AWS DynamoDB (Streams)

Stream

• Time ordered sequence of item level modifications
  • Change log of DynamoDB table
• Stored 24 hours
• Similar to "Triggers" in RDBMS
• Can use special adapter to connect to Kinesis Data Streams

Stream Record

• Single data modification
• Has sequence number
• View Type
  • Old Image, New Image, Old + New Image, Keys Only

Shards

• Organizes stream records
• Automatically managed by AWS
• Lineage: parent can split into child
  • Important to read parent BEFORE child
    • Otherwise data out-of-order

DynamoDB Streams Kinesis Adapter

• Use KCL to process DynamoDB stream
• Same interface - different implementation

DynamoDB Triggers

• Lambda Integration
• Write function that reacts to events (triggers)
• Unlike stored procedure in RDBMS they run "out-of-process"
  • Cannot bring down the database

Use Cases

• create indexes on attributes within document objects
• cross-region replication
  • AWS provides Lambda based application for this use case
  • Alternative to datapipeline
• De-dupe data
• Aggregation of metrics
• Multi-master replication (across regions)
  • Now supported by Global Tables
• Archiving TTL'ed items (e.g. to S3)