AWS Snowball

Overview

• Transferring large amount of data offline
  • Import data to S3
  • Export data from S3
• "Never understimate the bandwidth of truck full of hard drives".
• Devices "snowballs" can be combined together to create a really big "SNOWBALL"
• Transferring 50TB on 150 Mbps link (50% utilization) takes 63 days
• Management API

Device

• 50/80 TB 
• Tamper proof
• All data encrypted
• Network adapters (SPF+, RJ45) @10GbE
• e-ink for display

Snowball client

• Runs on-premise
• Used to unlock device
• Transfers data onto device
• Supports HDFS
• Works best if OS supports AES-NI

S3 Adapter

• Exposes S3 comptaible endpoint 
• Existing tools can just point to Snowball IP

Job (import)

• workflow for handling data import
  
  • Create job
  • Snowball shipped to customer
  • Download client
  • Plug device to the network/machine
    • place deep in the stack to prevent bottlenecks
  • Client uploads data onto device
    • Encrypts using KMS
    • Parallel upload
  • Ship device back to AWS 
  • AWS team plugs in to their network and uploads to S3 (in future EBS)

References

• https://aws.amazon.com/blogs/aws/aws-snowball-edge-more-storage-local-endpoints-lambda-functions/