AWS EC2 (EBS)

Overview

• Network block storage as a service.

Model

• Block Device
  • Moves data in blocks of bytes (does not understand filesystem on top of it)
  • Block is 256KiB
  • Supports random I/O and generally uses buffered I/O
• Volume Size: max 16TiB
• Replication
  • within the same AZ 
  • write issued to primary and mirror
  • write returns when both are ack'ed
    • compare drdb

Availability

• Replicated within AZ
• 99.999% availability (5 nines)
• AFR: 0.1-0.5% provided that you snapshot more often than every 20GB modified blocks
  • Snapshotted EBS volume is practicaly 0% AFR (S3 durability)
  • It grows slowly from there
• RAIDing (1/5/6) does not work as well as on physical volumes due to shared infrastructure

IOPS

• Input/Output operation per second 
  • Read or Write
• Measured in increments of 256KiB
  • 1 * 1024KiB  = 4 IOPS
  • 1024 * 1KiB = 1024 IOPS
• For very large chunks you may reach throughput limit before IOPS limit
  • Example
    • 1000GiB gp2 has 3000 IOPS limit and throughput 160 MiB/s
    • At 256KiB you reach max throughput at 640 IOPS
    • For smaller size (e.g. 16Kib) same volume sustains 3000 IOPS

Average Queue Length

• Queue length - number of I/O operations pending on the device
• Workload demand important to take full advantage of EBS volume

General Purpose (SSD) - gp2

• IOPS baseline (3 IOPS/GiB)
  • size:IOPS = 1:3
    • e.g. 50 GIB = 150 IOPS
  • At 1TiB you reach 3000 all the time (no more bursts)
  • At 3.333TiB you reach cap 10,000 IOPS
• Burst: 30 minutes @ 3000
  • 3 credits per GiB per second
  • Initial credit: 5.4M
  • Cap at 5.4M
• Throughput: 128MB/s or 160MB/s
  • Depends on Volume size (170GiB is cut-off)
  • Pairs well with 1000 Mbps EBS optimized
• Size: 1GiB - 16TiB

Provisioned IOPS (SSD) - io1

• Precise definition of performance
• IOPS: 100-32,000
  • Can achieve very high IOPS for small volume size
  • e.g. 50GiB = 1500 IOPS

• Max ratio size:IOPS = 1:50
  • 100GiB = max 5000 IOPS
  • 640GiB ~ max 32,000 IOPS (max)
  • Prevents tiny disks with enormous IOPS
• Max Throughput
  • 256KiB/s per IOPS
  • 500 MiB/s
• 4Gib-16 TiB

Throughput Optmized (HDD) - st1

• Low-cost magnetic storage
• Dominant attribute is throughput not IOPS
• Not bootable
• Good for sequential I/O (EMR, ETL, log processing)
• IOPS
  • Baseline 40 MiB/TB
  • Burst 250 MiB/TB
  • Max 500 MiB/s
  • I/O is 1MiB (inefficient for small reads/writes)
• Throughust
  • Burst bucket
  • May drop during snapshotting
• 500 GiB - 16 TiB

Cold HDD - sc1

• Same as st1 but ~3x lower performance
  • Baseline throughput (12 MiB)
  • Burst per 1TiB (80)
• Cheaper than st1

Magnetic - standard (previous generation)

• IOPS around 100 (1 seek = 10msec)
  • Not changeable
  • Can burst to hundreds
• Throughput: 40-90 MB/s
• Latency Read: 10-40ms
• Latency Write: 2-10 ms
• Size: 1GiB-1TiB

EBS Encryption

• Transparent for the OS
• Data in-transit
  • encryption/decryption occurs on EC2 instance (in memory only)
• Data at rest
  • disk, snapshots
• Protects against physical access
• Volume Encryption Key 
  • DataKey protected with region-specific CMK
  • Stored encrypted on the disk
  • Stored plaintext in-memory only
• Performance impact
  
  • Increased latency
  • Same IOPS
• Supported on all EBS types (gp2, io1, standard)
• Certain instance types are supported
  • Must support Intel AES New Instructions (AES-NI)
    • e.g. Not supported for t2 type
• Encryption Context: EBS volumeId

Snapshots

• Start using application when CreateSnapshot returns (list of blocks)
  • Tool: ec2-consistent-snapshot
• Restored from snapshot: you can read all volume upfront (lazy loading)
• Can be copied between regions
  • New snapshotId in destination
  • S3-SSE used
  • Up to 5 in progress
  • First is full - subsequent are incremental
  • Encrypted Snapshot
    
    • You can specify it should be encrypted (if source is not)
    • You can change encryption key
    • Cannot be made public

Prewarming (initialization)

• EBS created from snapshots have blocks lazy loaded from S3
  • Performance impact 5-50%
  • Provisioned IOPS volumes may display warning
  • Blocks can be loaded by running
    • dd - standard but single threaded
    • fio - requires apt-get, multithreaded
• EBS newly created (blank) does not require initialization

Monitoring

• Frequency
  
  • Basic (5 minutes)
  • Detailed (io1)
• Metrics
  • io1
    • VolumeThroughputPercentage - delivered/provisioned %
    • VolumeConsumedReadWriteOps - total consumed IOPS (in 256K units)
• Volume Status Check
  • Automated test running every 5 minutes: PASS or FAIL
  • Status: ok, impaired, insufficient-data
  • Automatically disable IO on failure and create event
    • Stop applications -> enable manually -> run fsck -> start applications
    • Set AutoEnableIO attribute if you don't care about potential corruption
• Volume I/O Performance check
  • io1 only
  • alerts when performance too low
    • warning if < 50% IOPS
  • Status: normal (ok), degraded (warning), severly degraded (warning), stalled (impaired)

Deletion

• When instance is terminated
  • Root volumes default to delete
  • Non-root volumes default to retain

Elastic Volumes

• Ability to modify EBS on-the-fly
  • Volume Type (io1, gp2, sc1, sc2, etc.)
  • Size
  • Provisioned IOPS (io1)

References

• http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSEncryption.html
• http://docs.aws.amazon.com/kms/latest/developerguide/services-ebs.html
• http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-copy-snapshot.html
• https://www.youtube.com/watch?v=2wKgha8CZ_w (STG403)
• https://aws.amazon.com/blogs/aws/amazon-ebs-update-new-elastic-volumes-change-everything/