AWS - Storage - Rebuild-InProgress

S3


  1. S3 is object based storage on AWS cloud. Objects are files like videos, pictures etc.
  2. 0byte to 5TB size files can be uploaded.
  3. There is no limit on upload.
  4. Files are stored in buckets.
  5. S3 is universal namespace (two same name bucket can't exist globally), so each bucket name must be unique.
  6. S3 url syntax https://S3-<region name>.amazonaws.com/<bucketname>.
  7. Putting new object on S3 gives read after write consistency (no delay in read).
  8. For any update or deletion of object propagation of change will take time.
  9. S3 fundamentals are Key(name), Value(data), VersionID, Metadata, ACLs
  10. Write/Upload to S3 is notified by HTTP-200 message.
  11. Faster upload of larger files can be done using multipart upload tool.

S3 storage classes/Tier

  1. S3 (durable, immediately available, frequently accessed).
  2. S3-IA (durable, immediately available, infrequently accessed).
  3. S3-Reduced redundancy object (data that can be quickly reproducible from backup, like thumbnail etc).
  4. Glacier-Archive data, where once request raised to access data it takes 3-5hours for response.

S3 Versioning

  1. By default, it is disabled.
  2. Once enabled, we can suspend it.
  3. First upload will not have any versionID associated, once versioning enabled versionID gets associated with every update of object. On suspension of versioning first uploaded object gets updated on every update.
  4. Versioning allows MFA authentication deletes.
  5. Cross region replication requires versioning to be in enabled state on source and destination bucket.
S3 object lifecycle management
  1. Can be used in conjunction with versioning or independent.
  2. Can be applied to current or previous versions.
  3. Following options to choose from
    1. Transition to S3-IA : 128KB minimum size and 30 days after creation date of object it can be moved to S3-IA.
    2. Archive to glacier storage : 60 days after creation of object or 30days after moving to S3-IA.
    3. Permanent deletion. 

Security

  1. By default, all newly created buckets are private.
  2. Access control to your buckets using
    1. Bucket policies.
    2. Access control lists.
  3. Setting up bucket access logs.

Encryption

  1. Data (object) in transit
    1. SSL/TLS (HTTPS)
  2. Data at rest
    1. Server side encryption
      1. SSE-S3 (AES-256)
      2. SSE-KMS (Key management service, envelope key, audit trail)
      3. SSE-C (Customer provided encryption key)
    2. Client side encryption
      1. Encrypt your own data and upload.

Comments

Post a Comment