Friday, June 23, 2017

AWS : S3 : Cross Region Replication & Transfer Acceleration

What is CRR ?
CRR is an Amazon S3 feature that automatically replicates data across AWS regions. With CRR, every object uploaded to an S3 bucket is automatically replicated to a destination bucket in a different AWS region that you choose.

What is use of CRR ?
It can be used  reduce latency for users in different regions

How do I enable CRR?
CRR is a bucket-level configuration. You enable a CRR configuration on your source bucket by specifying a destination bucket in a different region for replication.


     Requires versioning enabled on source bucket.
     Pick 3 things ‘destination region’ , ‘destination storage class’ , and ‘destination bucket’
     Give IAM also 

  • Note : Existing object will not be replicated , CRR replicates every future upload of every object to another bucket.

What does CRR replicate to the target bucket?CRR replicates
  • Object 
  • Associated Metadata
  • ACL
Any change to the underlying data, metadata, or ACLs on the object would trigger a new replication to the destination bucket. You can either choose to replicate all objects uploaded to a source bucket or just a subset of objects uploaded by specifying prefixes. Existing data in the bucket prior to enabling CRR is not replicated. You can use S3’s COPY API to copy the existing data into your destinatio

Can I use CRR with lifecycle rules ?
Yes, you can configure separate lifecycle rules on the source and destination buckets. For example, you can configure a lifecycle rule to migrate data from Standard to Standard - IA on the destination bucket or configure a lifecycle rule to archive data into Amazon Glacier.

What is Transfer Acceleration ?
Amazon S3 Transfer Acceleration enables fast, easy, and secure transfers of files over long distances between your client and your Amazon S3 bucket. Transfer Acceleration leverages Amazon CloudFront’s globally distributed AWS Edge Locations. As data arrives at an AWS Edge Location, data is routed to your Amazon S3 bucket over an optimized network path.

How to enable Transfer Acceleration ?
  • Enable it at bucket level
  • After Transfer Acceleration is enabled, you can point your Amazon S3 PUT and GET requests to the s3-accelerate endpoint domain name. Your data transfer application must use one of the following two types of endpoints to access the bucket for faster data transfer: 
  1. .s3-accelerate.amazonaws.com or 
  2. .s3-accelerate.dualstack.amazonaws.com for the “dual-stack” endpoint.

When should you use Transfer Acceleration ?
If you are uploading to a centralized bucket from geographically dispersed locations, or if you regularly transfer GBs or TBs of data across continents, you may save hours or days of data transfer time.

What if Transfer Acceleration isn't faster?
Each time you use Transfer Acceleration to upload an object, we will check whether Transfer Acceleration is likely to be faster than a regular Amazon S3 transfer. If we determine that Transfer Acceleration is not likely to be faster than a regular Amazon S3 transfer of the same object to the same destination AWS region, we will not charge for that use of Transfer Acceleration for that transfer, and may bypass the Transfer Acceleration system for that upload.

Can I use Transfer Acceleration with multipart uploads?
Yes, Transfer Acceleration supports all bucket level features including multipart upload.

Q. How should I choose between Transfer Acceleration and Amazon CloudFront’s PUT/POST?
Transfer Acceleration optimizes the TCP protocol and adds additional intelligence between the client and the S3 bucket, making Transfer Acceleration a better choice if a higher throughput is desired. If you have objects that are smaller than 1GB or if the data set is less than 1GB in size, you should consider using Amazon CloudFront's PUT/POST commands for optimal performance.

Q. How should I choose between Transfer Acceleration and AWS Snowball?
The AWS Import/Export Snowball is ideal for customers moving large batches of data at once. The AWS Snowball has a typical 5-7 days turnaround time.

todo compare Transfer Acceleration with other options ( AWS Direct Connect , Storage Gateway )


Q. How do I get started with IPv6 on Amazon S3?
You can get started by pointing your application to Amazon S3’s new “dual-stack” endpoint, which supports access over both IPv4 and IPv6. In most cases, no further configuration is required for access over IPv6, because most network clients prefer IPv6 addresses by default.

Expect no slowness

Do I need to update my bucket and IAM policies?

Yes, if you use policies to grant or restrict access via IP addresses, you will need to update those policies to include the associated IPv6 ranges before you switch to the “dual-stack” endpoint. If your bucket grants or restricts access to specific IAM users, you will also need to have the IAM policy administrator review those users’ IAM policies to ensure they have appropriate access to the associated IPv6 ranges before you switch to the “dual-stack” endpoint. Failure to do so may result in clients incorrectly losing or gaining access to the bucket when they start using IPv6.

Q: What can I do if my clients are impacted by policy, network, or other restrictions in using IPv6 for Amazon S3?Applications that are impacted by using IPv6 can switch back to the standard IPv4-only endpoints at any time.

Q: Can I use IPv6 with all Amazon S3 features?
No, IPv6 support is not currently available when using Website Hosting and access via BitTorrent. All other features should work as expected when accessing Amazon S3 using IPv6.

No comments: