How to Design and implementing a web scale cloud content storage and delivery infrastructure on AWS using S3

Architecture for AWS Storage and Delivery infrastructure :

The above proposed solution makes use of different aws services for getting a scalable and secure optimistic solution.

Two S3 buckets are considered for the complete solution.

Basically we considered two buckets one is primary and the other is secondary , where primary bucket is considered as primary site and secondary is considered as disaster site.

Disaster Recovery : To make best of use of disaster site, it’s always recommended to create the disaster site in a completely different region in a different geographical area, therefore we created two buckets in two different regions , one in north california and other in EU-London.AWS S3 Replication feature can help us to meet this requirement by replicating data from the bucket of one region to the other region.

Cost optimization is the big thing that we can provide to customer by making best use of lifecycle policy defined at bucket. We can move the object at configured time intervals to different storage classes like SIA and Glacier as the cost can be reduced, SIA can be used to move infrequent access data , however when required it will be instant. Glacier can be used to move the archive data which can be used for audit or for any year end activities. Following is the policy we defined on the buckets as per the current requirement, we will move the objects from Standard S3 to SIA after 75 days from the date of creation and then we will keep it there for few months until we complete a year from date of object creation.Once a year is completed we will keep the data in glacier for one more year which can be used for audit and then we can purge the data.

3)Authentication and Authorization is the other big thing that we can provide to the customer to make the data more secure. AWS provides security,identity and compliance for the the same. We can create users/groups with IAM and associate the same with the required bucket. Here is the snapshot of security policy for one of the bucket, the same can be applied to any bucket. You have a choice of configuring this along with S3 configuration after creating the users at IAM or you can directly go to IAM and do the configuration there as well by attaching the required bucket.

4) Geo-restrictions is the other thing we can offer through AWS , it helps the customer to enforce restrictions to a particular country. There might be use cases where customer want to enforce this because of some government policies. Following is the snapshot of the geo restrictions that we had for the country CUBA.

5) Reduced Latency : This is one big thing that any customer looks at. AWS provides a CDN offering called cloudfront for achieving the same. CloudFront can be configured to serve the content of S3 bucket so that data can be served fastly across the globe through the edge location deployed globally. Reliability can be there because of different edge locations. Following snapshot shows the cloudfront distribution and its configuration.

Leave a comment