S3

Now that we understand AWS regions, we should setup some infrastructure. We're going to start with S3 because it's simple and cheap. It's also one of the most fundamental and widely-used services in AWS.

In the next part we'll setup a static website on S3, but first, what is S3?

What is S3?

You can store virtually any amount of data with S3 all the way to exabytes with unmatched performance. S3 is fully elastic, automatically growing and shrinking as you add and remove data. There's no need to provision storage, and you pay only for what you use.

S3 (Simple Storage Service) is AWS's object storage service. Think of it as an infinitely large hard drive in the cloud where you an store your data.

Buckets and Objects

It's not a traditional file system with folders and directories, it's a flat storage model of buckets and objects. You put bytes in and you get bytes out.

  • Buckets are the containers for objects.
  • Objects are the files/bytes you store.

You an actually think of it as a bucket full of objects, where each object has a unique key (name) within a bucket.

This flat structure makes S3 incredibly scalable. You can store virtually unlimited objects without performance degradation.

S3 also stores multiple coppies of objects in seperate data centres for redundancy.

What can you do with S3?

  • Application Assets: Store images, videos, and documents
  • Backup & Archive: Durable storage for backups
  • Data Lakes: Store massive amounts of raw data for analytics (parquet)
  • Software Delivery: Distribute software updates and downloads
  • Media Storage: Host audio, video, and images
  • Web Hosting: Store and serve website files

S3 Pricing

Free Tier (Per Month)
  • 5GB of Amazon S3 storage in the S3 Standard storage class
  • 20,000 GET Requests
  • 2,000 PUT, COPY, POST, or LIST Requests
  • 100 GB of Data Transfer Out each month.
Standard Pricing us-east-1
ItemCost
First 50 TB / Month$0.023 per GB
PUT, COPY, POST, LIST requests (per 1,000 requests)$0.005
GET, SELECT, and all other requests (per 1,000 requests)$0.0004
All data transfer in$0.00 per GB
Data Transfer OUT From Amazon S3 To Internet First 10 TB / Month$0.09 per GB

Pricing varies by region and storage class, but S3 is pretty cheap for most use cases. And free for what we'll be using it for.

Summary

S3 is for storing objects. It's a good default storage choice for static assets and there's a lot we can do with it.

For now, let's keep it simple and use S3 to host a static website.

Show timestamps
00:00
The first piece of infrastructure I want us to set up is going to be an S3 bucket, because this is
00:04
an incredibly easy service to use. We're going to end up using it a lot, and it's a very, very
00:10
cheap service. So we can kind of just keep them running forever and not worry about them like we would
00:14
with EC2 instances. So I'm going to head into the playground account just to take a look at
00:18
what S3 is. If we go in here and search for S3, you can see it's an
00:25
object storage service. You can store and retrieve any amount of data from anywhere. So this is storage—
00:31
it's not compute. We're not going to run code here. We're just going to store bytes and retrieve bytes.
00:36
You know, we put in files and get them later on. But this really is a highly scalable and
00:41
basically infinite amount of storage that we can use. So we can really hand S3 as much data
00:46
like terabytes, petabytes, whatever, and it'll be able to distribute that and make sure that we
00:51
can always store and retrieve basically any amount of data that we want. And that's kind of
00:55
Amazon's guarantee here. And this isn't a mounted file system. It is flat storage, which means that
01:01
we have this idea of buckets, which are kind of like containers, and then objects within the bucket.
01:06
There's no other hierarchy there. You just have a bucket full of bytes, basically.
01:10
So we can come in here, create a new bucket. And general purpose is what we're going to be
01:15
using in this course. Bucket names just have to be unique—globally unique. So you have to pick a
01:20
unique name, maybe some unique cloud course name. Hopefully that's unique enough.
01:27
Then there's a whole bunch of configuration we can set up here. By default, everything is private.
01:32
It's locked down. So we can put files into an S3 bucket and absolutely no one will be able to
01:38
access them unless we configure these settings to allow access. We can do versioning,
01:43
we can do a whole bunch of stuff. But essentially, if we create a bucket—and I will do this now—
01:48
I'm just going to create it with the default settings. We now have a container where we can put an
01:52
unlimited number of bytes. We can put any files we want in here. So at a minimum, you could kind of
01:56
use this as a private Dropbox. I can click on this bucket and see an Objects tab here. I have no
02:01
objects, but I can just start manually dragging and dropping in files. I can retrieve those files
02:05
to my laptop if I wanted to. But this is really handy if I was creating a web application where I
02:09
needed to store users' videos or images or any assets like that. I could just dump them into S3,
02:15
and that is always going to work. It's always going to scale, no matter how much data I put into
02:19
it. I actually have a list of common use cases here. So you can do backups and archives—I often
02:23
do database backups to S3. You can use it for data lakes, software delivery, media storage.
02:29
Web hosting is actually what we're going to use this for in the next part, where you have a static
02:34
website and you can just put those files into S3 and use it to serve up your website for
02:39
you. The pricing model is pretty good. In the free tier, which, for a year right now,
02:43
you get all of this for free. So we're not going to spend any money on S3. Don't worry about it.
02:47
But when you have to start paying, it's still incredibly cheap. Like these numbers, you would have
02:51
to be storing a lot of data and paying for bandwidth costs—like streaming video or something—
02:55
before you'd see any significant impact on your bill from S3. It's always going to be probably
03:00
the cheapest cost on your bill. So I'm going to go back in here. I'm actually just going to delete this
03:04
bucket. We're not going to do anything with it. In the next section, we're going to set up an S3
03:09
bucket, put a static website on it, and
03:12
host that so people can visit a website that's all hosted on an S3 bucket.