IAM Roles for EC2
Now we are going to take a practical look at how roles and policies work for an EC2 instance by having an EC2 instance access S3 and CloudWatch.
We will start with a basic example using the cli. Then we'll build upon this to have our web app interact with S3 and CloudWatch.
Setup
Let's start by creating a new S3 bucket and EC2 instance. To keep things simple, do everything in the same region.
step 1:
step 2:
Once the instance is created, login to the instance using ssh. We're now going to install the aws cli on the instance to interact with the S3 bucket.
step 3:
step 4:
You should see the error:
Which is just saying we don't have permission to list the objects in that bucket. So let's setup an IAM role to allow the EC2 instance to access the S3 bucket.
Show timestamps
00:00
Now we're gonna take a look at how roles and policies work
00:03
with an EC2 instance by setting up an EC2 instance
00:06
that accesses an S3 bucket and CloudWatch.
00:10
And we're just gonna use the AWS CLI on an instance
00:14
for this example, then in the next parts,
00:17
we're gonna integrate this into our QR code web app.
00:21
So that application will be able
00:23
to directly access S3 and CloudWatch.
00:25
But to get started,
00:27
we're gonna go into the Playground account
00:29
and we'll start by setting up the S3 bucket.
00:32
And this is just gonna be a test bucket
00:34
so we can see how we might add files
00:36
and read files from a EC2 instance.
00:39
So I'm gonna create a new general purpose bucket.
00:42
We're gonna leave all of the settings
00:43
as the default settings.
00:45
And then I just need a unique name.
00:47
So my test bucket for IAM, I hope that's unique.
00:52
So let's try creating that bucket.
00:55
That is not unique, two.
00:57
Okay, so it created
00:59
my test bucket for IAM two.
01:01
So I got to remember this bucket name.
01:03
We're gonna have to use this a little bit later on.
01:05
And also I have done this in Oregon.
01:07
So I'm gonna do everything in Oregon for this example.
01:10
And we are just leaving all of the default settings
01:12
for this bucket because it's locked down by default.
01:15
So nothing can really access this.
01:16
And I'm just gonna leave this open in a new tab.
01:18
So I have access to this name for later on.
01:20
Then we're gonna set up the EC2 instance.
01:22
So in a new tab, I'm gonna open up the EC2 dashboard
01:25
and we're gonna launch a new instance to access this bucket.
01:28
So I'm gonna call this bucket.
01:28
I'm gonna call this IAM example.
01:31
And we're gonna set up an Ubuntu instance.
01:34
On the T2 micro, I'm gonna log in with my existing key pair.
01:39
And for the network settings,
01:42
I really just need to be able to log in with SSH.
01:44
So that is good.
01:45
Everything else is fine.
01:47
And actually one more thing I will mention,
01:49
if I already had an IAM role set up,
01:52
I could come into the advanced details here
01:53
and select the IAM instance profile right here,
01:57
but we don't have that set up yet.
01:58
So I'm just gonna launch the instance without it
02:00
and we can always update the instance later on.
02:02
And then once that instance is set up,
02:04
I'm going to connect to it using SSH.
02:07
So I'm gonna copy the example and paste that in,
02:10
but change the path to my .ssh directory.
02:15
And now I'm logged into that instance.
02:17
And like I said, for this example,
02:18
we're just gonna use the AWS CLI.
02:20
So I wanna be able to access that AWS CLI
02:23
from this Ubuntu instance.
02:24
But by default, it's not installed on the instance.
02:27
So I need to install that and I can find the instructions
02:29
just by Googling, install AWS CLI.
02:33
And the AWS docs will give you instructions
02:35
for any operating system, I think.
02:38
So since we're on Linux,
02:40
I can just copy and paste this into my Ubuntu instance.
02:45
And this is relying on unzip,
02:47
which I don't have installed.
02:48
So I can sudo app install, unzip.
02:51
And now I should be able to unzip that
02:54
and then run the installer.
02:56
Okay, so once that's done, I should be able to run AWS.
03:00
Yeah, so I have the CLI installed now,
03:02
and I just wanna run some commands against that S3 bucket.
03:06
So one of the simplest ones would be
03:08
to list the contents of an S3 bucket.
03:10
So that's AWS S3, and then we need the name of that bucket.
03:14
So I'm gonna head back to my bucket here
03:17
and I'm gonna copy the bucket name
03:19
and then just paste that into terminal right here.
03:22
And if I had permission to list the contents of the bucket,
03:26
this should work.
03:27
But instead what I'll get is unable to locate credentials
03:30
because it's looking for an access key
03:32
and a secret access key to use
03:34
to see if I have permission to access the bucket.
03:36
And it's telling me that I can configure credentials
03:38
running AWS configure,
03:40
but this is really only useful if I've set up an IAM user
03:43
and I have some credentials
03:44
that I'm gonna hard code onto this instance.
03:46
But since it's an EC2 instance, I'm gonna use a role,
03:48
which means that these credentials are temporary
03:52
and managed for me.
03:53
So I don't actually have to hard code them in here.
03:55
So don't run AWS configure.
03:57
Instead, what we need to do is set up an IAM role
03:59
for this EC2 instance.
04:01
So I am gonna head back over here
04:03
and I'm gonna open up IAM in a new tab
04:06
and we are gonna go to roles.
04:09
And there's some roles that are already set up.
04:11
There's already policy set up to some default things
04:14
that we might wanna use,
04:15
but we are gonna create a new role
04:18
specifically for this EC2 instance.
04:20
So we're gonna be using an AWS service.
04:23
We're gonna attach this role to an AWS service.
04:24
And all the services are listed down here.
04:27
There's the commonly used services, EC2 and Lambda.
04:29
And then there's a whole bunch of other things
04:30
we can create roles for,
04:32
but this is just for an EC2 instance.
04:34
We are gonna give an identity to this EC2 instance.
04:37
And we're just gonna leave the default use case set up,
04:39
allows an EC2 instance to call AWS services on your behalf.
04:42
So we'll click next.
04:44
And then we can attach policies to this instance.
04:47
And there are a bunch of policies already set up.
04:49
So if I wanted to allow everything on S3,
04:52
I could find, here we go, Amazon S3 full access.
04:53
So my instance would be able to create buckets,
04:56
delete buckets, do absolutely anything in S3,
04:58
which isn't necessarily what we want,
05:00
but we could expand this and see what the JSON policy is.
05:03
It's really simple.
05:04
This just allows us to do anything on any S3.
05:07
And there's a bunch of these policies pre-created for us.
05:10
And we will use one of these later
05:11
when we set up CloudWatch.
05:13
But for now, I'm actually gonna skip this.
05:15
We're gonna attach a policy later on.
05:16
So I have no policies attached to my EC2 instance.
05:20
And then I'm gonna set up the role name.
05:22
So this is for my test IAM EC2 instance.
05:25
So I'm just gonna call this test IAM role.
05:28
And then we're gonna leave everything as default
05:30
because this is just allowing the EC2 instance
05:32
to assume this role.
05:34
And we don't have any policies attached to this yet.
05:37
We just have a role.
05:38
So once I click create role, we have a role,
05:40
we have an identity,
05:42
but that identity isn't allowed to do anything.
05:44
It's just something that we can use with the EC2 instance.
05:46
So if I go back to my instance,
05:49
I'm gonna select the checkbox here, and I'm gonna go,
05:51
I'm gonna go to actions and then security
05:55
and then modify IAM role.
05:57
And here I'll be able to now attach that role
05:59
to the instance.
06:00
So I have my test IAM role.
06:01
I'm gonna select that and update the IAM role.
06:04
So now my EC2 instance has security credentials.
06:08
It can kind of log into my AWS account.
06:10
It just doesn't have any permissions yet.
06:12
But if I go back over to terminal
06:14
and try listing the contents of this bucket,
06:16
we do get a different error message now
06:18
because it is able to locate credentials.
06:20
It has those credentials
06:21
but it doesn't have the IAM role.
06:23
It's just now that we don't have access.
06:24
We don't have permission to access this S3 bucket.
06:27
So now the next step is to create a policy
06:30
that gives us permission to access this bucket.
06:32
So if we go back over to IAM,
06:35
I'm now gonna head over to policies
06:38
and I'm gonna create a custom policy just for this bucket.
06:41
So the service we're creating the policy for is now S3.
06:45
The role we're attaching to an EC2 instance,
06:48
but the policy we're gonna make is to allow permission
06:51
for the S3 bucket.
06:51
So this is going to allow us permission for the S3 bucket.
06:51
So this is going to allow us permission for the S3 bucket.
06:51
So this is going to allow us permission for the S3 bucket.
06:51
So this is going to allow us permission for the S3 bucket.
06:51
So this is for S3.
06:52
And then we have this option here, visual or JSON.
06:56
And the visual dashboard will allow us
06:58
to configure the permissions using all of these dropdowns
07:02
and check boxes.
07:03
But we could just come into the JSON mode
07:05
and modify this JSON directly
07:07
or get an LLM to generate this for us.
07:09
But we'll stick to visual for now
07:11
just so you can see how this works.
07:12
So I have to reselect S3.
07:14
And what we wanna do is allow list objects,
07:18
such the list bucket.
07:20
So this will allow us to,
07:20
list all of the objects that exist in the bucket.
07:24
Then once we've list those objects,
07:25
we wanna be able to get a single object from the bucket.
07:29
So this is get object right here.
07:30
And then I also want my EC2 instance
07:32
to be able to put new files, new objects into the bucket,
07:36
which is put object.
07:39
And I want it to be able to delete those files too.
07:41
So that is delete object.
07:44
And you can see there's tons of things
07:45
that you can do on an S3 bucket.
07:46
If we just allowed access to any of these actions,
07:49
there's a lot of things our instance would be able to do,
07:52
but the principle of least privilege,
07:53
we only wanna allow the exact things
07:56
that are required for our use case.
07:57
So that's good for all the actions we want.
08:00
Then on the resource,
08:02
we wanna select a specific bucket.
08:03
This isn't just for any S3 bucket in my account.
08:06
This is for a specific one.
08:07
So we're gonna add an ARN for the bucket.
08:10
And we just need to put in the bucket name here.
08:12
So that's this bucket name.
08:14
I'm gonna copy this again so I don't get it wrong.
08:17
And I'm gonna go back here.
08:18
I'm gonna paste that bucket name in.
08:19
So this is the ARN for the bucket,
08:21
but all I need to do is paste in that bucket name.
08:23
So I'm gonna add the bucket there.
08:25
And then I could specify a specific object
08:28
or specific objects that this EC2 instance
08:32
is able to read and write.
08:33
But for this case, I want it to be able to access
08:36
any object within that bucket.
08:38
So I'm gonna select any for that.
08:39
And then I'm gonna select next for the policy name.
08:43
I'm gonna say IAM demo S3 policy name.
08:49
And I'm gonna say IAM demo S3 policy.
08:52
And then we can see the permissions
08:54
that have been granted here.
08:55
And then I'll just create that policy.
08:57
So now if I search through policies, IAM demo,
09:01
there it is right there.
09:02
This is my custom policy that I've made.
09:04
And we can select the JSON view here
09:06
to see what the actual JSON for the policy might look like.
09:10
And remember that this is just a JSON document.
09:12
So in order to get these permissions,
09:14
we need to attach it to the IAM role.
09:17
So I'm gonna head back over to roles.
09:18
So I'm gonna go to my test IAM role
09:20
and then we need to attach policies to this.
09:23
So I'm gonna click add permissions
09:25
and we can attach a policy,
09:27
which is that JSON policy that I just made,
09:29
or we can create an inline policy.
09:31
And these essentially do the same thing.
09:33
But if I create a policy as a separate entity,
09:36
I could potentially share that between multiple roles
09:38
or multiple users or groups.
09:40
Whereas if I create an inline policy,
09:42
I would just be adding that policy directly to the role
09:46
and I wouldn't be able to share it across multiple identities.
09:48
And whether you add an inline policy like this
09:50
or create a policy as a separate entity
09:52
is completely up to you and your use cases.
09:54
Both are correct ways of doing things.
09:56
But since I've already made that role,
09:59
I'm gonna come back and I'm gonna attach that policy
10:02
that we made and that is my IAM demo S3 policy.
10:06
So I'm gonna select that and add permission.
10:09
So now my custom role has that policy attached to it.
10:13
If I go back over to terminal
10:15
and I try to list the contents of that S3 bucket,
10:18
I'm actually still getting access denied,
10:20
which is kind of weird.
10:21
So I'm gonna come back in here.
10:22
I can see, yeah, my test IAM role
10:24
that's attached to the instance has this policy.
10:26
So let's just double check the policy is correct.
10:29
So we can list, read and write.
10:31
That seems good.
10:33
If I go to JSON, that's weird.
10:35
It doesn't actually look correct.
10:37
I'm gonna edit this because what I would expect this to be
10:41
is that I would have my bucket listed as a resource,
10:45
which would be this bottom one here.
10:47
And I would have all the objects
10:48
in the bucket listed as a resource too.
10:50
So if I wanna put an object and get an object,
10:53
that's on any object within the bucket.
10:55
But if I wanna list all the objects in the bucket,
10:57
I'm running that directly against the bucket.
10:59
So I should have these two as my resources.
11:01
I don't know why that got a little bit messed up there.
11:03
I probably configured something wrong,
11:05
but the end policy should end up looking something like this.
11:07
I think that's the issue.
11:09
So if I go next and save the changes to this policy,
11:13
I'm hoping that my role now has access to the S3 bucket.
11:18
Perfect.
11:19
Okay, so the policy was just a little messed up there.
11:20
So I am able to list the contents of the bucket.
11:23
I'm not getting any issues here.
11:25
And this is exactly what I would expect.
11:27
There's nothing in the bucket,
11:28
so I'm not actually getting any contents.
11:30
Now let's add a file to that bucket from the EC2 instance.
11:34
So I'm gonna use vim, and I'm just gonna make notes.txt.
11:38
This is my test note, and then I can save that file.
11:43
And then instead of using LS, I am going to use AWS S3 CPT.
11:48
Notes.txt, and this should copy this file
11:53
from the instance to that S3 bucket.
11:55
And since we have put object
11:56
as one of the allowed permissions,
11:58
this should work and put that notes.txt
12:00
directly into that bucket.
12:02
So now if I run the LS command again,
12:05
I should be able to see that I have this single file
12:08
within the S3 bucket.
12:09
And if we really wanted to double check,
12:11
we could go back to the bucket over here,
12:13
and I can refresh this, and I can see, yeah,
12:16
this S3 bucket now contains notes,
12:17
because my instance,
12:18
this instance has permission to add files,
12:20
read files, and list the bucket.
12:22
And we were fine grained with what we were exactly allowing,
12:25
so I wouldn't be able to do any other thing.
12:26
So I wouldn't be able to, for example, AWS S3 LS,
12:29
which would list all the buckets in the account.
12:31
That's still access denied,
12:33
because I'm only allowed to do specific things
12:35
on that specific bucket.
12:36
And now we're gonna work with another service.
12:38
This is CloudWatch, where we can create log groups
12:41
and have our EC2 instance add system logs,
12:44
application logs, or EC2 metrics to this service.
12:48
And just to make sure we can't access this,
12:49
I'm gonna say AWS logs, so that's for CloudWatch.
12:53
I'm gonna create log group,
12:55
and I'm gonna give this a name, test group.
13:00
Okay, so this should create a new log group
13:02
in CloudWatch logs.
13:03
As we expected, we're getting this access denied exception,
13:06
because we do not have permission to create log group.
13:09
So let's go change that.
13:11
I'm gonna go back over to my IAM role.
13:14
So I'm already on my test IAM role.
13:16
That's the one that's attached to the EC2.
13:18
And we're going to add permissions,
13:21
and I'm gonna attach a policy again.
13:23
And this time, instead of creating a custom policy,
13:26
we're gonna use one of the default built-in policies
13:28
that AWS already gives us,
13:30
because there's one for CloudWatch agent server policy.
13:35
So this one right here.
13:36
This is already configured.
13:37
Actually, we can expand this,
13:39
and let's see what it looks like.
13:41
So this is already pre-configured
13:42
to allow most of the default things we would want,
13:45
something like an EC2 instance to be able to do,
13:47
with CloudWatch.
13:48
It's allowed to create log groups, push its metrics.
13:51
And this is all around set up exactly for this use case.
13:54
So instead of creating a custom one,
13:55
we are just gonna use this built-in one.
13:57
So this is the CloudWatch agent server policy.
14:00
And we're gonna add that permission to our test IAM role,
14:04
which is already attached to the instance.
14:06
So now if I go back over here
14:08
and try to create a new log group using the AWS CLI,
14:11
we should have permission now.
14:13
And that seemed to work.
14:14
So back in the web dashboard,
14:17
I'm gonna head over to the CloudWatch service
14:21
so that we can go into log groups
14:23
and see if it was actually created.
14:24
So I'm gonna select log groups over here.
14:25
That's the thing that we just made.
14:27
And I can see there's my test group.
14:28
So this instance now has permission to create log groups,
14:31
and I could push logs and do other things with CloudWatch
14:34
because I added that policy to the IAM role.
14:36
But since this was just a test,
14:38
I am going to delete that log group since we don't need it.
14:41
So that is how we can give an EC2 instance
14:45
access to other services using an IAM role,
14:47
and different policies.
14:49
In the next parts,
14:50
we're gonna set up the QR code web application again,
14:53
and have the application and instance
14:56
access the S3 bucket and CloudWatch
14:59
in a much more meaningful and realistic way.