Control access to any resource in AWS

Let's learn how to control access to any resource in AWS. That's a tall order, so we'll work through a 'simple' example that introduces the key IAM concepts you'll need to be effective.

We'll start with the basic IAM access control flow and elements of IAM policies. These security policy elements control a principal's ability to execute an API action that affects a resource.

Control access with IAM policies

IAM policies control whether a principal may act on a resource.

This diagram depicts a simplified IAM access control flow for an AWS API request:

Figure 1.1: Simplified IAM Access Control Flow

First, an application or person authenticates as an IAM role or user principal. A principal is an entity authenticated by AWS and assigned privileges to use within AWS. Then that principal requests an AWS API action. The AWS Identity and Access Management (IAM) system evaluates that request to determine if it is allowed. IAM does that by evaluating any:

• Identity policies attached to the principal
• Resource policies attached to the resource, e.g. S3 bucket
• Service Control policies attached to the AWS Account

Finally, IAM renders a decision either allowing the request to proceed to the target service API or responds with AccessDenied.

At its core, AWS IAM enables you to state whether a principal should be allowed or denied the ability to invoke an API action on a resource. Each of those emphasized terms are key elements of an IAM security policy statement. A policy statement describes which access control rules apply in a given situation.

Statements are collected into an IAM security policy document represented as JSON in this form:

Copy
{
  "Version": "2012-10-17",
  "Statement": [
    ... one or more Statement objects ...
    {
      "Sid": ... (Optional) Statement Identifier ...,
      "Effect": ...either "Allow" or "Deny" ...,
      "Action": [... array of Actions ...],
      "Resource": [ ... array of Resources ... ],
      "Principal": { ... one kind of principal ...
        "AWS": [... array of AWS accounts and IAM principals ...]
        "Service": [... array of AWS AWS services ...]
        ... others ...
      }
      "Condition": { ... (Optional) extra condition objects }
    }
  ]
}

Each policy document has a Version and Statement member element.

The Version member defines which version of the AWS Security Policy language the document is written in. Use the latest version, 2012-10-17.

The Statement member contains a list of one or more statement objects. Each Statement object has the following form:

Sid (optional): a string identifying the statement's purpose

Effect (required): whether to Allow or Deny access if the statement applies; Deny always wins when multiple statements apply

Principal (required): which principals the statement applies to, most importantly and commonly an AWS principal. The AWS Security Policy language supports several kinds of principals:

• AWS: an AWS account or IAM user or role principals within an account; may be specified as a single string or list of strings. Usually specifies an IAM entity within your AWS account, your organization, or a partner's account.
• Service: the name of an AWS service such as cloudtrail.amazonaws.com
• Federated: a principal from a federated identity provider such as a corporate IdP integrated via SAML or a public IdP such as Cognito, Google, Facebook, or Amazon
• Anonymous: allow anyone access via *; this can be narrowed via conditions

Action (required): one or more AWS API actions the statement will Allow or Deny the Principal to invoke as a string or list of strings. Supports wildcards , ? and *.

Resource (required): one or more AWS resources the statement applies to, specified as ARNs. Supports wildcards, ? and *.

Condition (optional): one or more conditions that qualify when the statement applies using context from the request to verify a request was made in a certain way, such as to an encrypted endpoint. Some conditions support wildcards, ? and *.

The Principal, Action, and Resource each have a negated form that can be used instead of the positive form: NotPrincipal, NotAction, and NotResource. The negated forms are primarily useful for advanced use cases such as granting limited access to an AWS service principal, but are difficult to use correctly. NotPrincipal is especially dangerous as it's easy to 'match' nearly every AWS account. So is NotResource combined with Allow, which makes it trivial to accidentally allow access to all buckets except one. Avoid NotPrincipal and NotResource.

While Principal, Action, and Resource (or their negative form), are required elements, AWS IAM infers the Principal for a policy when attached to an IAM user or role.

All AWS Security policy documents take the form described above. Consult the AWS IAM reference guide for the full IAM policy language grammar.

AWS supports five types of security policy, each applying to a different scope:

Take a moment to think about the level of control and flexibility this language provides. This makes the IAM security policy language very powerful, but also difficult to understand. We'll dive into why IAM is hard later.

Now let's see how these policy elements work in practice by controlling access to an S3 bucket.

Control Access to an S3 Bucket

Let’s start with a common deployment scenario and policy requirements.

Suppose we have a simple application deployed entirely in AWS:

Figure 1.2 Simple App Using Lambda & S3

The application:

• deploys to AWS using an automated delivery pipeline
• runs on AWS Lambda using the app IAM role
• is supported by a customer service team
• stores data in the sensitive-app-data bucket

The deployment must implement the organization’s high-level security policy requirements:

• implement least privilege, allowing only explicitly-specified principals the actions and access to data they need to perform their business function and denying access to all other principals
• require encryption at rest and in transport

Many people approach this by creating an Identity policy for the application's IAM role, so let's start there. We can create an Identity policy for the app role that grants read and write access to the sensitive-app-data bucket:

Copy
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowRestrictedReadData",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetObjectVersion",
        "s3:GetObject"
      ],
      "Resource": [
        "arn:aws:s3:::sensitive-app-data/*",  # for GetObject*
        "arn:aws:s3:::sensitive-app-data"     # for ListBucket
      ]
    },
    {
      "Sid": "AllowRestrictedWriteData",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:AbortMultipartUpload"
      ],
      "Resource": [
        "arn:aws:s3:::sensitive-app-data/*",
        "arn:aws:s3:::sensitive-app-data"
      ]
    }
  ]
}

Once attached to the app role, this Identity policy will allow the application to read and write data in the sensitive-app-data bucket.

But problems are lurking. This identity policy grants access to the app role in a minimal way, but it doesn't achieve the full mission of protecting the sensitive application data.

What if:

• The app role has another identity policy attached that allows it to delete objects in any bucket
• Other IAM users and roles have the ability to read objects from any S3 bucket in the account

These scenarios are very common. Many Identity policies include statements that do not limit the resources they apply to with wildcards, either:

• "Resource": "*" # all resources in the account
• "Resource": "arn:aws:s3:::*" # all S3 buckets

This is the case when using AWS Managed Policies like ReadOnlyAccess or AdministratorAccess. AWS Managed Policies always use "Resource": "*" since they must be usable in any customer's account.

Depending on the use case, wildcards are useful, necessary, and dangerous. Carefully analyze the scope of resources the wildcard matches now, and what it could match in the future as resources change. Then decide if the matched scope is appropriate.

We've just seen that we can allow access to sensitive data with an Identity policy. But we can't deny unintended resource access with Identity policies in a scalable and maintainable way.

It's impractical for engineers to continually be aware of and evaluate every policy for every identity in an account. It's really impractical to update Identity policies every time a new resource needs to be protected.

So how will we implement the least privilege and encryption requirements for our sensitive data?

We need to block unauthorized and insecure access to sensitive data with a different approach, Resource policies.

Actually implement Least Privilege

Let's turn the problem around. Deny unauthorized and insecure access to the sensitive data using a resource policy attached to the bucket.

This flowchart shows how AWS IAM evaluates policies:

Figure 1.3 AWS Policy Evaluation Logic

Whoa! That's a lot.

That's the high level process AWS uses to evaluate security policies and decide "can the caller execute this API action?"

Any Allow in the policy evaluation chain provides access to the resource unless there is an explicit Deny.

Notice that:

• Both Identity and Resource policies may Allow an action.
• Effects are calculated with this precedence:
  • Explicit Deny by matching statement, overriding Allow
  • Explicit Allow by matching statement
  • Implicit Deny when no statement matches

Resource policies apply security rules directly to a resource like an S3 bucket or KMS key. In practice, there are relatively few sensitive data resources in an account. So a more practical approach to protecting the sensitive data is to attach a resource policy that enforces least privilege and best practice.

Begin creating your resource policy by identifying precisely who needs access to the bucket and what kind of access they need.

Suppose interviews with the application delivery and customer support team reveal the following access requirements for IAM principals:

• ci user needs administration capabilities to deploy application updates
• admin role needs administration capabilities to fix urgent problems
• app role needs read and write data capabilities for the application to function
• cust-service role needs to read data capabilities to investigate problems

All of these identities and application resources exist within a single AWS account, 111.

Now we have enough information to provision the intended access in the steps that follow.

Deny, Deny, Deny

Start by creating a policy with a DenyEveryoneElse statement that blocks access from all principals who do not need access to the bucket:

Copy
{
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "DenyEveryoneElse",
          "Effect": "Deny",
          "Action": "s3:*",
          "Resource": [
            "arn:aws:s3:::sensitive-app-data",
            "arn:aws:s3:::sensitive-app-data/*"
          ],
          "Principal": {
            "AWS": "111"
          },
          "Condition": {
            "ArnNotEquals": {
              "aws:PrincipalArn": [
                "arn:aws:iam::111:role/admin",
                "arn:aws:iam::111:role/app",
                "arn:aws:iam::111:user/ci",
                "arn:aws:iam::111:role/cust-service"
              ]
            }
          }
        }
      ]
    }

The DenyEveryoneElse statement jumps right into the deep end of the IAM policy concepts. This statement denies all IAM principals in account 111 the ability to invoke any S3 api actions on this bucket, whenever the condition is met. That condition is when the requestor's IAM principal ARN is not one of the intended principals, admin, app, ci, or cust-service.

This statement narrows the scope of who can access the bucket to these four principals, no matter what their Identity policies allow. Remember explicit Deny takes precedence over explicit Allow in AWS IAM.

We also want to enforce the organization's encryption policy.

Require encryption in transport with a DenyInsecureCommunications statement:

Copy
{
  "Sid": "DenyInsecureCommunications",
  "Effect": "Deny",
  "Action": "s3:*",
  "Resource": [
    "arn:aws:s3:::sensitive-app-data/*",
    "arn:aws:s3:::sensitive-app-data"
  ],
  "Principal": {
    "AWS": "*"
  },
  "Condition": {
    "Bool": {
      "aws:SecureTransport": "false"
    }
  }
}

This denies all principals the ability to invoke an S3 api action using an insecure transport. This forces clients to use https. The AWS API will deny any request made with http when it reaches the AWS API endpoint. (Note: If requiring secure transport causes a problem, consult the relevant AWS SDK docs and reconfigure the application to use the https transport scheme.)

Enforce encryption at rest with theseDenyUnencryptedStorage and DenyStorageWithoutKMSEncryption statements:

Copy
{
  "Sid": "DenyUnencryptedStorage",
  "Effect": "Deny",
  "Action": "s3:PutObject",
  "Resource": "arn:aws:s3:::sensitive-app-data/*",
  "Principal": {
    "AWS": "*"
  },
  "Condition": {
    "Null": {
      "s3:x-amz-server-side-encryption": "true"
    }
  }
},
{
  "Sid": "DenyStorageWithoutKMSEncryption",
  "Effect": "Deny",
  "Action": "s3:PutObject",
  "Resource": "arn:aws:s3:::sensitive-app-data/*",
  "Principal": {
    "AWS": "*"
  },
  "Condition": {
    "StringNotEquals": {
      "s3:x-amz-server-side-encryption": "aws:kms"
    }
  }
}

These statements force clients to do two things. First, all S3:PutObject operations must use AWS' server side encryption services on every put. Second, the encryption must use an encryption key managed by the AWS Key Management Service (KMS). We'll discuss these choices in more detail later. For now, know they enforce encryption at rest with a service you can also use to enforce fine-grained data access controls, KMS.

Allow, Allow, Allow

Now let's explicitly grant the access each of the principals need with an appropriate statement.

Organize the Allow statements by needed access capability, not principal. Organizing by access capability simplifies understanding what API actions principals may and may not perform on the resource. You will always know where to look to see if, e.g. a principal is allowed to write data.

Grant an access capability with a statement using this form:

Copy
{
  "Sid": "Allow<CapabilityName>",
  "Effect": "Allow",
  "Action": [
    "s3:<CapabilityAction1>",
    "s3:<CapabilityAction2>",
    "s3:<CapabilityAction...N>",
  ],
  "Resource": [
    "arn:aws:s3:::sensitive-app-data/*",
    "arn:aws:s3:::sensitive-app-data"
  ],
  "Principal": {
    "AWS": "*"
  },
  "Condition": {
    "ArnEquals": {
      "aws:PrincipalArn": [
        "arn:aws:iam::111:<Identity1>",
        "arn:aws:iam::111:<Identity2>",
        "arn:aws:iam::111:<Identity...N>"
      ]
    }
  }
}

The Sid describes that the statement allows a particular capability such as reading data or administering resources.

The Resource element applies to:

• all objects in the bucket, arn:aws:s3:::sensitive-app-data/*
• the bucket itself, arn:aws:s3:::sensitive-app-data

Buckets and Bucket Objects are distinct resource types. This is an important detail. Access to a bucket is identified and evaluated separately from the objects inside it. Most AWS API actions apply to a single resource type, though some API actions interact with multiple.

The Principal matches all AWS IAM principals in the aws:principalArn array.

A common policy mistake is to mismatch the API action and covered resource type. Allowing S3:PutObject to arn:aws:s3:::sensitive-app-data is an example of this mistake. AWS may save the policy depending on what else is in it. But when you actually invoke S3:PutObject the operation will fail. The action allowed puts to arn:aws:s3:::sensitive-app-data, a bucket type. But S3:PutObject operates against objects.

Allow Writes

The correct allow statement looks like this, which allows writes into the bucket:

Copy
{
  "Sid": "AllowRestrictedWriteData",
  "Effect": "Allow",
  "Action": [
    "s3:PutObject",
    "s3:AbortMultipartUpload"
  ],
  "Resource": [
    "arn:aws:s3:::sensitive-app-data/*"
  ],
  "Principal": {
    "AWS": "*"
  },
  "Condition": {
    "ArnEquals": {
      "aws:PrincipalArn": [
        "arn:aws:iam::111:role/app"
      ]
    }
  }
}

This statement allows the app role to put objects and also abort object uploads that are in progress. AllowRestrictedWriteData's resource element was narrowed to cover only objects in the bucket because the API actions pertain only to objects. The AWS API docs document which resource types each action works with.

Allow Administration

Now allow the ci and admin principals to administer the bucket and its resources with a statement like:

Copy
{
  "Sid": "AllowRestrictedAdministerResource",
  "Effect": "Allow",
  "Action": [
    "s3:PutReplicationConfiguration",
    "s3:PutObjectVersionAcl",
    "s3:PutObjectRetention",
    "s3:PutObjectLegalHold",
    "s3:PutObjectAcl",
    "s3:PutMetricsConfiguration",
    "s3:PutLifecycleConfiguration",
    "s3:PutInventoryConfiguration",
    "s3:PutEncryptionConfiguration",
    "s3:PutBucketWebsite",
    "s3:PutBucketVersioning",
    "s3:PutBucketTagging",
    "s3:PutBucketRequestPayment",
    "s3:PutBucketPublicAccessBlock",
    "s3:PutBucketPolicy",
    "s3:PutBucketObjectLockConfiguration",
    "s3:PutBucketNotification",
    "s3:PutBucketLogging",
    "s3:PutBucketCORS",
    "s3:PutBucketAcl",
    "s3:PutAnalyticsConfiguration",
    "s3:PutAccelerateConfiguration",
    "s3:DeleteBucketWebsite",
    "s3:DeleteBucketPolicy",
    "s3:BypassGovernanceRetention"
  ],
  "Resource": [
    "arn:aws:s3:::sensitive-app-data/*",
    "arn:aws:s3:::sensitive-app-data"
  ],
  "Principal": {
    "AWS": "*"
  },
  "Condition": {
    "ArnEquals": {
      "aws:PrincipalArn": [
        "arn:aws:iam::111:user/ci",
        "arn:aws:iam::111:role/admin"
      ]
    }
  }
}

Notice the (long) list of actions includes some that operate on:

• buckets, e.g. s3:PutBucket*
• bucket objects, e.g. s3:PutObject*
• and some where it's not particularly clear what type of resource applies, e.g. s3:PutMetricsConfiguration.

We can create similar statements to allow reading configuration as well as reading or deleting data.

The full policy for this example appears in Appendix - Least Privilege Bucket Policy. We won't include it here because it's well over 200 lines.

To adopt this least privilege model for your own bucket, replace the principals and bucket name with your own. Then put the policy into effect by populating the policy contents into the bucket's policy attribute in CloudFormation or Terraform, the AWS cli, or console. These all invoke s3:PutBucketPolicy. Finally, verify principals can still access data as expected. Resource policies are stored within the scope of the resource and cannot be shared between resources directly. We'll scale implementing least privilege in AWS when we 'Simplify AWS IAM'.

Let's wrap up our 'simple' example.

Summary

This 'simple' example demonstrated a few things.

First, the AWS IAM security policy language is flexible and powerful enough to implement fine-grained access controls to AWS API actions and data. The AWS identity and resource policies created in this chapter implement our high-level goals:

1. allow only authorized principals to access data in the way we intend
2. enforce encryption in transport and at rest

Second, it wasn't easy or straightforward to implement the policies we needed. The policy language is complex and some effects are difficult to anticipate. We had to close off unexpected access from other principals with excess privileges in the account. Enforcing basic encryption requirements is possible, but took three separate statements.

And the final policies constitute hundreds of lines instead of a few tens of lines you find in most 'Getting Started' blog posts and examples.

But keep in mind that the access control capabilities provided by the AWS IAM security service (and equivalent in other hyperscale Clouds) have no common analogue to on-premises data centers.

AWS IAM is over 10 years old but it's still 'early', especially when it comes to abstractions that help teams go fast safely.

Keep reading to learn about the problems in AWS IAM and the abstractions you need to solve them.