AWS S3 Cross Account Bucket Permissions

Granting a third party account access to your AWS S3 bucket can be quite useful.

Introduction

Imagine you want your firm to be able to deliver files to a customer by writing data to their AWS S3 bucket. You can do this with “cross account bucket permissions” as described below. You can also refer to various bits of AWS documentation if you prefer, but there are some tricky steps that may not be clear in those articles.

In the General Idea and Preparation section we provide an overview of what is going on and the single step both your firm and the customer need to execute. If you are an expert AWS user, this may be all you need. But we also provide Detailed CLI Operations so you can start from scratch, create test users, grant the permissions, and verify everything works yourself. These detailed operations are particularly useful if you want to change this setup slightly to suit your situation.

The source code for this document is available in the aocks/aws-examples repo on GitHub and discussed (along with other information) in this article on the AOCKS blog.

General Idea and Preparation

Imagine you an AWS account for your firm and your customer has an AWS account. What you want to do is roughly the following

1. Have the Customer Account grant your Firm Account’s root user access to the customer’s bucket.
2. Have your Firm Account grant an IAM account access to the customer’s bucket.
   • This is the main stumbling block that confuses a lot of users and often isn’t documented well. If you try to have the Customer Account grant access to a non-root IAM account in your firm, you will generally get various errors.
   • Roughly speaking this step is necessary because a non-root IAM account in your Firm Account doesn’t have any permissions unless you grant them from an admin user in your Firm Account.

Customer Action

The customer needs to create a policy to grant the firm account’s root user bucket access and apply the policy to the bucket. This is illustrated below where you would replace ${FIRM_ACCT} with the AWS account number for your firm and replace ${BUCKET} with the name of the AWS bucket.

cat > /tmp/writer-policy.json<<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
	        "AWS": "arn:aws:iam::${FIRM_ACCT}:root"
            },
            "Action": [
                "s3:GetObject",
                "s3:ListBucket",
		"s3:GetLifecycleConfiguration",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::${BUCKET}",
                "arn:aws:s3:::${BUCKET}/*"
            ]
        }
    ]
}
EOF

and apply the policy:

aws s3api put-bucket-policy \
  --bucket ${BUCKET} --policy file:///tmp/writer-policy.json

Firm Action

After completing the required Customer Action, your firm’s root user will have access to the customer’s bucket. Generally, however, you should not be using your firm’s root user to do anything but delegate appropriate permissions to an IAM account in your firm.

Below we create an inline policy to allow bucket access and grant it to an IAM user in your firm’s account named $FIRM_IAM_USER:

cat > /tmp/writer-policy-iam.json<<EOF
{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Sid": "allowwriteruseriamaccess",
         "Effect": "Allow",
         "Action": [
            "s3:ListBucket",
            "s3:GetObject",
            "s3:PutObject",
            "s3:PutObjectAcl"
         ],
         "Resource": [
            "arn:aws:s3:::${BUCKET}",
            "arn:aws:s3:::${BUCKET}/*"
         ]
      }
   ]
}
EOF

Now we apply the policy.

aws iam put-user-policy --user-name $FIRM_IAM_USER \
  --policy-name writer-policy \
  --policy-document file:///tmp/writer-policy-iam.json

Detailed CLI Operations

The following provide documentation and commands to use in a command line interface (CLI). These should work on Linux, Mac, or WSL (if you are using Windows, I highly recommend WSL). You should basically be able to just cut and paste these commands into your terminal (with obvious minor modifications like setting environment variables to point to your accounts).

Setup variables and software

The first step is to setup a python virtual environment and install the AWS CLI into it:

python3 -m venv venv_aws
. venv_aws/bin/activate
pip install boto3 awscli

Next you will want to setup some environment variables:

• CUSTOMER_PROFILE: This should be the name of the profile in the ~/.aws/credentials for the customer’s account. In the example below, this is set to my_awesome_customer; you should change this to whatever account you are using to test or for the real customer’s account.
• FIRM_PROFILE: This should be the name of the profile in your ~/.aws/credentials for your firm’s account. In the example below, this is set to my_firm.
• CUSTOMER_USER: This is the IAM account that you or your customer will use for testing.
• CUSTOMER_AWS_ACCT: This is the account ID for the customer’s AWS account.
• BUCKET: Name of the bucket you want to use.
• FIRM_ACCT: This is the account ID for your firm’s AWS account.
• FIRM_IAM_USER: This is the name of the IAM user in your firm that will be granted ability to write to the customer’s bucket (by your firm’s root account).

Once you have decided on the values above, you should execute like the following to store these as environment variables so you can use them in the remainder of this example.

export CUSTOMER_PROFILE=my_awesome_customer
export CUSTOMER_USER=test-customer
export CUSTOMER_AWS_ACCT=ACCT_ID_FOR_THE_CUSTOMER
export BUCKET=my.test.bucket
export FIRM_ACCT=ACCT_ID_FOR_THE_BROAD
export FIRM_PROFILE=my_firm
export FIRM_IAM_USER=test-writer

Note: if you type something like cat ~/.aws/credentials, that command should show you something like

[my_awesome_customer]
aws_access_key_id=...
aws_secret_access_key=...
[my_firm]
aws_access_key_id=...
aws_secret_access_key=...

where the ellipsis are replaced with the appropriate access keys for your customer and your firm. If not, make sure to setup your ~/.aws/credentials appropriately.

Customer Granting Access to Firm

Here we basically just execute the Customer Action mentioned earlier. We create a policy so that the firm’s root user can read/write to the desired bucket:

cat > /tmp/policy-for-firm-root.json<<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
	        "AWS": "arn:aws:iam::${FIRM_ACCT}:root"
            },
            "Action": [
                "s3:GetObject",
                "s3:ListBucket",
		"s3:GetLifecycleConfiguration",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::${BUCKET}",
                "arn:aws:s3:::${BUCKET}/*"
            ]
        }
    ]
}
EOF

Now the customer applies the policy:

aws --profile ${CUSTOMER_PROFILE} s3api put-bucket-policy \
  --bucket $BUCKET --policy file:///tmp/policy-for-firm-root.json

Create Customer User

The steps below are not strictly necessary. They are provided so you or the customer can do testing. Note that these are executed by the customer so they use --profile $CUSTOMER_PROFILE. We create an IAM user on the customer side for testing reading and writing to the customer’s bucket:

aws --profile $CUSTOMER_PROFILE iam create-user \
  --user-name ${CUSTOMER_USER} | tee /tmp/customer-user.json

Next we will create credentials for the customer user:

aws --profile ${CUSTOMER_PROFILE} iam create-access-key \
  --user-name ${CUSTOMER_USER} | tee /tmp/creds_customer_user.json

Now we can create the policy for the bucket so that the customer’s IAM user can read/write files to the bucket (note that this overwrites the previously applied policy-for-firm-root.json so we will need to add that back after testing):

cat > /tmp/policy.json<<EOF
{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Sid": "statement1",
         "Effect": "Allow",
         "Principal": {
            "AWS": "arn:aws:iam::${CUSTOMER_AWS_ACCT}:user/${CUSTOMER_USER}"
         },
         "Action": [
            "s3:GetBucketLocation",
            "s3:ListBucket"
         ],
         "Resource": [
            "arn:aws:s3:::${BUCKET}"
         ]
      },
      {
         "Sid": "statement2",
         "Effect": "Allow",
         "Principal": {
            "AWS": "arn:aws:iam::${CUSTOMER_AWS_ACCT}:user/${CUSTOMER_USER}"
         },
         "Action": [
             "s3:GetObject",
             "s3:PutObject",
             "s3:PutObjectAcl"
         ],
         "Resource": [
            "arn:aws:s3:::${BUCKET}/*"
         ]
      }
   ]
}
EOF

Now apply policy so customer can access bucket:

aws --profile ${CUSTOMER_PROFILE} s3api put-bucket-policy \
  --bucket $BUCKET --policy file:///tmp/policy.json

Verify customer can write to the bucket

echo "Example file with date `date`" > /tmp/example-file.txt
AWS_ACCESS_KEY_ID=`jq -r .AccessKey.AccessKeyId /tmp/creds_customer_user.json` \
  AWS_SECRET_ACCESS_KEY=`jq -r .AccessKey.SecretAccessKey /tmp/creds_customer_user.json` \
  aws s3api put-object --bucket $BUCKET --key example-file.txt \
  --body /tmp/example-file.txt
AWS_ACCESS_KEY_ID=`jq -r .AccessKey.AccessKeyId /tmp/creds_customer_user.json` \
  AWS_SECRET_ACCESS_KEY=`jq -r .AccessKey.SecretAccessKey /tmp/creds_customer_user.json` \
  aws s3api get-object --bucket $BUCKET --key example-file.txt /tmp/out.txt
cat /tmp/out.txt

Now that our testing is done, add back the policy for the firm root:

aws --profile ${CUSTOMER_PROFILE} s3api put-bucket-policy \
  --bucket $BUCKET --policy file:///tmp/policy-for-firm-root.json

Create Firm User and Policy

In this section we create an IAM user account in the firm’s AWS account that can access the customer’s bucket. Note that most of these commands use --profile ${FIRM_PROFILE} since they are executing in the firm’s AWS account.

Generally you would only need to create this firm user once. For this setup, you would need to apply the policy to the firm user for each customer bucket or create a more sophisticated policy with wild cards that would apply to all customer buckets but that is beyond the scope of this article.

Now we make an IAM account for your firm who can write to the desired bucket:

aws --profile ${FIRM_PROFILE} iam create-user \
  --user-name $FIRM_IAM_USER | tee /tmp/writer-user.json

Now we create access keys for the new user:

aws --profile ${FIRM_PROFILE} iam create-access-key \
  --user-name $FIRM_IAM_USER | tee /tmp/writer-creds.json

Now we setup a policy for the firm’s non-root user so it can access the bucket which root has been granted access to buy the customer.

cat > /tmp/writer-policy-iam.json<<EOF
{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Sid": "allowwriteruseriamaccess",
         "Effect": "Allow",
         "Action": [
            "s3:ListBucket",
            "s3:GetObject",
            "s3:PutObject",
            "s3:PutObjectAcl"
         ],
         "Resource": [
            "arn:aws:s3:::${BUCKET}",
            "arn:aws:s3:::${BUCKET}/*"
         ]
      }
   ]
}
EOF

Now we apply the policy where the firm’s admin is granting access to the non-root $FIRM_IAM_USER.

aws --profile ${FIRM_PROFILE} \
  iam put-user-policy --user-name $FIRM_IAM_USER --policy-name writer-policy \
  --policy-document file:///tmp/writer-policy-iam.json

Now we test writing and reading.

rm -f /tmp/out.txt
echo "Example file with date `date`" > /tmp/example-file.txt
AWS_ACCESS_KEY_ID=`jq -r .AccessKey.AccessKeyId /tmp/writer-creds.json` \
  AWS_SECRET_ACCESS_KEY=`jq -r .AccessKey.SecretAccessKey /tmp/writer-creds.json` \
  aws s3api put-object --bucket $BUCKET --key example-file.txt \
  --body /tmp/example-file.txt
AWS_ACCESS_KEY_ID=`jq -r .AccessKey.AccessKeyId /tmp/writer-creds.json` \
  AWS_SECRET_ACCESS_KEY=`jq -r .AccessKey.SecretAccessKey /tmp/writer-creds.json` \
  aws s3api get-object --bucket $BUCKET --key example-file.txt /tmp/out.txt
cat /tmp/out.txt

Cleanup

Now clean-up the user we created for testing on the customer side.

aws --profile ${CUSTOMER_PROFILE} iam delete-access-key \
   --user-name ${CUSTOMER_USER} --access-key-id \
     `jq -r .AccessKey.AccessKeyId /tmp/creds_customer_user.json`
aws --profile ${CUSTOMER_PROFILE} iam delete-user \
  --user-name $CUSTOMER_USER

Now clean-up the user we created for the firm.

aws --profile ${FIRM_PROFILE} iam delete-user-policy \
  --user-name $FIRM_IAM_USER --policy-name writer-policy
aws --profile ${FIRM_PROFILE} iam delete-access-key \
  --access-key-id `jq -r .AccessKey.AccessKeyId /tmp/writer-creds.json` \
  --user-name $FIRM_IAM_USER
aws --profile ${FIRM_PROFILE} iam delete-user \
  --user-name $FIRM_IAM_USER

You may also want to delete the bucket if desired.

• software
• aws