6

Goal: Get files

  • from Bucket 1 in ca-central-1 in Account A
  • to Bucket 2 in us-east-1 in Account B
  • using the AWS CLI from a third machine using an the IAM role with correct S3 read and write permissions (assume unless that's unlikely)

I got the error:

ClientError: An error occurred (AccessDenied) when calling the CopyObject operation: VPC endpoints do not support cross-region requests

and this broke my mental model about how S3 works. I thought that S3 was not behind any VPC and that VPC endpoints were just about an alternate routing pathway (other than the internet) for a machine within a private subnet.

But if you're using the CLI and asking to transfer files from one S3 bucket to another, why would a VPC come into play at all?

Ben Ogorek
  • 163

1 Answers1

5

I assume since you say "using the AWS CLI from a third machine using an the IAM role" that your "third machine" (why third?) is an EC2 instance inside a VPC.

EC2 instances are inside VPCs. If that VPC has an S3 VPC endpoint then the EC2 instance will use it, as use of endpoints is based on DNS. You've found that VPC endpoints don't support cross-region copying.

Options:

  • Use built in S3 replication to copy the bucket / folder, but this is probably not flexible enough to do individual files
  • Remove your VPC endpoints
  • Use the --endpoint-url of the S3 CLI to manually specify the URL of a public S3 endpoint. I'm not 100% sure this will work but it's worth a try
  • Write a lambda function to do the copy, make sure it runs outside the VPC - this is the default unless you configure it to run inside the VPC
Tim
  • 33,870
  • 7
  • 56
  • 84