DevOps Stack Exchange
DevOps Stack Exchange

Questions tagged [amazon-s3]

Amazon Simple Storage Service. For questions about storing and retrieving DevOps-related data in S3, or using S3 for building a code pipeline.

Amazon S3 (Simple Storage Service) is an online storage web service offered by Amazon Web Services. Amazon S3 provides storage through a simple web services interface. It gives any developer access to the same highly scalable, reliable, secure, fast, inexpensive infrastructure that Amazon uses to run its own global network of web sites.

References

63 questions
34
votes
2 answers

What is a good strategy to keep my site online when S3 goes offline?

What is a good strategy to keep my site online when S3 goes offline? If S3 US East 1 goes offline, how should I have my app configured/structured to prevent that taking my entire site offline? What are the best strategies to diversify in this sort…
jcolebrand
  • 1,275
  • 10
  • 14
23
votes
1 answer

How do I work around the fact that AWS SQS is not HIPAA compliant?

I have a use case where data from S3 is queued into AWS SQS, which is in turn connected to CloudWatch, whose metrics will be triggering AWS Lambda. However, I want the architecture to be HIPAA compliant. So, I have come up with this idea: Once my…
Dawny33
  • 2,816
  • 3
  • 24
  • 62
21
votes
6 answers

View all AWS S3 buckets and list each buckets storage used

I have multiple AWS accounts and I need to list all S3 buckets per account and then view each buckets total size. Currently, I can only view the storage size of a single S3 bucket with: aws s3 ls s3://mybucket --recursive --human-readable…
Kyle Steenkamp
  • 1,172
  • 1
  • 9
  • 18
20
votes
2 answers

Is cross-region replication 100% fool-proof for S3 region outages?

Amazon S3 has an option of cross-region replication which should be pretty fault-tolerant against region/zone outages. Does that mean those who are ranting about the outage did not make use of this aspect? Or is that cross-region replication is not…
Dawny33
  • 2,816
  • 3
  • 24
  • 62
17
votes
2 answers

How to order physical backup of S3 data?

I'd like to order backup of data to be copied out from S3 to a physical drive (SATA HD) and to be shipped to the address. I know it's possible using Amazon Glacier, however I cannot find such option anywhere. What would be the easiest way to do…
kenorb
  • 8,011
  • 14
  • 43
  • 80
13
votes
2 answers

Can you use `aws-sdk` to store objects in Google Storage?

Google Storage claims they support interoperability in their solution with Amazon AWS S3 and Eucalyptus Systems APIs. source: https://cloud.google.com/storage/docs/interoperability Can the aws-sdk for Java, or Node.js, or Python, be used to store…
Evgeny Zislis
  • 9,023
  • 5
  • 39
  • 72
11
votes
3 answers

Best practices for external backups for S3 buckets?

I would like to know how everyone is making external backups of S3 buckets and best practices/recommendations for this. Regardless of how good AWS is it is still a single point of failure. Especially on a more corporate scale, vast amounts of…
Kyle Steenkamp
  • 1,172
  • 1
  • 9
  • 18
9
votes
1 answer

Is there any way I can rotate AWS KMS keys every time interval < 1 year?

According AWS KMS, keys can be rotated once every year. However, there is no way I can define a custom time period for that. However, I can keep changing the same in the bucket's policy manually. So, is there anyway I can automate that?
Dawny33
  • 2,816
  • 3
  • 24
  • 62
8
votes
5 answers

How to provide a user to access only a particular bucket in AWS S3?

I have a list of buckets in AWS S3. I have created an IAM user. I have an option to provide S3 full or read only access for a user using groups. Is there any options to provide access only to a particular bucket?
Sathishkumar Jayaraj
  • 183
  • 2
  • 6
6
votes
1 answer

Is it necessary to specify resource in S3 Bucket Policies?

Is it necessary to specify a resource attribute within the statements that make up an S3 bucket policy? The Access Policy Language Overview seems to suggest that the resource attribute is always included, though it doesn't come straight out and say…
Nic
  • 351
  • 1
  • 4
  • 12
5
votes
3 answers

How to monitor AWS S3 cross-region replication process?

Amazon S3 has a cross-region replication which will handle copy of new/updated objects to additional region. The problem is that solution does not provide visibility on state for replication process, for example at the moment there's no way to…
rombob
  • 657
  • 5
  • 16
5
votes
1 answer

How to apply using Terraform to launch multiple EC2 Resource with different configs (VPC not maintained by TF)

We are planning to use Terraform for provisioning EC2 instance which will host Oracle DB. While we are able to achieve this, there are some questions for which I am searching for suggestions / feedback's. Below is the directory structure we are…
Dvusrgme
  • 153
  • 1
  • 4
4
votes
1 answer

Triggering a SNS notification when an object is accessed (downloaded) in AWS S3

I have managed to trigger events on my bucket when objects are uploaded (PUT) but I am surprised that there are no triggers called when objects are downloaded (GETs), as can be seen in this screenshot of the interface: I want to be notified of each…
W.M.
  • 187
  • 5
4
votes
4 answers

Is it possible to mount an s3 bucket as a point in a docker container?

I have a Java EE packaged as war file stored in an AWS s3 bucket. I will like to mount the folder containing the .war file as a point in my docker container. I have managed to do this on my local machine. Below is an example of a JBoss wildfly…
christian Martin
  • 85
  • 1
  • 1
  • 5
4
votes
2 answers

Impact on policy after deleting IAM user

I have created one IAM User named "test" with S3 and Glacier Permission. Now with user "test" i created one life cycle policy to move data from S3 to Glacier at every 5 days. Now, i delete IAM user "test" because i don't need it anymore. So my…
Jay
  • 1,064
  • 2
  • 12
  • 21
1
2 3 4 5