Postgres backup and WAL to S3

Question

We are looking for a solution for the following problem:

We have set up streaming replication so we have a master DB and a slave DB, we want to have basebackups and WAL files sent to our S3 storage periodically.

I have looked at:

• WAL-e which seemed to suit us perfectly, except that it does not support multiple tablespaces, which is a deal breaker.

• Barman supports this but does not support S3, which is also a deal breaker, although it may be possible with pre- and post-backup hooks which I am currently looking into.

• OmniPITR doesn't support S3 either.

Is there an application that does all these things already, or will I have to do some scripting?

Answer 1

You may want to reconsider wal-e from Instagram which does just this,

wal-e Simple Continuous Archiving for Postgres

It supports AWS. And as of November 2013, shortly after you asked the question, it supports table spaces.

Answer 2

After a lot of research, I ended up using Barman and a custom post-backup script, it works very good!