Re: Backup PostgreSQL from RDS straight to S3

2019-09-18 Thread Steven Lembark
s3fs available on linux allows mounting S3 directly as a local filesystem. At that point something like: pg_dump ... | gzip -9 -c > /mnt/s3-mount-point/$basename.pg_dump.gz; will do the deed nicely. If your S3 volume is something like your_name_here.com/pg_dump then you could parallize it by

Re: Backup PostgreSQL from RDS straight to S3

2019-09-18 Thread Adrian Klaver
On 9/18/19 12:04 PM, Anthony DeBarros wrote: Great, thanks. The question I have, which at first glance isn't covered there, is whether those instructions will at any point bring the dump file onto the EC2 box, either in memory or temp file storage, on its way to S3? I don't know enough about ho

Re: Backup PostgreSQL from RDS straight to S3

2019-09-18 Thread Anthony DeBarros
Great, thanks. The question I have, which at first glance isn't covered there, is whether those instructions will at any point bring the dump file onto the EC2 box, either in memory or temp file storage, on its way to S3? I don't know enough about how Linux handles data piped from one command to th

Re: Backup PostgreSQL from RDS straight to S3

2019-09-18 Thread Adrian Klaver
On 9/18/19 11:32 AM, Anthony DeBarros wrote: Hi, folks -- I'm a longtime PostgreSQL user but a bit of a noob when it comes to maintenance. Question: I'm running PostgreSQL 11 on Amazon RDS. Also have an EC2 box running Ubuntu that runs some Python scripts that collect data into PostgreSQL. W

Backup PostgreSQL from RDS straight to S3

2019-09-18 Thread Anthony DeBarros
Hi, folks -- I'm a longtime PostgreSQL user but a bit of a noob when it comes to maintenance. Question: I'm running PostgreSQL 11 on Amazon RDS. Also have an EC2 box running Ubuntu that runs some Python scripts that collect data into PostgreSQL. We're doing the standard RDS backups. However, I'd