Re: Using AWS ephemeral SSD storage for production database workload?

2018-01-29 Thread Pritam Barhate
Thanks Ben and Sam for sharing your experience. On Jan 30, 2018 8:52 AM, "Ben Chobot" wrote: > > > On Jan 29, 2018, at 8:05 PM, Sam Gendler > wrote: > > > > Why not use EBS storage, but don’t use provisioned iops SSDs (io1) for > the ebs volume. Just use the default storage type (gp2) and live

Re: Using AWS ephemeral SSD storage for production database workload?

2018-01-29 Thread Ben Chobot
> On Jan 29, 2018, at 8:05 PM, Sam Gendler wrote: > > Why not use EBS storage, but don’t use provisioned iops SSDs (io1) for the > ebs volume. Just use the default storage type (gp2) and live with the 3000 > IOPS peak for 30 minutes that that allows. You’d be amazed at just how much > I/o can

Re: Using AWS ephemeral SSD storage for production database workload?

2018-01-29 Thread Sam Gendler
Why not use EBS storage, but don’t use provisioned iops SSDs (io1) for the ebs volume. Just use the default storage type (gp2) and live with the 3000 IOPS peak for 30 minutes that that allows. You’d be amazed at just how much I/o can be handled within the default IOPS allowance, though bear in mind

Re: Using AWS ephemeral SSD storage for production database workload?

2018-01-29 Thread Ben Madin
G'day all, We have been doing this in production for about five years, the client is aware of the trade off between speed, cost and availability. (By this I mean, if it goes down for a few minutes, no big concern to them). We had around 2 million users, but very small payloads. We take full datab

Re: Using AWS ephemeral SSD storage for production database workload?

2018-01-29 Thread Pritam Barhate
Thank you, Paul and Steven, for the information. @Paul: Thanks for the link. Planning to read through most of fdr's comments on Hacker News. @Steven: > Q: Why not just use RDS? > It'll be simpler. Already using it for multiple deployments. I am primarily a programmer. But now want to get out o

Re: Using AWS ephemeral SSD storage for production database workload?

2018-01-29 Thread Paul A Jungwirth
> I have come across some ideas on the Internet > where people hinted at running production PostgreSQL workloads > on AWS ephemeral SSD storage. I think people were more serious about that before AWS introduced PIOPS. I wouldn't do this unless I had streaming replication to a standby, plus regular

Re: Using AWS ephemeral SSD storage for production database workload?

2018-01-29 Thread Steven Lembark
On Mon, 29 Jan 2018 23:27:32 +0530 Pritam Barhate wrote: > In short, I am just trying to learn from other people's experience. This is identical to solutions that use tmpfs on linux for database storage or dealing with a fully failed storage system. Think about what you'd do if a RAID controll

Re: Using AWS ephemeral SSD storage for production database workload?

2018-01-29 Thread Pritam Barhate
>> Both log shipping and async replication are ancient features, and should >> be well understood. What exactly is unclear? I know about these and I know how to operate them also. The only part I am concerned about is the ephemeral storage. The risk appetite around it and the steps people take in

Re: Using AWS ephemeral SSD storage for production database workload?

2018-01-29 Thread Tomas Vondra
On 01/29/2018 05:41 PM, Pritam Barhate wrote: > Hi everyone,  > > As you may know, EBS volumes though durable are very costly when you > need provisioned IOPS. As opposed to this AWS instance attached > ephemeral SSD is very fast but isn't durable. > > I have come across some ideas on the Inte

Using AWS ephemeral SSD storage for production database workload?

2018-01-29 Thread Pritam Barhate
Hi everyone, As you may know, EBS volumes though durable are very costly when you need provisioned IOPS. As opposed to this AWS instance attached ephemeral SSD is very fast but isn't durable. I have come across some ideas on the Internet where people hinted at running production PostgreSQL worklo