Re: [PERFORM] slow query - will CLUSTER help?

2013-12-23 Thread Shaun Thomas
> What are your thoughts on the right way to use SSDs in a RAID to > enhance postgres I/O performance? In an earlier reply, you > indicated one of a "RAID1+0 consisting of several spindles, > NVRAM-based solution (SSD or PCIe card), or a SAN" Well, it's a tiered approach. If you can identify yo

Re: [PERFORM] slow query - will CLUSTER help?

2013-12-20 Thread Gavin Flower
On 21/12/13 05:11, Shaun Thomas wrote: [...] . Of course, don't forget to buy modules in multiples of four, otherwise you're not taking advantage of all the CPU's memory channels. :) Note some processors have 3 (three) memory channels! And I know of some with 4 memory channels. So it is im

Re: [PERFORM] slow query - will CLUSTER help?

2013-12-20 Thread Shaun Thomas
On 12/20/2013 09:57 AM, Sev Zaslavsky wrote: There is a separate RAID-1 for WAL, another for tablespace and another for operating system. I tend to stick to DB-size / 10 as a minimum, but I also have an OLTP system. For a more OLAP-type, the ratio is negotiable. The easiest way to tell is t

Re: [PERFORM] slow query - will CLUSTER help?

2013-12-20 Thread Shaun Thomas
On 12/19/2013 03:24 PM, Sergey Konoplev wrote: 2. You are limited with IO I would also suggest you to upgrade your storage in this case. I think this is the case. If I recall correctly, his setup includes a single RAID-1 for everything, and he only has 32GB of RAM. In fact, the WAL traffic f

Re: [PERFORM] slow query - will CLUSTER help?

2013-12-19 Thread Sergey Konoplev
On Thu, Dec 19, 2013 at 12:54 PM, Sev Zaslavsky wrote: > On 12/19/2013 3:34 PM, Sergey Konoplev wrote: >> On Thu, Dec 12, 2013 at 9:30 AM, Sev Zaslavsky wrote: >>> Table rt_h_nbbo contains several hundred million rows. All rows for a >>> given >>> entry_date are appended to this table in an over

Re: [PERFORM] slow query - will CLUSTER help?

2013-12-19 Thread Sergey Konoplev
On Thu, Dec 12, 2013 at 9:30 AM, Sev Zaslavsky wrote: [...] > Table rt_h_nbbo contains several hundred million rows. All rows for a given > entry_date are appended to this table in an overnight process every night - > on the order of several million rows per day. [...] > I perceive an ineffici

Re: [PERFORM] slow query - will CLUSTER help?

2013-12-19 Thread Jeff Janes
On Thu, Dec 12, 2013 at 9:30 AM, Sev Zaslavsky wrote: > Hello, > > I've got a very simple table with a very simple SELECT query, but it takes > longer on the initial run than I'd like, so I want to see if there is a > strategy to optimize this. > > Table rt_h_nbbo contains several hundred millio

Re: [PERFORM] slow query - will CLUSTER help?

2013-12-19 Thread Kevin Grittner
Sev Zaslavsky wrote: I want to agree with everything Shaun said and add a tiny bit. > Does loading 24Gb of data in 21 sec seem "about right"? It's a little on the slow side.  You said 1634 page reads.  At 9 ms per read that would be 14.7 seconds.  But I'm basing the 9 ms per page read on my Lin

Re: [PERFORM] slow query - will CLUSTER help?

2013-12-19 Thread Shaun Thomas
On 12/12/2013 11:30 AM, Sev Zaslavsky wrote: _First question_ is: Does loading 24Gb of data in 21 sec seem "about right" (hardware specs at bottom of email)? That's actually pretty good. 24GB is a lot of data to process. _Second question_: Is it possible to tell postgres to physically store

[PERFORM] slow query - will CLUSTER help?

2013-12-19 Thread Sev Zaslavsky
Hello,_ _ I've got a very simple table with a very simple SELECT query, but it takes longer on the initial run than I'd like, so I want to see if there is a strategy to optimize this. Table rt_h_nbbo contains several hundred million rows. All rows for a given entry_date are appended to this