[PERFORM] Hardware advice for scalable warehouse db

2011-07-15 Thread chris
Hi list, My employer will be donated a NetApp FAS 3040 SAN [1] and we want to run our warehouse DB on it. The pg9.0 DB currently comprises ~1.5TB of tables, 200GB of indexes, and grows ~5%/month. The DB is not update critical, but undergoes larger read and insert operations frequently. My

Re: [PERFORM] Hardware advice for scalable warehouse db

2011-07-15 Thread Greg Smith
chris wrote: My employer is a university with little funds and we have to find a cheap way to scale for the next 3 years, so the SAN seems a good chance to us. A SAN is rarely ever the cheapest way to scale anything; you're paying extra for reliability instead. I was thinking to put the

Re: [PERFORM] Inoptimal query plan for max() and multicolumn index

2011-07-15 Thread Gaetano Mendola
On 20/06/2011 07:35, Vladimir Kulev wrote: But this does not work: # explain analyze select max(timestamp) from sms where number in ('5502712','5802693','5801981'); Try to rewrite that query this way: explain analyze select timestamp from sms where number in ('5502712','5802693','5801981')

Re: [PERFORM] Inoptimal query plan for max() and multicolumn index

2011-07-15 Thread Gaetano Mendola
On 20/06/2011 07:35, Vladimir Kulev wrote: But this does not work: # explain analyze select max(timestamp) from sms where number in ('5502712','5802693','5801981'); Try to rewrite that query this way: explain analyze select timestamp from sms where number in ('5502712','5802693','5801981')

Re: [PERFORM] Hardware advice for scalable warehouse db

2011-07-15 Thread jesper
1 x Intel Xeon X5670, 6C, 2.93GHz, 12M Cache 16 GB (4x4GB) Low Volt DDR3 1066Mhz PERC H700 SAS RAID controller 4 x 300 GB 10k SAS 6Gbps 2.5 in RAID 10 Apart from Gregs excellent recommendations. I would strongly suggest more memory. 16GB in 2011 is really on the low side. PG is using

Re: [PERFORM] Hardware advice for scalable warehouse db

2011-07-15 Thread Robert Schnabel
On 7/15/2011 2:10 AM, Greg Smith wrote: chris wrote: My employer is a university with little funds and we have to find a cheap way to scale for the next 3 years, so the SAN seems a good chance to us. A SAN is rarely ever the cheapest way to scale anything; you're paying extra for reliability

Re: [PERFORM] Unexpected seq scans when expected result is 1 row out of milions

2011-07-15 Thread Tom Lane
Svetlin Manavski svetlin.manav...@gmail.com writes: The question is: why do we get a seq scan on appqosdata.tcpsessiondata_1 and appqosdata.tcpsessiondata_2 even if the planner estimates correctly 1 row out of millions could potentially be selected? As you can see ~90% of the time is spent on

Re: [PERFORM] Hardware advice for scalable warehouse db

2011-07-15 Thread Scott Marlowe
On Fri, Jul 15, 2011 at 12:34 AM, chris chri...@gmx.net wrote: I was thinking to put the WAL and the indexes on the local disks, and the rest on the SAN. If funds allow, we might downgrade the disks to SATA and add a 50 GB SATA SSD for the WAL (SAS/SATA mixup not possible). Just to add to the

Re: [PERFORM] Hardware advice for scalable warehouse db

2011-07-15 Thread Scott Marlowe
On Fri, Jul 15, 2011 at 10:39 AM, Robert Schnabel schnab...@missouri.edu wrote: I'm curious what people think of these: http://www.pc-pitstop.com/sas_cables_enclosures/scsase166g.asp I currently have my database on two of these and for my purpose they seem to be fine and are quite a bit less

Re: [PERFORM] Hardware advice for scalable warehouse db

2011-07-15 Thread Josh Berkus
Just to add to the conversation, there's no real advantage to putting WAL on SSD. Indexes can benefit from them, but WAL is mosty seqwuential throughput and for that a pair of SATA 1TB drives at 7200RPM work just fine for most folks. Actually, there's a strong disadvantage to putting WAL

Re: [PERFORM] Hardware advice for scalable warehouse db

2011-07-15 Thread chris r.
Hi list, Thanks a lot for your very helpful feedback! I've tested MD1000, MD1200, and MD1220 arrays before, and always gotten seriously good performance relative to the dollars spent Great hint, but I'm afraid that's too expensive for us. But it's a great way to scale over the years, I'll keep

Re: [PERFORM] Hardware advice for scalable warehouse db

2011-07-15 Thread Rob Wultsch
On Fri, Jul 15, 2011 at 11:49 AM, chris r. chri...@gmx.net wrote: Hi list, Thanks a lot for your very helpful feedback! I've tested MD1000, MD1200, and MD1220 arrays before, and always gotten seriously good performance relative to the dollars spent Great hint, but I'm afraid that's too

Re: [PERFORM] Hardware advice for scalable warehouse db

2011-07-15 Thread Josh Berkus
On 7/14/11 11:34 PM, chris wrote: Any comments on the configuration? Any experiences with iSCSI vs. Fibre Channel for SANs and PostgreSQL? If the SAN setup sucks, do you see a cheap alternative how to connect as many as 16 x 2TB disks as DAS? Here's the problem with iSCSI: on gigabit ethernet,

Re: [PERFORM] Statistics and Multi-Column indexes

2011-07-15 Thread lars
On 07/10/2011 02:31 PM, Samuel Gendler wrote: What about partitioning tables by tenant id and then maintaining indexes on each partition independent of tenant id, since constraint exclusion should handle filtering by tenant id for you. That seems like a potentially more tolerable variant of

Re: [PERFORM] UPDATEDs slowing SELECTs in a fully cached database

2011-07-15 Thread lars
On 07/14/2011 04:03 PM, Jeff Janes wrote: On Wed, Jul 13, 2011 at 3:41 PM, larslhofha...@yahoo.com wrote: On 07/13/2011 11:42 AM, Kevin Grittner wrote: So transactions without an XID *are* sensitive to synchronous_commit. That's likely a useful clue. How much did it help the run time of the