Re: [PERFORM] How to setup disk spindles for best performance

2008-08-20 Thread Joshua D. Drake
Christiaan Willemsen wrote: So, what you are basically saying, is that a single mirror is in general more than enough to facilitate the transaction log. http://www.commandprompt.com/blogs/joshua_drake/2008/04/is_that_performance_i_smell_ext2_vs_ext3_on_50_spindles_testing_for_postgresql/ http:/

Re: [PERFORM] How to setup disk spindles for best performance

2008-08-20 Thread Christiaan Willemsen
So, what you are basically saying, is that a single mirror is in general more than enough to facilitate the transaction log. So it would not be smart to put the indexes onto a separate disk spindle to improve index performance? On Aug 21, 2008, at 3:49 AM, Merlin Moncure wrote: On Wed, Au

Re: [PERFORM] How to setup disk spindles for best performance

2008-08-20 Thread Scott Marlowe
On Wed, Aug 20, 2008 at 4:25 PM, Christiaan Willemsen <[EMAIL PROTECTED]> wrote: > I'm currently trying to find out what the best configuration is for our new > database server. It will server a database of about 80 GB and growing fast. > The new machine has plenty of memory (64GB) and 16 SAS disks

Re: [PERFORM] How to setup disk spindles for best performance

2008-08-20 Thread Merlin Moncure
On Wed, Aug 20, 2008 at 6:25 PM, Christiaan Willemsen <[EMAIL PROTECTED]> wrote: > I'm currently trying to find out what the best configuration is for our new > database server. It will server a database of about 80 GB and growing fast. > The new machine has plenty of memory (64GB) and 16 SAS disks

[PERFORM] How to setup disk spindles for best performance

2008-08-20 Thread Christiaan Willemsen
I'm currently trying to find out what the best configuration is for our new database server. It will server a database of about 80 GB and growing fast. The new machine has plenty of memory (64GB) and 16 SAS disks, of wich two are already in use as a mirror for the OS. The rest can be used f

Re: [PERFORM] Postgres not using array

2008-08-20 Thread André Volpato
David Wilson escreveu: On Wed, Aug 20, 2008 at 2:30 PM, André Volpato <[EMAIL PROTECTED]> wrote: The CPU is 100% used since a few hours ago. Can anyone tell why? Sounds like you've just got a CPU bound query. The data may even all be in cache. Some information on database size, alon

Re: [PERFORM] Postgres not using array

2008-08-20 Thread David Wilson
On Wed, Aug 20, 2008 at 2:30 PM, André Volpato <[EMAIL PROTECTED]> wrote: > The CPU is 100% used since a few hours ago. Can anyone tell why? Sounds like you've just got a CPU bound query. The data may even all be in cache. Some information on database size, along with EXPLAIN results for your qu

[PERFORM] Postgres not using array

2008-08-20 Thread André Volpato
Folks, In a fresh Pg8.3.3 box, we created a 3-stripe RAID0 array. The write speed is about 180MB/s, as shown by dd : # dd if=/dev/zero of=/postgres/base/teste2 bs=1024 count=500 500+0 records in 500+0 records out 512000 bytes (5,1 GB) copied, 28,035 seconds, 183 MB/s BTW, /postg

Re: [PERFORM] Slow query with a lot of data

2008-08-20 Thread Tom Lane
"Scott Carey" <[EMAIL PROTECTED]> writes: > The planner actually thinks there will only be 28704 rows returned of width > 12. But it chooses to sort 53 million rows before aggregating. Thats > either a bug or there's something else wrong here. That is the wrong way > to aggregate those results

Re: [PERFORM] Optimizing a VIEW

2008-08-20 Thread Tom Lane
Decibel! <[EMAIL PROTECTED]> writes: > On Aug 16, 2008, at 9:19 PM, Gurjeet Singh wrote: >> Awesome Any comments on the performance of hstore? > I've looked at it but haven't actually used it. One thing I wish it > did was to keep a catalog somewhere of the "names" that it's seen so > that

Re: [PERFORM] Slow query with a lot of data

2008-08-20 Thread Scott Carey
Ok, so the problem boils down to the sort at the end. The query up through the merge join on domain is as fast as its going to get. The sort at the end however, should not happen ideally. There are not that many rows returned, and it should hash_aggregate if it thinks there is enough space to do

Re: [PERFORM] Optimizing a VIEW

2008-08-20 Thread Decibel!
On Aug 17, 2008, at 10:21 AM, Madison Kelly wrote: Truth be told, I sort of expected this would be what I had to do. I think I asked this more in hoping that there might be some "magic" I didn't know about, but I see now that's not the case. :) As my data points grow to 500,000+, the time it

Re: [PERFORM] Optimizing a VIEW

2008-08-20 Thread Decibel!
On Aug 16, 2008, at 9:19 PM, Gurjeet Singh wrote: For you very specific case, I recommend you check out contrib/hstore: http://www.postgresql.org/docs/current/static/hstore.html Awesome Any comments on the performance of hstore? I've looked at it but haven't actually used it. One thing I

Re: [PERFORM] Slow query with a lot of data

2008-08-20 Thread Scott Carey
More work_mem will make the sort fit more in memory and less on disk, even with the same query plan. On Wed, Aug 20, 2008 at 12:54 AM, Moritz Onken <[EMAIL PROTECTED]>wrote: > > Am 19.08.2008 um 17:23 schrieb Moritz Onken: > > >> Am 19.08.2008 um 16:49 schrieb Scott Carey: >> >> What is your wo

Re: [PERFORM] PostgreSQL+Hibernate Performance

2008-08-20 Thread Mark Lewis
Yes, we use connection pooling. As I recall Hibernate ships with c3p0 connection pooling built-in, which is what we use. We were happy enough with c3p0 that we ended up moving our other non-hibernate apps over to it, away from DBCP. pgpool does connection pooling at a socket level instead of in

Re: [PERFORM] PostgreSQL+Hibernate Performance

2008-08-20 Thread Mark Lewis
The tradeoffs for multiple indexes are more or less as follows: 1. Having the right indexes makes queries faster, often dramatically so. 2. But more indexes makes inserts/updates slower, although generally not dramatically slower. 3. Each index requires disk space. With several indexes, you can

Re: [PERFORM] PostgreSQL+Hibernate Performance

2008-08-20 Thread Matthew Wakeling
On Wed, 20 Aug 2008, Kranti K K Parisa™ wrote: creating multiple indexes on same column will effect performance?  for example: index1 : column1, column2, column3 index2: column1 index3: column2, index4: column3 index5: column1,column2 The sole purpose of indexes is to affect performance. Howe

Re: [PERFORM] PostgreSQL+Hibernate Performance

2008-08-20 Thread Kranti K K Parisa™
creating multiple indexes on same column will effect performance? for example: index1 : column1, column2, column3 index2: column1 index3: column2, index4: column3 index5: column1,column2 which means, i am trying fire the SQL queries keeping columns in the where conditions. and the possibilities

Re: [PERFORM] Software vs. Hardware RAID Data

2008-08-20 Thread Mark Wong
On Wed, Aug 20, 2008 at 12:53 AM, Tommy Gildseth <[EMAIL PROTECTED]> wrote: > Mark Wong wrote: >> >> Hi all, >> >> We started an attempt to slice the data we've been collecting in >> another way, to show the results of software vs. hardware RAID: >> >> >> http://wiki.postgresql.org/wiki/HP_ProLiant

Re: [PERFORM] PostgreSQL+Hibernate Performance

2008-08-20 Thread Nikolas Everett
The only thing thats bitten me about hibernate + postgres is that when inserting into partitioned tables, postgres does not reply with the number of rows that hibernate expected. My current (not great) solution is to define a specific SQLInsert annotation and tell it not to do any checking like so

Re: [PERFORM] Software vs. Hardware RAID Data

2008-08-20 Thread Mark Wong
On Tue, Aug 19, 2008 at 10:49 PM, <[EMAIL PROTECTED]> wrote: > On Tue, 19 Aug 2008, Mark Wong wrote: > >> Hi all, >> >> We started an attempt to slice the data we've been collecting in >> another way, to show the results of software vs. hardware RAID: >> >> >> http://wiki.postgresql.org/wiki/HP_Pr

Re: [PERFORM] PostgreSQL+Hibernate Performance

2008-08-20 Thread Kranti K K Parisa™
Hi Mark, Thank you very much for the information. I will analyse the DB structure and create indexes on PG directly. Are you using any connection pooling like DBCP? or PG POOL? Regards, KP On Wed, Aug 20, 2008 at 8:05 PM, Mark Lewis <[EMAIL PROTECTED]> wrote: > On Wed, 2008-08-20 at 17:55 +053

Re: [PERFORM] PostgreSQL+Hibernate Performance

2008-08-20 Thread Mark Lewis
On Wed, 2008-08-20 at 17:55 +0530, Kranti K K Parisa™ wrote: > Hi, > > Can anyone suggest the performance tips for PostgreSQL using > Hibernate. > > One of the queries: > > - PostgreSQL has INDEX concept and Hibernate also has Column INDEXes. > Which is better among them? or creating either of t

Re: [PERFORM] pgsql do not handle NULL constants in the view

2008-08-20 Thread Sergey Hripchenko
Thx it helps. Shame on me %) I forgot that NULL itself has no type, and thought that each constant in the view are casted to the resulting type at the creation time. -Original Message- From: Tom Lane [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 20, 2008 4:54 PM To: Sergey Hripchenko

Re: [PERFORM] pgsql do not handle NULL constants in the view

2008-08-20 Thread Tom Lane
Sergey Hripchenko <[EMAIL PROTECTED]> writes: > CREATE OR REPLACE VIEW bar AS > SELECT * > FROM ( > ( > SELECT calldate, duration, billsec, > get_asterisk_cdr_caller_id(accountcode) AS caller_id > FROM asterisk_cdr > ) UNION ALL ( > SELECT start_time, get_interval_s

[PERFORM] PostgreSQL+Hibernate Performance

2008-08-20 Thread Kranti K K Parisa™
Hi, Can anyone suggest the performance tips for PostgreSQL using Hibernate. One of the queries: - PostgreSQL has INDEX concept and Hibernate also has Column INDEXes. Which is better among them? or creating either of them is enough? or need to create both of them? and any more performace aspects

[PERFORM] pgsql do not handle NULL constants in the view

2008-08-20 Thread Sergey Hripchenko
Hi all, Looks like I found a bug with views optimization: For example create a test view: CREATE OR REPLACE VIEW bar AS SELECT * FROM ( ( SELECT calldate, duration, billsec, get_asterisk_cdr_caller_id(accountcode) AS caller_id FROM asterisk_cdr ) UNION ALL ( SELE

Re: [PERFORM] pgsql do not handle NULL constants in the view

2008-08-20 Thread Sergey Hripchenko
Forgot to add asteriskpilot=> SELECT version(); version PostgreSQL 8.2.9 on i386-redhat-linux-gnu, compiled by GCC gcc (GCC) 4.1.2 20070925 (Re

Re: [PERFORM] Slow query with a lot of data

2008-08-20 Thread Zoltan Boszormenyi
Moritz Onken írta: > > Am 19.08.2008 um 17:23 schrieb Moritz Onken: > >> >> Am 19.08.2008 um 16:49 schrieb Scott Carey: >> >>> What is your work_mem set to? The default? >>> >>> Try increasing it significantly if you have the RAM and seeing if >>> that affects the explain plan. You may even want

Re: [PERFORM] Slow query with a lot of data

2008-08-20 Thread Moritz Onken
Am 19.08.2008 um 17:23 schrieb Moritz Onken: Am 19.08.2008 um 16:49 schrieb Scott Carey: What is your work_mem set to? The default? Try increasing it significantly if you have the RAM and seeing if that affects the explain plan. You may even want to set it to a number larger than the

Re: [PERFORM] Software vs. Hardware RAID Data

2008-08-20 Thread Tommy Gildseth
Mark Wong wrote: Hi all, We started an attempt to slice the data we've been collecting in another way, to show the results of software vs. hardware RAID: http://wiki.postgresql.org/wiki/HP_ProLiant_DL380_G5_Tuning_Guide#Hardware_vs._Software_Raid Comments, suggestions, criticisms, et al. welco