Re: [PERFORM] Index on two columns not used

2006-10-23 Thread Péter Kovács
Markus, Thank you for your kind explanation. Peter Markus Schaber wrote: Hi, Peter, Péter Kovács wrote: Sorry for the amateurish question, but what are "heap tuples"? Also, my understanding is that the following statement applies only for composite indexes: "PostgreSQL can't use the valu

Re: [PERFORM] Best COPY Performance

2006-10-23 Thread Craig A. James
Jim C. Nasby wrote: http://stats.distributed.net used to use a perl script to do some transformations before loading data into the database. IIRC, when we switched to using C we saw 100x improvement in speed, so I suspect that if you want performance perl isn't the way to go. I think you can comp

[PERFORM] Problems using a function in a where clause

2006-10-23 Thread Mara Dalponte
Hello, I have a query with several join operations and applying the same filter condition over each involved table. This condition is a complex predicate over an indexed timestamp field, depending on some parameters. To factorize code, I wrote the filter into a plpgsql function, but the resulti

[PERFORM] Copy database performance issue

2006-10-23 Thread Steve
Hello there; I've got an application that has to copy an existing database to a new database on the same machine. I used to do this with a pg_dump command piped to psql to perform the copy; however the database is 18 gigs large on disk and this takes a LONG time to do. So I read up, found

Re: [PERFORM] Best COPY Performance

2006-10-23 Thread Joshua D. Drake
> Ultimately, you might be best of using triggers instead of rules for the > partitioning since then you could use copy. Or go to raw insert commands > that are wrapped in a transaction. My experience is that triggers are quite a bit faster than rules in any kind of partitioning that involves mor

Re: [PERFORM] Optimizing disk throughput on quad Opteron

2006-10-23 Thread Jim C. Nasby
On Sat, Oct 21, 2006 at 08:43:05AM -0700, John Philips wrote: > I heard some say that the transaction log should be on > it's own array, others say it doesn't hurt to have it > on the same array as the OS. Is it really worthwhile > to put it on it's own array? It all depends on the controller and

Re: [PERFORM] Best COPY Performance

2006-10-23 Thread Jim C. Nasby
On Mon, Oct 23, 2006 at 11:10:19AM -0400, Worky Workerson wrote: > >> I am most interested in loading two tables, one with about 21 (small) > >> VARCHARs where each record is about 200 bytes, and another with 7 > >> INTEGERs, 3 TIMESTAMPs, and 1 BYTEA where each record is about 350 > >> bytes. > >

Re: [PERFORM] New hardware thoughts

2006-10-23 Thread Jim C. Nasby
On Sat, Oct 21, 2006 at 12:12:59AM +0930, Shane Ambler wrote: > Generally more disks at slower speed - 2 10K disks in raid 0 is faster > than 1 15K disk. More disks also allow more options. Not at writing they're not (unless you're using RAID0... ugh). -- Jim Nasby

Re: [PERFORM] Vacuum and Memory Loss

2006-10-23 Thread Jim C. Nasby
On Mon, Oct 23, 2006 at 09:45:59AM +0100, Richard Huxton wrote: > Mike wrote: > >Hello friends, > > > >I am responsible for maintaining a high volume website using postgresql > >8.1.4. Given the amount of reads and writes, I vacuum full the server a > >few times a week around 1, 2 AM shutting down

Re: [PERFORM] New hardware thoughts

2006-10-23 Thread Joshua D. Drake
Vivek Khera wrote: > > On Oct 23, 2006, at 5:08 PM, Joshua D. Drake wrote: > >>> >>> They don't randomly change the controllers under the same name. If you >>> order a PERC4e/Si controller you will get the same controller every >>> time. >> >> Actually Vivek this isn't true. Yes the hardware wil

Re: [PERFORM] New hardware thoughts

2006-10-23 Thread Vivek Khera
On Oct 23, 2006, at 5:08 PM, Joshua D. Drake wrote: They don't randomly change the controllers under the same name. If you order a PERC4e/Si controller you will get the same controller every time. Actually Vivek this isn't true. Yes the hardware will likely be the same, but the firmware

Re: [PERFORM] Optimizing disk throughput on quad Opteron

2006-10-23 Thread Joshua D. Drake
Vivek Khera wrote: > > On Oct 23, 2006, at 4:59 PM, Joshua D. Drake wrote: > >>> If I had $50k budget, I'd be buying the SunFire X4500 and running >>> Solaris + ZFS on it. However, you're limited to 2 dual core Opterons, >>> it seems. >> >> The HP 585 will give you quad dual core :) > > but can

Re: [PERFORM] Optimizing disk throughput on quad Opteron

2006-10-23 Thread Vivek Khera
On Oct 23, 2006, at 4:59 PM, Joshua D. Drake wrote: If I had $50k budget, I'd be buying the SunFire X4500 and running Solaris + ZFS on it. However, you're limited to 2 dual core Opterons, it seems. The HP 585 will give you quad dual core :) but can you sling the bits to and from the dis

Re: [PERFORM] New hardware thoughts

2006-10-23 Thread Joshua D. Drake
> > FUD!!! > > They don't randomly change the controllers under the same name. If you > order a PERC4e/Si controller you will get the same controller every > time. Actually Vivek this isn't true. Yes the hardware will likely be the same, but the firmware rev will likely be different and I have

Re: [PERFORM] New hardware thoughts

2006-10-23 Thread Vivek Khera
On Oct 20, 2006, at 10:58 AM, Dave Cramer wrote: My advice is to find another supplier. check the archives for Dell. Not necessarily bad to go with Dell. There are *some* of their controllers that are wicked fast in some configurations. However, finding which ones are fast is very trick

Re: [PERFORM] Optimizing disk throughput on quad Opteron

2006-10-23 Thread Joshua D. Drake
Vivek Khera wrote: > > On Oct 21, 2006, at 11:43 AM, John Philips wrote: > >> Can you guys see any glaring bottlenecks in my layout? >> Any other suggestions to offer (throw in more >> controllers, different RAID layout, etc.)? Our budget >> limit is $50k. > > If I had $50k budget, I'd be buyi

Re: [PERFORM] Optimizing disk throughput on quad Opteron

2006-10-23 Thread Vivek Khera
On Oct 21, 2006, at 11:43 AM, John Philips wrote: Can you guys see any glaring bottlenecks in my layout? Any other suggestions to offer (throw in more controllers, different RAID layout, etc.)? Our budget limit is $50k. If I had $50k budget, I'd be buying the SunFire X4500 and running Sol

Re: [PERFORM] New hardware thoughts

2006-10-23 Thread Bucky Jordan
-logic raid-controller) and found it to be a very nice machine. > > But again, they also offer (the same?) Broadcom networking on board. > Just like Dell and HP. And it is a LSI Logic sas-controller on board, so > if FBSD has trouble with either of those, its hard to find anything > suitable at al

Re: [PERFORM] Index on two columns not used

2006-10-23 Thread Tom Lane
Markus Schaber <[EMAIL PROTECTED]> writes: > Alvaro Herrera wrote: >> Actually, when the UPDATE puts a new row version in the same heap page, >> the index must be updated anyway. > AFAICS only, when the index covers (directly or via function) a column > that's actually changed. > Changing columns

Re: [PERFORM] Best COPY Performance

2006-10-23 Thread Worky Workerson
> I am most interested in loading two tables, one with about 21 (small) > VARCHARs where each record is about 200 bytes, and another with 7 > INTEGERs, 3 TIMESTAMPs, and 1 BYTEA where each record is about 350 > bytes. indexes/keys? more memory for sorting during index creation can have a dramati

Re: [PERFORM] Index on two columns not used

2006-10-23 Thread Markus Schaber
Hi, Alvaro, Alvaro Herrera wrote: >> Additionally, in most UPDATE cases, the new row version will fit into >> the same page as the old version. In this case, the index does not have >> to be changed, which is an additional speed improvement. > Actually, when the UPDATE puts a new row version in t

Re: [PERFORM] Best COPY Performance

2006-10-23 Thread Luke Lonergan
Markus, On 10/23/06 2:27 AM, "Markus Schaber" <[EMAIL PROTECTED]> wrote: > On a table with no indices, triggers and contstraints, we managed to > COPY about 7-8 megabytes/second with psql over our 100 MBit network, so > here the network was the bottleneck. We routinely get 10-12MB/s on I/O hardw

Re: [PERFORM] Index on two columns not used

2006-10-23 Thread Alvaro Herrera
Markus Schaber wrote: > Additionally, in most UPDATE cases, the new row version will fit into > the same page as the old version. In this case, the index does not have > to be changed, which is an additional speed improvement. Actually, when the UPDATE puts a new row version in the same heap page

Re: [PERFORM] Index on two columns not used

2006-10-23 Thread Markus Schaber
Hi, Peter, Péter Kovács wrote: > Sorry for the amateurish question, but what are "heap tuples"? > > Also, my understanding is that the following statement applies only for > composite indexes: "PostgreSQL can't use the values stored in the index > to check the join condition". I assume that Postg

Re: [PERFORM] Optimizing disk throughput on quad Opteron

2006-10-23 Thread John Philips
> The thing I would ask is would you not be better > with SAS drives? > > Since the comments on Dell, and the highlighted > issues I have been > looking at HP and the the Smart Array P600 > controller with 512 BBWC. > Although I am looking to stick with the 8 internal > disks, rather than >

Re: [PERFORM] Best COPY Performance

2006-10-23 Thread Markus Schaber
Hi, Worky, Worky Workerson wrote: > I am currently getting > between 10K and 15K inserts/second. > I ran trivial little insert into a table with a single integer row and > came close to 250K inserts/second using psql's \copy, so I'm thinking > that my code could be optimized a bit more, but want

Re: [PERFORM] Vacuum and Memory Loss

2006-10-23 Thread Richard Huxton
Mike wrote: Hello friends, I am responsible for maintaining a high volume website using postgresql 8.1.4. Given the amount of reads and writes, I vacuum full the server a few times a week around 1, 2 AM shutting down the site for a few minutes. The next day morning around 10 - 11 AM the server s

Re: [PERFORM] New hardware thoughts

2006-10-23 Thread Arjen van der Meijden
On 20-10-2006 22:33 Ben Suffolk wrote: How about the Fujitsu Siemens Sun Clones? I have not really looked at them but have heard the odd good thing about them. Fujitsu doesn't build Sun clones! That really is insulting for them ;-) They do offer Sparc-hardware, but that's a bit higher up the m