Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Tom Lane
Scott Carey writes: > On a similar note, is Postgres' Quicksort a dual-pivot quicksort? This can > be up to 2x as fast as a normal quicksort (25% fewer swap operations, and > swap operations are more expensive than compares for most sorts). In Postgres, the swaps are pretty much free compared

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Jose Ildefonso Camargo Tolosa
Hi! Thanks for the review link! Ildefonso. On Mon, Aug 30, 2010 at 6:01 PM, Greg Smith wrote: > Clemens Eisserer wrote: > > Hi, > > > > This isn't an older Opteron, its 6 core, 6MB L3 cache "Istanbul".  Its not > the newer stuff either. > > > Everything before Magny Cours is now an older Optero

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Scott Carey
On a similar note, is Postgres' Quicksort a dual-pivot quicksort? This can be up to 2x as fast as a normal quicksort (25% fewer swap operations, and swap operations are more expensive than compares for most sorts). Just google 'dual pivot quicksort' for more info. And before anyone asks --

Re: [PERFORM] slow DDL creation

2010-08-30 Thread David Fetter
On Mon, Aug 30, 2010 at 04:28:25PM -0600, Kevin Kempter wrote: > Hi all ; > > we have an automated partition creation process that includes the creation of > an FK constraint. we have a few other servers with similar scenarios and this > is the only server that stinks per when we create the new

Re: [PERFORM] slow DDL creation

2010-08-30 Thread bricklen
On Mon, Aug 30, 2010 at 3:28 PM, Kevin Kempter wrote: > Hi all ; > > we have an automated partition creation process that includes the creation of > an FK constraint. we have a few other servers with similar scenarios and this > is the only server that stinks per when we create the new partitions.

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread David Boreham
On 8/30/2010 3:18 PM, Chris Browne wrote: ... As long as you're willing to rewrite PostgreSQL in Occam 2... Just re-write it in Google's new language 'Go' : it's close enough to Occam and they'd probably fund the project.. ;) -- Sent via pgsql-performance mailing list (pgsql-performance@

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Greg Smith
Clemens Eisserer wrote: Hi, This isn't an older Opteron, its 6 core, 6MB L3 cache "Istanbul". Its not the newer stuff either. Everything before Magny Cours is now an older Opteron from my perspective. The 6-cores are identical to Magny Cours (except that Magny Cours has two o

[PERFORM] slow DDL creation

2010-08-30 Thread Kevin Kempter
Hi all ; we have an automated partition creation process that includes the creation of an FK constraint. we have a few other servers with similar scenarios and this is the only server that stinks per when we create the new partitions. Anyone have any thoughts on how to debug this? were running

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Chris Browne
david_l...@boreham.org (David Boreham) writes: > Feels like I fell through a worm hole in space/time, back to inmos in > 1987, and a guy from marketing has just > walked in the office going on about there's a customer who wants to > use our massively parallel hardware to speed up databases... ...

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Jeff Davis
On Mon, 2010-08-30 at 09:51 -0400, Eliot Gable wrote: > Not sure if anyone else saw this, but it struck me as an interesting > idea if it could be added to PostgreSQL. GPU accelerated database > operations could be very... interesting. Of course, this could be > difficult to do in a way that useful

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread David Boreham
Feels like I fell through a worm hole in space/time, back to inmos in 1987, and a guy from marketing has just walked in the office going on about there's a customer who wants to use our massively parallel hardware to speed up databases... -- Sent via pgsql-performance mailing list (pgsql-per

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Scott Marlowe
On Mon, Aug 30, 2010 at 8:56 AM, Gaël Le Mignot wrote: > Hello, > > In my humble opinion, while  it can sound interesting from a theorical > point of view to outloads some  operations to the GPU, there is a huge > pratical problem in current world  : databases which are big enough to > require suc

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Clemens Eisserer
Hi, >> This isn't an older Opteron, its 6 core, 6MB L3 cache "Istanbul".  Its not >> the newer stuff either. > > Everything before Magny Cours is now an older Opteron from my perspective. The 6-cores are identical to Magny Cours (except that Magny Cours has two of those beast in one package). -

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Yeb Havinga
Greg Smith wrote: This comes up every year or so. The ability of GPU offloading to help with sorting has to overcome the additional latency that comes from copying everything over to it and then getting all the results back. If you look at the typical types of sorting people see in PostgreSQL

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Eliot Gable
Well, from that perspective, it becomes a "chicken and egg" problem. Without the software support to use a GPU in a server for acceleration, nobody's going to build a server with a GPU. However, as previously stated, I can understand the challenges with determining whether the offloading would eve

Re: [PERFORM] Using Between

2010-08-30 Thread Ozer, Pam
Yes. ANALYZE was run after we loaded the data. Thanks for your assistance Here is the full Query. select distinct VehicleUsed.VehicleUsedId as VehicleUsedId , VehicleUsed.VehicleUsedDisplayPriority as VehicleUsedDisplayPriority , VehicleUsed.HasVehicleUsedThumbnail as HasVehicleUsedThumbnail ,

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Tom Lane
Gerhard Wiesinger writes: > BTW: I have the old data setup. /var/lib/pgsql-old. Is there a fast setup > with old version on different TCP port possible to compare query plans? You'll need to reinstall the old executables. If you put the new executables in the same directories, it's not going to

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Gerhard Wiesinger
On Mon, 30 Aug 2010, Tom Lane wrote: Gerhard Wiesinger writes: I know the drawbacks of an EAV design but I don't want to discuss that. I want to discuss the major performance decrease of PostgreSQL 8.3 (performance was ok) to PostgreSQL 8.4 (performance is NOT ok). Any further ideas how I c

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Tom Lane
Gerhard Wiesinger writes: > I know the drawbacks of an EAV design but I don't want to discuss that. I > want to discuss the major performance decrease of PostgreSQL 8.3 > (performance was ok) to PostgreSQL 8.4 (performance is NOT ok). > Any further ideas how I can track this down? > Can someone

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Gerhard Wiesinger
On Mon, 30 Aug 2010, Pavel Stehule wrote: 2010/8/30 Gerhard Wiesinger : On Mon, 30 Aug 2010, Pavel Stehule wrote: Hello 2010/8/30 Andreas Kretschmer : Gerhard Wiesinger wrote: I know that the data model is key/value pairs but it worked well in 8.3. I need this flexibility. Any ideas?

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Jose Ildefonso Camargo Tolosa
Hi! Thanks you all for this great amount of information! What memory/motherboard (ie, chipset) is installed on the phenom ii one? it looks like it peaks to ~6.2GB/s with 4 threads. Also, what kernel is on it? (uname -a would be nice). Now, this looks like sustained memory speed, what about ran

Re: [PERFORM] [Fwd: postgres 8.4.1 number of connections]

2010-08-30 Thread Maria L. Wilson
thanks for your response we have 3 app servers that attach to this one particular database server. What kind of load issues did you find on 8.4.1? I'd be interested in anything documented on it - this might make an upgrade a higher priority! thanks, Maria Scott Marlowe wrote: On Thu, Au

[PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Eliot Gable
Not sure if anyone else saw this, but it struck me as an interesting idea if it could be added to PostgreSQL. GPU accelerated database operations could be very... interesting. Of course, this could be difficult to do in a way that usefully increases performance of PostgreSQL, but I'll leave that up

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Gaël Le Mignot
Hello, In my humble opinion, while it can sound interesting from a theorical point of view to outloads some operations to the GPU, there is a huge pratical problem in current world : databases which are big enough to require such heavy optimization are usually runned on server hardware, which v

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Greg Smith
Yeb Havinga wrote: model name : AMD Phenom(tm) II X4 940 Processor @ 3.00GHz cpu cores : 4 stream compiled with -O3 Function Rate (MB/s) Avg time Min time Max time Triad: 5395.1815 0.0089 0.0089 0.0089 For comparison sake, an only moderately e

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Greg Smith
Scott Carey wrote: The 2427 should do 12.8 GB/sec theoretical peak (dual channel 800Mhz DDR2) per processor socket (so 2x that if multithreaded and 2 Sockets). A Nehalem will do ~2x that (triple channel, 1066Mhz) and is also significantly faster clock for clock. But a Core2 based Xeon on Sock

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Greg Smith
Eliot Gable wrote: Not sure if anyone else saw this, but it struck me as an interesting idea if it could be added to PostgreSQL. GPU accelerated database operations could be very... interesting. Of course, this could be difficult to do in a way that usefully increases performance of PostgreSQL, b

[PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Eliot Gable
Not sure if anyone else saw this, but it struck me as an interesting idea if it could be added to PostgreSQL. GPU accelerated database operations could be very... interesting. Of course, this could be difficult to do in a way that usefully increases performance of PostgreSQL, but I'll leave that up

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Yeb Havinga
Scott Marlowe wrote: On Mon, Aug 30, 2010 at 1:58 AM, Yeb Havinga wrote: four parallel r...@p:~/ff/www.cs.virginia.edu/stream/FTP/Code# ./a.out & ./a.out & ./a.out & ./a.out You know you can just do "stream 4" to get 4 parallel streams right? Which version is that? The stream.c sou

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Scott Marlowe
On Mon, Aug 30, 2010 at 1:58 AM, Yeb Havinga wrote: > four parallel > r...@p:~/ff/www.cs.virginia.edu/stream/FTP/Code# ./a.out & ./a.out & ./a.out > & ./a.out You know you can just do "stream 4" to get 4 parallel streams right? -- Sent via pgsql-performance mailing list (pgsql-performance@postg

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Scott Carey
On Aug 27, 2010, at 10:25 AM, Greg Smith wrote: > Scott Carey wrote: >> But the select count(*) query, cached in RAM is 3x faster in one system than >> the other. The CPUs aren't 3x different performance wise. Something else >> may be wrong here. >> >> An individual Core2 Duo 2.93Ghz should

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Pavel Stehule
2010/8/30 Gerhard Wiesinger : > On Mon, 30 Aug 2010, Pavel Stehule wrote: > >> Hello >> >> 2010/8/30 Andreas Kretschmer : >>> >>> Gerhard Wiesinger wrote: >>> I know that the data model is key/value pairs but it worked well in 8.3. I need this flexibility. Any ideas? >>> >>> If

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Gerhard Wiesinger
On Mon, 30 Aug 2010, Pavel Stehule wrote: Hello 2010/8/30 Andreas Kretschmer : Gerhard Wiesinger wrote: I know that the data model is key/value pairs but it worked well in 8.3. I need this flexibility. Any ideas? If i understand the query correctly it's a pivot-table, right? no - it's

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Gerhard Wiesinger
On Mon, 30 Aug 2010, Scott Marlowe wrote: On Mon, Aug 30, 2010 at 1:25 AM, Gerhard Wiesinger wrote: On Mon, 30 Aug 2010, Scott Marlowe wrote: On Mon, Aug 30, 2010 at 12:20 AM, Gerhard Wiesinger wrote: Hello, I just upgraded with pg_dump/restore from PostgreSQL 8.3.11 to 8.4.4 but I'm hav

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Yeb Havinga
Jose Ildefonso Camargo Tolosa wrote: Also, nowadays, Intel has better performance than AMD, at least when comparing Athlon 64 vs Core2, I'm still saving to get a Phenom II system in order to benchmark them and see how it goes (does anyone have one of these for testing?). r...@p:~/ff/www.cs.virgi

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Gerhard Wiesinger
On Mon, 30 Aug 2010, Andreas Kretschmer wrote: Gerhard Wiesinger wrote: I know that the data model is key/value pairs but it worked well in 8.3. I need this flexibility. Any ideas? If i understand the query correctly it's a pivot-table, right? The view flattens the key/value structure (

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Pavel Stehule
Hello 2010/8/30 Andreas Kretschmer : > Gerhard Wiesinger wrote: > >> I know that the data model is key/value pairs but it worked well in 8.3. >> I need this flexibility. >> >> Any ideas? > > If i understand the query correctly it's a pivot-table, right? > no - it's just EAV table on very large d

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Scott Marlowe
On Mon, Aug 30, 2010 at 1:25 AM, Gerhard Wiesinger wrote: > On Mon, 30 Aug 2010, Scott Marlowe wrote: > >> On Mon, Aug 30, 2010 at 12:20 AM, Gerhard Wiesinger >> wrote: >>> >>> Hello, >>> >>> I just upgraded with pg_dump/restore from PostgreSQL 8.3.11 to 8.4.4 but >>> I'm >>> having major perform

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Gerhard Wiesinger
On Mon, 30 Aug 2010, Scott Marlowe wrote: On Mon, Aug 30, 2010 at 12:20 AM, Gerhard Wiesinger wrote: Hello, I just upgraded with pg_dump/restore from PostgreSQL 8.3.11 to 8.4.4 but I'm having major performance problems with a query with many left joins. Problem is that costs are now very, ver

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Andreas Kretschmer
Gerhard Wiesinger wrote: > I know that the data model is key/value pairs but it worked well in 8.3. > I need this flexibility. > > Any ideas? If i understand the query correctly it's a pivot-table, right? If yes, and if i where you, i would try to rewrite this query, to something like: select

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Scott Marlowe
On Mon, Aug 30, 2010 at 12:20 AM, Gerhard Wiesinger wrote: > Hello, > > I just upgraded with pg_dump/restore from PostgreSQL 8.3.11 to 8.4.4 but I'm > having major performance problems with a query with many left joins. Problem > is that costs are now very, very, very high (was ok in 8.3). Analyze