Re: [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL community

2006-06-16 Thread Robert Lor
Arjen van der Meijden wrote: I can already confirm very good scalability (with our workload) on postgresql on that machine. We've been testing a 32thread/16G-version and it shows near-linear scaling when enabling 1, 2, 4, 6 and 8 cores (with all four threads enabled). The threads are a bit

Re: [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-06-16 Thread Tatsuo Ishii
> Tatsuo Ishii <[EMAIL PROTECTED]> writes: > > Interesting. We (some Japanese companies including SRA OSS, > > Inc. Japan) did some PG scalability testing using a Unisys's big 16 > > (physical) CPU machine and found PG scales up to 8 CPUs. However > > beyond 8 CPU PG does not scale anymore. The res

Re: [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-06-16 Thread Tom Lane
Tatsuo Ishii <[EMAIL PROTECTED]> writes: > Interesting. We (some Japanese companies including SRA OSS, > Inc. Japan) did some PG scalability testing using a Unisys's big 16 > (physical) CPU machine and found PG scales up to 8 CPUs. However > beyond 8 CPU PG does not scale anymore. The result can be

Re: [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-06-16 Thread Tatsuo Ishii
> I am thrill to inform you all that Sun has just donated a fully loaded > T2000 system to the PostgreSQL community, and it's being setup by Corey > Shields at OSL (osuosl.org) and should be online probably early next > week. The system has > > * 8 cores, 4 hw threads/core @ 1.2 GHz. Solaris se

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL community

2006-06-16 Thread Josh Berkus
Arjen, > I can already confirm very good scalability (with our workload) on > postgresql on that machine. We've been testing a 32thread/16G-version > and it shows near-linear scaling when enabling 1, 2, 4, 6 and 8 cores > (with all four threads enabled). Keen. We're trying to keep the linear sc

Re: [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL community

2006-06-16 Thread Arjen van der Meijden
On 16-6-2006 17:18, Robert Lor wrote: I think this system is well suited for PG scalability testing, among others. We did an informal test using an internal OLTP benchmark and noticed that PG can scale to around 8 CPUs. Would be really cool if all 32 virtual CPUs can be utilized!!! I can al

Re: [PERFORM] Question about clustering multiple columns

2006-06-16 Thread Bruno Wolff III
On Fri, Jun 16, 2006 at 11:11:59 -0700, Benjamin Arai <[EMAIL PROTECTED]> wrote: > Hi, > > Thanks for the reply. I have one more question. Does it matter in which > order that I make the index? Please keep replies copied to the lists so that other people can learn from and crontibute to the di

Re: [PERFORM] SAN performance mystery

2006-06-16 Thread Jeff Trout
On Jun 16, 2006, at 5:11 AM, Tim Allen wrote: One curious thing is that some postgres backends seem to spend an inordinate amount of time in uninterruptible iowait state. I found a posting to this list from December 2004 from someone who reported that very same thing. For example, bringin

Re: [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL community

2006-06-16 Thread Josh Berkus
Folks, > I am thrill to inform you all that Sun has just donated a fully loaded > T2000 system to the PostgreSQL community, and it's being setup by Corey > Shields at OSL (osuosl.org) and should be online probably early next > week. The system has So this system will be hosted by Open Source Lab

Re: [PERFORM] Question about clustering multiple columns

2006-06-16 Thread Bruno Wolff III
On Tue, Jun 13, 2006 at 09:04:15 -0700, Benjamin Arai <[EMAIL PROTECTED]> wrote: > Hi, > > I have a database where there are three columns (name,date,data). The > queries are almost always something like SELECT date,data FROM table WHERE > name=blah AND date > 1/1/2005 AND date < 1/1/2006;. I

[PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL community

2006-06-16 Thread Robert Lor
I am thrill to inform you all that Sun has just donated a fully loaded T2000 system to the PostgreSQL community, and it's being setup by Corey Shields at OSL (osuosl.org) and should be online probably early next week. The system has * 8 cores, 4 hw threads/core @ 1.2 GHz. Solaris sees the sy

Re: [PERFORM] SAN performance mystery

2006-06-16 Thread Merlin Moncure
On 6/16/06, Mikael Carneholm <[EMAIL PROTECTED]> wrote: We've seen similar results with our EMC CX200 (fully equipped) when compared to a single (1) SCSI disk machine. For sequential reads/writes (import, export, updates on 5-10 30M+ row tables), performance is downright awful. A big DB update to

Re: [PERFORM] Optimizer internals

2006-06-16 Thread Jonah H. Harris
On 16 Jun 2006 09:21:01 -0400, Greg Stark <[EMAIL PROTECTED]> wrote: Well Oracle has to do almost all that same work, it's just doing it in a separate place called a rollback segment. Well, it's not really the same work. The process by which Oracle manages UNDO is actually pretty simple and ef

Re: [PERFORM] Delete operation VERY slow...

2006-06-16 Thread Tom Lane
David Leangen <[EMAIL PROTECTED]> writes: > The only inconvenience is that I need to remove all the foreign key > constraints before truncating, then put them back after. I was about to ask if you had any. Usually the reason for DELETE being slow is that you have foreign key references to (not fr

Re: [PERFORM] Optimizer internals

2006-06-16 Thread Greg Stark
"Jonah H. Harris" <[EMAIL PROTECTED]> writes: > Now, if we're considering UPDATES (the worst case for PostgreSQL's > current MVCC architecture), then this is (IMHO) a true statement. > There aren't many *successful* commercial databases that incur the > additional overhead of creating another vers

Re: [PERFORM] Optimizer internals

2006-06-16 Thread Jonah H. Harris
On 16 Jun 2006 07:23:26 -0400, Greg Stark <[EMAIL PROTECTED]> wrote: The flip side is that Oracle and others like it have to do a lot of extra footwork to do if you query data that hasn't been committed yet. That footwork has performance implications. Not disagreeing here at all, but considerin

Re: [PERFORM] how to partition disks

2006-06-16 Thread Michael Stone
On Wed, Jun 14, 2006 at 04:32:23PM +0200, Sven Geisler wrote: For example, You run two queries with two clients and each queries needs to read some indices from disk. In this case it more efficient to read from different volumes than to read from one large volume where the disc arms has to jump

Re: [PERFORM] SAN performance mystery

2006-06-16 Thread Mikael Carneholm
We've seen similar results with our EMC CX200 (fully equipped) when compared to a single (1) SCSI disk machine. For sequential reads/writes (import, export, updates on 5-10 30M+ row tables), performance is downright awful. A big DB update took 5-6h in pre-prod (single SCSI), and 10-14?h (don't reca

Re: [PERFORM] SAN performance mystery

2006-06-16 Thread Greg Stark
"Alex Turner" <[EMAIL PROTECTED]> writes: > Given the fact that most SATA drives have only an 8MB cache, and your RAID > controller should have at least 64MB, I would argue that the system with the > RAID controller should always be faster. If it's not, you're getting > short-changed somewhere,

Re: [PERFORM] Optimizer internals

2006-06-16 Thread Greg Stark
Mark Lewis <[EMAIL PROTECTED]> writes: > On Thu, 2006-06-15 at 14:05 -0400, John Vincent wrote: > > Now I've been told by our DBA that we should have been able to wholy > > satisfy that query via the indexes. > > DB2 can satisfy the query using only indexes because DB2 doesn't do > MVCC. Well it

Re: [PERFORM] Delete operation VERY slow...

2006-06-16 Thread PFC
Wow! That was almost instantaneous. I can't believe the difference. The only inconvenience is that I need to remove all the foreign key constraints before truncating, then put them back after. But I suppose it is a small price to pay for this incredible optimization. In that case, your DELET

Re: [PERFORM] SAN performance mystery

2006-06-16 Thread Tim Allen
Tim Allen wrote: We have a customer who are having performance problems. They have a large (36G+) postgres 8.1.3 database installed on an 8-way opteron with 8G RAM, attached to an EMC SAN via fibre-channel (I don't have details of the EMC SAN model, or the type of fibre-channel card at the mome

Re: [PERFORM] Delete operation VERY slow...

2006-06-16 Thread David Leangen
Wow! That was almost instantaneous. I can't believe the difference. The only inconvenience is that I need to remove all the foreign key constraints before truncating, then put them back after. But I suppose it is a small price to pay for this incredible optimization. Thank you! On Fri, 2006-

Re: [PERFORM] SAN performance mystery

2006-06-16 Thread Stefan Kaltenbrunner
Tim Allen wrote: > We have a customer who are having performance problems. They have a > large (36G+) postgres 8.1.3 database installed on an 8-way opteron with > 8G RAM, attached to an EMC SAN via fibre-channel (I don't have details > of the EMC SAN model, or the type of fibre-channel card at the

Re: [PERFORM] Delete operation VERY slow...

2006-06-16 Thread Gourish Singbal
  David,   Truncate table would be a good idea if u want to delete all the data in the table. You need not perform vacuum in this case since there are no dead rows created.   ~gourish  On 6/16/06, David Leangen <[EMAIL PROTECTED]> wrote: Hello!I am trying to delete an entire table. There are about

Re: [PERFORM] Delete operation VERY slow...

2006-06-16 Thread A. Kretschmer
am 16.06.2006, um 15:58:46 +0900 mailte David Leangen folgendes: > > Hello! > > I am trying to delete an entire table. There are about 41,000 rows in > the table (based on count(*)). > > I am using the SQL comment: delete from table; Use TRUNCATE table. Andreas -- Andreas Kretschmer(Kon

[PERFORM] Delete operation VERY slow...

2006-06-16 Thread David Leangen
Hello! I am trying to delete an entire table. There are about 41,000 rows in the table (based on count(*)). I am using the SQL comment: delete from table; The operation seems to take in the order of hours, rather than seconds or minutes. "Explain delete from table" gives me: