Re: [PERFORM] count(*) slow on large tables

2003-10-06 Thread Shridhar Daithankar
Bruce Momjian wrote: OK, I beefed up the TODO: * Use a fixed row count and a +/- count with MVCC visibility rules to allow fast COUNT(*) queries with no WHERE clause(?) I can always give the details if someone asks. It doesn't seem complex enough for a separate TODO.detail

Re: [PERFORM] Postgres low end processing.

2003-10-06 Thread Shridhar Daithankar
Stef wrote: On Fri, 03 Oct 2003 12:32:00 -0400 Tom Lane [EMAIL PROTECTED] wrote: = What exactly is failing? And what's the platform, anyway? Nothing is really failing atm, except the funds for better hardware. JBOSS and some other servers need to be run on these machines, along with linux,

Re: [PERFORM] Postgres low end processing.

2003-10-06 Thread Stef
Thanks for the replies, On Fri, 3 Oct 2003 11:08:48 -0700 Josh Berkus [EMAIL PROTECTED] wrote: = 1. Make sure that the WAL files (pg_xlog) are on a seperate disk from the = database files, either through mounting or symlinking. I'm not sure I understand how this helps? = 2. Tweak the .conf

Re: [PERFORM] reindex/vacuum locking/performance?

2003-10-06 Thread Jeff
On Sun, 5 Oct 2003, Neil Conway wrote: I don't know any portable way to do that :-( For the non-portable way of doing this, are you referring to O_DIRECT? Even if it isn't available everywhere, it might be worth considering this at least for the platforms on which it is supported. I

Re: [PERFORM] reindex/vacuum locking/performance?

2003-10-06 Thread Shridhar Daithankar
Jeff wrote: I'd be interested in tinkering with this, but I'm more interested at the moment of why (with proof, not antecdotal) Solaris is so much slower than Linux and what we cna do about this. We're looking to move a rather large Informix db to PG and ops has reservations about ditching Sun

Re: [PERFORM] Postgres low end processing.

2003-10-06 Thread Bruno Wolff III
On Mon, Oct 06, 2003 at 09:55:51 +0200, Stef [EMAIL PROTECTED] wrote: Thanks, I'll try some of these, and post the results. The actual machines seem to be Pentium I machines, with 32M RAM. I've gathered that it is theoretically possible, so no to go try it. I am running 7.4beta2 on a

[PERFORM] locking/performance, Solaris performance discovery

2003-10-06 Thread Jeff
On Mon, 6 Oct 2003, Andrew Sullivan wrote: There's plenty of academic work which purports to show that LRU is far from the best choice. Just in principle, it seems obvious that a single-case seqscan-type operation (such as vacuum does) is a good way to lose your cache for no real gain.

Re: [PERFORM] locking/performance, Solaris performance discovery

2003-10-06 Thread Tom Lane
Jeff [EMAIL PROTECTED] writes: I've started profiling and running tests... currently it is leaning towards the sysv semaphores. I see in src/backend/port/ that pg_sema.c is linked to the sysv implementation. Does Solaris have Posix semaphores? You could try using those instead.

[PERFORM] Shopping for hardware

2003-10-06 Thread Jason Hihn
Ok, I asked this on [novice], but I was told it's be better to post it here... I've got some money to spend on a new servers. The biggest concern is the PostgreSQL database server that will be the company. (*Everyone* uses the database server in some form or another) I'm looking for hot-swappable

Re: [PERFORM] locking/performance, Solaris performance discovery

2003-10-06 Thread Jeff
On Mon, 6 Oct 2003, Tom Lane wrote: Does Solaris have Posix semaphores? You could try using those instead. regards, tom lane Yep. It does. I learned them quick enough (using posix_sema.c as a guide) and found out that at least on Sol 2.6 they are slower than sysv -

Re: [PERFORM] Shopping for hardware

2003-10-06 Thread scott.marlowe
On Mon, 6 Oct 2003, Jason Hihn wrote: Ok, I asked this on [novice], but I was told it's be better to post it here... I've got some money to spend on a new servers. The biggest concern is the PostgreSQL database server that will be the company. (*Everyone* uses the database server in some

Re: [PERFORM] Shopping for hardware

2003-10-06 Thread Vivek Khera
JH == Jason Hihn [EMAIL PROTECTED] writes: JH The configuration that is going on in my head is: JH RAID 1, 200gig disks JH 1 server, 4g ram JH Linux 2.4 or 2.6 (depends on when we deploy and 2.6's track record at that JH time) My recommendation is to get more disks (smaller and faster) rather

Re: [PERFORM] Shopping for hardware

2003-10-06 Thread Josh Berkus
Jason, In terms of numbers, we expect have an average of 100 active connections (most of which are idle 9/10ths of the time), with about 85% reading traffic. I hope to have one server host about 1000-2000 active databases, with the largest being about 60 meg (no blobs). Inactive databases

[PERFORM] Seqscan buffer promotion (was: reindex/vacuum locking/performance?)

2003-10-06 Thread James Rogers
On Mon, 2003-10-06 at 05:15, Andrew Sullivan wrote: There's plenty of academic work which purports to show that LRU is far from the best choice. Just in principle, it seems obvious that a single-case seqscan-type operation (such as vacuum does) is a good way to lose your cache for no real

Re: [PERFORM] count(*) slow on large tables

2003-10-06 Thread Sean Chittenden
How it will help? This is in addition to trigger proposal that came up earlier. With triggers it's not possible to make values visible across backends unless trigger updates a table, which eventually leads to vacuum/dead tuples problem. 1. User creates a trigger to check updates/inserts for

[PERFORM] SOlaris updates

2003-10-06 Thread Jeff
Ran the test on another linux box - the one that generated the dump the sun loaded (which should have similar data...) and I got a profile plan similar to the Sun. Which makes me feel more comfortable. Still interesting why that other box gave me the different profile. Now off the fun and

Re: [PERFORM] Postgres low end processing.

2003-10-06 Thread Josh Berkus
Stef, = 1. Make sure that the WAL files (pg_xlog) are on a seperate disk from the = database files, either through mounting or symlinking. I'm not sure I understand how this helps? It gives you better fsync write performance on a low-end disk setup. Otherwise, the disk is forced to do

Re: [PERFORM] reindex/vacuum locking/performance?

2003-10-06 Thread Larry Rosenman
--On Monday, October 06, 2003 14:26:10 -0400 Tom Lane [EMAIL PROTECTED] wrote: Neil Conway [EMAIL PROTECTED] writes: On Sun, 2003-10-05 at 19:50, Neil Conway wrote: I was hoping you'd reply to this, Tom -- you were referring to O_DIRECT, right? Not necessarily --- as you point out, it's not

Re: [PERFORM] reindex/vacuum locking/performance?

2003-10-06 Thread Jason Hihn
Stepping out on a limb... (I'm not a disk kernel guy) I have long thought that as part of a cache descriptor, there should be a process-definable replacement-strategy (RS). Each cache entry would be associated to each process's replacement-strategy variable and the page-replacement algorithm

Re: [PERFORM] reindex/vacuum locking/performance?

2003-10-06 Thread Ronald Khoo
This would be relatively easy to fix as far as our own buffering is concerned, but the thing that's needed to make it really useful is to prevent caching of seqscan-read pages in the kernel disk buffers. I don't know any portable way to do that :-( raw disc ? :-)