Re: [PERFORM] High context switches occurring

2005-12-19 Thread Jignesh K. Shah
It basically says pg_xlog is the bottleneck and move it to the disk with the best response time that you can afford. :-) Increasing checkpoint_segments doesn't seem to help much. Playing with wal_sync_method might change the behavior. For proof .. On Solaris, the /tmp is like a RAM Drive...Of c

Re: [PERFORM] separate drives for WAL or pgdata files

2005-12-19 Thread David Lang
On Mon, 19 Dec 2005, David Lang wrote: this is getting dangerously close to being able to fit in ram. I saw an article over the weekend that Samsung is starting to produce 8G DIMM's, that can go 8 to a controller (instead of 4 per as is currently done), when motherboards come out that support

Re: [PERFORM] separate drives for WAL or pgdata files

2005-12-19 Thread David Lang
On Mon, 19 Dec 2005, Anjan Dave wrote: I am not sure if there's an obvious answer to this...If there's a choice of an external RAID10 (Fiber Channel 6 or 8 15Krpm drives) enabled drives, what is more beneficial to store on it, the WAL, or the Database files? One of the other would go on the loca

Re: [PERFORM] High context switches occurring

2005-12-19 Thread Oleg Bartunov
Hi there, I see a very low performance and high context switches on our dual itanium2 slackware box (Linux ptah 2.6.14 #1 SMP) with 8Gb of RAM, running 8.1_STABLE. Any tips here ? [EMAIL PROTECTED]:~/cvs/8.1/pgsql/contrib/pgbench$ time pgbench -s 10 -c 10 -t 3000 pgbench starting vacuum...end.

Re: [PERFORM] PostgreSQL and Ultrasparc T1

2005-12-19 Thread Jignesh K. Shah
I guess it depends on what you term as your metric for measurement. If it is just one query execution time .. It may not be the best on UltraSPARC T1. But if you have more than 8 complex queries running simultaneously, UltraSPARC T1 can do well compared comparatively provided the application ca

Re: [PERFORM] High context switches occurring

2005-12-19 Thread Juan Casero
Guys - Help me out here as I try to understand this benchmark. What is the Sun hardware and operating system we are talking about here and what is the intel hardware and operating system? What was the Sun version of PostgreSQL compiled with? Gcc on Solaris (assuming sparc) or Sun studio? W

Re: [PERFORM] High context switches occurring

2005-12-19 Thread Anjan Dave
Re-ran it 3 times on each host - Sun: -bash-3.00$ time pgbench -s 10 -c 10 -t 3000 pgbench starting vacuum...end. transaction type: TPC-B (sort of) scaling factor: 1 number of clients: 10 number of transactions per client: 3000 number of transactions actually processed: 3/3 tps = 827.810

Re: [PERFORM] PostgreSQL and Ultrasparc T1

2005-12-19 Thread Luke Lonergan
Jignesh, On 12/19/05 12:21 PM, "Jignesh Shah" <[EMAIL PROTECTED]> wrote: > extended device statistics > r/sw/s Mr/s Mw/s wait actv wsvc_t asvc_t %w %b device > 0.8 14.00.00.0 0.0 0.30.0 17.8 0 4 c3t0d0 >91.40.0 91.40.0 0.0

Re: [PERFORM] PostgreSQL and Ultrasparc T1

2005-12-19 Thread Luke Lonergan
Jignesh, On 12/19/05 12:21 PM, "Jignesh Shah" <[EMAIL PROTECTED]> wrote: > I got about 720 MB/sec to 730 MB/sec with plain dd tests on my current > storage configuration (8 LUNS on 4 fibers) which slowed me down (10K rpm 146 > GB disks FC) with 4 LUNS going through a longer pass to the disks (

Re: [PERFORM] SAN/NAS options

2005-12-19 Thread Anjan Dave
Usually manufacturer's claims are tested in 'ideal' conditions, it may not translate well on bandwidth seen on the host side. A 2Gbps Fiber Channel connection would (ideally) give you about 250MB/sec per HBA. Not sure how it translates for GigE considering scsi protocol overheads, but you may wa

Re: [PERFORM] SAN/NAS options

2005-12-19 Thread Matthew Schumacher
Jim C. Nasby wrote: > On Wed, Dec 14, 2005 at 01:56:10AM -0500, Charles Sprickman wrote: > You'll note that I'm being somewhat driven by my OS of choice, FreeBSD. > >>Unlike Solaris or other commercial offerings, there is no nice volume >>management available. While I'd love to keep managing a

Re: [PERFORM] PostgreSQL and Ultrasparc T1

2005-12-19 Thread Juan Casero
Ok. That is what I wanted to know. Right now this database is a PostgreSQL 7.4.8 system. I am using it in a sort of DSS role. I have weekly summaries of the sales for our division going back three years. I have a PHP based webapp that I wrote to give the managers access to this data. The

Re: [PERFORM] Is the optimizer choice right?

2005-12-19 Thread Greg Stark
Carlos Benkendorf <[EMAIL PROTECTED]> writes: > Hi, > > We´re running 8.03 and I´m trying to understand why the following SELECT > doesn´t use iarchave05 index. > > If you disable seqscan then iarchave05 index is used and the total runtime > is about 50% less than when iarchave05 i

Re: [PERFORM] Any way to optimize GROUP BY queries?

2005-12-19 Thread Greg Stark
"Cristian Prieto" <[EMAIL PROTECTED]> writes: > SELECT adv, pub, web, country, date_trunc('hour', tiempo), sum(num) > FROM mytmp GROUP BY adv, pub, web, country, date_trunc('hour', tiempo) > > I've tried to create index in different columns but it seems that the group > by clause doesn't use the

[PERFORM] Is the optimizer choice right?

2005-12-19 Thread Carlos Benkendorf
Hi,   We´re running 8.03 and I´m trying to understand why the following SELECT doesn´t use iarchave05 index.   If you disable seqscan then iarchave05 index is used and the total runtime is about 50% less than when iarchave05 index is not used.   Why is the optimizer not using iarchave05 index

Re: [PERFORM] PostgreSQL and Ultrasparc T1

2005-12-19 Thread Jignesh Shah
Hi Luke, I got about 720 MB/sec to 730 MB/sec with plain dd tests on my current storage configuration (8 LUNS on 4 fibers) which slowed me down (10K rpm 146 GB disks FC) with 4 LUNS going through a longer pass to the disks (via a controller master array to slave JBODs to provide ) .

[PERFORM] separate drives for WAL or pgdata files

2005-12-19 Thread Anjan Dave
Hi,   I am not sure if there’s an obvious answer to this…If there’s a choice of an external RAID10 (Fiber Channel 6 or 8 15Krpm drives) enabled drives, what is more beneficial to store on it, the WAL, or the Database files? One of the other would go on the local RAID10 (4 drives, 15Krpm)

Re: [PERFORM] PostgreSQL and Ultrasparc T1

2005-12-19 Thread Luke Lonergan
Jignesh, On 12/19/05 11:29 AM, "Jignesh Shah" <[EMAIL PROTECTED]> wrote: > I have gone to the max with 4 fibers on Sun Fire T2000. But I am not sure > about the answers that you asked. Let me see if I can get answers for them. I > am going to try to max out the IO on these systems with 8 fibers a

Re: [PERFORM] PostgreSQL and Ultrasparc T1

2005-12-19 Thread Jignesh Shah
Hi Luke, I have gone to the max with 4 fibers on Sun Fire T2000. But I am not sure about the answers that you asked. Let me see if I can get answers for them. I am going to try to max out the IO on these systems with 8 fibers as soon as I get additional storage so stay tuned for that. By the

Re: [PERFORM] make bulk deletes faster?

2005-12-19 Thread James Klo
Mitch Skinner wrote: Have you considered partitioning? http://www.postgresql.org/docs/8.1/interactive/ddl-partitioning.html If you can partition your timeblock table so that you archive an entire partition at a time, then you can delete the archived rows by just dropping (or truncating) that p

[PERFORM] Any way to optimize GROUP BY queries?

2005-12-19 Thread Cristian Prieto
I have the following table:   CREATE TABLE mytmp (     Adv integer,     Pub integer,     Web integer,     Tiempo timestamp,     Num integer,     Country varchar(2) );   CREATE INDEX idx_mytmp ON mytmp(adv, pub, web);   And with 16

Re: [PERFORM] PostgreSQL and Ultrasparc T1

2005-12-19 Thread Luke Lonergan
Jignesh, On 12/19/05 6:27 AM, "Jignesh K. Shah" <[EMAIL PROTECTED]> wrote: > Sun Fire T2000 has 3 PCI-E and 1PCI-X slot free when shipped. Using > dual fiber channel 2G adapters you can get about 200MB x 8 = 1600MB/sec > IO bandwidth. Plus when 4G HBAs are supported that will double up. Now I >

Re: [PERFORM] PostgreSQL and Ultrasparc T1

2005-12-19 Thread Anjan Dave
8 HBAs at 200MB/sec would require a pretty significant Storage Processor backend unless cost is not a factor. Once you achieve that, there's a question of sharing/balancing I/O requirements of various other applications/databases on that same shared backend storage... Anjan -Original Message

Re: [PERFORM] make bulk deletes faster?

2005-12-19 Thread Ang Chin Han
On 12/18/05, James Klo <[EMAIL PROTECTED]> wrote: > explain analyze delete from timeblock where timeblockid = 66 > > Index Scan using timeblockid_idx on timeblock (cost=0.00..5.28 rows=1 > width=6) (actual time=0.022..0.022 rows=0 loops=1) > Index Cond: (timeblockid = 66) > Total runtime

Re: [PERFORM] PostgreSQL and Ultrasparc T1

2005-12-19 Thread Jignesh K. Shah
Sun Fire T2000 has 3 PCI-E and 1PCI-X slot free when shipped. Using dual fiber channel 2G adapters you can get about 200MB x 8 = 1600MB/sec IO bandwidth. Plus when 4G HBAs are supported that will double up. Now I think generally that's good enough for 1TB raw data or 2-3 TB Database size. Of c

Re: [PERFORM] make bulk deletes faster?

2005-12-19 Thread Mitch Skinner
On Sat, 2005-12-17 at 21:10 -0800, James Klo wrote: > I need to routinely move data from the timeblock table to an archive > table with the same schema named timeblock_archive. I really need this > to happen as quickly as possible, as the archive operation appears to > really tax the db server.

Re: [PERFORM] make bulk deletes faster?

2005-12-19 Thread James Klo
In article <[EMAIL PROTECTED]>, [EMAIL PROTECTED] (Michael Fuhr) wrote: > On Sat, Dec 17, 2005 at 09:10:40PM -0800, James Klo wrote: > > I'd like some suggestions on how to get the deletes to happen faster, as > > while deleting individually appears to extremely fast, when I go to > > delete lo