Re: [PERFORM] Query planner issue
Mark Lewis <[EMAIL PROTECTED]> writes: > You really, really want to upgrade as soon as possible, No, sooner than that. Show your boss the list of known data-loss-causing bugs in 7.2.1, and refuse to take responsibility if the database eats all your data before the "in good time" upgrade. The release note pages starting here: http://developer.postgresql.org/docs/postgres/release-7-2-8.html mention the problems we found while 7.2 was still supported. It's likely that some of the 7.3 bugs found later than 2005-05-09 also apply to 7.2. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] Where is my bottleneck?
Arnau Rebassa Villalonga wrote: The configuration of postgresql is the default, I tried to tune the postgresql.conf and the results where disappointing, so I left again the default values. That's the first thing to fix. Go to the page below and read through the "Performance Tuning" article. http://www.varlena.com/varlena/GeneralBits/Tidbits/index.php -- Richard Huxton Archonet Ltd ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PERFORM] Query planner issue
On Mon, Jan 30, 2006 at 03:26:23PM -0800, Mark Lewis wrote: > You have lots of dead rows. Do a vacuum full to get it under control, > then run VACUUM more frequently and/or increase your FSM settings to > keep dead rows in check. In 7.2 vacuum is pretty intrusive; it will be > much better behaved once you can upgrade to a more recent version. > > You really, really want to upgrade as soon as possible, and refer to the > on-line docs about what to do with your FSM settings. > Thanks! Vacuum full did it. I will now play with fsm settings to avoid running a full vacuum daily... -- Emmanuel Lacour Easter-eggs 44-46 rue de l'Ouest - 75014 Paris - France - Métro Gaité Phone: +33 (0) 1 43 35 00 37- Fax: +33 (0) 1 41 35 00 76 mailto:[EMAIL PROTECTED] -http://www.easter-eggs.com ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] Query planner issue
with Postgresql 7.2.1 you will need to do BOTH vacuum and reindex and with a table that gets many updates/deletes, you should run vacuum more than daily. Both issues have been solved in 8.1. Jim -- Original Message --- From: Emmanuel Lacour <[EMAIL PROTECTED]> To: pgsql-performance@postgresql.org Sent: Mon, 30 Jan 2006 23:57:11 +0100 Subject: [PERFORM] Query planner issue > Hi everybody, > > I have the following problem, on a test server, if I do a fresh import > of production data then run > 'explain analyze select count(*) from mandats;' > > I get this result: > > Aggregate (cost=6487.32..6487.32 rows=1 width=0) (actual time=607.61..607.61 > rows=1 loops=1) > -> Seq Scan on mandats (cost=0.00..6373.26 rows=45626 width=0) (actual > time=0.14..496.20 rows=45626 > loops=1) Total runtime: 607.95 msec > > On the production server, if I do the same (without other use of the server), > I get: > > Aggregate (cost=227554.33..227554.33 rows=1 width=0) (actual > time=230705.79..230705.79 rows=1 loops=1) > -> Seq Scan on mandats (cost=0.00..227440.26 rows=45626 width=0) (actual > time=0.03..230616.64 rows=45760 > loops=1) Total runtime: 230706.08 msec > > Is there anyone having an idea on how yo solve this poor performances? I > think it is caused by many delete/insert on this table every day, but > how to solve it, I need to run this qury each hour :(. I run > vacuum each night, postgresql is unfortunatly 7.2.1 :( (no upgrade > before 2 or 3 months). > > -- > Emmanuel Lacour Easter-eggs > 44-46 rue de l'Ouest - 75014 Paris - France - Métro Gaité > Phone: +33 (0) 1 43 35 00 37- Fax: +33 (0) 1 41 35 00 76 > mailto:[EMAIL PROTECTED] -http://www.easter-eggs.com > > ---(end of broadcast)--- > TIP 6: explain analyze is your friend --- End of Original Message --- ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PERFORM] Query planner issue
You have lots of dead rows. Do a vacuum full to get it under control, then run VACUUM more frequently and/or increase your FSM settings to keep dead rows in check. In 7.2 vacuum is pretty intrusive; it will be much better behaved once you can upgrade to a more recent version. You really, really want to upgrade as soon as possible, and refer to the on-line docs about what to do with your FSM settings. -- Mark Lewis On Mon, 2006-01-30 at 23:57 +0100, Emmanuel Lacour wrote: > Hi everybody, > > I have the following problem, on a test server, if I do a fresh import > of production data then run > 'explain analyze select count(*) from mandats;' > > I get this result: > > Aggregate (cost=6487.32..6487.32 rows=1 width=0) (actual time=607.61..607.61 > rows=1 loops=1) > -> Seq Scan on mandats (cost=0.00..6373.26 rows=45626 width=0) (actual > time=0.14..496.20 rows=45626 loops=1) > Total runtime: 607.95 msec > > > On the production server, if I do the same (without other use of the server), > I get: > > Aggregate (cost=227554.33..227554.33 rows=1 width=0) (actual > time=230705.79..230705.79 rows=1 loops=1) > -> Seq Scan on mandats (cost=0.00..227440.26 rows=45626 width=0) (actual > time=0.03..230616.64 rows=45760 loops=1) > Total runtime: 230706.08 msec > > > > Is there anyone having an idea on how yo solve this poor performances? I > think it is caused by many delete/insert on this table every day, but > how to solve it, I need to run this qury each hour :(. I run > vacuum each night, postgresql is unfortunatly 7.2.1 :( (no upgrade > before 2 or 3 months). > ---(end of broadcast)--- TIP 6: explain analyze is your friend
[PERFORM] Query planner issue
Hi everybody, I have the following problem, on a test server, if I do a fresh import of production data then run 'explain analyze select count(*) from mandats;' I get this result: Aggregate (cost=6487.32..6487.32 rows=1 width=0) (actual time=607.61..607.61 rows=1 loops=1) -> Seq Scan on mandats (cost=0.00..6373.26 rows=45626 width=0) (actual time=0.14..496.20 rows=45626 loops=1) Total runtime: 607.95 msec On the production server, if I do the same (without other use of the server), I get: Aggregate (cost=227554.33..227554.33 rows=1 width=0) (actual time=230705.79..230705.79 rows=1 loops=1) -> Seq Scan on mandats (cost=0.00..227440.26 rows=45626 width=0) (actual time=0.03..230616.64 rows=45760 loops=1) Total runtime: 230706.08 msec Is there anyone having an idea on how yo solve this poor performances? I think it is caused by many delete/insert on this table every day, but how to solve it, I need to run this qury each hour :(. I run vacuum each night, postgresql is unfortunatly 7.2.1 :( (no upgrade before 2 or 3 months). -- Emmanuel Lacour Easter-eggs 44-46 rue de l'Ouest - 75014 Paris - France - Métro Gaité Phone: +33 (0) 1 43 35 00 37- Fax: +33 (0) 1 41 35 00 76 mailto:[EMAIL PROTECTED] -http://www.easter-eggs.com ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PERFORM] Huge Data sets, simple queries
On Fri, Jan 27, 2006 at 07:05:04PM -0800, Luke Lonergan wrote: > Sounds like you are running into the limits of your disk subsystem. You are > scanning all of the data in the transactions table, so you will be limited > by the disk bandwidth you have ? and using RAID-10, you should divide the > number of disk drives by 2 and multiply by their indiividual bandwidth > (around 60MB/s) and that?s what you can expect in terms of performance. So, > if you have 8 drives, you should expect to get 4 x 60 MB/s = 240 MB/s in > bandwidth. That means that if you are dealing with 24,000 MB of data in the > ?transactions? table, then you will scan it in 100 seconds. Why divide by 2? A good raid controller should be able to send read requests to both drives out of the mirrored set to fully utilize the bandwidth. Of course, that probably won't come into play unless the OS decides that it's going to read-ahead fairly large chunks of the table at a time... Also, some vmstat output would certainly help clarify where the bottleneck is... -- Jim C. Nasby, Sr. Engineering Consultant [EMAIL PROTECTED] Pervasive Software http://pervasive.comwork: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461 ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] Where is my bottleneck?
On Tue, Jan 24, 2006 at 07:40:22PM +0100, Arnau Rebassa Villalonga wrote: > Hi all, > > I have a performance problem and I don't know where is my bottleneck. > I have postgresql 7.4.2 running on a debian server with kernel You should really upgrade to the latest 7.4 version. You're probably vulnerable to some data-loss issues. > 2.4.26-1-686-smp with two Xeon(TM) at 2.80GHz and 4GB of RAM and a RAID > 5 made with SCSI disks. Maybe its not the latest hardware but I think Generally speaking, databases (or anything else that does a lot of random writes) don't like RAID5. > My problem is that the general performance is not good enough and I > don't know where is the bottleneck. It could be because the queries are > not optimized as they should be, but I also think it can be a postgresql > configuration problem or hardware problem (HDs not beeing fast enough, > not enough RAM, ... ) What kind of performance are you expecting? What are you actually seeing? > The configuration of postgresql is the default, I tried to tune the > postgresql.conf and the results where disappointing, so I left again the > default values. Probably not so good... you'll most likely want to tune shared_buffers, sort_mem and effective_cache_size at a minimum. Granted, that might not be your current bottleneck, but it would probably be your next. -- Jim C. Nasby, Sr. Engineering Consultant [EMAIL PROTECTED] Pervasive Software http://pervasive.comwork: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461 ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PERFORM] Huge Data sets, simple queries
Depesz, On 1/30/06 9:53 AM, "hubert depesz lubaczewski" <[EMAIL PROTECTED]> wrote: >> double the performance on a reasonable number of drives. > > how many is reasonable? What I mean by that is: given a set of disks N, the read performance of RAID will be equal to the drive read rate A times the number of drives used for reading by the RAID algorithm. In the case of RAID5, that number is (N-1), so the read rate is A x (N-1). In the case of RAID10, that number is N/2, so the read rate is A x (N/2). So, the ratio of read performance RAID5/RAID10 is (N-1)/(N/2) = 2 x (N-1)/N. For numbers of drives, this ratio looks like this: N RAID5/RAID10 3 1.33 6 1.67 8 1.75 14 1.86 So - I think reasonable would be 6-8, which are common disk configurations. - Luke ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PERFORM] Huge Data sets, simple queries
On 1/29/06, Luke Lonergan <[EMAIL PROTECTED]> wrote: > Oh - and about RAID 10 - for large data work it's more often a waste of > disk performance-wise compared to RAID 5 these days. RAID5 will almost > double the performance on a reasonable number of drives. how many is reasonable? depesz ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] Where is my bottleneck?
Arnau, On 1/24/06 10:40 AM, "Arnau Rebassa Villalonga" <[EMAIL PROTECTED]> wrote: > I know it's a problem with a very big scope, but could you give me a > hint about where I should look to? Try this: time bash -c "dd if=/dev/zero of=bigfile bs=8k count=200 && sync" time dd if=bigfile of=/dev/null bs=8k And report the results back here. If it takes too long to complete (more than a couple of minutes), bring up another window and run "vmstat 1" then report the values in the columns "bi" and "bo". - Luke ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PERFORM] Where is my bottleneck?
On Tue, Jan 24, 2006 at 07:40:22PM +0100, Arnau Rebassa Villalonga wrote: I have a performance problem and I don't know where is my bottleneck. [snip] Most of the time the idle value is even higher than 60%. It's generally a fairly safe bet that if you are running slow and your cpu is idle, your i/o isn't fast enough. Mike Stone ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
