Re: [PERFORM] Talking about optimizer, my long dream

2011-02-07 Thread Achilleas Mantzios
Greg, 1st off, thanx for your great book, and i really hope i find the time to read it thoroughly. (since i am still stuck somewhere in the middle of "Administration Cookbook" lol!) Well, people, speaking from the point of the occasional poster and frequent lurker i can see that smth is going a

Re: [PERFORM] Indexes with condition using immutable functions applied to column not used

2011-02-07 Thread Jesper Krogh
On 2011-02-08 01:14, Sylvain Rabot wrote: CREATE INDEX directory_id_user_mod_cons_hash_0_btree_idx ON mike.directory USING btree (id_user) WHERE __mod_cons_hash(id_user, 4) = 0; CREATE INDEX directory_id_user_mod_cons_hash_1_btree_idx ON mike.directory USING btree (id_user) WHERE __mod_cons_has

Re: [PERFORM] Really really slow select count(*)

2011-02-07 Thread Scott Marlowe
On Mon, Feb 7, 2011 at 8:17 PM, felix wrote: > > On Mon, Feb 7, 2011 at 6:05 AM, Shaun Thomas wrote: >> >> That’s one of the things I talked about. To be safe, PG will start to shut >> down but disallow new connections, and *that’s all*. Old connections are >> grandfathered in until they disconne

Re: [PERFORM] Really really slow select count(*)

2011-02-07 Thread felix
On Mon, Feb 7, 2011 at 6:05 AM, Shaun Thomas wrote: > > That’s one of the things I talked about. To be safe, PG will start to shut > down but disallow new connections, and **that’s all**. Old connections are > grandfathered in until they disconnect, and when they all go away, it shuts > down grac

Re: [PERFORM] Write-heavy pg_stats_collector on mostly idle server

2011-02-07 Thread Josh Berkus
> Ooops. > It looks like you are right, see ./src/backend/postmaster/pgstat.c > > 3c2313f4 (Tom Lane 2008-11-03 01:17:08 + 2926) > if (last_statwrite < last_statrequest) > 70d75697 (Magnus Hagander2008-08-05 12:09:30 + 2927) > pgstat_write_statsfile(false); This i

[PERFORM] Indexes with condition using immutable functions applied to column not used

2011-02-07 Thread Sylvain Rabot
Hi, I am trying to understand how indexes works to get the most of them. First I would like to know if there is more advantage than overhead to split an index in several ones using conditions e.g. doing : CREATE INDEX directory_id_user_0_btree_idx ON mike.directory USING btree (id_user) WHERE i

Re: [PERFORM] Really really slow select count(*)

2011-02-07 Thread Craig Ringer
On 02/08/2011 03:05 AM, Greg Smith wrote: Accordingly I would expect any serious attempt to add some auto-reduction behavior to be beset with argument, and I'd never consider writing such a thing as a result. Too many non-controversial things I could work on instead. Yep. I expressed my own do

Re: [PERFORM] Really really slow select count(*)

2011-02-07 Thread Craig Ringer
On 02/07/2011 06:30 PM, Marti Raudsepp wrote: On Mon, Feb 7, 2011 at 05:03, Craig Ringer wrote: What would possibly help would be if Pg could fall back to lower shared_buffers automatically, screaming about it in the logs but still launching. OTOH, many people don't check the logs, so they'd th

Re: [PERFORM] Write-heavy pg_stats_collector on mostly idle server

2011-02-07 Thread Cédric Villemain
2011/2/8 Joshua D. Drake : > On Mon, 2011-02-07 at 14:58 -0800, Josh Berkus wrote: >> >> Anyone seen anything like this before? >> >> >> > >> > it is the expected behavior, IIRC >> >> OK.  It just seems kind of pathological for stats file writing to be 10X >> the volume of data writing.  I see why

Re: [PERFORM] Write-heavy pg_stats_collector on mostly idle server

2011-02-07 Thread Joshua D. Drake
On Mon, 2011-02-07 at 14:58 -0800, Josh Berkus wrote: > >> Anyone seen anything like this before? > >> > > > > it is the expected behavior, IIRC > > OK. It just seems kind of pathological for stats file writing to be 10X > the volume of data writing. I see why it's happening, but I think it's >

Re: [PERFORM] Write-heavy pg_stats_collector on mostly idle server

2011-02-07 Thread Josh Berkus
>> Anyone seen anything like this before? >> > > it is the expected behavior, IIRC OK. It just seems kind of pathological for stats file writing to be 10X the volume of data writing. I see why it's happening, but I think it's something we should fix. -- -- J

Re: [PERFORM] Really really slow select count(*)

2011-02-07 Thread Greg Smith
Craig Ringer wrote: What would possibly help would be if Pg could fall back to lower shared_buffers automatically, screaming about it in the logs but still launching. This is exactly what initdb does when it produces an initial setting for shared_buffers that goes into the postgresql.conf file

Re: [PERFORM] Really really slow select count(*)

2011-02-07 Thread Scott Marlowe
On Mon, Feb 7, 2011 at 8:05 AM, felix wrote: > +1 > this is exactly what I was looking for at the time:  a -t (configtest) > option to pg_ctl > and I think it should fall back to lower shared buffers and log it. > SHOW ALL; would show the used value however, much like apache, this might not have

Re: [PERFORM] Really really slow select count(*)

2011-02-07 Thread felix
+1 this is exactly what I was looking for at the time: a -t (configtest) option to pg_ctl and I think it should fall back to lower shared buffers and log it. SHOW ALL; would show the used value On Mon, Feb 7, 2011 at 11:30 AM, Marti Raudsepp wrote: > On Mon, Feb 7, 2011 at 05:03, Craig Rin

Re: [PERFORM] Different execution plans for semantically equivalent queries

2011-02-07 Thread Marti Raudsepp
On Mon, Feb 7, 2011 at 00:03, Mikkel Lauritsen wrote: >>> SELECT * FROM table t1 WHERE 0 = (SELECT COUNT(*) FROM table t2 WHERE >>>     t2.type = t1.type AND t2.timestamp > t1.timestamp) >> >> I suspect that *any* database is going to have trouble optimizing that. > Just out of curiosity I've bee

Re: [PERFORM] Really really slow select count(*)

2011-02-07 Thread Marti Raudsepp
On Mon, Feb 7, 2011 at 05:03, Craig Ringer wrote: > What would possibly help would be if Pg could fall back to lower > shared_buffers automatically, screaming about it in the logs but still > launching. OTOH, many people don't check the logs, so they'd think their > new setting had taken effect an

Re: [PERFORM] getting the most of out multi-core systems for repeated complex SELECT statements

2011-02-07 Thread Vitalii Tymchyshyn
Hi, all My small thoughts about parallelizing single query. AFAIK in the cases where it is needed, there is usually one single operation that takes a lot of CPU, e.g. hashing or sorting. And this are usually tasks that has well known algorithms to parallelize. The main problem, as for me, is th

Re: [PERFORM] High load,

2011-02-07 Thread Greg Smith
Michael Kohl wrote: HDD: 2x 120 GB OCZ Vertex 2 SSD; RAID 1 As a general warning here, as far as I know the regular Vertex 2 SSD doesn't cache writes properly for database use. It's possible to have a crash that leaves the database corrupted, if the drive has writes queued up in its cach