Re: [PERFORM] Why is the number of dead tuples causing the performance of deferred triggers to degrading so rapidly (exponentionally)?

2004-08-17 Thread Josh Berkus
Frank,

> It seems in this case the time needed for a single deferred trigger somehow
> depends on the number of dead tuples in the table, because a vacuum of the
> table will 'reset' the query-times. However, even if I wanted to, vacuum is
> not allowed from within a function.
>
> What is happening here? And more importantly, what can I do to prevent
> this?

I'm not clear on all of the work you're doing in the trigger.  However, it 
seems obvious that you're deleting and/or updating a large number of rows.  
The escalating execution times would be consistent with that.

> NB. My real-world application 'collects' id's in need for deferred work,
> but this work is both costly and only needed once per base record. So I use
> an 'update' table whose content I join with the actual tables in order to
> do the work for _all_ the base records involved upon the first execution of
> the deferred trigger. At the end of the trigger, this 'update' table is
> emptied so any additional deferred triggers on the same table will hardly
> lose any time. Or at least, that was the intention

I think you're doing a lot more than is wise to do in triggers.   Deferrable 
triggers aren't really intended for running long procedures with the creation 
of types and temporary tables (your post got a bit garbled, so pardon me if 
I'm misreading it).   I'd suggest reconsidering your approach to this 
application problem.

At the very least, increase max_fsm_relations to some high value, which may 
help (or not). 

-Josh

-- 
__Aglio Database Solutions___
Josh BerkusConsultant
[EMAIL PROTECTED]www.agliodbs.com
Ph: 415-752-2500Fax: 415-752-2387
2166 Hayes Suite 200San Francisco, CA

---(end of broadcast)---
TIP 8: explain analyze is your friend


[PERFORM] Why is the number of dead tuples causing the performance of deferred triggers to degrading so rapidly (exponentionally)?

2004-08-17 Thread Frank van Vugt
Obviously,

this part of tr_f_def():

**
-- delete the contents
--  delete from f;
IF EXISTS (SELECT 1 FROM f) THEN
DELETE FROM F;
VACUUM F;
END IF;
**


should simply read:

**
-- delete the contents
delete from f;
**



--
Best,




Frank.


---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


[PERFORM] Why is the number of dead tuples causing the performance of deferred triggers to degrading so rapidly (exponentionally)?

2004-08-17 Thread Frank van Vugt
Hi,

I'm seeing the following behaviour with the table and functions given below:

db=# insert into f select * from full_sequence(1, 1000);
INSERT 0 1000
Time: 197,507 ms
db=# insert into f select * from full_sequence(1, 1000);
INSERT 0 1000
Time: 341,880 ms
db=# insert into f select * from full_sequence(1, 1000);
INSERT 0 1000
Time: 692,603 ms
db=# insert into f select * from full_sequence(1, 1000);
INSERT 0 1000
Time: 985,253 ms
db=# insert into f select * from full_sequence(1, 1000);
INSERT 0 1000
Time: 1241,334 ms

Or even worse (fresh drop/create of the table and functions):

db=# insert into f select id from full_sequence(1, 1);
INSERT 0 1
Time: 22255,767 ms
db=# insert into f select id from full_sequence(1, 1);
INSERT 0 1
Time: 45398,433 ms
db=# insert into f select id from full_sequence(1, 1);
INSERT 0 1
Time: 67993,476 ms

Wrapping the commands in a transaction only accumulates the penalty at commit.

It seems in this case the time needed for a single deferred trigger somehow 
depends on the number of dead tuples in the table, because a vacuum of the 
table will 'reset' the query-times. However, even if I wanted to, vacuum is 
not allowed from within a function.

What is happening here? And more importantly, what can I do to prevent this?

NB. My real-world application 'collects' id's in need for deferred work, but 
this work is both costly and only needed once per base record. So I use an 
'update' table whose content I join with the actual tables in order to do the 
work for _all_ the base records involved upon the first execution of the 
deferred trigger. At the end of the trigger, this 'update' table is emptied 
so any additional deferred triggers on the same table will hardly lose any 
time. Or at least, that was the intention

*** demo script ***
drop table f cascade;
drop function tr_f_def() cascade;
drop function full_sequence(integer, integer);
drop type full_sequence_type;

create table f (id int);
create function tr_f_def() RETURNS trigger LANGUAGE 'plpgsql' STABLE STRICT
SECURITY INVOKER AS '
DECLARE
BEGIN
-- do stuff with all the ids in the table

-- delete the contents
--  delete from f;
IF EXISTS (SELECT 1 FROM f) THEN
DELETE FROM F;
VACUUM F;
END IF;

RETURN NULL;
END;';
create type full_sequence_type as (id int);
create function full_sequence(integer, integer)
RETURNS SETOF full_sequence_type
LANGUAGE 'plpgsql'
IMMUTABLE
STRICT
SECURITY INVOKER
AS 'DECLARE
my_from ALIAS FOR $1;
my_to ALIAS FOR $2;
result full_sequence_type%ROWTYPE;
BEGIN
-- just loop
FOR i IN my_from..my_to LOOP
result.id = i;
RETURN NEXT result;
END LOOP;

-- finish
RETURN;
END;';
CREATE CONSTRAINT TRIGGER f_def AFTER INSERT ON f DEFERRABLE INITIALLY 
DEFERRED FOR EACH ROW EXECUTE PROCEDURE tr_f_def();
*** demo script ***

db=# select version();
   version
-
 PostgreSQL 7.4.3 on i686-pc-linux-gnu, compiled by GCC egcs-2.91.66





-- 
Best,




Frank.


---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faqs/FAQ.html