Re: [PERFORM] Excessive memory used for INSERT

2015-01-06 Thread Alessandro Ipe
Hi, Doing the UPDATE on the child table (provided that the table does exist) as you recommended solved all my memory consumption issue. Thanks a lot, Alessandro. On Tuesday 23 December 2014 15:27:41 Tom Lane wrote: > Alessandro Ipe writes: > > I guess the memory consumption is depending o

Re: [PERFORM] Excessive memory used for INSERT

2014-12-23 Thread Tom Lane
Alessandro Ipe writes: > I guess the memory consumption is depending on the size of my database, so > only giving a reduced version of it won't allow to hit the issue. > The pg_dumpall file of my database can be found at the address > https://gerb.oma.be/owncloud/public.php?service=files&t=5e0e9

Re: [PERFORM] Excessive memory used for INSERT

2014-12-23 Thread Alessandro Ipe
Hi, I guess the memory consumption is depending on the size of my database, so only giving a reduced version of it won't allow to hit the issue. The pg_dumpall file of my database can be found at the address https://gerb.oma.be/owncloud/public.php?service=files&t=5e0e9e1bb06dce1d12c95662a9ee1c0

Re: [PERFORM] Excessive memory used for INSERT

2014-12-18 Thread Alessandro Ipe
Hi, I can send a full dump of my database (< 2MB) if it is OK for you. Thanks, A. On Thursday 18 December 2014 12:05:45 Tom Lane wrote: > Alessandro Ipe writes: > > Hi, > > I tried also with an upsert function > > CREATE FUNCTION upsert_func(sql_insert text, sql_update text) RETURNS void

Re: [PERFORM] Excessive memory used for INSERT

2014-12-18 Thread Tom Lane
Alessandro Ipe writes: > Hi, > I tried also with an upsert function > CREATE FUNCTION upsert_func(sql_insert text, sql_update text) RETURNS void > LANGUAGE plpgsql > AS $$ > BEGIN > EXECUTE sql_update; > IF FOUND THEN > RETURN; > END IF; > BEGIN > EXECUTE sql_insert; > EXCE

Re: [PERFORM] Excessive memory used for INSERT

2014-12-18 Thread Alessandro Ipe
Hi, I tried also with an upsert function CREATE FUNCTION upsert_func(sql_insert text, sql_update text) RETURNS void LANGUAGE plpgsql AS $$ BEGIN EXECUTE sql_update; IF FOUND THEN RETURN; END IF; BEGIN EXECUTE sql_insert; EXCEPTION WHEN OTHERS THEN EXECUTE sql_update; END;

Re: [PERFORM] Excessive memory used for INSERT

2014-12-18 Thread Alessandro Ipe
On Thursday 18 December 2014 08:51:47 Torsten Förtsch wrote: > On 17/12/14 16:14, Alessandro Ipe wrote: > > 2014-12-15 17:54:07 GMT LOG: server process (PID 21897) was terminated > > by signal 9: Killed > > since it was killed by SIGKILL, maybe it's the kernel's OOM killer? Indeed and this hopef

Re: [PERFORM] Excessive memory used for INSERT

2014-12-18 Thread Alessandro Ipe
Hi, A grep in a nightly dump of this database did not return any AFTER trigger. The only keys are primary on each daily table, through ADD CONSTRAINT "MSG_-MM-DD_pkey" PRIMARY KEY (slot, msg); and on the global table ADD CONSTRAINT msg_pkey PRIMARY KEY (slot, msg); Regards, A. On Wedne

Re: [PERFORM] Excessive memory used for INSERT

2014-12-17 Thread Torsten Förtsch
On 17/12/14 16:14, Alessandro Ipe wrote: > 2014-12-15 17:54:07 GMT LOG: server process (PID 21897) was terminated > by signal 9: Killed since it was killed by SIGKILL, maybe it's the kernel's OOM killer? Torsten -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To m

Re: [PERFORM] Excessive memory used for INSERT

2014-12-17 Thread Tom Lane
Alessandro Ipe writes: > My dtrigger definition is > CREATE TRIGGER msg_trigger BEFORE INSERT ON msg FOR EACH ROW EXECUTE > PROCEDURE msg_function(); > so it seems that it is a BEFORE trigger. Hm, no AFTER triggers anywhere? Are there foreign keys, perhaps? regards, tom

Re: [PERFORM] Excessive memory used for INSERT

2014-12-17 Thread Alessandro Ipe
Hi, My dtrigger definition is CREATE TRIGGER msg_trigger BEFORE INSERT ON msg FOR EACH ROW EXECUTE PROCEDURE msg_function(); so it seems that it is a BEFORE trigger. To be totally honest, I have "really" limited knownledge in SQL and postgresql and all these were gathered from recipes found on

Re: [PERFORM] Excessive memory used for INSERT

2014-12-17 Thread Alessandro Ipe
Hi Torsten, Thanks for your answer. I have modified (SELECT * FROM upsert) to (SELECT * FROM upsert WHERE slot=to_timestamp('201212032145', 'MMDDHH24MI') and MSG=2) according to your suggestion to reduce the result-set to a single row. However, the INSERT process is still consuming the sam

Re: [PERFORM] Excessive memory used for INSERT

2014-12-17 Thread Tom Lane
Torsten Zuehlsdorff writes: > How many rows is "(SELECT * FROM upsert)" returning? Without knowing > more i would guess, that the result-set is very big and that could be > the reason for the memory usage. Result sets are not ordinarily accumulated on the server side. Alessandro didn't show th

Re: [PERFORM] Excessive memory used for INSERT

2014-12-17 Thread Torsten Zuehlsdorff
Hello Alessandro, 2014-12-15 17:54:07 GMT DETAIL: Failed process was running: WITH upsert AS (update MSG set (slot,MSG,HRV,VIS006,VIS008,IR_016,IR_039,WV_062,WV_073,IR_087,IR_097,IR_108,IR_120,IR_134,PRO,EPI,CLM,TAPE) = (to_timestamp('201212032145', 'MMDDHH24MI'),2,'\xff','\xff','\xff','

[PERFORM] Excessive memory used for INSERT

2014-12-17 Thread Alessandro Ipe
Hi, Software and hardware running postgresql are: - postgresql92-9.2.3-1.1.1.x86_64 - openSuSE 12.3 x64_86 - 16 GB of RAM - 2 GB of swap - 8-core Intel(R) Xeon(R) CPU E5-2407 0 @ 2.20GHz - ext4 filesystem hold on a hardware Dell PERC H710 RAID10 with 4x4TB SATA HDs. - 2 GB of RAM are reserved for