I have installed PostgreSQL 7.3.2 on FreeBSD 4.7, running on PC with CPU Pentium II 400MHz and 384Mb RAM.
Problem is that SQL statement (see below) is running too long. With current WHERE clause 'SUBSTR(2, 2) IN ('NL', 'NM') return 25 records. With 1 record, SELECT time is about 50 minutes and takes approx. 120Mb RAM. With 25 records SELECT takes about 600Mb of memory and ends after about 10 hours with error: "Memory exhausted in AllocSetAlloc(32)".
***
How can I speed up processing? Why query (IMHO not too complex) executes so long? :(
***
Information about configuration, data structures and table sizes see below. Model picture attached.
Current postgresql.conf settings (some) are:
=== Cut === max_connections = 8
shared_buffers = 8192 max_fsm_relations = 256 max_fsm_pages = 65536 max_locks_per_transaction = 16 wal_buffers = 256
sort_mem = 131072 vacuum_mem = 16384
checkpoint_segments = 4 checkpoint_timeout = 300 commit_delay = 32000 commit_siblings = 4 fsync = false
enable_seqscan = false
effective_cache_size = 65536 === Cut ===
SELECT statement is:
SELECT showcalc('B00204', dd, r020, t071) AS s04
FROM v_file02wide
WHERE a011 = 3
AND inrepdate(data)
AND SUBSTR(ncks, 2, 2) IN ('NL', 'NM')
AND r030 = 980;Query plan is:
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Aggregate (cost=174200202474.99..174200202474.99 rows=1 width=143)
-> Hash Join (cost=174200199883.63..174200202474.89 rows=43 width=143)
Hash Cond: ("outer".id_k041 = "inner".id_k041)
-> Hash Join (cost=174200199880.57..174200202471.07 rows=43 width=139)
Hash Cond: ("outer".id_r030 = "inner".id_r030)
-> Hash Join (cost=174200199865.31..174200202410.31 rows=8992 width=135)
Hash Cond: ("outer".id_r020 = "inner".id_r020)
-> Hash Join (cost=174200199681.91..174200202069.55 rows=8992 width=124)
Hash Cond: ("outer".id_dd = "inner".id_dd)
-> Merge Join (cost=174200199676.04..174200201906.32 rows=8992 width=114)
Merge Cond: ("outer".id_v = "inner".id_v)
Join Filter: (("outer".data >= CASE WHEN ("inner".dataa IS NOT NULL) THEN "inner".dataa WHEN ("outer".data IS NOT NULL) THEN "outer".data ELSE NULL::date END) AND ("outer".data <= CASE WHEN ("inner".datab IS NOT NULL) THEN "inner".datab WHEN ("outer".data IS NOT NULL) THEN "outer".data ELSE NULL::date END))
-> Sort (cost=42528.39..42933.04 rows=161858 width=65)
Sort Key: filexxr.id_v
-> Hash Join (cost=636.25..28524.10 rows=161858 width=65)
Hash Cond: ("outer".id_obl = "inner".id_obl)
-> Hash Join (cost=632.67..25687.99 rows=161858 width=61)
Hash Cond: ("outer".id_r = "inner".id_r)
-> Index Scan using index_file02_k041 on file02 (cost=0.00..18951.63 rows=816093 width=32)
-> Hash (cost=615.41..615.41 rows=6903 width=29)
-> Index Scan using index_filexxr_a011 on filexxr (cost=0.00..615.41 rows=6903 width=29)
Index Cond: (id_a011 = 3)
Filter: inrepdate(data)
-> Hash (cost=3.47..3.47 rows=43 width=4)
-> Index Scan using kod_obl_pkey on kod_obl obl (cost=0.00..3.47 rows=43 width=4)
-> Sort (cost=174200157147.65..174200157150.57 rows=1167 width=49)
Sort Key: dov_tvbv.id_v
-> Merge Join (cost=0.00..174200157088.20 rows=1167 width=49)
Merge Cond: ("outer".id_bnk = "inner".id_bnk)
-> Index Scan using dov_bank_pkey on dov_bank (cost=0.00..290100261328.45 rows=1450 width=13)
Filter: (subplan)
SubPlan
-> Materialize (cost=100000090.02..100000090.02 rows=29 width=11)
-> Seq Scan on dov_bank (cost=100000000.00..100000090.02 rows=29 width=11)
Filter: ((substr((box)::text, 2, 2) = 'NL'::text) OR (substr((box)::text, 2, 2) = 'NM'::text))
-> Index Scan using index_dov_tvbv_bnk on dov_tvbv (cost=0.00..142.42 rows=2334 width=36)
-> Hash (cost=5.83..5.83 rows=16 width=10)
-> Index Scan using ek_pok_r_pkey on ek_pok_r epr (cost=0.00..5.83 rows=16 width=10)
-> Hash (cost=178.15..178.15 rows=2100 width=11)
-> Index Scan using kl_r020_pkey on kl_r020 (cost=0.00..178.15 rows=2100 width=11)
-> Hash (cost=15.26..15.26 rows=1 width=4)
-> Index Scan using kl_r030_pkey on kl_r030 r030 (cost=0.00..15.26 rows=1 width=4)
Filter: ((r030)::text = '980'::text)
-> Hash (cost=3.04..3.04 rows=4 width=4)
-> Index Scan using kl_k041_pkey on kl_k041 (cost=0.00..3.04 rows=4 width=4)
(45 rows)
Function showcalc definition is:
CREATE OR REPLACE FUNCTION showcalc(VARCHAR(10), VARCHAR(2), VARCHAR(4), NUMERIC(16)) RETURNS NUMERIC(16)
LANGUAGE SQL AS '
-- Parameters: code, dd, r020, t071
SELECT COALESCE(
(SELECT sc.koef * $4
FROM showing AS s NATURAL JOIN showcomp AS sc
WHERE s.kod LIKE $1
AND NOT SUBSTR(acc_mask, 1, 1) LIKE ''[''
AND SUBSTR(acc_mask, 1, 4) LIKE $3
AND SUBSTR(acc_mask, 5, 1) LIKE SUBSTR($2, 1, 1)),
(SELECT SUM(sc.koef * COALESCE(showcalc(SUBSTR(acc_mask, 2, LENGTH(acc_mask) - 2), $2, $3, $4), 0))
FROM showing AS s NATURAL JOIN showcomp AS sc
WHERE s.kod LIKE $1
AND SUBSTR(acc_mask, 1, 1) LIKE ''[''),
0) AS showing;
';
View v_file02wide is:
CREATE VIEW v_file02wide AS
SELECT id_a011 AS a011, data, obl.ko, obl.nazva AS oblast, b030, banx.box AS ncks, banx.nazva AS bank,
epr.dd, r020, r030, a3, r030.nazva AS valuta, k041,
-- Sum equivalent in national currency
t071 * get_kurs(id_r030, data) AS t070,
t071
FROM v_file02 AS vf02
JOIN kod_obl AS obl USING(id_obl)
JOIN (dov_bank NATURAL JOIN dov_tvbv) AS banx
ON banx.id_v = vf02.id_v
AND data BETWEEN COALESCE(banx.dataa, data)
AND COALESCE(banx.datab, data)
JOIN ek_pok_r AS epr USING(id_dd)
JOIN kl_r020 USING(id_r020)
JOIN kl_r030 AS r030 USING(id_r030)
JOIN kl_k041 USING(id_k041);
Function inrepdate is:
CREATE OR REPLACE FUNCTION inrepdate(DATE) RETURNS BOOL
LANGUAGE SQL AS '
-- Returns true if given date is in repdate
SELECT (SELECT COUNT(*) FROM repdate
WHERE $1 BETWEEN COALESCE(data1, CURRENT_DATE)
AND COALESCE(data2, CURRENT_DATE))
> 0;
'; Table sizes (records)
filexxr 34712
file02 816589
v_file02 816589
kod_obl 43
banx 2334
ek_pok_r 16
kl_r020 2100
kl_r030 208
kl_r041 4
v_file02wide
showing 2787
showcomp 13646
repdate 1Table has indexes almost for all selected fields.
showcalc in this query selects and uses 195 rows.
Total query size is 8066 records (COUNT(*) executes about 33 seconds and uses 120Mb RAM).
With best regards Yaroslav Mazurak.
<<inline: model.gif>>
smime.p7s
Description: S/MIME Cryptographic Signature
