Re: Temporary files usage in explain

2021-04-27 Thread Matteo Bonardi
Okay, I had a misconception of the buffers option. As I reread the documentation, I realized how stupid the question was. I confirm that with the buffers option I now see: Buffers: shared hit=9617011 read=1328356 dirtied=793 written=397, temp read=2996659 written=5956399 Thank you both, Matteo

Re: Temporary files usage in explain

2021-04-27 Thread David Rowley
On Tue, 27 Apr 2021 at 21:15, Matteo Bonardi wrote: > Explain plan: https://explain.depesz.com/s/BXGT > > Usually I'm looking for "external merge Disk" to see temp files usage but, in > this case, the only reference to that is 299,368kB in the last but one node > of explain. > Can anyone help me

Re: Temporary files usage in explain

2021-04-27 Thread Laurenz Albe
On Tue, 2021-04-27 at 11:15 +0200, Matteo Bonardi wrote: > I have a query that creates a large number of temporary files, in this > example ~ 45GB. > Looking at the query plan I can't figure out where temporary files are being > generated. > > Explain plan: https://ex

Temporary files usage in explain

2021-04-27 Thread Matteo Bonardi
Hi everybody, I have a query that creates a large number of temporary files, in this example ~ 45GB. Looking at the query plan I can't figure out where temporary files are being generated. Explain plan: https://explain.depesz.com/s/BXGT Usually I'm looking for "external merge Di

Re: Quere keep using temporary files

2019-10-25 Thread Tom Lane
rihad writes: > On 10/25/2019 05:49 PM, Tom Lane wrote: >> You'd need to provide a lot more detail about what that query is doing >> for anyone to be able to guess where the temp file usage is coming from. > I just checked and saw that the function "foo_xml_v2" above returns > table. Is this eno

Re: Quere keep using temporary files

2019-10-25 Thread rihad
On 10/25/2019 05:49 PM, Tom Lane wrote: rihad writes: Hi, we frequently run many query involving XML that use a smallish temporary file, despite having increased local work_mem in that transaction to 16GB. FreeBSD's top shows that the memory isn't actually being used - it remains free. Basicall

Re: Quere keep using temporary files

2019-10-25 Thread Tom Lane
rihad writes: > Hi, we frequently run many query involving XML that use a smallish > temporary file, despite having increased local work_mem in that > transaction to 16GB. FreeBSD's top shows that the memory isn't actually > being used - it remains free. Basically many such queries are run with

Re: Quere keep using temporary files

2019-10-25 Thread Luca Ferrari
On Fri, Oct 25, 2019 at 10:23 AM rihad wrote: > LOG: temporary file: path "base/pgsql_tmp/pgsql_tmp92452.1079", size 166518 > STATEMENT: DELETE FROM "foo" WHERE ((col1, col2, col3) in (select col1, > col2, col3 from foo_xml_v2(''))) AND > "foo"."col_id" IN ($1, $2, $3, $4, $5, $6, $7, $8, $9)

Quere keep using temporary files

2019-10-25 Thread rihad
Hi, we frequently run many query involving XML that use a smallish temporary file, despite having increased local work_mem in that transaction to 16GB. FreeBSD's top shows that the memory isn't actually being used - it remains free. Basically many such queries are run within a single transactio

Re: temporary files

2019-10-08 Thread dangal
thank you very much you take for your time We raised the work_mem to 130 mb and there was no more problem! Now we are seeing to improve the query, it is complicated because it is generated by a product that we have installed in production! -- Sent from: https://www.postgresql-archive.org/Postgre

Re: temporary files

2019-10-07 Thread Tomas Vondra
On Mon, Oct 07, 2019 at 01:52:41PM -0700, dangal wrote: Hello everyone, I have a question to see if you can help me, I have set work_mem in 100 MB but I have the following in the pgbadger Queries generating the most temporary files (N) Count Total size Min size Max sizeAvg size

temporary files

2019-10-07 Thread dangal
Hello everyone, I have a question to see if you can help me, I have set work_mem in 100 MB but I have the following in the pgbadger Queries generating the most temporary files (N) Count Total size Min size Max sizeAvg size    58 3.24 GiB 57.15 MiB 57.15 MiB 57.15 MiB How could

Re: PostgreSQL 9.6 Temporary files

2018-03-20 Thread Jimmy Augustine
2018-03-20 15:00 GMT+01:00 Adrian Klaver : > On 03/20/2018 03:16 AM, Jimmy Augustine wrote: > >> Thanks all for your response, >> >> $du -h $MY_DATA/base/$BASE_OID/ returns 162GB but when I execute this >> query: >> >> |SELECT stats.relname AS table, >> pg_size_pretty(pg_relation_size(statsio.rel

Re: PostgreSQL 9.6 Temporary files

2018-03-20 Thread Adrian Klaver
On 03/20/2018 03:16 AM, Jimmy Augustine wrote: Thanks all for your response, $du -h $MY_DATA/base/$BASE_OID/ returns 162GB but when I execute this query: |SELECT stats.relname AS table, pg_size_pretty(pg_relation_size(statsio.relid)) AS table_size, pg_size_pretty(pg_total_relation_size(statsi

Re: PostgreSQL 9.6 Temporary files

2018-03-20 Thread Jimmy Augustine
Thanks all for your response, $du -h $MY_DATA/base/$BASE_OID/ returns 162GB but when I execute this query: SELECT stats.relname AS table, pg_size_pretty(pg_relation_size(statsio.relid)) AS table_size, pg_size_pretty(pg_total_relation_size(statsio.relid)

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Adrian Klaver
On 03/19/2018 10:27 AM, Jimmy Augustine wrote: I tried this query and my database size is equal to 162GB. Well you can always look in $DATA directly. The database will be under $DATA/base/. You can find the like this: select oid, datname from pg_database where datname=''; -- Adrian Kla

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Adrian Klaver
On 03/19/2018 10:29 AM, Jimmy Augustine wrote: That's aggregated. Not current values. Ah did you know some documentation about that ? https://www.postgresql.org/docs/10/static/monitoring-stats.html#PG-STAT-DATABASE-VIEW Andreas -- 2ndQuadrant - The PostgreSQL Suppor

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Jimmy Augustine
> >>> > >>> > >>> Exactly how did you determine this? > >>> > >>> I used this command and sum result for all database : > >>> SELECT pg_size_pretty(pg_total_relation_size('table_name')); > >>> >

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Jimmy Augustine
> >>>> Dear Friends, >>>> >>>> I am newbie to postgresql. >>>> I have 162 GB on my database but when I check size of all >>>> tables, I approximately obtain 80 GB. >>>> I also see that I

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Adrian Klaver
On 03/19/2018 10:17 AM, Melvin Davidson wrote: * *I think your problem is that SELECT pg_size_pretty(pg_total_relation_size('table_name')); only looks at the current database * *but SELECT pg_size_pretty(pg_database_size('Database Name')); looks at ALL databases. Not according to he

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Andreas Kretschmer
Dear Friends, >>> >>> I am newbie to postgresql. >>> I have 162 GB on my database but when I check size >of all >>> tables, I approximately obtain 80 GB. >>> I also see that I have 68

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Jimmy Augustine
I have 162 GB on my database but when I check size of all >> tables, I approximately obtain 80 GB. >> I also see that I have 68GB of temporary files however >> I only >> found 2.4MB at postgres/data/bas

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Melvin Davidson
drian.kla...@aklaver.com> wrote: >> >>> On 03/19/2018 09:31 AM, Jimmy Augustine wrote: >>> >>>> Dear Friends, >>>> >>>> I am newbie to postgresql. >>>> I have 162 GB on my database but when I check size of all tables, I

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Melvin Davidson
...@aklaver.com>>: >>> >>> On 03/19/2018 09:31 AM, Jimmy Augustine wrote: >>> >>> Dear Friends, >>> >>> I am newbie to postgresql. >>> I have 162 GB on my database but when I check size of all >>>

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Jimmy Augustine
>> >>> I am newbie to postgresql. >>> I have 162 GB on my database but when I check size of all tables, I >>> approximately obtain 80 GB. >>> I also see that I have 68GB of temporary files however I only found >>> 2.4MB at postgres/data/base/pgs

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Adrian Klaver
approximately obtain 80 GB.         I also see that I have 68GB of temporary files however I only         found 2.4MB at postgres/data/base/pgsql_tmp.     Exactly how did you determine this? I used this command and sum result for all database : SELECT

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Jimmy Augustine
>> Dear Friends, >> >> I am newbie to postgresql. >> I have 162 GB on my database but when I check size of all >> tables, I approximately obtain 80 GB. >> I also see that I have 68GB of temporary files however I onl

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Adrian Klaver
n I check size of all tables, I approximately obtain 80 GB. I also see that I have 68GB of temporary files however I only found 2.4MB at postgres/data/base/pgsql_tmp. Exactly how did you determine this? I used this command and sum result for all database : SELECT pg_si

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Jimmy Augustine
2018-03-19 17:45 GMT+01:00 Adrian Klaver : > On 03/19/2018 09:31 AM, Jimmy Augustine wrote: > >> Dear Friends, >> >> I am newbie to postgresql. >> I have 162 GB on my database but when I check size of all tables, I >> approximately obtain 80 GB. >> I al

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Melvin Davidson
B. >> I also see that I have 68GB of temporary files however I only found 2.4MB >> at postgres/data/base/pgsql_tmp. >> > > Exactly how did you determine this? > > >> Could you tell me what are those temporary files and where are they at? >> Can I delete some of t

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Jimmy Augustine
> >approximately obtain 80 GB. > > > Indexes? > > Indexes are included into 80 GB that I mentioned. >I also see that I have 68GB of temporary files however I only found > > Where can you see that? > > I used pgAdmin 4 and I see statistics on my global database.

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Adrian Klaver
On 03/19/2018 09:31 AM, Jimmy Augustine wrote: Dear Friends, I am newbie to postgresql. I have 162 GB on my database but when I check size of all tables, I approximately obtain 80 GB. I also see that I have 68GB of temporary files however I only found 2.4MB at postgres/data/base/pgsql_tmp

Re: PostgreSQL 9.6 Temporary files

2018-03-19 Thread Andreas Kretschmer
On 19 March 2018 17:31:20 CET, Jimmy Augustine wrote: >Dear Friends, > >I am newbie to postgresql. >I have 162 GB on my database but when I check size of all tables, I >approximately obtain 80 GB. Indexes? >I also see that I have 68GB of temporary files however I only found

PostgreSQL 9.6 Temporary files

2018-03-19 Thread Jimmy Augustine
Dear Friends, I am newbie to postgresql. I have 162 GB on my database but when I check size of all tables, I approximately obtain 80 GB. I also see that I have 68GB of temporary files however I only found 2.4MB at postgres/data/base/pgsql_tmp. Could you tell me what are those temporary files and