Re: Select count(*) on a 2B Rows Tables Takes ~20 Hours

2018-09-17 Thread Laurenz Albe
Fred Habash wrote: > If I'm reading this correctly, it took 57M ms out of an elapsed time of 61M > ms to read 45M pages from the filesystem? > If the average service time per sarr is < 5 ms, Is this a case of bloated > index where re-indexing is warranted? > > explain (analyze,buffers,timing,ve

Re: Select count(*) on a 2B Rows Tables Takes ~20 Hours

2018-09-17 Thread Fred Habash
Buffers: shared hit=72620045 read=45,297,330 I/O Timings: read=57,489,958.088 Execution time: 61,141,110.516 ms If I'm reading this correctly, it took 57M ms out of an elapsed time of 61M ms to read 45M pages from the filesystem? If the average service time per sarr is < 5 ms, Is this a case of bl

Re: Select count(*) on a 2B Rows Tables Takes ~20 Hours

2018-09-13 Thread Andres Freund
Hi, On 2018-09-13 14:12:02 -0400, Tom Lane wrote: > > This is an Aurora cluster running on r4.2xlarge (8 vCPU, 61g). > > Don't know much about Aurora, but I wonder whether you paid for > guaranteed (provisioned) IOPS, and if so what service level. Given that aurora uses direct-io and has the sto

RE: Select count(*) on a 2B Rows Tables Takes ~20 Hours

2018-09-13 Thread Fd Habash
Cc: pgsql-performance@lists.postgresql.org Subject: Re: Select count(*) on a 2B Rows Tables Takes ~20 Hours Fd Habash writes: > Based on my research in the forums and Google , it is described in multiple > places that ‘select count(*)’ is expected to be slow in Postgres because of > the MVCC con

Re: Select count(*) on a 2B Rows Tables Takes ~20 Hours

2018-09-13 Thread Tom Lane
Fd Habash writes: > Based on my research in the forums and Google , it is described in multiple > places that ‘select count(*)’ is expected to be slow in Postgres because of > the MVCC controls imposed upon the query leading a table scan. Also, the > elapsed time increase linearly with table si

Re: Select count(*) on a 2B Rows Tables Takes ~20 Hours

2018-09-13 Thread Justin Pryzby
On Thu, Sep 13, 2018 at 01:33:54PM -0400, Fd Habash wrote: > Is this ET expected? If not, what could be slowing it down? I’m currently > running explain analyze and I’ll share the final output when done. explain(analyze,BUFFERS) is what's probably interesting You're getting an index-only-scan,

Select count(*) on a 2B Rows Tables Takes ~20 Hours

2018-09-13 Thread Fd Habash
Based on my research in the forums and Google , it is described in multiple places that ‘select count(*)’ is expected to be slow in Postgres because of the MVCC controls imposed upon the query leading a table scan. Also, the elapsed time increase linearly with table size. However, I do not kno