Re: [PERFORM] performance of SELECT * much faster than SELECT with large offset

2014-10-02 Thread Tom Lane
Marc Slemko writes: > I ran into this oddity lately that goes against everything I thought I > understood and was wondering if anyone had any insight. SELECT * avoids a projection step ... see ExecAssignScanProjectionInfo. regards, tom lane -- Sent via pgsql-performanc

[PERFORM] performance of SELECT * much faster than SELECT with large offset

2014-10-02 Thread Marc Slemko
I ran into this oddity lately that goes against everything I thought I understood and was wondering if anyone had any insight. Version/env details at the end. The root of it is these query times: marcs=# select * from ccrimes offset 514 limit 1; [...data omitted...] (1 row) Time: 650.280 ms

Re: [PERFORM] Yet another abort-early plan disaster on 9.3

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 12:56 PM, Josh Berkus wrote: > Yes, it's only intractable if you're wedded to the idea of a tiny, > fixed-size sample. If we're allowed to sample, say, 1% of the table, we > can get a MUCH more accurate n_distinct estimate using multiple > algorithms, of which HLL is one.

Re: [PERFORM] Yet another abort-early plan disaster on 9.3

2014-10-02 Thread Josh Berkus
On 10/02/2014 02:30 AM, Peter Geoghegan wrote: > On Thu, Oct 2, 2014 at 1:19 AM, Simon Riggs wrote: >> Having read papers on it, I believe the problem is intractable. Coding >> is not the issue. To anyone: please prove me wrong, in detail, with >> references so it can be coded. > > I think it mig

Re: [PERFORM] auto vaccum is dying

2014-10-02 Thread Jeff Janes
On Wed, Oct 1, 2014 at 9:43 PM, Rodrigo Barboza wrote: > Hello, I have a table that receives lots of updates and inserts. > Auto vaccum is always being cancelled on that table. > Do you have a scheduled task that clusters or reindexes the table? Newer versions of PostgreSQL will log the conflic

Re: [PERFORM] Yet another abort-early plan disaster on 9.3

2014-10-02 Thread Ryan Johnson
On 29/09/2014 9:00 AM, Merlin Moncure wrote: On Fri, Sep 26, 2014 at 3:06 AM, Simon Riggs wrote: The problem, as I see it, is different. We assume that if there are 100 distinct values and you use LIMIT 1 that you would only need to scan 1% of rows. We assume that the data is arranged in the ta

Re: [PERFORM] Yet another abort-early plan disaster on 9.3

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 1:19 AM, Simon Riggs wrote: >> I disagree that (1) is not worth fixing just because we've provided >> users with an API to override the stats. It would unquestionably be >> better for us to have a better n_distinct estimate in the first place. >> Further, this is an easier

Re: [PERFORM] Yet another abort-early plan disaster on 9.3

2014-10-02 Thread Simon Riggs
On 1 October 2014 19:56, Josh Berkus wrote: > On 09/30/2014 04:01 PM, Simon Riggs wrote: >> On 30 September 2014 18:28, Jeff Janes wrote: >> Anyway, in the particular case I posted fixing n_distinct to realistic numbers (%) fixed the query plan. >>> >>> >>> But wouldn't fixing the absol

Re: [PERFORM] auto vaccum is dying

2014-10-02 Thread Dorian Hoxha
I think I've read that when auto-vacuum takes too long, run it more often. On Thu, Oct 2, 2014 at 8:53 AM, Heikki Linnakangas wrote: > On 10/02/2014 07:43 AM, Rodrigo Barboza wrote: > >> Hello, I have a table that receives lots of updates and inserts. >> Auto vaccum is always being cancelled on