Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-06-17 Thread Tomas Vondra
Hi, On 05/13/15 23:07, Jeff Janes wrote: With the warning it is very hard to correlate the discrepancy you do see with which column is causing it, as the warnings don't include table or column names (Assuming of course that you run it on a substantial database--if you just run it on a few toy ca

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-05-15 Thread Josh Berkus
On 05/15/2015 12:58 PM, Robert Haas wrote: > On Fri, May 15, 2015 at 3:35 PM, Josh Berkus wrote: >> On 05/15/2015 11:30 AM, Robert Haas wrote: >>> Once we enter beta (or even feature freeze), it's too late to whack >>> around the algorithm heavily. We're pretty much committed to >>> releasing and

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-05-15 Thread Robert Haas
On Fri, May 15, 2015 at 3:35 PM, Josh Berkus wrote: > On 05/15/2015 11:30 AM, Robert Haas wrote: >> Once we enter beta (or even feature freeze), it's too late to whack >> around the algorithm heavily. We're pretty much committed to >> releasing and supporting whatever we have got at that point.

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-05-15 Thread Josh Berkus
On 05/15/2015 11:30 AM, Robert Haas wrote: > Once we enter beta (or even feature freeze), it's too late to whack > around the algorithm heavily. We're pretty much committed to > releasing and supporting whatever we have got at that point. I guess > we could revert it if it doesn't work out, but t

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-05-15 Thread Robert Haas
On Wed, May 13, 2015 at 5:07 PM, Jeff Janes wrote: > With the warning it is very hard to correlate the discrepancy you do see > with which column is causing it, as the warnings don't include table or > column names (Assuming of course that you run it on a substantial > database--if you just run it

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-05-13 Thread Jeff Janes
On Thu, Apr 30, 2015 at 6:20 PM, Tomas Vondra wrote: > Hi, > > On 04/30/15 22:57, Robert Haas wrote: > >> On Tue, Mar 31, 2015 at 3:02 PM, Tomas Vondra >> wrote: >> >>> attached is v4 of the patch implementing adaptive ndistinct estimator. >>> >> >> So, I took a look at this today. It's interest

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-05-01 Thread Robert Haas
On Thu, Apr 30, 2015 at 9:20 PM, Tomas Vondra wrote: > I agree that this is not ready for 9.5 - it was meant as an experiment > (hence printing the estimate in a WARNING, to make it easier to compare > the value to the current estimator). Without that it'd be much more > complicated to compare the

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-30 Thread Tomas Vondra
On 05/01/15 00:18, Robert Haas wrote: On Thu, Apr 30, 2015 at 5:31 PM, Heikki Linnakangas wrote: You can override the ndistinct estimate with ALTER TABLE. I think that's enough for an escape hatch. I'm not saying that isn't nice to have, but I don't think it really helps much here. Setting

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-30 Thread Tomas Vondra
Hi, On 04/30/15 22:57, Robert Haas wrote: On Tue, Mar 31, 2015 at 3:02 PM, Tomas Vondra wrote: attached is v4 of the patch implementing adaptive ndistinct estimator. So, I took a look at this today. It's interesting work, but it looks more like a research project than something we can commit

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-30 Thread Robert Haas
On Thu, Apr 30, 2015 at 5:31 PM, Heikki Linnakangas wrote: > On 04/30/2015 01:57 PM, Robert Haas wrote: >> 2. There should be a compatibility GUC to restore the old behavior. >> The new behavior should be the default, because if we're not confident >> that the new behavior will be better for most

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-30 Thread Heikki Linnakangas
On 04/30/2015 01:57 PM, Robert Haas wrote: 2. There should be a compatibility GUC to restore the old behavior. The new behavior should be the default, because if we're not confident that the new behavior will be better for most people, we have no business installing it in the first place (plus f

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-30 Thread Robert Haas
On Tue, Mar 31, 2015 at 3:02 PM, Tomas Vondra wrote: > attached is v4 of the patch implementing adaptive ndistinct estimator. So, I took a look at this today. It's interesting work, but it looks more like a research project than something we can commit to 9.5. As far as I can see, this still co

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-19 Thread Jeff Janes
On Tue, Apr 14, 2015 at 11:45 PM, Jeff Janes wrote: > On Tue, Mar 31, 2015 at 12:02 PM, Tomas Vondra < > tomas.von...@2ndquadrant.com> wrote: > >> Hi all, >> >> attached is v4 of the patch implementing adaptive ndistinct estimator. >> > > Hi Tomas, > > I have a case here where the adaptive algori

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-14 Thread Jeff Janes
On Tue, Mar 31, 2015 at 12:02 PM, Tomas Vondra wrote: > Hi all, > > attached is v4 of the patch implementing adaptive ndistinct estimator. > Hi Tomas, I have a case here where the adaptive algorithm underestimates ndistinct by a factor of 7 while the default estimator is pretty close. 5MB file

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-04 Thread Tomas Vondra
Hi, On 04/03/15 15:46, Greg Stark wrote: > The simple workaround for this was adding a fallback to GEE when f[1] or f[2] is 0. GEE is another estimator described in the paper, behaving much better in those cases. For completeness, what's the downside in just always using GEE? That's a good q

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-03 Thread Greg Stark
> The simple workaround for this was adding a fallback to GEE when f[1] or f[2] is 0. GEE is another estimator described in the paper, behaving much better in those cases. For completeness, what's the downside in just always using GEE?

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-03-31 Thread Tomas Vondra
Hi all, attached is v4 of the patch implementing adaptive ndistinct estimator. I've been looking into the strange estimates, mentioned on 2014/12/07: > values currentadaptive > -- > 106 99 107 > 1068 6449190