Re: [HACKERS] KNN-GiST with recheck

2015-05-18 Thread Heikki Linnakangas
On 05/16/2015 12:42 AM, Jim Nasby wrote: On 5/14/15 6:30 PM, Heikki Linnakangas wrote: On 05/15/2015 02:28 AM, Heikki Linnakangas wrote: I think this is now ready for committing, but I'm pretty tired now so I'll read through this one more time in the morning, so that I won't wake up to a red

Re: [HACKERS] KNN-GiST with recheck

2015-05-15 Thread Alexander Korotkov
On Fri, May 15, 2015 at 2:30 AM, Heikki Linnakangas hlinn...@iki.fi wrote: On 05/15/2015 02:28 AM, Heikki Linnakangas wrote: I think this is now ready for committing, but I'm pretty tired now so I'll read through this one more time in the morning, so that I won't wake up to a red buildfarm.

Re: [HACKERS] KNN-GiST with recheck

2015-05-15 Thread Alexander Korotkov
On Fri, May 15, 2015 at 2:48 PM, Heikki Linnakangas hlinn...@iki.fi wrote: On 05/15/2015 11:31 AM, Alexander Korotkov wrote: On Fri, May 15, 2015 at 2:30 AM, Heikki Linnakangas hlinn...@iki.fi wrote: On 05/15/2015 02:28 AM, Heikki Linnakangas wrote: I think this is now ready for

Re: [HACKERS] KNN-GiST with recheck

2015-05-15 Thread Heikki Linnakangas
On 05/15/2015 11:31 AM, Alexander Korotkov wrote: On Fri, May 15, 2015 at 2:30 AM, Heikki Linnakangas hlinn...@iki.fi wrote: On 05/15/2015 02:28 AM, Heikki Linnakangas wrote: I think this is now ready for committing, but I'm pretty tired now so I'll read through this one more time in the

Re: [HACKERS] KNN-GiST with recheck

2015-05-15 Thread Bruce Momjian
On Fri, May 15, 2015 at 02:48:29PM +0300, Heikki Linnakangas wrote: On 05/15/2015 11:31 AM, Alexander Korotkov wrote: On Fri, May 15, 2015 at 2:30 AM, Heikki Linnakangas hlinn...@iki.fi wrote: On 05/15/2015 02:28 AM, Heikki Linnakangas wrote: I think this is now ready for committing, but

Re: [HACKERS] KNN-GiST with recheck

2015-05-15 Thread Jim Nasby
On 5/14/15 6:30 PM, Heikki Linnakangas wrote: On 05/15/2015 02:28 AM, Heikki Linnakangas wrote: I think this is now ready for committing, but I'm pretty tired now so I'll read through this one more time in the morning, so that I won't wake up to a red buildfarm. If anyone feels motivated to

Re: [HACKERS] KNN-GiST with recheck

2015-05-15 Thread Alexander Korotkov
On Fri, May 15, 2015 at 2:49 PM, Alexander Korotkov aekorot...@gmail.com wrote: On Fri, May 15, 2015 at 2:48 PM, Heikki Linnakangas hlinn...@iki.fi wrote: On 05/15/2015 11:31 AM, Alexander Korotkov wrote: On Fri, May 15, 2015 at 2:30 AM, Heikki Linnakangas hlinn...@iki.fi wrote: On

Re: [HACKERS] KNN-GiST with recheck

2015-05-14 Thread Heikki Linnakangas
On 05/15/2015 02:28 AM, Heikki Linnakangas wrote: I think this is now ready for committing, but I'm pretty tired now so I'll read through this one more time in the morning, so that I won't wake up to a red buildfarm. Forgot to attach the latest patch, here you go. - Heikki From

Re: [HACKERS] KNN-GiST with recheck

2015-05-14 Thread Heikki Linnakangas
On 05/14/2015 01:43 PM, Alexander Korotkov wrote: On Wed, May 13, 2015 at 10:17 PM, Alexander Korotkov aekorot...@gmail.com wrote: One quick comment: It would be good to avoid the extra comparisons of the distances, when the index doesn't return any lossy items. As the patch stands, it adds

Re: [HACKERS] KNN-GiST with recheck

2015-05-14 Thread Alexander Korotkov
On Wed, May 13, 2015 at 10:17 PM, Alexander Korotkov aekorot...@gmail.com wrote: One quick comment: It would be good to avoid the extra comparisons of the distances, when the index doesn't return any lossy items. As the patch stands, it adds one extra copyDistances() call and a

Re: [HACKERS] KNN-GiST with recheck

2015-05-13 Thread Alexander Korotkov
On Wed, May 13, 2015 at 10:16 PM, Heikki Linnakangas hlinn...@iki.fi wrote: On 04/17/2015 12:05 PM, Alexander Korotkov wrote: On Wed, Feb 25, 2015 at 12:15 PM, Alexander Korotkov aekorot...@gmail.com wrote: Hi! On Tue, Feb 24, 2015 at 5:39 PM, Tomas Vondra

Re: [HACKERS] KNN-GiST with recheck

2015-05-13 Thread Heikki Linnakangas
On 04/17/2015 12:05 PM, Alexander Korotkov wrote: On Wed, Feb 25, 2015 at 12:15 PM, Alexander Korotkov aekorot...@gmail.com wrote: Hi! On Tue, Feb 24, 2015 at 5:39 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: On 17.2.2015 14:21, Alexander Korotkov wrote: On Sun, Feb 15, 2015 at

Re: [HACKERS] KNN-GiST with recheck

2015-04-17 Thread Alexander Korotkov
On Wed, Feb 25, 2015 at 12:15 PM, Alexander Korotkov aekorot...@gmail.com wrote: Hi! On Tue, Feb 24, 2015 at 5:39 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: On 17.2.2015 14:21, Alexander Korotkov wrote: On Sun, Feb 15, 2015 at 2:08 PM, Alexander Korotkov aekorot...@gmail.com

Re: [HACKERS] KNN-GiST with recheck

2015-02-25 Thread Alexander Korotkov
Hi! On Tue, Feb 24, 2015 at 5:39 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: On 17.2.2015 14:21, Alexander Korotkov wrote: On Sun, Feb 15, 2015 at 2:08 PM, Alexander Korotkov aekorot...@gmail.com mailto:aekorot...@gmail.com wrote: Revised patch with reordering in GiST is

Re: [HACKERS] KNN-GiST with recheck

2015-02-24 Thread Tomas Vondra
Hi, On 17.2.2015 14:21, Alexander Korotkov wrote: On Sun, Feb 15, 2015 at 2:08 PM, Alexander Korotkov aekorot...@gmail.com mailto:aekorot...@gmail.com wrote: Revised patch with reordering in GiST is attached (knn-gist-recheck-in-gist.patch) as well as testing script (test.py). I meant to

Re: [HACKERS] KNN-GiST with recheck

2015-02-17 Thread Alexander Korotkov
On Sun, Feb 15, 2015 at 2:08 PM, Alexander Korotkov aekorot...@gmail.com wrote: Following changes has been made in attached patch: * Get sort operators from pathkeys. * Recheck argument of distance function has been reverted. Few comments were added and pairing heap comparison function

Re: [HACKERS] KNN-GiST with recheck

2015-02-15 Thread Alexander Korotkov
On Thu, Jan 8, 2015 at 1:12 AM, Alexander Korotkov aekorot...@gmail.com wrote: On Tue, Dec 16, 2014 at 4:37 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Patch attached. It should be applied on top of my pairing heap patch at

Re: [HACKERS] KNN-GiST with recheck

2015-01-07 Thread Alexander Korotkov
On Tue, Dec 16, 2014 at 4:37 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Patch attached. It should be applied on top of my pairing heap patch at http://www.postgresql.org/message-id/548ffa2c.7060...@vmware.com. Some caveats: * The signature of the distance function is unchanged,

Re: [HACKERS] KNN-GiST with recheck

2014-12-16 Thread Heikki Linnakangas
On 10/06/2014 12:36 PM, Emre Hasegeli wrote: Thanks. The main question now is design of this patch. Currently, it does all the work inside access method. We already have some discussion of pro and cons of this method. I would like to clarify alternatives now. I can see following way: 1.

Re: [HACKERS] KNN-GiST with recheck

2014-12-15 Thread Heikki Linnakangas
On 08/03/2014 04:48 PM, Emre Hasegeli wrote: 1. This patch introduces a new polygon - point operator. That seems useful on its own, with or without this patch. Yeah, but exact-knn cant come with no one implementation. But it would better come in a separate patch. I tried to split them.

Re: [HACKERS] KNN-GiST with recheck

2014-12-14 Thread Michael Paquier
On Tue, Jan 28, 2014 at 10:54 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: 1. This patch introduces a new polygon - point operator. That seems useful on its own, with or without this patch. This patch is tracked with this entry in the commit fest app and is marked as Ready for

Re: [HACKERS] KNN-GiST with recheck

2014-11-28 Thread Alexander Korotkov
On Sat, Nov 22, 2014 at 2:20 AM, Peter Geoghegan p...@heroku.com wrote: On Mon, Jan 13, 2014 at 9:17 AM, Alexander Korotkov aekorot...@gmail.com wrote: This patch was split from thread: http://www.postgresql.org/message-id/CAPpHfdscOX5an71nHd8WSUH6GNOCf=V7wgDaTXdDd9=gon-...@mail.gmail.com

Re: [HACKERS] KNN-GiST with recheck

2014-10-06 Thread Emre Hasegeli
Thanks. The main question now is design of this patch. Currently, it does all the work inside access method. We already have some discussion of pro and cons of this method. I would like to clarify alternatives now. I can see following way: 1. Implement new executor node which performs

Re: [HACKERS] KNN-GiST with recheck

2014-09-29 Thread Alexander Korotkov
On Mon, Sep 29, 2014 at 6:16 AM, Bruce Momjian br...@momjian.us wrote: On Fri, Sep 26, 2014 at 10:49:42AM +0400, Alexander Korotkov wrote: Does this also fix the identical PostGIS problem or is there something PostGIS needs to do? This patch provides general infrastructure for

Re: [HACKERS] KNN-GiST with recheck

2014-09-28 Thread Bruce Momjian
On Fri, Sep 26, 2014 at 10:49:42AM +0400, Alexander Korotkov wrote: Does this also fix the identical PostGIS problem or is there something PostGIS needs to do? This patch provides general infrastructure for recheck in KNN-GiST. PostGIS need corresponding change in its GiST opclass.

Re: [HACKERS] KNN-GiST with recheck

2014-09-26 Thread Alexander Korotkov
On Fri, Sep 26, 2014 at 5:18 AM, Bruce Momjian br...@momjian.us wrote: On Sun, Sep 14, 2014 at 11:34:26PM +0400, Alexander Korotkov wrote: Cost estimation of GiST is a big problem anyway. It doesn't care (and can't) about amount of recheck for regular operators. In this patch,

Re: [HACKERS] KNN-GiST with recheck

2014-09-25 Thread Emre Hasegeli
Fixed, thanks. Here are my questions and comments about the code. doc/src/sgml/gist.sgml:812: be rechecked from heap tuple before tuple is returned. If literalrecheck/ flag isn't set then it's true by default for compatibility reasons. The literalrecheck/ flag can be

Re: [HACKERS] KNN-GiST with recheck

2014-09-25 Thread Alexander Korotkov
On Thu, Sep 25, 2014 at 9:00 PM, Emre Hasegeli e...@hasegeli.com wrote: Fixed, thanks. Here are my questions and comments about the code. doc/src/sgml/gist.sgml:812: be rechecked from heap tuple before tuple is returned. If literalrecheck/ flag isn't set then it's true by

Re: [HACKERS] KNN-GiST with recheck

2014-09-25 Thread Bruce Momjian
On Sun, Sep 14, 2014 at 11:34:26PM +0400, Alexander Korotkov wrote: Cost estimation of GiST is a big problem anyway. It doesn't care (and can't) about amount of recheck for regular operators. In this patch, same would be for knn recheck. The problem is that touching heap from

Re: [HACKERS] KNN-GiST with recheck

2014-09-17 Thread Emre Hasegeli
While looking it at I found a bug. It returns the second column in wrong order when both of the distance functions return recheck = true. Test script attached to run on the regression database. I tried to fix but could not. searchTreeItemDistanceRecheck function is not very easy to

Re: [HACKERS] KNN-GiST with recheck

2014-09-17 Thread Emre Hasegeli
I managed to break it again by ordering rows only by the second column of the index. Test script attached. I was confused. It is undefined behavior. Sorry for the noise. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] KNN-GiST with recheck

2014-09-17 Thread Alexander Korotkov
On Wed, Sep 17, 2014 at 12:30 PM, Emre Hasegeli e...@hasegeli.com wrote: I managed to break it again by ordering rows only by the second column of the index. Test script attached. I was confused. It is undefined behavior. Sorry for the noise. No problem. Thanks a lot for testing.

Re: [HACKERS] KNN-GiST with recheck

2014-09-14 Thread Emre Hasegeli
I added the point to polygon distance operator patch to the open CommitFest as ready for committer and added myself as reviewer to both of the patches. I think that for most use cases just some operators require further sorting and some of them not. But it could appear one day that some index

Re: [HACKERS] KNN-GiST with recheck

2014-09-14 Thread Alexander Korotkov
On Sun, Sep 14, 2014 at 10:09 PM, Emre Hasegeli e...@hasegeli.com wrote: I added the point to polygon distance operator patch to the open CommitFest as ready for committer and added myself as reviewer to both of the patches. Thanks. Cost estimation of GiST is a big problem anyway. It

Re: [HACKERS] KNN-GiST with recheck

2014-08-04 Thread Alexander Korotkov
On Sun, Aug 3, 2014 at 5:48 PM, Emre Hasegeli e...@hasegeli.com wrote: 1. This patch introduces a new polygon - point operator. That seems useful on its own, with or without this patch. Yeah, but exact-knn cant come with no one implementation. But it would better come in a separate

Re: [HACKERS] KNN-GiST with recheck

2014-08-03 Thread Emre Hasegeli
1. This patch introduces a new polygon - point operator. That seems useful on its own, with or without this patch. Yeah, but exact-knn cant come with no one implementation. But it would better come in a separate patch. I tried to split them. Separated patches are attached. I changed the

Re: [HACKERS] KNN-GiST with recheck

2014-02-03 Thread Alexander Korotkov
On Tue, Jan 28, 2014 at 9:32 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 01/28/2014 04:12 PM, Alexander Korotkov wrote: On Tue, Jan 28, 2014 at 5:54 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: 4. (as you mentioned in the other thread: ) It's a modularity

Re: [HACKERS] KNN-GiST with recheck

2014-01-28 Thread Heikki Linnakangas
On 01/13/2014 07:17 PM, Alexander Korotkov wrote: Here goes a desription of this patch same as in original thread. KNN-GiST provides ability to get ordered results from index, but this order is based only on index information. For instance, GiST index contains bounding rectangles for polygons,

Re: [HACKERS] KNN-GiST with recheck

2014-01-28 Thread Alexander Korotkov
On Tue, Jan 28, 2014 at 5:54 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 01/13/2014 07:17 PM, Alexander Korotkov wrote: Here goes a desription of this patch same as in original thread. KNN-GiST provides ability to get ordered results from index, but this order is based only

Re: [HACKERS] KNN-GiST with recheck

2014-01-28 Thread Heikki Linnakangas
On 01/28/2014 04:12 PM, Alexander Korotkov wrote: On Tue, Jan 28, 2014 at 5:54 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: 4. (as you mentioned in the other thread: ) It's a modularity violation that you peek into the heap tuple from gist. I think the proper way to do this would be

[HACKERS] KNN-GiST with recheck

2014-01-13 Thread Alexander Korotkov
Hackers! This patch was split from thread: http://www.postgresql.org/message-id/CAPpHfdscOX5an71nHd8WSUH6GNOCf=V7wgDaTXdDd9=gon-...@mail.gmail.com I've split it to separate thead, because it's related to partial sort only conceptually not technically. Also I renamed it to knn-gist-recheck from