Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-10-29 Thread Hannu Krosing
On Thu, 2009-10-29 at 09:32 -0400, Tom Lane wrote: > Hannu Krosing writes: > > Or maybe we could just extract the hashes form some version of stock > > postgresql (say 8.3) and then make those available in contrib under the > > name "stable_hashes" ? > > A better name would be "wishful_thinking"

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-10-29 Thread Peter Eisentraut
On Thu, 2009-10-29 at 09:32 -0400, Tom Lane wrote: > Hannu Krosing writes: > > Or maybe we could just extract the hashes form some version of stock > > postgresql (say 8.3) and then make those available in contrib under the > > name "stable_hashes" ? > > A better name would be "wishful_thinking"

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-10-29 Thread Tom Lane
Hannu Krosing writes: > Or maybe we could just extract the hashes form some version of stock > postgresql (say 8.3) and then make those available in contrib under the > name "stable_hashes" ? A better name would be "wishful_thinking" ... contrib does not have control over some of the main risk fa

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-10-29 Thread Hannu Krosing
On Thu, 2009-10-29 at 09:47 +0200, Peter Eisentraut wrote: > On Wed, 2009-10-28 at 12:51 -0700, Jeff Davis wrote: > > Trying to develop and document a set of standardized, stable hash > > functions covering a wide range of possible use cases sounds like it may > > be better served by an extension.

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-10-29 Thread Peter Eisentraut
On Wed, 2009-10-28 at 12:51 -0700, Jeff Davis wrote: > Trying to develop and document a set of standardized, stable hash > functions covering a wide range of possible use cases sounds like it may > be better served by an extension. I suspect that some of the participants in this thread have PL/Pro

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-10-28 Thread Hannu Krosing
On Wed, 2009-10-28 at 15:31 -0400, Tom Lane wrote: > Hannu Krosing writes: > > I had never checked the docs for hash functions, but I had assumed, that > > internal functions are prefixed by pg_ and anything else is public, free > > to use functionality. > > Sure, it's free to use. It's not free

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-10-28 Thread Hannu Krosing
On Wed, 2009-10-28 at 12:51 -0700, Jeff Davis wrote: > On Wed, 2009-10-28 at 21:09 +0200, Hannu Krosing wrote: > > Is at least the fact that they "are undocumented, have changed in the > > past, and are likely to change again in the future" documented ? > > That's a little confusing to me: how do

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-10-28 Thread Jeff Davis
On Wed, 2009-10-28 at 21:09 +0200, Hannu Krosing wrote: > Is at least the fact that they "are undocumented, have changed in the > past, and are likely to change again in the future" documented ? That's a little confusing to me: how do we document that something is undocumented? And where do we sto

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-10-28 Thread Tom Lane
Kenneth Marshall writes: > On Wed, Oct 28, 2009 at 03:31:17PM -0400, Tom Lane wrote: >> Hash indexes are so far from being production-grade that this argument >> is not significant. > In addition that change from 8.3 -> 8.4 to store only the hash and not > the value in the index means that a rein

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-10-28 Thread Kenneth Marshall
On Wed, Oct 28, 2009 at 03:31:17PM -0400, Tom Lane wrote: > Hannu Krosing writes: > > I had never checked the docs for hash functions, but I had assumed, that > > internal functions are prefixed by pg_ and anything else is public, free > > to use functionality. > > Sure, it's free to use. It's n

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-10-28 Thread Tom Lane
Hannu Krosing writes: > I had never checked the docs for hash functions, but I had assumed, that > internal functions are prefixed by pg_ and anything else is public, free > to use functionality. Sure, it's free to use. It's not free to assume that we promise never to change it. > Changing hash

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-10-28 Thread Hannu Krosing
On Wed, 2009-02-11 at 11:22 -0500, Tom Lane wrote: > Asko Oja writes: > > Did this change hashtext() visible to users? We have been using it quite > > widely for partitioning our databases. If so then it should be marked quite > > visibly in release notes as there might be others who will be hit b

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-02-11 Thread Tom Lane
Asko Oja writes: > Did this change hashtext() visible to users? We have been using it quite > widely for partitioning our databases. If so then it should be marked quite > visibly in release notes as there might be others who will be hit by this. The hash functions are undocumented, have changed

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-02-11 Thread Asko Oja
Did this change hashtext() visible to users? We have been using it quite widely for partitioning our databases. If so then it should be marked quite visibly in release notes as there might be others who will be hit by this. regards Asko On Mon, Feb 9, 2009 at 11:22 PM, Tom Lane wrote: > Kenneth

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-02-09 Thread Tom Lane
Kenneth Marshall writes: > I have updated the patch posted by Jeff Davis on January 9th > to include the micro-patch above as well as updated the polymorphism > regressions tests. This applies cleanly to the latest CVS pull. Applied --- thanks for being persistent about resolving the doubts on th

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-01-27 Thread Kenneth Marshall
On Sun, Jan 25, 2009 at 10:27:03PM -0600, Kenneth Marshall wrote: > On Sat, Jan 10, 2009 at 01:36:25PM -0500, Tom Lane wrote: > > Jeff Davis writes: > > > I ran 5 times on both old and new code, eliminating the top and bottom > > > and taking the average of the remaining 3, and I got a 6.9% perfor

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-01-25 Thread Kenneth Marshall
On Sat, Jan 10, 2009 at 01:36:25PM -0500, Tom Lane wrote: > Jeff Davis writes: > > I ran 5 times on both old and new code, eliminating the top and bottom > > and taking the average of the remaining 3, and I got a 6.9% performance > > improvement with the new code. > > The question that has been c

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-01-10 Thread Kenneth Marshall
On Sat, Jan 10, 2009 at 01:57:27PM -0500, Gregory Stark wrote: > Tom Lane writes: > > > Jeff Davis writes: > >> I ran 5 times on both old and new code, eliminating the top and bottom > >> and taking the average of the remaining 3, and I got a 6.9% performance > >> improvement with the new code.

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-01-10 Thread Kenneth Marshall
On Sat, Jan 10, 2009 at 10:56:15AM -0800, Jeff Davis wrote: > On Sat, 2009-01-10 at 13:36 -0500, Tom Lane wrote: > > Jeff Davis writes: > > > I ran 5 times on both old and new code, eliminating the top and bottom > > > and taking the average of the remaining 3, and I got a 6.9% performance > > > i

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-01-10 Thread Jeff Davis
On Sat, 2009-01-10 at 13:57 -0500, Gregory Stark wrote: > But I gather it's a 6% speedup in the time spent actually in the > hash function. That's correct. I was not able to detect any difference at all between the old and new code using HashAggregate, which is where most of my testing effort went

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-01-10 Thread Gregory Stark
Tom Lane writes: > Jeff Davis writes: >> I ran 5 times on both old and new code, eliminating the top and bottom >> and taking the average of the remaining 3, and I got a 6.9% performance >> improvement with the new code. > > The question that has been carefully evaded throughout the discussion >

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-01-10 Thread Jeff Davis
On Sat, 2009-01-10 at 13:36 -0500, Tom Lane wrote: > Jeff Davis writes: > > I ran 5 times on both old and new code, eliminating the top and bottom > > and taking the average of the remaining 3, and I got a 6.9% performance > > improvement with the new code. > > The question that has been carefull

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-01-10 Thread Tom Lane
Jeff Davis writes: > I ran 5 times on both old and new code, eliminating the top and bottom > and taking the average of the remaining 3, and I got a 6.9% performance > improvement with the new code. The question that has been carefully evaded throughout the discussion of this patch is whether the

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-01-10 Thread Jeff Davis
On Sat, 2009-01-10 at 11:06 -0600, Kenneth Marshall wrote: > > Separating mix() and final() should have some performance benefit, > > right? > > > Yes, it does but the results can be swamped by other latencies in the > code path. Tests such as Tom's benchmark of the underlying functions is > neede

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-01-10 Thread Kenneth Marshall
On Fri, Jan 09, 2009 at 02:00:39PM -0800, Jeff Davis wrote: > On Fri, 2009-01-09 at 14:29 -0600, Kenneth Marshall wrote: > > Jeff, > > > > Thanks for the review. I would not really expect any differences in hash > > index build times other than normal noise variances. The most definitive > > bench

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-01-09 Thread Jeff Davis
On Fri, 2009-01-09 at 14:29 -0600, Kenneth Marshall wrote: > Jeff, > > Thanks for the review. I would not really expect any differences in hash > index build times other than normal noise variances. The most definitive > benchmark that I have seen was done with my original patch submission > in Ma

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-01-09 Thread Kenneth Marshall
On Fri, Jan 09, 2009 at 12:04:15PM -0800, Jeff Davis wrote: > On Mon, 2008-12-22 at 13:47 -0600, Kenneth Marshall wrote: > > Dear PostgreSQL developers, > > > > I am re-sending this to keep this last change to the > > internal hash function on the radar. > > > > Hi Ken, > > A few comments: >

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2009-01-09 Thread Jeff Davis
On Mon, 2008-12-22 at 13:47 -0600, Kenneth Marshall wrote: > Dear PostgreSQL developers, > > I am re-sending this to keep this last change to the > internal hash function on the radar. > Hi Ken, A few comments: 1. New patch with very minor changes attached. 2. I reverted the change you made

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2008-12-22 Thread Alvaro Herrera
Kenneth Marshall wrote: > Dear PostgreSQL developers, > > I am re-sending this to keep this last change to the > internal hash function on the radar. Please add it to the commitfest wiki page, http://wiki.postgresql.org/wiki/CommitFest_2008-11 Thanks -- Alvaro Herrera

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2008-12-22 Thread Kenneth Marshall
Dear PostgreSQL developers, I am re-sending this to keep this last change to the internal hash function on the radar. Ken Sorry about the delay for this update to the new hash index implementation. I was trying to get the WAL logging in place and forgot to post the actual patch. The

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2008-11-04 Thread Kenneth Marshall
Oleg, Here is a little more information on the use of CRC32 as a hash function, with some warning caveats: http://home.comcast.net/~bretm/hash/8.html Regards, Ken On Tue, Nov 04, 2008 at 03:15:44PM -0600, Kenneth Marshall wrote: > On Tue, Nov 04, 2008 at 11:32:47PM +0300, Oleg Bartunov wrote: >

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2008-11-04 Thread Kenneth Marshall
On Tue, Nov 04, 2008 at 11:32:47PM +0300, Oleg Bartunov wrote: > Just interested if you repeat your tests not with cracklib-dict, > but using 8-bit words. From our experience we found many hash functions > are optimized for 7-bit words and produce too many collisions > for 8-bit words. That's why w

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2008-11-04 Thread Oleg Bartunov
Just interested if you repeat your tests not with cracklib-dict, but using 8-bit words. From our experience we found many hash functions are optimized for 7-bit words and produce too many collisions for 8-bit words. That's why we use crc32. Oleg On Tue, 4 Nov 2008, Kenneth Marshall wrote: Sorry

Re: [HACKERS] [PATCHES] updated hash functions for postgresql v1

2008-11-04 Thread Kenneth Marshall
Sorry about the delay for this update to the new hash index implementation. I was trying to get the WAL logging in place and forgot to post the actual patch. The WAL for hash indexes will need to wait for 8.5, but I did want to add back in the piece of the Bob Jenkins 2006 hash function that was st