Re: storing the hash multiplier instead of the hash value

2010-04-11 Thread Sean Kelly
"Robert Jacques" wrote: > On Tue, 30 Mar 2010 16:17:20 -0300, Sea Kelly > wrote: > > > I like murmurhash as well, and I'd even had it in druntime briefly, > > but > > it uses a proprietary license and I couldn't get permision from the > > author to use it. It's also an order of magnitude slower

Re: storing the hash multiplier instead of the hash value

2010-03-30 Thread Robert Jacques
On Tue, 30 Mar 2010 16:17:20 -0300, Sea Kelly wrote: I like murmurhash as well, and I'd even had it in druntime briefly, but it uses a proprietary license and I couldn't get permision from the author to use it. It's also an order of magnitude slower on sparc than x86. That seems weird. The

Re: storing the hash multiplier instead of the hash value

2010-03-30 Thread Sea Kelly
I like murmurhash as well, and I'd even had it in druntime briefly, but it uses a proprietary license and I couldn't get permision from the author to use it. It's also an order of magnitude slower on sparc than x86. Fawzi Mohamed wrote: > > On 22-mar-10, at 21:04, Andrei Alexandrescu wrote: > >

Re: storing the hash multiplier instead of the hash value

2010-03-23 Thread Fawzi Mohamed
On 23-mar-10, at 23:07, Walter Bright wrote: Fawzi Mohamed wrote: On 23-mar-10, at 19:04, Andrei Alexandrescu wrote: What I'm pushing for as of now is to move the associative array definition from opacity into templated goodies in object_.d. that would be nice, that is one the main reasons S

Re: storing the hash multiplier instead of the hash value

2010-03-23 Thread Fawzi Mohamed
On 23-mar-10, at 23:02, Walter Bright wrote: Fawzi Mohamed wrote: I think that the public interface should be exposed (or re-exposed) somewhere to the outside so that one can easily create efficient hashes for user defined types. For example it is nice to be able to chain hash functions (

Re: storing the hash multiplier instead of the hash value

2010-03-23 Thread Walter Bright
Andrei Alexandrescu wrote: Walter is afraid that that's going to mean the balkanization of D - everyone will define their own object.d. That may be a risk, but I strongly believe it's a risk worth taking. It would mean the end of precompiled libraries.

Re: storing the hash multiplier instead of the hash value

2010-03-23 Thread Walter Bright
Fawzi Mohamed wrote: On 23-mar-10, at 19:04, Andrei Alexandrescu wrote: What I'm pushing for as of now is to move the associative array definition from opacity into templated goodies in object_.d. that would be nice, that is one the main reasons Steven implementation is faster. It would be

Re: storing the hash multiplier instead of the hash value

2010-03-23 Thread Walter Bright
Fawzi Mohamed wrote: I think that the public interface should be exposed (or re-exposed) somewhere to the outside so that one can easily create efficient hashes for user defined types. For example it is nice to be able to chain hash functions (something that the default one does not allow). J

Re: storing the hash multiplier instead of the hash value

2010-03-23 Thread Andrei Alexandrescu
On 03/23/2010 02:34 PM, Fawzi Mohamed wrote: On 23-mar-10, at 19:04, Andrei Alexandrescu wrote: What I'm pushing for as of now is to move the associative array definition from opacity into templated goodies in object_.d. that would be nice, that is one the main reasons Steven implementation

Re: storing the hash multiplier instead of the hash value

2010-03-23 Thread Fawzi Mohamed
On 23-mar-10, at 19:04, Andrei Alexandrescu wrote: What I'm pushing for as of now is to move the associative array definition from opacity into templated goodies in object_.d. that would be nice, that is one the main reasons Steven implementation is faster. It would be nice if this would b

Re: storing the hash multiplier instead of the hash value

2010-03-23 Thread Andrei Alexandrescu
On 03/23/2010 12:08 PM, Fawzi Mohamed wrote: On 22-mar-10, at 21:04, Andrei Alexandrescu wrote: Better suggestions are always welcome. For integrals I'm unclear on what we could use to make things better. (Clearly we could and should get rid of the extraneous field.) I like murmurhash, that I

Re: storing the hash multiplier instead of the hash value

2010-03-23 Thread Fawzi Mohamed
On 22-mar-10, at 21:04, Andrei Alexandrescu wrote: On 03/22/2010 02:50 PM, Daniel Keep wrote: Sadly, I lack the background to make any sort of objective judgement of the hash function *itself*, so I'll just reiterate what I've heard repeated to me over and over again by numerous people: D's

Re: storing the hash multiplier instead of the hash value

2010-03-23 Thread Andrei Alexandrescu
On 03/23/2010 08:11 AM, Michael Rynn wrote: On Mon, 22 Mar 2010 16:59:36 -0500, Andrei Alexandrescu wrote: On 03/22/2010 04:03 PM, Andrei Alexandrescu wrote: On 03/22/2010 03:36 PM, Walter Bright wrote: Andrei Alexandrescu wrote: Better suggestions are always welcome. For integrals I'm uncle

Re: storing the hash multiplier instead of the hash value

2010-03-23 Thread Michael Rynn
On Mon, 22 Mar 2010 16:59:36 -0500, Andrei Alexandrescu wrote: > On 03/22/2010 04:03 PM, Andrei Alexandrescu wrote: >> On 03/22/2010 03:36 PM, Walter Bright wrote: >>> Andrei Alexandrescu wrote: Better suggestions are always welcome. For integrals I'm unclear on what we could use to make

Re: storing the hash multiplier instead of the hash value

2010-03-23 Thread Leandro Lucarella
Andrei Alexandrescu, el 22 de marzo a las 15:04 me escribiste: > On 03/22/2010 02:50 PM, Daniel Keep wrote: > > > >How about just *fixing* the hashtable so it doesn't generate false > >pointers in the first place? Maybe I'm just strange, but that seems > >like a more direct, efficacious solution..

Re: storing the hash multiplier instead of the hash value

2010-03-22 Thread Andrei Alexandrescu
On 03/22/2010 08:32 PM, Robert Jacques wrote: On Mon, 22 Mar 2010 19:00:57 -0300, Andrei Alexandrescu wrote: [snip] And the last idea before I forget (just talked to Walter about it) is that the GC could and should offer functions: GC.preCollect(void delegate()); GC.postCollect(void delegate()

Re: storing the hash multiplier instead of the hash value

2010-03-22 Thread Robert Jacques
On Mon, 22 Mar 2010 17:04:56 -0300, Andrei Alexandrescu wrote: On 03/22/2010 02:50 PM, Daniel Keep wrote: How about just *fixing* the hashtable so it doesn't generate false pointers in the first place? Maybe I'm just strange, but that seems like a more direct, efficacious solution... Thi

Re: storing the hash multiplier instead of the hash value

2010-03-22 Thread Robert Jacques
On Mon, 22 Mar 2010 19:00:57 -0300, Andrei Alexandrescu wrote: [snip] And the last idea before I forget (just talked to Walter about it) is that the GC could and should offer functions: GC.preCollect(void delegate()); GC.postCollect(void delegate()); That way we can effectively implement w

Re: storing the hash multiplier instead of the hash value

2010-03-22 Thread Lionello Lunesu
On 23-3-2010 1:51, Andrei Alexandrescu wrote: > Currently, each entry in a D hashtable stores the hash of the key for > efficiency purposes. > > There is a bit of redundancy: as soon as you entered a hash bucket you > know that the hashkey % tablesize is a specific number, call it k. But k > is no

Re: storing the hash multiplier instead of the hash value

2010-03-22 Thread Andrei Alexandrescu
On 03/22/2010 04:59 PM, Andrei Alexandrescu wrote: On 03/22/2010 04:03 PM, Andrei Alexandrescu wrote: On 03/22/2010 03:36 PM, Walter Bright wrote: Andrei Alexandrescu wrote: Better suggestions are always welcome. For integrals I'm unclear on what we could use to make things better. (Clearly we

Re: storing the hash multiplier instead of the hash value

2010-03-22 Thread Andrei Alexandrescu
On 03/22/2010 04:03 PM, Andrei Alexandrescu wrote: On 03/22/2010 03:36 PM, Walter Bright wrote: Andrei Alexandrescu wrote: Better suggestions are always welcome. For integrals I'm unclear on what we could use to make things better. (Clearly we could and should get rid of the extraneous field.)

Re: storing the hash multiplier instead of the hash value

2010-03-22 Thread Walter Bright
Andrei Alexandrescu wrote: As we discussed, if nodes are allocated in bunches, you could store 5 nodes in a 64-byte block instead of 4. That's more than a 25% increase in effectiveness because the per-block bookkeeping is also slashed by 5. Right, but the downside is that the nodes will not be

Re: storing the hash multiplier instead of the hash value

2010-03-22 Thread Andrei Alexandrescu
On 03/22/2010 03:54 PM, Walter Bright wrote: Andrei Alexandrescu wrote: Then you might be glad to hear that Walter has recently improved the default hashtable implementation significantly (at least for heavy-duty applications). All I did was switch from a tree used to resolve a bucket collisio

Re: storing the hash multiplier instead of the hash value

2010-03-22 Thread Andrei Alexandrescu
On 03/22/2010 03:36 PM, Walter Bright wrote: Andrei Alexandrescu wrote: Better suggestions are always welcome. For integrals I'm unclear on what we could use to make things better. (Clearly we could and should get rid of the extraneous field.) Unfortunately, it won't be much of a win. Memory a

Re: storing the hash multiplier instead of the hash value

2010-03-22 Thread Walter Bright
Andrei Alexandrescu wrote: Then you might be glad to hear that Walter has recently improved the default hashtable implementation significantly (at least for heavy-duty applications). All I did was switch from a tree used to resolve a bucket collision with a singly linked list. The improvement

Re: storing the hash multiplier instead of the hash value

2010-03-22 Thread Walter Bright
Andrei Alexandrescu wrote: Better suggestions are always welcome. For integrals I'm unclear on what we could use to make things better. (Clearly we could and should get rid of the extraneous field.) Unfortunately, it won't be much of a win. Memory allocation is done in buckets of size 16, 32,

Re: storing the hash multiplier instead of the hash value

2010-03-22 Thread Andrei Alexandrescu
On 03/22/2010 02:50 PM, Daniel Keep wrote: How about just *fixing* the hashtable so it doesn't generate false pointers in the first place? Maybe I'm just strange, but that seems like a more direct, efficacious solution... This was of course the first thing Walter and I discussed. It turns out

Re: storing the hash multiplier instead of the hash value

2010-03-22 Thread Daniel Keep
How about just *fixing* the hashtable so it doesn't generate false pointers in the first place? Maybe I'm just strange, but that seems like a more direct, efficacious solution... As for the redundancy, I was under the impression that the hash was cached so make resizing more efficient: if the nu

storing the hash multiplier instead of the hash value

2010-03-22 Thread Andrei Alexandrescu
Currently, each entry in a D hashtable stores the hash of the key for efficiency purposes. There is a bit of redundancy: as soon as you entered a hash bucket you know that the hashkey % tablesize is a specific number, call it k. But k is not enough information to restore the hash, so the actua