Re: New hash

2012-03-23 Thread Andrej Mitrovic
I thought I'd open this topic for discussion of issues with the new hash implementation. Anyways, this doesn't seem to work: AA!(string,int[]) hash; D:\dev\projects\New-AA-implementation>rdmd -ID:\DMD\dmd2\src\druntime\src newAATest.d newAA.d(581): Error: template newAA.A

Re: New hash

2012-03-23 Thread H. S. Teoh
On Sat, Mar 24, 2012 at 02:39:35AM +0100, Andrej Mitrovic wrote: > I thought I'd open this topic for discussion of issues with the new > hash implementation. Thanks for taking the time to test the code! > Anyways, this doesn't seem to work: > > AA!(string,int[]) hash;

Re: New hash

2012-03-23 Thread H. S. Teoh
On Sat, Mar 24, 2012 at 02:39:35AM +0100, Andrej Mitrovic wrote: > I thought I'd open this topic for discussion of issues with the new > hash implementation. > > Anyways, this doesn't seem to work: > > AA!(string,int[]) hash; [...] Argh... in the process of trying t

Re: New hash

2012-03-23 Thread Andrej Mitrovic
On 3/24/12, H. S. Teoh wrote: > Argh. Looks like it's caused by more const madness. :-( Try this diff: Ahaha: patching file newAA.d Assertion failed: hunk, file ../patch-2.5.9-src/patch.c, line 354 This application has requested the Runtime to terminate it in an unusual way. Please contact the ap

Re: New hash

2012-03-23 Thread H. S. Teoh
On Sat, Mar 24, 2012 at 03:11:32AM +0100, Andrej Mitrovic wrote: > On 3/24/12, H. S. Teoh wrote: > > Argh. Looks like it's caused by more const madness. :-( Try this diff: > > Ahaha: > patching file newAA.d > Assertion failed: hunk, file ../patch-2.5.9-src/patch.c, line 354 > This application has

Re: New hash

2012-03-23 Thread Andrej Mitrovic
On 3/24/12, Andrej Mitrovic wrote: > Anyways, this doesn't seem to work: > > AA!(string,int[]) hash; Ok it works now w/ your patch.

Re: New hash

2012-03-23 Thread H. S. Teoh
On Sat, Mar 24, 2012 at 03:15:49AM +0100, Andrej Mitrovic wrote: > On 3/24/12, Andrej Mitrovic wrote: > > Anyways, this doesn't seem to work: > > > > AA!(string,int[]) hash; > > Ok it works now w/ your patch. You probably want to git pull, 'cos I found 2-3 other places where I forgot about inout

Re: New hash

2012-03-23 Thread Andrej Mitrovic
On 3/24/12, Andrej Mitrovic wrote: > Anyways, this doesn't seem to work: > > AA!(string,int[]) hash; Ok it works now w/ your patch.

Re: New hash

2012-03-23 Thread H. S. Teoh
On Sat, Mar 24, 2012 at 02:39:35AM +0100, Andrej Mitrovic wrote: > I thought I'd open this topic for discussion of issues with the new > hash implementation. [...] Another issue: AA!(string,int[]) aa; auto x = aa.get("abc", []); This fails to compile,

Re: New hash

2012-03-23 Thread Andrej Mitrovic
On 3/24/12, H. S. Teoh wrote: > On Sat, Mar 24, 2012 at 02:39:35AM +0100, Andrej Mitrovic wrote: >> I thought I'd open this topic for discussion of issues with the new >> hash implementation. > [...] > > Another issue: > > AA!(string,int[]) aa; >

Re: New hash

2012-03-23 Thread H. S. Teoh
On Sat, Mar 24, 2012 at 04:21:21AM +0100, Andrej Mitrovic wrote: > On 3/24/12, H. S. Teoh wrote: [...] > > Another issue: > > > > AA!(string,int[]) aa; > > auto x = aa.get("abc", []); > > Yeah, templates are unforgiving sometimes. Well, I wish that compiler magic that does these inferenc

Re: New hash

2012-03-23 Thread Andrej Mitrovic
On 3/24/12, H. S. Teoh wrote: > Fix pushed to github. :-) Okay. Then I have some eye-candy for you: template Assoc(T) if (std.traits.isAssociativeArray!T) { alias AA!(GetTypesTuple!T) Assoc; } import std.typetuple; template GetTypesTuple(T) if (std.traits.isAssociativeArray!T) {

Re: New hash

2012-03-23 Thread Andrej Mitrovic
On 3/24/12, Andrej Mitrovic wrote: > Anyways, this doesn't seem to work. Another bug: struct Foo { string s; } AA!(string, Foo) test; newAA.d(378): Error: template newAA.toHash(T) if (is(T == char) || is(T == const(char)) || is(T == immutable(char))) toHash(T) if (is(T == char) || is(T == c

Re: New hash

2012-03-23 Thread H. S. Teoh
On Sat, Mar 24, 2012 at 05:12:00AM +0100, Andrej Mitrovic wrote: > On 3/24/12, H. S. Teoh wrote: > > Fix pushed to github. :-) > > Okay. Then I have some eye-candy for you: I have a much simpler version: template Assoc(T) { static if (is(T K : V[K], V))

Re: New hash

2012-03-23 Thread Andrej Mitrovic
On 3/24/12, Andrej Mitrovic wrote: > snip I've managed to test the hashes on a small-sized closed-source project (9K lines) which used hashes a lot. I've found no issues so far (no memory corruption or anything). Performance did drop a bit from 812msecs to 898msecs. I can't extensively test this

Re: New hash

2012-03-23 Thread Andrej Mitrovic
t; > :-) Yeah but with a type string[int][int] this will store a AA!(int, string[int]), IOW you will be using a druntime hash inside of your new hash. Mine converts this to AA!(int, AA!(string, int)) to properly test the new AA nested hashes. Also your version won't work since AA!(int, s

Re: New hash

2012-03-23 Thread Andrej Mitrovic
On 3/24/12, H. S. Teoh wrote: > Cool. Should I just replace the current AA alias with Assoc? I think > this is far more concise, and much easier to convert to "real" AA syntax > when we integrate with druntime/dmd. Sure. I kept wondering whether AA!(int, string) is a hash with an int or a string

Re: New hash

2012-03-23 Thread H. S. Teoh
On Sat, Mar 24, 2012 at 05:47:55AM +0100, Andrej Mitrovic wrote: [...] > I've managed to test the hashes on a small-sized closed-source project > (9K lines) which used hashes a lot. I've found no issues so far (no > memory corruption or anything). Performance did drop a bit from > 812msecs to 898ms

Re: New hash

2012-03-23 Thread Andrej Mitrovic
On 3/24/12, H. S. Teoh wrote: > Note that if hash literals are used, then they can be inefficient > because of the current hack of copying from the current AA (so it will > involve two copies Okie. I was mostly measuring lookups though. I can't accurately measure writes because I'm taking data fr

Re: New hash

2012-03-23 Thread H. S. Teoh
On Sat, Mar 24, 2012 at 06:08:40AM +0100, Andrej Mitrovic wrote: > On 3/24/12, H. S. Teoh wrote: > > Note that if hash literals are used, then they can be inefficient > > because of the current hack of copying from the current AA (so it > > will involve two copies > > Okie. I was mostly measuring

Re: New hash

2012-03-23 Thread H. S. Teoh
On Sat, Mar 24, 2012 at 05:56:27AM +0100, Andrej Mitrovic wrote: > On 3/24/12, H. S. Teoh wrote: > > Cool. Should I just replace the current AA alias with Assoc? I think > > this is far more concise, and much easier to convert to "real" AA > > syntax when we integrate with druntime/dmd. > > Sure.

Re: New hash

2012-03-23 Thread Andrej Mitrovic
On 3/24/12, H. S. Teoh wrote: > OK, I found that your template doesn't handle the case where the key > type is an AA. Heh, yeah, I've never used hashes as key types before. > However, one case remains unsolved: either I can't find the right way to > express this, or the new AA template needs fix

Re: New hash

2012-03-24 Thread bearophile
H. S. Teoh: > Someday, we have to revisit this whole const thing and how to make it > work nicely with containers... I found that it is causing >40% of my > troubles with the AA implementation. :-( This is a very good idea, but I see two problems in it: - Do you know someone intelligent enough t

Re: New hash

2012-03-24 Thread Timon Gehr
On 03/24/2012 03:39 AM, H. S. Teoh wrote: On Sat, Mar 24, 2012 at 02:39:35AM +0100, Andrej Mitrovic wrote: I thought I'd open this topic for discussion of issues with the new hash implementation. [...] Another issue: AA!(string,int[]) aa; auto x = aa.get("abc&qu

Re: New hash

2012-03-24 Thread Andrei Alexandrescu
s seem to be a tiny bit slower. I wonder what the matter is. What hashing function does the new hash use? Let me reiterate the suggestion that different hash algorithms are used depending on the string length. Andrei

Re: New hash

2012-03-24 Thread H. S. Teoh
On Sat, Mar 24, 2012 at 06:50:12AM +0100, Andrej Mitrovic wrote: > On 3/24/12, H. S. Teoh wrote: [...] > > However, one case remains unsolved: either I can't find the right > > way to express this, or the new AA template needs fixing: > > > > AA!(string[const AA!(int[int])]) meta; > > Not sur

Re: New hash

2012-03-24 Thread H. S. Teoh
> >work with the new hashes but I'll fix that) and so I need a 30second > >run that first fills the hashes before doing work on them. But it > >does seem to be a tiny bit slower. > > I wonder what the matter is. What hashing function does the new hash > use? Let me r

Re: New hash

2012-03-24 Thread Andrej Mitrovic
On 3/24/12, H. S. Teoh wrote: > But this can't be the source of the problem, > because, if anything, the hash function I substituted (which is the same > as that for char[] and string currently) should be *faster*. I'll try and make a test-case to get to the bottom of this.

Re: New hash

2012-03-28 Thread Andrej Mitrovic
On 3/24/12, Andrej Mitrovic wrote: > snip I've noticed this: // FIXME: this shouldn't be duplicated for every template instance. static immutable size_t[] prime_list = [ I think you can make this an enum.

Re: New hash

2012-03-28 Thread H. S. Teoh
On Wed, Mar 28, 2012 at 09:44:59PM +0200, Andrej Mitrovic wrote: > On 3/24/12, Andrej Mitrovic wrote: > > snip > > I've noticed this: > > // FIXME: this shouldn't be duplicated for every template instance. > static immutable size_t[] prime_list = [ > > I think you can make this an enum. You're

New hash API: Update

2012-06-24 Thread Johannes Pfau
I'm mostly finished with my hash API proposal. I also ported the existing crc, md5 and the proposed sha1 hash to this new API. I changed the namespace to std.util.digest. Andrei once said he thinks std.digest/std.hash is a too narrow package and someone else said putting crc into std.crypto.digest

New hash API: namespace

2012-06-25 Thread Johannes Pfau
OK, so I understand std.util is probably not a good idea. So the candidates for the namespace are: * std.crypto.hash * std.checksum * std.crypto.hash and std.checksum * std.hash and the same with hash replaced by digest. So which one should we use?

Re: New hash API: Update

2012-06-24 Thread Piotr Szturmaj
Johannes Pfau wrote: I'm mostly finished with my hash API proposal. I also ported the existing crc, md5 and the proposed sha1 hash to this new API. I changed the namespace to std.util.digest. Andrei once said he thinks std.digest/std.hash is a too narrow package and someone else said putting crc

Re: New hash API: Update

2012-06-24 Thread Masahiro Nakagawa
On Sunday, 24 June 2012 at 15:23:19 UTC, Johannes Pfau wrote: I'm mostly finished with my hash API proposal. I also ported the existing crc, md5 and the proposed sha1 hash to this new API. Great! I will read docs and souce code later. I changed the namespace to std.util.digest. Andrei once sai

Re: New hash API: Update

2012-06-24 Thread David
Am 24.06.2012 18:13, schrieb Masahiro Nakagawa: I disagree this point. The name 'util' does not make any sense. 'util' seems to be very subjective. Right `util` can be everything.

Re: New hash API: Update

2012-06-24 Thread Johannes Pfau
Am Sun, 24 Jun 2012 18:21:06 +0200 schrieb David : > Am 24.06.2012 18:13, schrieb Masahiro Nakagawa: > > I disagree this point. The name 'util' does not make any sense. > > 'util' seems to be very subjective. > > > > Right `util` can be everything. > Yep that's the idea. You put everything in t

Re: New hash API: Update

2012-06-24 Thread Johannes Pfau
Am Sun, 24 Jun 2012 18:07:53 +0200 schrieb Piotr Szturmaj : > Johannes Pfau wrote: > > I'm mostly finished with my hash API proposal. I also ported the > > existing crc, md5 and the proposed sha1 hash to this new API. > > > > I changed the namespace to std.util.digest. Andrei once said he > > thin

Re: New hash API: Update

2012-06-24 Thread Dmitry Olshansky
On 24-Jun-12 19:23, Johannes Pfau wrote: I'm mostly finished with my hash API proposal. I also ported the existing crc, md5 and the proposed sha1 hash to this new API. I changed the namespace to std.util.digest. Andrei once said he thinks std.digest/std.hash is a too narrow package and someone e

Re: New hash API: Update

2012-06-24 Thread David Nadlinger
On Sunday, 24 June 2012 at 18:10:03 UTC, Dmitry Olshansky wrote: To be frank almost anything could 'nicely fit' into util. That's why we should avoid this abomination at all costs. I tend to agree. An »util« package might make sense in an application, where it could e.g. hold smaller self-cont

Re: New hash API: Update

2012-06-24 Thread Alex Rønne Petersen
On 24-06-2012 17:23, Johannes Pfau wrote: I'm mostly finished with my hash API proposal. I also ported the existing crc, md5 and the proposed sha1 hash to this new API. Thanks. I changed the namespace to std.util.digest. Andrei once said he thinks std.digest/std.hash is a too narrow package

Re: New hash API: Update

2012-06-24 Thread Dmitry Olshansky
On 24-Jun-12 19:23, Johannes Pfau wrote: There are still some open questions: OOP interface: Digest.finish() This can only throw if the supplied buffer is too small. Make this nothrow & throw an Error on too small buffer? Or check buffer only in debug mode using asserts or preconditions? Yu

Re: New hash API: Update

2012-06-24 Thread Jonathan M Davis
On Sunday, June 24, 2012 17:23:18 Johannes Pfau wrote: > I'm mostly finished with my hash API proposal. I also ported the > existing crc, md5 and the proposed sha1 hash to this new API. > > I changed the namespace to std.util.digest. Andrei once said he thinks > std.digest/std.hash is a too narrow

Re: New hash API: Update

2012-06-25 Thread Johannes Pfau
Am Sun, 24 Jun 2012 17:58:47 -0700 schrieb Jonathan M Davis : > On Sunday, June 24, 2012 17:23:18 Johannes Pfau wrote: > > I'm mostly finished with my hash API proposal. I also ported the > > existing crc, md5 and the proposed sha1 hash to this new API. > > > > I changed the namespace to std.util

Re: New hash API: Update

2012-06-25 Thread Johannes Pfau
Am Mon, 25 Jun 2012 00:02:00 +0400 schrieb Dmitry Olshansky : > On 24-Jun-12 19:23, Johannes Pfau wrote: > > > > There are still some open questions: > > > > OOP interface: Digest.finish() > > > > This can only throw if the supplied buffer is too small. Make this > > nothrow & throw an Error on to

Re: New hash API: Update

2012-06-25 Thread Johannes Pfau
Am Sun, 24 Jun 2012 21:40:53 +0200 schrieb Alex Rønne Petersen : > Also, most (if not all) Digest methods really should be pure nothrow. > Same for some free functions (I think it's good practice to mark > template functions as pure nothrow explicitly if you want to > guarantee this). I still d

Re: New hash API: namespace

2012-06-25 Thread Jonathan M Davis
On Monday, June 25, 2012 11:35:33 Johannes Pfau wrote: > OK, so I understand std.util is probably not a good idea. > > So the candidates for the namespace are: > * std.crypto.hash > * std.checksum > * std.crypto.hash and std.checksum > * std.hash > > and the same with hash replaced by digest. > S

Re: New hash API: namespace

2012-06-25 Thread Piotr Szturmaj
Jonathan M Davis wrote: On Monday, June 25, 2012 11:35:33 Johannes Pfau wrote: OK, so I understand std.util is probably not a good idea. So the candidates for the namespace are: * std.crypto.hash * std.checksum * std.crypto.hash and std.checksum * std.hash and the same with hash replaced by di

Re: New hash API: Update

2012-06-25 Thread Jonathan M Davis
On Monday, June 25, 2012 11:30:41 Johannes Pfau wrote: > Am Sun, 24 Jun 2012 21:40:53 +0200 > > schrieb Alex Rønne Petersen : > > Also, most (if not all) Digest methods really should be pure nothrow. > > Same for some free functions (I think it's good practice to mark > > template functions as pur

Re: New hash API: Update

2012-06-25 Thread Alex Rønne Petersen
On 25-06-2012 11:13, Johannes Pfau wrote: Am Sun, 24 Jun 2012 17:58:47 -0700 schrieb Jonathan M Davis : On Sunday, June 24, 2012 17:23:18 Johannes Pfau wrote: I'm mostly finished with my hash API proposal. I also ported the existing crc, md5 and the proposed sha1 hash to this new API. I chang

Re: New hash API: namespace

2012-06-25 Thread Jacob Carlborg
On 2012-06-25 12:24, Piotr Szturmaj wrote: Jonathan M Davis wrote: On Monday, June 25, 2012 11:35:33 Johannes Pfau wrote: OK, so I understand std.util is probably not a good idea. So the candidates for the namespace are: * std.crypto.hash * std.checksum * std.crypto.hash and std.checksum * std

Re: New hash API: namespace

2012-06-25 Thread Jonathan M Davis
On Monday, June 25, 2012 12:24:44 Piotr Szturmaj wrote: > Jonathan M Davis wrote: > > On Monday, June 25, 2012 11:35:33 Johannes Pfau wrote: > >> OK, so I understand std.util is probably not a good idea. > >> > >> So the candidates for the namespace are: > >> * std.crypto.hash > >> * std.checksum

Re: New hash API: namespace

2012-06-25 Thread Felix Hufnagel
+1 for hashes into std.hash and cryptographic primitives into std.crypto and we should have a std.net (std.uri, std.socket, std.socketstream , std.net.curl, ...), std.io. for (Outbuffer, file, ) and probably std.database or something like that for (csv, json, xml, ...) ... Am 25.06.2012

Re: New hash API: namespace

2012-06-25 Thread nazriel
On Monday, 25 June 2012 at 16:09:43 UTC, Felix Hufnagel wrote: +1 for hashes into std.hash and cryptographic primitives into std.crypto and we should have a std.net (std.uri, std.socket, std.socketstream , std.net.curl, ...), std.io. for (Outbuffer, file, ) and probably std.database or som

Re: New hash API: namespace

2012-06-25 Thread Dmitry Olshansky
On 25-Jun-12 20:09, Felix Hufnagel wrote: +1 for hashes into std.hash and cryptographic primitives into std.crypto Another +1 here and we should have std.net (std.uri, std.socket, std.socketstream , std.net.curl, ...), std.io for proper I/O framework. and probably std.data or something like t

Re: New hash API: namespace

2012-06-25 Thread Jesse Phillips
On Monday, 25 June 2012 at 16:09:43 UTC, Felix Hufnagel wrote: +1 for hashes into std.hash and cryptographic primitives into std.crypto and we should have a std.net (std.uri, std.socket, std.socketstream , std.net.curl, ...), std.io. for (Outbuffer, file, ) and probably std.database or som

Re: New hash API: namespace

2012-06-29 Thread Don Clugston
On 25/06/12 20:04, Jesse Phillips wrote: On Monday, 25 June 2012 at 16:09:43 UTC, Felix Hufnagel wrote: +1 for hashes into std.hash and cryptographic primitives into std.crypto and we should have a std.net (std.uri, std.socket, std.socketstream , std.net.curl, ...), std.io. for (Outbuffer, file

Re: New hash API: namespace

2012-06-29 Thread Piotr Szturmaj
Don Clugston wrote: On 25/06/12 20:04, Jesse Phillips wrote: On Monday, 25 June 2012 at 16:09:43 UTC, Felix Hufnagel wrote: +1 for hashes into std.hash and cryptographic primitives into std.crypto and we should have a std.net (std.uri, std.socket, std.socketstream , std.net.curl, ...), std.io.