Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-21 Thread Daniel Kozak via Digitalmars-d-learn
Dne so 20. 1. 2024 21:21 uživatel Renato via Digitalmars-d-learn < digitalmars-d-learn@puremagic.com> napsal: > On Saturday, 20 January 2024 at 19:45:19 UTC, Daniel Kozak wrote: > > On Sat, Jan 20, 2024 at 2:11 PM Renato via Digitalmars-d-learn > > < digitalmars-d-learn@puremagic.com> wrote: > >

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-20 Thread Renato via Digitalmars-d-learn
On Saturday, 20 January 2024 at 19:45:19 UTC, Daniel Kozak wrote: On Sat, Jan 20, 2024 at 2:11 PM Renato via Digitalmars-d-learn < digitalmars-d-learn@puremagic.com> wrote: Wow, fantastic feedback from lots of people, thanks so much! ... > evilrat: > There is another important difference, i

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-20 Thread Daniel Kozak via Digitalmars-d-learn
On Sat, Jan 20, 2024 at 2:11 PM Renato via Digitalmars-d-learn < digitalmars-d-learn@puremagic.com> wrote: > Wow, fantastic feedback from lots of people, thanks so much! > ... > > > evilrat: > > There is another important difference, i quickly looked up D > > associative array implementation and

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-20 Thread Renato via Digitalmars-d-learn
On Saturday, 20 January 2024 at 13:07:39 UTC, Renato wrote: D overhead with the fastest compiler, LDC, compared with Rust: ``` 1.0 1.707627119 1.919527897 1.954595186 1.351342502 1.556217748 ``` Oh sorry, I only posted the rates for the Linux benchmark... On Mac M1, for some reason, D was

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-20 Thread Renato via Digitalmars-d-learn
Wow, fantastic feedback from lots of people, thanks so much! I will try to answer all points raised in the several answers I've got here since yesterday. On Saturday, 20 January 2024 at 01:27:50 UTC, H. S. Teoh wrote: I'm confused by the chained hashing of the digits. Why is that

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-20 Thread Siarhei Siamashka via Digitalmars-d-learn
On Saturday, 20 January 2024 at 01:27:50 UTC, H. S. Teoh wrote: On Sat, Jan 20, 2024 at 01:35:44AM +0100, Daniel Kozak via Digitalmars-d-learn wrote: [...] > Try addressing the points I wrote above and see if it makes a > difference. I have tried it (all of it) even before you wrote

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-19 Thread H. S. Teoh via Digitalmars-d-learn
On Sat, Jan 20, 2024 at 01:35:44AM +0100, Daniel Kozak via Digitalmars-d-learn wrote: [...] >> Try addressing the points I wrote above and see if it makes a >> difference. > >I have tried it (all of it) even before you wrote it here, because >I have completely the same ideas, but

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-19 Thread Daniel Kozak via Digitalmars-d-learn
On Fri, Jan 19, 2024 at 4:44 PM H. S. Teoh via Digitalmars-d-learn < digitalmars-d-learn@puremagic.com> wrote: > Taking a look at this code: > ... > Try addressing the points I wrote above and see if it makes a > difference. > > I have tried it (all of it) even before you wrote it here, because

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-19 Thread ryuukk_ via Digitalmars-d-learn
On Friday, 19 January 2024 at 17:18:36 UTC, evilrat wrote: On Friday, 19 January 2024 at 16:55:25 UTC, ryuukk_ wrote: You do hash map lookup for every character in D, it's slow, whereas in Rust you do it via pattern matching, java does the same, pattern matching Yet another reason to

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-19 Thread Jordan Wilson via Digitalmars-d-learn
On Friday, 19 January 2024 at 08:57:40 UTC, Renato wrote: Do you know why the whole thread seems to have disappeared?? There's a lot of good stuff in the thread, it would be a huge shame to lose all that! I agree! Thanks for posting your benchmarks, I thought your whole benching setup was

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-19 Thread evilrat via Digitalmars-d-learn
On Friday, 19 January 2024 at 16:55:25 UTC, ryuukk_ wrote: You do hash map lookup for every character in D, it's slow, whereas in Rust you do it via pattern matching, java does the same, pattern matching Yet another reason to advocate for pattern matching in D and switch as expression

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-19 Thread ryuukk_ via Digitalmars-d-learn
On Friday, 19 January 2024 at 13:40:39 UTC, Renato wrote: On Friday, 19 January 2024 at 10:15:57 UTC, evilrat wrote: On Friday, 19 January 2024 at 09:08:17 UTC, Renato wrote: I forgot to mention: the Java version is using a Trie... and it consistently beats the Rust numeric algorithm (which

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-19 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Jan 19, 2024 at 01:40:39PM +, Renato via Digitalmars-d-learn wrote: > On Friday, 19 January 2024 at 10:15:57 UTC, evilrat wrote: [...] > > Additionally if you comparing D by measuring DMD performance - > > don't. It is valuable in developing for fast iterations, but it > > lacks many

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-19 Thread Renato via Digitalmars-d-learn
On Friday, 19 January 2024 at 10:15:57 UTC, evilrat wrote: On Friday, 19 January 2024 at 09:08:17 UTC, Renato wrote: I forgot to mention: the Java version is using a Trie... and it consistently beats the Rust numeric algorithm (which means it's still faster than your D solution), but the

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-19 Thread Renato via Digitalmars-d-learn
On Friday, 19 January 2024 at 10:15:57 UTC, evilrat wrote: On Friday, 19 January 2024 at 09:08:17 UTC, Renato wrote: I forgot to mention: the Java version is using a Trie... and it consistently beats the Rust numeric algorithm (which means it's still faster than your D solution), but the

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-19 Thread Renato via Digitalmars-d-learn
On Friday, 19 January 2024 at 05:17:51 UTC, H. S. Teoh wrote: On Thu, Jan 18, 2024 at 04:23:16PM +, Renato via Digitalmars-d-learn wrote: [...] Ok, last time I'm running this for someone else :D ``` Proc,Run,Memory(bytes),Time(ms) ===> ./rust ./rust,23920640,30 ./rust,24018944,147

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-18 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Jan 18, 2024 at 04:23:16PM +, Renato via Digitalmars-d-learn wrote: [...] > Ok, last time I'm running this for someone else :D > > ``` > Proc,Run,Memory(bytes),Time(ms) > ===> ./rust > ./rust,23920640,30 > ./rust,24018944,147 > ./rust,24068096,592 > ./rust,24150016,1187 >

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-18 Thread Renato via Digitalmars-d-learn
On Wednesday, 17 January 2024 at 16:54:00 UTC, H. S. Teoh wrote: On Wed, Jan 17, 2024 at 07:57:02AM -0800, H. S. Teoh via Digitalmars-d-learn wrote: [...] I'll push the code to github. [...] Here: https://github.com/quickfur/prechelt/blob/master/encode_phone.d T BTW here's you main

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-18 Thread Renato via Digitalmars-d-learn
On Wednesday, 17 January 2024 at 16:54:00 UTC, H. S. Teoh wrote: On Wed, Jan 17, 2024 at 07:57:02AM -0800, H. S. Teoh via Digitalmars-d-learn wrote: [...] I'll push the code to github. [...] Here: https://github.com/quickfur/prechelt/blob/master/encode_phone.d T Ok, last time I'm

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-17 Thread Renato via Digitalmars-d-learn
On Wednesday, 17 January 2024 at 16:30:08 UTC, H. S. Teoh wrote: On Wed, Jan 17, 2024 at 07:19:39AM +, Renato via Digitalmars-d-learn wrote: [...] But pls run the benchmarks yourself as I am not going to keep running it for you, and would be nice if you posted your solution on a Gist for

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-17 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jan 17, 2024 at 07:57:02AM -0800, H. S. Teoh via Digitalmars-d-learn wrote: [...] > I'll push the code to github. [...] Here: https://github.com/quickfur/prechelt/blob/master/encode_phone.d T -- Why do conspiracy theories always come from the same people??

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-17 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jan 17, 2024 at 07:19:39AM +, Renato via Digitalmars-d-learn wrote: [...] > But pls run the benchmarks yourself as I am not going to keep running > it for you, and would be nice if you posted your solution on a Gist > for example, pasting lots of code in the forum makes it difficult to

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-17 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jan 17, 2024 at 07:19:39AM +, Renato via Digitalmars-d-learn wrote: > On Tuesday, 16 January 2024 at 22:13:55 UTC, H. S. Teoh wrote: > > used for the recursive calls. Getting rid of the .format ought to > > speed it up a bit. Will try that now... > > > > That will make no difference

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-17 Thread Renato via Digitalmars-d-learn
On Wednesday, 17 January 2024 at 11:56:19 UTC, evilrat wrote: On Wednesday, 17 January 2024 at 11:20:14 UTC, Renato wrote: That means the input file is still not ASCII (or UTF-8) as it should. Java is reading files with the ASCII encoding so it should've worked fine. It seems that it is

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-17 Thread evilrat via Digitalmars-d-learn
On Wednesday, 17 January 2024 at 11:20:14 UTC, Renato wrote: That means the input file is still not ASCII (or UTF-8) as it should. Java is reading files with the ASCII encoding so it should've worked fine. It seems that it is only works with ASCII encoding though.

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-17 Thread Renato via Digitalmars-d-learn
On Wednesday, 17 January 2024 at 10:50:26 UTC, evilrat wrote: On Wednesday, 17 January 2024 at 10:43:22 UTC, Renato wrote: On Wednesday, 17 January 2024 at 10:24:31 UTC, Renato wrote: It's not Java writing the file, it's the bash script

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-17 Thread evilrat via Digitalmars-d-learn
On Wednesday, 17 January 2024 at 10:43:22 UTC, Renato wrote: On Wednesday, 17 January 2024 at 10:24:31 UTC, Renato wrote: It's not Java writing the file, it's the bash script [`benchmark.sh`](https://github.com/renatoathaydes/prechelt-phone-number-encoding/blob/master/benchmark.sh#L31): ```

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-17 Thread Renato via Digitalmars-d-learn
On Wednesday, 17 January 2024 at 10:24:31 UTC, Renato wrote: It's not Java writing the file, it's the bash script [`benchmark.sh`](https://github.com/renatoathaydes/prechelt-phone-number-encoding/blob/master/benchmark.sh#L31): ``` java -cp "build/util" util.GeneratePhoneNumbers 1000 >

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-17 Thread Renato via Digitalmars-d-learn
On Wednesday, 17 January 2024 at 09:15:02 UTC, evilrat wrote: On Wednesday, 17 January 2024 at 07:11:02 UTC, Renato wrote: If you want to check your performance, you know you can run the `./benchmark.sh` yourself? Out of curiosity I've tried to manually run this on Windows and it seems

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-17 Thread evilrat via Digitalmars-d-learn
On Wednesday, 17 January 2024 at 07:06:25 UTC, Renato wrote: On Tuesday, 16 January 2024 at 22:15:04 UTC, Siarhei Siamashka wrote: On Tuesday, 16 January 2024 at 21:15:19 UTC, Renato wrote: It's a GC allocations fest. Things like this make it slow: ```diff { -string digit =

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-17 Thread evilrat via Digitalmars-d-learn
On Wednesday, 17 January 2024 at 07:11:02 UTC, Renato wrote: If you want to check your performance, you know you can run the `./benchmark.sh` yourself? Out of curiosity I've tried to manually run this on Windows and it seems that Java generator for these numbers files is "broken", the

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-16 Thread Renato via Digitalmars-d-learn
On Tuesday, 16 January 2024 at 22:13:55 UTC, H. S. Teoh wrote: used for the recursive calls. Getting rid of the .format ought to speed it up a bit. Will try that now... That will make no difference for the `count` option which is where your solution was very slow. To run the slow test

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-16 Thread Renato via Digitalmars-d-learn
On Tuesday, 16 January 2024 at 22:15:04 UTC, Siarhei Siamashka wrote: On Tuesday, 16 January 2024 at 21:15:19 UTC, Renato wrote: For the record (I already posted this on GitHub)... here's [my current fastest

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-16 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Jan 16, 2024 at 10:15:04PM +, Siarhei Siamashka via Digitalmars-d-learn wrote: > On Tuesday, 16 January 2024 at 21:15:19 UTC, Renato wrote: [...] > > ... what I am really curious about is what the code I wrote is doing > > wrong that causes it to run 4x slower than Rust despite doing

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-16 Thread Siarhei Siamashka via Digitalmars-d-learn
On Tuesday, 16 January 2024 at 21:15:19 UTC, Renato wrote: For the record (I already posted this on GitHub)... here's [my current fastest solution](https://github.com/renatoathaydes/prechelt-phone-number-encoding/blob/dlang-key-hash-incremental/src/d/src/dencoder.d) time using the same

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-16 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Jan 16, 2024 at 09:15:19PM +, Renato via Digitalmars-d-learn wrote: > On Tuesday, 16 January 2024 at 20:34:48 UTC, H. S. Teoh wrote: > > On Tue, Jan 16, 2024 at 12:28:49PM -0800, H. S. Teoh via > > Digitalmars-d-learn wrote: [...] > > > Anyway, I've fixed the problem, now my program

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-16 Thread Siarhei Siamashka via Digitalmars-d-learn
On Tuesday, 16 January 2024 at 21:15:19 UTC, Renato wrote: I can't explain why it's so incredibly fast, specially for the `count` case. I tried using the same hashing function on my solution, but that didn't really help me! That's dynamic programming with memoization. Basically caching the

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-16 Thread Renato via Digitalmars-d-learn
On Tuesday, 16 January 2024 at 20:34:48 UTC, H. S. Teoh wrote: On Tue, Jan 16, 2024 at 12:28:49PM -0800, H. S. Teoh via Digitalmars-d-learn wrote: [...] Anyway, I've fixed the problem, now my program produces the exact same output as Renato's repo. Code is posted below. [...] Great, I ran

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-16 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Jan 16, 2024 at 12:28:49PM -0800, H. S. Teoh via Digitalmars-d-learn wrote: [...] > Anyway, I've fixed the problem, now my program produces the exact same > output as Renato's repo. Code is posted below. [...] Oops, forgot to actually paste the code. Here it is:

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-16 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Jan 16, 2024 at 06:54:56PM +, Renato via Digitalmars-d-learn wrote: > On Tuesday, 16 January 2024 at 16:56:04 UTC, Siarhei Siamashka wrote: [...] > > You are not allowed to emit "1" as the first token in the output as > > long as there are any dictionary word matches at that position.

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-16 Thread Renato via Digitalmars-d-learn
On Tuesday, 16 January 2024 at 16:56:04 UTC, Siarhei Siamashka wrote: On Tuesday, 16 January 2024 at 15:50:35 UTC, H. S. Teoh wrote: Unfortunately there seems to be some discrepancy between the output I got and the prescribed output in your repository. For example, in your output the number

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-16 Thread Siarhei Siamashka via Digitalmars-d-learn
On Tuesday, 16 January 2024 at 15:50:35 UTC, H. S. Teoh wrote: Unfortunately there seems to be some discrepancy between the output I got and the prescribed output in your repository. For example, in your output the number 1556/0 does not have an encoding, but isn't "1 Mai 0" a valid encoding

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-16 Thread H. S. Teoh via Digitalmars-d-learn
P.S. Compiling my program with `ldc -O2`, it runs so fast that I couldn't measure any meaningful running time that's greater than startup overhead. So I wrote a helper program to generate random phone numbers up to 50 characters long, and found that it could encode 1 million phone numbers in 2.2

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-16 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Jan 16, 2024 at 07:50:35AM -0800, H. S. Teoh via Digitalmars-d-learn wrote: [...] > Unfortunately there seems to be some discrepancy between the output I > got and the prescribed output in your repository. For example, in your > output the number 1556/0 does not have an encoding, but

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-16 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Jan 15, 2024 at 08:10:55PM +, Renato via Digitalmars-d-learn wrote: > On Monday, 15 January 2024 at 01:10:14 UTC, Sergey wrote: > > On Sunday, 14 January 2024 at 17:11:27 UTC, Renato wrote: > > > If anyone can find any flaw in my methodology or optmise my code so > > > that it can

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-15 Thread Renato via Digitalmars-d-learn
On Monday, 15 January 2024 at 01:10:14 UTC, Sergey wrote: On Sunday, 14 January 2024 at 17:11:27 UTC, Renato wrote: If anyone can find any flaw in my methodology or optmise my code so that it can still get a couple of times faster, approaching Rust's performance, I would greatly appreciate

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-14 Thread Sergey via Digitalmars-d-learn
On Sunday, 14 January 2024 at 17:11:27 UTC, Renato wrote: If anyone can find any flaw in my methodology or optmise my code so that it can still get a couple of times faster, approaching Rust's performance, I would greatly appreciate that! But for now, my understanding is that the most

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-14 Thread Renato via Digitalmars-d-learn
On Sunday, 14 January 2024 at 10:02:58 UTC, Jordan Wilson wrote: On Saturday, 13 January 2024 at 11:03:42 UTC, Renato wrote: I like to use a phone encoding problem to determine the strenghtness and weaknesses of programming languages because this problem is easy enough I can write solutions in

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-14 Thread Renato via Digitalmars-d-learn
On Saturday, 13 January 2024 at 23:20:32 UTC, Sergey wrote: On Saturday, 13 January 2024 at 19:35:57 UTC, Renato wrote: On Saturday, 13 January 2024 at 17:00:58 UTC, Anonymouse wrote: On Saturday, 13 January 2024 at 12:55:27 UTC, Renato wrote: [...] I will have to try it... I thought that

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-14 Thread Jordan Wilson via Digitalmars-d-learn
On Saturday, 13 January 2024 at 11:03:42 UTC, Renato wrote: I like to use a phone encoding problem to determine the strenghtness and weaknesses of programming languages because this problem is easy enough I can write solutions in any language in a few hours, but complex enough to exercise lots

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-14 Thread Anonymouse via Digitalmars-d-learn
On Saturday, 13 January 2024 at 23:20:32 UTC, Sergey wrote: I would suggest to rewrite in the same way as Rust implemented. Probably you would like to try: [...] I would strongly argue for profiling first instead of optimising based on conjecture. If you profile you have solid evidence on

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-13 Thread Sergey via Digitalmars-d-learn
On Saturday, 13 January 2024 at 19:35:57 UTC, Renato wrote: On Saturday, 13 January 2024 at 17:00:58 UTC, Anonymouse wrote: On Saturday, 13 January 2024 at 12:55:27 UTC, Renato wrote: [...] I will have to try it... I thought that `BigInt` was to blame for the slowness (from what I could read

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-13 Thread Renato via Digitalmars-d-learn
On Saturday, 13 January 2024 at 17:00:58 UTC, Anonymouse wrote: On Saturday, 13 January 2024 at 12:55:27 UTC, Renato wrote: [...] Not a great profiling experience :). Anyone has a better suggestion to "parse" the trace file? As a drive-by suggestion and I hope it doesn't derail anything,

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-13 Thread Anonymouse via Digitalmars-d-learn
On Saturday, 13 January 2024 at 12:55:27 UTC, Renato wrote: [...] Not a great profiling experience :). Anyone has a better suggestion to "parse" the trace file? As a drive-by suggestion and I hope it doesn't derail anything, but if you have the opportunity to run it on linux, have you tried

Re: Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-13 Thread Renato via Digitalmars-d-learn
On Saturday, 13 January 2024 at 11:03:42 UTC, Renato wrote: I tried to profile the D code but the profiler seems to be broken on my OS (Mac): I profiled it on Linux and stored [the trace.log file](https://gist.github.com/renatoathaydes/fd8752ed81b0cf792ed7ef5aa3d68acd) on a public Gist.

Help optimize D solution to phone encoding problem: extremely slow performace.

2024-01-13 Thread Renato via Digitalmars-d-learn
I like to use a phone encoding problem to determine the strenghtness and weaknesses of programming languages because this problem is easy enough I can write solutions in any language in a few hours, but complex enough to exercise lots of interesting parts of the language. You can check [my