Re: The GC and performance, but not what you expect

2014-06-12 Thread Etienne via Digitalmars-d
On 2014-06-10 5:53 AM, Adam Sakareassen via Digitalmars-d wrote: Hi, Yes my little GC project is coming along. It allocates much faster in multi threaded applications when all threads are competing for the lock. D's current GC is very slow when there is contention for the GC-lock which is acq

Re: The GC and performance, but not what you expect

2014-06-11 Thread Nordlöw
Only a little. In scripts where I deliberately introduce contention my allocator is quicker. It's much closer when there is little contention. Very interesting... It would be nice to see single-threaded benchmarks aswell for completeness. Thx-

Re: The GC and performance, but not what you expect

2014-06-10 Thread Rainer Schuetze via Digitalmars-d
On 10.06.2014 17:15, Sean Kelly wrote: On Thursday, 29 May 2014 at 23:39:02 UTC, Marco Leise wrote: Nice try, but destructors called by the GC are currently effectively @nogc. So don't try that at home. When did that happen? Some effort was made at one point to ensure that allocations work

Re: The GC and performance, but not what you expect

2014-06-10 Thread Adam Sakareassen via Digitalmars-d
On 11/06/2014 6:39 AM, "Nordlöw" via Digitalmars-d wrote: Have you benchmarked your lock-free GC against D's builtin with different allocation sizes? I've seen comments on the need for that... Only a little. In scripts where I deliberately introduce contention my allocator is quicker. It's mu

Re: The GC and performance, but not what you expect

2014-06-10 Thread Nordlöw
Yes my little GC project is coming along. It allocates much faster in multi threaded applications when all threads are competing for the lock. Have you benchmarked your lock-free GC against D's builtin with different allocation sizes? I've seen comments on the need for that... Anyhow very exc

Re: The GC and performance, but not what you expect

2014-06-10 Thread Joakim via Digitalmars-d
On Tuesday, 10 June 2014 at 17:58:56 UTC, Sean Kelly wrote: On Tuesday, 10 June 2014 at 17:07:23 UTC, Joakim wrote: On Tuesday, 10 June 2014 at 15:15:46 UTC, Sean Kelly wrote: On Thursday, 29 May 2014 at 23:39:02 UTC, Marco Leise wrote: Nice try, but destructors called by the GC are currently

Re: The GC and performance, but not what you expect

2014-06-10 Thread Sean Kelly via Digitalmars-d
On Tuesday, 10 June 2014 at 17:07:23 UTC, Joakim wrote: On Tuesday, 10 June 2014 at 15:15:46 UTC, Sean Kelly wrote: On Thursday, 29 May 2014 at 23:39:02 UTC, Marco Leise wrote: Nice try, but destructors called by the GC are currently effectively @nogc. So don't try that at home. When did tha

Re: The GC and performance, but not what you expect

2014-06-10 Thread Joakim via Digitalmars-d
On Tuesday, 10 June 2014 at 15:15:46 UTC, Sean Kelly wrote: On Thursday, 29 May 2014 at 23:39:02 UTC, Marco Leise wrote: Nice try, but destructors called by the GC are currently effectively @nogc. So don't try that at home. When did that happen? Some effort was made at one point to ensure t

Re: The GC and performance, but not what you expect

2014-06-10 Thread Sean Kelly via Digitalmars-d
On Thursday, 29 May 2014 at 23:39:02 UTC, Marco Leise wrote: Nice try, but destructors called by the GC are currently effectively @nogc. So don't try that at home. When did that happen? Some effort was made at one point to ensure that allocations worked from dtors. Not that I'm in favor, b

Re: The GC and performance, but not what you expect

2014-06-10 Thread Adam Sakareassen via Digitalmars-d
Hi, Yes my little GC project is coming along. It allocates much faster in multi threaded applications when all threads are competing for the lock. D's current GC is very slow when there is contention for the GC-lock which is acquired for every malloc. The other hidden lock usage is when a

Re: The GC and performance, but not what you expect

2014-06-01 Thread Marco Leise via Digitalmars-d
Am Sat, 31 May 2014 08:52:34 + schrieb "Kagamin" : > The design of TCMalloc seems to be compatible with Boehm GC, so D > GC can probably use same ideas. Did I mention that Orvid planned on doing that for his replacement GC? It looks like there are now a lot of proof of concept GC modificatio

Re: The GC and performance, but not what you expect

2014-05-31 Thread Kagamin via Digitalmars-d
The design of TCMalloc seems to be compatible with Boehm GC, so D GC can probably use same ideas.

Re: The GC and performance, but not what you expect

2014-05-30 Thread Marco Leise via Digitalmars-d
Am Fri, 30 May 2014 15:54:57 + schrieb "Ola Fosheim Grøstad" : > On Friday, 30 May 2014 at 09:46:10 UTC, Marco Leise wrote: > > simplicity. But as soon as I added a single CAS I was already > > over the time that TCMalloc needs. That way I learned that CAS > > is not as cheap as it looks and t

Re: The GC and performance, but not what you expect

2014-05-30 Thread Brad Anderson via Digitalmars-d
On Friday, 30 May 2014 at 05:45:52 UTC, Andrei Alexandrescu wrote: Thing is the allocation patterns in D and Java are mightily different. Java does need a good GC because it allocates everywhere for everything. I think we need to come up with our own measurements and statistics when it comes

Re: The GC and performance, but not what you expect

2014-05-30 Thread via Digitalmars-d
This is also one of many good reasons to not make allocators library based, but do it in the compiler and backend. That also allows you do allocate multiple objects in a single CAS. (i.e. the compiler collects multiple allocs calls and replace it with one)

Re: The GC and performance, but not what you expect

2014-05-30 Thread via Digitalmars-d
On Friday, 30 May 2014 at 09:46:10 UTC, Marco Leise wrote: simplicity. But as soon as I added a single CAS I was already over the time that TCMalloc needs. That way I learned that CAS is not as cheap as it looks and the fastest allocators work thread local as long as possible. 22 cycles latency

Re: The GC and performance, but not what you expect

2014-05-30 Thread Nordlöw
On the topic of perf, I found a stupid trick the other day and wrote it down on the wiki: http://wiki.dlang.org/Perf Great tip! Now I want bash completion for it. But I can't seem to get it by reusing _kill in bash-completion: https://superuser.com/questions/760613/reusing-kill-completion-in

Re: The GC and performance, but not what you expect

2014-05-30 Thread Marco Leise via Digitalmars-d
Am Fri, 30 May 2014 10:29:58 +0200 schrieb Rainer Schuetze : > > > On 29.05.2014 12:09, Atila Neves wrote: > > The GC is preventing me from beating Java, but not because of > > collections. It's the locking it does to allocate instead! I > > don't know about the rest of you but I definitely didn

Re: The GC and performance, but not what you expect

2014-05-30 Thread Rainer Schuetze via Digitalmars-d
On 29.05.2014 12:09, Atila Neves wrote: The GC is preventing me from beating Java, but not because of collections. It's the locking it does to allocate instead! I don't know about the rest of you but I definitely didn't see that one coming. A lock should not be more than a CAS operation that

Re: The GC and performance, but not what you expect

2014-05-30 Thread Atila Neves via Digitalmars-d
I tried modifying the runtime but unfortunately vibe.d doesn't compile with the latest dmd. I don't feel like fixing it or figuring out how to compile gdc either. As mentioned later, I wouldn't be able to claim to beat Java, but I would be able to see how much faster it would go without the lock,

Re: The GC and performance, but not what you expect

2014-05-29 Thread Andrei Alexandrescu via Digitalmars-d
On 5/29/14, 4:03 AM, Wanderer wrote: It will be hard to beat Java's garbage collector. They use generational, sequential memory approach, while D's memory manager (correct me if I'm wrong) uses more common approach and has to deal with possible memory fragmentation. Allocating a new object is ext

Re: The GC and performance, but not what you expect

2014-05-29 Thread Martin Nowak via Digitalmars-d
On Thursday, 29 May 2014 at 13:06:20 UTC, Steven Schveighoffer wrote: More and more, it's looking like we are going to start needing thread-local pools for thread-local allocations. This is the best solution, get rid of the locking for most allocations even in multi-threaded programs.

Re: The GC and performance, but not what you expect

2014-05-29 Thread Marco Leise via Digitalmars-d
Am Thu, 29 May 2014 18:00:13 + schrieb "Brian Schott" : > On Thursday, 29 May 2014 at 10:09:18 UTC, Atila Neves wrote: > > If you're profiling binaries on Linux, this thing is a must > > have and I have no idea how I'd never heard about it before. > > On the topic of perf, I found a stupid t

Re: The GC and performance, but not what you expect

2014-05-29 Thread Marco Leise via Digitalmars-d
Am Thu, 29 May 2014 12:35:46 + schrieb "safety0ff" : > On Thursday, 29 May 2014 at 10:09:18 UTC, Atila Neves wrote: > > > > The GC is preventing me from beating Java, but not because of > > collections. It's the locking it does to allocate instead! I > > don't know about the rest of you but I

Re: The GC and performance, but not what you expect

2014-05-29 Thread Marco Leise via Digitalmars-d
Am Thu, 29 May 2014 20:01:16 + schrieb "Sean Kelly" : > On Thursday, 29 May 2014 at 19:00:24 UTC, Walter Bright wrote: > > > > If it's single threaded, and the single thread is doing the > > collecting, who is starting up a new thread? > > class Foo { > ~this() { > auto t = n

Re: The GC and performance, but not what you expect

2014-05-29 Thread Steven Schveighoffer via Digitalmars-d
On Thu, 29 May 2014 16:01:16 -0400, Sean Kelly wrote: On Thursday, 29 May 2014 at 19:00:24 UTC, Walter Bright wrote: If it's single threaded, and the single thread is doing the collecting, who is starting up a new thread? class Foo { ~this() { auto t = new Thread({ auto

Re: The GC and performance, but not what you expect

2014-05-29 Thread Sean Kelly via Digitalmars-d
On Thursday, 29 May 2014 at 19:00:24 UTC, Walter Bright wrote: If it's single threaded, and the single thread is doing the collecting, who is starting up a new thread? class Foo { ~this() { auto t = new Thread({ auto a = new char[100]; }); t.start(); } }

Re: The GC and performance, but not what you expect

2014-05-29 Thread Sean Kelly via Digitalmars-d
On Thursday, 29 May 2014 at 10:09:18 UTC, Atila Neves wrote: The GC is preventing me from beating Java, but not because of collections. It's the locking it does to allocate instead! I don't know about the rest of you but I definitely didn't see that one coming. This is sadly unavoidable with

Re: The GC and performance, but not what you expect

2014-05-29 Thread Steven Schveighoffer via Digitalmars-d
On Thu, 29 May 2014 14:42:19 -0400, Ali Çehreli wrote: On 05/29/2014 06:06 AM, Steven Schveighoffer wrote: > The explanation in the comment is that finalizers may spawn a new > thread. See > https://github.com/D-Programming-Language/druntime/blob/master/src/gc/gc.d#L410 The strange thin

Re: The GC and performance, but not what you expect

2014-05-29 Thread Walter Bright via Digitalmars-d
On 5/29/2014 4:01 AM, Robert Schadek via Digitalmars-d wrote: On 05/29/2014 12:41 PM, Jacob Carlborg via Digitalmars-d wrote: On 2014-05-29 12:09, Atila Neves wrote: The GC is preventing me from beating Java, but not because of collections. It's the locking it does to allocate instead! I don't

Re: The GC and performance, but not what you expect

2014-05-29 Thread Ali Çehreli via Digitalmars-d
On 05/29/2014 06:06 AM, Steven Schveighoffer wrote: > The explanation in the comment is that finalizers may spawn a new > thread. See > https://github.com/D-Programming-Language/druntime/blob/master/src/gc/gc.d#L410 The strange thing is, finalizers may not even be executed. :-/ I would simp

Re: The GC and performance, but not what you expect

2014-05-29 Thread Brian Schott via Digitalmars-d
On Thursday, 29 May 2014 at 10:09:18 UTC, Atila Neves wrote: If you're profiling binaries on Linux, this thing is a must have and I have no idea how I'd never heard about it before. On the topic of perf, I found a stupid trick the other day and wrote it down on the wiki: http://wiki.dlang.org/

Re: The GC and performance, but not what you expect

2014-05-29 Thread Steven Schveighoffer via Digitalmars-d
On Thu, 29 May 2014 06:09:17 -0400, Atila Neves wrote: The GC is preventing me from beating Java, but not because of collections. It's the locking it does to allocate instead! I don't know about the rest of you but I definitely didn't see that one coming. Just checked. It used to be a lon

Re: The GC and performance, but not what you expect

2014-05-29 Thread safety0ff via Digitalmars-d
On Thursday, 29 May 2014 at 10:09:18 UTC, Atila Neves wrote: The GC is preventing me from beating Java, but not because of collections. It's the locking it does to allocate instead! I don't know about the rest of you but I definitely didn't see that one coming. I would have seen it coming if

Re: The GC and performance, but not what you expect

2014-05-29 Thread Jacob Carlborg via Digitalmars-d
On 2014-05-29 14:18, Shammah Chancellor wrote: I was under the impression that the D gc does move objects around on the heap and then update the pointers. It does not. It would require barriers (read or write, don't remember which) and my people here are against that. -- /Jacob Carlborg

Re: The GC and performance, but not what you expect

2014-05-29 Thread Jacob Carlborg via Digitalmars-d
On 2014-05-29 13:01, Robert Schadek via Digitalmars-d wrote: properly, but collections is not atomar and you have to prevent threads that are created during collection from collecting. so you still need to lock The GC already stops the world, can a thread then be created during a collection?

Re: The GC and performance, but not what you expect

2014-05-29 Thread Shammah Chancellor via Digitalmars-d
On 2014-05-29 11:03:11 +, Wanderer said: It will be hard to beat Java's garbage collector. They use generational, sequential memory approach, while D's memory manager (correct me if I'm wrong) uses more common approach and has to deal with possible memory fragmentation. Allocating a new ob

Re: The GC and performance, but not what you expect

2014-05-29 Thread Wanderer via Digitalmars-d
It will be hard to beat Java's garbage collector. They use generational, sequential memory approach, while D's memory manager (correct me if I'm wrong) uses more common approach and has to deal with possible memory fragmentation. Allocating a new object is extremely cheap in Java, literally a f

Re: The GC and performance, but not what you expect

2014-05-29 Thread Robert Schadek via Digitalmars-d
On 05/29/2014 12:41 PM, Jacob Carlborg via Digitalmars-d wrote: > On 2014-05-29 12:09, Atila Neves wrote: > >> The GC is preventing me from beating Java, but not because of >> collections. It's the locking it does to allocate instead! I >> don't know about the rest of you but I definitely didn't se

Re: The GC and performance, but not what you expect

2014-05-29 Thread Jacob Carlborg via Digitalmars-d
On 2014-05-29 12:09, Atila Neves wrote: The GC is preventing me from beating Java, but not because of collections. It's the locking it does to allocate instead! I don't know about the rest of you but I definitely didn't see that one coming. Doesn't the runtime know how many threads currently e

The GC and performance, but not what you expect

2014-05-29 Thread Atila Neves via Digitalmars-d
So after the last day of DConf was over, back at the Aloft Brian Schott taught me and Jonathan Crapuchettes about perf (https://perf.wiki.kernel.org/index.php/Main_Page). If you're profiling binaries on Linux, this thing is a must have and I have no idea how I'd never heard about it before. Anywa