Re: Easy huge GC optimizations

2014-05-23 Thread Rainer Schuetze via Digitalmars-d
On 22.05.2014 21:04, Etienne wrote: On 2014-05-22 2:12 PM, Rainer Schuetze wrote: NO_INTERIOR is currently only used for the hash array used by associative arrays. It is a bit dangerous to use as any pointer,slice or register still operating on the array is ignored, so collecting it might

Re: Easy huge GC optimizations

2014-05-23 Thread Chris via Digitalmars-d
On Friday, 23 May 2014 at 06:17:43 UTC, Rainer Schuetze wrote: On 22.05.2014 21:04, Etienne wrote: On 2014-05-22 2:12 PM, Rainer Schuetze wrote: NO_INTERIOR is currently only used for the hash array used by associative arrays. It is a bit dangerous to use as any pointer,slice or register

Re: Easy huge GC optimizations

2014-05-23 Thread John Colvin via Digitalmars-d
On Friday, 23 May 2014 at 13:43:53 UTC, Chris wrote: On Friday, 23 May 2014 at 06:17:43 UTC, Rainer Schuetze wrote: On 22.05.2014 21:04, Etienne wrote: On 2014-05-22 2:12 PM, Rainer Schuetze wrote: NO_INTERIOR is currently only used for the hash array used by associative arrays. It is a

Re: Easy huge GC optimizations

2014-05-23 Thread Etienne via Digitalmars-d
On 2014-05-23 2:17 AM, Rainer Schuetze wrote: On 22.05.2014 21:04, Etienne wrote: On 2014-05-22 2:12 PM, Rainer Schuetze wrote: NO_INTERIOR is currently only used for the hash array used by associative arrays. It is a bit dangerous to use as any pointer,slice or register still operating on

Re: Easy huge GC optimizations

2014-05-23 Thread Etienne via Digitalmars-d
On 2014-05-23 11:41 AM, John Colvin wrote: Bear in mind here that most code goes though a whole bunch of machine learning algorithms in the CPU itself. Like it or not, it has proved extremely successful. I'm happy you're here to say this. Machine learning is the future of algorithms

Re: Easy huge GC optimizations

2014-05-23 Thread Etienne via Digitalmars-d
On 2014-05-23 12:33 PM, Etienne wrote: It only skips the inner search of the pool, like marking it NO_SCAN if a sample of the pointers that pointed to it are still alive. Sorry that's not true. It's like marking it NO_INTERIOR while it being still SCAN. By default, all the pages would be

Re: Easy huge GC optimizations

2014-05-23 Thread Chris via Digitalmars-d
On Friday, 23 May 2014 at 15:41:39 UTC, John Colvin wrote: On Friday, 23 May 2014 at 13:43:53 UTC, Chris wrote: On Friday, 23 May 2014 at 06:17:43 UTC, Rainer Schuetze wrote: On 22.05.2014 21:04, Etienne wrote: On 2014-05-22 2:12 PM, Rainer Schuetze wrote: NO_INTERIOR is currently only

Re: Easy huge GC optimizations

2014-05-23 Thread Etienne via Digitalmars-d
On 2014-05-23 1:29 PM, Chris wrote: I know that CPU's do a good bit of guessing. But that's not the same thing. If they err, they make up for it (Ooops, it's not in the cache! Will get it from HD, just a nanosec!). If the GC errs, how do you make up for it? Please educate me. If the GC errs,

Re: Easy huge GC optimizations

2014-05-23 Thread Chris via Digitalmars-d
On Friday, 23 May 2014 at 17:38:27 UTC, Etienne wrote: On 2014-05-23 1:29 PM, Chris wrote: I know that CPU's do a good bit of guessing. But that's not the same thing. If they err, they make up for it (Ooops, it's not in the cache! Will get it from HD, just a nanosec!). If the GC errs, how do

Re: Easy huge GC optimizations

2014-05-23 Thread Rainer Schuetze via Digitalmars-d
On 23.05.2014 18:41, Etienne wrote: On 2014-05-23 12:33 PM, Etienne wrote: It only skips the inner search of the pool, like marking it NO_SCAN if a sample of the pointers that pointed to it are still alive. Sorry that's not true. It's like marking it NO_INTERIOR while it being still SCAN.

Re: Easy huge GC optimizations

2014-05-23 Thread Etienne via Digitalmars-d
On 2014-05-23 2:08 PM, Chris wrote: Fair enough. But what about programs that allocate a lot and run for ages (a server app for example)? A server app? Couldn't have asked me for a better example. You can see my native events fork here (I'm working on replacing libevent):

Re: Easy huge GC optimizations

2014-05-23 Thread Etienne via Digitalmars-d
On 2014-05-23 2:14 PM, Rainer Schuetze wrote: BTW: How do you detect the sample pointers are alive? Or do you mean just the roots? You store a void** and the original value, dereference it to see if it's the same value as the original. Loop through 20 of those if you have 500, and you update

Re: Easy huge GC optimizations

2014-05-23 Thread Rainer Schuetze via Digitalmars-d
On 23.05.2014 20:52, Etienne wrote: On 2014-05-23 2:14 PM, Rainer Schuetze wrote: BTW: How do you detect the sample pointers are alive? Or do you mean just the roots? You store a void** and the original value, dereference it to see if it's the same value as the original. Loop through 20 of

Re: Easy huge GC optimizations

2014-05-23 Thread via Digitalmars-d
On Friday, 23 May 2014 at 15:41:39 UTC, John Colvin wrote: Bear in mind here that most code goes though a whole bunch of machine learning algorithms in the CPU itself. Like it or not, it has proved extremely successful. What kind of machine learning? Branch prediction?

Re: Easy huge GC optimizations

2014-05-23 Thread Etienne via Digitalmars-d
On 2014-05-23 3:08 PM, Rainer Schuetze wrote: AFAICT your test case does not measure garbage collection, but manual memory management using the GC as the memory manager. delete/free are not meant to be called by user code as these are unsafe operations. Yes, you're right, the test without the

Easy huge GC optimizations

2014-05-22 Thread Etienne via Digitalmars-d
I was thinking of how the GC could be optimized further and came across some sweet flags that are barely used throughout Phobos in favor of a witch-hunting against the GC: GC.BlkAttr.NO_SCAN GC.BlkAttr.NO_INTERIOR When using the NO_SCAN attribute with GC.setAttr(p, NO_SCAN), you're basically

Re: Easy huge GC optimizations

2014-05-22 Thread Rainer Schuetze via Digitalmars-d
On 22.05.2014 18:42, Etienne wrote: I was thinking of how the GC could be optimized further and came across some sweet flags that are barely used throughout Phobos in favor of a witch-hunting against the GC: GC.BlkAttr.NO_SCAN GC.BlkAttr.NO_INTERIOR When using the NO_SCAN attribute with

Re: Easy huge GC optimizations

2014-05-22 Thread Steven Schveighoffer via Digitalmars-d
On Thu, 22 May 2014 14:12:38 -0400, Rainer Schuetze r.sagita...@gmx.de wrote: I'm not sure where allocating a struct ends up, so maybe there is some potential there. It's done similarly to arrays. The function is here:

Re: Easy huge GC optimizations

2014-05-22 Thread Etienne via Digitalmars-d
On 2014-05-22 2:12 PM, Rainer Schuetze wrote: NO_INTERIOR is currently only used for the hash array used by associative arrays. It is a bit dangerous to use as any pointer,slice or register still operating on the array is ignored, so collecting it might corrupt your memory. That's quite a