Am Mon, 12 May 2014 09:32:58 +0000 schrieb "Paulo Pinto" <pj...@progtools.org>:
> On Monday, 12 May 2014 at 09:05:39 UTC, John Colvin wrote: > > On Monday, 12 May 2014 at 08:45:56 UTC, Walter Bright wrote: > >> On 5/12/2014 12:12 AM, Manu via Digitalmars-d wrote: > >>> What? You've never offered me a practical solution. > >> > >> I have, you've just rejected them. > >> > >> > >>> What do I do? > >> > >> 1. you can simply do C++ style memory management. > >> shared_ptr<>, etc. > >> > >> 2. you can have the non-pausible code running in a thread that > >> is not registered with the gc, so the gc won't pause it. This > >> requires that this thread not allocate gc memory, but it can > >> use gc memory allocated by other threads, as long as those > >> other threads retain a root to it. > >> > >> 3. D allows you to create and use any memory management scheme > >> you want. You are simply not locked into GC. For example, I > >> rewrote my Empire game into D and it did not do any allocation > >> at all - no GC, not even malloc. I know that you'll need to do > >> allocation, I'm just pointing out that GC allocations and > >> pauses are hardly inevitable. > >> > >> 4. for my part, I have implemented @nogc so you can track down > >> gc usage in code. I have also been working towards refactoring > >> Phobos to eliminate unnecessary GC allocations and provide > >> alternatives that do not allocate GC memory. Unfortunately, > >> these PR's just sit there. > >> > >> 5. you can divide your app into multiple processes that > >> communicate via interprocess communication. One of them > >> pausing will not pause the others. You can even do things like > >> turn off the GC collections in those processes, and when they > >> run out of memory just kill them and restart them. (This is > >> not an absurd idea, I've heard of people doing that > >> effectively.) > >> > >> 6. If you call C++ libs, they won't be allocating memory with > >> the D GC. D code can call C++ code. If you run those C++ libs > >> in separate threads, they won't get paused, either (see (2)). > >> > >> 7. The Warp program I wrote avoids GC pauses by allocating > >> ephemeral memory with malloc/free, and (ironically) only using > >> GC for persistent data structures that should never be free'd. > >> Then, I just turned off GC collections, because they'd never > >> free anything anyway. > >> > >> 8. you can disable and enable collections, and you can cause > >> collections to be run at times when nothing is happening (like > >> when the user has not input anything for a while). > >> > >> > >> The point is, the fact that D has 'new' that allocates GC > >> memory simply does not mean you are obliged to use it. The GC > >> is not going to pause your program if you don't allocate with > >> it. Nor will it ever run a collection at uncontrollable, > >> random, asynchronous times. > > > > The only solutions to the libraries problem that I can see here > > require drastic separation of calls to said libraries from any > > even vaguely time critical code. This is quite restrictive. > > > > Yes, calling badly optimised libraries from a hot loop is a bad > > idea anyway, but the GC changes this from > > > > "well it might take a little more time than usual, but we can > > spare a few nano-seconds and it'll show up easily in the > > profiler" > > > > to > > > > "it might, sometimes, cause the GC to run a full collection on > > our 3.96 / 4.00 GB heap with an associated half-second pause." > > > > And here we go again, "I can't use that library, it's memory > > management scheme is incompatible with my needs, I'll have to > > rewrite it myself..." > > A badly placed malloc() in library code can also trigger OS > virtualization mechanisms and make processes being swapped out to > disk, with the respective overhead in disk access and time spent > on kernel code. > > So it is just not the "we can spare a few nano-seconds". > > -- > Paulo Yes, it could easily extend to a longer wait. I think we all know programs that hang while the system is swapping out. Don't let it get to that! A PC game would typically reduce caches or texture resolutions before running out of RAM. Linux has a threshold of free pages it tries to keep available at any time to satisfy occasional small allocations. http://www.science.unitn.it/~fiorella/guidelinux/tlk/node39.html All-in-all malloc is less likely to cause long pauses. It just allocates and doesn't ask itself if there might be dead memory to salvage to satisfy a request. Time will tell if all "well written" D libraries will be @nogc to move the question of allocations to the user. -- Marco