[Issue 3463] Integrate Precise Heap Scanning Into the GC
http://d.puremagic.com/issues/show_bug.cgi?id=3463 Jacob Carlborg changed: What|Removed |Added CC||d...@me.com --- Comment #126 from Jacob Carlborg 2011-04-15 00:27:53 PDT --- Why not just add an additional garbage collector with this new implementation and leave the old one as it is and then developers can choose which one to use at link time. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
[Issue 3463] Integrate Precise Heap Scanning Into the GC
http://d.puremagic.com/issues/show_bug.cgi?id=3463
Sean Cavanaugh changed:
What|Removed |Added
CC||worksonmymach...@gmail.com
--- Comment #127 from Sean Cavanaugh 2011-04-15
01:40:35 PDT ---
(In reply to comment #120)
> (In reply to comment #118)
> > True, and it works tolerably well. To do a moving gc, however, you need more
> > precise information.
>
> I don't want a moving GC. I want a fast GC.
>
> ("I" in this context means D users with the same requirements, mainly video
> game developers.)
>
> I understand the advantages of a moving GC - heap compaction allowing for an
> overall smaller managed heap etc., but I hope you understand that sacrificing
> speed for these goals is not an unilateral improvement for everyone.
I am a game developer, and this thread is fairly fascinating to me, as memory
management and good support for Intel SSE2(and AVX) or PowerPC VMX are two of
the biggest issues to me when considering alternative languages or the question
of 'will this language be suitable in the future'. The SSE problem seems
workable with extern C'd C++ DLLs code to handle the heavy math, which leaves
the GC as a big 'what does this mean' when evaluating the landscape.
The reality is a lot of game engines allocate a surprising amount of memory at
run time. The speed of malloc itself is rarely an issue as most searches take
reasonably similar amount of time. The real problems with heavy use of malloc
are thread lock contention in the allocator, and fragmentation. Fragmentation
causes two problems: large allocation failures when memory is low (say 1 MB
allocation when 30 MB is 'free'), and virtual pages are unable to be reclaimed
due to a stray allocation or two within the page.
Lock contention is solved by making separate heaps. Fragmentation is fought
also fought by separating the heaps, but organizing the allocations coherently
either time-wise or by allocation type where like sized objects pooled into a
special pool for objects of that size. As a bonus fixed size object pools have
const time for allocation, except when the pool has to grow, but we try real
hard to pre-size these to the worst case values. On my last project we had
about 8 dlmalloc based heaps and 15 fixed sized allocator pools, to solve these
problems.
I would greatly prefer a GC to compact the heap to keep the peak memory down,
because in embeded (console) environments memory is a constant but time is
fungible. VM might be available on the environments, but it isn't going to be
backed by disk. Instead the idea of the VM is that it is a tool to fight
fragmentation of the underlying physical pages, and to help you get contiguous
space to work with. There is also pressure to use larger (64k, 1MB, 4MB pages)
pages to keep the TLB lookups fast, which hurts even more with fragmentation.
Tiny allocations holding onto these big pages prevents them from being
reclaimed, which makes getting those allocations moved somewhere better pretty
important.
Now the good news is a huge amount of resources in a game do not need to be
allocated into a garbage collected space. For the most part anything you send
to the GPU data is far better off being written into its memory system and left
alone. Physics data and Audio data have similar behaviors for the most part
and can be allocated through malloc or aligned forms of malloc (for SSE
friendlies).
So from a game's developers point of I need to know when the GC will run either
by configuration or by manually driving it. Both allow me to run a frame with
most of the AI and physics disabled to give more of the time to the collector.
A panic execution GC pass that I wasn't expecting is acceptable, provided I get
notified of it, as I would expect this to be an indicator memory is getting
tight to the point an Out of Memory is imminent. A panic GC is a QA problem as
we can tell them where and how often the are occurring and they can in turn
tell the designers making the art/levels that they need to start trimming the
memory usage a bit.
Ideally the GC would be able to run in less time than a single frame (say
10-15ms for a 30fps game). Taking away some amount of time every frame is also
acceptable. For example spending 1ms of every frame to do 1ms worth of data
movement or analysis for compacting would be a reasonable thing to allow, even
if it was in addition to the multi-millisecond spikes at some time interval (30
frames, 30 seconds whatever). Making the whole thing friendly to having lots
of CPU cores wouldn't hurt either.
--
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
[Issue 3463] Integrate Precise Heap Scanning Into the GC
http://d.puremagic.com/issues/show_bug.cgi?id=3463 --- Comment #128 from Vladimir 2011-04-15 02:14:17 PDT --- (In reply to comment #127) Thank you for your insight! > So from a game's developers point of I need to know when the GC will run > either > by configuration or by manually driving it. You can disable automatic garbage collection and manually invoke a collection right now. > Both allow me to run a frame with > most of the AI and physics disabled to give more of the time to the > collector. This won't work for multiplayer games where the game state must be kept in sync on all sides. > Ideally the GC would be able to run in less time than a single frame (say > 10-15ms for a 30fps game). Moving GCs are bound to be slower than the current one, but heap compaction probably doesn't need to happen as often as a simple GC run to reclaim memory. > Taking away some amount of time every frame is also acceptable. > For example spending 1ms of every frame to do 1ms worth of data > movement or analysis for compacting would be a reasonable thing to allow, The current GC doesn't support incremental runs. Jeremie Pelletier has written a garbage collector some time ago which can do a shallow scan and only collect objects with no immediate references: http://pastebin.com/f7a3b4c4a -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
[Issue 5465] AA.keys with char keys
http://d.puremagic.com/issues/show_bug.cgi?id=5465 Pedro Rodrigues changed: What|Removed |Added CC||pdfrodrig...@gmail.com --- Comment #2 from Pedro Rodrigues 2011-04-15 06:31:27 PDT --- I've tested with version 2.052 and obtained the following output: th xpliaems So it's probably fixed now. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
[Issue 5438] Thread.sleep doesn't sleep
http://d.puremagic.com/issues/show_bug.cgi?id=5438 Pedro Rodrigues changed: What|Removed |Added CC||pdfrodrig...@gmail.com --- Comment #1 from Pedro Rodrigues 2011-04-15 07:09:38 PDT --- According to the documentation of Thread.sleep it receives as argument "The minimum duration the calling thread should be suspended, in 100 nanosecond intervals.". That means that calling with value 10_000 will sleep for 1 millisecond, which is too small to be noticeable. I tried with 100_000_000 (10 seconds) in DMD v2.052 and it seemed to be working as expected. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
[Issue 5438] Thread.sleep doesn't sleep
http://d.puremagic.com/issues/show_bug.cgi?id=5438 Steven Schveighoffer changed: What|Removed |Added Status|NEW |RESOLVED CC||schvei...@yahoo.com Resolution||WORKSFORME --- Comment #2 from Steven Schveighoffer 2011-04-15 07:43:16 PDT --- This bug needs more details, as to exactly what is expected to happen. Not having any details, I can't tell what exactly you expect should happen. I'm closing this, if you feel it was closed in error, please reopen with more details. Specifically, you should outline what the behavior currently is, and what behavior you expect. Also, any evidence showing the erroneous behavior would be appreciated. As a tip, Thread.sleep now accepts a core.time.Duration struct, which can be initialized with units other than hnsecs. e.g. Thread.sleep(dur!"seconds"(5)); // sleep for 5 seconds -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
[Issue 5465] AA.keys with char keys
http://d.puremagic.com/issues/show_bug.cgi?id=5465 bearophile_h...@eml.cc changed: What|Removed |Added Status|NEW |RESOLVED Resolution||FIXED -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
[Issue 1366] Inconsistent __traits usage
http://d.puremagic.com/issues/show_bug.cgi?id=1366 Don changed: What|Removed |Added Status|NEW |RESOLVED Resolution||FIXED --- Comment #2 from Don 2011-04-15 21:19:08 PDT --- This is one of at least 8 bugs fixed by this commit: https://github.com/donc/dmd/commit/fc67046cf1e66182d959309fb15ef9e2d4c266b9 -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
[Issue 2841] char[] incorrectly accepted as a template value argument in D2
http://d.puremagic.com/issues/show_bug.cgi?id=2841 Don changed: What|Removed |Added Status|NEW |RESOLVED Resolution||FIXED --- Comment #4 from Don 2011-04-15 21:22:04 PDT --- The example in comment 2 is fixed in this commit. https://github.com/donc/dmd/commit/fc67046cf1e66182d959309fb15ef9e2d4c266b9 -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
[Issue 1386] "string expected" when using allMembers-element in __traits(getMember, ...)
http://d.puremagic.com/issues/show_bug.cgi?id=1386 Don changed: What|Removed |Added CC||clugd...@yahoo.com.au --- Comment #9 from Don 2011-04-15 21:33:24 PDT --- The error message is: test2.d(22): Error: no property '__T4TempTkZ' for type 'test2.Asdf' Some kind of junk tuple members are being included in allMembers. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
[Issue 5678] new enum struct re-allocated at compile time
http://d.puremagic.com/issues/show_bug.cgi?id=5678 Don changed: What|Removed |Added Status|NEW |RESOLVED CC||clugd...@yahoo.com.au Resolution||FIXED --- Comment #1 from Don 2011-04-15 22:34:17 PDT --- Fixed https://github.com/D-Programming-Language/dmd/commit/47d46bf1364c63df5bb01406db4098c771d701dd -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
