So now pgen.d works (without simpledisplay version an reusing trueimage). The simpledisplay version still has problems, but I'll try (manually (de)allocating)/(reusing) the data
2013/5/14 Adam D. Ruppe <destructiona...@gmail.com> > On Monday, 13 May 2013 at 21:28:34 UTC, maarten van damme wrote: > >> Adam checks this thread so he'll probably read the errors about >> simpledisplay. if not I'll copy paste and put in a bug report on github. >> > > I won't be on my linux box with free time until at least tomorrow > afternoon but i'll check then. I've used simpledisplay on linux before, but > never with a 64 bit app so i'll have to try it. > > > So, do we actually know where the memory problems came from? is anyone >> actually running out of memory too or am I the only one? >> > > I didn't actually get that far since i'm on my slow computer, but my guess > is one of those copies got pointed into. > > The image in your code is something like 8 MB of memory, and is generated > pretty rapidly, as fast as teh cpu could do it in the other thread. > > The old function took that 8MB and made at least two copies of it, once in > my function and once in std.zlib's function. If the usable address space is > 2 GB, any random 32 bit number would have about a 1/128 chance of pointing > into it. > > If you consider the gc was probably scanning at least one copy of that > data for pointers - std.zlib's one was constructed as a void[], so even if > allocator told the gc what type it was (idk if it does or not), void[] has > to be scanned conservatively because it could be an array of pointers. > > And then add in random numbers on the stack, and the odds are pretty good > the gc will get a false positive and then fail to free the memory block. > > > Repeat as it loops through the images, and you've got a pretty big memory > leak adding up. > > > I'm not 100% sure that's what happened, but I think it is pretty likely. > The reason the new function fixes it is because it manages the big copies > manually, freeing them at the end of the function without worrying about > false pointers. >