Mike Lambert wrote: > Should this be a configure.pl-determined constant? Should we hardcode it > to sizeof(void*)? Is this behavior guaranteed by the C spec? Can we > assume it across all platforms even if it is not guaranteed?
I would be in favour of making it configuration-determined, just in case there is some platform that likes a packed stack - but we can probably get away with a fairly simple test, as I think we're unlikely to find many exceptions. > Might I ask what your motivation was for the header linked list? The original aim of the first version of grey was to find further uses for the interpreter cycle count. As mentioned before, only parts of that code have been implemented in the current version of grey. I think perhaps a summary of the first version of grey may be in order (that is the one that does 5000 lives in 60 seconds on my box; still by far the fastest) 1) Decouple dead-object detection from dead-object freeing. In my profiling, I discovered that the process of actually freeing the dead objects was a fairly lengthy part of the DOD process, being proportional to the total number of objects allocated. Although the DOD scan has to go through everything, there is no need to free objects in a pool that is not under pressure; instead that can be delayed until a specific pool needs more resources. By changing the cycle count field in each header from its original meaning of 'birthdate' to 'last live' and setting it to the current interpreter cycle when marking a header as live, that information remains available for later use - so when a resource pool's free list is empty, we can scan for resources that have a 'last live' field before the cycle count when the last dod scan was performed. Of course, a full dod scan has to be forced whenever the counter wraps. My next thought was that this could be integrated into memory pool compaction - when a pool is compacted, we could simply ignore dead headers, but it would be better to actually put them onto the free list. At this point I came up with the idea of allocating pool memory in pages instead of a single large chunk. The long time goal was to aim for a pseduo-generational collection method, the short time goal was to reduce the peak memory usage by not requiring having double the pool memory allocated during the compaction run. This only works if compact_pool can not only allocate memory in pages, but also free each page as it has been processed. This is only possible if compact_pool runs through the pool, rather than through the headers - but we had no link from pool to header - thus I invented the linked list of buffers by pool page. Then the compaction process becomes: allocate one new page of memory at the start, loop through each 'old' page, allocating new pages as required; free each 'old' page once completed. The allocation and freeing of memory pages was then changed so that old pages were not returned to the system, but stored in a free page pool, from where they were reallocated. This has another benefit in reducing the interaction with the system memory allocator. Initially these lists were simply connected to each memory page, without specific sequencing - then I realised that by sorting them in bufstart sequence, I could implement a different version of COW, with no further impact on the String header (of course, it already has the cycle count, next/prev header pointers and the pointer to the header pool - so it is way more expensive than strstart). As you pointed out, this method of COW allows a fairly effective means of decowing; as well as the example you mention, it can also handle the more complicated case of: set S0, some_large_data_file_contents substr S1, S0, 0, 1 #get first character as COW substr S2, S0, 214761, 1 #get last character as COW set S0, "" sweep collect [Aside: I would like that 'set S0, ""' to be at least 'clear S0' or better 'free S0', where the former would zero the register, and the latter would return the header to the free pool immediately] The page-by-page compaction process has the ability to be suspended at the end of any page; this could be used in either a round-robin type approach, starting from the oldest page, and stopping when we achieved a target amount of free space; or, if we suspect that a lot of pool usage is short life span, we could always start with the most recently filled page and would back. I have not yet tried any of these approaches - I always just did a full compaction pass. Hopefully this explains how the linked list approach came about. Remember, grey has always been experimental - the above does not mention at least a dozen ideas that caused performance to deteriorate and were discarded. Only changes that improved performance (or had a very small detrimental effect but were required for subsequent changes) survived. -- Peter Gibbs EmKel Systems
