Hi all, SCIFIOCellImg with Memory Service > * Development branch: https://github.com/scifio/scifio/tree/cache-imps > * Refactored SCIFIOCells to cache via phantom reference instead of > finalization (done) > * Created MemoryService to manage a new RefHandler class of plugins and > provide configurable memory safeguards (done) > * Taking a test case of re-writing each pixel in a dataset, improved cache > performance from 59.6s write, 57.1s read to 35.2s write, 1.6s read (done) > * Two tests still failing unpredictably due to garbage collection reliance > (I think). As soon as these are fixed, the update SCIFIOCell/MemoryService > functionality will be released in SCIFIO 0.7.4. > * Integration to KNIME started but not finished (easy task, thanks Mark!) > > > Just wanted to let people know that this functionality is now available in the SCIFIO-0.7.4 release <https://github.com/scifio/scifio/releases>.
I ended up focusing the changes to the PhantomReference refactoring and performance improvements. If anyone's interested why, I wrote up an explanation on the original ticket<https://github.com/scifio/scifio/issues/69> . To elaborate on the write/read test I was using as a metric, Christian Dietz provided a ~90MB dataset and made a simple program that iterates over each pixel, writing it to a random value, then iterates over the dataset again reading each pixel's value. With the 0.7.3 cell cache, the writing operation actually took over 100s when limiting the JVM to 50MB ram. Reading the data back took at least 50s. By adjusting when certain operations run, moving a lot of cleanup to separate threads, and avoiding unnecessary disk checks, as of 0.7.4 the same operation now takes ~16s to write and ~2s to read on the same machine. And of course the code is significantly improved, as we're no longer relying on overridden finalize methods or System.gc calls. Anyway, I'm sure there will still be more improvements to make but I think the cell cache was significantly enhanced. Thanks to Christian for providing the motivation + PhantomReference info! - Mark
_______________________________________________ ImageJ-devel mailing list [email protected] http://imagej.net/mailman/listinfo/imagej-devel
