phaniarnab opened a new pull request #932: URL: https://github.com/apache/systemml/pull/932
This patch contains handling of multiple cache entries pointing to the same data (due to multilevel caching). 1) All the entries with the same values are connected with a linkedlist. Even though they output same data, they have different computation time. 2) Eviction logic marks an entry for deferred spilling/removal if other entries are linked to that. If all the entries in a list are marked for spilling or removal, only then we evict the item. 3) Disk write and read happen only once for all the items connected to a single matrix. This way single read and write restores multiple entries to cache and clears more space respectively. 4) Initial experiments show huge improvements in cache management. Now the cache can store many more entries (this patch fixes duplicate size calculations), need reduced number of disk I/O. These changes overall improve cache hit count. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org