On 9/15/2018 6:31 AM, Duy Nguyen wrote:
On Wed, Sep 12, 2018 at 6:18 PM Ben Peart <benpe...@microsoft.com> wrote:
This patch helps address the CPU cost of loading the index by creating
multiple threads to divide the work of loading and converting the cache
entries across all available CPU cores.
It accomplishes this by having the primary thread loop across the index file
tracking the offset and (for V4 indexes) expanding the name. It creates a
thread to process each block of entries as it comes to them.
I used p0002-read-cache.sh to generate some performance data:
Test w/100,000 files Baseline Parallel entries
---------------------------------------------------------------------------
read_cache/discard_cache 1000 times 14.08(0.01+0.10) 9.72(0.03+0.06) -31.0%
Test w/1,000,000 files Baseline Parallel entries
------------------------------------------------------------------------------
read_cache/discard_cache 1000 times 202.95(0.01+0.07) 154.14(0.03+0.06) -24.1%
The numbers here and the previous patch to load extensions in parallel
are exactly the same. What do these numbers mean? With both changes?
It means I messed up when creating my commit message for the extension
patch and copy/pasted the wrong numbers. Yes, these numbers are with
both changes (the correct numbers for the extension only are not as good).
