Marcelo Tosatti wrote: > Hi, > > The following is an improvement on top of an earlier patch by Izik. It > increases pagefault scalability for SMP guests by allowing concurrent > guest walking, allocation and instruction emulation on the fault path. > > The test being used is pft, which starts a number of threads > allocating and writing malloc()'ed memory. pft.c can be found at > http://lkml.org/lkml/2004/8/15/58 > > The script being used is: > > bytes=$((400*1024*1024)) > ./pft -t -b$bytes -r10 -f1 > ./pft -b$bytes -r10 -f2 > ./pft -b$bytes -r10 -f3 > ./pft -b$bytes -r10 -f4 > ./pft -b$bytes -r10 -f8 > > This is a 4-way guest. > > One important detail from the results is that there is no difference > for the two threads case, but beyond that we see a clear improvement. > follow_page() is showing up high in profiling, so I believe this > is partly due to the fact that it does follow_page() twice while > holding the lock, once in mmu_set_spte() from walk_addr() and again in > mmu_set_spte() in fetch() - I'm looking into removing those duplicated > calls for the same gfn. > > The patch still lacks the copy_from_user_inatomic() change in > prefetch_page() to avoid a potential sleep in case the page is swapped > out. > > Another issue is that now fetch() will re-read the pte's after > instantiating a shadow page, but in theory they could be swapped out. I > believe that is safe since walk_addr() just touched the pte's bringing > them in from swap. >
We need to convert that to kvm_read_guest_atomic() to avoid even that theoretical race. If the read fails, we can simply return and let the guest retry the faulting instruction. > Switching the shadow lock to a spinlock also reduces the pagefault > latency in > 2 CPU's case to 50%. > > > stock KVM: > Gb Rep Threads User System Wall flt/cpu/s fault/wsec > 0 10 1 0.292s 5.744s 6.006s169638.179 168754.614 > 0 10 2 0.476s 7.340s 4.032s131005.143 236904.678 > 0 10 3 0.620s 10.972s 4.058s 88331.275 223355.103 > 0 10 4 0.588s 14.784s 4.062s 66610.464 221352.545 > 0 10 8 0.616s 15.100s 4.074s 65152.462 215861.148 > > KVM + kvm-scale.patch: > Gb Rep Threads User System Wall flt/cpu/s fault/wsec > 0 10 1 0.328s 6.668s 7.002s146360.233 145746.022 > 0 10 2 0.472s 7.708s 4.036s125175.570 234513.121 > 0 10 3 0.500s 8.480s 3.038s114024.057 302227.769 > 0 10 4 0.508s 9.156s 3.012s105953.654 328156.474 > 0 10 8 1.144s 9.260s 3.028s 98417.544 311395.745 > > > Very impressive results (and patch). -- error compiling committee.c: too many arguments to function ------------------------------------------------------------------------- SF.Net email is sponsored by: Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace _______________________________________________ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
