On Mon, Aug 07, 2017 at 19:52:16 -0400, Emilio G. Cota wrote: > This series applies on top of the "multiple TCG contexts" series, v4: > https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg06769.html > > Highlights: > > - First, fix a few typos I encountered while working on this (patches 1-3). > I could send them separately to qemu-trivial if you prefer. > - QHT: use a proper cmp function, instead of just checking pointer values > to determine equality of matches. > - Use a binary search tree for each TCG region. > - Make l1_map lockless by using cmpxchg > - Introduce page locks (for !user-mode), so that tb_lock is not > needed when operating on a page > - Introduce page_collection, to lock a range of pages > - Introduce tb->jmp_lock to protect TB jump lists. > - Remove tb_lock. User-mode uses just mmap_lock and tb->jmp_lock's; > !user-mode uses the same jump locks as well as page locks. > > Performance numbers are in patch 22. We get nice speedups, but I still > see a lot of idling when booting many cores. I suspect it comes from > cross-CPU events (e.g. TLB invalidations), but I need to profile it > better (perf is not good for this; mutrace doesn't quite work). But > anyway that's for another patchset.
The idling is due to BQL contention related to interrupt handling. In the case of ARM, this boils down to the GICv3 code being single-threaded. I don't have time right now to make it multi-threaded, but at least we know where the scalability bottleneck is. BTW if there's interest I can submit the lock profiler to the list. The code is in this branch: https://github.com/cota/qemu/tree/lock-profiler The first commit has sample output: https://github.com/cota/qemu/commit/c5bda634 Also, any feedback on the parent (tb_lock removal) patchset would be appreciated. To make the 2.11 merge easier, I rebased this patchset (as well as the multi-tcg-v4 set it is based on) on top of rth's tcg-generic-15, fixing a good bunch of annoying conflicts. The resulting branch is available at: https://github.com/cota/qemu/tree/tcg-generic-15%2Bmulti-tcg-v4-parallel Thanks, Emilio