Supercedes: 20220812180806.2128593-1-richard.hender...@linaro.org ("accel/tcg: minimize tlb lookups during translate + user-only PROT_EXEC fixes")
A few changes to the PROT_EXEC work that I posted last week, and then continuing to the main event. My initial goal was to reduce the overhead of TB flushing, which Alex Bennee identified as a significant issue with respect to booting AArch64 kernels under avocado. Our initial guess was that we need a more efficient data structure for walking TBs associated with a physical page. While I was looking at some of those numbers, I noted that we were seeing up to 16000 TBs attached to a single page, which is well more than I expected to see, and means that a new data structure isn't going to help as much as simply reducing the number of translations. It turns out the retranslation is due to the guest kernel's userland address space randomization. Each process gets e.g. libc mapped to a different virtual address, which caused a new translation. This, then, introduces some infrastructure for writing "pc-relative" translation blocks, in which the guest pc is treated as a variable just like any other guest cpu register. The hashing for these TBs are adjusted to compare the physical address. The target/arm backend is adjusted to use the new feature. This does result in a significant reduction in translation. From the BootLinuxAarch64.test_virt_tcg_gicv2 test, at the login prompt: Before: gen code size 160684739/1073736704 TB count 289808 TB flush count 1 TB invalidate count 235143 After: gen code size 277992547/1073736704 TB count 503882 TB flush count 0 TB invalidate count 69282 Before TARGET_TB_PCREL, we generate approximately 1.1GB of TBs (overflow 1GB, flush, and fill 153MB again). Afterward, we only generate 265MB of TBs. Surprisingly, this does not affect wall-clock times nearly as much as I would have expected: before after change BootLinuxAarch64.test_virt_tcg_gicv2: 97.35 85.11 -12% BootLinuxAarch64.test_virt_tcg_gicv3: 102.75 96.87 -5% Change in profile, top 10 entries before, matched up with after: before after 9.01% qemu-system-aar [.] helper_lookup_tb_ptr 10.67% 4.92% qemu-system-aar [.] qht_lookup_custom 5.06% 4.79% qemu-system-aar [.] get_phys_addr_lpae 5.24% 2.57% qemu-system-aar [.] address_space_ldq_le 2.77% 2.33% qemu-system-aar [.] liveness_pass_1 0.60% 2.24% qemu-system-aar [.] cpu_get_tb_cpu_state 2.58% 1.76% qemu-system-aar [.] address_space_translate_internal 1.75% 1.71% qemu-system-aar [.] tb_lookup_cmp 1.92% 1.65% qemu-system-aar [.] tcg_gen_code 0.44% 1.64% qemu-system-aar [.] do_tb_phys_invalidate 0.09% r~ Ilya Leoshkevich (1): accel/tcg: Introduce is_same_page() Richard Henderson (32): linux-user/arm: Mark the commpage executable linux-user/hppa: Allocate page zero as a commpage linux-user/x86_64: Allocate vsyscall page as a commpage linux-user: Honor PT_GNU_STACK tests/tcg/i386: Move smc_code2 to an executable section accel/tcg: Remove PageDesc code_bitmap accel/tcg: Use bool for page_find_alloc accel/tcg: Make tb_htable_lookup static accel/tcg: Move qemu_ram_addr_from_host_nofail to physmem.c accel/tcg: Properly implement get_page_addr_code for user-only accel/tcg: Use probe_access_internal for softmmu get_page_addr_code_hostp accel/tcg: Add nofault parameter to get_page_addr_code_hostp accel/tcg: Unlock mmap_lock after longjmp accel/tcg: Raise PROT_EXEC exception early accel/tcg: Remove translator_ldsw accel/tcg: Add pc and host_pc params to gen_intermediate_code accel/tcg: Add fast path for translator_ld* accel/tcg: Use DisasContextBase in plugin_gen_tb_start accel/tcg: Do not align tb->page_addr[0] include/hw/core: Create struct CPUJumpCache accel/tcg: Introduce tb_pc and tb_pc_log accel/tcg: Introduce TARGET_TB_PCREL accel/tcg: Split log_cpu_exec into inline and slow path target/arm: Introduce curr_insn_len target/arm: Change gen_goto_tb to work on displacements target/arm: Change gen_*set_pc_im to gen_*update_pc target/arm: Change gen_exception_insn* to work on displacements target/arm: Change gen_exception_internal to work on displacements target/arm: Change gen_jmp* to work on displacements target/arm: Introduce gen_pc_plus_diff for aarch64 target/arm: Introduce gen_pc_plus_diff for aarch32 target/arm: Enable TARGET_TB_PCREL include/elf.h | 1 + include/exec/cpu-common.h | 1 + include/exec/cpu-defs.h | 3 + include/exec/exec-all.h | 138 +++++++------- include/exec/plugin-gen.h | 7 +- include/exec/translator.h | 85 +++++++-- include/hw/core/cpu.h | 9 +- linux-user/arm/target_cpu.h | 4 +- linux-user/qemu.h | 1 + target/arm/cpu-param.h | 2 + target/arm/translate-a32.h | 2 +- target/arm/translate.h | 21 ++- accel/tcg/cpu-exec.c | 222 +++++++++++++--------- accel/tcg/cputlb.c | 98 +++------- accel/tcg/plugin-gen.c | 23 +-- accel/tcg/translate-all.c | 197 +++++++------------- accel/tcg/translator.c | 122 +++++++++--- accel/tcg/user-exec.c | 15 ++ linux-user/elfload.c | 81 +++++++- softmmu/physmem.c | 12 ++ target/alpha/translate.c | 5 +- target/arm/cpu.c | 23 +-- target/arm/translate-a64.c | 174 ++++++++++------- target/arm/translate-m-nocp.c | 6 +- target/arm/translate-mve.c | 2 +- target/arm/translate-vfp.c | 10 +- target/arm/translate.c | 237 +++++++++++++++--------- target/avr/cpu.c | 2 +- target/avr/translate.c | 5 +- target/cris/translate.c | 5 +- target/hexagon/cpu.c | 2 +- target/hexagon/translate.c | 6 +- target/hppa/cpu.c | 4 +- target/hppa/translate.c | 5 +- target/i386/tcg/tcg-cpu.c | 2 +- target/i386/tcg/translate.c | 7 +- target/loongarch/cpu.c | 2 +- target/loongarch/translate.c | 6 +- target/m68k/translate.c | 5 +- target/microblaze/cpu.c | 2 +- target/microblaze/translate.c | 5 +- target/mips/tcg/exception.c | 2 +- target/mips/tcg/sysemu/special_helper.c | 2 +- target/mips/tcg/translate.c | 5 +- target/nios2/translate.c | 5 +- target/openrisc/cpu.c | 2 +- target/openrisc/translate.c | 6 +- target/ppc/translate.c | 5 +- target/riscv/cpu.c | 4 +- target/riscv/translate.c | 5 +- target/rx/cpu.c | 2 +- target/rx/translate.c | 5 +- target/s390x/tcg/translate.c | 5 +- target/sh4/cpu.c | 4 +- target/sh4/translate.c | 5 +- target/sparc/cpu.c | 2 +- target/sparc/translate.c | 5 +- target/tricore/cpu.c | 2 +- target/tricore/translate.c | 6 +- target/xtensa/translate.c | 6 +- tcg/tcg.c | 6 +- tests/tcg/i386/test-i386.c | 2 +- 62 files changed, 979 insertions(+), 666 deletions(-) -- 2.34.1