Splitting Matt's larger patch set:
> 4 tcg: add tcg_gen_goto_jc_{i32,i64,tl}()
>
> Preparation. A second dispatch interface alongside
> tcg_gen_lookup_and_goto_ptr(), which is unchanged. The caller passes
> the destination PC and thereby states that the CPU state is already
> the destination's, which is what an inline lookup needs and what the
> existing interface cannot promise. --enable-debug-tcg checks that
> claim at runtime. Five targets are migrated; every other target and
> every unmigrated call site is untouched.
>
> 5 accel/tcg: add CF_NO_GOTO_JC, set while a breakpoint is present
>
> Preparation. The one thing an inline jump cache probe cannot check is
> breakpoints, and it does not have to: the probe compares cflags, so a
> cflag set while cpu->breakpoints is non-empty keeps such blocks both
> from dispatching inline and from being reached by a block that does.
> Nothing reads it yet.
>
> 6 RFC: tcg: probe the TB jump cache inline instead of calling a helper
>
> 95.8% of those 8.4 billion helper_lookup_tb_ptr() calls hit the jump
> cache. Emit the probe inline (hash, four guarded loads, goto_ptr) and
> call the helper only on a miss. -34.67%, -25.94% wall
Changes for v7:
* All prerequisites merged.
* Use the current TB data for complex comparisons, when possible.
x86_64 sample:
49 3b 5c 24 18 cmpq 0x18(%r12), %rbx
75 25 jne 1f
4d 8b 64 24 10 movq 0x10(%r12), %r12
4d 85 e4 testq %r12, %r12
74 1b je 1f
49 83 7c 24 08 00 cmpq $0, 8(%r12)
75 13 jne 1f
49 8b 44 24 10 movq 0x10(%r12), %rax
48 3b 05 3f fe ff ff cmpq -0x1c1(%rip), %rax # tb->flags
75 05 jne 1f
41 ff 64 24 18 jmpq *0x18(%r12)
1:
aarch64 sample:
a94142b5 ldp x21, x16, [x21, #0x10]
f10002bf cmp x21, #0
fa541200 ccmp x16, x20, #0, ne
54000101 b.ne 1f
a940c6b0 ldp x16, x17, [x21, #8]
f9400eb5 ldr x21, [x21, #0x18]
58fff41e ldr x30, 0xe4c32c000210 # tb->flags
eb1e023f cmp x17, x30
fa400a00 ccmp x16, #0, #0, eq
54000041 b.ne 1f
d61f02a0 br x21
1:
s390x sample:
eb12 c010 0004 lmg %r1, %r2, 0x10(%r12)
ec2d 0012 6064 cgrjlh %r2, %r13, 1f
ec18 000f 007c cgije %r1, 0, 1f
eb02 1008 0004 lmg %r0, %r2, 8(%r1)
ec06 0009 007c cgijlh %r0, 0, 1f
c618 ffff fef6 cgrl %r1, 0x3ff18000310 # tb->flags
a764 0003 jlh 1f
07f2 br %r2
1:
Changes for v6:
* Split target patches from infrastructure patches.
* Use tcg-op.c rewrite infrastructure.
* Add tcg_gen_goto_jc3_*, for the guest to pass all 3 values.
* Fix bugs in target/
- alpha: Missed several system-mode points at which
we cannot use goto_jc (mtpr ps, mtpr fen, halt), and one
where we cannot use goto_jc, but can use goto_jc3 (call_pal).
- loongarch: Only use goto_jc for branches. There are several
other corner cases lurking in the priviledged insns, and
they can't be as important as branches.
- mips: Droped everything. I expect that this could be made
to work using goto_jc3, but all of the tb->flags frobbing
related to delayed branches needs some thought.
- s390x: Only use jc for branches. There are several other
corner cases with SACF and SAM insns. EX emulation requires
special treatment: always use goto_jc3 so that we only look
for TBs with ex_value == 0.
* Use tcg_gen_lea in gen_jc_probe.
* Add goto_jc opcode.
r~
Matt Turner (5):
tcg: add tcg_gen_goto_jc{3}_{i32,i64,tl}
target/loongarch: Use tcg_gen_goto_jc_tl for JIRL
target/ppc: Use tcg_gen_goto_jc_tl
accel/tcg: add CF_NO_GOTO_JC, set while a breakpoint is present
tcg: Probe the TB jump cache inline instead of calling a helper
Richard Henderson (9):
target/alpha: Use tcg_gen_goto_jc{3}_i64
target/s390x/tcg: Remove DISAS_PC_UPDATED
target/s390x/tcg: Use tcg_gen_goto_jc3_i64 for branches
include/exec: Move tc within TranslationBlock
tcg: Add INDEX_op_goto_jc
tcg/x86_64: Implement INDEX_op_goto_jc
tcg/aarch64: Implement INDEX_op_goto_jc
tcg/s390x: Implement INDEX_op_goto_jc
accel/tcg: Move tb_jmp_cache to CPUNegativeOffsetState
docs/devel/tcg-ops.rst | 11 ++
accel/tcg/tcg-runtime.h | 4 +
include/exec/translation-block.h | 9 +-
include/hw/core/cpu.h | 4 +-
include/tcg/tcg-op-common.h | 29 +++
include/tcg/tcg-op.h | 4 +
include/tcg/tcg-opc.h | 1 +
tcg/aarch64/tcg-target-con-set.h | 1 +
tcg/s390x/tcg-target-con-set.h | 1 +
tcg/s390x/tcg-target-con-str.h | 1 +
tcg/tcg-internal.h | 10 ++
include/tcg/tcg-op-def.h.inc | 2 +
accel/tcg/cpu-exec-common.c | 13 +-
accel/tcg/cpu-exec.c | 37 +++-
accel/tcg/cputlb.c | 2 +-
accel/tcg/tb-maint.c | 2 +-
accel/tcg/translate-all.c | 2 +-
cpu-common.c | 27 ++-
target/alpha/translate.c | 60 ++++---
target/loongarch/tcg/translate.c | 2 +-
target/ppc/translate.c | 4 +-
target/s390x/tcg/translate.c | 32 ++--
tcg/tcg-op.c | 167 +++++++++++++++++-
tcg/tcg.c | 23 +++
.../tcg/insn_trans/trans_branch.c.inc | 2 +-
tcg/aarch64/tcg-target.c.inc | 131 +++++++++++++-
tcg/loongarch64/tcg-target.c.inc | 4 +
tcg/ppc64/tcg-target.c.inc | 4 +
tcg/riscv64/tcg-target.c.inc | 4 +
tcg/s390x/tcg-target.c.inc | 62 +++++++
tcg/sparc64/tcg-target.c.inc | 4 +
tcg/tci/tcg-target.c.inc | 4 +
tcg/x86_64/tcg-target.c.inc | 79 +++++++++
33 files changed, 667 insertions(+), 75 deletions(-)
--
2.53.0
