With Ztso, plain loads and stores follow RVTSO, but the barriers around them use TCG_MO_ALL. That also orders a store before a later load, which TSO doesn't require, and is expensive: on x86 hosts it adds a locked instruction around every load and store in a multi-threaded guest.
The barrier after a load now only keeps later loads and stores behind it, and the barrier before a store only keeps earlier loads and stores ahead of it. That is all TSO needs. Signed-off-by: Rick Richard <[email protected]> --- I found this while looking into why riscv64 guests were so much slower than aarch64 ones under bsd-user on FreeBSD. The default -cpu max enables Ztso, and turning it off with ztso=false accounted for most of the difference. On an x86-64 host the patch made xz almost four times faster and cut the time of a clang compile by about 30%, the same as running with ztso=false. The output didn't change. I also tested it on a Raspberry Pi 3B+, which is arm64 and doesn't give TSO ordering in hardware, to make sure the weaker barriers still hold up there. Everything ran correctly, and xz was about 12% faster. target/riscv/tcg/insn_trans/trans_rvi.c.inc | 4 ++-- target/riscv/tcg/insn_trans/trans_rvv.c.inc | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/target/riscv/tcg/insn_trans/trans_rvi.c.inc b/target/riscv/tcg/insn_trans/trans_rvi.c.inc index 40b4c9aa..d31e4295 100644 --- a/target/riscv/tcg/insn_trans/trans_rvi.c.inc +++ b/target/riscv/tcg/insn_trans/trans_rvi.c.inc @@ -424,7 +424,7 @@ static bool gen_load(DisasContext *ctx, arg_lb *a, MemOp memop) } if (ctx->ztso) { - tcg_gen_mb(TCG_MO_ALL | TCG_BAR_LDAQ); + tcg_gen_mb(TCG_MO_LD_LD | TCG_MO_LD_ST | TCG_BAR_LDAQ); } return out; @@ -485,7 +485,7 @@ static bool gen_store_tl(DisasContext *ctx, arg_sb *a, MemOp memop) TCGv data = get_gpr(ctx, a->rs2, EXT_NONE); if (ctx->ztso) { - tcg_gen_mb(TCG_MO_ALL | TCG_BAR_STRL); + tcg_gen_mb(TCG_MO_LD_ST | TCG_MO_ST_ST | TCG_BAR_STRL); } tcg_gen_qemu_st_tl(data, addr, ctx->mem_idx, memop); diff --git a/target/riscv/tcg/insn_trans/trans_rvv.c.inc b/target/riscv/tcg/insn_trans/trans_rvv.c.inc index a22e2cae..8c21ab87 100644 --- a/target/riscv/tcg/insn_trans/trans_rvv.c.inc +++ b/target/riscv/tcg/insn_trans/trans_rvv.c.inc @@ -731,7 +731,7 @@ static bool ldst_us_trans(uint32_t vd, uint32_t rs1, uint32_t data, * around the loop. */ if (is_store && s->ztso) { - tcg_gen_mb(TCG_MO_ALL | TCG_BAR_STRL); + tcg_gen_mb(TCG_MO_LD_ST | TCG_MO_ST_ST | TCG_BAR_STRL); } mark_vs_dirty(s); @@ -739,7 +739,7 @@ static bool ldst_us_trans(uint32_t vd, uint32_t rs1, uint32_t data, fn(dest, mask, base, tcg_env, desc); if (!is_store && s->ztso) { - tcg_gen_mb(TCG_MO_ALL | TCG_BAR_LDAQ); + tcg_gen_mb(TCG_MO_LD_LD | TCG_MO_LD_ST | TCG_BAR_LDAQ); } finalize_rvv_inst(s); -- 2.43.0
