Re: [PATCH 2/2] scsi-disk: Fix crash for VM configured with USB CDROM after live migration

[Prasad Pandit]

Hi,

On Fri, 24 May 2024 at 16:23, Yong Huang  wrote:
> I'm not testing the latest QEMU version while theoretically it is
> reproducible, I'll check it and give a conclusion.

* Yes, that'll help. Thank you.

> I'm not sure this usage is common but in our production environment, it is 
> used.

* I see. If it's being used in a production environment and the crash
occurs there, then it's a reasonable change.

 > IMHO, resetting the CDROM device may be a work around because
> the request SHOULD not be lost. No requests are found may be
> caused by other reasons, resetting the CD ROM seems crude.
> The path that executes the scsi_get_buf() is in a USB mass storage
> device,  and it called by the UHCI controller originally, which just
> handles the Frame List blindly, reset solution is kind of complicated
> in implementation
>
> Migrating the requests may be a graceful solution.

* Yes, true. Migration should migrate guest's devices along with their
state and data. Resetting was suggested for the case if CDROM is not
used in production and so the migration was not required. But you are
using it in a production environment so migrating SCSI requests makes
sense.

Thank you.
---
  - Prasad

Re: [PATCH v2 01/18] migration: Fix file migration with fdset

[Prasad Pandit]

On Fri, 24 May 2024 at 18:00, Fabiano Rosas  wrote:
> That's the point. If offset==0 we truncate all the way, if not, we truncate 
> to the offset.

* Yes, I was wondering if the migration file has some data, but still
'offset' ends up being zero(0). If that's unlikely to happen, then we
are good.

Thank you.
---
  - Prasad

[PATCH 3/3] tests/qtest/migration-test: Use custom asm bios for ppc64

[Nicholas Piggin]

Similar to other archs, build a custom bios memory updater. Running the
test with OF code is a cool trick, but SLOF takes a long time to boot.
This reduces test time by around 3x (150s to 50s).

Signed-off-by: Nicholas Piggin 
---
 tests/migration/migration-test.h   |  1 +
 tests/migration/ppc64/a-b-kernel.h | 42 +++
 tests/qtest/migration-test.c   | 35 +++-
 tests/migration/Makefile   |  2 +-
 tests/migration/ppc64/Makefile | 15 +++
 tests/migration/ppc64/a-b-kernel.S | 66 ++
 6 files changed, 130 insertions(+), 31 deletions(-)
 create mode 100644 tests/migration/ppc64/a-b-kernel.h
 create mode 100644 tests/migration/ppc64/Makefile
 create mode 100644 tests/migration/ppc64/a-b-kernel.S

diff --git a/tests/migration/migration-test.h b/tests/migration/migration-test.h
index 68512c0b1b..194df7df6f 100644
--- a/tests/migration/migration-test.h
+++ b/tests/migration/migration-test.h
@@ -22,6 +22,7 @@
 /* PPC */
 #define PPC_TEST_MEM_START (1 * 1024 * 1024)
 #define PPC_TEST_MEM_END   (100 * 1024 * 1024)
+#define PPC_H_PUT_TERM_CHAR 0x58
 
 /* ARM */
 #define ARM_TEST_MEM_START (0x4000 + 1 * 1024 * 1024)
diff --git a/tests/migration/ppc64/a-b-kernel.h 
b/tests/migration/ppc64/a-b-kernel.h
new file mode 100644
index 00..673317efdb
--- /dev/null
+++ b/tests/migration/ppc64/a-b-kernel.h
@@ -0,0 +1,42 @@
+/* This file is automatically generated from the assembly file in
+ * tests/migration/ppc64. Edit that file and then run "make all"
+ * inside tests/migration to update, and then remember to send both
+ * the header and the assembler differences in your patch submission.
+ */
+unsigned char ppc64_kernel[] = {
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x7d, 0x20, 0x00, 0xa6, 0x39, 0x40, 0xff, 0xff,
+  0x79, 0x49, 0xf8, 0x0e, 0x7d, 0x20, 0x01, 0x64, 0x3e, 0x80, 0x00, 0x10,
+  0x62, 0x94, 0x00, 0x00, 0x3d, 0x20, 0x06, 0x40, 0x61, 0x29, 0x00, 0x00,
+  0x7e, 0xb4, 0x48, 0x50, 0x39, 0x40, 0x10, 0x00, 0x7e, 0xb5, 0x53, 0xd2,
+  0x38, 0x60, 0x00, 0x58, 0x38, 0x80, 0x00, 0x00, 0x38, 0xa0, 0x00, 0x01,
+  0x38, 0xc0, 0x00, 0x41, 0x78, 0xc6, 0xc1, 0xc6, 0x44, 0x00, 0x00, 0x22,
+  0x38, 0x60, 0x00, 0x00, 0x7e, 0x89, 0xa3, 0x78, 0x7e, 0xa9, 0x03, 0xa6,
+  0x98, 0x69, 0x00, 0x00, 0x39, 0x29, 0x10, 0x00, 0x42, 0x00, 0xff, 0xf8,
+  0x7e, 0x89, 0xa3, 0x78, 0x7e, 0xa9, 0x03, 0xa6, 0x88, 0x69, 0x00, 0x00,
+  0x38, 0x63, 0x00, 0x01, 0x98, 0x69, 0x00, 0x00, 0x39, 0x29, 0x10, 0x00,
+  0x42, 0x00, 0xff, 0xf0, 0x38, 0x60, 0x00, 0x58, 0x38, 0x80, 0x00, 0x00,
+  0x38, 0xa0, 0x00, 0x01, 0x38, 0xc0, 0x00, 0x42, 0x78, 0xc6, 0xc1, 0xc6,
+  0x44, 0x00, 0x00, 0x22, 0x4b, 0xff, 0xff, 0xcc
+};
+
diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index b8617cc843..94d5057857 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -128,6 +128,7 @@ static char *bootpath;
  */
 #include "tests/migration/i386/a-b-bootblock.h"
 #include "tests/migration/aarch64/a-b-kernel.h"
+#include "tests/migration/ppc64/a-b-kernel.h"
 #include "tests/migration/s390x/a-b-bios.h"
 
 static void bootfile_create(char *dir, bool suspend_me)
@@ -147,10 +148,8 @@ static void bootfile_create(char *dir, bool suspend_me)
 content = s390x_elf;
 len = sizeof(s390x_elf);
 } else if (strcmp(arch, "ppc64") == 0) {
-/*
- * sane architectures can be programmed at the boot prompt
-   

[PATCH 2/3] tests/qtest/migration-test: Enable on ppc64

[Nicholas Piggin]

ppc64 with TCG seems to no longer be failing this test. Let's try to
enable it. s390x is still hanging about 1 in 10 runs.

Signed-off-by: Nicholas Piggin 
---
 tests/qtest/migration-test.c | 16 +++-
 1 file changed, 3 insertions(+), 13 deletions(-)

diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index c13535c37d..b8617cc843 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -3454,19 +3454,9 @@ int main(int argc, char **argv)
 #endif
 
 /*
- * On ppc64, the test only works with kvm-hv, but not with kvm-pr and TCG
- * is touchy due to race conditions on dirty bits (especially on PPC for
- * some reason)
- */
-if (g_str_equal(arch, "ppc64") &&
-(!has_kvm || access("/sys/module/kvm_hv", F_OK))) {
-g_test_message("Skipping tests: kvm_hv not available");
-goto test_add_done;
-}
-
-/*
- * Similar to ppc64, s390x seems to be touchy with TCG, so disable it
- * there until the problems are resolved
+ * On s390x, the test seems to be touchy with TCG, perhaps due to race
+ * conditions on dirty bits, so disable it there until the problems are
+ * resolved.
  */
 if (g_str_equal(arch, "s390x") && !has_kvm) {
 g_test_message("Skipping tests: s390x host with KVM is required");
-- 
2.43.0

[PATCH 1/3] tests/qtest/migration-test: Quieten ppc64 QEMU warnigns

[Nicholas Piggin]

Signed-off-by: Nicholas Piggin 
---
 tests/qtest/migration-test.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index b7e3406471..c13535c37d 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -21,6 +21,7 @@
 #include "chardev/char.h"
 #include "crypto/tlscredspsk.h"
 #include "qapi/qmp/qlist.h"
+#include "libqos/libqos-spapr.h"
 
 #include "migration-helpers.h"
 #include "tests/migration/migration-test.h"
@@ -742,7 +743,8 @@ static int test_migrate_start(QTestState **from, QTestState 
**to,
   "until'", end_address, start_address);
 machine_alias = "pseries";
 machine_opts = "vsmt=8";
-arch_opts = g_strdup("-nodefaults");
+arch_opts = g_strdup_printf("-nodefaults "
+"-machine " PSERIES_DEFAULT_CAPABILITIES);
 } else if (strcmp(arch, "aarch64") == 0) {
 memory_size = "150M";
 machine_alias = "virt";
-- 
2.43.0

[PATCH 0/3] tests/qtest/migration-test: Improve and enable on ppc64

[Nicholas Piggin]

ppc64 seems to be stable now, only problem is the speed, so I
rewrote the test to avoid SLOF and use special bios blob like
the other archs.

Thanks,
Nick

Nicholas Piggin (3):
  tests/qtest/migration-test: Quieten ppc64 QEMU warnigns
  tests/qtest/migration-test: Enable on ppc64
  tests/qtest/migration-test: Use custom asm bios for ppc64

 tests/migration/migration-test.h   |  1 +
 tests/migration/ppc64/a-b-kernel.h | 42 +++
 tests/qtest/migration-test.c   | 53 +---
 tests/migration/Makefile   |  2 +-
 tests/migration/ppc64/Makefile | 15 +++
 tests/migration/ppc64/a-b-kernel.S | 66 ++
 6 files changed, 135 insertions(+), 44 deletions(-)
 create mode 100644 tests/migration/ppc64/a-b-kernel.h
 create mode 100644 tests/migration/ppc64/Makefile
 create mode 100644 tests/migration/ppc64/a-b-kernel.S

-- 
2.43.0

[PATCH v2 19/67] target/arm: Convert FADD, FSUB, FDIV, FMUL to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/helper-a64.h|   4 +
 target/arm/tcg/translate.h |   5 +
 target/arm/tcg/a64.decode  |  27 +
 target/arm/tcg/translate-a64.c | 205 +
 target/arm/tcg/vec_helper.c|   4 +
 5 files changed, 143 insertions(+), 102 deletions(-)

diff --git a/target/arm/tcg/helper-a64.h b/target/arm/tcg/helper-a64.h
index b79751a717..371388f61b 100644
--- a/target/arm/tcg/helper-a64.h
+++ b/target/arm/tcg/helper-a64.h
@@ -133,6 +133,10 @@ DEF_HELPER_4(cpyfp, void, env, i32, i32, i32)
 DEF_HELPER_4(cpyfm, void, env, i32, i32, i32)
 DEF_HELPER_4(cpyfe, void, env, i32, i32, i32)
 
+DEF_HELPER_FLAGS_5(gvec_fdiv_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_fdiv_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_fdiv_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
+
 DEF_HELPER_FLAGS_5(gvec_fmulx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 DEF_HELPER_FLAGS_5(gvec_fmulx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 DEF_HELPER_FLAGS_5(gvec_fmulx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h
index 80e85096a8..ecfa242eef 100644
--- a/target/arm/tcg/translate.h
+++ b/target/arm/tcg/translate.h
@@ -252,6 +252,11 @@ static inline int shl_12(DisasContext *s, int x)
 return x << 12;
 }
 
+static inline int xor_2(DisasContext *s, int x)
+{
+return x ^ 2;
+}
+
 static inline int neon_3same_fp_size(DisasContext *s, int x)
 {
 /* Convert 0==fp32, 1==fp16 into a MO_* value */
diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 2e0e01be01..82daafbef5 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -21,6 +21,7 @@
 
 %rd 0:5
 %esz_sd 22:1 !function=plus_2
+%esz_hsd22:2 !function=xor_2
 %hl 11:1 21:1
 %hlm11:1 20:2
 
@@ -37,6 +38,7 @@
 
 @rrr_h   ... rm:5 .. rn:5 rd:5  &rrr_e esz=1
 @rrr_sd  ... rm:5 .. rn:5 rd:5  &rrr_e esz=%esz_sd
+@rrr_hsd ... rm:5 .. rn:5 rd:5  &rrr_e esz=%esz_hsd
 
 @rrx_h   .. .. rm:4  . . rn:5 rd:5  &rrx_e esz=1 idx=%hlm
 @rrx_s   .. . rm:5   . . rn:5 rd:5  &rrx_e esz=2 idx=%hl
@@ -697,22 +699,47 @@ INS_element 0 1   10 1110 000 di:5  0 si:4 1 rn:5 rd:5
 
 ### Advanced SIMD scalar three same
 
+FADD_s  0001 1110 ..1 . 0010 10 . . @rrr_hsd
+FSUB_s  0001 1110 ..1 . 0011 10 . . @rrr_hsd
+FDIV_s  0001 1110 ..1 . 0001 10 . . @rrr_hsd
+FMUL_s  0001 1110 ..1 .  10 . . @rrr_hsd
+
 FMULX_s 0101 1110 010 . 00011 1 . . @rrr_h
 FMULX_s 0101 1110 0.1 . 11011 1 . . @rrr_sd
 
 ### Advanced SIMD three same
 
+FADD_v  0.00 1110 010 . 00010 1 . . @qrrr_h
+FADD_v  0.00 1110 0.1 . 11010 1 . . @qrrr_sd
+
+FSUB_v  0.00 1110 110 . 00010 1 . . @qrrr_h
+FSUB_v  0.00 1110 1.1 . 11010 1 . . @qrrr_sd
+
+FDIV_v  0.10 1110 010 . 00111 1 . . @qrrr_h
+FDIV_v  0.10 1110 0.1 . 1 1 . . @qrrr_sd
+
+FMUL_v  0.10 1110 010 . 00011 1 . . @qrrr_h
+FMUL_v  0.10 1110 0.1 . 11011 1 . . @qrrr_sd
+
 FMULX_v 0.00 1110 010 . 00011 1 . . @qrrr_h
 FMULX_v 0.00 1110 0.1 . 11011 1 . . @qrrr_sd
 
 ### Advanced SIMD scalar x indexed element
 
+FMUL_si 0101  00 ..  1001 . 0 . .   @rrx_h
+FMUL_si 0101  10 . . 1001 . 0 . .   @rrx_s
+FMUL_si 0101  11 0 . 1001 . 0 . .   @rrx_d
+
 FMULX_si0111  00 ..  1001 . 0 . .   @rrx_h
 FMULX_si0111  10 . . 1001 . 0 . .   @rrx_s
 FMULX_si0111  11 0 . 1001 . 0 . .   @rrx_d
 
 ### Advanced SIMD vector x indexed element
 
+FMUL_vi 0.00  00 ..  1001 . 0 . .   @qrrx_h
+FMUL_vi 0.00  10 . . 1001 . 0 . .   @qrrx_s
+FMUL_vi 0.00  11 0 . 1001 . 0 . .   @qrrx_d
+
 FMULX_vi0.10  00 ..  1001 . 0 . .   @qrrx_h
 FMULX_vi0.10  10 . . 1001 . 0 . .   @qrrx_s
 FMULX_vi0.10  11 0 . 1001 . 0 . .   @qrrx_d
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 8cbe6cd70f..97c3d758d6 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -4887,6 +4887,34 @@ static bool do_fp3_scalar(DisasContext *s, arg_rrr_e *a, 
const FPScalar *f)
 return true;
 }
 
+static const FPScalar f_scalar_fadd = {
+gen_helper_vfp_addh,
+gen_helper_vfp_adds,
+gen_helper_vfp_addd,
+};
+TRANS(FADD_s, do_fp3_scalar, a, &f_

[PATCH v2 50/67] target/arm: Convert ADD, SUB (vector) to decodetree

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  6 ++
 target/arm/tcg/translate-a64.c | 34 +++---
 2 files changed, 17 insertions(+), 23 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 96ce35ad40..44383b4fc7 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -765,6 +765,9 @@ UQSHL_s 0111 1110 ..1 . 01001 1 . . 
@rrr_e
 SQRSHL_s0101 1110 ..1 . 01011 1 . . @rrr_e
 UQRSHL_s0111 1110 ..1 . 01011 1 . . @rrr_e
 
+ADD_s   0101 1110 111 . 1 1 . . @rrr_d
+SUB_s   0111 1110 111 . 1 1 . . @rrr_d
+
 ### Advanced SIMD scalar pairwise
 
 FADDP_s 0101 1110 0011  1101 10 . . @rr_h
@@ -895,6 +898,9 @@ UQSHL_v 0.10 1110 ..1 . 01001 1 . . 
@qrrr_e
 SQRSHL_v0.00 1110 ..1 . 01011 1 . . @qrrr_e
 UQRSHL_v0.10 1110 ..1 . 01011 1 . . @qrrr_e
 
+ADD_v   0.00 1110 ..1 . 1 1 . . @qrrr_e
+SUB_v   0.10 1110 ..1 . 1 1 . . @qrrr_e
+
 ### Advanced SIMD scalar x indexed element
 
 FMUL_si 0101  00 ..  1001 . 0 . .   @rrx_h
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index b76682cabf..77a64923e7 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5118,6 +5118,8 @@ TRANS(SSHL_s, do_int3_scalar_d, a, gen_sshl_i64)
 TRANS(USHL_s, do_int3_scalar_d, a, gen_ushl_i64)
 TRANS(SRSHL_s, do_int3_scalar_d, a, gen_helper_neon_rshl_s64)
 TRANS(URSHL_s, do_int3_scalar_d, a, gen_helper_neon_rshl_u64)
+TRANS(ADD_s, do_int3_scalar_d, a, tcg_gen_add_i64)
+TRANS(SUB_s, do_int3_scalar_d, a, tcg_gen_sub_i64)
 
 typedef struct ENVScalar2 {
 NeonGenTwoOpEnvFn *gen_bhs[3];
@@ -5432,6 +5434,8 @@ TRANS(UQSHL_v, do_gvec_fn3, a, gen_neon_uqshl)
 TRANS(SQRSHL_v, do_gvec_fn3, a, gen_neon_sqrshl)
 TRANS(UQRSHL_v, do_gvec_fn3, a, gen_neon_uqrshl)
 
+TRANS(ADD_v, do_gvec_fn3, a, tcg_gen_gvec_add)
+TRANS(SUB_v, do_gvec_fn3, a, tcg_gen_gvec_sub)
 
 /*
  * Advanced SIMD scalar/vector x indexed element
@@ -9444,13 +9448,6 @@ static void handle_3same_64(DisasContext *s, int opcode, 
bool u,
 }
 gen_cmtst_i64(tcg_rd, tcg_rn, tcg_rm);
 break;
-case 0x10: /* ADD, SUB */
-if (u) {
-tcg_gen_sub_i64(tcg_rd, tcg_rn, tcg_rm);
-} else {
-tcg_gen_add_i64(tcg_rd, tcg_rn, tcg_rm);
-}
-break;
 default:
 case 0x1: /* SQADD / UQADD */
 case 0x5: /* SQSUB / UQSUB */
@@ -9458,6 +9455,7 @@ static void handle_3same_64(DisasContext *s, int opcode, 
bool u,
 case 0x9: /* SQSHL, UQSHL */
 case 0xa: /* SRSHL, URSHL */
 case 0xb: /* SQRSHL, UQRSHL */
+case 0x10: /* ADD, SUB */
 g_assert_not_reached();
 }
 }
@@ -9482,7 +9480,6 @@ static void disas_simd_scalar_three_reg_same(DisasContext 
*s, uint32_t insn)
 case 0x6: /* CMGT, CMHI */
 case 0x7: /* CMGE, CMHS */
 case 0x11: /* CMTST, CMEQ */
-case 0x10: /* ADD, SUB (vector) */
 if (size != 3) {
 unallocated_encoding(s);
 return;
@@ -9501,6 +9498,7 @@ static void disas_simd_scalar_three_reg_same(DisasContext 
*s, uint32_t insn)
 case 0x9: /* SQSHL, UQSHL */
 case 0xa: /* SRSHL, URSHL */
 case 0xb: /* SQRSHL, UQRSHL */
+case 0x10: /* ADD, SUB (vector) */
 unallocated_encoding(s);
 return;
 }
@@ -10958,6 +10956,11 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 
 case 0x01: /* SQADD, UQADD */
 case 0x05: /* SQSUB, UQSUB */
+case 0x08: /* SSHL, USHL */
+case 0x09: /* SQSHL, UQSHL */
+case 0x0a: /* SRSHL, URSHL */
+case 0x0b: /* SQRSHL, UQRSHL */
+case 0x10: /* ADD, SUB */
 unallocated_encoding(s);
 return;
 }
@@ -10995,13 +10998,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_saba, size);
 }
 return;
-case 0x10: /* ADD, SUB */
-if (u) {
-gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_sub, size);
-} else {
-gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_add, size);
-}
-return;
 case 0x13: /* MUL, PMUL */
 if (!u) { /* MUL */
 gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_mul, size);
@@ -11044,14 +11040,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
  vec_full_reg_offset(s, rm),
  is_q ? 16 : 8, vec_full_reg_size(s));
 return;
-
-case 0x01: /* SQADD, UQADD */
-case 0x05: /* SQSUB, UQSUB */
-case 0x08: /* SSHL, USHL */
-case 0x09: /* SQSHL, UQSHL */
-case 0x0a: /* SRSHL, URSHL */
-case 0x0b: /* SQRSHL, UQRSHL */
-g_assert_not_reached();
 }
 
 if (size == 

[PATCH v2 27/67] target/arm: Convert FRECPS, FRSQRTS to decodetree

[Richard Henderson]

These are the last instructions within handle_3same_float
and disas_simd_scalar_three_reg_same_fp16 so remove them.

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  12 ++
 target/arm/tcg/translate-a64.c | 293 -
 2 files changed, 46 insertions(+), 259 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index a852b5f06f..84cb38f1dd 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -731,6 +731,12 @@ FACGT_s 0111 1110 1.1 . 11101 1 . . 
@rrr_sd
 FABD_s  0111 1110 110 . 00010 1 . . @rrr_h
 FABD_s  0111 1110 1.1 . 11010 1 . . @rrr_sd
 
+FRECPS_s0101 1110 010 . 00111 1 . . @rrr_h
+FRECPS_s0101 1110 0.1 . 1 1 . . @rrr_sd
+
+FRSQRTS_s   0101 1110 110 . 00111 1 . . @rrr_h
+FRSQRTS_s   0101 1110 1.1 . 1 1 . . @rrr_sd
+
 ### Advanced SIMD three same
 
 FADD_v  0.00 1110 010 . 00010 1 . . @qrrr_h
@@ -784,6 +790,12 @@ FACGT_v 0.10 1110 1.1 . 11101 1 . . 
@qrrr_sd
 FABD_v  0.10 1110 110 . 00010 1 . . @qrrr_h
 FABD_v  0.10 1110 1.1 . 11010 1 . . @qrrr_sd
 
+FRECPS_v0.00 1110 010 . 00111 1 . . @qrrr_h
+FRECPS_v0.00 1110 0.1 . 1 1 . . @qrrr_sd
+
+FRSQRTS_v   0.00 1110 110 . 00111 1 . . @qrrr_h
+FRSQRTS_v   0.00 1110 1.1 . 1 1 . . @qrrr_sd
+
 ### Advanced SIMD scalar x indexed element
 
 FMUL_si 0101  00 ..  1001 . 0 . .   @rrx_h
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 633384d2a5..a7537a5104 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5035,6 +5035,20 @@ static const FPScalar f_scalar_fabd = {
 };
 TRANS(FABD_s, do_fp3_scalar, a, &f_scalar_fabd)
 
+static const FPScalar f_scalar_frecps = {
+gen_helper_recpsf_f16,
+gen_helper_recpsf_f32,
+gen_helper_recpsf_f64,
+};
+TRANS(FRECPS_s, do_fp3_scalar, a, &f_scalar_frecps)
+
+static const FPScalar f_scalar_frsqrts = {
+gen_helper_rsqrtsf_f16,
+gen_helper_rsqrtsf_f32,
+gen_helper_rsqrtsf_f64,
+};
+TRANS(FRSQRTS_s, do_fp3_scalar, a, &f_scalar_frsqrts)
+
 static bool do_fp3_vector(DisasContext *s, arg_qrrr_e *a,
   gen_helper_gvec_3_ptr * const fns[3])
 {
@@ -5182,6 +5196,20 @@ static gen_helper_gvec_3_ptr * const f_vector_fabd[3] = {
 };
 TRANS(FABD_v, do_fp3_vector, a, f_vector_fabd)
 
+static gen_helper_gvec_3_ptr * const f_vector_frecps[3] = {
+gen_helper_gvec_recps_h,
+gen_helper_gvec_recps_s,
+gen_helper_gvec_recps_d,
+};
+TRANS(FRECPS_v, do_fp3_vector, a, f_vector_frecps)
+
+static gen_helper_gvec_3_ptr * const f_vector_frsqrts[3] = {
+gen_helper_gvec_rsqrts_h,
+gen_helper_gvec_rsqrts_s,
+gen_helper_gvec_rsqrts_d,
+};
+TRANS(FRSQRTS_v, do_fp3_vector, a, f_vector_frsqrts)
+
 /*
  * Advanced SIMD scalar/vector x indexed element
  */
@@ -9308,107 +9336,6 @@ static void handle_3same_64(DisasContext *s, int 
opcode, bool u,
 }
 }
 
-/* Handle the 3-same-operands float operations; shared by the scalar
- * and vector encodings. The caller must filter out any encodings
- * not allocated for the encoding it is dealing with.
- */
-static void handle_3same_float(DisasContext *s, int size, int elements,
-   int fpopcode, int rd, int rn, int rm)
-{
-int pass;
-TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR);
-
-for (pass = 0; pass < elements; pass++) {
-if (size) {
-/* Double */
-TCGv_i64 tcg_op1 = tcg_temp_new_i64();
-TCGv_i64 tcg_op2 = tcg_temp_new_i64();
-TCGv_i64 tcg_res = tcg_temp_new_i64();
-
-read_vec_element(s, tcg_op1, rn, pass, MO_64);
-read_vec_element(s, tcg_op2, rm, pass, MO_64);
-
-switch (fpopcode) {
-case 0x1f: /* FRECPS */
-gen_helper_recpsf_f64(tcg_res, tcg_op1, tcg_op2, fpst);
-break;
-case 0x3f: /* FRSQRTS */
-gen_helper_rsqrtsf_f64(tcg_res, tcg_op1, tcg_op2, fpst);
-break;
-default:
-case 0x18: /* FMAXNM */
-case 0x19: /* FMLA */
-case 0x1a: /* FADD */
-case 0x1b: /* FMULX */
-case 0x1c: /* FCMEQ */
-case 0x1e: /* FMAX */
-case 0x38: /* FMINNM */
-case 0x39: /* FMLS */
-case 0x3a: /* FSUB */
-case 0x3e: /* FMIN */
-case 0x5b: /* FMUL */
-case 0x5c: /* FCMGE */
-case 0x5d: /* FACGE */
-case 0x5f: /* FDIV */
-case 0x7a: /* FABD */
-case 0x7c: /* FCMGT */
-case 0x7d: /* FACGT */
-g_assert_not_reached();
-}
-
-   

[PATCH v2 35/67] target/arm: Convert FMLAL, FMLSL to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  10 +++
 target/arm/tcg/translate-a64.c | 144 ++---
 2 files changed, 51 insertions(+), 103 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 22dfe8568d..7e993ed345 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -797,6 +797,11 @@ FMLA_v  0.00 1110 0.1 . 11001 1 . . 
@qrrr_sd
 FMLS_v  0.00 1110 110 . 1 1 . . @qrrr_h
 FMLS_v  0.00 1110 1.1 . 11001 1 . . @qrrr_sd
 
+FMLAL_v 0.00 1110 001 . 11101 1 . . @qrrr_h
+FMLSL_v 0.00 1110 101 . 11101 1 . . @qrrr_h
+FMLAL2_v0.10 1110 001 . 11001 1 . . @qrrr_h
+FMLSL2_v0.10 1110 101 . 11001 1 . . @qrrr_h
+
 FCMEQ_v 0.00 1110 010 . 00100 1 . . @qrrr_h
 FCMEQ_v 0.00 1110 0.1 . 11100 1 . . @qrrr_sd
 
@@ -877,3 +882,8 @@ FMLS_vi 0.00  11 0 . 0101 . 0 . .   
@qrrx_d
 FMULX_vi0.10  00 ..  1001 . 0 . .   @qrrx_h
 FMULX_vi0.10  10 . . 1001 . 0 . .   @qrrx_s
 FMULX_vi0.10  11 0 . 1001 . 0 . .   @qrrx_d
+
+FMLAL_vi0.00  10 ..   . 0 . .   @qrrx_h
+FMLSL_vi0.00  10 ..  0100 . 0 . .   @qrrx_h
+FMLAL2_vi   0.10  10 ..  1000 . 0 . .   @qrrx_h
+FMLSL2_vi   0.10  10 ..  1100 . 0 . .   @qrrx_h
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 9fe70a939b..a4ff1fd202 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5256,6 +5256,24 @@ static gen_helper_gvec_3_ptr * const f_vector_fminnmp[3] 
= {
 };
 TRANS(FMINNMP_v, do_fp3_vector, a, f_vector_fminnmp)
 
+static bool do_fmlal(DisasContext *s, arg_qrrr_e *a, bool is_s, bool is_2)
+{
+if (fp_access_check(s)) {
+int data = (is_2 << 1) | is_s;
+tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd),
+   vec_full_reg_offset(s, a->rn),
+   vec_full_reg_offset(s, a->rm), tcg_env,
+   a->q ? 16 : 8, vec_full_reg_size(s),
+   data, gen_helper_gvec_fmlal_a64);
+}
+return true;
+}
+
+TRANS_FEAT(FMLAL_v, aa64_fhm, do_fmlal, a, false, false)
+TRANS_FEAT(FMLSL_v, aa64_fhm, do_fmlal, a, true, false)
+TRANS_FEAT(FMLAL2_v, aa64_fhm, do_fmlal, a, false, true)
+TRANS_FEAT(FMLSL2_v, aa64_fhm, do_fmlal, a, true, true)
+
 TRANS(ADDP_v, do_gvec_fn3, a, gen_gvec_addp)
 TRANS(SMAXP_v, do_gvec_fn3_no64, a, gen_gvec_smaxp)
 TRANS(SMINP_v, do_gvec_fn3_no64, a, gen_gvec_sminp)
@@ -5447,6 +5465,24 @@ static bool do_fmla_vector_idx(DisasContext *s, 
arg_qrrx_e *a, bool neg)
 TRANS(FMLA_vi, do_fmla_vector_idx, a, false)
 TRANS(FMLS_vi, do_fmla_vector_idx, a, true)
 
+static bool do_fmlal_idx(DisasContext *s, arg_qrrx_e *a, bool is_s, bool is_2)
+{
+if (fp_access_check(s)) {
+int data = (a->idx << 2) | (is_2 << 1) | is_s;
+tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd),
+   vec_full_reg_offset(s, a->rn),
+   vec_full_reg_offset(s, a->rm), tcg_env,
+   a->q ? 16 : 8, vec_full_reg_size(s),
+   data, gen_helper_gvec_fmlal_idx_a64);
+}
+return true;
+}
+
+TRANS_FEAT(FMLAL_vi, aa64_fhm, do_fmlal_idx, a, false, false)
+TRANS_FEAT(FMLSL_vi, aa64_fhm, do_fmlal_idx, a, true, false)
+TRANS_FEAT(FMLAL2_vi, aa64_fhm, do_fmlal_idx, a, false, true)
+TRANS_FEAT(FMLSL2_vi, aa64_fhm, do_fmlal_idx, a, true, true)
+
 /*
  * Advanced SIMD scalar pairwise
  */
@@ -10911,78 +10947,6 @@ static void disas_simd_3same_logic(DisasContext *s, 
uint32_t insn)
 }
 }
 
-/* Floating point op subgroup of C3.6.16. */
-static void disas_simd_3same_float(DisasContext *s, uint32_t insn)
-{
-/* For floating point ops, the U, size[1] and opcode bits
- * together indicate the operation. size[0] indicates single
- * or double.
- */
-int fpopcode = extract32(insn, 11, 5)
-| (extract32(insn, 23, 1) << 5)
-| (extract32(insn, 29, 1) << 6);
-int is_q = extract32(insn, 30, 1);
-int size = extract32(insn, 22, 1);
-int rm = extract32(insn, 16, 5);
-int rn = extract32(insn, 5, 5);
-int rd = extract32(insn, 0, 5);
-
-if (size == 1 && !is_q) {
-unallocated_encoding(s);
-return;
-}
-
-switch (fpopcode) {
-case 0x1d: /* FMLAL  */
-case 0x3d: /* FMLSL  */
-case 0x59: /* FMLAL2 */
-case 0x79: /* FMLSL2 */
-if (size & 1 || !dc_isar_feature(aa64_fhm, s)) {
-unallocated_encoding(s);
-return;
-}
-if (fp_access_check(s)) {
-int is_s = extract32(insn, 23, 1);
-int is_2 = extract32(insn, 29, 1);
-   

[PATCH v2 53/67] target/arm: Use TCG_COND_TSTNE in gen_cmtst_vec

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/gengvec.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c
index 1d6bc6021d..1895c3b19f 100644
--- a/target/arm/tcg/gengvec.c
+++ b/target/arm/tcg/gengvec.c
@@ -943,9 +943,7 @@ void gen_cmtst_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
 
 static void gen_cmtst_vec(unsigned vece, TCGv_vec d, TCGv_vec a, TCGv_vec b)
 {
-tcg_gen_and_vec(vece, d, a, b);
-tcg_gen_dupi_vec(vece, a, 0);
-tcg_gen_cmp_vec(TCG_COND_NE, vece, d, d, a);
+tcg_gen_cmp_vec(TCG_COND_TSTNE, vece, d, a, b);
 }
 
 void gen_gvec_cmtst(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
-- 
2.34.1

[PATCH v2 66/67] target/arm: Convert FMADD, FMSUB, FNMADD, FNMSUB to decodetree

[Richard Henderson]

These are the only instructions in the 3 source scalar class.

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  10 ++
 target/arm/tcg/translate-a64.c | 233 -
 2 files changed, 93 insertions(+), 150 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index f7f897f9fc..6f6cd805b7 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -32,6 +32,7 @@
 &rr_e   rd rn esz
 &rrr_e  rd rn rm esz
 &rrx_e  rd rn rm idx esz
+&_e rd rn rm ra esz
 &qrr_e  q rd rn esz
 &qrrr_e q rd rn rm esz
 &qrrx_e q rd rn rm idx esz
@@ -998,3 +999,12 @@ SQDMULH_vi  0.00  10 . . 1100 . 0 . .  
 @qrrx_s
 
 SQRDMULH_vi 0.00  01 ..  1101 . 0 . .   @qrrx_h
 SQRDMULH_vi 0.00  10 . . 1101 . 0 . .   @qrrx_s
+
+# Floating-point data-processing (3 source)
+
+@_hsd     .. . rm:5  . ra:5  rn:5  rd:5 &_e 
esz=%esz_hsd
+
+FMADD   0001  .. 0 . 0 . . .@_hsd
+FMSUB   0001  .. 0 . 1 . . .@_hsd
+FNMADD  0001  .. 1 . 0 . . .@_hsd
+FNMSUB  0001  .. 1 . 1 . . .@_hsd
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 14226c56cf..3c2963ebaa 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5866,6 +5866,88 @@ static bool trans_ADDP_s(DisasContext *s, arg_rr_e *a)
 return true;
 }
 
+/*
+ * Floating-point data-processing (3 source)
+ */
+
+static bool do_fmadd(DisasContext *s, arg__e *a, bool neg_a, bool neg_n)
+{
+TCGv_ptr fpst;
+
+/*
+ * These are fused multiply-add.  Note that doing the negations here
+ * as separate steps is correct: an input NaN should come out with
+ * its sign bit flipped if it is a negated-input.
+ */
+switch (a->esz) {
+case MO_64:
+if (fp_access_check(s)) {
+TCGv_i64 tn = read_fp_dreg(s, a->rn);
+TCGv_i64 tm = read_fp_dreg(s, a->rm);
+TCGv_i64 ta = read_fp_dreg(s, a->ra);
+
+if (neg_a) {
+gen_vfp_negd(ta, ta);
+}
+if (neg_n) {
+gen_vfp_negd(tn, tn);
+}
+fpst = fpstatus_ptr(FPST_FPCR);
+gen_helper_vfp_muladdd(ta, tn, tm, ta, fpst);
+write_fp_dreg(s, a->rd, ta);
+}
+break;
+
+case MO_32:
+if (fp_access_check(s)) {
+TCGv_i32 tn = read_fp_sreg(s, a->rn);
+TCGv_i32 tm = read_fp_sreg(s, a->rm);
+TCGv_i32 ta = read_fp_sreg(s, a->ra);
+
+if (neg_a) {
+gen_vfp_negs(ta, ta);
+}
+if (neg_n) {
+gen_vfp_negs(tn, tn);
+}
+fpst = fpstatus_ptr(FPST_FPCR);
+gen_helper_vfp_muladds(ta, tn, tm, ta, fpst);
+write_fp_sreg(s, a->rd, ta);
+}
+break;
+
+case MO_16:
+if (!dc_isar_feature(aa64_fp16, s)) {
+return false;
+}
+if (fp_access_check(s)) {
+TCGv_i32 tn = read_fp_hreg(s, a->rn);
+TCGv_i32 tm = read_fp_hreg(s, a->rm);
+TCGv_i32 ta = read_fp_hreg(s, a->ra);
+
+if (neg_a) {
+gen_vfp_negh(ta, ta);
+}
+if (neg_n) {
+gen_vfp_negh(tn, tn);
+}
+fpst = fpstatus_ptr(FPST_FPCR_F16);
+gen_helper_advsimd_muladdh(ta, tn, tm, ta, fpst);
+write_fp_sreg(s, a->rd, ta);
+}
+break;
+
+default:
+return false;
+}
+return true;
+}
+
+TRANS(FMADD, do_fmadd, a, false, false)
+TRANS(FNMADD, do_fmadd, a, true, true)
+TRANS(FMSUB, do_fmadd, a, false, true)
+TRANS(FNMSUB, do_fmadd, a, true, false)
+
 /* Shift a TCGv src by TCGv shift_amount, put result in dst.
  * Note that it is the caller's responsibility to ensure that the
  * shift amount is in range (ie 0..31 or 0..63) and provide the ARM
@@ -7665,152 +7747,6 @@ static void disas_fp_1src(DisasContext *s, uint32_t 
insn)
 }
 }
 
-/* Floating-point data-processing (3 source) - single precision */
-static void handle_fp_3src_single(DisasContext *s, bool o0, bool o1,
-  int rd, int rn, int rm, int ra)
-{
-TCGv_i32 tcg_op1, tcg_op2, tcg_op3;
-TCGv_i32 tcg_res = tcg_temp_new_i32();
-TCGv_ptr fpst = fpstatus_ptr(FPST_FPCR);
-
-tcg_op1 = read_fp_sreg(s, rn);
-tcg_op2 = read_fp_sreg(s, rm);
-tcg_op3 = read_fp_sreg(s, ra);
-
-/* These are fused multiply-add, and must be done as one
- * floating point operation with no rounding between the
- * multiplication and addition steps.
- * NB that doing the negations here as separate steps is
- * correct : an input NaN should come out with its sign bit

[PATCH v2 54/67] target/arm: Convert SHADD, UHADD to gvec

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/helper.h |   6 --
 target/arm/tcg/translate.h  |   5 ++
 target/arm/tcg/gengvec.c| 144 
 target/arm/tcg/neon_helper.c|  27 --
 target/arm/tcg/translate-a64.c  |  17 ++--
 target/arm/tcg/translate-neon.c |   4 +-
 6 files changed, 158 insertions(+), 45 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index 9a89c9cea7..b26bfcb079 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -268,12 +268,6 @@ DEF_HELPER_FLAGS_2(fjcvtzs, TCG_CALL_NO_RWG, i64, f64, ptr)
 DEF_HELPER_FLAGS_3(check_hcr_el2_trap, TCG_CALL_NO_WG, void, env, i32, i32)
 
 /* neon_helper.c */
-DEF_HELPER_2(neon_hadd_s8, i32, i32, i32)
-DEF_HELPER_2(neon_hadd_u8, i32, i32, i32)
-DEF_HELPER_2(neon_hadd_s16, i32, i32, i32)
-DEF_HELPER_2(neon_hadd_u16, i32, i32, i32)
-DEF_HELPER_2(neon_hadd_s32, s32, s32, s32)
-DEF_HELPER_2(neon_hadd_u32, i32, i32, i32)
 DEF_HELPER_2(neon_rhadd_s8, i32, i32, i32)
 DEF_HELPER_2(neon_rhadd_u8, i32, i32, i32)
 DEF_HELPER_2(neon_rhadd_s16, i32, i32, i32)
diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h
index 048cb45ebe..dd99d76bf2 100644
--- a/target/arm/tcg/translate.h
+++ b/target/arm/tcg/translate.h
@@ -472,6 +472,11 @@ void gen_neon_sqrshl(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 void gen_neon_uqrshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
  uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
 
+void gen_gvec_shadd(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_uhadd(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+
 void gen_cmtst_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b);
 void gen_ushl_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b);
 void gen_sshl_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b);
diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c
index 1895c3b19f..0627cec6b2 100644
--- a/target/arm/tcg/gengvec.c
+++ b/target/arm/tcg/gengvec.c
@@ -1852,3 +1852,147 @@ void gen_gvec_uminp(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 tcg_debug_assert(vece <= MO_32);
 tcg_gen_gvec_3_ool(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, 0, fns[vece]);
 }
+
+static void gen_shadd8_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+TCGv_i64 t = tcg_temp_new_i64();
+
+tcg_gen_and_i64(t, a, b);
+tcg_gen_vec_sar8i_i64(a, a, 1);
+tcg_gen_vec_sar8i_i64(b, b, 1);
+tcg_gen_andi_i64(t, t, dup_const(MO_8, 1));
+tcg_gen_vec_add8_i64(d, a, b);
+tcg_gen_vec_add8_i64(d, d, t);
+}
+
+static void gen_shadd16_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+TCGv_i64 t = tcg_temp_new_i64();
+
+tcg_gen_and_i64(t, a, b);
+tcg_gen_vec_sar16i_i64(a, a, 1);
+tcg_gen_vec_sar16i_i64(b, b, 1);
+tcg_gen_andi_i64(t, t, dup_const(MO_16, 1));
+tcg_gen_vec_add16_i64(d, a, b);
+tcg_gen_vec_add16_i64(d, d, t);
+}
+
+static void gen_shadd_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b)
+{
+TCGv_i32 t = tcg_temp_new_i32();
+
+tcg_gen_and_i32(t, a, b);
+tcg_gen_sari_i32(a, a, 1);
+tcg_gen_sari_i32(b, b, 1);
+tcg_gen_andi_i32(t, t, 1);
+tcg_gen_add_i32(d, a, b);
+tcg_gen_add_i32(d, d, t);
+}
+
+static void gen_shadd_vec(unsigned vece, TCGv_vec d, TCGv_vec a, TCGv_vec b)
+{
+TCGv_vec t = tcg_temp_new_vec_matching(d);
+
+tcg_gen_and_vec(vece, t, a, b);
+tcg_gen_sari_vec(vece, a, a, 1);
+tcg_gen_sari_vec(vece, b, b, 1);
+tcg_gen_and_vec(vece, t, t, tcg_constant_vec_matching(d, vece, 1));
+tcg_gen_add_vec(vece, d, a, b);
+tcg_gen_add_vec(vece, d, d, t);
+}
+
+void gen_gvec_shadd(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+static const TCGOpcode vecop_list[] = {
+INDEX_op_sari_vec, INDEX_op_add_vec, 0
+};
+static const GVecGen3 g[] = {
+{ .fni8 = gen_shadd8_i64,
+  .fniv = gen_shadd_vec,
+  .opt_opc = vecop_list,
+  .vece = MO_8 },
+{ .fni8 = gen_shadd16_i64,
+  .fniv = gen_shadd_vec,
+  .opt_opc = vecop_list,
+  .vece = MO_16 },
+{ .fni4 = gen_shadd_i32,
+  .fniv = gen_shadd_vec,
+  .opt_opc = vecop_list,
+  .vece = MO_32 },
+};
+tcg_debug_assert(vece <= MO_32);
+tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &g[vece]);
+}
+
+static void gen_uhadd8_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+TCGv_i64 t = tcg_temp_new_i64();
+
+tcg_gen_and_i64(t, a, b);
+tcg_gen_vec_shr8i_i64(a, a, 1);
+tcg_gen_vec_shr8i_i64(b, b, 1);
+tcg_gen_andi_i64(t, t, dup_const(MO_8, 1));
+tcg_gen_vec_add8_i64(d, a, b);
+tcg_gen_vec_add8_i64(d, d, t);
+}
+
+static void gen_uhadd16_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+TCGv_i64 t = tcg_temp_new_i64();
+
+tcg_gen_and_i64(t, a, b);
+   

[PATCH v2 46/67] target/arm: Convert SQSHL and UQSHL (register) to gvec

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/helper.h |  8 
 target/arm/tcg/translate.h  |  4 
 target/arm/tcg/neon-dp.decode   | 10 ++---
 target/arm/tcg/gengvec.c| 24 ++
 target/arm/tcg/neon_helper.c| 36 +
 target/arm/tcg/translate-a64.c  | 17 +++-
 target/arm/tcg/translate-neon.c |  6 ++
 7 files changed, 83 insertions(+), 22 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index 25eb7bf5df..f345087ddb 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -326,6 +326,14 @@ DEF_HELPER_3(neon_qrshl_u32, i32, env, i32, i32)
 DEF_HELPER_3(neon_qrshl_s32, i32, env, i32, i32)
 DEF_HELPER_3(neon_qrshl_u64, i64, env, i64, i64)
 DEF_HELPER_3(neon_qrshl_s64, i64, env, i64, i64)
+DEF_HELPER_FLAGS_5(neon_sqshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(neon_sqshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(neon_sqshl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(neon_sqshl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(neon_uqshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(neon_uqshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(neon_uqshl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(neon_uqshl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 
 DEF_HELPER_FLAGS_4(gvec_srshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_4(gvec_srshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h
index ea63ffc47b..6c6d4d49e7 100644
--- a/target/arm/tcg/translate.h
+++ b/target/arm/tcg/translate.h
@@ -463,6 +463,10 @@ void gen_gvec_srshl(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
 void gen_gvec_urshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
 uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_neon_sqshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_neon_uqshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
 
 void gen_cmtst_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b);
 void gen_ushl_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b);
diff --git a/target/arm/tcg/neon-dp.decode b/target/arm/tcg/neon-dp.decode
index 8525c65c0d..6d4996b8d8 100644
--- a/target/arm/tcg/neon-dp.decode
+++ b/target/arm/tcg/neon-dp.decode
@@ -109,14 +109,8 @@ VSHL_U_3s 001 1 0 . ..   0100 . . . 0 
 @3same_rev
 @3same_64_rev ... . . . 11    . q:1 . .  \
  &3same vm=%vn_dp vn=%vm_dp vd=%vd_dp size=3
 
-{
-  VQSHL_S64_3s    001 0 0 . ..   0100 . . . 1  @3same_64_rev
-  VQSHL_S_3s  001 0 0 . ..   0100 . . . 1  @3same_rev
-}
-{
-  VQSHL_U64_3s    001 1 0 . ..   0100 . . . 1  @3same_64_rev
-  VQSHL_U_3s  001 1 0 . ..   0100 . . . 1  @3same_rev
-}
+VQSHL_S_3s    001 0 0 . ..   0100 . . . 1  @3same_rev
+VQSHL_U_3s    001 1 0 . ..   0100 . . . 1  @3same_rev
 VRSHL_S_3s    001 0 0 . ..   0101 . . . 0  @3same_rev
 VRSHL_U_3s    001 1 0 . ..   0101 . . . 0  @3same_rev
 {
diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c
index d9a9132722..773dbf41d3 100644
--- a/target/arm/tcg/gengvec.c
+++ b/target/arm/tcg/gengvec.c
@@ -1239,6 +1239,30 @@ void gen_gvec_urshl(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 tcg_gen_gvec_3_ool(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, 0, fns[vece]);
 }
 
+void gen_neon_sqshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+static gen_helper_gvec_3_ptr * const fns[] = {
+gen_helper_neon_sqshl_b, gen_helper_neon_sqshl_h,
+gen_helper_neon_sqshl_s, gen_helper_neon_sqshl_d,
+};
+tcg_debug_assert(vece <= MO_64);
+tcg_gen_gvec_3_ptr(rd_ofs, rn_ofs, rm_ofs, tcg_env,
+   opr_sz, max_sz, 0, fns[vece]);
+}
+
+void gen_neon_uqshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+static gen_helper_gvec_3_ptr * const fns[] = {
+gen_helper_neon_uqshl_b, gen_helper_neon_uqshl_h,
+gen_helper_neon_uqshl_s, gen_helper_neon_uqshl_d,
+};
+tcg_debug_assert(vece <= MO_64);
+tcg_gen_gvec_3_ptr(rd_ofs, rn_ofs, rm_ofs, tcg_env,
+   opr_sz, max_sz, 0, fns[vece]);
+}
+
 void gen_uqadd_bhs(TCGv_i64 res, TCGv_i64 qc, TCGv_i64 a, TCGv_i64 b, MemOp 
esz)
 {
 uint64_t max = MAKE_64BIT_MASK(0, 8 << esz);
diff --git 

[PATCH v2 49/67] target/arm: Convert SQRSHL, UQRSHL to decodetree

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  4 +++
 target/arm/tcg/translate-a64.c | 48 --
 2 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 85caf37948..96ce35ad40 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -762,6 +762,8 @@ SRSHL_s 0101 1110 111 . 01010 1 . . 
@rrr_d
 URSHL_s 0111 1110 111 . 01010 1 . . @rrr_d
 SQSHL_s 0101 1110 ..1 . 01001 1 . . @rrr_e
 UQSHL_s 0111 1110 ..1 . 01001 1 . . @rrr_e
+SQRSHL_s0101 1110 ..1 . 01011 1 . . @rrr_e
+UQRSHL_s0111 1110 ..1 . 01011 1 . . @rrr_e
 
 ### Advanced SIMD scalar pairwise
 
@@ -890,6 +892,8 @@ SRSHL_v 0.00 1110 ..1 . 01010 1 . . 
@qrrr_e
 URSHL_v 0.10 1110 ..1 . 01010 1 . . @qrrr_e
 SQSHL_v 0.00 1110 ..1 . 01001 1 . . @qrrr_e
 UQSHL_v 0.10 1110 ..1 . 01001 1 . . @qrrr_e
+SQRSHL_v0.00 1110 ..1 . 01011 1 . . @qrrr_e
+UQRSHL_v0.10 1110 ..1 . 01011 1 . . @qrrr_e
 
 ### Advanced SIMD scalar x indexed element
 
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index b0004e2c6f..b76682cabf 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5162,6 +5162,22 @@ static const ENVScalar2 f_scalar_uqshl = {
 };
 TRANS(UQSHL_s, do_env_scalar2, a, &f_scalar_uqshl)
 
+static const ENVScalar2 f_scalar_sqrshl = {
+{ gen_helper_neon_qrshl_s8,
+  gen_helper_neon_qrshl_s16,
+  gen_helper_neon_qrshl_s32 },
+gen_helper_neon_qrshl_s64,
+};
+TRANS(SQRSHL_s, do_env_scalar2, a, &f_scalar_sqrshl)
+
+static const ENVScalar2 f_scalar_uqrshl = {
+{ gen_helper_neon_qrshl_u8,
+  gen_helper_neon_qrshl_u16,
+  gen_helper_neon_qrshl_u32 },
+gen_helper_neon_qrshl_u64,
+};
+TRANS(UQRSHL_s, do_env_scalar2, a, &f_scalar_uqrshl)
+
 static bool do_fp3_vector(DisasContext *s, arg_qrrr_e *a,
   gen_helper_gvec_3_ptr * const fns[3])
 {
@@ -5413,6 +5429,8 @@ TRANS(SRSHL_v, do_gvec_fn3, a, gen_gvec_srshl)
 TRANS(URSHL_v, do_gvec_fn3, a, gen_gvec_urshl)
 TRANS(SQSHL_v, do_gvec_fn3, a, gen_neon_sqshl)
 TRANS(UQSHL_v, do_gvec_fn3, a, gen_neon_uqshl)
+TRANS(SQRSHL_v, do_gvec_fn3, a, gen_neon_sqrshl)
+TRANS(UQRSHL_v, do_gvec_fn3, a, gen_neon_uqrshl)
 
 
 /*
@@ -9426,13 +9444,6 @@ static void handle_3same_64(DisasContext *s, int opcode, 
bool u,
 }
 gen_cmtst_i64(tcg_rd, tcg_rn, tcg_rm);
 break;
-case 0xb: /* SQRSHL, UQRSHL */
-if (u) {
-gen_helper_neon_qrshl_u64(tcg_rd, tcg_env, tcg_rn, tcg_rm);
-} else {
-gen_helper_neon_qrshl_s64(tcg_rd, tcg_env, tcg_rn, tcg_rm);
-}
-break;
 case 0x10: /* ADD, SUB */
 if (u) {
 tcg_gen_sub_i64(tcg_rd, tcg_rn, tcg_rm);
@@ -9446,6 +9457,7 @@ static void handle_3same_64(DisasContext *s, int opcode, 
bool u,
 case 0x8: /* SSHL, USHL */
 case 0x9: /* SQSHL, UQSHL */
 case 0xa: /* SRSHL, URSHL */
+case 0xb: /* SQRSHL, UQRSHL */
 g_assert_not_reached();
 }
 }
@@ -9467,8 +9479,6 @@ static void disas_simd_scalar_three_reg_same(DisasContext 
*s, uint32_t insn)
 TCGv_i64 tcg_rd;
 
 switch (opcode) {
-case 0xb: /* SQRSHL, UQRSHL */
-break;
 case 0x6: /* CMGT, CMHI */
 case 0x7: /* CMGE, CMHS */
 case 0x11: /* CMTST, CMEQ */
@@ -9490,6 +9500,7 @@ static void disas_simd_scalar_three_reg_same(DisasContext 
*s, uint32_t insn)
 case 0x8: /* SSHL, USHL */
 case 0x9: /* SQSHL, UQSHL */
 case 0xa: /* SRSHL, URSHL */
+case 0xb: /* SQRSHL, UQRSHL */
 unallocated_encoding(s);
 return;
 }
@@ -9516,16 +9527,6 @@ static void 
disas_simd_scalar_three_reg_same(DisasContext *s, uint32_t insn)
 void (*genfn)(TCGv_i64, TCGv_i64, TCGv_i64, TCGv_i64, MemOp) = NULL;
 
 switch (opcode) {
-case 0xb: /* SQRSHL, UQRSHL */
-{
-static NeonGenTwoOpEnvFn * const fns[3][2] = {
-{ gen_helper_neon_qrshl_s8, gen_helper_neon_qrshl_u8 },
-{ gen_helper_neon_qrshl_s16, gen_helper_neon_qrshl_u16 },
-{ gen_helper_neon_qrshl_s32, gen_helper_neon_qrshl_u32 },
-};
-genenvfn = fns[size][u];
-break;
-}
 case 0x16: /* SQDMULH, SQRDMULH */
 {
 static NeonGenTwoOpEnvFn * const fns[2][2] = {
@@ -9540,6 +9541,7 @@ static void disas_simd_scalar_three_reg_same(DisasContext 
*s, uint32_t insn)
 case 0x1: /* SQADD, UQADD */
 case 0x5: /* SQSUB, UQSUB */
 case 0x9: /* SQSHL, UQSHL */
+case 0xb: /* SQRSHL, UQRSHL */
 g_assert_not_reached();
 }
 
@@ -10965,13 +10967,6 @@ static void disas_simd_3same_int(DisasContext *

[PATCH v2 55/67] target/arm: Convert SHADD, UHADD to decodetree

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  2 ++
 target/arm/tcg/translate-a64.c | 11 +++
 2 files changed, 5 insertions(+), 8 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 3061e26242..e33d91fd0a 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -912,6 +912,8 @@ CMGE_v  0.00 1110 ..1 . 00111 1 . . 
@qrrr_e
 CMHS_v  0.10 1110 ..1 . 00111 1 . . @qrrr_e
 CMTST_v 0.00 1110 ..1 . 10001 1 . . @qrrr_e
 CMEQ_v  0.10 1110 ..1 . 10001 1 . . @qrrr_e
+SHADD_v 0.00 1110 ..1 . 0 1 . . @qrrr_e
+UHADD_v 0.10 1110 ..1 . 0 1 . . @qrrr_e
 
 ### Advanced SIMD scalar x indexed element
 
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 5f3423513d..00c04425c1 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5454,6 +5454,8 @@ TRANS(UQRSHL_v, do_gvec_fn3, a, gen_neon_uqrshl)
 
 TRANS(ADD_v, do_gvec_fn3, a, tcg_gen_gvec_add)
 TRANS(SUB_v, do_gvec_fn3, a, tcg_gen_gvec_sub)
+TRANS(SHADD_v, do_gvec_fn3_no64, a, gen_gvec_shadd)
+TRANS(UHADD_v, do_gvec_fn3_no64, a, gen_gvec_uhadd)
 
 static bool do_cmop_v(DisasContext *s, arg_qrrr_e *a, TCGCond cond)
 {
@@ -10920,7 +10922,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 return;
 }
 /* fall through */
-case 0x0: /* SHADD, UHADD */
 case 0x2: /* SRHADD, URHADD */
 case 0x4: /* SHSUB, UHSUB */
 case 0xc: /* SMAX, UMAX */
@@ -10946,6 +10947,7 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 }
 break;
 
+case 0x0: /* SHADD, UHADD */
 case 0x01: /* SQADD, UQADD */
 case 0x05: /* SQSUB, UQSUB */
 case 0x06: /* CMGT, CMHI */
@@ -10965,13 +10967,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 }
 
 switch (opcode) {
-case 0x00: /* SHADD, UHADD */
-if (u) {
-gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_uhadd, size);
-} else {
-gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_shadd, size);
-}
-return;
 case 0x0c: /* SMAX, UMAX */
 if (u) {
 gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_umax, size);
-- 
2.34.1

[PATCH v2 62/67] target/arm: Convert MUL, PMUL to decodetree

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  5 
 target/arm/tcg/translate-a64.c | 51 +-
 2 files changed, 25 insertions(+), 31 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 07b604ec30..3ea0643370 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -926,6 +926,8 @@ SABD_v  0.00 1110 ..1 . 01110 1 . . 
@qrrr_e
 UABD_v  0.10 1110 ..1 . 01110 1 . . @qrrr_e
 SABA_v  0.00 1110 ..1 . 0 1 . . @qrrr_e
 UABA_v  0.10 1110 ..1 . 0 1 . . @qrrr_e
+MUL_v   0.00 1110 ..1 . 10011 1 . . @qrrr_e
+PMUL_v  0.10 1110 001 . 10011 1 . . @qrrr_b
 
 ### Advanced SIMD scalar x indexed element
 
@@ -967,3 +969,6 @@ FMLAL_vi0.00  10 ..   . 0 . .   
@qrrx_h
 FMLSL_vi0.00  10 ..  0100 . 0 . .   @qrrx_h
 FMLAL2_vi   0.10  10 ..  1000 . 0 . .   @qrrx_h
 FMLSL2_vi   0.10  10 ..  1100 . 0 . .   @qrrx_h
+
+MUL_vi  0.00  01 ..  1000 . 0 . .   @qrrx_h
+MUL_vi  0.00  10 . . 1000 . 0 . .   @qrrx_s
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 61afbc434f..1909d1426c 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5468,6 +5468,8 @@ TRANS(SABA_v, do_gvec_fn3_no64, a, gen_gvec_saba)
 TRANS(UABA_v, do_gvec_fn3_no64, a, gen_gvec_uaba)
 TRANS(SABD_v, do_gvec_fn3_no64, a, gen_gvec_sabd)
 TRANS(UABD_v, do_gvec_fn3_no64, a, gen_gvec_uabd)
+TRANS(MUL_v, do_gvec_fn3_no64, a, tcg_gen_gvec_mul)
+TRANS(PMUL_v, do_gvec_op3_ool, a, 0, gen_helper_gvec_pmul_b)
 
 static bool do_cmop_v(DisasContext *s, arg_qrrr_e *a, TCGCond cond)
 {
@@ -5694,6 +5696,22 @@ TRANS_FEAT(FMLSL_vi, aa64_fhm, do_fmlal_idx, a, true, 
false)
 TRANS_FEAT(FMLAL2_vi, aa64_fhm, do_fmlal_idx, a, false, true)
 TRANS_FEAT(FMLSL2_vi, aa64_fhm, do_fmlal_idx, a, true, true)
 
+static bool do_int3_vector_idx(DisasContext *s, arg_qrrx_e *a,
+   gen_helper_gvec_3 * const fns[2])
+{
+assert(a->esz == MO_16 || a->esz == MO_32);
+if (fp_access_check(s)) {
+gen_gvec_op3_ool(s, a->q, a->rd, a->rn, a->rm, a->idx, fns[a->esz - 
1]);
+}
+return true;
+}
+
+static gen_helper_gvec_3 * const f_vector_idx_mul[2] = {
+gen_helper_gvec_mul_idx_h,
+gen_helper_gvec_mul_idx_s,
+};
+TRANS(MUL_vi, do_int3_vector_idx, a, f_vector_idx_mul)
+
 /*
  * Advanced SIMD scalar pairwise
  */
@@ -10927,12 +10945,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 int rd = extract32(insn, 0, 5);
 
 switch (opcode) {
-case 0x13: /* MUL, PMUL */
-if (u && size != 0) {
-unallocated_encoding(s);
-return;
-}
-/* fall through */
 case 0x12: /* MLA, MLS */
 if (size == 3) {
 unallocated_encoding(s);
@@ -10969,6 +10981,7 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 case 0x0f: /* SABA, UABA */
 case 0x10: /* ADD, SUB */
 case 0x11: /* CMTST, CMEQ */
+case 0x13: /* MUL, PMUL */
 unallocated_encoding(s);
 return;
 }
@@ -10978,13 +10991,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 }
 
 switch (opcode) {
-case 0x13: /* MUL, PMUL */
-if (!u) { /* MUL */
-gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_mul, size);
-} else {  /* PMUL */
-gen_gvec_op3_ool(s, is_q, rd, rn, rm, 0, gen_helper_gvec_pmul_b);
-}
-return;
 case 0x12: /* MLA, MLS */
 if (u) {
 gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_mls, size);
@@ -12198,7 +12204,6 @@ static void disas_simd_indexed(DisasContext *s, 
uint32_t insn)
 TCGv_ptr fpst;
 
 switch (16 * u + opcode) {
-case 0x08: /* MUL */
 case 0x10: /* MLA */
 case 0x14: /* MLS */
 if (is_scalar) {
@@ -12285,6 +12290,7 @@ static void disas_simd_indexed(DisasContext *s, 
uint32_t insn)
 case 0x01: /* FMLA */
 case 0x04: /* FMLSL */
 case 0x05: /* FMLS */
+case 0x08: /* MUL */
 case 0x09: /* FMUL */
 case 0x18: /* FMLAL2 */
 case 0x19: /* FMULX */
@@ -12407,22 +12413,6 @@ static void disas_simd_indexed(DisasContext *s, 
uint32_t insn)
 }
 return;
 
-case 0x08: /* MUL */
-if (!is_long && !is_scalar) {
-static gen_helper_gvec_3 * const fns[3] = {
-gen_helper_gvec_mul_idx_h,
-gen_helper_gvec_mul_idx_s,
-gen_helper_gvec_mul_idx_d,
-};
-tcg_gen_gvec_3_ool(vec_full_reg_offset(s, rd),
-   vec_full_reg_offset(s, rn),
-   vec_full_reg_offset(s, rm),
-   is_q ? 16 : 8, vec_full_reg_size(s),
- 

[PATCH v2 52/67] target/arm: Use TCG_COND_TSTNE in gen_cmtst_{i32, i64}

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/gengvec.c | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c
index 51e66ccf5f..1d6bc6021d 100644
--- a/target/arm/tcg/gengvec.c
+++ b/target/arm/tcg/gengvec.c
@@ -933,14 +933,12 @@ void gen_gvec_mls(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 /* CMTST : test is "if (X & Y != 0)". */
 static void gen_cmtst_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b)
 {
-tcg_gen_and_i32(d, a, b);
-tcg_gen_negsetcond_i32(TCG_COND_NE, d, d, tcg_constant_i32(0));
+tcg_gen_negsetcond_i32(TCG_COND_TSTNE, d, a, b);
 }
 
 void gen_cmtst_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
 {
-tcg_gen_and_i64(d, a, b);
-tcg_gen_negsetcond_i64(TCG_COND_NE, d, d, tcg_constant_i64(0));
+tcg_gen_negsetcond_i64(TCG_COND_TSTNE, d, a, b);
 }
 
 static void gen_cmtst_vec(unsigned vece, TCGv_vec d, TCGv_vec a, TCGv_vec b)
-- 
2.34.1

[PATCH v2 47/67] target/arm: Convert SQSHL, UQSHL to decodetree

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  4 ++
 target/arm/tcg/translate-a64.c | 74 ++
 2 files changed, 53 insertions(+), 25 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 9e02776036..85caf37948 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -760,6 +760,8 @@ SSHL_s  0101 1110 111 . 01000 1 . . 
@rrr_d
 USHL_s  0111 1110 111 . 01000 1 . . @rrr_d
 SRSHL_s 0101 1110 111 . 01010 1 . . @rrr_d
 URSHL_s 0111 1110 111 . 01010 1 . . @rrr_d
+SQSHL_s 0101 1110 ..1 . 01001 1 . . @rrr_e
+UQSHL_s 0111 1110 ..1 . 01001 1 . . @rrr_e
 
 ### Advanced SIMD scalar pairwise
 
@@ -886,6 +888,8 @@ SSHL_v  0.00 1110 ..1 . 01000 1 . . 
@qrrr_e
 USHL_v  0.10 1110 ..1 . 01000 1 . . @qrrr_e
 SRSHL_v 0.00 1110 ..1 . 01010 1 . . @qrrr_e
 URSHL_v 0.10 1110 ..1 . 01010 1 . . @qrrr_e
+SQSHL_v 0.00 1110 ..1 . 01001 1 . . @qrrr_e
+UQSHL_v 0.10 1110 ..1 . 01001 1 . . @qrrr_e
 
 ### Advanced SIMD scalar x indexed element
 
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 50b653bb4d..f8d2760bea 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5119,6 +5119,49 @@ TRANS(USHL_s, do_int3_scalar_d, a, gen_ushl_i64)
 TRANS(SRSHL_s, do_int3_scalar_d, a, gen_helper_neon_rshl_s64)
 TRANS(URSHL_s, do_int3_scalar_d, a, gen_helper_neon_rshl_u64)
 
+typedef struct ENVScalar2 {
+NeonGenTwoOpEnvFn *gen_bhs[3];
+NeonGenTwo64OpEnvFn *gen_d;
+} ENVScalar2;
+
+static bool do_env_scalar2(DisasContext *s, arg_rrr_e *a, const ENVScalar2 *f)
+{
+if (!fp_access_check(s)) {
+return true;
+}
+if (a->esz == MO_64) {
+TCGv_i64 t0 = read_fp_dreg(s, a->rn);
+TCGv_i64 t1 = read_fp_dreg(s, a->rm);
+f->gen_d(t0, tcg_env, t0, t1);
+write_fp_dreg(s, a->rd, t0);
+} else {
+TCGv_i32 t0 = tcg_temp_new_i32();
+TCGv_i32 t1 = tcg_temp_new_i32();
+
+read_vec_element_i32(s, t0, a->rn, 0, a->esz);
+read_vec_element_i32(s, t1, a->rm, 0, a->esz);
+f->gen_bhs[a->esz](t0, tcg_env, t0, t1);
+write_fp_sreg(s, a->rd, t0);
+}
+return true;
+}
+
+static const ENVScalar2 f_scalar_sqshl = {
+{ gen_helper_neon_qshl_s8,
+  gen_helper_neon_qshl_s16,
+  gen_helper_neon_qshl_s32 },
+gen_helper_neon_qshl_s64,
+};
+TRANS(SQSHL_s, do_env_scalar2, a, &f_scalar_sqshl)
+
+static const ENVScalar2 f_scalar_uqshl = {
+{ gen_helper_neon_qshl_u8,
+  gen_helper_neon_qshl_u16,
+  gen_helper_neon_qshl_u32 },
+gen_helper_neon_qshl_u64,
+};
+TRANS(UQSHL_s, do_env_scalar2, a, &f_scalar_uqshl)
+
 static bool do_fp3_vector(DisasContext *s, arg_qrrr_e *a,
   gen_helper_gvec_3_ptr * const fns[3])
 {
@@ -5368,6 +5411,8 @@ TRANS(SSHL_v, do_gvec_fn3, a, gen_gvec_sshl)
 TRANS(USHL_v, do_gvec_fn3, a, gen_gvec_ushl)
 TRANS(SRSHL_v, do_gvec_fn3, a, gen_gvec_srshl)
 TRANS(URSHL_v, do_gvec_fn3, a, gen_gvec_urshl)
+TRANS(SQSHL_v, do_gvec_fn3, a, gen_neon_sqshl)
+TRANS(UQSHL_v, do_gvec_fn3, a, gen_neon_uqshl)
 
 
 /*
@@ -9381,13 +9426,6 @@ static void handle_3same_64(DisasContext *s, int opcode, 
bool u,
 }
 gen_cmtst_i64(tcg_rd, tcg_rn, tcg_rm);
 break;
-case 0x9: /* SQSHL, UQSHL */
-if (u) {
-gen_helper_neon_qshl_u64(tcg_rd, tcg_env, tcg_rn, tcg_rm);
-} else {
-gen_helper_neon_qshl_s64(tcg_rd, tcg_env, tcg_rn, tcg_rm);
-}
-break;
 case 0xb: /* SQRSHL, UQRSHL */
 if (u) {
 gen_helper_neon_qrshl_u64(tcg_rd, tcg_env, tcg_rn, tcg_rm);
@@ -9406,6 +9444,7 @@ static void handle_3same_64(DisasContext *s, int opcode, 
bool u,
 case 0x1: /* SQADD / UQADD */
 case 0x5: /* SQSUB / UQSUB */
 case 0x8: /* SSHL, USHL */
+case 0x9: /* SQSHL, UQSHL */
 case 0xa: /* SRSHL, URSHL */
 g_assert_not_reached();
 }
@@ -9428,7 +9467,6 @@ static void disas_simd_scalar_three_reg_same(DisasContext 
*s, uint32_t insn)
 TCGv_i64 tcg_rd;
 
 switch (opcode) {
-case 0x9: /* SQSHL, UQSHL */
 case 0xb: /* SQRSHL, UQRSHL */
 break;
 case 0x6: /* CMGT, CMHI */
@@ -9450,6 +9488,7 @@ static void disas_simd_scalar_three_reg_same(DisasContext 
*s, uint32_t insn)
 case 0x1: /* SQADD, UQADD */
 case 0x5: /* SQSUB, UQSUB */
 case 0x8: /* SSHL, USHL */
+case 0x9: /* SQSHL, UQSHL */
 case 0xa: /* SRSHL, URSHL */
 unallocated_encoding(s);
 return;
@@ -9477,16 +9516,6 @@ static void 
disas_simd_scalar_three_reg_same(DisasContext *s, uint32_t insn)
 void (*genfn)(TCGv_i64, TCGv_i64, TCGv_i64, TCGv_i64, MemOp) = NULL;
 
 switch (opcode) {
-case 0x9: /* 

[PATCH v2 63/67] target/arm: Convert MLA, MLS to decodetree

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  8 
 target/arm/tcg/translate-a64.c | 77 ++
 2 files changed, 31 insertions(+), 54 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 3ea0643370..2dea68a0a9 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -928,6 +928,8 @@ SABA_v  0.00 1110 ..1 . 0 1 . . 
@qrrr_e
 UABA_v  0.10 1110 ..1 . 0 1 . . @qrrr_e
 MUL_v   0.00 1110 ..1 . 10011 1 . . @qrrr_e
 PMUL_v  0.10 1110 001 . 10011 1 . . @qrrr_b
+MLA_v   0.00 1110 ..1 . 10010 1 . . @qrrr_e
+MLS_v   0.10 1110 ..1 . 10010 1 . . @qrrr_e
 
 ### Advanced SIMD scalar x indexed element
 
@@ -972,3 +974,9 @@ FMLSL2_vi   0.10  10 ..  1100 . 0 . .   
@qrrx_h
 
 MUL_vi  0.00  01 ..  1000 . 0 . .   @qrrx_h
 MUL_vi  0.00  10 . . 1000 . 0 . .   @qrrx_s
+
+MLA_vi  0.10  01 ..   . 0 . .   @qrrx_h
+MLA_vi  0.10  10 . .  . 0 . .   @qrrx_s
+
+MLS_vi  0.10  01 ..  0100 . 0 . .   @qrrx_h
+MLS_vi  0.10  10 . . 0100 . 0 . .   @qrrx_s
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 1909d1426c..c4601cde2f 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5470,6 +5470,8 @@ TRANS(SABD_v, do_gvec_fn3_no64, a, gen_gvec_sabd)
 TRANS(UABD_v, do_gvec_fn3_no64, a, gen_gvec_uabd)
 TRANS(MUL_v, do_gvec_fn3_no64, a, tcg_gen_gvec_mul)
 TRANS(PMUL_v, do_gvec_op3_ool, a, 0, gen_helper_gvec_pmul_b)
+TRANS(MLA_v, do_gvec_fn3_no64, a, gen_gvec_mla)
+TRANS(MLS_v, do_gvec_fn3_no64, a, gen_gvec_mls)
 
 static bool do_cmop_v(DisasContext *s, arg_qrrr_e *a, TCGCond cond)
 {
@@ -5712,6 +5714,24 @@ static gen_helper_gvec_3 * const f_vector_idx_mul[2] = {
 };
 TRANS(MUL_vi, do_int3_vector_idx, a, f_vector_idx_mul)
 
+static bool do_mla_vector_idx(DisasContext *s, arg_qrrx_e *a, bool sub)
+{
+static gen_helper_gvec_4 * const fns[2][2] = {
+{ gen_helper_gvec_mla_idx_h, gen_helper_gvec_mls_idx_h },
+{ gen_helper_gvec_mla_idx_s, gen_helper_gvec_mls_idx_s },
+};
+
+assert(a->esz == MO_16 || a->esz == MO_32);
+if (fp_access_check(s)) {
+gen_gvec_op4_ool(s, a->q, a->rd, a->rn, a->rm, a->rd,
+ a->idx, fns[a->esz - 1][sub]);
+}
+return true;
+}
+
+TRANS(MLA_vi, do_mla_vector_idx, a, false)
+TRANS(MLS_vi, do_mla_vector_idx, a, true)
+
 /*
  * Advanced SIMD scalar pairwise
  */
@@ -10945,12 +10965,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 int rd = extract32(insn, 0, 5);
 
 switch (opcode) {
-case 0x12: /* MLA, MLS */
-if (size == 3) {
-unallocated_encoding(s);
-return;
-}
-break;
 case 0x16: /* SQDMULH, SQRDMULH */
 if (size == 0 || size == 3) {
 unallocated_encoding(s);
@@ -10981,6 +10995,7 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 case 0x0f: /* SABA, UABA */
 case 0x10: /* ADD, SUB */
 case 0x11: /* CMTST, CMEQ */
+case 0x12: /* MLA, MLS */
 case 0x13: /* MUL, PMUL */
 unallocated_encoding(s);
 return;
@@ -10991,13 +11006,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 }
 
 switch (opcode) {
-case 0x12: /* MLA, MLS */
-if (u) {
-gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_mls, size);
-} else {
-gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_mla, size);
-}
-return;
 case 0x16: /* SQDMULH, SQRDMULH */
 {
 static gen_helper_gvec_3_ptr * const fns[2][2] = {
@@ -12204,13 +12212,6 @@ static void disas_simd_indexed(DisasContext *s, 
uint32_t insn)
 TCGv_ptr fpst;
 
 switch (16 * u + opcode) {
-case 0x10: /* MLA */
-case 0x14: /* MLS */
-if (is_scalar) {
-unallocated_encoding(s);
-return;
-}
-break;
 case 0x02: /* SMLAL, SMLAL2 */
 case 0x12: /* UMLAL, UMLAL2 */
 case 0x06: /* SMLSL, SMLSL2 */
@@ -12292,6 +12293,8 @@ static void disas_simd_indexed(DisasContext *s, 
uint32_t insn)
 case 0x05: /* FMLS */
 case 0x08: /* MUL */
 case 0x09: /* FMUL */
+case 0x10: /* MLA */
+case 0x14: /* MLS */
 case 0x18: /* FMLAL2 */
 case 0x19: /* FMULX */
 case 0x1c: /* FMLSL2 */
@@ -12412,40 +12415,6 @@ static void disas_simd_indexed(DisasContext *s, 
uint32_t insn)
: gen_helper_gvec_fcmlah_idx);
 }
 return;
-
-case 0x10: /* MLA */
-if (!is_long && !is_scalar) {
-static gen_helper_gvec_4 * const fns[3] = {
-gen_helper_gvec_mla_idx_h,
-gen_helper_gvec_m

[PATCH v2 05/67] target/arm: Fix decode of FMOV (hp) vs MOVI

[Richard Henderson]

The decode of FMOV (vector, immediate, half-precision) vs
invalid cases of MOVI are incorrect.

Fixes RISU mismatch for invalid insn 0x2f01fd31.

Fixes: 70b4e6a4457 ("arm/translate-a64: add FP16 FMOV to simd_mod_imm")
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/translate-a64.c | 24 ++--
 1 file changed, 14 insertions(+), 10 deletions(-)

diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index d97acdbaf9..5455ae3685 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -7904,27 +7904,31 @@ static void disas_simd_mod_imm(DisasContext *s, 
uint32_t insn)
 bool is_q = extract32(insn, 30, 1);
 uint64_t imm = 0;
 
-if (o2 != 0 || ((cmode == 0xf) && is_neg && !is_q)) {
-/* Check for FMOV (vector, immediate) - half-precision */
-if (!(dc_isar_feature(aa64_fp16, s) && o2 && cmode == 0xf)) {
+if (o2) {
+if (cmode != 0xf || is_neg) {
 unallocated_encoding(s);
 return;
 }
-}
-
-if (!fp_access_check(s)) {
-return;
-}
-
-if (cmode == 15 && o2 && !is_neg) {
 /* FMOV (vector, immediate) - half-precision */
+if (!dc_isar_feature(aa64_fp16, s)) {
+unallocated_encoding(s);
+return;
+}
 imm = vfp_expand_imm(MO_16, abcdefgh);
 /* now duplicate across the lanes */
 imm = dup_const(MO_16, imm);
 } else {
+if (cmode == 0xf && is_neg && !is_q) {
+unallocated_encoding(s);
+return;
+}
 imm = asimd_imm_const(abcdefgh, cmode, is_neg);
 }
 
+if (!fp_access_check(s)) {
+return;
+}
+
 if (!((cmode & 0x9) == 0x1 || (cmode & 0xd) == 0x9)) {
 /* MOVI or MVNI, with MVNI negation handled above.  */
 tcg_gen_gvec_dup_imm(MO_64, vec_full_reg_offset(s, rd), is_q ? 16 : 8,
-- 
2.34.1

[PATCH v2 38/67] target/arm: Convert SUQADD and USQADD to gvec

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/helper.h|  16 +
 target/arm/tcg/translate-a64.h |   6 ++
 target/arm/tcg/gengvec64.c | 106 +++
 target/arm/tcg/translate-a64.c | 113 ++---
 target/arm/tcg/vec_helper.c|  64 +++
 5 files changed, 241 insertions(+), 64 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index f830531dd3..de2c5c9aef 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -836,6 +836,22 @@ DEF_HELPER_FLAGS_5(gvec_sqsub_s, TCG_CALL_NO_RWG,
void, ptr, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_5(gvec_sqsub_d, TCG_CALL_NO_RWG,
void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_usqadd_b, TCG_CALL_NO_RWG,
+   void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_usqadd_h, TCG_CALL_NO_RWG,
+   void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_usqadd_s, TCG_CALL_NO_RWG,
+   void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_usqadd_d, TCG_CALL_NO_RWG,
+   void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_suqadd_b, TCG_CALL_NO_RWG,
+   void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_suqadd_h, TCG_CALL_NO_RWG,
+   void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_suqadd_s, TCG_CALL_NO_RWG,
+   void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_suqadd_d, TCG_CALL_NO_RWG,
+   void, ptr, ptr, ptr, ptr, i32)
 
 DEF_HELPER_FLAGS_5(gvec_fmlal_a32, TCG_CALL_NO_RWG,
void, ptr, ptr, ptr, ptr, i32)
diff --git a/target/arm/tcg/translate-a64.h b/target/arm/tcg/translate-a64.h
index 91750f0ca9..b5cb26f8a2 100644
--- a/target/arm/tcg/translate-a64.h
+++ b/target/arm/tcg/translate-a64.h
@@ -197,6 +197,12 @@ void gen_gvec_eor3(unsigned vece, uint32_t d, uint32_t n, 
uint32_t m,
uint32_t a, uint32_t oprsz, uint32_t maxsz);
 void gen_gvec_bcax(unsigned vece, uint32_t d, uint32_t n, uint32_t m,
uint32_t a, uint32_t oprsz, uint32_t maxsz);
+void gen_gvec_suqadd_qc(unsigned vece, uint32_t rd_ofs,
+uint32_t rn_ofs, uint32_t rm_ofs,
+uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_usqadd_qc(unsigned vece, uint32_t rd_ofs,
+uint32_t rn_ofs, uint32_t rm_ofs,
+uint32_t opr_sz, uint32_t max_sz);
 
 void gen_sve_ldr(DisasContext *s, TCGv_ptr, int vofs, int len, int rn, int 
imm);
 void gen_sve_str(DisasContext *s, TCGv_ptr, int vofs, int len, int rn, int 
imm);
diff --git a/target/arm/tcg/gengvec64.c b/target/arm/tcg/gengvec64.c
index 093b498b13..4b76e476a0 100644
--- a/target/arm/tcg/gengvec64.c
+++ b/target/arm/tcg/gengvec64.c
@@ -188,3 +188,109 @@ void gen_gvec_bcax(unsigned vece, uint32_t d, uint32_t n, 
uint32_t m,
 tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op);
 }
 
+static void gen_suqadd_vec(unsigned vece, TCGv_vec t, TCGv_vec qc,
+   TCGv_vec a, TCGv_vec b)
+{
+TCGv_vec max =
+tcg_constant_vec_matching(t, vece, (1ull << ((8 << vece) - 1)) - 1);
+TCGv_vec u = tcg_temp_new_vec_matching(t);
+
+/* Maximum value that can be added to @a without overflow. */
+tcg_gen_sub_vec(vece, u, max, a);
+
+/* Constrain addend so that the next addition never overflows. */
+tcg_gen_umin_vec(vece, u, u, b);
+tcg_gen_add_vec(vece, t, u, a);
+
+/* Compute QC by comparing the adjusted @b. */
+tcg_gen_xor_vec(vece, u, u, b);
+tcg_gen_or_vec(vece, qc, qc, u);
+}
+
+void gen_gvec_suqadd_qc(unsigned vece, uint32_t rd_ofs,
+uint32_t rn_ofs, uint32_t rm_ofs,
+uint32_t opr_sz, uint32_t max_sz)
+{
+static const TCGOpcode vecop_list[] = {
+INDEX_op_add_vec, INDEX_op_sub_vec, INDEX_op_umin_vec, 0
+};
+static const GVecGen4 ops[4] = {
+{ .fniv = gen_suqadd_vec,
+  .fno = gen_helper_gvec_suqadd_b,
+  .opt_opc = vecop_list,
+  .write_aofs = true,
+  .vece = MO_8 },
+{ .fniv = gen_suqadd_vec,
+  .fno = gen_helper_gvec_suqadd_h,
+  .opt_opc = vecop_list,
+  .write_aofs = true,
+  .vece = MO_16 },
+{ .fniv = gen_suqadd_vec,
+  .fno = gen_helper_gvec_suqadd_s,
+  .opt_opc = vecop_list,
+  .write_aofs = true,
+  .vece = MO_32 },
+{ .fniv = gen_suqadd_vec,
+  .fno = gen_helper_gvec_suqadd_d,
+  .opt_opc = vecop_list,
+  .write_aofs = true,
+  .vece = MO_64 },
+};
+tcg_gen_gvec_4(rd_ofs, offsetof(CPUARMState, vfp.qc),
+   rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
+}
+
+static void gen_usqadd_vec(unsigned vece, TCGv_vec t, TCGv_vec qc,
+   TCGv_vec a, TCGv_vec b)
+{
+TCGv_vec u = tcg_temp_new_vec_matching(t);
+TCGv_vec z

[PATCH v2 34/67] target/arm: Use gvec for neon pmax, pmin

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/translate-neon.c | 78 ++---
 1 file changed, 4 insertions(+), 74 deletions(-)

diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c
index 6c5a7a98e1..18b048611b 100644
--- a/target/arm/tcg/translate-neon.c
+++ b/target/arm/tcg/translate-neon.c
@@ -831,6 +831,10 @@ DO_3SAME_NO_SZ_3(VABA_S, gen_gvec_saba)
 DO_3SAME_NO_SZ_3(VABD_U, gen_gvec_uabd)
 DO_3SAME_NO_SZ_3(VABA_U, gen_gvec_uaba)
 DO_3SAME_NO_SZ_3(VPADD, gen_gvec_addp)
+DO_3SAME_NO_SZ_3(VPMAX_S, gen_gvec_smaxp)
+DO_3SAME_NO_SZ_3(VPMIN_S, gen_gvec_sminp)
+DO_3SAME_NO_SZ_3(VPMAX_U, gen_gvec_umaxp)
+DO_3SAME_NO_SZ_3(VPMIN_U, gen_gvec_uminp)
 
 #define DO_3SAME_CMP(INSN, COND)\
 static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \
@@ -1003,80 +1007,6 @@ DO_3SAME_32_ENV(VQSHL_U, qshl_u)
 DO_3SAME_32_ENV(VQRSHL_S, qrshl_s)
 DO_3SAME_32_ENV(VQRSHL_U, qrshl_u)
 
-static bool do_3same_pair(DisasContext *s, arg_3same *a, NeonGenTwoOpFn *fn)
-{
-/* Operations handled pairwise 32 bits at a time */
-TCGv_i32 tmp, tmp2, tmp3;
-
-if (!arm_dc_feature(s, ARM_FEATURE_NEON)) {
-return false;
-}
-
-/* UNDEF accesses to D16-D31 if they don't exist. */
-if (!dc_isar_feature(aa32_simd_r32, s) &&
-((a->vd | a->vn | a->vm) & 0x10)) {
-return false;
-}
-
-if (a->size == 3) {
-return false;
-}
-
-if (!vfp_access_check(s)) {
-return true;
-}
-
-assert(a->q == 0); /* enforced by decode patterns */
-
-/*
- * Note that we have to be careful not to clobber the source operands
- * in the "vm == vd" case by storing the result of the first pass too
- * early. Since Q is 0 there are always just two passes, so instead
- * of a complicated loop over each pass we just unroll.
- */
-tmp = tcg_temp_new_i32();
-tmp2 = tcg_temp_new_i32();
-tmp3 = tcg_temp_new_i32();
-
-read_neon_element32(tmp, a->vn, 0, MO_32);
-read_neon_element32(tmp2, a->vn, 1, MO_32);
-fn(tmp, tmp, tmp2);
-
-read_neon_element32(tmp3, a->vm, 0, MO_32);
-read_neon_element32(tmp2, a->vm, 1, MO_32);
-fn(tmp3, tmp3, tmp2);
-
-write_neon_element32(tmp, a->vd, 0, MO_32);
-write_neon_element32(tmp3, a->vd, 1, MO_32);
-
-return true;
-}
-
-#define DO_3SAME_PAIR(INSN, func)   \
-static bool trans_##INSN##_3s(DisasContext *s, arg_3same *a)\
-{   \
-static NeonGenTwoOpFn * const fns[] = { \
-gen_helper_neon_##func##8,  \
-gen_helper_neon_##func##16, \
-gen_helper_neon_##func##32, \
-};  \
-if (a->size > 2) {  \
-return false;   \
-}   \
-return do_3same_pair(s, a, fns[a->size]);   \
-}
-
-/* 32-bit pairwise ops end up the same as the elementwise versions.  */
-#define gen_helper_neon_pmax_s32  tcg_gen_smax_i32
-#define gen_helper_neon_pmax_u32  tcg_gen_umax_i32
-#define gen_helper_neon_pmin_s32  tcg_gen_smin_i32
-#define gen_helper_neon_pmin_u32  tcg_gen_umin_i32
-
-DO_3SAME_PAIR(VPMAX_S, pmax_s)
-DO_3SAME_PAIR(VPMIN_S, pmin_s)
-DO_3SAME_PAIR(VPMAX_U, pmax_u)
-DO_3SAME_PAIR(VPMIN_U, pmin_u)
-
 #define DO_3SAME_VQDMULH(INSN, FUNC)\
 WRAP_ENV_FN(gen_##INSN##_tramp16, gen_helper_neon_##FUNC##_s16);\
 WRAP_ENV_FN(gen_##INSN##_tramp32, gen_helper_neon_##FUNC##_s32);\
-- 
2.34.1

[PATCH v2 42/67] target/arm: Convert SUQADD, USQADD to decodetree

[Richard Henderson]

These are faux 2-operand instructions, reading from rd.
Sort them next to the other three-operand same insns for clarity.

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  8 +
 target/arm/tcg/translate-a64.c | 64 --
 2 files changed, 14 insertions(+), 58 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 19010af03b..7c350ba833 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -45,6 +45,7 @@
 @rrr_sd  ... rm:5 .. rn:5 rd:5  &rrr_e esz=%esz_sd
 @rrr_hsd ... rm:5 .. rn:5 rd:5  &rrr_e esz=%esz_hsd
 @rrr_e   esz:2 . rm:5 .. rn:5 rd:5  &rrr_e
+@r2r_e   esz:2 . . .. rm:5 rd:5 &rrr_e rn=%rd
 
 @rrx_h   .. .. rm:4  . . rn:5 rd:5  &rrx_e esz=1 idx=%hlm
 @rrx_s   .. . rm:5   . . rn:5 rd:5  &rrx_e esz=2 idx=%hl
@@ -60,6 +61,7 @@
 @qrrr_h . q:1 .. ... rm:5 .. rn:5 rd:5  &qrrr_e esz=1
 @qrrr_sd. q:1 .. ... rm:5 .. rn:5 rd:5  &qrrr_e esz=%esz_sd
 @qrrr_e . q:1 .. esz:2 . rm:5 .. rn:5 rd:5  &qrrr_e
+@qr2r_e . q:1 .. esz:2 . . .. rm:5 rd:5 &qrrr_e rn=%rd
 
 @qrrx_h . q:1 ..  .. .. rm:4  . . rn:5 rd:5 \
 &qrrx_e esz=1 idx=%hlm
@@ -750,6 +752,9 @@ UQADD_s 0111 1110 ..1 . 1 1 . . 
@rrr_e
 SQSUB_s 0101 1110 ..1 . 00101 1 . . @rrr_e
 UQSUB_s 0111 1110 ..1 . 00101 1 . . @rrr_e
 
+SUQADD_s0101 1110 ..1 0 00111 0 . . @r2r_e
+USQADD_s0111 1110 ..1 0 00111 0 . . @r2r_e
+
 ### Advanced SIMD scalar pairwise
 
 FADDP_s 0101 1110 0011  1101 10 . . @rr_h
@@ -868,6 +873,9 @@ UQADD_v 0.10 1110 ..1 . 1 1 . . 
@qrrr_e
 SQSUB_v 0.00 1110 ..1 . 00101 1 . . @qrrr_e
 UQSUB_v 0.10 1110 ..1 . 00101 1 . . @qrrr_e
 
+SUQADD_v0.00 1110 ..1 0 00111 0 . . @qr2r_e
+USQADD_v0.10 1110 ..1 0 00111 0 . . @qr2r_e
+
 ### Advanced SIMD scalar x indexed element
 
 FMUL_si 0101  00 ..  1001 . 0 . .   @rrx_h
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 2f7298811d..fbcf18f92a 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5096,6 +5096,8 @@ TRANS(SQADD_s, do_satacc_s, a, MO_SIGN, MO_SIGN, 
gen_sqadd_bhs, gen_sqadd_d)
 TRANS(SQSUB_s, do_satacc_s, a, MO_SIGN, MO_SIGN, gen_sqsub_bhs, gen_sqsub_d)
 TRANS(UQADD_s, do_satacc_s, a, 0, 0, gen_uqadd_bhs, gen_uqadd_d)
 TRANS(UQSUB_s, do_satacc_s, a, 0, 0, gen_uqsub_bhs, gen_uqsub_d)
+TRANS(SUQADD_s, do_satacc_s, a, MO_SIGN, 0, gen_suqadd_bhs, gen_suqadd_d)
+TRANS(USQADD_s, do_satacc_s, a, 0, MO_SIGN, gen_usqadd_bhs, gen_usqadd_d)
 
 static bool do_fp3_vector(DisasContext *s, arg_qrrr_e *a,
   gen_helper_gvec_3_ptr * const fns[3])
@@ -5339,6 +5341,8 @@ TRANS(SQADD_v, do_gvec_fn3, a, gen_gvec_sqadd_qc)
 TRANS(UQADD_v, do_gvec_fn3, a, gen_gvec_uqadd_qc)
 TRANS(SQSUB_v, do_gvec_fn3, a, gen_gvec_sqsub_qc)
 TRANS(UQSUB_v, do_gvec_fn3, a, gen_gvec_uqsub_qc)
+TRANS(SUQADD_v, do_gvec_fn3, a, gen_gvec_suqadd_qc)
+TRANS(USQADD_v, do_gvec_fn3, a, gen_gvec_usqadd_qc)
 
 /*
  * Advanced SIMD scalar/vector x indexed element
@@ -10009,48 +10013,6 @@ static void handle_2misc_narrow(DisasContext *s, bool 
scalar,
 clear_vec_high(s, is_q, rd);
 }
 
-/* Remaining saturating accumulating ops */
-static void handle_2misc_satacc(DisasContext *s, bool is_scalar, bool is_u,
-bool is_q, unsigned size, int rn, int rd)
-{
-TCGv_i64 res, qc, a, b;
-
-if (!is_scalar) {
-gen_gvec_fn3(s, is_q, rd, rd, rn,
- is_u ? gen_gvec_usqadd_qc : gen_gvec_suqadd_qc, size);
-return;
-}
-
-res = tcg_temp_new_i64();
-qc = tcg_temp_new_i64();
-a = tcg_temp_new_i64();
-b = tcg_temp_new_i64();
-
-/* Read and extend scalar inputs to 64-bits. */
-read_vec_element(s, a, rd, 0, size | (is_u ? 0 : MO_SIGN));
-read_vec_element(s, b, rn, 0, size | (is_u ? MO_SIGN : 0));
-tcg_gen_ld_i64(qc, tcg_env, offsetof(CPUARMState, vfp.qc));
-
-if (size == MO_64) {
-if (is_u) {
-gen_usqadd_d(res, qc, a, b);
-} else {
-gen_suqadd_d(res, qc, a, b);
-}
-} else {
-if (is_u) {
-gen_usqadd_bhs(res, qc, a, b, size);
-} else {
-gen_suqadd_bhs(res, qc, a, b, size);
-/* Truncate signed 64-bit result for writeback. */
-tcg_gen_ext_i64(res, res, size);
-}
-}
-
-write_fp_dreg(s, rd, res);
-tcg_gen_st_i64(qc, tcg_env, offsetof(CPUARMState, vfp.qc));
-}
-
 /* AdvSIMD scalar two reg misc
  *  31 30  29 28   24 23  22 21   17 1612 11 10 95 40
  * +

[PATCH v2 21/67] target/arm: Introduce vfp_load_reg16

[Richard Henderson]

Load and zero-extend float16 into a TCGv_i32 before
all scalar operations.

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/translate-vfp.c | 39 +++---
 1 file changed, 22 insertions(+), 17 deletions(-)

diff --git a/target/arm/tcg/translate-vfp.c b/target/arm/tcg/translate-vfp.c
index b9af03b7c3..8e755fcde8 100644
--- a/target/arm/tcg/translate-vfp.c
+++ b/target/arm/tcg/translate-vfp.c
@@ -48,6 +48,12 @@ static inline void vfp_store_reg32(TCGv_i32 var, int reg)
 tcg_gen_st_i32(var, tcg_env, vfp_reg_offset(false, reg));
 }
 
+static inline void vfp_load_reg16(TCGv_i32 var, int reg)
+{
+tcg_gen_ld16u_i32(var, tcg_env,
+  vfp_reg_offset(false, reg) + HOST_BIG_ENDIAN * 2);
+}
+
 /*
  * The imm8 encodes the sign bit, enough bits to represent an exponent in
  * the range 011xx to 100xx, and the most significant 4 bits of
@@ -902,8 +908,7 @@ static bool trans_VMOV_half(DisasContext *s, 
arg_VMOV_single *a)
 if (a->l) {
 /* VFP to general purpose register */
 tmp = tcg_temp_new_i32();
-vfp_load_reg32(tmp, a->vn);
-tcg_gen_andi_i32(tmp, tmp, 0x);
+vfp_load_reg16(tmp, a->vn);
 store_reg(s, a->rt, tmp);
 } else {
 /* general purpose register to VFP */
@@ -1453,11 +1458,11 @@ static bool do_vfp_3op_hp(DisasContext *s, 
VFPGen3OpSPFn *fn,
 fd = tcg_temp_new_i32();
 fpst = fpstatus_ptr(FPST_FPCR_F16);
 
-vfp_load_reg32(f0, vn);
-vfp_load_reg32(f1, vm);
+vfp_load_reg16(f0, vn);
+vfp_load_reg16(f1, vm);
 
 if (reads_vd) {
-vfp_load_reg32(fd, vd);
+vfp_load_reg16(fd, vd);
 }
 fn(fd, f0, f1, fpst);
 vfp_store_reg32(fd, vd);
@@ -1633,7 +1638,7 @@ static bool do_vfp_2op_hp(DisasContext *s, VFPGen2OpSPFn 
*fn, int vd, int vm)
 }
 
 f0 = tcg_temp_new_i32();
-vfp_load_reg32(f0, vm);
+vfp_load_reg16(f0, vm);
 fn(f0, f0);
 vfp_store_reg32(f0, vd);
 
@@ -2106,13 +2111,13 @@ static bool do_vfm_hp(DisasContext *s, arg_VFMA_sp *a, 
bool neg_n, bool neg_d)
 vm = tcg_temp_new_i32();
 vd = tcg_temp_new_i32();
 
-vfp_load_reg32(vn, a->vn);
-vfp_load_reg32(vm, a->vm);
+vfp_load_reg16(vn, a->vn);
+vfp_load_reg16(vm, a->vm);
 if (neg_n) {
 /* VFNMS, VFMS */
 gen_helper_vfp_negh(vn, vn);
 }
-vfp_load_reg32(vd, a->vd);
+vfp_load_reg16(vd, a->vd);
 if (neg_d) {
 /* VFNMA, VFNMS */
 gen_helper_vfp_negh(vd, vd);
@@ -2456,11 +2461,11 @@ static bool trans_VCMP_hp(DisasContext *s, arg_VCMP_sp 
*a)
 vd = tcg_temp_new_i32();
 vm = tcg_temp_new_i32();
 
-vfp_load_reg32(vd, a->vd);
+vfp_load_reg16(vd, a->vd);
 if (a->z) {
 tcg_gen_movi_i32(vm, 0);
 } else {
-vfp_load_reg32(vm, a->vm);
+vfp_load_reg16(vm, a->vm);
 }
 
 if (a->e) {
@@ -2700,7 +2705,7 @@ static bool trans_VRINTR_hp(DisasContext *s, 
arg_VRINTR_sp *a)
 }
 
 tmp = tcg_temp_new_i32();
-vfp_load_reg32(tmp, a->vm);
+vfp_load_reg16(tmp, a->vm);
 fpst = fpstatus_ptr(FPST_FPCR_F16);
 gen_helper_rinth(tmp, tmp, fpst);
 vfp_store_reg32(tmp, a->vd);
@@ -2773,7 +2778,7 @@ static bool trans_VRINTZ_hp(DisasContext *s, 
arg_VRINTZ_sp *a)
 }
 
 tmp = tcg_temp_new_i32();
-vfp_load_reg32(tmp, a->vm);
+vfp_load_reg16(tmp, a->vm);
 fpst = fpstatus_ptr(FPST_FPCR_F16);
 tcg_rmode = gen_set_rmode(FPROUNDING_ZERO, fpst);
 gen_helper_rinth(tmp, tmp, fpst);
@@ -2853,7 +2858,7 @@ static bool trans_VRINTX_hp(DisasContext *s, 
arg_VRINTX_sp *a)
 }
 
 tmp = tcg_temp_new_i32();
-vfp_load_reg32(tmp, a->vm);
+vfp_load_reg16(tmp, a->vm);
 fpst = fpstatus_ptr(FPST_FPCR_F16);
 gen_helper_rinth_exact(tmp, tmp, fpst);
 vfp_store_reg32(tmp, a->vd);
@@ -3270,7 +3275,7 @@ static bool trans_VCVT_hp_int(DisasContext *s, 
arg_VCVT_sp_int *a)
 
 fpst = fpstatus_ptr(FPST_FPCR_F16);
 vm = tcg_temp_new_i32();
-vfp_load_reg32(vm, a->vm);
+vfp_load_reg16(vm, a->vm);
 
 if (a->s) {
 if (a->rz) {
@@ -3383,8 +3388,8 @@ static bool trans_VINS(DisasContext *s, arg_VINS *a)
 /* Insert low half of Vm into high half of Vd */
 rm = tcg_temp_new_i32();
 rd = tcg_temp_new_i32();
-vfp_load_reg32(rm, a->vm);
-vfp_load_reg32(rd, a->vd);
+vfp_load_reg16(rm, a->vm);
+vfp_load_reg16(rd, a->vd);
 tcg_gen_deposit_i32(rd, rd, rm, 16, 16);
 vfp_store_reg32(rd, a->vd);
 return true;
-- 
2.34.1

[PATCH v2 51/67] target/arm: Convert CMGT, CMHI, CMGE, CMHS, CMTST, CMEQ to decodetree

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  12 +++
 target/arm/tcg/translate-a64.c | 132 -
 2 files changed, 60 insertions(+), 84 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 44383b4fc7..3061e26242 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -767,6 +767,12 @@ UQRSHL_s0111 1110 ..1 . 01011 1 . . 
@rrr_e
 
 ADD_s   0101 1110 111 . 1 1 . . @rrr_d
 SUB_s   0111 1110 111 . 1 1 . . @rrr_d
+CMGT_s  0101 1110 111 . 00110 1 . . @rrr_d
+CMHI_s  0111 1110 111 . 00110 1 . . @rrr_d
+CMGE_s  0101 1110 111 . 00111 1 . . @rrr_d
+CMHS_s  0111 1110 111 . 00111 1 . . @rrr_d
+CMTST_s 0101 1110 111 . 10001 1 . . @rrr_d
+CMEQ_s  0111 1110 111 . 10001 1 . . @rrr_d
 
 ### Advanced SIMD scalar pairwise
 
@@ -900,6 +906,12 @@ UQRSHL_v0.10 1110 ..1 . 01011 1 . . 
@qrrr_e
 
 ADD_v   0.00 1110 ..1 . 1 1 . . @qrrr_e
 SUB_v   0.10 1110 ..1 . 1 1 . . @qrrr_e
+CMGT_v  0.00 1110 ..1 . 00110 1 . . @qrrr_e
+CMHI_v  0.10 1110 ..1 . 00110 1 . . @qrrr_e
+CMGE_v  0.00 1110 ..1 . 00111 1 . . @qrrr_e
+CMHS_v  0.10 1110 ..1 . 00111 1 . . @qrrr_e
+CMTST_v 0.00 1110 ..1 . 10001 1 . . @qrrr_e
+CMEQ_v  0.10 1110 ..1 . 10001 1 . . @qrrr_e
 
 ### Advanced SIMD scalar x indexed element
 
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 77a64923e7..3c6cfc2952 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5180,6 +5180,24 @@ static const ENVScalar2 f_scalar_uqrshl = {
 };
 TRANS(UQRSHL_s, do_env_scalar2, a, &f_scalar_uqrshl)
 
+static bool do_cmop_d(DisasContext *s, arg_rrr_e *a, TCGCond cond)
+{
+if (fp_access_check(s)) {
+TCGv_i64 t0 = read_fp_dreg(s, a->rn);
+TCGv_i64 t1 = read_fp_dreg(s, a->rm);
+tcg_gen_negsetcond_i64(cond, t0, t0, t1);
+write_fp_dreg(s, a->rd, t0);
+}
+return true;
+}
+
+TRANS(CMGT_s, do_cmop_d, a, TCG_COND_GT)
+TRANS(CMHI_s, do_cmop_d, a, TCG_COND_GTU)
+TRANS(CMGE_s, do_cmop_d, a, TCG_COND_GE)
+TRANS(CMHS_s, do_cmop_d, a, TCG_COND_GEU)
+TRANS(CMEQ_s, do_cmop_d, a, TCG_COND_EQ)
+TRANS(CMTST_s, do_cmop_d, a, TCG_COND_TSTNE)
+
 static bool do_fp3_vector(DisasContext *s, arg_qrrr_e *a,
   gen_helper_gvec_3_ptr * const fns[3])
 {
@@ -5437,6 +5455,28 @@ TRANS(UQRSHL_v, do_gvec_fn3, a, gen_neon_uqrshl)
 TRANS(ADD_v, do_gvec_fn3, a, tcg_gen_gvec_add)
 TRANS(SUB_v, do_gvec_fn3, a, tcg_gen_gvec_sub)
 
+static bool do_cmop_v(DisasContext *s, arg_qrrr_e *a, TCGCond cond)
+{
+if (a->esz == MO_64 && !a->q) {
+return false;
+}
+if (fp_access_check(s)) {
+tcg_gen_gvec_cmp(cond, a->esz,
+ vec_full_reg_offset(s, a->rd),
+ vec_full_reg_offset(s, a->rn),
+ vec_full_reg_offset(s, a->rm),
+ a->q ? 16 : 8, vec_full_reg_size(s));
+}
+return true;
+}
+
+TRANS(CMGT_v, do_cmop_v, a, TCG_COND_GT)
+TRANS(CMHI_v, do_cmop_v, a, TCG_COND_GTU)
+TRANS(CMGE_v, do_cmop_v, a, TCG_COND_GE)
+TRANS(CMHS_v, do_cmop_v, a, TCG_COND_GEU)
+TRANS(CMEQ_v, do_cmop_v, a, TCG_COND_EQ)
+TRANS(CMTST_v, do_gvec_fn3, a, gen_gvec_cmtst)
+
 /*
  * Advanced SIMD scalar/vector x indexed element
  */
@@ -9421,45 +9461,6 @@ static void 
disas_simd_scalar_three_reg_diff(DisasContext *s, uint32_t insn)
 }
 }
 
-static void handle_3same_64(DisasContext *s, int opcode, bool u,
-TCGv_i64 tcg_rd, TCGv_i64 tcg_rn, TCGv_i64 tcg_rm)
-{
-/* Handle 64x64->64 opcodes which are shared between the scalar
- * and vector 3-same groups. We cover every opcode where size == 3
- * is valid in either the three-reg-same (integer, not pairwise)
- * or scalar-three-reg-same groups.
- */
-TCGCond cond;
-
-switch (opcode) {
-case 0x6: /* CMGT, CMHI */
-cond = u ? TCG_COND_GTU : TCG_COND_GT;
-do_cmop:
-/* 64 bit integer comparison, result = test ? -1 : 0. */
-tcg_gen_negsetcond_i64(cond, tcg_rd, tcg_rn, tcg_rm);
-break;
-case 0x7: /* CMGE, CMHS */
-cond = u ? TCG_COND_GEU : TCG_COND_GE;
-goto do_cmop;
-case 0x11: /* CMTST, CMEQ */
-if (u) {
-cond = TCG_COND_EQ;
-goto do_cmop;
-}
-gen_cmtst_i64(tcg_rd, tcg_rn, tcg_rm);
-break;
-default:
-case 0x1: /* SQADD / UQADD */
-case 0x5: /* SQSUB / UQSUB */
-case 0x8: /* SSHL, USHL */
-case 0x9: /* SQSHL, UQSHL */
-case 0xa: /* SRSHL, URSHL */
-case 0xb: /* SQRSHL, UQRSHL */
-case 0x10: /* ADD, SUB */
-   

[PATCH v2 41/67] target/arm: Convert SQADD, SQSUB, UQADD, UQSUB to decodetree

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  11 
 target/arm/tcg/translate-a64.c | 100 +++--
 2 files changed, 68 insertions(+), 43 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index f48adef5bb..19010af03b 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -44,6 +44,7 @@
 @rrr_h   ... rm:5 .. rn:5 rd:5  &rrr_e esz=1
 @rrr_sd  ... rm:5 .. rn:5 rd:5  &rrr_e esz=%esz_sd
 @rrr_hsd ... rm:5 .. rn:5 rd:5  &rrr_e esz=%esz_hsd
+@rrr_e   esz:2 . rm:5 .. rn:5 rd:5  &rrr_e
 
 @rrx_h   .. .. rm:4  . . rn:5 rd:5  &rrx_e esz=1 idx=%hlm
 @rrx_s   .. . rm:5   . . rn:5 rd:5  &rrx_e esz=2 idx=%hl
@@ -744,6 +745,11 @@ FRECPS_s0101 1110 0.1 . 1 1 . . 
@rrr_sd
 FRSQRTS_s   0101 1110 110 . 00111 1 . . @rrr_h
 FRSQRTS_s   0101 1110 1.1 . 1 1 . . @rrr_sd
 
+SQADD_s 0101 1110 ..1 . 1 1 . . @rrr_e
+UQADD_s 0111 1110 ..1 . 1 1 . . @rrr_e
+SQSUB_s 0101 1110 ..1 . 00101 1 . . @rrr_e
+UQSUB_s 0111 1110 ..1 . 00101 1 . . @rrr_e
+
 ### Advanced SIMD scalar pairwise
 
 FADDP_s 0101 1110 0011  1101 10 . . @rr_h
@@ -857,6 +863,11 @@ BSL_v   0.10 1110 011 . 00011 1 . . 
@qrrr_b
 BIT_v   0.10 1110 101 . 00011 1 . . @qrrr_b
 BIF_v   0.10 1110 111 . 00011 1 . . @qrrr_b
 
+SQADD_v 0.00 1110 ..1 . 1 1 . . @qrrr_e
+UQADD_v 0.10 1110 ..1 . 1 1 . . @qrrr_e
+SQSUB_v 0.00 1110 ..1 . 00101 1 . . @qrrr_e
+UQSUB_v 0.10 1110 ..1 . 00101 1 . . @qrrr_e
+
 ### Advanced SIMD scalar x indexed element
 
 FMUL_si 0101  00 ..  1001 . 0 . .   @rrx_h
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index ca7ba6b1e8..2f7298811d 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5060,6 +5060,43 @@ static const FPScalar f_scalar_frsqrts = {
 };
 TRANS(FRSQRTS_s, do_fp3_scalar, a, &f_scalar_frsqrts)
 
+static bool do_satacc_s(DisasContext *s, arg_rrr_e *a,
+MemOp sgn_n, MemOp sgn_m,
+void (*gen_bhs)(TCGv_i64, TCGv_i64, TCGv_i64, TCGv_i64, MemOp),
+void (*gen_d)(TCGv_i64, TCGv_i64, TCGv_i64, TCGv_i64))
+{
+TCGv_i64 t0, t1, t2, qc;
+MemOp esz = a->esz;
+
+if (!fp_access_check(s)) {
+return true;
+}
+
+t0 = tcg_temp_new_i64();
+t1 = tcg_temp_new_i64();
+t2 = tcg_temp_new_i64();
+qc = tcg_temp_new_i64();
+read_vec_element(s, t1, a->rn, 0, esz | sgn_n);
+read_vec_element(s, t2, a->rm, 0, esz | sgn_m);
+tcg_gen_ld_i64(qc, tcg_env, offsetof(CPUARMState, vfp.qc));
+
+if (esz == MO_64) {
+gen_d(t0, qc, t1, t2);
+} else {
+gen_bhs(t0, qc, t1, t2, esz);
+tcg_gen_ext_i64(t0, t0, esz);
+}
+
+write_fp_dreg(s, a->rd, t0);
+tcg_gen_st_i64(qc, tcg_env, offsetof(CPUARMState, vfp.qc));
+return true;
+}
+
+TRANS(SQADD_s, do_satacc_s, a, MO_SIGN, MO_SIGN, gen_sqadd_bhs, gen_sqadd_d)
+TRANS(SQSUB_s, do_satacc_s, a, MO_SIGN, MO_SIGN, gen_sqsub_bhs, gen_sqsub_d)
+TRANS(UQADD_s, do_satacc_s, a, 0, 0, gen_uqadd_bhs, gen_uqadd_d)
+TRANS(UQSUB_s, do_satacc_s, a, 0, 0, gen_uqsub_bhs, gen_uqsub_d)
+
 static bool do_fp3_vector(DisasContext *s, arg_qrrr_e *a,
   gen_helper_gvec_3_ptr * const fns[3])
 {
@@ -5298,6 +5335,11 @@ TRANS(BSL_v, do_bitsel, a->q, a->rd, a->rd, a->rn, a->rm)
 TRANS(BIT_v, do_bitsel, a->q, a->rd, a->rm, a->rn, a->rd)
 TRANS(BIF_v, do_bitsel, a->q, a->rd, a->rm, a->rd, a->rn)
 
+TRANS(SQADD_v, do_gvec_fn3, a, gen_gvec_sqadd_qc)
+TRANS(UQADD_v, do_gvec_fn3, a, gen_gvec_uqadd_qc)
+TRANS(SQSUB_v, do_gvec_fn3, a, gen_gvec_sqsub_qc)
+TRANS(UQSUB_v, do_gvec_fn3, a, gen_gvec_uqsub_qc)
+
 /*
  * Advanced SIMD scalar/vector x indexed element
  */
@@ -9291,29 +9333,8 @@ static void handle_3same_64(DisasContext *s, int opcode, 
bool u,
  * or scalar-three-reg-same groups.
  */
 TCGCond cond;
-TCGv_i64 qc;
 
 switch (opcode) {
-case 0x1: /* SQADD */
-qc = tcg_temp_new_i64();
-tcg_gen_ld_i64(qc, tcg_env, offsetof(CPUARMState, vfp.qc));
-if (u) {
-gen_uqadd_d(tcg_rd, qc, tcg_rn, tcg_rm);
-} else {
-gen_sqadd_d(tcg_rd, qc, tcg_rn, tcg_rm);
-}
-tcg_gen_st_i64(qc, tcg_env, offsetof(CPUARMState, vfp.qc));
-break;
-case 0x5: /* SQSUB */
-qc = tcg_temp_new_i64();
-tcg_gen_ld_i64(qc, tcg_env, offsetof(CPUARMState, vfp.qc));
-if (u) {
-gen_uqsub_d(tcg_rd, qc, tcg_rn, tcg_rm);
-} else {
-gen_sqsub_d(tcg_rd, qc, tcg_rn, tcg_rm);
-  

[PATCH v2 31/67] target/arm: Convert ADDP to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/helper.h|   5 ++
 target/arm/tcg/translate.h |   3 +
 target/arm/tcg/a64.decode  |   6 ++
 target/arm/tcg/gengvec.c   |  12 
 target/arm/tcg/translate-a64.c | 128 ++---
 target/arm/tcg/vec_helper.c|  30 
 6 files changed, 77 insertions(+), 107 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index 065460ea80..d3579a101f 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -1061,6 +1061,11 @@ DEF_HELPER_FLAGS_5(gvec_fminnump_h, TCG_CALL_NO_RWG, 
void, ptr, ptr, ptr, ptr, i
 DEF_HELPER_FLAGS_5(gvec_fminnump_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 DEF_HELPER_FLAGS_5(gvec_fminnump_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 
+DEF_HELPER_FLAGS_4(gvec_addp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_addp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_addp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_addp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+
 #ifdef TARGET_AARCH64
 #include "tcg/helper-a64.h"
 #include "tcg/helper-sve.h"
diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h
index b05a9eb668..04771f483b 100644
--- a/target/arm/tcg/translate.h
+++ b/target/arm/tcg/translate.h
@@ -514,6 +514,9 @@ void gen_gvec_saba(unsigned vece, uint32_t rd_ofs, uint32_t 
rn_ofs,
 void gen_gvec_uaba(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
 
+void gen_gvec_addp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+   uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+
 /*
  * Forward to the isar_feature_* tests given a DisasContext pointer.
  */
diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 43557fdccc..84f5bcc0e0 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -38,6 +38,7 @@
 &q_eq rd rn rm ra esz
 
 @rr_h    ... . .. rn:5 rd:5 &rr_e esz=1
+@rr_d    ... . .. rn:5 rd:5 &rr_e esz=3
 @rr_sd   ... . .. rn:5 rd:5 &rr_e esz=%esz_sd
 
 @rrr_h   ... rm:5 .. rn:5 rd:5  &rrr_e esz=1
@@ -56,6 +57,7 @@
 
 @qrrr_h . q:1 .. ... rm:5 .. rn:5 rd:5  &qrrr_e esz=1
 @qrrr_sd. q:1 .. ... rm:5 .. rn:5 rd:5  &qrrr_e esz=%esz_sd
+@qrrr_e . q:1 .. esz:2 . rm:5 .. rn:5 rd:5  &qrrr_e
 
 @qrrx_h . q:1 ..  .. .. rm:4  . . rn:5 rd:5 \
 &qrrx_e esz=1 idx=%hlm
@@ -758,6 +760,8 @@ FMAXNMP_s   0111 1110 0.11  1100 10 . . 
@rr_sd
 FMINNMP_s   0101 1110 1011  1100 10 . . @rr_h
 FMINNMP_s   0111 1110 1.11  1100 10 . . @rr_sd
 
+ADDP_s  0101 1110  0001 1011 10 . . @rr_d
+
 ### Advanced SIMD three same
 
 FADD_v  0.00 1110 010 . 00010 1 . . @qrrr_h
@@ -832,6 +836,8 @@ FMAXNMP_v   0.10 1110 0.1 . 11000 1 . . 
@qrrr_sd
 FMINNMP_v   0.10 1110 110 . 0 1 . . @qrrr_h
 FMINNMP_v   0.10 1110 1.1 . 11000 1 . . @qrrr_sd
 
+ADDP_v  0.00 1110 ..1 . 10111 1 . . @qrrr_e
+
 ### Advanced SIMD scalar x indexed element
 
 FMUL_si 0101  00 ..  1001 . 0 . .   @rrx_h
diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c
index 7a1856253f..f010dd5a0e 100644
--- a/target/arm/tcg/gengvec.c
+++ b/target/arm/tcg/gengvec.c
@@ -1610,3 +1610,15 @@ void gen_gvec_uaba(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 };
 tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
 }
+
+void gen_gvec_addp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+   uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+static gen_helper_gvec_3 * const fns[4] = {
+gen_helper_gvec_addp_b,
+gen_helper_gvec_addp_h,
+gen_helper_gvec_addp_s,
+gen_helper_gvec_addp_d,
+};
+tcg_gen_gvec_3_ool(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, 0, fns[vece]);
+}
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 07415bd285..b8add91112 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5245,6 +5245,8 @@ static gen_helper_gvec_3_ptr * const f_vector_fminnmp[3] 
= {
 };
 TRANS(FMINNMP_v, do_fp3_vector, a, f_vector_fminnmp)
 
+TRANS(ADDP_v, do_gvec_fn3, a, gen_gvec_addp)
+
 /*
  * Advanced SIMD scalar/vector x indexed element
  */
@@ -5485,6 +5487,20 @@ TRANS(FMINP_s, do_fp3_scalar_pair, a, &f_scalar_fmin)
 TRANS(FMAXNMP_s, do_fp3_scalar_pair, a, &f_scalar_fmaxnm)
 TRANS(FMINNMP_s, do_fp3_scalar_pair, a, &f_scalar_fminnm)
 
+static bool trans_ADDP_s(DisasContext *s, arg_rr_e *a)
+{
+if (fp_access_check(s)) {
+TCGv_i64 t0 = tcg_temp_new_i64();
+TCGv_

[PATCH v2 64/67] target/arm: Tidy SQDMULH, SQRDMULH (vector)

[Richard Henderson]

We already have a gvec helper for the operations, but we aren't
using it on the aa32 neon side.  Create a unified expander for
use by both aa32 and aa64 translators.

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/translate.h  |  4 
 target/arm/tcg/gengvec.c| 20 
 target/arm/tcg/translate-a64.c  | 23 ---
 target/arm/tcg/translate-neon.c | 23 +++
 4 files changed, 31 insertions(+), 39 deletions(-)

diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h
index 3b1e68b779..aba21f730f 100644
--- a/target/arm/tcg/translate.h
+++ b/target/arm/tcg/translate.h
@@ -539,6 +539,10 @@ void gen_gvec_sri(unsigned vece, uint32_t rd_ofs, uint32_t 
rm_ofs,
 void gen_gvec_sli(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
   int64_t shift, uint32_t opr_sz, uint32_t max_sz);
 
+void gen_gvec_sqdmulh_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_sqrdmulh_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+  uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
 void gen_gvec_sqrdmlah_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
   uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
 void gen_gvec_sqrdmlsh_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c
index 32caabd126..462c185f9a 100644
--- a/target/arm/tcg/gengvec.c
+++ b/target/arm/tcg/gengvec.c
@@ -34,6 +34,26 @@ static void gen_gvec_fn3_qc(uint32_t rd_ofs, uint32_t 
rn_ofs, uint32_t rm_ofs,
opr_sz, max_sz, 0, fn);
 }
 
+void gen_gvec_sqdmulh_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+static gen_helper_gvec_3_ptr * const fns[2] = {
+gen_helper_neon_sqdmulh_h, gen_helper_neon_sqdmulh_s
+};
+tcg_debug_assert(vece >= 1 && vece <= 2);
+gen_gvec_fn3_qc(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, fns[vece - 1]);
+}
+
+void gen_gvec_sqrdmulh_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+static gen_helper_gvec_3_ptr * const fns[2] = {
+gen_helper_neon_sqrdmulh_h, gen_helper_neon_sqrdmulh_s
+};
+tcg_debug_assert(vece >= 1 && vece <= 2);
+gen_gvec_fn3_qc(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, fns[vece - 1]);
+}
+
 void gen_gvec_sqrdmlah_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
   uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
 {
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index c4601cde2f..c673b95ec7 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -724,19 +724,6 @@ static void gen_gvec_op3_fpst(DisasContext *s, bool is_q, 
int rd, int rn,
is_q ? 16 : 8, vec_full_reg_size(s), data, fn);
 }
 
-/* Expand a 3-operand + qc + operation using an out-of-line helper.  */
-static void gen_gvec_op3_qc(DisasContext *s, bool is_q, int rd, int rn,
-int rm, gen_helper_gvec_3_ptr *fn)
-{
-TCGv_ptr qc_ptr = tcg_temp_new_ptr();
-
-tcg_gen_addi_ptr(qc_ptr, tcg_env, offsetof(CPUARMState, vfp.qc));
-tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, rd),
-   vec_full_reg_offset(s, rn),
-   vec_full_reg_offset(s, rm), qc_ptr,
-   is_q ? 16 : 8, vec_full_reg_size(s), 0, fn);
-}
-
 /* Expand a 4-operand operation using an out-of-line helper.  */
 static void gen_gvec_op4_ool(DisasContext *s, bool is_q, int rd, int rn,
  int rm, int ra, int data, gen_helper_gvec_4 *fn)
@@ -11007,12 +10994,10 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 
 switch (opcode) {
 case 0x16: /* SQDMULH, SQRDMULH */
-{
-static gen_helper_gvec_3_ptr * const fns[2][2] = {
-{ gen_helper_neon_sqdmulh_h, gen_helper_neon_sqrdmulh_h },
-{ gen_helper_neon_sqdmulh_s, gen_helper_neon_sqrdmulh_s },
-};
-gen_gvec_op3_qc(s, is_q, rd, rn, rm, fns[size - 1][u]);
+if (u) {
+gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_sqrdmulh_qc, size);
+} else {
+gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_sqdmulh_qc, size);
 }
 return;
 }
diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c
index f9a8753906..915c9e56db 100644
--- a/target/arm/tcg/translate-neon.c
+++ b/target/arm/tcg/translate-neon.c
@@ -937,28 +937,11 @@ DO_SHA2(SHA256SU1, gen_helper_crypto_sha256su1)
 }
 
 #define DO_3SAME_VQDMULH(INSN, FUNC)\
-WRAP_ENV_FN(gen_##INSN##_tramp16, gen_helper_neon_##FUNC##_s16);\
-WRAP_ENV_FN(gen_##INSN##_t

[PATCH v2 48/67] target/arm: Convert SQRSHL and UQRSHL (register) to gvec

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/helper.h |  8 ++
 target/arm/tcg/translate.h  |  4 +++
 target/arm/tcg/neon-dp.decode   | 17 ++--
 target/arm/tcg/gengvec.c| 24 
 target/arm/tcg/neon_helper.c| 24 
 target/arm/tcg/translate-a64.c  | 17 +---
 target/arm/tcg/translate-neon.c | 49 ++---
 7 files changed, 71 insertions(+), 72 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index f345087ddb..9a89c9cea7 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -334,6 +334,14 @@ DEF_HELPER_FLAGS_5(neon_uqshl_b, TCG_CALL_NO_RWG, void, 
ptr, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_5(neon_uqshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 DEF_HELPER_FLAGS_5(neon_uqshl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 DEF_HELPER_FLAGS_5(neon_uqshl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(neon_sqrshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(neon_sqrshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(neon_sqrshl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(neon_sqrshl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(neon_uqrshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(neon_uqrshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(neon_uqrshl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(neon_uqrshl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 
 DEF_HELPER_FLAGS_4(gvec_srshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_4(gvec_srshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h
index 6c6d4d49e7..048cb45ebe 100644
--- a/target/arm/tcg/translate.h
+++ b/target/arm/tcg/translate.h
@@ -467,6 +467,10 @@ void gen_neon_sqshl(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
 void gen_neon_uqshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
 uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_neon_sqrshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_neon_uqrshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
 
 void gen_cmtst_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b);
 void gen_ushl_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b);
diff --git a/target/arm/tcg/neon-dp.decode b/target/arm/tcg/neon-dp.decode
index 6d4996b8d8..788578c8fa 100644
--- a/target/arm/tcg/neon-dp.decode
+++ b/target/arm/tcg/neon-dp.decode
@@ -102,25 +102,12 @@ VCGE_U_3s 001 1 0 . ..   0011 . . . 1 
 @3same
 
 VSHL_S_3s 001 0 0 . ..   0100 . . . 0  @3same_rev
 VSHL_U_3s 001 1 0 . ..   0100 . . . 0  @3same_rev
-
-# Insns operating on 64-bit elements (size!=0b11 handled elsewhere)
-# The _rev suffix indicates that Vn and Vm are reversed (as explained
-# by the comment for the @3same_rev format).
-@3same_64_rev ... . . . 11    . q:1 . .  \
- &3same vm=%vn_dp vn=%vm_dp vd=%vd_dp size=3
-
 VQSHL_S_3s    001 0 0 . ..   0100 . . . 1  @3same_rev
 VQSHL_U_3s    001 1 0 . ..   0100 . . . 1  @3same_rev
 VRSHL_S_3s    001 0 0 . ..   0101 . . . 0  @3same_rev
 VRSHL_U_3s    001 1 0 . ..   0101 . . . 0  @3same_rev
-{
-  VQRSHL_S64_3s   001 0 0 . ..   0101 . . . 1  @3same_64_rev
-  VQRSHL_S_3s 001 0 0 . ..   0101 . . . 1  @3same_rev
-}
-{
-  VQRSHL_U64_3s   001 1 0 . ..   0101 . . . 1  @3same_64_rev
-  VQRSHL_U_3s 001 1 0 . ..   0101 . . . 1  @3same_rev
-}
+VQRSHL_S_3s   001 0 0 . ..   0101 . . . 1  @3same_rev
+VQRSHL_U_3s   001 1 0 . ..   0101 . . . 1  @3same_rev
 
 VMAX_S_3s 001 0 0 . ..   0110 . . . 0  @3same
 VMAX_U_3s 001 1 0 . ..   0110 . . . 0  @3same
diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c
index 773dbf41d3..51e66ccf5f 100644
--- a/target/arm/tcg/gengvec.c
+++ b/target/arm/tcg/gengvec.c
@@ -1263,6 +1263,30 @@ void gen_neon_uqshl(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
opr_sz, max_sz, 0, fns[vece]);
 }
 
+void gen_neon_sqrshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+static gen_helper_gvec_3_ptr * const fns[] = {
+gen_helper_neon_sqrshl_b, gen_helper_neon_sqrshl_h,
+gen_helper_neon_sqrshl_s, gen_helper_neon_sqrshl_d,
+};
+tcg_debug_assert(vece

[PATCH v2 37/67] target/arm: Improve vector UQADD, UQSUB, SQADD, SQSUB

[Richard Henderson]

No need for a full comparison; xor produces non-zero bits
for QC just fine.

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/gengvec.c | 32 
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c
index 22c9d17dce..bfe6885a01 100644
--- a/target/arm/tcg/gengvec.c
+++ b/target/arm/tcg/gengvec.c
@@ -1217,21 +1217,21 @@ void gen_gvec_sshl(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
 }
 
-static void gen_uqadd_vec(unsigned vece, TCGv_vec t, TCGv_vec sat,
+static void gen_uqadd_vec(unsigned vece, TCGv_vec t, TCGv_vec qc,
   TCGv_vec a, TCGv_vec b)
 {
 TCGv_vec x = tcg_temp_new_vec_matching(t);
 tcg_gen_add_vec(vece, x, a, b);
 tcg_gen_usadd_vec(vece, t, a, b);
-tcg_gen_cmp_vec(TCG_COND_NE, vece, x, x, t);
-tcg_gen_or_vec(vece, sat, sat, x);
+tcg_gen_xor_vec(vece, x, x, t);
+tcg_gen_or_vec(vece, qc, qc, x);
 }
 
 void gen_gvec_uqadd_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
 {
 static const TCGOpcode vecop_list[] = {
-INDEX_op_usadd_vec, INDEX_op_cmp_vec, INDEX_op_add_vec, 0
+INDEX_op_usadd_vec, INDEX_op_add_vec, 0
 };
 static const GVecGen4 ops[4] = {
 { .fniv = gen_uqadd_vec,
@@ -1259,21 +1259,21 @@ void gen_gvec_uqadd_qc(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
 }
 
-static void gen_sqadd_vec(unsigned vece, TCGv_vec t, TCGv_vec sat,
+static void gen_sqadd_vec(unsigned vece, TCGv_vec t, TCGv_vec qc,
   TCGv_vec a, TCGv_vec b)
 {
 TCGv_vec x = tcg_temp_new_vec_matching(t);
 tcg_gen_add_vec(vece, x, a, b);
 tcg_gen_ssadd_vec(vece, t, a, b);
-tcg_gen_cmp_vec(TCG_COND_NE, vece, x, x, t);
-tcg_gen_or_vec(vece, sat, sat, x);
+tcg_gen_xor_vec(vece, x, x, t);
+tcg_gen_or_vec(vece, qc, qc, x);
 }
 
 void gen_gvec_sqadd_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
 {
 static const TCGOpcode vecop_list[] = {
-INDEX_op_ssadd_vec, INDEX_op_cmp_vec, INDEX_op_add_vec, 0
+INDEX_op_ssadd_vec, INDEX_op_add_vec, 0
 };
 static const GVecGen4 ops[4] = {
 { .fniv = gen_sqadd_vec,
@@ -1301,21 +1301,21 @@ void gen_gvec_sqadd_qc(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
 }
 
-static void gen_uqsub_vec(unsigned vece, TCGv_vec t, TCGv_vec sat,
+static void gen_uqsub_vec(unsigned vece, TCGv_vec t, TCGv_vec qc,
   TCGv_vec a, TCGv_vec b)
 {
 TCGv_vec x = tcg_temp_new_vec_matching(t);
 tcg_gen_sub_vec(vece, x, a, b);
 tcg_gen_ussub_vec(vece, t, a, b);
-tcg_gen_cmp_vec(TCG_COND_NE, vece, x, x, t);
-tcg_gen_or_vec(vece, sat, sat, x);
+tcg_gen_xor_vec(vece, x, x, t);
+tcg_gen_or_vec(vece, qc, qc, x);
 }
 
 void gen_gvec_uqsub_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
 {
 static const TCGOpcode vecop_list[] = {
-INDEX_op_ussub_vec, INDEX_op_cmp_vec, INDEX_op_sub_vec, 0
+INDEX_op_ussub_vec, INDEX_op_sub_vec, 0
 };
 static const GVecGen4 ops[4] = {
 { .fniv = gen_uqsub_vec,
@@ -1343,21 +1343,21 @@ void gen_gvec_uqsub_qc(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
 }
 
-static void gen_sqsub_vec(unsigned vece, TCGv_vec t, TCGv_vec sat,
+static void gen_sqsub_vec(unsigned vece, TCGv_vec t, TCGv_vec qc,
   TCGv_vec a, TCGv_vec b)
 {
 TCGv_vec x = tcg_temp_new_vec_matching(t);
 tcg_gen_sub_vec(vece, x, a, b);
 tcg_gen_sssub_vec(vece, t, a, b);
-tcg_gen_cmp_vec(TCG_COND_NE, vece, x, x, t);
-tcg_gen_or_vec(vece, sat, sat, x);
+tcg_gen_xor_vec(vece, x, x, t);
+tcg_gen_or_vec(vece, qc, qc, x);
 }
 
 void gen_gvec_sqsub_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
 {
 static const TCGOpcode vecop_list[] = {
-INDEX_op_sssub_vec, INDEX_op_cmp_vec, INDEX_op_sub_vec, 0
+INDEX_op_sssub_vec, INDEX_op_sub_vec, 0
 };
 static const GVecGen4 ops[4] = {
 { .fniv = gen_sqsub_vec,
-- 
2.34.1

[PATCH v2 57/67] target/arm: Convert SHSUB, UHSUB to decodetree

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  2 ++
 target/arm/tcg/translate-a64.c | 11 +++
 2 files changed, 5 insertions(+), 8 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index e33d91fd0a..b1bbcb144e 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -914,6 +914,8 @@ CMTST_v 0.00 1110 ..1 . 10001 1 . . 
@qrrr_e
 CMEQ_v  0.10 1110 ..1 . 10001 1 . . @qrrr_e
 SHADD_v 0.00 1110 ..1 . 0 1 . . @qrrr_e
 UHADD_v 0.10 1110 ..1 . 0 1 . . @qrrr_e
+SHSUB_v 0.00 1110 ..1 . 00100 1 . . @qrrr_e
+UHSUB_v 0.10 1110 ..1 . 00100 1 . . @qrrr_e
 
 ### Advanced SIMD scalar x indexed element
 
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 63f7a59f94..6571b999f4 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5456,6 +5456,8 @@ TRANS(ADD_v, do_gvec_fn3, a, tcg_gen_gvec_add)
 TRANS(SUB_v, do_gvec_fn3, a, tcg_gen_gvec_sub)
 TRANS(SHADD_v, do_gvec_fn3_no64, a, gen_gvec_shadd)
 TRANS(UHADD_v, do_gvec_fn3_no64, a, gen_gvec_uhadd)
+TRANS(SHSUB_v, do_gvec_fn3_no64, a, gen_gvec_shsub)
+TRANS(UHSUB_v, do_gvec_fn3_no64, a, gen_gvec_uhsub)
 
 static bool do_cmop_v(DisasContext *s, arg_qrrr_e *a, TCGCond cond)
 {
@@ -10923,7 +10925,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 }
 /* fall through */
 case 0x2: /* SRHADD, URHADD */
-case 0x4: /* SHSUB, UHSUB */
 case 0xc: /* SMAX, UMAX */
 case 0xd: /* SMIN, UMIN */
 case 0xe: /* SABD, UABD */
@@ -10949,6 +10950,7 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 
 case 0x0: /* SHADD, UHADD */
 case 0x01: /* SQADD, UQADD */
+case 0x04: /* SHSUB, UHSUB */
 case 0x05: /* SQSUB, UQSUB */
 case 0x06: /* CMGT, CMHI */
 case 0x07: /* CMGE, CMHS */
@@ -10967,13 +10969,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 }
 
 switch (opcode) {
-case 0x04: /* SHSUB, UHSUB */
-if (u) {
-gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_uhsub, size);
-} else {
-gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_shsub, size);
-}
-return;
 case 0x0c: /* SMAX, UMAX */
 if (u) {
 gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_umax, size);
-- 
2.34.1

[PATCH v2 45/67] target/arm: Convert SRSHL, URSHL to decodetree

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  4 
 target/arm/tcg/translate-a64.c | 22 +++---
 2 files changed, 11 insertions(+), 15 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index ea897d6732..9e02776036 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -758,6 +758,8 @@ USQADD_s0111 1110 ..1 0 00111 0 . . 
@r2r_e
 
 SSHL_s  0101 1110 111 . 01000 1 . . @rrr_d
 USHL_s  0111 1110 111 . 01000 1 . . @rrr_d
+SRSHL_s 0101 1110 111 . 01010 1 . . @rrr_d
+URSHL_s 0111 1110 111 . 01010 1 . . @rrr_d
 
 ### Advanced SIMD scalar pairwise
 
@@ -882,6 +884,8 @@ USQADD_v0.10 1110 ..1 0 00111 0 . . 
@qr2r_e
 
 SSHL_v  0.00 1110 ..1 . 01000 1 . . @qrrr_e
 USHL_v  0.10 1110 ..1 . 01000 1 . . @qrrr_e
+SRSHL_v 0.00 1110 ..1 . 01010 1 . . @qrrr_e
+URSHL_v 0.10 1110 ..1 . 01010 1 . . @qrrr_e
 
 ### Advanced SIMD scalar x indexed element
 
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 2dffda36a8..24f2025997 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5116,6 +5116,8 @@ static bool do_int3_scalar_d(DisasContext *s, arg_rrr_e 
*a,
 
 TRANS(SSHL_s, do_int3_scalar_d, a, gen_sshl_i64)
 TRANS(USHL_s, do_int3_scalar_d, a, gen_ushl_i64)
+TRANS(SRSHL_s, do_int3_scalar_d, a, gen_helper_neon_rshl_s64)
+TRANS(URSHL_s, do_int3_scalar_d, a, gen_helper_neon_rshl_u64)
 
 static bool do_fp3_vector(DisasContext *s, arg_qrrr_e *a,
   gen_helper_gvec_3_ptr * const fns[3])
@@ -5364,6 +5366,8 @@ TRANS(USQADD_v, do_gvec_fn3, a, gen_gvec_usqadd_qc)
 
 TRANS(SSHL_v, do_gvec_fn3, a, gen_gvec_sshl)
 TRANS(USHL_v, do_gvec_fn3, a, gen_gvec_ushl)
+TRANS(SRSHL_v, do_gvec_fn3, a, gen_gvec_srshl)
+TRANS(URSHL_v, do_gvec_fn3, a, gen_gvec_urshl)
 
 
 /*
@@ -9384,13 +9388,6 @@ static void handle_3same_64(DisasContext *s, int opcode, 
bool u,
 gen_helper_neon_qshl_s64(tcg_rd, tcg_env, tcg_rn, tcg_rm);
 }
 break;
-case 0xa: /* SRSHL, URSHL */
-if (u) {
-gen_helper_neon_rshl_u64(tcg_rd, tcg_rn, tcg_rm);
-} else {
-gen_helper_neon_rshl_s64(tcg_rd, tcg_rn, tcg_rm);
-}
-break;
 case 0xb: /* SQRSHL, UQRSHL */
 if (u) {
 gen_helper_neon_qrshl_u64(tcg_rd, tcg_env, tcg_rn, tcg_rm);
@@ -9409,6 +9406,7 @@ static void handle_3same_64(DisasContext *s, int opcode, 
bool u,
 case 0x1: /* SQADD / UQADD */
 case 0x5: /* SQSUB / UQSUB */
 case 0x8: /* SSHL, USHL */
+case 0xa: /* SRSHL, URSHL */
 g_assert_not_reached();
 }
 }
@@ -9433,7 +9431,6 @@ static void disas_simd_scalar_three_reg_same(DisasContext 
*s, uint32_t insn)
 case 0x9: /* SQSHL, UQSHL */
 case 0xb: /* SQRSHL, UQRSHL */
 break;
-case 0xa: /* SRSHL, URSHL */
 case 0x6: /* CMGT, CMHI */
 case 0x7: /* CMGE, CMHS */
 case 0x11: /* CMTST, CMEQ */
@@ -9453,6 +9450,7 @@ static void disas_simd_scalar_three_reg_same(DisasContext 
*s, uint32_t insn)
 case 0x1: /* SQADD, UQADD */
 case 0x5: /* SQSUB, UQSUB */
 case 0x8: /* SSHL, USHL */
+case 0xa: /* SRSHL, URSHL */
 unallocated_encoding(s);
 return;
 }
@@ -10937,13 +10935,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 }
 
 switch (opcode) {
-case 0x0a: /* SRSHL, URSHL */
-if (u) {
-gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_urshl, size);
-} else {
-gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_srshl, size);
-}
-return;
 case 0x0c: /* SMAX, UMAX */
 if (u) {
 gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_umax, size);
@@ -11025,6 +11016,7 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 case 0x01: /* SQADD, UQADD */
 case 0x05: /* SQSUB, UQSUB */
 case 0x08: /* SSHL, USHL */
+case 0x0a: /* SRSHL, URSHL */
 g_assert_not_reached();
 }
 
-- 
2.34.1

[PATCH v2 08/67] target/arm: Split out gengvec64.c

[Richard Henderson]

Split some routines out of translate-a64.c and translate-sve.c
that are used by both.

Reviewed-by: Peter Maydell 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/translate-a64.h |   4 +
 target/arm/tcg/gengvec64.c | 190 +
 target/arm/tcg/translate-a64.c |  26 -
 target/arm/tcg/translate-sve.c | 145 +
 target/arm/tcg/meson.build |   1 +
 5 files changed, 197 insertions(+), 169 deletions(-)
 create mode 100644 target/arm/tcg/gengvec64.c

diff --git a/target/arm/tcg/translate-a64.h b/target/arm/tcg/translate-a64.h
index 7b811b8ac5..91750f0ca9 100644
--- a/target/arm/tcg/translate-a64.h
+++ b/target/arm/tcg/translate-a64.h
@@ -193,6 +193,10 @@ void gen_gvec_rax1(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 void gen_gvec_xar(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
   uint32_t rm_ofs, int64_t shift,
   uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_eor3(unsigned vece, uint32_t d, uint32_t n, uint32_t m,
+   uint32_t a, uint32_t oprsz, uint32_t maxsz);
+void gen_gvec_bcax(unsigned vece, uint32_t d, uint32_t n, uint32_t m,
+   uint32_t a, uint32_t oprsz, uint32_t maxsz);
 
 void gen_sve_ldr(DisasContext *s, TCGv_ptr, int vofs, int len, int rn, int 
imm);
 void gen_sve_str(DisasContext *s, TCGv_ptr, int vofs, int len, int rn, int 
imm);
diff --git a/target/arm/tcg/gengvec64.c b/target/arm/tcg/gengvec64.c
new file mode 100644
index 00..093b498b13
--- /dev/null
+++ b/target/arm/tcg/gengvec64.c
@@ -0,0 +1,190 @@
+/*
+ *  AArch64 generic vector expansion
+ *
+ *  Copyright (c) 2013 Alexander Graf 
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see .
+ */
+
+#include "qemu/osdep.h"
+#include "translate.h"
+#include "translate-a64.h"
+
+
+static void gen_rax1_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m)
+{
+tcg_gen_rotli_i64(d, m, 1);
+tcg_gen_xor_i64(d, d, n);
+}
+
+static void gen_rax1_vec(unsigned vece, TCGv_vec d, TCGv_vec n, TCGv_vec m)
+{
+tcg_gen_rotli_vec(vece, d, m, 1);
+tcg_gen_xor_vec(vece, d, d, n);
+}
+
+void gen_gvec_rax1(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+   uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+static const TCGOpcode vecop_list[] = { INDEX_op_rotli_vec, 0 };
+static const GVecGen3 op = {
+.fni8 = gen_rax1_i64,
+.fniv = gen_rax1_vec,
+.opt_opc = vecop_list,
+.fno = gen_helper_crypto_rax1,
+.vece = MO_64,
+};
+tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &op);
+}
+
+static void gen_xar8_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, int64_t sh)
+{
+TCGv_i64 t = tcg_temp_new_i64();
+uint64_t mask = dup_const(MO_8, 0xff >> sh);
+
+tcg_gen_xor_i64(t, n, m);
+tcg_gen_shri_i64(d, t, sh);
+tcg_gen_shli_i64(t, t, 8 - sh);
+tcg_gen_andi_i64(d, d, mask);
+tcg_gen_andi_i64(t, t, ~mask);
+tcg_gen_or_i64(d, d, t);
+}
+
+static void gen_xar16_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, int64_t sh)
+{
+TCGv_i64 t = tcg_temp_new_i64();
+uint64_t mask = dup_const(MO_16, 0x >> sh);
+
+tcg_gen_xor_i64(t, n, m);
+tcg_gen_shri_i64(d, t, sh);
+tcg_gen_shli_i64(t, t, 16 - sh);
+tcg_gen_andi_i64(d, d, mask);
+tcg_gen_andi_i64(t, t, ~mask);
+tcg_gen_or_i64(d, d, t);
+}
+
+static void gen_xar_i32(TCGv_i32 d, TCGv_i32 n, TCGv_i32 m, int32_t sh)
+{
+tcg_gen_xor_i32(d, n, m);
+tcg_gen_rotri_i32(d, d, sh);
+}
+
+static void gen_xar_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, int64_t sh)
+{
+tcg_gen_xor_i64(d, n, m);
+tcg_gen_rotri_i64(d, d, sh);
+}
+
+static void gen_xar_vec(unsigned vece, TCGv_vec d, TCGv_vec n,
+TCGv_vec m, int64_t sh)
+{
+tcg_gen_xor_vec(vece, d, n, m);
+tcg_gen_rotri_vec(vece, d, d, sh);
+}
+
+void gen_gvec_xar(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+  uint32_t rm_ofs, int64_t shift,
+  uint32_t opr_sz, uint32_t max_sz)
+{
+static const TCGOpcode vecop[] = { INDEX_op_rotli_vec, 0 };
+static const GVecGen3i ops[4] = {
+{ .fni8 = gen_xar8_i64,
+  .fniv = gen_xar_vec,
+  .fno = gen_helper_sve2_xar_b,
+  .opt_opc = vecop,
+  .vece = MO_8 },
+{ .fni8 = gen_xar16_i64,
+  .fniv =

[PATCH v2 32/67] target/arm: Use gvec for neon padd

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/helper.h | 2 --
 target/arm/tcg/neon_helper.c| 5 -
 target/arm/tcg/translate-neon.c | 3 +--
 3 files changed, 1 insertion(+), 9 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index d3579a101f..51ed49aa50 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -354,8 +354,6 @@ DEF_HELPER_3(neon_qrshl_s64, i64, env, i64, i64)
 
 DEF_HELPER_2(neon_add_u8, i32, i32, i32)
 DEF_HELPER_2(neon_add_u16, i32, i32, i32)
-DEF_HELPER_2(neon_padd_u8, i32, i32, i32)
-DEF_HELPER_2(neon_padd_u16, i32, i32, i32)
 DEF_HELPER_2(neon_sub_u8, i32, i32, i32)
 DEF_HELPER_2(neon_sub_u16, i32, i32, i32)
 DEF_HELPER_2(neon_mul_u8, i32, i32, i32)
diff --git a/target/arm/tcg/neon_helper.c b/target/arm/tcg/neon_helper.c
index bc6c4a54e9..a0b51c8809 100644
--- a/target/arm/tcg/neon_helper.c
+++ b/target/arm/tcg/neon_helper.c
@@ -745,11 +745,6 @@ uint32_t HELPER(neon_add_u16)(uint32_t a, uint32_t b)
 return (a + b) ^ mask;
 }
 
-#define NEON_FN(dest, src1, src2) dest = src1 + src2
-NEON_POP(padd_u8, neon_u8, 4)
-NEON_POP(padd_u16, neon_u16, 2)
-#undef NEON_FN
-
 #define NEON_FN(dest, src1, src2) dest = src1 - src2
 NEON_VOP(sub_u8, neon_u8, 4)
 NEON_VOP(sub_u16, neon_u16, 2)
diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c
index 2326a05a0a..6c5a7a98e1 100644
--- a/target/arm/tcg/translate-neon.c
+++ b/target/arm/tcg/translate-neon.c
@@ -830,6 +830,7 @@ DO_3SAME_NO_SZ_3(VABD_S, gen_gvec_sabd)
 DO_3SAME_NO_SZ_3(VABA_S, gen_gvec_saba)
 DO_3SAME_NO_SZ_3(VABD_U, gen_gvec_uabd)
 DO_3SAME_NO_SZ_3(VABA_U, gen_gvec_uaba)
+DO_3SAME_NO_SZ_3(VPADD, gen_gvec_addp)
 
 #define DO_3SAME_CMP(INSN, COND)\
 static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \
@@ -1070,13 +1071,11 @@ static bool do_3same_pair(DisasContext *s, arg_3same 
*a, NeonGenTwoOpFn *fn)
 #define gen_helper_neon_pmax_u32  tcg_gen_umax_i32
 #define gen_helper_neon_pmin_s32  tcg_gen_smin_i32
 #define gen_helper_neon_pmin_u32  tcg_gen_umin_i32
-#define gen_helper_neon_padd_u32  tcg_gen_add_i32
 
 DO_3SAME_PAIR(VPMAX_S, pmax_s)
 DO_3SAME_PAIR(VPMIN_S, pmin_s)
 DO_3SAME_PAIR(VPMAX_U, pmax_u)
 DO_3SAME_PAIR(VPMIN_U, pmin_u)
-DO_3SAME_PAIR(VPADD, padd_u)
 
 #define DO_3SAME_VQDMULH(INSN, FUNC)\
 WRAP_ENV_FN(gen_##INSN##_tramp16, gen_helper_neon_##FUNC##_s16);\
-- 
2.34.1

[PATCH v2 40/67] target/arm: Inline scalar SQADD, UQADD, SQSUB, UQSUB

[Richard Henderson]

This eliminates the last uses of these neon helpers.
Incorporate the MO_64 expanders as an option to the vector expander.

Signed-off-by: Richard Henderson 
---
 target/arm/helper.h|  17 
 target/arm/tcg/translate.h |  15 +++
 target/arm/tcg/gengvec.c   | 116 +++
 target/arm/tcg/neon_helper.c   | 162 -
 target/arm/tcg/translate-a64.c |  67 --
 5 files changed, 169 insertions(+), 208 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index c76158d6d3..a14c040451 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -268,23 +268,6 @@ DEF_HELPER_FLAGS_2(fjcvtzs, TCG_CALL_NO_RWG, i64, f64, ptr)
 DEF_HELPER_FLAGS_3(check_hcr_el2_trap, TCG_CALL_NO_WG, void, env, i32, i32)
 
 /* neon_helper.c */
-DEF_HELPER_FLAGS_3(neon_qadd_u8, TCG_CALL_NO_RWG, i32, env, i32, i32)
-DEF_HELPER_FLAGS_3(neon_qadd_s8, TCG_CALL_NO_RWG, i32, env, i32, i32)
-DEF_HELPER_FLAGS_3(neon_qadd_u16, TCG_CALL_NO_RWG, i32, env, i32, i32)
-DEF_HELPER_FLAGS_3(neon_qadd_s16, TCG_CALL_NO_RWG, i32, env, i32, i32)
-DEF_HELPER_FLAGS_3(neon_qadd_u32, TCG_CALL_NO_RWG, i32, env, i32, i32)
-DEF_HELPER_FLAGS_3(neon_qadd_s32, TCG_CALL_NO_RWG, i32, env, i32, i32)
-DEF_HELPER_3(neon_qsub_u8, i32, env, i32, i32)
-DEF_HELPER_3(neon_qsub_s8, i32, env, i32, i32)
-DEF_HELPER_3(neon_qsub_u16, i32, env, i32, i32)
-DEF_HELPER_3(neon_qsub_s16, i32, env, i32, i32)
-DEF_HELPER_3(neon_qsub_u32, i32, env, i32, i32)
-DEF_HELPER_3(neon_qsub_s32, i32, env, i32, i32)
-DEF_HELPER_3(neon_qadd_u64, i64, env, i64, i64)
-DEF_HELPER_3(neon_qadd_s64, i64, env, i64, i64)
-DEF_HELPER_3(neon_qsub_u64, i64, env, i64, i64)
-DEF_HELPER_3(neon_qsub_s64, i64, env, i64, i64)
-
 DEF_HELPER_2(neon_hadd_s8, i32, i32, i32)
 DEF_HELPER_2(neon_hadd_u8, i32, i32, i32)
 DEF_HELPER_2(neon_hadd_s16, i32, i32, i32)
diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h
index 3abdbedfe5..87439dcc61 100644
--- a/target/arm/tcg/translate.h
+++ b/target/arm/tcg/translate.h
@@ -466,12 +466,27 @@ void gen_sshl_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b);
 void gen_ushl_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b);
 void gen_sshl_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b);
 
+void gen_uqadd_bhs(TCGv_i64 res, TCGv_i64 qc,
+   TCGv_i64 a, TCGv_i64 b, MemOp esz);
+void gen_uqadd_d(TCGv_i64 d, TCGv_i64 q, TCGv_i64 a, TCGv_i64 b);
 void gen_gvec_uqadd_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+
+void gen_sqadd_bhs(TCGv_i64 res, TCGv_i64 qc,
+   TCGv_i64 a, TCGv_i64 b, MemOp esz);
+void gen_sqadd_d(TCGv_i64 d, TCGv_i64 q, TCGv_i64 a, TCGv_i64 b);
 void gen_gvec_sqadd_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+
+void gen_uqsub_bhs(TCGv_i64 res, TCGv_i64 qc,
+   TCGv_i64 a, TCGv_i64 b, MemOp esz);
+void gen_uqsub_d(TCGv_i64 d, TCGv_i64 q, TCGv_i64 a, TCGv_i64 b);
 void gen_gvec_uqsub_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+
+void gen_sqsub_bhs(TCGv_i64 res, TCGv_i64 qc,
+   TCGv_i64 a, TCGv_i64 b, MemOp esz);
+void gen_sqsub_d(TCGv_i64 d, TCGv_i64 q, TCGv_i64 a, TCGv_i64 b);
 void gen_gvec_sqsub_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
 
diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c
index bfe6885a01..66a514ba86 100644
--- a/target/arm/tcg/gengvec.c
+++ b/target/arm/tcg/gengvec.c
@@ -1217,6 +1217,28 @@ void gen_gvec_sshl(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
 }
 
+void gen_uqadd_bhs(TCGv_i64 res, TCGv_i64 qc, TCGv_i64 a, TCGv_i64 b, MemOp 
esz)
+{
+uint64_t max = MAKE_64BIT_MASK(0, 8 << esz);
+TCGv_i64 tmp = tcg_temp_new_i64();
+
+tcg_gen_add_i64(tmp, a, b);
+tcg_gen_umin_i64(res, tmp, tcg_constant_i64(max));
+tcg_gen_xor_i64(tmp, tmp, res);
+tcg_gen_or_i64(qc, qc, tmp);
+}
+
+void gen_uqadd_d(TCGv_i64 res, TCGv_i64 qc, TCGv_i64 a, TCGv_i64 b)
+{
+TCGv_i64 t = tcg_temp_new_i64();
+
+tcg_gen_add_i64(t, a, b);
+tcg_gen_movcond_i64(TCG_COND_LTU, res, t, a,
+tcg_constant_i64(UINT64_MAX), t);
+tcg_gen_xor_i64(t, t, res);
+tcg_gen_or_i64(qc, qc, t);
+}
+
 static void gen_uqadd_vec(unsigned vece, TCGv_vec t, TCGv_vec qc,
   TCGv_vec a, TCGv_vec b)
 {
@@ -1250,6 +1272,7 @@ void gen_gvec_uqadd_qc(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
   .opt_opc = vecop_list,
   .vece = MO_32 },
 { .fniv = gen_uqadd_vec,
+  .fni8 = gen_uqadd_d,
   .fno = gen_helper_gvec_uqadd_d,
   .write_aofs = true,
   .opt_opc = vecop_list,
@@ -1259,6 +1282,41 @@ void gen_gvec_uqad

[PATCH v2 30/67] target/arm: Use gvec for neon faddp, fmaxp, fminp

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/helper.h |  7 -
 target/arm/tcg/translate-neon.c | 55 ++---
 target/arm/tcg/vec_helper.c | 45 ---
 3 files changed, 3 insertions(+), 104 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index 3268477329..065460ea80 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -650,13 +650,6 @@ DEF_HELPER_FLAGS_6(gvec_fcmlas_idx, TCG_CALL_NO_RWG,
 DEF_HELPER_FLAGS_6(gvec_fcmlad, TCG_CALL_NO_RWG,
void, ptr, ptr, ptr, ptr, ptr, i32)
 
-DEF_HELPER_FLAGS_5(neon_paddh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
-DEF_HELPER_FLAGS_5(neon_pmaxh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
-DEF_HELPER_FLAGS_5(neon_pminh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
-DEF_HELPER_FLAGS_5(neon_padds, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
-DEF_HELPER_FLAGS_5(neon_pmaxs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
-DEF_HELPER_FLAGS_5(neon_pmins, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
-
 DEF_HELPER_FLAGS_4(gvec_sstoh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_4(gvec_sitos, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_4(gvec_ustoh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
diff --git a/target/arm/tcg/translate-neon.c b/target/arm/tcg/translate-neon.c
index 144f18ba22..2326a05a0a 100644
--- a/target/arm/tcg/translate-neon.c
+++ b/target/arm/tcg/translate-neon.c
@@ -1144,6 +1144,9 @@ DO_3S_FP_GVEC(VFMA, gen_helper_gvec_vfma_s, 
gen_helper_gvec_vfma_h)
 DO_3S_FP_GVEC(VFMS, gen_helper_gvec_vfms_s, gen_helper_gvec_vfms_h)
 DO_3S_FP_GVEC(VRECPS, gen_helper_gvec_recps_nf_s, gen_helper_gvec_recps_nf_h)
 DO_3S_FP_GVEC(VRSQRTS, gen_helper_gvec_rsqrts_nf_s, 
gen_helper_gvec_rsqrts_nf_h)
+DO_3S_FP_GVEC(VPADD, gen_helper_gvec_faddp_s, gen_helper_gvec_faddp_h)
+DO_3S_FP_GVEC(VPMAX, gen_helper_gvec_fmaxp_s, gen_helper_gvec_fmaxp_h)
+DO_3S_FP_GVEC(VPMIN, gen_helper_gvec_fminp_s, gen_helper_gvec_fminp_h)
 
 WRAP_FP_GVEC(gen_VMAXNM_fp32_3s, FPST_STD, gen_helper_gvec_fmaxnum_s)
 WRAP_FP_GVEC(gen_VMAXNM_fp16_3s, FPST_STD_F16, gen_helper_gvec_fmaxnum_h)
@@ -1180,58 +1183,6 @@ static bool trans_VMINNM_fp_3s(DisasContext *s, 
arg_3same *a)
 return do_3same(s, a, gen_VMINNM_fp32_3s);
 }
 
-static bool do_3same_fp_pair(DisasContext *s, arg_3same *a,
- gen_helper_gvec_3_ptr *fn)
-{
-/* FP pairwise operations */
-TCGv_ptr fpstatus;
-
-if (!arm_dc_feature(s, ARM_FEATURE_NEON)) {
-return false;
-}
-
-/* UNDEF accesses to D16-D31 if they don't exist. */
-if (!dc_isar_feature(aa32_simd_r32, s) &&
-((a->vd | a->vn | a->vm) & 0x10)) {
-return false;
-}
-
-if (!vfp_access_check(s)) {
-return true;
-}
-
-assert(a->q == 0); /* enforced by decode patterns */
-
-
-fpstatus = fpstatus_ptr(a->size == MO_16 ? FPST_STD_F16 : FPST_STD);
-tcg_gen_gvec_3_ptr(vfp_reg_offset(1, a->vd),
-   vfp_reg_offset(1, a->vn),
-   vfp_reg_offset(1, a->vm),
-   fpstatus, 8, 8, 0, fn);
-
-return true;
-}
-
-/*
- * For all the functions using this macro, size == 1 means fp16,
- * which is an architecture extension we don't implement yet.
- */
-#define DO_3S_FP_PAIR(INSN,FUNC)\
-static bool trans_##INSN##_fp_3s(DisasContext *s, arg_3same *a) \
-{   \
-if (a->size == MO_16) { \
-if (!dc_isar_feature(aa32_fp16_arith, s)) { \
-return false;   \
-}   \
-return do_3same_fp_pair(s, a, FUNC##h); \
-}   \
-return do_3same_fp_pair(s, a, FUNC##s); \
-}
-
-DO_3S_FP_PAIR(VPADD, gen_helper_neon_padd)
-DO_3S_FP_PAIR(VPMAX, gen_helper_neon_pmax)
-DO_3S_FP_PAIR(VPMIN, gen_helper_neon_pmin)
-
 static bool do_vector_2sh(DisasContext *s, arg_2reg_shift *a, GVecGen2iFn *fn)
 {
 /* Handle a 2-reg-shift insn which can be vectorized. */
diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c
index 79e1fdcaa9..26a9ca9c14 100644
--- a/target/arm/tcg/vec_helper.c
+++ b/target/arm/tcg/vec_helper.c
@@ -2192,51 +2192,6 @@ DO_ABA(gvec_uaba_d, uint64_t)
 
 #undef DO_ABA
 
-#define DO_NEON_PAIRWISE(NAME, OP)  \
-void HELPER(NAME##s)(void *vd, void *vn, void *vm,  \
- void *stat, uint32_t oprsz)\
-{   \
-float_status *fpst = stat;  \
-float32 *d = vd;

[PATCH v2 28/67] target/arm: Convert FADDP to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/helper.h|  4 ++
 target/arm/tcg/a64.decode  | 12 +
 target/arm/tcg/translate-a64.c | 87 ++
 target/arm/tcg/vec_helper.c| 23 +
 4 files changed, 105 insertions(+), 21 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index ff6e3094f4..8441b49d1f 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -1048,6 +1048,10 @@ DEF_HELPER_FLAGS_5(gvec_uclamp_s, TCG_CALL_NO_RWG,
 DEF_HELPER_FLAGS_5(gvec_uclamp_d, TCG_CALL_NO_RWG,
void, ptr, ptr, ptr, ptr, i32)
 
+DEF_HELPER_FLAGS_5(gvec_faddp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(gvec_faddp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(gvec_faddp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+
 #ifdef TARGET_AARCH64
 #include "tcg/helper-a64.h"
 #include "tcg/helper-sve.h"
diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 84cb38f1dd..d2a02365e1 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -29,6 +29,7 @@
 &ri rd imm
 &rri_sf rd rn imm sf
 &i  imm
+&rr_e   rd rn esz
 &rrr_e  rd rn rm esz
 &rrx_e  rd rn rm idx esz
 &qrr_e  q rd rn esz
@@ -36,6 +37,9 @@
 &qrrx_e q rd rn rm idx esz
 &q_eq rd rn rm ra esz
 
+@rr_h    ... . .. rn:5 rd:5 &rr_e esz=1
+@rr_sd   ... . .. rn:5 rd:5 &rr_e esz=%esz_sd
+
 @rrr_h   ... rm:5 .. rn:5 rd:5  &rrr_e esz=1
 @rrr_sd  ... rm:5 .. rn:5 rd:5  &rrr_e esz=%esz_sd
 @rrr_hsd ... rm:5 .. rn:5 rd:5  &rrr_e esz=%esz_hsd
@@ -737,6 +741,11 @@ FRECPS_s0101 1110 0.1 . 1 1 . . 
@rrr_sd
 FRSQRTS_s   0101 1110 110 . 00111 1 . . @rrr_h
 FRSQRTS_s   0101 1110 1.1 . 1 1 . . @rrr_sd
 
+### Advanced SIMD scalar pairwise
+
+FADDP_s 0101 1110 0011  1101 10 . . @rr_h
+FADDP_s 0111 1110 0.11  1101 10 . . @rr_sd
+
 ### Advanced SIMD three same
 
 FADD_v  0.00 1110 010 . 00010 1 . . @qrrr_h
@@ -796,6 +805,9 @@ FRECPS_v0.00 1110 0.1 . 1 1 . . 
@qrrr_sd
 FRSQRTS_v   0.00 1110 110 . 00111 1 . . @qrrr_h
 FRSQRTS_v   0.00 1110 1.1 . 1 1 . . @qrrr_sd
 
+FADDP_v 0.10 1110 010 . 00010 1 . . @qrrr_h
+FADDP_v 0.10 1110 0.1 . 11010 1 . . @qrrr_sd
+
 ### Advanced SIMD scalar x indexed element
 
 FMUL_si 0101  00 ..  1001 . 0 . .   @rrx_h
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index a7537a5104..78949ab34f 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5210,6 +5210,13 @@ static gen_helper_gvec_3_ptr * const f_vector_frsqrts[3] 
= {
 };
 TRANS(FRSQRTS_v, do_fp3_vector, a, f_vector_frsqrts)
 
+static gen_helper_gvec_3_ptr * const f_vector_faddp[3] = {
+gen_helper_gvec_faddp_h,
+gen_helper_gvec_faddp_s,
+gen_helper_gvec_faddp_d,
+};
+TRANS(FADDP_v, do_fp3_vector, a, f_vector_faddp)
+
 /*
  * Advanced SIMD scalar/vector x indexed element
  */
@@ -5395,6 +5402,56 @@ static bool do_fmla_vector_idx(DisasContext *s, 
arg_qrrx_e *a, bool neg)
 TRANS(FMLA_vi, do_fmla_vector_idx, a, false)
 TRANS(FMLS_vi, do_fmla_vector_idx, a, true)
 
+/*
+ * Advanced SIMD scalar pairwise
+ */
+
+static bool do_fp3_scalar_pair(DisasContext *s, arg_rr_e *a, const FPScalar *f)
+{
+switch (a->esz) {
+case MO_64:
+if (fp_access_check(s)) {
+TCGv_i64 t0 = tcg_temp_new_i64();
+TCGv_i64 t1 = tcg_temp_new_i64();
+
+read_vec_element(s, t0, a->rn, 0, MO_64);
+read_vec_element(s, t1, a->rn, 1, MO_64);
+f->gen_d(t0, t0, t1, fpstatus_ptr(FPST_FPCR));
+write_fp_dreg(s, a->rd, t0);
+}
+break;
+case MO_32:
+if (fp_access_check(s)) {
+TCGv_i32 t0 = tcg_temp_new_i32();
+TCGv_i32 t1 = tcg_temp_new_i32();
+
+read_vec_element_i32(s, t0, a->rn, 0, MO_32);
+read_vec_element_i32(s, t1, a->rn, 1, MO_32);
+f->gen_s(t0, t0, t1, fpstatus_ptr(FPST_FPCR));
+write_fp_sreg(s, a->rd, t0);
+}
+break;
+case MO_16:
+if (!dc_isar_feature(aa64_fp16, s)) {
+return false;
+}
+if (fp_access_check(s)) {
+TCGv_i32 t0 = tcg_temp_new_i32();
+TCGv_i32 t1 = tcg_temp_new_i32();
+
+read_vec_element_i32(s, t0, a->rn, 0, MO_16);
+read_vec_element_i32(s, t1, a->rn, 1, MO_16);
+f->gen_h(t0, t0, t1, fpstatus_ptr(FPST_FPCR_F16));
+write_fp_sreg(s, a->rd, t0);
+}
+break;
+default:
+g_assert_

[PATCH v2 29/67] target/arm: Convert FMAXP, FMINP, FMAXNMP, FMINNMP to decodetree

[Richard Henderson]

These are the last instructions within disas_simd_three_reg_same_fp16,
so remove it.

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/helper.h|  16 ++
 target/arm/tcg/a64.decode  |  24 +++
 target/arm/tcg/translate-a64.c | 296 ++---
 target/arm/tcg/vec_helper.c|  16 ++
 4 files changed, 107 insertions(+), 245 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index 8441b49d1f..3268477329 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -1052,6 +1052,22 @@ DEF_HELPER_FLAGS_5(gvec_faddp_h, TCG_CALL_NO_RWG, void, 
ptr, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_5(gvec_faddp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 DEF_HELPER_FLAGS_5(gvec_faddp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 
+DEF_HELPER_FLAGS_5(gvec_fmaxp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(gvec_fmaxp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(gvec_fmaxp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+
+DEF_HELPER_FLAGS_5(gvec_fminp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(gvec_fminp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(gvec_fminp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+
+DEF_HELPER_FLAGS_5(gvec_fmaxnump_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(gvec_fmaxnump_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(gvec_fmaxnump_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+
+DEF_HELPER_FLAGS_5(gvec_fminnump_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(gvec_fminnump_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(gvec_fminnump_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+
 #ifdef TARGET_AARCH64
 #include "tcg/helper-a64.h"
 #include "tcg/helper-sve.h"
diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index d2a02365e1..43557fdccc 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -746,6 +746,18 @@ FRSQRTS_s   0101 1110 1.1 . 1 1 . . 
@rrr_sd
 FADDP_s 0101 1110 0011  1101 10 . . @rr_h
 FADDP_s 0111 1110 0.11  1101 10 . . @rr_sd
 
+FMAXP_s 0101 1110 0011   10 . . @rr_h
+FMAXP_s 0111 1110 0.11   10 . . @rr_sd
+
+FMINP_s 0101 1110 1011   10 . . @rr_h
+FMINP_s 0111 1110 1.11   10 . . @rr_sd
+
+FMAXNMP_s   0101 1110 0011  1100 10 . . @rr_h
+FMAXNMP_s   0111 1110 0.11  1100 10 . . @rr_sd
+
+FMINNMP_s   0101 1110 1011  1100 10 . . @rr_h
+FMINNMP_s   0111 1110 1.11  1100 10 . . @rr_sd
+
 ### Advanced SIMD three same
 
 FADD_v  0.00 1110 010 . 00010 1 . . @qrrr_h
@@ -808,6 +820,18 @@ FRSQRTS_v   0.00 1110 1.1 . 1 1 . . 
@qrrr_sd
 FADDP_v 0.10 1110 010 . 00010 1 . . @qrrr_h
 FADDP_v 0.10 1110 0.1 . 11010 1 . . @qrrr_sd
 
+FMAXP_v 0.10 1110 010 . 00110 1 . . @qrrr_h
+FMAXP_v 0.10 1110 0.1 . 0 1 . . @qrrr_sd
+
+FMINP_v 0.10 1110 110 . 00110 1 . . @qrrr_h
+FMINP_v 0.10 1110 1.1 . 0 1 . . @qrrr_sd
+
+FMAXNMP_v   0.10 1110 010 . 0 1 . . @qrrr_h
+FMAXNMP_v   0.10 1110 0.1 . 11000 1 . . @qrrr_sd
+
+FMINNMP_v   0.10 1110 110 . 0 1 . . @qrrr_h
+FMINNMP_v   0.10 1110 1.1 . 11000 1 . . @qrrr_sd
+
 ### Advanced SIMD scalar x indexed element
 
 FMUL_si 0101  00 ..  1001 . 0 . .   @rrx_h
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 78949ab34f..07415bd285 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5217,6 +5217,34 @@ static gen_helper_gvec_3_ptr * const f_vector_faddp[3] = 
{
 };
 TRANS(FADDP_v, do_fp3_vector, a, f_vector_faddp)
 
+static gen_helper_gvec_3_ptr * const f_vector_fmaxp[3] = {
+gen_helper_gvec_fmaxp_h,
+gen_helper_gvec_fmaxp_s,
+gen_helper_gvec_fmaxp_d,
+};
+TRANS(FMAXP_v, do_fp3_vector, a, f_vector_fmaxp)
+
+static gen_helper_gvec_3_ptr * const f_vector_fminp[3] = {
+gen_helper_gvec_fminp_h,
+gen_helper_gvec_fminp_s,
+gen_helper_gvec_fminp_d,
+};
+TRANS(FMINP_v, do_fp3_vector, a, f_vector_fminp)
+
+static gen_helper_gvec_3_ptr * const f_vector_fmaxnmp[3] = {
+gen_helper_gvec_fmaxnump_h,
+gen_helper_gvec_fmaxnump_s,
+gen_helper_gvec_fmaxnump_d,
+};
+TRANS(FMAXNMP_v, do_fp3_vector, a, f_vector_fmaxnmp)
+
+static gen_helper_gvec_3_ptr * const f_vector_fminnmp[3] = {
+gen_helper_gvec_fminnump_h,
+gen_helper_gvec_fminnump_s,
+gen_helper_gvec_fminnump_d,
+};
+TRANS(FMINNMP_v, do_fp3_vector, a, f_vector_fminnmp)
+
 /*
  * Advanced SIMD scalar/vector 

[PATCH v2 58/67] target/arm: Convert SRHADD, URHADD to gvec

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/helper.h |   7 --
 target/arm/tcg/translate.h  |   4 +
 target/arm/tcg/gengvec.c| 144 
 target/arm/tcg/neon_helper.c|  27 --
 target/arm/tcg/translate-a64.c  |  48 ++-
 target/arm/tcg/translate-neon.c |  26 +-
 6 files changed, 158 insertions(+), 98 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index b95f24ed0a..85f9302563 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -268,13 +268,6 @@ DEF_HELPER_FLAGS_2(fjcvtzs, TCG_CALL_NO_RWG, i64, f64, ptr)
 DEF_HELPER_FLAGS_3(check_hcr_el2_trap, TCG_CALL_NO_WG, void, env, i32, i32)
 
 /* neon_helper.c */
-DEF_HELPER_2(neon_rhadd_s8, i32, i32, i32)
-DEF_HELPER_2(neon_rhadd_u8, i32, i32, i32)
-DEF_HELPER_2(neon_rhadd_s16, i32, i32, i32)
-DEF_HELPER_2(neon_rhadd_u16, i32, i32, i32)
-DEF_HELPER_2(neon_rhadd_s32, s32, s32, s32)
-DEF_HELPER_2(neon_rhadd_u32, i32, i32, i32)
-
 DEF_HELPER_2(neon_pmin_u8, i32, i32, i32)
 DEF_HELPER_2(neon_pmin_s8, i32, i32, i32)
 DEF_HELPER_2(neon_pmin_u16, i32, i32, i32)
diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h
index 315e0afd04..3b1e68b779 100644
--- a/target/arm/tcg/translate.h
+++ b/target/arm/tcg/translate.h
@@ -480,6 +480,10 @@ void gen_gvec_shsub(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
 void gen_gvec_uhsub(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
 uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_srhadd(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_urhadd(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
 
 void gen_cmtst_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b);
 void gen_ushl_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b);
diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c
index 6a54ad2d21..32caabd126 100644
--- a/target/arm/tcg/gengvec.c
+++ b/target/arm/tcg/gengvec.c
@@ -2140,3 +2140,147 @@ void gen_gvec_uhsub(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 assert(vece <= MO_32);
 tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &g[vece]);
 }
+
+static void gen_srhadd8_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+TCGv_i64 t = tcg_temp_new_i64();
+
+tcg_gen_or_i64(t, a, b);
+tcg_gen_vec_sar8i_i64(a, a, 1);
+tcg_gen_vec_sar8i_i64(b, b, 1);
+tcg_gen_andi_i64(t, t, dup_const(MO_8, 1));
+tcg_gen_vec_add8_i64(d, a, b);
+tcg_gen_vec_add8_i64(d, d, t);
+}
+
+static void gen_srhadd16_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+TCGv_i64 t = tcg_temp_new_i64();
+
+tcg_gen_or_i64(t, a, b);
+tcg_gen_vec_sar16i_i64(a, a, 1);
+tcg_gen_vec_sar16i_i64(b, b, 1);
+tcg_gen_andi_i64(t, t, dup_const(MO_16, 1));
+tcg_gen_vec_add16_i64(d, a, b);
+tcg_gen_vec_add16_i64(d, d, t);
+}
+
+static void gen_srhadd_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b)
+{
+TCGv_i32 t = tcg_temp_new_i32();
+
+tcg_gen_or_i32(t, a, b);
+tcg_gen_sari_i32(a, a, 1);
+tcg_gen_sari_i32(b, b, 1);
+tcg_gen_andi_i32(t, t, 1);
+tcg_gen_add_i32(d, a, b);
+tcg_gen_add_i32(d, d, t);
+}
+
+static void gen_srhadd_vec(unsigned vece, TCGv_vec d, TCGv_vec a, TCGv_vec b)
+{
+TCGv_vec t = tcg_temp_new_vec_matching(d);
+
+tcg_gen_or_vec(vece, t, a, b);
+tcg_gen_sari_vec(vece, a, a, 1);
+tcg_gen_sari_vec(vece, b, b, 1);
+tcg_gen_and_vec(vece, t, t, tcg_constant_vec_matching(d, vece, 1));
+tcg_gen_add_vec(vece, d, a, b);
+tcg_gen_add_vec(vece, d, d, t);
+}
+
+void gen_gvec_srhadd(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+static const TCGOpcode vecop_list[] = {
+INDEX_op_sari_vec, INDEX_op_add_vec, 0
+};
+static const GVecGen3 g[] = {
+{ .fni8 = gen_srhadd8_i64,
+  .fniv = gen_srhadd_vec,
+  .opt_opc = vecop_list,
+  .vece = MO_8 },
+{ .fni8 = gen_srhadd16_i64,
+  .fniv = gen_srhadd_vec,
+  .opt_opc = vecop_list,
+  .vece = MO_16 },
+{ .fni4 = gen_srhadd_i32,
+  .fniv = gen_srhadd_vec,
+  .opt_opc = vecop_list,
+  .vece = MO_32 },
+};
+assert(vece <= MO_32);
+tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &g[vece]);
+}
+
+static void gen_urhadd8_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+TCGv_i64 t = tcg_temp_new_i64();
+
+tcg_gen_or_i64(t, a, b);
+tcg_gen_vec_shr8i_i64(a, a, 1);
+tcg_gen_vec_shr8i_i64(b, b, 1);
+tcg_gen_andi_i64(t, t, dup_const(MO_8, 1));
+tcg_gen_vec_add8_i64(d, a, b);
+tcg_gen_vec_add8_i64(d, d, t);
+}
+
+static void gen_urhadd16_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+TCGv_i64 t = tcg_temp_new_i64();
+
+tcg_gen_or_i64(t, 

[PATCH v2 09/67] target/arm: Convert Cryptographic AES to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  | 21 +++--
 target/arm/tcg/translate-a64.c | 86 +++---
 2 files changed, 54 insertions(+), 53 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 0e7656fd15..1de09903dc 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -19,11 +19,17 @@
 # This file is processed by scripts/decodetree.py
 #
 
-&r   rn
-&ri  rd imm
-&rri_sf  rd rn imm sf
-&i   imm
+%rd 0:5
 
+&r  rn
+&ri rd imm
+&rri_sf rd rn imm sf
+&i  imm
+&qrr_e  q rd rn esz
+&qrrr_e q rd rn rm esz
+
+@rr_q1e0  .. rn:5 rd:5  &qrr_e q=1 esz=0
+@r2r_q1e0     .. rm:5 rd:5  &qrrr_e rn=%rd q=1 
esz=0
 
 ### Data Processing - Immediate
 
@@ -590,3 +596,10 @@ CPYFE   00 011 0 01100 .  01 . . 
@cpy
 CPYP00 011 1 01000 .  01 . . @cpy
 CPYM00 011 1 01010 .  01 . . @cpy
 CPYE00 011 1 01100 .  01 . . @cpy
+
+### Cryptographic AES
+
+AESE01001110 00 10100 00100 10 . .  @r2r_q1e0
+AESD01001110 00 10100 00101 10 . .  @r2r_q1e0
+AESMC   01001110 00 10100 00110 10 . .  @rr_q1e0
+AESIMC  01001110 00 10100 00111 10 . .  @rr_q1e0
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 8842ff634d..3894db4bee 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -1313,6 +1313,34 @@ bool sme_enabled_check_with_svcr(DisasContext *s, 
unsigned req)
 return true;
 }
 
+/*
+ * Expanders for AdvSIMD translation functions.
+ */
+
+static bool do_gvec_op2_ool(DisasContext *s, arg_qrr_e *a, int data,
+gen_helper_gvec_2 *fn)
+{
+if (!a->q && a->esz == MO_64) {
+return false;
+}
+if (fp_access_check(s)) {
+gen_gvec_op2_ool(s, a->q, a->rd, a->rn, data, fn);
+}
+return true;
+}
+
+static bool do_gvec_op3_ool(DisasContext *s, arg_qrrr_e *a, int data,
+gen_helper_gvec_3 *fn)
+{
+if (!a->q && a->esz == MO_64) {
+return false;
+}
+if (fp_access_check(s)) {
+gen_gvec_op3_ool(s, a->q, a->rd, a->rn, a->rm, data, fn);
+}
+return true;
+}
+
 /*
  * This utility function is for doing register extension with an
  * optional shift. You will likely want to pass a temporary for the
@@ -4560,6 +4588,15 @@ static bool trans_EXTR(DisasContext *s, arg_extract *a)
 return true;
 }
 
+/*
+ * Cryptographic AES
+ */
+
+TRANS_FEAT(AESE, aa64_aes, do_gvec_op3_ool, a, 0, gen_helper_crypto_aese)
+TRANS_FEAT(AESD, aa64_aes, do_gvec_op3_ool, a, 0, gen_helper_crypto_aesd)
+TRANS_FEAT(AESMC, aa64_aes, do_gvec_op2_ool, a, 0, gen_helper_crypto_aesmc)
+TRANS_FEAT(AESIMC, aa64_aes, do_gvec_op2_ool, a, 0, gen_helper_crypto_aesimc)
+
 /* Shift a TCGv src by TCGv shift_amount, put result in dst.
  * Note that it is the caller's responsibility to ensure that the
  * shift amount is in range (ie 0..31 or 0..63) and provide the ARM
@@ -13460,54 +13497,6 @@ static void disas_simd_indexed(DisasContext *s, 
uint32_t insn)
 }
 }
 
-/* Crypto AES
- *  31 24 23  22 21   17 1612 11 10 95 40
- * +-+--+---++-+--+--+
- * | 0 1 0 0 1 1 1 0 | size | 1 0 1 0 0 | opcode | 1 0 |  Rn  |  Rd  |
- * +-+--+---++-+--+--+
- */
-static void disas_crypto_aes(DisasContext *s, uint32_t insn)
-{
-int size = extract32(insn, 22, 2);
-int opcode = extract32(insn, 12, 5);
-int rn = extract32(insn, 5, 5);
-int rd = extract32(insn, 0, 5);
-gen_helper_gvec_2 *genfn2 = NULL;
-gen_helper_gvec_3 *genfn3 = NULL;
-
-if (!dc_isar_feature(aa64_aes, s) || size != 0) {
-unallocated_encoding(s);
-return;
-}
-
-switch (opcode) {
-case 0x4: /* AESE */
-genfn3 = gen_helper_crypto_aese;
-break;
-case 0x6: /* AESMC */
-genfn2 = gen_helper_crypto_aesmc;
-break;
-case 0x5: /* AESD */
-genfn3 = gen_helper_crypto_aesd;
-break;
-case 0x7: /* AESIMC */
-genfn2 = gen_helper_crypto_aesimc;
-break;
-default:
-unallocated_encoding(s);
-return;
-}
-
-if (!fp_access_check(s)) {
-return;
-}
-if (genfn2) {
-gen_gvec_op2_ool(s, true, rd, rn, 0, genfn2);
-} else {
-gen_gvec_op3_ool(s, true, rd, rd, rn, 0, genfn3);
-}
-}
-
 /* Crypto three-reg SHA
  *  31 24 23  22  21 20  16  15 1412 11 10 95 40
  * +-+--+---+--+---++-+--+--+
@@ -13917,7 +13906,6 @@ static const AArch64DecodeTabl

[PATCH v2 36/67] target/arm: Convert disas_simd_3same_logic to decodetree

[Richard Henderson]

This includes AND, ORR, EOR, BIC, ORN, BSF, BIT, BIF.

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  | 10 +
 target/arm/tcg/translate-a64.c | 68 ++
 2 files changed, 29 insertions(+), 49 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 7e993ed345..f48adef5bb 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -55,6 +55,7 @@
 @rrr_q1e3    ... rm:5 .. rn:5 rd:5  &qrrr_e q=1 esz=3
 @_q1e3   ... rm:5 . ra:5 rn:5 rd:5  &q_e q=1 esz=3
 
+@qrrr_b . q:1 .. ... rm:5 .. rn:5 rd:5  &qrrr_e esz=0
 @qrrr_h . q:1 .. ... rm:5 .. rn:5 rd:5  &qrrr_e esz=1
 @qrrr_sd. q:1 .. ... rm:5 .. rn:5 rd:5  &qrrr_e esz=%esz_sd
 @qrrr_e . q:1 .. esz:2 . rm:5 .. rn:5 rd:5  &qrrr_e
@@ -847,6 +848,15 @@ SMINP_v 0.00 1110 ..1 . 10101 1 . . 
@qrrr_e
 UMAXP_v 0.10 1110 ..1 . 10100 1 . . @qrrr_e
 UMINP_v 0.10 1110 ..1 . 10101 1 . . @qrrr_e
 
+AND_v   0.00 1110 001 . 00011 1 . . @qrrr_b
+BIC_v   0.00 1110 011 . 00011 1 . . @qrrr_b
+ORR_v   0.00 1110 101 . 00011 1 . . @qrrr_b
+ORN_v   0.00 1110 111 . 00011 1 . . @qrrr_b
+EOR_v   0.10 1110 001 . 00011 1 . . @qrrr_b
+BSL_v   0.10 1110 011 . 00011 1 . . @qrrr_b
+BIT_v   0.10 1110 101 . 00011 1 . . @qrrr_b
+BIF_v   0.10 1110 111 . 00011 1 . . @qrrr_b
+
 ### Advanced SIMD scalar x indexed element
 
 FMUL_si 0101  00 ..  1001 . 0 . .   @rrx_h
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index a4ff1fd202..9167e4d0bd 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5280,6 +5280,24 @@ TRANS(SMINP_v, do_gvec_fn3_no64, a, gen_gvec_sminp)
 TRANS(UMAXP_v, do_gvec_fn3_no64, a, gen_gvec_umaxp)
 TRANS(UMINP_v, do_gvec_fn3_no64, a, gen_gvec_uminp)
 
+TRANS(AND_v, do_gvec_fn3, a, tcg_gen_gvec_and)
+TRANS(BIC_v, do_gvec_fn3, a, tcg_gen_gvec_andc)
+TRANS(ORR_v, do_gvec_fn3, a, tcg_gen_gvec_or)
+TRANS(ORN_v, do_gvec_fn3, a, tcg_gen_gvec_orc)
+TRANS(EOR_v, do_gvec_fn3, a, tcg_gen_gvec_xor)
+
+static bool do_bitsel(DisasContext *s, bool is_q, int d, int a, int b, int c)
+{
+if (fp_access_check(s)) {
+gen_gvec_fn4(s, is_q, d, a, b, c, tcg_gen_gvec_bitsel, 0);
+}
+return true;
+}
+
+TRANS(BSL_v, do_bitsel, a->q, a->rd, a->rd, a->rn, a->rm)
+TRANS(BIT_v, do_bitsel, a->q, a->rd, a->rm, a->rn, a->rd)
+TRANS(BIF_v, do_bitsel, a->q, a->rd, a->rm, a->rd, a->rn)
+
 /*
  * Advanced SIMD scalar/vector x indexed element
  */
@@ -10901,52 +10919,6 @@ static void disas_simd_three_reg_diff(DisasContext *s, 
uint32_t insn)
 }
 }
 
-/* Logic op (opcode == 3) subgroup of C3.6.16. */
-static void disas_simd_3same_logic(DisasContext *s, uint32_t insn)
-{
-int rd = extract32(insn, 0, 5);
-int rn = extract32(insn, 5, 5);
-int rm = extract32(insn, 16, 5);
-int size = extract32(insn, 22, 2);
-bool is_u = extract32(insn, 29, 1);
-bool is_q = extract32(insn, 30, 1);
-
-if (!fp_access_check(s)) {
-return;
-}
-
-switch (size + 4 * is_u) {
-case 0: /* AND */
-gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_and, 0);
-return;
-case 1: /* BIC */
-gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_andc, 0);
-return;
-case 2: /* ORR */
-gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_or, 0);
-return;
-case 3: /* ORN */
-gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_orc, 0);
-return;
-case 4: /* EOR */
-gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_xor, 0);
-return;
-
-case 5: /* BSL bitwise select */
-gen_gvec_fn4(s, is_q, rd, rd, rn, rm, tcg_gen_gvec_bitsel, 0);
-return;
-case 6: /* BIT, bitwise insert if true */
-gen_gvec_fn4(s, is_q, rd, rm, rn, rd, tcg_gen_gvec_bitsel, 0);
-return;
-case 7: /* BIF, bitwise insert if false */
-gen_gvec_fn4(s, is_q, rd, rm, rd, rn, tcg_gen_gvec_bitsel, 0);
-return;
-
-default:
-g_assert_not_reached();
-}
-}
-
 /* Integer op subgroup of C3.6.16. */
 static void disas_simd_3same_int(DisasContext *s, uint32_t insn)
 {
@@ -11212,12 +11184,10 @@ static void disas_simd_three_reg_same(DisasContext 
*s, uint32_t insn)
 int opcode = extract32(insn, 11, 5);
 
 switch (opcode) {
-case 0x3: /* logic ops */
-disas_simd_3same_logic(s, insn);
-break;
 default:
 disas_simd_3same_int(s, insn);
 break;
+case 0x3: /* logic ops */
 case 0x14: /* SMAXP, UMAXP */
 case 0x15: /* SMINP, UMINP */
 case 0x17: /* ADDP */
-- 
2.34.1

[PATCH v2 56/67] target/arm: Convert SHSUB, UHSUB to gvec

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/helper.h |   6 --
 target/arm/tcg/translate.h  |   4 +
 target/arm/tcg/gengvec.c| 144 
 target/arm/tcg/neon_helper.c|  27 --
 target/arm/tcg/translate-a64.c  |  17 ++--
 target/arm/tcg/translate-neon.c |   4 +-
 6 files changed, 157 insertions(+), 45 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index b26bfcb079..b95f24ed0a 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -274,12 +274,6 @@ DEF_HELPER_2(neon_rhadd_s16, i32, i32, i32)
 DEF_HELPER_2(neon_rhadd_u16, i32, i32, i32)
 DEF_HELPER_2(neon_rhadd_s32, s32, s32, s32)
 DEF_HELPER_2(neon_rhadd_u32, i32, i32, i32)
-DEF_HELPER_2(neon_hsub_s8, i32, i32, i32)
-DEF_HELPER_2(neon_hsub_u8, i32, i32, i32)
-DEF_HELPER_2(neon_hsub_s16, i32, i32, i32)
-DEF_HELPER_2(neon_hsub_u16, i32, i32, i32)
-DEF_HELPER_2(neon_hsub_s32, s32, s32, s32)
-DEF_HELPER_2(neon_hsub_u32, i32, i32, i32)
 
 DEF_HELPER_2(neon_pmin_u8, i32, i32, i32)
 DEF_HELPER_2(neon_pmin_s8, i32, i32, i32)
diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h
index dd99d76bf2..315e0afd04 100644
--- a/target/arm/tcg/translate.h
+++ b/target/arm/tcg/translate.h
@@ -476,6 +476,10 @@ void gen_gvec_shadd(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
 void gen_gvec_uhadd(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
 uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_shsub(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_uhsub(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
 
 void gen_cmtst_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b);
 void gen_ushl_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b);
diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c
index 0627cec6b2..6a54ad2d21 100644
--- a/target/arm/tcg/gengvec.c
+++ b/target/arm/tcg/gengvec.c
@@ -1996,3 +1996,147 @@ void gen_gvec_uhadd(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 tcg_debug_assert(vece <= MO_32);
 tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &g[vece]);
 }
+
+static void gen_shsub8_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+TCGv_i64 t = tcg_temp_new_i64();
+
+tcg_gen_andc_i64(t, b, a);
+tcg_gen_vec_sar8i_i64(a, a, 1);
+tcg_gen_vec_sar8i_i64(b, b, 1);
+tcg_gen_andi_i64(t, t, dup_const(MO_8, 1));
+tcg_gen_vec_sub8_i64(d, a, b);
+tcg_gen_vec_sub8_i64(d, d, t);
+}
+
+static void gen_shsub16_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+TCGv_i64 t = tcg_temp_new_i64();
+
+tcg_gen_andc_i64(t, b, a);
+tcg_gen_vec_sar16i_i64(a, a, 1);
+tcg_gen_vec_sar16i_i64(b, b, 1);
+tcg_gen_andi_i64(t, t, dup_const(MO_16, 1));
+tcg_gen_vec_sub16_i64(d, a, b);
+tcg_gen_vec_sub16_i64(d, d, t);
+}
+
+static void gen_shsub_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b)
+{
+TCGv_i32 t = tcg_temp_new_i32();
+
+tcg_gen_andc_i32(t, b, a);
+tcg_gen_sari_i32(a, a, 1);
+tcg_gen_sari_i32(b, b, 1);
+tcg_gen_andi_i32(t, t, 1);
+tcg_gen_sub_i32(d, a, b);
+tcg_gen_sub_i32(d, d, t);
+}
+
+static void gen_shsub_vec(unsigned vece, TCGv_vec d, TCGv_vec a, TCGv_vec b)
+{
+TCGv_vec t = tcg_temp_new_vec_matching(d);
+
+tcg_gen_andc_vec(vece, t, b, a);
+tcg_gen_sari_vec(vece, a, a, 1);
+tcg_gen_sari_vec(vece, b, b, 1);
+tcg_gen_and_vec(vece, t, t, tcg_constant_vec_matching(d, vece, 1));
+tcg_gen_sub_vec(vece, d, a, b);
+tcg_gen_sub_vec(vece, d, d, t);
+}
+
+void gen_gvec_shsub(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+static const TCGOpcode vecop_list[] = {
+INDEX_op_sari_vec, INDEX_op_sub_vec, 0
+};
+static const GVecGen3 g[4] = {
+{ .fni8 = gen_shsub8_i64,
+  .fniv = gen_shsub_vec,
+  .opt_opc = vecop_list,
+  .vece = MO_8 },
+{ .fni8 = gen_shsub16_i64,
+  .fniv = gen_shsub_vec,
+  .opt_opc = vecop_list,
+  .vece = MO_16 },
+{ .fni4 = gen_shsub_i32,
+  .fniv = gen_shsub_vec,
+  .opt_opc = vecop_list,
+  .vece = MO_32 },
+};
+assert(vece <= MO_32);
+tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &g[vece]);
+}
+
+static void gen_uhsub8_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+TCGv_i64 t = tcg_temp_new_i64();
+
+tcg_gen_andc_i64(t, b, a);
+tcg_gen_vec_shr8i_i64(a, a, 1);
+tcg_gen_vec_shr8i_i64(b, b, 1);
+tcg_gen_andi_i64(t, t, dup_const(MO_8, 1));
+tcg_gen_vec_sub8_i64(d, a, b);
+tcg_gen_vec_sub8_i64(d, d, t);
+}
+
+static void gen_uhsub16_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+TCGv_i64 t = tcg_temp_new_i64();
+
+tcg_gen_andc_i64(t, b, a);
+tcg_gen_vec_shr16i_i64(a,

[PATCH v2 04/67] target/arm: Zero-extend writeback for fp16 FCVTZS (scalar, integer)

[Richard Henderson]

Fixes RISU mismatch for "fcvtzs h31, h0, #14".

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/translate-a64.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 4126aaa27e..d97acdbaf9 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -8707,6 +8707,9 @@ static void handle_simd_shift_fpint_conv(DisasContext *s, 
bool is_scalar,
 read_vec_element_i32(s, tcg_op, rn, pass, size);
 fn(tcg_op, tcg_op, tcg_shift, tcg_fpstatus);
 if (is_scalar) {
+if (size == MO_16 && !is_u) {
+tcg_gen_ext16u_i32(tcg_op, tcg_op);
+}
 write_fp_sreg(s, rd, tcg_op);
 } else {
 write_vec_element_i32(s, tcg_op, rd, pass, size);
-- 
2.34.1

[PATCH v2 60/67] target/arm: Convert SMAX, SMIN, UMAX, UMIN to decodetree

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  4 
 target/arm/tcg/translate-a64.c | 22 ++
 2 files changed, 10 insertions(+), 16 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 1c448b4f7c..bc98963bc5 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -918,6 +918,10 @@ SHSUB_v 0.00 1110 ..1 . 00100 1 . . 
@qrrr_e
 UHSUB_v 0.10 1110 ..1 . 00100 1 . . @qrrr_e
 SRHADD_v0.00 1110 ..1 . 00010 1 . . @qrrr_e
 URHADD_v0.10 1110 ..1 . 00010 1 . . @qrrr_e
+SMAX_v  0.00 1110 ..1 . 01100 1 . . @qrrr_e
+UMAX_v  0.10 1110 ..1 . 01100 1 . . @qrrr_e
+SMIN_v  0.00 1110 ..1 . 01101 1 . . @qrrr_e
+UMIN_v  0.10 1110 ..1 . 01101 1 . . @qrrr_e
 
 ### Advanced SIMD scalar x indexed element
 
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 9ef5de6755..db6f59df17 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5460,6 +5460,10 @@ TRANS(SHSUB_v, do_gvec_fn3_no64, a, gen_gvec_shsub)
 TRANS(UHSUB_v, do_gvec_fn3_no64, a, gen_gvec_uhsub)
 TRANS(SRHADD_v, do_gvec_fn3_no64, a, gen_gvec_srhadd)
 TRANS(URHADD_v, do_gvec_fn3_no64, a, gen_gvec_urhadd)
+TRANS(SMAX_v, do_gvec_fn3_no64, a, tcg_gen_gvec_smax)
+TRANS(UMAX_v, do_gvec_fn3_no64, a, tcg_gen_gvec_umax)
+TRANS(SMIN_v, do_gvec_fn3_no64, a, tcg_gen_gvec_smin)
+TRANS(UMIN_v, do_gvec_fn3_no64, a, tcg_gen_gvec_umin)
 
 static bool do_cmop_v(DisasContext *s, arg_qrrr_e *a, TCGCond cond)
 {
@@ -10925,8 +10929,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 return;
 }
 /* fall through */
-case 0xc: /* SMAX, UMAX */
-case 0xd: /* SMIN, UMIN */
 case 0xe: /* SABD, UABD */
 case 0xf: /* SABA, UABA */
 case 0x12: /* MLA, MLS */
@@ -10959,6 +10961,8 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 case 0x09: /* SQSHL, UQSHL */
 case 0x0a: /* SRSHL, URSHL */
 case 0x0b: /* SQRSHL, UQRSHL */
+case 0x0c: /* SMAX, UMAX */
+case 0x0d: /* SMIN, UMIN */
 case 0x10: /* ADD, SUB */
 case 0x11: /* CMTST, CMEQ */
 unallocated_encoding(s);
@@ -10970,20 +10974,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 }
 
 switch (opcode) {
-case 0x0c: /* SMAX, UMAX */
-if (u) {
-gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_umax, size);
-} else {
-gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_smax, size);
-}
-return;
-case 0x0d: /* SMIN, UMIN */
-if (u) {
-gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_umin, size);
-} else {
-gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_smin, size);
-}
-return;
 case 0xe: /* SABD, UABD */
 if (u) {
 gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_uabd, size);
-- 
2.34.1

[PATCH v2 12/67] target/arm: Convert Cryptographic 3-register SHA512 to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  | 11 
 target/arm/tcg/translate-a64.c | 97 --
 2 files changed, 32 insertions(+), 76 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 350afabc77..c342c27608 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -31,6 +31,7 @@
 @rr_q1e0  .. rn:5 rd:5  &qrr_e q=1 esz=0
 @r2r_q1e0     .. rm:5 rd:5  &qrrr_e rn=%rd q=1 
esz=0
 @rrr_q1e0    ... rm:5 .. rn:5 rd:5  &qrrr_e q=1 esz=0
+@rrr_q1e3    ... rm:5 .. rn:5 rd:5  &qrrr_e q=1 esz=3
 
 ### Data Processing - Immediate
 
@@ -620,3 +621,13 @@ SHA256SU1   0101 1110 000 . 011000 . .  
@rrr_q1e0
 SHA1H   0101 1110 0010 1000  10 . . @rr_q1e0
 SHA1SU1 0101 1110 0010 1000 0001 10 . . @rr_q1e0
 SHA256SU0   0101 1110 0010 1000 0010 10 . . @rr_q1e0
+
+### Cryptographic three-register SHA512
+
+SHA512H 1100 1110 011 . 10 . .  @rrr_q1e0
+SHA512H21100 1110 011 . 11 . .  @rrr_q1e0
+SHA512SU1   1100 1110 011 . 100010 . .  @rrr_q1e0
+RAX11100 1110 011 . 100011 . .  @rrr_q1e3
+SM3PARTW1   1100 1110 011 . 11 . .  @rrr_q1e0
+SM3PARTW2   1100 1110 011 . 110001 . .  @rrr_q1e0
+SM4EKEY 1100 1110 011 . 110010 . .  @rrr_q1e0
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 1d20bf0c35..77b24cd52e 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -1341,6 +1341,17 @@ static bool do_gvec_op3_ool(DisasContext *s, arg_qrrr_e 
*a, int data,
 return true;
 }
 
+static bool do_gvec_fn3(DisasContext *s, arg_qrrr_e *a, GVecGen3Fn *fn)
+{
+if (!a->q && a->esz == MO_64) {
+return false;
+}
+if (fp_access_check(s)) {
+gen_gvec_fn3(s, a->q, a->rd, a->rn, a->rm, fn, a->esz);
+}
+return true;
+}
+
 /*
  * This utility function is for doing register extension with an
  * optional shift. You will likely want to pass a temporary for the
@@ -4589,7 +4600,7 @@ static bool trans_EXTR(DisasContext *s, arg_extract *a)
 }
 
 /*
- * Cryptographic AES, SHA
+ * Cryptographic AES, SHA, SHA512
  */
 
 TRANS_FEAT(AESE, aa64_aes, do_gvec_op3_ool, a, 0, gen_helper_crypto_aese)
@@ -4610,6 +4621,15 @@ TRANS_FEAT(SHA1H, aa64_sha1, do_gvec_op2_ool, a, 0, 
gen_helper_crypto_sha1h)
 TRANS_FEAT(SHA1SU1, aa64_sha1, do_gvec_op2_ool, a, 0, 
gen_helper_crypto_sha1su1)
 TRANS_FEAT(SHA256SU0, aa64_sha256, do_gvec_op2_ool, a, 0, 
gen_helper_crypto_sha256su0)
 
+TRANS_FEAT(SHA512H, aa64_sha512, do_gvec_op3_ool, a, 0, 
gen_helper_crypto_sha512h)
+TRANS_FEAT(SHA512H2, aa64_sha512, do_gvec_op3_ool, a, 0, 
gen_helper_crypto_sha512h2)
+TRANS_FEAT(SHA512SU1, aa64_sha512, do_gvec_op3_ool, a, 0, 
gen_helper_crypto_sha512su1)
+TRANS_FEAT(RAX1, aa64_sha3, do_gvec_fn3, a, gen_gvec_rax1)
+TRANS_FEAT(SM3PARTW1, aa64_sm3, do_gvec_op3_ool, a, 0, 
gen_helper_crypto_sm3partw1)
+TRANS_FEAT(SM3PARTW2, aa64_sm3, do_gvec_op3_ool, a, 0, 
gen_helper_crypto_sm3partw2)
+TRANS_FEAT(SM4EKEY, aa64_sm4, do_gvec_op3_ool, a, 0, gen_helper_crypto_sm4ekey)
+
+
 /* Shift a TCGv src by TCGv shift_amount, put result in dst.
  * Note that it is the caller's responsibility to ensure that the
  * shift amount is in range (ie 0..31 or 0..63) and provide the ARM
@@ -13510,80 +13530,6 @@ static void disas_simd_indexed(DisasContext *s, 
uint32_t insn)
 }
 }
 
-/* Crypto three-reg SHA512
- *  31   21 20  16 15  14  13 12  11  10  95 40
- * +---+--+---+---+-++--+--+
- * | 1 1 0 0 1 1 1 0 0 1 1 |  Rm  | 1 | O | 0 0 | opcode |  Rn  |  Rd  |
- * +---+--+---+---+-++--+--+
- */
-static void disas_crypto_three_reg_sha512(DisasContext *s, uint32_t insn)
-{
-int opcode = extract32(insn, 10, 2);
-int o =  extract32(insn, 14, 1);
-int rm = extract32(insn, 16, 5);
-int rn = extract32(insn, 5, 5);
-int rd = extract32(insn, 0, 5);
-bool feature;
-gen_helper_gvec_3 *oolfn = NULL;
-GVecGen3Fn *gvecfn = NULL;
-
-if (o == 0) {
-switch (opcode) {
-case 0: /* SHA512H */
-feature = dc_isar_feature(aa64_sha512, s);
-oolfn = gen_helper_crypto_sha512h;
-break;
-case 1: /* SHA512H2 */
-feature = dc_isar_feature(aa64_sha512, s);
-oolfn = gen_helper_crypto_sha512h2;
-break;
-case 2: /* SHA512SU1 */
-feature = dc_isar_feature(aa64_sha512, s);
-oolfn = gen_helper_crypto_sha512su1;
-break;
-case 3: /* RAX1 */
-feature = dc_isar_feature(aa64_sha3, s);
-gvecfn = gen_gvec_rax1;
-brea

[PATCH v2 67/67] target/arm: Convert FCSEL to decodetree

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |   4 ++
 target/arm/tcg/translate-a64.c | 108 ++---
 2 files changed, 49 insertions(+), 63 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 6f6cd805b7..5dadbc74d7 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -1000,6 +1000,10 @@ SQDMULH_vi  0.00  10 . . 1100 . 0 . 
.   @qrrx_s
 SQRDMULH_vi 0.00  01 ..  1101 . 0 . .   @qrrx_h
 SQRDMULH_vi 0.00  10 . . 1101 . 0 . .   @qrrx_s
 
+# Floating-point conditional select
+
+FCSEL   0001 1110 .. 1 rm:5 cond:4 11 rn:5 rd:5 esz=%esz_hsd
+
 # Floating-point data-processing (3 source)
 
 @_hsd     .. . rm:5  . ra:5  rn:5  rd:5 &_e 
esz=%esz_hsd
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 3c2963ebaa..845aaa2bfb 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5866,6 +5866,50 @@ static bool trans_ADDP_s(DisasContext *s, arg_rr_e *a)
 return true;
 }
 
+/*
+ * Floating-point conditional select
+ */
+
+static bool trans_FCSEL(DisasContext *s, arg_FCSEL *a)
+{
+TCGv_i64 t_true, t_false;
+DisasCompare64 c;
+
+switch (a->esz) {
+case MO_32:
+case MO_64:
+break;
+case MO_16:
+if (!dc_isar_feature(aa64_fp16, s)) {
+return false;
+}
+break;
+default:
+return false;
+}
+
+if (!fp_access_check(s)) {
+return true;
+}
+
+/* Zero extend sreg & hreg inputs to 64 bits now.  */
+t_true = tcg_temp_new_i64();
+t_false = tcg_temp_new_i64();
+read_vec_element(s, t_true, a->rn, 0, a->esz);
+read_vec_element(s, t_false, a->rm, 0, a->esz);
+
+a64_test_cc(&c, a->cond);
+tcg_gen_movcond_i64(c.cond, t_true, c.value, tcg_constant_i64(0),
+t_true, t_false);
+
+/*
+ * Note that sregs & hregs write back zeros to the high bits,
+ * and we've already done the zero-extension.
+ */
+write_fp_dreg(s, a->rd, t_true);
+return true;
+}
+
 /*
  * Floating-point data-processing (3 source)
  */
@@ -7332,68 +7376,6 @@ static void disas_fp_ccomp(DisasContext *s, uint32_t 
insn)
 }
 }
 
-/* Floating point conditional select
- *   31  30  29 28   24 23  22  21 20  16 15  12 11 10 95 40
- * +---+---+---+---+--+---+--+--+-+--+--+
- * | M | 0 | S | 1 1 1 1 0 | type | 1 |  Rm  | cond | 1 1 |  Rn  |  Rd  |
- * +---+---+---+---+--+---+--+--+-+--+--+
- */
-static void disas_fp_csel(DisasContext *s, uint32_t insn)
-{
-unsigned int mos, type, rm, cond, rn, rd;
-TCGv_i64 t_true, t_false;
-DisasCompare64 c;
-MemOp sz;
-
-mos = extract32(insn, 29, 3);
-type = extract32(insn, 22, 2);
-rm = extract32(insn, 16, 5);
-cond = extract32(insn, 12, 4);
-rn = extract32(insn, 5, 5);
-rd = extract32(insn, 0, 5);
-
-if (mos) {
-unallocated_encoding(s);
-return;
-}
-
-switch (type) {
-case 0:
-sz = MO_32;
-break;
-case 1:
-sz = MO_64;
-break;
-case 3:
-sz = MO_16;
-if (dc_isar_feature(aa64_fp16, s)) {
-break;
-}
-/* fallthru */
-default:
-unallocated_encoding(s);
-return;
-}
-
-if (!fp_access_check(s)) {
-return;
-}
-
-/* Zero extend sreg & hreg inputs to 64 bits now.  */
-t_true = tcg_temp_new_i64();
-t_false = tcg_temp_new_i64();
-read_vec_element(s, t_true, rn, 0, sz);
-read_vec_element(s, t_false, rm, 0, sz);
-
-a64_test_cc(&c, cond);
-tcg_gen_movcond_i64(c.cond, t_true, c.value, tcg_constant_i64(0),
-t_true, t_false);
-
-/* Note that sregs & hregs write back zeros to the high bits,
-   and we've already done the zero-extension.  */
-write_fp_dreg(s, rd, t_true);
-}
-
 /* Floating-point data-processing (1 source) - half precision */
 static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn)
 {
@@ -8205,7 +8187,7 @@ static void disas_data_proc_fp(DisasContext *s, uint32_t 
insn)
 break;
 case 3:
 /* Floating point conditional select */
-disas_fp_csel(s, insn);
+unallocated_encoding(s); /* in decodetree */
 break;
 case 0:
 switch (ctz32(extract32(insn, 12, 4))) {
-- 
2.34.1

[PATCH v2 20/67] target/arm: Convert FMAX, FMIN, FMAXNM, FMINNM to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/helper.h|   4 +
 target/arm/tcg/a64.decode  |  17 
 target/arm/tcg/translate-a64.c | 168 +
 target/arm/tcg/vec_helper.c|   4 +
 4 files changed, 113 insertions(+), 80 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index 2b02733305..7ee15b9651 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -748,15 +748,19 @@ DEF_HELPER_FLAGS_5(gvec_facgt_s, TCG_CALL_NO_RWG, void, 
ptr, ptr, ptr, ptr, i32)
 
 DEF_HELPER_FLAGS_5(gvec_fmax_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_5(gvec_fmax_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_fmax_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 
 DEF_HELPER_FLAGS_5(gvec_fmin_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_5(gvec_fmin_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_fmin_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 
 DEF_HELPER_FLAGS_5(gvec_fmaxnum_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 DEF_HELPER_FLAGS_5(gvec_fmaxnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(gvec_fmaxnum_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 
 DEF_HELPER_FLAGS_5(gvec_fminnum_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 DEF_HELPER_FLAGS_5(gvec_fminnum_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(gvec_fminnum_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 
 DEF_HELPER_FLAGS_5(gvec_recps_nf_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 DEF_HELPER_FLAGS_5(gvec_recps_nf_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 82daafbef5..e2678d919e 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -704,6 +704,11 @@ FSUB_s  0001 1110 ..1 . 0011 10 . . 
@rrr_hsd
 FDIV_s  0001 1110 ..1 . 0001 10 . . @rrr_hsd
 FMUL_s  0001 1110 ..1 .  10 . . @rrr_hsd
 
+FMAX_s  0001 1110 ..1 . 0100 10 . . @rrr_hsd
+FMIN_s  0001 1110 ..1 . 0101 10 . . @rrr_hsd
+FMAXNM_s0001 1110 ..1 . 0110 10 . . @rrr_hsd
+FMINNM_s0001 1110 ..1 . 0111 10 . . @rrr_hsd
+
 FMULX_s 0101 1110 010 . 00011 1 . . @rrr_h
 FMULX_s 0101 1110 0.1 . 11011 1 . . @rrr_sd
 
@@ -721,6 +726,18 @@ FDIV_v  0.10 1110 0.1 . 1 1 . . 
@qrrr_sd
 FMUL_v  0.10 1110 010 . 00011 1 . . @qrrr_h
 FMUL_v  0.10 1110 0.1 . 11011 1 . . @qrrr_sd
 
+FMAX_v  0.00 1110 010 . 00110 1 . . @qrrr_h
+FMAX_v  0.00 1110 0.1 . 0 1 . . @qrrr_sd
+
+FMIN_v  0.00 1110 110 . 00110 1 . . @qrrr_h
+FMIN_v  0.00 1110 1.1 . 0 1 . . @qrrr_sd
+
+FMAXNM_v0.00 1110 010 . 0 1 . . @qrrr_h
+FMAXNM_v0.00 1110 0.1 . 11000 1 . . @qrrr_sd
+
+FMINNM_v0.00 1110 110 . 0 1 . . @qrrr_h
+FMINNM_v0.00 1110 1.1 . 11000 1 . . @qrrr_sd
+
 FMULX_v 0.00 1110 010 . 00011 1 . . @qrrr_h
 FMULX_v 0.00 1110 0.1 . 11011 1 . . @qrrr_sd
 
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 97c3d758d6..6f8207d842 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -4915,6 +4915,34 @@ static const FPScalar f_scalar_fmul = {
 };
 TRANS(FMUL_s, do_fp3_scalar, a, &f_scalar_fmul)
 
+static const FPScalar f_scalar_fmax = {
+gen_helper_advsimd_maxh,
+gen_helper_vfp_maxs,
+gen_helper_vfp_maxd,
+};
+TRANS(FMAX_s, do_fp3_scalar, a, &f_scalar_fmax)
+
+static const FPScalar f_scalar_fmin = {
+gen_helper_advsimd_minh,
+gen_helper_vfp_mins,
+gen_helper_vfp_mind,
+};
+TRANS(FMIN_s, do_fp3_scalar, a, &f_scalar_fmin)
+
+static const FPScalar f_scalar_fmaxnm = {
+gen_helper_advsimd_maxnumh,
+gen_helper_vfp_maxnums,
+gen_helper_vfp_maxnumd,
+};
+TRANS(FMAXNM_s, do_fp3_scalar, a, &f_scalar_fmaxnm)
+
+static const FPScalar f_scalar_fminnm = {
+gen_helper_advsimd_minnumh,
+gen_helper_vfp_minnums,
+gen_helper_vfp_minnumd,
+};
+TRANS(FMINNM_s, do_fp3_scalar, a, &f_scalar_fminnm)
+
 static const FPScalar f_scalar_fmulx = {
 gen_helper_advsimd_mulxh,
 gen_helper_vfp_mulxs,
@@ -4978,6 +5006,34 @@ static gen_helper_gvec_3_ptr * const f_vector_fmul[3] = {
 };
 TRANS(FMUL_v, do_fp3_vector, a, f_vector_fmul)
 
+static gen_helper_gvec_3_ptr * const f_vector_fmax[3] = {
+gen_helper_gvec_fmax_h,
+gen_helper_gvec_fmax_s,
+gen_helper_gvec_fmax_d,
+};
+TRANS(FMAX_v, do_fp3_vector, a, f_vector_fmax)
+
+static gen_helper_gvec_3_ptr * const f_vector_fmin[3] = {
+gen_helper_gvec_fmin_h,
+gen_helper_gve

[PATCH v2 61/67] target/arm: Convert SABA, SABD, UABA, UABD to decodetree

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  4 
 target/arm/tcg/translate-a64.c | 22 ++
 2 files changed, 10 insertions(+), 16 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index bc98963bc5..07b604ec30 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -922,6 +922,10 @@ SMAX_v  0.00 1110 ..1 . 01100 1 . . 
@qrrr_e
 UMAX_v  0.10 1110 ..1 . 01100 1 . . @qrrr_e
 SMIN_v  0.00 1110 ..1 . 01101 1 . . @qrrr_e
 UMIN_v  0.10 1110 ..1 . 01101 1 . . @qrrr_e
+SABD_v  0.00 1110 ..1 . 01110 1 . . @qrrr_e
+UABD_v  0.10 1110 ..1 . 01110 1 . . @qrrr_e
+SABA_v  0.00 1110 ..1 . 0 1 . . @qrrr_e
+UABA_v  0.10 1110 ..1 . 0 1 . . @qrrr_e
 
 ### Advanced SIMD scalar x indexed element
 
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index db6f59df17..61afbc434f 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5464,6 +5464,10 @@ TRANS(SMAX_v, do_gvec_fn3_no64, a, tcg_gen_gvec_smax)
 TRANS(UMAX_v, do_gvec_fn3_no64, a, tcg_gen_gvec_umax)
 TRANS(SMIN_v, do_gvec_fn3_no64, a, tcg_gen_gvec_smin)
 TRANS(UMIN_v, do_gvec_fn3_no64, a, tcg_gen_gvec_umin)
+TRANS(SABA_v, do_gvec_fn3_no64, a, gen_gvec_saba)
+TRANS(UABA_v, do_gvec_fn3_no64, a, gen_gvec_uaba)
+TRANS(SABD_v, do_gvec_fn3_no64, a, gen_gvec_sabd)
+TRANS(UABD_v, do_gvec_fn3_no64, a, gen_gvec_uabd)
 
 static bool do_cmop_v(DisasContext *s, arg_qrrr_e *a, TCGCond cond)
 {
@@ -10929,8 +10933,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 return;
 }
 /* fall through */
-case 0xe: /* SABD, UABD */
-case 0xf: /* SABA, UABA */
 case 0x12: /* MLA, MLS */
 if (size == 3) {
 unallocated_encoding(s);
@@ -10963,6 +10965,8 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 case 0x0b: /* SQRSHL, UQRSHL */
 case 0x0c: /* SMAX, UMAX */
 case 0x0d: /* SMIN, UMIN */
+case 0x0e: /* SABD, UABD */
+case 0x0f: /* SABA, UABA */
 case 0x10: /* ADD, SUB */
 case 0x11: /* CMTST, CMEQ */
 unallocated_encoding(s);
@@ -10974,20 +10978,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 }
 
 switch (opcode) {
-case 0xe: /* SABD, UABD */
-if (u) {
-gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_uabd, size);
-} else {
-gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_sabd, size);
-}
-return;
-case 0xf: /* SABA, UABA */
-if (u) {
-gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_uaba, size);
-} else {
-gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_saba, size);
-}
-return;
 case 0x13: /* MUL, PMUL */
 if (!u) { /* MUL */
 gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_mul, size);
-- 
2.34.1

[PATCH v2 16/67] target/arm: Convert XAR to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  4 
 target/arm/tcg/translate-a64.c | 43 +++---
 2 files changed, 18 insertions(+), 29 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 1292312a7f..7f354af25d 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -654,3 +654,7 @@ SM3TT1A 11001110 010 . 10 .. 00 . . 
@crypto3i
 SM3TT1B 11001110 010 . 10 .. 01 . . @crypto3i
 SM3TT2A 11001110 010 . 10 .. 10 . . @crypto3i
 SM3TT2B 11001110 010 . 10 .. 11 . . @crypto3i
+
+### Cryptographic XAR
+
+XAR 1100 1110 100 rm:5 imm:6 rn:5 rd:5
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index cf3a7dfa99..75f1e6a7b9 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -4688,6 +4688,20 @@ TRANS_FEAT(SM3TT1B, aa64_sm3, do_crypto3i, a, 
gen_helper_crypto_sm3tt1b)
 TRANS_FEAT(SM3TT2A, aa64_sm3, do_crypto3i, a, gen_helper_crypto_sm3tt2a)
 TRANS_FEAT(SM3TT2B, aa64_sm3, do_crypto3i, a, gen_helper_crypto_sm3tt2b)
 
+static bool trans_XAR(DisasContext *s, arg_XAR *a)
+{
+if (!dc_isar_feature(aa64_sha3, s)) {
+return false;
+}
+if (fp_access_check(s)) {
+gen_gvec_xar(MO_64, vec_full_reg_offset(s, a->rd),
+ vec_full_reg_offset(s, a->rn),
+ vec_full_reg_offset(s, a->rm), a->imm, 16,
+ vec_full_reg_size(s));
+}
+return true;
+}
+
 /* Shift a TCGv src by TCGv shift_amount, put result in dst.
  * Note that it is the caller's responsibility to ensure that the
  * shift amount is in range (ie 0..31 or 0..63) and provide the ARM
@@ -13588,34 +13602,6 @@ static void disas_simd_indexed(DisasContext *s, 
uint32_t insn)
 }
 }
 
-/* Crypto XAR
- *  31   21 20  16 1510 95 40
- * +---+--++--+--+
- * | 1 1 0 0 1 1 1 0 1 0 0 |  Rm  |  imm6  |  Rn  |  Rd  |
- * +---+--++--+--+
- */
-static void disas_crypto_xar(DisasContext *s, uint32_t insn)
-{
-int rm = extract32(insn, 16, 5);
-int imm6 = extract32(insn, 10, 6);
-int rn = extract32(insn, 5, 5);
-int rd = extract32(insn, 0, 5);
-
-if (!dc_isar_feature(aa64_sha3, s)) {
-unallocated_encoding(s);
-return;
-}
-
-if (!fp_access_check(s)) {
-return;
-}
-
-gen_gvec_xar(MO_64, vec_full_reg_offset(s, rd),
- vec_full_reg_offset(s, rn),
- vec_full_reg_offset(s, rm), imm6, 16,
- vec_full_reg_size(s));
-}
-
 /* C3.6 Data processing - SIMD, inc Crypto
  *
  * As the decode gets a little complex we are using a table based
@@ -13644,7 +13630,6 @@ static const AArch64DecodeTable data_proc_simd[] = {
 { 0x5e000400, 0xdfe08400, disas_simd_scalar_copy },
 { 0x5f00, 0xdf000400, disas_simd_indexed }, /* scalar indexed */
 { 0x5f000400, 0xdf800400, disas_simd_scalar_shift_imm },
-{ 0xce80, 0xffe0, disas_crypto_xar },
 { 0x0e400400, 0x9f60c400, disas_simd_three_reg_same_fp16 },
 { 0x0e780800, 0x8f7e0c00, disas_simd_two_reg_misc_fp16 },
 { 0x5e400400, 0xdf60c400, disas_simd_scalar_three_reg_same_fp16 },
-- 
2.34.1

[PATCH v2 65/67] target/arm: Convert SQDMULH, SQRDMULH to decodetree

[Richard Henderson]

These are the last instructions within disas_simd_three_reg_same
and disas_simd_scalar_three_reg_same, so remove them.

Signed-off-by: Richard Henderson 
---
 target/arm/helper.h|  10 ++
 target/arm/tcg/a64.decode  |  18 +++
 target/arm/tcg/translate-a64.c | 276 ++---
 target/arm/tcg/vec_helper.c|  64 
 4 files changed, 172 insertions(+), 196 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index 85f9302563..24feecee9b 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -968,6 +968,16 @@ DEF_HELPER_FLAGS_5(neon_sqrdmulh_h, TCG_CALL_NO_RWG,
 DEF_HELPER_FLAGS_5(neon_sqrdmulh_s, TCG_CALL_NO_RWG,
void, ptr, ptr, ptr, ptr, i32)
 
+DEF_HELPER_FLAGS_5(neon_sqdmulh_idx_h, TCG_CALL_NO_RWG,
+   void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(neon_sqdmulh_idx_s, TCG_CALL_NO_RWG,
+   void, ptr, ptr, ptr, ptr, i32)
+
+DEF_HELPER_FLAGS_5(neon_sqrdmulh_idx_h, TCG_CALL_NO_RWG,
+   void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(neon_sqrdmulh_idx_s, TCG_CALL_NO_RWG,
+   void, ptr, ptr, ptr, ptr, i32)
+
 DEF_HELPER_FLAGS_4(sve2_sqdmulh_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_4(sve2_sqdmulh_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_4(sve2_sqdmulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 2dea68a0a9..f7f897f9fc 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -774,6 +774,9 @@ CMHS_s  0111 1110 111 . 00111 1 . . 
@rrr_d
 CMTST_s 0101 1110 111 . 10001 1 . . @rrr_d
 CMEQ_s  0111 1110 111 . 10001 1 . . @rrr_d
 
+SQDMULH_s   0101 1110 ..1 . 10110 1 . . @rrr_e
+SQRDMULH_s  0111 1110 ..1 . 10110 1 . . @rrr_e
+
 ### Advanced SIMD scalar pairwise
 
 FADDP_s 0101 1110 0011  1101 10 . . @rr_h
@@ -931,6 +934,9 @@ PMUL_v  0.10 1110 001 . 10011 1 . . 
@qrrr_b
 MLA_v   0.00 1110 ..1 . 10010 1 . . @qrrr_e
 MLS_v   0.10 1110 ..1 . 10010 1 . . @qrrr_e
 
+SQDMULH_v   0.00 1110 ..1 . 10110 1 . . @qrrr_e
+SQRDMULH_v  0.10 1110 ..1 . 10110 1 . . @qrrr_e
+
 ### Advanced SIMD scalar x indexed element
 
 FMUL_si 0101  00 ..  1001 . 0 . .   @rrx_h
@@ -949,6 +955,12 @@ FMULX_si0111  00 ..  1001 . 0 . .  
 @rrx_h
 FMULX_si0111  10 . . 1001 . 0 . .   @rrx_s
 FMULX_si0111  11 0 . 1001 . 0 . .   @rrx_d
 
+SQDMULH_si  0101  01 ..  1100 . 0 . .   @rrx_h
+SQDMULH_si  0101  10 ..  1100 . 0 . .   @rrx_s
+
+SQRDMULH_si 0101  01 ..  1101 . 0 . .   @rrx_h
+SQRDMULH_si 0101  10 . . 1101 . 0 . .   @rrx_s
+
 ### Advanced SIMD vector x indexed element
 
 FMUL_vi 0.00  00 ..  1001 . 0 . .   @qrrx_h
@@ -980,3 +992,9 @@ MLA_vi  0.10  10 . .  . 0 . .   
@qrrx_s
 
 MLS_vi  0.10  01 ..  0100 . 0 . .   @qrrx_h
 MLS_vi  0.10  10 . . 0100 . 0 . .   @qrrx_s
+
+SQDMULH_vi  0.00  01 ..  1100 . 0 . .   @qrrx_h
+SQDMULH_vi  0.00  10 . . 1100 . 0 . .   @qrrx_s
+
+SQRDMULH_vi 0.00  01 ..  1101 . 0 . .   @qrrx_h
+SQRDMULH_vi 0.00  10 . . 1101 . 0 . .   @qrrx_s
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index c673b95ec7..14226c56cf 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -1350,6 +1350,14 @@ static bool do_gvec_fn3_no64(DisasContext *s, arg_qrrr_e 
*a, GVecGen3Fn *fn)
 return true;
 }
 
+static bool do_gvec_fn3_no8_no64(DisasContext *s, arg_qrrr_e *a, GVecGen3Fn 
*fn)
+{
+if (a->esz == MO_8) {
+return false;
+}
+return do_gvec_fn3_no64(s, a, fn);
+}
+
 static bool do_gvec_fn4(DisasContext *s, arg_q_e *a, GVecGen4Fn *fn)
 {
 if (!a->q && a->esz == MO_64) {
@@ -5167,6 +5175,25 @@ static const ENVScalar2 f_scalar_uqrshl = {
 };
 TRANS(UQRSHL_s, do_env_scalar2, a, &f_scalar_uqrshl)
 
+static bool do_env_scalar2_hs(DisasContext *s, arg_rrr_e *a,
+  const ENVScalar2 *f)
+{
+if (a->esz == MO_16 || a->esz == MO_32) {
+return do_env_scalar2(s, a, f);
+}
+return false;
+}
+
+static const ENVScalar2 f_scalar_sqdmulh = {
+{ NULL, gen_helper_neon_qdmulh_s16, gen_helper_neon_qdmulh_s32 }
+};
+TRANS(SQDMULH_s, do_env_scalar2_hs, a, &f_scalar_sqdmulh)
+
+static const ENVScalar2 f_scalar_sqrdmulh = {
+{ NULL, gen_helper_neon_qrdmulh_s16, gen_helper_neon_qrdmulh_s32 }
+};
+TRANS(SQRDMULH_s, do_env_scalar2_hs, a, &f_scalar_sqrdmulh)
+
 static bool do_cmop_d(DisasContext *s, a

[PATCH v2 59/67] target/arm: Convert SRHADD, URHADD to decodetree

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  2 ++
 target/arm/tcg/translate-a64.c | 11 +++
 2 files changed, 5 insertions(+), 8 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index b1bbcb144e..1c448b4f7c 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -916,6 +916,8 @@ SHADD_v 0.00 1110 ..1 . 0 1 . . 
@qrrr_e
 UHADD_v 0.10 1110 ..1 . 0 1 . . @qrrr_e
 SHSUB_v 0.00 1110 ..1 . 00100 1 . . @qrrr_e
 UHSUB_v 0.10 1110 ..1 . 00100 1 . . @qrrr_e
+SRHADD_v0.00 1110 ..1 . 00010 1 . . @qrrr_e
+URHADD_v0.10 1110 ..1 . 00010 1 . . @qrrr_e
 
 ### Advanced SIMD scalar x indexed element
 
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 40aa7a9d57..9ef5de6755 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5458,6 +5458,8 @@ TRANS(SHADD_v, do_gvec_fn3_no64, a, gen_gvec_shadd)
 TRANS(UHADD_v, do_gvec_fn3_no64, a, gen_gvec_uhadd)
 TRANS(SHSUB_v, do_gvec_fn3_no64, a, gen_gvec_shsub)
 TRANS(UHSUB_v, do_gvec_fn3_no64, a, gen_gvec_uhsub)
+TRANS(SRHADD_v, do_gvec_fn3_no64, a, gen_gvec_srhadd)
+TRANS(URHADD_v, do_gvec_fn3_no64, a, gen_gvec_urhadd)
 
 static bool do_cmop_v(DisasContext *s, arg_qrrr_e *a, TCGCond cond)
 {
@@ -10923,7 +10925,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 return;
 }
 /* fall through */
-case 0x2: /* SRHADD, URHADD */
 case 0xc: /* SMAX, UMAX */
 case 0xd: /* SMIN, UMIN */
 case 0xe: /* SABD, UABD */
@@ -10949,6 +10950,7 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 
 case 0x0: /* SHADD, UHADD */
 case 0x01: /* SQADD, UQADD */
+case 0x02: /* SRHADD, URHADD */
 case 0x04: /* SHSUB, UHSUB */
 case 0x05: /* SQSUB, UQSUB */
 case 0x06: /* CMGT, CMHI */
@@ -10968,13 +10970,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 }
 
 switch (opcode) {
-case 0x02: /* SRHADD, URHADD */
-if (u) {
-gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_urhadd, size);
-} else {
-gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_srhadd, size);
-}
-return;
 case 0x0c: /* SMAX, UMAX */
 if (u) {
 gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_umax, size);
-- 
2.34.1

[PATCH v2 43/67] target/arm: Convert SSHL, USHL to decodetree

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  7 ++
 target/arm/tcg/translate-a64.c | 40 +-
 2 files changed, 32 insertions(+), 15 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 7c350ba833..ea897d6732 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -42,6 +42,7 @@
 @rr_sd   ... . .. rn:5 rd:5 &rr_e esz=%esz_sd
 
 @rrr_h   ... rm:5 .. rn:5 rd:5  &rrr_e esz=1
+@rrr_d   ... rm:5 .. rn:5 rd:5  &rrr_e esz=3
 @rrr_sd  ... rm:5 .. rn:5 rd:5  &rrr_e esz=%esz_sd
 @rrr_hsd ... rm:5 .. rn:5 rd:5  &rrr_e esz=%esz_hsd
 @rrr_e   esz:2 . rm:5 .. rn:5 rd:5  &rrr_e
@@ -755,6 +756,9 @@ UQSUB_s 0111 1110 ..1 . 00101 1 . . 
@rrr_e
 SUQADD_s0101 1110 ..1 0 00111 0 . . @r2r_e
 USQADD_s0111 1110 ..1 0 00111 0 . . @r2r_e
 
+SSHL_s  0101 1110 111 . 01000 1 . . @rrr_d
+USHL_s  0111 1110 111 . 01000 1 . . @rrr_d
+
 ### Advanced SIMD scalar pairwise
 
 FADDP_s 0101 1110 0011  1101 10 . . @rr_h
@@ -876,6 +880,9 @@ UQSUB_v 0.10 1110 ..1 . 00101 1 . . 
@qrrr_e
 SUQADD_v0.00 1110 ..1 0 00111 0 . . @qr2r_e
 USQADD_v0.10 1110 ..1 0 00111 0 . . @qr2r_e
 
+SSHL_v  0.00 1110 ..1 . 01000 1 . . @qrrr_e
+USHL_v  0.10 1110 ..1 . 01000 1 . . @qrrr_e
+
 ### Advanced SIMD scalar x indexed element
 
 FMUL_si 0101  00 ..  1001 . 0 . .   @rrx_h
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index fbcf18f92a..8d39a9663e 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5099,6 +5099,24 @@ TRANS(UQSUB_s, do_satacc_s, a, 0, 0, gen_uqsub_bhs, 
gen_uqsub_d)
 TRANS(SUQADD_s, do_satacc_s, a, MO_SIGN, 0, gen_suqadd_bhs, gen_suqadd_d)
 TRANS(USQADD_s, do_satacc_s, a, 0, MO_SIGN, gen_usqadd_bhs, gen_usqadd_d)
 
+static bool do_int3_scalar_d(DisasContext *s, arg_rrr_e *a,
+ void (*fn)(TCGv_i64, TCGv_i64, TCGv_i64))
+{
+if (fp_access_check(s)) {
+TCGv_i64 t0 = tcg_temp_new_i64();
+TCGv_i64 t1 = tcg_temp_new_i64();
+
+read_vec_element(s, t0, a->rn, 0, MO_64);
+read_vec_element(s, t1, a->rm, 0, MO_64);
+fn(t0, t0, t1);
+write_fp_dreg(s, a->rd, t0);
+}
+return true;
+}
+
+TRANS(SSHL_s, do_int3_scalar_d, a, gen_sshl_i64)
+TRANS(USHL_s, do_int3_scalar_d, a, gen_ushl_i64)
+
 static bool do_fp3_vector(DisasContext *s, arg_qrrr_e *a,
   gen_helper_gvec_3_ptr * const fns[3])
 {
@@ -5344,6 +5362,10 @@ TRANS(UQSUB_v, do_gvec_fn3, a, gen_gvec_uqsub_qc)
 TRANS(SUQADD_v, do_gvec_fn3, a, gen_gvec_suqadd_qc)
 TRANS(USQADD_v, do_gvec_fn3, a, gen_gvec_usqadd_qc)
 
+TRANS(SSHL_v, do_gvec_fn3, a, gen_gvec_sshl)
+TRANS(USHL_v, do_gvec_fn3, a, gen_gvec_ushl)
+
+
 /*
  * Advanced SIMD scalar/vector x indexed element
  */
@@ -9355,13 +9377,6 @@ static void handle_3same_64(DisasContext *s, int opcode, 
bool u,
 }
 gen_cmtst_i64(tcg_rd, tcg_rn, tcg_rm);
 break;
-case 0x8: /* SSHL, USHL */
-if (u) {
-gen_ushl_i64(tcg_rd, tcg_rn, tcg_rm);
-} else {
-gen_sshl_i64(tcg_rd, tcg_rn, tcg_rm);
-}
-break;
 case 0x9: /* SQSHL, UQSHL */
 if (u) {
 gen_helper_neon_qshl_u64(tcg_rd, tcg_env, tcg_rn, tcg_rm);
@@ -9393,6 +9408,7 @@ static void handle_3same_64(DisasContext *s, int opcode, 
bool u,
 default:
 case 0x1: /* SQADD / UQADD */
 case 0x5: /* SQSUB / UQSUB */
+case 0x8: /* SSHL, USHL */
 g_assert_not_reached();
 }
 }
@@ -9417,7 +9433,6 @@ static void disas_simd_scalar_three_reg_same(DisasContext 
*s, uint32_t insn)
 case 0x9: /* SQSHL, UQSHL */
 case 0xb: /* SQRSHL, UQRSHL */
 break;
-case 0x8: /* SSHL, USHL */
 case 0xa: /* SRSHL, URSHL */
 case 0x6: /* CMGT, CMHI */
 case 0x7: /* CMGE, CMHS */
@@ -9437,6 +9452,7 @@ static void disas_simd_scalar_three_reg_same(DisasContext 
*s, uint32_t insn)
 default:
 case 0x1: /* SQADD, UQADD */
 case 0x5: /* SQSUB, UQSUB */
+case 0x8: /* SSHL, USHL */
 unallocated_encoding(s);
 return;
 }
@@ -10921,13 +10937,6 @@ static void disas_simd_3same_int(DisasContext *s, 
uint32_t insn)
 }
 
 switch (opcode) {
-case 0x08: /* SSHL, USHL */
-if (u) {
-gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_ushl, size);
-} else {
-gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_sshl, size);
-}
-return;
 case 0x0c: /* SMAX, UMAX */
 if (u) {
 gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_umax, size);
@@ -11008,6 +11017,7 

[PATCH v2 44/67] target/arm: Convert SRSHL and URSHL (register) to gvec

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 target/arm/helper.h | 10 +
 target/arm/tcg/translate.h  |  4 
 target/arm/tcg/neon-dp.decode   | 10 ++---
 target/arm/tcg/gengvec.c| 22 +++
 target/arm/tcg/neon_helper.c| 38 -
 target/arm/tcg/translate-a64.c  | 17 ++-
 target/arm/tcg/translate-neon.c |  6 ++
 7 files changed, 84 insertions(+), 23 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index a14c040451..25eb7bf5df 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -327,6 +327,16 @@ DEF_HELPER_3(neon_qrshl_s32, i32, env, i32, i32)
 DEF_HELPER_3(neon_qrshl_u64, i64, env, i64, i64)
 DEF_HELPER_3(neon_qrshl_s64, i64, env, i64, i64)
 
+DEF_HELPER_FLAGS_4(gvec_srshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_srshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_srshl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_srshl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+
+DEF_HELPER_FLAGS_4(gvec_urshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_urshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_urshl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_urshl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+
 DEF_HELPER_2(neon_add_u8, i32, i32, i32)
 DEF_HELPER_2(neon_add_u16, i32, i32, i32)
 DEF_HELPER_2(neon_sub_u8, i32, i32, i32)
diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h
index 87439dcc61..ea63ffc47b 100644
--- a/target/arm/tcg/translate.h
+++ b/target/arm/tcg/translate.h
@@ -459,6 +459,10 @@ void gen_gvec_sshl(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
 void gen_gvec_ushl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_srshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_urshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
 
 void gen_cmtst_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b);
 void gen_ushl_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b);
diff --git a/target/arm/tcg/neon-dp.decode b/target/arm/tcg/neon-dp.decode
index fd3a01bfa0..8525c65c0d 100644
--- a/target/arm/tcg/neon-dp.decode
+++ b/target/arm/tcg/neon-dp.decode
@@ -117,14 +117,8 @@ VSHL_U_3s 001 1 0 . ..   0100 . . . 0 
 @3same_rev
   VQSHL_U64_3s    001 1 0 . ..   0100 . . . 1  @3same_64_rev
   VQSHL_U_3s  001 1 0 . ..   0100 . . . 1  @3same_rev
 }
-{
-  VRSHL_S64_3s    001 0 0 . ..   0101 . . . 0  @3same_64_rev
-  VRSHL_S_3s  001 0 0 . ..   0101 . . . 0  @3same_rev
-}
-{
-  VRSHL_U64_3s    001 1 0 . ..   0101 . . . 0  @3same_64_rev
-  VRSHL_U_3s  001 1 0 . ..   0101 . . . 0  @3same_rev
-}
+VRSHL_S_3s    001 0 0 . ..   0101 . . . 0  @3same_rev
+VRSHL_U_3s    001 1 0 . ..   0101 . . . 0  @3same_rev
 {
   VQRSHL_S64_3s   001 0 0 . ..   0101 . . . 1  @3same_64_rev
   VQRSHL_S_3s 001 0 0 . ..   0101 . . . 1  @3same_rev
diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c
index 66a514ba86..d9a9132722 100644
--- a/target/arm/tcg/gengvec.c
+++ b/target/arm/tcg/gengvec.c
@@ -1217,6 +1217,28 @@ void gen_gvec_sshl(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
 }
 
+void gen_gvec_srshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+static gen_helper_gvec_3 * const fns[] = {
+gen_helper_gvec_srshl_b, gen_helper_gvec_srshl_h,
+gen_helper_gvec_srshl_s, gen_helper_gvec_srshl_d,
+};
+tcg_debug_assert(vece <= MO_64);
+tcg_gen_gvec_3_ool(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, 0, fns[vece]);
+}
+
+void gen_gvec_urshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+static gen_helper_gvec_3 * const fns[] = {
+gen_helper_gvec_urshl_b, gen_helper_gvec_urshl_h,
+gen_helper_gvec_urshl_s, gen_helper_gvec_urshl_d,
+};
+tcg_debug_assert(vece <= MO_64);
+tcg_gen_gvec_3_ool(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, 0, fns[vece]);
+}
+
 void gen_uqadd_bhs(TCGv_i64 res, TCGv_i64 qc, TCGv_i64 a, TCGv_i64 b, MemOp 
esz)
 {
 uint64_t max = MAKE_64BIT_MASK(0, 8 << esz);
diff --git a/target/arm/tcg/neon_helper.c b/target/arm/tcg/neon_helper.c
index 0af15e9f6e..516ecc1dcb 100644
--- a/target/arm/tcg/neon_helper.c
+++ b/target/arm/tcg/neon_helper.c
@@ -6,10 +6,11

[PATCH v2 17/67] target/arm: Convert Advanced SIMD copy to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  13 +
 target/arm/tcg/translate-a64.c | 426 +++--
 2 files changed, 152 insertions(+), 287 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 7f354af25d..d5bfeae7a8 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -658,3 +658,16 @@ SM3TT2B 11001110 010 . 10 .. 11 . . 
@crypto3i
 ### Cryptographic XAR
 
 XAR 1100 1110 100 rm:5 imm:6 rn:5 rd:5
+
+### Advanced SIMD scalar copy
+
+DUP_element_s   0101 1110 000 imm:5 0  1 rn:5 rd:5
+
+### Advanced SIMD copy
+
+DUP_element_v   0 q:1 00 1110 000 imm:5 0  1 rn:5 rd:5
+DUP_general 0 q:1 00 1110 000 imm:5 0 0001 1 rn:5 rd:5
+INS_general 0 1   00 1110 000 imm:5 0 0011 1 rn:5 rd:5
+SMOV0 q:1 00 1110 000 imm:5 0 0101 1 rn:5 rd:5
+UMOV0 q:1 00 1110 000 imm:5 0 0111 1 rn:5 rd:5
+INS_element 0 1   10 1110 000 di:5  0 si:4 1 rn:5 rd:5
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 75f1e6a7b9..1a12bf22fd 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -4702,6 +4702,145 @@ static bool trans_XAR(DisasContext *s, arg_XAR *a)
 return true;
 }
 
+/*
+ * Advanced SIMD copy
+ */
+
+static bool decode_esz_idx(int imm, MemOp *pesz, unsigned *pidx)
+{
+unsigned esz = ctz32(imm);
+if (esz <= MO_64) {
+*pesz = esz;
+*pidx = imm >> (esz + 1);
+return true;
+}
+return false;
+}
+
+static bool trans_DUP_element_s(DisasContext *s, arg_DUP_element_s *a)
+{
+MemOp esz;
+unsigned idx;
+
+if (!decode_esz_idx(a->imm, &esz, &idx)) {
+return false;
+}
+if (fp_access_check(s)) {
+/*
+ * This instruction just extracts the specified element and
+ * zero-extends it into the bottom of the destination register.
+ */
+TCGv_i64 tmp = tcg_temp_new_i64();
+read_vec_element(s, tmp, a->rn, idx, esz);
+write_fp_dreg(s, a->rd, tmp);
+}
+return true;
+}
+
+static bool trans_DUP_element_v(DisasContext *s, arg_DUP_element_v *a)
+{
+MemOp esz;
+unsigned idx;
+
+if (!decode_esz_idx(a->imm, &esz, &idx)) {
+return false;
+}
+if (esz == MO_64 && !a->q) {
+return false;
+}
+if (fp_access_check(s)) {
+tcg_gen_gvec_dup_mem(esz, vec_full_reg_offset(s, a->rd),
+ vec_reg_offset(s, a->rn, idx, esz),
+ a->q ? 16 : 8, vec_full_reg_size(s));
+}
+return true;
+}
+
+static bool trans_DUP_general(DisasContext *s, arg_DUP_general *a)
+{
+MemOp esz;
+unsigned idx;
+
+if (!decode_esz_idx(a->imm, &esz, &idx)) {
+return false;
+}
+if (esz == MO_64 && !a->q) {
+return false;
+}
+if (fp_access_check(s)) {
+tcg_gen_gvec_dup_i64(esz, vec_full_reg_offset(s, a->rd),
+ a->q ? 16 : 8, vec_full_reg_size(s),
+ cpu_reg(s, a->rn));
+}
+return true;
+}
+
+static bool do_smov_umov(DisasContext *s, arg_SMOV *a, MemOp is_signed)
+{
+MemOp esz;
+unsigned idx;
+
+if (!decode_esz_idx(a->imm, &esz, &idx)) {
+return false;
+}
+if (is_signed) {
+if (esz == MO_64 || (esz == MO_32 && !a->q)) {
+return false;
+}
+} else {
+if (esz == MO_64 ? !a->q : a->q) {
+return false;
+}
+}
+if (fp_access_check(s)) {
+TCGv_i64 tcg_rd = cpu_reg(s, a->rd);
+read_vec_element(s, tcg_rd, a->rn, idx, esz | is_signed);
+if (is_signed && !a->q) {
+tcg_gen_ext32u_i64(tcg_rd, tcg_rd);
+}
+}
+return true;
+}
+
+TRANS(SMOV, do_smov_umov, a, MO_SIGN)
+TRANS(UMOV, do_smov_umov, a, 0)
+
+static bool trans_INS_general(DisasContext *s, arg_INS_general *a)
+{
+MemOp esz;
+unsigned idx;
+
+if (!decode_esz_idx(a->imm, &esz, &idx)) {
+return false;
+}
+if (fp_access_check(s)) {
+write_vec_element(s, cpu_reg(s, a->rn), a->rd, idx, esz);
+clear_vec_high(s, true, a->rd);
+}
+return true;
+}
+
+static bool trans_INS_element(DisasContext *s, arg_INS_element *a)
+{
+MemOp esz;
+unsigned didx, sidx;
+
+if (!decode_esz_idx(a->di, &esz, &didx)) {
+return false;
+}
+sidx = a->si >> esz;
+if (fp_access_check(s)) {
+TCGv_i64 tmp = tcg_temp_new_i64();
+
+read_vec_element(s, tmp, a->rn, sidx, esz);
+write_vec_element(s, tmp, a->rd, didx, esz);
+
+/* INS is considered a 128-bit write for SVE. */
+clear_vec_high(s, true, a->rd);
+}
+return true;
+}
+
 /* Shift a TCGv src by TCGv shift_amount, put result in dst.
  * Note that it is the caller's responsibility to ensure that the
  * shift amount is in range (ie 0..31 or 0..63) and provide the AR

[PATCH v2 15/67] target/arm: Convert Cryptographic 3-register, imm2 to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  | 10 
 target/arm/tcg/translate-a64.c | 43 ++
 2 files changed, 22 insertions(+), 31 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index ef6902e86a..1292312a7f 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -644,3 +644,13 @@ SM4E1100 1110 110 0 11 . .  
@r2r_q1e0
 EOR31100 1110 000 . 0 . . . @_q1e3
 BCAX1100 1110 001 . 0 . . . @_q1e3
 SM3SS1  1100 1110 010 . 0 . . . @_q1e3
+
+### Cryptographic three-register, imm2
+
+&crypto3i   rd rn rm imm
+@crypto3i    ... rm:5 .. imm:2 .. rn:5 rd:5 &crypto3i
+
+SM3TT1A 11001110 010 . 10 .. 00 . . @crypto3i
+SM3TT1B 11001110 010 . 10 .. 01 . . @crypto3i
+SM3TT2A 11001110 010 . 10 .. 10 . . @crypto3i
+SM3TT2B 11001110 010 . 10 .. 11 . . @crypto3i
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 2951e7eb59..cf3a7dfa99 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -4676,6 +4676,18 @@ static bool trans_SM3SS1(DisasContext *s, arg_SM3SS1 *a)
 return true;
 }
 
+static bool do_crypto3i(DisasContext *s, arg_crypto3i *a, gen_helper_gvec_3 
*fn)
+{
+if (fp_access_check(s)) {
+gen_gvec_op3_ool(s, true, a->rd, a->rn, a->rm, a->imm, fn);
+}
+return true;
+}
+TRANS_FEAT(SM3TT1A, aa64_sm3, do_crypto3i, a, gen_helper_crypto_sm3tt1a)
+TRANS_FEAT(SM3TT1B, aa64_sm3, do_crypto3i, a, gen_helper_crypto_sm3tt1b)
+TRANS_FEAT(SM3TT2A, aa64_sm3, do_crypto3i, a, gen_helper_crypto_sm3tt2a)
+TRANS_FEAT(SM3TT2B, aa64_sm3, do_crypto3i, a, gen_helper_crypto_sm3tt2b)
+
 /* Shift a TCGv src by TCGv shift_amount, put result in dst.
  * Note that it is the caller's responsibility to ensure that the
  * shift amount is in range (ie 0..31 or 0..63) and provide the ARM
@@ -13604,36 +13616,6 @@ static void disas_crypto_xar(DisasContext *s, uint32_t 
insn)
  vec_full_reg_size(s));
 }
 
-/* Crypto three-reg imm2
- *  31   21 20  16 15  14 13 12  11  10  95 40
- * +---+--+-+--++--+--+
- * | 1 1 0 0 1 1 1 0 0 1 0 |  Rm  | 1 0 | imm2 | opcode |  Rn  |  Rd  |
- * +---+--+-+--++--+--+
- */
-static void disas_crypto_three_reg_imm2(DisasContext *s, uint32_t insn)
-{
-static gen_helper_gvec_3 * const fns[4] = {
-gen_helper_crypto_sm3tt1a, gen_helper_crypto_sm3tt1b,
-gen_helper_crypto_sm3tt2a, gen_helper_crypto_sm3tt2b,
-};
-int opcode = extract32(insn, 10, 2);
-int imm2 = extract32(insn, 12, 2);
-int rm = extract32(insn, 16, 5);
-int rn = extract32(insn, 5, 5);
-int rd = extract32(insn, 0, 5);
-
-if (!dc_isar_feature(aa64_sm3, s)) {
-unallocated_encoding(s);
-return;
-}
-
-if (!fp_access_check(s)) {
-return;
-}
-
-gen_gvec_op3_ool(s, true, rd, rn, rm, imm2, fns[opcode]);
-}
-
 /* C3.6 Data processing - SIMD, inc Crypto
  *
  * As the decode gets a little complex we are using a table based
@@ -13663,7 +13645,6 @@ static const AArch64DecodeTable data_proc_simd[] = {
 { 0x5f00, 0xdf000400, disas_simd_indexed }, /* scalar indexed */
 { 0x5f000400, 0xdf800400, disas_simd_scalar_shift_imm },
 { 0xce80, 0xffe0, disas_crypto_xar },
-{ 0xce408000, 0xffe0c000, disas_crypto_three_reg_imm2 },
 { 0x0e400400, 0x9f60c400, disas_simd_three_reg_same_fp16 },
 { 0x0e780800, 0x8f7e0c00, disas_simd_two_reg_misc_fp16 },
 { 0x5e400400, 0xdf60c400, disas_simd_scalar_three_reg_same_fp16 },
-- 
2.34.1

[PATCH v2 22/67] target/arm: Expand vfp neg and abs inline

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/helper.h|  6 
 target/arm/tcg/translate.h | 30 +++
 target/arm/tcg/translate-a64.c | 44 +--
 target/arm/tcg/translate-vfp.c | 54 +-
 target/arm/vfp_helper.c| 30 ---
 5 files changed, 79 insertions(+), 85 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index 7ee15b9651..0fd01c9c52 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -132,12 +132,6 @@ DEF_HELPER_3(vfp_maxnumd, f64, f64, f64, ptr)
 DEF_HELPER_3(vfp_minnumh, f16, f16, f16, ptr)
 DEF_HELPER_3(vfp_minnums, f32, f32, f32, ptr)
 DEF_HELPER_3(vfp_minnumd, f64, f64, f64, ptr)
-DEF_HELPER_1(vfp_negh, f16, f16)
-DEF_HELPER_1(vfp_negs, f32, f32)
-DEF_HELPER_1(vfp_negd, f64, f64)
-DEF_HELPER_1(vfp_absh, f16, f16)
-DEF_HELPER_1(vfp_abss, f32, f32)
-DEF_HELPER_1(vfp_absd, f64, f64)
 DEF_HELPER_2(vfp_sqrth, f16, f16, env)
 DEF_HELPER_2(vfp_sqrts, f32, f32, env)
 DEF_HELPER_2(vfp_sqrtd, f64, f64, env)
diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h
index ecfa242eef..b05a9eb668 100644
--- a/target/arm/tcg/translate.h
+++ b/target/arm/tcg/translate.h
@@ -406,6 +406,36 @@ static inline void gen_swstep_exception(DisasContext *s, 
int isv, int ex)
  */
 uint64_t vfp_expand_imm(int size, uint8_t imm8);
 
+static inline void gen_vfp_absh(TCGv_i32 d, TCGv_i32 s)
+{
+tcg_gen_andi_i32(d, s, INT16_MAX);
+}
+
+static inline void gen_vfp_abss(TCGv_i32 d, TCGv_i32 s)
+{
+tcg_gen_andi_i32(d, s, INT32_MAX);
+}
+
+static inline void gen_vfp_absd(TCGv_i64 d, TCGv_i64 s)
+{
+tcg_gen_andi_i64(d, s, INT64_MAX);
+}
+
+static inline void gen_vfp_negh(TCGv_i32 d, TCGv_i32 s)
+{
+tcg_gen_xori_i32(d, s, 1u << 15);
+}
+
+static inline void gen_vfp_negs(TCGv_i32 d, TCGv_i32 s)
+{
+tcg_gen_xori_i32(d, s, 1u << 31);
+}
+
+static inline void gen_vfp_negd(TCGv_i64 d, TCGv_i64 s)
+{
+tcg_gen_xori_i64(d, s, 1ull << 63);
+}
+
 /* Vector operations shared between ARM and AArch64.  */
 void gen_gvec_ceq0(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
uint32_t opr_sz, uint32_t max_sz);
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 6f8207d842..878f83298f 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -6591,10 +6591,10 @@ static void handle_fp_1src_half(DisasContext *s, int 
opcode, int rd, int rn)
 tcg_gen_mov_i32(tcg_res, tcg_op);
 break;
 case 0x1: /* FABS */
-tcg_gen_andi_i32(tcg_res, tcg_op, 0x7fff);
+gen_vfp_absh(tcg_res, tcg_op);
 break;
 case 0x2: /* FNEG */
-tcg_gen_xori_i32(tcg_res, tcg_op, 0x8000);
+gen_vfp_negh(tcg_res, tcg_op);
 break;
 case 0x3: /* FSQRT */
 fpst = fpstatus_ptr(FPST_FPCR_F16);
@@ -6645,10 +6645,10 @@ static void handle_fp_1src_single(DisasContext *s, int 
opcode, int rd, int rn)
 tcg_gen_mov_i32(tcg_res, tcg_op);
 goto done;
 case 0x1: /* FABS */
-gen_helper_vfp_abss(tcg_res, tcg_op);
+gen_vfp_abss(tcg_res, tcg_op);
 goto done;
 case 0x2: /* FNEG */
-gen_helper_vfp_negs(tcg_res, tcg_op);
+gen_vfp_negs(tcg_res, tcg_op);
 goto done;
 case 0x3: /* FSQRT */
 gen_helper_vfp_sqrts(tcg_res, tcg_op, tcg_env);
@@ -6720,10 +6720,10 @@ static void handle_fp_1src_double(DisasContext *s, int 
opcode, int rd, int rn)
 
 switch (opcode) {
 case 0x1: /* FABS */
-gen_helper_vfp_absd(tcg_res, tcg_op);
+gen_vfp_absd(tcg_res, tcg_op);
 goto done;
 case 0x2: /* FNEG */
-gen_helper_vfp_negd(tcg_res, tcg_op);
+gen_vfp_negd(tcg_res, tcg_op);
 goto done;
 case 0x3: /* FSQRT */
 gen_helper_vfp_sqrtd(tcg_res, tcg_op, tcg_env);
@@ -6949,7 +6949,7 @@ static void handle_fp_2src_single(DisasContext *s, int 
opcode,
 switch (opcode) {
 case 0x8: /* FNMUL */
 gen_helper_vfp_muls(tcg_res, tcg_op1, tcg_op2, fpst);
-gen_helper_vfp_negs(tcg_res, tcg_res);
+gen_vfp_negs(tcg_res, tcg_res);
 break;
 default:
 case 0x0: /* FMUL */
@@ -6983,7 +6983,7 @@ static void handle_fp_2src_double(DisasContext *s, int 
opcode,
 switch (opcode) {
 case 0x8: /* FNMUL */
 gen_helper_vfp_muld(tcg_res, tcg_op1, tcg_op2, fpst);
-gen_helper_vfp_negd(tcg_res, tcg_res);
+gen_vfp_negd(tcg_res, tcg_res);
 break;
 default:
 case 0x0: /* FMUL */
@@ -7017,7 +7017,7 @@ static void handle_fp_2src_half(DisasContext *s, int 
opcode,
 switch (opcode) {
 case 0x8: /* FNMUL */
 gen_helper_advsimd_mulh(tcg_res, tcg_op1, tcg_op2, fpst);
-tcg_gen_xori_i32(tcg_res, tcg_res, 0x8000);
+gen_vfp_negh(tcg_res, tcg_res);
 break;
 default:
 case 0x0: /* FMUL */
@@ -7102,11 +7102,11 @@ static void handle_fp_3src_sin

[PATCH v2 07/67] target/arm: Split out gengvec.c

[Richard Henderson]

Reviewed-by: Peter Maydell 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/translate.h |5 +
 target/arm/tcg/gengvec.c   | 1612 
 target/arm/tcg/translate.c | 1588 ---
 target/arm/tcg/meson.build |1 +
 4 files changed, 1618 insertions(+), 1588 deletions(-)
 create mode 100644 target/arm/tcg/gengvec.c

diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h
index dc66ff2190..80e85096a8 100644
--- a/target/arm/tcg/translate.h
+++ b/target/arm/tcg/translate.h
@@ -445,6 +445,11 @@ void gen_gvec_ssra(unsigned vece, uint32_t rd_ofs, 
uint32_t rm_ofs,
 void gen_gvec_usra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
int64_t shift, uint32_t opr_sz, uint32_t max_sz);
 
+void gen_srshr32_i32(TCGv_i32 d, TCGv_i32 a, int32_t sh);
+void gen_srshr64_i64(TCGv_i64 d, TCGv_i64 a, int64_t sh);
+void gen_urshr32_i32(TCGv_i32 d, TCGv_i32 a, int32_t sh);
+void gen_urshr64_i64(TCGv_i64 d, TCGv_i64 a, int64_t sh);
+
 void gen_gvec_srshr(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
 int64_t shift, uint32_t opr_sz, uint32_t max_sz);
 void gen_gvec_urshr(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c
new file mode 100644
index 00..7a1856253f
--- /dev/null
+++ b/target/arm/tcg/gengvec.c
@@ -0,0 +1,1612 @@
+/*
+ *  ARM generic vector expansion
+ *
+ *  Copyright (c) 2003 Fabrice Bellard
+ *  Copyright (c) 2005-2007 CodeSourcery
+ *  Copyright (c) 2007 OpenedHand, Ltd.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see .
+ */
+
+#include "qemu/osdep.h"
+#include "translate.h"
+
+
+static void gen_gvec_fn3_qc(uint32_t rd_ofs, uint32_t rn_ofs, uint32_t rm_ofs,
+uint32_t opr_sz, uint32_t max_sz,
+gen_helper_gvec_3_ptr *fn)
+{
+TCGv_ptr qc_ptr = tcg_temp_new_ptr();
+
+tcg_gen_addi_ptr(qc_ptr, tcg_env, offsetof(CPUARMState, vfp.qc));
+tcg_gen_gvec_3_ptr(rd_ofs, rn_ofs, rm_ofs, qc_ptr,
+   opr_sz, max_sz, 0, fn);
+}
+
+void gen_gvec_sqrdmlah_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+  uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+static gen_helper_gvec_3_ptr * const fns[2] = {
+gen_helper_gvec_qrdmlah_s16, gen_helper_gvec_qrdmlah_s32
+};
+tcg_debug_assert(vece >= 1 && vece <= 2);
+gen_gvec_fn3_qc(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, fns[vece - 1]);
+}
+
+void gen_gvec_sqrdmlsh_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+  uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+static gen_helper_gvec_3_ptr * const fns[2] = {
+gen_helper_gvec_qrdmlsh_s16, gen_helper_gvec_qrdmlsh_s32
+};
+tcg_debug_assert(vece >= 1 && vece <= 2);
+gen_gvec_fn3_qc(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, fns[vece - 1]);
+}
+
+#define GEN_CMP0(NAME, COND)  \
+void NAME(unsigned vece, uint32_t d, uint32_t m,  \
+  uint32_t opr_sz, uint32_t max_sz)   \
+{ tcg_gen_gvec_cmpi(COND, vece, d, m, 0, opr_sz, max_sz); }
+
+GEN_CMP0(gen_gvec_ceq0, TCG_COND_EQ)
+GEN_CMP0(gen_gvec_cle0, TCG_COND_LE)
+GEN_CMP0(gen_gvec_cge0, TCG_COND_GE)
+GEN_CMP0(gen_gvec_clt0, TCG_COND_LT)
+GEN_CMP0(gen_gvec_cgt0, TCG_COND_GT)
+
+#undef GEN_CMP0
+
+static void gen_ssra8_i64(TCGv_i64 d, TCGv_i64 a, int64_t shift)
+{
+tcg_gen_vec_sar8i_i64(a, a, shift);
+tcg_gen_vec_add8_i64(d, d, a);
+}
+
+static void gen_ssra16_i64(TCGv_i64 d, TCGv_i64 a, int64_t shift)
+{
+tcg_gen_vec_sar16i_i64(a, a, shift);
+tcg_gen_vec_add16_i64(d, d, a);
+}
+
+static void gen_ssra32_i32(TCGv_i32 d, TCGv_i32 a, int32_t shift)
+{
+tcg_gen_sari_i32(a, a, shift);
+tcg_gen_add_i32(d, d, a);
+}
+
+static void gen_ssra64_i64(TCGv_i64 d, TCGv_i64 a, int64_t shift)
+{
+tcg_gen_sari_i64(a, a, shift);
+tcg_gen_add_i64(d, d, a);
+}
+
+static void gen_ssra_vec(unsigned vece, TCGv_vec d, TCGv_vec a, int64_t sh)
+{
+tcg_gen_sari_vec(vece, a, a, sh);
+tcg_gen_add_vec(vece, d, d, a);
+}
+
+void gen_gvec_ssra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+   int64_t shift, uint32_t opr_sz, uint32_t max_sz)
+{
+static const TCGOpcode vecop

[PATCH v2 11/67] target/arm: Convert Cryptographic 2-register SHA to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  6 
 target/arm/tcg/translate-a64.c | 54 +++---
 2 files changed, 10 insertions(+), 50 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 7590659ee6..350afabc77 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -614,3 +614,9 @@ SHA1SU0 0101 1110 000 . 001100 . .  
@rrr_q1e0
 SHA256H 0101 1110 000 . 01 . .  @rrr_q1e0
 SHA256H20101 1110 000 . 010100 . .  @rrr_q1e0
 SHA256SU1   0101 1110 000 . 011000 . .  @rrr_q1e0
+
+### Cryptographic two-register SHA
+
+SHA1H   0101 1110 0010 1000  10 . . @rr_q1e0
+SHA1SU1 0101 1110 0010 1000 0001 10 . . @rr_q1e0
+SHA256SU0   0101 1110 0010 1000 0010 10 . . @rr_q1e0
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 5bef39d4e7..1d20bf0c35 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -4606,6 +4606,10 @@ TRANS_FEAT(SHA256H, aa64_sha256, do_gvec_op3_ool, a, 0, 
gen_helper_crypto_sha256
 TRANS_FEAT(SHA256H2, aa64_sha256, do_gvec_op3_ool, a, 0, 
gen_helper_crypto_sha256h2)
 TRANS_FEAT(SHA256SU1, aa64_sha256, do_gvec_op3_ool, a, 0, 
gen_helper_crypto_sha256su1)
 
+TRANS_FEAT(SHA1H, aa64_sha1, do_gvec_op2_ool, a, 0, gen_helper_crypto_sha1h)
+TRANS_FEAT(SHA1SU1, aa64_sha1, do_gvec_op2_ool, a, 0, 
gen_helper_crypto_sha1su1)
+TRANS_FEAT(SHA256SU0, aa64_sha256, do_gvec_op2_ool, a, 0, 
gen_helper_crypto_sha256su0)
+
 /* Shift a TCGv src by TCGv shift_amount, put result in dst.
  * Note that it is the caller's responsibility to ensure that the
  * shift amount is in range (ie 0..31 or 0..63) and provide the ARM
@@ -13506,55 +13510,6 @@ static void disas_simd_indexed(DisasContext *s, 
uint32_t insn)
 }
 }
 
-/* Crypto two-reg SHA
- *  31 24 23  22 21   17 1612 11 10 95 40
- * +-+--+---++-+--+--+
- * | 0 1 0 1 1 1 1 0 | size | 1 0 1 0 0 | opcode | 1 0 |  Rn  |  Rd  |
- * +-+--+---++-+--+--+
- */
-static void disas_crypto_two_reg_sha(DisasContext *s, uint32_t insn)
-{
-int size = extract32(insn, 22, 2);
-int opcode = extract32(insn, 12, 5);
-int rn = extract32(insn, 5, 5);
-int rd = extract32(insn, 0, 5);
-gen_helper_gvec_2 *genfn;
-bool feature;
-
-if (size != 0) {
-unallocated_encoding(s);
-return;
-}
-
-switch (opcode) {
-case 0: /* SHA1H */
-feature = dc_isar_feature(aa64_sha1, s);
-genfn = gen_helper_crypto_sha1h;
-break;
-case 1: /* SHA1SU1 */
-feature = dc_isar_feature(aa64_sha1, s);
-genfn = gen_helper_crypto_sha1su1;
-break;
-case 2: /* SHA256SU0 */
-feature = dc_isar_feature(aa64_sha256, s);
-genfn = gen_helper_crypto_sha256su0;
-break;
-default:
-unallocated_encoding(s);
-return;
-}
-
-if (!feature) {
-unallocated_encoding(s);
-return;
-}
-
-if (!fp_access_check(s)) {
-return;
-}
-gen_gvec_op2_ool(s, true, rd, rn, 0, genfn);
-}
-
 /* Crypto three-reg SHA512
  *  31   21 20  16 15  14  13 12  11  10  95 40
  * +---+--+---+---+-++--+--+
@@ -13849,7 +13804,6 @@ static const AArch64DecodeTable data_proc_simd[] = {
 { 0x5e000400, 0xdfe08400, disas_simd_scalar_copy },
 { 0x5f00, 0xdf000400, disas_simd_indexed }, /* scalar indexed */
 { 0x5f000400, 0xdf800400, disas_simd_scalar_shift_imm },
-{ 0x5e280800, 0xff3e0c00, disas_crypto_two_reg_sha },
 { 0xce608000, 0xffe0b000, disas_crypto_three_reg_sha512 },
 { 0xcec08000, 0xf000, disas_crypto_two_reg_sha512 },
 { 0xce00, 0xff808000, disas_crypto_four_reg },
-- 
2.34.1

[PATCH v2 10/67] target/arm: Convert Cryptographic 3-register SHA to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  | 11 +
 target/arm/tcg/translate-a64.c | 78 +-
 2 files changed, 21 insertions(+), 68 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 1de09903dc..7590659ee6 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -30,6 +30,7 @@
 
 @rr_q1e0  .. rn:5 rd:5  &qrr_e q=1 esz=0
 @r2r_q1e0     .. rm:5 rd:5  &qrrr_e rn=%rd q=1 
esz=0
+@rrr_q1e0    ... rm:5 .. rn:5 rd:5  &qrrr_e q=1 esz=0
 
 ### Data Processing - Immediate
 
@@ -603,3 +604,13 @@ AESE01001110 00 10100 00100 10 . .  
@r2r_q1e0
 AESD01001110 00 10100 00101 10 . .  @r2r_q1e0
 AESMC   01001110 00 10100 00110 10 . .  @rr_q1e0
 AESIMC  01001110 00 10100 00111 10 . .  @rr_q1e0
+
+### Cryptographic three-register SHA
+
+SHA1C   0101 1110 000 . 00 . .  @rrr_q1e0
+SHA1P   0101 1110 000 . 000100 . .  @rrr_q1e0
+SHA1M   0101 1110 000 . 001000 . .  @rrr_q1e0
+SHA1SU0 0101 1110 000 . 001100 . .  @rrr_q1e0
+SHA256H 0101 1110 000 . 01 . .  @rrr_q1e0
+SHA256H20101 1110 000 . 010100 . .  @rrr_q1e0
+SHA256SU1   0101 1110 000 . 011000 . .  @rrr_q1e0
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 3894db4bee..5bef39d4e7 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -4589,7 +4589,7 @@ static bool trans_EXTR(DisasContext *s, arg_extract *a)
 }
 
 /*
- * Cryptographic AES
+ * Cryptographic AES, SHA
  */
 
 TRANS_FEAT(AESE, aa64_aes, do_gvec_op3_ool, a, 0, gen_helper_crypto_aese)
@@ -4597,6 +4597,15 @@ TRANS_FEAT(AESD, aa64_aes, do_gvec_op3_ool, a, 0, 
gen_helper_crypto_aesd)
 TRANS_FEAT(AESMC, aa64_aes, do_gvec_op2_ool, a, 0, gen_helper_crypto_aesmc)
 TRANS_FEAT(AESIMC, aa64_aes, do_gvec_op2_ool, a, 0, gen_helper_crypto_aesimc)
 
+TRANS_FEAT(SHA1C, aa64_sha1, do_gvec_op3_ool, a, 0, gen_helper_crypto_sha1c)
+TRANS_FEAT(SHA1P, aa64_sha1, do_gvec_op3_ool, a, 0, gen_helper_crypto_sha1p)
+TRANS_FEAT(SHA1M, aa64_sha1, do_gvec_op3_ool, a, 0, gen_helper_crypto_sha1m)
+TRANS_FEAT(SHA1SU0, aa64_sha1, do_gvec_op3_ool, a, 0, 
gen_helper_crypto_sha1su0)
+
+TRANS_FEAT(SHA256H, aa64_sha256, do_gvec_op3_ool, a, 0, 
gen_helper_crypto_sha256h)
+TRANS_FEAT(SHA256H2, aa64_sha256, do_gvec_op3_ool, a, 0, 
gen_helper_crypto_sha256h2)
+TRANS_FEAT(SHA256SU1, aa64_sha256, do_gvec_op3_ool, a, 0, 
gen_helper_crypto_sha256su1)
+
 /* Shift a TCGv src by TCGv shift_amount, put result in dst.
  * Note that it is the caller's responsibility to ensure that the
  * shift amount is in range (ie 0..31 or 0..63) and provide the ARM
@@ -13497,72 +13506,6 @@ static void disas_simd_indexed(DisasContext *s, 
uint32_t insn)
 }
 }
 
-/* Crypto three-reg SHA
- *  31 24 23  22  21 20  16  15 1412 11 10 95 40
- * +-+--+---+--+---++-+--+--+
- * | 0 1 0 1 1 1 1 0 | size | 0 |  Rm  | 0 | opcode | 0 0 |  Rn  |  Rd  |
- * +-+--+---+--+---++-+--+--+
- */
-static void disas_crypto_three_reg_sha(DisasContext *s, uint32_t insn)
-{
-int size = extract32(insn, 22, 2);
-int opcode = extract32(insn, 12, 3);
-int rm = extract32(insn, 16, 5);
-int rn = extract32(insn, 5, 5);
-int rd = extract32(insn, 0, 5);
-gen_helper_gvec_3 *genfn;
-bool feature;
-
-if (size != 0) {
-unallocated_encoding(s);
-return;
-}
-
-switch (opcode) {
-case 0: /* SHA1C */
-genfn = gen_helper_crypto_sha1c;
-feature = dc_isar_feature(aa64_sha1, s);
-break;
-case 1: /* SHA1P */
-genfn = gen_helper_crypto_sha1p;
-feature = dc_isar_feature(aa64_sha1, s);
-break;
-case 2: /* SHA1M */
-genfn = gen_helper_crypto_sha1m;
-feature = dc_isar_feature(aa64_sha1, s);
-break;
-case 3: /* SHA1SU0 */
-genfn = gen_helper_crypto_sha1su0;
-feature = dc_isar_feature(aa64_sha1, s);
-break;
-case 4: /* SHA256H */
-genfn = gen_helper_crypto_sha256h;
-feature = dc_isar_feature(aa64_sha256, s);
-break;
-case 5: /* SHA256H2 */
-genfn = gen_helper_crypto_sha256h2;
-feature = dc_isar_feature(aa64_sha256, s);
-break;
-case 6: /* SHA256SU1 */
-genfn = gen_helper_crypto_sha256su1;
-feature = dc_isar_feature(aa64_sha256, s);
-break;
-default:
-unallocated_encoding(s);
-return;
-}
-
-if (!feature) {
-unallocated_encoding(s);
-return;
-}
-
-if (!fp_access_check(s)) {
-return;
-}
-gen_gvec_op3_ool(s, true, rd, rn, rm, 0

[PATCH v2 06/67] target/arm: Verify sz=0 for Advanced SIMD scalar pairwise (fp16)

[Richard Henderson]

All of these insns have "if sz == '1' then UNDEFINED" in their pseudocode.
Fixes a RISU miscompare for invalid insn 0x5ef0c87a.

Fixes: 5c36d89567c ("arm/translate-a64: add all FP16 ops in 
simd_scalar_pairwise")
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/translate-a64.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 5455ae3685..0bdddb8517 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -8006,7 +8006,7 @@ static void disas_simd_scalar_pairwise(DisasContext *s, 
uint32_t insn)
 case 0x2f: /* FMINP */
 /* FP op, size[0] is 32 or 64 bit*/
 if (!u) {
-if (!dc_isar_feature(aa64_fp16, s)) {
+if ((size & 1) || !dc_isar_feature(aa64_fp16, s)) {
 unallocated_encoding(s);
 return;
 } else {
-- 
2.34.1

[PATCH v2 23/67] target/arm: Convert FNMUL to decodetree

[Richard Henderson]

This is the last instruction within disas_fp_2src,
so remove that and its subroutines.

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |   1 +
 target/arm/tcg/translate-a64.c | 177 +
 2 files changed, 27 insertions(+), 151 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index e2678d919e..cde4b86303 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -703,6 +703,7 @@ FADD_s  0001 1110 ..1 . 0010 10 . . 
@rrr_hsd
 FSUB_s  0001 1110 ..1 . 0011 10 . . @rrr_hsd
 FDIV_s  0001 1110 ..1 . 0001 10 . . @rrr_hsd
 FMUL_s  0001 1110 ..1 .  10 . . @rrr_hsd
+FNMUL_s 0001 1110 ..1 . 1000 10 . . @rrr_hsd
 
 FMAX_s  0001 1110 ..1 . 0100 10 . . @rrr_hsd
 FMIN_s  0001 1110 ..1 . 0101 10 . . @rrr_hsd
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 878f83298f..5ba30ba7c8 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -4950,6 +4950,31 @@ static const FPScalar f_scalar_fmulx = {
 };
 TRANS(FMULX_s, do_fp3_scalar, a, &f_scalar_fmulx)
 
+static void gen_fnmul_h(TCGv_i32 d, TCGv_i32 n, TCGv_i32 m, TCGv_ptr s)
+{
+gen_helper_vfp_mulh(d, n, m, s);
+gen_vfp_negh(d, d);
+}
+
+static void gen_fnmul_s(TCGv_i32 d, TCGv_i32 n, TCGv_i32 m, TCGv_ptr s)
+{
+gen_helper_vfp_muls(d, n, m, s);
+gen_vfp_negs(d, d);
+}
+
+static void gen_fnmul_d(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_ptr s)
+{
+gen_helper_vfp_muld(d, n, m, s);
+gen_vfp_negd(d, d);
+}
+
+static const FPScalar f_scalar_fnmul = {
+gen_fnmul_h,
+gen_fnmul_s,
+gen_fnmul_d,
+};
+TRANS(FNMUL_s, do_fp3_scalar, a, &f_scalar_fnmul)
+
 static bool do_fp3_vector(DisasContext *s, arg_qrrr_e *a,
   gen_helper_gvec_3_ptr * const fns[3])
 {
@@ -6932,156 +6957,6 @@ static void disas_fp_1src(DisasContext *s, uint32_t 
insn)
 }
 }
 
-/* Floating-point data-processing (2 source) - single precision */
-static void handle_fp_2src_single(DisasContext *s, int opcode,
-  int rd, int rn, int rm)
-{
-TCGv_i32 tcg_op1;
-TCGv_i32 tcg_op2;
-TCGv_i32 tcg_res;
-TCGv_ptr fpst;
-
-tcg_res = tcg_temp_new_i32();
-fpst = fpstatus_ptr(FPST_FPCR);
-tcg_op1 = read_fp_sreg(s, rn);
-tcg_op2 = read_fp_sreg(s, rm);
-
-switch (opcode) {
-case 0x8: /* FNMUL */
-gen_helper_vfp_muls(tcg_res, tcg_op1, tcg_op2, fpst);
-gen_vfp_negs(tcg_res, tcg_res);
-break;
-default:
-case 0x0: /* FMUL */
-case 0x1: /* FDIV */
-case 0x2: /* FADD */
-case 0x3: /* FSUB */
-case 0x4: /* FMAX */
-case 0x5: /* FMIN */
-case 0x6: /* FMAXNM */
-case 0x7: /* FMINNM */
-g_assert_not_reached();
-}
-
-write_fp_sreg(s, rd, tcg_res);
-}
-
-/* Floating-point data-processing (2 source) - double precision */
-static void handle_fp_2src_double(DisasContext *s, int opcode,
-  int rd, int rn, int rm)
-{
-TCGv_i64 tcg_op1;
-TCGv_i64 tcg_op2;
-TCGv_i64 tcg_res;
-TCGv_ptr fpst;
-
-tcg_res = tcg_temp_new_i64();
-fpst = fpstatus_ptr(FPST_FPCR);
-tcg_op1 = read_fp_dreg(s, rn);
-tcg_op2 = read_fp_dreg(s, rm);
-
-switch (opcode) {
-case 0x8: /* FNMUL */
-gen_helper_vfp_muld(tcg_res, tcg_op1, tcg_op2, fpst);
-gen_vfp_negd(tcg_res, tcg_res);
-break;
-default:
-case 0x0: /* FMUL */
-case 0x1: /* FDIV */
-case 0x2: /* FADD */
-case 0x3: /* FSUB */
-case 0x4: /* FMAX */
-case 0x5: /* FMIN */
-case 0x6: /* FMAXNM */
-case 0x7: /* FMINNM */
-g_assert_not_reached();
-}
-
-write_fp_dreg(s, rd, tcg_res);
-}
-
-/* Floating-point data-processing (2 source) - half precision */
-static void handle_fp_2src_half(DisasContext *s, int opcode,
-int rd, int rn, int rm)
-{
-TCGv_i32 tcg_op1;
-TCGv_i32 tcg_op2;
-TCGv_i32 tcg_res;
-TCGv_ptr fpst;
-
-tcg_res = tcg_temp_new_i32();
-fpst = fpstatus_ptr(FPST_FPCR_F16);
-tcg_op1 = read_fp_hreg(s, rn);
-tcg_op2 = read_fp_hreg(s, rm);
-
-switch (opcode) {
-case 0x8: /* FNMUL */
-gen_helper_advsimd_mulh(tcg_res, tcg_op1, tcg_op2, fpst);
-gen_vfp_negh(tcg_res, tcg_res);
-break;
-default:
-case 0x0: /* FMUL */
-case 0x1: /* FDIV */
-case 0x2: /* FADD */
-case 0x3: /* FSUB */
-case 0x4: /* FMAX */
-case 0x5: /* FMIN */
-case 0x6: /* FMAXNM */
-case 0x7: /* FMINNM */
-g_assert_not_reached();
-}
-
-write_fp_sreg(s, rd, tcg_res);
-}
-
-/* Floating point data-processing (2 source)
- *   31  30  29 28   24 23  22  21 20  16 1512 11 10 95 40
- * +---+---+---+---+--+---+--++---

[PATCH v2 01/67] target/arm: Add neoverse-n1 to qemu-arm (DO NOT MERGE)

[Richard Henderson]

Hack, because there should be a better way to do this without
duplicating code between cpu32.c and cpu64.c.  Hack, because
qemu-arm crashes without ARM_FEATURE_AARCH64 disabled.

Needed in order to compare RISU results with aarch64.ci.qemu.org.

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/cpu32.c | 73 ++
 1 file changed, 73 insertions(+)

diff --git a/target/arm/tcg/cpu32.c b/target/arm/tcg/cpu32.c
index bdd82d912a..6ee055c78b 100644
--- a/target/arm/tcg/cpu32.c
+++ b/target/arm/tcg/cpu32.c
@@ -978,6 +978,78 @@ static void arm_max_initfn(Object *obj)
 }
 #endif /* !TARGET_AARCH64 */
 
+#ifdef CONFIG_USER_ONLY
+static void aarch64_neoverse_n1_initfn(Object *obj)
+{
+ARMCPU *cpu = ARM_CPU(obj);
+
+cpu->dtb_compatible = "arm,neoverse-n1";
+set_feature(&cpu->env, ARM_FEATURE_V8);
+set_feature(&cpu->env, ARM_FEATURE_NEON);
+set_feature(&cpu->env, ARM_FEATURE_GENERIC_TIMER);
+set_feature(&cpu->env, ARM_FEATURE_BACKCOMPAT_CNTFRQ);
+// set_feature(&cpu->env, ARM_FEATURE_AARCH64);
+set_feature(&cpu->env, ARM_FEATURE_CBAR_RO);
+set_feature(&cpu->env, ARM_FEATURE_EL2);
+set_feature(&cpu->env, ARM_FEATURE_EL3);
+set_feature(&cpu->env, ARM_FEATURE_PMU);
+
+/* Ordered by B2.4 AArch64 registers by functional group */
+cpu->clidr = 0x8223;
+cpu->ctr = 0x8444c004;
+cpu->dcz_blocksize = 4;
+cpu->isar.id_aa64dfr0  = 0x000110305408ull;
+cpu->isar.id_aa64isar0 = 0x100010211120ull;
+cpu->isar.id_aa64isar1 = 0x0011ull;
+cpu->isar.id_aa64mmfr0 = 0x00101125ull;
+cpu->isar.id_aa64mmfr1 = 0x10212122ull;
+cpu->isar.id_aa64mmfr2 = 0x1011ull;
+cpu->isar.id_aa64pfr0  = 0x11001012ull; /* GIC filled in later */
+cpu->isar.id_aa64pfr1  = 0x0020ull;
+cpu->id_afr0   = 0x;
+cpu->isar.id_dfr0  = 0x04010088;
+cpu->isar.id_isar0 = 0x02101110;
+cpu->isar.id_isar1 = 0x13112111;
+cpu->isar.id_isar2 = 0x21232042;
+cpu->isar.id_isar3 = 0x01112131;
+cpu->isar.id_isar4 = 0x00010142;
+cpu->isar.id_isar5 = 0x01011121;
+cpu->isar.id_isar6 = 0x0010;
+cpu->isar.id_mmfr0 = 0x10201105;
+cpu->isar.id_mmfr1 = 0x4000;
+cpu->isar.id_mmfr2 = 0x0126;
+cpu->isar.id_mmfr3 = 0x02122211;
+cpu->isar.id_mmfr4 = 0x00021110;
+cpu->isar.id_pfr0  = 0x10010131;
+cpu->isar.id_pfr1  = 0x0001; /* GIC filled in later */
+cpu->isar.id_pfr2  = 0x0011;
+cpu->midr = 0x414fd0c1;  /* r4p1 */
+cpu->revidr = 0;
+
+/* From B2.23 CCSIDR_EL1 */
+cpu->ccsidr[0] = 0x701fe01a; /* 64KB L1 dcache */
+cpu->ccsidr[1] = 0x201fe01a; /* 64KB L1 icache */
+cpu->ccsidr[2] = 0x70ffe03a; /* 1MB L2 cache */
+
+/* From B2.98 SCTLR_EL3 */
+cpu->reset_sctlr = 0x30c50838;
+
+/* From B4.23 ICH_VTR_EL2 */
+cpu->gic_num_lrs = 4;
+cpu->gic_vpribits = 5;
+cpu->gic_vprebits = 5;
+cpu->gic_pribits = 5;
+
+/* From B5.1 AdvSIMD AArch64 register summary */
+cpu->isar.mvfr0 = 0x10110222;
+cpu->isar.mvfr1 = 0x1321;
+cpu->isar.mvfr2 = 0x0043;
+
+/* From D5.1 AArch64 PMU register summary */
+cpu->isar.reset_pmcr_el0 = 0x410c3000;
+}
+#endif /* CONFIG_USER_ONLY */
+
 static const ARMCPUInfo arm_tcg_cpus[] = {
 { .name = "arm926",  .initfn = arm926_initfn },
 { .name = "arm946",  .initfn = arm946_initfn },
@@ -1018,6 +1090,7 @@ static const ARMCPUInfo arm_tcg_cpus[] = {
 { .name = "max", .initfn = arm_max_initfn },
 #endif
 #ifdef CONFIG_USER_ONLY
+{ .name = "neoverse-n1", .initfn = aarch64_neoverse_n1_initfn },
 { .name = "any", .initfn = arm_max_initfn },
 #endif
 };
-- 
2.34.1

[PATCH v2 39/67] target/arm: Inline scalar SUQADD and USQADD

[Richard Henderson]

This eliminates the last uses of these neon helpers.
Incorporate the MO_64 expanders as an option to the vector expander.

Signed-off-by: Richard Henderson 
---
 target/arm/helper.h|   8 --
 target/arm/tcg/translate-a64.h |   8 ++
 target/arm/tcg/gengvec64.c |  71 ++
 target/arm/tcg/neon_helper.c   | 165 -
 target/arm/tcg/translate-a64.c |  73 +--
 5 files changed, 103 insertions(+), 222 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index de2c5c9aef..c76158d6d3 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -274,14 +274,6 @@ DEF_HELPER_FLAGS_3(neon_qadd_u16, TCG_CALL_NO_RWG, i32, 
env, i32, i32)
 DEF_HELPER_FLAGS_3(neon_qadd_s16, TCG_CALL_NO_RWG, i32, env, i32, i32)
 DEF_HELPER_FLAGS_3(neon_qadd_u32, TCG_CALL_NO_RWG, i32, env, i32, i32)
 DEF_HELPER_FLAGS_3(neon_qadd_s32, TCG_CALL_NO_RWG, i32, env, i32, i32)
-DEF_HELPER_FLAGS_3(neon_uqadd_s8, TCG_CALL_NO_RWG, i32, env, i32, i32)
-DEF_HELPER_FLAGS_3(neon_uqadd_s16, TCG_CALL_NO_RWG, i32, env, i32, i32)
-DEF_HELPER_FLAGS_3(neon_uqadd_s32, TCG_CALL_NO_RWG, i32, env, i32, i32)
-DEF_HELPER_FLAGS_3(neon_uqadd_s64, TCG_CALL_NO_RWG, i64, env, i64, i64)
-DEF_HELPER_FLAGS_3(neon_sqadd_u8, TCG_CALL_NO_RWG, i32, env, i32, i32)
-DEF_HELPER_FLAGS_3(neon_sqadd_u16, TCG_CALL_NO_RWG, i32, env, i32, i32)
-DEF_HELPER_FLAGS_3(neon_sqadd_u32, TCG_CALL_NO_RWG, i32, env, i32, i32)
-DEF_HELPER_FLAGS_3(neon_sqadd_u64, TCG_CALL_NO_RWG, i64, env, i64, i64)
 DEF_HELPER_3(neon_qsub_u8, i32, env, i32, i32)
 DEF_HELPER_3(neon_qsub_s8, i32, env, i32, i32)
 DEF_HELPER_3(neon_qsub_u16, i32, env, i32, i32)
diff --git a/target/arm/tcg/translate-a64.h b/target/arm/tcg/translate-a64.h
index b5cb26f8a2..0fcf7cb63a 100644
--- a/target/arm/tcg/translate-a64.h
+++ b/target/arm/tcg/translate-a64.h
@@ -197,9 +197,17 @@ void gen_gvec_eor3(unsigned vece, uint32_t d, uint32_t n, 
uint32_t m,
uint32_t a, uint32_t oprsz, uint32_t maxsz);
 void gen_gvec_bcax(unsigned vece, uint32_t d, uint32_t n, uint32_t m,
uint32_t a, uint32_t oprsz, uint32_t maxsz);
+
+void gen_suqadd_bhs(TCGv_i64 res, TCGv_i64 qc,
+TCGv_i64 a, TCGv_i64 b, MemOp esz);
+void gen_suqadd_d(TCGv_i64 res, TCGv_i64 qc, TCGv_i64 a, TCGv_i64 b);
 void gen_gvec_suqadd_qc(unsigned vece, uint32_t rd_ofs,
 uint32_t rn_ofs, uint32_t rm_ofs,
 uint32_t opr_sz, uint32_t max_sz);
+
+void gen_usqadd_bhs(TCGv_i64 res, TCGv_i64 qc,
+TCGv_i64 a, TCGv_i64 b, MemOp esz);
+void gen_usqadd_d(TCGv_i64 res, TCGv_i64 qc, TCGv_i64 a, TCGv_i64 b);
 void gen_gvec_usqadd_qc(unsigned vece, uint32_t rd_ofs,
 uint32_t rn_ofs, uint32_t rm_ofs,
 uint32_t opr_sz, uint32_t max_sz);
diff --git a/target/arm/tcg/gengvec64.c b/target/arm/tcg/gengvec64.c
index 4b76e476a0..dad4c1853b 100644
--- a/target/arm/tcg/gengvec64.c
+++ b/target/arm/tcg/gengvec64.c
@@ -188,6 +188,38 @@ void gen_gvec_bcax(unsigned vece, uint32_t d, uint32_t n, 
uint32_t m,
 tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op);
 }
 
+/*
+ * Set @res to the correctly saturated result.
+ * Set @qc non-zero if saturation occured.
+ */
+void gen_suqadd_bhs(TCGv_i64 res, TCGv_i64 qc,
+TCGv_i64 a, TCGv_i64 b, MemOp esz)
+{
+TCGv_i64 max = tcg_constant_i64((1ull << ((8 << esz) - 1)) - 1);
+TCGv_i64 t = tcg_temp_new_i64();
+
+tcg_gen_add_i64(t, a, b);
+tcg_gen_smin_i64(res, t, max);
+tcg_gen_xor_i64(t, t, res);
+tcg_gen_or_i64(qc, qc, t);
+}
+
+void gen_suqadd_d(TCGv_i64 res, TCGv_i64 qc, TCGv_i64 a, TCGv_i64 b)
+{
+TCGv_i64 max = tcg_constant_i64(INT64_MAX);
+TCGv_i64 t = tcg_temp_new_i64();
+
+/* Maximum value that can be added to @a without overflow. */
+tcg_gen_sub_i64(t, max, a);
+
+/* Constrain addend so that the next addition never overflows. */
+tcg_gen_umin_i64(t, t, b);
+tcg_gen_add_i64(res, a, t);
+
+tcg_gen_xor_i64(t, t, b);
+tcg_gen_or_i64(qc, qc, t);
+}
+
 static void gen_suqadd_vec(unsigned vece, TCGv_vec t, TCGv_vec qc,
TCGv_vec a, TCGv_vec b)
 {
@@ -231,6 +263,7 @@ void gen_gvec_suqadd_qc(unsigned vece, uint32_t rd_ofs,
   .write_aofs = true,
   .vece = MO_32 },
 { .fniv = gen_suqadd_vec,
+  .fni8 = gen_suqadd_d,
   .fno = gen_helper_gvec_suqadd_d,
   .opt_opc = vecop_list,
   .write_aofs = true,
@@ -240,6 +273,43 @@ void gen_gvec_suqadd_qc(unsigned vece, uint32_t rd_ofs,
rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
 }
 
+void gen_usqadd_bhs(TCGv_i64 res, TCGv_i64 qc,
+TCGv_i64 a, TCGv_i64 b, MemOp esz)
+{
+TCGv_i64 max = tcg_constant_i64(MAKE_64BIT_MASK(0, 8 << esz));
+TCGv_i64 zero = tcg_constant_i64(0);
+TCGv_i64 tmp = tcg_temp_new_i64();
+
+tcg_gen_add_i64(tmp, a, b);
+tcg_gen_smin_i64(res, tmp, max)

[PATCH v2 25/67] target/arm: Convert FCMEQ, FCMGE, FCMGT, FACGE, FACGT to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/helper.h|   5 +
 target/arm/tcg/a64.decode  |  30 ++
 target/arm/tcg/translate-a64.c | 188 +++--
 target/arm/tcg/vec_helper.c|  30 ++
 4 files changed, 174 insertions(+), 79 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index e021c18517..8d076011c1 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -727,18 +727,23 @@ DEF_HELPER_FLAGS_5(gvec_fabd_s, TCG_CALL_NO_RWG, void, 
ptr, ptr, ptr, ptr, i32)
 
 DEF_HELPER_FLAGS_5(gvec_fceq_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_5(gvec_fceq_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_fceq_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 
 DEF_HELPER_FLAGS_5(gvec_fcge_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_5(gvec_fcge_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_fcge_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 
 DEF_HELPER_FLAGS_5(gvec_fcgt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_5(gvec_fcgt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_fcgt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 
 DEF_HELPER_FLAGS_5(gvec_facge_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 DEF_HELPER_FLAGS_5(gvec_facge_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(gvec_facge_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 
 DEF_HELPER_FLAGS_5(gvec_facgt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 DEF_HELPER_FLAGS_5(gvec_facgt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(gvec_facgt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
 
 DEF_HELPER_FLAGS_5(gvec_fmax_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_5(gvec_fmax_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 11527bb5e5..7fc3277be6 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -713,6 +713,21 @@ FMINNM_s0001 1110 ..1 . 0111 10 . . 
@rrr_hsd
 FMULX_s 0101 1110 010 . 00011 1 . . @rrr_h
 FMULX_s 0101 1110 0.1 . 11011 1 . . @rrr_sd
 
+FCMEQ_s 0101 1110 010 . 00100 1 . . @rrr_h
+FCMEQ_s 0101 1110 0.1 . 11100 1 . . @rrr_sd
+
+FCMGE_s 0111 1110 010 . 00100 1 . . @rrr_h
+FCMGE_s 0111 1110 0.1 . 11100 1 . . @rrr_sd
+
+FCMGT_s 0111 1110 110 . 00100 1 . . @rrr_h
+FCMGT_s 0111 1110 1.1 . 11100 1 . . @rrr_sd
+
+FACGE_s 0111 1110 010 . 00101 1 . . @rrr_h
+FACGE_s 0111 1110 0.1 . 11101 1 . . @rrr_sd
+
+FACGT_s 0111 1110 110 . 00101 1 . . @rrr_h
+FACGT_s 0111 1110 1.1 . 11101 1 . . @rrr_sd
+
 ### Advanced SIMD three same
 
 FADD_v  0.00 1110 010 . 00010 1 . . @qrrr_h
@@ -748,6 +763,21 @@ FMLA_v  0.00 1110 0.1 . 11001 1 . . 
@qrrr_sd
 FMLS_v  0.00 1110 110 . 1 1 . . @qrrr_h
 FMLS_v  0.00 1110 1.1 . 11001 1 . . @qrrr_sd
 
+FCMEQ_v 0.00 1110 010 . 00100 1 . . @qrrr_h
+FCMEQ_v 0.00 1110 0.1 . 11100 1 . . @qrrr_sd
+
+FCMGE_v 0.10 1110 010 . 00100 1 . . @qrrr_h
+FCMGE_v 0.10 1110 0.1 . 11100 1 . . @qrrr_sd
+
+FCMGT_v 0.10 1110 110 . 00100 1 . . @qrrr_h
+FCMGT_v 0.10 1110 1.1 . 11100 1 . . @qrrr_sd
+
+FACGE_v 0.10 1110 010 . 00101 1 . . @qrrr_h
+FACGE_v 0.10 1110 0.1 . 11101 1 . . @qrrr_sd
+
+FACGT_v 0.10 1110 110 . 00101 1 . . @qrrr_h
+FACGT_v 0.10 1110 1.1 . 11101 1 . . @qrrr_sd
+
 ### Advanced SIMD scalar x indexed element
 
 FMUL_si 0101  00 ..  1001 . 0 . .   @rrx_h
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index f84c12378d..75b0c1a005 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -4975,6 +4975,41 @@ static const FPScalar f_scalar_fnmul = {
 };
 TRANS(FNMUL_s, do_fp3_scalar, a, &f_scalar_fnmul)
 
+static const FPScalar f_scalar_fcmeq = {
+gen_helper_advsimd_ceq_f16,
+gen_helper_neon_ceq_f32,
+gen_helper_neon_ceq_f64,
+};
+TRANS(FCMEQ_s, do_fp3_scalar, a, &f_scalar_fcmeq)
+
+static const FPScalar f_scalar_fcmge = {
+gen_helper_advsimd_cge_f16,
+gen_helper_neon_cge_f32,
+gen_helper_neon_cge_f64,
+};
+TRANS(FCMGE_s, do_fp3_scalar, a, &f_scalar_fcmge)
+
+static const FPScalar f_scalar_fcmgt = {
+gen_helper_advsimd_cgt_f16,
+gen_helper_neon_cgt_f32,
+gen_helper_neon_cgt_f64,
+};
+TRANS(FCMGT_s, do_fp3_scalar, a, &f_scalar_fcmgt)
+
+static co

[PATCH v2 18/67] target/arm: Convert FMULX to decodetree

[Richard Henderson]

Convert all forms (scalar, vector, scalar indexed, vector indexed),
which allows us to remove switch table entries elsewhere.

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/helper-a64.h|   8 ++
 target/arm/tcg/a64.decode  |  45 +++
 target/arm/tcg/translate-a64.c | 221 +++--
 target/arm/tcg/vec_helper.c|  39 +++---
 4 files changed, 259 insertions(+), 54 deletions(-)

diff --git a/target/arm/tcg/helper-a64.h b/target/arm/tcg/helper-a64.h
index 0518165399..b79751a717 100644
--- a/target/arm/tcg/helper-a64.h
+++ b/target/arm/tcg/helper-a64.h
@@ -132,3 +132,11 @@ DEF_HELPER_4(cpye, void, env, i32, i32, i32)
 DEF_HELPER_4(cpyfp, void, env, i32, i32, i32)
 DEF_HELPER_4(cpyfm, void, env, i32, i32, i32)
 DEF_HELPER_4(cpyfe, void, env, i32, i32, i32)
+
+DEF_HELPER_FLAGS_5(gvec_fmulx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(gvec_fmulx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+DEF_HELPER_FLAGS_5(gvec_fmulx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, 
i32)
+
+DEF_HELPER_FLAGS_5(gvec_fmulx_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, 
ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_fmulx_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, 
ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_fmulx_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, 
ptr, i32)
diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index d5bfeae7a8..2e0e01be01 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -20,21 +20,44 @@
 #
 
 %rd 0:5
+%esz_sd 22:1 !function=plus_2
+%hl 11:1 21:1
+%hlm11:1 20:2
 
 &r  rn
 &ri rd imm
 &rri_sf rd rn imm sf
 &i  imm
+&rrr_e  rd rn rm esz
+&rrx_e  rd rn rm idx esz
 &qrr_e  q rd rn esz
 &qrrr_e q rd rn rm esz
+&qrrx_e q rd rn rm idx esz
 &q_eq rd rn rm ra esz
 
+@rrr_h   ... rm:5 .. rn:5 rd:5  &rrr_e esz=1
+@rrr_sd  ... rm:5 .. rn:5 rd:5  &rrr_e esz=%esz_sd
+
+@rrx_h   .. .. rm:4  . . rn:5 rd:5  &rrx_e esz=1 idx=%hlm
+@rrx_s   .. . rm:5   . . rn:5 rd:5  &rrx_e esz=2 idx=%hl
+@rrx_d   .. . rm:5   idx:1 . rn:5 rd:5  &rrx_e esz=3
+
 @rr_q1e0  .. rn:5 rd:5  &qrr_e q=1 esz=0
 @r2r_q1e0     .. rm:5 rd:5  &qrrr_e rn=%rd q=1 
esz=0
 @rrr_q1e0    ... rm:5 .. rn:5 rd:5  &qrrr_e q=1 esz=0
 @rrr_q1e3    ... rm:5 .. rn:5 rd:5  &qrrr_e q=1 esz=3
 @_q1e3   ... rm:5 . ra:5 rn:5 rd:5  &q_e q=1 esz=3
 
+@qrrr_h . q:1 .. ... rm:5 .. rn:5 rd:5  &qrrr_e esz=1
+@qrrr_sd. q:1 .. ... rm:5 .. rn:5 rd:5  &qrrr_e esz=%esz_sd
+
+@qrrx_h . q:1 ..  .. .. rm:4  . . rn:5 rd:5 \
+&qrrx_e esz=1 idx=%hlm
+@qrrx_s . q:1 ..  .. . rm:5   . . rn:5 rd:5 \
+&qrrx_e esz=2 idx=%hl
+@qrrx_d . q:1 ..  .. . rm:5   idx:1 . rn:5 rd:5 \
+&qrrx_e esz=3
+
 ### Data Processing - Immediate
 
 # PC-rel addressing
@@ -671,3 +694,25 @@ INS_general 0 1   00 1110 000 imm:5 0 0011 1 rn:5 rd:5
 SMOV0 q:1 00 1110 000 imm:5 0 0101 1 rn:5 rd:5
 UMOV0 q:1 00 1110 000 imm:5 0 0111 1 rn:5 rd:5
 INS_element 0 1   10 1110 000 di:5  0 si:4 1 rn:5 rd:5
+
+### Advanced SIMD scalar three same
+
+FMULX_s 0101 1110 010 . 00011 1 . . @rrr_h
+FMULX_s 0101 1110 0.1 . 11011 1 . . @rrr_sd
+
+### Advanced SIMD three same
+
+FMULX_v 0.00 1110 010 . 00011 1 . . @qrrr_h
+FMULX_v 0.00 1110 0.1 . 11011 1 . . @qrrr_sd
+
+### Advanced SIMD scalar x indexed element
+
+FMULX_si0111  00 ..  1001 . 0 . .   @rrx_h
+FMULX_si0111  10 . . 1001 . 0 . .   @rrx_s
+FMULX_si0111  11 0 . 1001 . 0 . .   @rrx_d
+
+### Advanced SIMD vector x indexed element
+
+FMULX_vi0.10  00 ..  1001 . 0 . .   @qrrx_h
+FMULX_vi0.10  10 . . 1001 . 0 . .   @qrrx_s
+FMULX_vi0.10  11 0 . 1001 . 0 . .   @qrrx_d
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 1a12bf22fd..8cbe6cd70f 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -4841,6 +4841,178 @@ static bool trans_INS_element(DisasContext *s, 
arg_INS_element *a)
 return true;
 }
 
+/*
+ * Advanced SIMD three same
+ */
+
+typedef struct FPScalar {
+void (*gen_h)(TCGv_i32, TCGv_i32, TCGv_i32, TCGv_ptr);
+void (*gen_s)(TCGv_i32, TCGv_i32, TCGv_i32, TCGv_ptr);
+void (*gen_d)(TCGv_i64, TCGv_i64, TCGv_i64, TCGv_ptr);
+} FPScalar;
+
+static bool do_fp3_scalar(DisasContext *s, arg_rrr_e *a, const FPScalar *f)
+{
+switch (a->esz) {
+

[PATCH v2 13/67] target/arm: Convert Cryptographic 2-register SHA512 to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |  5 
 target/arm/tcg/translate-a64.c | 50 ++
 2 files changed, 8 insertions(+), 47 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index c342c27608..5a46205751 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -631,3 +631,8 @@ RAX11100 1110 011 . 100011 . .  
@rrr_q1e3
 SM3PARTW1   1100 1110 011 . 11 . .  @rrr_q1e0
 SM3PARTW2   1100 1110 011 . 110001 . .  @rrr_q1e0
 SM4EKEY 1100 1110 011 . 110010 . .  @rrr_q1e0
+
+### Cryptographic two-register SHA512
+
+SHA512SU0   1100 1110 110 0 10 . .  @rr_q1e0
+SM4E1100 1110 110 0 11 . .  @r2r_q1e0
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 77b24cd52e..eed0abe912 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -4629,6 +4629,9 @@ TRANS_FEAT(SM3PARTW1, aa64_sm3, do_gvec_op3_ool, a, 0, 
gen_helper_crypto_sm3part
 TRANS_FEAT(SM3PARTW2, aa64_sm3, do_gvec_op3_ool, a, 0, 
gen_helper_crypto_sm3partw2)
 TRANS_FEAT(SM4EKEY, aa64_sm4, do_gvec_op3_ool, a, 0, gen_helper_crypto_sm4ekey)
 
+TRANS_FEAT(SHA512SU0, aa64_sha512, do_gvec_op2_ool, a, 0, 
gen_helper_crypto_sha512su0)
+TRANS_FEAT(SM4E, aa64_sm4, do_gvec_op3_ool, a, 0, gen_helper_crypto_sm4e)
+
 
 /* Shift a TCGv src by TCGv shift_amount, put result in dst.
  * Note that it is the caller's responsibility to ensure that the
@@ -13530,52 +13533,6 @@ static void disas_simd_indexed(DisasContext *s, 
uint32_t insn)
 }
 }
 
-/* Crypto two-reg SHA512
- *  31 12  11  10  95 40
- * +-++--+--+
- * | 1 1 0 0 1 1 1 0 1 1 0 0 0 0 0 0 1 0 0 0 | opcode |  Rn  |  Rd  |
- * +-++--+--+
- */
-static void disas_crypto_two_reg_sha512(DisasContext *s, uint32_t insn)
-{
-int opcode = extract32(insn, 10, 2);
-int rn = extract32(insn, 5, 5);
-int rd = extract32(insn, 0, 5);
-bool feature;
-
-switch (opcode) {
-case 0: /* SHA512SU0 */
-feature = dc_isar_feature(aa64_sha512, s);
-break;
-case 1: /* SM4E */
-feature = dc_isar_feature(aa64_sm4, s);
-break;
-default:
-unallocated_encoding(s);
-return;
-}
-
-if (!feature) {
-unallocated_encoding(s);
-return;
-}
-
-if (!fp_access_check(s)) {
-return;
-}
-
-switch (opcode) {
-case 0: /* SHA512SU0 */
-gen_gvec_op2_ool(s, true, rd, rn, 0, gen_helper_crypto_sha512su0);
-break;
-case 1: /* SM4E */
-gen_gvec_op3_ool(s, true, rd, rd, rn, 0, gen_helper_crypto_sm4e);
-break;
-default:
-g_assert_not_reached();
-}
-}
-
 /* Crypto four-register
  *  31   23 22 21 20  16 15  14  10 95 40
  * +---+-+--+---+--+--+--+
@@ -13750,7 +13707,6 @@ static const AArch64DecodeTable data_proc_simd[] = {
 { 0x5e000400, 0xdfe08400, disas_simd_scalar_copy },
 { 0x5f00, 0xdf000400, disas_simd_indexed }, /* scalar indexed */
 { 0x5f000400, 0xdf800400, disas_simd_scalar_shift_imm },
-{ 0xcec08000, 0xf000, disas_crypto_two_reg_sha512 },
 { 0xce00, 0xff808000, disas_crypto_four_reg },
 { 0xce80, 0xffe0, disas_crypto_xar },
 { 0xce408000, 0xffe0c000, disas_crypto_three_reg_imm2 },
-- 
2.34.1

[PATCH v2 14/67] target/arm: Convert Cryptographic 4-register to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a64.decode  |   8 ++
 target/arm/tcg/translate-a64.c | 132 +++--
 2 files changed, 51 insertions(+), 89 deletions(-)

diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 5a46205751..ef6902e86a 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -27,11 +27,13 @@
 &i  imm
 &qrr_e  q rd rn esz
 &qrrr_e q rd rn rm esz
+&q_eq rd rn rm ra esz
 
 @rr_q1e0  .. rn:5 rd:5  &qrr_e q=1 esz=0
 @r2r_q1e0     .. rm:5 rd:5  &qrrr_e rn=%rd q=1 
esz=0
 @rrr_q1e0    ... rm:5 .. rn:5 rd:5  &qrrr_e q=1 esz=0
 @rrr_q1e3    ... rm:5 .. rn:5 rd:5  &qrrr_e q=1 esz=3
+@_q1e3   ... rm:5 . ra:5 rn:5 rd:5  &q_e q=1 esz=3
 
 ### Data Processing - Immediate
 
@@ -636,3 +638,9 @@ SM4EKEY 1100 1110 011 . 110010 . .  
@rrr_q1e0
 
 SHA512SU0   1100 1110 110 0 10 . .  @rr_q1e0
 SM4E1100 1110 110 0 11 . .  @r2r_q1e0
+
+### Cryptographic four-register
+
+EOR31100 1110 000 . 0 . . . @_q1e3
+BCAX1100 1110 001 . 0 . . . @_q1e3
+SM3SS1  1100 1110 010 . 0 . . . @_q1e3
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index eed0abe912..2951e7eb59 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -1352,6 +1352,17 @@ static bool do_gvec_fn3(DisasContext *s, arg_qrrr_e *a, 
GVecGen3Fn *fn)
 return true;
 }
 
+static bool do_gvec_fn4(DisasContext *s, arg_q_e *a, GVecGen4Fn *fn)
+{
+if (!a->q && a->esz == MO_64) {
+return false;
+}
+if (fp_access_check(s)) {
+gen_gvec_fn4(s, a->q, a->rd, a->rn, a->rm, a->ra, fn, a->esz);
+}
+return true;
+}
+
 /*
  * This utility function is for doing register extension with an
  * optional shift. You will likely want to pass a temporary for the
@@ -4632,6 +4643,38 @@ TRANS_FEAT(SM4EKEY, aa64_sm4, do_gvec_op3_ool, a, 0, 
gen_helper_crypto_sm4ekey)
 TRANS_FEAT(SHA512SU0, aa64_sha512, do_gvec_op2_ool, a, 0, 
gen_helper_crypto_sha512su0)
 TRANS_FEAT(SM4E, aa64_sm4, do_gvec_op3_ool, a, 0, gen_helper_crypto_sm4e)
 
+TRANS_FEAT(EOR3, aa64_sha3, do_gvec_fn4, a, gen_gvec_eor3)
+TRANS_FEAT(BCAX, aa64_sha3, do_gvec_fn4, a, gen_gvec_bcax)
+
+static bool trans_SM3SS1(DisasContext *s, arg_SM3SS1 *a)
+{
+if (!dc_isar_feature(aa64_sm3, s)) {
+return false;
+}
+if (fp_access_check(s)) {
+TCGv_i32 tcg_op1 = tcg_temp_new_i32();
+TCGv_i32 tcg_op2 = tcg_temp_new_i32();
+TCGv_i32 tcg_op3 = tcg_temp_new_i32();
+TCGv_i32 tcg_res = tcg_temp_new_i32();
+unsigned vsz, dofs;
+
+read_vec_element_i32(s, tcg_op1, a->rn, 3, MO_32);
+read_vec_element_i32(s, tcg_op2, a->rm, 3, MO_32);
+read_vec_element_i32(s, tcg_op3, a->ra, 3, MO_32);
+
+tcg_gen_rotri_i32(tcg_res, tcg_op1, 20);
+tcg_gen_add_i32(tcg_res, tcg_res, tcg_op2);
+tcg_gen_add_i32(tcg_res, tcg_res, tcg_op3);
+tcg_gen_rotri_i32(tcg_res, tcg_res, 25);
+
+/* Clear the whole register first, then store bits [127:96]. */
+vsz = vec_full_reg_size(s);
+dofs = vec_full_reg_offset(s, a->rd);
+tcg_gen_gvec_dup_imm(MO_64, dofs, vsz, vsz, 0);
+write_vec_element_i32(s, tcg_res, a->rd, 3, MO_32);
+}
+return true;
+}
 
 /* Shift a TCGv src by TCGv shift_amount, put result in dst.
  * Note that it is the caller's responsibility to ensure that the
@@ -13533,94 +13576,6 @@ static void disas_simd_indexed(DisasContext *s, 
uint32_t insn)
 }
 }
 
-/* Crypto four-register
- *  31   23 22 21 20  16 15  14  10 95 40
- * +---+-+--+---+--+--+--+
- * | 1 1 0 0 1 1 1 0 0 | Op0 |  Rm  | 0 |  Ra  |  Rn  |  Rd  |
- * +---+-+--+---+--+--+--+
- */
-static void disas_crypto_four_reg(DisasContext *s, uint32_t insn)
-{
-int op0 = extract32(insn, 21, 2);
-int rm = extract32(insn, 16, 5);
-int ra = extract32(insn, 10, 5);
-int rn = extract32(insn, 5, 5);
-int rd = extract32(insn, 0, 5);
-bool feature;
-
-switch (op0) {
-case 0: /* EOR3 */
-case 1: /* BCAX */
-feature = dc_isar_feature(aa64_sha3, s);
-break;
-case 2: /* SM3SS1 */
-feature = dc_isar_feature(aa64_sm3, s);
-break;
-default:
-unallocated_encoding(s);
-return;
-}
-
-if (!feature) {
-unallocated_encoding(s);
-return;
-}
-
-if (!fp_access_check(s)) {
-return;
-}
-
-if (op0 < 2) {
-TCGv_i64 tcg_op1, tcg_op2, tcg_op3, tcg_res[2];
-int pass;
-
-tcg_op1 = tcg_temp_new_i64();
-tcg

[PATCH v2 24/67] target/arm: Convert FMLA, FMLS to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/helper.h|   2 +
 target/arm/tcg/a64.decode  |  22 +++
 target/arm/tcg/translate-a64.c | 241 +
 target/arm/tcg/vec_helper.c|  14 ++
 4 files changed, 163 insertions(+), 116 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index 0fd01c9c52..e021c18517 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -770,9 +770,11 @@ DEF_HELPER_FLAGS_5(gvec_fmls_s, TCG_CALL_NO_RWG, void, 
ptr, ptr, ptr, ptr, i32)
 
 DEF_HELPER_FLAGS_5(gvec_vfma_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_5(gvec_vfma_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_vfma_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 
 DEF_HELPER_FLAGS_5(gvec_vfms_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_5(gvec_vfms_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_vfms_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 
 DEF_HELPER_FLAGS_5(gvec_ftsmul_h, TCG_CALL_NO_RWG,
void, ptr, ptr, ptr, ptr, i32)
diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index cde4b86303..11527bb5e5 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -742,12 +742,26 @@ FMINNM_v0.00 1110 1.1 . 11000 1 . . 
@qrrr_sd
 FMULX_v 0.00 1110 010 . 00011 1 . . @qrrr_h
 FMULX_v 0.00 1110 0.1 . 11011 1 . . @qrrr_sd
 
+FMLA_v  0.00 1110 010 . 1 1 . . @qrrr_h
+FMLA_v  0.00 1110 0.1 . 11001 1 . . @qrrr_sd
+
+FMLS_v  0.00 1110 110 . 1 1 . . @qrrr_h
+FMLS_v  0.00 1110 1.1 . 11001 1 . . @qrrr_sd
+
 ### Advanced SIMD scalar x indexed element
 
 FMUL_si 0101  00 ..  1001 . 0 . .   @rrx_h
 FMUL_si 0101  10 . . 1001 . 0 . .   @rrx_s
 FMUL_si 0101  11 0 . 1001 . 0 . .   @rrx_d
 
+FMLA_si 0101  00 ..  0001 . 0 . .   @rrx_h
+FMLA_si 0101  10 ..  0001 . 0 . .   @rrx_s
+FMLA_si 0101  11 0.  0001 . 0 . .   @rrx_d
+
+FMLS_si 0101  00 ..  0101 . 0 . .   @rrx_h
+FMLS_si 0101  10 ..  0101 . 0 . .   @rrx_s
+FMLS_si 0101  11 0.  0101 . 0 . .   @rrx_d
+
 FMULX_si0111  00 ..  1001 . 0 . .   @rrx_h
 FMULX_si0111  10 . . 1001 . 0 . .   @rrx_s
 FMULX_si0111  11 0 . 1001 . 0 . .   @rrx_d
@@ -758,6 +772,14 @@ FMUL_vi 0.00  00 ..  1001 . 0 . .  
 @qrrx_h
 FMUL_vi 0.00  10 . . 1001 . 0 . .   @qrrx_s
 FMUL_vi 0.00  11 0 . 1001 . 0 . .   @qrrx_d
 
+FMLA_vi 0.00  00 ..  0001 . 0 . .   @qrrx_h
+FMLA_vi 0.00  10 . . 0001 . 0 . .   @qrrx_s
+FMLA_vi 0.00  11 0 . 0001 . 0 . .   @qrrx_d
+
+FMLS_vi 0.00  00 ..  0101 . 0 . .   @qrrx_h
+FMLS_vi 0.00  10 . . 0101 . 0 . .   @qrrx_s
+FMLS_vi 0.00  11 0 . 0101 . 0 . .   @qrrx_d
+
 FMULX_vi0.10  00 ..  1001 . 0 . .   @qrrx_h
 FMULX_vi0.10  10 . . 1001 . 0 . .   @qrrx_s
 FMULX_vi0.10  11 0 . 1001 . 0 . .   @qrrx_d
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 5ba30ba7c8..f84c12378d 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5066,6 +5066,20 @@ static gen_helper_gvec_3_ptr * const f_vector_fmulx[3] = 
{
 };
 TRANS(FMULX_v, do_fp3_vector, a, f_vector_fmulx)
 
+static gen_helper_gvec_3_ptr * const f_vector_fmla[3] = {
+gen_helper_gvec_vfma_h,
+gen_helper_gvec_vfma_s,
+gen_helper_gvec_vfma_d,
+};
+TRANS(FMLA_v, do_fp3_vector, a, f_vector_fmla)
+
+static gen_helper_gvec_3_ptr * const f_vector_fmls[3] = {
+gen_helper_gvec_vfms_h,
+gen_helper_gvec_vfms_s,
+gen_helper_gvec_vfms_d,
+};
+TRANS(FMLS_v, do_fp3_vector, a, f_vector_fmls)
+
 /*
  * Advanced SIMD scalar/vector x indexed element
  */
@@ -5115,6 +5129,64 @@ static bool do_fp3_scalar_idx(DisasContext *s, arg_rrx_e 
*a, const FPScalar *f)
 TRANS(FMUL_si, do_fp3_scalar_idx, a, &f_scalar_fmul)
 TRANS(FMULX_si, do_fp3_scalar_idx, a, &f_scalar_fmulx)
 
+static bool do_fmla_scalar_idx(DisasContext *s, arg_rrx_e *a, bool neg)
+{
+switch (a->esz) {
+case MO_64:
+if (fp_access_check(s)) {
+TCGv_i64 t0 = read_fp_dreg(s, a->rd);
+TCGv_i64 t1 = read_fp_dreg(s, a->rn);
+TCGv_i64 t2 = tcg_temp_new_i64();
+
+read_vec_element(s, t2, a->rm, a->idx, MO_64);
+if (neg) {
+gen_vfp_negd(t1, t1);
+}
+gen_helper_v

[PATCH v2 02/67] target/arm: Use PLD, PLDW, PLI not NOP for t32

[Richard Henderson]

This fixes a bug in that neither PLI nor PLDW are present in ARMv6T2,
but are introduced with ARMv7 and ARMv7MP respectively.
For clarity, do not use NOP for PLD.

Note that there is no PLDW (literal) -- bit 5 of the first word
is not decoded, and is PLD (literal).  Confirmed on neoverse-n1
host which does *not* trap on the (0) bit in the decode.

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/t32.decode  | 25 -
 target/arm/tcg/translate.c |  4 ++--
 2 files changed, 14 insertions(+), 15 deletions(-)

diff --git a/target/arm/tcg/t32.decode b/target/arm/tcg/t32.decode
index f21ad0167a..d327178829 100644
--- a/target/arm/tcg/t32.decode
+++ b/target/arm/tcg/t32.decode
@@ -458,41 +458,41 @@ STR_ri    1000 1100   
@ldst_ri_pos
 # Note that Load, unsigned (literal) overlaps all other load encodings.
 {
   {
-NOP   1000 -001   # PLD
+PLD   1000 -001   # (literal)
 LDRB_ri   1000 .001   @ldst_ri_lit
   }
   {
-NOP   1000 1001   # PLD
+PLD   1000 1001   # (immediate T1)
 LDRB_ri   1000 1001   @ldst_ri_pos
   }
   LDRB_ri 1000 0001   1..1    @ldst_ri_idx
   {
-NOP   1000 0001   1100    # PLD
+PLD   1000 0001   1100    # (immediate T2)
 LDRB_ri   1000 0001   1100    @ldst_ri_neg
   }
   LDRBT_ri    1000 0001   1110    @ldst_ri_unp
   {
-NOP   1000 0001   00 --   # PLD
+PLD   1000 0001   00 --   # (register)
 LDRB_rr   1000 0001   00 ..   @ldst_rr
   }
 }
 {
   {
-NOP   1000 -011   # PLD
+PLD   1000 -011   # (literal)
 LDRH_ri   1000 .011   @ldst_ri_lit
   }
   {
-NOP   1000 1011   # PLDW
+PLDW  1000 1011   # (immediate T1)
 LDRH_ri   1000 1011   @ldst_ri_pos
   }
   LDRH_ri 1000 0011   1..1    @ldst_ri_idx
   {
-NOP   1000 0011   1100    # PLDW
+PLDW  1000 0011   1100    # (immediate T2)
 LDRH_ri   1000 0011   1100    @ldst_ri_neg
   }
   LDRHT_ri    1000 0011   1110    @ldst_ri_unp
   {
-NOP   1000 0011   00 --   # PLDW
+PLDW  1000 0011   00 --   # (register)
 LDRH_rr   1000 0011   00 ..   @ldst_rr
   }
 }
@@ -504,24 +504,23 @@ STR_ri    1000 1100   
@ldst_ri_pos
   LDRT_ri 1000 0101   1110    @ldst_ri_unp
   LDR_rr  1000 0101   00 ..   @ldst_rr
 }
-# NOPs here are PLI.
 {
   {
-NOP   1001 -001   
+PLI   1001 -001   # (literal T3)
 LDRSB_ri  1001 .001   @ldst_ri_lit
   }
   {
-NOP   1001 1001   
+PLI   1001 1001   # (immediate T1)
 LDRSB_ri  1001 1001   @ldst_ri_pos
   }
   LDRSB_ri    1001 0001   1..1    @ldst_ri_idx
   {
-NOP   1001 0001   1100 
+PLI   1001 0001   1100    # (immediate T2)
 LDRSB_ri  1001 0001   1100    @ldst_ri_neg
   }
   LDRSBT_ri   1001 0001   1110    @ldst_ri_unp
   {
-NOP   1001 0001   00 -- 
+PLI   1001 0001   00 --   # (register)
 LDRSB_rr  1001 0001   00 ..   @ldst_rr
   }
 }
diff --git a/target/arm/tcg/translate.c b/target/arm/tcg/translate.c
index d605e10f11..187eacffd9 100644
--- a/target/arm/tcg/translate.c
+++ b/target/arm/tcg/translate.c
@@ -8765,12 +8765,12 @@ static bool trans_PLD(DisasContext *s, arg_PLD *a)
 return ENABLE_ARCH_5TE;
 }
 
-static bool trans_PLDW(DisasContext *s, arg_PLD *a)
+static bool trans_PLDW(DisasContext *s, arg_PLDW *a)
 {
 return arm_dc_feature(s, ARM_FEATURE_V7MP);
 }
 
-static bool trans_PLI(DisasContext *s, arg_PLD *a)
+static bool trans_PLI(DisasContext *s, arg_PLI *a)
 {
 return ENABLE_ARCH_7;
 }
-- 
2.34.1

[PATCH v2 26/67] target/arm: Convert FABD to decodetree

[Richard Henderson]

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/helper.h|  1 +
 target/arm/tcg/a64.decode  |  6 
 target/arm/tcg/translate-a64.c | 60 ++
 target/arm/tcg/vec_helper.c|  6 
 4 files changed, 53 insertions(+), 20 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index 8d076011c1..ff6e3094f4 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -724,6 +724,7 @@ DEF_HELPER_FLAGS_5(gvec_fmul_d, TCG_CALL_NO_RWG, void, ptr, 
ptr, ptr, ptr, i32)
 
 DEF_HELPER_FLAGS_5(gvec_fabd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_5(gvec_fabd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_5(gvec_fabd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 
 DEF_HELPER_FLAGS_5(gvec_fceq_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_5(gvec_fceq_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32)
diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 7fc3277be6..a852b5f06f 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -728,6 +728,9 @@ FACGE_s 0111 1110 0.1 . 11101 1 . . 
@rrr_sd
 FACGT_s 0111 1110 110 . 00101 1 . . @rrr_h
 FACGT_s 0111 1110 1.1 . 11101 1 . . @rrr_sd
 
+FABD_s  0111 1110 110 . 00010 1 . . @rrr_h
+FABD_s  0111 1110 1.1 . 11010 1 . . @rrr_sd
+
 ### Advanced SIMD three same
 
 FADD_v  0.00 1110 010 . 00010 1 . . @qrrr_h
@@ -778,6 +781,9 @@ FACGE_v 0.10 1110 0.1 . 11101 1 . . 
@qrrr_sd
 FACGT_v 0.10 1110 110 . 00101 1 . . @qrrr_h
 FACGT_v 0.10 1110 1.1 . 11101 1 . . @qrrr_sd
 
+FABD_v  0.10 1110 110 . 00010 1 . . @qrrr_h
+FABD_v  0.10 1110 1.1 . 11010 1 . . @qrrr_sd
+
 ### Advanced SIMD scalar x indexed element
 
 FMUL_si 0101  00 ..  1001 . 0 . .   @rrx_h
diff --git a/target/arm/tcg/translate-a64.c b/target/arm/tcg/translate-a64.c
index 75b0c1a005..633384d2a5 100644
--- a/target/arm/tcg/translate-a64.c
+++ b/target/arm/tcg/translate-a64.c
@@ -5010,6 +5010,31 @@ static const FPScalar f_scalar_facgt = {
 };
 TRANS(FACGT_s, do_fp3_scalar, a, &f_scalar_facgt)
 
+static void gen_fabd_h(TCGv_i32 d, TCGv_i32 n, TCGv_i32 m, TCGv_ptr s)
+{
+gen_helper_vfp_subh(d, n, m, s);
+gen_vfp_absh(d, d);
+}
+
+static void gen_fabd_s(TCGv_i32 d, TCGv_i32 n, TCGv_i32 m, TCGv_ptr s)
+{
+gen_helper_vfp_subs(d, n, m, s);
+gen_vfp_abss(d, d);
+}
+
+static void gen_fabd_d(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_ptr s)
+{
+gen_helper_vfp_subd(d, n, m, s);
+gen_vfp_absd(d, d);
+}
+
+static const FPScalar f_scalar_fabd = {
+gen_fabd_h,
+gen_fabd_s,
+gen_fabd_d,
+};
+TRANS(FABD_s, do_fp3_scalar, a, &f_scalar_fabd)
+
 static bool do_fp3_vector(DisasContext *s, arg_qrrr_e *a,
   gen_helper_gvec_3_ptr * const fns[3])
 {
@@ -5150,6 +5175,13 @@ static gen_helper_gvec_3_ptr * const f_vector_facgt[3] = 
{
 };
 TRANS(FACGT_v, do_fp3_vector, a, f_vector_facgt)
 
+static gen_helper_gvec_3_ptr * const f_vector_fabd[3] = {
+gen_helper_gvec_fabd_h,
+gen_helper_gvec_fabd_s,
+gen_helper_gvec_fabd_d,
+};
+TRANS(FABD_v, do_fp3_vector, a, f_vector_fabd)
+
 /*
  * Advanced SIMD scalar/vector x indexed element
  */
@@ -9303,10 +9335,6 @@ static void handle_3same_float(DisasContext *s, int 
size, int elements,
 case 0x3f: /* FRSQRTS */
 gen_helper_rsqrtsf_f64(tcg_res, tcg_op1, tcg_op2, fpst);
 break;
-case 0x7a: /* FABD */
-gen_helper_vfp_subd(tcg_res, tcg_op1, tcg_op2, fpst);
-gen_vfp_absd(tcg_res, tcg_res);
-break;
 default:
 case 0x18: /* FMAXNM */
 case 0x19: /* FMLA */
@@ -9322,6 +9350,7 @@ static void handle_3same_float(DisasContext *s, int size, 
int elements,
 case 0x5c: /* FCMGE */
 case 0x5d: /* FACGE */
 case 0x5f: /* FDIV */
+case 0x7a: /* FABD */
 case 0x7c: /* FCMGT */
 case 0x7d: /* FACGT */
 g_assert_not_reached();
@@ -9344,10 +9373,6 @@ static void handle_3same_float(DisasContext *s, int 
size, int elements,
 case 0x3f: /* FRSQRTS */
 gen_helper_rsqrtsf_f32(tcg_res, tcg_op1, tcg_op2, fpst);
 break;
-case 0x7a: /* FABD */
-gen_helper_vfp_subs(tcg_res, tcg_op1, tcg_op2, fpst);
-gen_vfp_abss(tcg_res, tcg_res);
-break;
 default:
 case 0x18: /* FMAXNM */
 case 0x19: /* FMLA */
@@ -9363,6 +9388,7 @@ static void handle_3same_float(DisasContext *s, int size, 
int elements,
 case 0x5c: /* FCMGE */
 case 0x5d: /* FACGE */
 case 0

[PATCH v2 03/67] target/arm: Reject incorrect operands to PLD, PLDW, PLI

[Richard Henderson]

For all, rm == 15 is invalid.
Prior to v8, thumb with rm == 13 is invalid.
For PLDW, rn == 15 is invalid.

Fixes a RISU mismatch for the HINTSPACE pattern in t32.risu
compared to a neoverse-n1 host.

Signed-off-by: Richard Henderson 
---
 target/arm/tcg/a32-uncond.decode |  8 +++--
 target/arm/tcg/t32.decode|  7 ++--
 target/arm/tcg/translate.c   | 57 
 3 files changed, 66 insertions(+), 6 deletions(-)

diff --git a/target/arm/tcg/a32-uncond.decode b/target/arm/tcg/a32-uncond.decode
index 2339de2e94..e1b1780d37 100644
--- a/target/arm/tcg/a32-uncond.decode
+++ b/target/arm/tcg/a32-uncond.decode
@@ -24,7 +24,9 @@
 
 &empty   !extern
 &i   !extern imm
+&r   !extern rm
 &setend  E
+&nm  rn rm
 
 # Branch with Link and Exchange
 
@@ -61,9 +63,9 @@ PLD   0101 -101     # 
(imm, lit) 5te
 PLDW  0101 -001     # (imm, lit) 7mp
 PLI   0100 -101     # (imm, lit) 7
 
-PLD   0111 -101   - -- 0    # (register) 5te
-PLDW  0111 -001   - -- 0    # (register) 7mp
-PLI   0110 -101   - -- 0    # (register) 7
+PLD_rr    0111 -101   - -- 0 rm:4   &r
+PLDW_rr   0111 -001 rn:4  - -- 0 rm:4   &nm
+PLI_rr    0110 -101   - -- 0 rm:4   &r
 
 # Unallocated memory hints
 #
diff --git a/target/arm/tcg/t32.decode b/target/arm/tcg/t32.decode
index d327178829..1ec12442a4 100644
--- a/target/arm/tcg/t32.decode
+++ b/target/arm/tcg/t32.decode
@@ -28,6 +28,7 @@
 &rrr_rot !extern rd rn rm rot
 &rrr !extern rd rn rm
 &rr  !extern rd rm
+&nm  !extern rn rm
 &ri  !extern rd imm
 &r   !extern rm
 &i   !extern imm
@@ -472,7 +473,7 @@ STR_ri    1000 1100     
  @ldst_ri_pos
   }
   LDRBT_ri    1000 0001   1110    @ldst_ri_unp
   {
-PLD   1000 0001   00 --   # (register)
+PLD_rr    1000 0001   00 -- rm:4  &r
 LDRB_rr   1000 0001   00 ..   @ldst_rr
   }
 }
@@ -492,7 +493,7 @@ STR_ri    1000 1100     
  @ldst_ri_pos
   }
   LDRHT_ri    1000 0011   1110    @ldst_ri_unp
   {
-PLDW  1000 0011   00 --   # (register)
+PLDW_rr   1000 0011 rn:4  00 -- rm:4  &nm
 LDRH_rr   1000 0011   00 ..   @ldst_rr
   }
 }
@@ -520,7 +521,7 @@ STR_ri    1000 1100     
  @ldst_ri_pos
   }
   LDRSBT_ri   1001 0001   1110    @ldst_ri_unp
   {
-PLI   1001 0001   00 --   # (register)
+PLI_rr    1001 0001   00 -- rm:4  &r
 LDRSB_rr  1001 0001   00 ..   @ldst_rr
   }
 }
diff --git a/target/arm/tcg/translate.c b/target/arm/tcg/translate.c
index 187eacffd9..7c09153b6e 100644
--- a/target/arm/tcg/translate.c
+++ b/target/arm/tcg/translate.c
@@ -8775,6 +8775,63 @@ static bool trans_PLI(DisasContext *s, arg_PLI *a)
 return ENABLE_ARCH_7;
 }
 
+static bool prefetch_check_m(DisasContext *s, int rm)
+{
+switch (rm) {
+case 13:
+/* SP allowed in v8 or with A1 encoding; rejected with T1. */
+return ENABLE_ARCH_8 || !s->thumb;
+case 15:
+/* PC always rejected. */
+return false;
+default:
+return true;
+}
+}
+
+static bool trans_PLD_rr(DisasContext *s, arg_PLD_rr *a)
+{
+if (!ENABLE_ARCH_5TE) {
+return false;
+}
+/* We cannot return false, because that leads to LDRB for thumb. */
+if (!prefetch_check_m(s, a->rm)) {
+unallocated_encoding(s);
+}
+return true;
+}
+
+static bool trans_PLDW_rr(DisasContext *s, arg_PLDW_rr *a)
+{
+if (!arm_dc_feature(s, ARM_FEATURE_V7MP)) {
+return false;
+}
+/*
+ * For A1, rn == 15 is UNPREDICTABLE.
+ * For T1, rn == 15 is PLD (literal).
+ */
+if (a->rn == 15) {
+return false;
+}
+/* We cannot return false, because that leads to LDRH for thumb. */
+if (!prefetch_check_m(s, a->rm)) {
+unallocated_encoding(s);
+}
+return true;
+}
+
+static bool trans_PLI_rr(DisasContext *s, arg_PLI_rr *a)
+{
+if (!ENABLE_ARCH_7) {
+return false;
+}
+/* We cannot return false, because that leads to LDRSB for thumb. */
+if (!prefetch_check_m(s, a->rm)) {
+unallocated_encoding(s);
+}
+return true;
+}
+
 /*
  * If-then
  */
-- 
2.34.1

[PATCH v2 33/67] target/arm: Convert SMAXP, SMINP, UMAXP, UMINP to decodetree

[Richard Henderson]

These are the last instructions within handle_simd_3same_pair
so remove it.

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/helper.h|  16 +
 target/arm/tcg/translate.h |   8 +++
 target/arm/tcg/a64.decode  |   4 ++
 target/arm/tcg/gengvec.c   |  48 +
 target/arm/tcg/translate-a64.c | 119 +
 target/arm/tcg/vec_helper.c|  16 +
 6 files changed, 109 insertions(+), 102 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index 51ed49aa50..f830531dd3 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -1064,6 +1064,22 @@ DEF_HELPER_FLAGS_4(gvec_addp_h, TCG_CALL_NO_RWG, void, 
ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_4(gvec_addp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_4(gvec_addp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
 
+DEF_HELPER_FLAGS_4(gvec_smaxp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_smaxp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_smaxp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+
+DEF_HELPER_FLAGS_4(gvec_sminp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_sminp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_sminp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+
+DEF_HELPER_FLAGS_4(gvec_umaxp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_umaxp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_umaxp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+
+DEF_HELPER_FLAGS_4(gvec_uminp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_uminp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_uminp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+
 #ifdef TARGET_AARCH64
 #include "tcg/helper-a64.h"
 #include "tcg/helper-sve.h"
diff --git a/target/arm/tcg/translate.h b/target/arm/tcg/translate.h
index 04771f483b..3abdbedfe5 100644
--- a/target/arm/tcg/translate.h
+++ b/target/arm/tcg/translate.h
@@ -516,6 +516,14 @@ void gen_gvec_uaba(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 
 void gen_gvec_addp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_smaxp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_sminp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_umaxp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_uminp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
 
 /*
  * Forward to the isar_feature_* tests given a DisasContext pointer.
diff --git a/target/arm/tcg/a64.decode b/target/arm/tcg/a64.decode
index 84f5bcc0e0..22dfe8568d 100644
--- a/target/arm/tcg/a64.decode
+++ b/target/arm/tcg/a64.decode
@@ -837,6 +837,10 @@ FMINNMP_v   0.10 1110 110 . 0 1 . . 
@qrrr_h
 FMINNMP_v   0.10 1110 1.1 . 11000 1 . . @qrrr_sd
 
 ADDP_v  0.00 1110 ..1 . 10111 1 . . @qrrr_e
+SMAXP_v 0.00 1110 ..1 . 10100 1 . . @qrrr_e
+SMINP_v 0.00 1110 ..1 . 10101 1 . . @qrrr_e
+UMAXP_v 0.10 1110 ..1 . 10100 1 . . @qrrr_e
+UMINP_v 0.10 1110 ..1 . 10101 1 . . @qrrr_e
 
 ### Advanced SIMD scalar x indexed element
 
diff --git a/target/arm/tcg/gengvec.c b/target/arm/tcg/gengvec.c
index f010dd5a0e..22c9d17dce 100644
--- a/target/arm/tcg/gengvec.c
+++ b/target/arm/tcg/gengvec.c
@@ -1622,3 +1622,51 @@ void gen_gvec_addp(unsigned vece, uint32_t rd_ofs, 
uint32_t rn_ofs,
 };
 tcg_gen_gvec_3_ool(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, 0, fns[vece]);
 }
+
+void gen_gvec_smaxp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+static gen_helper_gvec_3 * const fns[4] = {
+gen_helper_gvec_smaxp_b,
+gen_helper_gvec_smaxp_h,
+gen_helper_gvec_smaxp_s,
+};
+tcg_debug_assert(vece <= MO_32);
+tcg_gen_gvec_3_ool(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, 0, fns[vece]);
+}
+
+void gen_gvec_sminp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+static gen_helper_gvec_3 * const fns[4] = {
+gen_helper_gvec_sminp_b,
+gen_helper_gvec_sminp_h,
+gen_helper_gvec_sminp_s,
+};
+tcg_debug_assert(vece <= MO_32);
+tcg_gen_gvec_3_ool(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, 0, fns[vece]);
+}
+
+void gen_gvec_umaxp(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+st

[PATCH v2 00/67] target/arm: Convert a64 advsimd to decodetree (part 1)

[Richard Henderson]

In the process, convert more code to gvec as well -- I will need
the gvec code for implementing SME2.  I guess this is about 1/3
of the job done, but there's no reason to wait until the patch
set is completely unwieldy.

Changes for v2:
  * Fix existing RISU failures vs neoverse-n1.
  * Introduce vfp_load_reg16, fixing a regression wrt VNEG (scalar, hp).
  * Fix typo in SUQADD vectorization.
  * Two more conversions.


r~


Richard Henderson (67):
  target/arm: Add neoverse-n1 to qemu-arm (DO NOT MERGE)
  target/arm: Use PLD, PLDW, PLI not NOP for t32
  target/arm: Reject incorrect operands to PLD, PLDW, PLI
  target/arm: Zero-extend writeback for fp16 FCVTZS (scalar, integer)
  target/arm: Fix decode of FMOV (hp) vs MOVI
  target/arm: Verify sz=0 for Advanced SIMD scalar pairwise (fp16)
  target/arm: Split out gengvec.c
  target/arm: Split out gengvec64.c
  target/arm: Convert Cryptographic AES to decodetree
  target/arm: Convert Cryptographic 3-register SHA to decodetree
  target/arm: Convert Cryptographic 2-register SHA to decodetree
  target/arm: Convert Cryptographic 3-register SHA512 to decodetree
  target/arm: Convert Cryptographic 2-register SHA512 to decodetree
  target/arm: Convert Cryptographic 4-register to decodetree
  target/arm: Convert Cryptographic 3-register, imm2 to decodetree
  target/arm: Convert XAR to decodetree
  target/arm: Convert Advanced SIMD copy to decodetree
  target/arm: Convert FMULX to decodetree
  target/arm: Convert FADD, FSUB, FDIV, FMUL to decodetree
  target/arm: Convert FMAX, FMIN, FMAXNM, FMINNM to decodetree
  target/arm: Introduce vfp_load_reg16
  target/arm: Expand vfp neg and abs inline
  target/arm: Convert FNMUL to decodetree
  target/arm: Convert FMLA, FMLS to decodetree
  target/arm: Convert FCMEQ, FCMGE, FCMGT, FACGE, FACGT to decodetree
  target/arm: Convert FABD to decodetree
  target/arm: Convert FRECPS, FRSQRTS to decodetree
  target/arm: Convert FADDP to decodetree
  target/arm: Convert FMAXP, FMINP, FMAXNMP, FMINNMP to decodetree
  target/arm: Use gvec for neon faddp, fmaxp, fminp
  target/arm: Convert ADDP to decodetree
  target/arm: Use gvec for neon padd
  target/arm: Convert SMAXP, SMINP, UMAXP, UMINP to decodetree
  target/arm: Use gvec for neon pmax, pmin
  target/arm: Convert FMLAL, FMLSL to decodetree
  target/arm: Convert disas_simd_3same_logic to decodetree
  target/arm: Improve vector UQADD, UQSUB, SQADD, SQSUB
  target/arm: Convert SUQADD and USQADD to gvec
  target/arm: Inline scalar SUQADD and USQADD
  target/arm: Inline scalar SQADD, UQADD, SQSUB, UQSUB
  target/arm: Convert SQADD, SQSUB, UQADD, UQSUB to decodetree
  target/arm: Convert SUQADD, USQADD to decodetree
  target/arm: Convert SSHL, USHL to decodetree
  target/arm: Convert SRSHL and URSHL (register) to gvec
  target/arm: Convert SRSHL, URSHL to decodetree
  target/arm: Convert SQSHL and UQSHL (register) to gvec
  target/arm: Convert SQSHL, UQSHL to decodetree
  target/arm: Convert SQRSHL and UQRSHL (register) to gvec
  target/arm: Convert SQRSHL, UQRSHL to decodetree
  target/arm: Convert ADD, SUB (vector) to decodetree
  target/arm: Convert CMGT, CMHI, CMGE, CMHS, CMTST, CMEQ to decodetree
  target/arm: Use TCG_COND_TSTNE in gen_cmtst_{i32,i64}
  target/arm: Use TCG_COND_TSTNE in gen_cmtst_vec
  target/arm: Convert SHADD, UHADD to gvec
  target/arm: Convert SHADD, UHADD to decodetree
  target/arm: Convert SHSUB, UHSUB to gvec
  target/arm: Convert SHSUB, UHSUB to decodetree
  target/arm: Convert SRHADD, URHADD to gvec
  target/arm: Convert SRHADD, URHADD to decodetree
  target/arm: Convert SMAX, SMIN, UMAX, UMIN to decodetree
  target/arm: Convert SABA, SABD, UABA, UABD to decodetree
  target/arm: Convert MUL, PMUL to decodetree
  target/arm: Convert MLA, MLS to decodetree
  target/arm: Tidy SQDMULH, SQRDMULH (vector)
  target/arm: Convert SQDMULH, SQRDMULH to decodetree
  target/arm: Convert FMADD, FMSUB, FNMADD, FNMSUB to decodetree
  target/arm: Convert FCSEL to decodetree

 target/arm/helper.h  |  164 +-
 target/arm/tcg/helper-a64.h  |   12 +
 target/arm/tcg/translate-a64.h   |   18 +
 target/arm/tcg/translate.h   |   95 +
 target/arm/tcg/a32-uncond.decode |8 +-
 target/arm/tcg/a64.decode|  430 ++-
 target/arm/tcg/neon-dp.decode|   37 +-
 target/arm/tcg/t32.decode|   26 +-
 target/arm/tcg/cpu32.c   |   73 +
 target/arm/tcg/gengvec.c | 2306 
 target/arm/tcg/gengvec64.c   |  367 +++
 target/arm/tcg/neon_helper.c |  511 +---
 target/arm/tcg/translate-a64.c   | 4440 ++
 target/arm/tcg/translate-neon.c  |  254 +-
 target/arm/tcg/translate-sve.c   |  145 +-
 target/arm/tcg/translate-vfp.c   |   93 +-
 target/arm/tcg/translate.c   | 1649 +--
 target/arm/tcg/vec_helper.c  |  349 ++-
 target/arm/vfp_helper.c  |   30 -
 target/arm/tcg/meson.build   |2 +
 20 files changed, 5446 insertions(+), 5563 deletions(-)
 create mode 100644 target/arm/

[PATCH V2 0/3] improve -overcommit cpu-pm=on|off

[Zide Chen]

Currently, if running "-overcommit cpu-pm=on" on hosts that don't
have MWAIT support, the MWAIT/MONITOR feature is advertised to the
guest and executing MWAIT/MONITOR on the guest triggers #UD.

V2:
- [PATCH 1]: took Thomas' suggestion for more generic fix
- [PATCH 2/3]: no changes

Zide Chen (3):
  vl: Allow multiple -overcommit commands
  target/i386: call cpu_exec_realizefn before x86_cpu_filter_features
  target/i386: Move host_cpu_enable_cpu_pm into kvm_cpu_realizefn()

 system/vl.c   |  4 ++--
 target/i386/cpu.c | 24 
 target/i386/host-cpu.c| 12 
 target/i386/kvm/kvm-cpu.c | 12 +---
 4 files changed, 23 insertions(+), 29 deletions(-)

-- 
2.34.1

[PATCH V2 3/3] target/i386: Move host_cpu_enable_cpu_pm into kvm_cpu_realizefn()

[Zide Chen]

It seems not a good idea to expand features in host_cpu_realizefn,
which is for host CPU only.  Additionally, cpu-pm option is KVM
specific, and it's cleaner to put it in kvm_cpu_realizefn(), together
with the WAITPKG code.

Fixes: f5cc5a5c1686 ("i386: split cpu accelerators from cpu.c, using 
AccelCPUClass")
Signed-off-by: Zide Chen 
---
 target/i386/host-cpu.c| 12 
 target/i386/kvm/kvm-cpu.c | 11 +--
 2 files changed, 9 insertions(+), 14 deletions(-)

diff --git a/target/i386/host-cpu.c b/target/i386/host-cpu.c
index 280e427c017c..8b8bf5afeccf 100644
--- a/target/i386/host-cpu.c
+++ b/target/i386/host-cpu.c
@@ -42,15 +42,6 @@ static uint32_t host_cpu_phys_bits(void)
 return host_phys_bits;
 }
 
-static void host_cpu_enable_cpu_pm(X86CPU *cpu)
-{
-CPUX86State *env = &cpu->env;
-
-host_cpuid(5, 0, &cpu->mwait.eax, &cpu->mwait.ebx,
-   &cpu->mwait.ecx, &cpu->mwait.edx);
-env->features[FEAT_1_ECX] |= CPUID_EXT_MONITOR;
-}
-
 static uint32_t host_cpu_adjust_phys_bits(X86CPU *cpu)
 {
 uint32_t host_phys_bits = host_cpu_phys_bits();
@@ -83,9 +74,6 @@ bool host_cpu_realizefn(CPUState *cs, Error **errp)
 X86CPU *cpu = X86_CPU(cs);
 CPUX86State *env = &cpu->env;
 
-if (cpu->max_features && enable_cpu_pm) {
-host_cpu_enable_cpu_pm(cpu);
-}
 if (env->features[FEAT_8000_0001_EDX] & CPUID_EXT2_LM) {
 uint32_t phys_bits = host_cpu_adjust_phys_bits(cpu);
 
diff --git a/target/i386/kvm/kvm-cpu.c b/target/i386/kvm/kvm-cpu.c
index 3adcedf0dbc3..197c892da89a 100644
--- a/target/i386/kvm/kvm-cpu.c
+++ b/target/i386/kvm/kvm-cpu.c
@@ -64,9 +64,16 @@ static bool kvm_cpu_realizefn(CPUState *cs, Error **errp)
  *   cpu_common_realizefn() (via xcc->parent_realize)
  */
 if (cpu->max_features) {
-if (enable_cpu_pm && kvm_has_waitpkg()) {
-env->features[FEAT_7_0_ECX] |= CPUID_7_0_ECX_WAITPKG;
+if (enable_cpu_pm) {
+if (kvm_has_waitpkg()) {
+env->features[FEAT_7_0_ECX] |= CPUID_7_0_ECX_WAITPKG;
+}
+
+host_cpuid(5, 0, &cpu->mwait.eax, &cpu->mwait.ebx,
+   &cpu->mwait.ecx, &cpu->mwait.edx);
+env->features[FEAT_1_ECX] |= CPUID_EXT_MONITOR;
 }
+
 if (cpu->ucode_rev == 0) {
 cpu->ucode_rev =
 kvm_arch_get_supported_msr_feature(kvm_state,
-- 
2.34.1

[PATCH V2 2/3] target/i386: call cpu_exec_realizefn before x86_cpu_filter_features

[Zide Chen]

cpu_exec_realizefn which calls the accel-specific realizefn may expand
features.  e.g., some accel-specific options may require extra features
to be enabled, and it's appropriate to expand these features in accel-
specific realizefn.

One such example is the cpu-pm option, which may add CPUID_EXT_MONITOR.

Thus, call cpu_exec_realizefn before x86_cpu_filter_features to ensure
that it won't expose features not supported by the host.

Fixes: 662175b91ff2 ("i386: reorder call to cpu_exec_realizefn")
Suggested-by: Xiaoyao Li 
Signed-off-by: Zide Chen 
---
 target/i386/cpu.c | 24 
 target/i386/kvm/kvm-cpu.c |  1 -
 2 files changed, 12 insertions(+), 13 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index bc2dceb647fa..a1c1c785bd2f 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -7604,6 +7604,18 @@ static void x86_cpu_realizefn(DeviceState *dev, Error 
**errp)
 }
 }
 
+/*
+ * note: the call to the framework needs to happen after feature expansion,
+ * but before the checks/modifications to ucode_rev, mwait, phys_bits.
+ * These may be set by the accel-specific code,
+ * and the results are subsequently checked / assumed in this function.
+ */
+cpu_exec_realizefn(cs, &local_err);
+if (local_err != NULL) {
+error_propagate(errp, local_err);
+return;
+}
+
 x86_cpu_filter_features(cpu, cpu->check_cpuid || cpu->enforce_cpuid);
 
 if (cpu->enforce_cpuid && x86_cpu_have_filtered_features(cpu)) {
@@ -7625,18 +7637,6 @@ static void x86_cpu_realizefn(DeviceState *dev, Error 
**errp)
 
 x86_cpu_set_sgxlepubkeyhash(env);
 
-/*
- * note: the call to the framework needs to happen after feature expansion,
- * but before the checks/modifications to ucode_rev, mwait, phys_bits.
- * These may be set by the accel-specific code,
- * and the results are subsequently checked / assumed in this function.
- */
-cpu_exec_realizefn(cs, &local_err);
-if (local_err != NULL) {
-error_propagate(errp, local_err);
-return;
-}
-
 if (xcc->host_cpuid_required && !accel_uses_host_cpuid()) {
 g_autofree char *name = x86_cpu_class_get_model_name(xcc);
 error_setg(&local_err, "CPU model '%s' requires KVM or HVF", name);
diff --git a/target/i386/kvm/kvm-cpu.c b/target/i386/kvm/kvm-cpu.c
index f76972e47e61..3adcedf0dbc3 100644
--- a/target/i386/kvm/kvm-cpu.c
+++ b/target/i386/kvm/kvm-cpu.c
@@ -50,7 +50,6 @@ static bool kvm_cpu_realizefn(CPUState *cs, Error **errp)
  * nothing else has been set by the user (or by accelerators) in
  * cpu->ucode_rev and cpu->phys_bits, and updates the CPUID results in
  * mwait.ecx.
- * This accel realization code also assumes cpu features are already 
expanded.
  *
  * realize order:
  *
-- 
2.34.1

[PATCH V2 1/3] vl: Allow multiple -overcommit commands

[Zide Chen]

Both cpu-pm and mem-lock are related to system resource overcommit, but
they are separate from each other, in terms of how they are realized,
and of course, they are applied to different system resources.

It's tempting to use separate command lines to specify their behavior.
e.g., in the following example, the cpu-pm command is quietly
overwritten, and it's not easy to notice it without careful inspection.

  --overcommit mem-lock=on
  --overcommit cpu-pm=on

Fixes: c8c9dc42b7ca ("Remove the deprecated -realtime option")
Suggested-by: Thomas Huth 
Signed-off-by: Zide Chen 
---

v2:

Thanks to Thomas' suggestion, changed to this better approach, which
is more generic and can handle situations like: "enabled the option in
the config file, and now you'd like to disable it on the command line
again".

 system/vl.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/system/vl.c b/system/vl.c
index a3eede5fa5b8..dfa6cdd9283b 100644
--- a/system/vl.c
+++ b/system/vl.c
@@ -3545,8 +3545,8 @@ void qemu_init(int argc, char **argv)
 if (!opts) {
 exit(1);
 }
-enable_mlock = qemu_opt_get_bool(opts, "mem-lock", false);
-enable_cpu_pm = qemu_opt_get_bool(opts, "cpu-pm", false);
+enable_mlock = qemu_opt_get_bool(opts, "mem-lock", 
enable_mlock);
+enable_cpu_pm = qemu_opt_get_bool(opts, "cpu-pm", 
enable_cpu_pm);
 break;
 case QEMU_OPTION_compat:
 {
-- 
2.34.1

Re: [RFC PATCH v3 18/18] hw/arm/virt: Set SMMU OAS based on CPU PARANGE

[Julien Grall]

Hi Mostafa,

On 29/04/2024 04:24, Mostafa Saleh wrote:

Use the new SMMU property to make the SMMU OAS match the CPU PARANGE.
That's according to SMMU manual ARM IHI 0070F.b: >  6.3.6 SMMU_IDR5, OAS 
must match the system physical address size.




Signed-off-by: Mostafa Saleh 
---
  hw/arm/virt.c  | 14 --
  target/arm/cpu.h   |  2 ++
  target/arm/cpu64.c |  5 +


When trying to build qemu-system-arm, I get the following error:

[1/3028] Generating subprojects/dtc/version_gen.h with a custom command
[2/3028] Generating qemu-version.h with a custom command (wrapped by 
meson to capture output)

[3/3021] Linking target qemu-system-aarch64
[4/3021] Linking target qemu-system-arm
FAILED: qemu-system-arm
clang -m64 -mcx16 @qemu-system-arm.rsp
libqemu-arm-softmmu.fa.p/hw_arm_virt.c.o: In function `get_system_oas':
/home/jgrall/works/oss/qemu/build/../hw/arm/virt.c:259: undefined 
reference to `cpu_arm_get_oas'
clang-11: error: linker command failed with exit code 1 (use -v to see 
invocation)

ninja: build stopped: subcommand failed.
make: *** [run-ninja] Error 1

I think you need to provide cpu_arm_get_oas() also for 32-bit arm (I 
guess it is implemented in target/arm/cpu.c).


Cheers,

--
Julien Grall

[PATCH v2 5/6] Move tcg implementation of x86 get_physical_address into common helper code.

[Don Porter]

Signed-off-by: Don Porter 
---
 target/i386/cpu.h|  41 ++
 target/i386/helper.c | 515 +
 target/i386/tcg/sysemu/excp_helper.c | 555 +--
 3 files changed, 561 insertions(+), 550 deletions(-)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index fc3ae55213..39ce49e61f 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -2094,6 +2094,42 @@ struct X86CPUClass {
 ResettablePhases parent_phases;
 };
 
+typedef struct X86TranslateParams {
+target_ulong addr;
+target_ulong cr3;
+int pg_mode;
+int mmu_idx;
+int ptw_idx;
+MMUAccessType access_type;
+} X86TranslateParams;
+
+typedef struct X86TranslateResult {
+hwaddr paddr;
+int prot;
+int page_size;
+} X86TranslateResult;
+
+typedef enum X86TranslateFaultStage2 {
+S2_NONE,
+S2_GPA,
+S2_GPT,
+} X86TranslateFaultStage2;
+
+typedef struct X86TranslateFault {
+int exception_index;
+int error_code;
+target_ulong cr2;
+X86TranslateFaultStage2 stage2;
+} X86TranslateFault;
+
+typedef struct X86PTETranslate {
+CPUX86State *env;
+X86TranslateFault *err;
+int ptw_idx;
+void *haddr;
+hwaddr gaddr;
+} X86PTETranslate;
+
 /* Intended to become a generic PTE type */
 typedef union PTE {
 uint64_t pte64_t;
@@ -2137,6 +2173,11 @@ void x86_cpu_list(void);
 int cpu_x86_support_mca_broadcast(CPUX86State *env);
 
 #ifndef CONFIG_USER_ONLY
+bool x86_cpu_get_physical_address(CPUX86State *env, vaddr addr,
+  MMUAccessType access_type, int mmu_idx,
+  X86TranslateResult *out,
+  X86TranslateFault *err, uint64_t ra);
+
 hwaddr x86_cpu_get_phys_page_attrs_debug(CPUState *cpu, vaddr addr,
  MemTxAttrs *attrs);
 int cpu_get_pic_interrupt(CPUX86State *s);
diff --git a/target/i386/helper.c b/target/i386/helper.c
index 48d1513a35..21445e84b2 100644
--- a/target/i386/helper.c
+++ b/target/i386/helper.c
@@ -26,6 +26,7 @@
 #include "sysemu/hw_accel.h"
 #include "monitor/monitor.h"
 #include "kvm/kvm_i386.h"
+#include "exec/cpu_ldst.h"
 #endif
 #include "qemu/log.h"
 #ifdef CONFIG_TCG
@@ -227,6 +228,520 @@ void cpu_x86_update_cr4(CPUX86State *env, uint32_t 
new_cr4)
 }
 
 #if !defined(CONFIG_USER_ONLY)
+
+static inline uint32_t ptw_ldl(const X86PTETranslate *in, uint64_t ra)
+{
+if (likely(in->haddr)) {
+return ldl_p(in->haddr);
+}
+return cpu_ldl_mmuidx_ra(in->env, in->gaddr, in->ptw_idx, ra);
+}
+
+static inline uint64_t ptw_ldq(const X86PTETranslate *in, uint64_t ra)
+{
+if (likely(in->haddr)) {
+return ldq_p(in->haddr);
+}
+return cpu_ldq_mmuidx_ra(in->env, in->gaddr, in->ptw_idx, ra);
+}
+/*
+ * Note that we can use a 32-bit cmpxchg for all page table entries,
+ * even 64-bit ones, because PG_PRESENT_MASK, PG_ACCESSED_MASK and
+ * PG_DIRTY_MASK are all in the low 32 bits.
+ */
+static bool ptw_setl_slow(const X86PTETranslate *in, uint32_t old, uint32_t 
new)
+{
+uint32_t cmp;
+
+/* Does x86 really perform a rmw cycle on mmio for ptw? */
+start_exclusive();
+cmp = cpu_ldl_mmuidx_ra(in->env, in->gaddr, in->ptw_idx, 0);
+if (cmp == old) {
+cpu_stl_mmuidx_ra(in->env, in->gaddr, new, in->ptw_idx, 0);
+}
+end_exclusive();
+return cmp == old;
+}
+
+static inline bool ptw_setl(const X86PTETranslate *in, uint32_t old,
+uint32_t set)
+{
+if (set & ~old) {
+uint32_t new = old | set;
+if (likely(in->haddr)) {
+old = cpu_to_le32(old);
+new = cpu_to_le32(new);
+return qatomic_cmpxchg((uint32_t *)in->haddr, old, new) == old;
+}
+return ptw_setl_slow(in, old, new);
+}
+return true;
+}
+
+
+static bool ptw_translate(X86PTETranslate *inout, hwaddr addr, uint64_t ra)
+{
+CPUTLBEntryFull *full;
+int flags;
+
+inout->gaddr = addr;
+flags = probe_access_full(inout->env, addr, 0, MMU_DATA_STORE,
+  inout->ptw_idx, true, &inout->haddr, &full, ra);
+
+if (unlikely(flags & TLB_INVALID_MASK)) {
+X86TranslateFault *err = inout->err;
+
+assert(inout->ptw_idx == MMU_NESTED_IDX);
+*err = (X86TranslateFault){
+.error_code = inout->env->error_code,
+.cr2 = addr,
+.stage2 = S2_GPT,
+};
+return false;
+}
+return true;
+}
+
+static bool x86_mmu_translate(CPUX86State *env, const X86TranslateParams *in,
+  X86TranslateResult *out,
+  X86TranslateFault *err, uint64_t ra)
+{
+const target_ulong addr = in->addr;
+const int pg_mode = in->pg_mode;
+const bool is_user = is_mmu_index_user(in->mmu_idx);
+const MMUAccessType access_type = in->access_type;
+uint64_t ptep, pte, rsvd_mask;
+X86PTETranslate pte_trans = {
+.env =

[PATCH v2 2/6] Convert 'info tlb' to use generic iterator

[Don Porter]

Signed-off-by: Don Porter 
---
 target/i386/monitor.c | 203 ++
 1 file changed, 28 insertions(+), 175 deletions(-)

diff --git a/target/i386/monitor.c b/target/i386/monitor.c
index d7aae99c73..adf95edfb4 100644
--- a/target/i386/monitor.c
+++ b/target/i386/monitor.c
@@ -430,201 +430,54 @@ void hmp_info_pg(Monitor *mon, const QDict *qdict)
 }
 
 static void print_pte(Monitor *mon, CPUArchState *env, hwaddr addr,
-  hwaddr pte, hwaddr mask)
+  hwaddr pte)
 {
-addr = addr_canonical(env, addr);
-
-monitor_printf(mon, HWADDR_FMT_plx ": " HWADDR_FMT_plx
-   " %c%c%c%c%c%c%c%c%c\n",
-   addr,
-   pte & mask,
-   pte & PG_NX_MASK ? 'X' : '-',
-   pte & PG_GLOBAL_MASK ? 'G' : '-',
-   pte & PG_PSE_MASK ? 'P' : '-',
-   pte & PG_DIRTY_MASK ? 'D' : '-',
-   pte & PG_ACCESSED_MASK ? 'A' : '-',
-   pte & PG_PCD_MASK ? 'C' : '-',
-   pte & PG_PWT_MASK ? 'T' : '-',
-   pte & PG_USER_MASK ? 'U' : '-',
-   pte & PG_RW_MASK ? 'W' : '-');
-}
+char buf[128];
+char *pos = buf;
 
-static void tlb_info_32(Monitor *mon, CPUArchState *env)
-{
-unsigned int l1, l2;
-uint32_t pgd, pde, pte;
+addr = addr_canonical(env, addr);
 
-pgd = env->cr[3] & ~0xfff;
-for(l1 = 0; l1 < 1024; l1++) {
-cpu_physical_memory_read(pgd + l1 * 4, &pde, 4);
-pde = le32_to_cpu(pde);
-if (pde & PG_PRESENT_MASK) {
-if ((pde & PG_PSE_MASK) && (env->cr[4] & CR4_PSE_MASK)) {
-/* 4M pages */
-print_pte(mon, env, (l1 << 22), pde, ~((1 << 21) - 1));
-} else {
-for(l2 = 0; l2 < 1024; l2++) {
-cpu_physical_memory_read((pde & ~0xfff) + l2 * 4, &pte, 4);
-pte = le32_to_cpu(pte);
-if (pte & PG_PRESENT_MASK) {
-print_pte(mon, env, (l1 << 22) + (l2 << 12),
-  pte & ~PG_PSE_MASK,
-  ~0xfff);
-}
-}
-}
-}
-}
-}
+pos += sprintf(pos, HWADDR_FMT_plx ": " HWADDR_FMT_plx " ", addr,
+   (hwaddr) (pte & PG_ADDRESS_MASK));
 
-static void tlb_info_pae32(Monitor *mon, CPUArchState *env)
-{
-unsigned int l1, l2, l3;
-uint64_t pdpe, pde, pte;
-uint64_t pdp_addr, pd_addr, pt_addr;
+pos += sprintf(pos, " %s", pg_bits(pte));
 
-pdp_addr = env->cr[3] & ~0x1f;
-for (l1 = 0; l1 < 4; l1++) {
-cpu_physical_memory_read(pdp_addr + l1 * 8, &pdpe, 8);
-pdpe = le64_to_cpu(pdpe);
-if (pdpe & PG_PRESENT_MASK) {
-pd_addr = pdpe & 0x3f000ULL;
-for (l2 = 0; l2 < 512; l2++) {
-cpu_physical_memory_read(pd_addr + l2 * 8, &pde, 8);
-pde = le64_to_cpu(pde);
-if (pde & PG_PRESENT_MASK) {
-if (pde & PG_PSE_MASK) {
-/* 2M pages with PAE, CR4.PSE is ignored */
-print_pte(mon, env, (l1 << 30) + (l2 << 21), pde,
-  ~((hwaddr)(1 << 20) - 1));
-} else {
-pt_addr = pde & 0x3f000ULL;
-for (l3 = 0; l3 < 512; l3++) {
-cpu_physical_memory_read(pt_addr + l3 * 8, &pte, 
8);
-pte = le64_to_cpu(pte);
-if (pte & PG_PRESENT_MASK) {
-print_pte(mon, env, (l1 << 30) + (l2 << 21)
-  + (l3 << 12),
-  pte & ~PG_PSE_MASK,
-  ~(hwaddr)0xfff);
-}
-}
-}
-}
-}
-}
+/* Trim line to fit screen */
+if (pos - buf > 79) {
+strcpy(buf + 77, "..");
 }
-}
 
-#ifdef TARGET_X86_64
-static void tlb_info_la48(Monitor *mon, CPUArchState *env,
-uint64_t l0, uint64_t pml4_addr)
-{
-uint64_t l1, l2, l3, l4;
-uint64_t pml4e, pdpe, pde, pte;
-uint64_t pdp_addr, pd_addr, pt_addr;
-
-for (l1 = 0; l1 < 512; l1++) {
-cpu_physical_memory_read(pml4_addr + l1 * 8, &pml4e, 8);
-pml4e = le64_to_cpu(pml4e);
-if (!(pml4e & PG_PRESENT_MASK)) {
-continue;
-}
-
-pdp_addr = pml4e & 0x3f000ULL;
-for (l2 = 0; l2 < 512; l2++) {
-cpu_physical_memory_read(pdp_addr + l2 * 8, &pdpe, 8);
-pdpe = le64_to_cpu(pdpe);
-if (!(pdpe & PG_PRESENT_MASK)) {
-continue;
-}
-
-if (pdpe & PG_PSE_MASK) {
-/* 1G pages, CR4.PSE is ignored *

[PATCH v2 6/6] Convert x86_mmu_translate() to use common code.

[Don Porter]

Signed-off-by: Don Porter 
---
 target/i386/arch_memory_mapping.c|  37 ++-
 target/i386/cpu.h|  11 +-
 target/i386/helper.c | 371 ++-
 target/i386/tcg/sysemu/excp_helper.c |   2 +-
 4 files changed, 128 insertions(+), 293 deletions(-)

diff --git a/target/i386/arch_memory_mapping.c 
b/target/i386/arch_memory_mapping.c
index 040464dd34..9ea5aeff16 100644
--- a/target/i386/arch_memory_mapping.c
+++ b/target/i386/arch_memory_mapping.c
@@ -33,7 +33,7 @@
  * Returns a hardware address on success.  Should not fail (i.e., caller is
  * responsible to ensure that a page table is actually present).
  */
-static hwaddr mmu_page_table_root(CPUState *cs, int *height)
+hwaddr mmu_page_table_root(CPUState *cs, int *height)
 {
 X86CPU *cpu = X86_CPU(cs);
 CPUX86State *env = &cpu->env;
@@ -228,6 +228,35 @@ static void _mmu_decode_va_parameters(CPUState *cs, int 
height,
 }
 }
 
+/**
+ * mmu_virtual_to_pte_index - Given a virtual address and height in the
+ *   page table radix tree, return the index that should be used
+ *   to look up the next page table entry (pte) in translating an
+ *   address.
+ *
+ * @cs - CPU state
+ * @vaddr - The virtual address to translate
+ * @height - height of node within the tree (leaves are 1, not 0).
+ *
+ * Example: In 32-bit x86 page tables, the virtual address is split
+ * into 10 bits at height 2, 10 bits at height 1, and 12 offset bits.
+ * So a call with VA and height 2 would return the first 10 bits of va,
+ * right shifted by 22.
+ */
+
+int mmu_virtual_to_pte_index(CPUState *cs, target_ulong vaddr, int height)
+{
+int shift = 0;
+int width = 0;
+int mask = 0;
+
+_mmu_decode_va_parameters(cs, height, &shift, &width);
+
+mask = (1 << width) - 1;
+
+return (vaddr >> shift) & mask;
+}
+
 /**
  * get_pte - Copy the contents of the page table entry at node[i] into 
pt_entry.
  *   Optionally, add the relevant bits to the virtual address in
@@ -247,7 +276,7 @@ static void _mmu_decode_va_parameters(CPUState *cs, int 
height,
  *  Optional parameter.
  */
 
-static void
+void
 get_pte(CPUState *cs, hwaddr node, int i, int height,
 PTE_t *pt_entry, target_ulong vaddr_parent, target_ulong *vaddr_pte,
 hwaddr *pte_paddr)
@@ -284,7 +313,7 @@ get_pte(CPUState *cs, hwaddr node, int i, int height,
 }
 
 
-static bool
+bool
 mmu_pte_check_bits(CPUState *cs, PTE_t *pte, int64_t mask)
 {
 X86CPU *cpu = X86_CPU(cs);
@@ -300,7 +329,7 @@ mmu_pte_check_bits(CPUState *cs, PTE_t *pte, int64_t mask)
  * mmu_pte_presetn - Return true if the pte is
  *   marked 'present'
  */
-static bool
+bool
 mmu_pte_present(CPUState *cs, PTE_t *pte)
 {
 return mmu_pte_check_bits(cs, pte, PG_PRESENT_MASK);
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 39ce49e61f..51d4a55e6b 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -2151,15 +2151,23 @@ int x86_cpu_write_elf64_qemunote(WriteCoreDumpFunction 
f, CPUState *cpu,
 int x86_cpu_write_elf32_qemunote(WriteCoreDumpFunction f, CPUState *cpu,
  DumpState *s);
 
+hwaddr mmu_page_table_root(CPUState *cs, int *height);
+bool mmu_pte_check_bits(CPUState *cs, PTE_t *pte, int64_t mask);
+bool mmu_pte_present(CPUState *cs, PTE_t *pte);
 bool mmu_pte_leaf(CPUState *cs, int height, PTE_t *pte);
 target_ulong mmu_pte_leaf_page_size(CPUState *cs, int height);
 hwaddr mmu_pte_child(CPUState *cs, PTE_t *pte, int height);
 int mmu_page_table_entries_per_node(CPUState *cs, int height);
+int mmu_virtual_to_pte_index(CPUState *cs, target_ulong vaddr, int height);
 bool for_each_pte(CPUState *cs,
   int (*fn)(CPUState *cs, void *data, PTE_t *pte,
 target_ulong vaddr, int height, int offset),
   void *data, bool visit_interior_nodes,
   bool visit_not_present);
+void get_pte(CPUState *cs, hwaddr node, int i, int height, PTE_t *pt_entry,
+ target_ulong vaddr_parent, target_ulong *vaddr_pte,
+ hwaddr *pte_paddr);
+
 
 bool x86_cpu_get_memory_mapping(CPUState *cpu, MemoryMappingList *list,
 Error **errp);
@@ -2176,7 +2184,8 @@ int cpu_x86_support_mca_broadcast(CPUX86State *env);
 bool x86_cpu_get_physical_address(CPUX86State *env, vaddr addr,
   MMUAccessType access_type, int mmu_idx,
   X86TranslateResult *out,
-  X86TranslateFault *err, uint64_t ra);
+  X86TranslateFault *err, uint64_t ra,
+  bool read_only);
 
 hwaddr x86_cpu_get_phys_page_attrs_debug(CPUState *cpu, vaddr addr,
  MemTxAttrs *attrs);
diff --git a/target/i386/helper.c b/target/i386/helper.c
index 21445e84b2..17ffba200d 100644
--- a/target/i386/helper.c
+++ b/target/i386/helper.c
@@ -

[PATCH v2 1/6] Add an "info pg" command that prints the current page tables

[Don Porter]

The new "info pg" monitor command prints the current page table,
including virtual address ranges, flag bits, and snippets of physical
page numbers.  Completely filled regions of the page table with
compatible flags are "folded", with the result that the complete
output for a freshly booted x86-64 Linux VM can fit in a single
terminal window.  The output looks like this:

VPN range Entry FlagsPhysical page
[7f000-7f000] PML4[0fe] ---DA--UWP
  [7f28c-7f28f]  PDP[0a3] ---DA--UWP
[7f28c4600-7f28c47ff]  PDE[023] ---DA--UWP
  [7f28c4655-7f28c4656]  PTE[055-056] X--D---U-P 007f14-007f15
  [7f28c465b-7f28c465b]  PTE[05b] A--U-P 001cfc
...
[ff800-ff800] PML4[1ff] ---DA--UWP
  [8-b]  PDP[1fe] ---DA---WP
[81000-81dff]  PDE[008-00e] -GSDA---WP 001000-001dff
  [c-f]  PDP[1ff] ---DA--UWP
[ff400-ff5ff]  PDE[1fa] ---DA--UWP
  [ff5fb-ff5fc]  PTE[1fb-1fc] XG-DACT-WP 0fec00 0fee00
[ff600-ff7ff]  PDE[1fb] ---DA--UWP
  [ff600-ff600]  PTE[000] -G-DA--U-P 001467

This draws heavy inspiration from Austin Clements' original patch.

This also adds a generic page table walker, which other monitor
and execution commands will be migrated to in subsequent patches.

Signed-off-by: Don Porter 
---
 hmp-commands-info.hx  |  26 ++
 include/monitor/hmp-target.h  |   1 +
 target/i386/arch_memory_mapping.c | 486 +-
 target/i386/cpu.h |  16 +
 target/i386/monitor.c | 380 +++
 5 files changed, 908 insertions(+), 1 deletion(-)

diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index 20a9835ea8..918b82015c 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -237,6 +237,32 @@ ERST
 .cmd= hmp_info_mtree,
 },
 
+#if defined(TARGET_I386)
+{
+.name   = "pg",
+.args_type  = "",
+.params = "",
+.help   = "show the page table",
+.cmd= hmp_info_pg,
+},
+#endif
+
+SRST   
|
+  ``info pg``  
|
+Show the active page table.
|
+ERST
+
+{
+.name   = "mtree",
+.args_type  = "flatview:-f,dispatch_tree:-d,owner:-o,disabled:-D",
+.params = "[-f][-d][-o][-D]",
+.help   = "show memory tree (-f: dump flat view for address 
spaces;"
+  "-d: dump dispatch tree, valid with -f only);"
+  "-o: dump region owners/parents;"
+  "-D: dump disabled regions",
+.cmd= hmp_info_mtree,
+},
+
 SRST
   ``info mtree``
 Show memory tree.
diff --git a/include/monitor/hmp-target.h b/include/monitor/hmp-target.h
index b679aaebbf..9af72ea58d 100644
--- a/include/monitor/hmp-target.h
+++ b/include/monitor/hmp-target.h
@@ -50,6 +50,7 @@ CPUState *mon_get_cpu(Monitor *mon);
 void hmp_info_mem(Monitor *mon, const QDict *qdict);
 void hmp_info_tlb(Monitor *mon, const QDict *qdict);
 void hmp_mce(Monitor *mon, const QDict *qdict);
+void hmp_info_pg(Monitor *mon, const QDict *qdict);
 void hmp_info_local_apic(Monitor *mon, const QDict *qdict);
 void hmp_info_sev(Monitor *mon, const QDict *qdict);
 void hmp_info_sgx(Monitor *mon, const QDict *qdict);
diff --git a/target/i386/arch_memory_mapping.c 
b/target/i386/arch_memory_mapping.c
index d1ff659128..00bf2a2116 100644
--- a/target/i386/arch_memory_mapping.c
+++ b/target/i386/arch_memory_mapping.c
@@ -15,6 +15,491 @@
 #include "cpu.h"
 #include "sysemu/memory_mapping.h"
 
+/**
+ ** code hook implementations for x86 ***
+ */
+
+#define PML4_ADDR_MASK 0xff000ULL /* selects bits 51:12 */
+
+/**
+ * mmu_page_table_root - Given a CPUState, return the physical address
+ *   of the current page table root, as well as
+ *   write the height of the tree into *height.
+ *
+ * @cs - CPU state
+ * @height - a pointer to an integer, to store the page table tree height
+ *
+ * Returns a hardware address on success.  Should not fail (i.e., caller is
+ * responsible to ensure that a page table is actually present).
+ */
+static hwaddr mmu_page_table_root(CPUState *cs, int *height)
+{
+X86CPU *cpu = X86_CPU(cs);
+CPUX86State *env = &cpu->env;
+/*
+ * DEP 5/15/24: Some original page table walking code sets the a20
+ * mask as a 32 bit integer and checks it on each level of hte
+ * page table walk; some only checks it against the final result.
+ * For 64 bits, I think we need to sign extend in the common case
+ * it is not set (and returns -1), or we will lose bits.
+ */
+int64_t a20_mask;
+
+assert(cpu_p

[PATCH v2 4/6] Convert x86_cpu_get_memory_mapping() to use generic iterators

[Don Porter]

Signed-off-by: Don Porter 
---
 target/i386/arch_memory_mapping.c | 318 --
 1 file changed, 40 insertions(+), 278 deletions(-)

diff --git a/target/i386/arch_memory_mapping.c 
b/target/i386/arch_memory_mapping.c
index 00bf2a2116..040464dd34 100644
--- a/target/i386/arch_memory_mapping.c
+++ b/target/i386/arch_memory_mapping.c
@@ -19,6 +19,7 @@
  ** code hook implementations for x86 ***
  */
 
+/* PAE Paging or IA-32e Paging */
 #define PML4_ADDR_MASK 0xff000ULL /* selects bits 51:12 */
 
 /**
@@ -499,302 +500,63 @@ bool for_each_pte(CPUState *cs,
 /**
  * Back to x86 hooks
  */
+struct memory_mapping_data {
+MemoryMappingList *list;
+};
 
-/* PAE Paging or IA-32e Paging */
-static void walk_pte(MemoryMappingList *list, AddressSpace *as,
- hwaddr pte_start_addr,
- int32_t a20_mask, target_ulong start_line_addr)
-{
-hwaddr pte_addr, start_paddr;
-uint64_t pte;
-target_ulong start_vaddr;
-int i;
-
-for (i = 0; i < 512; i++) {
-pte_addr = (pte_start_addr + i * 8) & a20_mask;
-pte = address_space_ldq(as, pte_addr, MEMTXATTRS_UNSPECIFIED, NULL);
-if (!(pte & PG_PRESENT_MASK)) {
-/* not present */
-continue;
-}
-
-start_paddr = (pte & ~0xfff) & ~(0x1ULL << 63);
-if (cpu_physical_memory_is_io(start_paddr)) {
-/* I/O region */
-continue;
-}
-
-start_vaddr = start_line_addr | ((i & 0x1ff) << 12);
-memory_mapping_list_add_merge_sorted(list, start_paddr,
- start_vaddr, 1 << 12);
-}
-}
-
-/* 32-bit Paging */
-static void walk_pte2(MemoryMappingList *list, AddressSpace *as,
-  hwaddr pte_start_addr, int32_t a20_mask,
-  target_ulong start_line_addr)
-{
-hwaddr pte_addr, start_paddr;
-uint32_t pte;
-target_ulong start_vaddr;
-int i;
-
-for (i = 0; i < 1024; i++) {
-pte_addr = (pte_start_addr + i * 4) & a20_mask;
-pte = address_space_ldl(as, pte_addr, MEMTXATTRS_UNSPECIFIED, NULL);
-if (!(pte & PG_PRESENT_MASK)) {
-/* not present */
-continue;
-}
-
-start_paddr = pte & ~0xfff;
-if (cpu_physical_memory_is_io(start_paddr)) {
-/* I/O region */
-continue;
-}
-
-start_vaddr = start_line_addr | ((i & 0x3ff) << 12);
-memory_mapping_list_add_merge_sorted(list, start_paddr,
- start_vaddr, 1 << 12);
-}
-}
-
-/* PAE Paging or IA-32e Paging */
-#define PLM4_ADDR_MASK 0xff000ULL /* selects bits 51:12 */
-
-static void walk_pde(MemoryMappingList *list, AddressSpace *as,
- hwaddr pde_start_addr,
- int32_t a20_mask, target_ulong start_line_addr)
+static int add_memory_mapping_to_list(CPUState *cs, void *data, PTE_t *pte,
+  target_ulong vaddr, int height,
+  int offset)
 {
-hwaddr pde_addr, pte_start_addr, start_paddr;
-uint64_t pde;
-target_ulong line_addr, start_vaddr;
-int i;
-
-for (i = 0; i < 512; i++) {
-pde_addr = (pde_start_addr + i * 8) & a20_mask;
-pde = address_space_ldq(as, pde_addr, MEMTXATTRS_UNSPECIFIED, NULL);
-if (!(pde & PG_PRESENT_MASK)) {
-/* not present */
-continue;
-}
-
-line_addr = start_line_addr | ((i & 0x1ff) << 21);
-if (pde & PG_PSE_MASK) {
-/* 2 MB page */
-start_paddr = (pde & ~0x1f) & ~(0x1ULL << 63);
-if (cpu_physical_memory_is_io(start_paddr)) {
-/* I/O region */
-continue;
-}
-start_vaddr = line_addr;
-memory_mapping_list_add_merge_sorted(list, start_paddr,
- start_vaddr, 1 << 21);
-continue;
-}
-
-pte_start_addr = (pde & PLM4_ADDR_MASK) & a20_mask;
-walk_pte(list, as, pte_start_addr, a20_mask, line_addr);
-}
-}
+X86CPU *cpu = X86_CPU(cs);
+CPUX86State *env = &cpu->env;
 
-/* 32-bit Paging */
-static void walk_pde2(MemoryMappingList *list, AddressSpace *as,
-  hwaddr pde_start_addr, int32_t a20_mask,
-  bool pse)
-{
-hwaddr pde_addr, pte_start_addr, start_paddr, high_paddr;
-uint32_t pde;
-target_ulong line_addr, start_vaddr;
-int i;
+struct memory_mapping_data *mm_data = (struct memory_mapping_data *) data;
 
-for (i = 0; i < 1024; i++) {
-pde_addr = (pde_start_addr + i * 4) & a20_mask;
-pde = address_space_ldl(as, pde_addr, MEMTXATTRS_UNSPECIFIED, NULL);
-if (!(pde & PG_PRESENT_MASK)) {
-/* not present */
-continue;
+hwaddr start_paddr = 0;
+size_t pg_size = mmu_pte_lea

[PATCH v2 0/6] Rework x86 page table walks

[Don Porter]

This version of the 'info pg' command adopts Peter Maydell's request
to write some guest-agnostic page table iterator and accessor code,
along with architecture-specific hooks.  The first patch in this
series contributes a generic page table iterator and an x86
instantiation.  As a client, we first introduce an 'info pg' monitor
command, as well as a compressing callback hook for creating succinct
page table representations.

After this, each successive patch replaces an exisitng x86 page table
walker with a use of common iterator code.

I could use advice on how to ensure this is sufficiently well tested.
I used 'make check' and 'make check-avocado', which both pass; what is
the typical standard for testing something like a page table related
change?

As far as generality, I have only attempted this on x86, but I expect
the design would work for any similar radix-tree style page table.

I am still new enough to the code base that I wasn't certain about
where to put the generic code, as well as naming conventions.

Per David Gilbert's suggestion, I was careful to ensure that monitor
calls do not perturb TLB state (see the read-only flag in some
functions).

I appreciate Nadav's suggestion about other ways to pursue the same
goal: I ended up deciding I would like to try my hand at consolidating
the x86 page table code.

Don Porter (6):
  Add an "info pg" command that prints the current page tables
  Convert 'info tlb' to use generic iterator
  Convert 'info mem' to use generic iterator
  Convert x86_cpu_get_memory_mapping() to use generic iterators
  Move tcg implementation of x86 get_physical_address into common helper
code.
  Convert x86_mmu_translate() to use common code.

 hmp-commands-info.hx |  26 +
 include/monitor/hmp-target.h |   1 +
 target/i386/arch_memory_mapping.c| 735 +++---
 target/i386/cpu.h|  66 ++
 target/i386/helper.c | 518 
 target/i386/monitor.c| 883 +--
 target/i386/tcg/sysemu/excp_helper.c | 555 +
 7 files changed, 1439 insertions(+), 1345 deletions(-)

--
2.34.1

[PATCH v2 3/6] Convert 'info mem' to use generic iterator

[Don Porter]

Signed-off-by: Don Porter 
---
 target/i386/monitor.c | 344 +-
 1 file changed, 35 insertions(+), 309 deletions(-)

diff --git a/target/i386/monitor.c b/target/i386/monitor.c
index adf95edfb4..147743392d 100644
--- a/target/i386/monitor.c
+++ b/target/i386/monitor.c
@@ -480,332 +480,58 @@ void hmp_info_tlb(Monitor *mon, const QDict *qdict)
 for_each_pte(cs, &mem_print_tlb, &state, false, false);
 }
 
-static void mem_print(Monitor *mon, CPUArchState *env,
-  hwaddr *pstart, int *plast_prot,
-  hwaddr end, int prot)
-{
-int prot1;
-prot1 = *plast_prot;
-if (prot != prot1) {
-if (*pstart != -1) {
-monitor_printf(mon, HWADDR_FMT_plx "-" HWADDR_FMT_plx " "
-   HWADDR_FMT_plx " %c%c%c\n",
-   addr_canonical(env, *pstart),
-   addr_canonical(env, end),
-   addr_canonical(env, end - *pstart),
-   prot1 & PG_USER_MASK ? 'u' : '-',
-   'r',
-   prot1 & PG_RW_MASK ? 'w' : '-');
-}
-if (prot != 0)
-*pstart = end;
-else
-*pstart = -1;
-*plast_prot = prot;
-}
-}
-
-static void mem_info_32(Monitor *mon, CPUArchState *env)
+static
+bool mem_print(CPUState *cs, struct mem_print_state *state)
 {
-unsigned int l1, l2;
-int prot, last_prot;
-uint32_t pgd, pde, pte;
-hwaddr start, end;
-
-pgd = env->cr[3] & ~0xfff;
-last_prot = 0;
-start = -1;
-for(l1 = 0; l1 < 1024; l1++) {
-cpu_physical_memory_read(pgd + l1 * 4, &pde, 4);
-pde = le32_to_cpu(pde);
-end = l1 << 22;
-if (pde & PG_PRESENT_MASK) {
-if ((pde & PG_PSE_MASK) && (env->cr[4] & CR4_PSE_MASK)) {
-prot = pde & (PG_USER_MASK | PG_RW_MASK | PG_PRESENT_MASK);
-mem_print(mon, env, &start, &last_prot, end, prot);
-} else {
-for(l2 = 0; l2 < 1024; l2++) {
-cpu_physical_memory_read((pde & ~0xfff) + l2 * 4, &pte, 4);
-pte = le32_to_cpu(pte);
-end = (l1 << 22) + (l2 << 12);
-if (pte & PG_PRESENT_MASK) {
-prot = pte & pde &
-(PG_USER_MASK | PG_RW_MASK | PG_PRESENT_MASK);
-} else {
-prot = 0;
-}
-mem_print(mon, env, &start, &last_prot, end, prot);
-}
-}
-} else {
-prot = 0;
-mem_print(mon, env, &start, &last_prot, end, prot);
-}
-}
-/* Flush last range */
-mem_print(mon, env, &start, &last_prot, (hwaddr)1 << 32, 0);
-}
+CPUArchState *env = state->env;
+int i = 0;
 
-static void mem_info_pae32(Monitor *mon, CPUArchState *env)
-{
-unsigned int l1, l2, l3;
-int prot, last_prot;
-uint64_t pdpe, pde, pte;
-uint64_t pdp_addr, pd_addr, pt_addr;
-hwaddr start, end;
-
-pdp_addr = env->cr[3] & ~0x1f;
-last_prot = 0;
-start = -1;
-for (l1 = 0; l1 < 4; l1++) {
-cpu_physical_memory_read(pdp_addr + l1 * 8, &pdpe, 8);
-pdpe = le64_to_cpu(pdpe);
-end = l1 << 30;
-if (pdpe & PG_PRESENT_MASK) {
-pd_addr = pdpe & 0x3f000ULL;
-for (l2 = 0; l2 < 512; l2++) {
-cpu_physical_memory_read(pd_addr + l2 * 8, &pde, 8);
-pde = le64_to_cpu(pde);
-end = (l1 << 30) + (l2 << 21);
-if (pde & PG_PRESENT_MASK) {
-if (pde & PG_PSE_MASK) {
-prot = pde & (PG_USER_MASK | PG_RW_MASK |
-  PG_PRESENT_MASK);
-mem_print(mon, env, &start, &last_prot, end, prot);
-} else {
-pt_addr = pde & 0x3f000ULL;
-for (l3 = 0; l3 < 512; l3++) {
-cpu_physical_memory_read(pt_addr + l3 * 8, &pte, 
8);
-pte = le64_to_cpu(pte);
-end = (l1 << 30) + (l2 << 21) + (l3 << 12);
-if (pte & PG_PRESENT_MASK) {
-prot = pte & pde & (PG_USER_MASK | PG_RW_MASK |
-PG_PRESENT_MASK);
-} else {
-prot = 0;
-}
-mem_print(mon, env, &start, &last_prot, end, prot);
-}
-}
-} else {
-prot = 0;
-mem_print(mon, env, &start, &last_prot, end, prot);
-}
-}
-} else {
-prot = 0;
-mem_print(mon, env, &start

Re: [PATCH] target/i386: disable jmp_opt if EFLAGS.RF is 1

[Richard Henderson]

On 5/24/24 08:33, Paolo Bonzini wrote:

If EFLAGS.RF is 1, special processing in gen_eob_worker() is needed and
therefore goto_tb cannot be used.

Suggested-by: Richard Henderson 
Cc: qemu-sta...@nongnu.org
Signed-off-by: Paolo Bonzini 
---
  target/i386/tcg/translate.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/i386/tcg/translate.c b/target/i386/tcg/translate.c
index 76be7425800..ebcff8766cf 100644
--- a/target/i386/tcg/translate.c
+++ b/target/i386/tcg/translate.c
@@ -4660,7 +4660,7 @@ static void i386_tr_init_disas_context(DisasContextBase 
*dcbase, CPUState *cpu)
  dc->cpuid_7_1_eax_features = env->features[FEAT_7_1_EAX];
  dc->cpuid_xsave_features = env->features[FEAT_XSAVE];
  dc->jmp_opt = !((cflags & CF_NO_GOTO_TB) ||
-(flags & (HF_TF_MASK | HF_INHIBIT_IRQ_MASK)));
+(flags & (HF_RF_MASK | HF_TF_MASK | HF_INHIBIT_IRQ_MASK)));
  /*
   * If jmp_opt, we want to handle each string instruction individually.
   * For icount also disable repz optimization so that each iteration


Reviewed-by: Richard Henderson 


r~

Re: [PATCH] target/i386: always go through gen_eob*()

[Richard Henderson]

On 5/24/24 08:33, Paolo Bonzini wrote:

Using DISAS_NORETURN does not process any of HF_INHIBIT_IRQ_MASK,
HF_RF_MASK or HF_TF_MASK.  Never use it, instead there is
DISAS_EOB_ONLY.

Cc: qemu-sta...@nongnu.org
Signed-off-by: Paolo Bonzini 
---
  target/i386/tcg/translate.c | 18 --
  target/i386/tcg/emit.c.inc  |  4 ++--
  2 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/target/i386/tcg/translate.c b/target/i386/tcg/translate.c
index ebcff8766cf..df10e7d8a6a 100644
--- a/target/i386/tcg/translate.c
+++ b/target/i386/tcg/translate.c
@@ -1406,7 +1406,7 @@ static void gen_exception(DisasContext *s, int trapno)
  gen_update_cc_op(s);
  gen_update_eip_cur(s);
  gen_helper_raise_exception(tcg_env, tcg_constant_i32(trapno));
-s->base.is_jmp = DISAS_NORETURN;
+s->base.is_jmp = DISAS_EOB_ONLY;


This is wrong, because we exit via exception, right here.
Anything you add afterward is unreachable.


  }
  
  /* Generate #UD for the current instruction.  The assumption here is that

@@ -2191,7 +2191,7 @@ static void gen_interrupt(DisasContext *s, uint8_t intno)
  gen_update_eip_cur(s);
  gen_helper_raise_interrupt(tcg_env, tcg_constant_i32(intno),
 cur_insn_len_i32(s));
-s->base.is_jmp = DISAS_NORETURN;
+s->base.is_jmp = DISAS_EOB_ONLY;


Likewise.


  }
  
  static void gen_set_hflag(DisasContext *s, uint32_t mask)

@@ -2354,7 +2354,7 @@ static void gen_jmp_rel(DisasContext *s, MemOp ot, int 
diff, int tb_num)
  tcg_gen_movi_tl(cpu_eip, new_eip);
  }
  tcg_gen_exit_tb(s->base.tb, tb_num);
-s->base.is_jmp = DISAS_NORETURN;
+s->base.is_jmp = DISAS_EOB_ONLY;


This is wrong because exit_tb exits, and anything you add after is unreachable.
I think you simply want to remove the exit_tb call as well, but there may be more cleanup 
possible in the wider context; I haven't checked.



  } else {
  if (!(tb_cflags(s->base.tb) & CF_PCREL)) {
  tcg_gen_movi_tl(cpu_eip, new_eip);
@@ -3520,7 +3520,7 @@ static void disas_insn_old(DisasContext *s, CPUState 
*cpu, int b)
  gen_update_cc_op(s);
  gen_update_eip_cur(s);
  gen_helper_rdpmc(tcg_env);
-s->base.is_jmp = DISAS_NORETURN;
+s->base.is_jmp = DISAS_EOB_ONLY;


This is wrong because helper_rdpmc is noreturn, always raising an exception.



@@ -3690,7 +3690,7 @@ static void disas_insn_old(DisasContext *s, CPUState 
*cpu, int b)
  gen_update_cc_op(s);
  gen_update_eip_cur(s);
  gen_helper_mwait(tcg_env, cur_insn_len_i32(s));
-s->base.is_jmp = DISAS_NORETURN;
+s->base.is_jmp = DISAS_EOB_ONLY;


Likewise.


@@ -3769,7 +3769,7 @@ static void disas_insn_old(DisasContext *s, CPUState 
*cpu, int b)
  gen_helper_vmrun(tcg_env, tcg_constant_i32(s->aflag - 1),
   cur_insn_len_i32(s));
  tcg_gen_exit_tb(NULL, 0);
-s->base.is_jmp = DISAS_NORETURN;
+s->base.is_jmp = DISAS_EOB_ONLY;


Calls exit_tb, which is probably bogus here and EOB_ONLY is correct.
But I'd need to look deeper into what vmrun does.


  switch (dc->base.is_jmp) {
  case DISAS_NORETURN:
+   /*
+* Nothing to do, gen_eob*() was already called.  DISAS_NORETURN is
+* never set explicitly except in gen_eob_worker(), because that is
+* where HF_INHIBIT_IRQ_MASK, HF_RF_MASK and HF_TF_MASK are handled.
+*/


Comment is wrong because exceptions *should* set NORETURN.
All of the masks are irrelevant to #gp or #ud etc.



@@ -1642,7 +1642,7 @@ static void gen_HLT(DisasContext *s, CPUX86State *env, 
X86DecodedInsn *decode)
  gen_update_cc_op(s);
  gen_update_eip_cur(s);
  gen_helper_hlt(tcg_env, cur_insn_len_i32(s));
-s->base.is_jmp = DISAS_NORETURN;
+s->base.is_jmp = DISAS_EOB_ONLY;


noreturn.


@@ -4022,7 +4022,7 @@ static void gen_XCHG(DisasContext *s, CPUX86State *env, 
X86DecodedInsn *decode)
  gen_update_cc_op(s);
  gen_update_eip_cur(s);
  gen_helper_pause(tcg_env, cur_insn_len_i32(s));
-s->base.is_jmp = DISAS_NORETURN;
+s->base.is_jmp = DISAS_EOB_ONLY;


noreturn.


r~

[PATCH 2/5] meson: remove unnecessary dependency

[Paolo Bonzini]

The dbus_display1_dep is not really used since all occurrences also
request gio independently.  Just list the generated sources and drop
dbus_display1_dep.

Signed-off-by: Paolo Bonzini 
---
 audio/meson.build   | 4 ++--
 tests/qtest/meson.build | 2 +-
 ui/meson.build  | 5 ++---
 3 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/audio/meson.build b/audio/meson.build
index 608f35e6af7..59f0a431d51 100644
--- a/audio/meson.build
+++ b/audio/meson.build
@@ -30,8 +30,8 @@ endforeach
 
 if dbus_display
 module_ss = ss.source_set()
-module_ss.add(when: [gio, dbus_display1_dep, pixman],
-  if_true: files('dbusaudio.c'))
+module_ss.add(when: [gio, pixman],
+  if_true: [dbus_display1, files('dbusaudio.c')])
 audio_modules += {'dbus': module_ss}
 endif
 
diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
index 86293051dce..b98fae6a6dd 100644
--- a/tests/qtest/meson.build
+++ b/tests/qtest/meson.build
@@ -354,7 +354,7 @@ if vnc.found()
 endif
 
 if dbus_display
-  qtests += {'dbus-display-test': [dbus_display1_dep, gio]}
+  qtests += {'dbus-display-test': [dbus_display1, gio]}
 endif
 
 qtest_executables = {}
diff --git a/ui/meson.build b/ui/meson.build
index 5d89986b0ee..cfbf29428df 100644
--- a/ui/meson.build
+++ b/ui/meson.build
@@ -91,8 +91,7 @@ if dbus_display
   '--interface-prefix', 'org.qemu.',
   '--c-namespace', 'QemuDBus',
   '--generate-c-code', '@BASENAME@'])
-  dbus_display1_dep = declare_dependency(sources: dbus_display1, dependencies: 
gio)
-  dbus_ss.add(when: [gio, dbus_display1_dep],
+  dbus_ss.add(when: gio,
   if_true: [files(
 'dbus-chardev.c',
 'dbus-clipboard.c',
@@ -100,7 +99,7 @@ if dbus_display
 'dbus-error.c',
 'dbus-listener.c',
 'dbus.c',
-  ), opengl, gbm, pixman])
+  ), opengl, gbm, pixman, dbus_display1])
   ui_modules += {'dbus' : dbus_ss}
 endif
 
-- 
2.45.1

[PATCH 1/5] meson: remove unnecessary reference to libm

[Paolo Bonzini]

libm is linked into all targets via libqemuutil, no need to specify it
explicitly.

Signed-off-by: Paolo Bonzini 
---
 block/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/meson.build b/block/meson.build
index e1f03fd773e..8993055c75e 100644
--- a/block/meson.build
+++ b/block/meson.build
@@ -110,7 +110,7 @@ foreach m : [
   [blkio, 'blkio', files('blkio.c')],
   [curl, 'curl', files('curl.c')],
   [glusterfs, 'gluster', files('gluster.c')],
-  [libiscsi, 'iscsi', [files('iscsi.c'), libm]],
+  [libiscsi, 'iscsi', files('iscsi.c')],
   [libnfs, 'nfs', files('nfs.c')],
   [libssh, 'ssh', files('ssh.c')],
   [rbd, 'rbd', files('rbd.c')],
-- 
2.45.1

[PATCH 4/5] meson: do not query modules before they are processed

[Paolo Bonzini]

Signed-off-by: Paolo Bonzini 
---
 block/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/meson.build b/block/meson.build
index 8993055c75e..158dc3b89db 100644
--- a/block/meson.build
+++ b/block/meson.build
@@ -119,7 +119,7 @@ foreach m : [
 module_ss = ss.source_set()
 module_ss.add(when: m[0], if_true: m[2])
 if enable_modules
-  modsrc += module_ss.all_sources()
+  modsrc += m[2]
 endif
 block_modules += {m[1] : module_ss}
   endif
-- 
2.45.1

[PATCH 5/5] migration: remove unnecessary zlib dependency

[Paolo Bonzini]

zlib code is only used by the emulators, not by the tests.

Signed-off-by: Paolo Bonzini 
---
 meson.build   | 2 +-
 migration/dirtyrate.c | 1 -
 migration/qemu-file.c | 1 -
 migration/meson.build | 2 +-
 4 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/meson.build b/meson.build
index 7fd82b5f48c..63866071445 100644
--- a/meson.build
+++ b/meson.build
@@ -3696,7 +3696,7 @@ libmigration = static_library('migration', sources: 
migration_files + genh,
   name_suffix: 'fa',
   build_by_default: false)
 migration = declare_dependency(link_with: libmigration,
-   dependencies: [zlib, qom, io])
+   dependencies: [qom, io])
 system_ss.add(migration)
 
 block_ss = block_ss.apply({})
diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c
index d02d70b7b4b..1d9db812990 100644
--- a/migration/dirtyrate.c
+++ b/migration/dirtyrate.c
@@ -12,7 +12,6 @@
 
 #include "qemu/osdep.h"
 #include "qemu/error-report.h"
-#include 
 #include "hw/core/cpu.h"
 #include "qapi/error.h"
 #include "exec/ramblock.h"
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 9ccbbb00991..b6d2f588bd7 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -22,7 +22,6 @@
  * THE SOFTWARE.
  */
 #include "qemu/osdep.h"
-#include 
 #include "qemu/madvise.h"
 #include "qemu/error-report.h"
 #include "qemu/iov.h"
diff --git a/migration/meson.build b/migration/meson.build
index 8815f808374..bdc3244bce0 100644
--- a/migration/meson.build
+++ b/migration/meson.build
@@ -29,7 +29,7 @@ system_ss.add(files(
   'socket.c',
   'tls.c',
   'threadinfo.c',
-), gnutls)
+), gnutls, zlib)
 
 if get_option('replication').allowed()
   system_ss.add(files('colo-failover.c', 'colo.c'))
-- 
2.45.1

[PATCH 3/5] tcg: include dependencies in static_library()

[Paolo Bonzini]

This ensures that for example libffi can be reached even if it is not
in /usr/include.

Signed-off-by: Paolo Bonzini 
---
 tcg/meson.build | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tcg/meson.build b/tcg/meson.build
index 8251589fd4e..ffbe754d8b3 100644
--- a/tcg/meson.build
+++ b/tcg/meson.build
@@ -32,19 +32,19 @@ tcg_ss = tcg_ss.apply({})
 libtcg_user = static_library('tcg_user',
  tcg_ss.sources() + genh,
  name_suffix: 'fa',
+ dependencies: tcg_ss.dependencies(),
  c_args: '-DCONFIG_USER_ONLY',
  build_by_default: false)
 
-tcg_user = declare_dependency(link_with: libtcg_user,
-  dependencies: tcg_ss.dependencies())
+tcg_user = declare_dependency(link_with: libtcg_user)
 user_ss.add(tcg_user)
 
 libtcg_system = static_library('tcg_system',
 tcg_ss.sources() + genh,
 name_suffix: 'fa',
+dependencies: tcg_ss.dependencies(),
 c_args: '-DCONFIG_SOFTMMU',
 build_by_default: false)
 
-tcg_system = declare_dependency(link_with: libtcg_system,
- dependencies: tcg_ss.dependencies())
+tcg_system = declare_dependency(link_with: libtcg_system)
 system_ss.add(tcg_system)
-- 
2.45.1

[PATCH 0/5] meson: small cleanups

[Paolo Bonzini]

All found while looking at Akihiko's issues with declare_dependency(
objects: ...).

https://patchew.org/QEMU/20240524-objects-v1-0-07cbbe961...@daynix.com/

Paolo Bonzini (5):
  meson: remove unnecessary reference to libm
  meson: remove unnecessary dependency
  tcg: include dependencies in static_library()
  meson: do not query modules before they are processed
  migration: remove unnecessary zlib dependency

 meson.build | 2 +-
 migration/dirtyrate.c   | 1 -
 migration/qemu-file.c   | 1 -
 audio/meson.build   | 4 ++--
 block/meson.build   | 4 ++--
 migration/meson.build   | 2 +-
 tcg/meson.build | 8 
 tests/qtest/meson.build | 2 +-
 ui/meson.build  | 5 ++---
 9 files changed, 13 insertions(+), 16 deletions(-)

-- 
2.45.1

Re: [RFC PATCH 1/4] target/riscv/kvm: add software breakpoints support

[Paolo Bonzini]

On Tue, Apr 16, 2024 at 11:23 AM Daniel Henrique Barboza
 wrote:
> > +int kvm_arch_insert_sw_breakpoint(CPUState *cs, struct kvm_sw_breakpoint 
> > *bp,
> > +  vaddr len)
> > +{
> > +if (len != 4 && len != 2) {
> > +return -EINVAL;
> > +}
>
> I wonder if this verification should be moved to kvm_insert_breakpoint(). Is
> there any known reason why other archs would use 'len' other than 2 or 4? The
> parent function can throw the EINVAL in this case. Otherwise all callers from
> all archs will need a similar EINVAL check.

I'm not sure how len is defined in the gdb protocol, but x86 has a
breakpoint length of 1 and an instruction length that can be any value
between 1 and 15.

Most architectures could assume that it's always one value, i.e. just
not care about checking len in kvm_arch_insert_sw_breakpoint.

The patches look good, feel free to take them through the RISC-V tree.

One thing that I was wondering is: could RISC-V just use always
c.ebreak if C instructions are supported, and ebreak if they're not?
But if for example that would that mess up the synchronization of the
disassembly in gdb, it's a good reason to add the len argument as you
did here.

Paolo

[PATCH] target/i386: always go through gen_eob*()

[Paolo Bonzini]

Using DISAS_NORETURN does not process any of HF_INHIBIT_IRQ_MASK,
HF_RF_MASK or HF_TF_MASK.  Never use it, instead there is
DISAS_EOB_ONLY.

Cc: qemu-sta...@nongnu.org
Signed-off-by: Paolo Bonzini 
---
 target/i386/tcg/translate.c | 18 --
 target/i386/tcg/emit.c.inc  |  4 ++--
 2 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/target/i386/tcg/translate.c b/target/i386/tcg/translate.c
index ebcff8766cf..df10e7d8a6a 100644
--- a/target/i386/tcg/translate.c
+++ b/target/i386/tcg/translate.c
@@ -1406,7 +1406,7 @@ static void gen_exception(DisasContext *s, int trapno)
 gen_update_cc_op(s);
 gen_update_eip_cur(s);
 gen_helper_raise_exception(tcg_env, tcg_constant_i32(trapno));
-s->base.is_jmp = DISAS_NORETURN;
+s->base.is_jmp = DISAS_EOB_ONLY;
 }
 
 /* Generate #UD for the current instruction.  The assumption here is that
@@ -2191,7 +2191,7 @@ static void gen_interrupt(DisasContext *s, uint8_t intno)
 gen_update_eip_cur(s);
 gen_helper_raise_interrupt(tcg_env, tcg_constant_i32(intno),
cur_insn_len_i32(s));
-s->base.is_jmp = DISAS_NORETURN;
+s->base.is_jmp = DISAS_EOB_ONLY;
 }
 
 static void gen_set_hflag(DisasContext *s, uint32_t mask)
@@ -2354,7 +2354,7 @@ static void gen_jmp_rel(DisasContext *s, MemOp ot, int 
diff, int tb_num)
 tcg_gen_movi_tl(cpu_eip, new_eip);
 }
 tcg_gen_exit_tb(s->base.tb, tb_num);
-s->base.is_jmp = DISAS_NORETURN;
+s->base.is_jmp = DISAS_EOB_ONLY;
 } else {
 if (!(tb_cflags(s->base.tb) & CF_PCREL)) {
 tcg_gen_movi_tl(cpu_eip, new_eip);
@@ -3520,7 +3520,7 @@ static void disas_insn_old(DisasContext *s, CPUState 
*cpu, int b)
 gen_update_cc_op(s);
 gen_update_eip_cur(s);
 gen_helper_rdpmc(tcg_env);
-s->base.is_jmp = DISAS_NORETURN;
+s->base.is_jmp = DISAS_EOB_ONLY;
 break;
 case 0x134: /* sysenter */
 /* For AMD SYSENTER is not valid in long mode */
@@ -3690,7 +3690,7 @@ static void disas_insn_old(DisasContext *s, CPUState 
*cpu, int b)
 gen_update_cc_op(s);
 gen_update_eip_cur(s);
 gen_helper_mwait(tcg_env, cur_insn_len_i32(s));
-s->base.is_jmp = DISAS_NORETURN;
+s->base.is_jmp = DISAS_EOB_ONLY;
 break;
 
 case 0xca: /* clac */
@@ -3769,7 +3769,7 @@ static void disas_insn_old(DisasContext *s, CPUState 
*cpu, int b)
 gen_helper_vmrun(tcg_env, tcg_constant_i32(s->aflag - 1),
  cur_insn_len_i32(s));
 tcg_gen_exit_tb(NULL, 0);
-s->base.is_jmp = DISAS_NORETURN;
+s->base.is_jmp = DISAS_EOB_ONLY;
 break;
 
 case 0xd9: /* VMMCALL */
@@ -4770,6 +4770,11 @@ static void i386_tr_tb_stop(DisasContextBase *dcbase, 
CPUState *cpu)
 
 switch (dc->base.is_jmp) {
 case DISAS_NORETURN:
+   /*
+* Nothing to do, gen_eob*() was already called.  DISAS_NORETURN is
+* never set explicitly except in gen_eob_worker(), because that is
+* where HF_INHIBIT_IRQ_MASK, HF_RF_MASK and HF_TF_MASK are handled.
+*/
 break;
 case DISAS_TOO_MANY:
 gen_update_cc_op(dc);
@@ -4793,6 +4798,7 @@ static void i386_tr_tb_stop(DisasContextBase *dcbase, 
CPUState *cpu)
 default:
 g_assert_not_reached();
 }
+assert(dc->base.is_jmp == DISAS_NORETURN);
 }
 
 static const TranslatorOps i386_tr_ops = {
diff --git a/target/i386/tcg/emit.c.inc b/target/i386/tcg/emit.c.inc
index c78e35b1e28..14464074d5a 100644
--- a/target/i386/tcg/emit.c.inc
+++ b/target/i386/tcg/emit.c.inc
@@ -1642,7 +1642,7 @@ static void gen_HLT(DisasContext *s, CPUX86State *env, 
X86DecodedInsn *decode)
 gen_update_cc_op(s);
 gen_update_eip_cur(s);
 gen_helper_hlt(tcg_env, cur_insn_len_i32(s));
-s->base.is_jmp = DISAS_NORETURN;
+s->base.is_jmp = DISAS_EOB_ONLY;
 #endif
 }
 
@@ -4022,7 +4022,7 @@ static void gen_XCHG(DisasContext *s, CPUX86State *env, 
X86DecodedInsn *decode)
 gen_update_cc_op(s);
 gen_update_eip_cur(s);
 gen_helper_pause(tcg_env, cur_insn_len_i32(s));
-s->base.is_jmp = DISAS_NORETURN;
+s->base.is_jmp = DISAS_EOB_ONLY;
 }
 /* No writeback.  */
 decode->op[0].unit = X86_OP_SKIP;
-- 
2.45.1

[PATCH] target/i386: disable jmp_opt if EFLAGS.RF is 1

[Paolo Bonzini]

If EFLAGS.RF is 1, special processing in gen_eob_worker() is needed and
therefore goto_tb cannot be used.

Suggested-by: Richard Henderson 
Cc: qemu-sta...@nongnu.org
Signed-off-by: Paolo Bonzini 
---
 target/i386/tcg/translate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/i386/tcg/translate.c b/target/i386/tcg/translate.c
index 76be7425800..ebcff8766cf 100644
--- a/target/i386/tcg/translate.c
+++ b/target/i386/tcg/translate.c
@@ -4660,7 +4660,7 @@ static void i386_tr_init_disas_context(DisasContextBase 
*dcbase, CPUState *cpu)
 dc->cpuid_7_1_eax_features = env->features[FEAT_7_1_EAX];
 dc->cpuid_xsave_features = env->features[FEAT_XSAVE];
 dc->jmp_opt = !((cflags & CF_NO_GOTO_TB) ||
-(flags & (HF_TF_MASK | HF_INHIBIT_IRQ_MASK)));
+(flags & (HF_RF_MASK | HF_TF_MASK | HF_INHIBIT_IRQ_MASK)));
 /*
  * If jmp_opt, we want to handle each string instruction individually.
  * For icount also disable repz optimization so that each iteration
-- 
2.45.1

Re: [PATCH v2 1/3] hw/riscv/virt: Add memory hotplugging and virtio-md-pci support

[Daniel Henrique Barboza]

On 5/24/24 12:02, David Hildenbrand wrote:

On 24.05.24 15:14, Daniel Henrique Barboza wrote:



On 5/21/24 07:56, Björn Töpel wrote:

From: Björn Töpel 

Virtio-based memory devices (virtio-mem/virtio-pmem) allows for
dynamic resizing of virtual machine memory, and requires proper
hotplugging (add/remove) support to work.

Add device memory support for RISC-V "virt" machine, and enable
virtio-md-pci with the corresponding missing hotplugging callbacks.

Signed-off-by: Björn Töpel 
---
   hw/riscv/Kconfig   |  2 +
   hw/riscv/virt.c    | 83 +-
   hw/virtio/virtio-mem.c |  5 ++-
   3 files changed, 87 insertions(+), 3 deletions(-)

diff --git a/hw/riscv/Kconfig b/hw/riscv/Kconfig
index a2030e3a6ff0..08f82dbb681a 100644
--- a/hw/riscv/Kconfig
+++ b/hw/riscv/Kconfig
@@ -56,6 +56,8 @@ config RISCV_VIRT
   select PLATFORM_BUS
   select ACPI
   select ACPI_PCI
+    select VIRTIO_MEM_SUPPORTED
+    select VIRTIO_PMEM_SUPPORTED
   config SHAKTI_C
   bool
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 4fdb66052587..443902f919d2 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -53,6 +53,8 @@
   #include "hw/pci-host/gpex.h"
   #include "hw/display/ramfb.h"
   #include "hw/acpi/aml-build.h"
+#include "hw/mem/memory-device.h"
+#include "hw/virtio/virtio-mem-pci.h"
   #include "qapi/qapi-visit-common.h"
   #include "hw/virtio/virtio-iommu.h"
@@ -1407,6 +1409,7 @@ static void virt_machine_init(MachineState *machine)
   DeviceState *mmio_irqchip, *virtio_irqchip, *pcie_irqchip;
   int i, base_hartid, hart_count;
   int socket_count = riscv_socket_count(machine);
+    hwaddr device_memory_base, device_memory_size;
   /* Check socket count limit */
   if (VIRT_SOCKETS_MAX < socket_count) {
@@ -1420,6 +1423,12 @@ static void virt_machine_init(MachineState *machine)
   exit(1);
   }
+    if (machine->ram_slots > ACPI_MAX_RAM_SLOTS) {
+    error_report("unsupported amount of memory slots: %"PRIu64,
+ machine->ram_slots);
+    exit(EXIT_FAILURE);
+    }
+
   /* Initialize sockets */
   mmio_irqchip = virtio_irqchip = pcie_irqchip = NULL;
   for (i = 0; i < socket_count; i++) {
@@ -1553,6 +1562,37 @@ static void virt_machine_init(MachineState *machine)
   memory_region_add_subregion(system_memory, memmap[VIRT_MROM].base,
   mask_rom);
+    /* device memory */
+    device_memory_base = ROUND_UP(s->memmap[VIRT_DRAM].base + 
machine->ram_size,
+  GiB);
+    device_memory_size = machine->maxram_size - machine->ram_size;
+    if (device_memory_size > 0) {
+    /*
+ * Each DIMM is aligned based on the backend's alignment value.
+ * Assume max 1G hugepage alignment per slot.
+ */
+    device_memory_size += machine->ram_slots * GiB;


We don't need to align to 1GiB. This calc can use 2MiB instead (or 4MiB if we're
running 32 bits).


+
+    if (riscv_is_32bit(&s->soc[0])) {
+    hwaddr memtop = device_memory_base + ROUND_UP(device_memory_size,
+  GiB);


Same here - alignment is 2/4 MiB.


+
+    if (memtop > UINT32_MAX) {
+    error_report("memory exceeds 32-bit limit by %lu bytes",
+ memtop - UINT32_MAX);
+    exit(EXIT_FAILURE);
+    }
+    }
+
+    if (device_memory_base + device_memory_size < device_memory_size) {
+    error_report("unsupported amount of device memory");
+    exit(EXIT_FAILURE);
+    }


Took another look and found this a bit strange. These are all unsigned vars, so
if (unsigned a + unsigned b < unsigned b) will always be 'false'. The compiler 
is
probably cropping this out.


No. Unsigned interger overflow is defined behavior and this is a common check 
to detect such overflow. tI's consistent with what we do for other 
architectures.



Oh, ok. We're so far away from UINT64_MAX that it didn't occur to me doing an 
overflow
check here. Fair enough.

 


The calc we need to do is to ensure that the extra ram_slots * alignment will 
fit into
the VIRT_DRAM block, i.e. maxram_size + (ram_slots * alignment) < 
memmap[VIRT_DRAM].size.


TBH I'm starting to have second thoughts about letting users hotplug whatever 
they want.
It seems cleaner to just force the 2/4 Mb alignment in pre_plug() and be done 
with it,
no need to allocate ram_slots * alignment and doing all these extra checks.


It's worth noting that if user space decides to specify addresses manually, it 
can mess up everything already. There are other events that can result in 
fragmentation of the memory device area (repeated hot(un)plug of differing 
DIMMs).

Assume you have 1 GiB range and hotplug a 512 MiB DIMM at offset 256 MiB. You 
won't be able to hotplug another 512 MiB DIMM even though we reserved space.

My take so far is: if the user want

Re: [PATCH 03/16] target/i386: document and group DISAS_* constants

[Paolo Bonzini]

On Fri, May 24, 2024 at 5:13 PM Richard Henderson
 wrote:
>
> On 5/24/24 08:04, Paolo Bonzini wrote:
> > ... and nope, it's the other way round - DISAS_NORETURN is a bug
> > waiting to happen for x86 translation because it doesn't process any
> > of HF_INHIBIT_IRQ_MASK, HF_RF_MASK or HF_TF_MASK.
>
> Do you need to suppress use_goto_tb in these cases?

HF_INHIBIT_IRQ_MASK and HF_TF_MASK already do, HF_RF_MASK is missing.
Nice catch.

Paolo

Re: [PATCH v3 2/2] cxl/core: add poison creation event handler

[Shiyang Ruan via]

在 2024/5/22 14:45, Dan Williams 写道:

Shiyang Ruan wrote:
[..]

My expectation is MF_ACTION_REQUIRED is not appropriate for CXL event
reported errors since action is only required for direct consumption
events and those need not be reported through the device event queue.

Got it.


I'm not very sure about 'Host write/read' type.  In my opinion, these
two types of event should be sent from device when CPU is accessing a
bad memory address, they could be thought of a sync event which needs


Hmm, no that's not my understanding of a sync event. I expect when error
notifications are synchronous the CPU is guaranteed not to make forward
progress past the point of encountering the error. MSI-signaled
component-events are always asynchronous by that definition because the
CPU is free running while the interrupt is in-flight.


Understood.  In OS-First path, it couldn't be a sync event.




the 'MF_ACTION_REQUIRED' flag.  Then, we can determine the flag by the
types like this:
- CXL_EVENT_TRANSACTION_READ | CXL_EVENT_TRANSACTION_WRITE
=> MF_ACTION_REQUIRED
- CXL_EVENT_TRANSACTION_INJECT_POISON => MF_SW_SIMULATED
- others  => 0


I doubt any reasonable policy can be inferred from the transaction type.
Consider that the CPU itself does not take a sychronous exception when
writes encounter poison. At most those are flagged via CMCI
(corrected machine check interrupt). The only events that cause
exceptions are CPU reads that consume poison. The device has no idea
whether read events are coming from a CPU or a DMA event.

MF_SW_SIMULATED is purely for software simulated poison events as
injected poison can stil cause system fatal damage if the poison is
ingested in an unrecoverable path.

So, I think all CXL poison notification events should trigger an action
optional memory_failure(). I expect this needs to make sure that
duplicates re not a problem. I.e. in the case of CPU consumption of CXL
poison, that causes a synchronous MF_ACTION_REQUIRED event via the MCE
path *and* it may trigger the device to send an error record for the
same page. As far as I can see, duplicate reports (MCE + CXL device) are
unavoidable.


I think my previous understanding about MCE was wrong.  Here is my 
current understanding after some research:


Since CXL device is a memory device, while CPU consumes a poison page of 
CXL device, it always triggers a MCE by interrupt (INT18), no matter 
which-First path is configured.  This is the first report.  Then 
currently, in FW-First path, the poison event is transferred according 
to the following process: CXL device -> firmware -> OS:ACPI->APEI->GHES 
-> MCE.  This is the second one.  These two MCEs represent the same 
poisoning page, which is the so-called "duplicate report", right?  Now, 
the memory_failure() handling I'm trying to add in OS-First path, is 
also another duplicate report.


So, the primary issue to be solved is the second MCE report.  As you 
suggested, make it a optional action.


Please correct me if I'm wrong.  Thank you very much!

--
Ruan.