[PATCH] D73786: [ARM,MVE] Fix vreinterpretq in big-endian mode.

[Simon Tatham via Phabricator via cfe-commits]

simon_tatham created this revision.
simon_tatham added reviewers: dmgreen, MarkMurrayARM, miyuki, ostannard.
Herald added subscribers: llvm-commits, cfe-commits, hiraditya, kristof.beyls.
Herald added projects: clang, LLVM.

In big-endian MVE, the simple vector load/store instructions (i.e.
both contiguous and non-widening) don't all store the bytes of a
register to memory in the same order: it matters whether you did a
VSTRB.8, VSTRH.16 or VSTRW.32. Put another way, the in-register
formats of different vector types relate to each other in a different
way from the in-memory formats.

So, if you want to 'bitcast' or 'reinterpret' one vector type as
another, you have to carefully specify which you mean: did you want to
reinterpret the //register// format of one type as that of the other,
or the //memory// format?

The ACLE `vreinterpretq` intrinsics are specified to reinterpret the
register format. But I had implemented them as LLVM IR bitcast, which
is specified for all types as a reinterpretation of the memory format.
So a `vreinterpretq` intrinsic, applied to values already in registers,
would code-generate incorrectly if compiled big-endian: instead of
emitting no code, it would emit a `vrev`.

To fix this, I've introduced a new IR intrinsic to perform a
register-format reinterpretation: `@llvm.arm.mve.vreinterpretq`. It's
implemented by a trivial isel pattern that expects the input in an
MQPR register, and just returns it unchanged.

In the clang codegen, I only emit this new intrinsic where it's
actually needed: I prefer a bitcast wherever it will have the right
effect, because LLVM understands bitcasts better. So we still generate
bitcasts in little-endian mode, and even in big-endian when you're
casting between two vector types with the same lane size.

For testing, I've moved all the codegen tests of vreinterpretq out
into their own file, so that they can have a different set of RUN
lines to check both big- and little-endian.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D73786

Files:
  clang/include/clang/Basic/arm_mve.td
  clang/include/clang/Basic/arm_mve_defs.td
  clang/lib/CodeGen/CGBuiltin.cpp
  clang/test/CodeGen/arm-mve-intrinsics/admin.c
  clang/test/CodeGen/arm-mve-intrinsics/reinterpret.c
  llvm/include/llvm/IR/IntrinsicsARM.td
  llvm/lib/Target/ARM/ARMISelLowering.cpp
  llvm/lib/Target/ARM/ARMISelLowering.h
  llvm/lib/Target/ARM/ARMInstrMVE.td
  llvm/test/CodeGen/Thumb2/mve-be.ll


Index: llvm/test/CodeGen/Thumb2/mve-be.ll
===================================================================
--- llvm/test/CodeGen/Thumb2/mve-be.ll
+++ llvm/test/CodeGen/Thumb2/mve-be.ll
@@ -295,3 +295,64 @@
   %3 = tail call <4 x i32> asm sideeffect "  VMULLB.s32 $0, $1, $1", "=&w,w"(<4 x i32> %2) #2
   ret <4 x i32> %3
 }
+
+; Test case demonstrating that 'bitcast' reinterprets the memory format of a
+; vector, as if stored and then loaded. So if it has to go between two
+; operations treating a register as having different lane sizes, then in
+; big-endian mode, it has to emit a vrev32.16, which is equivalent to the
+; effect that vstrw.32 followed by vldrh.16 would have.
+define arm_aapcs_vfpcc void @test_bitcast(<4 x i32>* readonly %in, <8 x i16>* %out) {
+; CHECK-LE-LABEL: test_bitcast:
+; CHECK-LE:       @ %bb.0: @ %entry
+; CHECK-LE-NEXT:    vldrw.u32 q0, [r0]
+; CHECK-LE-NEXT:    vmul.i32 q0, q0, q0
+; CHECK-LE-NEXT:    vmul.i16 q0, q0, q0
+; CHECK-LE-NEXT:    vstrw.32 q0, [r1]
+; CHECK-LE-NEXT:    bx lr
+;
+; CHECK-BE-LABEL: test_bitcast:
+; CHECK-BE:       @ %bb.0: @ %entry
+; CHECK-BE-NEXT:    vldrw.u32 q0, [r0]
+; CHECK-BE-NEXT:    vmul.i32 q0, q0, q0
+; CHECK-BE-NEXT:    vrev32.16 q0, q0
+; CHECK-BE-NEXT:    vmul.i16 q0, q0, q0
+; CHECK-BE-NEXT:    vstrh.16 q0, [r1]
+; CHECK-BE-NEXT:    bx lr
+entry:
+  %vin = load <4 x i32>, <4 x i32>* %in, align 8
+  %vdbl = mul <4 x i32> %vin, %vin
+  %cast = bitcast <4 x i32> %vdbl to <8 x i16>
+  %cdbl = mul <8 x i16> %cast, %cast
+  store <8 x i16> %cdbl, <8 x i16>* %out, align 8
+  ret void
+}
+
+; Similar test case but using the arm.mve.vreinterpretq intrinsic instead,
+; which is defined to reinterpret the in-register format, so it generates no
+; instruction in either endianness.
+define arm_aapcs_vfpcc void @test_vreinterpretq(<4 x i32>* readonly %in, <8 x i16>* %out) {
+; CHECK-LE-LABEL: test_vreinterpretq:
+; CHECK-LE:       @ %bb.0: @ %entry
+; CHECK-LE-NEXT:    vldrw.u32 q0, [r0]
+; CHECK-LE-NEXT:    vmul.i32 q0, q0, q0
+; CHECK-LE-NEXT:    vmul.i16 q0, q0, q0
+; CHECK-LE-NEXT:    vstrw.32 q0, [r1]
+; CHECK-LE-NEXT:    bx lr
+;
+; CHECK-BE-LABEL: test_vreinterpretq:
+; CHECK-BE:       @ %bb.0: @ %entry
+; CHECK-BE-NEXT:    vldrw.u32 q0, [r0]
+; CHECK-BE-NEXT:    vmul.i32 q0, q0, q0
+; CHECK-BE-NEXT:    vmul.i16 q0, q0, q0
+; CHECK-BE-NEXT:    vstrh.16 q0, [r1]
+; CHECK-BE-NEXT:    bx lr
+entry:
+  %vin = load <4 x i32>, <4 x i32>* %in, align 8
+  %vdbl = mul <4 x i32> %vin, %vin
+  %cast = call <8 x i16> @llvm.arm.mve.vreinterpretq.v8i16.v4i32(<4 x i32> %vdbl)
+  %cdbl = mul <8 x i16> %cast, %cast
+  store <8 x i16> %cdbl, <8 x i16>* %out, align 8
+  ret void
+}
+
+declare <8 x i16> @llvm.arm.mve.vreinterpretq.v8i16.v4i32(<4 x i32>)
Index: llvm/lib/Target/ARM/ARMInstrMVE.td
===================================================================
--- llvm/lib/Target/ARM/ARMInstrMVE.td
+++ llvm/lib/Target/ARM/ARMInstrMVE.td
@@ -3961,6 +3961,7 @@
 // vector types (v4i1<>v8i1, etc.) also as part of lowering vector shuffles.
 
 def predicate_cast : SDNode<"ARMISD::PREDICATE_CAST", SDTUnaryOp>;
+def vector_reg_cast : SDNode<"ARMISD::VECTOR_REG_CAST", SDTUnaryOp>;
 
 let Predicates = [HasMVEInt] in {
   foreach VT = [ v4i1, v8i1, v16i1 ] in {
@@ -3973,6 +3974,10 @@
       def : Pat<(VT  (predicate_cast (VT2 VCCR:$src))),
                 (VT  (COPY_TO_REGCLASS (VT2 VCCR:$src), VCCR))>;
   }
+
+  foreach VT = [ v16i8, v8i16, v8f16, v4i32, v4f32, v2i64, v2f64 ] in
+    foreach VT2 = [ v16i8, v8i16, v8f16, v4i32, v4f32, v2i64, v2f64 ] in
+      def : Pat<(VT (vector_reg_cast (VT2 MQPR:$src))), (VT MQPR:$src)>;
 }
 
 // end of MVE compares
Index: llvm/lib/Target/ARM/ARMISelLowering.h
===================================================================
--- llvm/lib/Target/ARM/ARMISelLowering.h
+++ llvm/lib/Target/ARM/ARMISelLowering.h
@@ -131,6 +131,7 @@
       LE,           // Low-overhead loops, Loop End
 
       PREDICATE_CAST, // Predicate cast for MVE i1 types
+      VECTOR_REG_CAST, // Reinterpret the current contents of a vector register
 
       VCMP,         // Vector compare.
       VCMPZ,        // Vector compare to zero.
Index: llvm/lib/Target/ARM/ARMISelLowering.cpp
===================================================================
--- llvm/lib/Target/ARM/ARMISelLowering.cpp
+++ llvm/lib/Target/ARM/ARMISelLowering.cpp
@@ -1605,6 +1605,7 @@
   case ARMISD::WIN__DBZCHK:   return "ARMISD::WIN__DBZCHK";
 
   case ARMISD::PREDICATE_CAST: return "ARMISD::PREDICATE_CAST";
+  case ARMISD::VECTOR_REG_CAST: return "ARMISD::VECTOR_REG_CAST";
   case ARMISD::VCMP:          return "ARMISD::VCMP";
   case ARMISD::VCMPZ:         return "ARMISD::VCMPZ";
   case ARMISD::VTST:          return "ARMISD::VTST";
@@ -3777,6 +3778,9 @@
   case Intrinsic::arm_mve_pred_v2i:
     return DAG.getNode(ARMISD::PREDICATE_CAST, SDLoc(Op), Op.getValueType(),
                        Op.getOperand(1));
+  case Intrinsic::arm_mve_vreinterpretq:
+    return DAG.getNode(ARMISD::VECTOR_REG_CAST, SDLoc(Op), Op.getValueType(),
+                       Op.getOperand(1));
   }
 }
 
Index: llvm/include/llvm/IR/IntrinsicsARM.td
===================================================================
--- llvm/include/llvm/IR/IntrinsicsARM.td
+++ llvm/include/llvm/IR/IntrinsicsARM.td
@@ -795,6 +795,8 @@
   [llvm_anyvector_ty], [llvm_i32_ty], [IntrNoMem]>;
 def int_arm_mve_pred_v2i : Intrinsic<
   [llvm_i32_ty], [llvm_anyvector_ty], [IntrNoMem]>;
+def int_arm_mve_vreinterpretq : Intrinsic<
+  [llvm_anyvector_ty], [llvm_anyvector_ty], [IntrNoMem]>;
 
 multiclass IntrinsicSignSuffix<list<LLVMType> rets, list<LLVMType> params = [],
                                     list<IntrinsicProperty> props = [],
Index: clang/test/CodeGen/arm-mve-intrinsics/reinterpret.c
===================================================================
--- /dev/null
+++ clang/test/CodeGen/arm-mve-intrinsics/reinterpret.c
@@ -0,0 +1,1629 @@
+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
+// RUN: %clang_cc1 -triple thumbv8.1m.main-arm-none-eabi -target-feature +mve.fp -mfloat-abi hard -fallow-half-arguments-and-returns -O0 -disable-O0-optnone -S -emit-llvm -o - %s | opt -S -mem2reg -sroa -early-cse | FileCheck %s --check-prefix=BOTH --check-prefix=LE
+// RUN: %clang_cc1 -triple thumbv8.1m.main-arm-none-eabi -target-feature +mve.fp -mfloat-abi hard -fallow-half-arguments-and-returns -O0 -disable-O0-optnone -DPOLYMORPHIC -S -emit-llvm -o - %s | opt -S -mem2reg -sroa -early-cse | FileCheck %s --check-prefix=BOTH --check-prefix=LE
+// RUN: %clang_cc1 -triple thumbebv8.1m.main-arm-none-eabi -target-feature +mve.fp -mfloat-abi hard -fallow-half-arguments-and-returns -O0 -disable-O0-optnone -S -emit-llvm -o - %s | opt -S -mem2reg -sroa -early-cse | FileCheck %s --check-prefix=BOTH --check-prefix=BE
+// RUN: %clang_cc1 -triple thumbebv8.1m.main-arm-none-eabi -target-feature +mve.fp -mfloat-abi hard -fallow-half-arguments-and-returns -O0 -disable-O0-optnone -DPOLYMORPHIC -S -emit-llvm -o - %s | opt -S -mem2reg -sroa -early-cse | FileCheck %s --check-prefix=BOTH --check-prefix=BE
+
+#include <arm_mve.h>
+
+// LE-LABEL: @test_vreinterpretq_f16_f32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <8 x half>
+// LE-NEXT:    ret <8 x half> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_f16_f32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x half> @llvm.arm.mve.vreinterpretq.v8f16.v4f32(<4 x float> [[A:%.*]])
+// BE-NEXT:    ret <8 x half> [[TMP0]]
+//
+float16x8_t test_vreinterpretq_f16_f32(float32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f16_f32(a);
+#endif /* POLYMORPHIC */
+}
+
+// BOTH-LABEL: @test_vreinterpretq_f16_s16(
+// BOTH-NEXT:  entry:
+// BOTH-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <8 x half>
+// BOTH-NEXT:    ret <8 x half> [[TMP0]]
+//
+float16x8_t test_vreinterpretq_f16_s16(int16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f16_s16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_f16_s32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <8 x half>
+// LE-NEXT:    ret <8 x half> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_f16_s32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x half> @llvm.arm.mve.vreinterpretq.v8f16.v4i32(<4 x i32> [[A:%.*]])
+// BE-NEXT:    ret <8 x half> [[TMP0]]
+//
+float16x8_t test_vreinterpretq_f16_s32(int32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f16_s32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_f16_s64(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <8 x half>
+// LE-NEXT:    ret <8 x half> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_f16_s64(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x half> @llvm.arm.mve.vreinterpretq.v8f16.v2i64(<2 x i64> [[A:%.*]])
+// BE-NEXT:    ret <8 x half> [[TMP0]]
+//
+float16x8_t test_vreinterpretq_f16_s64(int64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f16_s64(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_f16_s8(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <8 x half>
+// LE-NEXT:    ret <8 x half> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_f16_s8(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x half> @llvm.arm.mve.vreinterpretq.v8f16.v16i8(<16 x i8> [[A:%.*]])
+// BE-NEXT:    ret <8 x half> [[TMP0]]
+//
+float16x8_t test_vreinterpretq_f16_s8(int8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f16_s8(a);
+#endif /* POLYMORPHIC */
+}
+
+// BOTH-LABEL: @test_vreinterpretq_f16_u16(
+// BOTH-NEXT:  entry:
+// BOTH-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <8 x half>
+// BOTH-NEXT:    ret <8 x half> [[TMP0]]
+//
+float16x8_t test_vreinterpretq_f16_u16(uint16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f16_u16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_f16_u32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <8 x half>
+// LE-NEXT:    ret <8 x half> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_f16_u32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x half> @llvm.arm.mve.vreinterpretq.v8f16.v4i32(<4 x i32> [[A:%.*]])
+// BE-NEXT:    ret <8 x half> [[TMP0]]
+//
+float16x8_t test_vreinterpretq_f16_u32(uint32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f16_u32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_f16_u64(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <8 x half>
+// LE-NEXT:    ret <8 x half> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_f16_u64(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x half> @llvm.arm.mve.vreinterpretq.v8f16.v2i64(<2 x i64> [[A:%.*]])
+// BE-NEXT:    ret <8 x half> [[TMP0]]
+//
+float16x8_t test_vreinterpretq_f16_u64(uint64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f16_u64(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_f16_u8(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <8 x half>
+// LE-NEXT:    ret <8 x half> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_f16_u8(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x half> @llvm.arm.mve.vreinterpretq.v8f16.v16i8(<16 x i8> [[A:%.*]])
+// BE-NEXT:    ret <8 x half> [[TMP0]]
+//
+float16x8_t test_vreinterpretq_f16_u8(uint8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f16_u8(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_f32_f16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <4 x float>
+// LE-NEXT:    ret <4 x float> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_f32_f16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x float> @llvm.arm.mve.vreinterpretq.v4f32.v8f16(<8 x half> [[A:%.*]])
+// BE-NEXT:    ret <4 x float> [[TMP0]]
+//
+float32x4_t test_vreinterpretq_f32_f16(float16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f32_f16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_f32_s16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <4 x float>
+// LE-NEXT:    ret <4 x float> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_f32_s16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x float> @llvm.arm.mve.vreinterpretq.v4f32.v8i16(<8 x i16> [[A:%.*]])
+// BE-NEXT:    ret <4 x float> [[TMP0]]
+//
+float32x4_t test_vreinterpretq_f32_s16(int16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f32_s16(a);
+#endif /* POLYMORPHIC */
+}
+
+// BOTH-LABEL: @test_vreinterpretq_f32_s32(
+// BOTH-NEXT:  entry:
+// BOTH-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <4 x float>
+// BOTH-NEXT:    ret <4 x float> [[TMP0]]
+//
+float32x4_t test_vreinterpretq_f32_s32(int32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f32_s32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_f32_s64(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <4 x float>
+// LE-NEXT:    ret <4 x float> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_f32_s64(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x float> @llvm.arm.mve.vreinterpretq.v4f32.v2i64(<2 x i64> [[A:%.*]])
+// BE-NEXT:    ret <4 x float> [[TMP0]]
+//
+float32x4_t test_vreinterpretq_f32_s64(int64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f32_s64(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_f32_s8(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <4 x float>
+// LE-NEXT:    ret <4 x float> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_f32_s8(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x float> @llvm.arm.mve.vreinterpretq.v4f32.v16i8(<16 x i8> [[A:%.*]])
+// BE-NEXT:    ret <4 x float> [[TMP0]]
+//
+float32x4_t test_vreinterpretq_f32_s8(int8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f32_s8(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_f32_u16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <4 x float>
+// LE-NEXT:    ret <4 x float> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_f32_u16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x float> @llvm.arm.mve.vreinterpretq.v4f32.v8i16(<8 x i16> [[A:%.*]])
+// BE-NEXT:    ret <4 x float> [[TMP0]]
+//
+float32x4_t test_vreinterpretq_f32_u16(uint16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f32_u16(a);
+#endif /* POLYMORPHIC */
+}
+
+// BOTH-LABEL: @test_vreinterpretq_f32_u32(
+// BOTH-NEXT:  entry:
+// BOTH-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <4 x float>
+// BOTH-NEXT:    ret <4 x float> [[TMP0]]
+//
+float32x4_t test_vreinterpretq_f32_u32(uint32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f32_u32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_f32_u64(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <4 x float>
+// LE-NEXT:    ret <4 x float> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_f32_u64(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x float> @llvm.arm.mve.vreinterpretq.v4f32.v2i64(<2 x i64> [[A:%.*]])
+// BE-NEXT:    ret <4 x float> [[TMP0]]
+//
+float32x4_t test_vreinterpretq_f32_u64(uint64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f32_u64(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_f32_u8(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <4 x float>
+// LE-NEXT:    ret <4 x float> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_f32_u8(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x float> @llvm.arm.mve.vreinterpretq.v4f32.v16i8(<16 x i8> [[A:%.*]])
+// BE-NEXT:    ret <4 x float> [[TMP0]]
+//
+float32x4_t test_vreinterpretq_f32_u8(uint8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_f32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_f32_u8(a);
+#endif /* POLYMORPHIC */
+}
+
+// BOTH-LABEL: @test_vreinterpretq_s16_f16(
+// BOTH-NEXT:  entry:
+// BOTH-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <8 x i16>
+// BOTH-NEXT:    ret <8 x i16> [[TMP0]]
+//
+int16x8_t test_vreinterpretq_s16_f16(float16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s16_f16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s16_f32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <8 x i16>
+// LE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s16_f32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x i16> @llvm.arm.mve.vreinterpretq.v8i16.v4f32(<4 x float> [[A:%.*]])
+// BE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+int16x8_t test_vreinterpretq_s16_f32(float32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s16_f32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s16_s32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <8 x i16>
+// LE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s16_s32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x i16> @llvm.arm.mve.vreinterpretq.v8i16.v4i32(<4 x i32> [[A:%.*]])
+// BE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+int16x8_t test_vreinterpretq_s16_s32(int32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s16_s32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s16_s64(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <8 x i16>
+// LE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s16_s64(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x i16> @llvm.arm.mve.vreinterpretq.v8i16.v2i64(<2 x i64> [[A:%.*]])
+// BE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+int16x8_t test_vreinterpretq_s16_s64(int64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s16_s64(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s16_s8(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <8 x i16>
+// LE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s16_s8(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x i16> @llvm.arm.mve.vreinterpretq.v8i16.v16i8(<16 x i8> [[A:%.*]])
+// BE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+int16x8_t test_vreinterpretq_s16_s8(int8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s16_s8(a);
+#endif /* POLYMORPHIC */
+}
+
+// BOTH-LABEL: @test_vreinterpretq_s16_u16(
+// BOTH-NEXT:  entry:
+// BOTH-NEXT:    ret <8 x i16> [[A:%.*]]
+//
+int16x8_t test_vreinterpretq_s16_u16(uint16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s16_u16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s16_u32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <8 x i16>
+// LE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s16_u32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x i16> @llvm.arm.mve.vreinterpretq.v8i16.v4i32(<4 x i32> [[A:%.*]])
+// BE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+int16x8_t test_vreinterpretq_s16_u32(uint32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s16_u32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s16_u64(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <8 x i16>
+// LE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s16_u64(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x i16> @llvm.arm.mve.vreinterpretq.v8i16.v2i64(<2 x i64> [[A:%.*]])
+// BE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+int16x8_t test_vreinterpretq_s16_u64(uint64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s16_u64(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s16_u8(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <8 x i16>
+// LE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s16_u8(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x i16> @llvm.arm.mve.vreinterpretq.v8i16.v16i8(<16 x i8> [[A:%.*]])
+// BE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+int16x8_t test_vreinterpretq_s16_u8(uint8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s16_u8(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s32_f16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <4 x i32>
+// LE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s32_f16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x i32> @llvm.arm.mve.vreinterpretq.v4i32.v8f16(<8 x half> [[A:%.*]])
+// BE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+int32x4_t test_vreinterpretq_s32_f16(float16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s32_f16(a);
+#endif /* POLYMORPHIC */
+}
+
+// BOTH-LABEL: @test_vreinterpretq_s32_f32(
+// BOTH-NEXT:  entry:
+// BOTH-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <4 x i32>
+// BOTH-NEXT:    ret <4 x i32> [[TMP0]]
+//
+int32x4_t test_vreinterpretq_s32_f32(float32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s32_f32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s32_s16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <4 x i32>
+// LE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s32_s16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x i32> @llvm.arm.mve.vreinterpretq.v4i32.v8i16(<8 x i16> [[A:%.*]])
+// BE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+int32x4_t test_vreinterpretq_s32_s16(int16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s32_s16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s32_s64(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <4 x i32>
+// LE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s32_s64(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x i32> @llvm.arm.mve.vreinterpretq.v4i32.v2i64(<2 x i64> [[A:%.*]])
+// BE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+int32x4_t test_vreinterpretq_s32_s64(int64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s32_s64(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s32_s8(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <4 x i32>
+// LE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s32_s8(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x i32> @llvm.arm.mve.vreinterpretq.v4i32.v16i8(<16 x i8> [[A:%.*]])
+// BE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+int32x4_t test_vreinterpretq_s32_s8(int8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s32_s8(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s32_u16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <4 x i32>
+// LE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s32_u16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x i32> @llvm.arm.mve.vreinterpretq.v4i32.v8i16(<8 x i16> [[A:%.*]])
+// BE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+int32x4_t test_vreinterpretq_s32_u16(uint16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s32_u16(a);
+#endif /* POLYMORPHIC */
+}
+
+// BOTH-LABEL: @test_vreinterpretq_s32_u32(
+// BOTH-NEXT:  entry:
+// BOTH-NEXT:    ret <4 x i32> [[A:%.*]]
+//
+int32x4_t test_vreinterpretq_s32_u32(uint32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s32_u32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s32_u64(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <4 x i32>
+// LE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s32_u64(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x i32> @llvm.arm.mve.vreinterpretq.v4i32.v2i64(<2 x i64> [[A:%.*]])
+// BE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+int32x4_t test_vreinterpretq_s32_u64(uint64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s32_u64(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s32_u8(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <4 x i32>
+// LE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s32_u8(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x i32> @llvm.arm.mve.vreinterpretq.v4i32.v16i8(<16 x i8> [[A:%.*]])
+// BE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+int32x4_t test_vreinterpretq_s32_u8(uint8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s32_u8(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s64_f16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <2 x i64>
+// LE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s64_f16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <2 x i64> @llvm.arm.mve.vreinterpretq.v2i64.v8f16(<8 x half> [[A:%.*]])
+// BE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+int64x2_t test_vreinterpretq_s64_f16(float16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s64_f16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s64_f32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <2 x i64>
+// LE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s64_f32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <2 x i64> @llvm.arm.mve.vreinterpretq.v2i64.v4f32(<4 x float> [[A:%.*]])
+// BE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+int64x2_t test_vreinterpretq_s64_f32(float32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s64_f32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s64_s16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <2 x i64>
+// LE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s64_s16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <2 x i64> @llvm.arm.mve.vreinterpretq.v2i64.v8i16(<8 x i16> [[A:%.*]])
+// BE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+int64x2_t test_vreinterpretq_s64_s16(int16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s64_s16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s64_s32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <2 x i64>
+// LE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s64_s32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <2 x i64> @llvm.arm.mve.vreinterpretq.v2i64.v4i32(<4 x i32> [[A:%.*]])
+// BE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+int64x2_t test_vreinterpretq_s64_s32(int32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s64_s32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s64_s8(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <2 x i64>
+// LE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s64_s8(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <2 x i64> @llvm.arm.mve.vreinterpretq.v2i64.v16i8(<16 x i8> [[A:%.*]])
+// BE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+int64x2_t test_vreinterpretq_s64_s8(int8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s64_s8(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s64_u16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <2 x i64>
+// LE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s64_u16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <2 x i64> @llvm.arm.mve.vreinterpretq.v2i64.v8i16(<8 x i16> [[A:%.*]])
+// BE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+int64x2_t test_vreinterpretq_s64_u16(uint16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s64_u16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s64_u32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <2 x i64>
+// LE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s64_u32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <2 x i64> @llvm.arm.mve.vreinterpretq.v2i64.v4i32(<4 x i32> [[A:%.*]])
+// BE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+int64x2_t test_vreinterpretq_s64_u32(uint32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s64_u32(a);
+#endif /* POLYMORPHIC */
+}
+
+// BOTH-LABEL: @test_vreinterpretq_s64_u64(
+// BOTH-NEXT:  entry:
+// BOTH-NEXT:    ret <2 x i64> [[A:%.*]]
+//
+int64x2_t test_vreinterpretq_s64_u64(uint64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s64_u64(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s64_u8(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <2 x i64>
+// LE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s64_u8(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <2 x i64> @llvm.arm.mve.vreinterpretq.v2i64.v16i8(<16 x i8> [[A:%.*]])
+// BE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+int64x2_t test_vreinterpretq_s64_u8(uint8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s64_u8(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s8_f16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <16 x i8>
+// LE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s8_f16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <16 x i8> @llvm.arm.mve.vreinterpretq.v16i8.v8f16(<8 x half> [[A:%.*]])
+// BE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+int8x16_t test_vreinterpretq_s8_f16(float16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s8_f16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s8_f32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <16 x i8>
+// LE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s8_f32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <16 x i8> @llvm.arm.mve.vreinterpretq.v16i8.v4f32(<4 x float> [[A:%.*]])
+// BE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+int8x16_t test_vreinterpretq_s8_f32(float32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s8_f32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s8_s16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <16 x i8>
+// LE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s8_s16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <16 x i8> @llvm.arm.mve.vreinterpretq.v16i8.v8i16(<8 x i16> [[A:%.*]])
+// BE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+int8x16_t test_vreinterpretq_s8_s16(int16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s8_s16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s8_s32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <16 x i8>
+// LE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s8_s32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <16 x i8> @llvm.arm.mve.vreinterpretq.v16i8.v4i32(<4 x i32> [[A:%.*]])
+// BE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+int8x16_t test_vreinterpretq_s8_s32(int32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s8_s32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s8_s64(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <16 x i8>
+// LE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s8_s64(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <16 x i8> @llvm.arm.mve.vreinterpretq.v16i8.v2i64(<2 x i64> [[A:%.*]])
+// BE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+int8x16_t test_vreinterpretq_s8_s64(int64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s8_s64(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s8_u16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <16 x i8>
+// LE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s8_u16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <16 x i8> @llvm.arm.mve.vreinterpretq.v16i8.v8i16(<8 x i16> [[A:%.*]])
+// BE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+int8x16_t test_vreinterpretq_s8_u16(uint16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s8_u16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s8_u32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <16 x i8>
+// LE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s8_u32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <16 x i8> @llvm.arm.mve.vreinterpretq.v16i8.v4i32(<4 x i32> [[A:%.*]])
+// BE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+int8x16_t test_vreinterpretq_s8_u32(uint32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s8_u32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_s8_u64(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <16 x i8>
+// LE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_s8_u64(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <16 x i8> @llvm.arm.mve.vreinterpretq.v16i8.v2i64(<2 x i64> [[A:%.*]])
+// BE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+int8x16_t test_vreinterpretq_s8_u64(uint64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s8_u64(a);
+#endif /* POLYMORPHIC */
+}
+
+// BOTH-LABEL: @test_vreinterpretq_s8_u8(
+// BOTH-NEXT:  entry:
+// BOTH-NEXT:    ret <16 x i8> [[A:%.*]]
+//
+int8x16_t test_vreinterpretq_s8_u8(uint8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_s8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_s8_u8(a);
+#endif /* POLYMORPHIC */
+}
+
+// BOTH-LABEL: @test_vreinterpretq_u16_f16(
+// BOTH-NEXT:  entry:
+// BOTH-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <8 x i16>
+// BOTH-NEXT:    ret <8 x i16> [[TMP0]]
+//
+uint16x8_t test_vreinterpretq_u16_f16(float16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u16_f16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u16_f32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <8 x i16>
+// LE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u16_f32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x i16> @llvm.arm.mve.vreinterpretq.v8i16.v4f32(<4 x float> [[A:%.*]])
+// BE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+uint16x8_t test_vreinterpretq_u16_f32(float32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u16_f32(a);
+#endif /* POLYMORPHIC */
+}
+
+// BOTH-LABEL: @test_vreinterpretq_u16_s16(
+// BOTH-NEXT:  entry:
+// BOTH-NEXT:    ret <8 x i16> [[A:%.*]]
+//
+uint16x8_t test_vreinterpretq_u16_s16(int16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u16_s16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u16_s32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <8 x i16>
+// LE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u16_s32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x i16> @llvm.arm.mve.vreinterpretq.v8i16.v4i32(<4 x i32> [[A:%.*]])
+// BE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+uint16x8_t test_vreinterpretq_u16_s32(int32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u16_s32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u16_s64(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <8 x i16>
+// LE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u16_s64(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x i16> @llvm.arm.mve.vreinterpretq.v8i16.v2i64(<2 x i64> [[A:%.*]])
+// BE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+uint16x8_t test_vreinterpretq_u16_s64(int64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u16_s64(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u16_s8(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <8 x i16>
+// LE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u16_s8(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x i16> @llvm.arm.mve.vreinterpretq.v8i16.v16i8(<16 x i8> [[A:%.*]])
+// BE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+uint16x8_t test_vreinterpretq_u16_s8(int8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u16_s8(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u16_u32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <8 x i16>
+// LE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u16_u32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x i16> @llvm.arm.mve.vreinterpretq.v8i16.v4i32(<4 x i32> [[A:%.*]])
+// BE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+uint16x8_t test_vreinterpretq_u16_u32(uint32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u16_u32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u16_u64(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <8 x i16>
+// LE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u16_u64(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x i16> @llvm.arm.mve.vreinterpretq.v8i16.v2i64(<2 x i64> [[A:%.*]])
+// BE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+uint16x8_t test_vreinterpretq_u16_u64(uint64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u16_u64(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u16_u8(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <8 x i16>
+// LE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u16_u8(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <8 x i16> @llvm.arm.mve.vreinterpretq.v8i16.v16i8(<16 x i8> [[A:%.*]])
+// BE-NEXT:    ret <8 x i16> [[TMP0]]
+//
+uint16x8_t test_vreinterpretq_u16_u8(uint8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u16(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u16_u8(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u32_f16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <4 x i32>
+// LE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u32_f16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x i32> @llvm.arm.mve.vreinterpretq.v4i32.v8f16(<8 x half> [[A:%.*]])
+// BE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+uint32x4_t test_vreinterpretq_u32_f16(float16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u32_f16(a);
+#endif /* POLYMORPHIC */
+}
+
+// BOTH-LABEL: @test_vreinterpretq_u32_f32(
+// BOTH-NEXT:  entry:
+// BOTH-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <4 x i32>
+// BOTH-NEXT:    ret <4 x i32> [[TMP0]]
+//
+uint32x4_t test_vreinterpretq_u32_f32(float32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u32_f32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u32_s16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <4 x i32>
+// LE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u32_s16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x i32> @llvm.arm.mve.vreinterpretq.v4i32.v8i16(<8 x i16> [[A:%.*]])
+// BE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+uint32x4_t test_vreinterpretq_u32_s16(int16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u32_s16(a);
+#endif /* POLYMORPHIC */
+}
+
+// BOTH-LABEL: @test_vreinterpretq_u32_s32(
+// BOTH-NEXT:  entry:
+// BOTH-NEXT:    ret <4 x i32> [[A:%.*]]
+//
+uint32x4_t test_vreinterpretq_u32_s32(int32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u32_s32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u32_s64(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <4 x i32>
+// LE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u32_s64(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x i32> @llvm.arm.mve.vreinterpretq.v4i32.v2i64(<2 x i64> [[A:%.*]])
+// BE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+uint32x4_t test_vreinterpretq_u32_s64(int64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u32_s64(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u32_s8(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <4 x i32>
+// LE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u32_s8(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x i32> @llvm.arm.mve.vreinterpretq.v4i32.v16i8(<16 x i8> [[A:%.*]])
+// BE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+uint32x4_t test_vreinterpretq_u32_s8(int8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u32_s8(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u32_u16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <4 x i32>
+// LE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u32_u16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x i32> @llvm.arm.mve.vreinterpretq.v4i32.v8i16(<8 x i16> [[A:%.*]])
+// BE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+uint32x4_t test_vreinterpretq_u32_u16(uint16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u32_u16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u32_u64(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <4 x i32>
+// LE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u32_u64(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x i32> @llvm.arm.mve.vreinterpretq.v4i32.v2i64(<2 x i64> [[A:%.*]])
+// BE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+uint32x4_t test_vreinterpretq_u32_u64(uint64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u32_u64(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u32_u8(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <4 x i32>
+// LE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u32_u8(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <4 x i32> @llvm.arm.mve.vreinterpretq.v4i32.v16i8(<16 x i8> [[A:%.*]])
+// BE-NEXT:    ret <4 x i32> [[TMP0]]
+//
+uint32x4_t test_vreinterpretq_u32_u8(uint8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u32(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u32_u8(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u64_f16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <2 x i64>
+// LE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u64_f16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <2 x i64> @llvm.arm.mve.vreinterpretq.v2i64.v8f16(<8 x half> [[A:%.*]])
+// BE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+uint64x2_t test_vreinterpretq_u64_f16(float16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u64_f16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u64_f32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <2 x i64>
+// LE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u64_f32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <2 x i64> @llvm.arm.mve.vreinterpretq.v2i64.v4f32(<4 x float> [[A:%.*]])
+// BE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+uint64x2_t test_vreinterpretq_u64_f32(float32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u64_f32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u64_s16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <2 x i64>
+// LE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u64_s16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <2 x i64> @llvm.arm.mve.vreinterpretq.v2i64.v8i16(<8 x i16> [[A:%.*]])
+// BE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+uint64x2_t test_vreinterpretq_u64_s16(int16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u64_s16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u64_s32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <2 x i64>
+// LE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u64_s32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <2 x i64> @llvm.arm.mve.vreinterpretq.v2i64.v4i32(<4 x i32> [[A:%.*]])
+// BE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+uint64x2_t test_vreinterpretq_u64_s32(int32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u64_s32(a);
+#endif /* POLYMORPHIC */
+}
+
+// BOTH-LABEL: @test_vreinterpretq_u64_s64(
+// BOTH-NEXT:  entry:
+// BOTH-NEXT:    ret <2 x i64> [[A:%.*]]
+//
+uint64x2_t test_vreinterpretq_u64_s64(int64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u64_s64(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u64_s8(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <2 x i64>
+// LE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u64_s8(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <2 x i64> @llvm.arm.mve.vreinterpretq.v2i64.v16i8(<16 x i8> [[A:%.*]])
+// BE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+uint64x2_t test_vreinterpretq_u64_s8(int8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u64_s8(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u64_u16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <2 x i64>
+// LE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u64_u16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <2 x i64> @llvm.arm.mve.vreinterpretq.v2i64.v8i16(<8 x i16> [[A:%.*]])
+// BE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+uint64x2_t test_vreinterpretq_u64_u16(uint16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u64_u16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u64_u32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <2 x i64>
+// LE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u64_u32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <2 x i64> @llvm.arm.mve.vreinterpretq.v2i64.v4i32(<4 x i32> [[A:%.*]])
+// BE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+uint64x2_t test_vreinterpretq_u64_u32(uint32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u64_u32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u64_u8(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <2 x i64>
+// LE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u64_u8(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <2 x i64> @llvm.arm.mve.vreinterpretq.v2i64.v16i8(<16 x i8> [[A:%.*]])
+// BE-NEXT:    ret <2 x i64> [[TMP0]]
+//
+uint64x2_t test_vreinterpretq_u64_u8(uint8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u64(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u64_u8(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u8_f16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <16 x i8>
+// LE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u8_f16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <16 x i8> @llvm.arm.mve.vreinterpretq.v16i8.v8f16(<8 x half> [[A:%.*]])
+// BE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+uint8x16_t test_vreinterpretq_u8_f16(float16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u8_f16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u8_f32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <16 x i8>
+// LE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u8_f32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <16 x i8> @llvm.arm.mve.vreinterpretq.v16i8.v4f32(<4 x float> [[A:%.*]])
+// BE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+uint8x16_t test_vreinterpretq_u8_f32(float32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u8_f32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u8_s16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <16 x i8>
+// LE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u8_s16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <16 x i8> @llvm.arm.mve.vreinterpretq.v16i8.v8i16(<8 x i16> [[A:%.*]])
+// BE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+uint8x16_t test_vreinterpretq_u8_s16(int16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u8_s16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u8_s32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <16 x i8>
+// LE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u8_s32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <16 x i8> @llvm.arm.mve.vreinterpretq.v16i8.v4i32(<4 x i32> [[A:%.*]])
+// BE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+uint8x16_t test_vreinterpretq_u8_s32(int32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u8_s32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u8_s64(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <16 x i8>
+// LE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u8_s64(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <16 x i8> @llvm.arm.mve.vreinterpretq.v16i8.v2i64(<2 x i64> [[A:%.*]])
+// BE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+uint8x16_t test_vreinterpretq_u8_s64(int64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u8_s64(a);
+#endif /* POLYMORPHIC */
+}
+
+// BOTH-LABEL: @test_vreinterpretq_u8_s8(
+// BOTH-NEXT:  entry:
+// BOTH-NEXT:    ret <16 x i8> [[A:%.*]]
+//
+uint8x16_t test_vreinterpretq_u8_s8(int8x16_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u8_s8(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u8_u16(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <16 x i8>
+// LE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u8_u16(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <16 x i8> @llvm.arm.mve.vreinterpretq.v16i8.v8i16(<8 x i16> [[A:%.*]])
+// BE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+uint8x16_t test_vreinterpretq_u8_u16(uint16x8_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u8_u16(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u8_u32(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <16 x i8>
+// LE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u8_u32(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <16 x i8> @llvm.arm.mve.vreinterpretq.v16i8.v4i32(<4 x i32> [[A:%.*]])
+// BE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+uint8x16_t test_vreinterpretq_u8_u32(uint32x4_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u8_u32(a);
+#endif /* POLYMORPHIC */
+}
+
+// LE-LABEL: @test_vreinterpretq_u8_u64(
+// LE-NEXT:  entry:
+// LE-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <16 x i8>
+// LE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+// BE-LABEL: @test_vreinterpretq_u8_u64(
+// BE-NEXT:  entry:
+// BE-NEXT:    [[TMP0:%.*]] = call <16 x i8> @llvm.arm.mve.vreinterpretq.v16i8.v2i64(<2 x i64> [[A:%.*]])
+// BE-NEXT:    ret <16 x i8> [[TMP0]]
+//
+uint8x16_t test_vreinterpretq_u8_u64(uint64x2_t a)
+{
+#ifdef POLYMORPHIC
+    return vreinterpretq_u8(a);
+#else /* POLYMORPHIC */
+    return vreinterpretq_u8_u64(a);
+#endif /* POLYMORPHIC */
+}
Index: clang/test/CodeGen/arm-mve-intrinsics/admin.c
===================================================================
--- clang/test/CodeGen/arm-mve-intrinsics/admin.c
+++ clang/test/CodeGen/arm-mve-intrinsics/admin.c
@@ -122,1258 +122,6 @@
     return vcreateq_u8(a, b);
 }
 
-// CHECK-LABEL: @test_vreinterpretq_f16_f32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <8 x half>
-// CHECK-NEXT:    ret <8 x half> [[TMP0]]
-//
-float16x8_t test_vreinterpretq_f16_f32(float32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f16_f32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f16_s16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <8 x half>
-// CHECK-NEXT:    ret <8 x half> [[TMP0]]
-//
-float16x8_t test_vreinterpretq_f16_s16(int16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f16_s16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f16_s32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <8 x half>
-// CHECK-NEXT:    ret <8 x half> [[TMP0]]
-//
-float16x8_t test_vreinterpretq_f16_s32(int32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f16_s32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f16_s64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <8 x half>
-// CHECK-NEXT:    ret <8 x half> [[TMP0]]
-//
-float16x8_t test_vreinterpretq_f16_s64(int64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f16_s64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f16_s8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <8 x half>
-// CHECK-NEXT:    ret <8 x half> [[TMP0]]
-//
-float16x8_t test_vreinterpretq_f16_s8(int8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f16_s8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f16_u16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <8 x half>
-// CHECK-NEXT:    ret <8 x half> [[TMP0]]
-//
-float16x8_t test_vreinterpretq_f16_u16(uint16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f16_u16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f16_u32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <8 x half>
-// CHECK-NEXT:    ret <8 x half> [[TMP0]]
-//
-float16x8_t test_vreinterpretq_f16_u32(uint32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f16_u32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f16_u64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <8 x half>
-// CHECK-NEXT:    ret <8 x half> [[TMP0]]
-//
-float16x8_t test_vreinterpretq_f16_u64(uint64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f16_u64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f16_u8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <8 x half>
-// CHECK-NEXT:    ret <8 x half> [[TMP0]]
-//
-float16x8_t test_vreinterpretq_f16_u8(uint8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f16_u8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f32_f16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <4 x float>
-// CHECK-NEXT:    ret <4 x float> [[TMP0]]
-//
-float32x4_t test_vreinterpretq_f32_f16(float16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f32_f16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f32_s16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <4 x float>
-// CHECK-NEXT:    ret <4 x float> [[TMP0]]
-//
-float32x4_t test_vreinterpretq_f32_s16(int16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f32_s16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f32_s32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <4 x float>
-// CHECK-NEXT:    ret <4 x float> [[TMP0]]
-//
-float32x4_t test_vreinterpretq_f32_s32(int32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f32_s32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f32_s64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <4 x float>
-// CHECK-NEXT:    ret <4 x float> [[TMP0]]
-//
-float32x4_t test_vreinterpretq_f32_s64(int64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f32_s64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f32_s8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <4 x float>
-// CHECK-NEXT:    ret <4 x float> [[TMP0]]
-//
-float32x4_t test_vreinterpretq_f32_s8(int8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f32_s8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f32_u16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <4 x float>
-// CHECK-NEXT:    ret <4 x float> [[TMP0]]
-//
-float32x4_t test_vreinterpretq_f32_u16(uint16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f32_u16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f32_u32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <4 x float>
-// CHECK-NEXT:    ret <4 x float> [[TMP0]]
-//
-float32x4_t test_vreinterpretq_f32_u32(uint32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f32_u32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f32_u64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <4 x float>
-// CHECK-NEXT:    ret <4 x float> [[TMP0]]
-//
-float32x4_t test_vreinterpretq_f32_u64(uint64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f32_u64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_f32_u8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <4 x float>
-// CHECK-NEXT:    ret <4 x float> [[TMP0]]
-//
-float32x4_t test_vreinterpretq_f32_u8(uint8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_f32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_f32_u8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s16_f16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <8 x i16>
-// CHECK-NEXT:    ret <8 x i16> [[TMP0]]
-//
-int16x8_t test_vreinterpretq_s16_f16(float16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s16_f16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s16_f32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <8 x i16>
-// CHECK-NEXT:    ret <8 x i16> [[TMP0]]
-//
-int16x8_t test_vreinterpretq_s16_f32(float32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s16_f32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s16_s32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <8 x i16>
-// CHECK-NEXT:    ret <8 x i16> [[TMP0]]
-//
-int16x8_t test_vreinterpretq_s16_s32(int32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s16_s32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s16_s64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <8 x i16>
-// CHECK-NEXT:    ret <8 x i16> [[TMP0]]
-//
-int16x8_t test_vreinterpretq_s16_s64(int64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s16_s64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s16_s8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <8 x i16>
-// CHECK-NEXT:    ret <8 x i16> [[TMP0]]
-//
-int16x8_t test_vreinterpretq_s16_s8(int8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s16_s8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s16_u16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    ret <8 x i16> [[A:%.*]]
-//
-int16x8_t test_vreinterpretq_s16_u16(uint16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s16_u16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s16_u32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <8 x i16>
-// CHECK-NEXT:    ret <8 x i16> [[TMP0]]
-//
-int16x8_t test_vreinterpretq_s16_u32(uint32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s16_u32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s16_u64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <8 x i16>
-// CHECK-NEXT:    ret <8 x i16> [[TMP0]]
-//
-int16x8_t test_vreinterpretq_s16_u64(uint64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s16_u64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s16_u8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <8 x i16>
-// CHECK-NEXT:    ret <8 x i16> [[TMP0]]
-//
-int16x8_t test_vreinterpretq_s16_u8(uint8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s16_u8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s32_f16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <4 x i32>
-// CHECK-NEXT:    ret <4 x i32> [[TMP0]]
-//
-int32x4_t test_vreinterpretq_s32_f16(float16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s32_f16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s32_f32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <4 x i32>
-// CHECK-NEXT:    ret <4 x i32> [[TMP0]]
-//
-int32x4_t test_vreinterpretq_s32_f32(float32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s32_f32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s32_s16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <4 x i32>
-// CHECK-NEXT:    ret <4 x i32> [[TMP0]]
-//
-int32x4_t test_vreinterpretq_s32_s16(int16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s32_s16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s32_s64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <4 x i32>
-// CHECK-NEXT:    ret <4 x i32> [[TMP0]]
-//
-int32x4_t test_vreinterpretq_s32_s64(int64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s32_s64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s32_s8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <4 x i32>
-// CHECK-NEXT:    ret <4 x i32> [[TMP0]]
-//
-int32x4_t test_vreinterpretq_s32_s8(int8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s32_s8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s32_u16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <4 x i32>
-// CHECK-NEXT:    ret <4 x i32> [[TMP0]]
-//
-int32x4_t test_vreinterpretq_s32_u16(uint16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s32_u16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s32_u32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    ret <4 x i32> [[A:%.*]]
-//
-int32x4_t test_vreinterpretq_s32_u32(uint32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s32_u32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s32_u64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <4 x i32>
-// CHECK-NEXT:    ret <4 x i32> [[TMP0]]
-//
-int32x4_t test_vreinterpretq_s32_u64(uint64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s32_u64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s32_u8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <4 x i32>
-// CHECK-NEXT:    ret <4 x i32> [[TMP0]]
-//
-int32x4_t test_vreinterpretq_s32_u8(uint8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s32_u8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s64_f16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <2 x i64>
-// CHECK-NEXT:    ret <2 x i64> [[TMP0]]
-//
-int64x2_t test_vreinterpretq_s64_f16(float16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s64_f16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s64_f32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <2 x i64>
-// CHECK-NEXT:    ret <2 x i64> [[TMP0]]
-//
-int64x2_t test_vreinterpretq_s64_f32(float32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s64_f32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s64_s16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <2 x i64>
-// CHECK-NEXT:    ret <2 x i64> [[TMP0]]
-//
-int64x2_t test_vreinterpretq_s64_s16(int16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s64_s16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s64_s32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <2 x i64>
-// CHECK-NEXT:    ret <2 x i64> [[TMP0]]
-//
-int64x2_t test_vreinterpretq_s64_s32(int32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s64_s32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s64_s8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <2 x i64>
-// CHECK-NEXT:    ret <2 x i64> [[TMP0]]
-//
-int64x2_t test_vreinterpretq_s64_s8(int8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s64_s8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s64_u16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <2 x i64>
-// CHECK-NEXT:    ret <2 x i64> [[TMP0]]
-//
-int64x2_t test_vreinterpretq_s64_u16(uint16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s64_u16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s64_u32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <2 x i64>
-// CHECK-NEXT:    ret <2 x i64> [[TMP0]]
-//
-int64x2_t test_vreinterpretq_s64_u32(uint32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s64_u32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s64_u64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    ret <2 x i64> [[A:%.*]]
-//
-int64x2_t test_vreinterpretq_s64_u64(uint64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s64_u64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s64_u8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <2 x i64>
-// CHECK-NEXT:    ret <2 x i64> [[TMP0]]
-//
-int64x2_t test_vreinterpretq_s64_u8(uint8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s64_u8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s8_f16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <16 x i8>
-// CHECK-NEXT:    ret <16 x i8> [[TMP0]]
-//
-int8x16_t test_vreinterpretq_s8_f16(float16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s8_f16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s8_f32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <16 x i8>
-// CHECK-NEXT:    ret <16 x i8> [[TMP0]]
-//
-int8x16_t test_vreinterpretq_s8_f32(float32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s8_f32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s8_s16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <16 x i8>
-// CHECK-NEXT:    ret <16 x i8> [[TMP0]]
-//
-int8x16_t test_vreinterpretq_s8_s16(int16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s8_s16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s8_s32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <16 x i8>
-// CHECK-NEXT:    ret <16 x i8> [[TMP0]]
-//
-int8x16_t test_vreinterpretq_s8_s32(int32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s8_s32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s8_s64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <16 x i8>
-// CHECK-NEXT:    ret <16 x i8> [[TMP0]]
-//
-int8x16_t test_vreinterpretq_s8_s64(int64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s8_s64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s8_u16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <16 x i8>
-// CHECK-NEXT:    ret <16 x i8> [[TMP0]]
-//
-int8x16_t test_vreinterpretq_s8_u16(uint16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s8_u16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s8_u32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <16 x i8>
-// CHECK-NEXT:    ret <16 x i8> [[TMP0]]
-//
-int8x16_t test_vreinterpretq_s8_u32(uint32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s8_u32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s8_u64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <16 x i8>
-// CHECK-NEXT:    ret <16 x i8> [[TMP0]]
-//
-int8x16_t test_vreinterpretq_s8_u64(uint64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s8_u64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_s8_u8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    ret <16 x i8> [[A:%.*]]
-//
-int8x16_t test_vreinterpretq_s8_u8(uint8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_s8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_s8_u8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u16_f16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <8 x i16>
-// CHECK-NEXT:    ret <8 x i16> [[TMP0]]
-//
-uint16x8_t test_vreinterpretq_u16_f16(float16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u16_f16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u16_f32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <8 x i16>
-// CHECK-NEXT:    ret <8 x i16> [[TMP0]]
-//
-uint16x8_t test_vreinterpretq_u16_f32(float32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u16_f32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u16_s16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    ret <8 x i16> [[A:%.*]]
-//
-uint16x8_t test_vreinterpretq_u16_s16(int16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u16_s16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u16_s32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <8 x i16>
-// CHECK-NEXT:    ret <8 x i16> [[TMP0]]
-//
-uint16x8_t test_vreinterpretq_u16_s32(int32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u16_s32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u16_s64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <8 x i16>
-// CHECK-NEXT:    ret <8 x i16> [[TMP0]]
-//
-uint16x8_t test_vreinterpretq_u16_s64(int64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u16_s64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u16_s8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <8 x i16>
-// CHECK-NEXT:    ret <8 x i16> [[TMP0]]
-//
-uint16x8_t test_vreinterpretq_u16_s8(int8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u16_s8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u16_u32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <8 x i16>
-// CHECK-NEXT:    ret <8 x i16> [[TMP0]]
-//
-uint16x8_t test_vreinterpretq_u16_u32(uint32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u16_u32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u16_u64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <8 x i16>
-// CHECK-NEXT:    ret <8 x i16> [[TMP0]]
-//
-uint16x8_t test_vreinterpretq_u16_u64(uint64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u16_u64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u16_u8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <8 x i16>
-// CHECK-NEXT:    ret <8 x i16> [[TMP0]]
-//
-uint16x8_t test_vreinterpretq_u16_u8(uint8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u16(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u16_u8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u32_f16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <4 x i32>
-// CHECK-NEXT:    ret <4 x i32> [[TMP0]]
-//
-uint32x4_t test_vreinterpretq_u32_f16(float16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u32_f16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u32_f32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <4 x i32>
-// CHECK-NEXT:    ret <4 x i32> [[TMP0]]
-//
-uint32x4_t test_vreinterpretq_u32_f32(float32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u32_f32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u32_s16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <4 x i32>
-// CHECK-NEXT:    ret <4 x i32> [[TMP0]]
-//
-uint32x4_t test_vreinterpretq_u32_s16(int16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u32_s16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u32_s32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    ret <4 x i32> [[A:%.*]]
-//
-uint32x4_t test_vreinterpretq_u32_s32(int32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u32_s32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u32_s64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <4 x i32>
-// CHECK-NEXT:    ret <4 x i32> [[TMP0]]
-//
-uint32x4_t test_vreinterpretq_u32_s64(int64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u32_s64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u32_s8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <4 x i32>
-// CHECK-NEXT:    ret <4 x i32> [[TMP0]]
-//
-uint32x4_t test_vreinterpretq_u32_s8(int8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u32_s8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u32_u16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <4 x i32>
-// CHECK-NEXT:    ret <4 x i32> [[TMP0]]
-//
-uint32x4_t test_vreinterpretq_u32_u16(uint16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u32_u16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u32_u64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <4 x i32>
-// CHECK-NEXT:    ret <4 x i32> [[TMP0]]
-//
-uint32x4_t test_vreinterpretq_u32_u64(uint64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u32_u64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u32_u8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <4 x i32>
-// CHECK-NEXT:    ret <4 x i32> [[TMP0]]
-//
-uint32x4_t test_vreinterpretq_u32_u8(uint8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u32(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u32_u8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u64_f16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <2 x i64>
-// CHECK-NEXT:    ret <2 x i64> [[TMP0]]
-//
-uint64x2_t test_vreinterpretq_u64_f16(float16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u64_f16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u64_f32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <2 x i64>
-// CHECK-NEXT:    ret <2 x i64> [[TMP0]]
-//
-uint64x2_t test_vreinterpretq_u64_f32(float32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u64_f32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u64_s16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <2 x i64>
-// CHECK-NEXT:    ret <2 x i64> [[TMP0]]
-//
-uint64x2_t test_vreinterpretq_u64_s16(int16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u64_s16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u64_s32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <2 x i64>
-// CHECK-NEXT:    ret <2 x i64> [[TMP0]]
-//
-uint64x2_t test_vreinterpretq_u64_s32(int32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u64_s32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u64_s64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    ret <2 x i64> [[A:%.*]]
-//
-uint64x2_t test_vreinterpretq_u64_s64(int64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u64_s64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u64_s8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <2 x i64>
-// CHECK-NEXT:    ret <2 x i64> [[TMP0]]
-//
-uint64x2_t test_vreinterpretq_u64_s8(int8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u64_s8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u64_u16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <2 x i64>
-// CHECK-NEXT:    ret <2 x i64> [[TMP0]]
-//
-uint64x2_t test_vreinterpretq_u64_u16(uint16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u64_u16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u64_u32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <2 x i64>
-// CHECK-NEXT:    ret <2 x i64> [[TMP0]]
-//
-uint64x2_t test_vreinterpretq_u64_u32(uint32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u64_u32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u64_u8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <16 x i8> [[A:%.*]] to <2 x i64>
-// CHECK-NEXT:    ret <2 x i64> [[TMP0]]
-//
-uint64x2_t test_vreinterpretq_u64_u8(uint8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u64(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u64_u8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u8_f16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x half> [[A:%.*]] to <16 x i8>
-// CHECK-NEXT:    ret <16 x i8> [[TMP0]]
-//
-uint8x16_t test_vreinterpretq_u8_f16(float16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u8_f16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u8_f32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x float> [[A:%.*]] to <16 x i8>
-// CHECK-NEXT:    ret <16 x i8> [[TMP0]]
-//
-uint8x16_t test_vreinterpretq_u8_f32(float32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u8_f32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u8_s16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <16 x i8>
-// CHECK-NEXT:    ret <16 x i8> [[TMP0]]
-//
-uint8x16_t test_vreinterpretq_u8_s16(int16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u8_s16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u8_s32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <16 x i8>
-// CHECK-NEXT:    ret <16 x i8> [[TMP0]]
-//
-uint8x16_t test_vreinterpretq_u8_s32(int32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u8_s32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u8_s64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <16 x i8>
-// CHECK-NEXT:    ret <16 x i8> [[TMP0]]
-//
-uint8x16_t test_vreinterpretq_u8_s64(int64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u8_s64(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u8_s8(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    ret <16 x i8> [[A:%.*]]
-//
-uint8x16_t test_vreinterpretq_u8_s8(int8x16_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u8_s8(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u8_u16(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <8 x i16> [[A:%.*]] to <16 x i8>
-// CHECK-NEXT:    ret <16 x i8> [[TMP0]]
-//
-uint8x16_t test_vreinterpretq_u8_u16(uint16x8_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u8_u16(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u8_u32(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <4 x i32> [[A:%.*]] to <16 x i8>
-// CHECK-NEXT:    ret <16 x i8> [[TMP0]]
-//
-uint8x16_t test_vreinterpretq_u8_u32(uint32x4_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u8_u32(a);
-#endif /* POLYMORPHIC */
-}
-
-// CHECK-LABEL: @test_vreinterpretq_u8_u64(
-// CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[TMP0:%.*]] = bitcast <2 x i64> [[A:%.*]] to <16 x i8>
-// CHECK-NEXT:    ret <16 x i8> [[TMP0]]
-//
-uint8x16_t test_vreinterpretq_u8_u64(uint64x2_t a)
-{
-#ifdef POLYMORPHIC
-    return vreinterpretq_u8(a);
-#else /* POLYMORPHIC */
-    return vreinterpretq_u8_u64(a);
-#endif /* POLYMORPHIC */
-}
-
 // CHECK-LABEL: @test_vuninitializedq_polymorphic_f16(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    ret <8 x half> undef
Index: clang/lib/CodeGen/CGBuiltin.cpp
===================================================================
--- clang/lib/CodeGen/CGBuiltin.cpp
+++ clang/lib/CodeGen/CGBuiltin.cpp
@@ -7019,6 +7019,32 @@
   return Builder.CreateVectorSplat(Elements, V);
 }
 
+static llvm::Value *ARMMVEVectorReinterpret(CGBuilderTy &Builder,
+                                            CodeGenFunction *CGF,
+                                            llvm::Value *V,
+                                            llvm::Type *DestType) {
+  // Convert one MVE vector type into another by reinterpreting its in-register
+  // format.
+  //
+  // Little-endian, this is identical to a bitcast (which reinterprets the
+  // memory format). But big-endian, they're not necessarily the same, because
+  // the register and memory formats map to each other differently depending on
+  // the lane size.
+  //
+  // We generate a bitcast whenever we can (if we're little-endian, or if the
+  // lane sizes are the same anyway). Otherwise we fall back to an IR intrinsic
+  // that performs the different kind of reinterpretation.
+  if (CGF->getTarget().isBigEndian() &&
+      V->getType()->getScalarSizeInBits() != DestType->getScalarSizeInBits()) {
+    return Builder.CreateCall(
+        CGF->CGM.getIntrinsic(Intrinsic::arm_mve_vreinterpretq,
+                              {DestType, V->getType()}),
+        V);
+  } else {
+    return Builder.CreateBitCast(V, DestType);
+  }
+}
+
 Value *CodeGenFunction::EmitARMMVEBuiltinExpr(unsigned BuiltinID,
                                               const CallExpr *E,
                                               ReturnValueSlot ReturnValue,
Index: clang/include/clang/Basic/arm_mve_defs.td
===================================================================
--- clang/include/clang/Basic/arm_mve_defs.td
+++ clang/include/clang/Basic/arm_mve_defs.td
@@ -57,6 +57,10 @@
   // an argument.
   let prefix = func # "(Builder, ";
 }
+class CGFHelperFn<string func> : IRBuilderBase {
+  // Like CGHelperFn, but also takes the CodeGenFunction itself.
+  let prefix = func # "(Builder, this, ";
+}
 def add: IRBuilder<"CreateAdd">;
 def mul: IRBuilder<"CreateMul">;
 def not: IRBuilder<"CreateNot">;
@@ -89,6 +93,7 @@
 def xelt_var: IRBuilder<"CreateExtractElement">;
 def trunc: IRBuilder<"CreateTrunc">;
 def bitcast: IRBuilder<"CreateBitCast">;
+def vreinterpret: CGFHelperFn<"ARMMVEVectorReinterpret">;
 def extend: CGHelperFn<"SignOrZeroExtend"> {
   let special_params = [IRBuilderIntParam<2, "bool">];
 }
Index: clang/include/clang/Basic/arm_mve.td
===================================================================
--- clang/include/clang/Basic/arm_mve.td
+++ clang/include/clang/Basic/arm_mve.td
@@ -1063,7 +1063,7 @@
       !if(!eq(!cast<string>(desttype),!cast<string>(srctype)),[],[srctype])))
   in {
     def "vreinterpretq_" # desttype: Intrinsic<
-        VecOf<desttype>, (args Vector:$x), (bitcast $x, VecOf<desttype>)>;
+        VecOf<desttype>, (args Vector:$x), (vreinterpret $x, VecOf<desttype>)>;
   }
 }
 

_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits