The SVE gather and scatter costs are classified based on whether they do 4 loads per 128 bits (x32) or 2 loads per 128 bits (x64). The number after the "x" refers to the number of bits in each "container".
However, the test for which to use was based on the element size rather than the container size. This meant that we'd use the overly conservative x32 costs for VNx2SI gathers. VNx2SI gathers are really .D gathers in which the upper half of each extension result is ignored. This patch is necessary to switch -mtune=generic over to the "new" vector costs. Tested on aarch64-linux-gnu. Pushed as previously agreed with Tamar and Kyrill. Richard gcc/ * config/aarch64/aarch64.cc (aarch64_detect_vector_stmt_subtype) (aarch64_vector_costs::add_stmt_cost): Use the x64 cost rather than x32 cost for all VNx2 modes. --- gcc/config/aarch64/aarch64.cc | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc index 40dacfcf2e7..033ea61d3a8 100644 --- a/gcc/config/aarch64/aarch64.cc +++ b/gcc/config/aarch64/aarch64.cc @@ -16819,7 +16819,8 @@ aarch64_detect_vector_stmt_subtype (vec_info *vinfo, vect_cost_for_stmt kind, && STMT_VINFO_MEMORY_ACCESS_TYPE (stmt_info) == VMAT_GATHER_SCATTER) { unsigned int nunits = vect_nunits_for_cost (vectype); - if (GET_MODE_UNIT_BITSIZE (TYPE_MODE (vectype)) == 64) + /* Test for VNx2 modes, which have 64-bit containers. */ + if (known_eq (GET_MODE_NUNITS (TYPE_MODE (vectype)), aarch64_sve_vg)) return { sve_costs->gather_load_x64_cost, nunits }; return { sve_costs->gather_load_x32_cost, nunits }; } @@ -17309,7 +17310,9 @@ aarch64_vector_costs::add_stmt_cost (int count, vect_cost_for_stmt kind, const sve_vec_cost *sve_costs = aarch64_tune_params.vec_costs->sve; if (sve_costs) { - if (GET_MODE_UNIT_BITSIZE (TYPE_MODE (vectype)) == 64) + /* Test for VNx2 modes, which have 64-bit containers. */ + if (known_eq (GET_MODE_NUNITS (TYPE_MODE (vectype)), + aarch64_sve_vg)) m_sve_gather_scatter_init_cost += sve_costs->gather_load_x64_init_cost; else -- 2.25.1