[PATCH] tree-sra: fix compare_access_positions qsort comparator

[Alexander Monakov]

Hi,

The compare_access_positions qsort comparator lacks transitivity, although
somewhat surprisingly this issue didn't manifest on 64-bit x86 bootstraps.
The first invalid comparison step is here (tree-sra.c:1545):

      /* Put the integral type with the bigger precision first.  */
      else if (INTEGRAL_TYPE_P (f1->type)
               && INTEGRAL_TYPE_P (f2->type))
        return TYPE_PRECISION (f2->type) - TYPE_PRECISION (f1->type);

Imagine you have items A, B, C such that they compare equal according to
preceding comparison steps, A and C are integral and have precision 64 resp.
32, B is non-integral.  Then you have C < A according to this step, but
comparisons against B depend on TYPE_UID, so you can end up with A < B < C < A.

A minimal fix would be to order all integral items before/after non-integral,
like preceding code already does for aggregate/vector/complex types.

Bootstrapped/regtested on 32-bit x86.

Thanks.
Alexander

        * tree-sra (compare_access_positions): Order non-integral types before
        integral types.

diff --git a/gcc/tree-sra.c b/gcc/tree-sra.c
index 163b7a2d03b..4f9a8802aeb 100644
--- a/gcc/tree-sra.c
+++ b/gcc/tree-sra.c
@@ -1542,19 +1542,17 @@ compare_access_positions (const void *a, const void *b)
               && TREE_CODE (f2->type) != COMPLEX_TYPE
               && TREE_CODE (f2->type) != VECTOR_TYPE)
        return -1;
+      /* Put any non-integral type before any integral type.  */
+      else if (INTEGRAL_TYPE_P (f1->type)
+              && !INTEGRAL_TYPE_P (f2->type))
+       return 1;
+      else if (!INTEGRAL_TYPE_P (f1->type)
+              && INTEGRAL_TYPE_P (f2->type))
+       return -1;
       /* Put the integral type with the bigger precision first.  */
       else if (INTEGRAL_TYPE_P (f1->type)
               && INTEGRAL_TYPE_P (f2->type))
        return TYPE_PRECISION (f2->type) - TYPE_PRECISION (f1->type);
-      /* Put any integral type with non-full precision last.  */
-      else if (INTEGRAL_TYPE_P (f1->type)
-              && (TREE_INT_CST_LOW (TYPE_SIZE (f1->type))
-                  != TYPE_PRECISION (f1->type)))
-       return 1;
-      else if (INTEGRAL_TYPE_P (f2->type)
-              && (TREE_INT_CST_LOW (TYPE_SIZE (f2->type))
-                  != TYPE_PRECISION (f2->type)))
-       return -1;
       /* Stabilize the sort.  */
       return TYPE_UID (f1->type) - TYPE_UID (f2->type);
     }