[patch] Do not apply scalar storage order to pointer fields

[Eric Botcazou]

Hi,

I didn't really think of pointer fields (nor of vector fields originally) when 
implementing the scalar_storage_order attribute, so they are swapped as well.
As Ulrich pointed out, this is problematic to describe in DWARF and probably 
not very useful in any case, so the attached patch pulls them out.

Tested on x86-64/Linux, OK for mainline?


2021-05-07  Eric Botcazou  <ebotca...@adacore.com>

        * doc/extend.texi (scalar_storage_order): Mention effect on pointer
        and vector fields.
        * tree.h (reverse_storage_order_for_component_p): Return false if
        the type is a pointer.
c/
        * c-typeck.c (build_unary_op) <ADDR_EXPR>: Do not issue an error
        on the address of a pointer field in a record with reverse SSO.


2021-05-07  Eric Botcazou  <ebotca...@adacore.com>

        * gcc.dg/sso-12.c: New test.

-- 
Eric Botcazou
diff --git a/gcc/c/c-typeck.c b/gcc/c/c-typeck.c
index fdc7bb6125c..5bdc673d03a 100644
--- a/gcc/c/c-typeck.c
+++ b/gcc/c/c-typeck.c
@@ -4866,6 +4866,7 @@ build_unary_op (location_t location, enum tree_code code, tree xarg,
 	  if (TYPE_REVERSE_STORAGE_ORDER (TREE_TYPE (TREE_OPERAND (arg, 0))))
 	    {
 	      if (!AGGREGATE_TYPE_P (TREE_TYPE (arg))
+		  && !POINTER_TYPE_P (TREE_TYPE (arg))
 		  && !VECTOR_TYPE_P (TREE_TYPE (arg)))
 		{
 		  error_at (location, "cannot take address of scalar with "
diff --git a/gcc/doc/extend.texi b/gcc/doc/extend.texi
index c8caf36f293..fd9175d1b3b 100644
--- a/gcc/doc/extend.texi
+++ b/gcc/doc/extend.texi
@@ -8551,6 +8551,9 @@ or an array whose component is a @code{union} or a @code{struct}, and it is
 possible for these fields to have a different scalar storage order than the
 enclosing type.
 
+Note that neither pointer nor vector fields are considered scalar fields in
+this context, so the attribute has no effects on these fields.
+
 This attribute is supported only for targets that use a uniform default
 scalar storage order (fortunately, most of them), i.e.@: targets that store
 the scalars either all in big-endian or all in little-endian.
diff --git a/gcc/tree.h b/gcc/tree.h
index 6d3cfc4c588..784452ca490 100644
--- a/gcc/tree.h
+++ b/gcc/tree.h
@@ -4989,7 +4989,9 @@ static inline bool
 reverse_storage_order_for_component_p (tree t)
 {
   /* The storage order only applies to scalar components.  */
-  if (AGGREGATE_TYPE_P (TREE_TYPE (t)) || VECTOR_TYPE_P (TREE_TYPE (t)))
+  if (AGGREGATE_TYPE_P (TREE_TYPE (t))
+      || POINTER_TYPE_P (TREE_TYPE (t))
+      || VECTOR_TYPE_P (TREE_TYPE (t)))
     return false;
 
   if (TREE_CODE (t) == REALPART_EXPR || TREE_CODE (t) == IMAGPART_EXPR)

/* Test scalar_storage_order attribute and pointer fields */

/* { dg-do run } */
/* { dg-options "-Wno-pedantic" } */

#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
struct __attribute__((scalar_storage_order("big-endian"))) Rec
{
  int *p;
};
#else
struct __attribute__((scalar_storage_order("little-endian"))) Rec
{
  int *p;
};
#endif

int main (int argc)
{
  struct Rec r = { &argc };
  int *p = &argc;

  if (__builtin_memcmp (&r.p, &p, sizeof (int *)) != 0)
    __builtin_abort ();

  return 0;
}