[Bug middle-end/78412] New: attribute malloc ineffective

[msebor at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78412

            Bug ID: 78412
           Summary: attribute malloc ineffective
           Product: gcc
           Version: 7.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: middle-end
          Assignee: unassigned at gcc dot gnu.org
          Reporter: msebor at gcc dot gnu.org
  Target Milestone: ---

I came across this while testing my patch for bug 78284
(https://gcc.gnu.org/ml/gcc-patches/2016-11/msg01672.html) where I accidentally
removed attribute malloc from one of the built-ins.

Attribute malloc is document to have the effect that...

...the pointer P returned by the function cannot alias any other pointer valid
when the function returns, and moreover no pointers to valid objects occur in
any storage addressed by P.

Given this, I would expect the calls to abort to be eliminated in all the
functions below except g() because in all of them the non-null pointers
returned from successive calls to function f1 must be distinct from one another
and cannot point into the storage returned by the malloc-like function.  Yet
GCC retains all of them.

void* f0 (unsigned);
void* __attribute__ ((malloc)) f1 (unsigned);

void g0 (void)
{
  void *p = f0 (sizeof (void*));
  void *q = f0 (sizeof (void*));

  if (p && p == q) __builtin_abort ();   // expect to be retained
}

void g1 (void)
{
  void *p = f1 (sizeof (void*));
  void *q = f1 (sizeof (void*));

  if (p && p == q) __builtin_abort ();   // expected to be optimized away
}

void g2 (void)
{
  void *p = f1 (sizeof (void*));
  void *q = f1 (sizeof (void*));

  if (p && q && *(void**)q == p) __builtin_abort ();   // expected to be
optimized away
}

void g3 (void)
{
  void *p = __builtin_malloc (sizeof (void*));
  void *q = __builtin_malloc (sizeof (void*));

  if (p && p == q) __builtin_abort ();   // expected to be optimized away
}