[Bug tree-optimization/121023] New: musttail vs. IPA optimizations on the caller

[jakub at gcc dot gnu.org via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=121023

            Bug ID: 121023
           Summary: musttail vs. IPA optimizations on the caller
           Product: gcc
           Version: 16.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: jakub at gcc dot gnu.org
  Target Milestone: ---

struct S { int a, b; };

[[gnu::noipa]] int
foo (struct S x, int y, int z)
{
  return x.a + y + z;
}

[[gnu::noinline]] static int
bar (struct S x, int y, int z)
{
  [[gnu::musttail]] return foo ((struct S) { x.a, 0 }, y, 1);
}

int
baz (int x)
{
  return bar ((struct S) { 1, 2 }, x, 2) + bar ((struct S) { 2, 3 }, x + 1, 2);
}

is accepted at -O0 and -O2 on x86_64 and -O0 -m32 as well, but rejected for -O2
-m32:
pr121NNN.c: In function ‘bar.constprop.isra’:
pr121NNN.c:12:28: error: cannot tail-call: callee required more stack slots
than the caller
   12 |   [[gnu::musttail]] return foo ((struct S) { x.a, 0 }, y, 1);
      |                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The problem is that IPA optimizations optimized bar from
int bar (struct S x, int y, int z)
to
int bar.constprop.isra (int ISRA.6, int y)

Now, on x86_64 with 6 GPRs both of (struct S, int, int) and (int, int) don't
need any stack slots and so tail call is possible regardless whether we IPA
optimize the caller or not, but if there were more than 6 arguments we could
trigger even there; and on arches which pass everything on the stack removing
arguments or replacing them with arguments of different types can be always a
problem.