[Bug c/67769] New: VRP pass does wrong optimization

[bmei at broadcom dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67769

            Bug ID: 67769
           Summary: VRP pass does wrong optimization
           Product: gcc
           Version: 6.0
            Status: UNCONFIRMED
          Severity: major
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: bmei at broadcom dot com
  Target Milestone: ---

#include <stdlib.h>

static int
clamp (int x, int lo, int hi)
{
    return (x < lo) ? lo : ((x > hi) ? hi : x);
}


__attribute__((noinline))
short
foo (int N)
{
    short value =
        clamp (N, 0, 16);


    return value;
}

int main ()
{
  if (foo (-5) != 0)
    abort();
  return 0;
}


Compile this simple code and run. 

bash:bmei:xl-cam-21:34271> ~/scratch/install-x86/bin/gcc tst.c -O2
bash:bmei:xl-cam-21:34272> ./a.out
Aborted
bash:bmei:xl-cam-21:34273> ~/scratch/install-x86/bin/gcc -v
Using built-in specs.
COLLECT_GCC=/home/bmei/scratch/install-x86/bin/gcc
COLLECT_LTO_WRAPPER=/projects/firepath_tools1_scratch/bmei/install-x86/libexec/gcc/x86_64-unknown-linux-gnu/6.0.0/lto-wrapper
Target: x86_64-unknown-linux-gnu
Configured with: ../trunk/configure
--prefix=/projects/firepath_tools1_scratch/bmei/install-x86 --disable-nls
--with-mpfr=/projects/firepath_tools/work/bmei/packages/mpfr/2.4.1/x86-64
--with-gmp=/projects/firepath_tools/work/bmei/packages/gmp/4.3.0/x86-64
--with-mpc=/projects/firepath_tools/work/bmei/packages/mpc/0.8.1/x86-64
--disable-libsanitizer --disable-target-libsanitizer CFLAGS='-O0 -g3'
CXXFLAGS='-O0 -g3' --enable-languages=c --no-recursion --disable-bootstrap :
(reconfigured) ../trunk/configure
--prefix=/projects/firepath_tools1_scratch/bmei/install-x86 --disable-nls
--with-mpfr=/projects/firepath_tools/work/bmei/packages/mpfr/2.4.1/x86-64
--with-gmp=/projects/firepath_tools/work/bmei/packages/gmp/4.3.0/x86-64
--with-mpc=/projects/firepath_tools/work/bmei/packages/mpc/0.8.1/x86-64
--disable-libsanitizer --disable-target-libsanitizer CFLAGS='-O0 -g3'
CXXFLAGS='-O0 -g3' --disable-bootstrap --enable-languages=c,lto --no-create
--no-recursion
Thread model: posix
gcc version 6.0.0 20150929 (experimental) [trunk revision 143368] (GCC)


I looked into the tree dump, it seems that VRP2 pass. The second MAX_EXPR is
folded.

Folding statement: iftmp.0_3 = MIN_EXPR <N_2(D), 16>;
Not folded
Folding statement: iftmp.0_6 = MAX_EXPR <iftmp.0_3, 0>;
Folded into: iftmp.0_6 = iftmp.0_3;

Folding statement: value_4 = (short int) iftmp.0_6;
Not folded
Folding statement: return value_4;
Not folded
foo (int N)
[ noinline ]
{
  short int value;
  int iftmp.0_3;
  int iftmp.0_6;

  <bb 2>:
  iftmp.0_3 = MIN_EXPR <N_2(D), 16>;
  iftmp.0_6 = iftmp.0_3;
  value_4 = (short int) iftmp.0_6;
  return value_4;

}