https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67999
Bug ID: 67999
Summary: Wrong optimization of pointer comparisons
Product: gcc
Version: 5.2.0
Status: UNCONFIRMED
Severity: normal
Priority: P3
Component: c
Assignee: unassigned at gcc dot gnu.org
Reporter: ch3root at openwall dot com
Target Milestone: ---
The following program:
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
size_t len = 0xC0000000; /* 3GB */
char *buf = malloc(len);
if (!buf)
return 1;
printf("buf = %p\n", (void *)buf);
printf("len = 0x%08zx\n", len);
printf("buf + len = %p\n", (void *)(buf + len));
printf("buf + len < buf: ");
if (buf + len < buf)
printf("true\n");
else
printf("false\n");
return 0;
}
generates output like this:
$ gcc -m32 -O2 test.c && ./a.out
buf = 0x3751d008
len = 0xc0000000
buf + len = 0xf751d008
buf + len < buf: true
The result of the comparison "buf + len < buf" is wrong.
AFAICT the comparison "buf + len < buf" is simplified into "len < 0" where
"len" is treated as a signed value which is wrong when "len" is of an unsigned
type and has a large value.
