https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101810
Bug ID: 101810
Summary: libiberty/simple-object-xcoff.c segmentation fault
Product: gcc
Version: 12.0
Status: UNCONFIRMED
Severity: normal
Priority: P3
Component: plugins
Assignee: unassigned at gcc dot gnu.org
Reporter: amodra at gmail dot com
Target Milestone: ---
>From https://sourceware.org/bugzilla/show_bug.cgi?id=28179
binutils/nm-new --plugin ~/build/gcc-virgin/lto-plugin/.libs/liblto_plugin.so
-a pr28179
AddressSanitizer:DEADLYSIGNAL
=================================================================
==3630013==ERROR: AddressSanitizer: SEGV on unknown address 0x60200001000a (pc
0x7fc28ca928ea bp 0x000000000000 sp 0x7ffd425c36d0 T0)
==3630013==The signal is caused by a READ memory access.
#0 0x7fc28ca928ea in simple_object_xcoff_find_sections
/home/alan/src/gcc-virgin/libiberty/simple-object-xcoff.c:529:26
#1 0x7fc28ca874f7 in claim_file_handler
/home/alan/src/gcc-virgin/lto-plugin/lto-plugin.c:1189:16
#2 0x9ad923 in try_claim /home/alan/src/binutils-gdb/bfd/plugin.c:323:7
[snip]
A little analysis of the binutils testcase reveals the xcoff file header has
nsyms of 0x80000000. The file contains a number of places where ocr->nsyms *
SYMESZ is calculated. Since ocr->nsyms is an unsigned int and SYMESZ a plain
number (18), the expression overflows to zero. That results in a zero length
buffer being allocated and read from file, but 0x80000000 syms processed from
the buffer.
