Bug#786556: libpoppler46: segfault on OOM

[Jakub Wilk]

Package: libpoppler46
Version: 0.26.5-2




$ ulimit -v 80000
$ pdftoppm -r 300 test.pdf > /dev/null
Syntax Error: Out of memory<0a>
Syntax Error: Failed to decode J2K image<0a>
Syntax Warning: Did no succeed opening JPX Stream as JP2, trying as J2K.
Syntax Error: 00000000: expected a marker instead of 0<0a>
Syntax Warning: Did no succeed opening JPX Stream as J2K, trying as JPT.
Syntax Error: Forbidden value encounter in message header !!<0a>
Syntax Error: [JPT-stream] : Expecting Main header first [class_Id 0] !<0a>
Syntax Error: Did no succeed opening JPX Stream.
Out of memory
Syntax Error: Out of memory<0a>
Syntax Error: Failed to decode J2K image<0a>
Syntax Warning: Did no succeed opening JPX Stream as JP2, trying as J2K.
Syntax Error: 00000000: expected a marker instead of 0<0a>
Syntax Warning: Did no succeed opening JPX Stream as J2K, trying as JPT.
Syntax Error: Forbidden value encounter in message header !!<0a>
Syntax Error: [JPT-stream] : Expecting Main header first [class_Id 0] !<0a>
Syntax Error: Did no succeed opening JPX Stream.
Out of memory
Out of memory
Segmentation fault (core dumped)


Backtrace:
#0  memset () at ../sysdeps/x86_64/memset.S:78
#1  0x00007f68ed1031f0 in memset (__len=<optimized out>, __ch=<optimized out>, 
__dest=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/string3.h:84
#2  Splash::clear (this=this@entry=0x7f68eda804e0, color=color@entry=0x7ffd42edff60 
"", alpha=alpha@entry=0 '\000') at Splash.cc:1870
#3  0x00007f68ed0193ee in SplashOutputDev::drawSoftMaskedImage (this=0x7f68eda69970, 
state=<optimized out>, ref=<optimized out>, str=0x7f68eda800c0, width=2106, 
height=3406, colorMap=0x7f68eda807d0, interpolate=false, maskStr=0x7f68edb04f60, 
maskWidth=2106, maskHeight=3406, maskColorMap=0x7f68edb06e50, maskInterpolate=false) at 
SplashOutputDev.cc:3627
#4  0x00007f68ed065bec in Gfx::doImage (this=0x7f68eda6cad0, 
ref=0x7ffd42ee0240, str=0x7f68eda800c0, inlineImg=64, inlineImg@entry=false) at 
Gfx.cc:4646
#5  0x00007f68ed066e43 in Gfx::opXObject (this=0x7f68eda6cad0, args=<optimized out>, 
numArgs=<optimized out>) at Gfx.cc:4179
#6  0x00007f68ed060d98 in Gfx::go (this=this@entry=0x7f68eda6cad0, 
topLevel=topLevel@entry=true) at Gfx.cc:762
#7  0x00007f68ed061298 in Gfx::display (this=this@entry=0x7f68eda6cad0, 
obj=obj@entry=0x7ffd42ee0620, topLevel=topLevel@entry=true) at Gfx.cc:728
#8  0x00007f68ed0a9165 in Page::displaySlice (this=0x7f68eda6c130, 
out=0x7f68eda69970, hDPI=300, vDPI=300, rotate=0, useMediaBox=true, 
crop=<optimized out>, sliceX=0, sliceY=0, sliceW=2105, sliceH=3405, 
printing=false, abortCheckCbk=0x0, abortCheckCbkData=0x0, annotDisplayDecideCbk=0x0, 
annotDisplayDecideCbkData=0x0, copyXRef=false) at Page.cc:585
#9  0x00007f68ed606652 in savePageSlice (ppmFile=<optimized out>, pg_h=<optimized out>, pg_w=<optimized out>, 
h=7167525, w=-307848848, y=<optimized out>, x=<optimized out>, pg=<optimized out>, splashOut=<optimized 
out>, doc=<optimized out>) at pdftoppm.cc:222
#10 main (argc=2, argv=0x0) at pdftoppm.cc:521


-- System Information:
Debian Release: stretch/sid
 APT prefers unstable
 APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.0.0-1-amd64 (SMP w/2 CPU cores)
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages libpoppler46 depends on:
ii  libc6              2.19-18
ii  libfontconfig1     2.11.0-6.3
ii  libfreetype6       2.5.2-4
ii  libjpeg62-turbo    1:1.4.0-7
ii  liblcms2-2         2.6-3+b3
ii  libopenjpeg5       1:1.5.2-3
ii  libpng12-0         1.2.50-2+b2
ii  libstdc++6         5.1.1-6
ii  libtiff5           4.0.3-13
ii  multiarch-support  2.19-18

--
Jakub Wilk

Poppler segfaults when in runs out of (virtual) memory on the attached test PDF:

test.pdf.xz
Description: application/xz