Bug#783254: imlib2: cannot rescale+render images > 32767 width

2015-04-24 Thread Lars Stoltenow
On Fri, Apr 24, 2015 at 08:36:26PM +0200, Lars Stoltenow wrote:
> The bug originally appeared when using feh to view a wide PNG image in
> fullscreen (causing it to be downsampled while rendering).

Forgot to provide a quick way of testing:

convert -size 32770x100 xc:white test.png
feh --fullscreen test.png


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org



Bug#783254: imlib2: cannot rescale+render images > 32767 width

2015-04-24 Thread Lars Stoltenow
Source: imlib2
Severity: normal

Dear Maintainer,

rendering an imlib image to a X11 window while simultaneously scaling it, causes
a crash when the image width is > 32768 (or 32767 or so).

The bug originally appeared when using feh to view a wide PNG image in
fullscreen (causing it to be downsampled while rendering).

The (apparent) cause of the crash is the __imlib_CalcXPoints calculating
offsets (into image data I think) incorrectly. For not-so-wide images, all
offsets are positive, which makes sense. For wider images, all but the first
offsets are negative, which subsequently causes out-of-bounds memory accesses
and a crash.

I guess this is because the calculations happen with int = 32 bit (even on
amd64). Several intermediate calculations shift left by 16 -> sign bit flips
for > 32768 -> calculated offset becomes negative. (The resulting value is
right shifted by 16 later again, but then of course it is still negative).

A first quick fix that doesn't appear to completely fall apart is attached.
It appears to fix the problem, however I am not sure if there are other parts
that should also use 64 bit numbers.

-- System Information:
Debian Release: 8.0
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 3.12-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)
--- a/src/lib/scale.c	2013-12-22 05:20:31.0 +0100
+++ b/src/lib/scale.c	2015-04-24 20:29:59.623533552 +0200
@@ -112,7 +112,7 @@
if (dw > (b1 + b2))
  {
 val = (b1 << 16);
-inc = ((sw - b1 - b2) << 16) / (dw - b1 - b2);
+inc = ((long long)(sw - b1 - b2) << 16) / (dw - b1 - b2);
 for (i = 0; i < (dw - b1 - b2); i++)
   {
  p[j++] = (val >> 16);