Re: xfree86 4.2.1-6 build on alpha

[Anthony Towns]

severity 184057 serious
severity 187417 serious
merge 187417 184057
thanks

On Thu, Apr 03, 2003 at 11:07:16AM +0200, Falk Hueffner wrote:
> > Ah, the gzip killer bug. Works with -3, fails with -4 - where the
> > --rsyncable patch was introduced. However, StevenK claimed he couldn't
> > reproduce it, so I didn't file a bug about it.
> > Bug needs to be filed on gzip about the 'gzip killer' .bdf.
> Already reported 3 weeks ago as #184057.

So, here's the deal. On alpha, this bug is reproducible when compiled
with gcc-3.2 at any optimisation, but not reproducible with gcc-2.95
at -O2. When -DDEBUG is enabled, the assertion is triggered on alpha
with gcc-2.95 and gcc-3.2. With -DDEBUG enabled, the assertion is also
triggered on powerpc.

The problem appears to be that the checks in deflate_fast() and deflate(),
namely:

if (hash_head != NIL && strstart - hash_head <= MAX_DIST) {

and

if (hash_head != NIL && prev_length < max_lazy_match &&
strstart - hash_head <= MAX_DIST) {

preceeding calls to longest_match() do not actually ensure that the assertion:

Assert(strstart <= window_size-MIN_LOOKAHEAD, "insufficient lookahead");

in longest_match() actually passes. I don't really understand what's going
on exactly, but the thoughtless solution of adding the extra check from
the assertion explicitly seems to work (and, afaics, should work).

The two tests (strstart <= window_size - MIN_LOOKAHEAD, and strstart -
hash_head <= MAX_DIST) are equivalent when hash_head > WSIZE, but there's
no particular reason for that to be true, that I can see. I don't think this
can result in corrupted data, and while a buffer is overflown I think it's
only by reading, so apart from the segfault I don't _think_ there are any
problems caused by this bug. I'm not really sure though.

The patch looks like:

--- gzip-1.3.5/deflate.c2003-04-03 21:51:36.0 +1000
+++ gzip-1.3.5-aj/deflate.c 2003-04-03 21:56:38.0 +1000
@@ -643,7 +643,8 @@
 /* Find the longest match, discarding those <= prev_length.
  * At this point we have always match_length < MIN_MATCH
  */
-if (hash_head != NIL && strstart - hash_head <= MAX_DIST) {
+if (hash_head != NIL && strstart - hash_head <= MAX_DIST &&
+strstart <= window_size - MIN_LOOKAHEAD) {
 /* To simplify the code, we prevent matches with the string
  * of window index 0 (in particular we have to avoid a match
  * of the string with itself at the start of the input file).
@@ -737,7 +738,8 @@
 match_length = MIN_MATCH-1;
 
 if (hash_head != NIL && prev_length < max_lazy_match &&
-strstart - hash_head <= MAX_DIST) {
+strstart - hash_head <= MAX_DIST && 
+strstart <= window_size - MIN_LOOKAHEAD) {
 /* To simplify the code, we prevent matches with the string
  * of window index 0 (in particular we have to avoid a match
  * of the string with itself at the start of the input file).

As it stands, this patch decreases the effectiveness of gzip's deflate
implementation by, I guess, up to 258 bytes per file. For comparison:

$ echo `gzip  
I don't speak for anyone save myself. GPG signed mail preferred.

  ``Dear Anthony Towns: [...] Congratulations -- 
you are now certified as a Red Hat Certified Engineer!''

Re: xfree86 4.2.1-6 build on alpha

[Anthony Towns]

severity 184057 serious
severity 187417 serious
merge 187417 184057
thanks

On Thu, Apr 03, 2003 at 11:07:16AM +0200, Falk Hueffner wrote:
> > Ah, the gzip killer bug. Works with -3, fails with -4 - where the
> > --rsyncable patch was introduced. However, StevenK claimed he couldn't
> > reproduce it, so I didn't file a bug about it.
> > Bug needs to be filed on gzip about the 'gzip killer' .bdf.
> Already reported 3 weeks ago as #184057.

So, here's the deal. On alpha, this bug is reproducible when compiled
with gcc-3.2 at any optimisation, but not reproducible with gcc-2.95
at -O2. When -DDEBUG is enabled, the assertion is triggered on alpha
with gcc-2.95 and gcc-3.2. With -DDEBUG enabled, the assertion is also
triggered on powerpc.

The problem appears to be that the checks in deflate_fast() and deflate(),
namely:

if (hash_head != NIL && strstart - hash_head <= MAX_DIST) {

and

if (hash_head != NIL && prev_length < max_lazy_match &&
strstart - hash_head <= MAX_DIST) {

preceeding calls to longest_match() do not actually ensure that the assertion:

Assert(strstart <= window_size-MIN_LOOKAHEAD, "insufficient lookahead");

in longest_match() actually passes. I don't really understand what's going
on exactly, but the thoughtless solution of adding the extra check from
the assertion explicitly seems to work (and, afaics, should work).

The two tests (strstart <= window_size - MIN_LOOKAHEAD, and strstart -
hash_head <= MAX_DIST) are equivalent when hash_head > WSIZE, but there's
no particular reason for that to be true, that I can see. I don't think this
can result in corrupted data, and while a buffer is overflown I think it's
only by reading, so apart from the segfault I don't _think_ there are any
problems caused by this bug. I'm not really sure though.

The patch looks like:

--- gzip-1.3.5/deflate.c2003-04-03 21:51:36.0 +1000
+++ gzip-1.3.5-aj/deflate.c 2003-04-03 21:56:38.0 +1000
@@ -643,7 +643,8 @@
 /* Find the longest match, discarding those <= prev_length.
  * At this point we have always match_length < MIN_MATCH
  */
-if (hash_head != NIL && strstart - hash_head <= MAX_DIST) {
+if (hash_head != NIL && strstart - hash_head <= MAX_DIST &&
+strstart <= window_size - MIN_LOOKAHEAD) {
 /* To simplify the code, we prevent matches with the string
  * of window index 0 (in particular we have to avoid a match
  * of the string with itself at the start of the input file).
@@ -737,7 +738,8 @@
 match_length = MIN_MATCH-1;
 
 if (hash_head != NIL && prev_length < max_lazy_match &&
-strstart - hash_head <= MAX_DIST) {
+strstart - hash_head <= MAX_DIST && 
+strstart <= window_size - MIN_LOOKAHEAD) {
 /* To simplify the code, we prevent matches with the string
  * of window index 0 (in particular we have to avoid a match
  * of the string with itself at the start of the input file).

As it stands, this patch decreases the effectiveness of gzip's deflate
implementation by, I guess, up to 258 bytes per file. For comparison:

$ echo `gzip  
I don't speak for anyone save myself. GPG signed mail preferred.

  ``Dear Anthony Towns: [...] Congratulations -- 
you are now certified as a Red Hat Certified Engineer!''


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: xfree86 4.2.1-6 build on alpha

[Falk Hueffner]

Daniel Stone <[EMAIL PROTECTED]> writes:

> On Thu, Apr 03, 2003 at 05:01:50PM +1000, Anthony Towns scrawled:
> > xfree86 4.2.1-6 fails to build on alpha due to:
> > 
> > ...
> > LD_LIBRARY_PATH=../../../exports/lib  ../../../exports/bin/bdftopcf -t 
> > lutBS08.
> > bdf | gzip > lutBS08.pcf.gz
> > make[6]: *** [lutBS08.pcf.gz] Error 139
> > 
> > See http://buildd.debian.org/build.php?arch=alpha&pkg=xfree86&ver=4.2.1-6
> > 
> > Can this be looked into, please?
> 
> Ah, the gzip killer bug. Works with -3, fails with -4 - where the
> --rsyncable patch was introduced. However, StevenK claimed he couldn't
> reproduce it, so I didn't file a bug about it.
> 
> Bug needs to be filed on gzip about the 'gzip killer' .bdf.

Already reported 3 weeks ago as #184057.

-- 
Falk

Re: xfree86 4.2.1-6 build on alpha

[Daniel Stone]

On Thu, Apr 03, 2003 at 05:01:50PM +1000, Anthony Towns scrawled:
> xfree86 4.2.1-6 fails to build on alpha due to:
> 
> ...
> LD_LIBRARY_PATH=../../../exports/lib  ../../../exports/bin/bdftopcf -t 
> lutBS08.
> bdf | gzip > lutBS08.pcf.gz
> make[6]: *** [lutBS08.pcf.gz] Error 139
> 
> See http://buildd.debian.org/build.php?arch=alpha&pkg=xfree86&ver=4.2.1-6
> 
> Can this be looked into, please?

Ah, the gzip killer bug. Works with -3, fails with -4 - where the
--rsyncable patch was introduced. However, StevenK claimed he couldn't
reproduce it, so I didn't file a bug about it.

Bug needs to be filed on gzip about the 'gzip killer' .bdf.

(Yes, I can confirm this bug on my Alpha).

-d

-- 
Daniel Stone <[EMAIL PROTECTED]>
Developer, Trinity College, University of Melbourne


pgp2ZW8jkGlUu.pgp
Description: PGP signature

Re: xfree86 4.2.1-6 build on alpha

[Falk Hueffner]

Daniel Stone <[EMAIL PROTECTED]> writes:

> On Thu, Apr 03, 2003 at 05:01:50PM +1000, Anthony Towns scrawled:
> > xfree86 4.2.1-6 fails to build on alpha due to:
> > 
> > ...
> > LD_LIBRARY_PATH=../../../exports/lib  ../../../exports/bin/bdftopcf -t lutBS08.
> > bdf | gzip > lutBS08.pcf.gz
> > make[6]: *** [lutBS08.pcf.gz] Error 139
> > 
> > See http://buildd.debian.org/build.php?arch=alpha&pkg=xfree86&ver=4.2.1-6
> > 
> > Can this be looked into, please?
> 
> Ah, the gzip killer bug. Works with -3, fails with -4 - where the
> --rsyncable patch was introduced. However, StevenK claimed he couldn't
> reproduce it, so I didn't file a bug about it.
> 
> Bug needs to be filed on gzip about the 'gzip killer' .bdf.

Already reported 3 weeks ago as #184057.

-- 
Falk


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

xfree86 4.2.1-6 build on alpha

[Anthony Towns]

Hello,

xfree86 4.2.1-6 fails to build on alpha due to:

...
LD_LIBRARY_PATH=../../../exports/lib  ../../../exports/bin/bdftopcf -t lutBS08.
bdf | gzip > lutBS08.pcf.gz
make[6]: *** [lutBS08.pcf.gz] Error 139

See http://buildd.debian.org/build.php?arch=alpha&pkg=xfree86&ver=4.2.1-6

Can this be looked into, please?

Cheers,
aj

-- 
Anthony Towns <[EMAIL PROTECTED]> 
I don't speak for anyone save myself. GPG signed mail preferred.

  ``Dear Anthony Towns: [...] Congratulations -- 
you are now certified as a Red Hat Certified Engineer!''

Re: xfree86 4.2.1-6 build on alpha

[Daniel Stone]

On Thu, Apr 03, 2003 at 05:01:50PM +1000, Anthony Towns scrawled:
> xfree86 4.2.1-6 fails to build on alpha due to:
> 
> ...
> LD_LIBRARY_PATH=../../../exports/lib  ../../../exports/bin/bdftopcf -t lutBS08.
> bdf | gzip > lutBS08.pcf.gz
> make[6]: *** [lutBS08.pcf.gz] Error 139
> 
> See http://buildd.debian.org/build.php?arch=alpha&pkg=xfree86&ver=4.2.1-6
> 
> Can this be looked into, please?

Ah, the gzip killer bug. Works with -3, fails with -4 - where the
--rsyncable patch was introduced. However, StevenK claimed he couldn't
reproduce it, so I didn't file a bug about it.

Bug needs to be filed on gzip about the 'gzip killer' .bdf.

(Yes, I can confirm this bug on my Alpha).

-d

-- 
Daniel Stone <[EMAIL PROTECTED]>
Developer, Trinity College, University of Melbourne


pgp0.pgp
Description: PGP signature

xfree86 4.2.1-6 build on alpha

[Anthony Towns]

Hello,

xfree86 4.2.1-6 fails to build on alpha due to:

...
LD_LIBRARY_PATH=../../../exports/lib  ../../../exports/bin/bdftopcf -t lutBS08.
bdf | gzip > lutBS08.pcf.gz
make[6]: *** [lutBS08.pcf.gz] Error 139

See http://buildd.debian.org/build.php?arch=alpha&pkg=xfree86&ver=4.2.1-6

Can this be looked into, please?

Cheers,
aj

-- 
Anthony Towns <[EMAIL PROTECTED]> 
I don't speak for anyone save myself. GPG signed mail preferred.

  ``Dear Anthony Towns: [...] Congratulations -- 
you are now certified as a Red Hat Certified Engineer!''


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]