[gdal-dev] GeoTIFF and LZWDecode (LZWEncode?) errors
Hi When batch processing GeoTIFF files, I tend to get this problem a bit too often: Warning 1: LZWDecode:LZWDecode: Strip 11 not terminated with EOI code ERROR 1: LZWDecode:Not enough data at scanline 11 (short 30 bytes) ERROR 1: TIFFReadEncodedStrip() failed. band 1: IReadBlock failed at X offset 0, Y offset 11 ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 11 The strip, scanline and offsets vary. It normally happens when I process from clients (computers) with the files located on servers (on local network), and I can reprocess a file again without getting the error, or possible an error with a different strip, scanline or offset. This special case is caused by gdaldem.exe and the error is reported by gdal_translate.exe using GDAL 1.9, but I have experienced this error using GDAL 1.9.2 and I think I saw it using GDAL 1.10 also. I have two issues here: 1. Why the problem is happening to begin with? 2. Why does the Warning and/or Error not trigger ERRORLEVEL 1? The first bullet I think can be very difficult to debug, so I seek experiences here. The second bullet is really frustrating, because I have to look through all my clients looking for output similar to the message above - and then reprocess all crashed files again. One solution I have seen somewhere is to not compressing the files, but this results in quite an increase in disk space usage - way too high for me. Kind regards, Casper ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] GeoTIFF and LZWDecode (LZWEncode?) errors
Casper, I'm not sure to have understood if you have this problem while reading or generating compressed files. Could you provide a way (that is to say provide both the input data and the command line used) of reproducing this ? Best regards, Even > Hi > > When batch processing GeoTIFF files, I tend to get this problem a bit too > often: > > Warning 1: LZWDecode:LZWDecode: Strip 11 not terminated with EOI code > ERROR 1: LZWDecode:Not enough data at scanline 11 (short 30 bytes) > ERROR 1: TIFFReadEncodedStrip() failed. > > band 1: IReadBlock failed at X offset 0, Y offset 11 > ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 11 > > The strip, scanline and offsets vary. It normally happens when I process from > clients (computers) with the files located on servers (on local network), and > I can reprocess a file again without getting the error, or possible an error > with a different strip, scanline or offset. This special case is caused by > gdaldem.exe and the error is reported by gdal_translate.exe using GDAL 1.9, > but I have experienced this error using GDAL 1.9.2 and I think I saw it using > GDAL 1.10 also. > > I have two issues here: > > 1. Why the problem is happening to begin with? > > 2. Why does the Warning and/or Error not trigger ERRORLEVEL 1? > > The first bullet I think can be very difficult to debug, so I seek > experiences here. The second bullet is really frustrating, because I have to > look through all my clients looking for output similar to the message above - > and then reprocess all crashed files again. > > One solution I have seen somewhere is to not compressing the files, but this > results in quite an increase in disk space usage - way too high for me. > > > Kind regards, Casper > ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] GeoTIFF and LZWDecode (LZWEncode?) errors
Hi Even. I cannot reproduce the error, it just occurs when I'm batch processing and not necessarily at the same place/file. That's why I asked for experience. However, when I get the error again, I can provide the corrupted file and then its possible to see that ex. Gdal_translate doesn't set ERRORLEVEL 1 or similar even though the process cannot continue. Kind regards, Casper -Original Message- From: Even Rouault [mailto:even.roua...@mines-paris.org] Sent: 11. juli 2013 15:40 To: Casper Børgesen (CABO) Cc: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] GeoTIFF and LZWDecode (LZWEncode?) errors Casper, I'm not sure to have understood if you have this problem while reading or generating compressed files. Could you provide a way (that is to say provide both the input data and the command line used) of reproducing this ? Best regards, Even > Hi > > When batch processing GeoTIFF files, I tend to get this problem a bit > too > often: > > Warning 1: LZWDecode:LZWDecode: Strip 11 not terminated with EOI code > ERROR 1: LZWDecode:Not enough data at scanline 11 (short 30 bytes) > ERROR 1: TIFFReadEncodedStrip() failed. > > band 1: IReadBlock failed at X offset 0, Y offset 11 ERROR 1: > GetBlockRef failed at X block offset 0, Y block offset 11 > > The strip, scanline and offsets vary. It normally happens when I > process from clients (computers) with the files located on servers (on > local network), and I can reprocess a file again without getting the > error, or possible an error with a different strip, scanline or > offset. This special case is caused by gdaldem.exe and the error is > reported by gdal_translate.exe using GDAL 1.9, but I have experienced > this error using GDAL 1.9.2 and I think I saw it using GDAL 1.10 also. > > I have two issues here: > > 1. Why the problem is happening to begin with? > > 2. Why does the Warning and/or Error not trigger ERRORLEVEL 1? > > The first bullet I think can be very difficult to debug, so I seek > experiences here. The second bullet is really frustrating, because I > have to look through all my clients looking for output similar to the > message above - and then reprocess all crashed files again. > > One solution I have seen somewhere is to not compressing the files, > but this results in quite an increase in disk space usage - way too high for > me. > > > Kind regards, Casper > ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] GeoTIFF and LZWDecode (LZWEncode?) errors
Le jeudi 11 juillet 2013 15:56:56, Casper Børgesen (CABO) a écrit : > Hi Even. > > I cannot reproduce the error, it just occurs when I'm batch processing and > not necessarily at the same place/file. That's why I asked for experience. > However, when I get the error again, I can provide the corrupted file and > then its possible to see that ex. Gdal_translate doesn't set ERRORLEVEL 1 > or similar even though the process cannot continue. Normally it should return an error, but if the error happens during the closing of the dataset (writes might be deferred), then it would go unnoticed. That's potentially what happens in your case, but difficult to know without closer investigation. > > Kind regards, Casper > > > -Original Message- > From: Even Rouault [mailto:even.roua...@mines-paris.org] > Sent: 11. juli 2013 15:40 > To: Casper Børgesen (CABO) > Cc: gdal-dev@lists.osgeo.org > Subject: Re: [gdal-dev] GeoTIFF and LZWDecode (LZWEncode?) errors > > Casper, > > I'm not sure to have understood if you have this problem while reading or > generating compressed files. Could you provide a way (that is to say > provide both the input data and the command line used) of reproducing this > ? > > Best regards, > > Even > > > Hi > > > > When batch processing GeoTIFF files, I tend to get this problem a bit > > too > > often: > > > > Warning 1: LZWDecode:LZWDecode: Strip 11 not terminated with EOI code > > ERROR 1: LZWDecode:Not enough data at scanline 11 (short 30 bytes) > > ERROR 1: TIFFReadEncodedStrip() failed. > > > > band 1: IReadBlock failed at X offset 0, Y offset 11 ERROR 1: > > GetBlockRef failed at X block offset 0, Y block offset 11 > > > > The strip, scanline and offsets vary. It normally happens when I > > process from clients (computers) with the files located on servers (on > > local network), and I can reprocess a file again without getting the > > error, or possible an error with a different strip, scanline or > > offset. This special case is caused by gdaldem.exe and the error is > > reported by gdal_translate.exe using GDAL 1.9, but I have experienced > > this error using GDAL 1.9.2 and I think I saw it using GDAL 1.10 also. > > > > I have two issues here: > > > > 1. Why the problem is happening to begin with? > > > > 2. Why does the Warning and/or Error not trigger ERRORLEVEL 1? > > > > The first bullet I think can be very difficult to debug, so I seek > > experiences here. The second bullet is really frustrating, because I > > have to look through all my clients looking for output similar to the > > message above - and then reprocess all crashed files again. > > > > One solution I have seen somewhere is to not compressing the files, > > but this results in quite an increase in disk space usage - way too high > > for me. > > > > > > Kind regards, Casper -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
