> On Aug 8, 2026, at 08:00, Chao Li <[email protected]> wrote: > > > >> On Aug 8, 2026, at 03:42, Zsolt Parragi <[email protected]> wrote: >> >> Hello! >> >> Isn't this a generic problem, also present in gzip/lz4 compressed backups? > > Maybe, but I haven’t checked them yet. I will do that next week.
I just verified gzip has the same problem using a similar script:
```
workdir=$(mktemp -d /tmp/gzip-trunc.XXXXXX)
mkdir "$workdir/truncated"
dd if=/dev/zero of="$workdir/base.tar" bs=1024 count=2
gzip -c "$workdir/base.tar" > "$workdir/base.tar.gz"
size=$(stat -f %z "$workdir/base.tar.gz")
dd if="$workdir/base.tar.gz" of="$workdir/truncated/base.tar.gz" bs=1
count=$((size - 1))
manifest_prefix=$'{"PostgreSQL-Backup-Manifest-Version": 1,\n "Files": [],\n
"WAL-Ranges": [],\n'
printf '%s' "$manifest_prefix" > "$workdir/manifest-prefix"
manifest_checksum=$(shasum -a 256 "$workdir/manifest-prefix" | awk '{print $1}')
printf '%s"Manifest-Checksum": "%s"}\n' "$manifest_prefix" "$manifest_checksum"
> "$workdir/truncated/backup_manifest"
gzip -t "$workdir/truncated/base.tar.gz"
pg_verifybackup -F t -s "$workdir/truncated"
```
gzip fails, while pg_verifybackup succeeds.
Looking at the code, astreamer_gzip_decompressor_content() stops processing
when it has consumed the input buffer. However, the inflate() doc, see [2] and
[3], says that only Z_STREAM_END indicates that decompression has completed. My
debugging also shows that, when reading the truncated gzip file, inflate()
returns Z_OK, while it returns Z_STREAM_END for a valid gzip file. Therefore,
this patch records whether inflate() has returned Z_STREAM_END and checks that
state in astreamer_gzip_decompressor_finalize().
Unlike zstd, we don't need to call inflate() again in
astreamer_gzip_decompressor_finalize(). Once all input has been consumed
without Z_STREAM_END, calling inflate() with no input cannot complete the
stream.
See attached 0002 for the fix of gzip streamer. I will check the lz4 streamer
next.
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
v2-0001-Fix-detection-of-truncated-zstd-compressed-backup.patch
Description: Binary data
v2-0002-Fix-detection-of-truncated-gzip-compressed-backup.patch
Description: Binary data
