Package: diffoscope Severity: wishlist X-Debbugs-Cc: Sam James <s...@gentoo.org>
When comparing two XZ compressed files that decompress to identical
data, please compare the metadata before comparing compressed data.
The xz --list option can be used for this, from the manual page:
Print information about compressed files. No uncompressed output is
produced, and no files are created or removed. In list mode, the
program cannot read the compressed data from standard input or from
other unseekable sources.
The default listing shows basic information about files, one file
per line. To get more detailed information, use also the --verbose
option. For even more information, use --verbose twice, but note
that this may be slow, because getting all the extra information
requires many seeks. The width of verbose output exceeds 80
characters, so piping the output to, for example, less -S may be
convenient if the terminal isn't wide enough.
The exact output may vary between xz versions and different locales.
For machine-readable output, --robot --list should be used.
In addition, the filename printed is the one from the command-line,
there does not appear to be filename metadata embedded in XZ files,
so the filename could be stripped before comparison where possible,
since diffoscope already compares filenames at a different layer.
The human-readable output may change between xz versions, so the
filename stripping will be brittle and could break on xz updates,
but comparing human-readable output is more user-friendly than
comparing the robot output, so any stripping would have to be
best-effort, but since any tests for it could break on xz upgrades,
and probably many comparisons will be between the same filenames within
different dirs, it might not be worth adding filename stripping yet,
until xz itself offers an option for hiding the filenames.
In addition, currently the xz --list option only supports the xz file
format and does not support the lzma and lzip file formats.
$ echo foo > foo
$ xz -0 < foo > foo.0.xz
$ xz -9 < foo > foo.9.xz
$ diffoscope foo.0.xz foo.9.xz
--- foo.0.xz
+++ foo.9.xz
│┄ Format-specific differences are supported for XZ compressed files but no
file-specific differences were detected; falling back to a binary diff. file(1)
reports: XZ compressed data, checksum CRC64
@@ -1,4 +1,4 @@
00000000: fd37 7a58 5a00 0004 e6d6 b446 0200 2101 .7zXZ......F..!.
-00000010: 0c00 0000 8f98 419c 0100 0366 6f6f 0a00 ......A....foo..
+00000010: 1c00 0000 10cf 58cc 0100 0366 6f6f 0a00 ......X....foo..
00000020: ffd7 ac5a 3031 9cf2 0001 1c04 6f2c 9cc1 ...Z01......o,..
00000030: 1fb6 f37d 0100 0000 0004 595a ...}......YZ
$ diff -u <(xz --list foo.0.xz) <(xz --list foo.9.xz)
--- /dev/fd/63 2024-04-20 08:26:27.769377608 +0800
+++ /dev/fd/62 2024-04-20 08:26:27.769377608 +0800
@@ -1,2 +1,2 @@
Strms Blocks Compressed Uncompressed Ratio Check Filename
- 1 1 60 B 4 B --- CRC64 foo.0.xz
+ 1 1 60 B 4 B --- CRC64 foo.9.xz
$ diff -u <(xz --list --verbose foo.0.xz) <(xz --list --verbose foo.9.xz)
--- /dev/fd/63 2024-04-20 08:26:43.701196927 +0800
+++ /dev/fd/62 2024-04-20 08:26:43.705196881 +0800
@@ -1,4 +1,4 @@
-foo.0.xz (1/1)
+foo.9.xz (1/1)
Streams: 1
Blocks: 1
Compressed size: 60 B
$ diff -u <(xz --list --verbose --verbose foo.0.xz) <(xz --list --verbose
--verbose foo.9.xz)
--- /dev/fd/63 2024-04-20 08:26:56.029056126 +0800
+++ /dev/fd/62 2024-04-20 08:26:56.029056126 +0800
@@ -1,4 +1,4 @@
-foo.0.xz (1/1)
+foo.9.xz (1/1)
Streams: 1
Blocks: 1
Compressed size: 60 B
@@ -11,7 +11,7 @@
1 1 0 0 60
4 --- CRC64 0
Blocks:
Stream Block CompOffset UncompOffset TotalSize
UncompSize Ratio Check CheckVal Header Flags CompSize
MemUsage Filters
- 1 1 12 0 28
4 7.000 CRC64 f29c31305aacd7ff 12 -- 8
1 MiB --lzma2=dict=256KiB
- Memory needed: 1 MiB
+ 1 1 12 0 28
4 7.000 CRC64 f29c31305aacd7ff 12 -- 8
65 MiB --lzma2=dict=64MiB
+ Memory needed: 65 MiB
Sizes in headers: No
Minimum XZ Utils version: 5.0.0
$ diff -u <(xz --robot --list --verbose --verbose foo.0.xz) <(xz --robot
--list --verbose --verbose foo.9.xz)
--- /dev/fd/63 2024-04-20 08:31:42.445584805 +0800
+++ /dev/fd/62 2024-04-20 08:31:42.449584755 +0800
@@ -1,6 +1,6 @@
-name foo.0.xz
+name foo.9.xz
file 1 1 60 4 --- CRC64 0
stream 1 1 0 0 60 4 --- CRC64 0
-block 1 1 1 12 0 28 4 7.000
CRC64 f29c31305aacd7ff 12 -- 8 327736
--lzma2=dict=256KiB
-summary 327736 no 50000002
-totals 1 1 60 4 --- CRC64 0 1
327736 no 50000002
+block 1 1 1 12 0 28 4 7.000
CRC64 f29c31305aacd7ff 12 -- 8 67174456
--lzma2=dict=64MiB
+summary 67174456 no 50000002
+totals 1 1 60 4 --- CRC64 0 1
67174456 no 50000002
--
bye,
pabs
https://wiki.debian.org/PaulWise
signature.asc
Description: This is a digitally signed message part
