On Mon, Nov 06, 2023 at 10:42:05AM +0000, Chris Lamb wrote: > Christos Zoulas wrote: > > > For weeks now there is no useful output from diffoscope. Is there anything I > > can do to help debug the issue? > > > > tests.reproducible-builds.org [eeB] > > Wed 1 Nov 23:35:09 UTC 2023 - diffoscope 251 produced no output for > > x86_64-amd64/amd64/binary/sets/comp.tar.xz and was killed after running into > > timeout after 30m...
these issues seem to have been resolved, https://jenkins.debian.net/job/reproducible_netbsd/1029/console contains no such timeouts. > We'd actually love some help debugging this issue. As you can see from > the list archives, we have indeed been seeing this for a little while: > > https://alioth-lists.debian.net/pipermail/reproducible-builds/Week-of-Mon-20230828/thread.html > … as well as some threads in later weeks. I'm not sure those issues are related, maybe or maybe not. > However, are unsure whether this an issue with: > > a) The system running tests.reproducible-builds.org (which, I believe, > was upgraded around the time this issue appeared?) > > b) The code in diffoscope itself (which is somewhat unlikely, as there > have not been meaningful changes that might cause crashes or > timeouts.) https://jenkins.debian.net/job/reproducible_netbsd/1029/console contains this, which IMO should not happen and is an issue in diffoscope: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 767, in main sys.exit(run_diffoscope(parsed_args)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 718, in run_diffoscope difference = compare_root_paths(path1, path2) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 69, in compare_root_paths difference = compare_files(file1, file2) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 149, in compare_files return file1.compare(file2, source) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 532, in compare difference = self._compare_using_details(other, source) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 467, in _compare_using_details details.extend( File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/container.py", line 197, in compare_pair difference = compare_files( ^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 149, in compare_files return file1.compare(file2, source) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 616, in compare return self.compare_bytes(other, source) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 412, in compare_bytes return compare_binary_files(self, other, source) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 176, in compare_binary_files return Difference.from_operation( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 267, in from_operation return Difference.from_operation_exc( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 288, in from_operation_exc feeder1, operation1, excluded1 = operation_and_feeder(path1) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 285, in operation_and_feeder operation.start() File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/command.py", line 45, in start self._process = subprocess.run( ^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/subprocess.py", line 550, in run stdout, stderr = process.communicate(input, timeout=timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/subprocess.py", line 1209, in communicate stdout, stderr = self._communicate(input, endtime, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/subprocess.py", line 2128, in _communicate data = os.read(key.fd, 32768) ^^^^^^^^^^^^^^^^^^^^^^ MemoryError Fri 10 Nov 22:41:22 UTC 2023 - diffoscope 251 had trouble comparing the two builds. Please investigate x86_64-amd64/images/NetBSD-10.99.10-amd64-bios-install.img.gz Traceback (most recent call last): File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 767, in main sys.exit(run_diffoscope(parsed_args)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 718, in run_diffoscope difference = compare_root_paths(path1, path2) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 69, in compare_root_paths difference = compare_files(file1, file2) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 149, in compare_files return file1.compare(file2, source) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 532, in compare difference = self._compare_using_details(other, source) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 467, in _compare_using_details details.extend( File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/container.py", line 197, in compare_pair difference = compare_files( ^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 149, in compare_files return file1.compare(file2, source) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 616, in compare return self.compare_bytes(other, source) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 412, in compare_bytes return compare_binary_files(self, other, source) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 176, in compare_binary_files return Difference.from_operation( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 267, in from_operation return Difference.from_operation_exc( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 288, in from_operation_exc feeder1, operation1, excluded1 = operation_and_feeder(path1) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 285, in operation_and_feeder operation.start() File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/command.py", line 45, in start self._process = subprocess.run( ^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/subprocess.py", line 550, in run stdout, stderr = process.communicate(input, timeout=timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/subprocess.py", line 1209, in communicate stdout, stderr = self._communicate(input, endtime, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/subprocess.py", line 2128, in _communicate data = os.read(key.fd, 32768) ^^^^^^^^^^^^^^^^^^^^^^ MemoryError Fri 10 Nov 22:43:45 UTC 2023 - diffoscope 251 had trouble comparing the two builds. Please investigate x86_64-amd64/images/NetBSD-10.99.10-amd64-install.img.gz -- cheers, Holger ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org ⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C ⠈⠳⣄ 三人成虎- Three men make a tiger. In other words, if one guy says "there's a tiger over there" you might not believe them, if three guys in a row all say this- you think there's a tiger there. A lie, repeated often enough, will be accepted as truth.
signature.asc
Description: PGP signature
_______________________________________________ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
