Bug#1035704: proj: reproducible-builds: timezone-dependent timestamps in .gsb/.gtx files
Control: tags -1 - moreinfo + pending On 5/30/23 00:35, Vagrant Cascadian wrote: On 2023-05-29, Sebastiaan Couwenberg wrote: On 5/29/23 06:13, Vagrant Cascadian wrote: On 2023-05-29, Sebastiaan Couwenberg wrote: Does TZ=UTC also work when set in the environment? If so, that could be passed to the unshar commands in d/rules. I would expect that to work as well, which I though of shortly after sending the updated patch... though did not yet test it! Can you test that? Tested! Works! Patch attached! Thanks for confirming the theory. Applied in git. Kind Regards, Bas -- GPG Key ID: 4096R/6750F10AE88D4AF1 Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1
Bug#1035704: proj: reproducible-builds: timezone-dependent timestamps in .gsb/.gtx files
On 2023-05-29, Sebastiaan Couwenberg wrote: > On 5/29/23 06:13, Vagrant Cascadian wrote: >> On 2023-05-29, Sebastiaan Couwenberg wrote: >>> On 5/28/23 23:38, Vagrant Cascadian wrote: That said, I think it really is the touch commands in debian/datumgrids* as touch's timestamp modification is timezone dependent in many cases... The attached patch fixes this by setting the TZ=UTC as an environment variable in the debian/datumgrids*.shar files. ... >>> Patching the shar files is not ideal, when their content is modified >>> these changes will be lost. >>> >>> shar/unshar should be more likely be patched. I am not familiar with shar/unshar, but sure, adding support for SOURCE_DATE_EPOCH would be welcome... >>> Does TZ=UTC also work when set in the environment? If so, that could be >>> passed to the unshar commands in d/rules. >> >> I would expect that to work as well, which I though of shortly after >> sending the updated patch... though did not yet test it! > > Can you test that? Tested! Works! Patch attached! > Otherwise we'll have to upload to experimental. As much as I would love to see it fixed in time for bookworm, my guess is that it is a bit late already... live well, vagrant From 24865b8c2cda67a39774415e540dfc24ad243458 Mon Sep 17 00:00:00 2001 From: Vagrant Cascadian Date: Mon, 29 May 2023 15:28:36 -0700 Subject: [PATCH] debian/rules: Use UTC timezone when calling unshar. (Closes: #1035704) Without this, the touch calls in debian/datumgrids*.shar result in timezone-specific differences in various .gsb and .gtx files. https://reproducible-builds.org/docs/timezones/ --- debian/rules | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/debian/rules b/debian/rules index a778f4d..3a624ff 100755 --- a/debian/rules +++ b/debian/rules @@ -20,8 +20,8 @@ UPSTREAM_VERSION = $(shell echo $(DEB_VERSION_UPSTREAM) | sed -e 's/\+.*//') datumgrids: datumgrids-stamp datumgrids-stamp: - unshar -c -d $(CURDIR)/data $(CURDIR)/debian/datumgrids.shar - unshar -c -d $(CURDIR)/data $(CURDIR)/debian/datumgrids-ch.shar + TZ=UTC unshar -c -d $(CURDIR)/data $(CURDIR)/debian/datumgrids.shar + TZ=UTC unshar -c -d $(CURDIR)/data $(CURDIR)/debian/datumgrids-ch.shar touch $@ %: -- 2.39.2 signature.asc Description: PGP signature
Bug#1035704: proj: reproducible-builds: timezone-dependent timestamps in .gsb/.gtx files
On 5/29/23 06:13, Vagrant Cascadian wrote:
On 2023-05-29, Sebastiaan Couwenberg wrote:
On 5/28/23 23:38, Vagrant Cascadian wrote:
That said, I think it really is the touch commands in debian/datumgrids*
as touch's timestamp modification is timezone dependent in many cases...
The attached patch fixes this by setting the TZ=UTC as an environment
variable in the debian/datumgrids*.shar files.
I also had success with a patch where the touch calls are done with
--date=${SOURCE_DATE_EPOCH} which also worked for me (as touch assumes
to be TZ=UTC in this case)... if that would be preferable, I can also
provide a patch for that.
Patching the shar files is not ideal, when their content is modified
these changes will be lost.
shar/unshar should be more likely be patched.
Does TZ=UTC also work when set in the environment? If so, that could be
passed to the unshar commands in d/rules.
I would expect that to work as well, which I though of shortly after
sending the updated patch... though did not yet test it!
Can you test that? Otherwise we'll have to upload to experimental.
Kind Regards,
Bas
--
GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1
Bug#1035704: proj: reproducible-builds: timezone-dependent timestamps in .gsb/.gtx files
On 2023-05-29, Sebastiaan Couwenberg wrote:
> On 5/28/23 23:38, Vagrant Cascadian wrote:
>> That said, I think it really is the touch commands in debian/datumgrids*
>> as touch's timestamp modification is timezone dependent in many cases...
>>
>> The attached patch fixes this by setting the TZ=UTC as an environment
>> variable in the debian/datumgrids*.shar files.
>>
>> I also had success with a patch where the touch calls are done with
>> --date=${SOURCE_DATE_EPOCH} which also worked for me (as touch assumes
>> to be TZ=UTC in this case)... if that would be preferable, I can also
>> provide a patch for that.
>
> Patching the shar files is not ideal, when their content is modified
> these changes will be lost.
>
> shar/unshar should be more likely be patched.
>
> Does TZ=UTC also work when set in the environment? If so, that could be
> passed to the unshar commands in d/rules.
I would expect that to work as well, which I though of shortly after
sending the updated patch... though did not yet test it!
live well,
vagrant
signature.asc
Description: PGP signature
Bug#1035704: proj: reproducible-builds: timezone-dependent timestamps in .gsb/.gtx files
On 5/28/23 23:38, Vagrant Cascadian wrote:
On 2023-05-08, Vagrant Cascadian wrote:
On 2023-05-09, Sebastiaan Couwenberg wrote:
On 5/9/23 05:47, Sebastiaan Couwenberg wrote:
On 5/8/23 22:43, Vagrant Cascadian wrote:
On 2023-05-08, Sebastiaan Couwenberg wrote:
On 5/8/23 02:07, Vagrant Cascadian wrote:
The attached patch fixes this by not touching these files during the
build process.
From shar(1):
"
-m, --no-timestamp
do not restore modification times.
...
That should be used when generating the archives instead of your patch
to not have the mtimes touched when unpacking the archives.
...
But the diffoscope-results only show a few of the files from the shar
archives with different mtimes, and all of them get touched when
unpacking the archive just before the configure target is executed.
Well, I too was perplexed why other files were not affected, but it is
consistently those .gsb and .gtx files, and the submitted patch allows
to consistently build reproducibly in the dozens of times I've build
it...
shar actually makes the timestamps reproducible by always using the
mtime recorded in the archive instead of the time the files were
unpacked.
Something else is likely changing the mtime after the files are
unpacked. Some of these grids are used by the testsuite for example.
I will try to look into it further, but without really being familiar
with the proj codebase (or even what proj itself does)... any additional
very specific suggestions where to look next would definitely be
appreciated! :)
CMake's configure_file() is used to copy the .gsb & .gtx files from
CMAKE_CURRENT_SOURCE_DIR to CMAKE_CURRENT_BINARY_DIR, that might be
touching the mtimes too. See: data/CMakeLists.txt
Thanks, that is definitely worth taking a look at...
...
Will try to poke at it more tomorrow...
I had no luck with poking at that approach... though did not have great
ideas what to even try there.
That said, I think it really is the touch commands in debian/datumgrids*
as touch's timestamp modification is timezone dependent in many cases...
The attached patch fixes this by setting the TZ=UTC as an environment
variable in the debian/datumgrids*.shar files.
I also had success with a patch where the touch calls are done with
--date=${SOURCE_DATE_EPOCH} which also worked for me (as touch assumes
to be TZ=UTC in this case)... if that would be preferable, I can also
provide a patch for that.
Patching the shar files is not ideal, when their content is modified
these changes will be lost.
shar/unshar should be more likely be patched.
Does TZ=UTC also work when set in the environment? If so, that could be
passed to the unshar commands in d/rules.
Kind Regards,
Bas
--
GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1
Bug#1035704: proj: reproducible-builds: timezone-dependent timestamps in .gsb/.gtx files
On 2023-05-08, Vagrant Cascadian wrote:
> On 2023-05-09, Sebastiaan Couwenberg wrote:
>> On 5/9/23 05:47, Sebastiaan Couwenberg wrote:
>>> On 5/8/23 22:43, Vagrant Cascadian wrote:
On 2023-05-08, Sebastiaan Couwenberg wrote:
> On 5/8/23 02:07, Vagrant Cascadian wrote:
>> The attached patch fixes this by not touching these files during the
>> build process.
>
> From shar(1):
>
> "
> -m, --no-timestamp
> do not restore modification times.
...
> That should be used when generating the archives instead of your patch
> to not have the mtimes touched when unpacking the archives.
...
> But the diffoscope-results only show a few of the files from the shar
> archives with different mtimes, and all of them get touched when
> unpacking the archive just before the configure target is executed.
Well, I too was perplexed why other files were not affected, but it is
consistently those .gsb and .gtx files, and the submitted patch allows
to consistently build reproducibly in the dozens of times I've build
it...
> shar actually makes the timestamps reproducible by always using the
> mtime recorded in the archive instead of the time the files were
> unpacked.
>
> Something else is likely changing the mtime after the files are
> unpacked. Some of these grids are used by the testsuite for example.
I will try to look into it further, but without really being familiar
with the proj codebase (or even what proj itself does)... any additional
very specific suggestions where to look next would definitely be
appreciated! :)
>>>
>>> CMake's configure_file() is used to copy the .gsb & .gtx files from
>>> CMAKE_CURRENT_SOURCE_DIR to CMAKE_CURRENT_BINARY_DIR, that might be
>>> touching the mtimes too. See: data/CMakeLists.txt
>
> Thanks, that is definitely worth taking a look at...
...
> Will try to poke at it more tomorrow...
I had no luck with poking at that approach... though did not have great
ideas what to even try there.
That said, I think it really is the touch commands in debian/datumgrids*
as touch's timestamp modification is timezone dependent in many cases...
The attached patch fixes this by setting the TZ=UTC as an environment
variable in the debian/datumgrids*.shar files.
I also had success with a patch where the touch calls are done with
--date=${SOURCE_DATE_EPOCH} which also worked for me (as touch assumes
to be TZ=UTC in this case)... if that would be preferable, I can also
provide a patch for that.
live well,
vagrant
From 5e019591780bcf0b61511cc242d1636c9ec909ca Mon Sep 17 00:00:00 2001
From: Vagrant Cascadian
Date: Sun, 28 May 2023 12:54:59 -0700
Subject: [PATCH] debian/datumgrids*.shar: Use UTC timezone when touching
files.
Without this, the touch calls result in timezone-specific differences
in various .gsb and .gtx files.
https://reproducible-builds.org/docs/timezones/
---
debian/datumgrids-ch.shar | 4
debian/datumgrids.shar| 4
2 files changed, 8 insertions(+)
diff --git a/debian/datumgrids-ch.shar b/debian/datumgrids-ch.shar
index 308ca93..313a3c6 100644
--- a/debian/datumgrids-ch.shar
+++ b/debian/datumgrids-ch.shar
@@ -35,6 +35,10 @@ gettext_dir=
locale_dir=
set_echo=false
+# reproducible builds: Use UTC timezone to ensure consistent
+# timestamps on touched files.
+export TZ=UTC
+
for dir in $PATH
do
if test -f $dir/gettext \
diff --git a/debian/datumgrids.shar b/debian/datumgrids.shar
index d9a0161..2dec48e 100644
--- a/debian/datumgrids.shar
+++ b/debian/datumgrids.shar
@@ -48,6 +48,10 @@ gettext_dir=
locale_dir=
set_echo=false
+# reproducible builds: Use UTC timezone to ensure consistent
+# timestamps on touched files.
+export TZ=UTC
+
for dir in $PATH
do
if test -f $dir/gettext \
--
2.39.2
signature.asc
Description: PGP signature
Bug#1035704: proj: reproducible-builds: timezone-dependent timestamps in .gsb/.gtx files
On 2023-05-09, Sebastiaan Couwenberg wrote: > On 5/9/23 05:47, Sebastiaan Couwenberg wrote: >> On 5/8/23 22:43, Vagrant Cascadian wrote: >>> On 2023-05-08, Sebastiaan Couwenberg wrote: On 5/8/23 02:07, Vagrant Cascadian wrote: > The attached patch fixes this by not touching these files during the > build process. From shar(1): " -m, --no-timestamp do not restore modification times. Avoid generating 'touch' commands to restore the file modification dates when unpacking files from the archive. When file modification times are not preserved, project build programs like "make" will see built files older than the files they get built from. This is why, when this option is not used, a special effort is made to restore timestamps. " That should be used when generating the archives instead of your patch to not have the mtimes touched when unpacking the archives. >>> >>> Is it actually a problem to allow dpkg to normalize the timestamps on >>> these files rather than forcefully setting them to ... a value from a >>> shar archive? It is perhaps a naive question; I really do not know. >> >> Where does dpkg normalize the timestamps? I thought it did as part of dpkg-deb, from the dpkg-deb.1 manpage: SOURCE_DATE_EPOCH If set, it will be used as the timestamp (as seconds since the epoch) in the deb(5)'s ar(5) container and used to clamp the mtime in the tar(5) file entries. The other adjacent files appear to use a timestamp consistent with the last debian/changelog entry: -rw-r--r--···0·root·(0)·root·(0)·1097·2022-12-01·08:50:03.00·./usr/share/proj/CH Which is what the dpkg-buildpackage.1 manpage says is used to SOURCE_DATE_EPOCH... SOURCE_DATE_EPOCH This variable is set to the Unix timestamp since the epoch of the latest entry in debian/changelog, if it is not already defined. >> shar sets the timestamps when the archive is unpacked before the package >> built starts. >> >> Some of the files in the diffoscope-results are only installed in >> proj-data and not used otherwise during the build. >> >> * BETA2007.gsb is used in test/gie/DHDN_ETRS89.gie >> >> * CHENYX06.gsb/CHENYX06_etrs.gsb/CHENYX06a.gsb are only installed >> >> * egm96_15.gtx is used in test/gie/deformation.gie, >> test/gie/more_builtins.gie, test/gie/4D-API_cs2cs-style.gie, and >> test/cli/testdatumfile >> >> * ntf_r93.gsb is used in test/gie/more_builtins.gie, >> test/gie/4D-API_cs2cs-style.gie, and test/cli/testdatumfile >> >> * nzgd2kgrid0005.gsb is used in unit tests This seems like a strong lead, but I would expect the test suite to run (and thus modify the timestamps) before dpkg-deb sets the timestamps on the built packages... so I am still a bit perplexed, but probably just misunderstanding exactly what happens when and where. :) But the diffoscope-results only show a few of the files from the shar archives with different mtimes, and all of them get touched when unpacking the archive just before the configure target is executed. >>> >>> Well, I too was perplexed why other files were not affected, but it is >>> consistently those .gsb and .gtx files, and the submitted patch allows >>> to consistently build reproducibly in the dozens of times I've build >>> it... >>> >>> shar actually makes the timestamps reproducible by always using the mtime recorded in the archive instead of the time the files were unpacked. Something else is likely changing the mtime after the files are unpacked. Some of these grids are used by the testsuite for example. >>> >>> I will try to look into it further, but without really being familiar >>> with the proj codebase (or even what proj itself does)... any additional >>> very specific suggestions where to look next would definitely be >>> appreciated! :) >> >> CMake's configure_file() is used to copy the .gsb & .gtx files from >> CMAKE_CURRENT_SOURCE_DIR to CMAKE_CURRENT_BINARY_DIR, that might be >> touching the mtimes too. See: data/CMakeLists.txt Thanks, that is definitely worth taking a look at... > Seeing how the mtimes are off by two hours, this could be the difference > between UTC and CEST. For clarity, it is almost definitely timezone related, but actually UTC-14 and UTC+12 (e.g. off by 26 hours), which is used for the TZ variable on tests.reproducible-builds.org: ···83696·2018-02-22·07:28:23.00·./usr/share/proj/BETA2007.gsb vs. ···83696·2018-02-21·05:28:23.00·./usr/share/proj/BETA2007.gsb Note the difference of date... > The latter was in effect when the archives were > created: > > $ grep "Made on" debian/datumgrids*.shar > debian/datumgrids-ch.shar:# Made on 2018-02-26 22:27 CET by . >
Bug#1035704: proj: reproducible-builds: timezone-dependent timestamps in .gsb/.gtx files
On 5/9/23 05:47, Sebastiaan Couwenberg wrote: On 5/8/23 22:43, Vagrant Cascadian wrote: On 2023-05-08, Sebastiaan Couwenberg wrote: On 5/8/23 02:07, Vagrant Cascadian wrote: The attached patch fixes this by not touching these files during the build process. From shar(1): " -m, --no-timestamp do not restore modification times. Avoid generating 'touch' commands to restore the file modification dates when unpacking files from the archive. When file modification times are not preserved, project build programs like "make" will see built files older than the files they get built from. This is why, when this option is not used, a special effort is made to restore timestamps. " That should be used when generating the archives instead of your patch to not have the mtimes touched when unpacking the archives. Is it actually a problem to allow dpkg to normalize the timestamps on these files rather than forcefully setting them to ... a value from a shar archive? It is perhaps a naive question; I really do not know. Where does dpkg normalize the timestamps? shar sets the timestamps when the archive is unpacked before the package built starts. Some of the files in the diffoscope-results are only installed in proj-data and not used otherwise during the build. * BETA2007.gsb is used in test/gie/DHDN_ETRS89.gie * CHENYX06.gsb/CHENYX06_etrs.gsb/CHENYX06a.gsb are only installed * egm96_15.gtx is used in test/gie/deformation.gie, test/gie/more_builtins.gie, test/gie/4D-API_cs2cs-style.gie, and test/cli/testdatumfile * ntf_r93.gsb is used in test/gie/more_builtins.gie, test/gie/4D-API_cs2cs-style.gie, and test/cli/testdatumfile * nzgd2kgrid0005.gsb is used in unit tests But the diffoscope-results only show a few of the files from the shar archives with different mtimes, and all of them get touched when unpacking the archive just before the configure target is executed. Well, I too was perplexed why other files were not affected, but it is consistently those .gsb and .gtx files, and the submitted patch allows to consistently build reproducibly in the dozens of times I've build it... shar actually makes the timestamps reproducible by always using the mtime recorded in the archive instead of the time the files were unpacked. Something else is likely changing the mtime after the files are unpacked. Some of these grids are used by the testsuite for example. I will try to look into it further, but without really being familiar with the proj codebase (or even what proj itself does)... any additional very specific suggestions where to look next would definitely be appreciated! :) CMake's configure_file() is used to copy the .gsb & .gtx files from CMAKE_CURRENT_SOURCE_DIR to CMAKE_CURRENT_BINARY_DIR, that might be touching the mtimes too. See: data/CMakeLists.txt Seeing how the mtimes are off by two hours, this could be the difference between UTC and CEST. The latter was in effect when the archives were created: $ grep "Made on" debian/datumgrids*.shar debian/datumgrids-ch.shar:# Made on 2018-02-26 22:27 CET by . debian/datumgrids.shar:# Made on 2018-09-15 20:13 CEST by . But why does it only affect the binary GSB & GTX files, and not also the binary ntv1_can.dat file or text files like README.DATUMGRID and the init files (the ones without extensions)? Kind Regards, Bas -- GPG Key ID: 4096R/6750F10AE88D4AF1 Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1
Bug#1035704: proj: reproducible-builds: timezone-dependent timestamps in .gsb/.gtx files
On 5/8/23 22:43, Vagrant Cascadian wrote: On 2023-05-08, Sebastiaan Couwenberg wrote: On 5/8/23 02:07, Vagrant Cascadian wrote: The attached patch fixes this by not touching these files during the build process. From shar(1): " -m, --no-timestamp do not restore modification times. Avoid generating 'touch' commands to restore the file modification dates when unpacking files from the archive. When file modification times are not preserved, project build programs like "make" will see built files older than the files they get built from. This is why, when this option is not used, a special effort is made to restore timestamps. " That should be used when generating the archives instead of your patch to not have the mtimes touched when unpacking the archives. Is it actually a problem to allow dpkg to normalize the timestamps on these files rather than forcefully setting them to ... a value from a shar archive? It is perhaps a naive question; I really do not know. Where does dpkg normalize the timestamps? shar sets the timestamps when the archive is unpacked before the package built starts. Some of the files in the diffoscope-results are only installed in proj-data and not used otherwise during the build. * BETA2007.gsb is used in test/gie/DHDN_ETRS89.gie * CHENYX06.gsb/CHENYX06_etrs.gsb/CHENYX06a.gsb are only installed * egm96_15.gtx is used in test/gie/deformation.gie, test/gie/more_builtins.gie, test/gie/4D-API_cs2cs-style.gie, and test/cli/testdatumfile * ntf_r93.gsb is used in test/gie/more_builtins.gie, test/gie/4D-API_cs2cs-style.gie, and test/cli/testdatumfile * nzgd2kgrid0005.gsb is used in unit tests But the diffoscope-results only show a few of the files from the shar archives with different mtimes, and all of them get touched when unpacking the archive just before the configure target is executed. Well, I too was perplexed why other files were not affected, but it is consistently those .gsb and .gtx files, and the submitted patch allows to consistently build reproducibly in the dozens of times I've build it... shar actually makes the timestamps reproducible by always using the mtime recorded in the archive instead of the time the files were unpacked. Something else is likely changing the mtime after the files are unpacked. Some of these grids are used by the testsuite for example. I will try to look into it further, but without really being familiar with the proj codebase (or even what proj itself does)... any additional very specific suggestions where to look next would definitely be appreciated! :) CMake's configure_file() is used to copy the .gsb & .gtx files from CMAKE_CURRENT_SOURCE_DIR to CMAKE_CURRENT_BINARY_DIR, that might be touching the mtimes too. See: data/CMakeLists.txt Kind Regards, Bas -- GPG Key ID: 4096R/6750F10AE88D4AF1 Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1
Bug#1035704: proj: reproducible-builds: timezone-dependent timestamps in .gsb/.gtx files
On 2023-05-08, Sebastiaan Couwenberg wrote: > On 5/8/23 02:07, Vagrant Cascadian wrote: >> The attached patch fixes this by not touching these files during the >> build process. > > From shar(1): > > " >-m, --no-timestamp > do not restore modification times. > > Avoid generating 'touch' commands to restore the file > modification dates when unpacking files from the archive. > > When file modification times are not preserved, project build > programs like "make" will see built files older than the files > they get built from. This is why, when this option is not > used, a special effort is made to restore timestamps. > " > > That should be used when generating the archives instead of your patch > to not have the mtimes touched when unpacking the archives. Is it actually a problem to allow dpkg to normalize the timestamps on these files rather than forcefully setting them to ... a value from a shar archive? It is perhaps a naive question; I really do not know. > But the diffoscope-results only show a few of the files from the shar > archives with different mtimes, and all of them get touched when > unpacking the archive just before the configure target is executed. Well, I too was perplexed why other files were not affected, but it is consistently those .gsb and .gtx files, and the submitted patch allows to consistently build reproducibly in the dozens of times I've build it... > shar actually makes the timestamps reproducible by always using the > mtime recorded in the archive instead of the time the files were unpacked. > > Something else is likely changing the mtime after the files are > unpacked. Some of these grids are used by the testsuite for example. I will try to look into it further, but without really being familiar with the proj codebase (or even what proj itself does)... any additional very specific suggestions where to look next would definitely be appreciated! :) live well, vagrant signature.asc Description: PGP signature
Bug#1035704: proj: reproducible-builds: timezone-dependent timestamps in .gsb/.gtx files
Control: tags -1 moreinfo On 5/8/23 02:07, Vagrant Cascadian wrote: The attached patch fixes this by not touching these files during the build process. From shar(1): " -m, --no-timestamp do not restore modification times. Avoid generating 'touch' commands to restore the file modification dates when unpacking files from the archive. When file modification times are not preserved, project build programs like "make" will see built files older than the files they get built from. This is why, when this option is not used, a special effort is made to restore timestamps. " That should be used when generating the archives instead of your patch to not have the mtimes touched when unpacking the archives. But the diffoscope-results only show a few of the files from the shar archives with different mtimes, and all of them get touched when unpacking the archive just before the configure target is executed. shar actually makes the timestamps reproducible by always using the mtime recorded in the archive instead of the time the files were unpacked. Something else is likely changing the mtime after the files are unpacked. Some of these grids are used by the testsuite for example. Kind Regards, Bas -- GPG Key ID: 4096R/6750F10AE88D4AF1 Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1
Bug#1035704: proj: reproducible-builds: timezone-dependent timestamps in .gsb/.gtx files
Source: proj
Severity: normal
Tags: patch
User: reproducible-bui...@lists.alioth.debian.org
Usertags: timestamps timezone
X-Debbugs-Cc: reproducible-b...@lists.alioth.debian.org
Various .gsb and .gtx files get touched in the build process in a way
that results in a timezone-dependent timestamp for these files in the
shipped package:
https://tests.reproducible-builds.org/debian/rb-pkg/bookworm/amd64/diffoscope-results/proj.html
-rw-r--r--···0·root·(0)·root·(0)83696·2018-02-22·07:28:23.00·./usr/share/proj/BETA2007.gsb
vs.
-rw-r--r--···0·root·(0)·root·(0)83696·2018-02-21·05:28:23.00·./usr/share/proj/BETA2007.gsb
The attached patch fixes this by not touching these files during the
build process.
According to my local tests, with this patch applied, proj should build
reproducibly on tests.reproducible-builds.org once it lands in
bookworm/testing!
Unfortunately, there are unresolved issues with build paths, which are
tested in unstable and experimental, so will not show as reproducible
there without further fixes.
Thanks for maintaining proj!
live well,
vagrant
From 25dbf89dae803e4e5e207c253d9f83c889680ed9 Mon Sep 17 00:00:00 2001
From: Vagrant Cascadian
Date: Tue, 2 May 2023 16:39:31 -0700
Subject: [PATCH] debian/datumgrids*.shar: Avoid updating timestamps.
This results in timezone-specific differences in various .gsb and .gtx
files.
https://reproducible-builds.org/docs/timestamps/
---
debian/datumgrids-ch.shar | 21 ++---
debian/datumgrids.shar| 21 ++---
2 files changed, 4 insertions(+), 38 deletions(-)
diff --git a/debian/datumgrids-ch.shar b/debian/datumgrids-ch.shar
index 308ca93..37194b7 100644
--- a/debian/datumgrids-ch.shar
+++ b/debian/datumgrids-ch.shar
@@ -84,25 +84,8 @@ st2=123123592001.59
st2tr=123123592001.5 # old SysV 14-char limit
st3=1231235901
-if touch -am -t ${st1} ${f} >/dev/null 2>&1 && \
- test ! -f ${st1} && test -f ${f}; then
- shar_touch='touch -am -t $1$2$3$4$5$6.$7 "$8"'
-
-elif touch -am ${st2} ${f} >/dev/null 2>&1 && \
- test ! -f ${st2} && test ! -f ${st2tr} && test -f ${f}; then
- shar_touch='touch -am $3$4$5$6$1$2.$7 "$8"'
-
-elif touch -am ${st3} ${f} >/dev/null 2>&1 && \
- test ! -f ${st3} && test -f ${f}; then
- shar_touch='touch -am $3$4$5$6$2 "$8"'
-
-else
- shar_touch=:
- echo
- ${echo} 'WARNING: not restoring timestamps. Consider getting and
-installing GNU '\''touch'\'', distributed in GNU coreutils...'
- echo
-fi
+# Avoid mangling timestamps
+shar_touch=:
rm -f ${st1} ${st2} ${st2tr} ${st3} ${f}
#
if test ! -d ${lock_dir} ; then :
diff --git a/debian/datumgrids.shar b/debian/datumgrids.shar
index d9a0161..f41ae56 100644
--- a/debian/datumgrids.shar
+++ b/debian/datumgrids.shar
@@ -97,25 +97,8 @@ st2=123123592001.59
st2tr=123123592001.5 # old SysV 14-char limit
st3=1231235901
-if touch -am -t ${st1} ${f} >/dev/null 2>&1 && \
- test ! -f ${st1} && test -f ${f}; then
- shar_touch='touch -am -t $1$2$3$4$5$6.$7 "$8"'
-
-elif touch -am ${st2} ${f} >/dev/null 2>&1 && \
- test ! -f ${st2} && test ! -f ${st2tr} && test -f ${f}; then
- shar_touch='touch -am $3$4$5$6$1$2.$7 "$8"'
-
-elif touch -am ${st3} ${f} >/dev/null 2>&1 && \
- test ! -f ${st3} && test -f ${f}; then
- shar_touch='touch -am $3$4$5$6$2 "$8"'
-
-else
- shar_touch=:
- echo
- ${echo} 'WARNING: not restoring timestamps. Consider getting and
-installing GNU '\''touch'\'', distributed in GNU coreutils...'
- echo
-fi
+# avoid mangling timestamps on files
+shar_touch=:
rm -f ${st1} ${st2} ${st2tr} ${st3} ${f}
#
if test ! -d ${lock_dir} ; then :
--
2.39.2
signature.asc
Description: PGP signature
