Re: [nmh-workers] Reproducible build patch

[Bakul Shah]

On Mon, 23 Jul 2018 09:17:45 -0400 Ken Hornstein  wrote:
Ken Hornstein writes:
> We received the following bug report today:
>
>   http://savannah.nongnu.org/support/?109535
>
> It seems to me like reproducible builds are probably a pretty good idea
> and nmh should support them, and the patch looks relatively small.  But
> to be completely portable the date line should be (from here):
>
> https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal#Bash_.2F_POSIX_
> shell
>
> DATE_FMT="%Y-%m-%d %T +"
> SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(date +s)}"
> date=$(TZ=GMT0 date --date="@$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null || 
> date -u -r "$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null || date -u "+$DATE_F
> MT")
>
> I ... think that's right (note, I haven't tested it yet).  Everyone ok
> with this?

How about something like

timestamp() {
_TIME="${SOURCE_DATE_EPOCH:-$(date +%s)}"
if date --help 2>/dev/null; then src=-d@ ; else src=-r ; fi
date ${src}$_TIME +"%Y-%m-%d %T +"
}

Tested in linux, FreeBSD & Mac.

$  timestamp
2018-07-23 16:02:33 +
$ SOURCE_DATE_EPOCH=12345
$ timestamp
1969-12-31 19:25:45 +

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] Reproducible build patch

[Ken Hornstein]

>> >> DATE_FMT="%Y-%m-%d %T +"
>> >> SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(date +s)}"
>> >> date=$(TZ=GMT0 date --date="@$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null 
>> >> ||
>> >> date -u -r "$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null || date -u 
>> >> "+$DATE_FMT")
>
>> >I doubt that's right - at least on Fedora, /usr/bin/date doesn't work that 
>> >way
>>
>> Um, isn't /usr/bin/date on Fedora GNU date?  If so, shouldn't the --date
>> option work?
>
>Well, first off - last I heard we support nmh on more than just Linux.

Geez, Valdis, did you _read_ the link I sent along with this message??!?!

So, the "date +s" was typo on my part (I did say that I didn't test it),
but I kind of thought that obviously should have been "+%s".  But AFAICT
the other date commands all work on their respective operating systems.

--Ken

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] Reproducible build patch

[valdis . kletnieks]

On Tue, 24 Jul 2018 00:12:44 +0200, Leonardo Taccari said:

> This is what happens on NetBSD (if I'm not missing something the
> it is similar to what happens with GNU date):

Thanks for checking...

>  | + date -u -r 1532383046 '+%Y-%m-%d %T +' 2>/dev/null
>  | + date='2018-07-23 21:57:26 +'

Yes, the BSD -r seems to be the equivalent of GNU --date=.


pgpSX5EiqIvAZ.pgp
Description: PGP signature
-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] Reproducible build patch

[Leonardo Taccari]

Hello valdis, Ken and nmh-workers@,

valdis.kletni...@vt.edu writes:
> [...]
> Also, the "date +s" and the "date -u -r" don't do what you think they do...
> [...]

I think that `date +s' is a typo - probably missing a `%' as you've
noticed - while `date -u -r' (that's after the `||') should work on
most BSD systems.

> [...]
> I'm assuming that the BSD 'date'  takes -r as a timestamp rather than a 
> filename.  Somebody
> on a BSD system will need to check that.

This is what happens on NetBSD (if I'm not missing something the
it is similar to what happens with GNU date):

 | % cat /tmp/date.sh
 | DATE_FMT="%Y-%m-%d %T +"
 | SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(date +%s)}"
 | date=$(TZ=GMT0 date --date="@$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null || 
date -u -r "$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null || date -u 
"+$DATE_FMT")
 | echo $date
 | % date && sh -x date.sh
 | Mon Jul 23 23:57:26 CEST 2018
 | + DATE_FMT='%Y-%m-%d %T +'
 | + date '+%s'
 | + SOURCE_DATE_EPOCH=1532383046
 | + TZ=GMT0 date '--date=@1532383046' '+%Y-%m-%d %T +' 2>/dev/null
 | + date -u -r 1532383046 '+%Y-%m-%d %T +' 2>/dev/null
 | + date='2018-07-23 21:57:26 +'
 | + echo 2018-07-23 '21:57:26' +
 | 2018-07-23 21:57:26 +

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] Reproducible build patch

[valdis . kletnieks]

On Mon, 23 Jul 2018 14:09:13 -0400, Ken Hornstein said:
> >On Mon, 23 Jul 2018 09:17:45 -0400, Ken Hornstein said:
> >
> >> DATE_FMT="%Y-%m-%d %T +"
> >> SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(date +s)}"
> >> date=$(TZ=GMT0 date --date="@$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null 
> >> ||
> >> date -u -r "$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null || date -u 
> >> "+$DATE_FMT")

> >I doubt that's right - at least on Fedora, /usr/bin/date doesn't work that 
> >way
>
> Um, isn't /usr/bin/date on Fedora GNU date?  If so, shouldn't the --date
> option work?

Well, first off - last I heard we support nmh on more than just Linux.

Also, the "date +s" and the "date -u -r" don't do what you think they do...

16:24:10 1 [~] set -v
16:24:22 0 [~] set -x
set -x
16:24:27 0 [~] DATE_FMT="%Y-%m-%d %T +"
DATE_FMT="%Y-%m-%d %T +"
+ DATE_FMT='%Y-%m-%d %T +'
16:24:35 0 [~] SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(date +s)}"
SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(date +s)}"
++ date +s  Not set, so we take the replacement 
value...
+ SOURCE_DATE_EPOCH=s*** probably not what we intended
16:24:44 0 [~] date=$(TZ=GMT0 date --date="@$SOURCE_DATE_EPOCH" "+$DATE_FMT" 
2>/dev/null || date -u -r "$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null || date 
-u "+$DATE_FMT")
date=$(TZ=GMT0 date --date="@$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null || 
date -u -r "$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null || date -u 
"+$DATE_FMT")
++ TZ=GMT0
++ date --date=@s '+%Y-%m-%d %T +' (This dies because --date=@s  is 
bogus
++ date -u -r s '+%Y-%m-%d %T +'   (This dies because '-r s' is 
bogus as there's no file 's' to snarf a timestamp from
++ date -u '+%Y-%m-%d %T +'And we end up in the last case, 
which works.
+ date='2018-07-23 20:25:04 +'

Now, this may magically work if $SOURCE_DATE_EPOCH is set to something
sane before we enter this code - but if it isn't set, it only happens to work

Aha.  Found it.  In the debian.org link:

BUILD_DATE="$(date --utc --date="@${SOURCE_DATE_EPOCH:-$(date +%s)}" +%Y-%m-%d)"

date +%s
16:32:25 123 [~/src/nmh] date +%s
1532378290

6:38:59 0 [~] DATE_FMT="%Y-%m-%d %T +"
DATE_FMT="%Y-%m-%d %T +"
+ DATE_FMT='%Y-%m-%d %T +'
16:39:05 0 [~] SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(date +%s)}"
SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(date +%s)}"
++ date +%s
+ SOURCE_DATE_EPOCH=1532378360
16:39:20 0 [~] date=$(TZ=GMT0 date --date="@$SOURCE_DATE_EPOCH" "+$DATE_FMT" 
2>/dev/null || date -u -r "$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null || date 
-u "+$DATE_FMT")
date=$(TZ=GMT0 date --date="@$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null || 
date -u -r "$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null || date -u 
"+$DATE_FMT")
++ TZ=GMT0
++ date --date=@1532378360 '+%Y-%m-%d %T +'
+ date='2018-07-23 20:39:20 +'

That %s matters.  A lot.

I'm assuming that the BSD 'date'  takes -r as a timestamp rather than a 
filename.  Somebody
on a BSD system will need to check that.






pgpcsezBZATYB.pgp
Description: PGP signature
-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] Reproducible build patch

[Ken Hornstein]

>On Mon, 23 Jul 2018 09:17:45 -0400, Ken Hornstein said:
>
>> DATE_FMT="%Y-%m-%d %T +"
>> SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(date +s)}"
>> date=$(TZ=GMT0 date --date="@$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null ||
>date -u -r "$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null || date -u "+$DATE_FMT
>")
>>
>> I ... think that's right (note, I haven't tested it yet).  Everyone ok
>> with this?
>
>I doubt that's right - at least on Fedora, /usr/bin/date doesn't work that 
>way

Um, isn't /usr/bin/date on Fedora GNU date?  If so, shouldn't the --date
option work?

--Ken

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] Reproducible build patch

[valdis . kletnieks]

On Mon, 23 Jul 2018 09:17:45 -0400, Ken Hornstein said:

> DATE_FMT="%Y-%m-%d %T +"
> SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(date +s)}"
> date=$(TZ=GMT0 date --date="@$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null ||
date -u -r "$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null || date -u "+$DATE_FMT
")
>
> I ... think that's right (note, I haven't tested it yet).  Everyone ok
> with this?

I doubt that's right - at least on Fedora, /usr/bin/date doesn't work that 
way

date -r takes the next thing as a file whose mtime should be used
as the date source.

   -r, --reference=FILE
  display the last modification time of FILE

However, the next thing after -r is a bash variable that's probably not the 
name of a file.

Also, the 'date +s' is busted, it will just return "s".



pgpgXKVskZW6S.pgp
Description: PGP signature
-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

[nmh-workers] Reproducible build patch

[Ken Hornstein]

We received the following bug report today:

http://savannah.nongnu.org/support/?109535

It seems to me like reproducible builds are probably a pretty good idea
and nmh should support them, and the patch looks relatively small.  But
to be completely portable the date line should be (from here):

https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal#Bash_.2F_POSIX_shell

DATE_FMT="%Y-%m-%d %T +"
SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(date +s)}"
date=$(TZ=GMT0 date --date="@$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null || 
date -u -r "$SOURCE_DATE_EPOCH" "+$DATE_FMT" 2>/dev/null || date -u 
"+$DATE_FMT")

I ... think that's right (note, I haven't tested it yet).  Everyone ok
with this?

--Ken

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers