Re: [v3] Erlang
On Mon, May 23, 2016 at 01:48:11PM +0200, Pjotr Prins wrote: > Good news. The changelog for > > http://erlang.org/download/OTP-19.0-rc1.README > > says that > > OTP-13504Application(s): compiler > >The compiler will no longer put the compilation date >and time into BEAM files. That means that two BEAM >files compiled on the same computer from the same >source code and compilation options will be identical. > >Note: If you want to find out whether a BEAM file on >disk is different from the loaded code, compared the >MD5 value obtained from Mod:module_info(md5) with the >MD5 value obtained from beam_lib:md5(BeamFileForMod)` > > This means that Erlang and Elixir may go into GNU Guix :) Indeed, great news :) I will try building it with Guix today.
Re: [v3] Erlang
On Sat, Apr 09, 2016 at 12:56:49PM +0200, Pjotr Prins wrote: > On Tue, Apr 05, 2016 at 02:46:42PM -0400, Leo Famulari wrote: > > On Tue, Apr 05, 2016 at 12:02:28PM +0200, Pjotr Prins wrote: > > > I received a nice reply from Joe Armstrong. Basically he agrees with > > > the idea of deterministic builds and hashes for content of source > > > files > > > > > > http://joearms.github.io/2015/03/12/The_web_of_names.html > > > > > > and will support us if we raise our issue on the mailing list. Leo, do > > > you want to have a go - essentially sending the E-mail I wrote to Joe? > > > I can do it if you want me to. > > > > I'm more than happy to test things, but I think it's best if you send > > the message since you are actually using Erlang. But like I said before, > > if it's too much of a burden, I'll do it. Please let me know! > > A quick update: the Erlang authors are going to fix this issue as can > be followed in > > http://erlang.org/pipermail/erlang-questions/2016-April/088721.html Good news. The changelog for http://erlang.org/download/OTP-19.0-rc1.README says that OTP-13504Application(s): compiler The compiler will no longer put the compilation date and time into BEAM files. That means that two BEAM files compiled on the same computer from the same source code and compilation options will be identical. Note: If you want to find out whether a BEAM file on disk is different from the loaded code, compared the MD5 value obtained from Mod:module_info(md5) with the MD5 value obtained from beam_lib:md5(BeamFileForMod)` This means that Erlang and Elixir may go into GNU Guix :) Pj.
Re: [v3] Erlang
On Tue, Apr 05, 2016 at 02:46:42PM -0400, Leo Famulari wrote: > On Tue, Apr 05, 2016 at 12:02:28PM +0200, Pjotr Prins wrote: > > I received a nice reply from Joe Armstrong. Basically he agrees with > > the idea of deterministic builds and hashes for content of source > > files > > > > http://joearms.github.io/2015/03/12/The_web_of_names.html > > > > and will support us if we raise our issue on the mailing list. Leo, do > > you want to have a go - essentially sending the E-mail I wrote to Joe? > > I can do it if you want me to. > > I'm more than happy to test things, but I think it's best if you send > the message since you are actually using Erlang. But like I said before, > if it's too much of a burden, I'll do it. Please let me know! A quick update: the Erlang authors are going to fix this issue as can be followed in http://erlang.org/pipermail/erlang-questions/2016-April/088721.html Pj. --
Re: [v3] Erlang
On Tue, Apr 05, 2016 at 12:02:28PM +0200, Pjotr Prins wrote: > I received a nice reply from Joe Armstrong. Basically he agrees with > the idea of deterministic builds and hashes for content of source > files > > http://joearms.github.io/2015/03/12/The_web_of_names.html > > and will support us if we raise our issue on the mailing list. Leo, do > you want to have a go - essentially sending the E-mail I wrote to Joe? > I can do it if you want me to. I'm more than happy to test things, but I think it's best if you send the message since you are actually using Erlang. But like I said before, if it's too much of a burden, I'll do it. Please let me know!
Re: [v3] Erlang
I received a nice reply from Joe Armstrong. Basically he agrees with the idea of deterministic builds and hashes for content of source files http://joearms.github.io/2015/03/12/The_web_of_names.html and will support us if we raise our issue on the mailing list. Leo, do you want to have a go - essentially sending the E-mail I wrote to Joe? I can do it if you want me to. I'll sign up to the ML too. http://erlang.org/mailman/listinfo/erlang-questions Pj.
Re: [v3] Erlang
Dear Mr Armstrong,
First of, thanks for the hard work on Erlang. I read your book when it
came out and it is cool to see how use is exploding. My latest project
is actually written in Elixir.
You may have heard of GNU Guix, the modern (functional) package
manager of the GNU project. We are trying to add Erlang and Elixir to
Guix, but we are running into the problem that building the Erlang
compiler is not deterministic and therefore not reproducible, i.e. the
beam files contain time stamps.
For normal software built by Erlang this can be overriden with
SOURCE_DATE_EPOCH, but for the compiler itself we have not found how
to do this.
Do you have a suggestion how to bootstrap the compiler with
SOURCE_DATE_EPOCH set or disable the time stamps? I am sure as a FP
compiler designer you can appreciate determinism. Because GNU Guix is
deterministic there is no need to keep track of time stamps. For hot
reloading we can assume the start of EPOCH will do the trick, right?
Pj.
On Mon, Apr 04, 2016 at 01:49:44PM -0400, Leo Famulari wrote:
> On Mon, Apr 04, 2016 at 12:50:12PM -0400, Leo Famulari wrote:
> > On Mon, Apr 04, 2016 at 10:28:02AM +0200, Pjotr Prins wrote:
> > > On Sun, Apr 03, 2016 at 11:39:24PM -0400, Leo Famulari wrote:
> > > > Debian's package exhibits this problem. The timestamps are generated in
> > > > the following places in the source code. I don't know how to approach
> > > > this problem.
> > > >
> > > > lib/kernel/test/global_SUITE_data/global_trace.erl:io:format("The
> > > > trace was generated at ~p~n", [EndTime]),
> > > > lib/reltool/bin/reltool.escript:lists:flatten(io_lib:format("%% ~s
> > > > generated at ~w ~w\n~p.\n\n",
> > > > lib/reltool/src/reltool_server.erl:IoList = io_lib:format("%%
> > > > config generated at ~w ~w\n~p.\n\n",
> > > > lib/reltool/src/reltool_target.erl:RelIoList = io_lib:format("%%
> > > > rel generated at ~w ~w\n~p.\n\n",
> > > > lib/reltool/src/reltool_target.erl:ScriptIoList = io_lib:format("%%
> > > > script generated at ~w ~w\n~p.\n\n",
> > > > lib/reltool/src/reltool_target.erl:AppIoList =
> > > > io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
> > > > lib/reltool/src/reltool_target.erl:AppIoList =
> > > > io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
> > > > lib/runtime_tools/src/erts_alloc_config.erl:"generated at
> > > > ~w-~2..0w-~2..0w ~2..0w:~2..0w.~2..0w by "
> > > > lib/sasl/src/systools_make.erl: io:format(Fd, "%% script generated
> > > > at ~w ~w\n~p.\n",
> > > > lib/wx/src/gen/gl.erl:%% The program object's information log is
> > > > updated and the program is generated at the time
> > >
> > > If there is no easy work around I suggest simply patching them.
> > > Fortunately
> > > the Erlang compiler does not change much at this level.
> >
> > The ideal solution would be to use the value of the environment variable
> > SOURCE_DATE_EPOCH if it is set, and else to behave as it does now.
> >
> > > We can also contact Joe Armstrong, the author of Erlang, to discuss
> > > this point. He appears to be approachable. I am sure he is open to
> > > the idea of deterministic builds in a deterministic build system ;)
> >
> > I could go to the Erlang IRC channel or forums (whatever they use) and
> > ask for advice. Since you are actually using Erlang, I think you would
> > be the better person to contact Joe Armstrong himself, if we decide to
> > do that.
>
> I presented the situation on IRC and it was recommended that I start the
> discussion on a mailing list.
>
> I think that the erlang-questions list [0] could be a good place to
> start.
>
> Pjotr, would you like to start the conversation? I can do it if you are
> too busy or something.
>
> [0]
> http://www.erlang.org/community
>
--
Re: [v3] Erlang
On Mon, Apr 04, 2016 at 01:49:44PM -0400, Leo Famulari wrote: > I think that the erlang-questions list [0] could be a good place to > start. Possibly, though I think it is more user-oriented. > Pjotr, would you like to start the conversation? I can do it if you are > too busy or something. I'll try and approach Joe Armstrong directly and you can do the ML at the same time. Cc this list. I don't think, however, that we should wait for that fix. It may take a while before it reaches 'stable', or something other tools can depend upon. I had the same thing with ldc (the LLVM D compiler now in Guix) and patched tests for the time being (for ldc I'll probably be able to resolve it at Dconf in May). If we want something usable within 3-6 months I suggest we patch a hard coded date in. It is only the compiler itself we are talking about. A patch may also help explain things ;) I would love to have Erlang and Elixir in GNU Guix. GNU Guix targeting programmers makes a load of sense. Pj.
Re: [v3] Erlang
On Mon, Apr 04, 2016 at 12:50:12PM -0400, Leo Famulari wrote:
> On Mon, Apr 04, 2016 at 10:28:02AM +0200, Pjotr Prins wrote:
> > On Sun, Apr 03, 2016 at 11:39:24PM -0400, Leo Famulari wrote:
> > > Debian's package exhibits this problem. The timestamps are generated in
> > > the following places in the source code. I don't know how to approach
> > > this problem.
> > >
> > > lib/kernel/test/global_SUITE_data/global_trace.erl:io:format("The
> > > trace was generated at ~p~n", [EndTime]),
> > > lib/reltool/bin/reltool.escript:lists:flatten(io_lib:format("%% ~s
> > > generated at ~w ~w\n~p.\n\n",
> > > lib/reltool/src/reltool_server.erl:IoList = io_lib:format("%% config
> > > generated at ~w ~w\n~p.\n\n",
> > > lib/reltool/src/reltool_target.erl:RelIoList = io_lib:format("%% rel
> > > generated at ~w ~w\n~p.\n\n",
> > > lib/reltool/src/reltool_target.erl:ScriptIoList = io_lib:format("%%
> > > script generated at ~w ~w\n~p.\n\n",
> > > lib/reltool/src/reltool_target.erl:AppIoList =
> > > io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
> > > lib/reltool/src/reltool_target.erl:AppIoList =
> > > io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
> > > lib/runtime_tools/src/erts_alloc_config.erl: "generated at
> > > ~w-~2..0w-~2..0w ~2..0w:~2..0w.~2..0w by "
> > > lib/sasl/src/systools_make.erl: io:format(Fd, "%% script generated
> > > at ~w ~w\n~p.\n",
> > > lib/wx/src/gen/gl.erl:%% The program object's information log is updated
> > > and the program is generated at the time
> >
> > If there is no easy work around I suggest simply patching them. Fortunately
> > the Erlang compiler does not change much at this level.
>
> The ideal solution would be to use the value of the environment variable
> SOURCE_DATE_EPOCH if it is set, and else to behave as it does now.
>
> > We can also contact Joe Armstrong, the author of Erlang, to discuss
> > this point. He appears to be approachable. I am sure he is open to
> > the idea of deterministic builds in a deterministic build system ;)
>
> I could go to the Erlang IRC channel or forums (whatever they use) and
> ask for advice. Since you are actually using Erlang, I think you would
> be the better person to contact Joe Armstrong himself, if we decide to
> do that.
I presented the situation on IRC and it was recommended that I start the
discussion on a mailing list.
I think that the erlang-questions list [0] could be a good place to
start.
Pjotr, would you like to start the conversation? I can do it if you are
too busy or something.
[0]
http://www.erlang.org/community
Re: [v3] Erlang
On Mon, Apr 04, 2016 at 10:28:02AM +0200, Pjotr Prins wrote:
> On Sun, Apr 03, 2016 at 11:39:24PM -0400, Leo Famulari wrote:
> > Debian's package exhibits this problem. The timestamps are generated in
> > the following places in the source code. I don't know how to approach
> > this problem.
> >
> > lib/kernel/test/global_SUITE_data/global_trace.erl:io:format("The trace
> > was generated at ~p~n", [EndTime]),
> > lib/reltool/bin/reltool.escript:lists:flatten(io_lib:format("%% ~s
> > generated at ~w ~w\n~p.\n\n",
> > lib/reltool/src/reltool_server.erl:IoList = io_lib:format("%% config
> > generated at ~w ~w\n~p.\n\n",
> > lib/reltool/src/reltool_target.erl:RelIoList = io_lib:format("%% rel
> > generated at ~w ~w\n~p.\n\n",
> > lib/reltool/src/reltool_target.erl:ScriptIoList = io_lib:format("%%
> > script generated at ~w ~w\n~p.\n\n",
> > lib/reltool/src/reltool_target.erl:AppIoList =
> > io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
> > lib/reltool/src/reltool_target.erl:AppIoList =
> > io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
> > lib/runtime_tools/src/erts_alloc_config.erl:"generated at
> > ~w-~2..0w-~2..0w ~2..0w:~2..0w.~2..0w by "
> > lib/sasl/src/systools_make.erl: io:format(Fd, "%% script generated
> > at ~w ~w\n~p.\n",
> > lib/wx/src/gen/gl.erl:%% The program object's information log is updated
> > and the program is generated at the time
>
> If there is no easy work around I suggest simply patching them. Fortunately
> the Erlang compiler does not change much at this level.
The ideal solution would be to use the value of the environment variable
SOURCE_DATE_EPOCH if it is set, and else to behave as it does now.
> We can also contact Joe Armstrong, the author of Erlang, to discuss
> this point. He appears to be approachable. I am sure he is open to
> the idea of deterministic builds in a deterministic build system ;)
I could go to the Erlang IRC channel or forums (whatever they use) and
ask for advice. Since you are actually using Erlang, I think you would
be the better person to contact Joe Armstrong himself, if we decide to
do that.
Re: [v3] Erlang
On Sun, Apr 03, 2016 at 11:39:24PM -0400, Leo Famulari wrote:
> Debian's package exhibits this problem. The timestamps are generated in
> the following places in the source code. I don't know how to approach
> this problem.
>
> lib/kernel/test/global_SUITE_data/global_trace.erl:io:format("The trace
> was generated at ~p~n", [EndTime]),
> lib/reltool/bin/reltool.escript:lists:flatten(io_lib:format("%% ~s
> generated at ~w ~w\n~p.\n\n",
> lib/reltool/src/reltool_server.erl:IoList = io_lib:format("%% config
> generated at ~w ~w\n~p.\n\n",
> lib/reltool/src/reltool_target.erl:RelIoList = io_lib:format("%% rel
> generated at ~w ~w\n~p.\n\n",
> lib/reltool/src/reltool_target.erl:ScriptIoList = io_lib:format("%%
> script generated at ~w ~w\n~p.\n\n",
> lib/reltool/src/reltool_target.erl:AppIoList = io_lib:format("%%
> app generated at ~w ~w\n~p.\n\n",
> lib/reltool/src/reltool_target.erl:AppIoList = io_lib:format("%%
> app generated at ~w ~w\n~p.\n\n",
> lib/runtime_tools/src/erts_alloc_config.erl: "generated at ~w-~2..0w-~2..0w
> ~2..0w:~2..0w.~2..0w by "
> lib/sasl/src/systools_make.erl: io:format(Fd, "%% script generated
> at ~w ~w\n~p.\n",
> lib/wx/src/gen/gl.erl:%% The program object's information log is updated and
> the program is generated at the time
If there is no easy work around I suggest simply patching them. Fortunately
the Erlang compiler does not change much at this level.
We can also contact Joe Armstrong, the author of Erlang, to discuss
this point. He appears to be approachable. I am sure he is open to
the idea of deterministic builds in a deterministic build system ;)
Pj.
Re: [v3] Erlang
On Sun, Apr 03, 2016 at 11:26:48PM +0200, Ludovic Courtès wrote:
> Leo Famulari skribis:
> > All the .beam files have timestamps and build paths embedded [0]. This
> > is a known issue without a clear solution. The build paths don't seem so
> > bad to me, since my understanding is that they are set deterministically
> > by the Guix builder.
> A patch along the lines of the one of the page above should do the work
> for beam files, no? We’d have to change DEB_BUILD_DATE to
> SOURCE_DATE_EPOCH, and remove regexp-matching from there.
There is a more generic patch applied to the actual Debian package [0],
and I've applied it to ours in the attached revision. It will allow us
to build Erlang packages without the timestamp. Unfortunately, it seems
to have no effect when building Erlang itself.
> > There are also some .script files that also embed timestamps in a human
> > readable string. Grepping for the string makes it easy to find the
> > source of this. We could patch the timestamp format string with the
> > value of SOURCE_DATE_EPOCH.
Debian's package exhibits this problem. The timestamps are generated in
the following places in the source code. I don't know how to approach
this problem.
lib/kernel/test/global_SUITE_data/global_trace.erl:io:format("The trace was
generated at ~p~n", [EndTime]),
lib/reltool/bin/reltool.escript:lists:flatten(io_lib:format("%% ~s
generated at ~w ~w\n~p.\n\n",
lib/reltool/src/reltool_server.erl:IoList = io_lib:format("%% config
generated at ~w ~w\n~p.\n\n",
lib/reltool/src/reltool_target.erl:RelIoList = io_lib:format("%% rel
generated at ~w ~w\n~p.\n\n",
lib/reltool/src/reltool_target.erl:ScriptIoList = io_lib:format("%% script
generated at ~w ~w\n~p.\n\n",
lib/reltool/src/reltool_target.erl:AppIoList = io_lib:format("%%
app generated at ~w ~w\n~p.\n\n",
lib/reltool/src/reltool_target.erl:AppIoList = io_lib:format("%%
app generated at ~w ~w\n~p.\n\n",
lib/runtime_tools/src/erts_alloc_config.erl:"generated at ~w-~2..0w-~2..0w
~2..0w:~2..0w.~2..0w by "
lib/sasl/src/systools_make.erl: io:format(Fd, "%% script generated at ~w
~w\n~p.\n",
lib/wx/src/gen/gl.erl:%% The program object's information log is updated and
the program is generated at the time
[0]
https://sources.debian.net/src/erlang/1:18.3-dfsg-1/debian/patches/reproducible-build.patch/
>From 180f142a3f67f80aa029fc34c38f21cd8988eefe Mon Sep 17 00:00:00 2001
Message-Id: <180f142a3f67f80aa029fc34c38f21cd8988eefe.1459741136.git@famulari.name>
From: Steve Sprang
Date: Sat, 6 Feb 2016 12:40:53 -0800
Subject: [v3 1/1] gnu: Add erlang.
* gnu/packages/erlang.scm: New file.
* gnu-system.am (GNU_SYSTEM_MODULES): Add it.
* gnu/packages/patches/erlang-reproducible-build.patch: New file.
* gnu-system.am (dist_PATCH_DATA): Add it.
Co-authored by: Leo Famulari
---
gnu-system.am | 2 +
gnu/packages/erlang.scm| 119 +
.../patches/erlang-reproducible-build.patch| 30 ++
3 files changed, 151 insertions(+)
create mode 100644 gnu/packages/erlang.scm
create mode 100644 gnu/packages/patches/erlang-reproducible-build.patch
diff --git a/gnu-system.am b/gnu-system.am
index d45b3d1..498910f 100644
--- a/gnu-system.am
+++ b/gnu-system.am
@@ -102,6 +102,7 @@ GNU_SYSTEM_MODULES =\
gnu/packages/enchant.scm \
gnu/packages/engineering.scm \
gnu/packages/enlightenment.scm \
+ gnu/packages/erlang.scm \
gnu/packages/fcitx.scm \
gnu/packages/feh.scm \
gnu/packages/figlet.scm \
@@ -459,6 +460,7 @@ dist_patch_DATA = \
gnu/packages/patches/emacs-exec-path.patch \
gnu/packages/patches/emacs-scheme-complete-scheme-r5rs-info.patch \
gnu/packages/patches/emacs-source-date-epoch.patch \
+ gnu/packages/patches/erlang-reproducible-build.patch \
gnu/packages/patches/eudev-rules-directory.patch \
gnu/packages/patches/evilwm-lost-focus-bug.patch \
gnu/packages/patches/expat-CVE-2015-1283.patch \
diff --git a/gnu/packages/erlang.scm b/gnu/packages/erlang.scm
new file mode 100644
index 000..b241961
--- /dev/null
+++ b/gnu/packages/erlang.scm
@@ -0,0 +1,119 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2016 Steve Sprang
+;;; Copyright © 2016 Leo Famulari
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
