ok, Reproducible Builds are not so easy to get: each plugin that you use can
cause an issue
I really recommend you diffoscope to investigate differences, it really helps a
lot by easily giving you precise differences
Good luck for the investigations :)
Regards,
Hervé
----- Mail original -----
De: "Christofer Dutz" <christofer.d...@c-ware.de>
À: "Maven Developers List" <dev@maven.apache.org>
Envoyé: Dimanche 3 Novembre 2019 20:16:42
Objet: Re: last review of Reproducible Builds proposal
Hi Herve,
unfortunately I now have implemented some tooling to compare the stuff produced
by the updated reproducible builds.
And it does show quite a number of non binary equal files.
Will investigate what's the difference.
Chris
Am 03.11.19, 11:08 schrieb "Hervé BOUTEMY" <herve.bout...@free.fr>:
great feedback, thank you: this proves the initial set of only 3 plugins is
a
good basis.
For checking the many output artifacts from a build, future buildinfo file
[1]
should help a lot. I still have many usability concerns for Maven
multi-module
builds (starting with: should we generate only the root buildinfo or one
buildinfo per Maven module?)
Let's keep the ball rolling: today, it's plugins release day!!!
Regards,
Hervé
[1] https://reproducible-builds.org/docs/jvm/
Le vendredi 1 novembre 2019, 13:34:32 CET Christofer Dutz a écrit :
> Hi,
>
> so I just updated the versions of the 3 plugins and gave the Apache PLC4X
> build a little spin ...
> https://github.com/apache/plc4x/tree/feature/reproducible-builds
>
> Did two executions of this:
> mvn -U -P
>
with-boost,with-java,with-dotnet,with-cpp,with-python,with-proxies,with-san
> dbox,with
>
-DaltDeploymentRepository=snapshot-repo::default::file:./local-snapshots-di
> r clean deploy
(but with a different second local repo location)
>
> Then I used "diff" and "cmp" to compare individual files and it didn't
show
> differences with the artifacts I chose ...
now I guess I'll have to whip
> up some little bash script to sort of compare the artifacts from one
> directory with the other (Unfortunately the SNAPSHOT timestamps are a
> little annoying :-/
> We do have some C++, C# and Python modules ... but I wouldn't expect the
C++
> and C# to be reproducible.
> Chris
>
>
> Am 01.11.19, 12:04 schrieb "Hervé BOUTEMY" <herve.bout...@free.fr>:
>
> Le vendredi 1 novembre 2019, 09:40:40 CET Christofer Dutz a écrit :
>
> > Hi Herve,
> >
> > thanks for that will try it asap and report any findings back.
> >
> > But good to know that there is a difference between JDK major
versions
> > and
> > OSes ... so it would probably be best to stage releases on Linux
with
> > an
> > OpenJDK of the minimum supported version?
> > Just thinking how to make it
> > possible to verify without having to buy Mac or Windows licenses ...
> > guess
> > on every machine you could whip up a Ubuntu VM for verification.
Just
> > thinking about it ... perhaps it would be best to create a Docker
> > image for
doing the reproducible stuff ...
>
> just to be clear: the difference is on newline only, then Windows vs
> anything
else. You get the same result on Linux, MacOS, FreeBSD, or any
> other Unix.
> And if I want to be complete, if you get a source tarball from
Windows,
>
extract it to a Linux/MacOS/... box and build with "mvn -
> Dline.separator='\r\n'", in general, you get the same result as a
build
> under
Windows: I tested multiple times, it worked, but we'll see if it
> works always or just "in general"
>
>
> > Are there any plans on creating a plugin to allow verification?
> >
> > Sort of something like this:
> > "mvn package release:verify-reproduicble
> > -DstagingRepoUrl=a.b.c.de/repo/blahblahblah"
> > (Which doesn't deploy the
> > artifacts, but instead download them and do a binary comparison)
>
> yes, but for now it was completely another form: this is the
"Buildinfo
> file"
proposal https://reproducible-builds.org/docs/jvm/
> that I stopped to work on 1 year ago given I had no chance to get the
> same
output: it's now a good time to restart working on it as next steps
>
>
> > Also it could be great if the release-plugin could automatically set
> > the
> > property:
> > a) if it finds the "project.build.outputTimestamp" set to some
> > placeholder value b) if some switch tells it to prepare a
> > reproducible
> > build by using some sort of "switch" parameter
> > Guess that would sort of close the loop to get the biggest benefit
out
> > of
> > the reproducible builds.
>
> +1
> issue has been created
> https://issues.apache.org/jira/browse/MRELEASE-1029
But I didn't work on
> it yet, help welcome
>
>
> > I would be happy to help as I think this is one
> > of the greatest new features. (Ok ... perhaps besides the
> > sound-output-extension I learned about yesterday ;-) )
>
> +1
> the current step is important, but it's only the beginning of the
story
> from
an effective usage perspective
>
> Regards,
>
> Hervé
>
>
> >
> > Chris
> >
> >
> > Am 01.11.19, 09:24 schrieb "Hervé BOUTEMY" <herve.bout...@free.fr>:
> >
> >
> > Le jeudi 31 octobre 2019, 17:26:52 CET Christofer Dutz a écrit :
> >
> >
> >
> > > Hi all,
> > >
> > > as I can see you're voting on releasing the reproducible build
> > > extended
> > > plugin versions.
> >
> >
> >
> > Is there any documentation on how to use this new
> >
> >
> >
> > > feature?
> > >
> > > I had a look at the confluence page, but that seemed like a
> > > brainstorming
> > > session.
> >
> >
> >
> > ok, the Wiki page [1] started as a brainstorming session, was
> > updated to
> >
> > a proposal (the "Output Archive Entries Timestamp" parapgraph).
>
> And now I
>
> > probably should order paragraph a little bit, and add a "Making your
> > build
> > reproducible" section for end uses to have a quick explanation.
> >
> > I'll write the explanation here as a first try before working on
> > the
> >
> > Wiki:
>
>
>
> > 1. upgrade your plugins to reproducible version, particularly
> >
> > mpaven-source-plugin, maven-jar-plugin and maven-assembly-plugin to
> > vesion
> > 3.2.0
>
> 2. add project.build.outputTimestamp property with the timestamp
>
> > value that will be used in zip/jar/tar archives: <properties>
> >
> >
> >
> >
<project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.out
> > putTi
mestamp>
>
> </properties>
>
> >
> > Notice:
> > - there is no Maven version prerequisite, everything happens at
> > plugins
> >
> > level
>
> - Reproducible Builds require to have no version ranges in
>
> > dependencies, generally gives different result on Unixes vs Windows,
> > and
> > generally depend on the major version of JDK used to compile (see
"Out
> > of
> > Scope" paragraph)
> >
> > You have the basis configured, the output should be reproducible
> > now.
> > If something is still not reproducible, use diffoscope to find
> > the
> >
> > unstable output, find the plugin that generated this output and
check
> > if
> > there is a reproducible version available: if not, please open an
> > issue to
> > help plugin maintainers improving Reproducible Builds support at
> > every
> > plugin level.
>
>
>
> > [1]
> >
> >
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682
> > 318
>
>
>
> >
> >
> >
> >
> >
> >
> > > I would love to add this to the PLC4X build asap.
> >
> >
> >
> > I'd love to have feedback
> > Don't hesitate to ping me.
> >
> >
> >
> > >
> > > So I would like to test the release-candidates and vote too.
> >
> >
> >
> > I would love to have many tester and votes :)
> >
> >
> >
> >
> > >
> > > Chris
> > >
> > >
> > >
> > > Am 16.10.19, 14:42 schrieb "Hervé BOUTEMY"
> > > <herve.bout...@free.fr>:
> > >
> > >
> > >
> > > Le mercredi 16 octobre 2019, 13:40:48 CEST Andreas Sewe a
> > > écrit :
> > >
> > >
> > >
> > >
> > >
> > > > Emmanuel Bourg wrote:
> > > >
> > > >
> > > >
> > > > > Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit :
> > > > >
> > > > >
> > > > >
> > > > >> last question: now that we seem to fully understand
> > > > >> each
> > > > >> other,
> > > > >> does it
> > > > >> mean that you don't need any more "seconds since the
> > > > >> epoch"
> > > > >> format
> > > > >> support for the property?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > If Maven supports the SOURCE_DATE_EPOCH environment
> > > > > variable
> > > > > I
> > > > > don't
> > > > > think that's necessary, otherwise it would be nice to
be
> > > > > able
> > > > > to
> > > > > invoke
> > > > >
> > > > > Maven with:
> > > > >
> > > > >
> > > > >
> > > > > mvn package
> > > > > -Dproject.build.outputTimestamp=$SOURCE_DATE_EPOCH
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > and this means supporting a timestamp formatted as
> > > > > seconds
> > > > > since
> > > > > the
> > > > > epoch.
> > > >
> > > >
> > > >
> > > >
> > > > +1
> > > >
> > > > The above would be a nice, simple way of bridging the
gap
> > > > between
> > > > SOURCE_DATE_EPOCH and project.build.outputTimestamp.
> > >
> > >
> > >
> > >
> > >
> > > told like that, ok, why not
> > >
> > >
> > >
> > >
> > >
> > >
> > > >
> > > > If it is not too much trouble to implement the "\d+ ->
> > > > seconds
> > > > since
> > > > epoch" heuristic, them I would love to see it included.
> > >
> > >
> > >
> > >
> > >
> > > ok, I'll do and prepare the release
> > >
> > > Regards,
> > >
> > > Hervé
> > >
> > >
> > >
> > >
> > >
> > >
> > > >
> > > > Best wishes,
> > > >
> > > > Andreas
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
------------------------------------------------------------
> > > ------
> > > ---
> > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > For additional commands, e-mail: dev-h...@maven.apache.org
> > >
> > >
> > >
> > >
> > >
> > >
> > >
----------------------------------------------------------------
> > > -----
> > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > For additional commands, e-mail: dev-h...@maven.apache.org
> > >
> >
> >
> >
> >
> >
> >
> >
> >
> >
------------------------------------------------------------------
> > ---
> > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > For additional commands, e-mail: dev-h...@maven.apache.org
> >
> >
> >
> >
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
