Re: RFR: 8238650: Allow to override buildDate with SOURCE_DATE_EPOCH
On Sat, 27 Mar 2021 17:26:21 GMT, John Neffenger wrote: > The API states, "There is no guarantee that the name strings in the resulting > array will appear in any specific order; they are not, in particular, > guaranteed to appear in alphabetical order." That API was defined in December > 1998 with Java version 1.2, so there's no changing it now. Sorted output still complies with that API, though. If changing it at the Java level is too impactful, there could still be a sort-wrapper in Gradle or some other layer. - PR: https://git.openjdk.java.net/jfx/pull/99
Re: RFR: 8238650: Allow to override buildDate with SOURCE_DATE_EPOCH
On Fri, 26 Mar 2021 19:19:10 GMT, John Neffenger wrote: > objectDir.listFiles().sort() Now the interesting question is, if it is possible to change `listFiles` to always return deterministic output or if we have to patch an infinite number of callers instead. For those cases where people do not care about order, one could add `listUnsortedFiles` with the old behaviour. There is a small performance impact, but usually that is rather small (below 4%) compared to interacting with the filesystem and processing the files returned. - PR: https://git.openjdk.java.net/jfx/pull/99
Re: RFR: 8238650: Allow to override buildDate with SOURCE_DATE_EPOCH
On Mon, 8 Mar 2021 17:32:20 GMT, John Neffenger wrote: >> Long term, yes. >> >> I am currently low on time and my Java-foo is not that great, so I was >> hoping that someone would pick it up. > >> I am currently low on time and my Java-foo is not that great, so I was >> hoping that someone would pick it up. > > @bmwiedemann Thank you for getting this important change started. If you can > finish this pull request, I'll follow up with some additional changes to > support reproducible builds. Otherwise, if you're still short on time, I can > pick up where you left off. So far I managed to create reproducible Java > artifacts — all but the native shared libraries (and the > `javafx.graphics.jmod` file that includes them). @jgneff feel free to pick it up. When linking shared libraries, ensure that .o files are listed in deterministic order - i.e. `sort(glob(*.c))` and check if individual .c or .o files vary between builds. - PR: https://git.openjdk.java.net/jfx/pull/99
Re: RFR: 8264449: Enable reproducible builds with SOURCE_DATE_EPOCH
On Thu, 1 Apr 2021 17:50:47 GMT, Kevin Rushforth wrote: >> It would be better to add the flag in `libxml/CMakeLists.txt` or in >> `build.gradle` via `cmakeArgs`. > > Ideally the latter, if feasible, so that the logic to handle > `SOURCE_DATE_EPOCH` is more localized. IMHO, it would make sense to merge any clear improvement of the status-quo and debug+fix more in a later PR. "Perfect is the enemy of good." Regarding Webkit, I know of https://bugs.webkit.org/show_bug.cgi?id=174540 https://bugs.webkit.org/show_bug.cgi?id=188738 https://build.opensuse.org/request/show/667729 but your version is probably not *that* old. - PR: https://git.openjdk.java.net/jfx/pull/446
Re: RFR: 8238650: Allow to override buildDate with SOURCE_DATE_EPOCH
On Tue, 14 Apr 2020 11:48:55 GMT, Kevin Rushforth wrote: >> Finally getting to this (sorry for the delay). This seems OK to me, as it is >> not an intrusive change. I do not think >> that actually setting this property for production builds is a good idea, so >> I recommend against that (ultimately that >> will be up to Gluon or whoever produces builds). I'll test it once you >> change it from an env variable to a gradle >> property. > > @bmwiedemann are you planning to pursue this PR? Long term, yes. I am currently low on time and my Java-foo is not that great, so I was hoping that someone would pick it up. - PR: https://git.openjdk.java.net/jfx/pull/99
Re: RFR: 8238650: Allow to override buildDate with SOURCE_DATE_EPOCH
On Sat, 8 Feb 2020 11:14:29 GMT, Bernhard M. Wiedemann wrote: >> As an optional override, I am OK with the concept of having a way for the >> build to be reproducible. >> >> FWIW, I have scripts that will unpack the modular jar files and diff each >> class as well as doing the same for a src.zip, and it's pretty easy to tell >> if only VersionInfo (which is the class that records the time stamps) has >> changed. >> >> I note that in practice, this is useful for a certain class of builds (e.g., >> CI or nightly test builds), but each released build is necessarily going to >> be different because you want a unique time stamp and build number >> associated with it. >> >> I will review this (probably some time next week) and would like @johanvos >> to review as well. > >> FWIW, I have scripts that will unpack the modular jar files and diff each >> class > > I agree that such specialized diff tools have some value, yet, there are also > some limitations and downsides to them. E.g. you cannot simply tell another > party what the expected sha256sum of a build result is. > > https://www.suse.com/c/?p=42014 also has a section on problems with "the use > of specialized comparison tools like [openSUSE's] ‘build-compare‘ " > > I probably should write an FAQ entry about that topic... > >> each released build is necessarily going to be different because you want a >> unique time stamp and build number associated with it. > > For release builds, it is important that other people can take the released > sources and reproduce the same original binaries with the same release number > (and ideally same timestamps) to easily verify that the build was clean (not > corrupted by bad CPUs/RAM/HDDs or someone messing with the build machine). > I heard, some people even use that to save network bandwidth: add a small > patch locally+remotely, build it locally, tell the world the new build hash, > but have others upload their binaries with the right hash. Hi, did you find time to review this? - PR: https://git.openjdk.java.net/jfx/pull/99
Re: RFR: 8238650: Allow to override buildDate with SOURCE_DATE_EPOCH
On Fri, 7 Feb 2020 20:04:21 GMT, Kevin Rushforth wrote: >>> Do I have to open a separate bug for each of them? >> >> Every fix (meaning each pull request) needs a unique bug ID. > > As an optional override, I am OK with the concept of having a way for the > build to be reproducible. > > FWIW, I have scripts that will unpack the modular jar files and diff each > class as well as doing the same for a src.zip, and it's pretty easy to tell > if only VersionInfo (which is the class that records the time stamps) has > changed. > > I note that in practice, this is useful for a certain class of builds (e.g., > CI or nightly test builds), but each released build is necessarily going to > be different because you want a unique time stamp and build number associated > with it. > > I will review this (probably some time next week) and would like @johanvos to > review as well. > FWIW, I have scripts that will unpack the modular jar files and diff each > class I agree that such specialized diff tools have some value, yet, there are also some limitations and downsides to them. E.g. you cannot simply tell another party what the expected sha256sum of a build result is. https://www.suse.com/c/?p=42014 also has a section on problems with "the use of specialized comparison tools like [openSUSE's] ‘build-compare‘ " I probably should write an FAQ entry about that topic... > each released build is necessarily going to be different because you want a > unique time stamp and build number associated with it. For release builds, it is important that other people can take the released sources and reproduce the same original binaries with the same release number (and ideally same timestamps) to easily verify that the build was clean (not corrupted by bad CPUs/RAM/HDDs or someone messing with the build machine). I heard, some people even use that to save network bandwidth: add a small patch locally+remotely, build it locally, tell the world the new build hash, but have others upload their binaries with the right hash. - PR: https://git.openjdk.java.net/jfx/pull/99
Re: RFR: 8238650: Allow to override buildDate with SOURCE_DATE_EPOCH
On Fri, 7 Feb 2020 13:43:50 GMT, Kevin Rushforth wrote: >> So the contributor agreement is done. How can I find the bug? > > The OCA needs to be validated and recorded. As for the bug, it has been > transferred to the JDK project as > [JDK-8238650](https://bugs.openjdk.java.net/browse/JDK-8238650). > The OCA needs to be validated and recorded. https://www.oracle.com/technetwork/community/oca-486395.html#w now has `Bernhard Wiedemann - OpenJDK - bmwiedemann` > As for the bug, it has been transferred to the JDK project as > [JDK-8238650](https://bugs.openjdk.java.net/browse/JDK-8238650). Thanks, somehow I had not got an email notification for that. Also worth noting that this PR only fixes one of the sources of non-determinism in openjfx. Do I have to open a separate bug for each of them? - PR: https://git.openjdk.java.net/jfx/pull/99
RFR: 8238650: Allow to override buildDate with SOURCE_DATE_EPOCH
Allow to override buildDate with `SOURCE_DATE_EPOCH` in order to make builds reproducible. See https://reproducible-builds.org/ for why this is good and https://reproducible-builds.org/specs/source-date-epoch/ for the definition of this variable. This PR was done while working on reproducible builds for openSUSE. - Commits: - e7f71fb9: 8238650: Allow to override buildDate with SOURCE_DATE_EPOCH Changes: https://git.openjdk.java.net/jfx/pull/99/files Webrev: https://webrevs.openjdk.java.net/jfx/99/webrev.00 Issue: https://bugs.openjdk.java.net/browse/JDK-8238650 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jfx/pull/99.diff Fetch: git fetch https://git.openjdk.java.net/jfx pull/99/head:pull/99 PR: https://git.openjdk.java.net/jfx/pull/99
Re: RFR: 8238650: Allow to override buildDate with SOURCE_DATE_EPOCH
On Wed, 29 Jan 2020 12:48:35 GMT, Kevin Rushforth wrote: >> /signed >> >> I filed one bug and got review ID : 9063488 > >> I filed one bug and got review ID : 9063488 > > OK. So the contributor agreement is done. How can I find the bug? - PR: https://git.openjdk.java.net/jfx/pull/99
Re: RFR: 8238650: Allow to override buildDate with SOURCE_DATE_EPOCH
On Wed, 29 Jan 2020 12:33:00 GMT, Kevin Rushforth wrote: >> Allow to override buildDate with `SOURCE_DATE_EPOCH` >> in order to make builds reproducible. >> See https://reproducible-builds.org/ for why this is good >> and https://reproducible-builds.org/specs/source-date-epoch/ >> for the definition of this variable. >> >> This PR was done while working on reproducible builds for openSUSE. > > In addition to a signed OCA, we will need a bug filed to track this. Please > read the [CONTRIBUTING > guidelines](https://github.com/openjdk/jfx/blob/master/CONTRIBUTING.md). /signed I filed one bug and got review ID : 9063488 - PR: https://git.openjdk.java.net/jfx/pull/99
