Hello Eddy,
On 04/11/25 1:19 pm, Eduard Stefes wrote:
...
# the problem
due to the differences of the hardware implementation, some tests in
the openjdk JREG tests fail(tracked here[1][2])
- java/util/zip/CloseInflaterDeflaterTest.java
fails due to dfltcc's flushing behaviour. The test should check if
the openjdk wrapper around the jni and native library will
successfully detect writes to closed output streams. This was
created because there where bugs with and race conditions in the
write/close/flush.
I had a look at https://bugs.openjdk.org/browse/JDK-8339216. That JBS
issue lacks the actual failure/exception that happens for that test.
Would it be possible to attach the .jtr file from a test execution where
it fails? Depending on what the failure is, I think it could be
addressed in that test.
- tools/jlink/JLinkReproducibleTest.java:
- tools/jar/modularJar/JarToolModuleDescriptorReproducibilityTest.java:
This tests fail due to two calls to dfltcc will not generate the
same compressed data for the same input data. I want to
emphasize that RFC 1950 and 1951 do not guarantee the same output
for two deflate/zlib data streams if feed the same input data.
I will have to read up the RFCs in detail, but these reproducibility
tests have intentionally be written to catch issues like these. We want
the ZIP/JAR artifacts generated for the JDK to be reproducible. We have
been fixing bugs in the JDK, wherever possible, to make these artifacts
reproducible.
## reproducible compression
I don't know how much openjdk relies on reproducible data compression.
I assume it is preferable if its possible to create the same .jar twice
and get the same binary data. The zlib dfltcc implementation could be
controlled via an env variable to disable the hardware compression[5].
Also usually dftcc will only be used for certain compression levels and
this can also be changed via env variables[6].
I am not familiar with that zlib code linked in [5] and [6]. But if you
or anyone else is familiar with it, then instead of this being a
environment variable evaluated at runtime on the host where the JDK
runs, maybe it could be implemented as a build time macro(?) in that
project? That would then allow you to pass a specific value during build
time of that project and control this behaviour? You would then still
have to do the work of "bundling" this zlib with the vendor specific JDK
that you build, but at least for that, the OpenJDK build has the
necessary infrastructure, see "--with-zlib" option in the build docs of
OpenJDK.
# Finally
I hope i did not overwhelm you all with this email. As I don't have a
bugs.openjdk.org account, I thought the mailing list is the best place
to discuss this mater.
These mailing lists are indeed the right place to discuss this level of
details. We are aware that recently there's been more than one variant
of "zlib" being distributed by OS vendors, and we have noticed
differences in the generated output from those implementations. These
differences may (and will very likely) keep increasing over time. I
think we will have to address each of these issues on a case-by-case
basis for now.
-Jaikiran
