Hi Mark,
On 6/05/2020 1:50 am, Mark Kralj-Taylor wrote:
Hi David,
So the issue is that while glibc 2.12 (in OEL6.4) supports
clock_gettime(CLOCK_REALTIME) it requires a runtime dependency on
librt.so, which is an optional runtime dependency, hence the dynamic
lookup you mention. The complexity of dynamic lookup makes this
enhancement unattractive. But when openjdk updates to require glibc
=2.17 it would be trivial, because clock_gettime() was moved into the
core libc.so.
It would be good to be able to remove the need to link librt. In general
we try to avoid needing to track libc/glibc changes in detail, and which
versions on which Linux distributions contain which libc/glibc version -
and even whether they are necessarily equivalent ie. are all glibc 2.17
the same?
With glibc >= 2.17 there would still need to be a call to
clock_gettime() to see if the optional CLOCK_MONOTONIC is supported,
whereas CLOCK_REALTIME is required by the Posix spec and header docs.
I would have hoped/expected that glibc 2.17 either did or did not
support CLOCK_MONOTONIC without any need to do a dynamic runtime check.
:( You seem to be suggesting that two Linux distributions claiming to
have glibc 2.17 may have different support for CLOCK_MONOTONIC - is that
really the case?
---
This leads me to ask a different question:
What is the openjdk process for reviewing and updating its minimum dependencies?
- When are dependencies reviewed?
- What is the mailgroup to look at? (please let me know if there is a
better place to ask this question)
The info I found was last updated for JDK 13:
https://wiki.openjdk.java.net/display/Build/Supported+Build+Platforms
So there are two issues: build platforms and runtime platforms.
The minimum build platform is generally determined by the Build group
(build-...@openjdk.java.net) and is generally driven by the regular
toolchain upgrade process - ie if we want to support gcc X.Y and it no
longer runs on SomeLinux A.B then we have to decide if it is okay drop
that support and bump the minimum SomeLinux version. That may depend on
the other OpenJDK members and what they need to support in their build
farms etc. The wiki above should be tracking that but the build team
(aka Magnus and Erik) are busy folk and it can fall behind. This is
further complicated by the fact that we may use a devkit for an older OS
version, but run it on a later one.
For runtime platforms there is no official OpenJDK minimum supported
platform as far as I am aware. Obviously given the source code +
toolchain there is an implied minimum version, you just won't
necessarily find it clearly documented at any given point in time - nor
know when it suddenly changes. My understanding is that each OpenJDK
distributor determines their own minimum supported OS versions -
potentially maintaining custom patches to achieve that. Any OpenJDK
change that implicitly changes those minimum versions should result in
feedback that then influences whether such a change should be kept or
reverted. And of course there is a difference between an officially
supported OS version and one where it happens to still run okay. For
example, for a long time Windows XP was not officially supported but we
actively tried to not break it.
- Glibc 2.12 was released in 2010 and glibc 2.17 in 2012
- The current openjdk LTS release is Java SE 11 in September 2018,
supported through to October 2024 (extended support to September
2026): https://en.wikipedia.org/wiki/Java_version_history).
- Oracle Enterprise Linux 7.0 released in 2014 included glibc 2.17
These dates suggest an update of openjdk's glibc dependency might be due.
But I guess the driving factor is how the timelines of openjdk LTS
release support fits with those of Oracle Enterprise Linux versions.
And with other OpenJDK distributors/vendors. Here's the information for
the Oracle JDK 14:
https://www.oracle.com/java/technologies/javase/products-doc-jdk14certconfig.html
---
FYI some observations on my experience of building and testing openjdk.
I installed OEL6.4, but found it very tricky to build a `real` openjdk
distribution on a current Ubuntu that would run with all features on
OEL6.4.
Right that is not a supported build platform.
- The base openjdk build was easy to use, but it built a JDK with elf
dependencies based on my OS, so of course would not run on OLE6.
- The openjdk make/devkit infrastructure didn't work for me to make an
OEL6 build, after lots of attempts (including running in chroot or
docker isolation). I'm sure it works reliably, provided you have the
correct OS setup. But its not easy for an outsider to replicate that.
- The AdoptOpenJDK Docker build was super easy to use, and I liked
that it used Docker to use a reproducible and fully scripted OS build
environment. https://github.com/AdoptOpenJDK/openjdk-build
- But AdoptOpenJDK Docker build doesn't support building and using an
OEL6 devkit. AdoptOpenJDK build targets RHEL 7.4+:
https://github.com/AdoptOpenJDK/openjdk-build/wiki/%5BWIP%5D-Minimum-OS-levels
Yes I think 7.4 is current minimum build platform.
openjdk's devkit infra must pre-date the rise of Docker, and fully
reproducible build environments becoming the norm.
Have they become the norm? :) I don't play in that space so have no
idea. But yes this all pre-dates Docker et al by a very long time.
Cheers,
David
-----
Maybe one day there will be time to use Docker or similar to simplify
the openjdk build, and making it easy for anyone to reproduce.
As with using newer glibc features (glibc 2.17 vs 2.12), its likely to
be drastically simpler when openjdk is able to update dependencies to
newer OS versions. Especially if openjdk can be built on its minimum
required per-platform OS version in a (Docker) container.
Mark
On Tue, 14 Apr 2020 at 23:34, David Holmes <david.hol...@oracle.com> wrote:
Hi Mark,
On 15/04/2020 3:09 am, Mark Kralj-Taylor wrote:
David, Daniel,
What is the oldest (lowest) version of Linux and glibc for openjdk 15?
I'm not clear on that.
The availability of clock_gettime(CLOCK_REALTIME) on the oldest
Linux/glibc supported by openjdk 15 is likely to be the deciding
factor on if Hotspot Linux code can call
clock_gettime(CLOCK_REALTIME).
doc/building.md suggests minimum Linux is Oracle Enterprise Linux 6.4
(i.e. RHEL 6.4).
Which I think uses glibc 2.12-1.107 (based on
https://yum.oracle.com/repo/OracleLinux/OL6/4/base/x86_64/index_src.html).
Searching for glibc sources it looks like this supports
clock_gettime(CLOCK_REALTIME).
The wording of Linux docs suggests that clock_gettime(CLOCK_REALTIME)
should be supported if the clock_gettime() API exists. But other clock
sources are not mandatory. So CLOCK_REALTIME should be available even
if CLOCK_MONOTONIC is not.
- See: https://linux.die.net/man/2/clock_gettime.
- Also "POSIX.1-2008 marks gettimeofday() as obsolete, recommending
the use of clock_gettime(2) instead." from:
https://linux.die.net/man/2/gettimeofday
Note that Hotspot os_posix.cpp checks for non-error return from
clock_gettime(CLOCK_MONOTONIC) to guard setting the _clock_gettime
function pointer. Which was why in the patch I called clock_gettime
directly for Linux specific os_linux.cpp (a subset of Posix OS-s).
Also os_linux.cpp has:
#ifndef SUPPORTS_CLOCK_MONOTONIC
#error "Build platform doesn't support clock_gettime and related functionality"
#endif
Which made me wonder if openjdk might require CLOCK_MONOTONIC - which
would mean clock_gettime(CLOCK_REALTIME) is supported.
So we require that the build platform supports CLOCK_MONOTONIC and
clock_gettime, but not that the runtime platform supports
CLOCK_MONOTONIC (without which we don't/didn't need clock_gettime().
So we can switch to using clock_gettime(CLOCK_REALTIME) at build time
with no problem.
We can probably also require clock_gettime(CLOCK_REALTIME) at runtime,
but we need to double-check that. I recall encountering a platform where
clock_gettime was not available, but I can't recall if it was mainstream
or on one of the other OpenJDK projects - nor do I recall exactly how
long ago this was. Keeping the dynamic lookup of clock_gettime would be
a conservative approach here - but we would need to make the distinction
between the ability to use CLOCK_REALTIME and CLOCK_MONOTONIC.
Or someone puts in the time and effort to establish exactly what the
kernel and glibc dependencies are and whether we can just rely on
everything existing on all platforms we care about. I don't have time to
do that nor validate the results if someone else does it.
Cheers,
David
Mark
On Tue, 14 Apr 2020 at 18:04, Mark Kralj-Taylor <kralj.m...@gmail.com> wrote:
Daniel,
Yes System.currentTimeMillis() and Clock.systemUTC() must be
consistent, so should use the same OS time source (as shown by ).
The patch to os_linux.cpp ensures this by calling the same Linux API:
clock_gettime(CLOCK_REALTIME) for both, from:
- os::javaTimeMillis() that backs System.currentTimeMillis()
- os::javaTimeSystemUTC() that backs Clock.systemUTC()
Looking at Linux / glibc source I think that gettimeofday() and
clock_gettime(CLOCK_REALTIME) are supposed to be the same clocksource.
i.e. that given by: cat
/sys/devices/system/clocksource/clocksource0/current_clocksource
Mark
On Tue, 14 Apr 2020 at 13:29, Daniel Fuchs <daniel.fu...@oracle.com> wrote:
Hi,
On 11/04/2020 00:53, David Holmes wrote:
Update:
It's a holiday weekend so I can't dig into this right now but we tried
using a high-precision clock source for systemUTC() in the past but it
didn't work because systemUTC() and currentTimeMillis() have to use
the same time base, and currentTimeMillis() has to use gettimeofday().
I thought this cross-dependency was documented somewhere but can't
find it right now. If gettimeofday and clock_gettime(CLOCK_REALTIME)
actually have the same time characteristics wrt. wall-clock time then
changing both as suggested may indeed work.
Just to emphasize David's comment: System::currentTimeMillis() and
Clock::systemUTC() should use the same time source: if they don't - then
some tests will fail, and because it can be tricky to assert things
in tests, they might (and probably will) fail only intermittently.
I'm probably the culprit here, I added those tests when I upgraded
Clock::systemUTC() to report sub millisecond granularity [1] - as was
available in the underlying clock that System::currentTimeMillis()
already used.
However, I think I would be disturbed if System::currentTimeMillis()
and Clock::systemUTC() were using different clocks and could
report a different notion of time (by drifting away from each other).
best regards,
-- daniel
[1] https://bugs.openjdk.java.net/browse/JDK-8068730
