I believe Beam already has quite a few users that have forged ahead and used Java 11 with various runners, pre-portability. Mostly I believe the Java 11 limitations are with particular features (Schema codegen) and extensions/IOs/transitive deps.
When it comes to the container, I'd be interested in looking at test coverage. The Flink & Spark portable ValidatesRunner suites use EMBEDDED environment, so they don't exercise the container. The first testing of the Java SDK harness container against the Python-based Universal Local Runner is in pull request now [1]. Are there other test suites to highlight? How hard would it be to run Flink & Spark against the container(s) too? Kenn [1] https://github.com/apache/beam/pull/11792 (despite the name ValidatesRunner, in this case it is validating both the runner and harness, since we don't have a compliance test suite for SDK harnesses) On Fri, Jul 10, 2020 at 7:54 AM Tyson Hamilton <[email protected]> wrote: > What do we consider 'ready'? > > Maybe the only required outstanding bugs are supporting the direct runner > (BEAM-10085), core tests (BEAM-10081), IO tests (BEAM-10084) to start > with? Notably this would exclude failing tests like those for GCP core, > GCPIOs, Dataflow runner, Spark runner, Flink runner, Samza. > > > On Thu, Jul 9, 2020 at 4:44 PM Kyle Weaver <[email protected]> wrote: > >> My main question is, are we confident the Java 11 container is ready to >> release? AFAIK there are still a number of issues blocking full Java 11 >> support (cf [1] <https://issues.apache.org/jira/browse/BEAM-10090>; not >> sure how many of these, if any, affect the SDK harness specifically though.) >> >> For comparison, we recently decided to stop publishing Go SDK containers >> until the Go SDK is considered mature [2]. In the meantime, those who want >> to use the Go SDK can build their own container images from source. >> >> Do we already have a Gradle task to build Java 11 containers? If not, >> this would be a good intermediate step, letting users opt-in to Java >> 11 without us overpromising support. >> > > We do not. From what I can tell, the build.gradele [1] for the Java > container is only for the one version. There is a docker file used for > Jenkins tests. > > [1] > https://github.com/apache/beam/blob/master/sdks/java/container/build.gradle > > >> > When we eventually do the renaming, we can add a note to CHANGES.md [3]. >> >> [1] https://issues.apache.org/jira/browse/BEAM-10090 >> [2] https://issues.apache.org/jira/browse/BEAM-9685 >> [3] https://github.com/apache/beam/blob/master/CHANGES.md >> >> On Thu, Jul 9, 2020 at 3:44 PM Emily Ye <[email protected]> wrote: >> >>> Hi all, >>> >>> I'm getting ramped up on contributing and was looking into adding the >>> Java 11 harness container to releases ( >>> https://issues.apache.org/jira/browse/BEAM-8106) - should I rename the >>> current java container so we have two new images `beam_java8_sdk` and >>> `beam_java11_sdk` or hold off on renaming? If we do rename it, what steps >>> should I take to announce/document the change? >>> >>> Thanks, >>> Emily >>> >>
