Rather than have a user rebuild HBase for themselves and specify hadoop-three.version, why not bump the Hadoop3 version in our POM?
I also don't have an explanation for why the JDK11 + Hadoop3 build succeeds in Jenkins. The branch-2.4 build isn't downloading a flakey tests list, so I don't think these tests are being excluded. In fact, Jenkins records a successful execution of this test on JDK11/Hadoop3. https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/4/testReport/org.apache.hadoop.hbase.regionserver.wal/TestAsyncProtobufLog/ On Wed, Dec 9, 2020 at 4:31 PM 张铎(Duo Zhang) <palomino...@gmail.com> wrote: > OK, I think the problem is a bug in jetty 9.3, the JavaVersion > implementation for 9.3 and 9.4 are completely different, there is no > problem to parse 11.0.9.1 for jetty 9.4 but for jetty 9.3, it can only pass > version with two dots, i.e, 11.0.9. > > I think you could add -Dhadoop-three.version to set the hadoop 3 version > explicitly to a newer version which uses jetty 9.4 to solve the problem, > IIRC all the newest release for each active release line has upgrade to > jetty 9.4 and that's why we need to shade jetty as jetty 9.3 and 9.4 are > incompatible. > > Thanks. > > 张铎(Duo Zhang) <palomino...@gmail.com> 于2020年12月10日周四 上午8:21写道： > > > On nightly jdk11 build the jdk version is > > > > AdoptOpenJDK-11.0.6+10 > > > > Andrew Purtell <apurt...@apache.org> 于2020年12月10日周四 上午7:21写道： > > > >> Let me rephrase. > >> > >> I'm all for testing the optional configurations. I'm less supportive of > >> vetoing releases when an optional configuration has some issue due to a > >> third party component. I would like to see us veto only on the required > >> configurations, and otherwise file JIRAs to fix up the nits on the > >> optional > >> ones. > >> > >> > >> On Wed, Dec 9, 2020 at 3:19 PM Andrew Purtell <apurt...@apache.org> > >> wrote: > >> > >> > > parseJDK9:71, JavaVersion (org.eclipse.jetty.util) > >> > > >> > So unless I am mistaken, some Jetty utility class is not able to parse > >> the > >> > version string of your particular JDK/JRE. > >> > > >> > We can try to downgrade Jetty but I am not sure in general how we are > >> > supposed to take on the risk of third party dependencies doing the > wrong > >> > thing in an optional configuration. I for one do not want to deal > with a > >> > combinatorial explosion of transitive dependencies when releasing. > >> > > >> > > >> > On Wed, Dec 9, 2020 at 2:41 PM Nick Dimiduk <ndimi...@apache.org> > >> wrote: > >> > > >> >> This is coming out of Jetty + Hadoop. This build has a regression in > >> our > >> >> JDK11 support. Or perhaps there's a regression in the upstream Hadoop > >> >> against which JDK11 builds. > >> >> > >> >> I'm afraid I must vote -1 until we can sort out the issue. I'd > >> appreciate > >> >> it if someone else can attempt to repro, help ensure it's not just > me. > >> >> > >> >> Thanks, > >> >> Nick > >> >> > >> >> (Apologies for the crude stack trace; this is copied out of an > attached > >> >> debugger) > >> >> > >> >> parseJDK9:71, JavaVersion (org.eclipse.jetty.util) > >> >> parse:49, JavaVersion (org.eclipse.jetty.util) > >> >> <clinit>:43, JavaVersion (org.eclipse.jetty.util) > >> >> findAndFilterContainerPaths:185, WebInfConfiguration > >> >> (org.eclipse.jetty.webapp) > >> >> preConfigure:155, WebInfConfiguration (org.eclipse.jetty.webapp) > >> >> preConfigure:485, WebAppContext (org.eclipse.jetty.webapp) > >> >> doStart:521, WebAppContext (org.eclipse.jetty.webapp) > >> >> start:68, AbstractLifeCycle (org.eclipse.jetty.util.component) > >> >> start:131, ContainerLifeCycle (org.eclipse.jetty.util.component) > >> >> doStart:113, ContainerLifeCycle (org.eclipse.jetty.util.component) > >> >> doStart:61, AbstractHandler (org.eclipse.jetty.server.handler) > >> >> start:68, AbstractLifeCycle (org.eclipse.jetty.util.component) > >> >> start:131, ContainerLifeCycle (org.eclipse.jetty.util.component) > >> >> start:427, Server (org.eclipse.jetty.server) > >> >> doStart:105, ContainerLifeCycle (org.eclipse.jetty.util.component) > >> >> doStart:61, AbstractHandler (org.eclipse.jetty.server.handler) > >> >> doStart:394, Server (org.eclipse.jetty.server) > >> >> start:68, AbstractLifeCycle (org.eclipse.jetty.util.component) > >> >> start:1140, HttpServer2 (org.apache.hadoop.http) > >> >> start:177, NameNodeHttpServer > (org.apache.hadoop.hdfs.server.namenode) > >> >> startHttpServer:872, NameNode > (org.apache.hadoop.hdfs.server.namenode) > >> >> initialize:694, NameNode (org.apache.hadoop.hdfs.server.namenode) > >> >> <init>:940, NameNode (org.apache.hadoop.hdfs.server.namenode) > >> >> <init>:913, NameNode (org.apache.hadoop.hdfs.server.namenode) > >> >> createNameNode:1646, NameNode > (org.apache.hadoop.hdfs.server.namenode) > >> >> createNameNode:1314, MiniDFSCluster (org.apache.hadoop.hdfs) > >> >> configureNameService:1083, MiniDFSCluster (org.apache.hadoop.hdfs) > >> >> createNameNodesAndSetConf:958, MiniDFSCluster > (org.apache.hadoop.hdfs) > >> >> initMiniDFSCluster:890, MiniDFSCluster (org.apache.hadoop.hdfs) > >> >> <init>:518, MiniDFSCluster (org.apache.hadoop.hdfs) > >> >> build:477, MiniDFSCluster$Builder (org.apache.hadoop.hdfs) > >> >> startMiniDFSCluster:108, AsyncFSTestBase > >> >> (org.apache.hadoop.hbase.io.asyncfs) > >> >> setUp:87, TestFanOutOneBlockAsyncDFSOutput > >> >> (org.apache.hadoop.hbase.io.asyncfs) > >> >> invoke0:-1, NativeMethodAccessorImpl (jdk.internal.reflect) > >> >> invoke:62, NativeMethodAccessorImpl (jdk.internal.reflect) > >> >> invoke:43, DelegatingMethodAccessorImpl (jdk.internal.reflect) > >> >> invoke:566, Method (java.lang.reflect) > >> >> runReflectiveCall:59, FrameworkMethod$1 (org.junit.runners.model) > >> >> run:12, ReflectiveCallable (org.junit.internal.runners.model) > >> >> invokeExplosively:56, FrameworkMethod (org.junit.runners.model) > >> >> invokeMethod:33, RunBefores (org.junit.internal.runners.statements) > >> >> evaluate:24, RunBefores (org.junit.internal.runners.statements) > >> >> evaluate:27, RunAfters (org.junit.internal.runners.statements) > >> >> evaluate:38, SystemExitRule$1 (org.apache.hadoop.hbase) > >> >> call:288, FailOnTimeout$CallableStatement > >> >> (org.junit.internal.runners.statements) > >> >> call:282, FailOnTimeout$CallableStatement > >> >> (org.junit.internal.runners.statements) > >> >> run:264, FutureTask (java.util.concurrent) > >> >> run:834, Thread (java.lang) > >> >> > >> >> On Wed, Dec 9, 2020 at 2:08 PM Nick Dimiduk <ndimi...@apache.org> > >> wrote: > >> >> > >> >> > On Mon, Dec 7, 2020 at 1:51 PM Nick Dimiduk <ndimi...@apache.org> > >> >> wrote: > >> >> > > >> >> >> Has anyone successfully built/run this RC with JDK11 and Hadoop3 > >> >> profile? > >> >> >> I'm seeing test failures locally in the hbase-asyncfs module. > >> >> >> Reproducible with: > >> >> >> > >> >> >> $ > >> >> >> > >> >> > >> > JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home > >> >> >> mvn clean install -Dhadoop.profile=3.0 > >> >> >> > >> >> > >> > -Dtest=org.apache.hadoop.hbase.io.asyncfs.TestFanOutOneBlockAsyncDFSOutput > >> >> >> ... > >> >> >> [INFO] Running > >> >> >> org.apache.hadoop.hbase.io > .asyncfs.TestFanOutOneBlockAsyncDFSOutput > >> >> >> > >> >> >> > >> >> >> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time > >> elapsed: > >> >> >> 1.785 s <<< FAILURE! - in > >> >> >> org.apache.hadoop.hbase.io > .asyncfs.TestFanOutOneBlockAsyncDFSOutput > >> >> >> > >> >> >> [ERROR] > >> >> >> org.apache.hadoop.hbase.io > .asyncfs.TestFanOutOneBlockAsyncDFSOutput > >> >> Time > >> >> >> elapsed: 1.775 s <<< ERROR! > >> >> >> > >> >> >> java.lang.ExceptionInInitializerError > >> >> >> > >> >> >> > >> >> >> at > >> >> >> org.apache.hadoop.hbase.io > >> >> > >> > .asyncfs.TestFanOutOneBlockAsyncDFSOutput.setUp(TestFanOutOneBlockAsyncDFSOutput.java:87) > >> >> >> > >> >> >> Caused by: java.lang.IllegalArgumentException: Invalid Java > version > >> >> >> 11.0.9.1 > >> >> >> > >> >> >> at > >> >> >> org.apache.hadoop.hbase.io > >> >> > >> > .asyncfs.TestFanOutOneBlockAsyncDFSOutput.setUp(TestFanOutOneBlockAsyncDFSOutput.java:87) > >> >> >> > >> >> > > >> >> > This failure is not isolated to macOS. I ran this build on an > Ubuntu > >> VM > >> >> > with the same AdoptOpenJDK 11.0.9.1. Why don't we see this in > >> Jenkins? > >> >> > > >> >> > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time > >> elapsed: > >> >> > 0.011 s <<< FAILURE! - in > >> >> > org.apache.hadoop.hbase.regionserver.wal.TestAsyncProtobufLog > >> >> > > >> >> > [ERROR] > org.apache.hadoop.hbase.regionserver.wal.TestAsyncProtobufLog > >> >> > Time elapsed: 0.003 s <<< ERROR! > >> >> > > >> >> > java.lang.ExceptionInInitializerError > >> >> > > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.regionserver.wal.TestAsyncProtobufLog.setUpBeforeClass(TestAsyncProtobufLog.java:56) > >> >> > > >> >> > Caused by: java.lang.IllegalArgumentException: Invalid Java > >> >> version > >> >> > 11.0.9.1 > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.regionserver.wal.TestAsyncProtobufLog.setUpBeforeClass(TestAsyncProtobufLog.java:56) > >> >> > > >> >> > On Thu, Dec 3, 2020 at 4:05 PM Andrew Purtell <apurt...@apache.org > > > >> >> wrote: > >> >> >> > >> >> >>> Please vote on this Apache hbase release candidate, > hbase-2.4.0RC1 > >> >> >>> > >> >> >>> The VOTE will remain open for at least 72 hours. > >> >> >>> > >> >> >>> [ ] +1 Release this package as Apache hbase 2.4.0 > >> >> >>> [ ] -1 Do not release this package because ... > >> >> >>> > >> >> >>> The tag to be voted on is 2.4.0RC1: > >> >> >>> > >> >> >>> https://github.com/apache/hbase/tree/2.4.0RC1 > >> >> >>> > >> >> >>> The release files, including signatures, digests, as well as > >> >> CHANGES.md > >> >> >>> and RELEASENOTES.md included in this RC can be found at: > >> >> >>> > >> >> >>> https://dist.apache.org/repos/dist/dev/hbase/2.4.0RC1/ > >> >> >>> > >> >> >>> Customarily Maven artifacts would be available in a staging > >> >> repository. > >> >> >>> Unfortunately I was forced to terminate the Maven deploy step > after > >> >> >>> the upload ran for more than four hours and my build equipment > >> >> >>> needed to be relocated, with loss of network connectivity. This > RC > >> has > >> >> >>> been delayed long enough. A temporary Maven repository is not a > >> >> >>> requirement for a vote. I will retry Maven deploy tomorrow. I can > >> >> >>> promise the artifacts for this RC will be staged in Apache Nexus > >> and > >> >> >>> ready for release well ahead of the earliest possible time this > >> vote > >> >> >>> can complete. > >> >> >>> > >> >> >>> Artifacts were signed with the apurt...@apache.org key which can > >> be > >> >> >>> found > >> >> >>> in: > >> >> >>> > >> >> >>> https://dist.apache.org/repos/dist/release/hbase/KEYS > >> >> >>> > >> >> >>> The API compatibility report for this RC can be found at: > >> >> >>> > >> >> >>> > >> >> >>> > >> >> >>> > >> >> > >> > https://dist.apache.org/repos/dist/dev/hbase/2.4.0RC1/api_compare_2.4.0RC1_to_2.3.0.html > >> >> >>> > >> >> >>> The changes are mostly added methods, which conform to the > >> >> compatibility > >> >> >>> guidelines for a new minor release. There is one change to the > >> public > >> >> >>> Region interface that alters the return type of a method. This is > >> >> >>> equivalent to a removal then addition and can be a binary > >> >> compatibility > >> >> >>> problem. However to your RM's eye the change looks intentional > and > >> is > >> >> >>> part of an API improvement project, and a compatibility method is > >> not > >> >> >>> possible here because Java doesn't consider return type when > >> deciding > >> >> if > >> >> >>> one method signature duplicates another. > >> >> >>> > >> >> >>> To learn more about Apache HBase, please see > >> >> >>> > >> >> >>> http://hbase.apache.org/ > >> >> >>> > >> >> >>> Thanks, > >> >> >>> Your HBase Release Manager > >> >> >>> > >> >> >> > >> >> > >> > > >> > > >> > -- > >> > Best regards, > >> > Andrew > >> > > >> > Words like orphans lost among the crosstalk, meaning torn from truth's > >> > decrepit hands > >> > - A23, Crosstalk > >> > > >> > >> > >> -- > >> Best regards, > >> Andrew > >> > >> Words like orphans lost among the crosstalk, meaning torn from truth's > >> decrepit hands > >> - A23, Crosstalk > >> > > >
