On Wed, Dec 9, 2020 at 5:30 PM 张铎(Duo Zhang) <palomino...@gmail.com> wrote:
> I've explained to you why the nightly build has no problem... > > It is because we use 11.0.6 for nightly, while your jdk version is > 11.0.9.1, the JavaVersion class in jetty 9.3 can only pass the former > one... > Yes, I see that now (and I have read the offending Jetty code). My apologies, Duo, for speaking past your earlier comment. Nick Dimiduk <ndimi...@apache.org> 于2020年12月10日周四 上午9:10写道： > > > Rather than have a user rebuild HBase for themselves and specify > > hadoop-three.version, why not bump the Hadoop3 version in our POM? > > > > I also don't have an explanation for why the JDK11 + Hadoop3 build > succeeds > > in Jenkins. The branch-2.4 build isn't downloading a flakey tests list, > so > > I don't think these tests are being excluded. In fact, Jenkins records a > > successful execution of this test on JDK11/Hadoop3. > > > > > > > https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/4/testReport/org.apache.hadoop.hbase.regionserver.wal/TestAsyncProtobufLog/ > > > > On Wed, Dec 9, 2020 at 4:31 PM 张铎(Duo Zhang) <palomino...@gmail.com> > > wrote: > > > > > OK, I think the problem is a bug in jetty 9.3, the JavaVersion > > > implementation for 9.3 and 9.4 are completely different, there is no > > > problem to parse 11.0.9.1 for jetty 9.4 but for jetty 9.3, it can only > > pass > > > version with two dots, i.e, 11.0.9. > > > > > > I think you could add -Dhadoop-three.version to set the hadoop 3 > version > > > explicitly to a newer version which uses jetty 9.4 to solve the > problem, > > > IIRC all the newest release for each active release line has upgrade to > > > jetty 9.4 and that's why we need to shade jetty as jetty 9.3 and 9.4 > are > > > incompatible. > > > > > > Thanks. > > > > > > 张铎(Duo Zhang) <palomino...@gmail.com> 于2020年12月10日周四 上午8:21写道： > > > > > > > On nightly jdk11 build the jdk version is > > > > > > > > AdoptOpenJDK-11.0.6+10 > > > > > > > > Andrew Purtell <apurt...@apache.org> 于2020年12月10日周四 上午7:21写道： > > > > > > > >> Let me rephrase. > > > >> > > > >> I'm all for testing the optional configurations. I'm less supportive > > of > > > >> vetoing releases when an optional configuration has some issue due > to > > a > > > >> third party component. I would like to see us veto only on the > > required > > > >> configurations, and otherwise file JIRAs to fix up the nits on the > > > >> optional > > > >> ones. > > > >> > > > >> > > > >> On Wed, Dec 9, 2020 at 3:19 PM Andrew Purtell <apurt...@apache.org> > > > >> wrote: > > > >> > > > >> > > parseJDK9:71, JavaVersion (org.eclipse.jetty.util) > > > >> > > > > >> > So unless I am mistaken, some Jetty utility class is not able to > > parse > > > >> the > > > >> > version string of your particular JDK/JRE. > > > >> > > > > >> > We can try to downgrade Jetty but I am not sure in general how we > > are > > > >> > supposed to take on the risk of third party dependencies doing the > > > wrong > > > >> > thing in an optional configuration. I for one do not want to deal > > > with a > > > >> > combinatorial explosion of transitive dependencies when releasing. > > > >> > > > > >> > > > > >> > On Wed, Dec 9, 2020 at 2:41 PM Nick Dimiduk <ndimi...@apache.org> > > > >> wrote: > > > >> > > > > >> >> This is coming out of Jetty + Hadoop. This build has a regression > > in > > > >> our > > > >> >> JDK11 support. Or perhaps there's a regression in the upstream > > Hadoop > > > >> >> against which JDK11 builds. > > > >> >> > > > >> >> I'm afraid I must vote -1 until we can sort out the issue. I'd > > > >> appreciate > > > >> >> it if someone else can attempt to repro, help ensure it's not > just > > > me. > > > >> >> > > > >> >> Thanks, > > > >> >> Nick > > > >> >> > > > >> >> (Apologies for the crude stack trace; this is copied out of an > > > attached > > > >> >> debugger) > > > >> >> > > > >> >> parseJDK9:71, JavaVersion (org.eclipse.jetty.util) > > > >> >> parse:49, JavaVersion (org.eclipse.jetty.util) > > > >> >> <clinit>:43, JavaVersion (org.eclipse.jetty.util) > > > >> >> findAndFilterContainerPaths:185, WebInfConfiguration > > > >> >> (org.eclipse.jetty.webapp) > > > >> >> preConfigure:155, WebInfConfiguration (org.eclipse.jetty.webapp) > > > >> >> preConfigure:485, WebAppContext (org.eclipse.jetty.webapp) > > > >> >> doStart:521, WebAppContext (org.eclipse.jetty.webapp) > > > >> >> start:68, AbstractLifeCycle (org.eclipse.jetty.util.component) > > > >> >> start:131, ContainerLifeCycle (org.eclipse.jetty.util.component) > > > >> >> doStart:113, ContainerLifeCycle > (org.eclipse.jetty.util.component) > > > >> >> doStart:61, AbstractHandler (org.eclipse.jetty.server.handler) > > > >> >> start:68, AbstractLifeCycle (org.eclipse.jetty.util.component) > > > >> >> start:131, ContainerLifeCycle (org.eclipse.jetty.util.component) > > > >> >> start:427, Server (org.eclipse.jetty.server) > > > >> >> doStart:105, ContainerLifeCycle > (org.eclipse.jetty.util.component) > > > >> >> doStart:61, AbstractHandler (org.eclipse.jetty.server.handler) > > > >> >> doStart:394, Server (org.eclipse.jetty.server) > > > >> >> start:68, AbstractLifeCycle (org.eclipse.jetty.util.component) > > > >> >> start:1140, HttpServer2 (org.apache.hadoop.http) > > > >> >> start:177, NameNodeHttpServer > > > (org.apache.hadoop.hdfs.server.namenode) > > > >> >> startHttpServer:872, NameNode > > > (org.apache.hadoop.hdfs.server.namenode) > > > >> >> initialize:694, NameNode (org.apache.hadoop.hdfs.server.namenode) > > > >> >> <init>:940, NameNode (org.apache.hadoop.hdfs.server.namenode) > > > >> >> <init>:913, NameNode (org.apache.hadoop.hdfs.server.namenode) > > > >> >> createNameNode:1646, NameNode > > > (org.apache.hadoop.hdfs.server.namenode) > > > >> >> createNameNode:1314, MiniDFSCluster (org.apache.hadoop.hdfs) > > > >> >> configureNameService:1083, MiniDFSCluster > (org.apache.hadoop.hdfs) > > > >> >> createNameNodesAndSetConf:958, MiniDFSCluster > > > (org.apache.hadoop.hdfs) > > > >> >> initMiniDFSCluster:890, MiniDFSCluster (org.apache.hadoop.hdfs) > > > >> >> <init>:518, MiniDFSCluster (org.apache.hadoop.hdfs) > > > >> >> build:477, MiniDFSCluster$Builder (org.apache.hadoop.hdfs) > > > >> >> startMiniDFSCluster:108, AsyncFSTestBase > > > >> >> (org.apache.hadoop.hbase.io.asyncfs) > > > >> >> setUp:87, TestFanOutOneBlockAsyncDFSOutput > > > >> >> (org.apache.hadoop.hbase.io.asyncfs) > > > >> >> invoke0:-1, NativeMethodAccessorImpl (jdk.internal.reflect) > > > >> >> invoke:62, NativeMethodAccessorImpl (jdk.internal.reflect) > > > >> >> invoke:43, DelegatingMethodAccessorImpl (jdk.internal.reflect) > > > >> >> invoke:566, Method (java.lang.reflect) > > > >> >> runReflectiveCall:59, FrameworkMethod$1 (org.junit.runners.model) > > > >> >> run:12, ReflectiveCallable (org.junit.internal.runners.model) > > > >> >> invokeExplosively:56, FrameworkMethod (org.junit.runners.model) > > > >> >> invokeMethod:33, RunBefores > (org.junit.internal.runners.statements) > > > >> >> evaluate:24, RunBefores (org.junit.internal.runners.statements) > > > >> >> evaluate:27, RunAfters (org.junit.internal.runners.statements) > > > >> >> evaluate:38, SystemExitRule$1 (org.apache.hadoop.hbase) > > > >> >> call:288, FailOnTimeout$CallableStatement > > > >> >> (org.junit.internal.runners.statements) > > > >> >> call:282, FailOnTimeout$CallableStatement > > > >> >> (org.junit.internal.runners.statements) > > > >> >> run:264, FutureTask (java.util.concurrent) > > > >> >> run:834, Thread (java.lang) > > > >> >> > > > >> >> On Wed, Dec 9, 2020 at 2:08 PM Nick Dimiduk <ndimi...@apache.org > > > > > >> wrote: > > > >> >> > > > >> >> > On Mon, Dec 7, 2020 at 1:51 PM Nick Dimiduk < > ndimi...@apache.org > > > > > > >> >> wrote: > > > >> >> > > > > >> >> >> Has anyone successfully built/run this RC with JDK11 and > Hadoop3 > > > >> >> profile? > > > >> >> >> I'm seeing test failures locally in the hbase-asyncfs module. > > > >> >> >> Reproducible with: > > > >> >> >> > > > >> >> >> $ > > > >> >> >> > > > >> >> > > > >> > > > > > > JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home > > > >> >> >> mvn clean install -Dhadoop.profile=3.0 > > > >> >> >> > > > >> >> > > > >> > > > > > > -Dtest=org.apache.hadoop.hbase.io.asyncfs.TestFanOutOneBlockAsyncDFSOutput > > > >> >> >> ... > > > >> >> >> [INFO] Running > > > >> >> >> org.apache.hadoop.hbase.io > > > .asyncfs.TestFanOutOneBlockAsyncDFSOutput > > > >> >> >> > > > >> >> >> > > > >> >> >> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time > > > >> elapsed: > > > >> >> >> 1.785 s <<< FAILURE! - in > > > >> >> >> org.apache.hadoop.hbase.io > > > .asyncfs.TestFanOutOneBlockAsyncDFSOutput > > > >> >> >> > > > >> >> >> [ERROR] > > > >> >> >> org.apache.hadoop.hbase.io > > > .asyncfs.TestFanOutOneBlockAsyncDFSOutput > > > >> >> Time > > > >> >> >> elapsed: 1.775 s <<< ERROR! > > > >> >> >> > > > >> >> >> java.lang.ExceptionInInitializerError > > > >> >> >> > > > >> >> >> > > > >> >> >> at > > > >> >> >> org.apache.hadoop.hbase.io > > > >> >> > > > >> > > > > > > .asyncfs.TestFanOutOneBlockAsyncDFSOutput.setUp(TestFanOutOneBlockAsyncDFSOutput.java:87) > > > >> >> >> > > > >> >> >> Caused by: java.lang.IllegalArgumentException: Invalid Java > > > version > > > >> >> >> 11.0.9.1 > > > >> >> >> > > > >> >> >> at > > > >> >> >> org.apache.hadoop.hbase.io > > > >> >> > > > >> > > > > > > .asyncfs.TestFanOutOneBlockAsyncDFSOutput.setUp(TestFanOutOneBlockAsyncDFSOutput.java:87) > > > >> >> >> > > > >> >> > > > > >> >> > This failure is not isolated to macOS. I ran this build on an > > > Ubuntu > > > >> VM > > > >> >> > with the same AdoptOpenJDK 11.0.9.1. Why don't we see this in > > > >> Jenkins? > > > >> >> > > > > >> >> > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time > > > >> elapsed: > > > >> >> > 0.011 s <<< FAILURE! - in > > > >> >> > org.apache.hadoop.hbase.regionserver.wal.TestAsyncProtobufLog > > > >> >> > > > > >> >> > [ERROR] > > > org.apache.hadoop.hbase.regionserver.wal.TestAsyncProtobufLog > > > >> >> > Time elapsed: 0.003 s <<< ERROR! > > > >> >> > > > > >> >> > java.lang.ExceptionInInitializerError > > > >> >> > > > > >> >> > at > > > >> >> > > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.regionserver.wal.TestAsyncProtobufLog.setUpBeforeClass(TestAsyncProtobufLog.java:56) > > > >> >> > > > > >> >> > Caused by: java.lang.IllegalArgumentException: Invalid > > Java > > > >> >> version > > > >> >> > 11.0.9.1 > > > >> >> > at > > > >> >> > > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.regionserver.wal.TestAsyncProtobufLog.setUpBeforeClass(TestAsyncProtobufLog.java:56) > > > >> >> > > > > >> >> > On Thu, Dec 3, 2020 at 4:05 PM Andrew Purtell < > > apurt...@apache.org > > > > > > > >> >> wrote: > > > >> >> >> > > > >> >> >>> Please vote on this Apache hbase release candidate, > > > hbase-2.4.0RC1 > > > >> >> >>> > > > >> >> >>> The VOTE will remain open for at least 72 hours. > > > >> >> >>> > > > >> >> >>> [ ] +1 Release this package as Apache hbase 2.4.0 > > > >> >> >>> [ ] -1 Do not release this package because ... > > > >> >> >>> > > > >> >> >>> The tag to be voted on is 2.4.0RC1: > > > >> >> >>> > > > >> >> >>> https://github.com/apache/hbase/tree/2.4.0RC1 > > > >> >> >>> > > > >> >> >>> The release files, including signatures, digests, as well as > > > >> >> CHANGES.md > > > >> >> >>> and RELEASENOTES.md included in this RC can be found at: > > > >> >> >>> > > > >> >> >>> https://dist.apache.org/repos/dist/dev/hbase/2.4.0RC1/ > > > >> >> >>> > > > >> >> >>> Customarily Maven artifacts would be available in a staging > > > >> >> repository. > > > >> >> >>> Unfortunately I was forced to terminate the Maven deploy step > > > after > > > >> >> >>> the upload ran for more than four hours and my build > equipment > > > >> >> >>> needed to be relocated, with loss of network connectivity. > This > > > RC > > > >> has > > > >> >> >>> been delayed long enough. A temporary Maven repository is > not a > > > >> >> >>> requirement for a vote. I will retry Maven deploy tomorrow. I > > can > > > >> >> >>> promise the artifacts for this RC will be staged in Apache > > Nexus > > > >> and > > > >> >> >>> ready for release well ahead of the earliest possible time > this > > > >> vote > > > >> >> >>> can complete. > > > >> >> >>> > > > >> >> >>> Artifacts were signed with the apurt...@apache.org key which > > can > > > >> be > > > >> >> >>> found > > > >> >> >>> in: > > > >> >> >>> > > > >> >> >>> https://dist.apache.org/repos/dist/release/hbase/KEYS > > > >> >> >>> > > > >> >> >>> The API compatibility report for this RC can be found at: > > > >> >> >>> > > > >> >> >>> > > > >> >> >>> > > > >> >> >>> > > > >> >> > > > >> > > > > > > https://dist.apache.org/repos/dist/dev/hbase/2.4.0RC1/api_compare_2.4.0RC1_to_2.3.0.html > > > >> >> >>> > > > >> >> >>> The changes are mostly added methods, which conform to the > > > >> >> compatibility > > > >> >> >>> guidelines for a new minor release. There is one change to > the > > > >> public > > > >> >> >>> Region interface that alters the return type of a method. > This > > is > > > >> >> >>> equivalent to a removal then addition and can be a binary > > > >> >> compatibility > > > >> >> >>> problem. However to your RM's eye the change looks > intentional > > > and > > > >> is > > > >> >> >>> part of an API improvement project, and a compatibility > method > > is > > > >> not > > > >> >> >>> possible here because Java doesn't consider return type when > > > >> deciding > > > >> >> if > > > >> >> >>> one method signature duplicates another. > > > >> >> >>> > > > >> >> >>> To learn more about Apache HBase, please see > > > >> >> >>> > > > >> >> >>> http://hbase.apache.org/ > > > >> >> >>> > > > >> >> >>> Thanks, > > > >> >> >>> Your HBase Release Manager > > > >> >> >>> > > > >> >> >> > > > >> >> > > > >> > > > > >> > > > > >> > -- > > > >> > Best regards, > > > >> > Andrew > > > >> > > > > >> > Words like orphans lost among the crosstalk, meaning torn from > > truth's > > > >> > decrepit hands > > > >> > - A23, Crosstalk > > > >> > > > > >> > > > >> > > > >> -- > > > >> Best regards, > > > >> Andrew > > > >> > > > >> Words like orphans lost among the crosstalk, meaning torn from > truth's > > > >> decrepit hands > > > >> - A23, Crosstalk > > > >> > > > > > > > > > >
