I haven't got a chance to deep dive into HADOOP-18324 <https://issues.apache.org/jira/browse/HADOOP-18324> which is claimed to be the reason for these failures. Most probably will try to check next week if it is still there. >From the PR uploaded on HDFS-16853 <https://issues.apache.org/jira/browse/HDFS-16853> it looks like changing or tweaking the cleanup logic itself rather than with playing with tests or MiniDfsCluster, So, the clean up logic has issues but I still need to check what is the impact of that, If I have a service and that terminates in a non test setup, will the restart be an issue like these tests are facing, my initial hunch was No. But I need to carefully check and see what is the impact and what other issues it can cause. the original logic ain't something which can be decoded with just a few seconds of cursory look.
++ @Owen O'Malley <owen.omal...@gmail.com> is the original author of the Hadoop Jira, maybe he can share some pointers about that. -Ayush On Thu, 5 Jan 2023 at 07:04, Chris Nauroth <cnaur...@apache.org> wrote: > Is it a problem limited to MiniDFSCluster, or is it a broader problem of > RPC client resource cleanup? The patch is changing connection close > cleanup, so I assumed the latter. If so, then it could potentially impact > applications integrating with the RPC clients. > > If the problem is limited to MiniDFSCluster and restarts within a single > JVM, then I agree the impact is smaller. Then, we'd want to consider what > downstream projects have tests that do restarts on a MiniDFSCluster. > > Chris Nauroth > > > On Wed, Jan 4, 2023 at 4:22 PM Ayush Saxena <ayush...@gmail.com> wrote: > > > Hmm I'm looking at HADOOP-11867 related stuff but couldn't find it > >> mentioned anywhere in change log or release notes. Are they actually > >> up-to-date? > > > > > > I don't think there is any issue with the ReleaseNotes generation as such > > but with the Resolution type of this ticket, It ain't marked as Fixed but > > Done. The other ticket which is marked Done is also not part of the > release > > notes. [1] > > > > if I'm understanding the potential impact of HDFS-16853 > >> correctly, then it's serious enough to fix before a release. (I could > >> change my vote if someone wants to make a case that it's not that > >> serious.) > >> > > > > Chris, I just had a very quick look at HDFS-16853, I am not sure if this > > can happen outside a MiniDfsCluster setup? Just guessing from the > > description in the ticket. It looked like when we did a restart of the > > Namenode in the MiniDfsCluster, I guess that would be in the same single > > JVM, and that is why a previous blocked thread caused issues with the > > restart. That is what I understood, I haven't checked the code though. > > > > Second, In the same context, Being curious If this lands up being a > > MiniDfsCluster only issue, do we still consider this a release blocker? > Not > > saying in a way it won't be serious, MiniDfsCluster is very widely used > by > > downstream projects and all, so just wanted to know.... > > > > Regarding the Hive & Bouncy castle. The PR seems to have a valid binding > > veto, I am not sure if it will get done any time soon, so if the use case > > is something required, I would suggest handling it at Hadoop itself. It > > seems to be centric to Hive-3.x, I tried compiling the Hive master branch > > with 3.3.5 and it passed. Other than that Hive officially support only > > Hadoop-3.3.1 and that too only in the last 4.x release[2] > > > > > > [1] > > > https://issues.apache.org/jira/browse/HADOOP-11867?jql=project%20%3D%20HADOOP%20AND%20resolution%20%3D%20Done%20AND%20fixVersion%20%3D%203.3.5%20ORDER%20BY%20resolution%20DESC > > [2] https://issues.apache.org/jira/browse/HIVE-24484 > > > > -Ayush > > > > On Tue, 3 Jan 2023 at 23:51, Chris Nauroth <cnaur...@apache.org> wrote: > > > >> -1, because if I'm understanding the potential impact of HDFS-16853 > >> correctly, then it's serious enough to fix before a release. (I could > >> change my vote if someone wants to make a case that it's not that > >> serious.) > >> > >> Otherwise, this RC was looking good: > >> > >> * Verified all checksums. > >> * Verified all signatures. > >> * Built from source, including native code on Linux. > >> * mvn clean package -Pnative -Psrc -Drequire.openssl > -Drequire.snappy > >> -Drequire.zstd -DskipTests > >> * Tests passed. > >> * mvn --fail-never clean test -Pnative -Dparallel-tests > >> -Drequire.snappy -Drequire.zstd -Drequire.openssl > >> -Dsurefire.rerunFailingTestsCount=3 -DtestsThreadCount=8 > >> * Checked dependency tree to make sure we have all of the expected > library > >> updates that are mentioned in the release notes. > >> * mvn -o dependency:tree > >> * Farewell, S3Guard. > >> * Confirmed that hadoop-openstack is now just a stub placeholder > artifact > >> with no code. > >> * For ARM verification: > >> * Ran "file <X>" on all native binaries in the ARM tarball to > confirm > >> they actually came out with ARM as the architecture. > >> * Output of hadoop checknative -a on ARM looks good. > >> * Ran a MapReduce job with the native bzip2 codec for compression, > and > >> it worked fine. > >> * Ran a MapReduce job with YARN configured to use > >> LinuxContainerExecutor and verified launching the containers through > >> container-executor worked. > >> > >> My local setup didn't have the test failures mentioned by Viraj, though > >> there was some flakiness with a few HDFS snapshot tests timing out. > >> > >> Regarding Hive and Bouncy Castle, there is an existing issue and pull > >> request tracking an upgrade attempt. It's looking like some amount of > code > >> changes are required: > >> > >> https://issues.apache.org/jira/browse/HIVE-26648 > >> https://github.com/apache/hive/pull/3744 > >> > >> Chris Nauroth > >> > >> > >> On Tue, Jan 3, 2023 at 8:57 AM Chao Sun <sunc...@apache.org> wrote: > >> > >> > Hmm I'm looking at HADOOP-11867 related stuff but couldn't find it > >> > mentioned anywhere in change log or release notes. Are they actually > >> > up-to-date? > >> > > >> > On Mon, Jan 2, 2023 at 7:48 AM Masatake Iwasaki > >> > <iwasak...@oss.nttdata.com> wrote: > >> > > > >> > > > - building HBase 2.4.13 and Hive 3.1.3 against 3.3.5 failed due > >> to > >> > dependency change. > >> > > > >> > > For HBase, classes under com/sun/jersey/json/* and com/sun/xml/* are > >> not > >> > expected in hbase-shaded-with-hadoop-check-invariants. > >> > > Updating hbase-shaded/pom.xml is expected to be the fix as done in > >> > HBASE-27292. > >> > > > >> > > >> > https://github.com/apache/hbase/commit/00612106b5fa78a0dd198cbcaab610bd8b1be277 > >> > > > >> > > [INFO] --- exec-maven-plugin:1.6.0:exec > >> > (check-jar-contents-for-stuff-with-hadoop) @ > >> > hbase-shaded-with-hadoop-check-invariants --- > >> > > [ERROR] Found artifact with unexpected contents: > >> > > >> > '/home/rocky/srcs/bigtop/build/hbase/rpm/BUILD/hbase-2.4.13/hbase-shaded/hbase-shaded-client/target/hbase-shaded-client-2.4.13.jar' > >> > > Please check the following and either correct the build or > >> update > >> > > the allowed list with reasoning. > >> > > > >> > > com/ > >> > > com/sun/ > >> > > com/sun/jersey/ > >> > > com/sun/jersey/json/ > >> > > ... > >> > > > >> > > > >> > > For Hive, classes belonging to org.bouncycastle:bcprov-jdk15on:1.68 > >> seem > >> > to be problematic. > >> > > Excluding them on hive-jdbc might be the fix. > >> > > > >> > > [ERROR] Failed to execute goal > >> > org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on > >> > project hive-jdbc: Error creating shaded jar: Problem shading JAR > >> > > >> > /home/rocky/.m2/repository/org/bouncycastle/bcprov-jdk15on/1.68/bcprov-jdk15on-1.68.jar > >> > entry > >> > > >> > META-INF/versions/15/org/bouncycastle/jcajce/provider/asymmetric/edec/SignatureSpi$EdDSA.class: > >> > java.lang.IllegalArgumentException: Unsupported class file major > >> version 59 > >> > -> [Help 1] > >> > > ... > >> > > > >> > > > >> > > On 2023/01/02 22:02, Masatake Iwasaki wrote: > >> > > > Thanks for your great effort for the new release, Steve and > Mukund. > >> > > > > >> > > > +1 while it would be nice if we can address missed Javadocs. > >> > > > > >> > > > + verified the signature and checksum. > >> > > > + built from source tarball on Rocky Linux 8 and OpenJDK 8 with > >> native > >> > profile enabled. > >> > > > + launched pseudo distributed cluster including kms and httpfs > >> with > >> > Kerberos and SSL enabled. > >> > > > + created encryption zone, put and read files via httpfs. > >> > > > + ran example MR wordcount over encryption zone. > >> > > > + built rpm packages by Bigtop and ran smoke-tests on Rocky Linux > 8 > >> > (both x86_64 and aarch64). > >> > > > - building HBase 2.4.13 and Hive 3.1.3 against 3.3.5 failed due > >> to > >> > dependency change. > >> > > > # while building HBase 2.4.13 and Hive 3.1.3 against Hadoop > >> 3.3.4 > >> > worked. > >> > > > + skimmed the site contents. > >> > > > - Javadocs are not contained (under r3.3.5/api). > >> > > > # The issue can be reproduced even if I built site docs from > >> the > >> > source. > >> > > > > >> > > > Masatake Iwasaki > >> > > > > >> > > > On 2022/12/22 4:28, Steve Loughran wrote: > >> > > >> Mukund and I have put together a release candidate (RC0) for > Hadoop > >> > 3.3.5. > >> > > >> > >> > > >> Given the time of year it's a bit unrealistic to run a 5 day vote > >> and > >> > > >> expect people to be able to test it thoroughly enough to make > this > >> > the one > >> > > >> we can ship. > >> > > >> > >> > > >> What we would like is for anyone who can to verify the tarballs, > >> and > >> > test > >> > > >> the binaries, especially anyone who can try the arm64 binaries. > >> We've > >> > got > >> > > >> the building of those done and now the build file will > incorporate > >> > them > >> > > >> into the release -but neither of us have actually tested it yet. > >> > Maybe I > >> > > >> should try it on my pi400 over xmas. > >> > > >> > >> > > >> The maven artifacts are up on the apache staging repo -they are > the > >> > ones > >> > > >> from x86 build. Building and testing downstream apps will be > >> > incredibly > >> > > >> helpful. > >> > > >> > >> > > >> The RC is available at: > >> > > >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC0/ > >> > > >> > >> > > >> The git tag is release-3.3.5-RC0, commit 3262495904d > >> > > >> > >> > > >> The maven artifacts are staged at > >> > > >> > >> > > >> > https://repository.apache.org/content/repositories/orgapachehadoop-1365/ > >> > > >> > >> > > >> You can find my public key at: > >> > > >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > >> > > >> > >> > > >> Change log > >> > > >> > >> > > >> > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC0/CHANGELOG.md > >> > > >> > >> > > >> Release notes > >> > > >> > >> > > >> > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC0/RELEASENOTES.md > >> > > >> > >> > > >> This is off branch-3.3 and is the first big release since 3.3.2. > >> > > >> > >> > > >> Key changes include > >> > > >> > >> > > >> * Big update of dependencies to try and keep those reports of > >> > > >> transitive CVEs under control -both genuine and false > positive. > >> > > >> * HDFS RBF enhancements > >> > > >> * Critical fix to ABFS input stream prefetching for correct > >> reading. > >> > > >> * Vectored IO API for all FSDataInputStream implementations, with > >> > > >> high-performance versions for file:// and s3a:// filesystems. > >> > > >> file:// through java native io > >> > > >> s3a:// parallel GET requests. > >> > > >> * This release includes Arm64 binaries. Please can anyone with > >> > > >> compatible systems validate these. > >> > > >> > >> > > >> > >> > > >> Please try the release and vote on it, even though i don't know > >> what > >> > is a > >> > > >> good timeline here...i'm actually going on holiday in early jan. > >> > Mukund is > >> > > >> around and so can drive the process while I'm offline. > >> > > >> > >> > > >> Assuming we do have another iteration, the RC1 will not be before > >> mid > >> > jan > >> > > >> for that reason > >> > > >> > >> > > >> Steve (and mukund) > >> > > >> > >> > > > > >> > > > > >> --------------------------------------------------------------------- > >> > > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > >> > > > For additional commands, e-mail: > common-dev-h...@hadoop.apache.org > >> > > > > >> > > > >> > > > --------------------------------------------------------------------- > >> > > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > >> > > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > >> > > > >> > > >> > --------------------------------------------------------------------- > >> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > >> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >> > > >> > > >> > > >
