Re: [VOTE] Release Apache Hadoop 3.3.4
thanks. the release is up for download as is the site; i will do the announcement now. also automated a lot more of the work of doing the release, inc testing across projects https://github.com/steveloughran/validate-hadoop-client-artifacts On Thu, 4 Aug 2022 at 19:08, Stack wrote: > +1 (Sorry, took me a while) > > Ran: ./dev-support/hadoop-vote.sh --source > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/ > > * Signature: ok > > * Checksum : failed > > * Rat check (17.0.1): ok > > - mvn clean apache-rat:check > > * Built from source (17.0.1): ok > > - mvn clean install -DskipTests > > * Built tar from source (17.0.1): ok > > - mvn clean package -Pdist -DskipTests -Dtar > -Dmaven.javadoc.skip=true > > Took a look at website. Home page says stuff like, “ARM Support: This is > the first release to support ARM architectures.“, which I don’t think is > true of 3.3.4 but otherwise, looks fine. > > Only played with HDFS. UIs looked right. > > Deployed to ten node arm64 cluster. Ran the hbase verification job on top > of it and all passed. Did some kills, stuff came back. > > I didn't spend time on unit tests but one set passed on a local rig here: > > [image: image.png] > Stack > > On Fri, Jul 29, 2022 at 11:48 AM Steve Loughran > wrote: > >> I have put together a release candidate (RC1) for Hadoop 3.3.4 >> >> The RC is available at: >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/ >> >> The git tag is release-3.3.4-RC1, commit a585a73c3e0 >> >> The maven artifacts are staged at >> https://repository.apache.org/content/repositories/orgapachehadoop-1358/ >> >> You can find my public key at: >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS >> >> Change log >> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/CHANGELOG.md >> >> Release notes >> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/RELEASENOTES.md >> >> There's a very small number of changes, primarily critical code/packaging >> issues and security fixes. >> >> See the release notes for details. >> >> Please try the release and vote. The vote will run for 5 days. >> >> steve >> >
Re: [VOTE] Release Apache Hadoop 3.3.4
+1 (Sorry, took me a while) Ran: ./dev-support/hadoop-vote.sh --source https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/ * Signature: ok * Checksum : failed * Rat check (17.0.1): ok - mvn clean apache-rat:check * Built from source (17.0.1): ok - mvn clean install -DskipTests * Built tar from source (17.0.1): ok - mvn clean package -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true Took a look at website. Home page says stuff like, “ARM Support: This is the first release to support ARM architectures.“, which I don’t think is true of 3.3.4 but otherwise, looks fine. Only played with HDFS. UIs looked right. Deployed to ten node arm64 cluster. Ran the hbase verification job on top of it and all passed. Did some kills, stuff came back. I didn't spend time on unit tests but one set passed on a local rig here: [image: image.png] Stack On Fri, Jul 29, 2022 at 11:48 AM Steve Loughran wrote: > I have put together a release candidate (RC1) for Hadoop 3.3.4 > > The RC is available at: > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/ > > The git tag is release-3.3.4-RC1, commit a585a73c3e0 > > The maven artifacts are staged at > https://repository.apache.org/content/repositories/orgapachehadoop-1358/ > > You can find my public key at: > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > Change log > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/CHANGELOG.md > > Release notes > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/RELEASENOTES.md > > There's a very small number of changes, primarily critical code/packaging > issues and security fixes. > > See the release notes for details. > > Please try the release and vote. The vote will run for 5 days. > > steve >
Re: [VOTE] Release Apache Hadoop 3.3.4
I'm still trying to get a successful run from Spark side here: https://github.com/apache/spark/pull/37281. The last CI run failed, but most likely due to some issue not related to Hadoop. On Thu, Aug 4, 2022 at 10:27 AM Steve Loughran wrote: > > The vote passed with the following result. > > Binding PMC members: > > +1 Chris Nauroth > +1 Steve Loughran > +1 Masatake Iwasaki > > Non binding votes: > > +1 Ashutosh Gupta > > Cheng Pan was worried about the new transient kotlin dependency. They are > working on a PR there which we can target for the successor to this release > https://issues.apache.org/jira/browse/HDFS-16714 > > I'm going to publish the artifacts, site docs, maven artifacts, then > announce. > > Thank you to all who helped to push this release out. > > > On Thu, 4 Aug 2022 at 11:48, Ashutosh Gupta > wrote: > > > +1 (non-binding) > > > > * Builds from source looks good. > > * Checksums and signatures are correct. > > * Running basic HDFS commands and running simple MapReduce jobs looks good. > > * Skimmed through the contents of site documentation and it looks good. > > > > Thanks Steve for driving this release. > > > > Ashutosh > > > > > > On Wed, Aug 3, 2022 at 9:39 PM Chris Nauroth wrote: > > > > > +1 (binding) > > > > > > * Verified all checksums. > > > * Verified all signatures. > > > * Built from source, including native code on Linux. > > > * mvn clean package -Pnative -Psrc -Drequire.openssl -Drequire.snappy > > > -Drequire.zstd -DskipTests > > > * Tests passed. > > > * mvn --fail-never clean test -Pnative -Dparallel-tests > > > -Drequire.snappy -Drequire.zstd -Drequire.openssl > > > -Dsurefire.rerunFailingTestsCount=3 -DtestsThreadCount=8 > > > * Checked dependency tree to make sure we have all of the expected > > library > > > updates that are mentioned in the release notes. > > > * mvn -o dependency:tree > > > > > > I saw a LibHDFS test failure, but I know it's something flaky that's > > > already tracked in a JIRA issue. The release looks good. Steve, thank you > > > for driving this. > > > > > > Chris Nauroth > > > > > > > > > On Wed, Aug 3, 2022 at 11:27 AM Steve Loughran > > > > > > > > wrote: > > > > > > > my vote for this is +1, binding. > > > > > > > > obviously I`m biased, but i do not want to have to issue any more > > interim > > > > releases before the feature release off branch-3.3, so I am trying to > > be > > > > ruthless. > > > > > > > > my client vaidator ant project has a more targets to help with > > releasing, > > > > and now builds a lot mor of my local projects > > > > https://github.com/steveloughran/validate-hadoop-client-artifacts > > > > all good as far as my test coverage goes, with these projects > > validating > > > > the staged dependencies. > > > > > > > > now, who else can review > > > > > > > > On Fri, 29 Jul 2022 at 19:47, Steve Loughran > > > wrote: > > > > > > > > > > > > > > > > > > > I have put together a release candidate (RC1) for Hadoop 3.3.4 > > > > > > > > > > The RC is available at: > > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/ > > > > > > > > > > The git tag is release-3.3.4-RC1, commit a585a73c3e0 > > > > > > > > > > The maven artifacts are staged at > > > > > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1358/ > > > > > > > > > > You can find my public key at: > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > > > > > > > Change log > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/CHANGELOG.md > > > > > > > > > > Release notes > > > > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/RELEASENOTES.md > > > > > > > > > > There's a very small number of changes, primarily critical > > > code/packaging > > > > > issues and security fixes. > > > > > > > > > > See the release notes for details. > > > > > > > > > > Please try the release and vote. The vote will run for 5 days. > > > > > > > > > > steve > > > > > > > > > > > > > > - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.3.4
The vote passed with the following result. Binding PMC members: +1 Chris Nauroth +1 Steve Loughran +1 Masatake Iwasaki Non binding votes: +1 Ashutosh Gupta Cheng Pan was worried about the new transient kotlin dependency. They are working on a PR there which we can target for the successor to this release https://issues.apache.org/jira/browse/HDFS-16714 I'm going to publish the artifacts, site docs, maven artifacts, then announce. Thank you to all who helped to push this release out. On Thu, 4 Aug 2022 at 11:48, Ashutosh Gupta wrote: > +1 (non-binding) > > * Builds from source looks good. > * Checksums and signatures are correct. > * Running basic HDFS commands and running simple MapReduce jobs looks good. > * Skimmed through the contents of site documentation and it looks good. > > Thanks Steve for driving this release. > > Ashutosh > > > On Wed, Aug 3, 2022 at 9:39 PM Chris Nauroth wrote: > > > +1 (binding) > > > > * Verified all checksums. > > * Verified all signatures. > > * Built from source, including native code on Linux. > > * mvn clean package -Pnative -Psrc -Drequire.openssl -Drequire.snappy > > -Drequire.zstd -DskipTests > > * Tests passed. > > * mvn --fail-never clean test -Pnative -Dparallel-tests > > -Drequire.snappy -Drequire.zstd -Drequire.openssl > > -Dsurefire.rerunFailingTestsCount=3 -DtestsThreadCount=8 > > * Checked dependency tree to make sure we have all of the expected > library > > updates that are mentioned in the release notes. > > * mvn -o dependency:tree > > > > I saw a LibHDFS test failure, but I know it's something flaky that's > > already tracked in a JIRA issue. The release looks good. Steve, thank you > > for driving this. > > > > Chris Nauroth > > > > > > On Wed, Aug 3, 2022 at 11:27 AM Steve Loughran > > > > > wrote: > > > > > my vote for this is +1, binding. > > > > > > obviously I`m biased, but i do not want to have to issue any more > interim > > > releases before the feature release off branch-3.3, so I am trying to > be > > > ruthless. > > > > > > my client vaidator ant project has a more targets to help with > releasing, > > > and now builds a lot mor of my local projects > > > https://github.com/steveloughran/validate-hadoop-client-artifacts > > > all good as far as my test coverage goes, with these projects > validating > > > the staged dependencies. > > > > > > now, who else can review > > > > > > On Fri, 29 Jul 2022 at 19:47, Steve Loughran > > wrote: > > > > > > > > > > > > > > > I have put together a release candidate (RC1) for Hadoop 3.3.4 > > > > > > > > The RC is available at: > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/ > > > > > > > > The git tag is release-3.3.4-RC1, commit a585a73c3e0 > > > > > > > > The maven artifacts are staged at > > > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1358/ > > > > > > > > You can find my public key at: > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > > > > > Change log > > > > > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/CHANGELOG.md > > > > > > > > Release notes > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/RELEASENOTES.md > > > > > > > > There's a very small number of changes, primarily critical > > code/packaging > > > > issues and security fixes. > > > > > > > > See the release notes for details. > > > > > > > > Please try the release and vote. The vote will run for 5 days. > > > > > > > > steve > > > > > > > > > >
Re: [VOTE] Release Apache Hadoop 3.3.4
+1 (non-binding) * Builds from source looks good. * Checksums and signatures are correct. * Running basic HDFS commands and running simple MapReduce jobs looks good. * Skimmed through the contents of site documentation and it looks good. Thanks Steve for driving this release. Ashutosh On Wed, Aug 3, 2022 at 9:39 PM Chris Nauroth wrote: > +1 (binding) > > * Verified all checksums. > * Verified all signatures. > * Built from source, including native code on Linux. > * mvn clean package -Pnative -Psrc -Drequire.openssl -Drequire.snappy > -Drequire.zstd -DskipTests > * Tests passed. > * mvn --fail-never clean test -Pnative -Dparallel-tests > -Drequire.snappy -Drequire.zstd -Drequire.openssl > -Dsurefire.rerunFailingTestsCount=3 -DtestsThreadCount=8 > * Checked dependency tree to make sure we have all of the expected library > updates that are mentioned in the release notes. > * mvn -o dependency:tree > > I saw a LibHDFS test failure, but I know it's something flaky that's > already tracked in a JIRA issue. The release looks good. Steve, thank you > for driving this. > > Chris Nauroth > > > On Wed, Aug 3, 2022 at 11:27 AM Steve Loughran > > wrote: > > > my vote for this is +1, binding. > > > > obviously I`m biased, but i do not want to have to issue any more interim > > releases before the feature release off branch-3.3, so I am trying to be > > ruthless. > > > > my client vaidator ant project has a more targets to help with releasing, > > and now builds a lot mor of my local projects > > https://github.com/steveloughran/validate-hadoop-client-artifacts > > all good as far as my test coverage goes, with these projects validating > > the staged dependencies. > > > > now, who else can review > > > > On Fri, 29 Jul 2022 at 19:47, Steve Loughran > wrote: > > > > > > > > > > > I have put together a release candidate (RC1) for Hadoop 3.3.4 > > > > > > The RC is available at: > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/ > > > > > > The git tag is release-3.3.4-RC1, commit a585a73c3e0 > > > > > > The maven artifacts are staged at > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1358/ > > > > > > You can find my public key at: > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > > > Change log > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/CHANGELOG.md > > > > > > Release notes > > > > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/RELEASENOTES.md > > > > > > There's a very small number of changes, primarily critical > code/packaging > > > issues and security fixes. > > > > > > See the release notes for details. > > > > > > Please try the release and vote. The vote will run for 5 days. > > > > > > steve > > > > > >
Re: [VOTE] Release Apache Hadoop 3.3.4
+1 (binding) * Verified all checksums. * Verified all signatures. * Built from source, including native code on Linux. * mvn clean package -Pnative -Psrc -Drequire.openssl -Drequire.snappy -Drequire.zstd -DskipTests * Tests passed. * mvn --fail-never clean test -Pnative -Dparallel-tests -Drequire.snappy -Drequire.zstd -Drequire.openssl -Dsurefire.rerunFailingTestsCount=3 -DtestsThreadCount=8 * Checked dependency tree to make sure we have all of the expected library updates that are mentioned in the release notes. * mvn -o dependency:tree I saw a LibHDFS test failure, but I know it's something flaky that's already tracked in a JIRA issue. The release looks good. Steve, thank you for driving this. Chris Nauroth On Wed, Aug 3, 2022 at 11:27 AM Steve Loughran wrote: > my vote for this is +1, binding. > > obviously I`m biased, but i do not want to have to issue any more interim > releases before the feature release off branch-3.3, so I am trying to be > ruthless. > > my client vaidator ant project has a more targets to help with releasing, > and now builds a lot mor of my local projects > https://github.com/steveloughran/validate-hadoop-client-artifacts > all good as far as my test coverage goes, with these projects validating > the staged dependencies. > > now, who else can review > > On Fri, 29 Jul 2022 at 19:47, Steve Loughran wrote: > > > > > > > I have put together a release candidate (RC1) for Hadoop 3.3.4 > > > > The RC is available at: > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/ > > > > The git tag is release-3.3.4-RC1, commit a585a73c3e0 > > > > The maven artifacts are staged at > > https://repository.apache.org/content/repositories/orgapachehadoop-1358/ > > > > You can find my public key at: > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > Change log > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/CHANGELOG.md > > > > Release notes > > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/RELEASENOTES.md > > > > There's a very small number of changes, primarily critical code/packaging > > issues and security fixes. > > > > See the release notes for details. > > > > Please try the release and vote. The vote will run for 5 days. > > > > steve > > >
Re: [VOTE] Release Apache Hadoop 3.3.4
my vote for this is +1, binding. obviously I`m biased, but i do not want to have to issue any more interim releases before the feature release off branch-3.3, so I am trying to be ruthless. my client vaidator ant project has a more targets to help with releasing, and now builds a lot mor of my local projects https://github.com/steveloughran/validate-hadoop-client-artifacts all good as far as my test coverage goes, with these projects validating the staged dependencies. now, who else can review On Fri, 29 Jul 2022 at 19:47, Steve Loughran wrote: > > > I have put together a release candidate (RC1) for Hadoop 3.3.4 > > The RC is available at: > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/ > > The git tag is release-3.3.4-RC1, commit a585a73c3e0 > > The maven artifacts are staged at > https://repository.apache.org/content/repositories/orgapachehadoop-1358/ > > You can find my public key at: > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > Change log > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/CHANGELOG.md > > Release notes > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/RELEASENOTES.md > > There's a very small number of changes, primarily critical code/packaging > issues and security fixes. > > See the release notes for details. > > Please try the release and vote. The vote will run for 5 days. > > steve >
Re: [VOTE] Release Apache Hadoop 3.3.4
+1(binding) * skimmed the contents of site documentation. * built the source tarball on Rocky Linux 8 (x86_64) by OpenJDK 8 with `-Pnative`. * launched pseudo distributed cluster including kms and httpfs with Kerberos and SSL enabled. * created encryption zone, put and read files via httpfs. * ran example MR wordcount over encryption zone. * launched 3-node docker cluster with NN-HA and RM-HA enabled and ran some example MR jobs. * built HBase 2.4.13, Hive 3.1.3 and Spark 3.2.1 against Hadoop 3.3.4 RC1 on CentOS 7 (x86_64) by using Bigtop and ran smoke-tests. https://github.com/apache/bigtop/pull/964 * HBsae needed to fix relocation rules due to dependency on okhttp added by HDFS-16453. * built Spark 3.3.0 using the staging repository. Thanks, Masatake Iwasaki On 2022/07/30 3:47, Steve Loughran wrote: I have put together a release candidate (RC1) for Hadoop 3.3.4 The RC is available at: https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/ The git tag is release-3.3.4-RC1, commit a585a73c3e0 The maven artifacts are staged at https://repository.apache.org/content/repositories/orgapachehadoop-1358/ You can find my public key at: https://dist.apache.org/repos/dist/release/hadoop/common/KEYS Change log https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/CHANGELOG.md Release notes https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/RELEASENOTES.md There's a very small number of changes, primarily critical code/packaging issues and security fixes. See the release notes for details. Please try the release and vote. The vote will run for 5 days. steve - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.3.4
cancelling this RC so i can issue a new one with an updated reload4j https://issues.apache.org/jira/browse/HADOOP-18354 i will also do a pr to update aws sdk, which is less critical (the jackson databind classes don't seem to get used), -upgrading will stop audit tools overreacting On Thu, 21 Jul 2022 at 19:07, Steve Loughran wrote: > > I have put together a release candidate (RC0) for Hadoop 3.3.4 > > The RC is available at: > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC0/ > > The git tag is release-3.3.4-RC0, commit c679bc76d26 > > The maven artifacts are staged at > https://repository.apache.org/content/repositories/orgapachehadoop-1356/ > > You can find my public key at: > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > Change log > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC0/CHANGELOG.md > > Release notes > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC0/RELEASENOTES.md > > There's a very small number of changes, primarily critical code/packaging > issues and security fixes. > > See the release notes for details. > > Please try the release and vote. The vote will run for 5 days. > > -Steve > > >
