Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

[Steve Loughran]

Here are the result of the vote.

Binding PMC members

+1 Stack
+1 Masatake Iwasaki
+1 Xiaoqiao He
+1 Chao Sun
+1 Chris Nauroth


Non binding votes

+1 Ayush Saxena
+1 Viraj Jasani
+1 Mukund Thakur


There were no negative votes, and maven is happy.
I'm pleased to announce therefore, that this release candidate has been
approved to become the final release.

I'm going to go through the next steps of the process: publishing the
artifacts, site docs, maven artifacts, announcements etc.

Thank you to all who reviewed this!

-Steve

PS: I'm going to start some discussion on correcting/tuning the release
process, once I've got through it.

On Mon, 16 May 2022 at 19:16, Chris Nauroth  wrote:

> +1 (binding)
>
> - Verified all checksums.
> - Verified all signatures.
> - Built from source, including native code on Linux.
> - Ran several examples successfully.
>
> Chris Nauroth
>
>
> On Mon, May 16, 2022 at 10:06 AM Chao Sun  wrote:
>
> > +1
> >
> > - Compiled from source
> > - Verified checksums & signatures
> > - Launched a pseudo HDFS cluster and ran some simple commands
> > - Ran full Spark tests with the RC
> >
> > Thanks Steve!
> >
> > Chao
> >
> > On Mon, May 16, 2022 at 2:19 AM Ayush Saxena  wrote:
> > >
> > > +1,
> > > * Built from source.
> > > * Successful native build on Ubuntu 18.04
> > > * Verified Checksums.
> > >
> >
> (CHANGELOG.md,RELEASENOTES.md,hadoop-3.3.3-rat.txt,hadoop-3.3.3-site.tar.gz,hadoop-3.3.3-src.tar.gz,hadoop-3.3.3.tar.gz)
> > > * Verified Signature.
> > > * Successful RAT check
> > > * Ran basic HDFS shell commands.
> > > * Ran basic YARN shell commands.
> > > * Verified version in hadoop version command and UI
> > > * Ran some MR example Jobs.
> > > * Browsed
> UI(Namenode/Datanode/ResourceManager/NodeManager/HistoryServer)
> > > * Browsed the contents of Maven Artifacts.
> > > * Browsed the contents of the website.
> > >
> > > Thanx Steve for driving the release, Good Luck!!!
> > >
> > > -Ayush
> > >
> > > On Mon, 16 May 2022 at 08:20, Xiaoqiao He 
> wrote:
> > >
> > > > +1(binding)
> > > >
> > > > * Verified signature and checksum of the source tarball.
> > > > * Built the source code on Ubuntu and OpenJDK 11 by `mvn clean
> package
> > > > -DskipTests -Pnative -Pdist -Dtar`.
> > > > * Setup pseudo cluster with HDFS and YARN.
> > > > * Run simple FsShell - mkdir/put/get/mv/rm and check the result.
> > > > * Run example mr applications and check the result - Pi & wordcount.
> > > > * Check the Web UI of NameNode/DataNode/Resourcemanager/NodeManager
> > etc.
> > > >
> > > > Thanks Steve for your work.
> > > >
> > > > - He Xiaoqiao
> > > >
> > > > On Mon, May 16, 2022 at 4:25 AM Viraj Jasani 
> > wrote:
> > > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > * Signature: ok
> > > > > * Checksum : ok
> > > > > * Rat check (1.8.0_301): ok
> > > > >  - mvn clean apache-rat:check
> > > > > * Built from source (1.8.0_301): ok
> > > > >  - mvn clean install  -DskipTests
> > > > > * Built tar from source (1.8.0_301): ok
> > > > >  - mvn clean package  -Pdist -DskipTests -Dtar
> > -Dmaven.javadoc.skip=true
> > > > >
> > > > > HDFS, MapReduce and HBase (2.5) CRUD functional testing on
> > > > > pseudo-distributed mode looks good.
> > > > >
> > > > >
> > > > > On Wed, May 11, 2022 at 10:26 AM Steve Loughran
> > > > 
> > > > > wrote:
> > > > >
> > > > > > I have put together a release candidate (RC1) for Hadoop 3.3.3
> > > > > >
> > > > > > The RC is available at:
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/
> > > > > >
> > > > > > The git tag is release-3.3.3-RC1, commit d37586cbda3
> > > > > >
> > > > > > The maven artifacts are staged at
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1349/
> > > > > >
> > > > > > You can find my public key at:
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > > > > >
> > > > > > Change log
> > > > > >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/CHANGELOG.md
> > > > > >
> > > > > > Release notes
> > > > > >
> > > >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/RELEASENOTES.md
> > > > > >
> > > > > > There's a very small number of changes, primarily critical
> > > > code/packaging
> > > > > > issues and security fixes.
> > > > > >
> > > > > > * The critical fixes which shipped in the 3.2.3 release.
> > > > > > * CVEs in our code and dependencies
> > > > > > * Shaded client packaging issues.
> > > > > > * A switch from log4j to reload4j
> > > > > >
> > > > > > reload4j is an active fork of the log4j 1.17 library with the
> > classes
> > > > > > which contain CVEs removed. Even though hadoop never used those
> > > > classes,
> > > > > > they regularly raised alerts on security scans and concen from
> > users.
> > > > > > Switching to the forked project allows us to ship a secure
> logging
> > > > > > framework. It will complicate the builds of downstream
> > > > > > maven/ivy/gradle projects which exclude our log4j artifacts, as
> > 

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

[Chris Nauroth]

+1 (binding)

- Verified all checksums.
- Verified all signatures.
- Built from source, including native code on Linux.
- Ran several examples successfully.

Chris Nauroth


On Mon, May 16, 2022 at 10:06 AM Chao Sun  wrote:

> +1
>
> - Compiled from source
> - Verified checksums & signatures
> - Launched a pseudo HDFS cluster and ran some simple commands
> - Ran full Spark tests with the RC
>
> Thanks Steve!
>
> Chao
>
> On Mon, May 16, 2022 at 2:19 AM Ayush Saxena  wrote:
> >
> > +1,
> > * Built from source.
> > * Successful native build on Ubuntu 18.04
> > * Verified Checksums.
> >
> (CHANGELOG.md,RELEASENOTES.md,hadoop-3.3.3-rat.txt,hadoop-3.3.3-site.tar.gz,hadoop-3.3.3-src.tar.gz,hadoop-3.3.3.tar.gz)
> > * Verified Signature.
> > * Successful RAT check
> > * Ran basic HDFS shell commands.
> > * Ran basic YARN shell commands.
> > * Verified version in hadoop version command and UI
> > * Ran some MR example Jobs.
> > * Browsed UI(Namenode/Datanode/ResourceManager/NodeManager/HistoryServer)
> > * Browsed the contents of Maven Artifacts.
> > * Browsed the contents of the website.
> >
> > Thanx Steve for driving the release, Good Luck!!!
> >
> > -Ayush
> >
> > On Mon, 16 May 2022 at 08:20, Xiaoqiao He  wrote:
> >
> > > +1(binding)
> > >
> > > * Verified signature and checksum of the source tarball.
> > > * Built the source code on Ubuntu and OpenJDK 11 by `mvn clean package
> > > -DskipTests -Pnative -Pdist -Dtar`.
> > > * Setup pseudo cluster with HDFS and YARN.
> > > * Run simple FsShell - mkdir/put/get/mv/rm and check the result.
> > > * Run example mr applications and check the result - Pi & wordcount.
> > > * Check the Web UI of NameNode/DataNode/Resourcemanager/NodeManager
> etc.
> > >
> > > Thanks Steve for your work.
> > >
> > > - He Xiaoqiao
> > >
> > > On Mon, May 16, 2022 at 4:25 AM Viraj Jasani 
> wrote:
> > > >
> > > > +1 (non-binding)
> > > >
> > > > * Signature: ok
> > > > * Checksum : ok
> > > > * Rat check (1.8.0_301): ok
> > > >  - mvn clean apache-rat:check
> > > > * Built from source (1.8.0_301): ok
> > > >  - mvn clean install  -DskipTests
> > > > * Built tar from source (1.8.0_301): ok
> > > >  - mvn clean package  -Pdist -DskipTests -Dtar
> -Dmaven.javadoc.skip=true
> > > >
> > > > HDFS, MapReduce and HBase (2.5) CRUD functional testing on
> > > > pseudo-distributed mode looks good.
> > > >
> > > >
> > > > On Wed, May 11, 2022 at 10:26 AM Steve Loughran
> > > 
> > > > wrote:
> > > >
> > > > > I have put together a release candidate (RC1) for Hadoop 3.3.3
> > > > >
> > > > > The RC is available at:
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/
> > > > >
> > > > > The git tag is release-3.3.3-RC1, commit d37586cbda3
> > > > >
> > > > > The maven artifacts are staged at
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1349/
> > > > >
> > > > > You can find my public key at:
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > > > >
> > > > > Change log
> > > > >
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/CHANGELOG.md
> > > > >
> > > > > Release notes
> > > > >
> > >
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/RELEASENOTES.md
> > > > >
> > > > > There's a very small number of changes, primarily critical
> > > code/packaging
> > > > > issues and security fixes.
> > > > >
> > > > > * The critical fixes which shipped in the 3.2.3 release.
> > > > > * CVEs in our code and dependencies
> > > > > * Shaded client packaging issues.
> > > > > * A switch from log4j to reload4j
> > > > >
> > > > > reload4j is an active fork of the log4j 1.17 library with the
> classes
> > > > > which contain CVEs removed. Even though hadoop never used those
> > > classes,
> > > > > they regularly raised alerts on security scans and concen from
> users.
> > > > > Switching to the forked project allows us to ship a secure logging
> > > > > framework. It will complicate the builds of downstream
> > > > > maven/ivy/gradle projects which exclude our log4j artifacts, as
> they
> > > > > need to cut the new dependency instead/as well.
> > > > >
> > > > > See the release notes for details.
> > > > >
> > > > > This is the second release attempt. It is the same git commit as
> > > before,
> > > > > but
> > > > > fully recompiled with another republish to maven staging, which
> has bee
> > > > > verified by building spark, as well as a minimal test project.
> > > > >
> > > > > Please try the release and vote. The vote will run for 5 days.
> > > > >
> > > > > -Steve
> > > > >
> > >
> > > -
> > > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> > > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> > >
> > >
>
> -
> To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
>

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

[Chao Sun]

+1

- Compiled from source
- Verified checksums & signatures
- Launched a pseudo HDFS cluster and ran some simple commands
- Ran full Spark tests with the RC

Thanks Steve!

Chao

On Mon, May 16, 2022 at 2:19 AM Ayush Saxena  wrote:
>
> +1,
> * Built from source.
> * Successful native build on Ubuntu 18.04
> * Verified Checksums.
> (CHANGELOG.md,RELEASENOTES.md,hadoop-3.3.3-rat.txt,hadoop-3.3.3-site.tar.gz,hadoop-3.3.3-src.tar.gz,hadoop-3.3.3.tar.gz)
> * Verified Signature.
> * Successful RAT check
> * Ran basic HDFS shell commands.
> * Ran basic YARN shell commands.
> * Verified version in hadoop version command and UI
> * Ran some MR example Jobs.
> * Browsed UI(Namenode/Datanode/ResourceManager/NodeManager/HistoryServer)
> * Browsed the contents of Maven Artifacts.
> * Browsed the contents of the website.
>
> Thanx Steve for driving the release, Good Luck!!!
>
> -Ayush
>
> On Mon, 16 May 2022 at 08:20, Xiaoqiao He  wrote:
>
> > +1(binding)
> >
> > * Verified signature and checksum of the source tarball.
> > * Built the source code on Ubuntu and OpenJDK 11 by `mvn clean package
> > -DskipTests -Pnative -Pdist -Dtar`.
> > * Setup pseudo cluster with HDFS and YARN.
> > * Run simple FsShell - mkdir/put/get/mv/rm and check the result.
> > * Run example mr applications and check the result - Pi & wordcount.
> > * Check the Web UI of NameNode/DataNode/Resourcemanager/NodeManager etc.
> >
> > Thanks Steve for your work.
> >
> > - He Xiaoqiao
> >
> > On Mon, May 16, 2022 at 4:25 AM Viraj Jasani  wrote:
> > >
> > > +1 (non-binding)
> > >
> > > * Signature: ok
> > > * Checksum : ok
> > > * Rat check (1.8.0_301): ok
> > >  - mvn clean apache-rat:check
> > > * Built from source (1.8.0_301): ok
> > >  - mvn clean install  -DskipTests
> > > * Built tar from source (1.8.0_301): ok
> > >  - mvn clean package  -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true
> > >
> > > HDFS, MapReduce and HBase (2.5) CRUD functional testing on
> > > pseudo-distributed mode looks good.
> > >
> > >
> > > On Wed, May 11, 2022 at 10:26 AM Steve Loughran
> > 
> > > wrote:
> > >
> > > > I have put together a release candidate (RC1) for Hadoop 3.3.3
> > > >
> > > > The RC is available at:
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/
> > > >
> > > > The git tag is release-3.3.3-RC1, commit d37586cbda3
> > > >
> > > > The maven artifacts are staged at
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1349/
> > > >
> > > > You can find my public key at:
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > > >
> > > > Change log
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/CHANGELOG.md
> > > >
> > > > Release notes
> > > >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/RELEASENOTES.md
> > > >
> > > > There's a very small number of changes, primarily critical
> > code/packaging
> > > > issues and security fixes.
> > > >
> > > > * The critical fixes which shipped in the 3.2.3 release.
> > > > * CVEs in our code and dependencies
> > > > * Shaded client packaging issues.
> > > > * A switch from log4j to reload4j
> > > >
> > > > reload4j is an active fork of the log4j 1.17 library with the classes
> > > > which contain CVEs removed. Even though hadoop never used those
> > classes,
> > > > they regularly raised alerts on security scans and concen from users.
> > > > Switching to the forked project allows us to ship a secure logging
> > > > framework. It will complicate the builds of downstream
> > > > maven/ivy/gradle projects which exclude our log4j artifacts, as they
> > > > need to cut the new dependency instead/as well.
> > > >
> > > > See the release notes for details.
> > > >
> > > > This is the second release attempt. It is the same git commit as
> > before,
> > > > but
> > > > fully recompiled with another republish to maven staging, which has bee
> > > > verified by building spark, as well as a minimal test project.
> > > >
> > > > Please try the release and vote. The vote will run for 5 days.
> > > >
> > > > -Steve
> > > >
> >
> > -
> > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> >
> >

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

[Ayush Saxena]

+1,
* Built from source.
* Successful native build on Ubuntu 18.04
* Verified Checksums.
(CHANGELOG.md,RELEASENOTES.md,hadoop-3.3.3-rat.txt,hadoop-3.3.3-site.tar.gz,hadoop-3.3.3-src.tar.gz,hadoop-3.3.3.tar.gz)
* Verified Signature.
* Successful RAT check
* Ran basic HDFS shell commands.
* Ran basic YARN shell commands.
* Verified version in hadoop version command and UI
* Ran some MR example Jobs.
* Browsed UI(Namenode/Datanode/ResourceManager/NodeManager/HistoryServer)
* Browsed the contents of Maven Artifacts.
* Browsed the contents of the website.

Thanx Steve for driving the release, Good Luck!!!

-Ayush

On Mon, 16 May 2022 at 08:20, Xiaoqiao He  wrote:

> +1(binding)
>
> * Verified signature and checksum of the source tarball.
> * Built the source code on Ubuntu and OpenJDK 11 by `mvn clean package
> -DskipTests -Pnative -Pdist -Dtar`.
> * Setup pseudo cluster with HDFS and YARN.
> * Run simple FsShell - mkdir/put/get/mv/rm and check the result.
> * Run example mr applications and check the result - Pi & wordcount.
> * Check the Web UI of NameNode/DataNode/Resourcemanager/NodeManager etc.
>
> Thanks Steve for your work.
>
> - He Xiaoqiao
>
> On Mon, May 16, 2022 at 4:25 AM Viraj Jasani  wrote:
> >
> > +1 (non-binding)
> >
> > * Signature: ok
> > * Checksum : ok
> > * Rat check (1.8.0_301): ok
> >  - mvn clean apache-rat:check
> > * Built from source (1.8.0_301): ok
> >  - mvn clean install  -DskipTests
> > * Built tar from source (1.8.0_301): ok
> >  - mvn clean package  -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true
> >
> > HDFS, MapReduce and HBase (2.5) CRUD functional testing on
> > pseudo-distributed mode looks good.
> >
> >
> > On Wed, May 11, 2022 at 10:26 AM Steve Loughran
> 
> > wrote:
> >
> > > I have put together a release candidate (RC1) for Hadoop 3.3.3
> > >
> > > The RC is available at:
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/
> > >
> > > The git tag is release-3.3.3-RC1, commit d37586cbda3
> > >
> > > The maven artifacts are staged at
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1349/
> > >
> > > You can find my public key at:
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > >
> > > Change log
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/CHANGELOG.md
> > >
> > > Release notes
> > >
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/RELEASENOTES.md
> > >
> > > There's a very small number of changes, primarily critical
> code/packaging
> > > issues and security fixes.
> > >
> > > * The critical fixes which shipped in the 3.2.3 release.
> > > * CVEs in our code and dependencies
> > > * Shaded client packaging issues.
> > > * A switch from log4j to reload4j
> > >
> > > reload4j is an active fork of the log4j 1.17 library with the classes
> > > which contain CVEs removed. Even though hadoop never used those
> classes,
> > > they regularly raised alerts on security scans and concen from users.
> > > Switching to the forked project allows us to ship a secure logging
> > > framework. It will complicate the builds of downstream
> > > maven/ivy/gradle projects which exclude our log4j artifacts, as they
> > > need to cut the new dependency instead/as well.
> > >
> > > See the release notes for details.
> > >
> > > This is the second release attempt. It is the same git commit as
> before,
> > > but
> > > fully recompiled with another republish to maven staging, which has bee
> > > verified by building spark, as well as a minimal test project.
> > >
> > > Please try the release and vote. The vote will run for 5 days.
> > >
> > > -Steve
> > >
>
> -
> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

[Xiaoqiao He]

+1(binding)

* Verified signature and checksum of the source tarball.
* Built the source code on Ubuntu and OpenJDK 11 by `mvn clean package
-DskipTests -Pnative -Pdist -Dtar`.
* Setup pseudo cluster with HDFS and YARN.
* Run simple FsShell - mkdir/put/get/mv/rm and check the result.
* Run example mr applications and check the result - Pi & wordcount.
* Check the Web UI of NameNode/DataNode/Resourcemanager/NodeManager etc.

Thanks Steve for your work.

- He Xiaoqiao

On Mon, May 16, 2022 at 4:25 AM Viraj Jasani  wrote:
>
> +1 (non-binding)
>
> * Signature: ok
> * Checksum : ok
> * Rat check (1.8.0_301): ok
>  - mvn clean apache-rat:check
> * Built from source (1.8.0_301): ok
>  - mvn clean install  -DskipTests
> * Built tar from source (1.8.0_301): ok
>  - mvn clean package  -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true
>
> HDFS, MapReduce and HBase (2.5) CRUD functional testing on
> pseudo-distributed mode looks good.
>
>
> On Wed, May 11, 2022 at 10:26 AM Steve Loughran 
> wrote:
>
> > I have put together a release candidate (RC1) for Hadoop 3.3.3
> >
> > The RC is available at:
> > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/
> >
> > The git tag is release-3.3.3-RC1, commit d37586cbda3
> >
> > The maven artifacts are staged at
> > https://repository.apache.org/content/repositories/orgapachehadoop-1349/
> >
> > You can find my public key at:
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> > Change log
> > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/CHANGELOG.md
> >
> > Release notes
> > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/RELEASENOTES.md
> >
> > There's a very small number of changes, primarily critical code/packaging
> > issues and security fixes.
> >
> > * The critical fixes which shipped in the 3.2.3 release.
> > * CVEs in our code and dependencies
> > * Shaded client packaging issues.
> > * A switch from log4j to reload4j
> >
> > reload4j is an active fork of the log4j 1.17 library with the classes
> > which contain CVEs removed. Even though hadoop never used those classes,
> > they regularly raised alerts on security scans and concen from users.
> > Switching to the forked project allows us to ship a secure logging
> > framework. It will complicate the builds of downstream
> > maven/ivy/gradle projects which exclude our log4j artifacts, as they
> > need to cut the new dependency instead/as well.
> >
> > See the release notes for details.
> >
> > This is the second release attempt. It is the same git commit as before,
> > but
> > fully recompiled with another republish to maven staging, which has bee
> > verified by building spark, as well as a minimal test project.
> >
> > Please try the release and vote. The vote will run for 5 days.
> >
> > -Steve
> >

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

[Viraj Jasani]

+1 (non-binding)

* Signature: ok
* Checksum : ok
* Rat check (1.8.0_301): ok
 - mvn clean apache-rat:check
* Built from source (1.8.0_301): ok
 - mvn clean install  -DskipTests
* Built tar from source (1.8.0_301): ok
 - mvn clean package  -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true

HDFS, MapReduce and HBase (2.5) CRUD functional testing on
pseudo-distributed mode looks good.


On Wed, May 11, 2022 at 10:26 AM Steve Loughran 
wrote:

> I have put together a release candidate (RC1) for Hadoop 3.3.3
>
> The RC is available at:
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/
>
> The git tag is release-3.3.3-RC1, commit d37586cbda3
>
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1349/
>
> You can find my public key at:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Change log
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/CHANGELOG.md
>
> Release notes
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/RELEASENOTES.md
>
> There's a very small number of changes, primarily critical code/packaging
> issues and security fixes.
>
> * The critical fixes which shipped in the 3.2.3 release.
> * CVEs in our code and dependencies
> * Shaded client packaging issues.
> * A switch from log4j to reload4j
>
> reload4j is an active fork of the log4j 1.17 library with the classes
> which contain CVEs removed. Even though hadoop never used those classes,
> they regularly raised alerts on security scans and concen from users.
> Switching to the forked project allows us to ship a secure logging
> framework. It will complicate the builds of downstream
> maven/ivy/gradle projects which exclude our log4j artifacts, as they
> need to cut the new dependency instead/as well.
>
> See the release notes for details.
>
> This is the second release attempt. It is the same git commit as before,
> but
> fully recompiled with another republish to maven staging, which has bee
> verified by building spark, as well as a minimal test project.
>
> Please try the release and vote. The vote will run for 5 days.
>
> -Steve
>

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

[Masatake Iwasaki]

+1 (binding)

* verified signature and checksum of the source tarball.
* built the source code on Rocky Linux 8 (x86_64) and OpenJDK 8
  by `mvn install -DskipTests -Pnative -Pdist`.
* launched pseudo distributed cluster with Kerberos security enabled and ran 
sample MR jobs.
* launched HA enabled 3-nodes docker cluster and ran sample MR jobs.
* built site documentation by `mvn site site:stage -Preleasedocs` and skimmed 
the contents.
* built Spark 3.2.1 against 3.3.3 RC1 using the staging repository
  (after `rm -rf ~/.m2/repository/org/apache/hadoop`).

Thanks,
Masatake Iwasaki

On 2022/05/12 2:25, Steve Loughran wrote:

I have put together a release candidate (RC1) for Hadoop 3.3.3

The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/

The git tag is release-3.3.3-RC1, commit d37586cbda3

The maven artifacts are staged at
https://repository.apache.org/content/repositories/orgapachehadoop-1349/

You can find my public key at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS

Change log
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/CHANGELOG.md

Release notes
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/RELEASENOTES.md

There's a very small number of changes, primarily critical code/packaging
issues and security fixes.

* The critical fixes which shipped in the 3.2.3 release.
* CVEs in our code and dependencies
* Shaded client packaging issues.
* A switch from log4j to reload4j

reload4j is an active fork of the log4j 1.17 library with the classes
which contain CVEs removed. Even though hadoop never used those classes,
they regularly raised alerts on security scans and concen from users.
Switching to the forked project allows us to ship a secure logging
framework. It will complicate the builds of downstream
maven/ivy/gradle projects which exclude our log4j artifacts, as they
need to cut the new dependency instead/as well.

See the release notes for details.

This is the second release attempt. It is the same git commit as before, but
fully recompiled with another republish to maven staging, which has bee
verified by building spark, as well as a minimal test project.

Please try the release and vote. The vote will run for 5 days.

-Steve



-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

[Mukund Madhav Thakur]

+1 (binding)

Signature : good
Checksum : good.
Compiled from source , good

mvn clean install -DskipTests
Ran aws integration tests, good.

 mvn  -Dparallel-tests -DtestsThreadCount=8 clean verify
Compiled gcs using staging repo with version 3.3.3, successful.

 mvn clean install -DskipTests -Dhadoop.version=3.3.3
-Psnapshots-and-staging


On Thu, May 12, 2022 at 7:56 PM Stack  wrote:

> +1 (binding)
>
> * Signature: ok
> * Checksum : ok
> * Rat check (10.0.2): ok
>  - mvn clean apache-rat:check
> * Built from source (10.0.2): ok
>  - mvn clean install  -DskipTests
> * Unit tests pass (10.0.2): ok
>  - mvn package -P runAllTests  -Dsurefire.rerunFailingTestsCount=3
>
> 
> [INFO] Apache Hadoop Cloud Storage Project  SUCCESS [
>  0.026 s]
> [INFO]
> 
> [INFO] BUILD SUCCESS
> [INFO]
> 
> [INFO] Total time:  12:51 h
> [INFO] Finished at: 2022-05-12T06:25:19-07:00
> [INFO]
> 
> [WARNING] The requested profile "runAllTests" could not be activated
> because it does not exist.
>
> Built a downstreamer against this RC and ran it in-the-small. Seemed fine.
>
> S
>
>
> On Wed, May 11, 2022 at 10:25 AM Steve Loughran
> 
> wrote:
>
> > I have put together a release candidate (RC1) for Hadoop 3.3.3
> >
> > The RC is available at:
> > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/
> >
> > The git tag is release-3.3.3-RC1, commit d37586cbda3
> >
> > The maven artifacts are staged at
> > https://repository.apache.org/content/repositories/orgapachehadoop-1349/
> >
> > You can find my public key at:
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> > Change log
> > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/CHANGELOG.md
> >
> > Release notes
> > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/RELEASENOTES.md
> >
> > There's a very small number of changes, primarily critical code/packaging
> > issues and security fixes.
> >
> > * The critical fixes which shipped in the 3.2.3 release.
> > * CVEs in our code and dependencies
> > * Shaded client packaging issues.
> > * A switch from log4j to reload4j
> >
> > reload4j is an active fork of the log4j 1.17 library with the classes
> > which contain CVEs removed. Even though hadoop never used those classes,
> > they regularly raised alerts on security scans and concen from users.
> > Switching to the forked project allows us to ship a secure logging
> > framework. It will complicate the builds of downstream
> > maven/ivy/gradle projects which exclude our log4j artifacts, as they
> > need to cut the new dependency instead/as well.
> >
> > See the release notes for details.
> >
> > This is the second release attempt. It is the same git commit as before,
> > but
> > fully recompiled with another republish to maven staging, which has bee
> > verified by building spark, as well as a minimal test project.
> >
> > Please try the release and vote. The vote will run for 5 days.
> >
> > -Steve
> >
>

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

[Stack]

+1 (binding)

* Signature: ok
* Checksum : ok
* Rat check (10.0.2): ok
 - mvn clean apache-rat:check
* Built from source (10.0.2): ok
 - mvn clean install  -DskipTests
* Unit tests pass (10.0.2): ok
 - mvn package -P runAllTests  -Dsurefire.rerunFailingTestsCount=3


[INFO] Apache Hadoop Cloud Storage Project  SUCCESS [
 0.026 s]
[INFO]

[INFO] BUILD SUCCESS
[INFO]

[INFO] Total time:  12:51 h
[INFO] Finished at: 2022-05-12T06:25:19-07:00
[INFO]

[WARNING] The requested profile "runAllTests" could not be activated
because it does not exist.

Built a downstreamer against this RC and ran it in-the-small. Seemed fine.

S


On Wed, May 11, 2022 at 10:25 AM Steve Loughran 
wrote:

> I have put together a release candidate (RC1) for Hadoop 3.3.3
>
> The RC is available at:
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/
>
> The git tag is release-3.3.3-RC1, commit d37586cbda3
>
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1349/
>
> You can find my public key at:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Change log
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/CHANGELOG.md
>
> Release notes
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/RELEASENOTES.md
>
> There's a very small number of changes, primarily critical code/packaging
> issues and security fixes.
>
> * The critical fixes which shipped in the 3.2.3 release.
> * CVEs in our code and dependencies
> * Shaded client packaging issues.
> * A switch from log4j to reload4j
>
> reload4j is an active fork of the log4j 1.17 library with the classes
> which contain CVEs removed. Even though hadoop never used those classes,
> they regularly raised alerts on security scans and concen from users.
> Switching to the forked project allows us to ship a secure logging
> framework. It will complicate the builds of downstream
> maven/ivy/gradle projects which exclude our log4j artifacts, as they
> need to cut the new dependency instead/as well.
>
> See the release notes for details.
>
> This is the second release attempt. It is the same git commit as before,
> but
> fully recompiled with another republish to maven staging, which has bee
> verified by building spark, as well as a minimal test project.
>
> Please try the release and vote. The vote will run for 5 days.
>
> -Steve
>

[VOTE] Release Apache Hadoop 3.3.3 (RC1)

[Steve Loughran]

I have put together a release candidate (RC1) for Hadoop 3.3.3

The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/

The git tag is release-3.3.3-RC1, commit d37586cbda3

The maven artifacts are staged at
https://repository.apache.org/content/repositories/orgapachehadoop-1349/

You can find my public key at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS

Change log
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/CHANGELOG.md

Release notes
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/RELEASENOTES.md

There's a very small number of changes, primarily critical code/packaging
issues and security fixes.

* The critical fixes which shipped in the 3.2.3 release.
* CVEs in our code and dependencies
* Shaded client packaging issues.
* A switch from log4j to reload4j

reload4j is an active fork of the log4j 1.17 library with the classes
which contain CVEs removed. Even though hadoop never used those classes,
they regularly raised alerts on security scans and concen from users.
Switching to the forked project allows us to ship a secure logging
framework. It will complicate the builds of downstream
maven/ivy/gradle projects which exclude our log4j artifacts, as they
need to cut the new dependency instead/as well.

See the release notes for details.

This is the second release attempt. It is the same git commit as before, but
fully recompiled with another republish to maven staging, which has bee
verified by building spark, as well as a minimal test project.

Please try the release and vote. The vote will run for 5 days.

-Steve

Re: [VOTE] Release Apache Hadoop 3.3.3

[Steve Loughran]

update: got a stub project whose test run requires the client api and
shaded artifacts, and whose clean target wiil delete all artifacts of a
specific version from your local repo, so as to ensure all it untainted

https://github.com/steveloughran/validate-hadoop-client-artifacts

I do now have a version in staging with the files; still don't know how the
previous delete lacked them, I will do more testing tomorrow before putting
up the next RC. which will be off the same git commit as before, just a
rebuild, repackage and republish.

-Steve

On Tue, 10 May 2022 at 00:18, Steve Loughran  wrote:

> I've done another docker build and the client jars appear to be there.
> I'll test tomorrow before putting up another vote. it will be exactly the
> same commit as before, just recompiled/republished
>
> On Mon, 9 May 2022 at 17:45, Chao Sun  wrote:
>
>> Agreed, that step #10 is out-dated and should be removed (I skipped
>> that when releasing Hadoop 3.3.2 but didn't update it, sorry).
>>
>> > How about using
>> https://repository.apache.org/content/repositories/orgapachehadoop-1348/
>>
>> Akira, I tried this too but it didn't work. I think we'd need the
>> artifacts to be properly pushed to the staging repository.
>>
>> > Could you please let me know how I can consume the Hadoop 3 jars in
>> maven?
>>
>> Gautham (if you are following this thread), you'll need to add the
>> following:
>>
>> 
>>staged
>>staged-releases
>>https://repository.apache.org/content/repositories/staging/
>> 
>>
>>  true
>>
>>
>>  true
>>
>>  
>>
>> to the `` section in your Maven pom.xml file.
>>
>> On Mon, May 9, 2022 at 8:52 AM Steve Loughran
>>  wrote:
>> >
>> > I didn't do that as the docker image was doing it itself...I discussed
>> this
>> > with Akira and Ayush & they agreed. so whatever went wrong. it was
>> > something else.
>> >
>> > I have been building a list of things I'd like to change there; cutting
>> > that line was one of them. but I need to work out the correct workflow.
>> >
>> > trying again, and creating a stub module to verify the client is in
>> staging
>> >
>> > On Mon, 9 May 2022 at 15:19, Masatake Iwasaki <
>> iwasak...@oss.nttdata.co.jp>
>> > wrote:
>> >
>> > > It seems to be caused by obsolete instruction in HowToRelease Wiki?
>> > >
>> > > After HADOOP-15058, `mvn deploy` is kicked by
>> > > `dev-support/bin/create-release --asfrelease`.
>> > > https://issues.apache.org/jira/browse/HADOOP-15058
>> > >
>> > > Step #10 in "Creating the release candidate (X.Y.Z-RC)" section
>> > > of the Wiki still instructs to run `mvn deploy` with `-DskipShade`.
>> > >
>> > > 2 sets of artifact are deployed after creating RC based on the
>> instruction.
>> > > The latest one contains empty shaded jars.
>> > >
>> > > hadoop-client-api and hadoop-client-runtime of already released 3.2.3
>> > > looks having same issue...
>> > >
>> > > Masatake Iwasaki
>> > >
>> > > On 2022/05/08 6:45, Akira Ajisaka wrote:
>> > > > Hi Chao,
>> > > >
>> > > > How about using
>> > > >
>> https://repository.apache.org/content/repositories/orgapachehadoop-1348/
>> > > > instead of
>> https://repository.apache.org/content/repositories/staging/ ?
>> > > >
>> > > > Akira
>> > > >
>> > > > On Sat, May 7, 2022 at 10:52 AM Ayush Saxena 
>> wrote:
>> > > >
>> > > >> Hmm, I see the artifacts ideally should have got overwritten by
>> the new
>> > > >> RC, but they didn’t. The reason seems like the staging path shared
>> > > doesn’t
>> > > >> have any jars…
>> > > >> That is why it was picking the old jars. I think Steve needs to
>> run mvn
>> > > >> deploy again…
>> > > >>
>> > > >> Sent from my iPhone
>> > > >>
>> > > >>> On 07-May-2022, at 7:12 AM, Chao Sun  wrote:
>> > > >>>
>> > > >>> 
>> > > 
>> > >  Chao can you use the one that Steve mentioned in the mail?
>> > > >>>
>> > > >>> Hmm how do I do that? Typically after closing the RC in nexus the
>> > > >>> release bits will show up in
>> > > >>>
>> > > >>
>> > >
>> https://repository.apache.org/content/repositories/staging/org/apache/hadoop
>> > > >>> and Spark build will be able to pick them up for testing. However
>> in
>> > > >>> this case I don't see any 3.3.3 jars in the URL.
>> > > >>>
>> > >  On Fri, May 6, 2022 at 6:24 PM Ayush Saxena 
>> > > wrote:
>> > > 
>> > >  There were two 3.3.3 staged. The earlier one was with skipShade,
>> the
>> > > >> date was also april 22, I archived that. Chao can you use the one
>> that
>> > > >> Steve mentioned in the mail?
>> > > 
>> > > > On Sat, 7 May 2022 at 06:18, Chao Sun 
>> wrote:
>> > > >
>> > > > Seems there are some issues with the shaded client as I was not
>> able
>> > > > to compile Apache Spark with the RC
>> > > > (https://github.com/apache/spark/pull/36474). Looks like it's
>> > > compiled
>> > > > with the `-DskipShade` option and the hadoop-client-api JAR
>> doesn't
>> > > > contain any class:
>> > > >
>> > > 

Re: [VOTE] Release Apache Hadoop 3.3.3

[Steve Loughran]

I've done another docker build and the client jars appear to be there. I'll
test tomorrow before putting up another vote. it will be exactly the same
commit as before, just recompiled/republished

On Mon, 9 May 2022 at 17:45, Chao Sun  wrote:

> Agreed, that step #10 is out-dated and should be removed (I skipped
> that when releasing Hadoop 3.3.2 but didn't update it, sorry).
>
> > How about using
> https://repository.apache.org/content/repositories/orgapachehadoop-1348/
>
> Akira, I tried this too but it didn't work. I think we'd need the
> artifacts to be properly pushed to the staging repository.
>
> > Could you please let me know how I can consume the Hadoop 3 jars in
> maven?
>
> Gautham (if you are following this thread), you'll need to add the
> following:
>
> 
>staged
>staged-releases
>https://repository.apache.org/content/repositories/staging/
> 
>
>  true
>
>
>  true
>
>  
>
> to the `` section in your Maven pom.xml file.
>
> On Mon, May 9, 2022 at 8:52 AM Steve Loughran
>  wrote:
> >
> > I didn't do that as the docker image was doing it itself...I discussed
> this
> > with Akira and Ayush & they agreed. so whatever went wrong. it was
> > something else.
> >
> > I have been building a list of things I'd like to change there; cutting
> > that line was one of them. but I need to work out the correct workflow.
> >
> > trying again, and creating a stub module to verify the client is in
> staging
> >
> > On Mon, 9 May 2022 at 15:19, Masatake Iwasaki <
> iwasak...@oss.nttdata.co.jp>
> > wrote:
> >
> > > It seems to be caused by obsolete instruction in HowToRelease Wiki?
> > >
> > > After HADOOP-15058, `mvn deploy` is kicked by
> > > `dev-support/bin/create-release --asfrelease`.
> > > https://issues.apache.org/jira/browse/HADOOP-15058
> > >
> > > Step #10 in "Creating the release candidate (X.Y.Z-RC)" section
> > > of the Wiki still instructs to run `mvn deploy` with `-DskipShade`.
> > >
> > > 2 sets of artifact are deployed after creating RC based on the
> instruction.
> > > The latest one contains empty shaded jars.
> > >
> > > hadoop-client-api and hadoop-client-runtime of already released 3.2.3
> > > looks having same issue...
> > >
> > > Masatake Iwasaki
> > >
> > > On 2022/05/08 6:45, Akira Ajisaka wrote:
> > > > Hi Chao,
> > > >
> > > > How about using
> > > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1348/
> > > > instead of
> https://repository.apache.org/content/repositories/staging/ ?
> > > >
> > > > Akira
> > > >
> > > > On Sat, May 7, 2022 at 10:52 AM Ayush Saxena 
> wrote:
> > > >
> > > >> Hmm, I see the artifacts ideally should have got overwritten by the
> new
> > > >> RC, but they didn’t. The reason seems like the staging path shared
> > > doesn’t
> > > >> have any jars…
> > > >> That is why it was picking the old jars. I think Steve needs to run
> mvn
> > > >> deploy again…
> > > >>
> > > >> Sent from my iPhone
> > > >>
> > > >>> On 07-May-2022, at 7:12 AM, Chao Sun  wrote:
> > > >>>
> > > >>> 
> > > 
> > >  Chao can you use the one that Steve mentioned in the mail?
> > > >>>
> > > >>> Hmm how do I do that? Typically after closing the RC in nexus the
> > > >>> release bits will show up in
> > > >>>
> > > >>
> > >
> https://repository.apache.org/content/repositories/staging/org/apache/hadoop
> > > >>> and Spark build will be able to pick them up for testing. However
> in
> > > >>> this case I don't see any 3.3.3 jars in the URL.
> > > >>>
> > >  On Fri, May 6, 2022 at 6:24 PM Ayush Saxena 
> > > wrote:
> > > 
> > >  There were two 3.3.3 staged. The earlier one was with skipShade,
> the
> > > >> date was also april 22, I archived that. Chao can you use the one
> that
> > > >> Steve mentioned in the mail?
> > > 
> > > > On Sat, 7 May 2022 at 06:18, Chao Sun 
> wrote:
> > > >
> > > > Seems there are some issues with the shaded client as I was not
> able
> > > > to compile Apache Spark with the RC
> > > > (https://github.com/apache/spark/pull/36474). Looks like it's
> > > compiled
> > > > with the `-DskipShade` option and the hadoop-client-api JAR
> doesn't
> > > > contain any class:
> > > >
> > > > ➜  hadoop-client-api jar tf 3.3.3/hadoop-client-api-3.3.3.jar
> > > > META-INF/
> > > > META-INF/MANIFEST.MF
> > > > META-INF/NOTICE.txt
> > > > META-INF/LICENSE.txt
> > > > META-INF/maven/
> > > > META-INF/maven/org.apache.hadoop/
> > > > META-INF/maven/org.apache.hadoop/hadoop-client-api/
> > > > META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.xml
> > > > META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.properties
> > > >
> > > > On Fri, May 6, 2022 at 4:24 PM Stack  wrote:
> > > >>
> > > >> +1 (binding)
> > > >>
> > > >>   * Signature: ok
> > > >>   * Checksum : passed
> > > >>   * Rat check (1.8.0_191): passed
> > > >>- mvn clean 

Re: [VOTE] Release Apache Hadoop 3.3.3

[Chao Sun]

Agreed, that step #10 is out-dated and should be removed (I skipped
that when releasing Hadoop 3.3.2 but didn't update it, sorry).

> How about using 
> https://repository.apache.org/content/repositories/orgapachehadoop-1348/

Akira, I tried this too but it didn't work. I think we'd need the
artifacts to be properly pushed to the staging repository.

> Could you please let me know how I can consume the Hadoop 3 jars in maven?

Gautham (if you are following this thread), you'll need to add the following:


   staged
   staged-releases
   https://repository.apache.org/content/repositories/staging/
   
 true
   
   
 true
   
 

to the `` section in your Maven pom.xml file.

On Mon, May 9, 2022 at 8:52 AM Steve Loughran
 wrote:
>
> I didn't do that as the docker image was doing it itself...I discussed this
> with Akira and Ayush & they agreed. so whatever went wrong. it was
> something else.
>
> I have been building a list of things I'd like to change there; cutting
> that line was one of them. but I need to work out the correct workflow.
>
> trying again, and creating a stub module to verify the client is in staging
>
> On Mon, 9 May 2022 at 15:19, Masatake Iwasaki 
> wrote:
>
> > It seems to be caused by obsolete instruction in HowToRelease Wiki?
> >
> > After HADOOP-15058, `mvn deploy` is kicked by
> > `dev-support/bin/create-release --asfrelease`.
> > https://issues.apache.org/jira/browse/HADOOP-15058
> >
> > Step #10 in "Creating the release candidate (X.Y.Z-RC)" section
> > of the Wiki still instructs to run `mvn deploy` with `-DskipShade`.
> >
> > 2 sets of artifact are deployed after creating RC based on the instruction.
> > The latest one contains empty shaded jars.
> >
> > hadoop-client-api and hadoop-client-runtime of already released 3.2.3
> > looks having same issue...
> >
> > Masatake Iwasaki
> >
> > On 2022/05/08 6:45, Akira Ajisaka wrote:
> > > Hi Chao,
> > >
> > > How about using
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1348/
> > > instead of https://repository.apache.org/content/repositories/staging/ ?
> > >
> > > Akira
> > >
> > > On Sat, May 7, 2022 at 10:52 AM Ayush Saxena  wrote:
> > >
> > >> Hmm, I see the artifacts ideally should have got overwritten by the new
> > >> RC, but they didn’t. The reason seems like the staging path shared
> > doesn’t
> > >> have any jars…
> > >> That is why it was picking the old jars. I think Steve needs to run mvn
> > >> deploy again…
> > >>
> > >> Sent from my iPhone
> > >>
> > >>> On 07-May-2022, at 7:12 AM, Chao Sun  wrote:
> > >>>
> > >>> 
> > 
> >  Chao can you use the one that Steve mentioned in the mail?
> > >>>
> > >>> Hmm how do I do that? Typically after closing the RC in nexus the
> > >>> release bits will show up in
> > >>>
> > >>
> > https://repository.apache.org/content/repositories/staging/org/apache/hadoop
> > >>> and Spark build will be able to pick them up for testing. However in
> > >>> this case I don't see any 3.3.3 jars in the URL.
> > >>>
> >  On Fri, May 6, 2022 at 6:24 PM Ayush Saxena 
> > wrote:
> > 
> >  There were two 3.3.3 staged. The earlier one was with skipShade, the
> > >> date was also april 22, I archived that. Chao can you use the one that
> > >> Steve mentioned in the mail?
> > 
> > > On Sat, 7 May 2022 at 06:18, Chao Sun  wrote:
> > >
> > > Seems there are some issues with the shaded client as I was not able
> > > to compile Apache Spark with the RC
> > > (https://github.com/apache/spark/pull/36474). Looks like it's
> > compiled
> > > with the `-DskipShade` option and the hadoop-client-api JAR doesn't
> > > contain any class:
> > >
> > > ➜  hadoop-client-api jar tf 3.3.3/hadoop-client-api-3.3.3.jar
> > > META-INF/
> > > META-INF/MANIFEST.MF
> > > META-INF/NOTICE.txt
> > > META-INF/LICENSE.txt
> > > META-INF/maven/
> > > META-INF/maven/org.apache.hadoop/
> > > META-INF/maven/org.apache.hadoop/hadoop-client-api/
> > > META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.xml
> > > META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.properties
> > >
> > > On Fri, May 6, 2022 at 4:24 PM Stack  wrote:
> > >>
> > >> +1 (binding)
> > >>
> > >>   * Signature: ok
> > >>   * Checksum : passed
> > >>   * Rat check (1.8.0_191): passed
> > >>- mvn clean apache-rat:check
> > >>   * Built from source (1.8.0_191): failed
> > >>- mvn clean install  -DskipTests
> > >>- mvn -fae --no-transfer-progress -DskipTests
> > >> -Dmaven.javadoc.skip=true
> > >> -Pnative -Drequire.openssl -Drequire.snappy -Drequire.valgrind
> > >> -Drequire.zstd -Drequire.test.libhadoop clean install
> > >>   * Unit tests pass (1.8.0_191):
> > >> - HDFS Tests passed (Didn't run more than this).
> > >>
> > >> Deployed a ten node ha hdfs cluster with three namenodes and five
> > >> 

Re: [VOTE] Release Apache Hadoop 3.3.3

[Steve Loughran]

I didn't do that as the docker image was doing it itself...I discussed this
with Akira and Ayush & they agreed. so whatever went wrong. it was
something else.

I have been building a list of things I'd like to change there; cutting
that line was one of them. but I need to work out the correct workflow.

trying again, and creating a stub module to verify the client is in staging

On Mon, 9 May 2022 at 15:19, Masatake Iwasaki 
wrote:

> It seems to be caused by obsolete instruction in HowToRelease Wiki?
>
> After HADOOP-15058, `mvn deploy` is kicked by
> `dev-support/bin/create-release --asfrelease`.
> https://issues.apache.org/jira/browse/HADOOP-15058
>
> Step #10 in "Creating the release candidate (X.Y.Z-RC)" section
> of the Wiki still instructs to run `mvn deploy` with `-DskipShade`.
>
> 2 sets of artifact are deployed after creating RC based on the instruction.
> The latest one contains empty shaded jars.
>
> hadoop-client-api and hadoop-client-runtime of already released 3.2.3
> looks having same issue...
>
> Masatake Iwasaki
>
> On 2022/05/08 6:45, Akira Ajisaka wrote:
> > Hi Chao,
> >
> > How about using
> > https://repository.apache.org/content/repositories/orgapachehadoop-1348/
> > instead of https://repository.apache.org/content/repositories/staging/ ?
> >
> > Akira
> >
> > On Sat, May 7, 2022 at 10:52 AM Ayush Saxena  wrote:
> >
> >> Hmm, I see the artifacts ideally should have got overwritten by the new
> >> RC, but they didn’t. The reason seems like the staging path shared
> doesn’t
> >> have any jars…
> >> That is why it was picking the old jars. I think Steve needs to run mvn
> >> deploy again…
> >>
> >> Sent from my iPhone
> >>
> >>> On 07-May-2022, at 7:12 AM, Chao Sun  wrote:
> >>>
> >>> 
> 
>  Chao can you use the one that Steve mentioned in the mail?
> >>>
> >>> Hmm how do I do that? Typically after closing the RC in nexus the
> >>> release bits will show up in
> >>>
> >>
> https://repository.apache.org/content/repositories/staging/org/apache/hadoop
> >>> and Spark build will be able to pick them up for testing. However in
> >>> this case I don't see any 3.3.3 jars in the URL.
> >>>
>  On Fri, May 6, 2022 at 6:24 PM Ayush Saxena 
> wrote:
> 
>  There were two 3.3.3 staged. The earlier one was with skipShade, the
> >> date was also april 22, I archived that. Chao can you use the one that
> >> Steve mentioned in the mail?
> 
> > On Sat, 7 May 2022 at 06:18, Chao Sun  wrote:
> >
> > Seems there are some issues with the shaded client as I was not able
> > to compile Apache Spark with the RC
> > (https://github.com/apache/spark/pull/36474). Looks like it's
> compiled
> > with the `-DskipShade` option and the hadoop-client-api JAR doesn't
> > contain any class:
> >
> > ➜  hadoop-client-api jar tf 3.3.3/hadoop-client-api-3.3.3.jar
> > META-INF/
> > META-INF/MANIFEST.MF
> > META-INF/NOTICE.txt
> > META-INF/LICENSE.txt
> > META-INF/maven/
> > META-INF/maven/org.apache.hadoop/
> > META-INF/maven/org.apache.hadoop/hadoop-client-api/
> > META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.xml
> > META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.properties
> >
> > On Fri, May 6, 2022 at 4:24 PM Stack  wrote:
> >>
> >> +1 (binding)
> >>
> >>   * Signature: ok
> >>   * Checksum : passed
> >>   * Rat check (1.8.0_191): passed
> >>- mvn clean apache-rat:check
> >>   * Built from source (1.8.0_191): failed
> >>- mvn clean install  -DskipTests
> >>- mvn -fae --no-transfer-progress -DskipTests
> >> -Dmaven.javadoc.skip=true
> >> -Pnative -Drequire.openssl -Drequire.snappy -Drequire.valgrind
> >> -Drequire.zstd -Drequire.test.libhadoop clean install
> >>   * Unit tests pass (1.8.0_191):
> >> - HDFS Tests passed (Didn't run more than this).
> >>
> >> Deployed a ten node ha hdfs cluster with three namenodes and five
> >> journalnodes. Ran a ten node hbase (older version of 2.5 branch
> built
> >> against 3.3.2) against it. Tried a small verification job. Good.
> Ran a
> >> bigger job with mild chaos. All seems to be working properly
> >> (recoveries,
> >> logs look fine). Killed a namenode. Failover worked promptly. UIs
> look
> >> good. Poked at the hdfs cli. Seems good.
> >>
> >> S
> >>
> >> On Tue, May 3, 2022 at 4:24 AM Steve Loughran
> >> 
> >> wrote:
> >>
> >>> I have put together a release candidate (rc0) for Hadoop 3.3.3
> >>>
> >>> The RC is available at:
> >>> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/
> >>>
> >>> The git tag is release-3.3.3-RC0, commit d37586cbda3
> >>>
> >>> The maven artifacts are staged at
> >>>
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1348/
> >>>
> >>> You can find my public key at:
> >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS

Re: [VOTE] Release Apache Hadoop 3.3.3

[Masatake Iwasaki]

Step #10 in "Creating the release candidate (X.Y.Z-RC)" section
of the Wiki still instructs to run `mvn deploy` with `-DskipShade`.


https://cwiki.apache.org/confluence/display/HADOOP2/HowToRelease#HowToRelease-Creatingthereleasecandidate(X.Y.Z-RC%3CN%3E)

On 2022/05/09 23:18, Masatake Iwasaki wrote:

It seems to be caused by obsolete instruction in HowToRelease Wiki?

After HADOOP-15058, `mvn deploy` is kicked by
`dev-support/bin/create-release --asfrelease`.
https://issues.apache.org/jira/browse/HADOOP-15058

Step #10 in "Creating the release candidate (X.Y.Z-RC)" section
of the Wiki still instructs to run `mvn deploy` with `-DskipShade`.

2 sets of artifact are deployed after creating RC based on the instruction.
The latest one contains empty shaded jars.

hadoop-client-api and hadoop-client-runtime of already released 3.2.3
looks having same issue...

Masatake Iwasaki

On 2022/05/08 6:45, Akira Ajisaka wrote:

Hi Chao,

How about using
https://repository.apache.org/content/repositories/orgapachehadoop-1348/
instead of https://repository.apache.org/content/repositories/staging/ ?

Akira

On Sat, May 7, 2022 at 10:52 AM Ayush Saxena  wrote:


Hmm, I see the artifacts ideally should have got overwritten by the new
RC, but they didn’t. The reason seems like the staging path shared doesn’t
have any jars…
That is why it was picking the old jars. I think Steve needs to run mvn
deploy again…

Sent from my iPhone


On 07-May-2022, at 7:12 AM, Chao Sun  wrote:




Chao can you use the one that Steve mentioned in the mail?


Hmm how do I do that? Typically after closing the RC in nexus the
release bits will show up in


https://repository.apache.org/content/repositories/staging/org/apache/hadoop

and Spark build will be able to pick them up for testing. However in
this case I don't see any 3.3.3 jars in the URL.


On Fri, May 6, 2022 at 6:24 PM Ayush Saxena  wrote:

There were two 3.3.3 staged. The earlier one was with skipShade, the

date was also april 22, I archived that. Chao can you use the one that
Steve mentioned in the mail?



On Sat, 7 May 2022 at 06:18, Chao Sun  wrote:

Seems there are some issues with the shaded client as I was not able
to compile Apache Spark with the RC
(https://github.com/apache/spark/pull/36474). Looks like it's compiled
with the `-DskipShade` option and the hadoop-client-api JAR doesn't
contain any class:

➜  hadoop-client-api jar tf 3.3.3/hadoop-client-api-3.3.3.jar
META-INF/
META-INF/MANIFEST.MF
META-INF/NOTICE.txt
META-INF/LICENSE.txt
META-INF/maven/
META-INF/maven/org.apache.hadoop/
META-INF/maven/org.apache.hadoop/hadoop-client-api/
META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.xml
META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.properties

On Fri, May 6, 2022 at 4:24 PM Stack  wrote:


+1 (binding)

  * Signature: ok
  * Checksum : passed
  * Rat check (1.8.0_191): passed
   - mvn clean apache-rat:check
  * Built from source (1.8.0_191): failed
   - mvn clean install  -DskipTests
   - mvn -fae --no-transfer-progress -DskipTests

-Dmaven.javadoc.skip=true

-Pnative -Drequire.openssl -Drequire.snappy -Drequire.valgrind
-Drequire.zstd -Drequire.test.libhadoop clean install
  * Unit tests pass (1.8.0_191):
    - HDFS Tests passed (Didn't run more than this).

Deployed a ten node ha hdfs cluster with three namenodes and five
journalnodes. Ran a ten node hbase (older version of 2.5 branch built
against 3.3.2) against it. Tried a small verification job. Good. Ran a
bigger job with mild chaos. All seems to be working properly

(recoveries,

logs look fine). Killed a namenode. Failover worked promptly. UIs look
good. Poked at the hdfs cli. Seems good.

S

On Tue, May 3, 2022 at 4:24 AM Steve Loughran



wrote:


I have put together a release candidate (rc0) for Hadoop 3.3.3

The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/

The git tag is release-3.3.3-RC0, commit d37586cbda3

The maven artifacts are staged at


https://repository.apache.org/content/repositories/orgapachehadoop-1348/


You can find my public key at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS

Change log
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/CHANGELOG.md

Release notes


https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/RELEASENOTES.md


There's a very small number of changes, primarily critical

code/packaging

issues and security fixes.


   - The critical fixes which shipped in the 3.2.3 release.
   -  CVEs in our code and dependencies
   - Shaded client packaging issues.
   - A switch from log4j to reload4j


reload4j is an active fork of the log4j 1.17 library with the

classes which

contain CVEs removed. Even though hadoop never used those classes,

they

regularly raised alerts on security scans and concen from users.

Switching

to the forked project allows us to ship a secure logging framework.

It will

complicate the builds of downstream maven/ivy/gradle projects which

exclude

our log4j artifacts, as they 

Re: [VOTE] Release Apache Hadoop 3.3.3

[Masatake Iwasaki]

It seems to be caused by obsolete instruction in HowToRelease Wiki?

After HADOOP-15058, `mvn deploy` is kicked by
`dev-support/bin/create-release --asfrelease`.
https://issues.apache.org/jira/browse/HADOOP-15058

Step #10 in "Creating the release candidate (X.Y.Z-RC)" section
of the Wiki still instructs to run `mvn deploy` with `-DskipShade`.

2 sets of artifact are deployed after creating RC based on the instruction.
The latest one contains empty shaded jars.

hadoop-client-api and hadoop-client-runtime of already released 3.2.3
looks having same issue...

Masatake Iwasaki

On 2022/05/08 6:45, Akira Ajisaka wrote:

Hi Chao,

How about using
https://repository.apache.org/content/repositories/orgapachehadoop-1348/
instead of https://repository.apache.org/content/repositories/staging/ ?

Akira

On Sat, May 7, 2022 at 10:52 AM Ayush Saxena  wrote:


Hmm, I see the artifacts ideally should have got overwritten by the new
RC, but they didn’t. The reason seems like the staging path shared doesn’t
have any jars…
That is why it was picking the old jars. I think Steve needs to run mvn
deploy again…

Sent from my iPhone


On 07-May-2022, at 7:12 AM, Chao Sun  wrote:




Chao can you use the one that Steve mentioned in the mail?


Hmm how do I do that? Typically after closing the RC in nexus the
release bits will show up in


https://repository.apache.org/content/repositories/staging/org/apache/hadoop

and Spark build will be able to pick them up for testing. However in
this case I don't see any 3.3.3 jars in the URL.


On Fri, May 6, 2022 at 6:24 PM Ayush Saxena  wrote:

There were two 3.3.3 staged. The earlier one was with skipShade, the

date was also april 22, I archived that. Chao can you use the one that
Steve mentioned in the mail?



On Sat, 7 May 2022 at 06:18, Chao Sun  wrote:

Seems there are some issues with the shaded client as I was not able
to compile Apache Spark with the RC
(https://github.com/apache/spark/pull/36474). Looks like it's compiled
with the `-DskipShade` option and the hadoop-client-api JAR doesn't
contain any class:

➜  hadoop-client-api jar tf 3.3.3/hadoop-client-api-3.3.3.jar
META-INF/
META-INF/MANIFEST.MF
META-INF/NOTICE.txt
META-INF/LICENSE.txt
META-INF/maven/
META-INF/maven/org.apache.hadoop/
META-INF/maven/org.apache.hadoop/hadoop-client-api/
META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.xml
META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.properties

On Fri, May 6, 2022 at 4:24 PM Stack  wrote:


+1 (binding)

  * Signature: ok
  * Checksum : passed
  * Rat check (1.8.0_191): passed
   - mvn clean apache-rat:check
  * Built from source (1.8.0_191): failed
   - mvn clean install  -DskipTests
   - mvn -fae --no-transfer-progress -DskipTests

-Dmaven.javadoc.skip=true

-Pnative -Drequire.openssl -Drequire.snappy -Drequire.valgrind
-Drequire.zstd -Drequire.test.libhadoop clean install
  * Unit tests pass (1.8.0_191):
- HDFS Tests passed (Didn't run more than this).

Deployed a ten node ha hdfs cluster with three namenodes and five
journalnodes. Ran a ten node hbase (older version of 2.5 branch built
against 3.3.2) against it. Tried a small verification job. Good. Ran a
bigger job with mild chaos. All seems to be working properly

(recoveries,

logs look fine). Killed a namenode. Failover worked promptly. UIs look
good. Poked at the hdfs cli. Seems good.

S

On Tue, May 3, 2022 at 4:24 AM Steve Loughran



wrote:


I have put together a release candidate (rc0) for Hadoop 3.3.3

The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/

The git tag is release-3.3.3-RC0, commit d37586cbda3

The maven artifacts are staged at


https://repository.apache.org/content/repositories/orgapachehadoop-1348/


You can find my public key at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS

Change log
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/CHANGELOG.md

Release notes


https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/RELEASENOTES.md


There's a very small number of changes, primarily critical

code/packaging

issues and security fixes.


   - The critical fixes which shipped in the 3.2.3 release.
   -  CVEs in our code and dependencies
   - Shaded client packaging issues.
   - A switch from log4j to reload4j


reload4j is an active fork of the log4j 1.17 library with the

classes which

contain CVEs removed. Even though hadoop never used those classes,

they

regularly raised alerts on security scans and concen from users.

Switching

to the forked project allows us to ship a secure logging framework.

It will

complicate the builds of downstream maven/ivy/gradle projects which

exclude

our log4j artifacts, as they need to cut the new dependency

instead/as

well.

See the release notes for details.

This is my first release through the new docker build process, do

please

validate artifact signing  to make sure it is good. I'll be trying

builds

of downstream projects.

We know there are some outstanding issues 

Re: [VOTE] Release Apache Hadoop 3.3.3

[Steve Loughran]

had a report of that staging playing up too from Gautham

I'm going to have to treat this as a failure from the maven deploy process,
so I'll have to cancel this RC and try and work out WTF went wrong.

I'll also put together some minimal apps to fetch and compile against
everything. i did do some myself (storediag, hboss), and did try to make
sure I had cleaned up my local repo first, but maybe it was tainted.

oh well

---

However, I get this error saying that it can’t find the maven plugins and
jars for Hadoop 3.3.3 –

[INFO] --< org.apache.hadoop:hadoop-common
>---

[INFO] Building Apache Hadoop Common 3.3.3
[5/9]

[INFO] [ jar
]-

Downloading from central:
https://repository.apache.org/content/repositories/orgapachehadoop-1348/org/apache/hadoop/hadoop-maven-plugins/3.3.3/hadoop-maven-plugins-3.3.3.pom

[WARNING] The POM for org.apache.hadoop:hadoop-maven-plugins:jar:3.3.3 is
missing, no dependency information available

Downloading from central:
https://repository.apache.org/content/repositories/orgapachehadoop-1348/org/apache/hadoop/hadoop-maven-plugins/3.3.3/hadoop-maven-plugins-3.3.3.jar

[INFO]


[INFO] Reactor Summary for Apache Hadoop Common Project 3.3.3:

[INFO]

[INFO] Apache Hadoop Annotations .. SUCCESS [
7.349 s]

[INFO] Apache Hadoop MiniKDC .. SUCCESS [
1.810 s]

[INFO] Apache Hadoop Auth . SUCCESS [
7.922 s]

[INFO] Apache Hadoop Auth Examples  SUCCESS [
0.375 s]

[INFO] Apache Hadoop Common ... FAILURE [
2.095 s]

[INFO] Apache Hadoop NFS .. SKIPPED

[INFO] Apache Hadoop KMS .. SKIPPED

[INFO] Apache Hadoop Registry . SKIPPED

[INFO] Apache Hadoop Common Project ... SKIPPED

[INFO]


[INFO] BUILD FAILURE

[INFO]


[INFO] Total time:  24.946 s

[INFO] Finished at: 2022-05-07T11:29:18+05:30

[INFO]


[ERROR] Plugin org.apache.hadoop:hadoop-maven-plugins:3.3.3 or one of its
dependencies could not be resolved: Could not find artifact
org.apache.hadoop:hadoop-maven-plugins:jar:3.3.3 in central (
https://repository.apache.org/content/repositories/orgapachehadoop-1348/)
-> [Help 1]

[ERROR]



Could you please let me know how I can consume the Hadoop 3 jars in maven?



Thanks,

--Gautham

On Sat, 7 May 2022 at 22:45, Akira Ajisaka  wrote:

> Hi Chao,
>
> How about using
> https://repository.apache.org/content/repositories/orgapachehadoop-1348/
> instead of https://repository.apache.org/content/repositories/staging/ ?
>
> Akira
>
> On Sat, May 7, 2022 at 10:52 AM Ayush Saxena  wrote:
>
> > Hmm, I see the artifacts ideally should have got overwritten by the new
> > RC, but they didn’t. The reason seems like the staging path shared
> doesn’t
> > have any jars…
> > That is why it was picking the old jars. I think Steve needs to run mvn
> > deploy again…
> >
> > Sent from my iPhone
> >
> > > On 07-May-2022, at 7:12 AM, Chao Sun  wrote:
> > >
> > > 
> > >>
> > >> Chao can you use the one that Steve mentioned in the mail?
> > >
> > > Hmm how do I do that? Typically after closing the RC in nexus the
> > > release bits will show up in
> > >
> >
> https://repository.apache.org/content/repositories/staging/org/apache/hadoop
> > > and Spark build will be able to pick them up for testing. However in
> > > this case I don't see any 3.3.3 jars in the URL.
> > >
> > >> On Fri, May 6, 2022 at 6:24 PM Ayush Saxena 
> wrote:
> > >>
> > >> There were two 3.3.3 staged. The earlier one was with skipShade, the
> > date was also april 22, I archived that. Chao can you use the one that
> > Steve mentioned in the mail?
> > >>
> > >>> On Sat, 7 May 2022 at 06:18, Chao Sun  wrote:
> > >>>
> > >>> Seems there are some issues with the shaded client as I was not able
> > >>> to compile Apache Spark with the RC
> > >>> (https://github.com/apache/spark/pull/36474). Looks like it's
> compiled
> > >>> with the `-DskipShade` option and the hadoop-client-api JAR doesn't
> > >>> contain any class:
> > >>>
> > >>> ➜  hadoop-client-api jar tf 3.3.3/hadoop-client-api-3.3.3.jar
> > >>> META-INF/
> > >>> META-INF/MANIFEST.MF
> > >>> META-INF/NOTICE.txt
> > >>> META-INF/LICENSE.txt
> > >>> META-INF/maven/
> > >>> META-INF/maven/org.apache.hadoop/
> > >>> META-INF/maven/org.apache.hadoop/hadoop-client-api/
> > >>> META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.xml
> > >>> META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.properties
> > >>>
> > >>> On Fri, May 6, 2022 at 4:24 PM Stack  wrote:
> > 

Re: [VOTE] Release Apache Hadoop 3.3.3

[Akira Ajisaka]

Hi Chao,

How about using
https://repository.apache.org/content/repositories/orgapachehadoop-1348/
instead of https://repository.apache.org/content/repositories/staging/ ?

Akira

On Sat, May 7, 2022 at 10:52 AM Ayush Saxena  wrote:

> Hmm, I see the artifacts ideally should have got overwritten by the new
> RC, but they didn’t. The reason seems like the staging path shared doesn’t
> have any jars…
> That is why it was picking the old jars. I think Steve needs to run mvn
> deploy again…
>
> Sent from my iPhone
>
> > On 07-May-2022, at 7:12 AM, Chao Sun  wrote:
> >
> > 
> >>
> >> Chao can you use the one that Steve mentioned in the mail?
> >
> > Hmm how do I do that? Typically after closing the RC in nexus the
> > release bits will show up in
> >
> https://repository.apache.org/content/repositories/staging/org/apache/hadoop
> > and Spark build will be able to pick them up for testing. However in
> > this case I don't see any 3.3.3 jars in the URL.
> >
> >> On Fri, May 6, 2022 at 6:24 PM Ayush Saxena  wrote:
> >>
> >> There were two 3.3.3 staged. The earlier one was with skipShade, the
> date was also april 22, I archived that. Chao can you use the one that
> Steve mentioned in the mail?
> >>
> >>> On Sat, 7 May 2022 at 06:18, Chao Sun  wrote:
> >>>
> >>> Seems there are some issues with the shaded client as I was not able
> >>> to compile Apache Spark with the RC
> >>> (https://github.com/apache/spark/pull/36474). Looks like it's compiled
> >>> with the `-DskipShade` option and the hadoop-client-api JAR doesn't
> >>> contain any class:
> >>>
> >>> ➜  hadoop-client-api jar tf 3.3.3/hadoop-client-api-3.3.3.jar
> >>> META-INF/
> >>> META-INF/MANIFEST.MF
> >>> META-INF/NOTICE.txt
> >>> META-INF/LICENSE.txt
> >>> META-INF/maven/
> >>> META-INF/maven/org.apache.hadoop/
> >>> META-INF/maven/org.apache.hadoop/hadoop-client-api/
> >>> META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.xml
> >>> META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.properties
> >>>
> >>> On Fri, May 6, 2022 at 4:24 PM Stack  wrote:
> 
>  +1 (binding)
> 
>   * Signature: ok
>   * Checksum : passed
>   * Rat check (1.8.0_191): passed
>    - mvn clean apache-rat:check
>   * Built from source (1.8.0_191): failed
>    - mvn clean install  -DskipTests
>    - mvn -fae --no-transfer-progress -DskipTests
> -Dmaven.javadoc.skip=true
>  -Pnative -Drequire.openssl -Drequire.snappy -Drequire.valgrind
>  -Drequire.zstd -Drequire.test.libhadoop clean install
>   * Unit tests pass (1.8.0_191):
> - HDFS Tests passed (Didn't run more than this).
> 
>  Deployed a ten node ha hdfs cluster with three namenodes and five
>  journalnodes. Ran a ten node hbase (older version of 2.5 branch built
>  against 3.3.2) against it. Tried a small verification job. Good. Ran a
>  bigger job with mild chaos. All seems to be working properly
> (recoveries,
>  logs look fine). Killed a namenode. Failover worked promptly. UIs look
>  good. Poked at the hdfs cli. Seems good.
> 
>  S
> 
>  On Tue, May 3, 2022 at 4:24 AM Steve Loughran
> 
>  wrote:
> 
> > I have put together a release candidate (rc0) for Hadoop 3.3.3
> >
> > The RC is available at:
> > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/
> >
> > The git tag is release-3.3.3-RC0, commit d37586cbda3
> >
> > The maven artifacts are staged at
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1348/
> >
> > You can find my public key at:
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> > Change log
> > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/CHANGELOG.md
> >
> > Release notes
> >
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/RELEASENOTES.md
> >
> > There's a very small number of changes, primarily critical
> code/packaging
> > issues and security fixes.
> >
> >
> >   - The critical fixes which shipped in the 3.2.3 release.
> >   -  CVEs in our code and dependencies
> >   - Shaded client packaging issues.
> >   - A switch from log4j to reload4j
> >
> >
> > reload4j is an active fork of the log4j 1.17 library with the
> classes which
> > contain CVEs removed. Even though hadoop never used those classes,
> they
> > regularly raised alerts on security scans and concen from users.
> Switching
> > to the forked project allows us to ship a secure logging framework.
> It will
> > complicate the builds of downstream maven/ivy/gradle projects which
> exclude
> > our log4j artifacts, as they need to cut the new dependency
> instead/as
> > well.
> >
> > See the release notes for details.
> >
> > This is my first release through the new docker build process, do
> please
> > validate artifact signing  to make sure it is good. I'll be trying
> builds
> > of downstream 

Re: [VOTE] Release Apache Hadoop 3.3.3

[Ayush Saxena]

Hmm, I see the artifacts ideally should have got overwritten by the new RC, but 
they didn’t. The reason seems like the staging path shared doesn’t have any 
jars…
That is why it was picking the old jars. I think Steve needs to run mvn deploy 
again…

Sent from my iPhone

> On 07-May-2022, at 7:12 AM, Chao Sun  wrote:
> 
> 
>> 
>> Chao can you use the one that Steve mentioned in the mail?
> 
> Hmm how do I do that? Typically after closing the RC in nexus the
> release bits will show up in
> https://repository.apache.org/content/repositories/staging/org/apache/hadoop
> and Spark build will be able to pick them up for testing. However in
> this case I don't see any 3.3.3 jars in the URL.
> 
>> On Fri, May 6, 2022 at 6:24 PM Ayush Saxena  wrote:
>> 
>> There were two 3.3.3 staged. The earlier one was with skipShade, the date 
>> was also april 22, I archived that. Chao can you use the one that Steve 
>> mentioned in the mail?
>> 
>>> On Sat, 7 May 2022 at 06:18, Chao Sun  wrote:
>>> 
>>> Seems there are some issues with the shaded client as I was not able
>>> to compile Apache Spark with the RC
>>> (https://github.com/apache/spark/pull/36474). Looks like it's compiled
>>> with the `-DskipShade` option and the hadoop-client-api JAR doesn't
>>> contain any class:
>>> 
>>> ➜  hadoop-client-api jar tf 3.3.3/hadoop-client-api-3.3.3.jar
>>> META-INF/
>>> META-INF/MANIFEST.MF
>>> META-INF/NOTICE.txt
>>> META-INF/LICENSE.txt
>>> META-INF/maven/
>>> META-INF/maven/org.apache.hadoop/
>>> META-INF/maven/org.apache.hadoop/hadoop-client-api/
>>> META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.xml
>>> META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.properties
>>> 
>>> On Fri, May 6, 2022 at 4:24 PM Stack  wrote:
 
 +1 (binding)
 
  * Signature: ok
  * Checksum : passed
  * Rat check (1.8.0_191): passed
   - mvn clean apache-rat:check
  * Built from source (1.8.0_191): failed
   - mvn clean install  -DskipTests
   - mvn -fae --no-transfer-progress -DskipTests -Dmaven.javadoc.skip=true
 -Pnative -Drequire.openssl -Drequire.snappy -Drequire.valgrind
 -Drequire.zstd -Drequire.test.libhadoop clean install
  * Unit tests pass (1.8.0_191):
- HDFS Tests passed (Didn't run more than this).
 
 Deployed a ten node ha hdfs cluster with three namenodes and five
 journalnodes. Ran a ten node hbase (older version of 2.5 branch built
 against 3.3.2) against it. Tried a small verification job. Good. Ran a
 bigger job with mild chaos. All seems to be working properly (recoveries,
 logs look fine). Killed a namenode. Failover worked promptly. UIs look
 good. Poked at the hdfs cli. Seems good.
 
 S
 
 On Tue, May 3, 2022 at 4:24 AM Steve Loughran 
 wrote:
 
> I have put together a release candidate (rc0) for Hadoop 3.3.3
> 
> The RC is available at:
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/
> 
> The git tag is release-3.3.3-RC0, commit d37586cbda3
> 
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1348/
> 
> You can find my public key at:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> 
> Change log
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/CHANGELOG.md
> 
> Release notes
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/RELEASENOTES.md
> 
> There's a very small number of changes, primarily critical code/packaging
> issues and security fixes.
> 
> 
>   - The critical fixes which shipped in the 3.2.3 release.
>   -  CVEs in our code and dependencies
>   - Shaded client packaging issues.
>   - A switch from log4j to reload4j
> 
> 
> reload4j is an active fork of the log4j 1.17 library with the classes 
> which
> contain CVEs removed. Even though hadoop never used those classes, they
> regularly raised alerts on security scans and concen from users. Switching
> to the forked project allows us to ship a secure logging framework. It 
> will
> complicate the builds of downstream maven/ivy/gradle projects which 
> exclude
> our log4j artifacts, as they need to cut the new dependency instead/as
> well.
> 
> See the release notes for details.
> 
> This is my first release through the new docker build process, do please
> validate artifact signing  to make sure it is good. I'll be trying 
> builds
> of downstream projects.
> 
> We know there are some outstanding issues with at least one library we are
> shipping (okhttp), but I don't want to hold this release up for it. If the
> docker based release process works smoothly enough we can do a followup
> security release in a few weeks.
> 
> Please try the release and vote. The vote will run for 5 days.
> 
> -Steve
> 
>>> 
>>> 

Re: [VOTE] Release Apache Hadoop 3.3.3

[Chao Sun]

> Chao can you use the one that Steve mentioned in the mail?

Hmm how do I do that? Typically after closing the RC in nexus the
release bits will show up in
https://repository.apache.org/content/repositories/staging/org/apache/hadoop
and Spark build will be able to pick them up for testing. However in
this case I don't see any 3.3.3 jars in the URL.

On Fri, May 6, 2022 at 6:24 PM Ayush Saxena  wrote:
>
> There were two 3.3.3 staged. The earlier one was with skipShade, the date was 
> also april 22, I archived that. Chao can you use the one that Steve mentioned 
> in the mail?
>
> On Sat, 7 May 2022 at 06:18, Chao Sun  wrote:
>>
>> Seems there are some issues with the shaded client as I was not able
>> to compile Apache Spark with the RC
>> (https://github.com/apache/spark/pull/36474). Looks like it's compiled
>> with the `-DskipShade` option and the hadoop-client-api JAR doesn't
>> contain any class:
>>
>> ➜  hadoop-client-api jar tf 3.3.3/hadoop-client-api-3.3.3.jar
>> META-INF/
>> META-INF/MANIFEST.MF
>> META-INF/NOTICE.txt
>> META-INF/LICENSE.txt
>> META-INF/maven/
>> META-INF/maven/org.apache.hadoop/
>> META-INF/maven/org.apache.hadoop/hadoop-client-api/
>> META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.xml
>> META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.properties
>>
>> On Fri, May 6, 2022 at 4:24 PM Stack  wrote:
>> >
>> > +1 (binding)
>> >
>> >   * Signature: ok
>> >   * Checksum : passed
>> >   * Rat check (1.8.0_191): passed
>> >- mvn clean apache-rat:check
>> >   * Built from source (1.8.0_191): failed
>> >- mvn clean install  -DskipTests
>> >- mvn -fae --no-transfer-progress -DskipTests -Dmaven.javadoc.skip=true
>> > -Pnative -Drequire.openssl -Drequire.snappy -Drequire.valgrind
>> > -Drequire.zstd -Drequire.test.libhadoop clean install
>> >   * Unit tests pass (1.8.0_191):
>> > - HDFS Tests passed (Didn't run more than this).
>> >
>> > Deployed a ten node ha hdfs cluster with three namenodes and five
>> > journalnodes. Ran a ten node hbase (older version of 2.5 branch built
>> > against 3.3.2) against it. Tried a small verification job. Good. Ran a
>> > bigger job with mild chaos. All seems to be working properly (recoveries,
>> > logs look fine). Killed a namenode. Failover worked promptly. UIs look
>> > good. Poked at the hdfs cli. Seems good.
>> >
>> > S
>> >
>> > On Tue, May 3, 2022 at 4:24 AM Steve Loughran 
>> > wrote:
>> >
>> > > I have put together a release candidate (rc0) for Hadoop 3.3.3
>> > >
>> > > The RC is available at:
>> > > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/
>> > >
>> > > The git tag is release-3.3.3-RC0, commit d37586cbda3
>> > >
>> > > The maven artifacts are staged at
>> > > https://repository.apache.org/content/repositories/orgapachehadoop-1348/
>> > >
>> > > You can find my public key at:
>> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> > >
>> > > Change log
>> > > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/CHANGELOG.md
>> > >
>> > > Release notes
>> > > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/RELEASENOTES.md
>> > >
>> > > There's a very small number of changes, primarily critical code/packaging
>> > > issues and security fixes.
>> > >
>> > >
>> > >- The critical fixes which shipped in the 3.2.3 release.
>> > >-  CVEs in our code and dependencies
>> > >- Shaded client packaging issues.
>> > >- A switch from log4j to reload4j
>> > >
>> > >
>> > > reload4j is an active fork of the log4j 1.17 library with the classes 
>> > > which
>> > > contain CVEs removed. Even though hadoop never used those classes, they
>> > > regularly raised alerts on security scans and concen from users. 
>> > > Switching
>> > > to the forked project allows us to ship a secure logging framework. It 
>> > > will
>> > > complicate the builds of downstream maven/ivy/gradle projects which 
>> > > exclude
>> > > our log4j artifacts, as they need to cut the new dependency instead/as
>> > > well.
>> > >
>> > > See the release notes for details.
>> > >
>> > > This is my first release through the new docker build process, do please
>> > > validate artifact signing  to make sure it is good. I'll be trying 
>> > > builds
>> > > of downstream projects.
>> > >
>> > > We know there are some outstanding issues with at least one library we 
>> > > are
>> > > shipping (okhttp), but I don't want to hold this release up for it. If 
>> > > the
>> > > docker based release process works smoothly enough we can do a followup
>> > > security release in a few weeks.
>> > >
>> > > Please try the release and vote. The vote will run for 5 days.
>> > >
>> > > -Steve
>> > >
>>
>> -
>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>>

-
To unsubscribe, e-mail: 

Re: [VOTE] Release Apache Hadoop 3.3.3

[Ayush Saxena]

There were two 3.3.3 staged. The earlier one was with skipShade, the date
was also april 22, I archived that. Chao can you use the one that Steve
mentioned in the mail?

On Sat, 7 May 2022 at 06:18, Chao Sun  wrote:

> Seems there are some issues with the shaded client as I was not able
> to compile Apache Spark with the RC
> (https://github.com/apache/spark/pull/36474). Looks like it's compiled
> with the `-DskipShade` option and the hadoop-client-api JAR doesn't
> contain any class:
>
> ➜  hadoop-client-api jar tf 3.3.3/hadoop-client-api-3.3.3.jar
> META-INF/
> META-INF/MANIFEST.MF
> META-INF/NOTICE.txt
> META-INF/LICENSE.txt
> META-INF/maven/
> META-INF/maven/org.apache.hadoop/
> META-INF/maven/org.apache.hadoop/hadoop-client-api/
> META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.xml
> META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.properties
>
> On Fri, May 6, 2022 at 4:24 PM Stack  wrote:
> >
> > +1 (binding)
> >
> >   * Signature: ok
> >   * Checksum : passed
> >   * Rat check (1.8.0_191): passed
> >- mvn clean apache-rat:check
> >   * Built from source (1.8.0_191): failed
> >- mvn clean install  -DskipTests
> >- mvn -fae --no-transfer-progress -DskipTests
> -Dmaven.javadoc.skip=true
> > -Pnative -Drequire.openssl -Drequire.snappy -Drequire.valgrind
> > -Drequire.zstd -Drequire.test.libhadoop clean install
> >   * Unit tests pass (1.8.0_191):
> > - HDFS Tests passed (Didn't run more than this).
> >
> > Deployed a ten node ha hdfs cluster with three namenodes and five
> > journalnodes. Ran a ten node hbase (older version of 2.5 branch built
> > against 3.3.2) against it. Tried a small verification job. Good. Ran a
> > bigger job with mild chaos. All seems to be working properly (recoveries,
> > logs look fine). Killed a namenode. Failover worked promptly. UIs look
> > good. Poked at the hdfs cli. Seems good.
> >
> > S
> >
> > On Tue, May 3, 2022 at 4:24 AM Steve Loughran
> 
> > wrote:
> >
> > > I have put together a release candidate (rc0) for Hadoop 3.3.3
> > >
> > > The RC is available at:
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/
> > >
> > > The git tag is release-3.3.3-RC0, commit d37586cbda3
> > >
> > > The maven artifacts are staged at
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1348/
> > >
> > > You can find my public key at:
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > >
> > > Change log
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/CHANGELOG.md
> > >
> > > Release notes
> > >
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/RELEASENOTES.md
> > >
> > > There's a very small number of changes, primarily critical
> code/packaging
> > > issues and security fixes.
> > >
> > >
> > >- The critical fixes which shipped in the 3.2.3 release.
> > >-  CVEs in our code and dependencies
> > >- Shaded client packaging issues.
> > >- A switch from log4j to reload4j
> > >
> > >
> > > reload4j is an active fork of the log4j 1.17 library with the classes
> which
> > > contain CVEs removed. Even though hadoop never used those classes, they
> > > regularly raised alerts on security scans and concen from users.
> Switching
> > > to the forked project allows us to ship a secure logging framework. It
> will
> > > complicate the builds of downstream maven/ivy/gradle projects which
> exclude
> > > our log4j artifacts, as they need to cut the new dependency instead/as
> > > well.
> > >
> > > See the release notes for details.
> > >
> > > This is my first release through the new docker build process, do
> please
> > > validate artifact signing  to make sure it is good. I'll be trying
> builds
> > > of downstream projects.
> > >
> > > We know there are some outstanding issues with at least one library we
> are
> > > shipping (okhttp), but I don't want to hold this release up for it. If
> the
> > > docker based release process works smoothly enough we can do a followup
> > > security release in a few weeks.
> > >
> > > Please try the release and vote. The vote will run for 5 days.
> > >
> > > -Steve
> > >
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.3.3

[Chao Sun]

Seems there are some issues with the shaded client as I was not able
to compile Apache Spark with the RC
(https://github.com/apache/spark/pull/36474). Looks like it's compiled
with the `-DskipShade` option and the hadoop-client-api JAR doesn't
contain any class:

➜  hadoop-client-api jar tf 3.3.3/hadoop-client-api-3.3.3.jar
META-INF/
META-INF/MANIFEST.MF
META-INF/NOTICE.txt
META-INF/LICENSE.txt
META-INF/maven/
META-INF/maven/org.apache.hadoop/
META-INF/maven/org.apache.hadoop/hadoop-client-api/
META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.xml
META-INF/maven/org.apache.hadoop/hadoop-client-api/pom.properties

On Fri, May 6, 2022 at 4:24 PM Stack  wrote:
>
> +1 (binding)
>
>   * Signature: ok
>   * Checksum : passed
>   * Rat check (1.8.0_191): passed
>- mvn clean apache-rat:check
>   * Built from source (1.8.0_191): failed
>- mvn clean install  -DskipTests
>- mvn -fae --no-transfer-progress -DskipTests -Dmaven.javadoc.skip=true
> -Pnative -Drequire.openssl -Drequire.snappy -Drequire.valgrind
> -Drequire.zstd -Drequire.test.libhadoop clean install
>   * Unit tests pass (1.8.0_191):
> - HDFS Tests passed (Didn't run more than this).
>
> Deployed a ten node ha hdfs cluster with three namenodes and five
> journalnodes. Ran a ten node hbase (older version of 2.5 branch built
> against 3.3.2) against it. Tried a small verification job. Good. Ran a
> bigger job with mild chaos. All seems to be working properly (recoveries,
> logs look fine). Killed a namenode. Failover worked promptly. UIs look
> good. Poked at the hdfs cli. Seems good.
>
> S
>
> On Tue, May 3, 2022 at 4:24 AM Steve Loughran 
> wrote:
>
> > I have put together a release candidate (rc0) for Hadoop 3.3.3
> >
> > The RC is available at:
> > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/
> >
> > The git tag is release-3.3.3-RC0, commit d37586cbda3
> >
> > The maven artifacts are staged at
> > https://repository.apache.org/content/repositories/orgapachehadoop-1348/
> >
> > You can find my public key at:
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> > Change log
> > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/CHANGELOG.md
> >
> > Release notes
> > https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/RELEASENOTES.md
> >
> > There's a very small number of changes, primarily critical code/packaging
> > issues and security fixes.
> >
> >
> >- The critical fixes which shipped in the 3.2.3 release.
> >-  CVEs in our code and dependencies
> >- Shaded client packaging issues.
> >- A switch from log4j to reload4j
> >
> >
> > reload4j is an active fork of the log4j 1.17 library with the classes which
> > contain CVEs removed. Even though hadoop never used those classes, they
> > regularly raised alerts on security scans and concen from users. Switching
> > to the forked project allows us to ship a secure logging framework. It will
> > complicate the builds of downstream maven/ivy/gradle projects which exclude
> > our log4j artifacts, as they need to cut the new dependency instead/as
> > well.
> >
> > See the release notes for details.
> >
> > This is my first release through the new docker build process, do please
> > validate artifact signing  to make sure it is good. I'll be trying builds
> > of downstream projects.
> >
> > We know there are some outstanding issues with at least one library we are
> > shipping (okhttp), but I don't want to hold this release up for it. If the
> > docker based release process works smoothly enough we can do a followup
> > security release in a few weeks.
> >
> > Please try the release and vote. The vote will run for 5 days.
> >
> > -Steve
> >

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.3.3

[Stack]

+1 (binding)

  * Signature: ok
  * Checksum : passed
  * Rat check (1.8.0_191): passed
   - mvn clean apache-rat:check
  * Built from source (1.8.0_191): failed
   - mvn clean install  -DskipTests
   - mvn -fae --no-transfer-progress -DskipTests -Dmaven.javadoc.skip=true
-Pnative -Drequire.openssl -Drequire.snappy -Drequire.valgrind
-Drequire.zstd -Drequire.test.libhadoop clean install
  * Unit tests pass (1.8.0_191):
- HDFS Tests passed (Didn't run more than this).

Deployed a ten node ha hdfs cluster with three namenodes and five
journalnodes. Ran a ten node hbase (older version of 2.5 branch built
against 3.3.2) against it. Tried a small verification job. Good. Ran a
bigger job with mild chaos. All seems to be working properly (recoveries,
logs look fine). Killed a namenode. Failover worked promptly. UIs look
good. Poked at the hdfs cli. Seems good.

S

On Tue, May 3, 2022 at 4:24 AM Steve Loughran 
wrote:

> I have put together a release candidate (rc0) for Hadoop 3.3.3
>
> The RC is available at:
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/
>
> The git tag is release-3.3.3-RC0, commit d37586cbda3
>
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1348/
>
> You can find my public key at:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Change log
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/CHANGELOG.md
>
> Release notes
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/RELEASENOTES.md
>
> There's a very small number of changes, primarily critical code/packaging
> issues and security fixes.
>
>
>- The critical fixes which shipped in the 3.2.3 release.
>-  CVEs in our code and dependencies
>- Shaded client packaging issues.
>- A switch from log4j to reload4j
>
>
> reload4j is an active fork of the log4j 1.17 library with the classes which
> contain CVEs removed. Even though hadoop never used those classes, they
> regularly raised alerts on security scans and concen from users. Switching
> to the forked project allows us to ship a secure logging framework. It will
> complicate the builds of downstream maven/ivy/gradle projects which exclude
> our log4j artifacts, as they need to cut the new dependency instead/as
> well.
>
> See the release notes for details.
>
> This is my first release through the new docker build process, do please
> validate artifact signing  to make sure it is good. I'll be trying builds
> of downstream projects.
>
> We know there are some outstanding issues with at least one library we are
> shipping (okhttp), but I don't want to hold this release up for it. If the
> docker based release process works smoothly enough we can do a followup
> security release in a few weeks.
>
> Please try the release and vote. The vote will run for 5 days.
>
> -Steve
>

Re: [VOTE] Release Apache Hadoop 3.3.3

[Ayush Saxena]

+1,
* Built from source
* Successful native build on Ubuntu 18.04
* Verified Checksums
(CHANGELOG.md,RELEASENOTES.md,hadoop-3.3.3-rat.txt,hadoop-3.3.3-site.tar.gz,hadoop-3.3.3-src.tar.gz,hadoop-3.3.3.tar.gz)
* Successful RAT check
* Ran some basic HDFS shell commands
* Ran some basic YARN shell commands
* Browsed through UI (NN ,DN, RM, NM & JHS)
* Tried some commands on Hive using Hive built on hive-master
* Verified Signature: Says Good Signature but "This key is not certified
with a trusted signature!"
* Ran some MR example jobs(TeraGen, TeraSort, TeraValidate, WordCount,
WordMean & Pi)
* Version & commit hash seems correct in UI as well as in hadoop version
output.
* Browsed through the ChangeLog & Release Notes (One place mentions hadoop
3.4.0 though, but we can survive I suppose)
* Browsed through the documentation.

Thanx Steve for driving the release, Good Luck!!!

-Ayush



On Tue, 3 May 2022 at 16:54, Steve Loughran 
wrote:

> I have put together a release candidate (rc0) for Hadoop 3.3.3
>
> The RC is available at:
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/
>
> The git tag is release-3.3.3-RC0, commit d37586cbda3
>
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1348/
>
> You can find my public key at:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Change log
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/CHANGELOG.md
>
> Release notes
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/RELEASENOTES.md
>
> There's a very small number of changes, primarily critical code/packaging
> issues and security fixes.
>
>
>- The critical fixes which shipped in the 3.2.3 release.
>-  CVEs in our code and dependencies
>- Shaded client packaging issues.
>- A switch from log4j to reload4j
>
>
> reload4j is an active fork of the log4j 1.17 library with the classes which
> contain CVEs removed. Even though hadoop never used those classes, they
> regularly raised alerts on security scans and concen from users. Switching
> to the forked project allows us to ship a secure logging framework. It will
> complicate the builds of downstream maven/ivy/gradle projects which exclude
> our log4j artifacts, as they need to cut the new dependency instead/as
> well.
>
> See the release notes for details.
>
> This is my first release through the new docker build process, do please
> validate artifact signing  to make sure it is good. I'll be trying builds
> of downstream projects.
>
> We know there are some outstanding issues with at least one library we are
> shipping (okhttp), but I don't want to hold this release up for it. If the
> docker based release process works smoothly enough we can do a followup
> security release in a few weeks.
>
> Please try the release and vote. The vote will run for 5 days.
>
> -Steve
>

Re: [VOTE] Release Apache Hadoop 3.3.3

[Viraj Jasani]

+1 (non-binding),

With a minor change  in
hadoop-vote,

* Signature: ok
* Checksum : ok
* Rat check (1.8.0_301): ok
 - mvn clean apache-rat:check
* Built from source (1.8.0_301): ok
 - mvn clean install  -DskipTests
* Built tar from source (1.8.0_301): ok
 - mvn clean package  -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true

HDFS and MapReduce functional testing looks good.

As per PR#4268 , except for a
few flakes, TestDistributedShell and TestCsiClient are consistently failing.


On Tue, May 3, 2022 at 4:24 AM Steve Loughran 
wrote:

> I have put together a release candidate (rc0) for Hadoop 3.3.3
>
> The RC is available at:
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/
>
> The git tag is release-3.3.3-RC0, commit d37586cbda3
>
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1348/
>
> You can find my public key at:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Change log
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/CHANGELOG.md
>
> Release notes
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/RELEASENOTES.md
>
> There's a very small number of changes, primarily critical code/packaging
> issues and security fixes.
>
>
>- The critical fixes which shipped in the 3.2.3 release.
>-  CVEs in our code and dependencies
>- Shaded client packaging issues.
>- A switch from log4j to reload4j
>
>
> reload4j is an active fork of the log4j 1.17 library with the classes which
> contain CVEs removed. Even though hadoop never used those classes, they
> regularly raised alerts on security scans and concen from users. Switching
> to the forked project allows us to ship a secure logging framework. It will
> complicate the builds of downstream maven/ivy/gradle projects which exclude
> our log4j artifacts, as they need to cut the new dependency instead/as
> well.
>
> See the release notes for details.
>
> This is my first release through the new docker build process, do please
> validate artifact signing  to make sure it is good. I'll be trying builds
> of downstream projects.
>
> We know there are some outstanding issues with at least one library we are
> shipping (okhttp), but I don't want to hold this release up for it. If the
> docker based release process works smoothly enough we can do a followup
> security release in a few weeks.
>
> Please try the release and vote. The vote will run for 5 days.
>
> -Steve
>

Re: [VOTE] Release Apache Hadoop 3.3.3

[Masatake Iwasaki]

+1 (binding)

* verified signature and checksum of the source tarball.
* built the source code on Rocky Linux 8 (x86_64) and OpenJDK 8
  by `mvn install -DskipTests -Pnative -Pdist`.
* launched pseudo distributed cluster with Kerberos security enabled and ran 
sample MR jobs.
* launched HA enabled 3-nodes docker cluster and ran sample MR jobs.
* launched pseudo distributed cluster and `spark-shell --master yarn`
  with spark-3.2.1-bin-without-hadoop and ran some tutorial code.
* built site documentation by `mvn site site:stage -Preleasedocs` and skimmed 
the contents.

Thanks,
Masatake Iwasaki

On 2022/05/03 20:18, Steve Loughran wrote:

I have put together a release candidate (rc0) for Hadoop 3.3.3

The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/

The git tag is release-3.3.3-RC0, commit d37586cbda3

The maven artifacts are staged at
https://repository.apache.org/content/repositories/orgapachehadoop-1348/

You can find my public key at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS

Change log
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/CHANGELOG.md

Release notes
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/RELEASENOTES.md

There's a very small number of changes, primarily critical code/packaging
issues and security fixes.


- The critical fixes which shipped in the 3.2.3 release.
-  CVEs in our code and dependencies
- Shaded client packaging issues.
- A switch from log4j to reload4j


reload4j is an active fork of the log4j 1.17 library with the classes which
contain CVEs removed. Even though hadoop never used those classes, they
regularly raised alerts on security scans and concen from users. Switching
to the forked project allows us to ship a secure logging framework. It will
complicate the builds of downstream maven/ivy/gradle projects which exclude
our log4j artifacts, as they need to cut the new dependency instead/as well.

See the release notes for details.

This is my first release through the new docker build process, do please
validate artifact signing  to make sure it is good. I'll be trying builds
of downstream projects.

We know there are some outstanding issues with at least one library we are
shipping (okhttp), but I don't want to hold this release up for it. If the
docker based release process works smoothly enough we can do a followup
security release in a few weeks.

Please try the release and vote. The vote will run for 5 days.

-Steve



-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.3.3

[Mukund Madhav Thakur]

+1 (binding)
Built tar from source in mac os and 1.8.0_161.   good

mvn clean package -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true


Able to access cloud s3 and abfs storage through command line.

Ran S3 and Abfs integration tests. All good.



On Tue, May 3, 2022 at 6:24 AM Steve Loughran 
wrote:

> I have put together a release candidate (rc0) for Hadoop 3.3.3
>
> The RC is available at:
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/
>
> The git tag is release-3.3.3-RC0, commit d37586cbda3
>
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1348/
>
> You can find my public key at:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Change log
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/CHANGELOG.md
>
> Release notes
> https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/RELEASENOTES.md
>
> There's a very small number of changes, primarily critical code/packaging
> issues and security fixes.
>
>
>- The critical fixes which shipped in the 3.2.3 release.
>-  CVEs in our code and dependencies
>- Shaded client packaging issues.
>- A switch from log4j to reload4j
>
>
> reload4j is an active fork of the log4j 1.17 library with the classes which
> contain CVEs removed. Even though hadoop never used those classes, they
> regularly raised alerts on security scans and concen from users. Switching
> to the forked project allows us to ship a secure logging framework. It will
> complicate the builds of downstream maven/ivy/gradle projects which exclude
> our log4j artifacts, as they need to cut the new dependency instead/as
> well.
>
> See the release notes for details.
>
> This is my first release through the new docker build process, do please
> validate artifact signing  to make sure it is good. I'll be trying builds
> of downstream projects.
>
> We know there are some outstanding issues with at least one library we are
> shipping (okhttp), but I don't want to hold this release up for it. If the
> docker based release process works smoothly enough we can do a followup
> security release in a few weeks.
>
> Please try the release and vote. The vote will run for 5 days.
>
> -Steve
>

[VOTE] Release Apache Hadoop 3.3.3

[Steve Loughran]

I have put together a release candidate (rc0) for Hadoop 3.3.3

The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/

The git tag is release-3.3.3-RC0, commit d37586cbda3

The maven artifacts are staged at
https://repository.apache.org/content/repositories/orgapachehadoop-1348/

You can find my public key at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS

Change log
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/CHANGELOG.md

Release notes
https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/RELEASENOTES.md

There's a very small number of changes, primarily critical code/packaging
issues and security fixes.


   - The critical fixes which shipped in the 3.2.3 release.
   -  CVEs in our code and dependencies
   - Shaded client packaging issues.
   - A switch from log4j to reload4j


reload4j is an active fork of the log4j 1.17 library with the classes which
contain CVEs removed. Even though hadoop never used those classes, they
regularly raised alerts on security scans and concen from users. Switching
to the forked project allows us to ship a secure logging framework. It will
complicate the builds of downstream maven/ivy/gradle projects which exclude
our log4j artifacts, as they need to cut the new dependency instead/as well.

See the release notes for details.

This is my first release through the new docker build process, do please
validate artifact signing  to make sure it is good. I'll be trying builds
of downstream projects.

We know there are some outstanding issues with at least one library we are
shipping (okhttp), but I don't want to hold this release up for it. If the
docker based release process works smoothly enough we can do a followup
security release in a few weeks.

Please try the release and vote. The vote will run for 5 days.

-Steve