Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

Here are the result of the vote. Binding PMC members +1 Stack +1 Masatake Iwasaki +1 Xiaoqiao He +1 Chao Sun +1 Chris Nauroth Non binding votes +1 Ayush Saxena +1 Viraj Jasani +1 Mukund Thakur There were no negative votes, and maven is happy. I'm pleased to announce therefore, that this rele

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

+1 (binding) - Verified all checksums. - Verified all signatures. - Built from source, including native code on Linux. - Ran several examples successfully. Chris Nauroth On Mon, May 16, 2022 at 10:06 AM Chao Sun wrote: > +1 > > - Compiled from source > - Verified checksums & signatures > - La

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

+1 - Compiled from source - Verified checksums & signatures - Launched a pseudo HDFS cluster and ran some simple commands - Ran full Spark tests with the RC Thanks Steve! Chao On Mon, May 16, 2022 at 2:19 AM Ayush Saxena wrote: > > +1, > * Built from source. > * Successful native build on Ubun

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

+1, * Built from source. * Successful native build on Ubuntu 18.04 * Verified Checksums. (CHANGELOG.md,RELEASENOTES.md,hadoop-3.3.3-rat.txt,hadoop-3.3.3-site.tar.gz,hadoop-3.3.3-src.tar.gz,hadoop-3.3.3.tar.gz) * Verified Signature. * Successful RAT check * Ran basic HDFS shell commands. * Ran basic

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

+1(binding) * Verified signature and checksum of the source tarball. * Built the source code on Ubuntu and OpenJDK 11 by `mvn clean package -DskipTests -Pnative -Pdist -Dtar`. * Setup pseudo cluster with HDFS and YARN. * Run simple FsShell - mkdir/put/get/mv/rm and check the result. * Run example

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

+1 (non-binding) * Signature: ok * Checksum : ok * Rat check (1.8.0_301): ok - mvn clean apache-rat:check * Built from source (1.8.0_301): ok - mvn clean install -DskipTests * Built tar from source (1.8.0_301): ok - mvn clean package -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true HDFS, M

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

+1 (binding) * verified signature and checksum of the source tarball. * built the source code on Rocky Linux 8 (x86_64) and OpenJDK 8 by `mvn install -DskipTests -Pnative -Pdist`. * launched pseudo distributed cluster with Kerberos security enabled and ran sample MR jobs. * launched HA enabled

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

+1 (binding) Signature : good Checksum : good. Compiled from source , good mvn clean install -DskipTests Ran aws integration tests, good. mvn -Dparallel-tests -DtestsThreadCount=8 clean verify Compiled gcs using staging repo with version 3.3.3, successful. mvn clean install -DskipTests -Dhad

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

+1 (binding) * Signature: ok * Checksum : ok * Rat check (10.0.2): ok - mvn clean apache-rat:check * Built from source (10.0.2): ok - mvn clean install -DskipTests * Unit tests pass (10.0.2): ok - mvn package -P runAllTests -Dsur

[VOTE] Release Apache Hadoop 3.3.3 (RC1)

I have put together a release candidate (RC1) for Hadoop 3.3.3 The RC is available at: https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC1/ The git tag is release-3.3.3-RC1, commit d37586cbda3 The maven artifacts are staged at https://repository.apache.org/content/repositories/orgapachehadoo

Re: [VOTE] Release Apache Hadoop 3.3.3

update: got a stub project whose test run requires the client api and shaded artifacts, and whose clean target wiil delete all artifacts of a specific version from your local repo, so as to ensure all it untainted https://github.com/steveloughran/validate-hadoop-client-artifacts I do now have a v

Re: [VOTE] Release Apache Hadoop 3.3.3

I've done another docker build and the client jars appear to be there. I'll test tomorrow before putting up another vote. it will be exactly the same commit as before, just recompiled/republished On Mon, 9 May 2022 at 17:45, Chao Sun wrote: > Agreed, that step #10 is out-dated and should be remo

Re: [VOTE] Release Apache Hadoop 3.3.3

Agreed, that step #10 is out-dated and should be removed (I skipped that when releasing Hadoop 3.3.2 but didn't update it, sorry). > How about using > https://repository.apache.org/content/repositories/orgapachehadoop-1348/ Akira, I tried this too but it didn't work. I think we'd need the artifa

Re: [VOTE] Release Apache Hadoop 3.3.3

I didn't do that as the docker image was doing it itself...I discussed this with Akira and Ayush & they agreed. so whatever went wrong. it was something else. I have been building a list of things I'd like to change there; cutting that line was one of them. but I need to work out the correct workf

Re: [VOTE] Release Apache Hadoop 3.3.3

Step #10 in "Creating the release candidate (X.Y.Z-RC)" section of the Wiki still instructs to run `mvn deploy` with `-DskipShade`. https://cwiki.apache.org/confluence/display/HADOOP2/HowToRelease#HowToRelease-Creatingthereleasecandidate(X.Y.Z-RC%3CN%3E) On 2022/05/09 23:18, Masatake Iwasaki wr

Re: [VOTE] Release Apache Hadoop 3.3.3

It seems to be caused by obsolete instruction in HowToRelease Wiki? After HADOOP-15058, `mvn deploy` is kicked by `dev-support/bin/create-release --asfrelease`. https://issues.apache.org/jira/browse/HADOOP-15058 Step #10 in "Creating the release candidate (X.Y.Z-RC)" section of the Wiki still in

Re: [VOTE] Release Apache Hadoop 3.3.3

had a report of that staging playing up too from Gautham I'm going to have to treat this as a failure from the maven deploy process, so I'll have to cancel this RC and try and work out WTF went wrong. I'll also put together some minimal apps to fetch and compile against everything. i did do some

Re: [VOTE] Release Apache Hadoop 3.3.3

Hi Chao, How about using https://repository.apache.org/content/repositories/orgapachehadoop-1348/ instead of https://repository.apache.org/content/repositories/staging/ ? Akira On Sat, May 7, 2022 at 10:52 AM Ayush Saxena wrote: > Hmm, I see the artifacts ideally should have got overwritten by

Re: [VOTE] Release Apache Hadoop 3.3.3

Hmm, I see the artifacts ideally should have got overwritten by the new RC, but they didn’t. The reason seems like the staging path shared doesn’t have any jars… That is why it was picking the old jars. I think Steve needs to run mvn deploy again… Sent from my iPhone > On 07-May-2022, at 7:12

Re: [VOTE] Release Apache Hadoop 3.3.3

> Chao can you use the one that Steve mentioned in the mail? Hmm how do I do that? Typically after closing the RC in nexus the release bits will show up in https://repository.apache.org/content/repositories/staging/org/apache/hadoop and Spark build will be able to pick them up for testing. However

Re: [VOTE] Release Apache Hadoop 3.3.3

There were two 3.3.3 staged. The earlier one was with skipShade, the date was also april 22, I archived that. Chao can you use the one that Steve mentioned in the mail? On Sat, 7 May 2022 at 06:18, Chao Sun wrote: > Seems there are some issues with the shaded client as I was not able > to compil

Re: [VOTE] Release Apache Hadoop 3.3.3

Seems there are some issues with the shaded client as I was not able to compile Apache Spark with the RC (https://github.com/apache/spark/pull/36474). Looks like it's compiled with the `-DskipShade` option and the hadoop-client-api JAR doesn't contain any class: ➜ hadoop-client-api jar tf 3.3.3/h

Re: [VOTE] Release Apache Hadoop 3.3.3

+1 (binding) * Signature: ok * Checksum : passed * Rat check (1.8.0_191): passed - mvn clean apache-rat:check * Built from source (1.8.0_191): failed - mvn clean install -DskipTests - mvn -fae --no-transfer-progress -DskipTests -Dmaven.javadoc.skip=true -Pnative -Drequire.openssl

Re: [VOTE] Release Apache Hadoop 3.3.3

+1, * Built from source * Successful native build on Ubuntu 18.04 * Verified Checksums (CHANGELOG.md,RELEASENOTES.md,hadoop-3.3.3-rat.txt,hadoop-3.3.3-site.tar.gz,hadoop-3.3.3-src.tar.gz,hadoop-3.3.3.tar.gz) * Successful RAT check * Ran some basic HDFS shell commands * Ran some basic YARN shell com

Re: [VOTE] Release Apache Hadoop 3.3.3

+1 (non-binding), With a minor change in hadoop-vote, * Signature: ok * Checksum : ok * Rat check (1.8.0_301): ok - mvn clean apache-rat:check * Built from source (1.8.0_301): ok - mvn clean install -DskipTests * Built tar from source (1.8.0_301): o

Re: [VOTE] Release Apache Hadoop 3.3.3

+1 (binding) * verified signature and checksum of the source tarball. * built the source code on Rocky Linux 8 (x86_64) and OpenJDK 8 by `mvn install -DskipTests -Pnative -Pdist`. * launched pseudo distributed cluster with Kerberos security enabled and ran sample MR jobs. * launched HA enabled

Re: [VOTE] Release Apache Hadoop 3.3.3

+1 (binding) Built tar from source in mac os and 1.8.0_161. good mvn clean package -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true Able to access cloud s3 and abfs storage through command line. Ran S3 and Abfs integration tests. All good. On Tue, May 3, 2022 at 6:24 AM Steve Loughran w

[VOTE] Release Apache Hadoop 3.3.3

I have put together a release candidate (rc0) for Hadoop 3.3.3 The RC is available at: https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/ The git tag is release-3.3.3-RC0, commit d37586cbda3 The maven artifacts are staged at https://repository.apache.org/content/repositories/orgapachehadoo