Re: [VOTE] Release Apache Hadoop 3.3.2 - RC0
> There was a discussion [1] and concluded that ARM binaries are optional. The release vote is only for source code, not binary. Cool, thanks for clarifying this Akira! I'll not include the ARM binaries in the next RC then. > Come January I will look at moving to the 1.12 SDK; I don't know how long that will take to stabilize. Sounds good. Hope everyone had a good holiday break! I'll address the above issues and create RC1 soon. Thanks, Chao On Thu, Dec 30, 2021 at 6:00 AM Steve Loughran wrote: > On Tue, 14 Dec 2021 at 22:56, Chao Sun wrote: > > > Thanks all for taking a look! looks like we need another RC addressing > the > > following issues. > > > > > 1. the overview page of the doc is for the Hadoop 3.0 release. It would > > be best to base the doc on top of Hadoop 3.3.0 overview page. (it's a > miss > > on my part... The overview page of 3.3.1 wasn't updated) > > > > For this, I just need to update > > the hadoop-project/src/site/markdown/index.md.vm and incorporate notable > > changes made in 3.3.1/3.3.2, is that correct? looks like the file hasn't > > been touched for a while. > > > > > 2. ARM binaries is not included. For the 3.3.1 release, I had to run > the > > create release script on an ARM machine separately to create the binary > > tarball. > > > > Hmm this might be challenging for me. Could you share the steps of how > you > > did it? especially where did you get an ARM machine. > > > > > 3. the jdiff version > > > https://github.com/apache/hadoop/blob/branch-3.3.2/hadoop-project-dist/pom.xml#L137 > > > > I just need to backport this commit: > > > https://github.com/apache/hadoop/commit/a77bf7cf07189911da99e305e3b80c589edbbfb5 > > to branch-3.3.2 (and potentially branch-3.3)? > > > > > The 3.3.1 binary tarball is 577mb. The 3.3.2 RC0 is 608mb. I'm curious > > what are added. > > > > The difference is mostly in aws-java-sdk-bundle jar: 3.3.1 uses > > > https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-bundle/1.11.901 > > while 3.3.2 uses > > > https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-bundle/1.11.1026 > . > > The difference is ~32.5mb. > > > > Chao > > > >> > >>> Wow, that's a big change. But the bundling of a shaded jar avoids so > many problems we've had in the past. Come January I will look at moving to > the 1.12 SDK; I don't know how long that will take to stabilize. >
Re: [VOTE] Release Apache Hadoop 3.3.2 - RC0
On Tue, 14 Dec 2021 at 22:56, Chao Sun wrote: > Thanks all for taking a look! looks like we need another RC addressing the > following issues. > > > 1. the overview page of the doc is for the Hadoop 3.0 release. It would > be best to base the doc on top of Hadoop 3.3.0 overview page. (it's a miss > on my part... The overview page of 3.3.1 wasn't updated) > > For this, I just need to update > the hadoop-project/src/site/markdown/index.md.vm and incorporate notable > changes made in 3.3.1/3.3.2, is that correct? looks like the file hasn't > been touched for a while. > > > 2. ARM binaries is not included. For the 3.3.1 release, I had to run the > create release script on an ARM machine separately to create the binary > tarball. > > Hmm this might be challenging for me. Could you share the steps of how you > did it? especially where did you get an ARM machine. > > > 3. the jdiff version > https://github.com/apache/hadoop/blob/branch-3.3.2/hadoop-project-dist/pom.xml#L137 > > I just need to backport this commit: > https://github.com/apache/hadoop/commit/a77bf7cf07189911da99e305e3b80c589edbbfb5 > to branch-3.3.2 (and potentially branch-3.3)? > > > The 3.3.1 binary tarball is 577mb. The 3.3.2 RC0 is 608mb. I'm curious > what are added. > > The difference is mostly in aws-java-sdk-bundle jar: 3.3.1 uses > https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-bundle/1.11.901 > while 3.3.2 uses > https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-bundle/1.11.1026. > The difference is ~32.5mb. > > Chao > >> >>> Wow, that's a big change. But the bundling of a shaded jar avoids so many problems we've had in the past. Come January I will look at moving to the 1.12 SDK; I don't know how long that will take to stabilize.
Re: [VOTE] Release Apache Hadoop 3.3.2 - RC0
Hi Chao, > For this, I just need to update the hadoop-project/src/site/markdown/index.md.vm and incorporate notable changes made in 3.3.1/3.3.2, is that correct? > ARM binaries There was a discussion [1] and concluded that ARM binaries are optional. The release vote is only for source code, not binary. BTW, I want to include https://issues.apache.org/jira/browse/YARN-11053 to 3.3.2 because it is a regression in 3.3.x releases. [1]: https://lists.apache.org/thread/ghcgq5s745zs5cc84n4owfphf1h21zz2 Thanks and regards, Akira On Wed, Dec 15, 2021 at 7:56 AM Chao Sun wrote: > Thanks all for taking a look! looks like we need another RC addressing the > following issues. > > > 1. the overview page of the doc is for the Hadoop 3.0 release. It would > be best to base the doc on top of Hadoop 3.3.0 overview page. (it's a miss > on my part... The overview page of 3.3.1 wasn't updated) > > For this, I just need to update > the hadoop-project/src/site/markdown/index.md.vm and incorporate notable > changes made in 3.3.1/3.3.2, is that correct? looks like the file hasn't > been touched for a while. > > > 2. ARM binaries is not included. For the 3.3.1 release, I had to run the > create release script on an ARM machine separately to create the binary > tarball. > > Hmm this might be challenging for me. Could you share the steps of how you > did it? especially where did you get an ARM machine. > > > 3. the jdiff version > > https://github.com/apache/hadoop/blob/branch-3.3.2/hadoop-project-dist/pom.xml#L137 > > I just need to backport this commit: > > https://github.com/apache/hadoop/commit/a77bf7cf07189911da99e305e3b80c589edbbfb5 > to branch-3.3.2 (and potentially branch-3.3)? > > > The 3.3.1 binary tarball is 577mb. The 3.3.2 RC0 is 608mb. I'm curious > what are added. > > The difference is mostly in aws-java-sdk-bundle jar: 3.3.1 uses > > https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-bundle/1.11.901 > while 3.3.2 uses > > https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-bundle/1.11.1026 > . > The difference is ~32.5mb. > > Chao > > On Tue, Dec 14, 2021 at 5:25 AM Steve Loughran > wrote: > > > I'll do my best to test this; I'm a bit broken right now. > > > > I think we should mention in a release notes that is the version of a > > log4j included in this and all previous releases is not vulnerable. But > > provide a list plus links to any that have been fixed > > > > On Fri, 10 Dec 2021 at 02:09, Chao Sun wrote: > > > >> Hi all, > >> > >> Sorry for the long delay. I've prepared RC0 for Hadoop 3.3.2 below: > >> > >> The RC is available at: > >> http://people.apache.org/~sunchao/hadoop-3.3.2-RC0/ > >> The RC tag is at: > >> https://github.com/apache/hadoop/releases/tag/release-3.3.2-RC0 > >> The Maven artifacts are staged at: > >> > https://repository.apache.org/content/repositories/orgapachehadoop-1330/ > >> > >> You can find my public key at: https://people.apache.org/~sunchao/KEYS > >> > >> Please evaluate the RC and vote. > >> > >> Thanks, > >> Chao > >> > > >
Re: [VOTE] Release Apache Hadoop 3.3.2 - RC0
Thanks all for taking a look! looks like we need another RC addressing the following issues. > 1. the overview page of the doc is for the Hadoop 3.0 release. It would be best to base the doc on top of Hadoop 3.3.0 overview page. (it's a miss on my part... The overview page of 3.3.1 wasn't updated) For this, I just need to update the hadoop-project/src/site/markdown/index.md.vm and incorporate notable changes made in 3.3.1/3.3.2, is that correct? looks like the file hasn't been touched for a while. > 2. ARM binaries is not included. For the 3.3.1 release, I had to run the create release script on an ARM machine separately to create the binary tarball. Hmm this might be challenging for me. Could you share the steps of how you did it? especially where did you get an ARM machine. > 3. the jdiff version https://github.com/apache/hadoop/blob/branch-3.3.2/hadoop-project-dist/pom.xml#L137 I just need to backport this commit: https://github.com/apache/hadoop/commit/a77bf7cf07189911da99e305e3b80c589edbbfb5 to branch-3.3.2 (and potentially branch-3.3)? > The 3.3.1 binary tarball is 577mb. The 3.3.2 RC0 is 608mb. I'm curious what are added. The difference is mostly in aws-java-sdk-bundle jar: 3.3.1 uses https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-bundle/1.11.901 while 3.3.2 uses https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-bundle/1.11.1026. The difference is ~32.5mb. Chao On Tue, Dec 14, 2021 at 5:25 AM Steve Loughran wrote: > I'll do my best to test this; I'm a bit broken right now. > > I think we should mention in a release notes that is the version of a > log4j included in this and all previous releases is not vulnerable. But > provide a list plus links to any that have been fixed > > On Fri, 10 Dec 2021 at 02:09, Chao Sun wrote: > >> Hi all, >> >> Sorry for the long delay. I've prepared RC0 for Hadoop 3.3.2 below: >> >> The RC is available at: >> http://people.apache.org/~sunchao/hadoop-3.3.2-RC0/ >> The RC tag is at: >> https://github.com/apache/hadoop/releases/tag/release-3.3.2-RC0 >> The Maven artifacts are staged at: >> https://repository.apache.org/content/repositories/orgapachehadoop-1330/ >> >> You can find my public key at: https://people.apache.org/~sunchao/KEYS >> >> Please evaluate the RC and vote. >> >> Thanks, >> Chao >> >
Re: [VOTE] Release Apache Hadoop 3.3.2 - RC0
I'll do my best to test this; I'm a bit broken right now. I think we should mention in a release notes that is the version of a log4j included in this and all previous releases is not vulnerable. But provide a list plus links to any that have been fixed On Fri, 10 Dec 2021 at 02:09, Chao Sun wrote: > Hi all, > > Sorry for the long delay. I've prepared RC0 for Hadoop 3.3.2 below: > > The RC is available at: > http://people.apache.org/~sunchao/hadoop-3.3.2-RC0/ > The RC tag is at: > https://github.com/apache/hadoop/releases/tag/release-3.3.2-RC0 > The Maven artifacts are staged at: > https://repository.apache.org/content/repositories/orgapachehadoop-1330/ > > You can find my public key at: https://people.apache.org/~sunchao/KEYS > > Please evaluate the RC and vote. > > Thanks, > Chao >
Re: [VOTE] Release Apache Hadoop 3.3.2 - RC0
Thanks a lot for pushing it forward! A few things I noticed that we should incorporate: 1. the overview page of the doc is for the Hadoop 3.0 release. It would be best to base the doc on top of Hadoop 3.3.0 overview page. (it's a miss on my part... The overview page of 3.3.1 wasn't updated) 2. ARM binaries is not included. For the 3.3.1 release, I had to run the create release script on an ARM machine separately to create the binary tarball. 3. the jdiff version https://github.com/apache/hadoop/blob/branch-3.3.2/hadoop-project-dist/pom.xml#L137 I am not sure exactly what this is used for but i think it should be updated to 3.3.2 (or 3.3.1) (it was updated in trunk but i forgot to update the branch-3.3) The 3.3.1 binary tarball is 577mb. The 3.3.2 RC0 is 608mb. I'm curious what are added. On Fri, Dec 10, 2021 at 10:09 AM Chao Sun wrote: > Hi all, > > Sorry for the long delay. I've prepared RC0 for Hadoop 3.3.2 below: > > The RC is available at: > http://people.apache.org/~sunchao/hadoop-3.3.2-RC0/ > The RC tag is at: > https://github.com/apache/hadoop/releases/tag/release-3.3.2-RC0 > The Maven artifacts are staged at: > https://repository.apache.org/content/repositories/orgapachehadoop-1330/ > > You can find my public key at: https://people.apache.org/~sunchao/KEYS > > Please evaluate the RC and vote. > > Thanks, > Chao >
Re: [VOTE] Release Apache Hadoop 3.3.2 - RC0
Thank you Chao Sun for preparing the RC. I've added your public key to the Hadoop KEYS file ( https://downloads.apache.org/hadoop/common/KEYS). Thanks, Akira On Fri, Dec 10, 2021 at 11:10 AM Chao Sun wrote: > Hi all, > > Sorry for the long delay. I've prepared RC0 for Hadoop 3.3.2 below: > > The RC is available at: > http://people.apache.org/~sunchao/hadoop-3.3.2-RC0/ > The RC tag is at: > https://github.com/apache/hadoop/releases/tag/release-3.3.2-RC0 > The Maven artifacts are staged at: > https://repository.apache.org/content/repositories/orgapachehadoop-1330/ > > You can find my public key at: https://people.apache.org/~sunchao/KEYS > > Please evaluate the RC and vote. > > Thanks, > Chao >
[VOTE] Release Apache Hadoop 3.3.2 - RC0
Hi all, Sorry for the long delay. I've prepared RC0 for Hadoop 3.3.2 below: The RC is available at: http://people.apache.org/~sunchao/hadoop-3.3.2-RC0/ The RC tag is at: https://github.com/apache/hadoop/releases/tag/release-3.3.2-RC0 The Maven artifacts are staged at: https://repository.apache.org/content/repositories/orgapachehadoop-1330/ You can find my public key at: https://people.apache.org/~sunchao/KEYS Please evaluate the RC and vote. Thanks, Chao
