Re: [VOTE] Release Hadoop-3.1.3-RC0
+1 (binding) Thanks Zhankun for all of your hard work on this release. I downloaded and built the source and ran it on an insecure multi-node pseudo cluster. I performed various YARN manual tests, including creating custom resources, creating queue submission ACLs, and queue refreshes. One concern is that preemption does not seem to be working when only the custom resources are over the queue capacity, but I don't think this is something introduced with this release. -Eric On Thursday, September 12, 2019, 3:04:44 AM CDT, Zhankun Tang wrote: Hi folks, Thanks to everyone's help on this release. Special thanks to Rohith, Wei-Chiu, Akira, Sunil, Wangda! I have created a release candidate (RC0) for Apache Hadoop 3.1.3. The RC release artifacts are available at: http://home.apache.org/~ztang/hadoop-3.1.3-RC0/ The maven artifacts are staged at: https://repository.apache.org/content/repositories/orgapachehadoop-1228/ The RC tag in git is here: https://github.com/apache/hadoop/tree/release-3.1.3-RC0 And my public key is at: https://dist.apache.org/repos/dist/release/hadoop/common/KEYS *This vote will run for 7 days, ending on Sept.19th at 11:59 pm PST.* For the testing, I have run several Spark and distributed shell jobs in my pseudo cluster. My +1 (non-binding) to start. BR, Zhankun On Wed, 4 Sep 2019 at 15:56, zhankun tang wrote: > Hi all, > > Thanks for everyone helping in resolving all the blockers targeting Hadoop > 3.1.3[1]. We've cleaned all the blockers and moved out non-blockers issues > to 3.1.4. > > I'll cut the branch today and call a release vote soon. Thanks! > > > [1]. https://s.apache.org/5hj5i > > BR, > Zhankun > > > On Wed, 21 Aug 2019 at 12:38, Zhankun Tang wrote: > >> Hi folks, >> >> We have Apache Hadoop 3.1.2 released on Feb 2019. >> >> It's been more than 6 months passed and there're >> >> 246 fixes[1]. 2 blocker and 4 critical Issues [2] >> >> (As Wei-Chiu Chuang mentioned, HDFS-13596 will be another blocker) >> >> >> I propose my plan to do a maintenance release of 3.1.3 in the next few >> (one or two) weeks. >> >> Hadoop 3.1.3 release plan: >> >> Code Freezing Date: *25th August 2019 PDT* >> >> Release Date: *31th August 2019 PDT* >> >> >> Please feel free to share your insights on this. Thanks! >> >> >> [1] https://s.apache.org/zw8l5 >> >> [2] https://s.apache.org/fjol5 >> >> >> BR, >> >> Zhankun >> > - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree
+1 (binding) Thanks to the Ozone folks for their efforts at maintaining good separation with HDFS and common. I took a lot of heat for the unpopular opinion that they should be separate, so I am glad the process has worked out well for both codebases. It looks like my concerns were addressed and I appreciate it. It is cool to see the evolution here. Aaron On Thu, Sep 19, 2019 at 3:37 AM Steve Loughran wrote: > in that case, > > +1 from me (binding) > > On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton wrote: > > > > one thing to consider here as you are giving up your ability to make > > > changes in hadoop-* modules, including hadoop-common, and their > > > dependencies, in sync with your own code. That goes for filesystem > > contract > > > tests. > > > > > > are you happy with that? > > > > > > Yes. I think we can live with it. > > > > Fortunatelly the Hadoop parts which are used by Ozone (security + rpc) > > are stable enough, we didn't need bigger changes until now (small > > patches are already included in 3.1/3.2). > > > > I think it's better to use released Hadoop bits in Ozone anyway, and > > worst (best?) case we can try to do more frequent patch releases from > > Hadoop (if required). > > > > > > m. > > > > > > >
Re: [VOTE] Release Hadoop-3.1.3-RC0
+1 (binding) - Downloaded source, setup a single node cluster - Verified basic HDFS operations, put/get/cat etc - Verified basic YARN restful apis, cluster/nodes/scheduler, all seem good - Run several distributed shell job Thanks Weiwei On Sep 19, 2019, 4:28 PM +0800, Sunil Govindan , wrote: > +1 (binding) > > Thanks Zhankun for putting up the release. Thanks for leading this. > > - verified signature > - ran a local cluster from tar ball > - ran some MR jobs > - perform CLI ops, and looks good > - UI seems fine > > Thanks > Sunil > > On Thu, Sep 12, 2019 at 1:34 PM Zhankun Tang wrote: > > > Hi folks, > > > > Thanks to everyone's help on this release. Special thanks to Rohith, > > Wei-Chiu, Akira, Sunil, Wangda! > > > > I have created a release candidate (RC0) for Apache Hadoop 3.1.3. > > > > The RC release artifacts are available at: > > http://home.apache.org/~ztang/hadoop-3.1.3-RC0/ > > > > The maven artifacts are staged at: > > https://repository.apache.org/content/repositories/orgapachehadoop-1228/ > > > > The RC tag in git is here: > > https://github.com/apache/hadoop/tree/release-3.1.3-RC0 > > > > And my public key is at: > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > *This vote will run for 7 days, ending on Sept.19th at 11:59 pm PST.* > > > > For the testing, I have run several Spark and distributed shell jobs in my > > pseudo cluster. > > > > My +1 (non-binding) to start. > > > > BR, > > Zhankun > > > > On Wed, 4 Sep 2019 at 15:56, zhankun tang wrote: > > > > > Hi all, > > > > > > Thanks for everyone helping in resolving all the blockers targeting > > Hadoop > > > 3.1.3[1]. We've cleaned all the blockers and moved out non-blockers > > issues > > > to 3.1.4. > > > > > > I'll cut the branch today and call a release vote soon. Thanks! > > > > > > > > > [1]. https://s.apache.org/5hj5i > > > > > > BR, > > > Zhankun > > > > > > > > > On Wed, 21 Aug 2019 at 12:38, Zhankun Tang wrote: > > > > > > > Hi folks, > > > > > > > > We have Apache Hadoop 3.1.2 released on Feb 2019. > > > > > > > > It's been more than 6 months passed and there're > > > > > > > > 246 fixes[1]. 2 blocker and 4 critical Issues [2] > > > > > > > > (As Wei-Chiu Chuang mentioned, HDFS-13596 will be another blocker) > > > > > > > > > > > > I propose my plan to do a maintenance release of 3.1.3 in the next few > > > > (one or two) weeks. > > > > > > > > Hadoop 3.1.3 release plan: > > > > > > > > Code Freezing Date: *25th August 2019 PDT* > > > > > > > > Release Date: *31th August 2019 PDT* > > > > > > > > > > > > Please feel free to share your insights on this. Thanks! > > > > > > > > > > > > [1] https://s.apache.org/zw8l5 > > > > > > > > [2] https://s.apache.org/fjol5 > > > > > > > > > > > > BR, > > > > > > > > Zhankun > > > > > > > > >
[jira] [Resolved] (HADOOP-16556) Fix some LGTM alerts
[ https://issues.apache.org/jira/browse/HADOOP-16556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16556. - Fix Version/s: 3.3.0 Assignee: Malcolm Taylor Resolution: Fixed > Fix some LGTM alerts > > > Key: HADOOP-16556 > URL: https://issues.apache.org/jira/browse/HADOOP-16556 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.3.0 >Reporter: Malcolm Taylor >Assignee: Malcolm Taylor >Priority: Minor > Fix For: 3.3.0 > > > LGTM analysis of Hadoop has raised some alerts > ([https://lgtm.com/projects/g/apache/hadoop/?mode=tree).] This issue is to > fix some of the more straightforward ones. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree
in that case, +1 from me (binding) On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton wrote: > > one thing to consider here as you are giving up your ability to make > > changes in hadoop-* modules, including hadoop-common, and their > > dependencies, in sync with your own code. That goes for filesystem > contract > > tests. > > > > are you happy with that? > > > Yes. I think we can live with it. > > Fortunatelly the Hadoop parts which are used by Ozone (security + rpc) > are stable enough, we didn't need bigger changes until now (small > patches are already included in 3.1/3.2). > > I think it's better to use released Hadoop bits in Ozone anyway, and > worst (best?) case we can try to do more frequent patch releases from > Hadoop (if required). > > > m. > > >
Re: [VOTE] Release Apache Hadoop 3.2.1 - RC0
Hi Rohith, Thanks for driving this release. +1 (binding) - built from the source on windows machine. - created a pseudo cluster. - ran PI job. - checked basic metrics with ATSv2 enabled. On Thu, Sep 19, 2019 at 12:30 PM Sunil Govindan wrote: > Hi Rohith > > Thanks for putting this together, appreciate the same. > > +1 (binding) > > - verified signature > - brought up a cluster from the tar ball > - Ran some basic MR jobs > - RM UI seems fine (old and new) > > > Thanks > Sunil > > On Wed, Sep 11, 2019 at 12:56 PM Rohith Sharma K S < > rohithsharm...@apache.org> wrote: > > > Hi folks, > > > > I have put together a release candidate (RC0) for Apache Hadoop 3.2.1. > > > > The RC is available at: > > http://home.apache.org/~rohithsharmaks/hadoop-3.2.1-RC0/ > > > > The RC tag in git is release-3.2.1-RC0: > > https://github.com/apache/hadoop/tree/release-3.2.1-RC0 > > > > > > The maven artifacts are staged at > > https://repository.apache.org/content/repositories/orgapachehadoop-1226/ > > > > You can find my public key at: > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > This vote will run for 7 days(5 weekdays), ending on 18th Sept at 11:59 > pm > > PST. > > > > I have done testing with a pseudo cluster and distributed shell job. My > +1 > > to start. > > > > Thanks & Regards > > Rohith Sharma K S > > > -- Regards, Abhishek Modi
Re: [VOTE] Release Hadoop-3.1.3-RC0
+1 (binding) Thanks Zhankun for putting up the release. Thanks for leading this. - verified signature - ran a local cluster from tar ball - ran some MR jobs - perform CLI ops, and looks good - UI seems fine Thanks Sunil On Thu, Sep 12, 2019 at 1:34 PM Zhankun Tang wrote: > Hi folks, > > Thanks to everyone's help on this release. Special thanks to Rohith, > Wei-Chiu, Akira, Sunil, Wangda! > > I have created a release candidate (RC0) for Apache Hadoop 3.1.3. > > The RC release artifacts are available at: > http://home.apache.org/~ztang/hadoop-3.1.3-RC0/ > > The maven artifacts are staged at: > https://repository.apache.org/content/repositories/orgapachehadoop-1228/ > > The RC tag in git is here: > https://github.com/apache/hadoop/tree/release-3.1.3-RC0 > > And my public key is at: > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > *This vote will run for 7 days, ending on Sept.19th at 11:59 pm PST.* > > For the testing, I have run several Spark and distributed shell jobs in my > pseudo cluster. > > My +1 (non-binding) to start. > > BR, > Zhankun > > On Wed, 4 Sep 2019 at 15:56, zhankun tang wrote: > > > Hi all, > > > > Thanks for everyone helping in resolving all the blockers targeting > Hadoop > > 3.1.3[1]. We've cleaned all the blockers and moved out non-blockers > issues > > to 3.1.4. > > > > I'll cut the branch today and call a release vote soon. Thanks! > > > > > > [1]. https://s.apache.org/5hj5i > > > > BR, > > Zhankun > > > > > > On Wed, 21 Aug 2019 at 12:38, Zhankun Tang wrote: > > > >> Hi folks, > >> > >> We have Apache Hadoop 3.1.2 released on Feb 2019. > >> > >> It's been more than 6 months passed and there're > >> > >> 246 fixes[1]. 2 blocker and 4 critical Issues [2] > >> > >> (As Wei-Chiu Chuang mentioned, HDFS-13596 will be another blocker) > >> > >> > >> I propose my plan to do a maintenance release of 3.1.3 in the next few > >> (one or two) weeks. > >> > >> Hadoop 3.1.3 release plan: > >> > >> Code Freezing Date: *25th August 2019 PDT* > >> > >> Release Date: *31th August 2019 PDT* > >> > >> > >> Please feel free to share your insights on this. Thanks! > >> > >> > >> [1] https://s.apache.org/zw8l5 > >> > >> [2] https://s.apache.org/fjol5 > >> > >> > >> BR, > >> > >> Zhankun > >> > > >
Re: [VOTE] Release Apache Hadoop 3.2.1 - RC0
Hi Rohith Thanks for putting this together, appreciate the same. +1 (binding) - verified signature - brought up a cluster from the tar ball - Ran some basic MR jobs - RM UI seems fine (old and new) Thanks Sunil On Wed, Sep 11, 2019 at 12:56 PM Rohith Sharma K S < rohithsharm...@apache.org> wrote: > Hi folks, > > I have put together a release candidate (RC0) for Apache Hadoop 3.2.1. > > The RC is available at: > http://home.apache.org/~rohithsharmaks/hadoop-3.2.1-RC0/ > > The RC tag in git is release-3.2.1-RC0: > https://github.com/apache/hadoop/tree/release-3.2.1-RC0 > > > The maven artifacts are staged at > https://repository.apache.org/content/repositories/orgapachehadoop-1226/ > > You can find my public key at: > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > This vote will run for 7 days(5 weekdays), ending on 18th Sept at 11:59 pm > PST. > > I have done testing with a pseudo cluster and distributed shell job. My +1 > to start. > > Thanks & Regards > Rohith Sharma K S >
[jira] [Created] (HADOOP-16587) Make AAD endpoint configurable on all Auth flows
Bilahari T H created HADOOP-16587: - Summary: Make AAD endpoint configurable on all Auth flows Key: HADOOP-16587 URL: https://issues.apache.org/jira/browse/HADOOP-16587 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Reporter: Bilahari T H Assignee: Bilahari T H Make AAD endpoint configurable on all Auth flows. Currently auth endpoint is hard coded for refreshtoken flow and MSI flow. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.2.1 - RC0
Thanks Brahma for voting and bringing this to my attention! On Thu, 19 Sep 2019 at 11:28, Brahma Reddy Battula wrote: > RohithThanks for driving the release > > +1 (Binding). > > --Built from the source > --Installed pseudo cluster > --Verified Basic hdfs shell command > --Ran Pi jobs > --Browsed the UI > > > *Rolling Upgrade:* > Following issue could have been merged.With out this, need to disable > token till rolling upgrade finalised. (Since one of main rolling upgrade > issue already merged (HDFS-13596)). > https://issues.apache.org/jira/browse/HDFS-14509 > This issue marked as blocker for 2.10 and still open!. Can anyone HDFS folks confirms this whether is is blocker for *hadoop-3.2.1* release? -Rohith Sharma K S
