Re: [VOTE] Release Apache Hadoop 3.1.2 - RC1

t):1" > > I think we are probably due for an overall review of LICENSE and NOTICE. I > saw some idiosyncrasies there but nothing that looked like a blocker. > > On Mon, Jan 28, 2019 at 10:20 PM Sunil G wrote: > >> Hi Folks, >> >> On behalf of Wangda

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

+1 . Thanks Wangda. - Sunil On Sat, Feb 2, 2019 at 3:54 AM Wangda Tan wrote: > Hi all, > > According to positive feedbacks from the thread [1] > > This is vote thread to start a new subproject named "hadoop-submarine" > which follows the release process already established for ozone. > > The

Re: [DISCUSS] Making submarine to different release model like Ozone

+1 from me on this. ML/DL is one of the fast growing areas and a runtime on YARN helps customers to have ML/DL workloads to run on same cluster where the ETL or other traditional big data workloads ingest or mine data. Faster release cadence can pace up the development for Submarine and more agile

[VOTE] Release Apache Hadoop 3.1.2 - RC1

Hi Folks, On behalf of Wangda, we have an RC1 for Apache Hadoop 3.1.2. The artifacts are available here: http://home.apache.org/~sunilg/hadoop-3.1.2-RC1/ The RC tag in git is release-3.1.2-RC1: https://github.com/apache/hadoop/commits/release-3.1.2-RC1 The maven artifacts are available via

[ANNOUNCE] Apache Hadoop 3.2.0 release

Greetings all, It gives me great pleasure to announce that the Apache Hadoop community has voted to release Apache Hadoop 3.2.0. Apache Hadoop 3.2.0 is the first release of Apache Hadoop 3.2 line for the year 2019, which includes 1092 fixes since previous Hadoop 3.1.0 release. Of these fixes:

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

Thanks everyone for helping to vote this release! With 7 binding votes, 10 non-binding votes and no veto, this vote stands passed, I'm going to work on staging the release. Thanks, Sunil On Tue, Jan 15, 2019 at 9:59 PM Weiwei Yang wrote: > +1 (binding) > > - Setup a cluster, run

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

ces [1] >>> 4. Executed basic HDFS commands >>> 5. Executed basic yarn example jobs >>> >>> Marton >>> >>> [1]: FTR: resources: >>> https://github.com/flokkr/k8s/tree/master/examples/hadoop , generator: >>> https://github.com

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

harder >> to work with hadoop packages; beacuse of the missing source attachments. >> >> example artifact which misses the sources: >> >> https://repository.apache.org/content/repositories/orgapachehadoop-1178/org/apache/hadoop/hadoop-mapreduce-client-core/3.2.0/ >> >> ch

[VOTE] Release Apache Hadoop 3.2.0 - RC1

Hi folks, Thanks to all of you who helped in this release [1] and for helping to vote for RC0. I have created second release candidate (RC1) for Apache Hadoop 3.2.0. Artifacts for this RC are available here: http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/ RC tag in git is

Re: Hadoop 3.2 Release Plan proposal

Hi Folks, Given all blockers are closed now, I am planning to do RC1 tomorrow. branch-3.2.0 remains closed for commit. Please let me know if any issues are missed. Thanks, Sunil G On Tue, Nov 13, 2018 at 8:11 PM Sunil G wrote: > Hi Folks, > > All blockers are closed by las

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

ases. For > example, when I change > yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take > effect unless I restart the RM. > > I am still investigating, but I thought I should bring this up asap. > > Thanks, > -Eric > > > > > On Friday, No

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

.default.disable_preemption, it doesn't take > effect unless I restart the RM. > > I am still investigating, but I thought I should bring this up asap. > > Thanks, > -Eric > > > > > On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G > wrote: > > > >

Re: Hadoop 3.2 Release Plan proposal

: > Another thing is that I see a bunch of jiras under HDFS-8707, which don't > have the Fix Version field listing 3.2, some have it just empty. > This means they will not be populated into release notes. > > Thanks, > --Konstantin > > On Thu, Oct 25, 2018 at 7:59 AM Sunil G wrote:

3.2.0 branch is closed for commits

Hi All, All blockers are closed for 3.2.0 as of now and RC is getting prepared. Hence 3.2.0 branch closed for commits. Please use branch-3.2 for any commit and set fixed version as 3.2.1 Thanks, Sunil

Re: Hadoop 3.2 Release Plan proposal

with this feature. > > Thanks, > --Konstantin > > On Mon, Oct 22, 2018 at 5:01 AM Steve Loughran > wrote: > >> its in. >> >> good catch! >> >> On 20 Oct 2018, at 01:35, Wei-Chiu Chuang > weic...@cloudera.com>> wrote: >> >> Thanks Suni

Re: Hadoop 3.2 Release Plan proposal

, 2018 at 11:43 PM Jonathan Bender wrote: > Hello, were there any updates around the 3.2.0 RC timing? All I see in the > current blockers are related to the new Submarine subproject, wasn't sure > if that is what is holding things up. > > Cheers, > Jon > > On Tue, Oct 2, 2018 at

Re: Hadoop 3.2 Release Plan proposal

t; > thanks > - Robert > > On Tue, Oct 2, 2018 at 11:44 AM Aaron Fabbri > wrote: > >> Trunk is not building for me.. Did you miss a 3.2.0-SNAPSHOT in the >> top-level pom.xml? >> >> >> On Tue, Oct 2, 2018 at 10:16 AM Sunil G wrote: >> >> >

Re: Hadoop 3.2 Release Plan proposal

Hi All As mentioned in earlier mail, I have cut branch-3.2 and reset trunk to 3.3.0-SNAPSHOT. I will share the RC details sooner once all necessary patches are pulled into branch-3.2. Thank You - Sunil On Mon, Sep 24, 2018 at 2:00 PM Sunil G wrote: > Hi All > > We are now down to

Re: Hadoop 3.2 Release Plan proposal

Hi All We are now down to the last Blocker and HADOOP-15407 is merged to trunk. Thanks for the support. *Plan for RC* 3.2 branch cut and reset trunk : *25th Tuesday* RC0 for 3.2: *28th Friday* Thank You Sunil On Mon, Sep 17, 2018 at 3:21 PM Sunil G wrote: > Hi All > > We are

Re: Hadoop 3.2 Release Plan proposal

Hi All We are down to 3 Blockers and 4 Critical now. Thanks all of you for helping in this. I am following up on these tickets, once its closed we will cut the 3.2 branch. Thanks Sunil Govindan On Wed, Sep 12, 2018 at 5:10 PM Sunil G wrote: > Hi All, > > Inline with the ori

Re: [Vote] Merge discussion for Node attribute support feature YARN-3409

Thanks Sean YARN-8768 is raised and we will get this in very soon. - Sunil On Wed, Sep 12, 2018 at 9:16 PM Sean Mackrory wrote: > `mvn install` fails on trunk right now due to javadoc error in yarn-api. Is > that related? If so, can we revert or fix ASAP please? > > On Wed, Sep 12, 2018 at

Re: Hadoop 3.2 Release Plan proposal

them to get in by Code Freeze date. [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND priority in (Blocker, Critical) AND resolution = Unresolved AND "Target Version/s" = 3.2.0 ORDER BY priority DESC Thanks, Sunil On Thu, Aug 30, 2018 at 9:59 PM Sunil G wrote: > Hi All, > &

Re: [Vote] Merge discussion for Node attribute support feature YARN-3409

+1 for the merge. - Sunil On Wed, Sep 5, 2018 at 6:01 PM Naganarasimha Garla < naganarasimha...@apache.org> wrote: > Hi All, > Thanks for feedback folks, based on the positive response starting > a Vote thread for merging YARN-3409 to master. > > Regards, > + Naga & Sunil > > On Wed,

Re: [Discuss] Merge discussion for Node attribute support feature YARN-3409

+1 for merge. Quickly checked all basic test runs in the branch - Add/replace/remove of attributes work good. - scheduler can now handle attribute based placement constraints. - tested DS shell with various constructs like java=1.8, python!=3, AND/OR constraints etc. - Documentation on attributes

Re: HADOOP-14163 proposal for new hadoop.apache.org

+1. Looks really good. - Sunil On Mon, Sep 3, 2018 at 10:51 AM Vinayakumar B wrote: > +1, New site looks great. > > Just one nit in README > '--refresh' flag for hugo server no longer available. > > -Vinay > > On Mon, 3 Sep 2018, 10:21 am Shashikant Banerjee, < > sbaner...@hortonworks.com> >

Re: Hadoop 3.2 Release Plan proposal

Sunil On Tue, Aug 14, 2018 at 10:30 PM Sunil G wrote: > Hi All, > > Thanks for the feedbacks. Inline with earlier communication dated 17th > July 2018, I would like to provide some updates. > > We are approaching previously proposed feature freeze date (Aug 21, about > 7

Re: Hadoop 3.2 Release Plan proposal

ot;Target Version/s" = 3.2.0 ORDER BY priority DESC Thanks, Sunil On Fri, Jul 20, 2018 at 8:03 AM Sunil G wrote: > Thanks Subru for the thoughts. > One of the main reason for a major release is to push out critical > features with a faster cadence to the users. If we are pulling mo

Re: YARN SLS improving idea

Hi Sichen 1. Add input support for the scheduling request format. Yes. I suppose this change is in SLS end (client to generate requests) 2. Add support for scheduling request resource format in NMSim. Makes sense. 3. Adding scheduling request support for the Capacity Scheduler(maybe it is already

Re: [VOTE] Release Apache Hadoop 3.1.1 - RC0

Thanks Wangda for the initiative. +1 for this RC. I have tested this RC built from source file. - Also ran few MR apps and verified both new YARN UI and old RM UI. - Tested Application priority and timeout - Inter Queue and Intra Queue Preemption cases were also verified - Tested

Re: Hadoop 3.2 Release Plan proposal

n Wed, Jul 18, 2018 at 6:20 PM Aaron Fabbri > wrote: > >> On Tue, Jul 17, 2018 at 7:21 PM Steve Loughran >> wrote: >> >> > >> > >> > On 16 Jul 2018, at 23:45, Sunil G > > sun...@apache.org>> wrote: >> > >> > I would al

Hadoop 3.2 Release Plan proposal

Hi All, To continue a faster cadence of releases to accommodate more features, we could plan a Hadoop 3.2 release around August end. To start the process sooner, and to establish a timeline, I propose to target Hadoop 3.2.0 release by August end 2018. (About 1.5 months from now). I would

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

Thanks. These patches are now restored. - Sunil On Fri, Jul 6, 2018 at 11:14 AM Vinod Kumar Vavilapalli wrote: > +1 > > Thanks > +Vinod > > > On Jul 6, 2018, at 11:12 AM, Sunil G wrote: > > I just checked. YARN-7556 and YARN-7451 can be cherry-picked. &g

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

I just checked. YARN-7556 and YARN-7451 can be cherry-picked. I cherry-picked in my local and compiled. Things are good. I can push this now which will restore trunk to its original. I can do this if there are no objection. - Sunil On Fri, Jul 6, 2018 at 11:10 AM Arpit Agarwal wrote: >

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

Yes these patches can be cherry-picked. git cherry-pick 17262470246232d0f0651d627a4961e55b1efe6a git cherry-pick 99febe7fd50c31c0f5dd40fa7f376f2c1f64f8c3 I will try this now and will compile. - Sunil On Fri, Jul 6, 2018 at 10:59 AM Giovanni Matteo Fumarola < giovanni.fumar...@gmail.com>

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

+1 for this. - Sunil On Thu, Jul 5, 2018 at 2:37 PM Subru Krishnan wrote: > Folks, > > There was a merge commit accidentally pushed to trunk, you can find the > details in the mail thread [1]. > > I have raised an INFRA ticket [2] to reset/force push to clean up trunk. > > Can we have a quick

Re: Why are some Jenkins builds failing in Yarn-UI

Hi Steve I was recently looking into this failure and this is due to a repo hosting problem. It seems Bower team has deprecated old registry url https://bower.herokuapp.com and need to change registry url to https://registry.bower.io. YARN-8457 addressed this problem is backported till

Re: [VOTE] Release Apache Hadoop 2.8.4 (RC0)

+1 (binding) 1. Build package from src 2. Ran few MR jobs and verified checked App Priority cases 3. Node Label basic functions are ok. Thanks Sunil On Tue, May 8, 2018 at 11:11 PM 俊平堵 wrote: > Hi all, > I've created the first release candidate (RC0) for Apache

Re: Apache Hadoop 3.1.1 release plan

Thanks Brahma. Yes, Billie is reviewing YARN-8265 and I am helping in YARN-8236. - Sunil On Thu, May 10, 2018 at 2:25 PM Brahma Reddy Battula < brahmareddy.batt...@huawei.com> wrote: > Thanks Wangda Tan for driving the 3.1.1 release.Yes,This can be better > addition to 3.1 line release for

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

+1 (binding) On Mon 2 Apr, 2018, 12:24 Sunil G, <sun...@apache.org> wrote: > Thanks Wangda for initiating the release. > > I tested this RC built from source file. > > >- Tested MR apps (sleep, wc) and verified both new YARN UI and old RM >UI. >

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

Thanks Wangda for initiating the release. I tested this RC built from source file. - Tested MR apps (sleep, wc) and verified both new YARN UI and old RM UI. - Below feature sanity is done - Application priority - Application timeout - Intra Queue preemption with priority

Re: [VOTE] Merge YARN-6592 feature branch to trunk

+1 - Sunil On Fri, Jan 26, 2018 at 8:58 PM Arun Suresh wrote: > Hello yarn-dev@ > > Based on the positive feedback from the DISCUSS thread [1], I'd like to > start a formal vote to merge YARN-6592 [2] to trunk. The vote will run for > 5 days, and will end Jan 31 7:30AM

Re: [DISCUSS] Merge YARN-6592 to trunk

+1. Thanks Arun. I did manual testing for check affinity and anti-affinity features with placement allocator. Also checked SLS to see any performance regression, and there are not much difference as Arun mentioned. Thanks all the folks for working on this. Kudos! - Sunil On Fri, Jan 26, 2018

Re: Missing some trunk commit history

Hi Eric. A branch merge has happened during that time, and hence you might have seen some old commits from that branch. If you go down further, you could see those commits. Copied from my git log: commit 40b0045ebe0752cd3d1d09be00acbabdea983799 Author: Weiwei Yang Date: Wed

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

TestCapacitySchedulerPerf#testUserLimitThroughputForTwoResources. Results are a bit off w.r.t 2.8 (~5% less). I will open a ticket and investigate by doing more tests to see if its to be addressed or not. - Sunil G On Sat, Dec 9, 2017 at 2:01 AM Andrew Wang <andrew.w...@cloudera.com> wrote: &g

Re: [VOTE] Release Apache Hadoop 2.8.3 (RC0)

NodeLabel scenarios. - Added some labels to couple of nodes - Verified old UI for labels - Submitted apps to labelled cluster and it works fine. - Also performed few cli commands related to nodelabel - Test basic HA cases Thanks Sunil G On Tue, Dec 5, 2017 at 3:28 PM

Re: [VOTE] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881) to trunk

Thank You all. We merged the branch to trunk and updated jiras accordingly. Thanks everyone who helped in this feature. - Sunil and Wangda On Thu, Dec 7, 2017 at 10:13 PM Sunil G <sun...@apache.org> wrote: > And lastly +1 (binding) from myself. > Vote passes with 6 (

Re: [VOTE] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881) to trunk

.org> wrote: > >> +1 >> >> On Nov 30, 2017 7:26 AM, "Sunil G" <sun...@apache.org> wrote: >> >> > Hi All, >> > >> > >> > Based on the discussion at [1], I'd like to start a vote to merge >> feature >> >

Re: [DISCUSS] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881) to trunk

00, Rohith Sharma K S < >> rohithsharm...@apache.org>, wrote: >> +1, thanks Sunil for working on this feature! >> >> -Rohith Sharma K S >> >> On 24 November 2017 at 23:19, Sunil G <sun...@apache.org> wrote: >> >> Hi All, >>

[VOTE] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881) to trunk

Hi All, Based on the discussion at [1], I'd like to start a vote to merge feature branch YARN-5881 to trunk. Vote will run for 7 days, ending Wednesday Dec 6 at 6:00PM PDT. This branch adds support to configure queue capacity as absolute resource in capacity scheduler. This will help admins

Re: [DISCUSS] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881) to trunk

any more even though it has pending resources. > > Thanks,Eric > > > From: Sunil G <sun...@apache.org> > To: "yarn-...@hadoop.apache.org" <yarn-...@hadoop.apache.org>; Hadoop > Common <common-dev@hadoop.apache.org>; Hdfs-dev < > hdfs-...@h

[DISCUSS] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881) to trunk

81 [3] : https://issues.apache.org/jira/browse/YARN-7533 [4] : https://issues.apache.org/jira/browse/YARN-7510 Thanks, Sunil G and Wangda Tan

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

+1 (binding) Built from source. - Tested few cases in an HA cluster and tried to do failover by using rmadmin commands etc. This seems works fine including submitting apps. - I also tested many MR apps and all are running fine w/o any issues. - Majorly tested below feature sanity

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC2)

Hi Mukul We have an RC3 release thread started as RC2 had an issue reported. Kindly help to check the same - Sunil On Tue, Nov 14, 2017 at 3:00 PM Mukul Kumar Singh wrote: > +1 (non-binding) > > I built from source on Mac OS X 10.13.1 Java 1.8.0_111 > > - Deployed on a

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC2)

+1 (binding) Deployed cluster built from source. - Tested few cases in an HA cluster and tried to do failover by using rmadmin commands etc. This seems works fine including submitting apps. - I also tested many MR apps and all are running fine w/o any issues. - Majorly tested below

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC0)

Hi Subru and Arun. Thanks for driving 2.9 release. Great work! I installed cluster built from source. - Ran few MR jobs with application priority enabled. Runs fine. - Accessed new UI and it also seems fine. However I am also getting same issue as Rohith reported. - Started an HA cluster -

Re: [VOTE] Merge yarn-native-services branch into trunk

g system if disabled. > > Special thanks to a team of folks who worked hard towards this: Billie > Rinaldi, Gour Saha, Vinod Kumar Vavilapalli, Jonathan Maron, Rohith Sharma > K S, Sunil G, Akhil PB, Eric Yang. This effort could not be possible > without their ideas and hard work. &g

Re: [VOTE] Merge YARN-3926 (resource profile) to trunk

Hi Daniel Thank you very much for the support. * When you say that the feature can be turned off, do you mean resource types or resource profiles? I know there's an off-by-default property that governs resource profiles, but I didn't see any way to turn off resource types.

Re: [VOTE] Merge YARN-3926 (resource profile) to trunk

Thank you very much Varun Vasudev, Wangda Tan, Daniel and all the folks who helped in getting this feature in this level. Starting with my +1 (binding). # Tested a 5 node cluster with resource profiles enabled/disabled (feature is disabled by default) # All apis added are marked as

Re: [VOTE] Merge feature branch YARN-5355 (Timeline Service v2) to trunk

Thank you very much Vrushali, Rohith, Varun and other folks who made this happen. Great work, really appreciate the same!! +1 (binding) from my side: # Tested ATSv2 cluster in a secure cluster. Ran some basic jobs # Accessed new YARN UI which shows various flows/flow activity etc. Seems fine. #

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

Thanks Konstantin +1 (binding) 1. Build tar ball from source package 2. Ran basic MR jobs and verified UI. 3. Enabled node labels and ran sleep job. Works fine. 4. Verified CLI commands related to node labels and its working fine. 5. RM WorkPreserving restart cases are also verified, and looks

[jira] [Created] (HADOOP-14658) branch-2 compilation is broken in hadoop-azure

Sunil G created HADOOP-14658: Summary: branch-2 compilation is broken in hadoop-azure Key: HADOOP-14658 URL: https://issues.apache.org/jira/browse/HADOOP-14658 Project: Hadoop Common Issue Type

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha4-RC0

+1 (non-binding) Tested from building tar ball from binary. - Ran few MR jobs with node labels - Verified old and new YARN ui - Checked preemption as well Thanks Sunil On Fri, Jun 30, 2017 at 8:11 AM Andrew Wang wrote: > Hi all, > > As always, thanks to the many,

Upgrading minimum version of Maven to 3.1 from 3.0

Hi Folks, Recently we were doing build framework up-gradation for Yarn Ui. In order to compile yarn-ui on various architectures, we were using frontend-maven-plugin 0.0.22 version. However build is failing in *ppc64le.* If we could use latest version of frontend-maven-plugin, we could resolve