Re: [DISCUSS] Apache YARN committers/contribu­t­ors meetup #5

Thanks for hosting this. Should people respond to this email for date preferences or do you want to use something like doodle? On Sat, Nov 11, 2017 at 8:59 AM Daniel Templeton wrote: > Sounds like there are no other volunteers, so we're happy to host here > at Cloudera.

[jira] [Resolved] (YARN-2814) RM: Clean-up the handling of "fatal" events

[ https://issues.apache.org/jira/browse/YARN-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla resolved YARN-2814. Resolution: Not A Problem I believe most of the items I called out had been fixed over

[jira] [Created] (YARN-7057) FSAppAttempt#getResourceUsage doesn't need to considered resources queued for preemption

Karthik Kambatla created YARN-7057: -- Summary: FSAppAttempt#getResourceUsage doesn't need to considered resources queued for preemption Key: YARN-7057 URL: https://issues.apache.org/jira/browse/YARN-7057

[jira] [Created] (YARN-7049) FSAppAttempt fields tracking containers/resources queued for preemption can use better names

Karthik Kambatla created YARN-7049: -- Summary: FSAppAttempt fields tracking containers/resources queued for preemption can use better names Key: YARN-7049 URL: https://issues.apache.org/jira/browse/YARN-7049

[jira] [Resolved] (YARN-6590) ResourceManager Master/Slave transition make all applications killed

[ https://issues.apache.org/jira/browse/YARN-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla resolved YARN-6590. Resolution: Invalid [~wuchang1989], in the future, please reach out on our user mailing list

[jira] [Created] (YARN-6582) FSAppAttempt demand can be updated atomically in updateDemand()

Karthik Kambatla created YARN-6582: -- Summary: FSAppAttempt demand can be updated atomically in updateDemand() Key: YARN-6582 URL: https://issues.apache.org/jira/browse/YARN-6582 Project: Hadoop YARN

[jira] [Resolved] (YARN-6448) Continuous scheduling thread crashes while sorting nodes

[ https://issues.apache.org/jira/browse/YARN-6448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla resolved YARN-6448. Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.9.0 Just committed

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

+1 (binding) * Built from source * Started a pseudo-distributed cluster with fairscheduler. * Ran sample jobs * Verified WebUI On Wed, Mar 22, 2017 at 11:56 AM, varunsax...@apache.org < varun.saxena.apa...@gmail.com> wrote: > Thanks Junping for creating the release. > > +1 (non-binding) > > *

[jira] [Created] (YARN-6246) Move app starvation identification out of the update thread

Karthik Kambatla created YARN-6246: -- Summary: Move app starvation identification out of the update thread Key: YARN-6246 URL: https://issues.apache.org/jira/browse/YARN-6246 Project: Hadoop YARN

[jira] [Resolved] (YARN-6230) Failing unit test TestFairScheduler.testMoveWouldViolateMaxResourcesConstraints

[ https://issues.apache.org/jira/browse/YARN-6230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla resolved YARN-6230. Resolution: Duplicate YARN-6231 duplicates this issue, but we have made more progress

[jira] [Created] (YARN-6210) FS: Node reservations can interfere with preemption

Karthik Kambatla created YARN-6210: -- Summary: FS: Node reservations can interfere with preemption Key: YARN-6210 URL: https://issues.apache.org/jira/browse/YARN-6210 Project: Hadoop YARN

[jira] [Resolved] (YARN-6038) Check other resource requests if cannot match the first one while identifying containers to preempt

[ https://issues.apache.org/jira/browse/YARN-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla resolved YARN-6038. Resolution: Duplicate This is a duplicate of YARN-5832. > Check other resource reque

[jira] [Created] (YARN-6194) Cluster capacity in SchedulingPolicy is updated only on allocation file reload

Karthik Kambatla created YARN-6194: -- Summary: Cluster capacity in SchedulingPolicy is updated only on allocation file reload Key: YARN-6194 URL: https://issues.apache.org/jira/browse/YARN-6194

[jira] [Created] (YARN-6193) DominantResourceFairnessPolicy#checkIfUsageOverFairShare considers all resources instead of just dominant resource

Karthik Kambatla created YARN-6193: -- Summary: DominantResourceFairnessPolicy#checkIfUsageOverFairShare considers all resources instead of just dominant resource Key: YARN-6193 URL: https://issues.apache.org/jira

[jira] [Resolved] (YARN-4067) available resource could be set negative

[ https://issues.apache.org/jira/browse/YARN-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla resolved YARN-4067. Resolution: Duplicate YARN-3933 is addressing the same issue. > available resource co

[jira] [Resolved] (YARN-4045) Negative avaialbleMB is being reported for root queue.

[ https://issues.apache.org/jira/browse/YARN-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla resolved YARN-4045. Resolution: Duplicate Looks like this is a duplicate of YARN-3933. > Negative avaialbl

[jira] [Created] (YARN-6163) FS Preemption is a trickle for severely starved applications

Karthik Kambatla created YARN-6163: -- Summary: FS Preemption is a trickle for severely starved applications Key: YARN-6163 URL: https://issues.apache.org/jira/browse/YARN-6163 Project: Hadoop YARN

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha2 RC0

iles within jars > should not reset the clock on the VOTE. -C > > > On Wed, Jan 25, 2017 at 11:14 AM, Karthik Kambatla <ka...@cloudera.com> > > wrote: > > > >> Thanks for driving the alphas, Andrew. I don't see the need to restart > the > >> vote an

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha2 RC0

Thanks for driving the alphas, Andrew. I don't see the need to restart the vote and I feel it is okay to fix the minor issues before releasing. +1 (binding). Downloaded source, stood up a pseudo-distributed cluster with FairScheduler, ran example jobs, and played around with the UI. Thanks

Re: [VOTE] Release cadence and EOL

Given the discussions, I feel we are not ready for VOTE on this yet. Sangjin, should we go back to the DISCUSS thread? IMO, these are guidelines and not policies we want to enforce. Maybe, the text should say something along the lines of: "The Hadoop community is inclined to". And, maybe, a

Re: [VOTE] Release cadence and EOL

+1 I would also like to see some process guidelines. I should have brought this up on the discussion thread, but I thought of them only now :( - Is an RM responsible for all maintenance releases against a minor release, or finding another RM to drive a maintenance release? In the past,

Re: [DISCUSS] Release cadence and EOL

topic has aged quite a bit in the discussion thread. Should > we take it to a vote? Do we need additional discussions? > > Regards, > Sangjin > > On Wed, Nov 9, 2016 at 11:11 PM, Karthik Kambatla <ka...@cloudera.com> > wrote: > >> Fair points, Sangjin and Andrew.

Merging YARN-4752 (FS preemption) to branch-2

Hi devs We merged YARN-4752 (FS preemption), which was one commit, to trunk more than a month ago and it has been doing okay. I am planning to merge it to branch-2 (filed YARN-6076). Since it is one commit and shouldn't even have been a branch, I am skipping the VOTE thread. Let me know if you

[jira] [Created] (YARN-6076) Backport YARN-4752 (FS preemption changes) to branch-2

Karthik Kambatla created YARN-6076: -- Summary: Backport YARN-4752 (FS preemption changes) to branch-2 Key: YARN-6076 URL: https://issues.apache.org/jira/browse/YARN-6076 Project: Hadoop YARN

Re: apply for "assign to myself" privilege of YARN project

Done. On Sun, Jan 8, 2017 at 10:27 PM, Yuanbo Liu wrote: > Hi, developers > I'd like to contribute to YARN project. Could anyone authorize "assign to > myself" to me. Thanks in advance. >

[jira] [Resolved] (YARN-6021) When your allocated minShare of all queue`s added up exceed cluster capacity you can get some queue for 0 fairshare

[ https://issues.apache.org/jira/browse/YARN-6021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla resolved YARN-6021. Resolution: Not A Problem I am closing this as "Not a Problem". Ple

[jira] [Created] (YARN-5990) [Umbrella] Follow-up (phase-2) FairScheduler preemption improvements

Karthik Kambatla created YARN-5990: -- Summary: [Umbrella] Follow-up (phase-2) FairScheduler preemption improvements Key: YARN-5990 URL: https://issues.apache.org/jira/browse/YARN-5990 Project: Hadoop

Re: [VOTE] Merge feature branch YARN-4752. FairScheduler preemption overhaul

org> wrote: > +1 > > On Thu, Nov 17, 2016 at 8:21 AM, Arun Suresh <asur...@apache.org> wrote: > >> +1 >> >> >> On Thu, Nov 17, 2016 at 6:21 AM, Daniel Templeton <dan...@cloudera.com> >> wrote: >> >> > +1 on the merge. Nice work, Kar

[VOTE] Merge feature branch YARN-4752. FairScheduler preemption overhaul

Hi folks, Based on the positive feedback on the DISCUSS thread [1], I would like to start a formal vote to merge feature branch YARN-4752 to trunk. The cumulative patch is posted to YARN-4752, and squashed into one commit here [2]. The changes include: 1. Preemption considers individual

Re: [DISCUSS] Merge FairScheduler preemption overhaul (YARN-4752) to trunk

all the testing bases were covered. >> >> To add to Karthik's points, YARN-5819 was committed early this morning, >> and the JIRA for his TODO bullet is YARN-5885. >> >> Daniel >> >> >> On 11/10/16 6:19 PM, Karthik Kambatla wrote: >> >>> Forg

[jira] [Created] (YARN-5893) Experiment with using PriorityQueue for storing starved applications

Karthik Kambatla created YARN-5893: -- Summary: Experiment with using PriorityQueue for storing starved applications Key: YARN-5893 URL: https://issues.apache.org/jira/browse/YARN-5893 Project: Hadoop

[jira] [Created] (YARN-5885) Cleanup YARN-4752 for merge

Karthik Kambatla created YARN-5885: -- Summary: Cleanup YARN-4752 for merge Key: YARN-5885 URL: https://issues.apache.org/jira/browse/YARN-5885 Project: Hadoop YARN Issue Type: Sub-task

Re: [DISCUSS] Merge FairScheduler preemption overhaul (YARN-4752) to trunk

PM, Karthik Kambatla <ka...@cloudera.com> wrote: > Hi folks > > We have been working on overhauling FairScheduler preemption on branch > YARN-4572. It is close to being ready for merge to trunk: > >1. Preemption considers individual ResourceRequests to satisfy.

[DISCUSS] Merge FairScheduler preemption overhaul (YARN-4752) to trunk

Hi folks We have been working on overhauling FairScheduler preemption on branch YARN-4572. It is close to being ready for merge to trunk: 1. Preemption considers individual ResourceRequests to satisfy. (YARN-5605) 2. Preemption now works within a leaf queue and across sibling leaf

Re: Updated 2.8.0-SNAPSHOT artifact

rs to the list before we release then I'd rather stay where >> > we're at and ship it ASAP. >> > >> > Jason >> > (1) https://issues.apache.org/jira/issues/?jql=project%20in% >> > 20%28hadoop%2C%20yarn%2C%20mapreduce%2C%20hdfs%29% >> > 20and%20re

Re: [DISCUSS] Release cadence and EOL

r. >>> > >>> > The proposal is a minor release on the latest major line every 6 >>> months, >>> > and a maintenance release on a minor release (as there may be >>> concurrently >>> > maintained minor releases) every 2 months. >>&g

[jira] [Created] (YARN-5863) Record that an application is starved for resources when it is

Karthik Kambatla created YARN-5863: -- Summary: Record that an application is starved for resources when it is Key: YARN-5863 URL: https://issues.apache.org/jira/browse/YARN-5863 Project: Hadoop YARN

[jira] [Created] (YARN-5832) Preemption should consider multiple resource-requests of a starved app as needed

Karthik Kambatla created YARN-5832: -- Summary: Preemption should consider multiple resource-requests of a starved app as needed Key: YARN-5832 URL: https://issues.apache.org/jira/browse/YARN-5832

[jira] [Created] (YARN-5831) Propagate allowPreemptionFrom flag all the way down to the app

Karthik Kambatla created YARN-5831: -- Summary: Propagate allowPreemptionFrom flag all the way down to the app Key: YARN-5831 URL: https://issues.apache.org/jira/browse/YARN-5831 Project: Hadoop YARN

[jira] [Created] (YARN-5830) Avoid preempting AM containers

Karthik Kambatla created YARN-5830: -- Summary: Avoid preempting AM containers Key: YARN-5830 URL: https://issues.apache.org/jira/browse/YARN-5830 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Created] (YARN-5829) FS preemption should reserve a node before considering containers on it for preemption

Karthik Kambatla created YARN-5829: -- Summary: FS preemption should reserve a node before considering containers on it for preemption Key: YARN-5829 URL: https://issues.apache.org/jira/browse/YARN-5829

[jira] [Created] (YARN-5824) Verify app starvation under custom preemption thresholds and timeouts

Karthik Kambatla created YARN-5824: -- Summary: Verify app starvation under custom preemption thresholds and timeouts Key: YARN-5824 URL: https://issues.apache.org/jira/browse/YARN-5824 Project

[jira] [Created] (YARN-5821) Drop left-over preemption-related code and clean up method visibilities in the Schedulable hierarchy

Karthik Kambatla created YARN-5821: -- Summary: Drop left-over preemption-related code and clean up method visibilities in the Schedulable hierarchy Key: YARN-5821 URL: https://issues.apache.org/jira/browse/YARN

[jira] [Created] (YARN-5819) Verify preemption works between applications in the same leaf queue

Karthik Kambatla created YARN-5819: -- Summary: Verify preemption works between applications in the same leaf queue Key: YARN-5819 URL: https://issues.apache.org/jira/browse/YARN-5819 Project: Hadoop

[jira] [Created] (YARN-5798) Handle FSPreemptionThread crashing due to a RuntimeException

Karthik Kambatla created YARN-5798: -- Summary: Handle FSPreemptionThread crashing due to a RuntimeException Key: YARN-5798 URL: https://issues.apache.org/jira/browse/YARN-5798 Project: Hadoop YARN

[jira] [Created] (YARN-5783) Unit tests to verify the identification of starved applications

Karthik Kambatla created YARN-5783: -- Summary: Unit tests to verify the identification of starved applications Key: YARN-5783 URL: https://issues.apache.org/jira/browse/YARN-5783 Project: Hadoop YARN

Re: Updated 2.8.0-SNAPSHOT artifact

Is there value in releasing current branch-2.8? Aren't we better off re-cutting the branch off of branch-2? On Tue, Oct 25, 2016 at 12:20 AM, Akira Ajisaka wrote: > It's almost a year since branch-2.8 has cut. > I'm thinking we need to release 2.8.0 ASAP. > >

Re: Chrome extension to collapse JIRA comments

Never included the link :) https://github.com/gezapeti/jira-comment-collapser On Mon, Oct 17, 2016 at 6:46 PM, Karthik Kambatla <ka...@cloudera.com> wrote: > Hi folks > > Sorry for the widespread email, but thought you would find this useful. > > My colleague, Pe

Chrome extension to collapse JIRA comments

Hi folks Sorry for the widespread email, but thought you would find this useful. My colleague, Peter, had put together this chrome extension to collapse comments from certain users (HadoopQA, Githubbot) that makes tracking conversations in JIRAs much easier. Cheers! Karthik

Re: [DISCUSS] Deprecating and removing DockerContainerExecutor

I ll go ahead and commit the patch end of this week if I don't hear any reservations either here or on the JIRA. On Wed, Oct 12, 2016 at 9:29 AM, Karthik Kambatla <ka...@cloudera.com> wrote: > Based on the lack of response, can I assume lack of interest in > DockerContainerExecut

Re: [DISCUSS] Deprecating and removing DockerContainerExecutor

Based on the lack of response, can I assume lack of interest in DockerContainerExecutor and hence agreement with its removal? On Thu, Oct 6, 2016 at 6:25 AM, Karthik Kambatla <ka...@cloudera.com> wrote: > Hi folks > > I am emailing here to bring attention to YARN-538

Re: [VOTE] Release Apache Hadoop 2.6.5 (RC1)

Thanks for putting the RC together, Sangjin. +1 (binding) Built from source, deployed pseudo distributed cluster and ran some example MR jobs. On Fri, Oct 7, 2016 at 6:01 PM, Yongjun Zhang wrote: > Hi Sangjin, > > Thanks a lot for your work here. > > My +1 (binding). > >

[DISCUSS] Deprecating and removing DockerContainerExecutor

Hi folks I am emailing here to bring attention to YARN-5388 that proposes to deprecate DockerContainerExecutor (DCE) in 2.x and its removal in 3.0. DCE was released in 2.6. Since then, the LinuxContainerExecutor has been augmented to support Docker containers alongside non-Docker ones. Given the

[jira] [Created] (YARN-5709) Cleanup Curator-based leader election code

Karthik Kambatla created YARN-5709: -- Summary: Cleanup Curator-based leader election code Key: YARN-5709 URL: https://issues.apache.org/jira/browse/YARN-5709 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-5625) FairScheduler should use FSContext more aggressively to avoid constructors with many parameters

Karthik Kambatla created YARN-5625: -- Summary: FairScheduler should use FSContext more aggressively to avoid constructors with many parameters Key: YARN-5625 URL: https://issues.apache.org/jira/browse/YARN-5625

[jira] [Created] (YARN-5607) Document TestContainerResourceUsage#waitForContainerCompletion

Karthik Kambatla created YARN-5607: -- Summary: Document TestContainerResourceUsage#waitForContainerCompletion Key: YARN-5607 URL: https://issues.apache.org/jira/browse/YARN-5607 Project: Hadoop YARN

[jira] [Created] (YARN-5605) Preempt containers (all on one node) to meet the requirement of starved applications

Karthik Kambatla created YARN-5605: -- Summary: Preempt containers (all on one node) to meet the requirement of starved applications Key: YARN-5605 URL: https://issues.apache.org/jira/browse/YARN-5605

[jira] [Resolved] (YARN-3997) An Application requesting multiple core containers can't preempt running application made of single core containers

[ https://issues.apache.org/jira/browse/YARN-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla resolved YARN-3997. Resolution: Duplicate > An Application requesting multiple core containers can't pree

[jira] [Resolved] (YARN-2457) FairScheduler: Handle preemption to help starved parent queues

[ https://issues.apache.org/jira/browse/YARN-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla resolved YARN-2457. Resolution: Duplicate > FairScheduler: Handle preemption to help starved parent que

Re: [DISCUSS] Release cadence and EOL

> > > Here is just an idea to get started. How about "a minor release line is > EOLed 2 years after it is released or there are 2 newer minor releases, > whichever is sooner. The community reserves the right to extend or shorten > the life of a release line if there is a good reason to do so." > >

Re: Short peaks in container memory usage

Jan, As part of YARN-1011 (oversubscription work), we are looking at better (faster) ways of monitoring and enforcement and considering putting all YARN containers under a cgroup with hard limit so YARN as a whole does not go over a limit, but let the individual containers run over. The details

[DISCUSS] Release cadence and EOL

Forking off this discussion from 2.6.5 release thread. Junping and Chris T have brought up important concerns regarding too many concurrent releases and the lack of EOL for our releases. First up, it would be nice to hear from others on our releases having the notion of EOL and other

Re: [Release thread] 2.6.5 release activities

Since there is sufficient interest in 2.6.5, we should probably do it. All the reasons Allen outlines make sense. That said, Junping brings up a very important point that we should think of for future releases. For a new user or a user that does not directly contribute to the project, more stable

YARN Contributor Meetup - 8/25

Hi folks You are all cordially invited to the YARN contributor meetup at Cloudera on 8/25. There will be a webex for folks who can't make it in person. Please RSVP - http://www.meetup.com/Hadoop-Contributors/events/233285258/ Thanks Karthik

Re: [DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

to alphaX to betaX to GA; this applies to both the Hadoop-2 model and the 3.0.0-alphaX model. On Tue, Aug 9, 2016 at 6:02 AM, Karthik Kambatla <ka...@cloudera.com> wrote: > Most people I talked to found 3.0.0-alpha, 3.1.0-alpha/beta confusing. I > am not aware of any other software s

Re: [DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

we should bump up trunk version to 4.x for landing new incompatible > changes. > > Thanks, > > Junping > ________ > From: Karthik Kambatla <ka...@cloudera.com> > Sent: Monday, August 08, 2016 6:54 PM > Cc: common-...@hadoop.apache.org;

Re: [DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

I like the 3.0.0-alphaX approach primarily for simpler understanding of compatibility guarantees. Calling 3.0.0 alpha and 3.1.0 beta is confusing because, it is not immediately clear that 3.0.0 and 3.1.0 could be incompatible in APIs. I am open to something like 2.98.x for alphas and 2.99.x for

Re: YARN Contributor Meetup

Thanks for voting, folks. Thursday 8/25 3 PM and 5 PM are at a tie. Let me look into the logistics and send out further details early next week. On Thu, Jul 28, 2016 at 7:25 PM, Karthik Kambatla <ka...@cloudera.com> wrote: > Gentle reminder: please mark all the timings that work for yo

Re: YARN Contributor Meetup

Gentle reminder: please mark all the timings that work for you. So far, we have 9 people interested. http://doodle.com/poll/hu678caa7wbeeqvw On Mon, Jul 25, 2016 at 11:28 AM, Karthik Kambatla <ka...@cloudera.com> wrote: > Hi folks > > Per our previous discussions, Cloudera wou

Re: [DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

Inline. > >> BTW, I never see we have a clear definition for alpha release. It is >> previously used as unstable in API definition (2.1-alpha, 2.2-alpha, etc.) >> but sometimes means unstable in production quality (2.7.0). I think we >> should clearly define it with major consensus so user won't

Re: [DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

Inline. > 1) Set the fix version for all a.b.c versions, where c > 0. > 2) For each major release line, set the lowest a.b.0 version. > Sounds reasonable. > > The -alphaX versions we're using leading up to 3.0.0 GA can be treated as > a.b.c versions, with alpha1 being the a.b.0 release. >

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

IIRR, the vote is on source artifacts and binaries are for convenience. If that is right, I am open to either options - do another RC or continue this vote and fix the binary artifacts. On Tue, Jul 26, 2016 at 12:11 PM, Vinod Kumar Vavilapalli < vino...@apache.org> wrote: > Thanks Daniel and

YARN Contributor Meetup

Hi folks Per our previous discussions, Cloudera would like to host the next YARN meetup. Could you please take a moment and pick all slots that work for you here [1]? The timings are in PT. Keep in mind that parking is a little tight at 3 PM. 5 PM should

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

+1 (binding) * Downloaded and build from source * Checked LICENSE and NOTICE * Pseudo-distributed cluster with FairScheduler * Ran MR and HDFS tests * Verified basic UI On Sun, Jul 24, 2016 at 1:07 PM, larry mccay wrote: > +1 binding > > * downloaded and built from source >

Re: [DISCUSS] YARN-5079 : Native YARN framework layer for services and Apache Slider

+1. Merging the core parts of Slider into YARN should help with faster development on service-oriented features. On Tue, Jul 19, 2016 at 12:38 PM, Wangda Tan wrote: > +1, this can avoid lots efforts for end user to run services on YARN. > > On Tue, Jul 19, 2016 at 7:12

[jira] [Created] (YARN-5380) NMTimelinePublisher should use getMemorySize instead of getMemory

Karthik Kambatla created YARN-5380: -- Summary: NMTimelinePublisher should use getMemorySize instead of getMemory Key: YARN-5380 URL: https://issues.apache.org/jira/browse/YARN-5380 Project: Hadoop

[jira] [Resolved] (YARN-5371) FairScheduer ContinuousScheduling thread throws Exception

[ https://issues.apache.org/jira/browse/YARN-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla resolved YARN-5371. Resolution: Duplicate > FairScheduer ContinuousScheduling thread throws Except

Re: [VOTE] merge feature branch YARN-2928

+0 I'm in favor of merging this, especially in time for Hadoop 3 alpha. However, I haven't looked at the code or tests. So, don't feel qualified to vote +1. Mobile > On Jul 2, 2016, at 10:17 AM, Arun Suresh wrote: > > +1(binding) > Really looking forward to this in

[jira] [Created] (YARN-5308) FairScheduler: Move continuous scheduling related threads to TestContinuousScheduling

Karthik Kambatla created YARN-5308: -- Summary: FairScheduler: Move continuous scheduling related threads to TestContinuousScheduling Key: YARN-5308 URL: https://issues.apache.org/jira/browse/YARN-5308

Re: [DISCUSS] merging YARN-2928 (Timeline Service v.2) to trunk

For the catch up, I meant during Hadoop Summit next week. On Tue, Jun 21, 2016 at 10:28 PM, Karthik Kambatla <ka...@cloudera.com> wrote: > The reasons for my asking about alternate implementations: (1) ease of > trying it out for Yarn devs and iteration for bug fixes, improvements an

Re: [DISCUSS] merging YARN-2928 (Timeline Service v.2) to trunk

ased implementation but it is strictly > >> for > >>> test purposes (as we write data into a local file). It does not support > >> all > >>> the features of Timeline Service v.2 as well. > >>>> Regarding LevelDB, Timeline Service v.2 has di

Re: [DISCUSS] merging YARN-2928 (Timeline Service v.2) to trunk

Firstly, thanks Sangjin and others for driving this major feature. Merging to trunk and including in 3.0.0-alpha1 seems reasonable, as it will give early access to downstream users. With regards to merging, it would help to have clear documentation on how to setup and use ATS. Slightly

Re: [DISCUSS] Increased use of feature branches

Thanks for clarifying Andrew. Inline. On Mon, Jun 13, 2016 at 3:59 PM, Andrew Wang <andrew.w...@cloudera.com> wrote: > > On Fri, Jun 10, 2016 at 9:39 PM, Karthik Kambatla <ka...@cloudera.com> > wrote: > >> I would like to understand the trunk-incompat part of t

Re: [DISCUSS] Increased use of feature branches

ous and significant: >>> >> > - A lot of commits (incompatible, risky, uncompleted feature, etc.) >>> have >>> >> > to wait to commit to trunk or put into a separated branch that could >>> >> delay >>> >> > feature development

Re: [DISCUSS] Increased use of feature branches

ing the approach for 3.x. The new proposal forces following these requirements and hence I like it more. > > Thanks, > > Junping > > From: Karthik Kambatla <ka...@cloudera.com> > Sent: Friday, June 10, 2016 7:49 AM > To: Andrew Wang > Cc: common-...@hado

Re: [DISCUSS] Increased use of feature branches

Thanks for restarting this thread Andrew. I really hope we can get this across to a VOTE so it is clear. I see a few advantages shipping from trunk: - The lack of need for one additional backport each time. - Feature rot in trunk Instead of creating branch-3, I recommend creating

[jira] [Created] (YARN-5184) Fix up incompatible changes introduced in YARN-2882

Karthik Kambatla created YARN-5184: -- Summary: Fix up incompatible changes introduced in YARN-2882 Key: YARN-5184 URL: https://issues.apache.org/jira/browse/YARN-5184 Project: Hadoop YARN

[jira] [Created] (YARN-5182) MockNodes.newNodes creates one more node per rack than requested

Karthik Kambatla created YARN-5182: -- Summary: MockNodes.newNodes creates one more node per rack than requested Key: YARN-5182 URL: https://issues.apache.org/jira/browse/YARN-5182 Project: Hadoop

[jira] [Created] (YARN-5181) ClusterNodeTracker: add method to get list of nodes matching a specific resourceName

Karthik Kambatla created YARN-5181: -- Summary: ClusterNodeTracker: add method to get list of nodes matching a specific resourceName Key: YARN-5181 URL: https://issues.apache.org/jira/browse/YARN-5181

Re: [DISCUSS] what exactly are the stability guarantees of the YARN APIs

using Public-Evolving to capture this intent. Should we add a third annotation in addition to Audience and Stability to capture whether a class can be extended? Given the few classes we anticipate being extended, this is likely lesser work. :) > > > On Sat, May 28, 2016 at 10:07 AM, Kart

Re: [DISCUSS] what exactly are the stability guarantees of the YARN APIs

Inline. On Sat, May 28, 2016 at 3:51 AM, Steve Loughran wrote: > > In SLIDER-1130, I discovered that some changes to some YARN classes had > broken my test code —new abstract methods had been defined, which broke the > subclasses I have to mock a YARN cluster. These are

[jira] [Created] (YARN-5106) Provide a builder interface for FairScheduler allocations for use in tests

Karthik Kambatla created YARN-5106: -- Summary: Provide a builder interface for FairScheduler allocations for use in tests Key: YARN-5106 URL: https://issues.apache.org/jira/browse/YARN-5106 Project

Re: Looking to a Hadoop 3 release

I am with Vinod on avoiding merging mostly_complete_branches to trunk since we are not shipping any release off it. If 3.x releases going off of trunk is going to help with this, I am fine with that approach. We should still make sure to keep trunk-incompat small and not include large features.

[jira] [Created] (YARN-5035) FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on

Karthik Kambatla created YARN-5035: -- Summary: FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on Key: YARN-5035 URL: https://issues.apache.org/jira/browse/YARN-5035 Project

[jira] [Created] (YARN-5030) Revisit o.a.h.y.s.rm.scheduler.ResourceUsage

Karthik Kambatla created YARN-5030: -- Summary: Revisit o.a.h.y.s.rm.scheduler.ResourceUsage Key: YARN-5030 URL: https://issues.apache.org/jira/browse/YARN-5030 Project: Hadoop YARN Issue

[jira] [Created] (YARN-5028) RMStateStore should trim down app state for completed applications

Karthik Kambatla created YARN-5028: -- Summary: RMStateStore should trim down app state for completed applications Key: YARN-5028 URL: https://issues.apache.org/jira/browse/YARN-5028 Project: Hadoop

[jira] [Created] (YARN-4927) TestRMHA#testTransitionedToActiveRefreshFail fails when FairScheduler is the default

Karthik Kambatla created YARN-4927: -- Summary: TestRMHA#testTransitionedToActiveRefreshFail fails when FairScheduler is the default Key: YARN-4927 URL: https://issues.apache.org/jira/browse/YARN-4927

[jira] [Created] (YARN-4885) Aggregate container utilization can be negative

Karthik Kambatla created YARN-4885: -- Summary: Aggregate container utilization can be negative Key: YARN-4885 URL: https://issues.apache.org/jira/browse/YARN-4885 Project: Hadoop YARN Issue

[jira] [Created] (YARN-4866) FairScheduler: AMs can consume all vcores leading to a livelock when using FAIR policy

Karthik Kambatla created YARN-4866: -- Summary: FairScheduler: AMs can consume all vcores leading to a livelock when using FAIR policy Key: YARN-4866 URL: https://issues.apache.org/jira/browse/YARN-4866

[jira] [Resolved] (YARN-2048) List all of the containers of an application from the yarn web

[ https://issues.apache.org/jira/browse/YARN-2048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla resolved YARN-2048. Resolution: Duplicate > List all of the containers of an application from the yarn

  1   2   3   4   5   >