[jira] [Commented] (YARN-1459) RM services should depend on ConfigurationProvider during startup too
[
https://issues.apache.org/jira/browse/YARN-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895775#comment-13895775
]
Hadoop QA commented on YARN-1459:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12627838/YARN-1459.4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3047//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3047//console
This message is automatically generated.
> RM services should depend on ConfigurationProvider during startup too
> -
>
> Key: YARN-1459
> URL: https://issues.apache.org/jira/browse/YARN-1459
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: 2.2.0
>Reporter: Karthik Kambatla
>Assignee: Xuan Gong
> Attachments: YARN-1459.1.patch, YARN-1459.2.patch, YARN-1459.3.patch,
> YARN-1459.4.patch, YARN-1459.4.patch
>
>
> YARN-1667, YARN-1668, YARN-1669 already changed RM to depend on a
> configuration provider so as to be able to refresh many configuration files
> across RM fail-over. The dependency on the configuration-provider by the RM
> should happen at its boot up time too.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1525) Web UI should redirect to active RM when HA is enabled.
[
https://issues.apache.org/jira/browse/YARN-1525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895768#comment-13895768
]
Hadoop QA commented on YARN-1525:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12627837/YARN1525.v7.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 2
release audit warnings.
{color:red}-1 core tests{color}. The following test timeouts occurred in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.client.api.impl.TestNMClient
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3046//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/3046//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/3046//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3046//console
This message is automatically generated.
> Web UI should redirect to active RM when HA is enabled.
> ---
>
> Key: YARN-1525
> URL: https://issues.apache.org/jira/browse/YARN-1525
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Xuan Gong
>Assignee: Cindy Li
> Attachments: YARN1525.patch.v1, YARN1525.patch.v2, YARN1525.patch.v3,
> YARN1525.v7.patch
>
>
> When failover happens, web UI should redirect to the current active rm.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1490) RM should optionally not kill all containers when an ApplicationMaster exits
[ https://issues.apache.org/jira/browse/YARN-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895763#comment-13895763 ] Bikas Saha commented on YARN-1490: -- Also, no containers are allocated until the application actually registers and so doing this upon app registration should be the same timeline-wise as doing this during app submission. > RM should optionally not kill all containers when an ApplicationMaster exits > > > Key: YARN-1490 > URL: https://issues.apache.org/jira/browse/YARN-1490 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: YARN-1490.1.patch, YARN-1490.10.patch, > YARN-1490.11.patch, YARN-1490.11.patch, YARN-1490.12.patch, > YARN-1490.2.patch, YARN-1490.3.patch, YARN-1490.4.patch, YARN-1490.5.patch, > YARN-1490.6.patch, YARN-1490.7.patch, YARN-1490.8.patch, YARN-1490.9.patch, > org.apache.oozie.service.TestRecoveryService_thread-dump.txt > > > This is needed to enable work-preserving AM restart. Some apps can chose to > reconnect with old running containers, some may not want to. This should be > an option. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1490) RM should optionally not kill all containers when an ApplicationMaster exits
[ https://issues.apache.org/jira/browse/YARN-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895761#comment-13895761 ] Bikas Saha commented on YARN-1490: -- The option to kill container on AM exit should come from AM.registerApplicationMaster and not from client.submitApplication(AppSubmissionContext). The AM is the one who knows whether it can recover running containers after restart. So its the AM who should tell the RM whether to kill containers upon AM restart. Consider the case when a generic client is submitted a bunch of a class of apps or the same app client is submitting apps with different versions of the AM code. Some apps/versions may support this feature and some may not. Its the app that should tell the RM what to do based on what the app is capable of. > RM should optionally not kill all containers when an ApplicationMaster exits > > > Key: YARN-1490 > URL: https://issues.apache.org/jira/browse/YARN-1490 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: YARN-1490.1.patch, YARN-1490.10.patch, > YARN-1490.11.patch, YARN-1490.11.patch, YARN-1490.12.patch, > YARN-1490.2.patch, YARN-1490.3.patch, YARN-1490.4.patch, YARN-1490.5.patch, > YARN-1490.6.patch, YARN-1490.7.patch, YARN-1490.8.patch, YARN-1490.9.patch, > org.apache.oozie.service.TestRecoveryService_thread-dump.txt > > > This is needed to enable work-preserving AM restart. Some apps can chose to > reconnect with old running containers, some may not want to. This should be > an option. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1459) RM services should depend on ConfigurationProvider during startup too
[ https://issues.apache.org/jira/browse/YARN-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-1459: Attachment: YARN-1459.4.patch add into findbugs-exclude.xml to ignore this findbug warning > RM services should depend on ConfigurationProvider during startup too > - > > Key: YARN-1459 > URL: https://issues.apache.org/jira/browse/YARN-1459 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 2.2.0 >Reporter: Karthik Kambatla >Assignee: Xuan Gong > Attachments: YARN-1459.1.patch, YARN-1459.2.patch, YARN-1459.3.patch, > YARN-1459.4.patch, YARN-1459.4.patch > > > YARN-1667, YARN-1668, YARN-1669 already changed RM to depend on a > configuration provider so as to be able to refresh many configuration files > across RM fail-over. The dependency on the configuration-provider by the RM > should happen at its boot up time too. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1698) Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore as default
[
https://issues.apache.org/jira/browse/YARN-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895749#comment-13895749
]
Hadoop QA commented on YARN-1698:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12627836/YARN-1698.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3045//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3045//console
This message is automatically generated.
> Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore
> as default
> --
>
> Key: YARN-1698
> URL: https://issues.apache.org/jira/browse/YARN-1698
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 2.4.0
>Reporter: Zhijie Shen
>Assignee: Zhijie Shen
> Attachments: YARN-1698.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1525) Web UI should redirect to active RM when HA is enabled.
[ https://issues.apache.org/jira/browse/YARN-1525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cindy Li updated YARN-1525: --- Attachment: YARN1525.v7.patch Added end-to-end test case > Web UI should redirect to active RM when HA is enabled. > --- > > Key: YARN-1525 > URL: https://issues.apache.org/jira/browse/YARN-1525 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Cindy Li > Attachments: YARN1525.patch.v1, YARN1525.patch.v2, YARN1525.patch.v3, > YARN1525.v7.patch > > > When failover happens, web UI should redirect to the current active rm. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1698) Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore as default
[ https://issues.apache.org/jira/browse/YARN-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-1698: -- Attachment: YARN-1698.1.patch Update ApplicationHistoryServer to use LeveldbApplicationTimelineStore as default. Straightforward change without test cases. > Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore > as default > -- > > Key: YARN-1698 > URL: https://issues.apache.org/jira/browse/YARN-1698 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.4.0 >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Attachments: YARN-1698.1.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Resolved] (YARN-1699) Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore as default
[ https://issues.apache.org/jira/browse/YARN-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen resolved YARN-1699. --- Resolution: Duplicate Create the same ticket twice by mistake. > Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore > as default > -- > > Key: YARN-1699 > URL: https://issues.apache.org/jira/browse/YARN-1699 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.4.0 >Reporter: Zhijie Shen >Assignee: Zhijie Shen > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (YARN-1699) Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore as default
Zhijie Shen created YARN-1699: - Summary: Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore as default Key: YARN-1699 URL: https://issues.apache.org/jira/browse/YARN-1699 Project: Hadoop YARN Issue Type: Sub-task Affects Versions: 2.4.0 Reporter: Zhijie Shen Assignee: Zhijie Shen -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (YARN-1698) Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore as default
Zhijie Shen created YARN-1698: - Summary: Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore as default Key: YARN-1698 URL: https://issues.apache.org/jira/browse/YARN-1698 Project: Hadoop YARN Issue Type: Sub-task Affects Versions: 2.4.0 Reporter: Zhijie Shen Assignee: Zhijie Shen -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1692) ConcurrentModificationException in fair scheduler AppSchedulable
[
https://issues.apache.org/jira/browse/YARN-1692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895638#comment-13895638
]
Sangjin Lee commented on YARN-1692:
---
This concerns thread safety, and it is not practical to have unit tests that
verify thread safety. I did a complete analysis of access to the map in
question.
> ConcurrentModificationException in fair scheduler AppSchedulable
>
>
> Key: YARN-1692
> URL: https://issues.apache.org/jira/browse/YARN-1692
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 2.0.5-alpha
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Attachments: yarn-1692.patch
>
>
> We saw a ConcurrentModificationException thrown in the fair scheduler:
> {noformat}
> 2014-02-07 01:40:01,978 ERROR
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler:
> Exception in fair scheduler UpdateThread
> java.util.ConcurrentModificationException
> at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926)
> at java.util.HashMap$ValueIterator.next(HashMap.java:954)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AppSchedulable.updateDemand(AppSchedulable.java:85)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.updateDemand(FSLeafQueue.java:125)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue.updateDemand(FSParentQueue.java:82)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.update(FairScheduler.java:217)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler$UpdateThread.run(FairScheduler.java:195)
> at java.lang.Thread.run(Thread.java:724)
> {noformat}
> The map that gets returned by FSSchedulerApp.getResourceRequests() are
> iterated on without proper synchronization.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1497) Expose moving apps between queues on the command line
[ https://issues.apache.org/jira/browse/YARN-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895598#comment-13895598 ] Hudson commented on YARN-1497: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1692 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1692/]) YARN-1497. Command line additions for moving apps between queues (Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565754) * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/YarnClient.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/YarnClientImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/YarnCLI.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java > Expose moving apps between queues on the command line > - > > Key: YARN-1497 > URL: https://issues.apache.org/jira/browse/YARN-1497 > Project: Hadoop YARN > Issue Type: Sub-task > Components: client >Affects Versions: 2.2.0 >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Fix For: 3.0.0 > > Attachments: YARN-1497-1.patch, YARN-1497.patch, YARN-1497.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1672) YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds
[ https://issues.apache.org/jira/browse/YARN-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895592#comment-13895592 ] Hudson commented on YARN-1672: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1692 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1692/]) YARN-1672. YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds (Naren Koneru via kasha) (kasha: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565866) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/NonAggregatingLogHandler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/TestNonAggregatingLogHandler.java > YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds > -- > > Key: YARN-1672 > URL: https://issues.apache.org/jira/browse/YARN-1672 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.2.0 >Reporter: Karthik Kambatla >Assignee: Naren Koneru >Priority: Trivial > Fix For: 2.4.0 > > Attachments: YARN-1672-1.patch > > > YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1041) Protocol changes for RM to bind and notify a restarted AM of existing containers
[ https://issues.apache.org/jira/browse/YARN-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895596#comment-13895596 ] Hudson commented on YARN-1041: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1692 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1692/]) YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt > Protocol changes for RM to bind and notify a restarted AM of existing > containers > > > Key: YARN-1041 > URL: https://issues.apache.org/jira/browse/YARN-1041 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: YARN-1041.1.patch, YARN-1041.2.patch, YARN-1041.3.patch, > YARN-1041.4.patch, YARN-1041.5.patch, YARN-1041.6.patch, YARN-1041.7.patch > > > For long lived containers we don't want the AM to be a SPOF. > When the RM restarts a (failed) AM, it should be given the list of containers > it had already been allocated. the AM should then be able to contact the NMs > to get details on them. NMs would also need to do any binding of the > containers needed to handle a moved/restarted AM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1493) Schedulers don't recognize apps separately from app-attempts
[ https://issues.apache.org/jira/browse/YARN-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895588#comment-13895588 ] Hudson commented on YARN-1493: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1692 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1692/]) YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt > Schedulers don't recognize apps separately from app-attempts > > > Key: YARN-1493 > URL: https://issues.apache.org/jira/browse/YARN-1493 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: YARN-1493.1.patch, YARN-1493.2.patch, YARN-1493.3.patch, > YARN-1493.4.patch, YARN-1493.5.patch, YARN-1493.6.patch, YARN-1493.7.patch, > YARN-1493.8.patch, YARN-1493.9.patch > > > Today, scheduler is tied to attempt only. > We need to separate app-level handling logic in scheduler. We can add new > app-level events to the scheduler and separate the app-level logic out. This > is good for work-preserving AM restart, RM restart, and also needed for > differentiating app-level metrics and attempt-level metrics. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1689) RMAppAttempt is not killed when RMApp is at ACCEPTED
[
https://issues.apache.org/jira/browse/YARN-1689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895590#comment-13895590
]
Hudson commented on YARN-1689:
--
SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1692 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1692/])
YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are
reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
> RMAppAttempt is not killed when RMApp is at ACCEPTED
>
>
> Key: YARN-1689
> URL: https://issues.apache.org/jira/browse/YARN-1689
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.4.0
>Reporter: Deepesh Khandelwal
>Assignee: Vinod Kumar Vavilapalli
>Priority: Critical
> Fix For: 2.4.0
>
> Attachments: RM_UI.png, YARN-1689-20140205.txt
>
>
> When running some Hive on Tez jobs, the RM after a while gets into an
> unusable state where no jobs run. In the RM log I see the following exception:
> {code}
> 2014-02-04 20:28:08,553 WARN ipc.Server (Server.java:run(1978)) - IPC Server
> handler 0 on 8030, call
> org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.registerApplicationMaster
> from 172.18.145.156:40474 Call#0 Retry#0: error:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.getTransferredContainers(AbstractYarnScheduler.java:48)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.registerApplicationMaster(ApplicationMasterService.java:278)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.registerApplicationMaster(ApplicationMasterProtocolPBServiceImpl.java:90)
> at
> org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:95)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1958)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1956)
> ..
> 2014-02-04 20:28:08,544 ERROR rmapp.RMAppImpl (RMAppImpl.java:handle(626)) -
> Can't handle this event at current state
> org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event:
> ATTEMPT_REGISTERED at KILLED
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at
> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:624)
> at
> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:81)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher.handle(ResourceManager.java:656)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher.handle(ResourceManager.java:640)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
> at java.lang.Thread.run(Thread.java:662)
> 2014-02-04 20:28:08,549 INFO resourcemanager.RMAuditLogger
> (RMAuditLogger.java:logSuccess(140)) - USER=hrt_qa IP=172.18.145.156
> OPERATION=Kill Application Request TARGET=ClientRMService
> RESULT=SUCCESS APPID=application_1391543307203_0001
> 2014-02-04 20:28:08,553 WARN ipc.Server (Server.java:run(1978)) - IPC Server
> handler 0 on 8030, call
> org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.registerApplicationMaster
> from 172.18.145.156:40474 Call#0 Retry#0: error:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.getTransferredContainers(AbstractYarnScheduler.java:48)
> at
> org.apache.hadoop.yarn
[jira] [Commented] (YARN-1566) Change distributed-shell to retain containers from previous AppAttempt
[ https://issues.apache.org/jira/browse/YARN-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895595#comment-13895595 ] Hudson commented on YARN-1566: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1692 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1692/]) YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt > Change distributed-shell to retain containers from previous AppAttempt > -- > > Key: YARN-1566 > URL: https://issues.apache.org/jira/browse/YARN-1566 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: YARN-1566.1.patch, YARN-1566.2.patch > > > Change distributed-shell to reuse previous AM's running containers when AM is > restarting. It can also be made configurable whether to enable this feature > or not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1166) YARN 'appsFailed' metric should be of type 'counter'
[ https://issues.apache.org/jira/browse/YARN-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895597#comment-13895597 ] Hudson commented on YARN-1166: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1692 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1692/]) YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt > YARN 'appsFailed' metric should be of type 'counter' > > > Key: YARN-1166 > URL: https://issues.apache.org/jira/browse/YARN-1166 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.1.0-beta >Reporter: Srimanth Gunturi >Assignee: Zhijie Shen >Priority: Blocker > Fix For: 2.4.0 > > Attachments: YARN-1166.2.patch, YARN-1166.3.patch, YARN-1166.4.patch, > YARN-1166.5.patch, YARN-1166.6.patch, YARN-1166.7.patch, YARN-1166.8.patch, > YARN-1166.9.patch, YARN-1166.9.patch, YARN-1166.patch > > > Currently in YARN's queue metrics, the cumulative metric 'appsFailed' is of > type 'guage' - which means the exact value will be reported. > All other cumulative queue metrics (AppsSubmitted, AppsCompleted, AppsKilled) > are all of type 'counter' - meaning Ganglia will use slope to provide deltas > between time-points. > To be consistent, AppsFailed metric should also be of type 'counter'. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1490) RM should optionally not kill all containers when an ApplicationMaster exits
[ https://issues.apache.org/jira/browse/YARN-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895594#comment-13895594 ] Hudson commented on YARN-1490: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1692 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1692/]) YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt > RM should optionally not kill all containers when an ApplicationMaster exits > > > Key: YARN-1490 > URL: https://issues.apache.org/jira/browse/YARN-1490 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: YARN-1490.1.patch, YARN-1490.10.patch, > YARN-1490.11.patch, YARN-1490.11.patch, YARN-1490.12.patch, > YARN-1490.2.patch, YARN-1490.3.patch, YARN-1490.4.patch, YARN-1490.5.patch, > YARN-1490.6.patch, YARN-1490.7.patch, YARN-1490.8.patch, YARN-1490.9.patch, > org.apache.oozie.service.TestRecoveryService_thread-dump.txt > > > This is needed to enable work-preserving AM restart. Some apps can chose to > reconnect with old running containers, some may not want to. This should be > an option. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1661) AppMaster logs says failing even if an application does succeed.
[
https://issues.apache.org/jira/browse/YARN-1661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895587#comment-13895587
]
Hudson commented on YARN-1661:
--
SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1692 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1692/])
YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are
reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
> AppMaster logs says failing even if an application does succeed.
>
>
> Key: YARN-1661
> URL: https://issues.apache.org/jira/browse/YARN-1661
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications/distributed-shell
>Affects Versions: 2.4.0
>Reporter: Tassapol Athiapinya
>Assignee: Vinod Kumar Vavilapalli
> Fix For: 2.4.0
>
> Attachments: YARN-1661.txt
>
>
> Run:
> /usr/bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client
> -jar -shell_command ls
> Open AM logs. Last line would indicate AM failure even though container logs
> print good ls result.
> {code}
> 2014-01-24 21:45:29,592 INFO [main] distributedshell.ApplicationMaster
> (ApplicationMaster.java:finish(599)) - Application completed. Signalling
> finish to RM
> 2014-01-24 21:45:29,612 INFO [main] impl.AMRMClientImpl
> (AMRMClientImpl.java:unregisterApplicationMaster(315)) - Waiting for
> application to be successfully unregistered.
> 2014-01-24 21:45:29,816 INFO [main] distributedshell.ApplicationMaster
> (ApplicationMaster.java:main(267)) - Application Master failed. exiting
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1635) Implement a Leveldb based ApplicationTimelineStore
[ https://issues.apache.org/jira/browse/YARN-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895589#comment-13895589 ] Hudson commented on YARN-1635: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1692 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1692/]) YARN-1635. Implemented a Leveldb based ApplicationTimelineStore. Contributed by Billie Rinaldi. (zjshen: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565868) * /hadoop/common/trunk/hadoop-project/pom.xml * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSPutErrors.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/apptimeline/TestApplicationTimelineRecords.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/pom.xml * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineReader.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineWriter.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/EntityId.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/EntityIdentifier.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/GenericObjectMapper.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/LeveldbApplicationTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/MemoryApplicationTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/ATSWebServices.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineStoreTestUtils.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/TestGenericObjectMapper.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/TestLeveldbApplicationTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/TestMemoryApplicationTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestATSWebServices.java > Implement a Leveldb based ApplicationTimelineStore > -- > > Key: YARN-1635 > URL: https://issues.apache.org/jira/browse/YARN-1635 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Billie Rinaldi > Fix For: 2.4.0 > > Attachments: YARN-1635.1.patch, YARN-1635.10.patch, > YARN-1635.11.patch, YARN-1635.12.patch, YARN-1635.2.patch, YARN-1635.3.patch, > YARN-1635.4.patch, YARN-1635.5.patch, YARN-1635.6.patch, YARN-1635.7.patch, > YARN-1635.8.patch, YARN-1635.9.patc
[jira] [Commented] (YARN-1672) YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds
[ https://issues.apache.org/jira/browse/YARN-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895557#comment-13895557 ] Hudson commented on YARN-1672: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1667 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1667/]) YARN-1672. YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds (Naren Koneru via kasha) (kasha: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565866) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/NonAggregatingLogHandler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/TestNonAggregatingLogHandler.java > YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds > -- > > Key: YARN-1672 > URL: https://issues.apache.org/jira/browse/YARN-1672 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.2.0 >Reporter: Karthik Kambatla >Assignee: Naren Koneru >Priority: Trivial > Fix For: 2.4.0 > > Attachments: YARN-1672-1.patch > > > YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1497) Expose moving apps between queues on the command line
[ https://issues.apache.org/jira/browse/YARN-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895563#comment-13895563 ] Hudson commented on YARN-1497: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1667 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1667/]) YARN-1497. Command line additions for moving apps between queues (Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565754) * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/YarnClient.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/YarnClientImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/YarnCLI.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java > Expose moving apps between queues on the command line > - > > Key: YARN-1497 > URL: https://issues.apache.org/jira/browse/YARN-1497 > Project: Hadoop YARN > Issue Type: Sub-task > Components: client >Affects Versions: 2.2.0 >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Fix For: 3.0.0 > > Attachments: YARN-1497-1.patch, YARN-1497.patch, YARN-1497.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1635) Implement a Leveldb based ApplicationTimelineStore
[ https://issues.apache.org/jira/browse/YARN-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895554#comment-13895554 ] Hudson commented on YARN-1635: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1667 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1667/]) YARN-1635. Implemented a Leveldb based ApplicationTimelineStore. Contributed by Billie Rinaldi. (zjshen: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565868) * /hadoop/common/trunk/hadoop-project/pom.xml * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSPutErrors.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/apptimeline/TestApplicationTimelineRecords.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/pom.xml * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineReader.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineWriter.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/EntityId.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/EntityIdentifier.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/GenericObjectMapper.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/LeveldbApplicationTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/MemoryApplicationTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/ATSWebServices.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineStoreTestUtils.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/TestGenericObjectMapper.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/TestLeveldbApplicationTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/TestMemoryApplicationTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestATSWebServices.java > Implement a Leveldb based ApplicationTimelineStore > -- > > Key: YARN-1635 > URL: https://issues.apache.org/jira/browse/YARN-1635 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Billie Rinaldi > Fix For: 2.4.0 > > Attachments: YARN-1635.1.patch, YARN-1635.10.patch, > YARN-1635.11.patch, YARN-1635.12.patch, YARN-1635.2.patch, YARN-1635.3.patch, > YARN-1635.4.patch, YARN-1635.5.patch, YARN-1635.6.patch, YARN-1635.7.patch, > YARN-1635.8.patch, YARN-1635.9.patch > > > As
[jira] [Commented] (YARN-1490) RM should optionally not kill all containers when an ApplicationMaster exits
[ https://issues.apache.org/jira/browse/YARN-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895559#comment-13895559 ] Hudson commented on YARN-1490: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1667 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1667/]) YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt > RM should optionally not kill all containers when an ApplicationMaster exits > > > Key: YARN-1490 > URL: https://issues.apache.org/jira/browse/YARN-1490 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: YARN-1490.1.patch, YARN-1490.10.patch, > YARN-1490.11.patch, YARN-1490.11.patch, YARN-1490.12.patch, > YARN-1490.2.patch, YARN-1490.3.patch, YARN-1490.4.patch, YARN-1490.5.patch, > YARN-1490.6.patch, YARN-1490.7.patch, YARN-1490.8.patch, YARN-1490.9.patch, > org.apache.oozie.service.TestRecoveryService_thread-dump.txt > > > This is needed to enable work-preserving AM restart. Some apps can chose to > reconnect with old running containers, some may not want to. This should be > an option. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1566) Change distributed-shell to retain containers from previous AppAttempt
[ https://issues.apache.org/jira/browse/YARN-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895560#comment-13895560 ] Hudson commented on YARN-1566: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1667 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1667/]) YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt > Change distributed-shell to retain containers from previous AppAttempt > -- > > Key: YARN-1566 > URL: https://issues.apache.org/jira/browse/YARN-1566 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: YARN-1566.1.patch, YARN-1566.2.patch > > > Change distributed-shell to reuse previous AM's running containers when AM is > restarting. It can also be made configurable whether to enable this feature > or not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1689) RMAppAttempt is not killed when RMApp is at ACCEPTED
[
https://issues.apache.org/jira/browse/YARN-1689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1389#comment-1389
]
Hudson commented on YARN-1689:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #1667 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1667/])
YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are
reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
> RMAppAttempt is not killed when RMApp is at ACCEPTED
>
>
> Key: YARN-1689
> URL: https://issues.apache.org/jira/browse/YARN-1689
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.4.0
>Reporter: Deepesh Khandelwal
>Assignee: Vinod Kumar Vavilapalli
>Priority: Critical
> Fix For: 2.4.0
>
> Attachments: RM_UI.png, YARN-1689-20140205.txt
>
>
> When running some Hive on Tez jobs, the RM after a while gets into an
> unusable state where no jobs run. In the RM log I see the following exception:
> {code}
> 2014-02-04 20:28:08,553 WARN ipc.Server (Server.java:run(1978)) - IPC Server
> handler 0 on 8030, call
> org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.registerApplicationMaster
> from 172.18.145.156:40474 Call#0 Retry#0: error:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.getTransferredContainers(AbstractYarnScheduler.java:48)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.registerApplicationMaster(ApplicationMasterService.java:278)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.registerApplicationMaster(ApplicationMasterProtocolPBServiceImpl.java:90)
> at
> org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:95)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1958)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1956)
> ..
> 2014-02-04 20:28:08,544 ERROR rmapp.RMAppImpl (RMAppImpl.java:handle(626)) -
> Can't handle this event at current state
> org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event:
> ATTEMPT_REGISTERED at KILLED
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at
> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:624)
> at
> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:81)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher.handle(ResourceManager.java:656)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher.handle(ResourceManager.java:640)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
> at java.lang.Thread.run(Thread.java:662)
> 2014-02-04 20:28:08,549 INFO resourcemanager.RMAuditLogger
> (RMAuditLogger.java:logSuccess(140)) - USER=hrt_qa IP=172.18.145.156
> OPERATION=Kill Application Request TARGET=ClientRMService
> RESULT=SUCCESS APPID=application_1391543307203_0001
> 2014-02-04 20:28:08,553 WARN ipc.Server (Server.java:run(1978)) - IPC Server
> handler 0 on 8030, call
> org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.registerApplicationMaster
> from 172.18.145.156:40474 Call#0 Retry#0: error:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.getTransferredContainers(AbstractYarnScheduler.java:48)
> at
> org.apache.hadoop.yarn.server.re
[jira] [Commented] (YARN-1166) YARN 'appsFailed' metric should be of type 'counter'
[ https://issues.apache.org/jira/browse/YARN-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895562#comment-13895562 ] Hudson commented on YARN-1166: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1667 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1667/]) YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt > YARN 'appsFailed' metric should be of type 'counter' > > > Key: YARN-1166 > URL: https://issues.apache.org/jira/browse/YARN-1166 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.1.0-beta >Reporter: Srimanth Gunturi >Assignee: Zhijie Shen >Priority: Blocker > Fix For: 2.4.0 > > Attachments: YARN-1166.2.patch, YARN-1166.3.patch, YARN-1166.4.patch, > YARN-1166.5.patch, YARN-1166.6.patch, YARN-1166.7.patch, YARN-1166.8.patch, > YARN-1166.9.patch, YARN-1166.9.patch, YARN-1166.patch > > > Currently in YARN's queue metrics, the cumulative metric 'appsFailed' is of > type 'guage' - which means the exact value will be reported. > All other cumulative queue metrics (AppsSubmitted, AppsCompleted, AppsKilled) > are all of type 'counter' - meaning Ganglia will use slope to provide deltas > between time-points. > To be consistent, AppsFailed metric should also be of type 'counter'. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1041) Protocol changes for RM to bind and notify a restarted AM of existing containers
[ https://issues.apache.org/jira/browse/YARN-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895561#comment-13895561 ] Hudson commented on YARN-1041: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1667 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1667/]) YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt > Protocol changes for RM to bind and notify a restarted AM of existing > containers > > > Key: YARN-1041 > URL: https://issues.apache.org/jira/browse/YARN-1041 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: YARN-1041.1.patch, YARN-1041.2.patch, YARN-1041.3.patch, > YARN-1041.4.patch, YARN-1041.5.patch, YARN-1041.6.patch, YARN-1041.7.patch > > > For long lived containers we don't want the AM to be a SPOF. > When the RM restarts a (failed) AM, it should be given the list of containers > it had already been allocated. the AM should then be able to contact the NMs > to get details on them. NMs would also need to do any binding of the > containers needed to handle a moved/restarted AM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1661) AppMaster logs says failing even if an application does succeed.
[
https://issues.apache.org/jira/browse/YARN-1661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895552#comment-13895552
]
Hudson commented on YARN-1661:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #1667 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1667/])
YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are
reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
> AppMaster logs says failing even if an application does succeed.
>
>
> Key: YARN-1661
> URL: https://issues.apache.org/jira/browse/YARN-1661
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications/distributed-shell
>Affects Versions: 2.4.0
>Reporter: Tassapol Athiapinya
>Assignee: Vinod Kumar Vavilapalli
> Fix For: 2.4.0
>
> Attachments: YARN-1661.txt
>
>
> Run:
> /usr/bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client
> -jar -shell_command ls
> Open AM logs. Last line would indicate AM failure even though container logs
> print good ls result.
> {code}
> 2014-01-24 21:45:29,592 INFO [main] distributedshell.ApplicationMaster
> (ApplicationMaster.java:finish(599)) - Application completed. Signalling
> finish to RM
> 2014-01-24 21:45:29,612 INFO [main] impl.AMRMClientImpl
> (AMRMClientImpl.java:unregisterApplicationMaster(315)) - Waiting for
> application to be successfully unregistered.
> 2014-01-24 21:45:29,816 INFO [main] distributedshell.ApplicationMaster
> (ApplicationMaster.java:main(267)) - Application Master failed. exiting
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1493) Schedulers don't recognize apps separately from app-attempts
[ https://issues.apache.org/jira/browse/YARN-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895553#comment-13895553 ] Hudson commented on YARN-1493: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1667 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1667/]) YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt > Schedulers don't recognize apps separately from app-attempts > > > Key: YARN-1493 > URL: https://issues.apache.org/jira/browse/YARN-1493 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: YARN-1493.1.patch, YARN-1493.2.patch, YARN-1493.3.patch, > YARN-1493.4.patch, YARN-1493.5.patch, YARN-1493.6.patch, YARN-1493.7.patch, > YARN-1493.8.patch, YARN-1493.9.patch > > > Today, scheduler is tied to attempt only. > We need to separate app-level handling logic in scheduler. We can add new > app-level events to the scheduler and separate the app-level logic out. This > is good for work-preserving AM restart, RM restart, and also needed for > differentiating app-level metrics and attempt-level metrics. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1661) AppMaster logs says failing even if an application does succeed.
[
https://issues.apache.org/jira/browse/YARN-1661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895516#comment-13895516
]
Hudson commented on YARN-1661:
--
SUCCESS: Integrated in Hadoop-Yarn-trunk #475 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/475/])
YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are
reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
> AppMaster logs says failing even if an application does succeed.
>
>
> Key: YARN-1661
> URL: https://issues.apache.org/jira/browse/YARN-1661
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications/distributed-shell
>Affects Versions: 2.4.0
>Reporter: Tassapol Athiapinya
>Assignee: Vinod Kumar Vavilapalli
> Fix For: 2.4.0
>
> Attachments: YARN-1661.txt
>
>
> Run:
> /usr/bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client
> -jar -shell_command ls
> Open AM logs. Last line would indicate AM failure even though container logs
> print good ls result.
> {code}
> 2014-01-24 21:45:29,592 INFO [main] distributedshell.ApplicationMaster
> (ApplicationMaster.java:finish(599)) - Application completed. Signalling
> finish to RM
> 2014-01-24 21:45:29,612 INFO [main] impl.AMRMClientImpl
> (AMRMClientImpl.java:unregisterApplicationMaster(315)) - Waiting for
> application to be successfully unregistered.
> 2014-01-24 21:45:29,816 INFO [main] distributedshell.ApplicationMaster
> (ApplicationMaster.java:main(267)) - Application Master failed. exiting
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1166) YARN 'appsFailed' metric should be of type 'counter'
[ https://issues.apache.org/jira/browse/YARN-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895526#comment-13895526 ] Hudson commented on YARN-1166: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #475 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/475/]) YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt > YARN 'appsFailed' metric should be of type 'counter' > > > Key: YARN-1166 > URL: https://issues.apache.org/jira/browse/YARN-1166 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.1.0-beta >Reporter: Srimanth Gunturi >Assignee: Zhijie Shen >Priority: Blocker > Fix For: 2.4.0 > > Attachments: YARN-1166.2.patch, YARN-1166.3.patch, YARN-1166.4.patch, > YARN-1166.5.patch, YARN-1166.6.patch, YARN-1166.7.patch, YARN-1166.8.patch, > YARN-1166.9.patch, YARN-1166.9.patch, YARN-1166.patch > > > Currently in YARN's queue metrics, the cumulative metric 'appsFailed' is of > type 'guage' - which means the exact value will be reported. > All other cumulative queue metrics (AppsSubmitted, AppsCompleted, AppsKilled) > are all of type 'counter' - meaning Ganglia will use slope to provide deltas > between time-points. > To be consistent, AppsFailed metric should also be of type 'counter'. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1635) Implement a Leveldb based ApplicationTimelineStore
[ https://issues.apache.org/jira/browse/YARN-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895518#comment-13895518 ] Hudson commented on YARN-1635: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #475 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/475/]) YARN-1635. Implemented a Leveldb based ApplicationTimelineStore. Contributed by Billie Rinaldi. (zjshen: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565868) * /hadoop/common/trunk/hadoop-project/pom.xml * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSPutErrors.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/apptimeline/TestApplicationTimelineRecords.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/pom.xml * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineReader.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineWriter.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/EntityId.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/EntityIdentifier.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/GenericObjectMapper.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/LeveldbApplicationTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/MemoryApplicationTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/ATSWebServices.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineStoreTestUtils.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/TestGenericObjectMapper.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/TestLeveldbApplicationTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/TestMemoryApplicationTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestATSWebServices.java > Implement a Leveldb based ApplicationTimelineStore > -- > > Key: YARN-1635 > URL: https://issues.apache.org/jira/browse/YARN-1635 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Billie Rinaldi > Fix For: 2.4.0 > > Attachments: YARN-1635.1.patch, YARN-1635.10.patch, > YARN-1635.11.patch, YARN-1635.12.patch, YARN-1635.2.patch, YARN-1635.3.patch, > YARN-1635.4.patch, YARN-1635.5.patch, YARN-1635.6.patch, YARN-1635.7.patch, > YARN-1635.8.patch, YARN-1635.9.patch > > > As p
[jira] [Commented] (YARN-1041) Protocol changes for RM to bind and notify a restarted AM of existing containers
[ https://issues.apache.org/jira/browse/YARN-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895525#comment-13895525 ] Hudson commented on YARN-1041: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #475 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/475/]) YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt > Protocol changes for RM to bind and notify a restarted AM of existing > containers > > > Key: YARN-1041 > URL: https://issues.apache.org/jira/browse/YARN-1041 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: YARN-1041.1.patch, YARN-1041.2.patch, YARN-1041.3.patch, > YARN-1041.4.patch, YARN-1041.5.patch, YARN-1041.6.patch, YARN-1041.7.patch > > > For long lived containers we don't want the AM to be a SPOF. > When the RM restarts a (failed) AM, it should be given the list of containers > it had already been allocated. the AM should then be able to contact the NMs > to get details on them. NMs would also need to do any binding of the > containers needed to handle a moved/restarted AM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1689) RMAppAttempt is not killed when RMApp is at ACCEPTED
[
https://issues.apache.org/jira/browse/YARN-1689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895519#comment-13895519
]
Hudson commented on YARN-1689:
--
SUCCESS: Integrated in Hadoop-Yarn-trunk #475 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/475/])
YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are
reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
> RMAppAttempt is not killed when RMApp is at ACCEPTED
>
>
> Key: YARN-1689
> URL: https://issues.apache.org/jira/browse/YARN-1689
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.4.0
>Reporter: Deepesh Khandelwal
>Assignee: Vinod Kumar Vavilapalli
>Priority: Critical
> Fix For: 2.4.0
>
> Attachments: RM_UI.png, YARN-1689-20140205.txt
>
>
> When running some Hive on Tez jobs, the RM after a while gets into an
> unusable state where no jobs run. In the RM log I see the following exception:
> {code}
> 2014-02-04 20:28:08,553 WARN ipc.Server (Server.java:run(1978)) - IPC Server
> handler 0 on 8030, call
> org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.registerApplicationMaster
> from 172.18.145.156:40474 Call#0 Retry#0: error:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.getTransferredContainers(AbstractYarnScheduler.java:48)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.registerApplicationMaster(ApplicationMasterService.java:278)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.registerApplicationMaster(ApplicationMasterProtocolPBServiceImpl.java:90)
> at
> org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:95)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1962)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1958)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1956)
> ..
> 2014-02-04 20:28:08,544 ERROR rmapp.RMAppImpl (RMAppImpl.java:handle(626)) -
> Can't handle this event at current state
> org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event:
> ATTEMPT_REGISTERED at KILLED
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at
> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:624)
> at
> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:81)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher.handle(ResourceManager.java:656)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher.handle(ResourceManager.java:640)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
> at java.lang.Thread.run(Thread.java:662)
> 2014-02-04 20:28:08,549 INFO resourcemanager.RMAuditLogger
> (RMAuditLogger.java:logSuccess(140)) - USER=hrt_qa IP=172.18.145.156
> OPERATION=Kill Application Request TARGET=ClientRMService
> RESULT=SUCCESS APPID=application_1391543307203_0001
> 2014-02-04 20:28:08,553 WARN ipc.Server (Server.java:run(1978)) - IPC Server
> handler 0 on 8030, call
> org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.registerApplicationMaster
> from 172.18.145.156:40474 Call#0 Retry#0: error:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.getTransferredContainers(AbstractYarnScheduler.java:48)
> at
> org.apache.hadoop.yarn.server.reso
[jira] [Commented] (YARN-1490) RM should optionally not kill all containers when an ApplicationMaster exits
[ https://issues.apache.org/jira/browse/YARN-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895523#comment-13895523 ] Hudson commented on YARN-1490: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #475 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/475/]) YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt > RM should optionally not kill all containers when an ApplicationMaster exits > > > Key: YARN-1490 > URL: https://issues.apache.org/jira/browse/YARN-1490 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: YARN-1490.1.patch, YARN-1490.10.patch, > YARN-1490.11.patch, YARN-1490.11.patch, YARN-1490.12.patch, > YARN-1490.2.patch, YARN-1490.3.patch, YARN-1490.4.patch, YARN-1490.5.patch, > YARN-1490.6.patch, YARN-1490.7.patch, YARN-1490.8.patch, YARN-1490.9.patch, > org.apache.oozie.service.TestRecoveryService_thread-dump.txt > > > This is needed to enable work-preserving AM restart. Some apps can chose to > reconnect with old running containers, some may not want to. This should be > an option. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1566) Change distributed-shell to retain containers from previous AppAttempt
[ https://issues.apache.org/jira/browse/YARN-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895524#comment-13895524 ] Hudson commented on YARN-1566: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #475 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/475/]) YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt > Change distributed-shell to retain containers from previous AppAttempt > -- > > Key: YARN-1566 > URL: https://issues.apache.org/jira/browse/YARN-1566 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: YARN-1566.1.patch, YARN-1566.2.patch > > > Change distributed-shell to reuse previous AM's running containers when AM is > restarting. It can also be made configurable whether to enable this feature > or not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1672) YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds
[ https://issues.apache.org/jira/browse/YARN-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895521#comment-13895521 ] Hudson commented on YARN-1672: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #475 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/475/]) YARN-1672. YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds (Naren Koneru via kasha) (kasha: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565866) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/NonAggregatingLogHandler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/TestNonAggregatingLogHandler.java > YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds > -- > > Key: YARN-1672 > URL: https://issues.apache.org/jira/browse/YARN-1672 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.2.0 >Reporter: Karthik Kambatla >Assignee: Naren Koneru >Priority: Trivial > Fix For: 2.4.0 > > Attachments: YARN-1672-1.patch > > > YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1497) Expose moving apps between queues on the command line
[ https://issues.apache.org/jira/browse/YARN-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895527#comment-13895527 ] Hudson commented on YARN-1497: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #475 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/475/]) YARN-1497. Command line additions for moving apps between queues (Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565754) * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/YarnClient.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/YarnClientImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/YarnCLI.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java > Expose moving apps between queues on the command line > - > > Key: YARN-1497 > URL: https://issues.apache.org/jira/browse/YARN-1497 > Project: Hadoop YARN > Issue Type: Sub-task > Components: client >Affects Versions: 2.2.0 >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Fix For: 3.0.0 > > Attachments: YARN-1497-1.patch, YARN-1497.patch, YARN-1497.patch > > -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1493) Schedulers don't recognize apps separately from app-attempts
[ https://issues.apache.org/jira/browse/YARN-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895517#comment-13895517 ] Hudson commented on YARN-1493: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #475 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/475/]) YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 are reverted from branch-2.3. Updating YARN's CHANGES.txt. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1565805) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt > Schedulers don't recognize apps separately from app-attempts > > > Key: YARN-1493 > URL: https://issues.apache.org/jira/browse/YARN-1493 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: YARN-1493.1.patch, YARN-1493.2.patch, YARN-1493.3.patch, > YARN-1493.4.patch, YARN-1493.5.patch, YARN-1493.6.patch, YARN-1493.7.patch, > YARN-1493.8.patch, YARN-1493.9.patch > > > Today, scheduler is tied to attempt only. > We need to separate app-level handling logic in scheduler. We can add new > app-level events to the scheduler and separate the app-level logic out. This > is good for work-preserving AM restart, RM restart, and also needed for > differentiating app-level metrics and attempt-level metrics. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1637) Implement a client library for java users to post entities+events
[
https://issues.apache.org/jira/browse/YARN-1637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895495#comment-13895495
]
Hadoop QA commented on YARN-1637:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12627793/YARN-1637.8.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3044//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3044//console
This message is automatically generated.
> Implement a client library for java users to post entities+events
> -
>
> Key: YARN-1637
> URL: https://issues.apache.org/jira/browse/YARN-1637
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Zhijie Shen
> Attachments: YARN-1637.1.patch, YARN-1637.2.patch, YARN-1637.3.patch,
> YARN-1637.4.patch, YARN-1637.5.patch, YARN-1637.6.patch, YARN-1637.7.patch,
> YARN-1637.8.patch
>
>
> This is a wrapper around the web-service to facilitate easy posting of
> entity+event data to the time-line server.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1637) Implement a client library for java users to post entities+events
[ https://issues.apache.org/jira/browse/YARN-1637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-1637: -- Attachment: YARN-1637.8.patch Thanks, Billie! Batch rebased. > Implement a client library for java users to post entities+events > - > > Key: YARN-1637 > URL: https://issues.apache.org/jira/browse/YARN-1637 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Zhijie Shen > Attachments: YARN-1637.1.patch, YARN-1637.2.patch, YARN-1637.3.patch, > YARN-1637.4.patch, YARN-1637.5.patch, YARN-1637.6.patch, YARN-1637.7.patch, > YARN-1637.8.patch > > > This is a wrapper around the web-service to facilitate easy posting of > entity+event data to the time-line server. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
