[jira] [Commented] (YARN-2431) NM restart: cgroup is not removed for reacquired containers
[ https://issues.apache.org/jira/browse/YARN-2431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122812#comment-14122812 ] Hudson commented on YARN-2431: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #671 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/671/]) YARN-2431. NM restart: cgroup is not removed for reacquired containers. Contributed by Jason Lowe (jlowe: rev 3fa5f728c4d6944302621965d6a0376827af5b51) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestLinuxContainerExecutor.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java NM restart: cgroup is not removed for reacquired containers --- Key: YARN-2431 URL: https://issues.apache.org/jira/browse/YARN-2431 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Affects Versions: 2.6.0 Reporter: Jason Lowe Assignee: Jason Lowe Fix For: 2.6.0 Attachments: YARN-2431.patch The cgroup for a reacquired container is not being removed when the container exits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2511) Allow All Origins by default when Cross Origin Filter is enabled
[ https://issues.apache.org/jira/browse/YARN-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122818#comment-14122818 ] Hudson commented on YARN-2511: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #671 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/671/]) YARN-2511. Allowed all origins by default when CrossOriginFilter is enabled. Contributed by Jonathan Eagles. (zjshen: rev 51a4faf52199b181471d591a9fde900fc166fc48) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/webapp/TestCrossOriginFilter.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/webapp/CrossOriginFilter.java * hadoop-yarn-project/CHANGES.txt Allow All Origins by default when Cross Origin Filter is enabled Key: YARN-2511 URL: https://issues.apache.org/jira/browse/YARN-2511 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Jonathan Eagles Assignee: Jonathan Eagles Fix For: 2.6.0 Attachments: YARN-2511-v1.patch This is the default for jetty 7 cross origin filter -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2509) Enable Cross Origin Filter for timeline server only and not all Yarn servers
[ https://issues.apache.org/jira/browse/YARN-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122817#comment-14122817 ] Hudson commented on YARN-2509: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #671 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/671/]) YARN-2509. Enable Cross Origin Filter for timeline server only and not all Yarn servers (Mit Desai via jeagles) (jeagles: rev 91d45f0f017281c8ab7ac858aa5d618accb9ea44) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java Enable Cross Origin Filter for timeline server only and not all Yarn servers Key: YARN-2509 URL: https://issues.apache.org/jira/browse/YARN-2509 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Jonathan Eagles Assignee: Mit Desai Fix For: 2.6.0 Attachments: YARN-2509.patch, YARN-2509.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2431) NM restart: cgroup is not removed for reacquired containers
[ https://issues.apache.org/jira/browse/YARN-2431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122940#comment-14122940 ] Hudson commented on YARN-2431: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1862 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1862/]) YARN-2431. NM restart: cgroup is not removed for reacquired containers. Contributed by Jason Lowe (jlowe: rev 3fa5f728c4d6944302621965d6a0376827af5b51) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestLinuxContainerExecutor.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java * hadoop-yarn-project/CHANGES.txt NM restart: cgroup is not removed for reacquired containers --- Key: YARN-2431 URL: https://issues.apache.org/jira/browse/YARN-2431 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Affects Versions: 2.6.0 Reporter: Jason Lowe Assignee: Jason Lowe Fix For: 2.6.0 Attachments: YARN-2431.patch The cgroup for a reacquired container is not being removed when the container exits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2511) Allow All Origins by default when Cross Origin Filter is enabled
[ https://issues.apache.org/jira/browse/YARN-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122947#comment-14122947 ] Hudson commented on YARN-2511: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1862 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1862/]) YARN-2511. Allowed all origins by default when CrossOriginFilter is enabled. Contributed by Jonathan Eagles. (zjshen: rev 51a4faf52199b181471d591a9fde900fc166fc48) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/webapp/TestCrossOriginFilter.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/webapp/CrossOriginFilter.java Allow All Origins by default when Cross Origin Filter is enabled Key: YARN-2511 URL: https://issues.apache.org/jira/browse/YARN-2511 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Jonathan Eagles Assignee: Jonathan Eagles Fix For: 2.6.0 Attachments: YARN-2511-v1.patch This is the default for jetty 7 cross origin filter -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2511) Allow All Origins by default when Cross Origin Filter is enabled
[ https://issues.apache.org/jira/browse/YARN-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122961#comment-14122961 ] Hudson commented on YARN-2511: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1887 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1887/]) YARN-2511. Allowed all origins by default when CrossOriginFilter is enabled. Contributed by Jonathan Eagles. (zjshen: rev 51a4faf52199b181471d591a9fde900fc166fc48) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/webapp/CrossOriginFilter.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/webapp/TestCrossOriginFilter.java Allow All Origins by default when Cross Origin Filter is enabled Key: YARN-2511 URL: https://issues.apache.org/jira/browse/YARN-2511 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Jonathan Eagles Assignee: Jonathan Eagles Fix For: 2.6.0 Attachments: YARN-2511-v1.patch This is the default for jetty 7 cross origin filter -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2438) yarn-env.sh cleanup
[ https://issues.apache.org/jira/browse/YARN-2438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123089#comment-14123089 ] John Smith commented on YARN-2438: -- How do the yarn heap sizes get set? yarn-env.sh cleanup --- Key: YARN-2438 URL: https://issues.apache.org/jira/browse/YARN-2438 Project: Hadoop YARN Issue Type: Bug Components: scripts Affects Versions: 3.0.0 Reporter: Allen Wittenauer Labels: newbie a) YARN_PROXYSERVER_OPTS and YARN_PROXYSERVER_HEAP are not documented b) Defaults should get moved to yarn-config.sh instead of being specifically set -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)
[ https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123134#comment-14123134 ] Vinod Kumar Vavilapalli commented on YARN-1708: --- ReservationUpdateResponse.setReservationName() is still public, it needs to be private. Add a public API to reserve resources (part of YARN-1051) - Key: YARN-1708 URL: https://issues.apache.org/jira/browse/YARN-1708 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Carlo Curino Assignee: Subramaniam Krishnan Attachments: YARN-1708.patch, YARN-1708.patch, YARN-1708.patch This JIRA tracks the definition of a new public API for YARN, which allows users to reserve resources (think of time-bounded queues). This is part of the admission control enhancement proposed in YARN-1051. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2515) Update ConverterUtils#toContainerId to parse epoch
[ https://issues.apache.org/jira/browse/YARN-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123218#comment-14123218 ] Jian He commented on YARN-2515: --- Patch looks good. ConvertUtils is marked private. Maybe we can provide a fromString method in ContainerId so that people can use that? Update ConverterUtils#toContainerId to parse epoch -- Key: YARN-2515 URL: https://issues.apache.org/jira/browse/YARN-2515 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Tsuyoshi OZAWA Assignee: Tsuyoshi OZAWA Attachments: YARN-2515.1.patch ContaienrId#toString was updated on YARN-2182. We should also update ConverterUtils#toContainerId to parse epoch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (YARN-2508) Cross Origin configuration parameters prefix are not honored
[ https://issues.apache.org/jira/browse/YARN-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mit Desai reassigned YARN-2508: --- Assignee: Mit Desai Cross Origin configuration parameters prefix are not honored Key: YARN-2508 URL: https://issues.apache.org/jira/browse/YARN-2508 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Jonathan Eagles Assignee: Mit Desai CrossOriginFilterInitializer specifies a prefix to have http server read parameters. However, it neither trims the prefix nor passes the prefix to the CrossOriginFilter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2509) Enable Cross Origin Filter for timeline server only and not all Yarn servers
[ https://issues.apache.org/jira/browse/YARN-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123249#comment-14123249 ] Zhijie Shen commented on YARN-2509: --- IMHO, it may be better to programmatically remove CrossOriginFilterInitializer if enabled = false, but I'm also fine if we can explicitly document the proper configuration. Enable Cross Origin Filter for timeline server only and not all Yarn servers Key: YARN-2509 URL: https://issues.apache.org/jira/browse/YARN-2509 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Jonathan Eagles Assignee: Mit Desai Fix For: 2.6.0 Attachments: YARN-2509.patch, YARN-2509.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2438) yarn-env.sh cleanup
[ https://issues.apache.org/jira/browse/YARN-2438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123288#comment-14123288 ] Allen Wittenauer commented on YARN-2438: I'm going to assume you are talking about the defaults that are listed in the yarn-env.sh file? The code sort of cheats. The actual default is set by HADOOP_HEAPSIZE. So really that default is a lie. Good catch! Looking at other stuff, I have a strong doubt that the yarn-policy file is actually used. I'll file a separate jira to deprecate that. The rest should get defined properly in the project init. yarn-env.sh cleanup --- Key: YARN-2438 URL: https://issues.apache.org/jira/browse/YARN-2438 Project: Hadoop YARN Issue Type: Bug Components: scripts Affects Versions: 3.0.0 Reporter: Allen Wittenauer Labels: newbie a) YARN_PROXYSERVER_OPTS and YARN_PROXYSERVER_HEAP are not documented b) Defaults should get moved to yarn-config.sh instead of being specifically set -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2515) Update ConverterUtils#toContainerId to parse epoch
[ https://issues.apache.org/jira/browse/YARN-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated YARN-2515: - Attachment: YARN-2515.2.patch Update ConverterUtils#toContainerId to parse epoch -- Key: YARN-2515 URL: https://issues.apache.org/jira/browse/YARN-2515 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Tsuyoshi OZAWA Assignee: Tsuyoshi OZAWA Attachments: YARN-2515.1.patch, YARN-2515.2.patch ContaienrId#toString was updated on YARN-2182. We should also update ConverterUtils#toContainerId to parse epoch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (YARN-2516) Deprecate yarn.policy.file
Allen Wittenauer created YARN-2516: -- Summary: Deprecate yarn.policy.file Key: YARN-2516 URL: https://issues.apache.org/jira/browse/YARN-2516 Project: Hadoop YARN Issue Type: Improvement Reporter: Allen Wittenauer It doesn't appear that yarn.policy.file is actually used anywhere, there isn't an example yarn-policy.xml file, etc, etc. So let's remove it from the shell code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2515) Update ConverterUtils#toContainerId to parse epoch
[ https://issues.apache.org/jira/browse/YARN-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123305#comment-14123305 ] Tsuyoshi OZAWA commented on YARN-2515: -- Thanks for your review, Jian. Updated to add ContainerId#fromString based on a following design: * I moved the contents of ConverterUtils#toContainerId into ContainerId#fromString because ConverterUtils is in hadoop-yarn-common and hadoop-yarn-api doesn't depend on hadoop-yarn-common. * A helper method, ConverterUtils#toApplicationAttemptId, is useful for implementing ContainerId#fromString, so the method is copied as ContainerId#toApplicationAttemptId. Update ConverterUtils#toContainerId to parse epoch -- Key: YARN-2515 URL: https://issues.apache.org/jira/browse/YARN-2515 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Tsuyoshi OZAWA Assignee: Tsuyoshi OZAWA Attachments: YARN-2515.1.patch, YARN-2515.2.patch ContaienrId#toString was updated on YARN-2182. We should also update ConverterUtils#toContainerId to parse epoch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2516) Deprecate yarn.policy.file
[ https://issues.apache.org/jira/browse/YARN-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated YARN-2516: --- Component/s: scripts Deprecate yarn.policy.file -- Key: YARN-2516 URL: https://issues.apache.org/jira/browse/YARN-2516 Project: Hadoop YARN Issue Type: Improvement Components: scripts Reporter: Allen Wittenauer Labels: newbie It doesn't appear that yarn.policy.file is actually used anywhere, there isn't an example yarn-policy.xml file, etc, etc. So let's remove it from the shell code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2516) Deprecate yarn.policy.file
[ https://issues.apache.org/jira/browse/YARN-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated YARN-2516: --- Labels: newbie (was: ) Deprecate yarn.policy.file -- Key: YARN-2516 URL: https://issues.apache.org/jira/browse/YARN-2516 Project: Hadoop YARN Issue Type: Improvement Components: scripts Reporter: Allen Wittenauer Labels: newbie It doesn't appear that yarn.policy.file is actually used anywhere, there isn't an example yarn-policy.xml file, etc, etc. So let's remove it from the shell code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2179) Initial cache manager structure and context
[ https://issues.apache.org/jira/browse/YARN-2179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated YARN-2179: --- Attachment: YARN-2179-trunk-v5.patch Attached v5. Update to match patch in YARN-1492. Initial cache manager structure and context --- Key: YARN-2179 URL: https://issues.apache.org/jira/browse/YARN-2179 Project: Hadoop YARN Issue Type: Sub-task Reporter: Chris Trezzo Assignee: Chris Trezzo Attachments: YARN-2179-trunk-v1.patch, YARN-2179-trunk-v2.patch, YARN-2179-trunk-v3.patch, YARN-2179-trunk-v4.patch, YARN-2179-trunk-v5.patch Implement the initial shared cache manager structure and context. The SCMContext will be used by a number of manager services (i.e. the backing store and the cleaner service). The AppChecker is used to gather the currently running applications on SCM startup (necessary for an scm that is backed by an in-memory store). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2180) In-memory backing store for cache manager
[ https://issues.apache.org/jira/browse/YARN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated YARN-2180: --- Attachment: YARN-2180-trunk-v3.patch Attached v3. Updated to match patch in YARN-1492. In-memory backing store for cache manager - Key: YARN-2180 URL: https://issues.apache.org/jira/browse/YARN-2180 Project: Hadoop YARN Issue Type: Sub-task Reporter: Chris Trezzo Assignee: Chris Trezzo Attachments: YARN-2180-trunk-v1.patch, YARN-2180-trunk-v2.patch, YARN-2180-trunk-v3.patch Implement an in-memory backing store for the cache manager. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2183) Cleaner service for cache manager
[ https://issues.apache.org/jira/browse/YARN-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated YARN-2183: --- Attachment: YARN-2183-trunk-v3.patch Attached v3. Updated to match patch in YARN-1492. Cleaner service for cache manager - Key: YARN-2183 URL: https://issues.apache.org/jira/browse/YARN-2183 Project: Hadoop YARN Issue Type: Sub-task Reporter: Chris Trezzo Assignee: Chris Trezzo Attachments: YARN-2183-trunk-v1.patch, YARN-2183-trunk-v2.patch, YARN-2183-trunk-v3.patch Implement the cleaner service for the cache manager along with metrics for the service. This service is responsible for cleaning up old resource references in the manager and removing stale entries from the cache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2186) Node Manager uploader service for cache manager
[ https://issues.apache.org/jira/browse/YARN-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated YARN-2186: --- Attachment: YARN-2186-trunk-v3.patch Attached v3. Updated to match patch in YARN-1492. Node Manager uploader service for cache manager --- Key: YARN-2186 URL: https://issues.apache.org/jira/browse/YARN-2186 Project: Hadoop YARN Issue Type: Sub-task Reporter: Chris Trezzo Assignee: Chris Trezzo Attachments: YARN-2186-trunk-v1.patch, YARN-2186-trunk-v2.patch, YARN-2186-trunk-v3.patch Implement the node manager uploader service for the cache manager. This service is responsible for communicating with the node manager when it uploads resources to the shared cache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2189) Admin service for cache manager
[ https://issues.apache.org/jira/browse/YARN-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated YARN-2189: --- Attachment: YARN-2189-trunk-v4.patch Attached v4. Updated to match patch in YARN-1492. Admin service for cache manager --- Key: YARN-2189 URL: https://issues.apache.org/jira/browse/YARN-2189 Project: Hadoop YARN Issue Type: Sub-task Reporter: Chris Trezzo Assignee: Chris Trezzo Attachments: YARN-2189-trunk-v1.patch, YARN-2189-trunk-v2.patch, YARN-2189-trunk-v3.patch, YARN-2189-trunk-v4.patch Implement the admin service for the shared cache manager. This service is responsible for handling administrative commands such as manually running a cleaner task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2203) Web UI for cache manager
[ https://issues.apache.org/jira/browse/YARN-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated YARN-2203: --- Attachment: YARN-2203-trunk-v3.patch Attached v3. Updated to match patch in YARN-1492. Web UI for cache manager Key: YARN-2203 URL: https://issues.apache.org/jira/browse/YARN-2203 Project: Hadoop YARN Issue Type: Sub-task Reporter: Chris Trezzo Assignee: Chris Trezzo Attachments: YARN-2203-trunk-v1.patch, YARN-2203-trunk-v2.patch, YARN-2203-trunk-v3.patch Implement the web server and web ui for the cache manager. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2515) Update ConverterUtils#toContainerId to parse epoch
[
https://issues.apache.org/jira/browse/YARN-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123353#comment-14123353
]
Hadoop QA commented on YARN-2515:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12666835/YARN-2515.2.patch
against trunk revision 9609b73.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4835//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4835//console
This message is automatically generated.
Update ConverterUtils#toContainerId to parse epoch
--
Key: YARN-2515
URL: https://issues.apache.org/jira/browse/YARN-2515
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Tsuyoshi OZAWA
Assignee: Tsuyoshi OZAWA
Attachments: YARN-2515.1.patch, YARN-2515.2.patch
ContaienrId#toString was updated on YARN-2182. We should also update
ConverterUtils#toContainerId to parse epoch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2154) FairScheduler: Improve preemption to preempt only those containers that would satisfy the incoming request
[
https://issues.apache.org/jira/browse/YARN-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123380#comment-14123380
]
Karthik Kambatla commented on YARN-2154:
At an abstract level, I propose the following:
# For each queue, the child {{Schedulable}}s should be sorted in the ascending
order of (desiredShare - fairShare). In case of FSLeafQueue, this corresponds
to running applications.
# Add {{FSQueue#estimateStarvation(ListFSAppAttempt)}} to find enough
applications, considering minshare and fairshare starvation in that order. Or,
should we have two methods, one for each kind of starvation.
# For a starved queue, we do a depth-first traversal (in the order of
deprivation) to find these applications. If none of child queues/applications
are under their shares, we pick the application (if it exists) with a positive
demand and least over its share.
# Once the application list is computed, we traverse through the
ResourceRequests (ordered first by application, second by priority) and preempt
any matching containers currently assigned to an application over its share.
# As a follow up, we can consider looking at this application list first at
schedule time.
Thoughts?
FairScheduler: Improve preemption to preempt only those containers that would
satisfy the incoming request
--
Key: YARN-2154
URL: https://issues.apache.org/jira/browse/YARN-2154
Project: Hadoop YARN
Issue Type: Improvement
Components: fairscheduler
Affects Versions: 2.4.0
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Priority: Critical
Today, FairScheduler uses a spray-gun approach to preemption. Instead, it
should only preempt resources that would satisfy the incoming request.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2154) FairScheduler: Improve preemption to preempt only those containers that would satisfy the incoming request
[
https://issues.apache.org/jira/browse/YARN-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123541#comment-14123541
]
Wei Yan commented on YARN-2154:
---
bq. 1. For each queue, the child {{Schedulable}}s should be sorted in the
ascending order of (desiredShare - fairShare). In case of FSLeafQueue, this
corresponds to running applications.
Do we still need to do the sorting? As we traverse from the root queue, and
take applications from the queues which are starvated for min/fair share. I
think we collect all applications from starvated queues.
bq. 4. Once the application list is computed, we traverse through the
ResourceRequests (ordered first by application, second by priority) and preempt
any matching containers currently assigned to an application over its share.
To sort the applications, do we still need to follow the policy? For example,
for two applications belong to the same FSLeafQueue, they should be sorted
according to the leafQueue's policy; for two applications belong to two
different leaf queues, may need to refer to parent queue's policy.
And, after preempting one container and assigning the resource to one
application in the list, we also need to check whether that application and its
parent queue is still starvated.
FairScheduler: Improve preemption to preempt only those containers that would
satisfy the incoming request
--
Key: YARN-2154
URL: https://issues.apache.org/jira/browse/YARN-2154
Project: Hadoop YARN
Issue Type: Improvement
Components: fairscheduler
Affects Versions: 2.4.0
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Priority: Critical
Today, FairScheduler uses a spray-gun approach to preemption. Instead, it
should only preempt resources that would satisfy the incoming request.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2154) FairScheduler: Improve preemption to preempt only those containers that would satisfy the incoming request
[ https://issues.apache.org/jira/browse/YARN-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123608#comment-14123608 ] Karthik Kambatla commented on YARN-2154: bq. Do we still need to do the sorting? As we traverse from the root queue, and take applications from the queues which are starvated for min/fair share. I think we collect all applications from starvated queues. Queues might starve without their children starving. The sorting is for those cases. bq. To sort the applications, do we still need to follow the policy? We shouldn't need to sort the applications. The first application corresponds to the queue that is starved the most even though the application itself might not be the most starved one. FairScheduler: Improve preemption to preempt only those containers that would satisfy the incoming request -- Key: YARN-2154 URL: https://issues.apache.org/jira/browse/YARN-2154 Project: Hadoop YARN Issue Type: Improvement Components: fairscheduler Affects Versions: 2.4.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Priority: Critical Today, FairScheduler uses a spray-gun approach to preemption. Instead, it should only preempt resources that would satisfy the incoming request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1492) truly shared cache for jars (jobjar/libjar)
[
https://issues.apache.org/jira/browse/YARN-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123617#comment-14123617
]
Chris Trezzo commented on YARN-1492:
The patch is now +1 overall. Please note that I have broken up this patch into
smaller patches that are located in each of the sub tasks on this issue. If you
would like to try out the feature, here are the simple requirements for setting
up the shared cache:
1. In HDFS, create the shared cache root directory (set to /sharedcache by
default).
2. In mapred-site.xml add the following parameter:
{noformat}
property
namemapreduce.job.sharedcache.mode/name
valuejobjar,libjars,files,archives/value
description
A comma delimited list of resource categories to submit to the shared cache.
The valid categories are: jobjar, libjars, files, archives.
If disabled is specified then the job submission code will not use
the shared cache.
/description
/property
{noformat}
3. In yarn-site.xml add the following parameter:
{noformat}
property
descriptionWhether the shared cache is enabled/description
nameyarn.sharedcache.enabled/name
valueenabled/value
/property
{noformat}
4. Start the SCM (shared cache manager) using the regular yarn shell scripts.
{noformat}
./yarn-daemon.sh start sharedcachemanager
{noformat}
With this setup all job jars, lib jars, files and archives specified by
MapReduce jobs will be automatically cached.
truly shared cache for jars (jobjar/libjar)
---
Key: YARN-1492
URL: https://issues.apache.org/jira/browse/YARN-1492
Project: Hadoop YARN
Issue Type: New Feature
Affects Versions: 2.0.4-alpha
Reporter: Sangjin Lee
Assignee: Chris Trezzo
Attachments: YARN-1492-all-trunk-v1.patch,
YARN-1492-all-trunk-v2.patch, YARN-1492-all-trunk-v3.patch,
YARN-1492-all-trunk-v4.patch, YARN-1492-all-trunk-v5.patch,
shared_cache_design.pdf, shared_cache_design_v2.pdf,
shared_cache_design_v3.pdf, shared_cache_design_v4.pdf,
shared_cache_design_v5.pdf
Currently there is the distributed cache that enables you to cache jars and
files so that attempts from the same job can reuse them. However, sharing is
limited with the distributed cache because it is normally on a per-job basis.
On a large cluster, sometimes copying of jobjars and libjars becomes so
prevalent that it consumes a large portion of the network bandwidth, not to
speak of defeating the purpose of bringing compute to where data is. This
is wasteful because in most cases code doesn't change much across many jobs.
I'd like to propose and discuss feasibility of introducing a truly shared
cache so that multiple jobs from multiple users can share and cache jars.
This JIRA is to open the discussion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-2508) Cross Origin configuration parameters prefix are not honored
[ https://issues.apache.org/jira/browse/YARN-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mit Desai updated YARN-2508: Attachment: YARN-2508.patch Attaching the patch. Cross Origin configuration parameters prefix are not honored Key: YARN-2508 URL: https://issues.apache.org/jira/browse/YARN-2508 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Jonathan Eagles Assignee: Mit Desai Attachments: YARN-2508.patch CrossOriginFilterInitializer specifies a prefix to have http server read parameters. However, it neither trims the prefix nor passes the prefix to the CrossOriginFilter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1708) Add a public API to reserve resources (part of YARN-1051)
[ https://issues.apache.org/jira/browse/YARN-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123687#comment-14123687 ] Vinod Kumar Vavilapalli commented on YARN-1708: --- Sorry, my bad. I misread it. Patch looks good. +1. Add a public API to reserve resources (part of YARN-1051) - Key: YARN-1708 URL: https://issues.apache.org/jira/browse/YARN-1708 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Carlo Curino Assignee: Subramaniam Krishnan Attachments: YARN-1708.patch, YARN-1708.patch, YARN-1708.patch This JIRA tracks the definition of a new public API for YARN, which allows users to reserve resources (think of time-bounded queues). This is part of the admission control enhancement proposed in YARN-1051. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2508) Cross Origin configuration parameters prefix are not honored
[
https://issues.apache.org/jira/browse/YARN-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123713#comment-14123713
]
Hadoop QA commented on YARN-2508:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12666908/YARN-2508.patch
against trunk revision 21c0cde.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4836//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4836//console
This message is automatically generated.
Cross Origin configuration parameters prefix are not honored
Key: YARN-2508
URL: https://issues.apache.org/jira/browse/YARN-2508
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Reporter: Jonathan Eagles
Assignee: Mit Desai
Attachments: YARN-2508.patch
CrossOriginFilterInitializer specifies a prefix to have http server read
parameters. However, it neither trims the prefix nor passes the prefix to the
CrossOriginFilter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-1530) [Umbrella] Store, manage and serve per-framework application-timeline data
[ https://issues.apache.org/jira/browse/YARN-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kanter updated YARN-1530: Attachment: ATS-Write-Pipeline-Design-Proposal.pdf Thanks Sangjin for posting those notes. I've written up a rough proposal based on some of those discussions on how we can improve the scalability and reliability of the ATS write path and attached it to this JIRA: ATS-Write-Pipeline-Design-Proposal.pdf. I'd like to use this to drive further discussions and hopefully let us all work towards this design (or a design that comes out of these discussions). [Umbrella] Store, manage and serve per-framework application-timeline data -- Key: YARN-1530 URL: https://issues.apache.org/jira/browse/YARN-1530 Project: Hadoop YARN Issue Type: Bug Reporter: Vinod Kumar Vavilapalli Attachments: ATS-Write-Pipeline-Design-Proposal.pdf, ATS-meet-up-8-28-2014-notes.pdf, application timeline design-20140108.pdf, application timeline design-20140116.pdf, application timeline design-20140130.pdf, application timeline design-20140210.pdf This is a sibling JIRA for YARN-321. Today, each application/framework has to do store, and serve per-framework data all by itself as YARN doesn't have a common solution. This JIRA attempts to solve the storage, management and serving of per-framework data from various applications, both running and finished. The aim is to change YARN to collect and store data in a generic manner with plugin points for frameworks to do their own thing w.r.t interpretation and serving. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (YARN-2517) Implement TimelineClientAsync
Zhijie Shen created YARN-2517: - Summary: Implement TimelineClientAsync Key: YARN-2517 URL: https://issues.apache.org/jira/browse/YARN-2517 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen In some scenarios, we'd like to put timeline entities in another thread no to block the current one. It's good to have a TimelineClientAsync like AMRMClientAsync and NMClientAsync. It can buffer entities, put them in a separate thread, and have callback to handle the responses. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2508) Cross Origin configuration parameters prefix are not honored
[ https://issues.apache.org/jira/browse/YARN-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123780#comment-14123780 ] Zhijie Shen commented on YARN-2508: --- +1, LGTM Cross Origin configuration parameters prefix are not honored Key: YARN-2508 URL: https://issues.apache.org/jira/browse/YARN-2508 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Jonathan Eagles Assignee: Mit Desai Attachments: YARN-2508.patch CrossOriginFilterInitializer specifies a prefix to have http server read parameters. However, it neither trims the prefix nor passes the prefix to the CrossOriginFilter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2508) Cross Origin configuration parameters prefix are not honored
[ https://issues.apache.org/jira/browse/YARN-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-2508: -- Hadoop Flags: Reviewed Cross Origin configuration parameters prefix are not honored Key: YARN-2508 URL: https://issues.apache.org/jira/browse/YARN-2508 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Jonathan Eagles Assignee: Mit Desai Attachments: YARN-2508.patch CrossOriginFilterInitializer specifies a prefix to have http server read parameters. However, it neither trims the prefix nor passes the prefix to the CrossOriginFilter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (YARN-2518) Support in-process container executor
BoYang created YARN-2518: Summary: Support in-process container executor Key: YARN-2518 URL: https://issues.apache.org/jira/browse/YARN-2518 Project: Hadoop YARN Issue Type: New Feature Components: nodemanager Affects Versions: 2.5.0 Environment: Linux, Windows Reporter: BoYang Priority: Minor Node Manage always creates a new process for a new application. We have hit a scenario where we want the node manager to execute the application inside its own process, so we get fast response time. It would be nice if Node Manager or YARN can provide native support for that. In general, the scenario is that we have a long running process which can accept requests and process the requests inside its own process. Since YARN is good at scheduling jobs, we want to use YARN to dispatch jobs (e.g. requests in JSON) to the long running process. In that case, we do not want YARN container to spin up a new process for each request. Instead, we want YARN container to send the request to the long running process for further processing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2518) Support in-process container executor
[ https://issues.apache.org/jira/browse/YARN-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123816#comment-14123816 ] BoYang commented on YARN-2518: -- We tried to use a custom Container Executor implementation, and hook it up with YARN. It seems working. But it would be still nice if YARN can provide native support for in-process container executor. Support in-process container executor - Key: YARN-2518 URL: https://issues.apache.org/jira/browse/YARN-2518 Project: Hadoop YARN Issue Type: New Feature Components: nodemanager Affects Versions: 2.5.0 Environment: Linux, Windows Reporter: BoYang Priority: Minor Labels: container, dispatch, in-process, job, node Node Manage always creates a new process for a new application. We have hit a scenario where we want the node manager to execute the application inside its own process, so we get fast response time. It would be nice if Node Manager or YARN can provide native support for that. In general, the scenario is that we have a long running process which can accept requests and process the requests inside its own process. Since YARN is good at scheduling jobs, we want to use YARN to dispatch jobs (e.g. requests in JSON) to the long running process. In that case, we do not want YARN container to spin up a new process for each request. Instead, we want YARN container to send the request to the long running process for further processing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1709) Admission Control: Reservation subsystem
[
https://issues.apache.org/jira/browse/YARN-1709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123885#comment-14123885
]
Chris Douglas commented on YARN-1709:
-
Overall, the patch lgtm. Just a few minor tweaks, then I'm +1
* very minor: Javadoc could be compressed a bit (empty lines)
{{InMemoryPlan}}
* The {{ZERO_RESOURCE}} instance escapes via {{getConsumptionForUser}}
* Some lines are more than 80 characters
* The logging can use built-in substitution more efficiently. Instead of:
{code}
String errMsg =
MessageFormat
.format(
The specified Reservation with ID {0} does not exist in the plan,
reservation.getReservationId());
LOG.error(errMsg);
{code}
Prefer:
{code}
LOG.error(The specified Reservation with ID {} does not exist in the plan,
reservation.getReservationId());
{code}
Some of the code already uses this construction, but a few still use
{{MessageFormat}}.
* This form is harder to read:
{code}
InMemoryReservationAllocation inMemReservation = null;
if (reservation instanceof InMemoryReservationAllocation) {
inMemReservation = (InMemoryReservationAllocation) reservation;
} else {
// [snip] log error
throw new RuntimeException(errMsg);
}
{code}
than the if (error) { throw; } construction used the other checks. Is it an
improvement over {{ClassCastException}}?
* {{addReservation}} doesn't need to hold the write lock while it checks
invariants on its arguments
* The private methods that assume locks ({{incrementAllocation}},
{{decrementAllocation}}, {{removeReservation}}, etc.) are held should probably
{{assert}} that precondition (e.g., {{RRWL::isWriteLockedByCurrentThread()}})
* {{getMinimumAllocation}} and {{getMaximumAllocation}} return mutable data
that should probably be cloned
{{InMemoryReservationAllocation}}
* minor style: redundant {{this}} in get methods
* {{toString}} should use {{StringBuilder}} instead of {{StringBuffer}}
{{PlanView}}
* Mismatched javadoc on {{getEarliestStartTime}}
* {{getLastEndTime}} specifies UTC. Is that enforced in the implementation?
{{ReservationInterval}}
* Can this be made immutable? It's a key in several maps
{{RLESparseResourceAllocation}}
* Though some methods in {{InMemoryPlan}}, the {{ZERO_RESOURCE}} internal
variable can escape via {{getCapacityAtTime}}.
Admission Control: Reservation subsystem
Key: YARN-1709
URL: https://issues.apache.org/jira/browse/YARN-1709
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Carlo Curino
Assignee: Subramaniam Krishnan
Attachments: YARN-1709.patch, YARN-1709.patch, YARN-1709.patch
This JIRA is about the key data structure used to track resources over time
to enable YARN-1051. The Reservation subsystem is conceptually a plan of
how the scheduler will allocate resources over-time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (YARN-2512) Allow for origin pattern matching in cross origin filter
[ https://issues.apache.org/jira/browse/YARN-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles reassigned YARN-2512: - Assignee: Jonathan Eagles Allow for origin pattern matching in cross origin filter Key: YARN-2512 URL: https://issues.apache.org/jira/browse/YARN-2512 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Jonathan Eagles Assignee: Jonathan Eagles -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2512) Allow for origin pattern matching in cross origin filter
[ https://issues.apache.org/jira/browse/YARN-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated YARN-2512: -- Attachment: YARN-2512-v1.patch Allow for origin pattern matching in cross origin filter Key: YARN-2512 URL: https://issues.apache.org/jira/browse/YARN-2512 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Jonathan Eagles Assignee: Jonathan Eagles Attachments: YARN-2512-v1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2512) Allow for origin pattern matching in cross origin filter
[ https://issues.apache.org/jira/browse/YARN-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated YARN-2512: -- Description: Extending the feature set of allowed origins. Now a * in a pattern indicates this allowed origin is a pattern and will be matched including multiple sub-domains. Allow for origin pattern matching in cross origin filter Key: YARN-2512 URL: https://issues.apache.org/jira/browse/YARN-2512 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Jonathan Eagles Assignee: Jonathan Eagles Attachments: YARN-2512-v1.patch Extending the feature set of allowed origins. Now a * in a pattern indicates this allowed origin is a pattern and will be matched including multiple sub-domains. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2512) Allow for origin pattern matching in cross origin filter
[ https://issues.apache.org/jira/browse/YARN-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14124248#comment-14124248 ] Jonathan Eagles commented on YARN-2512: --- [~zjshen], can you please review this cross-origin change for use with the timeline server? Allow for origin pattern matching in cross origin filter Key: YARN-2512 URL: https://issues.apache.org/jira/browse/YARN-2512 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Jonathan Eagles Assignee: Jonathan Eagles Attachments: YARN-2512-v1.patch Extending the feature set of allowed origins. Now a * in a pattern indicates this allowed origin is a pattern and will be matched including multiple sub-domains. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2512) Allow for origin pattern matching in cross origin filter
[
https://issues.apache.org/jira/browse/YARN-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14124271#comment-14124271
]
Hadoop QA commented on YARN-2512:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12666971/YARN-2512-v1.patch
against trunk revision e6420fe.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4837//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4837//console
This message is automatically generated.
Allow for origin pattern matching in cross origin filter
Key: YARN-2512
URL: https://issues.apache.org/jira/browse/YARN-2512
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Reporter: Jonathan Eagles
Assignee: Jonathan Eagles
Attachments: YARN-2512-v1.patch
Extending the feature set of allowed origins. Now a * in a pattern
indicates this allowed origin is a pattern and will be matched including
multiple sub-domains.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-2507) Document Cross Origin Filter Configuration for ATS
[ https://issues.apache.org/jira/browse/YARN-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated YARN-2507: -- Attachment: YARN-2507-v1.patch Document Cross Origin Filter Configuration for ATS -- Key: YARN-2507 URL: https://issues.apache.org/jira/browse/YARN-2507 Project: Hadoop YARN Issue Type: Sub-task Components: documentation, timelineserver Affects Versions: 2.6.0 Reporter: Jonathan Eagles Attachments: YARN-2507-v1.patch CORS support was added for ATS as part of YARN-2277. This jira is to document configuration for ATS CORS support. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2507) Document Cross Origin Filter Configuration for ATS
[
https://issues.apache.org/jira/browse/YARN-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14124292#comment-14124292
]
Hadoop QA commented on YARN-2507:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12666982/YARN-2507-v1.patch
against trunk revision e6420fe.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4838//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4838//console
This message is automatically generated.
Document Cross Origin Filter Configuration for ATS
--
Key: YARN-2507
URL: https://issues.apache.org/jira/browse/YARN-2507
Project: Hadoop YARN
Issue Type: Sub-task
Components: documentation, timelineserver
Affects Versions: 2.6.0
Reporter: Jonathan Eagles
Assignee: Jonathan Eagles
Attachments: YARN-2507-v1.patch
CORS support was added for ATS as part of YARN-2277. This jira is to document
configuration for ATS CORS support.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-2284) Find missing config options in YarnConfiguration and yarn-default.xml
[ https://issues.apache.org/jira/browse/YARN-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated YARN-2284: - Attachment: YARN-2284-07.patch Add a few properties to ignore. Find missing config options in YarnConfiguration and yarn-default.xml - Key: YARN-2284 URL: https://issues.apache.org/jira/browse/YARN-2284 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 2.4.1 Reporter: Ray Chiang Assignee: Ray Chiang Priority: Minor Labels: supportability Attachments: YARN-2284-04.patch, YARN-2284-05.patch, YARN-2284-06.patch, YARN-2284-07.patch, YARN2284-01.patch, YARN2284-02.patch, YARN2284-03.patch YarnConfiguration has one set of properties. yarn-default.xml has another set of properties. Ideally, there should be an automatic way to find missing properties in either location. This is analogous to MAPREDUCE-5130, but for yarn-default.xml. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2284) Find missing config options in YarnConfiguration and yarn-default.xml
[
https://issues.apache.org/jira/browse/YARN-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14124337#comment-14124337
]
Hadoop QA commented on YARN-2284:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12666987/YARN-2284-07.patch
against trunk revision e6420fe.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common:
org.apache.hadoop.ha.TestZKFailoverControllerStress
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4839//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4839//console
This message is automatically generated.
Find missing config options in YarnConfiguration and yarn-default.xml
-
Key: YARN-2284
URL: https://issues.apache.org/jira/browse/YARN-2284
Project: Hadoop YARN
Issue Type: Improvement
Affects Versions: 2.4.1
Reporter: Ray Chiang
Assignee: Ray Chiang
Priority: Minor
Labels: supportability
Attachments: YARN-2284-04.patch, YARN-2284-05.patch,
YARN-2284-06.patch, YARN-2284-07.patch, YARN2284-01.patch, YARN2284-02.patch,
YARN2284-03.patch
YarnConfiguration has one set of properties. yarn-default.xml has another
set of properties. Ideally, there should be an automatic way to find missing
properties in either location.
This is analogous to MAPREDUCE-5130, but for yarn-default.xml.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
