[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15369791#comment-15369791 ] Hudson commented on YARN-3623: -- SUCCESS: Integrated in Hadoop-trunk-Commit #10074 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10074/]) YARN-3623. Add a config to indicate the Timeline Service version. (sjlee: rev cf4666bb8ccc99a63c623455cd88157a3007775d) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: New Feature > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Fix For: 2.8.0 > > Attachments: YARN-3623-2015-11-19.1.patch, > YARN-3623-2015-12-09.patch, YARN-3623-addendum.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15051236#comment-15051236 ] Junping Du commented on YARN-3623: -- bq. "it means the cluster will and should bring up the timeline service v.1.5 (and nothing else)." Think over it again. I am still uncomfortable for the description - "(and nothing else)." It could be against for our possible solutions for YARN-4368. I prefer to remove this in an addendum patch or it could provide unnecessary restrictions if it get released in 2.8.0. Thoughts? > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: New Feature > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Fix For: 2.8.0 > > Attachments: YARN-3623-2015-11-19.1.patch, YARN-3623-2015-12-09.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15051478#comment-15051478 ] Hudson commented on YARN-3623: -- FAILURE: Integrated in Hadoop-trunk-Commit #8951 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8951/]) YARN-3623-Addendum: Improve the description for Timeline Service Version (xgong: rev 21daa6c68a0bff51a14e748bf14d56b2f5a5580f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: New Feature > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Fix For: 2.8.0 > > Attachments: YARN-3623-2015-11-19.1.patch, > YARN-3623-2015-12-09.patch, YARN-3623-addendum.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15051459#comment-15051459 ] Xuan Gong commented on YARN-3623: - +1 Checking this in > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: New Feature > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Fix For: 2.8.0 > > Attachments: YARN-3623-2015-11-19.1.patch, > YARN-3623-2015-12-09.patch, YARN-3623-addendum.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15051463#comment-15051463 ] Xuan Gong commented on YARN-3623: - Committed the addendum patch in trunk/branch-2. Thanks, junping. > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: New Feature > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Fix For: 2.8.0 > > Attachments: YARN-3623-2015-11-19.1.patch, > YARN-3623-2015-12-09.patch, YARN-3623-addendum.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15051335#comment-15051335 ] Naganarasimha G R commented on YARN-3623: - +1 lgtm > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: New Feature > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Fix For: 2.8.0 > > Attachments: YARN-3623-2015-11-19.1.patch, > YARN-3623-2015-12-09.patch, YARN-3623-addendum.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15051419#comment-15051419 ] Junping Du commented on YARN-3623: -- Yes. That's the flexibility we want here - we probably bring up old version ATS service for legacy running apps during upgrade. So we should pay more attentions here given configurations (especially in yarn-default.xml) serve as a public protocol to hadoop users. > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: New Feature > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Fix For: 2.8.0 > > Attachments: YARN-3623-2015-11-19.1.patch, > YARN-3623-2015-12-09.patch, YARN-3623-addendum.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15050551#comment-15050551 ] Junping Du commented on YARN-3623: -- Ok. I think I already find the answer from YARN-4234 that ATS 1.5 depends on this patch. Will go ahead to commit it to 2.8. > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Attachments: YARN-3623-2015-11-19.1.patch, YARN-3623-2015-12-09.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15050496#comment-15050496 ] Junping Du commented on YARN-3623: -- Oh. Just noticed target version is YARN-2928 but I think ATS 1.5 need this also. Isn't it? Shall we commit the patch to trunk/branch-2? > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Attachments: YARN-3623-2015-11-19.1.patch, YARN-3623-2015-12-09.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15050592#comment-15050592 ] Hudson commented on YARN-3623: -- FAILURE: Integrated in Hadoop-trunk-Commit #8950 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8950/]) YARN-3623. Add a config to indicate the Timeline Service version. (junping_du: rev f910e4f639dc311fcb257bfcb869b1aa8b2c0643) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: New Feature > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Fix For: 2.8.0 > > Attachments: YARN-3623-2015-11-19.1.patch, YARN-3623-2015-12-09.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15050494#comment-15050494 ] Junping Du commented on YARN-3623: -- +1. Committing. > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Attachments: YARN-3623-2015-11-19.1.patch, YARN-3623-2015-12-09.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15051264#comment-15051264 ] Sangjin Lee commented on YARN-3623: --- That's why I asked whether you were OK with the wording. How about "it means the cluster should bring up the timeline service specified by the version" (dropping the exclusive phrase)? > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: New Feature > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Fix For: 2.8.0 > > Attachments: YARN-3623-2015-11-19.1.patch, YARN-3623-2015-12-09.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15051377#comment-15051377 ] Li Lu commented on YARN-3623: - I'd like to make sure I understand this change correctly. Specifically, what kind of new cases do we allow by removing the "nothing else" part. IIUC, this allows us to bring up an old ATS server after the system has been upgraded. This looks fine with me. Am I missing some other cases introduced by this change? > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: New Feature > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Fix For: 2.8.0 > > Attachments: YARN-3623-2015-11-19.1.patch, > YARN-3623-2015-12-09.patch, YARN-3623-addendum.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15051426#comment-15051426 ] Li Lu commented on YARN-3623: - Agree. +1 for the change. > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: New Feature > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Fix For: 2.8.0 > > Attachments: YARN-3623-2015-11-19.1.patch, > YARN-3623-2015-12-09.patch, YARN-3623-addendum.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15051133#comment-15051133 ] Hudson commented on YARN-3623: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #682 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/682/]) YARN-3623. Add a config to indicate the Timeline Service version. (junping_du: rev f910e4f639dc311fcb257bfcb869b1aa8b2c0643) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml * hadoop-yarn-project/CHANGES.txt > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: New Feature > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Fix For: 2.8.0 > > Attachments: YARN-3623-2015-11-19.1.patch, YARN-3623-2015-12-09.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15049169#comment-15049169 ] Li Lu commented on YARN-3623: - Hi folks, seems like we're reaching agreement on the config itself, but still having some concerns with timeline v2 rolling upgrade. How about fix the description as Junping suggested, put this patch in while we keep our v2 discussion in YARN-3196? Looks like the only action item for this JIRA is to update the description? > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Attachments: YARN-3623-2015-11-19.1.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15049488#comment-15049488 ] Sangjin Lee commented on YARN-3623: --- I'm obviously +1 as that's exactly what I said. :) [~djp]? > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Attachments: YARN-3623-2015-11-19.1.patch, YARN-3623-2015-12-09.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15049449#comment-15049449 ] Li Lu commented on YARN-3623: - The new description LGTM. [~sjlee0] any suggestions? Thanks! > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Attachments: YARN-3623-2015-11-19.1.patch, YARN-3623-2015-12-09.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15049383#comment-15049383 ] Sangjin Lee commented on YARN-3623: --- I'm +1. Thanks. > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Attachments: YARN-3623-2015-11-19.1.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[
https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15049478#comment-15049478
]
Hadoop QA commented on YARN-3623:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
33s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 20s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 20s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 3s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
46s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 10s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 25s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 3s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 18s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 18s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 29s
{color} | {color:red} Patch generated 1 new checkstyle issues in
hadoop-yarn-project/hadoop-yarn (total was 214, now 214). {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 11s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 24s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 24s
{color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.8.0_66.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 5s
{color} | {color:green} hadoop-yarn-common in the patch passed with JDK
v1.8.0_66. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 26s
{color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.7.0_91.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 16s
{color} | {color:green} hadoop-yarn-common in the patch passed with JDK
v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 44m 28s {color}
| {color:black}
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[
https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15049005#comment-15049005
]
Sangjin Lee commented on YARN-3623:
---
Thanks for your comment [~djp]. Per comments above, let's move the v.1-v.2
compatibility discussion to YARN-3196. I just want to clarify my comments to
the extent it is relevant to this JIRA as I think they might have been
misunderstood.
{quote}
I would question on this. If "yarn.timeline-service.version" is 2 (after
cluster is being upgrade), and we don't serve 1/1.5 ATS service any more, how
can existing running applications survival for timeline services? Unless we
have a clear answer in v2 that we will continue to maintain a ATS v1/v1.5
service as a legacy daemon in v2 (I don't prefer this way), I don't think we
should mark this config to indicate an unique version of ATS service running in
the server side.
{quote}
When I said the cluster should bring up that exact version of the timeline
service, I didn't mean that we will not support any compatibility. I definitely
agree that the compatibility and support for a smooth rolling upgrade should be
an objective, and that's why we want to continue the discussion and work on
YARN-3196. What I meant to do is to separate the compatibility support (or
rolling upgrade support) from the main interpretation of this config on the
cluster.
It is true that the main mode of operation will be on the version that is
declared via timeline-service.version. Also, I think there are many options in
the way we can implement the rolling upgrade support. Supporting rolling
upgrade does not necessarily mean that the v.1 write/read endpoints must be up
in parallel with the v.2 write/read endpoints. We talked about having some kind
of a temporary proxy or something in the timeline client itself. There may be
other ways, but we're not mandating that the old endpoints must be up to
implement the rolling upgrade support. My point was that when we say
timeline-service.version = 2 doesn't *automatically* mean we still must bring
up end points of the previous version (or versions), as that's more of an
implementation choice for how to support rolling upgrade. I hope that clarifies
my earlier comments.
{quote}
This works if we don't consider rolling upgrade case. For roll up cases, an
running application/framework cannot switch its client version config if YARN
cluster is upgrading to a new version ATS. We shouldn't claim that
application's clients is expected to be no response if version is mis-match
with serve or the user would misunderstand they have to kill these applications
after upgrade. Instead, we should claim that client is not supposed to override
this config that vary with cluster config unless they are pretty sure what
cluster side are doing (like upgrading process, etc.).
{quote}
Again, I hope it is clear what I meant was NOT that we will not consider the
rolling upgrade use case. Even if the cluster is running with version = 2, with
a proper rolling upgrade support, it should be prepared to handle (during the
transition) calls that are coming in from running apps with version = 1 or 1.5.
That's why I said "depending on how robust the compatibility story is".
Let me know if this helps in any way.
> We should have a config to indicate the Timeline Service version
>
>
> Key: YARN-3623
> URL: https://issues.apache.org/jira/browse/YARN-3623
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
>Reporter: Zhijie Shen
>Assignee: Xuan Gong
> Attachments: YARN-3623-2015-11-19.1.patch
>
>
> So far RM, MR AM, DA AM added/changed new config to enable the feature to
> write the timeline data to v2 server. It's good to have a YARN
> timeline-service.version config like timeline-service.enable to indicate the
> version of the running timeline service with the given YARN cluster. It's
> beneficial for users to more smoothly move from v1 to v2, as they don't need
> to change the existing config, but switch this config from v1 to v2. And each
> framework doesn't need to have their own v1/v2 config.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047084#comment-15047084 ] Junping Du commented on YARN-3623: -- I don't quite familiar with requirement of ATS v1.5. However, in stands of ATS v2, I would agree with [~sjlee0]'s comments above to make this effect works on writer side only (TimelineClient). More clarifications: 1. This version configuration is to benefit application/framework to have flexibility to run on top of YARN cluster with ATS v1 or v2 running with indicating the latest stable version ATS service that the cluster can support. ATS v1 and v2 client are different binary bits and use different incompatible APIs to put information like event, metrics, etc. so far. With getting proper configuration from YARN, the application can aware the ATS service version when landing on YARN cluster and can choose different TimelineClient to push info and get rid of our pains in doing TestDistributedCache for v1/v2 timeline service. 2. We shouldn't break rolling upgrade scenario, or it could be seen as incompatible feature which cannot land on 2.x branch. That also means, we should support ATS v1 and v2 services at the same time during cluster upgrade so legacy/existing applications can still access their old ATS service which is the same as many rollup stories. 2 clarification is more related to this change: we'd better change "yarn.timeline-service.version" to "yarn.timeline-service.latest.version" and use "indicate to clients what is the latest stable version of the running timeline service" to get rid of any confusion here. Also it is better to explicitly mention that our support range for ATS is: [X-1, X] for rolling upgrade (assume X is latest stable ATS version). > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Attachments: YARN-3623-2015-11-19.1.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047758#comment-15047758 ] Li Lu commented on YARN-3623: - bq. it may be sufficient to note it here and carry on that discussion on a v.2 subtask. Sound good? Agree. We can further investigate this issue in the v2 branch. I'm also fine with the current config name in [~xgong]'s patch. > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Attachments: YARN-3623-2015-11-19.1.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047695#comment-15047695 ] Sangjin Lee commented on YARN-3623: --- I agree the rolling upgrade use case from v.1 to v.2 should be addressed. We had some offline discussion on this too. Since it is a pretty major item in and of itself and somewhat separate (being v.2-specific) from this specific JIRA, it may be sufficient to note it here and carry on that discussion on a v.2 subtask. Sound good? I'm fine with the current name "yarn.timeline-service.version". I just want to clarify the interpretation of this config on the cluster side and on the client side. On the cluster side, it *should* always be interpreted as precisely which version of the timeline service should be up. If "yarn.timeline-service.version" is 1.5, and "yarn.timeline-service.enabled" is true, it should be understood as the cluster should bring up the timeline service v.1.5 (and nothing else), and the client can expect that to be the case. On the client side, clearly a client that uses the same version should expect to succeed. If a client chooses to use a smaller version in spite of this, then depending on how robust the compatibility story is between versions, the results may vary (part of the rolling upgrade discussion included). > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Xuan Gong > Attachments: YARN-3623-2015-11-19.1.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[
https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15045609#comment-15045609
]
Sangjin Lee commented on YARN-3623:
---
{quote}
4. Can raise new jira and handle REST interface support to get the supported
ATS version and config from the server for ATS 1.5
5. Can raise new jira and handle version checking for the ATSv2 interface
methods in TimelineClient.and also fetching of version
{quote}
In my mind, trying to get the version by querying the REST interface seems
highly problematic. If we're going to have a configuration that clearly spells
out which version the cluster is running, why not rely on that config? Is there
a reason to believe that may not be correct?
In terms of issues of querying the REST interface for the version, first, the
write URLs are not really feasible. V.1.5 may not even have the write REST
endpoint up. For v.2, you really need to first discover the right URL (i.e.
server) to talk to. You do it anyway, but that's only to know which endpoint to
talk to, not to discover which version it needs to use.
Even if we were to use the read REST endpoint, it would mean that the timeline
client code would need to contact the list of all known version URLs one at a
time to see which read endpoint is up and running. It sounds to me that it can
be unnecessarily flaky and harder than it needs to be.
I think it makes sense to rely on the version config, and not worry about
querying it live. I don't see added value, and config will only be simpler and
more robust. Have I missed any important point?
> We should have a config to indicate the Timeline Service version
>
>
> Key: YARN-3623
> URL: https://issues.apache.org/jira/browse/YARN-3623
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
>Reporter: Zhijie Shen
>Assignee: Xuan Gong
> Attachments: YARN-3623-2015-11-19.1.patch
>
>
> So far RM, MR AM, DA AM added/changed new config to enable the feature to
> write the timeline data to v2 server. It's good to have a YARN
> timeline-service.version config like timeline-service.enable to indicate the
> version of the running timeline service with the given YARN cluster. It's
> beneficial for users to more smoothly move from v1 to v2, as they don't need
> to change the existing config, but switch this config from v1 to v2. And each
> framework doesn't need to have their own v1/v2 config.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15036654#comment-15036654 ] Li Lu commented on YARN-3623: - Hi [~Naganarasimha] sorry for the late reply (just came back from a vacation). Plan sounds good to me. One thing is that we may not need to mark this JIRA as ATS v1.5 since the fix here is a rather general one: from v1.5 on we need to handle this configuration in ATS correctly, thus I think it'll be a general JIRA and not attached with any specific version of ATS. Right now we have the patch for this JIRA so we can proceed with the rest of the plan? It will certainly be helpful if anyone has any concerns on Naga's plan. > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Naganarasimha G R > Attachments: YARN-3623-2015-11-19.1.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15023819#comment-15023819 ] Naganarasimha G R commented on YARN-3623: - Hi [~vinodkv], [~sjlee0], [~gtCarrera] & [~xgong], If the above solution is fine then we can have following steps # make this jira as subjira of 1.5 and introduce the timeline version # As part of YARN-4183, we can create *"yarn.timeline-service.client.require-delegation-token"* so that it fixes depency on *"yarn.timeline-service.enabled"* in the client side to get tokens # YARN-4356 or a *new jira* can handle modifications and updates of timeline version in ATSv2 # Can raise new jira and handle REST interface support to get the supported ATS version and config from the server for ATS 1.5 # Can raise new jira and handle version checking for the ATSv2 interface methods in TimelineClient.and also fetching of version > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Naganarasimha G R > Attachments: YARN-3623-2015-11-19.1.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15018486#comment-15018486 ] Naganarasimha G R commented on YARN-3623: - This would be good [~vinodkv], but was under the impression that timeline service version is not required for versions before ATS 1.5 and this would be introduced in YARN-4234 (as part of ATS 1.5) and the required modifications where ever applicable in ATSv2 would be handled in this jira (earlier ATS subjira). Ok anyway can create another subjira in 2928 to handle usage of this newly introduced version. > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Naganarasimha G R > Attachments: YARN-3623-2015-11-19.1.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[
https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15018657#comment-15018657
]
Naganarasimha G R commented on YARN-3623:
-
Hi [~vinodkv],
bq. There should be an explicit yarn.timeline-service.version which tells
YarnClient to get tokens or not - yes for non-present version (default), v1, v2
but no for v1.5.
IIUC this contradicts the [~sjlee0]'s proposal which was discussed in YARN-4183
and the approach which i had discussed in YARN-4234 with [~gtCarrera],
I can envisage it as follows (including the opinions of [~sjlee0]):
* timeline service will be configured with {{"yarn.timeline-service.enabled"}}
and {{"TIMELINE_SERVICE_VERSION"}} to be true based on which appropriate
timeline handler is selected
* RM/NM (SMP) will be configured with {{"yarn.timeline-service.enabled"}} and
{{yarn.resourcemanager.system-metrics-publisher.enabled}} to be set to true and
along with it {{"TIMELINE_SERVICE_VERSION"}} based on which appropriate client
will be selected.
* Client apps who ever want to communicate with Timeline server will enable
{{"yarn.timeline-service.enabled"}} and a new config like
{{yarn.timeline-service.client.require-delegation-token}} to be set to true to
publish the ATS events.
* If {{security , "yarn.timeline-service.enabled" and
"yarn.timeline-service.client.require-delegation-token"}} are enabled then
delegation token is automatically fetched in the yarn client as earlier
* when the timeline client is initialized it {{contacts server for the version
and related configs}} through REST and once it receives it initializes itself.
If user tries to use invalid methods (not appropriate to the server timeline
version) then timelineclient throws exception.
Thus this will avoid improper configurations between client and server and also
allows client to have flexible configurations from server to avoid getting
delegation tokens from ATS based on server config.
A note : In the above proposal have not considered the multiple versions which
is still under discussion in YARN-4368
> We should have a config to indicate the Timeline Service version
>
>
> Key: YARN-3623
> URL: https://issues.apache.org/jira/browse/YARN-3623
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
>Reporter: Zhijie Shen
>Assignee: Naganarasimha G R
> Attachments: YARN-3623-2015-11-19.1.patch
>
>
> So far RM, MR AM, DA AM added/changed new config to enable the feature to
> write the timeline data to v2 server. It's good to have a YARN
> timeline-service.version config like timeline-service.enable to indicate the
> version of the running timeline service with the given YARN cluster. It's
> beneficial for users to more smoothly move from v1 to v2, as they don't need
> to change the existing config, but switch this config from v1 to v2. And each
> framework doesn't need to have their own v1/v2 config.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[
https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014948#comment-15014948
]
Hadoop QA commented on YARN-3623:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} docker + precommit patch detected. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
44s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 42s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 34s
{color} | {color:green} trunk passed with JDK v1.7.0_85 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 8s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 30s
{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common in
trunk has 3 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 18s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 37s
{color} | {color:green} trunk passed with JDK v1.7.0_85 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 20s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 29s
{color} | {color:green} the patch passed with JDK v1.7.0_85 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 29s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 34s
{color} | {color:red} Patch generated 1 new checkstyle issues in
hadoop-yarn-project/hadoop-yarn (total was 212, now 212). {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
33s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 26s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 49s
{color} | {color:green} the patch passed with JDK v1.7.0_85 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 31s
{color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.8.0_66.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 21s {color}
| {color:red} hadoop-yarn-common in the patch failed with JDK v1.8.0_66.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 29s
{color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.7.0_85.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 26s {color}
| {color:red} hadoop-yarn-common in the patch failed with JDK v1.7.0_85.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} |
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014661#comment-15014661 ] Vinod Kumar Vavilapalli commented on YARN-3623: --- Copying comments from YARN-4183 w.r.t versioning: By Me - There should be an explicit yarn.timeline-service.version which tells YarnClient to get tokens or not - yes for non-present version (default), v1, v2 but no for v1.5. - We should also use the same property in the new API calls proposed for V1.5 YARN-4233 / V2 YARN-2928, lest the users think they can call any API independent of what is supported on server side. - The version field has semantics on both client and server side at the same time - it's picking a solution end-to-end. By [~sjlee0] - The version is a list or enum - https://issues.apache.org/jira/browse/YARN-4183?focusedCommentId=15004954=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15004954. Related JIRA: YARN-4368. > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Naganarasimha G R > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014756#comment-15014756 ] Xuan Gong commented on YARN-3623: - Attached a simple patch which added a new config to indicate the Timeline Service version. > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Naganarasimha G R > Attachments: YARN-3623-2015-11-19.1.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3623) We should have a config to indicate the Timeline Service version
[ https://issues.apache.org/jira/browse/YARN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15015061#comment-15015061 ] Li Lu commented on YARN-3623: - Patch LGTM. The findbugs warnings appears to be irrelevant to the fix here. > We should have a config to indicate the Timeline Service version > > > Key: YARN-3623 > URL: https://issues.apache.org/jira/browse/YARN-3623 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Naganarasimha G R > Attachments: YARN-3623-2015-11-19.1.patch > > > So far RM, MR AM, DA AM added/changed new config to enable the feature to > write the timeline data to v2 server. It's good to have a YARN > timeline-service.version config like timeline-service.enable to indicate the > version of the running timeline service with the given YARN cluster. It's > beneficial for users to more smoothly move from v1 to v2, as they don't need > to change the existing config, but switch this config from v1 to v2. And each > framework doesn't need to have their own v1/v2 config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
