[jira] [Commented] (YARN-2838) Issues with TimeLineServer (Application History)
[ https://issues.apache.org/jira/browse/YARN-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14204789#comment-14204789 ] Naganarasimha G R commented on YARN-2838: - Missed to add point 4 : In YARNClientIMPL;history data can be either got from HistoryServer (old manager) or from TimeLineServer (new) So historyServiceEnabled flag needs to check for both Timeline server configurations and ApplicationHistoryServer configurations, as data can be got from either of them. > Issues with TimeLineServer (Application History) > > > Key: YARN-2838 > URL: https://issues.apache.org/jira/browse/YARN-2838 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Affects Versions: 2.5.1 >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: IssuesInTimelineServer.pdf > > > Few issues in usage of Timeline server for generic application history access -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2838) Issues with TimeLineServer (Application History)
[ https://issues.apache.org/jira/browse/YARN-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14206634#comment-14206634 ] Naganarasimha G R commented on YARN-2838: - Hi [~zjshen], Can you please feedback on these issues ? As some issues requires discussion before rectifiction... > Issues with TimeLineServer (Application History) > > > Key: YARN-2838 > URL: https://issues.apache.org/jira/browse/YARN-2838 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Affects Versions: 2.5.1 >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: IssuesInTimelineServer.pdf > > > Few issues in usage of Timeline server for generic application history access -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2838) Issues with TimeLineServer (Application History)
[ https://issues.apache.org/jira/browse/YARN-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14206671#comment-14206671 ] Zhijie Shen commented on YARN-2838: --- [~Naganarasimha], sorry for not responding you immediately as being busy on finalizing 2.6. A quick scan through your issue document. Here's my clarification: 1. While the entry point of the this sub-module is still called ApplicationHistoryServer, it is actually generalized to be TimelineServer right now (definitely we need to refactor the code at some point). The baseline service provided the the timeline server is to allow the cluster and its apps to store their history information, metrics and so on by complying with the defined timeline data model. Later on, users and admins can query this information to do the analysis. 2. Application history (or we prefer to call it generic history service) is now a built-in service in the timeline server to record the generic history information of YARN apps. It was on a separate store (on FS), but after YARN-2033, it has been moved to the timeline store too, as a payload. We replace the old storage layer, but keep the existing interfaces (web UI, services, CLI) not changed to be the analog of what RM provides for running apps. We still didn't integrate TimelineClient and AHSClient, the latter of which is RPC interface of getting generic history information via RPC interface. APPLICATION_HISTORY_ENABLED is the only remaining old config to control whether we also want to pull the app info from the generic history service inside the timeline server. You may want to take a look at YARN-2033 to get more context about the change. Moreover, as a number of limitation of the old history store, we're no longer going to support it. 3. The document is definitely staled. I'll file separate document Jira, however, it's too late for 2.6. Let's target 2.7 for an up-to-date document about timeline service and its built-in generic history service. Does it sound good? > Issues with TimeLineServer (Application History) > > > Key: YARN-2838 > URL: https://issues.apache.org/jira/browse/YARN-2838 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Affects Versions: 2.5.1 >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: IssuesInTimelineServer.pdf > > > Few issues in usage of Timeline server for generic application history access -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2838) Issues with TimeLineServer (Application History)
[
https://issues.apache.org/jira/browse/YARN-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14207562#comment-14207562
]
Naganarasimha G R commented on YARN-2838:
-
Hi [~zjshen]
I will go through it (YARN-2033), but felt like some issues still stand
valid even if plan to continue as timeline server itself.
{quote}
# Whatever the CLI command user executes is historyserver or timelineserver it
looks
like ApplicationHistoryServer only run. So can we modify the name of the class
ApplicationHistoryServer to TimelineHistoryServer (or any other suitable name
as
it seems like any command user runs ApplicationHistoryServer is started)
# Instead of the "Starting the History Server anyway..." deprecated msg, can we
have
"Starting the Timeline History Server anyway...".
# Based on start or stop, deprecated message should get modified to "Starting
the
Timeline History Server anyway..." or "Stopping the Timeline History Server
anyway..."
{quote}
So if you comment on the individual issues/points would like to start fixing as
part of this jira
There is also a 4th issue which i mentioned
{quote}
Missed to add point 4 : In YARNClientIMPL;history data can be either got from
HistoryServer (old manager) or from TimeLineServer (new)
So historyServiceEnabled flag needs to check for both Timeline server
configurations and ApplicationHistoryServer configurations, as data can be got
from either of them.
{quote}
I think this is also related to the issue which you mentioned ??We still didn't
integrate TimelineClient and AHSClient, the latter of which is RPC interface of
getting generic history information via RPC interface.??. But any way we need
to fix this issue also right ? so already any jira is raised or shall i work as
part of this jira ?
And also please inform if this issue needs to be split into mulitple jiras
(apart from documentation which you have already raised) would like to split
and work on them.
As already i have started looking into these issues, was also planning to work
on documentation. If you don't mind can you assign the issue (YARN-2854) to me ?
> Issues with TimeLineServer (Application History)
>
>
> Key: YARN-2838
> URL: https://issues.apache.org/jira/browse/YARN-2838
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
>Affects Versions: 2.5.1
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: IssuesInTimelineServer.pdf
>
>
> Few issues in usage of Timeline server for generic application history access
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2838) Issues with TimeLineServer (Application History)
[ https://issues.apache.org/jira/browse/YARN-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14210199#comment-14210199 ] Zhijie Shen commented on YARN-2838: --- bq. 1. Whatever the CLI command user executes is historyserver or timelineserver it looks like ApplicationHistoryServer only run. So can we modify the name of the class ApplicationHistoryServer to TimelineHistoryServer (or any other suitable name as it seems like any command user runs ApplicationHistoryServer is started) Yes, not just the the main entry point class, but the whole sub-module needs to be refactor somehow to reflect the generalized conception (YARN-2043). bq. 2. Instead of the "Starting the History Server anyway..." deprecated msg, can we have "Starting the Timeline History Server anyway...". bq. 3. Based on start or stop, deprecated message should get modified to "Starting the Timeline History Server anyway..." or "Stopping the Timeline History Server anyway..." See the comment before. bq. But any way we need to fix this issue also right ? so already any jira is raised or shall i work as part of this jira ? See YARN-2522. We can work this issue there. bq. And also please inform if this issue needs to be split into mulitple jiras (apart from documentation which you have already raised) would like to split and work on them. If you agree, we can close this Jira, and work on separate Jiras that focus on each individual issues. bq. As already i have started looking into these issues, was also planning to work on documentation. If you don't mind can you assign the issue (YARN-2854) to me ? No problem, assigned it to you. > Issues with TimeLineServer (Application History) > > > Key: YARN-2838 > URL: https://issues.apache.org/jira/browse/YARN-2838 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Affects Versions: 2.5.1 >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: IssuesInTimelineServer.pdf > > > Few issues in usage of Timeline server for generic application history access -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2838) Issues with TimeLineServer (Application History)
[ https://issues.apache.org/jira/browse/YARN-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14214899#comment-14214899 ] Naganarasimha G R commented on YARN-2838: - Thanks [~zjshen] for reviewing and guiding under which issues each can be handled . Issue1 will be handled in [ YARN-2043 ] Issue2 will be handled in [ YARN-2033 ] Issue3 will be handled in [ YARN-2854 ] Issue4 will be handled in [ YARN-2522 ] Will start working on the assigned issues and update you at the earliest Hence we can resolve this issue as closed ... > Issues with TimeLineServer (Application History) > > > Key: YARN-2838 > URL: https://issues.apache.org/jira/browse/YARN-2838 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Affects Versions: 2.6.0, 2.5.1 >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: IssuesInTimelineServer.pdf > > > Few issues in usage of Timeline server for generic application history access -- This message was sent by Atlassian JIRA (v6.3.4#6332)
