[
https://issues.apache.org/jira/browse/YARN-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15010072#comment-15010072
]
Sangjin Lee commented on YARN-4183:
-----------------------------------
bq. May be we can try to fail the RM startup if SMP is not able to connect to
Timelineserver,but IIUC Jonathan Eagles and Jason Lowe in other ATS 1.5 jira
were informing that cluster should run though the timelineserver is not running
hence its not desirable to fail the RM startup.
I'm *not* suggesting that we fail the RM if the SMP cannot connect to the
timeline server. I'm suggesting we can *disable* the SMP (as opposed to having
continuous failures or failing the RM) if we know the timeline service is
disabled. That's why the config helps, as it is the clearly indicated intent
for running the cluster.
As you said, it is possible that yarn.timeline-service.enabled is set to false
even if the timeline service is up and vice versa. But I think we should assume
a reasonable use case here where it is fair to expect the timeline service to
be there if yarn.timeline-service.enabled is true. If that config is true but
the timeline service is not up, then I think it is acceptable to see continuous
timeline service failures (this is the existing behavior btw).
bq. if we still want to go ahead with yarn.timeline-service.enabled then we
might need to come up with a new configuration to indicate that client wants to
use the timeline server hence create the timeline client and the timelineserver
delegation tokens.
Yes, I agree. It's implied that we would need a different config/mechanism for
disabling getting the delegation token. I would advocate having a separate
client-side config. Whether the server has enabled the timeline service and
whether a particular client/app will use it are separate concerns, and separate
configs should drive them. The current state of tez and MR is a good example.
MR should be able to say I won't use the timeline service even if it is
available. Any MR code that uses the timeline service should basically check
both configs; i.e. the timeline service should be enabled and its config to use
the timeline service should be true.
This might be an idealistic argument on my part, but I think doing something
along that line would lead to cleaner separation of concerns and larger degrees
of freedom. My 2 cents.
> Enabling generic application history forces every job to get a timeline
> service delegation token
> ------------------------------------------------------------------------------------------------
>
> Key: YARN-4183
> URL: https://issues.apache.org/jira/browse/YARN-4183
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.7.1
> Reporter: Mit Desai
> Assignee: Mit Desai
> Attachments: YARN-4183.1.patch
>
>
> When enabling just the Generic History Server and not the timeline server,
> the system metrics publisher will not publish the events to the timeline
> store as it checks if the timeline server and system metrics publisher are
> enabled before creating a timeline client.
> To make it work, if the timeline service flag is turned on, it will force
> every yarn application to get a delegation token.
> Instead of checking if timeline service is enabled, we should be checking if
> application history server is enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
