[
https://issues.apache.org/jira/browse/YARN-6105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834435#comment-15834435
]
Varun Saxena edited comment on YARN-6105 at 1/23/17 1:15 PM:
-------------------------------------------------------------
bq. I discussed with cloud team today, constant clusterId or storing it
separately as info does not work for them.
Hmm...Trying to understand the issue better. What issues do they face if they
publish entities with a constant cluster ID in the cloud? Is there any
advantage in differentiating between different ephemeral clusters, and that too
as part of the context(i.e. via YARN tags) ? In a cloud scenario, does the user
even care about the different cluster IDs' each time his job is run? Wouldn't
cluster information be provided by the Cloud Manager or we plan to have Cloud
Manager get this info too from ATS? Agree that each cluster is unique, but if
cluster ID is dynamically generated at runtime as clusters come up and go away,
we need to see how much sense this cluster ID would make for end user.
Anyways, assuming that for cloud scenario, a constant cluster is not to be
given, if we want a list of all cluster IDs', I guess we would need it at real
time. In that case we cannot have a periodic job finding out unique cluster
IDs' and adding them to the new table as suggested above. Correct ?
I am fine with having a /clusterids per say as I mentioned on the other JIRA,
but for cloud use case, wouldn't this list become quite large? And as it
becomes quite large, we need to see how useful it is.
We can probably provide some API to register cluster if the use case for cloud
is strong enough and there is no feasible alternative. Thoughts ?
was (Author: varun_saxena):
bq. I discussed with cloud team today, constant clusterId or storing it
separately as info does not work for them.
Hmm...Trying to understand the issue better. What issues do they face if they
publish entities with a constant cluster ID across the cloud? Is there any
advantage in differentiating between different ephemeral clusters, and that too
as part of the context(i.e. via YARN tags) ? In a cloud scenario, does the user
even care about the different cluster IDs' each time his job is run? Wouldn't
cluster information be provided by the Cloud Manager or we plan to have Cloud
Manager get this info too from ATS ?
Anyways, assuming that for cloud scenario, a constant cluster is not to be
given, if we want a list of all cluster IDs', I guess we would need it at real
time. In that case we cannot have a periodic job finding out unique cluster
IDs' and adding them to the new table as suggested above. Correct ?
I am fine with having a /clusterids per say as I mentioned on the other JIRA,
but for cloud use case, wouldn't this list become quite large? And as it
becomes quite large, we need to see how useful it is.
> Support for new REST end point /clusterids
> ------------------------------------------
>
> Key: YARN-6105
> URL: https://issues.apache.org/jira/browse/YARN-6105
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Rohith Sharma K S
>
> As discussed in YARN-5378 and YARN-6095, it is required to have */clusterids*
> that returns list of clusterids that back end has is useful.
> Use case : In cloud, clusters are arbitrarily spin up and destroyed. Each
> cluster has its own clusterId which UI never knows about it. To all those
> newly spin up cluster, same ATS server has been used. And sam web UI has been
> used. Admin can select the clusterId and navigate to any pages. So, it is
> worth to list ClusterId's from ATS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
