[
https://issues.apache.org/jira/browse/YARN-7215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16172434#comment-16172434
]
Eric Yang edited comment on YARN-7215 at 9/19/17 10:19 PM:
-----------------------------------------------------------
[~jianhe] How does RM handle a service that is in stopped state? Stopped
slider application does not have any record in resource manager. Same slider
application can have multiple Application ID when the application has been
restarted. Slider uses HDFS file to persist the paused application, but having
resource manager to crawl through lists of HDFS directories to find stopped
service seems like potential load attack to namenode. It would be better to
have the operational record index, and cached by well known mechanism like a
SOLR collection. This also reduces having to brew another random read/write,
low latency, index, cache mechanism in YARN. Both HBase and SOLR have solved
random read/write on top of HDFS with some success. It would be better to use
existing libraries that have been baked for several years than inventing
something new for specialized purpose.
was (Author: eyang):
[~jianhe] How does RM handle a service that is in stopped state? Stopped
slider application does not have any record in resource manager. Same slider
application can have multiple Application ID when the application has been
restarted. Slider uses HDFS file to persist the paused application, but having
resource manager to crawl through lists of HDFS directories to find stopped
service seems like potential load attack to namenode. It would be better to
have the operational record index, and cached by well known mechanism like a
SOLR collection. This also reduces having to brew another random read/write,
low latency, index, cache mechanism in YARN. Both HBase and SOLR have solved
random read/write on top of HDFS with some success. It would be better to we
use existing libraries that have been baked for several years than inventing
something new for specialized purpose.
> REST API to list all deployed services by the same user
> -------------------------------------------------------
>
> Key: YARN-7215
> URL: https://issues.apache.org/jira/browse/YARN-7215
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: api, applications
> Reporter: Eric Yang
> Assignee: Eric Yang
>
> In Slider, it is possible to list deployed applications from the same user by
> using:
> {code}
> slider list
> {code}
> This API can help UI to display application and services deployed by the same
> user.
> Apiserver does not have ability to list all applications/services at this
> time. This API requires fast response to list all applications because it is
> a common UI operation. ApiServer deployed applications persist configuration
> in HDFS similar to slider, but using directory listing to display deployed
> application might cost too much overhead to namenode. We may want to use
> alternative storage mechanism to cache deployed application configuration to
> accelerate the response time of list deployed applications.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
