[jira] [Comment Edited] (YARN-7215) REST API to list all deployed services by the same user
[
https://issues.apache.org/jira/browse/YARN-7215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16172740#comment-16172740
]
Jian He edited comment on YARN-7215 at 9/20/17 4:48 AM:
To clarify a bit more, the new YARN UI service tab already list the services,
it does this by passing a yarn-service type filter to the RMWebServices. I
meant to implement similar thing in CLI to list services. Well, user can
already list the services today by running "yarn application -list -appTypes
yarn-service" . I think make it more explicit with a "yarn service list"
command would be more convenient to the user. Similarly, for REST API, we can
have an explicit endpoint to list services
was (Author: jianhe):
To clarify a bit more, the new YARN UI service tab already list the services,
it does this by passing a yarn-service type filter to the RMWebServices. I
meant to implement similar thing in CLI to list services. Well, user can still
get same result by passing a "yarn-service" filter using "yarn application
-list" command today. I think make it more explicit with a "yarn service list"
command would be more convenient to the user. Similarly, for REST API, we can
have an explicit endpoint to list services
> REST API to list all deployed services by the same user
> ---
>
> Key: YARN-7215
> URL: https://issues.apache.org/jira/browse/YARN-7215
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: api, applications
>Reporter: Eric Yang
>Assignee: Eric Yang
>
> In Slider, it is possible to list deployed applications from the same user by
> using:
> {code}
> slider list
> {code}
> This API can help UI to display application and services deployed by the same
> user.
> Apiserver does not have ability to list all applications/services at this
> time. This API requires fast response to list all applications because it is
> a common UI operation. ApiServer deployed applications persist configuration
> in HDFS similar to slider, but using directory listing to display deployed
> application might cost too much overhead to namenode. We may want to use
> alternative storage mechanism to cache deployed application configuration to
> accelerate the response time of list deployed applications.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7215) REST API to list all deployed services by the same user
[
https://issues.apache.org/jira/browse/YARN-7215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16172740#comment-16172740
]
Jian He edited comment on YARN-7215 at 9/20/17 4:46 AM:
To clarify a bit more, the new YARN UI service tab already list the services,
it does this by passing a yarn-service type filter to the RMWebServices. I
meant to implement similar thing in CLI to list services. Well, user can still
get same result by passing a "yarn-service" filter using "yarn application
-list" command today. I think make it more explicit with a "yarn service list"
command would be more convenient to the user. Similarly, for REST API, we can
have an explicit endpoint to list services
was (Author: jianhe):
To clarify a bit more, the new YARN UI service tab already list the services,
it does this by passing a yarn-service type filter to the RM. I meant to
implement similar thing in CLI to list services. Well, user can still get same
result by passing a "yarn-service" filter using "yarn application -list"
command today. I think make it more explicit with a "yarn service list" command
would be more convenient to the user.
> REST API to list all deployed services by the same user
> ---
>
> Key: YARN-7215
> URL: https://issues.apache.org/jira/browse/YARN-7215
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: api, applications
>Reporter: Eric Yang
>Assignee: Eric Yang
>
> In Slider, it is possible to list deployed applications from the same user by
> using:
> {code}
> slider list
> {code}
> This API can help UI to display application and services deployed by the same
> user.
> Apiserver does not have ability to list all applications/services at this
> time. This API requires fast response to list all applications because it is
> a common UI operation. ApiServer deployed applications persist configuration
> in HDFS similar to slider, but using directory listing to display deployed
> application might cost too much overhead to namenode. We may want to use
> alternative storage mechanism to cache deployed application configuration to
> accelerate the response time of list deployed applications.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7215) REST API to list all deployed services by the same user
[
https://issues.apache.org/jira/browse/YARN-7215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16172434#comment-16172434
]
Eric Yang edited comment on YARN-7215 at 9/19/17 10:19 PM:
---
[~jianhe] How does RM handle a service that is in stopped state? Stopped
slider application does not have any record in resource manager. Same slider
application can have multiple Application ID when the application has been
restarted. Slider uses HDFS file to persist the paused application, but having
resource manager to crawl through lists of HDFS directories to find stopped
service seems like potential load attack to namenode. It would be better to
have the operational record index, and cached by well known mechanism like a
SOLR collection. This also reduces having to brew another random read/write,
low latency, index, cache mechanism in YARN. Both HBase and SOLR have solved
random read/write on top of HDFS with some success. It would be better to use
existing libraries that have been baked for several years than inventing
something new for specialized purpose.
was (Author: eyang):
[~jianhe] How does RM handle a service that is in stopped state? Stopped
slider application does not have any record in resource manager. Same slider
application can have multiple Application ID when the application has been
restarted. Slider uses HDFS file to persist the paused application, but having
resource manager to crawl through lists of HDFS directories to find stopped
service seems like potential load attack to namenode. It would be better to
have the operational record index, and cached by well known mechanism like a
SOLR collection. This also reduces having to brew another random read/write,
low latency, index, cache mechanism in YARN. Both HBase and SOLR have solved
random read/write on top of HDFS with some success. It would be better to we
use existing libraries that have been baked for several years than inventing
something new for specialized purpose.
> REST API to list all deployed services by the same user
> ---
>
> Key: YARN-7215
> URL: https://issues.apache.org/jira/browse/YARN-7215
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: api, applications
>Reporter: Eric Yang
>Assignee: Eric Yang
>
> In Slider, it is possible to list deployed applications from the same user by
> using:
> {code}
> slider list
> {code}
> This API can help UI to display application and services deployed by the same
> user.
> Apiserver does not have ability to list all applications/services at this
> time. This API requires fast response to list all applications because it is
> a common UI operation. ApiServer deployed applications persist configuration
> in HDFS similar to slider, but using directory listing to display deployed
> application might cost too much overhead to namenode. We may want to use
> alternative storage mechanism to cache deployed application configuration to
> accelerate the response time of list deployed applications.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
