[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15552144#comment-15552144
]
Varun Saxena edited comment on YARN-5585 at 10/6/16 3:01 PM:
-------------------------------------------------------------
bq. I was thinking to use same REST API for both by using SingleColumnFilter.
One cons I see is table scan for all the entityType i.e reflect in read
performance.
We should not use SingleColumnValueFilter if we know the prefix because as you
said former will lead to a relatively slower read performance. Basically we
need to differentiate between not having a prefix for the entity type and user
unable to supply it.
bq. I would have thought that we store the entities in the reverse entity id
order, but it appears that the entity id is encoded into the row key as is
(EntityRowKey). Am I reading that right? If so, this is a bug to fix.
Entity IDs' can be anything. Even a completely alphabetical sequence can be an
entity ID. So it will not be possible to define a reverse order for every
generic entity ID. Is this your question ?
bq. Firstly about multi JVM which makes application programmer to define new
protocol for transferring prefixId.
Trying to understand this more. Can same DAG be executed by multiple Tez AMs' ?
bq. Secondly, what if users misses providing an prefixId in subsequent
updates.?
This should be caught during integration phase. Right ?
was (Author: varun_saxena):
bq. I was thinking to use same REST API for both by using SingleColumnFilter.
One cons I see is table scan for all the entityType i.e reflect in read
performance.
We should not use SingleColumnValueFilter if we know the prefix because as you
said former will lead to a relatively slower read performance. Basically we
need to differentiate between having a prefix for the entity type and user
unable to supply it.
bq. I would have thought that we store the entities in the reverse entity id
order, but it appears that the entity id is encoded into the row key as is
(EntityRowKey). Am I reading that right? If so, this is a bug to fix.
Entity IDs' can be anything. Even a completely alphabetical sequence can be an
entity ID. So it will not be possible to define a reverse order for every
generic entity ID. Is this your question ?
bq. Firstly about multi JVM which makes application programmer to define new
protocol for transferring prefixId.
Trying to understand this more. Can same DAG be executed by multiple Tez AMs' ?
bq. Secondly, what if users misses providing an prefixId in subsequent
updates.?
This should be caught during integration phase. Right ?
> [Atsv2] Add a new filter fromId in REST endpoints
> -------------------------------------------------
>
> Key: YARN-5585
> URL: https://issues.apache.org/jira/browse/YARN-5585
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelinereader
> Reporter: Rohith Sharma K S
> Assignee: Rohith Sharma K S
> Priority: Critical
> Attachments: 0001-YARN-5585.patch, YARN-5585-workaround.patch,
> YARN-5585.v0.patch
>
>
> TimelineReader REST API's provides lot of filters to retrieve the
> applications. Along with those, it would be good to add new filter i.e fromId
> so that entities can be retrieved after the fromId.
> Current Behavior : Default limit is set to 100. If there are 1000 entities
> then REST call gives first/last 100 entities. How to retrieve next set of 100
> entities i.e 101 to 200 OR 900 to 801?
> Example : If applications are stored database, app-1 app-2 ... app-10.
> *getApps?limit=5* gives app-1 to app-5. But to retrieve next 5 apps, there is
> no way to achieve this.
> So proposal is to have fromId in the filter like
> *getApps?limit=5&&fromId=app-5* which gives list of apps from app-6 to
> app-10.
> Since ATS is targeting large number of entities storage, it is very common
> use case to get next set of entities using fromId rather than querying all
> the entites. This is very useful for pagination in web UI.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
