[ https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15770183#comment-15770183 ]
Rohith Sharma K S commented on YARN-5585: ----------------------------------------- There couple of things to discuss more # I would be leaning towards adding fromId in filter only if fromId is single value ;-) More precisely, fromId could be only entityPrefixId. Basically, in our earlier discussion fromId is combination of idPrefix as well as id i.e *fromId=entityIdPrefix:entityId* OR *fromId=entityIdPrefix*. IMO, to achieve pagination for general entities, we do not really require to provide entityId at all if entities are published with entityId. So, I think we can separate fromId as two parts ## *fromIdPrefix* : takes the value of entityIdPrefix which is mandatory when fromId is given. This can also be a independent queryParam to achieve pagination. ## *fromId* : takes the value of entityId. Not necessarily mandatory to achieve pagination. But if fromId is provided then fromIdPrefix is mandatory. Thoughts on this? # {color:red}More importantly{color}, we have missed one point over here. This JIRA is focusing only on general entities pagination. But should also implement pagination for other REST API's. I just skimmed into flowActivityTable, flowRunTable and ApplicationTable. ## Query Flows : Difficult to achieve in existing table schema. ## Query Flow Runs: {color:green}Can be achieved with existing table schema{color}. *fromId=flowRunId* ## Query Apps for a Flow : {color:green}Can be achieved with existing table schema{color}. *fromId=appId* ## Query Apps for a Flow Run : {color:green}Can be achieved with existing table schema{color} *formId=appId* > [Atsv2] Reader side changes for entity prefix and support for pagination via > additional filters > ----------------------------------------------------------------------------------------------- > > Key: YARN-5585 > URL: https://issues.apache.org/jira/browse/YARN-5585 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelinereader > Reporter: Rohith Sharma K S > Assignee: Rohith Sharma K S > Priority: Critical > Labels: yarn-5355-merge-blocker > Attachments: 0001-YARN-5585.patch, YARN-5585-YARN-5355.0001.patch, > YARN-5585-YARN-5355.0002.patch, YARN-5585-YARN-5355.0003.patch, > YARN-5585-workaround.patch, YARN-5585.v0.patch > > > TimelineReader REST API's provides lot of filters to retrieve the > applications. Along with those, it would be good to add new filter i.e fromId > so that entities can be retrieved after the fromId. > Current Behavior : Default limit is set to 100. If there are 1000 entities > then REST call gives first/last 100 entities. How to retrieve next set of 100 > entities i.e 101 to 200 OR 900 to 801? > Example : If applications are stored database, app-1 app-2 ... app-10. > *getApps?limit=5* gives app-1 to app-5. But to retrieve next 5 apps, there is > no way to achieve this. > So proposal is to have fromId in the filter like > *getApps?limit=5&&fromId=app-5* which gives list of apps from app-6 to > app-10. > Since ATS is targeting large number of entities storage, it is very common > use case to get next set of entities using fromId rather than querying all > the entites. This is very useful for pagination in web UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org