----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73292/#review222954 -----------------------------------------------------------
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java Lines 466 (patched) <https://reviews.apache.org/r/73292/#comment312123> getNextMarker is computed 2 times. Consider local variable. repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java Lines 316 (patched) <https://reviews.apache.org/r/73292/#comment312121> Use camel case. repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java Lines 870 (patched) <https://reviews.apache.org/r/73292/#comment312122> This could be refactored into asertSearchProcessorWithMarker and asertSearchProcessorWithoutMarker. - Ashutosh Mestry On May 7, 2021, 12:40 p.m., Pinal Shah wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73292/ > ----------------------------------------------------------- > > (Updated May 7, 2021, 12:40 p.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, > and Sarath Subramanian. > > > Bugs: ATLAS-4254 > https://issues.apache.org/jira/browse/ATLAS-4254 > > > Repository: atlas > > > Description > ------- > > **Current Working of Pagination** > > - When user specifies page information i.e offset and limit in the request, > except some condition, offset is reset to 0 > - Query is then executed from 0 offset and limit > - Filter out the results obtained from query > - Generate result set by skipping the results from 0 to the given offset > > **New Working Structure** > > - New Parameter 'marker' (String) has been introduced in SearchParameters > which will have encrypted string as value. > - The marker should be set as '*' for the start of the page > - The response of above request (AtlasSearchResult) will have 'nextMarker' > field which will have encrypted String which is to be passed as value of > 'marker' in the subsequent request > - if marker is set then offset should not be greater than 0 > > > **Perfomance details** > > Took usecase of ranger tagsync: > - On searching tagged entities starting with offset: 0 and limit: 1000 > returned 10000 entities in ~ 21s > - On searching tagged entities starting with marker: * and limit: 1000 > returned 10000 entities in ~ 10s > (both the number are taken on the fresh start of the server, so as to avoid > caching affecting the numbers) > - Around 51% improvement > > - Improvement is seen as the page size increases. > > > Diffs > ----- > > intg/src/main/java/org/apache/atlas/model/discovery/AtlasSearchResult.java > e1c550ef7 > intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java > 9d2cd4ffc > > repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java > 647ff9ca0 > > repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java > a3ab6e38c > > repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java > 5dcff3b16 > > repository/src/main/java/org/apache/atlas/discovery/FreeTextSearchProcessor.java > 92152fff5 > > repository/src/main/java/org/apache/atlas/discovery/FullTextSearchProcessor.java > b37d93a83 > repository/src/main/java/org/apache/atlas/discovery/SearchContext.java > aa49121f6 > repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java > f9832c3b2 > > repository/src/main/java/org/apache/atlas/discovery/TermSearchProcessor.java > 45a815886 > > repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java > 027827af2 > > repository/src/test/java/org/apache/atlas/discovery/ClassificationSearchProcessorTest.java > e1ebbfc70 > webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java 4c7b6228d > > > Diff: https://reviews.apache.org/r/73292/diff/3/ > > > Testing > ------- > > MAnual testing > unit testcases > > > Thanks, > > Pinal Shah > >
