[
https://issues.apache.org/jira/browse/DRILL-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17610340#comment-17610340
]
ASF GitHub Bot commented on DRILL-8320:
---------------------------------------
jnturton commented on code in PR #2660:
URL: https://github.com/apache/drill/pull/2660#discussion_r981908934
##########
contrib/storage-http/src/main/java/org/apache/drill/exec/store/http/HttpBatchReader.java:
##########
@@ -66,7 +67,7 @@ public class HttpBatchReader implements
ManagedReader<SchemaNegotiator> {
private final int maxRecords;
protected final Paginator paginator;
protected String baseUrl;
- private JsonLoader jsonLoader;
+ private JsonLoaderImpl jsonLoader;
Review Comment:
I hadn't noticed that we'd already started depending on JsonLoaderImpl. Is
something preventing working with the JsonLoader interface?
> Prevent Infinite Pagination for Index Paginator
> -----------------------------------------------
>
> Key: DRILL-8320
> URL: https://issues.apache.org/jira/browse/DRILL-8320
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - HTTP
> Affects Versions: 1.20.2
> Reporter: Charles Givre
> Assignee: Charles Givre
> Priority: Critical
> Fix For: 2.0.0
>
>
> In some cases that use keyset/index pagination, if the API does not have a
> boolean column that indicates when to stop, Drill will send requests until
> the API stops returning data. This PR fixes this by making the boolean
> parameter optional.
> If that parameter is not present, if the index result is blank or the same as
> the previous request, pagination will end.
> Note, if the pagination parameters are buried in nested objects, this cannot
> be configured with a dataPath. If the user uses a dataPath, pagination will
> stop at the first page.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
