[
https://issues.apache.org/jira/browse/CAMEL-27179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-27179 started by Claus Ibsen.
-------------------------------------------
> camel-aws2-timestream - query has no next token header, results cannot be
> paged
> -------------------------------------------------------------------------------
>
> Key: CAMEL-27179
> URL: https://issues.apache.org/jira/browse/CAMEL-27179
> Project: Camel
> Issue Type: Improvement
> Components: camel-aws2
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Minor
> Fix For: 4.24.0
>
>
> h3. Problem
> The {{query}} operation of {{aws2-timestream}} (without {{pojoRequest}})
> builds the {{QueryRequest}} from two headers only:
> {{CamelAwsTimestreamQueryString}} and {{CamelAwsTimestreamQueryMaxRows}}.
> Timestream returns a result in pages, with a {{nextToken}} for the next page,
> but there is no header to send a next token, and none is set from the
> response. So only the first page of a result can be read. Setting
> {{CamelAwsTimestreamQueryMaxRows}} limits the page size, it does not give
> paging. The endpoint documentation already says that "the next page needs a
> QueryRequest with its nextToken and pojoRequest=true", that is Java code.
> The Kamelet {{aws-timestream-query-sink}} cannot offer paging because of this.
> h3. Where
> {{components/camel-aws/camel-aws2-timestream/src/main/java/org/apache/camel/component/aws2/timestream/query/Timestream2QueryProducer.java}},
> {{query(...)}}: the {{else}} branch reads only {{QUERY_STRING}} and
> {{QUERY_MAX_ROWS}}, and the body is set to the {{QueryResponse}} without any
> header from it.
> h3. Suggested change
> One or both of:
> * A header {{CamelAwsTimestreamQueryNextToken}}: read it into
> {{QueryRequest.Builder.nextToken(...)}}, and set it on the result from
> {{QueryResponse.nextToken()}} (absent on the last page), so that a route can
> loop over the pages, e.g. with {{loop doWhile}}.
> * An endpoint option (e.g. {{queryAllPages}}, default {{false}}) that follows
> the pages with {{timestreamQueryClient.queryPaginator(...)}} and gives all
> the rows in one result.
> Document the new header and option in {{Timestream2Constants}} and the
> component docs.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)