[
https://issues.apache.org/jira/browse/KAFKA-6560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16366392#comment-16366392
]
ASF GitHub Bot commented on KAFKA-6560:
---------------------------------------
guozhangwang opened a new pull request #4578: KAFKA-6560: Replace range query
with newly added single point query in Windowed Aggregation [WIP]
URL: https://github.com/apache/kafka/pull/4578
*More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.*
*Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.*
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Use single-point queries than range queries for windowed aggregation operators
> ------------------------------------------------------------------------------
>
> Key: KAFKA-6560
> URL: https://issues.apache.org/jira/browse/KAFKA-6560
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Guozhang Wang
> Assignee: Guozhang Wang
> Priority: Critical
> Labels: needs-kip
>
> Today for windowed aggregations in Streams DSL, the underlying implementation
> is leveraging the fetch(key, from, to) API to get all the related windows for
> a single record to update. However, this is a very inefficient operation with
> significant amount of CPU time iterating over window stores. On the other
> hand, since the operator implementation itself have full knowledge of the
> window specs it can actually translate this operation into multiple
> single-point queries with the accurate window start timestamp, which would
> largely reduce the overhead.
> The proposed approach is to add a single fetch API to the WindowedStore and
> use that in the KStreamWindowedAggregate / KStreamWindowedReduce operators.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
