[
https://issues.apache.org/jira/browse/KAFKA-8295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17388235#comment-17388235
]
Sagar Rao edited comment on KAFKA-8295 at 7/27/21, 6:15 PM:
------------------------------------------------------------
[~ableegoldman].. wanted to know, if we can have a KIP for merge() operator in
the KV store irrespective of using merge operator from Rocksdb or not(option 1
from above) ?
I will still be doing some benchmarks on it..
That allows users to use the store for counter specific use cases. WDYT?
was (Author: sagarrao):
[~ableegoldman].. wanted to know, if we can have a KIP for merge() operator in
the KV store irrespective of using merge operator from Rocksdb or not(option 1
from above) ? That allows users to use the store for counter specific use
cases. WDYT?
> Optimize count() using RocksDB merge operator
> ---------------------------------------------
>
> Key: KAFKA-8295
> URL: https://issues.apache.org/jira/browse/KAFKA-8295
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: A. Sophie Blee-Goldman
> Assignee: Sagar Rao
> Priority: Major
>
> In addition to regular put/get/delete RocksDB provides a fourth operation,
> merge. This essentially provides an optimized read/update/write path in a
> single operation. One of the built-in (C++) merge operators exposed over the
> Java API is a counter. We should be able to leverage this for a more
> efficient implementation of count()
>
> (Note: Unfortunately it seems unlikely we can use this to optimize general
> aggregations, even if RocksJava allowed for a custom merge operator, unless
> we provide a way for the user to specify and connect a C++ implemented
> aggregator – otherwise we incur too much cost crossing the jni for a net
> performance benefit)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
