[
https://issues.apache.org/jira/browse/KAFKA-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Bejeck updated KAFKA-5363:
-------------------------------
Description:
Currently, when restoring a state store in a Kafka Streams application, we put
one key-value at a time into the store.
This task aims to make this recovery more efficient by creating a new interface
with "restoreAll" functionality allowing for bulk writes by the underlying
state store implementation.
The proposal will also add "beginRestore" and "endRestore" callback methods
potentially used for
Tracking when the bulk restoration process begins and ends.
Keeping track of the number of records and last offset restored.
KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-167%3A+Add+a+restoreAll+method+to+StateRestoreCallback
was:
Add a new method {{restoreAll(List<KeyValue<byte[], byte[]>> records)}} to the
{{StateRestoreCallback}} to enable bulk writing to the underlying state store
vs individual {{restore(byte[] key, byte[] value)}} resulting in quicker
restore times.
KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-167%3A+Add+a+restoreAll+method+to+StateRestoreCallback
> Add ability to batch restore and receive restoration stats.
> -----------------------------------------------------------
>
> Key: KAFKA-5363
> URL: https://issues.apache.org/jira/browse/KAFKA-5363
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Bill Bejeck
> Assignee: Bill Bejeck
> Labels: kip
> Fix For: 0.11.1.0
>
>
> Currently, when restoring a state store in a Kafka Streams application, we
> put one key-value at a time into the store.
> This task aims to make this recovery more efficient by creating a new
> interface with "restoreAll" functionality allowing for bulk writes by the
> underlying state store implementation.
> The proposal will also add "beginRestore" and "endRestore" callback methods
> potentially used for
> Tracking when the bulk restoration process begins and ends.
> Keeping track of the number of records and last offset restored.
> KIP:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-167%3A+Add+a+restoreAll+method+to+StateRestoreCallback
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
