Boyang Chen created KAFKA-7816:
----------------------------------
Summary: Windowed topic should have window size as part of the
metadata
Key: KAFKA-7816
URL: https://issues.apache.org/jira/browse/KAFKA-7816
Project: Kafka
Issue Type: Improvement
Components: consumer, streams
Reporter: Boyang Chen
Assignee: Boyang Chen
Currently the Kafka window store topics require a windowed serde to properly
deserialize the records. One of the required config is `window.size.ms`, which
indicates the diff between (window.end - window.start). For space efficiency,
KStream only stores the windowed record with window start time, because as long
as the restore consumer knows size of the window, it would properly derive the
window end time by adding window.size.ms to window start time.
However, this makes the reuse of window topic very hard because another user
has to config the correct window size in order to deserialize the data. When we
extract the customized consumer as a template, every time new user has to
define their own window size. If we do wild-card matching consumer, things
could be even worse to work because different topics may have different window
size and user has to read through the application code to find that info.
To make the decoding of window topic easier, we are proposing to add a new
config to TopicMetadata called `windowSize` which could be used for
applications to properly deserialize the data without requirement to config a
window size. This could also make client side serde API easier.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
