[ https://issues.apache.org/jira/browse/KAFKA-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16748373#comment-16748373 ]
Boyang Chen commented on KAFKA-7816: ------------------------------------ Yea, good point here [~guozhang], from the same windowed topic, allowing user to interpret the data in customized way also makes sense. Let me close this Jira in a couple of days if no other opinion is bumped. > Windowed topic should have window size as part of the metadata > -------------------------------------------------------------- > > Key: KAFKA-7816 > URL: https://issues.apache.org/jira/browse/KAFKA-7816 > Project: Kafka > Issue Type: Improvement > Components: consumer, streams > Reporter: Boyang Chen > Assignee: Boyang Chen > Priority: Major > > Currently the Kafka window store topics require a windowed serde to properly > deserialize the records. One of the required config is `window.size.ms`, > which indicates the diff between (window.end - window.start). For space > efficiency, KStream only stores the windowed record with window start time, > because as long as the restore consumer knows size of the window, it would > properly derive the window end time by adding window.size.ms to window start > time. > However, this makes the reuse of window topic very hard because another user > has to config the correct window size in order to deserialize the data. When > we extract the customized consumer as a template, every time new user has to > define their own window size. If we do wild-card matching consumer, things > could be even worse to work because different topics may have different > window size and user has to read through the application code to find that > info. > To make the decoding of window topic easier, we are proposing to add a new > config to TopicMetadata called `windowSize` which could be used for > applications to properly deserialize the data without requirement to config a > window size. This could also make client side serde API easier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)