BewareMyPower edited a comment on issue #9317: URL: https://github.com/apache/pulsar/issues/9317#issuecomment-768882909
> What do you mean by 'compact the topic' after producing 10 messages? you mean you run the pulsar-admin compact command again? Yeah, the command is just a manual compaction for old messages. A manual compaction is just compacting the existed messages but not compact all old messages when a new message arrive, which is inefficient. Topic compaction's purpose is mainly for reducing storage. If you have a consumer that subscribes a topic, even if no compaction was done, the consumer will always read the latest message because each time a message is acknowledged, the consume position will be persisted as a **cursor**. However, for some scenarios like what you said, once a newer message has been read, the old messages are no longer meaningful. We needn't store these messages any more so we can compact the topic. Then these messages are useless for new subscription. See http://pulsar.apache.org/docs/en/concepts-topic-compaction/, There're two ways to do topic compaction: > Triggered automatically when the backlog reaches a certain size or can be triggered manually via the command line. See the Topic compaction cookbook The command, is just triggering compaction manually. I think what you want is doing the compaction for each message arrived. It's unnecessary except you never acknowledge any message and restart consuming or use a new subscription frequently. A suggestion is to use namespace policy or topic level policy for topic compaction. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
