gaoran10 commented on pull request #7461: URL: https://github.com/apache/pulsar/pull/7461#issuecomment-654883130
1. I think we couldn't add the config `pulsar.read-compact`, this config is a global config. It is up to users to query the raw data or the compacted data, so we could add metadata for users to add in the SQL statement. Such as `select price from pulsar.'public/default'.'stock-topic' where __compacted_topic__ = true` is used to query compacted data. Refer to the `PulsarInternalColumn.java`. 2. The compacted data should be read from two parts, one is the compacted ledger and the other are normal ledgers, like the compacted Topic reading. So we should check the topic compacted position first, and generate splits through the compacted ledger and normal ledgers. One split should have one property such as `read-compacted` indicate this split is used to read compacted ledger or normal topic ledgers. We could use the normal topic name to get its compacted ledger name. @jiazhai @codelipenghui Please confirm. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
