Chris,
This (understandably) requires access to the log segment files on disk. Managed
Kafka services are becoming more popular (Confluent Cloud, Amazon MSK) and they
do not expose the log segment files on disk. It’d be great to have an
equivalent functionality that would work on managed servic
Bump. I'd like to gather more feedback on this!
Chris
On 2020/08/17 20:23:51, "Christopher Beard (BLOOMBERG/ 919 3RD A)"
wrote:
> Hi everyone,
>
> I would like to start a discussion on KIP-640:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-640%3A+Add+log+compression+analysis+tool
>
Hi Alex, thanks for the question!
In the simplest sense, the tool doesn't know anything about the messages in the
log or any particular batch. The tool would compress the encrypted data to
measure the resulting size, but the results would likely show no reduction in
data size. Effectively, the
Hi, how will this work with encrypted data in logs if/when KIP-317 gets merged?
Encrypted data will be hard to compress, so the analyzer tool might need to
acquire the decryption key somewhere measure the compression stats.
On 2020/08/17 20:23:51, "Christopher Beard (BLOOMBERG/ 919 3RD A)"
wro
Hi everyone,
I would like to start a discussion on KIP-640:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-640%3A+Add+log+compression+analysis+tool
This KIP outlines a new CLI tool which helps compare how the various
compression types supported by Kafka reduce the size of a log (and there
