Re: [PR] KAFKA-7632: Support Compression Level [kafka]
dongjinleekr commented on PR #10826: URL: https://github.com/apache/kafka/pull/10826#issuecomment-2228295007 @junrao @mimaison @clolov @ozturkberkay I greatly appreciate you finalizing this PR! After changing jobs, I could not get involved with the Kafka community since I had to focus on our in-house fork, aka Navercorp (@naver) Kafka. Now, I just restarted KIP-780 and am testing it with our in-house tiered storage plugin. As soon as I complete the work, I will re-open the PR. Thank you again, and stay tuned! :bowing_man: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-7632: Support Compression Level [kafka]
mimaison commented on PR #10826: URL: https://github.com/apache/kafka/pull/10826#issuecomment-2154623171 Closing this PR since https://github.com/apache/kafka/pull/15516 has been merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-7632: Support Compression Level [kafka]
mimaison closed pull request #10826: KAFKA-7632: Support Compression Level URL: https://github.com/apache/kafka/pull/10826 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-7632: Support Compression Level [kafka]
mimaison commented on PR #10826: URL: https://github.com/apache/kafka/pull/10826#issuecomment-2048127235 I opened https://github.com/apache/kafka/pull/15516 to implement this KIP. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-7632: Support Compression Level [kafka]
ijuma commented on code in PR #10826:
URL: https://github.com/apache/kafka/pull/10826#discussion_r850506417
##
clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java:
##
@@ -188,6 +190,12 @@ public class ProducerConfig extends AbstractConfig {
+ " values are
none, gzip, snappy, lz4, or
zstd. "
+ "Compression is of
full batches of data, so the efficacy of batching will also impact the
compression ratio (more batching means better compression).";
+/** compression.level */
+public static final String COMPRESSION_LEVEL_CONFIG = "compression.level";
+private static final String COMPRESSION_LEVEL_DOC = "The compression level
for all data generated by the producer. The default level and valid value is up
to "
++ "compression.type. (none, snappy: not
available. gzip: 1~9. lz4: 1~17. "
Review Comment:
I think the reason why @dongjinleekr didn't include those is that they may
change at the compression library level.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] KAFKA-7632: Support Compression Level [kafka]
mimaison commented on PR #10826: URL: https://github.com/apache/kafka/pull/10826#issuecomment-1931723005 I updated the [KIP](https://cwiki.apache.org/confluence/display/KAFKA/KIP-390%3A+Support+Compression+Level) based on [Jun's suggestion](https://github.com/apache/kafka/pull/10826#issuecomment-1795952612) and sent an update to the VOTE thread. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-7632: Support Compression Level [kafka]
mimaison commented on PR #10826: URL: https://github.com/apache/kafka/pull/10826#issuecomment-1801941525 Yes it would be good to get this over the line. @dongjinleekr Do you think you'll have time to update it? or would you need some help? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-7632: Support Compression Level [kafka]
junrao commented on PR #10826:
URL: https://github.com/apache/kafka/pull/10826#issuecomment-1795952612
One discussion of the PR is around choosing the default value for
`compression.level`
(https://github.com/apache/kafka/pull/10826#discussion_r917108950). This is
tricky since it's specific to the compression type. I am wondering if it's
better to adjust the KIP to support `compression.{gzip|lz4|zstd}.level`
instead. This will make it more aligned with the future KIP
https://cwiki.apache.org/confluence/display/KAFKA/KIP-780%3A+Support+fine-grained+compression+options#KIP780:Supportfinegrainedcompressionoptions-Zstd.1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] KAFKA-7632: Support Compression Level [kafka]
ozturkberkay commented on PR #10826: URL: https://github.com/apache/kafka/pull/10826#issuecomment-1786847820 Hi, I was wondering if this PR would be prioritized in the foreseeable future given there has been a lot of work done already -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
