The KS system test did surface a bug. Bruno put a PR up for it, which is
already approved.
We should get it into 3.8 asap to unblock the release from our side.
-Matthias
On 7/8/24 8:51 AM, Josep Prat wrote:
Hi Dongjin,
KIP-390 is part of the 3.8 release because the JIRA associated with it:
https://issues.apache.org/jira/browse/KAFKA-7632 is closed as resolved,
hence the KIP is declared done and ready. I did some digging, and I saw
that Mickael was the one doing the PR that closed the JIRA ticket:
https://github.com/apache/kafka/pull/15516
This means that the KIP work is merged and unfortunately it is now quite
late to perform a rollback for this feature.
@Mickael Maison <[email protected]> let me know if anything I
mentioned is not accurate (as you were the one bringing the KIP to
completion).
Best,
On Mon, Jul 8, 2024 at 5:38 PM Dongjin Lee <[email protected]> wrote:
Hi Josep,
Thanks for managing the 3.8 release. I have a request: could you please
move the KIP-390 into the 3.9 release?
Here is the background: KIP-390 was adopted first but hasn't been released
for a long time. After some time, I proposed KIP-780 with further
improvements and also corrected an obvious design error
(`compression.level` → `compression.(gzip|lz4|zstd). level`), but it hasn't
been adopted due to the community's lack of response, my changing job,
focusing the in-house fork, etc. And last weekend, I found that KIP-380 has
been included in the 3.8 release plan.
- KIP-390:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-390%3A+Support+Compression+Level
- KIP-780:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-780%3A+Support+fine-grained+compression+options
However, shipping those two features at once has the following benefits:
1. Full functionality without design error.
We can provide full functionality, particularly useful with tiered storage
feature at once. I found that several users of tiered storage use
server-side recompression and want to improve the compression efficiency.
Of course, it does not include any design errors :)
2. More chance of testing.
Currently, I am managing an in-house fork of Apache Kafka and Cruise
Control[^1], running on thousands of clusters on k8s. With our ongoing work
on the tiered storage plugin, we can test both KIPs at once. Since we are
planning to move the terabytes of logs from thousands of microservices into
the object storage, some of them can be ideal testbeds.
If you are okay, I will re-initiate the discussion of KIP-780 and rework
KIP-380 on the latest trunk.
Thanks,
Dongjin
[^1]: For example: https://github.com/linkedin/cruise-control/pull/2145
On Mon, Feb 26, 2024 at 8:38 PM Josep Prat <[email protected]>
wrote:
Hi all,
I'd like to volunteer as release manager for the Apache Kafka 3.8.0
release.
If there are no objections, I'll start building a release plan (or
adapting
the one Colin made some weeks ago) in the wiki in the next days.
Thank you.
--
[image: Aiven] <https://www.aiven.io>
*Josep Prat*
Open Source Engineering Director, *Aiven*
[email protected] | +491715557497
aiven.io <https://www.aiven.io> | <
https://www.facebook.com/aivencloud
<https://www.linkedin.com/company/aiven/> <
https://twitter.com/aiven_io>
*Aiven Deutschland GmbH*
Alexanderufer 3-7, 10117 Berlin
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
Amtsgericht Charlottenburg, HRB 209739 B
--
*Dongjin Lee*
*A hitchhiker in the mathematical world.*
*github: <http://goog_969573159/>github.com/dongjinleekr
<https://github.com/dongjinleekr>keybase: https://keybase.io/dongjinleekr
<https://keybase.io/dongjinleekr>linkedin: kr.linkedin.com/in/dongjinleekr
<https://kr.linkedin.com/in/dongjinleekr>speakerdeck:
speakerdeck.com/dongjin
<https://speakerdeck.com/dongjin>*
