Re: [DISCUSS] KIP-729 Custom validation of records on the broker prior to log append

Interesting point. You are correct that at least KIP-729 cannot validate that. We could propose a different KIP for that which could enforce that in the upper layer. Personally, I would be hesitant to discard the data in that case, but just use metrics/logs to detect those and inform the

Re: [DISCUSS] KIP-729 Custom validation of records on the broker prior to log append

One use case we would like is to require that producers are sending compressed messages. Would this KIP (or KIP-686) allow the broker to detect that? From looking at both KIPs, it doesn't look it would help with my particular use case. Both of the KIPs are at the Record-level. Thanks, -James

Re: [DISCUSS] KIP-760: Increase minimum value of segment.ms and segment.bytes

Badai, Thanks for the KIP. We sometimes want to force compaction on a topic. This might be because there is a bad record in the topic, and we want to force it to get deleted. The way we do this is, we set segment.ms to a small value and write a record, in order to force a segment roll. And we

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #291

See

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #290

See

[jira] [Created] (KAFKA-13041) Support debugging system tests

Stanislav Vodetskyi created KAFKA-13041: --- Summary: Support debugging system tests Key: KAFKA-13041 URL: https://issues.apache.org/jira/browse/KAFKA-13041 Project: Kafka Issue Type:

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #289

See Changes: -- [...truncated 348601 lines...] [2021-07-06T22:04:54.385Z] [2021-07-06T22:04:54.385Z] PlaintextConsumerTest > testCommitSpecifiedOffsets() STARTED

[DISCUSS] KIP-760: Increase minimum value of segment.ms and segment.bytes

Hi all I have just created KIP-760 (https://cwiki.apache.org/confluence/display/KAFKA/KIP-760%3A+Increase+minimum+value+of+segment.ms+and+segment.bytes). I created this KIP because I have seen so many Kafka brokers crash due to small segment.ms and/or segment.bytes. Please let me know what you

[jira] [Resolved] (KAFKA-7760) Add broker configuration to set minimum value for segment.bytes and segment.ms

[ https://issues.apache.org/jira/browse/KAFKA-7760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Badai Aqrandista resolved KAFKA-7760. - Resolution: Duplicate > Add broker configuration to set minimum value for segment.bytes

[jira] [Created] (KAFKA-13040) Increase minimum value of segment.ms and segment.bytes

Badai Aqrandista created KAFKA-13040: Summary: Increase minimum value of segment.ms and segment.bytes Key: KAFKA-13040 URL: https://issues.apache.org/jira/browse/KAFKA-13040 Project: Kafka

Re: New release branch 3.0

Thanks for the heads up Konstantine: I am currently working on these issues below and should send new PRs by the end of the week. My changes should be merged into the 3.0 branch as well as trunk. I have marked them as "blockers" for tracking purposes and for the PR merges. *Add Missing

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #288

See Changes: -- [...truncated 410741 lines...] [2021-07-06T19:37:46.011Z] [2021-07-06T19:37:46.011Z] AuthorizerIntegrationTest >

New release branch 3.0

Hi Kafka developers and friends, The release branch for Apache Kafka 3.0 (with version 3.0.0) has been created (https://github.com/apache/kafka/tree/3.0). The trunk branch is about to be bumped to 3.1.0-SNAPSHOT via https://github.com/apache/kafka/pull/10981 At this point, I'll be reviewing the

[jira] [Resolved] (KAFKA-13035) Kafka Connect: Update documentation for POST /connectors/(string: name)/restart to include task Restart behavior

[ https://issues.apache.org/jira/browse/KAFKA-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-13035. --- Fix Version/s: 3.0.0 Reviewer: Randall Hauch Resolution: Fixed Merged to

[jira] [Resolved] (KAFKA-12979) Implement --find-hanging API in transaction tool

[ https://issues.apache.org/jira/browse/KAFKA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-12979. - Fix Version/s: 3.0.0 Resolution: Fixed > Implement --find-hanging API in

[jira] [Resolved] (KAFKA-12981) Ensure LogSegment.maxTimestampSoFar and LogSegment.offsetOfMaxTimestampSoFar are read/updated in sync

[ https://issues.apache.org/jira/browse/KAFKA-12981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-12981. - Fix Version/s: 3.0.0 Reviewer: David Jacot Resolution: Fixed > Ensure

[jira] [Created] (KAFKA-13039) kafka 0.10.1 gradle build failed

hantaoluo created KAFKA-13039: - Summary: kafka 0.10.1 gradle build failed Key: KAFKA-13039 URL: https://issues.apache.org/jira/browse/KAFKA-13039 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-13038) document IdentityReplicationPolicy

Ryanne Dolan created KAFKA-13038: Summary: document IdentityReplicationPolicy Key: KAFKA-13038 URL: https://issues.apache.org/jira/browse/KAFKA-13038 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-13037) "Thread state is already PENDING_SHUTDOWN" log spam

John Gray created KAFKA-13037: - Summary: "Thread state is already PENDING_SHUTDOWN" log spam Key: KAFKA-13037 URL: https://issues.apache.org/jira/browse/KAFKA-13037 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-13036) Replace EasyMock and PowerMock with Mockito for RocksDBMetricsRecorderTest

YI-CHEN WANG created KAFKA-13036: Summary: Replace EasyMock and PowerMock with Mockito for RocksDBMetricsRecorderTest Key: KAFKA-13036 URL: https://issues.apache.org/jira/browse/KAFKA-13036 Project: