[jira] [Commented] (KAFKA-15417) JoinWindow does not seem to work properly with a KStream - KStream - LeftJoin()
[
https://issues.apache.org/jira/browse/KAFKA-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768550#comment-17768550
]
Victor van den Hoven commented on KAFKA-15417:
--
Not sure what I can do about this:
> Task :streams:test
> org.apache.kafka.streams.integration.EOSUncleanShutdownIntegrationTest.shouldWorkWithUncleanShutdownWipeOutStateStore[exactly_once]
> failed, log available in
> /home/jenkins/jenkins-agent/workspace/Kafka_kafka-pr_PR-14426@2/streams/build/reports/testOutput/org.apache.kafka.streams.integration.EOSUncleanShutdownIntegrationTest.shouldWorkWithUncleanShutdownWipeOutStateStore[exactly_once].test.stdout
> Gradle Test Run :streams:test > Gradle Test Executor 85 >
> EOSUncleanShutdownIntegrationTest > [exactly_once] >
> shouldWorkWithUncleanShutdownWipeOutStateStore[exactly_once] FAILED
:(
> JoinWindow does not seem to work properly with a KStream - KStream -
> LeftJoin()
>
>
> Key: KAFKA-15417
> URL: https://issues.apache.org/jira/browse/KAFKA-15417
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Affects Versions: 3.4.0
>Reporter: Victor van den Hoven
>Assignee: Victor van den Hoven
>Priority: Major
> Attachments: SimpleStreamTopology.java, SimpleStreamTopologyTest.java
>
>
> In Kafka-streams 3.4.0 :
> According to the javadoc of the Joinwindow:
> _There are three different window configuration supported:_
> * _before = after = time-difference_
> * _before = 0 and after = time-difference_
> * _*before = time-difference and after = 0*_
>
> However if I use a joinWindow with *before = time-difference and after = 0*
> on a kstream-kstream-leftjoin the *after=0* part does not seem to work.
> When using _stream1.leftjoin(stream2, joinWindow)_ with
> {_}joinWindow{_}.{_}after=0 and joinWindow.before=30s{_}, any new message on
> stream 1 that can not be joined with any messages on stream2 should be joined
> with a null-record after the _joinWindow.after_ has ended and a new message
> has arrived on stream1.
> It does not.
> Only if the new message arrives after the value of _joinWindow.before_ the
> previous message will be joined with a null-record.
>
> Attached you can find two files with a TopologyTestDriver Unit test to
> reproduce.
> topology: stream1.leftjoin( stream2, joiner, joinwindow)
> joinWindow has before=5000ms and after=0
> message1(key1) -> stream1
> after 4000ms message2(key2) -> stream1 -> NO null-record join was made, but
> the after period was expired.
> after 4900ms message2(key2) -> stream1 -> NO null-record join was made, but
> the after period was expired.
> after 5000ms message2(key2) -> stream1 -> A null-record join was made,
> before period was expired.
> after 6000ms message2(key2) -> stream1 -> A null-record join was made,
> before period was expired.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [kafka] github-actions[bot] commented on pull request #13721: KAFKA-14782: Implementation Details Different from Documentation (del…
github-actions[bot] commented on PR #13721: URL: https://github.com/apache/kafka/pull/13721#issuecomment-1732850906 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge conflicts, please update it with the latest from trunk (or appropriate release branch) If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] github-actions[bot] commented on pull request #13919: MINOR: doc/streams/dsl-api, fix href of "KTable-KTable Foreign-Key Joins"
github-actions[bot] commented on PR #13919: URL: https://github.com/apache/kafka/pull/13919#issuecomment-1732850810 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge conflicts, please update it with the latest from trunk (or appropriate release branch) If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests
lianetm commented on code in PR #14364:
URL: https://github.com/apache/kafka/pull/14364#discussion_r1335307959
##
clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java:
##
@@ -0,0 +1,274 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.clients.consumer.internals;
+
+import org.apache.kafka.clients.ClientResponse;
+import org.apache.kafka.clients.consumer.ConsumerConfig;
+import org.apache.kafka.common.Node;
+import org.apache.kafka.common.Uuid;
+import org.apache.kafka.common.message.ConsumerGroupHeartbeatResponseData;
+import org.apache.kafka.common.protocol.ApiKeys;
+import org.apache.kafka.common.protocol.Errors;
+import org.apache.kafka.common.requests.ConsumerGroupHeartbeatResponse;
+import org.apache.kafka.common.requests.RequestHeader;
+import org.apache.kafka.common.serialization.StringDeserializer;
+import org.apache.kafka.common.utils.LogContext;
+import org.apache.kafka.common.utils.MockTime;
+import org.apache.kafka.common.utils.Time;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
+import org.junit.jupiter.params.provider.ValueSource;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Optional;
+import java.util.Properties;
+
+import static
org.apache.kafka.clients.consumer.ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG;
+import static
org.apache.kafka.clients.consumer.ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG;
+import static
org.apache.kafka.clients.consumer.ConsumerConfig.RETRY_BACKOFF_MS_CONFIG;
+import static
org.apache.kafka.clients.consumer.ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyLong;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.never;
+import static org.mockito.Mockito.spy;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+public class HeartbeatRequestManagerTest {
+
+private final int heartbeatInterval = 1000;
+private final long retryBackoffMaxMs = 3000;
+private final long retryBackoffMs = 100;
+private final String groupId = "group-id";
+
+private Time mockTime;
+private LogContext mockLogContext;
+private CoordinatorRequestManager mockCoordinatorRequestManager;
+private SubscriptionState mockSubscriptionState;
+private HeartbeatRequestManager heartbeatRequestManager;
+private MembershipManager mockMembershipManager;
+private HeartbeatRequestManager.HeartbeatRequestState
heartbeatRequestState;
+private ConsumerConfig config;
+
+private String memberId = "member-id";
+private int memberEpoch = 1;
+private ConsumerGroupHeartbeatResponseData.Assignment memberAssignment =
mockAssignment();
+private ErrorEventHandler errorEventHandler;
+
+private ConsumerGroupHeartbeatResponseData.Assignment mockAssignment() {
+return new ConsumerGroupHeartbeatResponseData.Assignment()
+.setAssignedTopicPartitions(Arrays.asList(
+new ConsumerGroupHeartbeatResponseData.TopicPartitions()
+.setTopicId(Uuid.randomUuid())
+.setPartitions(Arrays.asList(0, 1, 2)),
+new ConsumerGroupHeartbeatResponseData.TopicPartitions()
+.setTopicId(Uuid.randomUuid())
+.setPartitions(Arrays.asList(3, 4, 5))
+));
+}
+
+@BeforeEach
+public void setUp() {
+mockTime = new MockTime();
+mockLogContext = new LogContext();
+Properties properties = new Properties();
+properties.put(BOOTSTRAP_SERVERS_CONFIG, "localhost:");
+properties.put(KEY_DESERIALIZER_CLASS_CONFIG,
StringDeserializer.class);
+properties.put(VALUE_DESERIALIZER_CLASS_CONFIG,
StringDeserializer.class);
+properties.put(RETRY_BACKOFF_MS_CONFIG, "100");
+config = new ConsumerConfig(properties);
+
[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests
lianetm commented on code in PR #14364:
URL: https://github.com/apache/kafka/pull/14364#discussion_r1335309021
##
clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java:
##
@@ -0,0 +1,274 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.clients.consumer.internals;
+
+import org.apache.kafka.clients.ClientResponse;
+import org.apache.kafka.clients.consumer.ConsumerConfig;
+import org.apache.kafka.common.Node;
+import org.apache.kafka.common.Uuid;
+import org.apache.kafka.common.message.ConsumerGroupHeartbeatResponseData;
+import org.apache.kafka.common.protocol.ApiKeys;
+import org.apache.kafka.common.protocol.Errors;
+import org.apache.kafka.common.requests.ConsumerGroupHeartbeatResponse;
+import org.apache.kafka.common.requests.RequestHeader;
+import org.apache.kafka.common.serialization.StringDeserializer;
+import org.apache.kafka.common.utils.LogContext;
+import org.apache.kafka.common.utils.MockTime;
+import org.apache.kafka.common.utils.Time;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
+import org.junit.jupiter.params.provider.ValueSource;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Optional;
+import java.util.Properties;
+
+import static
org.apache.kafka.clients.consumer.ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG;
+import static
org.apache.kafka.clients.consumer.ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG;
+import static
org.apache.kafka.clients.consumer.ConsumerConfig.RETRY_BACKOFF_MS_CONFIG;
+import static
org.apache.kafka.clients.consumer.ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyLong;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.never;
+import static org.mockito.Mockito.spy;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+public class HeartbeatRequestManagerTest {
+
+private final int heartbeatInterval = 1000;
+private final long retryBackoffMaxMs = 3000;
+private final long retryBackoffMs = 100;
+private final String groupId = "group-id";
+
+private Time mockTime;
+private LogContext mockLogContext;
+private CoordinatorRequestManager mockCoordinatorRequestManager;
+private SubscriptionState mockSubscriptionState;
+private HeartbeatRequestManager heartbeatRequestManager;
+private MembershipManager mockMembershipManager;
+private HeartbeatRequestManager.HeartbeatRequestState
heartbeatRequestState;
+private ConsumerConfig config;
+
+private String memberId = "member-id";
+private int memberEpoch = 1;
+private ConsumerGroupHeartbeatResponseData.Assignment memberAssignment =
mockAssignment();
+private ErrorEventHandler errorEventHandler;
+
+private ConsumerGroupHeartbeatResponseData.Assignment mockAssignment() {
+return new ConsumerGroupHeartbeatResponseData.Assignment()
+.setAssignedTopicPartitions(Arrays.asList(
+new ConsumerGroupHeartbeatResponseData.TopicPartitions()
+.setTopicId(Uuid.randomUuid())
+.setPartitions(Arrays.asList(0, 1, 2)),
+new ConsumerGroupHeartbeatResponseData.TopicPartitions()
+.setTopicId(Uuid.randomUuid())
+.setPartitions(Arrays.asList(3, 4, 5))
+));
+}
+
+@BeforeEach
+public void setUp() {
+mockTime = new MockTime();
+mockLogContext = new LogContext();
+Properties properties = new Properties();
+properties.put(BOOTSTRAP_SERVERS_CONFIG, "localhost:");
+properties.put(KEY_DESERIALIZER_CLASS_CONFIG,
StringDeserializer.class);
+properties.put(VALUE_DESERIALIZER_CLASS_CONFIG,
StringDeserializer.class);
+properties.put(RETRY_BACKOFF_MS_CONFIG, "100");
+config = new ConsumerConfig(properties);
+
[GitHub] [kafka] vamossagar12 commented on pull request #14432: KAFKA-14516: [1/N] Adding logic to not fence a static member if it sends a LeaveGroup
vamossagar12 commented on PR #14432: URL: https://github.com/apache/kafka/pull/14432#issuecomment-1732788357 Test failures seem unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests
lianetm commented on code in PR #14364:
URL: https://github.com/apache/kafka/pull/14364#discussion_r1335307959
##
clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java:
##
@@ -0,0 +1,274 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.clients.consumer.internals;
+
+import org.apache.kafka.clients.ClientResponse;
+import org.apache.kafka.clients.consumer.ConsumerConfig;
+import org.apache.kafka.common.Node;
+import org.apache.kafka.common.Uuid;
+import org.apache.kafka.common.message.ConsumerGroupHeartbeatResponseData;
+import org.apache.kafka.common.protocol.ApiKeys;
+import org.apache.kafka.common.protocol.Errors;
+import org.apache.kafka.common.requests.ConsumerGroupHeartbeatResponse;
+import org.apache.kafka.common.requests.RequestHeader;
+import org.apache.kafka.common.serialization.StringDeserializer;
+import org.apache.kafka.common.utils.LogContext;
+import org.apache.kafka.common.utils.MockTime;
+import org.apache.kafka.common.utils.Time;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
+import org.junit.jupiter.params.provider.ValueSource;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Optional;
+import java.util.Properties;
+
+import static
org.apache.kafka.clients.consumer.ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG;
+import static
org.apache.kafka.clients.consumer.ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG;
+import static
org.apache.kafka.clients.consumer.ConsumerConfig.RETRY_BACKOFF_MS_CONFIG;
+import static
org.apache.kafka.clients.consumer.ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyLong;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.never;
+import static org.mockito.Mockito.spy;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+public class HeartbeatRequestManagerTest {
+
+private final int heartbeatInterval = 1000;
+private final long retryBackoffMaxMs = 3000;
+private final long retryBackoffMs = 100;
+private final String groupId = "group-id";
+
+private Time mockTime;
+private LogContext mockLogContext;
+private CoordinatorRequestManager mockCoordinatorRequestManager;
+private SubscriptionState mockSubscriptionState;
+private HeartbeatRequestManager heartbeatRequestManager;
+private MembershipManager mockMembershipManager;
+private HeartbeatRequestManager.HeartbeatRequestState
heartbeatRequestState;
+private ConsumerConfig config;
+
+private String memberId = "member-id";
+private int memberEpoch = 1;
+private ConsumerGroupHeartbeatResponseData.Assignment memberAssignment =
mockAssignment();
+private ErrorEventHandler errorEventHandler;
+
+private ConsumerGroupHeartbeatResponseData.Assignment mockAssignment() {
+return new ConsumerGroupHeartbeatResponseData.Assignment()
+.setAssignedTopicPartitions(Arrays.asList(
+new ConsumerGroupHeartbeatResponseData.TopicPartitions()
+.setTopicId(Uuid.randomUuid())
+.setPartitions(Arrays.asList(0, 1, 2)),
+new ConsumerGroupHeartbeatResponseData.TopicPartitions()
+.setTopicId(Uuid.randomUuid())
+.setPartitions(Arrays.asList(3, 4, 5))
+));
+}
+
+@BeforeEach
+public void setUp() {
+mockTime = new MockTime();
+mockLogContext = new LogContext();
+Properties properties = new Properties();
+properties.put(BOOTSTRAP_SERVERS_CONFIG, "localhost:");
+properties.put(KEY_DESERIALIZER_CLASS_CONFIG,
StringDeserializer.class);
+properties.put(VALUE_DESERIALIZER_CLASS_CONFIG,
StringDeserializer.class);
+properties.put(RETRY_BACKOFF_MS_CONFIG, "100");
+config = new ConsumerConfig(properties);
+
[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests
lianetm commented on code in PR #14364:
URL: https://github.com/apache/kafka/pull/14364#discussion_r1335304701
##
clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java:
##
@@ -0,0 +1,294 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.clients.consumer.internals;
+
+import org.apache.kafka.clients.CommonClientConfigs;
+import org.apache.kafka.clients.consumer.ConsumerConfig;
+import org.apache.kafka.common.KafkaException;
+import org.apache.kafka.common.errors.GroupAuthorizationException;
+import org.apache.kafka.common.message.ConsumerGroupHeartbeatRequestData;
+import org.apache.kafka.common.protocol.Errors;
+import org.apache.kafka.common.requests.ConsumerGroupHeartbeatRequest;
+import org.apache.kafka.common.requests.ConsumerGroupHeartbeatResponse;
+import org.apache.kafka.common.utils.LogContext;
+import org.apache.kafka.common.utils.Time;
+import org.apache.kafka.common.utils.Timer;
+import org.slf4j.Logger;
+
+import java.util.ArrayList;
+import java.util.Collections;
+
+/**
+ * Manages the request creation and response handling for the heartbeat. The
module creates a {@link ConsumerGroupHeartbeatRequest}
+ * using the state stored in the {@link MembershipManager} and enqueue it to
the network queue to be sent out. Once
+ * the response is received, the module will update the state in the {@link
MembershipManager} and handle any errors.
+ *
+ * The manager only emits heartbeat when the member is in a group, tries to
join a group, or tries rejoin the group.
+ * If the member does not have groupId configured, left the group, or
encountering fatal exceptions, the heartbeat will
+ * not be sent. If the coordinator not is not found, we will skip sending the
heartbeat and tries to find a coordinator first.
+ *
+ * If the heartbeat failed due to retriable errors, such as, TimeoutException.
The subsequent attempt will be backoff
+ * exponentially.
+ *
+ * If the member completes the partition revocation process, a heartbeat
request will be sent in the next event loop.
+ *
+ * {@link HeartbeatRequestState} for more details.
+ */
+public class HeartbeatRequestManager implements RequestManager {
+private final Logger logger;
+
+private final int rebalanceTimeoutMs;
+
+private final CoordinatorRequestManager coordinatorRequestManager;
+private final SubscriptionState subscriptions;
+private final HeartbeatRequestState heartbeatRequestState;
+private final MembershipManager membershipManager;
+private final ErrorEventHandler nonRetriableErrorHandler;
+
+public HeartbeatRequestManager(
+final Time time,
+final LogContext logContext,
+final ConsumerConfig config,
+final CoordinatorRequestManager coordinatorRequestManager,
+final SubscriptionState subscriptions,
+final MembershipManager membershipManager,
+final ErrorEventHandler nonRetriableErrorHandler) {
+this.coordinatorRequestManager = coordinatorRequestManager;
+this.logger = logContext.logger(getClass());
+this.subscriptions = subscriptions;
+this.membershipManager = membershipManager;
+this.nonRetriableErrorHandler = nonRetriableErrorHandler;
+this.rebalanceTimeoutMs =
config.getInt(CommonClientConfigs.MAX_POLL_INTERVAL_MS_CONFIG);
+long retryBackoffMs =
config.getLong(ConsumerConfig.RETRY_BACKOFF_MS_CONFIG);
+long retryBackoffMaxMs =
config.getLong(ConsumerConfig.RETRY_BACKOFF_MAX_MS_CONFIG);
+this.heartbeatRequestState = new HeartbeatRequestState(logContext,
time, 0, retryBackoffMs,
+retryBackoffMaxMs, rebalanceTimeoutMs);
+}
+
+// Visible for testing
+HeartbeatRequestManager(
+final Time time,
+final LogContext logContext,
+final ConsumerConfig config,
+final CoordinatorRequestManager coordinatorRequestManager,
+final SubscriptionState subscriptions,
+final MembershipManager membershipManager,
+final HeartbeatRequestState heartbeatRequestState,
+final ErrorEventHandler nonRetriableErrorHandler) {
+this.logger = logContext.logger(this.getClass());
+
[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests
lianetm commented on code in PR #14364:
URL: https://github.com/apache/kafka/pull/14364#discussion_r1335297785
##
clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java:
##
@@ -0,0 +1,294 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.clients.consumer.internals;
+
+import org.apache.kafka.clients.CommonClientConfigs;
+import org.apache.kafka.clients.consumer.ConsumerConfig;
+import org.apache.kafka.common.KafkaException;
+import org.apache.kafka.common.errors.GroupAuthorizationException;
+import org.apache.kafka.common.message.ConsumerGroupHeartbeatRequestData;
+import org.apache.kafka.common.protocol.Errors;
+import org.apache.kafka.common.requests.ConsumerGroupHeartbeatRequest;
+import org.apache.kafka.common.requests.ConsumerGroupHeartbeatResponse;
+import org.apache.kafka.common.utils.LogContext;
+import org.apache.kafka.common.utils.Time;
+import org.apache.kafka.common.utils.Timer;
+import org.slf4j.Logger;
+
+import java.util.ArrayList;
+import java.util.Collections;
+
+/**
+ * Manages the request creation and response handling for the heartbeat. The
module creates a {@link ConsumerGroupHeartbeatRequest}
+ * using the state stored in the {@link MembershipManager} and enqueue it to
the network queue to be sent out. Once
+ * the response is received, the module will update the state in the {@link
MembershipManager} and handle any errors.
+ *
+ * The manager only emits heartbeat when the member is in a group, tries to
join a group, or tries rejoin the group.
+ * If the member does not have groupId configured, left the group, or
encountering fatal exceptions, the heartbeat will
+ * not be sent. If the coordinator not is not found, we will skip sending the
heartbeat and tries to find a coordinator first.
+ *
+ * If the heartbeat failed due to retriable errors, such as, TimeoutException.
The subsequent attempt will be backoff
+ * exponentially.
+ *
+ * If the member completes the partition revocation process, a heartbeat
request will be sent in the next event loop.
+ *
+ * {@link HeartbeatRequestState} for more details.
+ */
+public class HeartbeatRequestManager implements RequestManager {
+private final Logger logger;
+
+private final int rebalanceTimeoutMs;
+
+private final CoordinatorRequestManager coordinatorRequestManager;
+private final SubscriptionState subscriptions;
+private final HeartbeatRequestState heartbeatRequestState;
+private final MembershipManager membershipManager;
+private final ErrorEventHandler nonRetriableErrorHandler;
+
+public HeartbeatRequestManager(
+final Time time,
+final LogContext logContext,
+final ConsumerConfig config,
+final CoordinatorRequestManager coordinatorRequestManager,
+final SubscriptionState subscriptions,
+final MembershipManager membershipManager,
+final ErrorEventHandler nonRetriableErrorHandler) {
+this.coordinatorRequestManager = coordinatorRequestManager;
+this.logger = logContext.logger(getClass());
+this.subscriptions = subscriptions;
+this.membershipManager = membershipManager;
+this.nonRetriableErrorHandler = nonRetriableErrorHandler;
+this.rebalanceTimeoutMs =
config.getInt(CommonClientConfigs.MAX_POLL_INTERVAL_MS_CONFIG);
+long retryBackoffMs =
config.getLong(ConsumerConfig.RETRY_BACKOFF_MS_CONFIG);
+long retryBackoffMaxMs =
config.getLong(ConsumerConfig.RETRY_BACKOFF_MAX_MS_CONFIG);
+this.heartbeatRequestState = new HeartbeatRequestState(logContext,
time, 0, retryBackoffMs,
+retryBackoffMaxMs, rebalanceTimeoutMs);
+}
+
+// Visible for testing
+HeartbeatRequestManager(
+final Time time,
+final LogContext logContext,
+final ConsumerConfig config,
+final CoordinatorRequestManager coordinatorRequestManager,
+final SubscriptionState subscriptions,
+final MembershipManager membershipManager,
+final HeartbeatRequestState heartbeatRequestState,
+final ErrorEventHandler nonRetriableErrorHandler) {
+this.logger = logContext.logger(this.getClass());
+
[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests
lianetm commented on code in PR #14364:
URL: https://github.com/apache/kafka/pull/14364#discussion_r1335295791
##
clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java:
##
@@ -115,25 +115,37 @@ public int memberEpoch() {
@Override
public void updateState(ConsumerGroupHeartbeatResponseData response) {
-if (response.errorCode() == Errors.NONE.code()) {
-this.memberId = response.memberId();
-this.memberEpoch = response.memberEpoch();
-ConsumerGroupHeartbeatResponseData.Assignment assignment =
response.assignment();
-if (assignment != null) {
-setTargetAssignment(assignment);
-}
-maybeTransitionToStable();
-} else {
-if (response.errorCode() == Errors.FENCED_MEMBER_EPOCH.code() ||
response.errorCode() == Errors.UNKNOWN_MEMBER_ID.code()) {
-resetEpoch();
-transitionTo(MemberState.FENCED);
-} else if (response.errorCode() ==
Errors.UNRELEASED_INSTANCE_ID.code()) {
-transitionTo(MemberState.FAILED);
-}
-// TODO: handle other errors here to update state accordingly,
mainly making the
-// distinction between the recoverable errors and the fatal ones,
that should FAILED
-// the member
+if (response.errorCode() != Errors.NONE.code()) {
+String errorMessage = String.format(
+"Unexpected error in Heartbeat response. Expected no
error, but received: %s",
+Errors.forCode(response.errorCode())
+);
+throw new IllegalStateException(errorMessage);
+}
+this.memberId = response.memberId();
+this.memberEpoch = response.memberEpoch();
+ConsumerGroupHeartbeatResponseData.Assignment assignment =
response.assignment();
+if (assignment != null) {
+setTargetAssignment(assignment);
}
+maybeTransitionToStable();
+}
+
+@Override
+public void fenceMember() {
+resetEpoch();
+transitionTo(MemberState.FENCED);
+}
+
+@Override
+public void transitionToFailure() {
+transitionTo(MemberState.FAILED);
+}
+
+@Override
+public boolean shouldSendHeartbeat() {
Review Comment:
`shouldSendHeartbeat` returning false when UNJOINED does not seem right. We
do need to send HB when UNJOINED to be able to join the group. I would say
FAILED is the only state we we shouldn't send HB.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests
lianetm commented on code in PR #14364:
URL: https://github.com/apache/kafka/pull/14364#discussion_r1335289660
##
clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java:
##
@@ -0,0 +1,294 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.clients.consumer.internals;
+
+import org.apache.kafka.clients.CommonClientConfigs;
+import org.apache.kafka.clients.consumer.ConsumerConfig;
+import org.apache.kafka.common.KafkaException;
+import org.apache.kafka.common.errors.GroupAuthorizationException;
+import org.apache.kafka.common.message.ConsumerGroupHeartbeatRequestData;
+import org.apache.kafka.common.protocol.Errors;
+import org.apache.kafka.common.requests.ConsumerGroupHeartbeatRequest;
+import org.apache.kafka.common.requests.ConsumerGroupHeartbeatResponse;
+import org.apache.kafka.common.utils.LogContext;
+import org.apache.kafka.common.utils.Time;
+import org.apache.kafka.common.utils.Timer;
+import org.slf4j.Logger;
+
+import java.util.ArrayList;
+import java.util.Collections;
+
+/**
+ * Manages the request creation and response handling for the heartbeat. The
module creates a {@link ConsumerGroupHeartbeatRequest}
+ * using the state stored in the {@link MembershipManager} and enqueue it to
the network queue to be sent out. Once
+ * the response is received, the module will update the state in the {@link
MembershipManager} and handle any errors.
+ *
+ * The manager only emits heartbeat when the member is in a group, tries to
join a group, or tries rejoin the group.
+ * If the member does not have groupId configured, left the group, or
encountering fatal exceptions, the heartbeat will
+ * not be sent. If the coordinator not is not found, we will skip sending the
heartbeat and tries to find a coordinator first.
+ *
+ * If the heartbeat failed due to retriable errors, such as, TimeoutException.
The subsequent attempt will be backoff
+ * exponentially.
+ *
+ * If the member completes the partition revocation process, a heartbeat
request will be sent in the next event loop.
+ *
+ * {@link HeartbeatRequestState} for more details.
+ */
+public class HeartbeatRequestManager implements RequestManager {
+private final Logger logger;
+
+private final int rebalanceTimeoutMs;
+
+private final CoordinatorRequestManager coordinatorRequestManager;
+private final SubscriptionState subscriptions;
+private final HeartbeatRequestState heartbeatRequestState;
+private final MembershipManager membershipManager;
+private final ErrorEventHandler nonRetriableErrorHandler;
+
+public HeartbeatRequestManager(
+final Time time,
+final LogContext logContext,
+final ConsumerConfig config,
+final CoordinatorRequestManager coordinatorRequestManager,
+final SubscriptionState subscriptions,
+final MembershipManager membershipManager,
+final ErrorEventHandler nonRetriableErrorHandler) {
+this.coordinatorRequestManager = coordinatorRequestManager;
+this.logger = logContext.logger(getClass());
+this.subscriptions = subscriptions;
+this.membershipManager = membershipManager;
+this.nonRetriableErrorHandler = nonRetriableErrorHandler;
+this.rebalanceTimeoutMs =
config.getInt(CommonClientConfigs.MAX_POLL_INTERVAL_MS_CONFIG);
+long retryBackoffMs =
config.getLong(ConsumerConfig.RETRY_BACKOFF_MS_CONFIG);
+long retryBackoffMaxMs =
config.getLong(ConsumerConfig.RETRY_BACKOFF_MAX_MS_CONFIG);
+this.heartbeatRequestState = new HeartbeatRequestState(logContext,
time, 0, retryBackoffMs,
Review Comment:
Got it, seeing that the `canSendRequest` considers inflight requests then it
makes sense to set an initial value of 0 I would say, so that we send the first
HB as soon as the HM manager starts. I would only suggest to add some tests for
the interval, including this case where we might not get a response to our
first HB request.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.ap
[GitHub] [kafka] ijuma commented on pull request #14433: KAFKA-15485: Support building with Java 21 (3/3)
ijuma commented on PR #14433: URL: https://github.com/apache/kafka/pull/14433#issuecomment-1732746476 Java 21 and Java 8 builds passed in the latest re-run. The failures for the other builds are unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KAFKA-15495) KRaft partition truncated when the only ISR member restarts with an empty disk
[
https://issues.apache.org/jira/browse/KAFKA-15495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768472#comment-17768472
]
Ismael Juma commented on KAFKA-15495:
-
Please check KIP-966 - it talks about this problem.
> KRaft partition truncated when the only ISR member restarts with an empty disk
> --
>
> Key: KAFKA-15495
> URL: https://issues.apache.org/jira/browse/KAFKA-15495
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 3.3.2, 3.4.1, 3.6.0, 3.5.1
>Reporter: Ron Dagostino
>Priority: Critical
>
> Assume a topic-partition in KRaft has just a single leader replica in the
> ISR. Assume next that this replica goes offline. This replica's log will
> define the contents of that partition when the replica restarts, which is
> correct behavior. However, assume now that the replica has a disk failure,
> and we then replace the failed disk with a new, empty disk that we also
> format with the storage tool so it has the correct cluster ID. If we then
> restart the broker, the topic-partition will have no data in it, and any
> other replicas that might exist will truncate their logs to match, which
> results in data loss. See below for a step-by-step demo of how to reproduce
> this.
> [KIP-858: Handle JBOD broker disk failure in
> KRaft|https://cwiki.apache.org/confluence/display/KAFKA/KIP-858%3A+Handle+JBOD+broker+disk+failure+in+KRaft]
> introduces the concept of a Disk UUID that we can use to solve this problem.
> Specifically, when the leader restarts with an empty (but
> correctly-formatted) disk, the actual UUID associated with the disk will be
> different. The controller will notice upon broker re-registration that its
> disk UUID differs from what was previously registered. Right now we have no
> way of detecting this situation, but the disk UUID gives us that capability.
> STEPS TO REPRODUCE:
> Create a single broker cluster with single controller. The standard files
> under config/kraft work well:
> bin/kafka-storage.sh random-uuid
> J8qXRwI-Qyi2G0guFTiuYw
> #ensure we start clean
> /bin/rm -rf /tmp/kraft-broker-logs /tmp/kraft-controller-logs
> bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config
> config/kraft/controller.properties
> bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config
> config/kraft/broker.properties
> bin/kafka-server-start.sh config/kraft/controller.properties
> bin/kafka-server-start.sh config/kraft/broker.properties
> bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic foo1
> --partitions 1 --replication-factor 1
> #create __consumer-offsets topics
> bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic foo1
> --from-beginning
> ^C
> #confirm that __consumer_offsets topic partitions are all created and on
> broker with node id 2
> bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe
> Now create 2 more brokers, with node IDs 11 and 12
> cat config/kraft/broker.properties | sed 's/node.id=2/node.id=11/' | sed
> 's/localhost:9092/localhost:9011/g' | sed
> 's#log.dirs=/tmp/kraft-broker-logs#log.dirs=/tmp/kraft-broker-logs11#' >
> config/kraft/broker11.properties
> cat config/kraft/broker.properties | sed 's/node.id=2/node.id=12/' | sed
> 's/localhost:9092/localhost:9012/g' | sed
> 's#log.dirs=/tmp/kraft-broker-logs#log.dirs=/tmp/kraft-broker-logs12#' >
> config/kraft/broker12.properties
> #ensure we start clean
> /bin/rm -rf /tmp/kraft-broker-logs11 /tmp/kraft-broker-logs12
> bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config
> config/kraft/broker11.properties
> bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config
> config/kraft/broker12.properties
> bin/kafka-server-start.sh config/kraft/broker11.properties
> bin/kafka-server-start.sh config/kraft/broker12.properties
> #create a topic with a single partition replicated on two brokers
> bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic foo2
> --partitions 1 --replication-factor 2
> #reassign partitions onto brokers with Node IDs 11 and 12
> echo '{"partitions":[{"topic": "foo2","partition": 0,"replicas": [11,12]}],
> "version":1}' > /tmp/reassign.json
> bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092
> --reassignment-json-file /tmp/reassign.json --execute
> bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092
> --reassignment-json-file /tmp/reassign.json --verify
> #make preferred leader 11 the actual leader if it not
> bin/kafka-leader-election.sh --bootstrap-server localhost:9092
> --all-topic-partitions --election-type preferred
> #Confirm both brokers are in ISR and 11 is the leader
> bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic
[jira] [Commented] (KAFKA-15485) Support building with Java 21 (LTS release)
[ https://issues.apache.org/jira/browse/KAFKA-15485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768460#comment-17768460 ] Ismael Juma commented on KAFKA-15485: - Yes. > Support building with Java 21 (LTS release) > --- > > Key: KAFKA-15485 > URL: https://issues.apache.org/jira/browse/KAFKA-15485 > Project: Kafka > Issue Type: Improvement >Reporter: Divij Vaidya >Assignee: Ismael Juma >Priority: Major > Fix For: 3.7.0 > > > JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This > ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20). > Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not > anticipate major hiccups while upgrading to JDK-21. > As part of this JIRA we want to: > 1. Upgrade Kafka to JDK 21 > 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below) > 3. Update the README (see[4]) to mention Kafka's support for JDK-21 > [1] [https://github.com/apache/kafka/pull/13840] > [2] [https://github.com/apache/kafka/pull/13582] > [3] [https://github.com/apache/kafka/pull/12948] > [4] [https://github.com/apache/kafka/pull/14061] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-15497) Refactor build.gradle and split each module configuration on the module itself
Said BOUDJELDA created KAFKA-15497: -- Summary: Refactor build.gradle and split each module configuration on the module itself Key: KAFKA-15497 URL: https://issues.apache.org/jira/browse/KAFKA-15497 Project: Kafka Issue Type: Improvement Reporter: Said BOUDJELDA The *build.gradle* file is getting too big and hard to maintain, a good reason to split this files over modules of the project, and let the root *build.gradle* file manage just the common parts of the projet, this will increase readability -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (KAFKA-15493) Ensure system tests work with Java 21
[ https://issues.apache.org/jira/browse/KAFKA-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Said BOUDJELDA reassigned KAFKA-15493: -- Assignee: Said BOUDJELDA > Ensure system tests work with Java 21 > - > > Key: KAFKA-15493 > URL: https://issues.apache.org/jira/browse/KAFKA-15493 > Project: Kafka > Issue Type: Improvement >Reporter: Ismael Juma >Assignee: Said BOUDJELDA >Priority: Major > Fix For: 3.7.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (KAFKA-15485) Support building with Java 21 (LTS release)
[ https://issues.apache.org/jira/browse/KAFKA-15485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768454#comment-17768454 ] Said BOUDJELDA edited comment on KAFKA-15485 at 9/24/23 10:50 PM: -- [~ijuma] Can I take the KAFKA-15493 task then was (Author: JIRAUSER301378): [~ijuma] Can I take the KAFKA-15493 task then ? > Support building with Java 21 (LTS release) > --- > > Key: KAFKA-15485 > URL: https://issues.apache.org/jira/browse/KAFKA-15485 > Project: Kafka > Issue Type: Improvement >Reporter: Divij Vaidya >Assignee: Ismael Juma >Priority: Major > Fix For: 3.7.0 > > > JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This > ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20). > Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not > anticipate major hiccups while upgrading to JDK-21. > As part of this JIRA we want to: > 1. Upgrade Kafka to JDK 21 > 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below) > 3. Update the README (see[4]) to mention Kafka's support for JDK-21 > [1] [https://github.com/apache/kafka/pull/13840] > [2] [https://github.com/apache/kafka/pull/13582] > [3] [https://github.com/apache/kafka/pull/12948] > [4] [https://github.com/apache/kafka/pull/14061] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (KAFKA-15485) Support building with Java 21 (LTS release)
[ https://issues.apache.org/jira/browse/KAFKA-15485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768454#comment-17768454 ] Said BOUDJELDA edited comment on KAFKA-15485 at 9/24/23 10:12 PM: -- [~ijuma] Can I take the KAFKA-15493 task then ? was (Author: JIRAUSER301378): [~ijuma] Can I take this task then ? > Support building with Java 21 (LTS release) > --- > > Key: KAFKA-15485 > URL: https://issues.apache.org/jira/browse/KAFKA-15485 > Project: Kafka > Issue Type: Improvement >Reporter: Divij Vaidya >Assignee: Ismael Juma >Priority: Major > Fix For: 3.7.0 > > > JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This > ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20). > Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not > anticipate major hiccups while upgrading to JDK-21. > As part of this JIRA we want to: > 1. Upgrade Kafka to JDK 21 > 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below) > 3. Update the README (see[4]) to mention Kafka's support for JDK-21 > [1] [https://github.com/apache/kafka/pull/13840] > [2] [https://github.com/apache/kafka/pull/13582] > [3] [https://github.com/apache/kafka/pull/12948] > [4] [https://github.com/apache/kafka/pull/14061] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KAFKA-15485) Support building with Java 21 (LTS release)
[ https://issues.apache.org/jira/browse/KAFKA-15485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768454#comment-17768454 ] Said BOUDJELDA commented on KAFKA-15485: [~ijuma] Can I take this task then ? > Support building with Java 21 (LTS release) > --- > > Key: KAFKA-15485 > URL: https://issues.apache.org/jira/browse/KAFKA-15485 > Project: Kafka > Issue Type: Improvement >Reporter: Divij Vaidya >Assignee: Ismael Juma >Priority: Major > Fix For: 3.7.0 > > > JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This > ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20). > Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not > anticipate major hiccups while upgrading to JDK-21. > As part of this JIRA we want to: > 1. Upgrade Kafka to JDK 21 > 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below) > 3. Update the README (see[4]) to mention Kafka's support for JDK-21 > [1] [https://github.com/apache/kafka/pull/13840] > [2] [https://github.com/apache/kafka/pull/13582] > [3] [https://github.com/apache/kafka/pull/12948] > [4] [https://github.com/apache/kafka/pull/14061] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KAFKA-15496) Upgrade Jetty to version 11.0.16
[ https://issues.apache.org/jira/browse/KAFKA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768453#comment-17768453 ] Said BOUDJELDA commented on KAFKA-15496: [~divijvaidya] I want to know what do you think about this > Upgrade Jetty to version 11.0.16 > > > Key: KAFKA-15496 > URL: https://issues.apache.org/jira/browse/KAFKA-15496 > Project: Kafka > Issue Type: Improvement >Reporter: Said BOUDJELDA >Priority: Major > > Check release notes > [https://github.com/eclipse/jetty.project/releases/tag/jetty-11.0.16] > This task cannot be achieved without the upgrade of *jackson-databind* > library please check for more details KAFKA-15208 for more details > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-15496) Upgrade Jetty to version 11.0.16
Said BOUDJELDA created KAFKA-15496: -- Summary: Upgrade Jetty to version 11.0.16 Key: KAFKA-15496 URL: https://issues.apache.org/jira/browse/KAFKA-15496 Project: Kafka Issue Type: Improvement Reporter: Said BOUDJELDA Check release notes [https://github.com/eclipse/jetty.project/releases/tag/jetty-11.0.16] This task cannot be achieved without the upgrade of *jackson-databind* library please check for more details KAFKA-15208 for more details -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (KAFKA-15485) Support building with Java 21 (LTS release)
[
https://issues.apache.org/jira/browse/KAFKA-15485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768452#comment-17768452
]
Ismael Juma edited comment on KAFKA-15485 at 9/24/23 9:43 PM:
--
[~bmscomp] Run the system tests as described below with Java 21:
[https://github.com/apache/kafka/tree/trunk/tests]
One relevant portion:
Run tests with a different JVM (it may be as easy as replacing 11 with 21)
{code:java}
bash tests/docker/ducker-ak up -j 'openjdk:11'; tests/docker/run_tests.sh{code}
For the tests that fail, we would have to investigate why.
was (Author: ijuma):
[~bmscomp] Run the system tests as described below with Java 21:
[https://github.com/apache/kafka/tree/trunk/tests]
One relevant portion:
Run tests with a different JVM (it may be as easy as replacing 11 with 21)
bash tests/docker/ducker-ak up -j 'openjdk:11'; tests/docker/run_tests.sh
For the tests that fail, we would have to investigate why.
> Support building with Java 21 (LTS release)
> ---
>
> Key: KAFKA-15485
> URL: https://issues.apache.org/jira/browse/KAFKA-15485
> Project: Kafka
> Issue Type: Improvement
>Reporter: Divij Vaidya
>Assignee: Ismael Juma
>Priority: Major
> Fix For: 3.7.0
>
>
> JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This
> ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20).
> Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not
> anticipate major hiccups while upgrading to JDK-21.
> As part of this JIRA we want to:
> 1. Upgrade Kafka to JDK 21
> 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below)
> 3. Update the README (see[4]) to mention Kafka's support for JDK-21
> [1] [https://github.com/apache/kafka/pull/13840]
> [2] [https://github.com/apache/kafka/pull/13582]
> [3] [https://github.com/apache/kafka/pull/12948]
> [4] [https://github.com/apache/kafka/pull/14061]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KAFKA-15485) Support building with Java 21 (LTS release)
[ https://issues.apache.org/jira/browse/KAFKA-15485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768452#comment-17768452 ] Ismael Juma commented on KAFKA-15485: - [~bmscomp] Run the system tests as described below with Java 21: [https://github.com/apache/kafka/tree/trunk/tests] One relevant portion: Run tests with a different JVM (it may be as easy as replacing 11 with 21) bash tests/docker/ducker-ak up -j 'openjdk:11'; tests/docker/run_tests.sh For the tests that fail, we would have to investigate why. > Support building with Java 21 (LTS release) > --- > > Key: KAFKA-15485 > URL: https://issues.apache.org/jira/browse/KAFKA-15485 > Project: Kafka > Issue Type: Improvement >Reporter: Divij Vaidya >Assignee: Ismael Juma >Priority: Major > Fix For: 3.7.0 > > > JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This > ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20). > Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not > anticipate major hiccups while upgrading to JDK-21. > As part of this JIRA we want to: > 1. Upgrade Kafka to JDK 21 > 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below) > 3. Update the README (see[4]) to mention Kafka's support for JDK-21 > [1] [https://github.com/apache/kafka/pull/13840] > [2] [https://github.com/apache/kafka/pull/13582] > [3] [https://github.com/apache/kafka/pull/12948] > [4] [https://github.com/apache/kafka/pull/14061] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [kafka] ijuma commented on pull request #14433: KAFKA-15485: Support building with Java 21 (3/3)
ijuma commented on PR #14433: URL: https://github.com/apache/kafka/pull/14433#issuecomment-1732673267 Different set of failures for JDK 21: > Build / JDK 21 and Scala 2.13 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationSSLTest.testSyncTopicConfigs() > Build / JDK 21 and Scala 2.13 / kafka.api.SaslScramSslEndToEndAuthorizationTest.testAuthentications(String).quorum=zk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KAFKA-15485) Support building with Java 21 (LTS release)
[ https://issues.apache.org/jira/browse/KAFKA-15485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768450#comment-17768450 ] Said BOUDJELDA commented on KAFKA-15485: [~ijuma] [~divijvaidya] do you have more details for the KAFKA-15493 I think I can give it a try > Support building with Java 21 (LTS release) > --- > > Key: KAFKA-15485 > URL: https://issues.apache.org/jira/browse/KAFKA-15485 > Project: Kafka > Issue Type: Improvement >Reporter: Divij Vaidya >Assignee: Ismael Juma >Priority: Major > Fix For: 3.7.0 > > > JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This > ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20). > Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not > anticipate major hiccups while upgrading to JDK-21. > As part of this JIRA we want to: > 1. Upgrade Kafka to JDK 21 > 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below) > 3. Update the README (see[4]) to mention Kafka's support for JDK-21 > [1] [https://github.com/apache/kafka/pull/13840] > [2] [https://github.com/apache/kafka/pull/13582] > [3] [https://github.com/apache/kafka/pull/12948] > [4] [https://github.com/apache/kafka/pull/14061] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15495) KRaft partition truncated when the only ISR member restarts with an empty disk
[
https://issues.apache.org/jira/browse/KAFKA-15495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ron Dagostino updated KAFKA-15495:
--
Description:
Assume a topic-partition in KRaft has just a single leader replica in the ISR.
Assume next that this replica goes offline. This replica's log will define the
contents of that partition when the replica restarts, which is correct
behavior. However, assume now that the replica has a disk failure, and we then
replace the failed disk with a new, empty disk that we also format with the
storage tool so it has the correct cluster ID. If we then restart the broker,
the topic-partition will have no data in it, and any other replicas that might
exist will truncate their logs to match, which results in data loss. See below
for a step-by-step demo of how to reproduce this.
[KIP-858: Handle JBOD broker disk failure in
KRaft|https://cwiki.apache.org/confluence/display/KAFKA/KIP-858%3A+Handle+JBOD+broker+disk+failure+in+KRaft]
introduces the concept of a Disk UUID that we can use to solve this problem.
Specifically, when the leader restarts with an empty (but correctly-formatted)
disk, the actual UUID associated with the disk will be different. The
controller will notice upon broker re-registration that its disk UUID differs
from what was previously registered. Right now we have no way of detecting
this situation, but the disk UUID gives us that capability.
STEPS TO REPRODUCE:
Create a single broker cluster with single controller. The standard files
under config/kraft work well:
bin/kafka-storage.sh random-uuid
J8qXRwI-Qyi2G0guFTiuYw
#ensure we start clean
/bin/rm -rf /tmp/kraft-broker-logs /tmp/kraft-controller-logs
bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config
config/kraft/controller.properties
bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config
config/kraft/broker.properties
bin/kafka-server-start.sh config/kraft/controller.properties
bin/kafka-server-start.sh config/kraft/broker.properties
bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic foo1
--partitions 1 --replication-factor 1
#create __consumer-offsets topics
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic foo1
--from-beginning
^C
#confirm that __consumer_offsets topic partitions are all created and on broker
with node id 2
bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe
Now create 2 more brokers, with node IDs 11 and 12
cat config/kraft/broker.properties | sed 's/node.id=2/node.id=11/' | sed
's/localhost:9092/localhost:9011/g' | sed
's#log.dirs=/tmp/kraft-broker-logs#log.dirs=/tmp/kraft-broker-logs11#' >
config/kraft/broker11.properties
cat config/kraft/broker.properties | sed 's/node.id=2/node.id=12/' | sed
's/localhost:9092/localhost:9012/g' | sed
's#log.dirs=/tmp/kraft-broker-logs#log.dirs=/tmp/kraft-broker-logs12#' >
config/kraft/broker12.properties
#ensure we start clean
/bin/rm -rf /tmp/kraft-broker-logs11 /tmp/kraft-broker-logs12
bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config
config/kraft/broker11.properties
bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config
config/kraft/broker12.properties
bin/kafka-server-start.sh config/kraft/broker11.properties
bin/kafka-server-start.sh config/kraft/broker12.properties
#create a topic with a single partition replicated on two brokers
bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic foo2
--partitions 1 --replication-factor 2
#reassign partitions onto brokers with Node IDs 11 and 12
echo '{"partitions":[{"topic": "foo2","partition": 0,"replicas": [11,12]}],
"version":1}' > /tmp/reassign.json
bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092
--reassignment-json-file /tmp/reassign.json --execute
bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092
--reassignment-json-file /tmp/reassign.json --verify
#make preferred leader 11 the actual leader if it not
bin/kafka-leader-election.sh --bootstrap-server localhost:9092
--all-topic-partitions --election-type preferred
#Confirm both brokers are in ISR and 11 is the leader
bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic foo2
Topic: foo2 TopicId: pbbQZ23UQ5mQqmZpoSRCLQ PartitionCount: 1
ReplicationFactor: 2Configs: segment.bytes=1073741824
Topic: foo2 Partition: 0Leader: 11 Replicas: 11,12 Isr:
12,11
#Emit some messages to the topic
bin/kafka-console-producer.sh --bootstrap-server localhost:9092 --topic foo2
1
2
3
4
5
^C
#confirm we see the messages
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic foo2
--from-beginning
1
2
3
4
5
^C
#Again confirm both brokers are in ISR, leader is 11
bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic foo2
Topic: foo2 TopicId: pbbQZ23UQ5mQqmZpoSRCL
[jira] [Updated] (KAFKA-15495) KRaft partition truncated when the only ISR member restarts with an empty disk
[ https://issues.apache.org/jira/browse/KAFKA-15495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Dagostino updated KAFKA-15495: -- Description: Assume a topic-partition in KRaft has just a single leader replica in the ISR. Assume next that this replica goes offline. This replica's log will define the contents of that partition when the replica restarts, which is correct behavior. However, assume now that the replica has a disk failure, and we then replace the failed disk with a new, empty disk that we also format with the storage tool so it has the correct cluster ID. If we then restart the broker, the topic-partition will have no data in it, and any other replicas that might exist will truncate their logs to match, which results in data loss. See below for a step-by-step demo of how to reproduce this. [KIP-858: Handle JBOD broker disk failure in KRaft|https://cwiki.apache.org/confluence/display/KAFKA/KIP-858%3A+Handle+JBOD+broker+disk+failure+in+KRaft] introduces the concept of a Disk UUID that we can use to solve this problem. Specifically, when the leader restarts with an empty (but correctly-formatted) disk, the actual UUID associated with the disk will be different. The controller will notice upon broker re-registration that its disk UUID differs from what was previously registered. Right now we have no way of detecting this situation, but the disk UUID gives us that capability. STEPS TO REPRODUCE: Create a single broker cluster with single controller. The standard files under config/kraft work well: bin/kafka-storage.sh random-uuid J8qXRwI-Qyi2G0guFTiuYw # ensure we start clean /bin/rm -rf /tmp/kraft-broker-logs /tmp/kraft-controller-logs bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config config/kraft/controller.properties bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config config/kraft/broker.properties bin/kafka-server-start.sh config/kraft/controller.properties bin/kafka-server-start.sh config/kraft/broker.properties bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic foo1 --partitions 1 --replication-factor 1 # create __consumer-offsets topics bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic foo1 --from-beginning ^C # confirm that __consumer_offsets topic partitions are all created and on broker with node id 2 bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe Now create 2 more brokers, with node IDs 11 and 12 cat config/kraft/broker.properties | sed 's/node.id=2/node.id=11/' | sed 's/localhost:9092/localhost:9011/g' | sed 's#log.dirs=/tmp/kraft-broker-logs#log.dirs=/tmp/kraft-broker-logs11#' > config/kraft/broker11.properties cat config/kraft/broker.properties | sed 's/node.id=2/node.id=12/' | sed 's/localhost:9092/localhost:9012/g' | sed 's#log.dirs=/tmp/kraft-broker-logs#log.dirs=/tmp/kraft-broker-logs12#' > config/kraft/broker12.properties # ensure we start clean /bin/rm -rf /tmp/kraft-broker-logs11 /tmp/kraft-broker-logs12 bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config config/kraft/broker11.properties bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config config/kraft/broker12.properties bin/kafka-server-start.sh config/kraft/broker11.properties bin/kafka-server-start.sh config/kraft/broker12.properties # create a topic with a single partition replicated on two brokers bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic foo2 --partitions 1 --replication-factor 2 # reassign partitions onto brokers with Node IDs 11 and 12 cat > /tmp/reassign.json
[GitHub] [kafka] yashmayya commented on a diff in pull request #14093: KAFKA-15248 Add BooleanConverter
yashmayya commented on code in PR #14093:
URL: https://github.com/apache/kafka/pull/14093#discussion_r1335220167
##
connect/runtime/src/main/java/org/apache/kafka/connect/converters/BooleanConverter.java:
##
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.connect.converters;
+
+import java.util.HashMap;
+import java.util.Map;
+import org.apache.kafka.common.config.ConfigDef;
+import org.apache.kafka.common.errors.SerializationException;
+import org.apache.kafka.common.serialization.BooleanDeserializer;
+import org.apache.kafka.common.serialization.BooleanSerializer;
+import org.apache.kafka.common.utils.Utils;
+import org.apache.kafka.connect.data.Schema;
+import org.apache.kafka.connect.data.Schema.Type;
+import org.apache.kafka.connect.data.SchemaAndValue;
+import org.apache.kafka.connect.errors.DataException;
+import org.apache.kafka.connect.storage.Converter;
+import org.apache.kafka.connect.storage.ConverterConfig;
+import org.apache.kafka.connect.storage.ConverterType;
+import org.apache.kafka.connect.storage.HeaderConverter;
+
+/**
+ * {@link Converter} and {@link HeaderConverter} implementation that supports
serializing to and
+ * deserializing from Boolean values.
+ *
+ * When converting from bytes to Kafka Connect format, the converter will
always return an optional
+ * BOOLEAN schema.
+ */
+public class BooleanConverter implements Converter, HeaderConverter {
Review Comment:
In https://issues.apache.org/jira/browse/KAFKA-15291 /
https://github.com/apache/kafka/pull/14159, all the built-in converters were
updated to implement the `Versioned` interface. Could we do the same here?
##
connect/runtime/src/main/java/org/apache/kafka/connect/converters/BooleanConverter.java:
##
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.connect.converters;
+
+import java.util.HashMap;
+import java.util.Map;
+import org.apache.kafka.common.config.ConfigDef;
+import org.apache.kafka.common.errors.SerializationException;
+import org.apache.kafka.common.serialization.BooleanDeserializer;
+import org.apache.kafka.common.serialization.BooleanSerializer;
+import org.apache.kafka.common.utils.Utils;
+import org.apache.kafka.connect.data.Schema;
+import org.apache.kafka.connect.data.Schema.Type;
+import org.apache.kafka.connect.data.SchemaAndValue;
+import org.apache.kafka.connect.errors.DataException;
+import org.apache.kafka.connect.storage.Converter;
+import org.apache.kafka.connect.storage.ConverterConfig;
+import org.apache.kafka.connect.storage.ConverterType;
+import org.apache.kafka.connect.storage.HeaderConverter;
+
+/**
+ * {@link Converter} and {@link HeaderConverter} implementation that supports
serializing to and
+ * deserializing from Boolean values.
+ *
+ * When converting from bytes to Kafka Connect format, the converter will
always return an optional
+ * BOOLEAN schema.
+ */
+public class BooleanConverter implements Converter, HeaderConverter {
+
+private static final ConfigDef CONFIG_DEF = ConverterConfig.newConfigDef();
+
+private final BooleanSerializer serializer = new BooleanSerializer();
+private final BooleanDeserializer deserializer = new BooleanDeserializer();
+
+@Override
+public ConfigDef config() {
+return CONFIG_DEF;
Review Comment:
I think it makes more sense to use `BooleanConverterConfig.configDef()` here
instead of re
[GitHub] [kafka] ijuma commented on pull request #14433: KAFKA-15485: Support building with Java 21 (3/3)
ijuma commented on PR #14433: URL: https://github.com/apache/kafka/pull/14433#issuecomment-1732624885 Two failures with JDK 21: > Build / JDK 21 and Scala 2.13 / kafka.zk.ZkMigrationIntegrationTest.[1] Type=ZK, MetadataVersion=3.4-IV0, Security=PLAINTEXT > Build / JDK 21 and Scala 2.13 / org.apache.kafka.streams.integration.NamedTopologyIntegrationTest.shouldRemoveOneNamedTopologyWhileAnotherContinuesProcessing() They seem flaky, kicked off another build. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (KAFKA-15495) KRaft partition truncated when the only ISR member restarts with an empty disk
[ https://issues.apache.org/jira/browse/KAFKA-15495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Dagostino updated KAFKA-15495: -- Description: Assume a topic-partition in KRaft has just a single leader replica in the ISR. Assume next that this replica goes offline. This replica's log will define the contents of that partition when the replica restarts, which is correct behavior. However, assume now that the replica has a disk failure, and we then replace the failed disk with a new, empty disk that we also format with the storage tool so it has the correct cluster ID. If we then restart the broker, the topic-partition will have no data in it, and any other replicas that might exist will truncate their logs to match, which results in data loss. See below for a step-by-step demo of how to reproduce this. [KIP-858: Handle JBOD broker disk failure in KRaft|https://cwiki.apache.org/confluence/display/KAFKA/KIP-858%3A+Handle+JBOD+broker+disk+failure+in+KRaft] introduces the concept of a Disk UUID that we can use to solve this problem. Specifically, when the leader restarts with an empty (but correctly-formatted) disk, the actual UUID associated with the disk will be different. The controller will notice upon broker re-registration that its disk UUID differs from what was previously registered. Right now we have no way of detecting this situation, but the disk UUID gives us that capability. STEPS TO REPRODUCE: Create a single broker cluster with single controller. The standard files under config/kraft work well: bin/kafka-storage.sh random-uuid J8qXRwI-Qyi2G0guFTiuYw # ensure we start clean /bin/rm -rf /tmp/kraft-broker-logs /tmp/kraft-controller-logs bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config config/kraft/controller.properties bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config config/kraft/broker.properties bin/kafka-server-start.sh config/kraft/controller.properties bin/kafka-server-start.sh config/kraft/broker.properties bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic foo1 --partitions 1 --replication-factor 1 # create __consumer-offsets topics bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic foo1 --from-beginning ^C # confirm that __consumer_offsets topic partitions are all created and on broker with node id 2 bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe Now create 2 more brokers, with node IDs 3 and 4 cat config/kraft/broker.properties | sed 's/node.id=2/node.id=11/' | sed 's/localhost:9092/localhost:9011/g' | sed 's#log.dirs=/tmp/kraft-broker-logs#log.dirs=/tmp/kraft-broker-logs11#' > config/kraft/broker11.properties cat config/kraft/broker.properties | sed 's/node.id=2/node.id=12/' | sed 's/localhost:9092/localhost:9012/g' | sed 's#log.dirs=/tmp/kraft-broker-logs#log.dirs=/tmp/kraft-broker-logs12#' > config/kraft/broker12.properties # ensure we start clean /bin/rm -rf /tmp/kraft-broker-logs11 /tmp/kraft-broker-logs12 bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config config/kraft/broker11.properties bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config config/kraft/broker12.properties bin/kafka-server-start.sh config/kraft/broker11.properties bin/kafka-server-start.sh config/kraft/broker12.properties # create a topic with a single partition replicated on two brokers bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic foo2 --partitions 1 --replication-factor 2 # reassign partitions onto brokers with Node IDs 11 and 12 cat > /tmp/reassign.json
[jira] [Updated] (KAFKA-15495) KRaft partition truncated when the only ISR member restarts with an empty disk
[
https://issues.apache.org/jira/browse/KAFKA-15495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ron Dagostino updated KAFKA-15495:
--
Summary: KRaft partition truncated when the only ISR member restarts with
an empty disk (was: KRaft partition truncated when the only ISR member
restarts with and empty disk)
> KRaft partition truncated when the only ISR member restarts with an empty disk
> --
>
> Key: KAFKA-15495
> URL: https://issues.apache.org/jira/browse/KAFKA-15495
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 3.3.2, 3.4.1, 3.6.0, 3.5.1
>Reporter: Ron Dagostino
>Priority: Critical
>
> Assume a topic-partition in KRaft has just a single leader replica in the
> ISR. Assume next that this replica goes offline. This replica's log will
> define the contents of that partition when the replica restarts, which is
> correct behavior. However, assume now that the replica has a disk failure,
> and we then replace the failed disk with a new, empty disk that we also
> format with the storage tool so it has the correct cluster ID. If we then
> restart the broker, the topic-partition will have no data in it, and any
> other replicas that might exist will truncate their logs to match. See below
> for a step-by-step demo of how to reproduce this.
> [KIP-858: Handle JBOD broker disk failure in
> KRaft|https://cwiki.apache.org/confluence/display/KAFKA/KIP-858%3A+Handle+JBOD+broker+disk+failure+in+KRaft]
> introduces the concept of a Disk UUID that we can use to solve this problem.
> Specifically, when the leader restarts with an empty (but
> correctly-formatted) disk, the actual UUID associated with the disk will be
> different. The controller will notice upon broker re-registration that its
> disk UUID differs from what was previously registered. Right now we have no
> way of detecting this situation, but the disk UUID gives us that capability.
> STEPS TO REPRODUCE:
> Create a single broker cluster with single controller. The standard files
> under config/kraft work well:
> bin/kafka-storage.sh random-uuid
> J8qXRwI-Qyi2G0guFTiuYw
> # ensure we start clean
> /bin/rm -rf /tmp/kraft-broker-logs /tmp/kraft-controller-logs
> bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config
> config/kraft/controller.properties
> bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config
> config/kraft/broker.properties
> bin/kafka-server-start.sh config/kraft/controller.properties
> bin/kafka-server-start.sh config/kraft/broker.properties
> bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic foo1
> --partitions 1 --replication-factor 1
> # create __consumer-offsets topics
> bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic foo1
> --from-beginning
> ^C
> # confirm that __consumer_offsets topic partitions are all created and on
> broker with node id 2
> bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe
> Now create 2 more brokers, with node IDs 3 and 4
> cat config/kraft/broker.properties | sed 's/node.id=2/node.id=11/' | sed
> 's/localhost:9092/localhost:9011/g' | sed
> 's#log.dirs=/tmp/kraft-broker-logs#log.dirs=/tmp/kraft-broker-logs11#' >
> config/kraft/broker11.properties
> cat config/kraft/broker.properties | sed 's/node.id=2/node.id=12/' | sed
> 's/localhost:9092/localhost:9012/g' | sed
> 's#log.dirs=/tmp/kraft-broker-logs#log.dirs=/tmp/kraft-broker-logs12#' >
> config/kraft/broker12.properties
> # ensure we start clean
> /bin/rm -rf /tmp/kraft-broker-logs11 /tmp/kraft-broker-logs12
> bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config
> config/kraft/broker11.properties
> bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config
> config/kraft/broker12.properties
> bin/kafka-server-start.sh config/kraft/broker11.properties
> bin/kafka-server-start.sh config/kraft/broker12.properties
> # create a topic with a single partition replicated on two brokers
> bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic foo2
> --partitions 1 --replication-factor 2
> # reassign partitions onto brokers with Node IDs 11 and 12
> cat > /tmp/reassign.json < {"partitions":[{"topic": "foo2","partition": 0,"replicas": [11,12]}],
> "version":1}
> DONE
> bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092
> --reassignment-json-file /tmp/reassign.json --execute
> bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092
> --reassignment-json-file /tmp/reassign.json --verify
> # make preferred leader 11 the actual leader if it not
> bin/kafka-leader-election.sh --bootstrap-server localhost:9092
> --all-topic-partitions --election-type preferred
> # Confirm both brokers are in ISR and 11 is the leader
> bin/kafka-topic
[jira] [Created] (KAFKA-15495) KRaft partition truncated when the only ISR member restarts with and empty disk
Ron Dagostino created KAFKA-15495: - Summary: KRaft partition truncated when the only ISR member restarts with and empty disk Key: KAFKA-15495 URL: https://issues.apache.org/jira/browse/KAFKA-15495 Project: Kafka Issue Type: Bug Affects Versions: 3.5.1, 3.4.1, 3.3.2, 3.6.0 Reporter: Ron Dagostino Assume a topic-partition in KRaft has just a single leader replica in the ISR. Assume next that this replica goes offline. This replica's log will define the contents of that partition when the replica restarts, which is correct behavior. However, assume now that the replica has a disk failure, and we then replace the failed disk with a new, empty disk that we also format with the storage tool so it has the correct cluster ID. If we then restart the broker, the topic-partition will have no data in it, and any other replicas that might exist will truncate their logs to match. See below for a step-by-step demo of how to reproduce this. [KIP-858: Handle JBOD broker disk failure in KRaft|https://cwiki.apache.org/confluence/display/KAFKA/KIP-858%3A+Handle+JBOD+broker+disk+failure+in+KRaft] introduces the concept of a Disk UUID that we can use to solve this problem. Specifically, when the leader restarts with an empty (but correctly-formatted) disk, the actual UUID associated with the disk will be different. The controller will notice upon broker re-registration that its disk UUID differs from what was previously registered. Right now we have no way of detecting this situation, but the disk UUID gives us that capability. STEPS TO REPRODUCE: Create a single broker cluster with single controller. The standard files under config/kraft work well: bin/kafka-storage.sh random-uuid J8qXRwI-Qyi2G0guFTiuYw # ensure we start clean /bin/rm -rf /tmp/kraft-broker-logs /tmp/kraft-controller-logs bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config config/kraft/controller.properties bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config config/kraft/broker.properties bin/kafka-server-start.sh config/kraft/controller.properties bin/kafka-server-start.sh config/kraft/broker.properties bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic foo1 --partitions 1 --replication-factor 1 # create __consumer-offsets topics bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic foo1 --from-beginning ^C # confirm that __consumer_offsets topic partitions are all created and on broker with node id 2 bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe Now create 2 more brokers, with node IDs 3 and 4 cat config/kraft/broker.properties | sed 's/node.id=2/node.id=11/' | sed 's/localhost:9092/localhost:9011/g' | sed 's#log.dirs=/tmp/kraft-broker-logs#log.dirs=/tmp/kraft-broker-logs11#' > config/kraft/broker11.properties cat config/kraft/broker.properties | sed 's/node.id=2/node.id=12/' | sed 's/localhost:9092/localhost:9012/g' | sed 's#log.dirs=/tmp/kraft-broker-logs#log.dirs=/tmp/kraft-broker-logs12#' > config/kraft/broker12.properties # ensure we start clean /bin/rm -rf /tmp/kraft-broker-logs11 /tmp/kraft-broker-logs12 bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config config/kraft/broker11.properties bin/kafka-storage.sh format --cluster-id J8qXRwI-Qyi2G0guFTiuYw --config config/kraft/broker12.properties bin/kafka-server-start.sh config/kraft/broker11.properties bin/kafka-server-start.sh config/kraft/broker12.properties # create a topic with a single partition replicated on two brokers bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic foo2 --partitions 1 --replication-factor 2 # reassign partitions onto brokers with Node IDs 11 and 12 cat > /tmp/reassign.json <
[jira] (KAFKA-15169) Add tests for RemoteIndexCache
[ https://issues.apache.org/jira/browse/KAFKA-15169 ] Arpit Goyal deleted comment on KAFKA-15169: - was (Author: JIRAUSER301926): [~divijvaidya] In the first test case Step no 3 Do we support this functionality fetchAndCreateIndex ? As per the codebase there is only getIndexEntry function which calls remote storage only if there is a miss ? > Add tests for RemoteIndexCache > -- > > Key: KAFKA-15169 > URL: https://issues.apache.org/jira/browse/KAFKA-15169 > Project: Kafka > Issue Type: Test >Reporter: Satish Duggana >Assignee: Arpit Goyal >Priority: Major > Labels: KIP-405 > Fix For: 3.7.0 > > > Follow-up from > https://github.com/apache/kafka/pull/13275#discussion_r1257490978 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15494) Remove deprecated calls in build.gradle for preparing future upgrades
[
https://issues.apache.org/jira/browse/KAFKA-15494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Said BOUDJELDA updated KAFKA-15494:
---
Description:
On the purpose of preparing the future upgrades of {*}Gradle wrapper{*}, we
need to get ride of the deprecated calls of certains methods as tasks
registrations, this will make the future upgrades to *Gradle 9* where plenty of
methods are marked for removal for this future big release, that will for sure
brings support for *JDK 21*
Running *Gradle* build with *--warning-mode all* reveals much warnings and much
deprecations for removal, and we have to be prepared from now to the future
changes
This is an example of deprecation warning
{code:java}
Build file '/Users/bmscomp/codes/kafka/build.gradle': line 3116
The org.gradle.api.plugins.Convention type has been deprecated. This is
scheduled to be removed in Gradle 9.0. Consult the upgrading guide for further
information:
https://docs.gradle.org/8.2.1/userguide/upgrading_version_8.html#deprecated_access_to_conventions
at
build_bpyr04xfzz0tpxxyqu97xn8xy$_run_closure58.doCall(/Users/bmscomp/codes/kafka/build.gradle:3116)
(Run with --stacktrace to get the full stack trace of this deprecation
warning.)
at
build_bpyr04xfzz0tpxxyqu97xn8xy.run(/Users/bmscomp/codes/kafka/build.gradle:3115)
(Run with --stacktrace to get the full stack trace of this deprecation
warning.) {code}
was:
On the purpose of preparing the future upgrades of {*}Gradle wrapper{*}, we
need to get ride of the deprecated calls of certains methods as tasks
registrations, this will make the future upgrades to *Gradle 9* where plenty of
methods are marked for removal for this future big release, that will for sure
brings support for *JDK 21*
Running *Gradle* build with *--warning-mode all* reveals much warnings and much
deprecations for removal, and we have to be prepared from now to the future
changes
> Remove deprecated calls in build.gradle for preparing future upgrades
> -
>
> Key: KAFKA-15494
> URL: https://issues.apache.org/jira/browse/KAFKA-15494
> Project: Kafka
> Issue Type: Improvement
>Reporter: Said BOUDJELDA
>Assignee: Said BOUDJELDA
>Priority: Major
>
> On the purpose of preparing the future upgrades of {*}Gradle wrapper{*}, we
> need to get ride of the deprecated calls of certains methods as tasks
> registrations, this will make the future upgrades to *Gradle 9* where plenty
> of methods are marked for removal for this future big release, that will for
> sure brings support for *JDK 21*
>
> Running *Gradle* build with *--warning-mode all* reveals much warnings and
> much deprecations for removal, and we have to be prepared from now to the
> future changes
>
>
> This is an example of deprecation warning
>
> {code:java}
> Build file '/Users/bmscomp/codes/kafka/build.gradle': line 3116
> The org.gradle.api.plugins.Convention type has been deprecated. This is
> scheduled to be removed in Gradle 9.0. Consult the upgrading guide for
> further information:
> https://docs.gradle.org/8.2.1/userguide/upgrading_version_8.html#deprecated_access_to_conventions
> at
> build_bpyr04xfzz0tpxxyqu97xn8xy$_run_closure58.doCall(/Users/bmscomp/codes/kafka/build.gradle:3116)
> (Run with --stacktrace to get the full stack trace of this
> deprecation warning.)
> at
> build_bpyr04xfzz0tpxxyqu97xn8xy.run(/Users/bmscomp/codes/kafka/build.gradle:3115)
> (Run with --stacktrace to get the full stack trace of this
> deprecation warning.) {code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (KAFKA-15494) Remove deprecated calls in build.gradle for preparing future upgrades
[
https://issues.apache.org/jira/browse/KAFKA-15494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Said BOUDJELDA updated KAFKA-15494:
---
Description:
On the purpose of preparing the future upgrades of {*}Gradle wrapper{*}, we
need to get ride of the deprecated calls of certains methods as tasks
registrations, this will make the future upgrades to *Gradle 9* where plenty of
methods are marked for removal for this future big release, that will for sure
brings support for *JDK 21*
Running *Gradle* build with *--warning-mode all* reveals much warnings and much
deprecations for removal, and we have to be prepared from now to the future
changes
was:
On the purpose of preparing the future upgrades of Gradle wrapper, we need to
get ride of the deprecated calls of certains methods as tasks registrations,
this will make the future upgrades to *Gradle 9* where plenty of methods are
marked for removal for this future big release, that will for sure brings
support for *JDK 21*
For now trying to upgrade to Gradle 8.3 shows more deprecations then previous
version
> Remove deprecated calls in build.gradle for preparing future upgrades
> -
>
> Key: KAFKA-15494
> URL: https://issues.apache.org/jira/browse/KAFKA-15494
> Project: Kafka
> Issue Type: Improvement
>Reporter: Said BOUDJELDA
>Assignee: Said BOUDJELDA
>Priority: Major
>
> On the purpose of preparing the future upgrades of {*}Gradle wrapper{*}, we
> need to get ride of the deprecated calls of certains methods as tasks
> registrations, this will make the future upgrades to *Gradle 9* where plenty
> of methods are marked for removal for this future big release, that will for
> sure brings support for *JDK 21*
>
> Running *Gradle* build with *--warning-mode all* reveals much warnings and
> much deprecations for removal, and we have to be prepared from now to the
> future changes
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (KAFKA-15494) Remove deprecated calls in build.gradle for preparing future upgrades
[ https://issues.apache.org/jira/browse/KAFKA-15494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Said BOUDJELDA updated KAFKA-15494: --- Summary: Remove deprecated calls in build.gradle for preparing future upgrades (was: Use tasks.register instead of task create in build.gradle) > Remove deprecated calls in build.gradle for preparing future upgrades > - > > Key: KAFKA-15494 > URL: https://issues.apache.org/jira/browse/KAFKA-15494 > Project: Kafka > Issue Type: Improvement >Reporter: Said BOUDJELDA >Assignee: Said BOUDJELDA >Priority: Major > > On the purpose of preparing the future upgrades of Gradle wrapper, we need to > get ride of the deprecated calls of certains methods as tasks registrations, > this will make the future upgrades to *Gradle 9* where plenty of methods are > marked for removal for this future big release, that will for sure brings > support for *JDK 21* > > For now trying to upgrade to Gradle 8.3 shows more deprecations then previous > version > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15494) Use tasks.register instead of task create in build.gradle
[ https://issues.apache.org/jira/browse/KAFKA-15494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Said BOUDJELDA updated KAFKA-15494: --- Description: On the purpose of preparing the future upgrades of Gradle wrapper, we need to get ride of the deprecated calls of certains methods as tasks registrations, this will make the future upgrades to *Gradle 9* where plenty of methods are marked for removal for this future big release, that will for sure brings support for *JDK 21* For now trying to upgrade to Gradle 8.3 shows more deprecations then previous version was: On the purpose of preparing the future upgrades of Gradle wrapper, we need to get ride of the deprecated calls of certains methods as tasks registrations, this will make the future upgrades to *Gradle 9* where plenty of methods are marked for removal for this future big release, that will for sure brings support for *JDK 21* > Use tasks.register instead of task create in build.gradle > - > > Key: KAFKA-15494 > URL: https://issues.apache.org/jira/browse/KAFKA-15494 > Project: Kafka > Issue Type: Improvement >Reporter: Said BOUDJELDA >Assignee: Said BOUDJELDA >Priority: Major > > On the purpose of preparing the future upgrades of Gradle wrapper, we need to > get ride of the deprecated calls of certains methods as tasks registrations, > this will make the future upgrades to *Gradle 9* where plenty of methods are > marked for removal for this future big release, that will for sure brings > support for *JDK 21* > > For now trying to upgrade to Gradle 8.3 shows more deprecations then previous > version > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15494) Use tasks.register instead of task create in build.gradle
[ https://issues.apache.org/jira/browse/KAFKA-15494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Said BOUDJELDA updated KAFKA-15494: --- Description: On the purpose of preparing the future upgrades of Gradle wrapper, we need to get ride of the deprecated calls of certains methods as tasks registrations, this will make the future upgrades to *Gradle 9* where plenty of methods are marked for removal for this future big release, that will for sure brings support for *JDK 21* For now trying to update was: On the purpose of preparing the future upgrades of Gradle wrapper, we need to get ride of the deprecated calls of certains methods as tasks registrations, this will make the future upgrades to *Gradle 9* where plenty of methods are marked for removal for this future big release, that will for sure brings support for *JDK 21* > Use tasks.register instead of task create in build.gradle > - > > Key: KAFKA-15494 > URL: https://issues.apache.org/jira/browse/KAFKA-15494 > Project: Kafka > Issue Type: Improvement >Reporter: Said BOUDJELDA >Assignee: Said BOUDJELDA >Priority: Major > > On the purpose of preparing the future upgrades of Gradle wrapper, we need to > get ride of the deprecated calls of certains methods as tasks registrations, > this will make the future upgrades to *Gradle 9* where plenty of methods are > marked for removal for this future big release, that will for sure brings > support for *JDK 21* > > For now trying to update > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15494) Use tasks.register instead of task create in build.gradle
[ https://issues.apache.org/jira/browse/KAFKA-15494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Said BOUDJELDA updated KAFKA-15494: --- Description: On the purpose of preparing the future upgrades of Gradle wrapper, we need to get ride of the deprecated calls of certains methods as tasks registrations, this will make the future upgrades to *Gradle 9* where plenty of methods are marked for removal for this future big release, that will for sure brings support for *JDK 21* was: On the purpose of preparing the future upgrades of Gradle wrapper, we need to get ride of the deprecated calls of certains methods as tasks registrations, this will make the future upgrades to *Gradle 9* where plenty of methods are marked for removal for this future big release, that will for sure brings support for *JDK 21* For now trying to update > Use tasks.register instead of task create in build.gradle > - > > Key: KAFKA-15494 > URL: https://issues.apache.org/jira/browse/KAFKA-15494 > Project: Kafka > Issue Type: Improvement >Reporter: Said BOUDJELDA >Assignee: Said BOUDJELDA >Priority: Major > > On the purpose of preparing the future upgrades of Gradle wrapper, we need to > get ride of the deprecated calls of certains methods as tasks registrations, > this will make the future upgrades to *Gradle 9* where plenty of methods are > marked for removal for this future big release, that will for sure brings > support for *JDK 21* > > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15494) Use tasks.register instead of task create in build.gradle
[ https://issues.apache.org/jira/browse/KAFKA-15494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Said BOUDJELDA updated KAFKA-15494: --- Summary: Use tasks.register instead of task create in build.gradle (was: Remove deprecation call of task registration of Gradle on build.gradle and use tasks.register instead) > Use tasks.register instead of task create in build.gradle > - > > Key: KAFKA-15494 > URL: https://issues.apache.org/jira/browse/KAFKA-15494 > Project: Kafka > Issue Type: Improvement >Reporter: Said BOUDJELDA >Assignee: Said BOUDJELDA >Priority: Major > > On the purpose of preparing the future upgrades of Gradle wrapper, we need to > get ride of the deprecated calls of certains methods as tasks registrations, > this will make the future upgrades to *Gradle 9* where plenty of methods are > marked for removal for this future big release, that will for sure brings > support for *JDK 21* > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15494) Remove deprecation call of task registration of Gradle on build.gradle and use tasks.register instead
[
https://issues.apache.org/jira/browse/KAFKA-15494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Said BOUDJELDA updated KAFKA-15494:
---
Description:
On the purpose of preparing the future upgrades of Gradle wrapper, we need to
get ride of the deprecated calls of certains methods as tasks registrations,
this will make the future upgrades to *Gradle 9* where plenty of methods are
marked for removal for this future big release, that will for sure brings
support for *JDK 21*
was:
On the purpose of preparing the future upgrades of Gradle wrapper, we need to
get ride of the deprecated calls of certains methods as tasks registrations,
this will make the future upgrades to *Gradle 9* where plenty of methods are
marked for removal for this future big release, that will for sure brings
support for *JDK 21*
an exemple of deprecated usage of task registration
{code:java}
task aggregatedJavadoc(type: Javadoc, dependsOn: compileJava) {
def projectsWithJavadoc = subprojects.findAll { it.javadoc.enabled }
source = projectsWithJavadoc.collect { it.sourceSets.main.allJava }
classpath = files(projectsWithJavadoc.collect {
it.sourceSets.main.compileClasspath })
includes = projectsWithJavadoc.collectMany { it.javadoc.getIncludes() }
excludes = projectsWithJavadoc.collectMany { it.javadoc.getExcludes() }
options.charSet = 'UTF-8'
options.docEncoding = 'UTF-8'
options.encoding = 'UTF-8'
// Turn off doclint for now, see
https://blog.joda.org/2014/02/turning-off-doclint-in-jdk-8-javadoc.html for
rationale
options.addStringOption('Xdoclint:none', '-quiet')
// The URL structure was changed to include the locale after Java 8
if (JavaVersion.current().isJava11Compatible())
options.links
"https://docs.oracle.com/en/java/javase/${JavaVersion.current().majorVersion}/docs/api/"
else
options.links "https://docs.oracle.com/javase/8/docs/api/";
}
{code}
> Remove deprecation call of task registration of Gradle on build.gradle and
> use tasks.register instead
> -
>
> Key: KAFKA-15494
> URL: https://issues.apache.org/jira/browse/KAFKA-15494
> Project: Kafka
> Issue Type: Improvement
>Reporter: Said BOUDJELDA
>Assignee: Said BOUDJELDA
>Priority: Major
>
> On the purpose of preparing the future upgrades of Gradle wrapper, we need to
> get ride of the deprecated calls of certains methods as tasks registrations,
> this will make the future upgrades to *Gradle 9* where plenty of methods are
> marked for removal for this future big release, that will for sure brings
> support for *JDK 21*
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KAFKA-15494) Remove deprecation call of task registration of Gradle on build.gradle and use tasks.register instead
Said BOUDJELDA created KAFKA-15494:
--
Summary: Remove deprecation call of task registration of Gradle on
build.gradle and use tasks.register instead
Key: KAFKA-15494
URL: https://issues.apache.org/jira/browse/KAFKA-15494
Project: Kafka
Issue Type: Improvement
Reporter: Said BOUDJELDA
Assignee: Said BOUDJELDA
On the purpose of preparing the future upgrades of Gradle wrapper, we need to
get ride of the deprecated calls of certains methods as tasks registrations,
this will make the future upgrades to *Gradle 9* where plenty of methods are
marked for removal for this future big release, that will for sure brings
support for *JDK 21*
an exemple of deprecated usage of task registration
{code:java}
task aggregatedJavadoc(type: Javadoc, dependsOn: compileJava) {
def projectsWithJavadoc = subprojects.findAll { it.javadoc.enabled }
source = projectsWithJavadoc.collect { it.sourceSets.main.allJava }
classpath = files(projectsWithJavadoc.collect {
it.sourceSets.main.compileClasspath })
includes = projectsWithJavadoc.collectMany { it.javadoc.getIncludes() }
excludes = projectsWithJavadoc.collectMany { it.javadoc.getExcludes() }
options.charSet = 'UTF-8'
options.docEncoding = 'UTF-8'
options.encoding = 'UTF-8'
// Turn off doclint for now, see
https://blog.joda.org/2014/02/turning-off-doclint-in-jdk-8-javadoc.html for
rationale
options.addStringOption('Xdoclint:none', '-quiet')
// The URL structure was changed to include the locale after Java 8
if (JavaVersion.current().isJava11Compatible())
options.links
"https://docs.oracle.com/en/java/javase/${JavaVersion.current().majorVersion}/docs/api/"
else
options.links "https://docs.oracle.com/javase/8/docs/api/";
}
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KAFKA-15485) Support building with Java 21 (LTS release)
[ https://issues.apache.org/jira/browse/KAFKA-15485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768387#comment-17768387 ] Ismael Juma commented on KAFKA-15485: - [~divijvaidya] I filed KAFKA-15493. I won't have time to work on that, but it would be great to do that before 3.7.0, so we can confidently claim Java 21 support. > Support building with Java 21 (LTS release) > --- > > Key: KAFKA-15485 > URL: https://issues.apache.org/jira/browse/KAFKA-15485 > Project: Kafka > Issue Type: Improvement >Reporter: Divij Vaidya >Assignee: Ismael Juma >Priority: Major > Fix For: 3.7.0 > > > JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This > ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20). > Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not > anticipate major hiccups while upgrading to JDK-21. > As part of this JIRA we want to: > 1. Upgrade Kafka to JDK 21 > 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below) > 3. Update the README (see[4]) to mention Kafka's support for JDK-21 > [1] [https://github.com/apache/kafka/pull/13840] > [2] [https://github.com/apache/kafka/pull/13582] > [3] [https://github.com/apache/kafka/pull/12948] > [4] [https://github.com/apache/kafka/pull/14061] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-15493) Ensure system tests work with Java 21
Ismael Juma created KAFKA-15493: --- Summary: Ensure system tests work with Java 21 Key: KAFKA-15493 URL: https://issues.apache.org/jira/browse/KAFKA-15493 Project: Kafka Issue Type: Improvement Reporter: Ismael Juma Fix For: 3.7.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15485) Support building with Java 21 (LTS release)
[ https://issues.apache.org/jira/browse/KAFKA-15485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-15485: Summary: Support building with Java 21 (LTS release) (was: Support building Java 21 (LTS release)) > Support building with Java 21 (LTS release) > --- > > Key: KAFKA-15485 > URL: https://issues.apache.org/jira/browse/KAFKA-15485 > Project: Kafka > Issue Type: Improvement >Reporter: Divij Vaidya >Assignee: Ismael Juma >Priority: Major > Fix For: 3.7.0 > > > JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This > ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20). > Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not > anticipate major hiccups while upgrading to JDK-21. > As part of this JIRA we want to: > 1. Upgrade Kafka to JDK 21 > 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below) > 3. Update the README (see[4]) to mention Kafka's support for JDK-21 > [1] [https://github.com/apache/kafka/pull/13840] > [2] [https://github.com/apache/kafka/pull/13582] > [3] [https://github.com/apache/kafka/pull/12948] > [4] [https://github.com/apache/kafka/pull/14061] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15485) Support building Java 21 (LTS release)
[ https://issues.apache.org/jira/browse/KAFKA-15485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-15485: Summary: Support building Java 21 (LTS release) (was: Support Java 21 (LTS release)) > Support building Java 21 (LTS release) > -- > > Key: KAFKA-15485 > URL: https://issues.apache.org/jira/browse/KAFKA-15485 > Project: Kafka > Issue Type: Improvement >Reporter: Divij Vaidya >Assignee: Ismael Juma >Priority: Major > Fix For: 3.7.0 > > > JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This > ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20). > Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not > anticipate major hiccups while upgrading to JDK-21. > As part of this JIRA we want to: > 1. Upgrade Kafka to JDK 21 > 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below) > 3. Update the README (see[4]) to mention Kafka's support for JDK-21 > [1] [https://github.com/apache/kafka/pull/13840] > [2] [https://github.com/apache/kafka/pull/13582] > [3] [https://github.com/apache/kafka/pull/12948] > [4] [https://github.com/apache/kafka/pull/14061] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [kafka] ijuma opened a new pull request, #14433: KAFKA-15485: Support Java 21 (3/3)
ijuma opened a new pull request, #14433: URL: https://github.com/apache/kafka/pull/14433 * Update CI to build with Java 21 instead of Java 20 * Disable spotbugs when building with Java 21 as it doesn't support it yet (filed KAFKA-15492 for fixing this) * Disable SslTransportLayerTest.testValidEndpointIdentificationCN with Java 21 (same as Java 20) ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (KAFKA-15485) Support Java 21 (LTS release)
[ https://issues.apache.org/jira/browse/KAFKA-15485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-15485: Summary: Support Java 21 (LTS release) (was: Upgrade to JDK-21 (LTS release)) > Support Java 21 (LTS release) > - > > Key: KAFKA-15485 > URL: https://issues.apache.org/jira/browse/KAFKA-15485 > Project: Kafka > Issue Type: Improvement >Reporter: Divij Vaidya >Assignee: Ismael Juma >Priority: Major > Fix For: 3.7.0 > > > JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This > ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20). > Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not > anticipate major hiccups while upgrading to JDK-21. > As part of this JIRA we want to: > 1. Upgrade Kafka to JDK 21 > 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below) > 3. Update the README (see[4]) to mention Kafka's support for JDK-21 > [1] [https://github.com/apache/kafka/pull/13840] > [2] [https://github.com/apache/kafka/pull/13582] > [3] [https://github.com/apache/kafka/pull/12948] > [4] [https://github.com/apache/kafka/pull/14061] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-15492) Enable spotbugs when building with Java 21
Ismael Juma created KAFKA-15492: --- Summary: Enable spotbugs when building with Java 21 Key: KAFKA-15492 URL: https://issues.apache.org/jira/browse/KAFKA-15492 Project: Kafka Issue Type: Improvement Reporter: Ismael Juma The latest version of spotbugs (4.7.3) doesn't support Java 21. In order not to delay Java 21 support, we disabled spotbugs when building with Java 21. This should be reverted once we upgrade to a version of spotbugs that supports Java 21. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15485) Upgrade to JDK-21 (LTS release)
[ https://issues.apache.org/jira/browse/KAFKA-15485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-15485: Description: JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20). Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not anticipate major hiccups while upgrading to JDK-21. As part of this JIRA we want to: 1. Upgrade Kafka to JDK 21 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below) 3. Update the README (see[4]) to mention Kafka's support for JDK-21 [1] [https://github.com/apache/kafka/pull/13840] [2] [https://github.com/apache/kafka/pull/13582] [3] [https://github.com/apache/kafka/pull/12948] [4] [https://github.com/apache/kafka/pull/14061] was: JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20). Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not anticipate major hiccups while upgrading to JDK-21. As part of this JIRA we want to: 1. Upgrade Kafka to JDK 21 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below) 3. Update the README (see[4]) to mention Kafka's support for JDK-21 [1] [https://github.com/apache/kafka/pull/13840] [2] [https://github.com/apache/kafka/pull/13582] [3] [https://github.com/apache/kafka/pull/12948] [4] [https://github.com/apache/kafka/pull/14061] > Upgrade to JDK-21 (LTS release) > --- > > Key: KAFKA-15485 > URL: https://issues.apache.org/jira/browse/KAFKA-15485 > Project: Kafka > Issue Type: Improvement >Reporter: Divij Vaidya >Assignee: Ismael Juma >Priority: Major > Fix For: 3.7.0 > > > JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This > ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20). > Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not > anticipate major hiccups while upgrading to JDK-21. > As part of this JIRA we want to: > 1. Upgrade Kafka to JDK 21 > 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below) > 3. Update the README (see[4]) to mention Kafka's support for JDK-21 > [1] [https://github.com/apache/kafka/pull/13840] > [2] [https://github.com/apache/kafka/pull/13582] > [3] [https://github.com/apache/kafka/pull/12948] > [4] [https://github.com/apache/kafka/pull/14061] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15485) Upgrade to JDK-21 (LTS release)
[ https://issues.apache.org/jira/browse/KAFKA-15485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-15485: Description: JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20). Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not anticipate major hiccups while upgrading to JDK-21. As part of this JIRA we want to: 1. Upgrade Kafka to JDK 21 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below) 3. Update the README (see[4]) to mention Kafka's support for JDK-21 [1] [https://github.com/apache/kafka/pull/13840] [2] [https://github.com/apache/kafka/pull/13582] [3] [https://github.com/apache/kafka/pull/12948] [4] [https://github.com/apache/kafka/pull/14061] was: JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20). Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not anticipate major hiccups while upgrading to JDK-21. As part of this JIRA we want to: 1. Upgrade Kafka to JDK 21 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below) 3. Update the README (see[4]) to mention Kafka's support for JDK-21 As a stretch goal for this JIRA, we want to: 1. Explore the new features for JDK-21 (like virtual threads) and create separate Jira tickets to explore their usage for Kafka [1] [https://github.com/apache/kafka/pull/13840] [2] [https://github.com/apache/kafka/pull/13582] [3] [https://github.com/apache/kafka/pull/12948] [4] [https://github.com/apache/kafka/pull/14061] > Upgrade to JDK-21 (LTS release) > --- > > Key: KAFKA-15485 > URL: https://issues.apache.org/jira/browse/KAFKA-15485 > Project: Kafka > Issue Type: Improvement >Reporter: Divij Vaidya >Assignee: Ismael Juma >Priority: Major > Fix For: 3.7.0 > > > JDK-21 is the latest LTS release which reached GA on 19th Sept 2023. This > ticket aims to upgrade JDK used by Kafka to JDK-21 (currently it's JDK20). > Thanks to proactive work done by [~ijuma] earlier [1][2][3], I do not > anticipate major hiccups while upgrading to JDK-21. > As part of this JIRA we want to: > 1. Upgrade Kafka to JDK 21 > 2. Replace the CI build for JDK 20 with JDK 21 (similar to [3] below) > 3. Update the README (see[4]) to mention Kafka's support for JDK-21 > [1] [https://github.com/apache/kafka/pull/13840] > [2] [https://github.com/apache/kafka/pull/13582] > [3] [https://github.com/apache/kafka/pull/12948] > [4] [https://github.com/apache/kafka/pull/14061] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15117) SslTransportLayerTest.testValidEndpointIdentificationCN fails with Java 20 & 21
[
https://issues.apache.org/jira/browse/KAFKA-15117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismael Juma updated KAFKA-15117:
Description:
All variations fail as seen below. These tests have been disabled when run with
Java 20 & 21 for now.
{code:java}
Gradle Test Run :clients:test > Gradle Test Executor 12 > SslTransportLayerTest
> testValidEndpointIdentificationCN(Args) > [1] tlsProtocol=TLSv1.2,
useInlinePem=false FAILED
org.opentest4j.AssertionFailedError: Channel 0 was not ready after 30
seconds ==> expected: but was:
at
app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
at
app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
at app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:211)
at
app//org.apache.kafka.common.network.NetworkTestUtils.waitForChannelReady(NetworkTestUtils.java:107)
at
app//org.apache.kafka.common.network.NetworkTestUtils.checkClientConnection(NetworkTestUtils.java:70)
at
app//org.apache.kafka.common.network.SslTransportLayerTest.verifySslConfigs(SslTransportLayerTest.java:1296)
at
app//org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(SslTransportLayerTest.java:202)
org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(Args)[2]
failed, log available in
/home/ijuma/src/kafka/clients/build/reports/testOutput/org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(Args)[2].test.stdout
Gradle Test Run :clients:test > Gradle Test Executor 12 > SslTransportLayerTest
> testValidEndpointIdentificationCN(Args) > [2] tlsProtocol=TLSv1.2,
useInlinePem=true FAILED
org.opentest4j.AssertionFailedError: Channel 0 was not ready after 30
seconds ==> expected: but was:
at
app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
at
app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
at app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:211)
at
app//org.apache.kafka.common.network.NetworkTestUtils.waitForChannelReady(NetworkTestUtils.java:107)
at
app//org.apache.kafka.common.network.NetworkTestUtils.checkClientConnection(NetworkTestUtils.java:70)
at
app//org.apache.kafka.common.network.SslTransportLayerTest.verifySslConfigs(SslTransportLayerTest.java:1296)
at
app//org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(SslTransportLayerTest.java:202)
org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(Args)[3]
failed, log available in
/home/ijuma/src/kafka/clients/build/reports/testOutput/org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(Args)[3].test.stdout
Gradle Test Run :clients:test > Gradle Test Executor 12 > SslTransportLayerTest
> testValidEndpointIdentificationCN(Args) > [3] tlsProtocol=TLSv1.3,
useInlinePem=false FAILED
org.opentest4j.AssertionFailedError: Channel 0 was not ready after 30
seconds ==> expected: but was:
at
app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
at
app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
at app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:211)
at
app//org.apache.kafka.common.network.NetworkTestUtils.waitForChannelReady(NetworkTestUtils.java:107)
at
app//org.apache.kafka.common.network.NetworkTestUtils.checkClientConnection(NetworkTestUtils.java:70)
at
app//org.apache.kafka.common.network.SslTransportLayerTest.verifySslConfigs(SslTransportLayerTest.java:1296)
at
app//org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(SslTransportLayerTest.java:202)
{code}
was:
All variations fail as seen below. These tests have been disabled when run with
Java 20 for now.
{code:java}
Gradle Test Run :clients:test > Gradle Test Executor 12 > SslTransportLayerTest
> testValidEndpointIdentificationCN(Args) > [1] tlsProtocol=TLSv1.2,
useInlinePem=false FAILED
org.opentest4j.AssertionFailedError: Channel 0 was not ready after 30
seconds ==> expected: but was:
[jira] [Updated] (KAFKA-15117) SslTransportLayerTest.testValidEndpointIdentificationCN fails with Java 20 & 21
[
https://issues.apache.org/jira/browse/KAFKA-15117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismael Juma updated KAFKA-15117:
Fix Version/s: 3.7.0
> SslTransportLayerTest.testValidEndpointIdentificationCN fails with Java 20 &
> 21
> ---
>
> Key: KAFKA-15117
> URL: https://issues.apache.org/jira/browse/KAFKA-15117
> Project: Kafka
> Issue Type: Bug
>Reporter: Ismael Juma
>Assignee: Purshotam Chauhan
>Priority: Major
> Fix For: 3.7.0
>
>
> All variations fail as seen below. These tests have been disabled when run
> with Java 20 & 21 for now.
> {code:java}
> Gradle Test Run :clients:test > Gradle Test Executor 12 >
> SslTransportLayerTest > testValidEndpointIdentificationCN(Args) > [1]
> tlsProtocol=TLSv1.2, useInlinePem=false FAILED
> org.opentest4j.AssertionFailedError: Channel 0 was not ready after 30
> seconds ==> expected: but was:
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at
> app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
> at
> app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
> at
> app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:211)
> at
> app//org.apache.kafka.common.network.NetworkTestUtils.waitForChannelReady(NetworkTestUtils.java:107)
> at
> app//org.apache.kafka.common.network.NetworkTestUtils.checkClientConnection(NetworkTestUtils.java:70)
> at
> app//org.apache.kafka.common.network.SslTransportLayerTest.verifySslConfigs(SslTransportLayerTest.java:1296)
> at
> app//org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(SslTransportLayerTest.java:202)
> org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(Args)[2]
> failed, log available in
> /home/ijuma/src/kafka/clients/build/reports/testOutput/org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(Args)[2].test.stdout
> Gradle Test Run :clients:test > Gradle Test Executor 12 >
> SslTransportLayerTest > testValidEndpointIdentificationCN(Args) > [2]
> tlsProtocol=TLSv1.2, useInlinePem=true FAILED
> org.opentest4j.AssertionFailedError: Channel 0 was not ready after 30
> seconds ==> expected: but was:
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at
> app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
> at
> app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
> at
> app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:211)
> at
> app//org.apache.kafka.common.network.NetworkTestUtils.waitForChannelReady(NetworkTestUtils.java:107)
> at
> app//org.apache.kafka.common.network.NetworkTestUtils.checkClientConnection(NetworkTestUtils.java:70)
> at
> app//org.apache.kafka.common.network.SslTransportLayerTest.verifySslConfigs(SslTransportLayerTest.java:1296)
> at
> app//org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(SslTransportLayerTest.java:202)
> org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(Args)[3]
> failed, log available in
> /home/ijuma/src/kafka/clients/build/reports/testOutput/org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(Args)[3].test.stdout
> Gradle Test Run :clients:test > Gradle Test Executor 12 >
> SslTransportLayerTest > testValidEndpointIdentificationCN(Args) > [3]
> tlsProtocol=TLSv1.3, useInlinePem=false FAILED
> org.opentest4j.AssertionFailedError: Channel 0 was not ready after 30
> seconds ==> expected: but was:
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at
> app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
> at
> app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
> at
> app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:211)
> at
> app//org.apache.kafka.common.network.NetworkTestUtils.waitForChannelReady(NetworkTestUtils.java:107)
> at
> app//org.apache.kafka.common.network.NetworkTestUtils.checkClientConnection(NetworkTestUtils.java:70)
> at
> app//org.apache.kafka.c
[jira] [Updated] (KAFKA-15117) SslTransportLayerTest.testValidEndpointIdentificationCN fails with Java 20 & 21
[
https://issues.apache.org/jira/browse/KAFKA-15117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismael Juma updated KAFKA-15117:
Summary: SslTransportLayerTest.testValidEndpointIdentificationCN fails with
Java 20 & 21 (was: SslTransportLayerTest.testValidEndpointIdentificationCN
fails with Java 20)
> SslTransportLayerTest.testValidEndpointIdentificationCN fails with Java 20 &
> 21
> ---
>
> Key: KAFKA-15117
> URL: https://issues.apache.org/jira/browse/KAFKA-15117
> Project: Kafka
> Issue Type: Bug
>Reporter: Ismael Juma
>Assignee: Purshotam Chauhan
>Priority: Major
>
>
> All variations fail as seen below. These tests have been disabled when run
> with Java 20 for now.
> {code:java}
> Gradle Test Run :clients:test > Gradle Test Executor 12 >
> SslTransportLayerTest > testValidEndpointIdentificationCN(Args) > [1]
> tlsProtocol=TLSv1.2, useInlinePem=false FAILED
> org.opentest4j.AssertionFailedError: Channel 0 was not ready after 30
> seconds ==> expected: but was:
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at
> app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
> at
> app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
> at
> app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:211)
> at
> app//org.apache.kafka.common.network.NetworkTestUtils.waitForChannelReady(NetworkTestUtils.java:107)
> at
> app//org.apache.kafka.common.network.NetworkTestUtils.checkClientConnection(NetworkTestUtils.java:70)
> at
> app//org.apache.kafka.common.network.SslTransportLayerTest.verifySslConfigs(SslTransportLayerTest.java:1296)
> at
> app//org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(SslTransportLayerTest.java:202)
> org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(Args)[2]
> failed, log available in
> /home/ijuma/src/kafka/clients/build/reports/testOutput/org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(Args)[2].test.stdout
> Gradle Test Run :clients:test > Gradle Test Executor 12 >
> SslTransportLayerTest > testValidEndpointIdentificationCN(Args) > [2]
> tlsProtocol=TLSv1.2, useInlinePem=true FAILED
> org.opentest4j.AssertionFailedError: Channel 0 was not ready after 30
> seconds ==> expected: but was:
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at
> app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
> at
> app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
> at
> app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:211)
> at
> app//org.apache.kafka.common.network.NetworkTestUtils.waitForChannelReady(NetworkTestUtils.java:107)
> at
> app//org.apache.kafka.common.network.NetworkTestUtils.checkClientConnection(NetworkTestUtils.java:70)
> at
> app//org.apache.kafka.common.network.SslTransportLayerTest.verifySslConfigs(SslTransportLayerTest.java:1296)
> at
> app//org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(SslTransportLayerTest.java:202)
> org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(Args)[3]
> failed, log available in
> /home/ijuma/src/kafka/clients/build/reports/testOutput/org.apache.kafka.common.network.SslTransportLayerTest.testValidEndpointIdentificationCN(Args)[3].test.stdout
> Gradle Test Run :clients:test > Gradle Test Executor 12 >
> SslTransportLayerTest > testValidEndpointIdentificationCN(Args) > [3]
> tlsProtocol=TLSv1.3, useInlinePem=false FAILED
> org.opentest4j.AssertionFailedError: Channel 0 was not ready after 30
> seconds ==> expected: but was:
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at
> app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
> at
> app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
> at
> app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:211)
> at
> app//org.apache.kafka.common.network.NetworkTestUtils.waitForChannelReady(NetworkTestUtils.java:107)
> at
> app//org.apache.kafk
[GitHub] [kafka] ijuma merged pull request #14430: MINOR: Update to Scala 2.13.12
ijuma merged PR #14430: URL: https://github.com/apache/kafka/pull/14430 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] ijuma merged pull request #14431: KAFKA-15485: Upgrade gradle, plugins and test libraries (2/N)
ijuma merged PR #14431: URL: https://github.com/apache/kafka/pull/14431 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #14431: KAFKA-15485: Upgrade gradle, plugins and test libraries (2/N)
ijuma commented on PR #14431: URL: https://github.com/apache/kafka/pull/14431#issuecomment-1732565485 Java 8 and 20 builds passed. Unrelated flakes for other builds below: > Build / JDK 11 and Scala 2.13 / org.apache.kafka.trogdor.coordinator.CoordinatorTest.testTaskRequestWithOldStartMsGetsUpdated() Build / JDK 17 and Scala 2.13 / org.apache.kafka.connect.mirror.integration.IdentityReplicationIntegrationTest.testSyncTopicConfigs() Build / JDK 17 and Scala 2.13 / integration.kafka.server.FetchFromFollowerIntegrationTest.testRackAwareRangeAssignor() -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] ijuma merged pull request #14427: KAFKA-15485: Fix "this-escape" compiler warnings introduced by JDK 21 (1/N)
ijuma merged PR #14427: URL: https://github.com/apache/kafka/pull/14427 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KAFKA-15169) Add tests for RemoteIndexCache
[ https://issues.apache.org/jira/browse/KAFKA-15169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768372#comment-17768372 ] Arpit Goyal commented on KAFKA-15169: - [~divijvaidya] In the first test case Step no 3 Do we support this functionality fetchAndCreateIndex ? As per the codebase there is only getIndexEntry function which calls remote storage only if there is a miss ? > Add tests for RemoteIndexCache > -- > > Key: KAFKA-15169 > URL: https://issues.apache.org/jira/browse/KAFKA-15169 > Project: Kafka > Issue Type: Test >Reporter: Satish Duggana >Assignee: Arpit Goyal >Priority: Major > Labels: KIP-405 > Fix For: 3.7.0 > > > Follow-up from > https://github.com/apache/kafka/pull/13275#discussion_r1257490978 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] (KAFKA-15169) Add tests for RemoteIndexCache
[ https://issues.apache.org/jira/browse/KAFKA-15169 ] Arpit Goyal deleted comment on KAFKA-15169: - was (Author: JIRAUSER301926): [~divijvaidya] I have started working on it . Can you help me understand what does "fetchAndCreateIndex' means . There is no such function exists in the remotestoragemanager class. Does it mean while fetching from the cache it should creates an index entry? > Add tests for RemoteIndexCache > -- > > Key: KAFKA-15169 > URL: https://issues.apache.org/jira/browse/KAFKA-15169 > Project: Kafka > Issue Type: Test >Reporter: Satish Duggana >Assignee: Arpit Goyal >Priority: Major > Labels: KIP-405 > Fix For: 3.7.0 > > > Follow-up from > https://github.com/apache/kafka/pull/13275#discussion_r1257490978 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (KAFKA-15169) Add tests for RemoteIndexCache
[ https://issues.apache.org/jira/browse/KAFKA-15169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768352#comment-17768352 ] Arpit Goyal edited comment on KAFKA-15169 at 9/24/23 10:55 AM: --- [~divijvaidya] I have started working on it . Can you help me understand what does "fetchAndCreateIndex' means . There is no such function exists in the remotestoragemanager class. Does it mean while fetching from the cache it should creates an index entry? was (Author: JIRAUSER301926): [~divijvaidya] I have started working on it . Can you help me understand what does "fetchAndCreateIndex' means . There is no such function exists in the remotestoragemanager class. > Add tests for RemoteIndexCache > -- > > Key: KAFKA-15169 > URL: https://issues.apache.org/jira/browse/KAFKA-15169 > Project: Kafka > Issue Type: Test >Reporter: Satish Duggana >Assignee: Arpit Goyal >Priority: Major > Labels: KIP-405 > Fix For: 3.7.0 > > > Follow-up from > https://github.com/apache/kafka/pull/13275#discussion_r1257490978 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KAFKA-6020) Broker side filtering
[ https://issues.apache.org/jira/browse/KAFKA-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768362#comment-17768362 ] Barak commented on KAFKA-6020: -- Same comment as [~Enzo90910] - Broker side filtering would help a lot in multiple use-cases. > Broker side filtering > - > > Key: KAFKA-6020 > URL: https://issues.apache.org/jira/browse/KAFKA-6020 > Project: Kafka > Issue Type: New Feature > Components: consumer >Reporter: Pavel Micka >Priority: Major > Labels: needs-kip > > Currently, it is not possible to filter messages on broker side. Filtering > messages on broker side is convenient for filter with very low selectivity > (one message in few thousands). In my case it means to transfer several GB of > data to consumer, throw it away, take one message and do it again... > While I understand that filtering by message body is not feasible (for > performance reasons), I propose to filter just by message key prefix. This > can be achieved even without any deserialization, as the prefix to be matched > can be passed as an array (hence the broker would do just array prefix > compare). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KAFKA-15169) Add tests for RemoteIndexCache
[ https://issues.apache.org/jira/browse/KAFKA-15169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768352#comment-17768352 ] Arpit Goyal commented on KAFKA-15169: - [~divijvaidya] I have started working on it . Can you help me understand what does "fetchAndCreateIndex' means . There is no such function exists in the remotestoragemanager class. > Add tests for RemoteIndexCache > -- > > Key: KAFKA-15169 > URL: https://issues.apache.org/jira/browse/KAFKA-15169 > Project: Kafka > Issue Type: Test >Reporter: Satish Duggana >Assignee: Arpit Goyal >Priority: Major > Labels: KIP-405 > Fix For: 3.7.0 > > > Follow-up from > https://github.com/apache/kafka/pull/13275#discussion_r1257490978 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [kafka] vamossagar12 commented on a diff in pull request #14432: KAFKA-14516: [1/N] Adding logic to not fence a static member if it sends a LeaveGroup
vamossagar12 commented on code in PR #14432:
URL: https://github.com/apache/kafka/pull/14432#discussion_r1335136107
##
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java:
##
@@ -929,12 +929,22 @@ private
CoordinatorResult consumerGr
ConsumerGroup group = getOrMaybeCreateConsumerGroup(groupId, false);
ConsumerGroupMember member = group.getOrMaybeCreateMember(memberId,
false);
-log.info("[GroupId " + groupId + "] Member " + memberId + " left the
consumer group.");
-
-List records = consumerGroupFenceMember(group, member);
+List records = new ArrayList<>();
+// The departing member is a static one. We don't need to fence this
member because it is
+// expected to come back within session timeout
+if (member.instanceId() != null) {
+log.info("Member {} with instance id {} is a static member and
will not be fenced from the group", memberId, member.instanceId());
+} else {
+log.info("[GroupId {}] Member {} left the consumer group.",
groupId, memberId);
Review Comment:
I have taken the liberty and updated the log line to use an argument based
loggers instead of the string concatenation based pattern that existed before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] vamossagar12 commented on pull request #14432: KAFKA-14516: [1/N] Adding logic to not fence a static member if it sends a LeaveGroup
vamossagar12 commented on PR #14432: URL: https://github.com/apache/kafka/pull/14432#issuecomment-1732507180 @dajac , do you think it is ok to have this small PR focussed only on static member departure? I plan to create such small PRs for other cases when a static member joins etc. Let me know if you prefer this or a single PR with all changes. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] vamossagar12 opened a new pull request, #14432: KAFKA-14516: [1/N] Adding logic to not fence a static member if it sends a LeaveGroup
vamossagar12 opened a new pull request, #14432: URL: https://github.com/apache/kafka/pull/14432 This is the first PR which aims at adding static membership to the new consumer group protocol. If focusses on not triggering a rebalance when a static member sends a ConsumerGroupHeartbeat message with an epoch indicating that it wants to leave the group. One important point to note is that in the KIP , when talking about [static membership](https://cwiki.apache.org/confluence/display/KAFKA/KIP-848%3A+The+Next+Generation+of+the+Consumer+Rebalance+Protocol#KIP848:TheNextGenerationoftheConsumerRebalanceProtocol-StaticMembership(KIP-345)), it is mentioned that a static member wants to leave the group, it needs to send a member epoch value equal to -2. I think that might not be needed as the Group Coordinator is able to figure out if the departing member is a static one or not. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
