[GitHub] [kafka] hachikuji commented on pull request #12190: KAFKA-13923; Generalize authorizer system test for kraft
hachikuji commented on PR #12190: URL: https://github.com/apache/kafka/pull/12190#issuecomment-1133482572 Successful test run here: http://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/2022-05-20--001.system-test-kafka-branch-builder--1653090157--hachikuji--KAFKA-13923--017f69bfd3/report.html. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] hachikuji opened a new pull request, #12190: KAFKA-13923; Generalize authorizer system test for kraft
hachikuji opened a new pull request, #12190: URL: https://github.com/apache/kafka/pull/12190 Now that KRaft supports an authorizer (from KIP-801), we should be covering it in our system tests. I kept around the test with KRaft and the zk `AclAuthorizer` since it seems useful in KRaft to test an implementation which uses an external source. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] rittikaadhikari commented on a diff in pull request #12005: KAFKA-13803: Refactor Leader API Access
rittikaadhikari commented on code in PR #12005:
URL: https://github.com/apache/kafka/pull/12005#discussion_r878592964
##
core/src/main/scala/kafka/server/LeaderEndPoint.scala:
##
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package kafka.server
+
+import kafka.cluster.BrokerEndPoint
+import kafka.server.AbstractFetcherThread.{ReplicaFetch, ResultWithPartitions}
+import org.apache.kafka.common.TopicPartition
+import org.apache.kafka.common.requests.FetchRequest
+import
org.apache.kafka.common.message.OffsetForLeaderEpochResponseData.EpochEndOffset
+import org.apache.kafka.common.message.{FetchResponseData,
OffsetForLeaderEpochRequestData}
+
+import scala.collection.Map
+
+/**
+ * This trait defines the APIs to be used on the client side to access a
broker that is a leader.
+ */
+trait LeaderEndPoint {
+
+ type FetchData = FetchResponseData.PartitionData
+ type EpochData = OffsetForLeaderEpochRequestData.OffsetForLeaderPartition
+
+ /**
+ * A boolean specifying if truncation when fetching from the leader is
supported
+ */
+ def isTruncationOnFetchSupported: Boolean
+
+ /**
+ * Initiate closing access to fetches from leader.
+ */
+ def initiateClose(): Unit
+
+ /**
+ * Closes access to fetches from leader.
+ * `initiateClose` must be called prior to invoking `close`.
+ */
+ def close(): Unit
+
+ /**
+ * The specific broker (host:port) we want to connect to.
+ */
+ def brokerEndPoint(): BrokerEndPoint
+
+ /**
+ * Given a fetchRequest, carries out the expected request and returns
+ * the results from fetching from the leader.
+ *
+ * @param fetchRequest The fetch request we want to carry out
+ *
+ * @return A map of topic partition -> fetch data
+ */
+ def fetch(fetchRequest: FetchRequest.Builder): Map[TopicPartition, FetchData]
+
+ /**
+ * Fetches the log start offset of the given topic partition, at a specific
+ * leader epoch, from the leader.
+ *
+ * @param topicPartition The topic partition that we want to fetch from
+ * @param currentLeaderEpoch An int representing the current leader epoch of
the requester
+ *
+ * @return A long representing the earliest offset in the leader's topic
partition.
Review Comment:
I can add some documentation to the `fetchEarliestOffset` in both
implemented endpoint classes to reflect that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] rittikaadhikari commented on a diff in pull request #12005: KAFKA-13803: Refactor Leader API Access
rittikaadhikari commented on code in PR #12005:
URL: https://github.com/apache/kafka/pull/12005#discussion_r878592706
##
core/src/main/scala/kafka/server/LeaderEndPoint.scala:
##
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package kafka.server
+
+import kafka.cluster.BrokerEndPoint
+import kafka.server.AbstractFetcherThread.{ReplicaFetch, ResultWithPartitions}
+import org.apache.kafka.common.TopicPartition
+import org.apache.kafka.common.requests.FetchRequest
+import
org.apache.kafka.common.message.OffsetForLeaderEpochResponseData.EpochEndOffset
+import org.apache.kafka.common.message.{FetchResponseData,
OffsetForLeaderEpochRequestData}
+
+import scala.collection.Map
+
+/**
+ * This trait defines the APIs to be used on the client side to access a
broker that is a leader.
+ */
+trait LeaderEndPoint {
+
+ type FetchData = FetchResponseData.PartitionData
+ type EpochData = OffsetForLeaderEpochRequestData.OffsetForLeaderPartition
+
+ /**
+ * A boolean specifying if truncation when fetching from the leader is
supported
+ */
+ def isTruncationOnFetchSupported: Boolean
+
+ /**
+ * Initiate closing access to fetches from leader.
+ */
+ def initiateClose(): Unit
+
+ /**
+ * Closes access to fetches from leader.
+ * `initiateClose` must be called prior to invoking `close`.
+ */
+ def close(): Unit
+
+ /**
+ * The specific broker (host:port) we want to connect to.
+ */
+ def brokerEndPoint(): BrokerEndPoint
+
+ /**
+ * Given a fetchRequest, carries out the expected request and returns
+ * the results from fetching from the leader.
+ *
+ * @param fetchRequest The fetch request we want to carry out
+ *
+ * @return A map of topic partition -> fetch data
+ */
+ def fetch(fetchRequest: FetchRequest.Builder): Map[TopicPartition, FetchData]
+
+ /**
+ * Fetches the log start offset of the given topic partition, at a specific
+ * leader epoch, from the leader.
+ *
+ * @param topicPartition The topic partition that we want to fetch from
+ * @param currentLeaderEpoch An int representing the current leader epoch of
the requester
+ *
+ * @return A long representing the earliest offset in the leader's topic
partition.
Review Comment:
I believe that this depends on whether we are using the
`RemoteLeaderEndPoint` vs. the `LocalLeaderEndPoint`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] rittikaadhikari commented on a diff in pull request #12005: KAFKA-13803: Refactor Leader API Access
rittikaadhikari commented on code in PR #12005:
URL: https://github.com/apache/kafka/pull/12005#discussion_r878591307
##
core/src/main/scala/kafka/server/BrokerBlockingSender.scala:
##
@@ -124,4 +128,8 @@ class ReplicaFetcherBlockingSend(sourceBroker:
BrokerEndPoint,
def close(): Unit = {
networkClient.close()
}
+
+ override def toString: String = {
+s"ReplicaFetcherBlockingSend($sourceBroker, fetcherId=$fetcherId)"
Review Comment:
Nice catch, I totally missed that in the refactor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] kowshik commented on a diff in pull request #12005: KAFKA-13803: Refactor Leader API Access
kowshik commented on code in PR #12005:
URL: https://github.com/apache/kafka/pull/12005#discussion_r878586639
##
core/src/main/scala/kafka/server/LeaderEndPoint.scala:
##
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package kafka.server
+
+import kafka.cluster.BrokerEndPoint
+import kafka.server.AbstractFetcherThread.{ReplicaFetch, ResultWithPartitions}
+import org.apache.kafka.common.TopicPartition
+import org.apache.kafka.common.requests.FetchRequest
+import
org.apache.kafka.common.message.OffsetForLeaderEpochResponseData.EpochEndOffset
+import org.apache.kafka.common.message.{FetchResponseData,
OffsetForLeaderEpochRequestData}
+
+import scala.collection.Map
+
+/**
+ * This trait defines the APIs to be used on the client side to access a
broker that is a leader.
+ */
+trait LeaderEndPoint {
+
+ type FetchData = FetchResponseData.PartitionData
+ type EpochData = OffsetForLeaderEpochRequestData.OffsetForLeaderPartition
+
+ /**
+ * A boolean specifying if truncation when fetching from the leader is
supported
+ */
+ def isTruncationOnFetchSupported: Boolean
+
+ /**
+ * Initiate closing access to fetches from leader.
+ */
+ def initiateClose(): Unit
+
+ /**
+ * Closes access to fetches from leader.
+ * `initiateClose` must be called prior to invoking `close`.
+ */
+ def close(): Unit
+
+ /**
+ * The specific broker (host:port) we want to connect to.
+ */
+ def brokerEndPoint(): BrokerEndPoint
+
+ /**
+ * Given a fetchRequest, carries out the expected request and returns
+ * the results from fetching from the leader.
+ *
+ * @param fetchRequest The fetch request we want to carry out
+ *
+ * @return A map of topic partition -> fetch data
+ */
+ def fetch(fetchRequest: FetchRequest.Builder): Map[TopicPartition, FetchData]
+
+ /**
+ * Fetches the log start offset of the given topic partition, at a specific
+ * leader epoch, from the leader.
+ *
+ * @param topicPartition The topic partition that we want to fetch from
+ * @param currentLeaderEpoch An int representing the current leader epoch of
the requester
+ *
+ * @return A long representing the earliest offset in the leader's topic
partition.
Review Comment:
Is this returning the leader's "local" log start offset, or the remote log
start offset?
Note: the earliest offset is not necessarily the local log start offset, it
could be the remote log start offset.
Depending on what this function is returning, we can improve the
documentation to mention "local" or "remote" log start offset.
##
core/src/main/scala/kafka/server/BrokerBlockingSender.scala:
##
@@ -124,4 +128,8 @@ class ReplicaFetcherBlockingSend(sourceBroker:
BrokerEndPoint,
def close(): Unit = {
networkClient.close()
}
+
+ override def toString: String = {
+s"ReplicaFetcherBlockingSend($sourceBroker, fetcherId=$fetcherId)"
Review Comment:
Lets use `BrokerBlockingSender` in the string.
##
core/src/test/scala/unit/kafka/server/epoch/util/ReplicaFetcherMockBlockingSend.scala:
##
@@ -16,19 +16,19 @@
*/
package kafka.server.epoch.util
-import java.net.SocketTimeoutException
-import java.util
import kafka.cluster.BrokerEndPoint
import kafka.server.BlockingSend
import org.apache.kafka.clients.{ClientRequest, ClientResponse, MockClient,
NetworkClientUtils}
-import org.apache.kafka.common.message.{FetchResponseData,
OffsetForLeaderEpochResponseData}
import
org.apache.kafka.common.message.OffsetForLeaderEpochResponseData.{EpochEndOffset,
OffsetForLeaderTopicResult}
+import org.apache.kafka.common.message.{FetchResponseData,
OffsetForLeaderEpochResponseData}
import org.apache.kafka.common.protocol.{ApiKeys, Errors}
import org.apache.kafka.common.requests.AbstractRequest.Builder
import org.apache.kafka.common.requests.{AbstractRequest, FetchResponse,
OffsetsForLeaderEpochResponse, FetchMetadata => JFetchMetadata}
import org.apache.kafka.common.utils.{SystemTime, Time}
import org.apache.kafka.common.{Node, TopicIdPartition, TopicPartition, Uuid}
+import java.net.SocketTimeoutException
+import java.util
import scala.collection.Map
/**
Review Comment:
[jira] [Commented] (KAFKA-13817) Schedule nextTimeToEmit to system time every time instead of just once
[ https://issues.apache.org/jira/browse/KAFKA-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17540294#comment-17540294 ] Matthias J. Sax commented on KAFKA-13817: - Thanks for the PR! I put it into my review backlog. Might take some time until I get to it. > Schedule nextTimeToEmit to system time every time instead of just once > -- > > Key: KAFKA-13817 > URL: https://issues.apache.org/jira/browse/KAFKA-13817 > Project: Kafka > Issue Type: Improvement > Components: streams >Reporter: Hao Li >Assignee: Lim Qing Wei >Priority: Minor > Labels: beginner, newbie > > [https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoin.java#L229-L231.] > > If this is just scheduled once, this can trigger emit every time if system > time jumps a lot suddenly. > > For example, > # nextTimeToEmit set to 1 and step is 1 > # If next system time jumps to 100, we will always emit for next 100 records -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [kafka] mjsax commented on pull request #12169: MINOR: improve description of `commit.interval.ms` config
mjsax commented on PR #12169: URL: https://github.com/apache/kafka/pull/12169#issuecomment-1133318073 I am not sure if bumping the priority from LOW to MEDIUM is the right thing to do? In general, we cannot also not explain all config cross-dependencies (ie, `linger.ms`). If there is no objection, I would merge the PR as-is? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] mjsax commented on a diff in pull request #12169: MINOR: improve description of `commit.interval.ms` config
mjsax commented on code in PR #12169:
URL: https://github.com/apache/kafka/pull/12169#discussion_r878518216
##
streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java:
##
@@ -452,7 +452,9 @@ public class StreamsConfig extends AbstractConfig {
/** {@code commit.interval.ms} */
@SuppressWarnings("WeakerAccess")
public static final String COMMIT_INTERVAL_MS_CONFIG =
"commit.interval.ms";
-private static final String COMMIT_INTERVAL_MS_DOC = "The frequency in
milliseconds with which to save the position of the processor." +
+private static final String COMMIT_INTERVAL_MS_DOC = "The frequency in
milliseconds with which to commit processing progress." +
+" For at-least-once processing, committing means to save the position
(ie, offsets) of the processor." +
+" For exactly-once processing, it means to commit the transaction
which includes to save the position." +
Review Comment:
Not sure -- would we go down the rabbit hole?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (KAFKA-13923) ZooKeeperAuthorizerTest should use standard authorizer for kraft
Jason Gustafson created KAFKA-13923: --- Summary: ZooKeeperAuthorizerTest should use standard authorizer for kraft Key: KAFKA-13923 URL: https://issues.apache.org/jira/browse/KAFKA-13923 Project: Kafka Issue Type: Improvement Reporter: Jason Gustafson Assignee: Jason Gustafson Our system test `ZooKeeperAuthorizerTest` relies on the zk-based `AclAuthorizer` even when running KRaft. We should update this test to use `StandardAuthorizer` (and probably change the name while we're at it). -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (KAFKA-13817) Schedule nextTimeToEmit to system time every time instead of just once
[ https://issues.apache.org/jira/browse/KAFKA-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17540259#comment-17540259 ] Lim Qing Wei commented on KAFKA-13817: -- Hi [~lihaosky] or [~mjsax] , will you be the one to review my PR? Or should I request in the dev mailing list? > Schedule nextTimeToEmit to system time every time instead of just once > -- > > Key: KAFKA-13817 > URL: https://issues.apache.org/jira/browse/KAFKA-13817 > Project: Kafka > Issue Type: Improvement > Components: streams >Reporter: Hao Li >Assignee: Lim Qing Wei >Priority: Minor > Labels: beginner, newbie > > [https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoin.java#L229-L231.] > > If this is just scheduled once, this can trigger emit every time if system > time jumps a lot suddenly. > > For example, > # nextTimeToEmit set to 1 and step is 1 > # If next system time jumps to 100, we will always emit for next 100 records -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (KAFKA-13914) Implement kafka-metadata-quorum.sh
[ https://issues.apache.org/jira/browse/KAFKA-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio updated KAFKA-13914: --- Issue Type: Improvement (was: Bug) > Implement kafka-metadata-quorum.sh > -- > > Key: KAFKA-13914 > URL: https://issues.apache.org/jira/browse/KAFKA-13914 > Project: Kafka > Issue Type: Improvement >Reporter: Jason Gustafson >Priority: Major > > KIP-595 documents a tool for describing quorum status > `kafka-metadata-quorum.sh`: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-595%3A+A+Raft+Protocol+for+the+Metadata+Quorum#KIP595:ARaftProtocolfortheMetadataQuorum-ToolingSupport.] > We need to implement this. > Note that this depends on the Admin API for `DescribeQuorum`, which is > proposed in KIP-836: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-836%3A+Addition+of+Information+in+DescribeQuorumResponse+about+Voter+Lag.] > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [kafka] divijvaidya commented on a diff in pull request #12189: Fix flaky test TopicCommandIntegrationTest.testDescribeAtMinIsrPartitions(String).quorum=kraft
divijvaidya commented on code in PR #12189:
URL: https://github.com/apache/kafka/pull/12189#discussion_r878437206
##
core/src/test/scala/integration/kafka/admin/TopicCommandIntegrationTest.scala:
##
@@ -697,6 +697,19 @@ class TopicCommandIntegrationTest extends
KafkaServerTestHarness with Logging wi
try {
killBroker(0)
killBroker(1)
+
+ val aliveServers = brokers.filterNot(brokers => brokers.config.brokerId
== 0 || brokers.config.brokerId == 1)
+
+ if (isKRaftTest()) {
+TestUtils.ensureConsistentKRaftMetadata(aliveServers, controllerServer,
Review Comment:
Thank you for the review and suggestion. I have made two changes based on
your suggestion:
1. Added a helper method `aliveBrokers` in KafkaServerTestHarness()
2. Modified KafkaServerTestHarness.ensureConsistentKRaftMetadata() to check
for only the brokers which are alive
Separately, if you have thoughts on further simplification (especially
around combining the logic of Zk and kraft in a single helper function), I
would be happy to make the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] hachikuji commented on a diff in pull request #12189: Fix flaky test TopicCommandIntegrationTest.testDescribeAtMinIsrPartitions(String).quorum=kraft
hachikuji commented on code in PR #12189:
URL: https://github.com/apache/kafka/pull/12189#discussion_r878330637
##
core/src/test/scala/integration/kafka/admin/TopicCommandIntegrationTest.scala:
##
@@ -697,6 +697,19 @@ class TopicCommandIntegrationTest extends
KafkaServerTestHarness with Logging wi
try {
killBroker(0)
killBroker(1)
+
+ val aliveServers = brokers.filterNot(brokers => brokers.config.brokerId
== 0 || brokers.config.brokerId == 1)
+
+ if (isKRaftTest()) {
+TestUtils.ensureConsistentKRaftMetadata(aliveServers, controllerServer,
Review Comment:
We have a helper `KafkaServerTestHarness.ensureConsistentKRaftMetadata`. I
think it would make sense to modify it to use only the alive brokers. Then we
could use it here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] dajac commented on pull request #12121: KAFKA-13846: Adding overloaded metricOrElseCreate method
dajac commented on PR #12121: URL: https://github.com/apache/kafka/pull/12121#issuecomment-1133028737 Do we also have concrete examples of usages of this new API? It would be helpful to update one or two metrics which would benefits from this change. This would be a good illustration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] dajac commented on pull request #12121: KAFKA-13846: Adding overloaded metricOrElseCreate method
dajac commented on PR #12121: URL: https://github.com/apache/kafka/pull/12121#issuecomment-1133027849 `Metrics` is actually published in our [javadoc](https://javadoc.io/doc/org.apache.kafka/kafka-clients/latest/org/apache/kafka/common/metrics/Metrics.html). This suggests that changing it would require a KIP, no? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] dajac commented on a diff in pull request #12121: KAFKA-13846: Adding overloaded metricOrElseCreate method
dajac commented on code in PR #12121:
URL: https://github.com/apache/kafka/pull/12121#discussion_r878262992
##
clients/src/main/java/org/apache/kafka/common/metrics/Metrics.java:
##
@@ -563,10 +615,15 @@ public synchronized void removeReporter(MetricsReporter
reporter) {
}
}
-synchronized void registerMetric(KafkaMetric metric) {
+synchronized void registerMetric(KafkaMetric metric, boolean
raiseIfMetricExists) {
Review Comment:
`registerMetric` is also called from the `Sensor` class so we have to check
if that works there as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Comment Edited] (KAFKA-13922) Unable to generate coverage reports for the whole project
[
https://issues.apache.org/jira/browse/KAFKA-13922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17540157#comment-17540157
]
Patrik Nagy edited comment on KAFKA-13922 at 5/20/22 2:53 PM:
--
There is a reportCoverage task in each module that runs jacoco or scoverage
report (scoverage for the core module and jacoco for any other). This works
well and needed for coverage generation.
There is also a reportCoverage task configured outside of subprojects scope
that depends on jacocoRootReport and the reportCoverage task of the core
module.
The jacocoRootReport depends on all java related project's test task.
I don't think this jacocoRootReport is needed in the project because:
* You can have the coverage data generated for all modules without it
* If you need something like a report to summarize everything that is what
sonar is much better for than an html/xml file
* I don't think jacoco aggregated report would include the core module's
scoverage report (I am sure it can be implemented with additional work, but I
am not sure it is worth instead of using simply a sonar), so even if the
aggregated report would work, it still would not be complete for the project,
core module would be missing
* It seems to me that Gradle advises to create another module for aggregated
reports that would be responsible to collect everything and generate the
report, not like the current one:
[https://docs.gradle.org/7.3.3/samples/sample_jvm_multi_project_with_code_coverage.html]
* I doubt that it is used by anyone because it is not working for a really
long time (tested in 2.5 and 3.2)
I think the jacocoRootReport should be considered to be deleted along with the
reportCoverage task outside of subprojects scope:
{code:java}
if (userEnableTestCoverage) {
task reportCoverage(dependsOn: ['jacocoRootReport', 'core:reportCoverage'])
} {code}
Without them, it would be easy to generate coverage for all modules as it is
documented.
was (Author: JIRAUSER289621):
There is a reportCoverage task in each module that runs jacoco or scoverage
report (scoverage for the core module and jacoco for any other). This works
well and needed for coverage generation.
There is also a reportCoverage task configured outside of subprojects scope
that depends on jacocoRootReport and the reportCoverage task of the core
module.
The jacocoRootReport depends on all java related project's test task.
I don't think this jacocoRootReport is needed in the project because:
* You can have the coverage data generated for all modules without it
* If you need something like a report to summarize everything that is what
sonar is much better for than an html/xml file
* I don't think jacoco aggregated report would include the core module's
scoverage report (I am sure it can be implemented with additional work, but I
am not sure it is worth instead of using simply a sonar), so even if the
aggregated report would work, it still would not be complete for the project,
core module would be missing
* It seems to me that Gradle advises to create another module for aggregated
reports that would be responsible to collect everything and generate the
report, not like the current one:
[https://docs.gradle.org/7.3.3/samples/sample_jvm_multi_project_with_code_coverage.html]
* I doubt that it is used by anyone because it is not working for a really
long time (tested in 2.5 and 3.2)
I think the jacocoRootReport should be considered to be deleted along with the
reportCoverage task outside of subprojects scope:
{code:java}
if (userEnableTestCoverage) {
task reportCoverage(dependsOn: ['jacocoRootReport', 'core:reportCoverage'])
} {code}
Without them, it would be easy to generate coverage for all modules as it is
documented.
> Unable to generate coverage reports for the whole project
> -
>
> Key: KAFKA-13922
> URL: https://issues.apache.org/jira/browse/KAFKA-13922
> Project: Kafka
> Issue Type: Bug
> Components: build
>Affects Versions: 2.5.0
>Reporter: Patrik Nagy
>Priority: Minor
>
> It is documented in the project that if we need code coverage reports for the
> whole project, we need to run something like this where we enabled the test
> coverage flag and run the reportCoverage task:
> {code:java}
> ./gradlew reportCoverage -PenableTestCoverage=true -Dorg.gradle.parallel=false
> {code}
> If I run it, the build will fail in the end because of jacocoRootReport:
> {code:java}
> 14:34:41 > Task :jacocoRootReport FAILED
> 14:34:41
> 14:34:41 FAILURE: Build failed with an exception.
> 14:34:41
> 14:34:41 * What went wrong:
> 14:34:41 Some problems were found with the configuration of task
> ':jacocoRootReport' (type 'JacocoReport').
> 14:34:41 - Type
[jira] [Commented] (KAFKA-13922) Unable to generate coverage reports for the whole project
[
https://issues.apache.org/jira/browse/KAFKA-13922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17540157#comment-17540157
]
Patrik Nagy commented on KAFKA-13922:
-
There is a reportCoverage task in each module that runs jacoco or scoverage
report (scoverage for the core module and jacoco for any other). This works
well and needed for coverage generation.
There is also a reportCoverage task configured outside of subprojects scope
that depends on jacocoRootReport and the reportCoverage task of the core
module.
The jacocoRootReport depends on all java related project's test task.
I don't think this jacocoRootReport is needed in the project because:
* You can have the coverage data generated for all modules without it
* If you need something like a report to summarize everything that is what
sonar is much better for than an html/xml file
* I don't think jacoco aggregated report would include the core module's
scoverage report (I am sure it can be implemented with additional work, but I
am not sure it is worth instead of using simply a sonar), so even if the
aggregated report would work, it still would not be complete for the project,
core module would be missing
* It seems to me that Gradle advises to create another module for aggregated
reports that would be responsible to collect everything and generate the
report, not like the current one:
[https://docs.gradle.org/7.3.3/samples/sample_jvm_multi_project_with_code_coverage.html]
* I doubt that it is used by anyone because it is not working for a really
long time (tested in 2.5 and 3.2)
I think the jacocoRootReport should be considered to be deleted along with the
reportCoverage task outside of subprojects scope:
{code:java}
if (userEnableTestCoverage) {
task reportCoverage(dependsOn: ['jacocoRootReport', 'core:reportCoverage'])
} {code}
Without them, it would be easy to generate coverage for all modules as it is
documented.
> Unable to generate coverage reports for the whole project
> -
>
> Key: KAFKA-13922
> URL: https://issues.apache.org/jira/browse/KAFKA-13922
> Project: Kafka
> Issue Type: Bug
> Components: build
>Affects Versions: 2.5.0
>Reporter: Patrik Nagy
>Priority: Minor
>
> It is documented in the project that if we need code coverage reports for the
> whole project, we need to run something like this where we enabled the test
> coverage flag and run the reportCoverage task:
> {code:java}
> ./gradlew reportCoverage -PenableTestCoverage=true -Dorg.gradle.parallel=false
> {code}
> If I run it, the build will fail in the end because of jacocoRootReport:
> {code:java}
> 14:34:41 > Task :jacocoRootReport FAILED
> 14:34:41
> 14:34:41 FAILURE: Build failed with an exception.
> 14:34:41
> 14:34:41 * What went wrong:
> 14:34:41 Some problems were found with the configuration of task
> ':jacocoRootReport' (type 'JacocoReport').
> 14:34:41 - Type 'org.gradle.testing.jacoco.tasks.JacocoReport' property
> 'jacocoClasspath' doesn't have a configured value.
> 14:34:41
> 14:34:41 Reason: This property isn't marked as optional and no value has
> been configured.
> 14:34:41
> 14:34:41 Possible solutions:
> 14:34:41 1. Assign a value to 'jacocoClasspath'.
> 14:34:41 2. Mark property 'jacocoClasspath' as optional.
> 14:34:41
> 14:34:41 Please refer to
> https://docs.gradle.org/7.3.3/userguide/validation_problems.html#value_not_set
> for more details about this problem.
> 14:34:41 - Type 'org.gradle.testing.jacoco.tasks.JacocoReport' property
> 'reports.enabledReports.html.outputLocation' doesn't have a configured value.
> 14:34:41
> 14:34:41 Reason: This property isn't marked as optional and no value has
> been configured.
> 14:34:41
> 14:34:41 Possible solutions:
> 14:34:41 1. Assign a value to
> 'reports.enabledReports.html.outputLocation'.
> 14:34:41 2. Mark property 'reports.enabledReports.html.outputLocation'
> as optional.
> 14:34:41
> 14:34:41 Please refer to
> https://docs.gradle.org/7.3.3/userguide/validation_problems.html#value_not_set
> for more details about this problem.
> 14:34:41 - Type 'org.gradle.testing.jacoco.tasks.JacocoReport' property
> 'reports.enabledReports.xml.outputLocation' doesn't have a configured value.
> 14:34:41
> 14:34:41 Reason: This property isn't marked as optional and no value has
> been configured.
> 14:34:41
> 14:34:41 Possible solutions:
> 14:34:41 1. Assign a value to
> 'reports.enabledReports.xml.outputLocation'.
> 14:34:41 2. Mark property 'reports.enabledReports.xml.outputLocation'
> as optional. {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [kafka] viktorsomogyi commented on pull request #10738: KAFKA-6945: KIP-373, allow users to create delegation token for others
viktorsomogyi commented on PR #10738: URL: https://github.com/apache/kafka/pull/10738#issuecomment-1132991630 @omkreddy would you please check it once you get some time? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] viktorsomogyi commented on pull request #12180: KAFKA-13917: Avoid calling lookupCoordinator() in tight loop
viktorsomogyi commented on PR #12180: URL: https://github.com/apache/kafka/pull/12180#issuecomment-1132983718 @mimaison would you please review this short change if you get some time? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (KAFKA-13922) Unable to generate coverage reports for the whole project
Patrik Nagy created KAFKA-13922:
---
Summary: Unable to generate coverage reports for the whole project
Key: KAFKA-13922
URL: https://issues.apache.org/jira/browse/KAFKA-13922
Project: Kafka
Issue Type: Bug
Components: build
Affects Versions: 2.5.0
Reporter: Patrik Nagy
It is documented in the project that if we need code coverage reports for the
whole project, we need to run something like this where we enabled the test
coverage flag and run the reportCoverage task:
{code:java}
./gradlew reportCoverage -PenableTestCoverage=true -Dorg.gradle.parallel=false
{code}
If I run it, the build will fail in the end because of jacocoRootReport:
{code:java}
14:34:41 > Task :jacocoRootReport FAILED
14:34:41
14:34:41 FAILURE: Build failed with an exception.
14:34:41
14:34:41 * What went wrong:
14:34:41 Some problems were found with the configuration of task
':jacocoRootReport' (type 'JacocoReport').
14:34:41 - Type 'org.gradle.testing.jacoco.tasks.JacocoReport' property
'jacocoClasspath' doesn't have a configured value.
14:34:41
14:34:41 Reason: This property isn't marked as optional and no value has
been configured.
14:34:41
14:34:41 Possible solutions:
14:34:41 1. Assign a value to 'jacocoClasspath'.
14:34:41 2. Mark property 'jacocoClasspath' as optional.
14:34:41
14:34:41 Please refer to
https://docs.gradle.org/7.3.3/userguide/validation_problems.html#value_not_set
for more details about this problem.
14:34:41 - Type 'org.gradle.testing.jacoco.tasks.JacocoReport' property
'reports.enabledReports.html.outputLocation' doesn't have a configured value.
14:34:41
14:34:41 Reason: This property isn't marked as optional and no value has
been configured.
14:34:41
14:34:41 Possible solutions:
14:34:41 1. Assign a value to
'reports.enabledReports.html.outputLocation'.
14:34:41 2. Mark property 'reports.enabledReports.html.outputLocation' as
optional.
14:34:41
14:34:41 Please refer to
https://docs.gradle.org/7.3.3/userguide/validation_problems.html#value_not_set
for more details about this problem.
14:34:41 - Type 'org.gradle.testing.jacoco.tasks.JacocoReport' property
'reports.enabledReports.xml.outputLocation' doesn't have a configured value.
14:34:41
14:34:41 Reason: This property isn't marked as optional and no value has
been configured.
14:34:41
14:34:41 Possible solutions:
14:34:41 1. Assign a value to 'reports.enabledReports.xml.outputLocation'.
14:34:41 2. Mark property 'reports.enabledReports.xml.outputLocation' as
optional. {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [kafka] vvcephei merged pull request #12186: MINOR: Deflake OptimizedKTableIntegrationTest
vvcephei merged PR #12186: URL: https://github.com/apache/kafka/pull/12186 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] soarez commented on a diff in pull request #12174: KAFKA-13907: Fix hanging ServerShutdownTest.testCleanShutdownWithKRaftControllerUnavailable
soarez commented on code in PR #12174: URL: https://github.com/apache/kafka/pull/12174#discussion_r878056323 ## core/src/main/scala/kafka/server/BrokerServer.scala: ## @@ -93,6 +93,9 @@ class BrokerServer( this.logIdent = logContext.logPrefix + // VisibleForTesting + private[kafka] var controlledShutdownTimeoutMillis: Long = 5 * 60 * 1000L // 5 minutes Review Comment: Thanks for reviewing this @dengziming. I agree it is not an elegant approach. And thanks for your suggestion, it seems to be in line with @hachikuji 's suggestion as well. I've updated this patch to follow your suggested approach. Please let me know if this is what you had in mind. However, I'm not very confident that this is better, since `KafkaServer` also has to implement the new `shutdown` variant and it has no use for the timeout. So it seems a bit strange to have an implementation where the timeout has to be ignored. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] divijvaidya commented on pull request #12189: Fix flaky test TopicCommandIntegrationTest.testDescribeAtMinIsrPartitions(String).quorum=kraft
divijvaidya commented on PR #12189: URL: https://github.com/apache/kafka/pull/12189#issuecomment-1132747105 @dengziming @guozhangwang perhaps you might be interested to code review this one? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] divijvaidya opened a new pull request, #12189: Fix flaky test TopicCommandIntegrationTest.testDescribeAtMinIsrPartitions(String).quorum=kraft
divijvaidya opened a new pull request, #12189: URL: https://github.com/apache/kafka/pull/12189 ## Problem Flaky test as failed in CI https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-12184/1/tests/ The test fails because it does not wait for metadata to be propagated across brokers before killing a broker which may lead to it getting stale information. Note that a similar test was done in #12104 for a different test. ## Change Wait for metadata propagation to complete after killing the broker. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] fvaleri commented on pull request #12159: Fix stuck SSL tests in case of authentication failure
fvaleri commented on PR #12159: URL: https://github.com/apache/kafka/pull/12159#issuecomment-1132744614 All selector's tests are passing. Failing tests seem to be unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (KAFKA-13921) Flaky test TopicCommandIntegrationTest testDescribeAtMinIsrPartitions(String).quorum=kraft
[
https://issues.apache.org/jira/browse/KAFKA-13921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Divij Vaidya reassigned KAFKA-13921:
Assignee: Divij Vaidya
> Flaky test TopicCommandIntegrationTest
> testDescribeAtMinIsrPartitions(String).quorum=kraft
> --
>
> Key: KAFKA-13921
> URL: https://issues.apache.org/jira/browse/KAFKA-13921
> Project: Kafka
> Issue Type: Bug
> Components: unit tests
>Reporter: Divij Vaidya
>Assignee: Divij Vaidya
>Priority: Minor
> Labels: flaky-test
>
> [https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-12184/1/tests]
> {code:java}
> org.opentest4j.AssertionFailedError: expected: <1> but was: <7> at
> app//org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55)
> at
> app//org.junit.jupiter.api.AssertionUtils.failNotEqual(AssertionUtils.java:62)
>at
> app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150)
> at
> app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145)
> at app//org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:527)
> at
> app//kafka.admin.TopicCommandIntegrationTest.testDescribeAtMinIsrPartitions(TopicCommandIntegrationTest.scala:704)
>at
> java.base@11.0.12/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base@11.0.12/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base@11.0.12/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base@11.0.12/java.lang.reflect.Method.invoke(Method.java:566)
> at
> app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
> at
> app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
>at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
> at
> app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
> at
> app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
> at
> app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:92)
>at
> app//org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
> at
> app//org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
> at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
> at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
>at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
> at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
> at
> app//org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
> at
> app//org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
> at
> app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
>at
> app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
>at
> app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
> at
> app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
> at
> app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
> at
> app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
> at
> app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
>at
> app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
> at
> app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
>at
> app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
> at
> app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
>at
>
[jira] [Updated] (KAFKA-13920) KafkaProducer.close waits idefinitely on ioThread that is not joining
[
https://issues.apache.org/jira/browse/KAFKA-13920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Muravskiy updated KAFKA-13920:
---
Description:
When closing (transactional) KafkaProducer (after aborting incomplete
transaction with {{KafkaProducer.abortTransaction()}}), it keeps waiting for
the ioThread to join:
{code:java}
"MainLoop" #24 prio=5 os_prio=0 tid=0x7f93d8afb800 nid=0x2c398 in
Object.wait() [0x7f93a0cf8000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1265)
- locked <0x85feb2c0> (a
org.apache.kafka.common.utils.KafkaThread)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1220)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1197)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1173){code}
while the ioThread is in this state:
{code:java}
"kafka-producer-network-thread | produce-ris-live-consume1.vega" #26 daemon
prio=5 os_prio=0 tid=0x7f92d01e1800 nid=0x2c399 waiting on condition
[0x7f93a03ca000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.kafka.common.utils.Utils.sleep(Utils.java:375)
at org.apache.kafka.common.utils.SystemTime.sleep(SystemTime.java:41)
at
org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:463)
at
org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:315)
at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:255)
at java.lang.Thread.run(Thread.java:750)
{code}
and does not quit.
was:
{{When closing (transactional) KafkaProducer (after aborting incomplete
transaction with }}{{{}KafkaProducer{}}}{{{}.abortTransaction(){}}}), it keeps
waiting for the ioThread to join:
{code:java}
"MainLoop" #24 prio=5 os_prio=0 tid=0x7f93d8afb800 nid=0x2c398 in
Object.wait() [0x7f93a0cf8000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1265)
- locked <0x85feb2c0> (a
org.apache.kafka.common.utils.KafkaThread)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1220)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1197)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1173){code}
{{while the ioThread is in this state:}}
{code:java}
"kafka-producer-network-thread | produce-ris-live-consume1.vega" #26 daemon
prio=5 os_prio=0 tid=0x7f92d01e1800 nid=0x2c399 waiting on condition
[0x7f93a03ca000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.kafka.common.utils.Utils.sleep(Utils.java:375)
at org.apache.kafka.common.utils.SystemTime.sleep(SystemTime.java:41)
at
org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:463)
at
org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:315)
at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:255)
at java.lang.Thread.run(Thread.java:750){code}
{{and does not quit.}}
> KafkaProducer.close waits idefinitely on ioThread that is not joining
> --
>
> Key: KAFKA-13920
> URL: https://issues.apache.org/jira/browse/KAFKA-13920
> Project: Kafka
> Issue Type: Bug
> Components: producer
>Affects Versions: 2.8.0
>Reporter: Oleg Muravskiy
>Priority: Major
>
> When closing (transactional) KafkaProducer (after aborting incomplete
> transaction with {{KafkaProducer.abortTransaction()}}), it keeps waiting for
> the ioThread to join:
> {code:java}
> "MainLoop" #24 prio=5 os_prio=0 tid=0x7f93d8afb800 nid=0x2c398 in
> Object.wait() [0x7f93a0cf8000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Thread.join(Thread.java:1265)
> - locked <0x85feb2c0> (a
> org.apache.kafka.common.utils.KafkaThread)
> at
> org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1220)
> at
> org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1197)
> at
> org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1173){code}
> while the ioThread is in this state:
> {code:java}
> "kafka-producer-network-thread | produce-ris-live-consume1.vega" #26 daemon
> prio=5 os_prio=0
[jira] [Updated] (KAFKA-13920) KafkaProducer.close waits idefinitely on ioThread that is not joining
[
https://issues.apache.org/jira/browse/KAFKA-13920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Muravskiy updated KAFKA-13920:
---
Description:
{{When closing (transactional) KafkaProducer (after aborting incomplete
transaction with }}{{{}KafkaProducer{}}}{{{}.abortTransaction(){}}}), it keeps
waiting for the ioThread to join:
{code:java}
"MainLoop" #24 prio=5 os_prio=0 tid=0x7f93d8afb800 nid=0x2c398 in
Object.wait() [0x7f93a0cf8000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1265)
- locked <0x85feb2c0> (a
org.apache.kafka.common.utils.KafkaThread)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1220)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1197)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1173){code}
{{while the ioThread is in this state:}}
{code:java}
"kafka-producer-network-thread | produce-ris-live-consume1.vega" #26 daemon
prio=5 os_prio=0 tid=0x7f92d01e1800 nid=0x2c399 waiting on condition
[0x7f93a03ca000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.kafka.common.utils.Utils.sleep(Utils.java:375)
at org.apache.kafka.common.utils.SystemTime.sleep(SystemTime.java:41)
at
org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:463)
at
org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:315)
at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:255)
at java.lang.Thread.run(Thread.java:750){code}
{{and does not quit.}}
was:
{}When closing (transactional) KafkaProducer (after aborting incomplete
transaction with {{{}}}{{{}KafkaProducer{}}}{{{}.abortTransaction()}}), it
keeps waiting for the ioThread to join:{}
{code:java}
"MainLoop" #24 prio=5 os_prio=0 tid=0x7f93d8afb800 nid=0x2c398 in
Object.wait() [0x7f93a0cf8000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1265)
- locked <0x85feb2c0> (a
org.apache.kafka.common.utils.KafkaThread)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1220)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1197)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1173){code}
{{while the ioThread is in this state:}}
{code:java}
"kafka-producer-network-thread | produce-ris-live-consume1.vega" #26 daemon
prio=5 os_prio=0 tid=0x7f92d01e1800 nid=0x2c399 waiting on condition
[0x7f93a03ca000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.kafka.common.utils.Utils.sleep(Utils.java:375)
at org.apache.kafka.common.utils.SystemTime.sleep(SystemTime.java:41)
at
org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:463)
at
org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:315)
at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:255)
at java.lang.Thread.run(Thread.java:750){code}
{{and does not quit.}}
> KafkaProducer.close waits idefinitely on ioThread that is not joining
> --
>
> Key: KAFKA-13920
> URL: https://issues.apache.org/jira/browse/KAFKA-13920
> Project: Kafka
> Issue Type: Bug
> Components: producer
>Affects Versions: 2.8.0
>Reporter: Oleg Muravskiy
>Priority: Major
>
> {{When closing (transactional) KafkaProducer (after aborting incomplete
> transaction with }}{{{}KafkaProducer{}}}{{{}.abortTransaction(){}}}), it
> keeps waiting for the ioThread to join:
> {code:java}
> "MainLoop" #24 prio=5 os_prio=0 tid=0x7f93d8afb800 nid=0x2c398 in
> Object.wait() [0x7f93a0cf8000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Thread.join(Thread.java:1265)
> - locked <0x85feb2c0> (a
> org.apache.kafka.common.utils.KafkaThread)
> at
> org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1220)
> at
> org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1197)
> at
> org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1173){code}
> {{while the ioThread is in this state:}}
> {code:java}
> "kafka-producer-network-thread |
[jira] [Updated] (KAFKA-13920) KafkaProducer.close waits idefinitely on ioThread that is not joining
[
https://issues.apache.org/jira/browse/KAFKA-13920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Muravskiy updated KAFKA-13920:
---
Description:
{}When closing (transactional) KafkaProducer (after aborting incomplete
transaction with {{{}}}{{{}KafkaProducer{}}}{{{}.abortTransaction()}}), it
keeps waiting for the ioThread to join:{}
{code:java}
"MainLoop" #24 prio=5 os_prio=0 tid=0x7f93d8afb800 nid=0x2c398 in
Object.wait() [0x7f93a0cf8000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1265)
- locked <0x85feb2c0> (a
org.apache.kafka.common.utils.KafkaThread)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1220)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1197)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1173){code}
{{while the ioThread is in this state:}}
{code:java}
"kafka-producer-network-thread | produce-ris-live-consume1.vega" #26 daemon
prio=5 os_prio=0 tid=0x7f92d01e1800 nid=0x2c399 waiting on condition
[0x7f93a03ca000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.kafka.common.utils.Utils.sleep(Utils.java:375)
at org.apache.kafka.common.utils.SystemTime.sleep(SystemTime.java:41)
at
org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:463)
at
org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:315)
at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:255)
at java.lang.Thread.run(Thread.java:750){code}
{{and does not quit.}}
was:
{{When closing (transactional) KafkaProducer, it keeps waiting for the ioThread
to join:}}
{code:java}
"MainLoop" #24 prio=5 os_prio=0 tid=0x7f93d8afb800 nid=0x2c398 in
Object.wait() [0x7f93a0cf8000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1265)
- locked <0x85feb2c0> (a
org.apache.kafka.common.utils.KafkaThread)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1220)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1197)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1173){code}
{{while the ioThread is in this state:}}
{code:java}
"kafka-producer-network-thread | produce-ris-live-consume1.vega" #26 daemon
prio=5 os_prio=0 tid=0x7f92d01e1800 nid=0x2c399 waiting on condition
[0x7f93a03ca000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.kafka.common.utils.Utils.sleep(Utils.java:375)
at org.apache.kafka.common.utils.SystemTime.sleep(SystemTime.java:41)
at
org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:463)
at
org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:315)
at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:255)
at java.lang.Thread.run(Thread.java:750){code}
{{and does not quit.}}
> KafkaProducer.close waits idefinitely on ioThread that is not joining
> --
>
> Key: KAFKA-13920
> URL: https://issues.apache.org/jira/browse/KAFKA-13920
> Project: Kafka
> Issue Type: Bug
> Components: producer
>Affects Versions: 2.8.0
>Reporter: Oleg Muravskiy
>Priority: Major
>
> {}When closing (transactional) KafkaProducer (after aborting incomplete
> transaction with {{{}}}{{{}KafkaProducer{}}}{{{}.abortTransaction()}}), it
> keeps waiting for the ioThread to join:{}
> {code:java}
> "MainLoop" #24 prio=5 os_prio=0 tid=0x7f93d8afb800 nid=0x2c398 in
> Object.wait() [0x7f93a0cf8000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Thread.join(Thread.java:1265)
> - locked <0x85feb2c0> (a
> org.apache.kafka.common.utils.KafkaThread)
> at
> org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1220)
> at
> org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1197)
> at
> org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1173){code}
> {{while the ioThread is in this state:}}
> {code:java}
> "kafka-producer-network-thread | produce-ris-live-consume1.vega" #26 daemon
> prio=5 os_prio=0 tid=0x7f92d01e1800 nid=0x2c399
[jira] [Updated] (KAFKA-13920) KafkaProducer.close waits idefinitely on ioThread that is not joining
[
https://issues.apache.org/jira/browse/KAFKA-13920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Muravskiy updated KAFKA-13920:
---
Description:
{{When closing (transactional) KafkaProducer, it keeps waiting for the ioThread
to join:}}
{code:java}
"MainLoop" #24 prio=5 os_prio=0 tid=0x7f93d8afb800 nid=0x2c398 in
Object.wait() [0x7f93a0cf8000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1265)
- locked <0x85feb2c0> (a
org.apache.kafka.common.utils.KafkaThread)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1220)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1197)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1173){code}
{{while the ioThread is in this state:}}
{code:java}
"kafka-producer-network-thread | produce-ris-live-consume1.vega" #26 daemon
prio=5 os_prio=0 tid=0x7f92d01e1800 nid=0x2c399 waiting on condition
[0x7f93a03ca000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.kafka.common.utils.Utils.sleep(Utils.java:375)
at org.apache.kafka.common.utils.SystemTime.sleep(SystemTime.java:41)
at
org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:463)
at
org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:315)
at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:255)
at java.lang.Thread.run(Thread.java:750){code}
{{and does not quit.}}
was:
{{When closing (transactional) KafkaProducer, it keeps waiting for the ioThread
to join:}}
{{}}
{code:java}
"MainLoop" #24 prio=5 os_prio=0 tid=0x7f93d8afb800 nid=0x2c398 in
Object.wait() [0x7f93a0cf8000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1265)
- locked <0x85feb2c0> (a
org.apache.kafka.common.utils.KafkaThread)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1220)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1197)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1173){code}
{{while the ioThread is in this state:}}
{{}}
{code:java}
"kafka-producer-network-thread | produce-ris-live-consume1.vega" #26 daemon
prio=5 os_prio=0 tid=0x7f92d01e1800 nid=0x2c399 waiting on condition
[0x7f93a03ca000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.kafka.common.utils.Utils.sleep(Utils.java:375)
at org.apache.kafka.common.utils.SystemTime.sleep(SystemTime.java:41)
at
org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:463)
at
org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:315)
at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:255)
at java.lang.Thread.run(Thread.java:750){code}
{{and does not quit.}}
> KafkaProducer.close waits idefinitely on ioThread that is not joining
> --
>
> Key: KAFKA-13920
> URL: https://issues.apache.org/jira/browse/KAFKA-13920
> Project: Kafka
> Issue Type: Bug
> Components: producer
>Affects Versions: 2.8.0
>Reporter: Oleg Muravskiy
>Priority: Major
>
> {{When closing (transactional) KafkaProducer, it keeps waiting for the
> ioThread to join:}}
> {code:java}
> "MainLoop" #24 prio=5 os_prio=0 tid=0x7f93d8afb800 nid=0x2c398 in
> Object.wait() [0x7f93a0cf8000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Thread.join(Thread.java:1265)
> - locked <0x85feb2c0> (a
> org.apache.kafka.common.utils.KafkaThread)
> at
> org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1220)
> at
> org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1197)
> at
> org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1173){code}
> {{while the ioThread is in this state:}}
> {code:java}
> "kafka-producer-network-thread | produce-ris-live-consume1.vega" #26 daemon
> prio=5 os_prio=0 tid=0x7f92d01e1800 nid=0x2c399 waiting on condition
> [0x7f93a03ca000]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
[jira] [Updated] (KAFKA-13920) KafkaProducer.close waits idefinitely on ioThread that is not joining
[
https://issues.apache.org/jira/browse/KAFKA-13920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Muravskiy updated KAFKA-13920:
---
Affects Version/s: 2.8.0
(was: 2.6.2)
> KafkaProducer.close waits idefinitely on ioThread that is not joining
> --
>
> Key: KAFKA-13920
> URL: https://issues.apache.org/jira/browse/KAFKA-13920
> Project: Kafka
> Issue Type: Bug
> Components: producer
>Affects Versions: 2.8.0
>Reporter: Oleg Muravskiy
>Priority: Major
>
> {{When closing (transactional) KafkaProducer, it keeps waiting for the
> ioThread to join:}}
> {{}}
> {code:java}
> "MainLoop" #24 prio=5 os_prio=0 tid=0x7f93d8afb800 nid=0x2c398 in
> Object.wait() [0x7f93a0cf8000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Thread.join(Thread.java:1265)
> - locked <0x85feb2c0> (a
> org.apache.kafka.common.utils.KafkaThread)
> at
> org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1220)
> at
> org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1197)
> at
> org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1173){code}
> {{while the ioThread is in this state:}}
> {{}}
> {code:java}
> "kafka-producer-network-thread | produce-ris-live-consume1.vega" #26 daemon
> prio=5 os_prio=0 tid=0x7f92d01e1800 nid=0x2c399 waiting on condition
> [0x7f93a03ca000]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.kafka.common.utils.Utils.sleep(Utils.java:375)
> at org.apache.kafka.common.utils.SystemTime.sleep(SystemTime.java:41)
> at
> org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:463)
> at
> org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:315)
> at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:255)
> at java.lang.Thread.run(Thread.java:750){code}
> {{and does not quit.}}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (KAFKA-13921) Flaky test TopicCommandIntegrationTest testDescribeAtMinIsrPartitions(String).quorum=kraft
Divij Vaidya created KAFKA-13921:
Summary: Flaky test TopicCommandIntegrationTest
testDescribeAtMinIsrPartitions(String).quorum=kraft
Key: KAFKA-13921
URL: https://issues.apache.org/jira/browse/KAFKA-13921
Project: Kafka
Issue Type: Bug
Components: unit tests
Reporter: Divij Vaidya
[https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-12184/1/tests]
{code:java}
org.opentest4j.AssertionFailedError: expected: <1> but was: <7> at
app//org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55) at
app//org.junit.jupiter.api.AssertionUtils.failNotEqual(AssertionUtils.java:62)
at
app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150) at
app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145) at
app//org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:527) at
app//kafka.admin.TopicCommandIntegrationTest.testDescribeAtMinIsrPartitions(TopicCommandIntegrationTest.scala:704)
at
java.base@11.0.12/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) at
java.base@11.0.12/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base@11.0.12/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base@11.0.12/java.lang.reflect.Method.invoke(Method.java:566) at
app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
at
app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at
app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at
app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at
app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:92)
at
app//org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at
app//org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at
app//org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at
app//org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at
app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
at
app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
at
app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
at
app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
at
app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
at
app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at
app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at
app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at
app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at
app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at
app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
at
app//org.junit.platform.engine.support.hierarchical.NodeTestTask$DefaultDynamicTestExecutor.execute(NodeTestTask.java:226)
at
[jira] [Created] (KAFKA-13920) KafkaProducer.close waits idefinitely on ioThread that is not joining
Oleg Muravskiy created KAFKA-13920:
--
Summary: KafkaProducer.close waits idefinitely on ioThread that is
not joining
Key: KAFKA-13920
URL: https://issues.apache.org/jira/browse/KAFKA-13920
Project: Kafka
Issue Type: Bug
Components: producer
Affects Versions: 2.6.2
Reporter: Oleg Muravskiy
{{When closing (transactional) KafkaProducer, it keeps waiting for the ioThread
to join:}}
{{}}
{code:java}
"MainLoop" #24 prio=5 os_prio=0 tid=0x7f93d8afb800 nid=0x2c398 in
Object.wait() [0x7f93a0cf8000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1265)
- locked <0x85feb2c0> (a
org.apache.kafka.common.utils.KafkaThread)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1220)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1197)
at
org.apache.kafka.clients.producer.KafkaProducer.close(KafkaProducer.java:1173){code}
{{while the ioThread is in this state:}}
{{}}
{code:java}
"kafka-producer-network-thread | produce-ris-live-consume1.vega" #26 daemon
prio=5 os_prio=0 tid=0x7f92d01e1800 nid=0x2c399 waiting on condition
[0x7f93a03ca000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.kafka.common.utils.Utils.sleep(Utils.java:375)
at org.apache.kafka.common.utils.SystemTime.sleep(SystemTime.java:41)
at
org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:463)
at
org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:315)
at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:255)
at java.lang.Thread.run(Thread.java:750){code}
{{and does not quit.}}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [kafka] calmera opened a new pull request, #12188: KAFKA-10892: Shared Readonly State Stores
calmera opened a new pull request, #12188: URL: https://github.com/apache/kafka/pull/12188 This PR implements KIP-813: support for shareable state stores. Tests were created to validate the structure of the resulting topology. Certain parts (MockProcessor eg.) have been migrated to the newer PAPI as well. Tests have been ran against the whole project to be sure this migration doesn't cause any side-effects. The contribution is my original work and I license the work to the project under the project's open source license -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] dajac opened a new pull request, #12187: KAFKA-13858; Kraft should not shutdown metadata listener until controller shutdown is finished
dajac opened a new pull request, #12187: URL: https://github.com/apache/kafka/pull/12187 WIP ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (KAFKA-13919) Add metric for log recovery progress
Luke Chen created KAFKA-13919: - Summary: Add metric for log recovery progress Key: KAFKA-13919 URL: https://issues.apache.org/jira/browse/KAFKA-13919 Project: Kafka Issue Type: New Feature Components: core, log Reporter: Luke Chen Assignee: Luke Chen expose metrics for log recovery progress. This metrics would let the admins have a way to monitor the log recovery progress. Details can be found here: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-831%3A+Add+metric+for+log+recovery+progress] -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [kafka] dengziming commented on pull request #11951: KAFKA-13836: Improve KRaft broker heartbeat logic
dengziming commented on PR #11951: URL: https://github.com/apache/kafka/pull/11951#issuecomment-1132601901 Bumped `BrokerHeartbeat` and use version 2 if MetadataVersion is above IBP_3_3_IV1, this is the first use case of FeatureVersion, PTAL. @hachikuji @mumrah -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] dengziming commented on a diff in pull request #12174: KAFKA-13907: Fix hanging ServerShutdownTest.testCleanShutdownWithKRaftControllerUnavailable
dengziming commented on code in PR #12174: URL: https://github.com/apache/kafka/pull/12174#discussion_r877836361 ## core/src/main/scala/kafka/server/BrokerServer.scala: ## @@ -93,6 +93,9 @@ class BrokerServer( this.logIdent = logContext.logPrefix + // VisibleForTesting + private[kafka] var controlledShutdownTimeoutMillis: Long = 5 * 60 * 1000L // 5 minutes Review Comment: This can solve the problem but is not very elegant, we can add a `shutdown(timeout)`, and use `Time` to compute a deadline, then we can use the deadline to compute a timeout for a controlled shutdown, WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] cadonna commented on pull request #12152: MINOR: Fix DSL typo in streams docs
cadonna commented on PR #12152: URL: https://github.com/apache/kafka/pull/12152#issuecomment-1132573699 Thanks for the idea! To get more information about the directory mapping you can consult the release process wiki: https://cwiki.apache.org/confluence/display/KAFKA/Release+Process . There are two parts that describe how docs are released, one is section "Website update process" and the other is the item "Update the website:" under section "After the vote passes". If you have tested your idea, feel free to propose it on the dev mailing list. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] milindmantri commented on pull request #12152: MINOR: Fix DSL typo in streams docs
milindmantri commented on PR #12152: URL: https://github.com/apache/kafka/pull/12152#issuecomment-1132547512 @cadonna, I looked into this. My initial idea was that adding a `docs-typo`-like label to a PR would trigger (after merge) a workflow/action on `kafka-site` that creates a PR there. I'm not sure about the directory mapping from `apache/kafka` to `apache/kafka-site`, but this can be explored. Does this resonate with you? Can you think of another way or how this can be improved? Is this even worth exploring further, considering the number/rate of PRs that are for docs related typos? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
