[jira] [Created] (KAFKA-15340) Test request quota for kraft controller apis

[Deng Ziming (Jira)]

Deng Ziming created KAFKA-15340:
---

 Summary: Test request quota for kraft controller apis
 Key: KAFKA-15340
 URL: https://issues.apache.org/jira/browse/KAFKA-15340
 Project: Kafka
  Issue Type: Improvement
  Components: kraft, unit tests
Reporter: Deng Ziming


The RequestQuotaTest only tests request quota for kraft broker apis and zk 
broker apis, we should also test kraft controller apis. 

further more, maybe there are others tests we need to complement for kraft 
controller apis(not only broker apis).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-15339) Transient I/O error happening in appending records could lead to the half of whole cluster

[Haoze Wu (Jira)]

Haoze Wu created KAFKA-15339:


 Summary: Transient I/O error happening in appending records could 
lead to the half of whole cluster
 Key: KAFKA-15339
 URL: https://issues.apache.org/jira/browse/KAFKA-15339
 Project: Kafka
  Issue Type: Improvement
  Components: connect, producer 
Affects Versions: 3.5.0
Reporter: Haoze Wu


We are running an integration test in which we start an Embedded Connect 
Cluster in the active 3.5 branch. However, because of transient disk error, we 
may encounter an IOException during appending records to one topic. As shown in 
the stack trace: 
{code:java}
[2023-08-13 16:53:51,016] ERROR Error while appending records to 
connect-config-topic-connect-cluster-0 in dir 
/tmp/EmbeddedKafkaCluster8003464883598783225 
(org.apache.kafka.storage.internals.log.LogDirFailureChannel:61)
java.io.IOException: 
        at 
org.apache.kafka.common.record.MemoryRecords.writeFullyTo(MemoryRecords.java:92)
        at 
org.apache.kafka.common.record.FileRecords.append(FileRecords.java:188)
        at kafka.log.LogSegment.append(LogSegment.scala:161)
        at kafka.log.LocalLog.append(LocalLog.scala:436)
        at kafka.log.UnifiedLog.append(UnifiedLog.scala:853)
        at kafka.log.UnifiedLog.appendAsLeader(UnifiedLog.scala:664)
        at 
kafka.cluster.Partition.$anonfun$appendRecordsToLeader$1(Partition.scala:1281)
        at kafka.cluster.Partition.appendRecordsToLeader(Partition.scala:1269)
        at 
kafka.server.ReplicaManager.$anonfun$appendToLocalLog$6(ReplicaManager.scala:977)
        at 
scala.collection.StrictOptimizedMapOps.map(StrictOptimizedMapOps.scala:28)
        at 
scala.collection.StrictOptimizedMapOps.map$(StrictOptimizedMapOps.scala:27)
        at scala.collection.mutable.HashMap.map(HashMap.scala:35)
        at 
kafka.server.ReplicaManager.appendToLocalLog(ReplicaManager.scala:965)
        at kafka.server.ReplicaManager.appendRecords(ReplicaManager.scala:623)
        at kafka.server.KafkaApis.handleProduceRequest(KafkaApis.scala:680)
        at kafka.server.KafkaApis.handle(KafkaApis.scala:180)
        at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:76)
        at java.lang.Thread.run(Thread.java:748) {code}
However, just because of failing to append the records to one partition. The 
fetcher for all the other partitions are removed, broker shutdown, and finally 
embedded connect cluster killed as whole. 
{code:java}
[2023-08-13 17:35:37,966] WARN Stopping serving logs in dir 
/tmp/EmbeddedKafkaCluster6777164631574762227 (kafka.log.LogManager:70)
[2023-08-13 17:35:37,968] ERROR Shutdown broker because all log dirs in 
/tmp/EmbeddedKafkaCluster6777164631574762227 have failed 
(kafka.log.LogManager:143)
[2023-08-13 17:35:37,968] WARN Abrupt service halt with code 1 and message null 
(org.apache.kafka.connect.util.clusters.EmbeddedConnectCluster:130)
[2023-08-13 17:35:37,968] ERROR [LogDirFailureHandler]: Error due to 
(kafka.server.ReplicaManager$LogDirFailureHandler:135)
org.apache.kafka.connect.util.clusters.UngracefulShutdownException: Abrupt 
service halt with code 1 and message null {code}
I am wondering if we could add configurable retry around the root cause to 
tolerate the possible I/O faults so that if the retry is successful, the 
embedded connect cluster could still operate. 

Any comments and suggestions would be appreciated.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2097

[Apache Jenkins Server]

See 

[jira] [Created] (KAFKA-15338) The metric group documentation for metrics added in KAFKA-13945 is incorrect

[Neil Buesing (Jira)]

Neil Buesing created KAFKA-15338:


 Summary: The metric group documentation for metrics added in 
KAFKA-13945 is incorrect
 Key: KAFKA-15338
 URL: https://issues.apache.org/jira/browse/KAFKA-15338
 Project: Kafka
  Issue Type: Bug
  Components: streams
Affects Versions: 3.5.1, 3.4.1, 3.5.0, 3.3.2, 3.3.1, 3.4.0, 3.3.0
Reporter: Neil Buesing


ops.html (docs/streams/ops.html) incorrectly states that the metrics type is 
"stream-processor-node-metrics", but in looking at the metrics and inspecting 
the code in TopicMetrics, these metrics have a type of "stream-topic-metrics".

4 metrics are in error "bytes-consumed-total", "bytes-produced-total", 
"records-consumed-total", and "records-produced-total".

Looks like the type was changed from the KIP, and the documentation still 
reflects the KIP.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)