[jira] [Created] (KAFKA-12429) Serdes for all message types in internal topic which is used in default implementation for RLMM.

2021-03-04 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-12429: -- Summary: Serdes for all message types in internal topic which is used in default implementation for RLMM. Key: KAFKA-12429 URL: https://issues.apache.org/jira/browse/KAFKA-124

[GitHub] [kafka] chia7712 commented on a change in pull request #10269: KAFKA-12410 KafkaAPis ought to group fetch data before generating fet…

2021-03-04 Thread GitBox
chia7712 commented on a change in pull request #10269: URL: https://github.com/apache/kafka/pull/10269#discussion_r588095220 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -811,31 +821,40 @@ class KafkaApis(val requestChannel: RequestChannel, .s

[jira] [Updated] (KAFKA-7739) Kafka Tiered Storage

2021-03-04 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana updated KAFKA-7739: -- Description: KIP:  [https://cwiki.apache.org/confluence/display/KAFKA/KIP-405%3A+Kafka+Tiered+St

[GitHub] [kafka] chia7712 commented on a change in pull request #10269: KAFKA-12410 KafkaAPis ought to group fetch data before generating fet…

2021-03-04 Thread GitBox
chia7712 commented on a change in pull request #10269: URL: https://github.com/apache/kafka/pull/10269#discussion_r588094923 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -811,31 +821,40 @@ class KafkaApis(val requestChannel: RequestChannel, .s

[GitHub] [kafka] dajac commented on a change in pull request #10269: KAFKA-12410 KafkaAPis ought to group fetch data before generating fet…

2021-03-04 Thread GitBox
dajac commented on a change in pull request #10269: URL: https://github.com/apache/kafka/pull/10269#discussion_r588094059 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -811,31 +821,40 @@ class KafkaApis(val requestChannel: RequestChannel, .setR

[GitHub] [kafka] dajac commented on a change in pull request #10269: KAFKA-12410 KafkaAPis ought to group fetch data before generating fet…

2021-03-04 Thread GitBox
dajac commented on a change in pull request #10269: URL: https://github.com/apache/kafka/pull/10269#discussion_r588093454 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -811,31 +821,40 @@ class KafkaApis(val requestChannel: RequestChannel, .setR

[GitHub] [kafka] chia7712 commented on a change in pull request #10269: KAFKA-12410 KafkaAPis ought to group fetch data before generating fet…

2021-03-04 Thread GitBox
chia7712 commented on a change in pull request #10269: URL: https://github.com/apache/kafka/pull/10269#discussion_r588091547 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -811,31 +821,40 @@ class KafkaApis(val requestChannel: RequestChannel, .s

[GitHub] [kafka] chia7712 commented on a change in pull request #10269: KAFKA-12410 KafkaAPis ought to group fetch data before generating fet…

2021-03-04 Thread GitBox
chia7712 commented on a change in pull request #10269: URL: https://github.com/apache/kafka/pull/10269#discussion_r588090869 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -811,31 +821,40 @@ class KafkaApis(val requestChannel: RequestChannel, .s

[GitHub] [kafka] dajac commented on a change in pull request #10269: KAFKA-12410 KafkaAPis ought to group fetch data before generating fet…

2021-03-04 Thread GitBox
dajac commented on a change in pull request #10269: URL: https://github.com/apache/kafka/pull/10269#discussion_r588075100 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -811,31 +821,40 @@ class KafkaApis(val requestChannel: RequestChannel, .setR

[jira] [Assigned] (KAFKA-12404) Provision to propagate error message from Authorizer to clients

2021-03-04 Thread YI-CHEN WANG (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YI-CHEN WANG reassigned KAFKA-12404: Assignee: YI-CHEN WANG > Provision to propagate error message from Authorizer to clients

[GitHub] [kafka] dajac commented on pull request #10269: KAFKA-12410 KafkaAPis ought to group fetch data before generating fet…

2021-03-04 Thread GitBox
dajac commented on pull request #10269: URL: https://github.com/apache/kafka/pull/10269#issuecomment-791199822 Nice PR! I will take a look at it on Monday. This is an automated message from the Apache Git Service. To respond

[jira] [Updated] (KAFKA-12416) Group Coordinator followers are failing with OffsetsOutOfOrderException

2021-03-04 Thread Sandeep (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep updated KAFKA-12416: Description: Upon failure of group coordinator, the followers of newly elected group coordinator are fail

[GitHub] [kafka] rodesai commented on a change in pull request #10266: KAFKA-10357: Add validation method for internal topics

2021-03-04 Thread GitBox
rodesai commented on a change in pull request #10266: URL: https://github.com/apache/kafka/pull/10266#discussion_r588053733 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java ## @@ -91,6 +104,225 @@ public InternalTopicMa

[jira] [Updated] (KAFKA-12416) Group Coordinator followers are failing with OffsetsOutOfOrderException

2021-03-04 Thread Sandeep (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep updated KAFKA-12416: Description: Upon failure of group coordinator, the followers of newly elected group coordinator are fail

[jira] [Commented] (KAFKA-12428) Add a last-heartbeat-seconds-ago metric to Kafka Consumer

2021-03-04 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17295791#comment-17295791 ] Guozhang Wang commented on KAFKA-12428: --- Thinking about this a bit, maybe it's bet

[GitHub] [kafka] chia7712 commented on a change in pull request #10269: KAFKA-12410 KafkaAPis ought to group fetch data before generating fet…

2021-03-04 Thread GitBox
chia7712 commented on a change in pull request #10269: URL: https://github.com/apache/kafka/pull/10269#discussion_r588050163 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -795,7 +795,17 @@ class KafkaApis(val requestChannel: RequestChannel, // the

[GitHub] [kafka] chia7712 commented on a change in pull request #10269: KAFKA-12410 KafkaAPis ought to group fetch data before generating fet…

2021-03-04 Thread GitBox
chia7712 commented on a change in pull request #10269: URL: https://github.com/apache/kafka/pull/10269#discussion_r588049874 ## File path: core/src/main/scala/kafka/server/FetchSession.scala ## @@ -294,14 +294,24 @@ trait FetchContext extends Logging { */ def updateAndG

[GitHub] [kafka] chia7712 commented on a change in pull request #10269: KAFKA-12410 KafkaAPis ought to group fetch data before generating fet…

2021-03-04 Thread GitBox
chia7712 commented on a change in pull request #10269: URL: https://github.com/apache/kafka/pull/10269#discussion_r588049196 ## File path: core/src/main/scala/kafka/server/FetchSession.scala ## @@ -411,54 +436,40 @@ class IncrementalFetchContext(private val time: Time, }

[GitHub] [kafka] chia7712 opened a new pull request #10269: KAFKA-12410 KafkaAPis ought to group fetch data before generating fet…

2021-03-04 Thread GitBox
chia7712 opened a new pull request #10269: URL: https://github.com/apache/kafka/pull/10269 The fetch data generated by `KafkaApis` is re-grouped when it is converted to `FetchResponse`. That is unnecessary since `KafkaApis` can keep a grouped collection for fetch data. The other main chang

[jira] [Updated] (KAFKA-12410) KafkaAPis ought to group fetch data before generating fetch response

2021-03-04 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated KAFKA-12410: --- Summary: KafkaAPis ought to group fetch data before generating fetch response (was: Remove

[jira] [Resolved] (KAFKA-12412) Group Coordinator followers are failing with OffsetsOutOfOrderException

2021-03-04 Thread Sandeep (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep resolved KAFKA-12412. - Resolution: Fixed > Group Coordinator followers are failing with OffsetsOutOfOrderException > --

[jira] [Reopened] (KAFKA-12416) Group Coordinator followers are failing with OffsetsOutOfOrderException

2021-03-04 Thread Sandeep (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep reopened KAFKA-12416: - > Group Coordinator followers are failing with OffsetsOutOfOrderException >

[GitHub] [kafka] highluck commented on a change in pull request #9851: KAFKA-10769 Remove JoinGroupRequest#containsValidPattern as it is dup…

2021-03-04 Thread GitBox
highluck commented on a change in pull request #9851: URL: https://github.com/apache/kafka/pull/9851#discussion_r588027499 ## File path: clients/src/main/java/org/apache/kafka/common/requests/JoinGroupRequest.java ## @@ -59,38 +59,12 @@ public String toString() { public s

[GitHub] [kafka] highluck commented on a change in pull request #9851: KAFKA-10769 Remove JoinGroupRequest#containsValidPattern as it is dup…

2021-03-04 Thread GitBox
highluck commented on a change in pull request #9851: URL: https://github.com/apache/kafka/pull/9851#discussion_r588027499 ## File path: clients/src/main/java/org/apache/kafka/common/requests/JoinGroupRequest.java ## @@ -59,38 +59,12 @@ public String toString() { public s

[GitHub] [kafka] chia7712 commented on a change in pull request #9851: KAFKA-10769 Remove JoinGroupRequest#containsValidPattern as it is dup…

2021-03-04 Thread GitBox
chia7712 commented on a change in pull request #9851: URL: https://github.com/apache/kafka/pull/9851#discussion_r588025352 ## File path: clients/src/main/java/org/apache/kafka/common/requests/JoinGroupRequest.java ## @@ -59,38 +59,12 @@ public String toString() { public s

[GitHub] [kafka] highluck commented on a change in pull request #9851: KAFKA-10769 Remove JoinGroupRequest#containsValidPattern as it is dup…

2021-03-04 Thread GitBox
highluck commented on a change in pull request #9851: URL: https://github.com/apache/kafka/pull/9851#discussion_r588023830 ## File path: clients/src/main/java/org/apache/kafka/common/requests/JoinGroupRequest.java ## @@ -59,38 +59,12 @@ public String toString() { public s

[GitHub] [kafka] rohitrmd commented on a change in pull request #9744: KAFKA-10062: Add a method to retrieve the current timestamp as known by the Streams app

2021-03-04 Thread GitBox
rohitrmd commented on a change in pull request #9744: URL: https://github.com/apache/kafka/pull/9744#discussion_r587996458 ## File path: streams/test-utils/src/main/java/org/apache/kafka/streams/processor/MockProcessorContext.java ## @@ -353,11 +371,19 @@ public void setHeader

[jira] [Commented] (KAFKA-12313) Consider deprecating the default.windowed.serde.inner.class configs

2021-03-04 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17295740#comment-17295740 ] Sagar Rao commented on KAFKA-12313: --- hey [~ableegoldman], is this something that I can

[GitHub] [kafka] chia7712 commented on a change in pull request #9851: KAFKA-10769 Remove JoinGroupRequest#containsValidPattern as it is dup…

2021-03-04 Thread GitBox
chia7712 commented on a change in pull request #9851: URL: https://github.com/apache/kafka/pull/9851#discussion_r588014280 ## File path: clients/src/main/java/org/apache/kafka/common/requests/JoinGroupRequest.java ## @@ -59,38 +59,12 @@ public String toString() { public s

[GitHub] [kafka] abbccdda commented on a change in pull request #10240: KAFKA-12381: remove live broker checks for forwarding topic creation

2021-03-04 Thread GitBox
abbccdda commented on a change in pull request #10240: URL: https://github.com/apache/kafka/pull/10240#discussion_r588010801 ## File path: tests/kafkatest/tests/core/security_test.py ## @@ -65,7 +65,7 @@ def test_client_ssl_endpoint_validation_failure(self, security_protocol,

[GitHub] [kafka] vamossagar12 commented on pull request #9779: KAFKA-10767: Adding test cases for all, reverseAll and reverseRange for ThreadCache

2021-03-04 Thread GitBox
vamossagar12 commented on pull request #9779: URL: https://github.com/apache/kafka/pull/9779#issuecomment-791122717 @cadonna , could you plz review this PR as well whenevr you get the chance? This is an automated message from

[GitHub] [kafka] showuon opened a new pull request #10268: MINOR: output meaningful error message

2021-03-04 Thread GitBox
showuon opened a new pull request #10268: URL: https://github.com/apache/kafka/pull/10268 Saw the test failed messages [here](https://ci-builds.apache.org/job/Kafka/job/kafka-trunk-jdk11/565/testReport/junit/org.apache.kafka.connect.mirror.integration/MirrorConnectorsIntegrationTest/testRep

[GitHub] [kafka] rohitrmd commented on a change in pull request #9744: KAFKA-10062: Add a method to retrieve the current timestamp as known by the Streams app

2021-03-04 Thread GitBox
rohitrmd commented on a change in pull request #9744: URL: https://github.com/apache/kafka/pull/9744#discussion_r587996458 ## File path: streams/test-utils/src/main/java/org/apache/kafka/streams/processor/MockProcessorContext.java ## @@ -353,11 +371,19 @@ public void setHeader

[GitHub] [kafka] dengziming commented on pull request #10021: KAFKA-12205: Delete snapshots less than the snapshot at the log start

2021-03-04 Thread GitBox
dengziming commented on pull request #10021: URL: https://github.com/apache/kafka/pull/10021#issuecomment-791092389 Rebase onto trunk, ping @hachikuji . This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] hachikuji commented on a change in pull request #10240: KAFKA-12381: remove live broker checks for forwarding topic creation

2021-03-04 Thread GitBox
hachikuji commented on a change in pull request #10240: URL: https://github.com/apache/kafka/pull/10240#discussion_r587955277 ## File path: tests/kafkatest/tests/core/security_test.py ## @@ -65,7 +65,7 @@ def test_client_ssl_endpoint_validation_failure(self, security_protocol,

[jira] [Created] (KAFKA-12428) Add a last-heartbeat-seconds-ago metric to Kafka Consumer

2021-03-04 Thread Guozhang Wang (Jira)
Guozhang Wang created KAFKA-12428: - Summary: Add a last-heartbeat-seconds-ago metric to Kafka Consumer Key: KAFKA-12428 URL: https://issues.apache.org/jira/browse/KAFKA-12428 Project: Kafka I

[jira] [Updated] (KAFKA-12428) Add a last-heartbeat-seconds-ago metric to Kafka Consumer

2021-03-04 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-12428: -- Component/s: streams > Add a last-heartbeat-seconds-ago metric to Kafka Consumer > ---

[GitHub] [kafka] splett2 commented on pull request #10267: KAFKA-12427: Don't update connection idle time for muted connections

2021-03-04 Thread GitBox
splett2 commented on pull request #10267: URL: https://github.com/apache/kafka/pull/10267#issuecomment-791045318 cc @rajinisivaram for reviews/thoughts This approach is fairly simple, and we still end up calling `pollSelectionKeys()` with muted channels. An alternative that I was thi

[GitHub] [kafka] splett2 opened a new pull request #10267: KAFKA-12427: Don't update connection idle time for muted connections

2021-03-04 Thread GitBox
splett2 opened a new pull request #10267: URL: https://github.com/apache/kafka/pull/10267 `Selector.poll()` will always call `pollSelectionKeys()` for channels with buffered data. `pollSelectionKeys()` will always update connection last idle time, even if the channel is muted and we don't

[GitHub] [kafka] hachikuji commented on pull request #10252: KAFKA-12403; Ensure local state deleted on `RemoveTopicRecord` received

2021-03-04 Thread GitBox
hachikuji commented on pull request #10252: URL: https://github.com/apache/kafka/pull/10252#issuecomment-791038017 Well, I _was_ going to write some integration tests, but it seems we are still awaiting some infrastructure for that. I tested it out manually and it works correctly. I see th

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10211: KAFKA-12347: updating TaskMetadata

2021-03-04 Thread GitBox
wcarlson5 commented on a change in pull request #10211: URL: https://github.com/apache/kafka/pull/10211#discussion_r587907372 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java ## @@ -1136,6 +1139,33 @@ public boolean commitNeeded(

[GitHub] [kafka] hachikuji commented on pull request #10252: KAFKA-12403; Ensure local state deleted on `RemoveTopicRecord` received

2021-03-04 Thread GitBox
hachikuji commented on pull request #10252: URL: https://github.com/apache/kafka/pull/10252#issuecomment-791027372 I am working on some integration tests here. Hopefully then we can wrap this up. This is an automated message

[jira] [Updated] (KAFKA-12426) Missing logic to create partition.metadata files in RaftReplicaManager

2021-03-04 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-12426: Description: As part of KIP-516, the broker should create a partition.metadata file for ea

[GitHub] [kafka] abbccdda commented on a change in pull request #10211: KAFKA-12347: updating TaskMetadata

2021-03-04 Thread GitBox
abbccdda commented on a change in pull request #10211: URL: https://github.com/apache/kafka/pull/10211#discussion_r587906332 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java ## @@ -1136,6 +1139,33 @@ public boolean commitNeeded()

[jira] [Created] (KAFKA-12427) Broker does not close muted idle connections with buffered data

2021-03-04 Thread David Mao (Jira)
David Mao created KAFKA-12427: - Summary: Broker does not close muted idle connections with buffered data Key: KAFKA-12427 URL: https://issues.apache.org/jira/browse/KAFKA-12427 Project: Kafka Is

[jira] [Updated] (KAFKA-12426) Missing logic to create partition.metadata files in RaftReplicaManager

2021-03-04 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-12426: Issue Type: Bug (was: Improvement) > Missing logic to create partition.metadata files in

[jira] [Created] (KAFKA-12426) Missing logic to create partition.metadata files in RaftReplicaManager

2021-03-04 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-12426: --- Summary: Missing logic to create partition.metadata files in RaftReplicaManager Key: KAFKA-12426 URL: https://issues.apache.org/jira/browse/KAFKA-12426 Project:

[jira] [Created] (KAFKA-12425) Connector status report could include a timestamp

2021-03-04 Thread James C. McPherson (Jira)
James C. McPherson created KAFKA-12425: -- Summary: Connector status report could include a timestamp Key: KAFKA-12425 URL: https://issues.apache.org/jira/browse/KAFKA-12425 Project: Kafka

[GitHub] [kafka] ableegoldman commented on pull request #10251: MINOR: add missing docs for record-e2e-latency metrics

2021-03-04 Thread GitBox
ableegoldman commented on pull request #10251: URL: https://github.com/apache/kafka/pull/10251#issuecomment-790998614 Merged to trunk and cherrypicked to 2.7 and 2.8 (cc @vvcephei , it's docs only) This is an automated messa

[GitHub] [kafka] ableegoldman merged pull request #10251: MINOR: add missing docs for record-e2e-latency metrics

2021-03-04 Thread GitBox
ableegoldman merged pull request #10251: URL: https://github.com/apache/kafka/pull/10251 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [kafka] ableegoldman commented on pull request #10251: MINOR: add missing docs for record-e2e-latency metrics

2021-03-04 Thread GitBox
ableegoldman commented on pull request #10251: URL: https://github.com/apache/kafka/pull/10251#issuecomment-790997202 Looks like all three Java builds passed, but some of the other builds failed (with timeout I believe). This is a docs-only change anyway so I think it's safe to merge ---

[GitHub] [kafka] hachikuji merged pull request #10256: MINOR: Raft max batch size needs to propagate to log config

2021-03-04 Thread GitBox
hachikuji merged pull request #10256: URL: https://github.com/apache/kafka/pull/10256 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10211: KAFKA-12347: updating TaskMetadata

2021-03-04 Thread GitBox
wcarlson5 commented on a change in pull request #10211: URL: https://github.com/apache/kafka/pull/10211#discussion_r587874211 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java ## @@ -3112,6 +3122,26 @@ void setChangelogOffset

[jira] [Resolved] (KAFKA-12409) Leaking gauge in ReplicaManager

2021-03-04 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-12409. Resolution: Fixed > Leaking gauge in ReplicaManager >

[jira] [Updated] (KAFKA-12409) Leaking gauge in ReplicaManager

2021-03-04 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12409: --- Fix Version/s: 2.6.2 2.7.1 3.0.0 > Lea

[GitHub] [kafka] ableegoldman commented on pull request #10260: KAFKA-12409: Leaking gauge in ReplicaManager

2021-03-04 Thread GitBox
ableegoldman commented on pull request #10260: URL: https://github.com/apache/kafka/pull/10260#issuecomment-790990198 Cherrypicked to 2.7 and 2.6 This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [kafka] ableegoldman commented on pull request #10260: KAFKA-12409: Leaking gauge in ReplicaManager

2021-03-04 Thread GitBox
ableegoldman commented on pull request #10260: URL: https://github.com/apache/kafka/pull/10260#issuecomment-790988978 Merged to trunk. Will also cherrypick to 2.7 and 2.6 since I'm still in the process of cutting the 2.6.2 RC -- you should check with @vvcephei where we are in the 2.8 relea

[GitHub] [kafka] ableegoldman merged pull request #10260: KAFKA-12409: Leaking gauge in ReplicaManager

2021-03-04 Thread GitBox
ableegoldman merged pull request #10260: URL: https://github.com/apache/kafka/pull/10260 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10211: KAFKA-12347: updating TaskMetadata

2021-03-04 Thread GitBox
wcarlson5 commented on a change in pull request #10211: URL: https://github.com/apache/kafka/pull/10211#discussion_r587869809 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java ## @@ -266,13 +272,13 @@ public void suspend() {

[GitHub] [kafka] hachikuji commented on a change in pull request #10252: KAFKA-12403; Ensure local state deleted on `RemoveTopicRecord` received

2021-03-04 Thread GitBox
hachikuji commented on a change in pull request #10252: URL: https://github.com/apache/kafka/pull/10252#discussion_r587857619 ## File path: core/src/main/scala/kafka/server/metadata/BrokerMetadataListener.scala ## @@ -142,28 +145,19 @@ class BrokerMetadataListener(brokerId: In

[GitHub] [kafka] hachikuji commented on a change in pull request #10252: KAFKA-12403; Ensure local state deleted on `RemoveTopicRecord` received

2021-03-04 Thread GitBox
hachikuji commented on a change in pull request #10252: URL: https://github.com/apache/kafka/pull/10252#discussion_r587850882 ## File path: core/src/main/scala/kafka/server/metadata/MetadataPartitions.scala ## @@ -171,25 +200,38 @@ class MetadataPartitionsBuilder(val brokerId:

[jira] [Created] (KAFKA-12424) Separate build logging in jenkins

2021-03-04 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-12424: --- Summary: Separate build logging in jenkins Key: KAFKA-12424 URL: https://issues.apache.org/jira/browse/KAFKA-12424 Project: Kafka Issue Type: Improveme

[GitHub] [kafka] hachikuji commented on a change in pull request #10256: MINOR: Raft max batch size needs to propagate to log config

2021-03-04 Thread GitBox
hachikuji commented on a change in pull request #10256: URL: https://github.com/apache/kafka/pull/10256#discussion_r587835595 ## File path: core/src/main/scala/kafka/raft/RaftManager.scala ## @@ -220,25 +220,14 @@ class KafkaRaftManager[T]( } private def buildMetadataLo

[jira] [Created] (KAFKA-12423) Consider which log configs should be overridable for the metadata topic

2021-03-04 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-12423: --- Summary: Consider which log configs should be overridable for the metadata topic Key: KAFKA-12423 URL: https://issues.apache.org/jira/browse/KAFKA-12423 Project

[GitHub] [kafka] abbccdda commented on a change in pull request #10211: KAFKA-12347: updating TaskMetadata

2021-03-04 Thread GitBox
abbccdda commented on a change in pull request #10211: URL: https://github.com/apache/kafka/pull/10211#discussion_r587832660 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java ## @@ -3112,6 +3122,26 @@ void setChangelogOffsets

[jira] [Created] (KAFKA-12422) Add getter for KafkaConsumer configs

2021-03-04 Thread Max Rothman (Jira)
Max Rothman created KAFKA-12422: --- Summary: Add getter for KafkaConsumer configs Key: KAFKA-12422 URL: https://issues.apache.org/jira/browse/KAFKA-12422 Project: Kafka Issue Type: Improvement

[jira] [Created] (KAFKA-12421) Improve controller's atomic grouping

2021-03-04 Thread Jose Armando Garcia Sancio (Jira)
Jose Armando Garcia Sancio created KAFKA-12421: -- Summary: Improve controller's atomic grouping Key: KAFKA-12421 URL: https://issues.apache.org/jira/browse/KAFKA-12421 Project: Kafka

[GitHub] [kafka] mumrah commented on a change in pull request #10256: MINOR: Raft max batch size needs to propagate to log config

2021-03-04 Thread GitBox
mumrah commented on a change in pull request #10256: URL: https://github.com/apache/kafka/pull/10256#discussion_r587822370 ## File path: core/src/test/scala/kafka/raft/KafkaMetadataLogTest.scala ## @@ -310,44 +288,109 @@ final class KafkaMetadataLogTest { log.close() -

[GitHub] [kafka] cadonna commented on a change in pull request #10266: KAFKA-10357: Add validation method for internal topics

2021-03-04 Thread GitBox
cadonna commented on a change in pull request #10266: URL: https://github.com/apache/kafka/pull/10266#discussion_r587817305 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/DescribeConfigsResult.java ## @@ -36,7 +36,7 @@ private final Map> futures; -

[GitHub] [kafka] cadonna commented on pull request #10266: KAFKA-10357: Add validation method for internal topics

2021-03-04 Thread GitBox
cadonna commented on pull request #10266: URL: https://github.com/apache/kafka/pull/10266#issuecomment-790933530 Call for review: @mjsax @rodesai @ableegoldman @guozhangwang This is an automated message from the Apache Git S

[GitHub] [kafka] cadonna opened a new pull request #10266: KAFKA-10357: Add validation method for internal topics

2021-03-04 Thread GitBox
cadonna opened a new pull request #10266: URL: https://github.com/apache/kafka/pull/10266 For KIP-698, we need a way to validate internal topics before we create them. This PR adds a validation method to the InternalTopicManager for that purpose. ### Committer Checklist (exclude

[GitHub] [kafka] hachikuji commented on pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

2021-03-04 Thread GitBox
hachikuji commented on pull request #9590: URL: https://github.com/apache/kafka/pull/9590#issuecomment-790932209 @jolshan The active segment would not be replaced during cleaning at a minimum. I think the recovery logic we want is to scan backwards through the `.swap` files and collect the

[GitHub] [kafka] lostiniceland closed pull request #2882: KAFKA-5008: Provide OSGi metadata for Kafka-Clients

2021-03-04 Thread GitBox
lostiniceland closed pull request #2882: URL: https://github.com/apache/kafka/pull/2882 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] Iskuskov commented on a change in pull request #9541: KAFKA-10675: Add schema name to ConnectSchema.validateValue() error message

2021-03-04 Thread GitBox
Iskuskov commented on a change in pull request #9541: URL: https://github.com/apache/kafka/pull/9541#discussion_r587811360 ## File path: connect/api/src/main/java/org/apache/kafka/connect/data/ConnectSchema.java ## @@ -266,10 +261,19 @@ public static void validateValue(String

[jira] [Created] (KAFKA-12420) Kafka network Selector class has many constructors; use a Builder pattern instead

2021-03-04 Thread Steve Rodrigues (Jira)
Steve Rodrigues created KAFKA-12420: --- Summary: Kafka network Selector class has many constructors; use a Builder pattern instead Key: KAFKA-12420 URL: https://issues.apache.org/jira/browse/KAFKA-12420

[GitHub] [kafka] hachikuji commented on a change in pull request #10256: MINOR: Raft max batch size needs to propagate to log config

2021-03-04 Thread GitBox
hachikuji commented on a change in pull request #10256: URL: https://github.com/apache/kafka/pull/10256#discussion_r587809203 ## File path: core/src/test/scala/kafka/raft/KafkaMetadataLogTest.scala ## @@ -310,44 +288,109 @@ final class KafkaMetadataLogTest { log.close()

[GitHub] [kafka] hachikuji commented on a change in pull request #10256: MINOR: Raft max batch size needs to propagate to log config

2021-03-04 Thread GitBox
hachikuji commented on a change in pull request #10256: URL: https://github.com/apache/kafka/pull/10256#discussion_r587803031 ## File path: core/src/main/scala/kafka/raft/RaftManager.scala ## @@ -220,25 +220,14 @@ class KafkaRaftManager[T]( } private def buildMetadataLo

[GitHub] [kafka] jeqo opened a new pull request #10265: To record value

2021-03-04 Thread GitBox
jeqo opened a new pull request #10265: URL: https://github.com/apache/kafka/pull/10265 Draft implementation of [KIP-634](https://cwiki.apache.org/confluence/display/KAFKA/KIP-634%3A+Complementary+support+for+headers+and+record+metadata+in+Kafka+Streams+DSL) ### Committer Checklist (e

[GitHub] [kafka] hachikuji merged pull request #10264: HOTFIX: Controller topic deletion should be atomic

2021-03-04 Thread GitBox
hachikuji merged pull request #10264: URL: https://github.com/apache/kafka/pull/10264 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [kafka] hachikuji commented on pull request #10264: HOTFIX: Controller topic deletion should be atomic

2021-03-04 Thread GitBox
hachikuji commented on pull request #10264: URL: https://github.com/apache/kafka/pull/10264#issuecomment-790911623 I will merge to trunk. The metadata tests are passing locally. This is an automated message from the Apache Gi

[GitHub] [kafka] hachikuji opened a new pull request #10264: HOTFIX: Controller topic deletion should be atomic

2021-03-04 Thread GitBox
hachikuji opened a new pull request #10264: URL: https://github.com/apache/kafka/pull/10264 We merged https://github.com/apache/kafka/pull/10253 and https://github.com/apache/kafka/pull/10184 at about the same time, which caused a build error. Topic deletions should be atomic. ### C

[GitHub] [kafka] hachikuji commented on a change in pull request #10256: MINOR: Raft max batch size needs to propagate to log config

2021-03-04 Thread GitBox
hachikuji commented on a change in pull request #10256: URL: https://github.com/apache/kafka/pull/10256#discussion_r58995 ## File path: core/src/test/scala/kafka/raft/KafkaMetadataLogTest.scala ## @@ -310,44 +288,109 @@ final class KafkaMetadataLogTest { log.close()

[GitHub] [kafka] hachikuji commented on a change in pull request #10256: MINOR: Raft max batch size needs to propagate to log config

2021-03-04 Thread GitBox
hachikuji commented on a change in pull request #10256: URL: https://github.com/apache/kafka/pull/10256#discussion_r587774537 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -297,10 +288,43 @@ final class KafkaMetadataLog private ( } object KafkaMe

[GitHub] [kafka] mumrah commented on a change in pull request #10256: MINOR: Raft max batch size needs to propagate to log config

2021-03-04 Thread GitBox
mumrah commented on a change in pull request #10256: URL: https://github.com/apache/kafka/pull/10256#discussion_r587767322 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -297,10 +288,43 @@ final class KafkaMetadataLog private ( } object KafkaMetad

[jira] [Commented] (KAFKA-9831) Failing test: EosIntegrationTest.shouldNotViolateEosIfOneTaskFailsWithState[exactly_once_beta]

2021-03-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17295547#comment-17295547 ] Matthias J. Sax commented on KAFKA-9831: Absolutely! Reassigning. > Failing test

[jira] [Assigned] (KAFKA-9831) Failing test: EosIntegrationTest.shouldNotViolateEosIfOneTaskFailsWithState[exactly_once_beta]

2021-03-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-9831: -- Assignee: Luke Chen (was: Matthias J. Sax) > Failing test: > EosIntegrationTest.shou

[GitHub] [kafka] hachikuji merged pull request #10184: MINOR: enable topic deletion in the KIP-500 controller

2021-03-04 Thread GitBox
hachikuji merged pull request #10184: URL: https://github.com/apache/kafka/pull/10184 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[jira] [Resolved] (KAFKA-12414) Group Coordinator followers are failing with OffsetsOutOfOrderException

2021-03-04 Thread Sandeep (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep resolved KAFKA-12414. - Resolution: Duplicate > Group Coordinator followers are failing with OffsetsOutOfOrderException > --

[jira] [Resolved] (KAFKA-12413) Group Coordinator followers are failing with OffsetsOutOfOrderException

2021-03-04 Thread Sandeep (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep resolved KAFKA-12413. - Resolution: Duplicate > Group Coordinator followers are failing with OffsetsOutOfOrderException > --

[jira] [Resolved] (KAFKA-12416) Group Coordinator followers are failing with OffsetsOutOfOrderException

2021-03-04 Thread Sandeep (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep resolved KAFKA-12416. - Resolution: Duplicate > Group Coordinator followers are failing with OffsetsOutOfOrderException > --

[jira] [Resolved] (KAFKA-12415) Prepare for Gradle 7.0 and restrict transitive scope for non api dependencies

2021-03-04 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-12415. - Resolution: Fixed > Prepare for Gradle 7.0 and restrict transitive scope for non api dependencie

[jira] [Resolved] (KAFKA-12411) Group Coordinator Followers failing with OutOfOrderOffsetException

2021-03-04 Thread Sandeep (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep resolved KAFKA-12411. - Resolution: Duplicate > Group Coordinator Followers failing with OutOfOrderOffsetException > ---

[GitHub] [kafka] ijuma merged pull request #10203: KAFKA-12415 Prepare for Gradle 7.0 and restrict transitive scope for non api dependencies

2021-03-04 Thread GitBox
ijuma merged pull request #10203: URL: https://github.com/apache/kafka/pull/10203 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [kafka] ijuma commented on pull request #10203: KAFKA-12415 Prepare for Gradle 7.0 and restrict transitive scope for non api dependencies

2021-03-04 Thread GitBox
ijuma commented on pull request #10203: URL: https://github.com/apache/kafka/pull/10203#issuecomment-790865459 No test failures, but one of the jobs was killed in the last run: https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-10203/19/testReport/ The previous runs confir

[GitHub] [kafka] hachikuji commented on pull request #10184: MINOR: enable topic deletion in the KIP-500 controller

2021-03-04 Thread GitBox
hachikuji commented on pull request #10184: URL: https://github.com/apache/kafka/pull/10184#issuecomment-790862249 Thanks for reviews. I will merge this on behalf of @cmccabe to trunk and 2.8. This is an automated message fro

[GitHub] [kafka] jolshan commented on pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

2021-03-04 Thread GitBox
jolshan commented on pull request #9590: URL: https://github.com/apache/kafka/pull/9590#issuecomment-790855545 @hachikuji In the example above, we could also have something like `35.swap` too, right, since we are updating the baseOffset of each segment. (Which means we will need to look at

[GitHub] [kafka] mumrah merged pull request #10253: KAFKA-12376: Apply atomic append to the log

2021-03-04 Thread GitBox
mumrah merged pull request #10253: URL: https://github.com/apache/kafka/pull/10253 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] hachikuji commented on pull request #10256: MINOR: Raft max batch size needs to propagate to log config

2021-03-04 Thread GitBox
hachikuji commented on pull request #10256: URL: https://github.com/apache/kafka/pull/10256#issuecomment-790833577 @ijuma Yes, probably. I'm working on it. This is an automated message from the Apache Git Service. To respond

[jira] [Updated] (KAFKA-12419) Remove Deprecated APIs of Kafka Streams in 3.0

2021-03-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12419: Labels: needs-kip (was: ) > Remove Deprecated APIs of Kafka Streams in 3.0 >

[GitHub] [kafka] ijuma commented on a change in pull request #10252: KAFKA-12403; Ensure local state deleted on `RemoveTopicRecord` received

2021-03-04 Thread GitBox
ijuma commented on a change in pull request #10252: URL: https://github.com/apache/kafka/pull/10252#discussion_r587677099 ## File path: metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java ## @@ -483,7 +475,7 @@ private ApiError createTopic(Creatab

  1   2   >