[GitHub] [kafka] chia7712 commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-10-29 Thread GitBox
chia7712 commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r514903001 ## File path: clients/src/main/resources/common/message/ProduceRequest.json ## @@ -33,21 +33,21 @@ "validVersions": "0-8", "flexibleVersions": "none"

[GitHub] [kafka] kowshik commented on pull request #9536: MINOR: KIP-584: Remove admin client facility to read features from controller

2020-10-29 Thread GitBox
kowshik commented on pull request #9536: URL: https://github.com/apache/kafka/pull/9536#issuecomment-719268329 cc @abbccdda @cmccabe @junrao -- this PR is ready for review. This is an automated message from the Apache Git Ser

[GitHub] [kafka] kowshik opened a new pull request #9536: MINOR: KIP-584: Remove admin client facility to read features from controller

2020-10-29 Thread GitBox
kowshik opened a new pull request #9536: URL: https://github.com/apache/kafka/pull/9536 In this PR, I have eliminated the facility in `Admin#describeFeatures` API and it's implementation to be able to optionally send a `describeFeatures` request to the controller. This feature was not seen

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514882473 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] ning2008wisc commented on pull request #9224: KAFKA-10304: refactor MM2 integration tests

2020-10-29 Thread GitBox
ning2008wisc commented on pull request #9224: URL: https://github.com/apache/kafka/pull/9224#issuecomment-719201216 A kind reminder for additional feedback and comments, based on my response above This is an automated messag

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514872477 ## File path: core/src/test/scala/unit/kafka/network/RequestConvertToJsonTest.scala ## @@ -0,0 +1,188 @@ +/** + * Licensed to the Apache Software Found

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514619891 ## File path: core/src/test/scala/unit/kafka/network/RequestConvertToJsonTest.scala ## @@ -0,0 +1,188 @@ +/** + * Licensed to the Apache Software Found

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514619891 ## File path: core/src/test/scala/unit/kafka/network/RequestConvertToJsonTest.scala ## @@ -0,0 +1,188 @@ +/** + * Licensed to the Apache Software Found

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514861991 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514861389 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] chia7712 commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-10-29 Thread GitBox
chia7712 commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r514819326 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java ## @@ -210,65 +142,42 @@ public String toString() { }

[GitHub] [kafka] dengziming opened a new pull request #9535: MINOR: remove redundant return statement

2020-10-29 Thread GitBox
dengziming opened a new pull request #9535: URL: https://github.com/apache/kafka/pull/9535 the result of `GroupMetadataManager.storeOffsets` is Unit, so remove the redundant return statement This is an automated message from

[jira] [Updated] (KAFKA-6217) Allow consumers to read messages from LEO

2020-10-29 Thread Ashok Bala (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashok Bala updated KAFKA-6217: -- Description: we have a use case(real time customer facing application) to support high throughput(redu

[GitHub] [kafka] ableegoldman opened a new pull request #9534: KAFKA-10664: Delete existing checkpoint when writing empty offsets

2020-10-29 Thread GitBox
ableegoldman opened a new pull request #9534: URL: https://github.com/apache/kafka/pull/9534 ...otherwise we can get stuck in an endless loop of initializing corrupted offsets, hitting OffsetOutOfRangeException and closing the task, then reviving the task with those same corrupted offsets.

[jira] [Commented] (KAFKA-10635) Streams application fails with OutOfOrderSequenceException after rolling restarts of brokers

2020-10-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223324#comment-17223324 ] Matthias J. Sax commented on KAFKA-10635: - \cc [~hachikuji] [~bob-barrett] > St

[jira] [Created] (KAFKA-10664) Streams fails to overwrite corrupted offsets leading to infinite OffsetOutOfRangeException loop

2020-10-29 Thread A. Sophie Blee-Goldman (Jira)
A. Sophie Blee-Goldman created KAFKA-10664: -- Summary: Streams fails to overwrite corrupted offsets leading to infinite OffsetOutOfRangeException loop Key: KAFKA-10664 URL: https://issues.apache.org/jira/b

[jira] [Comment Edited] (KAFKA-10645) Forwarding a record from a punctuator sometimes it results in a NullPointerException

2020-10-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17222393#comment-17222393 ] Matthias J. Sax edited comment on KAFKA-10645 at 10/30/20, 1:31 AM: --

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514636997 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514636692 ## File path: core/src/main/scala/kafka/network/RequestChannel.scala ## @@ -162,7 +161,7 @@ object RequestChannel extends Logging { } } -

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514635784 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] mjsax merged pull request #9513: MINOR: improve `null` checks for headers

2020-10-29 Thread GitBox
mjsax merged pull request #9513: URL: https://github.com/apache/kafka/pull/9513 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] mjsax commented on pull request #9513: MINOR: improve `null` checks for headers

2020-10-29 Thread GitBox
mjsax commented on pull request #9513: URL: https://github.com/apache/kafka/pull/9513#issuecomment-719085653 > However, it may be overkill since users don't create ConsumerRecord. Yeah, I tend to agree. Would not hurt to add it, but should not be strictly necessary. Will merge this a

[GitHub] [kafka] ableegoldman commented on pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
ableegoldman commented on pull request #9530: URL: https://github.com/apache/kafka/pull/9530#issuecomment-719085324 Cherrypicked to 2.6 & 2.7 cc/ @bbejeck This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] mjsax commented on pull request #9518: KAFKA-10645: Add null check to the array/Iterable values in RecordHeaders constructor

2020-10-29 Thread GitBox
mjsax commented on pull request #9518: URL: https://github.com/apache/kafka/pull/9518#issuecomment-719084829 @showuon Seems we worked on this in parallel. Sorry for that... This is an automated message from the Apache Git Ser

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514621561 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514619891 ## File path: core/src/test/scala/unit/kafka/network/RequestConvertToJsonTest.scala ## @@ -0,0 +1,188 @@ +/** + * Licensed to the Apache Software Found

[GitHub] [kafka] ableegoldman merged pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
ableegoldman merged pull request #9530: URL: https://github.com/apache/kafka/pull/9530 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] ableegoldman commented on pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
ableegoldman commented on pull request #9530: URL: https://github.com/apache/kafka/pull/9530#issuecomment-719074084 One flaky test failed: `EosBetaUpgradeIntegrationTest.shouldUpgradeFromEosAlphaToEosBeta[true]` Confirmed systems test passed. Merging to trunk --

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514617626 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514614216 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514610499 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] vvcephei commented on pull request #9467: KAFKA-10515: Properly initialize nullable Serdes with default values

2020-10-29 Thread GitBox
vvcephei commented on pull request #9467: URL: https://github.com/apache/kafka/pull/9467#issuecomment-719063925 Merged to 2.6 (cc @mimaison ) Thanks again for the contribution, @thake ! This is an automated message fro

[jira] [Updated] (KAFKA-10515) NPE: Foreign key join serde may not be initialized with default serde if application is distributed

2020-10-29 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler updated KAFKA-10515: - Fix Version/s: 2.6.1 > NPE: Foreign key join serde may not be initialized with default serde if

[GitHub] [kafka] vvcephei merged pull request #9467: KAFKA-10515: Properly initialize nullable Serdes with default values

2020-10-29 Thread GitBox
vvcephei merged pull request #9467: URL: https://github.com/apache/kafka/pull/9467 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] hachikuji commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r514594596 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java ## @@ -210,65 +142,42 @@ public String toString() { }

[GitHub] [kafka] gitlw opened a new pull request #9533: Show log end offset during truncation to help estimate data loss during ULE

2020-10-29 Thread GitBox
gitlw opened a new pull request #9533: URL: https://github.com/apache/kafka/pull/9533 During Unclean Leader Election, there could be data loss due to truncation at the resigned leader. This PR tries to add more logs to understand the scale of message loss during an unclean leader electi

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514585353 ## File path: raft/src/main/java/org/apache/kafka/raft/RaftClient.java ## @@ -26,30 +24,53 @@ interface Listener { /** - * Callbac

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514585353 ## File path: raft/src/main/java/org/apache/kafka/raft/RaftClient.java ## @@ -26,30 +24,53 @@ interface Listener { /** - * Callbac

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514584469 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -1688,14 +1694,39 @@ private long pollCurrentState(long currentTimeMs)

[GitHub] [kafka] bbejeck commented on a change in pull request #9532: MINOR: Move upgraded docs from site to kafak docs

2020-10-29 Thread GitBox
bbejeck commented on a change in pull request #9532: URL: https://github.com/apache/kafka/pull/9532#discussion_r514580970 ## File path: docs/upgrade.html ## @@ -21,6 +21,48 @@ Notable changes in 2.8.0 +Upgrading to 2.7.0 from any version 0.8.x through 2.6.x Review comme

[GitHub] [kafka] bbejeck commented on pull request #9532: MINOR: Move upgraded docs from site to kafak docs

2020-10-29 Thread GitBox
bbejeck commented on pull request #9532: URL: https://github.com/apache/kafka/pull/9532#issuecomment-719035513 Note that most of this PR already exists in 2.6 and doesn't need a close look This is an automated message from th

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514577992 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514577021 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514576105 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514575372 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514574814 ## File path: core/src/main/scala/kafka/network/RequestChannel.scala ## @@ -162,7 +161,7 @@ object RequestChannel extends Logging { } } -

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331 add a streams handler

2020-10-29 Thread GitBox
wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r514566700 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -346,26 +351,89 @@ public void setStateListener(final KafkaStreams.S

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331 add a streams handler

2020-10-29 Thread GitBox
wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r514567028 ## File path: streams/src/main/java/org/apache/kafka/streams/errors/StreamsUncaughtExceptionHandler.java ## @@ -0,0 +1,45 @@ +/* + * Licensed to the Apac

[GitHub] [kafka] lct45 commented on a change in pull request #9487: KAFKA-9331 add a streams handler

2020-10-29 Thread GitBox
lct45 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r514535737 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -282,6 +284,15 @@ public boolean isRunning() {

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514539737 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -329,8 +387,9 @@ private void appendLeaderChangeMessage(LeaderState stat

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514537694 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -228,35 +234,81 @@ private void updateLeaderEndOffsetAndTimestamp(

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514536828 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -228,35 +234,81 @@ private void updateLeaderEndOffsetAndTimestamp(

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514535761 ## File path: raft/src/main/java/org/apache/kafka/raft/internals/ThresholdPurgatory.java ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514533712 ## File path: raft/src/main/java/org/apache/kafka/raft/internals/RecordsBatchReader.java ## @@ -0,0 +1,210 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] [kafka] lct45 commented on a change in pull request #9487: KAFKA-9331 add a streams handler

2020-10-29 Thread GitBox
lct45 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r514527605 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -346,26 +351,89 @@ public void setStateListener(final KafkaStreams.State

[GitHub] [kafka] bbejeck opened a new pull request #9532: MINOR: Move upgraded docs from site to kafak docs

2020-10-29 Thread GitBox
bbejeck opened a new pull request #9532: URL: https://github.com/apache/kafka/pull/9532 For the 2.7 release, we need to migrate some docs changes that went to `kafka-site` but didn't go into `kafka/docs` This PR covers the `documentation.html` and `uprade.html` changes. Once these

[GitHub] [kafka] hachikuji opened a new pull request #9531: KAFKA-10661; Add new resigned state for graceful shutdown/initialization

2020-10-29 Thread GitBox
hachikuji opened a new pull request #9531: URL: https://github.com/apache/kafka/pull/9531 When initializing the raft state machine after shutting down as a leader, we were previously entering the "unattached" state, which means we have no leader and no voted candidate. This was a bug becau

[GitHub] [kafka] abbccdda merged pull request #9529: Revert initial principal from 2.7

2020-10-29 Thread GitBox
abbccdda merged pull request #9529: URL: https://github.com/apache/kafka/pull/9529 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514518360 ## File path: raft/src/main/java/org/apache/kafka/raft/BatchReader.java ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[jira] [Created] (KAFKA-10663) Flakey test ConsumerBounceTest#testSeekAndCommitWithBrokerFailures

2020-10-29 Thread Boyang Chen (Jira)
Boyang Chen created KAFKA-10663: --- Summary: Flakey test ConsumerBounceTest#testSeekAndCommitWithBrokerFailures Key: KAFKA-10663 URL: https://issues.apache.org/jira/browse/KAFKA-10663 Project: Kafka

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514516570 ## File path: raft/src/main/java/org/apache/kafka/raft/BatchReader.java ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514515763 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/DataInputStreamReadable.java ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Sof

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514514051 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/DataInputStreamReadable.java ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Sof

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514512213 ## File path: clients/src/main/java/org/apache/kafka/common/metrics/stats/Percentiles.java ## @@ -113,11 +113,11 @@ protected HistogramSample newSample(l

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514508755 ## File path: clients/src/main/java/org/apache/kafka/common/requests/AddOffsetsToTxnRequest.java ## @@ -57,6 +57,10 @@ public AddOffsetsToTxnRequest(S

[GitHub] [kafka] cadonna commented on pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
cadonna commented on pull request #9530: URL: https://github.com/apache/kafka/pull/9530#issuecomment-718965450 System tests run: https://jenkins.confluent.io/job/system-test-kafka-branch-builder/4276/ This is an automated me

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514504593 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -1688,14 +1694,39 @@ private long pollCurrentState(long currentTimeMs)

[GitHub] [kafka] cadonna commented on pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
cadonna commented on pull request #9530: URL: https://github.com/apache/kafka/pull/9530#issuecomment-718961222 This fix needs to be cherry-picked to 2.7 and 2.6 This is an automated message from the Apache Git Service. To re

[GitHub] [kafka] cadonna commented on pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
cadonna commented on pull request #9530: URL: https://github.com/apache/kafka/pull/9530#issuecomment-718960604 Call for review: @lct45 @ableegoldman @vvcephei This is an automated message from the Apache Git Service. To resp

[GitHub] [kafka] cadonna opened a new pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
cadonna opened a new pull request #9530: URL: https://github.com/apache/kafka/pull/9530 The system test StreamsUpgradeTest.test_version_probing_upgrade tries to verify the wrong version for version probing. ### Committer Checklist (excluded from commit message) - [ ] Verify desi

[jira] [Created] (KAFKA-10662) Possible hanging test in 2.6 on JDK 11

2020-10-29 Thread John Roesler (Jira)
John Roesler created KAFKA-10662: Summary: Possible hanging test in 2.6 on JDK 11 Key: KAFKA-10662 URL: https://issues.apache.org/jira/browse/KAFKA-10662 Project: Kafka Issue Type: Bug Af

[GitHub] [kafka] senthilm-ms commented on pull request #9495: KAFKA-10642: Expose the real stack trace if any exception occurred during SSL Client Trust Verification in extension

2020-10-29 Thread GitBox
senthilm-ms commented on pull request #9495: URL: https://github.com/apache/kafka/pull/9495#issuecomment-718944686 > @rajinisivaram : What do you think? Thanks. @junrao @rajinisivaram - take a look at the stack trace mentioned in the JIRA bug where you don't see the extension/custom

[GitHub] [kafka] abbccdda commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
abbccdda commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514482551 ## File path: core/src/main/scala/kafka/raft/TimingWheelExpirationService.scala ## @@ -0,0 +1,61 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] junrao commented on pull request #9495: KAFKA-10642: Expose the real stack trace if any exception occurred during SSL Client Trust Verification in extension

2020-10-29 Thread GitBox
junrao commented on pull request #9495: URL: https://github.com/apache/kafka/pull/9495#issuecomment-718942968 @rajinisivaram : What do you think? Thanks. This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] abbccdda commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
abbccdda commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r513200010 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/DataInputStreamReadable.java ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Soft

[GitHub] [kafka] junrao commented on pull request #6915: KAFKA-8334 Executor to retry delayed operations failed to obtain lock

2020-10-29 Thread GitBox
junrao commented on pull request #6915: URL: https://github.com/apache/kafka/pull/6915#issuecomment-718933493 @yinchuanwang : The short answer is that the retry logic could cause a watcher event not to be triggered. If there is no future event triggering the same watcher, the corresponding

[GitHub] [kafka] hachikuji commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r514467652 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java ## @@ -210,65 +142,42 @@ public String toString() { }

[jira] [Created] (KAFKA-10661) Add resigned state to raft state machine to preserve leader/epoch information

2020-10-29 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-10661: --- Summary: Add resigned state to raft state machine to preserve leader/epoch information Key: KAFKA-10661 URL: https://issues.apache.org/jira/browse/KAFKA-10661 P

[jira] [Commented] (KAFKA-9892) Producer state snapshot needs to be forced to disk

2020-10-29 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223094#comment-17223094 ] Jun Rao commented on KAFKA-9892: [~kbrajesh176]: Thanks for your interest. Just added you

[GitHub] [kafka] vvcephei merged pull request #9426: MINOR: Fix flaky shouldRejectNonExistentStoreName

2020-10-29 Thread GitBox
vvcephei merged pull request #9426: URL: https://github.com/apache/kafka/pull/9426 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[jira] [Commented] (KAFKA-9892) Producer state snapshot needs to be forced to disk

2020-10-29 Thread Brajesh Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223070#comment-17223070 ] Brajesh Kumar commented on KAFKA-9892: --- [~junrao] [~ijuma] Can I pick this up pl

[GitHub] [kafka] vvcephei commented on pull request #9426: MINOR: Fix flaky shouldRejectNonExistentStoreName

2020-10-29 Thread GitBox
vvcephei commented on pull request #9426: URL: https://github.com/apache/kafka/pull/9426#issuecomment-718900823 The only build failure was: `Build / JDK 11 / org.apache.kafka.streams.integration.StreamTableJoinTopologyOptimizationIntegrationTest.shouldDoStreamTableJoinWithDifferentNumber

[jira] [Resolved] (KAFKA-10638) QueryableStateIntegrationTest fails due to stricter store checking

2020-10-29 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-10638. -- Resolution: Fixed > QueryableStateIntegrationTest fails due to stricter store checking > -

[jira] [Updated] (KAFKA-10638) QueryableStateIntegrationTest fails due to stricter store checking

2020-10-29 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler updated KAFKA-10638: - Fix Version/s: (was: 2.7.1) (was: 2.8.0) 2.7.0 > Q

[jira] [Commented] (KAFKA-9948) Gradle Issue

2020-10-29 Thread Brajesh Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223044#comment-17223044 ] Brajesh Kumar commented on KAFKA-9948: --- [~dulvinw] Which command you are using to

[jira] [Comment Edited] (KAFKA-9948) Gradle Issue

2020-10-29 Thread Brajesh Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223044#comment-17223044 ] Brajesh Kumar edited comment on KAFKA-9948 at 10/29/20, 5:02 PM: -

[GitHub] [kafka] vvcephei commented on pull request #9414: KAFKA-10585: Kafka Streams should clean up the state store directory from cleanup

2020-10-29 Thread GitBox
vvcephei commented on pull request #9414: URL: https://github.com/apache/kafka/pull/9414#issuecomment-718889494 Hey @dongjinleekr , just a quick check-in: do you plan to take a look at the test failures? This is an automated

[jira] [Commented] (KAFKA-10373) Kafka Reassign Partition is stuck with Java OutOfMemory error

2020-10-29 Thread Brajesh Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223041#comment-17223041 ] Brajesh Kumar commented on KAFKA-10373: Hi What is your Environment configurati

[jira] [Comment Edited] (KAFKA-10373) Kafka Reassign Partition is stuck with Java OutOfMemory error

2020-10-29 Thread Brajesh Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223041#comment-17223041 ] Brajesh Kumar edited comment on KAFKA-10373 at 10/29/20, 4:59 PM: ---

[GitHub] [kafka] vvcephei merged pull request #9521: KAFKA-10638: Fix QueryableStateIntegrationTest

2020-10-29 Thread GitBox
vvcephei merged pull request #9521: URL: https://github.com/apache/kafka/pull/9521 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] vvcephei commented on pull request #9521: KAFKA-10638: Fix QueryableStateIntegrationTest

2020-10-29 Thread GitBox
vvcephei commented on pull request #9521: URL: https://github.com/apache/kafka/pull/9521#issuecomment-718885978 The tests passed, and I answered the two review questions. I'll go ahead and merge. This is an automated message

[GitHub] [kafka] vvcephei commented on pull request #9467: KAFKA-10515: Properly initialize nullable Serdes with default values

2020-10-29 Thread GitBox
vvcephei commented on pull request #9467: URL: https://github.com/apache/kafka/pull/9467#issuecomment-718882348 _Finally_ merged that Jenkinsfile PR (seeing a lot of timeouts, but decided just to go ahead and merge it). Merged in the 2.6 branch, and now the tests are running! We shou

[GitHub] [kafka] vvcephei merged pull request #9471: MINOR: Add Jenkinsfile to 2.6

2020-10-29 Thread GitBox
vvcephei merged pull request #9471: URL: https://github.com/apache/kafka/pull/9471 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[jira] [Created] (KAFKA-10660) Poll time out logstash

2020-10-29 Thread David (Jira)
David created KAFKA-10660: - Summary: Poll time out logstash Key: KAFKA-10660 URL: https://issues.apache.org/jira/browse/KAFKA-10660 Project: Kafka Issue Type: Bug Components: KafkaConnect

[GitHub] [kafka] abbccdda commented on pull request #9529: Revert initial principal from 2.7

2020-10-29 Thread GitBox
abbccdda commented on pull request #9529: URL: https://github.com/apache/kafka/pull/9529#issuecomment-718871156 @bbejeck @hachikuji for a review This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [kafka] abbccdda opened a new pull request #9529: Revert initial principal from 2.7

2020-10-29 Thread GitBox
abbccdda opened a new pull request #9529: URL: https://github.com/apache/kafka/pull/9529 Need to make sure 2.7 doesn't bump the request header unexpectedly. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage

[GitHub] [kafka] abbccdda commented on pull request #9522: MINOR: revert initial principal PR

2020-10-29 Thread GitBox
abbccdda commented on pull request #9522: URL: https://github.com/apache/kafka/pull/9522#issuecomment-718869803 Reopening this PR to merge to 2.7 This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [kafka] bbejeck opened a new pull request #9528: MINOR: Add releaseTarGz to args for building docs

2020-10-29 Thread GitBox
bbejeck opened a new pull request #9528: URL: https://github.com/apache/kafka/pull/9528 This PR adds the `releaseTarGz` command back into the `release.py::command_stage_docs` method when building docs for staging *Summary of testing strategy (including rationale) for the feature o

[GitHub] [kafka] hachikuji commented on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-10-29 Thread GitBox
hachikuji commented on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-718828042 @chia7712 One thing that would be useful is running the producer-performance test, just to make sure the the performance is inline. Might be worth checking flame graphs as well.

[GitHub] [kafka] hachikuji commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r514326022 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java ## @@ -210,65 +142,42 @@ public String toString() { }

  1   2   >