[GitHub] [kafka] dengziming opened a new pull request #10667: KAFKA-12772: Move all transaction state transition rules into their states

2021-05-10 Thread GitBox
dengziming opened a new pull request #10667: URL: https://github.com/apache/kafka/pull/10667 *More detailed description of your change* Similar to KAFKA-5258 which move all partition and replica state transition rules into their states, we move the transaction state transition rules into

[jira] [Created] (KAFKA-12772) Move all TransactionState transition rules into their states

2021-05-10 Thread dengziming (Jira)
dengziming created KAFKA-12772: -- Summary: Move all TransactionState transition rules into their states Key: KAFKA-12772 URL: https://issues.apache.org/jira/browse/KAFKA-12772 Project: Kafka Iss

[jira] [Commented] (KAFKA-12766) Consider Disabling WAL-related Options in RocksDB

2021-05-10 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342337#comment-17342337 ] Bruno Cadonna commented on KAFKA-12766: --- I have to admit that I blindly followed y

[jira] [Commented] (KAFKA-12635) Mirrormaker 2 offset sync is incorrect if the target partition is empty

2021-05-10 Thread Ning Zhang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342308#comment-17342308 ] Ning Zhang commented on KAFKA-12635: great, thanks for the feedback. I will proceed

[jira] [Commented] (KAFKA-12635) Mirrormaker 2 offset sync is incorrect if the target partition is empty

2021-05-10 Thread Frank Yi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342302#comment-17342302 ] Frank Yi commented on KAFKA-12635: -- fix works for me! Thanks > Mirrormaker 2 offset sy

[GitHub] [kafka] ableegoldman opened a new pull request #10666: MINOR: prevent cleanup() from being called while Streams is still shutting down

2021-05-10 Thread GitBox
ableegoldman opened a new pull request #10666: URL: https://github.com/apache/kafka/pull/10666 Currently `KafkaStreams#cleanUp` only throw an IllegalStateException if the state is RUNNING or REBALANCING, however the application could be in the process of shutting down in which case StreamT

[GitHub] [kafka] ijuma commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2

2021-05-10 Thread GitBox
ijuma commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r629848073 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java ## @@ -525,6 +525,11 @@ private TransactionManager configureTrans

[GitHub] [kafka] mjsax commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2

2021-05-10 Thread GitBox
mjsax commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r629845576 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java ## @@ -179,10 +179,18 @@ public void beginTransaction() throws Pro

[GitHub] [kafka] mjsax commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2

2021-05-10 Thread GitBox
mjsax commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r629845081 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java ## @@ -525,6 +525,11 @@ private TransactionManager configureTrans

[jira] [Assigned] (KAFKA-12718) SessionWindows are closed too early

2021-05-10 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-12718: --- Assignee: Juan C. Gonzalez-Zurita > SessionWindows are closed too early > -

[jira] [Commented] (KAFKA-12718) SessionWindows are closed too early

2021-05-10 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342293#comment-17342293 ] Matthias J. Sax commented on KAFKA-12718: - [~gonzur] it seems [~byusti] lost int

[jira] [Resolved] (KAFKA-12760) Delete My Account

2021-05-10 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-12760. - Resolution: Invalid [~byusti] – we cannot delete your account either. You could try to

[jira] [Commented] (KAFKA-12771) CheckStyle attempted upgrade (8.36.2 -->> 8.41.1) summons a pack of 'Indentation' errors

2021-05-10 Thread Dongjin Lee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342288#comment-17342288 ] Dongjin Lee commented on KAFKA-12771: - Hi [~dejan2609], This issue is addressed in K

[GitHub] [kafka] showuon commented on pull request #10665: KAFKA-9009: increase replica.lag.time.max.ms to make the test reliable

2021-05-10 Thread GitBox
showuon commented on pull request #10665: URL: https://github.com/apache/kafka/pull/10665#issuecomment-837734046 @edoardocomar @mimaison , could you help review this PR to make the test reliable. Thank you. -- This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [kafka] showuon opened a new pull request #10665: KAFKA-9009: increase replica.lag.time.max.ms to make the test reliable

2021-05-10 Thread GitBox
showuon opened a new pull request #10665: URL: https://github.com/apache/kafka/pull/10665 We used to set a low `replica.lag.time.max.ms` value (2 sec) to speed up the test, but the 2 sec is not long enough in slow Jenkins env, and caused the follower got kicked out from ISR, so the `UnderR

[jira] [Commented] (KAFKA-12757) Move server related common and public classes into separate module(s).

2021-05-10 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342272#comment-17342272 ] Satish Duggana commented on KAFKA-12757: [~junrao] I am not aware of cases like

[GitHub] [kafka] ableegoldman commented on pull request #10637: MINOR: remove storage/src/generated from tracked files

2021-05-10 Thread GitBox
ableegoldman commented on pull request #10637: URL: https://github.com/apache/kafka/pull/10637#issuecomment-837691327 Is it possible to set up the gitignore to automatically ignore anything under a `generated/` subdirectory? -- This is an automated message from the Apache Git Service. To

[jira] [Assigned] (KAFKA-9009) Flaky Test kafka.integration.MetricsDuringTopicCreationDeletionTest.testMetricsDuringTopicCreateDelete

2021-05-10 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen reassigned KAFKA-9009: Assignee: Luke Chen > Flaky Test > kafka.integration.MetricsDuringTopicCreationDeletionTest.testM

[jira] [Commented] (KAFKA-9009) Flaky Test kafka.integration.MetricsDuringTopicCreationDeletionTest.testMetricsDuringTopicCreateDelete

2021-05-10 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342261#comment-17342261 ] Luke Chen commented on KAFKA-9009: -- investigating > Flaky Test > kafka.integration.Met

[GitHub] [kafka] ableegoldman commented on pull request #10664: KAFKA-12749: Changelog topic config on suppressed KTable lost

2021-05-10 Thread GitBox
ableegoldman commented on pull request #10664: URL: https://github.com/apache/kafka/pull/10664#issuecomment-837607688 cc any of @cadonna @vvcephei @lct45 @wcarlson5 @mjsax @guozhangwang to review this -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] ableegoldman commented on a change in pull request #6592: KAFKA-8326: Introduce List Serde

2021-05-10 Thread GitBox
ableegoldman commented on a change in pull request #6592: URL: https://github.com/apache/kafka/pull/6592#discussion_r629768563 ## File path: clients/src/main/java/org/apache/kafka/common/serialization/Serdes.java ## @@ -265,4 +287,14 @@ public UUIDSerde() { static public

[GitHub] [kafka] vichu commented on pull request #10664: KAFKA-12749: Changelog topic config on suppressed KTable lost

2021-05-10 Thread GitBox
vichu commented on pull request #10664: URL: https://github.com/apache/kafka/pull/10664#issuecomment-837597679 @ableegoldman Would appreciate it if you can take a look at this PR when you get a chance. -- This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [kafka] vichu opened a new pull request #10664: KAFKA-12749: Changelog topic config on suppressed KTable lost

2021-05-10 Thread GitBox
vichu opened a new pull request #10664: URL: https://github.com/apache/kafka/pull/10664 Refactored `logConfig` to be passed appropriately when using `shutDownWhenFull` or `emitEarlyWhenFull`. Removed the constructor that doesn't accept a `logConfig` parameter so you're forced to specify it

[GitHub] [kafka] vitojeng commented on pull request #10657: KAFKA-5876: Apply InvalidStateStorePartitionException for Interactive Queries

2021-05-10 Thread GitBox
vitojeng commented on pull request #10657: URL: https://github.com/apache/kafka/pull/10657#issuecomment-837571301 @ableegoldman Thanks. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [kafka] ijuma commented on pull request #10606: KAFKA-12728: version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0 -->> 7.0.0)

2021-05-10 Thread GitBox
ijuma commented on pull request #10606: URL: https://github.com/apache/kafka/pull/10606#issuecomment-837551412 Looks like there is a change in behavior in Gradle 7 related to resource files that's causing a bunch of tests to fail -- This is an automated message from the Apache Git Servi

[GitHub] [kafka] ableegoldman commented on pull request #10657: KAFKA-5876: Apply InvalidStateStorePartitionException for Interactive Queries

2021-05-10 Thread GitBox
ableegoldman commented on pull request #10657: URL: https://github.com/apache/kafka/pull/10657#issuecomment-837549700 Merged to trunk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [kafka] ableegoldman merged pull request #10657: KAFKA-5876: Apply InvalidStateStorePartitionException for Interactive Queries

2021-05-10 Thread GitBox
ableegoldman merged pull request #10657: URL: https://github.com/apache/kafka/pull/10657 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, p

[GitHub] [kafka] ijuma commented on pull request #10306: MINOR: socket setup max should be 30 seconds

2021-05-10 Thread GitBox
ijuma commented on pull request #10306: URL: https://github.com/apache/kafka/pull/10306#issuecomment-837508654 @cmccabe We should update the KIP and share a note in the mailing list thread. Also, was this released with `127` as the default or did we change it before the first release? If t

[GitHub] [kafka] mdedetrich edited a comment on pull request #10648: KAFKA-9726: Add IdentityReplicationPolicy for MM2

2021-05-10 Thread GitBox
mdedetrich edited a comment on pull request #10648: URL: https://github.com/apache/kafka/pull/10648#issuecomment-837469565 KIP has been created at https://cwiki.apache.org/confluence/display/KAFKA/KIP-737%3A+Add+canTrackSource+to+ReplicationPolicy and a new thread has been started in the k

[GitHub] [kafka] mdedetrich commented on pull request #10648: KAFKA-9726: Add IdentityReplicationPolicy for MM2

2021-05-10 Thread GitBox
mdedetrich commented on pull request #10648: URL: https://github.com/apache/kafka/pull/10648#issuecomment-837469565 KIP has been created at https://cwiki.apache.org/confluence/display/KAFKA/KIP-737%3A+Add+canTrackSource+to+ReplicationPolicy and a new thread has been started in the apache-d

[GitHub] [kafka] dejan2609 commented on pull request #10656: MINOR: checkstyle version upgrade: 8.20 -->> 8.36.2

2021-05-10 Thread GitBox
dejan2609 commented on pull request #10656: URL: https://github.com/apache/kafka/pull/10656#issuecomment-837469070 @romani, @ijuma > Do you want to contribute the change to allow the checkstyle version to be specified via a parameter? Sure, I created this JIRA ticket for my s

[jira] [Updated] (KAFKA-12771) CheckStyle attempted upgrade (8.36.2 -->> 8.41.1) summons a pack of 'Indentation' errors

2021-05-10 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dejan Stojadinović updated KAFKA-12771: --- Description: ^*Prologue*: [https://github.com/apache/kafka/pull/10656#issuecomment-

[jira] [Created] (KAFKA-12771) CheckStyle attempted upgrade (8.36.2 -->> 8.41.1) summons a pack of 'Indentation' errors

2021-05-10 Thread Jira
Dejan Stojadinović created KAFKA-12771: -- Summary: CheckStyle attempted upgrade (8.36.2 -->> 8.41.1) summons a pack of 'Indentation' errors Key: KAFKA-12771 URL: https://issues.apache.org/jira/browse/KAFKA-127

[jira] [Created] (KAFKA-12770) Jenkins build: allow the CheckStyle version to be specified via parameter

2021-05-10 Thread Jira
Dejan Stojadinović created KAFKA-12770: -- Summary: Jenkins build: allow the CheckStyle version to be specified via parameter Key: KAFKA-12770 URL: https://issues.apache.org/jira/browse/KAFKA-12770

[jira] [Updated] (KAFKA-12728) Version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0 -->> 7.0.0)

2021-05-10 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-12728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dejan Stojadinović updated KAFKA-12728: --- Summary: Version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0

[GitHub] [kafka] ijuma commented on a change in pull request #10620: KAFKA-12736: KafkaProducer.flush holds onto completed ProducerBatch(s) until flush completes

2021-05-10 Thread GitBox
ijuma commented on a change in pull request #10620: URL: https://github.com/apache/kafka/pull/10620#discussion_r629703346 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java ## @@ -710,8 +710,11 @@ private boolean appendsInPro

[jira] [Updated] (KAFKA-12768) Mirrormaker2 consumer config not using newly assigned client id

2021-05-10 Thread Vincent (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent updated KAFKA-12768: Attachment: (was: mirroramker2.properties) > Mirrormaker2 consumer config not using newly assigned cli

[jira] [Updated] (KAFKA-12768) Mirrormaker2 consumer config not using newly assigned client id

2021-05-10 Thread Vincent (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent updated KAFKA-12768: Attachment: (was: mirrormaker.log) > Mirrormaker2 consumer config not using newly assigned client id >

[jira] [Updated] (KAFKA-12768) Mirrormaker2 consumer config not using newly assigned client id

2021-05-10 Thread Vincent (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent updated KAFKA-12768: Attachment: (was: instance2_log) > Mirrormaker2 consumer config not using newly assigned client id > -

[jira] [Updated] (KAFKA-12768) Mirrormaker2 consumer config not using newly assigned client id

2021-05-10 Thread Vincent (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent updated KAFKA-12768: Attachment: (was: instance1_log) > Mirrormaker2 consumer config not using newly assigned client id > -

[jira] [Created] (KAFKA-12769) Backport of KAFKA-8562

2021-05-10 Thread Josep Prat (Jira)
Josep Prat created KAFKA-12769: -- Summary: Backport of KAFKA-8562 Key: KAFKA-12769 URL: https://issues.apache.org/jira/browse/KAFKA-12769 Project: Kafka Issue Type: Task Components: net

[jira] [Commented] (KAFKA-12769) Backport of KAFKA-8562

2021-05-10 Thread Josep Prat (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342144#comment-17342144 ] Josep Prat commented on KAFKA-12769: I'll work on this > Backport of KAFKA-8562 > -

[jira] [Updated] (KAFKA-12768) Mirrormaker2 consumer config not using newly assigned client id

2021-05-10 Thread Vincent (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent updated KAFKA-12768: Description: Component: MirrorMaker2 from the 2.6.0 distribution. We tried to set quotas based client.id

[jira] [Created] (KAFKA-12768) Mirrormaker2 consumer config not using newly assigned client id

2021-05-10 Thread Vincent (Jira)
Vincent created KAFKA-12768: --- Summary: Mirrormaker2 consumer config not using newly assigned client id Key: KAFKA-12768 URL: https://issues.apache.org/jira/browse/KAFKA-12768 Project: Kafka Issue

[jira] [Updated] (KAFKA-12766) Consider Disabling WAL-related Options in RocksDB

2021-05-10 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12766: --- Fix Version/s: 3.0.0 > Consider Disabling WAL-related Options in RocksDB > -

[jira] [Commented] (KAFKA-12766) Consider Disabling WAL-related Options in RocksDB

2021-05-10 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342123#comment-17342123 ] A. Sophie Blee-Goldman commented on KAFKA-12766: Personally I agree that

[jira] [Updated] (KAFKA-12747) Flaky Test RocksDBStoreTest.shouldReturnUUIDsWithStringPrefix

2021-05-10 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-12747: -- Fix Version/s: 2.8.1 > Flaky Test RocksDBStoreTest.shouldReturnUUIDsWithStringPrefix > ---

[jira] [Commented] (KAFKA-12749) Changelog topic config on suppressed KTable lost

2021-05-10 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342120#comment-17342120 ] A. Sophie Blee-Goldman commented on KAFKA-12749: Thanks [~vishranganatha

[jira] [Assigned] (KAFKA-12749) Changelog topic config on suppressed KTable lost

2021-05-10 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman reassigned KAFKA-12749: -- Assignee: Viswanathan Ranganathan > Changelog topic config on suppres

[GitHub] [kafka] ableegoldman commented on a change in pull request #10568: KAFKA-8897: Upgrade RocksDB to 6.19.3

2021-05-10 Thread GitBox
ableegoldman commented on a change in pull request #10568: URL: https://github.com/apache/kafka/pull/10568#discussion_r629637264 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBGenericOptionsToDbOptionsColumnFamilyOptionsAdapter.java ## @@ -

[jira] [Assigned] (KAFKA-12747) Flaky Test RocksDBStoreTest.shouldReturnUUIDsWithStringPrefix

2021-05-10 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-12747: - Fix Version/s: 3.0.0 Assignee: Guozhang Wang Resolution: Fixed > Flaky

[GitHub] [kafka] guozhangwang commented on pull request #10643: KAFKA-12747: Fix flakiness in shouldReturnUUIDsWithStringPrefix

2021-05-10 Thread GitBox
guozhangwang commented on pull request #10643: URL: https://github.com/apache/kafka/pull/10643#issuecomment-837205952 Will cherry-pick to old branches as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [kafka] guozhangwang merged pull request #10643: KAFKA-12747: Fix flakiness in shouldReturnUUIDsWithStringPrefix

2021-05-10 Thread GitBox
guozhangwang merged pull request #10643: URL: https://github.com/apache/kafka/pull/10643 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, p

[GitHub] [kafka] g1geordie commented on pull request #10663: KAFKA-12708 Rewrite org.apache.kafka.test.Microbenchmarks by JMH

2021-05-10 Thread GitBox
g1geordie commented on pull request #10663: URL: https://github.com/apache/kafka/pull/10663#issuecomment-837173030 @chia7712 can you help me take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [kafka] g1geordie opened a new pull request #10663: KAFKA-12708 Rewrite org.apache.kafka.test.Microbenchmarks by JMH

2021-05-10 Thread GitBox
g1geordie opened a new pull request #10663: URL: https://github.com/apache/kafka/pull/10663 Rewrite org.apache.kafka.test.Microbenchmarks by JMH -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] wcarlson5 commented on pull request #10634: KAFKA-12754: Improve endOffsets for TaskMetadata

2021-05-10 Thread GitBox
wcarlson5 commented on pull request #10634: URL: https://github.com/apache/kafka/pull/10634#issuecomment-837165677 @ableegoldman Thanks for the review! I think I got to your comments. There was one miss understanding about how the `TaskMetadata` is updated but let me know if you still have

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10634: KAFKA-12754: Improve endOffsets for TaskMetadata

2021-05-10 Thread GitBox
wcarlson5 commented on a change in pull request #10634: URL: https://github.com/apache/kafka/pull/10634#discussion_r629609542 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -900,6 +902,14 @@ private long pollPhase() {

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10634: KAFKA-12754: Improve endOffsets for TaskMetadata

2021-05-10 Thread GitBox
wcarlson5 commented on a change in pull request #10634: URL: https://github.com/apache/kafka/pull/10634#discussion_r629604506 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -1162,6 +1162,15 @@ private void updateTaskMeta

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10634: KAFKA-12754: Improve endOffsets for TaskMetadata

2021-05-10 Thread GitBox
wcarlson5 commented on a change in pull request #10634: URL: https://github.com/apache/kafka/pull/10634#discussion_r629603915 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StandbyTask.java ## @@ -302,6 +302,11 @@ public void updateCommittedOf

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10634: KAFKA-12754: Improve endOffsets for TaskMetadata

2021-05-10 Thread GitBox
wcarlson5 commented on a change in pull request #10634: URL: https://github.com/apache/kafka/pull/10634#discussion_r629599232 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java ## @@ -1193,6 +1196,11 @@ public void updateCommittedO

[jira] [Assigned] (KAFKA-12708) Rewrite org.apache.kafka.test.Microbenchmarks by JMH

2021-05-10 Thread GeordieMai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GeordieMai reassigned KAFKA-12708: -- Assignee: GeordieMai > Rewrite org.apache.kafka.test.Microbenchmarks by JMH > ---

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10634: KAFKA-12754: Improve endOffsets for TaskMetadata

2021-05-10 Thread GitBox
wcarlson5 commented on a change in pull request #10634: URL: https://github.com/apache/kafka/pull/10634#discussion_r629595406 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/TaskMetadataIntegrationTest.java ## @@ -0,0 +1,200 @@ +/* + * Licensed to the

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10634: KAFKA-12754: Improve endOffsets for TaskMetadata

2021-05-10 Thread GitBox
wcarlson5 commented on a change in pull request #10634: URL: https://github.com/apache/kafka/pull/10634#discussion_r629592700 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/TaskMetadataIntegrationTest.java ## @@ -0,0 +1,200 @@ +/* + * Licensed to the

[GitHub] [kafka] mumrah commented on a change in pull request #10431: KAFKA-12543: Change RawSnapshotReader ownership model

2021-05-10 Thread GitBox
mumrah commented on a change in pull request #10431: URL: https://github.com/apache/kafka/pull/10431#discussion_r629541231 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -161,19 +162,24 @@ final class KafkaMetadataLog private ( override def trunc

[GitHub] [kafka] mumrah commented on a change in pull request #10431: KAFKA-12543: Change RawSnapshotReader ownership model

2021-05-10 Thread GitBox
mumrah commented on a change in pull request #10431: URL: https://github.com/apache/kafka/pull/10431#discussion_r629541231 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -161,19 +162,24 @@ final class KafkaMetadataLog private ( override def trunc

[GitHub] [kafka] lbradstreet commented on a change in pull request #10620: KAFKA-12736: KafkaProducer.flush holds onto completed ProducerBatch(s) until flush completes

2021-05-10 Thread GitBox
lbradstreet commented on a change in pull request #10620: URL: https://github.com/apache/kafka/pull/10620#discussion_r629543903 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java ## @@ -710,8 +710,11 @@ private boolean append

[GitHub] [kafka] cadonna commented on a change in pull request #10568: KAFKA-8897: Upgrade RocksDB to 6.19.3

2021-05-10 Thread GitBox
cadonna commented on a change in pull request #10568: URL: https://github.com/apache/kafka/pull/10568#discussion_r629541422 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBGenericOptionsToDbOptionsColumnFamilyOptionsAdapter.java ## @@ -1381,

[GitHub] [kafka] guozhangwang commented on a change in pull request #10568: KAFKA-8897: Upgrade RocksDB to 6.19.3

2021-05-10 Thread GitBox
guozhangwang commented on a change in pull request #10568: URL: https://github.com/apache/kafka/pull/10568#discussion_r629538083 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBGenericOptionsToDbOptionsColumnFamilyOptionsAdapter.java ## @@ -

[jira] [Commented] (KAFKA-12757) Move server related common and public classes into separate module(s).

2021-05-10 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342023#comment-17342023 ] Jun Rao commented on KAFKA-12757: - [~satish.duggana]: The implementation of server side

[GitHub] [kafka] mumrah commented on a change in pull request #10431: KAFKA-12543: Change RawSnapshotReader ownership model

2021-05-10 Thread GitBox
mumrah commented on a change in pull request #10431: URL: https://github.com/apache/kafka/pull/10431#discussion_r629531449 ## File path: raft/src/main/java/org/apache/kafka/snapshot/FileRawSnapshotReader.java ## @@ -54,8 +54,12 @@ public Records records() { } @Over

[GitHub] [kafka] guozhangwang commented on a change in pull request #10643: KAFKA-12747: Fix flakiness in shouldReturnUUIDsWithStringPrefix

2021-05-10 Thread GitBox
guozhangwang commented on a change in pull request #10643: URL: https://github.com/apache/kafka/pull/10643#discussion_r629530576 ## File path: streams/src/test/java/org/apache/kafka/streams/state/internals/RocksDBStoreTest.java ## @@ -451,17 +453,14 @@ public void shouldReturn

[GitHub] [kafka] jlprat commented on pull request #10662: KAFKA-12747: Fix flaky test RocksDBStoreTest.shouldReturnUUIDsWithStr…

2021-05-10 Thread GitBox
jlprat commented on pull request #10662: URL: https://github.com/apache/kafka/pull/10662#issuecomment-836981716 The other way around, I haven't seen your PR, you were first 😃 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [kafka] guozhangwang commented on a change in pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-05-10 Thread GitBox
guozhangwang commented on a change in pull request #10552: URL: https://github.com/apache/kafka/pull/10552#discussion_r629529360 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java ## @@ -387,58 +403,125 @@ private boolea

[GitHub] [kafka] mumrah commented on a change in pull request #10431: KAFKA-12543: Change RawSnapshotReader ownership model

2021-05-10 Thread GitBox
mumrah commented on a change in pull request #10431: URL: https://github.com/apache/kafka/pull/10431#discussion_r629529239 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -242,85 +246,116 @@ final class KafkaMetadataLog private ( } override def

[GitHub] [kafka] guozhangwang commented on pull request #10662: KAFKA-12747: Fix flaky test RocksDBStoreTest.shouldReturnUUIDsWithStr…

2021-05-10 Thread GitBox
guozhangwang commented on pull request #10662: URL: https://github.com/apache/kafka/pull/10662#issuecomment-836978921 Hey @jlprat sorry I was not aware you're also working on this ticket.. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[jira] [Commented] (KAFKA-12767) Properly set Streams system test runtime classpath

2021-05-10 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342014#comment-17342014 ] John Roesler commented on KAFKA-12767: -- Marked "affects version" to 3.0.0, since th

[jira] [Updated] (KAFKA-12767) Properly set Streams system test runtime classpath

2021-05-10 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler updated KAFKA-12767: - Affects Version/s: 3.0.0 > Properly set Streams system test runtime classpath >

[GitHub] [kafka] vvcephei commented on pull request #10631: MINOR: Stop using hamcrest in system tests

2021-05-10 Thread GitBox
vvcephei commented on pull request #10631: URL: https://github.com/apache/kafka/pull/10631#issuecomment-836964248 Thanks for this fix, @lct45 and @cadonna ! It's great to get these tests running again. FYI, I went ahead and filed https://issues.apache.org/jira/browse/KAFKA-12767 for

[jira] [Created] (KAFKA-12767) Properly set Streams system test runtime classpath

2021-05-10 Thread John Roesler (Jira)
John Roesler created KAFKA-12767: Summary: Properly set Streams system test runtime classpath Key: KAFKA-12767 URL: https://issues.apache.org/jira/browse/KAFKA-12767 Project: Kafka Issue Type

[GitHub] [kafka] C0urante commented on pull request #10659: MINOR: remove unnecessary placeholder from WorkerSourceTask#recordSent

2021-05-10 Thread GitBox
C0urante commented on pull request #10659: URL: https://github.com/apache/kafka/pull/10659#issuecomment-836857466 Blegh, sorry guys. Thanks for catching this! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[jira] [Updated] (KAFKA-12757) Move server related common and public classes into separate module(s).

2021-05-10 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana updated KAFKA-12757: --- Description: There are two sets of classes that we want to pull out here for server common/

[GitHub] [kafka] dejan2609 edited a comment on pull request #10658: POC for CheckStyle 8.42 regression (with 'Unnecessary Parentheses' errors)

2021-05-10 Thread GitBox
dejan2609 edited a comment on pull request #10658: URL: https://github.com/apache/kafka/pull/10658#issuecomment-836815768 Rebased onto trunk and force-pushed (as explained in a conversation/review above). -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [kafka] dejan2609 commented on pull request #10658: POC for CheckStyle 8.42 regression (with 'Unnecessary Parentheses' errors)

2021-05-10 Thread GitBox
dejan2609 commented on pull request #10658: URL: https://github.com/apache/kafka/pull/10658#issuecomment-836815768 Rebased onto trunk and force-pushed (as explained in a conversation above). -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] dejan2609 commented on a change in pull request #10658: POC for CheckStyle 8.42 regression (with 'Unnecessary Parentheses' errors)

2021-05-10 Thread GitBox
dejan2609 commented on a change in pull request #10658: URL: https://github.com/apache/kafka/pull/10658#discussion_r629430530 ## File path: checkstyle/checkstyle.xml ## @@ -91,7 +91,7 @@ - + Review comment: Roger that @romani. Ok, I

[GitHub] [kafka] mimaison commented on a change in pull request #10660: MINOR: Updating files with release 2.7.1

2021-05-10 Thread GitBox
mimaison commented on a change in pull request #10660: URL: https://github.com/apache/kafka/pull/10660#discussion_r629427422 ## File path: tests/docker/Dockerfile ## @@ -61,7 +61,7 @@ RUN mkdir -p "/opt/kafka-2.3.1" && chmod a+rw /opt/kafka-2.3.1 && curl -s "$KAFK RUN mkdir -

[GitHub] [kafka] jlprat commented on a change in pull request #10643: KAFKA-12747: Fix flakiness in shouldReturnUUIDsWithStringPrefix

2021-05-10 Thread GitBox
jlprat commented on a change in pull request #10643: URL: https://github.com/apache/kafka/pull/10643#discussion_r629341305 ## File path: streams/src/test/java/org/apache/kafka/streams/state/internals/RocksDBStoreTest.java ## @@ -451,17 +453,14 @@ public void shouldReturnUUIDsW

[GitHub] [kafka] showuon commented on pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-05-10 Thread GitBox
showuon commented on pull request #10552: URL: https://github.com/apache/kafka/pull/10552#issuecomment-836655715 I'll have some refine to this PR. Please wait for a while . Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [kafka] jlprat commented on pull request #10643: KAFKA-12747: Fix flakiness in shouldReturnUUIDsWithStringPrefix

2021-05-10 Thread GitBox
jlprat commented on pull request #10643: URL: https://github.com/apache/kafka/pull/10643#issuecomment-836631952 You can also take a look at https://github.com/apache/kafka/pull/10662/files#diff-944ff4a8c4eab2003ab210f5a2fceac1d124e71fcfe383b736add11803a4fd4cR468 (in there I was checking th

[GitHub] [kafka] jlprat closed pull request #10662: KAFKA-12747: Fix flaky test RocksDBStoreTest.shouldReturnUUIDsWithStr…

2021-05-10 Thread GitBox
jlprat closed pull request #10662: URL: https://github.com/apache/kafka/pull/10662 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [kafka] jlprat commented on pull request #10662: KAFKA-12747: Fix flaky test RocksDBStoreTest.shouldReturnUUIDsWithStr…

2021-05-10 Thread GitBox
jlprat commented on pull request #10662: URL: https://github.com/apache/kafka/pull/10662#issuecomment-836630173 Now I see another PR was already there: https://github.com/apache/kafka/pull/10643 -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[jira] [Commented] (KAFKA-12747) Flaky Test RocksDBStoreTest.shouldReturnUUIDsWithStringPrefix

2021-05-10 Thread Josep Prat (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17341872#comment-17341872 ] Josep Prat commented on KAFKA-12747: There was already another PR > Flaky Test Rock

[jira] [Assigned] (KAFKA-12747) Flaky Test RocksDBStoreTest.shouldReturnUUIDsWithStringPrefix

2021-05-10 Thread Josep Prat (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josep Prat reassigned KAFKA-12747: -- Assignee: (was: Josep Prat) > Flaky Test RocksDBStoreTest.shouldReturnUUIDsWithStringPref

[GitHub] [kafka] jlprat opened a new pull request #10662: KAFKA-12747: Fix flaky test RocksDBStoreTest.shouldReturnUUIDsWithStr…

2021-05-10 Thread GitBox
jlprat opened a new pull request #10662: URL: https://github.com/apache/kafka/pull/10662 …ingPrefix Fixes the test by checking if there is a prefix collision. Following comment on [JIRA ticket](https://issues.apache.org/jira/browse/KAFKA-12747?focusedCommentId=17338612&page=com.at

[GitHub] [kafka] chia7712 commented on a change in pull request #10660: MINOR: Updating files with release 2.7.1

2021-05-10 Thread GitBox
chia7712 commented on a change in pull request #10660: URL: https://github.com/apache/kafka/pull/10660#discussion_r629292359 ## File path: tests/docker/Dockerfile ## @@ -61,7 +61,7 @@ RUN mkdir -p "/opt/kafka-2.3.1" && chmod a+rw /opt/kafka-2.3.1 && curl -s "$KAFK RUN mkdir -

[GitHub] [kafka] satishd edited a comment on pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

2021-05-10 Thread GitBox
satishd edited a comment on pull request #10638: URL: https://github.com/apache/kafka/pull/10638#issuecomment-834307680 Thanks @junrao for the review comments. Pl see inline replies and the latest commits. -- This is an automated message from the Apache Git Service. To respond to the mes

[jira] [Assigned] (KAFKA-12747) Flaky Test RocksDBStoreTest.shouldReturnUUIDsWithStringPrefix

2021-05-10 Thread Josep Prat (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josep Prat reassigned KAFKA-12747: -- Assignee: Josep Prat > Flaky Test RocksDBStoreTest.shouldReturnUUIDsWithStringPrefix > --

[jira] [Commented] (KAFKA-12747) Flaky Test RocksDBStoreTest.shouldReturnUUIDsWithStringPrefix

2021-05-10 Thread Josep Prat (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17341834#comment-17341834 ] Josep Prat commented on KAFKA-12747: I'd like to try to solve this one. > Flaky Tes

[GitHub] [kafka] satishd commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

2021-05-10 Thread GitBox
satishd commented on a change in pull request #10638: URL: https://github.com/apache/kafka/pull/10638#discussion_r629269557 ## File path: build.gradle ## @@ -1345,6 +1349,62 @@ project(':raft') { } } +project(':server-common') { + archivesBaseName = "kafka-server-common"

[jira] [Updated] (KAFKA-12757) Move server related common and public classes into separate module(s).

2021-05-10 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana updated KAFKA-12757: --- Description: There are two sets of classes that we want to pull out here for server common/

[GitHub] [kafka] chia7712 opened a new pull request #10661: MINOR: upgrade pip from 20.2.2 to 21.1.1

2021-05-10 Thread GitBox
chia7712 opened a new pull request #10661: URL: https://github.com/apache/kafka/pull/10661 The following error happens on my mac m1 when building docker image for system tests. ``` Collecting pynacl Using cached PyNaCl-1.4.0.tar.gz (3.4 MB) Installing build dependencies

  1   2   >