[jira] [Commented] (KAFKA-10396) Overall memory of container keep on growing due to kafka stream / rocksdb and OOM killed once limit reached

2021-03-25 Thread Vagesh Mathapati (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309180#comment-17309180 ] Vagesh Mathapati commented on KAFKA-10396: -- Yes. By adding RocksDB config memory did not grow

[GitHub] [kafka] guozhangwang opened a new pull request #10412: KAFKA-12562: Remove deprecated APIs in KafkaStreams and returned state classes

2021-03-25 Thread GitBox
guozhangwang opened a new pull request #10412: URL: https://github.com/apache/kafka/pull/10412 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including

[jira] [Created] (KAFKA-12563) Something wrong with MM2 metrics

2021-03-25 Thread Bui Thanh MInh (Jira)
Bui Thanh MInh created KAFKA-12563: -- Summary: Something wrong with MM2 metrics Key: KAFKA-12563 URL: https://issues.apache.org/jira/browse/KAFKA-12563 Project: Kafka Issue Type: Bug

[GitHub] [kafka] ableegoldman opened a new pull request #10411: KAFKA-7606: Remove deprecated options from StreamsResetter

2021-03-25 Thread GitBox
ableegoldman opened a new pull request #10411: URL: https://github.com/apache/kafka/pull/10411 Removes deprecated flags from StreamsResetter: "--zookeeper" flag (deprecated in 1.0 via KIP-198) "--execute" flag (deprecated in 1.1 via KIP-171) -- This is an automated message from

[jira] [Assigned] (KAFKA-7606) Remove deprecated options from StreamsResetter

2021-03-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman reassigned KAFKA-7606: - Assignee: A. Sophie Blee-Goldman > Remove deprecated options from

[GitHub] [kafka] wenbingshen commented on pull request #10383: MINOR: Query topic describe and sort output by topic name when using adminClient

2021-03-25 Thread GitBox
wenbingshen commented on pull request #10383: URL: https://github.com/apache/kafka/pull/10383#issuecomment-807936687 @dajac Can you take the time to look at this pr again, if you are satisfied, can you help approved it? Thanks. :) -- This is an automated message from the Apache Git

[GitHub] [kafka] wenbingshen commented on pull request #10403: KAFKA-12556: Add --under-preferred-replica-partitions option to describe topics command

2021-03-25 Thread GitBox
wenbingshen commented on pull request #10403: URL: https://github.com/apache/kafka/pull/10403#issuecomment-807935845 > @wenbingshen Thanks for the PR. We need a KIP for adding a new flag to a command line tool as it is considered as part of the public API. Thank you for your

[jira] [Assigned] (KAFKA-8784) Remove default implementation of RocksDBConfigSetter#close

2021-03-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman reassigned KAFKA-8784: - Assignee: Walker Carlson (was: sats) > Remove default implementation

[jira] [Commented] (KAFKA-12419) Remove Deprecated APIs of Kafka Streams in 3.0

2021-03-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309167#comment-17309167 ] A. Sophie Blee-Goldman commented on KAFKA-12419: Ok I'm just gonna go for it and

[jira] [Commented] (KAFKA-7606) Remove deprecated options from StreamsResetter

2021-03-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309166#comment-17309166 ] A. Sophie Blee-Goldman commented on KAFKA-7606: --- Hey [~mjsax], you still planning on

[jira] [Assigned] (KAFKA-12283) Flaky Test RebalanceSourceConnectorsIntegrationTest#testMultipleWorkersRejoining

2021-03-25 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen reassigned KAFKA-12283: - Assignee: Luke Chen (was: Chia-Ping Tsai) > Flaky Test >

[GitHub] [kafka] dongjinleekr removed a comment on pull request #10402: MINOR: Remove unthrown exceptions, fix typo, etc.

2021-03-25 Thread GitBox
dongjinleekr removed a comment on pull request #10402: URL: https://github.com/apache/kafka/pull/10402#issuecomment-807925585 Retest this please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] dongjinleekr commented on pull request #10402: MINOR: Remove unthrown exceptions, fix typo, etc.

2021-03-25 Thread GitBox
dongjinleekr commented on pull request #10402: URL: https://github.com/apache/kafka/pull/10402#issuecomment-807925585 Retest this please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Issue Comment Deleted] (KAFKA-7870) Error sending fetch request (sessionId=1578860481, epoch=INITIAL) to node 2: java.io.IOException: Connection to 2 was disconnected before the response was r

2021-03-25 Thread kai.tian (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kai.tian updated KAFKA-7870: Comment: was deleted (was: We also encountered this problem in version 2.1.0, which resulted in the

[jira] [Created] (KAFKA-12562) Remove deprecated-overloaded "KafkaStreams#metadataForKey" and "KafkaStreams#store"

2021-03-25 Thread Guozhang Wang (Jira)
Guozhang Wang created KAFKA-12562: - Summary: Remove deprecated-overloaded "KafkaStreams#metadataForKey" and "KafkaStreams#store" Key: KAFKA-12562 URL: https://issues.apache.org/jira/browse/KAFKA-12562

[jira] [Resolved] (KAFKA-12527) Remove deprecated "PartitionGrouper" interface

2021-03-25 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-12527. --- Resolution: Duplicate > Remove deprecated "PartitionGrouper" interface >

[jira] [Assigned] (KAFKA-12524) Remove deprecated WindowBytesStoreSupplier#segments

2021-03-25 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-12524: - Assignee: Guozhang Wang > Remove deprecated WindowBytesStoreSupplier#segments >

[jira] [Updated] (KAFKA-7606) Remove deprecated options from StreamsResetter

2021-03-25 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-7606: - Parent: KAFKA-12419 Issue Type: Sub-task (was: Task) > Remove deprecated options from

[GitHub] [kafka] ableegoldman commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-25 Thread GitBox
ableegoldman commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r602006023 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -511,36 +500,82 @@ void

[jira] [Updated] (KAFKA-7106) Remove segment/segmentInterval from Window definition

2021-03-25 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-7106: - Parent: KAFKA-12419 Issue Type: Sub-task (was: Improvement) > Remove

[jira] [Updated] (KAFKA-7785) Remove PartitionGrouper interface and it's config and move DefaultPartitionGrouper to internal package

2021-03-25 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-7785: - Parent: KAFKA-12419 Issue Type: Sub-task (was: Improvement) > Remove PartitionGrouper

[jira] [Resolved] (KAFKA-12526) Remove deprecated long ms overloads

2021-03-25 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-12526. --- Resolution: Duplicate > Remove deprecated long ms overloads >

[jira] [Updated] (KAFKA-12536) Add Instant-based methods to ReadOnlySessionStore

2021-03-25 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-12536: -- Parent: KAFKA-12419 (was: KAFKA-10434) > Add Instant-based methods to ReadOnlySessionStore >

[jira] [Updated] (KAFKA-10434) Remove deprecated methods on WindowStore

2021-03-25 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-10434: -- Parent: KAFKA-12419 Issue Type: Sub-task (was: Improvement) > Remove deprecated

[jira] [Updated] (KAFKA-12450) Remove deprecated methods from ReadOnlyWindowStore

2021-03-25 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-12450: -- Parent: KAFKA-12419 (was: KAFKA-10434) > Remove deprecated methods from ReadOnlyWindowStore

[jira] [Updated] (KAFKA-12451) Remove deprecation annotation on long-based read operations in WindowStore

2021-03-25 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-12451: -- Parent: KAFKA-12419 (was: KAFKA-10434) > Remove deprecation annotation on long-based read

[GitHub] [kafka] guozhangwang commented on pull request #10302: KAFKA-7785: move internal DefaultPartitionGrouper

2021-03-25 Thread GitBox
guozhangwang commented on pull request #10302: URL: https://github.com/apache/kafka/pull/10302#issuecomment-807922328 @mjsax if you can review and merge @highluck 's PR let's do it, I can rebase the other afterwards. -- This is an automated message from the Apache Git Service. To

[jira] [Commented] (KAFKA-12283) Flaky Test RebalanceSourceConnectorsIntegrationTest#testMultipleWorkersRejoining

2021-03-25 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309147#comment-17309147 ] Chia-Ping Tsai commented on KAFKA-12283: Sure! > Flaky Test >

[GitHub] [kafka] showuon commented on pull request #10410: KAFKA-12561: don't set timeout for future.get

2021-03-25 Thread GitBox
showuon commented on pull request #10410: URL: https://github.com/apache/kafka/pull/10410#issuecomment-807918897 @mumrah , @hachikuji , could you review this PR? It failed a lot in recent builds. Thanks. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] showuon opened a new pull request #10410: KAFKA-12561: don't set timeout for future.get

2021-03-25 Thread GitBox
showuon opened a new pull request #10410: URL: https://github.com/apache/kafka/pull/10410 These tests failed quite often, because of `TimeoutException`. We cannot get the future result in time. I think the flaky tests tell us that we should not expect the results will return as soon as we

[jira] [Created] (KAFKA-12561) Fix flaky kafka.server.RaftClusterTest.testCreateClusterAndCreateListDeleteTopic()

2021-03-25 Thread Luke Chen (Jira)
Luke Chen created KAFKA-12561: - Summary: Fix flaky kafka.server.RaftClusterTest.testCreateClusterAndCreateListDeleteTopic() Key: KAFKA-12561 URL: https://issues.apache.org/jira/browse/KAFKA-12561

[GitHub] [kafka] showuon commented on pull request #10409: KAFKA-9295: increase heartbeat interval to avoid unneeded rebalance

2021-03-25 Thread GitBox
showuon commented on pull request #10409: URL: https://github.com/apache/kafka/pull/10409#issuecomment-807915216 @ableegoldman , I agree with you! Let me see if I can get some logs and see if you have any thoughts. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] ableegoldman edited a comment on pull request #10409: KAFKA-9295: increase heartbeat interval to avoid unneeded rebalance

2021-03-25 Thread GitBox
ableegoldman edited a comment on pull request #10409: URL: https://github.com/apache/kafka/pull/10409#issuecomment-807906627 > When the system is slow, it might not be able to send heartbeat within default 3 seconds Well, it would have to miss ~3 heartbeats in order to trigger a

[GitHub] [kafka] ableegoldman edited a comment on pull request #10409: KAFKA-9295: increase heartbeat interval to avoid unneeded rebalance

2021-03-25 Thread GitBox
ableegoldman edited a comment on pull request #10409: URL: https://github.com/apache/kafka/pull/10409#issuecomment-807906627 > When the system is slow, it might not be able to send heartbeat within default 3 seconds Well, it would have to miss ~3 heartbeats in order to trigger a

[GitHub] [kafka] ableegoldman commented on a change in pull request #10409: KAFKA-9295: increase heartbeat interval to avoid unneeded rebalance

2021-03-25 Thread GitBox
ableegoldman commented on a change in pull request #10409: URL: https://github.com/apache/kafka/pull/10409#discussion_r601991532 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/KTableKTableForeignKeyInnerJoinMultiIntegrationTest.java ## @@ -172,7

[GitHub] [kafka] ableegoldman commented on pull request #10409: KAFKA-9295: increase heartbeat interval to avoid unneeded rebalance

2021-03-25 Thread GitBox
ableegoldman commented on pull request #10409: URL: https://github.com/apache/kafka/pull/10409#issuecomment-807906627 > When the system is slow, it might not be able to send heartbeat within default 3 seconds Well, it would have to miss ~3 heartbeats in order to trigger a rebalance

[GitHub] [kafka] showuon commented on a change in pull request #10409: KAFKA-9295: increase heartbeat interval to avoid unneeded rebalance

2021-03-25 Thread GitBox
showuon commented on a change in pull request #10409: URL: https://github.com/apache/kafka/pull/10409#discussion_r601988722 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/KTableKTableForeignKeyInnerJoinMultiIntegrationTest.java ## @@ -184,26 +186,25

[GitHub] [kafka] ableegoldman commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-25 Thread GitBox
ableegoldman commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r601988472 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java ## @@ -774,48 +771,6 @@ public void

[GitHub] [kafka] showuon commented on a change in pull request #10409: KAFKA-9295: increase heartbeat interval to avoid unneeded rebalance

2021-03-25 Thread GitBox
showuon commented on a change in pull request #10409: URL: https://github.com/apache/kafka/pull/10409#discussion_r601988476 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/KTableKTableForeignKeyInnerJoinMultiIntegrationTest.java ## @@ -184,26 +186,25

[GitHub] [kafka] ableegoldman commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-25 Thread GitBox
ableegoldman commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r601988153 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamThreadTest.java ## @@ -2333,74 +2333,6 @@ void runOnce() {

[GitHub] [kafka] showuon commented on a change in pull request #10409: KAFKA-9295: increase heartbeat interval to avoid unneeded rebalance

2021-03-25 Thread GitBox
showuon commented on a change in pull request #10409: URL: https://github.com/apache/kafka/pull/10409#discussion_r601987807 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/KTableKTableForeignKeyInnerJoinMultiIntegrationTest.java ## @@ -172,7 +174,7 @@

[GitHub] [kafka] ableegoldman commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-25 Thread GitBox
ableegoldman commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r601987936 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -611,6 +607,9 @@ boolean runLoop() {

[GitHub] [kafka] showuon commented on a change in pull request #10409: KAFKA-9295: increase heartbeat interval to avoid unneeded rebalance

2021-03-25 Thread GitBox
showuon commented on a change in pull request #10409: URL: https://github.com/apache/kafka/pull/10409#discussion_r601987807 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/KTableKTableForeignKeyInnerJoinMultiIntegrationTest.java ## @@ -172,7 +174,7 @@

[GitHub] [kafka] ijuma merged pull request #10399: MINOR: Use Java 11 for generating aggregated javadoc in release.py

2021-03-25 Thread GitBox
ijuma merged pull request #10399: URL: https://github.com/apache/kafka/pull/10399 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [kafka] ijuma commented on pull request #10405: KAFKA-3968: fsync the parent directory of a segment file when the file is created

2021-03-25 Thread GitBox
ijuma commented on pull request #10405: URL: https://github.com/apache/kafka/pull/10405#issuecomment-807903146 Thanks for the PR. Did we check the performance impact of this change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] ableegoldman commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-25 Thread GitBox
ableegoldman commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r601987024 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -589,11 +589,7 @@ boolean runLoop() {

[jira] [Issue Comment Deleted] (KAFKA-12485) Speed up Consumer#committed by returning cached offsets for owned partitions

2021-03-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12485: --- Comment: was deleted (was: Side note: if we get this optimization in then

[jira] [Comment Edited] (KAFKA-12360) Improve documentation of max.task.idle.ms (kafka-streams)

2021-03-25 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309113#comment-17309113 ] John Roesler edited comment on KAFKA-12360 at 3/26/21, 2:57 AM: Hi

[jira] [Commented] (KAFKA-12360) Improve documentation of max.task.idle.ms (kafka-streams)

2021-03-25 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309113#comment-17309113 ] John Roesler commented on KAFKA-12360: -- Hi [~nicodds] , I can see your line of reasoning, but I

[GitHub] [kafka] vvcephei commented on pull request #10207: Fixing documentation source for issue KAFKA-12360

2021-03-25 Thread GitBox
vvcephei commented on pull request #10207: URL: https://github.com/apache/kafka/pull/10207#issuecomment-807898796 Hi @nicodds , I can see your line of reasoning, but I think there must be something else going on there. When a task is "idling", it does not block the poll loop.

[GitHub] [kafka] ableegoldman edited a comment on pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-25 Thread GitBox
ableegoldman edited a comment on pull request #10407: URL: https://github.com/apache/kafka/pull/10407#issuecomment-807879383 Ready for review @guozhangwang @vvcephei @mjsax @cadonna -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] showuon opened a new pull request #10409: KAFKA-9295: increase heartbeat interval to avoid unneeded rebalance

2021-03-25 Thread GitBox
showuon opened a new pull request #10409: URL: https://github.com/apache/kafka/pull/10409 After investigation, I found the test sometimes failed because the streams keep rebalancing before they start to read/process/write data, due to heartbeat timeout: ``` Preparing to rebalance

[jira] [Commented] (KAFKA-12419) Remove Deprecated APIs of Kafka Streams in 3.0

2021-03-25 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309109#comment-17309109 ] Matthias J. Sax commented on KAFKA-12419: - It would be best if we could re-ramp the

[GitHub] [kafka] ableegoldman commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-25 Thread GitBox
ableegoldman commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r601981022 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -167,18 +167,7 @@ void

[GitHub] [kafka] mjsax merged pull request #10300: KAFKA-12452: Remove deprecated overloads of ProcessorContext#forward

2021-03-25 Thread GitBox
mjsax merged pull request #10300: URL: https://github.com/apache/kafka/pull/10300 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[jira] [Commented] (KAFKA-12283) Flaky Test RebalanceSourceConnectorsIntegrationTest#testMultipleWorkersRejoining

2021-03-25 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309103#comment-17309103 ] Luke Chen commented on KAFKA-12283: --- [~chia7712], Mind if I take this ticket over? Thank you. > Flaky

[GitHub] [kafka] showuon commented on pull request #10408: KAFKA-12283: disable flaky testMultipleWorkersRejoining to stabilize build

2021-03-25 Thread GitBox
showuon commented on pull request #10408: URL: https://github.com/apache/kafka/pull/10408#issuecomment-80797 @ableegoldman , I agree to ignore this test first, and added back after the root cause fixed, to stabilize build. -- This is an automated message from the Apache Git Service.

[GitHub] [kafka] showuon opened a new pull request #10408: KAFKA-12283: disable flaky testMultipleWorkersRejoining to stabilize build

2021-03-25 Thread GitBox
showuon opened a new pull request #10408: URL: https://github.com/apache/kafka/pull/10408 Will address the root cause in this PR: https://github.com/apache/kafka/pull/10367, and added this test back in that PR, too. ### Committer Checklist (excluded from commit message) - [ ]

[GitHub] [kafka] highluck commented on pull request #9851: KAFKA-10769 Remove JoinGroupRequest#containsValidPattern as it is dup…

2021-03-25 Thread GitBox
highluck commented on pull request #9851: URL: https://github.com/apache/kafka/pull/9851#issuecomment-807886758 ping @chia7712 Please give me a review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [kafka] ableegoldman commented on pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-25 Thread GitBox
ableegoldman commented on pull request #10407: URL: https://github.com/apache/kafka/pull/10407#issuecomment-807879383 Still cleaning up the tests but here is the general idea of the fix -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] ableegoldman opened a new pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-25 Thread GitBox
ableegoldman opened a new pull request #10407: URL: https://github.com/apache/kafka/pull/10407 Clean up handling of TaskCorruptedException from `commitOffsetsOrTransaction` by (1) removing the offset commit in `TaskManager#handleCorruption`, and (2) filter out any revoked tasks from the

[GitHub] [kafka] dongjinleekr commented on pull request #10402: MINOR: Remove unthrown exceptions, fix typo, etc.

2021-03-25 Thread GitBox
dongjinleekr commented on pull request #10402: URL: https://github.com/apache/kafka/pull/10402#issuecomment-807859296 @ableegoldman Here it is. I removed one commit and changed the title. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] dengziming commented on pull request #10400: MINOR: fix package name in integration test

2021-03-25 Thread GitBox
dengziming commented on pull request #10400: URL: https://github.com/apache/kafka/pull/10400#issuecomment-807859034 @rondagostino Thank you for your suggestions, rechecked the other source files! -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] dongjinleekr commented on a change in pull request #10402: TRIVIAL: Minor improvements

2021-03-25 Thread GitBox
dongjinleekr commented on a change in pull request #10402: URL: https://github.com/apache/kafka/pull/10402#discussion_r601946043 ## File path: connect/mirror-client/src/main/java/org/apache/kafka/connect/mirror/RemoteClusterUtils.java ## @@ -1,94 +0,0 @@ -/* - * Licensed to

[GitHub] [kafka] alok123t opened a new pull request #10406: MINOR: Update testUpdate method with correct Crc32

2021-03-25 Thread GitBox
alok123t opened a new pull request #10406: URL: https://github.com/apache/kafka/pull/10406 In this PR I have updated the `testUpdate` method to test with the correct `Crc32` class ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation -

[jira] [Updated] (KAFKA-12560) Accidental delete of some log files kafka-authorizer.log and kafka-request.log can break topics in cluster

2021-03-25 Thread Rufus Skyfii (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rufus Skyfii updated KAFKA-12560: - Description: These two log files have 0 byte size and last created modified or updated  along

[jira] [Created] (KAFKA-12560) Accidental delete of some log files kafka-authorizer.log and kafka-request.log can break topics in cluster

2021-03-25 Thread Rufus Skyfii (Jira)
Rufus Skyfii created KAFKA-12560: Summary: Accidental delete of some log files kafka-authorizer.log and kafka-request.log can break topics in cluster Key: KAFKA-12560 URL:

[jira] [Commented] (KAFKA-12559) Add a top-level Streams config for bounding off-heap memory

2021-03-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309052#comment-17309052 ] A. Sophie Blee-Goldman commented on KAFKA-12559: We can also consider whether to apply

[jira] [Created] (KAFKA-12559) Add a top-level Streams config for bounding off-heap memory

2021-03-25 Thread A. Sophie Blee-Goldman (Jira)
A. Sophie Blee-Goldman created KAFKA-12559: -- Summary: Add a top-level Streams config for bounding off-heap memory Key: KAFKA-12559 URL: https://issues.apache.org/jira/browse/KAFKA-12559

[jira] [Resolved] (KAFKA-12508) Emit-on-change tables may lose updates on error or restart in at_least_once

2021-03-25 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-12508. -- Resolution: Fixed Disabled KIP-557 entirely. > Emit-on-change tables may lose updates on

[jira] [Updated] (KAFKA-12508) Emit-on-change tables may lose updates on error or restart in at_least_once

2021-03-25 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler updated KAFKA-12508: - Affects Version/s: 2.6.0 > Emit-on-change tables may lose updates on error or restart in

[GitHub] [kafka] ccding opened a new pull request #10405: [KAFKA-3968] fsync the parent directory of a segment file when the file is created

2021-03-25 Thread GitBox
ccding opened a new pull request #10405: URL: https://github.com/apache/kafka/pull/10405 Kafka does not call fsync() on the directory when a new log segment is created and flushed to disk. The problem is that following sequence of calls doesn't guarantee file durability: fd

[jira] [Commented] (KAFKA-6603) Kafka streams off heap memory usage does not match expected values from configuration

2021-03-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309033#comment-17309033 ] A. Sophie Blee-Goldman commented on KAFKA-6603: --- Hey [~igor.calabria] can we close this

[jira] [Commented] (KAFKA-10396) Overall memory of container keep on growing due to kafka stream / rocksdb and OOM killed once limit reached

2021-03-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309030#comment-17309030 ] A. Sophie Blee-Goldman commented on KAFKA-10396: Hey [~vmathapati] can we resolve this

[jira] [Updated] (KAFKA-12495) Unbalanced connectors/tasks distribution will happen in Connect's incremental cooperative assignor

2021-03-25 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-12495: -- Description: In Kafka Connect, we implement incremental cooperative rebalance algorithm based on

[jira] [Commented] (KAFKA-12283) Flaky Test RebalanceSourceConnectorsIntegrationTest#testMultipleWorkersRejoining

2021-03-25 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309020#comment-17309020 ] Luke Chen commented on KAFKA-12283: --- And, answer your question: Is this a bug in the test, or

[jira] [Updated] (KAFKA-8575) Investigate removing EAGER protocol & cleaning up task suspension in Streams rebalancing

2021-03-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-8575: -- Description: With KIP-429 the suspend/resume of tasks may have minimal gains

[jira] [Updated] (KAFKA-8575) Investigate removing EAGER protocol & cleaning up task suspension in Streams rebalancing

2021-03-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-8575: -- Priority: Blocker (was: Major) > Investigate removing EAGER protocol &

[jira] [Commented] (KAFKA-12283) Flaky Test RebalanceSourceConnectorsIntegrationTest#testMultipleWorkersRejoining

2021-03-25 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309002#comment-17309002 ] Luke Chen commented on KAFKA-12283: --- [~ableegoldman], sorry, I forgot to update status here. The PR to

[GitHub] [kafka] dhruvilshah3 commented on a change in pull request #10388: KAFKA-12520: Ensure log loading does not truncate producer state unless required

2021-03-25 Thread GitBox
dhruvilshah3 commented on a change in pull request #10388: URL: https://github.com/apache/kafka/pull/10388#discussion_r601867771 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -700,11 +700,13 @@ class Log(@volatile private var _dir: File, case _:

[GitHub] [kafka] dhruvilshah3 commented on a change in pull request #10388: KAFKA-12520: Ensure log loading does not truncate producer state unless required

2021-03-25 Thread GitBox
dhruvilshah3 commented on a change in pull request #10388: URL: https://github.com/apache/kafka/pull/10388#discussion_r601867672 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -700,11 +700,13 @@ class Log(@volatile private var _dir: File, case _:

[GitHub] [kafka] dhruvilshah3 commented on a change in pull request #10388: KAFKA-12520: Ensure log loading does not truncate producer state unless required

2021-03-25 Thread GitBox
dhruvilshah3 commented on a change in pull request #10388: URL: https://github.com/apache/kafka/pull/10388#discussion_r601867171 ## File path: core/src/main/scala/kafka/log/LogSegment.scala ## @@ -322,17 +323,14 @@ class LogSegment private[log] (val log: FileRecords,

[GitHub] [kafka] dhruvilshah3 commented on a change in pull request #10388: KAFKA-12520: Ensure log loading does not truncate producer state unless required

2021-03-25 Thread GitBox
dhruvilshah3 commented on a change in pull request #10388: URL: https://github.com/apache/kafka/pull/10388#discussion_r601865901 ## File path: core/src/main/scala/kafka/log/LogSegment.scala ## @@ -322,17 +323,14 @@ class LogSegment private[log] (val log: FileRecords,

[jira] [Commented] (KAFKA-12558) MM2 may not sync partition offsets correctly

2021-03-25 Thread Ryanne Dolan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308968#comment-17308968 ] Ryanne Dolan commented on KAFKA-12558: -- This sounds right. Updating the partition store without

[jira] [Comment Edited] (KAFKA-12468) Initial offsets are copied from source to target cluster

2021-03-25 Thread Alan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308965#comment-17308965 ] Alan edited comment on KAFKA-12468 at 3/25/21, 8:47 PM: I found a race condition

[jira] [Commented] (KAFKA-12468) Initial offsets are copied from source to target cluster

2021-03-25 Thread Alan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308965#comment-17308965 ] Alan commented on KAFKA-12468: -- I found a race condition in MirrorSourceTask.

[jira] [Updated] (KAFKA-12558) MM2 may not sync partition offsets correctly

2021-03-25 Thread Alan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan updated KAFKA-12558: - Description: There is a race condition in {{MirrorSourceTask}} where certain partition offsets may never be

[jira] [Updated] (KAFKA-12558) MM2 may not sync partition offsets correctly

2021-03-25 Thread Alan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan updated KAFKA-12558: - Description: There is a race condition in {{MirrorSourceTask}} where certain partition offsets may never be

[jira] [Created] (KAFKA-12558) MM2 may not sync partition offsets correctly

2021-03-25 Thread Alan (Jira)
Alan created KAFKA-12558: Summary: MM2 may not sync partition offsets correctly Key: KAFKA-12558 URL: https://issues.apache.org/jira/browse/KAFKA-12558 Project: Kafka Issue Type: Bug

[GitHub] [kafka] vvcephei merged pull request #10397: KAFKA-12508: Disable KIP-557

2021-03-25 Thread GitBox
vvcephei merged pull request #10397: URL: https://github.com/apache/kafka/pull/10397 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[jira] [Commented] (KAFKA-10575) StateRestoreListener#onRestoreEnd should always be triggered

2021-03-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308931#comment-17308931 ] A. Sophie Blee-Goldman commented on KAFKA-10575: [~high.lee] this ticket is aimed at

[GitHub] [kafka] vvcephei commented on a change in pull request #10404: KAFKA-12557: Fix hanging KafkaAdminClientTest

2021-03-25 Thread GitBox
vvcephei commented on a change in pull request #10404: URL: https://github.com/apache/kafka/pull/10404#discussion_r601785655 ## File path: clients/src/test/java/org/apache/kafka/clients/admin/AdminClientUnitTestEnv.java ## @@ -113,6 +114,9 @@ public MockClient kafkaClient() {

[GitHub] [kafka] vvcephei commented on pull request #10397: KAFKA-12508: Disable KIP-557

2021-03-25 Thread GitBox
vvcephei commented on pull request #10397: URL: https://github.com/apache/kafka/pull/10397#issuecomment-807342848 Thanks, @ableegoldman ! Yes, I plan to do it right now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] ableegoldman commented on a change in pull request #10402: TRIVIAL: Minor improvements

2021-03-25 Thread GitBox
ableegoldman commented on a change in pull request #10402: URL: https://github.com/apache/kafka/pull/10402#discussion_r601774493 ## File path: connect/mirror-client/src/main/java/org/apache/kafka/connect/mirror/RemoteClusterUtils.java ## @@ -1,94 +0,0 @@ -/* - * Licensed to

[GitHub] [kafka] ryannedolan commented on a change in pull request #10402: TRIVIAL: Minor improvements

2021-03-25 Thread GitBox
ryannedolan commented on a change in pull request #10402: URL: https://github.com/apache/kafka/pull/10402#discussion_r601772795 ## File path: connect/mirror-client/src/main/java/org/apache/kafka/connect/mirror/RemoteClusterUtils.java ## @@ -1,94 +0,0 @@ -/* - * Licensed to

[jira] [Commented] (KAFKA-12283) Flaky Test RebalanceSourceConnectorsIntegrationTest#testMultipleWorkersRejoining

2021-03-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308917#comment-17308917 ] A. Sophie Blee-Goldman commented on KAFKA-12283: Hey [~showuon] what's the status here?

[GitHub] [kafka] ableegoldman commented on a change in pull request #10402: TRIVIAL: Minor improvements

2021-03-25 Thread GitBox
ableegoldman commented on a change in pull request #10402: URL: https://github.com/apache/kafka/pull/10402#discussion_r601767570 ## File path: connect/mirror-client/src/main/java/org/apache/kafka/connect/mirror/RemoteClusterUtils.java ## @@ -1,94 +0,0 @@ -/* - * Licensed to

[GitHub] [kafka] ableegoldman commented on pull request #10397: KAFKA-12508: Disable KIP-557

2021-03-25 Thread GitBox
ableegoldman commented on pull request #10397: URL: https://github.com/apache/kafka/pull/10397#issuecomment-807283643 Nice test debugging  Just to confirm, you're going to backport this to 2.7 & 2.6 right? -- This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] ijuma commented on a change in pull request #10404: KAFKA-12557: Fix hanging KafkaAdminClientTest

2021-03-25 Thread GitBox
ijuma commented on a change in pull request #10404: URL: https://github.com/apache/kafka/pull/10404#discussion_r601759185 ## File path: clients/src/test/java/org/apache/kafka/clients/admin/AdminClientUnitTestEnv.java ## @@ -113,6 +114,9 @@ public MockClient kafkaClient() {

[GitHub] [kafka] vvcephei commented on pull request #10397: KAFKA-12508: Disable KIP-557

2021-03-25 Thread GitBox
vvcephei commented on pull request #10397: URL: https://github.com/apache/kafka/pull/10397#issuecomment-807271081 I looked into the Java 8 timeout. It should not be related, as it's caused by the KafkaAdminClientTest hanging. I submitted https://github.com/apache/kafka/pull/10404 to

  1   2   >