[jira] [Updated] (KAFKA-13633) Merging multiple KStreams in one operation

2022-01-31 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13633: Labels: needs-kip (was: ) > Merging multiple KStreams in one operation >

[jira] [Commented] (KAFKA-13633) Merging multiple KStreams in one operation

2022-01-31 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17485067#comment-17485067 ] Matthias J. Sax commented on KAFKA-13633: - Seems you close the PR... Btw: this ticket implies a

[GitHub] [kafka] hachikuji opened a new pull request #11725: KAFKA-13221; Implement `PartitionsWithLateTransactionsCount` metric

2022-01-31 Thread GitBox
hachikuji opened a new pull request #11725: URL: https://github.com/apache/kafka/pull/11725 This patch implements a new metric `PartitionsWithLateTransactionsCount` which tracks the number of partitions with late transactions in the cluster. This metric was documented in KIP-664:

[GitHub] [kafka] ableegoldman commented on a change in pull request #11424: KAFKA-13152: Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

2022-01-31 Thread GitBox
ableegoldman commented on a change in pull request #11424: URL: https://github.com/apache/kafka/pull/11424#discussion_r796144313 ## File path: streams/test-utils/src/main/java/org/apache/kafka/streams/TopologyTestDriver.java ## @@ -329,7 +330,7 @@ private

[GitHub] [kafka] ableegoldman commented on a change in pull request #11424: KAFKA-13152: Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

2022-01-31 Thread GitBox
ableegoldman commented on a change in pull request #11424: URL: https://github.com/apache/kafka/pull/11424#discussion_r796139442 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/namedtopology/TopologyConfig.java ## @@ -116,14 +123,30 @@ public

[GitHub] [kafka] ijuma commented on pull request #11691: KAFKA-13598: enable idempotence producer by default and validate the configs

2022-01-31 Thread GitBox
ijuma commented on pull request #11691: URL: https://github.com/apache/kafka/pull/11691#issuecomment-1026067014 > To be honest, I'm a bit hesitant about this patch going into a bug fix release. One of the tricky issues from the KIP was changing idempotent permission requirements. The

[GitHub] [kafka] hachikuji commented on a change in pull request #11691: KAFKA-13598: enable idempotence producer by default and validate the configs

2022-01-31 Thread GitBox
hachikuji commented on a change in pull request #11691: URL: https://github.com/apache/kafka/pull/11691#discussion_r795906556 ## File path: core/src/main/scala/kafka/tools/ConsoleProducer.scala ## @@ -168,7 +168,7 @@ object ConsoleProducer { .withRequiredArg

[GitHub] [kafka] Gerrrr commented on pull request #11447: KAFKA-13024

2022-01-31 Thread GitBox
Ge commented on pull request #11447: URL: https://github.com/apache/kafka/pull/11447#issuecomment-1026044886 Thanks for taking a look at this PR @guozhangwang ! > we are only fixing KStream, since for KTable it's always safe to filter null keys Yes > it's trying

[GitHub] [kafka] jasonk000 commented on pull request #11722: KAFKA-13630: reduce amount of time that producer network thread holds batch queue lock

2022-01-31 Thread GitBox
jasonk000 commented on pull request #11722: URL: https://github.com/apache/kafka/pull/11722#issuecomment-1026000556 This application has a single producer thread, a high `send()` rate. The change reduces spinlock CPU cycles from 14.6% to 2.5% of the `send()` path, or more clearly a 12.1%

[GitHub] [kafka] jasonk000 commented on pull request #11721: KAFKA-13629: use faster algorithm for ByteUtils sizeOfXxx algorithm

2022-01-31 Thread GitBox
jasonk000 commented on pull request #11721: URL: https://github.com/apache/kafka/pull/11721#issuecomment-1025992223 Overall, this results in ~6% reduction in CPU cycles in the `send()` path. Before flamegraph (see highlighted section) - starting from `send()`

[GitHub] [kafka] jonathan-albrecht-ibm commented on a change in pull request #11690: KAFKA-13599: Upgrade RocksDB to 6.27.3

2022-01-31 Thread GitBox
jonathan-albrecht-ibm commented on a change in pull request #11690: URL: https://github.com/apache/kafka/pull/11690#discussion_r795859434 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBGenericOptionsToDbOptionsColumnFamilyOptionsAdapter.java

[jira] [Updated] (KAFKA-13634) Kafka sends false information to the Debezium Connector in case the heartbeat topic does not exists

2022-01-31 Thread Andreas Gillmann (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andreas Gillmann updated KAFKA-13634: - Summary: Kafka sends false information to the Debezium Connector in case the heartbeat

[jira] [Created] (KAFKA-13634) false information to the Debezium Connector in case the heartbeat topic does not exists

2022-01-31 Thread Andreas Gillmann (Jira)
Andreas Gillmann created KAFKA-13634: Summary: false information to the Debezium Connector in case the heartbeat topic does not exists Key: KAFKA-13634 URL: https://issues.apache.org/jira/browse/KAFKA-13634

[jira] [Updated] (KAFKA-13633) Merging multiple KStreams in one operation

2022-01-31 Thread Nicholas Telford (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Telford updated KAFKA-13633: - Description: The {{KStream}} API provides {{merge(KStream)}} to merge another

[GitHub] [kafka] nicktelford closed pull request #11724: KAFKA-13633: Add vararg KStream#merge API

2022-01-31 Thread GitBox
nicktelford closed pull request #11724: URL: https://github.com/apache/kafka/pull/11724 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] nicktelford opened a new pull request #11724: KAFKA-13633: Add vararg KStream#merge API

2022-01-31 Thread GitBox
nicktelford opened a new pull request #11724: URL: https://github.com/apache/kafka/pull/11724 [KIP-819](https://cwiki.apache.org/confluence/display/KAFKA/KIP-819%3A+Merge+multiple+KStreams+in+one+operation) The `KStream#merge` API has been modified to permit merging many KStreams

[GitHub] [kafka] mimaison commented on pull request #11695: KAFKA-13595: Allow producing records with null fields in ConsoleProducer

2022-01-31 Thread GitBox
mimaison commented on pull request #11695: URL: https://github.com/apache/kafka/pull/11695#issuecomment-1025877880 Thanks @dajac for the review. I pushed an update -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Created] (KAFKA-13633) Merging multiple KStreams in one operation

2022-01-31 Thread Nicholas Telford (Jira)
Nicholas Telford created KAFKA-13633: Summary: Merging multiple KStreams in one operation Key: KAFKA-13633 URL: https://issues.apache.org/jira/browse/KAFKA-13633 Project: Kafka Issue

[GitHub] [kafka] mimaison commented on pull request #11599: KAFKA-13527: Add top-level error code field to DescribeLogDirsResponse

2022-01-31 Thread GitBox
mimaison commented on pull request #11599: URL: https://github.com/apache/kafka/pull/11599#issuecomment-1025855287 > @mimaison Should we update `KafkaApis` to correct fill the error code when v3 is used by the client? At the moment, the field is not used at all. @dajac Yes

[GitHub] [kafka] ijuma commented on a change in pull request #11691: KAFKA-13598: enable idempotence producer by default and validate the configs

2022-01-31 Thread GitBox
ijuma commented on a change in pull request #11691: URL: https://github.com/apache/kafka/pull/11691#discussion_r795714161 ## File path: docs/upgrade.html ## @@ -19,6 +19,13 @@

[GitHub] [kafka] ijuma commented on pull request #11722: KAFKA-13630: reduce amount of time that producer network thread holds batch queue lock

2022-01-31 Thread GitBox
ijuma commented on pull request #11722: URL: https://github.com/apache/kafka/pull/11722#issuecomment-1025708768 Thanks for the PR. Do you have some results to share regarding the impact this optimization has? -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] ijuma commented on pull request #11721: KAFKA-13629: use faster algorithm for ByteUtils sizeOfXxx algorithm

2022-01-31 Thread GitBox
ijuma commented on pull request #11721: URL: https://github.com/apache/kafka/pull/11721#issuecomment-1025704462 Thanks for the PR. Can you please share the results for the micro-benchmark as well as any improvement you observed in real workloads? -- This is an automated message from the

[GitHub] [kafka] serak commented on pull request #11549: KAFKA-13382: Automatic storage formatting

2022-01-31 Thread GitBox
serak commented on pull request #11549: URL: https://github.com/apache/kafka/pull/11549#issuecomment-1025692982 this would be very helpful for newbies kafka says storage not formated please run Kafka-storage.sh on windows machine and this file does not even exist in windows batch script

[jira] [Updated] (KAFKA-13623) Memory leak when multiple poll

2022-01-31 Thread Emanuel Velzi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emanuel Velzi updated KAFKA-13623: -- Attachment: image-2022-01-31-09-06-27-762.png > Memory leak when multiple poll >

[jira] [Updated] (KAFKA-13511) Add support for different unix precisions in TimestampConverter SMT

2022-01-31 Thread Julien Chanaud (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Chanaud updated KAFKA-13511: --- Summary: Add support for different unix precisions in TimestampConverter SMT (was: Update

[GitHub] [kafka] dajac commented on a change in pull request #11695: KAFKA-13595: Allow producing records with null fields in ConsoleProducer

2022-01-31 Thread GitBox
dajac commented on a change in pull request #11695: URL: https://github.com/apache/kafka/pull/11695#discussion_r795497752 ## File path: core/src/main/scala/kafka/tools/ConsoleProducer.scala ## @@ -362,11 +375,21 @@ object ConsoleProducer { } private def

[GitHub] [kafka] dajac commented on pull request #11599: KAFKA-13527: Add top-level error code field to DescribeLogDirsResponse

2022-01-31 Thread GitBox
dajac commented on pull request #11599: URL: https://github.com/apache/kafka/pull/11599#issuecomment-1025549712 @mimaison Should we update `KafkaApis` to correct fill the error code when v3 is used by the client? At the moment, the field is not used at all. -- This is an automated

[GitHub] [kafka] dajac commented on a change in pull request #11599: KAFKA-13527: Add top-level error code field to DescribeLogDirsResponse

2022-01-31 Thread GitBox
dajac commented on a change in pull request #11599: URL: https://github.com/apache/kafka/pull/11599#discussion_r795490547 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java ## @@ -2681,8 +2681,11 @@ public void

[GitHub] [kafka] dajac merged pull request #11718: MINOR: ConsoleConsumer should not always exit when Consumer::poll returns an empty record batch

2022-01-31 Thread GitBox
dajac merged pull request #11718: URL: https://github.com/apache/kafka/pull/11718 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Updated] (KAFKA-13632) MirrorMaker 2.0 NPE and Warning "Failure to commit records" for filtered records

2022-01-31 Thread Bert Baron (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bert Baron updated KAFKA-13632: --- Description: We have a setup where we filter records with MirrorMaker 2.0 (see below). This

[GitHub] [kafka] dajac commented on pull request #11718: MINOR: Fix transactions system tests

2022-01-31 Thread GitBox
dajac commented on pull request #11718: URL: https://github.com/apache/kafka/pull/11718#issuecomment-1025537099 System tests results:

[jira] [Updated] (KAFKA-13631) Warning "Ignoring await stop request for non-present connector..." when shutting down connector via API in distributed mode

2022-01-31 Thread Simon Schall (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Schall updated KAFKA-13631: - Description: * S{color:#172b4d}etup: kafka with version 2.8.1 with scala version 2.13{color} *

[jira] [Updated] (KAFKA-13631) Warning "Ignoring await stop request for non-present connector..." when shutting down connector via API in distributed mode

2022-01-31 Thread Simon Schall (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Schall updated KAFKA-13631: - Description: * S{color:#172b4d}etup: kafka with version 2.8.1 with scala version 2.13{color} *

[jira] [Updated] (KAFKA-13631) Warning "Ignoring await stop request for non-present connector..." when shutting down connector via API in distributed mode

2022-01-31 Thread Simon Schall (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Schall updated KAFKA-13631: - Summary: Warning "Ignoring await stop request for non-present connector..." when shutting down

[jira] [Updated] (KAFKA-13631) Warning "Ignoring await stop request for non-present connector..." when shuting down connector via API in distributed mode

2022-01-31 Thread Simon Schall (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Schall updated KAFKA-13631: - Description: * S{color:#172b4d}etu: kafka with version 2.8.1 with scala version 2.13{color} *

[jira] [Updated] (KAFKA-13631) Warning "Ignoring await stop request for non-present connector..." when shuting down connector via API in distributed mode

2022-01-31 Thread Simon Schall (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Schall updated KAFKA-13631: - Description: * S{color:#172b4d}etup: kafka with version 2.8.1 with scala version 2.13{color} *

[jira] [Updated] (KAFKA-13632) MirrorMaker 2.0 NPE and Warning "Failure to commit records" for filtered records

2022-01-31 Thread Bert Baron (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bert Baron updated KAFKA-13632: --- Description: We have a setup where we filter records with MirrorMaker 2.0 (see below). This

[GitHub] [kafka] lkokhreidze commented on pull request #10851: KAFKA-6718 / Rack aware standby task assignor

2022-01-31 Thread GitBox
lkokhreidze commented on pull request #10851: URL: https://github.com/apache/kafka/pull/10851#issuecomment-1025503003 Hi @cadonna , Sorry for the ping. Any chance we could review PR this week? Thanks -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] lkokhreidze commented on a change in pull request #10851: KAFKA-6718 / Rack aware standby task assignor

2022-01-31 Thread GitBox
lkokhreidze commented on a change in pull request #10851: URL: https://github.com/apache/kafka/pull/10851#discussion_r795434808 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/ClientTagAwareStandbyTaskAssignor.java ## @@ -0,0 +1,289

[jira] [Created] (KAFKA-13632) MirrorMaker 2.0 NPE and Warning "Failure to commit records" for filtered records

2022-01-31 Thread Bert Baron (Jira)
Bert Baron created KAFKA-13632: -- Summary: MirrorMaker 2.0 NPE and Warning "Failure to commit records" for filtered records Key: KAFKA-13632 URL: https://issues.apache.org/jira/browse/KAFKA-13632

[GitHub] [kafka] hi-rustin opened a new pull request #11723: MINOR: replace statement lambda with expression lambda

2022-01-31 Thread GitBox
hi-rustin opened a new pull request #11723: URL: https://github.com/apache/kafka/pull/11723 Replace statement lambda with expression lambda. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build

[jira] [Created] (KAFKA-13631) Warning "Ignoring await stop request for non-present connector..." when shuting down connector via API in distributed mode

2022-01-31 Thread Simon Schall (Jira)
Simon Schall created KAFKA-13631: Summary: Warning "Ignoring await stop request for non-present connector..." when shuting down connector via API in distributed mode Key: KAFKA-13631 URL: