[jira] [Created] (KAFKA-9835) Race condition with concurrent write allows reads above high watermark

Jason Gustafson created KAFKA-9835: -- Summary: Race condition with concurrent write allows reads above high watermark Key: KAFKA-9835 URL: https://issues.apache.org/jira/browse/KAFKA-9835 Project: Kaf

Build failed in Jenkins: kafka-trunk-jdk11 #1333

See Changes: [wangguoz] KAFKA-9801: Still trigger rebalance when static member joins in -- [...truncated 1.62 MB...] org.apache.kafka.streams.test.TestRecordTest > test

Build failed in Jenkins: kafka-trunk-jdk8 #4411

See Changes: [wangguoz] KAFKA-9801: Still trigger rebalance when static member joins in -- [...truncated 1.62 MB...] org.apache.kafka.streams.test.ConsumerRecordFactoryTe

回复：回复：回复：回复：回复：回复：[Vote] KIP-571: Add option to force remove members in StreamsResetter

Hi Boyang, Thanks for reminding me of that! I'm not sure about the convention, I thought it would need to re-collect votes if the KIP has changed~ Let's leave the vote thread here for 2 days, if no objection, I will take it as approved and update the PR accordingly. Thanks! Feyman

[jira] [Created] (KAFKA-9834) Add config to set ZSTD compresson level

jiamei xie created KAFKA-9834: - Summary: Add config to set ZSTD compresson level Key: KAFKA-9834 URL: https://issues.apache.org/jira/browse/KAFKA-9834 Project: Kafka Issue Type: Bug R

Jenkins build is back to normal : kafka-2.5-jdk8 #90

See

Re: 回复：回复：回复：回复：回复：[Vote] KIP-571: Add option to force remove members in StreamsResetter

You should already get enough votes if I'm counting correctly (Guozhang, John, Matthias) On Tue, Apr 7, 2020 at 6:59 PM feyman2009 wrote: > Hi, Boyang&Matthias > I think Matthias's proposal makes sense, but we can use the admin tool > for this scenario as Boyang mentioned or follow up later,

[VOTE] 2.5.0 RC3

Hello Kafka users, developers and client-developers, This is the forth candidate for release of Apache Kafka 2.5.0. * TLS 1.3 support (1.2 is now the default) * Co-groups for Kafka Streams * Incremental rebalance for Kafka Consumer * New metrics for better operational insight * Upgrade Zookeeper

[jira] [Resolved] (KAFKA-9833) Kafka Connect: Allow environment variable to set output directory of Kafka Connect logs

[ https://issues.apache.org/jira/browse/KAFKA-9833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Moe resolved KAFKA-9833. Resolution: Abandoned > Kafka Connect: Allow environment variable to set output directory of Kafka

[jira] [Created] (KAFKA-9833) Allow environment variable to set output directory of Kafka Connect logs

Brandon Moe created KAFKA-9833: -- Summary: Allow environment variable to set output directory of Kafka Connect logs Key: KAFKA-9833 URL: https://issues.apache.org/jira/browse/KAFKA-9833 Project: Kafka

Issue with stale consumer groups

Hi Dev, At my company we run Kafka version 0.10.1.1 in production. I noticed a strange behavior around stale consumers and wanted some help. So there are some consumer group ids which are no longer being used, but kafka seems to think that the group is rebalancing. root@kafka32:/opt/kafka/bin# ./

Build failed in Jenkins: kafka-trunk-jdk8 #4410

See Changes: [github] KAFKA-8107; Flaky Test -- [...truncated 5.99 MB...] org.apache.kafka.streams.TopologyTestDriverTest > shouldRespectTaskIdling[Eos enabled = false]

回复：回复：回复：回复：回复：回复：[Vote] KIP-571: Add option to force remove members in StreamsResetter

Hi, Boyang&Matthias I think Matthias's proposal makes sense, but we can use the admin tool for this scenario as Boyang mentioned or follow up later, so I prefer to keep this KIP unchanged to minimize the scope. Calling for vote ~ Thanks! Feyman --

Build failed in Jenkins: kafka-trunk-jdk11 #1332

See Changes: [github] KAFKA-8107; Flaky Test -- [...truncated 3.01 MB...] org.apache.kafka.streams.TopologyTestDriverTest > shouldPunctuateOnStreamsTime[Eos enabled = f

Jenkins build is back to normal : kafka-trunk-jdk11 #1331

See

[jira] [Created] (KAFKA-9832) Extend EOS system tests for EOS-beta

Matthias J. Sax created KAFKA-9832: -- Summary: Extend EOS system tests for EOS-beta Key: KAFKA-9832 URL: https://issues.apache.org/jira/browse/KAFKA-9832 Project: Kafka Issue Type: Sub-task

[DISCUSS] KIP-589 Add API to Update Replica State in Controller

Hey everyone, I'd like to start the discussion for KIP-589, part of the KIP-500 effort https://cwiki.apache.org/confluence/display/KAFKA/KIP-589+Add+API+to+update+Replica+state+in+Controller This is a proposal to use a new RPC instead of ZooKeeper for notifying the controller of an offline repli

Re: 回复：回复：回复：回复：回复：[Vote] KIP-571: Add option to force remove members in StreamsResetter

Hey Feyman, I think Matthias' suggestion is optional, and we could just use admin tool to remove single static members as well. Boyang On Tue, Apr 7, 2020 at 11:00 AM Matthias J. Sax wrote: > > Would you mind to elaborate why we still need that > > Sure. > > For static memership, the session t

[jira] [Created] (KAFKA-9831) Failing test: EosIntegrationTest.shouldNotViolateEosIfOneTaskFailsWithState[exactly_once_beta]

John Roesler created KAFKA-9831: --- Summary: Failing test: EosIntegrationTest.shouldNotViolateEosIfOneTaskFailsWithState[exactly_once_beta] Key: KAFKA-9831 URL: https://issues.apache.org/jira/browse/KAFKA-9831

Re: 回复：回复：回复：回复：回复：[Vote] KIP-571: Add option to force remove members in StreamsResetter

> Would you mind to elaborate why we still need that Sure. For static memership, the session timeout it usually set quite high. This make scaling in an application tricky: if you shut down one instance, no rebalance would happen until `session.timeout.ms` hits. This is specific to Kafka Streams,

Jenkins build is back to normal : kafka-trunk-jdk8 #4408

See

[jira] [Created] (KAFKA-9830) DeadLetterQueueReporter leaks KafkaProducer instance

Greg Harris created KAFKA-9830: -- Summary: DeadLetterQueueReporter leaks KafkaProducer instance Key: KAFKA-9830 URL: https://issues.apache.org/jira/browse/KAFKA-9830 Project: Kafka Issue Type: Bu

Build failed in Jenkins: kafka-trunk-jdk11 #1330

See Changes: [github] MINOR: Pass one action per unique resource name in -- [...truncated 3.01 MB...] org.apache.kafka.streams.test.ConsumerRecordFactoryTest > shouldRe

Re: [DISCUSS] (KAFKA-9806) authorize cluster operation when creating internal topics

On Tue, Apr 7, 2020, at 08:08, Paolo Moriello wrote: > Hi Colin, > > Thanks for your interest in this. I agree with you, this change could break > compatibility. However, changing the source principal is non trivial in > this case. In fact, here the problem is not in the internal topic creation >

Re: [jira] [Created] (KAFKA-9829) Kafka brokers are un-registered on Zookeeper node replacement

What value of zookeeper.connect is specified in server.properties of the broker. If it is having IPs rather than hostname then it is expected behaviour as all three Zookeeper nodes are destroyed and broker will not be able to communicate with Zookeeper. Thanks, Nitin On Tue, Apr 7, 2020 at 8:44 P

[jira] [Created] (KAFKA-9829) Kafka brokers are un-registered on Zookeeper node replacement

Pradeep created KAFKA-9829: -- Summary: Kafka brokers are un-registered on Zookeeper node replacement Key: KAFKA-9829 URL: https://issues.apache.org/jira/browse/KAFKA-9829 Project: Kafka Issue Type:

Re: [DISCUSS] (KAFKA-9806) authorize cluster operation when creating internal topics

Hi Colin, Thanks for your interest in this. I agree with you, this change could break compatibility. However, changing the source principal is non trivial in this case. In fact, here the problem is not in the internal topic creation - which succeeds - but in the two subsequent LeaderAndIsr and Upd

Re: 2.5.0 Producer.sendOffsetsToTransaction() and Backwards Compatibility

Thanks; Matthias, Very useful. On Mon, Apr 6, 2020 at 8:05 PM Matthias J. Sax wrote: > I guess one important point to mention is why Kafka Streams needs the > internal config though: it's about a save upgrade path. > > Even if the user tells us that they are on old brokers, we call the new > `s

Re: Shutdown Broker Because Log Dirs Failed

Hi Qarl, ideally you'd run Kafka on a linux host. If all you have is a Windows machine then Docker might be an option, Confluent has images available that you could use. Alternatively some form of virtual machine running linux would work as well. Best regards, Sönke Qarl schrieb am Di., 7. Apr.

回复：回复：回复：回复：回复：[Vote] KIP-571: Add option to force remove members in StreamsResetter

Hi, Matthias Thanks a lot! So you do not plan so support removing a _single static_ member via `StreamsResetter`? => Would you mind to elaborate why we still need that if we are able to batch remove active members with adminClient? Thanks! Feyman

RE: Shutdown Broker Because Log Dirs Failed

Thank you so much for the response! If this is unresolved, could you suggest any alternative for me to make this work? If that is possible. Any other solution is highly appreciated! Thank you. From: Sönke Liebau Sent: Tuesday, 7 April, 2020 6:56 PM To: dev Subject: Re: Shutdown Broker Because

Re: Shutdown Broker Because Log Dirs Failed

Hi Qarl, there probably (and sadly) is no good answer to your question. Windows is not an officially supported platform to run Kafka on and there is a large number of unresolved tickets in Jira on this topic. Most notable is probably KAFKA-1194 [1]. I haven't closely followed this, I thought we w

Shutdown Broker Because Log Dirs Failed

Hi, my name is Qarl. I am a newbie in software development, and I am using Kafka in this software. I have been facing this issue for weeks now. I’m running Kafka on Windows platform. The issue is when I run the Kafka server, and the retention period has reached, it will clear the logs file but t

Re: [VOTE] KIP-444: Refactor and Augment Metrics for Kafka Streams

Hi Guozhang, Thank you for the additions. Best, Bruno On Fri, Apr 3, 2020 at 2:08 AM Guozhang Wang wrote: > > A correction for the previous email: > > >- number of alive global threads, INFO >- number of alive restore threads, INFO > > > These two metrics are not going to be added in th

Build failed in Jenkins: kafka-2.4-jdk8 #184

See Changes: [jason] KAFKA-9815; Ensure consumer always re-joins if JoinGroup fails (#8420) -- [...truncated 5.58 MB...] org.apache.kafka.streams.TopologyTestDriverTest > sh

Build failed in Jenkins: kafka-trunk-jdk8 #4407

See Changes: [github] MINOR: Should cleanup the tasks after dirty close (#8433) [github] KAFKA-9818: improve error message to debug test (#8423) -- [...truncated 2.99 MB

[jira] [Created] (KAFKA-9828) Add partition to TestRecord in streams test-utils

Levani Kokhreidze created KAFKA-9828: Summary: Add partition to TestRecord in streams test-utils Key: KAFKA-9828 URL: https://issues.apache.org/jira/browse/KAFKA-9828 Project: Kafka Issue

Build failed in Jenkins: kafka-trunk-jdk11 #1329

See Changes: [github] MINOR: Should cleanup the tasks after dirty close (#8433) [github] KAFKA-9818: improve error message to debug test (#8423) -- [...truncated 3.01 M

Re: [DISCUSS] KIP-584: Versioning scheme for features

Hi Jun, I have updated the KIP for the item 111. I'm in the process of addressing 100.6, and will provide an update soon. I think item 110 is still under discussion given we are now providing a way to finalize all features to their latest version levels. In any case, please let us know how you fee

Re: [DISCUSS] KIP-584: Versioning scheme for features

Hi Colin, Thanks for the feedback, and suggestions! It is a great idea to provide a `--finalize-latest` flag. I agree it's a burden to ask the user to manually upgrade each feature to the latest version, after a release. I have now updated the KIP adding this idea. > What about a simple solution