[jira] [Resolved] (KAFKA-9834) Add interface to set ZSTD compresson level

[ https://issues.apache.org/jira/browse/KAFKA-9834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiamei xie resolved KAFKA-9834. --- Resolution: Duplicate Duplicate with KAFKA-7632 > Add interface to set ZSTD compresson level >

Jenkins build is back to normal : kafka-2.1-jdk8 #265

See

[jira] [Resolved] (KAFKA-9826) Log cleaning repeatedly picks same segment with no effect when first dirty offset is past start of active segment

[ https://issues.apache.org/jira/browse/KAFKA-9826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-9826. Fix Version/s: 2.6.0 Resolution: Fixed merged the PR to trunk > Log cleaning repeatedly picks same

[jira] [Created] (KAFKA-9871) Consumer group description showing duplicate partition information

OM Singh created KAFKA-9871: --- Summary: Consumer group description showing duplicate partition information Key: KAFKA-9871 URL: https://issues.apache.org/jira/browse/KAFKA-9871 Project: Kafka

Jenkins build is back to normal : kafka-trunk-jdk11 #1353

See

Build failed in Jenkins: kafka-2.2-jdk8 #45

See Changes: [matthias] MINOR: Upgrade ducktape to 0.7.7 (#8487) -- [...truncated 2.74 MB...] kafka.security.auth.SimpleAclAuthorizerTest >

[jira] [Created] (KAFKA-9870) Error is reported until the cluster is restarted. Error for partition [__consumer_offsets,19] to broker 0:org.apache.kafka.common.errors.NotLeaderForPartitionException: T

yonglu gao created KAFKA-9870: - Summary: Error is reported until the cluster is restarted. Error for partition [__consumer_offsets,19] to broker 0:org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that

[jira] [Created] (KAFKA-9869) kafka offset is discontinuous when appear when appear EOFException

Lee chen created KAFKA-9869: --- Summary: kafka offset is discontinuous when appear when appear EOFException Key: KAFKA-9869 URL: https://issues.apache.org/jira/browse/KAFKA-9869 Project: Kafka

Build failed in Jenkins: kafka-2.0-jdk8 #318

See Changes: [matthias] MINOR: Upgrade ducktape to 0.7.7 (#8487) -- [...truncated 436.35 KB...] kafka.coordinator.transaction.TransactionMarkerRequestCompletionHandlerTest

Re: [DISCUSS] KIP-591: Add Kafka Streams config to set default store type

Thanks for writing this up Matthias. I'm also thinking that maybe we can allow customized store types to be replaced as a single config knob. I'm even thinking if this can be extended to processor API besides DSL as well: e.g. if we allow classes like "KeyValueStoreBuilder" to take a constructor

Build failed in Jenkins: kafka-trunk-jdk8 #4432

See Changes: [github] MINOR: Upgrade ducktape to 0.7.7 (#8487) -- [...truncated 3.02 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Jenkins build is back to normal : kafka-1.0-jdk8 #293

See

Re: [DISCUSS] KIP-584: Versioning scheme for features

Hi Jun, Thanks a lot for the feedback and the questions! Please find my response below. > 200. The UpdateFeaturesRequest includes an AllowDowngrade field. It seems > that field needs to be persisted somewhere in ZK? (Kowshik): Great question! Below is my explanation. Please help me understand,

[jira] [Resolved] (KAFKA-5676) MockStreamsMetrics should be in o.a.k.test

[ https://issues.apache.org/jira/browse/KAFKA-5676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-5676. Resolution: Fixed > MockStreamsMetrics should be in o.a.k.test >

[jira] [Reopened] (KAFKA-5676) MockStreamsMetrics should be in o.a.k.test

[ https://issues.apache.org/jira/browse/KAFKA-5676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-5676: > MockStreamsMetrics should be in o.a.k.test > -- > >

Build failed in Jenkins: kafka-1.1-jdk8 #290

See Changes: [matthias] MINOR: Upgrade ducktape to 0.7.7 (#8487) -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building

Re: [DISCUSS] KIP-594: Expose output topic names from TopologyTestDriver

Andy, thanks for the KIP. The motivation is a little unclear to me: > This information allows all the outputs of a test run to be captured without > prior knowledge of the output topics. Given that the TTD users writes the `Topology` themselves, they should always have prior knowledge what

Re: [DISCUSS] KIP-591: Add Kafka Streams config to set default store type

While writing the KIP, the idea about custom stores also came to my mind, but I thought it could be done in a follow up KIP. However, given that two people asked about it, it might be better to just do it right away. Using a "super supplier interface" instead of an String config seems to be the

Re: [DISCUSS] KIP-584: Versioning scheme for features

Hi, Kowshik, Thanks for addressing those comments. Just a few more minor comments. 200. The UpdateFeaturesRequest includes an AllowDowngrade field. It seems that field needs to be persisted somewhere in ZK? 201. UpdateFeaturesResponse has the following top level fields. Should those fields be

Build failed in Jenkins: kafka-trunk-jdk11 #1352

See Changes: [github] KAFKA-9539; Add leader epoch in StopReplicaRequest (KIP-570) (#8257) -- [...truncated 3.04 MB...]

[Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

Hey all, I would like to start the vote for KIP-588: https://cwiki.apache.org/confluence/display/KAFKA/KIP-588 %3A+Allow+producers+to+recover+gracefully+from+transaction+timeouts Feel free to continue posting on discussion thread if you have any questions, thanks! Best, Boyang

[jira] [Resolved] (KAFKA-7544) Transient Failure: org.apache.kafka.streams.integration.EosIntegrationTest.shouldNotViolateEosIfOneTaskFails

[ https://issues.apache.org/jira/browse/KAFKA-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-7544. Resolution: Duplicate Closing this ticket as it links to the older, non-parameterized

[jira] [Created] (KAFKA-9868) Flaky Test EOSUncleanShutdownIntegrationTest.shouldWorkWithUncleanShutdownWipeOutStateStore

Sophie Blee-Goldman created KAFKA-9868: -- Summary: Flaky Test EOSUncleanShutdownIntegrationTest.shouldWorkWithUncleanShutdownWipeOutStateStore Key: KAFKA-9868 URL:

Re: Preferred Partition Leaders

Hi Lukasz, that sounds like a worthwhile addition to me! Just a few thoughts that occurred to me while reading your mail "Preferred Leader" as a term is somewhat taken in the Kafka namespace. It refers to the concept that the first leader from the list of replicas is the preferred leader and will

Jenkins build is back to normal : kafka-trunk-jdk11 #1351

See

Re: Preferred Partition Leaders

#nailedit On Tue, Apr 14, 2020 at 2:05 PM Jamie wrote: > Hi All, > > There is a KIP which might be of interest to you: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=120736982 - > it sounds like you want to blacklist brokers in DC3? > > Thanks, > > Jamie > > -Original

Re: Preferred Partition Leaders

Hi All,  There is a KIP which might be of interest to you:  https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=120736982 - it sounds like you want to blacklist brokers in DC3?  Thanks,  Jamie -Original Message- From: Michael K. Edwards To: dev@kafka.apache.org Sent: Tue,

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

Thanks Raymond for the proposal! Yea, adding a unified forwarding envelope request is a good idea, but it doesn't really solve our problem in this KIP. On Mon, Apr 13, 2020 at 11:14 PM Raymond Ng wrote: > Hi Boyang, > > Thanks for the KIP. Overall looks great. > > One suggestion: instead of

[jira] [Created] (KAFKA-9867) Log cleaner throws InvalidOffsetException for some partitions

Bradley created KAFKA-9867: -- Summary: Log cleaner throws InvalidOffsetException for some partitions Key: KAFKA-9867 URL: https://issues.apache.org/jira/browse/KAFKA-9867 Project: Kafka Issue Type:

Re: Preferred Partition Leaders

I have clients with a similar need relating to disaster recovery. (Three replicas per partition within a data center / AWS AZ/region, fourth replica elsewhere, ineligible to become the partition leader without manual intervention.) On Tue, Apr 14, 2020 at 12:31 PM Łukasz Antoniak wrote: > Hi

[jira] [Created] (KAFKA-9866) Do not attempt to elect preferred leader replicas which are outside ISR

Stanislav Kozlovski created KAFKA-9866: -- Summary: Do not attempt to elect preferred leader replicas which are outside ISR Key: KAFKA-9866 URL: https://issues.apache.org/jira/browse/KAFKA-9866

Re: Apache Feathercast

Thanks Jun! I'll reach out to Rich about a date. On Tue, 14 Apr 2020 at 19:11, Jun Rao wrote: > Hi, Sonke, > > That sounds like a good opportunity for promoting Kafka. Thanks for > volunteering. > > Jun > > On Tue, Apr 14, 2020 at 8:08 AM Sönke Liebau > wrote: > > > Hi all, > > > > Rich is

Preferred Partition Leaders

Hi Everyone, Recently I came across Kafka setup where two data centers are close to each other, but the company could not find a suitable place for the third one. As a result third DC is little further, lower network throughput, but still within range of decent network latency, qualifying for

[jira] [Resolved] (KAFKA-9539) Add leader epoch in StopReplicaRequest

[ https://issues.apache.org/jira/browse/KAFKA-9539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-9539. Fix Version/s: 2.6.0 Resolution: Fixed > Add leader epoch in StopReplicaRequest >

Re: [Discuss] KIP-581: Value of optional null field which has default value

Hi Cheng, Thanks for the KIP! I really appreciate the care that was taken to ensure backwards compatibility for existing users, and the minimal changes to public interface that are suggested to address this. I have two quick requests for clarification: 1) Where is the proposed

Re: [DISCUSS] KIP-591: Add Kafka Streams config to set default store type

Hi all, Thanks for starting this, Matthias! I've had multiple people mention this feature request to me. Actually, the most recent such request was from someone developing an LMDB-backed set of store implementations, as a drop-in replacement for RocksDB, so Sophie's suggestion seems relevant.

Re: Apache Feathercast

Hi, Sonke, That sounds like a good opportunity for promoting Kafka. Thanks for volunteering. Jun On Tue, Apr 14, 2020 at 8:08 AM Sönke Liebau wrote: > Hi all, > > Rich is trying to reboot the Feathercast series [1] and looking for > projects that would be willing to give a quick intro about

Essential of Kafka

Hi There! This is not really a development question, but I need some inspiration and feedback on my video about a Kafka streaming process. The link of the video is here: https://www.youtube.com/watch?v=G_kZc_eSLoo. And this is the process I have animated: [cid:image001.png@01D6124F.04183520]

[RESULTS] [VOTE] 2.5.0 RC3

Thanks everyone! The vote passes with 7 +1 votes (4 of which are binding) and no 0 or -1 votes. 4 binding +1 votes from PMC members Manikumar, Jun, Colin, and Matthias 1 committer +1 vote from Bill 2 community +1 votes from Israel Ekpo and Jonathan Santilli Voting email thread

[DISCUSS] KIP-594: Expose output topic names from TopologyTestDriver

Hey all, I would like to start off the discussion for KIP-594: https://cwiki.apache.org/confluence/display/KAFKA/KIP-594%3A+Expose+output+topic+names+from+TopologyTestDriver This KIP proposes to expose the names of the topics a topology produces records during a test run from the

Apache Feathercast

Hi all, Rich is trying to reboot the Feathercast series [1] and looking for projects that would be willing to give a quick intro about what they do. I'd be happy to do this for Kafka if no one has any objections. I am sure there are many people eminently more suited to this task than me though,

[jira] [Created] (KAFKA-9865) Expose output topic names from TopologyTestDriver

Andy Coates created KAFKA-9865: -- Summary: Expose output topic names from TopologyTestDriver Key: KAFKA-9865 URL: https://issues.apache.org/jira/browse/KAFKA-9865 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-9864) Avoid expensive QuotaViolationException usage

Lucas Bradstreet created KAFKA-9864: --- Summary: Avoid expensive QuotaViolationException usage Key: KAFKA-9864 URL: https://issues.apache.org/jira/browse/KAFKA-9864 Project: Kafka Issue

[jira] [Created] (KAFKA-9863) update the deprecated --zookeeper option in the documentation into --bootstrap-server

Luke Chen created KAFKA-9863: Summary: update the deprecated --zookeeper option in the documentation into --bootstrap-server Key: KAFKA-9863 URL: https://issues.apache.org/jira/browse/KAFKA-9863 Project:

Re: [kafka-clients] [VOTE] 2.5.0 RC3

Hello, I have ran the tests (passed) Follow the quick start guide with scala 2.12 (success) +1 Thanks! -- Jonathan On Tue, Apr 14, 2020 at 1:16 AM Colin McCabe wrote: > +1 (binding) > > verified checksums > ran unitTest > ran check > > best, > Colin > > On Tue, Apr 7, 2020, at 21:03, David

Build failed in Jenkins: kafka-trunk-jdk11 #1350

See Changes: [github] KAFKA-9853: Improve performance of Log.fetchOffsetByTimestamp (#8474) [github] KAFKA-9842; Add test case for OffsetsForLeaderEpoch grouping in Fetcher

Build failed in Jenkins: kafka-trunk-jdk11 #1349

See Changes: [github] HOTFIX: need to cleanup any tasks closed in TaskManager (#8463) -- [...truncated 4.77 MB...] kafka.zk.KafkaZkClientTest > testDeletePath STARTED

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

Hi Boyang, Thanks for the KIP. Overall looks great. One suggestion: instead of bumping the version and adding an optional field (PrincipalName) for a number of requests, can we consider adding a general ProxyRequest that acts as an "envelope" for the forwarded requests? A few advantages to this