Re: [jira] [Created] (KAFKA-9413) Add audit of Kafka cluster

2020-01-14 Thread Владимир Беруненко
On 2020/01/13 11:15:00, "Igor Martemyanov (Jira)" wrote: > Igor Martemyanov created KAFKA-9413: > --- > > Summary: Add audit of Kafka cluster > Key: KAFKA-9413 > URL:

Re: [VOTE] KIP-535: Allow state stores to serve stale reads during rebalance

2020-01-14 Thread Matthias J. Sax
Thanks. SGTM. -Matthias On 1/14/20 4:52 PM, John Roesler wrote: > Hey Matthias, > > Thanks for taking a look! I felt a little uneasy about it, but didn’t think > about the case you pointed out. Throwing an exception would indeed be safer. > > Given a choice between throwing in the default

Re: [DISCUSS] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

2020-01-14 Thread Matthias J. Sax
I was just checking the existing code, and we currently log at WARN level if the handler returns CONTINUE -- we did not have any complaints about it, hence, I don't see an issue with WARN -- and we should keep it consistent. I think the explicit mentioning of `ClassCastException` is based on the

CompletableFuture?

2020-01-14 Thread radai
Hi With kip-118 (Drop Support for Java 7) officially done, is there a timeline replacing usage of "plain" Futures with java 8 CompletableFutures? kafka 2.0 was mentioned at some point as a possible target for this ...

[jira] [Resolved] (KAFKA-9365) Add consumer group information to txn commit

2020-01-14 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen resolved KAFKA-9365. Resolution: Fixed > Add consumer group information to txn commit >

Build failed in Jenkins: kafka-trunk-jdk11 #1082

2020-01-14 Thread Apache Jenkins Server
See Changes: [github] KAFKA-9294: Add tests for Named parameter (#7927) [mumrah] KAFKA-8764: LogCleanerManager endless loop while compacting/cleaning [wangguoz] KAFKA-9365: Add server side change to include

Build failed in Jenkins: kafka-trunk-jdk8 #4160

2020-01-14 Thread Apache Jenkins Server
See Changes: [github] KAFKA-9294: Add tests for Named parameter (#7927) [mumrah] KAFKA-8764: LogCleanerManager endless loop while compacting/cleaning [wangguoz] KAFKA-9365: Add server side change to include

Re: [VOTE] KIP-535: Allow state stores to serve stale reads during rebalance

2020-01-14 Thread John Roesler
Hey Matthias, Thanks for taking a look! I felt a little uneasy about it, but didn’t think about the case you pointed out. Throwing an exception would indeed be safer. Given a choice between throwing in the default method or defining a new interface and throwing if the wrong interface is

Build failed in Jenkins: kafka-trunk-jdk11 #1081

2020-01-14 Thread Apache Jenkins Server
See Changes: [github] KAFKA-9420; Add flexible version support for converted protocols (#7931) -- [...truncated 5.72 MB...]

Re: [VOTE] KIP-535: Allow state stores to serve stale reads during rebalance

2020-01-14 Thread Matthias J. Sax
Thanks for the PR. That helps a lot. I actually do have a concern: the proposed default method, would disable the new feature to allow querying an active task during restore automatically. Hence, if a user has an existing custom store type, and would use the new KafkaStreams.store(..., true)

Re: [DISCUSS] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

2020-01-14 Thread Bill Bejeck
Hi Anna, Thanks for getting this KIP going again. I agree with pushing forward on option 0 as well. I a couple of questions about the KIP as written. The KIP states that any {{ClassCastException}} thrown plus any other unchecked exceptions will result in a log statement and not stop processing

Re: [DISCUSS] KIP-558: Track the set of actively used topics by connectors in Kafka Connect

2020-01-14 Thread Randall Hauch
Oh, one more thing: 9. There's no mention of how the status topic is partitioned, or how partitioning will be used by the new topic records. The KIP should probably outline this for clarity and completeness. Best regards, Randall On Tue, Jan 14, 2020 at 5:25 PM Randall Hauch wrote: > Thanks,

Re: [VOTE] On the new KIP-158: Kafka Connect allows source connectors to set topic settings when creating new topics

2020-01-14 Thread Bill Bejeck
Thanks for the KIP, Konstantine. The KIP is very clear and it seems like it will have a very positive impact. +1(binding) -Bill On Tue, Jan 14, 2020 at 2:02 PM Tom Bentley wrote: > +1 (non-binding). Thanks for the KIP! > > On Tue, Jan 14, 2020 at 6:45 PM Randall Hauch wrote: > > > Thanks

Jenkins build is back to normal : kafka-trunk-jdk8 #4159

2020-01-14 Thread Apache Jenkins Server
See

Re: [DISCUSS] KIP-558: Track the set of actively used topics by connectors in Kafka Connect

2020-01-14 Thread Randall Hauch
Thanks, Konstantine. Overall, this KIP looks interesting and really useful, and for the most part is spot on. I do have a number of questions/comments about specifics: 1. The topic records have a value that includes the connector name, task number that last reported the topic is used, and

Re: [VOTE] KIP-535: Allow state stores to serve stale reads during rebalance

2020-01-14 Thread John Roesler
Hi again all, I've sent a PR including this new option, and while implementing it, I realized we also have to make a (source-compatible) addition to the QueryableStoreType interface, so that the IQ store wrapper can pass the flag down to the composite store provider. See

Re: [ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

2020-01-14 Thread James Cheng
Congrats Colin, Vahid, and Manikumar! -James > On Jan 14, 2020, at 10:59 AM, Tom Bentley wrote: > > Congratulations! > > On Tue, Jan 14, 2020 at 6:57 PM Rajini Sivaram > wrote: > >> Congratulations Colin, Vahid and Manikumar! >> >> Regards, >> Rajini >> >> On Tue, Jan 14, 2020 at 6:32 PM

[jira] [Reopened] (KAFKA-8677) Flakey test GroupEndToEndAuthorizationTest#testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl

2020-01-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-8677: Reopening this ticket. Test failed again: 

Re: [DISCUSS] KIP-546: Add quota-specific APIs to the Admin Client, redux

2020-01-14 Thread Brian Byrne
Hi Colin, Your feedback is appreciated, thank you. On Tue, Jan 14, 2020 at 11:34 AM Colin McCabe wrote: > This is probably a nitpick, but it would be nice to specify that this list > is in order of highest priority to lowest. > Done. > Hmm. Maybe --show-overridden or --include-overridden

Re: [VOTE] KIP-555: An admin tools proposal to accomplish the deprecation of zookeeper access that is direct

2020-01-14 Thread Gwen Shapira
+1 (binding, re-vote) On Tue, Jan 14, 2020 at 11:23 AM Colin McCabe wrote: > > Hi all, > > I'm reposting this since I've been informed that gmail mashed the original > VOTE thread into a different email thread. Hopefully the different thread > title will prevent it doing that in this case. >

[jira] [Resolved] (KAFKA-9346) Consumer fetch offset back-off with pending transactions

2020-01-14 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen resolved KAFKA-9346. Resolution: Fixed > Consumer fetch offset back-off with pending transactions >

Re: [VOTE] KIP-535: Allow state stores to serve stale reads during rebalance

2020-01-14 Thread John Roesler
Thanks for calling this out, Matthias. You're correct that this looks like a harmful behavioral change. I'm fine with adding the new overload you mentioned, just a simple boolean flag to enable the new behavior. I'd actually propose that we call this flag "queryStaleData", with a default of

Jenkins build is back to normal : kafka-trunk-jdk11 #1080

2020-01-14 Thread Apache Jenkins Server
See

Re: [DISCUSS] KIP-515: Enable ZK client to use the new TLS supported authentication

2020-01-14 Thread Rajini Sivaram
Hi Ron, Thanks for the detailed explanation, sounds good to me. A few more questions: 1) At the moment, all sensitive broker configs including keystore/truststore passwords can be stored encrypted in ZooKeeper prior to starting up brokers. We are now adding SSL keystore/truststore passwords for

[jira] [Resolved] (KAFKA-9420) Bump APIs to enable flexible versions

2020-01-14 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-9420. Fix Version/s: 2.5.0 Resolution: Fixed > Bump APIs to enable flexible versions >

Re: [DISCUSS] KIP-546: Add quota-specific APIs to the Admin Client, redux

2020-01-14 Thread Colin McCabe
Hi Brian, Thanks for the KIP. The KIP says: > As represented by the current ZK node structure, the order in which quotas > are matched are as follows. Note is a specified user principal, > is a specified client ID, and is a special default > user/client ID that matches to all users or

Re: [VOTE] KIP-555: An admin tools proposal to accomplish the deprecation of zookeeper access that is direct

2020-01-14 Thread Ron Dagostino
Hi Colin. +1 from me (non-binding). (re-voting on this thread) Ron On Tue, Jan 14, 2020 at 2:23 PM Colin McCabe wrote: > > Hi all, > > I'm reposting this since I've been informed that gmail mashed the original > VOTE thread into a different email thread. Hopefully the different thread >

Re: [VOTE] KIP-555: Deprecate Direct Zookeeper access in Kafka Administrative Tools

2020-01-14 Thread Ron Dagostino
Hi Colin. +1 from me (non-binding). Ron On Tue, Jan 14, 2020 at 12:40 PM Colin McCabe wrote: > > Hi all, > > I'd like to start the vote on KIP-555: Deprecate Direct Zookeeper access in > Kafka Administrative Tools. > > KIP: https://cwiki.apache.org/confluence/x/Wg6dC > > Discussion thread:

[VOTE] KIP-555: An admin tools proposal to accomplish the deprecation of zookeeper access that is direct

2020-01-14 Thread Colin McCabe
Hi all, I'm reposting this since I've been informed that gmail mashed the original VOTE thread into a different email thread. Hopefully the different thread title will prevent it doing that in this case. I'd like to start the vote on KIP-555: Deprecate Direct Zookeeper access in Kafka

Re: [VOTE] KIP-535: Allow state stores to serve stale reads during rebalance

2020-01-14 Thread Matthias J. Sax
During the discussion of KIP-216 (https://cwiki.apache.org/confluence/display/KAFKA/KIP-216%3A+IQ+should+throw+different+exceptions+for+different+errors) we encountered that KIP-535 introduces a behavior change that is not backward compatible, hence, I would like to request a small change.

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

2020-01-14 Thread Matthias J. Sax
Vito, It's still unclear to me what the advantage is, to have both `StreamsNotRunningException` and `StateStoreNotAvailableException`? For both cased, the state is `PENDING_SHUTDOWN / NOT_RUNNING / ERROR` and thus, for a user point of view, why does it matter if the store is closed on not? I

Re: [VOTE] On the new KIP-158: Kafka Connect allows source connectors to set topic settings when creating new topics

2020-01-14 Thread Tom Bentley
+1 (non-binding). Thanks for the KIP! On Tue, Jan 14, 2020 at 6:45 PM Randall Hauch wrote: > Thanks for the updated KIP, Konstantine. I have a few minor nits, but all > are around the implementation details. > > +1 (binding) > > Best regards, > > Randall > > On Mon, Jan 13, 2020 at 10:16 AM

Re: [ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

2020-01-14 Thread Tom Bentley
Congratulations! On Tue, Jan 14, 2020 at 6:57 PM Rajini Sivaram wrote: > Congratulations Colin, Vahid and Manikumar! > > Regards, > Rajini > > On Tue, Jan 14, 2020 at 6:32 PM Mickael Maison > wrote: > > > Congrats Colin, Vahid and Manikumar! > > > > On Tue, Jan 14, 2020 at 5:43 PM Ismael Juma

Re: [ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

2020-01-14 Thread Matthias J. Sax
Congrats! On 1/14/20 10:49 AM, Rajini Sivaram wrote: > Congratulations Colin, Vahid and Manikumar! > > Regards, > Rajini > > On Tue, Jan 14, 2020 at 6:32 PM Mickael Maison > wrote: > >> Congrats Colin, Vahid and Manikumar! >> >> On Tue, Jan 14, 2020 at 5:43 PM Ismael Juma wrote: >>> >>>

Re: [VOTE] KIP-551: Expose disk read and write metrics

2020-01-14 Thread David Arthur
+1 binding This will be very nice to have. Thanks for the KIP, Colin. -David On Tue, Jan 14, 2020 at 11:39 AM Sönke Liebau wrote: > +1 (non-binding) > > Thanks for creating this! > > On Tue, 14 Jan 2020 at 17:36, Mitchell wrote: > > > +1 (non-binding)! > > Very useful kip. > > -mitch > > > >

Re: [ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

2020-01-14 Thread Rajini Sivaram
Congratulations Colin, Vahid and Manikumar! Regards, Rajini On Tue, Jan 14, 2020 at 6:32 PM Mickael Maison wrote: > Congrats Colin, Vahid and Manikumar! > > On Tue, Jan 14, 2020 at 5:43 PM Ismael Juma wrote: > > > > Congratulations Colin, Vahid and Manikumar! > > > > Ismael > > > > On Tue,

Build failed in Jenkins: kafka-trunk-jdk8 #4158

2020-01-14 Thread Apache Jenkins Server
See Changes: [github] KAFKA-7538: Reduce lock contention for Partition ISR lock (#5866) -- [...truncated 2.82 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Re: [VOTE] KIP-526: Reduce Producer Metadata Lookups for Large Number of Topics

2020-01-14 Thread Brian Byrne
Hello all, After further offline discussion, I've removed any efforts to control metadata RPC sizes. There are now only two items proposed in this KIP: (1) When encountering a new topic, only issue a metadata request for that particular topic. For all other cases, continue as it does today with

Re: [VOTE] On the new KIP-158: Kafka Connect allows source connectors to set topic settings when creating new topics

2020-01-14 Thread Randall Hauch
Thanks for the updated KIP, Konstantine. I have a few minor nits, but all are around the implementation details. +1 (binding) Best regards, Randall On Mon, Jan 13, 2020 at 10:16 AM Konstantine Karantasis < konstant...@confluent.io> wrote: > Hi everyone. > > I hope y'all had a nice break. The

Re: [ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

2020-01-14 Thread Mickael Maison
Congrats Colin, Vahid and Manikumar! On Tue, Jan 14, 2020 at 5:43 PM Ismael Juma wrote: > > Congratulations Colin, Vahid and Manikumar! > > Ismael > > On Tue, Jan 14, 2020 at 9:30 AM Gwen Shapira wrote: > > > Hi everyone, > > > > I'm happy to announce that Colin McCabe, Vahid Hashemian and

[jira] [Resolved] (KAFKA-9395) Improve Kafka scheduler's periodic maybeShrinkIsr()

2020-01-14 Thread Brian Byrne (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Byrne resolved KAFKA-9395. Assignee: Rajini Sivaram (was: Brian Byrne) Resolution: Done > Improve Kafka scheduler's

Re: [ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

2020-01-14 Thread Ismael Juma
Congratulations Colin, Vahid and Manikumar! Ismael On Tue, Jan 14, 2020 at 9:30 AM Gwen Shapira wrote: > Hi everyone, > > I'm happy to announce that Colin McCabe, Vahid Hashemian and Manikumar > Reddy are now members of Apache Kafka PMC. > > Colin and Manikumar became committers on Sept 2018

[jira] [Created] (KAFKA-9431) Expose API in KafkaStreams to fetch all local offset lags

2020-01-14 Thread Vinoth Chandar (Jira)
Vinoth Chandar created KAFKA-9431: - Summary: Expose API in KafkaStreams to fetch all local offset lags Key: KAFKA-9431 URL: https://issues.apache.org/jira/browse/KAFKA-9431 Project: Kafka

[VOTE] KIP-555: Deprecate Direct Zookeeper access in Kafka Administrative Tools

2020-01-14 Thread Colin McCabe
Hi all, I'd like to start the vote on KIP-555: Deprecate Direct Zookeeper access in Kafka Administrative Tools. KIP: https://cwiki.apache.org/confluence/x/Wg6dC Discussion thread:

[jira] [Created] (KAFKA-9430) Tighten up lag estimates when source topic optimization is on

2020-01-14 Thread Vinoth Chandar (Jira)
Vinoth Chandar created KAFKA-9430: - Summary: Tighten up lag estimates when source topic optimization is on Key: KAFKA-9430 URL: https://issues.apache.org/jira/browse/KAFKA-9430 Project: Kafka

[jira] [Created] (KAFKA-9429) Allow ability to control whether stale reads out of state stores are desirable

2020-01-14 Thread Vinoth Chandar (Jira)
Vinoth Chandar created KAFKA-9429: - Summary: Allow ability to control whether stale reads out of state stores are desirable Key: KAFKA-9429 URL: https://issues.apache.org/jira/browse/KAFKA-9429

Re: [DISCUSS] KIP-550: Mechanism to Delete Stray Partitions on Broker

2020-01-14 Thread Colin McCabe
Hi Dhruvil, Thanks for the KIP. I think there should be some way to turn this off, in case that becomes necessary. I'm also curious how long we intend to wait between detecting the duplication and deleting the extra logs. The KIP says "scheduled for deletion" but doesn't give a time frame

Re: [ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

2020-01-14 Thread Bill Bejeck
Congrats Colin, Vahid and Manikumar! Well deserved. -Bill On Tue, Jan 14, 2020 at 12:30 PM Gwen Shapira wrote: > Hi everyone, > > I'm happy to announce that Colin McCabe, Vahid Hashemian and Manikumar > Reddy are now members of Apache Kafka PMC. > > Colin and Manikumar became committers on

Re: [ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

2020-01-14 Thread Stanislav Kozlovski
Congratulations to all! Best, Stanislav On Tue, Jan 14, 2020 at 9:30 AM Gwen Shapira wrote: > Hi everyone, > > I'm happy to announce that Colin McCabe, Vahid Hashemian and Manikumar > Reddy are now members of Apache Kafka PMC. > > Colin and Manikumar became committers on Sept 2018 and Vahid on

[jira] [Created] (KAFKA-9428) Expose standby information in KafkaStreams via queryMetadataForKey API

2020-01-14 Thread Vinoth Chandar (Jira)
Vinoth Chandar created KAFKA-9428: - Summary: Expose standby information in KafkaStreams via queryMetadataForKey API Key: KAFKA-9428 URL: https://issues.apache.org/jira/browse/KAFKA-9428 Project:

[ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

2020-01-14 Thread Gwen Shapira
Hi everyone, I'm happy to announce that Colin McCabe, Vahid Hashemian and Manikumar Reddy are now members of Apache Kafka PMC. Colin and Manikumar became committers on Sept 2018 and Vahid on Jan 2019. They all contributed many patches, code reviews and participated in many KIP discussions. We

Re: Streams, Kafka windows

2020-01-14 Thread Sachin Mittal
You can try to convert the final resultant stream to table. Check this page for more info: https://docs.confluent.io/current/streams/faq.html#how-can-i-convert-a-kstream-to-a-ktable-without-an-aggregation-step This way table would always contain the latest (single) record for a given key. Sachin

Re: [DISCUSS] KIP-555: Deprecate Direct Zookeeper access in Kafka Administrative Tools

2020-01-14 Thread Colin McCabe
Hi Ron, Yes, the KIP-455 command line changes will be in 2.5. regards, Colin On Sat, Jan 11, 2020, at 03:21, Ron Dagostino wrote: > << Wow, I totally missed that! Great. > > Would the deprecation for kafka-reassign-partitions depend on KIP 455 > being merged? > > Ron > > > On Jan 10,

Re: [DISCUSS] KIP-515: Enable ZK client to use the new TLS supported authentication

2020-01-14 Thread Colin McCabe
Hi Ron, Thanks for the explanation. I guess thinking about it a little bit more, we should just add --zk-tls-config-file to all of these commands. We will be removing this option (plus ZK in general) from these commands in the next major release, but ZK is still supported in 2.5, so we should

[jira] [Created] (KAFKA-9427) StateRestoreListener.onRestoreEnd should return actual message count

2020-01-14 Thread Chris Stromberger (Jira)
Chris Stromberger created KAFKA-9427: Summary: StateRestoreListener.onRestoreEnd should return actual message count Key: KAFKA-9427 URL: https://issues.apache.org/jira/browse/KAFKA-9427 Project:

Re: [DISCUSS] KIP-390: Add producer option to adjust compression level

2020-01-14 Thread smiklos
Hi, Is there any update on this? I've done performance test with Avro data and Snappy compression. Setting the buffer from 32kb to 128kb brings a rough 10% decrease in storage which is a big deal. I could offer working on this as well. Best regards, Miklos

Re: [DISCUSS] KIP-515: Enable ZK client to use the new TLS supported authentication

2020-01-14 Thread Ron Dagostino
Hi Rajini. Thanks for pointing out kafka-acls as another bootstrap use case. I will update the KIP to mention it. (BTW, I sent an email on 1/8 stating that I had updated the KIP to reflect the current understanding and discussion, but that email did not go through -- the KIP has undergone a

Streams, Kafka windows

2020-01-14 Thread Viktor Markvardt
Hi, My name is Viktor. I'm currently working with Kafka streams and have several questions about Kafka and I can not find answers in the official docs. 1. Why suppress functionality does not work with Hopping windows? How to make it work? Example of the code: KStream finalStream = source

Re: [VOTE] KIP-551: Expose disk read and write metrics

2020-01-14 Thread Sönke Liebau
+1 (non-binding) Thanks for creating this! On Tue, 14 Jan 2020 at 17:36, Mitchell wrote: > +1 (non-binding)! > Very useful kip. > -mitch > > On Tue, Jan 14, 2020 at 10:26 AM Manikumar > wrote: > > > > +1 (binding). > > > > Thanks for the KIP. > > > > > > > > On Sun, Jan 12, 2020 at 1:23 AM

Re: [VOTE] KIP-551: Expose disk read and write metrics

2020-01-14 Thread Mitchell
+1 (non-binding)! Very useful kip. -mitch On Tue, Jan 14, 2020 at 10:26 AM Manikumar wrote: > > +1 (binding). > > Thanks for the KIP. > > > > On Sun, Jan 12, 2020 at 1:23 AM Lucas Bradstreet wrote: > > > +1 (non binding) > > > > On Sat, 11 Jan 2020 at 02:32, M. Manna wrote: > > > > > Hey

Build failed in Jenkins: kafka-trunk-jdk8 #4157

2020-01-14 Thread Apache Jenkins Server
See Changes: [jason] MINOR: Fix connect:mirror checkstyle (#7951) [wangguoz] KAFKA-9159: The caller of sendListOffsetRequest need to handle retriable [wangguoz] KAFKA-9346: Consumer back-off logic when

[jira] [Resolved] (KAFKA-6212) Kafka Streams - Incorrect partition rebalancing

2020-01-14 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-6212. -- Resolution: Cannot Reproduce > Kafka Streams - Incorrect partition rebalancing >

Re: [VOTE] KIP-551: Expose disk read and write metrics

2020-01-14 Thread Manikumar
+1 (binding). Thanks for the KIP. On Sun, Jan 12, 2020 at 1:23 AM Lucas Bradstreet wrote: > +1 (non binding) > > On Sat, 11 Jan 2020 at 02:32, M. Manna wrote: > > > Hey Colin, > > > > +1 - Really useful for folks managing a cluster by themselves. > > > > M. MAnna > > > > On Fri, 10 Jan 2020

Re: [DISCUSS] KIP-515: Enable ZK client to use the new TLS supported authentication

2020-01-14 Thread Rajini Sivaram
Resending this note which wasn't delivered when it was sent on Jan 6th: Hi Ron, Thanks for taking over this KIP. Is it possible to use SASL/GSSAPI with SSL for ZooKeeper like we do in Kafka? If not, we need to describe that in the migration section. We should also document the principals used

Re: [DISCUSS] KIP-515: Enable ZK client to use the new TLS supported authentication

2020-01-14 Thread Ron Dagostino
Hi Colin. <<< It seems like this [--zk-tls-config-file information] could just appear in a configuration file, which all of these tools already accept (I think) ZkSecurityMigrator has no such property file facility; adding a "--zk-tls-config-file" parameter is exactly for this purpose. If we

[jira] [Created] (KAFKA-9426) OffsetsForLeaderEpochClient Use Switch Statement

2020-01-14 Thread David Mollitor (Jira)
David Mollitor created KAFKA-9426: - Summary: OffsetsForLeaderEpochClient Use Switch Statement Key: KAFKA-9426 URL: https://issues.apache.org/jira/browse/KAFKA-9426 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-9425) InFlightRequests Class Uses Thread-Safe Counter Non-Thread-Safe Collection

2020-01-14 Thread David Mollitor (Jira)
David Mollitor created KAFKA-9425: - Summary: InFlightRequests Class Uses Thread-Safe Counter Non-Thread-Safe Collection Key: KAFKA-9425 URL: https://issues.apache.org/jira/browse/KAFKA-9425 Project:

Re: [DISCUSS] KIP-489 Kafka Consumer Record Latency Metric

2020-01-14 Thread Sean Glover
Hi Habib, Thank you for the reminder. I'll update the KIP this week and address the feedback from you and Gokul. Regards, Sean On Tue, Jan 14, 2020 at 9:06 AM Habib Nahas wrote: > Any chance of an update on the KIP? We are interested in seeing this move > forward. > > Thanks, > Habib > Sr

[jira] [Resolved] (KAFKA-7538) Improve locking model used to update ISRs and HW

2020-01-14 Thread Rajini Sivaram (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-7538. --- Fix Version/s: 2.5.0 Reviewer: Jason Gustafson Resolution: Fixed > Improve

Build failed in Jenkins: kafka-trunk-jdk11 #1079

2020-01-14 Thread Apache Jenkins Server
See Changes: [jason] MINOR: Fix connect:mirror checkstyle (#7951) [wangguoz] KAFKA-9159: The caller of sendListOffsetRequest need to handle retriable [wangguoz] KAFKA-9346: Consumer back-off logic when

Re: [VOTE] KIP-216: IQ should throw different exceptions for different errors

2020-01-14 Thread Navinder Brar
+1 (non-binding) With a small comment which was mentioned by Vinoth as well. Did we fix on the flag for StreamsRebalancingException, I don't see it in the KIP. -Navinder On Tuesday, 14 January, 2020, 08:00:11 pm IST, Vito Jeng wrote: Hi, all, I would like to start the vote for

[VOTE] KIP-216: IQ should throw different exceptions for different errors

2020-01-14 Thread Vito Jeng
Hi, all, I would like to start the vote for KIP-216. Currently, IQ throws InvalidStateStoreException for any types of error. With this KIP, user can distinguish different types of error. KIP is here:

Re: [DISCUSS] KIP-489 Kafka Consumer Record Latency Metric

2020-01-14 Thread Habib Nahas
Any chance of an update on the KIP? We are interested in seeing this move forward. Thanks, Habib Sr SDE, AWS On Wed, Dec 18, 2019, at 3:27 PM, Habib Nahas wrote: > Thanks Sean. Look forward to the updated KIP. > > Regards, > Habib > > On Fri, Dec 13, 2019, at 6:22 AM, Sean Glover wrote: > >

Re: [VOTE] KIP-545 support automated consumer offset sync across clusters in MM 2.0

2020-01-14 Thread Navinder Brar
+1 (non-binding) Navinder On Tuesday, 14 January, 2020, 07:24:02 pm IST, Ryanne Dolan wrote: Bump. We've got 4 non-binding and one binding vote. Ryanne On Fri, Dec 13, 2019, 1:44 AM Tom Bentley wrote: > +1 (non-binding) > > On Thu, Dec 12, 2019 at 6:33 PM Andrew Schofield < >

Re: [DISCUSS] KIP-550: Mechanism to Delete Stray Partitions on Broker

2020-01-14 Thread Dhruvil Shah
If there are no more questions or concerns, I will start a vote thread tomorrow. Thanks, Dhruvil On Mon, Jan 13, 2020 at 6:59 PM Dhruvil Shah wrote: > Hi Nikhil, > > Thanks for looking at the KIP. The kind of race condition you mention is > not possible as stray partition detection is done

[jira] [Created] (KAFKA-9424) Using AclCommand,avoid call the global method loadcache in SimpleAclAuthorizer

2020-01-14 Thread Steven Lu (Jira)
Steven Lu created KAFKA-9424: Summary: Using AclCommand,avoid call the global method loadcache in SimpleAclAuthorizer Key: KAFKA-9424 URL: https://issues.apache.org/jira/browse/KAFKA-9424 Project: Kafka

Re: [VOTE] KIP-545 support automated consumer offset sync across clusters in MM 2.0

2020-01-14 Thread Ryanne Dolan
Bump. We've got 4 non-binding and one binding vote. Ryanne On Fri, Dec 13, 2019, 1:44 AM Tom Bentley wrote: > +1 (non-binding) > > On Thu, Dec 12, 2019 at 6:33 PM Andrew Schofield < > andrew_schofi...@live.com> > wrote: > > > +1 (non-binding) > > > > On 12/12/2019, 14:20, "Mickael Maison" >

Re: Issues with triggering the build

2020-01-14 Thread Levani Kokhreidze
Hi Bruno, Thanks for trying. I’ve also tried with “retest this please” but didn’t do any good. Seems like it affects other PRs as well. - Levani > On Jan 14, 2020, at 12:13 PM, Bruno Cadonna wrote: > > Hi, > > I tried with "Retest this, please" but it didn't work. > > Best, > Bruno > > On

[jira] [Created] (KAFKA-9423) Refine layout of configuration options on website and make individual settings directly linkable

2020-01-14 Thread Jira
Sönke Liebau created KAFKA-9423: --- Summary: Refine layout of configuration options on website and make individual settings directly linkable Key: KAFKA-9423 URL: https://issues.apache.org/jira/browse/KAFKA-9423

Re: Issues with triggering the build

2020-01-14 Thread Bruno Cadonna
Hi, I tried with "Retest this, please" but it didn't work. Best, Bruno On Tue, Jan 14, 2020 at 9:01 AM Levani Kokhreidze wrote: > > Hello, > > Seems like there’re issues with triggering Jenkins builds. Latest commits for > my PR doesn’t trigger any of the builds. > Any ideas how to fix the

Issues with triggering the build

2020-01-14 Thread Levani Kokhreidze
Hello, Seems like there’re issues with triggering Jenkins builds. Latest commits for my PR doesn’t trigger any of the builds. Any ideas how to fix the issue? Here’s the example PR: https://github.com/apache/kafka/pull/7170 - Levani