[jira] [Created] (KAFKA-14155) Kafka Stream - State transition from RUNNING to ERROR

2022-08-09 Thread Harsha Nadig (Jira)
Harsha Nadig created KAFKA-14155: Summary: Kafka Stream - State transition from RUNNING to ERROR Key: KAFKA-14155 URL: https://issues.apache.org/jira/browse/KAFKA-14155 Project: Kafka Issue

Re: [VOTE] KIP - 405: Kafka Tiered Storage.

2021-02-11 Thread Harsha Chintalapani
+1 (binding). Thanks, Harsha On Thu, Feb 11, 2021 at 6:21 AM Satish Duggana wrote: > Hi All, > We would like to start voting on “KIP-405: Kafka Tiered Storage”. > > For reference here is the KIP: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-405%3A+Kafka+Tiered+

Re: [DISCUSS] KIP-688: Support dynamic update of delete.topic.enable config

2020-11-25 Thread Harsha Ch
to provide a simpler way to gate keep certain functionalities such as delete topic. Thanks, Harsha On Wed, Nov 25, 2020 at 10:25 AM Gwen Shapira wrote: > Thanks for bringing this up! > > I disagree that " there is a valid use-case to keep the flag > "delete.t

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2020-11-10 Thread Harsha Chintalapani
will be delivered and we can call that as preview. We will be running this in our production and continue to provide the data and metrics to push this feature to GA. On Tue, Nov 10, 2020 at 10:07 AM, Kowshik Prakasam wrote: > Hi Harsha/Satish, > > Thanks for the discussion today. Here is a link t

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2020-09-07 Thread Harsha Ch
Hi All, We are all working through the last meeting feedback. I'll cancel the tomorrow 's meeting and we can meanwhile continue our discussion in mailing list. We can start the regular meeting from next week onwards. Thanks, Harsha On Fri, Sep 04, 2020 at 8:41 AM, Satish Duggana

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2020-08-26 Thread Harsha Ch
Updated the KIP with Meeting Notes section https://cwiki.apache.org/confluence/display/KAFKA/KIP-405%3A+Kafka+Tiered+Storage#KIP405:KafkaTieredStorage-MeetingNotes On Tue, Aug 25, 2020 at 1:03 PM Jun Rao wrote: > Hi, Harsha, > > Thanks for the summary. Could you add th

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2020-08-25 Thread Harsha Chintalapani
dd to it I missed anything. Will produce a formal meeting notes from next meeting onwards. Thanks, Harsha On Mon, Aug 24, 2020 at 9:42 PM, Ying Zheng wrote: > We did some basic feature tests at Uber. The test cases and results are > shared in this google doc: > https://docs.google

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2020-08-24 Thread Harsha Ch
"Understand commitments towards driving design & implementation of the KIP further and how it aligns with participant interests in contributing to the efforts (ex: in the context of Uber’s Q3/Q4 roadmap)." What is that about? On Mon, Aug 24, 2020 at 11:05 AM Kowshik Prakasam wrote

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2020-08-19 Thread Harsha Ch
Hi Jun, Thanks. This will help a lot. Tuesday will work for us. -Harsha On Wed, Aug 19, 2020 at 1:24 PM Jun Rao wrote: > Hi, Satish, Ying, Harsha, > > Do you think it would be useful to have a regular virtual meeting to > discuss this KIP? The goal of the meeting wil

Re: [VOTE] KIP-651 - Support PEM format for SSL certificates and private key

2020-08-08 Thread Harsha Ch
+1 binding. Thanks, Harsha On Sat, Aug 8, 2020 at 2:07 AM Manikumar wrote: > +1 (binding) > > Thanks for the KIP. > > > On Fri, Aug 7, 2020 at 12:56 AM David Jacot wrote: > > > Supporting PEM is really nice. Thanks, Rajini. > > > > +1 (non-binding) >

Re: [VOTE] KIP-578: Add configuration to limit number of partitions

2020-08-05 Thread Harsha Ch
Thanks for the updates Gokul.  +1 (binding). Thanks, Harsha On Wed, Aug 05, 2020 at 8:18 AM, Gokul Ramanan Subramanian < gokul24...@gmail.com > wrote: > > > > Hi. > > > > I request the community to kindly resume the voting process for KIP-578. > If

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2020-07-15 Thread Harsha Ch
at this point to push the data to remote storage. We can ofcourse implement another RLMM that is optional for users to configure to push to remote. But that doesn't need to be addressed in this KIP. Thanks, Harsha On Wed, Jul 15, 2020 at 5:50 PM Colin McCabe wrote: > Hi Ying, > &g

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2020-07-09 Thread Harsha Chintalapani
already added some of the perf tests results in the KIP itself. It will be great if we can get design and code reviews from you and others in the community as we make progress. Thanks, Harsha On Tue, Jul 7, 2020 at 10:34 AM Jun Rao wrote: > Hi, Ying, > > Thanks for the update.

Re: [VOTE] KIP-578: Add configuration to limit number of partitions

2020-05-27 Thread Harsha Chintalapani
Thanks for the KIP Gokul. This will be really useful for our use cases as well. +1 (binding). -Harsha On Tue, May 26, 2020 at 12:33 AM, Gokul Ramanan Subramanian < gokul24...@gmail.com> wrote: > Hi. > > Any votes for this? > > Thanks. > > On Tue, May 12, 202

Re: [VOTE] KIP-602: Change default value for client.dns.lookup

2020-05-22 Thread Harsha Chintalapani
+1 (binding). -Harsha On Fri, May 22, 2020 at 10:04 AM Ismael Juma wrote: > Thanks for the KIP, +1 (binding). > > Ismael > > On Fri, May 22, 2020 at 1:40 AM Badai Aqrandista > wrote: > > > Hi All, > > > > I would like to start the vote on KIP-602: Change

Re: [VOTE] KIP-545 support automated consumer offset sync across clusters in MM 2.0

2020-05-21 Thread Harsha Ch
+1 (binding). Good addition to MM 2. -Harsha On Thu, May 21, 2020 at 8:36 AM, Manikumar < manikumar.re...@gmail.com > wrote: > > > > +1 (binding). > > > > Thanks for the KIP. > > > > On Thu, May 21, 2020 at 9:49 AM Maulin Vasavada <

Re: [VOTE] KIP-612: Ability to Limit Connection Creation Rate on Brokers

2020-05-18 Thread Harsha Ch
+1 (binding) On Mon, May 18 2020 at 4:54 PM, Anna Povzner < a...@confluent.io > wrote: > > > > Hi All, > > > > I would like to start the vote on KIP-612: Ability to limit connection > creation rate on brokers. > > > > For reference, here is the KIP wiki: >

Re: [VOTE] KIP-519: Make SSL context/engine configuration extensible

2020-03-30 Thread Harsha Chintalapani
Thanks for putting the KIP together. This is going to help quite a bit in customizing SSL. +1 (binding). Thanks, Harsha On Mon, Mar 30, 2020 at 7:44 PM Maulin Vasavada wrote: > Thanks Jun Rao for your vote and comments. > > For 1) Earlier it was the security.ssl package but after a

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2020-02-26 Thread Harsha Chintalapani
re is already > > available). > > The above discussion is based on the assumption that we need to > > cache the > > object metadata locally in every broker. I mentioned earlier that > > an > > alternative is to just store/retrieve those metadata in an external >

[jira] [Created] (KAFKA-9578) Kafka Tiered Storage - System Tests

2020-02-20 Thread Harsha (Jira)
Harsha created KAFKA-9578: - Summary: Kafka Tiered Storage - System Tests Key: KAFKA-9578 URL: https://issues.apache.org/jira/browse/KAFKA-9578 Project: Kafka Issue Type: Sub-task

Re: Possible to create Scrum board under Kafka project in JIRA?

2020-02-14 Thread Harsha Chintalapani
All the dependency JIRAs are created and linked to the EPIC here https://issues.apache.org/jira/browse/KAFKA-7739 . Lets drive it through that. -Harsha On Fri, Feb 14, 2020 at 2:28 AM, Alexandre Dupriez < alexandre.dupr...@gmail.com> wrote: > Good morning, > > Would it be po

Re: [DISCUSS] KIP-501 Avoid out-of-sync or offline partitions when follower fetch requests not processed in time

2020-02-11 Thread Harsha Chintalapani
it in the ISR and leave the disk read time on leader side out of this. Thanks, Harsha On Mon, Feb 10, 2020 at 9:26 PM, Lucas Bradstreet wrote: > Hi Harsha, > > Is the problem you'd like addressed the following? > > Assume 3 replicas, L and F1 and F2. > > 1. F1 and F2 are al

Re: [DISCUSS] KIP-501 Avoid out-of-sync or offline partitions when follower fetch requests not processed in time

2020-02-10 Thread Harsha Ch
Hi Jason & Jun,                  Do you have any feedback on the KIP or is it ok take it to voting?. Its good to have this config in Kafka to address disk failure scenarios as described in the KIP. Thanks, Harsha On Mon, Feb 10, 2020 at 5:10 PM, Brian Sang < bais...@yelp.com.invalid

Re: [VOTE] KIP-559: Make the Kafka Protocol Friendlier with L7 Proxies

2020-01-23 Thread Harsha Chintalapani
+1 ( binding). Much needed! -Harsha On Thu, Jan 23, 2020 at 7:17 PM, Guozhang Wang wrote: > +1 (binding) > > On Thu, Jan 23, 2020 at 1:55 PM Guozhang Wang wrote: > > Yeah that makes sense, it is a good-to-have if we can push through this in > 2.5 but if we do not have ban

Re: [DISCUSS] KIP-501 Avoid out-of-sync or offline partitions when follower fetch requests not processed in time

2019-12-04 Thread Harsha Chintalapani
. Thanks, Harsha On Mon, Nov 18, 2019 at 7:05 PM, Satish Duggana wrote: > Hi Jason, > Thanks for looking into the KIP. Apologies for my late reply. Increasing > replica max lag to 30-45 secs did not help as we observed that a few fetch > requests took more than 1-2 minutes. We do not want

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2019-12-03 Thread Harsha Chintalapani
. Having said that JBOD tooling needs to evolve to support production loads. Most of the users will be interested in using tiered storage without JBOD support support on day 1. Thanks, Harsha As for meeting, we could have a KIP e-meeting on this if needed, but it > will be open to everyone and will

Re: [ANNOUNCE] New committer: John Roesler

2019-11-13 Thread Harsha Chintalapani
Congrats, John! -Harsha On Wed, Nov 13, 2019 at 5:42 AM, aishwarya kumar wrote: > Congratulations John!! > > On Wed, Nov 13, 2019 at 8:17 AM Rajini Sivaram > wrote: > > Congratulations, John! > > Regards, > > Rajini > > On Wed, Nov 13, 201

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2019-11-04 Thread Harsha Chintalapani
Hi Jun, Can you please take a look at Satish's reply. Let us know if that answers your question. I would like to get yours and the rest of the community thoughts on the general direction we are going as we continue to make progress. Thanks, Harsha On Fri, Nov 1, 2019 at 3:06 AM Satish

Re: [VOTE] KIP-514 Add a bounded flush() API to Kafka Producer

2019-10-25 Thread Harsha Chintalapani
+1 (binding) -Harsha On Fri, Oct 25 2019 at 11:01 AM, wrote: > > +1 > > On Thu, Oct 24, 2019 at 9:33 PM radai wrote: > > Hello, > > > > > I'd like to initiate a vote on KIP-514. > > > > > links: > the kip - > > https://cwiki.apache.or

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2019-10-22 Thread Harsha Chintalapani
you think. Thanks, Harsha On Tue, Oct 22, 2019 at 3:53 PM Jun Rao wrote: > Hi, Harsha, > > Historically, we tried using a feature branch in 0.8. The experience > actually wasn't great. Merging the feature branch to the main branch > required additional review work and each merg

Re: [VOTE] KIP-537: Increase default zookeeper session timeout

2019-10-22 Thread Harsha Chintalapani
+1 (binding). Thanks, Harsha On Tue, Oct 22, 2019 at 3:20 PM Colin McCabe wrote: > +1 (binding). > > Thanks, Jason. > > best, > Colin > > > On Mon, Oct 21, 2019, at 17:27, Jason Gustafson wrote: > > I'd like to start a vote for KIP-537: > > > https:/

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2019-10-21 Thread Harsha
a feature branch in apache kafka git. It will allow us collaborate in open source community rather than in private branches. Please let me know if you have any objections to opening a feature branch in kafka's git repo. Thanks, Harsha On Mon, Apr 8, 2019, at 10:04 PM, Harsha wrote: > Thanks,

Re: [VOTE] KIP-511: Collect and Expose Client's Name and Version in the Brokers

2019-10-11 Thread Harsha Chintalapani
+1 (binding). Thanks for the KIP this is going to be super useful. Thanks, Harsha On Fri, Oct 11, 2019 at 2:57 PM Guozhang Wang wrote: > Hi David, > > Thanks for the KIP. I know I'm late on voting here but I'd be +1 on this > too! > > Just a quick clarification on the tag

Re: [VOTE] KIP-416: Notify SourceTask of ACK'd offsets, metadata

2019-10-04 Thread Harsha Chintalapani
+1 (binding). Thanks, Harsha On Fri, Oct 04, 2019 at 6:53 AM, Manikumar wrote: > Hi All, > > Please vote here for the formal approval of this KIP. > https://github.com/apache/kafka/pull/6295#discussion_r328867657 > > Thanks, > > > On Wed, Oct 2, 2019 at 5:50 AM R

Re: [VOTE] KIP-524: Allow users to choose config source when describing configs

2019-09-25 Thread Harsha Chintalapani
Thanks for the KIP. +1 (binding). -Harsha On Wed, Sep 25, 2019 at 11:55 AM Satish Duggana wrote: > Thanks Jason for the KIP. It looks to be a minor improvement but very > useful. > > +1 (non-binding) > > On Wed, Sep 25, 2019 at 9:29 PM Rajini Sivaram > wrote: > > &g

Re: [VOTE] KIP-525 - Return topic metadata and configs in CreateTopics response

2019-09-22 Thread Harsha Chintalapani
+1 (binding). -Harsha On Sat, Sep 21, 2019 at 12:13 AM, Manikumar wrote: > +1 (binding), Thanks for the KIP. > > Thanks, > > On Sat, Sep 21, 2019 at 2:49 AM Colin McCabe wrote: > > +1 (binding). Thanks, Rajini. > > best, > Colin > > On Fri, Sep 20, 2019, at

Re: [VOTE] KIP-517: Add consumer metrics to observe user poll behavior

2019-09-19 Thread Harsha Chintalapani
+1 (binding). Thanks for the KIP. -Harsha On Wed, Sep 18, 2019 at 9:07 AM Mickael Maison wrote: > +1 (non binding) > Thanks for the KIP, I can see this being really useful! > > On Wed, Sep 18, 2019 at 4:40 PM Kevin Lu wrote: > > > > Hi All, > > > > Sinc

Re: [VOTE] KIP-373: Allow users to create delegation tokens for other users

2019-09-16 Thread Harsha Ch
+1 (binding). Thanks for the KIP Viktor Thanks, Harsha On Mon, Sep 16, 2019 at 3:02 AM, Viktor Somogyi-Vass < viktorsomo...@gmail.com > wrote: > > > > Hi All, > > > > I'd like to bump this again in order to get some more binding votes and/or >

Re: [VOTE] KIP-309: Add toUpperCase support to sasl.kerberos.principal.to.local rule

2019-09-16 Thread Harsha Chintalapani
+1 (binding). Thanks, Harsha On Mon, Sep 16, 2019 at 4:21 AM, Manikumar wrote: > Hi all, > > I would like to start vote on this trivial KIP-309: https://cwiki.apache. > org/confluence/display/KAFKA/KIP > -309%3A+Add+toUpperCase+support+to+sasl.kerberos.principal.to.local+rule > > Thanks, >

Re: [DISCUSS] KIP-517: Add consumer metric indicating time between poll calls

2019-09-16 Thread Harsha Chintalapani
Thanks. +1 LGTM. On Mon, Sep 16, 2019 at 9:19 AM, Kevin Lu wrote: > Hi Harsha, > > Thanks for the feedback. I have added *last-poll-seconds-ago* to the KIP > (being consistent with *last-heartbeat-seconds-ago*). > > Regards, > Kevin > > On Sat, Sep 14, 2019 at

Re: [DISCUSS] KIP-491: Preferred Leader Deprioritized List (Temporary Blacklist)

2019-09-14 Thread Harsha Ch
orkaround given that Kafka itself knows when the topic retention would allow you to switch that replica to a leader" Not sure how its making it any complicated by having a single zk path to have a list of brokers. Thanks, Harsha On Mon, Sep 09, 2019 at 3:55 PM, Stanislav Kozlovski <

Re: [DISCUSS] KIP-517: Add consumer metric indicating time between poll calls

2019-09-14 Thread Harsha Chintalapani
Thanks Kevin for the KIP. Overall LGTM. On you second point, I think the metric will be really useful to indicate the perf bottlenecks on user code vs kakfa consumer/broker. Thanks, Harsha On Fri, Sep 13, 2019 at 2:41 PM, Kevin Lu wrote: > Hi Radai & Jason, > > Thanks fo

Re: [DISCUSS] KIP-491: Preferred Leader Deprioritized List (Temporary Blacklist)

2019-09-07 Thread Harsha Chintalapani
benefits having it in the system itself instead of asking users to hack bunch of json files to deal with outage scenario. Thanks, Harsha On Fri, Sep 6, 2019 at 4:36 PM George Li wrote: > Hi Colin, > > Thanks for the feedback. The "separate set of metadata about blacklists" &

Re: [VOTE] KIP-482: The Kafka Protocol should Support Optional Tagged Fields

2019-09-04 Thread Harsha Chintalapani
LGTM. +1 (binding) -Harsha On Wed, Sep 04, 2019 at 1:46 AM, Satish Duggana wrote: > +1 (non-binding) Thanks for the nice KIP. > > You may want to update the KIP saying that optional tagged fields do not > support complex types(or structs). > > On Wed, Sep 4, 2019 at 3:43 AM J

Re: [DISCUSS] KIP-515: Enable ZK client to use the new TLS supported authentication

2019-08-30 Thread Harsha Chintalapani
Thanks Pere. KIP looks good to me. -Harsha On Fri, Aug 30, 2019 at 10:05 AM, Pere Urbón Bayes wrote: > Not really, > my idea is to keep the JAAS parameter, so people don't see major > changes. But if you pass a properties file, then this takes precedence over > the other, w

Re: [DISCUSS] KIP-515: Enable ZK client to use the new TLS supported authentication

2019-08-30 Thread Harsha Chintalapani
nt section it will take precedence over zookeeper SSL configs? Thanks, Harsha On Fri, Aug 30, 2019 at 7:50 AM, Pere Urbón Bayes wrote: > Hi, > quick question, I saw in another mail that 2.4 release is planned for > September. I think it would be really awesome to have this for this > releas

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

2019-08-29 Thread Harsha Chintalapani
can move forward. Thanks, Harsha On Wed, Aug 28, 2019 at 1:51 PM, Maulin Vasavada wrote: > Hi Harsha > > As we synced-up offline on this topic, we hope you don't have any more > clarifications that you are seeking. If that is the case, can you please > help us move this forward a

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-22 Thread Harsha Ch
+1 (binding) Thanks, Harsha On Wed, Aug 21, 2019 at 4:23 PM, Robert Barrett < bob.barr...@confluent.io > wrote: > > > > +1 (non-binding) > > > > This will be great to have, thanks Jason! > > > > On Wed, Aug 21, 2019 at 4:29 AM Manikumar <

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

2019-08-20 Thread Harsha Chintalapani
. TrustManagerFactory tmf = TrustManagerFactory .getInstance(TrustManagerFactory.getDefaultAlgorithm());and use tmf.chekServerTrusted() or use https://docs.oracle.com/javase/7/docs/api/javax/net/ssl/TrustManagerFactory.html#getInstance(java.lang.String)if you want a specific provider. -Harsha

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

2019-08-20 Thread Harsha Chintalapani
the following file clearly indicates that it > does use custom algorithm- > > https://github.com/spiffe/java-spiffe/blob/master/src/main/java/spiffe/ > provider/SpiffeProvider.java#L17 > > > Algorithm value: > https://github.com/spiffe/java-spiffe/blob/master/src/main/java/spiff

Re: [VOTE] KIP-504 - Add new Java Authorizer Interface

2019-08-17 Thread Harsha Chintalapani
+1 (binding). Thanks, Harsha On Sat, Aug 17, 2019 at 2:53 AM, Manikumar wrote: > Hi, > > +1 (binding). > > Thanks for the KIP. LGTM. > > Regards, > Manikumar > > On Sat, Aug 17, 2019 at 4:41 AM Colin McCabe wrote: > > +1 (binding) > > Thanks, Ra

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-14 Thread Harsha Chintalapani
Thanks for the KIP Mickael. LGTM +1 (binding). -Harsha On Wed, Aug 14, 2019 at 1:10 PM, Colin McCabe wrote: > Thanks, Mickael. +1 (binding) > > best, > Colin > > On Wed, Aug 14, 2019, at 12:07, Gabor Somogyi wrote: > > +1 (non-binding) > I've read it through in depth

Re: [VOTE] KIP-503: deleted topics metric

2019-08-13 Thread Harsha Chintalapani
+1 (binding) Thanks, Harsha On Tue, Aug 13, 2019 at 12:08 PM, David Arthur wrote: > Hello all, > > I'd like to start the vote on KIP-503 > https://cwiki.apache.org/confluence/display/KAFKA/ > KIP-503%3A+Add+metric+for+number+of+topics+marked+for+deletion > > Thanks! > David >

Re: [DISCUSS] KIP-505 : Add new public method to only update assignment metadata in consumer

2019-08-12 Thread Harsha Chintalapani
requirements? Thanks, Harsha On Mon, Aug 12, 2019 at 2:53 PM, Jungtaek Lim wrote: > Thanks for the feedbacks Colin and Matthias. > > I agree with you regarding getting topics and partitions via AdminClient, > just curious how much the overhead would be. Would it be lighter, or > heavier?

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

2019-08-12 Thread Harsha Chintalapani
r > AdminClient, it would behave differently than what is proposed here. > Forcing every client to implement this change when calling auto create from > the producer specifically seems odd > I am not sure why its unintuitive , protocols change. We add or upgrade the existing protocols a

Re: [VOTE] KIP-499 - Unify connection name flag for command line tool

2019-08-08 Thread Harsha Chintalapani
+1 (binding). much needed!! On Thu, Aug 08, 2019 at 6:43 PM, Gwen Shapira wrote: > +1 (binding) THANK YOU. It would be +100 if I could. > > On Thu, Aug 8, 2019 at 6:37 PM Mitchell wrote: > > Hello Dev, > After the discussion I would like to start the vote for KIP-499 > > The following

Re: Ask for contributor access and write permission on wiki

2019-08-08 Thread Harsha Chintalapani
Hi Jungtaek, Gave you permissions on wiki. Please check. Thansk, Harsha On Thu, Aug 08, 2019 at 7:03 PM, Jungtaek Lim wrote: > Hi devs, > > I'd like to give a shot to make first contribution on Kafka community, as > I initiated thread on needs a new public API for met

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

2019-08-08 Thread Harsha Ch
are pretty much same as what you are asking for. Thanks, Harsha On Thu, Aug 8, 2019 at 3:47 PM Maulin Vasavada wrote: > Imagine a scenario like - We know we have a custom KMS and as a Kafka owner > we want to comply to using that KMS source to load keys/certs. As a Kafka > owner we

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

2019-08-08 Thread Harsha Chintalapani
Hi Maulin, On Thu, Aug 08, 2019 at 2:04 PM, Maulin Vasavada wrote: > Hi Harsha > > The reason we rejected the SslProvider route is that - we only needed a > custom way to load keys/certs. Not touch any policy that existing Providers > govern like SunJSSE Provider. > We hav

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

2019-08-08 Thread Harsha Chintalapani
security interfaces provided by Java itself. Thanks, Harsha On Thu, Aug 08, 2019 at 6:54 AM, Harsha Chintalapani wrote: > Hi Maulin, >Not sure if you looked at my previous replies. This changes > are not required as there is already security Provider to do what you are

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

2019-08-08 Thread Harsha Chintalapani
also addresses easy registration of such providers. Thanks, Harsha On Wed, Aug 07, 2019 at 11:31 PM, Maulin Vasavada wrote: > Bump! Can somebody please review this? > > On Tue, Jul 16, 2019 at 1:51 PM Maulin Vasavada > wrote: > > Bump! Can somebody please review this? > >

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

2019-08-07 Thread Harsha Chintalapani
On Wed, Aug 07, 2019 at 9:50 AM, Colin McCabe wrote: > On Wed, Aug 7, 2019, at 09:24, Harsha Ch wrote: > > On Tue, Aug 06, 2019 at 11:46 PM, Colin McCabe < cmcc...@apache.org > > wrote: > > On Tue, Aug 6, 2019, at 21:38, Harsha Ch wrote: > > Hi Colin, > "Hmm

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

2019-08-07 Thread Harsha Ch
On Tue, Aug 06, 2019 at 11:46 PM, Colin McCabe < cmcc...@apache.org > wrote: > > > > On Tue, Aug 6, 2019, at 21:38, Harsha Ch wrote: > > > >> >> >> Hi Colin, >> "Hmm... I'm not sure I follow. Users don't have to build their own

Re: [DISCUSS] KIP-504 - Add new Java Authorizer Interface

2019-08-07 Thread Harsha Chintalapani
Thanks for the KIP Rajini. Quick thought, it would be good to have a common module outside of clients that only applies to server side interfaces & changes. It looks like we are increasingly in favor of using Java interface for pluggable modules on the broker side. Thanks, Harsha On Tue,

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

2019-08-06 Thread Harsha Ch
't be exactly like Kafka consumer and why do we want to have producer to overrider server side config and while not allowing consumer to do so. We are not even allowing the same contract and this will cause more confusion from the users standpoint. Thanks, Harsha On Tue, Aug 6, 2019 at 9:09 PM Coli

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

2019-08-06 Thread Harsha Chintalapani
even broker side is turned off. Why can't we have the same behavior. You can still achieve the goal for this KIP by allowing auto.create.topics.enable to take precedence and have a similar behavior to that of consumer. Thanks, Harsha On Tue, Aug 06, 2019 at 6:06 PM, Harsha Ch wrote: >

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

2019-08-06 Thread Harsha Ch
turn them off. It will be the exact behavior as it is today, as far as producer is concerned. I am not understanding why not having server side configs to gateways such a hard requirement and this behavior exists today. As far I am concerned this breaks the backward compatibility. Thanks, Harsha

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

2019-08-06 Thread Harsha Chintalapani
anks, Harsha On Tue, Aug 06, 2019 at 7:41 AM, Ismael Juma wrote: > Hi Harsha, > > I mentioned policies and the authorizer. For example, with > CreateTopicPolicy, you can implement the limits you describe. If you have > ideas of how that should be improved, please submit a KIP. My point

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

2019-08-06 Thread Harsha Chintalapani
or not , if they are allowed set a ceiling on max no.of partitions and replication-factor. -Harsha On Mon, Aug 5 2019 at 8:58 PM, wrote: > Harsha, > > Rogue clients can use the admin client to create topics and partitions. > ACLs and policies can help in that case as well as this one. > >

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

2019-08-05 Thread Harsha Chintalapani
erver side configs of default topic, partitions should take higher precedence and client shouldn't be able to create a topic with higher no.of partitions, replication than what server config specifies. Thanks, Harsha On Mon, Aug 05, 2019 at 5:24 PM, Justine Olshan wrote: > Hi all, > I m

Re: [DISCUSS] KIP-503: deleted topics metric

2019-08-05 Thread Harsha Chintalapani
Thanks for the KIP. Its useful metric to have. LGTM. -Harsha On Mon, Aug 05, 2019 at 11:24 AM, David Arthur wrote: > Hello all, I'd like to start a discussion for > https://cwiki.apache.org/confluence/display/KAFKA/ > KIP-503%3A+Add+metric+for+number+of+topics+marked+for+deletion &

Re: [DISCUSS] KIP-499 - Unify connection name flag for command line tool

2019-08-01 Thread Harsha
+1 for the KIP. -Harsha On Thu, Aug 1, 2019, at 3:07 PM, Colin McCabe wrote: > On Wed, Jul 31, 2019, at 05:26, Mitchell wrote: > > Hi Jason, > > Thanks for looking at this! > > > > I wasn't exactly sure what to put in the compatibility section. I wrote > >

Re: [DISCUSS] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

2019-08-01 Thread Harsha
Hi Colin, Looks like KIP is missing the images , links are broken. Thanks, Harsha On Thu, Aug 1, 2019, at 2:05 PM, Colin McCabe wrote: > Hi all, > > I've written a KIP about removing ZooKeeper from Kafka. Please take a > look and let me know what you think

Re: [VOTE] KIP-492 Add java security providers in Kafka Security config

2019-07-29 Thread Harsha Chintalapani
Thanks for the KIP Sandeep. +1 (binding) Thanks, Harsha On Jul 29, 2019, 12:22 PM -0700, Sandeep Mopuri , wrote: > Hi all, after some good discussion > <https://www.mail-archive.com/dev@kafka.apache.org/msg99419.html> about the > KIP > <https://cwiki.apache.org/confluence/

Re: Fwd: [DISCUSS] KIP-492 Add java security providers in Kafka Security config

2019-07-25 Thread Harsha
there is a usecase where we benefit from having a provider config for SASL. On Thu, Jul 25, 2019, at 5:25 AM, Rajini Sivaram wrote: > Hi Sandeep/Harsha, > > I don't have any major concerns about this KIP since it solves a specific > issue and is a relatively minor change. I am unc

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2019-07-25 Thread Harsha
. This will not trigger any deletion of local segments. Thanks, Harsha On Thu, Jul 25, 2019, at 6:01 AM, Habib Nahas wrote: > Hi, > > Under the proposed definition of RemoteTier, would it be possible to > have an implementation that transfers older log segments to a slower > storag

Re: Fwd: [DISCUSS] KIP-492 Add java security providers in Kafka Security config

2019-07-24 Thread Harsha
Thanks for the details. Rajini, Can you please take a look and let us know if these addresses your concerns. Thanks, Harsha On Mon, Jul 22, 2019, at 9:36 AM, Sandeep Mopuri wrote: > Hi Rajini, > Thanks for raising the above questions. Please find the > replies below &g

Re: [DISCUSS] KIP-492 Add java security providers in Kafka Security config

2019-07-16 Thread Harsha
Thanks for the KIP Sandeep. LGTM. Mani & Rajini, can you please look at the KIP as well. Thanks, Harsha On Tue, Jul 16, 2019, at 2:54 PM, Sandeep Mopuri wrote: > Thanks for the suggestions, made changes accordingly. > > On Tue, Jul 16, 2019 at 9:27 AM Satish Duggana > wrote: &

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

2019-07-16 Thread Harsha
You can look at the implementation here for an example https://github.com/spiffe/spiffe-example/blob/master/java-spiffe/spiffe-security-provider/src/main/java/spiffe/api/provider/SpiffeProvider.java On Tue, Jul 16, 2019, at 9:00 PM, Harsha wrote: > Hi Mau

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

2019-07-16 Thread Harsha
KeyManagerFactory, TrustManagerFactory and add that your JVM settings and pass that factory name via configs exposed in KAFKA-8191. These are Java APIs and instead of adding custom apis like you are proposing in the KIP. -Harsha On Tue, Jul 16, 2019, at 1:51 PM, Maulin Vasavada wrote: > Bump!

Re: Possible implementation for KAFKA-560

2019-07-08 Thread Harsha
. Thanks, Harsha On Sun, Jun 23, 2019, at 9:40 PM, Carlos Manuel Duclos-Vergara wrote: > Hi, > Thanks for the answer. Looking at high water mark, then the logic would be > to flag the partitions that have > > high_watermark == log_start_offset > > In addition, I'm thinking t

Re: [VOTE] KIP-429: Kafka Consumer Incremental Rebalance Protocol

2019-05-22 Thread Harsha
+1 (binding). Thanks for the KIP looking forward for this to be avaiable in consumers. Thanks, Harsha On Wed, May 22, 2019, at 12:24 AM, Liquan Pei wrote: > +1 (non-binding) > > On Tue, May 21, 2019 at 11:34 PM Boyang Chen wrote: > > > Thank you Guozhang for all the har

Re: [VOTE] KIP-461 Improve Replica Fetcher behavior at handling partition failure

2019-05-09 Thread Harsha Chintalapani
Thanks for the KIP. +1(binding) Thanks, Harsha On May 9, 2019, 9:58 AM -0700, Jun Rao , wrote: > Hi, Aishwarya, > > Thanks for the KIP. +1 > > Jun > > On Wed, May 8, 2019 at 4:30 PM Aishwarya Gune > wrote: > > > Hi All! > > > > I would like to

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

2019-05-07 Thread Harsha
Thanks for the kip. LGTM +1. -Harsha On Mon, Apr 29, 2019, at 8:14 AM, Viktor Somogyi-Vass wrote: > Hi Jason, > > I too agree this is more of a problem in older versions and therefore we > could backport it. Were you thinking of any specific versions? I guess the > 2.x a

Re: [VOTE] KIP-433: Block old clients on brokers

2019-04-24 Thread Harsha
Hi Gwen & Ismael, Do you have any feedback on with the proposed approach, min.api.version allowing users to specify versions for every request. Thanks, Harsha On Fri, Apr 19, 2019, at 10:24 AM, Harsha wrote: > Thanks Ying for updating the KIP. >

Re: [VOTE] KIP-433: Block old clients on brokers

2019-04-19 Thread Harsha
Thanks Ying for updating the KIP. Hi Ismael, Given min.api.version allows admin/users to specifiy min.version for each request this should address your concerns right? Thanks, Harsha On Wed, Apr 17, 2019, at 2:29 PM, Ying Zheng wrote: > I have updated the config descript

Re: [VOTE] KIP-433: Block old clients on brokers

2019-04-12 Thread Harsha
release and it will be harder for users to figure out which release of sarama client they can use. Ying, if you have a different apporach which might address this issue please add. Thanks, Harsha On Fri, Apr 12, 2019, at 7:23 PM, Ismael Juma wrote: > Hi Harsha, > > There is no s

Re: [VOTE] KIP-433: Block old clients on brokers

2019-04-12 Thread Harsha
n in the error message that the broker only supports min.api.version and above. So that users can see a clear message and upgrade to a newer version. Thanks, Harsha On Fri, Apr 12, 2019, at 12:19 PM, Ismael Juma wrote: > Hi Ying, > > The actual reasons are important so that people can e

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2019-04-08 Thread Harsha
the remote storage might help in terms of efficiency but this requires Protocol changes and some way of syncing ACLs in Kafka to the Remote storage. Thanks, Harsha On Mon, Apr 8, 2019, at 8:48 AM, Ron Dagostino wrote: > Hi Harsha. A couple of questions. I think I know the answers, but it &g

Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always Round-Robin" Selection

2019-04-04 Thread Harsha
Looks like the KIP is passed with 3 binding votes. From Matthias, Bill Bejeck and myself you got 3 binding votes. You can do the full tally of the votes and send out a close of vote thread. Thanks, Harsha On Thu, Apr 4, 2019, at 12:24 PM, M. Manna wrote: > Hello, > > Trying

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2019-04-03 Thread Harsha
ver in a broker causes bigger than usual churn in the consumers? (I'm thinking about the time required to rebuild remote index files.)" Rebuild remote index files will only happen in case of remote storage missing all the copied index files. Fail-over will not trigger this rebuild. Hi Rya

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2019-04-01 Thread Harsha
arly days of Kafka, there is external module which used to contain kafka to hdfs copy tools and dependencies. We would like to have RLM (class implementation) and RSM(interface) to be in core and as you suggested, implementation of RSM could be in another package so that the dependencies of

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2019-03-27 Thread Harsha
Hi All, Thanks for your initial feedback. We updated the KIP. Please take a look and let us know if you have any questions. https://cwiki.apache.org/confluence/display/KAFKA/KIP-405%3A+Kafka+Tiered+Storage Thanks, Harsha On Wed, Feb 6, 2019, at 10:30 AM, Harsha wrote: > Thanks

Re: [VOTE] 2.2.0 RC2

2019-03-21 Thread Harsha
+1 (non-bidning) - Download artifacts, setup 3 node cluster - Ran producer/consumer clients Thanks, Harsha On Thu, Mar 21, 2019, at 5:54 AM, Andrew Schofield wrote: > +1 (non-binding) > > - Downloaded the artifacts > - Ran Kafka Connect connectors > > Thanks, > Andrew

Re: [VOTE] KIP-427: Add AtMinIsr topic partition category (new metric & TopicCommand option)

2019-03-08 Thread Harsha
+1 (binding) -Harsha On Thu, Mar 7, 2019, at 6:48 PM, hacker win7 wrote: > +1 (non-binding) > > > On Mar 8, 2019, at 02:32, Stanislav Kozlovski > > wrote: > > > > Thanks for the KIP, Kevin! This change will be a good improvement to > > Kafka's observab

Re: [VOTE] KIP-436 Add a metric indicating start time

2019-03-08 Thread Harsha
+1 (binding) Thanks, Harsha On Fri, Mar 8, 2019, at 2:55 AM, Dongjin Lee wrote: > +1 (non binding) > > 2 bindings, 3 non-bindings until now. (Colin, Manikumar / Satish, Mickael, > Dongjin) > > On Fri, Mar 8, 2019 at 7:44 PM Mickael Maison > wrote: > > &g

Re: [DISCUSS] KIP-426: Persist Broker Id to Zookeeper

2019-03-02 Thread Harsha
the concerns about a broker.id host mapping being available in zookeeper. Broker id belongs to Kafka and not in control pane. Thanks, Harsha On Sat, Mar 2, 2019, at 3:50 AM, Eno Thereska wrote: > Hi Harsha, Li Kan, > > What Colin mentioned is what I see in practice as well (at AWS and our &

Re: [DISCUSS] KIP-426: Persist Broker Id to Zookeeper

2019-03-01 Thread Harsha
reduce the reassignment step and allow us to just copy the data and start the new node with the previous broker.id which is what the KIP is proposing. I want to understand what are your concerns in moving this mapping which already exists on disk to zookeeper? Thanks, Harsha On Fri, Mar 1, 2019

Re: [DISCUSS] KIP-427: Add AtMinIsr topic partition category (new metric & TopicCommand option)

2019-02-28 Thread Harsha
are down and we are at 2 alive replicas this is stop everything to restore the cluster to a good state. Thanks, Harsha On Wed, Feb 27, 2019, at 11:17 PM, Dong Lin wrote: > Hey Kevin, > > Thanks for the update. > > The KIP suggests that AtMinIsr is better than UnderReplica

Re: [DISCUSS] KIP-433: Provide client API version to authorizer

2019-02-27 Thread Harsha
HI Colin, Overlooked the IDEMPOTENT_WRITE ACL. This along with client.min.version should solve the cases proposed in the KIP. Can we turn this KIP into adding min.client.version config to broker and it could be part of the dynamic config . Thanks, Harsha On Wed, Feb 27, 2019, at 12

  1   2   3   >