Build failed in Jenkins: kafka-trunk-jdk8 #2802

See Changes: [jason] KAFKA-6884; Consumer group command should use new admin client (#5032) [github] MINOR: update release.py (#5374) [github] KAFKA-4041: Update ZooKeeper to 3.4.13 (#5376) [wangguoz]

Re: KIP-327: Add describe all topics API to AdminClient

Thanks, Manikumar. best, Colin On Tue, Jul 17, 2018, at 19:44, Manikumar wrote: > Closing this KIP in favor of adding filtering support to the Metadata API > and KIP-142. Will open a new KIP when ready. > Thanks for your reviews. > > On Mon, Jul 16, 2018 at 8:38 AM Colin McCabe wrote: > > >

Build failed in Jenkins: kafka-2.0-jdk8 #73

See Changes: [matthias] MINOR: update release.py (#5374) [ismael] KAFKA-4041: Update ZooKeeper to 3.4.13 (#5376) [ismael] MINOR: Update Jetty to 9.4.11 (#5377) --

Re: KIP-327: Add describe all topics API to AdminClient

Closing this KIP in favor of adding filtering support to the Metadata API and KIP-142. Will open a new KIP when ready. Thanks for your reviews. On Mon, Jul 16, 2018 at 8:38 AM Colin McCabe wrote: > Thanks, Manikumar. I've been meaning to bring up KIP-142 again. It would > definitely be a nice

Re: [VOTE] KIP-338 Support to exclude the internal topics in kafka-topics.sh command

+1 (binding) Thank you for the improvement! On Mon, Jul 16, 2018 at 4:10 AM, Chia-Ping Tsai wrote: > hi folks, > > The discussion[1] of KIP-338[2] did not get any objection for last 6 days > so it is time to start the voting thread. > > Thanks for your time! > > [1]

Re: [VOTE] KIP-338 Support to exclude the internal topics in kafka-topics.sh command

hi Jason > I think eventually we'd want --exclude-internal-topics to be the default, (just a double check) the option name used in kip-338 is "--exclude-internal" rather than "--exclude-internal-topics". Do you prefer "--exclude-internal-topics"? Cheers, Chia-Ping On 2018/07/17 19:14:09,

[jira] [Created] (KAFKA-7177) Update 2.0 documentation to reflect changed quota behaviors by KIP-219

Jon Lee created KAFKA-7177: -- Summary: Update 2.0 documentation to reflect changed quota behaviors by KIP-219 Key: KAFKA-7177 URL: https://issues.apache.org/jira/browse/KAFKA-7177 Project: Kafka

[jira] [Created] (KAFKA-7176) State store metrics for migrated tasks are not removed

Sam Lendle created KAFKA-7176: - Summary: State store metrics for migrated tasks are not removed Key: KAFKA-7176 URL: https://issues.apache.org/jira/browse/KAFKA-7176 Project: Kafka Issue Type:

[jira] [Resolved] (KAFKA-5098) KafkaProducer.send() blocks and generates TimeoutException if topic name has illegal char

[ https://issues.apache.org/jira/browse/KAFKA-5098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy resolved KAFKA-5098. --- Resolution: Fixed > KafkaProducer.send() blocks and generates TimeoutException if topic name has >

[jira] [Created] (KAFKA-7175) Make version checking logic more flexible in streams_upgrade_test.py

Ted Yu created KAFKA-7175: - Summary: Make version checking logic more flexible in streams_upgrade_test.py Key: KAFKA-7175 URL: https://issues.apache.org/jira/browse/KAFKA-7175 Project: Kafka Issue

[jira] [Created] (KAFKA-7174) Improve version probing of subscription info

Ted Yu created KAFKA-7174: - Summary: Improve version probing of subscription info Key: KAFKA-7174 URL: https://issues.apache.org/jira/browse/KAFKA-7174 Project: Kafka Issue Type: Improvement

[DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

Hey group, I just created a new KIP about adding customizable SASL extensions to the OAuthBearer authentication mechanism. More details in the proposal KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-342%3A+Add+support+for+Custom+SASL+extensions+in+OAuthBearer+authentication JIRA:

Build failed in Jenkins: kafka-trunk-jdk8 #2801

See Changes: [github] MINOR: Update Jetty to 9.4.11 (#5377) [mjsax] MINOR: make the constructor of InMemoryKeyValueStore public so that it -- [...truncated 881.86 KB...]

Re: [DISCUSS] KIP-341: Update Sticky Assignor's User Data Protocol

Hi Jason, Thanks for the feedback. That sounds good to me. I'll update the KIP accordingly and move the local generation option to "Rejected Alternatives". I'll send a note when the revised KIP is ready. --Vahid From: Jason Gustafson To: dev Date: 07/17/2018 02:22 PM Subject:

Re: [DISCUSS] KIP-341: Update Sticky Assignor's User Data Protocol

Hey Vahid, I'm with Mike that it seems simpler to just use the consumer group generation. Even if you can figure out how to reason about the local generation, it still seems confusing to have something called "generation" which is not the consumer group generation. There doesn't seem to be much

[jira] [Resolved] (KAFKA-7103) Use bulkloading for RocksDBSegmentedBytesStore during init

[ https://issues.apache.org/jira/browse/KAFKA-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-7103. -- Resolution: Fixed Fix Version/s: 2.1.0 > Use bulkloading for RocksDBSegmentedBytesStore

[jira] [Resolved] (KAFKA-7170) NPE in ConsumerGroupCommand when describe consumer group

[ https://issues.apache.org/jira/browse/KAFKA-7170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7170. Resolution: Duplicate Resolving as duplicate. Please continue discussion in KAFKA-7044 if

Re: [VOTE] KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file

+1 On Tue, Jul 17, 2018 at 1:40 PM Jason Gustafson wrote: > +1. This is useful (though the naming inconsistencies in the tools are > vexing, as always). > > -Jason > > On Tue, Jul 17, 2018 at 12:24 PM, Dong Lin wrote: > > > Hi all, > > > > It seems that there is no further concern with the

Re: [VOTE] KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file

+1. This is useful (though the naming inconsistencies in the tools are vexing, as always). -Jason On Tue, Jul 17, 2018 at 12:24 PM, Dong Lin wrote: > Hi all, > > It seems that there is no further concern with the KIP-304. I guess it is > time to start the voting thread. > > The KIP can be

[jira] [Resolved] (KAFKA-7173) Update BrokerApiVersionsCommand to use new AdminClient

[ https://issues.apache.org/jira/browse/KAFKA-7173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-7173. Resolution: Duplicate Duplicate of KAFKA-5723. > Update BrokerApiVersionsCommand to use new

Re: [VOTE] 1.1.1 RC3

+1 - Did basic quickstarts on the 2.11 and 2.12 artifacts - Checked upgrade notes and documentation Thanks for managing the release, Dong! -Jason On Thu, Jul 12, 2018 at 2:36 PM, Harsha wrote: > +1. > 1. Ran unit tests > 2. Ran 3 node cluster to run few tests. > > Thanks, > Harsha > > On

[VOTE] KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file

Hi all, It seems that there is no further concern with the KIP-304. I guess it is time to start the voting thread. The KIP can be found at https://cwiki.apache.org/confluence/display/KAFKA/KIP-340%3A+Allow+kafka-reassign-partitions.sh+and+kafka-log-dirs.sh+to+take+admin+client+property+file .

Re: [VOTE] KIP-338 Support to exclude the internal topics in kafka-topics.sh command

Hi, thanks for the KIP. +1 from me. I think eventually we'd want --exclude-internal-topics to be the default, but it seems reasonable to keep the current behavior for compatibility. -Jason On Mon, Jul 16, 2018 at 2:23 PM, Dong Lin wrote: > +1 > > Thanks! > Dong > > On Mon, Jul 16, 2018 at

[jira] [Resolved] (KAFKA-6717) TopicPartition Assined twice to a consumer group for 2 consumer instances

[ https://issues.apache.org/jira/browse/KAFKA-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian resolved KAFKA-6717. Resolution: Duplicate Marking it as duplicate to keep all the discussion in the other

[jira] [Created] (KAFKA-7173) Update BrokerApiVersionsCommand to use new AdminClient

Jason Gustafson created KAFKA-7173: -- Summary: Update BrokerApiVersionsCommand to use new AdminClient Key: KAFKA-7173 URL: https://issues.apache.org/jira/browse/KAFKA-7173 Project: Kafka

Re: [DISCUSS] KIP-291: Have separate queues for control requests and data requests

@Becket 1. Thanks for the comment. You are right that normally there should be just one controller request because of muting, and I had NOT intended to say there would be many enqueued controller requests. I went through the KIP again, and I'm not sure which part conveys that info. I'd be happy to

Re: Restarting KIP-213 discussion

Hello Adam, Welcome back! Please feel free to update the the existing KIP. To bookkeep Jan's changes, I'd suggest adding a new section on the wiki. Guozhang On Tue, Jul 17, 2018 at 10:04 AM, Adam Bellemare wrote: > Hey all > > I recently took a crack at implementing my own version of

Re: [DISCUSS] KIP-291: Have separate queues for control requests and data requests

Hi Lucas, Thanks for the KIP. I am trying to understand why you think "The memory consumption can rise given the total number of queued requests can go up to 2x" in the impact section. Normally the requests from controller to a Broker are not high volume, right ? Thanks, Mayuresh On Tue, Jul

Re: [VOTE] KIP-291: Have separate queues for control requests and data requests

+1 on the KIP. (I'm not sure we actually necessary to introduce the condition variables for the concern that Jun raised, but it's an implementation detail that we can defer to a discussion in the PR.) On Sat, Jul 14, 2018 at 10:45 PM, Lucas Wang wrote: > Hi Jun, > > I agree by using the

Re: [DISCUSS] KIP-291: Have separate queues for control requests and data requests

Hey Becket - good point. Lucas and I were talking about this offline last week. It is true that there is only one request in flight for processing. However, there may be more during a controller failover but it should not be very high - basically the maximum number of controller failures that can

Re: [DISCUSS] KIP-320: Allow fetchers to detect and handle log truncation

Hi Jason, I also like your proposal and agree that KafkaConsumer#seekToCommitted() is more intuitive as a way to initialize both consumer's position and its fetch state. My understanding that KafkaConsumer#seekToCommitted() is purely for clients who store their offsets externally, right? And

Re: Contribution to Kafka

Thanks a lot. On Tue, 17 Jul 2018 at 22:25, Guozhang Wang wrote: > It's done. Cheers. > > > Guozhang > > On Tue, Jul 17, 2018 at 9:45 AM, Sayan Biswas > wrote: > > > Hi, > > I've signed up. My userid is: *Bitswazsky* > > > > On Tue, 17 Jul 2018 at 22:08, Guozhang Wang wrote: > > > > > You

Restarting KIP-213 discussion

Hey all I recently took a crack at implementing my own version of KIP-213 based on code written by the previous committee, Jan. I currently have it dev ready, but I want to restart some of the discussion based on some design decisions that I made, and I need some feedback on a few particular

Re: Contribution to Kafka

It's done. Cheers. Guozhang On Tue, Jul 17, 2018 at 9:45 AM, Sayan Biswas wrote: > Hi, > I've signed up. My userid is: *Bitswazsky* > > On Tue, 17 Jul 2018 at 22:08, Guozhang Wang wrote: > > > You just need to go do Apache JIRA and sign up with an id. > > > >

Re: Contribution to Kafka

Hi, I've signed up. My userid is: *Bitswazsky* On Tue, 17 Jul 2018 at 22:08, Guozhang Wang wrote: > You just need to go do Apache JIRA and sign up with an id. > > https://issues.apache.org/jira/secure/Dashboard.jspa > > > On Tue, Jul 17, 2018 at 9:22 AM, Sayan Biswas > wrote: > > > Hi, > > I

Re: Contribution to Kafka

You just need to go do Apache JIRA and sign up with an id. https://issues.apache.org/jira/secure/Dashboard.jspa On Tue, Jul 17, 2018 at 9:22 AM, Sayan Biswas wrote: > Hi, > I don't have an id yet. I had only subscribed to the dev mailing lists. Can > you please tell me how to get one? > > On

Re: Contribution to Kafka

Hi, I don't have an id yet. I had only subscribed to the dev mailing lists. Can you please tell me how to get one? On Tue, 17 Jul 2018 at 21:41, Guozhang Wang wrote: > Hello Sayan, > > What's your apache id? > > > Guozhang > > On Tue, Jul 17, 2018 at 8:14 AM, Sayan Biswas > wrote: > > > Hi

Re: Kafka Contributors

Hello Austin, What's your Apache id? Guozhang On Mon, Jul 16, 2018 at 4:23 PM, Austin Nobis wrote: > Hi, > > Can I be added to the list of contributors for Apache Kafka so I can assign > myself issues? > > Thanks, > Austin > -- -- Guozhang

Re: Contribution to Kafka

Hello Sayan, What's your apache id? Guozhang On Tue, Jul 17, 2018 at 8:14 AM, Sayan Biswas wrote: > Hi all, > > Can I be added to the Kafka contributor's list? > > Regards, > Sayan > > -- > Don't kill the dream - execute it. > -- -- Guozhang

Contribution to Kafka

Hi all, Can I be added to the Kafka contributor's list? Regards, Sayan -- Don't kill the dream - execute it.

[jira] [Created] (KAFKA-7172) Prometheus /metrics http endpoint for metrics monitoring integration

Hari Sekhon created KAFKA-7172: -- Summary: Prometheus /metrics http endpoint for metrics monitoring integration Key: KAFKA-7172 URL: https://issues.apache.org/jira/browse/KAFKA-7172 Project: Kafka

Kafka Contributors

Hi, Can I be added to the list of contributors for Apache Kafka so I can assign myself issues? Thanks, Austin

Re: [DISCUSS] KIP-291: Have separate queues for control requests and data requests

Thanks for the KIP, Lucas. Separating the control plane from the data plane makes a lot of sense. In the KIP you mentioned that the controller request queue may have many requests in it. Will this be a common case? The controller requests still goes through the SocketServer. The SocketServer will

[jira] [Created] (KAFKA-7171) KafkaPerformanceProducer crashes with same transaction id.

kaushik srinivas created KAFKA-7171: --- Summary: KafkaPerformanceProducer crashes with same transaction id. Key: KAFKA-7171 URL: https://issues.apache.org/jira/browse/KAFKA-7171 Project: Kafka

[jira] [Resolved] (KAFKA-7163) Topic is not marked for deletion when delete.topic.enable is set false

[ https://issues.apache.org/jira/browse/KAFKA-7163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaobo Liu resolved KAFKA-7163. --- Resolution: Not A Problem > Topic is not marked for deletion when delete.topic.enable is set false >

Re: [DISCUSSION] KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer

Hi Viktor, The approach where all the methods have a default implementation and the user chooses to override one of them seems the most appealing to me given the current state. It doesn't seem like we give up much in that case, what do you think? Ismael On Tue, Jul 10, 2018 at 7:15 AM Viktor

Re: [DISCUSS] KIP-333 Consider a faster form of rebalancing

Hi Becket, Thanks for reviewing this KIP. :) I probably did not explicitly state what we were trying to avoid by introducing this mode. As mentioned in the KIP, there is a offset lag which could result after a crash. Our main goal is to avoid this lag (i.e. the latency in terms of time that

Re: [DISCUSSION] KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer

here is my two cents. In order to keep the BC, SC, and correct deprecation cycle(deprecate Serializer/Deserializer in 2.0), the solution of this issue is to introduce a new class to replace the Serializer and Deserializer. The new class should be a subclass of Serializer/Deserializer and it

Jenkins build is back to normal : kafka-trunk-jdk8 #2800

See

Re: [DISCUSS] KIP-333 Consider a faster form of rebalancing

Hi Richard, Thanks for the KIP. I am a little confused on what is proposed. The KIP suggests that after recovery from a consumer crash, there will be two consumers consuming from the same partition. One consumes starting from the log end offset at the point of recovery, and another consumes

Build failed in Jenkins: kafka-0.10.2-jdk7 #227

See Changes: [github] MINOR: add release script (#5328) -- [...truncated 286.58 KB...] kafka.server.KafkaConfigTest > testInvalidAdvertisedListenersProtocol PASSED