Jenkins build is back to normal : kafka-2.1-jdk8 #92

See

Use KafkaAdminClient to get broker dynamic config ??leader.replication.throttled.rate?? and ??follower.replication.throttled.rate?? ??but returned null

Hi, everyone:I found KafkaAdminClient.describeConfigs can't return the dynamic broker config value. First I use kafka-reassign-partitions.sh to reassign partitions, and at the same time I add throttle as follows. And then use describeConfig to get the config value, but I can't get the

Build failed in Jenkins: kafka-2.0-jdk8 #209

See Changes: [jason] KAFKA-5503; Idempotent producer ignores shutdown while fetching -- [...truncated 2.66 MB...] org.apache.kafka.connect.util.ShutdownableThreadTest >

[jira] [Resolved] (KAFKA-1927) Replace requests in kafka.api with requests in org.apache.kafka.common.requests

[ https://issues.apache.org/jira/browse/KAFKA-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira resolved KAFKA-1927. - Resolution: Fixed Since the only remaining item is a refactoring, I think we can consider this

[jira] [Resolved] (KAFKA-1705) Add MR layer to Kafka

[ https://issues.apache.org/jira/browse/KAFKA-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira resolved KAFKA-1705. - Resolution: Won't Fix If I didn't do it by now... Also, MapReduce is so 2014 :) > Add MR layer

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

Thanks Gwen for the suggestion! +1 on the guidance of defining group.max.size. I guess a sample formula would be: 2 * (# of brokers * average metadata cache size * 80%) / (# of consumer groups * size of a single member metadata) if we assumed non-skewed partition assignment and pretty fair

[jira] [Resolved] (KAFKA-5503) Idempotent producer ignores shutdown while fetching ProducerId

[ https://issues.apache.org/jira/browse/KAFKA-5503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-5503. Resolution: Fixed Fix Version/s: 2.0.2 2.1.1 > Idempotent

[jira] [Created] (KAFKA-7783) Improve testing of idempotent/transactional producer shutdown

Jason Gustafson created KAFKA-7783: -- Summary: Improve testing of idempotent/transactional producer shutdown Key: KAFKA-7783 URL: https://issues.apache.org/jira/browse/KAFKA-7783 Project: Kafka

Re: [DISCUSS] KIP-392: Allow consumers to fetch from the closest replica

Hey Jun, Sorry for the late reply. I have been giving your comments some thought. Replies below: 1. The section on handling FETCH_OFFSET_TOO_LARGE error says "Use the > OffsetForLeaderEpoch API to verify the current position with the leader". > The OffsetForLeaderEpoch request returns log end

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

Sorry for joining the fun late, but I think the problem we are solving evolved a bit in the thread, and I'd like to have better understanding of the problem before voting :) Both KIP and discussion assert that large groups are a problem, but they are kinda inconsistent regarding why they are a

Re: [VOTE] KIP-345: Introduce static membership protocol to reduce consumer rebalances

Thanks Jason for the comment! I answered it on the discuss thread. Folks, could we continue the vote for this KIP? This is a very critical improvement for our streaming system stability and we need to get things rolling right at the start of 2019. Thank you for your time! Boyang

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

Hi Colin, Setting max.partition.fetch.bytes was discussed in the ticket. It's not as desirable if the message size is highly variable. Also this decrease the efficiency of network communication. In the case you mentioned below where a consumer can get messages from A, B, C and D but the

Jenkins build is back to normal : kafka-trunk-jdk8 #3276

See

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

Yep, that is correct! Get Outlook for iOS From: Mayuresh Gharat Sent: Wednesday, January 2, 2019 8:30 AM To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id Hi Boyang,

Re: [DISCUSS] : KIP-410: Add metric for request handler thread pool utilization by request type

Hi Stanislav, Thanks a lot for the feedback and sorry for the late reply. I agree that we can have the name "RequestHandlerPoolUsagePercent" instead of "RequestHandlerPoolUsageRate". The main reason, why I named it rate was because of the discussion we had on the corresponding jira. But now, that

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

Hi Boyang, Thanks a lot for the reply. I think, I get what you are saying. IIUC, once the group starts moving to dynamic membership from static membership, the group will have a list of new member.id's and the earlier mapping of group.instance.id -> member.id will be outdated and become inactive

Re: [DISCUSS] KIP-396: Add Commit/List Offsets Operations to AdminClient

It's now been a few weeks without any feedback. If I don't see anything new, I'll start a vote early next week. Thanks On Tue, Dec 11, 2018 at 6:00 PM Mickael Maison wrote: > > Thanks for the feedback Edo. > > 1) While the initial name was matching the Protocol, I agree it was > not great for

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

Jan I have been thinking a lot about the history of the discussion and your original proposal, and why you believe it is a better solution. The biggest problem with your original proposed design is that it seems to me to be non-deterministic. It is subject to race conditions that are dependent

Re: Permission to Create Content

What is your wiki ID? On 12/30/18 11:36 PM, Jamie wrote: > Hi All,  > Could you please grant me permission to create content so that I can create a > KIP?  > Many Thanks,  > Jamie > signature.asc Description: OpenPGP digital signature

[jira] [Created] (KAFKA-7782) cleanup policy for offsets topic should be configurable

Nir Barel created KAFKA-7782: Summary: cleanup policy for offsets topic should be configurable Key: KAFKA-7782 URL: https://issues.apache.org/jira/browse/KAFKA-7782 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-7781) Add validation check for Topic retention.ms property

Kamal Chandraprakash created KAFKA-7781: --- Summary: Add validation check for Topic retention.ms property Key: KAFKA-7781 URL: https://issues.apache.org/jira/browse/KAFKA-7781 Project: Kafka

RE: [EXTERNAL] [VOTE] KIP-382 MirrorMaker 2.0

+1 (non-binding) Jonathan Skrzypek -Original Message- From: Mickael Maison Sent: 02 January 2019 11:10 To: dev Subject: Re: [EXTERNAL] [VOTE] KIP-382 MirrorMaker 2.0 +1 (non-binding) Thanks Ryanne On Wed, Jan 2, 2019 at 4:47 AM McCaig, Rhys wrote: > > +1 (non-binding). Fantastic

[jira] [Created] (KAFKA-7780) Kafka client should validate topic-level configs and update only unique values

Manohar Vanam created KAFKA-7780: Summary: Kafka client should validate topic-level configs and update only unique values Key: KAFKA-7780 URL: https://issues.apache.org/jira/browse/KAFKA-7780

Re: [EXTERNAL] [VOTE] KIP-382 MirrorMaker 2.0

+1 (non-binding) Thanks Ryanne On Wed, Jan 2, 2019 at 4:47 AM McCaig, Rhys wrote: > > +1 (non-binding). Fantastic work on the KIP Ryanne. > > > On Dec 25, 2018, at 9:10 AM, Stephane Maarek > > wrote: > > > > +1 ! Great stuff > > > > Stephane > > > > On Mon., 24 Dec. 2018, 12:07 pm Edoardo

[jira] [Created] (KAFKA-7779) Avoid unnecessary loop iteration in leastLoadedNode

huxihx created KAFKA-7779: - Summary: Avoid unnecessary loop iteration in leastLoadedNode Key: KAFKA-7779 URL: https://issues.apache.org/jira/browse/KAFKA-7779 Project: Kafka Issue Type: Improvement