Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

Hi Luke and Justine. There are few updates on KIP-936 https://cwiki.apache.org/confluence/display/KAFKA/KIP-936%3A+Throttle+number+of+active+PIDs to introduce throttling on PIDs per User and would love to hear your feedback in the discussion thread

Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

Thanks, Luke for the feedback 1. how do we store value in bloom filter? It's unclear from this KIP that > what we store inside bloom filter, and how we throttle them. > My understanding is, we have a map with key = kafkaPrinciple, and value = > PID for each bloom filter. > And when new PID

Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

Hi Omnia, I finally got time to check this KIP. Thanks for putting all this together. Questions: 1. how do we store value in bloom filter? It's unclear from this KIP that what we store inside bloom filter, and how we throttle them. My understanding is, we have a map with key = kafkaPrinciple,

Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

Hi Justine, Thanks for having a look > One question I have is how will we handle a scenario where potentially each new client has a new Kafka Principal? Is that simply not covered by throttling? if any new client setup a new principal they will be throttled based on the throttling for

Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

Hi Omnia, Thanks for putting this together into the KIP! I'll have a look when available. Thanks. Luke On Thu, Jun 1, 2023 at 1:50 AM Justine Olshan wrote: > Hey Omnia, > > I was doing a bit of snooping (I actually just get updates for the KIP > page) and I saw this draft was in progress. I

Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

Hey Omnia, I was doing a bit of snooping (I actually just get updates for the KIP page) and I saw this draft was in progress. I shared it with some of my colleagues as well who I previously discussed the issue with. The initial look was pretty promising to me. I appreciate the detailing of the

Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

Hi Justine and Luke, I started a KIP draft here https://cwiki.apache.org/confluence/display/KAFKA/KIP-936%3A+Throttle+number+of+active+PIDs for a proposal would appreciate it if you could provide any initial feedback before opening a broader discussion. Thanks On Wed, Feb 22, 2023 at 4:35 PM

Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

*Hi Justine, * *My initial thought of throttling the initProducerId was to get ripped off the problem at the source (which creates too many PIDs per client) and fail faster but if having this on the produce request level is easier this should be fine. I am guessing it will be the same direction

Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

Hi Luke and Justine, Are there any thoughts or updates on this? I would love to help with this as we are hitting this more frequently now. best, On Mon, Oct 31, 2022 at 6:15 PM Omnia Ibrahim wrote: > Hi Luke and Justine, > >> For (3), you said: >> > - I have some concerns about the impact of

Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

Hi Luke and Justine, > For (3), you said: > > - I have some concerns about the impact of this option on the > transactional > producers, for example, what will happen to an ongoing transaction > associated with an expired PID? Would this leave the transactions in a > "hanging" state? > > - How

Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

Oops. I realized I just replied to Omnia 臘‍♀️ Here was my response for the mailing thread: Hey Omnia, Sorry to hear this is a problem for you as well. :( > * I have some concerns about the impact of this option on the transactional producers, for example, what will happen to an ongoing

Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

Hi Omnia, Thanks for your reply. For (3), you said: > - I have some concerns about the impact of this option on the transactional producers, for example, what will happen to an ongoing transaction associated with an expired PID? Would this leave the transactions in a "hanging" state? - How will

Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

Hi Luke & Justine, Thanks for looking into this issue, we have been experiencing this because of rouge clients as well. > 3. Having a limit to the number of active producer IDs (sort of like an LRU >cache) >-> The idea here is that if we hit a misconfigured client, we will expire >the older

Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

Bump this thread to see if there are any comments/thoughts. Thanks. Luke On Mon, Sep 26, 2022 at 11:06 AM Luke Chen wrote: > Hi devs, > > As stated in the motivation section in KIP-854 > :

[DISCUSS] solutions for broker OOM caused by many producer IDs

Hi devs, As stated in the motivation section in KIP-854 : With idempotent producers becoming the default in Kafka, this means that unless otherwise specified, all new producers will be given