Guozhang, The documentation is not very clear. Marc's response for producer purgatory makes sense. I am not entirely clear on fetch purgatory. How does broker use purgatory? Is it a temporary holding area? What happens to the messages if purge interval is exceeded in case of either/both producer and consumer? Are messages dropped in this case? Thanks, Priya
On Thu, Oct 31, 2013 at 2:47 PM, Guozhang Wang <wangg...@gmail.com> wrote: > Hello Priya, > > You can find the definitions of these two configs here: > > http://kafka.apache.org/documentation.html#brokerconfigs > > Guozhang > > > On Thu, Oct 31, 2013 at 11:20 AM, Marc Labbe <mrla...@gmail.com> wrote: > > > Hi Priya > > > > my understanding is producer requests will be delayed (and put in request > > purgatory) only if your producer uses ack=-1. It will be in the purgatory > > (delayed) until all brokers have acknowledged the messages to be > > replicated. The documentation suggests to monitor the > > ProducerRequestPurgatory size metrics , but it only applies if you're > using > > ack=-1, otherwise, this value will always be 0. > > > > For consumer requests, they'll be in purgatory (delayed) until the max > > allowed time to respond has been reached, unless it has enough messages > to > > fill the buffer before that. The request will not end up in the purgatory > > if you're making a blocking request (max wait <= 0). > > > > Not sure about the configuration interval though. > > > > marc > > > > > > On Thu, Oct 31, 2013 at 12:41 PM, Priya Matpadi < > > priya.matp...@ecofactor.com > > > wrote: > > > > > Hello, > > > What is purgatory? I believe the following two properties relate to > > > consumer and producer respectively. > > > Could someone please explain the significance of these? > > > fetch.purgatory.purge.interval.requests=100 > > > producer.purgatory.purge.interval.requests=100 > > > > > > Thanks, > > > Priya > > > > > > > > > -- > -- Guozhang >
