Hi, could you please be more specific on your use case? One of the theoretical advantages of a system like kafka is that you can decouple producers and consumers, so you don't need to to do backpressure. A different topic is how to handle lagging consumers, in that scenario you could scale up your service, etc.
Best. El mar., 19 feb. 2019 a las 15:43, Filipp Zhinkin (<filipp.zhin...@gmail.com>) escribió: > Hi! > > I'm trying to implement backpressure mechanism that asks producers to > stop doing any work when consumers are not able to process all > messages in time (producers require statistics calculated by consumers > in order to answer client requests, when consumers are lagging behind > we have to stop producers from making any responses). > > I see several ways to implement it: > - compute lag on consumer side and store it somewhere (zk, some db, etc); > - use separate service like Burrow; > - compute lag on every producer by getting commited and end offsets > for every partition via consumer API. > > Are there any downsides of the latter approach? Would is negatively > impact brokers performance? > > Thanks, > Filipp. >
