You should check out Kafka Streams Metrics (for upcoming 0.10.2 they are even more detailed).
There is not a lot of documentation for 0.10.0 or 0.10.1, but it work the same way as for consumer/producer metric that are documented. -Matthias On 1/24/17 10:38 PM, Sachin Mittal wrote: > Hi All, > I am running a kafka streaming application with a simple pipeline of: > source topic -> group -> aggregate by key -> for each > save to a sink. > > I source topic gets message at rate of 5000 - 10000 messages per second. > During peak load we see the delay reaching to 3 million messages. > > So I need to figure out where delay might be happening. > > 1. Is there any mechanism in kafka streams to log time spent within each > pipeline stage. > > 2. Also if I want to turn on custom logging to log some times how can I do > the same. > > I have a log4j.properties and I am packaging it inside a jar which has the > main class. > I place that jar in libs folder of kafka installation. > > However I see no logs generated under logs folder. > > So where are we suppose to add the log4j.properties. > > Thanks > Sachin >
signature.asc
Description: OpenPGP digital signature
