Hey Lin, Thanks for your interest. As Kartik mentioned in the blog post, LinkedIn Kafka team is developing a Kafka monitoring framework to help detect problem that may only be found, for example, when you run server/client with different versions for a long time using production traffic. We hope it can be used in complement to existing unit test and Ducktape test to make Kafka more robust.
But we don't know when it can be released -- it is still in its early stage of development. Thanks, Dong On Tue, Oct 13, 2015 at 10:47 PM, Lin Ma <lin...@huawei.com> wrote: > Hi, I read from website that effort has been put on developing Kafka > monitoring framework recently (detailed below). > Anyone knows when this framework can be released into kafka major version? > > Thanks. > > > https://engineering.linkedin.com/apache-kafka/how-we_re-improving-and-advancing-kafka-linkedin > Kafka Monitoring Framework > We have recently started working on this effort to have a standardized way > to monitor our Kafka clusters. The idea here is to run a set of test > applications which are producing and consuming data into kafka topics and > validating the basic guarantees (order, guaranteed delivery, data integrity > etc.) in addition to the end to end latencies for publishing and consuming > data. This effort is independent of the monitoring that we already have > where we monitor metrics emitted by the Kafka brokers. > > In addition to monitoring production systems, it will also be used for > validating a new Kafka build that we might pick from the open source trunk > to deploy to our production clusters. This will also help us ensure that > new versions of the kafka broker don't break existing older clients. >
