Hi , I am using burrow to monitor kafka Lags and I am having following queries :
1.On hitting the API /v2/kafka/local/consumer/group1/lag I am not able to view all the topics details present in that group and getting complete: false in the above JSON. What does this mean? Below mentioned is the json result of the above query. { "error": false, "message": "consumer group status returned", "status": { "cluster": "local", "group": "group1", "status": "OK", "complete": false, "partitions": [ { "topic": "topic1", "partition": 1, "status": "OK", "start": { "offset": 144, "timestamp": 1494566913489, "lag": 0, "max_offset": 144 }, "end": { "offset": 144, "timestamp": 1494566999000, "lag": 0, "max_offset": 144 } } ], "partition_count": 17, "maxlag": null, "totallag": 0 }, "request": { "url": "/v2/kafka/local/consumer/group1/lag", "host": "", "cluster": "local", "group": "group1", "topic": "" } } 2. Since burrow returns JSON is there any visualization tools that can be used to monitor the end results. 3. On hitting group describe command and burrow group lag command results are different burrow result is somewhat delayed then the results that I am getting while hitting group describes command on Kafka broker and I am getting the different result. Below mentioned is my burrow configuration: [general] logdir=log logconfig=/root/go/src/github.com/linkedin/Burrow/config/logging.cfg pidfile=burrow.pid client-id=burrow-lagchecker group-blacklist=^(console-consumer-|python-kafka-consumer-).*$ #group-whitelist=^(my-important-consumer).*$ [zookeeper] hostname=<zookeeper ip> port=2181 timeout=6 lock-path=/burrow/notifier [kafka "local"] broker=<Kafka Ip> broker-port=9092 zookeeper=<zookeeper ip> zookeeper-port=2181 zookeeper-path=/ offsets-topic=__consumer_offsets #[storm "local"] #zookeeper=zkhost01.example.com #zookeeper-port=2181 #zookeeper-path=/kafka-cluster/stormconsumers [tickers] broker-offsets=20 [lagcheck] intervals=10 expire-group=604800 [notify] interval=10 [httpserver] server=on port=8000 ; Alternatively, use listen (cannot be specified when port is) ; listen=host:port ; listen=host2:port2 [smtp] server=mailserver.example.com port=25 from=burrow-nore...@example.com template=config/default-email.tmpl [emailnotifier "b...@example.com"] group=local,critical-consumer-group group=local,other-consumer-group interval=60 [notify] interval=10 [httpnotifier] url=http://notification.server.example.com:9000/v1/alert interval=60 extra=app=burrow extra=tier=STG template-post=config/default-http-post.tmpl template-delete=config/default-http-delete.tmpl timeout=5 keepalive=30 So Can you please let me know what I am missing and how to fix these issues.Any help would be appreciated. Regards, Abhimanyu