Hi ,
I am using burrow to monitor kafka Lags and I am having following queries :
1.On hitting the API /v2/kafka/local/consumer/group1/lag I am not able to
view all the topics details present in that group and getting complete:
false in the above JSON. What does this mean? Below mentioned is the json
result of the above query.
{
"error": false,
"message": "consumer group status returned",
"status": {
"cluster": "local",
"group": "group1",
"status": "OK",
"complete": false,
"partitions": [
{
"topic": "topic1",
"partition": 1,
"status": "OK",
"start": {
"offset": 144,
"timestamp": 1494566913489,
"lag": 0,
"max_offset": 144
},
"end": {
"offset": 144,
"timestamp": 1494566999000,
"lag": 0,
"max_offset": 144
}
}
],
"partition_count": 17,
"maxlag": null,
"totallag": 0
},
"request": {
"url": "/v2/kafka/local/consumer/group1/lag",
"host": "",
"cluster": "local",
"group": "group1",
"topic": ""
}
}
2. Since burrow returns JSON is there any visualization tools that can be
used to monitor the end results.
3. On hitting group describe command and burrow group lag command results
are different burrow result is somewhat delayed then the results that I am
getting while hitting group describes command on Kafka broker and I am
getting the different result.
Below mentioned is my burrow configuration:
[general]
logdir=log
logconfig=/root/go/src/github.com/linkedin/Burrow/config/logging.cfg
pidfile=burrow.pid
client-id=burrow-lagchecker
group-blacklist=^(console-consumer-|python-kafka-consumer-).*$
#group-whitelist=^(my-important-consumer).*$
[zookeeper]
hostname=<zookeeper ip>
port=2181
timeout=6
lock-path=/burrow/notifier
[kafka "local"]
broker=<Kafka Ip>
broker-port=9092
zookeeper=<zookeeper ip>
zookeeper-port=2181
zookeeper-path=/
offsets-topic=__consumer_offsets
#[storm "local"]
#zookeeper=zkhost01.example.com
#zookeeper-port=2181
#zookeeper-path=/kafka-cluster/stormconsumers
[tickers]
broker-offsets=20
[lagcheck]
intervals=10
expire-group=604800
[notify]
interval=10
[httpserver]
server=on
port=8000
; Alternatively, use listen (cannot be specified when port is)
; listen=host:port
; listen=host2:port2
[smtp]
server=mailserver.example.com
port=25
from=burrow-nore...@example.com
template=config/default-email.tmpl
[emailnotifier "b...@example.com"]
group=local,critical-consumer-group
group=local,other-consumer-group
interval=60
[notify]
interval=10
[httpnotifier]
url=http://notification.server.example.com:9000/v1/alert
interval=60
extra=app=burrow
extra=tier=STG
template-post=config/default-http-post.tmpl
template-delete=config/default-http-delete.tmpl
timeout=5
keepalive=30
So Can you please let me know what I am missing and how to fix these
issues.Any help would be appreciated.
Regards,
Abhimanyu
