Hi, I and my team are beginning with kafka so I/we may not have read everything or understood everything yet.
The context is we’d like to bootstrap kafka in our own codebase, and first step is to have a deployable walking skeleton that the team can grow along our experience and needs. We are starting with kafka 0.9.0.0, and surprisingly there’s no status command for the broker, so we decided to do a comprehensive check but we are still a bit confused on how to write a healthcheck that is not fragile. Currently we would like to produce a message (e.g. a timestamp) on some healthcheck topic, and expect a consumer to read this message, a grep would check that the message has the same value. However we are facing some behaviour that we don’t yet quite understand. # first emit the message timestamp=$(date -u "%Y-%m-%dT%H:%M%SZ)"echo $timestamp | bin/kafka-console-producer.sh --broker-list localhost:9092 --topic healthcheck # try to consume the same message bin/kafka-console-consumer.sh --new-consumer --bootstrap-server localhost:9092 --topic healthcheck --max-messages 1 --from-beginning | grep $timestamp [ 0 == $? ] && echo "kafka onfline" || echo "kafka offline" In this case the consumer does not always *find* the matching message. We have tried different options there ; with or without --max-messages, --from-beginning, --timeout-ms, it does’nt behave as we thought by reading the documentation. Sometime the message is read some time it isn’t, but we don’t get why. ------------------------------ To be noted, I don’t understand how it is possible to read the latest message, > bin/kafka-topics.sh --zookeeper localhost:2181 --create --topic yat > --partitions 1 --replication-factor 1 Created topic "yat". > echo bob | bin/kafka-console-producer.sh --topic yat --broker-list > localhost:9092 > echo charles | bin/kafka-console-producer.sh --topic yat --broker-list > localhost:9092 > echo yohan | bin/kafka-console-producer.sh --topic yat --broker-list > localhost:9092 > echo carlos | bin/kafka-console-producer.sh --topic yat --broker-list > localhost:9092 # Running the simple command just outputs nothing, and I need to kill the process with ctrl-c, note the messages count is off by 1, or I'm missing something > bin/kafka-console-consumer.sh --new-consumer --bootstrap-server > localhost:9092 --topic yat ^CProcessed a total of 1 messages # Same with `--from-beginning`, the pocess don't finish up, and prints everything since the beginning (that is to be expected). bin/kafka-console-consumer.sh --new-consumer --bootstrap-server localhost:9092 --topic yat --from-beginning bob charles yohan carlos carlos yohan brice ^CProcessed a total of 8 messages # With `--from-beginning` and `--max-messages` the consumer stops at the very first message [root@3af0ccad3182 kafka]# bin/kafka-console-consumer.sh --new-consumer --bootstrap-server localhost:9092 --topic yat --from-beginning --max-messages 1 bob Processed a total of 1 messages # With `--max-messages` the latest message doesn't appear either and I have to kill the consumer too > bin/kafka-console-consumer.sh --new-consumer --bootstrap-server > localhost:9092 --topic yat --max-messages 1 ^CProcessed a total of 1 messages Also note the same happen with the old consumer with --zookeeper option. Is there some limitation with the console consumer regarding offsets ? Is there something I’m missing. Should we write our own healthcheck ? Thanks in advance for any feedback, pointers, etc. Cheers, — Brice