Re: Kafka 0.8.2.2 is using high CPU

2016-03-18 Thread li jinyu
I want to add more information. we've already setup a completely new Kafka cluster with 0.8.2.2, and the old cluster is still there, but no consumer or producer, it's really wired that the CPU usage is still high. also changed fetch.purgatory.purge.interval.requests and producer.purgatory.purge.i

Question regarding compression of topics in Kafka

2016-03-18 Thread R P
Hello All, Does kafka support compressing storage logs stored in log dir? What does compression.type=(gzip/snappy) in server.properties do? Based on documents I am assuming that it will compress the logs on local file system. I ran a quick experiment and found that my logs stored on local disk

Kafka 0.8.2.2 is using high CPU

2016-03-18 Thread li jinyu
hi all, we encountered problem with 0.8.1.1, as a lot of BadVersion error in log, we believe it's caused by issue https://issues.apache.org/jira/browse/KAFKA-1382, so we upgraded kafka to 0.8.2.2. but we found that kafka is using high CPU. following is the result of 'top' command: PID USER

Re: Larger Size Error Message

2016-03-18 Thread Fang Wong
Thanks Guozhang: I put server.log in the command line, got the the following error: -bash-4.1$ ./kafka-run-class.sh kafka.tools.DumpLogSegments --files /home/kafka/logs/server.log Dumping /home/sfdc/logs/liveAgent/kafka/logs/server.log Exception in thread "main" java.lang.NumberFormatException: F

Best practice to launch multiple Kafka Consumers

2016-03-18 Thread I PVP
Hi everyone, What is the best practice to launch multiple Kafka Consumers that will be running forever ? So far I have a single class that is called from a command line (.sh) launcher executed at the OS boot time, and that class main(String[] args) method will be starting many Consumers. Sho

Subscribe to this mailing list.

2016-03-18 Thread Amit kumar(Tech-Mantri)
Please add me to this mailing list. Thanks Amit Kumar

Re: Larger Size Error Message

2016-03-18 Thread Fang Wong
Thanks Guozhang! We are in the process of upgrading to 0.9.0.0. We will look into using ACLs. Is there a way to see what is the request in the kafka server, the request for my case is byte[]? Is there a way to turn on kafka logging to see the request on the kafka server side? Thanks, Fang On Wed