Pulling data from remote server

2015-12-29 Thread Bin Wang
Hi there, I have several different production servers running in different cloud providers, like AWS, Rackspace, ..etc. And I have one central big data environment inside company's network. I need to build a NRT(near real time) monitoring system which I need to stream and archive all the logs from

Kafka consumer hangs after reconnect to zookeeper

2015-11-10 Thread Bin Wang
I'm using the Java high level consumer API (my code is written in Scala). And when the connection of zookeeper is timeout and reconnected, the stream hangs and cannot get the new data. Can anybody help? Thanks! The version of Kafka is 0.8.2 that comes with cloudera. The code is written like the e