Re: Client improvement discussion

2013-07-28 Thread Jay Kreps
Hey Chris, Great questions. 1. Sync vs async. Actually I am saying the client will (1) always be async but (2) always return a response. So val resp = client.send(k, v) // returns immediately resp.await() // waits for request to complete resp.offset() // waits for request to complete (i

Re: Fatal issue (was RE: 0.8 throwing exception "Failed to find leader" and high-level consumer fails to make progress_

2013-07-28 Thread Jun Rao
Ok. So, it seems that the issue is there are lots of rebalances in the consumer. How long did you set the zk session expiration time? A typical reason for many rebalances is the consumer side GC. If so, you will see Zookeeper session expirations in the consumer log (grep for Expired). Occasional re