What I did for my project is I have a thread send metadata request to a random broker and monitor the metadata change periodically. The good thing is, to my knowledge, any broker in the cluster know the metadata for all the topics served in this cluster. Another options is you can always query zookeeper to pull the latest information about the cluster
On Wed, Nov 27, 2013 at 7:57 AM, Jun Rao <jun...@gmail.com> wrote: > Currently, regular consumers can only fetch from the leader replica. > Otherwise, they will get an error in response. We allow some special > consumers to read from follower replicas, but this is really for testing. > > Are you thinking of load balancing? Currently, we do load balancing across > partitions. We assume that there are many partitions in a cluster and try > to spread the leaders evenly among brokers. That way, even for a particular > partition, all clients have to talk to a single broker, the overall > workload is still balanced. > > Thanks, > > Jun > > > On Wed, Nov 27, 2013 at 2:13 AM, Simon Cooper < > simon.coo...@featurespace.co.uk> wrote: > > > Hi, > > > > I've been looking at the SimpleConsumer example, and that I've noticed > > that it always reads from the leader, and reacts to leader changes by > > reconnecting to the new leader. Is it possible to read from a replica in > > ISR that's not the leader? If so, how does the consumer get notified the > > replica it's reading from is no longer in ISR? > > > > Thanks, > > SimonC > > >
