Weide, This FetchSize is the fetch.message.max.bytes config in consumer configs:
http://kafka.apache.org/documentation.html#consumerconfigs Guozhang On Fri, Jul 11, 2014 at 6:18 PM, Weide Zhang <weo...@gmail.com> wrote: > Hi Guozhang, > > I have a couple of more questions about using the simpleConsumer API call. > > It seems when constructing the fetchRequest, it need a parameter called > FetchSize. Do you know what's the purpose of this parameter ? Does that > mean the call to Kafka won't be returned until the response message set > size is at least fetchSize ? I assuming the fetchRequest call to Kafka is a > blocking call. > > Thanks a lot, > > Weide > > > On Thu, Jul 10, 2014 at 3:50 PM, Guozhang Wang <wangg...@gmail.com> wrote: > > > Yes it can be shared. > > > > Guozhang > > > > > > On Thu, Jul 10, 2014 at 11:12 AM, Weide Zhang <weo...@gmail.com> wrote: > > > > > Hi Guozhang, > > > > > > Just a follow up question. Is one simpleconsumer instance can be shared > > by > > > multiple threads and fetch data from multiple topics and commit > offsets ? > > > > > > It seems all the implementation are synchronized which means it can be > > > shared between multiple threads ? Is my understanding correct ? > > > > > > Thanks, > > > > > > Weide > > > > > > > > > On Tue, Jul 1, 2014 at 5:36 PM, Guozhang Wang <wangg...@gmail.com> > > wrote: > > > > > > > Hi Weide, > > > > > > > > 1. The old consumer is still depending on the ZK no matter if you use > > ZK > > > > for offsets or not, since it depends on ZK for group manage anyways. > In > > > the > > > > new consumer (0.9) we have completely the ZK dependance. > > > > > > > > 2. Yes, it queries the broker for leader info. > > > > > > > > 3. You can use a single consumer for multiple topics, but a single > > > consumer > > > > can only talk to a single broker; if your topic spans across multiple > > > > brokers you would also need multiple simple consumers. > > > > > > > > Guozhang > > > > > > > > > > > > On Tue, Jul 1, 2014 at 5:14 PM, Weide Zhang <weo...@gmail.com> > wrote: > > > > > > > > > Hi , > > > > > > > > > > Just want to ask some basic question about kafka simple consumer. > > > > > > > > > > 1. if I'm using simple consumer and doesn't really depend on > > zookeeper > > > to > > > > > manage partition offset. (application manage offset themselves). > Will > > > > that > > > > > remove the zookeeper dependency for consumer ? > > > > > 2. if zookeeper dies, will simple consumer still able to get > > partition > > > > > leadership information from broker itself ? Or it indirectly uses > > > > zookeeper > > > > > to find out the partition leadership for particular topics ? > > > > > 3. does simpleconsumer api have topicfilter logic similar to high > > level > > > > > consumer provides ? Or I have to create multiple simpleconsumer for > > > > > multiple topics themselves explicitly ? > > > > > > > > > > Thanks a lot, > > > > > > > > > > Weide > > > > > > > > > > > > > > > > > > > > > -- > > > > -- Guozhang > > > > > > > > > > > > > > > -- > > -- Guozhang > > > -- -- Guozhang
