Jay, I don't think this line will bootstrap full metadata (for all topics). it will just construct the cluster object with bootstrap host. you need to do "metadata.add(topic)" to set interest of a topic's partition metadata.
Guozhang, I personally think this is ok. it just do a few DNS lookup or TCP connection before first send. On Mon, Jan 26, 2015 at 2:07 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > Oh, yes, I guess I thought you meant that construction of the client would > block on the metadata request. > > I don't personally think that is a problem because if it fails it will > retry in the background, right? > > But actually I think this is probably violating another desirable criteria > we had talked about which was keeping the producer from bootstrapping the > full metadata for all partitions. If it is doing that during construction > time presumably the resulting metadata request is for all partitions, no? > That isn't a huge problem, but I think isn't what was intended. > > -Jay > > On Mon, Jan 26, 2015 at 1:34 PM, Guozhang Wang <wangg...@gmail.com> wrote: > > > It will set the needUpdate flag to true and hence the background Sender > > will try to talk to the bootstrap servers. > > > > Guozhang > > > > On Mon, Jan 26, 2015 at 1:12 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > > > > > Hey Guozhang, > > > > > > That line shouldn't cause any connections to Kafka to be established, > > does > > > it? All that is doing is creating the Cluster pojo using the supplied > > > addresses. The use of InetSocketAddress may cause some dns stuff to > > happen, > > > though... > > > > > > -Jay > > > > > > On Mon, Jan 26, 2015 at 10:50 AM, Guozhang Wang <wangg...@gmail.com> > > > wrote: > > > > > > > Hi all, > > > > > > > > I am not sure if we have discussed about this before, but recently I > > > > realized that we have introduced boot dependency of the kafka-server > > > > specified by the "bootstrap.servers" config in the new producer. More > > > > specifically, although in the old producer we also have a similar > > config > > > > for specifying the broker list, the producer will not try to connect > to > > > > those brokers until the first message send call is triggered; whereas > > in > > > > the new producer, it will try to talk to them in construction time > via: > > > > > > > > update(Cluster.bootstrap(addresses), time.milliseconds()); > > > > > > > > > > > > I personally am neutral to this change, as in most cases the > > > corresponding > > > > kafka server should be up and running before the producer clients are > > > > deployed, but there are still some corner cases when it is not true, > > for > > > > example some standalone deployment tests of the app embedded with > some > > > > clients, etc. So I would like to bring this up to people's attention > if > > > we > > > > have not discussed about it before: do we think this is OK to > introduce > > > > this boot dependency in the new producer? > > > > > > > > -- Guozhang > > > > > > > > > > > > > > > -- > > -- Guozhang > > >
