I have not measured that value exactly, but I just thought the case when we use db; the frequent connection/disconnection causes overhead so that connection pool is generally used to solve the problem.
Does the kafka bring less overhead comparing with that case as much as what the connection pool doesn't need to be considered? Best regards bgkim 2016-03-22 11:40 GMT+09:00 Christian Posta <christian.po...@gmail.com>: > Can you explain more? Have you measured the overhead of opening the > connections? > > If I'm not mistaken, Kafka manages the connections under the covers to each > of the brokers that have topics (leader partitions) from which you're > consuming. The connection(s) to each partition leader will stay around as > long as you're consuming from them (ie, calling poll()) method. If you > aren't polling the broker for messages within the session timeout period, > the heartbeat mechanism will actually kick the consumer out and do a > rebalance. I'd be curious to understand more about your usecase to see > how/when you're experiencing overhead of the connection handshaking. > > On Mon, Mar 21, 2016 at 7:33 PM, BYEONG-GI KIM <bg...@bluedigm.com> wrote: > > > Hello. I have a question that the latest kafka, 0.9.0.1, provides any > APIs > > for managing connection pool of kafka on both consumer and producer > sides. > > > > I think the overhead which happens while establishing connection from > > consumer/producer to kafka broker(s) seems a little heavy. > > > > Thanks in advance! > > > > Best regards > > > > bgkim > > > > > > -- > *Christian Posta* > twitter: @christianposta > http://www.christianposta.com/blog > http://fabric8.io >
