Hi Gonzalo, Thanks for your response.
No, the Kafka sink connection is not the same all times.I have observed the connections closing and reconnecting. Sent from my HTC ----- Reply message ----- From: "Gonzalo Herreros" <[email protected]> To: "user" <[email protected]> Subject: Possibility of persisting the connection Date: Tue, Nov 17, 2015 11:08 PM For the sink, I would be surprised if the connection to kafka is not the same all the time. For the http source you could create a custom source where you keep a long lived http connection and have some way of detecting where a batch of events is sent (e.g. a new line character). Regards, Gonzalo On 17 November 2015 at 17:16, Hemanth Abbina <[email protected]<mailto:[email protected]>> wrote: Hi, Though it's against the basic design principle of Flume, I have one question. Is this possible to persist the connection between source & sink and re-use ? We are using HTTP source, File channel & Kafka sink and with that configuration, not getting the expected throughput because of the reconnections of the source & sink for every event. So, would it be possible to re-use the same HTTP and Kafka connections for multiple transactions ? (even with a custom source & sink) Thanks, Hemanth
