For the sink, I would be surprised if the connection to kafka is not the same all the time. For the http source you could create a custom source where you keep a long lived http connection and have some way of detecting where a batch of events is sent (e.g. a new line character).
Regards, Gonzalo On 17 November 2015 at 17:16, Hemanth Abbina <[email protected]> wrote: > Hi, > > > > Though it’s against the basic design principle of Flume, I have one > question. > > > > Is this possible to persist the connection between source & sink and > re-use ? > > > > We are using HTTP source, File channel & Kafka sink and with that > configuration, not getting the expected throughput because of the > reconnections of the source & sink for every event. > > > > So, would it be possible to re-use the same HTTP and Kafka connections for > multiple transactions ? (even with a custom source & sink) > > > > Thanks, > > Hemanth >
