Hey Tao, Interesting that you're seeing a lot of overhead constructing the new consumer instance each time. Granted it does have to fetch topic metadata and lookup the coordinator, but I wouldn't have expected that to be a big problem. How long is it typically taking?
-Jason On Mon, Jan 4, 2016 at 3:26 AM, Marko Bonaći <marko.bon...@sematext.com> wrote: > How are you consuming those topics? > > IF: I assume you have a consumer, so why not commit from within that > consumer, after you process the message (whatever "process" means to you). > > ELSE: couldn't you have a dedicated consumer for offset commit requests > that you don't shut down between requests? > > FINALLY: tell us more about your use case. > > Marko Bonaći > Monitoring | Alerting | Anomaly Detection | Centralized Log Management > Solr & Elasticsearch Support > Sematext <http://sematext.com/> | Contact > <http://sematext.com/about/contact.html> > > On Mon, Jan 4, 2016 at 12:18 PM, tao xiao <xiaotao...@gmail.com> wrote: > > > Hi team, > > > > I have a scenario where I want to write new offset for a list of topics > on > > demand. The list of topics is unknown until runtime and the interval > > between each commit is undetermined. what would be the best way to do so? > > > > One way I can think of is to create a new consumer and call > > commitSync(offsets) every time I want to commit. But it seems taking too > > much time to bootstrap the consumer. is there a lighter way to achieve > > this? > > >
