We did a relatively complete prototype but it isn't integrated into the mainline code yet and there isn't a target release date. There is rather a lot of testing and compatability work that would have to be done to fully productionize it. I suspect someone will pick it up in 2015 but I wouldn't block on it.
-Jay On Mon, Jan 26, 2015 at 1:55 PM, Yang <teddyyyy...@gmail.com> wrote: > well I guess this is what I need > > https://cwiki.apache.org/confluence/display/KAFKA/Transactional+Messaging+in+Kafka > > anyone knows how far this effort has gone? > > thanks > Yang > > On Mon, Jan 26, 2015 at 12:06 PM, Yang <teddyyyy...@gmail.com> wrote: > > > we are pulling from a DB, maybe 100k rows at a time. for each row we send > > it to kafka. > > > > now the problem is , if any errors happen during the db pulling (such as > > an earthquake), we stop. > > next time we wake up. we can't write the same record again into kafka, > > otherwise there might be over counting. > > > > > > does kafka support a "BATCH mode" with commit ? so basically at the start > > of my session, I declare "start transaction", and after all records have > > been pulled and sent to kafka, I declare "commit" . > > > > Thanks > > Yang > > >
