Whats your timeline Khurrum? Maybe we can work something out. On Wed, Nov 2, 2016 at 8:15 AM, Leigh Stewart <lstew...@twitter.com> wrote:
> We have in fact built something like this. No plans as yet to release, but > I think we would like to eventually. > > On Wed, Nov 2, 2016 at 2:43 AM, Khurrum Nasim <khurrumnas...@gmail.com> > wrote: > >> As part of implementing the kafka subscriber interface, I am wondering is >> there anyone use DL stream for storing the offsets? >> >> For example, if I have N streams (0..N-1), I need to track the read offset >> for each stream and store them somewhere. I can probably use other >> external >> services (like any key/value store) to store the offset. But it would >> introduce extra dependencies. I am thinking if I have a map of <stream, >> offset> and periodically flush the map into a separate stream (let's say >> __offset_ stream). With proper truncation/checkpoint mechanism, it would >> be >> very fast. >> >> This use case here is a very standard replicated state machine. I am also >> wondering do you guys think of providing some common library on >> distributedlog to simply implementing state machines. >> >> - KN >> > >
