Right that makes much more sense. You will probably want to make sure that your updates are idempotent (or you could just accept the risk), though in the SQL case you could commit your offset to the DB as part of the same transaction (requires more custom stuff).
Christian On Fri, Sep 12, 2014 at 5:45 PM, Patrick Barker <patrickbarke...@gmail.com> wrote: > O, I'm not trying to use it for persistence, I'm wanting to sync 3 > databases: sql, mongo, graph. I want to publish to kafka and then have it > update the db's. I'm wanting to keep this as efficient as possible. > > On Fri, Sep 12, 2014 at 6:39 PM, cac...@gmail.com <cac...@gmail.com> > wrote: > > > I would say that it depends upon what you mean by persistence. I don't > > believe Kafka is intended to be your permanent data store, but it would > > work if you were basically write once with appropriate query patterns. It > > would be an odd way to describe it though. > > > > Christian > > > > On Fri, Sep 12, 2014 at 4:05 PM, Stephen Boesch <java...@gmail.com> > wrote: > > > > > Hi Patrick, Kafka can be used at any scale including small ones > > > (initially anyways). The issues I ran into personally various issues > with > > > ZooKeeper management and a bug in deleting topics (is that fixed yet?) > > In > > > any case you might try out Kafka - given its highly performant, > > scalable, > > > and flexible backbone. After that you will have little worry about > > scale > > > - given Kafka's use within massive web scale deployments. > > > > > > 2014-09-12 15:18 GMT-07:00 Patrick Barker <patrickbarke...@gmail.com>: > > > > > > > Hey, I'm new to kafka and I'm trying to get a handle on how it all > > > works. I > > > > want to integrate polyglot persistence into my application. Kafka > looks > > > > like exactly what I want just on a smaller scale. I am currently only > > > > dealing with about 2,000 users, which may grow, but is kafka a good > > use > > > > case here, or is there another technology thats better suited? > > > > > > > > Thanks > > > > > > > > > >
