Re: open source projects based on kafka

2015-09-13 Thread Hisham Mardam-Bey
, at 10:34 PM, Li Tao wrote: > > > > > > > > Hi Hackers, > > > > > > > > This is Lee, a learner of kafka, i have read the original paper on > > kafka, > > > > and walked through the document. > > > > > > > > I think th

Re: Any drawbacks of running Kafka consumer in a web container

2015-08-21 Thread Hisham Mardam-Bey
insights. > > - > Venkatesh > -- Hisham Mardam-Bey -=[ CTO ]-=-[ Mate1 Inc. ]=- A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing in e-mail? -=[ Codito Ergo Sum ]=-

Re: Pulling Snapshots from Kafka, Log compaction last compact offset

2015-05-10 Thread Hisham Mardam-Bey
ead so far and stop as > > soon > > > > as I find a key beeing already in my set. I use this as an indicator > of > > > how > > > > far the log compaction has happened already and only pull up to this > > > point. > > > > This works quite well as I do not need to keep the messages but only > > > their > > > > keys in memory. > > > > > > > > The question I want to raise with the community is: > > > > > > > > How do you prevent pulling the same record twice (in different > > versions) > > > > and would it be beneficial if the "OffsetResponse" would also return > > the > > > > last offset that got compacted so far and the application would just > > pull > > > > up to this point? > > > > > > > > Looking forward for some recommendations and comments. > > > > > > > > Best > > > > Jan > > > > > > > > > > > > > > -- Hisham Mardam-Bey http://hisham.cc/

Re: [ANN] sqlstream: Simple MySQL binlog to Kafka stream

2015-03-17 Thread Hisham Mardam-Bey
ka. > > > > > > This is an early release and there are a few caveats (mentionned in the > > > README), mostly the poor partitioning which I'm going to evolve quickly > > > and the reconnection strategy which doesn't try to keep track of binlog > > > position, other than that, it should work as advertised. > > > > > > Cheers, > > > - pyr > > > -- Hisham Mardam-Bey http://hisham.cc/

Re: Plan for Scala 2.10+ support

2013-10-30 Thread Hisham Mardam-Bey
>>> Hi, > >>>>> We are building pub-sub applications over kafka in our company. Some > >> of > >>>>> our packages have been build on Scala 2.10+. Though only Kafka_2.9.2 > >> is > >>>>> available on maven repository. Ka

Identifying broker and partition for a message

2013-07-10 Thread Hisham Mardam-Bey
Hey guys, In 0.7 how can one identify the broker and partition that a message was pulled from in the high level consumer? Thanks! hmb. -- Hisham Mardam-Bey

Identifying broker and partition for a message

2013-07-10 Thread Hisham Mardam-Bey
Hey guys, In 0.7 how can one identify the broker and partition that a message was pulled from in the high level consumer? Thanks! hmb. -- Hisham Mardam-Bey