Re: reprocessing messages in kafka

2013-07-31 Thread Milind Parikh
It is possible to consume the same message more than once with the same consumer. However WHAT you actually do with the message (such as idempotent writes) is the tricker part. Regards Milind On Wed, Jul 31, 2013 at 8:22 AM, Oleg Ruchovets oruchov...@gmail.comwrote: Hi , I just don't know

Re: Truncated messages in fetch response

2013-06-27 Thread Milind Parikh
Jay is correct. ..it manifests itself when the size threshold condition is violated. For erlang client, this is fixed in 0.4.6 of mps. Github.com/milindparikh/mps Regards Milind On Jun 27, 2013 3:30 PM, Bob Potter bobby.pot...@gmail.com wrote: Vadim, I don't know under exactly what

Re: mps, an eventing framework for kafka

2013-06-15 Thread Milind Parikh
...@gmail.com wrote: Milind, Thanks for sharing. How did you consume Kafka data in Erlang? Jun On Fri, Jun 14, 2013 at 10:23 PM, Milind Parikh milindpar...@gmail.com wrote: There is a lot of useful discussion on the user group about data messages and eventing, I thought that I would

Re: Using Kafka for data messages

2013-06-15 Thread Milind Parikh
You might want to take a look at mps (github.com/milindparikh/mps). As I was thinking about blogging about it, your use-case magically surfaced on the mailing list. Since one of the supposed benefits of having a framework is to enable quick building of stuff on top of it, I took a crack at your

Re: Partitioning and scale

2013-05-23 Thread Milind Parikh
Number of files to manage by os, I suppose. Why wouldn't you use consistent hashing with deliberately engineered collisions to generate a limited number of topics / partitions and filter at the consumer level? Regards Milind On May 23, 2013 4:22 PM, Timothy Chen tnac...@gmail.com wrote: Hi

Kafka support

2013-04-12 Thread Milind Parikh
If a F500 company wants commercial support for Kafka, who would they turn to? It appears that there seems to be natural fit with real time processing schemes aka stormtrident. I am sure that someone in the community must have come across this issue. Thanks Milind