That is exactly why we've decided to stick with java. Also support for all
consumer settings out of the box.
Kind regards,
Radek Gruchalski
On 22 Dec 2012, at 19:17, David Arthur mum...@gmail.com wrote:
FWIW, message production is quite simpler than consumption. It does
not require the same complex coordination as the consumers. Producers
only use ZooKeeper to locate available brokers
Sent from my phone
On Dec 22, 2012, at 1:00 PM, Apoorva Gaurav apoorvagau...@gmail.com wrote:
Thanks Radek,
We also are thinking of Java / Scala for Consumers, for Producers whether
franz-kafka is a good choice?
--
Thanks Regards,
Apoorva
On Sat, Dec 22, 2012 at 9:38 PM, Radek Gruchalski
radek.gruchal...@portico.io wrote:
We started using node-kafka before we learned franz-kafka was available.
In node, franz-kafka would be my preferred choice now. But tbh, our
consumers are all java. node-kafka does not support consumer settings like
autooffset.reset and so on (or it is not obvious how to use those).
Afair franz-kafka offers those. Also, java zkconsumer gives you the jmx
monitoring tools, which may be helpful if you want to add some scaling
logic when consumer is lagging.
Our first choice is node too but we're consuming exclusively with java.
Hope this helps a little.
On 22 Dec 2012, at 05:21, Apoorva Gaurav apoorvagau...@gmail.com wrote:
Which is the best ZK based implementation of kafka in node.js. Our use
case
is that a pool of node js http servers will be listening to clients which
will send json over http. Using node js we'll do minimal decoration and
compression (preferably snappy) and write to brokers. We might also need
json to avro conversion but thats not a deal breaker. Consumers will be
writing these events to S3 (to begin with we don't plan to maintain HDFS
cluster). To begin with we'll have to support a peak load of 50K events /
second, average being much less, around 2K events / second. Suggestions
please. Is any one using franz-kafka in production. I'm only two days
into
kafka so don't know a lot, but franz-kafka looks clean and easy to work
with.
If none of the existing node.js implementation is capable of this then we
are willing to move to Scala or Java but node.js is the first choice.
Thanks Regards,
Apoorva
On Sat, Dec 22, 2012 at 2:25 AM, Radek Gruchalski
radek.gruchal...@portico.io wrote:
We are using https://github.com/radekg/node-kafka, occasionally pushing
about 2500 messages, 3.5K each / second. No issues so far. Different
story
with consumers. They are stable but under heavy load we experienced CPU
problems. I am the maintainer of that fork. The fork comes with ZK
integration. Another kafka module is this one:
https://github.com/dannycoates/franz-kafka.
Kind regards,
Radek Gruchalski
radek.gruchal...@technicolor.com (mailto:
radek.gruchal...@technicolor.com)
| radek.gruchal...@portico.io (mailto:radek.gruchal...@portico.io) |
ra...@gruchalski.com (mailto:ra...@gruchalski.com)
00447889948663
Confidentiality:
This communication is intended for the above-named person and may be
confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor
must you copy or show it to anyone; please delete/destroy and inform the
sender immediately.
On Thursday, 20 December 2012 at 18:31, Jun Rao wrote:
Chris,
Not sure how stable those node.js clients are. In 0.8, we plan to
provide a
native C version of the producer. A thin node.js layer can potentially
be
built on top of that.
Thanks,
Jun
On Thu, Dec 20, 2012 at 8:46 AM, Christopher Alexander
calexan...@gravycard.com (mailto:calexan...@gravycard.com) wrote:
During my due diligence to assess use of Kafka for both our activity
and
log message streams, I would like to ask the project committers and
community users about using Kafka with Node.js. Yes, I am aware that a
Kafka client exists for Node.js (
https://github.com/marcuswestin/node-kafka), which has spurred
further
interest by our front-end team. Here are my questions, excuse me if
they
seem noobish.
1. How reliable is the Node.js client (
https://github.com/marcuswestin/node-kafka) in production
applications?
If there are issues, what are they (the GitHub repo currently lists
none)?
2. To support real-time activity streams within Node.js, what is the
recommended consumer polling interval?
3. General advise observations on integrating a front-end based
Node.js
application with Kafka mediated messaging.
Thanks you!
Chris
