installation Kafka to existing hadoop environment

2013-04-03 Thread Oleg Ruchovets
Hi , I want to install it to the existing hadoop environment. I want to read data from Kafka and put it to HDFS. I use a Hortonworks distributions. Questions: I didn't find any guide how to install Kafka ( I read quick start http://kafka.apache.org/quickstart.html). Do I only need

Re: installation Kafka to existing hadoop environment

2013-04-03 Thread Jun Rao
The configs at LinkedIn can be found at https://cwiki.apache.org/confluence/display/KAFKA/Operations You may also want to checkout Kafka-Hadoop pipeline at LinkedIn: https://github.com/linkedin/camus Thanks, Jun On Wed, Apr 3, 2013 at 3:40 AM, Oleg Ruchovets oruchov...@gmail.com wrote: Hi ,

.net (c#) kafka client

2013-04-03 Thread Oleg Ruchovets
Hi , Is there a stable C# client for Kafka? Is there a rest API for Kafka? Thanks Oleg.

Re: .net (c#) kafka client

2013-04-03 Thread Oleg Ruchovets
I see , Is it a good Idea to use Node.js client? C# will produce messages to Node.js and Node.js will push it to the Kafka? Is there potential problem with such aproach? Thanks Oleg. On Wed, Apr 3, 2013 at 9:18 PM, Joel Koshy jjkosh...@gmail.com wrote: Unfortunately no - there is a legacy

Re: .net (c#) kafka client

2013-04-03 Thread David Arthur
What is the bridge between C# and Node.js? If you're writing some custom middle man, why not write it in Java or Scala so you can then use the official clients? -David On 4/3/13 3:22 PM, Oleg Ruchovets wrote: I see , Is it a good Idea to use Node.js client? C# will produce messages to

Re: .net (c#) kafka client

2013-04-03 Thread Oleg Ruchovets
Yes , I agree. So there is no way to use C# client or Rest API with current version of Kafka. Thanks Oleg, On Wed, Apr 3, 2013 at 10:29 PM, David Arthur mum...@gmail.com wrote: What is the bridge between C# and Node.js? If you're writing some custom middle man, why not write it in Java or

Re: .net (c#) kafka client

2013-04-03 Thread Matthew Rathbone
Maybe you could write a C# client. I don't think it would be super hard to write a basic one (that doesn't require zookeeper for example). C# has a lot of great features that would make it great for a solid kafka client. On Wed, Apr 3, 2013 at 3:58 PM, Oleg Ruchovets oruchov...@gmail.com wrote:

kafka 0.8 release schedule?

2013-04-03 Thread Soby Chacko
Hello, I am sure this question has been asked before. But, can someone tell me the tentative release schedule for Kafka 0.8? And also, when its released, is it going to be based on Scala 2.9.2 or default to 2.8? Regards, Soby Chacko.

Re: kafka 0.8 release schedule?

2013-04-03 Thread Ryan LeCompte
A 2.10 version would be awesome! Please please :-) On Wed, Apr 3, 2013 at 4:02 PM, Soby Chacko sobycha...@gmail.com wrote: Hello, I am sure this question has been asked before. But, can someone tell me the tentative release schedule for Kafka 0.8? And also, when its released, is it going

Re: kafka 0.8 release schedule?

2013-04-03 Thread Jun Rao
Soby, We plan to release 0.8 beta in Apr. Our build tool supports building on different versions of scala and the default one is 2.8. Thanks, Jun On Wed, Apr 3, 2013 at 4:02 PM, Soby Chacko sobycha...@gmail.com wrote: Hello, I am sure this question has been asked before. But, can someone

Re: installation Kafka to existing hadoop environment

2013-04-03 Thread Jun Rao
There should be a release-zip target that you can run from sbt in 0.7. It's broken in 0.8 and we plan to fix it ( https://issues.apache.org/jira/browse/KAFKA-843). Jun On Wed, Apr 3, 2013 at 8:29 AM, Oleg Ruchovets oruchov...@gmail.com wrote: Hi Jun Thank you for the answers. I've seen

Re: Zookeeper version

2013-04-03 Thread Jun Rao
Api-wise, ZK 3.4.5 should be fine. Not sure how stable it is. Thanks, Jun On Wed, Apr 3, 2013 at 3:50 PM, Ian Friedman i...@flurry.com wrote: Hey guys, I know LinkedIn uses Zookeeper 3.3.x, I'm wondering if Kafka has any hard dependency on that version or we would be OK using 3.4.5 since