Re: The motive to detach new consumer and zookeeper

2015-12-07 Thread Jason Gustafson
Hi Li, I think reducing the client's complexity and improving performance were two of the main reasons for the change. The rebalance protocol on top of Zookeeper was difficult to implement correctly, and I think a number of Kafka clients never actually got it working. Removing it as a dependence a

Re: how robust are file accesses?

2015-12-07 Thread Jay Kreps
Kafka should be robust against abrupt termination. -Jay On Sat, Dec 5, 2015 at 11:05 AM, Per Jahn wrote: > Hi > > Are files that kafka writes to the file system robust against abrupt > termination? > Including fs meta data. > And assuming it's used with a journaling fs. > Or do I need ups and a

Re: Good java consumer example with new 0.9 consumer api

2015-12-07 Thread Jay Kreps
Hey guys, Those docs were for a draft version of the API and I think they may be a bit out of date. I'd stick with the javadocs linked here: http://kafka.apache.org/documentation.html#newconsumerapi -Jay On Sun, Dec 6, 2015 at 7:40 AM, Dhyan Muralidharan wrote: > Thank you .These docs are good

Unable to set log.cleanup.policy on topic

2015-12-07 Thread Rakesh Vidyadharan
Hello, I upgraded to the latest 0.8 release 0.8.2.2, and tried to set the log cleanup policy on some of our topics (I got same error with 0.8.1, but thought it may have been some issue with that release). /opt/kafka/bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic metamorphosis.t

Re: Unable to set log.cleanup.policy on topic

2015-12-07 Thread Stevo Slavić
Hello Rakesh, log.cleanup.policy is broker configuration property, while cleanup.policy is topic configuration property (see http://kafka.apache.org/documentation.html#topic-config ). Since you are configuring particular topic, you need to use second one. Kind regards, Stevo Slavic. On Mon, Dec

Re: Unable to set log.cleanup.policy on topic

2015-12-07 Thread Ofir Manor
Hi, log.cleanup.policy is the server-level parameter. The per-topic parameter you are looking for is just cleanup.policy See here - http://kafka.apache.org/documentation.html#topic-config Also, it is listed in the help: $ kafka-topics.sh Create, delete, describe, or change a topic. Option

Re: Unable to set log.cleanup.policy on topic

2015-12-07 Thread Rakesh Vidyadharan
Duh…, did note the difference in property naming there. Thanks, was able to set the property. On 07/12/2015 14:37, "Stevo Slavić" wrote: >Hello Rakesh, > >log.cleanup.policy is broker configuration property, while cleanup.policy >is topic configuration property (see >http://kafka.apache.org

Re: Please add me to the mailing list...thanks

2015-12-07 Thread Guozhang Wang
Hello Mamadou, It is self-service: http://kafka.apache.org/contact.html Cheers, Guozhang On Sun, Dec 6, 2015 at 2:42 PM, Mamadou Sylla wrote: > Please add me to the mailing list...thanks > -- -- Guozhang

Kafka integration with Oracle QA

2015-12-07 Thread CY Kuek
Hi All, Good day, I have a question would like to seek for your advice. Is Kafka able to integrate with Oracle AQ for near real-time integration? Do you have any details on this? Thanks. Regards, Chiew Yea