Re: Need to use Kafka with spark

This is a very nice intro to spark and kafka, with some valuable details about partitioning and parallelism http://www.michael-noll.com/blog/2014/10/01/kafka-spark-streaming-integration-example-tutorial/ 2014-12-05 21:31 GMT+01:00 sanjeeb kumar : > Hi Team, > I am able to install Kafka in Ub

Re: consumer reaction to auto-create topics (0.8.1)

ted: > > https://issues.apache.org/jira/browse/KAFKA-1006 > > And we are still figuring the right way to fix it. Could you describe your > use cases where you need to start the consumer beforehand? > > Guozhang > > > On Thu, Mar 13, 2014 at 2:42 AM, David Morales de Frías

consumer reaction to auto-create topics (0.8.1)

Hi there, A few days ago i have written about an issue (0.8.0) when auto-create topic is enabled and consumer starts before producer (consumer never sees any message and they seem to be lost even by the brokers) Now i have done the same test with 0.8.1 and the issue is partially fixed but now the

Re: consumer reaction to auto-create topics

t; while rebalancing and issuing fetch requests for the topic. > > Could you try to run this from scratch (following the quickstart). Ensure > that the broker and zookeeper have been started successfully. > > Thanks, > Neha > > > On Thu, Mar 6, 2014 at 9:34 AM, David Mor

Re: consumer reaction to auto-create topics

0.8, thanks. 2014-03-06 18:27 GMT+01:00 Jun Rao : > Which version of Kafka are you using? > > Thanks, > > Jun > > > On Thu, Mar 6, 2014 at 2:10 AM, David Morales de Frías < > dmora...@paradigmatecnologico.com> wrote: > > > Hi there, > > > &g

consumer reaction to auto-create topics

Hi there, If i start a consumer in a non-existent topic (auto-create true) before the producer, the consumer never gets the messages. These are the steps: 1) kafka-console-consumer --topic newTopic (it doesn't exist) 2) kafka-console-producer --topic newTopic 3) Send some messages 4) I can see t

Re: Reg Partition and Replica?

Maybe these pictures can help you: https://kafka.apache.org/images/log_anatomy.png http://www.michael-noll.com/blog/uploads/kafka-topics-partitions-replicas.png http://www.michael-noll.com/blog/uploads/kafka-cluster-overview.png And of course, this post might help too: http://www.michael-noll.co

Re: calling ad-hoc topic creation from code

(), replicationFactor.intValue(), ""); zkClient.close(); Regards. 2014-02-25 18:50 GMT+01:00 David Morales de Frías < dmora...@paradigmatecnologico.com>: > Nope... i have checked this and replication factor is 1. Anyway, when i > tried to increase this param, an exception

Re: calling ad-hoc topic creation from code

okers in the > cluster is less than the replication factor at topic creation time. > > Thanks, > Neha > > > On Tue, Feb 25, 2014 at 8:39 AM, David Morales de Frías < > dmora...@paradigmatecnologico.com> wrote: > > > This is the code that i can see i

Re: calling ad-hoc topic creation from code

erCluster, 3, 3);* Thanks 2014-02-25 17:25 GMT+01:00 Jun Rao : > Is the ZK connection string + namespace the same btw the code and the > script? > > Thanks, > > Jun > > > On Tue, Feb 25, 2014 at 3:01 AM, David Morales de Frías < > dmora...@paradigmatecnologico.co

calling ad-hoc topic creation from code

Hi there, I'm trying to create a topic from java code, by calling CreateTopicCommand: *ZkClient zkClient = new ZkClient(zookeeperCluster, 3, 3);* *CreateTopicCommand.createTopic(zkClient, topic, numPartitions.intValue(),replicationFactor.intValue(), "");* *zkClient.close();* The prog