RE: Maven error

2014-08-19 Thread Georgy Abraham
Yes i meant delete the folder , but still the error isn't it ?? Try running a simple maven clean compile package install command .. Don't know what's causing your error -Original Message- From: researcher cs Sent: 19-08-2014 AM 08:20 To: user@storm.incubator.apache.org Subject: Re:

Re: Maven error

2014-08-19 Thread researcher cs
i got this after deleted folder [ERROR] BUILD ERROR [INFO] [INFO] Error building POM (may not be this project's POM). Project ID: org.clojure:clojure:jar:1.4.0 Reason: Cannot find parent: org.sonatype.oss:oss-parent for

RE: Supervisor keeps crashing

2014-08-19 Thread Bishnu Patro
From the error message, it seems some of the dependent jar file is missing from your storm lib directory. From: Kushan Maskey [mailto:kushan.mas...@mmillerassociates.com] Sent: 19 August 2014 02:29 To: user@storm.incubator.apache.org Subject: Supervisor keeps crashing I have two servers in

Re: kafka high level consumer - threads guaranteed to read a single partition?

2014-08-19 Thread Mattijs Ugen (DT)
For the kafka high level consumer, if I create exactly the number of threads as the number of partitions, is there a guarantee that each thread will be the only thread that reads from a particular partition? I'm following this example

Real time messages for testing Storm Kafka

2014-08-19 Thread Maisnam Ns
Hi, Can anyone suggest a tool that can generate enormous amount of text messages for testing Storm and Kafka. Thanks Ms

Re: java.lang.ArrayIndexOutOfBoundsException: 3 atbacktype.storm.utils.DisruptorQueue.consumeBatchToCursor

2014-08-19 Thread Kushan Maskey
Hi georgy, Thanks for the reply. I realized that it was coming from my code and I have resolved my problem. So what I found out is that even when there is no message in kafka to be read, KafkaSpout keep emitting a null or empty string fields. I take that emitted value and then i parse the data

Re: Supervisor keeps crashing

2014-08-19 Thread Kushan Maskey
I have installed the same storm package 0.9.2 on the servers and i have only removed the netty.3.2.2.jar files from there since it was conflicting with netty-3.6.3.Final.jar. Besides that I have not made any changes to other jars. If you can point me the jar you think is not correct/missing

What happens on a batch timeout?

2014-08-19 Thread Simon Cooper
When a batch times out, what happens to all the current in-flight tuples when the batch is replayed? Are they removed from the executor queues, or are they left in the queues, so they might be received by the executor as part of the replayed batch/next batch, if the executor is running behind?

RE: What happens on a batch timeout?

2014-08-19 Thread Simon Cooper
BTW, I'm referring to trident batches. From: Simon Cooper [mailto:simon.coo...@featurespace.co.uk] Sent: 19 August 2014 15:49 To: user@storm.incubator.apache.org Subject: What happens on a batch timeout? When a batch times out, what happens to all the current in-flight tuples when the batch is

RE: Maven error

2014-08-19 Thread Georgy Abraham
Hope you have given the required repositories in pom.xml for the dependencies U need . Some jars are not available in basic or default maven repository. U may need to add clojars or others repositories -Original Message- From: researcher cs Sent: 19-08-2014 AM 11:41 To:

RE: java.lang.ArrayIndexOutOfBoundsException: 3atbacktype.storm.utils.DisruptorQueue.consumeBatchToCursor

2014-08-19 Thread Georgy Abraham
I have tried Kafka spout of wrustmenier , the one that's getting integrated in 0.92 storm . That didn't give me any such problem . Are you sure its emitting empty messges . If any tulle coming from Kafka spout is not acknowledged , it will be replayed after the timeout. Is this your problem ??

Re: Real time messages for testing Storm Kafka

2014-08-19 Thread Adaryl Bob Wakefield, MBA
I was going to suggest connecting to Twitter at first but don’t they throttle the stream somehow? Adaryl Bob Wakefield, MBA Principal Mass Street Analytics 913.938.6685 www.linkedin.com/in/bobwakefieldmba Twitter: @BobLovesData From: Georgy Abraham Sent: Tuesday, August 19, 2014 2:14 PM To:

Re: java.lang.ArrayIndexOutOfBoundsException: 3atbacktype.storm.utils.DisruptorQueue.consumeBatchToCursor

2014-08-19 Thread Kushan Maskey
When i look at the worker logs, I see that KafkaSpout keep trying to get message from Kafka and then emits some thing even though there is no value and i see some logs that I print out in the bolt execute method being printed out. I do not see the error now as I have added a check in my code to

RE: java.lang.ArrayIndexOutOfBoundsException:3atbacktype.storm.utils.DisruptorQueue.consumeBatchToCursor

2014-08-19 Thread Georgy Abraham
One quick thing to check , if the message is being replayed the offset of Kafka queue won't get incremented for each message . For example, if there was 5 msgs earlier in Kafka queue , the log will show committing offset 5 . Of you pit a new message , ideally after processing the offset should

Re: Maven error

2014-08-19 Thread researcher cs
Thanks for helping me and for your effort and time . i'm new in maven . what about 1.4.0 ? is it a version or what ? and what exactly should i write it ? On Tue, Aug 19, 2014 at 9:01 PM, Georgy Abraham itsmegeo...@gmail.com wrote: Hope you have given the required repositories in pom.xml for

data cleansing in real time systems

2014-08-19 Thread Adaryl Bob Wakefield, MBA
I need help clearing something up. So I read this: http://nathanmarz.com/blog/how-to-beat-the-cap-theorem.html And in it he says: “Likewise, writing bad data has a clear path to recovery: delete the bad data and precompute the queries again. Since data is immutable and the master dataset is

Re: Storm on top of Mesos running on Docker

2014-08-19 Thread Ankit Toshniwal
On the storm UI are you seeing your topology getting the required number of workers and is the topology emitting any tuples? Also Storm UI should tell you if any of the supervisors were started and if they were on which machine and you can ssh to that machine to look at supervisor logs. But the

Worker logs not created

2014-08-19 Thread Palak Shah
Hi, I have a storm cluster running with 3 supervisor nodes. When I submit a topology in cluster mode, all the supervisors and workers are up and running, but the worker logs are not created. I can just see an empty folder called worker in /tmp/stormtmp (storm dataDir). previously someone had