Re: Topology Suggestion Required for Batching

Hi Padma, Firstly, thanks for responding. Here is how i am defining my topology conceptually.. - Spout waits for a request signal.. - once spout got a signal, it generates a request_id and broadcasts that request_id to 10 csv reader bolts.. - 10 csv reader bolts reads csv files line-by-line

Running topology from supervisor node

can I run topology form a supervisor node because when try it it gave me the current error : Exception in thread main java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused any help please ? best regards . -- *Al Fartakh

Doubts on Apache Storm

Hi, I am working on a project that need real time data computation with very low latency and high speed parallel processing. I went through the basics of Apache Storm and found it very interesting and it looks like it can meet my requirements. I have few doubts on Apache storm. Could you please

duplicated result

I'm using a bolt that receives tuples from another bolt (exclamation bolt ) and writes it on a file , the problem I got is that I have duplicated results four times , like when I emit a word , I found the word Written four times . where's the problem possibly could be ? public class

Re: Doubts on Apache Storm

Hi, Below are my inline comments: Does Apache storm supports cluster environment. If yes, does it supports load balancing as well? Yes, storm can be deployed in cluster environment. It supports load balancing too by distributing the task over cluster nodes. Is there any dependancy

Re: Can i increase the parallelism of the topology as per traffic

The number of executors and workers can be changed but the number of tasks is fixed at topology creation time. On May 6, 2014 1:40 AM, M.Tarkeshwar Rao tarkeshwa...@gmail.com wrote: Hi, Can i increase the parallelism of the topology as per traffic? I need inputs from you all. any link?

Re: duplicated result

You are creating your file writer with append set to true. It's it possible your topology was run more than once? On May 6, 2014 6:39 AM, Bilal Al Fartakh alfartaj.bi...@gmail.com wrote: I'm using a bolt that receives tuples from another bolt (exclamation bolt ) and writes it on a file , the

Re: duplicated result

HI ,Nathan and thank you for responding ,I appreciate it ! no I'm not , I just run this topology for the first time public class Ex { public static class ExclamationBolt extends BaseRichBolt { OutputCollector _collector; @Override public void prepare(Map conf,

Re: duplicated result

Can you share the contents of your input and output files? On May 6, 2014 7:54 AM, Bilal Al Fartakh alfartaj.bi...@gmail.com wrote: HI ,Nathan and thank you for responding ,I appreciate it ! no I'm not , I just run this topology for the first time public class Ex { public static

Re: duplicated result

Hi, The issue lies with the number of tasks/executors specified for spout. Try specifying 1 spout and see if you could see duplicates. I suppose there would be no duplicates in specifying 1 spout and 1 exclamtion bolt. On Tue, May 6, 2014 at 5:23 PM, Bilal Al Fartakh

Re: Doubts on Apache Storm

On Tue, May 06, 2014 at 03:21:13PM +0530, milind.pa...@polarisft.com wrote: Hi, Is Nimbus mandatory for storm? (Our development env is neither using Nimbus nor any other cloud environment) I think you might have misunderstood nimbus. It is a daemon that is part of storm, *not*

Re: duplicated result

Good point, but it shouldn't matter how many exclamation bolts there are. The number of spouts does because they are all reading the same file. On May 6, 2014 9:37 AM, padma priya chitturi padmapriy...@gmail.com wrote: Hi, The issue lies with the number of tasks/executors specified for spout.

Fw: Doubts on Apache Storm

Hi All, Thanks for clarifying the doubts. Can I request if there is any doc that demonstrate the deployment steps, pseudo code, FAQ, etc. (Basically basic building blocks) Regards Milind Patil | Intellect Liquidity cash management 8SWS 031 | Silver metropolis | Western express highway |

Re: duplicated result

Thank you :D specifying 1 spout was the solution . thank you Padma and Nathan again for your help ! 2014-05-06 14:40 GMT+01:00 Nathan Leung ncle...@gmail.com: Good point, but it shouldn't matter how many exclamation bolts there are. The number of spouts does because they are all reading the

RE: Doubts on Apache Storm

Milind, Have you looked at http://storm.incubator.apache.org/documentation/Home.html http://storm.incubator.apache.org/documentation/Creating-a-new-Storm-project.html Roger -Original Message- From: milind.pa...@polarisft.com [mailto:milind.pa...@polarisft.com] Sent: Tuesday, May 06,

question about TransactionalTridentKafkaSpout

Hi, I'm new to storm and trident. I have some general questions regarding usage of TransactionTridentKafkaSpout. I'm trying to understand the intervals but not sure where can i find some good resource. 1. Say one kafka topic has n partitions, how does storm worker/executor mapped to partitions ?

Re: Topology Suggestion Required for Batching

Can you tell us more about this use case? I don't really understand, but given what you've said so far, I might create a trident topology something like this: TridentTopology topology = new TridentTopology(); StormTopology = topology.newStream(spout1, spout) .each(new

Re: Question about OpaqueTridentKafkaSpout

I think it can. That is where the coordinator comes in picture. Coordinator defines the parameters of a batch and emitters do the job of emitting the sub portions of batch. On Mon, May 5, 2014 at 12:50 PM, Abhishek Bhattacharjee abhishek.bhattacharje...@gmail.com wrote: Are you sure that a