how to improve a trydent topology

Hi all, Can any one please suggest how to improve the performance of a trident topology? We are using storm trident for our application. Regards Tarkeshwar

Re: Trident transactional topology stuck re-emitting batches with Netty, but running fine with ZMQ (was Re: Topology is stuck)

In which version it is available. On 16 Sep 2014 19:01, "Danijel Schiavuzzi" wrote: > Yes, it's been fixed in 'master' for some time now. > > Danijel > > On Tuesday, September 16, 2014, M.Tarkeshwar Rao > wrote: > >> Hi Danijel, >

Re: Trident transactional topology stuck re-emitting batches with Netty, but running fine with ZMQ (was Re: Topology is stuck)

esday, September 16, 2014, M.Tarkeshwar Rao wrote: > Hi Danijel, > > Is the issue resolved in any version of the storm? > > Regards > Tarkeshwar > > On Thu, Jul 17, 2014 at 6:57 PM, Danijel Schiavuzzi < > dani...@schiavuzzi.com> wrote: > >> I've filled a

Re: Can I make Nimbus failsafe

, M.Tarkeshwar Rao wrote: > Ok we can do it together. How can we plan it? > > > On Tue, Sep 9, 2014 at 6:09 PM, 潘臻轩 wrote: > >> hi, I can not direct git you source code, it mix our company code, >> I have do many change for community branch. >> I will implement it

Re: Trident transactional topology stuck re-emitting batches with Netty, but running fine with ZMQ (was Re: Topology is stuck)

pology stops processing. See the above JIRA for > additional info. > > Danijel > > > > > On Thu, Jul 17, 2014 at 7:20 AM, M.Tarkeshwar Rao > wrote: > >> Thanks Danijel for helping me. >> >> >> On Thu, Jul 17, 2014 at 1:37 AM, Danijel S

Any one used to build their projects with storm 0.9.2 incubating using gradle?

Hi all, Any one used storm 0.9.2 incubating using gradle? We are getting compilation errors with this version with gradle but compilation is fine with 0.9.1. Compilation for 0.9.2 is fine with maven. any issue in the gradle? Regards Tarkeshwar

Re: I want to bind the spout with an IP and want to configure this ip wherever spout goes(at the time of crash of spout)

you in putting the virtual ip on machine which has as > queue (kafka or jms) and the read messages from that kafka instance in your > spout. > > > On Fri, Sep 12, 2014 at 12:05 PM, M.Tarkeshwar Rao > wrote: > >> Hello Friends, >> >> I want to bind the s

I want to bind the spout with an IP and want to configure this ip wherever spout goes(at the time of crash of spout)

Hello Friends, I want to bind the spout with an IP and want to configure this IP wherever spout runs in the cluster. Actually external clients sending data on this IP. We want that if the spout is up and running in any machine then this IP should also configured on that machine and if that spout

Re: Can I make Nimbus failsafe

t;> major new feature. :) >> >> >> On Tue, Sep 9, 2014 at 5:35 PM, M.Tarkeshwar Rao >> wrote: >> >>> Hi all, >>> >>> How we can achieve high availability of nimbus. >>> Can you please help us to achieve this? >>&g

Can I make Nimbus failsafe

Hi all, How we can achieve high availability of nimbus. Can you please help us to achieve this? Regards Tarkeshwar

want to read new assignment on zookeeper

Hi all, I want to read new assignment by nimbus on zookeeper. Can i do it ? I have a requirement to perform some action on any change in assignment. As our spout has the standard virtual ip. external world push the data to this spout. I want that when the spout accidentally killed and spawned

I need to set watcher on the bolt/spout crash. is It possible

Hi all, I have a small doubt in storm. I am new in this technology. I want to set the watcher on the zookeeper znode about any failure of any bolt of a topology. Can i get the notification about that? Please help we need to develop a java application which will monitor the any bolt/spout executo

Any Link which can tell extensive details of storm grouping ?

Hello friends, I need some documentation on storm grouping and how it is internally working. Can you please help me? Regards tarkeshwar

I want to setup complete storm-hdfs setup on my local system.Is it possible?

Hi, We have a application in which we are using storm as execution engine and hdfs,hbase,redis as distributed db. I want to test complete application and debug on my local laptop. Can i do it? can i launch Storm UI locally? is it possible to launch nimbus,supervisonrs and all the workers on my l

I am getting this connection loss error repeatedly from zookeeper. can you please suggest me?

Hi all , I am getting this error repeatedly. can you please suggest me? org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss at com.netflix.curator.ConnectionState.getZooKeeper(ConnectionState.java:72) ~[curator-client-1.0.1.jar:na] at

I want to enhance the performance of the running topology

Hi all, I want to enhance the performance of the running topology. If i done all standard configuration and then if i want to improve the performance then i have to re-balance and schedule the topology of the basis of traffic flow on the bolts. As i new to storm. am I going in right way? I wa

how trident managing if max transaction id reached.

Hi all, how trident managing if max transaction id reached? is it recycle the transaction ids. regards Tarkeshwar

how many master batch controller will created for connected graph

Hi all, I have a small confusion. I am using triedent for my application. i have one flow where : spout1-->bolt1->bolt2>bolt3 spout2--->bolt1 spout3->bolt1 I have above topology where spout1,spout2 and spout3 is connected to bolt1. I want to

Unique transaction id on coordinators running across JVM

Hi All, I have a requirement to generate "Unique transaction id on Master batch coordinator which should be unique across JVM". We have done some modification on trident source code where we want to generate "Unique transaction id" for single transaction. Presently Master batch coordinator assig

Can i get the metrics(data) from storm cluster regarding traffic load

Hi All I want to get some metrics from the storm cluster regarding what is traffic load on each link. like in following topology: A is the spout and rest all are bolts. I want to know the current traffic on each link like (A to B or C to D) A->B->C--->D--->E B--

Re: Trident transactional topology stuck re-emitting batches with Netty, but running fine with ZMQ (was Re: Topology is stuck)

producible as it was under 0.9.1, but the bug > seems nonetheless still present. I'll reduce the number of Storm slots and > topology workers as per your insights, hopefully this might make it easier > to reproduce the bug with a simplified Trident topology. > > > On Tuesday, July

Get some help for adaptive sheduling

Hi All, I want to implement adaptive scheduling for our product. I used custom scheduler. also i go through following links. http://www.dis.uniroma1.it/~midlab/articoli/ABQ13storm.pdf http://www.orgs.ttu.edu/debs2013/presentations/DEBS13-Paper88-Querzoni.pdf I liked the idea of topology based s

Re: Trident transactional topology stuck re-emitting batches with Netty, but running fine with ZMQ (was Re: Topology is stuck)

y special > configs enabled? > > Thanks, > > Danijel > > > On Mon, Jul 7, 2014 at 9:01 AM, M.Tarkeshwar Rao > wrote: > >> Hi Danijel, >> >> We are able to reproduce this issue with 0.9.2 as well. >> We have two worker setup to run the trident topology. &

Re: Trident transactional topology stuck re-emitting batches with Netty, but running fine with ZMQ (was Re: Topology is stuck)

Hi Danijel, We are able to reproduce this issue with 0.9.2 as well. We have two worker setup to run the trident topology. When we kill one of the worker and again when that killed worker spawn on same port(same slot) then that worker not able to communicate with 2nd worker. only transaction atte

Re: java.lang.ClassNotFoundException: storm.kafka.BrokerHosts

Hi Josh, i have similar issue. not able to compile the 0.9.2.Any suggestions as i am getting following errors. I am getting following error message when i am upgrading my project to storm 0.9.2. we are using gradle. Actually it is using thrift 7.0. Gradle is not able to download this. can you ple

I am getting following error message when i am upgrading my project to storm 0.9.2.

Hi all, I am getting following error message when i am upgrading my project to storm 0.9.2. we are using gradle. Actually it is using thrift 7.0. Gradle is not able to download this. can you please suggest what should we do? *1) 1st we tried with following option in build.gradle:* maven

Re: storm netty hang issue

uce it under 0.9.2 -- I will > prepare a simple topology and file a bug report on the JIRA eventually. > > > On Tue, Jul 1, 2014 at 11:41 AM, M.Tarkeshwar Rao > wrote: > >> Yes. >> >> >> On Tue, Jul 1, 2014 at 2:53 PM, Danijel Schiavuzzi < >> dani..

Re: storm netty hang issue

Yes. On Tue, Jul 1, 2014 at 2:53 PM, Danijel Schiavuzzi wrote: > Is the topology a Trident transactional topology maybe? > On Jul 1, 2014 9:02 AM, "M.Tarkeshwar Rao" wrote: > >> Hi All, >> >> We are using apache storm 9.01. >> Our topology have

storm netty hang issue

Hi All, We are using apache storm 9.01. Our topology have two worker processes. If one process of the worker killed our topology getting hanged. Any inputs? Runs fine with zeroMQ. Regards Tarkeshwar

Re: Facing issue in deployment of a topology.

Yes it is apache 0.9.0.1.Is there any problem in this version? On Mon, Jun 30, 2014 at 7:26 PM, Naresh Kosgi wrote: > are you using the apache version? > > > On Mon, Jun 30, 2014 at 8:56 AM, M.Tarkeshwar Rao > wrote: > >> Hi, >> >> Thanks Naresh for your

Re: Facing issue in deployment of a topology.

kill 9 storm processes from time to time without issue. >> >> >> On Sat, Jun 28, 2014 at 10:57 AM, M.Tarkeshwar Rao < >> tarkeshwa...@gmail.com> wrote: >> >>> Yes we are killing it by kill -9.but we are devloping the new product.we >>> have t

Re: Facing issue in deployment of a topology.

ust kill one worker (i'm guessing > your using kill -9 to kill one of the workers java process) > > > On Sat, Jun 28, 2014 at 8:51 AM, M.Tarkeshwar Rao > wrote: > >> Hi all, >> >> I am facing a issue in a deployment of a topology.we are using trident. >>

Facing issue in deployment of a topology.

Hi all, I am facing a issue in a deployment of a topology.we are using trident. Actually our topology working fine with one worker. when I set the number of worker 2 it is working.but when I kill one of the worker process it is giving issue. > Sometimes it is working but sometimes the communicatio

Getting unexpected error, closing socket connection and attempting reconnect

Getting following error.Can you please suggest why? we are using zookeeper with trident for fault report. During deleting the znode we getting following exception. 2014-06-18 16:22:51 o.a.z.ClientCnxn [WARN] Session 0x146ae10a9f401bb for server bl460cx2378/10.61.251.98:2000, unexpected error,

Not able get all the messages from zookeeper

Hi all, We are facing a issue in reading the data from zookeeper. Actually whenever there is a failure in processing on bolt, storm writes it on zookeeper.(by raising reportedfailedexception) we are using trident with some code changes in trident code base. We have 7 tuple(filenames) batch. we ar

when the worker dies,upon restart the acking stops between bolts.

Hi All, We need solution of following issue urgently. Can anyone help us on this regard? For latest implementation of isReady for Co ordinator,we would need the MasterBatchController and MetaDataCreator to run on the same worker. Hence brings in the need of scheduler.(we are using trident)

Facing exception on raising failure (java.lang.RuntimeException: backtype.storm.topology.ReportedFailedException: :1:0:null:null:null)

Hi All, We are facing this issue when we are raising the failure on bolt. Initially raise failure is not giving any issue. But now it is giving following exception. Can you please help me in resolving this issue? java.lang.RuntimeException: backtype.storm.topology.ReportedFailedException: :1:

Re: Is storm support bidirectional communication

7, 2014 at 8:08 AM, M.Tarkeshwar Rao > wrote: > >> Hi, >> >> Is storm support bi-directional communication? >> I want to implement event based processing like http(*request >> processing response*). >> >> is it possible? if yes can you please sugges

Is storm support bidirectional communication

Hi, Is storm support bi-directional communication? I want to implement event based processing like http(*request processing response*). is it possible? if yes can you please suggest. regards Tarkeshwar

How to handle events (request response in storm)

Hi, How to handle events(Request-response)? regards Tarkeshwar

What should be the best way to implement logging framework in storm on particular topology

Hi All, What should be the best way to implement logging framework in storm on particular topology? Regards Tarkeshwar

need help on timeout

Hi, Taking Storm distributed processing framework as a reference: Following two problems we are facing. (1) Find the optimal time out of batch processing based on traffic load and provision to set timeout at runtime in storm cluster. Regards Tarkeshwar

facing a problem in scheduling.

Hi all, I am facing a problem in scheduling. we are using trident. Initially co-ordinator and emiter scheduled in one worker machine and other bolts in different machines of the cluster. When Master-Batch-controller of trident goes down or intentionally killed, nimbus bring the MBC up but we are

Re: storm scheduler behavior

Hi, I am facing a problem in scheduling. we are using trident. Initially co-ordinator and emiter scheduled in one worker machine and other bolts in different machines of the cluster. When Master-Batch-controller of trident goes down or intentionally killed, nimbus bring the MBC up but we are gett

Can i increase the parallelism of the topology as per traffic

Hi, Can i increase the parallelism of the topology as per traffic? I need inputs from you all. any link? (Hint: find the bolt stats through nimbus service and increase the bolt’s executor by firing rebalancing command after fixed interval if load increases) regards tarkesh

[no subject]

Hi, Can i increase the parallelism of the topology as per traffic? I need inputs from you all. any link? (Hint: find the bolt stats through nimbus service and increase the bolt’s executor by firing rebalancing command after fixed interval if load increases) regards tarkesh

Re: Trident Stream Join with Self

Hi all, Taking Storm distributed processing framework as a reference: Following two problems we are facing. (1) Find the optimal time out of batch processing based on traffic load and provision to set timeout at runtime in storm cluster. (2) Computes the optimal degree of parallelism for eac

Need help in finding the optimal timeout and in parallelism

Hi all, Taking Storm distributed processing framework as a reference: Following two problems we are facing. (1) Find the optimal time out of batch processing based on traffic load and provision to set timeout at runtime in storm cluster. (2) Computes the optimal degree of parallelism for eac

Re: How dynamic are spouts/bolts in a topology?

Hi All, Any one have idea about how reportError is working. Can you please share some links for storm internals and how ackers are working. Regards On Sat, Apr 12, 2014 at 6:44 AM, Srinath C wrote: > Hi Otis, >The type of spouts and bolts and the inter-connectivity between them is > fixe

Any one have idea about how reportError is working

Hi All, Any one have idea about how reportError is working. Can you please share some links for storm internals and how ackers are working. Regards Tarkeshwar