Re: Can you help on this about performance of topology

Hi, For any kin of performance analysis, first determine the following 1. How many topologies? 2. What are the workload parameters (e.g. tuple arrival rate, number of executors per component, size of each tuple) 3. System environment ( Number of cores in each supervisor node, Available RAM) 4.

Number of topologies used in a mid sized company

Hi, Can any of you point me to a web link where I can get an idea about on average how many topologies are used in a mid sized company? Thanking you in advance -- Rudraneel Chakraborty Carleton University Real Time and Distributed Systems Research

Re: Worker process start time

to connect > to them. > Regards, > Florin > > On Fri, Jun 24, 2016 at 4:50 PM, Rudraneel chakraborty < > rudraneel.chakrabo...@gmail.com > <javascript:_e(%7B%7D,'cvml','rudraneel.chakrabo...@gmail.com');>> wrote: > >> My storm configuration is v1 3 nodes each wit

Re: Worker process start time

single worker process with 6 executors with no ackers). Do you >> guys find it normal? >> >> Can i speed things up?? >> >> >> -- >> Rudraneel Chakraborty >> Carleton University Real Time and Distributed Systems Reserach >> >> -- Rudraneel Chakraborty Carleton University Real Time and Distributed Systems Reserach

Worker process start time

Hi I am observing that my worker processes take a little over 15 seconds to start up (a single worker process with 6 executors with no ackers). Do you guys find it normal? Can i speed things up?? -- Rudraneel Chakraborty Carleton University Real Time and Distributed Systems Reserach

Re: Topology startup time

the many topologies > that I run. I’m sure there is room for improvement, but your experience > doesn’t seem like an outlier. > > -Aaron > > From: Rudraneel chakraborty <rudraneel.chakrabo...@gmail.com > <javascript:_e(%7B%7D,'cvml','rudraneel.chakrabo...@gmail.com');&

Topology startup time

Hello Whenever I try to resubmit a topology, it takes on an average 15 seconds to go completely functional. Is it normal? Is there any way to reduce this startup time -- Rudraneel Chakraborty Carleton University Real Time and Distributed Systems Reserach

Re: Difference between an assignable and an available slot

to my question is yes. Also, in the > documentation jargon, a "slot" translates to a worker process, right? > > Thank you again! > > Cheers, > Nick > > On Wed, Mar 9, 2016 at 4:04 PM, Rudraneel chakraborty < > rudraneel.chakrabo...@gmail.com > <javascript

Re: Difference between an assignable and an available slot

o, if an executor thread is assigned to a WorkerSlot returned by > getAvailableSlots(), can another executor thread be assigned to the same > WorkerSlot returned by the getAssignableSlots()? > > Thanks, > Nick > -- Rudraneel Chakraborty Carleton University Real Time and Distributed Systems Reserach

Re: Need Help regarding topology with numWorker>1

om multiple workers from a > performance point of view. > > Btw: I am a not sure what you mean by "I have to create a scenario..."? > > On 02/12/2016 11:42 PM, Rudraneel chakraborty wrote: > > Thanks. Actually i have to create a scenario where 2 worker performs > > be

Changing the default queue size of worker

-internal-message-buffers/ I need to change the default value of topology.receiver.buffer.size , but this doesnt work. Any help regarding this will be highly appreciated -- Rudraneel Chakraborty Carleton University Real Time and Distributed Systems Reserach

Re: Need Help regarding topology with numWorker>1

; > On 02/11/2016 05:14 AM, anshu shukla wrote: > > Not like that.. But i have used workers equal to number of cores. Each > > vm with 8 corea. > > > > On 11 Feb 2016 9:07 am, "Rudraneel chakraborty" > > <rudraneel.chakrabo...@gmail.com <javascript:;

Re: Need Help regarding topology with numWorker>1

; > On 02/11/2016 05:14 AM, anshu shukla wrote: > > Not like that.. But i have used workers equal to number of cores. Each > > vm with 8 corea. > > > > On 11 Feb 2016 9:07 am, "Rudraneel chakraborty" > > <rudraneel.chakrabo...@gmail.com <javascript:;

Re: Need Help regarding topology with numWorker>1

rk too. > > Any situation where you need to have tolerance in case of machine failure. > > > > On Thu, Feb 11, 2016 at 2:58 PM, Rudraneel chakraborty < > rudraneel.chakrabo...@gmail.com > <javascript:_e(%7B%7D,'cvml','rudraneel.chakrabo...@gmail.com');>

Re: Need Help regarding topology with numWorker>1

Topology param numWorkers i meant On Thursday, 11 February 2016, Rudraneel chakraborty < rudraneel.chakrabo...@gmail.com> wrote: > More specifically , i have seen a topology performs better if it is > assigned a single worker compared to more than one worker. > > I want

Re: Need Help regarding topology with numWorker>1

/2016 05:14 AM, anshu shukla wrote: > > Not like that.. But i have used workers equal to number of cores. Each > > vm with 8 corea. > > > > On 11 Feb 2016 9:07 am, "Rudraneel chakraborty" > > <rudraneel.chakrabo...@gmail.com <javascript:;> > > <m

Number of workers

Hi, I have developed a topology to parse normal single line csv lines from a kafka cluster with 2 brokers. I have 3 zookeeper nodes. and I have a 3 node storm cluster with 2 being supervisor nodes and one being the nimbus node. My observation is , the topology performs better in terms of latency

Need Help regarding topology with numWorker>1

Hello Good People, I desperately need an example of a topology which performs better with more than one worker process compared to a single worker. Could anyone help

Re: Need Help regarding topology with numWorker>1

more than one worker on same node ? Did u use custom schedule r? because by default, the workers would be spread throughout the cluster

Re: Storm Scheduler

> > find an good tutorial here: > > https://xumingming.sinaapp.com/885/twitter-storm-how-to-develop-a-pluggable-scheduler/ > > Btw: the scheduler is called periodically. > > -Matthias > > > On 12/09/2015 12:54 AM, Rudraneel chakraborty wrote: > > Hello , > &g

Storm Scheduler

Hello , I am trying to develop a custom storm scheduler for some specific needs. I wanted to know if one can control the calling of the scheduler ( or, if it is called periodically , how to control the frequency ) -- Rudraneel Chakraborty

Re: Multiple Spouts in Same topology or Topology per spout

t;>>> > >>>> > 2) Use different Spout (Spout A and Spout B and so on) within the >>>> same topology (say Topology A) to read messages from different queues . >>>> > >>>> > 3) Use Different Spouts one within eachTopology (Topology A

Help Required

topologies? -- Rudraneel Chakraborty Carleton University Real Time and Distributed Systems Reserach