Thank you
And i need this because i am developing a custom scheduler . And for this i
need to have topologies performing well with two worker processes instead
of one. And its becoming tough to saturate a single worker
On Saturday, 13 February 2016, Matthias J. Sax wrote:
> If your workload does
If your workload does not saturate a single machine, it will of course
be more efficient to run within a single worker as you avoid intra-JVM
communication.
As long as CPU, memory, or network of a single machine is not utilized
completely, you will not benefit from multiple workers from a
performa
Thanks for the reply. So after some thinking , i figured if i shorten the
size of the worker incoming queue , i can create the scenario where using
more than one worker might result in better performance. Any thoughts on
how to do that?
The property topology, receiver.buffer.size should be the the
Thanks for the reply. So after some thinking , i figured if i shorten the
size of the worker incoming queue , i can create the scenario where using
more than one worker might result in better performance. Any thoughts on
how to do that?
The property topology, receiver.buffer.size should be the the
Thanks. Actually i have to create a scenario where 2 worker performs better
than one worker. But in reality , topology with a single worker performs
considerably better.
I sending csv lines to kafka (5 partitions ) and reading them from a
topology with kafka spout (parallelism hint 5)
Any thought
Any situation where you require more CPU than 1 server can provide for you
- there are tuning parameters (e.g. localOrShuffleGrouping) that you can
use to reduce the amount of data sent over the network too.
Any situation where you need to have tolerance in case of machine failure.
On Thu, Feb
Topology param numWorkers i meant
On Thursday, 11 February 2016, Rudraneel chakraborty <
rudraneel.chakrabo...@gmail.com> wrote:
> More specifically , i have seen a topology performs better if it is
> assigned a single worker compared to more than one worker.
>
> I want a situation where a topolo
More specifically , i have seen a topology performs better if it is
assigned a single worker compared to more than one worker.
I want a situation where a topology performs better with more than one
worker.
And it doesnt matter if both workers are on same supervisor or different
supervisor
On Thu
I am not sure what you mean:
- number of worker slots per supervisor
or
- topology parameter "number of workers"
Can you clarify?
-Matthias
On 02/11/2016 05:14 AM, anshu shukla wrote:
> Not like that.. But i have used workers equal to number of cores. Each
> vm with 8 corea.
>
> On 11 Fe
Not like that.. But i have used workers equal to number of cores. Each vm
with 8 corea.
On 11 Feb 2016 9:07 am, "Rudraneel chakraborty" <
rudraneel.chakrabo...@gmail.com> wrote:
> more than one worker on same node ? Did u use custom schedule r? because
> by default, the workers would be spread t
more than one worker on same node ? Did u use custom schedule r? because by
default, the workers would be spread throughout the cluster
Since every worker process have only one thread for transfer of message
from network to executor queue so more often it becomes a bottkeneck when
input rate is high.. That forces us to have more than 1 worker on same
node.
I dont think other than this there is any logic dependent topo case whe
Hello Good People,
I desperately need an example of a topology which performs better with more
than one worker process compared to a single worker. Could anyone help
13 matches
Mail list logo
