Hi,
I know that I needs to be careful when I have blocking tasks but in the
context of my app, I have no blocking tasks (or blocking calls).
To give more details, my two nodes workers has a router (round robind
group) with 10 workers actors each. All my workers are stateless and
execute most of a time a job which is to read data from redis, put it in
s3. So, in the worst case, I can have a 10 jobs in parallel by node. The
thing is that my job are using Futures (reading in async way from redis,
uploading in async way in s3) like below and reading the logs, it seems
that sometimes, a worker executing a job can receive an another job to
execute. This cause for a single worker to execute more than one job and
use CPU resources.
override def receive = {
case jDump: JobDump =>
this.logReceiveMsg(jDump)
this.executeJobDump(jDump) pipeTo sender()
}
I'm doing some benchmarks to see how my cluster supports long running tests
with a high number of read/write from redis /to s3. Monitoring my worker
with jvisualvm, I see that my CPUs (during read / write) increase ~ 100 %
and then decrease at 0 % when no jobs is available. This is expected the
behavior but among the time, my CPU seems to not support a such load
(please see the screen that I attached to this answer). My test crashed
when the CPU do not decrease as excepted.
The reason why I want to tune my dispatcher is maybe with bounded thread
pool size, I can also limit the number of jobs to execute in parallel
without spawing workers already busy by a job.
Le jeudi 28 septembre 2017 10:39:13 UTC+2, Konrad Malawski a écrit :
>
> This is now how a fork-join pool works basically.
>
> Please read the docs about managing blocking:
>
> https://doc.akka.io/docs/akka/2.5/scala/dispatchers.html#blocking-needs-careful-management
>
> Tuning the default dispatcher like you’re attempting to here would not
> have a positive effect by the way.
> Please simply separate blocking or other behaviours on a different
> dispatcher, make it a thread-pool one etc.
>
> —
> Konrad `kto.so` Malawski
> Akka <http://akka.io> @ Lightbend <http://lightbend.com>
>
> On 28 September 2017 at 17:35:19, Kilic Ali-Firat (kilic.a...@gmail.com
> <javascript:>) wrote:
>
> Scala version : 2.11.7
> Akka version : 2.4.17
>
> -----
>
> Hi Akka Team,
>
> I'm writing an Akka application using Akka Cluster, Akka Actors and Akka
> Stream. I have a cluster of 3 nodes : 2 workers and 1 node consuming data
> from a source.
>
> I want three configs files : application.conf, node.worker.conf and
> node.source.conf. I limit the scope of this message to application.conf and
> node.worker.conf.
>
> Below you can find the content of node.worker.conf and application.conf :
>
> akka.cluster.roles = [workers]
>
>
> akka {
> loglevel = "INFO"
> loglevel = ${?FGS_AKKA_LOG_LEVEL}
>
> actor {
> provider = "akka.cluster.ClusterActorRefProvider"
>
> default-dispatcher {
> # This will be used if you have set "executor =
> "fork-join-executor""
> fork-join-executor {
> # Min number of threads to cap factor-based parallelism number to
> parallelism-min = 1
>
> # The parallelism factor is used to determine thread pool size
> using the
> # following formula: ceil(available processors * factor).
> Resulting size
> # is then bounded by the parallelism-min and parallelism-max
> values.
> parallelism-factor = 1
>
> # Max number of threads to cap factor-based parallelism number to
> parallelism-max = 8
> }
> throughput = 1024
> }
>
> }
>
>
> remote {
> enabled-transports = ["akka.remote.netty.tcp"]
> netty.tcp {
> hostname = ${clustering.node-ip}
> port = ${clustering.node-port}
>
>
>
>
> bind-hostname = "0.0.0.0"
> bind-port = ${clustering.node-port}
> }
> }
>
>
> cluster {
> seed-nodes =
> [
> "akka.tcp://"${clustering.cluster.name}"@"${clustering.fst-seed-
> node-ip}":"${clustering.fst-seed-node-port}
> "akka.tcp://"${clustering.cluster.name}"@"${clustering.snd-seed-
> node-ip}":"${clustering.snd-seed-node-port}
> ]
> }
> }
>
>
> clustering {
> # Choose the same need port for the seed nodes. Seed nodes must be on
> # different machines.
> fst-seed-node-port = 2551
> fst-seed-node-port = ${?FGS_FIRST_SEED_NODE_PORT}
>
> snd-seed-node-port = 2552
> snd-seed-node-port = ${?FGS_SECOND_SEED_NODE_PORT}
>
> fst-seed-node-ip="localhost"
> fst-seed-node-ip=${?FGS_FIRST_SEED_NODE_IP}
>
> snd-seed-node-ip="localhost"
> snd-seed-node-ip=${?FGS_SECOND_SEED_NODE_IP}
>
>
> // Must be the public IP
> node-ip = ${?FGS_CLUSTER_NODE_IP}
> node-port = ${?FGS_CLUSTER_NODE_PORT}
> cluster.name = "FGS"
> }
>
>
> I tried to tune some settings of my default-dispatcher : I want to limit
> the number of threads in the thread pool to 8. My laptop has 8 cores, this
> is why I put the parallelism-factor to 1.0
>
> But looking with jvisualm the number allocated threads, I got more than
> expected (I attached a screenshot). When I expected to have only 8 threads,
> I get it 21.
>
> This is how I create my ActorSystem :
>
> val akkaConfig = ConfigFactory.load()
>
>
> val actorSystemConfig =
> (ConfigFactory parseResources "node.worker.conf")
> .withFallback(akkaConfig)
> .resolve()
>
>
>
>
> lazy val actorSystemName = akkaConfig getString clusterNamePath
>
>
> implicit val system = ActorSystem(actorSystemName, actorSystemConfig)
>
> Am I wrong when creating my Actor System ?
> --
> >>>>>>>>>> Read the docs: http://akka.io/docs/
> >>>>>>>>>> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+...@googlegroups.com <javascript:>.
> To post to this group, send email to akka...@googlegroups.com
> <javascript:>.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
