Hi guys,
After some profiling, I can found that some functions of scala library as
slices was consuming CPU as I was not expected. So I removed them by
another functions.
But I want to know scan function from ForkJoinPool is the top consumer of
my CPU during the hot period (see pictures).
As you can see on the pictures, I have some hot periods in which actors
make some stuff and during these period, I has no blocking call so it is a
normal behaviour ?
Le jeudi 28 septembre 2017 12:40:54 UTC+2, Kilic Ali-Firat a écrit :
>
> Hi,
>
> I will run the code in local and make thread/stack dump.
>
> I attached all the living threads of my worker.
>
>
> Le jeudi 28 septembre 2017 12:18:56 UTC+2, johannes...@lightbend.com a
> écrit :
>>
>>
>> Hi Kilic,
>>
>> Try looking at stack traces during the busy periods (e.g. use `jstack` on
>> the command line to gather some), that should give you a clue what's going
>> on. In the picture you sent in your first email there were actually only 8
>> regular pool threads. Are there times where more is going on?
>>
>> Johannes
>>
>> On Thursday, September 28, 2017 at 11:25:26 AM UTC+2, Kilic Ali-Firat
>> wrote:
>>>
>>> Hi,
>>>
>>> I know that I needs to be careful when I have blocking tasks but in the
>>> context of my app, I have no blocking tasks (or blocking calls).
>>>
>>> To give more details, my two nodes workers has a router (round robind
>>> group) with 10 workers actors each. All my workers are stateless and
>>> execute most of a time a job which is to read data from redis, put it in
>>> s3. So, in the worst case, I can have a 10 jobs in parallel by node. The
>>> thing is that my job are using Futures (reading in async way from redis,
>>> uploading in async way in s3) like below and reading the logs, it seems
>>> that sometimes, a worker executing a job can receive an another job to
>>> execute. This cause for a single worker to execute more than one job and
>>> use CPU resources.
>>>
>>> override def receive = {
>>> case jDump: JobDump =>
>>> this.logReceiveMsg(jDump)
>>> this.executeJobDump(jDump) pipeTo sender()
>>> }
>>>
>>> I'm doing some benchmarks to see how my cluster supports long running
>>> tests with a high number of read/write from redis /to s3. Monitoring my
>>> worker with jvisualvm, I see that my CPUs (during read / write) increase ~
>>> 100 % and then decrease at 0 % when no jobs is available. This is expected
>>> the behavior but among the time, my CPU seems to not support a such load
>>> (please see the screen that I attached to this answer). My test crashed
>>> when the CPU do not decrease as excepted.
>>>
>>> The reason why I want to tune my dispatcher is maybe with bounded thread
>>> pool size, I can also limit the number of jobs to execute in parallel
>>> without spawing workers already busy by a job.
>>>
>>> Le jeudi 28 septembre 2017 10:39:13 UTC+2, Konrad Malawski a écrit :
>>>>
>>>> This is now how a fork-join pool works basically.
>>>>
>>>> Please read the docs about managing blocking:
>>>>
>>>> https://doc.akka.io/docs/akka/2.5/scala/dispatchers.html#blocking-needs-careful-management
>>>>
>>>> Tuning the default dispatcher like you’re attempting to here would not
>>>> have a positive effect by the way.
>>>> Please simply separate blocking or other behaviours on a different
>>>> dispatcher, make it a thread-pool one etc.
>>>>
>>>> —
>>>> Konrad `kto.so` Malawski
>>>> Akka <http://akka.io> @ Lightbend <http://lightbend.com>
>>>>
>>>> On 28 September 2017 at 17:35:19, Kilic Ali-Firat (kilic.a...@gmail.com)
>>>> wrote:
>>>>
>>>> Scala version : 2.11.7
>>>> Akka version : 2.4.17
>>>>
>>>> -----
>>>>
>>>> Hi Akka Team,
>>>>
>>>> I'm writing an Akka application using Akka Cluster, Akka Actors and
>>>> Akka Stream. I have a cluster of 3 nodes : 2 workers and 1 node consuming
>>>> data from a source.
>>>>
>>>> I want three configs files : application.conf, node.worker.conf and
>>>> node.source.conf. I limit the scope of this message to application.conf
>>>> and
>>>> node.worker.conf.
>>>>
>>>> Below you can find the content of node.worker.conf and application.conf
>>>> :
>>>>
>>>> akka.cluster.roles = [workers]
>>>>
>>>>
>>>> akka {
>>>> loglevel = "INFO"
>>>> loglevel = ${?FGS_AKKA_LOG_LEVEL}
>>>>
>>>> actor {
>>>> provider = "akka.cluster.ClusterActorRefProvider"
>>>>
>>>> default-dispatcher {
>>>> # This will be used if you have set "executor =
>>>> "fork-join-executor""
>>>> fork-join-executor {
>>>> # Min number of threads to cap factor-based parallelism number
>>>> to
>>>> parallelism-min = 1
>>>>
>>>> # The parallelism factor is used to determine thread pool size
>>>> using the
>>>> # following formula: ceil(available processors * factor).
>>>> Resulting size
>>>> # is then bounded by the parallelism-min and parallelism-max
>>>> values.
>>>> parallelism-factor = 1
>>>>
>>>> # Max number of threads to cap factor-based parallelism number
>>>> to
>>>> parallelism-max = 8
>>>> }
>>>> throughput = 1024
>>>> }
>>>>
>>>> }
>>>>
>>>>
>>>> remote {
>>>> enabled-transports = ["akka.remote.netty.tcp"]
>>>> netty.tcp {
>>>> hostname = ${clustering.node-ip}
>>>> port = ${clustering.node-port}
>>>>
>>>>
>>>>
>>>>
>>>> bind-hostname = "0.0.0.0"
>>>> bind-port = ${clustering.node-port}
>>>> }
>>>> }
>>>>
>>>>
>>>> cluster {
>>>> seed-nodes =
>>>> [
>>>> "akka.tcp://"${clustering.cluster.name}"@"${clustering.fst-seed
>>>> -node-ip}":"${clustering.fst-seed-node-port}
>>>> "akka.tcp://"${clustering.cluster.name}"@"${clustering.snd-seed
>>>> -node-ip}":"${clustering.snd-seed-node-port}
>>>> ]
>>>> }
>>>> }
>>>>
>>>>
>>>> clustering {
>>>> # Choose the same need port for the seed nodes. Seed nodes must be on
>>>> # different machines.
>>>> fst-seed-node-port = 2551
>>>> fst-seed-node-port = ${?FGS_FIRST_SEED_NODE_PORT}
>>>>
>>>> snd-seed-node-port = 2552
>>>> snd-seed-node-port = ${?FGS_SECOND_SEED_NODE_PORT}
>>>>
>>>> fst-seed-node-ip="localhost"
>>>> fst-seed-node-ip=${?FGS_FIRST_SEED_NODE_IP}
>>>>
>>>> snd-seed-node-ip="localhost"
>>>> snd-seed-node-ip=${?FGS_SECOND_SEED_NODE_IP}
>>>>
>>>>
>>>> // Must be the public IP
>>>> node-ip = ${?FGS_CLUSTER_NODE_IP}
>>>> node-port = ${?FGS_CLUSTER_NODE_PORT}
>>>> cluster.name = "FGS"
>>>> }
>>>>
>>>>
>>>> I tried to tune some settings of my default-dispatcher : I want to
>>>> limit the number of threads in the thread pool to 8. My laptop has 8
>>>> cores,
>>>> this is why I put the parallelism-factor to 1.0
>>>>
>>>> But looking with jvisualm the number allocated threads, I got more than
>>>> expected (I attached a screenshot). When I expected to have only 8
>>>> threads,
>>>> I get it 21.
>>>>
>>>> This is how I create my ActorSystem :
>>>>
>>>> val akkaConfig = ConfigFactory.load()
>>>>
>>>>
>>>> val actorSystemConfig =
>>>> (ConfigFactory parseResources "node.worker.conf")
>>>> .withFallback(akkaConfig)
>>>> .resolve()
>>>>
>>>>
>>>>
>>>>
>>>> lazy val actorSystemName = akkaConfig getString clusterNamePath
>>>>
>>>>
>>>> implicit val system = ActorSystem(actorSystemName,
>>>> actorSystemConfig)
>>>>
>>>> Am I wrong when creating my Actor System ?
>>>> --
>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>>>> >>>>>>>>>> Check the FAQ:
>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>> >>>>>>>>>> Search the archives:
>>>> https://groups.google.com/group/akka-user
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Akka User List" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to akka-user+...@googlegroups.com.
>>>> To post to this group, send email to akka...@googlegroups.com.
>>>> Visit this group at https://groups.google.com/group/akka-user.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
