Re: [akka-user] CPU spikes using fork-join-executor

2015-04-19 Thread Akara Sucharitakul
Thank you very much, everybody. Although we don't have the right tools to gather concrete evidence supporting this explanation, I agree it is the most likely cause. And thank you very much for filing the issue. The fact that we're on a VM and have double-layer scheduling (one by the hypervisor

Re: [akka-user] CPU spikes using fork-join-executor

2015-04-17 Thread Viktor Klang
I'd bet on this explanation too. Having looked at the code I think it should be possible to switch implementations to something without spinlocking. On Fri, Apr 17, 2015 at 8:20 AM, Roland Kuhn wrote: > Hi Akara, > > thanks for all the detailed information, I am 95% certain that I know what > h

Re: [akka-user] CPU spikes using fork-join-executor

2015-04-16 Thread Endre Varga
On Fri, Apr 17, 2015 at 8:20 AM, Roland Kuhn wrote: > Hi Akara, > > thanks for all the detailed information, I am 95% certain that I know what > happens—and I’d call it a bug. To the casual observer the code in > DirectByteBufferPool looks like a CAS loop, but it actually is an > open-coded primi

Re: [akka-user] CPU spikes using fork-join-executor

2015-04-16 Thread Roland Kuhn
Hi Akara, thanks for all the detailed information, I am 95% certain that I know what happens—and I’d call it a bug. To the casual observer the code in DirectByteBufferPool looks like a CAS loop, but it actually is an open-coded primitive spinlock. If you have more threads competing for this loc

Re: [akka-user] CPU spikes using fork-join-executor

2015-04-16 Thread Endre Varga
Hi Akara, The ForkJoinPool is a high-throughput executor pool and therefore has been designed with the assumption that most of the time threads will be busy. If you see the "scan()" method popping up as using the most CPU time then it means that your threads are mostly idle. This is quite likely s

Re: [akka-user] CPU spikes using fork-join-executor

2015-04-16 Thread Akara Sucharitakul
Nope, this is actually Ubuntu 10.10 on a 4vCPU VM (likely ESX but I can't tell for sure). Couple more pieces of information: - Further experiments with parallelism-factor lowered to 1.0 no longer shows the spikes. - Trying to profile the spikes with JMC would cause the spikes not to

Re: [akka-user] CPU spikes using fork-join-executor

2015-04-16 Thread Martynas Mickevičius
Hi, are you running this application on Windows? There is a known situation where high CPU usage might occur when using TCP with akka IO. On Tue, Apr 14, 2015 at 7:28 AM, Akara Sucharitakul < akara.sucharita...@gmail.com> wrote: > Scala 2.11.6, Akka 2.

[akka-user] CPU spikes using fork-join-executor

2015-04-13 Thread Akara Sucharitakul
Scala 2.11.6, Akka 2.3.9, Spray 1.3.2. When testing a particular workload, we ran into CPU spikes. The avg CPU load is less than 5% but about 6-7 minutes into the run, we start to see CPU spiking to near 100% lasting for several seconds. This repeats itself every 6-7 minutes. We can't correlate