Re: [ClusterLabs] pcsd processes using 100% CPU

2018-05-24 Thread Casey & Gina
> gcore is part of gdb: > https://packages.ubuntu.com/xenial/amd64/gdb/filelist > > Note that using the utility should have no observable influence > on the running process in question. When I ran gcore on the pid, it produced a whole bunch of memory read errors like this: warning: Memory read

Re: [ClusterLabs] pcsd processes using 100% CPU

2018-05-24 Thread Jan Pokorný
On 23/05/18 12:43 -0600, Casey & Gina wrote: > I don't have gcore installed and don't know which package might > provide it. I also don't have experience with gdb but am happy to > try anything suggested to help figure out what's going on. gcore is part of gdb:

Re: [ClusterLabs] pcsd processes using 100% CPU

2018-05-23 Thread Casey & Gina
Okay, I have this happening again on a couple servers right now, and am happy to let it spin and dig more into it. I'm not at all experienced with stuff like this though, so will need some explicit instruction on what to do beyond what I've documented here... I don't see anything of note in

Re: [ClusterLabs] pcsd processes using 100% CPU

2018-05-23 Thread Tomas Jelinek
Dne 22.5.2018 v 18:47 Shobe, Casey napsal(a): Not really enough info to debug... And I don't think I encountered this myself. Is there something more I can do to gather more information when this happens? I'm not very familiar with strace, just ran it on the PID and saw the screen fill up

Re: [ClusterLabs] pcsd processes using 100% CPU

2018-05-22 Thread Shobe, Casey
> Not really enough info to debug... And I don't think I encountered this > myself. Is there something more I can do to gather more information when this happens? I'm not very familiar with strace, just ran it on the PID and saw the screen fill up with sched_yield() lines... This happens

Re: [ClusterLabs] pcsd processes using 100% CPU

2018-05-22 Thread Casey & Gina
> Can you share some HW specs with us, at least the architecture > to start with -- x86_64=amd64, arm (gen/mode?), something else? It's x86_64, running Ubuntu 16.04; the latest package versions available from Ubuntu repositories. They are vmWare ESX nodes with 16 CPU cores and 64GB of memory

Re: [ClusterLabs] pcsd processes using 100% CPU

2018-05-22 Thread Jan Pokorný
On 18/05/18 20:04 +, Shobe, Casey wrote: > On a couple clusters that have been running for a little while > (without fencing), I'm seeing runaway server.rb processes using 100% > of a single CPU core each. > > When I look at ps, I can see that these have something to do with > pcsd: > > USER

Re: [ClusterLabs] pcsd processes using 100% CPU

2018-05-21 Thread Tomas Jelinek
Dne 18.5.2018 v 22:04 Shobe, Casey napsal(a): On a couple clusters that have been running for a little while (without fencing), I'm seeing runaway server.rb processes using 100% of a single CPU core each. When I look at ps, I can see that these have something to do with pcsd: USER PID

[ClusterLabs] pcsd processes using 100% CPU

2018-05-18 Thread Shobe, Casey
On a couple clusters that have been running for a little while (without fencing), I'm seeing runaway server.rb processes using 100% of a single CPU core each. When I look at ps, I can see that these have something to do with pcsd: USER PID %CPU %MEMVSZ RSS TTY STAT START