[ https://issues.apache.org/jira/browse/AURORA-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095046#comment-16095046 ]
Reza Motamedi commented on AURORA-1939: --------------------------------------- Added more logging to check the returned value for a process.cpu_times(). The numbers simply don't add up. What I see the same pid sampled by psutil showing a different name and CPU samples that fluctuate (decrease). Note the in this example pid:14712 was created on July 16th, and the log is from July 20th. So there is no chance that what we see in the log is because pid reused. {noformat} ~ $ grep 'pid=14712' thermos-observer.DEBUG D0720 17:01:04.148628 61755 process_collector_psutil.py:35] process:psutil.Process(pid=14712, name='mesos-slave') cpu times pcputimes(user=603.88, system=1104.31, children_user=0.0, children_system=0.0) D0720 17:06:02.358989 61755 process_collector_psutil.py:35] process:psutil.Process(pid=14712, name='mesos-slave') cpu times pcputimes(user=591.16, system=1097.5, children_user=0.0, children_system=0.0) D0720 17:10:55.258080 61755 process_collector_psutil.py:35] process:psutil.Process(pid=14712, name='python2.7') cpu times pcputimes(user=44.81, system=7.29, children_user=0.0, children_system=0.0) D0720 17:16:23.156296 61755 process_collector_psutil.py:35] process:psutil.Process(pid=14712, name='python2.7') cpu times pcputimes(user=596.01, system=1104.11, children_user=0.0, children_system=0.0) D0720 17:21:21.552978 61755 process_collector_psutil.py:35] process:psutil.Process(pid=14712, name='python2.7') cpu times pcputimes(user=44.9, system=7.3, children_user=0.0, children_system=0.0) ~ $ ps -o lstart= -p 14712 Sun Jul 16 11:39:19 2017 {noformat} > Thermos landing (host) page reports incorrect CPU rates when it is busy > ----------------------------------------------------------------------- > > Key: AURORA-1939 > URL: https://issues.apache.org/jira/browse/AURORA-1939 > Project: Aurora > Issue Type: Bug > Reporter: Reza Motamedi > Priority: Minor > > Thermos Observer uses `psutil` to monitor resource consumption of Thermos > Processes. On a busy machine, I have noticed negative CPU values when > visiting the Thermos landing page. > In my test I reproduced this by starting many processes that constantly > create short lived children. This indicates that in time between > `process_collector_psutil` looks up the Process children and the time it > calculates the CPU time the pid of the child is actually reused by another > much younger process, which leads to negative CPU times. -- This message was sent by Atlassian JIRA (v6.4.14#64029)