[
https://issues.apache.org/jira/browse/SAMZA-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Faris updated SAMZA-2779:
------------------------------
Description:
The updateOperatingSystemMetrics method is intended to report CPU usage for the
container but it's reporting for the entire host.
[https://github.com/apache/samza/blob/e37fe259cede587928cc0d543fc8384250c7e113/samza-core/src/main/scala/org/apache/samza/metrics/JvmMetrics.scala#L169]
{noformat}
gProcessCpuUsageProcessors.set(processCpuLoad *
operatingSystemMXBean.getAvailableProcessors)
gSystemCpuUsage.set(operatingSystemMXBean.getSystemCpuLoad * PCT)
{noformat}
* Java docs for
operatingSystemMXBean.[getAvailableProcessors|https://docs.oracle.com/javase/8/docs/api/java/lang/management/OperatingSystemMXBean.html?is-external=true#getAvailableProcessors%E2%80%93]
returns the number of processors available to the virtual machine
So what does this mean to Samza Metrics?
{quote}gProcessCpuUsageProcessors.set(processCpuLoad *
operatingSystemMXBean.getAvailableProcessors)
{quote}
The multiplier will change based on the number of processors a host will have.
Some hosts may have 24 cores, others 40. As the container moves form hostA to
hostB, this value will lose meaning.
Secondly it's not clear what ProcessCpuUsageProcessors represents. Why would
it be multiplied by the CPU count?
Here's a simple example class to demonstrate the behavior.
{code:java}
import java.lang.management.ManagementFactory;
import com.sun.management.OperatingSystemMXBean;
public class Count {
public static void main(String[] args) {
OperatingSystemMXBean osMXBean = (OperatingSystemMXBean)
ManagementFactory.getOperatingSystemMXBean();
int processorCount = osMXBean.getAvailableProcessors();
System.out.printf("Processor Count: %d\n", processorCount);
}
}
{code}
was:
The updateOperatingSystemMetrics method is intended to report CPU usage for the
container but it's reporting for the entire host.
[https://github.com/apache/samza/blob/e37fe259cede587928cc0d543fc8384250c7e113/samza-core/src/main/scala/org/apache/samza/metrics/JvmMetrics.scala#L169]
{noformat}
gProcessCpuUsageProcessors.set(processCpuLoad *
operatingSystemMXBean.getAvailableProcessors)
gSystemCpuUsage.set(operatingSystemMXBean.getSystemCpuLoad * PCT)
{noformat}
* Java docs for
operatingSystemMXBean.[getAvailableProcessors|https://docs.oracle.com/javase/8/docs/api/java/lang/management/OperatingSystemMXBean.html?is-external=true#getAvailableProcessors%E2%80%93]
returns the number of processors available to the virtual machine
So what does this mean to Samza Metrics?
{quote}gProcessCpuUsageProcessors.set(processCpuLoad *
operatingSystemMXBean.getAvailableProcessors)
{quote}
The multiplier will change based on the number of processors a host will have.
Some hosts may have 24 cores, others 40. As the container moves form hostA to
hostB, this value will lose meaning.
Secondly it's not clear what ProcessCpuUsageProcessors represents. Why would
it be multiplied by the CPU count?
Here's a simple example class to demonstrate the behavior.
{code:java}
import java.lang.management.ManagementFactory;
import com.sun.management.OperatingSystemMXBean;
public class Count {
public static void main(String[] args) {
OperatingSystemMXBean osMXBean = (OperatingSystemMXBean)
ManagementFactory.getOperatingSystemMXBean();
int processorCount = osMXBean.getAvailableProcessors();
System.out.printf("Processor Count: %d\n", processorCount);
}
}
{code}
> Container CPU metrics inconsistent.
> -----------------------------------
>
> Key: SAMZA-2779
> URL: https://issues.apache.org/jira/browse/SAMZA-2779
> Project: Samza
> Issue Type: Bug
> Reporter: Adam Faris
> Priority: Major
>
> The updateOperatingSystemMetrics method is intended to report CPU usage for
> the container but it's reporting for the entire host.
> [https://github.com/apache/samza/blob/e37fe259cede587928cc0d543fc8384250c7e113/samza-core/src/main/scala/org/apache/samza/metrics/JvmMetrics.scala#L169]
> {noformat}
> gProcessCpuUsageProcessors.set(processCpuLoad *
> operatingSystemMXBean.getAvailableProcessors)
> gSystemCpuUsage.set(operatingSystemMXBean.getSystemCpuLoad * PCT)
> {noformat}
> * Java docs for
> operatingSystemMXBean.[getAvailableProcessors|https://docs.oracle.com/javase/8/docs/api/java/lang/management/OperatingSystemMXBean.html?is-external=true#getAvailableProcessors%E2%80%93]
> returns the number of processors available to the virtual machine
> So what does this mean to Samza Metrics?
> {quote}gProcessCpuUsageProcessors.set(processCpuLoad *
> operatingSystemMXBean.getAvailableProcessors)
> {quote}
> The multiplier will change based on the number of processors a host will
> have. Some hosts may have 24 cores, others 40. As the container moves form
> hostA to hostB, this value will lose meaning.
> Secondly it's not clear what ProcessCpuUsageProcessors represents. Why would
> it be multiplied by the CPU count?
> Here's a simple example class to demonstrate the behavior.
> {code:java}
> import java.lang.management.ManagementFactory;
> import com.sun.management.OperatingSystemMXBean;
> public class Count {
> public static void main(String[] args) {
> OperatingSystemMXBean osMXBean = (OperatingSystemMXBean)
> ManagementFactory.getOperatingSystemMXBean();
> int processorCount = osMXBean.getAvailableProcessors();
> System.out.printf("Processor Count: %d\n", processorCount);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
