Re: [libvirt] RFC: add Intel CMT feature support
-Original Message-
From: Peter Krempa [mailto:pkre...@redhat.com]
Sent: Monday, June 29, 2015 5:44 PM
To: Ren, Qiaowei
Cc: libvir-list@redhat.com
Subject: Re: [libvirt] RFC: add Intel CMT feature support
On Sat, Jun 27, 2015 at 03:45:21 +, Ren, Qiaowei wrote:
Hi All,
Some Intel processor families (e.g. the Intel Xeon processor E5 v3 family)
introduced some PQos (Platform Qos) features to monitor or control shared
resource.
- CMT (Cache Monitoring Technology): measure the usage of cache by
applications running on the platform.
- CAT (Cache Allocation Technology): enable an OS or Hypervisor/VMM to
specify the amount of cache space into which an application can fill.
- MBM (Memory Bandwidth Monitoring): build on the CMT infrastructure to
allow monitoring of bandwidth from one level of the cache hierarchy to the
next
- in this case focusing on the L3 cache, which is typically backed directly by
system memory. As a result of this implementation, memory bandwidth can be
monitored.
For more information, see Intel 64 and IA-32 Architectures Software
Developer's Manual.
Among these PQos features, currently CMT patches has been merged into
Linux kernel mainline. So this patch series proposed in this mail will focus
on
CMT feature support in libvirt.
At the library API layer, I plan on adding cache related field into
virDomainInfo
and virNodeInfo:
Add cache member into virNodeInfo to get total size of cache in one node.
struct virNodeInfo {
...
unsigned int cores; /* number of cores per socket, total number of
processors in case of unusual NUMA topology*/
unsigned int threads; /* number of threads per core, 1 in case of
unusual numa topology */
+ unsigned int cache; /* cache size in bytes */
We don't allow changing/adding members to public structures. You probably will
be better of by adding this info to the capabilities XML.
Ok. So I have to modify capabilities XML and struct virCaps to add total cache
size related field.
};
Add cacheOcc member into virDomainInfo to get cache occupancy for one
VM.
struct virDomainInfo {
...
unsigned short nrVirtCpu; /* the number of virtual CPUs for the
domain */
unsigned long long cpuTime; /* the CPU time used in nanoseconds */
+ unsigned long long cacheOcc; /* the cache occupancy in Bytes */
Neither this structure can be modified.
};
With this change, those applications like OpenStack based on libvirt can
collect
cache usage for metering and monitoring (e.g. any VM using more than X% of
cache).
You can use the bulk stats API (virDomainListGetStats) to add this field
since that
uses the typed parameter approach which can be extended.
Got it. I will try to extend virDomainListGetStats API.
Because CMT implementation in Linux kernel is based on perf mechanism and
there is no nice and public API library for perf, I have to wrapper system
call for
perf in libvirt to use CMT feature, and enable/disable perf event for CMT when
VM is created/destroyed.
Any thoughts on this plan before I start submitting code patches? Do you
think
whether I need add several new APIs and XML element for CMT feature (or more
PQos features)?
Thanks,
Qiaowei
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] RFC: add Intel CMT feature support
On Sat, Jun 27, 2015 at 03:45:21 +, Ren, Qiaowei wrote:
Hi All,
Some Intel processor families (e.g. the Intel Xeon processor E5 v3 family)
introduced some PQos (Platform Qos) features to monitor or control shared
resource.
- CMT (Cache Monitoring Technology): measure the usage of cache by
applications running on the platform.
- CAT (Cache Allocation Technology): enable an OS or Hypervisor/VMM to
specify the amount of cache space into which an application can fill.
- MBM (Memory Bandwidth Monitoring): build on the CMT infrastructure to
allow monitoring of bandwidth from one level of the cache hierarchy to the
next - in this case focusing on the L3 cache, which is typically backed
directly by system memory. As a result of this implementation, memory
bandwidth can be monitored.
For more information, see Intel 64 and IA-32 Architectures Software
Developer's Manual.
Among these PQos features, currently CMT patches has been merged into Linux
kernel mainline. So this patch series proposed in this mail will focus on CMT
feature support in libvirt.
At the library API layer, I plan on adding cache related field into
virDomainInfo and virNodeInfo:
Add cache member into virNodeInfo to get total size of cache in one node.
struct virNodeInfo {
...
unsigned int cores; /* number of cores per socket, total number of
processors in case of unusual NUMA topology*/
unsigned int threads; /* number of threads per core, 1 in case of
unusual numa topology */
+ unsigned int cache; /* cache size in bytes */
We don't allow changing/adding members to public structures. You
probably will be better of by adding this info to the capabilities XML.
};
Add cacheOcc member into virDomainInfo to get cache occupancy for one VM.
struct virDomainInfo {
...
unsigned short nrVirtCpu; /* the number of virtual CPUs for the domain
*/
unsigned long long cpuTime; /* the CPU time used in nanoseconds */
+ unsigned long long cacheOcc; /* the cache occupancy in Bytes */
Neither this structure can be modified.
};
With this change, those applications like OpenStack based on libvirt can
collect cache usage for metering and monitoring (e.g. any VM using more than
X% of cache).
You can use the bulk stats API (virDomainListGetStats) to add this field
since that uses the typed parameter approach which can be extended.
Because CMT implementation in Linux kernel is based on perf mechanism and
there is no nice and public API library for perf, I have to wrapper system
call for perf in libvirt to use CMT feature, and enable/disable perf event
for CMT when VM is created/destroyed.
Any thoughts on this plan before I start submitting code patches? Do you
think whether I need add several new APIs and XML element for CMT feature (or
more PQos features)?
Thanks,
Qiaowei
Peter
signature.asc
Description: Digital signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] RFC: add Intel CMT feature support
Hi All,
Some Intel processor families (e.g. the Intel Xeon processor E5 v3 family)
introduced some PQos (Platform Qos) features to monitor or control shared
resource.
- CMT (Cache Monitoring Technology): measure the usage of cache by
applications running on the platform.
- CAT (Cache Allocation Technology): enable an OS or Hypervisor/VMM to
specify the amount of cache space into which an application can fill.
- MBM (Memory Bandwidth Monitoring): build on the CMT infrastructure to allow
monitoring of bandwidth from one level of the cache hierarchy to the next - in
this case focusing on the L3 cache, which is typically backed directly by
system memory. As a result of this implementation, memory bandwidth can be
monitored.
For more information, see Intel 64 and IA-32 Architectures Software
Developer's Manual.
Among these PQos features, currently CMT patches has been merged into Linux
kernel mainline. So this patch series proposed in this mail will focus on CMT
feature support in libvirt.
At the library API layer, I plan on adding cache related field into
virDomainInfo and virNodeInfo:
Add cache member into virNodeInfo to get total size of cache in one node.
struct virNodeInfo {
...
unsigned int cores; /* number of cores per socket, total number of
processors in case of unusual NUMA topology*/
unsigned int threads; /* number of threads per core, 1 in case of
unusual numa topology */
+ unsigned int cache; /* cache size in bytes */
};
Add cacheOcc member into virDomainInfo to get cache occupancy for one VM.
struct virDomainInfo {
...
unsigned short nrVirtCpu; /* the number of virtual CPUs for the domain */
unsigned long long cpuTime; /* the CPU time used in nanoseconds */
+ unsigned long long cacheOcc; /* the cache occupancy in Bytes */
};
With this change, those applications like OpenStack based on libvirt can
collect cache usage for metering and monitoring (e.g. any VM using more than X%
of cache).
Because CMT implementation in Linux kernel is based on perf mechanism and there
is no nice and public API library for perf, I have to wrapper system call for
perf in libvirt to use CMT feature, and enable/disable perf event for CMT when
VM is created/destroyed.
Any thoughts on this plan before I start submitting code patches? Do you think
whether I need add several new APIs and XML element for CMT feature (or more
PQos features)?
Thanks,
Qiaowei
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
