Re: [libvirt] [PATCH 0/8] Add perf and Intel CMT feature support
Hi, Daniel and Jiri, Do you have more comments about this new version? I guess that it should fix those comments on previous version. ^-^ Thanks, Qiaowei > -Original Message- > From: Ren, Qiaowei > Sent: Thursday, December 10, 2015 8:35 PM > To: libvir-list@redhat.com > Cc: Daniel P. Berrange; Jiri Denemark; Ren, Qiaowei > Subject: [PATCH 0/8] Add perf and Intel CMT feature support > > The series mainly adds Intel CMT feature support into libvirt. CMT is new > introduced PQos (Platform Qos) feature to monitor the usage of cache by > applications running on the platform. > > Currently CMT patches has been merged into Linux kernel mainline. > The CMT implementation in Linux kernel is based on perf mechanism and there is > no support for perf in libvirt, and so this series firstly add perf support > into libvirt, > including two public API and a set of util interfaces. And based on these > APIs and > interfaces, thie series implements CMT perf event support. > > TODO: > 1. add support for new APIs into libvirt-python library. > > Changes since v1: > * change perf APIs implementation to match new style of the API. > * add new xml element > * reenable all perf events previously enabled when libvirtd daemon > restart. > * redesign perf util functions. > > Changes since v2: > * add an example XML file to the test suite. > * add virPerfReadEvent(). > * change 'perf' xml element to new style. > * change 'perf' command to new stype. > > Qiaowei Ren (8): > perf: add new public APIs for perf event > perf: implement the remote protocol for perf event > perf: implement a set of util functions for perf event > qemu_driver: add support to perf event > perf: add new xml element > perf: reenable perf events when libvirtd restart > virsh: implement new command to support perf > virsh: extend domstats command > > daemon/remote.c | 47 > docs/schemas/domaincommon.rng | 27 +++ > include/libvirt/libvirt-domain.h | 19 ++ > include/libvirt/virterror.h | 1 + > src/Makefile.am | 1 + > src/conf/domain_conf.c| 111 ++ > src/conf/domain_conf.h| 10 + > src/driver-hypervisor.h | 12 + > src/libvirt-domain.c | 93 > src/libvirt_private.syms | 12 + > src/libvirt_public.syms | 6 + > src/qemu/qemu_domain.h| 3 + > src/qemu/qemu_driver.c| 195 + > src/qemu/qemu_process.c | 10 + > src/remote/remote_driver.c| 39 > src/remote/remote_protocol.x | 30 ++- > src/remote_protocol-structs | 18 ++ > src/util/virerror.c | 1 + > src/util/virperf.c| 303 > ++ > src/util/virperf.h| 63 ++ > tests/domainschemadata/domain-perf-simple.xml | 20 ++ > tools/virsh-domain-monitor.c | 7 + > tools/virsh-domain.c | 128 +++ > tools/virsh.pod | 27 ++- > 24 files changed, 1180 insertions(+), 3 deletions(-) create mode 100644 > src/util/virperf.c create mode 100644 src/util/virperf.h create mode 100644 > tests/domainschemadata/domain-perf-simple.xml > > -- > 1.9.1 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/8] Add perf and Intel CMT feature support
> -Original Message- > From: Daniel P. Berrange [mailto:berra...@redhat.com] > Sent: Tuesday, November 24, 2015 9:29 PM > To: Ren, Qiaowei; libvir-list@redhat.com > Subject: Re: [libvirt] [PATCH 0/8] Add perf and Intel CMT feature support > > On Tue, Nov 24, 2015 at 02:24:22PM +0100, Jiri Denemark wrote: > > On Tue, Nov 17, 2015 at 16:00:40 +0800, Qiaowei Ren wrote: > > > The series mainly adds Intel CMT feature support into libvirt. CMT > > > is new introduced PQos (Platform Qos) feature to monitor the usage > > > of cache by applications running on the platform. > > > > > > Currently CMT patches has been merged into Linux kernel mainline. > > > The CMT implementation in Linux kernel is based on perf mechanism > > > and there is no support for perf in libvirt, and so this series > > > firstly add perf support into libvirt, including two public API and > > > a set of util interfaces. And based on these APIs and interfaces, > > > thie series implements CMT perf event support. > > > > > > TODO: > > > 1. This series relys on keeping an open file descriptor for the guest. > > > We should add one patch to call sys_perf_event_open again iff > > > libvirtd is restarted. > > > > Yes, we should reenable perf events when reconnecting to running > > domains. Will we need to remember what events were enabled (in domain > > status XML) or is it something we can read back from the kernel? > > We need to record it somewhere. I guess this raises the question of whether we > should hide it in status XML, or persist the list of desired perf events in > the real > domain XML, so we can have the option of also having them enabled > immediately at startup, without requiring separate API call. > I checked the kernel perf interface and could not find the way to read back whether events were enabled from the kernel. So we have to record it somewhere according to Daniel. If we just persist the list of desired perf events in the real domain XML, users can have the option of having them enabled at startup. But it is also possible that users enable more events at runtime through API call for perf event. So should we record it in both real domain XML and status XML? Thanks, Qiaowei -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/8] Add perf and Intel CMT feature support
On Wed, Nov 25, 2015 at 13:16:34 +, Ren, Qiaowei wrote: > > > -Original Message- > > From: Daniel P. Berrange [mailto:berra...@redhat.com] > > Sent: Tuesday, November 24, 2015 9:29 PM > > To: Ren, Qiaowei; libvir-list@redhat.com > > Subject: Re: [libvirt] [PATCH 0/8] Add perf and Intel CMT feature support > > > > On Tue, Nov 24, 2015 at 02:24:22PM +0100, Jiri Denemark wrote: > > > On Tue, Nov 17, 2015 at 16:00:40 +0800, Qiaowei Ren wrote: > > > > The series mainly adds Intel CMT feature support into libvirt. CMT > > > > is new introduced PQos (Platform Qos) feature to monitor the usage > > > > of cache by applications running on the platform. > > > > > > > > Currently CMT patches has been merged into Linux kernel mainline. > > > > The CMT implementation in Linux kernel is based on perf mechanism > > > > and there is no support for perf in libvirt, and so this series > > > > firstly add perf support into libvirt, including two public API and > > > > a set of util interfaces. And based on these APIs and interfaces, > > > > thie series implements CMT perf event support. > > > > > > > > TODO: > > > > 1. This series relys on keeping an open file descriptor for the guest. > > > > We should add one patch to call sys_perf_event_open again iff > > > > libvirtd is restarted. > > > > > > Yes, we should reenable perf events when reconnecting to running > > > domains. Will we need to remember what events were enabled (in domain > > > status XML) or is it something we can read back from the kernel? > > > > We need to record it somewhere. I guess this raises the question of whether > > we > > should hide it in status XML, or persist the list of desired perf events in > > the real > > domain XML, so we can have the option of also having them enabled > > immediately at startup, without requiring separate API call. > > > > I checked the kernel perf interface and could not find the way to read > back whether events were enabled from the kernel. So we have to record > it somewhere according to Daniel. > > If we just persist the list of desired perf events in the real domain > XML, users can have the option of having them enabled at startup. But > it is also possible that users enable more events at runtime through > API call for perf event. So should we record it in both real domain > XML and status XML? The status XML always contains full domain XML of the running domain. Thus if you add to code to handle all this in domain XML, you don't need to do anything extra to store the info in status XML. However, you'll need to update live domain definition from the APIs which enable/disable the perf events. Jirka -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/8] Add perf and Intel CMT feature support
On Tue, Nov 24, 2015 at 02:24:22PM +0100, Jiri Denemark wrote: > On Tue, Nov 17, 2015 at 16:00:40 +0800, Qiaowei Ren wrote: > > The series mainly adds Intel CMT feature support into libvirt. CMT is > > new introduced PQos (Platform Qos) feature to monitor the usage of > > cache by applications running on the platform. > > > > Currently CMT patches has been merged into Linux kernel mainline. > > The CMT implementation in Linux kernel is based on perf mechanism and > > there is no support for perf in libvirt, and so this series firstly > > add perf support into libvirt, including two public API and a set of > > util interfaces. And based on these APIs and interfaces, thie series > > implements CMT perf event support. > > > > TODO: > > 1. This series relys on keeping an open file descriptor for the guest. > > We should add one patch to call sys_perf_event_open again iff libvirtd > > is restarted. > > Yes, we should reenable perf events when reconnecting to running > domains. Will we need to remember what events were enabled (in domain > status XML) or is it something we can read back from the kernel? We need to record it somewhere. I guess this raises the question of whether we should hide it in status XML, or persist the list of desired perf events in the real domain XML, so we can have the option of also having them enabled immediately at startup, without requiring separate API call. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/8] Add perf and Intel CMT feature support
On Tue, Nov 17, 2015 at 16:00:40 +0800, Qiaowei Ren wrote: > The series mainly adds Intel CMT feature support into libvirt. CMT is > new introduced PQos (Platform Qos) feature to monitor the usage of > cache by applications running on the platform. > > Currently CMT patches has been merged into Linux kernel mainline. > The CMT implementation in Linux kernel is based on perf mechanism and > there is no support for perf in libvirt, and so this series firstly > add perf support into libvirt, including two public API and a set of > util interfaces. And based on these APIs and interfaces, thie series > implements CMT perf event support. > > TODO: > 1. This series relys on keeping an open file descriptor for the guest. > We should add one patch to call sys_perf_event_open again iff libvirtd > is restarted. Yes, we should reenable perf events when reconnecting to running domains. Will we need to remember what events were enabled (in domain status XML) or is it something we can read back from the kernel? Jirka -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/8] Add perf and Intel CMT feature support
> -Original Message- > From: Daniel P. Berrange [mailto:berra...@redhat.com] > Sent: Tuesday, November 17, 2015 6:47 PM > To: Ren, Qiaowei > Cc: libvir-list@redhat.com; Jiri Denemark > Subject: Re: [PATCH 0/8] Add perf and Intel CMT feature support > > On Tue, Nov 17, 2015 at 04:00:40PM +0800, Qiaowei Ren wrote: > > The series mainly adds Intel CMT feature support into libvirt. CMT is > > new introduced PQos (Platform Qos) feature to monitor the usage of > > cache by applications running on the platform. > > > > Currently CMT patches has been merged into Linux kernel mainline. > > The CMT implementation in Linux kernel is based on perf mechanism and > > there is no support for perf in libvirt, and so this series firstly > > add perf support into libvirt, including two public API and a set of > > util interfaces. And based on these APIs and interfaces, thie series > > implements CMT perf event support. > > > > TODO: > > 1. This series relys on keeping an open file descriptor for the guest. > > We should add one patch to call sys_perf_event_open again iff libvirtd > > is restarted. > > I've not done a detailed patch review, but overall I think this design is > pretty > good now, so thanks for taking time to re-write this. > Daniel, thanks for your feedback. ^-^ Thanks, Qiaowei -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/8] Add perf and Intel CMT feature support
On Tue, Nov 17, 2015 at 04:00:40PM +0800, Qiaowei Ren wrote: > The series mainly adds Intel CMT feature support into libvirt. CMT is > new introduced PQos (Platform Qos) feature to monitor the usage of > cache by applications running on the platform. > > Currently CMT patches has been merged into Linux kernel mainline. > The CMT implementation in Linux kernel is based on perf mechanism and > there is no support for perf in libvirt, and so this series firstly > add perf support into libvirt, including two public API and a set of > util interfaces. And based on these APIs and interfaces, thie series > implements CMT perf event support. > > TODO: > 1. This series relys on keeping an open file descriptor for the guest. > We should add one patch to call sys_perf_event_open again iff libvirtd > is restarted. I've not done a detailed patch review, but overall I think this design is pretty good now, so thanks for taking time to re-write this. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
