Patch name: VM Power Management
Brief description: Verify VM power management in virtualized environments
Test Flag: Tested-by
Tester name: yong.liu at intel.com
Test environment:
OS: Fedora20 3.11.10-301.fc20.x86_64
GCC: gcc version 4.8.3 20140911
CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
NIC: Intel Corporation 82599ES 10-Gigabit SFI/SFP+
Network Connection [8086:10fb]
Test Tool Chain information:
Qemu: 1.6.1
libvirt: 1.1.3
Guest OS: Fedora20 3.11.10-301.fc20.x86_64
Guest GCC: gcc version 4.8.3 20140624
Commit ID: 72d3e7ad3183f42f8b9fb3bb1c12b3e1b39eef39
Detailed Testing information
DPDK SW Configuration:
Default x86_64-native-linuxapp-gcc configuration
Test Result Summary: Total 7 cases, 7 passed, 0 failed
Test Case - name:
VM Power Management Channel
Test Case - Description:
Check vm power management communication channels can
successfully connected
Test Case -command / instruction:
Create folder in system temporary filesystem for power
monitor socket
mkdir -p /tmp/powermonitor
Configure VM XML and pin VCPUs to specified CPUs
<vcpu placement='static'>5</vcpu>
<cputune>
<vcpupin vcpu='0' cpuset='1'/>
<vcpupin vcpu='1' cpuset='2'/>
<vcpupin vcpu='2' cpuset='3'/>
<vcpupin vcpu='3' cpuset='4'/>
<vcpupin vcpu='4' cpuset='5'/>
</cputune>
Configure VM XML to set up virtio serial ports
<channel type='unix'>
<source mode='bind'
path='/tmp/powermonitor/<vm_name>.<channel_num>'/>
<target type='virtio'
name='virtio.serial.port.poweragent.<channel_num>'/>
<address type='virtio-serial' controller='0'
bus='0' port='4'/>
</channel>
Run power-manager monitor in Host
./build/vm_power_mgr -c 0x3 -n 4
Startup VM and run guest_vm_power_mgr
guest_vm_power_mgr -c 0x1f -n 4 -- -i
Add vm in host and check vm_power_mgr can get frequency
normally
vmpower> add_vm <vm_name>
vmpower> add_channels <vm_name> all
vmpower> get_cpu_freq <core_num>
Check vcpu/cpu mapping can be detected normally
vmpower> show_vm <vm_name>
Test Case - expected test result:
VM power management communication channels can
sucessfully connected and host can get vm core information
Test Case - name:
VM Power Management Numa
Test Case - Description:
Check vm power management support manage cores in
different sockets
Test Case -command / instruction:
Get core and socket information by cpu_layout
./tools/cpu_layout.py
Configure VM XML to pin VCPUs on Socket1:
Repeat Case1
Check vcpu/cpu mapping can be detected normally
vmpower> show_vm <vm_name>
Test Case - expected test result:
VM power management communication channels can
sucessfully connected and show correct vm core information
Test Case - name:
VM scale cpu frequency down
Test Case - Description:
Check vm power management support VM configure self
cores frequency down
Test Case -command / instruction:
Setup VM power management environment
Send cpu frequency down hints to Host
vmpower(guest)> set_cpu_freq 0 down
Verify the frequency of physical CPU has been scaled
down correctly
vmpower> get_cpu_freq 1
Core 1 frequency: 2700000
Check other CPUs' frequency is not affected by actions
above
Check if the other VM works fine (if they use different
CPUs)
Repeat above actions several times
Test Case - expected test result:
Frequency for VM's core can be scaling down normally
Test Case - name:
VM scale cpu frequency up
Test Case - Description:
Check vm power management support VM configure self
cores frequency up
Test Case -command / instruction:
Setup VM power management environment
Send cpu frequency up hints to Host
vmpower(guest)> set_cpu_freq 0 up
Verify the frequency of physical CPU has been scaled up
correctly
vmpower> get_cpu_freq 1
Check other CPUs' frequency is not affected by actions
above
Check if the other VM works fine (if they use different
CPUs)
Repeat above actions several times
Test Case - expected test result:
Frequency for VM's core can be scaling up normally
Test Case - name:
VM Scale CPU Frequency to Min
Test Case - Description:
Check vm power management support VM configure self
cores frequency to minimum
Test Case -command / instruction:
Setup VM power management environment
Send cpu frequency scale to minimum hints
vmpower(guest)> set_cpu_freq 0 min
Verify the frequency of physical CPU has been scale to
min correctly
vmpower> get_cpu_freq 1
Core 1 frequency: 1200000
Check other CPUs' frequency is not affected by actions
above
check if the other VM works fine (if they use different
CPUs)
Test Case - expected test result:
Frequency for VM's core can be scaling to minimum
normally
Test Case - name:
VM Scale CPU Frequency to Max
Test Case - Description:
Check vm power management support VM configure self
cores frequency to maximum
Test Case -command / instruction:
Setup VM power management environment
Send cpu frequency scale to maximum hints
vmpower(guest)> set_cpu_freq 0 max
Verify the frequency of physical CPU has been scale to
max correctly
vmpower> get_cpu_freq 1
Core 1 frequency: 2800000
Check other CPUs' frequency is not affected by actions
above
check if the other VM works fine (if they use different
CPUs)
Test Case - expected test result:
Frequency for VM's core can be scaling to maximum
normally
Test Case - name:
VM Power Management Multi VMs
Test Case - Description:
Check vm power management support multiple VMs
Test Case -command / instruction:
Setup VM power management environment for VM1
Setup VM power management environment for VM2
Run power-manager in Host
./build/vm_power_mgr -c 0x3 -n 4
Startup VM1 and VM2
Add VM1 in host and check vm_power_mgr can get
frequency normally
vmpower> add_vm <vm1_name>
vmpower> add_channels <vm1_name> all
vmpower> get_cpu_freq <core_num>
Add VM2 in host and check vm_power_mgr can get
frequency normally
vmpower> add_vm <vm2_name>
vmpower> add_channels <vm2_name> all
vmpower> get_cpu_freq <core_num>
Check VM1 and VM2 cpu frequency can by modified by
guest_cli
Poweroff VM2 and remove VM2 from host vm_power_mgr
vmpower> rm_vm <vm2_name>
Test Case - expected test result:
VM power management supported multiple VMs add and
remove
> -----Original Message-----
> From: Carew, Alan
> Sent: Monday, October 13, 2014 3:36 AM
> To: dev at dpdk.org
> Cc: Liu, Yong
> Subject: [PATCH v4 00/10] VM Power Management
>
> Virtual Machine Power Management.
>
> The following patches add two DPDK sample applications and an alternate
> implementation of librte_power for use in virtualized environments.
> The idea is to provide librte_power functionality from within a VM to address
> the lack of MSRs to facilitate frequency changes from within a VM.
> It is ideally suited for Haswell which provides per core frequency scaling.
>
> The current librte_power affects frequency changes via the acpi-cpufreq
> 'userspace' power governor, accessed via sysfs.
>
> General Overview:(more information in each patch that follows).
> The VM Power Management solution provides two components:
>
> 1)VM: Allows for the a DPDK application in a VM to reuse the librte_power
> interface. Each lcore opens a Virto-Serial endpoint channel to the host,
> where the re-implementation of librte_power simply forwards the requests
> for
> frequency change to a host based monitor. The host monitor itself uses
> librte_power.
> Each lcore channel corresponds to a
> serial device '/dev/virtio-ports/virtio.serial.port.poweragent.<lcore_num>'
> which is opened in non-blocking mode.
> While each Virtual CPU can be mapped to multiple physical CPUs it is
> recommended that each vCPU should be mapped to a single core only.
>
> 2)Host: The host monitor is managed by a CLI, it allows for adding
> qemu/KVM
> virtual machines and associated channels to the monitor, manually changing
> CPU frequency, inspecting the state of VMs, vCPU to pCPU pinning and
> managing
> channels.
> Host channel endpoints are Virto-Serial endpoints configured as AF_UNIX
> file
> sockets which follow a specific naming convention
> i.e /tmp/powermonitor/<vm_name>.<channel_number>,
> each channel has an 1:1 mapping to a VM endpoint
> i.e. /dev/virtio-ports/virtio.serial.port.poweragent.<lcore_num>
> Host channel endpoints are opened in non-blocking mode and are
> monitored via epoll.
> Requests over each channel to change frequency are forwarded to the
> original
> librte_power.
>
> Channels must be manually configured as qemu-kvm command line
> arguments or
> libvirt domain definition(xml) e.g.
> <controller type='virtio-serial' index='0'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
> </controller>
> <channel type='unix'>
> <source mode='bind'
> path='/tmp/powermonitor/<vm_name>.<channel_num>'/>
> <target type='virtio' name='virtio.serial.port.poweragent.<channel_num>/>
> <address type='virtio-serial' controller='0' bus='0' port='<N>'/>
> </channel>
>
> Where multiple channels can be configured by specifying multiple <channel>
> elements, by replacing <vm_name>, <channel_num>.
> <N>(port number) should be incremented by 1 for each new channel
> element.
> More information on Virtio-Serial can be found here:
> http://fedoraproject.org/wiki/Features/VirtioSerial
> To enable the Hypervisor creation of channels, the host endpoint directory
> must be created with qemu permissions:
> mkdir /tmp/powermonitor
> chown qemu:qemu /tmp/powermonitor
>
> The host application runs on two separate lcores:
> Core N) CLI: For management of Virtual Machines adding channels to
> Monitor thread,
> inspecting state and manually setting CPU frequency [PATCH 02/09]
> Core N+1) Monitor Thread: An epoll based infinite loop that waits on channel
> events
> from VMs and calls the corresponding librte_power functions.
>
> A sample application is also provided to run on Virtual Machines, this
> application provides a CLI to manually set the frequency of a
> vCPU[PATCH 08/09]
>
> The current l3fwd-power sample application can also be run on a VM.
>
> Changes in V4:
> Fixed double free of channel during VM shutdown.
>
> Changes in V3:
> Fixed crash in Guest CLI when host application is not running.
> Renamed #defines to be more specific to the module they belong
> Added vCPU pinning via CLI
>
> Changes in V2:
> Runtime selection of librte_power implementations.
> Updated Unit tests to cover librte_power changes.
> PATCH[0/3] was sent twice, again as PATCH[0/4]
> Miscellaneous fixes.
>
> Alan Carew (10):
> Channel Manager and Monitor for VM Power Management(Host).
> VM Power Management CLI(Host).
> CPU Frequency Power Management(Host).
> VM Power Management application and Makefile.
> VM Power Management CLI(Guest).
> VM communication channels for VM Power Management(Guest).
> librte_power common interface for Guest and Host
> Packet format for VM Power Management(Host and Guest).
> Build system integration for VM Power Management(Guest and Host)
> VM Power Management Unit Tests
>
> app/test/Makefile | 3 +-
> app/test/autotest_data.py | 26 +
> app/test/test_power.c | 445 +-----------
> app/test/test_power_acpi_cpufreq.c | 544 ++++++++++++++
> app/test/test_power_kvm_vm.c | 308 ++++++++
> examples/vm_power_manager/Makefile | 57 ++
> examples/vm_power_manager/channel_manager.c | 804
> +++++++++++++++++++++
> examples/vm_power_manager/channel_manager.h | 314 ++++++++
> examples/vm_power_manager/channel_monitor.c | 231 ++++++
> examples/vm_power_manager/channel_monitor.h | 102 +++
> examples/vm_power_manager/guest_cli/Makefile | 56 ++
> examples/vm_power_manager/guest_cli/main.c | 87 +++
> examples/vm_power_manager/guest_cli/main.h | 52 ++
> .../guest_cli/vm_power_cli_guest.c | 155 ++++
> .../guest_cli/vm_power_cli_guest.h | 55 ++
> examples/vm_power_manager/main.c | 117 +++
> examples/vm_power_manager/main.h | 52 ++
> examples/vm_power_manager/power_manager.c | 244 +++++++
> examples/vm_power_manager/power_manager.h | 188 +++++
> examples/vm_power_manager/vm_power_cli.c | 669
> +++++++++++++++++
> examples/vm_power_manager/vm_power_cli.h | 47 ++
> lib/librte_power/Makefile | 3 +-
> lib/librte_power/channel_commands.h | 77 ++
> lib/librte_power/guest_channel.c | 162 +++++
> lib/librte_power/guest_channel.h | 89 +++
> lib/librte_power/rte_power.c | 540 ++------------
> lib/librte_power/rte_power.h | 120 ++-
> lib/librte_power/rte_power_acpi_cpufreq.c | 545 ++++++++++++++
> lib/librte_power/rte_power_acpi_cpufreq.h | 192 +++++
> lib/librte_power/rte_power_common.h | 39 +
> lib/librte_power/rte_power_kvm_vm.c | 135 ++++
> lib/librte_power/rte_power_kvm_vm.h | 179 +++++
> 32 files changed, 5725 insertions(+), 912 deletions(-)
> create mode 100644 app/test/test_power_acpi_cpufreq.c
> create mode 100644 app/test/test_power_kvm_vm.c
> create mode 100644 examples/vm_power_manager/Makefile
> create mode 100644 examples/vm_power_manager/channel_manager.c
> create mode 100644 examples/vm_power_manager/channel_manager.h
> create mode 100644 examples/vm_power_manager/channel_monitor.c
> create mode 100644 examples/vm_power_manager/channel_monitor.h
> create mode 100644 examples/vm_power_manager/guest_cli/Makefile
> create mode 100644 examples/vm_power_manager/guest_cli/main.c
> create mode 100644 examples/vm_power_manager/guest_cli/main.h
> create mode 100644
> examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
> create mode 100644
> examples/vm_power_manager/guest_cli/vm_power_cli_guest.h
> create mode 100644 examples/vm_power_manager/main.c
> create mode 100644 examples/vm_power_manager/main.h
> create mode 100644 examples/vm_power_manager/power_manager.c
> create mode 100644 examples/vm_power_manager/power_manager.h
> create mode 100644 examples/vm_power_manager/vm_power_cli.c
> create mode 100644 examples/vm_power_manager/vm_power_cli.h
> create mode 100644 lib/librte_power/channel_commands.h
> create mode 100644 lib/librte_power/guest_channel.c
> create mode 100644 lib/librte_power/guest_channel.h
> create mode 100644 lib/librte_power/rte_power_acpi_cpufreq.c
> create mode 100644 lib/librte_power/rte_power_acpi_cpufreq.h
> create mode 100644 lib/librte_power/rte_power_common.h
> create mode 100644 lib/librte_power/rte_power_kvm_vm.c
> create mode 100644 lib/librte_power/rte_power_kvm_vm.h
>
> --
> 1.9.3
