[Xen-devel] [PATCH v13 12/26] secondary vm suspend/resume/checkpoint code

2016-03-24 Thread Changlong Xie
From: Wen Congyang Secondary vm is running in colo mode. So we will do the following things again and again: 1. Resume secondary vm a. Send CHECKPOINT_SVM_READY to master. b. If it is not the first resume, call libxl__checkpoint_devices_preresume(). c. If it is the first resume(resume ri

[Xen-devel] [PATCH v13 08/26] libxc/migration: Specification update for DIRTY_PFN_LIST records

2016-03-24 Thread Changlong Xie
From: Wen Congyang Used by secondary to send it's dirty bitmap to primary under COLO. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang Signed-off-by: Changlong Xie Acked-by: Ian Jackson --- docs/specs/libxc-migration-stream.pandoc | 27 - tools/libxc/xc_sr_common.c

[Xen-devel] [PATCH v13 10/26] tools/libxl: add back channel support to write stream

2016-03-24 Thread Changlong Xie
From: Wen Congyang Add back channel support to write stream. If the write stream is a back channel stream, this means the write stream is used by Secondary to send some records back. Note: The function libxl__stream_write_checkpoint_state() will be used in later patches called "secondary vm susp

[Xen-devel] [PATCH v13 17/26] implement the cmdline for COLO

2016-03-24 Thread Changlong Xie
From: Wen Congyang Add a new option -c to the command 'xl remus'. If you want to use COLO HA instead of Remus HA, please use -c option. Update man pages to reflect the addition of a new option to 'xl remus' command. Also add a new option --colo to the internal command 'xl migrate-receive'. Sig

[Xen-devel] [PATCH v13 19/26] Introduce COLO mode and refactor relevant function

2016-03-24 Thread Changlong Xie
No functional changes. Signed-off-by: Wen Congyang Signed-off-by: Changlong Xie Acked-by: Ian Jackson --- tools/libxl/libxl_dm.c | 65 +- 1 file changed, 59 insertions(+), 6 deletions(-) diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm

[Xen-devel] [PATCH v13 14/26] primary vm suspend/resume/checkpoint code

2016-03-24 Thread Changlong Xie
From: Wen Congyang We will do the following things again and again: 1. Suspend primary vm a. Suspend primary vm b. do postsuspend c. Read CHECKPOINT_SVM_SUSPENDED sent by secondary 2. Checkpoint a. Write emulator xenstore data and emulator context b. Write checkpoint end record 3.

[Xen-devel] [PATCH v13 23/26] COLO nic: implement COLO nic subkind

2016-03-24 Thread Changlong Xie
From: Wen Congyang implement COLO nic subkind. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang Signed-off-by: Changlong Xie --- tools/hotplug/Linux/Makefile | 1 + tools/hotplug/Linux/colo-proxy-setup | 135 +++ tools/libxl/Makefile | 1 + too

[Xen-devel] [PATCH v13 15/26] libxc/restore: support COLO restore

2016-03-24 Thread Changlong Xie
From: Wen Congyang a. call callbacks resume/checkpoint/suspend while secondary vm status is consistent with primary b. send dirty pfn list to primary when checkpoint under colo c. send store gfn and console gfn to xl before resuming secondary vm Signed-off-by: Yang Hongyang Signed-off-by: We

[Xen-devel] [PATCH v13 22/26] COLO proxy: implement setup/teardown/preresume/postresume/checkpoint

2016-03-24 Thread Changlong Xie
From: Wen Congyang setup/teardown/preresume/postresume/checkpoint of COLO proxy module. we use netlink to communicate with proxy module. About colo-proxy module: http://www.spinics.net/lists/netdev/msg333520.html https://github.com/wencongyang/colo-proxy How to use: http://wiki.xen.org/wiki/COLO_

[Xen-devel] [PATCH v13 05/26] tools/libx{l, c}: add back channel to libxc

2016-03-24 Thread Changlong Xie
From: Wen Congyang In COLO mode, both VMs are running, and are considered in sync if the visible network traffic is identical. After some time, they fall out of sync. At this point, the two VMs have definitely diverged. Lets call the primary dirty bitmap set A, while the secondary dirty bitmap

[Xen-devel] [PATCH v13 16/26] libxc/save: support COLO save

2016-03-24 Thread Changlong Xie
From: Wen Congyang After suspend primary vm, get dirty bitmap on secondary vm, and send pages both dirty on primary/secondary to secondary. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang Signed-off-by: Changlong Xie CC: Andrew Cooper Acked-by: Ian Jackson --- tools/libxc/xc_sr_co

[Xen-devel] [PATCH v13 06/26] docs: add colo readme

2016-03-24 Thread Changlong Xie
From: Wen Congyang add colo readme, refer to http://wiki.xen.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang Signed-off-by: Changlong Xie Acked-by: Ian Campbell Acked-by: Wei Liu --- docs/README.colo | 9 + 1 file changed, 9 inser

[Xen-devel] [PATCH v13 18/26] COLO: introduce new API to prepare/start/do/get_error/stop replication

2016-03-24 Thread Changlong Xie
From: Wen Congyang We will use qemu block replication, and qemu provides some qmp commands to prepare replication, start replication, get replication error, and stop replication. Introduce new API to execute these qmp commands. Signed-off-by: Wen Congyang Signed-off-by: Changlong Xie Acked-by:

[Xen-devel] [PATCH v13 13/26] libxl_internal: move stream read manipulations to right place

2016-03-24 Thread Changlong Xie
No functional changes and this cleanup will make the later patch called "primary vm suspend/resume/checkpoint code" not too complicated. Signed-off-by: Wen Congyang Signed-off-by: Changlong Xie Acked-by: Ian Jackson --- tools/libxl/libxl_internal.h | 132 +--

[Xen-devel] [PATCH v13 07/26] docs/libxl: Introduce CHECKPOINT_CONTEXT to support migration v2 colo streams

2016-03-24 Thread Changlong Xie
From: Wen Congyang Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang Signed-off-by: Changlong Xie Acked-by: Ian Jackson --- docs/specs/libxl-migration-stream.pandoc | 59 ++-- tools/libxl/libxl_sr_stream_format.h | 11 ++ tools/python/xen/migration/

[Xen-devel] [PATCH v13 21/26] COLO: use qemu block replication

2016-03-24 Thread Changlong Xie
From: Wen Congyang Use qemu block replication as our block replication solution. Note that guest must be paused before starting COLO, otherwise, the disk won't be consistent between primary and secondary. Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang Signed-off-by: Changlong Xie Ac

[Xen-devel] [PATCH v13 11/26] tools/libxl: add back channel support to read stream

2016-03-24 Thread Changlong Xie
From: Wen Congyang This is used by primay to read records sent by secondary. Note: The function libxl__stream_read_checkpoint_state() will be used in later patches called "secondary vm suspend/resume/checkpoint code" and "primary vm suspend/resume/checkpoint code". Signed-off-by: Yang Hongyang

[Xen-devel] [PATCH v13 20/26] Support colo mode for qemu disk

2016-03-24 Thread Changlong Xie
From: Wen Congyang Usage: disk = ['...,colo,colo-host=xxx,colo-port=xxx,colo-export=xxx,active-disk=xxx,hidden-disk=xxx...'] For QEMU block replication details: http://wiki.qemu.org/Features/BlockReplication Note: we just introduce COLO framework, but don't implement COLO operations in this pat

[Xen-devel] [PATCH v13 24/26] setup and control colo proxy on primary side

2016-03-24 Thread Changlong Xie
From: Wen Congyang Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang Signed-off-by: Changlong Xie Acked-by: Ian Jackson --- tools/libxl/libxl_colo.h| 25 ++ tools/libxl/libxl_colo_save.c | 94 + tools/libxl/libxl_internal.h

[Xen-devel] [PATCH v13 25/26] setup and control colo proxy on secondary side

2016-03-24 Thread Changlong Xie
From: Wen Congyang Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang Signed-off-by: Changlong Xie --- tools/libxl/libxl_colo_restore.c | 28 +--- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/tools/libxl/libxl_colo_restore.c b/tools/libxl/libxl_

[Xen-devel] [PATCH v13 26/26] cmdline switches and config vars to control colo-proxy

2016-03-24 Thread Changlong Xie
From: Wen Congyang Add cmdline switches to 'xl migrate-receive' command to specify a domain-specific hotplug script to setup COLO proxy. Add a new config var 'colo.default.agentscript' to xl.conf, that allows the user to override the default global script used to setup COLO proxy. Signed-off-by

[Xen-devel] [PATCH v13 09/26] libxc/migration: export read_record for common use

2016-03-24 Thread Changlong Xie
From: Wen Congyang read_record() could be used by primary to read dirty bitmap record sent by secondary under COLO. When used by xc save side, we need to pass the backchannel fd instead of ctx->fd to read_record(), so we added a fd param to it. No functional changes. CC: Andrew Cooper Signed-of

[Xen-devel] [PATCH v13 00/26] COarse-grain LOck-stepping Virtual Machines for Non-stop Service

2016-03-24 Thread Changlong Xie
This patchset implemented the COLO feature for Xen. For detail/install/use of COLO feature, refer to: http://wiki.xen.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping You can get the codes from here: https://github.com/Pating/xen/tree/changlox/colo_v13 Changlog from v12 to v13 1. Rebase to the upstream

[Xen-devel] [PATCH v13 02/26] tools/libxl: introduce libxl__domain_common_switch_qemu_logdirty()

2016-03-24 Thread Changlong Xie
From: Wen Congyang Secondary vm is running in COLO mode, we need to send secondary vm's dirty page information to primary host at checkpoint, so we have to enable qemu logdirty on secondary. libxl__domain_suspend_common_switch_qemu_logdirty() is to enable qemu logdirty. But it uses libxl__domain

[Xen-devel] [PATCH v13 03/26] tools/libxl: Add back channel to allow migration target send data back

2016-03-24 Thread Changlong Xie
From: Wen Congyang In COLO mode, secondary needs to send the following data to primary: 1. In libxl Secondary sends the following CHECKPOINT_CONTEXT to primary: CHECKPOINT_SVM_SUSPENDED, CHECKPOINT_SVM_READY and CHECKPOINT_SVM_RESUMED 2. In libxc Secondary sends the dirty pfn list to pri

[Xen-devel] [PATCH v13 04/26] tools/libxl: Introduce new helper function dup_fd_helper()

2016-03-24 Thread Changlong Xie
From: Wen Congyang It is pure refactoring and no functional changes. Signed-off-by: Wen Congyang Signed-off-by: Changlong Xie Acked-by: Wei Liu --- tools/libxl/libxl_save_callout.c | 26 ++ 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/tools/libxl/lib

[Xen-devel] [PATCH v13 01/26] tools/libxl: introduction of libxl__qmp_restore to load qemu state

2016-03-24 Thread Changlong Xie
From: Wen Congyang In normal migration, the qemu state is passed to qemu as a parameter. With COLO, secondary vm is running. So we will do the following steps at every checkpoint: 1. suspend both primary vm and secondary vm 2. sync the state 3. resume both primary vm and secondary vm Primary will

Re: [Xen-devel] [PATCH v3 2/5] libxl: add new pvusb backend "qusb" provided by qemu

2016-03-24 Thread Chun Yan Liu
>>> On 3/25/2016 at 02:29 PM, in message <56f4dade.4040...@suse.com>, Juergen >>> Gross wrote: > On 25/03/16 03:23, Chun Yan Liu wrote: > > > > > On 3/23/2016 at 08:24 PM, in message > > <1458735847-9448-3-git-send-email-jgr...@suse.com>, Juergen Gross > > wrote: > >> Add a new

Re: [Xen-devel] [PATCH v12 14/26] primary vm suspend/resume/checkpoint code

2016-03-24 Thread Changlong Xie
On 03/24/2016 11:24 PM, Ian Jackson wrote: Changlong Xie writes ("[PATCH v12 14/26] primary vm suspend/resume/checkpoint code"): From: Wen Congyang ... Thanks. Most of this seems plausible (although I have done a review of rather limited depth): However, there are several instances like th

Re: [Xen-devel] [PATCH v3 4/5] libxl: check for dynamic device model start required

2016-03-24 Thread Chun Yan Liu
>>> On 3/25/2016 at 02:25 PM, in message <56f4d9d6.8030...@suse.com>, Juergen >>> Gross wrote: > On 25/03/16 03:06, Chun Yan Liu wrote: > > > > > On 3/23/2016 at 08:24 PM, in message > > <1458735847-9448-5-git-send-email-jgr...@suse.com>, Juergen Gross > > wrote: > >> Add a serv

Re: [Xen-devel] [PATCH v3 2/5] libxl: add new pvusb backend "qusb" provided by qemu

2016-03-24 Thread Juergen Gross
On 25/03/16 03:23, Chun Yan Liu wrote: > > On 3/23/2016 at 08:24 PM, in message > <1458735847-9448-3-git-send-email-jgr...@suse.com>, Juergen Gross > wrote: >> Add a new pvusb backend type "qusb" which is provided by qemu. It can >> be selected either by specifying the type directly in th

Re: [Xen-devel] [PATCH v3 4/5] libxl: check for dynamic device model start required

2016-03-24 Thread Juergen Gross
On 25/03/16 03:06, Chun Yan Liu wrote: > > On 3/23/2016 at 08:24 PM, in message > <1458735847-9448-5-git-send-email-jgr...@suse.com>, Juergen Gross > wrote: >> Add a service routine checking whether a device model must be started >> after adding a device to a domain. >> >> Signed-off-b

Re: [Xen-devel] [PATCH v3 5/5] libxl: add domain config parameter to force start of qemu

2016-03-24 Thread Juergen Gross
On 24/03/16 21:35, Wei Liu wrote: > On Thu, Mar 24, 2016 at 08:28:45PM +, Wei Liu wrote: >> On Wed, Mar 23, 2016 at 01:24:07PM +0100, Juergen Gross wrote: >>> Today the device model (qemu) is started for a pv domain only in case >>> a device requiring qemu is specified in the domain configurati

Re: [Xen-devel] [PATCH v3 5/5] libxl: add domain config parameter to force start of qemu

2016-03-24 Thread Juergen Gross
On 24/03/16 21:28, Wei Liu wrote: > On Wed, Mar 23, 2016 at 01:24:07PM +0100, Juergen Gross wrote: >> Today the device model (qemu) is started for a pv domain only in case >> a device requiring qemu is specified in the domain configuration >> (qdisk, vfb, channel). If there is no such device the de

Re: [Xen-devel] [PATCH v3 1/5] libxl: make libxl__need_xenpv_qemu() operate on domain config

2016-03-24 Thread Juergen Gross
On 24/03/16 21:28, Wei Liu wrote: > On Wed, Mar 23, 2016 at 01:24:03PM +0100, Juergen Gross wrote: >> libxl__need_xenpv_qemu() is called with configuration data for console, >> vfbs, disks and channels today in order to evaluate the need for >> starting a device model for a pv domain. >> >> The con

Re: [Xen-devel] [PATCH 3/3] libxl: add new pvusb backend "qusb" provided by qemu

2016-03-24 Thread Juergen Gross
On 24/03/16 21:07, Wei Liu wrote: > On Fri, Mar 18, 2016 at 09:24:26AM +0100, Juergen Gross wrote: >> On 17/03/16 17:55, George Dunlap wrote: >>> On Thu, Mar 10, 2016 at 3:00 PM, Juergen Gross wrote: Add a new pvusb backend type "qusb" which is provided by qemu. It can be selected either

Re: [Xen-devel] [PATCH v12 26/26] cmdline switches and config vars to control colo-proxy

2016-03-24 Thread Changlong Xie
On 03/25/2016 12:12 AM, Ian Jackson wrote: Changlong Xie writes ("[PATCH v12 26/26] cmdline switches and config vars to control colo-proxy"): From: Wen Congyang Add cmdline switches to 'xl migrate-receive' command to specify a domain-specific hotplug script to setup COLO proxy. +if (ni

Re: [Xen-devel] [PATCH v12 23/26] COLO nic: implement COLO nic subkind

2016-03-24 Thread Changlong Xie
On 03/25/2016 12:05 AM, Ian Jackson wrote: Changlong Xie writes ("[PATCH v12 23/26] COLO nic: implement COLO nic subkind"): From: Wen Congyang implement COLO nic subkind. ... diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl index a206d68..a8be078 100644 --- a/tools/libx

Re: [Xen-devel] [PATCH v8 3/3] VT-d: Fix vt-d Device-TLB flush timeout issue

2016-03-24 Thread Xu, Quan
On March 24, 2016 11:38pm, wrote: > On Thu, 2016-03-24 at 13:57 +0800, Quan Xu wrote: > > If Device-TLB flush timed out, we would hide the target ATS device and > > crash the domain owning this ATS device. If impacted domain is > > hardware domain, just throw out a warning. > > > > The hidden devi

[Xen-devel] [xen-4.5-testing test] 87166: tolerable FAIL - PUSHED

2016-03-24 Thread osstest service owner
flight 87166 xen-4.5-testing real [real] http://logs.test-lab.xenproject.org/osstest/logs/87166/ Failures :-/ but no regressions. Tests which are failing intermittently (not blocking): test-amd64-i386-libvirt 3 host-install(3) broken in 87050 pass in 87166 test-armhf-armhf-xl-arndale 6

Re: [Xen-devel] [PATCH v8 2/3] VT-d: Wrap a _sync version for all VT-d flush interfaces

2016-03-24 Thread Xu, Quan
On March 24, 2016 11:06pm, wrote: > On Thu, 2016-03-24 at 14:56 +0100, Dario Faggioli wrote: > > On Thu, 2016-03-24 at 13:57 +0800, Quan Xu wrote: > > > > > > @@ -134,8 +133,8 @@ int dev_invalidate_iotlb(struct iommu *iommu, > > > u16 > > > did, > > >  /* invalidate all translations: >

Re: [Xen-devel] [PATCH v12 26/26] cmdline switches and config vars to control colo-proxy

2016-03-24 Thread Changlong Xie
On 03/25/2016 12:12 AM, Ian Jackson wrote: Changlong Xie writes ("[PATCH v12 26/26] cmdline switches and config vars to control colo-proxy"): From: Wen Congyang Add cmdline switches to 'xl migrate-receive' command to specify a domain-specific hotplug script to setup COLO proxy. +if (ni

[Xen-devel] 4.4: INFO: rcu_sched self-detected stall on CPU

2016-03-24 Thread Steven Haigh
Hi all, Firstly, I've cross-posted this to xen-devel and the lkml - as this problem seems to only exist when using kernel 4.4 as a Xen DomU kernel. I have also CC'ed Greg KH for his awesome insight as maintainer. Please CC myself into replies - as I'm not a member of the kernel mailing list - I m

Re: [Xen-devel] [PATCH v12 23/26] COLO nic: implement COLO nic subkind

2016-03-24 Thread Changlong Xie
On 03/25/2016 12:05 AM, Ian Jackson wrote: Changlong Xie writes ("[PATCH v12 23/26] COLO nic: implement COLO nic subkind"): From: Wen Congyang implement COLO nic subkind. ... diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl index a206d68..a8be078 100644 --- a/tools/libx

Re: [Xen-devel] [PATCH v3 2/5] libxl: add new pvusb backend "qusb" provided by qemu

2016-03-24 Thread Chun Yan Liu
>>> On 3/23/2016 at 08:24 PM, in message <1458735847-9448-3-git-send-email-jgr...@suse.com>, Juergen Gross wrote: > Add a new pvusb backend type "qusb" which is provided by qemu. It can > be selected either by specifying the type directly in the configuration > or it is selected automatically

Re: [Xen-devel] [PATCH v3 4/5] libxl: check for dynamic device model start required

2016-03-24 Thread Chun Yan Liu
>>> On 3/23/2016 at 08:24 PM, in message <1458735847-9448-5-git-send-email-jgr...@suse.com>, Juergen Gross wrote: > Add a service routine checking whether a device model must be started > after adding a device to a domain. > > Signed-off-by: Juergen Gross > --- > tools/libxl/libxl.c

Re: [Xen-devel] [PATCH v12 19/26] Introduce COLO mode and refactor relevant function

2016-03-24 Thread Changlong Xie
On 03/24/2016 11:45 PM, Ian Jackson wrote: Changlong Xie writes ("[PATCH v12 19/26] Introduce COLO mode and refactor relevant function"): No functional changes. Thanks, this is quite helpful. Although: +static char *qemu_disk_scsi_drive_string(libxl__gc *gc, const char *pdev_path, +

Re: [Xen-devel] [PATCH v12 12/26] secondary vm suspend/resume/checkpoint code

2016-03-24 Thread Changlong Xie
On 03/24/2016 11:15 PM, Ian Jackson wrote: Changlong Xie writes ("[PATCH v12 12/26] secondary vm suspend/resume/checkpoint code"): From: Wen Congyang Thanks. This is all fine except for one small thing: +/* COLO only supports HVM now because it does not work very + * well with pv

Re: [Xen-devel] [PATCH v12 14/26] primary vm suspend/resume/checkpoint code

2016-03-24 Thread Changlong Xie
On 03/24/2016 11:24 PM, Ian Jackson wrote: Changlong Xie writes ("[PATCH v12 14/26] primary vm suspend/resume/checkpoint code"): From: Wen Congyang ... Thanks. Most of this seems plausible (although I have done a review of rather limited depth): However, there are several instances like th

[Xen-devel] [linux-mingo-tip-master test] 87146: regressions - FAIL

2016-03-24 Thread osstest service owner
flight 87146 linux-mingo-tip-master real [real] http://logs.test-lab.xenproject.org/osstest/logs/87146/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: build-i386-rumpuserxen6 xen-build fail REGR. vs. 60684 build-amd6

Re: [Xen-devel] [PATCH 3/7] oxenstored: refactor request processing

2016-03-24 Thread Boris Ostrovsky
On 03/24/2016 06:49 PM, Andrew Cooper wrote: On 24/03/2016 22:22, Boris Ostrovsky wrote: On 03/17/2016 01:51 PM, Jonathan Davies wrote: Encapsulate the request in a record that is passed from do_input to process_packet and input_handle_error. This will be helpful when keeping track of the requ

Re: [Xen-devel] [PATCH 3/7] oxenstored: refactor request processing

2016-03-24 Thread Andrew Cooper
On 24/03/2016 22:22, Boris Ostrovsky wrote: > On 03/17/2016 01:51 PM, Jonathan Davies wrote: >> Encapsulate the request in a record that is passed from do_input to >> process_packet and input_handle_error. >> >> This will be helpful when keeping track of the requests made as part >> of a >> transac

Re: [Xen-devel] Interested to participate in Outreachy Program

2016-03-24 Thread Wei Liu
On Wed, Mar 23, 2016 at 02:05:42PM -0500, Doug Goldstein wrote: [...] > > > > domid = find_domain(argv[optind]); > > if (!type) > > -libxl_primary_console_exec(ctx, domid); > > -else > > -libxl_console_exec(ctx, domid, num, type); > > +libxl_primary_console_e

Re: [Xen-devel] [PATCH 3/7] oxenstored: refactor request processing

2016-03-24 Thread Boris Ostrovsky
On 03/17/2016 01:51 PM, Jonathan Davies wrote: Encapsulate the request in a record that is passed from do_input to process_packet and input_handle_error. This will be helpful when keeping track of the requests made as part of a transaction. Signed-off-by: Jonathan Davies Reviewed-by: Andrew Co

[Xen-devel] [qemu-mainline test] 87141: regressions - FAIL

2016-03-24 Thread osstest service owner
flight 87141 qemu-mainline real [real] http://logs.test-lab.xenproject.org/osstest/logs/87141/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 9 debian-hvm-install fail REGR. vs. 86454 test

[Xen-devel] [xen-unstable-smoke test] 87229: tolerable all pass - PUSHED

2016-03-24 Thread osstest service owner
flight 87229 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/87229/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 12 migrate-support-checkfail never pass test-armhf-armhf-xl 12

Re: [Xen-devel] [PATCH v5 01/28] HYPERCALL_version_op. New hypercall mirroring XENVER_ but sane.

2016-03-24 Thread Konrad Rzeszutek Wilk
> Thanks! > > And Argh! > > I neglected to change the name of kernel_cache_init as Jan requested. Jan, here is an updated patch. I changed the name to capabilities_cache_init. Sorry about that! From e766ddb37acda16c3adbd65179d532b9381bda8e Mon Sep 17 00:00:00 2001 From: Konrad Rzeszutek Wilk

Re: [Xen-devel] [PATCH v5 02/28] libxc/libxl/python/xenstat/ocaml: Use new XEN_VERSION hypercall

2016-03-24 Thread Wei Liu
On Thu, Mar 24, 2016 at 04:00:14PM -0400, Konrad Rzeszutek Wilk wrote: [...] > + > +static int libxl__xc_version_wrapper(libxl__gc *gc, unsigned int cmd, char > *buf, ssize_t len, char **dst) Please wrap this long line. > +{ > +int r; > + > +r = xc_version(CTX->xch, cmd, buf, len); > +

Re: [Xen-devel] [PATCH v5 01/28] HYPERCALL_version_op. New hypercall mirroring XENVER_ but sane.

2016-03-24 Thread Konrad Rzeszutek Wilk
On March 24, 2016 4:22:29 PM EDT, Andrew Cooper wrote: >On 24/03/16 20:00, Konrad Rzeszutek Wilk wrote: >> This hypercall mirrors the XENVER_ in that it has similar >functionality. >> However it is designed differently: >> - No compat layer. The data structures are the same size on 32 >>as o

[Xen-devel] [linux-linus test] 87133: regressions - FAIL

2016-03-24 Thread osstest service owner
flight 87133 linux-linus real [real] http://logs.test-lab.xenproject.org/osstest/logs/87133/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: build-i386-rumpuserxen6 xen-build fail REGR. vs. 59254 build-amd64-rumpuserx

Re: [Xen-devel] [PATCH v3 5/5] libxl: add domain config parameter to force start of qemu

2016-03-24 Thread Wei Liu
On Thu, Mar 24, 2016 at 08:28:45PM +, Wei Liu wrote: > On Wed, Mar 23, 2016 at 01:24:07PM +0100, Juergen Gross wrote: > > Today the device model (qemu) is started for a pv domain only in case > > a device requiring qemu is specified in the domain configuration > > (qdisk, vfb, channel). If ther

Re: [Xen-devel] [PATCH v3 5/5] libxl: add domain config parameter to force start of qemu

2016-03-24 Thread Wei Liu
On Wed, Mar 23, 2016 at 01:24:07PM +0100, Juergen Gross wrote: > Today the device model (qemu) is started for a pv domain only in case > a device requiring qemu is specified in the domain configuration > (qdisk, vfb, channel). If there is no such device the device model > isn't started and hence it

Re: [Xen-devel] [PATCH v3 1/5] libxl: make libxl__need_xenpv_qemu() operate on domain config

2016-03-24 Thread Wei Liu
On Wed, Mar 23, 2016 at 01:24:03PM +0100, Juergen Gross wrote: > libxl__need_xenpv_qemu() is called with configuration data for console, > vfbs, disks and channels today in order to evaluate the need for > starting a device model for a pv domain. > > The console data is local to the caller and set

Re: [Xen-devel] [PATCH v3 3/5] libxl: add service function to check whether device model is running

2016-03-24 Thread Wei Liu
On Wed, Mar 23, 2016 at 01:24:05PM +0100, Juergen Gross wrote: > Add an internal service function to check for a running device model. > This can be used later when adding devices to a domain requiring a > device model for either printing an error message or starting the > device model in case it i

Re: [Xen-devel] [PATCH v5 01/28] HYPERCALL_version_op. New hypercall mirroring XENVER_ but sane.

2016-03-24 Thread Andrew Cooper
On 24/03/16 20:00, Konrad Rzeszutek Wilk wrote: > This hypercall mirrors the XENVER_ in that it has similar functionality. > However it is designed differently: > - No compat layer. The data structures are the same size on 32 >as on 64-bit. > - The hypercall accepts three arguments - the comm

Re: [Xen-devel] [PATCH 3/3] libxl: add new pvusb backend "qusb" provided by qemu

2016-03-24 Thread Wei Liu
On Fri, Mar 18, 2016 at 09:24:26AM +0100, Juergen Gross wrote: > On 17/03/16 17:55, George Dunlap wrote: > > On Thu, Mar 10, 2016 at 3:00 PM, Juergen Gross wrote: > >> Add a new pvusb backend type "qusb" which is provided by qemu. It can > >> be selected either by specifying the type directly in t

[Xen-devel] [PATCH v5 28/28] MAINTAINERS/xsplice: Add myself and Ross as the maintainers.

2016-03-24 Thread Konrad Rzeszutek Wilk
If you have a patch for xSplice send it our way! Signed-off-by: Ross Lagerwall Signed-off-by: Konrad Rzeszutek Wilk --- Cc: Ian Jackson Cc: Jan Beulich Cc: Keir Fraser Cc: Tim Deegan v5: Sort them F: fields (Jan) --- --- MAINTAINERS | 10 ++ 1 file changed, 10 insertions(+) diff

[Xen-devel] [PATCH v5 14/28] x86, xsplice: Print payload's symbol name and payload name in backtraces

2016-03-24 Thread Konrad Rzeszutek Wilk
Naturally the backtrace is presented when an instruction hits an bug_frame or %p is used. The payloads do not support bug_frames yet - however the functions the payloads call could hit an BUG() or WARN(). The traps.c has logic to scan for it this - and eventually it will find the correct bug_fram

[Xen-devel] [PATCH v5 08/28] xen-xsplice: Tool to manipulate xsplice payloads

2016-03-24 Thread Konrad Rzeszutek Wilk
A simple tool that allows an system admin to perform basic xsplice operations: - Upload a xsplice file (with an unique name) - List all the xsplice payloads loaded. - Apply, revert, replace, or unload the payload using the unique name. - Do all two - upload, and apply the payload in one go

[Xen-devel] [PATCH v5 04/28] vmap: Add vmalloc_cb and vfree_cb

2016-03-24 Thread Konrad Rzeszutek Wilk
For those users who want to supply their own vmap callback. To be called _after_ the pages have been allocated and the vmap API is ready to hand out virtual addresses. Instead of using the vmap ones it can call the callback which will be responsible for generating the virtual address. This allows

[Xen-devel] [PATCH v5 24/28] xsplice/xen_replace_world: Test-case for XSPLICE_ACTION_REPLACE

2016-03-24 Thread Konrad Rzeszutek Wilk
With this third payload one can do: -bash-4.1# xen-xsplice load xen_hello_world.xsplice Uploading xen_hello_world.xsplice (10148 bytes) Performing check: completed Performing apply:. completed [xen_hello_world depends on hypervisor build-id] -bash-4.1# xen-xsplice load xen_bye_world.xsplice Uploa

[Xen-devel] [PATCH v5 27/28] xsplice: Add support for shadow variables.

2016-03-24 Thread Konrad Rzeszutek Wilk
From: Ross Lagerwall Shadow variables are a piece of infrastructure to be used by xsplice modules. They are used to attach a new piece of data to an existing structure in memory. Martin Pohlack also mentions that using the SHADOW_SLOTS of small prime numbers has the advantage of having a simple

[Xen-devel] [PATCH v5 26/28] xsplice: Prevent duplicate payloads from being loaded.

2016-03-24 Thread Konrad Rzeszutek Wilk
From: Ross Lagerwall Signed-off-by: Ross Lagerwall Signed-off-by: Konrad Rzeszutek Wilk --- Cc: Ian Jackson Cc: Jan Beulich Cc: Keir Fraser Cc: Tim Deegan --- --- xen/common/xsplice.c | 21 + 1 file changed, 21 insertions(+) diff --git a/xen/common/xsplice.c b/xen/comm

[Xen-devel] [PATCH v5 22/28] xsplice: Print build_id in keyhandler and on bootup.

2016-03-24 Thread Konrad Rzeszutek Wilk
As it should be an useful debug mechanism. Signed-off-by: Konrad Rzeszutek Wilk -- Cc: Ian Jackson Cc: Jan Beulich Cc: Keir Fraser Cc: Tim Deegan v2: s/char */const void * v5: s/ssize_t/unsigned int/ --- xen/common/xsplice.c | 12 1 file changed, 12 insertions(+) diff --git a/

[Xen-devel] [PATCH v5 11/28] xsplice: Implement support for applying/reverting/replacing patches.

2016-03-24 Thread Konrad Rzeszutek Wilk
From: Ross Lagerwall Implement support for the apply, revert and replace actions. To perform and action on a payload, the hypercall sets up a data structure to schedule the work. A hook is added in all the return-to-guest paths to check for work to do and execute it if needed. In this way, patc

[Xen-devel] [PATCH v5 02/28] libxc/libxl/python/xenstat/ocaml: Use new XEN_VERSION hypercall

2016-03-24 Thread Konrad Rzeszutek Wilk
We change the xen_version libxc code to use the new hypercall. Which of course means every user in the code base has to be changed over. It is important to note that the xc_version_op has a different return semantic than the previous one. It returns negative values on error (like the old one), but

[Xen-devel] [PATCH v5 20/28] HYPERCALL_version_op: Add VERSION_build_id to retrieve build-id.

2016-03-24 Thread Konrad Rzeszutek Wilk
The VERSION hypercall provides the flexibility to expose the size of the build-id (so the callers can allocate the proper size before trying to retrieve it). It also allows in one nice swoop to retrieve the hypervisor build-id in the provided buffer. Signed-off-by: Konrad Rzeszutek Wilk --- Cc: D

[Xen-devel] [PATCH v5 19/28] build_id: Provide ld-embedded build-ids

2016-03-24 Thread Konrad Rzeszutek Wilk
This patch enables the Elf to be built with the build-id and provide in the Xen hypervisor the code to extract it. One can also retrieve the value of the build-id by doing 'readelf -n xen-syms'. For EFI builds we re-use the same build-id that the xen-syms was built with. The version of ld that f

[Xen-devel] [PATCH v5 07/28] libxc: Implementation of XEN_XSPLICE_op in libxc

2016-03-24 Thread Konrad Rzeszutek Wilk
The underlaying toolstack code to do the basic operations when using the XEN_XSPLICE_op syscalls: - upload the payload, - get status of an payload, - list all the payloads, - apply, check, replace, and revert the payload. Signed-off-by: Konrad Rzeszutek Wilk Signed-off-by: Ross Lagerwall Ack

[Xen-devel] [PATCH v5 13/28] xsplice, symbols: Implement symbol name resolution on address.

2016-03-24 Thread Konrad Rzeszutek Wilk
From: Ross Lagerwall If in the payload we do not have the old_addr we can resolve the virtual address based on the UNDEFined symbols. We also use an boolean flag: new_symbol to track symbols. The usual case this is used is by: * A payload may introduce a new symbol * A payload may override an e

[Xen-devel] [PATCH v5] xSplice v1 design and implementation.

2016-03-24 Thread Konrad Rzeszutek Wilk
Hey! Changelog: v4: http://lists.xen.org/archives/html/xen-devel/2016-03/msg01776.html - Lots of review. Lots of rework. Some patches checked in. v3: http://www.gossamer-threads.com/lists/xen/devel/418262 and http://lists.xenproject.org/archives/html/xen-devel/2016-02/msg04106.html - Ac

[Xen-devel] [PATCH v5 15/28] xsplice: Add .xsplice.hooks functions and test-case

2016-03-24 Thread Konrad Rzeszutek Wilk
From: Ross Lagerwall Add hook functions which run during patch apply and patch revert. Hook functions are used by xsplice payloads to manipulate data structures during patching, etc. Also add macros to be used by payloads for excluding functions or sections from being included in a patch. Furth

[Xen-devel] [PATCH v5 16/28] xsplice: Add support for bug frames.

2016-03-24 Thread Konrad Rzeszutek Wilk
From: Ross Lagerwall Add support for handling bug frames contained with xsplice modules. If a trap occurs search either the kernel bug table or an applied payload's bug table depending on the instruction pointer. We also include a test-case - which will test the function that was patched to make

[Xen-devel] [PATCH v5 05/28] xsplice: Design document

2016-03-24 Thread Konrad Rzeszutek Wilk
A mechanism is required to binarily patch the running hypervisor with new opcodes that have come about due to primarily security updates. This document describes the design of the API that would allow us to upload to the hypervisor binary patches. This document has been shaped by the input from:

[Xen-devel] [PATCH v5 06/28] xen/xsplice: Hypervisor implementation of XEN_XSPLICE_op

2016-03-24 Thread Konrad Rzeszutek Wilk
The implementation does not actually do any patching. It just adds the framework for doing the hypercalls, keeping track of ELF payloads, and the basic operations: - query which payloads exist, - query for specific payloads, - check*1, apply*1, replace*1, and unload payloads. *1: Which of cour

[Xen-devel] [PATCH v5 01/28] HYPERCALL_version_op. New hypercall mirroring XENVER_ but sane.

2016-03-24 Thread Konrad Rzeszutek Wilk
This hypercall mirrors the XENVER_ in that it has similar functionality. However it is designed differently: - No compat layer. The data structures are the same size on 32 as on 64-bit. - The hypercall accepts three arguments - the command, pointer to an buffer, and the length of the buffer

[Xen-devel] [PATCH v5 23/28] xsplice: Stacking build-id dependency checking.

2016-03-24 Thread Konrad Rzeszutek Wilk
We now expect that the ELF payloads be built with the --build-id. Also the .xsplice.deps section has to have the contents of the hypervisor (or a preceding payload) build-id. We already have the code to verify the Elf_Note build-id so export parts of it. This dependency means the hypervisor MUST

[Xen-devel] [PATCH v5 10/28] xsplice: Implement payload loading

2016-03-24 Thread Konrad Rzeszutek Wilk
From: Ross Lagerwall Add support for loading xsplice payloads. This is somewhat similar to the Linux kernel module loader, implementing the following steps: - Verify the elf file. - Parse the elf file. - Allocate a region of memory mapped within a free area of [xen_virt_end, XEN_VIRT_END]. - Co

[Xen-devel] [PATCH v5 17/28] xsplice: Add support for exception tables.

2016-03-24 Thread Konrad Rzeszutek Wilk
From: Ross Lagerwall Add support for exception tables contained within xSplice payloads. If an exception occurs search either the main exception table or a particular active payload's exception table depending on the instruction pointer. Also we add an test-case to make sure we have an exception

[Xen-devel] [PATCH v5 25/28] xsplice: Print dependency and payloads build_id in the keyhandler.

2016-03-24 Thread Konrad Rzeszutek Wilk
Signed-off-by: Konrad Rzeszutek Wilk --- Cc: Ian Jackson Cc: Jan Beulich Cc: Keir Fraser Cc: Tim Deegan --- --- xen/common/xsplice.c | 5 + 1 file changed, 5 insertions(+) diff --git a/xen/common/xsplice.c b/xen/common/xsplice.c index d7a65fe..d5d4b3c 100644 --- a/xen/common/xsplice.c ++

[Xen-devel] [PATCH v5 03/28] arm/x86: Use struct virtual_region to do bug, symbol, and (x86) exception tables lookup.

2016-03-24 Thread Konrad Rzeszutek Wilk
During execution of the hypervisor we have two regions of executable code - stext -> _etext, and _sinittext -> _einitext. The later is not needed after bootup. We also have various built-in macros and functions to search in between those two swaths depending on the state of the system. That is e

[Xen-devel] [PATCH v5 18/28] xsplice: Add support for alternatives

2016-03-24 Thread Konrad Rzeszutek Wilk
From: Ross Lagerwall Add support for applying alternative sections within xsplice payload. At payload load time, apply an alternative sections that are found. Also we add an test-case exercising a rather useless alternative (patching a NOP with a NOP) - but it does exercise the code-path. Signe

[Xen-devel] [PATCH v5 21/28] libxl: info: Display build_id of the hypervisor using XEN_VERSION_build_id

2016-03-24 Thread Konrad Rzeszutek Wilk
If the hypervisor is built with we will display it. Signed-off-by: Konrad Rzeszutek Wilk Acked-by: Wei Liu --- Cc: Ian Jackson Cc: Stefano Stabellini Cc: Wei Liu v2: Include HAVE_*, use libxl_zalloc, s/rc/ret/ v3: Retry with different size if 1020 is not enough. v4: Use VERSION_OP subops in

[Xen-devel] [PATCH v5 12/28] x86/xen_hello_world.xsplice: Test payload for patching 'xen_extra_version'.

2016-03-24 Thread Konrad Rzeszutek Wilk
This change demonstrates how to generate an xSplice ELF payload. The idea here is that we want to patch in the hypervisor the 'xen_version_extra' function with an function that will return 'Hello World'. The 'xl info | grep extraversion' will reflect the new value after the patching. To generate

[Xen-devel] [PATCH v5 09/28] xsplice: Add helper elf routines

2016-03-24 Thread Konrad Rzeszutek Wilk
From: Ross Lagerwall Add Elf routines and data structures in preparation for loading an xSplice payload. We make an assumption that the max number of sections an ELF payload can have is 64. We can in future make this be dependent on the names of the sections and verifying against a list, but for

Re: [Xen-devel] [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu

2016-03-24 Thread Wei Liu
On Thu, Mar 24, 2016 at 06:58:44PM +0100, Juergen Gross wrote: > On 24/03/16 15:03, Wei Liu wrote: > > On Tue, Mar 22, 2016 at 08:36:21AM +0100, Juergen Gross wrote: > >> Committers, > >> > >> anything missing on my side to have this series being committed? > >> All patches have Acked-by and Review

Re: [Xen-devel] [PATCH v4 03/34] xsm/xen_version: Add XSM for the xen_version hypercall

2016-03-24 Thread Konrad Rzeszutek Wilk
> >diff --git a/xen/xsm/flask/policy/security_classes > >b/xen/xsm/flask/policy/security_classes > >index ca191db..cde4e1a 100644 > >--- a/xen/xsm/flask/policy/security_classes > >+++ b/xen/xsm/flask/policy/security_classes > >@@ -18,5 +18,6 @@ class shadow > > class event > > class grant > > c

[Xen-devel] [xen-unstable-smoke test] 87212: tolerable all pass - PUSHED

2016-03-24 Thread osstest service owner
flight 87212 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/87212/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 12 migrate-support-checkfail never pass test-armhf-armhf-xl 12

Re: [Xen-devel] [PATCH v4 00/14] Load BIOS via toolstack instead of been embedded in hvmloader.

2016-03-24 Thread Jim Fehlig
Anthony PERARD wrote: > Hi all, > > Few changes in V4: > I leave the ACPI alone and only load the BIOS via libxl now. > > Detail of changes in each patches. > > I've look at loading the BIOS via the toolstack instead of having them > embedded > in the hvmloader binary. After this patch seri

Re: [Xen-devel] [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu

2016-03-24 Thread Juergen Gross
On 24/03/16 15:03, Wei Liu wrote: > On Tue, Mar 22, 2016 at 08:36:21AM +0100, Juergen Gross wrote: >> Committers, >> >> anything missing on my side to have this series being committed? >> All patches have Acked-by and Reviewed-by tags. >> > > Can you provide a branch with all the tags folded in?

  1   2   3   >