[Xen-devel] [xen-unstable test] 77209: regressions - trouble: broken/fail/pass

2016-01-07 Thread osstest service owner
flight 77209 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/77209/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-xsm   3 host-install(3) broken REGR. vs. 66879
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 9 debian-hvm-install fail 
REGR. vs. 66879

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-rumpuserxen-i386 10 guest-startfail like 66879
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail like 66879
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail like 66879
 test-amd64-amd64-libvirt-vhd  9 debian-di-installfail   like 66879

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop  fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd   9 debian-di-installfail   never pass

version targeted for testing:
 xen  ce0fac22e7f367ba72ebd762331f8c9bdf1e2519
baseline version:
 xen  bf925a9f1254391749f569c1b8fc606036340488

Last test of basis66879  2015-12-21 21:25:56 Z   16 days
Failing since 67053  2015-12-23 06:54:21 Z   15 days4 attempts
Testing same since77209  2016-01-06 04:21:37 Z1 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Feng Wu 
  Ian Campbell 
  Ian Jackson 
  Jan Beulich 
  Julien Grall 
  Kevin Tian 
  Samuel Thibault 
  Wei Liu 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-oldkern  pass
 build-i386-oldkern   pass
 build-amd64-prev pass
 build-i386-prev  pass
 build-amd64-pvops

Re: [Xen-devel] [PATCH v3 59/62] xen/arm: Add a hypercall for device mmio mapping

2016-01-07 Thread Shannon Zhao
Hi Jan,

On 2016/1/7 15:45, Jan Beulich wrote:
 On 07.01.16 at 07:58,  wrote:
>> > On 2015/11/17 19:04, Jan Beulich wrote:
>> > On 17.11.15 at 10:40,  wrote:
> >>> > --- a/xen/arch/arm/mm.c
> >>> > +++ b/xen/arch/arm/mm.c
> >>> > @@ -1138,6 +1138,10 @@ int xenmem_add_to_physmap_one(
> >>> >  rcu_unlock_domain(od);
> >>> >  break;
> >>> >  }
> >>> > +case XENMAPSPACE_dev_mmio:
> >>> > +rc = map_dev_mmio_region(d, gpfn, 1, idx);
> >>> > +return rc;
> >>> > +break;
>>> >> Blindly for any kind of domain? The XSM check in the
>>> >> XENMEM_add_to_physmap_batch handler (in common code) doesn't
>>> >> even know which map space is to be used...
>> > 
>> > Sorry, I know little about XSM. Could you suggest me how to add the
>> > check for this new type here?
> I'm sorry to push back here, but did you at least try to derive
> what is wanted from the multitude of other XSM checks present
> throughout the tree?

IIUC, you mean that it doean't need to change the XSM check itself, but
we should check if the current->domain is hardware domain and it maps
the space to itself before the XSM check, right?

If so, how about below patch on top of this patch?

diff --git a/xen/common/memory.c b/xen/common/memory.c
index 9ff1145..33feb2d 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -980,6 +980,13 @@ long do_memory_op(unsigned long cmd,
XEN_GUEST_HANDLE_PARAM(void) arg)
 if ( d == NULL )
 return -ESRCH;

+   /* Device MMIO mapping is only supported for Domain0 to map
these ranges
+* to itself
+*/
+if ( (xatp.space == XENMAPSPACE_dev_mmio) &&
+ (!is_hardware_domain(current->domain) || (d !=
current->domain)) )
+return -EOPNOTSUPP;
+
 rc = xsm_add_to_physmap(XSM_TARGET, current->domain, d);
 if ( rc )
 {
@@ -1024,6 +1031,13 @@ long do_memory_op(unsigned long cmd,
XEN_GUEST_HANDLE_PARAM(void) arg)
 if ( d == NULL )
 return -ESRCH;

+/* Device MMIO mapping is only supported for Domain0 to map
these ranges
+ * to itself
+ */
+if ( (xatpb.space == XENMAPSPACE_dev_mmio) &&
+ (!is_hardware_domain(current->domain) || (d !=
current->domain)) )
+return -EOPNOTSUPP;
+
  rc = xsm_add_to_physmap(XSM_TARGET, current->domain, d);
  if ( rc )
  {

Thanks,
-- 
Shannon


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Infiniband support

2016-01-07 Thread Ian Campbell
On Wed, 2016-01-06 at 06:54 +, Gohar Irfan wrote:
> Hi,
> 
> Can anyone guide me on how to compile Xen with Infiniband support?
> (Particularly Mellanox)
> I want to perform some RDMA read/write functionality from within the Xen
> code (it is for a course project) using the Verbs API. 

Xen itself doesn't typically contain I/O drivers at all, that is delegated
to the domain 0 or driver domain kernel(s).

IOW you need to be looking at how to compile Infiniband support into your
dom0 kernel, i.e. in Linux or BSD or whatever, I would presume there are
plenty of resources on that subject on the web.

Ian.


> 
> Thanks,
> Gohar
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH RFC LINUX v1] xen: arm: enable migration on ARM.

2016-01-07 Thread Ian Campbell
On Wed, 2016-01-06 at 17:55 +, Stefano Stabellini wrote:
> Please CC linux-arm for the non-RFC patches
> 
> On Wed, 9 Dec 2015, Ian Campbell wrote:
> > Replace various stub functions with real functionality, including
> > reestablishing the shared info page and the per-vcpu info pages on
> > restore.
> > 
> > Reestablishing the vcpu info page is a little subtle. The
> > VCPUOP_register_vcpu_info hypercall can only be called on either the
> > current VCPU or on an offline different VCPU. Since migration occurs
> > with all VCPUS online they are all therefore online at the point of
> > resume.
> > 
> > Therefore we must perform a cross VCPU call to each non-boot VCPU,
> > which cannot be done in the xen_arch_post_suspend() callback since
> > that is run from stop_machine() with interrupts disabled.
> > 
> > Furthermore VCPUOP_register_vcpu_info can only be called once per-VCPU
> > in a given domain, so it must not be called after a cancelled suspend
> > (which resumes in the same domain).
> > 
> > Therefore xen_arch_resume() gains a suspend_cancelled parameter and we
> > resume the secondary VCPUs there only if needed.
> 
> It is a bit complex but it seems better than what we do on x86, which
> is:

We cannot do this on ARM in any case, since we can't do VCPUOP_{up,down}
and we don't want to expose this. We might be able to do it via PCSI but
TBH I dislike this anyway since it defeats the purpose of migrating with
all VCPUs online, we may as well just bring them down before and bring them
back up afterwards (as we used to before the tools supported multivcpu
migration, i.e. the $dom/control/platform-feature-multiprocessor-suspend
node in xs).
> 
>   for_each_possible_cpu(cpu) {
>   bool other_cpu = (cpu != smp_processor_id());
>   bool is_up = HYPERVISOR_vcpu_op(VCPUOP_is_up, cpu, NULL);
> 
>   if (other_cpu && is_up &&
>   HYPERVISOR_vcpu_op(VCPUOP_down, cpu, NULL))
>   BUG();
> 
>   xen_setup_runstate_info(cpu);
> 
>   if (have_vcpu_info_placement)
>   xen_vcpu_setup(cpu);
> 
>   if (other_cpu && is_up &&
>   HYPERVISOR_vcpu_op(VCPUOP_up, cpu, NULL))
>   BUG();
>   }
> 
> 
> > +void xen_arch_post_suspend(int suspend_cancelled)
> > +{
> > +   xen_register_shared_info();
> > +   if (!suspend_cancelled)
> > +   xen_vcpu_restore();
> 
> could we wait and call xen_vcpu_restore for cpu0 from
> xen_vcpu_notify_resume?

I've been through almost every permutation of where this stuff can be done.
IIRC we need to take some events on this vcpu before then, else we get
wedged before the notify.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [qemu-upstream-4.2-testing test] 77180: regressions - FAIL

2016-01-07 Thread Ian Campbell
On Wed, 2016-01-06 at 18:28 +, osstest service owner wrote:
> flight 77180 qemu-upstream-4.2-testing real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/77180/
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  build-i3865 xen-build fail REGR. vs. 
> 62044
>  build-amd64   5 xen-build fail REGR. vs. 
> 62044

This is "man/xl.pod.1 around line 854: Expected text after =item, not a
bullet" exposed by the Jessie upgrade.

However per Ian in <22154.35021.750846.695...@mariner.uk.xensource.com> [0]
:

...] 4.2 has had no commits since October - in particular, none
of the recent security fixes - and I would be reluctant to give it a
veneer of activity.

So our choices WRT these fixes in qemu-xen.git#staging-4.2, given they have
already been pushed, are:

   1. Fix xen.git#staging-4.2 to build on Jessie and wait for it to propagate.
   2. Revert the fixes from qemu-xen.git#staging-4.2 and force push the
  resulting tree (which would be identical to something which previously
  passed).
   3. Rollback qemu-xen.git#staging-4.2.
   4. Force push.
   5. Drop a stop file.
   6. Shave yakks in osstest to allow per-branch selection of the Debian suite
  and pin xen-4.2 and earlier to Wheezy.

#1 is contrary to the quote above, which makes a reasonable argument IMHO.

#3, #4 and #5 all seem like bad ideas to me.

#2 is a bit odd (to have the fixes in the branch but reverted), but seems
least bad compared with #3..#5.

#6 is potentially a lot of work, but might be the right long term answer.

Ian.

[0] http://lists.xenproject.org/archives/html/xen-devel/2016-01/msg00112.html
> 
> Tests which did not succeed, but are not blocking:
>  build-i386-libvirt1 build-
> check(1)   blocked  n/a
>  test-amd64-i386-xl-qemuu-win7-amd64  1 build-
> check(1)  blocked n/a
>  test-amd64-i386-qemuu-rhel6hvm-intel  1 build-
> check(1) blocked n/a
>  test-i386-i386-xl-qemuu-winxpsp3  1 build-
> check(1)   blocked  n/a
>  test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-
> check(1)  blocked n/a
>  test-amd64-i386-qemuu-rhel6hvm-amd  1 build-
> check(1)   blocked n/a
>  test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  1 build-
> check(1) blocked n/a
>  test-amd64-i386-xend-qemuu-winxpsp3  1 build-
> check(1)  blocked n/a
>  test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-
> check(1) blocked n/a
>  build-amd64-libvirt   1 build-
> check(1)   blocked  n/a
>  test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-
> check(1)blocked n/a
>  test-amd64-amd64-xl-qemuu-win7-amd64  1 build-
> check(1) blocked n/a
>  test-amd64-amd64-xl-qemuu-winxpsp3  1 build-
> check(1)   blocked n/a
>  test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-
> check(1) blocked n/a
> 
> version targeted for testing:
>  qemuu5081fc1c773d2a83ec7a867f030323b8b6956cd1
> baseline version:
>  qemuuc17e602ae64fb24405ebd256679ba9a6f5819086
> 
> Last test of basis62044  2015-09-15 15:06:42 Z  113 days
> Testing same since66542  2015-12-18 16:37:10 Z   19 days   11
> attempts
> 
> 
> People who touched revisions under test:
>   Stefano Stabellini 
> 
> jobs:
>  build-amd64  fail
>  build-i386   fail
>  build-amd64-libvirt  blocked 
>  build-i386-libvirt   blocked 
>  build-amd64-pvopspass
>  build-i386-pvops pass
>  test-amd64-i386-qemuu-rhel6hvm-amd   blocked 
>  test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
>  test-amd64-i386-xl-qemuu-debianhvm-amd64 blocked 
>  test-amd64-amd64-xl-qemuu-ovmf-amd64 blocked 
>  test-amd64-i386-xl-qemuu-ovmf-amd64  blocked 
>  test-amd64-amd64-xl-qemuu-win7-amd64 blocked 
>  test-amd64-i386-xl-qemuu-win7-amd64  blocked 
>  test-amd64-i386-qemuu-rhel6hvm-intel blocked 
>  test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 blocked 
>  test-amd64-i386-xend-qemuu-winxpsp3  blocked 
>  test-amd64-amd64-xl-qemuu-winxpsp3   blocked 
>  test-i386-i386-xl-qemuu-winxpsp3 blocked 
> 
> 
> 
> sg-report-flight on osstest.test-lab.xenproject.org
> logs: /home/logs/logs
> images: /home/logs/images
> 
> Logs, config files, etc. are availabl

Re: [Xen-devel] [PATCH 1/2] vm_event: sync domctl

2016-01-07 Thread Ian Campbell
On Wed, 2016-01-06 at 19:29 +0100, Tamas K Lengyel wrote:
> 
> 
> On Wed, Jan 6, 2016 at 4:48 PM, Ian Campbell 
> wrote:
> > On Wed, 2015-12-23 at 15:53 +0100, Tamas K Lengyel wrote:
> > > Introduce new vm_event domctl option which allows an event subscriber
> > > to request all vCPUs not currently pending a vm_event request to be
> > > paused,
> > > thus allowing the subscriber to sync up on the state of the domain.
> > This
> > > is especially useful when the subscribed wants to disable certain
> > events
> > > from being delivered and wants to ensure no more requests are pending
> > on
> > > the
> > > ring before doing so.
> > >
> > > Cc: Ian Jackson 
> > > Cc: Stefano Stabellini 
> > > Cc: Ian Campbell 
> > > Cc: Wei Liu 
> > > Cc: Razvan Cojocaru 
> > > Signed-off-by: Tamas K Lengyel 
> > > ---
> > >  tools/libxc/include/xenctrl.h | 11 +++
> > >  tools/libxc/xc_vm_event.c | 16 
> > 
> > Tools side is pretty trivial, assuming there is agreement on the
> > underlying
> > hypercall interface:
> > 
> > Acked-by: Ian Campbell 
> Thanks, we've decided that this patch is actually not needed as the pause
> reference count is already good enough.

OK, thanks.

> > > +/***
> > >   * Memory sharing operations.
> > 
> > Do you also maintain this? If so do you fancy sending a patch to fix:
> > 
> > >   *
> > >   * Unles otherwise noted, these calls return 0 on succes, -1 and
> > errno on
> > 
> > "Unless" and "success" ?
> > 
> Sure, that could be done in a separate patch.

Yes, that's what I intended.

>  IMHO the whole sharing subsystem could use a cleanup series of its own
> to fix things like this, style issues and whatnot.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2 05/13] libxl: move xen-init-dom0 to tools/helpers

2016-01-07 Thread Ian Campbell
On Thu, 2016-01-07 at 07:15 +0100, Juergen Gross wrote:
> On 06/01/16 17:12, Ian Campbell wrote:
> > On Fri, 2015-12-18 at 14:14 +0100, Juergen Gross wrote:
> > > Move xen-init-dom0 from tools/libxl to tools/helpers, as it is just a
> > > helper program.
> > > 
> > > Signed-off-by: Juergen Gross 
> > > ---
> > >  tools/helpers/Makefile   | 10 ++
> > >  tools/{libxl => helpers}/xen-init-dom0.c |  0
> > >  tools/libxl/Makefile | 14 +++---
> > >  3 files changed, 13 insertions(+), 11 deletions(-)
> > >  rename tools/{libxl => helpers}/xen-init-dom0.c (100%)
> > > 
> > > diff --git a/tools/helpers/Makefile b/tools/helpers/Makefile
> > > index 52347fd..92aead4 100644
> > > --- a/tools/helpers/Makefile
> > > +++ b/tools/helpers/Makefile
> > > @@ -5,10 +5,16 @@
> > >  XEN_ROOT = $(CURDIR)/../..
> > >  include $(XEN_ROOT)/tools/Rules.mk
> > >  
> > > +PROGS += xen-init-dom0
> > >  ifeq ($(CONFIG_Linux),y)
> > >  PROGS += init-xenstore-domain
> > >  endif
> > >  
> > > +XEN_INIT_DOM0_OBJS = xen-init-dom0.o
> > > +$(XEN_INIT_DOM0_OBJS): CFLAGS += $(CFLAGS_libxenctrl)
> > 
> > I think the only use of this was for the xtl_* interfaces, which are
> > now in
> > libxentoollog.h (with a compat include via xenctrl.h). Would you mind
> > switching the tool over to use xentoollog directly (perhaps in a
> > separate
> > patch)?
> 
> Will do.

Thanks!

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2 03/13] libxl: provide a function to retrieve the xenstore domain id

2016-01-07 Thread Ian Campbell
On Thu, 2016-01-07 at 06:37 +0100, Juergen Gross wrote:
> On 06/01/16 16:59, Ian Campbell wrote:
> > On Fri, 2015-12-18 at 15:10 +0100, Juergen Gross wrote:
> > > On 18/12/15 14:53, Andrew Cooper wrote:
> > > > On 18/12/15 13:14, Juergen Gross wrote:
> > > > > Add libxl_xenstore_domid() to obtain the domain id of the
> > > > > xenstore
> > > > > domain.
> > > > > 
> > > > > Signed-off-by: Juergen Gross 
> > > > 
> > > > What are the expected semantics here? Would you expect it to return
> > > > domid 0 for a traditional setup, or are you wanting to use it as a
> > > > "does
> > > > a xenstored domain exist" test?
> > > 
> > > The latter. It will be used in patch 13 to decide which domain to
> > > stop via "xl shutdown --all".
> > 
> > ITYM "not stop".
> 
> Well, yes, if you select which domains to stop you select which domains
> are not stopped, too. :-)
> 
> I don't mind either wording. :-)

In the sense you meant I think you want "domains".

> > libxl already has interfaces for getting info about a
> > domain, libxl_domain_info libxl_list_domain etc, it seems like this
> > property should be added to that data rather than introducing a special
> > purpose API to get it. Especially given that xl shutdown already calls
> > libxl_list_domain.
> 
> Okay, I can change it that way.

Thanks.

> > I'm unsure if (lib)xl ought to be special casing XS in this way, as
> > opposed
> > to adding a more generic concept such as e.g. permanent domains, which
> > would be true for the xs domain but also for other special domains in
> > the
> > future and could even be controlled by the user or toolstack (I'm
> > thinking
> > you might want to set the flag on a driver domain for example).
> 
> The xs domain has to be handled separately, as it is needed to run in
> order to be able to stop other domains in a clean way.
> 
> In case dom0 reboot will be supported we need two different reboot
> modes: one with a hypervisor reboot implying all domains will be
> stopped (including the xs domain), and one without hypervisor reboot
> implying that all domains not requiring dom0 to be up all time will
> still be running after dom0 is up again.
> 
> For stopping all domains before doing a hypervisor reboot, driver
> domains should be stopped, too, but they should be stopped _after_
> all other domains. And even then the xs domain should be still
> running when the driver domains are being stopped.

So what we have is in effect some sort of reboot ordering or priority and a
threshold depending on what sort of reboot the host as a whole is
undergoing?

> IMO the generic concept you are asking for should be added in a
> separate patch handling stopping (and possibly rebooting) driver
> domains in a clean way.

Since libxl has a stable API once we add something we need to continue
supporting it, so we cannot (easily/cleanly) switch an xs specific scheme
into a generic one later. That argues then for supporting the XS case via
the generic mechanism now, even if we don't implement the other cases.

> 
> 
> Juergen
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Infiniband support

2016-01-07 Thread Gohar Irfan
Thanks a lot for the response.

I am not sure if I asked it right: I already have Infiniband installed on
my kernel and it is working fine in the user space. However, I would like
to use the RDMA Verbs API inside the Xen code, like calling RDMA functions
to send/receive data. That requires Infiniband headers to be included in
the Xen code, but I'm have difficulty with that. So is there any sample
code of someone already calling Infiniband/RDMA functions within Xen code
or any guide on how to recompile Xen with custom headers?

Thanks a lot!
On Thu, Jan 7, 2016 at 2:34 PM Ian Campbell  wrote:

> On Wed, 2016-01-06 at 06:54 +, Gohar Irfan wrote:
> > Hi,
> >
> > Can anyone guide me on how to compile Xen with Infiniband support?
> > (Particularly Mellanox)
> > I want to perform some RDMA read/write functionality from within the Xen
> > code (it is for a course project) using the Verbs API.
>
> Xen itself doesn't typically contain I/O drivers at all, that is delegated
> to the domain 0 or driver domain kernel(s).
>
> IOW you need to be looking at how to compile Infiniband support into your
> dom0 kernel, i.e. in Linux or BSD or whatever, I would presume there are
> plenty of resources on that subject on the web.
>
> Ian.
>
>
> >
> > Thanks,
> > Gohar
> >
> > ___
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
>
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Xen 4.7 Development Update

2016-01-07 Thread Andrew Cooper
On 07/01/16 05:41, Haozhong Zhang wrote:
> Hi Wei,
>
> On 01/04/16 10:15, Wei Liu wrote:
> [...] 
>> = Projects =
>>
>> == Hypervisor == 
> [...]
>> === x86 === 
> [...]
>> == Toolstack == 
>   *  vNVDIMM support
> -  Haozhong Zhang
>
> This is another item I'm working on and would like to see in 4.7. Not
> quite sure if it belongs to only toolstack, because it also contains
> patches in hypervisor/x86 to enable new instructions, but the primary
> part of v1 patch series is in toolstack.

The hypervisor side is small, basically mechanical, and already ready IMO.

The toolstacks side is going to be the interesting^Wcomplicated part of
this.  I would class this as a toolstack project at this point.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Unexpected error:

2016-01-07 Thread Wei Liu
CC George (who does the packaging for CentOS)

BTW this problem is better directed to appropriate mailing list of
CentOS (centos-virt? I can't remember the exact name).

Wei.

On Thu, Jan 07, 2016 at 10:53:00AM +0200, Jānis Andersons | Failiem.lv wrote:
> xen_major  : 4
> xen_minor  : 4
> xen_extra  : .3-3.el6
> CentOS release 6.7 (Final)
> 
> After:
> xm usb-hc-create demo_win2012_r2 2 4
> xm usb-list demo_win2012_r2
> WARNING: xend/xm is deprecated.
> Idx BE  state usb-ver  BE-path
> 0   0   1 USB2.0  /local/domain/0/backend/vusb/2/0
> port 1:
> port 2:
> port 3:
> port 4:
> xm usb-list-assignable-devices
> WARNING: xend/xm is deprecated.
> 1-1  : ID 090c:1000 SMI Corporation USB DISK
> 1-10 : ID 14dd:0002 Peppercon AG Multidevice
> 
> 
> xm usb-attach demo_win2012_r2 0 1 1-1
> 
> WARNING: xend/xm is deprecated.
> Unexpected error: 
> 
> Please report to xen-devel@lists.xen.org
> Traceback (most recent call last):
>   File "/usr/sbin/xm", line 20, in 
> main.main(sys.argv)
>   File "/usr/lib64/python2.6/site-packages/xen/xm/main.py", line 3946, in
> main
> _, rc = _run_cmd(cmd, cmd_name, args)
>   File "/usr/lib64/python2.6/site-packages/xen/xm/main.py", line 3970, in
> _run_cmd
> return True, cmd(args)
>   File "/usr/lib64/python2.6/site-packages/xen/xm/main.py", line 3011, in
> xm_usb_attach
> if vusb_util.bus_is_assigned(bus):
>   File "/usr/lib64/python2.6/site-packages/xen/util/vusb_util.py", line 275,
> in bus_is_assigned
> raise UsbDeviceParseError("Can't get assignment status: (%s)." % bus)
> xen.util.vusb_util.UsbDeviceParseError: vusb: Error parsing USB device info:
> Can't get assignment status: (1-1).
> 
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v3 2/4] libxc: support of linear p2m list for migration of pv-domains

2016-01-07 Thread Wei Liu
On Thu, Jan 07, 2016 at 11:21:04AM +0100, Juergen Gross wrote:
> On 06/01/16 16:40, Wei Liu wrote:
> > On Wed, Dec 16, 2015 at 10:24:18AM +0100, Juergen Gross wrote:
> > [...]
> >> @@ -698,21 +868,19 @@ static int normalise_pagetable(struct xc_sr_context 
> >> *ctx, const uint64_t *src,
> >>  /* 32bit guests can only use the first 4 entries of their L3 
> >> tables.
> >>   * All other are potentially used by Xen. */
> >>  xen_first = 4;
> >> -xen_last = 512;
> >> +xen_last = 511;
> > 
> > Is this a bug fix in its own right?
> 
> Hmm, bug fix is too much. It is a harmonization with the change below
> using macros to set xen_last to 511 at maximum. In fact it doesn't
> matter, because xen_last is used in:
> 
>if ( i >= xen_first && i <= xen_last )
> 
> with i being in the range from 0 to 511.
> 

Yes, that's what I was thinking. There seemed to be an off-by-one error
in the code. But as you said, i is within [0,511] so the code is fine.

Could you add a words or two about this hunk in commit message please.

With that:

Reviewed-by: Wei Liu 

Wei.

> 
> Juergen
> 
> > 
> > Wei.
> > 
> >>  break;
> >>  
> >>  case XEN_DOMCTL_PFINFO_L2TAB:
> >>  /* It is hard to spot Xen mappings in a 32bit guest's L2.  
> >> Most
> >>   * are normal but only a few will have Xen mappings.
> >> - *
> >> - * 428 = (HYPERVISOR_VIRT_START_PAE >> 
> >> L2_PAGETABLE_SHIFT_PAE)&0x1ff
> >> - *
> >> - * ...which is conveniently unavailable to us in a 64bit 
> >> build.
> >>   */
> >> -if ( pte_to_frame(src[428]) == ctx->x86_pv.compat_m2p_mfn0 )
> >> +i = (HYPERVISOR_VIRT_START_X86_32 >> L2_PAGETABLE_SHIFT_PAE) 
> >> & 511;
> >> +if ( pte_to_frame(src[i]) == ctx->x86_pv.compat_m2p_mfn0 )
> >>  {
> >> -xen_first = 428;
> >> -xen_last = 512;
> >> +xen_first = i;
> >> +xen_last = (HYPERVISOR_VIRT_END_X86_32 >>
> >> +L2_PAGETABLE_SHIFT_PAE) & 511;
> >>  }
> >>  break;
> >>  }
> >> -- 
> >> 2.6.2
> >>
> > 
> > ___
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
> > 
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2 13/13] tools: don't stop xenstore domain when stopping dom0

2016-01-07 Thread Ian Campbell
On Thu, 2016-01-07 at 07:52 +0100, Juergen Gross wrote:
> On 06/01/16 17:33, Ian Campbell wrote:
> > On Fri, 2015-12-18 at 15:53 +0100, Juergen Gross wrote:
> > > On 18/12/15 15:42, Andrew Cooper wrote:
> > > > On 18/12/15 13:14, Juergen Gross wrote:
> > > > > When restarting or shutting down dom0 the xendomains script tries
> > > > > to
> > > > > stop all other domains. Don't do this for the xenstore domain, as
> > > > > it
> > > > > might survive a dom0 reboot in the future.
> > > > > 
> > > > > The same applies to xl shutdown --all.
> > > > > 
> > > > > Signed-off-by: Juergen Gross 
> > > > > ---
> > > > >  tools/hotplug/Linux/xendomains.in | 17 +
> > > > >  tools/libxl/xl_cmdimpl.c  | 19 +++
> > > > >  2 files changed, 32 insertions(+), 4 deletions(-)
> > > > > 
> > > > > diff --git a/tools/hotplug/Linux/xendomains.in
> > > > > b/tools/hotplug/Linux/xendomains.in
> > > > > index dfe0b33..70b7f16 100644
> > > > > --- a/tools/hotplug/Linux/xendomains.in
> > > > > +++ b/tools/hotplug/Linux/xendomains.in
> > > > > @@ -196,6 +196,17 @@ rdnames()
> > > > >  done
> > > > >  }
> > > > >  
> > > > > +# set xenstore domain id (or 0 if no xenstore domain)
> > > > > +get_xsdomid()
> > > > 
> > > > A get/set mismatch.
> > > 
> > > Hmm, depends.
> > > 
> > > It is getting the domid of the xenstore domain and is setting
> > > XS_DOMID accordingly. The main semantics are to get the correct
> > > domid.
> > > 
> > > > 
> > > > > +{
> > > > > +${bindir}/xenstore-exists /tool/xenstored/domid
> > > > > +if test $? -ne 0; then
> > > > > +XS_DOMID=0
> > > > > +else
> > > > > +XS_DOMID=`${bindir}/xenstore-read /tool/xenstored/domid`
> > 
> > Please update docs/misc/xenstore-paths.markdown with this.
> 
> Okay.
> 
> > 
> > Did you mean /tools?
> 
> No. The xenstore path is /tool/...

You mean that are preexisting uses of this path?

/me looks.

Oh, so there is. Undocumented too :-(

> > 
> > Earlier in the series there was a patch which looped over xc_dom_info
> > looking for the xs domain -- if this is in xenstore can't it use that?
> 
> Hen and egg problem. You need to know how to connect to xenstore
> (domain or daemon) before being able to read xenstore.

Oh, of course.

Can you not infer from the presence of absence of the sockets in the local
f/s or do they always exist (i.e. stale from a previous configuration)?

We did once try switching to always using the domain ring, even if the
client and server were co-located in the same domain, but that can result
in uninterruptible sleeps in the kernel IIRC (a bug which might have since
been fixed, not sure). Anyway, that probably rules out the "solution" of
always using the domain.

The daemon would drop a pid file, but I suppose that might also be stale.

I'm mostly just brainstorming here, I don't really have a problem with the
scan in the earlier patch.

(FWIW in English idiom we usually say chicken and egg BTW)

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v3 2/4] libxc: support of linear p2m list for migration of pv-domains

2016-01-07 Thread Juergen Gross
On 07/01/16 11:33, Wei Liu wrote:
> On Thu, Jan 07, 2016 at 11:21:04AM +0100, Juergen Gross wrote:
>> On 06/01/16 16:40, Wei Liu wrote:
>>> On Wed, Dec 16, 2015 at 10:24:18AM +0100, Juergen Gross wrote:
>>> [...]
 @@ -698,21 +868,19 @@ static int normalise_pagetable(struct xc_sr_context 
 *ctx, const uint64_t *src,
  /* 32bit guests can only use the first 4 entries of their L3 
 tables.
   * All other are potentially used by Xen. */
  xen_first = 4;
 -xen_last = 512;
 +xen_last = 511;
>>>
>>> Is this a bug fix in its own right?
>>
>> Hmm, bug fix is too much. It is a harmonization with the change below
>> using macros to set xen_last to 511 at maximum. In fact it doesn't
>> matter, because xen_last is used in:
>>
>>if ( i >= xen_first && i <= xen_last )
>>
>> with i being in the range from 0 to 511.
>>
> 
> Yes, that's what I was thinking. There seemed to be an off-by-one error
> in the code. But as you said, i is within [0,511] so the code is fine.
> 
> Could you add a words or two about this hunk in commit message please.

Sure.

> 
> With that:
> 
> Reviewed-by: Wei Liu 

Thanks,

Juergen


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v3 59/62] xen/arm: Add a hypercall for device mmio mapping

2016-01-07 Thread Jan Beulich
>>> On 07.01.16 at 10:11,  wrote:
> Hi Jan,
> 
> On 2016/1/7 15:45, Jan Beulich wrote:
> On 07.01.16 at 07:58,  wrote:
>>> > On 2015/11/17 19:04, Jan Beulich wrote:
>>> > On 17.11.15 at 10:40,  wrote:
>> >>> > --- a/xen/arch/arm/mm.c
>> >>> > +++ b/xen/arch/arm/mm.c
>> >>> > @@ -1138,6 +1138,10 @@ int xenmem_add_to_physmap_one(
>> >>> >  rcu_unlock_domain(od);
>> >>> >  break;
>> >>> >  }
>> >>> > +case XENMAPSPACE_dev_mmio:
>> >>> > +rc = map_dev_mmio_region(d, gpfn, 1, idx);
>> >>> > +return rc;
>> >>> > +break;
 >> Blindly for any kind of domain? The XSM check in the
 >> XENMEM_add_to_physmap_batch handler (in common code) doesn't
 >> even know which map space is to be used...
>>> > 
>>> > Sorry, I know little about XSM. Could you suggest me how to add the
>>> > check for this new type here?
>> I'm sorry to push back here, but did you at least try to derive
>> what is wanted from the multitude of other XSM checks present
>> throughout the tree?
> 
> IIUC, you mean that it doean't need to change the XSM check itself, but
> we should check if the current->domain is hardware domain and it maps
> the space to itself before the XSM check, right?

No, I actually think that you need to add a new, secondary XSM
check. But you may want to consult with Daniel (who so far wasn't
even Cc-ed).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] RFC Userspace hypercalls

2016-01-07 Thread Andrew Cooper
On 07/01/16 10:42, Ian Campbell wrote:
> On Wed, 2016-01-06 at 11:44 +, Andrew Cooper wrote:
>> All console logging is synchronous (to ensure that log messages have
>> escaped the VM before an action occurs) and by default, an HVM test will
>> use the qemu debug port, console_io hypercall, and PV console (which
>> uses evtchn hypercalls).
> All three simultaneously, or it picks one depending on the scenario?

Currently all three (for simplicity), but I want to make the precise
setup configurable.

>
>> There are already scenarios under test where we cannot rely on the test
>> kernel having a fully functioning set of entry points (e.g. the DPL part
>> of the test above).  Therefore I specifically want to make it possible
>> to make userspace hypercalls, rather than simply making them possible to
>> be trapped-and-forwarded.
> And in these test cases there is useful logging to be done between the
> break the world and repair the world phases which I suppose follows if
> things didn't crash?

Precisely.

>
>> As a result, I proposing introducing a hypercall which allows a domain
>> to adjust its entry criteria for hypercalls (e.g. set_hypercall_iopl). 
>> Doing this for HVM guests is straight forward, but PV guests are harder,
>> as they bounce through Xen entrypoints.
>>
>> For PV guests, I propose that userspace hypercalls get implemented with
>> the int $0x82 path exclusively.  i.e. enabling userspace hypercalls
>> causes the hypercall page writing logic to consider the guest a ring1
>> kernel, and the int $0x82 entrypoint suitably delegates between a
>> regular hypercall and a compat hypercall.
>>
>> Thoughts?
> Would a xenconsoled mode which polls for updates (on specific guests only),
> along with the guest spinning waiting for the cons pointer to catch the
> prod one if it cares about synchronous logging be sufficient for this use
> case?

The framework already waits for cons to catch prod.

>
> Other random ideas:
> Implement the debug io port for PV guests too
> Log to a in guest buffer, as David suggested, possibly use xenaccess or
> similar to trap updates or as a doorbell.

Specifically not.  I have been bitten by that one too many times already.

In the case of XSA regression tests, or indeed the random x86
instruction executor which discovered XSA-44, the logging needs to have
escaped the host before the action is taken, or it all gets lost in a
host crash.

This is why console_io hypercalls are also used.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2 03/13] libxl: provide a function to retrieve the xenstore domain id

2016-01-07 Thread Ian Campbell
On Thu, 2016-01-07 at 11:44 +0100, Juergen Gross wrote:
> > > IMO the generic concept you are asking for should be added in a
> > > separate patch handling stopping (and possibly rebooting) driver
> > > domains in a clean way.
> > 
> > Since libxl has a stable API once we add something we need to continue
> > supporting it, so we cannot (easily/cleanly) switch an xs specific
> > scheme
> > into a generic one later. That argues then for supporting the XS case
> > via
> > the generic mechanism now, even if we don't implement the other cases.
> 
> I can't see a scenario where the xenstore domain would have to be
> stopped by dom0. Once you do it you'll never be able to connect to
> it again without changing the xenbus driver interface, too. It is
> the same reason why xenstored can't be restarted.
> 
> Driver domains are different and I think the interface to query a
> domain whether it is a driver domain or whether it might survive a
> dom0 reboot should be based on xenstore.
> 
> So a xenstore domain would always need special handling.

If there is really _never_ any reason to stop the xs domain then I think at
the libxl API level a class of "never stop" domains would be better than
special casing the xs, even if it turns out the only member of the set is
xs at least we've given ourselves wriggle room if something else comes up
in the future.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2 03/13] libxl: provide a function to retrieve the xenstore domain id

2016-01-07 Thread Juergen Gross
On 07/01/16 11:55, Ian Campbell wrote:
> On Thu, 2016-01-07 at 11:44 +0100, Juergen Gross wrote:
 IMO the generic concept you are asking for should be added in a
 separate patch handling stopping (and possibly rebooting) driver
 domains in a clean way.
>>>
>>> Since libxl has a stable API once we add something we need to continue
>>> supporting it, so we cannot (easily/cleanly) switch an xs specific
>>> scheme
>>> into a generic one later. That argues then for supporting the XS case
>>> via
>>> the generic mechanism now, even if we don't implement the other cases.
>>
>> I can't see a scenario where the xenstore domain would have to be
>> stopped by dom0. Once you do it you'll never be able to connect to
>> it again without changing the xenbus driver interface, too. It is
>> the same reason why xenstored can't be restarted.
>>
>> Driver domains are different and I think the interface to query a
>> domain whether it is a driver domain or whether it might survive a
>> dom0 reboot should be based on xenstore.
>>
>> So a xenstore domain would always need special handling.
> 
> If there is really _never_ any reason to stop the xs domain then I think at
> the libxl API level a class of "never stop" domains would be better than
> special casing the xs, even if it turns out the only member of the set is
> xs at least we've given ourselves wriggle room if something else comes up
> in the future.

Okay, so this would translate to either:

- add a "never stop" flag to libxl_dominfo (can I do this without
  breaking the API?)
- add a new call interface to either check a single domain to be of
  the "never stop" class or to return a list of those domains.

Preferences?


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] Which trees are supported (Was: Re: [qemu-upstream-4.2-testing test] 77180: regressions - FAIL)

2016-01-07 Thread Ian Campbell
So this arose because Stefano was unaware that 4.2 was no longer supported.
Neither am I ever confident about where the cut-off lie, e.g. I always have
to ask if I am doing backports for a security issue.

We should add rows to http://wiki.xen.org/wiki/Xen_Release_Features right
under Initial Release giving first the date until which that tree is
supported with backports and second the date until which security support
will exist. We might also want to add a third "status" row. e.g.
"Supported", "Security Support only", "EOL" (we'll deal with extended
support by a third party when that next arises).

I'm happy to make the edits, however I don't know what dates I would write 
here. Taking it to be 18 months of Support and a further 18 months of
security support I would get:

    Xen 4.0 Xen 4.1 Xen 4.2 Xen 4.3 
Xen 4.4 Xen 4.5 Xen 4.6
Initial Release 7 April 2010    25 March 2011   17 Sept 2012    9 July 
2013 10 March 2014   15 Jan 2015 13 Oct 2015 
Supported until EOL - ???   EOL - ???   EOL - ???   EOL - 
Jan 2015  EOL - Sept 2015 July 2016   April 2017
Security support tilEOL - ???   EOL - ???   EOL - ???   July 
2016   March 2016  Jan 2017Oct 2018

(maybe those EOLs - ??? could be whatever the respective dates were, I
didn't try and backtrack to try and find out if reality matched the plan)

Ian.

On Thu, 2016-01-07 at 09:56 +, Ian Campbell wrote:
> On Wed, 2016-01-06 at 18:28 +, osstest service owner wrote:
> > flight 77180 qemu-upstream-4.2-testing real [real]
> > http://logs.test-lab.xenproject.org/osstest/logs/77180/
> > 
> > Regressions :-(
> > 
> > Tests which did not succeed and are blocking,
> > including tests which could not be run:
> >  build-i3865 xen-build fail REGR.
> > vs. 62044
> >  build-amd64   5 xen-build fail REGR.
> > vs. 62044
> 
> This is "man/xl.pod.1 around line 854: Expected text after =item, not a
> bullet" exposed by the Jessie upgrade.
> 
> However per Ian in <22154.35021.750846.695...@mariner.uk.xensource.com> [
> 0]
> :
> 
> ...] 4.2 has had no commits since October - in particular, none
> of the recent security fixes - and I would be reluctant to give it a
> veneer of activity.
> 
> So our choices WRT these fixes in qemu-xen.git#staging-4.2, given they
> have
> already been pushed, are:
> 
>    1. Fix xen.git#staging-4.2 to build on Jessie and wait for it to
> propagate.
>    2. Revert the fixes from qemu-xen.git#staging-4.2 and force push the
>   resulting tree (which would be identical to something which
> previously
>   passed).
>    3. Rollback qemu-xen.git#staging-4.2.
>    4. Force push.
>    5. Drop a stop file.
>    6. Shave yakks in osstest to allow per-branch selection of the Debian
> suite
>   and pin xen-4.2 and earlier to Wheezy.
> 
> #1 is contrary to the quote above, which makes a reasonable argument
> IMHO.
> 
> #3, #4 and #5 all seem like bad ideas to me.
> 
> #2 is a bit odd (to have the fixes in the branch but reverted), but seems
> least bad compared with #3..#5.
> 
> #6 is potentially a lot of work, but might be the right long term answer.
> 
> Ian.
> 
> [0] http://lists.xenproject.org/archives/html/xen-devel/2016-01/msg00112.
> html
> > 
> > Tests which did not succeed, but are not blocking:
> >  build-i386-libvirt1 build-
> > check(1)   blocked  n/a
> >  test-amd64-i386-xl-qemuu-win7-amd64  1 build-
> > check(1)  blocked n/a
> >  test-amd64-i386-qemuu-rhel6hvm-intel  1 build-
> > check(1) blocked n/a
> >  test-i386-i386-xl-qemuu-winxpsp3  1 build-
> > check(1)   blocked  n/a
> >  test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-
> > check(1)  blocked n/a
> >  test-amd64-i386-qemuu-rhel6hvm-amd  1 build-
> > check(1)   blocked n/a
> >  test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  1 build-
> > check(1) blocked n/a
> >  test-amd64-i386-xend-qemuu-winxpsp3  1 build-
> > check(1)  blocked n/a
> >  test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-
> > check(1) blocked n/a
> >  build-amd64-libvirt   1 build-
> > check(1)   blocked  n/a
> >  test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-
> > check(1)blocked n/a
> >  test-amd64-amd64-xl-qemuu-win7-amd64  1 build-
> > check(1) blocked n/a
> >  test-amd64-amd64-xl-qemuu-winxpsp3  1 build-
> > check(1)   blocked n/a
> >  test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-
> > check(1) blocked n/a
> > 
> > version targeted for testing:
> >  qemuu5081fc1c773d2a83ec7a867f030323b8b6956cd1
> > baseline version:
> >  qemuuc17e602ae64fb24405ebd256679ba9a6f5819086
> > 
> > Last test of basis62044  2015-09-15 15:06:42 Z  113 days
> > Testing same since66542  2015-12-18 16:37:10

[Xen-devel] [distros-debian-wheezy test] 38599: regressions - trouble: blocked/broken/pass

2016-01-07 Thread Platform Team regression test user
flight 38599 distros-debian-wheezy real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/38599/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i3864 host-build-prep   fail REGR. vs. 38576

Tests which did not succeed, but are not blocking:
 test-amd64-i386-amd64-wheezy-netboot-pygrub  1 build-check(1)  blocked n/a
 test-amd64-i386-i386-wheezy-netboot-pvgrub  1 build-check(1)   blocked n/a

baseline version:
 flight   38576

jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   broken  
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-amd64-wheezy-netboot-pvgrub pass
 test-amd64-i386-i386-wheezy-netboot-pvgrub   blocked 
 test-amd64-i386-amd64-wheezy-netboot-pygrub  blocked 
 test-amd64-amd64-i386-wheezy-netboot-pygrub  pass



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [linux-linus test] 77218: regressions - FAIL

2016-01-07 Thread osstest service owner
flight 77218 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/77218/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-freebsd10-i386  6 xen-bootfail REGR. vs. 59254
 test-amd64-i386-xl-qemut-winxpsp3  6 xen-boot fail REGR. vs. 59254
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-qemut-rhel6hvm-intel  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-freebsd10-amd64  6 xen-boot   fail REGR. vs. 59254
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-winxpsp3  6 xen-boot fail REGR. vs. 59254
 test-amd64-i386-qemuu-rhel6hvm-intel  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 6 xen-boot fail REGR. vs. 
59254
 test-amd64-i386-xl-qemut-win7-amd64  6 xen-boot   fail REGR. vs. 59254
 test-amd64-i386-xl-qemut-debianhvm-amd64  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-rumpuserxen-i386  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-debianhvm-amd64  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-win7-amd64  6 xen-boot   fail REGR. vs. 59254
 test-amd64-i386-xl-xsm6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-ovmf-amd64  6 xen-boot   fail REGR. vs. 59254
 test-armhf-armhf-xl-xsm   6 xen-boot  fail REGR. vs. 59254
 test-armhf-armhf-xl   6 xen-boot  fail REGR. vs. 59254
 test-armhf-armhf-xl-arndale   6 xen-boot  fail REGR. vs. 59254
 test-armhf-armhf-xl-credit2   6 xen-boot  fail REGR. vs. 59254
 test-armhf-armhf-xl-cubietruck  6 xen-bootfail REGR. vs. 59254
 test-armhf-armhf-xl-multivcpu  6 xen-boot fail REGR. vs. 59254
 test-amd64-i386-pair 10 xen-boot/dst_host fail REGR. vs. 59254
 test-amd64-i386-pair  9 xen-boot/src_host fail REGR. vs. 59254

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-libvirt   6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-libvirt-xsm   6 xen-boot  fail REGR. vs. 59254
 test-armhf-armhf-libvirt  6 xen-boot  fail REGR. vs. 59254
 test-armhf-armhf-xl-rtds  6 xen-boot  fail REGR. vs. 59254
 test-armhf-armhf-libvirt-xsm  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 6 xen-boot fail baseline 
untested
 test-amd64-i386-xl-raw6 xen-bootfail baseline untested
 test-armhf-armhf-libvirt-qcow2  6 xen-boot  fail baseline untested
 test-armhf-armhf-libvirt-raw  6 xen-bootfail baseline untested
 test-armhf-armhf-xl-vhd   6 xen-bootfail baseline untested
 test-amd64-i386-libvirt-pair 10 xen-boot/dst_host   fail baseline untested
 test-amd64-i386-libvirt-pair  9 xen-boot/src_host   fail baseline untested
 test-amd64-amd64-libvirt-vhd  9 debian-di-install   fail baseline untested
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 59254

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-xl-pvh-intel 14 guest-saverestorefail  never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail never pass
 test-amd64-amd64-qemuu-nested-intel 13 xen-boot/l1 fail never pass
 test-amd64-amd64-qemuu-nested-amd 13 xen-boot/l1   fail never pass

version targeted for testing:
 linuxee9a7d2cb0cf1a1498478bc923d911f3d9c910ac
baseline version:
 linux45820c294fe1b1a9df495d57f40585ef2d069a39

Last test of basis59254  2015-07-09 04:20:48 Z  182 days
Failing since 59348  2015-07-10 04:24:05 Z  181 days  116 attempts
Testing same since77218  2016-01-06 07:53:49 Z1 days1 attempts


3437 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd

[Xen-devel] Which trees are supported (Was: Re: [qemu-upstream-4.2-testing test] 77180: regressions - FAIL)

2016-01-07 Thread Jan Beulich
>>> On 07.01.16 at 12:22,  wrote:
> So this arose because Stefano was unaware that 4.2 was no longer supported.
> Neither am I ever confident about where the cut-off lie, e.g. I always have
> to ask if I am doing backports for a security issue.
> 
> We should add rows to http://wiki.xen.org/wiki/Xen_Release_Features right
> under Initial Release giving first the date until which that tree is
> supported with backports and second the date until which security support
> will exist. We might also want to add a third "status" row. e.g.
> "Supported", "Security Support only", "EOL" (we'll deal with extended
> support by a third party when that next arises).
> 
> I'm happy to make the edits, however I don't know what dates I would write 
> here. Taking it to be 18 months of Support and a further 18 months of
> security support I would get:
> 
>   Xen 4.0 Xen 4.1 Xen 4.2 Xen 4.3 
> Xen 4.4 Xen 4.5 Xen 4.6
> Initial Release   7 April 201025 March 2011   17 Sept 20129 July 
> 2013 10 March 
> 2014  15 Jan 2015 13 Oct 2015 
> Supported until   EOL - ???   EOL - ???   EOL - ???   
> EOL - Jan 2015  EOL - Sept 2015 July 
> 2016  April 2017
> Security support til  EOL - ???   EOL - ???   EOL - ???   July 
> 2016   March 2016  Jan 
> 2017  Oct 2018

4.4 is going to have normal support ended with the 4.4.4 release only;
4.4.3 got released a little too early from that perspective.

> (maybe those EOLs - ??? could be whatever the respective dates were, I
> didn't try and backtrack to try and find out if reality matched the plan)

At least for the older ones it's probably not worth to reconstruct. 4.2 had
its security support ended in Sept 2015.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v3 2/2] libxc: Defer initialization of start_page for HVM guests

2016-01-07 Thread Roger Pau Monné
El 06/01/16 a les 21.03, Boris Ostrovsky ha escrit:
> With commit 8c45adec18e0 ("libxc: create unmapped initrd in domain
> builder if supported") location of ramdisk may not be available to
> HVMlite guests by the time alloc_magic_pages_hvm() is invoked if the
> guest supports unmapped initrd.
> 
> So let's move ramdisk info initialization (along with a few other
> operations that are not directly related to allocating magic/special
> pages) from alloc_magic_pages_hvm() to bootlate_hvm().
> 
> Signed-off-by: Boris Ostrovsky 
> Acked-by: Wei Liu 
> ---
>  tools/libxc/xc_dom_x86.c |  116 ++---
>  1 files changed, 67 insertions(+), 49 deletions(-)
> 
> diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
> index b8d2904..e102bd2 100644
> --- a/tools/libxc/xc_dom_x86.c
> +++ b/tools/libxc/xc_dom_x86.c
> @@ -586,23 +586,12 @@ static void build_hvm_info(void *hvm_info_page, struct 
> xc_dom_image *dom)
>  static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
>  {
>  unsigned long i;
> -void *hvm_info_page;
>  uint32_t *ident_pt, domid = dom->guest_domid;
>  int rc;
>  xen_pfn_t special_array[X86_HVM_NR_SPECIAL_PAGES];
>  xen_pfn_t ioreq_server_array[NR_IOREQ_SERVER_PAGES];
>  xc_interface *xch = dom->xch;
>  
> -if ( dom->device_model )
> -{
> -if ( (hvm_info_page = xc_map_foreign_range(
> -  xch, domid, PAGE_SIZE, PROT_READ | PROT_WRITE,
> -  HVM_INFO_PFN)) == NULL )
> -goto error_out;
> -build_hvm_info(hvm_info_page, dom);
> -munmap(hvm_info_page, PAGE_SIZE);
> -}
> -
>  /* Allocate and clear special pages. */
>  for ( i = 0; i < X86_HVM_NR_SPECIAL_PAGES; i++ )
>  special_array[i] = special_pfn(i);
> @@ -637,12 +626,9 @@ static int alloc_magic_pages_hvm(struct xc_dom_image 
> *dom)
>  if ( !dom->device_model )
>  {
>  struct xc_dom_seg seg;
> -struct hvm_start_info *start_info;
> -char *cmdline;
>  struct hvm_modlist_entry *modlist;

This can be removed if the conditional below is changed to:

if ( dom->ramdisk_blob )
-   start_info_size += sizeof(*modlist);
+   start_info_size += sizeof(struct hvm_modlist_entry);

Because AFAICT the variable itself is not used for anything else.

> -void *start_page;
>  size_t cmdline_size = 0;
> -size_t start_info_size = sizeof(*start_info);
> +size_t start_info_size = sizeof(struct hvm_start_info);
>  
>  if ( dom->cmdline )
>  {
> @@ -661,39 +647,6 @@ static int alloc_magic_pages_hvm(struct xc_dom_image 
> *dom)
>  goto out;
>  }
>  
> -start_page = xc_map_foreign_range(xch, domid, start_info_size,
> -  PROT_READ | PROT_WRITE,
> -  seg.pfn);
> -if ( start_page == NULL )
> -{
> -DOMPRINTF("Unable to map HVM start info page");
> -goto error_out;
> -}
> -
> -start_info = start_page;
> -cmdline = start_page + sizeof(*start_info);
> -modlist = start_page + sizeof(*start_info) + cmdline_size;
> -
> -if ( dom->cmdline )
> -{
> -strncpy(cmdline, dom->cmdline, cmdline_size);
> -start_info->cmdline_paddr = (seg.pfn << PAGE_SHIFT) +
> -((uintptr_t)cmdline - (uintptr_t)start_info);
> -}
> -
> -if ( dom->ramdisk_blob )
> -{
> -modlist[0].paddr = dom->ramdisk_seg.vstart - 
> dom->parms.virt_base;
> -modlist[0].size = dom->ramdisk_seg.vend - 
> dom->ramdisk_seg.vstart;
> -start_info->modlist_paddr = (seg.pfn << PAGE_SHIFT) +
> -((uintptr_t)modlist - (uintptr_t)start_info);
> -start_info->nr_modules = 1;
> -}
> -
> -start_info->magic = HVM_START_MAGIC_VALUE;
> -
> -munmap(start_page, start_info_size);
> -
>  dom->start_info_pfn = seg.pfn;
>  }
>  else
> @@ -1783,7 +1736,72 @@ static int alloc_pgtables_hvm(struct xc_dom_image *dom)
>  
>  static int bootlate_hvm(struct xc_dom_image *dom)
>  {
> -DOMPRINTF("%s: doing nothing", __func__);
> +uint32_t domid = dom->guest_domid;
> +xc_interface *xch = dom->xch;
> +
> +if ( !dom->device_model )
> +{
> +struct hvm_start_info *start_info;
> +size_t start_info_size = sizeof(*start_info);
> +void *start_page;
> +struct hvm_modlist_entry *modlist;
> +size_t cmdline_size = 0;
> +
> +if ( dom->cmdline )
> +{
> +cmdline_size = ROUNDUP(strlen(dom->cmdline) + 1, 8);
> +start_info_size += cmdline_size;
> +}
> +if ( dom->ramdisk_blob )
> +start_info_size += sizeof(*modlist); /* Limited to one module. */

The size calculations are duplicated, could you eith

Re: [Xen-devel] Xen 4.7 Development Update

2016-01-07 Thread Jan Beulich
>>> On 07.01.16 at 11:21,  wrote:
> On 07/01/16 05:41, Haozhong Zhang wrote:
>> Hi Wei,
>>
>> On 01/04/16 10:15, Wei Liu wrote:
>> [...] 
>>> = Projects =
>>>
>>> == Hypervisor == 
>> [...]
>>> === x86 === 
>> [...]
>>> == Toolstack == 
>>   *  vNVDIMM support
>> -  Haozhong Zhang
>>
>> This is another item I'm working on and would like to see in 4.7. Not
>> quite sure if it belongs to only toolstack, because it also contains
>> patches in hypervisor/x86 to enable new instructions, but the primary
>> part of v1 patch series is in toolstack.
> 
> The hypervisor side is small, basically mechanical, and already ready IMO.

To be honest, I'm not convinced: I didn't look at the patches yet,
but with memory management being the hypervisor's job I'm not
sure we can get away with those mechanical changes alone.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Which trees are supported (Was: Re: [qemu-upstream-4.2-testing test] 77180: regressions - FAIL)

2016-01-07 Thread Ian Campbell
On Thu, 2016-01-07 at 04:45 -0700, Jan Beulich wrote:
> > > > On 07.01.16 at 12:22,  wrote:
> > So this arose because Stefano was unaware that 4.2 was no longer
> > supported.
> > Neither am I ever confident about where the cut-off lie, e.g. I always
> > have
> > to ask if I am doing backports for a security issue.
> > 
> > We should add rows to http://wiki.xen.org/wiki/Xen_Release_Features
> > right
> > under Initial Release giving first the date until which that tree is
> > supported with backports and second the date until which security
> > support
> > will exist. We might also want to add a third "status" row. e.g.
> > "Supported", "Security Support only", "EOL" (we'll deal with extended
> > support by a third party when that next arises).
> > 
> > I'm happy to make the edits, however I don't know what dates I would
> > write 
> > here. Taking it to be 18 months of Support and a further 18 months of
> > security support I would get:
> > 
> >     Xen 4.0 Xen 4.1 Xen 4.2 Xen 4.3 
> > Xen 4.4 Xen 4.5 Xen 4.6
> > Initial Release 7 April 2010    25 March 2011   17 Sept 2012    9 July 
> > 2013 10 March 
> > 2014    15 Jan 2015 13 Oct 2015 
> > Supported until EOL - ???   EOL - ???   EOL - ???   
> > EOL - Jan 2015  EOL - Sept 2015 July 
> > 2016April 2017
> > Security support tilEOL - ???   EOL - ???   EOL - ???   
> > July 2016   March 2016  Jan 
> > 2017Oct 2018
> 
> 4.4 is going to have normal support ended with the 4.4.4 release only;
> 4.4.3 got released a little too early from that perspective.

Meaning it will be earlier later than September 2015.

4.4.3 was released in August which was too soon.

I think it is right to err on the side of stopping later than we said.

Did we stop adding backports to staging-4.4 in September, i.e. is 4.4.4
going to be fixes from August-September + security issues until the release
date?

> 
> > (maybe those EOLs - ??? could be whatever the respective dates were, I
> > didn't try and backtrack to try and find out if reality matched the
> > plan)
> 
> At least for the older ones it's probably not worth to reconstruct. 4.2
> had
> its security support ended in Sept 2015.

Thanks.

> 
> Jan
> 
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v4 4/4] libxc: set flag for support of linear p2m list in domain builder

2016-01-07 Thread Juergen Gross
Set the SIF_VIRT_P2M_4TOOLS flag for pv-domUs in the domain builder
to indicate the Xen tools have full support for the virtual mapped
linear p2m list.

This will enable pv-domUs to drop support of the 3 level p2m tree
and use the linear list only. Without setting this flag some kernels
might limit themselves to 512 GB memory size in order not to break
migration.

Signed-off-by: Juergen Gross 
Reviewed-by: Andrew Cooper 
Acked-by: Wei Liu 
Acked-by: Ian Campbell 
---
 docs/features/migration.pandoc| 7 ---
 tools/libxc/xc_dom_compat_linux.c | 2 +-
 tools/libxc/xc_dom_core.c | 2 ++
 3 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/docs/features/migration.pandoc b/docs/features/migration.pandoc
index 9852a19..151c50d 100644
--- a/docs/features/migration.pandoc
+++ b/docs/features/migration.pandoc
@@ -1,5 +1,5 @@
 % Migration
-% Revision 1
+% Revision 2
 
 \clearpage
 
@@ -95,7 +95,6 @@ scenarios, which will involve starting with VMs from Xen 4.5
 # Areas for improvement
 
 * Arm support
-* Linear P2M support for x86 PV
 * Live looping parameters
 
 # Known issues
@@ -105,7 +104,8 @@ scenarios, which will involve starting with VMs from Xen 4.5
 * x86 HVM with nested-virt (no relevant information included in the
   stream)
 * x86 PV ballooning (P2M marked dirty, target frame not marked)
-* x86 PV P2M structure changes (not noticed, stale mappings used)
+* x86 PV P2M structure changes (not noticed, stale mappings used) for
+  guests not using the linear p2m layout
 
 # References
 
@@ -120,4 +120,5 @@ for Migration v2
 Date   Revision Version  Notes
 --   ---
 2015-10-24 1Xen 4.6  Document written
+2015-12-11 2Xen 4.7  Support of linear p2m list
 --   ---
diff --git a/tools/libxc/xc_dom_compat_linux.c 
b/tools/libxc/xc_dom_compat_linux.c
index abbc09f..c922c61 100644
--- a/tools/libxc/xc_dom_compat_linux.c
+++ b/tools/libxc/xc_dom_compat_linux.c
@@ -59,7 +59,7 @@ int xc_linux_build(xc_interface *xch, uint32_t domid,
  ((rc = xc_dom_ramdisk_file(dom, initrd_name)) != 0) )
 goto out;
 
-dom->flags = flags;
+dom->flags |= flags;
 dom->console_evtchn = console_evtchn;
 dom->xenstore_evtchn = store_evtchn;
 
diff --git a/tools/libxc/xc_dom_core.c b/tools/libxc/xc_dom_core.c
index 2061ba6..55c779d 100644
--- a/tools/libxc/xc_dom_core.c
+++ b/tools/libxc/xc_dom_core.c
@@ -777,6 +777,8 @@ struct xc_dom_image *xc_dom_allocate(xc_interface *xch,
 dom->parms.elf_paddr_offset = UNSET_ADDR;
 dom->parms.p2m_base = UNSET_ADDR;
 
+dom->flags = SIF_VIRT_P2M_4TOOLS;
+
 dom->alloc_malloc += sizeof(*dom);
 return dom;
 
-- 
2.6.2


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Which trees are supported (Was: Re: [qemu-upstream-4.2-testing test] 77180: regressions - FAIL)

2016-01-07 Thread Ian Campbell
On Thu, 2016-01-07 at 12:44 +, Lars Kurth wrote:
> > On 7 Jan 2016, at 11:45, Jan Beulich  wrote:
> > 
> > > > > On 07.01.16 at 12:22,  wrote:
> > > So this arose because Stefano was unaware that 4.2 was no longer
> > > supported.
> > > Neither am I ever confident about where the cut-off lie, e.g. I
> > > always have
> > > to ask if I am doing backports for a security issue.
> > > 
> > > We should add rows to http://wiki.xen.org/wiki/Xen_Release_Features
> > > right
> > > under Initial Release giving first the date until which that tree is
> > > supported with backports and second the date until which security
> > > support
> > > will exist. We might also want to add a third "status" row. e.g.
> > > "Supported", "Security Support only", "EOL" (we'll deal with extended
> > > support by a third party when that next arises).
> > > 
> > > I'm happy to make the edits, however I don't know what dates I would
> > > write 
> > > here. Taking it to be 18 months of Support and a further 18 months of
> > > security support I would get:
> 
> Ian, that would be great. Can you ping me when done?

Done.

I dropped the "EOL - " prefixes, since it seems that if we forget to add
them as new things are EOL'd then there would be ambiguity between things
marked "EOL - Date" and things marked as just "Date" where Date is in the
past -- i.e. folks might think the EOL didn't occur.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v4 1/3] public/io/netif.h: document transmit and receive wire formats separately

2016-01-07 Thread Paul Durrant
Currently there is no documented wire format for guest receive-side
packets but the location of the 'wire format' comment block suggests
it is the same as transmit-side. This is almost true but there is a
subtle difference in the use of the 'size' field for the first fragment.

For clarity this patch creates separate comment blocks for receive
and transmit side packet wire formats, tries to be more clear about the
distinction between 'fragments' and 'extras', and documents the subtlety
concerning the size field of the first fragment.

Signed-off-by: Paul Durrant 
Cc: Ian Campbell 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Keir Fraser 
Cc: Tim Deegan 
---
 xen/include/public/io/netif.h | 39 ++-
 1 file changed, 26 insertions(+), 13 deletions(-)

diff --git a/xen/include/public/io/netif.h b/xen/include/public/io/netif.h
index e103cf3..1790ea0 100644
--- a/xen/include/public/io/netif.h
+++ b/xen/include/public/io/netif.h
@@ -151,22 +151,22 @@
  */
 
 /*
- * This is the 'wire' format for packets:
- *  Request 1: netif_tx_request_t -- NETTXF_* (any flags)
- * [Request 2: netif_extra_info_t] (only if request 1 has
- *  NETTXF_extra_info)
- * [Request 3: netif_extra_info_t] (only if request 2 has
- *  XEN_NETIF_EXTRA_MORE)
- *  Request 4: netif_tx_request_t -- NETTXF_more_data
- *  Request 5: netif_tx_request_t -- NETTXF_more_data
- *  ...
- *  Request N: netif_tx_request_t -- 0
- */
-
-/*
  * Guest transmit
  * ==
  *
+ * This is the 'wire' format for packets:
+ *  Fragment 1: netif_tx_request_t  - flags = NETTXF_*
+ *size = total packet size
+ * [Extra 1: netif_extra_info_t]- (only if fragment 1 flags include
+ * NETTXF_extra_info)
+ * [Extra N: netif_extra_info_t]- (only if extra N-1 flags include
+ * XEN_NETIF_EXTRA_MORE)
+ *  ...
+ *  Fragment N: netif_tx_request_t  - (only if fragment N-1 flags include
+ * NETTXF_more_data)
+ *flags = 0
+ *size = fragment size
+ *
  * Ring slot size is 12 octets, however not all request/response
  * structs use the full size.
  *
@@ -202,6 +202,19 @@
  * Guest receive
  * =
  *
+ * This is the 'wire' format for packets:
+ *  Fragment 1: netif_rx_request_t  - flags = NETRXF_*
+ *size = fragment size
+ * [Extra 1: netif_extra_info_t]- (only if fragment 1 flags include
+ * NETRXF_extra_info)
+ * [Extra N: netif_extra_info_t]- (only if extra N-1 flags include
+ * XEN_NETIF_EXTRA_MORE)
+ *  ...
+ *  Fragment N: netif_rx_request_t  - (only if fragment N-1 flags include
+ * NETRXF_more_data)
+ *flags = 0
+ *size = fragment size
+ *
  * Ring slot size is 8 octets.
  *
  * rx request (netif_rx_request_t)
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Which trees are supported (Was: Re: [qemu-upstream-4.2-testing test] 77180: regressions - FAIL)

2016-01-07 Thread Jan Beulich
>>> On 07.01.16 at 13:00,  wrote:
> On Thu, 2016-01-07 at 04:45 -0700, Jan Beulich wrote:
>> > > > On 07.01.16 at 12:22,  wrote:
>> > So this arose because Stefano was unaware that 4.2 was no longer
>> > supported.
>> > Neither am I ever confident about where the cut-off lie, e.g. I always
>> > have
>> > to ask if I am doing backports for a security issue.
>> > 
>> > We should add rows to http://wiki.xen.org/wiki/Xen_Release_Features 
>> > right
>> > under Initial Release giving first the date until which that tree is
>> > supported with backports and second the date until which security
>> > support
>> > will exist. We might also want to add a third "status" row. e.g.
>> > "Supported", "Security Support only", "EOL" (we'll deal with extended
>> > support by a third party when that next arises).
>> > 
>> > I'm happy to make the edits, however I don't know what dates I would
>> > write 
>> > here. Taking it to be 18 months of Support and a further 18 months of
>> > security support I would get:
>> > 
>> >Xen 4.0 Xen 4.1 Xen 4.2 Xen 4.3 
>> > Xen 4.4 Xen 4.5 Xen 4.6
>> > Initial Release7 April 201025 March 2011   17 Sept 20129 July 
>> > 2013 10 
> March 
>> > 2014   15 Jan 2015 13 Oct 2015 
>> > Supported untilEOL - ???   EOL - ???   EOL - ???   
>> > EOL - Jan 2015  EOL - Sept 2015 July 
> 
>> > 2016   April 2017
>> > Security support til   EOL - ???   EOL - ???   EOL - ???   
>> > July 2016   March 2016  Jan 
>> > 2017   Oct 2018
>> 
>> 4.4 is going to have normal support ended with the 4.4.4 release only;
>> 4.4.3 got released a little too early from that perspective.
> 
> Meaning it will be earlier later than September 2015.
> 
> 4.4.3 was released in August which was too soon.
> 
> I think it is right to err on the side of stopping later than we said.
> 
> Did we stop adding backports to staging-4.4 in September, i.e. is 4.4.4
> going to be fixes from August-September + security issues until the release
> date?

No, there was active backporting until December (and I expect to at
least put in Andrew's XSA-156 fixup before 4.4.4 goes out, which -
due to the osstest issues - is going to take a little more time anyway).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2 03/13] libxl: provide a function to retrieve the xenstore domain id

2016-01-07 Thread Wei Liu
On Thu, Jan 07, 2016 at 11:36:08AM +, Ian Campbell wrote:
> On Thu, 2016-01-07 at 12:21 +0100, Juergen Gross wrote:
> > On 07/01/16 11:55, Ian Campbell wrote:
> > > On Thu, 2016-01-07 at 11:44 +0100, Juergen Gross wrote:
> > > > > > IMO the generic concept you are asking for should be added in a
> > > > > > separate patch handling stopping (and possibly rebooting) driver
> > > > > > domains in a clean way.
> > > > > 
> > > > > Since libxl has a stable API once we add something we need to
> > > > > continue
> > > > > supporting it, so we cannot (easily/cleanly) switch an xs specific
> > > > > scheme
> > > > > into a generic one later. That argues then for supporting the XS
> > > > > case
> > > > > via
> > > > > the generic mechanism now, even if we don't implement the other
> > > > > cases.
> > > > 
> > > > I can't see a scenario where the xenstore domain would have to be
> > > > stopped by dom0. Once you do it you'll never be able to connect to
> > > > it again without changing the xenbus driver interface, too. It is
> > > > the same reason why xenstored can't be restarted.
> > > > 
> > > > Driver domains are different and I think the interface to query a
> > > > domain whether it is a driver domain or whether it might survive a
> > > > dom0 reboot should be based on xenstore.
> > > > 
> > > > So a xenstore domain would always need special handling.
> > > 
> > > If there is really _never_ any reason to stop the xs domain then I
> > > think at
> > > the libxl API level a class of "never stop" domains would be better
> > > than
> > > special casing the xs, even if it turns out the only member of the set
> > > is
> > > xs at least we've given ourselves wriggle room if something else comes
> > > up
> > > in the future.
> > 
> > Okay, so this would translate to either:
> > 
> > - add a "never stop" flag to libxl_dominfo (can I do this without
> >   breaking the API?)
> > - add a new call interface to either check a single domain to be of
> >   the "never stop" class or to return a list of those domains.
> > 
> > Preferences?
> 
> Definitely the former, with a LIBXL_HAVE_ #define in libxl.h so consumers
> know they can use it.
> 
> Wei, Ian, do you agree with this approach?
> 

My concern in other sub-thread is addressed so this approach is fine by
me.

Wei.

> Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v4 0/4] support linear p2m list in migrate stream v2

2016-01-07 Thread Ian Campbell
On Thu, 2016-01-07 at 13:36 +0100, Juergen Gross wrote:
> Add support for the virtual mapped linear p2m list of pv-domains in the
> v2 migrate stream. This will allow to migrate domains larger than 512
> GB.
> 
> Tested with 32- and 64-bit pv-domains both with and without linear p2m
> list and with a hvm domain.
> 
> Changes in V4:
> - correct format strings in patch 2 for x86-32 as requested by Ian
>   Campbell
> - modify commit note of patch 2 as requested by Wei Liu

Applied, thanks.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] tools: add distclean target to libs/toollog Makefile

2016-01-07 Thread Ian Campbell
On Thu, 2016-01-07 at 10:36 +, Ian Campbell wrote:
> On Thu, 2016-01-07 at 09:25 +0100, Juergen Gross wrote:
> > The new logging library Makefile doesn't support the distclean target.
> > Add it.
> > 
> > Also remove all created shared library versions via the clean target.
> > 
> > Signed-off-by: Juergen Gross 
> 
> Acked-by: Ian Campbell 

and applied.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v3] xen/arm: ignore writes to GICD_ICACTIVER ... GICD_ICACTIVERN

2016-01-07 Thread Ian Campbell
On Wed, 2016-01-06 at 17:21 +, Stefano Stabellini wrote:
> Injecting a fault to the guest just because it is writing to one of the
> GICD_ICACTIVER registers, which are part of the GICv2 and GICv3 specs,
> is harsh. Additionally it causes recent linux kernels to fail to boot on
> Xen.
> 
> Ignore writes to GICD_ICACTIVER ... GICD_ICACTIVERN instead, to solve
> the boot issue and for backportability. However implementing the
> registers properly might a better long term solution.
> 
> Signed-off-by: Stefano Stabellini 

Acked + applied.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [qemu-upstream-4.6-testing test] 77225: regressions - FAIL

2016-01-07 Thread osstest service owner
flight 77225 qemu-upstream-4.6-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/77225/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-ovmf-amd64  9 debian-hvm-install fail REGR. vs. 63071
 test-amd64-amd64-xl-qemuu-ovmf-amd64 9 debian-hvm-install fail REGR. vs. 63071

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qcow2 9 debian-di-install  fail in 77101 pass in 77225
 test-amd64-amd64-libvirt-vhd  9 debian-di-install   fail pass in 77101
 test-armhf-armhf-xl-arndale   6 xen-bootfail pass in 77101

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds  9 debian-installfail REGR. vs. 63071
 test-armhf-armhf-xl-vhd  15 guest-start.2fail blocked in 63071
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail like 63071
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 63071

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt-vhd 11 migrate-support-check fail in 77101 never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-check fail in 77101 never pass
 test-armhf-armhf-xl-arndale 13 saverestore-support-check fail in 77101 never 
pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass

version targeted for testing:
 qemuu9e304f572ac98265f5e7433b6490077962acda97
baseline version:
 qemuu980dfcf5b8d8161871f81d1987b2f8ea5d7d4db9

Last test of basis63071  2015-10-19 15:10:58 Z   79 days
Testing same since66535  2015-12-18 15:50:42 Z   19 days8 attempts


People who touched revisions under test:
  Stefano Stabellini 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-

Re: [Xen-devel] [PATCH v4 3/3] VT-d: Fix vt-d Device-TLB flush timeout issue.

2016-01-07 Thread Xu, Quan
> On January 07, 2016 9:28 PM,  wrote:
> >>> On 07.01.16 at 02:39,  wrote:
> > On January 06, 2016 7:26 PM,  wrote:
> >> > I didn't think about the full logic thoroughly now. But it would
> >> > always be good to not hide device now until a point where all
> >> > related states have been cleaned up in error handling path chained up.
> >> >
> >
> > Jan, could you help me to double check it? thanks.
> 
> I'm not sure I understand what you want or need, the more that I didn't even
> get around to look at the patches yet.
> 

Jan, 
Patch 2/3 and Patch 3/3 are based on v3 (Actually they are v3's patch 1/2 and 
patch 2/2).
We have discussed how to hide a device with pci_hide_device() when Device-TLB 
flush is error. 

Now there are 2 concerns:
  1. Hide the PCI device may break the code path. (then the pdev->domain is 
dom_xen)
  2. Is the blew logic right?
   --If Device-TLB flush is timeout, we'll hide the target ATS device 
and crash the domain owning this ATS device.
 If impacted domain is hardware domain, just throw out a warning, 
instead of crash the hardware domain.
The hided Device will be disallowed to be further assigned to any 
domain.

Kevin, correct me if I am wrong.


Quan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2 1/2] x86/hvm: allow guest to use clflushopt and clwb

2016-01-07 Thread Jan Beulich
>>> On 30.12.15 at 12:48,  wrote:
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -4583,21 +4583,30 @@ void hvm_cpuid(unsigned int input, unsigned int *eax, 
> unsigned int *ebx,
>  *edx &= ~cpufeat_mask(X86_FEATURE_PSE36);
>  break;
>  case 0x7:
> -if ( (count == 0) && !cpu_has_smep )
> -*ebx &= ~cpufeat_mask(X86_FEATURE_SMEP);
> +if ( count == 0 )
> +{
> +if ( !cpu_has_smep )
> +*ebx &= ~cpufeat_mask(X86_FEATURE_SMEP);
> +
> +if ( !cpu_has_smap )
> +*ebx &= ~cpufeat_mask(X86_FEATURE_SMAP);
> +
> +/* Don't expose MPX to hvm when VMX support is not available */
> +if ( !(vmx_vmexit_control & VM_EXIT_CLEAR_BNDCFGS) ||
> + !(vmx_vmentry_control & VM_ENTRY_LOAD_BNDCFGS) )
> +*ebx &= ~cpufeat_mask(X86_FEATURE_MPX);
>  
> -if ( (count == 0) && !cpu_has_smap )
> -*ebx &= ~cpufeat_mask(X86_FEATURE_SMAP);
> +/* Don't expose INVPCID to non-hap hvm. */
> +if ( !hap_enabled(d) )
> +*ebx &= ~cpufeat_mask(X86_FEATURE_INVPCID);
>  
> -/* Don't expose MPX to hvm when VMX support is not available */
> -if ( (count == 0) &&
> - (!(vmx_vmexit_control & VM_EXIT_CLEAR_BNDCFGS) ||
> -  !(vmx_vmentry_control & VM_ENTRY_LOAD_BNDCFGS)) )
> -*ebx &= ~cpufeat_mask(X86_FEATURE_MPX);
> +if ( !cpu_has_clflushopt )
> +*ebx &= ~cpufeat_mask(X86_FEATURE_CLFLUSHOPT);
> +
> +if ( !cpu_has_clwb )
> +*ebx &= ~cpufeat_mask(X86_FEATURE_CLWB);

I don't think we need this: Other than other things adjusted here,
there's nothing disabling these two features when found available,
and there are no extra conditions to consider. Otherwise, if we
were to follow this route, quite a bit of code would need to be
added to other case statements in this function. But that's all (I
think) going to be taken care of by Andrew's CPUID leveling series.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2 1/2] x86/hvm: allow guest to use clflushopt and clwb

2016-01-07 Thread Jan Beulich
>>> On 07.01.16 at 15:01,  wrote:
> On 07/01/16 13:49, Jan Beulich wrote:
> On 30.12.15 at 12:48,  wrote:
>>> --- a/xen/arch/x86/hvm/hvm.c
>>> +++ b/xen/arch/x86/hvm/hvm.c
>>> @@ -4583,21 +4583,30 @@ void hvm_cpuid(unsigned int input, unsigned int 
>>> *eax, 
> unsigned int *ebx,
>>>  *edx &= ~cpufeat_mask(X86_FEATURE_PSE36);
>>>  break;
>>>  case 0x7:
>>> -if ( (count == 0) && !cpu_has_smep )
>>> -*ebx &= ~cpufeat_mask(X86_FEATURE_SMEP);
>>> +if ( count == 0 )
>>> +{
>>> +if ( !cpu_has_smep )
>>> +*ebx &= ~cpufeat_mask(X86_FEATURE_SMEP);
>>> +
>>> +if ( !cpu_has_smap )
>>> +*ebx &= ~cpufeat_mask(X86_FEATURE_SMAP);
>>> +
>>> +/* Don't expose MPX to hvm when VMX support is not available */
>>> +if ( !(vmx_vmexit_control & VM_EXIT_CLEAR_BNDCFGS) ||
>>> + !(vmx_vmentry_control & VM_ENTRY_LOAD_BNDCFGS) )
>>> +*ebx &= ~cpufeat_mask(X86_FEATURE_MPX);
>>>  
>>> -if ( (count == 0) && !cpu_has_smap )
>>> -*ebx &= ~cpufeat_mask(X86_FEATURE_SMAP);
>>> +/* Don't expose INVPCID to non-hap hvm. */
>>> +if ( !hap_enabled(d) )
>>> +*ebx &= ~cpufeat_mask(X86_FEATURE_INVPCID);
>>>  
>>> -/* Don't expose MPX to hvm when VMX support is not available */
>>> -if ( (count == 0) &&
>>> - (!(vmx_vmexit_control & VM_EXIT_CLEAR_BNDCFGS) ||
>>> -  !(vmx_vmentry_control & VM_ENTRY_LOAD_BNDCFGS)) )
>>> -*ebx &= ~cpufeat_mask(X86_FEATURE_MPX);
>>> +if ( !cpu_has_clflushopt )
>>> +*ebx &= ~cpufeat_mask(X86_FEATURE_CLFLUSHOPT);
>>> +
>>> +if ( !cpu_has_clwb )
>>> +*ebx &= ~cpufeat_mask(X86_FEATURE_CLWB);
>> I don't think we need this: Other than other things adjusted here,
>> there's nothing disabling these two features when found available,
>> and there are no extra conditions to consider. Otherwise, if we
>> were to follow this route, quite a bit of code would need to be
>> added to other case statements in this function. But that's all (I
>> think) going to be taken care of by Andrew's CPUID leveling series.
> 
> My series does take care of all of this.
> 
> I would prefer that these two changes get taken as soon as they are
> ready, so I can rebase.

If we don't need the change quoted above, your rebase will actually
be easier to do.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH RFC] tools: add map files for libxen{store, ctrl, guest}.so

2016-01-07 Thread Andrew Cooper
On 07/01/16 13:51, Ian Campbell wrote:
> The map files highlight a number of namespacing inconsistencies
> (particularly with libxenguest using xc_* a significant amount).
>
> It also seems to highlight a bunch of libxenguest.so functionalty
> which appears to want to be exported (xc_*) but is not used in tree.
> The initial list was based on what was needed to compile everything in
> tree. I then looked through the list for xc_* and checked if any were
> exported in a public header, leading to adding the following functions
> which are intended to be public but not used in tree to the
> libxenguest.map:
>   - xc_cpuid_to_str
>   - xc_compression_add_page
>   - xc_compression_compress_pages
>   - xc_compression_create_context
>   - xc_compression_free_context
>   - xc_compression_reset_pagebuf
>   - xc_compression_uncompress_page

These compression functions became unused when I dropped legacy migration.

> diff --git a/tools/libxc/libxenctrl.map b/tools/libxc/libxenctrl.map
> new file mode 100644
> index 000..cc93a5b
> --- /dev/null
> +++ b/tools/libxc/libxenctrl.map
> @@ -0,0 +1,18 @@
> +{
> + global:
> + xc_*;
> +
> + /*
> +  * Supposedly internal functions which are also used
> +  * by libxenguest (only, it seems)
> +  */
> + read_exact;
> + write_exact;
> + writev_exact;

read/write_exact are used in libxc by xc_tmem.c, but only because the
tmem part of legacy migration split across the two libraries.

In the long term, they should move to being xenguest private.

~Andrew

> +
> + /* Other un-namespaced functions used elsewhere in tree */
> + do_xen_hypercall;
> + do_memory_op;
> +
> + local: *; /* Do not expose anything by default */
> +};


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 0/4] Allow schedulers to be selectable through Kconfig

2016-01-07 Thread Jan Beulich
>>> On 07.01.16 at 15:01,  wrote:

> Ian Campbell writes:
> 
>> I don't see this as contrary to your stated goals (e.g. ripping out all the
>> other schedulers), but I consider you to be within the expert camp for
>> wanting to do so (and having the chops to handle whatever pieces you find
>> yourselves with). I have no objections at all to allowing experts such as
>> yourselves to configure things and I applaud you for doing this in an
>> upstream way (it is the right thing to do).
>>
>> My concern is that while you rightly consider yourselves expert enough and
>> are building something for a specific (and AIUI targeted) use case many
>> normal users tend to think that if they are expert enough to find and flip
>> the switch then they are expert enough to deal with the consequences, when
>> they are not and/or they do not have the specific use case which the switch
>> was added to support i.e. they want common or garden Xen and we want that
>> to mean the same for everyone.
>>
>> It's those people (including general purpose distro maintainers) who I
>> think need to be strongly discouraged from messing with these options
>> because there will be a strong gravity towards them doing so.
> 
> So, if I add a patch in a v3 of this series that introduces a
> CONFIG_EXPERT option and hides all of the scheduler options behind that,
> would that be acceptible? That is a proposal that was mentioned on this
> thread before.

With me asking for that option to not have a visible prompt by default,
but nevertheless being settable. I do realize that this may not be
possible with the current kconfig tool, but that's imo the only way to
keep people from playing with expert options just because they see
there's a prompt. No textual warning will help this, I'm afraid.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v3 2/2] libxc: Defer initialization of start_page for HVM guests

2016-01-07 Thread Boris Ostrovsky

On 01/07/2016 06:43 AM, Roger Pau Monné wrote:

El 06/01/16 a les 21.03, Boris Ostrovsky ha escrit:
  
  static int bootlate_hvm(struct xc_dom_image *dom)

  {
-DOMPRINTF("%s: doing nothing", __func__);
+uint32_t domid = dom->guest_domid;
+xc_interface *xch = dom->xch;
+
+if ( !dom->device_model )
+{
+struct hvm_start_info *start_info;
+size_t start_info_size = sizeof(*start_info);
+void *start_page;
+struct hvm_modlist_entry *modlist;
+size_t cmdline_size = 0;
+
+if ( dom->cmdline )
+{
+cmdline_size = ROUNDUP(strlen(dom->cmdline) + 1, 8);
+start_info_size += cmdline_size;
+}
+if ( dom->ramdisk_blob )
+start_info_size += sizeof(*modlist); /* Limited to one module. */

The size calculations are duplicated, could you either stash
start_info_size into xc_dom_image, or simply do the memory allocation
(xc_dom_alloc_segment) inside of bootlate_hvm? (I think the latter would
be better if possible).


I didn't want to do the first because we'd use that information (two 
pieces --- we need to to know both the size of the extra chunk and where 
modlist starts) only once and it's not on a critical path. You can, of 
course, argue that we increase text size.


The problem with the second approach is that while it does seem to work 
I don't know whether we can delay allocations until bootlate(): notice 
how we print dom->virt_alloc_end in xc_dom_build_image() which to me 
indicates some "finality" as far as allocations for domain are concerned.



-boris

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [libvirt] [PATCH 1/2] xenconfig: support parsing and formatting vif bandwidth

2016-01-07 Thread Michal Privoznik
On 29.12.2015 02:09, Jim Fehlig wrote:
> Both xm and xl config have long supported specifying vif rate
> limiting, e.g.
> 
> vif = [ 'mac=00:16:3E:74:3d:76,bridge=br0,rate=10MB/s' ]
> 
> Add support for mapping rate to and from  in the xenconfig
> parser and formatter. rate is mapped to the required 'average' attribute
> of the  element, e.g.
> 
>   
> ...
> 
>   
> 
>   
> 
> Also add a unit test to check the conversion logic.
> 
> Signed-off-by: Jim Fehlig 
> ---
> 
> I used a bit of code from libxlu_vif.c to implement xenParseVifRate()
> instead of using the libxlutil lib directly, since in theory rate limiting
> applies to the old xen driver (no libxl) as well.
> 
>  src/xenconfig/xen_common.c   | 77 
> 
>  tests/xlconfigdata/test-vif-rate.cfg | 26 
>  tests/xlconfigdata/test-vif-rate.xml | 57 ++
>  tests/xlconfigtest.c |  1 +
>  4 files changed, 161 insertions(+)

ACK

Michal

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH RFC] tools: add map files for libxen{store, ctrl, guest}.so

2016-01-07 Thread Jan Beulich
>>> On 07.01.16 at 14:51,  wrote:
> --- /dev/null
> +++ b/tools/libxc/libxenctrl.map
> @@ -0,0 +1,18 @@
> +{
> + global:

No actual version identifier / name space?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH] tools/libxc: Initialise parameters in map_p2m_list() for error paths

2016-01-07 Thread Andrew Cooper
c/s 7bf7458 "libxc: support of linear p2m list for migration of
pv-domains" breaks compilation on CentOS 7 because of 'ptes' being
possibly uninitialised after the 'err:' label.

The migration will fail early for conditions which would cause the for()
loop not to run, but the compiler doesn't know this.

Initialise the parameters to safe default to make the function more
robust.

Signed-off-by: Andrew Cooper 
---
CC: Ian Campbell 
CC: Ian Jackson 
CC: Wei Liu 
CC: Juergen Gross 
---
 tools/libxc/xc_sr_save_x86_pv.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/libxc/xc_sr_save_x86_pv.c b/tools/libxc/xc_sr_save_x86_pv.c
index 4deb58f..ab4bbe0 100644
--- a/tools/libxc/xc_sr_save_x86_pv.c
+++ b/tools/libxc/xc_sr_save_x86_pv.c
@@ -316,9 +316,9 @@ static int map_p2m_list(struct xc_sr_context *ctx, uint64_t 
p2m_cr3)
 xc_interface *xch = ctx->xch;
 xen_vaddr_t p2m_vaddr, p2m_end, mask, off;
 xen_pfn_t p2m_mfn, mfn, saved_mfn, max_pfn;
-uint64_t *ptes;
+uint64_t *ptes = NULL;
 xen_pfn_t *mfns;
-unsigned fpp, n_pages, level, shift, idx_start, idx_end, idx, saved_idx;
+unsigned fpp, n_pages = 0, level, shift, idx_start, idx_end, idx, 
saved_idx;
 int rc = -1;
 
 p2m_mfn = cr3_to_mfn(ctx, p2m_cr3);
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 0/4] Allow schedulers to be selectable through Kconfig

2016-01-07 Thread Ian Campbell
On Thu, 2016-01-07 at 07:37 -0700, Jan Beulich wrote:
> > > > On 07.01.16 at 15:01,  wrote:
> 
> > Ian Campbell writes:
> > 
> > > I don't see this as contrary to your stated goals (e.g. ripping out all 
> > > the
> > > other schedulers), but I consider you to be within the expert camp for
> > > wanting to do so (and having the chops to handle whatever pieces you find
> > > yourselves with). I have no objections at all to allowing experts such as
> > > yourselves to configure things and I applaud you for doing this in an
> > > upstream way (it is the right thing to do).
> > > 
> > > My concern is that while you rightly consider yourselves expert enough and
> > > are building something for a specific (and AIUI targeted) use case many
> > > normal users tend to think that if they are expert enough to find and flip
> > > the switch then they are expert enough to deal with the consequences, when
> > > they are not and/or they do not have the specific use case which the 
> > > switch
> > > was added to support i.e. they want common or garden Xen and we want that
> > > to mean the same for everyone.
> > > 
> > > It's those people (including general purpose distro maintainers) who I
> > > think need to be strongly discouraged from messing with these options
> > > because there will be a strong gravity towards them doing so.
> > 
> > So, if I add a patch in a v3 of this series that introduces a
> > CONFIG_EXPERT option and hides all of the scheduler options behind that,
> > would that be acceptible? That is a proposal that was mentioned on this
> > thread before.

Thinking about it I think I'd avoid the specific name CONFIG_EXPERT due to
the expectations which Linux's use of the name has set.

If we invert the sense then we could call it e.g. CONFIG_STANDARD_PLATFORM
and default it to y, I expect it will be easier to discourage people from
turning such an option off than to discourage them from turning something
like CONFIG_EXPERT on.

> With me asking for that option to not have a visible prompt by default,
> but nevertheless being settable. I do realize that this may not be
> possible with the current kconfig tool, but that's imo the only way to
> keep people from playing with expert options just because they see
> there's a prompt. No textual warning will help this, I'm afraid.

While I have reasonably strong opinions about this issue, I do not think
they warrant forking Kconfig over.

With a suitably strong wording IMHO we have covered ourselves sufficiently.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] tools/libxc: Initialise parameters in map_p2m_list() for error paths

2016-01-07 Thread Ian Campbell
On Thu, 2016-01-07 at 14:55 +, Andrew Cooper wrote:
> c/s 7bf7458 "libxc: support of linear p2m list for migration of
> pv-domains" breaks compilation on CentOS 7 because of 'ptes' being
> possibly uninitialised after the 'err:' label.
> 
> The migration will fail early for conditions which would cause the for()
> loop not to run, but the compiler doesn't know this.

Isn't the issue the malloc goto err path before the loop? Looks like that
should have the error behaviour from the earlier half of the function
rather than the latter.

There might also be a path if ctx->x86_pv.levels == 0, in which case the
loop will never run, that's the sort of thing which could be checked (or
even perhaps asserted) on entry to the function.

> Initialise the parameters to safe default to make the function more
> robust.
> 
> Signed-off-by: Andrew Cooper 
> ---
> CC: Ian Campbell 
> CC: Ian Jackson 
> CC: Wei Liu 
> CC: Juergen Gross 
> ---
>  tools/libxc/xc_sr_save_x86_pv.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/libxc/xc_sr_save_x86_pv.c
> b/tools/libxc/xc_sr_save_x86_pv.c
> index 4deb58f..ab4bbe0 100644
> --- a/tools/libxc/xc_sr_save_x86_pv.c
> +++ b/tools/libxc/xc_sr_save_x86_pv.c
> @@ -316,9 +316,9 @@ static int map_p2m_list(struct xc_sr_context *ctx,
> uint64_t p2m_cr3)
>  xc_interface *xch = ctx->xch;
>  xen_vaddr_t p2m_vaddr, p2m_end, mask, off;
>  xen_pfn_t p2m_mfn, mfn, saved_mfn, max_pfn;
> -uint64_t *ptes;
> +uint64_t *ptes = NULL;
>  xen_pfn_t *mfns;
> -unsigned fpp, n_pages, level, shift, idx_start, idx_end, idx,
> saved_idx;
> +unsigned fpp, n_pages = 0, level, shift, idx_start, idx_end, idx,
> saved_idx;
>  int rc = -1;
>  
>  p2m_mfn = cr3_to_mfn(ctx, p2m_cr3);

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] tools/libxc: Initialise parameters in map_p2m_list() for error paths

2016-01-07 Thread Juergen Gross
On 07/01/16 16:06, Ian Campbell wrote:
> On Thu, 2016-01-07 at 14:55 +, Andrew Cooper wrote:
>> c/s 7bf7458 "libxc: support of linear p2m list for migration of
>> pv-domains" breaks compilation on CentOS 7 because of 'ptes' being
>> possibly uninitialised after the 'err:' label.
>>
>> The migration will fail early for conditions which would cause the for()
>> loop not to run, but the compiler doesn't know this.
> 
> Isn't the issue the malloc goto err path before the loop? Looks like that
> should have the error behaviour from the earlier half of the function
> rather than the latter.

Yes to both. OTOH with Andrew's patch the code behaves correctly.

> There might also be a path if ctx->x86_pv.levels == 0, in which case the
> loop will never run, that's the sort of thing which could be checked (or
> even perhaps asserted) on entry to the function.

As there is no function vector involved between setting levels and
consuming it I doubt this makes really sense.


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] tools/libxc: Initialise parameters in map_p2m_list() for error paths

2016-01-07 Thread Andrew Cooper
On 07/01/16 15:06, Ian Campbell wrote:
> On Thu, 2016-01-07 at 14:55 +, Andrew Cooper wrote:
>> c/s 7bf7458 "libxc: support of linear p2m list for migration of
>> pv-domains" breaks compilation on CentOS 7 because of 'ptes' being
>> possibly uninitialised after the 'err:' label.
>>
>> The migration will fail early for conditions which would cause the for()
>> loop not to run, but the compiler doesn't know this.
> Isn't the issue the malloc goto err path before the loop? Looks like that
> should have the error behaviour from the earlier half of the function
> rather than the latter.

So it is - I missed that one when looking the options.

>
> There might also be a path if ctx->x86_pv.levels == 0, in which case the
> loop will never run, that's the sort of thing which could be checked (or
> even perhaps asserted) on entry to the function.

I don't think asserting details like this is a scalable options.  Also,
it doesn't help the compilation error if someone ends up disabling
assert() by playing with NDEBUG.

All that is needed is an adjustment to the commit message IMO:

---8<---
tools/libxc: Initialise parameters in map_p2m_list() for error paths

c/s 7bf7458 "libxc: support of linear p2m list for migration of
pv-domains" breaks compilation on CentOS 7 because of 'ptes' being
possibly uninitialised after the 'err:' label.

Indeed, the malloc() failure path would end using 'ptes' while
uninitialised.  Initialise the parameters to safe defaults.

Signed-off-by: Andrew Cooper 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] OVMF related osstest failures on multiple branches

2016-01-07 Thread Ian Campbell
On Thu, 2016-01-07 at 15:36 +, Ian Jackson wrote:
>  endif
> -OVMF_UPSTREAM_REVISION ?= cb9a7ebabcd6b8a49dc0854b2f9592d732b5afbd
> +OVMF_UPSTREAM_REVISION ?= af9785a9ed61daea52b47f0bf448f1f228beee1e

This should be:
 OVMF_UPSTREAM_REVISION ?= 52a99493cce88a9d4ec8a02d7f1bd1a1001ce60d

You've picked up 04c5efb0a141 but not f046e501bbc I think, we should take
both, since that's what I've tested if nothing else.

Ian.

>  QEMU_UPSTREAM_REVISION ?= qemu-xen-4.6.0
>  MINIOS_UPSTREAM_REVISION ?= xen-RELEASE-4.6.0
>  # Fri Jun 26 11:58:40 2015 +0100

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 0/4] Allow schedulers to be selectable through Kconfig

2016-01-07 Thread Jan Beulich
>>> On 07.01.16 at 16:18,  wrote:
> On 1/7/16 9:00 AM, Ian Campbell wrote:
>> If we invert the sense then we could call it e.g. CONFIG_STANDARD_PLATFORM
>> and default it to y, I expect it will be easier to discourage people from
>> turning such an option off than to discourage them from turning something
>> like CONFIG_EXPERT on.
> 
> So I actually like this approach. Maybe tweak the name slightly to
> CONFIG_SUPPORTED_XEN? It can force settings a certain way and can also
> make menus invisible. But I'm hoping we can get Jan's buy in here before
> we post any patchset.

With Ian's proposed wording, and with it being either
STANDARD_PLATFORM as Ian suggests or just SUPPORTED, I'd
be fine I think.

> So Jonathan and I have been messing with the hidden option behind a
> non-menu option (e.g. environment variable). The only way we can get it
> to work is to require the environment variable to be passed at
> configuration time and at build time and I doubt you'd want the steps to
> be "CONFIG_EXPERT=n make menuconfig && CONFIG_EXPERT=n make" to build
> Xen. We'll play with this some more if that's desired but given Ian's
> response I don't know if it is.

If these overrides were only required for people wanting the non-
default setting, I think it wouldn't be too bad. In fact one more
thing to keep them from fiddling with the option.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [qemu-upstream-4.2-testing test] 77267: regressions - FAIL

2016-01-07 Thread osstest service owner
flight 77267 qemu-upstream-4.2-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/77267/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i3865 xen-build fail REGR. vs. 62044
 build-amd64   5 xen-build fail REGR. vs. 62044

Tests which did not succeed, but are not blocking:
 build-i386-libvirt1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-winxpsp3  1 build-check(1)   blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-i386-i386-xl-qemuu-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 test-amd64-i386-xend-qemuu-winxpsp3  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a

version targeted for testing:
 qemuu5081fc1c773d2a83ec7a867f030323b8b6956cd1
baseline version:
 qemuuc17e602ae64fb24405ebd256679ba9a6f5819086

Last test of basis62044  2015-09-15 15:06:42 Z  114 days
Testing same since66542  2015-12-18 16:37:10 Z   19 days   12 attempts


People who touched revisions under test:
  Stefano Stabellini 

jobs:
 build-amd64  fail
 build-i386   fail
 build-amd64-libvirt  blocked 
 build-i386-libvirt   blocked 
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-i386-qemuu-rhel6hvm-amd   blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-i386-xl-qemuu-debianhvm-amd64 blocked 
 test-amd64-amd64-xl-qemuu-ovmf-amd64 blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64  blocked 
 test-amd64-amd64-xl-qemuu-win7-amd64 blocked 
 test-amd64-i386-xl-qemuu-win7-amd64  blocked 
 test-amd64-i386-qemuu-rhel6hvm-intel blocked 
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 blocked 
 test-amd64-i386-xend-qemuu-winxpsp3  blocked 
 test-amd64-amd64-xl-qemuu-winxpsp3   blocked 
 test-i386-i386-xl-qemuu-winxpsp3 blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit 5081fc1c773d2a83ec7a867f030323b8b6956cd1
Author: Stefano Stabellini 
Date:   Fri Dec 18 15:45:14 2015 +

xenfb: avoid reading twice the same fields from the shared page

Reading twice the same field could give the guest an attack of
opportunity. In the case of event->type, gcc could compile the switch
statement into a jump table, effectively ending up reading the type
field multiple times.

This is part of XSA-155.

Signed-off-by: Stefano Stabellini 

commit 74fab2ef4c0ba42af477e9e445c9883cc45cf9e6
Author: Stefano Stabellini 
Date:   Fri Dec 18 15:44:58 2015 +

xen/blkif: Avoid double access to src->nr_segments

src is stored in shared memory and src->nr_segments is dereferenced
twice at the end of the function.  If a compiler decides to compile this
into two separate memory accesses then the size limitation could be
bypassed.

Fix it by removing the double access to src->nr_segments.

This is part of XSA-155.

Signed-off-by: Stefano Stabellini 


[Xen-devel] [xen-unstable-smoke test] 77380: tolerable all pass - PUSHED

2016-01-07 Thread osstest service owner
flight 77380 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/77380/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  68778eeaa3babedba9677400f63f1e7564bba561
baseline version:
 xen  ce0fac22e7f367ba72ebd762331f8c9bdf1e2519

Last test of basis77147  2016-01-05 15:01:04 Z2 days
Testing same since77380  2016-01-07 14:00:56 Z0 days1 attempts


People who touched revisions under test:
  Boris Ostrovsky 
  Ian Campbell 
  Juergen Gross 
  Stefano Stabellini 
  Wei Liu 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=xen-unstable-smoke
+ revision=68778eeaa3babedba9677400f63f1e7564bba561
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push xen-unstable-smoke 
68778eeaa3babedba9677400f63f1e7564bba561
+ branch=xen-unstable-smoke
+ revision=68778eeaa3babedba9677400f63f1e7564bba561
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable-smoke
+ qemuubranch=qemu-upstream-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' xqemu-upstream-unstable = x ']'
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable-smoke
+ prevxenbranch=xen-unstable
+ '[' x68778eeaa3babedba9677400f63f1e7564bba561 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/rumpuser-xen.git
+++ besteffort_repo https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ cached_repo https://github.com/rumpkernel/rumpkernel-netbsd-src 
'[fetch=try]'
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local 'options=[fetch=try]'
 getconfig GitCacheProxy
 perl -e '
use Osstest;
readglobalconfig();
print $c{"GitCacheProxy"} or die $!;
'
+++ local cache=git://cache:9419/
+++ '[' xgit:/

Re: [Xen-devel] [PATCH 0/4] Allow schedulers to be selectable through Kconfig

2016-01-07 Thread Jan Beulich
>>> On 07.01.16 at 16:30,  wrote:
> Some proposed text for discussion:
> 
> config STANDARD_PLATFORM
>   bool "Standard/Supported Platform"
>   default y
>   help
> This option enables everything which is part of the standard and
> supported Xen platform. You should say 'Y' to this question.
> 
> Turning off this option will expose additional options which will
> cause the resulting Xen binary to deviate from the supported
> configuration. Resulting configurations are not support by the Xen
> Project. Specifically configurations which disable 
> this option::
> 
>  * are not tested by the Xen Project; If you disable this 
> option you should
>perform your own QA.
>  * are not Supported by the Xen Project; Guidance will be 
> given but
>ultimately you are responsible for fixing the issues you 
> discover.
>  * do not receive security support; Issues which are seen 
> only with the
>option disabled are not treated as security bugs and are 
> not subject to
>the security process 
> http://www.xenproject.org/security-policy.html).
> 
> Again, you should say 'Y' to this question.
> 
> Too much? In particular is the second half of each bullet over egging the
> pudding somewhat?

I think these second halves are quite helpful.

> The state of this option should be logged during boot. Somewhere in this
> lot I suppose:
> 
> (XEN) Xen version 4.7-unstable (osstest@bad) (gcc (Debian 4.9.2-10) 4.9.2) 
> debug=y Mon Nov 30 10:33:03 UTC 2015
> (XEN) Latest ChangeSet: Thu Nov 26 16:01:27 2015 +0100 git:b1d398b
> 
> Either "Platform: Standard" or "Platform: Custom" perhaps? The first line
> is too long already so standard=y|n doesn't really work.

Perhaps there could be a 3rd line here saying: "Supported: yes
(xenproject.org)" or "Supported: no", allowing distros to further
customize this to direct their users to where support for the given
configuration can be got?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC PATCH v6 00/28] libxl: Deprivilege qemu

2016-01-07 Thread Wei Liu
On Tue, Dec 22, 2015 at 06:44:35PM +, Ian Jackson wrote:
> This is a new version of Stefano Stabellini's series
>   [PATCH v5 0/6] libxl: xs_restrict QEMU
> 
> I took Stefano's code as a spec for how to interact with qemu, and
> have reworked the whole series.  In particular, I have
>  - rebased onto staging
>  - split up some of the larger patches
>  - restructured the patches so that every intervening version should work
>  - redone the async task handling
>  - provided what seem to me to be appropriate configuration options
>  - shaved a few yaks (although I tried to limit this!)
>  - fixed the cross-version compatibility
>  - reduced the new permission grant for the domain to only .../physmap
> 

One aspect I don't see in this series is how save and restore for multiple
emulator context is handled.

Did I miss anything?

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [ovmf test] 77229: regressions - FAIL

2016-01-07 Thread osstest service owner
flight 77229 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/77229/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-ovmf-amd64 17 guest-start/debianhvm.repeat fail 
REGR. vs. 65543
 test-amd64-i386-xl-qemuu-ovmf-amd64 17 guest-start/debianhvm.repeat fail REGR. 
vs. 65543

version targeted for testing:
 ovmf c2a892d7c8a78143006bb7fdc95fb18f7e2fc685
baseline version:
 ovmf 5ac96e3a28dd26eabee421919f67fa7c443a47f1

Last test of basis65543  2015-12-08 08:45:15 Z   30 days
Failing since 65593  2015-12-08 23:44:51 Z   29 days   16 attempts
Testing same since77119  2016-01-05 09:02:24 Z2 days2 attempts


People who touched revisions under test:
  "Samer El-Haj-Mahmoud" 
  "Yao, Jiewen" 
  Andrew Fish 
  Ard Biesheuvel 
  Cecil Sheng 
  Chao Zhang 
  Dandan Bi 
  Daocheng Bu 
  Daryl McDaniel 
  Eric Dong 
  Eric Dong 
  Eugene Cohen 
  Feng Tian 
  Fu Siyuan 
  Hao Wu 
  Hess Chen 
  Heyi Guo 
  Jaben Carsey 
  Jeff Fan 
  Jiaxin Wu 
  Jim Dailey 
  Jordan Justen 
  Larry Hauch 
  Laszlo Ersek 
  Leekha Shaveta 
  Liming Gao 
  Mark Rutland 
  Michael Kinney 
  Paulo Alcantara 
  Qin Long 
  Qiu Shumin 
  Ruiyu Ni 
  Samer El-Haj-Mahmoud 
  Star Zeng 
  Yao Jiewen 
  Yao, Jiewen 
  Ye Ting 
  Yonghong Zhu 
  Zhang Lubo 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 3888 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v3 2/2] libxc: Defer initialization of start_page for HVM guests

2016-01-07 Thread Roger Pau Monné
El 07/01/16 a les 15.47, Boris Ostrovsky ha escrit:
> On 01/07/2016 06:43 AM, Roger Pau Monné wrote:
>> El 06/01/16 a les 21.03, Boris Ostrovsky ha escrit:
>>> static int bootlate_hvm(struct xc_dom_image *dom)
>>>   {
>>> -DOMPRINTF("%s: doing nothing", __func__);
>>> +uint32_t domid = dom->guest_domid;
>>> +xc_interface *xch = dom->xch;
>>> +
>>> +if ( !dom->device_model )
>>> +{
>>> +struct hvm_start_info *start_info;
>>> +size_t start_info_size = sizeof(*start_info);
>>> +void *start_page;
>>> +struct hvm_modlist_entry *modlist;
>>> +size_t cmdline_size = 0;
>>> +
>>> +if ( dom->cmdline )
>>> +{
>>> +cmdline_size = ROUNDUP(strlen(dom->cmdline) + 1, 8);
>>> +start_info_size += cmdline_size;
>>> +}
>>> +if ( dom->ramdisk_blob )
>>> +start_info_size += sizeof(*modlist); /* Limited to one
>>> module. */
>> The size calculations are duplicated, could you either stash
>> start_info_size into xc_dom_image, or simply do the memory allocation
>> (xc_dom_alloc_segment) inside of bootlate_hvm? (I think the latter would
>> be better if possible).
> 
> I didn't want to do the first because we'd use that information (two
> pieces --- we need to to know both the size of the extra chunk and where
> modlist starts) only once and it's not on a critical path. You can, of
> course, argue that we increase text size.

It's just that I don't want to get them out of synch, and that's what
usually happens when you perform the same calculations in two different
places, one might get out of synch with the other.

> The problem with the second approach is that while it does seem to work
> I don't know whether we can delay allocations until bootlate(): notice
> how we print dom->virt_alloc_end in xc_dom_build_image() which to me
> indicates some "finality" as far as allocations for domain are concerned.

For PV guests it probably matters, because we have to build the page
tables and the p2m, for HVMlite guests I don't think it matters at all
(or at least I don't see any obvious reason).

Another third option is to place the code that performs the size
calculations inside of a function that's called by both (bootlate_hvm
and alloc_magic_pages_hvm).

Roger.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 22/28] libxl: dm user: Document the default

2016-01-07 Thread Ian Campbell
On Tue, 2015-12-22 at 18:44 +, Ian Jackson wrote:
> Signed-off-by: Ian Jackson 
> CC: Stefano Stabellini 

Acked-by: Ian Campbell 
(could go in now despite RFC-ness of the series as a whole)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 27/28] libxl: Limit qemu physmap entries

2016-01-07 Thread Ian Campbell
On Tue, 2015-12-22 at 18:45 +, Ian Jackson wrote:
> Add a maximum limit of physmap entries to save, so that when the guest
> gets write access to physmap it cannot DOS the toolstack.
> 
> Signed-off-by: Stefano Stabellini 
> Signed-off-by: Ian Jackson 

Can we have a reference for where the number 12 comes from please.

With that I think this doesn't need to wait for the rest of the series?


> ---
> v6: Split out of xs permissions relaxation patch.
> ---
>  tools/libxl/libxl_dom.c |7 +++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
> index 6ded9c1..60e8f7f 100644
> --- a/tools/libxl/libxl_dom.c
> +++ b/tools/libxl/libxl_dom.c
> @@ -1431,6 +1431,8 @@ static void append_string(libxl__gc *gc, char
> **buf, uint32_t *len,
>  *len += extralen;
>  }
>  
> +#define MAX_PHYSMAP_ENTRIES 12
> +
>  int libxl__save_emulator_xenstore_data(libxl__domain_suspend_state *dss,
> char **callee_buf,
> uint32_t *callee_len)
> @@ -1450,6 +1452,11 @@ int
> libxl__save_emulator_xenstore_data(libxl__domain_suspend_state *dss,
>    &nr_entries);
>  if (!entries || nr_entries == 0) { rc = 0; goto out; }
>  
> +if (nr_entries > MAX_PHYSMAP_ENTRIES) {
> +LOG(ERROR, "Max physmap entries reached");
> +return ERROR_FAIL;
> +}
> +
>  for (i = 0; i < nr_entries; ++i) {
>  static const char *const physmap_subkeys[] = {
>  "start_addr", "size", "name"
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v3 0/5] Allow schedulers to be selectable through Kconfig

2016-01-07 Thread Jonathan Creekmore
Add machinery to allow the schedulers to be individually selectable
through the Kconfig interface. The first patch in the series sets up
the CONFIG_EXPERT Kconfig variable that is only enabled by passing an
environment variable to the build. The second patch in the series sets up
the Kconfig options for the schedulers and places the appropriate CONFIG_
options around the scheduler list. The third, fourth, and fifth patches
rework the scheduler list from being manually curated into a model
where just compiling any schedulers into the hypervisor causes the
scheduler list to be built up.

---
Changed since v2:
 * Added a predecessor patch that introduces a environment
   variable for the build to enable expert configuration options
   (1/5)
 * Hide the scheduler menu behind the expert option (2/5)
 * Provide static inlines for credit functions that are assumed to be
   present if it is compiled out (2/5)
 * Provide an absolute default of the credit scheduler if the 
   scheduler menu is not visible (2/5)

Changed since v1:
 * Marked credit2 as EXPERIMENTAL
 * Removed confusing language from the commit message
 * Alphabetize the schedulers
 * rename the __start and __end symbols to better match
   the rest of the file
 * Simplify the calculation of the number of schedulers
 * Make the scheduler ops structures static to their files

Jonathan Creekmore (5):
  build: Env var to enable expert config options
  build: Hook the schedulers into Kconfig
  build: Alloc space for sched list in the link file
  sched: Register the schedulers into the list
  sched: Use the auto-generated list of schedulers

 xen/Kconfig |  4 +++
 xen/Makefile|  1 +
 xen/arch/arm/xen.lds.S  |  4 +++
 xen/arch/x86/xen.lds.S  |  4 +++
 xen/common/Kconfig  | 67 +
 xen/common/Makefile |  8 +++---
 xen/common/sched_arinc653.c |  4 ++-
 xen/common/sched_credit.c   |  4 ++-
 xen/common/sched_credit2.c  |  4 ++-
 xen/common/sched_rt.c   |  4 ++-
 xen/common/schedule.c   | 20 ++
 xen/include/xen/sched-if.h  |  7 ++---
 xen/include/xen/sched.h |  5 
 13 files changed, 112 insertions(+), 24 deletions(-)

-- 
2.6.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v3 1/5] build: Env var to enable expert config options

2016-01-07 Thread Jonathan Creekmore
Add an additional environment variable, defaulting to disabled,
that enables the CONFIG_EXPERT configuration option. The purpose
of the CONFIG_EXPERT configuration option is to make non-standard
Kconfig options visible during the configuration process. The
CONFIG_EXPERT option is not, itself, visible during the Kconfig
configuration process, so typical users will never see it nor
any of the non-standard configuration options.

CC: Ian Campbell 
CC: Ian Jackson 
CC: Jan Beulich 
CC: Keir Fraser 
CC: Tim Deegan 
Signed-off-by: Jonathan Creekmore 
Reviewed-by: Doug Goldstein 

---
 xen/Kconfig  | 4 
 xen/Makefile | 1 +
 2 files changed, 5 insertions(+)

diff --git a/xen/Kconfig b/xen/Kconfig
index ffe3f45..fa8b27c 100644
--- a/xen/Kconfig
+++ b/xen/Kconfig
@@ -22,3 +22,7 @@ config DEFCONFIG_LIST
string
option defconfig_list
default "$ARCH_DEFCONFIG"
+
+config EXPERT
+   string
+   option env="XEN_CONFIG_EXPERT"
diff --git a/xen/Makefile b/xen/Makefile
index 9023863..4950afb 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -11,6 +11,7 @@ export XEN_DOMAIN ?= $(shell ([ -x /bin/dnsdomainname ] 
&& /bin/dnsdomainname) |
 export XEN_BUILD_DATE  ?= $(shell LC_ALL=C date)
 export XEN_BUILD_TIME  ?= $(shell LC_ALL=C date +%T)
 export XEN_BUILD_HOST  ?= $(shell hostname)
+export XEN_CONFIG_EXPERT ?= n
 
 export BASEDIR := $(CURDIR)
 export XEN_ROOT := $(BASEDIR)/..
-- 
2.6.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v3 2/5] build: Hook the schedulers into Kconfig

2016-01-07 Thread Jonathan Creekmore
Allow the schedulers to be independently enabled or disabled at
compile-time. To match existing behavior, all four schedulers are
compiled in by default, although the Credit2, RTDS, and ARINC653 are
marked EXPERIMENTAL to match their not currently supported status.

CC: George Dunlap 
CC: Dario Faggioli 
Signed-off-by: Jonathan Creekmore 
Reviewed-by: Doug Goldstein 

---
Changed since v2:

  * Hid the scheduler menu behind the EXPERT option
  * Provide static inlines for credit functions that are assumed to be
always available
  * Provide an absolute default of the credit scheduler if the
scheduler menu is not visible

Changed since v1:

  * Marked credit2 as EXPERIMENTAL
  * Removed confusing language from the commit message
  * Alphabetize the schedulers
---
 xen/common/Kconfig  | 67 +
 xen/common/Makefile |  8 +++---
 xen/common/schedule.c   | 12 +++--
 xen/include/xen/sched.h |  5 
 4 files changed, 86 insertions(+), 6 deletions(-)

diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index 046e257..db7411b 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -51,4 +51,71 @@ config KEXEC
 
  If unsure, say Y.
 
+# Enable schedulers
+menu "Schedulers"
+   visible if EXPERT = "y"
+
+config SCHED_CREDIT
+   bool "Credit scheduler support"
+   default y
+   ---help---
+ The traditional credit scheduler is a general purpose scheduler.
+
+ If unsure, say Y.
+
+config SCHED_CREDIT2
+   bool "Credit2 scheduler support (EXPERIMENTAL)"
+   default y
+   ---help---
+ The credit2 scheduler is a general purpose scheduler that is
+ optimized for lower latency and higher VM density.
+
+ If unsure, say Y.
+
+config SCHED_RTDS
+   bool "RTDS scheduler support (EXPERIMENTAL)"
+   default y
+   ---help---
+ The RTDS scheduler is a soft and firm real-time scheduler for
+ multicore, targeted for embedded, automotive, graphics and gaming
+ in the cloud, and general low-latency workloads.
+
+ If unsure, say N.
+
+config SCHED_ARINC653
+   bool "ARINC653 scheduler support (EXPERIMENTAL)"
+   default y
+   ---help---
+ The ARINC653 scheduler is a hard real-time scheduler for single
+ cores, targeted for avionics, drones, and medical devices.
+
+ If unsure, say N.
+
+choice
+   prompt "Default Scheduler?"
+   default SCHED_CREDIT_DEFAULT if SCHED_CREDIT
+   default SCHED_CREDIT2_DEFAULT if SCHED_CREDIT2
+   default SCHED_RTDS_DEFAULT if SCHED_RTDS
+   default SCHED_ARINC653_DEFAULT if SCHED_ARINC653
+
+   config SCHED_CREDIT_DEFAULT
+   bool "Credit Scheduler" if SCHED_CREDIT
+   config SCHED_CREDIT2_DEFAULT
+   bool "Credit2 Scheduler" if SCHED_CREDIT2
+   config SCHED_RTDS_DEFAULT
+   bool "RT Scheduler" if SCHED_RTDS
+   config SCHED_ARINC653_DEFAULT
+   bool "ARINC653 Scheduler" if SCHED_ARINC653
+endchoice
+
+config SCHED_DEFAULT
+   string
+   default "credit" if SCHED_CREDIT_DEFAULT
+   default "credit2" if SCHED_CREDIT2_DEFAULT
+   default "rtds" if SCHED_RTDS_DEFAULT
+   default "arinc653" if SCHED_ARINC653_DEFAULT
+   default "credit"
+
+endmenu
+
 endmenu
diff --git a/xen/common/Makefile b/xen/common/Makefile
index 8ab15ba..29a5916 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -30,10 +30,10 @@ obj-y += rangeset.o
 obj-y += radix-tree.o
 obj-y += rbtree.o
 obj-y += rcupdate.o
-obj-y += sched_credit.o
-obj-y += sched_credit2.o
-obj-y += sched_arinc653.o
-obj-y += sched_rt.o
+obj-$(CONFIG_SCHED_ARINC653) += sched_arinc653.o
+obj-$(CONFIG_SCHED_CREDIT) += sched_credit.o
+obj-$(CONFIG_SCHED_CREDIT2) += sched_credit2.o
+obj-$(CONFIG_SCHED_RTDS) += sched_rt.o
 obj-y += schedule.o
 obj-y += shutdown.o
 obj-y += softirq.o
diff --git a/xen/common/schedule.c b/xen/common/schedule.c
index d121896..2f98a48 100644
--- a/xen/common/schedule.c
+++ b/xen/common/schedule.c
@@ -38,8 +38,8 @@
 #include 
 #include 
 
-/* opt_sched: scheduler - default to credit */
-static char __initdata opt_sched[10] = "credit";
+/* opt_sched: scheduler - default to configured value */
+static char __initdata opt_sched[10] = CONFIG_SCHED_DEFAULT;
 string_param("sched", opt_sched);
 
 /* if sched_smt_power_savings is set,
@@ -65,10 +65,18 @@ DEFINE_PER_CPU(struct schedule_data, schedule_data);
 DEFINE_PER_CPU(struct scheduler *, scheduler);
 
 static const struct scheduler *schedulers[] = {
+#ifdef CONFIG_SCHED_CREDIT
 &sched_credit_def,
+#endif
+#ifdef CONFIG_SCHED_CREDIT2
 &sched_credit2_def,
+#endif
+#ifdef CONFIG_SCHED_ARINC653
 &sched_arinc653_def,
+#endif
+#ifdef CONFIG_SCHED_RTDS
 &sched_rtds_def,
+#endif
 };
 
 static struct scheduler __read_mostly ops;
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index fc61fc3..246338e 100644
--- a/xen/include/xen/sched.h
+++

[Xen-devel] [PATCH v3 4/5] sched: Register the schedulers into the list

2016-01-07 Thread Jonathan Creekmore
Adds a simple macro to place a pointer to a scheduler into an array
section at compile time. Also, goes ahead and generates the array
entries with each of the schedulers.

CC: George Dunlap 
CC: Dario Faggioli 
CC: Josh Whitehead 
CC: Robert VanVossen 
Signed-off-by: Jonathan Creekmore 
Acked-by: Dario Faggioli 
Reviewed-by: Andrew Cooper 
Reviewed-by: Doug Goldstein 
---
 xen/common/sched_arinc653.c | 2 ++
 xen/common/sched_credit.c   | 2 ++
 xen/common/sched_credit2.c  | 2 ++
 xen/common/sched_rt.c   | 2 ++
 xen/include/xen/sched-if.h  | 2 ++
 5 files changed, 10 insertions(+)

diff --git a/xen/common/sched_arinc653.c b/xen/common/sched_arinc653.c
index dbe02ed..3b59514 100644
--- a/xen/common/sched_arinc653.c
+++ b/xen/common/sched_arinc653.c
@@ -767,6 +767,8 @@ const struct scheduler sched_arinc653_def = {
 .tick_resume= NULL,
 };
 
+REGISTER_SCHEDULER(sched_arinc653_def);
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c
index 0dce790..e586248 100644
--- a/xen/common/sched_credit.c
+++ b/xen/common/sched_credit.c
@@ -2027,3 +2027,5 @@ const struct scheduler sched_credit_def = {
 .tick_suspend   = csched_tick_suspend,
 .tick_resume= csched_tick_resume,
 };
+
+REGISTER_SCHEDULER(sched_credit_def);
diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index 3c49ffa..38b02d0 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -2228,3 +2228,5 @@ const struct scheduler sched_credit2_def = {
 .alloc_domdata  = csched2_alloc_domdata,
 .free_domdata   = csched2_free_domdata,
 };
+
+REGISTER_SCHEDULER(sched_credit2_def);
diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
index 3f1d047..7640cd0 100644
--- a/xen/common/sched_rt.c
+++ b/xen/common/sched_rt.c
@@ -1199,3 +1199,5 @@ const struct scheduler sched_rtds_def = {
 .wake   = rt_vcpu_wake,
 .context_saved  = rt_context_saved,
 };
+
+REGISTER_SCHEDULER(sched_rtds_def);
diff --git a/xen/include/xen/sched-if.h b/xen/include/xen/sched-if.h
index 493d43f..9c6e0f5 100644
--- a/xen/include/xen/sched-if.h
+++ b/xen/include/xen/sched-if.h
@@ -170,6 +170,8 @@ extern const struct scheduler sched_credit2_def;
 extern const struct scheduler sched_arinc653_def;
 extern const struct scheduler sched_rtds_def;
 
+#define REGISTER_SCHEDULER(x) static const struct scheduler *x##_entry \
+  __used_section(".data.schedulers") = &x;
 
 struct cpupool
 {
-- 
2.6.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [xen-unstable-smoke test] 77400: trouble: blocked/broken/pass

2016-01-07 Thread osstest service owner
flight 77400 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/77400/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf   3 host-install(3) broken REGR. vs. 77380

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass

version targeted for testing:
 xen  03809ae77daf17a7e3104019758aa4c4b23b631c
baseline version:
 xen  68778eeaa3babedba9677400f63f1e7564bba561

Last test of basis77380  2016-01-07 14:00:56 Z0 days
Testing same since77400  2016-01-07 16:02:20 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Boris Ostrovsky 
  Daniel De Graaf 
  Doug Goldstein 
  Feng Wu 
  Jan Beulich 
  Joshua Otto 
  Kevin Tian 
  Paul Durrant 

jobs:
 build-amd64  pass
 build-armhf  broken  
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-step build-armhf host-install(3)

Not pushing.


commit 03809ae77daf17a7e3104019758aa4c4b23b631c
Author: Paul Durrant 
Date:   Thu Jan 7 15:28:33 2016 +0100

public/io/netif.h: document transmit and receive wire formats separately

Currently there is no documented wire format for guest receive-side
packets but the location of the 'wire format' comment block suggests
it is the same as transmit-side. This is almost true but there is a
subtle difference in the use of the 'size' field for the first fragment.

For clarity this patch creates separate comment blocks for receive
and transmit side packet wire formats, tries to be more clear about the
distinction between 'fragments' and 'extras', and documents the subtlety
concerning the size field of the first fragment.

Signed-off-by: Paul Durrant 

commit 9c419ee77f3e6c3a90a4a08c8682b4216bed8cb0
Author: Joshua Otto 
Date:   Thu Jan 7 15:28:08 2016 +0100

credit: remove pointless local variable initialization

Coverity CID 1343301

No functional changes.

Signed-off-by: Joshua Otto 

commit 373c3cf6dd0674fc2aa95f7db6b9add851817076
Author: Doug Goldstein 
Date:   Thu Jan 7 15:27:43 2016 +0100

remove dups in x86 and x86_64 variables

Currently the Xen build uses x86 and x86_64 variables as well as
CONFIG_X86 and CONFIG_X86_64. This just removes the duplication. The
CONFIG_ variables are now managed by Kconfig but existed previously so
this duplication existed prior to the Kconfig migration.

Signed-off-by: Doug Goldstein 
Acked-by: Andrew Cooper 
Acked-by: Feng Wu 

$(CONFIG_X86_64) -> y in x86 makefiles.
$(CONFIG_X86_64) -> $(CONFIG_X86) in non-x86 makefiles.

Signed-off-by: Jan Beulich 

commit fb424bf6b5b0df0155ab4e56a1b8f67e6470fa46
Author: Boris Ostrovsky 
Date:   Thu Jan 7 15:27:16 2016 +0100

x86/VPMU: don't allow any non-zero writes to MSR_IA32_PEBS_ENABLE

Calculation reserved bits for MSR_IA32_PEBS_ENABLE is model-dependent
and since we don't support PEBS anyway we shouldn't allow any writes to
it (but let's still permit guests wishing to disable PEBS).

We should also report PEBS as unsupported to HVM, just like we do on PV.

Signed-off-by: Boris Ostrovsky 
Acked-by: Kevin Tian 

commit 31af0d76759328161cb5db73b50b23dded51e15c
Author: Boris Ostrovsky 
Date:   Thu Jan 7 15:26:37 2016 +0100

x86/VPMU: check more carefully which bits are allowed to be written to MSRs

Current Intel VPMU emulation needs to perform more checks when writing
PMU MSRs on guest's behalf:
* MSR_CORE_PERF_GLOBAL_CTRL is not checked at all
* MSR_CORE_PERF_FIXED_CTR_CTRL has more reserved bits in PMU version 2
* MSR_CORE_PERF_GLOBAL_OVF_CTRL's bit 61 is allowed on versions grea

[Xen-devel] [qemu-mainline test] 77249: regressions - FAIL

2016-01-07 Thread osstest service owner
flight 77249 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/77249/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   5 xen-build fail REGR. vs. 66433
 build-i3865 xen-build fail REGR. vs. 66433
 build-amd64-xsm   5 xen-build fail REGR. vs. 66433
 build-i386-xsm5 xen-build fail REGR. vs. 66433
 build-armhf-xsm   5 xen-build fail REGR. vs. 66433
 build-armhf   5 xen-build fail REGR. vs. 66433

Tests which did not succeed, but are not blocking:
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1)blocked n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvh-amd   1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-winxpsp3  1 build-check(1)   blocked n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-pvh-intel  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-chec

Re: [Xen-devel] [PATCH v3 2/2] libxc: Defer initialization of start_page for HVM guests

2016-01-07 Thread Ian Campbell
On Thu, 2016-01-07 at 12:33 -0500, Boris Ostrovsky wrote:
> On 01/07/2016 12:06 PM, Ian Campbell wrote:
> > On Thu, 2016-01-07 at 17:54 +0100, Roger Pau Monné wrote:
> > > El 07/01/16 a les 15.47, Boris Ostrovsky ha escrit:
> > > > On 01/07/2016 06:43 AM, Roger Pau Monné wrote:
> > > > > El 06/01/16 a les 21.03, Boris Ostrovsky ha escrit:
> > > > > >  static int bootlate_hvm(struct xc_dom_image *dom)
> > > > > >    {
> > > > > > -DOMPRINTF("%s: doing nothing", __func__);
> > > > > > +uint32_t domid = dom->guest_domid;
> > > > > > +xc_interface *xch = dom->xch;
> > > > > > +
> > > > > > +if ( !dom->device_model )
> > > > > > +{
> > > > > > +struct hvm_start_info *start_info;
> > > > > > +size_t start_info_size = sizeof(*start_info);
> > > > > > +void *start_page;
> > > > > > +struct hvm_modlist_entry *modlist;
> > > > > > +size_t cmdline_size = 0;
> > > > > > +
> > > > > > +if ( dom->cmdline )
> > > > > > +{
> > > > > > +cmdline_size = ROUNDUP(strlen(dom->cmdline) + 1,
> > > > > > 8);
> > > > > > +start_info_size += cmdline_size;
> > > > > > +}
> > > > > > +if ( dom->ramdisk_blob )
> > > > > > +start_info_size += sizeof(*modlist); /* Limited to
> > > > > > one
> > > > > > module. */
> > > > > The size calculations are duplicated, could you either stash
> > > > > start_info_size into xc_dom_image, or simply do the memory
> > > > > allocation
> > > > > (xc_dom_alloc_segment) inside of bootlate_hvm? (I think the
> > > > > latter
> > > > > would
> > > > > be better if possible).
> > > > I didn't want to do the first because we'd use that information
> > > > (two
> > > > pieces --- we need to to know both the size of the extra chunk and
> > > > where
> > > > modlist starts) only once and it's not on a critical path. You can,
> > > > of
> > > > course, argue that we increase text size.
> > > It's just that I don't want to get them out of synch, and that's what
> > > usually happens when you perform the same calculations in two
> > > different
> > > places, one might get out of synch with the other.
> > > 
> > > > The problem with the second approach is that while it does seem to
> > > > work
> > > > I don't know whether we can delay allocations until bootlate():
> > > > notice
> > > > how we print dom->virt_alloc_end in xc_dom_build_image() which to
> > > > me
> > > > indicates some "finality" as far as allocations for domain are
> > > > concerned.
> > > For PV guests it probably matters, because we have to build the page
> > > tables and the p2m, for HVMlite guests I don't think it matters at
> > > all
> > > (or at least I don't see any obvious reason).
> > > 
> > > Another third option is to place the code that performs the size
> > > calculations inside of a function that's called by both (bootlate_hvm
> > > and alloc_magic_pages_hvm).
> > The call to xc_dom_alloc_segment initialises a xc_dom_seg, which in
> > this
> > case alloc_magic just throws away. If the location/size of that segment
> > is
> > required elsewhere then the normal approach would be to add it to the
> > handle struct (cf dom->{kernel,ramdisk}_seg et al) and to consume it in
> > the
> > other places.
> 
> xc_dom_alloc_segment() also updates domain's pfn_alloc_end and
> virt_alloc_end, which is what I was thinking about (i.e. that updating
> those values bootlate() time may be too late).

I was assuming the reason for calculating the size twice was
that xc_dom_alloc_segment() was called in the earlier hook, which is why I
mentioned stacking the xc_dom_seg.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [OSSTEST PATCH 0/7] Better database handling in Tcl

2016-01-07 Thread Ian Jackson
This series arranges for the owner daemon to be able to tolerate
database restarts, and is generally much more careful about database
errors in Tcl.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [OSSTEST PATCH 4/7] Database locking: Tcl: Always use db-execute-array for SELECT

2016-01-07 Thread Ian Jackson
We are going to make it wrong to use db-execute for SELECT statements.

Convert the two existing violation sites (providing a dummy arrayvar).

Signed-off-by: Ian Jackson 
---
 sg-execute-flight   |2 +-
 tcl/JobDB-Executive.tcl |2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/sg-execute-flight b/sg-execute-flight
index 4e3fcf2..008e661 100755
--- a/sg-execute-flight
+++ b/sg-execute-flight
@@ -30,7 +30,7 @@ proc check {} {
 
 jobdb::db-open
 
-set nqueued [jobdb::db-execute "
+set nqueued [jobdb::db-execute-array dummy "
 SELECT job FROM jobs j
  WHERE j.flight = $flight
AND j.status = 'queued'
diff --git a/tcl/JobDB-Executive.tcl b/tcl/JobDB-Executive.tcl
index 239f22c..bbce6fc 100644
--- a/tcl/JobDB-Executive.tcl
+++ b/tcl/JobDB-Executive.tcl
@@ -275,7 +275,7 @@ proc become-task {comment} {
 set username "[id user]@$hostname"
 
 transaction resources {
-set nrows [db-execute "
+set nrows [db-execute-array dummy "
 UPDATE tasks
SET username = [pg_quote $username],
comment = [pg_quote $comment]
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [OSSTEST PATCH 5/7] Database locking: Tcl: for errorCode, use pg_exec, not pg_execute

2016-01-07 Thread Ian Jackson
We would like to be able to retry db transactions.  To do this we need
to know why they failed (if they did).

But pg_execute does not set errorCode.  (This is clearly a bug.)  And
since it immediately discards a failed statement, any error
information has been lost by the time pg_execute returns.

So, instead, use pg_exec, and manually mess about with fishing
suitable information out of a failed statement handle, and generating
an appropriate errorCode.

There are no current consumers of this errorCode: that will come in a
moment.

A wrinkle is that as a result it is no longer possible to use
db-execute on a SELECT statement nor db-execute-array on a non-SELECT
statement.  This is because the 1. the `ok' status that we have to
check for is different for statements which are commands and ones
which return tupes and 2. we need to fish a different return value out
of the statement handle (-cmdTuples vs -numTuples).  But all uses in
the codebase are now fine for this distinction.

Signed-off-by: Ian Jackson 
---
 tcl/JobDB-Executive.tcl |   54 ---
 1 file changed, 51 insertions(+), 3 deletions(-)

diff --git a/tcl/JobDB-Executive.tcl b/tcl/JobDB-Executive.tcl
index bbce6fc..ed9abbb 100644
--- a/tcl/JobDB-Executive.tcl
+++ b/tcl/JobDB-Executive.tcl
@@ -121,13 +121,61 @@ proc db-execute-debug {stmt} {
puts stderr "EXECUTING >$stmt<"
 }
 }
+
+proc db--exec-check {shvar stmt expected_status body} {
+# pg_execute does not set errorCode and it throws away the
+# statement handle so we can't get the error out.  So
+# use pg_exec, as wrapped up here.
+
+# db--exec-check executes stmt and checks that the status is
+# `expected_status'.  If OK, executes body with $shvar set to the
+# stmt handle.   Otherwise throws with errorCode
+#   {OSSTEST-PSQL  }
+
+global errorInfo errorCode
+upvar 1 $shvar sh
+
+set sh [pg_exec dbh $stmt]
+
+set rc [catch {
+   set status [pg_result $sh -status]
+   if {[string compare $status $expected_status]} {
+   set emsg [pg_result $sh -error]
+   set sqlstate [pg_result $sh -error sqlstate]
+   if {![string length $emsg]} {
+   set emsg "osstest expected status $expected_status got $status"
+   }
+   set context [pg_result $sh -error context]
+   error $emsg \
+   "while executing SQL\n$stmt\nin SQL context\n$context" \
+   [list OSSTEST-PSQL $status $sqlstate]
+   }
+   uplevel 1 $body
+} emsg]
+
+set ei $errorInfo
+set ec $errorCode
+catch { pg_result $sh -clear }
+
+return -code $rc -errorinfo $ei -errorcode $ec $emsg
+}
+
 proc db-execute {stmt} {
 db-execute-debug $stmt
-uplevel 1 [list pg_execute dbh $stmt]
+db--exec-check sh $stmt PGRES_COMMAND_OK {
+   return [pg_result $sh -cmdTuples]
+}
 }
-proc db-execute-array {arrayvar stmt args} {
+proc db-execute-array {arrayvar stmt {body {}}} {
 db-execute-debug $stmt
-uplevel 1 [list pg_execute -array $arrayvar dbh $stmt] $args
+db--exec-check sh $stmt PGRES_TUPLES_OK {
+   set nrows [pg_result $sh -numTuples]
+   for {set row 0} {$row < $nrows} {incr row} {
+   uplevel 1 [list pg_result $sh -tupleArray $row $arrayvar]
+   uplevel 1 $body
+   }
+   return $nrows
+}
 }
 
 proc lock-tables {tables} {
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [OSSTEST PATCH 7/7] ms-ownerdaemon: Cope with db restart. Retry recording dead tasks.

2016-01-07 Thread Ian Jackson
In chan-destroy-stuff, instead of accessing the db directly, add the
dead task(s) to a queue, and arrange to look at that queue.

Errors are handled by setting an `after' handler which we cancel if we
are successful.

The after handler requeues a queue run attempt as the first thing
(which will arrange that a further retry will occur if things are
still broken) and then attempts to reconnect to the database.

I have tested this with a test instance by renaming the `tasks' table
under its feet, and it functions as expected.

DEPLOYMENT NOTE: The owner daemon cannot be restarted without shutting
everything down.  So this update should first be deployed in
Cambridge, probably, to see how it goes.  Also, it is less critical in
the main Xen production test lab because there the db and the owner
daemon are co-hosted on the same VM.

Signed-off-by: Ian Jackson 
---
 Osstest/Executive.pm |1 +
 ms-ownerdaemon   |   37 +
 2 files changed, 34 insertions(+), 4 deletions(-)

diff --git a/Osstest/Executive.pm b/Osstest/Executive.pm
index 2314577..d31fafb 100644
--- a/Osstest/Executive.pm
+++ b/Osstest/Executive.pm
@@ -113,6 +113,7 @@ augmentconfigdefaults(
 augmentconfigdefaults(
 OwnerDaemonHost => $c{ControlDaemonHost},
 QueueDaemonHost => $c{ControlDaemonHost},
+OwnerDaemonDbRetry => $c{QueueDaemonRetry},
 );
 
 #-- configuration reader etc. --
diff --git a/ms-ownerdaemon b/ms-ownerdaemon
index 502dcfe..318549a 100755
--- a/ms-ownerdaemon
+++ b/ms-ownerdaemon
@@ -22,16 +22,37 @@
 source ./tcl/daemonlib.tcl
 
 
+set dead_tasks {}
+
 proc chan-destroy-stuff {chan} {
+global dead_tasks
+
 upvar #0 chanawait($chan) await
 catch { unset await }
 
 upvar #0 chantasks($chan) tasks
 if {![info exists tasks]} return
 
+puts-chan-desc $chan "-- $tasks"
+
+foreach task $tasks {
+   lappend dead_tasks $task
+}
+after idle record-dead-tasks
+}
+
+proc record-dead-tasks {} {
+global c dead_tasks
+
+if {![llength $dead_tasks]} return
+
+puts "record-dead-tasks ... $dead_tasks"
+
+set retry [expr {$c(OwnerDaemonDbRetry) * 1000}]
+set eafter [after $retry record-dead-tasks-retry]
+
 jobdb::transaction resources {
-puts-chan-desc $chan "-- $tasks"
-foreach task $tasks {
+foreach task $dead_tasks {
 jobdb::db-execute "
 UPDATE tasks
SET live = 'f'
@@ -39,12 +60,20 @@ proc chan-destroy-stuff {chan} {
 "
 }
 }
-puts-chan-desc $chan "== $tasks"
-unset tasks
 
+after cancel $eafter
+puts "record-dead-tasks OK. $dead_tasks"
+set dead_tasks {}
 after idle await-endings-notify
 }
 
+proc record-dead-tasks-retry {} {
+after idle record-dead-tasks
+puts "** reconnecting/retrying **"
+catch { jobdb::db-close }
+jobdb::db-open
+}
+
 proc await-endings-notify {} {
 global chanawait
 foreach chan [array names chanawait] {
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [OSSTEST PATCH 6/7] Database locking: Tcl: Retry only on DEADLOCK DETECTED

2016-01-07 Thread Ian Jackson
Use the new errorCode coming out of db-execute* to tell when the error
is that we got a database deadlock, which is the situation in which we
should retry.

This involves combining the two catch blocks, so that there is only
one error handling strategy.  Previously errors on COMMIT would be
retried and others would not.  Now errors anywhere might be retried
but only if the DB indicated deadlock.

We now unconditionally execute ROLLBACK.  This is more correct, since
we always previously executed BEGIN.

And, we pass the errorInfo and errorCode from the $body to the caller.

I have tested this with a test db instance, using contrived means to
generate a database deadlock, and it does actually retry.

Signed-off-by: Ian Jackson 
---
 tcl/JobDB-Executive.tcl |   30 --
 1 file changed, 16 insertions(+), 14 deletions(-)

diff --git a/tcl/JobDB-Executive.tcl b/tcl/JobDB-Executive.tcl
index ed9abbb..63db4f0 100644
--- a/tcl/JobDB-Executive.tcl
+++ b/tcl/JobDB-Executive.tcl
@@ -283,25 +283,27 @@ proc transaction {tables script} {
db-execute "SET TRANSACTION ISOLATION LEVEL SERIALIZABLE"
lock-tables $tables
uplevel 1 $script
+   db-execute COMMIT
} result]
-   if {!$rc} {
-   if {[catch {
-   db-execute COMMIT
-   } emsg]} {
-   puts "commit failed: $emsg; retrying ..."
-   db-execute ROLLBACK
-   if {[incr retries -1] <= 0} {
-   error \
- "commit failed, too many retries: $emsg\n$errorInfo\n$errorCode\n"
+   set ei $errorInfo
+   set ec $errorCode
+   if {$rc} {
+   db-execute ROLLBACK
+   switch -glob $errorCode {
+   {OSSTEST-PSQL * 40P01} {
+   # DEADLOCK DETECTED
+   puts "transaction deadlock ($result) retrying ..."
+   if {[incr retries -1] <= 0} {
+   error \
+ "transaction failed, too many retries: $result\n$errorInfo\n$errorCode\n"
+   }
+   after 500
+   continue
}
-   after 500
-   continue
}
-   } else {
-   db-execute ROLLBACK
}
 db-close
-   return -code $rc $result
+   return -code $rc -errorinfo $ei -errorcode $ec $result
 }
 }
 
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [xen-unstable-smoke test] 77413: tolerable all pass - PUSHED

2016-01-07 Thread osstest service owner
flight 77413 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/77413/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  18eed42622b698e99f2950c293ae969ccaffe9ef
baseline version:
 xen  68778eeaa3babedba9677400f63f1e7564bba561

Last test of basis77380  2016-01-07 14:00:56 Z0 days
Failing since 77400  2016-01-07 16:02:20 Z0 days2 attempts
Testing same since77413  2016-01-07 18:01:49 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Bob Moore 
  Boris Ostrovsky 
  Daniel De Graaf 
  Doug Goldstein 
  Feng Wu 
  Graeme Gregory 
  Hanjun Guo 
  Jan Beulich 
  Joshua Otto 
  Kevin Tian 
  Len Brown 
  Lin Ming 
  Lv Zheng 
  Paul Durrant 
  Rafael J. Wysocki 
  Shannon Zhao 
  Stefano Stabellini 
  Tomasz Nowicki 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=xen-unstable-smoke
+ revision=18eed42622b698e99f2950c293ae969ccaffe9ef
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push xen-unstable-smoke 
18eed42622b698e99f2950c293ae969ccaffe9ef
+ branch=xen-unstable-smoke
+ revision=18eed42622b698e99f2950c293ae969ccaffe9ef
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable-smoke
+ qemuubranch=qemu-upstream-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' xqemu-upstream-unstable = x ']'
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable-smoke
+ prevxenbranch=xen-unstable
+ '[' x18eed42622b698e99f2950c293ae969ccaffe9ef = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/rumpuser-xen.git
+++ besteffort_repo https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ cached_repo https://github.com/rumpkernel/rumpkernel-netbsd-src 
'[fetch=try]'
+++ local repo=https://git

[Xen-devel] [qemu-upstream-4.5-testing test] 77247: regressions - FAIL

2016-01-07 Thread osstest service owner
flight 77247 qemu-upstream-4.5-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/77247/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-ovmf-amd64 9 debian-hvm-install fail REGR. vs. 62414
 test-amd64-i386-xl-qemuu-ovmf-amd64  9 debian-hvm-install fail REGR. vs. 62414

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-arndale 15 guest-start/debian.repeat fail in 66752 pass in 
77247
 test-armhf-armhf-xl-rtds  9 debian-install  fail pass in 66752

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-qemuu-win7-amd64 15 guest-localmigrate/x10 fail in 66752 
like 62075
 test-amd64-amd64-xl-rtds  6 xen-boot fail   like 62414
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 62414
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail like 62414

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds 13 saverestore-support-check fail in 66752 never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-check fail in 66752 never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 10 guest-start  fail never pass
 test-armhf-armhf-libvirt-raw 10 guest-start  fail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  10 guest-start  fail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass

version targeted for testing:
 qemuu32bba3499008c847e08858f310d65806e0bade36
baseline version:
 qemuue5a1bb22cfb307db909dbd3404c48e5bbeb9e66d

Last test of basis62414  2015-09-26 20:34:49 Z  102 days
Testing same since66534  2015-12-18 15:48:15 Z   20 days8 attempts


People who touched revisions under test:
  Stefano Stabellini 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl  pass
 test-armhf-armhf-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-amd64-qemuu-nested-amdfail
 test-amd64-amd64-xl-pvh-amd  fail
 test-amd64-i386-qemuu-rhel6hvm-amd   pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
 test-amd64-i386-freebsd10-amd64  pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail
 test-amd64-amd64-xl-qemuu-win7-amd64 fail
 test-amd64-i386-xl-qemuu-win7-amd64  fail
 test-armhf-armhf-xl-arndale  pass
 test-amd64-amd64-xl-credit2 

[Xen-devel] [seabios test] 77260: tolerable FAIL - PUSHED

2016-01-07 Thread osstest service owner
flight 77260 seabios real [real]
http://logs.test-lab.xenproject.org/osstest/logs/77260/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 77156

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass

version targeted for testing:
 seabios  16a9e7926a6c6845a98df2a6eac509c23c6206ba
baseline version:
 seabios  71479612401b794e6cb5025f61e5352c51f35877

Last test of basis77156  2016-01-05 17:41:30 Z2 days
Testing same since77260  2016-01-06 17:05:18 Z1 days1 attempts


People who touched revisions under test:
  Kevin O'Connor 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsmpass
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm pass
 test-amd64-amd64-qemuu-nested-amdfail
 test-amd64-i386-qemuu-rhel6hvm-amd   pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass
 test-amd64-amd64-xl-qemuu-win7-amd64 fail
 test-amd64-i386-xl-qemuu-win7-amd64  fail
 test-amd64-amd64-qemuu-nested-intel  pass
 test-amd64-i386-qemuu-rhel6hvm-intel pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 pass
 test-amd64-amd64-xl-qemuu-winxpsp3   pass
 test-amd64-i386-xl-qemuu-winxpsp3pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=seabios
+ revision=16a9e7926a6c6845a98df2a6eac509c23c6206ba
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push seabios 
16a9e7926a6c6845a98df2a6eac509c23c6206ba
+ branch=seabios
+ revision=16a9e7926a6c6845a98df2a6eac509c23c6206ba
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=seabios
+ xenbranch=xen-unstable
+ '[' xseabios = xlinux ']'
+ linuxbranch=
+ '[' x = x ']'
+ qemuubranch=qemu-upst

Re: [Xen-devel] [PATCH v3 59/62] xen/arm: Add a hypercall for device mmio mapping

2016-01-07 Thread Daniel De Graaf

On 01/07/2016 05:50 AM, Jan Beulich wrote:

On 07.01.16 at 10:11,  wrote:

Hi Jan,

On 2016/1/7 15:45, Jan Beulich wrote:

On 07.01.16 at 07:58,  wrote:

On 2015/11/17 19:04, Jan Beulich wrote:

On 17.11.15 at 10:40,  wrote:

--- a/xen/arch/arm/mm.c
+++ b/xen/arch/arm/mm.c
@@ -1138,6 +1138,10 @@ int xenmem_add_to_physmap_one(
  rcu_unlock_domain(od);
  break;
  }
+case XENMAPSPACE_dev_mmio:
+rc = map_dev_mmio_region(d, gpfn, 1, idx);
+return rc;
+break;

Blindly for any kind of domain? The XSM check in the
XENMEM_add_to_physmap_batch handler (in common code) doesn't
even know which map space is to be used...


Sorry, I know little about XSM. Could you suggest me how to add the
check for this new type here?

I'm sorry to push back here, but did you at least try to derive
what is wanted from the multitude of other XSM checks present
throughout the tree?


IIUC, you mean that it doean't need to change the XSM check itself, but
we should check if the current->domain is hardware domain and it maps
the space to itself before the XSM check, right?


No, I actually think that you need to add a new, secondary XSM
check. But you may want to consult with Daniel (who so far wasn't
even Cc-ed).


Looking at the original patch, I am not sure if I understand the
checks: it seems like the iomem_access_permitted check is being done
on the guest's page range instead of the actual IO memory, which
ends up allowing the guest to map anything as long as it maps it in
the right guest area.  The iomem_permit_access call there also seems
to be redundant because it is the same range that was just checked.

If the [start_gfn, start_gfn + nr) memory range actually describes
the physical addresses, then this operation is taking advantage of
the existing XSM checks on XEN_DOMCTL_iomem_permission, and the
only XSM check that is needed would be that current->domain has
permission to modify (d)'s mappings - and this is done by the
xsm_add_to_physmap check in XENMEM_add_to_physmap.

--
Daniel De Graaf
National Security Agency

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v5] x86/VPMU: implement ipc and arch filter flags

2016-01-07 Thread Brendan Gregg
On Thu, Jan 7, 2016 at 6:12 AM, Jan Beulich  wrote:

> >>> On 05.01.16 at 02:43,  wrote:
> > This introduces a way to have a restricted VPMU, by specifying one of two
> > predefined groups of PMCs to make available. For secure environments,
> this
> > allows the VPMU to be used without needing to enable all PMCs.
> >
> > Signed-off-by: Brendan Gregg 
> > Reviewed-by: Boris Ostrovsky 
>
> I was about to apply with
>
> >  static DEFINE_PER_CPU(struct vcpu *, last_vcpu);
> >
> > -static void __init parse_vpmu_param(char *s)
> > +static int parse_vpmu_param(char *s, int len)
>
> static bool_t __init parse_vpmu_param(char *s, unsigned int len)
>

Ok.


>
> >  {
> > +if ( ! *s || ! len )
>
> if ( !*s || !len )
>
>
Ok.


> > +return 0;
> > +if ( !strncmp(s, "bts", len) )
> > +vpmu_features |= XENPMU_FEATURE_INTEL_BTS;
> > +else if ( !strncmp(s, "ipc", len) )
> > +vpmu_features |= XENPMU_FEATURE_IPC_ONLY;
> > +else if ( !strncmp(s, "arch", len) )
> > +vpmu_features |= XENPMU_FEATURE_ARCH_ONLY;
> > +else
> > +return 1;
> > +return 0;
> > +}
> > +
> > +static void __init parse_vpmu_params(char *s)
> > +{
> > +char *sep, *p = s;
> > +
> >  switch ( parse_bool(s) )
> >  {
> >  case 0:
> >  break;
> >  default:
> > -if ( !strcmp(s, "bts") )
> > -vpmu_features |= XENPMU_FEATURE_INTEL_BTS;
> > -else if ( *s )
> > +while (1)
> >  {
> > -printk("VPMU: unknown flag: %s - vpmu disabled!\n", s);
> > -break;
> > +sep = strchr(p, ',');
> > +if ( sep == NULL )
> > +sep = strchr(p, 0);
> > +if ( parse_vpmu_param(p, sep - p) )
> > +goto error;
> > +if ( *sep == 0 )
>
> if ( !*sep )
>
>
Ok.


> > --- a/xen/arch/x86/cpu/vpmu_intel.c
> > +++ b/xen/arch/x86/cpu/vpmu_intel.c
> > @@ -604,12 +604,17 @@ static int core2_vpmu_do_wrmsr(unsigned int msr,
> > uint64_t msr_content,
> >   "MSR_PERF_GLOBAL_STATUS(0x38E)!\n");
> >  return -EINVAL;
> >  case MSR_IA32_PEBS_ENABLE:
> > +if ( vpmu_features & (XENPMU_FEATURE_IPC_ONLY |
> > + XENPMU_FEATURE_ARCH_ONLY) )
>
>   XENPMU_FEATURE_ARCH_ONLY) )
>
>
Ok, yes, neater.


> > @@ -656,12 +661,55 @@ static int core2_vpmu_do_wrmsr(unsigned int msr,
> uint64_t msr_content,
> >  tmp = msr - MSR_P6_EVNTSEL(0);
> >  if ( tmp >= 0 && tmp < arch_pmc_cnt )
> >  {
> > +bool_t blocked = 0;
> > +uint64_t umaskevent;
> >  struct xen_pmu_cntr_pair *xen_pmu_cntr_pair =
> >  vpmu_reg_pointer(core2_vpmu_cxt, arch_counters);
> >
> >  if ( msr_content & ARCH_CTRL_MASK )
> >  return -EINVAL;
> >
> > +/* PMC filters */
> > +umaskevent = msr_content & MSR_IA32_CMT_EVTSEL_UE_MASK;
> > +if ( vpmu_features & XENPMU_FEATURE_IPC_ONLY ||
> > + vpmu_features & XENPMU_FEATURE_ARCH_ONLY )
>
> if ( vpmu_features & (XENPMU_FEATURE_IPC_ONLY |
>   XENPMU_FEATURE_ARCH_ONLY) )
>
>
Ok.


> > +{
> > +blocked = 1;
> > +switch ( umaskevent )
> > +{
> > +/*
> > + * See the Pre-Defined Architectural Performance Events
> table
> > + * from the Intel 64 and IA-32 Architectures Software
> > + * Developer's Manual, Volume 3B, System Programming
> Guide,
> > + * Part 2.
> > + */
> > +case 0x003c: /* UnHalted Core Cycles */
> > +case 0x013c: /* UnHalted Reference Cycles */
> > +case 0x00c0: /* Instructions Retired */
> > +blocked = 0;
> > +default:
> > +break;
>
> dropped last two lines
>
>
Ok.


> > +}
> > +}
> > +
> > +if ( vpmu_features & XENPMU_FEATURE_ARCH_ONLY )
> > +{
> > +/* additional counters beyond IPC only; blocked already
> set */
>
> /* Additional counters beyond IPC only; blocked already
> set. */
>
> > +switch ( umaskevent )
> > +{
> > +case 0x4f2e: /* Last Level Cache References */
> > +case 0x412e: /* Last Level Cache Misses */
> > +case 0x00c4: /* Branch Instructions Retired */
> > +case 0x00c5: /* All Branch Mispredict Retired */
> > +blocked = 0;
> > +default:
> > +break;
>
> Again
>

Ok.


>
> > --- a/xen/include/public/pmu.h
> > +++ b/xen/include/public/pmu.h
> > @@ -84,9 +84,19 @@ DEFINE_XEN_GUEST_HANDLE(xen_pmu_params_t);
> >
> >  /*
> >   * PMU features:
> > - * - XENPMU_FEATURE_INTEL_BTS: Intel BTS support (ignor

[Xen-devel] [PATCH v4] libxc: Defer initialization of start_page for HVM guests

2016-01-07 Thread Boris Ostrovsky
With commit 8c45adec18e0 ("libxc: create unmapped initrd in domain
builder if supported") location of ramdisk may not be available to
HVMlite guests by the time alloc_magic_pages_hvm() is invoked if the
guest supports unmapped initrd.

So let's move ramdisk info initialization (along with a few other
operations that are not directly related to allocating magic/special
pages) from alloc_magic_pages_hvm() to bootlate_hvm().

Since we now split allocation and mapping of the start_info segment
let's stash it, along with cmdline length, in xc_dom_image so that we
can check whether we are mapping correctly-sized range.

We can also stop using xc_dom_image.start_info_pfn and leave it for
PV(H) guests only.

Signed-off-by: Boris Ostrovsky 
---
v4:
 * See the last two paragraphs from commit message above

 tools/libxc/include/xc_dom.h |2 +
 tools/libxc/xc_dom_x86.c |  140 +++--
 2 files changed, 80 insertions(+), 62 deletions(-)

diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
index 2460818..cac4698 100644
--- a/tools/libxc/include/xc_dom.h
+++ b/tools/libxc/include/xc_dom.h
@@ -71,6 +71,7 @@ struct xc_dom_image {
 
 /* arguments and parameters */
 char *cmdline;
+size_t cmdline_size;
 uint32_t f_requested[XENFEAT_NR_SUBMAPS];
 
 /* info from (elf) kernel image */
@@ -91,6 +92,7 @@ struct xc_dom_image {
 struct xc_dom_seg p2m_seg;
 struct xc_dom_seg pgtables_seg;
 struct xc_dom_seg devicetree_seg;
+struct xc_dom_seg start_info_seg; /* HVMlite only */
 xen_pfn_t start_info_pfn;
 xen_pfn_t console_pfn;
 xen_pfn_t xenstore_pfn;
diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
index b8d2904..8676c97 100644
--- a/tools/libxc/xc_dom_x86.c
+++ b/tools/libxc/xc_dom_x86.c
@@ -586,23 +586,12 @@ static void build_hvm_info(void *hvm_info_page, struct 
xc_dom_image *dom)
 static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
 {
 unsigned long i;
-void *hvm_info_page;
 uint32_t *ident_pt, domid = dom->guest_domid;
 int rc;
 xen_pfn_t special_array[X86_HVM_NR_SPECIAL_PAGES];
 xen_pfn_t ioreq_server_array[NR_IOREQ_SERVER_PAGES];
 xc_interface *xch = dom->xch;
 
-if ( dom->device_model )
-{
-if ( (hvm_info_page = xc_map_foreign_range(
-  xch, domid, PAGE_SIZE, PROT_READ | PROT_WRITE,
-  HVM_INFO_PFN)) == NULL )
-goto error_out;
-build_hvm_info(hvm_info_page, dom);
-munmap(hvm_info_page, PAGE_SIZE);
-}
-
 /* Allocate and clear special pages. */
 for ( i = 0; i < X86_HVM_NR_SPECIAL_PAGES; i++ )
 special_array[i] = special_pfn(i);
@@ -636,65 +625,25 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
 
 if ( !dom->device_model )
 {
-struct xc_dom_seg seg;
-struct hvm_start_info *start_info;
-char *cmdline;
-struct hvm_modlist_entry *modlist;
-void *start_page;
-size_t cmdline_size = 0;
-size_t start_info_size = sizeof(*start_info);
+size_t start_info_size = sizeof(struct hvm_start_info);
 
 if ( dom->cmdline )
 {
-cmdline_size = ROUNDUP(strlen(dom->cmdline) + 1, 8);
-start_info_size += cmdline_size;
-
+dom->cmdline_size = ROUNDUP(strlen(dom->cmdline) + 1, 8);
+start_info_size += dom->cmdline_size;
 }
+
+/* Limited to one module. */
 if ( dom->ramdisk_blob )
-start_info_size += sizeof(*modlist); /* Limited to one module. */
+start_info_size += sizeof(struct hvm_modlist_entry);
 
-rc = xc_dom_alloc_segment(dom, &seg, "HVMlite start info", 0,
-  start_info_size);
+rc = xc_dom_alloc_segment(dom, &dom->start_info_seg,
+  "HVMlite start info", 0, start_info_size);
 if ( rc != 0 )
 {
 DOMPRINTF("Unable to reserve memory for the start info");
 goto out;
 }
-
-start_page = xc_map_foreign_range(xch, domid, start_info_size,
-  PROT_READ | PROT_WRITE,
-  seg.pfn);
-if ( start_page == NULL )
-{
-DOMPRINTF("Unable to map HVM start info page");
-goto error_out;
-}
-
-start_info = start_page;
-cmdline = start_page + sizeof(*start_info);
-modlist = start_page + sizeof(*start_info) + cmdline_size;
-
-if ( dom->cmdline )
-{
-strncpy(cmdline, dom->cmdline, cmdline_size);
-start_info->cmdline_paddr = (seg.pfn << PAGE_SHIFT) +
-((uintptr_t)cmdline - (uintptr_t)start_info);
-}
-
-if ( dom->ramdisk_blob )
-{
-modlist[0].paddr = dom->ramdisk_seg.vstart - dom->parms.virt_base;
-modlist[0].size = dom->ramdisk_seg.

[Xen-devel] [PATCH v4.1] libxc: Defer initialization of start_page for HVM guests

2016-01-07 Thread Boris Ostrovsky
With commit 8c45adec18e0 ("libxc: create unmapped initrd in domain
builder if supported") location of ramdisk may not be available to
HVMlite guests by the time alloc_magic_pages_hvm() is invoked if the
guest supports unmapped initrd.

So let's move ramdisk info initialization (along with a few other
operations that are not directly related to allocating magic/special
pages) from alloc_magic_pages_hvm() to bootlate_hvm().

Since we now split allocation and mapping of the start_info segment
let's stash it, along with cmdline length, in xc_dom_image so that we
can check whether we are mapping correctly-sized range.

We can also stop using xc_dom_image.start_info_pfn and leave it for
PV(H) guests only.

Signed-off-by: Boris Ostrovsky 
---

v4:
 * See the last two paragraphs from commit message above

v4.1:
 * Inverted testing of start_info_size in bootlate_hvm().

 tools/libxc/include/xc_dom.h |2 +
 tools/libxc/xc_dom_x86.c |  140 +++--
 2 files changed, 80 insertions(+), 62 deletions(-)

diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
index 2460818..cac4698 100644
--- a/tools/libxc/include/xc_dom.h
+++ b/tools/libxc/include/xc_dom.h
@@ -71,6 +71,7 @@ struct xc_dom_image {
 
 /* arguments and parameters */
 char *cmdline;
+size_t cmdline_size;
 uint32_t f_requested[XENFEAT_NR_SUBMAPS];
 
 /* info from (elf) kernel image */
@@ -91,6 +92,7 @@ struct xc_dom_image {
 struct xc_dom_seg p2m_seg;
 struct xc_dom_seg pgtables_seg;
 struct xc_dom_seg devicetree_seg;
+struct xc_dom_seg start_info_seg; /* HVMlite only */
 xen_pfn_t start_info_pfn;
 xen_pfn_t console_pfn;
 xen_pfn_t xenstore_pfn;
diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
index b8d2904..8676c97 100644
--- a/tools/libxc/xc_dom_x86.c
+++ b/tools/libxc/xc_dom_x86.c
@@ -586,23 +586,12 @@ static void build_hvm_info(void *hvm_info_page, struct 
xc_dom_image *dom)
 static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
 {
 unsigned long i;
-void *hvm_info_page;
 uint32_t *ident_pt, domid = dom->guest_domid;
 int rc;
 xen_pfn_t special_array[X86_HVM_NR_SPECIAL_PAGES];
 xen_pfn_t ioreq_server_array[NR_IOREQ_SERVER_PAGES];
 xc_interface *xch = dom->xch;
 
-if ( dom->device_model )
-{
-if ( (hvm_info_page = xc_map_foreign_range(
-  xch, domid, PAGE_SIZE, PROT_READ | PROT_WRITE,
-  HVM_INFO_PFN)) == NULL )
-goto error_out;
-build_hvm_info(hvm_info_page, dom);
-munmap(hvm_info_page, PAGE_SIZE);
-}
-
 /* Allocate and clear special pages. */
 for ( i = 0; i < X86_HVM_NR_SPECIAL_PAGES; i++ )
 special_array[i] = special_pfn(i);
@@ -636,65 +625,25 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
 
 if ( !dom->device_model )
 {
-struct xc_dom_seg seg;
-struct hvm_start_info *start_info;
-char *cmdline;
-struct hvm_modlist_entry *modlist;
-void *start_page;
-size_t cmdline_size = 0;
-size_t start_info_size = sizeof(*start_info);
+size_t start_info_size = sizeof(struct hvm_start_info);
 
 if ( dom->cmdline )
 {
-cmdline_size = ROUNDUP(strlen(dom->cmdline) + 1, 8);
-start_info_size += cmdline_size;
-
+dom->cmdline_size = ROUNDUP(strlen(dom->cmdline) + 1, 8);
+start_info_size += dom->cmdline_size;
 }
+
+/* Limited to one module. */
 if ( dom->ramdisk_blob )
-start_info_size += sizeof(*modlist); /* Limited to one module. */
+start_info_size += sizeof(struct hvm_modlist_entry);
 
-rc = xc_dom_alloc_segment(dom, &seg, "HVMlite start info", 0,
-  start_info_size);
+rc = xc_dom_alloc_segment(dom, &dom->start_info_seg,
+  "HVMlite start info", 0, start_info_size);
 if ( rc != 0 )
 {
 DOMPRINTF("Unable to reserve memory for the start info");
 goto out;
 }
-
-start_page = xc_map_foreign_range(xch, domid, start_info_size,
-  PROT_READ | PROT_WRITE,
-  seg.pfn);
-if ( start_page == NULL )
-{
-DOMPRINTF("Unable to map HVM start info page");
-goto error_out;
-}
-
-start_info = start_page;
-cmdline = start_page + sizeof(*start_info);
-modlist = start_page + sizeof(*start_info) + cmdline_size;
-
-if ( dom->cmdline )
-{
-strncpy(cmdline, dom->cmdline, cmdline_size);
-start_info->cmdline_paddr = (seg.pfn << PAGE_SHIFT) +
-((uintptr_t)cmdline - (uintptr_t)start_info);
-}
-
-if ( dom->ramdisk_blob )
-{
-modlist[0].paddr = dom->ramdisk_seg.vstart - dom-

[Xen-devel] [PATCH v4.2] libxc: Defer initialization of start_page for HVM guests

2016-01-07 Thread Boris Ostrovsky
With commit 8c45adec18e0 ("libxc: create unmapped initrd in domain
builder if supported") location of ramdisk may not be available to
HVMlite guests by the time alloc_magic_pages_hvm() is invoked if the
guest supports unmapped initrd.

So let's move ramdisk info initialization (along with a few other
operations that are not directly related to allocating magic/special
pages) from alloc_magic_pages_hvm() to bootlate_hvm().

Since we now split allocation and mapping of the start_info segment
let's stash it, along with cmdline length, in xc_dom_image so that we
can check whether we are mapping correctly-sized range.

We can also stop using xc_dom_image.start_info_pfn and leave it for
PV(H) guests only.

Signed-off-by: Boris Ostrovsky 
---
v4:
 * See the last two paragraphs from commit message above

v4.1:
 * Inverted testing of start_info_size in bootlate_hvm().

v4.2
 *  Actually do what I said I'd do in 4.1

 tools/libxc/include/xc_dom.h |2 +
 tools/libxc/xc_dom_x86.c |  140 +++--
 2 files changed, 80 insertions(+), 62 deletions(-)

diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
index 2460818..cac4698 100644
--- a/tools/libxc/include/xc_dom.h
+++ b/tools/libxc/include/xc_dom.h
@@ -71,6 +71,7 @@ struct xc_dom_image {
 
 /* arguments and parameters */
 char *cmdline;
+size_t cmdline_size;
 uint32_t f_requested[XENFEAT_NR_SUBMAPS];
 
 /* info from (elf) kernel image */
@@ -91,6 +92,7 @@ struct xc_dom_image {
 struct xc_dom_seg p2m_seg;
 struct xc_dom_seg pgtables_seg;
 struct xc_dom_seg devicetree_seg;
+struct xc_dom_seg start_info_seg; /* HVMlite only */
 xen_pfn_t start_info_pfn;
 xen_pfn_t console_pfn;
 xen_pfn_t xenstore_pfn;
diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
index b8d2904..edf2db1 100644
--- a/tools/libxc/xc_dom_x86.c
+++ b/tools/libxc/xc_dom_x86.c
@@ -586,23 +586,12 @@ static void build_hvm_info(void *hvm_info_page, struct 
xc_dom_image *dom)
 static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
 {
 unsigned long i;
-void *hvm_info_page;
 uint32_t *ident_pt, domid = dom->guest_domid;
 int rc;
 xen_pfn_t special_array[X86_HVM_NR_SPECIAL_PAGES];
 xen_pfn_t ioreq_server_array[NR_IOREQ_SERVER_PAGES];
 xc_interface *xch = dom->xch;
 
-if ( dom->device_model )
-{
-if ( (hvm_info_page = xc_map_foreign_range(
-  xch, domid, PAGE_SIZE, PROT_READ | PROT_WRITE,
-  HVM_INFO_PFN)) == NULL )
-goto error_out;
-build_hvm_info(hvm_info_page, dom);
-munmap(hvm_info_page, PAGE_SIZE);
-}
-
 /* Allocate and clear special pages. */
 for ( i = 0; i < X86_HVM_NR_SPECIAL_PAGES; i++ )
 special_array[i] = special_pfn(i);
@@ -636,65 +625,25 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
 
 if ( !dom->device_model )
 {
-struct xc_dom_seg seg;
-struct hvm_start_info *start_info;
-char *cmdline;
-struct hvm_modlist_entry *modlist;
-void *start_page;
-size_t cmdline_size = 0;
-size_t start_info_size = sizeof(*start_info);
+size_t start_info_size = sizeof(struct hvm_start_info);
 
 if ( dom->cmdline )
 {
-cmdline_size = ROUNDUP(strlen(dom->cmdline) + 1, 8);
-start_info_size += cmdline_size;
-
+dom->cmdline_size = ROUNDUP(strlen(dom->cmdline) + 1, 8);
+start_info_size += dom->cmdline_size;
 }
+
+/* Limited to one module. */
 if ( dom->ramdisk_blob )
-start_info_size += sizeof(*modlist); /* Limited to one module. */
+start_info_size += sizeof(struct hvm_modlist_entry);
 
-rc = xc_dom_alloc_segment(dom, &seg, "HVMlite start info", 0,
-  start_info_size);
+rc = xc_dom_alloc_segment(dom, &dom->start_info_seg,
+  "HVMlite start info", 0, start_info_size);
 if ( rc != 0 )
 {
 DOMPRINTF("Unable to reserve memory for the start info");
 goto out;
 }
-
-start_page = xc_map_foreign_range(xch, domid, start_info_size,
-  PROT_READ | PROT_WRITE,
-  seg.pfn);
-if ( start_page == NULL )
-{
-DOMPRINTF("Unable to map HVM start info page");
-goto error_out;
-}
-
-start_info = start_page;
-cmdline = start_page + sizeof(*start_info);
-modlist = start_page + sizeof(*start_info) + cmdline_size;
-
-if ( dom->cmdline )
-{
-strncpy(cmdline, dom->cmdline, cmdline_size);
-start_info->cmdline_paddr = (seg.pfn << PAGE_SHIFT) +
-((uintptr_t)cmdline - (uintptr_t)start_info);
-}
-
-if ( dom->ramdisk_blob )
-{
-mo

[Xen-devel] [xen-4.6-testing test] 77259: regressions - FAIL

2016-01-07 Thread osstest service owner
flight 77259 xen-4.6-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/77259/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-libvirt  3 host-install(3) broken in 77154 REGR. vs. 65639
 test-amd64-i386-xl-qemuu-ovmf-amd64  9 debian-hvm-install fail REGR. vs. 65639
 test-amd64-amd64-xl-qemuu-ovmf-amd64 9 debian-hvm-install fail REGR. vs. 65639

Tests which are failing intermittently (not blocking):
 test-amd64-i386-rumpuserxen-i386 15 
rumpuserxen-demo-xenstorels/xenstorels.repeat fail pass in 77154
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 9 debian-hvm-install fail 
pass in 77154

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds  9 debian-installfail REGR. vs. 65639
 test-amd64-amd64-rumpuserxen-amd64 15 
rumpuserxen-demo-xenstorels/xenstorels.repeat fail in 77154 like 65639
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail like 65639
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 65639

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt  1 build-check(1)blocked in 77154 n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)  blocked in 77154 n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)blocked in 77154 n/a
 test-armhf-armhf-libvirt-xsm  1 build-check(1)blocked in 77154 n/a
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  828ac175e5f8f616b14e49f5353bca4dc2d0efd1
baseline version:
 xen  850bcd0f42e4912b6605a2caa42d5c466ed58bd1

Last test of basis65639  2015-12-09 21:22:40 Z   29 days
Failing since 66394  2015-12-15 18:17:32 Z   23 days   11 attempts
Testing same since77154  2016-01-05 17:23:57 Z2 days2 attempts


People who touched revisions under test:
  Boris Ostrovsky 
  Brendan Gregg 
  Daniel Kiper 
  Dario Faggioli 
  David Vrabel 
  Ed Swierk 
  Haozhong Zhang 
  Ian Campbell 
  Ian Jackson 
  Jan Beulich 
  Kevin Tian 
  Malcolm Crossley 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  

Re: [Xen-devel] [PATCH] xen: fix missing XSM_ENABLE change

2016-01-07 Thread Daniel De Graaf

On 01/07/2016 01:42 PM, Doug Goldstein wrote:

This is broken from "xen: convert XSM_ENABLE to Kconfig"
6d5293032f5fc1c65f7a73548afaa3caa8e0105a. This hunk was dropped when I
made my v2 for some reason.

Signed-off-by: Doug Goldstein 


Acked-by: Daniel De Graaf 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v4 3/3] VT-d: Fix vt-d Device-TLB flush timeout issue.

2016-01-07 Thread Xu, Quan
On January 07, 2016 9:47 PM,  wrote:
> Jan,
> Patch 2/3 and Patch 3/3 are based on v3 (Actually they are v3's patch 1/2 and
> patch 2/2).
> We have discussed how to hide a device with pci_hide_device() when
> Device-TLB flush is error.
> 
> Now there are 2 concerns:
>   1. Hide the PCI device may break the code path. (then the pdev->domain
> is dom_xen)
>   2. Is the blew logic right?
>--If Device-TLB flush is timeout, we'll hide the target ATS device
> and crash the domain owning this ATS device.
>  If impacted domain is hardware domain, just throw out a
> warning, instead of crash the hardware domain.
> The hided Device will be disallowed to be further assigned to any

Sorry, just correct it. 'hided Device' -> 'hidden device'. 



> domain.
> 
> Kevin, correct me if I am wrong.
> 
> 
> Quan
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2 2/2] x86/hvm: add support for pcommit instruction

2016-01-07 Thread Haozhong Zhang
On 01/07/16 06:53, Jan Beulich wrote:
> >>> On 30.12.15 at 12:48,  wrote:
> > --- a/xen/arch/x86/hvm/hvm.c
> > +++ b/xen/arch/x86/hvm/hvm.c
> > @@ -4605,6 +4605,9 @@ void hvm_cpuid(unsigned int input, unsigned int *eax, 
> > unsigned int *ebx,
> >  
> >  if ( !cpu_has_clwb )
> >  *ebx &= ~cpufeat_mask(X86_FEATURE_CLWB);
> > +
> > +if ( !cpu_has_pcommit )
> > +*ebx &= ~cpufeat_mask(X86_FEATURE_PCOMMIT);
> 
> Other than for patch 1, this not only need to stay, but needs to be
> extended along the lines of X86_FEATURE_MPX handling.
>

In section "PCOMMIT - Virtualization Support" of Intel Architecture
Instruction Set Extensions Programming Reference, it says

IA32_VMX_PROCBASED_CTLS2[53] (which enumerates support for the
1-setting of “PCOMMIT exiting”) is always the same as
CPUID.07H:EBX.PCOMMIT[bit 22].

so checking cpu_has_pcommit is enough here.

> > @@ -1075,6 +1076,9 @@ static int construct_vmcs(struct vcpu *v)
> >  __vmwrite(PLE_WINDOW, ple_window);
> >  }
> >  
> > +if ( cpu_has_vmx_pcommit )
> > +v->arch.hvm_vmx.secondary_exec_control &= ~SECONDARY_EXEC_PCOMMIT;
> 
> Why is this conditional? Instead of the if() there should be a comment
> imo.
>
The condition is not necessary. I'll remove it and add a comment in
the next version.

Haozhong

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v3 59/62] xen/arm: Add a hypercall for device mmio mapping

2016-01-07 Thread Shannon Zhao


On 2016/1/8 5:40, Daniel De Graaf wrote:
> On 01/07/2016 05:50 AM, Jan Beulich wrote:
> On 07.01.16 at 10:11,  wrote:
>>> Hi Jan,
>>>
>>> On 2016/1/7 15:45, Jan Beulich wrote:
>>> On 07.01.16 at 07:58,  wrote:
>> On 2015/11/17 19:04, Jan Beulich wrote:
>> On 17.11.15 at 10:40,  wrote:
 --- a/xen/arch/arm/mm.c
 +++ b/xen/arch/arm/mm.c
 @@ -1138,6 +1138,10 @@ int xenmem_add_to_physmap_one(
   rcu_unlock_domain(od);
   break;
   }
 +case XENMAPSPACE_dev_mmio:
 +rc = map_dev_mmio_region(d, gpfn, 1, idx);
 +return rc;
 +break;
 Blindly for any kind of domain? The XSM check in the
 XENMEM_add_to_physmap_batch handler (in common code) doesn't
 even know which map space is to be used...
>>
>> Sorry, I know little about XSM. Could you suggest me how to add the
>> check for this new type here?
 I'm sorry to push back here, but did you at least try to derive
 what is wanted from the multitude of other XSM checks present
 throughout the tree?
>>>
>>> IIUC, you mean that it doean't need to change the XSM check itself, but
>>> we should check if the current->domain is hardware domain and it maps
>>> the space to itself before the XSM check, right?
>>
>> No, I actually think that you need to add a new, secondary XSM
>> check. But you may want to consult with Daniel (who so far wasn't
>> even Cc-ed).
> 
> Looking at the original patch, I am not sure if I understand the
> checks: it seems like the iomem_access_permitted check is being done
> on the guest's page range instead of the actual IO memory, which
> ends up allowing the guest to map anything as long as it maps it in
> the right guest area. 
Yeah, since it's hard to know the MMIO range from the DSDT in XEN, we
permit full mmio capabilities for Dom0 and deny mmio access for some
devices e.g. uart. Then when Dom0 add those devices, call
XENMEM_add_to_physmap to map their MMIO ranges. This looks similar with
what x86 does.

/* The hardware domain is initially permitted full I/O capabilities. */
rc |= ioports_permit_access(d, 0, 0x);
rc |= iomem_permit_access(d, 0UL, ~0UL);
rc |= irqs_permit_access(d, 1, nr_irqs_gsi - 1);

> The iomem_permit_access call there also seems
> to be redundant because it is the same range that was just checked.
>
Ah, I'll drop this at next version.

> If the [start_gfn, start_gfn + nr) memory range actually describes
> the physical addresses, then this operation is taking advantage of
> the existing XSM checks on XEN_DOMCTL_iomem_permission, and the
> only XSM check that is needed would be that current->domain has
> permission to modify (d)'s mappings - and this is done by the
> xsm_add_to_physmap check in XENMEM_add_to_physmap.

Thanks,
-- 
Shannon


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v3 3/3] libxl: info: Display build_id of the hypervisor.

2016-01-07 Thread Konrad Rzeszutek Wilk
If the hypervisor is built with we will display it.

Signed-off-by: Konrad Rzeszutek Wilk 
---
v2: Include HAVE_*, use libxl_zalloc, s/rc/ret/
---
 tools/libxl/libxl.c | 24 
 tools/libxl/libxl.h |  5 +
 tools/libxl/libxl_types.idl |  1 +
 tools/libxl/xl_cmdimpl.c|  1 +
 4 files changed, 31 insertions(+)

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 9207621..b894c1f 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -5263,6 +5263,7 @@ libxl_numainfo *libxl_get_numainfo(libxl_ctx *ctx, int 
*nr)
 
 const libxl_version_info* libxl_get_version_info(libxl_ctx *ctx)
 {
+GC_INIT(ctx);
 union {
 xen_extraversion_t xen_extra;
 xen_compile_info_t xen_cc;
@@ -5270,8 +5271,10 @@ const libxl_version_info* 
libxl_get_version_info(libxl_ctx *ctx)
 xen_capabilities_info_t xen_caps;
 xen_platform_parameters_t p_parms;
 xen_commandline_t xen_commandline;
+xen_build_id_t build_id;
 } u;
 long xen_version;
+int ret;
 libxl_version_info *info = &ctx->version_info;
 
 if (info->xen_version_extra != NULL)
@@ -5304,6 +5307,27 @@ const libxl_version_info* 
libxl_get_version_info(libxl_ctx *ctx)
 xc_version(ctx->xch, XENVER_commandline, &u.xen_commandline);
 info->commandline = strdup(u.xen_commandline);
 
+u.build_id.len = sizeof(u) - sizeof(u.build_id);
+ret = xc_version(ctx->xch, XENVER_build_id, &u.build_id);
+switch ( ret ) {
+case -EPERM:
+case -ENODATA:
+case 0:
+info->build_id = strdup("");
+break;
+default:
+if (ret > 0) {
+unsigned int i;
+
+info->build_id = libxl__zalloc(NOGC, (ret * 2) + 1);
+
+for (i = 0; i < ret ; i++)
+snprintf(&info->build_id[i * 2], 3, "%02hhx", 
u.build_id.buf[i]);
+} else
+LOGEV(ERROR, ret, "getting build_id");
+break;
+}
+GC_FREE;
 return info;
 }
 
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 05606a7..b91d906 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -218,6 +218,11 @@
 #define LIBXL_HAVE_SOFT_RESET 1
 
 /*
+ * LIBXL_HAVE_BUILD_ID means that libxl_version_info has the extra
+ * field for the hypervisor build_id.
+ */
+#define LIBXL_HAVE_BUILD_ID 1
+/*
  * libxl ABI compatibility
  *
  * The only guarantee which libxl makes regarding ABI compatibility
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 9658356..6c29885 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -355,6 +355,7 @@ libxl_version_info = Struct("version_info", [
 ("virt_start",uint64),
 ("pagesize",  integer),
 ("commandline",   string),
+("build_id",  string),
 ], dir=DIR_OUT)
 
 libxl_domain_create_info = Struct("domain_create_info",[
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index f9933cb..1fbe7f3 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -5549,6 +5549,7 @@ static void output_xeninfo(void)
 printf("cc_compile_by  : %s\n", info->compile_by);
 printf("cc_compile_domain  : %s\n", info->compile_domain);
 printf("cc_compile_date: %s\n", info->compile_date);
+printf("build_id   : %s\n", info->build_id);
 
 return;
 }
-- 
2.1.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v3 1/3] xsm/xen_version: Add XSM for the xen_version hypercall (v6).

2016-01-07 Thread Konrad Rzeszutek Wilk
All of XENVER_* have now an XSM check.

The subop for XENVER_commandline is now a priviliged operation.
To not break guests we still return an string - but it is
just '\0'.

The rest: XENVER_[version|extraversion|capabilities|
parameters|get_features|page_size|guest_handle|changeset|
compile_info] behave as before - allowed by default for all guests.

This is with the XSM default (and non-default) policy and with
the dummy ones.

Signed-off-by: Konrad Rzeszutek Wilk 
---
v2: Do XSM check for all the XENVER_ ops.
v3: Add empty data conditions.
v4: Return  for priv subops.
v5: Move extraversion from priv to normal. Drop the XSM check
for the non-priv subops.
v6: Add +1 for strlen(xen_deny()) to include NULL. Move changeset,
compile_info to non-priv subops.
---
 tools/flask/policy/policy/modules/xen/xen.te |  4 
 xen/common/kernel.c  | 13 +++--
 xen/common/version.c |  5 +
 xen/include/xen/version.h|  1 +
 xen/include/xsm/dummy.h  | 21 +
 xen/include/xsm/xsm.h|  5 +
 xen/xsm/dummy.c  |  1 +
 xen/xsm/flask/hooks.c| 24 
 xen/xsm/flask/policy/access_vectors  |  2 ++
 9 files changed, 74 insertions(+), 2 deletions(-)

diff --git a/tools/flask/policy/policy/modules/xen/xen.te 
b/tools/flask/policy/policy/modules/xen/xen.te
index d35ae22..17f304e 100644
--- a/tools/flask/policy/policy/modules/xen/xen.te
+++ b/tools/flask/policy/policy/modules/xen/xen.te
@@ -73,6 +73,10 @@ allow dom0_t xen_t:xen2 {
 pmu_ctrl
 get_symbol
 };
+
+# Allow dom0 to use XENVER_commandline
+allow dom0_t xen_t:xen2 version_priv;
+
 allow dom0_t xen_t:mmu memorymap;
 
 # Allow dom0 to use these domctls on itself. For domctls acting on other
diff --git a/xen/common/kernel.c b/xen/common/kernel.c
index 6a3196a..2b3ccc4 100644
--- a/xen/common/kernel.c
+++ b/xen/common/kernel.c
@@ -13,6 +13,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -226,9 +227,10 @@ void __init do_initcalls(void)
 /*
  * Simple hypercalls.
  */
-
 DO(xen_version)(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
 {
+bool_t deny = !!xsm_version_op(XSM_OTHER, cmd);
+
 switch ( cmd )
 {
 case XENVER_version:
@@ -354,10 +356,17 @@ DO(xen_version)(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
 return 0;
 
 case XENVER_commandline:
-if ( copy_to_guest(arg, saved_cmdline, ARRAY_SIZE(saved_cmdline)) )
+{
+size_t len = ARRAY_SIZE(saved_cmdline);
+
+if ( deny )
+len = strlen(xen_deny()) + 1;
+
+if ( copy_to_guest(arg, deny ? xen_deny() : saved_cmdline, len) )
 return -EFAULT;
 return 0;
 }
+}
 
 return -ENOSYS;
 }
diff --git a/xen/common/version.c b/xen/common/version.c
index b152e27..95332a0 100644
--- a/xen/common/version.c
+++ b/xen/common/version.c
@@ -55,3 +55,8 @@ const char *xen_banner(void)
 {
 return XEN_BANNER;
 }
+
+const char *xen_deny(void)
+{
+return "\0";
+}
diff --git a/xen/include/xen/version.h b/xen/include/xen/version.h
index 81a3c7d..2015c0b 100644
--- a/xen/include/xen/version.h
+++ b/xen/include/xen/version.h
@@ -12,5 +12,6 @@ unsigned int xen_minor_version(void);
 const char *xen_extra_version(void);
 const char *xen_changeset(void);
 const char *xen_banner(void);
+const char *xen_deny(void);
 
 #endif /* __XEN_VERSION_H__ */
diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index 55b84f0..3f3614e 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -721,3 +721,24 @@ static XSM_INLINE int xsm_pmu_op (XSM_DEFAULT_ARG struct 
domain *d, unsigned int
 }
 
 #endif /* CONFIG_X86 */
+
+#include 
+static XSM_INLINE int xsm_version_op (XSM_DEFAULT_ARG uint32_t op)
+{
+XSM_ASSERT_ACTION(XSM_OTHER);
+switch ( op )
+{
+case XENVER_version:
+case XENVER_extraversion:
+case XENVER_compile_info:
+case XENVER_capabilities:
+case XENVER_changeset:
+case XENVER_platform_parameters:
+case XENVER_get_features:
+case XENVER_pagesize:
+case XENVER_guest_handle:
+return 0; /* These MUST always be accessible to any guest. */
+default:
+return xsm_default_action(XSM_PRIV, current->domain, NULL);
+}
+}
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 2c365cd..64f1fa3 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -192,6 +192,7 @@ struct xsm_operations {
 int (*ioport_mapping) (struct domain *d, uint32_t s, uint32_t e, uint8_t 
allow);
 int (*pmu_op) (struct domain *d, unsigned int op);
 #endif
+int (*version_op) (uint32_t cmd);
 };
 
 #ifdef CONFIG_XSM
@@ -730,6 +731,10 @@ static inline int xsm_pmu_op (xsm_default_t def, struct 
domain *d, unsigned int
 
 #endif /* CONFIG_X86 */
 
+static inline int xsm_version_op (xsm_default_t def, uint32_t op)
+{
+

[Xen-devel] [PATCH v3 2/3] XENVER_build_id: Provide ld-embedded build-ids (v8)

2016-01-07 Thread Konrad Rzeszutek Wilk
The mechanism to get this is via the XENVER hypercall and
we add a new sub-command to retrieve the binary build-id
called XENVER_build_id. The sub-hypercall parameter
allows an arbitrary size (the buffer and len is provided
to the hypervisor). A NULL parameter will probe the hypervisor
for the length of the build-id.

One can also retrieve the value of the build-id by doing
'readelf -h xen-syms'.

For EFI builds we re-use the same build-id that the xen-syms
was built with.

Note that there are no changes to the XSM files (dummy.c
and hooks.c) as the privileged subops fall in the default case.

The version of ld that first implemented --build-id is v2.18.
Hence we check for that or later version - if older version
found we do not build the hypervisor with the build-id
(and the return code is -ENODATA for that case).

Suggested-by: Andrew Cooper 
Signed-off-by: Martin Pohlack 
Signed-off-by: Konrad Rzeszutek Wilk 
---
v1: Rebase it on Martin's initial patch
v2: Move it to XENVER hypercall
v3: Fix EFI building (Ross's fix)
v4: Don't use the third argument for length.
v5: Use new structure for XENVER_build_id with variable buf.
v6: Include Ross's fix.
v7: Include detection of bin-utils for build-id support, add
probing for size, and return -EPERM for XSM denied calls.
v8: Build xen_build_id under ARM, required adding ELFSIZE in proper file.
---
 Config.mk   | 11 ++
 tools/libxc/xc_private.c|  7 +++
 tools/libxc/xc_private.h| 10 +
 xen/arch/arm/Makefile   |  6 +++---
 xen/arch/arm/xen.lds.S  |  6 ++
 xen/arch/x86/Makefile   | 16 +-
 xen/arch/x86/xen.lds.S  |  6 ++
 xen/common/kernel.c | 35 +++
 xen/common/version.c| 42 +
 xen/include/asm-arm/config.h|  2 ++
 xen/include/public/version.h| 16 +-
 xen/include/xen/version.h   |  1 +
 xen/xsm/flask/policy/access_vectors |  4 ++--
 13 files changed, 151 insertions(+), 11 deletions(-)

diff --git a/Config.mk b/Config.mk
index 62f8209..80d6972 100644
--- a/Config.mk
+++ b/Config.mk
@@ -126,6 +126,17 @@ endef
 check-$(gcc) = $(call cc-ver-check,CC,0x040100,"Xen requires at least gcc-4.1")
 $(eval $(check-y))
 
+ld-ver = $(shell if [ $$((`$(1) --version | head -1 | sed 's/[^0-9]/ /g' | awk 
\
+   '{ printf "0x%02x%02x", $$1, $$2}'`)) -ge $$(($(2))) ]; \
+   then echo y; else echo n; fi ;)
+
+# binutils 2.18 implement build-id.
+ifeq ($(call ld-ver,$(LD),0x0212),n)
+build_id :=
+else
+build_id := --build-id=sha1
+endif
+
 # as-insn: Check whether assembler supports an instruction.
 # Usage: cflags-y += $(call as-insn "insn",option-yes,option-no)
 as-insn = $(if $(shell echo 'void _(void) { asm volatile ( $(2) ); }' \
diff --git a/tools/libxc/xc_private.c b/tools/libxc/xc_private.c
index 6c0c0d6..876e0df 100644
--- a/tools/libxc/xc_private.c
+++ b/tools/libxc/xc_private.c
@@ -712,6 +712,13 @@ int xc_version(xc_interface *xch, int cmd, void *arg)
 case XENVER_commandline:
 sz = sizeof(xen_commandline_t);
 break;
+case XENVER_build_id:
+{
+xen_build_id_t *build_id = (xen_build_id_t *)arg;
+sz = sizeof(*build_id) + build_id->len;
+HYPERCALL_BOUNCE_SET_DIR(arg, XC_HYPERCALL_BUFFER_BOUNCE_BOTH);
+break;
+}
 default:
 ERROR("xc_version: unknown command %d\n", cmd);
 return -EINVAL;
diff --git a/tools/libxc/xc_private.h b/tools/libxc/xc_private.h
index f603c15..b1b1432 100644
--- a/tools/libxc/xc_private.h
+++ b/tools/libxc/xc_private.h
@@ -202,6 +202,16 @@ enum {
 #define DECLARE_HYPERCALL_BOUNCE(_ubuf, _sz, _dir) 
DECLARE_NAMED_HYPERCALL_BOUNCE(_ubuf, _ubuf, _sz, _dir)
 
 /*
+ * Change the direction.
+ *
+ * Can only be used if the bounce_pre/bounce_post commands have
+ * not been used.
+ */
+#define HYPERCALL_BOUNCE_SET_DIR(_buf, _dir) do { if 
((HYPERCALL_BUFFER(_buf))->hbuf) \
+assert(1); 
   \
+   
(HYPERCALL_BUFFER(_buf))->dir = _dir;  \
+} while (0)
+/*
  * Set the size of data to bounce. Useful when the size is not known
  * when the bounce buffer is declared.
  */
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 2f050f5..fdf0800 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -79,17 +79,17 @@ $(BASEDIR)/common/symbols-dummy.o:
$(MAKE) -f $(BASEDIR)/Rules.mk -C $(BASEDIR)/common symbols-dummy.o
 
 $(TARGET)-syms: prelink.o xen.lds $(BASEDIR)/common/symbols-dummy.o
-   $(LD) $(LDFLAGS) -T xen.lds -N prelink.o \
+   $(LD) $(LDFLAGS) -T xen.lds -N prelink.o $(build_id) \
$(BASEDIR)/common/symbols-dummy.o -o $(@D)/.$(@F).0
$(NM) -pa 

[Xen-devel] [PATCH v3] Add build-id to XENVER hypercall.

2016-01-07 Thread Konrad Rzeszutek Wilk
Since v2 (http://lists.xen.org/archives/html/xen-devel/2015-11/msg00630.html)
 - Made it work under ARM (build_id is part of the hypervisor).
 - Only made two sub-ops be 'priv' - commandline and build_id.
 - Applied all review comments.
 - Autodetect whether ld is able to use --build-id

v1 (http://lists.xen.org/archives/html/xen-devel/2015-10/msg01090.html)
 - Made it work on EFI
 - Compiles on ARM
 - Redid it per comments.

Attached are the three patches that will add XENVER_build_id and
add the proper bits in libxl/libxc.

However they also change the behavior of the existing hypercall
for XENVER_commandline - to return the string '' for non-priv
guests. This is with XSM enabled or disabled.

The new sub-op - XENVER_build_id on the other hand will return
-EPERM (XSM or not) for !priv guests.

Please take a look and provide your feedback at your leisure.

The patches are also at my git tree:

git://xenbits.xen.org/people/konradwilk/xen.git xenver.v3


 Config.mk| 11 +++
 tools/flask/policy/policy/modules/xen/xen.te |  4 +++
 tools/libxc/xc_private.c |  7 
 tools/libxc/xc_private.h | 10 ++
 tools/libxl/libxl.c  | 24 ++
 tools/libxl/libxl.h  |  5 +++
 tools/libxl/libxl_types.idl  |  1 +
 tools/libxl/xl_cmdimpl.c |  1 +
 xen/arch/arm/Makefile|  6 ++--
 xen/arch/arm/xen.lds.S   |  6 
 xen/arch/x86/Makefile| 16 +++---
 xen/arch/x86/xen.lds.S   |  6 
 xen/common/kernel.c  | 48 ++--
 xen/common/version.c | 47 +++
 xen/include/asm-arm/config.h |  2 ++
 xen/include/public/version.h | 16 +-
 xen/include/xen/version.h|  2 ++
 xen/include/xsm/dummy.h  | 21 
 xen/include/xsm/xsm.h|  5 +++
 xen/xsm/dummy.c  |  1 +
 xen/xsm/flask/hooks.c| 24 ++
 xen/xsm/flask/policy/access_vectors  |  2 ++
 22 files changed, 254 insertions(+), 11 deletions(-)


Konrad Rzeszutek Wilk (3):
  xsm/xen_version: Add XSM for the xen_version hypercall (v6).
  XENVER_build_id: Provide ld-embedded build-ids (v8)
  libxl: info: Display build_id of the hypervisor.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [seabios baseline-only test] 38601: tolerable FAIL

2016-01-07 Thread Platform Team regression test user
This run is configured for baseline tests only.

flight 38601 seabios real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/38601/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-qemuu-nested-intel 13 xen-boot/l1 fail like 38597
 test-amd64-amd64-xl-qemuu-winxpsp3  9 windows-install  fail like 38597

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail never pass

version targeted for testing:
 seabios  16a9e7926a6c6845a98df2a6eac509c23c6206ba
baseline version:
 seabios  71479612401b794e6cb5025f61e5352c51f35877

Last test of basis38597  2016-01-06 17:24:58 Z1 days
Testing same since38601  2016-01-07 20:56:04 Z0 days1 attempts


People who touched revisions under test:
  Kevin O'Connor 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsmpass
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm pass
 test-amd64-amd64-qemuu-nested-amdfail
 test-amd64-i386-qemuu-rhel6hvm-amd   pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass
 test-amd64-amd64-xl-qemuu-win7-amd64 fail
 test-amd64-i386-xl-qemuu-win7-amd64  fail
 test-amd64-amd64-qemuu-nested-intel  fail
 test-amd64-i386-qemuu-rhel6hvm-intel pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 pass
 test-amd64-amd64-xl-qemuu-winxpsp3   fail
 test-amd64-i386-xl-qemuu-winxpsp3pass



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


commit 16a9e7926a6c6845a98df2a6eac509c23c6206ba
Author: Kevin O'Connor 
Date:   Tue Dec 29 23:04:15 2015 -0500

tpm: Don't use 16bit BIOS return codes in TPM menu functions

Don't use the return codes from the 16bit BIOS spec in the internal
menu functions.  Only the 16bit BIOS interface code should need to
handle the details of that spec.  For functions that need to return
the TIS command status, return those codes directly instead of via a
pointer parameter.

Signed-off-by: Kevin O'Connor 

commit b8631eaeb3b2216371de028c76abf17186ac84ab
Author: Kevin O'Connor 
Date:   Wed Dec 30 12:51:27 2015 -0500

tpm: Don't use 16bit BIOS return codes in tpmhw_* functions

Don't use the return codes from the 16bit BIOS spec in the internal
tpmhw functions.  Only the 16bit BIOS interface code should need to
handle the details of that spec.

Signed-off-by: Kevin O'Connor 

commit 9ddea3b018fcb2e0d8d49a7e6c3c36763d4e93e0
Author: Kevin O'Connor 
Date:   Wed Dec 30 12:40:11 2015 -0500

tpm: Don't use 16bit BIOS return codes in tpm_log_event()

Don't use the return codes from the 16bit BIOS spec in the internal
tpm_log_event() and tpm_log_extend_event() functions.  Only the 16bit
BIOS interface code should need to handle the details of that spec.

Signed-off-by: Kevin O'Connor 

commit cac29f2504450a97e05a95fa6e76cdb9199eb879
Author: Kevin O'Connor 
Date:   Tue Dec 2

[Xen-devel] [PATCH] arch/x86: convert bigmem to Kconfig

2016-01-07 Thread Doug Goldstein
Convert the bigmem build option to Kconfig.

Signed-off-by: Doug Goldstein 
---
The test:

$ grep CONFIG_BIGMEM .config
# CONFIG_BIGMEM is not set
$ pahole xen-syms -C page_info

/* size: 32, cachelines: 1, members: 5 */



$ grep CONFIG_BIGMEM .config
CONFIG_BIGMEM=y
$ pahole xen-syms -C page_info

/* size: 48, cachelines: 1, members: 5 */
---
 xen/arch/x86/Kconfig  | 12 
 xen/arch/x86/Rules.mk |  2 --
 2 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 7d2ed96..90b4216 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -25,6 +25,18 @@ config ARCH_DEFCONFIG
 
 menu "Architecture Features"
 
+config BIGMEM
+   bool "big memory support"
+   default n
+   ---help---
+ Allows Xen to support up to 123Tb of memory.
+
+ This requires growing struct page_info from 32 to 48 bytes as well
+ as shrinking the always accessible direct mapped memory range from
+ 5Tb to 3.5Tb.
+
+ If unsure, say N.
+
 endmenu
 
 source "common/Kconfig"
diff --git a/xen/arch/x86/Rules.mk b/xen/arch/x86/Rules.mk
index b76a754..a108d24 100644
--- a/xen/arch/x86/Rules.mk
+++ b/xen/arch/x86/Rules.mk
@@ -23,7 +23,6 @@ $(call as-insn-check,CFLAGS,CC,".equ \"x\"$$(comma)1", \
  '-D__OBJECT_LABEL__=$(subst $(BASEDIR)/,,$(CURDIR))/$$@')
 
 shadow-paging ?= y
-bigmem?= n
 
 CFLAGS += -mno-red-zone -mno-sse -fpic
 CFLAGS += -fno-asynchronous-unwind-tables
@@ -33,4 +32,3 @@ CFLAGS += -DGCC_HAS_VISIBILITY_ATTRIBUTE
 endif
 
 CFLAGS-$(shadow-paging) += -DCONFIG_SHADOW_PAGING
-CFLAGS-$(bigmem)+= -DCONFIG_BIGMEM
-- 
2.4.10


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [qemu-upstream-4.4-testing test] 77292: regressions - FAIL

2016-01-07 Thread osstest service owner
flight 77292 qemu-upstream-4.4-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/77292/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-ovmf-amd64 9 debian-hvm-install fail REGR. vs. 62702
 test-amd64-i386-xl-qemuu-ovmf-amd64  9 debian-hvm-install fail REGR. vs. 62702

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 15 guest-localmigrate/x10 fail pass 
in 77190

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail like 62702

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass

version targeted for testing:
 qemuu2264b0c66075cc34c252a1386f019f8be6240890
baseline version:
 qemuu5fe74249f5ab528fe84a26fa60438a6de4c787f0

Last test of basis62702  2015-10-06 15:32:13 Z   93 days
Testing same since66539  2015-12-18 16:12:10 Z   20 days9 attempts


People who touched revisions under test:
  Stefano Stabellini 

jobs:
 build-amd64-xend pass
 build-i386-xend  pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-amd64-qemuu-nested-amdfail
 test-amd64-i386-qemuu-rhel6hvm-amd   pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
 test-amd64-i386-freebsd10-amd64  pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail
 test-amd64-amd64-xl-qemuu-win7-amd64 pass
 test-amd64-i386-xl-qemuu-win7-amd64  fail
 test-amd64-amd64-xl-credit2  pass
 test-amd64-i386-freebsd10-i386   pass
 test-amd64-amd64-qemuu-nested-intel  fail
 test-amd64-i386-qemuu-rhel6hvm-intel pass
 test-amd64-amd64-libvirt pass
 test-amd64-i386-libvirt  pass
 test-amd64-amd64-xl-multivcpupass
 test-amd64-amd64-pairpass
 test-amd64-i386-pair pass
 test-amd64-amd64-libvirt-pairpass
 test-amd64-i386-libvirt-pair pass
 test-amd64-amd64-pv  pass
 test-amd64-i386-pv   pass
 test-amd64-amd64-amd64-pvgrubpass
 test-amd64-amd64-i386-pvgrub pass
 test-amd64-amd64-pygrub  pass
 test-amd64-amd64-xl-qcow2pass
 test-amd64-i386-xl-raw   pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 fail
 test-amd64-amd64-libvirt-vhd pass
 test-amd64-amd64-xl-qemuu-winxpsp3   pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

--

Re: [Xen-devel] [PATCH v4.2] libxc: Defer initialization of start_page for HVM guests

2016-01-07 Thread Juergen Gross
On 07/01/16 23:19, Boris Ostrovsky wrote:
> With commit 8c45adec18e0 ("libxc: create unmapped initrd in domain
> builder if supported") location of ramdisk may not be available to
> HVMlite guests by the time alloc_magic_pages_hvm() is invoked if the
> guest supports unmapped initrd.
> 
> So let's move ramdisk info initialization (along with a few other
> operations that are not directly related to allocating magic/special
> pages) from alloc_magic_pages_hvm() to bootlate_hvm().
> 
> Since we now split allocation and mapping of the start_info segment
> let's stash it, along with cmdline length, in xc_dom_image so that we
> can check whether we are mapping correctly-sized range.
> 
> We can also stop using xc_dom_image.start_info_pfn and leave it for
> PV(H) guests only.
> 
> Signed-off-by: Boris Ostrovsky 
> ---
> v4:
>  * See the last two paragraphs from commit message above
> 
> v4.1:
>  * Inverted testing of start_info_size in bootlate_hvm().
> 
> v4.2
>  *  Actually do what I said I'd do in 4.1
> 
>  tools/libxc/include/xc_dom.h |2 +
>  tools/libxc/xc_dom_x86.c |  140 +++--
>  2 files changed, 80 insertions(+), 62 deletions(-)
> 
> diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
> index 2460818..cac4698 100644
> --- a/tools/libxc/include/xc_dom.h
> +++ b/tools/libxc/include/xc_dom.h
> @@ -71,6 +71,7 @@ struct xc_dom_image {
>  
>  /* arguments and parameters */
>  char *cmdline;
> +size_t cmdline_size;
>  uint32_t f_requested[XENFEAT_NR_SUBMAPS];
>  
>  /* info from (elf) kernel image */
> @@ -91,6 +92,7 @@ struct xc_dom_image {
>  struct xc_dom_seg p2m_seg;
>  struct xc_dom_seg pgtables_seg;
>  struct xc_dom_seg devicetree_seg;
> +struct xc_dom_seg start_info_seg; /* HVMlite only */

Instead of adding HVM specific members here, you could make use of
dom.arch_private and use just a local structure defined in xc_dom_x86.c.


Juergen

>  xen_pfn_t start_info_pfn;
>  xen_pfn_t console_pfn;
>  xen_pfn_t xenstore_pfn;
> diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
> index b8d2904..edf2db1 100644
> --- a/tools/libxc/xc_dom_x86.c
> +++ b/tools/libxc/xc_dom_x86.c
> @@ -586,23 +586,12 @@ static void build_hvm_info(void *hvm_info_page, struct 
> xc_dom_image *dom)
>  static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
>  {
>  unsigned long i;
> -void *hvm_info_page;
>  uint32_t *ident_pt, domid = dom->guest_domid;
>  int rc;
>  xen_pfn_t special_array[X86_HVM_NR_SPECIAL_PAGES];
>  xen_pfn_t ioreq_server_array[NR_IOREQ_SERVER_PAGES];
>  xc_interface *xch = dom->xch;
>  
> -if ( dom->device_model )
> -{
> -if ( (hvm_info_page = xc_map_foreign_range(
> -  xch, domid, PAGE_SIZE, PROT_READ | PROT_WRITE,
> -  HVM_INFO_PFN)) == NULL )
> -goto error_out;
> -build_hvm_info(hvm_info_page, dom);
> -munmap(hvm_info_page, PAGE_SIZE);
> -}
> -
>  /* Allocate and clear special pages. */
>  for ( i = 0; i < X86_HVM_NR_SPECIAL_PAGES; i++ )
>  special_array[i] = special_pfn(i);
> @@ -636,65 +625,25 @@ static int alloc_magic_pages_hvm(struct xc_dom_image 
> *dom)
>  
>  if ( !dom->device_model )
>  {
> -struct xc_dom_seg seg;
> -struct hvm_start_info *start_info;
> -char *cmdline;
> -struct hvm_modlist_entry *modlist;
> -void *start_page;
> -size_t cmdline_size = 0;
> -size_t start_info_size = sizeof(*start_info);
> +size_t start_info_size = sizeof(struct hvm_start_info);
>  
>  if ( dom->cmdline )
>  {
> -cmdline_size = ROUNDUP(strlen(dom->cmdline) + 1, 8);
> -start_info_size += cmdline_size;
> -
> +dom->cmdline_size = ROUNDUP(strlen(dom->cmdline) + 1, 8);
> +start_info_size += dom->cmdline_size;
>  }
> +
> +/* Limited to one module. */
>  if ( dom->ramdisk_blob )
> -start_info_size += sizeof(*modlist); /* Limited to one module. */
> +start_info_size += sizeof(struct hvm_modlist_entry);
>  
> -rc = xc_dom_alloc_segment(dom, &seg, "HVMlite start info", 0,
> -  start_info_size);
> +rc = xc_dom_alloc_segment(dom, &dom->start_info_seg,
> +  "HVMlite start info", 0, start_info_size);
>  if ( rc != 0 )
>  {
>  DOMPRINTF("Unable to reserve memory for the start info");
>  goto out;
>  }
> -
> -start_page = xc_map_foreign_range(xch, domid, start_info_size,
> -  PROT_READ | PROT_WRITE,
> -  seg.pfn);
> -if ( start_page == NULL )
> -{
> -DOMPRINTF("Unable to map HVM start info page");
> -goto error_out;
> -}
> -
> -start_info = st

Re: [Xen-devel] [libvirt] [PATCH 2/2] libxl: support vif outgoing bandwidth QoS

2016-01-07 Thread Jim Fehlig
On 01/07/2016 07:48 AM, Michal Privoznik wrote:
> On 29.12.2015 02:09, Jim Fehlig wrote:
>> The libxl_device_nic structure supports specifying an outgoing rate
>> limit based on a time interval and bytes allowed per interval. In xl
>> config a rate limit is specified as "/s@". INTERVAL
>> is optional and defaults to 50ms.
>>
>> libvirt expresses outgoing limits by average (required), peak, burst,
>> and floor attributes in units of KB/s. This patch supports the outgoing
>> bandwidth limit by converting the average KB/s to bytes per interval
>> based on the same default interval (50ms) used by xl.
>>
>> Signed-off-by: Jim Fehlig 
>> ---
>>  src/libxl/libxl_conf.c | 39 +++
>>  1 file changed, 39 insertions(+)
>>
>> diff --git a/src/libxl/libxl_conf.c b/src/libxl/libxl_conf.c
>> index 23c74e7..6320421 100644
>> --- a/src/libxl/libxl_conf.c
>> +++ b/src/libxl/libxl_conf.c
>> @@ -1093,6 +1093,7 @@ libxlMakeNic(virDomainDefPtr def,
>>  {
>>  bool ioemu_nic = def->os.type == VIR_DOMAIN_OSTYPE_HVM;
>>  virDomainNetType actual_type = virDomainNetGetActualType(l_nic);
>> +virNetDevBandwidthPtr actual_bw;
>>  
>>  /* TODO: Where is mtu stored?
>>   *
>> @@ -1206,6 +1207,44 @@ libxlMakeNic(virDomainDefPtr def,
>>  #endif
>>  }
>>  
>> +/*
>> + * Set bandwidth.
>> + * From $xen-sources/docs/misc/xl-network-configuration.markdown:
>> + *
>> + *
>> + * Specifies the rate at which the outgoing traffic will be limited to.
>> + * The default if this keyword is not specified is unlimited.
>> + *
>> + * The rate may be specified as "/s" or optionally 
>> "/s@".
>> + *
>> + * `RATE` is in bytes and can accept suffixes:
>> + * GB, MB, KB, B for bytes.
>> + * Gb, Mb, Kb, b for bits.
>> + * `INTERVAL` is in microseconds and can accept suffixes: ms, us, s.
>> + * It determines the frequency at which the vif transmission credit
>> + * is replenished. The default is 50ms.
>> +
>> + * Vif rate limiting is credit-based. It means that for "1MB/s@20ms",
>> + * the available credit will be equivalent of the traffic you would have
>> + * done at "1MB/s" during 20ms. This will results in a credit of 20,000
>> + * bytes replenished every 20,000 us.
>> + *
>> + *
>> + * libvirt doesn't support the notion of rate limiting over an interval.
>> + * Similar to xl's behavior when interval is not specified, set a 
>> default
>> + * interval of 50ms and calculate the number of bytes per interval based
>> + * on the specified average bandwidth.
>> + */
>> +actual_bw = virDomainNetGetActualBandwidth(l_nic);
>> +if (actual_bw && actual_bw->out && actual_bw->out->average) {
>> +uint64_t bytes_per_sec = actual_bw->out->average * 1024;
>> +uint64_t bytes_per_interval =
>> +(((uint64_t) bytes_per_sec * 5UL) / 100UL);
>> +
>> +x_nic->rate_bytes_per_interval = bytes_per_interval;
>> +x_nic->rate_interval_usecs =  5UL;
>> +}
>> +
> Interesting. I'd expect:
>
> x_nic->rate_bytes_per_interval = bytes_per_sec;
> x_nic->rate_interval_usecs = 1000*1000;

For the most part I mimicked the Xen code and wanted to stick with the default
interval of 50ms, which has been the default for a long time. It is even
mentioned in some old RHEL5 docs

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/Virtualization/sect-Virtualization-Tips_and_tricks-Limit_network_bandwidth_for_a_Xen_guest.html

BTW, here is the Xen code that inspired this logic

http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/libxl/libxlu_vif.c;h=0665e624dc178a6ca8058e04a7baacaf1475bd37;hb=HEAD#l131

rate_bytes_per_interval is set to (bytes/s * interval us)/100us

I guess we are saying the same thing, you're just setting interval to 1s (thus
rate_bytes_per_interval == bytes_per_sec) instead of the historical 50ms :-).

>
> I mean, if I understood the xl way of rate limiting correctly, one says
> how much bytes can be sent for how long. so for 1MB/s I'd expect to send
> 1024*1024 bytes each second.
>
> Or am I missing something?

Does the above explanation make sense? I might be missing something :-).  CC'd a
few Xen tools maintainer just in case.

Regards,
Jim


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [libvirt] [PATCH 1/2] xenconfig: support parsing and formatting vif bandwidth

2016-01-07 Thread Jim Fehlig
On 01/07/2016 07:48 AM, Michal Privoznik wrote:
> On 29.12.2015 02:09, Jim Fehlig wrote:
>> Both xm and xl config have long supported specifying vif rate
>> limiting, e.g.
>>
>> vif = [ 'mac=00:16:3E:74:3d:76,bridge=br0,rate=10MB/s' ]
>>
>> Add support for mapping rate to and from  in the xenconfig
>> parser and formatter. rate is mapped to the required 'average' attribute
>> of the  element, e.g.
>>
>>   
>> ...
>> 
>>   
>> 
>>   
>>
>> Also add a unit test to check the conversion logic.
>>
>> Signed-off-by: Jim Fehlig 
>> ---
>>
>> I used a bit of code from libxlu_vif.c to implement xenParseVifRate()
>> instead of using the libxlutil lib directly, since in theory rate limiting
>> applies to the old xen driver (no libxl) as well.
>>
>>  src/xenconfig/xen_common.c   | 77 
>> 
>>  tests/xlconfigdata/test-vif-rate.cfg | 26 
>>  tests/xlconfigdata/test-vif-rate.xml | 57 ++
>>  tests/xlconfigtest.c |  1 +
>>  4 files changed, 161 insertions(+)
> ACK

Hi Michal,

Thanks a lot for taking a look at this series! Note that I posted a V2 earlier
this week which includes support for parsing/formatting vif bandwidth in sexpr
config too

https://www.redhat.com/archives/libvir-list/2016-January/msg00045.html

Unfortunately it results in some changes to this patch, so I think it would be
best to peek at the V2 series before pushing it.

Regards,
Jim


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Missing libvirt+libxl functionality (Was: Re: [Xen-users] Programmatic administration of Xen machines)

2016-01-07 Thread Jim Fehlig
On 01/04/2016 05:37 AM, Ian Campbell wrote:
> On Mon, 2016-01-04 at 13:31 +0100, Andreas Pflug wrote:
>> Am 04.01.16 um 13:13 schrieb Ian Campbell:
>>> On Mon, 2016-01-04 at 12:47 +0100, Andreas Pflug wrote:
 Am 04.01.16 um 12:36 schrieb Ian Campbell:
> Sorry to hijack this thread.
>
> On Mon, 2015-12-28 at 12:56 +0100, Andreas Pflug wrote:
>> Actually, I find virsh a bad idea since its support for the new
>> xl
>> interface isn't feature complete as the old xm interface was. I
>> had
>> to
>> realize this the hard way when my virsh based python tools didn't
>> work
>> as expected after upgrading from xm to xl.
> Note that virsh speaks to libxl directly, not via xl.
>
> Please can you list the functionality of libvirt's libxl backed
> which
> is
> missing compared to the old xend based one? 
 libvirt only handles domains created through it, i.e. using xl at the
 same time is incompatible. Since dom0 can't be created using libvirt,
 it's missing from "virsh list" in any case.
>>> This works for me with a reasonably modern set of bits:
>>>
>>> root@marilith-n0:~# virsh list
>>>  IdName   State
>>> 
>>>  0 Domain-0   running
>>>  13d  running
>>>
>>> It probably requires the correct running of xen-init-dom0 during boot
>>> (likely via the xencommons initscript).
>>>
>>> I could believe it was broken at some point in history though.
>>>
  libvirt stores its own state, not using libxl/xenstore stuff.
>>> Please can you elaborate.
>> I tested on Debian with 4.4.1 and xl toolstack. See the author's blog
>> post:
>> https://blog.xenproject.org/2014/01/17/libvirt-support-for-xens-new-
>> libxenlight-toolstack/
> I think that particular aspect of the blog post is no longer valid, at
> least AFAICT with modern libvirt.

Commit 45697fe5 added dom0 management support to the libvirt libxl driver.
libvirt >= 1.2.18 contains the commit.

>
>> I had even more trouble, e.g. I wasn't able to use non-standard block
>> scripts (neither via /etc/xen/scripts/block-XXX nor via a script
>> parameter) which are mandatory for me.
> Looking at the code it seems like this is still missing.
>
> Copying the devel list and maintainer in case I've either missed something
> or there is a reason for this (other than "still on the TODO list", which I
> expect is most likely).

It is on a TODO list, but I'm not sure how to solve it :-/. Unlike the
libxl_device_disk struct, the corresponding libvirt struct does not support a
'script' field, and I doubt it ever will. Repeated attempts to add a 

[Xen-devel] [PATCH v3 2/5] remus: resume immediately if libxl__xc_domain_save_done() completes

2016-01-07 Thread Wen Congyang
For example: if the secondary host is down, and we fail to send the data to
the secondary host. xc_domain_save() returns 0. So in the function
libxl__xc_domain_save_done(), rc is 0(the helper program exits normally),
and retval is 0(it is xc_domain_save()'s return value). In such case, we
just need to complete the stream.

Signed-off-by: Wen Congyang 
---
 tools/libxl/libxl_stream_write.c | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/tools/libxl/libxl_stream_write.c b/tools/libxl/libxl_stream_write.c
index 80d9208..82e7719 100644
--- a/tools/libxl/libxl_stream_write.c
+++ b/tools/libxl/libxl_stream_write.c
@@ -354,8 +354,17 @@ void libxl__xc_domain_save_done(libxl__egc *egc, void 
*dss_void,
  * alive, and check_all_finished() may have torn it down around us.
  * If the stream is not still alive, we must not continue any work.
  */
-if (libxl__stream_write_inuse(stream))
-write_emulator_xenstore_record(egc, stream);
+if (libxl__stream_write_inuse(stream)) {
+if (dss->remus)
+/*
+ * For remus, if libxl__xc_domain_save_done() completes,
+ * there was an error sending data to the secondary.
+ * Resume the primary ASAP.
+ */
+stream_complete(egc, stream, 0);
+else
+write_emulator_xenstore_record(egc, stream);
+}
 }
 
 static void write_emulator_xenstore_record(libxl__egc *egc,
-- 
2.5.0




___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v3 3/5] tools/libxc: don't send end record if remus fails

2016-01-07 Thread Wen Congyang
Signed-off-by: Wen Congyang 
Reviewed-by: Andrew Cooper 
---
 tools/libxc/xc_sr_save.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c
index 88d85ef..e532168 100644
--- a/tools/libxc/xc_sr_save.c
+++ b/tools/libxc/xc_sr_save.c
@@ -795,7 +795,7 @@ static int save(struct xc_sr_context *ctx, uint16_t 
guest_type)
 
 rc = ctx->save.callbacks->checkpoint(ctx->save.callbacks->data);
 if ( rc <= 0 )
-ctx->save.checkpointed = false;
+goto err;
 }
 } while ( ctx->save.checkpointed );
 
-- 
2.5.0




___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v3 0/5] migration/remus: bug fix and cleanup

2016-01-07 Thread Wen Congyang

Wen Congyang (5):
  remus: don't call stream_continue() when doing failover
  remus: resume immediately if libxl__xc_domain_save_done() completes
  tools/libxc: don't send end record if remus fails
  tools/libxc: error handling for the postcopy() callback
  tools/libxl: remove unused function libxl__domain_save_device_model()

 tools/libxc/xc_sr_save.c |  6 ++-
 tools/libxl/libxl.c  |  4 --
 tools/libxl/libxl_dom.c  | 91 
 tools/libxl/libxl_internal.h |  6 ---
 tools/libxl/libxl_stream_read.c  | 21 +++---
 tools/libxl/libxl_stream_write.c | 13 +-
 6 files changed, 31 insertions(+), 110 deletions(-)

-- 
2.5.0




___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v3 1/5] remus: don't call stream_continue() when doing failover

2016-01-07 Thread Wen Congyang
stream_continue() is used for migration to read emulator
xenstore data and emulator context. For remus, if we do
failover, we have read it in the checkpoint cycle, and
we only need to complete the stream.

Signed-off-by: Wen Congyang 
Reviewed-by: Andrew Cooper 
---
 tools/libxl/libxl_stream_read.c | 21 -
 1 file changed, 16 insertions(+), 5 deletions(-)

diff --git a/tools/libxl/libxl_stream_read.c b/tools/libxl/libxl_stream_read.c
index 258dec4..65219d5 100644
--- a/tools/libxl/libxl_stream_read.c
+++ b/tools/libxl/libxl_stream_read.c
@@ -758,6 +758,9 @@ void libxl__xc_domain_restore_done(libxl__egc *egc, void 
*dcs_void,
 libxl__stream_read_state *stream = &dcs->srs;
 STATE_AO_GC(dcs->ao);
 
+/* convenience aliases */
+const int checkpointed_stream = dcs->restore_params.checkpointed_stream;
+
 if (rc)
 goto err;
 
@@ -777,11 +780,19 @@ void libxl__xc_domain_restore_done(libxl__egc *egc, void 
*dcs_void,
  * If the stream is not still alive, we must not continue any work.
  */
 if (libxl__stream_read_inuse(stream)) {
-/*
- * Libxc has indicated that it is done with the stream.  Resume reading
- * libxl records from it.
- */
-stream_continue(egc, stream);
+if (checkpointed_stream) {
+/*
+ * Failover from primary. Domain state is currently at a
+ * consistent checkpoint, ready to go.
+ */
+stream_complete(egc, stream, 0);
+} else {
+/*
+ * Libxc has indicated that it is done with the stream.
+ * Resume reading libxl records from it.
+ */
+stream_continue(egc, stream);
+}
 }
 }
 
-- 
2.5.0




___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v3 4/5] tools/libxc: error handling for the postcopy() callback

2016-01-07 Thread Wen Congyang
Signed-off-by: Wen Congyang 
Reviewed-by: Andrew Cooper 
---
 tools/libxc/xc_sr_save.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c
index e532168..e4ba560 100644
--- a/tools/libxc/xc_sr_save.c
+++ b/tools/libxc/xc_sr_save.c
@@ -791,7 +791,9 @@ static int save(struct xc_sr_context *ctx, uint16_t 
guest_type)
 if ( rc )
 goto err;
 
-ctx->save.callbacks->postcopy(ctx->save.callbacks->data);
+rc = ctx->save.callbacks->postcopy(ctx->save.callbacks->data);
+if ( rc <= 0 )
+goto err;
 
 rc = ctx->save.callbacks->checkpoint(ctx->save.callbacks->data);
 if ( rc <= 0 )
-- 
2.5.0




___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v3 5/5] tools/libxl: remove unused function libxl__domain_save_device_model()

2016-01-07 Thread Wen Congyang
After the commit d77570e7, libxl__domain_save_device_model() can
be dropped.

Signed-off-by: Wen Congyang 
---
 tools/libxl/libxl.c  |  4 --
 tools/libxl/libxl_dom.c  | 91 
 tools/libxl/libxl_internal.h |  6 ---
 3 files changed, 101 deletions(-)

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 9207621..c286881 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -1551,10 +1551,6 @@ static void stubdom_destroy_callback(libxl__egc *egc,
 dds->stubdom_finished = 1;
 savefile = libxl__device_model_savefile(gc, dis->domid);
 rc = libxl__remove_file(gc, savefile);
-/*
- * On suspend libxl__domain_save_device_model will have already
- * unlinked the save file.
- */
 if (rc) {
 LOG(ERROR, "failed to remove device-model savefile %s", savefile);
 }
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index 47971a9..2269998 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -1785,97 +1785,6 @@ static void stream_done(libxl__egc *egc,
 domain_save_done(egc, sws->dss, rc);
 }
 
-static void save_device_model_datacopier_done(libxl__egc *egc,
- libxl__datacopier_state *dc, int rc, int onwrite, int errnoval);
-
-void libxl__domain_save_device_model(libxl__egc *egc,
- libxl__domain_suspend_state *dss,
- libxl__save_device_model_cb *callback)
-{
-STATE_AO_GC(dss->ao);
-struct stat st;
-uint32_t qemu_state_len;
-int rc;
-
-dss->save_dm_callback = callback;
-
-/* Convenience aliases */
-const char *const filename = dss->dm_savefile;
-const int fd = dss->fd;
-
-libxl__datacopier_state *dc = &dss->save_dm_datacopier;
-memset(dc, 0, sizeof(*dc));
-dc->readwhat = GCSPRINTF("qemu save file %s", filename);
-dc->ao = ao;
-dc->readfd = -1;
-dc->writefd = fd;
-dc->maxsz = INT_MAX;
-dc->bytes_to_read = -1;
-dc->copywhat = GCSPRINTF("qemu save file for domain %"PRIu32, dss->domid);
-dc->writewhat = "save/migration stream";
-dc->callback = save_device_model_datacopier_done;
-
-dc->readfd = open(filename, O_RDONLY);
-if (dc->readfd < 0) {
-LOGE(ERROR, "unable to open %s", dc->readwhat);
-rc = ERROR_FAIL;
-goto out;
-}
-
-if (fstat(dc->readfd, &st))
-{
-LOGE(ERROR, "unable to fstat %s", dc->readwhat);
-rc = ERROR_FAIL;
-goto out;
-}
-
-if (!S_ISREG(st.st_mode)) {
-LOG(ERROR, "%s is not a plain file!", dc->readwhat);
-rc = ERROR_FAIL;
-goto out;
-}
-
-qemu_state_len = st.st_size;
-LOG(DEBUG, "%s is %d bytes", dc->readwhat, qemu_state_len);
-
-rc = libxl__datacopier_start(dc);
-if (rc) goto out;
-
-libxl__datacopier_prefixdata(egc, dc,
- QEMU_SIGNATURE, strlen(QEMU_SIGNATURE));
-
-libxl__datacopier_prefixdata(egc, dc,
- &qemu_state_len, sizeof(qemu_state_len));
-return;
-
- out:
-save_device_model_datacopier_done(egc, dc, rc, -1, EIO);
-}
-
-static void save_device_model_datacopier_done(libxl__egc *egc,
- libxl__datacopier_state *dc, int our_rc, int onwrite, int errnoval)
-{
-libxl__domain_suspend_state *dss =
-CONTAINER_OF(dc, *dss, save_dm_datacopier);
-STATE_AO_GC(dss->ao);
-
-/* Convenience aliases */
-const char *const filename = dss->dm_savefile;
-int rc;
-
-libxl__datacopier_kill(dc);
-
-if (dc->readfd >= 0) {
-close(dc->readfd);
-dc->readfd = -1;
-}
-
-rc = libxl__remove_file(gc, filename);
-if (!our_rc) our_rc = rc;
-
-dss->save_dm_callback(egc, dss, our_rc);
-}
-
 static void libxl__remus_teardown(libxl__egc *egc,
   libxl__domain_suspend_state *dss,
   int rc);
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index a556a38..233d44a 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -3103,9 +3103,6 @@ struct libxl__domain_suspend_state {
 libxl__logdirty_switch logdirty;
 void (*callback_common_done)(libxl__egc*,
  struct libxl__domain_suspend_state*, int ok);
-/* private for libxl__domain_save_device_model */
-libxl__save_device_model_cb *save_dm_callback;
-libxl__datacopier_state save_dm_datacopier;
 };
 
 
@@ -3498,9 +3495,6 @@ static inline bool libxl__save_helper_inuse(const 
libxl__save_helper_state *shs)
 /* Each time the dm needs to be saved, we must call suspend and then save */
 _hidden int libxl__domain_suspend_device_model(libxl__gc *gc,
libxl__domain_suspend_state *dss);
-_hidden void libxl__domain_save_device_model(libxl__egc *egc,
- libxl__domain_suspend_state *dss,
- 

Re: [Xen-devel] Missing libvirt+libxl functionality (Was: Re: [Xen-users] Programmatic administration of Xen machines)

2016-01-07 Thread Olaf Hering
On Thu, Jan 07, Jim Fehlig wrote:

> The 'iscsi:iqn.2006-09.de.suse@0ac47ee2-216e-452a-a341-a12624cd0225' blob was
> passed wholesale to xend, which would eat it and "do the right thing". AFAIK,
> libxl is not that forgiving. I've cc'd Olaf on this thread since we recently
> discussed how libvirt+libxl might support external block scripts. As I recall,
> we didn't have an novel ideas.

I think the "raw" string is known to libvirts libxl driver. It could do
a strncmp() and fill also ->script. Then libxl would call that thing by
itself, libvirt does not need to know about such internals. Of course
that would mean an additional "script=$path" is still not supported  by
libvirt. But thats not a loss because whoever is relying on can just
copy its own file into XEN_SCRIPT_DIR. After all only our own dmmd thing
requires a script because that is its whole purpose (carry the block
configuration along with the vm configuration).

Olaf

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


  1   2   >