Re: [Xen-devel] One question about the hypercall to translate gfn to mfn.

2014-12-15 Thread Jan Beulich
 On 15.12.14 at 07:25, kevin.t...@intel.com wrote:
  From: Jan Beulich [mailto:jbeul...@suse.com]
  On 12.12.14 at 08:24, kevin.t...@intel.com wrote:
  - how is BFN or unused address (what do you mean by address here?)
  allocated? does it need present in guest physical memory at boot time,
  or just finding some holes?
 
 Fitting this into holes should be fine.
 
 this is an interesting open to be further discussed. Here we need consider 
 the extreme case, i.e. a 64bit GPU page table can legitimately use up all 
 the system memory allocates to that VM, and considering dozens of VMs, 
 it means we need reserve a very large hole. 

Oh, it's guest RAM you want mapped, not frame buffer space. But still
you're never going to have to map more than the total amount of host
RAM, and (with Linux) we already assume everything can be mapped
through the 1:1 mapping. I.e. the only collision would be with excessive
PFN reservations for ballooning purposes.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Cache Allocation Technology(CAT) design for XEN

2014-12-15 Thread Chao Peng
On Fri, Dec 12, 2014 at 03:02:33PM +, Andrew Cooper wrote:
 On 12/12/14 12:27, Chao Peng wrote:
  Hi all, we plan to bring Intel CAT into XEN. This is the initial
  design for that. Comments/suggestions are welcome.
 
 
 Fantastic - this is a very clear and well presented document.  In terms
 of a plan of action, it looks fine.
Thank you for review.
 
 From my understanding, CAT is a largely orthogonal to CMT, but will
 share some of the base PSR infrastructure in Xen?
 
Yes, from functional perspective, they are independent features. But
they do share some common codes and also have similar designs in XEN.

Chao
 
 
 ___
 Xen-devel mailing list
 Xen-devel@lists.xen.org
 http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH] x86/AMD-ucode: correct multiple container handling

2014-12-15 Thread Jan Beulich
Avoid emitting an error message referring to an incorrect or corrupt
container file just because no entry was found for the running CPU.

Additionally switch the order of data validation and consumption in
cpu_request_microcode()'s first loop, and also check the types of
skipped blocks in container_fast_forward().

Signed-off-by: Jan Beulich jbeul...@suse.com

--- a/xen/arch/x86/microcode_amd.c
+++ b/xen/arch/x86/microcode_amd.c
@@ -331,12 +331,17 @@ static int container_fast_forward(const 
  header[1] == UCODE_EQUIV_CPU_TABLE_TYPE )
 break;
 
+if ( header[0] != UCODE_UCODE_TYPE )
+return -EINVAL;
 size = header[1] + SECTION_HDR_SIZE;
 if ( size  PATCH_HDR_SIZE || size_left  size )
 return -EINVAL;
 
 size_left -= size;
 *offset += size;
+
+if ( !size_left )
+return -ENODATA;
 }
 
 return 0;
@@ -386,10 +391,6 @@ static int cpu_request_microcode(int cpu
 break;
 }
 
-if ( find_equiv_cpu_id(mc_amd-equiv_cpu_table, current_cpu_id,
-   equiv_cpu_id) )
-break;
-
 /*
  * Could happen as we advance 'offset' early
  * in install_equiv_cpu_table
@@ -401,7 +402,16 @@ static int cpu_request_microcode(int cpu
 break;
 }
 
+if ( find_equiv_cpu_id(mc_amd-equiv_cpu_table, current_cpu_id,
+   equiv_cpu_id) )
+break;
+
 error = container_fast_forward(buf, bufsize - offset, offset);
+if ( error == -ENODATA )
+{
+ASSERT(offset == bufsize);
+break;
+}
 if ( error )
 {
 printk(KERN_ERR microcode: CPU%d incorrect or corrupt container 
file\n



x86/AMD-ucode: correct multiple container handling

Avoid emitting an error message referring to an incorrect or corrupt
container file just because no entry was found for the running CPU.

Additionally switch the order of data validation and consumption in
cpu_request_microcode()'s first loop, and also check the types of
skipped blocks in container_fast_forward().

Signed-off-by: Jan Beulich jbeul...@suse.com

--- a/xen/arch/x86/microcode_amd.c
+++ b/xen/arch/x86/microcode_amd.c
@@ -331,12 +331,17 @@ static int container_fast_forward(const 
  header[1] == UCODE_EQUIV_CPU_TABLE_TYPE )
 break;
 
+if ( header[0] != UCODE_UCODE_TYPE )
+return -EINVAL;
 size = header[1] + SECTION_HDR_SIZE;
 if ( size  PATCH_HDR_SIZE || size_left  size )
 return -EINVAL;
 
 size_left -= size;
 *offset += size;
+
+if ( !size_left )
+return -ENODATA;
 }
 
 return 0;
@@ -386,10 +391,6 @@ static int cpu_request_microcode(int cpu
 break;
 }
 
-if ( find_equiv_cpu_id(mc_amd-equiv_cpu_table, current_cpu_id,
-   equiv_cpu_id) )
-break;
-
 /*
  * Could happen as we advance 'offset' early
  * in install_equiv_cpu_table
@@ -401,7 +402,16 @@ static int cpu_request_microcode(int cpu
 break;
 }
 
+if ( find_equiv_cpu_id(mc_amd-equiv_cpu_table, current_cpu_id,
+   equiv_cpu_id) )
+break;
+
 error = container_fast_forward(buf, bufsize - offset, offset);
+if ( error == -ENODATA )
+{
+ASSERT(offset == bufsize);
+break;
+}
 if ( error )
 {
 printk(KERN_ERR microcode: CPU%d incorrect or corrupt container 
file\n
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] One question about the hypercall to translate gfn to mfn.

2014-12-15 Thread Jan Beulich
 On 15.12.14 at 10:05, kevin.t...@intel.com wrote:
 yes, definitely host RAM is the upper limit, and what I'm concerning here
 is how to reserve (at boot time) or allocate (on-demand) such large PFN
 resource, w/o collision with other PFN reservation usage (ballooning
 should be fine since it's operating existing RAM ranges in dom0 e820
 table).

I don't think ballooning is restricted to the regions named RAM in
Dom0's E820 table (at least it shouldn't be, and wasn't in the
classic Xen kernels).

 Maybe we can reserve a big-enough reserved region in dom0's 
 e820 table at boot time, for all PFN reservation usages, and then allocate
 them on-demand for specific usages?

What would big enough here mean (i.e. how would one determine
the needed size up front)? Plus any form of allocation would need a
reasonable approach to avoid fragmentation. And anyway I'm not
getting what position you're on: Do you expect to be able to fit
everything that needs mapping into the available mapping space (as
your reply above seems to imply) or do you think there won't be
enough mapping space (as earlier replies of yours appeared to
indicate)?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] libxl.c::macro definition::LIBXL_SCHEDULER_SEDF

2014-12-15 Thread Ian Campbell
On Fri, 2014-12-12 at 16:49 -0600, Jie Shen wrote:
 Hello Gurus,
 
 
 I am working on a project to change code of xen,
 Unfortunately I can not find the macro def of : LIBXL_SCHEDULER_

Check tools/libxl/{libxl_types.idl,idl.py,gentypes.py,idl.txt}

Ian.



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 00/12] Enable vTPM subsystem on TPM 2.0

2014-12-15 Thread Ian Campbell
On Sun, 2014-12-14 at 07:09 -0500, Quan Xu wrote:
 This series of patch enable the virtual Trusted Platform Module (vTPM)
 subsystem for Xen on TPM 2.0.

Please can you investigate the --thread option to git send-email
and/or git format-patch, such that all of the mails appear as replies
to this one or at least to the previous patch.

This will ensure that all the patches stay together in any decent MUA,
which is much easier for maintainers to cope with.

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] tools: libxl: link libxlu against libxl.

2014-12-15 Thread Ian Campbell
On Mon, 2014-08-04 at 10:58 +0100, Ian Campbell wrote:

Ping again. This issue has resurfaced in the Debian packaging of the 4.5
rcs. I think we should fix this for 4.5, the risks are minimal.

 It uses libxl_defbool_set and must therefore be linked against the
 right library.
 
 Spotted by dpkg-shlibdeps and pointed out by Bastian Blank:
 
 dpkg-shlibdeps: warning: symbol libxl_defbool_set used by 
 debian/libxen-4.4/usr/lib/libxlutil-4.4.so found in none of the libraries
 
 This required switching the make rule from $^ to an explicit
 LIBXLU_OBJS since the former now includes libxenlight.so.
 
 Signed-off-by: Ian Campbell ian.campb...@citrix.com
 Cc: Bastian Blank wa...@debian.org
 ---
  tools/libxl/Makefile |6 +++---
  1 file changed, 3 insertions(+), 3 deletions(-)
 
 diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
 index bd0db3b..e61048d 100644
 --- a/tools/libxl/Makefile
 +++ b/tools/libxl/Makefile
 @@ -35,7 +35,7 @@ LDFLAGS += $(PTHREAD_LDFLAGS)
  LIBXL_LIBS += $(PTHREAD_LIBS)
  LIBXL_LIBS += $(LIBXL_LIBS-y)
  
 -LIBXLU_LIBS =
 +LIBXLU_LIBS = libxenlight.so
  
  LIBXL_OBJS-y = osdeps.o libxl_paths.o libxl_bootloader.o flexarray.o
  ifeq ($(LIBXL_BLKTAP),y)
 @@ -213,8 +213,8 @@ libxlutil.so: libxlutil.so.$(XLUMAJOR)
  libxlutil.so.$(XLUMAJOR): libxlutil.so.$(XLUMAJOR).$(XLUMINOR)
   $(SYMLINK_SHLIB) $ $@
  
 -libxlutil.so.$(XLUMAJOR).$(XLUMINOR): $(LIBXLU_OBJS)
 - $(CC) $(LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,libxlutil.so.$(XLUMAJOR) 
 $(SHLIB_LDFLAGS) -o $@ $^ $(LIBXLU_LIBS) $(APPEND_LDFLAGS)
 +libxlutil.so.$(XLUMAJOR).$(XLUMINOR): $(LIBXLU_OBJS) libxenlight.so
 + $(CC) $(LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,libxlutil.so.$(XLUMAJOR) 
 $(SHLIB_LDFLAGS) -o $@ $(LIBXLU_OBJS) $(LIBXLU_LIBS) $(APPEND_LDFLAGS)
  
  libxlutil.a: $(LIBXLU_OBJS)
   $(AR) rcs libxlutil.a $^



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for 4.5] x86/VPMU: Clear last_vcpu when destroying VPMU

2014-12-15 Thread Jan Beulich
 On 12.12.14 at 22:20, boris.ostrov...@oracle.com wrote:
 --- a/xen/arch/x86/hvm/vpmu.c
 +++ b/xen/arch/x86/hvm/vpmu.c
 @@ -247,10 +247,32 @@ void vpmu_initialise(struct vcpu *v)
  }
  }
  
 +static void vpmu_clear_last(void *arg)
 +{
 +struct vcpu *v = (struct vcpu *)arg;

Please drop this pointless cast, or perhaps the entire variable, as ...

 +
 +if ( this_cpu(last_vcpu) == v )

... you don't really need it to be of struct vcpu * type - void *
is quite fine for a comparison.

 +this_cpu(last_vcpu) = NULL;
 +}
 +
  void vpmu_destroy(struct vcpu *v)
  {
  struct vpmu_struct *vpmu = vcpu_vpmu(v);
  
 +if ( vpmu_is_set(vpmu, VPMU_CONTEXT_ALLOCATED) )
 +{
 +/* Need to clear last_vcpu in case it points to v */
 +if ( vpmu-last_pcpu != smp_processor_id() )
 +on_selected_cpus(cpumask_of(vpmu-last_pcpu),
 + vpmu_clear_last, (void *)v, 1);

The cast here is pointless too. But considering your subsequent
reply this code may go away altogether anyway; if it doesn't,
explaining (in the commit message) why you need to use an IPI
here would seem necessary.

 +else
 +{
 +local_irq_disable();
 +vpmu_clear_last((void *)v);

And another pointless cast.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] xenconfig: fix boot device parsing

2014-12-15 Thread Ian Campbell
On Sun, 2014-12-14 at 14:58 +, Wei Liu wrote:
 The original code always checked *boot which was in effect boot[0]. It
 should use boot[i].
 
 Signed-off-by: Wei Liu wei.l...@citrix.com

Acked-by: Ian Campbell ian.campb...@citrix.com



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Ballooning dom0: insufficient memory (libxl) or CPU soft lockups (libvirt)

2014-12-15 Thread Jan Beulich
 On 14.12.14 at 04:20, mlati...@suse.com wrote:
 - If using virsh, the domain can be created while dom0 is still ballooning 
 down. This results in CPU soft lockups/performance degradation across the 
 entire host. (When creating a very large domain, the soft lockups can be 
 severe enough to kill the machine.)

This is an issue that needs handling independent of the tool stack
issues pointed out. Whether we should discuss this here depends
on whether we suspect the problem to be only with our kernels, or
also with upstream's. It's in any event quite odd for there to be
softlockups in the first place, as in the hypervisor the involved
hypercalls are all preemptible afaict. I'll have to go did through the
data you had provided earlier to see whether there's a complete
enough (and readable) log to be able to tell where there's a lack
of re-scheduling in the kernel.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC Patch v4 3/9] pass correct file to qemu if we use blktap2

2014-12-15 Thread Ian Campbell
On Sat, 2014-12-13 at 17:06 +, Wei Liu wrote:
 On Thu, Dec 11, 2014 at 04:45:09PM +, George Dunlap wrote:
  On Mon, Sep 22, 2014 at 6:59 AM, Wen Congyang we...@cn.fujitsu.com wrote:
   If we use blktap2, the correct file should be blktap device
   not the pdev_path.
  
   Signed-off-by: Wen Congyang we...@cn.fujitsu.com
   Cc: Shriram Rajagopalan rshri...@cs.ubc.ca
   Acked-by: Ian Campbell ian.campb...@citrix.com
  
  If I'm reading this correctly, this is actually a fairly serious bug
  in libxl -- it means that when using backend=tap with HVM domains,
  qemu will actually *bypass entirely* the tapdisk process and access
  the underlying file directly.
  
 
 Is it because qemu will corrupt the underlying file so it's very
 serious? 

I think it ends up being reasonably safe due to the unplug stuff, in
general only one of the two paths should be active at any given time and
there is appropriate flushing/syncing on the unplug etc.

In fact, I'm not 100% sure this wasn't a deliberate design decision at
one point to avoid the overhead of doing both qdisk and tapdisk. (I'm
not going to argue that this still makes sense though).

If qemu is corrupting files then that's a different matter, which would
indeed be serious, and which this change might paper over/avoid.

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [linux-linus test] 32374: regressions - FAIL

2014-12-15 Thread xen . org
flight 32374 linux-linus real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/32374/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   5 xen-build fail REGR. vs. 31241
 build-i386-pvops  5 kernel-build  fail REGR. vs. 31241
 build-i3865 xen-build fail REGR. vs. 31241

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl   9 guest-start  fail   like 31241

Tests which did not succeed, but are not blocking:
 test-amd64-i386-rumpuserxen-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumpuserxen-amd64  1 build-check(1)   blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  9 guest-start  fail   never pass
 build-amd64-rumpuserxen   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemut-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-pcipt-intel  1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-rhel6hvm-amd  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-sedf-pin  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-xl-credit21 build-check(1)   blocked  n/a
 build-i386-rumpuserxen1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-sedf  1 build-check(1)   blocked  n/a
 test-amd64-i386-rhel6hvm-intel  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-i386-qemut-rhel6hvm-intel  1 build-check(1) blocked n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-win7-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-win7-amd64  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemut-winxpsp3  1 build-check(1)   blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 test-amd64-i386-xl-winxpsp3-vcpus1  1 build-check(1)   blocked n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-winxpsp3   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-winxpsp3  1 build-check(1)   blocked n/a

version targeted for testing:
 linuxe6b5be2be4e30037eb551e0ed09dd97bd00d85d3
baseline version:
 linux9f76628da20f96a179ca62b504886f99ecc29223


1658 people touched revisions under test,
not listing them all


jobs:
 build-amd64  fail
 build-armhf  pass
 build-i386   fail
 build-amd64-libvirt  blocked 
 build-armhf-libvirt

[Xen-devel] Lots of ACPI warnings from newer iasl (20140926).

2014-12-15 Thread Ian Campbell
isal 20140926-1 in Debian Jessie is rather vocal about
tools/firmware/hvmloader/acpi. See below. Essentially, a few of these:
Control Method should be made Serialized (due to creation of
named objects within)
a slew of:
Object is not referenced ^  (Name is within method [_CRS])
and a load of:
^ Reserved method should not return a value (_L02)

4.4 seems to produce a similar looking set of warnings, so I think this
is a case of newer iasl having more warnings rather than a regression in
our code.

I also tried iasl from 20141107-1 Sid and it is similarly vocal.

Ian.

make: Entering directory 
'/local/scratch/ianc/devel/xen.git/tools/firmware/hvmloader'
make[1]: Entering directory 
'/local/scratch/ianc/devel/xen.git/tools/firmware/hvmloader'
make -C acpi all
make[2]: Entering directory 
'/local/scratch/ianc/devel/xen.git/tools/firmware/hvmloader/acpi'
iasl -vs -p ssdt_s3 -tc ssdt_s3.asl
ASL Input: ssdt_s3.asl - 33 lines, 172 bytes, 1 keywords
AML Output:ssdt_s3.aml - 49 bytes, 1 named objects, 0 executable opcodes
Hex Dump:  ssdt_s3.hex - 844 bytes

Compilation complete. 0 Errors, 0 Warnings, 0 Remarks, 4 Optimizations
sed -e 's/AmlCode/ssdt_s3/g' ssdt_s3.hex ssdt_s3.h
rm -f ssdt_s3.hex ssdt_s3.aml
iasl -vs -p ssdt_s4 -tc ssdt_s4.asl
ASL Input: ssdt_s4.asl - 33 lines, 172 bytes, 1 keywords
AML Output:ssdt_s4.aml - 49 bytes, 1 named objects, 0 executable opcodes
Hex Dump:  ssdt_s4.hex - 844 bytes

Compilation complete. 0 Errors, 0 Warnings, 0 Remarks, 4 Optimizations
sed -e 's/AmlCode/ssdt_s4/g' ssdt_s4.hex ssdt_s4.h
rm -f ssdt_s4.hex ssdt_s4.aml
iasl -vs -p ssdt_pm -tc ssdt_pm.asl
ssdt_pm.asl281: Method (HLPA, 0, NotSerialized)
Remark   2120 -   ^ Control Method should be made 
Serialized (due to creation of named objects within)

ssdt_pm.asl368: Method (_BST, 0, NotSerialized)
Remark   2120 -   ^ Control Method should be made 
Serialized (due to creation of named objects within)

ssdt_pm.asl406: Method (_BST, 0, NotSerialized)
Remark   2120 -   ^ Control Method should be made 
Serialized (due to creation of named objects within)

ASL Input: ssdt_pm.asl - 424 lines, 8311 bytes, 192 keywords
AML Output:ssdt_pm.aml - 1494 bytes, 64 named objects, 128 executable 
opcodes
Hex Dump:  ssdt_pm.hex - 14345 bytes

Compilation complete. 0 Errors, 0 Warnings, 3 Remarks, 31 Optimizations
sed -e 's/AmlCode/ssdt_pm/g' ssdt_pm.hex ssdt_pm.h
rm -f ssdt_pm.hex ssdt_pm.aml
iasl -vs -p ssdt_tpm -tc ssdt_tpm.asl
ASL Input: ssdt_tpm.asl - 33 lines, 250 bytes, 3 keywords
AML Output:ssdt_tpm.aml - 76 bytes, 3 named objects, 0 executable opcodes
Hex Dump:  ssdt_tpm.hex - 1070 bytes

Compilation complete. 0 Errors, 0 Warnings, 0 Remarks, 0 Optimizations
sed -e 's/AmlCode/ssdt_tpm/g' ssdt_tpm.hex ssdt_tpm.h
rm -f ssdt_tpm.hex ssdt_tpm.aml
gcc   -O1 -fno-omit-frame-pointer -m32 -march=i686 -g -fno-strict-aliasing 
-std=gnu99 -Wall -Wstrict-prototypes -Wdeclaration-after-statement 
-Wno-unused-but-set-variable -Wno-unused-local-typedefs   -D__XEN_TOOLS__ -MMD 
-MF .build.o.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE 
-fno-optimize-sibling-calls -mno-tls-direct-seg-refs  -Werror 
-fno-stack-protector -fno-exceptions -fno-builtin -msoft-float 
-I/local/scratch/ianc/devel/xen.git/tools/firmware/hvmloader/acpi/../../../../tools/include
  -c -o build.o build.c 
gcc -Wall -Werror -Wstrict-prototypes -O2 -fomit-frame-pointer 
-fno-strict-aliasing -Wdeclaration-after-statement 
-I/local/scratch/ianc/devel/xen.git/tools/firmware/hvmloader/acpi/../../../../tools/include
 -o mk_dsdt mk_dsdt.c
awk 'NR  1 {print s} {s=$0}' dsdt.asl  dsdt_anycpu.asl
./mk_dsdt --maxcpu any   dsdt_anycpu.asl
iasl -vs -p dsdt_anycpu -tc dsdt_anycpu.asl
dsdt_anycpu.asl110:Method (_CRS, 0, NotSerialized)
Remark   2120 -  ^ Control Method should be made 
Serialized (due to creation of named objects within)

dsdt_anycpu.asl151: ResourceProducer, PosDecode, 
MinFixed, MaxFixed,
Remark   2089 - Object is not referenced ^  
(Name is within method [_CRS])

dsdt_anycpu.asl151: ResourceProducer, PosDecode, 
MinFixed, MaxFixed,
Remark   2089 -   Object is not 
referenced ^  (Name is within method [_CRS])

dsdt_anycpu.asl151: ResourceProducer, PosDecode, 
MinFixed, MaxFixed,
Remark   2089 - Object is 
not referenced ^  (Name is within method [_CRS])

dsdt_anycpu.asl152: NonCacheable, ReadWrite,
Remark   2089 -  Object is not referenced ^  (Name is within 
method [_CRS])

dsdt_anycpu.asl152: NonCacheable, ReadWrite,
Remark   2089 - Object is not 

Re: [Xen-devel] [PATCH for-4.5] xl: print message to stdout when (!debug dryrun)

2014-12-15 Thread Ian Campbell
On Sat, 2014-12-13 at 16:54 +, Wei Liu wrote:
 In commit d36a3734a (xl: fix migration failure with xl migrate
 --debug), message is printed to stderr for both debug mode
 and dryrun mode. That caused rdname() in xendomains fails to parse
 domain name since it's expecting input from xl's stdout.
 
 So this patch separates those two cases. If xl is running in debug mode,
 then message is printed to stderr; if xl is running in dryrun mode and
 debug is not enabled, message is printed to stdout. This will fix
 xendomains and other scripts that use xl create --dryrun, as well as
 not re-introducing the old bug fixed in d36a3734a.
 
 Reported-by: Mark Pryor tlvie...@yahoo.com
 Signed-off-by: Wei Liu wei.l...@citrix.com
 Cc: M A Young m.a.yo...@durham.ac.uk
 Cc: Ian Campbell ian.campb...@citrix.com
 Cc: Konrad Wilk konrad.w...@oracle.com
 ---
 This is a regression, and this bug is so subtle that's a bit hard to
 debug from user's point of view. So I think this should go into 4.5.

Agreed.

 Mark posted a workaround in
 104017455.78913.1418434454763.javamail.ya...@jws10624.mail.bf1.yahoo.com
 but to be honest I don't think it's nice to have everyone patch their
 scripts in order to work around this.

Right.

 ---
  tools/libxl/xl_cmdimpl.c |4 +++-
  1 file changed, 3 insertions(+), 1 deletion(-)
 
 diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
 index 3737c7e..0a5f7c8 100644
 --- a/tools/libxl/xl_cmdimpl.c
 +++ b/tools/libxl/xl_cmdimpl.c
 @@ -2472,8 +2472,10 @@ static uint32_t create_domain(struct domain_create 
 *dom_info)
  }
  }
  
 -if (debug || dom_info-dryrun)
 +if (debug)
  printf_info(default_output_format, -1, d_config, stderr);
 +if (!debug  dom_info-dryrun)

   else if ( dom_info-dry-run )

is the same (right?) and less thinking for the reader.

Or the whole thing could be just:
if (debug || dom_info-dryrun)
  printf_info(default_output_format, -1, d_config, debug ? stderr : 
stdout);

Less repetitive.

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for-4.5] xl: print message to stdout when (!debug dryrun)

2014-12-15 Thread Ian Campbell
On Sat, 2014-12-13 at 13:46 -0500, Konrad Rzeszutek Wilk wrote:
 On Sat, Dec 13, 2014 at 06:42:09PM +, Wei Liu wrote:
  On Sat, Dec 13, 2014 at 01:37:16PM -0500, Konrad Rzeszutek Wilk wrote:
   On Sat, Dec 13, 2014 at 04:54:05PM +, Wei Liu wrote:
In commit d36a3734a (xl: fix migration failure with xl migrate
--debug), message is printed to stderr for both debug mode
and dryrun mode. That caused rdname() in xendomains fails to parse
domain name since it's expecting input from xl's stdout.

So this patch separates those two cases. If xl is running in debug mode,
then message is printed to stderr; if xl is running in dryrun mode and
debug is not enabled, message is printed to stdout. This will fix
xendomains and other scripts that use xl create --dryrun, as well as
not re-introducing the old bug fixed in d36a3734a.

Reported-by: Mark Pryor tlvie...@yahoo.com
Signed-off-by: Wei Liu wei.l...@citrix.com
Cc: M A Young m.a.yo...@durham.ac.uk
Cc: Ian Campbell ian.campb...@citrix.com
Cc: Konrad Wilk konrad.w...@oracle.com
---
This is a regression, and this bug is so subtle that's a bit hard to
debug from user's point of view. So I think this should go into 4.5.

Mark posted a workaround in
104017455.78913.1418434454763.javamail.ya...@jws10624.mail.bf1.yahoo.com
but to be honest I don't think it's nice to have everyone patch their
scripts in order to work around this.
---
 tools/libxl/xl_cmdimpl.c |4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 3737c7e..0a5f7c8 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -2472,8 +2472,10 @@ static uint32_t create_domain(struct 
domain_create *dom_info)
 }
 }
 
-if (debug || dom_info-dryrun)
+if (debug)
 printf_info(default_output_format, -1, d_config, stderr);
+if (!debug  dom_info-dryrun)
   
   Could this 'else if (dom_info-dry_run)' ?
   
   I know that semantically it is exactly the same as the 'if (!debug ..)' 
   but
   it just feels more proper? Thought since you are the maintainer in that
   area - your opinion on how you want to do that is of course authoritive.
   
  
  I don't have strong preference on this. I just happened to type in
  whatever style that flowed though my mind at that particular point. If
  Ian and you both feel strongly about this I can certainly resend. :-)
 
 Lets wait to see what Ian says (or what he ends up checking in).

I really ought to read the thread before replying. I do prefer the else
if style, or the ternary operator one I also proposed.



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Linux Xen Balloon Driver Improvement (Draft 2)

2014-12-15 Thread David Vrabel
On 27/10/14 12:33, Wei Liu wrote:
 
 ### Periodically exchange normal size pages with huge pages
 
 Worker thread wakes up periodically to check if there are enough pages
 in normal size page queue to coalesce into a huge page. If so, it will
 try to exchange that huge page into a number of normal size pages with
 XENMEM\_exchange hypercall.

As Andrew recently pointed out[1], changes to a guest's p2m are not
properly handled during migration.  This would have to be fixed before
we can have any sort of background memory exchange process.

David

[1] http://lists.xen.org/archives/html/xen-devel/2014-11/msg01954.html

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH for-4.5 v2] xl: print message to stdout when (!debug dryrun)

2014-12-15 Thread Wei Liu
In commit d36a3734a (xl: fix migration failure with xl migrate
--debug), message is printed to stderr for both debug mode
and dryrun mode. That caused rdname() in xendomains fails to parse
domain name since it's expecting input from xl's stdout.

So this patch separates those two cases. If xl is running in debug mode,
then message is printed to stderr; if xl is running in dryrun mode and
debug is not enabled, message is printed to stdout. This will fix
xendomains and other scripts that use xl create --dryrun, as well as
not re-introducing the old bug fixed in d36a3734a.

Reported-by: Mark Pryor tlvie...@yahoo.com
Signed-off-by: Wei Liu wei.l...@citrix.com
Cc: M A Young m.a.yo...@durham.ac.uk
Cc: Ian Campbell ian.campb...@citrix.com
Release-Acked-by: Konrad Wilk konrad.w...@oracle.com
---
 tools/libxl/xl_cmdimpl.c |3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 3737c7e..ed0d478 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -2473,7 +2473,8 @@ static uint32_t create_domain(struct domain_create 
*dom_info)
 }
 
 if (debug || dom_info-dryrun)
-printf_info(default_output_format, -1, d_config, stderr);
+printf_info(default_output_format, -1, d_config,
+debug ? stderr : stdout);
 
 ret = 0;
 if (dom_info-dryrun)
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] Mini-OS: netfront: Fix rx ring starvation in network_rx

2014-12-15 Thread Martin Lucina
samuel.thiba...@ens-lyon.org said:
  From: Martin Lucina mar...@lucina.net
  Date: Thu, 4 Dec 2014 14:33:53 +0100
  Subject: [PATCH] Mini-OS: netfront: Fix rx ring starvation in network_rx
  
  In network_rx() we must push the same amount of requests back onto the
  ring in the second loop that we consumed in the first loop. Otherwise
  the ring will eventually starve itself of free request slots and no
  packets will be delivered.
  
  Further, we make the HAVE_LIBC codepath clearer to follow by removing
  the some variable from the for loop initialisation and conditions.
  
  Signed-off-by: Martin Lucina mar...@lucina.net
 
 Acked-by: Samuel Thibault samuel.thiba...@ens-lyon.org

Thanks. I will merge a suitable version of this patch into our downstream
tree.

Is there anything else I need to do to ensure this change lands in xen.git?

Martin

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Linux Xen Balloon Driver Improvement (Draft 2)

2014-12-15 Thread Wei Liu
On Mon, Dec 15, 2014 at 10:52:40AM +, David Vrabel wrote:
 On 27/10/14 12:33, Wei Liu wrote:
  
  ### Periodically exchange normal size pages with huge pages
  
  Worker thread wakes up periodically to check if there are enough pages
  in normal size page queue to coalesce into a huge page. If so, it will
  try to exchange that huge page into a number of normal size pages with
  XENMEM\_exchange hypercall.
 
 As Andrew recently pointed out[1], changes to a guest's p2m are not
 properly handled during migration.  This would have to be fixed before
 we can have any sort of background memory exchange process.
 

Thanks for the pointer. I read that thread already.

Wei.

 David
 
 [1] http://lists.xen.org/archives/html/xen-devel/2014-11/msg01954.html

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [qemu-mainline test] 32377: regressions - FAIL

2014-12-15 Thread xen . org
flight 32377 qemu-mainline real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/32377/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i386-pvops  5 kernel-build  fail REGR. vs. 32194
 build-amd64   5 xen-build fail REGR. vs. 32194
 build-i3865 xen-build fail REGR. vs. 32194

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt  9 guest-start  fail   never pass
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pcipt-intel  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemut-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-rhel6hvm-amd  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-sedf  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-credit21 build-check(1)   blocked  n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-sedf-pin  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-win7-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemut-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-win7-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-rhel6hvm-intel  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qemut-winxpsp3  1 build-check(1)   blocked n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-armhf-armhf-xl  10 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-xl-qemuu-winxpsp3  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-winxpsp3-vcpus1  1 build-check(1)   blocked n/a
 test-amd64-i386-xl-qemuu-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-winxpsp3   1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a

version targeted for testing:
 qemuu99c9c3cb24e566258a0a141178934f9cb5198842
baseline version:
 qemuu45e1611de8be0eae55967694dd6e627c2dc354f2


People who touched revisions under test:
  Alex Bennée alex.ben...@linaro.org
  Amos Kong ak...@redhat.com
  Anton Blanchard an...@samba.org
  Antony Pavlov antonynpav...@gmail.com
  Bastian Koppelmann kbast...@mail.uni-paderborn.de
  Christoffer Dall christoffer.d...@linaro.org
  Dr. David Alan Gilbert dgilb...@redhat.com
  Fabian Aggeler aggel...@ethz.ch
  Fam Zheng f...@redhat.com
  Gerd Hoffmann kra...@redhat.com
  Gonglei arei.gong...@huawei.com
  Greg Bellows greg.bell...@linaro.org
  Jeff Cody jc...@redhat.com
  Jun Li junm...@gmail.com
  Kevin Wolf kw...@redhat.com
  lijun junm...@gmail.com
  Liviu Ionescu i...@livius.net
  Markus Armbruster arm...@redhat.com
  Max Reitz mre...@redhat.com
  Michael Mueller m...@linux.vnet.ibm.com
  Michael Tokarev m...@tls.msk.ru
  Nikita Belov zod...@ispras.ru
  Paolo Bonzini 

Re: [Xen-devel] Xen's Linux kernel config options

2014-12-15 Thread Juergen Gross

On 12/12/2014 05:44 PM, Konrad Rzeszutek Wilk wrote:

On Fri, Dec 12, 2014 at 02:17:27PM +0100, Juergen Gross wrote:

Hi,

This is a design proposal for a rework of the config options on the
Linux kernel which are related to Xen.

The need to do so arose from the fact that it is currently not
possible to build the Xen frontend drivers for a non-pvops kernel,
e.g. to run them in a HVM-domain. There are more drawbacks in the
current config options to which I'll come later.

Current Xen related config options are as follows:

Option  Selects Depends
-
XEN PARAVIRT_CLOCK(x86) PARAVIRT(x86)
 XEN_HAVE_PVMMU(x86)
 SWIOTLB_XEN(arm,arm64)
   PCI_XEN(x86)  SWIOTLB_XEN
   XEN_DOM0  PCI_XEN(x86)
 XEN_BACKEND
   XEN_BLKDEV_BACKEND
   XEN_PCIDEV_BACKEND(x86)
   XEN_SCSI_BACKEND
   XEN_NETDEV_BACKEND
 XEN_ACPI_HOTPLUG_MEMORY XEN_STUB
 XEN_ACPI_HOTPLUG_CPUXEN_STUB
 XEN_MCE_LOG(x86)
   XEN_PVHVM(x86)
 XEN_PVH(x86)
   XEN_MAX_DOMAIN_MEMORY(x86)
   XEN_SAVE_RESTORE(x86)
   XEN_DEBUG_FS(x86)
   XEN_FBDEV_FRONTENDXEN_XENBUS_FRONTEND
 INPUT_XEN_KBDDEV_FRONTEND
   XEN_BLKDEV_FRONTEND   XEN_XENBUS_FRONTEND
   HVC_XEN
 HVC_XEN_FRONTENDXEN_XENBUS_FRONTEND
   TCG_XEN   XEN_XENBUS_FRONTEND
   XEN_PCIDEV_FRONTEND   PCI_XEN
 XEN_XENBUS_FRONTEND
   XEN_SCSI_FRONTEND XEN_XENBUS_FRONTEND
   INPUT_XEN_KBDDEV_FRONTEND XEN_XENBUS_FRONTEND
   XEN_WDT
   XEN_BALLOON
 XEN_SELFBALLOONING  XEN_TMEM
 XEN_BALLOON_MEMORY_HOTPLUG
 XEN_SCRUB_PAGES
   XEN_DEV_EVTCHN
   XENFS XEN_PRIVCMD
 XEN_COMPAT_XENFS
   XEN_SYS_HYPERVISOR
   XEN_XENBUS_FRONTEND
   XEN_GNTDEV
   XEN_GRANT_DEV_ALLOC
   SWIOTLB_XEN
   XEN_TMEM(x86)
   XEN_PRIVCMD
   XEN_STUB(x86_64)  BROKEN
   XEN_ACPI_PROCESSOR(x86)
   XEN_HAVE_PVMMU
   XEN_EFI(x64)
   XEN_NETDEV_FRONTEND   XEN_XENBUS_FRONTEND

Legend:
- The first column are the Xen config options. Indentation in this
   column reflects the dependency between those options (e.g.
   XEN_SCSI_BACKEND depends on XEN_BACKEND, which in turn depends on
   XEN_DOM0, which depends on XEN).
- The second column shows the options which are selected automatically
   if the corresponding option in the first column is configured.
- The last column shows additional dependencies which can't be shown via
   the indentation level of the first column.
- Some options or dependencies are architecture specific, they are
   listed in brackets (e.g. XEN_TMEM which is available for x86 only).
- Non-Xen options are mentioned only if they seem to be really relevant,
   like e.g. PARAVIRT or BROKEN.

There are several reasons to change the config options:
- Be able to build Xen frontend drivers for HVM domains without the need
   for choosing a pvops kernel: currently the frontend drivers need XEN
   configured which depends on PARAVIRT.
- Be able to build a Dom0 using XEN_PVH without having to configure
   XEN_HAVE_PVMMU.
- Be able to build HVM driver domains.
- Some features are available for x86 only, in spite of being not
   architecture specific, e.g. XEN_DEBUG_FS or XEN_TMEM.

As a first draft I'd suggest the following:

Option  Selects Depends
--
XEN SWIOTLB_XEN(arm,arm64)
   XEN_PV(x86)   XEN_HAVE_PVMMU
 PARAVIRT
 PARAVIRT_CLOCK
   XEN_PVH(x86)  XEN_PVHVM
 PARAVIRT
 PARAVIRT_CLOCK
   XEN_BACKEND   XEN_PV(x86) ||
 XEN_PVH(x86) ||
 XEN_PVHVM
 XEN_BLKDEV_BACKEND
 XEN_PCIDEV_BACKEND(x86)
 XEN_SCSI_BACKEND
 XEN_NETDEV_BACKEND
   PCI_XEN(x86)  SWIOTLB_XEN
   XEN_DOM0  XEN_BACKEND XEN_PV(x86) ||
 PCI_XEN(x86)XEN_PVH(x86)


Could XEN_DOM0 be removed completly? And instead we just have
an 'backend' and 'frontend' type options?


What about (physical) memory/cpu hotplug and mce?

We could rename XEN_DOM0 to XEN_HWDOM, however.





 XEN_ACPI_HOTPLUG_MEMORY XEN_STUB
 XEN_ACPI_HOTPLUG_CPUXEN_STUB
 XEN_MCE_LOG(x86)
   

[Xen-devel] [linux-3.10 test] 32379: regressions - FAIL

2014-12-15 Thread xen . org
flight 32379 linux-3.10 real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/32379/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   5 xen-build fail REGR. vs. 26303
 build-i3865 xen-build fail REGR. vs. 26303
 build-i386-pvops  5 kernel-build  fail REGR. vs. 26303
 build-amd64-pvops 5 kernel-build  fail REGR. vs. 26303

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-rumpuserxen-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pcipt-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumpuserxen-amd64  1 build-check(1)   blocked n/a
 test-amd64-i386-qemut-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-rhel6hvm-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-sedf-pin  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  5 xen-boot fail   never pass
 test-amd64-i386-xl-credit21 build-check(1)   blocked  n/a
 test-amd64-i386-rhel6hvm-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-sedf  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   5 xen-boot fail   never pass
 test-amd64-i386-qemut-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64  1 build-check(1) blocked n/a
 build-amd64-rumpuserxen   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-winxpsp3-vcpus1  1 build-check(1)   blocked n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-winxpsp3   1 build-check(1)   blocked  n/a
 build-i386-rumpuserxen1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-win7-amd64  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-win7-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qemuu-winxpsp3  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemut-winxpsp3  1 build-check(1)   blocked n/a
 test-amd64-i386-xl-qemuu-winxpsp3  1 build-check(1)   blocked  n/a

version targeted for testing:
 linux2f9ac85b35cce77eb36e415f8f7a36aefb7d977d
baseline version:
 linuxbe67db109090b17b56eb8eb2190cd70700f107aa


797 people touched revisions under test,
not listing them all


jobs:
 build-amd64  fail
 build-armhf  pass
 build-i386   fail
 build-amd64-libvirt  blocked 
 build-armhf-libvirt   

Re: [Xen-devel] [PATCH for-4.5 v2] xl: print message to stdout when (!debug dryrun)

2014-12-15 Thread Ian Campbell
On Mon, 2014-12-15 at 10:56 +, Wei Liu wrote:
 In commit d36a3734a (xl: fix migration failure with xl migrate
 --debug), message is printed to stderr for both debug mode
 and dryrun mode. That caused rdname() in xendomains fails to parse
 domain name since it's expecting input from xl's stdout.
 
 So this patch separates those two cases. If xl is running in debug mode,
 then message is printed to stderr; if xl is running in dryrun mode and
 debug is not enabled, message is printed to stdout. This will fix
 xendomains and other scripts that use xl create --dryrun, as well as
 not re-introducing the old bug fixed in d36a3734a.
 
 Reported-by: Mark Pryor tlvie...@yahoo.com
 Signed-off-by: Wei Liu wei.l...@citrix.com
 Cc: M A Young m.a.yo...@durham.ac.uk
 Cc: Ian Campbell ian.campb...@citrix.com
 Release-Acked-by: Konrad Wilk konrad.w...@oracle.com

Acked-by: Ian Campbell ian.campb...@citrix.com




___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] SRIOV on a NIC card

2014-12-15 Thread Ian Campbell
On Sun, 2014-12-14 at 21:09 +0530, Galla Rao wrote:
 True it is not on Xen, it is on solaris platform
 
 
 As SRIOV (PCIe) spec is same on all platforms, the feature
 implementation
 should not change
 
 
 May be not an appropriate group for posting this query.

If you aren't using Xen then of course xen-devel (a list for the
development of Xen) isn't an appropriate forum.

Ian.



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] libxl.c::definition::can not find itb

2014-12-15 Thread Stefano Stabellini
On Fri, 12 Dec 2014, Jie Shen wrote:
 Hello Vincent and Stefano,

Hello Jie,
please ask libxl related questions on the xen mailing list
(xen-devel@lists.xen.org), CC'ed.



 I am working on a project to change xen's code.
 
 Unfortunately I can not find the MACRO def :
 such as :
 
 LIBXL_SCHEDULER_SEDF:
 
 where I can find it :
 
 thank you very much!
 
 
 =
   switch (sched) {
     case LIBXL_SCHEDULER_SEDF:
         ret=sched_sedf_domain_set(gc, domid, scinfo);
         break;
     case LIBXL_SCHEDULER_CREDIT:
         ret=sched_credit_domain_set(gc, domid, scinfo);
         break;
     case LIBXL_SCHEDULER_CREDIT2:
         ret=sched_credit2_domain_set(gc, domid, scinfo);
         break;
     case LIBXL_SCHEDULER_RTGLOBAL:
         ret=sched_rtglobal_domain_set(gc, domid, scinfo);
         break;
     case LIBXL_SCHEDULER_RTPARTITION:
         ret=sched_rtpartition_domain_set(gc, domid, scinfo);
         break;     
     case LIBXL_SCHEDULER_ARINC653:
         ret=sched_arinc653_domain_set(gc, domid, sc
 

It is defined in _libxl_types.h, that is autogenerated by gentypes.py.


 
 
 
 =
 
 
 
 
 
  Best regards
 Jie Shen
 
 
 Graduate Student in Computer Science
 Illinois Institute of Technology   
 Stuart Building  Chicago, IL 60616
 +1  312 404 0122
 
 
 ___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC Patch v4 3/9] pass correct file to qemu if we use blktap2

2014-12-15 Thread George Dunlap
On 12/15/2014 10:18 AM, Ian Campbell wrote:
 On Sat, 2014-12-13 at 17:06 +, Wei Liu wrote:
 On Thu, Dec 11, 2014 at 04:45:09PM +, George Dunlap wrote:
 On Mon, Sep 22, 2014 at 6:59 AM, Wen Congyang we...@cn.fujitsu.com wrote:
 If we use blktap2, the correct file should be blktap device
 not the pdev_path.

 Signed-off-by: Wen Congyang we...@cn.fujitsu.com
 Cc: Shriram Rajagopalan rshri...@cs.ubc.ca
 Acked-by: Ian Campbell ian.campb...@citrix.com

 If I'm reading this correctly, this is actually a fairly serious bug
 in libxl -- it means that when using backend=tap with HVM domains,
 qemu will actually *bypass entirely* the tapdisk process and access
 the underlying file directly.


 Is it because qemu will corrupt the underlying file so it's very
 serious? 
 
 I think it ends up being reasonably safe due to the unplug stuff, in
 general only one of the two paths should be active at any given time and
 there is appropriate flushing/syncing on the unplug etc.
 
 In fact, I'm not 100% sure this wasn't a deliberate design decision at
 one point to avoid the overhead of doing both qdisk and tapdisk. (I'm
 not going to argue that this still makes sense though).
 
 If qemu is corrupting files then that's a different matter, which would
 indeed be serious, and which this change might paper over/avoid.

Yes, I think when I wrote that I had forgotten that even with just PV /
emulated there's usually a duplicate datapath that we need to sort out,
and that we already sort that out with the device unplug protocol.

Since half the point of tapdisk was to be able to do things like
snapshotting / other fancy tricks as colo does, not just implement
specific image formats, then if it was deliberate it was kind of dumb.
I'd rather just assume that it was overlooked. :-)

In any case, as it's already fixed in 4.5, I don't think the exact
degree of seriousness matters too much: it's above the threshold for
should be backported IMHO, and well below the threshold of needs the
security response process.  So it should just be backported to 4.4,
4.3, and 4.2  (or some subset thereof, if we're designating stable trees).

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Xen's Linux kernel config options

2014-12-15 Thread Juergen Gross

On 12/12/2014 06:29 PM, David Vrabel wrote:

On 12/12/14 13:17, Juergen Gross wrote:



As a first draft I'd suggest the following:


Some rough thoughts below.


Option  Selects Depends
--
XEN SWIOTLB_XEN(arm,arm64)


XEN should get you basic support for making hypercalls and allowing
guest to identify themselves as running on Xen.  I don't think it should
select SWIOTLB_XEN even on ARM as it is only needed for guests with
access to real hardware.


Okay.




   XEN_PV(x86)   XEN_HAVE_PVMMU
 PARAVIRT
 PARAVIRT_CLOCK
   XEN_PVH(x86)  XEN_PVHVM
 PARAVIRT
 PARAVIRT_CLOCK
   XEN_BACKEND   XEN_PV(x86) ||
 XEN_PVH(x86) ||
 XEN_PVHVM
 XEN_BLKDEV_BACKEND
 XEN_PCIDEV_BACKEND(x86)
 XEN_SCSI_BACKEND
 XEN_NETDEV_BACKEND

[...]

XEN_PVHVM


Move XEN_PVHVM under XEN and have it select PARAVIRT and PARAVIRT_CLOCK.


Okay.




XEN_FRONTENDXEN_PV ||
 XEN_PVH ||
 XEN_PVHVM


This enables all the basic infrastructure for frontends: event channels,
grant tables and Xenbus.

Don't make XEN_FRONTEND depend on any XEN_* variant.  It should be
possible to have frontend drivers without support for any of the
PV/PVHVM/PVH guest types.  Frontends only need event channels, grant
table and xenbus.


Okay.


Perhaps have XEN_FRONTEND select XEN instead?


Hmm, with XEN not selecting anything else automatically (apart from
stuff needed by frontends AND backends) this would be okay, I think.


You need an additional option to enable the Xen platform PCI device.
This should depend on XEN_FRONTEND.


Yep.




   XEN_XENBUS_FRONTEND


Fold this into XEN_FRONTEND?


Hmm, what about Xenstore in an own domain? I'd say XEN_FRONTEND should
select XEN_XENBUS_FRONTEND, but XEN_XENBUS_FRONTEND should be available
to non-frontend domains as well.




   XEN_FBDEV_FRONTENDXEN_XENBUS_FRONTEND
 INPUT_XEN_KBDDEV_FRONTEND
   XEN_BLKDEV_FRONTEND   XEN_XENBUS_FRONTEND
   HVC_XEN_FRONTEND  XEN_XENBUS_FRONTEND
 HVC_XEN
   TCG_XEN   XEN_XENBUS_FRONTEND
   XEN_PCIDEV_FRONTEND   PCI_XEN
 XEN_XENBUS_FRONTEND
   XEN_SCSI_FRONTEND XEN_XENBUS_FRONTEND
   INPUT_XEN_KBDDEV_FRONTEND XEN_XENBUS_FRONTEND
   XEN_NETDEV_FRONTEND   XEN_XENBUS_FRONTEND

There might be some further adjustments needed (should XEN_DEV_EVTCHN
and XEN_GNTDEV belong to XEN_BACKEND?), but these are minor nits. The
main difference to the current status is the XEN setting no longer
controlling all other options.


XEN_DEV_EVTCH and XEN_GNTDEV are useful to non-backends (e.g., for
interdomain comms).


Okay.


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for-4.5] reset PCI devices on force removal even when QEMU returns error

2014-12-15 Thread Stefano Stabellini
On Fri, 12 Dec 2014, Konrad Rzeszutek Wilk wrote:
 On Fri, Dec 12, 2014 at 04:13:52PM +0100, Sander Eikelenboom wrote:
  Hi Konrad,
  
  This doesn't seem to be applied yet, nor does it seem to have a 
  release-(N)ACK 
  from you ?
 
 Hm, Stefano:
 
 - Is this a regression?

I don't think so. Probably a regression compared to the xend toolstack
though.


 - What are the risks of this not going in? I presume that it just means
   we haven't reset it in sysfs. But the xc_deassign_device operation
   if not done will not affect the hypervisor - which will move the
   device to dom0 upon guest teardown.

The device becomes unusable until somebody manually resets it.


  
  --
  Sander
  
  
  
  Friday, November 28, 2014, 5:53:09 PM, you wrote:
  
   On do_pci_remove when QEMU returns error, we just bail out early without
   resetting the device. On domain shutdown we are racing with QEMU exiting
   and most often QEMU closes the QMP connection before executing the
   requested command.
  
   In these cases if force=1, it makes sense to go ahead with rest of the
   PCI device removal, that includes resetting the device and calling
   xc_deassign_device. Otherwise we risk not resetting the device properly
   on domain shutdown.
  
   Signed-off-by: Stefano Stabellini stefano.stabell...@eu.citrix.com
  
   diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c
   index 316643c..0ac0b93 100644
   --- a/tools/libxl/libxl_pci.c
   +++ b/tools/libxl/libxl_pci.c
   @@ -1243,7 +1245,7 @@ static int do_pci_remove(libxl__gc *gc, uint32_t 
   domid,
rc = ERROR_INVAL;
goto out_fail;
}
   -if (rc) {
   +if (rc  !force) {
rc = ERROR_FAIL;
goto out_fail;
}
  
  
 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] One question about the hypercall to translate gfn to mfn.

2014-12-15 Thread Tian, Kevin
 From: Jan Beulich [mailto:jbeul...@suse.com]
 Sent: Monday, December 15, 2014 5:23 PM
 
  On 15.12.14 at 10:05, kevin.t...@intel.com wrote:
  yes, definitely host RAM is the upper limit, and what I'm concerning here
  is how to reserve (at boot time) or allocate (on-demand) such large PFN
  resource, w/o collision with other PFN reservation usage (ballooning
  should be fine since it's operating existing RAM ranges in dom0 e820
  table).
 
 I don't think ballooning is restricted to the regions named RAM in
 Dom0's E820 table (at least it shouldn't be, and wasn't in the
 classic Xen kernels).

well, nice to know that.

 
  Maybe we can reserve a big-enough reserved region in dom0's
  e820 table at boot time, for all PFN reservation usages, and then allocate
  them on-demand for specific usages?
 
 What would big enough here mean (i.e. how would one determine
 the needed size up front)? Plus any form of allocation would need a
 reasonable approach to avoid fragmentation. And anyway I'm not
 getting what position you're on: Do you expect to be able to fit
 everything that needs mapping into the available mapping space (as
 your reply above seems to imply) or do you think there won't be
 enough mapping space (as earlier replies of yours appeared to
 indicate)?
 

I expect to have everything mapped into the available mapping space,
and is asking for suggestions what's the best way to find and reserve
available PFNs in a way not conflicting with other usages (either
virtualization features like ballooning that you mentioned, or bare 
metal features like PCI hotplug or memory hotplug).

Tanks
Kevin

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Lots of ACPI warnings from newer iasl (20140926).

2014-12-15 Thread Jan Beulich
 On 15.12.14 at 11:44, ian.campb...@citrix.com wrote:
 isal 20140926-1 in Debian Jessie is rather vocal about
 tools/firmware/hvmloader/acpi. See below. Essentially, a few of these:
 Control Method should be made Serialized (due to creation of
 named objects within)
 a slew of:
 Object is not referenced ^  (Name is within method [_CRS])
 and a load of:
 ^ Reserved method should not return a value (_L02)
 
 4.4 seems to produce a similar looking set of warnings, so I think this
 is a case of newer iasl having more warnings rather than a regression in
 our code.
 
 I also tried iasl from 20141107-1 Sid and it is similarly vocal.

I had noticed some of these too, and was intending to address them
eventually. As usual, other things continued to take higher priority.
But the lack of serialization of some of the methods I didn't notice so
far, and those seem more relevant to fix than the others. I'll see to
get to this.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] x86/AMD-ucode: correct multiple container handling

2014-12-15 Thread Andrew Cooper
On 15/12/14 09:16, Jan Beulich wrote:
 Avoid emitting an error message referring to an incorrect or corrupt
 container file just because no entry was found for the running CPU.

 Additionally switch the order of data validation and consumption in
 cpu_request_microcode()'s first loop, and also check the types of
 skipped blocks in container_fast_forward().

 Signed-off-by: Jan Beulich jbeul...@suse.com

Reviewed-by: Andrew Cooper andrew.coop...@citrix.com


 --- a/xen/arch/x86/microcode_amd.c
 +++ b/xen/arch/x86/microcode_amd.c
 @@ -331,12 +331,17 @@ static int container_fast_forward(const 
   header[1] == UCODE_EQUIV_CPU_TABLE_TYPE )
  break;
  
 +if ( header[0] != UCODE_UCODE_TYPE )
 +return -EINVAL;
  size = header[1] + SECTION_HDR_SIZE;
  if ( size  PATCH_HDR_SIZE || size_left  size )
  return -EINVAL;
  
  size_left -= size;
  *offset += size;
 +
 +if ( !size_left )
 +return -ENODATA;
  }
  
  return 0;
 @@ -386,10 +391,6 @@ static int cpu_request_microcode(int cpu
  break;
  }
  
 -if ( find_equiv_cpu_id(mc_amd-equiv_cpu_table, current_cpu_id,
 -   equiv_cpu_id) )
 -break;
 -
  /*
   * Could happen as we advance 'offset' early
   * in install_equiv_cpu_table
 @@ -401,7 +402,16 @@ static int cpu_request_microcode(int cpu
  break;
  }
  
 +if ( find_equiv_cpu_id(mc_amd-equiv_cpu_table, current_cpu_id,
 +   equiv_cpu_id) )
 +break;
 +
  error = container_fast_forward(buf, bufsize - offset, offset);
 +if ( error == -ENODATA )
 +{
 +ASSERT(offset == bufsize);
 +break;
 +}
  if ( error )
  {
  printk(KERN_ERR microcode: CPU%d incorrect or corrupt container 
 file\n





 ___
 Xen-devel mailing list
 Xen-devel@lists.xen.org
 http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 1/4] xen: build infrastructure for generating hypercall depending symbols

2014-12-15 Thread David Vrabel
On 11/12/14 18:04, Juergen Gross wrote:
 Today there are several places in the kernel which build tables
 containing one entry for each possible Xen hypercall. Create an
 infrastructure to be able to generate these tables at build time.

Does arm and arm64 need something similar?  If so are the tools here
suitable for them?

 --- a/arch/x86/syscalls/Makefile
 +++ b/arch/x86/syscalls/Makefile

Why are these changes here and not in arch/x86/xen/Makefile?

 @@ -19,6 +19,9 @@ quiet_cmd_syshdr = SYSHDR  $@
  quiet_cmd_systbl = SYSTBL  $@
cmd_systbl = $(CONFIG_SHELL) '$(systbl)' $ $@
  
 +quiet_cmd_hypercalls = HYPERCALLS $@
 +  cmd_hypercalls = $(CONFIG_SHELL) '$' $@ $(filter-out $,$^)
 +
  syshdr_abi_unistd_32 := i386
  $(uapi)/unistd_32.h: $(syscall32) $(syshdr)
   $(call if_changed,syshdr)
 @@ -47,10 +50,16 @@ $(out)/syscalls_32.h: $(syscall32) $(systbl)
  $(out)/syscalls_64.h: $(syscall64) $(systbl)
   $(call if_changed,systbl)
  
 +$(out)/xen-hypercalls.h: $(srctree)/scripts/xen-hypercalls.sh
 + $(call if_changed,hypercalls)
 +
 +$(out)/xen-hypercalls.h: $(srctree)/include/xen/interface/xen*.h

The generated header should end up in asm/xen/

 --- /dev/null
 +++ b/scripts/xen-hypercalls.sh
 @@ -0,0 +1,11 @@
 +#!/bin/sh
 +out=$1
 +shift
 +in=$@
 +
 +for i in $in; do
 + eval $CPP $LINUXINCLUDE -dD -imacros $i -x c /dev/null
 +done | \
 +awk '$1 == #define  $2 ~ /__HYPERVISOR_[a-z][a-z_0-9]*/ { v[$3] = $2 }
 + END {for (i in v) if (!(v[i] in v))
 + print HYPERCALL(substr(v[i], 14))}' | sort -u $out

Include a comment in the generated output saying what generated it. e.g.,

/* auto-generated by scripts/xen-hypercall.sh */

David


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 2/4] xen: synchronize include/xen/interface/xen.h with xen

2014-12-15 Thread David Vrabel
On 11/12/14 18:04, Juergen Gross wrote:
 The header include/xen/interface/xen.h doesn't contain all definitions
 from Xen's version of that header. Update it accordingly.

Reviewed-by: David Vrabel david.vra...@citrix.com

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 3/4] xen: use generated hypervisor symbols in arch/x86/xen/trace.c

2014-12-15 Thread David Vrabel
On 11/12/14 18:04, Juergen Gross wrote:
 Instead of manually list all hypervisor calls in arch/x86/xen/trace.c
 use the auto generated list.

Reviewed-by: David Vrabel david.vra...@citrix.com

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 1/4] xen: build infrastructure for generating hypercall depending symbols

2014-12-15 Thread Jan Beulich
 On 15.12.14 at 12:38, david.vra...@citrix.com wrote:
 On 11/12/14 18:04, Juergen Gross wrote:
 --- a/arch/x86/syscalls/Makefile
 +++ b/arch/x86/syscalls/Makefile
 
 Why are these changes here and not in arch/x86/xen/Makefile?

Because this needs to be done in a step that (afaict) has no hook
in the Xen-specific Makefile.

 @@ -47,10 +50,16 @@ $(out)/syscalls_32.h: $(syscall32) $(systbl)
  $(out)/syscalls_64.h: $(syscall64) $(systbl)
  $(call if_changed,systbl)
  
 +$(out)/xen-hypercalls.h: $(srctree)/scripts/xen-hypercalls.sh
 +$(call if_changed,hypercalls)
 +
 +$(out)/xen-hypercalls.h: $(srctree)/include/xen/interface/xen*.h
 
 The generated header should end up in asm/xen/

Why is generated/asm/ not good enough?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v1] xen/arm: Manage pl011 uart TX interrupt correctly

2014-12-15 Thread Julien Grall
On 11/12/14 12:24, Ian Campbell wrote:
 These last two changes require that you cc the common serial maintainer,
 not just the ARM maintainers. In this case that means Keir, who I have
 CCd.

 ./scripts/get_maintainers.pl can help automate this.

The serial code lives in the THE REST category. So I don't think it's
mandatory to have Keir's ack on this patch.

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] xenstored crashes with SIGSEGV

2014-12-15 Thread Ian Campbell
On Fri, 2014-12-12 at 17:58 +, Ian Campbell wrote:
 (adding Ian J who knows a bit more about C xenstored than me...)
 
  On Fri, 2014-12-12 at 18:20 +0100, Philipp Hahn wrote:
  Hello Ian,
  
  On 12.12.2014 17:56, Ian Campbell wrote:
   On Fri, 2014-12-12 at 17:45 +0100, Philipp Hahn wrote:
   On 12.12.2014 17:32, Ian Campbell wrote:
   On Fri, 2014-12-12 at 17:14 +0100, Philipp Hahn wrote:
   We did enable tracing and now have the xenstored-trace.log of one 
   crash:
   It contains 1.6 billion lines and is 83 GiB.
   It just shows xenstored to crash on TRANSACTION_START.
  
   Is there some tool to feed that trace back into a newly launched 
   xenstored?
  
   Not that I know of I'm afraid.
  
   Okay, then I have to continue with my own tool.
   
   If you do end up developing a tool to replay a xenstore trace then I
   think that'd be something great to have in tree!
  
  I just need to figure out how to talk to xenstored on the wire: for some
  strange reason xenstored is closing the connection to the UNIX socket on
  the first write inside a transaction.
  Or switch to /usr/share/pyshared/xen/xend/xenstore/xstransact.py...
  
   Do you get a core dump when this happens? You might need to fiddle with
   ulimits (some distros disable by default). IIRC there is also some /proc
   nob which controls where core dumps go on the filesystem.
  
   Not for that specific trace: We first enabled generating core files, but
   only then discovered that this is not enough.
   
   How wasn't it enough? You mean you couldn't use gdb to extract a
   backtrace from the core file? Or was something else wrong?
  
  The 1st and 2nd trace look like this: ptr in frame #2 looks very bogus.
  
  (gdb) bt full
  #0  talloc_chunk_from_ptr (ptr=0xff) at talloc.c:116
  tc = value optimized out
  #1  0x00407edf in talloc_free (ptr=0xff) at talloc.c:551
  tc = value optimized out
  #2  0x0040a348 in tdb_open_ex (name=0x1941fb0
  /var/lib/xenstored/tdb.0x1935bb0,
 
 I've timed out for tonight will try and have another look next week.

I've had another dig, and have instrumented all of the error paths from
this function and I can't see any way for an invalid pointer to be
produced, let alone freed. I've been running under valgrind which should
have caught any uninitialised memory type errors.

  hash_size=value optimized out, tdb_flags=0, open_flags=value
  optimized out, mode=value optimized out,
  log_fn=0x4093b0 null_log_fn, hash_fn=value optimized out) at
  tdb.c:1958

Please can you confirm what is at line 1958 of your copy of tdb.c. I
think it will be tdb-locked, but I'd like to be sure.

You are running a 64-bit dom0, correct? I've only just noticed that
0xff is 32bits. My testing so far was 32-bit, I don't think it
should matter wrt use of uninitialised data etc.

I can't help feeling that 0xff must be some sort of magic
sentinel value to someone. I can't figure out what though.

Have you observed the xenstored processes growing especially large
before this happens? I'm wondering if there might be a leak somewhere
which after a time is resulting a 

I'm about to send out a patch which plumbs tdb's logging into
xenstored's logging, in the hopes that next time you see this it might
say something as it dies.

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH for-4.6] xenstored: log tdb message via xenstored's logging mechanisms

2014-12-15 Thread Ian Campbell
TDB provides us with a callback for this purpose. Use it in both
xenstored and xs_tdb_dump.

While at it make the existing log() macro tollerate memory failures.

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
 tools/xenstore/xenstored_core.c |   39 +--
 tools/xenstore/xs_tdb_dump.c|   12 +++-
 2 files changed, 44 insertions(+), 7 deletions(-)

diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c
index 4eaff57..3fd9a20 100644
--- a/tools/xenstore/xenstored_core.c
+++ b/tools/xenstore/xenstored_core.c
@@ -89,9 +89,14 @@ static void check_store(void);
 #define log(...)   \
do {\
char *s = talloc_asprintf(NULL, __VA_ARGS__);   \
-   trace(%s\n, s);   \
-   syslog(LOG_ERR, %s,  s);  \
-   talloc_free(s); \
+   if (s) {\
+   trace(%s\n, s);   \
+   syslog(LOG_ERR, %s,  s);  \
+   talloc_free(s); \
+   } else {\
+   trace(talloc failure during logging\n);   \
+   syslog(LOG_ERR, talloc failure during logging\n); \
+   }   \
} while (0)
 
 
@@ -1479,13 +1484,35 @@ static void manual_node(const char *name, const char 
*child)
talloc_free(node);
 }
 
+static void tdb_logger(TDB_CONTEXT *tdb, int level, const char * fmt, ...)
+{
+   va_list ap;
+   char *s;
+
+   va_start(ap, fmt);
+   s = talloc_vasprintf(NULL, fmt, ap);
+   va_end(ap);
+
+   if (s) {
+   trace(TDB: %s\n, s);
+   syslog(LOG_ERR, TDB: %s,  s);
+   if (verbose)
+   xprintf(TDB: %s, s);
+   talloc_free(s);
+   } else {
+   trace(talloc failure during logging\n);
+   syslog(LOG_ERR, talloc failure during logging\n);
+   }
+}
+
 static void setup_structure(void)
 {
char *tdbname;
tdbname = talloc_strdup(talloc_autofree_context(), xs_daemon_tdb());
 
if (!(tdb_flags  TDB_INTERNAL))
-   tdb_ctx = tdb_open(tdbname, 0, tdb_flags, O_RDWR, 0);
+   tdb_ctx = tdb_open_ex(tdbname, 0, tdb_flags, O_RDWR, 0,
+ tdb_logger, NULL);
 
if (tdb_ctx) {
/* XXX When we make xenstored able to restart, this will have
@@ -1516,8 +1543,8 @@ static void setup_structure(void)
talloc_free(tlocal);
}
else {
-   tdb_ctx = tdb_open(tdbname, 7919, tdb_flags, O_RDWR|O_CREAT,
-  0640);
+   tdb_ctx = tdb_open_ex(tdbname, 7919, tdb_flags, O_RDWR|O_CREAT,
+ 0640, tdb_logger, NULL);
if (!tdb_ctx)
barf_perror(Could not create tdb file %s, tdbname);
 
diff --git a/tools/xenstore/xs_tdb_dump.c b/tools/xenstore/xs_tdb_dump.c
index b91cdef..7a034c0 100644
--- a/tools/xenstore/xs_tdb_dump.c
+++ b/tools/xenstore/xs_tdb_dump.c
@@ -33,6 +33,15 @@ static char perm_to_char(enum xs_perm_type perm)
'?';
 }
 
+static void tdb_logger(TDB_CONTEXT *tdb, int level, const char * fmt, ...)
+{
+   va_list ap;
+
+   va_start(ap, fmt);
+   vfprintf(stderr, fmt, ap);
+   va_end(ap);
+}
+
 int main(int argc, char *argv[])
 {
TDB_DATA key;
@@ -41,7 +50,8 @@ int main(int argc, char *argv[])
if (argc != 2)
barf(Usage: xs_tdb_dump tdbfile);
 
-   tdb = tdb_open(talloc_strdup(NULL, argv[1]), 0, 0, O_RDONLY, 0);
+   tdb = tdb_open_ex(talloc_strdup(NULL, argv[1]), 0, 0, O_RDONLY, 0,
+ tdb_logger, NULL);
if (!tdb)
barf_perror(Could not open %s, argv[1]);
 
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [libvirt] [PATCH] xenconfig: fix boot device parsing

2014-12-15 Thread John Ferlan


On 12/14/2014 09:58 AM, Wei Liu wrote:
 The original code always checked *boot which was in effect boot[0]. It
 should use boot[i].

In the future, when you do the investigation, just reference the commit
id that caused the regression.  In this case it seems to be commit id
'547bd71a' (7/25/08 - quite a while ago)!

I'll work on getting this pushed shortly -

John
 
 Signed-off-by: Wei Liu wei.l...@citrix.com
 ---
  src/xenconfig/xen_common.c |2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
 
 diff --git a/src/xenconfig/xen_common.c b/src/xenconfig/xen_common.c
 index 7f4ec89..48431a7 100644
 --- a/src/xenconfig/xen_common.c
 +++ b/src/xenconfig/xen_common.c
 @@ -1071,7 +1071,7 @@ xenParseOS(virConfPtr conf, virDomainDefPtr def)
  return -1;
  
  for (i = 0; i  VIR_DOMAIN_BOOT_LAST  boot[i]; i++) {
 -switch (*boot) {
 +switch (boot[i]) {
  case 'a':
  def-os.bootDevs[i] = VIR_DOMAIN_BOOT_FLOPPY;
  break;
 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Porting document

2014-12-15 Thread Frediano Ziglio
2014-12-12 16:01 GMT+00:00 Ian Campbell ian.campb...@citrix.com:
 On Fri, 2014-12-12 at 15:59 +, Ian Campbell wrote:
 On Mon, 2014-12-08 at 15:47 +, Frediano Ziglio wrote:
  Hi,
while I was porting D01 platform
  (https://wiki.linaro.org/Boards/D01) to Xen I wrote a small document
  trying to describe the step I made and problems I encountered hoping
  it could useful to other people. The idea is to put such documentation
  in a wiki page or in the docs directory.
 
  Let me know what do you think.

 I've only skimmed the headings etc but I see no harm in posting it
 somewhere publicly. I don't know if wiki or in tree docs would be
 better. Maybe just post it on the list? At least then it will be in the
 archives.

The problem with ML is that is not that live, content is not easy changeable.

Frediano

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] xenstored crashes with SIGSEGV

2014-12-15 Thread Philipp Hahn
Hello Ian,

On 15.12.2014 14:17, Ian Campbell wrote:
 On Fri, 2014-12-12 at 17:58 +, Ian Campbell wrote:
  On Fri, 2014-12-12 at 18:20 +0100, Philipp Hahn wrote:
 On 12.12.2014 17:56, Ian Campbell wrote:
 On Fri, 2014-12-12 at 17:45 +0100, Philipp Hahn wrote:
 On 12.12.2014 17:32, Ian Campbell wrote:
 On Fri, 2014-12-12 at 17:14 +0100, Philipp Hahn wrote:
...
 The 1st and 2nd trace look like this: ptr in frame #2 looks very bogus.

 (gdb) bt full
 #0  talloc_chunk_from_ptr (ptr=0xff) at talloc.c:116
 tc = value optimized out
 #1  0x00407edf in talloc_free (ptr=0xff) at talloc.c:551
 tc = value optimized out
 #2  0x0040a348 in tdb_open_ex (name=0x1941fb0
 /var/lib/xenstored/tdb.0x1935bb0,

I just noticed something strange:

 #3  0x0040a684 in tdb_open (name=0xff Address
 0xff out of bounds, hash_size=0,
 tdb_flags=4254928, open_flags=-1, mode=3119127560) at tdb.c:1773
 #4  0x0040a70b in tdb_copy (tdb=0x192e540, outfile=0x1941fb0
 /var/lib/xenstored/tdb.0x1935bb0)

Why does gdb-7.0.1 print name=0xff00 here for frame 3, but for
frame 2 and 4 the pointers are correct again?
Verifying the values with an explicit print shows them as correct.

 I've timed out for tonight will try and have another look next week.
 
 I've had another dig, and have instrumented all of the error paths from
 this function and I can't see any way for an invalid pointer to be
 produced, let alone freed. I've been running under valgrind which should
 have caught any uninitialised memory type errors.

Thank you for testing that.

 hash_size=value optimized out, tdb_flags=0, open_flags=value
 optimized out, mode=value optimized out,
 log_fn=0x4093b0 null_log_fn, hash_fn=value optimized out) at
 tdb.c:1958
 
 Please can you confirm what is at line 1958 of your copy of tdb.c. I
 think it will be tdb-locked, but I'd like to be sure.

Yes, that's the line:
# sed -ne 1958p tdb.c
SAFE_FREE(tdb-locked);

 You are running a 64-bit dom0, correct?

yes: x86_64

 I've only just noticed that
 0xff is 32bits. My testing so far was 32-bit, I don't think it
 should matter wrt use of uninitialised data etc.
 
 I can't help feeling that 0xff must be some sort of magic
 sentinel value to someone. I can't figure out what though.

0xff is too much for bit flip errors. and also two crashes on different
machines in the same location very much rules out any HW error for me.

My 2nd idea was that someone decremented 0 one too many, but then that
would have to be an 8 bit value - reading the code I didn't see anything
like that.

 Have you observed the xenstored processes growing especially large
 before this happens? I'm wondering if there might be a leak somewhere
 which after a time is resulting a 

I have no monitoring of the memory usage for the crashed systems, but
the core files look reasonable sane.
Looking at the test-system running
/usr/share/pyshared/xen/xend/xenstore/tests/stress_xs.py the memory
usage stays constant since last Friday.

 I'm about to send out a patch which plumbs tdb's logging into
 xenstored's logging, in the hopes that next time you see this it might
 say something as it dies.

Thank you for the patch: I'll try to incorporate it and will continue
trying to reproduce the crash.


One more thing we noticed: /var/lib/xenstored/ contained the tdb file
and to bit-identical copies after the crash, so I would read that as two
transactions being in progress at the time of the crash. Might be that
this is important.
But /usr/share/pyshared/xen/xend/xenstore/tests/stress_xs.py seems to
create more transaction in parallel and my test system so far has
survived this since Friday.

Sincerely
Philipp Hahn

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] xenstored crashes with SIGSEGV

2014-12-15 Thread Ian Campbell
On Mon, 2014-12-15 at 15:19 +0100, Philipp Hahn wrote:
 Hello Ian,
 
 On 15.12.2014 14:17, Ian Campbell wrote:
  On Fri, 2014-12-12 at 17:58 +, Ian Campbell wrote:
   On Fri, 2014-12-12 at 18:20 +0100, Philipp Hahn wrote:
  On 12.12.2014 17:56, Ian Campbell wrote:
  On Fri, 2014-12-12 at 17:45 +0100, Philipp Hahn wrote:
  On 12.12.2014 17:32, Ian Campbell wrote:
  On Fri, 2014-12-12 at 17:14 +0100, Philipp Hahn wrote:
 ...
  The 1st and 2nd trace look like this: ptr in frame #2 looks very bogus.
 
  (gdb) bt full
  #0  talloc_chunk_from_ptr (ptr=0xff) at talloc.c:116
  tc = value optimized out
  #1  0x00407edf in talloc_free (ptr=0xff) at talloc.c:551
  tc = value optimized out
  #2  0x0040a348 in tdb_open_ex (name=0x1941fb0
  /var/lib/xenstored/tdb.0x1935bb0,
 
 I just noticed something strange:
 
  #3  0x0040a684 in tdb_open (name=0xff Address
  0xff out of bounds, hash_size=0,
  tdb_flags=4254928, open_flags=-1, mode=3119127560) at tdb.c:1773
  #4  0x0040a70b in tdb_copy (tdb=0x192e540, outfile=0x1941fb0
  /var/lib/xenstored/tdb.0x1935bb0)
 
 Why does gdb-7.0.1 print name=0xff00 here for frame 3, but for
 frame 2 and 4 the pointers are correct again?
 Verifying the values with an explicit print shows them as correct.

I has just noticed that and was wondering about that same thing. I'm
starting to worry that 0xff might just be a gdb thing, similar
to value optimized out, but infinitely more misleading.

I've also noticed in
https://forge.univention.org/bugzilla/show_bug.cgi?id=35104 that the
constant can be either 0xff00, 0xff or 0xff00 (6, 8
or 10 zeroes).

  hash_size=value optimized out, tdb_flags=0, open_flags=value
  optimized out, mode=value optimized out,
  log_fn=0x4093b0 null_log_fn, hash_fn=value optimized out) at
  tdb.c:1958
  
  Please can you confirm what is at line 1958 of your copy of tdb.c. I
  think it will be tdb-locked, but I'd like to be sure.
 
 Yes, that's the line:
 # sed -ne 1958p tdb.c
 SAFE_FREE(tdb-locked);

Good, thanks.

  You are running a 64-bit dom0, correct?
 
 yes: x86_64

Thanks for confirming. I'm resurrecting the 64-bit root partition on my
test box (which it turns out was still Debian Squeeze!)

 
  I've only just noticed that
  0xff is 32bits. My testing so far was 32-bit, I don't think it
  should matter wrt use of uninitialised data etc.
  
  I can't help feeling that 0xff must be some sort of magic
  sentinel value to someone. I can't figure out what though.
 
 0xff is too much for bit flip errors. and also two crashes on different
 machines in the same location very much rules out any HW error for me.
 
 My 2nd idea was that someone decremented 0 one too many, but then that
 would have to be an 8 bit value - reading the code I didn't see anything
 like that.

I was wondering if it was an overflow or sign-extension thing, but it
doesn't seem likely, not enough high bits set for one thing.

 One more thing we noticed: /var/lib/xenstored/ contained the tdb file
 and to bit-identical copies after the crash, so I would read that as two
 transactions being in progress at the time of the crash. Might be that
 this is important.

It's certainly worth noting, thanks.

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Xen's Linux kernel config options

2014-12-15 Thread Stefano Stabellini
On Fri, 12 Dec 2014, David Vrabel wrote:
 On 12/12/14 13:17, Juergen Gross wrote:
  
  
  As a first draft I'd suggest the following:
 
 Some rough thoughts below.
 
  Option  Selects Depends
  --
  XEN SWIOTLB_XEN(arm,arm64)
 
 XEN should get you basic support for making hypercalls and allowing
 guest to identify themselves as running on Xen.  I don't think it should
 select SWIOTLB_XEN even on ARM as it is only needed for guests with
 access to real hardware.

In that case XEN_BACKEND should select SWIOTLB_XEN.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for-4.6 2/4] xen/arm: vgic: Keep track of vIRQ used by a domain

2014-12-15 Thread Stefano Stabellini
On Fri, 12 Dec 2014, Julien Grall wrote:
 While it's easy to know which hardware IRQ is assigned to a domain, there
 is no way to know which IRQ is emulated by Xen for a specific domain.
 
 Introduce a bitmap to keep track of every vIRQ used by a domain. This
 will be used later to find free vIRQ for interrupt device assignment and
 emulated interrupt.
 
 Signed-off-by: Julien Grall julien.gr...@linaro.org
 ---
  xen/arch/arm/domain_build.c  |  6 +++
  xen/arch/arm/platforms/xgene-storm.c |  4 ++
  xen/arch/arm/vgic.c  | 76 
 
  xen/arch/arm/vtimer.c| 15 +++
  xen/include/asm-arm/domain.h |  1 +
  xen/include/asm-arm/vgic.h   | 13 ++
  6 files changed, 115 insertions(+)
 
 diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
 index de180d8..c238c8f 100644
 --- a/xen/arch/arm/domain_build.c
 +++ b/xen/arch/arm/domain_build.c
 @@ -968,6 +968,12 @@ static int map_device(struct domain *d, struct 
 dt_device_node *dev)
  irq = res;
  
  DPRINT(irq %u = %u\n, i, irq);
 +/*
 + * Checking the return of vgic_reserve_virq is not
 + * necessary. It should not fail except when we try to map
 + * twice the IRQ. This can happen if the IRQ is shared
 + */
 +vgic_reserve_virq(d, irq);
  res = route_irq_to_guest(d, irq, dt_node_name(dev));
  if ( res )
  {
 diff --git a/xen/arch/arm/platforms/xgene-storm.c 
 b/xen/arch/arm/platforms/xgene-storm.c
 index 0b3492d..416d42c 100644
 --- a/xen/arch/arm/platforms/xgene-storm.c
 +++ b/xen/arch/arm/platforms/xgene-storm.c
 @@ -71,6 +71,10 @@ static int map_one_spi(struct domain *d, const char *what,
  
  printk(Additional IRQ %u (%s)\n, irq, what);
  
 +if ( !vgic_reserve_virq(d, irq) )
 +printk(Failed to reserve the vIRQ %u on dom%d\n,
 +   irq, d-domain_id);
 +
  ret = route_irq_to_guest(d, irq, what);
  if ( ret )
  printk(Failed to route %s to dom%d\n, what, d-domain_id);
 diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
 index 75cb7ff..dbfc259 100644
 --- a/xen/arch/arm/vgic.c
 +++ b/xen/arch/arm/vgic.c
 @@ -87,6 +87,8 @@ int domain_vgic_init(struct domain *d)
  return -ENODEV;
  }
  
 +spin_lock_init(d-arch.vgic.lock);

you should probably explain in the commit message the reason why you are
making changes to the vgic lock


  d-arch.vgic.shared_irqs =
  xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_RANKS(d));
  if ( d-arch.vgic.shared_irqs == NULL )
 @@ -107,6 +109,15 @@ int domain_vgic_init(struct domain *d)
  
  d-arch.vgic.handler-domain_init(d);
  
 +d-arch.vgic.allocated_irqs =
 +xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d)));
 +if ( !d-arch.vgic.allocated_irqs )
 +return -ENOMEM;
 +
 +/* vIRQ0-15 (SGIs) are reserved */
 +for (i = 0; i = 15; i++)
 +set_bit(i, d-arch.vgic.allocated_irqs);
 +
  return 0;
  }
  
 @@ -119,6 +130,7 @@ void domain_vgic_free(struct domain *d)
  {
  xfree(d-arch.vgic.shared_irqs);
  xfree(d-arch.vgic.pending_irqs);
 +xfree(d-arch.vgic.allocated_irqs);
  }
  
  int vcpu_vgic_init(struct vcpu *v)
 @@ -441,6 +453,70 @@ int vgic_emulate(struct cpu_user_regs *regs, union hsr 
 hsr)
  return v-domain-arch.vgic.handler-emulate_sysreg(regs, hsr);
  }
  
 +bool_t vgic_reserve_virq(struct domain *d, unsigned int virq)
 +{
 +bool_t reserved;
 +
 +if ( virq = vgic_num_irqs(d) )
 +return 0;
 +
 +spin_lock(d-arch.vgic.lock);
 +reserved = !test_and_set_bit(virq, d-arch.vgic.allocated_irqs);
 +spin_unlock(d-arch.vgic.lock);

test_and_set_bit is atomic, why do you need to take the lock?


 +return reserved;
 +}
 +
 +int vgic_allocate_virq(struct domain *d, bool_t spi)
 +{
 +int ret = -1;
 +unsigned int virq;
 +
 +spin_lock(d-arch.vgic.lock);
 +if ( !spi )
 +{
 +virq = find_first_zero_bit(d-arch.vgic.allocated_irqs, 32);
 +if ( virq = 32 )
 +goto unlock;
 +}
 +else
 +{
 +virq = find_next_zero_bit(d-arch.vgic.allocated_irqs,
 +  32, vgic_num_irqs(d));
 +if ( virq = vgic_num_irqs(d) )
 +goto unlock;
 +}
 +
 +set_bit(virq, d-arch.vgic.allocated_irqs);
 +ret = virq;
 +
 +unlock:
 +spin_unlock(d-arch.vgic.lock);

you might be able to write this function without taking the lock too, by
using test_and_set_bit and retries:

retry:
virq = find_first_zero_bit;
if (test_and_set_bit(virq))
goto retry;



 +return ret;
 +}
 +
 +void vgic_free_virq(struct domain *d, unsigned int virq)
 +{
 +unsigned int spi;
 +
 +if ( is_hardware_domain(d) )
 +return;
 +
 +if ( virq  32  virq = vgic_num_irqs(d) )
 +return;
 +
 +spi = virq - 32;
 +
 +/* Taking the vGIC domain lock is not necessary. We don't care 

[Xen-devel] [xen-unstable test] 32361: tolerable FAIL - PUSHED

2014-12-15 Thread xen . org
flight 32361 xen-unstable real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/32361/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-sedf-pin  5 xen-bootfail pass in 32316
 test-amd64-i386-rumpuserxen-i386 3 host-install(3) broken in 32316 pass in 
32361
 test-amd64-i386-qemut-rhel6hvm-intel 3 host-install(3) broken in 32316 pass in 
32361
 test-amd64-amd64-xl   3 host-install(3)  broken in 32316 pass in 32361
 test-amd64-i386-xl3 host-install(3)  broken in 32316 pass in 32361
 test-amd64-i386-freebsd10-i386 3 host-install(3) broken in 32316 pass in 32361
 test-amd64-i386-xl-qemuu-debianhvm-amd64 3 host-install(3) broken in 32316 
pass in 32361
 test-amd64-i386-xl-qemuu-ovmf-amd64 3 host-install(3) broken in 32316 pass in 
32361
 test-amd64-i386-pair  4 host-install/dst_host(4) broken in 32316 pass in 32361

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt  9 guest-start  fail   never pass
 test-armhf-armhf-xl  10 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt   9 guest-start  fail   never pass
 test-amd64-amd64-xl-pcipt-intel  9 guest-start fail never pass
 test-armhf-armhf-libvirt  9 guest-start  fail   never pass
 test-amd64-i386-xl-qemut-winxpsp3 14 guest-stopfail never pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 14 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-winxpsp3 14 guest-stopfail never pass
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1 14 guest-stop fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 14 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 14 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 14 guest-stop fail never pass
 test-amd64-i386-xl-win7-amd64 14 guest-stop   fail  never pass
 test-amd64-amd64-xl-win7-amd64 14 guest-stop   fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 14 guest-stop  fail never pass
 test-amd64-i386-xl-winxpsp3-vcpus1 14 guest-stop   fail never pass
 test-amd64-amd64-xl-winxpsp3 14 guest-stop   fail   never pass
 test-amd64-amd64-xl-qemut-winxpsp3 14 guest-stop   fail never pass
 test-amd64-i386-xl-winxpsp3  14 guest-stop   fail   never pass
 test-amd64-amd64-xl-qemuu-winxpsp3 14 guest-stop   fail never pass

version targeted for testing:
 xen  7e73a6e7f12ae080222c5d339799905de3443a6e
baseline version:
 xen  60ce518a1b1caf2c1e4c1b203e87fb0b179ba687


People who touched revisions under test:
  David Vrabel david.vra...@citrix.com
  Ian Campbell ian.campb...@citrix.com
  Jan Beulich jbeul...@suse.com
  Kevin Tian kevin.t...@intel.com
  Mukesh Rathor mukesh.rat...@oracle.com
  Roger Pau Monné roger@citrix.com
  Tim Deegan t...@xen.org


jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-oldkern  pass
 build-i386-oldkern   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 build-amd64-rumpuserxen  pass
 build-i386-rumpuserxen   pass
 test-amd64-amd64-xl  pass
 test-armhf-armhf-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-i386-rhel6hvm-amd pass
 test-amd64-i386-qemut-rhel6hvm-amd   pass
 test-amd64-i386-qemuu-rhel6hvm-amd   pass
 test-amd64-amd64-xl-qemut-debianhvm-amd64pass
 test-amd64-i386-xl-qemut-debianhvm-amd64 pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
 test-amd64-i386-freebsd10-amd64  pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass
 test-amd64-amd64-rumpuserxen-amd64   pass
 

Re: [Xen-devel] [PATCH 11/12] vTPM/TPM2: Bind group keys and sectors data on disk

2014-12-15 Thread Daniel De Graaf

On 12/14/2014 07:09 AM, Quan Xu wrote:
[...]

+/*TPM 2.0 bind | TPM 1.x seal*/
+if (hw_is_tpm2()) {
+TPM2_disk_bind(dst, sblob, sizeof(sblob));
+} else {
+dst-pcr_selection = src-seals[i].pcr_selection;
+memcpy(dst-digest_release, src-seals[i].digest_release, 20);
+TPM_pcr_digest(dst-digest_at_seal, dst-pcr_selection);
+TPM_disk_seal(dst, sblob, sizeof(sblob));
+}


It appears that the secrets for the vTPMs are only being bound to the
presence of the physical TPM and not the measurements of the hypervisor
and other TCB components.  This does not provide as much security as it
did for TPM 1.2: an attacker with access to the boot disk can boot into
a compromised environment and extract the vTPM keys and disk images.

The TPM2_Create/TPM2_Unseal operations should be capable of performing
the same functionality.  If only SHA1 PCRs are used, they should be able
to be drop-in replacements, but supporting other hash algorithms may be
a feature that users who have a TPM2 will want.

--
Daniel De Graaf
National Security Agency

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] x86/AMD-ucode: correct multiple container handling

2014-12-15 Thread Aravind Gopalakrishnan

On 12/15/2014 5:29 AM, Andrew Cooper wrote:

On 15/12/14 09:16, Jan Beulich wrote:

Avoid emitting an error message referring to an incorrect or corrupt
container file just because no entry was found for the running CPU.

Additionally switch the order of data validation and consumption in
cpu_request_microcode()'s first loop, and also check the types of
skipped blocks in container_fast_forward().

Signed-off-by: Jan Beulichjbeul...@suse.com


Reviewed-by: Andrew Cooper andrew.coop...@citrix.com




Reviewed-by: Aravind Gopalakrishnan aravind.gopalakrish...@amd.com

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] One question about the hypercall to translate gfn to mfn.

2014-12-15 Thread Jan Beulich
 On 15.12.14 at 16:22, stefano.stabell...@eu.citrix.com wrote:
 On Mon, 15 Dec 2014, Jan Beulich wrote:
  On 15.12.14 at 10:05, kevin.t...@intel.com wrote:
  yes, definitely host RAM is the upper limit, and what I'm concerning here
  is how to reserve (at boot time) or allocate (on-demand) such large PFN
  resource, w/o collision with other PFN reservation usage (ballooning
  should be fine since it's operating existing RAM ranges in dom0 e820
  table).
 
 I don't think ballooning is restricted to the regions named RAM in
 Dom0's E820 table (at least it shouldn't be, and wasn't in the
 classic Xen kernels).
 
 Could you please elaborate more on this? It seems counter-intuitive at best.

I don't see what's counter-intuitive here. How can the hypervisor
(Dom0) or tool stack (DomU) know what ballooning intentions a
guest kernel may have? It's solely the guest kernel's responsibility
to make sure its ballooning activities don't collide with anything
else address-wise.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for-4.6 2/4] xen/arm: vgic: Keep track of vIRQ used by a domain

2014-12-15 Thread Julien Grall
Hi Stefano,

On 15/12/14 15:32, Stefano Stabellini wrote:
 On Fri, 12 Dec 2014, Julien Grall wrote:
 +spin_lock_init(d-arch.vgic.lock);
 
 you should probably explain in the commit message the reason why you are
 making changes to the vgic lock

Actually the domain vgic lock was never used. Only the per-vcpu vgic
lock was in used.

So I don't make any change to the vgic lock. If I don't use it, I will
add a patch to drop it.

 @@ -441,6 +453,70 @@ int vgic_emulate(struct cpu_user_regs *regs, union hsr 
 hsr)
  return v-domain-arch.vgic.handler-emulate_sysreg(regs, hsr);
  }
  
 +bool_t vgic_reserve_virq(struct domain *d, unsigned int virq)
 +{
 +bool_t reserved;
 +
 +if ( virq = vgic_num_irqs(d) )
 +return 0;
 +
 +spin_lock(d-arch.vgic.lock);
 +reserved = !test_and_set_bit(virq, d-arch.vgic.allocated_irqs);
 +spin_unlock(d-arch.vgic.lock);
 
 test_and_set_bit is atomic, why do you need to take the lock?

To avoid race condition with vgic_allocate_virq.

Anyway, I will dropped it with your suggestion to implement
vgic_allocate_virq without lock.

[..]

 +int vgic_allocate_virq(struct domain *d, bool_t spi)
 +{
 +int ret = -1;
 +unsigned int virq;
 +
 +spin_lock(d-arch.vgic.lock);
 +if ( !spi )
 +{
 +virq = find_first_zero_bit(d-arch.vgic.allocated_irqs, 32);
 +if ( virq = 32 )
 +goto unlock;
 +}
 +else
 +{
 +virq = find_next_zero_bit(d-arch.vgic.allocated_irqs,
 +  32, vgic_num_irqs(d));
 +if ( virq = vgic_num_irqs(d) )
 +goto unlock;
 +}
 +
 +set_bit(virq, d-arch.vgic.allocated_irqs);
 +ret = virq;
 +
 +unlock:
 +spin_unlock(d-arch.vgic.lock);
 
 you might be able to write this function without taking the lock too, by
 using test_and_set_bit and retries:
 
 retry:
 virq = find_first_zero_bit;
 if (test_and_set_bit(virq))
 goto retry;

I will give a look to it. I will also to limit the number of retry
(maybe to the number of vIRQ) for safety.

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] One question about the hypercall to translate gfn to mfn.

2014-12-15 Thread Stefano Stabellini
On Mon, 15 Dec 2014, Jan Beulich wrote:
  On 15.12.14 at 16:22, stefano.stabell...@eu.citrix.com wrote:
  On Mon, 15 Dec 2014, Jan Beulich wrote:
   On 15.12.14 at 10:05, kevin.t...@intel.com wrote:
   yes, definitely host RAM is the upper limit, and what I'm concerning here
   is how to reserve (at boot time) or allocate (on-demand) such large PFN
   resource, w/o collision with other PFN reservation usage (ballooning
   should be fine since it's operating existing RAM ranges in dom0 e820
   table).
  
  I don't think ballooning is restricted to the regions named RAM in
  Dom0's E820 table (at least it shouldn't be, and wasn't in the
  classic Xen kernels).
  
  Could you please elaborate more on this? It seems counter-intuitive at best.
 
 I don't see what's counter-intuitive here. How can the hypervisor
 (Dom0) or tool stack (DomU) know what ballooning intentions a
 guest kernel may have?

The hypervisor checks that the memory the guest is giving back is
actually ram, as a consequence the ballooning interface only supports
ram. Do you agree?

Ballooning is restricted to regions named RAM in the e820 table, because
Linux respects e820 in its pfn-mfn mappings. However it is true that
respecting the e820 in dom0 is not part of the interface.


 It's solely the guest kernel's responsibility
 to make sure its ballooning activities don't collide with anything
 else address-wise.

In the sense that it is in the guest kernel's responsibility to use the
interface properly.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 00/12] Enable vTPM subsystem on TPM 2.0

2014-12-15 Thread Daniel De Graaf

On 12/14/2014 07:09 AM, Quan Xu wrote:

This series of patch enable the virtual Trusted Platform Module (vTPM)
subsystem for Xen on TPM 2.0.

Noted, functionality for a virtual guest operating system (a DomU) is still
TPM 1.2. The main modifcation is on vtpmmgr-stubdom. The challenge is that
TPM 2.0 is not backward compatibility with TPM 1.2.



This series looks good.  Other than a minor comment on #9, my only concern is
the use of bind/unbind instead of seal/unseal (mentioned in my reply to #11,
but it really applies to the series as a whole).  Changing the data structures
to support hashes, authdata, and other measurements larger than 20 bytes is
probably best done sooner so that the management commands can be changed before
much infrastructure is built using the existing ones, but it doesn't have to be
done as part of the introduction of TPM2 support.

Due to the holidays, I will not be able to review further patches until January.

--
Daniel De Graaf
National Security Agency

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for 4.5] libxl: Tell qemu to use raw format when using a tapdisk

2014-12-15 Thread George Dunlap
On Wed, Dec 10, 2014 at 4:30 PM, Konrad Rzeszutek Wilk
konrad.w...@oracle.com wrote:
 On Tue, Dec 09, 2014 at 02:48:21PM +, Ian Campbell wrote:
 On Tue, 2014-12-09 at 14:04 +, George Dunlap wrote:
  At the moment libxl unconditinally passes the underlying file format
  to qemu in the device string.  However, when tapdisk is in use,
  tapdisk handles the underlying format and presents qemu with
  effectively a raw disk.  When qemu looks at the tapdisk block device
  and doesn't find the image format it was looking for, it will fail.
 
  This effectively means that tapdisk cannot be used with HVM domains at
  the moment except for raw files.
 
  Instead, if we're using a tapdisk backend, tell qemu to use a raw file
  format.
 
  Signed-off-by: George Dunlap george.dun...@eu.citrix.com

 Acked-by: Ian Campbell ian.campb...@citrix.com

 Release-Acked-by: Konrad Rzeszutek Wilk konrad.w...@oracle.com

This doesn't seem to have been applied yet.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH V2] libxl: Set path to console on domain startup.

2014-12-15 Thread Anthony PERARD
On Thu, Dec 11, 2014 at 04:23:15PM +, Ian Campbell wrote:
 On Thu, 2014-12-11 at 16:16 +, Anthony PERARD wrote:
  On Tue, Dec 09, 2014 at 03:56:02PM +, Ian Campbell wrote:
   On Tue, 2014-12-09 at 15:39 +, Anthony PERARD wrote:
The path to the pty of a Xen PV console is set only in
virDomainOpenConsole. But this is done too late. A call to
virDomainGetXMLDesc done before OpenConsole will not have the path to
the pty, but a call after OpenConsole will.

e.g. of the current issue.
Starting a domain with 'console type=pty/'
Then:
virDomainGetXMLDesc():
  devices
console type='pty'
  target type='xen' port='0'/
/console
  /devices
virDomainOpenConsole()
virDomainGetXMLDesc():
  devices
console type='pty' tty='/dev/pts/30'
  source path='/dev/pts/30'/
  target type='xen' port='0'/
/console
  /devices

The patch intend to have the TTY path on the first call of GetXMLDesc.
This is done by setting up the path at domain start up instead of in
OpenConsole.

https://bugzilla.redhat.com/show_bug.cgi?id=1170743

Signed-off-by: Anthony PERARD anthony.per...@citrix.com

---
Change in V2:
  Adding bug report link.
  Reword the last part of the patch description.
  Cleanup the code.
  Use VIR_FREE before VIR_STRDUP.
  Remove the code from OpenConsole as it is now a duplicate.
---
 src/libxl/libxl_domain.c | 20 
 src/libxl/libxl_driver.c | 15 ---
 2 files changed, 20 insertions(+), 15 deletions(-)

diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c
index 9c62291..325de79 100644
--- a/src/libxl/libxl_domain.c
+++ b/src/libxl/libxl_domain.c
@@ -1290,6 +1290,26 @@ libxlDomainStart(libxlDriverPrivatePtr driver, 
virDomainObjPtr vm,
 if (libxlDomainSetVcpuAffinities(driver, vm)  0)
 goto cleanup_dom;
 
+if (vm-def-nconsoles) {
+virDomainChrDefPtr chr = vm-def-consoles[0];
   
   Given vm-def-nconsoles should we loop and do them all?
  
  Maybe. libvirt it self will not be able to access any console that is
  not the first one (virDomainOpenConsole only provide access to console
  0), but a consumer of libvirt could.
  
   Also, and I really should know the answer to this (and sorry for not
   thinking of it earlier), but:
   
+ret = libxl_console_get_tty(priv-ctx, vm-def-id,
+chr-target.port, console_type,
+console);
   
   Might this race against xenconsoled writing the node to xenstore and
   therefore be prone to failing when starting multiple guests all at once?
  
  I've look through out libxl, xenconsoled and libvirt, and I could not
  find any synchronisation point. So I guest it's possible.
  
   Is there a hook which is called on virsh dumpxml which could update
   things on the fly (i.e. lookup the console iff it isn't already set)?
  
  I guest we could modify libxlDomainGetXMLDesc and libxlDomainOpenConsole
  to do the check, or having a xenstore watch on the path (if that is
  possible).
 
 The aop_console_how option to libxl_domain_create_new and
 libxl_domain_create_restore is documented as:
 
   /* A progress report will be made via ao_console_how, of type
* domain_create_console_available, when the domain's primary
* console is available and can be connected to.
*/
 
 Which sounds like exactly what is needed?

It looks like the progress is reported before the main function finish,
from the description of the type libxl_asyncprogress_how (in libxl.h).

Also, I tryied to use it, it works if xenconsoled is running. But if
xenconsoled is not running, then the callback is also called and
libxl_console_get_tty() return an empty string.

Or, maybe my test case is not relevant, so another question: Will
libxl wait for xenconsoled to process the new domain before calling the
callback?
Or, should I set the callback to NULL and have the
domain_create_console_available event sent through
the callback set by libxl_event_register_callbacks()?

-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [GIT PULL] xen: additional features for 3.19-rc0

2014-12-15 Thread David Vrabel
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Linus,

Please git pull the following tag:

 git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git 
stable/for-linus-3.19-rc0b-tag

xen: additional features for 3.19-rc0

- - Linear p2m for x86 PV guests which simplifies the p2m code, improves
  performance and will allow for  512 GB PV guests in the future.

Thanks.

A last-minute, configuration specific issue was discovered with this change
which is why it was not included in my previous pull request.  This is now been
fixed and tested.

David

 arch/x86/include/asm/pgtable_types.h |1 +
 arch/x86/include/asm/xen/page.h  |   64 +-
 arch/x86/mm/pageattr.c   |   20 +
 arch/x86/xen/mmu.c   |   40 +-
 arch/x86/xen/p2m.c   | 1172 +-
 arch/x86/xen/setup.c |  441 ++---
 arch/x86/xen/xen-ops.h   |6 +-
 7 files changed, 779 insertions(+), 965 deletions(-)

David Vrabel (1):
  Revert swiotlb-xen: pass dev_addr to swiotlb_tbl_unmap_single

Juergen Gross (13):
  xen: fix some style issues in p2m.c
  xen: Make functions static
  xen: use common page allocation function in p2m.c
  xen: Delay remapping memory of pv-domain
  xen: Delay m2p_override initialization
  xen: Delay invalidating extra memory
  x86: Introduce function to get pmd entry pointer
  xen: Hide get_phys_to_machine() to be able to tune common path
  xen: switch to linear virtual mapped sparse p2m list
  xen: Speed up set_phys_to_machine() by using read-only mappings
  xen: introduce helper functions to do safe read and write accesses
  xen: annotate xen_set_identity_and_remap_chunk() with __init
  xen: switch to post-init routines in xen mmu.c earlier
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (GNU/Linux)

iQEcBAEBAgAGBQJUjyK/AAoJEFxbo/MsZsTR2sQIAKpHjcGBVCKFoOGzfMjKGlab
cB/dAoTJD3iqde0ic0CH9hz2ZVHGz2yDA3PSBmuJf4QPMX2uCmE8K0RVZAtzntFj
YPsHWXDrsJBfBKBpi7gnmsNwA1VEVTW+3KYumPVprKOoAgYlVPzI/NdJxW+hucCo
gg8/f74OtEiyq8lOBo1Z7gYjh5nv4lc+1WgsaBGSWYWnn0zma6TGmB3jtBdryIuN
/KnPO4cFfkbx/nFwhlJ/mpMeC/ZI7aTU1u5Y1wAlcK3a9fItJT/WsD5orsGANddQ
rgeOceBCegd6YIQ6Q/CAu4uEqjNFgKz8G+0NyYI327/KaAQDykh8sKYQqUiK1Ew=
=UBAl
-END PGP SIGNATURE-

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] xenstored crashes with SIGSEGV

2014-12-15 Thread Ian Campbell
On Mon, 2014-12-15 at 14:50 +, Ian Campbell wrote:
 On Mon, 2014-12-15 at 15:19 +0100, Philipp Hahn wrote:
  I just noticed something strange:
  
   #3  0x0040a684 in tdb_open (name=0xff Address
   0xff out of bounds, hash_size=0,
   tdb_flags=4254928, open_flags=-1, mode=3119127560) at tdb.c:1773
   #4  0x0040a70b in tdb_copy (tdb=0x192e540, outfile=0x1941fb0
   /var/lib/xenstored/tdb.0x1935bb0)
  
  Why does gdb-7.0.1 print name=0xff00 here for frame 3, but for
  frame 2 and 4 the pointers are correct again?
  Verifying the values with an explicit print shows them as correct.
 
 I has just noticed that and was wondering about that same thing. I'm
 starting to worry that 0xff might just be a gdb thing, similar
 to value optimized out, but infinitely more misleading.

I'm reasonably convinced now that this is just a weird artefact of
running gdb on an optimised binary, probably a shortcoming in the debug
info leading to gdb getting confused.

Unfortunately this also calls into doubt the parameter to talloc_free,
perhaps in that context 0xff000 is a similar artefact.

Please can you print the entire contents of tdb in the second frame
(print *tdb ought to do it). I'm curious whether it is all sane or
not.

Please can you also print info regs at the point of the segv (in frame
0) as well as disas at that point.

Can you also p $_siginfo._sifields._sigfault.si_addr (in frame 0).
This ought to be the actual faulting address, which ought to give a hint
on how much we can trust the parameters in the stack trace.

Since I'm asking for the world I may as well ask you to dump the raw
stack too x/64x $sp ought to be a good starting point.

I notice in your bugzilla (for a different occurrence, I think):
 [2090451.721705] univention-conf[2512]: segfault at ff ip 
 0045e238 sp 768dfa30 error 6 in python2.6[40+21e000]

Which appears to have faulted access 0xff0 too. It looks like
this process is a python thing, it's nothing to do with xenstored I
assume? It seems rather coincidental that it should be accessing the 
same sort of address and be faulting.

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] Xen Project 4.5 RC4 is Ready Today; Test Day is Wednesday

2014-12-15 Thread Russ Pavlicek
Folks,

This Wednesday, December 17, is our fourth and FINAL Test Day
for the 4.5 release cycle (barring any changes which may result from
Wednesday's Test Day). Release Candidate 4 is available for
assessment today.

If you've held off testing the new release until it matures, delay no
longer!  Test and state your concerns now or the next release you
see could be the official one

Information about testing this release can be found here:
http://wiki.xenproject.org/wiki/Xen_4.5_RC4_test_instructions

To learn more about Test Days, including the scheduled date
for the final release, check out:
http://wiki.xenproject.org/wiki/Xen_Project_Test_Days

See you in #xentest on IRC this Wednesday for Test Day!

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v2 for 4.5] x86/VPMU: Clear last_vcpu when destroying VPMU

2014-12-15 Thread Boris Ostrovsky
We need to make sure that last_vcpu is not pointing to VCPU whose
VPMU is being destroyed. Otherwise we may try to dereference it in
the future, when VCPU is gone.

We have to do this via IPI since otherwise there is a (somewheat
theoretical) chance that between test and subsequent clearing
of last_vcpu the remote processor (i.e. vpmu-last_pcpu) might do
both load_vpmu() and then save_vpmu() for another VCPU. The former
will clear last_vcpu and the latter will set it to something else.

Performing this operation via IPI will guarantee that nothing can
happen on the remote processor between testing and clearing of
last_vcpu.

Signed-off-by: Boris Ostrovsky boris.ostrov...@oracle.com
---
 xen/arch/x86/hvm/vpmu.c |   20 
 1 files changed, 20 insertions(+), 0 deletions(-)

Changes in v2:
 * Test last_vcpu locally before IPI
 * Don't handle local pcpu as a special case --- on_selected_cpus
   will take care of that
 * Dont't cast variables unnecessarily

diff --git a/xen/arch/x86/hvm/vpmu.c b/xen/arch/x86/hvm/vpmu.c
index 1df74c2..37f0d9f 100644
--- a/xen/arch/x86/hvm/vpmu.c
+++ b/xen/arch/x86/hvm/vpmu.c
@@ -247,10 +247,30 @@ void vpmu_initialise(struct vcpu *v)
 }
 }
 
+static void vpmu_clear_last(void *arg)
+{
+if ( this_cpu(last_vcpu) == arg )
+this_cpu(last_vcpu) = NULL;
+}
+
 void vpmu_destroy(struct vcpu *v)
 {
 struct vpmu_struct *vpmu = vcpu_vpmu(v);
 
+if ( !vpmu_is_set(vpmu, VPMU_CONTEXT_ALLOCATED) )
+return;
+
+/*
+ * Need to clear last_vcpu in case it points to v.
+ * We can check here non-atomically whether it is 'v' since
+ * last_vcpu can never become 'v' again at this point.
+ * We will test it again in vpmu_clear_last() with interrupts
+ * disabled to make sure we don't clear someone else.
+ */
+if ( per_cpu(last_vcpu, vpmu-last_pcpu) == v )
+on_selected_cpus(cpumask_of(vpmu-last_pcpu),
+ vpmu_clear_last, v, 1);
+
 if ( vpmu-arch_vpmu_ops  vpmu-arch_vpmu_ops-arch_vpmu_destroy )
 vpmu-arch_vpmu_ops-arch_vpmu_destroy(v);
 }
-- 
1.7.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] xenstored crashes with SIGSEGV

2014-12-15 Thread Philipp Hahn
Hello Ian,

On 15.12.2014 18:45, Ian Campbell wrote:
 On Mon, 2014-12-15 at 14:50 +, Ian Campbell wrote:
 On Mon, 2014-12-15 at 15:19 +0100, Philipp Hahn wrote:
 I just noticed something strange:

 #3  0x0040a684 in tdb_open (name=0xff Address
 0xff out of bounds, hash_size=0,
 tdb_flags=4254928, open_flags=-1, mode=3119127560) at tdb.c:1773
...
 I'm reasonably convinced now that this is just a weird artefact of
 running gdb on an optimised binary, probably a shortcoming in the debug
 info leading to gdb getting confused.
 
 Unfortunately this also calls into doubt the parameter to talloc_free,
 perhaps in that context 0xff000 is a similar artefact.
 
 Please can you print the entire contents of tdb in the second frame
 (print *tdb ought to do it). I'm curious whether it is all sane or
 not.

(gdb) print *tdb
$1 = {name = 0x0, map_ptr = 0x0, fd = 47, map_size = 65280, read_only =
16711680,
  locked = 0xff00, ecode = 16711680, header = {
magic_food =
\000\000\000\000\000\000\000\000\000\377\000\000\000\000\377\000\000\000\000\000\000\000\000\000\000\377\000\000\000\000\377,
version = 0, hash_size = 0,
rwlocks = 65280, reserved = {16711680, 0, 0, 65280, 16711680, 0, 0,
65280,
  16711680, 0, 0, 65280, 16711680, 0, 0, 65280, 16711680, 0, 0,
65280, 16711680,
  0, 0, 65280, 16711680, 0, 0, 65280, 16711680, 0, 0}}, flags = 0,
travlocks = {
next = 0xff, off = 0, hash = 65280}, next = 0xff,
  device = 280375465082880, inode = 16711680, log_fn = 0x4093b0
null_log_fn,
  hash_fn = 0x4092f0 default_tdb_hash, open_flags = 2}

 Please can you also print info regs at the point of the segv (in frame
 0) as well as disas at that point.

(gdb) info registers
rax0x0  0
rbx0x16bff7023854960
rcx0x   -1
rdx0x40ecd0 4254928
rsi0x0  0
rdi0xff00   280375465082880
rbp0x7fcaed6c96a8   0x7fcaed6c96a8
rsp0x7fff9dc86330   0x7fff9dc86330
r8 0x7fcaece54c08   140509534571528
r9 0xff00   -72057594037927936
r100x7fcaed08c14c   140509536895308
r110x246582
r120xd  13
r130xff00   280375465082880
r140x4093b0 4232112
r150x167d62023582240
rip0x4075c4 0x4075c4 talloc_chunk_from_ptr+4
eflags 0x10206  [ PF IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0  0
es 0x0  0
fs 0x0  0
gs 0x0  0
fctrl  0x0  0
fstat  0x0  0
ftag   0x0  0
fiseg  0x0  0
fioff  0x0  0
foseg  0x0  0
fooff  0x0  0
fop0x0  0
mxcsr  0x0  [ ]

(gdb) disassemble
Dump of assembler code for function talloc_chunk_from_ptr:
0x004075c0 talloc_chunk_from_ptr+0:   sub$0x8,%rsp
0x004075c4 talloc_chunk_from_ptr+4:   mov-0x8(%rdi),%edx
0x004075c7 talloc_chunk_from_ptr+7:   lea-0x50(%rdi),%rax
0x004075cb talloc_chunk_from_ptr+11:  mov%edx,%ecx
0x004075cd talloc_chunk_from_ptr+13:  and
$0xfff0,%ecx
0x004075d0 talloc_chunk_from_ptr+16:  cmp$0xe814ec70,%ecx
0x004075d6 talloc_chunk_from_ptr+22:  jne0x4075e2
talloc_chunk_from_ptr+34
0x004075d8 talloc_chunk_from_ptr+24:  and$0x1,%edx
0x004075db talloc_chunk_from_ptr+27:  jne0x4075e2
talloc_chunk_from_ptr+34
0x004075dd talloc_chunk_from_ptr+29:  add$0x8,%rsp
0x004075e1 talloc_chunk_from_ptr+33:  retq
0x004075e2 talloc_chunk_from_ptr+34:  nopw   0x0(%rax,%rax,1)
0x004075e8 talloc_chunk_from_ptr+40:  callq  0x401b98 abort@plt

 Can you also p $_siginfo._sifields._sigfault.si_addr (in frame 0).
 This ought to be the actual faulting address, which ought to give a hint
 on how much we can trust the parameters in the stack trace.

Hmm, my gdb refused to access $_siginfo:
(gdb) show convenience
$_siginfo = Unable to read siginfo

 Since I'm asking for the world I may as well ask you to dump the raw
 stack too x/64x $sp ought to be a good starting point.

(gdb) x/64x $sp
0x7fff9dc86330: 0xed6c96a8  0x7fca  0x00407edf  0x
0x7fff9dc86340: 0x  0x  0x016bff70  0x
0x7fff9dc86350: 0xed6c96a8  0x7fca  0x000d  0x
0x7fff9dc86360: 0x  0x  0x004093b0  0x
0x7fff9dc86370: 0x0167d620  0x  0x0040a348  0x
0x7fff9dc86380: 0x  0x  0x  0x
0x7fff9dc86390: 0x  0x  0x  0x
0x7fff9dc863a0: 0x0011  0x  0x411d4816  0x
0x7fff9dc863b0: 0x0001  0x  0x81a0  0x
0x7fff9dc863c0: 

Re: [Xen-devel] [PATCH 11/12] vTPM/TPM2: Bind group keys and sectors data on disk

2014-12-15 Thread Xu, Quan


 -Original Message-
 From: Daniel De Graaf [mailto:dgde...@tycho.nsa.gov]
 Sent: Monday, December 15, 2014 11:56 PM
 To: Xu, Quan; xen-devel@lists.xen.org
 Cc: stefano.stabell...@eu.citrix.com; samuel.thiba...@ens-lyon.org
 Subject: Re: [PATCH 11/12] vTPM/TPM2: Bind group keys and sectors data
 on disk
 
 On 12/14/2014 07:09 AM, Quan Xu wrote:
 [...]
  +/*TPM 2.0 bind | TPM 1.x seal*/
  +if (hw_is_tpm2()) {
  +TPM2_disk_bind(dst, sblob, sizeof(sblob));
  +} else {
  +dst-pcr_selection = src-seals[i].pcr_selection;
  +memcpy(dst-digest_release, src-seals[i].digest_release,
 20);
  +TPM_pcr_digest(dst-digest_at_seal, dst-pcr_selection);
  +TPM_disk_seal(dst, sblob, sizeof(sblob));
  +}
 
 It appears that the secrets for the vTPMs are only being bound to the
 presence of the physical TPM and not the measurements of the hypervisor
 and other TCB components.  This does not provide as much security as it
 did for TPM 1.2: an attacker with access to the boot disk can boot into a
 compromised environment and extract the vTPM keys and disk images.
 
Agree with it.
I will bind more information, such as measurements of the hypervisor and other 
TCB components
In next version.


 The TPM2_Create/TPM2_Unseal operations should be capable of performing
 the same functionality.  If only SHA1 PCRs are used, they should be able to
 be drop-in replacements, but supporting other hash algorithms may be a
 feature that users who have a TPM2 will want.
 
Interesting:)..
I will continue to develop and maintain vTPM on TPM 2.0. Make it stable and 
robust.

 --
 Daniel De Graaf
 National Security Agency


Intel
Quan Xu

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [linux-linus test] 32386: regressions - trouble: broken/fail/pass

2014-12-15 Thread xen . org
flight 32386 linux-linus real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/32386/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-rumpuserxen-i386  8 guest-start   fail REGR. vs. 31241
 test-amd64-amd64-rumpuserxen-amd64  8 guest-start fail REGR. vs. 31241
 test-amd64-i386-qemuu-rhel6hvm-amd  3 host-install(3)   broken REGR. vs. 31241
 test-amd64-i386-xl-credit29 guest-start   fail REGR. vs. 31241
 test-amd64-amd64-xl  11 guest-saverestore fail REGR. vs. 31241
 test-amd64-i386-xl   11 guest-saverestore fail REGR. vs. 31241
 test-amd64-i386-xl-multivcpu 11 guest-saverestore fail REGR. vs. 31241
 test-amd64-i386-xl-qemuu-winxpsp3  3 host-install(3)broken REGR. vs. 31241
 test-amd64-amd64-pair16 guest-start/debianfail REGR. vs. 31241
 test-amd64-i386-xl-qemuu-debianhvm-amd64 11 guest-saverestore.2 fail REGR. vs. 
31241
 test-amd64-amd64-xl-qemut-debianhvm-amd64 3 host-install(3) broken REGR. vs. 
31241
 test-amd64-i386-pair 16 guest-start/debianfail REGR. vs. 31241

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-sedf-pin  9 guest-start   fail REGR. vs. 31241
 test-amd64-i386-freebsd10-i386  7 freebsd-install  fail like 31241
 test-armhf-armhf-xl   9 guest-start  fail   like 31241
 test-amd64-i386-freebsd10-amd64  7 freebsd-install fail like 31241
 test-amd64-amd64-xl-sedf 14 guest-localmigrate.2  fail REGR. vs. 31241
 test-amd64-amd64-xl-qemuu-winxpsp3  7 windows-install  fail like 31241

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt   9 guest-start  fail   never pass
 test-armhf-armhf-libvirt  9 guest-start  fail   never pass
 test-amd64-amd64-xl-pcipt-intel  9 guest-start fail never pass
 test-amd64-amd64-libvirt  9 guest-start  fail   never pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 14 guest-stop fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 14 guest-stop  fail never pass
 test-amd64-i386-xl-win7-amd64 14 guest-stop   fail  never pass
 test-amd64-i386-xl-qemuu-win7-amd64 14 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 14 guest-stop fail never pass
 test-amd64-amd64-xl-qemut-winxpsp3 14 guest-stop   fail never pass
 test-amd64-amd64-xl-win7-amd64 14 guest-stop   fail never pass
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1 14 guest-stop fail never pass
 test-amd64-i386-xl-winxpsp3-vcpus1 14 guest-stop   fail never pass
 test-amd64-i386-xl-winxpsp3  14 guest-stop   fail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64 14 guest-stop fail never pass
 test-amd64-i386-xl-qemut-winxpsp3 14 guest-stopfail never pass
 test-amd64-amd64-xl-winxpsp3 14 guest-stop   fail   never pass

version targeted for testing:
 linux67e2c3883828b39548cee2091b36656787775d95
baseline version:
 linux9f76628da20f96a179ca62b504886f99ecc29223


1671 people touched revisions under test,
not listing them all


jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 build-amd64-rumpuserxen  pass
 build-i386-rumpuserxen   pass
 test-amd64-amd64-xl  fail
 test-armhf-armhf-xl  fail
 test-amd64-i386-xl   fail
 test-amd64-i386-rhel6hvm-amd pass
 test-amd64-i386-qemut-rhel6hvm-amd   pass
 test-amd64-i386-qemuu-rhel6hvm-amd   broken  
 test-amd64-amd64-xl-qemut-debianhvm-amd64broken  
 test-amd64-i386-xl-qemut-debianhvm-amd64 pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64 fail
 

[Xen-devel] [RFC V9 4/4] domain snapshot design: libxl/libxlu

2014-12-15 Thread Chunyan Liu
Changes to V8:
  * remove libxl_domain_snapshot_create/delete/revert API
  * export disk snapshot functionality for both xl and libvirt usage

===
Libxl/libxlu Design

1. New Structures

libxl_disk_snapshot = Struct(disk_snapshot,[
# target disk
(disk,libxl_device_disk),

# disk snapshot name
(name,string),

# internal/external disk snapshot?
(external,bool),

# for external disk snapshot, specify following two field
(external_format, string),
(external_path,   string),
])


2. New Functions

Since there're already APIs for saving memory (libxl_domain_suspend)
and restoring domain from saved memory (libxl_domain_create_restore), to
xl domain snapshot tasks, the missing part is disk snapshot functionality.
And the disk snapshot functionality would be used by libvirt too.

Considering there is qmp handling in creating/deleting disk snapshot,
will add following new functions to libxl (?):

int libxl_disk_snapshot_create(libxl_ctx *ctx, uint32_t domid,
   libxl_disk_snapshot *snapshot, int nb);

Taking disk snapshots to a group of domain disks according to
configuration. For qcow2 disk backend type, it will call qmp
transaction command to do the work. For other disk backend types,
might call other external commands.

Parameters:
   ctx (INPUT):
   context
   domid (INPUT):
   domain id
   snapshot (INPUT):
   array of disk snapshot configuration. Has nb members.

   libxl_device_disk:
   structure to represent which disk.
   name:
   snapshot name.
   external:
   internal snapshot or external snapshot.
   'false' means internal disk snapshot. external_format and
   external_path will be ignored.
   'true' means external disk snapshot, then external_format
   and external_path should be provided.
   external_format:
   Should be provided when 'external' is true. If not provided,
   will use default format proper to the backend file.
   Ignored when 'external' is false.
   external_path:
   Must be provided when 'external' is true.
   Ignored when 'external' is false.
   nb (INPUT):
   number of disks that need to take disk snapshot.

Return:
   0 on success, -1 on error.


/*  This API might not be used by xl, since xl won't take care of deleting
 *  snapshots. But for libvirt, since libvirt manages snapshots and will
 *  delete snapshot, this API will be used.
 */
int libxl_disk_snapshot_delete(libxl_ctx *ctx, uint32_t domid,
   libxl_disk_snapshot *snapshot, int nb);

Delete disk snapshot of a group of domain disks according to
configuration. For qcow2 disk backend type, it will call qmp command
to delete internal disk snapshot. For other disk backend types, might
call other external commands.

Parameters:
   ctx (INPUT):
   context
   domid (INPUT):
   domain id
   snapshot (INPUT):
   array of disk snapshot configuration. Has nb members.
   nb (INPUT):
   number of disks that need to take disk snapshot.

Return:
   0 on success, -1 on error.


int libxl_disk_to_snapshot(libxl_ctx *ctx, uint32_t domid,
   libxl_disk_snapshot **snapshot, int *num);

This is for domain snapshot create. If user doesn't specify disks,
then by default it will take internal disk snapshot to each domain
disk. This function will fill libxl_disk_snapshot according to domain
disks info.

Parameters:
   ctx (INPUT):
   context
   domid (INPUT):
   domain id
   snapshot (OUTPUT):
   array of disk snapshot configuration.
   num (OUTPUT):
   number of disks.

Return:
   0 on success, -1 on error.



For disk snapshot revert, no qmp command for that, it always calls
external commands to finish the work, so put in libxlu (?):

int xlu_disk_snapshot_revert(libxl_disk_snapshot *snapshot, int nb);

Apply disk snapshot for a group of disks according to configuration. To
different disk backend types, call different external commands to do
the work.

Parameters:
   snapshot (INPUT):
   array of disk snapshot configuration. Has nb members.
   nb (INPUT):
   number of disks that need to take disk snapshot.

Return:
   0 on success, -1 on error.



3. Simple Architecture View

Creating domain snapshot:
(* means new functions we will need in libxl/libxlu)

  xl snapshot-create
 |
  parse configuration  libxl_disk_to_snapshot (*)
 |
  saving memory  libxl_domain_suspend
 |
 taking disk snapshot  libxl_disk_snapshot_create (*)
 |

[Xen-devel] [RFC V9 2/4] domain snapshot overview

2014-12-15 Thread Chunyan Liu
Changes to V8:
  * add an overview document, so that one can has a overall look
about the whole domain snapshot work, limits, requirements,
how to do, etc.

=
Domain snapshot overview

1. Purpose

Domain snapshot is a system checkpoint of a domain. Later, one can
roll back the domain to that checkpoint. It's a very useful backup
function. A domain snapshot contains the memory status at the
checkpoint and the disk status (which we called disk snapshot).

Domain snapshot functionality usually includes:
a) create a domain snapshot
b) roll back (or called revert) to a domain snapshot
c) delete a domain snapshot
d) list all domain snapshots

But following the existing xl idioms of managing storage and saved
VM images via existing CLI command (qemu-img, lvcreate, ls, mv,
cp etc), xl snapshot functionality would be kept as simple as
possible:
* xl will do a) and b), creating a snapshot and reverting a
  domain to a snapshot.
* xl will NOT do c) and d), xl won't manage snapshots, as xl
  doesn't maintain saved images created by 'xl save'. So xl
  will have no idea of the existence of domain snapshots and
  the chain relationship between snapshots. It will depends on
  user to take care of the snapshots, know the snapshot chain
  info, and delete snapshots.

Domain Snapshot Support and Not Support:
* support live snapshot
* support internal disk snapshot and external disk snapshot
* support different disk backend types.
  (Basic goal is to support 'raw' and 'qcow2' only).

* not support snapshot when domain is shutdowning or dying.
* not support disk-only snapshot [1].

 [1] To xl, it only concerns active domains, and even when domain
 is paused, there is no data flush to disk operation. So, take
 a disk-only snapshot and then resume, it is as if the guest
 had crashed. For this reason, disk-only snapshot is meaningless
 to xl. Should not support.


2. Requirements

General Requirements:
* ability to save/restore domain memory
* ability to create/delete/apply disk snapshot [2]
* ability to parse user config file

  [2] Disk snapshot requirements:
  - external tools: qemu-img, lvcreate, vhd-util, etc.
  - for basic goal, we support 'raw' and 'qcow2' backend types
only. Then it requires:
libxl qmp command or qemu-img (when qemu process does not
exist)


3. Interaction with other operations:

No.


4. General workflow

Create a snapshot:
  * parse user cfg file if passed in
  * check snapshot operation is allowed or not
  * save domain, saving memory status to file (refer to: save_domain)
  * take disk snapshot (e.g. call qmp command)
  * unpause domain

Revert to snapshot:
  * parse use cfg file (xl doesn't manage snapshots, so it has no
idea of snapshot existence. User MUST supply configuration file)
  * destroy this domain
  * create a new domain from snapshot info
- apply disk snapshot (e.g. call qemu-img)
- a process like restore domain

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [RFC V9 3/4] domain snapshot design: xl

2014-12-15 Thread Chunyan Liu
Changes to V8:
  * xl won't manage snapshots, that means it won't maintain json files,
won't maintain snapshot chain relationship, and then as a result
won't take care of deleting snapshot and listing snapshots.
  * remove snapshot-delete and snapshot-list interface
  * update snapshot-revert interface
  * update snapshot-create/revert implementaion

===

XL Design

1. User Interface

xl snapshot-create:
  Create a snapshot (disk and RAM) of a domain.

  SYNOPSIS:
snapshot-create domain [cfgfile] [--name string] [--live]

  OPTIONS:
--name string  snapshot name
--live   take a live snapshot

If option includes --live, then the domain is not paused while creating
the snapshot, like live migration. This increases size of the memory
dump file, but reducess downtime of the guest.

If option doens't include --name, a default name will be generated
according to the creation time.

If specify @cfgfile, use cfgfile. (e.g. if --name specifies a name,
meanwhile there is name specified in cfgfile, name in cfgfile will
be used.)


xl snapshot-revert:
  Revert domain to status of a snapshot.

  SYNOPSIS:
  snapshot-revert domain cfgfile [--running] [--force]

  OPTIONS:
--runningafter reverting, change state to running
--force  try harder on risky reverts

Normally, the domain will revert to the same state the domain was in while
the snapshot was taken (whether running, or paused).

If option includes --running, then overrides the snapshot state to
guarantee a running domain after the revert.



2. cfgfile syntax

#snapshot name. If user doesn't provide a VM snapshot name, xl will generate
#a name automatically by the creation time.
name=

#snapshot description. Default is NULL.
description=

#memory location. This field should be filled when memory=1. Default is NULL.
memory_path=

#disk snapshot information
#For easier parse config work, reuse disk configuration in xl.cfg, but
#with different meanings.
#disk syntax meaning: 'external path, external format, target device'

#e.g. to specify exernal disk snapshot, like this:
#disks=['/tmp/hda_snapshot.qcow2,qcow2,hda',
'/tmp/hdb_snapshot.qcow2,qcow2,hdb',]

#e.g. to specify internal disk snapshot, like this:
disks=[',,hda',',,hdb',]


3. xl snapshot-xxx implementation

xl snapshot-create

1), parse args or user configuration file.
2), save domain (store saved memory to memory_path)
3), create disk snapshots according to disk snapshot configuration
4), unpause domain

xl snapshot-revert

1), parse user configuration file
2), destroy current domain
3), revert disk snapshots according to disk snapshot configuration
4), restore domain from saved memory.

4. Notes

* user should take care of those snapshots, like: saved memory file, disk
  snapshots info (internal, external, etc.), snapshot chain relationship
* user should delete snapshots by themselves with CLI commands like: rm,
  qemu-img, etc.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel