[Xen-devel] [linux-3.14 test] 61316: regressions - trouble: broken/fail/pass

2015-09-04 Thread osstest service owner
flight 61316 linux-3.14 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/61316/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-rumpuserxen  5 rumpuserxen-build fail in 61096 REGR. vs. 60666
 build-i386-xsm5 xen-buildfail in 61096 REGR. vs. 60666
 build-armhf   5 xen-buildfail in 61096 REGR. vs. 60666
 test-amd64-i386-xl-vhd  13 guest-saverestore fail in 61263 REGR. vs. 60666
 test-amd64-i386-xl-qcow213 guest-saverestore fail in 61263 REGR. vs. 60666
 test-amd64-amd64-xl-vhd  19 guest-start.2fail in 61263 REGR. vs. 60666
 test-amd64-amd64-xl-qcow219 guest-start.2fail in 61263 REGR. vs. 60666

Tests which are failing intermittently (not blocking):
 test-amd64-i386-qemut-rhel6hvm-intel  3 host-install(3)   broken pass in 61263
 test-amd64-i386-xl-vhd3 host-install(3)   broken pass in 61263
 test-amd64-i386-xl-qcow2  3 host-install(3)   broken pass in 61263
 test-amd64-amd64-xl-vhd   3 host-install(3)   broken pass in 61263
 test-amd64-amd64-libvirt-qcow2  3 host-install(3) broken pass in 61263
 test-amd64-i386-libvirt   3 host-install(3)   broken pass in 61263
 test-amd64-amd64-xl-pvh-intel  3 host-install(3)  broken pass in 61263
 test-amd64-i386-xl-xsm3 host-install(3)   broken pass in 61263
 test-amd64-i386-xl3 host-install(3)   broken pass in 61263
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 3 host-install(3) broken 
pass in 61263
 test-amd64-i386-xl-qemuu-ovmf-amd64  3 host-install(3)broken pass in 61263
 test-amd64-amd64-xl-credit2   3 host-install(3)   broken pass in 61263
 test-amd64-i386-qemut-rhel6hvm-amd  3 host-install(3) broken pass in 61263
 test-amd64-amd64-libvirt-raw  3 host-install(3)   broken pass in 61263
 test-amd64-i386-xl-qemuu-win7-amd64  3 host-install(3)broken pass in 61263
 test-amd64-amd64-xl-pvh-amd   3 host-install(3)   broken pass in 61263
 test-amd64-i386-freebsd10-i386  3 host-install(3) broken pass in 61263
 test-amd64-amd64-libvirt-vhd  3 host-install(3)   broken pass in 61263
 test-amd64-i386-libvirt-xsm   3 host-install(3)   broken pass in 61263
 test-amd64-i386-libvirt-vhd   3 host-install(3)   broken pass in 61263
 test-amd64-i386-xl-qemuu-debianhvm-amd64 3 host-install(3) broken pass in 61263
 test-amd64-i386-qemuu-rhel6hvm-amd  3 host-install(3) broken pass in 61263
 test-amd64-i386-xl-raw3 host-install(3)   broken pass in 61263
 test-amd64-amd64-rumpuserxen-amd64  3 host-install(3) broken pass in 61263
 test-amd64-i386-rumpuserxen-i386  3 host-install(3)   broken pass in 61263
 test-amd64-amd64-libvirt  3 host-install(3)   broken pass in 61263
 test-amd64-amd64-xl-qemut-winxpsp3  3 host-install(3) broken pass in 61263
 test-amd64-i386-libvirt-raw   3 host-install(3)   broken pass in 61263
 test-amd64-amd64-xl-xsm   3 host-install(3)   broken pass in 61263
 test-amd64-i386-qemuu-rhel6hvm-intel  3 host-install(3)   broken pass in 61263
 test-amd64-i386-freebsd10-amd64  3 host-install(3)broken pass in 61263
 test-amd64-i386-xl-qemut-debianhvm-amd64 3 host-install(3) broken pass in 61263
 test-amd64-i386-xl-qemut-win7-amd64  3 host-install(3)broken pass in 61263
 test-amd64-amd64-xl-qcow2 3 host-install(3)   broken pass in 61263
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 3 host-install(3) broken pass in 
61263
 test-amd64-amd64-xl-qemuu-win7-amd64  3 host-install(3)   broken pass in 61263
 test-amd64-amd64-xl-multivcpu  3 host-install(3)  broken pass in 61263
 test-amd64-amd64-xl-qemuu-ovmf-amd64  3 host-install(3)   broken pass in 61263
 test-amd64-amd64-i386-pvgrub  3 host-install(3)   broken pass in 61263
 test-amd64-amd64-xl-qemut-win7-amd64  3 host-install(3)   broken pass in 61263
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 3 host-install(3) broken 
pass in 61263
 test-amd64-i386-libvirt-qcow2  3 host-install(3)  broken pass in 61263
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 3 host-install(3) broken 
pass in 61263
 test-amd64-amd64-amd64-pvgrub  3 host-install(3)  broken pass in 61263
 test-amd64-amd64-pygrub   3 host-install(3)   broken pass in 61263
 test-amd64-amd64-xl-qemut-debianhvm-amd64 3 host-install(3) broken pass in 
61263
 test-amd64-amd64-xl-raw   3 host-install(3)   broken pass in 61263
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 3 host-install(3) broken pass in 
61263
 test-amd64-amd64-xl-rtds  3 host-install(3)   broken pass in 61263
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 3 host-install(3) broken pass in 
61263
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 3 host-install(3) broken pass in 
61263
 test-amd64-i386-xl-qemut-winxpsp3-vcpus

[Xen-devel] [libvirt test] 61350: trouble: blocked/broken/pass

2015-09-04 Thread osstest service owner
flight 61350 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/61350/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-xsm   3 host-install(3) broken REGR. vs. 61304
 build-armhf   3 host-install(3) broken REGR. vs. 61304
 build-armhf-pvops 3 host-install(3) broken REGR. vs. 61304
 build-amd64   3 host-install(3) broken REGR. vs. 61304
 build-i386-xsm3 host-install(3) broken REGR. vs. 61304
 build-i386-libvirt3 host-install(3) broken REGR. vs. 61304
 build-amd64-xsm   3 host-install(3) broken REGR. vs. 61304
 build-amd64-pvops 3 host-install(3) broken REGR. vs. 61304

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-vhd  1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-vhd   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-xsm  1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a

version targeted for testing:
 libvirt  a39ab909081ab126990ca705852d38ff056b8f13
baseline version:
 libvirt  5c668a78d85b0d71e6ac8e23f2c605058b44df65

Last test of basis61304  2015-09-02 12:32:03 Z2 days
Testing same since61350  2015-09-04 13:14:40 Z0 days1 attempts


People who touched revisions under test:
  Erik Skultety 
  Jim Fehlig 
  John Ferlan 
  Laine Stump 
  Michal Privoznik 
  Pavel Hrdina 

jobs:
 build-amd64-xsm  broken  
 build-armhf-xsm  broken  
 build-i386-xsm   broken  
 build-amd64  broken  
 build-armhf  broken  
 build-i386   pass
 build-amd64-libvirt  blocked 
 build-armhf-libvirt  blocked 
 build-i386-libvirt   broken  
 build-amd64-pvopsbroken  
 build-armhf-pvopsbroken  
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmblocked 
 test-amd64-amd64-libvirt-xsm blocked 
 test-armhf-armhf-libvirt-xsm blocked 
 test-amd64-i386-libvirt-xsm  blocked 
 test-amd64-amd64-libvirt blocked 
 test-armhf-armhf-libvirt blocked 
 test-amd64-i386-libvirt  blocked 
 test-amd64-amd64-libvirt-pairblocked 
 test-amd64-i386-libvirt-pair blocked 
 test-amd64-amd64-libvirt-qcow2   blocked 
 test-armhf-armhf-libvirt-qcow2   blocked 
 test-amd64-i386-libvirt-qcow2blocked 
 test-amd64-amd64-libvirt-raw blocked 
 test-armhf-armhf-libvirt-raw blocked 
 test-amd64-i386-libvirt-raw  blo

[Xen-devel] [PATCH for 4.6 v2 1/5] libxc: clearer migration v2 debug message

2015-09-04 Thread Wei Liu
Previous the message was like:

SAVE:
xc: detail: 32 bits, 3 levels
xc: detail: max_pfn 0xf, p2m_frames 1024
xc: detail: max_mfn 0x13

RESTORE:
xc: detail: max_mfn 0x13
xc: detail: 32 bits, 3 levels
xc: detail: Expanded p2m from 0 to 0xf

It's not immediately clear that the last line in restore messages was
referring to max_pfn. Change the debug message a bit to make that
clearer.

Signed-off-by: Wei Liu 
Reviewed-by: Andrew Cooper 
---
 tools/libxc/xc_sr_restore_x86_pv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libxc/xc_sr_restore_x86_pv.c 
b/tools/libxc/xc_sr_restore_x86_pv.c
index ef26c64..c65a2f1 100644
--- a/tools/libxc/xc_sr_restore_x86_pv.c
+++ b/tools/libxc/xc_sr_restore_x86_pv.c
@@ -66,7 +66,7 @@ static int expand_p2m(struct xc_sr_context *ctx, unsigned 
long max_pfn)
 for ( i = (old_end_frame ? old_end_frame + 1 : 0); i <= end_frame; ++i )
 ctx->x86_pv.p2m_pfns[i] = INVALID_MFN;
 
-DPRINTF("Expanded p2m from %#lx to %#lx", old_max, max_pfn);
+DPRINTF("Changed max_pfn from %#lx to %#lx", old_max, max_pfn);
 return 0;
 }
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH for 4.6 v2 2/5] libxc: migration v2 prefix Memory -> Frames

2015-09-04 Thread Wei Liu
The prefix "Memory" is confusing because the numbers shown after that
are referring to frames. They have no bearing on how many pages a domain
actually owns or how many actual pages are processed.

Signed-off-by: Wei Liu 
---
 tools/libxc/xc_sr_save.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c
index 58667af..924c425 100644
--- a/tools/libxc/xc_sr_save.c
+++ b/tools/libxc/xc_sr_save.c
@@ -659,7 +659,7 @@ static int send_domain_memory_nonlive(struct xc_sr_context 
*ctx)
 if ( rc )
 goto err;
 
-xc_set_progress_prefix(xch, "Memory");
+xc_set_progress_prefix(xch, "Frames");
 
 rc = send_all_pages(ctx);
 if ( rc )
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH for 4.6 v2 3/5] libxc: fix indentation

2015-09-04 Thread Wei Liu
Signed-off-by: Wei Liu 
Reviewed-by: Andrew Cooper 
---
 tools/libxc/xc_sr_restore_x86_pv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libxc/xc_sr_restore_x86_pv.c 
b/tools/libxc/xc_sr_restore_x86_pv.c
index c65a2f1..bc604b3 100644
--- a/tools/libxc/xc_sr_restore_x86_pv.c
+++ b/tools/libxc/xc_sr_restore_x86_pv.c
@@ -970,7 +970,7 @@ static int x86_pv_localise_page(struct xc_sr_context *ctx,
 }
 
 if ( to_populate && populate_pfns(ctx, to_populate, pfns, NULL) )
-return -1;
+return -1;
 
 for ( i = 0; i < (PAGE_SIZE / sizeof(uint64_t)); ++i )
 {
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH for 4.6 v2 5/5] libxc: add assertion to avoid setting same bit more than once

2015-09-04 Thread Wei Liu
Signed-off-by: Wei Liu 
Reviewed-by: Andrew Cooper 
---
 tools/libxc/xc_sr_restore.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/libxc/xc_sr_restore.c b/tools/libxc/xc_sr_restore.c
index 924dd55..f48e7fc 100644
--- a/tools/libxc/xc_sr_restore.c
+++ b/tools/libxc/xc_sr_restore.c
@@ -181,6 +181,7 @@ static int pfn_set_populated(struct xc_sr_context *ctx, 
xen_pfn_t pfn)
 ctx->restore.max_populated_pfn = new_max;
 }
 
+assert(!test_bit(pfn, ctx->restore.populated_pfns));
 set_bit(pfn, ctx->restore.populated_pfns);
 
 return 0;
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH for 4.6 v2 4/5] libxc: don't populate same pfn more than once in populate_pfns

2015-09-04 Thread Wei Liu
The original implementation of populate_pfns didn't consider the same
pfn can be present multiple times in the array. The mechanism to prevent
populating the same pfn multiple times only worked if the recurring pfn
appeared in different batches.

This bug is discovered by Linux 4.1 32 bit kernel save / restore test,
which has several ptes pointing to same pfn, which results in an array
containing recurring pfn.  When libxc called x86_pv_localise_page, the
original implementation would populate the same pfn more than once.

The fix is to set bit in populated bitmap as we generate list of pfns to
be populated.

Signed-off-by: Wei Liu 
---
 tools/libxc/xc_sr_restore.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/libxc/xc_sr_restore.c b/tools/libxc/xc_sr_restore.c
index df885b6..924dd55 100644
--- a/tools/libxc/xc_sr_restore.c
+++ b/tools/libxc/xc_sr_restore.c
@@ -214,6 +214,9 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned count,
   types[i] != XEN_DOMCTL_PFINFO_BROKEN))) &&
  !pfn_is_populated(ctx, original_pfns[i]) )
 {
+rc = pfn_set_populated(ctx, original_pfns[i]);
+if ( rc )
+goto err;
 pfns[nr_pfns] = mfns[nr_pfns] = original_pfns[i];
 ++nr_pfns;
 }
@@ -238,9 +241,6 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned count,
 goto err;
 }
 
-rc = pfn_set_populated(ctx, pfns[i]);
-if ( rc )
-goto err;
 ctx->restore.ops.set_gfn(ctx, pfns[i], mfns[i]);
 }
 }
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH for 4.6 v2 0/5] Migration v2 fix

2015-09-04 Thread Wei Liu
Wei Liu (5):
  libxc: clearer migration v2 debug message
  libxc: migration v2 prefix Memory -> Frames
  libxc: fix indentation
  libxc: don't populate same pfn more than once in populate_pfns
  libxc: add assertion to avoid setting same bit more than once

 tools/libxc/xc_sr_restore.c| 7 ---
 tools/libxc/xc_sr_restore_x86_pv.c | 4 ++--
 tools/libxc/xc_sr_save.c   | 2 +-
 3 files changed, 7 insertions(+), 6 deletions(-)

-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 11/29] xen/x86: add bitmap of enabled emulated devices

2015-09-04 Thread Andrew Cooper

On 04/09/15 14:55, Jan Beulich wrote:

On 04.09.15 at 15:51,  wrote:

El 04/09/15 a les 14.25, Wei Liu ha escrit:

On Fri, Sep 04, 2015 at 02:08:50PM +0200, Roger Pau Monne wrote:

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 045f6ff..fe9504f 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -555,6 +555,29 @@ int arch_domain_create(struct domain *d, unsigned int

domcr_flags,

 d->domain_id);
  }
  
+if ( is_hvm_domain(d) )

+{
+uint32_t emulation_mask = (XEN_X86_EMU_LAPIC | XEN_X86_EMU_HPET |
+   XEN_X86_EMU_PMTIMER | XEN_X86_EMU_RTC |
+   XEN_X86_EMU_IOAPIC | XEN_X86_EMU_PIC |
+   XEN_X86_EMU_PMU | XEN_X86_EMU_VGA |
+   XEN_X86_EMU_IOMMU);

This is repetitive. Could you consolidate all these to

   #define XEN_X86_EMU_ALL ...

?

That sounds fine, I would place it in the public header where all the
XEN_X86_EMU_* are defined. I will wait for Andrew's opinion, since he
already acked this patch.

This doesn't belong in the public ABI, so if at all it should be added
there inside #ifdef __XEN__. Alternatively (and perhaps preferably)
this would go into another (internal) header.


Agreed.

Having an ALL mask will be useful for checking invalid values, but this 
logic is going to get a bit more complicated as soon as we see about 
introducing situations which permit the use of the vLAPIC, and I don't 
see the ALL mask being useful anywhere else.


I am not fussed either way.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [xen-4.5-testing test] 61309: regressions - trouble: blocked/broken/fail/pass

2015-09-04 Thread osstest service owner
flight 61309 xen-4.5-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/61309/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-debianhvm-amd64 19 guest-start/debianhvm.repeat fail 
REGR. vs. 60723

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-libvirt-vhd  3 host-install(3) broken REGR. vs. 60723
 test-amd64-i386-libvirt  14 guest-saverestore fail REGR. vs. 60723
 test-amd64-amd64-libvirt-vhd 13 guest-saverestore fail REGR. vs. 60723
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail like 60672
 test-armhf-armhf-xl-rtds 11 guest-start  fail   like 60695
 test-amd64-amd64-libvirt-raw  9 debian-di-installfail   like 60723
 test-amd64-i386-libvirt-raw   9 debian-di-installfail   like 60723
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail like 60723
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail like 60723

Tests which did not succeed, but are not blocking:
 test-amd64-i386-migrupgrade   1 build-check(1)   blocked  n/a
 test-amd64-amd64-migrupgrade  1 build-check(1)   blocked  n/a
 build-amd64-prev  5 xen-buildfail   never pass
 build-i386-prev   5 xen-buildfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-armhf-armhf-xl-qcow2 9 debian-di-installfail   never pass
 test-armhf-armhf-xl-vhd   9 debian-di-installfail   never pass
 test-armhf-armhf-xl-raw   9 debian-di-installfail   never pass
 test-armhf-armhf-libvirt-qcow2  9 debian-di-installfail never pass
 test-armhf-armhf-libvirt-raw  9 debian-di-installfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qcow2 9 debian-di-installfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-pair 21 guest-migrate/src_host/dst_host fail never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qcow2 11 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-pair 21 guest-migrate/src_host/dst_host fail never 
pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-amd64-amd64-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-amd64-i386-libvirt-vhd  11 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass

version targeted for testing:
 xen  ef89dc8c00087c8c1819e60bdad5527db70425e1
baseline version:
 xen  7f7642f778b78e8e204fc082ce03072bb26887c7

Last test of basis60723  2015-08-16 12:18:15 Z   19 days
Testing same since61309  2015-09-02 13:18:00 Z2 days1 attempts


People who touched revisions under test:
  Ian Campbell 
  Julien Grall 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-prev fail
 build-i386-prev  fail
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass

Re: [Xen-devel] PAGE_SIZE (64KB), while block driver 'struct request' deals with < PAGE_SIZE (up to 44Kb). Was:Re: [RFC] Support of non-indirect grant backend on 64KB guest

2015-09-04 Thread Julien Grall

Hi Konrad,

On 04/09/2015 18:32, Konrad Rzeszutek Wilk wrote:

On Fri, Sep 04, 2015 at 05:15:13PM +0100, Julien Grall wrote:

On 04/09/15 16:41, Konrad Rzeszutek Wilk wrote:

Maybe we could fall back to the previous plan of modifying xen-blkfront
for the moment?


Which afaic need to be reposted?


Right. Although I didn't see any comment on the patch. All the comments
was about the problem. Does it mean that you and Roger are "happy" with
the way it's done?


There were some #idef I think? And I recall seeing the segment limit being
advertised as PAGE_SIZE / 2?


There is no ifdef but a PAGE_SIZE / 2. I know about the latter and plan 
to replace it. It was only to have a quick patch to expose my problem.



I dug in the other block drivers to figure out what they
do when the underlaying storage cannot handle < PAGE_SIZE data.
And I couldn't find them. Which means this should be really dealt
on the drivers side (xen-blkfront) as an quirk.

Anyhow what I am going to do is - once it is reposted, force the
driver under x86 to work under this condition. That is disable
persistent support and only use 2048 .. and then drive some IO.
If all goes well I will send it in a git pull to Jens.


It will also depends on 64KB series which I plan to repost next week. I 
will send as follow-up.


To test it properly on x86 it will be necessary to drop on BUG_ON in the 
code which ensure that the extra req is never use for 4KB

(see BUG_ON((XEN_PAGE_SIZE == PAGE_SIZE) && require_extra_req)).

Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for 4.6 5/5] libxc: de-duplicate gpfns in populate_pfns

2015-09-04 Thread Andrew Cooper

On 04/09/15 19:32, Wei Liu wrote:

The original implementation didn't consider there can be same gpfns
present multiple times in the passed in array. The mechanism to prevent
populating same gpfn multiple times only works if the same gpfn appear
in different batch.

This bug is discovered by save / restore Linux 4.1 32 bit kernel, which
has several PTEs pointing to same gpfn. And gpfn pointed to by those
PTEs are populated in one batch by libxc.  When libxc calls
x86_pv_localise_page, the original implementation failed to detect
duplications in one batch.

The fix is to de-duplicate gpfns in populate_pfns.

Signed-off-by: Wei Liu 


The original version of populate_pfns() synchronously populated pfns 
when they were found referenced in PTEs ahead of the appropriate 
PAGE_DATA record arriving, but this proved to be very inefficient 
depends on the pagetable layout of the guest.


I agree that this is a bug.


---
  tools/libxc/xc_sr_restore.c | 19 ---
  1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/tools/libxc/xc_sr_restore.c b/tools/libxc/xc_sr_restore.c
index adb48da..09fe4c0 100644
--- a/tools/libxc/xc_sr_restore.c
+++ b/tools/libxc/xc_sr_restore.c
@@ -198,7 +198,7 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned count,
  xc_interface *xch = ctx->xch;
  xen_pfn_t *mfns = malloc(count * sizeof(*mfns)),
  *pfns = malloc(count * sizeof(*pfns));
-unsigned i, nr_pfns = 0;
+unsigned i, j, nr_pfns = 0;
  int rc = -1;
  
  if ( !mfns || !pfns )

@@ -209,14 +209,27 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned 
count,
  }
  
  for ( i = 0; i < count; ++i )

+pfns[i] = mfns[i] = INVALID_MFN;
+
+for ( i = 0; i < count; ++i )
  {
  if ( (!types || (types &&
   (types[i] != XEN_DOMCTL_PFINFO_XTAB &&
types[i] != XEN_DOMCTL_PFINFO_BROKEN))) &&
   !pfn_is_populated(ctx, original_pfns[i]) )
  {
-pfns[nr_pfns] = mfns[nr_pfns] = original_pfns[i];
-++nr_pfns;
+bool present = false;
+
+/* De-duplicate gpfns to avoid populating the same one twice */


Just pfns to match the rest of the code.  (I notice some other memory 
terminology needing cleaning up - I will formulate some patches when I 
am next in a position to do so.)



+for ( j = 0; j < nr_pfns; ++j )
+if ( pfns[j] == original_pfns[i] )
+present = true;


Adding this nested loop introduces O(N^2) behavior on what is typically 
1024-length inputs.


I recommend moving the pfn_set_populated() call from the subsequent loop 
into this loop, which will cause the pfn_is_populated() call in this 
loop condition to catch repeat populations even in the same batch.


The only way pfn_set_populated() can fail is out of memory, and any 
error anywhere in this function is fatal to migration, so there is no 
chance of proceeding with migration with a pfn marked as populated, but 
set to INVALID_MFN in the p2m.


~Andrew


+
+if ( !present )
+{
+pfns[nr_pfns] = mfns[nr_pfns] = original_pfns[i];
+++nr_pfns;
+}
  }
  }
  



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 15/16] xen/vtd: prevent from assign the device with shared rmrr

2015-09-04 Thread Tamas K Lengyel
On Fri, Sep 4, 2015 at 2:17 AM, Jan Beulich  wrote:

> >>> On 03.09.15 at 21:39,  wrote:
> > So this change in 4.6 prevents me from passing through devices that have
> > worked previously with VT-d:
> >
> > (XEN) [VT-D] cannot assign :00:1a.0 with shared RMRR at ae8a9000 for
> > Dom30.
> > (XEN) [VT-D] cannot assign :00:1d.0 with shared RMRR at ae8a9000 for
> > Dom31.
> >
> > The devices are the USB 2.0 devices on a DQ67SW motherboard:
> >
> > 00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset
> > Family USB Enhanced Host Controller #2 (rev 04)
> > 00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset
> > Family USB Enhanced Host Controller #1 (rev 04)
>
> Please don't top post. And I'm also puzzled by you sending this to
> Ian rather than the author.
>

Hm, I've just hit reply-all to the latest message I've found in the thread.


>
> Technically - Tiejun, should this perhaps be permitted in relaxed
> mode, at least until group assignment gets implemented? (Or
> Tamas, do you actually mean to assign these to _different_
> guests, considering the log fragment above?)
>

No, I actually want to assign them to the same domain. The domain creation
fails with either of those devices specified for passthrough whether they
are to be attached to the same domain or not.

Tamas
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for 4.6 2/5] libxc: migration v2 prefix Memory -> Memory (P2M)

2015-09-04 Thread Wei Liu
On Fri, Sep 04, 2015 at 09:40:55PM +0100, Andrew Cooper wrote:
> On 04/09/15 19:32, Wei Liu wrote:
> >The prefix "Memory" is confusing because the numbers shown after that
> >are referring to P2M entries. They have no bearing on how many pages a
> >domain actually owns or how many actual pages are processed.
> >
> >Signed-off-by: Wei Liu 
> >---
> >  tools/libxc/xc_sr_save.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> >diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c
> >index 58667af..bb7ff54 100644
> >--- a/tools/libxc/xc_sr_save.c
> >+++ b/tools/libxc/xc_sr_save.c
> >@@ -659,7 +659,7 @@ static int send_domain_memory_nonlive(struct 
> >xc_sr_context *ctx)
> >  if ( rc )
> >  goto err;
> >-xc_set_progress_prefix(xch, "Memory");
> >+xc_set_progress_prefix(xch, "Memory (P2M)");
> 
> This is incorrect for autotranslated guests.  I would recommend "Frames"
> instead.
> 

Fine by me of course.

Wei.

> ~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for 4.6 4/5] libxc: add assertion to avoid setting same bit more than once

2015-09-04 Thread Wei Liu
On Fri, Sep 04, 2015 at 09:43:16PM +0100, Andrew Cooper wrote:
> On 04/09/15 19:32, Wei Liu wrote:
> >Signed-off-by: Wei Liu 
> 
> This surely isn't bisectable with patch 5?  The change is fine (so
> Reviewed-by: Andrew Cooper ), provided you either
> reverse patches 4 and 5, or fold them together.
> 

Oh sure. I will reverse them.

Wei.

> ~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for 4.6 4/5] libxc: add assertion to avoid setting same bit more than once

2015-09-04 Thread Andrew Cooper

On 04/09/15 19:32, Wei Liu wrote:

Signed-off-by: Wei Liu 


This surely isn't bisectable with patch 5?  The change is fine (so 
Reviewed-by: Andrew Cooper ), provided you 
either reverse patches 4 and 5, or fold them together.


~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for 4.6 3/5] libxc: fix indentation

2015-09-04 Thread Andrew Cooper

On 04/09/15 19:32, Wei Liu wrote:

Signed-off-by: Wei Liu 


Reviewed-by: Andrew Cooper 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for 4.6 2/5] libxc: migration v2 prefix Memory -> Memory (P2M)

2015-09-04 Thread Andrew Cooper

On 04/09/15 19:32, Wei Liu wrote:

The prefix "Memory" is confusing because the numbers shown after that
are referring to P2M entries. They have no bearing on how many pages a
domain actually owns or how many actual pages are processed.

Signed-off-by: Wei Liu 
---
  tools/libxc/xc_sr_save.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c
index 58667af..bb7ff54 100644
--- a/tools/libxc/xc_sr_save.c
+++ b/tools/libxc/xc_sr_save.c
@@ -659,7 +659,7 @@ static int send_domain_memory_nonlive(struct xc_sr_context 
*ctx)
  if ( rc )
  goto err;
  
-xc_set_progress_prefix(xch, "Memory");

+xc_set_progress_prefix(xch, "Memory (P2M)");


This is incorrect for autotranslated guests.  I would recommend "Frames" 
instead.


~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for 4.6 1/5] libxc: clearer migration v2 debug message

2015-09-04 Thread Andrew Cooper

On 04/09/15 19:32, Wei Liu wrote:

Previous the message was like:

SAVE:
xc: detail: 32 bits, 3 levels
xc: detail: max_pfn 0xf, p2m_frames 1024
xc: detail: max_mfn 0x13

RESTORE:
xc: detail: max_mfn 0x13
xc: detail: 32 bits, 3 levels
xc: detail: Expanded p2m from 0 to 0xf

It's not immediately clear that the last line in restore messages was
referring to max_pfn. Change the debug message a bit to make that
clearer.


I am not sure that this is any clearer; the former means exactly the 
same although I suppose spelling out max_pfn is more consistent with the 
save side.


Reviewed-by: Andrew Cooper 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Linux 4.1 reports wrong number of pages to toolstack

2015-09-04 Thread Andrew Cooper

On 04/09/15 20:46, Wei Liu wrote:



When I looked at write_batch() I found some snippets that I thought to
be wrong. But I didn't what to make the judgement when I didn't have a
clear head.

write_batch() is a complicated function but it can't usefully be split any
further.  I would be happy to explain bits or expand the existing comments,
but it is also possible that it is buggy.


I think write_batch is correct. I overlooked one function call. I'm not
overly happy with the handling of balloon pages and the use of deferred
array in non-live transfer, but those things are not buggy in itself.


Handling of ballooned pages is broken at several layers.  This was 
covered in my talk at Seattle.  Fixing it is non-trivial.


The use of the deferred array is necessary for live migrates, and used 
in non-live migrates to avoid diverging the algorithm.  Nothing in the 
non-live side queries the deferred array (which itself is a contributory 
factor to the ballooning issue, as there is no interlock to prevent 
something else issuing population/depopoulation hypercalls on behalf of 
the paused domain).


~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Linux 4.1 reports wrong number of pages to toolstack

2015-09-04 Thread Wei Liu
On Fri, Sep 04, 2015 at 07:39:27PM +0100, Andrew Cooper wrote:
> 
> 
> On 04/09/15 12:35, Wei Liu wrote:
> >On Fri, Sep 04, 2015 at 10:35:52AM +0100, Andrew Cooper wrote:
> >>On 04/09/15 09:28, Jan Beulich wrote:
> >>On 04.09.15 at 05:38,  wrote:
> On 09/04/2015 02:40 AM, Wei Liu wrote:
> >This issue is exposed by the introduction of migration v2. The symptom 
> >is that
> >a guest with 32 bit 4.1 kernel can't be restored because it's asking for 
> >too
> >many pages.
> >
> >Note that all guests have 512MB memory, which means they have 131072 
> >pages.
> >
> >Both 3.14 tests [2] [3] get the correct number of pages.  Like:
> >
> > xc: detail: max_pfn 0x1, p2m_frames 256
> > ...
> > xc: detail: Memory: 2048/1310721%
> > ...
> >
> >However in both 4.1 [0] [1] the number of pages are quite wrong.
> >
> >4.1 32 bit:
> >
> > xc: detail: max_pfn 0xf, p2m_frames 1024
> > ...
> > xc: detail: Memory: 11264/10485761%
> > ...
> >
> >It thinks it has 4096MB memory.
> >
> >4.1 64 bit:
> >
> > xc: detail: max_pfn 0x3, p2m_frames 512
> > ...
> > xc: detail: Memory: 3072/2621441%
> > ...
> >
> >It thinks it has 1024MB memory.
> >
> >The total number of pages is determined in libxc by calling
> >xc_domain_nr_gpfns, which yanks shared_info->arch.max_pfn from
> >hypervisor. And that value is clearly touched by Linux in some way.
> Sure. shared_info->arch.max_pfn holds the number of pfns the p2m list
> can handle. This is not the memory size of the domain.
> 
> >I now think this is a bug in Linux kernel. The biggest suspect is the
> >introduction of linear P2M.  If you think this is a bug in toolstack,
> >please let me know.
> I absolutely think it is a toolstack bug. Even without the linear p2m
> things would go wrong in case a ballooned down guest would be migrated,
> as shared_info->arch.max_pfn would hold the upper limit of the guest
> in this case and not the current size.
> >>>I don't think this necessarily is a tool stack bug, at least not in
> >>>the sense implied above - since (afaik) migrating ballooned guests
> >>>(at least PV ones) has been working before, there ought to be
> >>>logic to skip ballooned pages (and I certainly recall having seen
> >>>migration slowly move up to e.g. 50% and the skip the other
> >>>half due to being ballooned, albeit that recollection certainly is
> >>>from before v2). And pages above the highest populated one
> >>>ought to be considered ballooned just as much. With the
> >>>information provided by Wei I don't think we can judge about
> >>>this, since it only shows the values the migration process starts
> >>>from, not when, why, or how it fails.
> >>Max pfn reported by migration v2 is max pfn, not the number of pages of RAM
> >>in the guest.
> >>
> >I understand that by looking at the code. Just the log itself
> >is very confusing.
> >
> >I propose we rename the log a bit. Maybe change "Memory" to "P2M" or
> >something else?
> 
> P2M would be wrong for HVM guests.  Memory was the same term used by the
> legacy code iirc.
> 
> "Frames" is probably the best term.
> 
> >
> >>It is used for the size of the bitmaps used by migration v2, including the
> >>logdirty op calls.
> >>
> >>All frames between 0 and max pfn will have their type queried, and acted
> >>upon appropriately, including doing nothing if the frame was ballooned out.
> >In short, do you think this is a bug in migration v2?
> 
> There is insufficient information in this thread to say either way. Maybe.
> Maybe a Linux kernel bug.
> 
> >
> >When I looked at write_batch() I found some snippets that I thought to
> >be wrong. But I didn't what to make the judgement when I didn't have a
> >clear head.
> 
> write_batch() is a complicated function but it can't usefully be split any
> further.  I would be happy to explain bits or expand the existing comments,
> but it is also possible that it is buggy.
> 

I think write_batch is correct. I overlooked one function call. I'm not
overly happy with the handling of balloon pages and the use of deferred
array in non-live transfer, but those things are not buggy in itself.

See my patch series for the real bug I discover. Gosh, took me a whole
day to identity the culprit.

Wei.

> ~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] What is the difference between XenAPI and XenAPI-Extensions

2015-09-04 Thread D'Mita Levy
Hello,

I am a student at FIU working on a school project so I am a not very
experienced. Right now I am trying to use the Xen API code I found at
https://github.com/xenserver/xenadmin to Snapshot a VM. The project I have
so far uses the XenAPI.VM functions and I have been successful in starting
and stopping a VM. As an example I perform those actions like this:

 XenRef vmRef = VM.get_by_uuid(_session, vmUUID);
 VM.start(_session, vmRef.opaque_ref, false, true);

My mentors would like me to try and clone a VM. I have copied the Run()
function from VMSnapshotCreateAction.cs and I tried to make a snapshot with
the following code:

   XenRef vmRef = VM.get_by_uuid(_session, GetUUIDByName(vmName));
   SnapshotType m_Type = SnapshotType.DISK; //hardcoded snapshot type
   string snapshot_name = "Testing_Snapshot";

   XenAPI.VM.async_snapshot(_session, vmRef.opaque_ref, snapshot_name);

I have two VM's. A Windows XP SP3 VM (No actual OS installed, just the base
VM) and an Ubuntu 14.04 Trusty Tahr VM (installed OS and running). The
Windows XP SP3 Snapshot gets created successfully but the Ubuntu snapshot
does not.

Upon furthure inspection I notice that XenAPI.VM works for start/stop but
XenAPI-Extensions.VM is needed to make snapshots - based on
VMSnapshotCreateAction.cs

Why is it that I need XenAPI-Extensions.VM to createa  vm reference to make
snapshots and cannot use XenAPI.VM? What is the difference between XenAPI
and Extensions? Finally, is it possible just to pull out the code so that I
can have the core functionality; it seems adding it to my simple WPF
project requires a ton of dependencies and references

Thanks,
D'Mita
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Linux 4.1 reports wrong number of pages to toolstack

2015-09-04 Thread Andrew Cooper



On 04/09/15 12:35, Wei Liu wrote:

On Fri, Sep 04, 2015 at 10:35:52AM +0100, Andrew Cooper wrote:

On 04/09/15 09:28, Jan Beulich wrote:

On 04.09.15 at 05:38,  wrote:

On 09/04/2015 02:40 AM, Wei Liu wrote:

This issue is exposed by the introduction of migration v2. The symptom is that
a guest with 32 bit 4.1 kernel can't be restored because it's asking for too
many pages.

Note that all guests have 512MB memory, which means they have 131072 pages.

Both 3.14 tests [2] [3] get the correct number of pages.  Like:

 xc: detail: max_pfn 0x1, p2m_frames 256
 ...
 xc: detail: Memory: 2048/1310721%
 ...

However in both 4.1 [0] [1] the number of pages are quite wrong.

4.1 32 bit:

 xc: detail: max_pfn 0xf, p2m_frames 1024
 ...
 xc: detail: Memory: 11264/10485761%
 ...

It thinks it has 4096MB memory.

4.1 64 bit:

 xc: detail: max_pfn 0x3, p2m_frames 512
 ...
 xc: detail: Memory: 3072/2621441%
 ...

It thinks it has 1024MB memory.

The total number of pages is determined in libxc by calling
xc_domain_nr_gpfns, which yanks shared_info->arch.max_pfn from
hypervisor. And that value is clearly touched by Linux in some way.

Sure. shared_info->arch.max_pfn holds the number of pfns the p2m list
can handle. This is not the memory size of the domain.


I now think this is a bug in Linux kernel. The biggest suspect is the
introduction of linear P2M.  If you think this is a bug in toolstack,
please let me know.

I absolutely think it is a toolstack bug. Even without the linear p2m
things would go wrong in case a ballooned down guest would be migrated,
as shared_info->arch.max_pfn would hold the upper limit of the guest
in this case and not the current size.

I don't think this necessarily is a tool stack bug, at least not in
the sense implied above - since (afaik) migrating ballooned guests
(at least PV ones) has been working before, there ought to be
logic to skip ballooned pages (and I certainly recall having seen
migration slowly move up to e.g. 50% and the skip the other
half due to being ballooned, albeit that recollection certainly is

>from before v2). And pages above the highest populated one

ought to be considered ballooned just as much. With the
information provided by Wei I don't think we can judge about
this, since it only shows the values the migration process starts
from, not when, why, or how it fails.

Max pfn reported by migration v2 is max pfn, not the number of pages of RAM
in the guest.


I understand that by looking at the code. Just the log itself
is very confusing.

I propose we rename the log a bit. Maybe change "Memory" to "P2M" or
something else?


P2M would be wrong for HVM guests.  Memory was the same term used by the 
legacy code iirc.


"Frames" is probably the best term.




It is used for the size of the bitmaps used by migration v2, including the
logdirty op calls.

All frames between 0 and max pfn will have their type queried, and acted
upon appropriately, including doing nothing if the frame was ballooned out.

In short, do you think this is a bug in migration v2?


There is insufficient information in this thread to say either way. 
Maybe.  Maybe a Linux kernel bug.




When I looked at write_batch() I found some snippets that I thought to
be wrong. But I didn't what to make the judgement when I didn't have a
clear head.


write_batch() is a complicated function but it can't usefully be split 
any further.  I would be happy to explain bits or expand the existing 
comments, but it is also possible that it is buggy.


~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH for 4.6 0/5] Migration v2 fix

2015-09-04 Thread Wei Liu
Wei Liu (5):
  libxc: clearer migration v2 debug message
  libxc: migration v2 prefix Memory -> Memory (P2M)
  libxc: fix indentation
  libxc: add assertion to avoid setting same bit more than once
  libxc: de-duplicate gpfns in populate_pfns

 tools/libxc/xc_sr_restore.c| 20 +---
 tools/libxc/xc_sr_restore_x86_pv.c |  4 ++--
 tools/libxc/xc_sr_save.c   |  2 +-
 3 files changed, 20 insertions(+), 6 deletions(-)

-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH for 4.6 1/5] libxc: clearer migration v2 debug message

2015-09-04 Thread Wei Liu
Previous the message was like:

SAVE:
xc: detail: 32 bits, 3 levels
xc: detail: max_pfn 0xf, p2m_frames 1024
xc: detail: max_mfn 0x13

RESTORE:
xc: detail: max_mfn 0x13
xc: detail: 32 bits, 3 levels
xc: detail: Expanded p2m from 0 to 0xf

It's not immediately clear that the last line in restore messages was
referring to max_pfn. Change the debug message a bit to make that
clearer.

Signed-off-by: Wei Liu 
---
 tools/libxc/xc_sr_restore_x86_pv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libxc/xc_sr_restore_x86_pv.c 
b/tools/libxc/xc_sr_restore_x86_pv.c
index ef26c64..c65a2f1 100644
--- a/tools/libxc/xc_sr_restore_x86_pv.c
+++ b/tools/libxc/xc_sr_restore_x86_pv.c
@@ -66,7 +66,7 @@ static int expand_p2m(struct xc_sr_context *ctx, unsigned 
long max_pfn)
 for ( i = (old_end_frame ? old_end_frame + 1 : 0); i <= end_frame; ++i )
 ctx->x86_pv.p2m_pfns[i] = INVALID_MFN;
 
-DPRINTF("Expanded p2m from %#lx to %#lx", old_max, max_pfn);
+DPRINTF("Changed max_pfn from %#lx to %#lx", old_max, max_pfn);
 return 0;
 }
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH for 4.6 3/5] libxc: fix indentation

2015-09-04 Thread Wei Liu
Signed-off-by: Wei Liu 
---
 tools/libxc/xc_sr_restore_x86_pv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libxc/xc_sr_restore_x86_pv.c 
b/tools/libxc/xc_sr_restore_x86_pv.c
index c65a2f1..bc604b3 100644
--- a/tools/libxc/xc_sr_restore_x86_pv.c
+++ b/tools/libxc/xc_sr_restore_x86_pv.c
@@ -970,7 +970,7 @@ static int x86_pv_localise_page(struct xc_sr_context *ctx,
 }
 
 if ( to_populate && populate_pfns(ctx, to_populate, pfns, NULL) )
-return -1;
+return -1;
 
 for ( i = 0; i < (PAGE_SIZE / sizeof(uint64_t)); ++i )
 {
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH for 4.6 2/5] libxc: migration v2 prefix Memory -> Memory (P2M)

2015-09-04 Thread Wei Liu
The prefix "Memory" is confusing because the numbers shown after that
are referring to P2M entries. They have no bearing on how many pages a
domain actually owns or how many actual pages are processed.

Signed-off-by: Wei Liu 
---
 tools/libxc/xc_sr_save.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c
index 58667af..bb7ff54 100644
--- a/tools/libxc/xc_sr_save.c
+++ b/tools/libxc/xc_sr_save.c
@@ -659,7 +659,7 @@ static int send_domain_memory_nonlive(struct xc_sr_context 
*ctx)
 if ( rc )
 goto err;
 
-xc_set_progress_prefix(xch, "Memory");
+xc_set_progress_prefix(xch, "Memory (P2M)");
 
 rc = send_all_pages(ctx);
 if ( rc )
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH for 4.6 5/5] libxc: de-duplicate gpfns in populate_pfns

2015-09-04 Thread Wei Liu
The original implementation didn't consider there can be same gpfns
present multiple times in the passed in array. The mechanism to prevent
populating same gpfn multiple times only works if the same gpfn appear
in different batch.

This bug is discovered by save / restore Linux 4.1 32 bit kernel, which
has several PTEs pointing to same gpfn. And gpfn pointed to by those
PTEs are populated in one batch by libxc.  When libxc calls
x86_pv_localise_page, the original implementation failed to detect
duplications in one batch.

The fix is to de-duplicate gpfns in populate_pfns.

Signed-off-by: Wei Liu 
---
 tools/libxc/xc_sr_restore.c | 19 ---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/tools/libxc/xc_sr_restore.c b/tools/libxc/xc_sr_restore.c
index adb48da..09fe4c0 100644
--- a/tools/libxc/xc_sr_restore.c
+++ b/tools/libxc/xc_sr_restore.c
@@ -198,7 +198,7 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned count,
 xc_interface *xch = ctx->xch;
 xen_pfn_t *mfns = malloc(count * sizeof(*mfns)),
 *pfns = malloc(count * sizeof(*pfns));
-unsigned i, nr_pfns = 0;
+unsigned i, j, nr_pfns = 0;
 int rc = -1;
 
 if ( !mfns || !pfns )
@@ -209,14 +209,27 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned 
count,
 }
 
 for ( i = 0; i < count; ++i )
+pfns[i] = mfns[i] = INVALID_MFN;
+
+for ( i = 0; i < count; ++i )
 {
 if ( (!types || (types &&
  (types[i] != XEN_DOMCTL_PFINFO_XTAB &&
   types[i] != XEN_DOMCTL_PFINFO_BROKEN))) &&
  !pfn_is_populated(ctx, original_pfns[i]) )
 {
-pfns[nr_pfns] = mfns[nr_pfns] = original_pfns[i];
-++nr_pfns;
+bool present = false;
+
+/* De-duplicate gpfns to avoid populating the same one twice */
+for ( j = 0; j < nr_pfns; ++j )
+if ( pfns[j] == original_pfns[i] )
+present = true;
+
+if ( !present )
+{
+pfns[nr_pfns] = mfns[nr_pfns] = original_pfns[i];
+++nr_pfns;
+}
 }
 }
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH for 4.6 4/5] libxc: add assertion to avoid setting same bit more than once

2015-09-04 Thread Wei Liu
Signed-off-by: Wei Liu 
---
 tools/libxc/xc_sr_restore.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/libxc/xc_sr_restore.c b/tools/libxc/xc_sr_restore.c
index df885b6..adb48da 100644
--- a/tools/libxc/xc_sr_restore.c
+++ b/tools/libxc/xc_sr_restore.c
@@ -181,6 +181,7 @@ static int pfn_set_populated(struct xc_sr_context *ctx, 
xen_pfn_t pfn)
 ctx->restore.max_populated_pfn = new_max;
 }
 
+assert(!test_bit(pfn, ctx->restore.populated_pfns));
 set_bit(pfn, ctx->restore.populated_pfns);
 
 return 0;
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [xen-unstable test] 61306: regressions - FAIL

2015-09-04 Thread osstest service owner
flight 61306 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/61306/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-multivcpu 16 guest-start/debian.repeat fail REGR. vs. 61059

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 16 
guest-start/debianhvm.repeat fail REGR. vs. 61059
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 15 guest-localmigrate.2 
fail blocked in 61059
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 9 debian-hvm-install fail 
like 61059
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail like 61059
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail like 61059

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-armhf-armhf-xl-raw   9 debian-di-installfail   never pass
 test-armhf-armhf-libvirt-raw  9 debian-di-installfail   never pass
 test-armhf-armhf-xl-qcow2 9 debian-di-installfail   never pass
 test-armhf-armhf-xl-vhd   9 debian-di-installfail   never pass
 test-armhf-armhf-libvirt-vhd  9 debian-di-installfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-qcow2  9 debian-di-installfail never pass
 test-amd64-amd64-libvirt-pair 21 guest-migrate/src_host/dst_host fail never 
pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-pair 21 guest-migrate/src_host/dst_host fail never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-raw  11 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-raw 11 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-amd64-i386-libvirt-vhd  11 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-amd64-i386-libvirt-qcow2 11 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-qcow2 11 migrate-support-checkfail never pass

version targeted for testing:
 xen  afc13fe5e21d18c09e44f8ae6f7f4484e9f1de7f
baseline version:
 xen  801ab48e5556cb54f67e3cb57f077f47e8663ced

Last test of basis61059  2015-08-30 15:26:08 Z5 days
Failing since 61248  2015-09-01 10:13:44 Z3 days2 attempts
Testing same since61306  2015-09-02 13:01:17 Z2 days1 attempts


People who touched revisions under test:
  David Scott 
  Doug Goldstein 
  George Dunlap 
  Ian Campbell 
  Ian Jackson 
  Jan Beulich 
  Jim Fehlig 
  Jonathan Creekmore 
  Julien Grall 
  Wei Liu 
  Zhigang Wang 

jobs:
 buil

Re: [Xen-devel] PAGE_SIZE (64KB), while block driver 'struct request' deals with < PAGE_SIZE (up to 44Kb). Was:Re: [RFC] Support of non-indirect grant backend on 64KB guest

2015-09-04 Thread Konrad Rzeszutek Wilk
On Fri, Sep 04, 2015 at 05:15:13PM +0100, Julien Grall wrote:
> On 04/09/15 16:41, Konrad Rzeszutek Wilk wrote:
> >> Maybe we could fall back to the previous plan of modifying xen-blkfront
> >> for the moment?
> > 
> > Which afaic need to be reposted?
> 
> Right. Although I didn't see any comment on the patch. All the comments
> was about the problem. Does it mean that you and Roger are "happy" with
> the way it's done?

There were some #idef I think? And I recall seeing the segment limit being
advertised as PAGE_SIZE / 2?


I dug in the other block drivers to figure out what they
do when the underlaying storage cannot handle < PAGE_SIZE data.
And I couldn't find them. Which means this should be really dealt
on the drivers side (xen-blkfront) as an quirk.
 
Anyhow what I am going to do is - once it is reposted, force the
driver under x86 to work under this condition. That is disable
persistent support and only use 2048 .. and then drive some IO.
If all goes well I will send it in a git pull to Jens.
   

> 
> Regards,
> 
> -- 
> Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] xen-blkback: free requests on disconnection

2015-09-04 Thread Konrad Rzeszutek Wilk
On Fri, Sep 04, 2015 at 02:51:10PM +0100, Julien Grall wrote:
> Hi Roger,
> 
> On 04/09/15 11:08, Roger Pau Monne wrote:
> > Request allocation has been moved to connect_ring, which is called every
> > time blkback connects to the frontend (this can happen multiple times during
> > a blkback instance life cycle). On the other hand, request freeing has not
> > been moved, so it's only called when destroying the backend instance. Due to
> > this mismatch, blkback can allocate the request pool multiple times, without
> > freeing it.
> > 
> > In order to fix it, move the freeing of requests to xen_blkif_disconnect to
> > restore the symmetry between request allocation and freeing.
> > 
> > Reported-by: Julien Grall 
> > Signed-off-by: Roger Pau Monné 
> > Cc: Julien Grall 
> > Cc: Konrad Rzeszutek Wilk 
> > Cc: Boris Ostrovsky 
> > Cc: David Vrabel 
> > Cc: xen-de...@lists.xenproject.org
> 
> The patch is fixing my problem when using UEFI in the guest. Thank you!
> 
> Tested-by: Julien Grall 

Testing it now for regressions and should be sending an git pull next week
for it.

Thanks!
> 
> Regards,
> 
> -- 
> Julien Grall
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v5 4/5] x86/pvh: Handle hypercalls for 32b PVH guests

2015-09-04 Thread Boris Ostrovsky
Signed-off-by: Boris Ostrovsky 
Reviewed-by: Jan Beulich 
---
 xen/arch/x86/hvm/hvm.c | 36 ++--
 1 file changed, 30 insertions(+), 6 deletions(-)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 6f6cadc..ff94a9c 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -5113,7 +5113,6 @@ static hvm_hypercall_t *const 
hvm_hypercall32_table[NR_hypercalls] = {
 [ __HYPERVISOR_arch_1 ] = (hvm_hypercall_t *)paging_domctl_continuation
 };
 
-/* PVH 32bitfixme. */
 static hvm_hypercall_t *const pvh_hypercall64_table[NR_hypercalls] = {
 HYPERCALL(platform_op),
 HYPERCALL(memory_op),
@@ -5133,6 +5132,30 @@ static hvm_hypercall_t *const 
pvh_hypercall64_table[NR_hypercalls] = {
 [ __HYPERVISOR_arch_1 ] = (hvm_hypercall_t *)paging_domctl_continuation
 };
 
+extern int compat_mmuext_op(XEN_GUEST_HANDLE_PARAM(void) cmp_uops,
+unsigned int count,
+XEN_GUEST_HANDLE_PARAM(uint) pdone,
+unsigned int foreigndom);
+static hvm_hypercall_t *const pvh_hypercall32_table[NR_hypercalls] = {
+HYPERCALL(platform_op),
+COMPAT_CALL(memory_op),
+HYPERCALL(xen_version),
+HYPERCALL(console_io),
+[ __HYPERVISOR_grant_table_op ]  =
+(hvm_hypercall_t *)hvm_grant_table_op_compat32,
+COMPAT_CALL(vcpu_op),
+COMPAT_CALL(mmuext_op),
+HYPERCALL(xsm_op),
+COMPAT_CALL(sched_op),
+HYPERCALL(event_channel_op),
+[ __HYPERVISOR_physdev_op ] = (hvm_hypercall_t *)hvm_physdev_op_compat32,
+HYPERCALL(hvm_op),
+HYPERCALL(sysctl),
+HYPERCALL(domctl),
+HYPERCALL(xenpmu_op),
+[ __HYPERVISOR_arch_1 ] = (hvm_hypercall_t *)paging_domctl_continuation
+};
+
 extern const uint8_t hypercall_args_table[], compat_hypercall_args_table[];
 
 int hvm_do_hypercall(struct cpu_user_regs *regs)
@@ -5163,8 +5186,8 @@ int hvm_do_hypercall(struct cpu_user_regs *regs)
 return viridian_hypercall(regs);
 
 if ( (eax >= NR_hypercalls) ||
- (is_pvh_domain(currd) ? !pvh_hypercall64_table[eax]
-   : !hvm_hypercall32_table[eax]) )
+ !(is_pvh_domain(currd) ? pvh_hypercall32_table[eax]
+: hvm_hypercall32_table[eax]) )
 {
 regs->eax = -ENOSYS;
 return HVM_HCALL_completed;
@@ -5219,8 +5242,6 @@ int hvm_do_hypercall(struct cpu_user_regs *regs)
 }
 #endif
 }
-else if ( unlikely(is_pvh_domain(currd)) )
-regs->_eax = -ENOSYS; /* PVH 32bitfixme. */
 else
 {
 unsigned int ebx = regs->_ebx;
@@ -5246,7 +5267,10 @@ int hvm_do_hypercall(struct cpu_user_regs *regs)
 }
 #endif
 
-regs->_eax = hvm_hypercall32_table[eax](ebx, ecx, edx, esi, edi, ebp);
+regs->_eax = (is_pvh_vcpu(curr)
+  ? pvh_hypercall32_table
+  : hvm_hypercall32_table)[eax](ebx, ecx, edx,
+esi, edi, ebp);
 
 #ifndef NDEBUG
 if ( !curr->arch.hvm_vcpu.hcall_preempted )
-- 
1.8.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v5 1/5] x86/pvh: Set 32b PVH guest mode in XEN_DOMCTL_set_address_size

2015-09-04 Thread Boris Ostrovsky
Signed-off-by: Boris Ostrovsky 
Reviewed-by: Jan Beulich 
---
 xen/arch/x86/domain.c | 27 ---
 xen/arch/x86/hvm/hvm.c| 24 +++-
 xen/arch/x86/hvm/vmx/vmcs.c   |  2 +-
 xen/arch/x86/hvm/vmx/vmx.c| 19 +++
 xen/include/asm-x86/hvm/hvm.h |  2 ++
 5 files changed, 61 insertions(+), 13 deletions(-)

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 045f6ff..7fa8b9c 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -366,7 +366,11 @@ int switch_native(struct domain *d)
 for_each_vcpu( d, v )
 {
 free_compat_arg_xlat(v);
-release_compat_l4(v);
+
+if ( !is_pvh_domain(d) )
+release_compat_l4(v);
+else
+hvm_set_mode(v, 8);
 }
 
 return 0;
@@ -377,25 +381,26 @@ int switch_compat(struct domain *d)
 struct vcpu *v;
 int rc;
 
-if ( is_pvh_domain(d) )
-{
-printk(XENLOG_G_INFO
-   "Xen currently does not support 32bit PVH guests\n");
-return -EINVAL;
-}
-
 if ( !may_switch_mode(d) )
 return -EACCES;
 if ( is_pv_32bit_domain(d) )
 return 0;
 
-d->arch.is_32bit_pv = d->arch.has_32bit_shinfo = 1;
+d->arch.has_32bit_shinfo = 1;
+if ( is_pv_domain(d) )
+d->arch.is_32bit_pv = 1;
 
 for_each_vcpu( d, v )
 {
 rc = setup_compat_arg_xlat(v);
 if ( !rc )
-rc = setup_compat_l4(v);
+{
+if ( !is_pvh_domain(d) )
+rc = setup_compat_l4(v);
+else
+rc = hvm_set_mode(v, 4);
+}
+
 if ( rc )
 goto undo_and_fail;
 }
@@ -410,7 +415,7 @@ int switch_compat(struct domain *d)
 {
 free_compat_arg_xlat(v);
 
-if ( !pagetable_is_null(v->arch.guest_table) )
+if ( !is_pvh_domain(d) && !pagetable_is_null(v->arch.guest_table) )
 release_compat_l4(v);
 }
 
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 615fa89..90ba676 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -2424,7 +2424,6 @@ int hvm_vcpu_initialise(struct vcpu *v)
 
 if ( is_pvh_domain(d) )
 {
-v->arch.hvm_vcpu.hcall_64bit = 1;/* PVH 32bitfixme. */
 /* This is for hvm_long_mode_enabled(v). */
 v->arch.hvm_vcpu.guest_efer = EFER_LMA | EFER_LME;
 return 0;
@@ -6825,6 +6824,29 @@ bool_t altp2m_vcpu_emulate_ve(struct vcpu *v)
 return 0;
 }
 
+int hvm_set_mode(struct vcpu *v, int mode)
+{
+
+switch ( mode )
+{
+case 4:
+v->arch.hvm_vcpu.guest_efer &= ~(EFER_LMA | EFER_LME);
+break;
+case 8:
+v->arch.hvm_vcpu.guest_efer |= (EFER_LMA | EFER_LME);
+break;
+default:
+return -EOPNOTSUPP;
+}
+
+hvm_update_guest_efer(v);
+
+if ( hvm_funcs.set_mode )
+return hvm_funcs.set_mode(v, mode);
+
+return 0;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index a0a97e7..08f2078 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -1160,7 +1160,7 @@ static int construct_vmcs(struct vcpu *v)
 __vmwrite(GUEST_FS_AR_BYTES, 0xc093);
 __vmwrite(GUEST_GS_AR_BYTES, 0xc093);
 if ( is_pvh_domain(d) )
-/* CS.L == 1, exec, read/write, accessed. PVH 32bitfixme. */
+/* CS.L == 1, exec, read/write, accessed. */
 __vmwrite(GUEST_CS_AR_BYTES, 0xa09b);
 else
 __vmwrite(GUEST_CS_AR_BYTES, 0xc09b); /* exec/read, accessed */
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 2582cdd..bbec0e8 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -1886,6 +1886,24 @@ static bool_t vmx_vcpu_emulate_ve(struct vcpu *v)
 return rc;
 }
 
+static int vmx_set_mode(struct vcpu *v, int mode)
+{
+unsigned long attr;
+
+if ( !is_pvh_vcpu(v) )
+return 0;
+
+ASSERT((mode == 4) || (mode == 8));
+
+attr = (mode == 4) ? 0xc09b : 0xa09b;
+
+vmx_vmcs_enter(v);
+__vmwrite(GUEST_CS_AR_BYTES, attr);
+vmx_vmcs_exit(v);
+
+return 0;
+}
+
 static struct hvm_function_table __initdata vmx_function_table = {
 .name = "VMX",
 .cpu_up_prepare   = vmx_cpu_up_prepare,
@@ -1945,6 +1963,7 @@ static struct hvm_function_table __initdata 
vmx_function_table = {
 .hypervisor_cpuid_leaf = vmx_hypervisor_cpuid_leaf,
 .enable_msr_exit_interception = vmx_enable_msr_exit_interception,
 .is_singlestep_supported = vmx_is_singlestep_supported,
+.set_mode = vmx_set_mode,
 .altp2m_vcpu_update_p2m = vmx_vcpu_update_eptp,
 .altp2m_vcpu_update_vmfunc_ve = vmx_vcpu_update_vmfunc_ve,
 .altp2m_vcpu_emulate_ve = vmx_vcpu_emulate_ve,
diff --git a/xen/include/asm-x86/hvm/hvm.h b/xen/include/asm-x86/hvm/hvm.h
index 68b216c..c21a768 100644
--- a/xen/include/asm-x86/hvm/hvm.h
+++ b/xen/include/asm-x86/hvm/hvm.h
@

[Xen-devel] [PATCH v5 0/5] 32-bit domU PVH support

2015-09-04 Thread Boris Ostrovsky
32-bit PVH-classic support

Changes in v5:
* Added patch 2 that prevents a 32-bit PVH guest from turning off PAE

Changes in v4:
* Add xenpmu_op hypercall to pvh_hypercall32_table[] (patch 3)
* Adjust 'is_pvh_domain(currd)' test to match a similar test further in the
  routine (patch 3)

Changes in v3:
* Swapped patches 1 and 2
* Added is_pvh_32bit_domain() macro
* Dropped a few unnecessary tests for 32b mode
* Added error for unsupported modes
* Added changes to switch_native() similar to those in switch_compat()
* Fully declared compat_mmuext_op() in hvm.c

Boris Ostrovsky (5):
  x86/pvh: Set 32b PVH guest mode in XEN_DOMCTL_set_address_size
  x86/pvh: Do not allow 32-bit PVH guests to clear CR4's PAE bit
  x86/compat: Test both PV and PVH guests for compat mode
  x86/pvh: Handle hypercalls for 32b PVH guests
  libxc/x86/pvh: Allow creation of 32b PVH guests

 tools/libxc/xc_dom_x86.c  | 32 +-
 xen/arch/x86/domain.c | 33 +++
 xen/arch/x86/domctl.c |  5 +--
 xen/arch/x86/hvm/hvm.c| 76 ---
 xen/arch/x86/hvm/vmx/vmcs.c   |  2 +-
 xen/arch/x86/hvm/vmx/vmx.c| 19 +++
 xen/include/asm-x86/domain.h  |  1 +
 xen/include/asm-x86/hvm/hvm.h |  2 ++
 8 files changed, 125 insertions(+), 45 deletions(-)

-- 
1.8.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v5 3/5] x86/compat: Test both PV and PVH guests for compat mode

2015-09-04 Thread Boris Ostrovsky
Add is_pvh_32bit_domain() macro and use it alongside is_pv_32bit_domain() where
necessary.

Since PVH guests cannot change execution mode, has_32bit_shinfo is a good
indicator of whether the guest is PVH and 32-bit.

Signed-off-by: Boris Ostrovsky 
---
 xen/arch/x86/domain.c| 6 +++---
 xen/arch/x86/domctl.c| 5 +++--
 xen/include/asm-x86/domain.h | 1 +
 3 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 7fa8b9c..327b13f 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -358,7 +358,7 @@ int switch_native(struct domain *d)
 
 if ( !may_switch_mode(d) )
 return -EACCES;
-if ( !is_pv_32bit_domain(d) )
+if ( !is_pv_32bit_domain(d) && !is_pvh_32bit_domain(d) )
 return 0;
 
 d->arch.is_32bit_pv = d->arch.has_32bit_shinfo = 0;
@@ -383,7 +383,7 @@ int switch_compat(struct domain *d)
 
 if ( !may_switch_mode(d) )
 return -EACCES;
-if ( is_pv_32bit_domain(d) )
+if ( is_pv_32bit_domain(d) || is_pvh_32bit_domain(d) )
 return 0;
 
 d->arch.has_32bit_shinfo = 1;
@@ -777,7 +777,7 @@ int arch_set_info_guest(
 
 /* The context is a compat-mode one if the target domain is compat-mode;
  * we expect the tools to DTRT even in compat-mode callers. */
-compat = is_pv_32bit_domain(d);
+compat = is_pv_32bit_domain(d) || is_pvh_32bit_domain(d);
 
 #define c(fld) (compat ? (c.cmp->fld) : (c.nat->fld))
 flags = c(flags);
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index bf62a88..6172c0d 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -349,7 +349,8 @@ long arch_do_domctl(
 
 case XEN_DOMCTL_get_address_size:
 domctl->u.address_size.size =
-is_pv_32bit_domain(d) ? 32 : BITS_PER_LONG;
+(is_pv_32bit_domain(d) || is_pvh_32bit_domain(d)) ?
+32 : BITS_PER_LONG;
 copyback = 1;
 break;
 
@@ -1203,7 +1204,7 @@ void arch_get_info_guest(struct vcpu *v, 
vcpu_guest_context_u c)
 {
 unsigned int i;
 const struct domain *d = v->domain;
-bool_t compat = is_pv_32bit_domain(d);
+bool_t compat = is_pv_32bit_domain(d) || is_pvh_32bit_domain(d);
 #define c(fld) (!compat ? (c.nat->fld) : (c.cmp->fld))
 
 if ( !is_pv_domain(d) )
diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h
index 0fce09e..6f73d08 100644
--- a/xen/include/asm-x86/domain.h
+++ b/xen/include/asm-x86/domain.h
@@ -14,6 +14,7 @@
 #define has_32bit_shinfo(d)((d)->arch.has_32bit_shinfo)
 #define is_pv_32bit_domain(d)  ((d)->arch.is_32bit_pv)
 #define is_pv_32bit_vcpu(v)(is_pv_32bit_domain((v)->domain))
+#define is_pvh_32bit_domain(d) (is_pvh_domain(d) && has_32bit_shinfo(d))
 
 #define is_hvm_pv_evtchn_domain(d) (has_hvm_container_domain(d) && \
 d->arch.hvm_domain.irq.callback_via_type == HVMIRQ_callback_vector)
-- 
1.8.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [BUG] Xen live migration dom0 memory

2015-09-04 Thread Ian Campbell
On Fri, 2015-09-04 at 17:38 +0100, Ian Campbell wrote:
> On Fri, 2015-09-04 at 17:04 +0100, Ian Campbell wrote:
> > On Tue, 2015-07-07 at 17:26 +0200, Epiontis IT wrote:
> > > Hello,
> > > 
> > > I already posted on the xen-users list and Ian told me to post a bug 
> > > report here. We have the following problem:
> > > 
> > > When I start a migration with "xl migrate  " the 
> > > destination machine sets up a vm "--incoming" and seems to sync 
> > > memory because "xl list" shows "Mem" for the incoming vm going up 
> > > stepwise until 2048. After that though the vm doesn't get launched. 
> > > The 
> > > 
> > > vm is frozen for about a minute, the dom0 begins swapping out the 2GB 
> > > 
> > > to 
> > > disk (because it only has 512M available for itself) and a process
> > 
> > I spoke with Ian Jackson about this this afternoon and we spent some
> > staring at the pmap, and we are now both very perplexed...
> > 
> > So I've just tried to repro this.
> > 
> > I installed latest 4.5-testing and a 512M dom0 (which as it happens is 
> > also
> > what our automated test system uses). I then migrated a 3GB domU with 
> > "xl
> > migrate  localhost" and it completed successfully. I had to do
> > localhost migrate due to only having one box, but I don't think this 
> > should
> > matter.
> >  
> > I observed the save helper with pmap and never saw any large 
> > allocations.
> > 
> > This matches the behaviour that both Ian and I expected.
> > 
> > Your logs show you are running 4.5.1-rc1. I've looked over
> > git log 4.5.1-rc1..origin/staging-4.5 -- tools/libx[cl]/
> > and I don't see anything like a fix for a leak etc.
> > 
> > Next step is to revert my test box from staging-4.5 to 4.5.1-rc1 and 
> > see
> > what changes.
> 
> Nothing changed.
> 
> I then tried building this tree with 
> 
> EXTRA_CFLAGS_XEN_TOOLS="-g -O2 -fstack-protector-strong -Wformat 
> -Werror=format-security" APPEND_CPPFLAGS="-D_FORTIFY_SOURCE=2" 
> APPEND_LDFLAGS="-Wl,-z,relro" 
> 
> which is how the Debian package is built, still no change.
> 
> My next step was to build patched with the Debian patches, still no 
> change.

I see in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=797205 that 4.4
also demonstrates this for you.

So I have now tried the packaged versions of Xen 4.4.1 + Linux 3.16 (from
Jessie) and Xen 4.4.1 + Linux 4.1 (from jessie-backports) and still no luck
reproducing the issue.

I'm afraid at this point I'm giving up. I think the most practical next
step would be for you to arrange run the restore helper under valgrind as I
describe below.

Ian.

> 
> So I'm at even more of a loss than I was before :-/
> 
> Your report said you were running Debian 8.0 (Jessie) but with Xen 4.5.1
> -rc1, while Jessie only has 4.4.1.
> 
> Was 4.5.1-rc1 from experimental or built yourself? Did you install 
> anything
> else from Sid/Experimental (or Stretch)?
> 
> If not from the packages in experimental then how did you install Xen?
> 
> If you can still reproduce then you could try installing the latest
> valgrind, move libxl-save-helper aside and replace it with a script which
> runs the tool under valgrind.
> 
> Modern valgrind understands many Xen hypercalls etc and so is quite 
> useful
> even on the Xen toolstack binaries.
> 
> Ian.
> 
> > 
> > Ian.
> > 
> > > 
> > > /usr/lib/xen-4.5/bin/libxl-save-helper --restore-domain
> > > 
> > > takes about 90% of the dom0 memory. After that the vm ist launched, 
> > > memory consumption goes back to normal, swap space is freed.
> > > 
> > > I attached several log and output files in xen_bug_report.txt. In 
> > > particular Ian wanted to see the pmap of the save helper process 
> > > which 
> > > you can find at the end of the file taken during several phases of 
> > > the 
> > > migration process.
> > > 
> > > 
> > > Thank you,
> > > Alex
> > > 
> > > ___
> > > Xen-devel mailing list
> > > Xen-devel@lists.xen.org
> > > http://lists.xen.org/xen-devel
> > 
> > ___
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v5 2/5] x86/pvh: Do not allow 32-bit PVH guests to clear CR4's PAE bit

2015-09-04 Thread Boris Ostrovsky
.. since we only support 32-bit PV(H) guests in PAE mode.

Signed-off-by: Boris Ostrovsky 
---
 xen/arch/x86/hvm/hvm.c | 16 
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 90ba676..6f6cadc 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -3524,11 +3524,19 @@ int hvm_set_cr4(unsigned long value, bool_t may_defer)
 goto gpf;
 }
 
-if ( !(value & X86_CR4_PAE) && hvm_long_mode_enabled(v) )
+if ( !(value & X86_CR4_PAE) )
 {
-HVM_DBG_LOG(DBG_LEVEL_1, "Guest cleared CR4.PAE while "
-"EFER.LMA is set");
-goto gpf;
+if ( hvm_long_mode_enabled(v) )
+{
+HVM_DBG_LOG(DBG_LEVEL_1, "Guest cleared CR4.PAE while "
+"EFER.LMA is set");
+goto gpf;
+}
+if ( is_pvh_vcpu(v) )
+{
+HVM_DBG_LOG(DBG_LEVEL_1, "32-bit PVH guest cleared CR4.PAE");
+goto gpf;
+}
 }
 
 old_cr = v->arch.hvm_vcpu.guest_cr[4];
-- 
1.8.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v5 5/5] libxc/x86/pvh: Allow creation of 32b PVH guests

2015-09-04 Thread Boris Ostrovsky
Signed-off-by: Boris Ostrovsky 
Acked-by: Ian Campbell 
---
 tools/libxc/xc_dom_x86.c | 32 +++-
 1 file changed, 15 insertions(+), 17 deletions(-)

diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
index 3d40fa4..05fb0ce 100644
--- a/tools/libxc/xc_dom_x86.c
+++ b/tools/libxc/xc_dom_x86.c
@@ -300,7 +300,8 @@ static int setup_pgtables_x86_32_pae(struct xc_dom_image 
*dom)
 pgpfn = (addr - dom->parms.virt_base) >> PAGE_SHIFT_X86;
 l1tab[l1off] =
 pfn_to_paddr(xc_dom_p2m_guest(dom, pgpfn)) | L1_PROT;
-if ( (addr >= dom->pgtables_seg.vstart) &&
+if ( (!dom->pvh_enabled)&&
+ (addr >= dom->pgtables_seg.vstart) &&
  (addr < dom->pgtables_seg.vend) )
 l1tab[l1off] &= ~_PAGE_RW; /* page tables are r/o */
 
@@ -590,22 +591,9 @@ static int vcpu_x86_32(struct xc_dom_image *dom, void *ptr)
 
 DOMPRINTF_CALLED(dom->xch);
 
-if ( dom->pvh_enabled )
-{
-xc_dom_panic(dom->xch, XC_INTERNAL_ERROR,
- "%s: PVH not supported for 32bit guests.", __FUNCTION__);
-return -1;
-}
-
 /* clear everything */
 memset(ctxt, 0, sizeof(*ctxt));
 
-ctxt->user_regs.ds = FLAT_KERNEL_DS_X86_32;
-ctxt->user_regs.es = FLAT_KERNEL_DS_X86_32;
-ctxt->user_regs.fs = FLAT_KERNEL_DS_X86_32;
-ctxt->user_regs.gs = FLAT_KERNEL_DS_X86_32;
-ctxt->user_regs.ss = FLAT_KERNEL_SS_X86_32;
-ctxt->user_regs.cs = FLAT_KERNEL_CS_X86_32;
 ctxt->user_regs.eip = dom->parms.virt_entry;
 ctxt->user_regs.esp =
 dom->parms.virt_base + (dom->bootstack_pfn + 1) * PAGE_SIZE_X86;
@@ -613,9 +601,6 @@ static int vcpu_x86_32(struct xc_dom_image *dom, void *ptr)
 dom->parms.virt_base + (dom->start_info_pfn) * PAGE_SIZE_X86;
 ctxt->user_regs.eflags = 1 << 9; /* Interrupt Enable */
 
-ctxt->kernel_ss = ctxt->user_regs.ss;
-ctxt->kernel_sp = ctxt->user_regs.esp;
-
 ctxt->flags = VGCF_in_kernel_X86_32 | VGCF_online_X86_32;
 if ( dom->parms.pae == 2 /* extended_cr3 */ ||
  dom->parms.pae == 3 /* bimodal */ )
@@ -626,6 +611,19 @@ static int vcpu_x86_32(struct xc_dom_image *dom, void *ptr)
 DOMPRINTF("%s: cr3: pfn 0x%" PRIpfn " mfn 0x%" PRIpfn "",
   __FUNCTION__, dom->pgtables_seg.pfn, cr3_pfn);
 
+if ( dom->pvh_enabled )
+return 0;
+
+ctxt->user_regs.ds = FLAT_KERNEL_DS_X86_32;
+ctxt->user_regs.es = FLAT_KERNEL_DS_X86_32;
+ctxt->user_regs.fs = FLAT_KERNEL_DS_X86_32;
+ctxt->user_regs.gs = FLAT_KERNEL_DS_X86_32;
+ctxt->user_regs.ss = FLAT_KERNEL_SS_X86_32;
+ctxt->user_regs.cs = FLAT_KERNEL_CS_X86_32;
+
+ctxt->kernel_ss = ctxt->user_regs.ss;
+ctxt->kernel_sp = ctxt->user_regs.esp;
+
 return 0;
 }
 
-- 
1.8.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [BUG] Xen live migration dom0 memory

2015-09-04 Thread Ian Campbell
On Fri, 2015-09-04 at 17:04 +0100, Ian Campbell wrote:
> On Tue, 2015-07-07 at 17:26 +0200, Epiontis IT wrote:
> > Hello,
> > 
> > I already posted on the xen-users list and Ian told me to post a bug 
> > report here. We have the following problem:
> > 
> > When I start a migration with "xl migrate  " the 
> > destination machine sets up a vm "--incoming" and seems to sync 
> > memory because "xl list" shows "Mem" for the incoming vm going up 
> > stepwise until 2048. After that though the vm doesn't get launched. The 
> > 
> > vm is frozen for about a minute, the dom0 begins swapping out the 2GB 
> > to 
> > disk (because it only has 512M available for itself) and a process
> 
> I spoke with Ian Jackson about this this afternoon and we spent some
> staring at the pmap, and we are now both very perplexed...
> 
> So I've just tried to repro this.
> 
> I installed latest 4.5-testing and a 512M dom0 (which as it happens is 
> also
> what our automated test system uses). I then migrated a 3GB domU with "xl
> migrate  localhost" and it completed successfully. I had to do
> localhost migrate due to only having one box, but I don't think this 
> should
> matter.
>  
> I observed the save helper with pmap and never saw any large allocations.
> 
> This matches the behaviour that both Ian and I expected.
> 
> Your logs show you are running 4.5.1-rc1. I've looked over
> git log 4.5.1-rc1..origin/staging-4.5 -- tools/libx[cl]/
> and I don't see anything like a fix for a leak etc.
> 
> Next step is to revert my test box from staging-4.5 to 4.5.1-rc1 and see
> what changes.

Nothing changed.

I then tried building this tree with 

EXTRA_CFLAGS_XEN_TOOLS="-g -O2 -fstack-protector-strong -Wformat 
-Werror=format-security" APPEND_CPPFLAGS="-D_FORTIFY_SOURCE=2" 
APPEND_LDFLAGS="-Wl,-z,relro" 

which is how the Debian package is built, still no change.

My next step was to build patched with the Debian patches, still no change.

So I'm at even more of a loss than I was before :-/

Your report said you were running Debian 8.0 (Jessie) but with Xen 4.5.1
-rc1, while Jessie only has 4.4.1.

Was 4.5.1-rc1 from experimental or built yourself? Did you install anything
else from Sid/Experimental (or Stretch)?

If not from the packages in experimental then how did you install Xen?

If you can still reproduce then you could try installing the latest
valgrind, move libxl-save-helper aside and replace it with a script which
runs the tool under valgrind.

Modern valgrind understands many Xen hypercalls etc and so is quite useful
even on the Xen toolstack binaries.

Ian.

> 
> Ian.
> 
> > 
> > /usr/lib/xen-4.5/bin/libxl-save-helper --restore-domain
> > 
> > takes about 90% of the dom0 memory. After that the vm ist launched, 
> > memory consumption goes back to normal, swap space is freed.
> > 
> > I attached several log and output files in xen_bug_report.txt. In 
> > particular Ian wanted to see the pmap of the save helper process which 
> > you can find at the end of the file taken during several phases of the 
> > migration process.
> > 
> > 
> > Thank you,
> > Alex
> > 
> > ___
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [Draft C] Boot ABI for HVM guests without a device-model

2015-09-04 Thread Ian Campbell
On Fri, 2015-09-04 at 17:47 +0200, Roger Pau Monné wrote:
> El 04/09/15 a les 17.21, Jan Beulich ha escrit:
> > > > > AP startup
> > > > > > > > ==
> > > > > > > > 
> > > > > > > > AP startup is performed using hypercalls. The following 
> > > > > > > > VCPU operations
> > > > > > > > are used in order to bring up secondary vCPUs:
> > > > > > > > 
> > > > > > > >  * VCPUOP_initialise is used to set the initial state of 
> > > > > > > > the vCPU. The
> > > > > > > >argument passed to the hypercall must be of the type 
> > > > > > > > vcpu_hvm_context.
> > > > > > 
> > > > > > VCPUOP_initialise takes a struct vcpu_guest_context; I don't 
> > > > > > think
> > > > > > we can or should change that.
> > > > 
> > > > Didn't we agree that vcpu_guest_context was not suitable for 
> > > > HVM/PVH guests?
> > Yes we did.
> > 
> > > > Patch 24 of my HVM-without-dm series already introduces this new
> > > > structure and the necessary helpers.
> > I didn't look at most of the series yet (despite it already being at 
> > v6;
> > I'm sorry, I just didn't get around so far). But I think you agree that
> > we can't just change an existing hypercall. Iirc along with agreeing
> > on vcpu_guest_context not being suitable we also agreed that this
> > will need to be a new sub-op, and I wondered whether calling it
> > VCPUOP_initialize would be too subtle.
> 
> VCPUOP_initialize was never available to HVM guests, so I don't think
> changing the argument is a problem. However, I understand that for the
> sake of clarity overloading an hypercall this way is not the best
> practice. What about naming it VCPUOP_hvm_initialise?

If the new interface could support both PV (vcpu_guest_context) and the new
thing (i.e. with a type field and a union perhaps), or if the new interface
can work for PV some other way then it's not unheard of to rename the
existing number with _compat and take over the name with a new number.

It just needs some compat __XEN_INTERFACE_VERSION__ stuff in the headers,
like with e.g. __HYPERVISOR_sched_op vs __HYPERVISOR_sched_op_compat.

(I've not looked at this interface and I don't really remember what the old
one looks like, so maybe this is an insane idea in this case)

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [Draft C] Boot ABI for HVM guests without a device-model

2015-09-04 Thread Ian Campbell
On Fri, 2015-09-04 at 10:01 -0600, Jan Beulich wrote:
> > 
> > > > On 04.09.15 at 17:26,  wrote:
> > On Fri, 2015-09-04 at 09:21 -0600, Jan Beulich wrote:
> > > > > > On 04.09.15 at 16:31,  wrote:
> > > > El 04/09/15 a les 16.08, Jan Beulich ha escrit:
> > > > > > > > On 04.09.15 at 14:11,  wrote:
> > > > > > AP startup
> > > > > > ==
> > > > > > 
> > > > > > AP startup is performed using hypercalls. The following VCPU 
> > > > > > operations
> > > > > > are used in order to bring up secondary vCPUs:
> > > > > > 
> > > > > >  * VCPUOP_initialise is used to set the initial state of the 
> > > > > > vCPU. 
> > > > > > The
> > > > > >argument passed to the hypercall must be of the type > > > > 
> > > > > > 
> > vcpu_hvm_context.
> > > > > 
> > > > > VCPUOP_initialise takes a struct vcpu_guest_context; I don't 
> > > > > think
> > > > > we can or should change that.
> > > > 
> > > > Didn't we agree that vcpu_guest_context was not suitable for 
> > > > HVM/PVH 
> > > > guests?
> > > 
> > > Yes we did.
> > > 
> > > > Patch 24 of my HVM-without-dm series already introduces this new
> > > > structure and the necessary helpers.
> > > 
> > > I didn't look at most of the series yet (despite it already being at 
> > > v6;
> > > I'm sorry, I just didn't get around so far). But I think you agree 
> > > that
> > > we can't just change an existing hypercall. Iirc along with agreeing
> > > on vcpu_guest_context not being suitable we also agreed that this
> > > will need to be a new sub-op, and I wondered whether calling it
> > > VCPUOP_initialize would be too subtle.
> > 
> > You mean literally only s/s/z/?
> 
> Indeed ;-)
> 
> > In which case, yes, far far to subtle.
> 
> I was afraid I would get feedback like this (and I was surprised I
> didn't already when I first suggested this).

I think even if I'd have seen it the first time I'd have thought you were
joking...

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [Draft C] Boot ABI for HVM guests without a device-model

2015-09-04 Thread Roger Pau Monné
El 04/09/15 a les 18.03, Jan Beulich ha escrit:
 On 04.09.15 at 17:47,  wrote:
>> VCPUOP_initialize was never available to HVM guests, so I don't think
>> changing the argument is a problem. However, I understand that for the
>> sake of clarity overloading an hypercall this way is not the best
>> practice. What about naming it VCPUOP_hvm_initialise?
> 
> Hmm, don't know, the more that we want it primarily for PVH.

Well, it will be available to HVM guests in general, both pure HVM and
PVH guests.

Roger.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] PAGE_SIZE (64KB), while block driver 'struct request' deals with < PAGE_SIZE (up to 44Kb). Was:Re: [RFC] Support of non-indirect grant backend on 64KB guest

2015-09-04 Thread Julien Grall
On 04/09/15 16:41, Konrad Rzeszutek Wilk wrote:
>> Maybe we could fall back to the previous plan of modifying xen-blkfront
>> for the moment?
> 
> Which afaic need to be reposted?

Right. Although I didn't see any comment on the patch. All the comments
was about the problem. Does it mean that you and Roger are "happy" with
the way it's done?

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [BUG] Xen live migration dom0 memory

2015-09-04 Thread Ian Campbell
On Tue, 2015-07-07 at 17:26 +0200, Epiontis IT wrote:
> Hello,
> 
> I already posted on the xen-users list and Ian told me to post a bug 
> report here. We have the following problem:
> 
> When I start a migration with "xl migrate  " the 
> destination machine sets up a vm "--incoming" and seems to sync 
> memory because "xl list" shows "Mem" for the incoming vm going up 
> stepwise until 2048. After that though the vm doesn't get launched. The 
> vm is frozen for about a minute, the dom0 begins swapping out the 2GB to 
> disk (because it only has 512M available for itself) and a process

I spoke with Ian Jackson about this this afternoon and we spent some
staring at the pmap, and we are now both very perplexed...

So I've just tried to repro this.

I installed latest 4.5-testing and a 512M dom0 (which as it happens is also
what our automated test system uses). I then migrated a 3GB domU with "xl
migrate  localhost" and it completed successfully. I had to do
localhost migrate due to only having one box, but I don't think this should
matter.
 
I observed the save helper with pmap and never saw any large allocations.

This matches the behaviour that both Ian and I expected.

Your logs show you are running 4.5.1-rc1. I've looked over
git log 4.5.1-rc1..origin/staging-4.5 -- tools/libx[cl]/
and I don't see anything like a fix for a leak etc.

Next step is to revert my test box from staging-4.5 to 4.5.1-rc1 and see
what changes.

Ian.

> 
> /usr/lib/xen-4.5/bin/libxl-save-helper --restore-domain
> 
> takes about 90% of the dom0 memory. After that the vm ist launched, 
> memory consumption goes back to normal, swap space is freed.
> 
> I attached several log and output files in xen_bug_report.txt. In 
> particular Ian wanted to see the pmap of the save helper process which 
> you can find at the end of the file taken during several phases of the 
> migration process.
> 
> 
> Thank you,
> Alex
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [Draft C] Boot ABI for HVM guests without a device-model

2015-09-04 Thread Jan Beulich
>>> On 04.09.15 at 17:47,  wrote:
> VCPUOP_initialize was never available to HVM guests, so I don't think
> changing the argument is a problem. However, I understand that for the
> sake of clarity overloading an hypercall this way is not the best
> practice. What about naming it VCPUOP_hvm_initialise?

Hmm, don't know, the more that we want it primarily for PVH.

> Would it make sense to add aliases to have:
> 
> #define VCPU_hvm_up VCPU_up
> #define VCPU_hvm_down VCPU_down
> #define VCPU_hvm_is_up VCPU_is_up
> 
> Just for symmetry reasons?

If using the above, maybe. But I'd be afraid of this just becoming
useless clutter.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [Draft C] Boot ABI for HVM guests without a device-model

2015-09-04 Thread Jan Beulich
>>> On 04.09.15 at 17:26,  wrote:
> On Fri, 2015-09-04 at 09:21 -0600, Jan Beulich wrote:
>> > > > On 04.09.15 at 16:31,  wrote:
>> > El 04/09/15 a les 16.08, Jan Beulich ha escrit:
>> > > > > > On 04.09.15 at 14:11,  wrote:
>> > > > AP startup
>> > > > ==
>> > > > 
>> > > > AP startup is performed using hypercalls. The following VCPU 
>> > > > operations
>> > > > are used in order to bring up secondary vCPUs:
>> > > > 
>> > > >  * VCPUOP_initialise is used to set the initial state of the vCPU. 
>> > > > The
>> > > >argument passed to the hypercall must be of the type > > > > 
> vcpu_hvm_context.
>> > > 
>> > > VCPUOP_initialise takes a struct vcpu_guest_context; I don't think
>> > > we can or should change that.
>> > 
>> > Didn't we agree that vcpu_guest_context was not suitable for HVM/PVH 
>> > guests?
>> 
>> Yes we did.
>> 
>> > Patch 24 of my HVM-without-dm series already introduces this new
>> > structure and the necessary helpers.
>> 
>> I didn't look at most of the series yet (despite it already being at v6;
>> I'm sorry, I just didn't get around so far). But I think you agree that
>> we can't just change an existing hypercall. Iirc along with agreeing
>> on vcpu_guest_context not being suitable we also agreed that this
>> will need to be a new sub-op, and I wondered whether calling it
>> VCPUOP_initialize would be too subtle.
> 
> You mean literally only s/s/z/?

Indeed ;-)

> In which case, yes, far far to subtle.

I was afraid I would get feedback like this (and I was surprised I
didn't already when I first suggested this).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 13/18] Update IRTE according to guest interrupt config changes

2015-09-04 Thread Jan Beulich
>>> On 25.08.15 at 03:57,  wrote:

First of all - an empty Cc list on a patch is suspicious.

> @@ -198,6 +199,109 @@ void free_hvm_irq_dpci(struct hvm_irq_dpci *dpci)
>  xfree(dpci);
>  }
>  
> +/*
> + * This routine handles lowest-priority interrupts using vector-hashing
> + * mechanism. As an example, modern Intel CPUs use this method to handle
> + * lowest-priority interrupts.
> + *
> + * Here is the details about the vector-hashing mechanism:
> + * 1. For lowest-priority interrupts, store all the possible destination
> + *vCPUs in an array.
> + * 2. Use "gvec % max number of destination vCPUs" to find the right
> + *destination vCPU in the array for the lowest-priority interrupt.
> + */
> +static struct vcpu *vector_hashing_dest(const struct domain *d,
> +uint32_t dest_id,
> +bool_t dest_mode,
> +uint8_t gvec)
> +
> +{
> +unsigned long *dest_vcpu_bitmap;
> +unsigned int dest_vcpus = 0, idx;
> +unsigned int bitmap_array_size = BITS_TO_LONGS(d->max_vcpus);
> +struct vcpu *v, *dest = NULL;
> +unsigned int i;
> +
> +dest_vcpu_bitmap = xzalloc_array(unsigned long, bitmap_array_size);
> +if ( !dest_vcpu_bitmap )
> +{
> +dprintk(XENLOG_G_INFO,
> +"dom%d: failed to allocate memory\n", d->domain_id);

This dprintk() won't really help much. Please remove it.

> +return NULL;
> +}
> +
> +for_each_vcpu ( d, v )
> +{
> +if ( !vlapic_match_dest(vcpu_vlapic(v), NULL, APIC_DEST_NOSHORT,
> +dest_id, dest_mode) )
> +continue;
> +
> +__set_bit(v->vcpu_id, dest_vcpu_bitmap);
> +dest_vcpus++;
> +}
> +
> +if ( dest_vcpus != 0 )
> +{
> +unsigned int mod = gvec % dest_vcpus;
> +idx = 0;

You don't use idx before here. Declare it here, which also avoids
me telling you that you placed the blank line wrongly.

> +for ( i = 0; i <= mod; i++ )
> +{
> +idx = find_next_bit(dest_vcpu_bitmap, d->max_vcpus, idx) + 1;
> +BUG_ON(idx >= d->max_vcpus);
> +}
> +idx--;
> +
> +dest = d->vcpu[idx];

Just [idx - 1] please.

> +static struct vcpu *pi_find_dest_vcpu(const struct domain *d, uint32_t 
> dest_id,
> +  bool_t dest_mode, uint8_t 
> delivery_mode,
> +  uint8_t gvec)
> +{
> +unsigned int dest_vcpus = 0;
> +struct vcpu *v, *dest = NULL;
> +
> +if ( delivery_mode == dest_LowestPrio )
> +return vector_hashing_dest(d, dest_id, dest_mode, gvec);

So at this point delivery_mode == dest_Fixed, right?

> +for_each_vcpu ( d, v )
> +{
> +if ( !vlapic_match_dest(vcpu_vlapic(v), NULL, APIC_DEST_NOSHORT,
> +dest_id, dest_mode) )
> +continue;
> +
> +dest_vcpus++;
> +dest = v;
> +}
> +
> +/*
> + * For fixed destination, we only handle single-destination
> + * interrupts.
> + */
> +if ( dest_vcpus == 1 )
> +return dest;

Is it thus even possible for the if() condition to be false? If it isn't,
returning early from the loop would seem the better option. And
even if it is, I would question whether delivering the interrupt to
the first match isn't going to be better than dropping it.

> @@ -329,11 +433,30 @@ int pt_irq_create_bind(
>  /* Calculate dest_vcpu_id for MSI-type pirq migration. */
>  dest = pirq_dpci->gmsi.gflags & VMSI_DEST_ID_MASK;
>  dest_mode = !!(pirq_dpci->gmsi.gflags & VMSI_DM_MASK);
> +delivery_mode = (pirq_dpci->gmsi.gflags >> GFLAGS_SHIFT_DELIV_MODE) &
> +VMSI_DELIV_MASK;

In numbers (gflags >> 12) & 0x7000, which is likely not what you
want.

>  dest_vcpu_id = hvm_girq_dest_2_vcpu_id(d, dest, dest_mode);
>  pirq_dpci->gmsi.dest_vcpu_id = dest_vcpu_id;
>  spin_unlock(&d->event_lock);
>  if ( dest_vcpu_id >= 0 )
>  hvm_migrate_pirqs(d->vcpu[dest_vcpu_id]);
> +
> +/* Use interrupt posting if it is supported. */
> +if ( iommu_intpost )
> +{
> +const struct vcpu *vcpu = pi_find_dest_vcpu(d, dest, dest_mode,
> +  delivery_mode, 
> pirq_dpci->gmsi.gvec);
> +
> +if ( vcpu )
> +{
> +rc = pi_update_irte( vcpu, info, pirq_dpci->gmsi.gvec );
> +if ( unlikely(rc) )
> +dprintk(XENLOG_G_INFO,
> +"%pv: failed to update PI IRTE, gvec:%02x\n",
> +vcpu, pirq_dpci->gmsi.gvec);

Even if only a debug build printk() - aren't you afraid that if this
ever triggers, it will trigger a lot? And hence be pretty useless?

> +}

(Not only) depending on the answer, I'd consider adding an "else

Re: [Xen-devel] [Draft C] Boot ABI for HVM guests without a device-model

2015-09-04 Thread Roger Pau Monné
El 04/09/15 a les 17.21, Jan Beulich ha escrit:
 AP startup
 >>> ==
 >>>
 >>> AP startup is performed using hypercalls. The following VCPU operations
 >>> are used in order to bring up secondary vCPUs:
 >>>
 >>>  * VCPUOP_initialise is used to set the initial state of the vCPU. The
 >>>argument passed to the hypercall must be of the type 
 >>> vcpu_hvm_context.
>>> >> 
>>> >> VCPUOP_initialise takes a struct vcpu_guest_context; I don't think
>>> >> we can or should change that.
>> > 
>> > Didn't we agree that vcpu_guest_context was not suitable for HVM/PVH 
>> > guests?
> Yes we did.
> 
>> > Patch 24 of my HVM-without-dm series already introduces this new
>> > structure and the necessary helpers.
> I didn't look at most of the series yet (despite it already being at v6;
> I'm sorry, I just didn't get around so far). But I think you agree that
> we can't just change an existing hypercall. Iirc along with agreeing
> on vcpu_guest_context not being suitable we also agreed that this
> will need to be a new sub-op, and I wondered whether calling it
> VCPUOP_initialize would be too subtle.

VCPUOP_initialize was never available to HVM guests, so I don't think
changing the argument is a problem. However, I understand that for the
sake of clarity overloading an hypercall this way is not the best
practice. What about naming it VCPUOP_hvm_initialise?

Would it make sense to add aliases to have:

#define VCPU_hvm_up VCPU_up
#define VCPU_hvm_down VCPU_down
#define VCPU_hvm_is_up VCPU_is_up

Just for symmetry reasons?

Roger.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] xen: switch extra memory accounting to use pfns

2015-09-04 Thread David Vrabel
On 04/09/15 13:05, Juergen Gross wrote:
> Instead of using physical addresses for accounting of extra memory
> areas available for ballooning switch to pfns as this is much less
> error prone regarding partial pages.

Applied to for-linus-4.3, thanks.

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] xen: limit memory to architectural maximum

2015-09-04 Thread David Vrabel
On 04/09/15 13:18, Juergen Gross wrote:
> When a pv-domain (including dom0) is started it tries to size it's
> p2m list according to the maximum possible memory amount it ever can
> achieve. Limit the initial maximum memory size to the architectural
> limit of the hardware in order to avoid overflows during remapping
> of memory.
> 
> This problem will occur when dom0 is started with an initial memory
> size being a multiple of 1GB, but without specifying it's maximum
> memory size. The kernel must be configured without
> CONFIG_XEN_BALLOON_MEMORY_HOTPLUG for the problem to happen.

Applied to for-linus-4.3, thanks.

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [Draft C] Boot ABI for HVM guests without a device-model

2015-09-04 Thread Ian Campbell
On Fri, 2015-09-04 at 09:21 -0600, Jan Beulich wrote:
> > 
> > > > On 04.09.15 at 16:31,  wrote:
> > El 04/09/15 a les 16.08, Jan Beulich ha escrit:
> > > > > > On 04.09.15 at 14:11,  wrote:
> > > > The format of the structure passed in the %ebx register is the 
> > > > following:
> > > 
> > > Even if it may sound like splitting hair: Please use precise wording. 
> > > It's
> > > not the structure that's contained in %ebx, but %ebx hold the address
> > > of that structure.
> > 
> > Would you be fine with replacing this sentence with:
> > 
> > The format of the boot start info structure is the following:
> 
> Yes (maybe insert "(pointed to be %ebx)").
> 
> > > > struct hvm_start_info {
> > > > #define HVM_START_MAGIC_VALUE 0x336ec578
> > > > uint32_t magic; /* Contains the magic value 
> > > > 0x336ec578   */
> > > > /* ("xEn3" with the 0x80 bit of the 
> > > > "E" set).*/
> > > > uint32_t flags; /* SIF_xxx flags.  
> > > >   */
> > > 
> > > Do really mean to re-use the SIF_* flags here?
> > 
> > We can introduce a new set of flags, HVM_INIT_*, which ATM is only 
> > going
> > to be:
> > 
> > #define HVM_FLAGS_INITDOMAIN (1<<0)
> 
> From an abstract pov I'd prefer that. Maybe I'm overlooking
> something which would be simplified by using the same values...
> 
> > > > AP startup
> > > > ==
> > > > 
> > > > AP startup is performed using hypercalls. The following VCPU 
> > > > operations
> > > > are used in order to bring up secondary vCPUs:
> > > > 
> > > >  * VCPUOP_initialise is used to set the initial state of the vCPU. 
> > > > The
> > > >argument passed to the hypercall must be of the type > > > > 
> > > > vcpu_hvm_context.
> > > 
> > > VCPUOP_initialise takes a struct vcpu_guest_context; I don't think
> > > we can or should change that.
> > 
> > Didn't we agree that vcpu_guest_context was not suitable for HVM/PVH 
> > guests?
> 
> Yes we did.
> 
> > Patch 24 of my HVM-without-dm series already introduces this new
> > structure and the necessary helpers.
> 
> I didn't look at most of the series yet (despite it already being at v6;
> I'm sorry, I just didn't get around so far). But I think you agree that
> we can't just change an existing hypercall. Iirc along with agreeing
> on vcpu_guest_context not being suitable we also agreed that this
> will need to be a new sub-op, and I wondered whether calling it
> VCPUOP_initialize would be too subtle.

You mean literally only s/s/z/? In which case, yes, far far to subtle.

Even initialise2 would be better than that alternative...

> 
> Jan
> 
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] PAGE_SIZE (64KB), while block driver 'struct request' deals with < PAGE_SIZE (up to 44Kb). Was:Re: [RFC] Support of non-indirect grant backend on 64KB guest

2015-09-04 Thread Konrad Rzeszutek Wilk
On Fri, Sep 04, 2015 at 03:04:18PM +0100, Stefano Stabellini wrote:
> On Thu, 27 Aug 2015, Julien Grall wrote:
> > On 21/08/15 18:10, Konrad Rzeszutek Wilk wrote:
> > > On Fri, Aug 21, 2015 at 05:08:35PM +0100, David Vrabel wrote:
> > >> On 21/08/15 17:05, Konrad Rzeszutek Wilk wrote:
> > >>>
> > >>> I have to concur with that. We can't mandate that ARM 64k page MUST use
> > >>> indirect descriptors.
> > >>
> > >> Then it has to be fixed in the block layer to allow < PAGE_SIZE segments
> > >> and to get the block layer to split requests for blkfront.
> > > 
> > > Hey Jens,
> > > 
> > > I am hoping you can help us figure this problem out.
> > > 
> > > The Linux ARM is capable of using 4KB pages and 64KB pages. Our block
> > > driver (xen-blkfront) was built with 4KB pages in mind and without using
> > > any fancy flags (which some backends lack) the maximum amount of I/O it 
> > > can
> > > fit on a ring is 44KB.
> > > 
> > > This has the unfortunate effect that when the xen-blkfront
> > > gets an 'struct request' it can have on page (64KB) and it can't actually
> > > fit it on the ring! And the lowest segment size it advertises is PAGE_SIZE
> > > (64KB). I believe Julien (who found this) tried initially advertising
> > > smaller segment size than PAGE_SIZE (32KB). However looking at
> > > __blk_segment_map_sg it looks to assume smallest size is PAGE_SIZE so
> > > that would explain why it did not work.
> > 
> > To be honest, I haven't tried to see how the block layer will act if I
> > dropped those checks in blk-settings.c until today.
> > 
> > I don't see any assumption about PAGE_SIZE in __blk_segment_map_sg.
> > Although dropping the checks in blk-settings (see quick patch [1]),
> > I got the following error in the frontend:
> > 
> > bio too big device xvda (128 > 88)
> > Buffer I/O error on dev xvda, logical block 0, async page read
> > bio too big device xvda (128 > 88)
> > Buffer I/O error on dev xvda, logical block 0, async page read
> > 
> > The "bio too big device ..." comes from generic_make_request_checks
> > (linux/block/blk-core.c) and the stack trace is:
> > 
> > [] dump_backtrace+0x0/0x124
> > [] show_stack+0x10/0x1c
> > [] dump_stack+0x78/0xbc
> > [] warn_slowpath_common+0x98/0xd0
> > [] warn_slowpath_null+0x14/0x20
> > [] generic_make_request_checks+0x114/0x230
> > [] generic_make_request+0x10/0x108
> > [] submit_bio+0x94/0x1e0
> > [] submit_bh_wbc.isra.36+0x100/0x1a8
> > [] block_read_full_page+0x320/0x3e8
> > [] blkdev_readpage+0x14/0x20
> > [] do_read_cache_page+0x16c/0x1a0
> > [] read_cache_page+0x10/0x1c
> > [] read_dev_sector+0x30/0x9c
> > [] msdos_partition+0x84/0x554
> > [] check_partition+0xf8/0x21c
> > [] rescan_partitions+0xb0/0x2a4
> > [] __blkdev_get+0x228/0x34c
> > [] blkdev_get+0x40/0x364
> > [] add_disk+0x398/0x424
> > [] blkback_changed+0x1200/0x152c
> > [] xenbus_otherend_changed+0x9c/0xa4
> > [] backend_changed+0xc/0x18
> > [] xenwatch_thread+0xa0/0x13c
> > [] kthread+0xd8/0xf0
> > 
> > The fs buffer code seems to assume that the block driver will always support
> > at least a bio of PAGE_SIZE.
> > 
> > > One wya to make this work is for the driver (xen-blkfront) to split
> > > the 'struct request' I/O in two internal requests.
> > > 
> > > But this has to be a normal problem. Surely there are other drivers
> > > (MMC?) that can't handle PAGE_SIZE and have to break their I/Os.
> > > Would it make sense for the common block code to be able to deal
> > > with this?
> > 
> > It will take me a bit of time to fully understand the block layer
> > as the changes doesn't seem trivial from POV (I don't have any
> > knowledge in it). So I will wait a feedback from Jens before
> > going further on this approach.
> 
> Maybe we could fall back to the previous plan of modifying xen-blkfront
> for the moment?

Which afaic need to be reposted?

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [Pkg-xen-devel] Bug#795330: Suggests noapic but doesn't support it

2015-09-04 Thread Jan Beulich
>>> On 04.09.15 at 16:22,  wrote:
> Oh, in which case removing (from Xen) the recommendation to try noapic
> might be useful.

Hmm - so far I didn't the option was not working. I also don't
understand the reference to pv-ops kernels in the doc. Sadly it's
not clear where this comes from, since Ian has imported it from
the wiki, and in the wiki it has been there from the beginning
(with no hint to where that first version came from).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [Pkg-xen-devel] Bug#795330: Suggests noapic but doesn't support it

2015-09-04 Thread Jan Beulich
>>> On 04.09.15 at 16:13,  wrote:
> On Fri, 2015-09-04 at 15:04 +0100, David Vrabel wrote:
>> On 04/09/15 14:38, Ian Campbell wrote:
>> > On Thu, 2015-08-13 at 00:59 +0200, Guido Günther wrote:
>> > > Package: xen-hypervisor-4.5-amd64
>> > > Severity: normal
>> > > 
>> > > Hi,
>> > > when running xen inside of kvm the hypervisor fails to set up the 
>> > > proper
>> > > IRQ routing and suggests to use noapic.
>> > 
>> > FTR, it seems to say:
>> > 
>> > panic("IO-APIC + timer doesn't work!  Boot with 
>> > apic_verbosity=debug "
>> >   "and send a report.  Then try booting with the 'noapic' 
>> > option");
>> 
>> This is the Linux kernel making a recommendation for its command line.
> 
> I cut-n-paste that message from xen.git. Although maybe in a file which
> originates in Linux.
> 
> (So now I realised I don't really know which entity was complaining)

The reference to apic_verbosity= makes it Xen (Linux uses apic= here).

Jan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [Draft C] Boot ABI for HVM guests without a device-model

2015-09-04 Thread Jan Beulich
>>> On 04.09.15 at 16:31,  wrote:
> El 04/09/15 a les 16.08, Jan Beulich ha escrit:
> On 04.09.15 at 14:11,  wrote:
>>> The format of the structure passed in the %ebx register is the following:
>> 
>> Even if it may sound like splitting hair: Please use precise wording. It's
>> not the structure that's contained in %ebx, but %ebx hold the address
>> of that structure.
> 
> Would you be fine with replacing this sentence with:
> 
> The format of the boot start info structure is the following:

Yes (maybe insert "(pointed to be %ebx)").

>>> struct hvm_start_info {
>>> #define HVM_START_MAGIC_VALUE 0x336ec578
>>> uint32_t magic; /* Contains the magic value 0x336ec578  
>>>  */
>>> /* ("xEn3" with the 0x80 bit of the "E" 
>>> set).*/
>>> uint32_t flags; /* SIF_xxx flags.   
>>>  */
>> 
>> Do really mean to re-use the SIF_* flags here?
> 
> We can introduce a new set of flags, HVM_INIT_*, which ATM is only going
> to be:
> 
> #define HVM_FLAGS_INITDOMAIN (1<<0)

>From an abstract pov I'd prefer that. Maybe I'm overlooking
something which would be simplified by using the same values...

>>> AP startup
>>> ==
>>>
>>> AP startup is performed using hypercalls. The following VCPU operations
>>> are used in order to bring up secondary vCPUs:
>>>
>>>  * VCPUOP_initialise is used to set the initial state of the vCPU. The
>>>argument passed to the hypercall must be of the type vcpu_hvm_context.
>> 
>> VCPUOP_initialise takes a struct vcpu_guest_context; I don't think
>> we can or should change that.
> 
> Didn't we agree that vcpu_guest_context was not suitable for HVM/PVH guests?

Yes we did.

> Patch 24 of my HVM-without-dm series already introduces this new
> structure and the necessary helpers.

I didn't look at most of the series yet (despite it already being at v6;
I'm sorry, I just didn't get around so far). But I think you agree that
we can't just change an existing hypercall. Iirc along with agreeing
on vcpu_guest_context not being suitable we also agreed that this
will need to be a new sub-op, and I wondered whether calling it
VCPUOP_initialize would be too subtle.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [Draft C] Boot ABI for HVM guests without a device-model

2015-09-04 Thread Jan Beulich
>>> On 04.09.15 at 16:31,  wrote:
> El 04/09/15 a les 16.08, Jan Beulich ha escrit:
> On 04.09.15 at 14:11,  wrote:
>>>  * tr: must be a 32-bit TSS (active) with a base of '0' and a limit of 
>>> '0xFF'.
>> 
>> Already on the previous version I asked about this strange 0xFF,
>> and I don't recall any answer.
> 
> My bad, I have actually changed this in the code (see v6), but forgot to
> update the design document. 0x67 is perfectly fine, and is what should
> be used.
> 
> Just for the record, I've initially set it to 0xff because that's how
> it's done in construct_vmcs.

Which seems bogus too. Jun, Kevin?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 12/18] x86: move some APIC related macros to apicdef.h

2015-09-04 Thread Jan Beulich
>>> On 25.08.15 at 03:57,  wrote:
> --- a/xen/include/asm-x86/apicdef.h
> +++ b/xen/include/asm-x86/apicdef.h
> @@ -124,6 +124,10 @@
>  
>  #define MAX_IO_APICS 128
>  
> +#define APIC_SHORT_MASK  0xc
> +#define APIC_DEST_NOSHORT0x0
> +#define APIC_DEST_MASK   0x800

Moving them into this header is certainly the right thing to do, but
please put them where they belong inside the header (i.e. the first
next to ACPI_DEST_{SELF,ALLINC,ALLBUT} and the latter before
or after APIC_DEST_{LOGICAL,PHYSICAL}.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 02/18] Add cmpxchg16b support for x86-64

2015-09-04 Thread Jan Beulich
>>> On 25.08.15 at 03:57,  wrote:
> --- a/xen/include/asm-x86/x86_64/system.h
> +++ b/xen/include/asm-x86/x86_64/system.h
> @@ -6,6 +6,34 @@
> (unsigned long)(n),sizeof(*(ptr
>  
>  /*
> + * Atomic 16 bytes compare and exchange.  Compare OLD with MEM, if
> + * identical, store NEW in MEM.  Return the initial value in MEM.
> + * Success is indicated by comparing RETURN with OLD.
> + *
> + * This function can only be called when cpu_has_cx16 is true.
> + */
> +
> +static always_inline __uint128_t __cmpxchg16b(
> +volatile void *ptr, __uint128_t *old, __uint128_t *new)

Noted just now - the latter two should both be const.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 11/18] vt-d: Add API to update IRTE when VT-d PI is used

2015-09-04 Thread Jan Beulich
>>> On 25.08.15 at 03:57,  wrote:
> This patch adds an API which is used to update the IRTE
> for posted-interrupt when guest changes MSI/MSI-X information.
> 
> CC: Yang Zhang 
> CC: Kevin Tian 
> CC: Keir Fraser 
> CC: Jan Beulich 
> CC: Andrew Cooper 
> Signed-off-by: Feng Wu 
> Acked-by: Kevin Tian 
> ---
> v6:
> - In some error cases, the desc->lock will be unlocked twice, fix it.

Bug fixes invalidate prior acks.

> --- a/xen/drivers/passthrough/vtd/intremap.c
> +++ b/xen/drivers/passthrough/vtd/intremap.c
> @@ -899,3 +899,110 @@ void iommu_disable_x2apic_IR(void)
>  for_each_drhd_unit ( drhd )
>  disable_qinval(drhd->iommu);
>  }
> +
> +static void setup_posted_irte(struct iremap_entry *new_ire,
> +const struct pi_desc *pi_desc, const uint8_t gvec)
> +{
> +new_ire->post.urg = 0;
> +new_ire->post.vector = gvec;
> +new_ire->post.pda_l = virt_to_maddr(pi_desc) >> (32 - PDA_LOW_BIT);
> +new_ire->post.pda_h = virt_to_maddr(pi_desc) >> 32;
> +
> +new_ire->post.res_1 = 0;
> +new_ire->post.res_2 = 0;
> +new_ire->post.res_3 = 0;
> +new_ire->post.res_4 = 0;
> +new_ire->post.res_5 = 0;
> +
> +new_ire->post.im = 1;
> +}

I think it would be better to just clear out the structure first. This
may then also allow for no longer naming all of the bitfield res_*
member, making it even more obvious that they're reserved. Or
wait - looking at the use below you seem to be updating the
descriptor. Why would you need to zero out reserved fields in
that case?

> +int pi_update_irte(const struct vcpu *v, const struct pirq *pirq,
> +const uint8_t gvec)
> +{
> +struct irq_desc *desc;
> +const struct msi_desc *msi_desc;
> +int remap_index;
> +int rc = 0;
> +const struct pci_dev *pci_dev;
> +const struct acpi_drhd_unit *drhd;
> +struct iommu *iommu;
> +struct ir_ctrl *ir_ctrl;
> +struct iremap_entry *iremap_entries = NULL, *p = NULL;
> +struct iremap_entry new_ire, old_ire;
> +const struct pi_desc *pi_desc = &v->arch.hvm_vmx.pi_desc;
> +__uint128_t ret;
> +
> +desc = pirq_spin_lock_irq_desc(pirq, NULL);
> +if ( !desc )
> +return -EINVAL;
> +
> +msi_desc = desc->msi_desc;
> +if ( !msi_desc )
> +{
> +rc = -ENODEV;
> +goto unlock_out;
> +}
> +
> +pci_dev = msi_desc->dev;
> +if ( !pci_dev )
> +{
> +rc = -ENODEV;
> +goto unlock_out;
> +}
> +
> +remap_index = msi_desc->remap_index;
> +
> +spin_unlock_irq(&desc->lock);
> +
> +ASSERT(spin_is_locked(&pcidevs_lock));
> +
> +/*
> + * For performance concern, we will store the 'iommu' pointer in

DYM "For performance reasons we should store ..."?

> + * 'struct msi_desc' in some other place, so we don't need to waste
> + * time searching it here. I will fix this later.

Instead of this last sentence I'd rather see the comment flagged
with FIXME or alike.

> + */
> +drhd = acpi_find_matched_drhd_unit(pci_dev);
> +if ( !drhd )
> +return -ENODEV;
> +
> +iommu = drhd->iommu;
> +ir_ctrl = iommu_ir_ctrl(iommu);
> +if ( !ir_ctrl )
> +return -ENODEV;
> +
> +spin_lock_irq(&ir_ctrl->iremap_lock);
> +
> +GET_IREMAP_ENTRY(ir_ctrl->iremap_maddr, remap_index, iremap_entries, p);
> +
> +old_ire = new_ire = *p;
> +
> +/* Setup/Update interrupt remapping table entry. */
> +setup_posted_irte(&new_ire, pi_desc, gvec);
> +ret = cmpxchg16b(p, &old_ire, &new_ire);
> +
> +/*
> + * In the above, we use cmpxchg16 to atomically update the 128-bit IRTE,
> + * and the hardware cannot update the IRTE behind us, so the return value
> + * of cmpxchg16 should be the same as old_ire. This ASSERT validate it.
> + */
> +ASSERT(ret == *(__uint128_t *)&old_ire);

I continue to object to such casts. Make the union have a
__uint128_t variant, which you can then check here.

Jan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Linux 4.1 reports wrong number of pages to toolstack

2015-09-04 Thread David Vrabel
On 04/09/15 15:53, Wei Liu wrote:
> On Fri, Sep 04, 2015 at 03:42:06PM +0100, David Vrabel wrote:
>> On 04/09/15 09:53, Ian Campbell wrote:
>>> On Fri, 2015-09-04 at 01:40 +0100, Wei Liu wrote:
 Hi David

 This issue is exposed by the introduction of migration v2. The symptom is 
 that
 a guest with 32 bit 4.1 kernel can't be restored because it's asking for 
 too
 many pages.
>>>
>>> FWIW my adhoc tests overnight gave me:
>>>
>>> 37858: b953c0d234bc72e8489d3bf51a276c5c4ec85345 v4.1Fail
>>> 37862: 39a8804455fb23f09157341d3ba7db6d7ae6ee76 v4.0Fail
>>> 37860: bfa76d49576599a4b9f9b7a71f23d73d6dcff735 v3.19   Fail
>>>
>>> 37872: e36f014edff70fc02b3d3d79cead1d58f289332e v3.19-rc7   Fail
>>> 37866: 26bc420b59a38e4e6685a73345a0def461136dce v3.19-rc6   Fail
>>> 37868: ec6f34e5b552fb0a52e6aae1a5afbbb1605cc6cc v3.19-rc5   Fail
>>> 37864: eaa27f34e91a14cdceed26ed6c6793ec1d186115 v3.19-rc4   Fail *
>>> 37867: b1940cd21c0f4abdce101253e860feff547291b0 v3.19-rc3   Pass *
>>> 37865: b7392d2247cfe6771f95d256374f1a8e6a6f48d6 v3.19-rc2   Pass
>>>
>>> 37863: 97bf6af1f928216fd6c5a66e8a57bfa95a659672 v3.19-rc1   Pass
>>>
>>> 37861: b2776bf7149bddd1f4161f14f79520f17fc1d71d v3.18   Pass
>>>
>>> I have set the adhoc bisector working on the ~200 commits between rc3 and
>>> rc4. It's running in the Citrix instance (which is quieter) so the interim
>>> results are only visible within our network at http://osstest.xs.citrite.ne
>>> t/~osstest/testlogs/results-adhoc/bisect/xen-unstable/test-amd64-i386
>>> -xl..html.
>>>
>>> So far it has confirmed the basis fail and it is now rechecking the basis
>>> pass.
>>>
>>> Slightly strange though is:
>>> $ git log --oneline v3.19-rc3..v3.19-rc4 -- drivers/xen/ arch/x86/xen/ 
>>> include/xen/
>>> $
>>>
>>> i.e. there are no relevant seeming xen commits in that range. Maybe the
>>> last one of this is more relevant?
>>
>> Since this bisect attempt appears to have disappeared into the weeds I
>> did my own and it fingered:
>>
>> 633d6f17cd91ad5bf2370265946f716e42d388c6 (x86/xen: prepare p2m list for
>> memory hotplug) which was introduced in 4.0-rc7.
>>
>> This looks a lot more plausible as the Linux change triggering the
>> migration failures.
>>
> 
> FWIW. Same 32bit kernel, 128MB memory, migration is OK.

This commit is only bad with 64-bit guests -- with a 32-bit guest the
maximum p2m size covers only 64 GiB.  It will also requires
XEN_BALLOON_MEMORY_HOTPLUG to be enabled.

This commit is exposing a toolstack bug.

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 09/18] VT-d: Remove pointless casts

2015-09-04 Thread Jan Beulich
>>> On 25.08.15 at 03:57,  wrote:
> Remove pointless casts.

Much appreciated, but ...

> @@ -281,11 +280,10 @@ static void dump_iommu_info(unsigned char key)
>  
>  printk("   %02x:  %04x   %x%x   %x   %x   %x%x"
>  "%x %02x\n", i,
> -(u32)remap->index_0_14 | ((u32)remap->index_15 << 15),
> -(u32)remap->format, (u32)remap->mask, 
> (u32)remap->trigger,
> -(u32)remap->irr, (u32)remap->polarity,
> -(u32)remap->delivery_status, (u32)remap->delivery_mode,
> -(u32)remap->vector);
> +remap->index_0_14 | ((u32)remap->index_15 << 15),

... why are you retaining this one?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Linux 4.1 reports wrong number of pages to toolstack

2015-09-04 Thread Wei Liu
On Fri, Sep 04, 2015 at 03:42:06PM +0100, David Vrabel wrote:
> On 04/09/15 09:53, Ian Campbell wrote:
> > On Fri, 2015-09-04 at 01:40 +0100, Wei Liu wrote:
> >> Hi David
> >>
> >> This issue is exposed by the introduction of migration v2. The symptom is 
> >> that
> >> a guest with 32 bit 4.1 kernel can't be restored because it's asking for 
> >> too
> >> many pages.
> > 
> > FWIW my adhoc tests overnight gave me:
> > 
> > 37858: b953c0d234bc72e8489d3bf51a276c5c4ec85345 v4.1Fail
> > 37862: 39a8804455fb23f09157341d3ba7db6d7ae6ee76 v4.0Fail
> > 37860: bfa76d49576599a4b9f9b7a71f23d73d6dcff735 v3.19   Fail
> > 
> > 37872: e36f014edff70fc02b3d3d79cead1d58f289332e v3.19-rc7   Fail
> > 37866: 26bc420b59a38e4e6685a73345a0def461136dce v3.19-rc6   Fail
> > 37868: ec6f34e5b552fb0a52e6aae1a5afbbb1605cc6cc v3.19-rc5   Fail
> > 37864: eaa27f34e91a14cdceed26ed6c6793ec1d186115 v3.19-rc4   Fail *
> > 37867: b1940cd21c0f4abdce101253e860feff547291b0 v3.19-rc3   Pass *
> > 37865: b7392d2247cfe6771f95d256374f1a8e6a6f48d6 v3.19-rc2   Pass
> > 
> > 37863: 97bf6af1f928216fd6c5a66e8a57bfa95a659672 v3.19-rc1   Pass
> > 
> > 37861: b2776bf7149bddd1f4161f14f79520f17fc1d71d v3.18   Pass
> > 
> > I have set the adhoc bisector working on the ~200 commits between rc3 and
> > rc4. It's running in the Citrix instance (which is quieter) so the interim
> > results are only visible within our network at http://osstest.xs.citrite.ne
> > t/~osstest/testlogs/results-adhoc/bisect/xen-unstable/test-amd64-i386
> > -xl..html.
> > 
> > So far it has confirmed the basis fail and it is now rechecking the basis
> > pass.
> > 
> > Slightly strange though is:
> > $ git log --oneline v3.19-rc3..v3.19-rc4 -- drivers/xen/ arch/x86/xen/ 
> > include/xen/
> > $
> > 
> > i.e. there are no relevant seeming xen commits in that range. Maybe the
> > last one of this is more relevant?
> 
> Since this bisect attempt appears to have disappeared into the weeds I
> did my own and it fingered:
> 
> 633d6f17cd91ad5bf2370265946f716e42d388c6 (x86/xen: prepare p2m list for
> memory hotplug) which was introduced in 4.0-rc7.
> 
> This looks a lot more plausible as the Linux change triggering the
> migration failures.
> 

FWIW. Same 32bit kernel, 128MB memory, migration is OK.

> David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 08/18] vmx: Suppress posting interrupts when 'SN' is set

2015-09-04 Thread Jan Beulich
>>> On 25.08.15 at 03:57,  wrote:
> --- a/xen/arch/x86/hvm/vmx/vmx.c
> +++ b/xen/arch/x86/hvm/vmx/vmx.c
> @@ -1701,8 +1701,36 @@ static void vmx_deliver_posted_intr(struct vcpu *v, u8 
> vector)
>   */
>  pi_set_on(&v->arch.hvm_vmx.pi_desc);
>  }
> -else if ( !pi_test_and_set_on(&v->arch.hvm_vmx.pi_desc) )
> +else
>  {
> +struct pi_desc old, new, prev;
> +
> +/* To skip over first check in the loop below. */
> +prev.control = 0;

Why don't you just read the field instead of adding the comment?

> +do {
> +/*
> + * Currently, we don't support urgent interrupt, all
> + * interrupts are recognized as non-urgent interrupt,
> + * so we cannot send posted-interrupt when 'SN' is set.
> + * Besides that, if 'ON' is already set, we cannot set
> + * posted-interrupts as well.
> + */
> +if ( pi_test_sn(&prev) || pi_test_on(&prev) )
> +{
> +vcpu_kick(v);
> +return;
> +}
> +
> +old.control = v->arch.hvm_vmx.pi_desc.control &
> +  ~( 1 << POSTED_INTR_ON | 1 << POSTED_INTR_SN );
> +new.control = v->arch.hvm_vmx.pi_desc.control |
> +  1 << POSTED_INTR_ON;

Missing parentheses around the shifts. In the former case there are
also stray blanks inside the parentheses.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2] public/io/netif.h: move and amend multicast control documentation

2015-09-04 Thread Ian Campbell
On Fri, 2015-09-04 at 14:22 +0100, Wei Liu wrote:
> On Wed, Sep 02, 2015 at 12:17:05PM +0100, Paul Durrant wrote:
> > netif.h contains a specification of the 
> > XEN_NETIF_EXTRA_TYPE_MCAST_{ADD,DEL}
> > extra info messages require to manipulate a multicast filter list 
> > maintained
> > by a backend and specifies the xenstore negotiation protocol in a 
> > comment
> > just above the structure defintion, which is easy to miss.
> > 
> > This patch moves the documentation of the xenstore negotiation to be
> > co-located with the documentation for other features and also amends 
> > the
> > wording to be clearer.
> > 
> > Signed-off-by: Paul Durrant 
> > Cc: Ian Campbell 
> > Cc: Ian Jackson 
> > Cc: Jan Beulich 
> > Cc: Keir Fraser 
> > Cc: Tim Deegan 
> 
> Acked-by: Wei Liu 

Applied, with:

> > + * connected state.
> 
> I would prefer adding a blank line here if possible.

"\n *" added.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 07/18] vmx: Initialize VT-d Posted-Interrupts Descriptor

2015-09-04 Thread Jan Beulich
>>> On 25.08.15 at 03:57,  wrote:
> --- a/xen/arch/x86/hvm/vmx/vmcs.c
> +++ b/xen/arch/x86/hvm/vmx/vmcs.c
> @@ -39,6 +39,7 @@
>  #include 
>  #include 
>  #include 
> +#include 

I can't seem to spot what this is needed for.

> @@ -951,6 +952,20 @@ void virtual_vmcs_vmwrite(void *vvmcs, u32 
> vmcs_encoding, u64 val)
>  virtual_vmcs_exit(vvmcs);
>  }
>  
> +static void pi_desc_init(struct vcpu *v)
> +{
> +uint32_t dest;
> +
> +v->arch.hvm_vmx.pi_desc.nv = posted_intr_vector;
> +
> +dest = cpu_physical_id(v->processor);
> +
> +if ( x2apic_enabled )
> +v->arch.hvm_vmx.pi_desc.ndst = dest;
> +else
> +v->arch.hvm_vmx.pi_desc.ndst = MASK_INSR(dest, PI_xAPIC_NDST_MASK);
> +}
> +
>  static int construct_vmcs(struct vcpu *v)
>  {
>  struct domain *d = v->domain;
> @@ -1089,6 +1104,9 @@ static int construct_vmcs(struct vcpu *v)
>  
>  if ( cpu_has_vmx_posted_intr_processing )
>  {
> +if ( iommu_intpost )
> +pi_desc_init(v);

If this is going to remain the only call site of the function, I'd suggest
expanding it here. This is because calling the function from elsewhere
is, at least at the first glance, unsafe: It doesn't update pi_desc
atomically, which is in (only apparent?) conflict with the atomic
modifications helpers added in an earlier patch.

If further call sites will get added by later patches, clarifying in a
comment why the non-atomic updates are okay would seem
necessary; alternatively change them to become atomic.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Commit moratorium

2015-09-04 Thread Ian Campbell
On Fri, 2015-09-04 at 14:19 +0100, Wei Liu wrote:
> On Fri, Sep 04, 2015 at 01:56:20PM +0100, Ian Campbell wrote:
> > On Fri, 2015-09-04 at 13:12 +0100, Wei Liu wrote:
> > > 
> > > But this point is moot since I said in the other email I was happy 
> > > with
> > > that one-liner applied today.
> > 
> > I will do so.
> > 
> > If we are having a push to staging anyway then I also have these 
> > candidates
> > in my folder:
> > 
> > tools/xen-access: use PRI_xen_pfn
> > xen/dt: Handle correctly node with interrupt-map in 
> > dt_for_each_irq_map
> > build: update top-level make help
> > MAINTAINERS: tools/ocaml: update David Scott's email address
> > public/io/netif.h: move and amend multicast control documentation
> > 
> > The last one is docs only, but isn't yet acked (I've not read it myself
> > either).
> > 
> > Would you rather I pushed these now at the same time as the arm64 fix 
> > or
> > wait until later?
> > 
> 
> At the same time - since most are docs update and two patches that touch
> code are well understood.

All done.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Linux 4.1 reports wrong number of pages to toolstack

2015-09-04 Thread David Vrabel
On 04/09/15 09:53, Ian Campbell wrote:
> On Fri, 2015-09-04 at 01:40 +0100, Wei Liu wrote:
>> Hi David
>>
>> This issue is exposed by the introduction of migration v2. The symptom is 
>> that
>> a guest with 32 bit 4.1 kernel can't be restored because it's asking for too
>> many pages.
> 
> FWIW my adhoc tests overnight gave me:
> 
> 37858: b953c0d234bc72e8489d3bf51a276c5c4ec85345 v4.1  Fail
> 37862: 39a8804455fb23f09157341d3ba7db6d7ae6ee76 v4.0  Fail
> 37860: bfa76d49576599a4b9f9b7a71f23d73d6dcff735 v3.19 Fail
> 
> 37872: e36f014edff70fc02b3d3d79cead1d58f289332e v3.19-rc7 Fail
> 37866: 26bc420b59a38e4e6685a73345a0def461136dce v3.19-rc6 Fail
> 37868: ec6f34e5b552fb0a52e6aae1a5afbbb1605cc6cc v3.19-rc5 Fail
> 37864: eaa27f34e91a14cdceed26ed6c6793ec1d186115 v3.19-rc4 Fail *
> 37867: b1940cd21c0f4abdce101253e860feff547291b0 v3.19-rc3 Pass *
> 37865: b7392d2247cfe6771f95d256374f1a8e6a6f48d6 v3.19-rc2 Pass
> 
> 37863: 97bf6af1f928216fd6c5a66e8a57bfa95a659672 v3.19-rc1 Pass
> 
> 37861: b2776bf7149bddd1f4161f14f79520f17fc1d71d v3.18 Pass
> 
> I have set the adhoc bisector working on the ~200 commits between rc3 and
> rc4. It's running in the Citrix instance (which is quieter) so the interim
> results are only visible within our network at http://osstest.xs.citrite.ne
> t/~osstest/testlogs/results-adhoc/bisect/xen-unstable/test-amd64-i386
> -xl..html.
> 
> So far it has confirmed the basis fail and it is now rechecking the basis
> pass.
> 
> Slightly strange though is:
> $ git log --oneline v3.19-rc3..v3.19-rc4 -- drivers/xen/ arch/x86/xen/ 
> include/xen/
> $
> 
> i.e. there are no relevant seeming xen commits in that range. Maybe the
> last one of this is more relevant?

Since this bisect attempt appears to have disappeared into the weeds I
did my own and it fingered:

633d6f17cd91ad5bf2370265946f716e42d388c6 (x86/xen: prepare p2m list for
memory hotplug) which was introduced in 4.0-rc7.

This looks a lot more plausible as the Linux change triggering the
migration failures.

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2] public/io/netif.h: move and amend multicast control documentation

2015-09-04 Thread Paul Durrant
> -Original Message-
> From: Ian Campbell [mailto:ian.campb...@citrix.com]
> Sent: 04 September 2015 15:42
> To: Wei Liu; Paul Durrant
> Cc: xen-de...@lists.xenproject.org; Keir (Xen.org); Tim (Xen.org); Ian
> Jackson; Jan Beulich
> Subject: Re: [Xen-devel] [PATCH v2] public/io/netif.h: move and amend
> multicast control documentation
> 
> On Fri, 2015-09-04 at 14:22 +0100, Wei Liu wrote:
> > On Wed, Sep 02, 2015 at 12:17:05PM +0100, Paul Durrant wrote:
> > > netif.h contains a specification of the
> > > XEN_NETIF_EXTRA_TYPE_MCAST_{ADD,DEL}
> > > extra info messages require to manipulate a multicast filter list
> > > maintained
> > > by a backend and specifies the xenstore negotiation protocol in a
> > > comment
> > > just above the structure defintion, which is easy to miss.
> > >
> > > This patch moves the documentation of the xenstore negotiation to be
> > > co-located with the documentation for other features and also amends
> > > the
> > > wording to be clearer.
> > >
> > > Signed-off-by: Paul Durrant 
> > > Cc: Ian Campbell 
> > > Cc: Ian Jackson 
> > > Cc: Jan Beulich 
> > > Cc: Keir Fraser 
> > > Cc: Tim Deegan 
> >
> > Acked-by: Wei Liu 
> 
> Applied, with:
> 
> > > + * connected state.
> >
> > I would prefer adding a blank line here if possible.
> 
> "\n *" added.

Cool. Thanks,

  Paul
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] MAINTAINERS: tools/ocaml: update David Scott's email address

2015-09-04 Thread Ian Campbell
On Wed, 2015-09-02 at 13:25 +0100, David Scott wrote:
> > 
> > On 2 Sep 2015, at 11:57, Ian Campbell  wrote:
> > 
> > On Wed, 2015-09-02 at 11:04 +0100, David Scott wrote:
> > > From: David Scott 
> > > 
> > > Replace my sometimes unreliable  address 
> > > with
> > > my reliable permanent address.
> > > 
> > 
> > I'd add (assuming I've got cause and effect right):
> > 
> > Reported-by: Doug Goldstein 
> 
> I think you have :-)

Added.

> > > Signed-off-by: David Scott 
> > > ---
> > > MAINTAINERS | 2 +-
> > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > 
> > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > index 73a96c9..a7fad84 100644
> > > --- a/MAINTAINERS
> > > +++ b/MAINTAINERS
> > > @@ -237,7 +237,7 @@ F:config/MiniOS.mk
> > > F:extras/mini-os/
> > > 
> > > OCAML TOOLS
> > > -M:   David Scott 
> > > +M:   David Scott 
> > 
> > Given your previous Ack of Doug's patch and the real From (not the 
> > pseudo
> > one) you used when sending I have to ask: Is this what you meant rather
> > than  (i.e. just dropping the eu.)
> 
> Yes — I was intending to switch to  anyway relatively 
> soon. I Acked Doug’s patch before I saw the confusion over (dave|david) 
> and (|eu) and thought this was a good time to make the switch. Maybe I’ve 
> confused things further!

Acked + applied.


> 
> 
> Cheers,
> David Scott

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 01/31] xen/dt: Handle correctly node with interrupt-map in dt_for_each_irq_map

2015-09-04 Thread Ian Campbell
On Wed, 2015-09-02 at 16:45 +0100, Julien Grall wrote:
> On 02/09/15 16:28, Ian Campbell wrote:
> > On Mon, 2015-08-31 at 15:20 +0100, Julien Grall wrote:
> > > Hi Vijay,
> > > 
> > > On 31/08/2015 12:06, vijay.kil...@gmail.com wrote:
> > > > From: Vijaya Kumar K 
> > > > 
> > > > dt_for_each_irq_map() returns error if no irq mapping is found.
> > > > With this patch, Ignore error and return success
> > > 
> > > NIT: s/Ignore/ignore/
> > > 
> > > > 
> > > > Signed-off-by: Vijaya Kumar K 
> > > 
> > > I think this could go in Xen 4.6 as it's a bug fix:
> > > 
> > > Reviewed-by: Julien Grall 
> > 
> > I replied to v5, forgetting I had a v6 in my hand too. I also forgot to 
> > CC
> > Wei. Lets try again here...
> > 
> > Acked-by: Ian Campbell 
> > 
> > Wei -- I think this fix would be good to have for 4.6. It is not an 
> > error
> > to process a zero-length (==non-existent) array I think.
> > 
> > I think the title should be s/with/without/, Julien was that what you 
> > meant
> > when you suggested it?
> 
> Yes.

Applied with that changed.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] build: update top-level make help

2015-09-04 Thread Ian Campbell
On Wed, 2015-09-02 at 15:31 +0100, Ian Campbell wrote:
> On Wed, 2015-09-02 at 09:22 -0500, Doug Goldstein wrote:
> > On 9/2/15 9:14 AM, Jan Beulich wrote:
> > > > > > On 02.09.15 at 15:34,  wrote:
> > > > On Wed, 2015-09-02 at 07:14 -0600, Jan Beulich wrote:
> > > > > > > > On 02.09.15 at 15:09,  wrote:
> > > > > > On Wed, Sep 2, 2015 at 3:28 AM, Jan Beulich  
> > > > > > 
> > > > > > wrote:
> > > > > > > > > > Doug Goldstein  09/01/15 10:10 PM 
> > > > > > > > > > >>>
> > > > > > > > --- a/Makefile
> > > > > > > > +++ b/Makefile
> > > > > > > > @@ -228,16 +228,23 @@ help:
> > > > > > > > @echo '  install-stubdom   - build and install the 
> > > > > > > > stubdomain 
> > > > > > > > images'
> > > > > > > > @echo '  install-docs  - build and install user 
> > > > > > > > documentation'
> > > > > > > > @echo ''
> > > > > > > > -  @echo 'Building targets:'
> > > > > > > > +  @echo 'Local dist targets:'
> > > > > > > > @echo '  dist  - build and install 
> > > > > > > > everything 
> > > > > > > > into 
> > > > > > > > local dist directory'
> > > > > > > > @echo '  world - clean everything then make 
> > > > > > > > 
> > > > > > > > dist'
> > > > > > > > -  @echo '  xen   - build and install 
> > > > > > > > Xen 
> > > > > > > > 
> > > > > > > > hypervisor'
> > > > > > > > -  @echo '  tools - build and install 
> > > > > > > > tools'
> > > > > > > > -  @echo '  stubdom   - build and install 
> > > > > > > > the 
> > > > > > > > 
> > > > > > > > stubdomain images'
> > > > > > > > -  @echo '  docs  - build and install 
> > > > > > > > user 
> > > > > > > > documentation'
> > > > > > > 
> > > > > > > Why are these being removed? At least the xen and tools ones 
> > > > > > > work 
> > > > > > > fine for me.
> > > > > > 
> > > > > > In the makefile they're marked as "legacy" and they are wired 
> > > > > > straight
> > > > > > to their "dist-" equivalents.
> > > > > 
> > > > > But still they're there and they work (and they're faster to type 
> > > > > 
> > > > > than
> > > > > their "dist-" equivalents).
> > > > 
> > > > If they are legacy then it is correct to not list them here.
> > > 
> > > Hmm - why does "legacy" imply "not documented here"?
> > > 
> > > But anyway - I don't mean to stand in the way of you committing the
> > > patch as is, as long as those (legacy) build targets don't go away.
> > > 
> > > Jan
> > > 
> > 
> > I definitely had no intention of removing them. I was merely trying to
> > update the help output to match what I saw when reading the contents of
> > the Makefile.
> > 
> > Do I still need to resubmit or is my S-o-b on the ML ok?
> 
> A post-facto S-o-b like you did is fine.

And now I've applied.

> 
> Ian.
> 
> 
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for 4.6] tools/xen-access: use PRI_xen_pfn

2015-09-04 Thread Ian Campbell
On Thu, 2015-09-03 at 19:27 +0100, Wei Liu wrote:
> Otherwise when building with 32bit compiler, we get:
> 
> xen-access.c: In function 'xenaccess_init':
> xen-access.c:263:5: error: format '%llx' expects argument of type 'long 
> long unsigned int', but argument 3 has type 'xen_pfn_t' [-Werror=format]
> cc1: all warnings being treated as errors
> 
> Signed-off-by: Wei Liu 

Applied with both acks.

> ---
> Cc: Razvan Cojocaru 
> Cc: Tamas K Lengyel 
> Cc: Ian Jackson 
> Cc: Ian Campbell 
> ---
>  tools/tests/xen-access/xen-access.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/tests/xen-access/xen-access.c b/tools/tests/xen
> -access/xen-access.c
> index cdb8f4e..a52ca6e 100644
> --- a/tools/tests/xen-access/xen-access.c
> +++ b/tools/tests/xen-access/xen-access.c
> @@ -260,7 +260,7 @@ xenaccess_t *xenaccess_init(xc_interface **xch_r, 
> domid_t domain_id)
>  goto err;
>  }
>  
> -DPRINTF("max_gpfn = %"PRIx64"\n", xenaccess->max_gpfn);
> +DPRINTF("max_gpfn = %"PRI_xen_pfn"\n", xenaccess->max_gpfn);
>  
>  return xenaccess;
>  

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] xen/arm64: do not (incorrectly) limit size of xenheap

2015-09-04 Thread Ian Campbell
On Fri, 2015-09-04 at 14:16 +0100, Wei Liu wrote:
> On Fri, Sep 04, 2015 at 01:57:00PM +0100, Julien Grall wrote:
> > The commit 88e3ed61642bb393458acc7a9bd2f96edc337190 "x86/NUMA: make
> > init_node_heap() respect Xen heap limit" breaks boot on the arm64 board
> > X-Gene.
> > 
> > The xenheap bits variable is used to know the last RAM MFN always 
> > mapped
> > in Xen virtual memory. If the value is 0, it means that all the memory 
> > is
> > always mapped in Xen virtual memory.
> > 
> > On X-gene the RAM bank resides above 128GB and last xenheap MFN is
> > 0x440. With the new way to calculate the number of bits, 
> > xenheap_bits
> > will be equal to 38 bits. This will result to hide all the RAM and the
> > impossibility to allocate xenheap memory.
> > 
> > Given that aarch64 have always all the memory mapped in Xen virtual
> > memory, it's not necessary to call xenheap_max_mfn which set the number
> > of bits.
> > 
> > Suggested-by: Jan Beulich 
> > Signed-off-by: Julien Grall 
> > Acked-by: Ian Campbell 
> > 
> 
> Release-acked-by: Wei Liu 

Applied.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 06/18] vmx: Add some helper functions for Posted-Interrupts

2015-09-04 Thread Jan Beulich
>>> On 25.08.15 at 03:57,  wrote:
> @@ -121,11 +122,31 @@ static inline int pi_test_and_clear_on(struct pi_desc 
> *pi_desc)
>  return test_and_clear_bit(POSTED_INTR_ON, &pi_desc->control);
>  }
>  
> +static inline int pi_test_on(struct pi_desc *pi_desc)
> +{
> +return test_bit(POSTED_INTR_ON, &pi_desc->control);
> +}

For this and ...

> +static inline int pi_test_sn(struct pi_desc *pi_desc)
> +{
> +return test_bit(POSTED_INTR_SN, &pi_desc->control);
> +}

... this I wonder whether using the bitfield you defined in the
previous patch wouldn't allow the compiler more freedom in
how to carry this out.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [Draft C] Boot ABI for HVM guests without a device-model

2015-09-04 Thread Roger Pau Monné
El 04/09/15 a les 16.08, Jan Beulich ha escrit:
 On 04.09.15 at 14:11,  wrote:
>>  * cs: must be a 32-bit read/execute code segment with an offset of ‘0’
>>and a limit of ‘0x’. The selector value is unspecified.
>>
>>  * ds, es: must be a 32-bit read/write data segment with an offset of
>>‘0’ and a limit of ‘0x’. The selector values are all unspecified.
> 
> In both cases s/offset/base/.

Right, sorry.

> 
>>  * tr: must be a 32-bit TSS (active) with a base of '0' and a limit of 
>> '0xFF'.
> 
> Already on the previous version I asked about this strange 0xFF,
> and I don't recall any answer.

My bad, I have actually changed this in the code (see v6), but forgot to
update the design document. 0x67 is perfectly fine, and is what should
be used.

Just for the record, I've initially set it to 0xff because that's how
it's done in construct_vmcs.

>>  * eflags: bit 17 (VM) must be cleared. Bit 9 (IF) must be cleared.
>>Other bits are all unspecified.
> 
> At the very least I'd also require TF to be clear.

Yes, that makes sense. I'm quite surprised that multiboot1 doesn't
mandate TF to also be cleared.

>> The format of the structure passed in the %ebx register is the following:
> 
> Even if it may sound like splitting hair: Please use precise wording. It's
> not the structure that's contained in %ebx, but %ebx hold the address
> of that structure.

Would you be fine with replacing this sentence with:

The format of the boot start info structure is the following:

>> struct hvm_start_info {
>> #define HVM_START_MAGIC_VALUE 0x336ec578
>> uint32_t magic; /* Contains the magic value 0x336ec578   
>> */
>> /* ("xEn3" with the 0x80 bit of the "E" 
>> set).*/
>> uint32_t flags; /* SIF_xxx flags.
>> */
> 
> Do really mean to re-use the SIF_* flags here?

We can introduce a new set of flags, HVM_INIT_*, which ATM is only going
to be:

#define HVM_FLAGS_INITDOMAIN (1<<0)

> 
>> AP startup
>> ==
>>
>> AP startup is performed using hypercalls. The following VCPU operations
>> are used in order to bring up secondary vCPUs:
>>
>>  * VCPUOP_initialise is used to set the initial state of the vCPU. The
>>argument passed to the hypercall must be of the type vcpu_hvm_context.
> 
> VCPUOP_initialise takes a struct vcpu_guest_context; I don't think
> we can or should change that.

Didn't we agree that vcpu_guest_context was not suitable for HVM/PVH guests?

Patch 24 of my HVM-without-dm series already introduces this new
structure and the necessary helpers.

Roger.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 04/18] vt-d: VT-d Posted-Interrupts feature detection

2015-09-04 Thread Jan Beulich
>>> On 25.08.15 at 03:57,  wrote:
> --- a/xen/drivers/passthrough/vtd/iommu.c
> +++ b/xen/drivers/passthrough/vtd/iommu.c
> @@ -2079,6 +2079,9 @@ static int init_vtd_hw(void)
>  disable_intremap(drhd->iommu);
>  }
>  
> +if ( !iommu_intremap )
> +iommu_intpost = 0;

Why is this being repeated here? iommu_setup() is already taking
care of this thanks to the earlier patch.

> @@ -2176,6 +2179,14 @@ int __init intel_vtd_setup(void)
>  if ( iommu_intremap && !ecap_intr_remap(iommu->ecap) )
>  iommu_intremap = 0;
>  
> +/*
> + * We cannot use posted interrupt if X86_FEATURE_CX16 is
> + * not supported, since we count on this feature to
> + * atomically update 16-byte IRTE in posted format.
> + */
> +if ( !iommu_intremap || !cap_intr_post(iommu->cap) || !cpu_has_cx16 )
> +iommu_intpost = 0;

You should check iommu_intpost instead of iommu_intremap to
match the other checks above here.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 05/18] vmx: Extend struct pi_desc to support VT-d Posted-Interrupts

2015-09-04 Thread Jan Beulich
>>> On 25.08.15 at 03:57,  wrote:
> Extend struct pi_desc according to VT-d Posted-Interrupts Spec.
> 
> CC: Kevin Tian 
> CC: Keir Fraser 
> CC: Jan Beulich 
> CC: Andrew Cooper 
> Signed-off-by: Feng Wu 
> Reviewed-by: Andrew Cooper 
> Acked-by: Kevin Tian 
> Reviewed-by: Konrad Rzeszutek Wilk 
> ---
> v3:
> - Use u32 instead of u64 for the bitfield in 'struct pi_desc'
> 
>  xen/include/asm-x86/hvm/vmx/vmcs.h | 15 +--
>  1 file changed, 13 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/include/asm-x86/hvm/vmx/vmcs.h 
> b/xen/include/asm-x86/hvm/vmx/vmcs.h
> index f1126d4..7e81752 100644
> --- a/xen/include/asm-x86/hvm/vmx/vmcs.h
> +++ b/xen/include/asm-x86/hvm/vmx/vmcs.h
> @@ -80,8 +80,19 @@ struct vmx_domain {
>  
>  struct pi_desc {
>  DECLARE_BITMAP(pir, NR_VECTORS);
> -u32 control;
> -u32 rsvd[7];
> +union {
> +struct
> +{

The brace belongs on the previous line.

> +u16 on : 1,  /* bit 256 - Outstanding Notification */
> +sn : 1,  /* bit 257 - Suppress Notification */
> +rsvd_1 : 14; /* bit 271:258 - Reserved */
> +u8  nv;  /* bit 279:272 - Notification Vector */
> +u8  rsvd_2;  /* bit 287:280 - Reserved */
> +u32 ndst;/* bit 319:288 - Notification Destination */

Too little indentation.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [Pkg-xen-devel] Bug#795330: Suggests noapic but doesn't support it

2015-09-04 Thread David Vrabel
On 04/09/15 15:13, Ian Campbell wrote:
> On Fri, 2015-09-04 at 15:04 +0100, David Vrabel wrote:
>> On 04/09/15 14:38, Ian Campbell wrote:
>>> Control: tag -1 upstream 
>>>
>>> Jan/Andy & David,
>>>
>>> Is there anything we can improve here on either the Xen or kernel side 
>>> do
>>> you think?
>>>
>>> On Thu, 2015-08-13 at 00:59 +0200, Guido Günther wrote:
 Package: xen-hypervisor-4.5-amd64
 Severity: normal

 Hi,
 when running xen inside of kvm the hypervisor fails to set up the 
 proper
 IRQ routing and suggests to use noapic.
>>>
>>> FTR, it seems to say:
>>>
>>> panic("IO-APIC + timer doesn't work!  Boot with 
>>> apic_verbosity=debug "
>>>   "and send a report.  Then try booting with the 'noapic' 
>>> option");
>>
>> This is the Linux kernel making a recommendation for its command line.
> 
> I cut-n-paste that message from xen.git. Although maybe in a file which
> originates in Linux.
> 
> (So now I realised I don't really know which entity was complaining)

Oh, in which case removing (from Xen) the recommendation to try noapic
might be useful.

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 03/18] iommu: Add iommu_intpost to control VT-d Posted-Interrupts feature

2015-09-04 Thread Jan Beulich
>>> On 25.08.15 at 03:57,  wrote:
> VT-d Posted-Interrupts is an enhancement to CPU side Posted-Interrupt.
> With VT-d Posted-Interrupts enabled, external interrupts from
> direct-assigned devices can be delivered to guests without VMM
> intervention when guest is running in non-root mode.
> 
> This patch adds variable 'iommu_intpost' to control whether enable VT-d
> posted-interrupt or not in the generic IOMMU code.
> 
> Signed-off-by: Feng Wu 
> Reviewed-by: Kevin Tian 
> Reviewed-by: Konrad Rzeszutek Wilk 

Acked-by: Jan Beulich 




___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 02/18] Add cmpxchg16b support for x86-64

2015-09-04 Thread Jan Beulich
>>> On 25.08.15 at 03:57,  wrote:
> This patch adds cmpxchg16b support for x86-64, so software
> can perform 128-bit atomic write/read.
> 
> CC: Keir Fraser 
> CC: Jan Beulich 
> CC: Andrew Cooper 
> Signed-off-by: Feng Wu 
> ---
> v6:
> - Fix a typo
> 
> v5:
> - Change back the parameters of __cmpxchg16b() to __uint128_t *
> - Remove pointless cast for 'ptr'
> - Remove pointless parentheses
> - Use A constraint for the output

There are a few comments I had on v4 which neither have been
addressed verbally nor have got incorporated into the patch. See
http://lists.xenproject.org/archives/html/xen-devel/2015-07/msg04694.html

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [Pkg-xen-devel] Bug#795330: Suggests noapic but doesn't support it

2015-09-04 Thread Ian Campbell
On Fri, 2015-09-04 at 15:04 +0100, David Vrabel wrote:
> On 04/09/15 14:38, Ian Campbell wrote:
> > Control: tag -1 upstream 
> > 
> > Jan/Andy & David,
> > 
> > Is there anything we can improve here on either the Xen or kernel side 
> > do
> > you think?
> > 
> > On Thu, 2015-08-13 at 00:59 +0200, Guido Günther wrote:
> > > Package: xen-hypervisor-4.5-amd64
> > > Severity: normal
> > > 
> > > Hi,
> > > when running xen inside of kvm the hypervisor fails to set up the 
> > > proper
> > > IRQ routing and suggests to use noapic.
> > 
> > FTR, it seems to say:
> > 
> > panic("IO-APIC + timer doesn't work!  Boot with 
> > apic_verbosity=debug "
> >   "and send a report.  Then try booting with the 'noapic' 
> > option");
> 
> This is the Linux kernel making a recommendation for its command line.

I cut-n-paste that message from xen.git. Although maybe in a file which
originates in Linux.

(So now I realised I don't really know which entity was complaining)

> > Did you also try the first thing it suggested? What was the result?
> > 
> > >  If you add this via
> > > 
> > > GRUB_CMDLINE_XEN_DEFAULT
> > > 
> > > it will boot futher but then report that noapic isn't supported.
> 
> Don't add Linux command line options to the Xen command line.
> 
> David
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] xen: limit memory to architectural maximum

2015-09-04 Thread Roger Pau Monné
El 04/09/15 a les 14.18, Juergen Gross ha escrit:
> When a pv-domain (including dom0) is started it tries to size it's
> p2m list according to the maximum possible memory amount it ever can
> achieve. Limit the initial maximum memory size to the architectural
> limit of the hardware in order to avoid overflows during remapping
> of memory.
> 
> This problem will occur when dom0 is started with an initial memory
> size being a multiple of 1GB, but without specifying it's maximum
> memory size. The kernel must be configured without
> CONFIG_XEN_BALLOON_MEMORY_HOTPLUG for the problem to happen.
> 
> Reported-by: Roger Pau Monné 
> Signed-off-by: Juergen Gross 

Tested-by: Roger Pau Monné 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] xen: switch extra memory accounting to use pfns

2015-09-04 Thread Roger Pau Monné
El 04/09/15 a les 14.05, Juergen Gross ha escrit:
> Instead of using physical addresses for accounting of extra memory
> areas available for ballooning switch to pfns as this is much less
> error prone regarding partial pages.
> 
> Reported-by: Roger Pau Monné 
> Signed-off-by: Juergen Gross 

Tested-by: Roger Pau Monné 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] PAGE_SIZE (64KB), while block driver 'struct request' deals with < PAGE_SIZE (up to 44Kb). Was:Re: [RFC] Support of non-indirect grant backend on 64KB guest

2015-09-04 Thread Stefano Stabellini
On Thu, 27 Aug 2015, Julien Grall wrote:
> On 21/08/15 18:10, Konrad Rzeszutek Wilk wrote:
> > On Fri, Aug 21, 2015 at 05:08:35PM +0100, David Vrabel wrote:
> >> On 21/08/15 17:05, Konrad Rzeszutek Wilk wrote:
> >>>
> >>> I have to concur with that. We can't mandate that ARM 64k page MUST use
> >>> indirect descriptors.
> >>
> >> Then it has to be fixed in the block layer to allow < PAGE_SIZE segments
> >> and to get the block layer to split requests for blkfront.
> > 
> > Hey Jens,
> > 
> > I am hoping you can help us figure this problem out.
> > 
> > The Linux ARM is capable of using 4KB pages and 64KB pages. Our block
> > driver (xen-blkfront) was built with 4KB pages in mind and without using
> > any fancy flags (which some backends lack) the maximum amount of I/O it can
> > fit on a ring is 44KB.
> > 
> > This has the unfortunate effect that when the xen-blkfront
> > gets an 'struct request' it can have on page (64KB) and it can't actually
> > fit it on the ring! And the lowest segment size it advertises is PAGE_SIZE
> > (64KB). I believe Julien (who found this) tried initially advertising
> > smaller segment size than PAGE_SIZE (32KB). However looking at
> > __blk_segment_map_sg it looks to assume smallest size is PAGE_SIZE so
> > that would explain why it did not work.
> 
> To be honest, I haven't tried to see how the block layer will act if I
> dropped those checks in blk-settings.c until today.
> 
> I don't see any assumption about PAGE_SIZE in __blk_segment_map_sg.
> Although dropping the checks in blk-settings (see quick patch [1]),
> I got the following error in the frontend:
> 
> bio too big device xvda (128 > 88)
> Buffer I/O error on dev xvda, logical block 0, async page read
> bio too big device xvda (128 > 88)
> Buffer I/O error on dev xvda, logical block 0, async page read
> 
> The "bio too big device ..." comes from generic_make_request_checks
> (linux/block/blk-core.c) and the stack trace is:
> 
> [] dump_backtrace+0x0/0x124
> [] show_stack+0x10/0x1c
> [] dump_stack+0x78/0xbc
> [] warn_slowpath_common+0x98/0xd0
> [] warn_slowpath_null+0x14/0x20
> [] generic_make_request_checks+0x114/0x230
> [] generic_make_request+0x10/0x108
> [] submit_bio+0x94/0x1e0
> [] submit_bh_wbc.isra.36+0x100/0x1a8
> [] block_read_full_page+0x320/0x3e8
> [] blkdev_readpage+0x14/0x20
> [] do_read_cache_page+0x16c/0x1a0
> [] read_cache_page+0x10/0x1c
> [] read_dev_sector+0x30/0x9c
> [] msdos_partition+0x84/0x554
> [] check_partition+0xf8/0x21c
> [] rescan_partitions+0xb0/0x2a4
> [] __blkdev_get+0x228/0x34c
> [] blkdev_get+0x40/0x364
> [] add_disk+0x398/0x424
> [] blkback_changed+0x1200/0x152c
> [] xenbus_otherend_changed+0x9c/0xa4
> [] backend_changed+0xc/0x18
> [] xenwatch_thread+0xa0/0x13c
> [] kthread+0xd8/0xf0
> 
> The fs buffer code seems to assume that the block driver will always support
> at least a bio of PAGE_SIZE.
> 
> > One wya to make this work is for the driver (xen-blkfront) to split
> > the 'struct request' I/O in two internal requests.
> > 
> > But this has to be a normal problem. Surely there are other drivers
> > (MMC?) that can't handle PAGE_SIZE and have to break their I/Os.
> > Would it make sense for the common block code to be able to deal
> > with this?
> 
> It will take me a bit of time to fully understand the block layer
> as the changes doesn't seem trivial from POV (I don't have any
> knowledge in it). So I will wait a feedback from Jens before
> going further on this approach.

Maybe we could fall back to the previous plan of modifying xen-blkfront
for the moment?

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [Draft C] Boot ABI for HVM guests without a device-model

2015-09-04 Thread Jan Beulich
>>> On 04.09.15 at 14:11,  wrote:
>  * cs: must be a 32-bit read/execute code segment with an offset of ‘0’
>and a limit of ‘0x’. The selector value is unspecified.
> 
>  * ds, es: must be a 32-bit read/write data segment with an offset of
>‘0’ and a limit of ‘0x’. The selector values are all unspecified.

In both cases s/offset/base/.

>  * tr: must be a 32-bit TSS (active) with a base of '0' and a limit of '0xFF'.

Already on the previous version I asked about this strange 0xFF,
and I don't recall any answer.

>  * eflags: bit 17 (VM) must be cleared. Bit 9 (IF) must be cleared.
>Other bits are all unspecified.

At the very least I'd also require TF to be clear.

> The format of the structure passed in the %ebx register is the following:

Even if it may sound like splitting hair: Please use precise wording. It's
not the structure that's contained in %ebx, but %ebx hold the address
of that structure.

> struct hvm_start_info {
> #define HVM_START_MAGIC_VALUE 0x336ec578
> uint32_t magic; /* Contains the magic value 0x336ec578   
> */
> /* ("xEn3" with the 0x80 bit of the "E" 
> set).*/
> uint32_t flags; /* SIF_xxx flags.
> */

Do really mean to re-use the SIF_* flags here?

> AP startup
> ==
> 
> AP startup is performed using hypercalls. The following VCPU operations
> are used in order to bring up secondary vCPUs:
> 
>  * VCPUOP_initialise is used to set the initial state of the vCPU. The
>argument passed to the hypercall must be of the type vcpu_hvm_context.

VCPUOP_initialise takes a struct vcpu_guest_context; I don't think
we can or should change that.

Jan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [Pkg-xen-devel] Bug#795330: Suggests noapic but doesn't support it

2015-09-04 Thread David Vrabel
On 04/09/15 14:38, Ian Campbell wrote:
> Control: tag -1 upstream 
> 
> Jan/Andy & David,
> 
> Is there anything we can improve here on either the Xen or kernel side do
> you think?
> 
> On Thu, 2015-08-13 at 00:59 +0200, Guido Günther wrote:
>> Package: xen-hypervisor-4.5-amd64
>> Severity: normal
>>
>> Hi,
>> when running xen inside of kvm the hypervisor fails to set up the proper
>> IRQ routing and suggests to use noapic.
> 
> FTR, it seems to say:
> 
> panic("IO-APIC + timer doesn't work!  Boot with apic_verbosity=debug "
>   "and send a report.  Then try booting with the 'noapic' option");

This is the Linux kernel making a recommendation for its command line.

> Did you also try the first thing it suggested? What was the result?
> 
>>  If you add this via
>>
>> GRUB_CMDLINE_XEN_DEFAULT
>>
>> it will boot futher but then report that noapic isn't supported.

Don't add Linux command line options to the Xen command line.

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 11/29] xen/x86: add bitmap of enabled emulated devices

2015-09-04 Thread Wei Liu
On Fri, Sep 04, 2015 at 03:51:17PM +0200, Roger Pau Monné wrote:
> El 04/09/15 a les 14.25, Wei Liu ha escrit:
> > On Fri, Sep 04, 2015 at 02:08:50PM +0200, Roger Pau Monne wrote:
> >> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> >> index 045f6ff..fe9504f 100644
> >> --- a/xen/arch/x86/domain.c
> >> +++ b/xen/arch/x86/domain.c
> >> @@ -555,6 +555,29 @@ int arch_domain_create(struct domain *d, unsigned int 
> >> domcr_flags,
> >> d->domain_id);
> >>  }
> >>  
> >> +if ( is_hvm_domain(d) )
> >> +{
> >> +uint32_t emulation_mask = (XEN_X86_EMU_LAPIC | XEN_X86_EMU_HPET |
> >> +   XEN_X86_EMU_PMTIMER | XEN_X86_EMU_RTC |
> >> +   XEN_X86_EMU_IOAPIC | XEN_X86_EMU_PIC |
> >> +   XEN_X86_EMU_PMU | XEN_X86_EMU_VGA |
> >> +   XEN_X86_EMU_IOMMU);
> > 
> > This is repetitive. Could you consolidate all these to
> > 
> >   #define XEN_X86_EMU_ALL ...
> > 
> > ?
> 
> That sounds fine, I would place it in the public header where all the
> XEN_X86_EMU_* are defined. I will wait for Andrew's opinion, since he
> already acked this patch.

Of course he has the final authority here. I don't want block this patch
just for such cosmetic comment. :-)

Wei.

> 
> Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 11/29] xen/x86: add bitmap of enabled emulated devices

2015-09-04 Thread Jan Beulich
>>> On 04.09.15 at 15:51,  wrote:
> El 04/09/15 a les 14.25, Wei Liu ha escrit:
>> On Fri, Sep 04, 2015 at 02:08:50PM +0200, Roger Pau Monne wrote:
>>> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
>>> index 045f6ff..fe9504f 100644
>>> --- a/xen/arch/x86/domain.c
>>> +++ b/xen/arch/x86/domain.c
>>> @@ -555,6 +555,29 @@ int arch_domain_create(struct domain *d, unsigned int 
> domcr_flags,
>>> d->domain_id);
>>>  }
>>>  
>>> +if ( is_hvm_domain(d) )
>>> +{
>>> +uint32_t emulation_mask = (XEN_X86_EMU_LAPIC | XEN_X86_EMU_HPET |
>>> +   XEN_X86_EMU_PMTIMER | XEN_X86_EMU_RTC |
>>> +   XEN_X86_EMU_IOAPIC | XEN_X86_EMU_PIC |
>>> +   XEN_X86_EMU_PMU | XEN_X86_EMU_VGA |
>>> +   XEN_X86_EMU_IOMMU);
>> 
>> This is repetitive. Could you consolidate all these to
>> 
>>   #define XEN_X86_EMU_ALL ...
>> 
>> ?
> 
> That sounds fine, I would place it in the public header where all the
> XEN_X86_EMU_* are defined. I will wait for Andrew's opinion, since he
> already acked this patch.

This doesn't belong in the public ABI, so if at all it should be added
there inside #ifdef __XEN__. Alternatively (and perhaps preferably)
this would go into another (internal) header.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for 4.6] build: fix tarball stubdom build

2015-09-04 Thread Ian Campbell
On Fri, 2015-09-04 at 08:48 -0500, Doug Goldstein wrote:
> On 8/27/15 12:58 PM, Ian Jackson wrote:
> > Wei Liu writes ("[PATCH for 4.6] build: fix tarball stubdom build"):
> > > When we create a source code tarball, mini-os is extracted to
> > > extras/mini-os directory. When building a source code tarball, we
> > > shouldn't clone mini-os again.
> > > 
> > > Only clone mini-os when that directory doesn't exist. This fixes 
> > > tarball
> > > build and doesn't affect non-tarball build.
> > 
> > Acked-by: Ian Jackson 
> > 
> > I am about to commit this.
> > 
> > Thanks,
> > Ian.
> 
> Ping. Do we want this to land for 4.6?

It is:

commit 0cc73e9870a96e18fc076618c0b419919794ae06
Author: Wei Liu 
Date:   Thu Aug 27 16:54:01 2015 +0100

build: fix tarball stubdom build

>  Since all its doing is
> documenting the Makefile targets in the actual 4.6 Makefile I would
> think its safe.

But that makes me think you hit rpely on the wrong email.

If you meant "build: update top-level make help" then that is mentioned in 
http://lists.xen.org/archives/html/xen-devel/2015-09/msg00533.html (and
I've not read Wei's reply yet)

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] xen-blkback: free requests on disconnection

2015-09-04 Thread Julien Grall
Hi Roger,

On 04/09/15 11:08, Roger Pau Monne wrote:
> Request allocation has been moved to connect_ring, which is called every
> time blkback connects to the frontend (this can happen multiple times during
> a blkback instance life cycle). On the other hand, request freeing has not
> been moved, so it's only called when destroying the backend instance. Due to
> this mismatch, blkback can allocate the request pool multiple times, without
> freeing it.
> 
> In order to fix it, move the freeing of requests to xen_blkif_disconnect to
> restore the symmetry between request allocation and freeing.
> 
> Reported-by: Julien Grall 
> Signed-off-by: Roger Pau Monné 
> Cc: Julien Grall 
> Cc: Konrad Rzeszutek Wilk 
> Cc: Boris Ostrovsky 
> Cc: David Vrabel 
> Cc: xen-de...@lists.xenproject.org

The patch is fixing my problem when using UEFI in the guest. Thank you!

Tested-by: Julien Grall 

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 11/29] xen/x86: add bitmap of enabled emulated devices

2015-09-04 Thread Roger Pau Monné
El 04/09/15 a les 14.25, Wei Liu ha escrit:
> On Fri, Sep 04, 2015 at 02:08:50PM +0200, Roger Pau Monne wrote:
>> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
>> index 045f6ff..fe9504f 100644
>> --- a/xen/arch/x86/domain.c
>> +++ b/xen/arch/x86/domain.c
>> @@ -555,6 +555,29 @@ int arch_domain_create(struct domain *d, unsigned int 
>> domcr_flags,
>> d->domain_id);
>>  }
>>  
>> +if ( is_hvm_domain(d) )
>> +{
>> +uint32_t emulation_mask = (XEN_X86_EMU_LAPIC | XEN_X86_EMU_HPET |
>> +   XEN_X86_EMU_PMTIMER | XEN_X86_EMU_RTC |
>> +   XEN_X86_EMU_IOAPIC | XEN_X86_EMU_PIC |
>> +   XEN_X86_EMU_PMU | XEN_X86_EMU_VGA |
>> +   XEN_X86_EMU_IOMMU);
> 
> This is repetitive. Could you consolidate all these to
> 
>   #define XEN_X86_EMU_ALL ...
> 
> ?

That sounds fine, I would place it in the public header where all the
XEN_X86_EMU_* are defined. I will wait for Andrew's opinion, since he
already acked this patch.

Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for 4.6] build: fix tarball stubdom build

2015-09-04 Thread Doug Goldstein
On 8/27/15 12:58 PM, Ian Jackson wrote:
> Wei Liu writes ("[PATCH for 4.6] build: fix tarball stubdom build"):
>> When we create a source code tarball, mini-os is extracted to
>> extras/mini-os directory. When building a source code tarball, we
>> shouldn't clone mini-os again.
>>
>> Only clone mini-os when that directory doesn't exist. This fixes tarball
>> build and doesn't affect non-tarball build.
> 
> Acked-by: Ian Jackson 
> 
> I am about to commit this.
> 
> Thanks,
> Ian.

Ping. Do we want this to land for 4.6? Since all its doing is
documenting the Makefile targets in the actual 4.6 Makefile I would
think its safe.

-- 
Doug Goldstein



signature.asc
Description: OpenPGP digital signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v11 10/11] libxl: add LIBXL_DEVICE_MODEL_SAVE_FILE

2015-09-04 Thread Vitaly Kuznetsov
Use this in libxl_dm instead of hard-coding.

Signed-off-by: Vitaly Kuznetsov 
Acked-by: Ian Campbell 
---
Changes since v10:
- s,XC,LIBXL,g to meet the XC_DEVICE_MODEL_RESTORE_FILE ->
  LIBXL_DEVICE_MODEL_RESTORE_FILE change.
---
 tools/libxl/libxl_dm.c   | 2 +-
 tools/libxl/libxl_internal.h | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
index c84085e..b7c6592 100644
--- a/tools/libxl/libxl_dm.c
+++ b/tools/libxl/libxl_dm.c
@@ -31,7 +31,7 @@ static const char *libxl_tapif_script(libxl__gc *gc)
 
 const char *libxl__device_model_savefile(libxl__gc *gc, uint32_t domid)
 {
-return libxl__sprintf(gc, "/var/lib/xen/qemu-save.%d", domid);
+return libxl__sprintf(gc, LIBXL_DEVICE_MODEL_SAVE_FILE".%d", domid);
 }
 
 static const char *qemu_xen_path(libxl__gc *gc)
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 6ea6c83..c3c3b7b 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -90,6 +90,7 @@
 /* QEMU may be slow to load and start due to a bug in Linux where the I/O
  * subsystem sometime produce high latency under load. */
 #define LIBXL_DEVICE_MODEL_START_TIMEOUT 60
+#define LIBXL_DEVICE_MODEL_SAVE_FILE "/var/lib/xen/qemu-save" /* .$domid */
 #define LIBXL_DEVICE_MODEL_RESTORE_FILE "/var/lib/xen/qemu-resume" /* .$domid 
*/
 #define LIBXL_STUBDOM_START_TIMEOUT 30
 #define LIBXL_QEMU_BODGE_TIMEOUT 2
-- 
2.4.3


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v11 03/11] xl: introduce enum domain_restart_type

2015-09-04 Thread Vitaly Kuznetsov
As a preparation before adding new restart type (soft reset) put all
restart types into an enum.

Signed-off-by: Vitaly Kuznetsov 
Acked-by: Ian Campbell 
Reviewed-by: Konrad Rzeszutek Wilk 
---
Changes since v10:
- None.
---
 tools/libxl/xl.h |  6 ++
 tools/libxl/xl_cmdimpl.c | 23 ++-
 2 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/tools/libxl/xl.h b/tools/libxl/xl.h
index 13bccba..6c19c0d 100644
--- a/tools/libxl/xl.h
+++ b/tools/libxl/xl.h
@@ -190,6 +190,12 @@ enum output_format {
 };
 extern enum output_format default_output_format;
 
+typedef enum {
+DOMAIN_RESTART_NONE = 0, /* No domain restart */
+DOMAIN_RESTART_NORMAL,   /* Domain should be restarted */
+DOMAIN_RESTART_RENAME,   /* Domain should be renamed and restarted */
+} domain_restart_type;
+
 extern void printf_info_sexp(int domid, libxl_domain_config *d_config, FILE 
*fh);
 
 #define XL_GLOBAL_CONFIG XEN_CONFIG_DIR "/xl.conf"
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 6fdc874..c9bd839 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -2411,15 +2411,12 @@ static void reload_domain_config(uint32_t domid,
 }
 }
 
-/* Returns 1 if domain should be restarted,
- * 2 if domain should be renamed then restarted, or 0
- * Can update r_domid if domain is destroyed etc */
-static int handle_domain_death(uint32_t *r_domid,
-   libxl_event *event,
-   libxl_domain_config *d_config)
-
+/* Can update r_domid if domain is destroyed */
+static domain_restart_type handle_domain_death(uint32_t *r_domid,
+   libxl_event *event,
+   libxl_domain_config *d_config)
 {
-int restart = 0;
+domain_restart_type restart = DOMAIN_RESTART_NONE;
 libxl_action_on_shutdown action;
 
 switch (event->u.domain_shutdown.shutdown_reason) {
@@ -2474,12 +2471,12 @@ static int handle_domain_death(uint32_t *r_domid,
 
 case LIBXL_ACTION_ON_SHUTDOWN_RESTART_RENAME:
 reload_domain_config(*r_domid, d_config);
-restart = 2;
+restart = DOMAIN_RESTART_RENAME;
 break;
 
 case LIBXL_ACTION_ON_SHUTDOWN_RESTART:
 reload_domain_config(*r_domid, d_config);
-restart = 1;
+restart = DOMAIN_RESTART_NORMAL;
 /* fall-through */
 case LIBXL_ACTION_ON_SHUTDOWN_DESTROY:
 LOG("Domain %d needs to be cleaned up: destroying the domain",
@@ -2946,7 +2943,7 @@ start:
 event->u.domain_shutdown.shutdown_reason,
 event->u.domain_shutdown.shutdown_reason);
 switch (handle_domain_death(&domid, event, &d_config)) {
-case 2:
+case DOMAIN_RESTART_RENAME:
 if (!preserve_domain(&domid, event, &d_config)) {
 /* If we fail then exit leaving the old domain in place. */
 ret = -1;
@@ -2954,7 +2951,7 @@ start:
 }
 
 /* Otherwise fall through and restart. */
-case 1:
+case DOMAIN_RESTART_NORMAL:
 libxl_event_free(ctx, event);
 libxl_evdisable_domain_death(ctx, deathw);
 deathw = NULL;
@@ -2990,7 +2987,7 @@ start:
 sleep(2);
 goto start;
 
-case 0:
+case DOMAIN_RESTART_NONE:
 LOG("Done. Exiting now");
 libxl_event_free(ctx, event);
 ret = 0;
-- 
2.4.3


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v11 07/11] flask: DOMCTL_soft_reset support

2015-09-04 Thread Vitaly Kuznetsov
Add new soft_reset vector to domain2 class, add it to create_domain
in the default policy.

Signed-off-by: Vitaly Kuznetsov 
Acked-by: Daniel De Graaf 
---
Changes since v10:
- None.
---
 tools/flask/policy/policy/modules/xen/xen.if | 2 +-
 xen/xsm/flask/hooks.c| 3 +++
 xen/xsm/flask/policy/access_vectors  | 2 ++
 3 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/tools/flask/policy/policy/modules/xen/xen.if 
b/tools/flask/policy/policy/modules/xen/xen.if
index a2f25e1..32dd7b3 100644
--- a/tools/flask/policy/policy/modules/xen/xen.if
+++ b/tools/flask/policy/policy/modules/xen/xen.if
@@ -52,7 +52,7 @@ define(`create_domain_common', `
getaffinity setaffinity setvcpuextstate };
allow $1 $2:domain2 { set_cpuid settsc setscheduler setclaim
set_max_evtchn set_vnumainfo get_vnumainfo cacheflush
-   psr_cmt_op psr_cat_op };
+   psr_cmt_op psr_cat_op soft_reset };
allow $1 $2:security check_context;
allow $1 $2:shadow enable;
allow $1 $2:mmu { map_read map_write adjust memorymap physmap pinpage 
mmuext_op updatemp };
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index 7a4522e..aa37096 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -738,6 +738,9 @@ static int flask_domctl(struct domain *d, int cmd)
 case XEN_DOMCTL_psr_cat_op:
 return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__PSR_CAT_OP);
 
+case XEN_DOMCTL_soft_reset:
+return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__SOFT_RESET);
+
 default:
 printk("flask_domctl: Unknown op %d\n", cmd);
 return -EPERM;
diff --git a/xen/xsm/flask/policy/access_vectors 
b/xen/xsm/flask/policy/access_vectors
index 71495fd..a11c788 100644
--- a/xen/xsm/flask/policy/access_vectors
+++ b/xen/xsm/flask/policy/access_vectors
@@ -232,6 +232,8 @@ class domain2
 # XEN_DOMCTL_monitor_op
 # XEN_DOMCTL_vm_event_op
 vm_event
+# XEN_DOMCTL_soft_reset
+soft_reset
 # XENMEM_access_op
 mem_access
 # XENMEM_paging_op
-- 
2.4.3


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v11 06/11] xen: Introduce XEN_DOMCTL_soft_reset

2015-09-04 Thread Vitaly Kuznetsov
New domctl resets state for a domain allowing it to 'start over': register
vcpu_info, switch to FIFO ABI for event channels. Still active grants are
being logged to help debugging misbehaving backends.

Signed-off-by: Vitaly Kuznetsov 
Acked-by: Jan Beulich 
---
Changes since v10:
- return when evtchn_reset() fails [Konrad Rzeszutek Wilk, Jan Beulich].
---
 xen/common/domain.c | 49 +
 xen/common/domctl.c |  9 +
 xen/include/public/domctl.h |  1 +
 xen/include/xen/sched.h |  2 ++
 4 files changed, 53 insertions(+), 8 deletions(-)

diff --git a/xen/common/domain.c b/xen/common/domain.c
index 1b9fcfc..42c8caf 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -110,6 +110,16 @@ static void vcpu_check_shutdown(struct vcpu *v)
 spin_unlock(&d->shutdown_lock);
 }
 
+static void vcpu_info_reset(struct vcpu *v)
+{
+struct domain *d = v->domain;
+
+v->vcpu_info = ((v->vcpu_id < XEN_LEGACY_MAX_VCPUS)
+? (vcpu_info_t *)&shared_info(d, vcpu_info[v->vcpu_id])
+: &dummy_vcpu_info);
+v->vcpu_info_mfn = INVALID_MFN;
+}
+
 struct vcpu *alloc_vcpu(
 struct domain *d, unsigned int vcpu_id, unsigned int cpu_id)
 {
@@ -145,10 +155,7 @@ struct vcpu *alloc_vcpu(
 v->runstate.state = RUNSTATE_offline;
 v->runstate.state_entry_time = NOW();
 set_bit(_VPF_down, &v->pause_flags);
-v->vcpu_info = ((vcpu_id < XEN_LEGACY_MAX_VCPUS)
-? (vcpu_info_t *)&shared_info(d, vcpu_info[vcpu_id])
-: &dummy_vcpu_info);
-v->vcpu_info_mfn = INVALID_MFN;
+vcpu_info_reset(v);
 init_waitqueue_vcpu(v);
 }
 
@@ -1038,6 +1045,34 @@ void domain_unpause_except_self(struct domain *d)
 domain_unpause(d);
 }
 
+int domain_soft_reset(struct domain *d)
+{
+struct vcpu *v;
+int rc;
+
+spin_lock(&d->shutdown_lock);
+for_each_vcpu ( d, v )
+if ( !v->paused_for_shutdown )
+{
+spin_unlock(&d->shutdown_lock);
+return -EINVAL;
+}
+spin_unlock(&d->shutdown_lock);
+
+rc = evtchn_reset(d);
+if ( rc )
+return rc;
+
+grant_table_warn_active_grants(d);
+
+for_each_vcpu ( d, v )
+unmap_vcpu_info(v);
+
+domain_resume(d);
+
+return 0;
+}
+
 int vcpu_reset(struct vcpu *v)
 {
 struct domain *d = v->domain;
@@ -1149,8 +1184,7 @@ int map_vcpu_info(struct vcpu *v, unsigned long gfn, 
unsigned offset)
 
 /*
  * Unmap the vcpu info page if the guest decided to place it somewhere
- * else.  This is only used from arch_domain_destroy, so there's no
- * need to do anything clever.
+ * else. This is used from arch_domain_destroy and domain_soft_reset.
  */
 void unmap_vcpu_info(struct vcpu *v)
 {
@@ -1163,8 +1197,7 @@ void unmap_vcpu_info(struct vcpu *v)
 unmap_domain_page_global((void *)
  ((unsigned long)v->vcpu_info & PAGE_MASK));
 
-v->vcpu_info = &dummy_vcpu_info;
-v->vcpu_info_mfn = INVALID_MFN;
+vcpu_info_reset(v);
 
 put_page_and_type(mfn_to_page(mfn));
 }
diff --git a/xen/common/domctl.c b/xen/common/domctl.c
index 7f959f3..41891b1 100644
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -704,6 +704,15 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) 
u_domctl)
 break;
 }
 
+case XEN_DOMCTL_soft_reset:
+if ( d == current->domain )
+{
+ret = -EINVAL;
+break;
+}
+ret = domain_soft_reset(d);
+break;
+
 case XEN_DOMCTL_destroydomain:
 ret = domain_kill(d);
 if ( ret == -ERESTART )
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 675f021..794d4d5 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1139,6 +1139,7 @@ struct xen_domctl {
 #define XEN_DOMCTL_psr_cmt_op75
 #define XEN_DOMCTL_monitor_op77
 #define XEN_DOMCTL_psr_cat_op78
+#define XEN_DOMCTL_soft_reset79
 #define XEN_DOMCTL_gdbsx_guestmemio1000
 #define XEN_DOMCTL_gdbsx_pausevcpu 1001
 #define XEN_DOMCTL_gdbsx_unpausevcpu   1002
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 73d3bc8..8053b5a 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -610,6 +610,8 @@ void domain_shutdown(struct domain *d, u8 reason);
 void domain_resume(struct domain *d);
 void domain_pause_for_debugger(void);
 
+int domain_soft_reset(struct domain *d);
+
 int vcpu_start_shutdown_deferral(struct vcpu *v);
 void vcpu_end_shutdown_deferral(struct vcpu *v);
 
-- 
2.4.3


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v11 09/11] libxc: support XEN_DOMCTL_soft_reset operation

2015-09-04 Thread Vitaly Kuznetsov
Introduce xc_domain_soft_reset() function supporting XEN_DOMCTL_soft_reset.

Signed-off-by: Vitaly Kuznetsov 
Acked-by: Wei Liu 
Reviewed-by: Konrad Rzeszutek Wilk 
Acked-by: Ian Jackson 
---
Changes since v10:
- None.
---
 tools/libxc/include/xenctrl.h | 3 +++
 tools/libxc/xc_domain.c   | 9 +
 2 files changed, 12 insertions(+)

diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index de3c0ad..cbe98de 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -1288,6 +1288,9 @@ int xc_domain_setvnuma(xc_interface *xch,
 unsigned int *vcpu_to_vnode,
 unsigned int *vnode_to_pnode);
 
+int xc_domain_soft_reset(xc_interface *xch,
+ uint32_t domid);
+
 #if defined(__i386__) || defined(__x86_64__)
 /*
  * PC BIOS standard E820 types and structure.
diff --git a/tools/libxc/xc_domain.c b/tools/libxc/xc_domain.c
index 780797f..62b2e45 100644
--- a/tools/libxc/xc_domain.c
+++ b/tools/libxc/xc_domain.c
@@ -2493,6 +2493,15 @@ int xc_domain_setvnuma(xc_interface *xch,
 return rc;
 }
 
+
+int xc_domain_soft_reset(xc_interface *xch,
+ uint32_t domid)
+{
+DECLARE_DOMCTL;
+domctl.cmd = XEN_DOMCTL_soft_reset;
+domctl.domain = (domid_t)domid;
+return do_domctl(xch, &domctl);
+}
 /*
  * Local variables:
  * mode: C
-- 
2.4.3


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v11 05/11] xen: grant_table: implement grant_table_warn_active_grants()

2015-09-04 Thread Vitaly Kuznetsov
Log first 10 active grants for a domain. This function is going to be used
for soft reset, active grants on this path usually mean misbehaving backends
refusing to release their mappings on shutdown. We need that in addition to
the already existent 'g' keyhandler as such misbehaving backends can cause a
domain to crash right after the soft reset operation and 'g' option won't be
available in this case.

Signed-off-by: Vitaly Kuznetsov 
Reviewed-by: Konrad Rzeszutek Wilk 
Acked-by: Jan Beulich 
---
Changes since v10:
- None.
---
 xen/common/grant_table.c  | 35 +++
 xen/include/xen/grant_table.h |  5 +
 2 files changed, 40 insertions(+)

diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 2b449d5..23185e7 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -3352,6 +3352,41 @@ gnttab_release_mappings(
 }
 }
 
+void grant_table_warn_active_grants(struct domain *d)
+{
+struct grant_table *gt = d->grant_table;
+struct active_grant_entry *act;
+grant_ref_t ref;
+unsigned int nr_active = 0;
+
+#define WARN_GRANT_MAX 10
+
+read_lock(>->lock);
+
+for ( ref = 0; ref != nr_grant_entries(gt); ref++ )
+{
+act = active_entry_acquire(gt, ref);
+if ( !act->pin )
+{
+active_entry_release(act);
+continue;
+}
+
+nr_active++;
+if ( nr_active <= WARN_GRANT_MAX )
+printk(XENLOG_G_DEBUG "Dom%d has an active grant: GFN: %lx (MFN: 
%lx)\n",
+   d->domain_id, act->gfn, act->frame);
+active_entry_release(act);
+}
+
+if ( nr_active > WARN_GRANT_MAX )
+printk(XENLOG_G_DEBUG "Dom%d has too many (%d) active grants to 
report\n",
+   d->domain_id, nr_active);
+
+read_unlock(>->lock);
+
+#undef WARN_GRANT_MAX
+}
 
 void
 grant_table_destroy(
diff --git a/xen/include/xen/grant_table.h b/xen/include/xen/grant_table.h
index 5263fd6..1c29cee 100644
--- a/xen/include/xen/grant_table.h
+++ b/xen/include/xen/grant_table.h
@@ -89,6 +89,11 @@ void grant_table_destroy(
 struct domain *d);
 void grant_table_init_vcpu(struct vcpu *v);
 
+/*
+ * Check if domain has active grants and log first 10 of them.
+ */
+void grant_table_warn_active_grants(struct domain *d);
+
 /* Domain death release of granted mappings of other domains' memory. */
 void
 gnttab_release_mappings(
-- 
2.4.3


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v11 01/11] xen: introduce SHUTDOWN_soft_reset shutdown reason

2015-09-04 Thread Vitaly Kuznetsov
This special type of shutdown is supposed to be used by PVHVM guests when
they want to perform some sort of kexec/kdump.

Signed-off-by: Vitaly Kuznetsov 
Acked-by: Jan Beulich 
Reviewed-by: Konrad Rzeszutek Wilk 
---
Changes since v10:
- None
---
 xen/common/shutdown.c  |  6 ++
 xen/include/public/sched.h | 11 ++-
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/xen/common/shutdown.c b/xen/common/shutdown.c
index 9cfbf7a..03a8641 100644
--- a/xen/common/shutdown.c
+++ b/xen/common/shutdown.c
@@ -66,6 +66,12 @@ void hwdom_shutdown(u8 reason)
 machine_restart(0);
 break; /* not reached */
 
+case SHUTDOWN_soft_reset:
+printk("Hardware domain %d did unsupported soft reset, rebooting.\n",
+   hardware_domain->domain_id);
+machine_restart(0);
+break; /* not reached */
+
 default:
 printk("Hardware Dom%u shutdown (unknown reason %u): ",
hardware_domain->domain_id, reason);
diff --git a/xen/include/public/sched.h b/xen/include/public/sched.h
index 4000ac9..2219696 100644
--- a/xen/include/public/sched.h
+++ b/xen/include/public/sched.h
@@ -159,7 +159,16 @@ DEFINE_XEN_GUEST_HANDLE(sched_watchdog_t);
 #define SHUTDOWN_suspend2  /* Clean up, save suspend info, kill. */
 #define SHUTDOWN_crash  3  /* Tell controller we've crashed. */
 #define SHUTDOWN_watchdog   4  /* Restart because watchdog time expired. */
-#define SHUTDOWN_MAX4  /* Maximum valid shutdown reason. */
+
+/*
+ * Domain asked to perform 'soft reset' for it. The expected behavior is to
+ * reset internal Xen state for the domain returning it to the point where it
+ * was created but leaving the domain's memory contents and vCPU contexts
+ * intact. This will allow the domain to start over and set up all Xen specific
+ * interfaces again.
+ */
+#define SHUTDOWN_soft_reset 5
+#define SHUTDOWN_MAX5  /* Maximum valid shutdown reason. */
 /* ` } */
 
 #endif /* __XEN_PUBLIC_SCHED_H__ */
-- 
2.4.3


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


  1   2   3   >