Re: [Xen-devel] [PATCH v9 06/13] efi: create new early memory allocator

2016-10-05 Thread Daniel Kiper
On Fri, Sep 30, 2016 at 03:46:54AM -0600, Jan Beulich wrote:
> >>> On 29.09.16 at 23:42,  wrote:
> > +#else
> > +static void __init free_ebmalloc_unused_mem(void)
> > +{
> > +}
> > +#endif
>
> Did you build test this for ARM? The function ought to be unused,
> as ...
>
> > @@ -1251,6 +1301,8 @@ void __init efi_init_memory(void)
> >  } *extra, *extra_head = NULL;
> >  #endif
> >
> > +free_ebmalloc_unused_mem();
>
> ... the whole function here doesn't get built on ARM.
>
> Julien - we're still awaiting your input on general aspects here.

Julien, ping? I would like to go forward with this patchset. Could
you tell us is current solution OK for you (of course except this
stupid build issue spotted by Jan)? Should anything change?

Daniel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] [distros-debian-squeeze test] 67795: tolerable FAIL

2016-10-05 Thread Platform Team regression test user
flight 67795 distros-debian-squeeze real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/67795/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-amd64-squeeze-netboot-pygrub 9 debian-di-install fail like 
6
 test-amd64-amd64-amd64-squeeze-netboot-pygrub 9 debian-di-install fail like 
6
 test-amd64-i386-i386-squeeze-netboot-pygrub 9 debian-di-install fail like 6
 test-amd64-amd64-i386-squeeze-netboot-pygrub 9 debian-di-install fail like 
6

baseline version:
 flight   6

jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-amd64-squeeze-netboot-pygrubfail
 test-amd64-i386-amd64-squeeze-netboot-pygrub fail
 test-amd64-amd64-i386-squeeze-netboot-pygrub fail
 test-amd64-i386-i386-squeeze-netboot-pygrub  fail



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] [PATCH] [Exynos] Fix to be error handled when 10ms delayed for cpu_on

2016-10-05 Thread JEUNGWOO, YOO
From: casionwoo 

Comment of origin code said "wait max 10 ms until cpu is on"
Origin code expects to print "CPU%d power enable failed", if cpu do not on 
until 10ms
But actual code do not reach to print even it wait 10 ms (actually it waits 
11ms not 10ms)
Because the comparing is like bellow
"if ( timeout-- == 0 )"
So I modified the code to wait 10ms and print the error statement
Let me simulate about origin code and modified code.

Origin code)

timeoutdelayed time   timeout
(before while)  (mdelay(1)) (timeout--)
  101   9
  9 2   
8
  8 3   
7
  7 4   
6
  6 5   
5
  5 6   
4
  4 7   
3
  3 8   
2
  2 9   
1
  1 10  0
  0 11  
-1

Modified code)

timeoutdelayed time   timeout
(before while)  (mdelay(1)) (--timeout)
  101   9
  9 2   
8
  8 3   
7
  7 4   
6
  6 5   
5
  5 6   
4
  4 7   
3
  3 8   
2
  2 9   
1
  1 10  0

Signed-off-by: JEUNGWOO, YOO 
---
 xen/arch/arm/platforms/exynos5.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/xen/arch/arm/platforms/exynos5.c b/xen/arch/arm/platforms/exynos5.c
index c43934f..9edd866 100644
--- a/xen/arch/arm/platforms/exynos5.c
+++ b/xen/arch/arm/platforms/exynos5.c
@@ -175,10 +175,10 @@ static int exynos5_cpu_power_up(void __iomem *power, int 
cpu)
 /* wait max 10 ms until cpu is on */
 while ( exynos_cpu_power_state(power, cpu) != S5P_CORE_LOCAL_PWR_EN )
 {
-if ( timeout-- == 0 )
-break;
-
 mdelay(1);
+
+if ( --timeout == 0 )
+break;
 }
 
 if ( timeout == 0 )
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] [xen-4.6-testing test] 101257: tolerable FAIL - PUSHED

2016-10-05 Thread osstest service owner
flight 101257 xen-4.6-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/101257/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds 11 guest-start  fail REGR. vs. 101189
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 101189
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stopfail like 101189
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail like 101189
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 101189

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-rumprun-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-rumprun-i386  1 build-check(1)   blocked  n/a
 build-i386-rumprun5 rumprun-buildfail   never pass
 build-amd64-rumprun   5 rumprun-buildfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass

version targeted for testing:
 xen  4b41252d00f1caf25a9811580a9b344db3c03708
baseline version:
 xen  ef005cc1f86de8db0880c7b1e233ef9d2b44b4ef

Last test of basis   101189  2016-09-28 15:10:42 Z6 days
Testing same since   101257  2016-10-04 13:48:30 Z0 days1 attempts


People who touched revisions under test:
  Jan Beulich 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64-xtf  pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-prev pass
 build-i386-prev  pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops  

Re: [Xen-devel] [PATCH] xen/x86: Initialize per_cpu(xen_vcpu, 0) a little earlier

2016-10-05 Thread David Vrabel
On 02/10/16 23:45, Boris Ostrovsky wrote:
> xen_cpuhp_setup() calls mutex_lock() which, when CONFIG_DEBUG_MUTEXES
> is defined, ends up calling xen_save_fl(). That routine expects
> per_cpu(xen_vcpu, 0) to be already initialized.

Applied to for-linus-4.9, thanks.

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] [xen-unstable-coverity test] 101279: all pass - PUSHED

2016-10-05 Thread osstest service owner
flight 101279 xen-unstable-coverity real [real]
http://logs.test-lab.xenproject.org/osstest/logs/101279/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xen  b7dd797c7fe4cd849018f78f6c7b9eb3d33b89d8
baseline version:
 xen  b3d54cead6459567d9786ad415149868ee7f2f5b

Last test of basis   101243  2016-10-02 09:19:38 Z3 days
Testing same since   101279  2016-10-05 09:19:19 Z0 days1 attempts


People who touched revisions under test:
  Anthony PERARD 
  casionwoo 
  Ian Jackson 
  Jan Beulich 
  JEUNGWOO, YOO 
  Wei Liu 

jobs:
 coverity-amd64   pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=xen-unstable-coverity
+ revision=b7dd797c7fe4cd849018f78f6c7b9eb3d33b89d8
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push 
xen-unstable-coverity b7dd797c7fe4cd849018f78f6c7b9eb3d33b89d8
+ branch=xen-unstable-coverity
+ revision=b7dd797c7fe4cd849018f78f6c7b9eb3d33b89d8
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable-coverity
+ qemuubranch=qemu-upstream-unstable-coverity
+ qemuubranch=qemu-upstream-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' xqemu-upstream-unstable = x ']'
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable-coverity
+ prevxenbranch=xen-4.7-testing
+ '[' xb7dd797c7fe4cd849018f78f6c7b9eb3d33b89d8 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/linux-firmware.git
++ : osst...@xenbits.xen.org:/home/osstest/ext/linux-firmware.git
++ : git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
++ : osst...@xenbits.xen.org:/home/xen/git/linux-pvops.git
++ : git://xenbits.xen.org/linux-pvops.git
++ : tested/linux-3.14
++ : tested/linux-arm-xen
++ '[' xgit://xenbits.xen.org/linux-pvops.git = x ']'
++ '[' x = x ']'
++ : git://xenbits.xen.org/linux-pvops.git
++ : tested/linux-arm-xen
++ : git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git
++ : tested/2.6.39.x
++ : daily-cron.xen-unstable-coverity
++ : daily-cron.xen-unstable-coverity
++ : daily-cro

Re: [Xen-devel] [PATCH RFC] x86/Intel: virtualize support for cpuid faulting

2016-10-05 Thread Jan Beulich
>>> On 05.10.16 at 06:49,  wrote:
> Ok.  Adding an hvm_get_cpl seems pretty straightforward.  I'll do that.

I've got such a patch already, but it'll be usefully posted only when we
get closer to re-opening the tree for 4.9. It definitely wants to do more
than just adding the new wrapper.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] Inconsistent use of set_context_data()?

2016-10-05 Thread Jan Beulich
Hello,

what's the point of this being used by hvmemul_read() and
hvmemul_cmpxchg(), but (namely but not limited to) not by
hvmemul_write()?

Thanks, Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] Inconsistent use of set_context_data()?

2016-10-05 Thread Razvan Cojocaru
Hello,

> what's the point of this being used by hvmemul_read() and
> hvmemul_cmpxchg(), but (namely but not limited to) not by
> hvmemul_write()?

To do introspection work, we sometimes need to modify the guest memory,
and there are cases, namely during hibernate / resume of Windows guests,
when we need to serve the "old" version of that memory to the current
instruction reading from it for the process to work reliably.

The design choice here has been that the introspection application is
smart enough to handle writes (after all, it is the one managing the
buffer sent via vm_event reply), so it is intended behaviour.


Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] Inconsistent use of set_context_data()?

2016-10-05 Thread Jan Beulich
>>> On 05.10.16 at 14:22,  wrote:
>> what's the point of this being used by hvmemul_read() and
>> hvmemul_cmpxchg(), but (namely but not limited to) not by
>> hvmemul_write()?
> 
> To do introspection work, we sometimes need to modify the guest memory,
> and there are cases, namely during hibernate / resume of Windows guests,
> when we need to serve the "old" version of that memory to the current
> instruction reading from it for the process to work reliably.
> 
> The design choice here has been that the introspection application is
> smart enough to handle writes (after all, it is the one managing the
> buffer sent via vm_event reply), so it is intended behaviour.

Well - the confusing thing is that for cmpxchg it's the value to be
written which gets altered, not the value to be compared against,
i.e. it acts as if set_context_data() was also intended to be
present in hvmemul_write().

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] Inconsistent use of set_context_data()?

2016-10-05 Thread Jan Beulich
>>> On 05.10.16 at 14:29,  wrote:
 On 05.10.16 at 14:22,  wrote:
>>> what's the point of this being used by hvmemul_read() and
>>> hvmemul_cmpxchg(), but (namely but not limited to) not by
>>> hvmemul_write()?
>> 
>> To do introspection work, we sometimes need to modify the guest memory,
>> and there are cases, namely during hibernate / resume of Windows guests,
>> when we need to serve the "old" version of that memory to the current
>> instruction reading from it for the process to work reliably.
>> 
>> The design choice here has been that the introspection application is
>> smart enough to handle writes (after all, it is the one managing the
>> buffer sent via vm_event reply), so it is intended behaviour.
> 
> Well - the confusing thing is that for cmpxchg it's the value to be
> written which gets altered, not the value to be compared against,
> i.e. it acts as if set_context_data() was also intended to be
> present in hvmemul_write().

And note how this is the only case where caller supplied data gets
modified - after all at least the p_new pointer should really be const
(for p_old one might argue that upon failure the old value could be
passed back to the caller). And if that altering of caller data was
intended, then there's at least one case where it doesn't take
effect right now: protmode_load_seg()'s updating of the accessed
bit or-s in a_flag into the local copy instead of using new_desc_b.

IOW - I don't really understand what (consistent) behavior is
expected here.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] Inconsistent use of set_context_data()?

2016-10-05 Thread Andrew Cooper
On 05/10/16 13:29, Jan Beulich wrote:
 On 05.10.16 at 14:22,  wrote:
>>> what's the point of this being used by hvmemul_read() and
>>> hvmemul_cmpxchg(), but (namely but not limited to) not by
>>> hvmemul_write()?
>> To do introspection work, we sometimes need to modify the guest memory,
>> and there are cases, namely during hibernate / resume of Windows guests,
>> when we need to serve the "old" version of that memory to the current
>> instruction reading from it for the process to work reliably.
>>
>> The design choice here has been that the introspection application is
>> smart enough to handle writes (after all, it is the one managing the
>> buffer sent via vm_event reply), so it is intended behaviour.
> Well - the confusing thing is that for cmpxchg it's the value to be
> written which gets altered, not the value to be compared against,
> i.e. it acts as if set_context_data() was also intended to be
> present in hvmemul_write().

Can I highly suggest that writing an Introspection feature doc,
explaining some bits and pieces like this might be a very good idea?

Also, commiseration to whichever poor sole had to debug that issue...

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] Inconsistent use of set_context_data()?

2016-10-05 Thread Razvan Cojocaru
On 10/05/2016 03:29 PM, Jan Beulich wrote:
 On 05.10.16 at 14:22,  wrote:
>>> what's the point of this being used by hvmemul_read() and
>>> hvmemul_cmpxchg(), but (namely but not limited to) not by
>>> hvmemul_write()?
>>
>> To do introspection work, we sometimes need to modify the guest memory,
>> and there are cases, namely during hibernate / resume of Windows guests,
>> when we need to serve the "old" version of that memory to the current
>> instruction reading from it for the process to work reliably.
>>
>> The design choice here has been that the introspection application is
>> smart enough to handle writes (after all, it is the one managing the
>> buffer sent via vm_event reply), so it is intended behaviour.
> 
> Well - the confusing thing is that for cmpxchg it's the value to be
> written which gets altered, not the value to be compared against,
> i.e. it acts as if set_context_data() was also intended to be
> present in hvmemul_write().

That's an accident, in hvmemul_cmpxchg() p_old is not being used at all
so IIRC I've simply latched onto p_new. All we're interested in are
reads from the supplied buffer overriding the guest's actual memory.


Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] fix EFI part of "symbols: Generate an xen-sym.map"

2016-10-05 Thread Jan Beulich
>>> On 21.09.16 at 18:17,  wrote:
> On Wed, Sep 21, 2016 at 10:04:21AM -0600, Jan Beulich wrote:
>> >>> On 21.09.16 at 17:59,  wrote:
>> > The fix can be done two ways:
>> >  a) See if xen.efi.map exists and then copy it
>> >  b) Or link xen.efi.map to xen-syms.map (similar to how xen.efi is linked
>> > against xen).
>> > 
>> > The patch chooses the latter.
>> 
>> Well, if the ARM maintainers like that ... I don't really see a point in
>> installing the same file twice without its second incarnation having a
>> specific purpose.
> 
> I also have the a) part ready, which is simple:

Ping? I'd really like to see this bogus error gone from the build.

Jan

> diff --git a/xen/Makefile b/xen/Makefile
> index e989a20..678f188 100644
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -67,7 +67,9 @@ _install: $(TARGET)$(CONFIG_XEN_INSTALL_SUFFIX)
>   if [ -r $(TARGET).efi -a -n '$(EFI_DIR)' ]; then \
>   [ -d $(D)$(EFI_DIR) ] || $(INSTALL_DIR) $(D)$(EFI_DIR); \
>   $(INSTALL_DATA) $(TARGET).efi 
> $(D)$(EFI_DIR)/$(T)-$(XEN_FULLVERSION).efi; \
> - $(INSTALL_DATA) $(TARGET).efi.map 
> $(D)$(DEBUG_DIR)/$(T)-$(XEN_FULLVERSION).efi.map; \
> +if [ -e $(TARGET).efi.map ]; then \
> + $(INSTALL_DATA) $(TARGET).efi.map 
> $(D)$(DEBUG_DIR)/$(T)-$(XEN_FULLVERSION).efi.map; \
> +fi; \
>   ln -sf $(T)-$(XEN_FULLVERSION).efi 
> $(D)$(EFI_DIR)/$(T)-$(XEN_VERSION).$(XEN_SUBVERSION).efi; \
>   ln -sf $(T)-$(XEN_FULLVERSION).efi 
> $(D)$(EFI_DIR)/$(T)-$(XEN_VERSION).efi; \
>   ln -sf $(T)-$(XEN_FULLVERSION).efi $(D)$(EFI_DIR)/$(T).efi; \
> 
> Either fix works.




___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] Inconsistent use of set_context_data()?

2016-10-05 Thread Jan Beulich
>>> On 05.10.16 at 14:44,  wrote:
> On 10/05/2016 03:29 PM, Jan Beulich wrote:
> On 05.10.16 at 14:22,  wrote:
 what's the point of this being used by hvmemul_read() and
 hvmemul_cmpxchg(), but (namely but not limited to) not by
 hvmemul_write()?
>>>
>>> To do introspection work, we sometimes need to modify the guest memory,
>>> and there are cases, namely during hibernate / resume of Windows guests,
>>> when we need to serve the "old" version of that memory to the current
>>> instruction reading from it for the process to work reliably.
>>>
>>> The design choice here has been that the introspection application is
>>> smart enough to handle writes (after all, it is the one managing the
>>> buffer sent via vm_event reply), so it is intended behaviour.
>> 
>> Well - the confusing thing is that for cmpxchg it's the value to be
>> written which gets altered, not the value to be compared against,
>> i.e. it acts as if set_context_data() was also intended to be
>> present in hvmemul_write().
> 
> That's an accident, in hvmemul_cmpxchg() p_old is not being used at all
> so IIRC I've simply latched onto p_new. All we're interested in are
> reads from the supplied buffer overriding the guest's actual memory.

So am I to understand you'll submit a patch replacing p_old with
p_new there (which would have the same effect as simply deleting
that code, except for the doc aspect)?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] Inconsistent use of set_context_data()?

2016-10-05 Thread Razvan Cojocaru
On 10/05/2016 03:47 PM, Jan Beulich wrote:
 On 05.10.16 at 14:44,  wrote:
>> On 10/05/2016 03:29 PM, Jan Beulich wrote:
>> On 05.10.16 at 14:22,  wrote:
> what's the point of this being used by hvmemul_read() and
> hvmemul_cmpxchg(), but (namely but not limited to) not by
> hvmemul_write()?

 To do introspection work, we sometimes need to modify the guest memory,
 and there are cases, namely during hibernate / resume of Windows guests,
 when we need to serve the "old" version of that memory to the current
 instruction reading from it for the process to work reliably.

 The design choice here has been that the introspection application is
 smart enough to handle writes (after all, it is the one managing the
 buffer sent via vm_event reply), so it is intended behaviour.
>>>
>>> Well - the confusing thing is that for cmpxchg it's the value to be
>>> written which gets altered, not the value to be compared against,
>>> i.e. it acts as if set_context_data() was also intended to be
>>> present in hvmemul_write().
>>
>> That's an accident, in hvmemul_cmpxchg() p_old is not being used at all
>> so IIRC I've simply latched onto p_new. All we're interested in are
>> reads from the supplied buffer overriding the guest's actual memory.
> 
> So am I to understand you'll submit a patch replacing p_old with
> p_new there (which would have the same effect as simply deleting
> that code, except for the doc aspect)?

Yes, I'll get on that. I'll also const-ify the p_new pointer unless
there are objections.


Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [Xen-users] xl info displays less ram than the total available

2016-10-05 Thread George Dunlap
On 05/10/16 13:52, bla...@riseup.net wrote:
> George Dunlap:
>> On Wed, Oct 5, 2016 at 11:16 AM,   wrote:
>>>
>>> Hi all,
>>> I have been wondering if I could manage to make Xen function on my
>>> laptop, using as dom0 my Arch Linux installation. As far as I understood
>>> there is no currently supported package in the AUR nor, obviously, in
>>> the official repos. What to do? Compile it by myself! I went under all
>>> the process, in particular
>>>
>>> % PYTHON="/usr/bin/python2" ./configure --enable-stubdom \
>>>--enable-systemd --prefix=/usr
>>>
>>> % PYTHON="/usr/bin/python2" NO_WERROR=1 make dist
>>>
>>> % sudo PYTHON="/usr/bin/python2" NO_WERROR=1 make install
>>>
>>> After that I manually added a grub entry to boot into my dom0
>>> chainbooted by Xen.
>>>
>>> And it seemed I succeeded, as far as I manage to boot into my system.
>>> Everything seem to work, except that if I look at the features of my
>>> system
>>>
>>> % sudo xl info
>>>
>>> I get a discouraging indication on the total amount of ram I have on my
>>> system
>>>
>>> total_memory   : 1948
>>> free_memory: 120
>>>
>>> while the total memory is (as measured from Arch booted without Xen)
>>>
>>> % free -m
>>>total   used free shared  buff/cache  available
>>>Mem:15924   8756 2206 737 49616100
>>>Swap:   819108191
>>>
>>> What to do? I am very new to Xen, how can I debug it?
>>
>> Thanks for your report.  Can you please attach the full output of "xl
>> info", as well as the output of "xl dmesg"?
>>
>> Thanks,
>> -George
> 
> I attach the output of xl info, xl dmesg in dom0 *and* dmesg in a normal
> Arch Linux boot (without Xen).

That's quite strange -- the e820 maps reported by Linux and Xen appear
to be almost completely different:

(XEN) Xen-e820 RAM map:
(XEN)   - 0009ec00 (usable)
(XEN)  0009ec00 - 000a (reserved)
(XEN)  000e - 0010 (reserved)
(XEN)  0010 - 79cbe000 (usable)
(XEN)  79cbe000 - bcd2f000 (reserved)
(XEN)  bcd2f000 - bce7f000 (ACPI NVS)
(XEN)  bce7f000 - bceff000 (ACPI data)
(XEN)  bceff000 - bfa0 (reserved)
(XEN)  f800 - fc00 (reserved)
(XEN)  fec0 - fec01000 (reserved)
(XEN)  fed08000 - fed09000 (reserved)
(XEN)  fed1 - fed1a000 (reserved)
(XEN)  fed1c000 - fed2 (reserved)
(XEN)  fee0 - fee01000 (reserved)
(XEN)  ffc0 - 00043e60 (reserved)

BIOS-e820: [mem 0x-0x00057fff] usable
BIOS-e820: [mem 0x00058000-0x00058fff] reserved
BIOS-e820: [mem 0x00059000-0x0008bfff] usable
BIOS-e820: [mem 0x0008c000-0x000b] reserved
BIOS-e820: [mem 0x000e-0x000f] reserved
BIOS-e820: [mem 0x0010-0x7b82dfff] usable
BIOS-e820: [mem 0x7b82e000-0x7ba2] reserved
BIOS-e820: [mem 0x7ba3-0xba1fcfff] usable
BIOS-e820: [mem 0xba1fd000-0xba3fcfff] type 20
BIOS-e820: [mem 0xba3fd000-0xbcd2efff] reserved
BIOS-e820: [mem 0xbcd2f000-0xbce7efff] ACPI NVS
BIOS-e820: [mem 0xbce7f000-0xbcefefff] ACPI data
BIOS-e820: [mem 0xbceff000-0xbcef] usable
BIOS-e820: [mem 0xbcf0-0xbf9f] reserved
BIOS-e820: [mem 0xf80f8000-0xf80f8fff] reserved
BIOS-e820: [mem 0xfed1c000-0xfed1] reserved
BIOS-e820: [mem 0x0001-0x00043e5f] usable

With the key thing being that Xen's version has the range from ~2G-3G
and 4G-17G as 'reserved', while Linux's version has those ranges as
'usable'.

Jan / Andy, any idea what's going on here?

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] Inconsistent use of set_context_data()?

2016-10-05 Thread Razvan Cojocaru
On 10/05/2016 04:02 PM, Jan Beulich wrote:
 On 05.10.16 at 14:53,  wrote:
>> On 10/05/2016 03:47 PM, Jan Beulich wrote:
>> On 05.10.16 at 14:44,  wrote:
 On 10/05/2016 03:29 PM, Jan Beulich wrote:
 On 05.10.16 at 14:22,  wrote:
>>> what's the point of this being used by hvmemul_read() and
>>> hvmemul_cmpxchg(), but (namely but not limited to) not by
>>> hvmemul_write()?
>>
>> To do introspection work, we sometimes need to modify the guest memory,
>> and there are cases, namely during hibernate / resume of Windows guests,
>> when we need to serve the "old" version of that memory to the current
>> instruction reading from it for the process to work reliably.
>>
>> The design choice here has been that the introspection application is
>> smart enough to handle writes (after all, it is the one managing the
>> buffer sent via vm_event reply), so it is intended behaviour.
>
> Well - the confusing thing is that for cmpxchg it's the value to be
> written which gets altered, not the value to be compared against,
> i.e. it acts as if set_context_data() was also intended to be
> present in hvmemul_write().

 That's an accident, in hvmemul_cmpxchg() p_old is not being used at all
 so IIRC I've simply latched onto p_new. All we're interested in are
 reads from the supplied buffer overriding the guest's actual memory.
>>>
>>> So am I to understand you'll submit a patch replacing p_old with
>>> p_new there (which would have the same effect as simply deleting
>>> that code, except for the doc aspect)?
>>
>> Yes, I'll get on that. I'll also const-ify the p_new pointer unless
>> there are objections.
> 
> You won't be able to - that's where I started actually.

You're right - just found out the hard way. I'll just submit the smaller
patch then in a moment.


Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] Inconsistent use of set_context_data()?

2016-10-05 Thread Jan Beulich
>>> On 05.10.16 at 14:53,  wrote:
> On 10/05/2016 03:47 PM, Jan Beulich wrote:
> On 05.10.16 at 14:44,  wrote:
>>> On 10/05/2016 03:29 PM, Jan Beulich wrote:
>>> On 05.10.16 at 14:22,  wrote:
>> what's the point of this being used by hvmemul_read() and
>> hvmemul_cmpxchg(), but (namely but not limited to) not by
>> hvmemul_write()?
>
> To do introspection work, we sometimes need to modify the guest memory,
> and there are cases, namely during hibernate / resume of Windows guests,
> when we need to serve the "old" version of that memory to the current
> instruction reading from it for the process to work reliably.
>
> The design choice here has been that the introspection application is
> smart enough to handle writes (after all, it is the one managing the
> buffer sent via vm_event reply), so it is intended behaviour.

 Well - the confusing thing is that for cmpxchg it's the value to be
 written which gets altered, not the value to be compared against,
 i.e. it acts as if set_context_data() was also intended to be
 present in hvmemul_write().
>>>
>>> That's an accident, in hvmemul_cmpxchg() p_old is not being used at all
>>> so IIRC I've simply latched onto p_new. All we're interested in are
>>> reads from the supplied buffer overriding the guest's actual memory.
>> 
>> So am I to understand you'll submit a patch replacing p_old with
>> p_new there (which would have the same effect as simply deleting
>> that code, except for the doc aspect)?
> 
> Yes, I'll get on that. I'll also const-ify the p_new pointer unless
> there are objections.

You won't be able to - that's where I started actually.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [Xen-users] xl info displays less ram than the total available

2016-10-05 Thread Andrew Cooper
On 05/10/16 14:02, George Dunlap wrote:
> On 05/10/16 13:52, bla...@riseup.net wrote:
>> George Dunlap:
>>> On Wed, Oct 5, 2016 at 11:16 AM,   wrote:
 Hi all,
 I have been wondering if I could manage to make Xen function on my
 laptop, using as dom0 my Arch Linux installation. As far as I understood
 there is no currently supported package in the AUR nor, obviously, in
 the official repos. What to do? Compile it by myself! I went under all
 the process, in particular

 % PYTHON="/usr/bin/python2" ./configure --enable-stubdom \
--enable-systemd --prefix=/usr

 % PYTHON="/usr/bin/python2" NO_WERROR=1 make dist

 % sudo PYTHON="/usr/bin/python2" NO_WERROR=1 make install

 After that I manually added a grub entry to boot into my dom0
 chainbooted by Xen.

 And it seemed I succeeded, as far as I manage to boot into my system.
 Everything seem to work, except that if I look at the features of my
 system

 % sudo xl info

 I get a discouraging indication on the total amount of ram I have on my
 system

 total_memory   : 1948
 free_memory: 120

 while the total memory is (as measured from Arch booted without Xen)

 % free -m
total   used free shared  buff/cache  available
Mem:15924   8756 2206 737 49616100
Swap:   819108191

 What to do? I am very new to Xen, how can I debug it?
>>> Thanks for your report.  Can you please attach the full output of "xl
>>> info", as well as the output of "xl dmesg"?
>>>
>>> Thanks,
>>> -George
>> I attach the output of xl info, xl dmesg in dom0 *and* dmesg in a normal
>> Arch Linux boot (without Xen).
> That's quite strange -- the e820 maps reported by Linux and Xen appear
> to be almost completely different:
>
> (XEN) Xen-e820 RAM map:
> (XEN)   - 0009ec00 (usable)
> (XEN)  0009ec00 - 000a (reserved)
> (XEN)  000e - 0010 (reserved)
> (XEN)  0010 - 79cbe000 (usable)
> (XEN)  79cbe000 - bcd2f000 (reserved)
> (XEN)  bcd2f000 - bce7f000 (ACPI NVS)
> (XEN)  bce7f000 - bceff000 (ACPI data)
> (XEN)  bceff000 - bfa0 (reserved)
> (XEN)  f800 - fc00 (reserved)
> (XEN)  fec0 - fec01000 (reserved)
> (XEN)  fed08000 - fed09000 (reserved)
> (XEN)  fed1 - fed1a000 (reserved)
> (XEN)  fed1c000 - fed2 (reserved)
> (XEN)  fee0 - fee01000 (reserved)
> (XEN)  ffc0 - 00043e60 (reserved)
>
> BIOS-e820: [mem 0x-0x00057fff] usable
> BIOS-e820: [mem 0x00058000-0x00058fff] reserved
> BIOS-e820: [mem 0x00059000-0x0008bfff] usable
> BIOS-e820: [mem 0x0008c000-0x000b] reserved
> BIOS-e820: [mem 0x000e-0x000f] reserved
> BIOS-e820: [mem 0x0010-0x7b82dfff] usable
> BIOS-e820: [mem 0x7b82e000-0x7ba2] reserved
> BIOS-e820: [mem 0x7ba3-0xba1fcfff] usable
> BIOS-e820: [mem 0xba1fd000-0xba3fcfff] type 20
> BIOS-e820: [mem 0xba3fd000-0xbcd2efff] reserved
> BIOS-e820: [mem 0xbcd2f000-0xbce7efff] ACPI NVS
> BIOS-e820: [mem 0xbce7f000-0xbcefefff] ACPI data
> BIOS-e820: [mem 0xbceff000-0xbcef] usable
> BIOS-e820: [mem 0xbcf0-0xbf9f] reserved
> BIOS-e820: [mem 0xf80f8000-0xf80f8fff] reserved
> BIOS-e820: [mem 0xfed1c000-0xfed1] reserved
> BIOS-e820: [mem 0x0001-0x00043e5f] usable
>
> With the key thing being that Xen's version has the range from ~2G-3G
> and 4G-17G as 'reserved', while Linux's version has those ranges as
> 'usable'.
>
> Jan / Andy, any idea what's going on here?

Not specifically.  Can you boot with e820-verbose on the Xen command
line, which will provide more information?

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] Inconsistent use of set_context_data()?

2016-10-05 Thread Razvan Cojocaru
On 10/05/2016 03:40 PM, Andrew Cooper wrote:
> On 05/10/16 13:29, Jan Beulich wrote:
> On 05.10.16 at 14:22,  wrote:
 what's the point of this being used by hvmemul_read() and
 hvmemul_cmpxchg(), but (namely but not limited to) not by
 hvmemul_write()?
>>> To do introspection work, we sometimes need to modify the guest memory,
>>> and there are cases, namely during hibernate / resume of Windows guests,
>>> when we need to serve the "old" version of that memory to the current
>>> instruction reading from it for the process to work reliably.
>>>
>>> The design choice here has been that the introspection application is
>>> smart enough to handle writes (after all, it is the one managing the
>>> buffer sent via vm_event reply), so it is intended behaviour.
>> Well - the confusing thing is that for cmpxchg it's the value to be
>> written which gets altered, not the value to be compared against,
>> i.e. it acts as if set_context_data() was also intended to be
>> present in hvmemul_write().
> 
> Can I highly suggest that writing an Introspection feature doc,
> explaining some bits and pieces like this might be a very good idea?

Fair point, I suppose what we'd need to figure out at this point is A)
where would the best place for this information to appear be (I assume
somewhere on the project Wiki), and B) what information is interesting,
or at least complex, enough to warrant higher level descriptions?

Tamas probably has additional thoughts on this.


Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] [PATCH] x86/hvm: Set the emulation context correctly in hvmemul_cmpxchg()

2016-10-05 Thread Razvan Cojocaru
hvmemul_cmpxchg() sets the read emulation context in p_new instead
of p_old, which is inconsistent (and wrong). We are now setting
p_old (even though it's unused) and adding a comment explaining
the change.

Suggested-by: Jan Beulich 
Signed-off-by: Razvan Cojocaru 
---
 xen/arch/x86/hvm/emulate.c | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index d759d3f..0cbb16e 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -1031,13 +1031,17 @@ static int hvmemul_cmpxchg(
 
 if ( unlikely(hvmemul_ctxt->set_context) )
 {
-int rc = set_context_data(p_new, bytes);
+int rc = set_context_data(p_old, bytes);
 
 if ( rc != X86EMUL_OKAY )
 return rc;
 }
 
-/* Fix this in case the guest is really relying on r-m-w atomicity. */
+/*
+ * Fix this in case the guest is really relying on r-m-w atomicity.
+ * Please note that while the set_context code is provided here for
+ * consistency, p_old is unused.
+ */
 return hvmemul_write(seg, offset, p_new, bytes, ctxt);
 }
 
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] Inconsistent use of set_context_data()?

2016-10-05 Thread Andrew Cooper
On 05/10/16 14:12, Razvan Cojocaru wrote:
> On 10/05/2016 03:40 PM, Andrew Cooper wrote:
>> On 05/10/16 13:29, Jan Beulich wrote:
>> On 05.10.16 at 14:22,  wrote:
> what's the point of this being used by hvmemul_read() and
> hvmemul_cmpxchg(), but (namely but not limited to) not by
> hvmemul_write()?
 To do introspection work, we sometimes need to modify the guest memory,
 and there are cases, namely during hibernate / resume of Windows guests,
 when we need to serve the "old" version of that memory to the current
 instruction reading from it for the process to work reliably.

 The design choice here has been that the introspection application is
 smart enough to handle writes (after all, it is the one managing the
 buffer sent via vm_event reply), so it is intended behaviour.
>>> Well - the confusing thing is that for cmpxchg it's the value to be
>>> written which gets altered, not the value to be compared against,
>>> i.e. it acts as if set_context_data() was also intended to be
>>> present in hvmemul_write().
>> Can I highly suggest that writing an Introspection feature doc,
>> explaining some bits and pieces like this might be a very good idea?
> Fair point, I suppose what we'd need to figure out at this point is A)
> where would the best place for this information to appear be (I assume
> somewhere on the project Wiki)

docs/features/  in tree please.

See the migration and levelling docs as examples, and the template for
suggested layout.

> , and B) what information is interesting,
> or at least complex, enough to warrant higher level descriptions?

In this case, I would suggested suggest that the Technical Details
section include some details as to what an introspection agent needs to
be able to do, using the interface, (and preferably why).

Something like "override the values from memory on an emulated read"
would certainly be helpful when referring to the emulation code.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] x86/hvm: Set the emulation context correctly in hvmemul_cmpxchg()

2016-10-05 Thread Jan Beulich
>>> On 05.10.16 at 15:19,  wrote:
> --- a/xen/arch/x86/hvm/emulate.c
> +++ b/xen/arch/x86/hvm/emulate.c
> @@ -1031,13 +1031,17 @@ static int hvmemul_cmpxchg(
>  
>  if ( unlikely(hvmemul_ctxt->set_context) )
>  {
> -int rc = set_context_data(p_new, bytes);
> +int rc = set_context_data(p_old, bytes);

So this addresses one half of the problem mentioned, but not the
other: You're still (unlike in all other cases where set_context_data()
is being used) modifying data owned by the caller, which may end
in it getting confused. I admit the semantics of the ->cmpxchg()
callback aren't well defined, but current behavior is clearly to leave
both buffers untouched even if at least p_new can't be constified.
And if p_old was meant to get modified in any way, it clearly could
only be to return back the old value an actual CMPXCHG may have
found in memory (which afaict could be different from whatever
override the introspection app may have enforced).

>  if ( rc != X86EMUL_OKAY )
>  return rc;
>  }
>  
> -/* Fix this in case the guest is really relying on r-m-w atomicity. */
> +/*
> + * Fix this in case the guest is really relying on r-m-w atomicity.
> + * Please note that while the set_context code is provided here for
> + * consistency, p_old is unused.
> + */
>  return hvmemul_write(seg, offset, p_new, bytes, ctxt);
>  }

So with this I wonder btw. why your patch (mostly) fixing this
shortcoming (while adding proper LOCK handling) never made it
to a version that could be committed.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] x86/hvm: Set the emulation context correctly in hvmemul_cmpxchg()

2016-10-05 Thread Razvan Cojocaru
On 10/05/2016 04:43 PM, Jan Beulich wrote:
 On 05.10.16 at 15:19,  wrote:
>> --- a/xen/arch/x86/hvm/emulate.c
>> +++ b/xen/arch/x86/hvm/emulate.c
>> @@ -1031,13 +1031,17 @@ static int hvmemul_cmpxchg(
>>  
>>  if ( unlikely(hvmemul_ctxt->set_context) )
>>  {
>> -int rc = set_context_data(p_new, bytes);
>> +int rc = set_context_data(p_old, bytes);
> 
> So this addresses one half of the problem mentioned, but not the
> other: You're still (unlike in all other cases where set_context_data()
> is being used) modifying data owned by the caller, which may end
> in it getting confused. I admit the semantics of the ->cmpxchg()
> callback aren't well defined, but current behavior is clearly to leave
> both buffers untouched even if at least p_new can't be constified.
> And if p_old was meant to get modified in any way, it clearly could
> only be to return back the old value an actual CMPXCHG may have
> found in memory (which afaict could be different from whatever
> override the introspection app may have enforced).

I understand. I'll just remove the set_context code then, and the newly
added comment along with it, and send out V2.

>>  if ( rc != X86EMUL_OKAY )
>>  return rc;
>>  }
>>  
>> -/* Fix this in case the guest is really relying on r-m-w atomicity. */
>> +/*
>> + * Fix this in case the guest is really relying on r-m-w atomicity.
>> + * Please note that while the set_context code is provided here for
>> + * consistency, p_old is unused.
>> + */
>>  return hvmemul_write(seg, offset, p_new, bytes, ctxt);
>>  }
> 
> So with this I wonder btw. why your patch (mostly) fixing this
> shortcoming (while adding proper LOCK handling) never made it
> to a version that could be committed.

I was under the impression that your stand on the rwlock patch had
remained that you prefer a stub version to it, for possible performance
reasons, hence I've not pressed the issue. If I've misunderstood I'm
happy to try to rework it for staging.

I thought that the only acceptable solution was adding an actual stub
running on the physical VCPU, and unfortunately I didn't get to work one
out, in part because I had to tackle other issues, and partly because
it's not very clear how to go about that in this case.


Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [Xen-users] xl info displays less ram than the total available

2016-10-05 Thread Jan Beulich
>>> On 05.10.16 at 15:32,  wrote:
> Here it comes xl dmesg with Xen booted with e820-verbose=true

I have to admit that the only way I can see

(XEN) Initial Xen-e820 RAM map:
(XEN)   - 0009ec00 (usable)
(XEN)  0009ec00 - 000a (reserved)
(XEN)  000e - 0010 (reserved)
(XEN)  0010 - 79cbe000 (usable)
(XEN)  79cbe000 - bcd2f000 (reserved)
(XEN)  bcd2f000 - bce7f000 (ACPI NVS)
(XEN)  bce7f000 - bceff000 (ACPI data)
(XEN)  bceff000 - bfa0 (reserved)
(XEN)  f800 - fc00 (reserved)
(XEN)  fec0 - fec01000 (reserved)
(XEN)  fed08000 - fed09000 (reserved)
(XEN)  fed1 - fed18000 (reserved)
(XEN)  fed18000 - fed19000 (reserved)
(XEN)  fed19000 - fed1a000 (reserved)
(XEN)  fed1c000 - fed2 (reserved)
(XEN)  fee0 - fee01000 (reserved)
(XEN)  ffc0 - 0001 (reserved)
(XEN)  0001 - 00043e60 (reserved)

having all those reserved regions above 2Gb is for the boot
loader to behave oddly. Since Linux and Xen use different paths
in the boot loader, one can't really draw conclusions from Linux
getting to see a better memory map.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] x86/hvm: Set the emulation context correctly in hvmemul_cmpxchg()

2016-10-05 Thread Jan Beulich
>>> On 05.10.16 at 15:54,  wrote:
> On 10/05/2016 04:43 PM, Jan Beulich wrote:
>> So with this I wonder btw. why your patch (mostly) fixing this
>> shortcoming (while adding proper LOCK handling) never made it
>> to a version that could be committed.
> 
> I was under the impression that your stand on the rwlock patch had
> remained that you prefer a stub version to it, for possible performance
> reasons, hence I've not pressed the issue. If I've misunderstood I'm
> happy to try to rework it for staging.
> 
> I thought that the only acceptable solution was adding an actual stub
> running on the physical VCPU, and unfortunately I didn't get to work one
> out, in part because I had to tackle other issues, and partly because
> it's not very clear how to go about that in this case.

Hmm, I have to admit I don't recall any stubs to be in the picture
here. What I recall is that the locked region was too large, and
covered cases which don't need a lock in the first place.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] x86/hvm: Set the emulation context correctly in hvmemul_cmpxchg()

2016-10-05 Thread Razvan Cojocaru
On 10/05/2016 05:05 PM, Jan Beulich wrote:
 On 05.10.16 at 15:54,  wrote:
>> On 10/05/2016 04:43 PM, Jan Beulich wrote:
>>> So with this I wonder btw. why your patch (mostly) fixing this
>>> shortcoming (while adding proper LOCK handling) never made it
>>> to a version that could be committed.
>>
>> I was under the impression that your stand on the rwlock patch had
>> remained that you prefer a stub version to it, for possible performance
>> reasons, hence I've not pressed the issue. If I've misunderstood I'm
>> happy to try to rework it for staging.
>>
>> I thought that the only acceptable solution was adding an actual stub
>> running on the physical VCPU, and unfortunately I didn't get to work one
>> out, in part because I had to tackle other issues, and partly because
>> it's not very clear how to go about that in this case.
> 
> Hmm, I have to admit I don't recall any stubs to be in the picture
> here. What I recall is that the locked region was too large, and
> covered cases which don't need a lock in the first place.

Andrew I think suggested a stub first:

https://lists.xenproject.org/archives/html/xen-devel/2016-04/msg02050.html

then George brought it up again:

https://lists.xenproject.org/archives/html/xen-devel/2016-04/msg03294.html

Andrew also talked about XenServer's performance testing with the
original patch:

https://lists.xenproject.org/archives/html/xen-devel/2016-04/msg03354.html

There's actually a version of it in XenServer's patch queue for 4.7:

https://github.com/xenserver/xen-4.7.pg/blob/master/master/xen-x86-emulate-syncrhonise-LOCKed-instruction-emulation.patch

If there's no impediment, I'm happy to start working on it again.


Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] [PATCH V2] x86/hvm: Remove emulation context setting from hvmemul_cmpxchg()

2016-10-05 Thread Razvan Cojocaru
hvmemul_cmpxchg() sets the read emulation context in p_new instead
of p_old, which is inconsistent (and wrong). Since p_old is
unused in any case and cmpxchg() semantics would be altered even
if it wasn't, remove the emulation context setting code.

Suggested-by: Jan Beulich 
Signed-off-by: Razvan Cojocaru 

---
Changes since V1:
 - Modified patch subject and description.
 - Removed the context setting code altogether.
---
 xen/arch/x86/hvm/emulate.c | 11 ---
 1 file changed, 11 deletions(-)

diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index d759d3f..6ed7486 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -1026,17 +1026,6 @@ static int hvmemul_cmpxchg(
 unsigned int bytes,
 struct x86_emulate_ctxt *ctxt)
 {
-struct hvm_emulate_ctxt *hvmemul_ctxt =
-container_of(ctxt, struct hvm_emulate_ctxt, ctxt);
-
-if ( unlikely(hvmemul_ctxt->set_context) )
-{
-int rc = set_context_data(p_new, bytes);
-
-if ( rc != X86EMUL_OKAY )
-return rc;
-}
-
 /* Fix this in case the guest is really relying on r-m-w atomicity. */
 return hvmemul_write(seg, offset, p_new, bytes, ctxt);
 }
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] [PATCH for-4.8] libelf: fix symtab/strtab loading for 32bit domains

2016-10-05 Thread Roger Pau Monne
Commit ed04ca introduced a bug in the symtab/strtab loading for 32bit
guests, that corrupted the section headers array due to the padding
introduced by the elf_shdr union.

The Elf section header array on 32bit should be accessible as an array of
Elf32_Shdr elements, and the union with Elf64_Shdr done in elf_shdr was
breaking this due to size differences between Elf32_Shdr and Elf64_Shdr.

Fix this by copying each section header one by one, and using the proper
size depending on the bitness of the guest kernel.

Reported-by: Brian Marcotte 
Signed-off-by: Roger Pau Monné 
---
Cc: Brian Marcotte 
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 
---
Should be backported to Xen 4.7 stable branch.
---
 xen/common/libelf/libelf-loader.c | 34 ++
 1 file changed, 30 insertions(+), 4 deletions(-)

diff --git a/xen/common/libelf/libelf-loader.c 
b/xen/common/libelf/libelf-loader.c
index 2626a40..3faff62 100644
--- a/xen/common/libelf/libelf-loader.c
+++ b/xen/common/libelf/libelf-loader.c
@@ -262,13 +262,14 @@ static void elf_load_bsdsyms(struct elf_binary *elf)
 } __attribute__((packed)) header;
 
 ELF_HANDLE_DECL(elf_ehdr) header_handle;
-unsigned long shdr_size;
+unsigned long shdr_size, ehdr_size;
 ELF_HANDLE_DECL(elf_shdr) section_handle;
-unsigned int link, rc;
+unsigned int link, rc, i;
 elf_ptrval header_base;
 elf_ptrval elf_header_base;
 elf_ptrval symtab_base;
 elf_ptrval strtab_base;
+void *shdr;
 
 if ( !elf->bsd_symtab_pstart )
 return;
@@ -394,15 +395,40 @@ do {  
  \
 header.size = strtab_base + elf_uval(elf, section_handle, sh_size) -
   elf_header_base;
 
-/* Load the headers. */
+/* Load the size plus elf header. */
+ehdr_size = sizeof(header) - sizeof(header.elf_header.section);
 rc = elf_load_image(elf, header_base, ELF_REALPTR2PTRVAL(&header),
-sizeof(header), sizeof(header));
+ehdr_size, ehdr_size);
 if ( rc != 0 )
 {
 elf_mark_broken(elf, "unable to load ELF headers into guest memory");
 return;
 }
 
+/*
+ * Load the section headers.
+ *
+ * NB: this _must_ be done one by one, and taking the bitness into account,
+ * so that the guest can treat this as an array of type Elf{32/64}_Shdr.
+ */
+shdr_size = elf_64bit(elf) ? sizeof(Elf64_Shdr) : sizeof(Elf32_Shdr);
+for ( i = 0; i < ELF_BSDSYM_SECTIONS; i++ )
+{
+if ( elf_64bit(elf) )
+shdr = &header.elf_header.section[i].e64;
+else
+shdr = &header.elf_header.section[i].e32;
+
+rc = elf_load_image(elf, header_base + ehdr_size + shdr_size * i,
+ELF_REALPTR2PTRVAL(shdr), shdr_size, shdr_size);
+if ( rc != 0 )
+{
+elf_mark_broken(elf,
+"unable to load ELF section header into guest memory");
+return;
+}
+}
+
 /* Remove permissions from elf_memcpy_safe. */
 elf->caller_xdest_base = NULL;
 elf->caller_xdest_size = 0;
-- 
2.7.4 (Apple Git-66)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] [xen-unstable-smoke test] 101282: tolerable all pass - PUSHED

2016-10-05 Thread osstest service owner
flight 101282 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/101282/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  9f5eff08a6a6f58645fb48382c843973674042c9
baseline version:
 xen  b7dd797c7fe4cd849018f78f6c7b9eb3d33b89d8

Last test of basis   101264  2016-10-04 21:01:23 Z0 days
Testing same since   101282  2016-10-05 13:03:25 Z0 days1 attempts


People who touched revisions under test:
  Jan Beulich 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=xen-unstable-smoke
+ revision=9f5eff08a6a6f58645fb48382c843973674042c9
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push xen-unstable-smoke 
9f5eff08a6a6f58645fb48382c843973674042c9
+ branch=xen-unstable-smoke
+ revision=9f5eff08a6a6f58645fb48382c843973674042c9
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable-smoke
+ qemuubranch=qemu-upstream-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' xqemu-upstream-unstable = x ']'
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable-smoke
+ prevxenbranch=xen-4.7-testing
+ '[' x9f5eff08a6a6f58645fb48382c843973674042c9 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/linux-firmware.git
++ : osst...@xenbits.xen.org:/home/osstest/ext/linux-firmware.git
++ : git://git.kernel.org/pub/scm/linux/kernel/git/firmware/

Re: [Xen-devel] [PATCH v2 net-next 2/7] xen-netback: retire guest rx side prefix GSO feature

2016-10-05 Thread Roger Pau Monné
On Tue, Oct 04, 2016 at 10:24:04AM -0400, Konrad Rzeszutek Wilk wrote:
> On Tue, Oct 04, 2016 at 01:35:41PM +, Paul Durrant wrote:
> > > -Original Message-
> > > From: Konrad Rzeszutek Wilk [mailto:konrad.w...@oracle.com]
> > > Sent: 04 October 2016 13:52
> > > To: Paul Durrant ; annie...@oracle.com;
> > > joao.m.mart...@oracle.com
> > > Cc: net...@vger.kernel.org; xen-de...@lists.xenproject.org; Wei Liu
> > > 
> > > Subject: Re: [Xen-devel] [PATCH v2 net-next 2/7] xen-netback: retire guest
> > > rx side prefix GSO feature
> > > 
> > > On Tue, Oct 04, 2016 at 10:29:13AM +0100, Paul Durrant wrote:
> > > > As far as I am aware only very old Windows network frontends make use
> > > > of this style of passing GSO packets from backend to frontend. These
> > > > frontends can easily be replaced by the freely available Xen Project
> > > > Windows PV network frontend, which uses the 'default' mechanism for
> > > > passing GSO packets, which is also used by all Linux frontends.
> > > 
> > > It is not that simple. Some companies have extra juice in their Windows
> > > frontends so can't easily swap over to the Xen Project one.
> > 
> > Ok, then those frontends will continue to work, but they won't get GSO 
> > packets any more. Prefix GSO has never been specified in the canonical 
> > netif header and so has been in a limbo state forever so such frontends 
> > have always been on borrowed time and only just happened to work against a 
> > linux backend. If someone wants to actually specify prefix GSO properly 
> > then it could be added back in, but it should not be necessary now that the 
> > RX side req<->rsp identity relation is documented 
> > (http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/include/public/io/netif.h;hb=HEAD#l729).
> > 
> > > 
> > > Either way CC-ing Annie
> > > 
> > > Also would it make sense to CC the FreeBSD and NetBSD maintainers of their
> > > PV drivers just to make sure? (Or has that been confirmed)
> > > 
> > 
> > I could do that, but I'd hope that they would be subscribed to xen-devel 
> > and will chime in if there's likely to be a problem.
> 
> Usually one CCs those folks. I think you are asking me to do
> the legwork and find them and CC them here?
> 
> CC-ing Roger and  Manuel Bouyer.

Thanks. FreeBSD is using the same method as current Linux in order to both 
send and receive GSO packets. That is using an extra slot in the ring, 
filled with a netif_extra_info of type XEN_NETIF_EXTRA_TYPE_GSO. Full code 
can be found here [0], but AFAICT FreeBSD is not using this prefix stuff.

Also, IIRC NetBSD doesn't have a Xen GSO implementation [1], but I would let 
Manuel answer that one.

Roger.

[0] http://fxr.watson.org/fxr/source/dev/xen/netfront/netfront.c
[1] https://github.com/jsonn/src/blob/trunk/sys/arch/xen/xen/if_xennet_xenbus.c

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2 net-next 2/7] xen-netback: retire guest rx side prefix GSO feature

2016-10-05 Thread Manuel Bouyer
On Wed, Oct 05, 2016 at 05:30:26PM +0200, Roger Pau Monné wrote:
> [...]
> Also, IIRC NetBSD doesn't have a Xen GSO implementation [1], but I would let 
> Manuel answer that one.

I confirm, we don't support GSO at this time.

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v9 06/13] efi: create new early memory allocator

2016-10-05 Thread Julien Grall

Hello Daniel,

On 05/10/2016 00:02, Daniel Kiper wrote:

On Fri, Sep 30, 2016 at 03:46:54AM -0600, Jan Beulich wrote:

On 29.09.16 at 23:42,  wrote:

+#else
+static void __init free_ebmalloc_unused_mem(void)
+{
+}
+#endif


Did you build test this for ARM? The function ought to be unused,
as ...


@@ -1251,6 +1301,8 @@ void __init efi_init_memory(void)
 } *extra, *extra_head = NULL;
 #endif

+free_ebmalloc_unused_mem();


... the whole function here doesn't get built on ARM.

Julien - we're still awaiting your input on general aspects here.


Julien, ping? I would like to go forward with this patchset. Could
you tell us is current solution OK for you (of course except this
stupid build issue spotted by Jan)? Should anything change?


Sorry I have been in conferences for the past 2 weeks with limited 
review bandwidth. I will try to give a look today.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for-4.8] libelf: fix symtab/strtab loading for 32bit domains

2016-10-05 Thread Jan Beulich
>>> On 05.10.16 at 17:11,  wrote:
> --- a/xen/common/libelf/libelf-loader.c
> +++ b/xen/common/libelf/libelf-loader.c
> @@ -262,13 +262,14 @@ static void elf_load_bsdsyms(struct elf_binary *elf)
>  } __attribute__((packed)) header;
>  
>  ELF_HANDLE_DECL(elf_ehdr) header_handle;
> -unsigned long shdr_size;
> +unsigned long shdr_size, ehdr_size;
>  ELF_HANDLE_DECL(elf_shdr) section_handle;
> -unsigned int link, rc;
> +unsigned int link, rc, i;
>  elf_ptrval header_base;
>  elf_ptrval elf_header_base;
>  elf_ptrval symtab_base;
>  elf_ptrval strtab_base;
> +void *shdr;

I'd appreciate if you moved this into the scope where it's needed.
Also I think it could be const.

> @@ -394,15 +395,40 @@ do {   
>  \
>  header.size = strtab_base + elf_uval(elf, section_handle, sh_size) -
>elf_header_base;
>  
> -/* Load the headers. */
> +/* Load the size plus elf header. */
> +ehdr_size = sizeof(header) - sizeof(header.elf_header.section);

I think offsetof(typeof(header), elf_header.section) would be the
safer expression here, removing the dependency on the packed
attribute.

>  rc = elf_load_image(elf, header_base, ELF_REALPTR2PTRVAL(&header),
> -sizeof(header), sizeof(header));
> +ehdr_size, ehdr_size);
>  if ( rc != 0 )
>  {
>  elf_mark_broken(elf, "unable to load ELF headers into guest memory");
>  return;
>  }
>  
> +/*
> + * Load the section headers.
> + *
> + * NB: this _must_ be done one by one, and taking the bitness into 
> account,
> + * so that the guest can treat this as an array of type Elf{32/64}_Shdr.
> + */
> +shdr_size = elf_64bit(elf) ? sizeof(Elf64_Shdr) : sizeof(Elf32_Shdr);
> +for ( i = 0; i < ELF_BSDSYM_SECTIONS; i++ )
> +{
> +if ( elf_64bit(elf) )
> +shdr = &header.elf_header.section[i].e64;
> +else
> +shdr = &header.elf_header.section[i].e32;
> +
> +rc = elf_load_image(elf, header_base + ehdr_size + shdr_size * i,
> +ELF_REALPTR2PTRVAL(shdr), shdr_size, shdr_size);

You shouldn't read shdr_size bytes here, but only sizeof() ones.

Which btw also applies to the earlier memset()ing of the SHN_UNDEF
header, which it then also looks like gets overwritten by the loading
done here (irrespective of the change you make).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH for-4.8] libelf: fix symtab/strtab loading for 32bit domains

2016-10-05 Thread Roger Pau Monne
On Wed, Oct 05, 2016 at 09:51:06AM -0600, Jan Beulich wrote:
> >>> On 05.10.16 at 17:11,  wrote:
> > --- a/xen/common/libelf/libelf-loader.c
> > +++ b/xen/common/libelf/libelf-loader.c
> > @@ -262,13 +262,14 @@ static void elf_load_bsdsyms(struct elf_binary *elf)
> >  } __attribute__((packed)) header;
> >  
> >  ELF_HANDLE_DECL(elf_ehdr) header_handle;
> > -unsigned long shdr_size;
> > +unsigned long shdr_size, ehdr_size;
> >  ELF_HANDLE_DECL(elf_shdr) section_handle;
> > -unsigned int link, rc;
> > +unsigned int link, rc, i;
> >  elf_ptrval header_base;
> >  elf_ptrval elf_header_base;
> >  elf_ptrval symtab_base;
> >  elf_ptrval strtab_base;
> > +void *shdr;
> 
> I'd appreciate if you moved this into the scope where it's needed.
> Also I think it could be const.

Done.
 
> > @@ -394,15 +395,40 @@ do {  
> >  
> >  \
> >  header.size = strtab_base + elf_uval(elf, section_handle, sh_size) -
> >elf_header_base;
> >  
> > -/* Load the headers. */
> > +/* Load the size plus elf header. */
> > +ehdr_size = sizeof(header) - sizeof(header.elf_header.section);
> 
> I think offsetof(typeof(header), elf_header.section) would be the
> safer expression here, removing the dependency on the packed
> attribute.

Right, I was dubious about which one would be better. I've removed the 
packed attribute of elf_header, but the one from header needs to say (so 
that the elf_header is just after the uint32_t). 

> >  rc = elf_load_image(elf, header_base, ELF_REALPTR2PTRVAL(&header),
> > -sizeof(header), sizeof(header));
> > +ehdr_size, ehdr_size);
> >  if ( rc != 0 )
> >  {
> >  elf_mark_broken(elf, "unable to load ELF headers into guest 
> > memory");
> >  return;
> >  }
> >  
> > +/*
> > + * Load the section headers.
> > + *
> > + * NB: this _must_ be done one by one, and taking the bitness into 
> > account,
> > + * so that the guest can treat this as an array of type 
> > Elf{32/64}_Shdr.
> > + */
> > +shdr_size = elf_64bit(elf) ? sizeof(Elf64_Shdr) : sizeof(Elf32_Shdr);
> > +for ( i = 0; i < ELF_BSDSYM_SECTIONS; i++ )
> > +{
> > +if ( elf_64bit(elf) )
> > +shdr = &header.elf_header.section[i].e64;
> > +else
> > +shdr = &header.elf_header.section[i].e32;
> > +
> > +rc = elf_load_image(elf, header_base + ehdr_size + shdr_size * i,
> > +ELF_REALPTR2PTRVAL(shdr), shdr_size, 
> > shdr_size);
> 
> You shouldn't read shdr_size bytes here, but only sizeof() ones.

Well, shdr_size is just the result of a sizeof.

> Which btw also applies to the earlier memset()ing of the SHN_UNDEF
> header, which it then also looks like gets overwritten by the loading
> done here (irrespective of the change you make).

Hm, I'm not sure I follow. Memset()ing of 
header.elf_header.section[SHN_UNDEF] is required, so that we don't copy 
contents of the stack into guest memory. This memset is done to the local 
data, which is then copied into the guest memory space here.

Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] [PATCH] xen/x86: Update topology map for PV VCPUs

2016-10-05 Thread Boris Ostrovsky
Early during boot topology_update_package_map() computes
logical_pkg_ids for all present processors.

Later, when processors are brought up, identify_cpu() updates
these values based on phys_pkg_id which is a function of
initial_apicid. On PV guests the latter may point to a
non-existing node, causing logical_pkg_ids to be set to -1.

Intel's RAPL uses logical_pkg_id (as topology_logical_package_id())
to index its arrays and therefore in this case will point to index
65535 (since logical_pkg_id is a u16). This could lead to either a
crash or may actually access random memory location.

As a workaround, we recompute topology during CPU bringup to reset
logical_pkg_id to a valid value.

(The reason for initial_apicid being bogus is because it is
initial_apicid of the processor from which the guest is launched.
This value is CPUID(1).EBX[31:24])

Signed-off-by: Boris Ostrovsky 
Cc: sta...@vger.kernel.org
---

Copying Andrew for the CPUID part.

 arch/x86/xen/smp.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index 311acad..9fa27ce 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -87,6 +87,12 @@ static void cpu_bringup(void)
cpu_data(cpu).x86_max_cores = 1;
set_cpu_sibling_map(cpu);
 
+   /*
+* identify_cpu() may have set logical_pkg_id to -1 due
+* to incorrect phys_proc_id. Let's re-comupte it.
+*/
+   topology_update_package_map(apic->cpu_present_to_apicid(cpu), cpu);
+
xen_setup_cpu_clockevents();
 
notify_cpu_starting(cpu);
-- 
1.8.3.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] [xen-4.4-testing test] 101268: tolerable FAIL - PUSHED

2016-10-05 Thread osstest service owner
flight 101268 xen-4.4-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/101268/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-cubietruck 16 guest-start.2  fail in 101256 pass in 101268
 test-amd64-amd64-xl-qemuu-ovmf-amd64 16 guest-stop fail in 101256 pass in 
101268
 test-amd64-amd64-xl-qcow2 18 guest-start/debian.repeat fail in 101256 pass in 
101268
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 17 guest-start/debianhvm.repeat fail 
in 101256 pass in 101268
 test-amd64-amd64-xl-qemut-win7-amd64 10 migrate-support-check fail in 101256 
pass in 101268
 test-amd64-i386-xl-qemut-debianhvm-amd64 10 migrate-support-check fail in 
101256 pass in 101268
 test-amd64-i386-xl-qemuu-debianhvm-amd64 14 guest-saverestore.2 fail in 101256 
pass in 101268
 test-amd64-i386-libvirt 17 guest-start/debian.repeat fail in 101256 pass in 
101268
 test-amd64-i386-xl-qemut-debianhvm-amd64 11 saverestore-support-check fail in 
101256 pass in 101268
 test-amd64-i386-xl-qemuu-win7-amd64 13 guest-localmigrate  fail pass in 101256
 test-amd64-amd64-xl-qemut-debianhvm-amd64 15 guest-localmigrate/x10 fail pass 
in 101256
 test-amd64-i386-xend-qemut-winxpsp3  9 windows-install fail pass in 101256

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop   fail in 101256 like 100815
 test-armhf-armhf-xl-multivcpu 15 guest-start/debian.repeatfail like 100815
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail like 100815
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 100815

Tests which did not succeed, but are not blocking:
 test-amd64-i386-rumprun-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumprun-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-xend-qemut-winxpsp3 20 leak-check/check fail in 101256 never 
pass
 build-amd64-rumprun   5 rumprun-buildfail   never pass
 build-i386-rumprun5 rumprun-buildfail   never pass
 test-armhf-armhf-libvirt-qcow2  9 debian-di-installfail never pass
 test-armhf-armhf-libvirt 11 guest-start  fail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw  9 debian-di-installfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-xtf-amd64-amd64-2   10 xtf-fep  fail   never pass
 test-xtf-amd64-amd64-2   16 xtf/test-pv32pae-selftestfail   never pass
 test-xtf-amd64-amd64-2  41 xtf/test-pv64-pv-iopl~hypercall fail never pass
 test-xtf-amd64-amd64-2   42 xtf/test-pv64-pv-iopl~vmassist fail never pass
 test-xtf-amd64-amd64-1   10 xtf-fep  fail   never pass
 test-xtf-amd64-amd64-1   16 xtf/test-pv32pae-selftestfail   never pass
 test-xtf-amd64-amd64-1  41 xtf/test-pv64-pv-iopl~hypercall fail never pass
 test-xtf-amd64-amd64-1   42 xtf/test-pv64-pv-iopl~vmassist fail never pass
 test-xtf-amd64-amd64-4   10 xtf-fep  fail   never pass
 test-xtf-amd64-amd64-4   16 xtf/test-pv32pae-selftestfail   never pass
 test-xtf-amd64-amd64-4  41 xtf/test-pv64-pv-iopl~hypercall fail never pass
 test-xtf-amd64-amd64-4   42 xtf/test-pv64-pv-iopl~vmassist fail never pass
 test-xtf-amd64-amd64-5   10 xtf-fep  fail   never pass
 test-xtf-amd64-amd64-5   16 xtf/test-pv32pae-selftestfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-xtf-amd64-amd64-5  41 xtf/test-pv64-pv-iopl~hypercall fail never pass
 test-xtf-amd64-amd64-5   42 xtf/test-pv64-pv-iopl~vmassist fail never pass
 test-xtf-amd64-amd64-3   10 xtf-fep  fail   never pass
 test-xtf-amd64-amd64-3   16 xtf/test-pv32pae-selftestfail   never pass
 test-xtf-amd64-amd64-3  41 xtf/test-pv64-pv-iopl~hypercall fail never pass
 test-xtf-amd64-amd64-3   42 xtf/test-pv64-pv-iopl~vmassist fail never pass
 test-amd64-amd64-qe

Re: [Xen-devel] [PATCH] xen/x86: Update topology map for PV VCPUs

2016-10-05 Thread Andrew Cooper
On 05/10/16 18:09, Boris Ostrovsky wrote:
> Early during boot topology_update_package_map() computes
> logical_pkg_ids for all present processors.
>
> Later, when processors are brought up, identify_cpu() updates
> these values based on phys_pkg_id which is a function of
> initial_apicid. On PV guests the latter may point to a
> non-existing node, causing logical_pkg_ids to be set to -1.
>
> Intel's RAPL uses logical_pkg_id (as topology_logical_package_id())
> to index its arrays and therefore in this case will point to index
> 65535 (since logical_pkg_id is a u16). This could lead to either a
> crash or may actually access random memory location.
>
> As a workaround, we recompute topology during CPU bringup to reset
> logical_pkg_id to a valid value.
>
> (The reason for initial_apicid being bogus is because it is
> initial_apicid of the processor from which the guest is launched.
> This value is CPUID(1).EBX[31:24])
>
> Signed-off-by: Boris Ostrovsky 
> Cc: sta...@vger.kernel.org
> ---
>
> Copying Andrew for the CPUID part.

Yeah - that leaf is usually fiction.  (Specifically, the fiction of
whichever cpu a specific toolstack function happened to sample at the
point in time that it was choosing which cpuid values to fake up for the
guest).

I am currently working on fixing the reported topology information to be
architecturally plausible, but current and previous hypervisors will be
wrong.

~Andrew

>
>  arch/x86/xen/smp.c | 6 ++
>  1 file changed, 6 insertions(+)
>
> diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
> index 311acad..9fa27ce 100644
> --- a/arch/x86/xen/smp.c
> +++ b/arch/x86/xen/smp.c
> @@ -87,6 +87,12 @@ static void cpu_bringup(void)
>   cpu_data(cpu).x86_max_cores = 1;
>   set_cpu_sibling_map(cpu);
>  
> + /*
> +  * identify_cpu() may have set logical_pkg_id to -1 due
> +  * to incorrect phys_proc_id. Let's re-comupte it.
> +  */
> + topology_update_package_map(apic->cpu_present_to_apicid(cpu), cpu);
> +
>   xen_setup_cpu_clockevents();
>  
>   notify_cpu_starting(cpu);


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] New Outreachy Applicant

2016-10-05 Thread George Dunlap
On Tue, Oct 4, 2016 at 3:23 PM, George Dunlap  wrote:
> Thanks for this -- if you're up for it, let me see what kinds of next
> steps you can do (obviously when you have the opportunity).

So there are a couple of things we could do next.

* If you want a simple change that you can use to get experience with
sending patches to the list:

There are a number of instances in tools/libxl/xl_cmdimpl.c where
where domid is listed as uint32_t (or libxl_domid), but the printf
specifier is used as '%d'.  Change this to '%u'.

* If you want an investigation to do:

At least a couple of times I've accidentally run a golang program with
libxl functionality on a system that wasn't running Xen (i.e., I'd
rebooted onto Linux baremetal).  Instead of throwing an error, as you
would expect, it crashed with a SEGV.

Try to reproduce this bug, and see if you can fix it.

* If you want to dive into the project:

I can send you my most recent version of libxl.go (which has a few
more things implemented).  Modify it so that it builds as a package (I
think "xenproject.org/xenlight" is probably the best name), and wire
it into the build system in tools/golang/xenlight, and installs into
$prefix/share/gocode/src/xenproject.org/.

The key thing with the last one is not to get too bogged down in
systems you're not familiar with -- if you get stuck ask for help
relatively quickly; and if it's turning into a mess, I can just do the
Makefile side of things so that you can get to the actual Go side of
the project.

What do you think?

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] [xen-4.6-testing baseline-only test] 67796: regressions - trouble: blocked/broken/fail/pass

2016-10-05 Thread Platform Team regression test user
This run is configured for baseline tests only.

flight 67796 xen-4.6-testing real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/67796/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 2 hosts-allocate broken 
REGR. vs. 67782
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 2 hosts-allocate broken 
REGR. vs. 67782
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 2 hosts-allocate broken REGR. vs. 
67782
 test-amd64-i386-libvirt-xsm   2 hosts-allocate  broken REGR. vs. 67782
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 2 hosts-allocate broken REGR. vs. 
67782
 test-amd64-amd64-xl-xsm   2 hosts-allocate  broken REGR. vs. 67782
 test-amd64-i386-xl-xsm2 hosts-allocate  broken REGR. vs. 67782
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 2 hosts-allocate broken REGR. 
vs. 67782
 test-amd64-i386-xl-raw2 hosts-allocate  broken REGR. vs. 67782
 test-xtf-amd64-amd64-32 hosts-allocate  broken REGR. vs. 67782
 test-xtf-amd64-amd64-22 hosts-allocate  broken REGR. vs. 67782
 test-amd64-i386-xl-qemut-win7-amd64  2 hosts-allocate   broken REGR. vs. 67782
 test-xtf-amd64-amd64-52 hosts-allocate  broken REGR. vs. 67782
 test-amd64-i386-xl-qemuu-debianhvm-amd64 2 hosts-allocate broken REGR. vs. 
67782
 test-amd64-i386-xl-qemuu-win7-amd64  2 hosts-allocate   broken REGR. vs. 67782
 test-amd64-i386-xl-qemuu-ovmf-amd64  2 hosts-allocate   broken REGR. vs. 67782
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1 2 hosts-allocate broken REGR. vs. 
67782
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 2 hosts-allocate broken REGR. vs. 
67782
 test-xtf-amd64-amd64-42 hosts-allocate  broken REGR. vs. 67782
 test-amd64-i386-qemut-rhel6hvm-amd  2 hosts-allocatebroken REGR. vs. 67782
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 2 hosts-allocate broken 
REGR. vs. 67782
 test-amd64-i386-xl3 host-install(3) broken REGR. vs. 67782
 test-armhf-armhf-xl  15 guest-start/debian.repeat fail REGR. vs. 67782
 test-xtf-amd64-amd64-119 xtf/test-hvm32-invlpg~shadow fail REGR. vs. 67782
 test-xtf-amd64-amd64-1 26 xtf/test-hvm32pae-invlpg~shadow fail REGR. vs. 67782
 test-xtf-amd64-amd64-134 xtf/test-hvm64-invlpg~shadow fail REGR. vs. 67782
 test-amd64-amd64-i386-pvgrub 10 guest-start   fail REGR. vs. 67782

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 67782
 test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail like 67782
 test-amd64-amd64-xl-qemut-winxpsp3  9 windows-install  fail like 67782

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-rumprun-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-rumprun-i386  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 build-amd64-rumprun   5 rumprun-buildfail   never pass
 build-i386-rumprun5 rumprun-buildfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-midway   12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-midway   13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   ne

[Xen-devel] [qemu-mainline test] 101269: tolerable FAIL - PUSHED

2016-10-05 Thread osstest service owner
flight 101269 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/101269/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds15 guest-start/debian.repeat fail REGR. vs. 101240
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 101240
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 101240
 test-amd64-amd64-xl-rtds  9 debian-install   fail  like 101240

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass

version targeted for testing:
 qemuubbc4c3f4f3c624e2de64fdcb79f4dd8c1a508e9d
baseline version:
 qemuuc5d128ffeb5357df1ea3e6de0c13b3d6a09f6064

Last test of basis   101240  2016-10-01 11:46:19 Z4 days
Failing since101251  2016-10-04 10:44:48 Z1 days2 attempts
Testing same since   101269  2016-10-05 01:18:25 Z0 days1 attempts


People who touched revisions under test:
  Alistair Francis 
  Alistair Francis 
  Alistair Francis 
  Andrew Jones 
  Denis V. Lunev 
  Dr. David Alan Gilbert 
  Edgar E. Iglesias 
  Eduardo Habkost 
  Eric Auger 
  Evgeny Yakovlev 
  Fam Zheng 
  Jakub Jermar 
  John Snow 
  Kevin Wolf 
  Marc-André Lureau 
  Markus Armbruster 
  Pavel Fedin 
  Peter Lieven 
  Peter Maydell 
  Shannon Zhao 
  Stefano Stabellini 
  Vijay Kumar B 
  Vijay Kumar B. 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass  

Re: [Xen-devel] [PATCH v9 06/13] efi: create new early memory allocator

2016-10-05 Thread Julien Grall

Hi Jan,

Sorry for the late answer, I have been traveling the past 2 weeks.

On 30/09/2016 02:46, Jan Beulich wrote:

On 29.09.16 at 23:42,  wrote:

+#else
+static void __init free_ebmalloc_unused_mem(void)
+{
+}
+#endif


Did you build test this for ARM? The function ought to be unused,
as ...


@@ -1251,6 +1301,8 @@ void __init efi_init_memory(void)
 } *extra, *extra_head = NULL;
 #endif

+free_ebmalloc_unused_mem();


... the whole function here doesn't get built on ARM.

Julien - we're still awaiting your input on general aspects here.


efi_init_memory would need to be called during Xen boot on ARM. I am not 
sure where as I we don't yet have runtime support on ARM.


Other than that, the patch looks good to me.

Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [OSSTEST PATCH 1/2] libvirt: Check migration capabilities using proper XML parser

2016-10-05 Thread Julien Grall

Hi Ian,

On 04/10/2016 10:05, Ian Jackson wrote:

Ian Jackson writes ("[OSSTEST PATCH 1/2] libvirt: Check migration capabilities using 
proper XML parser"):

Do not grep the virsh capabilities output (!)  Instead, parse the XML
using perl's XML modules and look for the specific feature flag using
an XPATH pattern.

...>

+sub _check_capability ($$) {

...

+   $rc = $self->check_capability('/capabilities/host/migration_features');

 ^

Missing _.  I didn't test this again before sending it.  I'd still
like a review from libvirt (and Xen/ARM) folks.


I am not sure what kind of input you would need from Xen/ARM. This patch 
set looks very libvirt specific.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] Xen virtual IOMMU high level design doc

2016-10-05 Thread Konrad Rzeszutek Wilk
On Thu, Sep 15, 2016 at 10:22:36PM +0800, Lan, Tianyu wrote:
> Hi Andrew:
> Sorry to bother you. To make sure we are on the right direction, it's
> better to get feedback from you before we go further step. Could you
> have a look? Thanks.
> 
> On 8/17/2016 8:05 PM, Lan, Tianyu wrote:
> > Hi All:
> >  The following is our Xen vIOMMU high level design for detail
> > discussion. Please have a look. Very appreciate for your comments.
> > This design doesn't cover changes when root port is moved to hypervisor.
> > We may design it later.
> > 
> > 
> > Content:
> > ===
> > 
> > 1. Motivation of vIOMMU
> > 1.1 Enable more than 255 vcpus
> > 1.2 Support VFIO-based user space driver
> > 1.3 Support guest Shared Virtual Memory (SVM)
> > 2. Xen vIOMMU Architecture
> > 2.1 2th level translation overview
> > 2.2 Interrupt remapping overview
> > 3. Xen hypervisor
> > 3.1 New vIOMMU hypercall interface
> > 3.2 2nd level translation
> > 3.3 Interrupt remapping
> > 3.4 1st level translation
> > 3.5 Implementation consideration
> > 4. Qemu
> > 4.1 Qemu vIOMMU framework
> > 4.2 Dummy xen-vIOMMU driver
> > 4.3 Q35 vs. i440x
> > 4.4 Report vIOMMU to hvmloader
> > 
> > 
> > 1 Motivation for Xen vIOMMU
> > ===
> > 
> > 1.1 Enable more than 255 vcpu support
> > HPC virtualization requires more than 255 vcpus support in a single VM
> > to meet parallel computing requirement. More than 255 vcpus support
> > requires interrupt remapping capability present on vIOMMU to deliver
> > interrupt to #vcpu >255 Otherwise Linux guest fails to boot up with >255
> > vcpus if interrupt remapping is absent.
> > 
> > 
> > 1.2 Support VFIO-based user space driver (e.g. DPDK) in the guest
> > It relies on the 2nd level translation capability (IOVA->GPA) on
> > vIOMMU. pIOMMU 2nd level becomes a shadowing structure of
> > vIOMMU to isolate DMA requests initiated by user space driver.
> > 
> > 
> > 1.3 Support guest SVM (Shared Virtual Memory)
> > It relies on the 1st level translation table capability (GVA->GPA) on
> > vIOMMU. pIOMMU needs to enable both 1st level and 2nd level translation
> > in nested mode (GVA->GPA->HPA) for passthrough device. IGD passthrough
> > is the main usage today (to support OpenCL 2.0 SVM feature). In the
> > future SVM might be used by other I/O devices too.
> > 
> > 2. Xen vIOMMU Architecture
> > 
> > 
> > 
> > * vIOMMU will be inside Xen hypervisor for following factors
> > 1) Avoid round trips between Qemu and Xen hypervisor
> > 2) Ease of integration with the rest of the hypervisor
> > 3) HVMlite/PVH doesn't use Qemu
> > * Dummy xen-vIOMMU in Qemu as a wrapper of new hypercall to create
> > /destory vIOMMU in hypervisor and deal with virtual PCI device's 2th
> > level translation.
> > 
> > 2.1 2th level translation overview
> > For Virtual PCI device, dummy xen-vIOMMU does translation in the
> > Qemu via new hypercall.
> > 
> > For physical PCI device, vIOMMU in hypervisor shadows IO page table from
> > IOVA->GPA to IOVA->HPA and load page table to physical IOMMU.
> > 
> > The following diagram shows 2th level translation architecture.
> > +-+
> > |Qemu++   |
> > || Virtual|   |
> > ||   PCI device   |   |
> > |||   |
> > |++   |
> > ||DMA |
> > |V|
> > |  ++   Request  ++   |
> > |  |+<---+|   |
> > |  |  Dummy xen vIOMMU  | Target GPA |  Memory region |   |
> > |  |+--->+|   |
> > |  +-+--++---++   |
> > ||   ||
> > ||Hypercall  ||
> > +++
> > |Hypervisor  |   ||
> > ||   ||
> > |v   ||
> > | +--+--+||
> > | |   vIOMMU|||
> > | +--+--+||
> > ||   ||
> > |v   ||
> > | +--+--+||
> > | | 

Re: [Xen-devel] Fwd: Openindiana, using -machine pc, accel=xen in qemu - Success!

2016-10-05 Thread jim burns
Pls cc: me with any replies.

On Monday, 26 September 2016, 15:27:59 EDT, jim burns wrote:
> Unless you have any other ideas, I'm giving up for now. Thanx for your help.

Things have changed on the Hipster branch of OI, giving me a new environment 
to test.

Mainly, they have ditched legacy grub in favor of a Forth based boot loader. 
Whereas I still cannot boot the grub based install dvds in xen, I can with the 
Forth based vhd. However, things are still in flux. A lot of packages on 
Hipster are changing - mostly the the python packages. Instead of being 
updated, they are being removed, then, if appropriate, treated as a new 
install.

The effect is that the boot process is unreliable. I have to retry the boot 4 
- 10 times before it stops stalling. This is true for both kvm and xen. If I 
had to guess, I'd say that the init software is having trouble bringing up the 
extra vcpus. When you do an 'xl vcpu-list ...' during a boot stall, only vcpu0 
is 'b' or 'r'; the others are 'p'. When I do boot successfully, all vcpus are 
'b' or 'r'. Once booted, the system is reliable.

As such, installing a new OI Hipster system would be a 2 step procedure - use 
kvm to install OI on the vhd, run a complete system update, then convert over 
to xen with the notes below.

One other change I made which may or not be necessary: I noticed that a lot of 
my panic / stack traces involve platform dependant kernel modules that should 
not be loaded in an hvm machine - namely xnf, and sometimes xpvd, which are 
from the para-virtualization path in OI. I uninstalled the package xvm/pv, 
which is used by Solaris' lightweight container virtualization solution - 
Zones, which removes the /platform/i86hvm directory. There is also a /
platform/i86xpv directory which is used by the now unmaintained xen drivers. I 
couldn't remove the package corresponding to that directory without removing 
the non para-virtualization drivers in /platform/i86pc, so I just renamed the 
i86xpv directory. This step has to be done every time you get an update for 
the package kernel/platform. Once things appear more solid, including booting, 
these changes can easily be reversed.

The following guest domain cfg options are different from the cfg I posted 
earlier in this thread:

vcpus=2
memory=3076
boot="cd"
xen_platform_pci=0
#viridian=1
#viridian=0
viridian=[ "all" ]
hap=1
nestedhvm=1
#pvh=1 # causes panic w/stack trace
#hdtype='ahci' # causes panic w/stack trace

The new boot loader has a sub menu for setting kernel options, like verbose. I 
noticed on occasion that the stall occurs after initializing vcpus 0 & 1, with 
vcpus=4, so I reduced the number of vcpus.

I increased memory to match my kvm options.

Instead of booting off grub on the dvd, then chaining to Forth, I eliminated 
any left over initialization problems from grub by booting directly off the 
vhd.

The value of xen_platform_pci can be 0 or 1, with no difference - except, 
curiously enough, the network card generation number changes: e1000g1 for kvm 
and xen_platform_pci=0, e1000g2 otherwise. I would have thought the -netdev 
option would have more effect than the -machine option.

All the viridian options above are equally valid. The default is viridian=1.

Tho' commented out before, hap=1 is the default if your system supports it, as 
my Haswell icore-5 does.

When adding nestedhvm=1, the cpuid option 'vmx' shows up for the first time in 
the boot log.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [Xen-users] Fwd: Openindiana, using -machine pc, accel=xen in qemu - Success!

2016-10-05 Thread jim burns
On Wednesday, 5 October 2016, 11:57:11 EDT, Kevin Brooks wrote:
> Sure. Jim can access webmail at https://mail.netgate.net/webmail
> 
> Jim should have a document from us with all of his settings, including the
> link to webmail.

 - can someone other than the OP verify that this is a legitimate web 
site, and I have not just been phished? I'm not enabling javascript for this 
site until I get confirmation.

Thank you.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] Clarification regarding MEM_ACCESS_* flags usage

2016-10-05 Thread Julien Grall

Hello Tamas and Ravzan,

I have been looking into mem access support on ARM and I am wondering 
how we expect the flags MEM_ACCESS_{R,W,X} to be used when the 
permission fault is happening during stage 1 page table walk.


For instance, if the fault is happening when the processor is loading an 
instruction, MEM_ACCESS_X will be set. However, the table walker may 
have failed because it is not possible to read the entry or update it 
(e.g dirty management).


Let say the region has been protected read-write (I think it is 
XENMEM_access_x), it means that mem access will think it doesn't have to 
deal with the error and bail out. So the guest vCPU will get stuck 
forever repeating the stage-1 page table walk and getting an instruction 
fault.


Similarly, the bit ESR_EL2.WnR during a data abort indicates whether the 
instruction was a load or store and not whether the page table walker 
was reading or writing the entry (see more details on [1]).


So what is the expectation of the flags MEM_ACCESS_R (e.g 
npfec.read_access) and MEM_ACCESS_W (e.g npfec.write_access) for stage-2 
abort on stage-1 page table walk?


Regards,

[1] https://patchwork.kernel.org/patch/9356377/

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] [libvirt test] 101270: tolerable FAIL - PUSHED

2016-10-05 Thread osstest service owner
flight 101270 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/101270/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass

version targeted for testing:
 libvirt  66bfc7cc61ca0f6ca8d57811024565d2c89f47f6
baseline version:
 libvirt  a88c65e490d45e73715823b455799a58869ddd0e

Last test of basis   101232  2016-10-01 04:24:13 Z4 days
Testing same since   101270  2016-10-05 04:23:38 Z0 days1 attempts


People who touched revisions under test:
  Daniel Veillard 
  John Ferlan 
  Michal Privoznik 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-libvirt-xsm pass
 test-armhf-armhf-libvirt-xsm fail
 test-amd64-i386-libvirt-xsm  pass
 test-amd64-amd64-libvirt pass
 test-armhf-armhf-libvirt fail
 test-amd64-i386-libvirt  pass
 test-amd64-amd64-libvirt-pairpass
 test-amd64-i386-libvirt-pair pass
 test-armhf-armhf-libvirt-qcow2   fail
 test-armhf-armhf-libvirt-raw fail
 test-amd64-amd64-libvirt-vhd pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=libvirt
+ revision=66bfc7cc61ca0f6ca8d57811024565d2c89f47f6
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push libvirt 
66bfc7cc61ca0f6ca8d57811024565d2c89f47f6
+ branch=libvirt
+ revi

Re: [Xen-devel] Clarification regarding MEM_ACCESS_* flags usage

2016-10-05 Thread Razvan Cojocaru
Hello Julien,

> I have been looking into mem access support on ARM and I am wondering
> how we expect the flags MEM_ACCESS_{R,W,X} to be used when the
> permission fault is happening during stage 1 page table walk.
> 
> For instance, if the fault is happening when the processor is loading an
> instruction, MEM_ACCESS_X will be set. However, the table walker may
> have failed because it is not possible to read the entry or update it
> (e.g dirty management).
> 
> Let say the region has been protected read-write (I think it is
> XENMEM_access_x), it means that mem access will think it doesn't have to
> deal with the error and bail out. So the guest vCPU will get stuck
> forever repeating the stage-1 page table walk and getting an instruction
> fault.
> 
> Similarly, the bit ESR_EL2.WnR during a data abort indicates whether the
> instruction was a load or store and not whether the page table walker
> was reading or writing the entry (see more details on [1]).
> 
> So what is the expectation of the flags MEM_ACCESS_R (e.g
> npfec.read_access) and MEM_ACCESS_W (e.g npfec.write_access) for stage-2
> abort on stage-1 page table walk?
> 
> Regards,
> 
> [1] https://patchwork.kernel.org/patch/9356377/

I'm not sure what the right way forward is here, but I do know that
there's some "EPT misconfiguration" talk in the Intel SDM, specifically:

"25.2.3.1 EPT Misconfigurations

AN EPT misconfiguration occurs if any of the following is identified
while translating a guest-physical address:

• The value of bits 2:0 of an EPT paging-structure entry is either 010b
(write-only) or 110b (write/execute).

• The value of bits 2:0 of an EPT paging-structure entry is 100b
(execute-only) and this value is not supported by the logical processor.
Software should read the VMX capability MSR IA32_VMX_EPT_VPID_CAP to
determine whether this value is supported (see Appendix G.10).

• The value of bits 2:0 of an EPT paging-structure entry is not 000b
(the entry is present) and one of the following holds:

—   A reserved bit is set. This includes the setting of a bit in the
range 51:12 that is beyond the logical processor’s physical-address width.

—   The entry is the last one used to translate a guest physical address
(either an EPT PDE with bit 7 set to 1 or an EPT PTE) and the value of
bits 5:3 (EPT memory type) is 2, 3, or 7 (these values are reserved).

EPT misconfigurations result when an EPT paging-structure entry is
configured with settings reserved for future functionality.
Software developers should be aware that such settings may be used in
the future and that an EPT paging-structure entry that causes an EPT
misconfiguration on one processor might not do so in the future."

IIRC, an EPT misconfiguration usually triggers a triple fault in Xen.


Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] Clarification regarding MEM_ACCESS_* flags usage

2016-10-05 Thread Tamas K Lengyel
Hi Julien,
It is expected that certain combinations of mem_access flags will put the
domain into unstable condition, resulting in a crash or a hang. As Razvan
mentioned, on x86 we can end up triggering EPT misconfiguration with the
wrong set of flags. The user of the API is expected to know what he/she is
doing in this regard, we don't do any enforcements or sanity checking on
the Xen side.

As to the issue you describe, indeed that can happen. If the user marks a
pagetable area non-readable/non-writable and the way ARM reports a walk for
an instruction-fetch as an execute violation when it traps, it will hang
the VM in a continuous violation state as no execute-violation was
requested to be triggered on the gfn by the user. There are other
situations where this can happen, as on ARM there is no such thing as
execute-only memory, so any time the user requests memory to be
execute-only or writable-executable will lead to problems like this -
instruction fetch violation when the user only requested read-violations.
But again, the users are expected to know what they are doing and perform
their own sanity checks as appropriate.

Tamas

On Wed, Oct 5, 2016 at 2:06 PM, Razvan Cojocaru 
wrote:

> Hello Julien,
>
> > I have been looking into mem access support on ARM and I am wondering
> > how we expect the flags MEM_ACCESS_{R,W,X} to be used when the
> > permission fault is happening during stage 1 page table walk.
> >
> > For instance, if the fault is happening when the processor is loading an
> > instruction, MEM_ACCESS_X will be set. However, the table walker may
> > have failed because it is not possible to read the entry or update it
> > (e.g dirty management).
> >
> > Let say the region has been protected read-write (I think it is
> > XENMEM_access_x), it means that mem access will think it doesn't have to
> > deal with the error and bail out. So the guest vCPU will get stuck
> > forever repeating the stage-1 page table walk and getting an instruction
> > fault.
> >
> > Similarly, the bit ESR_EL2.WnR during a data abort indicates whether the
> > instruction was a load or store and not whether the page table walker
> > was reading or writing the entry (see more details on [1]).
> >
> > So what is the expectation of the flags MEM_ACCESS_R (e.g
> > npfec.read_access) and MEM_ACCESS_W (e.g npfec.write_access) for stage-2
> > abort on stage-1 page table walk?
> >
> > Regards,
> >
> > [1] https://patchwork.kernel.org/patch/9356377/
>
> I'm not sure what the right way forward is here, but I do know that
> there's some "EPT misconfiguration" talk in the Intel SDM, specifically:
>
> "25.2.3.1 EPT Misconfigurations
>
> AN EPT misconfiguration occurs if any of the following is identified
> while translating a guest-physical address:
>
> • The value of bits 2:0 of an EPT paging-structure entry is either 010b
> (write-only) or 110b (write/execute).
>
> • The value of bits 2:0 of an EPT paging-structure entry is 100b
> (execute-only) and this value is not supported by the logical processor.
> Software should read the VMX capability MSR IA32_VMX_EPT_VPID_CAP to
> determine whether this value is supported (see Appendix G.10).
>
> • The value of bits 2:0 of an EPT paging-structure entry is not 000b
> (the entry is present) and one of the following holds:
>
> —   A reserved bit is set. This includes the setting of a bit in the
> range 51:12 that is beyond the logical processor’s physical-address width.
>
> —   The entry is the last one used to translate a guest physical address
> (either an EPT PDE with bit 7 set to 1 or an EPT PTE) and the value of
> bits 5:3 (EPT memory type) is 2, 3, or 7 (these values are reserved).
>
> EPT misconfigurations result when an EPT paging-structure entry is
> configured with settings reserved for future functionality.
> Software developers should be aware that such settings may be used in
> the future and that an EPT paging-structure entry that causes an EPT
> misconfiguration on one processor might not do so in the future."
>
> IIRC, an EPT misconfiguration usually triggers a triple fault in Xen.
>
>
> Thanks,
> Razvan
>
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
>
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] Clarification regarding MEM_ACCESS_* flags usage

2016-10-05 Thread Julien Grall



On 05/10/2016 13:23, Tamas K Lengyel wrote:

Hi Julien,
It is expected that certain combinations of mem_access flags will put
the domain into unstable condition, resulting in a crash or a hang. As
Razvan mentioned, on x86 we can end up triggering EPT misconfiguration
with the wrong set of flags. The user of the API is expected to know
what he/she is doing in this regard, we don't do any enforcements or
sanity checking on the Xen side.

As to the issue you describe, indeed that can happen. If the user marks
a pagetable area non-readable/non-writable and the way ARM reports a
walk for an instruction-fetch as an execute violation when it traps, it
will hang the VM in a continuous violation state as no execute-violation
was requested to be triggered on the gfn by the user. There are other
situations where this can happen, as on ARM there is no such thing as
execute-only memory, so any time the user requests memory to be
execute-only or writable-executable will lead to problems like this -
instruction fetch violation when the user only requested
read-violations. But again, the users are expected to know what they are
doing and perform their own sanity checks as appropriate.


I think the problem I described is neither the fault of the user, 
neither a misconfiguration of the page table. Let me clarify it.


The user can purposefully restrict the access to stage-1 page table to 
detect when the OS is modifying them. By side effect, this will also 
impact the page table walker.


A prefetch abort (e.g when an error occurs when the processor is trying 
to load the instruction) can either occur during a stage-1 page table 
walk (e.g the underlying memory of stage-1 page table has been 
protected) or because the permission in the stage-2 entry has been 
restricted.


In the case of the latter, this will always be because the memory is not 
executable. However, for the former may happen if the page table walker 
(i.e the MMU) is reading/writing the entry.


However, Xen ARM today is always considering that a prefetch abort will 
happen because it was not possible to execute the instruction.


I requested clarification about the flags because we need to fix this 
valid issue. From the usage on ARM and in the vm event app, it is not 
clear how those flags should be used.


Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] Xen Code Review Dashboard - Outreachy Program Project

2016-10-05 Thread Jesus M. Gonzalez-Barahona
On Tue, 2016-10-04 at 20:18 -0400, tevin.k.mall...@gmail.com wrote:
> Hello Jesus!
>  
> Thank you for taking the time out of your busy schedule to responded.
> I would love a summary of the small contribution over email, as this
> will allow me to get started on the project sooner. I am located in
> Florida USA, on Eastern Daylight Time and available anytime on
> October 6th to discuss details. If you have certain times that work
> best for you I can easily adjust my schedule. Just let me know when
> you would like to chat via IRC and I will be there. For the most part
> my schedule is very flexible, I am always available on Mondays,
> Tuesdays and Thursdays at anytime. Thank you once again and I look
> forward to hearing from you.

OK, here we are. With respect to the project in general, I assume
you're familiar with

https://wiki.xenproject.org/wiki/Outreach_Program_Projects#Xen_Code_Rev
iew_Dashboard

Right?

The main aim of the project is to reproduce all the process up to
having something like https://xen.biterg.io/ But using only GrimoireLab
tools

http://grimoirelab.github.io

Now it is using a mixture of MetricsGrimoire and custom scripts

The first step is to get information both from mailing lists and git
repositories using Perceval, and storing it in ElasticSearch. Later,
there are some scripts that should be ported to use this ElasticSearch
data (instead of the SQL data they are using now). With that, produce
the ElasticSearch indexes for the dashboard. Then, if possible, improve
the dashbosrd and make it more useful for the Xen community

Now, about the microtask.

I guess you know you can start with one microtask to show that you are
likely to be the right person for this project, according to Outreachy
requirements.

In this case, the microtask would be getting data from a mailing list
with Perceval. The mailing list is xen-devel. You get its archives
analyzed by Perceval, and store the resulting raw index in
ElasticSearch.

Once you're good with this, you do the same for the Xen git repo

And once you're good with this, you write an script which produces a
new index with some information for each commit, plus the branch in
which it was committed

Mboxes are at

https://lists.xenproject.org/archives/html/mbox/

They are then named xen-devel--, e.g. xen-devel-2016-09

You can write a little script using wget or curl to downoad several of
them at once. To begin with, you can start with some of them (say 5-10)

The code contribution result of the microtask would be the
identification of the branches, based on the output of Perceval / git

The setup would be you getting all the info from some mboxes in
ElasticSearch, a git repo in ElasticSearch, and a simple index,
combined of both plus branches information, again in ElasticSearch

Once you have those in Elasticsearch, just produce the result of
querying some of the items in ElasticSearch with curl, and the code for
the identification of branches. All of this can be stored in a git
repository for verification.

I can support you via irc and email if you have any trouble.

We're compiling some information on how to use GrimoireLib in

https://jgbarah.gitbooks.io/grimoirelab-training

Maybe that's a good place to start.

Is all of this ok with you?

Jesus.

-- 
Bitergia: http://bitergia.com
/me at Twitter: https://twitter.com/jgbarah


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] [xen-4.4-testing baseline-only test] 67805: trouble: blocked/broken

2016-10-05 Thread Platform Team regression test user
This run is configured for baseline tests only.

flight 67805 xen-4.4-testing real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/67805/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i386-xend   2 hosts-allocate  broken REGR. vs. 67679
 build-i3862 hosts-allocate  broken REGR. vs. 67679
 build-armhf   2 hosts-allocate  broken REGR. vs. 67679
 build-armhf-pvops 2 hosts-allocate  broken REGR. vs. 67679
 build-amd64-pvops 2 hosts-allocate  broken REGR. vs. 67679
 build-amd64   2 hosts-allocate  broken REGR. vs. 67679
 build-i386-pvops  2 hosts-allocate  broken REGR. vs. 67679
 build-amd64-xend  2 hosts-allocate  broken REGR. vs. 67679

Regressions which are regarded as allowable (not blocking):
 build-i386-xend   3 capture-logs   broken blocked in 67679
 build-i386-pvops  3 capture-logs   broken blocked in 67679
 build-i3863 capture-logs   broken blocked in 67679
 build-armhf   3 capture-logs   broken blocked in 67679
 build-armhf-pvops 3 capture-logs   broken blocked in 67679
 build-amd64-xend  3 capture-logs   broken blocked in 67679
 build-amd64-pvops 3 capture-logs   broken blocked in 67679
 build-amd64   3 capture-logs   broken blocked in 67679

Tests which did not succeed, but are not blocking:
 test-amd64-i386-rumprun-i386  1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 build-amd64-rumprun   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumprun-amd64  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 build-i386-rumprun1 build-check(1)   blocked  n/a
 test-xtf-amd64-amd64-11 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-midway1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-xtf-amd64-amd64-21 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-xtf-amd64-amd64-41 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-xtf-amd64-amd64-31 build-check(1)   blocked  n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-xtf-amd64-amd64-51 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemut-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-qemut-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-amd64-amd64-pv   1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-i386-pv1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check

[Xen-devel] [qemu-mainline baseline-only test] 67806: trouble: blocked/broken

2016-10-05 Thread Platform Team regression test user
This run is configured for baseline tests only.

flight 67806 qemu-mainline real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/67806/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i386-xsm2 hosts-allocate  broken REGR. vs. 67788
 build-armhf-pvops 2 hosts-allocate  broken REGR. vs. 67788
 build-amd64-pvops 2 hosts-allocate  broken REGR. vs. 67788
 build-i3862 hosts-allocate  broken REGR. vs. 67788
 build-i386-pvops  2 hosts-allocate  broken REGR. vs. 67788
 build-armhf-xsm   2 hosts-allocate  broken REGR. vs. 67788
 build-amd64-xsm   2 hosts-allocate  broken REGR. vs. 67788
 build-armhf   2 hosts-allocate  broken REGR. vs. 67788
 build-amd64   2 hosts-allocate  broken REGR. vs. 67788
 build-armhf-pvops 3 capture-logsbroken REGR. vs. 67788

Regressions which are regarded as allowable (not blocking):
 build-armhf-xsm   3 capture-logs   broken blocked in 67788
 build-i386-xsm3 capture-logs   broken blocked in 67788
 build-i386-pvops  3 capture-logs   broken blocked in 67788
 build-i3863 capture-logs   broken blocked in 67788
 build-amd64-pvops 3 capture-logs   broken blocked in 67788
 build-amd64-xsm   3 capture-logs   broken blocked in 67788
 build-armhf   3 capture-logs   broken blocked in 67788
 build-amd64   3 capture-logs   broken blocked in 67788

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-midway1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvh-amd   1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl   1 build-chec

Re: [Xen-devel] [OSSTEST PATCH 1/2] libvirt: Check migration capabilities using proper XML parser

2016-10-05 Thread Jim Fehlig

On 10/04/2016 11:02 AM, Ian Jackson wrote:

Do not grep the virsh capabilities output (!)  Instead, parse the XML
using perl's XML modules and look for the specific feature flag using
an XPATH pattern.

AFAICT from looking at the XML, that's


  

  

But the original code does not test for .

Xen could in principle (and AIUI might in the future, on ARM) support
save/restore but not live migration.  Currently it supports neither.

I don't know whether libvirt's capabilities system can capture this
distinction.  libvirt.git#1d37a4c4 "libxl: detect support for save and
restore" suggests not.


That's correct. But as Martin suggested in 2/2, libvirt could be enhanced to 
report the distinction if/when needed.




Perhaps relatedly, I am not sure whether this test should be changed
to look for the xpath
  /capabilities/host/migration_features/live
instead.  The schema (libvirt.git/docs/schemas/capability.rng) seems
to suggest that it probably should.


Agreed. migrate_check() should look for
/capabilities/host/migration_features/live.

Regards,
Jim



For now, this osstest commit has no ultimate functional change (with
libvirt output as it currently appears to be on real hosts).

Signed-off-by: Ian Jackson 
CC: Julien Grall 
CC: Jim Fehlig 
---
 Osstest/Toolstack/libvirt.pm | 17 ++---
 1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/Osstest/Toolstack/libvirt.pm b/Osstest/Toolstack/libvirt.pm
index 69ff0bb..b7db7af 100644
--- a/Osstest/Toolstack/libvirt.pm
+++ b/Osstest/Toolstack/libvirt.pm
@@ -21,6 +21,8 @@ use strict;
 use warnings;

 use Osstest::TestSupport;
+use XML::LibXML::XPathContext;
+use XML::LibXML;

 sub new {
 my ($class, $ho, $methname,$asset) = @_;
@@ -72,6 +74,17 @@ sub shutdown_wait ($$$) {
 guest_await_destroy($gho,$timeout);
 }

+sub _check_capability ($$) {
+my ($self, $xpath) = @_;
+my $ho = $self->{Host};
+my $caps = target_cmd_output_root($ho, 'virsh capabilities');
+my $stash = open_unique_stashfile('virsh-capabilities');
+my $dom = XML::LibXML->load_xml(string => $caps);
+my $xc = XML::LibXML::XPathContext->new($dom);
+my @nodes = $xc->findnodes($xpath);
+return !!@nodes;
+}
+
 sub migrate_check ($$) {
 my ($self, $local) = @_;
 my $rc;
@@ -80,9 +93,7 @@ sub migrate_check ($$) {
 # local migration is not supported
 $rc = 1;
 } else {
-   my $ho = $self->{Host};
-   my $caps = target_cmd_output_root($ho, "virsh capabilities");
-   $rc = ($caps =~ m//) ? 0 : 1
+   $rc = $self->check_capability('/capabilities/host/migration_features');
 }

 logm("rc=$rc");




___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [OSSTEST PATCH 2/2] libvirt: Do not attempt save/restore when migration not advertised

2016-10-05 Thread Jim Fehlig

On 10/04/2016 11:02 AM, Ian Jackson wrote:

Currently, osstest wrongly thinks that ARM can do save/restore,
because `virsh help' does mention the save command (on all
architectures).

Additionally, check the virth capabilities xpath
   /capabilities/host/migration_features
to try to see whether this host supports migration.

I am not sure if this is the right path to check.  Perhaps
   /capabilities/host/migration_features/live
is more correct, but this may be wrong if Xen comes to support save/restore
on ARM, but not live migration (but perhaps libvirt cannot express this
distinction in which case perhaps it's right after all).


Looking at the capabilities generation code again, I see that 
virCapabilitiesNew() takes 'offlineMigrate' and 'liveMigrate' parameters. I 
assume offline in this context means save, copy, restore. Martin, is that 
assumption correct?


If so, I think the saverestore_check() below can look for
/capabilities/host/migration_features. The migration check in 1/2 can look for
/capabilities/host/migration_features/live. Is it fair to assume save/restore is 
available when live migration is supported?


Regards,
Jim



Signed-off-by: Ian Jackson 
CC: Julien Grall 
CC: Jim Fehlig 
---
 Osstest/Toolstack/libvirt.pm | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/Osstest/Toolstack/libvirt.pm b/Osstest/Toolstack/libvirt.pm
index b7db7af..250fe47 100644
--- a/Osstest/Toolstack/libvirt.pm
+++ b/Osstest/Toolstack/libvirt.pm
@@ -111,7 +111,9 @@ sub check_for_command($$) {

 sub saverestore_check ($) {
 my ($self) = @_;
-return check_for_command($self, "save");
+return
+   _check_capability($self, '/capabilities/host/migration_features') &&
+   check_for_command($self, "save");
 }

 sub migrate () {




___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] [xen-4.7-testing test] 101272: tolerable FAIL - PUSHED

2016-10-05 Thread osstest service owner
flight 101272 xen-4.7-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/101272/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds  9 debian-install   fail  like 101197
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 101197
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail like 101197
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 101197

Tests which did not succeed, but are not blocking:
 test-amd64-i386-rumprun-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumprun-amd64  1 build-check(1)   blocked  n/a
 build-i386-rumprun5 rumprun-buildfail   never pass
 build-amd64-rumprun   5 rumprun-buildfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass

version targeted for testing:
 xen  3903db165ec9f823bfc0aca91e213c2cc221dc45
baseline version:
 xen  506182e00772ece038ca5207c54dfab3de57adef

Last test of basis   101197  2016-09-28 23:51:34 Z7 days
Testing same since   101255  2016-10-04 13:47:30 Z1 days2 attempts


People who touched revisions under test:
  Jan Beulich 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64-xtf  pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-prev pass
 build-i386-prev  pass
 buil

[Xen-devel] [xen-unstable test] 101274: regressions - FAIL

2016-10-05 Thread osstest service owner
flight 101274 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/101274/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-vhd   9 debian-di-installfail REGR. vs. 101250
 test-armhf-armhf-xl-credit2 15 guest-start/debian.repeat fail REGR. vs. 101250
 test-armhf-armhf-libvirt-qcow2  6 xen-boot   fail REGR. vs. 101250

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds 15 guest-start/debian.repeatfail  like 101246
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail like 101250
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 101250
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stopfail like 101250
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 101250
 test-amd64-amd64-xl-rtds  9 debian-install   fail  like 101250

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-rumprun-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-rumprun-i386  1 build-check(1)   blocked  n/a
 build-amd64-rumprun   5 rumprun-buildfail   never pass
 build-i386-rumprun5 rumprun-buildfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass

version targeted for testing:
 xen  b7dd797c7fe4cd849018f78f6c7b9eb3d33b89d8
baseline version:
 xen  40277cded320efa32b86c0c9f01e33145eab5ee4

Last test of basis   101250  2016-10-04 02:36:21 Z2 days
Failing since101254  2016-10-04 13:18:43 Z1 days2 attempts
Testing same since   101274  2016-10-05 05:24:58 Z0 days1 attempts


People who touched revisions under test:
  Anthony PERARD 
  casionwoo 
  Ian Jackson 
  Jan Beulich 
  JEUNGWOO, YOO 
  Wei Liu 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64-xtf  pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 b

[Xen-devel] [ovmf test] 101297: all pass - PUSHED

2016-10-05 Thread osstest service owner
flight 101297 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/101297/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 2cf9ecd2262e4098171fa27c97b56d483f5cf3b4
baseline version:
 ovmf c0b7e2b2bfc2748112607bfe83fc99cf48c97b48

Last test of basis   101249  2016-10-04 01:14:18 Z2 days
Testing same since   101297  2016-10-05 22:45:55 Z0 days1 attempts


People who touched revisions under test:
  Tapan Shah 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=ovmf
+ revision=2cf9ecd2262e4098171fa27c97b56d483f5cf3b4
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push ovmf 
2cf9ecd2262e4098171fa27c97b56d483f5cf3b4
+ branch=ovmf
+ revision=2cf9ecd2262e4098171fa27c97b56d483f5cf3b4
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=ovmf
+ xenbranch=xen-unstable
+ '[' xovmf = xlinux ']'
+ linuxbranch=
+ '[' x = x ']'
+ qemuubranch=qemu-upstream-unstable
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable
+ prevxenbranch=xen-4.7-testing
+ '[' x2cf9ecd2262e4098171fa27c97b56d483f5cf3b4 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/linux-firmware.git
++ : osst...@xenbits.xen.org:/home/osstest/ext/linux-firmware.git
++ : git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
++ : osst...@xenbits.xen.org:/home/xen/git/linux-pvops.g

[Xen-devel] [xen-4.5-testing test] 101275: tolerable FAIL - PUSHED

2016-10-05 Thread osstest service owner
flight 101275 xen-4.5-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/101275/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-qemuu-nested-intel 10 nested-setup fail in 101258 pass in 
101275
 test-armhf-armhf-xl   6 xen-boot   fail pass in 101258
 test-amd64-amd64-libvirt 14 guest-saverestore  fail pass in 101258
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 15 guest-localmigrate/x10 fail pass 
in 101258

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds 15 guest-start/debian.repeat fail in 101258 like 
101088
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 101070
 test-amd64-amd64-xl-qemuu-winxpsp3 15 guest-localmigrate/x10  fail like 101088
 test-armhf-armhf-xl-rtds 11 guest-start  fail  like 101106
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail like 101106
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 101106
 test-amd64-amd64-xl-rtds  6 xen-boot fail  like 101106
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stopfail like 101106

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-rumprun-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-rumprun-i386  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl 12 migrate-support-check fail in 101258 never pass
 test-armhf-armhf-xl 13 saverestore-support-check fail in 101258 never pass
 test-armhf-armhf-xl-rtds12 migrate-support-check fail in 101258 never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-check fail in 101258 never pass
 build-i386-rumprun5 rumprun-buildfail   never pass
 build-amd64-rumprun   5 rumprun-buildfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 10 guest-start  fail   never pass
 test-armhf-armhf-libvirt-qcow2 10 guest-start  fail never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-xl-vhd  10 guest-start  fail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass

version targeted for testing:
 xen  cfe165d08054b717825a5b26f468ea0b04b3f212
baseline version:
 xen  84e4e56aa643444f13686362535d8ca8a06839c6

Last test of basis   101106  2016-09-22 14:57:23 Z   13 days
Testing same since   101258  2016-10-04 13:48:08 Z1 days2 attempts


People who touched revisions under test:
  Jan Beulich 

jobs:
 build-amd64-xtf  pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-prev pass
 build-i386-prev  pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 build-amd64-rumprun  fail
 build-i386-rumprun   fail
 test-xtf-amd64-amd64-1   pass
 test-xtf-amd64-amd64-2   pas