Re: [Xen-devel] [libvirt] domXML modeling question

2019-03-06 Thread Daniel P . Berrangé
On Mon, Mar 04, 2019 at 04:42:32PM -0700, Jim Fehlig wrote:
> Adding xen-devel to cc in case anyone there wants to comment on my latest
> proposal...
> 
> On 2/20/19 5:20 PM, Jim Fehlig wrote:
> > There have been a few requests [1][2] to support Xen's max_grant_frames
> > setting in libvirt domXML, but I'm not quite sure how to model it. The
> > documentation [3] on this setting states:
> > 
> > Specify the maximum number of grant frames the domain is allowed to have.  
> > This
> > value controls how many pages the domain is able to grant access to for 
> > other
> > domains, needed e.g. for the operation of paravirtualized devices.  The 
> > default
> > is settable via xl.conf(5).
> 
> I've sent a patch to introduce an analogous default in the libvirt libxl 
> driver
> 
> https://www.redhat.com/archives/libvir-list/2019-March/msg00123.html
> 
> > 
> > It smells of a  setting, e.g. the amount of memory a domain can
> > share, but doesn't map to any of the existing settings. A new subelement
> >  doesn't feel right. Does anyone suggest a better way of
> > modeling max_grant_frames?
> 
> After discussing the max_grant_frames setting a bit more with Juergen I had
> the idea to model it as IO buffer space (or DMA space) of a xenbus
> "controller". All PV devices in the guest connect to the xenbus controller
> and make use of the available I/O buffer space. Guests with more PV devices
> requiring more buffer can increase the space on the xenbus controller
> device.
> 
> One small wrinkle in this idea is that we currently don't model xenbus in
> libvirt. I'd need to add support for a new xenbus controller type and start
> implicitly creating it when creating guests with PV devices, similar to
> auto-creation of controllers in the qemu driver. Also, there is no existing
> controller setting for specifying buffer space. Perhaps a 'ram' attribute
> could be added, similar to specifying memory for  devices? E.g.
> 
>   
> 
> Any opinion on this approach? Or other ideas for modeling this setting
> in libvirt?

Regardless of max grant frames support I think modeling xenbus as a
 is a reasonable thing to want to do. I don't have a
preference on whether you call it "ram" or explicitly "maxGrantFrames"
as an attribute.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] Can someone pls repair patchwork?

2019-03-06 Thread Oleksandr Andrushchenko

To whom it may concern

Since late 2017 the very useful Patchwork resource [1]
stopped working after (as I assume) Xen-devel list has changed
its address from xen-de...@lists.xen.org to the current one.
Patchwork is still configured to the old one, so recent
patches are not archived.
Could the respective owner from Xen community please take a look at [2]
and make Patchwork work again? In particular Patchwork is
very useful when you need a patch in mbox format without pain.

Thank you,
Oleksandr

[1] https://patchwork.kernel.org/project/xen-devel/list/
[2] https://patchwork.kernel.org/project/xen-devel/

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-coverity test] 133615: all pass - PUSHED

2019-03-06 Thread osstest service owner
flight 133615 xen-unstable-coverity real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133615/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xen  eeb31ee522c7bb8541eb4c037be2c42bfcf0a3c3
baseline version:
 xen  f393b82fe5ba3ed9cfe2b306ffa53368e55b75af

Last test of basis   133557  2019-03-03 09:30:21 Z3 days
Testing same since   133615  2019-03-06 09:18:51 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Ian Jackson 
  Jan Beulich 
  Juergen Gross 
  Manuel Bouyer 
  Roger Pau Monné 
  Sergey Dyasli 
  Wei Liu 

jobs:
 coverity-amd64   pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   f393b82fe5..eeb31ee522  eeb31ee522c7bb8541eb4c037be2c42bfcf0a3c3 -> 
coverity-tested/smoke

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Xen ARM GPU passthrough without IOMMU

2019-03-06 Thread Andrii Anisov

Hello Jinch,

On 05.03.19 15:35, Jinch wrote:

Thank you for your reply,
Now I can set the disk and some basic devices to run Android domu to console,
The main issue is the display GUI of Android.


Here you should distinguish clearly display subsystem and 3D rendering engine. 
Yet you need them both for your Android showing you something.
We have a quite generic PV DRM implementation, both FE and BE, which is called 
to serve as a display subsystem.
But 3D rendering is really vendor specific, and no generic approach 
implementation is known to me. Except software rendering for sure:)

You  might look at the hwcomposer we are using in our setup [1], but it is 
still IMG specific in terms of 3D.


I can use the drm PV drivers at https://github.com/xen-troops/displ_be to set 
the drm driver,
Or the next step is to research how to set drm device passthrough.


Sorry I didn't get that clearly. Do you say that you managed to bringup PV DRM 
for Android? Or it is your next step? And why do you say about DRM passthrough?


But the imxqxp board I use doesn’t have IOMMU, the GPU passthrough may have 
some problems.

I guess you should ask your GPU vendor about their virtualization solution 
first. That might give you a clue how can you provide your GPU access to other 
VM address space.
On our side we requested hypervisor to translate IPA to PA [2], and then fed 
those addresses to GPU from its driver. We do that for buggy silicon revision 
where GPU's IOMMU does not work.
Yet it makes more boards available for development, rather than for production.


Do you have some tutorials about how to set display and GUI when running 
Android as domu?

You might grab more information about PV DRM here [3].
XEN vdispl configuration you will find here [4].
Setting up Android display and GUI - you will not find any tutorials. You need 
your hwcomposer and gralloc which are 3D vendor specific, so contact your 
vendor.

[1] 
https://github.com/xen-troops/android_external_drm_hwcomposer/commits/android-9.0.0_r3-xt0.2
[2] https://github.com/xen-troops/xen/blob/master/xen/arch/arm/mm.c#L1332
[3] https://lwn.net/Articles/750258/
[4] https://xenbits.xen.org/docs/unstable-staging/man/xl.cfg.5.html#Devices

--
Sincerely,
Andrii Anisov.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Can someone pls repair patchwork?

2019-03-06 Thread Julien Grall

(+ Lars)

On 06/03/2019 10:04, Oleksandr Andrushchenko wrote:

To whom it may concern


Hi Oleksandr,



Since late 2017 the very useful Patchwork resource [1]
stopped working after (as I assume) Xen-devel list has changed
its address from xen-de...@lists.xen.org to the current one.
Patchwork is still configured to the old one, so recent
patches are not archived.
Could the respective owner from Xen community please take a look at [2]
and make Patchwork work again? In particular Patchwork is
very useful when you need a patch in mbox format without pain.


Patchwork is hosted by the kernel community. So it would be best if you contact 
them directly. [3].


Cheers,


[1] https://patchwork.kernel.org/project/xen-devel/list/
[2] https://patchwork.kernel.org/project/xen-devel/


[3] https://www.kernel.org/category/contact-us.html

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] Intel TXT maintainter update

2019-03-06 Thread Hawrylko, Lukasz
Due to personal changes at Intel, I am new TXT maintainer for XEN.
Adding patch that updates maintainers list.

Thanks,
Lukasz
diff --git a/MAINTAINERS b/MAINTAINERS
index a0cda4f7a1..4c47294706 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -237,7 +237,7 @@ F:	xen/arch/x86/debug.c
 F:	tools/debugger/gdbsx/
 
 INTEL(R) TRUSTED EXECUTION TECHNOLOGY (TXT)
-M:	Gang Wei 
+M:	Lukasz Hawrylko 
 M:	Shane Wang 
 S:	Supported
 F:	xen/arch/x86/tboot.c


smime.p7s
Description: S/MIME cryptographic signature


Intel Technology Poland sp. z o.o.
ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial 
Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | 
Kapital zakladowy 200.000 PLN.

Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i 
moze zawierac informacje poufne. W razie przypadkowego otrzymania tej 
wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; 
jakiekolwiek
przegladanie lub rozpowszechnianie jest zabronione.
This e-mail and any attachments may contain confidential material for the sole 
use of the intended recipient(s). If you are not the intended recipient, please 
contact the sender and delete all copies; any review or distribution by
others is strictly prohibited.
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 8/9] viridian: add implementation of synthetic timers

2019-03-06 Thread Paul Durrant
> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 25 February 2019 14:54
> To: Paul Durrant 
> Cc: Julien Grall ; Andrew Cooper 
> ; Roger Pau Monne
> ; Wei Liu ; George Dunlap 
> ; Ian
> Jackson ; Stefano Stabellini 
> ; xen-devel  de...@lists.xenproject.org>; Konrad Rzeszutek Wilk ; 
> Tim (Xen.org)
> 
> Subject: Re: [PATCH v3 8/9] viridian: add implementation of synthetic timers
> 
> >>> On 31.01.19 at 11:47,  wrote:
> > --- a/xen/arch/x86/hvm/viridian/synic.c
> > +++ b/xen/arch/x86/hvm/viridian/synic.c
> > @@ -329,7 +329,53 @@ void viridian_synic_domain_deinit(struct domain *d)
> >
> >  void viridian_synic_poll_messages(struct vcpu *v)
> >  {
> > -/* There are currently no message sources */
> > +viridian_time_poll_timers(v);
> > +}
> > +
> > +bool viridian_synic_deliver_timer_msg(struct vcpu *v, unsigned int sintx,
> > +  unsigned int index,
> > +  int64_t expiration, int64_t delivery)
> > +{
> > +const union viridian_sint_msr *vs = &v->arch.hvm.viridian->sint[sintx];
> > +HV_MESSAGE *msg = v->arch.hvm.viridian->simp.ptr;
> > +struct {
> > +uint32_t TimerIndex;
> > +uint32_t Reserved;
> > +uint64_t ExpirationTime;
> > +uint64_t DeliveryTime;
> > +} payload = {
> > +.TimerIndex = index,
> > +.ExpirationTime = expiration,
> > +.DeliveryTime = delivery,
> > +};
> > +
> > +if ( test_bit(sintx, &v->arch.hvm.viridian->msg_pending) )
> > +return false;
> > +
> > +BUILD_BUG_ON(sizeof(*msg) != HV_MESSAGE_SIZE);
> > +msg += sintx;
> > +
> > +/*
> > + * To avoid using an atomic test-and-set this function must be called
> > + * in context of the vcpu receiving the message.
> > + */
> > +ASSERT(v == current);
> > +if ( msg->Header.MessageType != HvMessageTypeNone )
> > +{
> > +msg->Header.MessageFlags.MessagePending = 1;
> > +set_bit(sintx, &v->arch.hvm.viridian->msg_pending);
> 
> As per the comment above this is always in context of the subject
> vCPU. It looks to me as if this was also the case for the two
> clear_bit() on the field in the prior patch. If so, all three could be
> the non-atomic variants instead.

The only slight subtlety I think is the one in the wrmsr function, which can be 
called in context of a domain restore. I think it's still ok for it to be 
non-atomic in this case but I'll assert (v = current || !v->running), which I 
think should cover it.

> 
> > +return false;
> > +}
> > +
> > +msg->Header.MessageType = HvMessageTimerExpired;
> > +msg->Header.MessageFlags.MessagePending = 0;
> > +msg->Header.PayloadSize = sizeof(payload);
> > +memcpy(msg->Payload, &payload, sizeof(payload));
> > +
> > +if ( !vs->fields.mask )
> > +vlapic_set_irq(vcpu_vlapic(v), vs->fields.vector, 0);
> 
> If this wasn't with v == current, I think you'd also need a barrier
> here. Could you extend the comment above to also mention this
> aspect?

Ok.

> 
> > @@ -118,14 +119,237 @@ static int64_t time_ref_count(struct domain *d)
> >  return raw_trc_val(d) + trc->off;
> >  }
> >
> > +static int64_t time_now(struct domain *d)
> 
> Why would this return a signed value? And can't the function
> parameter be const?

The function parameter can be const, but I think the result needs to be signed 
for the missed ticks logic in start_timer() to work correctly.

> 
> > +{
> > +const struct viridian_page *rt = &d->arch.hvm.viridian->reference_tsc;
> > +HV_REFERENCE_TSC_PAGE *p = rt->ptr;
> > +uint32_t start, end;
> > +__int128_t tsc;
> > +__int128_t scale;
> 
> I don't think you need both of them be 128 bits wide. I also don't
> see why either would want to be of a signed type.

The spec says (as in the comment below):

"The partition reference time is computed by the following formula:

ReferenceTime = ((VirtualTsc * TscScale) >> 64) + TscOffset

The multiplication is a 64 bit multiplication, which results in a 128 bit 
number which is then shifted 64 times to the right to obtain the high 64 
bits.TscScale"

Again, I'm using signed arithmetic as I think it's necessary for the missed 
ticks logic to work correctly in the event of an overflow.

> 
> > +int64_t offset;
> > +
> > +/*
> > + * If the reference TSC page is not enabled, or has been invalidated
> > + * fall back to the partition reference counter.
> > + */
> > +if ( !p || !p->TscSequence )
> > +return time_ref_count(d);
> > +
> > +/*
> > + * The following sampling algorithm for tsc, scale and offset is
> > + * documented in the specifiction.
> > + */
> > +start = p->TscSequence;
> > +
> > +do {
> > +tsc = rdtsc();
> > +scale = p->TscScale;
> > +offset = p->TscOffset;
> > +
> > +smp_mb();
> > +end = p->TscSequence;
> 
> Why is this a full barrier, rather than just 

Re: [Xen-devel] [PATCH 0/6] iomem cacheability

2019-03-06 Thread Amit Tomer
Hi
> Thanks for testing! You might have found a real bug in the series. Could
> you please also attach the full device tree?

Please find the attached DTS and DTB file used for testing.

Thanks
-Amit


r8a7795-h3ulcb.dts
Description: audio/vnd.dts


r8a7795-h3ulcb.dtb
Description: Binary data
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next RFC 3/4] pygrub: convert python files with 2to3

2019-03-06 Thread Wei Liu
On Tue, Mar 05, 2019 at 05:51:04PM +, Andrew Cooper wrote:
> On 05/03/2019 16:42, Wei Liu wrote:
> > Signed-off-by: Wei Liu 
> > ---
> > Not sure this works with python 2.4, but it should work with 2.7 since
> > the changes look more or less in the same vein as the changes in
> > libxl.
> >
> > The conversion of the import is interesting. This definitely needs
> > some testing.
> > ---
> >  tools/pygrub/src/ExtLinuxConf.py | 16 
> >  tools/pygrub/src/GrubConf.py | 36 ++--
> >  tools/pygrub/src/LiloConf.py | 16 
> >  3 files changed, 34 insertions(+), 34 deletions(-)
> >
> > diff --git a/tools/pygrub/src/ExtLinuxConf.py 
> > b/tools/pygrub/src/ExtLinuxConf.py
> > index d1789bf020..60da960c4b 100644
> > --- a/tools/pygrub/src/ExtLinuxConf.py
> > +++ b/tools/pygrub/src/ExtLinuxConf.py
> > @@ -12,7 +12,7 @@
> >  
> >  import sys, re, os
> >  import logging
> > -import GrubConf
> > +from . import GrubConf
> 
> Relative imports definitely don't exist in Py 2.4
> 
> >  
> >  class ExtLinuxImage(object):
> >  def __init__(self, lines, path):
> > @@ -32,7 +32,7 @@ class ExtLinuxImage(object):
> >  self.lines = []
> >  self.path = path
> >  self.root = ""
> > -map(self.set_from_line, lines)
> > +list(map(self.set_from_line, lines))
> 
> This an abuse of map() in the first place, but the automatic
> transformation makes the result even more confusing.

Right. I tried to find the justification for this transformation but the
document doesn't provide that.

I will drop this and the relative import -- I _think_ the original code
should still work with python 3.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next RFC 3/4] pygrub: convert python files with 2to3

2019-03-06 Thread Andrew Cooper
On 06/03/2019 11:31, Wei Liu wrote:
> On Tue, Mar 05, 2019 at 05:51:04PM +, Andrew Cooper wrote:
>> On 05/03/2019 16:42, Wei Liu wrote:
>>> Signed-off-by: Wei Liu 
>>> ---
>>> Not sure this works with python 2.4, but it should work with 2.7 since
>>> the changes look more or less in the same vein as the changes in
>>> libxl.
>>>
>>> The conversion of the import is interesting. This definitely needs
>>> some testing.
>>> ---
>>>  tools/pygrub/src/ExtLinuxConf.py | 16 
>>>  tools/pygrub/src/GrubConf.py | 36 ++--
>>>  tools/pygrub/src/LiloConf.py | 16 
>>>  3 files changed, 34 insertions(+), 34 deletions(-)
>>>
>>> diff --git a/tools/pygrub/src/ExtLinuxConf.py 
>>> b/tools/pygrub/src/ExtLinuxConf.py
>>> index d1789bf020..60da960c4b 100644
>>> --- a/tools/pygrub/src/ExtLinuxConf.py
>>> +++ b/tools/pygrub/src/ExtLinuxConf.py
>>> @@ -12,7 +12,7 @@
>>>  
>>>  import sys, re, os
>>>  import logging
>>> -import GrubConf
>>> +from . import GrubConf
>> Relative imports definitely don't exist in Py 2.4
>>
>>>  
>>>  class ExtLinuxImage(object):
>>>  def __init__(self, lines, path):
>>> @@ -32,7 +32,7 @@ class ExtLinuxImage(object):
>>>  self.lines = []
>>>  self.path = path
>>>  self.root = ""
>>> -map(self.set_from_line, lines)
>>> +list(map(self.set_from_line, lines))
>> This an abuse of map() in the first place, but the automatic
>> transformation makes the result even more confusing.
> Right. I tried to find the justification for this transformation but the
> document doesn't provide that.

The expected use of map is in the form:

x = map(fn, y)

which would leave x as a list in Py2, and a generator in Py3.

In most code, wrapping map with list() is the correct transformation to
make, because a) a lot of code written for Py2 expects it to be a list
and b) you cant programmatically evaluate whether leaving it in its
generator form is safe in context.

For this piece of code (and the other similar examples), map() is not
the correct construct to use in the first place, and probably wants
fixing for clarity alone, irrespective of the Py3 transformation.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 0/6] iomem cacheability

2019-03-06 Thread Amit Tomer
Hi,
> Looking at the stack trace, this is very likely due to the issue I pointed 
> out earlier on. I.e reserved-memory region should be described in the memory 
> nodes.

Do you mean, something like this(reserved-memory node is under memory node) ?

--- a/arch/arm64/boot/dts/renesas/r8a7795-h3ulcb.dts
+++ b/arch/arm64/boot/dts/renesas/r8a7795-h3ulcb.dts
@@ -17,23 +17,10 @@
memory@4800 {
device_type = "memory";
/* first 128MB is reserved for secure area. */
-   reg = <0x0 0x4800 0x0 0x3800>;
-   };
-
-   memory@5 {
-   device_type = "memory";
-   reg = <0x5 0x 0x0 0x4000>;
-   };
-
-   memory@6 {
-   device_type = "memory";
-   reg = <0x6 0x 0x0 0x4000>;
-   };
-
-   memory@7 {
-   device_type = "memory";
-   reg = <0x7 0x 0x0 0x4000>;
-   };
+  reg = <0x0 0x4800 0x0 0x3800>,
+<0x5 0x 0x0 0x4000>,
+<0x6 0x 0x0 0x4000>,
+<0x7 0x 0x0 0x4000>;

reserved-memory {
#address-cells = <2>;
@@ -61,6 +48,7 @@
reg = <0x 0x7000 0x0 0x1000>;
};
};
+   };

Thanks
-Amit

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 8/9] viridian: add implementation of synthetic timers

2019-03-06 Thread Paul Durrant
> -Original Message-
> From: Xen-devel [mailto:xen-devel-boun...@lists.xenproject.org] On Behalf Of 
> Paul Durrant
> Sent: 06 March 2019 11:24
> To: 'Jan Beulich' 
> Cc: Stefano Stabellini ; Wei Liu 
> ; Konrad Rzeszutek Wilk
> ; Andrew Cooper ; Tim 
> (Xen.org) ;
> George Dunlap ; Julien Grall 
> ; xen-devel  de...@lists.xenproject.org>; Ian Jackson ; Roger Pau 
> Monne
> 
> Subject: Re: [Xen-devel] [PATCH v3 8/9] viridian: add implementation of 
> synthetic timers
> 
> > -Original Message-
> > From: Jan Beulich [mailto:jbeul...@suse.com]
> > Sent: 25 February 2019 14:54
> > To: Paul Durrant 
> > Cc: Julien Grall ; Andrew Cooper 
> > ; Roger Pau Monne
> > ; Wei Liu ; George Dunlap 
> > ; Ian
> > Jackson ; Stefano Stabellini 
> > ; xen-devel  > de...@lists.xenproject.org>; Konrad Rzeszutek Wilk 
> > ; Tim (Xen.org)
> > 
> > Subject: Re: [PATCH v3 8/9] viridian: add implementation of synthetic timers
> >
> > >>> On 31.01.19 at 11:47,  wrote:
> > > --- a/xen/arch/x86/hvm/viridian/synic.c
> > > +++ b/xen/arch/x86/hvm/viridian/synic.c
> > > @@ -329,7 +329,53 @@ void viridian_synic_domain_deinit(struct domain *d)
> > >
> > >  void viridian_synic_poll_messages(struct vcpu *v)
> > >  {
> > > -/* There are currently no message sources */
> > > +viridian_time_poll_timers(v);
> > > +}
> > > +
> > > +bool viridian_synic_deliver_timer_msg(struct vcpu *v, unsigned int sintx,
> > > +  unsigned int index,
> > > +  int64_t expiration, int64_t 
> > > delivery)
> > > +{
> > > +const union viridian_sint_msr *vs = 
> > > &v->arch.hvm.viridian->sint[sintx];
> > > +HV_MESSAGE *msg = v->arch.hvm.viridian->simp.ptr;
> > > +struct {
> > > +uint32_t TimerIndex;
> > > +uint32_t Reserved;
> > > +uint64_t ExpirationTime;
> > > +uint64_t DeliveryTime;
> > > +} payload = {
> > > +.TimerIndex = index,
> > > +.ExpirationTime = expiration,
> > > +.DeliveryTime = delivery,
> > > +};
> > > +
> > > +if ( test_bit(sintx, &v->arch.hvm.viridian->msg_pending) )
> > > +return false;
> > > +
> > > +BUILD_BUG_ON(sizeof(*msg) != HV_MESSAGE_SIZE);
> > > +msg += sintx;
> > > +
> > > +/*
> > > + * To avoid using an atomic test-and-set this function must be called
> > > + * in context of the vcpu receiving the message.
> > > + */
> > > +ASSERT(v == current);
> > > +if ( msg->Header.MessageType != HvMessageTypeNone )
> > > +{
> > > +msg->Header.MessageFlags.MessagePending = 1;
> > > +set_bit(sintx, &v->arch.hvm.viridian->msg_pending);
> >
> > As per the comment above this is always in context of the subject
> > vCPU. It looks to me as if this was also the case for the two
> > clear_bit() on the field in the prior patch. If so, all three could be
> > the non-atomic variants instead.
> 
> The only slight subtlety I think is the one in the wrmsr function, which can 
> be called in context of a
> domain restore. I think it's still ok for it to be non-atomic in this case 
> but I'll assert (v =
> current || !v->running), which I think should cover it.
> 
> >
> > > +return false;
> > > +}
> > > +
> > > +msg->Header.MessageType = HvMessageTimerExpired;
> > > +msg->Header.MessageFlags.MessagePending = 0;
> > > +msg->Header.PayloadSize = sizeof(payload);
> > > +memcpy(msg->Payload, &payload, sizeof(payload));
> > > +
> > > +if ( !vs->fields.mask )
> > > +vlapic_set_irq(vcpu_vlapic(v), vs->fields.vector, 0);
> >
> > If this wasn't with v == current, I think you'd also need a barrier
> > here. Could you extend the comment above to also mention this
> > aspect?
> 
> Ok.
> 
> >
> > > @@ -118,14 +119,237 @@ static int64_t time_ref_count(struct domain *d)
> > >  return raw_trc_val(d) + trc->off;
> > >  }
> > >
> > > +static int64_t time_now(struct domain *d)
> >
> > Why would this return a signed value? And can't the function
> > parameter be const?
> 
> The function parameter can be const, but I think the result needs to be 
> signed for the missed ticks
> logic in start_timer() to work correctly.
> 
> >
> > > +{
> > > +const struct viridian_page *rt = 
> > > &d->arch.hvm.viridian->reference_tsc;
> > > +HV_REFERENCE_TSC_PAGE *p = rt->ptr;
> > > +uint32_t start, end;
> > > +__int128_t tsc;
> > > +__int128_t scale;
> >
> > I don't think you need both of them be 128 bits wide. I also don't
> > see why either would want to be of a signed type.
> 
> The spec says (as in the comment below):
> 
> "The partition reference time is computed by the following formula:
> 
> ReferenceTime = ((VirtualTsc * TscScale) >> 64) + TscOffset
> 
> The multiplication is a 64 bit multiplication, which results in a 128 bit 
> number which is then shifted
> 64 times to the right to obtain the high 64 bits.TscScale"
> 
> Again, I'm using signed arithmetic as I think it's necessary for the missed 
> ti

Re: [Xen-devel] Can someone pls repair patchwork?

2019-03-06 Thread Oleksandr Andrushchenko

+webmas...@kernel.org

Hi, there!

Could you please fix wrong mailing list for Xen project at [2]?
The correct mailing list now lives at "xen-devel@lists.xenproject.org"

Thank you in advance,
Oleksandr

On 3/6/19 1:17 PM, Julien Grall wrote:

(+ Lars)

On 06/03/2019 10:04, Oleksandr Andrushchenko wrote:

To whom it may concern


Hi Oleksandr,



Since late 2017 the very useful Patchwork resource [1]
stopped working after (as I assume) Xen-devel list has changed
its address from xen-de...@lists.xen.org to the current one.
Patchwork is still configured to the old one, so recent
patches are not archived.
Could the respective owner from Xen community please take a look at [2]
and make Patchwork work again? In particular Patchwork is
very useful when you need a patch in mbox format without pain.


Patchwork is hosted by the kernel community. So it would be best if 
you contact them directly. [3].



Ah, I thought that somebody from Xen community has admin rights

Cheers,


[1] https://patchwork.kernel.org/project/xen-devel/list/
[2] https://patchwork.kernel.org/project/xen-devel/


[3] https://www.kernel.org/category/contact-us.html




___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH RESEND 3/3] OvmfPkg/XenSupport: turn off address decoding before BAR sizing

2019-03-06 Thread Igor Druzhinin
On Xen, hvmloader firmware leaves address decoding enabled for
enumerated PCI device before jumping into OVMF. OVMF seems to
expect it to be disabled and tries to size PCI BARs in several places
without disabling it which causes BAR64, for example, being
incorrectly placed by QEMU.

Fix it by disabling PCI address decoding explicitly before the
first attempt to size BARs on Xen.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Igor Druzhinin 
---
 OvmfPkg/Library/PciHostBridgeLib/XenSupport.c | 34 +++
 1 file changed, 34 insertions(+)

diff --git a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c 
b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
index 408fb24..9940335 100644
--- a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
+++ b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
@@ -55,6 +55,33 @@ PcatPciRootBridgeBarExisted (
   EnableInterrupts ();
 }
 
+#define EFI_PCI_COMMAND_DECODE ((UINT16)(EFI_PCI_COMMAND_IO_SPACE | \
+ EFI_PCI_COMMAND_MEMORY_SPACE))
+STATIC
+VOID
+PcatPciRootBridgeDecoding (
+  IN  UINTN  Address,
+  IN  BOOLEANEnabled,
+  OUT UINT16 *OriginalValue
+  )
+{
+  UINT16Value;
+
+  //
+  // Preserve the original value
+  //
+  Value = *OriginalValue;
+  *OriginalValue = PciRead16 (Address);
+
+  if (!Enabled) {
+if (*OriginalValue & EFI_PCI_COMMAND_DECODE)
+   PciWrite16 (Address, *OriginalValue & ~EFI_PCI_COMMAND_DECODE);
+  } else {
+if (Value & EFI_PCI_COMMAND_DECODE)
+  PciWrite16 (Address, Value);
+  }
+}
+
 STATIC
 VOID
 PcatPciRootBridgeParseBars (
@@ -76,6 +103,7 @@ PcatPciRootBridgeParseBars (
   UINT32Value;
   UINT32OriginalUpperValue;
   UINT32UpperValue;
+  UINT16OriginalCommand;
   UINT64Mask;
   UINTN Offset;
   UINT64Base;
@@ -83,6 +111,12 @@ PcatPciRootBridgeParseBars (
   UINT64Limit;
   PCI_ROOT_BRIDGE_APERTURE  *MemAperture;
 
+  // Disable address decoding for every device before OVMF starts sizing it
+  PcatPciRootBridgeDecoding (
+PCI_LIB_ADDRESS (Bus, Device, Function, PCI_COMMAND_OFFSET),
+FALSE, &OriginalCommand
+  );
+
   for (Offset = BarOffsetBase; Offset < BarOffsetEnd; Offset += sizeof 
(UINT32)) {
 PcatPciRootBridgeBarExisted (
   PCI_LIB_ADDRESS (Bus, Device, Function, Offset),
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH RESEND 2/3] OvmfPkg/XenSupport: use a correct PCI host bridge aperture for BAR64

2019-03-06 Thread Igor Druzhinin
In case BAR64 is placed below 4G choose the correct aperture.
This fixes a failed assertion down the code path.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Igor Druzhinin 
---
 OvmfPkg/Library/PciHostBridgeLib/XenSupport.c | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c 
b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
index c23c46d..408fb24 100644
--- a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
+++ b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
@@ -145,7 +145,11 @@ PcatPciRootBridgeParseBars (
   Length = Length | LShiftU64 ((UINT64) UpperValue, 32);
   Length = (~Length) + 1;
 
-  MemAperture = MemAbove4G;
+  if (Base < 0x1) {
+MemAperture = Mem;
+  } else {
+MemAperture = MemAbove4G;
+  }
 }
 
 Limit = Base + Length - 1;
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH RESEND 1/3] OvmfPkg/XenSupport: remove usage of prefetchable PCI host bridge aperture

2019-03-06 Thread Igor Druzhinin
This aperture doesn't exist in OVMF and trying to use it causes
failing assertions later in cases there are prefetchable and
non-prefetchable BARs following each other. This configuration is
quite likely with some PCI passthrough devices.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Igor Druzhinin 
---
 OvmfPkg/Library/PciHostBridgeLib/XenSupport.c | 12 ++--
 1 file changed, 2 insertions(+), 10 deletions(-)

diff --git a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c 
b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
index 9204179..c23c46d 100644
--- a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
+++ b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
@@ -129,11 +129,7 @@ PcatPciRootBridgeParseBars (
   //
   Length = ((~Length) + 1) & 0x;
 
-  if ((Value & BIT3) == BIT3) {
-MemAperture = PMem;
-  } else {
-MemAperture = Mem;
-  }
+  MemAperture = Mem;
 } else {
   //
   // 64bit
@@ -149,11 +145,7 @@ PcatPciRootBridgeParseBars (
   Length = Length | LShiftU64 ((UINT64) UpperValue, 32);
   Length = (~Length) + 1;
 
-  if ((Value & BIT3) == BIT3) {
-MemAperture = PMemAbove4G;
-  } else {
-MemAperture = MemAbove4G;
-  }
+  MemAperture = MemAbove4G;
 }
 
 Limit = Base + Length - 1;
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH RESEND 0/3] Xen PCI passthrough fixes

2019-03-06 Thread Igor Druzhinin
Resend to include xen-devel@lists.xenproject.org to CC

Igor Druzhinin (3):
  OvmfPkg/XenSupport: remove usage of prefetchable PCI host bridge
aperture
  OvmfPkg/XenSupport: use a correct PCI host bridge aperture for BAR64
  OvmfPkg/XenSupport: turn off address decoding before BAR sizing

 OvmfPkg/Library/PciHostBridgeLib/XenSupport.c | 44 ++-
 1 file changed, 37 insertions(+), 7 deletions(-)

-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next RFC 4/4] pygrub: make it build with python 3

2019-03-06 Thread Wei Liu
On Tue, Mar 05, 2019 at 07:17:07PM +0100, Marek Marczykowski-Górecki wrote:
> On Tue, Mar 05, 2019 at 05:48:10PM +, Wei Liu wrote:
> > On Tue, Mar 05, 2019 at 05:42:07PM +, Andrew Cooper wrote:
> > > On 05/03/2019 16:42, Wei Liu wrote:
> > > > With the help of two porting guides and cpython source code:
> > > >
> > > > 1. Use PyUnicode to replace PyString counterparts.
> > > > 2. Use PyVarObject_HEAD_INIT and provide compatibility for 2.5 and
> > > >earlier.
> > > > 3. Remove usage of Py_FindMethod.
> > > > 4. Use new module initialisation routine.
> > > >
> > > > For #3, Py_FindMethod was removed, yet an alternative wasn't
> > > > documented.  The code is the result of reverse-engineering cpython
> > > > commit 6116d4a1d1
> > > >
> > > > https://docs.python.org/3/howto/cporting.html
> > > > http://python3porting.com/cextensions.html
> > > >
> > > > Signed-off-by: Wei Liu 
> > > 
> > > Marek already made the tools/python/* libraries compatible with Py2 and 
> > > Py3
> > > 
> > > The following commits are the relevant ones:
> > > 
> > > * be6b316 - python: handle long type in scripts (2 years ago)  > > Marczykowski-Górecki>
> > > * e16c705 - python: adjust module initalization for Python3 (2 years ago) 
> > > 
> > > * dd986cd - python: use PyLong_* for constructing 'int' type in Python3 
> > > (2 years ago) 
> > > * 121d9d4 - python: use PyBytes/PyUnicode instead of PyString (2 years 
> > > ago) 
> > > * 0c8981f - python: initialize specific fields of PyTypeObject (2 years 
> > > ago) 
> > > * 7b1e5f7 - python: use Py_TYPE instead of looking directly into 
> > > PyObject_HEAD (2 years ago) 
> > > * 96d1ee6 - python: drop tp_getattr implementation (2 years ago)  > > Marczykowski-Górecki>
> > > * 6b28df3 - python: check return value of PyErr_NewException (2 years 
> > > ago) 
> > 
> > I knew.
> > 
> > > 
> > > Which in particular handle strings differently in the Py2 case.
> > 
> > 
> > I am not sure his changes for the string APIs are correct -- they seem
> > to deviate from the official porting guide. But hey, I don't use these
> > bindings myself, so he probably knows better.
> 
> That was intentional, because in py2 str type is the same as bytes
> types, and in fact some of those str should really be bytes. It's in the
> commit message:
> 
> python: use PyBytes/PyUnicode instead of PyString
> 
> In Python2 PyBytes is the same as PyString, but in Python3 PyString is
> gone and 'str' is really PyUnicode in C-API.
> When handling arbitrary data, use PyBytes - which is the right thing to
> do in Python3, and pose no API change in Python2. When handling
> xenstore paths and transaction ids, which have well defined format, use
> PyUnicode - to ease API usage - no need to prefix all xenstore paths
> with 'b' when migrating scripts to Python3.
> 
> I'm not sure if the same reasoning applies to pygrub, but I guess it
> may. For example fsimage_file_read sounds like handling binary data, not
> really UTF-8 strings. Using PyUnicode for arbitrary binary data may lead
> to various UnicodeDecodeErrors.

Good point. Let me dig into this a bit more. Frankly I trust you more
than I trust myself on this matter. :-)

Wei.

> 
> -- 
> Best Regards,
> Marek Marczykowski-Górecki
> Invisible Things Lab
> A: Because it messes up the order in which people normally read text.
> Q: Why is top-posting such a bad thing?



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next RFC 3/4] pygrub: convert python files with 2to3

2019-03-06 Thread Wei Liu
On Wed, Mar 06, 2019 at 11:46:24AM +, Andrew Cooper wrote:
> On 06/03/2019 11:31, Wei Liu wrote:
> > On Tue, Mar 05, 2019 at 05:51:04PM +, Andrew Cooper wrote:
> >> On 05/03/2019 16:42, Wei Liu wrote:
> >>> Signed-off-by: Wei Liu 
> >>> ---
> >>> Not sure this works with python 2.4, but it should work with 2.7 since
> >>> the changes look more or less in the same vein as the changes in
> >>> libxl.
> >>>
> >>> The conversion of the import is interesting. This definitely needs
> >>> some testing.
> >>> ---
> >>>  tools/pygrub/src/ExtLinuxConf.py | 16 
> >>>  tools/pygrub/src/GrubConf.py | 36 
> >>> ++--
> >>>  tools/pygrub/src/LiloConf.py | 16 
> >>>  3 files changed, 34 insertions(+), 34 deletions(-)
> >>>
> >>> diff --git a/tools/pygrub/src/ExtLinuxConf.py 
> >>> b/tools/pygrub/src/ExtLinuxConf.py
> >>> index d1789bf020..60da960c4b 100644
> >>> --- a/tools/pygrub/src/ExtLinuxConf.py
> >>> +++ b/tools/pygrub/src/ExtLinuxConf.py
> >>> @@ -12,7 +12,7 @@
> >>>  
> >>>  import sys, re, os
> >>>  import logging
> >>> -import GrubConf
> >>> +from . import GrubConf
> >> Relative imports definitely don't exist in Py 2.4
> >>
> >>>  
> >>>  class ExtLinuxImage(object):
> >>>  def __init__(self, lines, path):
> >>> @@ -32,7 +32,7 @@ class ExtLinuxImage(object):
> >>>  self.lines = []
> >>>  self.path = path
> >>>  self.root = ""
> >>> -map(self.set_from_line, lines)
> >>> +list(map(self.set_from_line, lines))
> >> This an abuse of map() in the first place, but the automatic
> >> transformation makes the result even more confusing.
> > Right. I tried to find the justification for this transformation but the
> > document doesn't provide that.
> 
> The expected use of map is in the form:
> 
> x = map(fn, y)
> 
> which would leave x as a list in Py2, and a generator in Py3.

Oh, so we should indeed force it to evaluate.

> 
> In most code, wrapping map with list() is the correct transformation to
> make, because a) a lot of code written for Py2 expects it to be a list
> and b) you cant programmatically evaluate whether leaving it in its
> generator form is safe in context.
> 
> For this piece of code (and the other similar examples), map() is not
> the correct construct to use in the first place, and probably wants
> fixing for clarity alone, irrespective of the Py3 transformation.
> 

Sure.

Wei.

> ~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 8/9] viridian: add implementation of synthetic timers

2019-03-06 Thread Jan Beulich
>>> On 06.03.19 at 12:23,  wrote:
>> From: Jan Beulich [mailto:jbeul...@suse.com]
>> Sent: 25 February 2019 14:54
>> 
>> >>> On 31.01.19 at 11:47,  wrote:
>> > @@ -118,14 +119,237 @@ static int64_t time_ref_count(struct domain *d)
>> >  return raw_trc_val(d) + trc->off;
>> >  }
>> >
>> > +static int64_t time_now(struct domain *d)
>> 
>> Why would this return a signed value? And can't the function
>> parameter be const?
> 
> The function parameter can be const, but I think the result needs to be 
> signed for the missed ticks logic in start_timer() to work correctly.

If something requires signed arithmetic, then this should be enforced
there, not by the return type of an otherwise sufficiently generic
function. Then again NOW() also produces a signed value ...

>> > +{
>> > +const struct viridian_page *rt = &d->arch.hvm.viridian->reference_tsc;
>> > +HV_REFERENCE_TSC_PAGE *p = rt->ptr;
>> > +uint32_t start, end;
>> > +__int128_t tsc;
>> > +__int128_t scale;
>> 
>> I don't think you need both of them be 128 bits wide. I also don't
>> see why either would want to be of a signed type.
> 
> The spec says (as in the comment below):
> 
> "The partition reference time is computed by the following formula:
> 
> ReferenceTime = ((VirtualTsc * TscScale) >> 64) + TscOffset
> 
> The multiplication is a 64 bit multiplication, which results in a 128 bit 
> number which is then shifted 64 times to the right to obtain the high 64 
> bits.TscScale"

Well, yes, you want a 128-bit result. But for that you don't need to
multiply 128-bit quantities. See e.g. our own scale_delta() or
hvm_scale_tsc().

>> > +case HV_X64_MSR_STIMER0_CONFIG:
>> > +case HV_X64_MSR_STIMER1_CONFIG:
>> > +case HV_X64_MSR_STIMER2_CONFIG:
>> > +case HV_X64_MSR_STIMER3_CONFIG:
>> > +{
>> > +unsigned int stimerx = (idx - HV_X64_MSR_STIMER0_CONFIG) / 2;
>> > +struct viridian_stimer *vs = 
>> > &v->arch.hvm.viridian->stimer[stimerx];
>> > +
>> > +if ( !(viridian_feature_mask(d) & HVMPV_stimer) )
>> > +return X86EMUL_EXCEPTION;
>> > +
>> > +stop_stimer(vs);
>> > +
>> > +vs->config.raw = val;
>> > +
>> > +if ( !vs->config.fields.sintx )
>> > +vs->config.fields.enabled = 0;
>> > +
>> > +if ( vs->config.fields.enabled )
>> > +start_stimer(vs);
>> > +
>> > +break;
>> > +}
>> > +case HV_X64_MSR_STIMER0_COUNT:
>> 
>> Missing blank line again (and also further down here as well as in the
>> rdmsr code).
>> 
> 
> Ok. TBH I've always thought the normal style was to omit the blank line if 
> the case statement has braces.

Not sure about this specific sub-case.

>> > @@ -201,6 +476,32 @@ int viridian_time_rdmsr(const struct vcpu *v, 
>> > uint32_t idx, uint64_t *val)
>> >  break;
>> >  }
>> >
>> > +case HV_X64_MSR_STIMER0_CONFIG:
>> > +case HV_X64_MSR_STIMER1_CONFIG:
>> > +case HV_X64_MSR_STIMER2_CONFIG:
>> > +case HV_X64_MSR_STIMER3_CONFIG:
>> > +{
>> > +unsigned int stimerx = (idx - HV_X64_MSR_STIMER0_CONFIG) / 2;
>> > +
>> > +if ( !(viridian_feature_mask(d) & HVMPV_stimer) )
>> > +return X86EMUL_EXCEPTION;
>> > +
>> > +*val = v->arch.hvm.viridian->stimer[stimerx].config.raw;
>> 
>> While more noticeable here and ...
>> 
>> > +break;
>> > +}
>> > +case HV_X64_MSR_STIMER0_COUNT:
>> > +case HV_X64_MSR_STIMER1_COUNT:
>> > +case HV_X64_MSR_STIMER2_COUNT:
>> > +case HV_X64_MSR_STIMER3_COUNT:
>> > +{
>> > +unsigned int stimerx = (idx - HV_X64_MSR_STIMER0_COUNT) / 2;
>> > +
>> > +if ( !(viridian_feature_mask(d) & HVMPV_stimer) )
>> > +return X86EMUL_EXCEPTION;
>> > +
>> > +*val = v->arch.hvm.viridian->stimer[stimerx].count;
>> 
>> ... here, array_access_nospec() are probably needed not just here,
>> but also in the wrmsr logic.
> 
> Really? stimerx is calculated based on hitting the case statement in the 
> first place.

And any of the branches of the switch() can be (mis)speculated into.

>> > --- a/xen/include/asm-x86/hvm/viridian.h
>> > +++ b/xen/include/asm-x86/hvm/viridian.h
>> > @@ -40,6 +40,33 @@ union viridian_sint_msr
>> >  } fields;
>> >  };
>> >
>> > +union viridian_stimer_config_msr
>> > +{
>> > +uint64_t raw;
>> > +struct
>> > +{
>> > +uint64_t enabled:1;
>> > +uint64_t periodic:1;
>> > +uint64_t lazy:1;
>> > +uint64_t auto_enable:1;
>> > +uint64_t vector:8;
>> > +uint64_t direct_mode:1;
>> > +uint64_t reserved_zero1:3;
>> > +uint64_t sintx:4;
>> > +uint64_t reserved_zero2:44;
>> > +} fields;
>> > +};
>> > +
>> > +struct viridian_stimer {
>> > +struct vcpu *v;
>> 
>> Isn't a full 8-byte pointer a little too much overhead here? You could
>> instead store the timer index ...
> 
> I think I need it in stimer_expire() which can be called in any vcpu context 
> IIUC.
> 
>> 
>> > +stru

[Xen-devel] [PATCH v1 3/4] libxl: add libxl_get_parameters() function

2019-03-06 Thread Vasilis Liaskovitis
Add a new libxl function to get hypervisor parameters at runtime.

Signed-off-by: Vasilis Liaskovitis 
---
 tools/libxl/libxl.c | 15 +++
 tools/libxl/libxl.h |  1 +
 2 files changed, 16 insertions(+)

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index ec71574e99..124033e5a3 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -669,6 +669,21 @@ int libxl_set_parameters(libxl_ctx *ctx, char *params)
 return 0;
 }
 
+int libxl_get_parameters(libxl_ctx *ctx, char *params, char *values)
+{
+int ret;
+GC_INIT(ctx);
+
+ret = xc_get_parameters(ctx->xch, params, values);
+if (ret < 0) {
+LOGEV(ERROR, ret, "getting parameters");
+GC_FREE;
+return ret;//ERROR_FAIL;
+}
+GC_FREE;
+return 0;
+}
+
 static int fd_set_flags(libxl_ctx *ctx, int fd,
 int fcntlgetop, int fcntlsetop, const char *fl,
 int flagmask, int set_p)
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index a38e5cdba2..360a757a06 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -2307,6 +2307,7 @@ int libxl_send_trigger(libxl_ctx *ctx, uint32_t domid,
 int libxl_send_sysrq(libxl_ctx *ctx, uint32_t domid, char sysrq);
 int libxl_send_debug_keys(libxl_ctx *ctx, char *keys);
 int libxl_set_parameters(libxl_ctx *ctx, char *params);
+int libxl_get_parameters(libxl_ctx *ctx, char *params, char *values);
 
 typedef struct libxl__xen_console_reader libxl_xen_console_reader;
 
-- 
2.20.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v1 2/4] libxc: add function to get hypervisor parameters

2019-03-06 Thread Vasilis Liaskovitis
Add a new libxc function to get hypervisor parameters at runtime.

Signed-off-by: Vasilis Liaskovitis 
---
 tools/libxc/include/xenctrl.h |  1 +
 tools/libxc/xc_misc.c | 26 ++
 2 files changed, 27 insertions(+)

diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 31cdda76c6..281185063d 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -1228,6 +1228,7 @@ int xc_readconsolering(xc_interface *xch,
 
 int xc_send_debug_keys(xc_interface *xch, char *keys);
 int xc_set_parameters(xc_interface *xch, char *params);
+int xc_get_parameters(xc_interface *xch, char *params, char *values);
 
 typedef struct xen_sysctl_physinfo xc_physinfo_t;
 typedef struct xen_sysctl_cputopo xc_cputopo_t;
diff --git a/tools/libxc/xc_misc.c b/tools/libxc/xc_misc.c
index 5e6714ae2b..439ad91194 100644
--- a/tools/libxc/xc_misc.c
+++ b/tools/libxc/xc_misc.c
@@ -208,6 +208,32 @@ int xc_set_parameters(xc_interface *xch, char *params)
 return ret;
 }
 
+int xc_get_parameters(xc_interface *xch, char *params, char *values)
+{
+int ret, len = strlen(params);
+DECLARE_SYSCTL;
+DECLARE_HYPERCALL_BOUNCE(params, len, XC_HYPERCALL_BUFFER_BOUNCE_IN);
+DECLARE_HYPERCALL_BOUNCE(values, 1023, XC_HYPERCALL_BUFFER_BOUNCE_OUT);
+
+if ( xc_hypercall_bounce_pre(xch, params) )
+return -1;
+if ( xc_hypercall_bounce_pre(xch, values) )
+return -1;
+
+sysctl.cmd = XEN_SYSCTL_get_parameter;
+set_xen_guest_handle(sysctl.u.get_parameter.params, params);
+set_xen_guest_handle(sysctl.u.get_parameter.values, values);
+sysctl.u.get_parameter.size = len;
+memset(sysctl.u.get_parameter.pad, 0, sizeof(sysctl.u.get_parameter.pad));
+
+ret = do_sysctl(xch, &sysctl);
+
+xc_hypercall_bounce_post(xch, params);
+xc_hypercall_bounce_post(xch, values);
+
+return ret;
+}
+
 int xc_physinfo(xc_interface *xch,
 xc_physinfo_t *put_info)
 {
-- 
2.20.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v1 4/4] xl: add new xl command get-parameters

2019-03-06 Thread Vasilis Liaskovitis
Add a new xl command "get-parameters" to get hypervisor parameters at
runtime.

Signed-off-by: Vasilis Liaskovitis 
---
 docs/man/xl.1.pod.in   |  5 +
 tools/xl/xl.h  |  1 +
 tools/xl/xl_cmdtable.c |  5 +
 tools/xl/xl_misc.c | 25 +
 4 files changed, 36 insertions(+)

diff --git a/docs/man/xl.1.pod.in b/docs/man/xl.1.pod.in
index 4310fcd818..a1fff4d382 100644
--- a/docs/man/xl.1.pod.in
+++ b/docs/man/xl.1.pod.in
@@ -827,6 +827,11 @@ Send debug I to Xen. It is the same as pressing the 
Xen
 Set hypervisor parameters as specified in I. This allows for some
 boot parameters of the hypervisor to be modified in the running systems.
 
+=item B I
+
+Get hypervisor parameters as specified in I. This allows for some
+boot parameters of the hypervisor to be read in the running systems.
+
 =item B [I]
 
 Reads the Xen message buffer, similar to dmesg on a Linux system.  The
diff --git a/tools/xl/xl.h b/tools/xl/xl.h
index cf4202bc89..af3843e5b0 100644
--- a/tools/xl/xl.h
+++ b/tools/xl/xl.h
@@ -219,6 +219,7 @@ int main_psr_mba_set(int argc, char **argv);
 int main_psr_mba_show(int argc, char **argv);
 #endif
 int main_qemu_monitor_command(int argc, char **argv);
+int main_get_parameters(int argc, char **argv);
 
 void help(const char *command);
 
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 89716badcb..a18481619b 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -662,6 +662,11 @@ struct cmd_spec cmd_table[] = {
   "Issue a qemu monitor command to the device model of a domain",
   " ",
 },
+{ "get-parameters",
+  &main_get_parameters, 0, 1,
+  "Get hypervisor parameters",
+  "",
+},
 };
 
 int cmdtable_len = sizeof(cmd_table)/sizeof(struct cmd_spec);
diff --git a/tools/xl/xl_misc.c b/tools/xl/xl_misc.c
index dcf940a6d4..811f231b78 100644
--- a/tools/xl/xl_misc.c
+++ b/tools/xl/xl_misc.c
@@ -364,6 +364,31 @@ int main_config_update(int argc, char **argv)
 return 0;
 }
 
+int main_get_parameters(int argc, char **argv)
+{
+int opt, ret;
+char *params;
+char values[1023];
+
+SWITCH_FOREACH_OPT(opt, "", NULL, "get-parameters", 1) {
+/* No options */
+}
+
+params = argv[optind];
+
+if (!params) {
+   fprintf(stderr, "no parameter specified\n");
+   return EXIT_FAILURE;
+}
+else if ((ret = libxl_get_parameters(ctx, params, values))) {
+fprintf(stderr, "cannot get parameters: %s : %d\n", params, ret);
+fprintf(stderr, "Use \"xl dmesg\" to look for possible reason.\n");
+return EXIT_FAILURE;
+}
+fprintf(stderr, "%s : %s\n", params, values);
+
+return EXIT_SUCCESS;
+}
 /*
  * Local variables:
  * mode: C
-- 
2.20.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v1 0/4] Support for reading hypervisor parameters at runtime

2019-03-06 Thread Vasilis Liaskovitis
Currently parameters of the hypervisor cannot be inspected at runtime
through an xl command. Such a command would be a useful diagnostic
tool e.g. used in conjunction with the "xl set-parameters" command.

This patch series implements a new xl command "xl get-parameters"
which takes a string of input parameters and returns their current
values in the hypervisor settings.

Examples:

xl get-parameters "gnttab_max_frames gnttab_max_maptrack_frames"
gnttab_max_frames gnttab_max_maptrack_frames : 64 1024

xl set-parameters gnttab_max_frames=128

xl get-parameters gnttab_max_frames
gnttab_max_frames : 128

xl get-parameters "gnttab_max_frames gnttab_max_maptrack_frames"
gnttab_max_frames gnttab_max_maptrack_frames : 128 1024

Vasilis Liaskovitis (4):
  xen: add hypercall for getting parameters at runtime
  libxc: add function to get hypervisor parameters
  libxl: add libxl_get_parameters() function
  xl: add new xl command set-parameters

 docs/man/xl.1.pod.in|   5 ++
 tools/flask/policy/modules/dom0.te  |   2 +-
 tools/libxc/include/xenctrl.h   |   1 +
 tools/libxc/xc_misc.c   |  26 +++
 tools/libxl/libxl.c |  15 
 tools/libxl/libxl.h |   1 +
 tools/xl/xl.h   |   1 +
 tools/xl/xl_cmdtable.c  |   5 ++
 tools/xl/xl_misc.c  |  25 +++
 xen/common/kernel.c | 109 
 xen/common/sysctl.c |  45 
 xen/include/public/sysctl.h |  18 +
 xen/include/xen/lib.h   |   1 +
 xen/xsm/flask/hooks.c   |   3 +
 xen/xsm/flask/policy/access_vectors |   2 +
 15 files changed, 258 insertions(+), 1 deletion(-)

-- 
2.20.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v1 1/4] xen: add hypercall for getting parameters at runtime

2019-03-06 Thread Vasilis Liaskovitis
Add a sysctl hypercall to support getting hypervisor parameters
at runtime.

Signed-off-by: Vasilis Liaskovitis 
---
 tools/flask/policy/modules/dom0.te  |   2 +-
 xen/common/kernel.c | 109 
 xen/common/sysctl.c |  45 
 xen/include/public/sysctl.h |  18 +
 xen/include/xen/lib.h   |   1 +
 xen/xsm/flask/hooks.c   |   3 +
 xen/xsm/flask/policy/access_vectors |   2 +
 7 files changed, 179 insertions(+), 1 deletion(-)

diff --git a/tools/flask/policy/modules/dom0.te 
b/tools/flask/policy/modules/dom0.te
index a347d664f8..681d1a101b 100644
--- a/tools/flask/policy/modules/dom0.te
+++ b/tools/flask/policy/modules/dom0.te
@@ -16,7 +16,7 @@ allow dom0_t xen_t:xen {
 allow dom0_t xen_t:xen2 {
resource_op psr_cmt_op psr_alloc pmu_ctrl get_symbol
get_cpu_levelling_caps get_cpu_featureset livepatch_op
-   coverage_op set_parameter
+   coverage_op set_parameter get_parameter
 };
 
 # Allow dom0 to use all XENVER_ subops that have checks.
diff --git a/xen/common/kernel.c b/xen/common/kernel.c
index 612575430f..83225afd93 100644
--- a/xen/common/kernel.c
+++ b/xen/common/kernel.c
@@ -52,6 +52,110 @@ static int assign_integer_param(const struct kernel_param 
*param, uint64_t val)
 return 0;
 }
 
+static int get_integer_param(const struct kernel_param *param, uint64_t *val)
+{
+switch ( param->len )
+{
+case sizeof(uint8_t):
+*val = *(uint8_t *)param->par.var;
+break;
+case sizeof(uint16_t):
+*val = *(uint16_t *)param->par.var;
+break;
+case sizeof(uint32_t):
+*val = *(uint32_t *)param->par.var;
+break;
+case sizeof(uint64_t):
+*val = *(uint64_t *)param->par.var;
+break;
+default:
+BUG();
+}
+
+return 0;
+}
+
+static int get_params(const char *cmdline, char *values,
+  const struct kernel_param *start,
+  const struct kernel_param *end)
+{
+char opt[128], *optkey, *q;
+const char *p = cmdline, *val = values;
+const struct kernel_param *param;
+int len, rc = 0;
+uint64_t param_int;
+bool found;
+
+if (!values)
+return -EFAULT;
+
+for ( ; ; )
+{
+/* Skip whitespace. */
+while ( *p == ' ' )
+p++;
+if ( *p == '\0' )
+break;
+
+/* Grab the next whitespace-delimited option. */
+q = optkey = opt;
+while ( (*p != ' ') && (*p != '\0') )
+{
+if ( (q-opt) < (sizeof(opt)-1) ) /* avoid overflow */
+*q++ = *p;
+p++;
+}
+*q = '\0';
+
+/* Boolean parameters can be inverted with 'no-' prefix. */
+
+found = false;
+for ( param = start; param < end; param++ )
+{
+
+if ( strcmp(param->name, optkey) )
+continue;
+
+found = true;
+switch ( param->type )
+{
+case OPT_STR:
+len = snprintf((char*)val, sizeof(values), "%s ",
+   (char*)param->par.var);
+val += len;
+break;
+case OPT_UINT:
+get_integer_param(param, ¶m_int);
+len = snprintf((char*)val, sizeof(values), "%lu ", param_int);
+val += len;
+break;
+case OPT_BOOL:
+get_integer_param(param, ¶m_int);
+len = snprintf((char*)val, sizeof(values), "%s",
+   param_int ? "true" : "false");
+val += len;
+break;
+case OPT_SIZE:
+case OPT_CUSTOM:
+rc = -EINVAL;
+break;
+default:
+BUG();
+break;
+}
+}
+
+if ( !found )
+{
+printk("get-parameters: parameter \"%s\" unknown!\n", optkey);
+rc = -EINVAL;
+}
+}
+*val = '\0';
+
+return rc;
+}
+
 static int parse_params(const char *cmdline, const struct kernel_param *start,
 const struct kernel_param *end)
 {
@@ -199,6 +303,11 @@ int runtime_parse(const char *line)
 return parse_params(line, __param_start, __param_end);
 }
 
+int runtime_get_parameter(const char *line, char *values)
+{
+return get_params(line, values, __param_start, __param_end);
+}
+
 /**
  *cmdline_parse -- parses the xen command line.
  * If CONFIG_CMDLINE is set, it would be parsed prior to @cmdline.
diff --git a/xen/common/sysctl.c b/xen/common/sysctl.c
index c0aa6bde4e..d8597de9cd 100644
--- a/xen/common/sysctl.c
+++ b/xen/common/sysctl.c
@@ -501,6 +501,51 @@ long do_sysctl(XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) 
u_sysctl)
 
 break;
 }
+case XEN_SYSCTL_get_parameter:
+{
+#define XEN_GET_PARAMETER_MAX_SIZE 1023
+char *params;
+char 

Re: [Xen-devel] [PATCH v3 8/9] viridian: add implementation of synthetic timers

2019-03-06 Thread Jan Beulich
>>> On 06.03.19 at 12:47,  wrote:
>> From: Xen-devel [mailto:xen-devel-boun...@lists.xenproject.org] On Behalf Of 
>> Paul Durrant
>> Sent: 06 March 2019 11:24
>> 
>> > -Original Message-
>> > From: Jan Beulich [mailto:jbeul...@suse.com]
>> > Sent: 25 February 2019 14:54
>> >
>> > >>> On 31.01.19 at 11:47,  wrote:
>> > > @@ -118,14 +119,237 @@ static int64_t time_ref_count(struct domain *d)
>> > >  return raw_trc_val(d) + trc->off;
>> > >  }
>> > >
>> > > +static int64_t time_now(struct domain *d)
>> > > +{
>> > > +const struct viridian_page *rt = 
>> > > &d->arch.hvm.viridian->reference_tsc;
>> > > +HV_REFERENCE_TSC_PAGE *p = rt->ptr;
>> > > +uint32_t start, end;
>> > > +__int128_t tsc;
>> > > +__int128_t scale;
>> >
>> > I don't think you need both of them be 128 bits wide. I also don't
>> > see why either would want to be of a signed type.
>> 
>> The spec says (as in the comment below):
>> 
>> "The partition reference time is computed by the following formula:
>> 
>> ReferenceTime = ((VirtualTsc * TscScale) >> 64) + TscOffset
>> 
>> The multiplication is a 64 bit multiplication, which results in a 128 bit 
>> number which is then shifted
>> 64 times to the right to obtain the high 64 bits.TscScale"
>> 
>> Again, I'm using signed arithmetic as I think it's necessary for the missed 
>> ticks logic to work
>> correctly in the event of an overflow.
> 
> FAOD the code that I am concerned about is:
> 
> /*
>  * The timer is already started, so we're re-scheduling.
>  * Hence advance the timer expiration by one tick.
>  */
> next = vs->expiration + vs->count;
> 
> /* Now check to see if any expirations have been missed */
> if ( now - next > 0 )
> missed = (now - next) / vs->count;
> 
> If now and next were unsigned then next may overflow such that (now - next) 
> ends up being very large, rather than negative, so I'd end up calculating a 
> completely bogus value for missed.

And this is also what I've been referring to: If signedness matters, there
should be a cast here rather than enforcing signedness onto everyone by
a function logically never returning a signed value.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 8/9] viridian: add implementation of synthetic timers

2019-03-06 Thread Paul Durrant
> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 06 March 2019 13:00
> To: Paul Durrant 
> Cc: Julien Grall ; Andrew Cooper 
> ; George Dunlap
> ; Ian Jackson ; Roger Pau 
> Monne
> ; Wei Liu ; Stefano Stabellini 
> ;
> xen-devel ; Konrad Rzeszutek Wilk 
> ; Tim
> (Xen.org) 
> Subject: RE: [PATCH v3 8/9] viridian: add implementation of synthetic timers
> 
> >>> On 06.03.19 at 12:47,  wrote:
> >> From: Xen-devel [mailto:xen-devel-boun...@lists.xenproject.org] On Behalf 
> >> Of Paul Durrant
> >> Sent: 06 March 2019 11:24
> >>
> >> > -Original Message-
> >> > From: Jan Beulich [mailto:jbeul...@suse.com]
> >> > Sent: 25 February 2019 14:54
> >> >
> >> > >>> On 31.01.19 at 11:47,  wrote:
> >> > > @@ -118,14 +119,237 @@ static int64_t time_ref_count(struct domain *d)
> >> > >  return raw_trc_val(d) + trc->off;
> >> > >  }
> >> > >
> >> > > +static int64_t time_now(struct domain *d)
> >> > > +{
> >> > > +const struct viridian_page *rt = 
> >> > > &d->arch.hvm.viridian->reference_tsc;
> >> > > +HV_REFERENCE_TSC_PAGE *p = rt->ptr;
> >> > > +uint32_t start, end;
> >> > > +__int128_t tsc;
> >> > > +__int128_t scale;
> >> >
> >> > I don't think you need both of them be 128 bits wide. I also don't
> >> > see why either would want to be of a signed type.
> >>
> >> The spec says (as in the comment below):
> >>
> >> "The partition reference time is computed by the following formula:
> >>
> >> ReferenceTime = ((VirtualTsc * TscScale) >> 64) + TscOffset
> >>
> >> The multiplication is a 64 bit multiplication, which results in a 128 bit 
> >> number which is then
> shifted
> >> 64 times to the right to obtain the high 64 bits.TscScale"
> >>
> >> Again, I'm using signed arithmetic as I think it's necessary for the 
> >> missed ticks logic to work
> >> correctly in the event of an overflow.
> >
> > FAOD the code that I am concerned about is:
> >
> > /*
> >  * The timer is already started, so we're re-scheduling.
> >  * Hence advance the timer expiration by one tick.
> >  */
> > next = vs->expiration + vs->count;
> >
> > /* Now check to see if any expirations have been missed */
> > if ( now - next > 0 )
> > missed = (now - next) / vs->count;
> >
> > If now and next were unsigned then next may overflow such that (now - next)
> > ends up being very large, rather than negative, so I'd end up calculating a
> > completely bogus value for missed.
> 
> And this is also what I've been referring to: If signedness matters, there
> should be a cast here rather than enforcing signedness onto everyone by
> a function logically never returning a signed value.

Ok, I'll redefine the function to return an unsigned value and leave now and 
next as int64_t.

  Paul

> 
> Jan
> 


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 8/9] viridian: add implementation of synthetic timers

2019-03-06 Thread Paul Durrant
> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 06 March 2019 12:57
> To: Paul Durrant 
> Cc: Julien Grall ; Andrew Cooper 
> ; George Dunlap
> ; Ian Jackson ; Roger Pau 
> Monne
> ; Wei Liu ; Stefano Stabellini 
> ;
> xen-devel ; Konrad Rzeszutek Wilk 
> ; Tim
> (Xen.org) 
> Subject: RE: [PATCH v3 8/9] viridian: add implementation of synthetic timers
> 
> >>> On 06.03.19 at 12:23,  wrote:
> >> From: Jan Beulich [mailto:jbeul...@suse.com]
> >> Sent: 25 February 2019 14:54
> >>
> >> >>> On 31.01.19 at 11:47,  wrote:
> >> > @@ -118,14 +119,237 @@ static int64_t time_ref_count(struct domain *d)
> >> >  return raw_trc_val(d) + trc->off;
> >> >  }
> >> >
> >> > +static int64_t time_now(struct domain *d)
> >>
> >> Why would this return a signed value? And can't the function
> >> parameter be const?
> >
> > The function parameter can be const, but I think the result needs to be
> > signed for the missed ticks logic in start_timer() to work correctly.
> 
> If something requires signed arithmetic, then this should be enforced
> there, not by the return type of an otherwise sufficiently generic
> function. Then again NOW() also produces a signed value ...
> 
> >> > +{
> >> > +const struct viridian_page *rt = 
> >> > &d->arch.hvm.viridian->reference_tsc;
> >> > +HV_REFERENCE_TSC_PAGE *p = rt->ptr;
> >> > +uint32_t start, end;
> >> > +__int128_t tsc;
> >> > +__int128_t scale;
> >>
> >> I don't think you need both of them be 128 bits wide. I also don't
> >> see why either would want to be of a signed type.
> >
> > The spec says (as in the comment below):
> >
> > "The partition reference time is computed by the following formula:
> >
> > ReferenceTime = ((VirtualTsc * TscScale) >> 64) + TscOffset
> >
> > The multiplication is a 64 bit multiplication, which results in a 128 bit
> > number which is then shifted 64 times to the right to obtain the high 64
> > bits.TscScale"
> 
> Well, yes, you want a 128-bit result. But for that you don't need to
> multiply 128-bit quantities. See e.g. our own scale_delta() or
> hvm_scale_tsc().

Testing showed that by not casting first things were broken. I assumed this was 
because the result of the multiplication was being truncated to 64-bits before 
assignment, but I can check the generated code. I'll also have a look at the 
examples you cite.

  Paul


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 8/9] viridian: add implementation of synthetic timers

2019-03-06 Thread Paul Durrant
> -Original Message-
> From: Paul Durrant
> Sent: 06 March 2019 13:06
> To: 'Jan Beulich' 
> Cc: Julien Grall ; Andrew Cooper 
> ; George Dunlap
> ; Ian Jackson ; Roger Pau 
> Monne
> ; Wei Liu ; Stefano Stabellini 
> ;
> xen-devel ; Konrad Rzeszutek Wilk 
> ; Tim
> (Xen.org) 
> Subject: RE: [PATCH v3 8/9] viridian: add implementation of synthetic timers
> 
> > -Original Message-
> > From: Jan Beulich [mailto:jbeul...@suse.com]
> > Sent: 06 March 2019 12:57
> > To: Paul Durrant 
> > Cc: Julien Grall ; Andrew Cooper 
> > ; George Dunlap
> > ; Ian Jackson ; Roger Pau 
> > Monne
> > ; Wei Liu ; Stefano Stabellini 
> > ;
> > xen-devel ; Konrad Rzeszutek Wilk 
> > ; Tim
> > (Xen.org) 
> > Subject: RE: [PATCH v3 8/9] viridian: add implementation of synthetic timers
> >
> > >>> On 06.03.19 at 12:23,  wrote:
> > >> From: Jan Beulich [mailto:jbeul...@suse.com]
> > >> Sent: 25 February 2019 14:54
> > >>
> > >> >>> On 31.01.19 at 11:47,  wrote:
> > >> > @@ -118,14 +119,237 @@ static int64_t time_ref_count(struct domain *d)
> > >> >  return raw_trc_val(d) + trc->off;
> > >> >  }
> > >> >
> > >> > +static int64_t time_now(struct domain *d)
> > >>
> > >> Why would this return a signed value? And can't the function
> > >> parameter be const?
> > >
> > > The function parameter can be const, but I think the result needs to be
> > > signed for the missed ticks logic in start_timer() to work correctly.
> >
> > If something requires signed arithmetic, then this should be enforced
> > there, not by the return type of an otherwise sufficiently generic
> > function. Then again NOW() also produces a signed value ...
> >
> > >> > +{
> > >> > +const struct viridian_page *rt = 
> > >> > &d->arch.hvm.viridian->reference_tsc;
> > >> > +HV_REFERENCE_TSC_PAGE *p = rt->ptr;
> > >> > +uint32_t start, end;
> > >> > +__int128_t tsc;
> > >> > +__int128_t scale;
> > >>
> > >> I don't think you need both of them be 128 bits wide. I also don't
> > >> see why either would want to be of a signed type.
> > >
> > > The spec says (as in the comment below):
> > >
> > > "The partition reference time is computed by the following formula:
> > >
> > > ReferenceTime = ((VirtualTsc * TscScale) >> 64) + TscOffset
> > >
> > > The multiplication is a 64 bit multiplication, which results in a 128 bit
> > > number which is then shifted 64 times to the right to obtain the high 64
> > > bits.TscScale"
> >
> > Well, yes, you want a 128-bit result. But for that you don't need to
> > multiply 128-bit quantities. See e.g. our own scale_delta() or
> > hvm_scale_tsc().
> 
> Testing showed that by not casting first things were broken. I assumed this 
> was because the result of
> the multiplication was being truncated to 64-bits before assignment, but I 
> can check the generated
> code. I'll also have a look at the examples you cite.

Both those examples do the multiplication by inline asm (presumably to avoid 
truncation). Is that what you'd prefer?

  Paul

> 
>   Paul


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH RESEND 3/3] OvmfPkg/XenSupport: turn off address decoding before BAR sizing

2019-03-06 Thread Laszlo Ersek
On 03/06/19 13:40, Igor Druzhinin wrote:
> On Xen, hvmloader firmware leaves address decoding enabled for
> enumerated PCI device before jumping into OVMF. OVMF seems to
> expect it to be disabled and tries to size PCI BARs in several places
> without disabling it which causes BAR64, for example, being
> incorrectly placed by QEMU.
> 
> Fix it by disabling PCI address decoding explicitly before the
> first attempt to size BARs on Xen.
> 
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Igor Druzhinin 
> ---
>  OvmfPkg/Library/PciHostBridgeLib/XenSupport.c | 34 
> +++
>  1 file changed, 34 insertions(+)
> 
> diff --git a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c 
> b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
> index 408fb24..9940335 100644
> --- a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
> +++ b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
> @@ -55,6 +55,33 @@ PcatPciRootBridgeBarExisted (
>EnableInterrupts ();
>  }
>  
> +#define EFI_PCI_COMMAND_DECODE ((UINT16)(EFI_PCI_COMMAND_IO_SPACE | \
> + EFI_PCI_COMMAND_MEMORY_SPACE))

I thought I asked you not to define a macro here that started with the
"EFI_" prefix :/

http://mid.mail-archive.com/dd8e3c9e-cb76-d3fe-6a10-c0a41c714b56@redhat.com

Laszlo

> +STATIC
> +VOID
> +PcatPciRootBridgeDecoding (
> +  IN  UINTN  Address,
> +  IN  BOOLEANEnabled,
> +  OUT UINT16 *OriginalValue
> +  )
> +{
> +  UINT16Value;
> +
> +  //
> +  // Preserve the original value
> +  //
> +  Value = *OriginalValue;
> +  *OriginalValue = PciRead16 (Address);
> +
> +  if (!Enabled) {
> +if (*OriginalValue & EFI_PCI_COMMAND_DECODE)
> +   PciWrite16 (Address, *OriginalValue & ~EFI_PCI_COMMAND_DECODE);
> +  } else {
> +if (Value & EFI_PCI_COMMAND_DECODE)
> +  PciWrite16 (Address, Value);
> +  }
> +}
> +
>  STATIC
>  VOID
>  PcatPciRootBridgeParseBars (
> @@ -76,6 +103,7 @@ PcatPciRootBridgeParseBars (
>UINT32Value;
>UINT32OriginalUpperValue;
>UINT32UpperValue;
> +  UINT16OriginalCommand;
>UINT64Mask;
>UINTN Offset;
>UINT64Base;
> @@ -83,6 +111,12 @@ PcatPciRootBridgeParseBars (
>UINT64Limit;
>PCI_ROOT_BRIDGE_APERTURE  *MemAperture;
>  
> +  // Disable address decoding for every device before OVMF starts sizing it
> +  PcatPciRootBridgeDecoding (
> +PCI_LIB_ADDRESS (Bus, Device, Function, PCI_COMMAND_OFFSET),
> +FALSE, &OriginalCommand
> +  );
> +
>for (Offset = BarOffsetBase; Offset < BarOffsetEnd; Offset += sizeof 
> (UINT32)) {
>  PcatPciRootBridgeBarExisted (
>PCI_LIB_ADDRESS (Bus, Device, Function, Offset),
> 


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-4.10-testing test] 133594: regressions - FAIL

2019-03-06 Thread osstest service owner
flight 133594 xen-4.10-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133594/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-ovmf-amd64 15 guest-saverestore.2 fail REGR. vs. 
133487
 test-armhf-armhf-xl   5 host-ping-check-native   fail REGR. vs. 133487

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-win7-amd64 13 guest-saverestore  fail like 133359
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop fail like 133487
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  7842419a6b85edb4a5b9bee8b1179de4c8b84b60
baseline version:
 xen  a016b8f207c7a3fe8bdd2b6f7c080020e3e1c823

Last test of basis   133487  2019-02-28 18:48:38 Z5 days
Testing same since   133594  2019-03-05 14:36:48 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Jan Beulich 
  Juergen Gross 
  Manuel Bouyer 
  Sergey Dyasli 

jobs:
 bu

Re: [Xen-devel] [PATCH for-next RFC 0/4] tools: Python 3 compatibility

2019-03-06 Thread George Dunlap
On 3/5/19 4:42 PM, Wei Liu wrote:
> This series makes tools build with Python 3.
> 
> Compile test only with 2.7 and 3.5 thus far, hence the RFC. This should be 
> able
> to give people some idea what sort of work is involved.
> 
> You will also need Andrew's "tools/xen-foreign: Update python scripts to be
> Py3 compatible".

Tossing this out there: Given that python2 is (theoretically) EOL in
less than a year, is it worth maintaining compatibility with python2, or
would it be better to just go whole-hog into python3?

 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next RFC 0/4] tools: Python 3 compatibility

2019-03-06 Thread Wei Liu
On Wed, Mar 06, 2019 at 01:49:18PM +, George Dunlap wrote:
> On 3/5/19 4:42 PM, Wei Liu wrote:
> > This series makes tools build with Python 3.
> > 
> > Compile test only with 2.7 and 3.5 thus far, hence the RFC. This should be 
> > able
> > to give people some idea what sort of work is involved.
> > 
> > You will also need Andrew's "tools/xen-foreign: Update python scripts to be
> > Py3 compatible".
> 
> Tossing this out there: Given that python2 is (theoretically) EOL in
> less than a year, is it worth maintaining compatibility with python2, or
> would it be better to just go whole-hog into python3?

Some enterprise-y distros still ship ancient python AFAICT.

Given that the work involved seems to be manageable so far I would say
let's keep the compatibility for now.

Wei.

> 
>  -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next RFC 0/4] tools: Python 3 compatibility

2019-03-06 Thread George Dunlap
On 3/6/19 1:49 PM, George Dunlap wrote:
> On 3/5/19 4:42 PM, Wei Liu wrote:
>> This series makes tools build with Python 3.
>>
>> Compile test only with 2.7 and 3.5 thus far, hence the RFC. This should be 
>> able
>> to give people some idea what sort of work is involved.
>>
>> You will also need Andrew's "tools/xen-foreign: Update python scripts to be
>> Py3 compatible".
> 
> Tossing this out there: Given that python2 is (theoretically) EOL in
> less than a year, is it worth maintaining compatibility with python2, or
> would it be better to just go whole-hog into python3?

I mean -- looking at some of the discussions about how differently
certain kinds of things are interpreted between python2 and python3, it
seems a bit mad to try to write code that works in both: we're just
asking for trouble.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next RFC 0/4] tools: Python 3 compatibility

2019-03-06 Thread Wei Liu
On Wed, Mar 06, 2019 at 01:53:01PM +, George Dunlap wrote:
> On 3/6/19 1:49 PM, George Dunlap wrote:
> > On 3/5/19 4:42 PM, Wei Liu wrote:
> >> This series makes tools build with Python 3.
> >>
> >> Compile test only with 2.7 and 3.5 thus far, hence the RFC. This should be 
> >> able
> >> to give people some idea what sort of work is involved.
> >>
> >> You will also need Andrew's "tools/xen-foreign: Update python scripts to be
> >> Py3 compatible".
> > 
> > Tossing this out there: Given that python2 is (theoretically) EOL in
> > less than a year, is it worth maintaining compatibility with python2, or
> > would it be better to just go whole-hog into python3?
> 
> I mean -- looking at some of the discussions about how differently
> certain kinds of things are interpreted between python2 and python3, it
> seems a bit mad to try to write code that works in both: we're just
> asking for trouble.

I think Python is not very consistent even within a major version. New
syntax and constructs get added to point releases. Some of the
discussions re code isn't about the differences between 2 and 3.  There
is at least one about 2.4 vs 2.7.

I am all for moving away from 2.4.

Wei.

> 
>  -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next RFC 0/4] tools: Python 3 compatibility

2019-03-06 Thread Andrew Cooper
On 06/03/2019 13:49, George Dunlap wrote:
> On 3/5/19 4:42 PM, Wei Liu wrote:
>> This series makes tools build with Python 3.
>>
>> Compile test only with 2.7 and 3.5 thus far, hence the RFC. This should be 
>> able
>> to give people some idea what sort of work is involved.
>>
>> You will also need Andrew's "tools/xen-foreign: Update python scripts to be
>> Py3 compatible".
> Tossing this out there: Given that python2 is (theoretically) EOL in
> less than a year, is it worth maintaining compatibility with python2, or
> would it be better to just go whole-hog into python3?

Not when we've got supported distros which only ship Py2 by default. 
(CentOS 6 - I'm looking at you in particular...)

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next RFC 0/4] tools: Python 3 compatibility

2019-03-06 Thread Anthony PERARD
On Wed, Mar 06, 2019 at 01:57:55PM +, Wei Liu wrote:
> On Wed, Mar 06, 2019 at 01:53:01PM +, George Dunlap wrote:
> > On 3/6/19 1:49 PM, George Dunlap wrote:
> > > On 3/5/19 4:42 PM, Wei Liu wrote:
> > >> This series makes tools build with Python 3.
> > >>
> > >> Compile test only with 2.7 and 3.5 thus far, hence the RFC. This should 
> > >> be able
> > >> to give people some idea what sort of work is involved.
> > >>
> > >> You will also need Andrew's "tools/xen-foreign: Update python scripts to 
> > >> be
> > >> Py3 compatible".
> > > 
> > > Tossing this out there: Given that python2 is (theoretically) EOL in
> > > less than a year, is it worth maintaining compatibility with python2, or
> > > would it be better to just go whole-hog into python3?
> > 
> > I mean -- looking at some of the discussions about how differently
> > certain kinds of things are interpreted between python2 and python3, it
> > seems a bit mad to try to write code that works in both: we're just
> > asking for trouble.
> 
> I think Python is not very consistent even within a major version. New
> syntax and constructs get added to point releases. Some of the
> discussions re code isn't about the differences between 2 and 3.  There
> is at least one about 2.4 vs 2.7.

I think it's because they backport syntax changes from 3.X to 2.6 and
2.7. Also all the _future_X imports. All of that makes it easier to have
scripts compatible with both 2.7 and 3.X.

> I am all for moving away from 2.4.

Moving to a min of 2.6 would be nice. CentOS 6 comes it.

-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next RFC 0/4] tools: Python 3 compatibility

2019-03-06 Thread Wei Liu
On Wed, Mar 06, 2019 at 02:17:31PM +, Anthony PERARD wrote:
> On Wed, Mar 06, 2019 at 01:57:55PM +, Wei Liu wrote:
> > On Wed, Mar 06, 2019 at 01:53:01PM +, George Dunlap wrote:
> > > On 3/6/19 1:49 PM, George Dunlap wrote:
> > > > On 3/5/19 4:42 PM, Wei Liu wrote:
> > > >> This series makes tools build with Python 3.
> > > >>
> > > >> Compile test only with 2.7 and 3.5 thus far, hence the RFC. This 
> > > >> should be able
> > > >> to give people some idea what sort of work is involved.
> > > >>
> > > >> You will also need Andrew's "tools/xen-foreign: Update python scripts 
> > > >> to be
> > > >> Py3 compatible".
> > > > 
> > > > Tossing this out there: Given that python2 is (theoretically) EOL in
> > > > less than a year, is it worth maintaining compatibility with python2, or
> > > > would it be better to just go whole-hog into python3?
> > > 
> > > I mean -- looking at some of the discussions about how differently
> > > certain kinds of things are interpreted between python2 and python3, it
> > > seems a bit mad to try to write code that works in both: we're just
> > > asking for trouble.
> > 
> > I think Python is not very consistent even within a major version. New
> > syntax and constructs get added to point releases. Some of the
> > discussions re code isn't about the differences between 2 and 3.  There
> > is at least one about 2.4 vs 2.7.
> 
> I think it's because they backport syntax changes from 3.X to 2.6 and
> 2.7. Also all the _future_X imports. All of that makes it easier to have
> scripts compatible with both 2.7 and 3.X.
> 
> > I am all for moving away from 2.4.
> 
> Moving to a min of 2.6 would be nice. CentOS 6 comes it.

My next version will have a patch to do that. Any objection?

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH RESEND 3/3] OvmfPkg/XenSupport: turn off address decoding before BAR sizing

2019-03-06 Thread Igor Druzhinin
On 06/03/2019 13:22, Laszlo Ersek wrote:
> On 03/06/19 13:40, Igor Druzhinin wrote:
>> On Xen, hvmloader firmware leaves address decoding enabled for
>> enumerated PCI device before jumping into OVMF. OVMF seems to
>> expect it to be disabled and tries to size PCI BARs in several places
>> without disabling it which causes BAR64, for example, being
>> incorrectly placed by QEMU.
>>
>> Fix it by disabling PCI address decoding explicitly before the
>> first attempt to size BARs on Xen.
>>
>> Contributed-under: TianoCore Contribution Agreement 1.1
>> Signed-off-by: Igor Druzhinin 
>> ---
>>  OvmfPkg/Library/PciHostBridgeLib/XenSupport.c | 34 
>> +++
>>  1 file changed, 34 insertions(+)
>>
>> diff --git a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c 
>> b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
>> index 408fb24..9940335 100644
>> --- a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
>> +++ b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
>> @@ -55,6 +55,33 @@ PcatPciRootBridgeBarExisted (
>>EnableInterrupts ();
>>  }
>>  
>> +#define EFI_PCI_COMMAND_DECODE ((UINT16)(EFI_PCI_COMMAND_IO_SPACE | \
>> + EFI_PCI_COMMAND_MEMORY_SPACE))
> 
> I thought I asked you not to define a macro here that started with the
> "EFI_" prefix :/
> 
> http://mid.mail-archive.com/dd8e3c9e-cb76-d3fe-6a10-c0a41c714b56@redhat.com
> 

This is a resend of v1 patch series to get Xen folks into CC and gather
comments. I expect v2.

Igor

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-4.11-testing test] 133595: regressions - FAIL

2019-03-06 Thread osstest service owner
flight 133595 xen-4.11-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133595/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-credit1   7 xen-boot fail REGR. vs. 132938

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass

version targeted for testing:
 xen  e984846dad81218bbd8cbaec6df8e8a3530726dc
baseline version:
 xen  87f51bf366ca79b98e1e201bf9bd7a9c164631e2

Last test of basis   132938  2019-02-05 12:06:45 Z   29 days
Testing same since   133595  2019-03-05 14:36:57 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Jan Beulich 
  Juergen Gross 
  Manuel Bouyer 
  Sergey Dyasli 

jobs:
 build-

Re: [Xen-devel] [PATCH v3 8/9] viridian: add implementation of synthetic timers

2019-03-06 Thread Jan Beulich
>>> On 06.03.19 at 14:09,  wrote:
>> From: Paul Durrant
>> Sent: 06 March 2019 13:06
>> 
>> > From: Jan Beulich [mailto:jbeul...@suse.com]
>> > Sent: 06 March 2019 12:57
>> > Subject: RE: [PATCH v3 8/9] viridian: add implementation of synthetic 
>> > timers
>> >
>> > >>> On 06.03.19 at 12:23,  wrote:
>> > >> From: Jan Beulich [mailto:jbeul...@suse.com]
>> > >> Sent: 25 February 2019 14:54
>> > >>
>> > >> >>> On 31.01.19 at 11:47,  wrote:
>> > >> > +{
>> > >> > +const struct viridian_page *rt = 
>> > >> > &d->arch.hvm.viridian->reference_tsc;
>> > >> > +HV_REFERENCE_TSC_PAGE *p = rt->ptr;
>> > >> > +uint32_t start, end;
>> > >> > +__int128_t tsc;
>> > >> > +__int128_t scale;
>> > >>
>> > >> I don't think you need both of them be 128 bits wide. I also don't
>> > >> see why either would want to be of a signed type.
>> > >
>> > > The spec says (as in the comment below):
>> > >
>> > > "The partition reference time is computed by the following formula:
>> > >
>> > > ReferenceTime = ((VirtualTsc * TscScale) >> 64) + TscOffset
>> > >
>> > > The multiplication is a 64 bit multiplication, which results in a 128 bit
>> > > number which is then shifted 64 times to the right to obtain the high 64
>> > > bits.TscScale"
>> >
>> > Well, yes, you want a 128-bit result. But for that you don't need to
>> > multiply 128-bit quantities. See e.g. our own scale_delta() or
>> > hvm_scale_tsc().
>> 
>> Testing showed that by not casting first things were broken. I assumed this 
>> was because the result of
>> the multiplication was being truncated to 64-bits before assignment, but I 
>> can check the generated
>> code. I'll also have a look at the examples you cite.
> 
> Both those examples do the multiplication by inline asm (presumably to avoid 
> truncation). Is that what you'd prefer?

That would imo be best, not the least because of making us independent
of possible issues with 128-bit arithmetic on older compilers.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 8/9] viridian: add implementation of synthetic timers

2019-03-06 Thread Paul Durrant
> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 06 March 2019 14:38
> To: Paul Durrant 
> Cc: Julien Grall ; Andrew Cooper 
> ; George Dunlap
> ; Ian Jackson ; Roger Pau 
> Monne
> ; Wei Liu ; Stefano Stabellini 
> ;
> xen-devel ; Konrad Rzeszutek Wilk 
> ; Tim
> (Xen.org) 
> Subject: RE: [PATCH v3 8/9] viridian: add implementation of synthetic timers
> 
> >>> On 06.03.19 at 14:09,  wrote:
> >> From: Paul Durrant
> >> Sent: 06 March 2019 13:06
> >>
> >> > From: Jan Beulich [mailto:jbeul...@suse.com]
> >> > Sent: 06 March 2019 12:57
> >> > Subject: RE: [PATCH v3 8/9] viridian: add implementation of synthetic 
> >> > timers
> >> >
> >> > >>> On 06.03.19 at 12:23,  wrote:
> >> > >> From: Jan Beulich [mailto:jbeul...@suse.com]
> >> > >> Sent: 25 February 2019 14:54
> >> > >>
> >> > >> >>> On 31.01.19 at 11:47,  wrote:
> >> > >> > +{
> >> > >> > +const struct viridian_page *rt = 
> >> > >> > &d->arch.hvm.viridian->reference_tsc;
> >> > >> > +HV_REFERENCE_TSC_PAGE *p = rt->ptr;
> >> > >> > +uint32_t start, end;
> >> > >> > +__int128_t tsc;
> >> > >> > +__int128_t scale;
> >> > >>
> >> > >> I don't think you need both of them be 128 bits wide. I also don't
> >> > >> see why either would want to be of a signed type.
> >> > >
> >> > > The spec says (as in the comment below):
> >> > >
> >> > > "The partition reference time is computed by the following formula:
> >> > >
> >> > > ReferenceTime = ((VirtualTsc * TscScale) >> 64) + TscOffset
> >> > >
> >> > > The multiplication is a 64 bit multiplication, which results in a 128 
> >> > > bit
> >> > > number which is then shifted 64 times to the right to obtain the high 
> >> > > 64
> >> > > bits.TscScale"
> >> >
> >> > Well, yes, you want a 128-bit result. But for that you don't need to
> >> > multiply 128-bit quantities. See e.g. our own scale_delta() or
> >> > hvm_scale_tsc().
> >>
> >> Testing showed that by not casting first things were broken. I assumed 
> >> this was because the result
> of
> >> the multiplication was being truncated to 64-bits before assignment, but I 
> >> can check the generated
> >> code. I'll also have a look at the examples you cite.
> >
> > Both those examples do the multiplication by inline asm (presumably to avoid
> > truncation). Is that what you'd prefer?
> 
> That would imo be best, not the least because of making us independent
> of possible issues with 128-bit arithmetic on older compilers.
> 

Ok, I think I have figured out the necessary runes so I'll do that.

  Paul

> Jan
> 


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next RFC 1/4] build/m4: make python_devel.m4 work with both python 2 and 3

2019-03-06 Thread Anthony PERARD
On Tue, Mar 05, 2019 at 04:42:03PM +, Wei Liu wrote:
> Do the following:
> 
> 1. Change the form of "print".
> 2. Check for ABI flags -- this is complicated because it is only
>introduced in 3.2.

Is this a recommanded way of doing this? I may have a better way of
fixing this macro, see below.

> 3. Fix library name in AC_CHECK_LIB.
> 4. Remove other-libs in AC_CHECK_LIB.

Why did you remove the other libs? Also, with this change, PYTHON_LIBS
isn't used anywhere anymore, and can be removed.

> 
> Signed-off-by: Wei Liu 
> ---
> I doubt the non python-pkg branch works, because the paths generated
> seem rather off. It definitely doesn't work on my machine, but I
> don't know how other systems could possibly be configured before the
> existence of python-config.
> ---
>  m4/python_devel.m4 | 27 ---
>  tools/configure| 34 --
>  2 files changed, 36 insertions(+), 25 deletions(-)
> 
> diff --git a/m4/python_devel.m4 b/m4/python_devel.m4
> index 05ea4ef7e2..1e2f41b6aa 100644
> --- a/m4/python_devel.m4
> +++ b/m4/python_devel.m4
> @@ -2,37 +2,42 @@ AC_DEFUN([AX_CHECK_PYTHON_DEVEL], [
>  ac_previous_cppflags=$CPPFLAGS
>  ac_previous_ldflags=$LDFLAGS
>  ac_python_version=`$PYTHON -c 'import distutils.sysconfig; \
> -print distutils.sysconfig.get_config_var("VERSION")'`
> +print(distutils.sysconfig.get_config_var("VERSION"))'`
> +ac_python_abiflags=
>  AC_PATH_PROG([pyconfig], [$PYTHON-config], [no])
>  AS_IF([test x"$pyconfig" = x"no"], [
>  dnl For those that don't have python-config
>  CPPFLAGS="$CFLAGS `$PYTHON -c 'import distutils.sysconfig; \
>  print "-I" + distutils.sysconfig.get_config_var("INCLUDEPY")'`"
>  CPPFLAGS="$CPPFLAGS `$PYTHON -c 'import distutils.sysconfig; \
> -print distutils.sysconfig.get_config_var("CFLAGS")'`"
> +print(distutils.sysconfig.get_config_var("CFLAGS"))'`"
>  PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
> -print distutils.sysconfig.get_config_var("LIBS")'`"
> +print(distutils.sysconfig.get_config_var("LIBS"))'`"
>  PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
> -print distutils.sysconfig.get_config_var("SYSLIBS")'`"
> +print(distutils.sysconfig.get_config_var("SYSLIBS"))'`"
>  LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
> -print "-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
> -standard_lib=1) + "/config"'`"
> +print("-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
> +standard_lib=1) + "/config")'`"
>  LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
> -print distutils.sysconfig.get_config_var("LINKFORSHARED")'`"
> +print(distutils.sysconfig.get_config_var("LINKFORSHARED"))'`"
>  LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
> -print distutils.sysconfig.get_config_var("LDFLAGS")'`"
> +print(distutils.sysconfig.get_config_var("LDFLAGS"))'`"
>  ], [
>  dnl If python-config is found use it
>  CPPFLAGS="$CFLAGS `$PYTHON-config --cflags`"
>  LDFLAGS="$LDFLAGS `$PYTHON-config --ldflags`"
>  PYTHON_LIBS="$LIBS `$PYTHON-config --libs`"
> +abiflags="`$PYTHON-config --abiflags`"
> +if test "$?" == "0"
> +then
> +ac_python_abiflags="$abiflags"
> +fi
>  ])
>  
>  AC_CHECK_HEADER([Python.h], [],
>  [AC_MSG_ERROR([Unable to find Python development headers])],)
> -AC_CHECK_LIB(python$ac_python_version, PyArg_ParseTuple, [],
> -[AC_MSG_ERROR([Unable to find a suitable python development library])],
> -[$PYTHON_LIBS])
> +AC_CHECK_LIB(python$ac_python_version$ac_python_abiflags, PyArg_ParseTuple, 
> [],
> +[AC_MSG_ERROR([Unable to find a suitable python development library])])

So, AC_CHECK_LIB seems to only be used to check if PyArg_ParseTuple
exist, and requires as argument the name of the lib which is now
complicated.

But, AC_CHECK_LIB do test compilation using the LDFLAGS which already
contain the python lib we want, so instead, we could only do the part of
the jobs that we need:

AC_LINK_IFELSE([AC_LANG_CALL([], [PyArg_ParseTuple])], [],
[AC_MSG_ERROR([Unable to find a suitable python development library])])

That generate a main.c with PyArg_ParseTuple() call like AC_CHECK_LIB
do, and do build/link. If that fails, throw an error.

That avoid to use the --abiflags, which we don't need.

What do you thing?

Some progress message can be added, similair to AC_CHECK_LIB:
AC_MSG_CHECKING([for PyArg_ParseTuple])
and [AC_MSG_RESULT([yes])] on success.

(I think AC_CHECK_LIB would also update $LIBS, but I don't think our
build system is using that.)

>  CPPFLAGS=$ac_previous_cppflags
>  LDFLAGS=$ac_previous_ldflags
>  ])

-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v11 3/9] xen: introduce DECLARE_BOUNDS

2019-03-06 Thread Jan Beulich
>>> On 05.03.19 at 23:38,  wrote:
> --- a/xen/include/xen/lib.h
> +++ b/xen/include/xen/lib.h
> @@ -173,4 +173,92 @@ void init_constructors(void);
>  void *bsearch(const void *key, const void *base, size_t num, size_t size,
>int (*cmp)(const void *key, const void *elt));
>  
> + /*
> +  * Declare start and end array variables in C corresponding to existing
> +  * linker symbols.
> +  *
> +  * These macros, or an alternative technique, MUST be used any time
> +  * linker symbols are imported into C via the `extern []' idiom.
> +  *
> +  *__DECLARE_BOUNDS(TYPE, START, START, END)

START used twice here makes it ambiguous which one is which in the
subsequent text.

> +  *  introduces the following two constant expressions
> +  *
> +  *const TYPE *START;
> +  *const struct abstract_NAME *END;

For one these are declarations, not (constant) expressions. And
then the declarations produce array types, not pointer types.
Please let's not have a comment which is out of sync with what
it describes from the very beginning.

> +  *  whose values are the linker symbols START and END; these
> +  *  should be the start and end of a memory region.
> +  *
> +  *  You may then use these two inline functions:
> +  *
> +  *bool NAME_lt(const TYPE *s1, const struct abstract_NAME *s2);
> +  *ptrdiff_t NAME_diff(const TYPE *s1, const struct abstract_NAME *s2);
> +  *
> +  *  lt returns true iff s1 < s2.
> +  *  diff returns the s2-s1 in units of TYPE.
> +  *
> +  *

Stray double blank comment lines and no mention of _bytediff.

> +static inline ptrdiff_t name ## _diff(const type s1[],   
>  \
> +  const struct abstract_ ## name s2[])   
>  \
> +{
>  \
> +BUILD_BUG_ON(alignof(*s1) != alignof(*s2));  
>  \
> +return (ptrdiff_t)((uintptr_t)s2 - (uintptr_t)s1) /  
>  \
> +   (ptrdiff_t)sizeof(*s1);   
>   \
> +}   

I had specifically asked for this to simply call _bytediff, to limit
redundancy and in particular the total number of casts.

> +static inline ptrdiff_t name ## _bytediff(const type s1[],   
>  \
> +  const struct abstract_ ## name 
> s2[])\
> +{
>  \
> +BUILD_BUG_ON(alignof(*s1) != alignof(*s2));  
>  \
> +return (ptrdiff_t)((uintptr_t)s2 - (uintptr_t)s1);   
>  \
> +}

What's the value of the intermediate casting to uintptr_t? Why not
cast to ptrdiff_t right away?

I also don't think the BUILD_BUG_ON() is helpful in this latter case.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next RFC 1/4] build/m4: make python_devel.m4 work with both python 2 and 3

2019-03-06 Thread Wei Liu
On Wed, Mar 06, 2019 at 03:07:16PM +, Anthony PERARD wrote:
> On Tue, Mar 05, 2019 at 04:42:03PM +, Wei Liu wrote:
> > Do the following:
> > 
> > 1. Change the form of "print".
> > 2. Check for ABI flags -- this is complicated because it is only
> >introduced in 3.2.
> 
> Is this a recommanded way of doing this? I may have a better way of
> fixing this macro, see below.

Nope. I figured this out myself. There doesn't seem to be a recommended
way to do it.

> 
> > 3. Fix library name in AC_CHECK_LIB.
> > 4. Remove other-libs in AC_CHECK_LIB.
> 
> Why did you remove the other libs? Also, with this change, PYTHON_LIBS
> isn't used anywhere anymore, and can be removed.
> 

Because

"The other-libraries argument should be limited to cases where it is
desirable to test for one library in the presence of another that is not
already in LIBS."

Its original usage is wrong. At least in the python-config case,
PYTHON_LIBS is "-lpython2.7 -lpthread -ldl -lutil -lm". ./configure
already knows how to use LDFLAGS as far as I can tell from the log.

Yes. PYTHON_LIBS can be removed. I think.


> > 
> > Signed-off-by: Wei Liu 
> > ---
> > I doubt the non python-pkg branch works, because the paths generated
> > seem rather off. It definitely doesn't work on my machine, but I
> > don't know how other systems could possibly be configured before the
> > existence of python-config.
> > ---
> >  m4/python_devel.m4 | 27 ---
> >  tools/configure| 34 --
> >  2 files changed, 36 insertions(+), 25 deletions(-)
> > 
> > diff --git a/m4/python_devel.m4 b/m4/python_devel.m4
> > index 05ea4ef7e2..1e2f41b6aa 100644
> > --- a/m4/python_devel.m4
> > +++ b/m4/python_devel.m4
> > @@ -2,37 +2,42 @@ AC_DEFUN([AX_CHECK_PYTHON_DEVEL], [
> >  ac_previous_cppflags=$CPPFLAGS
> >  ac_previous_ldflags=$LDFLAGS
> >  ac_python_version=`$PYTHON -c 'import distutils.sysconfig; \
> > -print distutils.sysconfig.get_config_var("VERSION")'`
> > +print(distutils.sysconfig.get_config_var("VERSION"))'`
> > +ac_python_abiflags=
> >  AC_PATH_PROG([pyconfig], [$PYTHON-config], [no])
> >  AS_IF([test x"$pyconfig" = x"no"], [
> >  dnl For those that don't have python-config
> >  CPPFLAGS="$CFLAGS `$PYTHON -c 'import distutils.sysconfig; \
> >  print "-I" + distutils.sysconfig.get_config_var("INCLUDEPY")'`"
> >  CPPFLAGS="$CPPFLAGS `$PYTHON -c 'import distutils.sysconfig; \
> > -print distutils.sysconfig.get_config_var("CFLAGS")'`"
> > +print(distutils.sysconfig.get_config_var("CFLAGS"))'`"
> >  PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
> > -print distutils.sysconfig.get_config_var("LIBS")'`"
> > +print(distutils.sysconfig.get_config_var("LIBS"))'`"
> >  PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
> > -print distutils.sysconfig.get_config_var("SYSLIBS")'`"
> > +print(distutils.sysconfig.get_config_var("SYSLIBS"))'`"
> >  LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
> > -print "-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
> > -standard_lib=1) + "/config"'`"
> > +print("-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
> > +standard_lib=1) + "/config")'`"
> >  LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
> > -print distutils.sysconfig.get_config_var("LINKFORSHARED")'`"
> > +print(distutils.sysconfig.get_config_var("LINKFORSHARED"))'`"
> >  LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
> > -print distutils.sysconfig.get_config_var("LDFLAGS")'`"
> > +print(distutils.sysconfig.get_config_var("LDFLAGS"))'`"
> >  ], [
> >  dnl If python-config is found use it
> >  CPPFLAGS="$CFLAGS `$PYTHON-config --cflags`"
> >  LDFLAGS="$LDFLAGS `$PYTHON-config --ldflags`"
> >  PYTHON_LIBS="$LIBS `$PYTHON-config --libs`"
> > +abiflags="`$PYTHON-config --abiflags`"
> > +if test "$?" == "0"
> > +then
> > +ac_python_abiflags="$abiflags"
> > +fi
> >  ])
> >  
> >  AC_CHECK_HEADER([Python.h], [],
> >  [AC_MSG_ERROR([Unable to find Python development headers])],)
> > -AC_CHECK_LIB(python$ac_python_version, PyArg_ParseTuple, [],
> > -[AC_MSG_ERROR([Unable to find a suitable python development library])],
> > -[$PYTHON_LIBS])
> > +AC_CHECK_LIB(python$ac_python_version$ac_python_abiflags, 
> > PyArg_ParseTuple, [],
> > +[AC_MSG_ERROR([Unable to find a suitable python development library])])
> 
> So, AC_CHECK_LIB seems to only be used to check if PyArg_ParseTuple
> exist, and requires as argument the name of the lib which is now
> complicated.
> 
> But, AC_CHECK_LIB do test compilation using the LDFLAGS which already
> contain the python lib we want, so instead, we could only do the part of
> the jobs that we need:
> 
> AC_LINK_IFELSE([AC_LANG_CALL([], [PyArg_ParseTuple])], [],
> [AC_MSG_ERROR([Unable 

Re: [Xen-devel] [PATCH v11 5/9] xen/x86: use DECLARE_BOUNDS as required

2019-03-06 Thread Jan Beulich
>>> On 05.03.19 at 23:38,  wrote:
> --- a/xen/arch/x86/percpu.c
> +++ b/xen/arch/x86/percpu.c
> @@ -13,7 +13,8 @@ unsigned long __per_cpu_offset[NR_CPUS];
>   * context of PV guests.
>   */
>  #define INVALID_PERCPU_AREA (0x8000L - (long)__per_cpu_start)
> -#define PERCPU_ORDER get_order_from_bytes(__per_cpu_data_end - 
> __per_cpu_start)
> +#define PERCPU_ORDER get_order_from_bytes(per_cpu_diff(__per_cpu_start, \
> +   __per_cpu_data_end))

Please use _bytediff() when bytes are meant (i.e. also below, and
perhaps elsewhere).

> @@ -600,7 +602,9 @@ static void noinline init_done(void)
>  unregister_init_virtual_region();
>  
>  /* Zero the .init code and data. */
> -for ( va = __init_begin; va < _p(__init_end); va += PAGE_SIZE )
> +for ( va = (char *)__init_begin;
> +  init_lt(va, __init_end);
> +  va += PAGE_SIZE )

Is the line wrapping really needed here?

> --- a/xen/drivers/vpci/vpci.c
> +++ b/xen/drivers/vpci/vpci.c
> @@ -31,9 +31,9 @@ struct vpci_register {
>  };
>  
>  #ifdef __XEN__
> -extern vpci_register_init_t *const __start_vpci_array[];
> -extern vpci_register_init_t *const __end_vpci_array[];
> -#define NUM_VPCI_INIT (__end_vpci_array - __start_vpci_array)
> +typedef vpci_register_init_t *const vpci_array_t;

You don't want to keep the const here - DECLARE_BOUNDS() will
suitably add it.

Also how about vcpi_init_t or vpci_reg_init_t or some such? The
defined type is not really an array after all.

> +DECLARE_BOUNDS(vpci_array, __start_vpci_array, __end_vpci_array);
> +#define NUM_VPCI_INIT (vpci_array_diff(__start_vpci_array, __end_vpci_array))

Unnecessary outermost parentheses.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next RFC 1/4] build/m4: make python_devel.m4 work with both python 2 and 3

2019-03-06 Thread Anthony PERARD
On Wed, Mar 06, 2019 at 03:24:43PM +, Wei Liu wrote:
> On Wed, Mar 06, 2019 at 03:07:16PM +, Anthony PERARD wrote:
> > > -AC_CHECK_LIB(python$ac_python_version, PyArg_ParseTuple, [],
> > > -[AC_MSG_ERROR([Unable to find a suitable python development 
> > > library])],
> > > -[$PYTHON_LIBS])
> > > +AC_CHECK_LIB(python$ac_python_version$ac_python_abiflags, 
> > > PyArg_ParseTuple, [],
> > > +[AC_MSG_ERROR([Unable to find a suitable python development 
> > > library])])
> > 
> > So, AC_CHECK_LIB seems to only be used to check if PyArg_ParseTuple
> > exist, and requires as argument the name of the lib which is now
> > complicated.
> > 
> > But, AC_CHECK_LIB do test compilation using the LDFLAGS which already
> > contain the python lib we want, so instead, we could only do the part of
> > the jobs that we need:
> > 
> > AC_LINK_IFELSE([AC_LANG_CALL([], [PyArg_ParseTuple])], [],
> > [AC_MSG_ERROR([Unable to find a suitable python development library])])
> > 
> > That generate a main.c with PyArg_ParseTuple() call like AC_CHECK_LIB
> > do, and do build/link. If that fails, throw an error.
> > 
> > That avoid to use the --abiflags, which we don't need.
> > 
> > What do you thing?
> 
> Definitely looks better. Let me try this.

Actually, I think I just found better, same things but using a macro
that already do what we want:

AC_CHECK_FUNC([PyArg_ParseTuple], [],
[AC_MSG_ERROR([Unable to find a suitable python development library])])

-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v11 6/9] xen/common: use DECLARE_BOUNDS as required

2019-03-06 Thread Jan Beulich
>>> On 05.03.19 at 23:38,  wrote:
> --- a/xen/common/kernel.c
> +++ b/xen/common/kernel.c
> @@ -306,20 +306,25 @@ void add_taint(unsigned int flag)
>  tainted |= flag;
>  }
>  
> -extern const initcall_t __initcall_start[], __presmp_initcall_end[],
> -__initcall_end[];
> +DECLARE_ARRAY_BOUNDS(initcall);
> +typedef initcall_t presmp_initcall_t;
> +DECLARE_ARRAY_BOUNDS(presmp_initcall);
>  
>  void __init do_presmp_initcalls(void)
>  {
>  const initcall_t *call;
> -for ( call = __initcall_start; call < __presmp_initcall_end; call++ )
> +for ( call = __presmp_initcall_start;
> +   presmp_initcall_lt(call, __presmp_initcall_end);
> +   call++ )

Hard tabs slipped in. Also would you mind adding the missing blank line
ahead of the one you modify?

>  (*call)();
>  }
>  
>  void __init do_initcalls(void)
>  {
>  const initcall_t *call;
> -for ( call = __presmp_initcall_end; call < __initcall_end; call++ )
> +for ( call = __initcall_start;
> +  initcall_lt(call, __initcall_end);
> +  call++ )

Again no need for splitting the line as it seems.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] L1TF Patch Series v8

2019-03-06 Thread Jan Beulich
>>> On 27.02.19 at 17:13,  wrote:
> This patch series attempts to mitigate the issue that have been raised in the
> XSA-289 (https://xenbits.xen.org/xsa/advisory-289.html). To block speculative
> execution on Intel hardware, an lfence instruction is required to make sure
> that selected checks are not bypassed. Speculative out-of-bound accesses can
> be prevented by using the array_index_nospec macro.
> 
> The major change compared to version 8 is in grant_table handling, protecting
> a few more version comparisons.

Apart from that last patch this series looks to have been ready to go in
for about a week. Would you still want to allow it in, or rather defer it
until after the release?

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v11 5/9] xen/x86: use DECLARE_BOUNDS as required

2019-03-06 Thread Jan Beulich
>>> On 05.03.19 at 23:38,  wrote:
> @@ -600,7 +602,9 @@ static void noinline init_done(void)
>  unregister_init_virtual_region();
>  
>  /* Zero the .init code and data. */
> -for ( va = __init_begin; va < _p(__init_end); va += PAGE_SIZE )
> +for ( va = (char *)__init_begin;
> +  init_lt(va, __init_end);
> +  va += PAGE_SIZE )
>  clear_page(va);

At the example of this - is casting away of const-ness not also a
potential certification hindrance?

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Can someone pls repair patchwork?

2019-03-06 Thread Ian Jackson
Lars Kurth writes ("Re: [Xen-devel] Can someone pls repair patchwork?"):
> Hi all, (+ Florian & +Ian, as NEC runs their own patchwork instance and may 
> have some insights)
> 
> before I approach I wanted to ask whether we are sure this has to do with the 
> list change. I am assuming that patchwork gets mails from a registered 
> account on xen-devel. So it is not clear whether the domain change would 
> cause this: see 
> https://patchwork-freedesktop.readthedocs.io/en/latest/installation.html#subscribe-a-local-address-to-the-mailing-list
> 
> Looking at registered e-mails (see png), there appear to be two patchwork 
> instances registered with xen-devel@
> * patchwork-xen-de...@patchwork.codeaurora.org
> * patchwork-xen-de...@patchwork.kernel.org 
> 
> Note that https://patchwork.codeaurora.org/project/xen-devel/list/ seems to 
> have broken at the same time as the kernel.org one
> 
> @Ian: do you know what we did to the lists and/or e-mail handling around that 
> time? Could this be primarily an issue caused by some infrastructure change?

I don't remember any dates but we did change the lists from
f...@lists.xen.org to f...@lists.xenproject.org for some corporate
tradmark branding kind of reason.  I doubt you want to revert that.

> Is there a way to check whether mails are actually sent from xen-devel@ to 
> the patchwork instances?
> If so, maybe a Credativ ticket is needed

I doubt this is the problem.  I think what is needed is for the
patchwork instance owners to update their configuration for our new
mailing list name.

If you look here at this URL Julien provided
  https://patchwork.kernel.org/project/xen-devel/
you see that it says
  List address  xen-de...@lists.xen.org

That is what needs fixing.  Similarly for the codeaurora one I
presume.

I don't know who "owns" this inside the patchwork.kernel.org system.
It says "Maintainers" which is maybe the person who can update the
settings, but it is blank.  Maybe the thing is managed by the site
administrators then.

Unfortunately I could not find contact details for the site admins
for either of these anywhere on those websites.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Can someone pls repair patchwork?

2019-03-06 Thread Julien Grall

Hi Ian,

On 06/03/2019 16:00, Ian Jackson wrote:

Unfortunately I could not find contact details for the site admins
for either of these anywhere on those websites.


You can find the contact on kernel.org (see [1]). Oleksandr already CCed them on 
another e-mail.


Cheers,

[1] https://www.kernel.org/category/contact-us.html



Ian.



--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] L1TF Patch Series v8

2019-03-06 Thread Juergen Gross
On 06/03/2019 16:42, Jan Beulich wrote:
 On 27.02.19 at 17:13,  wrote:
>> This patch series attempts to mitigate the issue that have been raised in the
>> XSA-289 (https://xenbits.xen.org/xsa/advisory-289.html). To block speculative
>> execution on Intel hardware, an lfence instruction is required to make sure
>> that selected checks are not bypassed. Speculative out-of-bound accesses can
>> be prevented by using the array_index_nospec macro.
>>
>> The major change compared to version 8 is in grant_table handling, protecting
>> a few more version comparisons.
> 
> Apart from that last patch this series looks to have been ready to go in
> for about a week. Would you still want to allow it in, or rather defer it
> until after the release?

I'd like to defer them.


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Can someone pls repair patchwork?

2019-03-06 Thread Lars Kurth


On 06/03/2019, 16:00, "Ian Jackson"  wrote:

Lars Kurth writes ("Re: [Xen-devel] Can someone pls repair patchwork?"):
> Hi all, (+ Florian & +Ian, as NEC runs their own patchwork instance and 
may have some insights)
> 
> before I approach I wanted to ask whether we are sure this has to do with 
the list change. I am assuming that patchwork gets mails from a registered 
account on xen-devel. So it is not clear whether the domain change would cause 
this: see 
https://patchwork-freedesktop.readthedocs.io/en/latest/installation.html#subscribe-a-local-address-to-the-mailing-list
> 
> Looking at registered e-mails (see png), there appear to be two patchwork 
instances registered with xen-devel@
> * patchwork-xen-de...@patchwork.codeaurora.org
> * patchwork-xen-de...@patchwork.kernel.org 
> 
> Note that https://patchwork.codeaurora.org/project/xen-devel/list/ seems 
to have broken at the same time as the kernel.org one
> 
> @Ian: do you know what we did to the lists and/or e-mail handling around 
that time? Could this be primarily an issue caused by some infrastructure 
change?

I don't remember any dates but we did change the lists from
f...@lists.xen.org to f...@lists.xenproject.org for some corporate
tradmark branding kind of reason.  I doubt you want to revert that.

> Is there a way to check whether mails are actually sent from xen-devel@ 
to the patchwork instances?
> If so, maybe a Credativ ticket is needed

I doubt this is the problem.  I think what is needed is for the
patchwork instance owners to update their configuration for our new
mailing list name.

If you look here at this URL Julien provided
  https://patchwork.kernel.org/project/xen-devel/
you see that it says
  List address  xen-de...@lists.xen.org

That is what needs fixing.  Similarly for the codeaurora one I
presume.

I don't know who "owns" this inside the patchwork.kernel.org system.
It says "Maintainers" which is maybe the person who can update the
settings, but it is blank.  Maybe the thing is managed by the site
administrators then.

Unfortunately I could not find contact details for the site admins
for either of these anywhere on those websites.

Oleksandr sent a mail to +webmas...@kernel.org
Let's see whether anything comes back

If not, I can try and do this via the LF's infrastructure team: they are 
probably handling this
Please ping me in a week or so, if that is the case

Best Regards
Lars



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next RFC 2/4] libxl: make python scripts work with python 2 and 3

2019-03-06 Thread Wei Liu
On Tue, Mar 05, 2019 at 05:34:13PM +, Andrew Cooper wrote:
> On 05/03/2019 16:42, Wei Liu wrote:
> > All scripts are transformed by 2to3.
> >
> > The only addition is "from __future__ import print_function" so that
> > print("BLAH", file=sys.stderr) can work.
> >
> > https://python-future.org/compatible_idioms.html
> >
> > Tested with 2.7 and 3.5.
> >
> > Signed-off-by: Wei Liu 
> > ---
> > I don't have environment to test 2.4 -- it is almost 15 years old. We
> > may want to consider bumping the minimum requirement to 2.7?
> 
> The compatible way to do this is  sys.stderr.write(msg + "\n") and using
> print() without the future import.
> 
> > @@ -269,7 +271,7 @@ class KeyedUnion(Aggregate):
> >  if not isinstance(keyvar_type, Enumeration):
> >  raise ValueError
> >  
> > -kv_kwargs = dict([(x.lstrip('keyvar_'),y) for (x,y) in 
> > kwargs.items() if x.startswith('keyvar_')])
> > +kv_kwargs = dict([(x.lstrip('keyvar_'),y) for (x,y) in 
> > list(kwargs.items()) if x.startswith('keyvar_')])
> 
> This shouldn't need changing.  List comprehensions are one of the few
> uses of .items() which is compatible with older versions of python IIRC.
> 
> > @@ -362,11 +364,10 @@ def parse(f):
> >  globs[n] = t
> >  
> >  try:
> > -execfile(f, globs, locs)
> > -except SyntaxError,e:
> > -raise SyntaxError, \
> > -  "Errors were found at line %d while processing %s:\n\t%s"\
> > -  %(e.lineno,f,e.text)
> > +exec(compile(open(f).read(), f, 'exec'), globs, locs)
> > +except SyntaxError as e:
> 
> This is the only really awkward bit, and isn't Py 2.4 compatible.
> 
> The only option here to retain pre 2.6 compatibility is:
> 
> try:
>     ...
> except SyntaxError:
>     _, e = sys.exc_info()[:2]
>     ...

Since we will bump python requirement to 2.6, I think the transformation
made by 2to3 should be fine.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [distros-debian-squeeze test] 83710: trouble: blocked/broken

2019-03-06 Thread Platform Team regression test user
flight 83710 distros-debian-squeeze real [real]
http://osstest.xensource.com/osstest/logs/83710/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-pvopsbroken
 build-i386   broken
 build-amd64-pvopsbroken
 build-armhf  broken
 build-amd64  broken
 build-i386-pvops broken
 build-armhf-pvops 3 syslog-serverrunning
 build-armhf   3 syslog-serverrunning

Tests which did not succeed, but are not blocking:
 test-amd64-i386-i386-squeeze-netboot-pygrub  1 build-check(1)  blocked n/a
 test-amd64-amd64-i386-squeeze-netboot-pygrub  1 build-check(1) blocked n/a
 test-amd64-i386-amd64-squeeze-netboot-pygrub  1 build-check(1) blocked n/a
 test-amd64-amd64-amd64-squeeze-netboot-pygrub  1 build-check(1)blocked n/a
 build-armhf-pvops 4 host-install(4)  broken like 83674
 build-armhf   4 host-install(4)  broken like 83674
 build-amd64-pvops 4 host-install(4)  broken like 83674
 build-amd64   4 host-install(4)  broken like 83674
 build-i386-pvops  4 host-install(4)  broken like 83674
 build-i3864 host-install(4)  broken like 83674
 build-armhf-pvops 5 capture-logs broken like 83674
 build-armhf   5 capture-logs broken like 83674

baseline version:
 flight   83674

jobs:
 build-amd64  broken  
 build-armhf  broken  
 build-i386   broken  
 build-amd64-pvopsbroken  
 build-armhf-pvopsbroken  
 build-i386-pvops broken  
 test-amd64-amd64-amd64-squeeze-netboot-pygrubblocked 
 test-amd64-i386-amd64-squeeze-netboot-pygrub blocked 
 test-amd64-amd64-i386-squeeze-netboot-pygrub blocked 
 test-amd64-i386-i386-squeeze-netboot-pygrub  blocked 



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xensource.com/osstest/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next RFC 1/4] build/m4: make python_devel.m4 work with both python 2 and 3

2019-03-06 Thread Wei Liu
On Wed, Mar 06, 2019 at 03:35:53PM +, Anthony PERARD wrote:
> On Wed, Mar 06, 2019 at 03:24:43PM +, Wei Liu wrote:
> > On Wed, Mar 06, 2019 at 03:07:16PM +, Anthony PERARD wrote:
> > > > -AC_CHECK_LIB(python$ac_python_version, PyArg_ParseTuple, [],
> > > > -[AC_MSG_ERROR([Unable to find a suitable python development 
> > > > library])],
> > > > -[$PYTHON_LIBS])
> > > > +AC_CHECK_LIB(python$ac_python_version$ac_python_abiflags, 
> > > > PyArg_ParseTuple, [],
> > > > +[AC_MSG_ERROR([Unable to find a suitable python development 
> > > > library])])
> > > 
> > > So, AC_CHECK_LIB seems to only be used to check if PyArg_ParseTuple
> > > exist, and requires as argument the name of the lib which is now
> > > complicated.
> > > 
> > > But, AC_CHECK_LIB do test compilation using the LDFLAGS which already
> > > contain the python lib we want, so instead, we could only do the part of
> > > the jobs that we need:
> > > 
> > > AC_LINK_IFELSE([AC_LANG_CALL([], [PyArg_ParseTuple])], [],
> > > [AC_MSG_ERROR([Unable to find a suitable python development 
> > > library])])
> > > 
> > > That generate a main.c with PyArg_ParseTuple() call like AC_CHECK_LIB
> > > do, and do build/link. If that fails, throw an error.
> > > 
> > > That avoid to use the --abiflags, which we don't need.
> > > 
> > > What do you thing?
> > 
> > Definitely looks better. Let me try this.
> 
> Actually, I think I just found better, same things but using a macro
> that already do what we want:
> 
> AC_CHECK_FUNC([PyArg_ParseTuple], [],
> [AC_MSG_ERROR([Unable to find a suitable python development library])])
> 

This works. Thank you.

Wei.

> -- 
> Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v11 8/9] xen: use DECLARE_BOUNDS in alternative.c

2019-03-06 Thread Jan Beulich
>>> On 05.03.19 at 23:38,  wrote:
> @@ -193,8 +191,10 @@ void init_or_livepatch apply_alternatives(struct 
> alt_instr *start,

Seeing this relevant part of the function parameters, ...

>   *
>   * So be careful if you want to change the scan order to any other
>   * order.
> + *
> + * start and end could be pointers to different objects.
>   */
> -for ( a = base = start; a < end; a++ )
> +for ( a = base = (struct alt_instr *)start; alt_instr_lt(a, end); a++ )

... why the cast?

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [GIT PULL] (xen) stable/for-jens-5.1 to your 'for-5.1/block' branch.

2019-03-06 Thread Jens Axboe
On 3/5/19 7:25 PM, Konrad Rzeszutek Wilk wrote:
> Hi Jens,
> 
> Apologies for doing it right at the merge window time. This patchset has been 
> brewing
> for quite a while.
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git 
> stable/for-jens-5.1
> 
> This patchset makes the backend more robust by reading a negotiation
> variable only once and not twice.

Pulled, thanks.

-- 
Jens Axboe


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-smoke test] 133621: tolerable all pass - PUSHED

2019-03-06 Thread osstest service owner
flight 133621 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133621/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  4deeaf2a3ee50b096426eea41a4c9b96ded0f029
baseline version:
 xen  eeb31ee522c7bb8541eb4c037be2c42bfcf0a3c3

Last test of basis   133607  2019-03-05 21:00:48 Z0 days
Testing same since   133621  2019-03-06 15:00:38 Z0 days1 attempts


People who touched revisions under test:
  George Dunlap 
  Wei Liu 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   eeb31ee522..4deeaf2a3e  4deeaf2a3ee50b096426eea41a4c9b96ded0f029 -> smoke

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next RFC 4/4] pygrub: make it build with python 3

2019-03-06 Thread Wei Liu
On Tue, Mar 05, 2019 at 04:42:06PM +, Wei Liu wrote:
> +
>  PyMODINIT_FUNC
>  initxenfsimage(void)

So Python 3 requires the initialisation function to be called
PyInit_xenfsimage, otherwise it can't find the entry point of this
module.

I have fixed this in my next version.

Wei.

>  {
> +#if PY_MAJOR_VERSION < 3
>   Py_InitModule("xenfsimage", fsimage_module_methods);
> +#else
> + return PyModule_Create(&fsimage_module_def);
> +#endif
>  }
> -- 
> 2.11.0
> 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-4.7-testing test] 133596: tolerable FAIL - PUSHED

2019-03-06 Thread osstest service owner
flight 133596 xen-4.7-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133596/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds 12 guest-start  fail REGR. vs. 130859

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-credit2  10 debian-install  fail blocked in 130773
 test-xtf-amd64-amd64-1  50 xtf/test-hvm64-lbr-tsx-vmentry fail like 130773
 test-xtf-amd64-amd64-5   69 xtf/test-hvm64-xsa-278   fail  like 130773
 test-xtf-amd64-amd64-3  50 xtf/test-hvm64-lbr-tsx-vmentry fail like 130826
 test-xtf-amd64-amd64-5  50 xtf/test-hvm64-lbr-tsx-vmentry fail like 130826
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 130859
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 130859
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 130859
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 130859
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 130859
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 130859
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop fail like 130859
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 130859
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 130859
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 130859
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 130859
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass

version targeted for testing:
 xen  88f936d44d2e34ca2d0827cc828ea9d3aeef3fe8
baseline version:
 xen  710cc096971019bc2e5a9aabb9af1acca0b5b9e7

Last test of basis   130859  2018-11-29 11:42:49 Z   97 days
Testing same since   133596  2019-03-05 15:06:04 Z1 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Jan Beulich 
  Juergen Gross 
  Man

Re: [Xen-devel] [PATCH RESEND 3/3] OvmfPkg/XenSupport: turn off address decoding before BAR sizing

2019-03-06 Thread Laszlo Ersek
On 03/06/19 15:26, Igor Druzhinin wrote:
> On 06/03/2019 13:22, Laszlo Ersek wrote:
>> On 03/06/19 13:40, Igor Druzhinin wrote:
>>> On Xen, hvmloader firmware leaves address decoding enabled for
>>> enumerated PCI device before jumping into OVMF. OVMF seems to
>>> expect it to be disabled and tries to size PCI BARs in several places
>>> without disabling it which causes BAR64, for example, being
>>> incorrectly placed by QEMU.
>>>
>>> Fix it by disabling PCI address decoding explicitly before the
>>> first attempt to size BARs on Xen.
>>>
>>> Contributed-under: TianoCore Contribution Agreement 1.1
>>> Signed-off-by: Igor Druzhinin 
>>> ---
>>>  OvmfPkg/Library/PciHostBridgeLib/XenSupport.c | 34 
>>> +++
>>>  1 file changed, 34 insertions(+)
>>>
>>> diff --git a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c 
>>> b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
>>> index 408fb24..9940335 100644
>>> --- a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
>>> +++ b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
>>> @@ -55,6 +55,33 @@ PcatPciRootBridgeBarExisted (
>>>EnableInterrupts ();
>>>  }
>>>  
>>> +#define EFI_PCI_COMMAND_DECODE ((UINT16)(EFI_PCI_COMMAND_IO_SPACE | \
>>> + EFI_PCI_COMMAND_MEMORY_SPACE))
>>
>> I thought I asked you not to define a macro here that started with the
>> "EFI_" prefix :/
>>
>> http://mid.mail-archive.com/dd8e3c9e-cb76-d3fe-6a10-c0a41c714b56@redhat.com
>>
> 
> This is a resend of v1 patch series to get Xen folks into CC and gather
> comments. I expect v2.

My bad, thanks for the clarification. On edk2-devel we very rarely post
RESENDs without updates, and so I missed the implications now.

Thanks
Laszlo

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH for-next v2 v2 4/5] pygrub: make it build with python 3

2019-03-06 Thread Wei Liu
With the help of two porting guides and cpython source code:

1. Use PyBytes to replace PyString counterparts.
2. Use PyVarObject_HEAD_INIT and provide compatibility for 2.5 and
   earlier.
3. Remove usage of Py_FindMethod.
4. Use new module initialisation routine.

For #3, Py_FindMethod was removed, yet an alternative wasn't
documented.  The code is the result of reverse-engineering cpython
commit 6116d4a1d1

https://docs.python.org/3/howto/cporting.html
http://python3porting.com/cextensions.html

Signed-off-by: Wei Liu 
---
v2: use PyBytes.
---
 tools/pygrub/src/fsimage/fsimage.c | 96 ++
 1 file changed, 88 insertions(+), 8 deletions(-)

diff --git a/tools/pygrub/src/fsimage/fsimage.c 
b/tools/pygrub/src/fsimage/fsimage.c
index 743a3fb7b8..7e124f7bb3 100644
--- a/tools/pygrub/src/fsimage/fsimage.c
+++ b/tools/pygrub/src/fsimage/fsimage.c
@@ -26,11 +26,15 @@
 #include 
 #include 
 
+#if PY_MAJOR_VERSION < 3
 #if (PYTHON_API_VERSION >= 1011)
 #define PY_PAD 
0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L
 #else
 #define PY_PAD 0L,0L,0L,0L
 #endif
+#else
+#define PY_PAD 0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L
+#endif
 
 typedef struct fsimage_fs {
PyObject_HEAD
@@ -66,12 +70,24 @@ fsimage_file_read(fsimage_file_t *file, PyObject *args, 
PyObject *kwargs)
 
bufsize = size ? size : 4096;
 
-   if ((buffer = PyString_FromStringAndSize(NULL, bufsize)) == NULL)
+   buffer =
+#if PY_MAJOR_VERSION < 3
+   PyString_FromStringAndSize(NULL, bufsize);
+#else
+   PyBytes_FromStringAndSize(NULL, bufsize);
+#endif
+
+   if (buffer == NULL)
return (NULL);
  
while (1) {
int err;
-   void *buf = PyString_AS_STRING(buffer) + bytesread;
+   void *buf =
+#if PY_MAJOR_VERSION < 3
+   PyString_AS_STRING(buffer) + bytesread;
+#else
+   PyBytes_AS_STRING(buffer) + bytesread;
+#endif
 
err = fsi_pread_file(file->file, buf, bufsize,
bytesread + offset);
@@ -91,12 +107,20 @@ fsimage_file_read(fsimage_file_t *file, PyObject *args, 
PyObject *kwargs)
if (bufsize == 0)
break;
} else {
+#if PY_MAJOR_VERSION < 3
if (_PyString_Resize(&buffer, bytesread + bufsize) < 0)
+#else
+   if (_PyBytes_Resize(&buffer, bytesread + bufsize) < 0)
+#endif
return (NULL);
}
}
 
+#if PY_MAJOR_VERSION < 3
_PyString_Resize(&buffer, bytesread);
+#else
+   _PyBytes_Resize(&buffer, bytesread);
+#endif
return (buffer);
 }
 
@@ -113,11 +137,13 @@ static struct PyMethodDef fsimage_file_methods[] = {
{ NULL, NULL, 0, NULL } 
 };
 
+#if PY_MAJOR_VERSION < 3
 static PyObject *
 fsimage_file_getattr(fsimage_file_t *file, char *name)
 {
return (Py_FindMethod(fsimage_file_methods, (PyObject *)file, name));
 }
+#endif
 
 static void
 fsimage_file_dealloc(fsimage_file_t *file)
@@ -128,16 +154,25 @@ fsimage_file_dealloc(fsimage_file_t *file)
PyObject_DEL(file);
 }
 
+/* Compatibility for 2.5 and earlier */
+#ifndef PyVarObject_HEAD_INIT
+#define PyVarObject_HEAD_INIT(type, size) \
+   PyObject_HEAD_INIT(type) size,
+#endif
+
 static char fsimage_file_type__doc__[] = "Filesystem image file";
 PyTypeObject fsimage_file_type = {
-   PyObject_HEAD_INIT(&PyType_Type)
-   0,  /* ob_size */
+   PyVarObject_HEAD_INIT(&PyType_Type, 0)
"xenfsimage.file",  /* tp_name */
sizeof(fsimage_file_t), /* tp_size */
0,  /* tp_itemsize */
(destructor) fsimage_file_dealloc,  /* tp_dealloc */
0,  /* tp_print */
+#if PY_MAJOR_VERSION < 3
(getattrfunc) fsimage_file_getattr, /* tp_getattr */
+#else
+   0,  /* tp_getattr */
+#endif
0,  /* tp_setattr */
0,  /* tp_compare */
0,  /* tp_repr */
@@ -151,7 +186,16 @@ PyTypeObject fsimage_file_type = {
0,  /* tp_setattro */
0,  /* tp_as_buffer */
Py_TPFLAGS_DEFAULT, /* tp_flags */
-   fsimage_file_type__doc__,
+   fsimage_file_type__doc__,   /* tp_doc */
+#if PY_MAJOR_VERSION >= 3
+   0,  /* tp_traverse */
+   0,  /* tp_clear */
+   0,  /* tp_richcompare */
+   0,  /* tp_weaklistoffset */
+   0,

[Xen-devel] [PATCH for-next v2 v2 3/5] pygrub: convert python scripts to work with 2.6 and up

2019-03-06 Thread Wei Liu
Run 2to3 and pick the sensible suggestions.

Signed-off-by: Wei Liu 
---
 tools/pygrub/src/ExtLinuxConf.py | 15 ---
 tools/pygrub/src/GrubConf.py | 36 ++--
 tools/pygrub/src/LiloConf.py | 15 ---
 3 files changed, 34 insertions(+), 32 deletions(-)

diff --git a/tools/pygrub/src/ExtLinuxConf.py b/tools/pygrub/src/ExtLinuxConf.py
index d1789bf020..b84bbf8454 100644
--- a/tools/pygrub/src/ExtLinuxConf.py
+++ b/tools/pygrub/src/ExtLinuxConf.py
@@ -32,7 +32,8 @@ class ExtLinuxImage(object):
 self.lines = []
 self.path = path
 self.root = ""
-map(self.set_from_line, lines)
+for line in lines:
+self.set_from_line(line)
 
 def set_from_line(self, line, replace = None):
 (com, arg) = GrubConf.grub_exact_split(line, 2)
@@ -67,7 +68,7 @@ class ExtLinuxImage(object):
 setattr(self, "initrd", a.replace("initrd=", ""))
 arg = arg.replace(a, "")
 
-if com is not None and self.commands.has_key(com):
+if com is not None and com in self.commands:
 if self.commands[com] is not None:
 setattr(self, self.commands[com], re.sub('^"(.+)"$', r"\1", 
arg.strip()))
 else:
@@ -136,7 +137,7 @@ class ExtLinuxConfigFile(object):
 def parse(self, buf = None):
 if buf is None:
 if self.filename is None:
-raise ValueError, "No config file defined to parse!"
+raise ValueError("No config file defined to parse!")
 
 f = open(self.filename, 'r')
 lines = f.readlines()
@@ -167,7 +168,7 @@ class ExtLinuxConfigFile(object):
 
 (com, arg) = GrubConf.grub_exact_split(l, 2)
 com = com.lower()
-if self.commands.has_key(com):
+if com in self.commands:
 if self.commands[com] is not None:
 setattr(self, self.commands[com], arg.strip())
 else:
@@ -207,8 +208,8 @@ class ExtLinuxConfigFile(object):
 
 if __name__ == "__main__":
 if len(sys.argv) < 2:
-raise RuntimeError, "Need a configuration file to read"
+raise RuntimeError("Need a configuration file to read")
 g = ExtLinuxConfigFile(sys.argv[1])
 for i in g.images:
-print i
-print g.default
+print(i)
+print(g.default)
diff --git a/tools/pygrub/src/GrubConf.py b/tools/pygrub/src/GrubConf.py
index dc810d55cb..97913f3993 100644
--- a/tools/pygrub/src/GrubConf.py
+++ b/tools/pygrub/src/GrubConf.py
@@ -44,7 +44,7 @@ def get_path(s):
 return (None, s)
 idx = s.find(')')
 if idx == -1:
-raise ValueError, "Unable to find matching ')'"
+raise ValueError("Unable to find matching ')'")
 d = s[:idx]
 return (GrubDiskPart(d), s[idx + 1:])
 
@@ -100,7 +100,7 @@ class _GrubImage(object):
 "  initrd: %s\n" %(self.title, self.root, self.kernel,
self.args, self.initrd))
 def _parse(self, lines):
-map(self.set_from_line, lines)
+list(map(self.set_from_line, lines))
 
 def reset(self, lines):
 self._root = self._initrd = self._kernel = self._args = None
@@ -141,7 +141,7 @@ class GrubImage(_GrubImage):
 def set_from_line(self, line, replace = None):
 (com, arg) = grub_exact_split(line, 2)
 
-if self.commands.has_key(com):
+if com in self.commands:
 if self.commands[com] is not None:
 setattr(self, self.commands[com], arg.strip())
 else:
@@ -177,7 +177,7 @@ class _GrubConfigFile(object):
 self.parse()
 
 def parse(self, buf = None):
-raise RuntimeError, "unimplemented parse function"   
+raise RuntimeError("unimplemented parse function")
 
 def hasPasswordAccess(self):
 return self.passwordAccess
@@ -201,7 +201,7 @@ class _GrubConfigFile(object):
 import crypt
 if crypt.crypt(password, pwd[1]) == pwd[1]:
 return True
-except Exception, e:
+except Exception as e:
 self.passExc = "Can't verify password: %s" % str(e)
 return False
 
@@ -213,7 +213,7 @@ class _GrubConfigFile(object):
 
 def set(self, line):
 (com, arg) = grub_exact_split(line, 2)
-if self.commands.has_key(com):
+if com in self.commands:
 if self.commands[com] is not None:
 setattr(self, self.commands[com], arg.strip())
 else:
@@ -233,7 +233,7 @@ class _GrubConfigFile(object):
 self._default = val
 
 if self._default < 0:
-raise ValueError, "default must be positive number"
+raise ValueError("default must be positive number")
 default = property(_get_default, _set_default)
 
 def set_splash(self, val):
@@ -265,7 +265,7 @@ class GrubConfigFile(_Gr

[Xen-devel] [PATCH for-next v2 v2 2/5] libxl: make python scripts work with python 2.6 and up

2019-03-06 Thread Wei Liu
Go through the transformations suggested by 2to3 and pick the
necessary ones.

Use sys.stderr.write to avoid importing from __future__.

Signed-off-by: Wei Liu 
---
 tools/libxl/gentest.py  |  2 +-
 tools/libxl/gentypes.py | 10 +-
 tools/libxl/idl.py  | 13 ++---
 3 files changed, 12 insertions(+), 13 deletions(-)

diff --git a/tools/libxl/gentest.py b/tools/libxl/gentest.py
index 989959fc68..81e13b437c 100644
--- a/tools/libxl/gentest.py
+++ b/tools/libxl/gentest.py
@@ -86,7 +86,7 @@ def gen_rand_init(ty, v, indent = "", parent = None):
 
 if __name__ == '__main__':
 if len(sys.argv) < 3:
-print >>sys.stderr, "Usage: gentest.py  "
+sys.stderr.write("Usage: gentest.py  \n")
 sys.exit(1)
 
 random.seed(os.getenv('LIBXL_TESTIDL_SEED'))
diff --git a/tools/libxl/gentypes.py b/tools/libxl/gentypes.py
index 88e5c5f30e..656c157c01 100644
--- a/tools/libxl/gentypes.py
+++ b/tools/libxl/gentypes.py
@@ -576,14 +576,14 @@ def libxl_C_enum_from_string(ty, str, e, indent = ""):
 
 if __name__ == '__main__':
 if len(sys.argv) != 6:
-print >>sys.stderr, "Usage: gentypes.py   
  "
+sys.stderr.write("Usage: gentypes.py
 \n")
 sys.exit(1)
 
 (_, idlname, header, header_private, header_json, impl) = sys.argv
 
 (builtins,types) = idl.parse(idlname)
 
-print "outputting libxl type definitions to %s" % header
+print("outputting libxl type definitions to %s" % header)
 
 f = open(header, "w")
 
@@ -633,7 +633,7 @@ if __name__ == '__main__':
 f.write("""#endif /* %s */\n""" % (header_define))
 f.close()
 
-print "outputting libxl JSON definitions to %s" % header_json
+print("outputting libxl JSON definitions to %s" % header_json)
 
 f = open(header_json, "w")
 
@@ -657,7 +657,7 @@ if __name__ == '__main__':
 f.write("""#endif /* %s */\n""" % header_json_define)
 f.close()
 
-print "outputting libxl type internal definitions to %s" % header_private
+print("outputting libxl type internal definitions to %s" % header_private)
 
 f = open(header_private, "w")
 
@@ -683,7 +683,7 @@ if __name__ == '__main__':
 f.write("""#endif /* %s */\n""" % header_json_define)
 f.close()
 
-print "outputting libxl type implementations to %s" % impl
+print("outputting libxl type implementations to %s" % impl)
 
 f = open(impl, "w")
 f.write("""
diff --git a/tools/libxl/idl.py b/tools/libxl/idl.py
index 2a7f3c44fe..b5bfc66b50 100644
--- a/tools/libxl/idl.py
+++ b/tools/libxl/idl.py
@@ -11,7 +11,7 @@ DIR_BOTH = 3
 _default_namespace = ""
 def namespace(s):
 if type(s) != str:
-raise TypeError, "Require a string for the default namespace."
+raise TypeError("Require a string for the default namespace.")
 global _default_namespace
 _default_namespace = s
 
@@ -346,7 +346,7 @@ class OrderedDict(dict):
 return [(x,self[x]) for x in self.__ordered]
 
 def parse(f):
-print >>sys.stderr, "Parsing %s" % f
+sys.stderr.write("Parsing %s\n" % f)
 
 globs = {}
 locs = OrderedDict()
@@ -362,11 +362,10 @@ def parse(f):
 globs[n] = t
 
 try:
-execfile(f, globs, locs)
-except SyntaxError,e:
-raise SyntaxError, \
-  "Errors were found at line %d while processing %s:\n\t%s"\
-  %(e.lineno,f,e.text)
+exec(compile(open(f).read(), f, 'exec'), globs, locs)
+except SyntaxError as e:
+raise SyntaxError("Errors were found at line %d while processing 
%s:\n\t%s"\
+  %(e.lineno,f,e.text))
 
 types = [t for t in locs.ordered_values() if isinstance(t,Type)]
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH for-next v2 v2 5/5] Update python requirement

2019-03-06 Thread Wei Liu
CentOS 5, which was the reason of the 2.4 restriction, is EOL. CentOS
6 ships 2.6.

Bump the version to 2.6 in README. Now that all scripts are 3
compatible, remove the restriction on python 2 as well.

Update the check in configure.ac.

Signed-off-by: Wei Liu 
---
 README | 10 ++
 tools/configure|  8 
 tools/configure.ac |  2 +-
 3 files changed, 7 insertions(+), 13 deletions(-)

diff --git a/README b/README
index c19409efa2..3049201d91 100644
--- a/README
+++ b/README
@@ -46,7 +46,7 @@ provided by your OS distributor:
 - GCC 4.8 or later
 - GNU Binutils 2.24 or later
 * Development install of zlib (e.g., zlib-dev)
-* Development install of Python 2, v2.4 or later (e.g., python-dev)
+* Development install of Python v2.6 or later (e.g., python-dev)
 * Development install of curses (e.g., libncurses-dev)
 * Development install of openssl (e.g., openssl-dev)
 * Development install of x11 (e.g. xorg-x11-dev)
@@ -177,16 +177,10 @@ Python Runtime Libraries
 
 Various tools, such as pygrub, have the following runtime dependencies:
 
-* Python 2, v2.4 or later.
+* Python v2.6 or later.
   URL:http://www.python.org/
   Debian: python
 
-Note that the build system expects `python` to be python2.  If your system
-has `python` pointing to python3 (as in the case of Arch Linux or Anaconda),
-you'll need to specify a path to a python2 binary when running configure:
-
-PYTHON=/usr/bin/python2 ./configure
-
 Intel(R) Trusted Execution Technology Support
 =
 
diff --git a/tools/configure b/tools/configure
index 632ccce445..e1fa5d6b0f 100755
--- a/tools/configure
+++ b/tools/configure
@@ -7002,15 +7002,15 @@ if test x"${PYTHONPATH}" = x"no"
 then
 as_fn_error $? "Unable to find $PYTHON, please install $PYTHON" "$LINENO" 5
 fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for python version >= 2.3 " 
>&5
-$as_echo_n "checking for python version >= 2.3 ... " >&6; }
-`$PYTHON -c 'import sys; sys.exit(eval("sys.version_info < (2, 3)"))'`
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for python version >= 2.6 " 
>&5
+$as_echo_n "checking for python version >= 2.6 ... " >&6; }
+`$PYTHON -c 'import sys; sys.exit(eval("sys.version_info < (2, 6)"))'`
 if test "$?" != "0"
 then
 python_version=`$PYTHON -V 2>&1`
 { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
 $as_echo "no" >&6; }
-as_fn_error $? "$python_version is too old, minimum required version is 
2.3" "$LINENO" 5
+as_fn_error $? "$python_version is too old, minimum required version is 
2.6" "$LINENO" 5
 else
 { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
 $as_echo "yes" >&6; }
diff --git a/tools/configure.ac b/tools/configure.ac
index 1499344ce6..c9fd69ddfa 100644
--- a/tools/configure.ac
+++ b/tools/configure.ac
@@ -358,7 +358,7 @@ AS_IF([echo "$PYTHON" | grep -q "^/"], [
 ],[test -z "$PYTHON"], [PYTHON="python"],
 [AC_MSG_ERROR([PYTHON specified, but is not an absolute path])])
 AX_PATH_PROG_OR_FAIL([PYTHONPATH], [$PYTHON])
-AX_CHECK_PYTHON_VERSION([2], [3])
+AX_CHECK_PYTHON_VERSION([2], [6])
 
 AS_IF([test "$cross_compiling" != yes], [
 AX_CHECK_PYTHON_DEVEL()
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH for-next v2 v2 0/5] tools: Python 3 compatibility

2019-03-06 Thread Wei Liu
This series makes tools work with Python 2 and 3.

No more RFC because I have tested it on my testbox.

You need Andrew's "tools/xen-foreign: Update python scripts to be   

Py3 compatible" as well.

Wei.

Wei Liu (5):
  build/m4: make python_devel.m4 work with both python 2 and 3
  libxl: make python scripts work with python 2.6 and up
  pygrub: convert python scripts to work with 2.6 and up
  pygrub: make it build with python 3
  Update python requirement

 README | 10 +---
 m4/python_devel.m4 | 23 -
 tools/configure| 72 +---
 tools/configure.ac |  2 +-
 tools/libxl/gentest.py |  2 +-
 tools/libxl/gentypes.py| 10 ++--
 tools/libxl/idl.py | 13 +++---
 tools/pygrub/src/ExtLinuxConf.py   | 15 +++---
 tools/pygrub/src/GrubConf.py   | 36 +++---
 tools/pygrub/src/LiloConf.py   | 15 +++---
 tools/pygrub/src/fsimage/fsimage.c | 96 ++
 11 files changed, 157 insertions(+), 137 deletions(-)

-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH for-next v2 v2 1/5] build/m4: make python_devel.m4 work with both python 2 and 3

2019-03-06 Thread Wei Liu
Do the following:

1. Change the form of "print".
2. Use AC_CHECK_FUNC to avoid the need to generate library name.
3. Remove unused stuff.

Signed-off-by: Wei Liu 
---
I doubt the non-python-config branch works, because the paths generated
seem rather off. It definitely doesn't work on my machine, but I
don't know how other systems could possibly be configured before the
existence of python-config.
---
 m4/python_devel.m4 | 23 +++-
 tools/configure| 64 +++---
 2 files changed, 16 insertions(+), 71 deletions(-)

diff --git a/m4/python_devel.m4 b/m4/python_devel.m4
index 05ea4ef7e2..f9cb23aee1 100644
--- a/m4/python_devel.m4
+++ b/m4/python_devel.m4
@@ -1,38 +1,31 @@
 AC_DEFUN([AX_CHECK_PYTHON_DEVEL], [
 ac_previous_cppflags=$CPPFLAGS
 ac_previous_ldflags=$LDFLAGS
-ac_python_version=`$PYTHON -c 'import distutils.sysconfig; \
-print distutils.sysconfig.get_config_var("VERSION")'`
 AC_PATH_PROG([pyconfig], [$PYTHON-config], [no])
 AS_IF([test x"$pyconfig" = x"no"], [
 dnl For those that don't have python-config
 CPPFLAGS="$CFLAGS `$PYTHON -c 'import distutils.sysconfig; \
 print "-I" + distutils.sysconfig.get_config_var("INCLUDEPY")'`"
 CPPFLAGS="$CPPFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-print distutils.sysconfig.get_config_var("CFLAGS")'`"
-PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-print distutils.sysconfig.get_config_var("LIBS")'`"
-PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-print distutils.sysconfig.get_config_var("SYSLIBS")'`"
+print(distutils.sysconfig.get_config_var("CFLAGS"))'`"
 LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-print "-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
-standard_lib=1) + "/config"'`"
+print("-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
+standard_lib=1) + "/config")'`"
 LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-print distutils.sysconfig.get_config_var("LINKFORSHARED")'`"
+print(distutils.sysconfig.get_config_var("LINKFORSHARED"))'`"
 LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-print distutils.sysconfig.get_config_var("LDFLAGS")'`"
+print(distutils.sysconfig.get_config_var("LDFLAGS"))'`"
 ], [
 dnl If python-config is found use it
 CPPFLAGS="$CFLAGS `$PYTHON-config --cflags`"
 LDFLAGS="$LDFLAGS `$PYTHON-config --ldflags`"
-PYTHON_LIBS="$LIBS `$PYTHON-config --libs`"
 ])
 
 AC_CHECK_HEADER([Python.h], [],
 [AC_MSG_ERROR([Unable to find Python development headers])],)
-AC_CHECK_LIB(python$ac_python_version, PyArg_ParseTuple, [],
-[AC_MSG_ERROR([Unable to find a suitable python development library])],
-[$PYTHON_LIBS])
+AC_CHECK_FUNC([PyArg_ParseTuple], [],
+[AC_MSG_ERROR([Unable to find a suitable python development library])]) 
+
 CPPFLAGS=$ac_previous_cppflags
 LDFLAGS=$ac_previous_ldflags
 ])
diff --git a/tools/configure b/tools/configure
index acc857510e..632ccce445 100755
--- a/tools/configure
+++ b/tools/configure
@@ -7418,8 +7418,6 @@ if test "$cross_compiling" != yes; then :
 
 ac_previous_cppflags=$CPPFLAGS
 ac_previous_ldflags=$LDFLAGS
-ac_python_version=`$PYTHON -c 'import distutils.sysconfig; \
-print distutils.sysconfig.get_config_var("VERSION")'`
 # Extract the first word of "$PYTHON-config", so it can be a program name with 
args.
 set dummy $PYTHON-config; ac_word=$2
 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5
@@ -7466,24 +7464,19 @@ if test x"$pyconfig" = x"no"; then :
 CPPFLAGS="$CFLAGS `$PYTHON -c 'import distutils.sysconfig; \
 print "-I" + distutils.sysconfig.get_config_var("INCLUDEPY")'`"
 CPPFLAGS="$CPPFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-print distutils.sysconfig.get_config_var("CFLAGS")'`"
-PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-print distutils.sysconfig.get_config_var("LIBS")'`"
-PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-print distutils.sysconfig.get_config_var("SYSLIBS")'`"
+print(distutils.sysconfig.get_config_var("CFLAGS"))'`"
 LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-print "-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
-standard_lib=1) + "/config"'`"
+print("-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
+standard_lib=1) + "/config")'`"
 LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-print distutils.sysconfig.get_config_var("LINKFORSHARED")'`"
+print(distutils.sysconfig.get_config_var("LINKFORSHARED"))'`"
 LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-print distutils.sysconfig.get_config_var("LDFLAGS")'`"
+print(distutils.sysconfig.get_config_var("LDFLAGS"))'`"
 
 else
 
 C

Re: [Xen-devel] [PATCH for-next v2 v2 0/5] tools: Python 3 compatibility

2019-03-06 Thread Wei Liu
On Wed, Mar 06, 2019 at 05:52:05PM +, Wei Liu wrote:
> This series makes tools work with Python 2 and 3.
> 
> No more RFC because I have tested it on my testbox.

Spoke too soon. The testbox wasn't using the files I installed. I will
retest tomorrow.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next v2 v2 2/5] libxl: make python scripts work with python 2.6 and up

2019-03-06 Thread Andrew Cooper
On 06/03/2019 17:52, Wei Liu wrote:
> @@ -362,11 +362,10 @@ def parse(f):
>  globs[n] = t
>  
>  try:
> -execfile(f, globs, locs)
> -except SyntaxError,e:
> -raise SyntaxError, \
> -  "Errors were found at line %d while processing %s:\n\t%s"\
> -  %(e.lineno,f,e.text)
> +exec(compile(open(f).read(), f, 'exec'), globs, locs)
> +except SyntaxError as e:
> +raise SyntaxError("Errors were found at line %d while processing 
> %s:\n\t%s"\
> +  %(e.lineno,f,e.text))

As you're editing this, the \ can go, and it would be nice to properly
indent the second line.  (Even better if we can get spaces in sensible
places).

Otherwise, Reviewed-by: Andrew Cooper 

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next v2 v2 3/5] pygrub: convert python scripts to work with 2.6 and up

2019-03-06 Thread Andrew Cooper
On 06/03/2019 17:52, Wei Liu wrote:
> diff --git a/tools/pygrub/src/GrubConf.py b/tools/pygrub/src/GrubConf.py
> index dc810d55cb..97913f3993 100644
> --- a/tools/pygrub/src/GrubConf.py
> +++ b/tools/pygrub/src/GrubConf.py
> @@ -100,7 +100,7 @@ class _GrubImage(object):
>  "  initrd: %s\n" %(self.title, self.root, self.kernel,
> self.args, self.initrd))
>  def _parse(self, lines):
> -map(self.set_from_line, lines)
> +list(map(self.set_from_line, lines))

Another site which wants a transform into a for loop.

>  
>  def reset(self, lines):
>  self._root = self._initrd = self._kernel = self._args = None
> @@ -462,12 +462,12 @@ class Grub2ConfigFile(_GrubConfigFile):
>  
>  if __name__ == "__main__":
>  if len(sys.argv) < 3:
> -raise RuntimeError, "Need a grub version (\"grub\" or \"grub2\") and 
> a grub.conf or grub.cfg to read"
> +raise RuntimeError("Need a grub version (\"grub\" or \"grub2\") and 
> a grub.conf or grub.cfg to read")

As you're editing this anyway, it would be better to switch to

RuntimeError('Need a grub version ("grub" or "grub2") and a grub.conf or 
grub.cfg to read')


To drop the unnecessary escaping.

Otherwise, Reviewed-by: Andrew Cooper 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next v2 v2 4/5] pygrub: make it build with python 3

2019-03-06 Thread Andrew Cooper
On 06/03/2019 17:52, Wei Liu wrote:
> @@ -128,16 +154,25 @@ fsimage_file_dealloc(fsimage_file_t *file)
>   PyObject_DEL(file);
>  }
>  
> +/* Compatibility for 2.5 and earlier */
> +#ifndef PyVarObject_HEAD_INIT
> +#define PyVarObject_HEAD_INIT(type, size) \
> + PyObject_HEAD_INIT(type) size,
> +#endif
> +
>  static char fsimage_file_type__doc__[] = "Filesystem image file";
>  PyTypeObject fsimage_file_type = {
> - PyObject_HEAD_INIT(&PyType_Type)
> - 0,  /* ob_size */
> + PyVarObject_HEAD_INIT(&PyType_Type, 0)
>   "xenfsimage.file",  /* tp_name */
>   sizeof(fsimage_file_t), /* tp_size */
>   0,  /* tp_itemsize */
>   (destructor) fsimage_file_dealloc,  /* tp_dealloc */
>   0,  /* tp_print */
> +#if PY_MAJOR_VERSION < 3
>   (getattrfunc) fsimage_file_getattr, /* tp_getattr */
> +#else
> + 0,  /* tp_getattr */
> +#endif
>   0,  /* tp_setattr */
>   0,  /* tp_compare */
>   0,  /* tp_repr */
> @@ -151,7 +186,16 @@ PyTypeObject fsimage_file_type = {
>   0,  /* tp_setattro */
>   0,  /* tp_as_buffer */
>   Py_TPFLAGS_DEFAULT, /* tp_flags */
> - fsimage_file_type__doc__,
> + fsimage_file_type__doc__,   /* tp_doc */
> +#if PY_MAJOR_VERSION >= 3
> + 0,  /* tp_traverse */
> + 0,  /* tp_clear */
> + 0,  /* tp_richcompare */
> + 0,  /* tp_weaklistoffset */
> + 0,  /* tp_iter */
> + 0,  /* tp_iternext */
> + fsimage_file_methods,   /* tp_methods */
> +#endif
>   PY_PAD

PY_PAD is very WTF.  I've got no idea why it is necessary in the first
place.

Either way, most of the fields are zero, so why not use named
initialisation?

PyTypeObject fsimage_file_type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
.tp_name = "xenfsimage.file",
.tp_size = sizeof(fsimage_file_t),
...
};


~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v11 3/9] xen: introduce DECLARE_BOUNDS

2019-03-06 Thread Stefano Stabellini
On Wed, 6 Mar 2019, Jan Beulich wrote:
> > +static inline ptrdiff_t name ## _diff(const type s1[], 
> >\
> > +  const struct abstract_ ## name s2[]) 
> >\
> > +{  
> >\
> > +BUILD_BUG_ON(alignof(*s1) != alignof(*s2));
> >\
> > +return (ptrdiff_t)((uintptr_t)s2 - (uintptr_t)s1) /
> >\
> > +   (ptrdiff_t)sizeof(*s1); 
> > \
> > +}  
> >  
> 
> I had specifically asked for this to simply call _bytediff, to limit
> redundancy and in particular the total number of casts.
> 
> > +static inline ptrdiff_t name ## _bytediff(const type s1[], 
> >\
> > +  const struct abstract_ ## name 
> > s2[])\
> > +{  
> >\
> > +BUILD_BUG_ON(alignof(*s1) != alignof(*s2));
> >\
> > +return (ptrdiff_t)((uintptr_t)s2 - (uintptr_t)s1); 
> >\
> > +}
> 
> What's the value of the intermediate casting to uintptr_t? Why not
> cast to ptrdiff_t right away?

uintptr_t is the integer type corresponding to a pointer, so we should
use uintptr_t first. ptrdiff_t is the difference type so we should cast
to it afterwards. Specifically, uintptr_t is unsigned and ptrdiff_t is
signed. So I don't think it would be correct to do:

  return (ptrdiff_t)((ptrdiff_t)s2 - (ptrdiff_t)s1);

Or am I missing your point?


I'll address all the other comments.


> I also don't think the BUILD_BUG_ON() is helpful in this latter case.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-next v2 v2 2/5] libxl: make python scripts work with python 2.6 and up

2019-03-06 Thread Hans van Kranenburg
On 3/6/19 6:52 PM, Wei Liu wrote:
> Go through the transformations suggested by 2to3 and pick the
> necessary ones.
> 
> Use sys.stderr.write to avoid importing from __future__.
> 
> Signed-off-by: Wei Liu 
> ---
>  tools/libxl/gentest.py  |  2 +-
>  tools/libxl/gentypes.py | 10 +-
>  tools/libxl/idl.py  | 13 ++---
>  3 files changed, 12 insertions(+), 13 deletions(-)
> 
> diff --git a/tools/libxl/gentest.py b/tools/libxl/gentest.py
> index 989959fc68..81e13b437c 100644
> --- a/tools/libxl/gentest.py
> +++ b/tools/libxl/gentest.py
> @@ -86,7 +86,7 @@ def gen_rand_init(ty, v, indent = "", parent = None):
>  
>  if __name__ == '__main__':
>  if len(sys.argv) < 3:
> -print >>sys.stderr, "Usage: gentest.py  "
> +sys.stderr.write("Usage: gentest.py  \n")
>  sys.exit(1)
>  
>  random.seed(os.getenv('LIBXL_TESTIDL_SEED'))
> diff --git a/tools/libxl/gentypes.py b/tools/libxl/gentypes.py
> index 88e5c5f30e..656c157c01 100644
> --- a/tools/libxl/gentypes.py
> +++ b/tools/libxl/gentypes.py
> @@ -576,14 +576,14 @@ def libxl_C_enum_from_string(ty, str, e, indent = "
> "):
>  
>  if __name__ == '__main__':
>  if len(sys.argv) != 6:
> -print >>sys.stderr, "Usage: gentypes.py   
>   "
> +sys.stderr.write("Usage: gentypes.py
>  \n")
>  sys.exit(1)
>  
>  (_, idlname, header, header_private, header_json, impl) = sys.argv
>  
>  (builtins,types) = idl.parse(idlname)
>  
> -print "outputting libxl type definitions to %s" % header
> +print("outputting libxl type definitions to %s" % header)

Note that print is *not* a function in python 2, unless you..

  from __future__ import print_function

...which should not be avoided.

It looks like print() just works as a function, but it's not, and I
would recommend against introducing this kind of misleading code.

In print("hallo"), the parentheses are part of an expression, like in
  ((1+ 1) * 2)

Other syntax with parentheses creates tuples.

Python 2.7.13 (>>> replaced with ===)

=== type(())

=== type((1+1))

=== type((1, 2))


=== print () <- tuple
()
=== print (1+1) <- expression
2
=== print (1, 2) <- tuple
(1, 2)

=== from __future__ import print_function
=== print()

=== print(1+1)
2
=== print(1, 2)
1 2

>  f = open(header, "w")
>  
> @@ -633,7 +633,7 @@ if __name__ == '__main__':
>  f.write("""#endif /* %s */\n""" % (header_define))
>  f.close()
>  
> -print "outputting libxl JSON definitions to %s" % header_json
> +print("outputting libxl JSON definitions to %s" % header_json)
>  
>  f = open(header_json, "w")
>  
> @@ -657,7 +657,7 @@ if __name__ == '__main__':
>  f.write("""#endif /* %s */\n""" % header_json_define)
>  f.close()
>  
> -print "outputting libxl type internal definitions to %s" % header_private
> +print("outputting libxl type internal definitions to %s" % 
> header_private)
>  
>  f = open(header_private, "w")
>  
> @@ -683,7 +683,7 @@ if __name__ == '__main__':
>  f.write("""#endif /* %s */\n""" % header_json_define)
>  f.close()
>  
> -print "outputting libxl type implementations to %s" % impl
> +print("outputting libxl type implementations to %s" % impl)
>  
>  f = open(impl, "w")
>  f.write("""
> diff --git a/tools/libxl/idl.py b/tools/libxl/idl.py
> index 2a7f3c44fe..b5bfc66b50 100644
> --- a/tools/libxl/idl.py
> +++ b/tools/libxl/idl.py
> @@ -11,7 +11,7 @@ DIR_BOTH = 3
>  _default_namespace = ""
>  def namespace(s):
>  if type(s) != str:
> -raise TypeError, "Require a string for the default namespace."
> +raise TypeError("Require a string for the default namespace.")
>  global _default_namespace
>  _default_namespace = s
>  
> @@ -346,7 +346,7 @@ class OrderedDict(dict):
>  return [(x,self[x]) for x in self.__ordered]
>  
>  def parse(f):
> -print >>sys.stderr, "Parsing %s" % f
> +sys.stderr.write("Parsing %s\n" % f)

And if you have it from future anyway...

print("Parsing", f, file=sys.stderr)

>  globs = {}
>  locs = OrderedDict()
> @@ -362,11 +362,10 @@ def parse(f):
>  globs[n] = t
>  
>  try:
> -execfile(f, globs, locs)
> -except SyntaxError,e:
> -raise SyntaxError, \
> -  "Errors were found at line %d while processing %s:\n\t%s"\
> -  %(e.lineno,f,e.text)
> +exec(compile(open(f).read(), f, 'exec'), globs, locs)
> +except SyntaxError as e:
> +raise SyntaxError("Errors were found at line %d while processing 
> %s:\n\t%s"\
> +  %(e.lineno,f,e.text))
>  
>  types = [t for t in locs.ordered_values() if isinstance(t,Type)]
>  
> 

Hans

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v11 5/9] xen/x86: use DECLARE_BOUNDS as required

2019-03-06 Thread Stefano Stabellini
On Wed, 6 Mar 2019, Jan Beulich wrote:
> >>> On 05.03.19 at 23:38,  wrote:
> > --- a/xen/arch/x86/percpu.c
> > +++ b/xen/arch/x86/percpu.c
> > @@ -13,7 +13,8 @@ unsigned long __per_cpu_offset[NR_CPUS];
> >   * context of PV guests.
> >   */
> >  #define INVALID_PERCPU_AREA (0x8000L - (long)__per_cpu_start)
> > -#define PERCPU_ORDER get_order_from_bytes(__per_cpu_data_end - 
> > __per_cpu_start)
> > +#define PERCPU_ORDER get_order_from_bytes(per_cpu_diff(__per_cpu_start,
> >  \
> > +   __per_cpu_data_end))
> 
> Please use _bytediff() when bytes are meant (i.e. also below, and
> perhaps elsewhere).

OK


> > @@ -600,7 +602,9 @@ static void noinline init_done(void)
> >  unregister_init_virtual_region();
> >  
> >  /* Zero the .init code and data. */
> > -for ( va = __init_begin; va < _p(__init_end); va += PAGE_SIZE )
> > +for ( va = (char *)__init_begin;
> > +  init_lt(va, __init_end);
> > +  va += PAGE_SIZE )
> 
> Is the line wrapping really needed here?

It would end at 80 characters exactly otherwise. I am happy to do as you
prefer.


> > --- a/xen/drivers/vpci/vpci.c
> > +++ b/xen/drivers/vpci/vpci.c
> > @@ -31,9 +31,9 @@ struct vpci_register {
> >  };
> >  
> >  #ifdef __XEN__
> > -extern vpci_register_init_t *const __start_vpci_array[];
> > -extern vpci_register_init_t *const __end_vpci_array[];
> > -#define NUM_VPCI_INIT (__end_vpci_array - __start_vpci_array)
> > +typedef vpci_register_init_t *const vpci_array_t;
> 
> You don't want to keep the const here - DECLARE_BOUNDS() will
> suitably add it.

OK


> Also how about vcpi_init_t or vpci_reg_init_t or some such? The
> defined type is not really an array after all.

OK


> > +DECLARE_BOUNDS(vpci_array, __start_vpci_array, __end_vpci_array);
> > +#define NUM_VPCI_INIT (vpci_array_diff(__start_vpci_array, 
> > __end_vpci_array))
> 
> Unnecessary outermost parentheses.

OK

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v11 6/9] xen/common: use DECLARE_BOUNDS as required

2019-03-06 Thread Stefano Stabellini
On Wed, 6 Mar 2019, Jan Beulich wrote:
> >>> On 05.03.19 at 23:38,  wrote:
> > --- a/xen/common/kernel.c
> > +++ b/xen/common/kernel.c
> > @@ -306,20 +306,25 @@ void add_taint(unsigned int flag)
> >  tainted |= flag;
> >  }
> >  
> > -extern const initcall_t __initcall_start[], __presmp_initcall_end[],
> > -__initcall_end[];
> > +DECLARE_ARRAY_BOUNDS(initcall);
> > +typedef initcall_t presmp_initcall_t;
> > +DECLARE_ARRAY_BOUNDS(presmp_initcall);
> >  
> >  void __init do_presmp_initcalls(void)
> >  {
> >  const initcall_t *call;
> > -for ( call = __initcall_start; call < __presmp_initcall_end; call++ )
> > +for ( call = __presmp_initcall_start;
> > + presmp_initcall_lt(call, __presmp_initcall_end);
> > + call++ )
> 
> Hard tabs slipped in. Also would you mind adding the missing blank line
> ahead of the one you modify?

Argh! I'll do.


> >  (*call)();
> >  }
> >  
> >  void __init do_initcalls(void)
> >  {
> >  const initcall_t *call;
> > -for ( call = __presmp_initcall_end; call < __initcall_end; call++ )
> > +for ( call = __initcall_start;
> > +  initcall_lt(call, __initcall_end);
> > +  call++ )
> 
> Again no need for splitting the line as it seems.

Actually it was 79 in the other case and 78 here, so I'll fix both.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [qemu-mainline baseline-only test] 83711: trouble: blocked/broken

2019-03-06 Thread Platform Team regression test user
This run is configured for baseline tests only.

flight 83711 qemu-mainline real [real]
http://osstest.xensource.com/osstest/logs/83711/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64  broken
 build-i386   broken
 build-armhf-pvopsbroken
 build-i386-xsm   broken
 build-amd64-xsm  broken
 build-amd64-pvopsbroken
 build-i386-pvops broken
 build-armhf  broken
 build-armhf-pvops 3 syslog-serverrunning
 build-armhf   3 syslog-serverrunning

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-shadow1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-midway1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-win10-i386  1 build-check(1)  blocked n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-pvshim 1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win10-i386  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1)blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-shadow 1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-a

Re: [Xen-devel] [PATCH v11 1/9] xen: use __UINTPTR_TYPE__ for uintptr_t

2019-03-06 Thread Stefano Stabellini
On Wed, 6 Mar 2019, Jan Beulich wrote:
> >>> On 05.03.19 at 23:38,  wrote:
> > Use __UINTPTR_TYPE__ to define uintptr_t. A later patch will make use of
> > __PTRDIFF_TYPE__ to define ptrdiff_t.
> > 
> > Signed-off-by: Stefano Stabellini 
> > Reviewed-by: Ian Jackson 
> 
> As before - I object to this change without the description supplying
> both a reason (which would better also explain why the current way
> of defining uintptr_t is detrimental) and a discussion why it is okay for
> us to use __UINTPTR_TYPE__, despite (at least) gcc making this
> available only under certain conditions (i.e. it would need to be
> confirmed that whatever the conditions they're always met for us).

Is the following good enough:

Use __UINTPTR_TYPE__ to define uintptr_t for consistency with ptrdiff_t.
__UINTPTR_TYPE__ is safe to use as it is a common predefined macro of
GNU C; it is defined to the correct underlying type as per GCC
documentation[1].

[1] https://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html

Also, it is not a good idea to use __mode__(__pointer__) to define
uintptr_t, because it relies on an obscurely defined GCC feature whose
semantics might be taken to imply that the things are really pointers.


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v11 5/9] xen/x86: use DECLARE_BOUNDS as required

2019-03-06 Thread Stefano Stabellini
On Wed, 6 Mar 2019, Jan Beulich wrote:
> >>> On 05.03.19 at 23:38,  wrote:
> > @@ -600,7 +602,9 @@ static void noinline init_done(void)
> >  unregister_init_virtual_region();
> >  
> >  /* Zero the .init code and data. */
> > -for ( va = __init_begin; va < _p(__init_end); va += PAGE_SIZE )
> > +for ( va = (char *)__init_begin;
> > +  init_lt(va, __init_end);
> > +  va += PAGE_SIZE )
> >  clear_page(va);
> 
> At the example of this - is casting away of const-ness not also a
> potential certification hindrance?

Darn, well spotted! No, it is not permitted (Rule 11.8).
In this case const is not correct because we actually modify the object
few lines below:

  base->priv = 1;


This is problematic. We have also the following instances in this series
to deal with:

xen/arch/arm/percpu.c:_free_percpu_area
  char *p = (char *)__per_cpu_start + __per_cpu_offset[cpu];

xen/arch/x86/percpu.c:_free_percpu_area
  char *p = (char *)__per_cpu_start + __per_cpu_offset[cpu];

xen/arch/x86/setup.c:init_done
  for ( va = (char *)__init_begin; init_lt(va, __init_end); va += PAGE_SIZE )

xen/arch/x86/alternative.c:apply_alternatives
  for ( a = base = (struct alt_instr *)start; alt_instr_lt(a, end); a++ )


In all these cases we actually end up modifying the object. I suggest
we remove the const from either __DECLARE_BOUNDS (so from everywhere),
or just for per_cpu, init, and alt_instr by introducing another MACRO.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v11 8/9] xen: use DECLARE_BOUNDS in alternative.c

2019-03-06 Thread Stefano Stabellini
On Wed, 6 Mar 2019, Jan Beulich wrote:
> >>> On 05.03.19 at 23:38,  wrote:
> > @@ -193,8 +191,10 @@ void init_or_livepatch apply_alternatives(struct 
> > alt_instr *start,
> 
> Seeing this relevant part of the function parameters, ...
> 
> >   *
> >   * So be careful if you want to change the scan order to any other
> >   * order.
> > + *
> > + * start and end could be pointers to different objects.
> >   */
> > -for ( a = base = start; a < end; a++ )
> > +for ( a = base = (struct alt_instr *)start; alt_instr_lt(a, end); a++ )
> 
> ... why the cast?

To remove const (see the other email about the problematic sites).

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-4.8-testing test] 133598: regressions - FAIL

2019-03-06 Thread osstest service owner
flight 133598 xen-4.8-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133598/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-xtf-amd64-amd64-2 50 xtf/test-hvm64-lbr-tsx-vmentry fail REGR. vs. 130965
 test-xtf-amd64-amd64-2   69 xtf/test-hvm64-xsa-278   fail REGR. vs. 130965
 test-amd64-amd64-libvirt-pair 22 guest-migrate/src_host/dst_host fail REGR. 
vs. 130965
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 16 guest-localmigrate/x10 fail REGR. 
vs. 130965

Tests which did not succeed, but are not blocking:
 test-xtf-amd64-amd64-3  50 xtf/test-hvm64-lbr-tsx-vmentry fail like 130965
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 130965
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 130965
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 130965
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 130965
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 130965
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 130965
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 130965
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 130965
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 130965
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop fail like 130965
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass

version targeted for testing:
 xen  a1f8fe062899dca34fe2353ea27c6348c5d7cd7d
baseline version:
 xen  90

Re: [Xen-devel] [PATCH 0/6] iomem cacheability

2019-03-06 Thread Stefano Stabellini
On Wed, 6 Mar 2019, Amit Tomer wrote:
> Hi,
> > Looking at the stack trace, this is very likely due to the issue I pointed 
> > out earlier on. I.e reserved-memory region should be described in the 
> > memory nodes.
> 
> Do you mean, something like this(reserved-memory node is under memory node) ?

No, I think Julien meant that all the reserved-memory regions ranges
should be "covered" by the ranges of the memory node.

If a reserved-memory node range is 0x5400 - 0x5700, then we need
to make sure that the memory node includes that range. In your case, the
first memory bank is 0x4800 - 0x8000 so it would clearly include
the range 0x5400 - 0x5700. So, it looks like it is correct from
that point of view.

I am suspecting there is a bug in the patch "xen/arm: map
reserved-memory regions as normal memory in dom0", specifically in the
implementation of check_reserved_memory.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 0/6] iomem cacheability

2019-03-06 Thread Julien Grall
Hi,

On 06/03/2019 22:42, Stefano Stabellini wrote:
> On Wed, 6 Mar 2019, Amit Tomer wrote:
>> Hi,
>>> Looking at the stack trace, this is very likely due to the issue I pointed 
>>> out earlier on. I.e reserved-memory region should be described in the 
>>> memory nodes.
>>
>> Do you mean, something like this(reserved-memory node is under memory node) ?
> 
> No, I think Julien meant that all the reserved-memory regions ranges
> should be "covered" by the ranges of the memory node.
> 
> If a reserved-memory node range is 0x5400 - 0x5700, then we need
> to make sure that the memory node includes that range. In your case, the
> first memory bank is 0x4800 - 0x8000 so it would clearly include
> the range 0x5400 - 0x5700. So, it looks like it is correct from
> that point of view.

Not really, the DT provided by Amit is for the host. The memory node 
will be rewritten by Xen for Dom0  and does not include the 
reserved-memory regions so far.

> 
> I am suspecting there is a bug in the patch "xen/arm: map
> reserved-memory regions as normal memory in dom0", specifically in the
> implementation of check_reserved_memory.

I don't think the issue is in that code (see above).

Cheers,

-- 
Julien Grall
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [linux-4.9 test] 133600: tolerable FAIL - PUSHED

2019-03-06 Thread osstest service owner
flight 133600 linux-4.9 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133600/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 133491
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 133491
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 133491
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 133491
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 133491
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvhv2-intel 12 guest-start fail never pass
 test-amd64-amd64-xl-pvhv2-amd 12 guest-start  fail  never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass

version targeted for testing:
 linuxf422a02f865a93f9d3db0d8f2de08aab455fd1dc
baseline version:
 linux5507839a723e4edeed4efda2fa2249c4713fe0bb

Last test of basis   133491  2019-03-01 04:56:59 Z5 days
Testing same since   133600  2019-03-05 17:11:40 Z1 days1 attempts

-

Re: [Xen-devel] [PATCH v4.1 4/6] xen/x86: Allow stubdom access to irq created for msi.

2019-03-06 Thread Marek Marczykowski-Górecki
On Fri, Feb 22, 2019 at 11:42:22AM +0100, Roger Pau Monné wrote:
> On Thu, Feb 21, 2019 at 06:40:40PM +0100, Marek Marczykowski-Górecki wrote:
> > On Thu, Feb 21, 2019 at 05:47:51PM +0100, Roger Pau Monné wrote:
> > > On Fri, Feb 08, 2019 at 11:17:05AM +0100, Marek Marczykowski-Górecki 
> > > wrote:
> > > >  {
> > > >  int irq, ret;
> > > >  struct irq_desc *desc;
> > > > @@ -190,19 +190,19 @@ int create_irq(nodeid_t node)
> > > >  desc->arch.used = IRQ_UNUSED;
> > > >  irq = ret;
> > > >  }
> > > > -else if ( hardware_domain )
> > > > +else if ( dm_domain )
> > > 
> > > Can you guarantee that dm_domain is always current->domain?
> > 
> > No, in some cases it will be hardware_domain.
> 
> Right, but in that case current->domain == hardware_domain I guess?
> 
> > 
> > > I think you need to assert for this, or else take a reference to
> > > dm_domain (get_domain) before accessing it's fields, or else you risk
> > > the domain being destroyed while modifying it's fields.
> > 
> > Can hardware_domain be destroyed without panicking Xen? If so,
> > get_domain would be indeed needed.
> 
> What about other callers that are not the hardware_domain? You need to
> make sure those domains are not destroyed while {create/destroy}_irq
> is changing the permissions.
> 
> If you can guarantee that dm_domain == current->domain then you are
> safe, if not you need to get a reference before modifying any fields
> of the domain struct.
> 
> So IMO you either need to add an assert or a get_domain depending on
> the answer to the question above.

I've added an assert, and I think I have the answer to the above question:

(XEN) Assertion 'd == current->domain' failed at irq.c:232
(XEN) [ Xen-4.12.0-rc  x86_64  debug=y   Not tainted ]
(XEN) CPU:2
(XEN) RIP:e008:[] destroy_irq+0x126/0x143
(XEN) RFLAGS: 00010206   CONTEXT: hypervisor
(...)
(XEN) Xen call trace:
(XEN)[] destroy_irq+0x126/0x143
(XEN)[] msi_free_irq+0x51/0x1b8
(XEN)[] unmap_domain_pirq+0x487/0x4d4
(XEN)[] free_domain_pirqs+0x71/0x8f
(XEN)[] arch_domain_destroy+0x41/0xa1
(XEN)[] domain.c#complete_domain_destroy+0x86/0x159
(XEN)[] rcupdate.c#rcu_process_callbacks+0xa5/0x1cc
(XEN)[] softirq.c#__do_softirq+0x78/0x9a
(XEN)[] do_softirq+0x13/0x15
(XEN)[] domain.c#idle_loop+0x63/0xb9

In this case, current->domain obviously isn't the stubdomain, because at
this point it is already destroyed (it keeps reference to the target
domain, so target domain couldn't be destroyed before its stubdomain).

In fact, in this case get_dm_domain() returns wrong value, since it
isn't called by device model. At the point where stubdomain is already
destroyed, I think it should return NULL, but it returns
hardware_domain. But it isn't that easy, because target domain don't
have any reference to its stubdomain. Especially already destroyed one.

I's defined as:

static struct domain *get_dm_domain(struct domain *d)
{
return current->domain->target == d ? current->domain :
  hardware_domain;
}

Since hardware domain couldn't be destroyed(*) while the system is
running, in practice this wrong return value it isn't a problem.
Hardware didn't have permission over this IRQ (if stubdomain have
created it), so irq_deny_access is a no-op.

So, I would adjust assert in destroy_irq to allow also hardware_domain,
and add a comment that get_dm_domain may return hardware_domain during
domain destruction. Is that ok?

(*) is this actually true? I see shutdown_domain(hardware_domain)
cause Xen to shutdown. But I don't see anything like this in
domain_kill/domain_destroy. Normally no other domain than dom0 is able
to destroy other domains (and domain can't destroy itself), so it should
be ok. But with some XSM policy, I think it could be possible to allow
other domain to destroy hardware domain. In fact, just having hardware
domain != dom0 is enough to violate this assumption.
Am I missing anything?

Full crash message:
(XEN) Assertion 'd == current->domain' failed at irq.c:232
(XEN) [ Xen-4.12.0-rc  x86_64  debug=y   Not tainted ]
(XEN) CPU:2
(XEN) RIP:e008:[] destroy_irq+0x126/0x143
(XEN) RFLAGS: 00010206   CONTEXT: hypervisor
(XEN) rax: 830422264000   rbx: 001e   rcx: 
(XEN) rdx: 8304222de000   rsi: 830422235000   rdi: 001e
(XEN) rbp: 83042225fcc0   rsp: 83042225fc90   r8:  
(XEN) r9:  830385868880   r10: 0004   r11: 8304222ba010
(XEN) r12: 830422235000   r13: 001e   r14: 83042228
(XEN) r15: 83042d8d1000   cr0: 8005003b   cr4: 000426e0
(XEN) cr3: ca49d000   cr2: 88011a179c00
(XEN) fsb:    gsb: 88013564   gss: 
(XEN) ds: 002b   es: 002b   fs:    gs:    ss: e010   c

[Xen-devel] [xen-4.7-testing baseline-only test] 83712: trouble: blocked/broken

2019-03-06 Thread Platform Team regression test user
This run is configured for baseline tests only.

flight 83712 xen-4.7-testing real [real]
http://osstest.xensource.com/osstest/logs/83712/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64  broken
 build-amd64-prev broken
 build-i386   broken
 build-armhf-pvopsbroken
 build-i386-xsm   broken
 build-amd64-xtf  broken
 build-amd64-xsm  broken
 build-amd64-pvopsbroken
 build-i386-pvops broken
 build-armhf  broken
 build-i386-prev  broken
 build-armhf-pvops 4 host-install(4) broken REGR. vs. 75629
 build-armhf   4 host-install(4) broken REGR. vs. 75629
 build-i386-xsm4 host-install(4) broken REGR. vs. 75629
 build-amd64   4 host-install(4) broken REGR. vs. 75629
 build-amd64-pvops 4 host-install(4) broken REGR. vs. 75629
 build-amd64-xsm   4 host-install(4) broken REGR. vs. 75629
 build-amd64-xtf   4 host-install(4) broken REGR. vs. 75629
 build-i386-pvops  4 host-install(4) broken REGR. vs. 75629
 build-amd64-prev  4 host-install(4) broken REGR. vs. 75629
 build-i3864 host-install(4) broken REGR. vs. 75629
 build-i386-prev   4 host-install(4) broken REGR. vs. 75629
 build-armhf-pvops 3 syslog-serverrunning
 build-armhf   3 syslog-serverrunning

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-xtf-amd64-amd64-11 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-shadow1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-midway1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm  1 build-check(1) blocked n/a
 test-amd64-amd64-migrupgrade  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-win10-i386  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked 
n/a
 test-amd64-i386-xl-qemuu-win10-i386  1 build-check(1)  blocked n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qemut-win10-i386  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-qemut-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 build-i386-rumprun1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win10-i386  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-xtf-amd64-amd64-21 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 build-amd64-rumprun   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-am

[Xen-devel] [linux-4.14 test] 133601: regressions - FAIL

2019-03-06 Thread osstest service owner
flight 133601 linux-4.14 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133601/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-arndale   5 host-ping-check-native   fail REGR. vs. 133508
 test-armhf-armhf-libvirt-raw 15 guest-start/debian.repeat fail REGR. vs. 133508

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass

version targeted for testing:
 linux99403097be0cbe12042775d9ca3a66f2018adc3e
baseline version:
 linux30921fc1e5fcf904f9afddeece1288f5b16ba017

Last test of basis   133508  2019-03-01 22:06:06 Z5 days
Testing same since   133601  2019-03-05 17:11:57 Z1 days1 attempts


People who touched revisions under test:
  Aaron Hill 
  Alex Deucher 
  Andrew F. Davis 
  Andy Lutomirski 
  Atsushi Nemoto 
  Balaji Pothunoori 
  Bo He 
  Bob Copeland 
  Bob Copeland 
  Borislav Petkov 
  BOUGH CHEN 
  Chaitanya Tata 
  Chaitanya Tata 
  Dan Car

[Xen-devel] [PATCH] xen, cpu_hotplug: Prevent an out of bounds access

2019-03-06 Thread Dan Carpenter
The "cpu" variable comes from the sscanf() so Smatch marks it as
untrusted data.  We can't pass a higher value than "nr_cpu_ids" to
cpu_possible() or it results in an out of bounds access.

Fixes: d68d82afd4c8 ("xen: implement CPU hotplugging")
Signed-off-by: Dan Carpenter 
---
 drivers/xen/cpu_hotplug.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/xen/cpu_hotplug.c b/drivers/xen/cpu_hotplug.c
index b1357aa4bc55..f192b6f42da9 100644
--- a/drivers/xen/cpu_hotplug.c
+++ b/drivers/xen/cpu_hotplug.c
@@ -54,7 +54,7 @@ static int vcpu_online(unsigned int cpu)
 }
 static void vcpu_hotplug(unsigned int cpu)
 {
-   if (!cpu_possible(cpu))
+   if (cpu >= nr_cpu_ids || !cpu_possible(cpu))
return;
 
switch (vcpu_online(cpu)) {
-- 
2.17.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen, cpu_hotplug: Prevent an out of bounds access

2019-03-06 Thread Juergen Gross
On 07/03/2019 06:41, Dan Carpenter wrote:
> The "cpu" variable comes from the sscanf() so Smatch marks it as
> untrusted data.  We can't pass a higher value than "nr_cpu_ids" to
> cpu_possible() or it results in an out of bounds access.
> 
> Fixes: d68d82afd4c8 ("xen: implement CPU hotplugging")
> Signed-off-by: Dan Carpenter 

Reviewed-by: Juergen Gross 


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel