Re: [Xen-devel] [PATCH v3 07/15] argo: implement the register op

2019-01-14 Thread Christopher Clark
On Mon, Jan 14, 2019 at 6:19 AM Jan Beulich  wrote:
>
> >>> On 07.01.19 at 08:42,  wrote:
> > --- a/xen/common/argo.c
> > +++ b/xen/common/argo.c
> > @@ -23,16 +23,41 @@
> >  #include 
> >  #include 
> >  #include 
> > +#include 
> > +#include 
> >  #include 
> >  #include 
> >
> > +#define MAX_RINGS_PER_DOMAIN128U
> > +
> > +/* All messages on the ring are padded to a multiple of the slot size. */
> > +#define ROUNDUP_MESSAGE(a) (ROUNDUP((a), XEN_ARGO_MSG_SLOT_SIZE))
>
> Pointless outermost pair of parentheses.

ack, removed

>
> > @@ -198,6 +223,31 @@ static DEFINE_RWLOCK(argo_lock); /* L1 */
> >  #define argo_dprintk(format, ... ) ((void)0)
> >  #endif
> >
> > +/*
> > + * This hash function is used to distribute rings within the per-domain
> > + * hash tables (d->argo->ring_hash and d->argo_send_hash). The hash table
> > + * will provide a struct if a match is found with a 'argo_ring_id' key:
> > + * ie. the key is a (domain id, port, partner domain id) tuple.
> > + * Since port number varies the most in expected use, and the Linux driver
> > + * allocates at both the high and low ends, incorporate high and low bits 
> > to
> > + * help with distribution.
> > + * Apply array_index_nospec as a defensive measure since this operates
> > + * on user-supplied input and the array size that it indexes into is known.
> > + */
> > +static unsigned int
> > +hash_index(const struct argo_ring_id *id)
> > +{
> > +unsigned int hash;
> > +
> > +hash = (uint16_t)(id->port >> 16);
> > +hash ^= (uint16_t)id->port;
>
> I may have asked this before, but are the casts really needed
> with ...
>
> > +hash ^= id->domain_id;
> > +hash ^= id->partner_id;
> > +hash &= (ARGO_HTABLE_SIZE - 1);
>
> ... the masking done here?
>
> > +return array_index_nospec(hash, ARGO_HTABLE_SIZE);
>
> With the masking above - is this really needed?
>
> And then the question is whether the quality of the hash is
> sufficient: There won't be more set bits in the result than
> are in any of the three input values, so if they're all small,
> higher hash table entries won't be used at all. I would
> assume the goal to be that by the time 32 entities appear,
> chances be good that at least about 30 of the hash table
> entries are in use.

ok, I'll replace this function and address the above.
I'm out of time today so have added a FIXME for today's series posting
and will get it done tomorrow.

>
> > @@ -219,6 +269,78 @@ ring_unmap(struct argo_ring_info *ring_info)
> >  }
> >  }
> >
> > +static int
> > +ring_map_page(struct argo_ring_info *ring_info, unsigned int i, void 
> > **out_ptr)
> > +{
> > +if ( i >= ring_info->nmfns )
> > +{
> > +gprintk(XENLOG_ERR,
> > +   "argo: ring (vm%u:%x vm%d) %p attempted to map page  %u of 
> > %u\n",
>
> ring_info->id.{domain,partner}_id look to be of the same type -
> why once %u and once %d? Same elsewhere.

Fixed across the series to use %u for domid_t output.

>
> > +ring_info->id.domain_id, ring_info->id.port,
> > +ring_info->id.partner_id, ring_info, i, ring_info->nmfns);
> > +return -ENOMEM;
> > +}
>
> i = array_index_nospec(i, ring_info->nmfns);
>
> considering the array indexes here? Of course at this point only
> zero can be passed in, but I assume this changes in later patches
> and the index is at least indirectly guest controlled.

Added, thanks.

>
> > @@ -371,6 +493,418 @@ partner_rings_remove(struct domain *src_d)
> >  }
> >  }
> >
> > +static int
> > +find_ring_mfn(struct domain *d, gfn_t gfn, mfn_t *mfn)
>
> So you have find_ring_mfn(), find_ring_mfns(), and ring_find_info().
> Any chance you could use a consistent ordering of "ring" and "find"?
> Or is there a reason behind the apparent mismatch?

I've renamed them to use 'find_' as the common prefix. Look cleaner to
me. thanks.

>
> > +{
> > +p2m_type_t p2mt;
> > +int ret = 0;
> > +
> > +#ifdef CONFIG_X86
> > +*mfn = get_gfn_unshare(d, gfn_x(gfn), );
> > +#else
> > +*mfn = p2m_lookup(d, gfn, );
> > +#endif
> > +
> > +if ( !mfn_valid(*mfn) )
> > +ret = -EINVAL;
> > +#ifdef CONFIG_X86
> > +else if ( p2m_is_paging(p2mt) || (p2mt == p2m_ram_logdirty) )
> > +ret = -EAGAIN;
> > +#endif
> > +else if ( (p2mt != p2m_ram_rw) ||
> > +  !get_page_and_type(mfn_to_page(*mfn), d, PGT_writable_page) )
> > +ret = -EINVAL;
> > +
> > +#ifdef CONFIG_X86
> > +put_gfn(d, gfn_x(gfn));
> > +#endif
> > +
> > +return ret;
> > +}
>
> Please check whether you could leverage check_get_page_from_gfn()
> here. If you can't, please at least take inspiration as to e.g. the
> #ifdef-s from that function.

Have added a temporary FIXME for this and will do this tomorrow.

>
> > +static int
> > +find_ring_mfns(struct domain *d, struct argo_ring_info *ring_info,
> > +   uint32_t npage,
> > +   XEN_GUEST_HANDLE_PARAM(xen_argo_page_descr_t) pg_descr_hnd,
> > +  

Re: [Xen-devel] [PATCH v3 04/15] argo: init, destroy and soft-reset, with enable command line opt

2019-01-14 Thread Christopher Clark
On Mon, Jan 14, 2019 at 7:12 AM Jan Beulich  wrote:
>
> >>> On 14.01.19 at 15:58,  wrote:
> > On 07/01/2019 07:42, Christopher Clark wrote:
> >> --- a/xen/common/argo.c
> >> +++ b/xen/common/argo.c
> >>  long
> >>  do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1,
> >
> > I know I'm commenting on the wrong patch, but please use unsigned long
> > cmd, so the type definition here doesn't truncate the caller provided
> > value.  We have similar buggy code all over Xen, but its too late to fix
> > that, and I'd prefer not to propagate the error.
>
> Why buggy? It all depends on how the interface is specified. If
> the input is 32 bits wide, it is clear that higher bits are supposed
> to be ignored. Nothing says that the full register width is
> significant.

I've left this as is (ie. unsigned int) but I can change it if it should change.

>
> >> XEN_GUEST_HANDLE_PARAM(void) arg2, unsigned long arg3,
> >> unsigned long arg4)
> >>  {
> >> -return -ENOSYS;
> >> +struct domain *currd = current->domain;
> >> +long rc = -EFAULT;
> >> +
> >> +argo_dprintk("->do_argo_op(%u,%p,%p,%d,%d)\n", cmd,
> >> + (void *)arg1.p, (void *)arg2.p, (int) arg3, (int) arg4);
> >
> > For debugging purposes, you don't want to truncate any of these values,
> > or you'll have a print message which doesn't match what the guest
> > provided.  I'd use %ld for arg3 and arg4.
>
> Perhaps better %lx, for the output being easier to recognize
> for both bitmaps (e.g. flag values) and sufficiently large values.

ack, done

>
> >> +
> >> +if ( unlikely(!opt_argo_enabled) )
> >> +{
> >> +rc = -EOPNOTSUPP;
> >
> > Shouldn't this be ENOSYS instead?  There isn't a conceptual difference
> > between CONFIG_ARGO compiled out, and opt_argo clear on the command
> > line, and I don't think a guest should be able to tell the difference.
>
> I admit it's a boundary case, but I think ENOSYS should strictly
> only ever be (and have been) returned for unrecognized major
> hypercall numbers.

ack

Christopher

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 04/15] argo: init, destroy and soft-reset, with enable command line opt

2019-01-14 Thread Christopher Clark
On Mon, Jan 14, 2019 at 6:58 AM Andrew Cooper  wrote:
>
> On 07/01/2019 07:42, Christopher Clark wrote:
> > Initialises basic data structures and performs teardown of argo state
> > for domain shutdown.
> >
> > Inclusion of the Argo implementation is dependent on CONFIG_ARGO.
> >
> > Introduces a new Xen command line parameter 'argo': bool to enable/disable
> > the argo hypercall. Defaults to disabled.
> >
> > New headers:
> >   public/argo.h: with definions of addresses and ring structure, including
> >   indexes for atomic update for communication between domain and hypervisor.
> >
> >   xen/argo.h: to expose the hooks for integration into domain lifecycle:
> > argo_init: per-domain init of argo data structures for domain_create.
> > argo_destroy: teardown for domain_destroy and the error exit
> >   path of domain_create.
> > argo_soft_reset: reset of domain state for domain_soft_reset.
> >
> > Adds two new fields to struct domain:
> > rwlock_t argo_lock;
> > struct argo_domain *argo;
> >
> > In accordance with recent work on _domain_destroy, argo_destroy is
> > idempotent. It will tear down: all rings registered by this domain, all
> > rings where this domain is the single sender (ie. specified partner,
> > non-wildcard rings), and all pending notifications where this domain is
> > awaiting signal about available space in the rings of other domains.
> >
> > A count will be maintained of the number of rings that a domain has
> > registered in order to limit it below the fixed maximum limit defined here.
> >
> > The software license on the public header is the BSD license, standard
> > procedure for the public Xen headers. The public header was originally
> > posted under a GPL license at: [1]:
> > https://lists.xenproject.org/archives/html/xen-devel/2013-05/msg02710.html
> >
> > The following ACK by Lars Kurth is to confirm that only people being
> > employees of Citrix contributed to the header files in the series posted at
> > [1] and that thus the copyright of the files in question is fully owned by
> > Citrix. The ACK also confirms that Citrix is happy for the header files to
> > be published under a BSD license in this series (which is based on [1]).
> >
> > Signed-off-by: Christopher Clark 
> > Acked-by: Lars Kurth 
>
> I hope I've not trodden on the toes of any other reviews.  I've got some
> minor requests, but hopefully its all fairly trivial.
>
> > diff --git a/docs/misc/xen-command-line.pandoc 
> > b/docs/misc/xen-command-line.pandoc
> > index a755a67..aea13eb 100644
> > --- a/docs/misc/xen-command-line.pandoc
> > +++ b/docs/misc/xen-command-line.pandoc
> > @@ -182,6 +182,17 @@ Permit Xen to use "Always Running APIC Timer" support 
> > on compatible hardware
> >  in combination with cpuidle.  This option is only expected to be useful for
> >  developers wishing Xen to fall back to older timing methods on newer 
> > hardware.
> >
> > +### argo
> > +> `= `
> > +
> > +> Default: `false`
> > +
> > +Enable the Argo hypervisor-mediated interdomain communication mechanism.
> > +
> > +This allows domains access to the Argo hypercall, which supports 
> > registration
> > +of memory rings with the hypervisor to receive messages, sending messages 
> > to
> > +other domains by hypercall and querying the ring status of other domains.
>
> Please do include a note about CONFIG_ARGO.  I know this doc is
> inconsistent on the matter (as Kconfig postdates the written entries
> here), but I have been trying to fix up, and now about half of the
> documentation does mention appropriate Kconfig information.

Ack, note added.

>
> > diff --git a/xen/common/argo.c b/xen/common/argo.c
> > index 6f782f7..86195d3 100644
> > --- a/xen/common/argo.c
> > +++ b/xen/common/argo.c
> >  long
> >  do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1,
>
> I know I'm commenting on the wrong patch, but please use unsigned long
> cmd, so the type definition here doesn't truncate the caller provided
> value.  We have similar buggy code all over Xen, but its too late to fix
> that, and I'd prefer not to propagate the error.

On this one, given Jan's reply, I've left it as is for the series that
I'll push tonight. That patch is carrying an Ack from Jan at the
moment, so I wasn't going to touch it if it's not required. If there's
consensus that it should change, let me know and I'll switch it.

>
> > XEN_GUEST_HANDLE_PARAM(void) arg2, unsigned long arg3,
> > unsigned long arg4)
> >  {
> > -return -ENOSYS;
> > +struct domain *currd = current->domain;
> > +long rc = -EFAULT;
> > +
> > +argo_dprintk("->do_argo_op(%u,%p,%p,%d,%d)\n", cmd,
> > + (void *)arg1.p, (void *)arg2.p, (int) arg3, (int) arg4);
>
> For debugging purposes, you don't want to truncate any of these values,
> or you'll have a print message which doesn't match what the guest
> provided.  I'd use %ld for arg3 and arg4.

I've gone with %lu for arg3 and %lx for arg4, for the 

Re: [Xen-devel] [PATCH v3 10/15] argo: implement the notify op

2019-01-14 Thread Christopher Clark
On Thu, Jan 10, 2019 at 4:22 AM Roger Pau Monné  wrote:
>
>  On Mon, Jan 7, 2019 at 8:44 AM Christopher Clark
>  wrote:
> >
> > Queries for data about space availability in registered rings and
> > causes notification to be sent when space has become available.
> >
> > The hypercall op populates a supplied data structure with information about
> > ring state, and if insufficent space is currently available in a given ring,
>
> insufficient

ack, fixed

> > the hypervisor will record the domain's expressed interest and notify it
> > when it observes that space has become available.
> >
> > Checks for free space occur when this notify op is invoked, so it may be
> > intentionally invoked with no data structure to populate
> > (ie. a NULL argument) to trigger such a check and consequent notifications.
> >
> > Limit the maximum number of notify requests in a single operation to a
> > simple fixed limit of 256.

> >
> > +static struct argo_ring_info *
> > +ring_find_info(const struct domain *d, const struct argo_ring_id *id);
> > +
> > +static struct argo_ring_info *
> > +ring_find_info_by_match(const struct domain *d, uint32_t port,
> > +domid_t partner_id);
>
> Can you place the static functions such that you don't need prototypes for 
> them?

Ack, yes. These have now been pulled to the top of the file and the
prototypes removed.

> > +
> >  /*
> >   * This hash function is used to distribute rings within the per-domain
> >   * hash tables (d->argo->ring_hash and d->argo_send_hash). The hash table
> > @@ -265,6 +275,17 @@ signal_domain(struct domain *d)
> >  }
> >
> >  static void
> > +signal_domid(domid_t domain_id)
> > +{
> > +struct domain *d = get_domain_by_id(domain_id);
>
> Newline.

ack

>
> > +if ( !d )
> > +return;
> > +
> > +signal_domain(d);
> > +put_domain(d);
> > +}
> > +
> > +static void
> >  ring_unmap(struct argo_ring_info *ring_info)
> >  {
> >  unsigned int i;
> > @@ -473,6 +494,62 @@ get_sanitized_ring(xen_argo_ring_t *ring, struct 
> > argo_ring_info *ring_info)
> >  return 0;
> >  }
> >
> > +static uint32_t
> > +ringbuf_payload_space(struct domain *d, struct argo_ring_info *ring_info)
> > +{
> > +xen_argo_ring_t ring;
> > +uint32_t len;
> > +int32_t ret;
>
> You use a signed type to internally store the return value, but the
> return type from the function itself is unsigned. Is it guaranteed
> that ret < INT32_MAX always?

It is, yes, so I've added a explanatory comment:

/*
 * In a sanitized ring, we can rely on:
 *  (rx_ptr < ring_info->len)   &&
 *  (tx_ptr < ring_info->len)   &&
 *  (ring_info->len <= XEN_ARGO_MAX_RING_SIZE)
 *
 * and since: XEN_ARGO_MAX_RING_SIZE < INT32_MAX
 * therefore right here: ret < INT32_MAX
 * and we are safe to return it as a unsigned value from this function.
 * The subtractions below cannot increase its value.
 */

>
> > +
> > +ASSERT(spin_is_locked(_info->lock));
> > +
> > +len = ring_info->len;
> > +if ( !len )
> > +return 0;
> > +
> > +ret = get_sanitized_ring(, ring_info);
> > +if ( ret )
> > +return 0;
> > +
> > +argo_dprintk("sanitized ringbuf_payload_space: tx_ptr=%d rx_ptr=%d\n",
> > + ring.tx_ptr, ring.rx_ptr);
> > +
> > +/*
> > + * rx_ptr == tx_ptr means that the ring has been emptied, so return
> > + * the maximum payload size that can be accepted -- see message size
> > + * checking logic in the entry to ringbuf_insert which ensures that
> > + * there is always one message slot (of size ROUNDUP_MESSAGE(1)) left
> > + * available, preventing a ring from being entirely filled. This 
> > ensures
> > + * that matching ring indexes always indicate an empty ring and not a
> > + * full one.
> > + * The subtraction here will not underflow due to minimum size 
> > constraints
> > + * enforced on ring size elsewhere.
> > + */
> > +if ( ring.rx_ptr == ring.tx_ptr )
> > +return len - sizeof(struct xen_argo_ring_message_header)
> > +   - ROUNDUP_MESSAGE(1);
>
> Why not do something like:
>
> ret = ring.rx_ptr - ring.tx_ptr;
> if ( ret <= 0)
> ret += len;
>
> Instead of this early return?
>
> The only difference when the ring is full is that len should be used
> instead of the ptr difference.

That's much nicer -- thanks. Done.

> >
> >  static int
> > +fill_ring_data(const struct domain *currd,
> > +   XEN_GUEST_HANDLE(xen_argo_ring_data_ent_t) data_ent_hnd)
> > +{
> > +xen_argo_ring_data_ent_t ent;
> > +struct domain *dst_d;
> > +struct argo_ring_info *ring_info;
> > +int ret;
> > +
> > +ASSERT(rw_is_locked(_lock));
> > +
> > +ret = __copy_from_guest(, data_ent_hnd, 1) ? -EFAULT : 0;
> > +if ( ret )
> > +goto out;
>
> if ( __copy_from_guest(, data_ent_hnd, 1) )
> return -EFAULT;
>
> And you can get rid of the out label.

[Xen-devel] [linux-4.4 test] 131953: trouble: broken/fail/pass

2019-01-14 Thread osstest service owner
flight 131953 linux-4.4 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/131953/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-credit1  broken
 test-amd64-i386-xl-xsm   broken
 test-amd64-amd64-xl-credit2  broken
 test-amd64-amd64-libvirt-vhd broken
 test-amd64-i386-freebsd10-i386 broken
 test-amd64-i386-xl-raw   broken
 test-amd64-i386-xl   broken
 test-amd64-i386-xl-qemuu-ovmf-amd64   broken in 131939
 test-amd64-amd64-rumprun-amd64broken in 131939
 test-amd64-amd64-amd64-pvgrub broken in 131939
 test-amd64-amd64-xl-qemut-win10-i386  broken in 131939
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm  broken in 
131939
 test-amd64-i386-xl-shadowbroken  in 131939
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm broken in 131939
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm  broken in 131939

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-qemuu-ovmf-amd64 4 host-install(4) broken in 131939 pass in 
131953
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 4 host-install(4) broken in 
131939 pass in 131953
 test-amd64-amd64-amd64-pvgrub 4 host-install(4) broken in 131939 pass in 131953
 test-amd64-amd64-xl-qemut-win10-i386 4 host-install(4) broken in 131939 pass 
in 131953
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 4 host-install(4) broken 
in 131939 pass in 131953
 test-amd64-amd64-rumprun-amd64 4 host-install(4) broken in 131939 pass in 
131953
 test-amd64-i386-xl-shadow4 host-install(4) broken in 131939 pass in 131953
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 4 host-install(4) broken in 
131939 pass in 131953
 test-amd64-amd64-xl-credit2   4 host-install(4)  broken pass in 131939
 test-amd64-i386-freebsd10-i386  4 host-install(4)broken pass in 131939
 test-amd64-amd64-xl-credit1   4 host-install(4)  broken pass in 131939
 test-amd64-i386-xl-xsm4 host-install(4)  broken pass in 131939
 test-amd64-i386-xl-raw4 host-install(4)  broken pass in 131939
 test-amd64-amd64-libvirt-vhd  4 host-install(4)  broken pass in 131939
 test-amd64-i386-xl4 host-install(4)  broken pass in 131939
 test-armhf-armhf-libvirt 5 host-ping-check-native fail in 131939 pass in 131953
 test-amd64-i386-pair 10 xen-boot/src_host  fail pass in 131939
 test-amd64-i386-pair 11 xen-boot/dst_host  fail pass in 131939
 test-armhf-armhf-examine  8 reboot fail pass in 131939

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt-vhd 12 migrate-support-check fail in 131939 never pass
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvhv2-amd 12 guest-start  fail  never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvhv2-intel 12 guest-start fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-check  

[Xen-devel] Hikey: Enable Xen + Mainline Linux Kernel

2019-01-14 Thread Leo Yan
Hi all,

Sorry I bring up this question at here if it's duplicate due I googled
and found on the mailing list there have several related discussion for
enabling Xen on Hikey (I use Hikey but not Hikey960); but what I saw it
seems a new issue.

Below is the configuration at my side:

- Linux kernel: mainline kernel with 5.0-rc2;

  make defconfig
  make -j16 Image dtbs

- Xen: latest code base

  git clone git://xenbits.xen.org/xen.git
  cd xen
  make dist-xen XEN_TARGET_ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-

- Prepare my boot image:

  $ mount -o loop,rw,sync boot-linux.uefi.img boot-fat
  $ cp Image hi6220-hikey.dtb boot-fat/

  # prepare xen.cfg and startup.nsh

  For xen.cfg, it have below content:
  options=dom0_mem=512M dom0_max_vcpus=8 conswitch=x console=dtuart
  dtuart=/smb/uart@f7113000
  kernel=Image console=hvc root=/dev/mmcblk0p9 rootwait rw 3 mem=256M
  dtb=hi6220-hikey.dtb

  For startup.nsh, it have below content:
  FS0:
  Image console=ttyAMA3,115200 root=/dev/mmcblk0p9 rootwait rw efi=noruntime 
earlycon=pl011,0xf7113000

  $ umount boot-fat

- Flash the boot image into 'boot' partition and UEFI will find it by
  default:

  $ fastboot flash boot boot-linux.uefi.img

- After run into UEFI shell, after execute 'xen' command I can see
  below kernel panic; it's related with mmc driver initialization and
  it tries to use xen_swiotlb_alloc_coherent() for DMA memory
  allocation but failed.

  I tried to reduce the memory size in xen.cfg (options=dom0_mem=512M)
  for Xen and Linux kernel (mem=256M) but it doesn't work.

  Also enclosed the complete booting log.

  Very appreciate if you could give some suggestions for this.

Thanks,
Leo Yan

[1.807591] Modules linked in:
[1.810717] CPU: 4 PID: 1 Comm: swapper/0 Not tainted 
5.0.0-rc2-1-g5b47dea3757c #3
[1.818691] Hardware name: HiKey Development Board (DT)
[1.823983] pstate: 4005 (nZcv daif -PAN -UAO)
[1.828848] pc : xen_swiotlb_alloc_coherent+0x64/0x1e8
[1.834044] lr : dma_alloc_attrs+0xf4/0x110
[1.838289] sp : 10073a50
[1.841671] x29: 10073a50 x28: 0007 
[1.847047] x27: 11150068 x26: 80001b6ddd60 
[1.852429] x25: 10caaa70 x24:  
[1.857800] x23: 1000 x22: 1000 
[1.863177] x21:  x20: 80001c2edc10 
[1.868553] x19: 111fd000 x18:  
[1.873930] x17:  x16:  
[1.879306] x15: 111fd6c8 x14: 900737b7 
[1.884683] x13: 100737c5 x12: 11215000 
[1.890060] x11: 05f5e0ff x10: 111fd940 
[1.895436] x9 :  x8 : 80001bb0e700 
[1.900813] x7 :  x6 :  
[1.906189] x5 : 105bdbb8 x4 :  
[1.911566] x3 : 006000c0 x2 : 80001b6ddd60 
[1.916943] x1 : 1000 x0 :  
[1.922326] Process swapper/0 (pid: 1, stack limit = 0x(ptrval))
[1.929084] Call trace:
[1.931602]  xen_swiotlb_alloc_coherent+0x64/0x1e8
[1.936456]  dma_alloc_attrs+0xf4/0x110
[1.940359]  dmam_alloc_attrs+0x64/0xb8
[1.944264]  dw_mci_probe+0x5f8/0xb00
[1.947990]  dw_mci_pltfm_register+0xa0/0xd0
[1.952327]  dw_mci_k3_probe+0x2c/0x38
[1.956145]  platform_drv_probe+0x50/0xa0
[1.960218]  really_probe+0x1f0/0x298
[1.963947]  driver_probe_device+0x58/0x100
[1.968196]  __driver_attach+0xd4/0xd8
[1.972017]  bus_for_each_dev+0x74/0xc8
[1.975914]  driver_attach+0x20/0x28
[1.979556]  bus_add_driver+0x1ac/0x218
[1.983458]  driver_register+0x60/0x110
[1.987361]  __platform_driver_register+0x40/0x48
[1.992138]  dw_mci_k3_pltfm_driver_init+0x18/0x20
[1.996988]  do_one_initcall+0x5c/0x178
[2.000891]  kernel_init_freeable+0x198/0x240
[2.005314]  kernel_init+0x10/0x108
[2.008867]  ret_from_fork+0x10/0x18
[2.012516] Code: f9414280 aa1503e4 aa1a03e2 aa1703e1 (f945) 
[2.018680] ---[ end trace baf3ac16eabafa1f ]---
[2.023394] Kernel panic - not syncing: Attempted to kill init! 
exitcode=0x000b
[2.031068] SMP: stopping secondary CPUs
[2.035101] Kernel Offset: disabled
[2.038611] CPU features: 0x002,24002004
[2.042604] Memory Limit: 256 MB
[2.045898] ---[ end Kernel panic - not syncing: Attempted to kill init! 
exitcode=0x000b ]---
xen
Loading driver at 0x0003783D000 EntryPoint=0x000378E1398
Loading driver at 0x0003783D000 EntryPoint=0x000378E1398 
3hXen 4.12-unstable (c/s Mon Dec 17 09:22:59 2018 + git:a5b0eb3636) EFI 
loader

Using configuration file 'xen.cfg'
hi6220-hikey.dtb: 0x37b1-0x37b195c5
Image: 0x3653b000-0x3783ca00
 Xen 4.12-unstable
(XEN) Xen version 4.12-unstable (leoy@) (aarch64-linux-gnu-gcc (Linaro GCC 
6.2-2016.11) 6.2.1 20161016) debug=y  Mon Jan 14 11:16:40 CST 2019
(XEN) Latest ChangeSet: Mon Dec 17 09:22:59 2018 + 

Re: [Xen-devel] [PATCH 9/9] xen/privcmd-buf.c: Convert to use vm_insert_range_buggy

2019-01-14 Thread Souptick Joarder
On Tue, Jan 15, 2019 at 5:01 AM Boris Ostrovsky
 wrote:
>
> On 1/11/19 10:13 AM, Souptick Joarder wrote:
> > Convert to use vm_insert_range_buggy() to map range of kernel
> > memory to user vma.
> >
> > This driver has ignored vm_pgoff. We could later "fix" these drivers
> > to behave according to the normal vm_pgoff offsetting simply by
> > removing the _buggy suffix on the function name and if that causes
> > regressions, it gives us an easy way to revert.
> >
> > Signed-off-by: Souptick Joarder 
> > ---
> >  drivers/xen/privcmd-buf.c | 8 ++--
> >  1 file changed, 2 insertions(+), 6 deletions(-)
> >
> > diff --git a/drivers/xen/privcmd-buf.c b/drivers/xen/privcmd-buf.c
> > index de01a6d..a9d7e97 100644
> > --- a/drivers/xen/privcmd-buf.c
> > +++ b/drivers/xen/privcmd-buf.c
> > @@ -166,12 +166,8 @@ static int privcmd_buf_mmap(struct file *file, struct 
> > vm_area_struct *vma)
> >   if (vma_priv->n_pages != count)
> >   ret = -ENOMEM;
> >   else
> > - for (i = 0; i < vma_priv->n_pages; i++) {
> > - ret = vm_insert_page(vma, vma->vm_start + i * 
> > PAGE_SIZE,
> > -  vma_priv->pages[i]);
> > - if (ret)
> > - break;
> > - }
> > + ret = vm_insert_range_buggy(vma, vma_priv->pages,
> > + vma_priv->n_pages);
>
> This can use the non-buggy version. But since the original code was
> indeed buggy in this respect I can submit this as a separate patch later.
>
> So
>
> Reviewed-by: Boris Ostrovsky 

Thanks Boris.
>
>
> >
> >   if (ret)
> >   privcmd_buf_vmapriv_free(vma_priv);
>

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 8/9] xen/gntdev.c: Convert to use vm_insert_range

2019-01-14 Thread Souptick Joarder
On Tue, Jan 15, 2019 at 4:58 AM Boris Ostrovsky
 wrote:
>
> On 1/11/19 10:12 AM, Souptick Joarder wrote:
> > Convert to use vm_insert_range() to map range of kernel
> > memory to user vma.
> >
> > Signed-off-by: Souptick Joarder 
>
> Reviewed-by: Boris Ostrovsky 
>
> (although it would be good to mention in the commit that you are also
> replacing count with vma_pages(vma), and why)

The original code was using count ( *count = vma_pages(vma)* )
which is same as this patch. Do I need capture it change log ?

>
>
> > ---
> >  drivers/xen/gntdev.c | 16 ++--
> >  1 file changed, 6 insertions(+), 10 deletions(-)
> >
> > diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
> > index b0b02a5..ca4acee 100644
> > --- a/drivers/xen/gntdev.c
> > +++ b/drivers/xen/gntdev.c
> > @@ -1082,18 +1082,17 @@ static int gntdev_mmap(struct file *flip, struct 
> > vm_area_struct *vma)
> >  {
> >   struct gntdev_priv *priv = flip->private_data;
> >   int index = vma->vm_pgoff;
> > - int count = vma_pages(vma);
> >   struct gntdev_grant_map *map;
> > - int i, err = -EINVAL;
> > + int err = -EINVAL;
> >
> >   if ((vma->vm_flags & VM_WRITE) && !(vma->vm_flags & VM_SHARED))
> >   return -EINVAL;
> >
> >   pr_debug("map %d+%d at %lx (pgoff %lx)\n",
> > - index, count, vma->vm_start, vma->vm_pgoff);
> > + index, vma_pages(vma), vma->vm_start, vma->vm_pgoff);
> >
> >   mutex_lock(>lock);
> > - map = gntdev_find_map_index(priv, index, count);
> > + map = gntdev_find_map_index(priv, index, vma_pages(vma));
> >   if (!map)
> >   goto unlock_out;
> >   if (use_ptemod && map->vma)
> > @@ -1145,12 +1144,9 @@ static int gntdev_mmap(struct file *flip, struct 
> > vm_area_struct *vma)
> >   goto out_put_map;
> >
> >   if (!use_ptemod) {
> > - for (i = 0; i < count; i++) {
> > - err = vm_insert_page(vma, vma->vm_start + i*PAGE_SIZE,
> > - map->pages[i]);
> > - if (err)
> > - goto out_put_map;
> > - }
> > + err = vm_insert_range(vma, map->pages, map->count);
> > + if (err)
> > + goto out_put_map;
> >   } else {
> >  #ifdef CONFIG_X86
> >   /*
>

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [libvirt test] 131952: regressions - trouble: blocked/broken/fail/pass

2019-01-14 Thread osstest service owner
flight 131952 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/131952/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm   broken
 test-amd64-i386-libvirt  broken
 test-amd64-amd64-libvirt-xsm broken
 test-amd64-i386-libvirt-xsm  broken
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 4 host-install(4) broken 
REGR. vs. 131857
 test-amd64-amd64-libvirt-xsm  4 host-install(4)broken REGR. vs. 131857
 build-armhf-libvirt   6 libvirt-buildfail REGR. vs. 131857

Tests which are failing intermittently (not blocking):
 test-amd64-i386-libvirt   4 host-install(4)  broken pass in 131935
 test-amd64-i386-libvirt-xsm   4 host-install(4)  broken pass in 131935

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm 13 migrate-support-check fail in 131935 never pass
 test-amd64-i386-libvirt 13 migrate-support-check fail in 131935 never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass

version targeted for testing:
 libvirt  d40b820c5d3b0c9d5222844110881af66cdbb746
baseline version:
 libvirt  45b439c3af000eb41c819068d093406810dd036c

Last test of basis   131857  2019-01-09 04:18:51 Z6 days
Failing since131894  2019-01-10 04:18:50 Z5 days5 attempts
Testing same since   131925  2019-01-12 04:18:56 Z3 days3 attempts


People who touched revisions under test:
  Andrea Bolognani 
  Daniel P. Berrangé 
  Jiri Denemark 
  John Ferlan 
  Ján Tomko 
  Laine Stump 
  Michal Privoznik 
  Yuval Shaia 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  fail
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmbroken  
 test-amd64-amd64-libvirt-xsm broken  
 test-amd64-i386-libvirt-xsm  broken  
 test-amd64-amd64-libvirt pass
 test-armhf-armhf-libvirt blocked 
 test-amd64-i386-libvirt  broken  
 test-amd64-amd64-libvirt-pairpass
 test-amd64-i386-libvirt-pair pass
 test-armhf-armhf-libvirt-raw blocked 
 test-amd64-amd64-libvirt-vhd pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm broken
broken-job test-amd64-i386-libvirt broken
broken-job test-amd64-amd64-libvirt-xsm broken
broken-job test-amd64-i386-libvirt-xsm broken
broken-step test-amd64-i386-libvirt host-install(4)
broken-step test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm host-install(4)
broken-step test-amd64-i386-libvirt-xsm host-install(4)
broken-step test-amd64-amd64-libvirt-xsm host-install(4)
broken-job test-amd64-amd64-libvirt-xsm broken
broken-job test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm broken

Not pushing.

(No revision log; it would be 301 lines long.)

___
Xen-devel mailing list

[Xen-devel] [qemu-mainline test] 131951: trouble: broken/fail/pass

2019-01-14 Thread osstest service owner
flight 131951 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/131951/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm   broken
 test-amd64-i386-qemuu-rhel6hvm-intel broken
 test-amd64-amd64-xl-multivcpu broken
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm   broken
 test-amd64-i386-qemuu-rhel6hvm-intel 4 host-install(4) broken REGR. vs. 131842
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 4 host-install(4) broken 
REGR. vs. 131842
 test-amd64-amd64-xl-multivcpu  4 host-install(4)   broken REGR. vs. 131842
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 4 host-install(4) broken REGR. 
vs. 131842

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 131842
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 131842
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 131842
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 131842
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 131842
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass

version targeted for testing:
 qemuu27df21ca3886fff4dd3d70e515517667963a52f1
baseline version:
 qemuu147923b1a901a0370f83a0f4c58ec1baffef22f0

Last test of basis   131842  2019-01-09 00:37:22 Z6 days
Failing since131892  2019-01-09 23:37:00 Z5 days5 attempts
Testing same since   131923  2019-01-11 21:07:18 Z3 days3 attempts


People who touched revisions under test:
  Alex Williamson 
  Alexandro Sanchez Bach 
  Alexey Kardashevskiy 
  BALATON Zoltan 
  Cédric Le Goater 
  David Gibson 
  David Hildenbrand 
  Eduardo Habkost 
  Eric Blake 
  Fam Zheng 
  Frediano Ziglio 
  Gerd Hoffmann 
  Greg Kurz 
  Guenter Roeck 
  Kashyap Chamarthy 
  Laurent Vivier 
  Laurent Vivier 
  Li Feng 
  Li Qiang 
  Marc-André Lureau 
  Mark Cave-Ayland 
  Michael Roth 
  Paolo Bonzini 
  Peng Hao 
  Peter Maydell 
  Philippe Mathieu-Daudé 
  Priit Laes 
  Richard Henderson 
  Roman Bolshakov 
  Sreejith Mohanan 
  Stefan Hajnoczi 
  Thomas Huth 
  Tom Deseyn 
  Wainer dos Santos Moschetta 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf

[Xen-devel] [PATCH v2] pvcalls-front: fix potential null dereference

2019-01-14 Thread Wen Yang
 static checker warning:
drivers/xen/pvcalls-front.c:373 alloc_active_ring()
error: we previously assumed 'map->active.ring' could be null
   (see line 357)

drivers/xen/pvcalls-front.c
351 static int alloc_active_ring(struct sock_mapping *map)
352 {
353 void *bytes;
354
355 map->active.ring = (struct pvcalls_data_intf *)
356 get_zeroed_page(GFP_KERNEL);
357 if (!map->active.ring)
^
Check

358 goto out;
359
360 map->active.ring->ring_order = PVCALLS_RING_ORDER;
361 bytes = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
362 PVCALLS_RING_ORDER);
363 if (!bytes)
364 goto out;
365
366 map->active.data.in = bytes;
367 map->active.data.out = bytes +
368 XEN_FLEX_RING_SIZE(PVCALLS_RING_ORDER);
369
370 return 0;
371
372 out:
--> 373 free_active_ring(map);
 ^^^
Add null check on map->active.ring before dereferencing it to avoid
any NULL pointer dereferences.

Fixes: 9f51c05dc41a ("pvcalls-front: Avoid get_free_pages(GFP_KERNEL)
under spinlock")
Reported-by: Dan Carpenter 
Suggested-by: Boris Ostrovsky 
Signed-off-by: Wen Yang 
CC: Boris Ostrovsky 
CC: Juergen Gross 
CC: Stefano Stabellini 
CC: Dan Carpenter 
CC: xen-devel@lists.xenproject.org
CC: linux-ker...@vger.kernel.org
---
v2->v1:
- Add NULL check on map->active.ring and return
  immediately if it is.

 drivers/xen/pvcalls-front.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
index 307861f..8a249c9 100644
--- a/drivers/xen/pvcalls-front.c
+++ b/drivers/xen/pvcalls-front.c
@@ -343,6 +343,9 @@ int pvcalls_front_socket(struct socket *sock)
 
 static void free_active_ring(struct sock_mapping *map)
 {
+   if (!map->active.ring)
+   return;
+
free_pages((unsigned long)map->active.data.in,
map->active.ring->ring_order);
free_page((unsigned long)map->active.ring);
-- 
2.9.5


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [linux-4.14 test] 131949: regressions - trouble: broken/fail/pass

2019-01-14 Thread osstest service owner
flight 131949 linux-4.14 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/131949/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-xsm   broken
 test-amd64-amd64-xl-rtds broken
 test-amd64-i386-qemut-rhel6hvm-intel broken
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm   broken
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrictbroken
 test-amd64-amd64-xl-pvhv2-intel broken
 test-amd64-i386-xl-qemuu-ovmf-amd64 broken
 test-amd64-i386-qemuu-rhel6hvm-intel broken
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsmbroken
 test-amd64-i386-examine   5 host-install   broken REGR. vs. 131663
 test-amd64-i386-qemut-rhel6hvm-intel 4 host-install(4) broken REGR. vs. 131663
 test-amd64-i386-xl-xsm4 host-install(4)broken REGR. vs. 131663
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 4 host-install(4) broken REGR. 
vs. 131663
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 4 host-install(4) broken REGR. 
vs. 131663
 test-amd64-i386-qemuu-rhel6hvm-intel 4 host-install(4) broken REGR. vs. 131663
 test-amd64-amd64-xl-pvhv2-intel  4 host-install(4) broken REGR. vs. 131663
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 4 host-install(4) broken 
REGR. vs. 131663
 test-amd64-i386-xl-qemuu-ovmf-amd64  4 host-install(4) broken REGR. vs. 131663
 test-amd64-amd64-xl-credit2   7 xen-boot fail REGR. vs. 131663

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds  4 host-install(4)broken REGR. vs. 131663

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 

Re: [Xen-devel] [PATCH] pvcalls-front: Replace GFP_KERNEL with GFP_ATOMIC in create_active

2019-01-14 Thread Boris Ostrovsky
On 1/14/19 10:48 AM, wangbo wrote:
> Create_active may called inside spinlock,replace GFP_KERNEL with GFP_ATOMIC

https://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git/commit/?h=for-linus-4.21=9f51c05dc41a6d69423e3d03d18eb7ab22f9ec19
is queued and addresses this problem.

(Please run scripts/get_maintainer.pl on your patches, otherwise they
may be missed and delayed)

-boris


>
> Signed-off-by: wangbo 
> ---
>  drivers/xen/pvcalls-front.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
> index 77224d8..31bd3c9 100644
> --- a/drivers/xen/pvcalls-front.c
> +++ b/drivers/xen/pvcalls-front.c
> @@ -344,11 +344,11 @@ static int create_active(struct sock_mapping *map, int 
> *evtchn)
>   init_waitqueue_head(>active.inflight_conn_req);
>  
>   map->active.ring = (struct pvcalls_data_intf *)
> - __get_free_page(GFP_KERNEL | __GFP_ZERO);
> + __get_free_page(GFP_ATOMIC | __GFP_ZERO);
>   if (map->active.ring == NULL)
>   goto out_error;
>   map->active.ring->ring_order = PVCALLS_RING_ORDER;
> - bytes = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
> + bytes = (void *)__get_free_pages(GFP_ATOMIC | __GFP_ZERO,
>   PVCALLS_RING_ORDER);
>   if (bytes == NULL)
>   goto out_error;


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 9/9] xen/privcmd-buf.c: Convert to use vm_insert_range_buggy

2019-01-14 Thread Boris Ostrovsky
On 1/11/19 10:13 AM, Souptick Joarder wrote:
> Convert to use vm_insert_range_buggy() to map range of kernel
> memory to user vma.
>
> This driver has ignored vm_pgoff. We could later "fix" these drivers
> to behave according to the normal vm_pgoff offsetting simply by
> removing the _buggy suffix on the function name and if that causes
> regressions, it gives us an easy way to revert.
>
> Signed-off-by: Souptick Joarder 
> ---
>  drivers/xen/privcmd-buf.c | 8 ++--
>  1 file changed, 2 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/xen/privcmd-buf.c b/drivers/xen/privcmd-buf.c
> index de01a6d..a9d7e97 100644
> --- a/drivers/xen/privcmd-buf.c
> +++ b/drivers/xen/privcmd-buf.c
> @@ -166,12 +166,8 @@ static int privcmd_buf_mmap(struct file *file, struct 
> vm_area_struct *vma)
>   if (vma_priv->n_pages != count)
>   ret = -ENOMEM;
>   else
> - for (i = 0; i < vma_priv->n_pages; i++) {
> - ret = vm_insert_page(vma, vma->vm_start + i * PAGE_SIZE,
> -  vma_priv->pages[i]);
> - if (ret)
> - break;
> - }
> + ret = vm_insert_range_buggy(vma, vma_priv->pages,
> + vma_priv->n_pages);

This can use the non-buggy version. But since the original code was
indeed buggy in this respect I can submit this as a separate patch later.

So

Reviewed-by: Boris Ostrovsky 


>  
>   if (ret)
>   privcmd_buf_vmapriv_free(vma_priv);


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 8/9] xen/gntdev.c: Convert to use vm_insert_range

2019-01-14 Thread Boris Ostrovsky
On 1/11/19 10:12 AM, Souptick Joarder wrote:
> Convert to use vm_insert_range() to map range of kernel
> memory to user vma.
>
> Signed-off-by: Souptick Joarder 

Reviewed-by: Boris Ostrovsky 

(although it would be good to mention in the commit that you are also
replacing count with vma_pages(vma), and why)


> ---
>  drivers/xen/gntdev.c | 16 ++--
>  1 file changed, 6 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
> index b0b02a5..ca4acee 100644
> --- a/drivers/xen/gntdev.c
> +++ b/drivers/xen/gntdev.c
> @@ -1082,18 +1082,17 @@ static int gntdev_mmap(struct file *flip, struct 
> vm_area_struct *vma)
>  {
>   struct gntdev_priv *priv = flip->private_data;
>   int index = vma->vm_pgoff;
> - int count = vma_pages(vma);
>   struct gntdev_grant_map *map;
> - int i, err = -EINVAL;
> + int err = -EINVAL;
>  
>   if ((vma->vm_flags & VM_WRITE) && !(vma->vm_flags & VM_SHARED))
>   return -EINVAL;
>  
>   pr_debug("map %d+%d at %lx (pgoff %lx)\n",
> - index, count, vma->vm_start, vma->vm_pgoff);
> + index, vma_pages(vma), vma->vm_start, vma->vm_pgoff);
>  
>   mutex_lock(>lock);
> - map = gntdev_find_map_index(priv, index, count);
> + map = gntdev_find_map_index(priv, index, vma_pages(vma));
>   if (!map)
>   goto unlock_out;
>   if (use_ptemod && map->vma)
> @@ -1145,12 +1144,9 @@ static int gntdev_mmap(struct file *flip, struct 
> vm_area_struct *vma)
>   goto out_put_map;
>  
>   if (!use_ptemod) {
> - for (i = 0; i < count; i++) {
> - err = vm_insert_page(vma, vma->vm_start + i*PAGE_SIZE,
> - map->pages[i]);
> - if (err)
> - goto out_put_map;
> - }
> + err = vm_insert_range(vma, map->pages, map->count);
> + if (err)
> + goto out_put_map;
>   } else {
>  #ifdef CONFIG_X86
>   /*


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [freebsd-master test] 131955: trouble: blocked/broken

2019-01-14 Thread osstest service owner
flight 131955 freebsd-master real [real]
http://logs.test-lab.xenproject.org/osstest/logs/131955/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-freebsd  broken
 build-amd64-freebsd   5 host-install(5)broken REGR. vs. 131783

Tests which did not succeed, but are not blocking:
 build-amd64-xen-freebsd   1 build-check(1)   blocked  n/a
 build-amd64-freebsd-again 1 build-check(1)   blocked  n/a

version targeted for testing:
 freebsd  05dc4d60c8e036139ddd0d27118b45d9e681c7ce
baseline version:
 freebsd  2a52bc55467e95f92e1024cd558df3930df99594

Last test of basis   131783  2019-01-07 09:19:04 Z7 days
Failing since131876  2019-01-09 09:19:19 Z5 days3 attempts
Testing same since   131955  2019-01-14 09:19:32 Z0 days1 attempts


People who touched revisions under test:
  0mp <0...@freebsd.org>
  ae 
  allanjude 
  andrew 
  arichardson 
  avos 
  bapt 
  brooks 
  cem 
  chuck 
  cognet 
  cperciva 
  cy 
  delphij 
  des 
  emaste 
  fsu 
  glebius 
  gonzo 
  hselasky 
  ian 
  imp 
  jah 
  jhibbits 
  jkim 
  kevans 
  kib 
  kp 
  lme 
  manu 
  markj 
  ngie 
  np 
  nyan 
  pfg 
  pjd 
  ram 
  rgrimes 
  sef 
  shurd 
  trasz 
  tuexen 
  vmaffione 

jobs:
 build-amd64-freebsd-againblocked 
 build-amd64-freebsd  broken  
 build-amd64-xen-freebsd  blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-amd64-freebsd broken
broken-step build-amd64-freebsd host-install(5)

Not pushing.

(No revision log; it would be 1412 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v6 1/4] xen: introduce SYMBOL

2019-01-14 Thread Stefano Stabellini
Hi Jan,

One question below to make a decision on the way forward.

On Mon, 14 Jan 2019, Jan Beulich wrote:
> >>> On 14.01.19 at 04:45,  wrote:
> > So let's keep the linker-accessible variable as a type that works for the
> > linker (which really could be anything as long as you use the address, not
> > the value), but name it something else - a name that screams "DON'T USE ME
> > UNLESS YOU KNOW WHAT YOU'RE DOING". And then before the first use, copy
> > that value to "uintptr_t _start;".
> > 
> > The following is a quick proof of concept for aarch64. I changed the type
> > of _start and _end, and added code to copy the linker-assigned value to
> > _start and _end. Upon booting, I see the correct values:
> 
> Global symbols starting with underscores should already be shouting
> enough. But what's worse - the whole idea if using array types is to
> avoid the intermediate variables.
> 
> > --- a/xen/arch/arm/setup.c
> > +++ b/xen/arch/arm/setup.c
> > @@ -726,6 +726,12 @@ static void __init setup_mm(unsigned long dtb_paddr, 
> > size_t dtb_size)
> >  
> >  size_t __read_mostly dcache_line_bytes;
> >  
> > +typedef char TYPE_DOESNT_MATTER;
> > +extern TYPE_DOESNT_MATTER _start_linker_assigned_dont_use_me,
> > +  _end_linker_assigned_dont_use_me;
> 
> This and ...
> 
> > @@ -770,10 +776,17 @@ void __init start_xen(unsigned long boot_phys_offset,
> >  printk("Command line: %s\n", cmdline);
> >  cmdline_parse(cmdline);
> >  
> > +_start = (uintptr_t)&_start_linker_assigned_dont_use_me;
> > +_end = (uintptr_t)&_end_linker_assigned_dont_use_me;
> 
> ... this violates what the symbol names say. And if you want to
> avoid issues, you'd want to keep out of C files uses of those
> symbols altogether anyway, and you easily can: In any
> assembly file, have
> 
> _start:   .long _start_linker_assigned_dont_use_me
> _end: .long _end_linker_assigned_dont_use_me
> 
> In particular, they don't need to be runtime initialized, saving
> you from needing to set them before first use. But as said -
> things are the way they are precisely to avoid such variables.
> 
> >> But, instead of converting _start to unsigned long via SYMBOL_HIDE, we
> >> could convert it to uintptr_t instead, it would be a trivial change on
> >> top of the existing unsigned long series. Not sure if it is beneficial.
> > 
> > The difference would be whether we want to rely on implementation-defined
> > behavior or not.
> 
> Why not? Simply specify that compilers with implementation defined
> behavior not matching our expectations are unsuitable. And btw, I
> suppose this is just the tiny tip of the iceberg of our reliance on
> implementation defined behavior.

The reason is that relying on undefined behavior is not reliable, it is
not C compliant, it is not allowed by MISRA-C, and not guaranteed to
work with any compiler. Yes, this instance is only the tip of the
iceberg, we have a long road ahead, but we shouldn't really give up
because it is going to be difficult :-) Stewart's approach would
actually be compliant and help toward reducing reliance on undefined
behavior.

Would you be OK if I rework the series to follow his approach using
intermediate variables? See the attached patch as a reference, it only
"converts" _start and _end as an example. Fortunately, it will be
textually similar to the previous SYMBOL returning unsigned long version
of the series.

If you are OK with it, do you have any suggestions on how would you like
the intermediate variables to be called? I went with _start/start_ and
_end/end_ but I am open to suggestions. Also to which assembly file you
would like the new variables being added -- I created a new one for the
purpose named var.S in the attached example.diff --git a/xen/arch/arm/alternative.c b/xen/arch/arm/alternative.c
index 52ed7ed..b79536d 100644
--- a/xen/arch/arm/alternative.c
+++ b/xen/arch/arm/alternative.c
@@ -187,8 +187,8 @@ static int __apply_alternatives_multi_stop(void *unused)
 {
 int ret;
 struct alt_region region;
-mfn_t xen_mfn = virt_to_mfn(_start);
-paddr_t xen_size = _end - _start;
+mfn_t xen_mfn = virt_to_mfn(start_);
+paddr_t xen_size = end_ - start_;
 unsigned int xen_order = get_order_from_bytes(xen_size);
 void *xenmap;
 
@@ -206,7 +206,7 @@ static int __apply_alternatives_multi_stop(void *unused)
 region.begin = __alt_instructions;
 region.end = __alt_instructions_end;
 
-ret = __apply_alternatives(, xenmap - (void *)_start);
+ret = __apply_alternatives(, (uintptr_t)xenmap - start_);
 /* The patching is not expected to fail during boot. */
 BUG_ON(ret != 0);
 
diff --git a/xen/arch/arm/arm32/Makefile b/xen/arch/arm/arm32/Makefile
index 0ac254f..983fb82 100644
--- a/xen/arch/arm/arm32/Makefile
+++ b/xen/arch/arm/arm32/Makefile
@@ -10,4 +10,5 @@ obj-y += proc-v7.o proc-caxx.o
 obj-y += smpboot.o
 obj-y += traps.o
 obj-y += vfp.o
+obj-y += 

[Xen-devel] [RFC v1] kexec: Prototype for signature verification within Xen

2019-01-14 Thread Eric DeVolder
These changes work in conjunction with the signature
verification support for Xen I published recently.

Prior to this change, kexec supported the following
three modes of operation:

kexec_load:
- unverified loading of kernel into Linux (original mode)
- unverified loading of kernel into Xen
kexec_file_load (the -s option to kexec):
- verified loading of kernel into Linux

With the verified loading of a kernel into Linux, the scope
of kexec changed drastically as the kernel performs most of
the work that kexec previously did; the kernel does so so as
to reduce the risk of compromise.

For example, the unverified loading of a kernel into Linux
involves locating memory within the system to load the
various pieces of data (kernel, initramdisk, command line)
as well as reserving additional memory such as the first 1MB
on x86 for legacy reasons as well as something known as
'purgatory', a trampoline that checks the integrity of the
contents of loaded pieces of data, before invoking that
loaded kernel. The management of purgatory involves
manipulating an embedded ELF purgatory object file to insert
a memory hash value, and rewrite a few run-time switches
based on kexec command line parameters.

By contrast, the verified loading essentially just passes
file handles for the kernel, initramdisk, and command line
pointer, and the kernel takes care of the rest, by
performing all the work that the unverified kexec load would
do, but inside the kernel using trusted kernel code.

This changeset adds a fourth mode to kexec:

- verified loading of kernel into Xen

In general, Xen performs the signature verification on the
loaded kernel, much as Linux does, but that is where the
similarities end.  In the current Xen implementation, no
infrastructure is present to support reading from [Linux
dom0] file handles, or for manipulating ELF objects. As
such, without Xen support for these actions, Xen relies upon
kexec to provide these services, which is what this mode
does.

To achieve this, this mode of operation essentially vectors
the verified load for Xen through the non-verified path,
which performs all the needed actions for kexec to work, but
then makes an adjustment to pass the entire kernel file, not
just the loadable portion of the kernel file, to Xen in
order to provide the proper image for signature
verification.

The loading of kexec images for signature verification for
Xen is indicated with the -s switch, just like for Linux.

Changes to configure.ac are for detecting whether or not the
Xen version supports this kexec_file_load hypercall op.

Changes to kexec-bzImage64.c are for recording what the
change to the kernel image entry needs to be (the entire
kernel file, not just the loadable portion), as well as
vectoring kexec_file_load through kexec_load for Xen.

Changes to kexec-xen.c are to invoke the new Xen
kexec_file_load hypercall op, from kexec_load.

Changes to kexec.c are to vector kexec_file_load for Xen
throgh kexec_load for Xen, as well as make the correction
for passing the complete kernel file to Xen.

Signed-off-by: Eric DeVolder 
---
 configure.ac|  8 
 kexec/arch/x86_64/kexec-bzImage64.c | 18 ++
 kexec/kexec-xen.c   |  7 +++
 kexec/kexec.c   | 38 +
 4 files changed, 71 insertions(+)

diff --git a/configure.ac b/configure.ac
index e05d601..a11787b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -190,6 +190,14 @@ if test "$ac_cv_lib_xenctrl_xc_kexec_load" = yes ; then
AC_MSG_NOTICE([The kexec_status call is not available]))
 fi
 
+dnl Check for the Xen kexec_status hypercall - reachable from --with-xen=yes|dl
+if test "$ac_cv_lib_xenctrl_xc_kexec_load" = yes ; then
+   AC_CHECK_LIB(xenctrl, xc_kexec_file_load,
+   AC_DEFINE(HAVE_XEN_KEXEC_FILE_LOAD, 1,
+   [The Xen kexec_file_load call is available]),
+   AC_MSG_NOTICE([The Xen kexec_file_load call is not available]))
+fi
+
 dnl ---Sanity checks
 if test "$CC"  = "no"; then AC_MSG_ERROR([cc not found]); fi
 if test "$CPP" = "no"; then AC_MSG_ERROR([cpp not found]); fi
diff --git a/kexec/arch/x86_64/kexec-bzImage64.c 
b/kexec/arch/x86_64/kexec-bzImage64.c
index 8edb3e4..98e9d50 100644
--- a/kexec/arch/x86_64/kexec-bzImage64.c
+++ b/kexec/arch/x86_64/kexec-bzImage64.c
@@ -207,6 +207,20 @@ static int do_bzImage64_load(struct kexec_info *info,
align = real_mode->kernel_alignment;
addr = add_buffer(info, kernel + kern16_size, k_size,
  size, align, 0x10, -1, -1);
+#ifdef HAVE_XEN_KEXEC_FILE_LOAD
+if (xen_present() && info->file_mode)
+{
+/* Record info for post-purgatory hash computation replacement with 
kernel file */
+extern char *original_kernel;
+extern off_t original_kernel_len;
+extern char *replacement_kernel;
+extern off_t replacement_kernel_len;
+

[Xen-devel] [RFC v1 2/8] kexec: implement kexec_file_load() for PECOFF+Authenticode files

2019-01-14 Thread Eric DeVolder
This change adds to Xen the kexec_file_load() entry point.  The
kexec_file_load() is nearly identical to kexec_load(), but with the
added code to handle checking and handling of PECOFF Authenticode
signature verification.

Signed-off-by: Eric DeVolder 
---
 xen/common/kexec.c | 131 -
 1 file changed, 129 insertions(+), 2 deletions(-)

diff --git a/xen/common/kexec.c b/xen/common/kexec.c
index 44ae95d..b013514 100644
--- a/xen/common/kexec.c
+++ b/xen/common/kexec.c
@@ -33,6 +33,10 @@
 #include 
 #endif
 
+#include "ped.h"
+#include "TrustedCert.h"
+int verify_openssl (pecoff_image_t *pe, const uint8_t *TrustedCert, int 
TrustedCertSize);
+
 bool_t kexecing = FALSE;
 
 /* Memory regions to store the per cpu register state etc. on a crash. */
@@ -1112,6 +1116,126 @@ error:
 return ret;
 }
 
+static int kexec_file_load(XEN_GUEST_HANDLE_PARAM(void) uarg)
+{
+xen_kexec_load_t load;
+xen_kexec_segment_t *segments;
+struct kexec_image *kimage = NULL;
+int ret;
+int k, numSigned = 0, numPassed = 0;
+
+if ( copy_from_guest(, uarg, 1) )
+return -EFAULT;
+
+if ( load.nr_segments >= KEXEC_SEGMENT_MAX )
+return -EINVAL;
+
+segments = xmalloc_array(xen_kexec_segment_t, load.nr_segments);
+if ( segments == NULL )
+return -ENOMEM;
+
+if ( copy_from_guest(segments, load.segments.h, load.nr_segments) )
+{
+ret = -EFAULT;
+goto error;
+}
+
+/* Handle signature verification of signed segments */
+for (k = 0; k < load.nr_segments; ++k)
+{
+xen_kexec_segment_t *segment = [k];
+uint8_t *imageBase = NULL;
+size_t imageSize;
+pecoff_image_t *pe = NULL;
+int j; (void)j;
+
+if (NULL == segment->buf.h.p) continue;
+
+imageSize = segment->buf_size;
+imageBase = xmalloc_array(unsigned char, imageSize);
+if (NULL == imageBase)
+{
+printk("Ooops %u\n", (unsigned)imageSize);
+ret = -ENOMEM;
+goto error;
+}
+if ( copy_from_guest(imageBase, segment->buf.h, segment->buf_size) )
+{
+xfree(imageBase);
+ret = -EFAULT;
+goto error;
+}
+
+/* Handle PECOFF w/ Authenticode, only ... */
+pe = pecoff_image_decode(imageBase, imageSize);
+if (pe && pe->iddc.dd && pe->iddc.dd->CertificateTable.VirtualAddress)
+{
+++numSigned;
+pecoff_setup_verify(pe);
+ret = verify_openssl(pe, TrustedCert, sizeof(TrustedCert));
+
+/* if all is well ... */
+if (1 == ret)
+{
+coff_header_t *ch = pe->chc.ch;
+unsigned x;
+
+++numPassed; /* success! */
+
+/* point to text executable */
+for (x = 0; x < ch->NumberOfSections; ++x)
+{
+pecoff_section_t *s = pe->sectioncs[x].s;
+if (
+(s->Name[0] == '.') &&
+(s->Name[1] == 't') &&
+(s->Name[2] == 'e') &&
+(s->Name[3] == 'x') &&
+(s->Name[4] == 't')
+)
+{
+/* Adjust segment info for proper load */
+uint8_t *p = (uint8_t *)segment->buf.h.p;
+/* adjust to point to start of .text */
+p += s->PointerToRawData;
+segment->buf.h.p = p;
+/* adjust size accordingly */
+segment->buf_size -= s->PointerToRawData;
+}
+}
+}
+}
+if (pe) pecoff_image_free(pe);
+if (imageBase) xfree(imageBase);
+}
+printk("KEXEC_file_load signed %d passed %d\n", numSigned, numPassed);
+
+if (! ((numPassed == numSigned) && (numSigned > 0)) )
+ret = -ENOEXEC;
+else
+ret = kimage_alloc(, load.type, load.arch, load.entry_maddr,
+   load.nr_segments, segments);
+
+if ( ret < 0 )
+goto error;
+
+ret = kimage_load_segments(kimage);
+if ( ret < 0 )
+goto error;
+
+ret = kexec_load_slot(kimage);
+if ( ret < 0 )
+goto error;
+
+return 0;
+
+error:
+if ( ! kimage )
+xfree(segments);
+kimage_free(kimage);
+return ret;
+}
+
 static int kexec_do_unload(xen_kexec_unload_t *unload)
 {
 struct kexec_image *old_kimage;
@@ -1237,8 +1361,11 @@ static int do_kexec_op_internal(unsigned long op,
 ret = kexec_unload(uarg);
 break;
 case KEXEC_CMD_kexec_status:
-   ret = kexec_status(uarg);
-   break;
+   ret = kexec_status(uarg);
+   break;
+case KEXEC_CMD_kexec_file_load:
+ret = kexec_file_load(uarg);
+break;
 }
 
 

[Xen-devel] [RFC v1 1/8] kexec: add kexec_file_load to libxenctrl

2019-01-14 Thread Eric DeVolder
This change adds a new entry point in libxenctrl for the
kexec_file_load() operation. The code for kexec_file_load() is nearly
identical to kexec_load() other than the use of hypercall op
KEXEC_CMD_kexec_file_load rather than KEXEC_CMD_kexec_load.

Signed-off-by: Eric DeVolder 
---
 tools/libxc/xc_kexec.c | 41 +
 tools/libxc/xenctrl.h  |  4 
 xen/include/public/kexec.h |  4 +++-
 3 files changed, 48 insertions(+), 1 deletion(-)

diff --git a/tools/libxc/xc_kexec.c b/tools/libxc/xc_kexec.c
index 5003556..10762af 100644
--- a/tools/libxc/xc_kexec.c
+++ b/tools/libxc/xc_kexec.c
@@ -112,6 +112,47 @@ out:
 return ret;
 }
 
+int xc_kexec_file_load(xc_interface *xch, uint8_t type, uint16_t arch,
+  uint64_t entry_maddr,
+  uint32_t nr_segments, xen_kexec_segment_t *segments)
+{
+int ret = -1;
+DECLARE_HYPERCALL;
+DECLARE_HYPERCALL_BOUNCE(segments, sizeof(*segments) * nr_segments,
+ XC_HYPERCALL_BUFFER_BOUNCE_IN);
+DECLARE_HYPERCALL_BUFFER(xen_kexec_load_t, load);
+
+if ( xc_hypercall_bounce_pre(xch, segments) )
+{
+PERROR("Could not allocate bounce buffer for kexec load hypercall");
+goto out;
+}
+load = xc_hypercall_buffer_alloc(xch, load, sizeof(*load));
+if ( load == NULL )
+{
+PERROR("Could not allocate buffer for kexec load hypercall");
+goto out;
+}
+
+load->type = type;
+load->arch = arch;
+load->entry_maddr = entry_maddr;
+load->nr_segments = nr_segments;
+set_xen_guest_handle(load->segments.h, segments);
+
+hypercall.op = __HYPERVISOR_kexec_op;
+hypercall.arg[0] = KEXEC_CMD_kexec_file_load; // only difference with 
xc_kexec_load()
+hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(load);
+
+ret = do_xen_hypercall(xch, );
+
+out:
+xc_hypercall_buffer_free(xch, load);
+xc_hypercall_bounce_post(xch, segments);
+
+return ret;
+}
+
 int xc_kexec_unload(xc_interface *xch, int type)
 {
 DECLARE_HYPERCALL;
diff --git a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h
index f61a61c..6243e5a 100644
--- a/tools/libxc/xenctrl.h
+++ b/tools/libxc/xenctrl.h
@@ -2488,6 +2488,10 @@ int xc_kexec_load(xc_interface *xch, uint8_t type, 
uint16_t arch,
   uint64_t entry_maddr,
   uint32_t nr_segments, xen_kexec_segment_t *segments);
 
+int xc_kexec_file_load(xc_interface *xch, uint8_t type, uint16_t arch,
+  uint64_t entry_maddr,
+  uint32_t nr_segments, xen_kexec_segment_t *segments);
+
 /*
  * Unload a kexec image.
  *
diff --git a/xen/include/public/kexec.h b/xen/include/public/kexec.h
index 022e160..eae98d4 100644
--- a/xen/include/public/kexec.h
+++ b/xen/include/public/kexec.h
@@ -203,6 +203,7 @@ DEFINE_XEN_GUEST_HANDLE(xen_kexec_segment_t);
  */
 
 #define KEXEC_CMD_kexec_load 4
+#define KEXEC_CMD_kexec_file_load 7
 typedef struct xen_kexec_load {
 uint8_t  type;/* One of KEXEC_TYPE_* */
 uint8_t  _pad;
@@ -213,8 +214,9 @@ typedef struct xen_kexec_load {
 uint64_t _pad;
 } segments;
 uint64_t entry_maddr; /* image entry point machine address. */
-} xen_kexec_load_t;
+} xen_kexec_load_t, xen_kexec_file_load_t;
 DEFINE_XEN_GUEST_HANDLE(xen_kexec_load_t);
+DEFINE_XEN_GUEST_HANDLE(xen_kexec_file_load_t);
 
 /*
  * Unload a kexec image.
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC v1 3/8] kexec: new file openssl-1.1.0i.patch

2019-01-14 Thread Eric DeVolder
This patch creates a new file that is in turn a patch that contains a
small number of changes needed in order to soldier through the
compilation of all the OpenSSL sources within Xen.

This patch is applied to OpenSSL in xen.spec.

Signed-off-by: Eric DeVolder 
---
 openssl-1.1.0i.patch | 378 +++
 1 file changed, 378 insertions(+)
 create mode 100644 openssl-1.1.0i.patch

diff --git a/openssl-1.1.0i.patch b/openssl-1.1.0i.patch
new file mode 100644
index 000..12ea07f
--- /dev/null
+++ b/openssl-1.1.0i.patch
@@ -0,0 +1,378 @@
+diff -x configdata.pm -x Makefile -rwbup 
/root/openssl-1.1.0i/crypto/aes/aes_locl.h 
xen/common/openssl-1.1.0i/crypto/aes/aes_locl.h
+--- /root/openssl-1.1.0i/crypto/aes/aes_locl.h 2018-08-14 08:45:06.0 
-0400
 xen/common/openssl-1.1.0i/crypto/aes/aes_locl.h2018-11-12 
17:59:01.885376275 -0500
+@@ -13,7 +13,7 @@
+ # include 
+ # include 
+ # include 
+-# include 
++# include 
+ 
+ # if defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_AMD64) || 
defined(_M_X64))
+ #  define SWAP(x) (_lrotl(x, 8) & 0x00ff00ff | _lrotr(x, 8) & 0xff00ff00)
+@@ -25,12 +25,12 @@
+ # endif
+ 
+ # ifdef AES_LONG
+-typedef unsigned long u32;
++//typedef unsigned long u32;
+ # else
+-typedef unsigned int u32;
++//typedef unsigned int u32;
+ # endif
+-typedef unsigned short u16;
+-typedef unsigned char u8;
++//typedef unsigned short u16;
++//typedef unsigned char u8;
+ 
+ # define MAXKC   (256/32)
+ # define MAXKB   (256/8)
+diff -x configdata.pm -x Makefile -rwbup 
/root/openssl-1.1.0i/crypto/asn1/evp_asn1.c 
xen/common/openssl-1.1.0i/crypto/asn1/evp_asn1.c
+--- /root/openssl-1.1.0i/crypto/asn1/evp_asn1.c2018-08-14 
08:45:06.0 -0400
 xen/common/openssl-1.1.0i/crypto/asn1/evp_asn1.c   2018-11-12 
17:59:01.886376275 -0500
+@@ -56,7 +56,7 @@ ASN1_SEQUENCE(asn1_int_oct) = {
+ ASN1_SIMPLE(asn1_int_oct, oct, ASN1_OCTET_STRING)
+ } static_ASN1_SEQUENCE_END(asn1_int_oct)
+ 
+-DECLARE_ASN1_ITEM(asn1_int_oct)
++//DECLARE_ASN1_ITEM(asn1_int_oct)
+ 
+ int ASN1_TYPE_set_int_octetstring(ASN1_TYPE *a, long num, unsigned char *data,
+   int len)
+diff -x configdata.pm -x Makefile -rwbup 
/root/openssl-1.1.0i/crypto/asn1/n_pkey.c 
xen/common/openssl-1.1.0i/crypto/asn1/n_pkey.c
+--- /root/openssl-1.1.0i/crypto/asn1/n_pkey.c  2018-08-14 08:45:06.0 
-0400
 xen/common/openssl-1.1.0i/crypto/asn1/n_pkey.c 2018-11-12 
17:59:01.886376275 -0500
+@@ -43,8 +43,8 @@ ASN1_BROKEN_SEQUENCE(NETSCAPE_ENCRYPTED_
+ ASN1_SIMPLE(NETSCAPE_ENCRYPTED_PKEY, enckey, X509_SIG)
+ } static_ASN1_BROKEN_SEQUENCE_END(NETSCAPE_ENCRYPTED_PKEY)
+ 
+-DECLARE_ASN1_FUNCTIONS_const(NETSCAPE_ENCRYPTED_PKEY)
+-DECLARE_ASN1_ENCODE_FUNCTIONS_const(NETSCAPE_ENCRYPTED_PKEY,NETSCAPE_ENCRYPTED_PKEY)
++//DECLARE_ASN1_FUNCTIONS_const(NETSCAPE_ENCRYPTED_PKEY)
++//DECLARE_ASN1_ENCODE_FUNCTIONS_const(NETSCAPE_ENCRYPTED_PKEY,NETSCAPE_ENCRYPTED_PKEY)
+ IMPLEMENT_ASN1_FUNCTIONS_const(NETSCAPE_ENCRYPTED_PKEY)
+ 
+ ASN1_SEQUENCE(NETSCAPE_PKEY) = {
+@@ -53,8 +53,8 @@ ASN1_SEQUENCE(NETSCAPE_PKEY) = {
+ ASN1_SIMPLE(NETSCAPE_PKEY, private_key, ASN1_OCTET_STRING)
+ } static_ASN1_SEQUENCE_END(NETSCAPE_PKEY)
+ 
+-DECLARE_ASN1_FUNCTIONS_const(NETSCAPE_PKEY)
+-DECLARE_ASN1_ENCODE_FUNCTIONS_const(NETSCAPE_PKEY,NETSCAPE_PKEY)
++//DECLARE_ASN1_FUNCTIONS_const(NETSCAPE_PKEY)
++//DECLARE_ASN1_ENCODE_FUNCTIONS_const(NETSCAPE_PKEY,NETSCAPE_PKEY)
+ IMPLEMENT_ASN1_FUNCTIONS_const(NETSCAPE_PKEY)
+ 
+ # endif /* OPENSSL_NO_RC4 */
+diff -x configdata.pm -x Makefile -rwbup 
/root/openssl-1.1.0i/crypto/bio/b_print.c 
xen/common/openssl-1.1.0i/crypto/bio/b_print.c
+--- /root/openssl-1.1.0i/crypto/bio/b_print.c  2018-08-14 08:45:06.0 
-0400
 xen/common/openssl-1.1.0i/crypto/bio/b_print.c 2018-11-12 
17:59:01.887376275 -0500
+@@ -24,6 +24,7 @@
+ #ifdef HAVE_LONG_DOUBLE
+ # define LDOUBLE long double
+ #else
++#define double long
+ # define LDOUBLE double
+ #endif
+ 
+@@ -31,13 +32,18 @@ static int fmtstr(char **, char **, size
+   const char *, int, int, int);
+ static int fmtint(char **, char **, size_t *, size_t *,
+   int64_t, int, int, int, int);
++#if 0
+ static int fmtfp(char **, char **, size_t *, size_t *,
+  LDOUBLE, int, int, int, int);
++#else
++#define fmtfp(A,B,C,D,E,F,G,H,I) fmtint(A,B,C,D,E,F,G,H,I)
++#endif
+ static int doapr_outch(char **, char **, size_t *, size_t *, int);
+ static int _dopr(char **sbuffer, char **buffer,
+  size_t *maxlen, size_t *retlen, int *truncated,
+  const char *format, va_list args);
+ 
++
+ /* format read states */
+ #define DP_S_DEFAULT0
+ #define DP_S_FLAGS  1
+@@ -512,6 +518,7 @@ fmtint(char **sbuffer,
+ return 1;
+ }
+ 
++#if 0
+ static LDOUBLE abs_val(LDOUBLE value)
+ {
+ LDOUBLE result = value;
+@@ -798,6 +805,7 @@ fmtfp(char **sbuffer,
+ }
+ return 1;
+ 

[Xen-devel] [RFC v1 6/8] kexec: support files for PECOFF Authenticode signature verification

2019-01-14 Thread Eric DeVolder
This patch contains the files from the standalone utility that
are to be integrated into Xen to provide signature verification
of a PECOFF file.

- TrustedCert.h is the public root certificate for Oracle signed
  binaries. This created by taking the DER encoded file at
  http://cacerts.digicert.com/DigiCertHighAssuranceEVRootCA.crt and
  converting it into this compilable C header file.
- dlcl.h is my double linked circular list set of helper macros, and
  should be replaced with corresponding Xen list handling.
- pecoff.h is a transcription of the data structures and constants
  described in the Microsoft spec
  
https://msdn.microsoft.com/en-us/library/windows/desktop/ms680547(v=vs.85).aspx.
- ped.[ch] is the PECOFF decoder/parser I wrote, and probably could
  stand to be replaced with a more mature library.
- v_openssl.c is the "glue" code that utilizes OpenSSL routines to
  perform signature verification. This file essentially contains the
  two files 
https://github.com/vathpela/verify/Cryptlib/Pk/[CryptPkcs7.c|CryptAuthenticode.c]
  which together provide the signature verification capability.

Signed-off-by: Eric DeVolder 
---
 xen/common/TrustedCert.h |  113 
 xen/common/dlcl.h|  323 +++
 xen/common/pecoff.h  |  283 ++
 xen/common/ped.c |  579 
 xen/common/ped.h |  128 +
 xen/common/v_openssl.c   | 1348 ++
 6 files changed, 2774 insertions(+)
 create mode 100644 xen/common/TrustedCert.h
 create mode 100755 xen/common/dlcl.h
 create mode 100644 xen/common/pecoff.h
 create mode 100644 xen/common/ped.c
 create mode 100644 xen/common/ped.h
 create mode 100644 xen/common/v_openssl.c

diff --git a/xen/common/TrustedCert.h b/xen/common/TrustedCert.h
new file mode 100644
index 000..5b744e4
--- /dev/null
+++ b/xen/common/TrustedCert.h
@@ -0,0 +1,113 @@
+
+uint8_t TrustedCert[] = 
+{
+0x30,  0x82,  0x03,  0xC5,  0x30,  0x82,  0x02,  0xAD,  0xA0,  
+0x03,  0x02,  0x01,  0x02,  0x02,  0x10,  0x02,  0xAC,  0x5C,  
+0x26,  0x6A,  0x0B,  0x40,  0x9B,  0x8F,  0x0B,  0x79,  0xF2,  
+0xAE,  0x46,  0x25,  0x77,  0x30,  0x0D,  0x06,  0x09,  0x2A,  
+0x86,  0x48,  0x86,  0xF7,  0x0D,  0x01,  0x01,  0x05,  0x05,  
+0x00,  0x30,  0x6C,  0x31,  0x0B,  0x30,  0x09,  0x06,  0x03,  
+0x55,  0x04,  0x06,  0x13,  0x02,  0x55,  0x53,  0x31,  0x15,  
+0x30,  0x13,  0x06,  0x03,  0x55,  0x04,  0x0A,  0x13,  0x0C,  
+0x44,  0x69,  0x67,  0x69,  0x43,  0x65,  0x72,  0x74,  0x20,  
+0x49,  0x6E,  0x63,  0x31,  0x19,  0x30,  0x17,  0x06,  0x03,  
+0x55,  0x04,  0x0B,  0x13,  0x10,  0x77,  0x77,  0x77,  0x2E,  
+0x64,  0x69,  0x67,  0x69,  0x63,  0x65,  0x72,  0x74,  0x2E,  
+0x63,  0x6F,  0x6D,  0x31,  0x2B,  0x30,  0x29,  0x06,  0x03,  
+0x55,  0x04,  0x03,  0x13,  0x22,  0x44,  0x69,  0x67,  0x69,  
+0x43,  0x65,  0x72,  0x74,  0x20,  0x48,  0x69,  0x67,  0x68,  
+0x20,  0x41,  0x73,  0x73,  0x75,  0x72,  0x61,  0x6E,  0x63,  
+0x65,  0x20,  0x45,  0x56,  0x20,  0x52,  0x6F,  0x6F,  0x74,  
+0x20,  0x43,  0x41,  0x30,  0x1E,  0x17,  0x0D,  0x30,  0x36,  
+0x31,  0x31,  0x31,  0x30,  0x30,  0x30,  0x30,  0x30,  0x30,  
+0x30,  0x5A,  0x17,  0x0D,  0x33,  0x31,  0x31,  0x31,  0x31,  
+0x30,  0x30,  0x30,  0x30,  0x30,  0x30,  0x30,  0x5A,  0x30,  
+0x6C,  0x31,  0x0B,  0x30,  0x09,  0x06,  0x03,  0x55,  0x04,  
+0x06,  0x13,  0x02,  0x55,  0x53,  0x31,  0x15,  0x30,  0x13,  
+0x06,  0x03,  0x55,  0x04,  0x0A,  0x13,  0x0C,  0x44,  0x69,  
+0x67,  0x69,  0x43,  0x65,  0x72,  0x74,  0x20,  0x49,  0x6E,  
+0x63,  0x31,  0x19,  0x30,  0x17,  0x06,  0x03,  0x55,  0x04,  
+0x0B,  0x13,  0x10,  0x77,  0x77,  0x77,  0x2E,  0x64,  0x69,  
+0x67,  0x69,  0x63,  0x65,  0x72,  0x74,  0x2E,  0x63,  0x6F,  
+0x6D,  0x31,  0x2B,  0x30,  0x29,  0x06,  0x03,  0x55,  0x04,  
+0x03,  0x13,  0x22,  0x44,  0x69,  0x67,  0x69,  0x43,  0x65,  
+0x72,  0x74,  0x20,  0x48,  0x69,  0x67,  0x68,  0x20,  0x41,  
+0x73,  0x73,  0x75,  0x72,  0x61,  0x6E,  0x63,  0x65,  0x20,  
+0x45,  0x56,  0x20,  0x52,  0x6F,  0x6F,  0x74,  0x20,  0x43,  
+0x41,  0x30,  0x82,  0x01,  0x22,  0x30,  0x0D,  0x06,  0x09,  
+0x2A,  0x86,  0x48,  0x86,  0xF7,  0x0D,  0x01,  0x01,  0x01,  
+0x05,  0x00,  0x03,  0x82,  0x01,  0x0F,  0x00,  0x30,  0x82,  
+0x01,  0x0A,  0x02,  0x82,  0x01,  0x01,  0x00,  0xC6,  0xCC,  
+0xE5,  0x73,  0xE6,  0xFB,  0xD4,  0xBB,  0xE5,  0x2D,  0x2D,  
+0x32,  0xA6,  0xDF,  0xE5,  0x81,  0x3F,  0xC9,  0xCD,  0x25,  
+0x49,  0xB6,  0x71,  0x2A,  0xC3,  0xD5,  0x94,  0x34,  0x67,  
+0xA2,  0x0A,  0x1C,  0xB0,  0x5F,  0x69,  0xA6,  0x40,  0xB1,  
+0xC4,  0xB7,  0xB2,  0x8F,  0xD0,  0x98,  0xA4,  0xA9,  0x41,  
+0x59,  0x3A,  0xD3,  0xDC,  0x94,  0xD6,  0x3C,  0xDB,  0x74,  
+0x38,  0xA4,  0x4A,  0xCC,  0x4D,  0x25,  0x82,  0xF7,  0x4A,  
+0xA5,  0x53,  0x12,  0x38,  0xEE,  0xF3,  0x49,  0x6D,  0x71,  
+0x91,  0x7E,  0x63,  0xB6,  0xAB,  0xA6,  0x5F,  0xC3,  0xA4,  
+0x84,  0xF8,  0x4F,  0x62,  0x51,  0xBE,  0xF8,  0xC5,  0xEC,  
+0xDB,  

[Xen-devel] [RFC v1 4/8] kexec: xen/common/Makefile: include building of OpenSSL

2019-01-14 Thread Eric DeVolder
Changes needed to the xen/common/Makefile in order to stitch in the
compiling of OpenSSL as well as the PECOFF file decoder and signature
verification code.

Signed-off-by: Eric DeVolder 
---
 xen/common/Makefile | 4 
 1 file changed, 4 insertions(+)

diff --git a/xen/common/Makefile b/xen/common/Makefile
index b1a5c42..bf91227 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -74,3 +74,7 @@ subdir-$(coverage) += gcov
 
 subdir-y += libelf
 subdir-$(HAS_DEVICE_TREE) += libfdt
+
+obj-$(HAS_KEXEC) += ped.o v_openssl.o
+subdir-$(HAS_KEXEC) += openssl-1.1.0i
+
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC v1 5/8] kexec: changes to facilitate compiling OpenSSL within Xen

2019-01-14 Thread Eric DeVolder
This is a collection of small changes to Xen sources to
allow compiling of OpenSSL within Xen kernel. There are a
few necessary code stubs, but vast majority is header file
manipulation to pass compilation.

Note I created an include2/ directory to place all the
standard include files needed by the (userland) OpenSSL
source files, as opposed to modifying about 145 source files.

Signed-off-by: Eric DeVolder 
---
 xen/arch/x86/Rules.mk   |   2 +
 xen/common/xmalloc_tlsf.c   |  25 
 xen/include/asm-x86/types.h |   2 +
 xen/include/xen/types.h |   3 +
 xen/include/xen/xmalloc.h   |   1 +
 xen/include2/assert.h   |   1 +
 xen/include2/bits/syslog-path.h |   1 +
 xen/include2/ctype.h|   1 +
 xen/include2/errno.h|   1 +
 xen/include2/features.h |   1 +
 xen/include2/inttypes.h |   1 +
 xen/include2/limits.h   |   1 +
 xen/include2/memory.h   |   1 +
 xen/include2/stdarg.h   |   1 +
 xen/include2/stddef.h   |   1 +
 xen/include2/stdint.h   |   1 +
 xen/include2/stdio.h|   1 +
 xen/include2/stdlib.h   |   1 +
 xen/include2/string.h   |   1 +
 xen/include2/strings.h  |   1 +
 xen/include2/sys/time.h |   1 +
 xen/include2/sys/types.h|   1 +
 xen/include2/syslog.h   |   1 +
 xen/include2/time.h |   1 +
 xen/include2/unistd.h   |   1 +
 xen/include2/xenossl.h  | 130 
 26 files changed, 183 insertions(+)
 create mode 100644 xen/include2/assert.h
 create mode 100644 xen/include2/bits/syslog-path.h
 create mode 100644 xen/include2/ctype.h
 create mode 100644 xen/include2/errno.h
 create mode 100644 xen/include2/features.h
 create mode 100644 xen/include2/inttypes.h
 create mode 100644 xen/include2/limits.h
 create mode 100644 xen/include2/memory.h
 create mode 100644 xen/include2/stdarg.h
 create mode 100644 xen/include2/stddef.h
 create mode 100644 xen/include2/stdint.h
 create mode 100644 xen/include2/stdio.h
 create mode 100644 xen/include2/stdlib.h
 create mode 100644 xen/include2/string.h
 create mode 100644 xen/include2/strings.h
 create mode 100644 xen/include2/sys/time.h
 create mode 100644 xen/include2/sys/types.h
 create mode 100644 xen/include2/syslog.h
 create mode 100644 xen/include2/time.h
 create mode 100644 xen/include2/unistd.h
 create mode 100644 xen/include2/xenossl.h

diff --git a/xen/arch/x86/Rules.mk b/xen/arch/x86/Rules.mk
index fe820d2..768026a 100644
--- a/xen/arch/x86/Rules.mk
+++ b/xen/arch/x86/Rules.mk
@@ -29,6 +29,8 @@ CFLAGS += -DXEN_FILE_ALIGN=PAGE_SIZE
 CFLAGS += -I$(BASEDIR)/include -Wa,-I$(BASEDIR)/include
 CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-generic
 CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-default
+CFLAGS += -I$(BASEDIR)/include2
+CFLAGS += -I$(BASEDIR)/common/openssl-1.1.0i/include
 
 CFLAGS += -ffunction-sections
 CFLAGS += -fdata-sections
diff --git a/xen/common/xmalloc_tlsf.c b/xen/common/xmalloc_tlsf.c
index b13317e..2d1ce40 100644
--- a/xen/common/xmalloc_tlsf.c
+++ b/xen/common/xmalloc_tlsf.c
@@ -569,6 +569,31 @@ static void tlsf_init(void)
 #define ZERO_BLOCK_PTR ((void *)-1L)
 #endif
 
+void *xrealloc (void *ptr, unsigned long size)
+{
+struct bhdr *b;
+b = (struct bhdr *)((char *) ptr - BHDR_OVERHEAD);
+if (0 == size)
+{
+xfree(ptr);
+ptr = NULL;
+}
+else if (size <= b->size)
+b->size = size;
+else /* size > b->size */
+{
+/* FIX!!! brute force alloc new and copy */
+char *newptr = xmalloc_array(char, size);
+if (newptr)
+{
+memcpy(newptr, ptr, size);
+}
+xfree(ptr);
+ptr = newptr;
+}
+return ptr;
+}
+
 void *_xmalloc(unsigned long size, unsigned long align)
 {
 void *p = NULL;
diff --git a/xen/include/asm-x86/types.h b/xen/include/asm-x86/types.h
index b82fa58..cb1e97a 100644
--- a/xen/include/asm-x86/types.h
+++ b/xen/include/asm-x86/types.h
@@ -40,6 +40,8 @@ typedef __SIZE_TYPE__ size_t;
 typedef unsigned long size_t;
 #endif
 typedef signed long ssize_t;
+# define __ssize_t_defined
+
 
 typedef char bool_t;
 #define test_and_set_bool(b)   xchg(&(b), 1)
diff --git a/xen/include/xen/types.h b/xen/include/xen/types.h
index 8596ded..8d4d365 100644
--- a/xen/include/xen/types.h
+++ b/xen/include/xen/types.h
@@ -24,6 +24,7 @@ typedef unsigned char   u_char;
 typedef unsigned short  u_short;
 typedef unsigned intu_int;
 typedef unsigned long   u_long;
+#  define __u_char_defined
 
 /* sysv */
 typedef unsigned char   unchar;
@@ -46,6 +47,8 @@ typedef __s32   int32_t;
 typedef __u64   uint64_t;
 typedef __u64   u_int64_t;
 typedef __s64   int64_t;
+#  define __int8_t_defined
+
 
 struct domain;
 struct vcpu;
diff --git a/xen/include/xen/xmalloc.h b/xen/include/xen/xmalloc.h
index 

[Xen-devel] [RFC v1 8/8] kexec: include OpenSSL build in xen.spec

2019-01-14 Thread Eric DeVolder
The changes to xen.spec are needed to unpack, configure and
generate a Makefile for building OpenSSL within Xen.

The changes to xen.spec also apply patches which are primarily
the new files as part of the signature verification effort,
as well as some tweaks to Xen files to facilitate compiling of
OpenSSL.

Signed-off-by: Eric DeVolder 
---
 xen.spec | 78 
 1 file changed, 78 insertions(+)

diff --git a/xen.spec b/xen.spec
index 4e36f7e..e026f28 100644
--- a/xen.spec
+++ b/xen.spec
@@ -1,5 +1,7 @@
 %{!?buildid: %{expand: %%define buildid 1}}
 
+%define openssl openssl-1.1.0i
+
 Name: xen
 Version: 4.4.4
 Release: %{buildid}%{?dist}
@@ -9,6 +11,15 @@ License: GPL
 URL: http://www.xenproject.org/
 BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root
 Source0: xen-4.4.tar.gz
+Source100: https://www.openssl.org/source/%{openssl}.tar.gz
+# FIX!!! This needs also for nosource 100
+Patch101: 0001-kexec-add-kexec_file_load-to-libxenctrl.patch
+Patch102: 0002-kexec-implement-kexec_file_load-for-PECOFF-Authentic.patch
+Patch103: 0003-kexec-new-file-openssl-1.1.0i.patch.patch
+Patch104: 0004-kexec-xen-common-Makefile-include-building-of-OpenSS.patch
+Patch105: 0005-kexec-changes-to-facilitate-compiling-OpenSSL-within.patch
+Patch106: 0006-kexec-support-files-for-PECOFF-Authenticode-signatur.patch
+Patch107: 0007-kexec-Xen-compatible-makefile-for-OpenSSL.patch
 
 %if 0%{?sbsignxen}
 Source21: securebootca.cer
@@ -72,6 +83,73 @@ manage Xen virtual machines.
 
 %prep
 %setup -q -n xen-4.4
+%patch101 -p1
+%patch102 -p1
+%patch104 -p1
+%patch105 -p1
+%patch106 -p1
+%patch107 -p1
+
+tar -z -x -v -f %{SOURCE100}
+%patch103 -p2
+mkdir -p $RPM_BUILD_DIR/openssl
+
+# Taken from EDK2 UEFI linux-x86_64
+(cd %{openssl} ; ./Configure --prefix=$RPM_BUILD_DIR/openssl \
+UEFI \
+no-afalgeng \
+no-asm \
+no-async \
+no-autoalginit \
+no-autoerrinit \
+no-bf \
+no-blake2 \
+no-camellia \
+no-capieng \
+no-cast \
+no-chacha \
+no-ct \
+no-deprecated \
+no-dgram \
+no-dsa \
+no-dso \
+no-dynamic-engine \
+no-ec \
+no-ec2m \
+no-engine \
+no-err \
+no-filenames \
+no-gost \
+no-hw \
+no-idea \
+no-mdc2 \
+no-ocb \
+no-poly1305 \
+no-posix-io \
+no-rc2 \
+no-rfc3779 \
+no-rmd160 \
+no-scrypt \
+no-seed \
+no-shared \
+no-sock \
+no-srp \
+no-ssl \
+no-stdio \
+no-threads \
+no-ts \
+no-ui \
+no-whirlpool \
+)
+#   no-pic \
+#   no-cms \
+
+# Essential
+(cd %{openssl} ; make build_all_generated)
+
+mv %{openssl}/Makefile %{openssl}/Makefile.original
+mv Makefile.%{openssl} %{openssl}/Makefile
+mv %{openssl} xen/common
 
 %build
 %configure --enable-xend --enable-ovmf
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC v1 0/8] Prototype for kexec signature verification within Xen

2019-01-14 Thread Eric DeVolder
On April 20, 2018, I posted to xen-devel an RFC inquiring about
support for signature verification of kexec within Xen:

https://lists.xenproject.org/archives/html/xen-devel/2018-04/msg01655.html

Since then, I've worked towards a solution. For the purposes of
understanding signature verification, I built a standalone utility to
parse the xen.mb.efi PECOFF file, hash it contents, and extract its
digitial certificate and perform the Authenticode signature
verification. Once this was all working, I integrated the files into
Xen.

I have a working prototype, which integrates [enough] OpenSSL into
Xen to enable kexec signature verification. Alas I now have different
priorities, but my employer did ask that I post this set of changes.
You may do with them as you wish. I would be available for consultation
should somebody wish to pursue this further.

Being a prototype, it has the following known-to-me shortcomings:

1: Does not following Xen coding standard. There may be areas where I
do not use the most appropriate Xen style, call or macro, or error
checking.

2: The adaptation of OpenSSL into Xen is incomplete. There are a number
of stub routines that have not been implemented (but currently do not
seem to interfere with the signature verification operation). Some
possible ways to address this are:
 - Properly implement these routines
 - Investigate further the OpenSSL configury to see if these can be
   configured away (Note that I chose OpenSSL-1.1.0i specifically
   because that is what EDK2 uses, and EDK2 is as close to Xen
   embedded/kernel environment (Otherwise OpenSSL is primarily a
   userland package)).
 - All 150+ OpenSSL files are compiled-in, could look at eliminating
   files manually.
 - Maybe look at newer OpenSSL versions, which might have additional
   configurability?
 - Perhaps instead utilize libgcrypt + libksba instead of OpenSSL.

3: A configure option is needed for the signature verification. This
option should simultaneously disable kexec_load while enabling
kexec_file_load.

4: Linux has infrastructure to support multiple file types as well as
multiple signature verification techniques. By contrast, this prototype
is hardwired for PECOFF+Authenticode (EFI) format.

5: Linux has keyring infrastructure to support multiple certificates.
Currently the appropriate root certificate to satisfy Oracle-signed
Xen kernel is compiled-in. This area alone would need significant
attention if any hope in upstreaming is to occur.

5: There is probably a better PECOFF decoder than the one currently in
use.

6: Convert the usage of DLCL macros to Xen standard list operations.

7: For the include2/ xenossl.h header file hack to facilitate
compiling OpenSSL within Xen; that needs to be revisited. I did
this to deal with the standard header files the (userland) OpenSSL
expects present; rather than changing nearly every OpenSSL source
file.

8: Analysis to understand the compiled-size increase, as well
as the run-time size increase?

9: A true security audit on these changes? For example, this prototype
still relies upon the kexec userland tool to provide the purgatory
executable. For obvious security reasons, this needs to be migrated
within Xen, as Linux does (note that involves some level of ELF
parsing and relocation support).

10: Licensing of the various pieces may be problematic.

Note that there is a corresponding change to kexec-tools to
allow/enable the Xen kexec_file_load() hypercall. Those changes
are not part of this change set, but will be posted separately.

Anyway, this does work, for me.
eric



Eric DeVolder (8):
  kexec: add kexec_file_load to libxenctrl
  kexec: implement kexec_file_load() for PECOFF+Authenticode files
  kexec: new file openssl-1.1.0i.patch
  kexec: xen/common/Makefile: include building of OpenSSL
  kexec: changes to facilitate compiling OpenSSL within Xen
  kexec: support files for PECOFF Authenticode signature verification
  kexec: Xen compatible makefile for OpenSSL
  kexec: include OpenSSL build in xen.spec

 Makefile.openssl-1.1.0i |  480 ++
 openssl-1.1.0i.patch|  378 +++
 tools/libxc/xc_kexec.c  |   41 ++
 tools/libxc/xenctrl.h   |4 +
 xen.spec|   78 +++
 xen/arch/x86/Rules.mk   |2 +
 xen/common/Makefile |4 +
 xen/common/TrustedCert.h|  113 
 xen/common/dlcl.h   |  323 ++
 xen/common/kexec.c  |  131 +++-
 xen/common/pecoff.h |  283 
 xen/common/ped.c|  579 +
 xen/common/ped.h|  128 
 xen/common/v_openssl.c  | 1348 +++
 xen/common/xmalloc_tlsf.c   |   25 +
 xen/include/asm-x86/types.h |2 +
 xen/include/public/kexec.h  |4 +-
 xen/include/xen/types.h |3 +
 xen/include/xen/xmalloc.h   |1 +
 xen/include2/assert.h   |1 +
 

[Xen-devel] [RFC v1 7/8] kexec: Xen compatible makefile for OpenSSL

2019-01-14 Thread Eric DeVolder
This is the Xen compatible makefile for use in building
OpenSSL within Xen.

This file was generated by capturing a "normal" build of
OpenSSL and parsing that build to ensure the correct list of
options and files.

Signed-off-by: Eric DeVolder 
---
 Makefile.openssl-1.1.0i | 480 
 1 file changed, 480 insertions(+)
 create mode 100644 Makefile.openssl-1.1.0i

diff --git a/Makefile.openssl-1.1.0i b/Makefile.openssl-1.1.0i
new file mode 100644
index 000..bd15ecb
--- /dev/null
+++ b/Makefile.openssl-1.1.0i
@@ -0,0 +1,480 @@
+# NOTE: This file generated by doit.sh
+# NOTE: It is in form needed by Xen build infrastructure
+# NOTE: Original OpenSSL makefile has been renamed
+CFLAGS += -Iinclude
+CFLAGS += -Icrypto/include
+CFLAGS += -Icrypto
+CFLAGS += -Icrypto/modes
+CFLAGS += -I.
+CFLAGS += -DENGINESDIR="\"/root/myroot/lib/engines-1.1\""
+CFLAGS += -DL_ENDIAN
+CFLAGS += -DOPENSSL_USE_NODELETE
+CFLAGS += -DNDEBUG
+CFLAGS += -DOPENSSLDIR="\"/root/myroot/ssl\""
+CFLAGS += -DOPENSSL_API_COMPAT=0x1010L
+CFLAGS += -DOPENSSL_NO_DYNAMIC_ENGINE
+CFLAGS += -DOPENSSL_PIC
+
+obj-$(HAS_KEXEC) += crypto/aes/aes_cbc.o
+obj-$(HAS_KEXEC) += crypto/aes/aes_cfb.o
+obj-$(HAS_KEXEC) += crypto/aes/aes_core.o
+obj-$(HAS_KEXEC) += crypto/aes/aes_ecb.o
+obj-$(HAS_KEXEC) += crypto/aes/aes_ige.o
+obj-$(HAS_KEXEC) += crypto/aes/aes_misc.o
+obj-$(HAS_KEXEC) += crypto/aes/aes_ofb.o
+obj-$(HAS_KEXEC) += crypto/aes/aes_wrap.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_bitstr.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_d2i_fp.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_digest.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_dup.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_gentm.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_i2d_fp.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_int.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_mbstr.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_object.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_octet.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_print.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_sign.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_strex.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_strnid.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_time.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_type.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_utctm.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_utf8.o
+obj-$(HAS_KEXEC) += crypto/asn1/a_verify.o
+obj-$(HAS_KEXEC) += crypto/asn1/ameth_lib.o
+obj-$(HAS_KEXEC) += crypto/asn1/asn1_err.o
+obj-$(HAS_KEXEC) += crypto/asn1/asn1_gen.o
+obj-$(HAS_KEXEC) += crypto/asn1/asn1_lib.o
+obj-$(HAS_KEXEC) += crypto/asn1/asn1_par.o
+obj-$(HAS_KEXEC) += crypto/asn1/asn_mime.o
+obj-$(HAS_KEXEC) += crypto/asn1/asn_moid.o
+obj-$(HAS_KEXEC) += crypto/asn1/asn_mstbl.o
+obj-$(HAS_KEXEC) += crypto/asn1/asn_pack.o
+obj-$(HAS_KEXEC) += crypto/asn1/bio_asn1.o
+obj-$(HAS_KEXEC) += crypto/asn1/bio_ndef.o
+obj-$(HAS_KEXEC) += crypto/asn1/d2i_pr.o
+obj-$(HAS_KEXEC) += crypto/asn1/d2i_pu.o
+obj-$(HAS_KEXEC) += crypto/asn1/evp_asn1.o
+obj-$(HAS_KEXEC) += crypto/asn1/f_int.o
+obj-$(HAS_KEXEC) += crypto/asn1/f_string.o
+obj-$(HAS_KEXEC) += crypto/asn1/i2d_pr.o
+obj-$(HAS_KEXEC) += crypto/asn1/i2d_pu.o
+obj-$(HAS_KEXEC) += crypto/asn1/n_pkey.o
+obj-$(HAS_KEXEC) += crypto/asn1/nsseq.o
+obj-$(HAS_KEXEC) += crypto/asn1/p5_pbe.o
+obj-$(HAS_KEXEC) += crypto/asn1/p5_pbev2.o
+obj-$(HAS_KEXEC) += crypto/asn1/p5_scrypt.o
+obj-$(HAS_KEXEC) += crypto/asn1/p8_pkey.o
+obj-$(HAS_KEXEC) += crypto/asn1/t_bitst.o
+obj-$(HAS_KEXEC) += crypto/asn1/t_pkey.o
+obj-$(HAS_KEXEC) += crypto/asn1/t_spki.o
+obj-$(HAS_KEXEC) += crypto/asn1/tasn_dec.o
+obj-$(HAS_KEXEC) += crypto/asn1/tasn_enc.o
+obj-$(HAS_KEXEC) += crypto/asn1/tasn_fre.o
+obj-$(HAS_KEXEC) += crypto/asn1/tasn_new.o
+obj-$(HAS_KEXEC) += crypto/asn1/tasn_prn.o
+obj-$(HAS_KEXEC) += crypto/asn1/tasn_scn.o
+obj-$(HAS_KEXEC) += crypto/asn1/tasn_typ.o
+obj-$(HAS_KEXEC) += crypto/asn1/tasn_utl.o
+obj-$(HAS_KEXEC) += crypto/asn1/x_algor.o
+obj-$(HAS_KEXEC) += crypto/asn1/x_bignum.o
+obj-$(HAS_KEXEC) += crypto/asn1/x_info.o
+obj-$(HAS_KEXEC) += crypto/asn1/x_int64.o
+obj-$(HAS_KEXEC) += crypto/asn1/x_long.o
+obj-$(HAS_KEXEC) += crypto/asn1/x_pkey.o
+obj-$(HAS_KEXEC) += crypto/asn1/x_sig.o
+obj-$(HAS_KEXEC) += crypto/asn1/x_spki.o
+obj-$(HAS_KEXEC) += crypto/asn1/x_val.o
+obj-$(HAS_KEXEC) += crypto/async/arch/async_null.o
+obj-$(HAS_KEXEC) += crypto/async/arch/async_posix.o
+obj-$(HAS_KEXEC) += crypto/async/arch/async_win.o
+obj-$(HAS_KEXEC) += crypto/async/async.o
+obj-$(HAS_KEXEC) += crypto/async/async_err.o
+obj-$(HAS_KEXEC) += crypto/async/async_wait.o
+obj-$(HAS_KEXEC) += crypto/bio/b_addr.o
+obj-$(HAS_KEXEC) += crypto/bio/b_dump.o
+obj-$(HAS_KEXEC) += crypto/bio/b_print.o
+obj-$(HAS_KEXEC) += crypto/bio/b_sock.o
+obj-$(HAS_KEXEC) += crypto/bio/b_sock2.o
+obj-$(HAS_KEXEC) += crypto/bio/bf_buff.o
+obj-$(HAS_KEXEC) += crypto/bio/bf_lbuf.o
+obj-$(HAS_KEXEC) += crypto/bio/bf_nbio.o
+obj-$(HAS_KEXEC) += crypto/bio/bf_null.o
+obj-$(HAS_KEXEC) += crypto/bio/bio_cb.o
+obj-$(HAS_KEXEC) += crypto/bio/bio_err.o
+obj-$(HAS_KEXEC) += crypto/bio/bio_lib.o
+obj-$(HAS_KEXEC) += 

Re: [Xen-devel] [PATCH v3 04/15] argo: init, destroy and soft-reset, with enable command line opt

2019-01-14 Thread Roger Pau Monné
On Mon, Jan 14, 2019 at 3:48 PM Wei Liu  wrote:
>
> Hi all
>
> The locking scheme seems to be remaining sticking point. The rest are
> mostly cosmetic issues (FAOD, they still need to be addressed).  Frankly
> I don't think there is enough time to address all the technical details,
> but let me sum up each side's position and see if we can reach an
> amicable solution.
>
> From maintainers and reviewers' point of view:
>
> 1. Maintainers / reviewers don't like complexity unless absolutely
>necessary.
> 2. Maintainers / reviewers feel they have a responsibility to understand
>the code and algorithm.
>
> Yet being the gatekeepers doesn't necessarily mean we understand every
> technical details and every usecase. We would like to, but most of the
> time it is unrealistic.
>
> Down to this specific patch series:
>
> Roger thinks the locking scheme is too complex. Christopher argues
> that's necessary for short-live channels to be performant.
>
> Both have their point.
>
> I think having a complex locking scheme is inevitable, just like we did
> for performant grant table several years ago.  Regardless of the timing
> issue we have at hand, asking Christopher to implement a stripped down
> version creates more work for him.
>
> Yet ignoring Roger's concerns is unfair to him as well, since he put in
> so much time and effort to understand the algorithm and provide
> suggestions. It is in fact unreasonable to ask anyone to fully
> understand the locking mechanism and check the implementation is correct
> in a few days (given the series was posted in Dec and there were major
> holidays in between, plus everyone had other commitments).
>
> To unblock this, how about we make Christopher maintainer of Argo? He
> and OpenXT will be on the hook for further improvement. And I believe it
> would be in their best interest to keep Argo bug-free and eventually
> make it become supported.
>
> So:
>
> 1. Make sure Argo is self-contained -- this requires careful review for
>interaction between Argo and other parts of the hypervisor.
> 2. Argo is going to be experimental and off-by-default -- this is the
>default status for new feature anyway.
> 3. Make Christopher maintainer of Argo -- this would be a natural thing
>to do anyway.
>
> Does this work for everyone?

I think this is fine.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] tools/firmware: update OVMF Makefile

2019-01-14 Thread Olaf Hering
Am Mon, 14 Jan 2019 17:44:57 +
schrieb Wei Liu :

>   -   $(GIT) submodule update --init --recursive
>   +   [ -d .git ] && $(GIT) submodule update --init --recursive

This syntax fails, but this works for me:

if test -d .git ; then $(GIT) submodule update --init --recursive ; fi


Olaf


pgpZqZSMHOzoP.pgp
Description: Digitale Signatur von OpenPGP
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable test] 131947: trouble: broken/fail/pass

2019-01-14 Thread osstest service owner
flight 131947 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/131947/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadowbroken
 test-amd64-amd64-xl-rtds broken
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm  broken
 test-xtf-amd64-amd64-3   broken
 test-amd64-i386-qemuu-rhel6hvm-intel broken
 test-amd64-amd64-xl  broken
 test-amd64-i386-xl-xsm   broken
 test-amd64-amd64-xl-pvshim   broken
 test-xtf-amd64-amd64-34 host-install(4)broken REGR. vs. 131787
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm  broken in 
131927
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 broken in 131927
 test-amd64-amd64-libvirt-vhd broken  in 131927
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict  broken in 
131927

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 4 host-install(4) broken 
in 131927 pass in 131947
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 4 host-install(4) broken 
in 131927 pass in 131947
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 4 host-install(4) broken in 131927 
pass in 131947
 test-amd64-amd64-libvirt-vhd 4 host-install(4) broken in 131927 pass in 131947
 test-amd64-i386-qemuu-rhel6hvm-intel  4 host-install(4)  broken pass in 131927
 test-amd64-i386-xl-xsm4 host-install(4)  broken pass in 131927
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 4 host-install(4) broken 
pass in 131927
 test-amd64-amd64-xl-pvshim4 host-install(4)  broken pass in 131927
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 4 host-install(4) broken pass 
in 131927
 test-amd64-amd64-xl-rtds  4 host-install(4)  broken pass in 131927
 test-amd64-amd64-xl   4 host-install(4)  broken pass in 131927
 test-amd64-i386-libvirt-xsm 18 guest-start/debian.repeat fail in 131927 pass 
in 131947
 test-amd64-amd64-livepatch7 xen-boot   fail pass in 131927

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail in 131927 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 131787
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 131787
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 131787
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 131787
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 131787
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 131787
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 131787
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 131787
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 131787
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 

Re: [Xen-devel] [PATCH v5 16/24] hw: acpi: Fix memory hotplug AML generation error

2019-01-14 Thread Michael S. Tsirkin
On Thu, Nov 08, 2018 at 03:23:41PM +0100, Igor Mammedov wrote:
> On Mon,  5 Nov 2018 02:40:39 +0100
> Samuel Ortiz  wrote:
> 
> > From: Yang Zhong 
> > 
> > When using the generated memory hotplug AML, the iasl
> > compiler would give the following error:
> > 
> > dsdt.dsl 266: Return (MOST (_UID, Arg0, Arg1, Arg2))
> > Error 6080 - Called method returns no value ^
> > 
> > Signed-off-by: Yang Zhong 
> Reviewed-by: Igor Mammedov 
> 
> I suggest to put this patch at the beginning of the series
> before reference tables in test are updated.

Samuel how about a separate small series with just bugfixes
for starters?


> > ---
> >  hw/acpi/memory_hotplug.c | 10 +-
> >  1 file changed, 5 insertions(+), 5 deletions(-)
> > 
> > diff --git a/hw/acpi/memory_hotplug.c b/hw/acpi/memory_hotplug.c
> > index db2c4df961..893fc2bd27 100644
> > --- a/hw/acpi/memory_hotplug.c
> > +++ b/hw/acpi/memory_hotplug.c
> > @@ -686,15 +686,15 @@ void build_memory_hotplug_aml(Aml *table, uint32_t 
> > nr_mem,
> >  
> >  method = aml_method("_OST", 3, AML_NOTSERIALIZED);
> >  s = MEMORY_SLOT_OST_METHOD;
> > -aml_append(method, aml_return(aml_call4(
> > -s, aml_name("_UID"), aml_arg(0), aml_arg(1), aml_arg(2)
> > -)));
> > +aml_append(method,
> > +   aml_call4(s, aml_name("_UID"), aml_arg(0),
> > + aml_arg(1), aml_arg(2)));
> >  aml_append(dev, method);
> >  
> >  method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> >  s = MEMORY_SLOT_EJECT_METHOD;
> > -aml_append(method, aml_return(aml_call2(
> > -   s, aml_name("_UID"), aml_arg(0;
> > +aml_append(method,
> > +   aml_call2(s, aml_name("_UID"), aml_arg(0)));
> >  aml_append(dev, method);
> >  
> >  aml_append(dev_container, dev);

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] PING ARM [PATCH v2] xen/cmdline: Fix buggy strncmp(s, LITERAL, ss - s) construct

2019-01-14 Thread Andrew Cooper
On 14/01/2019 17:16, Julien Grall wrote:
> Hi Andrew,
>
> On 14/01/2019 16:59, Andrew Cooper wrote:
>> On 14/01/2019 16:07, Julien Grall wrote:
>>> On 14/01/2019 15:17, Andrew Cooper wrote:
> diff --git a/xen/arch/arm/efi/efi-boot.h
> b/xen/arch/arm/efi/efi-boot.h
> index ca655ff..22a86ec 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -212,7 +212,7 @@ EFI_STATUS __init
> fdt_add_uefi_nodes(EFI_SYSTEM_TABLE *sys_table,
>    break;
>      type = fdt_getprop(fdt, node, "device_type", );
> -    if ( type && strncmp(type, "memory", len) == 0 )
> +    if ( type && len == 6 && strncmp(type, "memory", 6) == 0 )
>>>
>>> string property terminates with NUL and is included in the len. So I
>>> don't think this change is correct.
>>
>> Are you saying that len is 7 here then?
>
> Yes. But I don't think this change is necessary as we already include
> NUL in the comparison.

Ah - fair point.  I'll drop this hunk then.

Are you happy with the adjustment in arch/arm/cpuerrata.c ?

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] tools/firmware: update OVMF Makefile

2019-01-14 Thread Anthony PERARD
On Mon, Jan 14, 2019 at 05:44:57PM +, Wei Liu wrote:
> On Mon, Jan 14, 2019 at 06:27:44PM +0100, Olaf Hering wrote:
> > Am Mon, 14 Jan 2019 17:11:56 +
> > schrieb Anthony PERARD :
> > 
> > > I think it's fine to keep the current `submodule update` call where it
> > > is. We could just make it check that it's an actual git worktree by
> > > checking for the presence of ".git" (file or directory) before executing
> > > git.
> > > 
> > > Would that be good enough?
> > 
> > Maybe. Whatever works with env WGET/GIT=/bin/false ./configure $options.
> > 
> 
> Can you try this?
> 
> diff --git a/tools/firmware/ovmf-makefile b/tools/firmware/ovmf-makefile
> index 3de2fc0300..649482bca8 100644
> --- a/tools/firmware/ovmf-makefile
> +++ b/tools/firmware/ovmf-makefile
> @@ -16,7 +16,7 @@ all: build
> 
>  .PHONY: build
>   build:
>   -   $(GIT) submodule update --init --recursive
>   +   [ -d .git ] && $(GIT) submodule update --init --recursive

Please, use -e or -r, it is perfectly reasonable to have .git been a
file. (someone could use `git worktree`, or even a submodule)

Also, I think we should use `if`, As make must not fail if it isn't a
git worktree. (or "|| true" or "||:", but that would hide git failures.)

>   OvmfPkg/build.sh -a X64 -b $(TARGET) -n 4
> cp Build/OvmfX64/$(TARGET)_GCC*/FV/OVMF.fd ovmf.bin
> 

-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 04/15] argo: init, destroy and soft-reset, with enable command line opt

2019-01-14 Thread Christopher Clark
On Mon, Jan 14, 2019 at 6:47 AM Wei Liu  wrote:
>
> Hi all
>
> The locking scheme seems to be remaining sticking point. The rest are
> mostly cosmetic issues (FAOD, they still need to be addressed).  Frankly
> I don't think there is enough time to address all the technical details,
> but let me sum up each side's position and see if we can reach an
> amicable solution.
>
> From maintainers and reviewers' point of view:
>
> 1. Maintainers / reviewers don't like complexity unless absolutely
>necessary.
> 2. Maintainers / reviewers feel they have a responsibility to understand
>the code and algorithm.
>
> Yet being the gatekeepers doesn't necessarily mean we understand every
> technical details and every usecase. We would like to, but most of the
> time it is unrealistic.
>
> Down to this specific patch series:
>
> Roger thinks the locking scheme is too complex. Christopher argues
> that's necessary for short-live channels to be performant.
>
> Both have their point.
>
> I think having a complex locking scheme is inevitable, just like we did
> for performant grant table several years ago.  Regardless of the timing
> issue we have at hand, asking Christopher to implement a stripped down
> version creates more work for him.
>
> Yet ignoring Roger's concerns is unfair to him as well, since he put in
> so much time and effort to understand the algorithm and provide
> suggestions. It is in fact unreasonable to ask anyone to fully
> understand the locking mechanism and check the implementation is correct
> in a few days (given the series was posted in Dec and there were major
> holidays in between, plus everyone had other commitments).
>
> To unblock this, how about we make Christopher maintainer of Argo? He
> and OpenXT will be on the hook for further improvement. And I believe it
> would be in their best interest to keep Argo bug-free and eventually
> make it become supported.
>
> So:
>
> 1. Make sure Argo is self-contained -- this requires careful review for
>interaction between Argo and other parts of the hypervisor.
> 2. Argo is going to be experimental and off-by-default -- this is the
>default status for new feature anyway.
> 3. Make Christopher maintainer of Argo -- this would be a natural thing
>to do anyway.
>
> Does this work for everyone?

Yes, this will work for me. Thanks for the summary and proposal.

Christopher

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 ] always clear the X2APIC_ENABLE bit for PV guest

2019-01-14 Thread Boris Ostrovsky
On 1/14/19 7:43 AM, Igor Druzhinin wrote:
> ping?

Applied to for-linus-4.21 (nka 5.0)

(this should have been copied to linux-ker...@vger.kernel.org and to me,
which is partly the reason why it was missed)

-boris


>
> On 10/12/2018 10:03, Xin Li wrote:
>> From: Talons Lee 
>>
>> Commit e657fcc clears cpu capability bit instead of using fake cpuid
>> value, the EXTD should always be off for PV guest without depending
>> on cpuid value. So remove the cpuid check in xen_read_msr_safe() to
>> always clear the X2APIC_ENABLE bit.
>>
>> Signed-off-by: Talons Lee 
>> Reviewed-by: Juergen Gross 
>>
>> ---
>> CC: Igor Druzhinin 
>> CC: Sergey Dyasli 
>> CC: Andrew Cooper 
>> CC: Juergen Gross 
>>
>> v2:
>> don't use fake cpuid to cheat xen_read_msr_safe(), just always clear
>> the EXTD bit.
>> ---
>>  arch/x86/xen/enlighten_pv.c | 5 +
>>  1 file changed, 1 insertion(+), 4 deletions(-)
>>
>> diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
>> index 4b20082..17cf92b 100644
>> --- a/arch/x86/xen/enlighten_pv.c
>> +++ b/arch/x86/xen/enlighten_pv.c
>> @@ -900,10 +900,7 @@ static u64 xen_read_msr_safe(unsigned int msr, int *err)
>>  val = native_read_msr_safe(msr, err);
>>  switch (msr) {
>>  case MSR_IA32_APICBASE:
>> -#ifdef CONFIG_X86_X2APIC
>> -if (!(cpuid_ecx(1) & (1 << (X86_FEATURE_X2APIC & 31
>> -#endif
>> -val &= ~X2APIC_ENABLE;
>> +val &= ~X2APIC_ENABLE;
>>  break;
>>  }
>>  return val;
>>
> ___
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Xen 4.12 Development Update

2019-01-14 Thread Stefano Stabellini
On Mon, 14 Jan 2019, Lars Kurth wrote:
> Hi Juergen,
> 
> > On 14 Jan 2019, at 10:13, Juergen Gross  wrote:
> > 
> > This email only tracks big items for xen.git tree. Please reply for items 
> > you
> > would like to see in 4.12 so that people have an idea what is going on and
> > prioritise accordingly.
> > 
> > You're welcome to provide description and use cases of the feature you're
> > working on.
> > 
> > = Timeline =
> > 
> > We now adopt a fixed cut-off date scheme. We will release about every 8 
> > months.
> > The upcoming 4.12 timeline are as followed:
> > 
> > * Last posting date: December 14th, 2018
> >  Last posting date for patches touching ARM code: December 1st, 2018
> > * Hard code freeze: January 11th, 2019
> >  Hard code freeze for patches touching ARM code: December 21st, 2018
> > --> we are here
> > * RC1: TBD
> > * Release: March 7th, 2019
> > 
> > Note that we don't have freeze exception scheme anymore. All patches
> > that wish to go into 4.12 must be posted initially no later than the
> > last posting date and finally no later than the hard code freeze. All
> > patches posted after that date will be automatically queued into next
> > release.
> > 
> > RCs will be arranged immediately after freeze.
> 
> We should start planning on a Test Day schedule.
> 
> > We recently introduced a jira instance to track all the tasks (not only big)
> > for the project. See: https://xenproject.atlassian.net/projects/XEN/issues.
> > 
> > Some of the tasks tracked by this e-mail also have a corresponding jira task
> > referred by XEN-N.
> > 
> > I have started to include the version number of series associated to each
> > feature. Can each owner send an update on the version number if the series
> > was posted upstream?
> > 
> > = Projects =
> > 
> > == Hypervisor == 
> > 
> > *  Improvements to domain creation (v2)
> >  -  Andrew Cooper
> > 
> > *  Argo (inter-VM communication) (v3)
> >  -  Christopher Clark
> > 
> > *  Core aware scheduling (RFC v1)
> >  -  Dario Faggioli
> > 
> > *  Core aware scheduling for credit2 (RFC v1)
> >  -  Dario Faggioli
> > 
> > === x86 === 
> > 
> > *  hypervisor x86 instruction emulator additions for AVX512 (v7)
> >  -  Jan Beulich
> > 
> > *  qemu deprivilege (v4)
> >  -  George Dunlap
> > 
> > *  Fixes to #DB injection
> >  -  Andrew Cooper
> > 
> > *  CPUID/MSR Xen/toolstack improvements
> >  -  Andrew Cooper
> > 
> > *  Improvements to domain_crash()
> >  -  Andrew Cooper
> > 
> > === ARM === 
> > 
> > == Completed == 
> > 
> > *  guest resource mapping
> >  -  Paul Durrant
> > 
> > *  PV-only hypervisor
> >  -  Wei Liu
> > 
> > *  HVM-only hypervisor
> >  -  Wei Liu
> > 
> > *  Make credit2 scheduler the default
> >  -  George Dunlap
> > 
> > *  Grub2: Support PVH guest boot
> >  -  Juergen Gross
> > 
> > *  Fix VGA logdirty related display freezes with altp2m
> >  -  Razvan Cojocaru
> > 
> > *  dom0less (boot multiple domains from device tree)
> >  -  Stefano Stabellini
> > 
> > *  Implement Set/Way operations
> >  -  Julien Grall
> 
> @Stefano:
> Didn't the ARM KCONFIG stuff go in *after* 4.11? If so, this should probably 
> be added. 
> Can't recall the series name

Yes, after 4.11. xen/arch/arm/configs/tiny64.conf is new in 4.12.


> Also, I think the Aggios changes went in after 4.11 was released also.
> The series was "xen/arm64: Suspend preconditions and CPU hotplug fixes"

Yes, that's right


> @ALL: also, for any major new features and/or enablers, we should look at the 
> docs and make sure they are in place and up-to-date, that SUPPORT.md is 
> updated and that any worthy/big enough features are listed. Also, if you 
> contributed a larger series/feature and it is not on this list, please let us 
> know.  
> 
> Thank you to everyone contributing to the project


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] tools/firmware: update OVMF Makefile

2019-01-14 Thread Wei Liu
On Mon, Jan 14, 2019 at 06:27:44PM +0100, Olaf Hering wrote:
> Am Mon, 14 Jan 2019 17:11:56 +
> schrieb Anthony PERARD :
> 
> > I think it's fine to keep the current `submodule update` call where it
> > is. We could just make it check that it's an actual git worktree by
> > checking for the presence of ".git" (file or directory) before executing
> > git.
> > 
> > Would that be good enough?
> 
> Maybe. Whatever works with env WGET/GIT=/bin/false ./configure $options.
> 

Can you try this?

diff --git a/tools/firmware/ovmf-makefile b/tools/firmware/ovmf-makefile
index 3de2fc0300..649482bca8 100644
--- a/tools/firmware/ovmf-makefile
+++ b/tools/firmware/ovmf-makefile
@@ -16,7 +16,7 @@ all: build

 .PHONY: build
  build:
  -   $(GIT) submodule update --init --recursive
  +   [ -d .git ] && $(GIT) submodule update --init --recursive
  OvmfPkg/build.sh -a X64 -b $(TARGET) -n 4
  cp Build/OvmfX64/$(TARGET)_GCC*/FV/OVMF.fd ovmf.bin


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] tools/firmware: update OVMF Makefile

2019-01-14 Thread Wei Liu
On Mon, Jan 14, 2019 at 05:11:56PM +, Anthony PERARD wrote:
[...]
> 
> I think it's fine to keep the current `submodule update` call where it
> is. We could just make it check that it's an actual git worktree by
> checking for the presence of ".git" (file or directory) before executing
> git.

I concur.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PULL 00/25] Xen queue v2

2019-01-14 Thread Peter Maydell
On Mon, 14 Jan 2019 at 13:52, Anthony PERARD  wrote:
>
> The following changes since commit 7260438b7056469610ee166f7abe9ff8a26b8b16:
>
>   Merge remote-tracking branch 
> 'remotes/palmer/tags/riscv-for-master-3.2-part2' into staging (2019-01-14 
> 11:41:43 +)
>
> are available in the Git repository at:
>
>   https://xenbits.xen.org/git-http/people/aperard/qemu-dm.git 
> tags/pull-xen-20190114
>
> for you to fetch changes up to c6025bd197d0dbcc5067553fd12538d8b29383c2:
>
>   xen-block: avoid repeated memory allocation (2019-01-14 13:45:40 +)
>
> 
> Xen queue
>
> * Xen PV backend 'qdevification'.
>   Starting with xen_disk.
> * Performance improvements for xen-block.
> * Remove of the Xen PV domain builder.
> * bug fixes.
>


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/4.0
for any user-visible changes.

-- PMM

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [ovmf test] 131950: regressions - FAIL

2019-01-14 Thread osstest service owner
flight 131950 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/131950/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i3866 xen-buildfail REGR. vs. 129475
 build-i386-xsm6 xen-buildfail REGR. vs. 129475
 build-amd64-xsm   6 xen-buildfail REGR. vs. 129475
 build-amd64   6 xen-buildfail REGR. vs. 129475

Tests which did not succeed, but are not blocking:
 build-i386-libvirt1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a

version targeted for testing:
 ovmf 28ce4cb3590bc3aaa91c3be75429d4e8722415e2
baseline version:
 ovmf 5ae3184d8c59f7bbb84bad482df6b8020ba58188

Last test of basis   129475  2018-11-05 21:13:11 Z   69 days
Failing since129526  2018-11-06 20:49:26 Z   68 days  264 attempts
Testing same since   131950  2019-01-13 23:11:26 Z0 days1 attempts


People who touched revisions under test:
  Achin Gupta 
  Alex James 
  Ard Biesheuvel 
  Ashish Singhal 
  Bob Feng 
  bob.c.f...@intel.com 
  BobCF 
  Carsey, Jaben 
  Chasel Chiu 
  Chasel, Chiu 
  Chen A Chen 
  Chu, Maggie 
  Dandan Bi 
  David Wei 
  Derek Lin 
  Eric Dong 
  Eugene Cohen 
  Feng, Bob C 
  Fu Siyuan 
  Gary Lin 
  Hao Wu 
  Jaben Carsey 
  Jagadeesh Ujja 
  Jeff Brasen 
  Jian J Wang 
  Jiaxin Wu 
  Jiewen Yao 
  Laszlo Ersek 
  Leif Lindholm 
  Liming Gao 
  Liu Yu 
  Maggie Chu 
  Marc Zyngier 
  Marcin Wojtas 
  Mike Maslenkin 
  Ming Huang 
  Pedroa Liu 
  Ruiyu Ni 
  Sami Mujawar 
  shenglei 
  Shenglei Zhang 
  Siyuan Fu 
  Songpeng Li 
  Star Zeng 
  Sughosh Ganu 
  Sumit Garg 
  Sun, Zailiang 
  Thomas Abraham 
  Thomas Rydman 
  Ting Ye 
  Tomasz Michalec 
  Vijayenthiran Subramaniam 
  Vladimir Olovyannikov 
  Wang BinX A 
  Wu Jiaxin 
  Ye Ting 
  Yonghong Zhu 
  yuchenlin 
  Zailiang Sun 
  Zhang, Chao B 
  Zhao, ZhiqiangX 
  Zhiju.Fan 
  zhijufan 
  ZhiqiangX Zhao 
  zwei4 

jobs:
 build-amd64-xsm  fail
 build-i386-xsm   fail
 build-amd64  fail
 build-i386   fail
 build-amd64-libvirt  blocked 
 build-i386-libvirt   blocked 
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64  blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 5829 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] tools/firmware: update OVMF Makefile

2019-01-14 Thread Olaf Hering
Am Mon, 14 Jan 2019 17:11:56 +
schrieb Anthony PERARD :

> I think it's fine to keep the current `submodule update` call where it
> is. We could just make it check that it's an actual git worktree by
> checking for the presence of ".git" (file or directory) before executing
> git.
> 
> Would that be good enough?

Maybe. Whatever works with env WGET/GIT=/bin/false ./configure $options.

Olaf


pgpvmPhXJuy60.pgp
Description: Digitale Signatur von OpenPGP
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v6 1/4] xen: introduce SYMBOL

2019-01-14 Thread Julien Grall

Hi Jan,

On 14/01/2019 16:44, Jan Beulich wrote:


extern struct my_struct __start[];
extern struct my_struct __end[];

void test(const struct my_struct *);

void foo(int i) {
test(i ? __start : __end);
}


Your example doesn't contain any potential undefined behavior. So, how this 
relevant with our discussion here?


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] tools/firmware: update OVMF Makefile

2019-01-14 Thread Anthony PERARD
On Mon, Jan 14, 2019 at 12:48:52PM +0100, Olaf Hering wrote:
> Am Mon, 14 Jan 2019 11:28:57 +
> schrieb Wei Liu :
> 
> > Are you saying that the breakage is shown when you put a snapshot of
> > ovmf under xen.git? How does ovmf distribute their snapshot? How is that
> > generated? Does it contain snapshots of submodules it needs already?
> 
> I export all required sources recursively and store them in the appropriate 
> directories.
> qemu itself requires only the keycodemapdb, it does no download-on-demand.
> 
> And now that I had a chance to look at the sources, qemu-xen-dir-find calls a 
> custom script that will do nothing if the -remote already exists.
> 
> I think instead of changing the custom Makefile, the submodules have to be 
> handled in git-checkout.sh which is invoked by ovmf-dir.

I'm not sure if that's a good idea, we would end up downloading a lot of
data for just one submodules that xen.git needs. (QEMU as quite a few
submodules that are never needed).

Also, git-checkout.sh isn't the only place where `git` is called. All
the *-force-update targets calls git without the script, `mktarball`
is another script that calls git.

I think it's fine to keep the current `submodule update` call where it
is. We could just make it check that it's an actual git worktree by
checking for the presence of ".git" (file or directory) before executing
git.

Would that be good enough?

> If I remember correctly, when keycodemapdb became a hard requirement nothing 
> in xen.git was adjusted.

That's not exactly true, we had to adjust the `src-tarball-release` make
target to dl QEMU's submodules (or rather call QEMU's own release
tarball script).

And qemu.git and edk2.git are different. QEMU's build
system takes care of its own submodules, like xen.git does,
dl-on-demand. edk2.git doesn't do anything and let the developper
download openssl on its own, either via git submodule or by dl a
specific release of openssl[1].

[1] https://github.com/tianocore/tianocore.github.io/wiki/UDK2018-How-to-Build

-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] PING ARM [PATCH v2] xen/cmdline: Fix buggy strncmp(s, LITERAL, ss - s) construct

2019-01-14 Thread Julien Grall

Hi Andrew,

On 14/01/2019 16:59, Andrew Cooper wrote:

On 14/01/2019 16:07, Julien Grall wrote:

On 14/01/2019 15:17, Andrew Cooper wrote:

diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index ca655ff..22a86ec 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -212,7 +212,7 @@ EFI_STATUS __init
fdt_add_uefi_nodes(EFI_SYSTEM_TABLE *sys_table,
   break;
     type = fdt_getprop(fdt, node, "device_type", );
-    if ( type && strncmp(type, "memory", len) == 0 )
+    if ( type && len == 6 && strncmp(type, "memory", 6) == 0 )


string property terminates with NUL and is included in the len. So I
don't think this change is correct.


Are you saying that len is 7 here then?


Yes. But I don't think this change is necessary as we already include NUL in the 
comparison.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] PING ARM [PATCH v2] xen/cmdline: Fix buggy strncmp(s, LITERAL, ss - s) construct

2019-01-14 Thread Andrew Cooper
On 14/01/2019 16:07, Julien Grall wrote:
> Hi Andrew,
>
> On 14/01/2019 15:17, Andrew Cooper wrote:
>>> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
>>> index ca655ff..22a86ec 100644
>>> --- a/xen/arch/arm/efi/efi-boot.h
>>> +++ b/xen/arch/arm/efi/efi-boot.h
>>> @@ -212,7 +212,7 @@ EFI_STATUS __init
>>> fdt_add_uefi_nodes(EFI_SYSTEM_TABLE *sys_table,
>>>   break;
>>>     type = fdt_getprop(fdt, node, "device_type", );
>>> -    if ( type && strncmp(type, "memory", len) == 0 )
>>> +    if ( type && len == 6 && strncmp(type, "memory", 6) == 0 )
>
> string property terminates with NUL and is included in the len. So I
> don't think this change is correct.

Are you saying that len is 7 here then?

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [XEN][ARM64]: Qemu unhandled level 1 translation fault and p2m failure

2019-01-14 Thread Julien Grall

(+ Stefano and Oleksandr)

On 09/01/2019 06:54, vikram k.s. wrote:

Hello,


Hello,

I have seen this e-mail posted a couple of times. I will only answer to this 
e-mail.

First of all, I would recommend to CC people that might be interested with the 
topic, so the thread get flagged in our inbox. Otherwise, you may have to wait 
longer for an answer. Don't worry if you don't get the right people, they will 
CC relevant people.




  * We are using XEN-4.12.


Xen 4.12 is not yet released. So what is the commit used?


  * Linux 4.14 as dom0 and Linux4.19 as domU.


Please provide more details such as the hardware you use, the guest 
configuration, the full log (Xen + Dom0 + DomU).




When domU is create getting below log.
qemu-system-i38[3478]: unhandled level 1 translation fault (11) at 0x0010, 
esr 0x9205, in libxengnttab.so.1.2[7faf9cd000+3000]


What PV drivers are you using?



Also  p2m is failing .
(XEN) p2m.c:1456: d2v4: gvirt_to_maddr failed va=0x80001dfa1090 flags=0x1 
par=0x80


I think this is unrelated, this is more likely related to KPTI and a known issue 
that needs to be fixed. It is in my TODO list, not sure when it is going to be 
resolved though. Any help would be welcomed here.


Best regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] pvcalls-front: Replace GFP_KERNEL with GFP_ATOMIC in create_active

2019-01-14 Thread wangbo
Create_active may called inside spinlock,replace GFP_KERNEL with GFP_ATOMIC

Signed-off-by: wangbo 
---
 drivers/xen/pvcalls-front.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
index 77224d8..31bd3c9 100644
--- a/drivers/xen/pvcalls-front.c
+++ b/drivers/xen/pvcalls-front.c
@@ -344,11 +344,11 @@ static int create_active(struct sock_mapping *map, int 
*evtchn)
init_waitqueue_head(>active.inflight_conn_req);
 
map->active.ring = (struct pvcalls_data_intf *)
-   __get_free_page(GFP_KERNEL | __GFP_ZERO);
+   __get_free_page(GFP_ATOMIC | __GFP_ZERO);
if (map->active.ring == NULL)
goto out_error;
map->active.ring->ring_order = PVCALLS_RING_ORDER;
-   bytes = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
+   bytes = (void *)__get_free_pages(GFP_ATOMIC | __GFP_ZERO,
PVCALLS_RING_ORDER);
if (bytes == NULL)
goto out_error;
-- 
2.7.4



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] x86emul: fix test harness and fuzzer build dependencies

2019-01-14 Thread Ian Jackson
Jan Beulich writes ("Re: [Xen-devel] [PATCH] x86emul: fix test harness and 
fuzzer build dependencies"):
> So how do we make progress here? For the two changes that
> you dislike I don't formally need your ack, and I have Andrew's.
> I would (have to) respect an active NAK of yours, of course.

As I say, I think you are trying to something that is not well
supported by recursive make.  It follows that if any situations cause
particular trouble, they should be documented rather than fixed; or
maybe the general case should be documented too.  ("Only top-level
make targets are fully supported; you may use deeper targets directly
with make -C or by invoking make in a subdir, but at risk of
erroneously not (re)building other parts of the tree".)

But I don't think this is important enough for a NAK.

I'm not sure what you mean by "make progress".  Do you mean "how can
we dispose of this disagreement?"  I think our views are
irreconcilable.

If you and other committers have listened to me and still want to
commit this then so be it.

> For the one change that I need your (or Wei's) ack on, I didn't
> see any strong objection so far, and this fixes an actual issue
> with the overall tools build, i.e. _outside_ of the area you're
> concerned about. The $(MAKE) invocation there is not overly
> nice, but I thought I did convince you that - with the way
> tools/include/ gets dealt with from the top level - this should
> not be an issue. Plus I'm just moving it.

FAOD, the only thing I am objecting to is this kind of thing:

   +ifeq ($(MAKELEVEL),0)
   +$(XEN_ROOT)/tools/include/xen/lib/x86/cpuid-autogen.h: FORCE
   +   $(MAKE) -C $(XEN_ROOT)/tools/include build
   +endif

This appears twice in your v2.  The rest I have no difficulty with.

I don't think it is sensible to turn this into an argument about whose
bailiwick various hunks are in.  I am not going to cry foul if this
gets committed, particularly if Andrew has reviewed this conversation
and tells us that his ack stands.  I have said my piece.

HTH.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v6 1/4] xen: introduce SYMBOL

2019-01-14 Thread Jan Beulich
>>> On 14.01.19 at 17:28,  wrote:
> Hi Jan,
> 
> On 14/01/2019 15:52, Jan Beulich wrote:
> On 14.01.19 at 16:41,  wrote:
>>> Hi Jan,
>>>
>>> On 14/01/2019 10:11, Jan Beulich wrote:
>>> On 11.01.19 at 19:04,  wrote:
> On Fri, 11 Jan 2019, Jan Beulich wrote:
> On 11.01.19 at 03:14,  wrote:
>>> Hi Juergen, Jan,
>>>
>>> I spoke with Julien: we are both convinced that the unsigned long
>>> solution is best. But Julien also did some research and he thinks that
>>> Jan's version (returning pointer type) not only does not help with
>>> MISRA-C, but also doesn't solve the potential GCC problem either. A
>>> description of the GCC issue is available here:
>>>
>>>
>
>>> 
> https://kristerw.blogspot.com/2016/12/pointer-comparison-invalid-optimization 
>>> .h
> tml?m=1
>>
>> I've read through it, and besides not agreeing with some of the
>> author's arguments I wasn't able to spot where it tells me why/how
>> the suggested approach doesn't solve the problem.
>>
>>> (Also keep in mind that Linux uses the unsigned long solution to solve
>>> the GCC issue, deviating from it doesn't seem wise.)
>>
>> Which specific gcc issue (that is not solved by retaining type)?
>
> I am hoping Julien and his team will be able to provide the more
> decisive information next week for us to make a decision, but it looks
> like the issue is not clear-cut and people on the GCC list disagree on
> how it should be handled.
>
>
> The C standard says that "Two pointers compare equal if and only if both
> are null pointers, both are pointers to the same object (including a
> pointer to an object and a subobject at its beginning) or function, both
> are pointers to one past the last element of the same array object, or
> one is a pointer to one past the end of one array object and the other
> is a pointer to the start of a different array object that happens to
> immediately follow the first array object in the address space."
>
> In short, the compiler is free to return false in a pointer comparison
> if it believes that the pointers point to different non-consecutive
> object.

 And it is this "it believes" which we undermine with the construct:
 As long as the compiler can't prove two pointers point to different
 objects, it can't eliminate the actual comparison.
>>>
>>> May I ask where does this come from? A compiler could technically be free to
>>> assume the inverse. I.e as long as it can't prove two pointers point to
>>> different objects, it can rely on the undefined behavior to optimize it.
>> 
>> No. As long as there's a chance that both pointers point to the same
>> object, it can't do bad things, because _if_ they do, the result of the
>> comparison has to be correct (as per the text still quoted above).
> 
> In the following example (taken from [1]):
> 
> extern struct my_struct __start[];
> extern struct my_struct __end[];
> 
> void foo(void)
> {
>  for (struct my_struct *x = __start; x != __end; x++)
>  do_something(x);
> }
> 
> The compiler can't be sure that __start and __end are not equal.

You're inverting what was said before: Of course the compiler
can#t be sure the addresses are not equal. But from the mere
language structure it knows that __start[] and __end[] are
two different objects.

> Yet it may 
> decide they are always different and optimize it to an infinite loop. So 
> surely, 
> the compiler can do bad things with even simple code.

Of course.

> I am struggling to understand how using "asm volatile" and still returning a 
> pointer would help here. If the compiler managed to infer that __start and 
> __end 
> are always different, then there are no reason for this to not happen with 
> the 
> new construct.

Of course there is: There's no connection to the original object(s)
anymore. Same with

extern struct my_struct __start[];
extern struct my_struct __end[];

void test(const struct my_struct *);

void foo(int i) {
test(i ? __start : __end);
}

Wherever test() is defined, the compiler can't make assumptions
anymore (unless of course it gets to see the definition, e.g. via
whole program optimization). But if the function's implementation
lives in a different binary, the compiler just can't make any
assumptions anymore.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v6 1/4] xen: introduce SYMBOL

2019-01-14 Thread Jan Beulich
>>> On 14.01.19 at 17:26,  wrote:
> On Monday, January 14, 2019 10:53 AM, Jan Beulich wrote:
>> > Hi Jan,
>> >
>> > On 14/01/2019 10:11, Jan Beulich wrote:
>> > On 11.01.19 at 19:04,  wrote:
>> >>> On Fri, 11 Jan 2019, Jan Beulich wrote:
>> >>> On 11.01.19 at 03:14,  wrote:
>> > Hi Juergen, Jan,
>> >
>> > I spoke with Julien: we are both convinced that the unsigned long
>> > solution is best. But Julien also did some research and he thinks that
>> > Jan's version (returning pointer type) not only does not help with
>> > MISRA-C, but also doesn't solve the potential GCC problem either. A
>> > description of the GCC issue is available here:
>> >
>> >
>> >>>
>> > 
> https://kristerw.blogspot.com/2016/12/pointer-comparison-invalid-optimization 
>> > .h
>> >>> tml?m=1
>> 
>>  I've read through it, and besides not agreeing with some of the
>>  author's arguments I wasn't able to spot where it tells me why/how
>>  the suggested approach doesn't solve the problem.
>> 
>> > (Also keep in mind that Linux uses the unsigned long solution to solve
>> > the GCC issue, deviating from it doesn't seem wise.)
>> 
>>  Which specific gcc issue (that is not solved by retaining type)?
>> >>>
>> >>> I am hoping Julien and his team will be able to provide the more
>> >>> decisive information next week for us to make a decision, but it looks
>> >>> like the issue is not clear-cut and people on the GCC list disagree on
>> >>> how it should be handled.
>> >>>
>> >>>
>> >>> The C standard says that "Two pointers compare equal if and only if both
>> >>> are null pointers, both are pointers to the same object (including a
>> >>> pointer to an object and a subobject at its beginning) or function, both
>> >>> are pointers to one past the last element of the same array object, or
>> >>> one is a pointer to one past the end of one array object and the other
>> >>> is a pointer to the start of a different array object that happens to
>> >>> immediately follow the first array object in the address space."
>> >>>
>> >>> In short, the compiler is free to return false in a pointer comparison
>> >>> if it believes that the pointers point to different non-consecutive
>> >>> object.
>> >>
>> >> And it is this "it believes" which we undermine with the construct:
>> >> As long as the compiler can't prove two pointers point to different
>> >> objects, it can't eliminate the actual comparison.
>> >
>> > May I ask where does this come from? A compiler could technically be free 
>> > to
>> > assume the inverse. I.e as long as it can't prove two pointers point to
>> > different objects, it can rely on the undefined behavior to optimize it.
>> 
>> No. As long as there's a chance that both pointers point to the same
>> object, it can't do bad things, because _if_ they do, the result of the
>> comparison has to be correct (as per the text still quoted above).
>> 
>> Jan
> 
> In the following declaration:
> extern char _start[], _end[];
> it's still a valid interpretation that _start and _end point to different 
> objects. In fact, I think is already making this assumption, given that GCC 
> 7.3 will emit the following warning if "extern" is removed: "warning: array 
> ‘_end’ assumed to have one element"

I'm afraid you're mixing up things. Removing the "extern" converts the
construct from a declaration to a (latent, determined once the entire
CU has been parsed) definition. The expansion to one element that
the compiler does is specifically to avoid pointers to compare equal
despite pointing at different objects (if there was no element, the
next sequential object in memory would have the same address, and
hence pointers would compare equal).

> Who's to say that GCC someday won't become smart enough to sniff out all the 
> inline assembly and pointer-type casts and still draw the conclusion that 
> they point to separate objects?

Looking through casts between pointer types is already happening.
Inspecting the actual text of an asm() is, otoh, something that
should never happen, as it would be extremely error prone.

> What if GCC someday will make these decisions 
> based on some sort of link-time optimization, like using an intermediary or 
> iterative symbol/linker map of sorts?

That's why hiding it through an asm() is imo best, because the asm()
will be retained in the internal representation, and hence remain
immune from such optimizations.

> Also, the above text only concerns equality comparisons. What about 
> greater/less/neq comparisons, arithmetic, and dereferencing?

Different subject.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v6 1/2] xen/blkback: add stack variable 'blkif' in connect_ring()

2019-01-14 Thread Dongli Zhang
As 'be->blkif' is used for many times in connect_ring(), the stack variable
'blkif' is added to substitute 'be-blkif'.

Suggested-by: Paul Durrant 
Signed-off-by: Dongli Zhang 
Reviewed-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
---
 drivers/block/xen-blkback/xenbus.c | 27 ++-
 1 file changed, 14 insertions(+), 13 deletions(-)

diff --git a/drivers/block/xen-blkback/xenbus.c 
b/drivers/block/xen-blkback/xenbus.c
index a4bc74e..a4aadac 100644
--- a/drivers/block/xen-blkback/xenbus.c
+++ b/drivers/block/xen-blkback/xenbus.c
@@ -1023,6 +1023,7 @@ static int read_per_ring_refs(struct xen_blkif_ring 
*ring, const char *dir)
 static int connect_ring(struct backend_info *be)
 {
struct xenbus_device *dev = be->dev;
+   struct xen_blkif *blkif = be->blkif;
unsigned int pers_grants;
char protocol[64] = "";
int err, i;
@@ -1033,25 +1034,25 @@ static int connect_ring(struct backend_info *be)
 
pr_debug("%s %s\n", __func__, dev->otherend);
 
-   be->blkif->blk_protocol = BLKIF_PROTOCOL_DEFAULT;
+   blkif->blk_protocol = BLKIF_PROTOCOL_DEFAULT;
err = xenbus_scanf(XBT_NIL, dev->otherend, "protocol",
   "%63s", protocol);
if (err <= 0)
strcpy(protocol, "unspecified, assuming default");
else if (0 == strcmp(protocol, XEN_IO_PROTO_ABI_NATIVE))
-   be->blkif->blk_protocol = BLKIF_PROTOCOL_NATIVE;
+   blkif->blk_protocol = BLKIF_PROTOCOL_NATIVE;
else if (0 == strcmp(protocol, XEN_IO_PROTO_ABI_X86_32))
-   be->blkif->blk_protocol = BLKIF_PROTOCOL_X86_32;
+   blkif->blk_protocol = BLKIF_PROTOCOL_X86_32;
else if (0 == strcmp(protocol, XEN_IO_PROTO_ABI_X86_64))
-   be->blkif->blk_protocol = BLKIF_PROTOCOL_X86_64;
+   blkif->blk_protocol = BLKIF_PROTOCOL_X86_64;
else {
xenbus_dev_fatal(dev, err, "unknown fe protocol %s", protocol);
return -ENOSYS;
}
pers_grants = xenbus_read_unsigned(dev->otherend, "feature-persistent",
   0);
-   be->blkif->vbd.feature_gnt_persistent = pers_grants;
-   be->blkif->vbd.overflow_max_grants = 0;
+   blkif->vbd.feature_gnt_persistent = pers_grants;
+   blkif->vbd.overflow_max_grants = 0;
 
/*
 * Read the number of hardware queues from frontend.
@@ -1067,16 +1068,16 @@ static int connect_ring(struct backend_info *be)
requested_num_queues, xenblk_max_queues);
return -ENOSYS;
}
-   be->blkif->nr_rings = requested_num_queues;
-   if (xen_blkif_alloc_rings(be->blkif))
+   blkif->nr_rings = requested_num_queues;
+   if (xen_blkif_alloc_rings(blkif))
return -ENOMEM;
 
pr_info("%s: using %d queues, protocol %d (%s) %s\n", dev->nodename,
-be->blkif->nr_rings, be->blkif->blk_protocol, protocol,
+blkif->nr_rings, blkif->blk_protocol, protocol,
 pers_grants ? "persistent grants" : "");
 
-   if (be->blkif->nr_rings == 1)
-   return read_per_ring_refs(>blkif->rings[0], dev->otherend);
+   if (blkif->nr_rings == 1)
+   return read_per_ring_refs(>rings[0], dev->otherend);
else {
xspathsize = strlen(dev->otherend) + xenstore_path_ext_size;
xspath = kmalloc(xspathsize, GFP_KERNEL);
@@ -1085,10 +1086,10 @@ static int connect_ring(struct backend_info *be)
return -ENOMEM;
}
 
-   for (i = 0; i < be->blkif->nr_rings; i++) {
+   for (i = 0; i < blkif->nr_rings; i++) {
memset(xspath, 0, xspathsize);
snprintf(xspath, xspathsize, "%s/queue-%u", 
dev->otherend, i);
-   err = read_per_ring_refs(>blkif->rings[i], xspath);
+   err = read_per_ring_refs(>rings[i], xspath);
if (err) {
kfree(xspath);
return err;
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v6 2/2] xen/blkback: rework connect_ring() to avoid inconsistent xenstore 'ring-page-order' set by malicious blkfront

2019-01-14 Thread Dongli Zhang
The xenstore 'ring-page-order' is used globally for each blkback queue and
therefore should be read from xenstore only once. However, it is obtained
in read_per_ring_refs() which might be called multiple times during the
initialization of each blkback queue.

If the blkfront is malicious and the 'ring-page-order' is set in different
value by blkfront every time before blkback reads it, this may end up at
the "WARN_ON(i != (XEN_BLKIF_REQS_PER_PAGE * blkif->nr_ring_pages));" in
xen_blkif_disconnect() when frontend is destroyed.

This patch reworks connect_ring() to read xenstore 'ring-page-order' only
once.

Signed-off-by: Dongli Zhang 
---
Changed since v1:
  * change the order of xenstore read in read_per_ring_refs
  * use xenbus_read_unsigned() in connect_ring()

Changed since v2:
  * simplify the condition check as "(err != 1 && nr_grefs > 1)"
  * avoid setting err as -EINVAL to remove extra one line of code

Changed since v3:
  * exit at the beginning if !nr_grefs
  * change the if statements to avoid test (err != 1) twice
  * initialize a 'blkif' stack variable (refer to PATCH 1/2)

Changed since v4:
  * use BUG_ON() when (nr_grefs == 0) to reminder the developer
  * set err = -EINVAL before xenbus_dev_fatal()

Changed since v5:
  * use WARN_ON() instead of BUG_ON()

 drivers/block/xen-blkback/xenbus.c | 72 +++---
 1 file changed, 43 insertions(+), 29 deletions(-)

diff --git a/drivers/block/xen-blkback/xenbus.c 
b/drivers/block/xen-blkback/xenbus.c
index a4aadac..0878e55 100644
--- a/drivers/block/xen-blkback/xenbus.c
+++ b/drivers/block/xen-blkback/xenbus.c
@@ -926,7 +926,7 @@ static int read_per_ring_refs(struct xen_blkif_ring *ring, 
const char *dir)
int err, i, j;
struct xen_blkif *blkif = ring->blkif;
struct xenbus_device *dev = blkif->be->dev;
-   unsigned int ring_page_order, nr_grefs, evtchn;
+   unsigned int nr_grefs, evtchn;
 
err = xenbus_scanf(XBT_NIL, dir, "event-channel", "%u",
  );
@@ -936,43 +936,42 @@ static int read_per_ring_refs(struct xen_blkif_ring 
*ring, const char *dir)
return err;
}
 
-   err = xenbus_scanf(XBT_NIL, dev->otherend, "ring-page-order", "%u",
- _page_order);
-   if (err != 1) {
-   err = xenbus_scanf(XBT_NIL, dir, "ring-ref", "%u", 
_ref[0]);
+   nr_grefs = blkif->nr_ring_pages;
+
+   if (unlikely(!nr_grefs)) {
+   WARN_ON(true);
+   return -EINVAL;
+   }
+
+   for (i = 0; i < nr_grefs; i++) {
+   char ring_ref_name[RINGREF_NAME_LEN];
+
+   snprintf(ring_ref_name, RINGREF_NAME_LEN, "ring-ref%u", i);
+   err = xenbus_scanf(XBT_NIL, dir, ring_ref_name,
+  "%u", _ref[i]);
+
if (err != 1) {
+   if (nr_grefs == 1)
+   break;
+
err = -EINVAL;
-   xenbus_dev_fatal(dev, err, "reading %s/ring-ref", dir);
+   xenbus_dev_fatal(dev, err, "reading %s/%s",
+dir, ring_ref_name);
return err;
}
-   nr_grefs = 1;
-   } else {
-   unsigned int i;
+   }
+
+   if (err != 1) {
+   WARN_ON(nr_grefs != 1);
 
-   if (ring_page_order > xen_blkif_max_ring_order) {
+   err = xenbus_scanf(XBT_NIL, dir, "ring-ref", "%u",
+  _ref[0]);
+   if (err != 1) {
err = -EINVAL;
-   xenbus_dev_fatal(dev, err, "%s/request %d ring page 
order exceed max:%d",
-dir, ring_page_order,
-xen_blkif_max_ring_order);
+   xenbus_dev_fatal(dev, err, "reading %s/ring-ref", dir);
return err;
}
-
-   nr_grefs = 1 << ring_page_order;
-   for (i = 0; i < nr_grefs; i++) {
-   char ring_ref_name[RINGREF_NAME_LEN];
-
-   snprintf(ring_ref_name, RINGREF_NAME_LEN, "ring-ref%u", 
i);
-   err = xenbus_scanf(XBT_NIL, dir, ring_ref_name,
-  "%u", _ref[i]);
-   if (err != 1) {
-   err = -EINVAL;
-   xenbus_dev_fatal(dev, err, "reading %s/%s",
-dir, ring_ref_name);
-   return err;
-   }
-   }
}
-   blkif->nr_ring_pages = nr_grefs;
 
for (i = 0; i < nr_grefs * XEN_BLKIF_REQS_PER_PAGE; i++) {
req = kzalloc(sizeof(*req), GFP_KERNEL);
@@ -1031,6 +1030,7 @@ static int connect_ring(struct backend_info *be)
size_t xspathsize;
   

Re: [Xen-devel] [PATCH v6 1/4] xen: introduce SYMBOL

2019-01-14 Thread Julien Grall

Hi Jan,

On 14/01/2019 15:52, Jan Beulich wrote:

On 14.01.19 at 16:41,  wrote:

Hi Jan,

On 14/01/2019 10:11, Jan Beulich wrote:

On 11.01.19 at 19:04,  wrote:

On Fri, 11 Jan 2019, Jan Beulich wrote:

On 11.01.19 at 03:14,  wrote:

Hi Juergen, Jan,

I spoke with Julien: we are both convinced that the unsigned long
solution is best. But Julien also did some research and he thinks that
Jan's version (returning pointer type) not only does not help with
MISRA-C, but also doesn't solve the potential GCC problem either. A
description of the GCC issue is available here:





https://kristerw.blogspot.com/2016/12/pointer-comparison-invalid-optimization
.h

tml?m=1


I've read through it, and besides not agreeing with some of the
author's arguments I wasn't able to spot where it tells me why/how
the suggested approach doesn't solve the problem.


(Also keep in mind that Linux uses the unsigned long solution to solve
the GCC issue, deviating from it doesn't seem wise.)


Which specific gcc issue (that is not solved by retaining type)?


I am hoping Julien and his team will be able to provide the more
decisive information next week for us to make a decision, but it looks
like the issue is not clear-cut and people on the GCC list disagree on
how it should be handled.


The C standard says that "Two pointers compare equal if and only if both
are null pointers, both are pointers to the same object (including a
pointer to an object and a subobject at its beginning) or function, both
are pointers to one past the last element of the same array object, or
one is a pointer to one past the end of one array object and the other
is a pointer to the start of a different array object that happens to
immediately follow the first array object in the address space."

In short, the compiler is free to return false in a pointer comparison
if it believes that the pointers point to different non-consecutive
object.


And it is this "it believes" which we undermine with the construct:
As long as the compiler can't prove two pointers point to different
objects, it can't eliminate the actual comparison.


May I ask where does this come from? A compiler could technically be free to
assume the inverse. I.e as long as it can't prove two pointers point to
different objects, it can rely on the undefined behavior to optimize it.


No. As long as there's a chance that both pointers point to the same
object, it can't do bad things, because _if_ they do, the result of the
comparison has to be correct (as per the text still quoted above).


In the following example (taken from [1]):

extern struct my_struct __start[];
extern struct my_struct __end[];

void foo(void)
{
for (struct my_struct *x = __start; x != __end; x++)
do_something(x);
}

The compiler can't be sure that __start and __end are not equal. Yet it may 
decide they are always different and optimize it to an infinite loop. So surely, 
the compiler can do bad things with even simple code.


I am struggling to understand how using "asm volatile" and still returning a 
pointer would help here. If the compiler managed to infer that __start and __end 
are always different, then there are no reason for this to not happen with the 
new construct.


I have been told that -fno-strict-aliasing may help us for pointer arithmetic. 
But I am still haven't find any evidence yet.


Cheers,

[1] 
https://kristerw.blogspot.com/2016/12/pointer-comparison-invalid-optimization.html?m=1



--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 07/15] x86/acpi: Add Hygon Dhyana support

2019-01-14 Thread Jan Beulich
>>> On 20.12.18 at 14:14,  wrote:
> Add Hygon Dhyana support to the acpi cpufreq subsystem by using the code
> path of AMD.

... cpufreq and cpuidle subsystems ...

> @@ -660,8 +661,9 @@ int cpufreq_cpu_init(unsigned int cpuid)
>  {
>  int ret;
>  
> -/* Currently we only handle Intel and AMD processor */
> +/* Currently we only handle Intel and AMD and Hygon processor */

Please replace the first "and" by a comma.

>  if ( (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL ) ||
> + (boot_cpu_data.x86_vendor == X86_VENDOR_HYGON ) ||
>   (boot_cpu_data.x86_vendor == X86_VENDOR_AMD ) )
>  ret = cpufreq_add_cpu(cpuid);
>  else

At the very least here I think you want to change to switch().

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v6 1/4] xen: introduce SYMBOL

2019-01-14 Thread Stewart Hildebrand
On Monday, January 14, 2019 10:53 AM, Jan Beulich wrote:
> > Hi Jan,
> >
> > On 14/01/2019 10:11, Jan Beulich wrote:
> > On 11.01.19 at 19:04,  wrote:
> >>> On Fri, 11 Jan 2019, Jan Beulich wrote:
> >>> On 11.01.19 at 03:14,  wrote:
> > Hi Juergen, Jan,
> >
> > I spoke with Julien: we are both convinced that the unsigned long
> > solution is best. But Julien also did some research and he thinks that
> > Jan's version (returning pointer type) not only does not help with
> > MISRA-C, but also doesn't solve the potential GCC problem either. A
> > description of the GCC issue is available here:
> >
> >
> >>>
> > https://kristerw.blogspot.com/2016/12/pointer-comparison-invalid-optimization
> > .h
> >>> tml?m=1
> 
>  I've read through it, and besides not agreeing with some of the
>  author's arguments I wasn't able to spot where it tells me why/how
>  the suggested approach doesn't solve the problem.
> 
> > (Also keep in mind that Linux uses the unsigned long solution to solve
> > the GCC issue, deviating from it doesn't seem wise.)
> 
>  Which specific gcc issue (that is not solved by retaining type)?
> >>>
> >>> I am hoping Julien and his team will be able to provide the more
> >>> decisive information next week for us to make a decision, but it looks
> >>> like the issue is not clear-cut and people on the GCC list disagree on
> >>> how it should be handled.
> >>>
> >>>
> >>> The C standard says that "Two pointers compare equal if and only if both
> >>> are null pointers, both are pointers to the same object (including a
> >>> pointer to an object and a subobject at its beginning) or function, both
> >>> are pointers to one past the last element of the same array object, or
> >>> one is a pointer to one past the end of one array object and the other
> >>> is a pointer to the start of a different array object that happens to
> >>> immediately follow the first array object in the address space."
> >>>
> >>> In short, the compiler is free to return false in a pointer comparison
> >>> if it believes that the pointers point to different non-consecutive
> >>> object.
> >>
> >> And it is this "it believes" which we undermine with the construct:
> >> As long as the compiler can't prove two pointers point to different
> >> objects, it can't eliminate the actual comparison.
> >
> > May I ask where does this come from? A compiler could technically be free to
> > assume the inverse. I.e as long as it can't prove two pointers point to
> > different objects, it can rely on the undefined behavior to optimize it.
> 
> No. As long as there's a chance that both pointers point to the same
> object, it can't do bad things, because _if_ they do, the result of the
> comparison has to be correct (as per the text still quoted above).
> 
> Jan

In the following declaration:
extern char _start[], _end[];
it's still a valid interpretation that _start and _end point to different 
objects. In fact, I think is already making this assumption, given that GCC 7.3 
will emit the following warning if "extern" is removed: "warning: array ‘_end’ 
assumed to have one element"

Who's to say that GCC someday won't become smart enough to sniff out all the 
inline assembly and pointer-type casts and still draw the conclusion that they 
point to separate objects? What if GCC someday will make these decisions based 
on some sort of link-time optimization, like using an intermediary or iterative 
symbol/linker map of sorts? It seems that using pointer types is at best a 
cat-and-mouse chase. GCC has too many unregulated moving pieces - it's a much 
stronger argument just to adhere to the standard. My vote still goes to 
uintptr_t.

Also, the above text only concerns equality comparisons. What about 
greater/less/neq comparisons, arithmetic, and dereferencing?

Stew
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 04/15] x86/cpu/mce: Add Hygon Dhyana support to the MCA infrastructure

2019-01-14 Thread Jan Beulich
>>> On 20.12.18 at 14:12,  wrote:
> --- a/xen/arch/x86/cpu/mcheck/mce_amd.c
> +++ b/xen/arch/x86/cpu/mcheck/mce_amd.c
> @@ -162,7 +162,8 @@ mcequirk_lookup_amd_quirkdata(struct cpuinfo_x86 *c)
>  {
>  int i;
>  
> -BUG_ON(c->x86_vendor != X86_VENDOR_AMD);
> +if (c->x86_vendor != X86_VENDOR_AMD)
> +return 0;

Please can you leave this untouched and change the single
caller instead? If need be down the road (but of course you'll
never introduce quirky behavior), we'd then add
mcequirk_lookup_hygon_quirkdata().

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] pvcalls-front: fix potential null dereference

2019-01-14 Thread Boris Ostrovsky
On 1/13/19 10:20 PM, Wen Yang wrote:
>  static checker warning:
> drivers/xen/pvcalls-front.c:373 alloc_active_ring()
> error: we previously assumed 'map->active.ring' could be null
>(see line 357)
>
> drivers/xen/pvcalls-front.c
> 351 static int alloc_active_ring(struct sock_mapping *map)
> 352 {
> 353 void *bytes;
> 354
> 355 map->active.ring = (struct pvcalls_data_intf *)
> 356 get_zeroed_page(GFP_KERNEL);
> 357 if (!map->active.ring)
> ^
> Check
>
> 358 goto out;
> 359
> 360 map->active.ring->ring_order = PVCALLS_RING_ORDER;
> 361 bytes = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
> 362 PVCALLS_RING_ORDER);
> 363 if (!bytes)
> 364 goto out;
> 365
> 366 map->active.data.in = bytes;
> 367 map->active.data.out = bytes +
> 368 XEN_FLEX_RING_SIZE(PVCALLS_RING_ORDER);
> 369
> 370 return 0;
> 371
> 372 out:
> --> 373 free_active_ring(map);
>  ^^^
> Replace map->active.ring->ring_order with PVCALLS_RING_ORDER
> to avoid potential null dereference.
>
> Fixes: 9f51c05dc41a ("pvcalls-front: Avoid get_free_pages(GFP_KERNEL)
> under spinlock")
> Reported-by: Dan Carpenter 
> Signed-off-by: Wen Yang 
> CC: Boris Ostrovsky 
> CC: Juergen Gross 
> CC: Stefano Stabellini 
> CC: Dan Carpenter 
> CC: xen-devel@lists.xenproject.org
> CC: linux-ker...@vger.kernel.org
> ---
>  drivers/xen/pvcalls-front.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
> index 307861f..e56f9a3 100644
> --- a/drivers/xen/pvcalls-front.c
> +++ b/drivers/xen/pvcalls-front.c
> @@ -344,7 +344,7 @@ int pvcalls_front_socket(struct socket *sock)
>  static void free_active_ring(struct sock_mapping *map)
>  {
>   free_pages((unsigned long)map->active.data.in,
> - map->active.ring->ring_order);
> + PVCALLS_RING_ORDER);

I would prefer to have a NULL check on map->active.ring and return
immediately if it is.

Otherwise this code assumes that map->active.ring->ring_order is always
PVCALLS_RING_ORDER, and I don't know whether we want to make this
assumption (even if this is true now).

-boris

>   free_page((unsigned long)map->active.ring);
>  }
>  


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] PING ARM [PATCH v2] xen/cmdline: Fix buggy strncmp(s, LITERAL, ss - s) construct

2019-01-14 Thread Julien Grall

Hi Andrew,

On 14/01/2019 15:17, Andrew Cooper wrote:

diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index ca655ff..22a86ec 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -212,7 +212,7 @@ EFI_STATUS __init fdt_add_uefi_nodes(EFI_SYSTEM_TABLE 
*sys_table,
  break;
  
  type = fdt_getprop(fdt, node, "device_type", );

-if ( type && strncmp(type, "memory", len) == 0 )
+if ( type && len == 6 && strncmp(type, "memory", 6) == 0 )


string property terminates with NUL and is included in the len. So I don't think 
this change is correct.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 03/15] x86/cpu/vpmu: Add Hygon Dhyana support for vPMU

2019-01-14 Thread Jan Beulich
>>> On 20.12.18 at 14:12,  wrote:
> --- a/xen/arch/x86/cpu/vpmu.c
> +++ b/xen/arch/x86/cpu/vpmu.c
> @@ -473,6 +473,7 @@ static int vpmu_arch_initialise(struct vcpu *v)
>  
>  switch ( vendor )
>  {
> +case X86_VENDOR_HYGON:
>  case X86_VENDOR_AMD:
>  ret = svm_vpmu_initialise(v);
>  break;
> @@ -890,6 +891,7 @@ static int __init vpmu_init(void)
>  
>  switch ( vendor )
>  {
> +case X86_VENDOR_HYGON:
>  case X86_VENDOR_AMD:
>  if ( amd_vpmu_init() )
> vpmu_mode = XENPMU_MODE_OFF;

Here and everywhere else, may I ask that you do your insertions
below the respective AMD ones instead of above?

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v6 1/4] xen: introduce SYMBOL

2019-01-14 Thread Jan Beulich
>>> On 14.01.19 at 16:41,  wrote:
> Hi Jan,
> 
> On 14/01/2019 10:11, Jan Beulich wrote:
> On 11.01.19 at 19:04,  wrote:
>>> On Fri, 11 Jan 2019, Jan Beulich wrote:
>>> On 11.01.19 at 03:14,  wrote:
> Hi Juergen, Jan,
>
> I spoke with Julien: we are both convinced that the unsigned long
> solution is best. But Julien also did some research and he thinks that
> Jan's version (returning pointer type) not only does not help with
> MISRA-C, but also doesn't solve the potential GCC problem either. A
> description of the GCC issue is available here:
>
>
>>> 
> https://kristerw.blogspot.com/2016/12/pointer-comparison-invalid-optimization 
> .h
>>> tml?m=1

 I've read through it, and besides not agreeing with some of the
 author's arguments I wasn't able to spot where it tells me why/how
 the suggested approach doesn't solve the problem.

> (Also keep in mind that Linux uses the unsigned long solution to solve
> the GCC issue, deviating from it doesn't seem wise.)

 Which specific gcc issue (that is not solved by retaining type)?
>>>
>>> I am hoping Julien and his team will be able to provide the more
>>> decisive information next week for us to make a decision, but it looks
>>> like the issue is not clear-cut and people on the GCC list disagree on
>>> how it should be handled.
>>>
>>>
>>> The C standard says that "Two pointers compare equal if and only if both
>>> are null pointers, both are pointers to the same object (including a
>>> pointer to an object and a subobject at its beginning) or function, both
>>> are pointers to one past the last element of the same array object, or
>>> one is a pointer to one past the end of one array object and the other
>>> is a pointer to the start of a different array object that happens to
>>> immediately follow the first array object in the address space."
>>>
>>> In short, the compiler is free to return false in a pointer comparison
>>> if it believes that the pointers point to different non-consecutive
>>> object.
>> 
>> And it is this "it believes" which we undermine with the construct:
>> As long as the compiler can't prove two pointers point to different
>> objects, it can't eliminate the actual comparison.
> 
> May I ask where does this come from? A compiler could technically be free to 
> assume the inverse. I.e as long as it can't prove two pointers point to 
> different objects, it can rely on the undefined behavior to optimize it.

No. As long as there's a chance that both pointers point to the same
object, it can't do bad things, because _if_ they do, the result of the
comparison has to be correct (as per the text still quoted above).

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] x86emul: fix test harness and fuzzer build dependencies

2019-01-14 Thread Jan Beulich
>>> On 14.01.19 at 16:09,  wrote:
> Jan Beulich writes ("Re: [Xen-devel] [PATCH] x86emul: fix test harness and 
> fuzzer build dependencies"):
>> On 21.12.18 at 15:16,  wrote:
>> > Why is this particular inter-directory dependency unusual ?  Do we
>> > plan to introduce similar MAKELEVEL-based invocation of dependency
>> > directory makefiles everyhere ?
>> 
>> If need be, anywhere where independent building of the sub-tree
>> is specifically meant to work as a standalone operation. That
>> invocation of "make -C /tools/include && ..." is in particular not
>> meant to be necessary for the test harness'es "run" target. It is
>> also not intended to be used for the fuzzer builds (as per
>> tools/fuzz/README.afl), albeit there one might as well call it a doc
>> omission.
> 
> This interface intent is inconsistent with the design principles for
> recursive make, and can not, in general, be realised.
> 
> It can in some special cases, including I think this one, be realised
> using MAKELEVEL, but only at the cost of significant extra complexity.
> I see you've sent another patch based on MAKELEVEL.  I don't have a
> strong enough objection to this change in this one place to block it.

But from the rest of your reply it sounds as if you don't want to
see it go in either. I'm feeling left in the dark as to how to make
progress here.

> However, I worry is that this is setting a precedent.
> What is wrong is the idea that in general it is OK in xen.git to
> start with a clean tree, or modify an existing build tree,, run
>make -C some/direct/ory
> and expect everything to be (re)built as needed.  The result of
> trying to implement that everywhere via MAKELEVEL would be awful.
> 
> Can you promise me not to send any more patches based on using
> MAKELEVEL this way ?

I don't intend to, but promise? No. For the fuzzer target I could
accept a doc change making the re-build of the include directory
a necessary prereq step. For the test harness'es "run" target,
though, I don't view this as a viable alternative. The name of the
goal is very clearly (to me at least) indicating that this is
(supposed to be) a standalone one. Any other harnesses
under tools/test/ may want to have something similar (some
already have a "run" target).

>  If not, where does it end ?  Noting that even
> your updated patch does not work for (eg):
>make -C tools/fuzz

Of course it doesn't; it's not meant to be. I don't see how this
relates here though - it wouldn't work without the change here
either. I'm not even convinced "make -C tools" works, as
opposed to "make tools", due to the very prereq step to (re-)
make tools/include/.

So how do we make progress here? For the two changes that
you dislike I don't formally need your ack, and I have Andrew's.
I would (have to) respect an active NAK of yours, of course.

For the one change that I need your (or Wei's) ack on, I didn't
see any strong objection so far, and this fixes an actual issue
with the overall tools build, i.e. _outside_ of the area you're
concerned about. The $(MAKE) invocation there is not overly
nice, but I thought I did convince you that - with the way
tools/include/ gets dealt with from the top level - this should
not be an issue. Plus I'm just moving it.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v6 1/4] xen: introduce SYMBOL

2019-01-14 Thread Julien Grall

Hi Jan,

On 14/01/2019 10:11, Jan Beulich wrote:

On 11.01.19 at 19:04,  wrote:

On Fri, 11 Jan 2019, Jan Beulich wrote:

On 11.01.19 at 03:14,  wrote:

Hi Juergen, Jan,

I spoke with Julien: we are both convinced that the unsigned long
solution is best. But Julien also did some research and he thinks that
Jan's version (returning pointer type) not only does not help with
MISRA-C, but also doesn't solve the potential GCC problem either. A
description of the GCC issue is available here:



https://kristerw.blogspot.com/2016/12/pointer-comparison-invalid-optimization.h
tml?m=1


I've read through it, and besides not agreeing with some of the
author's arguments I wasn't able to spot where it tells me why/how
the suggested approach doesn't solve the problem.


(Also keep in mind that Linux uses the unsigned long solution to solve
the GCC issue, deviating from it doesn't seem wise.)


Which specific gcc issue (that is not solved by retaining type)?


I am hoping Julien and his team will be able to provide the more
decisive information next week for us to make a decision, but it looks
like the issue is not clear-cut and people on the GCC list disagree on
how it should be handled.


The C standard says that "Two pointers compare equal if and only if both
are null pointers, both are pointers to the same object (including a
pointer to an object and a subobject at its beginning) or function, both
are pointers to one past the last element of the same array object, or
one is a pointer to one past the end of one array object and the other
is a pointer to the start of a different array object that happens to
immediately follow the first array object in the address space."

In short, the compiler is free to return false in a pointer comparison
if it believes that the pointers point to different non-consecutive
object.


And it is this "it believes" which we undermine with the construct:
As long as the compiler can't prove two pointers point to different
objects, it can't eliminate the actual comparison.


May I ask where does this come from? A compiler could technically be free to 
assume the inverse. I.e as long as it can't prove two pointers point to 
different objects, it can rely on the undefined behavior to optimize it.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 07/15] argo: implement the register op

2019-01-14 Thread Andrew Cooper
On 07/01/2019 07:42, Christopher Clark wrote:
> diff --git a/docs/misc/xen-command-line.pandoc 
> b/docs/misc/xen-command-line.pandoc
> index aea13eb..68d4415 100644
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -193,6 +193,21 @@ This allows domains access to the Argo hypercall, which 
> supports registration
>  of memory rings with the hypervisor to receive messages, sending messages to
>  other domains by hypercall and querying the ring status of other domains.
>  
> +### argo-mac
> +> `= permissive | enforcing`

Are these command line options already in use in the OpenXT community?

I ask, because we are trying to avoid gaining multiple top level options
for related functionatliy.

IMO, this functionality could be covered more succinctly with:

  argo = List of [ , mac = permissive | enforcing ]

which also allows for cleaner addition of future options.

(Unfortunately, to implement this, you need my cmdline_strcmp() fixes,
which are still pending an ack.)

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 04/15] argo: init, destroy and soft-reset, with enable command line opt

2019-01-14 Thread Lars Kurth
Adding Juergen,

to make sure he is aware of the discussion, as this is an excellent summary of 
the status of this series.

> On 14 Jan 2019, at 14:46, Wei Liu  wrote:
> 
> Hi all
> 
> The locking scheme seems to be remaining sticking point. The rest are
> mostly cosmetic issues (FAOD, they still need to be addressed).  Frankly
> I don't think there is enough time to address all the technical details,
> but let me sum up each side's position and see if we can reach an
> amicable solution.

snip

> To unblock this, how about we make Christopher maintainer of Argo? He
> and OpenXT will be on the hook for further improvement. And I believe it
> would be in their best interest to keep Argo bug-free and eventually
> make it become supported.
> 
> So:
> 
> 1. Make sure Argo is self-contained -- this requires careful review for
>   interaction between Argo and other parts of the hypervisor.
> 2. Argo is going to be experimental and off-by-default -- this is the
>   default status for new feature anyway.
> 3. Make Christopher maintainer of Argo -- this would be a natural thing
>   to do anyway.
> 
> Does this work for everyone?

I think this is a good way forward. Thank you Wei for putting this mail 
together.

Best Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Xen 4.12 Development Update

2019-01-14 Thread Lars Kurth
Hi Juergen,

> On 14 Jan 2019, at 10:13, Juergen Gross  wrote:
> 
> This email only tracks big items for xen.git tree. Please reply for items you
> would like to see in 4.12 so that people have an idea what is going on and
> prioritise accordingly.
> 
> You're welcome to provide description and use cases of the feature you're
> working on.
> 
> = Timeline =
> 
> We now adopt a fixed cut-off date scheme. We will release about every 8 
> months.
> The upcoming 4.12 timeline are as followed:
> 
> * Last posting date: December 14th, 2018
>  Last posting date for patches touching ARM code: December 1st, 2018
> * Hard code freeze: January 11th, 2019
>  Hard code freeze for patches touching ARM code: December 21st, 2018
> --> we are here
> * RC1: TBD
> * Release: March 7th, 2019
> 
> Note that we don't have freeze exception scheme anymore. All patches
> that wish to go into 4.12 must be posted initially no later than the
> last posting date and finally no later than the hard code freeze. All
> patches posted after that date will be automatically queued into next
> release.
> 
> RCs will be arranged immediately after freeze.

We should start planning on a Test Day schedule.

> We recently introduced a jira instance to track all the tasks (not only big)
> for the project. See: https://xenproject.atlassian.net/projects/XEN/issues.
> 
> Some of the tasks tracked by this e-mail also have a corresponding jira task
> referred by XEN-N.
> 
> I have started to include the version number of series associated to each
> feature. Can each owner send an update on the version number if the series
> was posted upstream?
> 
> = Projects =
> 
> == Hypervisor == 
> 
> *  Improvements to domain creation (v2)
>  -  Andrew Cooper
> 
> *  Argo (inter-VM communication) (v3)
>  -  Christopher Clark
> 
> *  Core aware scheduling (RFC v1)
>  -  Dario Faggioli
> 
> *  Core aware scheduling for credit2 (RFC v1)
>  -  Dario Faggioli
> 
> === x86 === 
> 
> *  hypervisor x86 instruction emulator additions for AVX512 (v7)
>  -  Jan Beulich
> 
> *  qemu deprivilege (v4)
>  -  George Dunlap
> 
> *  Fixes to #DB injection
>  -  Andrew Cooper
> 
> *  CPUID/MSR Xen/toolstack improvements
>  -  Andrew Cooper
> 
> *  Improvements to domain_crash()
>  -  Andrew Cooper
> 
> === ARM === 
> 
> == Completed == 
> 
> *  guest resource mapping
>  -  Paul Durrant
> 
> *  PV-only hypervisor
>  -  Wei Liu
> 
> *  HVM-only hypervisor
>  -  Wei Liu
> 
> *  Make credit2 scheduler the default
>  -  George Dunlap
> 
> *  Grub2: Support PVH guest boot
>  -  Juergen Gross
> 
> *  Fix VGA logdirty related display freezes with altp2m
>  -  Razvan Cojocaru
> 
> *  dom0less (boot multiple domains from device tree)
>  -  Stefano Stabellini
> 
> *  Implement Set/Way operations
>  -  Julien Grall

@Stefano:
Didn't the ARM KCONFIG stuff go in *after* 4.11? If so, this should probably be 
added. 
Can't recall the series name

Also, I think the Aggios changes went in after 4.11 was released also.
The series was "xen/arm64: Suspend preconditions and CPU hotplug fixes"

@ALL: also, for any major new features and/or enablers, we should look at the 
docs and make sure they are in place and up-to-date, that SUPPORT.md is updated 
and that any worthy/big enough features are listed. Also, if you contributed a 
larger series/feature and it is not on this list, please let us know.  

Thank you to everyone contributing to the project

Best Regards
Lars 
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] PING ARM [PATCH v2] xen/cmdline: Fix buggy strncmp(s, LITERAL, ss - s) construct

2019-01-14 Thread Andrew Cooper
There are ARM changes in here, and this bugfix is needed for 4.12, and
for backporting.

The one change not represented here is switching strncmp() with memcmp()
as recommended by Jan.

~Andrew

On 04/01/2019 17:17, Andrew Cooper wrote:
> When the command line parsing was updated to use const strings and no longer
> tokenise with NUL characters, string matches could no longer be made with
> strcmp().
>
> Unfortunately, the replacement was buggy.  strncmp(s, "opt", ss - s) matches
> "o", "op" and "opt" on the command line, as ss - s may be shorter than the
> passed literal.  Furthermore, parse_bool() is affected by this, so substrings
> such as "d", "e" and "o" are considered valid, with the latter being ambiguous
> between "on" and "off".
>
> Introduce a new strcmp-like function for the task, which looks for exact
> string matches, but declares success when the NUL of the literal matches a
> comma, colon or semicolon in the command line fragment.
>
> No change to the intended parsing functionality, but fixes cases where a
> partial string on the command line will inadvertently trigger options.
>
> A few areas were more than just a trivial change:
>
>  * fdt_add_uefi_nodes(), while not command line parsing, had the same broken
>strncmp() pattern.  As a fix, perform an explicit length check first.
>  * parse_irq_vector_map_param() gained some style corrections.
>  * parse_vpmu_params() was rewritten to use the normal list-of-options form,
>rather than just fixing up parse_vpmu_param() and leaving the parsing being
>hard to follow.
>  * Instead of making the trivial fix of adding an explicit length check in
>parse_bool(), use the length to select which token to we search for, which
>is more efficient than the previous linear search over all possible tokens.
>
> Signed-off-by: Andrew Cooper 
> ---
> CC: Jan Beulich 
> CC: Wei Liu 
> CC: Roger Pau Monné 
> CC: Stefano Stabellini 
> CC: Julien Grall 
>
> v2:
>  * Fix inverted check in parse_bool() for "no".  Drop extranious else's.
>  * Fix cmdline_strcmp() to not have implementation defined results.
> ---
>  xen/arch/arm/cpuerrata.c  |  6 +--
>  xen/arch/arm/efi/efi-boot.h   |  2 +-
>  xen/arch/x86/cpu/vpmu.c   | 49 --
>  xen/arch/x86/irq.c| 12 +++---
>  xen/arch/x86/psr.c|  4 +-
>  xen/arch/x86/spec_ctrl.c  |  6 +--
>  xen/arch/x86/x86_64/mmconfig-shared.c |  4 +-
>  xen/common/efi/boot.c |  4 +-
>  xen/common/kernel.c   | 79 
> ---
>  xen/drivers/cpufreq/cpufreq.c |  6 +--
>  xen/drivers/passthrough/iommu.c   | 28 ++---
>  xen/drivers/passthrough/pci.c |  4 +-
>  xen/include/xen/lib.h |  7 
>  13 files changed, 125 insertions(+), 86 deletions(-)
>
> diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c
> index adf88e7..f4815ca 100644
> --- a/xen/arch/arm/cpuerrata.c
> +++ b/xen/arch/arm/cpuerrata.c
> @@ -257,11 +257,11 @@ static int __init parse_spec_ctrl(const char *s)
>  {
>  s += 5;
>  
> -if ( !strncmp(s, "force-disable", ss - s) )
> +if ( !cmdline_strcmp(s, "force-disable") )
>  ssbd_state = ARM_SSBD_FORCE_DISABLE;
> -else if ( !strncmp(s, "runtime", ss - s) )
> +else if ( !cmdline_strcmp(s, "runtime") )
>  ssbd_state = ARM_SSBD_RUNTIME;
> -else if ( !strncmp(s, "force-enable", ss - s) )
> +else if ( !cmdline_strcmp(s, "force-enable") )
>  ssbd_state = ARM_SSBD_FORCE_ENABLE;
>  else
>  rc = -EINVAL;
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index ca655ff..22a86ec 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -212,7 +212,7 @@ EFI_STATUS __init fdt_add_uefi_nodes(EFI_SYSTEM_TABLE 
> *sys_table,
>  break;
>  
>  type = fdt_getprop(fdt, node, "device_type", );
> -if ( type && strncmp(type, "memory", len) == 0 )
> +if ( type && len == 6 && strncmp(type, "memory", 6) == 0 )
>  {
>  fdt_del_node(fdt, node);
>  continue;
> diff --git a/xen/arch/x86/cpu/vpmu.c b/xen/arch/x86/cpu/vpmu.c
> index 8a4f753..13da7d0 100644
> --- a/xen/arch/x86/cpu/vpmu.c
> +++ b/xen/arch/x86/cpu/vpmu.c
> @@ -61,42 +61,31 @@ static unsigned vpmu_count;
>  
>  static DEFINE_PER_CPU(struct vcpu *, last_vcpu);
>  
> -static int parse_vpmu_param(const char *s, unsigned int len)
> -{
> -if ( !*s || !len )
> -return 0;
> -if ( !strncmp(s, "bts", len) )
> -vpmu_features |= XENPMU_FEATURE_INTEL_BTS;
> -else if ( !strncmp(s, "ipc", len) )
> -vpmu_features |= XENPMU_FEATURE_IPC_ONLY;
> -else if ( !strncmp(s, "arch", len) )
> -vpmu_features |= XENPMU_FEATURE_ARCH_ONLY;
> -else
> -return 1;
> -  

Re: [Xen-devel] [PATCH v3 04/15] argo: init, destroy and soft-reset, with enable command line opt

2019-01-14 Thread Jan Beulich
>>> On 14.01.19 at 15:58,  wrote:
> On 07/01/2019 07:42, Christopher Clark wrote:
>> --- a/xen/common/argo.c
>> +++ b/xen/common/argo.c
>>  long
>>  do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1,
> 
> I know I'm commenting on the wrong patch, but please use unsigned long
> cmd, so the type definition here doesn't truncate the caller provided
> value.  We have similar buggy code all over Xen, but its too late to fix
> that, and I'd prefer not to propagate the error.

Why buggy? It all depends on how the interface is specified. If
the input is 32 bits wide, it is clear that higher bits are supposed
to be ignored. Nothing says that the full register width is
significant.

>> XEN_GUEST_HANDLE_PARAM(void) arg2, unsigned long arg3,
>> unsigned long arg4)
>>  {
>> -return -ENOSYS;
>> +struct domain *currd = current->domain;
>> +long rc = -EFAULT;
>> +
>> +argo_dprintk("->do_argo_op(%u,%p,%p,%d,%d)\n", cmd,
>> + (void *)arg1.p, (void *)arg2.p, (int) arg3, (int) arg4);
> 
> For debugging purposes, you don't want to truncate any of these values,
> or you'll have a print message which doesn't match what the guest
> provided.  I'd use %ld for arg3 and arg4.

Perhaps better %lx, for the output being easier to recognize
for both bitmaps (e.g. flag values) and sufficiently large values.

>> +
>> +if ( unlikely(!opt_argo_enabled) )
>> +{
>> +rc = -EOPNOTSUPP;
> 
> Shouldn't this be ENOSYS instead?  There isn't a conceptual difference
> between CONFIG_ARGO compiled out, and opt_argo clear on the command
> line, and I don't think a guest should be able to tell the difference.

I admit it's a boundary case, but I think ENOSYS should strictly
only ever be (and have been) returned for unrecognized major
hypercall numbers.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] x86emul: fix test harness and fuzzer build dependencies

2019-01-14 Thread Ian Jackson
Jan Beulich writes ("Re: [Xen-devel] [PATCH] x86emul: fix test harness and 
fuzzer build dependencies"):
> On 21.12.18 at 15:16,  wrote:
> > Why is this particular inter-directory dependency unusual ?  Do we
> > plan to introduce similar MAKELEVEL-based invocation of dependency
> > directory makefiles everyhere ?
> 
> If need be, anywhere where independent building of the sub-tree
> is specifically meant to work as a standalone operation. That
> invocation of "make -C /tools/include && ..." is in particular not
> meant to be necessary for the test harness'es "run" target. It is
> also not intended to be used for the fuzzer builds (as per
> tools/fuzz/README.afl), albeit there one might as well call it a doc
> omission.

This interface intent is inconsistent with the design principles for
recursive make, and can not, in general, be realised.

It can in some special cases, including I think this one, be realised
using MAKELEVEL, but only at the cost of significant extra complexity.
I see you've sent another patch based on MAKELEVEL.  I don't have a
strong enough objection to this change in this one place to block it.

However, I worry is that this is setting a precedent.
What is wrong is the idea that in general it is OK in xen.git to
start with a clean tree, or modify an existing build tree,, run
   make -C some/direct/ory
and expect everything to be (re)built as needed.  The result of
trying to implement that everywhere via MAKELEVEL would be awful.

Can you promise me not to send any more patches based on using
MAKELEVEL this way ?  If not, where does it end ?  Noting that even
your updated patch does not work for (eg):
   make -C tools/fuzz

Sorry,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 08/15] argo: implement the unregister op

2019-01-14 Thread Jan Beulich
>>> On 07.01.19 at 08:42,  wrote:
> @@ -666,6 +667,105 @@ ring_find_info(const struct domain *d, const struct 
> argo_ring_id *id)
>  return NULL;
>  }
>  
> +static struct argo_send_info *
> +send_find_info(const struct domain *d, const struct argo_ring_id *id)

As per the comment on patch 7, perhaps find_send_info()?

> +{
> +struct hlist_node *node;

const?

> +static long
> +unregister_ring(struct domain *currd,
> +XEN_GUEST_HANDLE_PARAM(xen_argo_unregister_ring_t) unreg_hnd)
> +{
> +xen_argo_unregister_ring_t unreg;
> +struct argo_ring_id ring_id;
> +struct argo_ring_info *ring_info;
> +struct argo_send_info *send_info;
> +struct domain *dst_d = NULL;
> +int ret;
> +
> +ret = copy_from_guest(, unreg_hnd, 1) ? -EFAULT : 0;
> +if ( ret )
> +goto out;
> +
> +ret = unreg.pad ? -EINVAL : 0;
> +if ( ret )
> +goto out;
> +
> +ring_id.partner_id = unreg.partner_id;
> +ring_id.port = unreg.port;
> +ring_id.domain_id = currd->domain_id;
> +
> +read_lock(_lock);
> +
> +if ( !currd->argo )
> +{
> +ret = -ENODEV;
> +goto out_unlock;
> +}
> +
> +write_lock(>argo->lock);
> +
> +ring_info = ring_find_info(currd, _id);
> +if ( ring_info )
> +{
> +ring_remove_info(currd, ring_info);
> +currd->argo->ring_count--;
> +}
> +
> +dst_d = get_domain_by_id(ring_id.partner_id);
> +if ( dst_d )
> +{
> +if ( dst_d->argo )
> +{
> +spin_lock(_d->argo->send_lock);
> +
> +send_info = send_find_info(dst_d, _id);
> +if ( send_info )
> +{
> +hlist_del(_info->node);
> +xfree(send_info);
> +}
> +
> +spin_unlock(_d->argo->send_lock);

As per the comment to an earlier patch, if at all possible call
allocation (and hence also freeing) functions with as little
locks held as possible. Pulling it out of the innermost lock
here looks straightforward at least.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 3/3] libxl: fix build (missing CLONE_NEWIPC) on astonishingly old systems

2019-01-14 Thread Wei Liu
On Mon, Jan 14, 2019 at 02:59:37PM +, Ian Jackson wrote:
> CLONE_NEWIPC was introduced in Linux 2.6.19, on the 29th of November
> 2006, which was 12 years, 1 month, and 14 days ago.
> 
> Nevertheless apparently some people are trying to build Xen on systems
> whose kernel headers are that old.  Placate these people by providing
> a fallback #define for CLONE_NEWIPC.
> 
> The actual binary value will of course remain constant, because of the
> kernel API promise, so this is and will be correct on all platforms
> where the CLONE_NEWIPC is supported.  (Even if for some reason we miss
> the right #includes.)
> 
> Of course at runtime this value will not work on older kernels.  It
> will be rejected as unknown by anything except some pre-2.6.18
> kernels.  On those kernels we do not want to support dm_restrict, and
> an attempt to use it will fail.  It is OK for the failure to be a
> messy EINVAL syscall failure.  (The IPC namespace unshare is necessary
> to avoid a suborned deprivileged qemu from causing trouble with shm,
> sem, etc.)
> 
> On the very old kernels, the feature is totally out of scope.
> (We are only interested, here, in making the build work, to avoid
> blocking people who aren't using this feature.)
> 
> CC: Wei Liu 
> CC: Juergen Gross 
> CC: Jan Beulich 
> Signed-off-by: Ian Jackson 
> 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 2/3] Revert "libxl: fix build on rather old systems"

2019-01-14 Thread Wei Liu
On Mon, Jan 14, 2019 at 02:59:36PM +, Ian Jackson wrote:
> This reverts commit 1bce5f9baf0f4a4e50722f32b44afe4fdefc6b35.
> 
> This situation should be handled by disabling the dm restrict
> feature, not silently falling back to lower protection.
> 
> Also this #ifdeffery is bad style.
> 
> Signed-off-by: Ian Jackson 


Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 1/3] docs/features/qemu-deprivilege.pandoc: No support with Linux <2.6.18

2019-01-14 Thread Wei Liu
On Mon, Jan 14, 2019 at 02:59:35PM +, Ian Jackson wrote:
> Some early kernesl are known not to reject unknown flags to
> unshare().  There may be other problems.
> 
> CC: Jan Beulich 
> Signed-off-by: Ian Jackson 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 2/3] Revert "libxl: fix build on rather old systems"

2019-01-14 Thread Ian Jackson
This reverts commit 1bce5f9baf0f4a4e50722f32b44afe4fdefc6b35.

This situation should be handled by disabling the dm restrict
feature, not silently falling back to lower protection.

Also this #ifdeffery is bad style.

Signed-off-by: Ian Jackson 
---
 tools/libxl/libxl_linux.c | 16 ++--
 1 file changed, 2 insertions(+), 14 deletions(-)

diff --git a/tools/libxl/libxl_linux.c b/tools/libxl/libxl_linux.c
index a4c2f28dbf..6475cca64b 100644
--- a/tools/libxl/libxl_linux.c
+++ b/tools/libxl/libxl_linux.c
@@ -334,24 +334,12 @@ int libxl__local_dm_preexec_restrict(libxl__gc *gc)
 unsigned i;
 
 /* Unshare mount and IPC namespaces.  These are unused by QEMU. */
-r = unshare(CLONE_NEWNS);
+r = unshare(CLONE_NEWNS | CLONE_NEWIPC);
 if (r) {
-LOGE(ERROR, "libxl: Mount namespace unshare failed");
+LOGE(ERROR, "libxl: Mount and IPC namespace unfailed");
 return ERROR_FAIL;
 }
 
-#ifndef CLONE_NEWIPC /* Available as of Linux 2.6.19 / glibc 2.8 */
-# define CLONE_NEWIPC 0x0800
-#endif
-r = unshare(CLONE_NEWIPC);
-if (r) {
-if (r && errno != EINVAL) {
-LOGE(ERROR, "libxl: IPC namespace unshare failed");
-return ERROR_FAIL;
-}
-LOG(WARN, "libxl: IPC namespace unshare unavailable");
-}
-
 /* Set various "easy" rlimits */
 for (i = 0; rlimits[i].resource != RLIMIT_NLIMITS; i++) {
 struct rlimit rlim;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 3/3] libxl: fix build (missing CLONE_NEWIPC) on astonishingly old systems

2019-01-14 Thread Ian Jackson
CLONE_NEWIPC was introduced in Linux 2.6.19, on the 29th of November
2006, which was 12 years, 1 month, and 14 days ago.

Nevertheless apparently some people are trying to build Xen on systems
whose kernel headers are that old.  Placate these people by providing
a fallback #define for CLONE_NEWIPC.

The actual binary value will of course remain constant, because of the
kernel API promise, so this is and will be correct on all platforms
where the CLONE_NEWIPC is supported.  (Even if for some reason we miss
the right #includes.)

Of course at runtime this value will not work on older kernels.  It
will be rejected as unknown by anything except some pre-2.6.18
kernels.  On those kernels we do not want to support dm_restrict, and
an attempt to use it will fail.  It is OK for the failure to be a
messy EINVAL syscall failure.  (The IPC namespace unshare is necessary
to avoid a suborned deprivileged qemu from causing trouble with shm,
sem, etc.)

On the very old kernels, the feature is totally out of scope.
(We are only interested, here, in making the build work, to avoid
blocking people who aren't using this feature.)

CC: Wei Liu 
CC: Juergen Gross 
CC: Jan Beulich 
Signed-off-by: Ian Jackson 

---
v3: Adjust commit message with facts about some even older kernels
v2: Get rid of spurious X
---
 tools/libxl/libxl_linux.c | 8 
 1 file changed, 8 insertions(+)

diff --git a/tools/libxl/libxl_linux.c b/tools/libxl/libxl_linux.c
index 6475cca64b..59dd945bc1 100644
--- a/tools/libxl/libxl_linux.c
+++ b/tools/libxl/libxl_linux.c
@@ -18,6 +18,14 @@
 #include 
 #include "libxl_internal.h"
 
+
+/* Workarounds for Linux-specific lacks can go here: */
+
+#ifndef CLONE_NEWIPC /* Available as of Linux 2.6.19 / glibc 2.8 */
+# define CLONE_NEWIPC 0x0800
+#endif
+
+
 int libxl__try_phy_backend(mode_t st_mode)
 {
 if (S_ISBLK(st_mode) || S_ISREG(st_mode)) {
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 1/3] docs/features/qemu-deprivilege.pandoc: No support with Linux <2.6.18

2019-01-14 Thread Ian Jackson
Some early kernesl are known not to reject unknown flags to
unshare().  There may be other problems.

CC: Jan Beulich 
Signed-off-by: Ian Jackson 
---
v3: New in this version of the series.
---
 docs/features/qemu-deprivilege.pandoc | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/docs/features/qemu-deprivilege.pandoc 
b/docs/features/qemu-deprivilege.pandoc
index eb05981a83..20d6ac2189 100644
--- a/docs/features/qemu-deprivilege.pandoc
+++ b/docs/features/qemu-deprivilege.pandoc
@@ -112,6 +112,9 @@ The following features still need to be implemented:
  * Inserting a new cdrom while the guest is running (xl cdrom-insert)
  * Migration / save / restore
 
+dm_restrict is totally unsupported and may have unexpected security
+problems if used with a dom0 Linux kernel earlier than 2.6.18.
+
 Additionally, getting PCI passthrough to work securely would require a
 significant rework of how passthrough works at the moment.  It may be
 implemented at some point but is not a near-term priority.
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 04/15] argo: init, destroy and soft-reset, with enable command line opt

2019-01-14 Thread Andrew Cooper
On 07/01/2019 07:42, Christopher Clark wrote:
> Initialises basic data structures and performs teardown of argo state
> for domain shutdown.
>
> Inclusion of the Argo implementation is dependent on CONFIG_ARGO.
>
> Introduces a new Xen command line parameter 'argo': bool to enable/disable
> the argo hypercall. Defaults to disabled.
>
> New headers:
>   public/argo.h: with definions of addresses and ring structure, including
>   indexes for atomic update for communication between domain and hypervisor.
>
>   xen/argo.h: to expose the hooks for integration into domain lifecycle:
> argo_init: per-domain init of argo data structures for domain_create.
> argo_destroy: teardown for domain_destroy and the error exit
>   path of domain_create.
> argo_soft_reset: reset of domain state for domain_soft_reset.
>
> Adds two new fields to struct domain:
> rwlock_t argo_lock;
> struct argo_domain *argo;
>
> In accordance with recent work on _domain_destroy, argo_destroy is
> idempotent. It will tear down: all rings registered by this domain, all
> rings where this domain is the single sender (ie. specified partner,
> non-wildcard rings), and all pending notifications where this domain is
> awaiting signal about available space in the rings of other domains.
>
> A count will be maintained of the number of rings that a domain has
> registered in order to limit it below the fixed maximum limit defined here.
>
> The software license on the public header is the BSD license, standard
> procedure for the public Xen headers. The public header was originally
> posted under a GPL license at: [1]:
> https://lists.xenproject.org/archives/html/xen-devel/2013-05/msg02710.html
>
> The following ACK by Lars Kurth is to confirm that only people being
> employees of Citrix contributed to the header files in the series posted at
> [1] and that thus the copyright of the files in question is fully owned by
> Citrix. The ACK also confirms that Citrix is happy for the header files to
> be published under a BSD license in this series (which is based on [1]).
>
> Signed-off-by: Christopher Clark 
> Acked-by: Lars Kurth 

I hope I've not trodden on the toes of any other reviews.  I've got some
minor requests, but hopefully its all fairly trivial.

> diff --git a/docs/misc/xen-command-line.pandoc 
> b/docs/misc/xen-command-line.pandoc
> index a755a67..aea13eb 100644
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -182,6 +182,17 @@ Permit Xen to use "Always Running APIC Timer" support on 
> compatible hardware
>  in combination with cpuidle.  This option is only expected to be useful for
>  developers wishing Xen to fall back to older timing methods on newer 
> hardware.
>  
> +### argo
> +> `= `
> +
> +> Default: `false`
> +
> +Enable the Argo hypervisor-mediated interdomain communication mechanism.
> +
> +This allows domains access to the Argo hypercall, which supports registration
> +of memory rings with the hypervisor to receive messages, sending messages to
> +other domains by hypercall and querying the ring status of other domains.

Please do include a note about CONFIG_ARGO.  I know this doc is
inconsistent on the matter (as Kconfig postdates the written entries
here), but I have been trying to fix up, and now about half of the
documentation does mention appropriate Kconfig information.

> diff --git a/xen/common/argo.c b/xen/common/argo.c
> index 6f782f7..86195d3 100644
> --- a/xen/common/argo.c
> +++ b/xen/common/argo.c
>  long
>  do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1,

I know I'm commenting on the wrong patch, but please use unsigned long
cmd, so the type definition here doesn't truncate the caller provided
value.  We have similar buggy code all over Xen, but its too late to fix
that, and I'd prefer not to propagate the error.

> XEN_GUEST_HANDLE_PARAM(void) arg2, unsigned long arg3,
> unsigned long arg4)
>  {
> -return -ENOSYS;
> +struct domain *currd = current->domain;
> +long rc = -EFAULT;
> +
> +argo_dprintk("->do_argo_op(%u,%p,%p,%d,%d)\n", cmd,
> + (void *)arg1.p, (void *)arg2.p, (int) arg3, (int) arg4);

For debugging purposes, you don't want to truncate any of these values,
or you'll have a print message which doesn't match what the guest
provided.  I'd use %ld for arg3 and arg4.

> +
> +if ( unlikely(!opt_argo_enabled) )
> +{
> +rc = -EOPNOTSUPP;

Shouldn't this be ENOSYS instead?  There isn't a conceptual difference
between CONFIG_ARGO compiled out, and opt_argo clear on the command
line, and I don't think a guest should be able to tell the difference.

> +return rc;
> +}
> +
> +domain_lock(currd);
> +
> +switch (cmd)
> +{
> +default:
> +rc = -EOPNOTSUPP;
> +break;
> +}
> +
> +domain_unlock(currd);
> +
> +argo_dprintk("<-do_argo_op(%u)=%ld\n", cmd, rc);
> +
> +return rc;
> +}
> +
> +static void
> 

[Xen-devel] [linux-4.9 test] 131946: trouble: broken/fail/pass

2019-01-14 Thread osstest service owner
flight 131946 linux-4.9 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/131946/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-xsm   broken
 test-amd64-amd64-xl-qemut-debianhvm-amd64   broken
 test-amd64-i386-xl   broken
 test-amd64-amd64-xl-shadow   broken
 test-amd64-amd64-xl-qemuu-ovmf-amd64 broken
 test-amd64-i386-freebsd10-amd64 broken
 test-amd64-i386-xl-qemut-debianhvm-amd64broken
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsmbroken
 test-amd64-amd64-xl-credit2  broken
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsmbroken
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 4 host-install(4) broken REGR. 
vs. 131645
 test-amd64-amd64-xl-qemut-debianhvm-amd64 4 host-install(4) broken REGR. vs. 
131645
 test-amd64-i386-freebsd10-amd64  4 host-install(4) broken REGR. vs. 131645
 test-amd64-amd64-xl-qemuu-ovmf-amd64 4 host-install(4) broken REGR. vs. 131645
 test-amd64-i386-examine   5 host-install   broken REGR. vs. 131645
 test-amd64-amd64-xl-credit2   4 host-install(4)broken REGR. vs. 131645
 test-amd64-i386-xl-xsm4 host-install(4)broken REGR. vs. 131645
 test-amd64-i386-xl4 host-install(4)broken REGR. vs. 131645
 test-amd64-i386-xl-qemut-debianhvm-amd64 4 host-install(4) broken REGR. vs. 
131645
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 4 host-install(4) broken REGR. 
vs. 131645
 test-amd64-amd64-xl-shadow4 host-install(4)broken REGR. vs. 131645

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-rumprun-amd64 17 rumprun-demo-xenstorels/xenstorels.repeat 
fail REGR. vs. 131645

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 131645
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 131645
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 131645
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 131645
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 131645
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-amd64-xl-pvhv2-intel 12 guest-start fail never pass
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-amd64-xl-pvhv2-amd 12 guest-start  fail  never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 

Re: [Xen-devel] [PATCH v3 04/15] argo: init, destroy and soft-reset, with enable command line opt

2019-01-14 Thread Wei Liu
Hi all

The locking scheme seems to be remaining sticking point. The rest are
mostly cosmetic issues (FAOD, they still need to be addressed).  Frankly
I don't think there is enough time to address all the technical details,
but let me sum up each side's position and see if we can reach an
amicable solution.

From maintainers and reviewers' point of view:

1. Maintainers / reviewers don't like complexity unless absolutely
   necessary.
2. Maintainers / reviewers feel they have a responsibility to understand
   the code and algorithm.

Yet being the gatekeepers doesn't necessarily mean we understand every
technical details and every usecase. We would like to, but most of the
time it is unrealistic.

Down to this specific patch series:

Roger thinks the locking scheme is too complex. Christopher argues
that's necessary for short-live channels to be performant.

Both have their point.

I think having a complex locking scheme is inevitable, just like we did
for performant grant table several years ago.  Regardless of the timing
issue we have at hand, asking Christopher to implement a stripped down
version creates more work for him.

Yet ignoring Roger's concerns is unfair to him as well, since he put in
so much time and effort to understand the algorithm and provide
suggestions. It is in fact unreasonable to ask anyone to fully
understand the locking mechanism and check the implementation is correct
in a few days (given the series was posted in Dec and there were major
holidays in between, plus everyone had other commitments).

To unblock this, how about we make Christopher maintainer of Argo? He
and OpenXT will be on the hook for further improvement. And I believe it
would be in their best interest to keep Argo bug-free and eventually
make it become supported.

So:

1. Make sure Argo is self-contained -- this requires careful review for
   interaction between Argo and other parts of the hypervisor.
2. Argo is going to be experimental and off-by-default -- this is the
   default status for new feature anyway.
3. Make Christopher maintainer of Argo -- this would be a natural thing
   to do anyway.

Does this work for everyone?

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH V3] x86/vm_event: block interrupt injection for sync vm_events

2019-01-14 Thread Razvan Cojocaru

On 1/14/19 4:42 PM, Juergen Gross wrote:

On 14/01/2019 11:56, Razvan Cojocaru wrote:

On 1/14/19 11:53 AM, Jan Beulich wrote:

On 14.01.19 at 10:34,  wrote:

On 1/12/19 12:04 AM, Boris Ostrovsky wrote:

On 12/14/18 6:49 AM, Razvan Cojocaru wrote:

Block interrupts (in vmx_intr_assist()) for the duration of
processing a sync vm_event (similarly to the strategy
currently used for single-stepping). Otherwise, attempting
to emulate an instruction when requested by a vm_event
reply may legitimately need to call e.g.
hvm_inject_page_fault(), which then overwrites the active
interrupt in the VMCS.

The sync vm_event handling path on x86/VMX is (roughly):
monitor_traps() -> process vm_event -> vmx_intr_assist()
(possibly writing VM_ENTRY_INTR_INFO) ->
hvm_vm_event_do_resume() -> hvm_emulate_one_vm_event()
(possibly overwriting the VM_ENTRY_INTR_INFO value).

This patch may also be helpful for the future removal
of may_defer in hvm_set_cr{0,3,4} and hvm_set_msr().

Signed-off-by: Razvan Cojocaru 



Reviewed-by: Boris Ostrovsky 


Thanks! So now we have three reviewed-bys, if I'm not mistaken all we
need is Tamas' (for the vm_event part) and Julien / Stefano's (for ARM)
acks (or otherwise).


And you'd need to talk Jürgen into allowing this in, now that we're
past the freeze point.


(Adding Jürgen to the conversation.)

Right, that would be ideal if possible - the code has absolutely no
impact on anything unless vm_event is active on the domain, which is
never the case for the use-cases considered for a Xen release.

So the case I'm making for the patch to go in 4.12 is that:

1. It's perfectly harmless (it affects nothing, except for introspection).

2. It's trivial and thus very easy to see that it's correct.

3. It fixes a major headache for us, and thus it is a great improvement
from an introspection standpoint (fixes OS crashes / hangs which we'd
otherwise need to work around in rather painful ways).

4. V3 of the patch has been sent out on Dec 14th - it's just that
reviewers have had other priorities and it did not gather all acks in time.

However, if it's not possible or desirable to allow this in the next
best thing is to at least have all the acks necessary for it to go in
first thing once the freeze is over.

 From Julien's reply I understand that the last ack necessary is Tamas'.


With that ack just arrived:

Release-acked-by: Juergen Gross 


Thanks! Very much appreciated.


Razvan

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH V3] x86/vm_event: block interrupt injection for sync vm_events

2019-01-14 Thread Juergen Gross
On 14/01/2019 11:56, Razvan Cojocaru wrote:
> On 1/14/19 11:53 AM, Jan Beulich wrote:
> On 14.01.19 at 10:34,  wrote:
>>> On 1/12/19 12:04 AM, Boris Ostrovsky wrote:
 On 12/14/18 6:49 AM, Razvan Cojocaru wrote:
> Block interrupts (in vmx_intr_assist()) for the duration of
> processing a sync vm_event (similarly to the strategy
> currently used for single-stepping). Otherwise, attempting
> to emulate an instruction when requested by a vm_event
> reply may legitimately need to call e.g.
> hvm_inject_page_fault(), which then overwrites the active
> interrupt in the VMCS.
>
> The sync vm_event handling path on x86/VMX is (roughly):
> monitor_traps() -> process vm_event -> vmx_intr_assist()
> (possibly writing VM_ENTRY_INTR_INFO) ->
> hvm_vm_event_do_resume() -> hvm_emulate_one_vm_event()
> (possibly overwriting the VM_ENTRY_INTR_INFO value).
>
> This patch may also be helpful for the future removal
> of may_defer in hvm_set_cr{0,3,4} and hvm_set_msr().
>
> Signed-off-by: Razvan Cojocaru 


 Reviewed-by: Boris Ostrovsky 
>>>
>>> Thanks! So now we have three reviewed-bys, if I'm not mistaken all we
>>> need is Tamas' (for the vm_event part) and Julien / Stefano's (for ARM)
>>> acks (or otherwise).
>>
>> And you'd need to talk Jürgen into allowing this in, now that we're
>> past the freeze point.
> 
> (Adding Jürgen to the conversation.)
> 
> Right, that would be ideal if possible - the code has absolutely no
> impact on anything unless vm_event is active on the domain, which is
> never the case for the use-cases considered for a Xen release.
> 
> So the case I'm making for the patch to go in 4.12 is that:
> 
> 1. It's perfectly harmless (it affects nothing, except for introspection).
> 
> 2. It's trivial and thus very easy to see that it's correct.
> 
> 3. It fixes a major headache for us, and thus it is a great improvement
> from an introspection standpoint (fixes OS crashes / hangs which we'd
> otherwise need to work around in rather painful ways).
> 
> 4. V3 of the patch has been sent out on Dec 14th - it's just that
> reviewers have had other priorities and it did not gather all acks in time.
> 
> However, if it's not possible or desirable to allow this in the next
> best thing is to at least have all the acks necessary for it to go in
> first thing once the freeze is over.
> 
> From Julien's reply I understand that the last ack necessary is Tamas'.

With that ack just arrived:

Release-acked-by: Juergen Gross 


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH V3] x86/vm_event: block interrupt injection for sync vm_events

2019-01-14 Thread Tamas K Lengyel
On Fri, Dec 14, 2018 at 4:50 AM Razvan Cojocaru
 wrote:
>
> Block interrupts (in vmx_intr_assist()) for the duration of
> processing a sync vm_event (similarly to the strategy
> currently used for single-stepping). Otherwise, attempting
> to emulate an instruction when requested by a vm_event
> reply may legitimately need to call e.g.
> hvm_inject_page_fault(), which then overwrites the active
> interrupt in the VMCS.
>
> The sync vm_event handling path on x86/VMX is (roughly):
> monitor_traps() -> process vm_event -> vmx_intr_assist()
> (possibly writing VM_ENTRY_INTR_INFO) ->
> hvm_vm_event_do_resume() -> hvm_emulate_one_vm_event()
> (possibly overwriting the VM_ENTRY_INTR_INFO value).
>
> This patch may also be helpful for the future removal
> of may_defer in hvm_set_cr{0,3,4} and hvm_set_msr().
>
> Signed-off-by: Razvan Cojocaru 

Acked-by: Tamas K Lengyel 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 2/2] libxl: fix build (missing CLONE_NEWIPC) on astonishingly old systems

2019-01-14 Thread Jan Beulich
>>> On 14.01.19 at 15:22,  wrote:
> Wei Liu writes ("Re: [PATCH v2 2/2] libxl: fix build (missing CLONE_NEWIPC) 
> on astonishingly old systems"):
>> On Mon, Jan 14, 2019 at 02:47:58AM -0700, Jan Beulich wrote:
>> > On 11.01.19 at 20:23,  wrote:
>> > > CLONE_NEWIPC was introduced in Linux 2.6.19, on the 29th of November
>> > > 2006, which was 12 years, 1 month, and 14 days ago.
>> > 
>> > Thanks for the very precise counting, the latter part which will be
>> > wrong - even if just slightly - by the time you commit it ;-)
> ...
>> > Sadly the situation is more complicated: The check to disallow
>> > unknown flags was introduced only in 2.6.17 [1], and apparently
>> > never backported to 2.6.16 or older stable trees despite the
>> > description talking about it going into 2.6.16. Since it didn't
>> > matter in my variant of the workaround, I didn't mention this.
> 
> Good grief.
> 
>> > Of course a pretty reasonable position to take would be to
>> > consider the 2.6.18-based XenoLinux tree a "baseline", beyond
>> > which we don't care about undesirable behavior here.
>> 
>> I think using 2.6.18 as baseline is very reasonable. 
> 
> I guess we need to write this in the SUPPORT.md statement for
> dm_restrict.

Ah yes, we should.

> TBH how about writing somewhere general in SUPPORT.md that "all bets
> are off if you use Linux before 2.6.18" ?  Do we even have a limit
> anywhere for security supported Linux versions ?

I don't think so, and leaving this specific case aside it's also
unclear to me why we should.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 04/15] argo: init, destroy and soft-reset, with enable command line opt

2019-01-14 Thread Rich Persaud

> On Jan 14, 2019, at 06:32, Roger Pau Monné  wrote:
> 
> On Mon, Jan 14, 2019 at 9:32 AM Christopher Clark
>  wrote:
>> 
>> I've written a document about the locking to add to the tree with the
>> series, and a copy is at github here:
>> 
>> https://github.com/dozylynx/xen/blob/0cb95385eba696ecf4856075a524c5e528e60455/docs/misc/argo-locking.md
> 
> Thanks. It would have been better to send the contents of the document
> to the list, so inline comments can be added. It's hard to comment on
> the document now since it's only on github AFAICT.

Here's an inline copy of the doc:

=== begin document ===

# Argo: Locking

## Introduction

Argo is an interdomain communication mechanism. It has requirements for 
performance isolation between domains, to prevent negative performance impact 
from malicious or disruptive activity of other domains, or even other vcpus of 
the same domain operating other rings.

Since Argo operates a data path between domains, sections of this code are 
*hot* when the communication paths are in use. To encourage high performance, a 
goal is to limit mutual exclusion to only where required and enable significant 
concurrency.

Avoidance of deadlock is essential and since state must frequently be updated 
that pertains to more than one domain, a locking protocol defines which locks 
are needed and the order of their acquistion.

## Structure

The granular locking structure of Argo enables:

1. Performance isolation of guests
2. Avoidance of DoS of rings by domains that are not authorized to send to them
3. Deadlock-free teardown of state across multiple domains on domain destroy
4. Performance of guests using Argo with concurrent operation of rings.

Argo uses three per-domain locks to protect three separate data structures.  
Access to the ring_hash data structure is confined to domains that a 
ring-registering domain has authorized to send data via the ring.  The complete 
set of Argo locks is:

* Global : `L1_global_argo_rwlock`
* Per-domain: `rings_L2_rwlock`
* Per-domain: `send_L2_lock`
* Per-domain: `wildcard_L2_lock`
* Per-ring: `L3_lock`

## Protected State

The data structures being protected by the locks are all per-domain. The only 
global Argo state is the `L1_global_argo_rwlock` used to coordinate access to 
data structures of other domains.

### State: Rings registered and owned by a domain

This includes the state to run that ring, such as memory frame numbers and 
established mappings. Per-ring state is protected by its own lock, so that 
multiple VCPUs of the same domain operating different rings do not inhibit the 
performance of each other.

The per-domain ring state also includes the list of pending notifications for 
other domains that are waiting for ring space availability.

### State: Partner rings registered by other domains that this domain is the 
single allowed sender

This state belonging to the permitted sender is written to when a ring is 
registered by another domain. The lock that protects this state is subject to 
locking at arbitrary frequency by those foreign domains when registering rings 
-- which do not need any permission granted by this domain in order to register 
a ring to communicate with it --  so it must not inhibit the domain's own 
ability to use its own rings, to protect them from DoS. For this reason, this 
state is protected by its own lock.

### State: Pending notifications to this domain about wildcard rings registered 
by other domains

This data structure is needed when a domain is destroyed, to cancel the 
outstanding space availability notifications about the wildcard rings of other 
domains that this domain has queried.

Data is entered into this data structure by the domain that owns it, either by 
a space-inhibited sendv or a notify operation.

Data is removed from this data structure in one of three cases: when space 
becomes available in the destination ring and the notification is sent, when 
the ring is torn down, or when the awaiting domain is destroyed.

In the case where a notification is sent, access to the data structure is 
triggered by the ring owner domain, rather than the domain waiting for 
notification. This data structure is protected by its own lock since doing so 
entails less contention than the alternative of reusing an existing lock owned 
by the domain.

## Hierarchical Locking Model and Protocol

The locking discipline within the Argo code is heirarchical and utilizes 
reader/writer locks to enable increased concurrency when operations do not 
conflict. None of the Argo locks are reentrant.

The hierarchy:

* There is a global rwlock (`L1`) to protect access to all of the per-domain 
argo data structures. 
* There is a rwlock per-domain (`rings_L2`) to protect the hashtable of the 
per-ring data structures. 
* There is a lock per ring (`L3`) to protect the per-ring data structure, 
`struct argo_ring_info`. 

There are a two other per-domain L2 locks; their operation is similar and they 
are described 

Re: [Xen-devel] [PATCH v2 2/2] libxl: fix build (missing CLONE_NEWIPC) on astonishingly old systems

2019-01-14 Thread Ian Jackson
Wei Liu writes ("Re: [PATCH v2 2/2] libxl: fix build (missing CLONE_NEWIPC) on 
astonishingly old systems"):
> On Mon, Jan 14, 2019 at 02:47:58AM -0700, Jan Beulich wrote:
> > On 11.01.19 at 20:23,  wrote:
> > > CLONE_NEWIPC was introduced in Linux 2.6.19, on the 29th of November
> > > 2006, which was 12 years, 1 month, and 14 days ago.
> > 
> > Thanks for the very precise counting, the latter part which will be
> > wrong - even if just slightly - by the time you commit it ;-)
...
> > Sadly the situation is more complicated: The check to disallow
> > unknown flags was introduced only in 2.6.17 [1], and apparently
> > never backported to 2.6.16 or older stable trees despite the
> > description talking about it going into 2.6.16. Since it didn't
> > matter in my variant of the workaround, I didn't mention this.

Good grief.

> > Of course a pretty reasonable position to take would be to
> > consider the 2.6.18-based XenoLinux tree a "baseline", beyond
> > which we don't care about undesirable behavior here.
> 
> I think using 2.6.18 as baseline is very reasonable. 

I guess we need to write this in the SUPPORT.md statement for
dm_restrict.

TBH how about writing somewhere general in SUPPORT.md that "all bets
are off if you use Linux before 2.6.18" ?  Do we even have a limit
anywhere for security supported Linux versions ?

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 07/15] argo: implement the register op

2019-01-14 Thread Jan Beulich
>>> On 07.01.19 at 08:42,  wrote:
> --- a/xen/common/argo.c
> +++ b/xen/common/argo.c
> @@ -23,16 +23,41 @@
>  #include 
>  #include 
>  #include 
> +#include 
> +#include 
>  #include 
>  #include 
>  
> +#define MAX_RINGS_PER_DOMAIN128U
> +
> +/* All messages on the ring are padded to a multiple of the slot size. */
> +#define ROUNDUP_MESSAGE(a) (ROUNDUP((a), XEN_ARGO_MSG_SLOT_SIZE))

Pointless outermost pair of parentheses.

> @@ -198,6 +223,31 @@ static DEFINE_RWLOCK(argo_lock); /* L1 */
>  #define argo_dprintk(format, ... ) ((void)0)
>  #endif
>  
> +/*
> + * This hash function is used to distribute rings within the per-domain
> + * hash tables (d->argo->ring_hash and d->argo_send_hash). The hash table
> + * will provide a struct if a match is found with a 'argo_ring_id' key:
> + * ie. the key is a (domain id, port, partner domain id) tuple.
> + * Since port number varies the most in expected use, and the Linux driver
> + * allocates at both the high and low ends, incorporate high and low bits to
> + * help with distribution.
> + * Apply array_index_nospec as a defensive measure since this operates
> + * on user-supplied input and the array size that it indexes into is known.
> + */
> +static unsigned int
> +hash_index(const struct argo_ring_id *id)
> +{
> +unsigned int hash;
> +
> +hash = (uint16_t)(id->port >> 16);
> +hash ^= (uint16_t)id->port;

I may have asked this before, but are the casts really needed
with ...

> +hash ^= id->domain_id;
> +hash ^= id->partner_id;
> +hash &= (ARGO_HTABLE_SIZE - 1);

... the masking done here?

> +return array_index_nospec(hash, ARGO_HTABLE_SIZE);

With the masking above - is this really needed?

And then the question is whether the quality of the hash is
sufficient: There won't be more set bits in the result than
are in any of the three input values, so if they're all small,
higher hash table entries won't be used at all. I would
assume the goal to be that by the time 32 entities appear,
chances be good that at least about 30 of the hash table
entries are in use.

> @@ -219,6 +269,78 @@ ring_unmap(struct argo_ring_info *ring_info)
>  }
>  }
>  
> +static int
> +ring_map_page(struct argo_ring_info *ring_info, unsigned int i, void 
> **out_ptr)
> +{
> +if ( i >= ring_info->nmfns )
> +{
> +gprintk(XENLOG_ERR,
> +   "argo: ring (vm%u:%x vm%d) %p attempted to map page  %u of 
> %u\n",

ring_info->id.{domain,partner}_id look to be of the same type -
why once %u and once %d? Same elsewhere.

> +ring_info->id.domain_id, ring_info->id.port,
> +ring_info->id.partner_id, ring_info, i, ring_info->nmfns);
> +return -ENOMEM;
> +}

i = array_index_nospec(i, ring_info->nmfns);

considering the array indexes here? Of course at this point only
zero can be passed in, but I assume this changes in later patches
and the index is at least indirectly guest controlled.

> @@ -371,6 +493,418 @@ partner_rings_remove(struct domain *src_d)
>  }
>  }
>  
> +static int
> +find_ring_mfn(struct domain *d, gfn_t gfn, mfn_t *mfn)

So you have find_ring_mfn(), find_ring_mfns(), and ring_find_info().
Any chance you could use a consistent ordering of "ring" and "find"?
Or is there a reason behind the apparent mismatch?

> +{
> +p2m_type_t p2mt;
> +int ret = 0;
> +
> +#ifdef CONFIG_X86
> +*mfn = get_gfn_unshare(d, gfn_x(gfn), );
> +#else
> +*mfn = p2m_lookup(d, gfn, );
> +#endif
> +
> +if ( !mfn_valid(*mfn) )
> +ret = -EINVAL;
> +#ifdef CONFIG_X86
> +else if ( p2m_is_paging(p2mt) || (p2mt == p2m_ram_logdirty) )
> +ret = -EAGAIN;
> +#endif
> +else if ( (p2mt != p2m_ram_rw) ||
> +  !get_page_and_type(mfn_to_page(*mfn), d, PGT_writable_page) )
> +ret = -EINVAL;
> +
> +#ifdef CONFIG_X86
> +put_gfn(d, gfn_x(gfn));
> +#endif
> +
> +return ret;
> +}

Please check whether you could leverage check_get_page_from_gfn()
here. If you can't, please at least take inspiration as to e.g. the
#ifdef-s from that function.

> +static int
> +find_ring_mfns(struct domain *d, struct argo_ring_info *ring_info,
> +   uint32_t npage,
> +   XEN_GUEST_HANDLE_PARAM(xen_argo_page_descr_t) pg_descr_hnd,
> +   uint32_t len)

Noticing it here, but perhaps still an issue elsewhere as well: Didn't
we agree on removing unnecessary use of fixed width types? Or
was that in the context on an earlier patch of v3?

> +{
> +unsigned int i;
> +int ret = 0;
> +mfn_t *mfns;
> +uint8_t **mfn_mapping;
> +
> +/*
> + * first bounds check on npage here also serves as an overflow check
> + * before left shifting it
> + */
> +if ( (unlikely(npage > (XEN_ARGO_MAX_RING_SIZE >> PAGE_SHIFT))) ||

Isn't this redundant with the check in do_argo_p()?

> + ((npage << PAGE_SHIFT) < len) )
> +return -EINVAL;
> +
> +if ( ring_info->mfns )
> +{
> +/* Ring already 

[Xen-devel] [PULL 14/25] xen: remove 'ioreq' struct/varable/field names from dataplane/xen-block.c

2019-01-14 Thread Anthony PERARD
From: Paul Durrant 

This is a purely cosmetic patch that purges the name 'ioreq' from struct,
variable and field names. (This name has been problematic for a long time
as 'ioreq' is the name used for generic I/O requests coming from Xen).
The patch replaces 'struct ioreq' with a new 'XenBlockRequest' type and
'ioreq' field/variable names with 'request', and then does necessary
fix-up to adhere to coding style.

Function names are not modified by this patch. They will be dealt with in
a subsequent patch.

No functional change.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
Signed-off-by: Anthony PERARD 
---
 hw/block/dataplane/xen-block.c | 310 +
 1 file changed, 156 insertions(+), 154 deletions(-)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 1ff464973c..6788bbf338 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -28,7 +28,7 @@
 #include "sysemu/iothread.h"
 #include "xen-block.h"
 
-struct ioreq {
+typedef struct XenBlockRequest {
 blkif_request_t req;
 int16_t status;
 off_t start;
@@ -39,9 +39,9 @@ struct ioreq {
 int aio_inflight;
 int aio_errors;
 XenBlockDataPlane *dataplane;
-QLIST_ENTRY(ioreq) list;
+QLIST_ENTRY(XenBlockRequest) list;
 BlockAcctCookie acct;
-};
+} XenBlockRequest;
 
 struct XenBlockDataPlane {
 XenDevice *xendev;
@@ -54,9 +54,9 @@ struct XenBlockDataPlane {
 int protocol;
 blkif_back_rings_t rings;
 int more_work;
-QLIST_HEAD(inflight_head, ioreq) inflight;
-QLIST_HEAD(finished_head, ioreq) finished;
-QLIST_HEAD(freelist_head, ioreq) freelist;
+QLIST_HEAD(inflight_head, XenBlockRequest) inflight;
+QLIST_HEAD(finished_head, XenBlockRequest) finished;
+QLIST_HEAD(freelist_head, XenBlockRequest) freelist;
 int requests_total;
 int requests_inflight;
 int requests_finished;
@@ -67,68 +67,68 @@ struct XenBlockDataPlane {
 AioContext *ctx;
 };
 
-static void ioreq_reset(struct ioreq *ioreq)
+static void ioreq_reset(XenBlockRequest *request)
 {
-memset(>req, 0, sizeof(ioreq->req));
-ioreq->status = 0;
-ioreq->start = 0;
-ioreq->buf = NULL;
-ioreq->size = 0;
-ioreq->presync = 0;
+memset(>req, 0, sizeof(request->req));
+request->status = 0;
+request->start = 0;
+request->buf = NULL;
+request->size = 0;
+request->presync = 0;
 
-ioreq->aio_inflight = 0;
-ioreq->aio_errors = 0;
+request->aio_inflight = 0;
+request->aio_errors = 0;
 
-ioreq->dataplane = NULL;
-memset(>list, 0, sizeof(ioreq->list));
-memset(>acct, 0, sizeof(ioreq->acct));
+request->dataplane = NULL;
+memset(>list, 0, sizeof(request->list));
+memset(>acct, 0, sizeof(request->acct));
 
-qemu_iovec_reset(>v);
+qemu_iovec_reset(>v);
 }
 
-static struct ioreq *ioreq_start(XenBlockDataPlane *dataplane)
+static XenBlockRequest *ioreq_start(XenBlockDataPlane *dataplane)
 {
-struct ioreq *ioreq = NULL;
+XenBlockRequest *request = NULL;
 
 if (QLIST_EMPTY(>freelist)) {
 if (dataplane->requests_total >= dataplane->max_requests) {
 goto out;
 }
 /* allocate new struct */
-ioreq = g_malloc0(sizeof(*ioreq));
-ioreq->dataplane = dataplane;
+request = g_malloc0(sizeof(*request));
+request->dataplane = dataplane;
 dataplane->requests_total++;
-qemu_iovec_init(>v, 1);
+qemu_iovec_init(>v, 1);
 } else {
 /* get one from freelist */
-ioreq = QLIST_FIRST(>freelist);
-QLIST_REMOVE(ioreq, list);
+request = QLIST_FIRST(>freelist);
+QLIST_REMOVE(request, list);
 }
-QLIST_INSERT_HEAD(>inflight, ioreq, list);
+QLIST_INSERT_HEAD(>inflight, request, list);
 dataplane->requests_inflight++;
 
 out:
-return ioreq;
+return request;
 }
 
-static void ioreq_finish(struct ioreq *ioreq)
+static void ioreq_finish(XenBlockRequest *request)
 {
-XenBlockDataPlane *dataplane = ioreq->dataplane;
+XenBlockDataPlane *dataplane = request->dataplane;
 
-QLIST_REMOVE(ioreq, list);
-QLIST_INSERT_HEAD(>finished, ioreq, list);
+QLIST_REMOVE(request, list);
+QLIST_INSERT_HEAD(>finished, request, list);
 dataplane->requests_inflight--;
 dataplane->requests_finished++;
 }
 
-static void ioreq_release(struct ioreq *ioreq, bool finish)
+static void ioreq_release(XenBlockRequest *request, bool finish)
 {
-XenBlockDataPlane *dataplane = ioreq->dataplane;
+XenBlockDataPlane *dataplane = request->dataplane;
 
-QLIST_REMOVE(ioreq, list);
-ioreq_reset(ioreq);
-ioreq->dataplane = dataplane;
-QLIST_INSERT_HEAD(>freelist, ioreq, list);
+QLIST_REMOVE(request, list);
+ioreq_reset(request);
+request->dataplane = dataplane;
+QLIST_INSERT_HEAD(>freelist, request, list);
 if (finish) {
 dataplane->requests_finished--;
 } else {
@@ -140,18 

[Xen-devel] [PULL 25/25] xen-block: avoid repeated memory allocation

2019-01-14 Thread Anthony PERARD
From: Tim Smith 

The xen-block dataplane currently allocates memory to hold the data for
each request as that request is used, and frees it afterwards. Because
it requires page-aligned blocks, this interacts poorly with non-page-
aligned allocations and balloons the heap.

Instead, allocate the maximum possible buffer size required for the
protocol, which is BLKIF_MAX_SEGMENTS_PER_REQUEST (currently 11) pages
when the request structure is created, and keep that buffer until it is
destroyed. Since the requests are re-used via a free list, this should
actually improve memory usage.

Signed-off-by: Tim Smith 

Re-based and commit comment adjusted.

Signed-off-by: Paul Durrant 
Acked-by: Anthony PERARD 
Signed-off-by: Anthony PERARD 
---
 hw/block/dataplane/xen-block.c | 14 +-
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 35bfccfba7..d0d8905a33 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -70,7 +70,6 @@ static void reset_request(XenBlockRequest *request)
 memset(>req, 0, sizeof(request->req));
 request->status = 0;
 request->start = 0;
-request->buf = NULL;
 request->size = 0;
 request->presync = 0;
 
@@ -95,6 +94,14 @@ static XenBlockRequest 
*xen_block_start_request(XenBlockDataPlane *dataplane)
 /* allocate new struct */
 request = g_malloc0(sizeof(*request));
 request->dataplane = dataplane;
+/*
+ * We cannot need more pages per requests than this, and since we
+ * re-use requests, allocate the memory once here. It will be freed
+ * xen_block_dataplane_destroy() when the request list is freed.
+ */
+request->buf = qemu_memalign(XC_PAGE_SIZE,
+ BLKIF_MAX_SEGMENTS_PER_REQUEST *
+ XC_PAGE_SIZE);
 dataplane->requests_total++;
 qemu_iovec_init(>v, 1);
 } else {
@@ -272,14 +279,12 @@ static void xen_block_complete_aio(void *opaque, int ret)
 if (ret == 0) {
 xen_block_copy_request(request);
 }
-qemu_vfree(request->buf);
 break;
 case BLKIF_OP_WRITE:
 case BLKIF_OP_FLUSH_DISKCACHE:
 if (!request->req.nr_segments) {
 break;
 }
-qemu_vfree(request->buf);
 break;
 default:
 break;
@@ -360,12 +365,10 @@ static int xen_block_do_aio(XenBlockRequest *request)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 
-request->buf = qemu_memalign(XC_PAGE_SIZE, request->size);
 if (request->req.nr_segments &&
 (request->req.operation == BLKIF_OP_WRITE ||
  request->req.operation == BLKIF_OP_FLUSH_DISKCACHE) &&
 xen_block_copy_request(request)) {
-qemu_vfree(request->buf);
 goto err;
 }
 
@@ -665,6 +668,7 @@ void xen_block_dataplane_destroy(XenBlockDataPlane 
*dataplane)
 request = QLIST_FIRST(>freelist);
 QLIST_REMOVE(request, list);
 qemu_iovec_destroy(>v);
+qemu_vfree(request->buf);
 g_free(request);
 }
 
-- 
Anthony PERARD


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PULL 15/25] xen: purge 'blk' and 'ioreq' from function names in dataplane/xen-block.c

2019-01-14 Thread Anthony PERARD
From: Paul Durrant 

This is a purely cosmetic patch that purges remaining use of 'blk' and
'ioreq' in local function names, and then makes sure all functions are
prefixed with 'xen_block_'.

No functional change.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
Signed-off-by: Anthony PERARD 
---
 hw/block/dataplane/xen-block.c | 90 +-
 1 file changed, 46 insertions(+), 44 deletions(-)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 6788bbf338..8e3965e171 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -67,7 +67,7 @@ struct XenBlockDataPlane {
 AioContext *ctx;
 };
 
-static void ioreq_reset(XenBlockRequest *request)
+static void reset_request(XenBlockRequest *request)
 {
 memset(>req, 0, sizeof(request->req));
 request->status = 0;
@@ -86,7 +86,7 @@ static void ioreq_reset(XenBlockRequest *request)
 qemu_iovec_reset(>v);
 }
 
-static XenBlockRequest *ioreq_start(XenBlockDataPlane *dataplane)
+static XenBlockRequest *xen_block_start_request(XenBlockDataPlane *dataplane)
 {
 XenBlockRequest *request = NULL;
 
@@ -111,7 +111,7 @@ static XenBlockRequest *ioreq_start(XenBlockDataPlane 
*dataplane)
 return request;
 }
 
-static void ioreq_finish(XenBlockRequest *request)
+static void xen_block_finish_request(XenBlockRequest *request)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 
@@ -121,12 +121,12 @@ static void ioreq_finish(XenBlockRequest *request)
 dataplane->requests_finished++;
 }
 
-static void ioreq_release(XenBlockRequest *request, bool finish)
+static void xen_block_release_request(XenBlockRequest *request, bool finish)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 
 QLIST_REMOVE(request, list);
-ioreq_reset(request);
+reset_request(request);
 request->dataplane = dataplane;
 QLIST_INSERT_HEAD(>freelist, request, list);
 if (finish) {
@@ -140,7 +140,7 @@ static void ioreq_release(XenBlockRequest *request, bool 
finish)
  * translate request into iovec + start offset
  * do sanity checks along the way
  */
-static int ioreq_parse(XenBlockRequest *request)
+static int xen_block_parse_request(XenBlockRequest *request)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 size_t len;
@@ -201,7 +201,7 @@ static int ioreq_parse(XenBlockRequest *request)
 return -1;
 }
 
-static int ioreq_grant_copy(XenBlockRequest *request)
+static int xen_block_copy_request(XenBlockRequest *request)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 XenDevice *xendev = dataplane->xendev;
@@ -247,9 +247,9 @@ static int ioreq_grant_copy(XenBlockRequest *request)
 return 0;
 }
 
-static int ioreq_runio_qemu_aio(XenBlockRequest *request);
+static int xen_block_do_aio(XenBlockRequest *request);
 
-static void qemu_aio_complete(void *opaque, int ret)
+static void xen_block_complete_aio(void *opaque, int ret)
 {
 XenBlockRequest *request = opaque;
 XenBlockDataPlane *dataplane = request->dataplane;
@@ -266,7 +266,7 @@ static void qemu_aio_complete(void *opaque, int ret)
 request->aio_inflight--;
 if (request->presync) {
 request->presync = 0;
-ioreq_runio_qemu_aio(request);
+xen_block_do_aio(request);
 goto done;
 }
 if (request->aio_inflight > 0) {
@@ -277,7 +277,7 @@ static void qemu_aio_complete(void *opaque, int ret)
 case BLKIF_OP_READ:
 /* in case of failure request->aio_errors is increased */
 if (ret == 0) {
-ioreq_grant_copy(request);
+xen_block_copy_request(request);
 }
 qemu_vfree(request->buf);
 break;
@@ -293,7 +293,7 @@ static void qemu_aio_complete(void *opaque, int ret)
 }
 
 request->status = request->aio_errors ? BLKIF_RSP_ERROR : BLKIF_RSP_OKAY;
-ioreq_finish(request);
+xen_block_finish_request(request);
 
 switch (request->req.operation) {
 case BLKIF_OP_WRITE:
@@ -318,9 +318,9 @@ static void qemu_aio_complete(void *opaque, int ret)
 aio_context_release(dataplane->ctx);
 }
 
-static bool blk_split_discard(XenBlockRequest *request,
-  blkif_sector_t sector_number,
-  uint64_t nr_sectors)
+static bool xen_block_split_discard(XenBlockRequest *request,
+blkif_sector_t sector_number,
+uint64_t nr_sectors)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 int64_t byte_offset;
@@ -343,7 +343,7 @@ static bool blk_split_discard(XenBlockRequest *request,
 byte_chunk = byte_remaining > limit ? limit : byte_remaining;
 request->aio_inflight++;
 blk_aio_pdiscard(dataplane->blk, byte_offset, byte_chunk,
- qemu_aio_complete, request);
+ xen_block_complete_aio, request);
 byte_remaining -= byte_chunk;
 byte_offset += byte_chunk;

[Xen-devel] [PULL 21/25] Remove broken Xen PV domain builder

2019-01-14 Thread Anthony PERARD
It is broken since Xen 4.9 [1] and it will not build in Xen 4.12. Also,
it is not built by default since QEMU 2.6.

[1] https://lists.xenproject.org/archives/html/xen-devel/2018-09/msg00313.html

Signed-off-by: Anthony PERARD 
Acked-by: Stefano Stabellini 
---
 configure   |  17 --
 hw/xenpv/Makefile.objs  |   2 -
 hw/xenpv/xen_domainbuild.c  | 299 
 hw/xenpv/xen_domainbuild.h  |  13 --
 hw/xenpv/xen_machine_pv.c   |  14 --
 include/hw/xen/xen.h|   1 -
 include/hw/xen/xen_common.h |  18 ---
 qemu-options.hx |   8 -
 vl.c|   7 -
 9 files changed, 379 deletions(-)
 delete mode 100644 hw/xenpv/xen_domainbuild.c
 delete mode 100644 hw/xenpv/xen_domainbuild.h

diff --git a/configure b/configure
index 4ea3f14883..f992709b89 100755
--- a/configure
+++ b/configure
@@ -357,7 +357,6 @@ vnc_png=""
 xkbcommon=""
 xen=""
 xen_ctrl_version=""
-xen_pv_domain_build="no"
 xen_pci_passthrough=""
 linux_aio=""
 cap_ng=""
@@ -1119,10 +1118,6 @@ for opt do
   ;;
   --enable-xen-pci-passthrough) xen_pci_passthrough="yes"
   ;;
-  --disable-xen-pv-domain-build) xen_pv_domain_build="no"
-  ;;
-  --enable-xen-pv-domain-build) xen_pv_domain_build="yes"
-  ;;
   --disable-brlapi) brlapi="no"
   ;;
   --enable-brlapi) brlapi="yes"
@@ -1685,8 +1680,6 @@ Advanced options (experts only):
   --tls-priority   default TLS protocol/cipher priority string
   --enable-gprof   QEMU profiling with gprof
   --enable-profilerprofiler support
-  --enable-xen-pv-domain-build
-   xen pv domain builder
   --enable-debug-stack-usage
track the maximum stack usage of stacks created by 
qemu_alloc_stack
 
@@ -2678,12 +2671,6 @@ if test "$xen_pci_passthrough" != "no"; then
   fi
 fi
 
-if test "$xen_pv_domain_build" = "yes" &&
-   test "$xen" != "yes"; then
-error_exit "User requested Xen PV domain builder support" \
-  "which requires Xen support."
-fi
-
 ##
 # Windows Hypervisor Platform accelerator (WHPX) check
 if test "$whpx" != "no" ; then
@@ -6069,7 +6056,6 @@ fi
 echo "xen support   $xen"
 if test "$xen" = "yes" ; then
   echo "xen ctrl version  $xen_ctrl_version"
-  echo "pv dom build  $xen_pv_domain_build"
 fi
 echo "brlapi support$brlapi"
 echo "bluez  support$bluez"
@@ -6539,9 +6525,6 @@ fi
 if test "$xen" = "yes" ; then
   echo "CONFIG_XEN_BACKEND=y" >> $config_host_mak
   echo "CONFIG_XEN_CTRL_INTERFACE_VERSION=$xen_ctrl_version" >> 
$config_host_mak
-  if test "$xen_pv_domain_build" = "yes" ; then
-echo "CONFIG_XEN_PV_DOMAIN_BUILD=y" >> $config_host_mak
-  fi
 fi
 if test "$linux_aio" = "yes" ; then
   echo "CONFIG_LINUX_AIO=y" >> $config_host_mak
diff --git a/hw/xenpv/Makefile.objs b/hw/xenpv/Makefile.objs
index bbf5873fd1..8bfa4586ab 100644
--- a/hw/xenpv/Makefile.objs
+++ b/hw/xenpv/Makefile.objs
@@ -1,4 +1,2 @@
 # Xen PV machine support
 obj-$(CONFIG_XEN) += xen_machine_pv.o
-# Xen PV machine builder support
-obj-$(CONFIG_XEN_PV_DOMAIN_BUILD) += xen_domainbuild.o
diff --git a/hw/xenpv/xen_domainbuild.c b/hw/xenpv/xen_domainbuild.c
deleted file mode 100644
index 1d9926e8d6..00
--- a/hw/xenpv/xen_domainbuild.c
+++ /dev/null
@@ -1,299 +0,0 @@
-#include "qemu/osdep.h"
-#include "qemu/units.h"
-#include "hw/xen/xen-legacy-backend.h"
-#include "xen_domainbuild.h"
-#include "qemu/timer.h"
-#include "qemu/log.h"
-
-#include 
-
-static int xenstore_domain_mkdir(char *path)
-{
-struct xs_permissions perms_ro[] = {{
-.id= 0, /* set owner: dom0 */
-},{
-.id= xen_domid,
-.perms = XS_PERM_READ,
-}};
-struct xs_permissions perms_rw[] = {{
-.id= 0, /* set owner: dom0 */
-},{
-.id= xen_domid,
-.perms = XS_PERM_READ | XS_PERM_WRITE,
-}};
-const char *writable[] = { "device", "control", "error", NULL };
-char subpath[256];
-int i;
-
-if (!xs_mkdir(xenstore, 0, path)) {
-fprintf(stderr, "%s: xs_mkdir %s: failed\n", __func__, path);
-return -1;
-}
-if (!xs_set_permissions(xenstore, 0, path, perms_ro, 2)) {
-fprintf(stderr, "%s: xs_set_permissions failed\n", __func__);
-return -1;
-}
-
-for (i = 0; writable[i]; i++) {
-snprintf(subpath, sizeof(subpath), "%s/%s", path, writable[i]);
-if (!xs_mkdir(xenstore, 0, subpath)) {
-fprintf(stderr, "%s: xs_mkdir %s: failed\n", __func__, subpath);
-return -1;
-}
-if (!xs_set_permissions(xenstore, 0, subpath, perms_rw, 2)) {
-fprintf(stderr, "%s: xs_set_permissions failed\n", __func__);
-return -1;
-}
-}
-return 0;
-}
-
-int xenstore_domain_init1(const char *kernel, const char *ramdisk,
-  const char *cmdline)
-{
-char *dom, uuid_string[42], vm[256], 

[Xen-devel] [PULL 24/25] xen-block: improve response latency

2019-01-14 Thread Anthony PERARD
From: Tim Smith 

If the I/O ring is full, the guest cannot send any more requests
until some responses are sent. Only sending all available responses
just before checking for new work does not leave much time for the
guest to supply new work, so this will cause stalls if the ring gets
full. Also, not completing reads as soon as possible adds latency
to the guest.

To alleviate that, complete IO requests as soon as they come back.
xen_block_send_response() already returns a value indicating whether
a notify should be sent, which is all the batching we need.

Signed-off-by: Tim Smith 

Re-based and commit comment adjusted.

Signed-off-by: Paul Durrant 
Acked-by: Anthony PERARD 
Signed-off-by: Anthony PERARD 
---
 hw/block/dataplane/xen-block.c | 56 +++---
 1 file changed, 18 insertions(+), 38 deletions(-)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index acd23a74a8..35bfccfba7 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -55,11 +55,9 @@ struct XenBlockDataPlane {
 blkif_back_rings_t rings;
 int more_work;
 QLIST_HEAD(inflight_head, XenBlockRequest) inflight;
-QLIST_HEAD(finished_head, XenBlockRequest) finished;
 QLIST_HEAD(freelist_head, XenBlockRequest) freelist;
 int requests_total;
 int requests_inflight;
-int requests_finished;
 unsigned int max_requests;
 BlockBackend *blk;
 QEMUBH *bh;
@@ -116,12 +114,10 @@ static void xen_block_finish_request(XenBlockRequest 
*request)
 XenBlockDataPlane *dataplane = request->dataplane;
 
 QLIST_REMOVE(request, list);
-QLIST_INSERT_HEAD(>finished, request, list);
 dataplane->requests_inflight--;
-dataplane->requests_finished++;
 }
 
-static void xen_block_release_request(XenBlockRequest *request, bool finish)
+static void xen_block_release_request(XenBlockRequest *request)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 
@@ -129,11 +125,7 @@ static void xen_block_release_request(XenBlockRequest 
*request, bool finish)
 reset_request(request);
 request->dataplane = dataplane;
 QLIST_INSERT_HEAD(>freelist, request, list);
-if (finish) {
-dataplane->requests_finished--;
-} else {
-dataplane->requests_inflight--;
-}
+dataplane->requests_inflight--;
 }
 
 /*
@@ -248,6 +240,7 @@ static int xen_block_copy_request(XenBlockRequest *request)
 }
 
 static int xen_block_do_aio(XenBlockRequest *request);
+static int xen_block_send_response(XenBlockRequest *request);
 
 static void xen_block_complete_aio(void *opaque, int ret)
 {
@@ -312,6 +305,18 @@ static void xen_block_complete_aio(void *opaque, int ret)
 default:
 break;
 }
+if (xen_block_send_response(request)) {
+Error *local_err = NULL;
+
+xen_device_notify_event_channel(dataplane->xendev,
+dataplane->event_channel,
+_err);
+if (local_err) {
+error_report_err(local_err);
+}
+}
+xen_block_release_request(request);
+
 qemu_bh_schedule(dataplane->bh);
 
 done:
@@ -419,7 +424,7 @@ static int xen_block_do_aio(XenBlockRequest *request)
 return -1;
 }
 
-static int xen_block_send_response_one(XenBlockRequest *request)
+static int xen_block_send_response(XenBlockRequest *request)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 int send_notify = 0;
@@ -474,29 +479,6 @@ static int xen_block_send_response_one(XenBlockRequest 
*request)
 return send_notify;
 }
 
-/* walk finished list, send outstanding responses, free requests */
-static void xen_block_send_response_all(XenBlockDataPlane *dataplane)
-{
-XenBlockRequest *request;
-int send_notify = 0;
-
-while (!QLIST_EMPTY(>finished)) {
-request = QLIST_FIRST(>finished);
-send_notify += xen_block_send_response_one(request);
-xen_block_release_request(request, true);
-}
-if (send_notify) {
-Error *local_err = NULL;
-
-xen_device_notify_event_channel(dataplane->xendev,
-dataplane->event_channel,
-_err);
-if (local_err) {
-error_report_err(local_err);
-}
-}
-}
-
 static int xen_block_get_request(XenBlockDataPlane *dataplane,
  XenBlockRequest *request, RING_IDX rc)
 {
@@ -547,7 +529,6 @@ static void xen_block_handle_requests(XenBlockDataPlane 
*dataplane)
 rp = dataplane->rings.common.sring->req_prod;
 xen_rmb(); /* Ensure we see queued requests up to 'rp'. */
 
-xen_block_send_response_all(dataplane);
 /*
  * If there was more than IO_PLUG_THRESHOLD requests in flight
  * when we got here, this is an indication that there the bottleneck
@@ -591,7 +572,7 @@ static void xen_block_handle_requests(XenBlockDataPlane 
*dataplane)
 break;
 };
 
-

[Xen-devel] [PULL 20/25] xen: remove the legacy 'xen_disk' backend

2019-01-14 Thread Anthony PERARD
From: Paul Durrant 

This backend has now been replaced by the 'xen-qdisk' XenDevice.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
Signed-off-by: Anthony PERARD 
---
 hw/block/Makefile.objs |1 -
 hw/block/xen_disk.c| 1011 
 2 files changed, 1012 deletions(-)
 delete mode 100644 hw/block/xen_disk.c

diff --git a/hw/block/Makefile.objs b/hw/block/Makefile.objs
index f34813a377..e206b8e712 100644
--- a/hw/block/Makefile.objs
+++ b/hw/block/Makefile.objs
@@ -5,7 +5,6 @@ common-obj-$(CONFIG_NAND) += nand.o
 common-obj-$(CONFIG_PFLASH_CFI01) += pflash_cfi01.o
 common-obj-$(CONFIG_PFLASH_CFI02) += pflash_cfi02.o
 common-obj-$(CONFIG_XEN) += xen-block.o
-common-obj-$(CONFIG_XEN) += xen_disk.o
 common-obj-$(CONFIG_ECC) += ecc.o
 common-obj-$(CONFIG_ONENAND) += onenand.o
 common-obj-$(CONFIG_NVME_PCI) += nvme.o
diff --git a/hw/block/xen_disk.c b/hw/block/xen_disk.c
deleted file mode 100644
index 477dfcca9f..00
--- a/hw/block/xen_disk.c
+++ /dev/null
@@ -1,1011 +0,0 @@
-/*
- *  xen paravirt block device backend
- *
- *  (c) Gerd Hoffmann 
- *
- *  This program is free software; you can redistribute it and/or modify
- *  it under the terms of the GNU General Public License as published by
- *  the Free Software Foundation; under version 2 of the License.
- *
- *  This program is distributed in the hope that it will be useful,
- *  but WITHOUT ANY WARRANTY; without even the implied warranty of
- *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- *  GNU General Public License for more details.
- *
- *  You should have received a copy of the GNU General Public License along
- *  with this program; if not, see .
- *
- *  Contributions after 2012-01-13 are licensed under the terms of the
- *  GNU GPL, version 2 or (at your option) any later version.
- */
-
-#include "qemu/osdep.h"
-#include "qemu/units.h"
-#include 
-#include 
-
-#include "hw/hw.h"
-#include "hw/xen/xen-legacy-backend.h"
-#include "xen_blkif.h"
-#include "sysemu/blockdev.h"
-#include "sysemu/iothread.h"
-#include "sysemu/block-backend.h"
-#include "qapi/error.h"
-#include "qapi/qmp/qdict.h"
-#include "qapi/qmp/qstring.h"
-#include "trace.h"
-
-/* - */
-
-#define BLOCK_SIZE  512
-#define IOCB_COUNT  (BLKIF_MAX_SEGMENTS_PER_REQUEST + 2)
-
-struct ioreq {
-blkif_request_t req;
-int16_t status;
-
-/* parsed request */
-off_t   start;
-QEMUIOVectorv;
-void*buf;
-size_t  size;
-int presync;
-
-/* aio status */
-int aio_inflight;
-int aio_errors;
-
-struct XenBlkDev*blkdev;
-QLIST_ENTRY(ioreq)   list;
-BlockAcctCookie acct;
-};
-
-#define MAX_RING_PAGE_ORDER 4
-
-struct XenBlkDev {
-struct XenLegacyDevicexendev;  /* must be first */
-char*params;
-char*mode;
-char*type;
-char*dev;
-char*devtype;
-booldirectiosafe;
-const char  *fileproto;
-const char  *filename;
-unsigned intring_ref[1 << MAX_RING_PAGE_ORDER];
-unsigned intnr_ring_ref;
-void*sring;
-int64_t file_blk;
-int64_t file_size;
-int protocol;
-blkif_back_rings_t  rings;
-int more_work;
-
-/* request lists */
-QLIST_HEAD(, ioreq) inflight;
-QLIST_HEAD(, ioreq) finished;
-QLIST_HEAD(, ioreq) freelist;
-int requests_total;
-int requests_inflight;
-int requests_finished;
-unsigned intmax_requests;
-
-gbooleanfeature_discard;
-
-/* qemu block driver */
-DriveInfo   *dinfo;
-BlockBackend*blk;
-QEMUBH  *bh;
-
-IOThread*iothread;
-AioContext  *ctx;
-};
-
-/* - */
-
-static void ioreq_reset(struct ioreq *ioreq)
-{
-memset(>req, 0, sizeof(ioreq->req));
-ioreq->status = 0;
-ioreq->start = 0;
-ioreq->buf = NULL;
-ioreq->size = 0;
-ioreq->presync = 0;
-
-ioreq->aio_inflight = 0;
-ioreq->aio_errors = 0;
-
-ioreq->blkdev = NULL;
-memset(>list, 0, sizeof(ioreq->list));
-memset(>acct, 0, sizeof(ioreq->acct));
-
-qemu_iovec_reset(>v);
-}
-
-static struct ioreq *ioreq_start(struct XenBlkDev *blkdev)
-{
-struct ioreq *ioreq = NULL;
-
-if (QLIST_EMPTY(>freelist)) {
-if (blkdev->requests_total >= blkdev->max_requests) {
-goto out;
-}
-/* allocate new struct */
-ioreq = g_malloc0(sizeof(*ioreq));
-ioreq->blkdev = blkdev;
-blkdev->requests_total++;
-

[Xen-devel] [PULL 18/25] xen: automatically create XenBlockDevice-s

2019-01-14 Thread Anthony PERARD
From: Paul Durrant 

This patch adds create and destroy function for XenBlockDevice-s so that
they can be created automatically when the Xen toolstack instantiates a new
PV backend via xenstore. When the XenBlockDevice is created this way it is
also necessary to create a 'drive' which matches the configuration that the
Xen toolstack has written into xenstore. This is done by formulating the
parameters necessary for each 'blockdev' layer of the drive and then using
qmp_blockdev_add() to create the layers. Also, for compatibility with the
legacy 'xen_disk' implementation, an iothread is automatically created for
the new XenBlockDevice. This, like the driver layers, will be destroyed
after the XenBlockDevice is unrealized.

The legacy backend scan for 'qdisk' is removed by this patch, which makes
the 'xen_disk' code is redundant. The code will be removed by a subsequent
patch.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony PERARD 
Signed-off-by: Anthony PERARD 
---
 hw/block/trace-events   |   4 +
 hw/block/xen-block.c| 375 
 hw/xen/xen-legacy-backend.c |   1 -
 include/hw/xen/xen-block.h  |  12 ++
 4 files changed, 391 insertions(+), 1 deletion(-)

diff --git a/hw/block/trace-events b/hw/block/trace-events
index 89e258319c..55e5a5500c 100644
--- a/hw/block/trace-events
+++ b/hw/block/trace-events
@@ -137,3 +137,7 @@ xen_disk_realize(void) ""
 xen_disk_unrealize(void) ""
 xen_cdrom_realize(void) ""
 xen_cdrom_unrealize(void) ""
+xen_block_blockdev_add(char *str) "%s"
+xen_block_blockdev_del(const char *node_name) "%s"
+xen_block_device_create(unsigned int number) "%u"
+xen_block_device_destroy(unsigned int number) "%u"
diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
index a7c37c185a..be28b63442 100644
--- a/hw/block/xen-block.c
+++ b/hw/block/xen-block.c
@@ -7,12 +7,20 @@
 
 #include "qemu/osdep.h"
 #include "qemu/cutils.h"
+#include "qemu/option.h"
 #include "qapi/error.h"
+#include "qapi/qapi-commands-block-core.h"
+#include "qapi/qapi-commands-misc.h"
+#include "qapi/qapi-visit-block-core.h"
+#include "qapi/qobject-input-visitor.h"
 #include "qapi/visitor.h"
+#include "qapi/qmp/qdict.h"
+#include "qapi/qmp/qstring.h"
 #include "hw/hw.h"
 #include "hw/xen/xen_common.h"
 #include "hw/block/xen_blkif.h"
 #include "hw/xen/xen-block.h"
+#include "hw/xen/xen-backend.h"
 #include "sysemu/blockdev.h"
 #include "sysemu/block-backend.h"
 #include "sysemu/iothread.h"
@@ -474,6 +482,7 @@ static void xen_block_class_init(ObjectClass *class, void 
*data)
 DeviceClass *dev_class = DEVICE_CLASS(class);
 XenDeviceClass *xendev_class = XEN_DEVICE_CLASS(class);
 
+xendev_class->backend = "qdisk";
 xendev_class->device = "vbd";
 xendev_class->get_name = xen_block_get_name;
 xendev_class->realize = xen_block_realize;
@@ -586,3 +595,369 @@ static void xen_block_register_types(void)
 }
 
 type_init(xen_block_register_types)
+
+static void xen_block_blockdev_del(const char *node_name, Error **errp)
+{
+trace_xen_block_blockdev_del(node_name);
+
+qmp_blockdev_del(node_name, errp);
+}
+
+static char *xen_block_blockdev_add(const char *id, QDict *qdict,
+Error **errp)
+{
+const char *driver = qdict_get_try_str(qdict, "driver");
+BlockdevOptions *options = NULL;
+Error *local_err = NULL;
+char *node_name;
+Visitor *v;
+
+if (!driver) {
+error_setg(errp, "no 'driver' parameter");
+return NULL;
+}
+
+node_name = g_strdup_printf("%s-%s", id, driver);
+qdict_put_str(qdict, "node-name", node_name);
+
+trace_xen_block_blockdev_add(node_name);
+
+v = qobject_input_visitor_new(QOBJECT(qdict));
+visit_type_BlockdevOptions(v, NULL, , _err);
+visit_free(v);
+
+if (local_err) {
+error_propagate(errp, local_err);
+goto fail;
+}
+
+qmp_blockdev_add(options, _err);
+
+if (local_err) {
+error_propagate(errp, local_err);
+goto fail;
+}
+
+qapi_free_BlockdevOptions(options);
+
+return node_name;
+
+fail:
+if (options) {
+qapi_free_BlockdevOptions(options);
+}
+g_free(node_name);
+
+return NULL;
+}
+
+static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
+{
+char *node_name = drive->node_name;
+
+if (node_name) {
+Error *local_err = NULL;
+
+xen_block_blockdev_del(node_name, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+return;
+}
+g_free(node_name);
+drive->node_name = NULL;
+}
+g_free(drive->id);
+g_free(drive);
+}
+
+static XenBlockDrive *xen_block_drive_create(const char *id,
+ const char *device_type,
+ QDict *opts, Error **errp)
+{
+const char *params = qdict_get_try_str(opts, "params");
+const char *mode = qdict_get_try_str(opts, "mode");
+const 

[Xen-devel] [PULL 23/25] xen-block: improve batching behaviour

2019-01-14 Thread Anthony PERARD
From: Tim Smith 

When I/O consists of many small requests, performance is improved by
batching them together in a single io_submit() call. When there are
relatively few requests, the extra overhead is not worth it. This
introduces a check to start batching I/O requests via blk_io_plug()/
blk_io_unplug() in an amount proportional to the number which were
already in flight at the time we started reading the ring.

Signed-off-by: Tim Smith 

Re-based and commit comment adjusted.

Signed-off-by: Paul Durrant 
Acked-by: Anthony PERARD 
Signed-off-by: Anthony PERARD 
---
 hw/block/dataplane/xen-block.c | 35 ++
 1 file changed, 35 insertions(+)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 8e3965e171..acd23a74a8 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -528,10 +528,18 @@ static int xen_block_get_request(XenBlockDataPlane 
*dataplane,
 return 0;
 }
 
+/*
+ * Threshold of in-flight requests above which we will start using
+ * blk_io_plug()/blk_io_unplug() to batch requests.
+ */
+#define IO_PLUG_THRESHOLD 1
+
 static void xen_block_handle_requests(XenBlockDataPlane *dataplane)
 {
 RING_IDX rc, rp;
 XenBlockRequest *request;
+int inflight_atstart = dataplane->requests_inflight;
+int batched = 0;
 
 dataplane->more_work = 0;
 
@@ -540,6 +548,18 @@ static void xen_block_handle_requests(XenBlockDataPlane 
*dataplane)
 xen_rmb(); /* Ensure we see queued requests up to 'rp'. */
 
 xen_block_send_response_all(dataplane);
+/*
+ * If there was more than IO_PLUG_THRESHOLD requests in flight
+ * when we got here, this is an indication that there the bottleneck
+ * is below us, so it's worth beginning to batch up I/O requests
+ * rather than submitting them immediately. The maximum number
+ * of requests we're willing to batch is the number already in
+ * flight, so it can grow up to max_requests when the bottleneck
+ * is below us.
+ */
+if (inflight_atstart > IO_PLUG_THRESHOLD) {
+blk_io_plug(dataplane->blk);
+}
 while (rc != rp) {
 /* pull request from ring */
 if (RING_REQUEST_CONS_OVERFLOW(>rings.common, rc)) {
@@ -585,7 +605,22 @@ static void xen_block_handle_requests(XenBlockDataPlane 
*dataplane)
 continue;
 }
 
+if (inflight_atstart > IO_PLUG_THRESHOLD &&
+batched >= inflight_atstart) {
+blk_io_unplug(dataplane->blk);
+}
 xen_block_do_aio(request);
+if (inflight_atstart > IO_PLUG_THRESHOLD) {
+if (batched >= inflight_atstart) {
+blk_io_plug(dataplane->blk);
+batched = 0;
+} else {
+batched++;
+}
+}
+}
+if (inflight_atstart > IO_PLUG_THRESHOLD) {
+blk_io_unplug(dataplane->blk);
 }
 
 if (dataplane->more_work &&
-- 
Anthony PERARD


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PULL 17/25] xen: add a mechanism to automatically create XenDevice-s...

2019-01-14 Thread Anthony PERARD
From: Paul Durrant 

...that maintains compatibility with existing Xen toolstacks.

Xen toolstacks instantiate PV backends by simply writing information into
xenstore and expecting a backend implementation to be watching for this.

This patch adds a new 'xen-backend' module to allow individual XenDevice
implementations to register create and destroy functions. The creator
will be called when a tool-stack instantiates a new backend in this way,
and the destructor will then be called after the resulting XenDevice
object is unrealized.

To support this it is also necessary to add new watchers into the XenBus
implementation to handle enumeration of new backends and also destruction
of XenDevice-s when the toolstack sets the backend 'online' key to 0.

NOTE: This patch only adds the framework. A subsequent patch will add a
  creator function for xen-block devices.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
Signed-off-by: Anthony PERARD 
---
 hw/xen/Makefile.objs |   2 +-
 hw/xen/trace-events  |   3 +
 hw/xen/xen-backend.c | 165 +++
 hw/xen/xen-bus.c | 164 +-
 include/hw/xen/xen-backend.h |  39 +
 include/hw/xen/xen-bus.h |   1 +
 include/qemu/module.h|   3 +
 7 files changed, 375 insertions(+), 2 deletions(-)
 create mode 100644 hw/xen/xen-backend.c
 create mode 100644 include/hw/xen/xen-backend.h

diff --git a/hw/xen/Makefile.objs b/hw/xen/Makefile.objs
index 77c0868190..84df60a928 100644
--- a/hw/xen/Makefile.objs
+++ b/hw/xen/Makefile.objs
@@ -1,5 +1,5 @@
 # xen backend driver support
-common-obj-$(CONFIG_XEN) += xen-legacy-backend.o xen_devconfig.o xen_pvdev.o 
xen-common.o xen-bus.o xen-bus-helper.o
+common-obj-$(CONFIG_XEN) += xen-legacy-backend.o xen_devconfig.o xen_pvdev.o 
xen-common.o xen-bus.o xen-bus-helper.o xen-backend.o
 
 obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen-host-pci-device.o
 obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen_pt.o xen_pt_config_init.o 
xen_pt_graphics.o xen_pt_msi.o
diff --git a/hw/xen/trace-events b/hw/xen/trace-events
index d4651bdb30..f6944624b2 100644
--- a/hw/xen/trace-events
+++ b/hw/xen/trace-events
@@ -16,6 +16,9 @@ xen_domid_restrict(int err) "err: %u"
 # include/hw/xen/xen-bus.c
 xen_bus_realize(void) ""
 xen_bus_unrealize(void) ""
+xen_bus_enumerate(void) ""
+xen_bus_type_enumerate(const char *type) "type: %s"
+xen_bus_backend_create(const char *type, const char *path) "type: %s path: %s"
 xen_bus_add_watch(const char *node, const char *key, char *token) "node: %s 
key: %s token: %s"
 xen_bus_remove_watch(const char *node, const char *key, char *token) "node: %s 
key: %s token: %s"
 xen_bus_watch(const char *token) "token: %s"
diff --git a/hw/xen/xen-backend.c b/hw/xen/xen-backend.c
new file mode 100644
index 00..da065f81b7
--- /dev/null
+++ b/hw/xen/xen-backend.c
@@ -0,0 +1,165 @@
+/*
+ * Copyright (c) 2018  Citrix Systems Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
+#include "hw/xen/xen-backend.h"
+#include "hw/xen/xen-bus.h"
+
+typedef struct XenBackendImpl {
+const char *type;
+XenBackendDeviceCreate create;
+XenBackendDeviceDestroy destroy;
+} XenBackendImpl;
+
+struct XenBackendInstance {
+QLIST_ENTRY(XenBackendInstance) entry;
+const XenBackendImpl *impl;
+XenBus *xenbus;
+char *name;
+XenDevice *xendev;
+};
+
+static GHashTable *xen_backend_table_get(void)
+{
+static GHashTable *table;
+
+if (table == NULL) {
+table = g_hash_table_new(g_str_hash, g_str_equal);
+}
+
+return table;
+}
+
+static void xen_backend_table_add(XenBackendImpl *impl)
+{
+g_hash_table_insert(xen_backend_table_get(), (void *)impl->type, impl);
+}
+
+static const XenBackendImpl *xen_backend_table_lookup(const char *type)
+{
+return g_hash_table_lookup(xen_backend_table_get(), type);
+}
+
+void xen_backend_register(const XenBackendInfo *info)
+{
+XenBackendImpl *impl = g_new0(XenBackendImpl, 1);
+
+g_assert(info->type);
+
+if (xen_backend_table_lookup(info->type)) {
+error_report("attempt to register duplicate Xen backend type '%s'",
+ info->type);
+abort();
+}
+
+if (!info->create) {
+error_report("backend type '%s' has no creator", info->type);
+abort();
+}
+
+impl->type = info->type;
+impl->create = info->create;
+impl->destroy = info->destroy;
+
+xen_backend_table_add(impl);
+}
+
+static QLIST_HEAD(, XenBackendInstance) backend_list;
+
+static void xen_backend_list_add(XenBackendInstance *backend)
+{
+QLIST_INSERT_HEAD(_list, backend, entry);
+}
+
+static XenBackendInstance *xen_backend_list_find(XenDevice *xendev)
+{
+XenBackendInstance *backend;
+
+QLIST_FOREACH(backend, _list, entry) {
+

[Xen-devel] [PULL 13/25] xen: remove 'XenBlkDev' and 'blkdev' names from dataplane/xen-block

2019-01-14 Thread Anthony PERARD
From: Paul Durrant 

This is a purely cosmetic patch that substitutes the old 'struct XenBlkDev'
name with 'XenBlockDataPlane' and 'blkdev' field/variable names with
'dataplane', and then does necessary fix-up to adhere to coding style.

No functional change.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
Signed-off-by: Anthony PERARD 
---
 hw/block/dataplane/xen-block.c | 352 +
 hw/block/dataplane/xen-block.h |   2 +-
 2 files changed, 183 insertions(+), 171 deletions(-)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index ed2b91..1ff464973c 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -38,12 +38,12 @@ struct ioreq {
 int presync;
 int aio_inflight;
 int aio_errors;
-struct XenBlkDev *blkdev;
+XenBlockDataPlane *dataplane;
 QLIST_ENTRY(ioreq) list;
 BlockAcctCookie acct;
 };
 
-struct XenBlkDev {
+struct XenBlockDataPlane {
 XenDevice *xendev;
 XenEventChannel *event_channel;
 unsigned int *ring_ref;
@@ -79,33 +79,33 @@ static void ioreq_reset(struct ioreq *ioreq)
 ioreq->aio_inflight = 0;
 ioreq->aio_errors = 0;
 
-ioreq->blkdev = NULL;
+ioreq->dataplane = NULL;
 memset(>list, 0, sizeof(ioreq->list));
 memset(>acct, 0, sizeof(ioreq->acct));
 
 qemu_iovec_reset(>v);
 }
 
-static struct ioreq *ioreq_start(struct XenBlkDev *blkdev)
+static struct ioreq *ioreq_start(XenBlockDataPlane *dataplane)
 {
 struct ioreq *ioreq = NULL;
 
-if (QLIST_EMPTY(>freelist)) {
-if (blkdev->requests_total >= blkdev->max_requests) {
+if (QLIST_EMPTY(>freelist)) {
+if (dataplane->requests_total >= dataplane->max_requests) {
 goto out;
 }
 /* allocate new struct */
 ioreq = g_malloc0(sizeof(*ioreq));
-ioreq->blkdev = blkdev;
-blkdev->requests_total++;
+ioreq->dataplane = dataplane;
+dataplane->requests_total++;
 qemu_iovec_init(>v, 1);
 } else {
 /* get one from freelist */
-ioreq = QLIST_FIRST(>freelist);
+ioreq = QLIST_FIRST(>freelist);
 QLIST_REMOVE(ioreq, list);
 }
-QLIST_INSERT_HEAD(>inflight, ioreq, list);
-blkdev->requests_inflight++;
+QLIST_INSERT_HEAD(>inflight, ioreq, list);
+dataplane->requests_inflight++;
 
 out:
 return ioreq;
@@ -113,26 +113,26 @@ static struct ioreq *ioreq_start(struct XenBlkDev *blkdev)
 
 static void ioreq_finish(struct ioreq *ioreq)
 {
-struct XenBlkDev *blkdev = ioreq->blkdev;
+XenBlockDataPlane *dataplane = ioreq->dataplane;
 
 QLIST_REMOVE(ioreq, list);
-QLIST_INSERT_HEAD(>finished, ioreq, list);
-blkdev->requests_inflight--;
-blkdev->requests_finished++;
+QLIST_INSERT_HEAD(>finished, ioreq, list);
+dataplane->requests_inflight--;
+dataplane->requests_finished++;
 }
 
 static void ioreq_release(struct ioreq *ioreq, bool finish)
 {
-struct XenBlkDev *blkdev = ioreq->blkdev;
+XenBlockDataPlane *dataplane = ioreq->dataplane;
 
 QLIST_REMOVE(ioreq, list);
 ioreq_reset(ioreq);
-ioreq->blkdev = blkdev;
-QLIST_INSERT_HEAD(>freelist, ioreq, list);
+ioreq->dataplane = dataplane;
+QLIST_INSERT_HEAD(>freelist, ioreq, list);
 if (finish) {
-blkdev->requests_finished--;
+dataplane->requests_finished--;
 } else {
-blkdev->requests_inflight--;
+dataplane->requests_inflight--;
 }
 }
 
@@ -142,7 +142,7 @@ static void ioreq_release(struct ioreq *ioreq, bool finish)
  */
 static int ioreq_parse(struct ioreq *ioreq)
 {
-struct XenBlkDev *blkdev = ioreq->blkdev;
+XenBlockDataPlane *dataplane = ioreq->dataplane;
 size_t len;
 int i;
 
@@ -165,12 +165,12 @@ static int ioreq_parse(struct ioreq *ioreq)
 };
 
 if (ioreq->req.operation != BLKIF_OP_READ &&
-blk_is_read_only(blkdev->blk)) {
+blk_is_read_only(dataplane->blk)) {
 error_report("error: write req for ro device");
 goto err;
 }
 
-ioreq->start = ioreq->req.sector_number * blkdev->file_blk;
+ioreq->start = ioreq->req.sector_number * dataplane->file_blk;
 for (i = 0; i < ioreq->req.nr_segments; i++) {
 if (i == BLKIF_MAX_SEGMENTS_PER_REQUEST) {
 error_report("error: nr_segments too big");
@@ -180,16 +180,16 @@ static int ioreq_parse(struct ioreq *ioreq)
 error_report("error: first > last sector");
 goto err;
 }
-if (ioreq->req.seg[i].last_sect * blkdev->file_blk >= XC_PAGE_SIZE) {
+if (ioreq->req.seg[i].last_sect * dataplane->file_blk >= XC_PAGE_SIZE) 
{
 error_report("error: page crossing");
 goto err;
 }
 
 len = (ioreq->req.seg[i].last_sect -
-   ioreq->req.seg[i].first_sect + 1) * blkdev->file_blk;
+   ioreq->req.seg[i].first_sect + 1) * dataplane->file_blk;
 ioreq->size += len;
 }
-if 

[Xen-devel] [PULL 11/25] xen: remove unnecessary code from dataplane/xen-block.c

2019-01-14 Thread Anthony PERARD
From: Paul Durrant 

Not all of the code duplicated from xen_disk.c is required as the basis for
the new dataplane implementation so this patch removes extraneous code,
along with the legacy #includes and calls to the legacy xen_pv_printf()
function. Error messages are changed to be reported using error_report().

NOTE: The code is still not yet built. Further transformations will be
  required to make it correctly interface to the new XenBus/XenDevice
  framework. They will be delivered in a subsequent patch.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
Signed-off-by: Anthony PERARD 
---
 hw/block/dataplane/xen-block.c | 429 ++---
 1 file changed, 23 insertions(+), 406 deletions(-)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 9fae50534e..228472320a 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -1,45 +1,23 @@
 /*
- *  xen paravirt block device backend
+ * Copyright (c) 2018  Citrix Systems Inc.
+ * (c) Gerd Hoffmann 
  *
- *  (c) Gerd Hoffmann 
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; under version 2 of the License.
  *
- *  This program is free software; you can redistribute it and/or modify
- *  it under the terms of the GNU General Public License as published by
- *  the Free Software Foundation; under version 2 of the License.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
  *
- *  This program is distributed in the hope that it will be useful,
- *  but WITHOUT ANY WARRANTY; without even the implied warranty of
- *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- *  GNU General Public License for more details.
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see .
  *
- *  You should have received a copy of the GNU General Public License along
- *  with this program; if not, see .
- *
- *  Contributions after 2012-01-13 are licensed under the terms of the
- *  GNU GPL, version 2 or (at your option) any later version.
+ * Contributions after 2012-01-13 are licensed under the terms of the
+ * GNU GPL, version 2 or (at your option) any later version.
  */
 
-#include "qemu/osdep.h"
-#include "qemu/units.h"
-#include 
-#include 
-
-#include "hw/hw.h"
-#include "hw/xen/xen_backend.h"
-#include "xen_blkif.h"
-#include "sysemu/blockdev.h"
-#include "sysemu/iothread.h"
-#include "sysemu/block-backend.h"
-#include "qapi/error.h"
-#include "qapi/qmp/qdict.h"
-#include "qapi/qmp/qstring.h"
-#include "trace.h"
-
-/* - */
-
-#define BLOCK_SIZE  512
-#define IOCB_COUNT  (BLKIF_MAX_SEGMENTS_PER_REQUEST + 2)
-
 struct ioreq {
 blkif_request_t req;
 int16_t status;
@@ -101,8 +79,6 @@ struct XenBlkDev {
 AioContext  *ctx;
 };
 
-/* - */
-
 static void ioreq_reset(struct ioreq *ioreq)
 {
 memset(>req, 0, sizeof(ioreq->req));
@@ -183,11 +159,6 @@ static int ioreq_parse(struct ioreq *ioreq)
 size_t len;
 int i;
 
-xen_pv_printf(
-xendev, 3,
-"op %d, nr %d, handle %d, id %" PRId64 ", sector %" PRId64 "\n",
-ioreq->req.operation, ioreq->req.nr_segments,
-ioreq->req.handle, ioreq->req.id, ioreq->req.sector_number);
 switch (ioreq->req.operation) {
 case BLKIF_OP_READ:
 break;
@@ -202,28 +173,27 @@ static int ioreq_parse(struct ioreq *ioreq)
 case BLKIF_OP_DISCARD:
 return 0;
 default:
-xen_pv_printf(xendev, 0, "error: unknown operation (%d)\n",
-  ioreq->req.operation);
+error_report("error: unknown operation (%d)", ioreq->req.operation);
 goto err;
 };
 
 if (ioreq->req.operation != BLKIF_OP_READ && blkdev->mode[0] != 'w') {
-xen_pv_printf(xendev, 0, "error: write req for ro device\n");
+error_report("error: write req for ro device");
 goto err;
 }
 
 ioreq->start = ioreq->req.sector_number * blkdev->file_blk;
 for (i = 0; i < ioreq->req.nr_segments; i++) {
 if (i == BLKIF_MAX_SEGMENTS_PER_REQUEST) {
-xen_pv_printf(xendev, 0, "error: nr_segments too big\n");
+error_report("error: nr_segments too big");
 goto err;
 }
 if (ioreq->req.seg[i].first_sect > ioreq->req.seg[i].last_sect) {
-xen_pv_printf(xendev, 0, "error: first > last sector\n");
+error_report("error: first > last sector");
 goto err;
 }
 if 

[Xen-devel] [PULL 12/25] xen: add header and build dataplane/xen-block.c

2019-01-14 Thread Anthony PERARD
From: Paul Durrant 

This patch adds the transformations necessary to get dataplane/xen-block.c
to build against the new XenBus/XenDevice framework. MAINTAINERS is also
updated due to the introduction of dataplane/xen-block.h.

NOTE: Existing data structure names are retained for the moment. These will
  be modified by subsequent patches. A typedef for XenBlockDataPlane
  has been added to the header (based on the old struct XenBlkDev name
  for the moment) so that the old names don't need to leak out of the
  dataplane code.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
Signed-off-by: Anthony PERARD 
---
 MAINTAINERS  |   1 +
 hw/block/dataplane/Makefile.objs |   1 +
 hw/block/dataplane/xen-block.c   | 356 ++-
 hw/block/dataplane/xen-block.h   |  29 +++
 4 files changed, 287 insertions(+), 100 deletions(-)
 create mode 100644 hw/block/dataplane/xen-block.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 4c98b34853..43b2691b5d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -419,6 +419,7 @@ F: hw/block/dataplane/xen*
 F: hw/xen/
 F: hw/xenpv/
 F: hw/i386/xen/
+F: include/hw/block/dataplane/xen*
 F: include/hw/xen/
 F: include/sysemu/xen-mapcache.h
 
diff --git a/hw/block/dataplane/Makefile.objs b/hw/block/dataplane/Makefile.objs
index e786f66421..c6c68dbc00 100644
--- a/hw/block/dataplane/Makefile.objs
+++ b/hw/block/dataplane/Makefile.objs
@@ -1 +1,2 @@
 obj-y += virtio-blk.o
+obj-$(CONFIG_XEN) += xen-block.o
diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 228472320a..ed2b91 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -18,65 +18,53 @@
  * GNU GPL, version 2 or (at your option) any later version.
  */
 
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
+#include "hw/hw.h"
+#include "hw/xen/xen_common.h"
+#include "hw/block/xen_blkif.h"
+#include "sysemu/block-backend.h"
+#include "sysemu/iothread.h"
+#include "xen-block.h"
+
 struct ioreq {
-blkif_request_t req;
-int16_t status;
-
-/* parsed request */
-off_t   start;
-QEMUIOVectorv;
-void*buf;
-size_t  size;
-int presync;
-
-/* aio status */
-int aio_inflight;
-int aio_errors;
-
-struct XenBlkDev*blkdev;
-QLIST_ENTRY(ioreq)   list;
-BlockAcctCookie acct;
+blkif_request_t req;
+int16_t status;
+off_t start;
+QEMUIOVector v;
+void *buf;
+size_t size;
+int presync;
+int aio_inflight;
+int aio_errors;
+struct XenBlkDev *blkdev;
+QLIST_ENTRY(ioreq) list;
+BlockAcctCookie acct;
 };
 
-#define MAX_RING_PAGE_ORDER 4
-
 struct XenBlkDev {
-struct XenLegacyDevicexendev;  /* must be first */
-char*params;
-char*mode;
-char*type;
-char*dev;
-char*devtype;
-booldirectiosafe;
-const char  *fileproto;
-const char  *filename;
-unsigned intring_ref[1 << MAX_RING_PAGE_ORDER];
-unsigned intnr_ring_ref;
-void*sring;
-int64_t file_blk;
-int64_t file_size;
-int protocol;
-blkif_back_rings_t  rings;
-int more_work;
-
-/* request lists */
+XenDevice *xendev;
+XenEventChannel *event_channel;
+unsigned int *ring_ref;
+unsigned int nr_ring_ref;
+void *sring;
+int64_t file_blk;
+int64_t file_size;
+int protocol;
+blkif_back_rings_t rings;
+int more_work;
 QLIST_HEAD(inflight_head, ioreq) inflight;
 QLIST_HEAD(finished_head, ioreq) finished;
 QLIST_HEAD(freelist_head, ioreq) freelist;
-int requests_total;
-int requests_inflight;
-int requests_finished;
-unsigned intmax_requests;
-
-gbooleanfeature_discard;
-
-/* qemu block driver */
-DriveInfo   *dinfo;
-BlockBackend*blk;
-QEMUBH  *bh;
-
-IOThread*iothread;
-AioContext  *ctx;
+int requests_total;
+int requests_inflight;
+int requests_finished;
+unsigned int max_requests;
+BlockBackend *blk;
+QEMUBH *bh;
+IOThread *iothread;
+AioContext *ctx;
 };
 
 static void ioreq_reset(struct ioreq *ioreq)
@@ -155,7 +143,6 @@ static void ioreq_release(struct ioreq *ioreq, bool finish)
 static int ioreq_parse(struct ioreq *ioreq)
 {
 struct XenBlkDev *blkdev = ioreq->blkdev;
-struct XenLegacyDevice *xendev = >xendev;
 size_t len;
 int i;
 
@@ -177,7 +164,8 @@ static int ioreq_parse(struct ioreq *ioreq)
 goto err;
 };
 
-if (ioreq->req.operation != BLKIF_OP_READ && blkdev->mode[0] != 'w') {
+if 

[Xen-devel] [PULL 19/25] MAINTAINERS: add myself as a Xen maintainer

2019-01-14 Thread Anthony PERARD
From: Paul Durrant 

I have made many significant contributions to the Xen code in QEMU,
particularly the recent patches introducing a new PV device framework.
I intend to make further significant contributions, porting other PV back-
ends to the new framework with the intent of eventually removing the
legacy code. It therefore seems reasonable that I become a maintainer of
the Xen code.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
Acked-by: Stefano Stabellini 
Signed-off-by: Anthony PERARD 
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 43b2691b5d..2a1520dee7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -407,6 +407,7 @@ Guest CPU Cores (Xen):
 X86
 M: Stefano Stabellini 
 M: Anthony Perard 
+M: Paul Durrant 
 L: xen-devel@lists.xenproject.org
 S: Supported
 F: */xen*
-- 
Anthony PERARD


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PULL 16/25] xen: add implementations of xen-block connect and disconnect functions...

2019-01-14 Thread Anthony PERARD
From: Paul Durrant 

...and wire in the dataplane.

This patch adds the remaining code to make the xen-block XenDevice
functional. The parameters that a block frontend expects to find are
populated in the backend xenstore area, and the 'ring-ref' and
'event-channel' values specified in the frontend xenstore area are
mapped/bound and used to set up the dataplane.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
Signed-off-by: Anthony PERARD 
---
 hw/block/xen-block.c   | 166 
 hw/xen/trace-events|   3 +
 hw/xen/xen-bus.c   | 187 -
 include/hw/xen/xen-block.h |   9 ++
 include/hw/xen/xen-bus.h   |  14 ++-
 5 files changed, 353 insertions(+), 26 deletions(-)

diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
index 3a963b0383..a7c37c185a 100644
--- a/hw/block/xen-block.c
+++ b/hw/block/xen-block.c
@@ -10,7 +10,13 @@
 #include "qapi/error.h"
 #include "qapi/visitor.h"
 #include "hw/hw.h"
+#include "hw/xen/xen_common.h"
+#include "hw/block/xen_blkif.h"
 #include "hw/xen/xen-block.h"
+#include "sysemu/blockdev.h"
+#include "sysemu/block-backend.h"
+#include "sysemu/iothread.h"
+#include "dataplane/xen-block.h"
 #include "trace.h"
 
 static char *xen_block_get_name(XenDevice *xendev, Error **errp)
@@ -28,6 +34,8 @@ static void xen_block_disconnect(XenDevice *xendev, Error 
**errp)
 XenBlockVdev *vdev = >props.vdev;
 
 trace_xen_block_disconnect(type, vdev->disk, vdev->partition);
+
+xen_block_dataplane_stop(blockdev->dataplane);
 }
 
 static void xen_block_connect(XenDevice *xendev, Error **errp)
@@ -35,8 +43,72 @@ static void xen_block_connect(XenDevice *xendev, Error 
**errp)
 XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
 const char *type = object_get_typename(OBJECT(blockdev));
 XenBlockVdev *vdev = >props.vdev;
+unsigned int order, nr_ring_ref, *ring_ref, event_channel, protocol;
+char *str;
 
 trace_xen_block_connect(type, vdev->disk, vdev->partition);
+
+if (xen_device_frontend_scanf(xendev, "ring-page-order", "%u",
+  ) != 1) {
+nr_ring_ref = 1;
+ring_ref = g_new(unsigned int, nr_ring_ref);
+
+if (xen_device_frontend_scanf(xendev, "ring-ref", "%u",
+  _ref[0]) != 1) {
+error_setg(errp, "failed to read ring-ref");
+g_free(ring_ref);
+return;
+}
+} else if (order <= blockdev->props.max_ring_page_order) {
+unsigned int i;
+
+nr_ring_ref = 1 << order;
+ring_ref = g_new(unsigned int, nr_ring_ref);
+
+for (i = 0; i < nr_ring_ref; i++) {
+const char *key = g_strdup_printf("ring-ref%u", i);
+
+if (xen_device_frontend_scanf(xendev, key, "%u",
+  _ref[i]) != 1) {
+error_setg(errp, "failed to read %s", key);
+g_free((gpointer)key);
+g_free(ring_ref);
+return;
+}
+
+g_free((gpointer)key);
+}
+} else {
+error_setg(errp, "invalid ring-page-order (%d)", order);
+return;
+}
+
+if (xen_device_frontend_scanf(xendev, "event-channel", "%u",
+  _channel) != 1) {
+error_setg(errp, "failed to read event-channel");
+g_free(ring_ref);
+return;
+}
+
+if (xen_device_frontend_scanf(xendev, "protocol", "%ms",
+  ) != 1) {
+protocol = BLKIF_PROTOCOL_NATIVE;
+} else {
+if (strcmp(str, XEN_IO_PROTO_ABI_X86_32) == 0) {
+protocol = BLKIF_PROTOCOL_X86_32;
+} else if (strcmp(str, XEN_IO_PROTO_ABI_X86_64) == 0) {
+protocol = BLKIF_PROTOCOL_X86_64;
+} else {
+protocol = BLKIF_PROTOCOL_NATIVE;
+}
+
+free(str);
+}
+
+xen_block_dataplane_start(blockdev->dataplane, ring_ref, nr_ring_ref,
+  event_channel, protocol, errp);
+
+g_free(ring_ref);
 }
 
 static void xen_block_unrealize(XenDevice *xendev, Error **errp)
@@ -56,6 +128,9 @@ static void xen_block_unrealize(XenDevice *xendev, Error 
**errp)
 /* Disconnect from the frontend in case this has not already happened */
 xen_block_disconnect(xendev, NULL);
 
+xen_block_dataplane_destroy(blockdev->dataplane);
+blockdev->dataplane = NULL;
+
 if (blockdev_class->unrealize) {
 blockdev_class->unrealize(blockdev, errp);
 }
@@ -68,6 +143,7 @@ static void xen_block_realize(XenDevice *xendev, Error 
**errp)
 XEN_BLOCK_DEVICE_GET_CLASS(xendev);
 const char *type = object_get_typename(OBJECT(blockdev));
 XenBlockVdev *vdev = >props.vdev;
+BlockConf *conf = >props.conf;
 Error *local_err = NULL;
 
 if (vdev->type == XEN_BLOCK_VDEV_TYPE_INVALID) {
@@ -81,8 +157,62 @@ static void xen_block_realize(XenDevice *xendev, Error 
**errp)

[Xen-devel] [PULL 22/25] xen: Replace few mentions of xend by libxl

2019-01-14 Thread Anthony PERARD
xend have been replaced by libxenlight (libxl) for many Xen releases
now.

Signed-off-by: Anthony PERARD 
Acked-by: Stefano Stabellini 
---
 hw/xenpv/xen_machine_pv.c | 2 +-
 include/hw/xen/xen.h  | 2 +-
 qemu-options.hx   | 4 ++--
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/hw/xenpv/xen_machine_pv.c b/hw/xenpv/xen_machine_pv.c
index 8d68fef25e..dcaf2a01a3 100644
--- a/hw/xenpv/xen_machine_pv.c
+++ b/hw/xenpv/xen_machine_pv.c
@@ -43,7 +43,7 @@ static void xen_init_pv(MachineState *machine)
 
 switch (xen_mode) {
 case XEN_ATTACH:
-/* nothing to do, xend handles everything */
+/* nothing to do, libxl handles everything */
 break;
 case XEN_EMULATE:
 error_report("xen emulation not implemented (yet)");
diff --git a/include/hw/xen/xen.h b/include/hw/xen/xen.h
index 978aaa31fb..ba039c146d 100644
--- a/include/hw/xen/xen.h
+++ b/include/hw/xen/xen.h
@@ -15,7 +15,7 @@
 /* xen-machine.c */
 enum xen_mode {
 XEN_EMULATE = 0,  // xen emulation, using xenner (default)
-XEN_ATTACH// attach to xen domain created by xend
+XEN_ATTACH// attach to xen domain created by libxl
 };
 
 extern uint32_t xen_domid;
diff --git a/qemu-options.hx b/qemu-options.hx
index 7c323c9406..521511ec13 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -3396,7 +3396,7 @@ DEF("xen-domid", HAS_ARG, QEMU_OPTION_xen_domid,
 "-xen-domid id   specify xen guest domain id\n", QEMU_ARCH_ALL)
 DEF("xen-attach", 0, QEMU_OPTION_xen_attach,
 "-xen-attach attach to existing xen domain\n"
-"xend will use this when starting QEMU\n",
+"libxl will use this when starting QEMU\n",
 QEMU_ARCH_ALL)
 DEF("xen-domid-restrict", 0, QEMU_OPTION_xen_domid_restrict,
 "-xen-domid-restrict restrict set of available xen operations\n"
@@ -3410,7 +3410,7 @@ Specify xen guest domain @var{id} (XEN only).
 @item -xen-attach
 @findex -xen-attach
 Attach to existing xen domain.
-xend will use this when starting QEMU (XEN only).
+libxl will use this when starting QEMU (XEN only).
 @findex -xen-domid-restrict
 Restrict set of available xen operations to specified domain id (XEN only).
 ETEXI
-- 
Anthony PERARD


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

  1   2   >