Re: [Xen-devel] [PATCH v7 09/15] argo: implement the sendv op; evtchn: expose send_guest_global_virq
>>> On 05.02.19 at 01:52, wrote: > To make Argo's current Experimental status clearer, with the ABI stability > status that accords, I propose the following addition to SUPPORT.md: > > Within section: ## Virtual Hardware, Hypervisor > > ### Argo: Inter-domain message delivery by hypercall. > > Status: Experimental Ah yes. It's odd enough that no-one so far has noticed the need for an addition there. I think we're still on the path of learning that feature additions need new entries there. Jan ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v7 09/15] argo: implement the sendv op; evtchn: expose send_guest_global_virq
On Mon, Feb 4, 2019 at 6:41 AM Jan Beulich wrote: > > >>> On 31.01.19 at 05:28, wrote: > > @@ -1237,6 +1864,54 @@ compat_argo_op(unsigned int cmd, > > XEN_GUEST_HANDLE_PARAM(void) arg1, > > break; > > } > > > > +case XEN_ARGO_OP_sendv: > > +{ > > +xen_argo_send_addr_t send_addr; > > +xen_argo_iov_t iovs[XEN_ARGO_MAXIOV]; > > +unsigned int i; > > +unsigned int niov; > > + > > +XEN_GUEST_HANDLE_PARAM(xen_argo_send_addr_t) send_addr_hnd = > > +guest_handle_cast(arg1, xen_argo_send_addr_t); > > +/* arg2: iovs, arg3: niov, arg4: message_type */ > > + > > +rc = copy_from_guest(&send_addr, send_addr_hnd, 1) ? -EFAULT : 0; > > +if ( rc ) > > +break; > > + > > +if ( unlikely(arg3 > XEN_ARGO_MAXIOV) ) > > +{ > > +rc = -EINVAL; > > +break; > > +} > > +niov = array_index_nospec(arg3, XEN_ARGO_MAXIOV + 1); > > + > > +/* > > + * Limited scope for compat_iovs array: enables a single > > copy_from_guest > > + * call and discards the array from the stack before calling sendv. > > + */ > > What makes you think the array gets removed from the stack again > before the call? The typical way of setting up stack frames for a > function is to allocate the full chunk of space needed at the start > of the function, and remove it before returning. Without the > argo_dprintk() after the switch() there would be the potential of > the sendv() carried out as a tail call, but you can't rely on that. OK. I've revised the comment. > With the current XEN_ARGO_MAXIOV value of 8 the overall frame > size is still tolerable, I would say. But I think you want to add > BUILD_BUG_ON()s here and in the native handler, such that > careless bumping of the value won't go unnoticed (but also see > below). ack, I've added BUILD_BUG_ON to both. > > > --- a/xen/include/public/argo.h > > +++ b/xen/include/public/argo.h > > @@ -43,6 +43,28 @@ > > /* Fixed-width type for "argo port" number. Nothing to do with evtchns. */ > > typedef uint32_t xen_argo_port_t; > > > > +/* > > + * XEN_ARGO_MAXIOV : maximum number of iovs accepted in a single sendv. > > + * Caution is required if this value is increased: this determines the > > size of > > + * an array of xen_argo_iov_t structs on the hypervisor stack, so could > > cause > > + * stack overflow if the value is too large. > > + * The Linux Argo driver never passes more than two iovs. > > + * > > + * This value should also not exceed 128 to ensure that the total amount > > of data > > + * posted in a single Argo sendv operation cannot exceed 2^31 bytes, to > > reduce > > + * risk of integer overflow defects: > > + * Each argo iov can hold ~ 2^24 bytes, so XEN_ARGO_MAXIOV <= 2^(31-24), > > + * ie. keep XEN_ARGO_MAXIOV <= 128. > > +*/ > > +#define XEN_ARGO_MAXIOV 8U > > How does 2^31 come into play here? uint32_t can hold up to 2^32, and > you shouldn't be using signed arithmetic anywhere by this time anymore. > I'm also struggling to see what the "~ 2^24 bytes" refers to - I see nothing > along these lines added to the public header, and ... > > > +typedef struct xen_argo_iov > > +{ > > +XEN_GUEST_HANDLE(uint8) iov_hnd; > > +uint32_t iov_len; > > ... the field here allows for 2^32-1. Oh, it's XEN_ARGO_MAX_RING_SIZE. > It would help if the comment cross referenced that name. I've removed the second paragraph of the comment entirely as it's no longer accurate or required due to the bounds checking in iov_count. > Btw., neither of these two maximum values look to be architectural limits, > so I wonder whether, before declaring the ABI stable, these constants > shouldn't be purged and replaced by settings the guest is to retrieve via > hypercall. That could potentially be useful; though it hasn't been necessary so far. (fwiw: A determined guest can already retrieve these settings via hypercall.) To make Argo's current Experimental status clearer, with the ABI stability status that accords, I propose the following addition to SUPPORT.md: Within section: ## Virtual Hardware, Hypervisor ### Argo: Inter-domain message delivery by hypercall. Status: Experimental Christopher ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v7 09/15] argo: implement the sendv op; evtchn: expose send_guest_global_virq
>>> On 31.01.19 at 05:28, wrote: > @@ -1237,6 +1864,54 @@ compat_argo_op(unsigned int cmd, > XEN_GUEST_HANDLE_PARAM(void) arg1, > break; > } > > +case XEN_ARGO_OP_sendv: > +{ > +xen_argo_send_addr_t send_addr; > +xen_argo_iov_t iovs[XEN_ARGO_MAXIOV]; > +unsigned int i; > +unsigned int niov; > + > +XEN_GUEST_HANDLE_PARAM(xen_argo_send_addr_t) send_addr_hnd = > +guest_handle_cast(arg1, xen_argo_send_addr_t); > +/* arg2: iovs, arg3: niov, arg4: message_type */ > + > +rc = copy_from_guest(&send_addr, send_addr_hnd, 1) ? -EFAULT : 0; > +if ( rc ) > +break; > + > +if ( unlikely(arg3 > XEN_ARGO_MAXIOV) ) > +{ > +rc = -EINVAL; > +break; > +} > +niov = array_index_nospec(arg3, XEN_ARGO_MAXIOV + 1); > + > +/* > + * Limited scope for compat_iovs array: enables a single > copy_from_guest > + * call and discards the array from the stack before calling sendv. > + */ What makes you think the array gets removed from the stack again before the call? The typical way of setting up stack frames for a function is to allocate the full chunk of space needed at the start of the function, and remove it before returning. Without the argo_dprintk() after the switch() there would be the potential of the sendv() carried out as a tail call, but you can't rely on that. With the current XEN_ARGO_MAXIOV value of 8 the overall frame size is still tolerable, I would say. But I think you want to add BUILD_BUG_ON()s here and in the native handler, such that careless bumping of the value won't go unnoticed (but also see below). > --- a/xen/include/public/argo.h > +++ b/xen/include/public/argo.h > @@ -43,6 +43,28 @@ > /* Fixed-width type for "argo port" number. Nothing to do with evtchns. */ > typedef uint32_t xen_argo_port_t; > > +/* > + * XEN_ARGO_MAXIOV : maximum number of iovs accepted in a single sendv. > + * Caution is required if this value is increased: this determines the size > of > + * an array of xen_argo_iov_t structs on the hypervisor stack, so could cause > + * stack overflow if the value is too large. > + * The Linux Argo driver never passes more than two iovs. > + * > + * This value should also not exceed 128 to ensure that the total amount of > data > + * posted in a single Argo sendv operation cannot exceed 2^31 bytes, to > reduce > + * risk of integer overflow defects: > + * Each argo iov can hold ~ 2^24 bytes, so XEN_ARGO_MAXIOV <= 2^(31-24), > + * ie. keep XEN_ARGO_MAXIOV <= 128. > +*/ > +#define XEN_ARGO_MAXIOV 8U How does 2^31 come into play here? uint32_t can hold up to 2^32, and you shouldn't be using signed arithmetic anywhere by this time anymore. I'm also struggling to see what the "~ 2^24 bytes" refers to - I see nothing along these lines added to the public header, and ... > +typedef struct xen_argo_iov > +{ > +XEN_GUEST_HANDLE(uint8) iov_hnd; > +uint32_t iov_len; ... the field here allows for 2^32-1. Oh, it's XEN_ARGO_MAX_RING_SIZE. It would help if the comment cross referenced that name. Btw., neither of these two maximum values look to be architectural limits, so I wonder whether, before declaring the ABI stable, these constants shouldn't be purged and replaced by settings the guest is to retrieve via hypercall. Jan ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v7 09/15] argo: implement the sendv op; evtchn: expose send_guest_global_virq
On Thu, Jan 31, 2019 at 8:38 AM Roger Pau Monné wrote: > > On Wed, Jan 30, 2019 at 08:28:14PM -0800, Christopher Clark wrote: > > sendv operation is invoked to perform a synchronous send of buffers > > contained in iovs to a remote domain's registered ring. > > > > It takes: > > * A destination address (domid, port) for the ring to send to. > >It performs a most-specific match lookup, to allow for wildcard. > > * A source address, used to inform the destination of where to reply. > > * The address of an array of iovs containing the data to send > > * .. and the length of that array of iovs > > * and a 32-bit message type, available to communicate message context > >data (eg. kernel-to-kernel, separate from the application data). > > > > If insufficient space exists in the destination ring, it will return > > -EAGAIN and Xen will notify the caller when sufficient space becomes > > available. > > > > Accesses to the ring indices are appropriately atomic. The rings are > > mapped into Xen's private address space to write as needed and the > > mappings are retained for later use. > > > > Notifications are sent to guests via VIRQ and send_guest_global_virq is > > exposed in the change to enable argo to call it. VIRQ_ARGO is claimed > > from the VIRQ previously reserved for this purpose (#11). > > > > The VIRQ notification method is used rather than sending events using > > evtchn functions directly because: > > > > * no current event channel type is an exact fit for the intended > > behaviour. ECS_IPI is closest, but it disallows migration to > > other VCPUs which is not necessarily a requirement for Argo. > > > > * at the point of argo_init, allocation of an event channel is > > complicated by none of the guest VCPUs being initialized yet > > and the event channel logic expects that a valid event channel > > has a present VCPU. > > > > * at the point of signalling a notification, the VIRQ logic is already > > defensive: if d->vcpu[0] is NULL, the notification is just silently > > dropped, whereas the evtchn_send logic is not so defensive: vcpu[0] > > must not be NULL, otherwise a null pointer dereference occurs. > > > > Using a VIRQ removes the need for the guest to query to determine which > > event channel notifications will be delivered on. This is also likely to > > simplify establishing future L0/L1 nested hypervisor argo communication. > > > > Signed-off-by: Christopher Clark > > Tested-by: Chris Patterson > > There's one style nit that I think can be fixed while committing: > > Reviewed-by: Roger Pau Monné Thanks. > Despite the usage of the open-coded mask below. As with previous > patches this is argos code, so I'm not going to oppose, but again I > think using such open coded masks is bad, and can lead to bugs in the > code. It can be fixed by a follow up patch. Have responded with a proposed fix to address this on the other thread. > > > +static int > > +ringbuf_insert(const struct domain *d, struct argo_ring_info *ring_info, > > + const struct argo_ring_id *src_id, xen_argo_iov_t *iovs, > > + unsigned int niov, uint32_t message_type, > > + unsigned long *out_len) > > +{ > > +xen_argo_ring_t ring; > > +struct xen_argo_ring_message_header mh = { }; > > +int sp, ret; > > +unsigned int len = 0; > > +xen_argo_iov_t *piov; > > +XEN_GUEST_HANDLE(uint8) NULL_hnd = { }; > > + > > +ASSERT(LOCKING_L3(d, ring_info)); > > + > > +/* > > + * Obtain the total size of data to transmit -- sets the 'len' variable > > + * -- and sanity check that the iovs conform to size and number limits. > > + * Enforced below: no more than 'len' bytes of guest data > > + * (plus the message header) will be sent in this operation. > > + */ > > +ret = iov_count(iovs, niov, &len); > > +if ( ret ) > > +return ret; > > + > > +/* > > + * Upper bound check the message len against the ring size. > > + * The message must not fill the ring; there must be at least one slot > > + * remaining so we can distinguish a full ring from an empty one. > > + * iov_count has already verified: len <= MAX_ARGO_MESSAGE_SIZE. > > + */ > > +if ( (ROUNDUP_MESSAGE(len) + sizeof(struct > > xen_argo_ring_message_header)) >missing space ^ > > +>= ring_info->len ) > > Align of >= also looks weird, should be aligned to the parenthesis > before ROUNDUP_. ack > > @@ -1175,6 +1766,42 @@ do_argo_op(unsigned int cmd, > > XEN_GUEST_HANDLE_PARAM(void) arg1, > > break; > > } > > > > +case XEN_ARGO_OP_sendv: > > +{ > > +xen_argo_send_addr_t send_addr; > > +xen_argo_iov_t iovs[XEN_ARGO_MAXIOV]; > > +unsigned int niov; > > + > > +XEN_GUEST_HANDLE_PARAM(xen_argo_send_addr_t) send_addr_hnd = > > +guest_handle_cast(arg1, xen_argo_send_addr_t); > > +XEN_GUES
Re: [Xen-devel] [PATCH v7 09/15] argo: implement the sendv op; evtchn: expose send_guest_global_virq
>>> On 31.01.19 at 17:35, wrote: > On Wed, Jan 30, 2019 at 08:28:14PM -0800, Christopher Clark wrote: >> +static int >> +ringbuf_insert(const struct domain *d, struct argo_ring_info *ring_info, >> + const struct argo_ring_id *src_id, xen_argo_iov_t *iovs, >> + unsigned int niov, uint32_t message_type, >> + unsigned long *out_len) >> +{ >> +xen_argo_ring_t ring; >> +struct xen_argo_ring_message_header mh = { }; >> +int sp, ret; >> +unsigned int len = 0; >> +xen_argo_iov_t *piov; >> +XEN_GUEST_HANDLE(uint8) NULL_hnd = { }; >> + >> +ASSERT(LOCKING_L3(d, ring_info)); >> + >> +/* >> + * Obtain the total size of data to transmit -- sets the 'len' variable >> + * -- and sanity check that the iovs conform to size and number limits. >> + * Enforced below: no more than 'len' bytes of guest data >> + * (plus the message header) will be sent in this operation. >> + */ >> +ret = iov_count(iovs, niov, &len); >> +if ( ret ) >> +return ret; >> + >> +/* >> + * Upper bound check the message len against the ring size. >> + * The message must not fill the ring; there must be at least one slot >> + * remaining so we can distinguish a full ring from an empty one. >> + * iov_count has already verified: len <= MAX_ARGO_MESSAGE_SIZE. >> + */ >> +if ( (ROUNDUP_MESSAGE(len) + sizeof(struct >> xen_argo_ring_message_header)) >missing space > ^ >> +>= ring_info->len ) > > Align of >= also looks weird, should be aligned to the parenthesis > before ROUNDUP_. Well, to be precise the >= belongs at the end of the previous line, so perhaps the line wrapping wants to be changed altogether. Jan ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v7 09/15] argo: implement the sendv op; evtchn: expose send_guest_global_virq
On Wed, Jan 30, 2019 at 08:28:14PM -0800, Christopher Clark wrote: > sendv operation is invoked to perform a synchronous send of buffers > contained in iovs to a remote domain's registered ring. > > It takes: > * A destination address (domid, port) for the ring to send to. >It performs a most-specific match lookup, to allow for wildcard. > * A source address, used to inform the destination of where to reply. > * The address of an array of iovs containing the data to send > * .. and the length of that array of iovs > * and a 32-bit message type, available to communicate message context >data (eg. kernel-to-kernel, separate from the application data). > > If insufficient space exists in the destination ring, it will return > -EAGAIN and Xen will notify the caller when sufficient space becomes > available. > > Accesses to the ring indices are appropriately atomic. The rings are > mapped into Xen's private address space to write as needed and the > mappings are retained for later use. > > Notifications are sent to guests via VIRQ and send_guest_global_virq is > exposed in the change to enable argo to call it. VIRQ_ARGO is claimed > from the VIRQ previously reserved for this purpose (#11). > > The VIRQ notification method is used rather than sending events using > evtchn functions directly because: > > * no current event channel type is an exact fit for the intended > behaviour. ECS_IPI is closest, but it disallows migration to > other VCPUs which is not necessarily a requirement for Argo. > > * at the point of argo_init, allocation of an event channel is > complicated by none of the guest VCPUs being initialized yet > and the event channel logic expects that a valid event channel > has a present VCPU. > > * at the point of signalling a notification, the VIRQ logic is already > defensive: if d->vcpu[0] is NULL, the notification is just silently > dropped, whereas the evtchn_send logic is not so defensive: vcpu[0] > must not be NULL, otherwise a null pointer dereference occurs. > > Using a VIRQ removes the need for the guest to query to determine which > event channel notifications will be delivered on. This is also likely to > simplify establishing future L0/L1 nested hypervisor argo communication. > > Signed-off-by: Christopher Clark > Tested-by: Chris Patterson There's one style nit that I think can be fixed while committing: Reviewed-by: Roger Pau Monné Despite the usage of the open-coded mask below. As with previous patches this is argos code, so I'm not going to oppose, but again I think using such open coded masks is bad, and can lead to bugs in the code. It can be fixed by a follow up patch. > +static int > +ringbuf_insert(const struct domain *d, struct argo_ring_info *ring_info, > + const struct argo_ring_id *src_id, xen_argo_iov_t *iovs, > + unsigned int niov, uint32_t message_type, > + unsigned long *out_len) > +{ > +xen_argo_ring_t ring; > +struct xen_argo_ring_message_header mh = { }; > +int sp, ret; > +unsigned int len = 0; > +xen_argo_iov_t *piov; > +XEN_GUEST_HANDLE(uint8) NULL_hnd = { }; > + > +ASSERT(LOCKING_L3(d, ring_info)); > + > +/* > + * Obtain the total size of data to transmit -- sets the 'len' variable > + * -- and sanity check that the iovs conform to size and number limits. > + * Enforced below: no more than 'len' bytes of guest data > + * (plus the message header) will be sent in this operation. > + */ > +ret = iov_count(iovs, niov, &len); > +if ( ret ) > +return ret; > + > +/* > + * Upper bound check the message len against the ring size. > + * The message must not fill the ring; there must be at least one slot > + * remaining so we can distinguish a full ring from an empty one. > + * iov_count has already verified: len <= MAX_ARGO_MESSAGE_SIZE. > + */ > +if ( (ROUNDUP_MESSAGE(len) + sizeof(struct xen_argo_ring_message_header)) missing space ^ > +>= ring_info->len ) Align of >= also looks weird, should be aligned to the parenthesis before ROUNDUP_. > @@ -1175,6 +1766,42 @@ do_argo_op(unsigned int cmd, > XEN_GUEST_HANDLE_PARAM(void) arg1, > break; > } > > +case XEN_ARGO_OP_sendv: > +{ > +xen_argo_send_addr_t send_addr; > +xen_argo_iov_t iovs[XEN_ARGO_MAXIOV]; > +unsigned int niov; > + > +XEN_GUEST_HANDLE_PARAM(xen_argo_send_addr_t) send_addr_hnd = > +guest_handle_cast(arg1, xen_argo_send_addr_t); > +XEN_GUEST_HANDLE_PARAM(xen_argo_iov_t) iovs_hnd = > +guest_handle_cast(arg2, xen_argo_iov_t); > +/* arg3 is niov */ > +/* arg4 is message_type. Must be a 32-bit value. */ > + > +rc = copy_from_guest(&send_addr, send_addr_hnd, 1) ? -EFAULT : 0; > +if ( rc ) > +break; > + > +/* > + * Reje
[Xen-devel] [PATCH v7 09/15] argo: implement the sendv op; evtchn: expose send_guest_global_virq
sendv operation is invoked to perform a synchronous send of buffers contained in iovs to a remote domain's registered ring. It takes: * A destination address (domid, port) for the ring to send to. It performs a most-specific match lookup, to allow for wildcard. * A source address, used to inform the destination of where to reply. * The address of an array of iovs containing the data to send * .. and the length of that array of iovs * and a 32-bit message type, available to communicate message context data (eg. kernel-to-kernel, separate from the application data). If insufficient space exists in the destination ring, it will return -EAGAIN and Xen will notify the caller when sufficient space becomes available. Accesses to the ring indices are appropriately atomic. The rings are mapped into Xen's private address space to write as needed and the mappings are retained for later use. Notifications are sent to guests via VIRQ and send_guest_global_virq is exposed in the change to enable argo to call it. VIRQ_ARGO is claimed from the VIRQ previously reserved for this purpose (#11). The VIRQ notification method is used rather than sending events using evtchn functions directly because: * no current event channel type is an exact fit for the intended behaviour. ECS_IPI is closest, but it disallows migration to other VCPUs which is not necessarily a requirement for Argo. * at the point of argo_init, allocation of an event channel is complicated by none of the guest VCPUs being initialized yet and the event channel logic expects that a valid event channel has a present VCPU. * at the point of signalling a notification, the VIRQ logic is already defensive: if d->vcpu[0] is NULL, the notification is just silently dropped, whereas the evtchn_send logic is not so defensive: vcpu[0] must not be NULL, otherwise a null pointer dereference occurs. Using a VIRQ removes the need for the guest to query to determine which event channel notifications will be delivered on. This is also likely to simplify establishing future L0/L1 nested hypervisor argo communication. Signed-off-by: Christopher Clark Tested-by: Chris Patterson --- v6 #09 Jan: introduce compat ABI v6 : hoist read of iovs from guest from ringbuf_insert to do_argo_op v6 : simplify init of the NULL_hnd in ringbuf_insert v6 #09 Jan: xlat.lst remove argo_iov as generated macro not in use v6 #04 Jan: xlat.lst: move argo struct entries to alphabetical position v6 #09 Roger: fix reference to VIRQ_ARGO in commit message v6 #09 Roger: use list_for_each_entry for looping v5 #09 Roger: add comment explaining post-iovs tx_ptr round up + wrap v5 #09 Roger: remove redundant len bounds check vs MAX_ARGO_MESSAGE_SIZE v5 #09 Roger: ringbuf_insert: WARN not ERR on empty iovs v5 #09 Roger: bugfix: set rc = -EFAULT if !guest_handle_okay v5 use EBUSY when cannot add to the pending ent queue: many domains active v5: add compat validation macros to primary source file: common/argo.c v5: dropped external file for compat macros: common/compat/argo.c v5 : convert hypercall arg structs to struct form for compat checking v5 : switch argo_iov to needs translation in xlat.lst v4 Jan: remove use of fixed-width types from iov_count, ringbuf_insert v4 #07 Jan: shrink critical sections in sendv v3 #07 Jan: header: note 32-bitness of hypercall message tuype arg v4 : use standard data structures as per common code v4 self: bugfix memcpy_to_guest_ring: head_len must check (offset + len) v4 #09 Roger: drop MESSAGE from VIRQ_ARGO_MESSAGE v3 #07 Jan: rename ring_find_info* to find_ring_info* v3 #07 Jan: fix numeric entries in printk format strings v3 #10 Roger: move find functions to top of file and drop prototypes v3 #04 Jan: meld compat struct checking for hypercall args v3 #04 Roger/Jan: make lock names clearer and assert their state v3 #04 Jan: port -> aport with type; distinguish argo port from evtchn v3 feedback #09 Eric: fix len & offset sanity check in memcpy_to_guest_ring v3 feedback #04 Roger: newline fix in wildcard_pending_list_insert v3 feedback #04 Roger: drop npages struct member, calculate from len v3 #09 Roger: simplify EFAULT return in memcpy_to_guest_ring v3 #09 Roger: add newline before return in get_sanitized_ring v3 #09 Roger: replace while with for loop in iov_count v3 #09 Roger: drop 0 in struct init in ringbuf_insert v3 #09 Roger: comment for XEN_ARGO_MAXIOV: warn of stack overflow risk v3 #09 Roger: simplify while loop: for instead in ringbuf_insert v3 #09 Roger: drop out label for returns in ringbuf_insert v3 #09 Roger: drop newline in pending_queue v3 #09 Roger: replace second goto label with error path unlock in sendv v3 #09 Jason: check iov_len vs MAX_ARGO_MESSAGE_SIZE in iov_count v3 #09 Jason: check padding is zeroed in sendv op v3 #09 Jason: memcpy_to_guest_ring: simpler code with better loop v2 self: use ring_info backpointer in pending_ent to maintain npending v2 feedback Jan: drop cookie, implement teardown v2 self: pending_queue: