[Xen-devel] [DRAFT v5] PV Calls protocol design document (former XenSock)
Hi all, This is the design document of the PV Calls protocol. You can find prototypes of the Linux frontend and backend drivers here: git://git.kernel.org/pub/scm/linux/kernel/git/sstabellini/xen.git pvcalls-5 To use them, make sure to enable CONFIG_PVCALLS in your kernel config and add "pvcalls=1" to the command line of your DomU Linux kernel. You also need the toolstack to create the initial xenstore nodes for the protocol. To do that, please apply the attached patch to libxl (the patch is based on Xen 4.7.0-rc3) and add "pvcalls=1" to your DomU config file. Note that previous versions of the protocols were named XenSock. It has been renamed for clarity of scope and to avoid confusion with hv_sock and vsock, which are used for inter-VMs communications. Cheers, Stefano Changes in v5: - clarify text - rename id to req_id - rename sockid to id - move id to request and response specific fields - add version node to xenstore Changes in v4: - rename xensock to pvcalls Changes in v3: - add a dummy element to struct xen_xensock_request to make sure the size of the struct is the same on both x86_32 and x86_64 Changes in v2: - add max-dataring-page-order - add "Publish backend features and transport parameters" to backend xenbus workflow - update new cmd values - update xen_xensock_request - add backlog parameter to listen and binary layout - add description of new data ring format (interface+data) - modify connect and accept to reflect new data ring format - add link to POSIX docs - add error numbers - add address format section and relevant numeric definitions - add explicit mention of unimplemented commands - add protocol node name - add xenbus shutdown diagram - add socket operation --- # PV Calls Protocol version 1 ## Rationale PV Calls is a paravirtualized protocol that allows the implementation of a set of POSIX functions in a different domain. The PV Calls frontend sends POSIX function calls to the backend, which implements them and returns a value to the frontend. This version of the document covers networking function calls, such as connect, accept, bind, release, listen, poll, recvmsg and sendmsg; but the protocol is meant to be easily extended to cover different sets of calls. Unimplemented commands return ENOTSUPP. PV Calls provide the following benefits: * full visibility of the guest behavior on the backend domain, allowing for inexpensive filtering and manipulation of any guest calls * excellent performance Specifically, PV Calls for networking offer these advantages: * guest networking works out of the box with VPNs, wireless networks and any other complex configurations on the host * guest services listen on ports bound directly to the backend domain IP addresses * localhost becomes a secure namespace for inter-VMs communications ## Design ### Xenstore The frontend and the backend connect to each other exchanging information via xenstore. The toolstack creates front and back nodes with state XenbusStateInitialising. The protocol node name is **pvcalls**. There can only be one PV Calls frontend per domain. Frontend XenBus Nodes port Values: The identifier of the Xen event channel used to signal activity in the ring buffer. ring-ref Values: The Xen grant reference granting permission for the backend to map the sole page in a single page sized ring buffer. Backend XenBus Nodes version Values: Protocol version supported by the backend. max-dataring-page-order Values: The maximum supported size of the data ring in units of lb(machine pages). (e.g. 0 == 1 page, 1 = 2 pages, 2 == 4 pages, etc.). State Machine Initialization: *Front* *Back* XenbusStateInitialising XenbusStateInitialising - Query virtual device- Query backend device properties. identification data. - Setup OS device instance. - Publish backend features - Allocate and initialize the and transport parameters request ring. | - Publish transport parameters | that will be in effect during V this connection.XenbusStateInitWait | | V XenbusStateInitialised - Query frontend transport parameters. - Connect to the request ring and event channel. | | V XenbusStateConnected - Query backend device properties. - Finalize OS virtual d
Re: [Xen-devel] [DRAFT v5] PV Calls protocol design document (former XenSock)
Ping? On Thu, 4 Aug 2016, Stefano Stabellini wrote: > Hi all, > > This is the design document of the PV Calls protocol. You can find > prototypes of the Linux frontend and backend drivers here: > > git://git.kernel.org/pub/scm/linux/kernel/git/sstabellini/xen.git pvcalls-5 > > To use them, make sure to enable CONFIG_PVCALLS in your kernel config > and add "pvcalls=1" to the command line of your DomU Linux kernel. You > also need the toolstack to create the initial xenstore nodes for the > protocol. To do that, please apply the attached patch to libxl (the > patch is based on Xen 4.7.0-rc3) and add "pvcalls=1" to your DomU config > file. > > Note that previous versions of the protocols were named XenSock. It has > been renamed for clarity of scope and to avoid confusion with hv_sock > and vsock, which are used for inter-VMs communications. > > Cheers, > > Stefano > > Changes in v5: > - clarify text > - rename id to req_id > - rename sockid to id > - move id to request and response specific fields > - add version node to xenstore > > Changes in v4: > - rename xensock to pvcalls > > Changes in v3: > - add a dummy element to struct xen_xensock_request to make sure the > size of the struct is the same on both x86_32 and x86_64 > > Changes in v2: > - add max-dataring-page-order > - add "Publish backend features and transport parameters" to backend > xenbus workflow > - update new cmd values > - update xen_xensock_request > - add backlog parameter to listen and binary layout > - add description of new data ring format (interface+data) > - modify connect and accept to reflect new data ring format > - add link to POSIX docs > - add error numbers > - add address format section and relevant numeric definitions > - add explicit mention of unimplemented commands > - add protocol node name > - add xenbus shutdown diagram > - add socket operation > > --- > > # PV Calls Protocol version 1 > > ## Rationale > > PV Calls is a paravirtualized protocol that allows the implementation of > a set of POSIX functions in a different domain. The PV Calls frontend > sends POSIX function calls to the backend, which implements them and > returns a value to the frontend. > > This version of the document covers networking function calls, such as > connect, accept, bind, release, listen, poll, recvmsg and sendmsg; but > the protocol is meant to be easily extended to cover different sets of > calls. Unimplemented commands return ENOTSUPP. > > PV Calls provide the following benefits: > * full visibility of the guest behavior on the backend domain, allowing > for inexpensive filtering and manipulation of any guest calls > * excellent performance > > Specifically, PV Calls for networking offer these advantages: > * guest networking works out of the box with VPNs, wireless networks and > any other complex configurations on the host > * guest services listen on ports bound directly to the backend domain IP > addresses > * localhost becomes a secure namespace for inter-VMs communications > > > ## Design > > ### Xenstore > > The frontend and the backend connect to each other exchanging information via > xenstore. The toolstack creates front and back nodes with state > XenbusStateInitialising. The protocol node name is **pvcalls**. There can only > be one PV Calls frontend per domain. > > Frontend XenBus Nodes > > port > Values: > > The identifier of the Xen event channel used to signal activity > in the ring buffer. > > ring-ref > Values: > > The Xen grant reference granting permission for the backend to map > the sole page in a single page sized ring buffer. > > Backend XenBus Nodes > > version > Values: > > Protocol version supported by the backend. > > max-dataring-page-order > Values: > > The maximum supported size of the data ring in units of lb(machine > pages). (e.g. 0 == 1 page, 1 = 2 pages, 2 == 4 pages, etc.). > > State Machine > > Initialization: > > *Front* *Back* > XenbusStateInitialising XenbusStateInitialising > - Query virtual device- Query backend device > properties. identification data. > - Setup OS device instance. - Publish backend features > - Allocate and initialize the and transport parameters > request ring. | > - Publish transport parameters | > that will be in effect during V > this connection.XenbusStateInitWait > | > | > V >XenbusStateInitialised > > - Query frontend transport > parameters. > - Connect to the request ring and >
Re: [Xen-devel] [DRAFT v5] PV Calls protocol design document (former XenSock)
The PV Calls (formerly XenSock) protocol design has recently attracted interest within the OpenXT software development community, as it is a novel interdomain communication mechanism and protocol. We have a longstanding and active interest in interdomain communication transport. v4v is a protocol and mechanism originally designed at Citrix for XenClient and XenClient XT, now in use by OpenXT, and has been deployed in production systems for several years at this point. It has benefitted from previous reviews by the Xen Community, and continued engagement from its original designers, and is our selection of protocol and implementation to meet the particular software requirements of our project. Members of the OpenXT community also have interdomain protocol experience through being involved in the development of both vchan and the XSM architecture. The PV Calls (formerly XenSock) work described in this thread is interesting and quite a different technology, evidently with a different set of design constraints and focus on solving a different problem: it enables the insertion of VM boundaries and separation in-between components that are communicating via the POSIX socket API, which could not otherwise be so isolated from each other. In contrast, v4v prioritizes isolation between VMs over seamless integration of existing components, preferring no memory sharing between VMs, and mandatory access control enforced on communication channels by the hypervisor. So while PV Calls is not a fit for the needs of OpenXT, this proposed approach looks very good for the problem that it aims to address. It is complimentary to the other mechanisms available in the Xen ecosystem. I would like to convey my positive support for the PV Calls proposal. Christopher Clark BAE Systems, OpenXT Project http://openxt.org On Thu, Aug 4, 2016 at 10:17 AM, Stefano Stabellini wrote: > Hi all, > > This is the design document of the PV Calls protocol. You can find > prototypes of the Linux frontend and backend drivers here: > > git://git.kernel.org/pub/scm/linux/kernel/git/sstabellini/xen.git > pvcalls-5 > > To use them, make sure to enable CONFIG_PVCALLS in your kernel config > and add "pvcalls=1" to the command line of your DomU Linux kernel. You > also need the toolstack to create the initial xenstore nodes for the > protocol. To do that, please apply the attached patch to libxl (the > patch is based on Xen 4.7.0-rc3) and add "pvcalls=1" to your DomU config > file. > > Note that previous versions of the protocols were named XenSock. It has > been renamed for clarity of scope and to avoid confusion with hv_sock > and vsock, which are used for inter-VMs communications. > > Cheers, > > Stefano > > Changes in v5: > - clarify text > - rename id to req_id > - rename sockid to id > - move id to request and response specific fields > - add version node to xenstore > > Changes in v4: > - rename xensock to pvcalls > > Changes in v3: > - add a dummy element to struct xen_xensock_request to make sure the > size of the struct is the same on both x86_32 and x86_64 > > Changes in v2: > - add max-dataring-page-order > - add "Publish backend features and transport parameters" to backend > xenbus workflow > - update new cmd values > - update xen_xensock_request > - add backlog parameter to listen and binary layout > - add description of new data ring format (interface+data) > - modify connect and accept to reflect new data ring format > - add link to POSIX docs > - add error numbers > - add address format section and relevant numeric definitions > - add explicit mention of unimplemented commands > - add protocol node name > - add xenbus shutdown diagram > - add socket operation > > --- > > # PV Calls Protocol version 1 > > ## Rationale > > PV Calls is a paravirtualized protocol that allows the implementation of > a set of POSIX functions in a different domain. The PV Calls frontend > sends POSIX function calls to the backend, which implements them and > returns a value to the frontend. > > This version of the document covers networking function calls, such as > connect, accept, bind, release, listen, poll, recvmsg and sendmsg; but > the protocol is meant to be easily extended to cover different sets of > calls. Unimplemented commands return ENOTSUPP. > > PV Calls provide the following benefits: > * full visibility of the guest behavior on the backend domain, allowing > for inexpensive filtering and manipulation of any guest calls > * excellent performance > > Specifically, PV Calls for networking offer these advantages: > * guest networking works out of the box with VPNs, wireless networks and > any other complex configurations on the host > * guest services listen on ports bound directly to the backend domain IP > addresses > * localhost becomes a secure namespace for inter-VMs communications > > > ## Design > > ### Xenstore > > The frontend and the backend connect to each other exchanging information > via > xenstore. The toolstack cre
Re: [Xen-devel] [DRAFT v5] PV Calls protocol design document (former XenSock)
Thank you! On Sun, 21 Aug 2016, Christopher Clark wrote: > The PV Calls (formerly XenSock) protocol design has recently attracted > interest within the OpenXT software > development community, as it is a novel interdomain communication mechanism > and protocol. > > We have a longstanding and active interest in interdomain communication > transport. v4v is a protocol and mechanism > originally designed at Citrix for XenClient and XenClient XT, now in use by > OpenXT, and has been deployed in > production systems for several years at this point. It has benefitted from > previous reviews by the Xen Community, > and continued engagement from its original designers, and is our selection of > protocol and implementation to meet > the particular software requirements of our project. Members of the OpenXT > community also have interdomain > protocol experience through being involved in the development of both vchan > and the XSM architecture. > > The PV Calls (formerly XenSock) work described in this thread is interesting > and quite a different technology, > evidently with a different set of design constraints and focus on solving a > different problem: it enables the > insertion of VM boundaries and separation in-between components that are > communicating via the POSIX socket API, > which could not otherwise be so isolated from each other. > > In contrast, v4v prioritizes isolation between VMs over seamless integration > of existing components, preferring no > memory sharing between VMs, and mandatory access control enforced on > communication channels by the hypervisor. > > So while PV Calls is not a fit for the needs of OpenXT, this proposed > approach looks very good for the problem > that it aims to address. It is complimentary to the other mechanisms > available in the Xen ecosystem. > > I would like to convey my positive support for the PV Calls proposal. > > Christopher Clark > BAE Systems, OpenXT Project > http://openxt.org > > > On Thu, Aug 4, 2016 at 10:17 AM, Stefano Stabellini > wrote: > Hi all, > > This is the design document of the PV Calls protocol. You can find > prototypes of the Linux frontend and backend drivers here: > > git://git.kernel.org/pub/scm/linux/kernel/git/sstabellini/xen.git > pvcalls-5 > > To use them, make sure to enable CONFIG_PVCALLS in your kernel config > and add "pvcalls=1" to the command line of your DomU Linux kernel. You > also need the toolstack to create the initial xenstore nodes for the > protocol. To do that, please apply the attached patch to libxl (the > patch is based on Xen 4.7.0-rc3) and add "pvcalls=1" to your DomU config > file. > > Note that previous versions of the protocols were named XenSock. It has > been renamed for clarity of scope and to avoid confusion with hv_sock > and vsock, which are used for inter-VMs communications. > > Cheers, > > Stefano > > Changes in v5: > - clarify text > - rename id to req_id > - rename sockid to id > - move id to request and response specific fields > - add version node to xenstore > > Changes in v4: > - rename xensock to pvcalls > > Changes in v3: > - add a dummy element to struct xen_xensock_request to make sure the > size of the struct is the same on both x86_32 and x86_64 > > Changes in v2: > - add max-dataring-page-order > - add "Publish backend features and transport parameters" to backend > xenbus workflow > - update new cmd values > - update xen_xensock_request > - add backlog parameter to listen and binary layout > - add description of new data ring format (interface+data) > - modify connect and accept to reflect new data ring format > - add link to POSIX docs > - add error numbers > - add address format section and relevant numeric definitions > - add explicit mention of unimplemented commands > - add protocol node name > - add xenbus shutdown diagram > - add socket operation > > --- > > # PV Calls Protocol version 1 > > ## Rationale > > PV Calls is a paravirtualized protocol that allows the implementation of > a set of POSIX functions in a different domain. The PV Calls frontend > sends POSIX function calls to the backend, which implements them and > returns a value to the frontend. > > This version of the document covers networking function calls, such as > connect, accept, bind, release, listen, poll, recvmsg and sendmsg; but > the protocol is meant to be easily extended to cover different sets of > calls. Unimplemented commands return ENOTSUPP. > > PV Calls provide the following benefits: > * full visibility of the guest behavior on the backend domain, allowing > for inexpensive filtering and manipulation of any guest calls >