Re: [Qemu-devel] [PATCH v3 0/9] Introduce I/O channels framework
On 11/12/2015 11:53, Daniel P. Berrange wrote: > Ping, unless anyone wants to review this further I'll send a pull > request for it once 2.6 opens up, so I can focus on getting the > other dependant patch series reviewed & merged in 2.6 too. Yes, please do. Paolo
Re: [Qemu-devel] [PATCH v3 0/9] Introduce I/O channels framework
Ping, unless anyone wants to review this further I'll send a pull
request for it once 2.6 opens up, so I can focus on getting the
other dependant patch series reviewed & merged in 2.6 too.
Regards,
Daniel
On Wed, Nov 18, 2015 at 06:42:43PM +, Daniel P. Berrange wrote:
> This series of patches is a followup submission for review of another
> chunk of my large series supporting TLS across chardevs, VNC and
> migration, previously shown here:
>
> RFC: https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00829.html
> v1: https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg04853.html
> v2: https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg03439.html
>
> In this series, I have provided only the general I/O channels
> framework, which will ultimately be used by VNC, chardev and
> migration code, which currently work as follows:
>
> - VNC: uses raw sockets APIs directly, layering in TLS, SASL
>and websockets where needed. This has resulted in what should
>be fairly generic code for TLS and websockets being entangled
>with the rest of the VNC server code.
>
> - Chardevs: uses GLib's GIOChannel framework. This only provides
>implementations for sockets and files. Its API lacks support for
>using struct iovec for read/writes, file descriptor passing,
>misc sockets ioctls/fcntls. While you can work around these
>problems by accessing the underling file descriptor directly,
>this breaks the encapsulation, requiring callers to know about
>specific implementations. It also does not have integration
>with QEMU Error reporting framework. So while the GIOChannel
>is extensible, extending it would result in throwing away
>pretty much the entire of the existing implementations
>
> - Migration: uses QEMUFile framework. The provides an abstract
>interface for I/O channels used during migration. It has
>impls for sockets, files, memory buffers & commands. While
>it has struct iovec support for writes, it does not have
>the same for reads. It also doesn't support file descriptor
>passing or control of the sockets ioctls/fcntls. It also
>does not have any explicit event loop integration, expecting
>the callers to directly access the underling file desctriptor
>and setup events on that. This limits its utility in cases
>where you need channels which are not backed by file
>descriptors. It has no integration with QEMU Error object
>for error reporting, has fixed semantics wrt writes
>ie must always do a full write, no partial writes, and
>most strangely forces either input or output mode, but
>refuses to allow both, so no bi-directional channels!
>
> Out of all of these, the migration code is probably closest
> to what is needed, but is still a good way off from being a
> generic framework that be can reused outside of the migration
> code.
>
> There is also the GLib GIO library which provides a generic
> framework, but we can't depend on that due to the minimum
> GLib requirement. It also has various missing pieces such as
> file descriptor passing, and no support for struct iovec
> either.
>
> Hence, this series was born, which tries to take the best of
> the designs for the GIOChannel, QIO and QEMUFile code to
> provide QIOChannel. Right from the start this new framework
> is explicitly isolated from any other QEMU subsystem, so its
> implementation will not get mixed up with specific use cases.
>
> The QIOChannel abstract base class defines the overall API
> contract
>
> - qio_channel_{write,writev,write_full} for writing data. The
>underling impl always uses struct iovec, but non-iov
>APIs are provided as simple wrappers for convenience
>
> - qio_channel_{read,readv,read_full} for reading data. The
>underling impl always uses struct iovec, but non-iov
>APIs are provided as simple wrappers for convenience
>
> - qio_channel_has_feature to determine which optional
>features the channel supports - eg file descriptor
>passing, nagle, etc
>
> - qio_channel_set_{blocking,delay,cork} for various fcntl
>and ioctl controls
>
> - qio_channel_{close,shutdown} for closing the I/O stream
>or individual channels
>
> - qio_channel_seek for random access channels
>
> - qio_channel_{add,create}_watch for integration into the
>main loop for event notifications
>
> - qio_channel_wait for blocking of execution pending an
>event notification
>
> - qio_channel_yield for switching coroutine until an
>event notification
>
> All the APIs support Error ** object arguments where needed.
> The object is built using QOM, in order to get reference
> counting and sub-classing with type checking. They are *not*
> user creatable/visible objects though - these are internal
> infrastructure, so we will be free to adapt APIs/objects at
> will over time.
>
> In this series there are a variety of implementations. Some
> of them provide an actual base layer data
[Qemu-devel] [PATCH v3 0/9] Introduce I/O channels framework
This series of patches is a followup submission for review of another
chunk of my large series supporting TLS across chardevs, VNC and
migration, previously shown here:
RFC: https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00829.html
v1: https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg04853.html
v2: https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg03439.html
In this series, I have provided only the general I/O channels
framework, which will ultimately be used by VNC, chardev and
migration code, which currently work as follows:
- VNC: uses raw sockets APIs directly, layering in TLS, SASL
and websockets where needed. This has resulted in what should
be fairly generic code for TLS and websockets being entangled
with the rest of the VNC server code.
- Chardevs: uses GLib's GIOChannel framework. This only provides
implementations for sockets and files. Its API lacks support for
using struct iovec for read/writes, file descriptor passing,
misc sockets ioctls/fcntls. While you can work around these
problems by accessing the underling file descriptor directly,
this breaks the encapsulation, requiring callers to know about
specific implementations. It also does not have integration
with QEMU Error reporting framework. So while the GIOChannel
is extensible, extending it would result in throwing away
pretty much the entire of the existing implementations
- Migration: uses QEMUFile framework. The provides an abstract
interface for I/O channels used during migration. It has
impls for sockets, files, memory buffers & commands. While
it has struct iovec support for writes, it does not have
the same for reads. It also doesn't support file descriptor
passing or control of the sockets ioctls/fcntls. It also
does not have any explicit event loop integration, expecting
the callers to directly access the underling file desctriptor
and setup events on that. This limits its utility in cases
where you need channels which are not backed by file
descriptors. It has no integration with QEMU Error object
for error reporting, has fixed semantics wrt writes
ie must always do a full write, no partial writes, and
most strangely forces either input or output mode, but
refuses to allow both, so no bi-directional channels!
Out of all of these, the migration code is probably closest
to what is needed, but is still a good way off from being a
generic framework that be can reused outside of the migration
code.
There is also the GLib GIO library which provides a generic
framework, but we can't depend on that due to the minimum
GLib requirement. It also has various missing pieces such as
file descriptor passing, and no support for struct iovec
either.
Hence, this series was born, which tries to take the best of
the designs for the GIOChannel, QIO and QEMUFile code to
provide QIOChannel. Right from the start this new framework
is explicitly isolated from any other QEMU subsystem, so its
implementation will not get mixed up with specific use cases.
The QIOChannel abstract base class defines the overall API
contract
- qio_channel_{write,writev,write_full} for writing data. The
underling impl always uses struct iovec, but non-iov
APIs are provided as simple wrappers for convenience
- qio_channel_{read,readv,read_full} for reading data. The
underling impl always uses struct iovec, but non-iov
APIs are provided as simple wrappers for convenience
- qio_channel_has_feature to determine which optional
features the channel supports - eg file descriptor
passing, nagle, etc
- qio_channel_set_{blocking,delay,cork} for various fcntl
and ioctl controls
- qio_channel_{close,shutdown} for closing the I/O stream
or individual channels
- qio_channel_seek for random access channels
- qio_channel_{add,create}_watch for integration into the
main loop for event notifications
- qio_channel_wait for blocking of execution pending an
event notification
- qio_channel_yield for switching coroutine until an
event notification
All the APIs support Error ** object arguments where needed.
The object is built using QOM, in order to get reference
counting and sub-classing with type checking. They are *not*
user creatable/visible objects though - these are internal
infrastructure, so we will be free to adapt APIs/objects at
will over time.
In this series there are a variety of implementations. Some
of them provide an actual base layer data transport, while
others provide a data translation layer:
In this series there are a variety of implementations. Some
of them provide an actual base layer data transport, while
others provide a data translation layer:
- QIOChannelSocket - for socket based I/O, IPv4, IPV6 & UNIX,
both stream and datagram.
- QIOChannelFile - any non-socket file, plain file, char dev,
fifo, pipe, etc
- QIOChannelTLS - data translation layer to
