Hello Eric, thank you for the feedback!
On Mon, Oct 11, 2021 at 4:17 PM Eric Blake <ebl...@redhat.com> wrote:
>
> On Sat, Oct 09, 2021 at 04:56:11AM -0300, Leonardo Bras wrote:
> > Adds io_async_writev and io_async_flush as optional callback to
> > QIOChannelClass,
>
> Are these names accurate?
I am sorry, I think I missed some renaming before generating the patchset.
As you suggested those names are incorrect (as they reflect a previous
naming used in v3).
I will replace them for io_{writev,flush}_zerocopy in v5.
>
> > allowing the implementation of asynchronous writes by subclasses.
> >
> > How to use them:
> > - Write data using qio_channel_writev_zerocopu(),
>
> s/copu/copy/
Thanks, I will fix that.
>
> > - Wait write completion with qio_channel_flush_zerocopy().
> >
> > Notes:
> > As some zerocopy implementations work asynchronously, it's
> > recommended to keep the write buffer untouched until the return of
> > qio_channel_flush_zerocopy(), by the risk of sending an updated buffer
>
> s/by/to avoid/
>
> > instead of the one at the write.
> >
> > As the new callbacks are optional, if a subclass does not implement them,
> > then:
> > - io_async_writev will return -1,
> > - io_async_flush will return 0 without changing anything.
>
> Are these names accurate?
They are not, I will replace them for io_{writev,flush}_zerocopy in v5.
>
> >
> > Also, some functions like qio_channel_writev_full_all() were adapted to
> > receive a flag parameter. That allows shared code between zerocopy and
> > non-zerocopy writev.
> >
> > Signed-off-by: Leonardo Bras <leob...@redhat.com>
> > ---
> > include/io/channel.h | 103 +++++++++++++++++++++++++++++++++++--------
> > io/channel.c | 74 +++++++++++++++++++++++--------
> > 2 files changed, 141 insertions(+), 36 deletions(-)
> >
> > diff --git a/include/io/channel.h b/include/io/channel.h
> > index 88988979f8..e7d4e1521f 100644
> > --- a/include/io/channel.h
> > +++ b/include/io/channel.h
> > @@ -32,12 +32,15 @@ OBJECT_DECLARE_TYPE(QIOChannel, QIOChannelClass,
> >
> > #define QIO_CHANNEL_ERR_BLOCK -2
> >
> > +#define QIO_CHANNEL_WRITE_FLAG_ZEROCOPY 0x1
> > +
> > typedef enum QIOChannelFeature QIOChannelFeature;
> >
> > enum QIOChannelFeature {
> > QIO_CHANNEL_FEATURE_FD_PASS,
> > QIO_CHANNEL_FEATURE_SHUTDOWN,
> > QIO_CHANNEL_FEATURE_LISTEN,
> > + QIO_CHANNEL_FEATURE_WRITE_ZEROCOPY,
> > };
> >
> >
> > @@ -136,6 +139,12 @@ struct QIOChannelClass {
> > IOHandler *io_read,
> > IOHandler *io_write,
> > void *opaque);
> > + ssize_t (*io_writev_zerocopy)(QIOChannel *ioc,
> > + const struct iovec *iov,
> > + size_t niov,
> > + Error **errp);
> > + int (*io_flush_zerocopy)(QIOChannel *ioc,
> > + Error **errp);
>
> Indentation is off by one.
Thanks, I will fix that for v5.
>
> > };
> >
> > /* General I/O handling functions */
> > @@ -222,12 +231,13 @@ ssize_t qio_channel_readv_full(QIOChannel *ioc,
> >
> >
> > /**
> > - * qio_channel_writev_full:
> > + * qio_channel_writev_full_flags:
> > * @ioc: the channel object
> > * @iov: the array of memory regions to write data from
> > * @niov: the length of the @iov array
> > * @fds: an array of file handles to send
> > * @nfds: number of file handles in @fds
> > + * @flags: write flags (QIO_CHANNEL_WRITE_FLAG_*)
> > * @errp: pointer to a NULL-initialized error object
> > *
> > * Write data to the IO channel, reading it from the
> > @@ -242,6 +252,10 @@ ssize_t qio_channel_readv_full(QIOChannel *ioc,
> > * guaranteed. If the channel is non-blocking and no
> > * data can be sent, it will return QIO_CHANNEL_ERR_BLOCK
> > *
> > + * If flag QIO_CHANNEL_WRITE_FLAG_ZEROCOPY is passed,
> > + * function will return once each buffer was queued for
> > + * sending.
>
> This would be a good place to document the requirement to keep the
> buffer unchanged until the zerocopy sequence completes.
That makes sense, even though that may be true for just some implementations,
it makes sense to document it here.
>
> > Error **errp);
> >
> > /**
> > - * qio_channel_writev_full_all:
> > + * qio_channel_writev_full_all_flags:
> > * @ioc: the channel object
> > * @iov: the array of memory regions to write data from
> > * @niov: the length of the @iov array
> > * @fds: an array of file handles to send
> > * @nfds: number of file handles in @fds
> > + * @flags: write flags (QIO_CHANNEL_WRITE_FLAG_*)
> > * @errp: pointer to a NULL-initialized error object
> > *
> > *
> > @@ -846,13 +868,58 @@ int qio_channel_readv_full_all(QIOChannel *ioc,
> > * to be written, yielding from the current coroutine
> > * if required.
> > *
> > + * If QIO_CHANNEL_WRITE_FLAG_ZEROCOPY is passed in flags,
> > + * instead of waiting for all requested data to be written,
> > + * this function will wait until it's all queued for writing.
>
> Another good place to document restrictions on buffer stability.
Ok :)
>
> > + *
> > * Returns: 0 if all bytes were written, or -1 on error
> > */
> >
> > -int qio_channel_writev_full_all(QIOChannel *ioc,
> > - const struct iovec *iov,
> > - size_t niov,
> > - int *fds, size_t nfds,
> > - Error **errp);
> > +int qio_channel_writev_full_all_flags(QIOChannel *ioc,
> > + const struct iovec *iov,
> > + size_t niov,
> > + int *fds, size_t nfds,
> > + int flags, Error **errp);
> > +#define qio_channel_writev_full_all(ioc, iov, niov, fds, nfds, errp) \
> > + qio_channel_writev_full_all_flags(ioc, iov, niov, fds, nfds, 0, errp)
> > +
> > +/**
> > + * qio_channel_writev_zerocopy:
> > + * @ioc: the channel object
> > + * @iov: the array of memory regions to write data from
> > + * @niov: the length of the @iov array
> > + * @errp: pointer to a NULL-initialized error object
> > + *
> > + * Behaves like qio_channel_writev_full_all_flags, but will write
> > + * data asynchronously while avoiding unnecessary data copy.
> > + * This function may return before any data is actually written,
> > + * but should queue every buffer for writting.
>
> writing
>
> Another place to document buffer stability considerations.
Ok,
Is it enough to document it in a single one of the places suggested, or
would you recommend documenting it in all suggested places?
>
> > + *
> > + * If at some point it's necessary wait for all data to be
>
> s/wait/to wait/
I will fix that for v5, thanks!
>
> > + * written, use qio_channel_flush_zerocopy().
> > + *
> > + * If zerocopy is not available, returns -1 and set errp.
> > + */
> > +
> > +ssize_t qio_channel_writev_zerocopy(QIOChannel *ioc,
> > + const struct iovec *iov,
> > + size_t niov,
> > + Error **errp);
> > +
> > +/**
> > + * qio_channel_flush_zerocopy:
> > + * @ioc: the channel object
> > + * @errp: pointer to a NULL-initialized error object
> > + *
> > + * Will lock until every packet queued with
>
> s/lock/block/
Yeah, I should have fixed it in v4.
Thanks for pointing this out.
>
> > + * qio_channel_writev_zerocopy() is sent, or return
> > + * in case of any error.
> > + *
> > + * Returns -1 if any error is found, 0 otherwise.
> > + * If not implemented, returns 0 without changing anything.
> > + */
> > +
> > +int qio_channel_flush_zerocopy(QIOChannel *ioc,
> > + Error **errp);
> >
> > #endif /* QIO_CHANNEL_H */
> > diff --git a/io/channel.c b/io/channel.c
> > index e8b019dc36..811c93ae23 100644
> > --- a/io/channel.c
> > +++ b/io/channel.c
>
> > +int qio_channel_flush_zerocopy(QIOChannel *ioc,
> > + Error **errp)
> > +{
> > + QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc);
> > +
> > + if (!klass->io_flush_zerocopy ||
> > + !qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_WRITE_ZEROCOPY))
> > {
> > + return 0;
>
> Matches your documentation, but an ideal app should not be trying to
> flush if the write failed in the first place. So wouldn't it be
> better to return -1 or even abort() on a coding error?
The point here is that any valid user of zrocopy_flush would have
already used zerocopy_writev
at some point, and failed if not supported / enabled.
Having this not returning error can help the user keep a simpler
approach when using
a setup in which the writev can happen in both zerocopy or default behavior.
I mean, the user will not need to check if zerocopy was or was not
enabled, and just flush anyway.
But if it's not good behavior, or you guys think it's a better
approach to fail here, I can also do that.
>
> > + }
> > +
> > + return klass->io_flush_zerocopy(ioc, errp);
> > +}
> > +
> > +
> > static void qio_channel_restart_read(void *opaque)
> > {
> > QIOChannel *ioc = opaque;
> > --
> > 2.33.0
> >
>
> --
> Eric Blake, Principal Software Engineer
> Red Hat, Inc. +1-919-301-3266
> Virtualization: qemu.org | libvirt.org
>
Thank you for reviewing!
Best regards,
Leo
