Re: [libvirt] [PATCH v3 09/31] Introduce virStreamSparseSendAll

[John Ferlan]

On 05/16/2017 10:03 AM, Michal Privoznik wrote:
> This is just a wrapper over new function that have been just
> introduced: virStreamSendHole() . It's very similar to
> virStreamSendAll() except it handles sparse streams well.
> 
> Signed-off-by: Michal Privoznik 
> ---
>  include/libvirt/libvirt-stream.h |  65 +++-
>  src/libvirt-stream.c | 159 
> +++
>  src/libvirt_public.syms  |   1 +
>  3 files changed, 222 insertions(+), 3 deletions(-)
> 

[...]

> diff --git a/src/libvirt-stream.c b/src/libvirt-stream.c
> index 6bf4c4f29..4cbe5eee1 100644
> --- a/src/libvirt-stream.c
> +++ b/src/libvirt-stream.c
> @@ -574,6 +574,165 @@ virStreamSendAll(virStreamPtr stream,
>  }
>  
>  
> +/**
> + * virStreamSparseSendAll:
> + * @stream: pointer to the stream object
> + * @handler: source callback for reading data from application
> + * @holeHandler: source callback for determining holes
> + * @skipHandler: skip holes as reported by @holeHandler
> + * @opaque: application defined data
> + *
> + * Send the entire data stream, reading the data from the
> + * requested data source. This is simply a convenient alternative
> + * to virStreamSend, for apps that do blocking-I/O.
> + *
> + * An example using this with a hypothetical file upload
> + * API looks like
> + *
> + *   int mysource(virStreamPtr st, char *buf, int nbytes, void *opaque) {
> + *   int *fd = opaque;
> + *
> + *   return read(*fd, buf, nbytes);
> + *   }
> + *
> + *   int myskip(virStreamPtr st, long long offset, void *opaque) {
> + *   int *fd = opaque;
> + *
> + *   return lseek(*fd, offset, SEEK_CUR) == (off_t) -1 ? -1 : 0;
> + *   }

Similar notations here regarding "long long" values that are being used
in/for lseek() which expects "off_t"...

> + *
> + *   int myindata(virStreamPtr st, int *inData,
> + *long long *offset, void *opaque) {
> + *   int *fd = opaque;
> + *
> + *   if (@fd in hole) {
> + *   *inData = 0;
> + *   *offset = holeSize;
> + *   } else {
> + *   *inData = 1;
> + *   *offset = dataSize;
> + *   }
> + *
> + *   return 0;
> + *   }
> + *
> + *   virStreamPtr st = virStreamNew(conn, 0);
> + *   int fd = open("demo.iso", O_RDONLY);
> + *
> + *   virConnectUploadFile(conn, st);

^^ This doesn't exist either and is a straight copy from
virStreamSendAll... Should it also have the "Sparse" though?

Reviewed-by: John Ferlan 


John

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH v3 09/31] Introduce virStreamSparseSendAll

[Michal Privoznik]

This is just a wrapper over new function that have been just
introduced: virStreamSendHole() . It's very similar to
virStreamSendAll() except it handles sparse streams well.

Signed-off-by: Michal Privoznik 
---
 include/libvirt/libvirt-stream.h |  65 +++-
 src/libvirt-stream.c | 159 +++
 src/libvirt_public.syms  |   1 +
 3 files changed, 222 insertions(+), 3 deletions(-)

diff --git a/include/libvirt/libvirt-stream.h b/include/libvirt/libvirt-stream.h
index a5e69a1c1..d18d43140 100644
--- a/include/libvirt/libvirt-stream.h
+++ b/include/libvirt/libvirt-stream.h
@@ -71,9 +71,9 @@ int virStreamRecvHole(virStreamPtr,
  * @nbytes: size of the data array
  * @opaque: optional application provided data
  *
- * The virStreamSourceFunc callback is used together
- * with the virStreamSendAll function for libvirt to
- * obtain the data that is to be sent.
+ * The virStreamSourceFunc callback is used together with
+ * the virStreamSendAll and virStreamSparseSendAll functions
+ * for libvirt to obtain the data that is to be sent.
  *
  * The callback will be invoked multiple times,
  * fetching data in small chunks. The application
@@ -96,6 +96,65 @@ int virStreamSendAll(virStreamPtr st,
  virStreamSourceFunc handler,
  void *opaque);
 
+/**
+ * virStreamSourceHoleFunc:
+ * @st: the stream object
+ * @inData: are we in data section
+ * @length: how long is the section we are currently in
+ * @opaque: optional application provided data
+ *
+ * The virStreamSourceHoleFunc callback is used together with the
+ * virStreamSparseSendAll function for libvirt to obtain the
+ * length of section stream is currently in.
+ *
+ * Moreover, upon successful return, @length should be updated
+ * with how many bytes are left until the current section ends
+ * (either data section or hole section). Also the stream is
+ * currently in data section, @inData should be set to a non-zero
+ * value and vice versa.
+ *
+ * NB: there's an implicit hole at the end of each file. If
+ * that's the case, @inData and @length should be both set to 0.
+ *
+ * This function should not adjust the current position within
+ * the file.
+ *
+ * Returns 0 on success,
+ *-1 upon error
+ */
+typedef int (*virStreamSourceHoleFunc)(virStreamPtr st,
+   int *inData,
+   long long *length,
+   void *opaque);
+
+/**
+ * virStreamSourceSkipFunc:
+ * @st: the stream object
+ * @length: stream hole size
+ * @opaque: optional application provided data
+ *
+ * This callback is used together with the virStreamSparseSendAll
+ * to skip holes in the underlying file as reported by
+ * virStreamSourceHoleFunc.
+ *
+ * The callback may be invoked multiple times as holes are found
+ * during processing a stream. The application should skip
+ * processing the hole in the stream source and then return.
+ * A return value of -1 at any time will abort the send operation.
+ *
+ * Returns 0 on success,
+ *-1 upon error.
+ */
+typedef int (*virStreamSourceSkipFunc)(virStreamPtr st,
+   long long length,
+   void *opaque);
+
+int virStreamSparseSendAll(virStreamPtr st,
+   virStreamSourceFunc handler,
+   virStreamSourceHoleFunc holeHandler,
+   virStreamSourceSkipFunc skipHandler,
+   void *opaque);
+
 /**
  * virStreamSinkFunc:
  *
diff --git a/src/libvirt-stream.c b/src/libvirt-stream.c
index 6bf4c4f29..4cbe5eee1 100644
--- a/src/libvirt-stream.c
+++ b/src/libvirt-stream.c
@@ -574,6 +574,165 @@ virStreamSendAll(virStreamPtr stream,
 }
 
 
+/**
+ * virStreamSparseSendAll:
+ * @stream: pointer to the stream object
+ * @handler: source callback for reading data from application
+ * @holeHandler: source callback for determining holes
+ * @skipHandler: skip holes as reported by @holeHandler
+ * @opaque: application defined data
+ *
+ * Send the entire data stream, reading the data from the
+ * requested data source. This is simply a convenient alternative
+ * to virStreamSend, for apps that do blocking-I/O.
+ *
+ * An example using this with a hypothetical file upload
+ * API looks like
+ *
+ *   int mysource(virStreamPtr st, char *buf, int nbytes, void *opaque) {
+ *   int *fd = opaque;
+ *
+ *   return read(*fd, buf, nbytes);
+ *   }
+ *
+ *   int myskip(virStreamPtr st, long long offset, void *opaque) {
+ *   int *fd = opaque;
+ *
+ *   return lseek(*fd, offset, SEEK_CUR) == (off_t) -1 ? -1 : 0;
+ *   }
+ *
+ *   int myindata(virStreamPtr st, int *inData,
+ *long long *offset, void *opaque) {
+ *   int *fd = opaque;
+ *
+ *   if (@fd in hole) {
+ *   *inData = 0;
+ *   *offset = holeSize;
+ *   } else {
+ *