Re: [PATCH 2/2] libgnttab: Add support for Linux dma-buf offset
Hello, all!
I would like to bring back this old thread as it seems it has stuck long
time ago
without clear NAck or Ack. I didn't rebase the changes because the
change itself
requires answers on the way we should go here: new ioctl (seems to be
better) or
extension of the existing one (not so great)
Thank you in advance,
Oleksandr
On 01.10.20 09:35, Oleksandr Andrushchenko wrote:
> Hi,
>
> On 9/28/20 6:20 PM, Ian Jackson wrote:
>> Oleksandr Andrushchenko writes ("[PATCH 2/2] libgnttab: Add support for
>> Linux dma-buf offset"):
>>> From: Oleksandr Andrushchenko
>>>
>>> Add version 2 of the dma-buf ioctls which adds data_ofs parameter.
>>>
>>> dma-buf is backed by a scatter-gather table and has offset parameter
>>> which tells where the actual data starts. Relevant ioctls are extended
>>> to support that offset:
>>> - when dma-buf is created (exported) from grant references then
>>> data_ofs is used to set the offset field in the scatter list
>>> of the new dma-buf
>>> - when dma-buf is imported and grant references provided then
>>> data_ofs is used to report that offset to user-space
>> Thanks. I'm not a DMA expert, but I think this is probably going in
>> roughly the right direction. I will probably want a review from a DMA
>> expert too, but let me get on with my questions:
>>
>> When you say "the protocol changes are already accepted" I think you
>> mean the Linux ioctl changes ? If not, what *do* you mean ?
> I mean that the relevant protocol changes are already part of both Xen [1]
>
> and Linux trees [2]. What is missing is ioctl implementation in the kernel and
>
> its support in Xen' tools. This is why I have marked the patch as RFC in order
>
> to get some view on the matter from Xen community. Once we agree on the
>
> naming, structure etc. I'll send patches for both Xen and Linux
>
>>> +/*
>>> + * Version 2 of the ioctls adds @data_ofs parameter.
>>> + *
>>> + * dma-buf is backed by a scatter-gather table and has offset
>>> + * parameter which tells where the actual data starts.
>>> + * Relevant ioctls are extended to support that offset:
>>> + * - when dma-buf is created (exported) from grant references then
>>> + * @data_ofs is used to set the offset field in the scatter list
>>> + * of the new dma-buf
>>> + * - when dma-buf is imported and grant references are provided then
>>> + * @data_ofs is used to report that offset to user-space
>>> + */
>>> +#define IOCTL_GNTDEV_DMABUF_EXP_FROM_REFS_V2 \
>>> +_IOC(_IOC_NONE, 'G', 13, \
>> I think this was copied from a Linux header file ? If so please quote
>> the precise file and revision in the commit message.
> This is not upstream yet, please see explanation above
>> And be sure to
>> copy the copyright informtaion appropriately.
>>
>>> +int osdep_gnttab_dmabuf_exp_from_refs_v2(xengnttab_handle *xgt, uint32_t
>>> domid,
>>> + uint32_t flags, uint32_t count,
>>> + const uint32_t *refs,
>>> + uint32_t *dmabuf_fd, uint32_t
>>> data_ofs)
>>> +{
>>> +abort();
>> I'm pretty sure this is wrong.
> First of all, Linux dma-bufs are only supported on Linux, so neither FreeBSD
> nor Mini-OS
>
> will have that. If you are referring to "abort()" here, so I am just aligning
> to what previously
>
> was there, e.g. all non-relevant dma-buf OS specifics were implemented like
> that.
>
>> This leads me to ask about compatibility, both across versions of the
>> various components, and API compatibility across different platforms.
>>
>> libxengnttab is supposed to have a stable API and ABI. This means
>> that old programs should work with the new library - which I think you
>> have achieved.
> Yes
>> But I think it also means that it should work with new programs, and
>> the new library, on old kernels. What is your compatibility story
>> here ? What is the intended mode of use by an application ?
> Well, this is a tough story. If we have new software and new library, but old
>
> kernel it means that the offset we are trying to get with the new ioctl will
> be
>
> unavailable to that new software. In most cases we can use offset of 0, but
> some
>
> platforms (iMX8) use offset of 64. So, we can workaround that for most(?)
> platforms
>
> by reporting offset 0, but some platforms will fail. I am not sure if this is
> good to state that
>
> this combination of software (as described above) "will mostly work" or just
> let
>
> the system fail at run-time, by letting Linux return ENOTSUPP for the new
> ioctl.
>
> By fail I mean that the display backend may decide if to use the previous
> version of the ioctl
>
> without the offset field.
>
>> And the same application code should be useable, so far as possible,
>> across different plaatforms that support Xen.
>>
>> What fallback would be possible for application do if the v2 function
>> is not available ? I think that
Re: [PATCH 2/2] libgnttab: Add support for Linux dma-buf offset
Hi,
On 9/28/20 6:20 PM, Ian Jackson wrote:
> Oleksandr Andrushchenko writes ("[PATCH 2/2] libgnttab: Add support for Linux
> dma-buf offset"):
>> From: Oleksandr Andrushchenko
>>
>> Add version 2 of the dma-buf ioctls which adds data_ofs parameter.
>>
>> dma-buf is backed by a scatter-gather table and has offset parameter
>> which tells where the actual data starts. Relevant ioctls are extended
>> to support that offset:
>>- when dma-buf is created (exported) from grant references then
>> data_ofs is used to set the offset field in the scatter list
>> of the new dma-buf
>>- when dma-buf is imported and grant references provided then
>> data_ofs is used to report that offset to user-space
> Thanks. I'm not a DMA expert, but I think this is probably going in
> roughly the right direction. I will probably want a review from a DMA
> expert too, but let me get on with my questions:
>
> When you say "the protocol changes are already accepted" I think you
> mean the Linux ioctl changes ? If not, what *do* you mean ?
I mean that the relevant protocol changes are already part of both Xen [1]
and Linux trees [2]. What is missing is ioctl implementation in the kernel and
its support in Xen' tools. This is why I have marked the patch as RFC in order
to get some view on the matter from Xen community. Once we agree on the
naming, structure etc. I'll send patches for both Xen and Linux
>
>> +/*
>> + * Version 2 of the ioctls adds @data_ofs parameter.
>> + *
>> + * dma-buf is backed by a scatter-gather table and has offset
>> + * parameter which tells where the actual data starts.
>> + * Relevant ioctls are extended to support that offset:
>> + * - when dma-buf is created (exported) from grant references then
>> + * @data_ofs is used to set the offset field in the scatter list
>> + * of the new dma-buf
>> + * - when dma-buf is imported and grant references are provided then
>> + * @data_ofs is used to report that offset to user-space
>> + */
>> +#define IOCTL_GNTDEV_DMABUF_EXP_FROM_REFS_V2 \
>> +_IOC(_IOC_NONE, 'G', 13, \
> I think this was copied from a Linux header file ? If so please quote
> the precise file and revision in the commit message.
This is not upstream yet, please see explanation above
>And be sure to
> copy the copyright informtaion appropriately.
>
>> +int osdep_gnttab_dmabuf_exp_from_refs_v2(xengnttab_handle *xgt, uint32_t
>> domid,
>> + uint32_t flags, uint32_t count,
>> + const uint32_t *refs,
>> + uint32_t *dmabuf_fd, uint32_t
>> data_ofs)
>> +{
>> +abort();
> I'm pretty sure this is wrong.
First of all, Linux dma-bufs are only supported on Linux, so neither FreeBSD
nor Mini-OS
will have that. If you are referring to "abort()" here, so I am just aligning
to what previously
was there, e.g. all non-relevant dma-buf OS specifics were implemented like
that.
>
> This leads me to ask about compatibility, both across versions of the
> various components, and API compatibility across different platforms.
>
> libxengnttab is supposed to have a stable API and ABI. This means
> that old programs should work with the new library - which I think you
> have achieved.
Yes
>
> But I think it also means that it should work with new programs, and
> the new library, on old kernels. What is your compatibility story
> here ? What is the intended mode of use by an application ?
Well, this is a tough story. If we have new software and new library, but old
kernel it means that the offset we are trying to get with the new ioctl will be
unavailable to that new software. In most cases we can use offset of 0, but some
platforms (iMX8) use offset of 64. So, we can workaround that for most(?)
platforms
by reporting offset 0, but some platforms will fail. I am not sure if this is
good to state that
this combination of software (as described above) "will mostly work" or just let
the system fail at run-time, by letting Linux return ENOTSUPP for the new ioctl.
By fail I mean that the display backend may decide if to use the previous
version of the ioctl
without the offset field.
>
> And the same application code should be useable, so far as possible,
> across different plaatforms that support Xen.
>
> What fallback would be possible for application do if the v2 function
> is not available ? I think that fallback action needs to be
> selectable at runtime, to support new userspace on old kernels.
Well, as I said before, for the platforms with offset 0 we are "fine" ignoring
the offset and
using v1 of the ioctl without the offset field. For the platforms with non-zero
offset it results
at least in slight screen distortion and they do need v2 of the ioctl
>
> What architectures is the new Linux ioctl available on ?
x86/ARM
>
>> diff --git a/tools/libs/gnttab/include/xengnttab.h
>> b/tools/libs/gnttab/include/xengnttab.h
>>
Re: [PATCH 2/2] libgnttab: Add support for Linux dma-buf offset
Oleksandr Andrushchenko writes ("[PATCH 2/2] libgnttab: Add support for Linux
dma-buf offset"):
> From: Oleksandr Andrushchenko
>
> Add version 2 of the dma-buf ioctls which adds data_ofs parameter.
>
> dma-buf is backed by a scatter-gather table and has offset parameter
> which tells where the actual data starts. Relevant ioctls are extended
> to support that offset:
> - when dma-buf is created (exported) from grant references then
> data_ofs is used to set the offset field in the scatter list
> of the new dma-buf
> - when dma-buf is imported and grant references provided then
> data_ofs is used to report that offset to user-space
Thanks. I'm not a DMA expert, but I think this is probably going in
roughly the right direction. I will probably want a review from a DMA
expert too, but let me get on with my questions:
When you say "the protocol changes are already accepted" I think you
mean the Linux ioctl changes ? If not, what *do* you mean ?
> +/*
> + * Version 2 of the ioctls adds @data_ofs parameter.
> + *
> + * dma-buf is backed by a scatter-gather table and has offset
> + * parameter which tells where the actual data starts.
> + * Relevant ioctls are extended to support that offset:
> + * - when dma-buf is created (exported) from grant references then
> + * @data_ofs is used to set the offset field in the scatter list
> + * of the new dma-buf
> + * - when dma-buf is imported and grant references are provided then
> + * @data_ofs is used to report that offset to user-space
> + */
> +#define IOCTL_GNTDEV_DMABUF_EXP_FROM_REFS_V2 \
> +_IOC(_IOC_NONE, 'G', 13, \
I think this was copied from a Linux header file ? If so please quote
the precise file and revision in the commit message. And be sure to
copy the copyright informtaion appropriately.
> +int osdep_gnttab_dmabuf_exp_from_refs_v2(xengnttab_handle *xgt, uint32_t
> domid,
> + uint32_t flags, uint32_t count,
> + const uint32_t *refs,
> + uint32_t *dmabuf_fd, uint32_t
> data_ofs)
> +{
> +abort();
I'm pretty sure this is wrong.
This leads me to ask about compatibility, both across versions of the
various components, and API compatibility across different platforms.
libxengnttab is supposed to have a stable API and ABI. This means
that old programs should work with the new library - which I think you
have achieved.
But I think it also means that it should work with new programs, and
the new library, on old kernels. What is your compatibility story
here ? What is the intended mode of use by an application ?
And the same application code should be useable, so far as possible,
across different plaatforms that support Xen.
What fallback would be possible for application do if the v2 function
is not available ? I think that fallback action needs to be
selectable at runtime, to support new userspace on old kernels.
What architectures is the new Linux ioctl available on ?
> diff --git a/tools/libs/gnttab/include/xengnttab.h
> b/tools/libs/gnttab/include/xengnttab.h
> index 111fc88caeb3..0956bd91e0df 100644
> --- a/tools/libs/gnttab/include/xengnttab.h
> +++ b/tools/libs/gnttab/include/xengnttab.h
> @@ -322,12 +322,19 @@ int xengnttab_grant_copy(xengnttab_handle *xgt,
> * Returns 0 if dma-buf was successfully created and the corresponding
> * dma-buf's file descriptor is returned in @fd.
> *
> +
> + * Version 2 also accepts @data_ofs offset of the data in the buffer.
> + *
> * [1]
> https://elixir.bootlin.com/linux/latest/source/Documentation/driver-api/dma-buf.rst
> */
> int xengnttab_dmabuf_exp_from_refs(xengnttab_handle *xgt, uint32_t domid,
> uint32_t flags, uint32_t count,
> const uint32_t *refs, uint32_t *fd);
>
> +int xengnttab_dmabuf_exp_from_refs_v2(xengnttab_handle *xgt, uint32_t domid,
> + uint32_t flags, uint32_t count,
> + const uint32_t *refs, uint32_t *fd,
> + uint32_t data_ofs);
I think the information about the meaning of @data_ofs must be in the
doc comment. Indeed, that should be the primary location.
Conversely there is no need to duplicate information between the patch
contents, and the commit message.
Is _v2 really the best name for this ? Are we likely to want to
extend this again in future ? Perhaps it should be called ..._offset
or something ? Please think about this and tell me your opinion.
> +int osdep_gnttab_dmabuf_exp_from_refs_v2(xengnttab_handle *xgt, uint32_t
> domid,
> + uint32_t flags, uint32_t count,
> + const uint32_t *refs,
> + uint32_t *dmabuf_fd,
> + uint32_t data_ofs)
> +{
> +
Re: [PATCH 2/2] libgnttab: Add support for Linux dma-buf offset
Hello, Ian, Wei!
Initially I have sent this patch as a part of the series for the display
protocol changes,
but later decided to split. At the moment the protocol changes are already
accepted, but this part is still missing feedback and is still wanted.
I would really appreciate if you could have a look at the change below.
Thank you in advance,
Oleksandr Andrushchenko
On 5/20/20 12:04 PM, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko
>
> Add version 2 of the dma-buf ioctls which adds data_ofs parameter.
>
> dma-buf is backed by a scatter-gather table and has offset parameter
> which tells where the actual data starts. Relevant ioctls are extended
> to support that offset:
>- when dma-buf is created (exported) from grant references then
> data_ofs is used to set the offset field in the scatter list
> of the new dma-buf
>- when dma-buf is imported and grant references provided then
> data_ofs is used to report that offset to user-space
>
> Signed-off-by: Oleksandr Andrushchenko
> ---
> tools/include/xen-sys/Linux/gntdev.h | 53 +
> tools/libs/gnttab/Makefile| 2 +-
> tools/libs/gnttab/freebsd.c | 15 +
> tools/libs/gnttab/gnttab_core.c | 17 ++
> tools/libs/gnttab/include/xengnttab.h | 13
> tools/libs/gnttab/libxengnttab.map| 6 ++
> tools/libs/gnttab/linux.c | 86 +++
> tools/libs/gnttab/minios.c| 15 +
> tools/libs/gnttab/private.h | 9 +++
> 9 files changed, 215 insertions(+), 1 deletion(-)
>
> diff --git a/tools/include/xen-sys/Linux/gntdev.h
> b/tools/include/xen-sys/Linux/gntdev.h
> index d16076044c71..0c43393cbee5 100644
> --- a/tools/include/xen-sys/Linux/gntdev.h
> +++ b/tools/include/xen-sys/Linux/gntdev.h
> @@ -274,4 +274,57 @@ struct ioctl_gntdev_dmabuf_imp_release {
> uint32_t reserved;
> };
>
> +/*
> + * Version 2 of the ioctls adds @data_ofs parameter.
> + *
> + * dma-buf is backed by a scatter-gather table and has offset
> + * parameter which tells where the actual data starts.
> + * Relevant ioctls are extended to support that offset:
> + * - when dma-buf is created (exported) from grant references then
> + * @data_ofs is used to set the offset field in the scatter list
> + * of the new dma-buf
> + * - when dma-buf is imported and grant references are provided then
> + * @data_ofs is used to report that offset to user-space
> + */
> +#define IOCTL_GNTDEV_DMABUF_EXP_FROM_REFS_V2 \
> +_IOC(_IOC_NONE, 'G', 13, \
> + sizeof(struct ioctl_gntdev_dmabuf_exp_from_refs_v2))
> +struct ioctl_gntdev_dmabuf_exp_from_refs_v2 {
> +/* IN parameters. */
> +/* Specific options for this dma-buf: see GNTDEV_DMA_FLAG_XXX. */
> +uint32_t flags;
> +/* Number of grant references in @refs array. */
> +uint32_t count;
> +/* Offset of the data in the dma-buf. */
> +uint32_t data_ofs;
> +/* OUT parameters. */
> +/* File descriptor of the dma-buf. */
> +uint32_t fd;
> +/* The domain ID of the grant references to be mapped. */
> +uint32_t domid;
> +/* Variable IN parameter. */
> +/* Array of grant references of size @count. */
> +uint32_t refs[1];
> +};
> +
> +#define IOCTL_GNTDEV_DMABUF_IMP_TO_REFS_V2 \
> +_IOC(_IOC_NONE, 'G', 14, \
> + sizeof(struct ioctl_gntdev_dmabuf_imp_to_refs_v2))
> +struct ioctl_gntdev_dmabuf_imp_to_refs_v2 {
> +/* IN parameters. */
> +/* File descriptor of the dma-buf. */
> +uint32_t fd;
> +/* Number of grant references in @refs array. */
> +uint32_t count;
> +/* The domain ID for which references to be granted. */
> +uint32_t domid;
> +/* Reserved - must be zero. */
> +uint32_t reserved;
> +/* OUT parameters. */
> +/* Offset of the data in the dma-buf. */
> +uint32_t data_ofs;
> +/* Array of grant references of size @count. */
> +uint32_t refs[1];
> +};
> +
> #endif /* __LINUX_PUBLIC_GNTDEV_H__ */
> diff --git a/tools/libs/gnttab/Makefile b/tools/libs/gnttab/Makefile
> index 2da8fbbb7f6f..5ee2d965214f 100644
> --- a/tools/libs/gnttab/Makefile
> +++ b/tools/libs/gnttab/Makefile
> @@ -2,7 +2,7 @@ XEN_ROOT = $(CURDIR)/../../..
> include $(XEN_ROOT)/tools/Rules.mk
>
> MAJOR= 1
> -MINOR= 2
> +MINOR= 3
> LIBNAME := gnttab
> USELIBS := toollog toolcore
>
> diff --git a/tools/libs/gnttab/freebsd.c b/tools/libs/gnttab/freebsd.c
> index 886b588303a0..baf0f60aa4d3 100644
> --- a/tools/libs/gnttab/freebsd.c
> +++ b/tools/libs/gnttab/freebsd.c
> @@ -319,6 +319,14 @@ int osdep_gnttab_dmabuf_exp_from_refs(xengnttab_handle
> *xgt, uint32_t domid,
> abort();
> }
>
> +int osdep_gnttab_dmabuf_exp_from_refs_v2(xengnttab_handle *xgt, uint32_t
> domid,
> + uint32_t flags, uint32_t count,
> + const uint32_t *refs,
> +
Re: [PATCH 2/2] libgnttab: Add support for Linux dma-buf offset
Ian, Wei, would you mind looking at the below please?
Thank you in advance,
Oleksandr
On 5/20/20 12:04 PM, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko
>
> Add version 2 of the dma-buf ioctls which adds data_ofs parameter.
>
> dma-buf is backed by a scatter-gather table and has offset parameter
> which tells where the actual data starts. Relevant ioctls are extended
> to support that offset:
>- when dma-buf is created (exported) from grant references then
> data_ofs is used to set the offset field in the scatter list
> of the new dma-buf
>- when dma-buf is imported and grant references provided then
> data_ofs is used to report that offset to user-space
>
> Signed-off-by: Oleksandr Andrushchenko
> ---
> tools/include/xen-sys/Linux/gntdev.h | 53 +
> tools/libs/gnttab/Makefile| 2 +-
> tools/libs/gnttab/freebsd.c | 15 +
> tools/libs/gnttab/gnttab_core.c | 17 ++
> tools/libs/gnttab/include/xengnttab.h | 13
> tools/libs/gnttab/libxengnttab.map| 6 ++
> tools/libs/gnttab/linux.c | 86 +++
> tools/libs/gnttab/minios.c| 15 +
> tools/libs/gnttab/private.h | 9 +++
> 9 files changed, 215 insertions(+), 1 deletion(-)
>
> diff --git a/tools/include/xen-sys/Linux/gntdev.h
> b/tools/include/xen-sys/Linux/gntdev.h
> index d16076044c71..0c43393cbee5 100644
> --- a/tools/include/xen-sys/Linux/gntdev.h
> +++ b/tools/include/xen-sys/Linux/gntdev.h
> @@ -274,4 +274,57 @@ struct ioctl_gntdev_dmabuf_imp_release {
> uint32_t reserved;
> };
>
> +/*
> + * Version 2 of the ioctls adds @data_ofs parameter.
> + *
> + * dma-buf is backed by a scatter-gather table and has offset
> + * parameter which tells where the actual data starts.
> + * Relevant ioctls are extended to support that offset:
> + * - when dma-buf is created (exported) from grant references then
> + * @data_ofs is used to set the offset field in the scatter list
> + * of the new dma-buf
> + * - when dma-buf is imported and grant references are provided then
> + * @data_ofs is used to report that offset to user-space
> + */
> +#define IOCTL_GNTDEV_DMABUF_EXP_FROM_REFS_V2 \
> +_IOC(_IOC_NONE, 'G', 13, \
> + sizeof(struct ioctl_gntdev_dmabuf_exp_from_refs_v2))
> +struct ioctl_gntdev_dmabuf_exp_from_refs_v2 {
> +/* IN parameters. */
> +/* Specific options for this dma-buf: see GNTDEV_DMA_FLAG_XXX. */
> +uint32_t flags;
> +/* Number of grant references in @refs array. */
> +uint32_t count;
> +/* Offset of the data in the dma-buf. */
> +uint32_t data_ofs;
> +/* OUT parameters. */
> +/* File descriptor of the dma-buf. */
> +uint32_t fd;
> +/* The domain ID of the grant references to be mapped. */
> +uint32_t domid;
> +/* Variable IN parameter. */
> +/* Array of grant references of size @count. */
> +uint32_t refs[1];
> +};
> +
> +#define IOCTL_GNTDEV_DMABUF_IMP_TO_REFS_V2 \
> +_IOC(_IOC_NONE, 'G', 14, \
> + sizeof(struct ioctl_gntdev_dmabuf_imp_to_refs_v2))
> +struct ioctl_gntdev_dmabuf_imp_to_refs_v2 {
> +/* IN parameters. */
> +/* File descriptor of the dma-buf. */
> +uint32_t fd;
> +/* Number of grant references in @refs array. */
> +uint32_t count;
> +/* The domain ID for which references to be granted. */
> +uint32_t domid;
> +/* Reserved - must be zero. */
> +uint32_t reserved;
> +/* OUT parameters. */
> +/* Offset of the data in the dma-buf. */
> +uint32_t data_ofs;
> +/* Array of grant references of size @count. */
> +uint32_t refs[1];
> +};
> +
> #endif /* __LINUX_PUBLIC_GNTDEV_H__ */
> diff --git a/tools/libs/gnttab/Makefile b/tools/libs/gnttab/Makefile
> index 2da8fbbb7f6f..5ee2d965214f 100644
> --- a/tools/libs/gnttab/Makefile
> +++ b/tools/libs/gnttab/Makefile
> @@ -2,7 +2,7 @@ XEN_ROOT = $(CURDIR)/../../..
> include $(XEN_ROOT)/tools/Rules.mk
>
> MAJOR= 1
> -MINOR= 2
> +MINOR= 3
> LIBNAME := gnttab
> USELIBS := toollog toolcore
>
> diff --git a/tools/libs/gnttab/freebsd.c b/tools/libs/gnttab/freebsd.c
> index 886b588303a0..baf0f60aa4d3 100644
> --- a/tools/libs/gnttab/freebsd.c
> +++ b/tools/libs/gnttab/freebsd.c
> @@ -319,6 +319,14 @@ int osdep_gnttab_dmabuf_exp_from_refs(xengnttab_handle
> *xgt, uint32_t domid,
> abort();
> }
>
> +int osdep_gnttab_dmabuf_exp_from_refs_v2(xengnttab_handle *xgt, uint32_t
> domid,
> + uint32_t flags, uint32_t count,
> + const uint32_t *refs,
> + uint32_t *dmabuf_fd, uint32_t
> data_ofs)
> +{
> +abort();
> +}
> +
> int osdep_gnttab_dmabuf_exp_wait_released(xengnttab_handle *xgt,
> uint32_t fd, uint32_t wait_to_ms)
> {
> @@ -331,6 +339,13 @@ int
