Re: [Xen-devel] xen/mem-reservation API and out-of-tree kernel modules

[Oleksandr Andrushchenko]

On 2/1/19 10:27 AM, Christoph Hellwig wrote:
> On Thu, Jan 31, 2019 at 01:44:15PM -0800, Stefano Stabellini wrote:
>> The alternative would be to turn xenmem_reservation_scrub_page into a
>> regular function (not a static inline)?
> All that is a moot point until said currently out of tree module gets
> submitted for inclusion anyway.
Indeed this is a moot point, so I can't argue here.
But this is how it is and unfortunately we have to live
with those modules and depend on 3rd parties willing or not
to disclose their sources to public...
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] xen/mem-reservation API and out-of-tree kernel modules

[Oleksandr Andrushchenko]

Hello,

I am working on porting an out-of-tree kernel driver to the kernel
5.0 and that driver uses functionality provided by 
drivers/xen/mem-reservation.c
module.  Since commit [1] it is not possible to build a kernel module
which uses mem-reservation API as xen_scrub_pages variable, which is 
checked in
xenmem_reservation_scrub_page, became a kernel module parameter and is 
now only
accessible for built-in modules:

static inline void xenmem_reservation_scrub_page(struct page *page)
^
{
     if (xen_scrub_pages)
     ^^^
         clear_highpage(page);
}

This results in link-time warning:

     WARNING: "xen_scrub_pages" [yourmodule.ko] undefined!

and thus not allowing the module to run. At the moment I can only see a 
possible fix
for this by making the following change:

diff --git a/drivers/xen/mem-reservation.c b/drivers/xen/mem-reservation.c
index 3782cf070338..85fecfec50e1 100644
--- a/drivers/xen/mem-reservation.c
+++ b/drivers/xen/mem-reservation.c
@@ -18,6 +18,7 @@

  bool __read_mostly xen_scrub_pages = 
IS_ENABLED(CONFIG_XEN_SCRUB_PAGES_DEFAULT);
  core_param(xen_scrub_pages, xen_scrub_pages, bool, 0);
+EXPORT_SYMBOL(xen_scrub_pages);

but this looks a bit unusual for the kernel?

I am looking for community advice here and help

Thank you,
Oleksandr

[1] 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=197ecb3802c04499d8ff4f8cb28f6efa008067db
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 0/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

Konrad, could you please review?

Thank you,
Oleksandr

On 1/15/19 11:38 AM, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko 
>
> Hello!
>
> At the moment Xen [1] already supports some virtual multimedia
> features [2] such as virtual display, sound. It supports keyboards,
> pointers and multi-touch devices all allowing Xen to be used in
> automotive appliances, In-Vehicle Infotainment (IVI) systems
> and many more.
>
> Frontend implementation is available at [3] and the corresponding
> backend at [4]. These are work in progress, but frontend already
> passes v4l2-compliance test for V4L2 drivers. libxl preliminary
> changes are available at [5].
>
> This work adds a new Xen para-virtualized protocol for a virtual
> camera device which extends multimedia capabilities of Xen even
> farther: video conferencing, IVI, high definition maps etc.
>
> The initial goal is to support most needed functionality with the
> final idea to make it possible to extend the protocol if need be:
>
> 1. Provide means for base virtual device configuration:
>   - pixel formats
>   - resolutions
>   - frame rates
> 2. Support basic camera controls:
>   - contrast
>   - brightness
>   - hue
>   - saturation
> 3. Support streaming control
>
> I would like to thank Hans Verkuil  for valuable
> comments and help.
>
> Thank you,
> Oleksandr Andrushchenko
>
> Changes since v3:
> =
>
> 1. Add trimming example for short FOURCC labels, e.g. Y16 and Y16-BE
> 2. Remove from XENCAMERA_OP_CONFIG_XXX requests colorspace, xfer_func,
> ycbcr_enc, quantization and move those into the corresponding response
> 3. Extend description of XENCAMERA_OP_BUF_REQUEST.num_bufs: limit to
> maximum buffers and num_bufs == 0 case
> 4. Extend decription of XENCAMERA_OP_BUF_CREATE.index and specify its
> range
> 5. Make XENCAMERA_EVT_FRAME_AVAIL.seq_num 32-bit instead of 64-bit
>
> Changes since v2:
> =
>
> 1. Add "max-buffers" frontend configuration entry, e.g.
> the maximum number of camera buffers a frontend may use.
> 2. Add big-endian pixel-format support:
>   - "formats" configuration string length changed from 4 to 7
> octets, so we can also manage BE pixel-formats
>   - add corresponding comments to FOURCC mappings description
> 3. New commands added to the protocol and documented:
>   - XENCAMERA_OP_CONFIG_VALIDATE
>   - XENCAMERA_OP_FRAME_RATE_SET
>   - XENCAMERA_OP_BUF_GET_LAYOUT
> 4.-Add defaults for colorspace, xfer, ycbcr_enc and quantization
> 5. Remove XENCAMERA_EVT_CONFIG_CHANGE event
> 6. Move plane offsets to XENCAMERA_OP_BUF_REQUEST as offsets
> required for the frontend might not be known at the configuration time
> 7. Clean up and address comments to v2 of the protocol
>
> Changes since v1:
> =
>
> 1. Added XenStore entries:
>   - frame-rates
> 2. Do not require the FOURCC code in XenStore to be upper case only
> 3. Added/changed command set:
>   - configuration get/set
>   - buffer queue/dequeue
>   - control get
> 4. Added control flags, e.g. read-only etc.
> 5. Added colorspace configuration support, relevant constants
> 6. Added events:
>   - configuration change
>   - control change
> 7. Changed control values to 64-bit
> 8. Added sequence number to frame avail event
> 9. Coding style cleanup
>
> [1] https://www.xenproject.org/
> [2] https://xenbits.xen.org/gitweb/?p=xen.git;a=tree;f=xen/include/public/io
> [3] https://github.com/andr2000/linux/tree/camera_front_v1/drivers/media/xen
> [4] https://github.com/andr2000/camera_be
> [5] https://github.com/andr2000/xen/tree/vcamera
>
> Oleksandr Andrushchenko (1):
>cameraif: add ABI for para-virtual camera
>
>   xen/include/public/io/cameraif.h | 1364 ++
>   1 file changed, 1364 insertions(+)
>   create mode 100644 xen/include/public/io/cameraif.h
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

Dropped in favor of https://lkml.org/lkml/2019/1/29/910

On 1/24/19 5:02 PM, Julien Grall wrote:
>
>
> On 24/01/2019 14:34, Oleksandr Andrushchenko wrote:
>> Hello, Julien!
>
> Hi,
>
>> On 1/22/19 1:44 PM, Julien Grall wrote:
>>>
>>>
>>> On 1/22/19 10:28 AM, Oleksandr Andrushchenko wrote:
>>>> Hello, Julien!
>>>
>>> Hi,
>>>
>>>> On 1/21/19 7:09 PM, Julien Grall wrote:
>>>> Well, I didn't get the attributes of pages at the backend side, but 
>>>> IMO
>>>> those
>>>> do not matter in my use-case (for simplicity I am not using
>>>> zero-copying at
>>>> backend side):
>>>
>>> They are actually important no matter what is your use case. If you
>>> access the same physical page with different attributes, then you are
>>> asking for trouble.
>> So, we have:
>>
>> DomU: frontend side
>> 
>> !PTE_RDONLY + PTE_PXN + PTE_SHARED + PTE_AF + PTE_UXN +
>> PTE_ATTRINDX(MT_NORMAL)
>
> I still don't understand how you came up with MT_NORMAL when you seem 
> to confirm...
>
>>
>> DomD: backend side
>> 
>> PTE_USER + !PTE_RDONLY + PTE_PXN + PTE_NG + PTE_CONT + PTE_TABLE_BIT +
>> PTE_UXN + PTE_ATTRINDX(MT_NORMAL)
>>
>>   From the above it seems that I don't violate cached/non-cached
>> agreement here
>>>
>>> This is why Xen imposes all the pages shared to have their memory
>>> attributes following some rules. Actually, speaking with Mark R., we
>>> may want to tight a bit more the attributes.
>>>
>>>>
>>>> 1. Frontend device allocates display buffer pages which come from 
>>>> shmem
>>>> and have these attributes:
>>>> !PTE_RDONLY + PTE_PXN + PTE_SHARED + PTE_AF + PTE_UXN +
>>>> PTE_ATTRINDX(MT_NORMAL)
>>>
>>> My knowledge of Xen DRM is inexistent. However, looking at the code in
>>> 5.0-rc2, I don't seem to find the same attributes. For instance
>>> xen_drm_front_gem_prime_vmap and gem_mmap_obj are using
>>> pgprot_writecombine. So it looks like, the mapping will be
>>> non-cacheable on Arm64.
>>>
>>> Can you explain how you came up to these attributes?
>> pgprot_writecombine is PTE_ATTRINDX(MT_NORMAL_NC), so it seems to be
>> applicable here? [1]
>
> ... that MT_NORMAL_NC is used for the frontend pages.
>
> MT_NORMAL_NC is different from MT_NORMAL. The use of the former will 
> result to non-cacheable memory while the latter will result to 
> cacheable memory.
>
> To me, this looks like the exact reason why you see artifact on the 
> display buffer. As the author of this code, can you explain why you 
> decided to use pgprot_writecombine here instead of relying on the 
> default VMA prot?
>
> [...]
>
>>> We actually never required to use cache flush in other PV protocol, so
>>> I still don't understand why the PV DRM should be different here.
>> Well, you are right. But at the same time not flushing the buffer makes
>> troubles,
>> so this is why I am trying to figure out what is wrong here.
>
> The cache flush is likely hiding the real problem rather than solving it.
>
>>>
>>> To me, it looks like that you are either missing some barriers
>> Barriers for the buffer? Not sure what you mean here.
>
> If you share information between two entities, you may need some 
> ordering so the information are seen consistently by the consumer 
> side. This can be achieved by using barriers.
>
>> Even more, we have
>> a use case
>> when the buffer is not touched by CPU in DomD and is solely owned by 
>> the HW.
>
> Memory ordering issues are subtle. The fact that one of your use-case 
> works does not imply that barriers are not necessary. I am not saying 
> there are a missing barriers, I am only pointed out potential reasons.
>
> Anyway, I don't think your problem is a missing barriers here. It is 
> more likely because of mismatch memory attributes (see above).
>
> Cheers,
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] drm/xen-front: Fix mmap attributes for display buffers

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

When GEM backing storage is allocated those are normal pages,
so there is no point using pgprot_writecombine while mmaping.
This fixes mismatch of buffer pages' memory attributes between
the frontend and backend which may cause screen artifacts.

Fixes: c575b7eeb89f ("drm/xen-front: Add support for Xen PV display frontend")

Signed-off-by: Oleksandr Andrushchenko 
Suggested-by: Julien Grall 
---
 drivers/gpu/drm/xen/xen_drm_front_gem.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index d303a2e17f5e..9d5c03d7668d 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -235,8 +235,7 @@ static int gem_mmap_obj(struct xen_gem_object *xen_obj,
vma->vm_flags &= ~VM_PFNMAP;
vma->vm_flags |= VM_MIXEDMAP;
vma->vm_pgoff = 0;
-   vma->vm_page_prot =
-   pgprot_writecombine(vm_get_page_prot(vma->vm_flags));
+   vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);
 
/*
 * vm_operations_struct.fault handler will be called if CPU access
@@ -283,7 +282,7 @@ void *xen_drm_front_gem_prime_vmap(struct drm_gem_object 
*gem_obj)
return NULL;
 
return vmap(xen_obj->pages, xen_obj->num_pages,
-   VM_MAP, pgprot_writecombine(PAGE_KERNEL));
+   VM_MAP, PAGE_KERNEL);
 }
 
 void xen_drm_front_gem_prime_vunmap(struct drm_gem_object *gem_obj,
-- 
2.20.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

On 1/24/19 5:02 PM, Julien Grall wrote:
>
>
> On 24/01/2019 14:34, Oleksandr Andrushchenko wrote:
>> Hello, Julien!
>
> Hi,
>
>> On 1/22/19 1:44 PM, Julien Grall wrote:
>>>
>>>
>>> On 1/22/19 10:28 AM, Oleksandr Andrushchenko wrote:
>>>> Hello, Julien!
>>>
>>> Hi,
>>>
>>>> On 1/21/19 7:09 PM, Julien Grall wrote:
>>>> Well, I didn't get the attributes of pages at the backend side, but 
>>>> IMO
>>>> those
>>>> do not matter in my use-case (for simplicity I am not using
>>>> zero-copying at
>>>> backend side):
>>>
>>> They are actually important no matter what is your use case. If you
>>> access the same physical page with different attributes, then you are
>>> asking for trouble.
>> So, we have:
>>
>> DomU: frontend side
>> 
>> !PTE_RDONLY + PTE_PXN + PTE_SHARED + PTE_AF + PTE_UXN +
>> PTE_ATTRINDX(MT_NORMAL)
>
> I still don't understand how you came up with MT_NORMAL when you seem 
> to confirm...
>
>>
>> DomD: backend side
>> 
>> PTE_USER + !PTE_RDONLY + PTE_PXN + PTE_NG + PTE_CONT + PTE_TABLE_BIT +
>> PTE_UXN + PTE_ATTRINDX(MT_NORMAL)
>>
>>   From the above it seems that I don't violate cached/non-cached
>> agreement here
>>>
>>> This is why Xen imposes all the pages shared to have their memory
>>> attributes following some rules. Actually, speaking with Mark R., we
>>> may want to tight a bit more the attributes.
>>>
>>>>
>>>> 1. Frontend device allocates display buffer pages which come from 
>>>> shmem
>>>> and have these attributes:
>>>> !PTE_RDONLY + PTE_PXN + PTE_SHARED + PTE_AF + PTE_UXN +
>>>> PTE_ATTRINDX(MT_NORMAL)
>>>
>>> My knowledge of Xen DRM is inexistent. However, looking at the code in
>>> 5.0-rc2, I don't seem to find the same attributes. For instance
>>> xen_drm_front_gem_prime_vmap and gem_mmap_obj are using
>>> pgprot_writecombine. So it looks like, the mapping will be
>>> non-cacheable on Arm64.
>>>
>>> Can you explain how you came up to these attributes?
>> pgprot_writecombine is PTE_ATTRINDX(MT_NORMAL_NC), so it seems to be
>> applicable here? [1]
>
> ... that MT_NORMAL_NC is used for the frontend pages.
>
> MT_NORMAL_NC is different from MT_NORMAL. The use of the former will 
> result to non-cacheable memory while the latter will result to 
> cacheable memory.
>
> To me, this looks like the exact reason why you see artifact on the 
> display buffer. As the author of this code, can you explain why you 
> decided to use pgprot_writecombine here instead of relying on the 
> default VMA prot?
>
> [...]
Sorry for late reply, it took me quite some time to re-check all the 
use-cases
that we have with PV DRM.
First of all I found a bug in my debug code which reported page 
attributes and
now I can confirm that all the use-cases sue MT_NORMAL, both backend and 
frontend.
You are right: there is no reason to have pgprot_writecombine and indeed 
I have
to rely on almost default VMA prot. I came up with the following (used 
by omap drm,
for example):

diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index ae28ad4b4254..867800a2ed42 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -238,8 +238,7 @@ static int gem_mmap_obj(struct xen_gem_object *xen_obj,
     vma->vm_flags &= ~VM_PFNMAP;
     vma->vm_flags |= VM_MIXEDMAP;
     vma->vm_pgoff = 0;
-   vma->vm_page_prot =
- pgprot_writecombine(vm_get_page_prot(vma->vm_flags));
+   vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);

     {
     int cnt = xen_obj->num_pages > 5 ? 5 : xen_obj->num_pages;
@@ -295,7 +294,7 @@ void *xen_drm_front_gem_prime_vmap(struct 
drm_gem_object *gem_obj)
     return NULL;

     return vmap(xen_obj->pages, xen_obj->num_pages,
-   VM_MAP, pgprot_writecombine(PAGE_KERNEL));
+   VM_MAP, PAGE_KERNEL);
  }

With the above all the artifacts are gone now as page attributes are the 
same across
all domains. So, I consider this as a fix and will send it as v3 or better
drop this patch and have a new one.

THANK YOU for helping figuring this out!
>
>>> We actually never required to use cache flush in other PV protocol, so
>>> I still don't understand why the PV DRM should be different here.
>> Well, you are right. But at the same time not flushing the buffer makes
>> troubles,
>> 

Re: [Xen-devel] [PATCH] arch/arm/xen: Remove duplicate header

[Oleksandr Andrushchenko]

+Boris and Juergen who can also help getting it in

On 1/28/19 8:34 AM, Souptick Joarder wrote:

On Mon, Jan 14, 2019 at 4:08 PM Oleksandr Andrushchenko
 wrote:

On 1/7/19 7:37 PM, Souptick Joarder wrote:

Remove duplicate header which is included twice.

Signed-off-by: Souptick Joarder 

Reviewed-by: Oleksandr Andrushchenko 

Can we get this patch in queue for 5.1 ?

---
   arch/arm/xen/mm.c | 1 -
   1 file changed, 1 deletion(-)

diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index cb44aa2..e1d44b9 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -7,7 +7,6 @@
   #include 
   #include 
   #include 
-#include 
   #include 
   #include 




___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH -next] drm/xen-front: Drop pointless static qualifier in fb_destroy()

[Oleksandr Andrushchenko]

On 1/26/19 2:05 PM, YueHaibing wrote:
> There is no need to have the 'struct drm_framebuffer *fb' variable
> static since new value always be assigned before use it.
>
> Signed-off-by: YueHaibing 
Good catch, thank you!
Reviewed-by: Oleksandr Andrushchenko 
> ---
>   drivers/gpu/drm/xen/xen_drm_front_kms.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/xen/xen_drm_front_kms.c 
> b/drivers/gpu/drm/xen/xen_drm_front_kms.c
> index 860da05..c2955d3 100644
> --- a/drivers/gpu/drm/xen/xen_drm_front_kms.c
> +++ b/drivers/gpu/drm/xen/xen_drm_front_kms.c
> @@ -54,7 +54,7 @@ static void fb_destroy(struct drm_framebuffer *fb)
> const struct drm_mode_fb_cmd2 *mode_cmd)
>   {
>   struct xen_drm_front_drm_info *drm_info = dev->dev_private;
> - static struct drm_framebuffer *fb;
> + struct drm_framebuffer *fb;
>   struct drm_gem_object *gem_obj;
>   int ret;
>   
>
>
>
>
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

Hello, Julien!
Sorry for the late reply - it took quite some time to collect the data 
requested.

On 1/22/19 1:44 PM, Julien Grall wrote:
>
>
> On 1/22/19 10:28 AM, Oleksandr Andrushchenko wrote:
>> Hello, Julien!
>
> Hi,
>
>> On 1/21/19 7:09 PM, Julien Grall wrote:
>> Well, I didn't get the attributes of pages at the backend side, but IMO
>> those
>> do not matter in my use-case (for simplicity I am not using 
>> zero-copying at
>> backend side):
>
> They are actually important no matter what is your use case. If you 
> access the same physical page with different attributes, then you are 
> asking for trouble.
So, we have:

DomU: frontend side

!PTE_RDONLY + PTE_PXN + PTE_SHARED + PTE_AF + PTE_UXN + 
PTE_ATTRINDX(MT_NORMAL)

DomD: backend side

PTE_USER + !PTE_RDONLY + PTE_PXN + PTE_NG + PTE_CONT + PTE_TABLE_BIT + 
PTE_UXN + PTE_ATTRINDX(MT_NORMAL)

 From the above it seems that I don't violate cached/non-cached 
agreement here
>
> This is why Xen imposes all the pages shared to have their memory 
> attributes following some rules. Actually, speaking with Mark R., we 
> may want to tight a bit more the attributes.
>
>>
>> 1. Frontend device allocates display buffer pages which come from shmem
>> and have these attributes:
>> !PTE_RDONLY + PTE_PXN + PTE_SHARED + PTE_AF + PTE_UXN +
>> PTE_ATTRINDX(MT_NORMAL)
>
> My knowledge of Xen DRM is inexistent. However, looking at the code in 
> 5.0-rc2, I don't seem to find the same attributes. For instance 
> xen_drm_front_gem_prime_vmap and gem_mmap_obj are using 
> pgprot_writecombine. So it looks like, the mapping will be 
> non-cacheable on Arm64.
>
> Can you explain how you came up to these attributes?
pgprot_writecombine is PTE_ATTRINDX(MT_NORMAL_NC), so it seems to be 
applicable here? [1]
>
>>
>> 2. Frontend grants references to these pages and shares those with the
>> backend
>>
>> 3. Backend is a user-space application (Weston client), so it uses
>> gntdev kernel
>> driver to mmap the pages. The pages, which are used by gntdev, are those
>> coming
>> from the Xen balloon driver and I believe they are all normal memory and
>> shouldn't be non-cached.
>>
>> 4. Once the frontend starts displaying it flips the buffers and backend
>> does *memcpy*
>> from the frontend-backend shared buffer into Weston's buffer. This means
>> no HW at the backend side touches the shared buffer.
>>
>> 5. I can see distorted picture.
>>
>> Previously I used setup with zero-copying, so then the picture becomes
>> more complicated
>> in terms of buffers and how those used by the backed, but anyways it
>> seems that the
>> very basic scenario with memory copying doesn't work for me.
>>
>> Using DMA API on frontend's side does help - no artifacts are seen.
>> This is why I'm thinking that this is related to frontend/kernel side
>> rather then to
>> the backend side. This is why I'm thinking this is related to caches and
>> trying to figure
>> out what can be done here instead of using DMA API.
>
> We actually never required to use cache flush in other PV protocol, so 
> I still don't understand why the PV DRM should be different here.
Well, you are right. But at the same time not flushing the buffer makes 
troubles,
so this is why I am trying to figure out what is wrong here.
>
> To me, it looks like that you are either missing some barriers
Barriers for the buffer? Not sure what you mean here. Even more, we have 
a use case
when the buffer is not touched by CPU in DomD and is solely owned by the HW.
> or the memory attributes are not correct.
Please see above - I do need your advice here
>
> Cheers,
>
Thank you very much for your time,
Oleksandr

[1] 
https://elixir.bootlin.com/linux/v5.0-rc2/source/arch/arm64/include/asm/pgtable.h#L414
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

Hello, Julien!

On 1/21/19 7:09 PM, Julien Grall wrote:
> Hello,
>
> On 21/01/2019 12:43, Oleksandr Andrushchenko wrote:
>> On 1/18/19 1:43 PM, Julien Grall wrote:
>>> On 18/01/2019 09:40, Oleksandr Andrushchenko wrote:
>>>> On 1/17/19 11:18 AM, Christoph Hellwig wrote:
>>>>> On Wed, Jan 16, 2019 at 06:43:29AM +, Oleksandr Andrushchenko
>>>>> wrote:
>>>>>>> This whole issue keeps getting more and more confusing.
>>>>>> Well, I don't really do DMA here, but instead the buffers in
>>>>>> question are shared with other Xen domain, so effectively it
>>>>>> could be thought of some sort of DMA here, where the "device" is
>>>>>> that remote domain. If the buffers are not flushed then the
>>>>>> remote part sees some inconsistency which in my case results
>>>>>> in artifacts on screen while displaying the buffers.
>>>>>> When buffers are allocated via DMA API then there are no artifacts;
>>>>>> if buffers are allocated with shmem + DMA mapping then there are no
>>>>>> artifacts as well.
>>>>>> The only offending use-case is when I use shmem backed buffers,
>>>>>> but do not flush them
>>>>> The right answer would be to implement cache maintainance hooks for
>>>>> this case in the Xen arch code.  These would basically look the same
>>>>> as the low-level cache maintainance used by the DMA ops, but without
>>>>> going through the DMA mapping layer, in fact they should probably
>>>>> reuse the same low-level assembly routines.
>>>>>
>>>>> I don't think this is the first usage of such Xen buffer sharing, so
>>>>> what do the other users do?
>>>> I'll have to get even deeper into it. Initially I
>>>> looked at the code, but didn't find anything useful.
>>>> Or maybe I have just overlooked obvious things there
>>>  From Xen on Arm ABI:
>>>
>>> "All memory which is shared with other entities in the system
>>> (including the hypervisor and other guests) must reside in memory
>>> which is mapped as Normal Inner Write-Back Outer Write-Back
>>> Inner-Shareable.
>>> This applies to:
>>>    - hypercall arguments passed via a pointer to guest memory.
>>>    - memory shared via the grant table mechanism (including PV I/O
>>>  rings etc).
>>>    - memory shared with the hypervisor (struct shared_info, struct
>>>  vcpu_info, the grant table, etc).
>>> "
>>>
>>> So you should not need any cache maintenance here. Can you provide
>>> more details on the memory attribute you use for memory shared in both
>>> the backend and frontend?
>>>
>> It takes quite some time to collect this (because many components are
>> involved in the
>> use-case), but for now the pages in the guest domain are:
>> !PTE_RDONLY + PTE_PXN + PTE_SHARED + PTE_AF + PTE_UXN +
>> PTE_ATTRINDX(MT_NORMAL)
>
> So that's the attribute for the page mapped in the frontend, right? 
> How about the backend side?
Please see below
>
> Also, could that page be handed to the graphic card correctly?
Yes, if we use zero-copying. But please see below
> If so, is your graphic card coherent?
Yes, it is
>
> If one of your components is mapping with non-cacheable attributes 
> then you are already not following the Xen Arm ABI. In that case, we 
> would need to discuss how to extend the ABI.
>
> Cheers,
>
Well, I didn't get the attributes of pages at the backend side, but IMO 
those
do not matter in my use-case (for simplicity I am not using zero-copying at
backend side):

1. Frontend device allocates display buffer pages which come from shmem
and have these attributes:
!PTE_RDONLY + PTE_PXN + PTE_SHARED + PTE_AF + PTE_UXN + 
PTE_ATTRINDX(MT_NORMAL)

2. Frontend grants references to these pages and shares those with the 
backend

3. Backend is a user-space application (Weston client), so it uses 
gntdev kernel
driver to mmap the pages. The pages, which are used by gntdev, are those 
coming
from the Xen balloon driver and I believe they are all normal memory and
shouldn't be non-cached.

4. Once the frontend starts displaying it flips the buffers and backend 
does *memcpy*
from the frontend-backend shared buffer into Weston's buffer. This means
no HW at the backend side touches the shared buffer.

5. I can see distorted picture.

Previously I used setup with zero-copying, so then the picture becomes 
more complicated
in terms of buffers and how those used by the backed, but anyways it 
seems that the
very basic scenario with memory copying doesn't work for me.

Using DMA API on frontend's side does help - no artifacts are seen.
This is why I'm thinking that this is related to frontend/kernel side 
rather then to
the backend side. This is why I'm thinking this is related to caches and 
trying to figure
out what can be done here instead of using DMA API.

Thank you,
Olkesandr
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

On 1/18/19 1:43 PM, Julien Grall wrote:
> (+ Stefano)
>
> Hi,
>
> Sorry for jumping late in the conversation.
>
> On 18/01/2019 09:40, Oleksandr Andrushchenko wrote:
>> On 1/17/19 11:18 AM, Christoph Hellwig wrote:
>>> On Wed, Jan 16, 2019 at 06:43:29AM +, Oleksandr Andrushchenko 
>>> wrote:
>>>>> This whole issue keeps getting more and more confusing.
>>>> Well, I don't really do DMA here, but instead the buffers in
>>>> question are shared with other Xen domain, so effectively it
>>>> could be thought of some sort of DMA here, where the "device" is
>>>> that remote domain. If the buffers are not flushed then the
>>>> remote part sees some inconsistency which in my case results
>>>> in artifacts on screen while displaying the buffers.
>>>> When buffers are allocated via DMA API then there are no artifacts;
>>>> if buffers are allocated with shmem + DMA mapping then there are no
>>>> artifacts as well.
>>>> The only offending use-case is when I use shmem backed buffers,
>>>> but do not flush them
>>> The right answer would be to implement cache maintainance hooks for
>>> this case in the Xen arch code.  These would basically look the same
>>> as the low-level cache maintainance used by the DMA ops, but without
>>> going through the DMA mapping layer, in fact they should probably
>>> reuse the same low-level assembly routines.
>>>
>>> I don't think this is the first usage of such Xen buffer sharing, so
>>> what do the other users do?
>> I'll have to get even deeper into it. Initially I
>> looked at the code, but didn't find anything useful.
>> Or maybe I have just overlooked obvious things there
> From Xen on Arm ABI:
>
> "All memory which is shared with other entities in the system
> (including the hypervisor and other guests) must reside in memory
> which is mapped as Normal Inner Write-Back Outer Write-Back 
> Inner-Shareable.
> This applies to:
>   - hypercall arguments passed via a pointer to guest memory.
>   - memory shared via the grant table mechanism (including PV I/O
>     rings etc).
>   - memory shared with the hypervisor (struct shared_info, struct
>     vcpu_info, the grant table, etc).
> "
>
> So you should not need any cache maintenance here. Can you provide 
> more details on the memory attribute you use for memory shared in both 
> the backend and frontend?
>
It takes quite some time to collect this (because many components are 
involved in the
use-case), but for now the pages in the guest domain are:
!PTE_RDONLY + PTE_PXN + PTE_SHARED + PTE_AF + PTE_UXN + 
PTE_ATTRINDX(MT_NORMAL)

> Cheers,
>
>>
>> Thank you,
>> Oleksandr
>> ___
>> Xen-devel mailing list
>> Xen-devel@lists.xenproject.org
>> https://lists.xenproject.org/mailman/listinfo/xen-devel
>>
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

On 1/17/19 11:18 AM, Christoph Hellwig wrote:
> On Wed, Jan 16, 2019 at 06:43:29AM +0000, Oleksandr Andrushchenko wrote:
>>> This whole issue keeps getting more and more confusing.
>> Well, I don't really do DMA here, but instead the buffers in
>> question are shared with other Xen domain, so effectively it
>> could be thought of some sort of DMA here, where the "device" is
>> that remote domain. If the buffers are not flushed then the
>> remote part sees some inconsistency which in my case results
>> in artifacts on screen while displaying the buffers.
>> When buffers are allocated via DMA API then there are no artifacts;
>> if buffers are allocated with shmem + DMA mapping then there are no
>> artifacts as well.
>> The only offending use-case is when I use shmem backed buffers,
>> but do not flush them
> The right answer would be to implement cache maintainance hooks for
> this case in the Xen arch code.  These would basically look the same
> as the low-level cache maintainance used by the DMA ops, but without
> going through the DMA mapping layer, in fact they should probably
> reuse the same low-level assembly routines.
>
> I don't think this is the first usage of such Xen buffer sharing, so
> what do the other users do?
I'll have to get even deeper into it. Initially I
looked at the code, but didn't find anything useful.
Or maybe I have just overlooked obvious things there

Thank you,
Oleksandr
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

On 1/16/19 8:36 AM, Christoph Hellwig wrote:
> On Wed, Jan 16, 2019 at 07:30:02AM +0100, Gerd Hoffmann wrote:
>>Hi,
>>
>>> +   if (!dma_map_sg(dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents,
>>> +   DMA_BIDIRECTIONAL)) {
>>> +   ret = -EFAULT;
>>> +   goto fail_free_sgt;
>>> +   }
>> Hmm, so it seems the arm guys could not come up with a suggestion how to
>> solve that one in a better way.  Ok, lets go with this then.
>>
>> But didn't we agree that this deserves a comment exmplaining the purpose
>> of the dma_map_sg() call is to flush caches and that there is no actual
>> DMA happening here?
> Using a dma mapping call to flush caches is complete no-go.  But the
> real question is why you'd even want to flush cashes if you do not
> want a dma mapping?
>
> This whole issue keeps getting more and more confusing.
Well, I don't really do DMA here, but instead the buffers in
question are shared with other Xen domain, so effectively it
could be thought of some sort of DMA here, where the "device" is
that remote domain. If the buffers are not flushed then the
remote part sees some inconsistency which in my case results
in artifacts on screen while displaying the buffers.
When buffers are allocated via DMA API then there are no artifacts;
if buffers are allocated with shmem + DMA mapping then there are no
artifacts as well.
The only offending use-case is when I use shmem backed buffers,
but do not flush them
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

On 1/16/19 8:30 AM, Gerd Hoffmann wrote:
>Hi,
>
>> +if (!dma_map_sg(dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents,
>> +DMA_BIDIRECTIONAL)) {
>> +ret = -EFAULT;
>> +goto fail_free_sgt;
>> +}
> Hmm, so it seems the arm guys could not come up with a suggestion how to
> solve that one in a better way.  Ok, lets go with this then.
>
> But didn't we agree that this deserves a comment exmplaining the purpose
> of the dma_map_sg() call is to flush caches and that there is no actual
> DMA happening here?
My bad, will add the comment
> cheers,
>Gerd
>
Thank you,
Oleksandr
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] drm: Split out drm_probe_helper.h

[Oleksandr Andrushchenko]

rs/gpu/drm/radeon/radeon_acpi.c  |  1 +
  drivers/gpu/drm/radeon/radeon_connectors.c|  1 +
  drivers/gpu/drm/radeon/radeon_device.c|  1 +
  drivers/gpu/drm/radeon/radeon_display.c   |  1 +
  drivers/gpu/drm/radeon/radeon_dp_mst.c|  1 +
  drivers/gpu/drm/radeon/radeon_drv.c   |  1 +
  drivers/gpu/drm/radeon/radeon_irq_kms.c   |  1 +
  drivers/gpu/drm/rcar-du/rcar_du_crtc.c|  2 +-
  drivers/gpu/drm/rcar-du/rcar_du_drv.c |  2 +-
  drivers/gpu/drm/rcar-du/rcar_du_encoder.c |  2 +-
  drivers/gpu/drm/rcar-du/rcar_du_kms.c |  2 +-
  drivers/gpu/drm/rcar-du/rcar_du_plane.c   |  2 +-
  drivers/gpu/drm/rcar-du/rcar_du_vsp.c |  2 +-
  drivers/gpu/drm/rcar-du/rcar_lvds.c   |  2 +-
  .../gpu/drm/rockchip/analogix_dp-rockchip.c   |  2 +-
  drivers/gpu/drm/rockchip/cdn-dp-core.c|  2 +-
  drivers/gpu/drm/rockchip/cdn-dp-core.h|  2 +-
  drivers/gpu/drm/rockchip/dw_hdmi-rockchip.c   |  2 +-
  drivers/gpu/drm/rockchip/inno_hdmi.c  |  2 +-
  drivers/gpu/drm/rockchip/rockchip_drm_drv.c   |  2 +-
  drivers/gpu/drm/rockchip/rockchip_drm_fb.c|  2 +-
  drivers/gpu/drm/rockchip/rockchip_drm_fbdev.c |  2 +-
  drivers/gpu/drm/rockchip/rockchip_drm_psr.c   |  2 +-
  drivers/gpu/drm/rockchip/rockchip_drm_vop.c   |  2 +-
  drivers/gpu/drm/rockchip/rockchip_lvds.c  |  2 +-
  drivers/gpu/drm/rockchip/rockchip_rgb.c   |  2 +-
  drivers/gpu/drm/shmobile/shmob_drm_crtc.c |  1 +
  drivers/gpu/drm/shmobile/shmob_drm_drv.c  |  1 +
  drivers/gpu/drm/shmobile/shmob_drm_kms.c  |  1 +
  drivers/gpu/drm/sti/sti_crtc.c|  2 +-
  drivers/gpu/drm/sti/sti_drv.c |  2 +-
  drivers/gpu/drm/sti/sti_dvo.c |  2 +-
  drivers/gpu/drm/sti/sti_hda.c |  2 +-
  drivers/gpu/drm/sti/sti_hdmi.c|  2 +-
  drivers/gpu/drm/sti/sti_tvout.c   |  2 +-
  drivers/gpu/drm/stm/drv.c |  2 +-
  drivers/gpu/drm/stm/ltdc.c|  2 +-
  drivers/gpu/drm/sun4i/sun4i_backend.c |  2 +-
  drivers/gpu/drm/sun4i/sun4i_crtc.c|  2 +-
  drivers/gpu/drm/sun4i/sun4i_drv.c |  2 +-
  drivers/gpu/drm/sun4i/sun4i_hdmi_enc.c|  2 +-
  drivers/gpu/drm/sun4i/sun4i_lvds.c|  2 +-
  drivers/gpu/drm/sun4i/sun4i_rgb.c |  2 +-
  drivers/gpu/drm/sun4i/sun4i_tcon.c|  2 +-
  drivers/gpu/drm/sun4i/sun4i_tv.c  |  2 +-
  drivers/gpu/drm/sun4i/sun6i_mipi_dsi.c|  2 +-
  drivers/gpu/drm/sun4i/sun8i_mixer.c   |  2 +-
  drivers/gpu/drm/sun4i/sun8i_ui_layer.c|  2 +-
  drivers/gpu/drm/sun4i/sun8i_vi_layer.c|  2 +-
  drivers/gpu/drm/tegra/drm.h   |  2 +-
  drivers/gpu/drm/tegra/fb.c|  1 +
  drivers/gpu/drm/tegra/hdmi.c  |  2 +-
  drivers/gpu/drm/tegra/hub.c   |  2 +-
  drivers/gpu/drm/tilcdc/tilcdc_drv.c   |  1 +
  drivers/gpu/drm/tilcdc/tilcdc_drv.h   |  1 -
  drivers/gpu/drm/tilcdc/tilcdc_external.c  |  1 +
  drivers/gpu/drm/tilcdc/tilcdc_panel.c |  1 +
  drivers/gpu/drm/tilcdc/tilcdc_tfp410.c|  1 +
  drivers/gpu/drm/tinydrm/core/tinydrm-core.c   |  2 +-
  drivers/gpu/drm/tinydrm/core/tinydrm-pipe.c   |  2 +-
  drivers/gpu/drm/tve200/tve200_drv.c   |  2 +-
  drivers/gpu/drm/udl/udl_connector.c   |  1 +
  drivers/gpu/drm/udl/udl_drv.c |  1 +
  drivers/gpu/drm/udl/udl_main.c|  1 +
  drivers/gpu/drm/vc4/vc4_crtc.c|  2 +-
  drivers/gpu/drm/vc4/vc4_dpi.c |  2 +-
  drivers/gpu/drm/vc4/vc4_dsi.c |  2 +-
  drivers/gpu/drm/vc4/vc4_hdmi.c|  2 +-
  drivers/gpu/drm/vc4/vc4_kms.c |  2 +-
  drivers/gpu/drm/vc4/vc4_txp.c |  2 +-
  drivers/gpu/drm/vc4/vc4_vec.c |  2 +-
  drivers/gpu/drm/virtio/virtgpu_display.c  |  2 +-
  drivers/gpu/drm/virtio/virtgpu_drv.h  |  2 +-
  drivers/gpu/drm/vkms/vkms_crtc.c  |  2 +-
  drivers/gpu/drm/vkms/vkms_drv.c   |  2 +-
  drivers/gpu/drm/vkms/vkms_output.c|  2 +-
  drivers/gpu/drm/vmwgfx/vmwgfx_kms.h   |  2 +-
  drivers/gpu/drm/xen/xen_drm_front.c   |  2 +-
  drivers/gpu/drm/xen/xen_drm_front_conn.c  |  2 +-
  drivers/gpu/drm/xen/xen_drm_front_gem.c   |  2 +-
  drivers/gpu/drm/xen/xen_drm_front_kms.c   |  2 +-

For Xen:
Acked-by: Oleksandr Andrushchenko 

  drivers/gpu/drm/zte/zx_drm_drv.c  |  2 +-
  drivers/gpu/drm/zte/zx_hdmi.c |  2 +-
  drivers/gpu/drm/zte/zx_tvenc.c|  2 +-
  drivers/gpu/drm/zte/zx_vga.c  |  2 +-
  drivers/gpu/drm/zte/zx_vou.c  |  2 +-
  drivers/staging/vboxvideo/vbox_irq.c  |  4 +-
  drivers/staging/vboxvideo/vbox_mode.c |  2 +-
  include/drm/drm_crtc_helper.h   

Re: [Xen-devel] [PATCH v4 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

On 1/15/19 4:44 PM, Hans Verkuil wrote:
> Hi Oleksandr,
Hello, Hans!
> Just two remaining comments:
>
> On 1/15/19 10:38 AM, Oleksandr Andrushchenko wrote:
>> From: Oleksandr Andrushchenko 
>>
>> This is the ABI for the two halves of a para-virtualized
>> camera driver which extends Xen's reach multimedia capabilities even
>> farther enabling it for video conferencing, In-Vehicle Infotainment,
>> high definition maps etc.
>>
>> The initial goal is to support most needed functionality with the
>> final idea to make it possible to extend the protocol if need be:
>>
>> 1. Provide means for base virtual device configuration:
>>   - pixel formats
>>   - resolutions
>>   - frame rates
>> 2. Support basic camera controls:
>>   - contrast
>>   - brightness
>>   - hue
>>   - saturation
>> 3. Support streaming control
>>
>> Signed-off-by: Oleksandr Andrushchenko 
>> ---
>>   xen/include/public/io/cameraif.h | 1364 ++
>>   1 file changed, 1364 insertions(+)
>>   create mode 100644 xen/include/public/io/cameraif.h
>>
>> diff --git a/xen/include/public/io/cameraif.h 
>> b/xen/include/public/io/cameraif.h
>> new file mode 100644
>> index ..246eb2457f40
>> --- /dev/null
>> +++ b/xen/include/public/io/cameraif.h
>> @@ -0,0 +1,1364 @@
> 
>
>> +/*
>> + 
>> **
>> + * EVENT CODES
>> + 
>> **
>> + */
>> +#define XENCAMERA_EVT_FRAME_AVAIL  0x00
>> +#define XENCAMERA_EVT_CTRL_CHANGE  0x01
>> +
>> +/* Resolution has changed. */
>> +#define XENCAMERA_EVT_CFG_FLG_RESOL(1 << 0)
> I think this flag is a left-over from v2 and should be removed.
Good catch, will remove
>
> 
>
>> + * Request number of buffers to be used:
>> + * 01 2   3octet
>> + * +++++
>> + * |   id| _OP_BUF_REQUEST|   reserved | 4
>> + * +++++
>> + * | reserved  | 8
>> + * +++++
>> + * |num_bufs| reserved | 12
>> + * +++++
>> + * | reserved  | 16
>> + * +++++
>> + * |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/|
>> + * +++++
>> + * | reserved  | 64
>> + * +++++
>> + *
>> + * num_bufs - uint8_t, desired number of buffers to be used. This is
>> + *   limited to the value configured in XenStore.max-buffers.
>> + *   Passing zero num_bufs in this request (after streaming has stopped
>> + *   and all buffers destroyed) unblocks camera configuration changes.
> I think the phrase 'unblocks camera configuration changes' is confusing.
>
> In v3 this sentence came after the third note below, and so it made sense
> in that context, but now the order has been reversed and it became hard to
> understand.
>
> I'm not sure what the best approach is to fix this. One option is to remove
> the third note and integrate it somehow in the sentence above. Or perhaps
> do away with the 'notes' at all and just write a more extensive documentation
> for this op. I leave that up to you.
I'll think about what can be done here
>> + *
>> + * See response format for this request.
>> + *
>> + * Notes:
>> + *  - frontend must check the corresponding response in order to see
>> + *if the values reported back by the backend do match the desired ones
>> + *and can be accepted.
>> + *  - frontend may send multiple XENCAMERA_OP_BUF_REQUEST requests before
>> + *sending XENCAMERA_OP_STREAM_START request to update or tune the
>> + *configuration.
>> + *  - after this request camera configuration cannot be changed, unless
> camera configuration -> the camera configuration
Will fix
>> + *streaming is stopped and buffers destroyed
>> + */
> Regards,
>
>   Hans
Thank you for helping with this,
Oleksandr
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

When GEM backing storage is allocated with drm_gem_get_pages
the backing pages may be cached, thus making it possible that
the backend sees only partial content of the buffer which may
lead to screen artifacts. Make sure that the frontend's
memory is coherent and the backend always sees correct display
buffer content.

Fixes: c575b7eeb89f ("drm/xen-front: Add support for Xen PV display frontend")

Signed-off-by: Oleksandr Andrushchenko 

---
Changes since v1:
 - Remove GFP_USER|__GFP_DMA32 mapping flags (Gerd)
 - Use drm_prime_pages_to_sg directly (Noralf)

 drivers/gpu/drm/xen/xen_drm_front_gem.c | 38 ++---
 1 file changed, 34 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index 28bc501af450..0b0d9b4f97dc 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -32,8 +32,11 @@ struct xen_gem_object {
/* set for buffers allocated by the backend */
bool be_alloc;
 
-   /* this is for imported PRIME buffer */
-   struct sg_table *sgt_imported;
+   /*
+* this is for imported PRIME buffer or the one allocated via
+* drm_gem_get_pages.
+*/
+   struct sg_table *sgt;
 };
 
 static inline struct xen_gem_object *
@@ -124,8 +127,28 @@ static struct xen_gem_object *gem_create(struct drm_device 
*dev, size_t size)
goto fail;
}
 
+   xen_obj->sgt = drm_prime_pages_to_sg(xen_obj->pages,
+xen_obj->num_pages);
+   if (IS_ERR_OR_NULL(xen_obj->sgt)) {
+   ret = PTR_ERR(xen_obj->sgt);
+   xen_obj->sgt = NULL;
+   goto fail_put_pages;
+   }
+
+   if (!dma_map_sg(dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents,
+   DMA_BIDIRECTIONAL)) {
+   ret = -EFAULT;
+   goto fail_free_sgt;
+   }
+
return xen_obj;
 
+fail_free_sgt:
+   sg_free_table(xen_obj->sgt);
+   xen_obj->sgt = NULL;
+fail_put_pages:
+   drm_gem_put_pages(_obj->base, xen_obj->pages, true, false);
+   xen_obj->pages = NULL;
 fail:
DRM_ERROR("Failed to allocate buffer with size %zu\n", size);
return ERR_PTR(ret);
@@ -148,7 +171,7 @@ void xen_drm_front_gem_free_object_unlocked(struct 
drm_gem_object *gem_obj)
struct xen_gem_object *xen_obj = to_xen_gem_obj(gem_obj);
 
if (xen_obj->base.import_attach) {
-   drm_prime_gem_destroy(_obj->base, xen_obj->sgt_imported);
+   drm_prime_gem_destroy(_obj->base, xen_obj->sgt);
gem_free_pages_array(xen_obj);
} else {
if (xen_obj->pages) {
@@ -157,6 +180,13 @@ void xen_drm_front_gem_free_object_unlocked(struct 
drm_gem_object *gem_obj)
xen_obj->pages);
gem_free_pages_array(xen_obj);
} else {
+   if (xen_obj->sgt) {
+   dma_unmap_sg(xen_obj->base.dev->dev,
+xen_obj->sgt->sgl,
+xen_obj->sgt->nents,
+DMA_BIDIRECTIONAL);
+   sg_free_table(xen_obj->sgt);
+   }
drm_gem_put_pages(_obj->base,
  xen_obj->pages, true, false);
}
@@ -202,7 +232,7 @@ xen_drm_front_gem_import_sg_table(struct drm_device *dev,
if (ret < 0)
return ERR_PTR(ret);
 
-   xen_obj->sgt_imported = sgt;
+   xen_obj->sgt = sgt;
 
ret = drm_prime_sg_to_page_addr_arrays(sgt, xen_obj->pages,
   NULL, xen_obj->num_pages);
-- 
2.20.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v4 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
 - pixel formats
 - resolutions
 - frame rates
2. Support basic camera controls:
 - contrast
 - brightness
 - hue
 - saturation
3. Support streaming control

Signed-off-by: Oleksandr Andrushchenko 
---
 xen/include/public/io/cameraif.h | 1364 ++
 1 file changed, 1364 insertions(+)
 create mode 100644 xen/include/public/io/cameraif.h

diff --git a/xen/include/public/io/cameraif.h b/xen/include/public/io/cameraif.h
new file mode 100644
index ..246eb2457f40
--- /dev/null
+++ b/xen/include/public/io/cameraif.h
@@ -0,0 +1,1364 @@
+/**
+ * cameraif.h
+ *
+ * Unified camera device I/O interface for Xen guest OSes.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright (C) 2018-2019 EPAM Systems Inc.
+ *
+ * Author: Oleksandr Andrushchenko 
+ */
+
+#ifndef __XEN_PUBLIC_IO_CAMERAIF_H__
+#define __XEN_PUBLIC_IO_CAMERAIF_H__
+
+#include "ring.h"
+#include "../grant_table.h"
+
+/*
+ **
+ *   Protocol version
+ **
+ */
+#define XENCAMERA_PROTOCOL_VERSION "1"
+
+/*
+ **
+ *  Feature and Parameter Negotiation
+ **
+ *
+ * Front->back notifications: when enqueuing a new request, sending a
+ * notification can be made conditional on xencamera_req (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Backends must set
+ * xencamera_req appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
+ *
+ * Back->front notifications: when enqueuing a new response, sending a
+ * notification can be made conditional on xencamera_resp (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Frontends must set
+ * xencamera_resp appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
+ *
+ * The two halves of a para-virtual camera driver utilize nodes within
+ * XenStore to communicate capabilities and to negotiate operating parameters.
+ * This section enumerates these nodes which reside in the respective front and
+ * backend portions of XenStore, following the XenBus convention.
+ *
+ * All data in XenStore is stored as strings. Nodes specifying numeric
+ * values are encoded in decimal. Integer value ranges listed below are
+ * expressed as fixed sized integer types capable of storing the conversion
+ * of a properly formatted node string, without loss of information.
+ *
+ **
+ *Example configuration
+ **
+ *
+ * This is an example of backend and frontend configuration:
+ *
+ *- Backend ---
+ *
+ * /local/domain/0/backend/vcamera/1/0/frontend-id = "1"
+ * /local/domain/0/backend/vcamera/1/0/frontend = 
"/local/domain/1/device/vcamera/0"
+ * /local/domain/0/backend/vcamera/1/0/state = "4"
+ * /local/domain/0/backend/vcamera/1/0/versions = "1,2"
+ *
+ *- Frontend --
+ *
+ 

[Xen-devel] [PATCH v4 0/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Hello!

At the moment Xen [1] already supports some virtual multimedia
features [2] such as virtual display, sound. It supports keyboards,
pointers and multi-touch devices all allowing Xen to be used in
automotive appliances, In-Vehicle Infotainment (IVI) systems
and many more.

Frontend implementation is available at [3] and the corresponding
backend at [4]. These are work in progress, but frontend already
passes v4l2-compliance test for V4L2 drivers. libxl preliminary
changes are available at [5].

This work adds a new Xen para-virtualized protocol for a virtual
camera device which extends multimedia capabilities of Xen even
farther: video conferencing, IVI, high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
 - pixel formats
 - resolutions
 - frame rates
2. Support basic camera controls:
 - contrast
 - brightness
 - hue
 - saturation
3. Support streaming control

I would like to thank Hans Verkuil  for valuable
comments and help.

Thank you,
Oleksandr Andrushchenko

Changes since v3:
=

1. Add trimming example for short FOURCC labels, e.g. Y16 and Y16-BE
2. Remove from XENCAMERA_OP_CONFIG_XXX requests colorspace, xfer_func,
   ycbcr_enc, quantization and move those into the corresponding response
3. Extend description of XENCAMERA_OP_BUF_REQUEST.num_bufs: limit to
   maximum buffers and num_bufs == 0 case
4. Extend decription of XENCAMERA_OP_BUF_CREATE.index and specify its
   range
5. Make XENCAMERA_EVT_FRAME_AVAIL.seq_num 32-bit instead of 64-bit

Changes since v2:
=

1. Add "max-buffers" frontend configuration entry, e.g.
   the maximum number of camera buffers a frontend may use.
2. Add big-endian pixel-format support:
 - "formats" configuration string length changed from 4 to 7
   octets, so we can also manage BE pixel-formats
 - add corresponding comments to FOURCC mappings description
3. New commands added to the protocol and documented:
 - XENCAMERA_OP_CONFIG_VALIDATE
 - XENCAMERA_OP_FRAME_RATE_SET
 - XENCAMERA_OP_BUF_GET_LAYOUT
4.-Add defaults for colorspace, xfer, ycbcr_enc and quantization
5. Remove XENCAMERA_EVT_CONFIG_CHANGE event
6. Move plane offsets to XENCAMERA_OP_BUF_REQUEST as offsets
   required for the frontend might not be known at the configuration time
7. Clean up and address comments to v2 of the protocol

Changes since v1:
=

1. Added XenStore entries:
 - frame-rates
2. Do not require the FOURCC code in XenStore to be upper case only
3. Added/changed command set:
 - configuration get/set
 - buffer queue/dequeue
 - control get
4. Added control flags, e.g. read-only etc.
5. Added colorspace configuration support, relevant constants
6. Added events:
 - configuration change
 - control change
7. Changed control values to 64-bit
8. Added sequence number to frame avail event
9. Coding style cleanup

[1] https://www.xenproject.org/
[2] https://xenbits.xen.org/gitweb/?p=xen.git;a=tree;f=xen/include/public/io
[3] https://github.com/andr2000/linux/tree/camera_front_v1/drivers/media/xen
[4] https://github.com/andr2000/camera_be
[5] https://github.com/andr2000/xen/tree/vcamera

Oleksandr Andrushchenko (1):
  cameraif: add ABI for para-virtual camera

 xen/include/public/io/cameraif.h | 1364 ++
 1 file changed, 1364 insertions(+)
 create mode 100644 xen/include/public/io/cameraif.h

-- 
2.20.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] arch/arm/xen: Remove duplicate header

[Oleksandr Andrushchenko]

On 1/7/19 7:37 PM, Souptick Joarder wrote:

Remove duplicate header which is included twice.

Signed-off-by: Souptick Joarder 

Reviewed-by: Oleksandr Andrushchenko 

---
  arch/arm/xen/mm.c | 1 -
  1 file changed, 1 deletion(-)

diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index cb44aa2..e1d44b9 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -7,7 +7,6 @@
  #include 
  #include 
  #include 
-#include 
  #include 
  #include 
  



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

Hello, Hans!
Could you please take a look at my answers below and kindly let me know
if we are ready for (final?) v4 from your point of view.

Konrad, Xen-devel - do you have any objections/comments on this?

Thank you,
Oleksandr

On 12/17/18 9:37 AM, Oleksandr Andrushchenko wrote:

Hello, Hans!

Thank you for reviewing, please find my answers inline

On 12/14/18 2:14 PM, Hans Verkuil wrote:

Hi Oleksandr,

This is looking a lot better than v2. I do have a few remaining 
comments about

some things that are a bit unclear to me.

On 12/12/18 10:49 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
  - pixel formats
  - resolutions
  - frame rates
2. Support basic camera controls:
  - contrast
  - brightness
  - hue
  - saturation
3. Support streaming control

Signed-off-by: Oleksandr Andrushchenko 


---
  xen/include/public/io/cameraif.h | 1374 
++

  1 file changed, 1374 insertions(+)
  create mode 100644 xen/include/public/io/cameraif.h

diff --git a/xen/include/public/io/cameraif.h 
b/xen/include/public/io/cameraif.h

new file mode 100644
index ..9aae0f47743b
--- /dev/null
+++ b/xen/include/public/io/cameraif.h
@@ -0,0 +1,1374 @@
+/** 


+ * cameraif.h
+ *
+ * Unified camera device I/O interface for Xen guest OSes.
+ *
+ * Permission is hereby granted, free of charge, to any person 
obtaining a copy
+ * of this software and associated documentation files (the 
"Software"), to
+ * deal in the Software without restriction, including without 
limitation the
+ * rights to use, copy, modify, merge, publish, distribute, 
sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the 
Software is

+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be 
included in

+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 
EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 
MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO 
EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR 
OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, 
ARISING

+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright (C) 2018 EPAM Systems Inc.
+ *
+ * Author: Oleksandr Andrushchenko 
+ */
+
+#ifndef __XEN_PUBLIC_IO_CAMERAIF_H__
+#define __XEN_PUBLIC_IO_CAMERAIF_H__
+
+#include "ring.h"
+#include "../grant_table.h"
+
+/*
+ 
**

+ *   Protocol version
+ 
**

+ */
+#define XENCAMERA_PROTOCOL_VERSION "1"
+
+/*
+ 
**

+ *  Feature and Parameter Negotiation
+ 
**

+ *
+ * Front->back notifications: when enqueuing a new request, sending a
+ * notification can be made conditional on xencamera_req (i.e., the 
generic

+ * hold-off mechanism provided by the ring macros). Backends must set
+ * xencamera_req appropriately (e.g., using 
RING_FINAL_CHECK_FOR_REQUESTS()).

+ *
+ * Back->front notifications: when enqueuing a new response, sending a
+ * notification can be made conditional on xencamera_resp (i.e., 
the generic

+ * hold-off mechanism provided by the ring macros). Frontends must set
+ * xencamera_resp appropriately (e.g., using 
RING_FINAL_CHECK_FOR_RESPONSES()).

+ *
+ * The two halves of a para-virtual camera driver utilize nodes within
+ * XenStore to communicate capabilities and to negotiate operating 
parameters.
+ * This section enumerates these nodes which reside in the 
respective front and

+ * backend portions of XenStore, following the XenBus convention.
+ *
+ * All data in XenStore is stored as strings. Nodes specifying numeric
+ * values are encoded in decimal. Integer value ranges listed below 
are
+ * expressed as fixed sized integer types capable of storing the 
conversion

+ * of a properly formatted node string, without loss of information.
+ *
+ 

Re: [Xen-devel] [PATCH 5/9] drm/xen/xen_drm_front_gem.c: Convert to use vm_insert_range

[Oleksandr Andrushchenko]

On 1/11/19 5:10 PM, Souptick Joarder wrote:

Convert to use vm_insert_range() to map range of kernel
memory to user vma.

Signed-off-by: Souptick Joarder 

Reviewed-by: Oleksandr Andrushchenko 

---
  drivers/gpu/drm/xen/xen_drm_front_gem.c | 18 +-
  1 file changed, 5 insertions(+), 13 deletions(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index 47ff019..9990c2f 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -225,8 +225,7 @@ struct drm_gem_object *
  static int gem_mmap_obj(struct xen_gem_object *xen_obj,
struct vm_area_struct *vma)
  {
-   unsigned long addr = vma->vm_start;
-   int i;
+   int ret;
  
  	/*

 * clear the VM_PFNMAP flag that was set by drm_gem_mmap(), and set the
@@ -247,18 +246,11 @@ static int gem_mmap_obj(struct xen_gem_object *xen_obj,
 * FIXME: as we insert all the pages now then no .fault handler must
 * be called, so don't provide one
 */
-   for (i = 0; i < xen_obj->num_pages; i++) {
-   int ret;
-
-   ret = vm_insert_page(vma, addr, xen_obj->pages[i]);
-   if (ret < 0) {
-   DRM_ERROR("Failed to insert pages into vma: %d\n", ret);
-   return ret;
-   }
+   ret = vm_insert_range(vma, xen_obj->pages, xen_obj->num_pages);
+   if (ret < 0)
+   DRM_ERROR("Failed to insert pages into vma: %d\n", ret);
  
-		addr += PAGE_SIZE;

-   }
-   return 0;
+   return ret;
  }
  
  int xen_drm_front_gem_mmap(struct file *filp, struct vm_area_struct *vma)



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

On 12/20/18 8:38 PM, Christoph Hellwig wrote:

On Thu, Dec 20, 2018 at 07:35:15PM +0100, Daniel Vetter wrote:

Err, with streaming DMA buffer sharing is trivial.  The coherent DMA
allocator is what causes all kinds of horrible hacks that can't actually
work on various platforms.

Hm, I thought the streaming dma api is the one that causes bounce
buffers and all that fun. If you're unlucky at least.

Yes it may.  But even if that happens everything will actually work,
just slower.  While the dma coherent API is simply broken.

But if you don't want bounce buffering you need to use the dma
noncoherent allocator as proposed here:


https://lists.linuxfoundation.org/pipermail/iommu/2018-December/031982.html

which combines allocating memory that doesn't need to be bounce
buffered with a sharing scheme that can actually work.

So, the bottom line will be: I can use DMA API for what I need, but:
1. I need to remove GFP_USER
2. No need for DMA32 (so no chance for bouncing to step in)
3. I may need to check if mapping and unmapping of the buffer
at once will also help, e.g. no need to have the buffer mapped until
it is destroyed
Did I get it all right?

Thank you,
Oleksandr

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

On 12/20/18 5:36 PM, Christoph Hellwig wrote:

On Tue, Dec 18, 2018 at 08:20:22PM +0100, Noralf Trønnes wrote:

+   if (!dma_map_sg(dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents,
+   DMA_BIDIRECTIONAL)) {


Are you using the DMA streaming API as a way to flush the caches?

This looks rather broken.  Please send the whole patch series to
the iommu list for proper review.

This is the only patch [1], no series. And at the moment I think
there is nothing to review as I am not sure how to deal with those
shmem pages: this patch is rather to start a discussion on how shmem
pages can be flushed on ARM (the only workaround I have so far is
in this patch which uses DMA API). This is where I am looking for
some advice, so I can implement the patch the right way.

Does this mean that GFP_USER isn't making the buffer coherent?

How could GFP_USER make memory coherent in any way?

I am no way an expert here, but other DRM drivers allocate buffers
from shmem and then use DMA API [2], for example [3]

[1] https://patchwork.kernel.org/patch/10700089/
[2] https://elixir.bootlin.com/linux/v4.20-rc7/ident/drm_gem_get_pages
[3] 
https://elixir.bootlin.com/linux/v4.20-rc7/source/drivers/gpu/drm/omapdrm/omap_gem.c#L248


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] drm_gem_get_pages and proper flushing/coherency

[Oleksandr Andrushchenko]

+ARM mailing list for help and suggestions

Dear ARM community!

Xen hypervizor para-virtualized DRM frontend driver [1] uses shmem
backed pages for display buffers and shares those with the host
driver. Everything works just fine, but in some scenarios I see
artifacts on the screen which are because those shmem pages of the
buffer are not flushed (please see the mail below for more
details from DRM point of view).
For x86 the flushing of the pages can be done with DRM helper [2]
and seem to help. But clflushopt, which is used there, is not defined
for ARM. I had a suggestion to use set_pages_array_*(),
but again it is only defined for x86, not ARM [3].

The implementation/workaround that I have [4] is based on the DMA
approach, e.g. I map shmem pages and it seem to help, but the whole
DMA approach here seems to be an overkill for that.

Now to the question: could anyone on ARM community help understand the
right way to flush those pages, so I don't need to use DMA for that?

Thank you,
Oleksandr

On 11/26/18 2:15 PM, Oleksandr Andrushchenko wrote:

Hello, all!

My driver (Xen para-virtualized frontend) in some scenarios uses
drm_gem_get_pages for allocating backing storage for dumb buffers.
There are use-cases which showed some artifacts on the screen
(modetest, other) which were worked around by flushing pages of the
buffer on page flip with drm_clflush_pages. But, the problem here
is that drm_clflush_pages is not available on ARM platforms (it is a NOP)
and doing flushes on every page flip seems to be non-optimal.

Other drivers that use drm_gem_get_pages seem to use DMA map/unmap
on the shmem backed buffer (this is from where drm_gem_get_pages
allocates the pages) and this is an obvious approach as the buffer needs
to be shared with real HW for DMA - please correct me if my understanding
here is wrong.

This is the part I missed in my implementation as I don't really have a
HW device which needs DMA, but a backend running in a different Xen 
domain.


Thus, as the buffer is backed with cachable pages the backend may see

artifacts on its side.


I am looking for some advices on what would be the best option to
make sure dumb buffers are not flushed every page flip and still
the memory remains coherent to the backend. I have implemented a
DMA map/unmap of the shmem pages on GEM object creation/destruction
and this does solve the problem, but as the backend is not really
a DMA device this is a bit misleading.

Is there any other (more?) suitable/preferable way(s) to achieve the 
same?


Thank you,
Oleksandr


[1] https://elixir.bootlin.com/linux/v4.20-rc7/source/drivers/gpu/drm/xen
[2] 
https://elixir.bootlin.com/linux/v4.20-rc7/source/drivers/gpu/drm/drm_cache.c#L45
[3] 
https://elixir.bootlin.com/linux/v4.20-rc7/source/arch/x86/include/asm/set_memory.h

[4] https://patchwork.kernel.org/patch/10700089/

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

On 12/19/18 6:14 PM, Noralf Trønnes wrote:


Den 19.12.2018 09.18, skrev Oleksandr Andrushchenko:

On 12/18/18 9:20 PM, Noralf Trønnes wrote:


Den 27.11.2018 11.32, skrev Oleksandr Andrushchenko:

From: Oleksandr Andrushchenko 

When GEM backing storage is allocated with drm_gem_get_pages
the backing pages may be cached, thus making it possible that
the backend sees only partial content of the buffer which may
lead to screen artifacts. Make sure that the frontend's
memory is coherent and the backend always sees correct display
buffer content.

Fixes: c575b7eeb89f ("drm/xen-front: Add support for Xen PV display 
frontend")


Signed-off-by: Oleksandr Andrushchenko 


---
  drivers/gpu/drm/xen/xen_drm_front_gem.c | 62 
+++--

  1 file changed, 48 insertions(+), 14 deletions(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
b/drivers/gpu/drm/xen/xen_drm_front_gem.c

index 47ff019d3aef..c592735e49d2 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -33,8 +33,11 @@ struct xen_gem_object {
  /* set for buffers allocated by the backend */
  bool be_alloc;
  -    /* this is for imported PRIME buffer */
-    struct sg_table *sgt_imported;
+    /*
+ * this is for imported PRIME buffer or the one allocated via
+ * drm_gem_get_pages.
+ */
+    struct sg_table *sgt;
  };
    static inline struct xen_gem_object *
@@ -77,10 +80,21 @@ static struct xen_gem_object 
*gem_create_obj(struct drm_device *dev,

  return xen_obj;
  }
  +struct sg_table *xen_drm_front_gem_get_sg_table(struct 
drm_gem_object *gem_obj)

+{
+    struct xen_gem_object *xen_obj = to_xen_gem_obj(gem_obj);
+
+    if (!xen_obj->pages)
+    return ERR_PTR(-ENOMEM);
+
+    return drm_prime_pages_to_sg(xen_obj->pages, xen_obj->num_pages);
+}
+
  static struct xen_gem_object *gem_create(struct drm_device *dev, 
size_t size)

  {
  struct xen_drm_front_drm_info *drm_info = dev->dev_private;
  struct xen_gem_object *xen_obj;
+    struct address_space *mapping;
  int ret;
    size = round_up(size, PAGE_SIZE);
@@ -113,10 +127,14 @@ static struct xen_gem_object 
*gem_create(struct drm_device *dev, size_t size)

  xen_obj->be_alloc = true;
  return xen_obj;
  }
+
  /*
   * need to allocate backing pages now, so we can share those
   * with the backend
   */



Let's see if I understand what you're doing:

Here you say that the pages should be DMA accessible for devices 
that can

only see 4GB.


Yes, your understanding is correct. As we are a para-virtualized 
device we


do not have strict requirements for 32-bit DMA. But, via dma-buf export,

the buffer we create can be used by real HW, e.g. one can pass-through

real HW devices into a guest domain and they can import our buffer (yes,

they can be IOMMU backed and other conditions may apply).

So, this is why we are limiting to DMA32 here, just to allow more 
possible


use-cases




+    mapping = xen_obj->base.filp->f_mapping;
+    mapping_set_gfp_mask(mapping, GFP_USER | __GFP_DMA32);
+
  xen_obj->num_pages = DIV_ROUND_UP(size, PAGE_SIZE);
  xen_obj->pages = drm_gem_get_pages(_obj->base);
  if (IS_ERR_OR_NULL(xen_obj->pages)) {
@@ -125,8 +143,27 @@ static struct xen_gem_object 
*gem_create(struct drm_device *dev, size_t size)

  goto fail;
  }
  +    xen_obj->sgt = xen_drm_front_gem_get_sg_table(_obj->base);
+    if (IS_ERR_OR_NULL(xen_obj->sgt)){
+    ret = PTR_ERR(xen_obj->sgt);
+    xen_obj->sgt = NULL;
+    goto fail_put_pages;
+    }
+
+    if (!dma_map_sg(dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents,
+    DMA_BIDIRECTIONAL)) {



Are you using the DMA streaming API as a way to flush the caches?

Yes

Does this mean that GFP_USER isn't making the buffer coherent?


No, it didn't help. I had a question [1] if there are any other 
better way


to achieve the same, but didn't have any response yet. So, I implemented

it via DMA API which helped.


As Gerd says asking on the arm list is probably the best way of finding a
future proof solution and understanding what's going on.

Yes, it seems so


But if you don't get any help there and you end up with the present
solution I suggest you add a comment that this is for flushing the caches
on arm. With the current code one can be led to believe that the driver
uses the dma address somewhere.

Makes sense


What about x86, does the problem exist there?


Yes, but there I could do drm_clflush_pages which is not implemented for ARM

I wonder if you can call dma_unmap_sg() right away since the flushing has
already happened. That would contain this flushing "hack" inside the
gem_create function.

Yes, I was thinking about this "solution" as well


I also suggest calling drm_prime_pages_to_sg() directly to increase
readability, since the check in xen_drm_front_ge

Re: [Xen-devel] [PATCH] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

On 12/19/18 4:10 PM, Gerd Hoffmann wrote:

   Hi,


Sure this actually helps?  It's below 4G in guest physical address
space, so it can be backed by pages which are actually above 4G in host
physical address space ...

Yes, you are right here. This is why I wrote about the IOMMU
and other conditions. E.g. you can have a device which only
expects 32-bit, but thanks to IOMMU it can access pages above
4GiB seamlessly. So, this is why I *hope* that this code *may* help
such devices. Do you think I don't need that and have to remove?

I would try without that, and maybe add a runtime option (module
parameter) later if it turns out some hardware actually needs that.
Devices which can do 32bit DMA only become less and less common these
days.

Good point, will remove then

+    if (!dma_map_sg(dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents,
+    DMA_BIDIRECTIONAL)) {

Are you using the DMA streaming API as a way to flush the caches?

Yes

Does this mean that GFP_USER isn't making the buffer coherent?

No, it didn't help. I had a question [1] if there are any other better way
to achieve the same, but didn't have any response yet. So, I implemented
it via DMA API which helped.

set_pages_array_*() ?

See arch/x86/include/asm/set_memory.h

Well, x86... I am on arm which doesn't define that...

Oh, arm.  Maybe ask on a arm list then.  I know on arm you have to care
about caching a lot more, but that also is where my knowledge ends ...

Using dma_map_sg for cache flushing looks like a sledge hammer approach
to me.

It is. This is why I am so unsure this is way to go

   But maybe it is needed to make xen flush the caches (xen guests
have their own dma mapping implementation, right?  Or is this different
on arm than on x86?).

I'll try to figure out

cheers,
   Gerd


Thank you,
Oleksandr

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

On 12/19/18 3:14 PM, Gerd Hoffmann wrote:

   Hi,


+    mapping = xen_obj->base.filp->f_mapping;
+    mapping_set_gfp_mask(mapping, GFP_USER | __GFP_DMA32);

Let's see if I understand what you're doing:

Here you say that the pages should be DMA accessible for devices that can
only see 4GB.

Yes, your understanding is correct. As we are a para-virtualized device we
do not have strict requirements for 32-bit DMA. But, via dma-buf export,
the buffer we create can be used by real HW, e.g. one can pass-through
real HW devices into a guest domain and they can import our buffer (yes,
they can be IOMMU backed and other conditions may apply).
So, this is why we are limiting to DMA32 here, just to allow more possible
use-cases

Sure this actually helps?  It's below 4G in guest physical address
space, so it can be backed by pages which are actually above 4G in host
physical address space ...


Yes, you are right here. This is why I wrote about the IOMMU

and other conditions. E.g. you can have a device which only

expects 32-bit, but thanks to IOMMU it can access pages above

4GiB seamlessly. So, this is why I *hope* that this code *may* help

such devices. Do you think I don't need that and have to remove?


+    if (!dma_map_sg(dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents,
+    DMA_BIDIRECTIONAL)) {


Are you using the DMA streaming API as a way to flush the caches?

Yes

Does this mean that GFP_USER isn't making the buffer coherent?

No, it didn't help. I had a question [1] if there are any other better way
to achieve the same, but didn't have any response yet. So, I implemented
it via DMA API which helped.

set_pages_array_*() ?

See arch/x86/include/asm/set_memory.h

Well, x86... I am on arm which doesn't define that...

HTH,
   Gerd


Thank you,

Oleksandr


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [XEN][ARM64] PV DRM failing to convert virtual to physical address

[Oleksandr Andrushchenko]

Sorry for top-posting.

Well, from the logs I see that those failures come before PV DRM

even creates any shared buffers which could be the problem:

when you see

(XEN) p2m.c:1456: d2v1: gvirt_to_maddr failed va=0x80001df5205f 
flags=0x1 par=0x809
(XEN) p2m.c:1456: d2v3: gvirt_to_maddr failed va=0x80001df8005f 
flags=0x1 par=0x809


in your logs this at the time when PV DRM had read the configuration and

created shared rings, but it hasn't yet allocated any buffers so far 
which might


be the source of p2m complaints. So, unfortunately, I think you would 
need to debug more


to understand from where those come.

And, please rebuild the PV DRM as a module and collect DRM logs in DomU.

On 12/19/18 7:45 AM, Vikram K wrote:

Hello Oleksandr,

Please find the attached log file.

Could please provide some pointers on how test DomU display.


modetest, weston - whatever you have in your rootfs
On Tue, Dec 18, 2018 at 12:06 PM Oleksandr Andrushchenko 
mailto:andr2...@gmail.com>> wrote:

Hello, Vikram!
>   * We are using 64 bit arm platform.
>   * Linux 4.20 Kernel in DomU with PV DRM front-end drivers.
>
>> Do you have [1] enabled in your DomD kernel?
 Earlier this configuration was not enabled. Enabled it now.


Well, this just adds zero-copying, but shouldn't change the behavior so much

>
>   * Xen-4.12 unstable version.
>
>> We never tested PV DRM with 4.12 so far, did you try with 4.10 for 
example?
 Xen-4.8 is working on our platform. Xen-4.9 to Xen-4.11 version 
fails while initializing CPU on our platform. Recently
 we received patch from Julien for Xen-4.12 unstable version, 
which has fix. Tried to apply patch on Xen-4.10 stable facing build issue.


  This is where we are at the moment and have all PV drivers 
running smoothly


>   * Able to build displ_be application and also its dependencies.
>
> Added below configuration in Domain-U config file.*|
> |*
> *|vdispl = [ 
'backend=Domain-0,be-alloc=0,connectors=HDMI-A-1:1920x1080']|*

>
>> Do you really have DomD as your configuration says?
Sorry it not DomD it is Domain-0.

>> Not Dom0 running the backend?
 Domain-0 is running in Dom0 and debian is running as DomU.

> Before launching the DomU ran the displ_be application in the
> background.
>  $ displ_be -m DRM -v *:Debug &
Please add display back-end logs

>
> Started Domain-U.
> $ xl create -c debian.cfg
>
>> Could you please build the PV DRM driver as module,
>> Built as module.

then run the following commands:

echo 0xff > /sys/module/drm/parameters/debug
echo 8 > /proc/sys/kernel/printk

This needs to be done *before* you insmod the PV DRM driver


>> and get back with DRM logs in DomU?
Please find the attached log file
--
Thanks & Regards
Vikram KS

This message contains confidential information and is intended only 
for the individual(s) named.If you are not the intended recipient, you 
are notified that disclosing, copying, distributing or taking any 
action in reliance on the contents of this mail and attached file/s is 
strictly prohibited. Please notify the sender immediately and delete 
this e-mail from your system. E-mail transmission cannot be guaranteed 
to be secured or error-free as information could be intercepted, 
corrupted, lost, destroyed, arrive late or incomplete, or contain 
viruses. The sender therefore does not accept liability for any errors 
or omissions in the contents of this message, which arise as a result 
of e-mail transmission.




___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

On 12/18/18 9:20 PM, Noralf Trønnes wrote:


Den 27.11.2018 11.32, skrev Oleksandr Andrushchenko:

From: Oleksandr Andrushchenko 

When GEM backing storage is allocated with drm_gem_get_pages
the backing pages may be cached, thus making it possible that
the backend sees only partial content of the buffer which may
lead to screen artifacts. Make sure that the frontend's
memory is coherent and the backend always sees correct display
buffer content.

Fixes: c575b7eeb89f ("drm/xen-front: Add support for Xen PV display 
frontend")


Signed-off-by: Oleksandr Andrushchenko 


---
  drivers/gpu/drm/xen/xen_drm_front_gem.c | 62 +++--
  1 file changed, 48 insertions(+), 14 deletions(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
b/drivers/gpu/drm/xen/xen_drm_front_gem.c

index 47ff019d3aef..c592735e49d2 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -33,8 +33,11 @@ struct xen_gem_object {
  /* set for buffers allocated by the backend */
  bool be_alloc;
  -    /* this is for imported PRIME buffer */
-    struct sg_table *sgt_imported;
+    /*
+ * this is for imported PRIME buffer or the one allocated via
+ * drm_gem_get_pages.
+ */
+    struct sg_table *sgt;
  };
    static inline struct xen_gem_object *
@@ -77,10 +80,21 @@ static struct xen_gem_object 
*gem_create_obj(struct drm_device *dev,

  return xen_obj;
  }
  +struct sg_table *xen_drm_front_gem_get_sg_table(struct 
drm_gem_object *gem_obj)

+{
+    struct xen_gem_object *xen_obj = to_xen_gem_obj(gem_obj);
+
+    if (!xen_obj->pages)
+    return ERR_PTR(-ENOMEM);
+
+    return drm_prime_pages_to_sg(xen_obj->pages, xen_obj->num_pages);
+}
+
  static struct xen_gem_object *gem_create(struct drm_device *dev, 
size_t size)

  {
  struct xen_drm_front_drm_info *drm_info = dev->dev_private;
  struct xen_gem_object *xen_obj;
+    struct address_space *mapping;
  int ret;
    size = round_up(size, PAGE_SIZE);
@@ -113,10 +127,14 @@ static struct xen_gem_object *gem_create(struct 
drm_device *dev, size_t size)

  xen_obj->be_alloc = true;
  return xen_obj;
  }
+
  /*
   * need to allocate backing pages now, so we can share those
   * with the backend
   */



Let's see if I understand what you're doing:

Here you say that the pages should be DMA accessible for devices that can
only see 4GB.


Yes, your understanding is correct. As we are a para-virtualized device we

do not have strict requirements for 32-bit DMA. But, via dma-buf export,

the buffer we create can be used by real HW, e.g. one can pass-through

real HW devices into a guest domain and they can import our buffer (yes,

they can be IOMMU backed and other conditions may apply).

So, this is why we are limiting to DMA32 here, just to allow more possible

use-cases




+    mapping = xen_obj->base.filp->f_mapping;
+    mapping_set_gfp_mask(mapping, GFP_USER | __GFP_DMA32);
+
  xen_obj->num_pages = DIV_ROUND_UP(size, PAGE_SIZE);
  xen_obj->pages = drm_gem_get_pages(_obj->base);
  if (IS_ERR_OR_NULL(xen_obj->pages)) {
@@ -125,8 +143,27 @@ static struct xen_gem_object *gem_create(struct 
drm_device *dev, size_t size)

  goto fail;
  }
  +    xen_obj->sgt = xen_drm_front_gem_get_sg_table(_obj->base);
+    if (IS_ERR_OR_NULL(xen_obj->sgt)){
+    ret = PTR_ERR(xen_obj->sgt);
+    xen_obj->sgt = NULL;
+    goto fail_put_pages;
+    }
+
+    if (!dma_map_sg(dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents,
+    DMA_BIDIRECTIONAL)) {



Are you using the DMA streaming API as a way to flush the caches?

Yes

Does this mean that GFP_USER isn't making the buffer coherent?


No, it didn't help. I had a question [1] if there are any other better way

to achieve the same, but didn't have any response yet. So, I implemented

it via DMA API which helped.



Noralf.


+    ret = -EFAULT;
+    goto fail_free_sgt;
+    }
+
  return xen_obj;
  +fail_free_sgt:
+    sg_free_table(xen_obj->sgt);
+    xen_obj->sgt = NULL;
+fail_put_pages:
+    drm_gem_put_pages(_obj->base, xen_obj->pages, true, false);
+    xen_obj->pages = NULL;
  fail:
  DRM_ERROR("Failed to allocate buffer with size %zu\n", size);
  return ERR_PTR(ret);
@@ -149,7 +186,7 @@ void 
xen_drm_front_gem_free_object_unlocked(struct drm_gem_object *gem_obj)

  struct xen_gem_object *xen_obj = to_xen_gem_obj(gem_obj);
    if (xen_obj->base.import_attach) {
-    drm_prime_gem_destroy(_obj->base, xen_obj->sgt_imported);
+    drm_prime_gem_destroy(_obj->base, xen_obj->sgt);
  gem_free_pages_array(xen_obj);
  } else {
  if (xen_obj->pages) {
@@ -158,6 +195,13 @@ void 
xen_drm_front_gem_free_object_unlocked(struct drm_gem_object *gem_obj)

  xen_ob

Re: [Xen-devel] [PATCH v2 2/3] drm/xen-front: Use Xen common shared buffer implementation

[Oleksandr Andrushchenko]

On 12/18/18 20:31, Boris Ostrovsky wrote:

On 12/18/18 11:15 AM, Noralf Trønnes wrote:

Den 30.11.2018 08.42, skrev Oleksandr Andrushchenko:

From: Oleksandr Andrushchenko 

Use page directory based shared buffer implementation
now available as common code for Xen frontend drivers.

Remove flushing of shared buffer on page flip as this
workaround needs a proper fix.

Signed-off-by: Oleksandr Andrushchenko

---

Reviewed-by: Noralf Trønnes 

Thank you, Noralf!


Now that all 3 have been acked/reviewed

Applied to for-linus-4.21

Thank you, Boris, Juergen!




___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 2/3] drm/xen-front: Use Xen common shared buffer implementation

[Oleksandr Andrushchenko]

On 12/17/18 5:26 PM, Boris Ostrovsky wrote:

On 12/17/18 10:03 AM, Oleksandr Andrushchenko wrote:

On 12/17/18 4:52 PM, Boris Ostrovsky wrote:

On 12/17/18 5:19 AM, Oleksandr Andrushchenko wrote:

Hello, Juergen, Boris!

As this DRM part of the series is the only one which needs ack/nack

(and it might take quite some time to complete) could we please

merge the patches 1 and 3 now that already have ack/r-b?



TBH I am not sure it makes sense to do this without the second patch.
Refactoring (and IIUIC this series is purely refactoring --- is it not?)
is done to reduce amount of code, and with only first and third patch we
end up with quite a significant increase in the number of LoC. (I am
going purely by diffstat)

Of course, the other reason for refactoring is to eliminate code
duplication, but without second patch that will not happen.

Agree, but this is the basis for the new pv camera frontend

I am working on now [1], so even if we do not remove the code from DRM

then we at least do not add it to the camera driver


Since 1 and 3 are already ACKed you should be able to start the camera
series with these two patches as pre-requisites even if patch 2 is still
stalled by the time your camera code is posted (which I assume will be
4.22 or later).

Agreed, maybe by that time DRM part will also get its r-b/ack



-boris



-boris

Thank you,

Oleksandr

[1]
https://github.com/andr2000/linux/blob/camera_front_v1/drivers/media/xen/Kconfig#L6



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [XEN][ARM64] PV DRM failing to convert virtual to physical address

[Oleksandr Andrushchenko]

Hello, Vikram!

First of all what makes you think this is related to PV DRM?

Please see inline for more comments

Thank you,

Oleksandr

On 12/18/18 7:26 AM, Vikram K wrote:

Hello,

When PV DRM is enabled and Domain-U is started xen is not able to 
converted virtual address to physical. Please provide input on 
resolving this issue.


Please find the details below.

  * We are using 64 bit arm platform.
  * Linux 4.20 Kernel in DomU with PV DRM front-end drivers.


Do you have [1] enabled in your DomD kernel?


  * Xen-4.12 unstable version.


We never tested PV DRM with 4.12 so far, did you try with 4.10 for example?

This is where we are at the moment and have all PV drivers running smoothly


  * Able to build displ_be application and also its dependencies.

Added below configuration in Domain-U config file.*|
|*
*|vdispl = [ 'backend=DomD,be-alloc=0,connectors=HDMI-A-1:1920x1080']|*


Do you really have DomD as your configuration says?

Not Dom0 running the backend?

Before launching the application ran the displ_be application in the 
background.

 $ displ_be -m DRM -v *:Debug

Please add display backend logs


Started Domain-U.
$ xl create -c debian.cfg


Could you please build the PV DRM driver as module,

then run the following commands:

echo 0xff > /sys/module/drm/parameters/debug
echo 8 > /proc/sys/kernel/printk

and get back with DRM logs in DomU?


Getting below logs.
Welcome to Debian GNU/Linux 9 (stretch)!

(XEN) p2m.c:1456: d2v2: gvirt_to_maddr failed va=0x80003df6c050 
flags=0x1 par=0x809

[    2.601162] systemd[1]: Set hostname to .
[    2.613526] random: systemd: uninitialized urandom read (16 bytes 
re(XEN) p2m.c:1456: d2v2: gvirt_to_maddr failed va=0x80003df6c05f 
flags=0x1 par=0x809

ad)
(XEN) p2m.c:1456: d2v2: gvirt_to_maddr failed va=0x80003df6c05f 
flags=0x1 par=0x809
(XEN) p2m.c:1456: d2v2: gvirt_to_maddr failed va=0x80003df6c050 
flags=0x1 par=0x809
(XEN) p2m.c:1456: d2v2: gvirt_to_maddr failed va=0x80003df6c05f 
flags=0x1 par=0x809



[snip]



--
Thanks & Regards
Vikram KS

This message contains confidential information and is intended only 
for the individual(s) named.If you are not the intended recipient, you 
are notified that disclosing, copying, distributing or taking any 
action in reliance on the contents of this mail and attached file/s is 
strictly prohibited. Please notify the sender immediately and delete 
this e-mail from your system. E-mail transmission cannot be guaranteed 
to be secured or error-free as information could be intercepted, 
corrupted, lost, destroyed, arrive late or incomplete, or contain 
viruses. The sender therefore does not accept liability for any errors 
or omissions in the contents of this message, which arise as a result 
of e-mail transmission.



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[1] 
https://elixir.bootlin.com/linux/v4.20-rc7/source/drivers/xen/Kconfig#L175


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 2/3] drm/xen-front: Use Xen common shared buffer implementation

[Oleksandr Andrushchenko]

On 12/17/18 4:52 PM, Boris Ostrovsky wrote:

On 12/17/18 5:19 AM, Oleksandr Andrushchenko wrote:

Hello, Juergen, Boris!

As this DRM part of the series is the only one which needs ack/nack

(and it might take quite some time to complete) could we please

merge the patches 1 and 3 now that already have ack/r-b?




TBH I am not sure it makes sense to do this without the second patch.
Refactoring (and IIUIC this series is purely refactoring --- is it not?)
is done to reduce amount of code, and with only first and third patch we
end up with quite a significant increase in the number of LoC. (I am
going purely by diffstat)

Of course, the other reason for refactoring is to eliminate code
duplication, but without second patch that will not happen.


Agree, but this is the basis for the new pv camera frontend

I am working on now [1], so even if we do not remove the code from DRM

then we at least do not add it to the camera driver


-boris


Thank you,

Oleksandr

[1] 
https://github.com/andr2000/linux/blob/camera_front_v1/drivers/media/xen/Kconfig#L6



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 2/3] drm/xen-front: Use Xen common shared buffer implementation

[Oleksandr Andrushchenko]

Hello, Juergen, Boris!

As this DRM part of the series is the only one which needs ack/nack

(and it might take quite some time to complete) could we please

merge the patches 1 and 3 now that already have ack/r-b?

Thank you,

Oleksandr

On 12/13/18 12:16 PM, Oleksandr Andrushchenko wrote:

bump

On 12/5/18 10:20 AM, Oleksandr Andrushchenko wrote:

Hello, Daniel!

Could you please ack/nack the patch, so either we can merge the

series or I can address your comments if any

Thank you,

Oleksandr

On 11/30/18 9:42 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

Use page directory based shared buffer implementation
now available as common code for Xen frontend drivers.

Remove flushing of shared buffer on page flip as this
workaround needs a proper fix.

Signed-off-by: Oleksandr Andrushchenko 


---
  drivers/gpu/drm/xen/Kconfig   |   1 +
  drivers/gpu/drm/xen/Makefile  |   1 -
  drivers/gpu/drm/xen/xen_drm_front.c   |  65 ++--
  drivers/gpu/drm/xen/xen_drm_front_gem.c   |   1 -
  drivers/gpu/drm/xen/xen_drm_front_shbuf.c | 414 
--

  drivers/gpu/drm/xen/xen_drm_front_shbuf.h |  64 
  6 files changed, 26 insertions(+), 520 deletions(-)
  delete mode 100644 drivers/gpu/drm/xen/xen_drm_front_shbuf.c
  delete mode 100644 drivers/gpu/drm/xen/xen_drm_front_shbuf.h

diff --git a/drivers/gpu/drm/xen/Kconfig b/drivers/gpu/drm/xen/Kconfig
index 4cca160782ab..f969d486855d 100644
--- a/drivers/gpu/drm/xen/Kconfig
+++ b/drivers/gpu/drm/xen/Kconfig
@@ -12,6 +12,7 @@ config DRM_XEN_FRONTEND
  select DRM_KMS_HELPER
  select VIDEOMODE_HELPERS
  select XEN_XENBUS_FRONTEND
+    select XEN_FRONT_PGDIR_SHBUF
  help
    Choose this option if you want to enable a para-virtualized
    frontend DRM/KMS driver for Xen guest OSes.
diff --git a/drivers/gpu/drm/xen/Makefile 
b/drivers/gpu/drm/xen/Makefile

index 712afff5ffc3..825905f67faa 100644
--- a/drivers/gpu/drm/xen/Makefile
+++ b/drivers/gpu/drm/xen/Makefile
@@ -4,7 +4,6 @@ drm_xen_front-objs := xen_drm_front.o \
    xen_drm_front_kms.o \
    xen_drm_front_conn.o \
    xen_drm_front_evtchnl.o \
-  xen_drm_front_shbuf.o \
    xen_drm_front_cfg.o \
    xen_drm_front_gem.o
  diff --git a/drivers/gpu/drm/xen/xen_drm_front.c 
b/drivers/gpu/drm/xen/xen_drm_front.c

index 6b6d5ab82ec3..4d3d36fc3a5d 100644
--- a/drivers/gpu/drm/xen/xen_drm_front.c
+++ b/drivers/gpu/drm/xen/xen_drm_front.c
@@ -19,6 +19,7 @@
  #include 
  #include 
  +#include 
  #include 
    #include "xen_drm_front.h"
@@ -26,28 +27,20 @@
  #include "xen_drm_front_evtchnl.h"
  #include "xen_drm_front_gem.h"
  #include "xen_drm_front_kms.h"
-#include "xen_drm_front_shbuf.h"
    struct xen_drm_front_dbuf {
  struct list_head list;
  u64 dbuf_cookie;
  u64 fb_cookie;
-    struct xen_drm_front_shbuf *shbuf;
+
+    struct xen_front_pgdir_shbuf shbuf;
  };
  -static int dbuf_add_to_list(struct xen_drm_front_info *front_info,
-    struct xen_drm_front_shbuf *shbuf, u64 dbuf_cookie)
+static void dbuf_add_to_list(struct xen_drm_front_info *front_info,
+ struct xen_drm_front_dbuf *dbuf, u64 dbuf_cookie)
  {
-    struct xen_drm_front_dbuf *dbuf;
-
-    dbuf = kzalloc(sizeof(*dbuf), GFP_KERNEL);
-    if (!dbuf)
-    return -ENOMEM;
-
  dbuf->dbuf_cookie = dbuf_cookie;
-    dbuf->shbuf = shbuf;
  list_add(>list, _info->dbuf_list);
-    return 0;
  }
    static struct xen_drm_front_dbuf *dbuf_get(struct list_head 
*dbuf_list,
@@ -62,15 +55,6 @@ static struct xen_drm_front_dbuf *dbuf_get(struct 
list_head *dbuf_list,

  return NULL;
  }
  -static void dbuf_flush_fb(struct list_head *dbuf_list, u64 
fb_cookie)

-{
-    struct xen_drm_front_dbuf *buf, *q;
-
-    list_for_each_entry_safe(buf, q, dbuf_list, list)
-    if (buf->fb_cookie == fb_cookie)
-    xen_drm_front_shbuf_flush(buf->shbuf);
-}
-
  static void dbuf_free(struct list_head *dbuf_list, u64 dbuf_cookie)
  {
  struct xen_drm_front_dbuf *buf, *q;
@@ -78,8 +62,8 @@ static void dbuf_free(struct list_head *dbuf_list, 
u64 dbuf_cookie)

  list_for_each_entry_safe(buf, q, dbuf_list, list)
  if (buf->dbuf_cookie == dbuf_cookie) {
  list_del(>list);
-    xen_drm_front_shbuf_unmap(buf->shbuf);
-    xen_drm_front_shbuf_free(buf->shbuf);
+    xen_front_pgdir_shbuf_unmap(>shbuf);
+    xen_front_pgdir_shbuf_free(>shbuf);
  kfree(buf);
  break;
  }
@@ -91,8 +75,8 @@ static void dbuf_free_all(struct list_head 
*dbuf_list)

    list_for_each_entry_safe(buf, q, dbuf_list, list) {
  list_del(>list);
-    xen_drm_front_shbuf_unmap(buf->shbuf);
-    xen_drm_front_shbuf_free(buf->shbuf);
+    xen_front_pgdir_shbuf_unmap(>shbuf);
+   

Re: [Xen-devel] [PATCH] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

On 12/14/18 10:35 AM, Daniel Vetter wrote:

On Fri, Dec 14, 2018 at 09:09:45AM +0200, Oleksandr Andrushchenko wrote:

On 12/13/18 5:48 PM, Daniel Vetter wrote:

On Thu, Dec 13, 2018 at 12:17:54PM +0200, Oleksandr Andrushchenko wrote:

Daniel, could you please comment?

Cross-revieweing someone else's stuff would scale better,

fair enough

   I don't think
I'll get around to anything before next year.

I put you on CC explicitly because you had comments on other patch [1]

and this one tries to solve the issue raised (I tried to figure out

at [2] if this is the way to go, but it seems I have no alternative here).

While at it [3] (I hope) addresses your comments and the series just

needs your single ack/nack to get in: all the rest ack/r-b are already

there. Do you mind looking at it?

As mentioned, much better if you aim for more per review with others, not
just me. And all that dma coherency stuff isn't something a really
understand all that well (I just know we have lots of pain). For options
maybe work together with Gerd Hoffman or Noralf Tronnes, I think either
has some patches pending that also need some review.


Fair enough,

thank you


-Daniel


-Daniel

Thank you very much for your time,

Oleksandr


Thank you

On 11/27/18 12:32 PM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

When GEM backing storage is allocated with drm_gem_get_pages
the backing pages may be cached, thus making it possible that
the backend sees only partial content of the buffer which may
lead to screen artifacts. Make sure that the frontend's
memory is coherent and the backend always sees correct display
buffer content.

Fixes: c575b7eeb89f ("drm/xen-front: Add support for Xen PV display frontend")

Signed-off-by: Oleksandr Andrushchenko 
---
drivers/gpu/drm/xen/xen_drm_front_gem.c | 62 +++--
1 file changed, 48 insertions(+), 14 deletions(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index 47ff019d3aef..c592735e49d2 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -33,8 +33,11 @@ struct xen_gem_object {
/* set for buffers allocated by the backend */
bool be_alloc;
-   /* this is for imported PRIME buffer */
-   struct sg_table *sgt_imported;
+   /*
+* this is for imported PRIME buffer or the one allocated via
+* drm_gem_get_pages.
+*/
+   struct sg_table *sgt;
};
static inline struct xen_gem_object *
@@ -77,10 +80,21 @@ static struct xen_gem_object *gem_create_obj(struct 
drm_device *dev,
return xen_obj;
}
+struct sg_table *xen_drm_front_gem_get_sg_table(struct drm_gem_object *gem_obj)
+{
+   struct xen_gem_object *xen_obj = to_xen_gem_obj(gem_obj);
+
+   if (!xen_obj->pages)
+   return ERR_PTR(-ENOMEM);
+
+   return drm_prime_pages_to_sg(xen_obj->pages, xen_obj->num_pages);
+}
+
static struct xen_gem_object *gem_create(struct drm_device *dev, size_t 
size)
{
struct xen_drm_front_drm_info *drm_info = dev->dev_private;
struct xen_gem_object *xen_obj;
+   struct address_space *mapping;
int ret;
size = round_up(size, PAGE_SIZE);
@@ -113,10 +127,14 @@ static struct xen_gem_object *gem_create(struct 
drm_device *dev, size_t size)
xen_obj->be_alloc = true;
return xen_obj;
}
+
/*
 * need to allocate backing pages now, so we can share those
 * with the backend
 */
+   mapping = xen_obj->base.filp->f_mapping;
+   mapping_set_gfp_mask(mapping, GFP_USER | __GFP_DMA32);
+
xen_obj->num_pages = DIV_ROUND_UP(size, PAGE_SIZE);
xen_obj->pages = drm_gem_get_pages(_obj->base);
if (IS_ERR_OR_NULL(xen_obj->pages)) {
@@ -125,8 +143,27 @@ static struct xen_gem_object *gem_create(struct drm_device 
*dev, size_t size)
goto fail;
}
+   xen_obj->sgt = xen_drm_front_gem_get_sg_table(_obj->base);
+   if (IS_ERR_OR_NULL(xen_obj->sgt)){
+   ret = PTR_ERR(xen_obj->sgt);
+   xen_obj->sgt = NULL;
+   goto fail_put_pages;
+   }
+
+   if (!dma_map_sg(dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents,
+   DMA_BIDIRECTIONAL)) {
+   ret = -EFAULT;
+   goto fail_free_sgt;
+   }
+
return xen_obj;
+fail_free_sgt:
+   sg_free_table(xen_obj->sgt);
+   xen_obj->sgt = NULL;
+fail_put_pages:
+   drm_gem_put_pages(_obj->base, xen_obj->pages, true, false);
+   xen_obj->pages = NULL;
fail:
DRM_ERROR("Failed to allocate buffer with size %zu\n", size);
return ERR_PTR(ret);
@@ -149,7 +186,7 @@ void xen_drm_front_gem_free_object_unlocked(struct 
drm_gem_object *gem_obj)
struct xen_gem_object *xen_obj = to_xen_

Re: [Xen-devel] [PATCH v3 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

Hello, Hans!

Thank you for reviewing, please find my answers inline

On 12/14/18 2:14 PM, Hans Verkuil wrote:

Hi Oleksandr,

This is looking a lot better than v2. I do have a few remaining comments about
some things that are a bit unclear to me.

On 12/12/18 10:49 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
  - pixel formats
  - resolutions
  - frame rates
2. Support basic camera controls:
  - contrast
  - brightness
  - hue
  - saturation
3. Support streaming control

Signed-off-by: Oleksandr Andrushchenko 
---
  xen/include/public/io/cameraif.h | 1374 ++
  1 file changed, 1374 insertions(+)
  create mode 100644 xen/include/public/io/cameraif.h

diff --git a/xen/include/public/io/cameraif.h b/xen/include/public/io/cameraif.h
new file mode 100644
index ..9aae0f47743b
--- /dev/null
+++ b/xen/include/public/io/cameraif.h
@@ -0,0 +1,1374 @@
+/**
+ * cameraif.h
+ *
+ * Unified camera device I/O interface for Xen guest OSes.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright (C) 2018 EPAM Systems Inc.
+ *
+ * Author: Oleksandr Andrushchenko 
+ */
+
+#ifndef __XEN_PUBLIC_IO_CAMERAIF_H__
+#define __XEN_PUBLIC_IO_CAMERAIF_H__
+
+#include "ring.h"
+#include "../grant_table.h"
+
+/*
+ **
+ *   Protocol version
+ **
+ */
+#define XENCAMERA_PROTOCOL_VERSION "1"
+
+/*
+ **
+ *  Feature and Parameter Negotiation
+ **
+ *
+ * Front->back notifications: when enqueuing a new request, sending a
+ * notification can be made conditional on xencamera_req (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Backends must set
+ * xencamera_req appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
+ *
+ * Back->front notifications: when enqueuing a new response, sending a
+ * notification can be made conditional on xencamera_resp (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Frontends must set
+ * xencamera_resp appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
+ *
+ * The two halves of a para-virtual camera driver utilize nodes within
+ * XenStore to communicate capabilities and to negotiate operating parameters.
+ * This section enumerates these nodes which reside in the respective front and
+ * backend portions of XenStore, following the XenBus convention.
+ *
+ * All data in XenStore is stored as strings. Nodes specifying numeric
+ * values are encoded in decimal. Integer value ranges listed below are
+ * expressed as fixed sized integer types capable of storing the conversion
+ * of a properly formatted node string, without loss of information.
+ *
+ **
+ *Example configuration
+ **
+ *
+ * This is an example of backend and frontend configuration:
+ *
+ *- Backend ---
+ *
+ * /local/domain/0/backend/vcamera/1/0/frontend-id = "1"
+ 

Re: [Xen-devel] [PATCH v3 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

PFA the diff between v2 and v3 for your convenience

On 12/12/18 11:49 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
  - pixel formats
  - resolutions
  - frame rates
2. Support basic camera controls:
  - contrast
  - brightness
  - hue
  - saturation
3. Support streaming control

Signed-off-by: Oleksandr Andrushchenko 
---
  xen/include/public/io/cameraif.h | 1374 ++
  1 file changed, 1374 insertions(+)
  create mode 100644 xen/include/public/io/cameraif.h

diff --git a/xen/include/public/io/cameraif.h b/xen/include/public/io/cameraif.h
new file mode 100644
index ..9aae0f47743b
--- /dev/null
+++ b/xen/include/public/io/cameraif.h
@@ -0,0 +1,1374 @@
+/**
+ * cameraif.h
+ *
+ * Unified camera device I/O interface for Xen guest OSes.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright (C) 2018 EPAM Systems Inc.
+ *
+ * Author: Oleksandr Andrushchenko 
+ */
+
+#ifndef __XEN_PUBLIC_IO_CAMERAIF_H__
+#define __XEN_PUBLIC_IO_CAMERAIF_H__
+
+#include "ring.h"
+#include "../grant_table.h"
+
+/*
+ **
+ *   Protocol version
+ **
+ */
+#define XENCAMERA_PROTOCOL_VERSION "1"
+
+/*
+ **
+ *  Feature and Parameter Negotiation
+ **
+ *
+ * Front->back notifications: when enqueuing a new request, sending a
+ * notification can be made conditional on xencamera_req (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Backends must set
+ * xencamera_req appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
+ *
+ * Back->front notifications: when enqueuing a new response, sending a
+ * notification can be made conditional on xencamera_resp (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Frontends must set
+ * xencamera_resp appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
+ *
+ * The two halves of a para-virtual camera driver utilize nodes within
+ * XenStore to communicate capabilities and to negotiate operating parameters.
+ * This section enumerates these nodes which reside in the respective front and
+ * backend portions of XenStore, following the XenBus convention.
+ *
+ * All data in XenStore is stored as strings. Nodes specifying numeric
+ * values are encoded in decimal. Integer value ranges listed below are
+ * expressed as fixed sized integer types capable of storing the conversion
+ * of a properly formatted node string, without loss of information.
+ *
+ **
+ *Example configuration
+ **
+ *
+ * This is an example of backend and frontend configuration:
+ *
+ *- Backend ---
+ *
+ * /local/domain/0/backend/vcamera/1/0/frontend-id = "1"
+ * /local/domain/0/backend/vcamera/1/0/frontend = 
"/local/domain/1/device/vcamera/0"
+ * /local/domain/0/backend/vcamera/1/0/state = "4"
+ * /local/domain/0/

Re: [Xen-devel] [PATCH] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

On 12/13/18 5:48 PM, Daniel Vetter wrote:

On Thu, Dec 13, 2018 at 12:17:54PM +0200, Oleksandr Andrushchenko wrote:

Daniel, could you please comment?

Cross-revieweing someone else's stuff would scale better,

fair enough

  I don't think
I'll get around to anything before next year.


I put you on CC explicitly because you had comments on other patch [1]

and this one tries to solve the issue raised (I tried to figure out

at [2] if this is the way to go, but it seems I have no alternative here).

While at it [3] (I hope) addresses your comments and the series just

needs your single ack/nack to get in: all the rest ack/r-b are already

there. Do you mind looking at it?


-Daniel


Thank you very much for your time,

Oleksandr


Thank you

On 11/27/18 12:32 PM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

When GEM backing storage is allocated with drm_gem_get_pages
the backing pages may be cached, thus making it possible that
the backend sees only partial content of the buffer which may
lead to screen artifacts. Make sure that the frontend's
memory is coherent and the backend always sees correct display
buffer content.

Fixes: c575b7eeb89f ("drm/xen-front: Add support for Xen PV display frontend")

Signed-off-by: Oleksandr Andrushchenko 
---
   drivers/gpu/drm/xen/xen_drm_front_gem.c | 62 +++--
   1 file changed, 48 insertions(+), 14 deletions(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index 47ff019d3aef..c592735e49d2 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -33,8 +33,11 @@ struct xen_gem_object {
/* set for buffers allocated by the backend */
bool be_alloc;
-   /* this is for imported PRIME buffer */
-   struct sg_table *sgt_imported;
+   /*
+* this is for imported PRIME buffer or the one allocated via
+* drm_gem_get_pages.
+*/
+   struct sg_table *sgt;
   };
   static inline struct xen_gem_object *
@@ -77,10 +80,21 @@ static struct xen_gem_object *gem_create_obj(struct 
drm_device *dev,
return xen_obj;
   }
+struct sg_table *xen_drm_front_gem_get_sg_table(struct drm_gem_object *gem_obj)
+{
+   struct xen_gem_object *xen_obj = to_xen_gem_obj(gem_obj);
+
+   if (!xen_obj->pages)
+   return ERR_PTR(-ENOMEM);
+
+   return drm_prime_pages_to_sg(xen_obj->pages, xen_obj->num_pages);
+}
+
   static struct xen_gem_object *gem_create(struct drm_device *dev, size_t size)
   {
struct xen_drm_front_drm_info *drm_info = dev->dev_private;
struct xen_gem_object *xen_obj;
+   struct address_space *mapping;
int ret;
size = round_up(size, PAGE_SIZE);
@@ -113,10 +127,14 @@ static struct xen_gem_object *gem_create(struct 
drm_device *dev, size_t size)
xen_obj->be_alloc = true;
return xen_obj;
}
+
/*
 * need to allocate backing pages now, so we can share those
 * with the backend
 */
+   mapping = xen_obj->base.filp->f_mapping;
+   mapping_set_gfp_mask(mapping, GFP_USER | __GFP_DMA32);
+
xen_obj->num_pages = DIV_ROUND_UP(size, PAGE_SIZE);
xen_obj->pages = drm_gem_get_pages(_obj->base);
if (IS_ERR_OR_NULL(xen_obj->pages)) {
@@ -125,8 +143,27 @@ static struct xen_gem_object *gem_create(struct drm_device 
*dev, size_t size)
goto fail;
}
+   xen_obj->sgt = xen_drm_front_gem_get_sg_table(_obj->base);
+   if (IS_ERR_OR_NULL(xen_obj->sgt)){
+   ret = PTR_ERR(xen_obj->sgt);
+   xen_obj->sgt = NULL;
+   goto fail_put_pages;
+   }
+
+   if (!dma_map_sg(dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents,
+   DMA_BIDIRECTIONAL)) {
+   ret = -EFAULT;
+   goto fail_free_sgt;
+   }
+
return xen_obj;
+fail_free_sgt:
+   sg_free_table(xen_obj->sgt);
+   xen_obj->sgt = NULL;
+fail_put_pages:
+   drm_gem_put_pages(_obj->base, xen_obj->pages, true, false);
+   xen_obj->pages = NULL;
   fail:
DRM_ERROR("Failed to allocate buffer with size %zu\n", size);
return ERR_PTR(ret);
@@ -149,7 +186,7 @@ void xen_drm_front_gem_free_object_unlocked(struct 
drm_gem_object *gem_obj)
struct xen_gem_object *xen_obj = to_xen_gem_obj(gem_obj);
if (xen_obj->base.import_attach) {
-   drm_prime_gem_destroy(_obj->base, xen_obj->sgt_imported);
+   drm_prime_gem_destroy(_obj->base, xen_obj->sgt);
gem_free_pages_array(xen_obj);
} else {
if (xen_obj->pages) {
@@ -158,6 +195,13 @@ void xen_drm_front_gem_free_object_unlocked(struct 
drm_gem_object *gem_obj)
  

Re: [Xen-devel] [PATCH v2 2/3] drm/xen-front: Use Xen common shared buffer implementation

[Oleksandr Andrushchenko]

bump

On 12/5/18 10:20 AM, Oleksandr Andrushchenko wrote:

Hello, Daniel!

Could you please ack/nack the patch, so either we can merge the

series or I can address your comments if any

Thank you,

Oleksandr

On 11/30/18 9:42 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

Use page directory based shared buffer implementation
now available as common code for Xen frontend drivers.

Remove flushing of shared buffer on page flip as this
workaround needs a proper fix.

Signed-off-by: Oleksandr Andrushchenko 


---
  drivers/gpu/drm/xen/Kconfig   |   1 +
  drivers/gpu/drm/xen/Makefile  |   1 -
  drivers/gpu/drm/xen/xen_drm_front.c   |  65 ++--
  drivers/gpu/drm/xen/xen_drm_front_gem.c   |   1 -
  drivers/gpu/drm/xen/xen_drm_front_shbuf.c | 414 --
  drivers/gpu/drm/xen/xen_drm_front_shbuf.h |  64 
  6 files changed, 26 insertions(+), 520 deletions(-)
  delete mode 100644 drivers/gpu/drm/xen/xen_drm_front_shbuf.c
  delete mode 100644 drivers/gpu/drm/xen/xen_drm_front_shbuf.h

diff --git a/drivers/gpu/drm/xen/Kconfig b/drivers/gpu/drm/xen/Kconfig
index 4cca160782ab..f969d486855d 100644
--- a/drivers/gpu/drm/xen/Kconfig
+++ b/drivers/gpu/drm/xen/Kconfig
@@ -12,6 +12,7 @@ config DRM_XEN_FRONTEND
  select DRM_KMS_HELPER
  select VIDEOMODE_HELPERS
  select XEN_XENBUS_FRONTEND
+    select XEN_FRONT_PGDIR_SHBUF
  help
    Choose this option if you want to enable a para-virtualized
    frontend DRM/KMS driver for Xen guest OSes.
diff --git a/drivers/gpu/drm/xen/Makefile b/drivers/gpu/drm/xen/Makefile
index 712afff5ffc3..825905f67faa 100644
--- a/drivers/gpu/drm/xen/Makefile
+++ b/drivers/gpu/drm/xen/Makefile
@@ -4,7 +4,6 @@ drm_xen_front-objs := xen_drm_front.o \
    xen_drm_front_kms.o \
    xen_drm_front_conn.o \
    xen_drm_front_evtchnl.o \
-  xen_drm_front_shbuf.o \
    xen_drm_front_cfg.o \
    xen_drm_front_gem.o
  diff --git a/drivers/gpu/drm/xen/xen_drm_front.c 
b/drivers/gpu/drm/xen/xen_drm_front.c

index 6b6d5ab82ec3..4d3d36fc3a5d 100644
--- a/drivers/gpu/drm/xen/xen_drm_front.c
+++ b/drivers/gpu/drm/xen/xen_drm_front.c
@@ -19,6 +19,7 @@
  #include 
  #include 
  +#include 
  #include 
    #include "xen_drm_front.h"
@@ -26,28 +27,20 @@
  #include "xen_drm_front_evtchnl.h"
  #include "xen_drm_front_gem.h"
  #include "xen_drm_front_kms.h"
-#include "xen_drm_front_shbuf.h"
    struct xen_drm_front_dbuf {
  struct list_head list;
  u64 dbuf_cookie;
  u64 fb_cookie;
-    struct xen_drm_front_shbuf *shbuf;
+
+    struct xen_front_pgdir_shbuf shbuf;
  };
  -static int dbuf_add_to_list(struct xen_drm_front_info *front_info,
-    struct xen_drm_front_shbuf *shbuf, u64 dbuf_cookie)
+static void dbuf_add_to_list(struct xen_drm_front_info *front_info,
+ struct xen_drm_front_dbuf *dbuf, u64 dbuf_cookie)
  {
-    struct xen_drm_front_dbuf *dbuf;
-
-    dbuf = kzalloc(sizeof(*dbuf), GFP_KERNEL);
-    if (!dbuf)
-    return -ENOMEM;
-
  dbuf->dbuf_cookie = dbuf_cookie;
-    dbuf->shbuf = shbuf;
  list_add(>list, _info->dbuf_list);
-    return 0;
  }
    static struct xen_drm_front_dbuf *dbuf_get(struct list_head 
*dbuf_list,
@@ -62,15 +55,6 @@ static struct xen_drm_front_dbuf *dbuf_get(struct 
list_head *dbuf_list,

  return NULL;
  }
  -static void dbuf_flush_fb(struct list_head *dbuf_list, u64 fb_cookie)
-{
-    struct xen_drm_front_dbuf *buf, *q;
-
-    list_for_each_entry_safe(buf, q, dbuf_list, list)
-    if (buf->fb_cookie == fb_cookie)
-    xen_drm_front_shbuf_flush(buf->shbuf);
-}
-
  static void dbuf_free(struct list_head *dbuf_list, u64 dbuf_cookie)
  {
  struct xen_drm_front_dbuf *buf, *q;
@@ -78,8 +62,8 @@ static void dbuf_free(struct list_head *dbuf_list, 
u64 dbuf_cookie)

  list_for_each_entry_safe(buf, q, dbuf_list, list)
  if (buf->dbuf_cookie == dbuf_cookie) {
  list_del(>list);
-    xen_drm_front_shbuf_unmap(buf->shbuf);
-    xen_drm_front_shbuf_free(buf->shbuf);
+    xen_front_pgdir_shbuf_unmap(>shbuf);
+    xen_front_pgdir_shbuf_free(>shbuf);
  kfree(buf);
  break;
  }
@@ -91,8 +75,8 @@ static void dbuf_free_all(struct list_head *dbuf_list)
    list_for_each_entry_safe(buf, q, dbuf_list, list) {
  list_del(>list);
-    xen_drm_front_shbuf_unmap(buf->shbuf);
-    xen_drm_front_shbuf_free(buf->shbuf);
+    xen_front_pgdir_shbuf_unmap(>shbuf);
+    xen_front_pgdir_shbuf_free(>shbuf);
  kfree(buf);
  }
  }
@@ -171,9 +155,9 @@ int xen_drm_front_dbuf_create(struct 
xen_drm_front_info *front_info,

    u32 bpp, u64 size, struct page **pages)
  {
  struct xen_drm_front_evtchnl *evtchnl;
-    struct xen_drm_front_shbuf *shbuf;
+   

[Xen-devel] [PATCH v3 0/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Hello!

At the moment Xen [1] already supports some virtual multimedia
features [2] such as virtual display, sound. It supports keyboards,
pointers and multi-touch devices all allowing Xen to be used in
automotive appliances, In-Vehicle Infotainment (IVI) systems
and many more.

Frontend implementation is available at [3] and the corresponding
backend at [4]. These are work in progress, but frontend already
passes v4l2-compliance test for V4L2 drivers. libxl changes are
available at [5].

This work adds a new Xen para-virtualized protocol for a virtual
camera device which extends multimedia capabilities of Xen even
farther: video conferencing, IVI, high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
 - pixel formats
 - resolutions
 - frame rates
2. Support basic camera controls:
 - contrast
 - brightness
 - hue
 - saturation
3. Support streaming control

I would like to thank Hans Verkuil  for valuable
comments and help.

Thank you,
Oleksandr Andrushchenko

Changes since v2:
=

1. Add "max-buffers" frontend configuration entry, e.g.
   the maximum number of camera buffers a frontend may use.
2. Add big-endian pixel-format support:
 - "formats" configuration string length changed from 4 to 7
   octets, so we can also manage BE pixel-formats
 - add corresponding comments to FOURCC mappings description
3. New commands added to the protocol and documented:
 - XENCAMERA_OP_CONFIG_VALIDATE
 - XENCAMERA_OP_FRAME_RATE_SET
 - XENCAMERA_OP_BUF_GET_LAYOUT
4.-Add defaults for colorspace, xfer, ycbcr_enc and quantization
5. Remove XENCAMERA_EVT_CONFIG_CHANGE event
6. Move plane offsets to XENCAMERA_OP_BUF_REQUEST as offsets
   required for the frontend might not be known at the configuration time
7. Clean up and address comments to v2 of the protocol

Changes since v1:
=

1. Added XenStore entries:
 - frame-rates
2. Do not require the FOURCC code in XenStore to be upper case only
3. Added/changed command set:
 - configuration get/set
 - buffer queue/dequeue
 - control get
4. Added control flags, e.g. read-only etc.
5. Added colorspace configuration support, relevant constants
6. Added events:
 - configuration change
 - control change
7. Changed control values to 64-bit
8. Added sequence number to frame avail event
9. Coding style cleanup

[1] https://www.xenproject.org/
[2] https://xenbits.xen.org/gitweb/?p=xen.git;a=tree;f=xen/include/public/io
[3] https://github.com/andr2000/linux/tree/camera_front_v1/drivers/media/xen
[4] https://github.com/andr2000/camera_be
[5] https://github.com/andr2000/xen/tree/vcamera

Oleksandr Andrushchenko (1):
  cameraif: add ABI for para-virtual camera

 xen/include/public/io/cameraif.h | 1374 ++
 1 file changed, 1374 insertions(+)
 create mode 100644 xen/include/public/io/cameraif.h

-- 
2.19.2


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
 - pixel formats
 - resolutions
 - frame rates
2. Support basic camera controls:
 - contrast
 - brightness
 - hue
 - saturation
3. Support streaming control

Signed-off-by: Oleksandr Andrushchenko 
---
 xen/include/public/io/cameraif.h | 1374 ++
 1 file changed, 1374 insertions(+)
 create mode 100644 xen/include/public/io/cameraif.h

diff --git a/xen/include/public/io/cameraif.h b/xen/include/public/io/cameraif.h
new file mode 100644
index ..9aae0f47743b
--- /dev/null
+++ b/xen/include/public/io/cameraif.h
@@ -0,0 +1,1374 @@
+/**
+ * cameraif.h
+ *
+ * Unified camera device I/O interface for Xen guest OSes.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright (C) 2018 EPAM Systems Inc.
+ *
+ * Author: Oleksandr Andrushchenko 
+ */
+
+#ifndef __XEN_PUBLIC_IO_CAMERAIF_H__
+#define __XEN_PUBLIC_IO_CAMERAIF_H__
+
+#include "ring.h"
+#include "../grant_table.h"
+
+/*
+ **
+ *   Protocol version
+ **
+ */
+#define XENCAMERA_PROTOCOL_VERSION "1"
+
+/*
+ **
+ *  Feature and Parameter Negotiation
+ **
+ *
+ * Front->back notifications: when enqueuing a new request, sending a
+ * notification can be made conditional on xencamera_req (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Backends must set
+ * xencamera_req appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
+ *
+ * Back->front notifications: when enqueuing a new response, sending a
+ * notification can be made conditional on xencamera_resp (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Frontends must set
+ * xencamera_resp appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
+ *
+ * The two halves of a para-virtual camera driver utilize nodes within
+ * XenStore to communicate capabilities and to negotiate operating parameters.
+ * This section enumerates these nodes which reside in the respective front and
+ * backend portions of XenStore, following the XenBus convention.
+ *
+ * All data in XenStore is stored as strings. Nodes specifying numeric
+ * values are encoded in decimal. Integer value ranges listed below are
+ * expressed as fixed sized integer types capable of storing the conversion
+ * of a properly formatted node string, without loss of information.
+ *
+ **
+ *Example configuration
+ **
+ *
+ * This is an example of backend and frontend configuration:
+ *
+ *- Backend ---
+ *
+ * /local/domain/0/backend/vcamera/1/0/frontend-id = "1"
+ * /local/domain/0/backend/vcamera/1/0/frontend = 
"/local/domain/1/device/vcamera/0"
+ * /local/domain/0/backend/vcamera/1/0/state = "4"
+ * /local/domain/0/backend/vcamera/1/0/versions = "1,2"
+ *
+ *- Frontend --
+ *
+ 

Re: [Xen-devel] [PATCH 6/7] drm/xen: Don't set the dpms hook

[Oleksandr Andrushchenko]

On 12/10/18 12:03 PM, Daniel Vetter wrote:

Doesn't do anything for atomic.

Signed-off-by: Daniel Vetter 
Cc: Oleksandr Andrushchenko 
Cc: xen-de...@lists.xen.org
---
  drivers/gpu/drm/xen/xen_drm_front_conn.c | 1 -
  1 file changed, 1 deletion(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front_conn.c 
b/drivers/gpu/drm/xen/xen_drm_front_conn.c
index c91ae532fa55..54af2669b1b3 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_conn.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_conn.c
@@ -89,7 +89,6 @@ static const struct drm_connector_helper_funcs 
connector_helper_funcs = {
  };
  
  static const struct drm_connector_funcs connector_funcs = {

-   .dpms = drm_helper_connector_dpms,
.fill_modes = drm_helper_probe_single_connector_modes,
.destroy = drm_connector_cleanup,
.reset = drm_atomic_helper_connector_reset,


Reviewed-by: Oleksandr Andrushchenko 

Thank you,

Oleksandr


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 2/3] drm/xen-front: Use Xen common shared buffer implementation

[Oleksandr Andrushchenko]

Hello, Daniel!

Could you please ack/nack the patch, so either we can merge the

series or I can address your comments if any

Thank you,

Oleksandr

On 11/30/18 9:42 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

Use page directory based shared buffer implementation
now available as common code for Xen frontend drivers.

Remove flushing of shared buffer on page flip as this
workaround needs a proper fix.

Signed-off-by: Oleksandr Andrushchenko 
---
  drivers/gpu/drm/xen/Kconfig   |   1 +
  drivers/gpu/drm/xen/Makefile  |   1 -
  drivers/gpu/drm/xen/xen_drm_front.c   |  65 ++--
  drivers/gpu/drm/xen/xen_drm_front_gem.c   |   1 -
  drivers/gpu/drm/xen/xen_drm_front_shbuf.c | 414 --
  drivers/gpu/drm/xen/xen_drm_front_shbuf.h |  64 
  6 files changed, 26 insertions(+), 520 deletions(-)
  delete mode 100644 drivers/gpu/drm/xen/xen_drm_front_shbuf.c
  delete mode 100644 drivers/gpu/drm/xen/xen_drm_front_shbuf.h

diff --git a/drivers/gpu/drm/xen/Kconfig b/drivers/gpu/drm/xen/Kconfig
index 4cca160782ab..f969d486855d 100644
--- a/drivers/gpu/drm/xen/Kconfig
+++ b/drivers/gpu/drm/xen/Kconfig
@@ -12,6 +12,7 @@ config DRM_XEN_FRONTEND
select DRM_KMS_HELPER
select VIDEOMODE_HELPERS
select XEN_XENBUS_FRONTEND
+   select XEN_FRONT_PGDIR_SHBUF
help
  Choose this option if you want to enable a para-virtualized
  frontend DRM/KMS driver for Xen guest OSes.
diff --git a/drivers/gpu/drm/xen/Makefile b/drivers/gpu/drm/xen/Makefile
index 712afff5ffc3..825905f67faa 100644
--- a/drivers/gpu/drm/xen/Makefile
+++ b/drivers/gpu/drm/xen/Makefile
@@ -4,7 +4,6 @@ drm_xen_front-objs := xen_drm_front.o \
  xen_drm_front_kms.o \
  xen_drm_front_conn.o \
  xen_drm_front_evtchnl.o \
- xen_drm_front_shbuf.o \
  xen_drm_front_cfg.o \
  xen_drm_front_gem.o
  
diff --git a/drivers/gpu/drm/xen/xen_drm_front.c b/drivers/gpu/drm/xen/xen_drm_front.c

index 6b6d5ab82ec3..4d3d36fc3a5d 100644
--- a/drivers/gpu/drm/xen/xen_drm_front.c
+++ b/drivers/gpu/drm/xen/xen_drm_front.c
@@ -19,6 +19,7 @@
  #include 
  #include 
  
+#include 

  #include 
  
  #include "xen_drm_front.h"

@@ -26,28 +27,20 @@
  #include "xen_drm_front_evtchnl.h"
  #include "xen_drm_front_gem.h"
  #include "xen_drm_front_kms.h"
-#include "xen_drm_front_shbuf.h"
  
  struct xen_drm_front_dbuf {

struct list_head list;
u64 dbuf_cookie;
u64 fb_cookie;
-   struct xen_drm_front_shbuf *shbuf;
+
+   struct xen_front_pgdir_shbuf shbuf;
  };
  
-static int dbuf_add_to_list(struct xen_drm_front_info *front_info,

-   struct xen_drm_front_shbuf *shbuf, u64 dbuf_cookie)
+static void dbuf_add_to_list(struct xen_drm_front_info *front_info,
+struct xen_drm_front_dbuf *dbuf, u64 dbuf_cookie)
  {
-   struct xen_drm_front_dbuf *dbuf;
-
-   dbuf = kzalloc(sizeof(*dbuf), GFP_KERNEL);
-   if (!dbuf)
-   return -ENOMEM;
-
dbuf->dbuf_cookie = dbuf_cookie;
-   dbuf->shbuf = shbuf;
list_add(>list, _info->dbuf_list);
-   return 0;
  }
  
  static struct xen_drm_front_dbuf *dbuf_get(struct list_head *dbuf_list,

@@ -62,15 +55,6 @@ static struct xen_drm_front_dbuf *dbuf_get(struct list_head 
*dbuf_list,
return NULL;
  }
  
-static void dbuf_flush_fb(struct list_head *dbuf_list, u64 fb_cookie)

-{
-   struct xen_drm_front_dbuf *buf, *q;
-
-   list_for_each_entry_safe(buf, q, dbuf_list, list)
-   if (buf->fb_cookie == fb_cookie)
-   xen_drm_front_shbuf_flush(buf->shbuf);
-}
-
  static void dbuf_free(struct list_head *dbuf_list, u64 dbuf_cookie)
  {
struct xen_drm_front_dbuf *buf, *q;
@@ -78,8 +62,8 @@ static void dbuf_free(struct list_head *dbuf_list, u64 
dbuf_cookie)
list_for_each_entry_safe(buf, q, dbuf_list, list)
if (buf->dbuf_cookie == dbuf_cookie) {
list_del(>list);
-   xen_drm_front_shbuf_unmap(buf->shbuf);
-   xen_drm_front_shbuf_free(buf->shbuf);
+   xen_front_pgdir_shbuf_unmap(>shbuf);
+   xen_front_pgdir_shbuf_free(>shbuf);
kfree(buf);
break;
}
@@ -91,8 +75,8 @@ static void dbuf_free_all(struct list_head *dbuf_list)
  
  	list_for_each_entry_safe(buf, q, dbuf_list, list) {

list_del(>list);
-   xen_drm_front_shbuf_unmap(buf->shbuf);
-   xen_drm_front_shbuf_free(buf->shbuf);
+   xen_front_pgdir_shbuf_unmap(>shbuf);
+   xen_front_pgdir_shbuf_free(>shbuf);
kfree(buf);
}
  }
@@ -171,9 +155,9 @@ int xe

Re: [Xen-devel] drm_gem_get_pages and proper flushing/coherency

[Oleksandr Andrushchenko]

On 11/26/18 2:15 PM, Oleksandr Andrushchenko wrote:

Hello, all!

My driver (Xen para-virtualized frontend) in some scenarios uses
drm_gem_get_pages for allocating backing storage for dumb buffers.
There are use-cases which showed some artifacts on the screen
(modetest, other) which were worked around by flushing pages of the
buffer on page flip with drm_clflush_pages. But, the problem here
is that drm_clflush_pages is not available on ARM platforms (it is a NOP)
and doing flushes on every page flip seems to be non-optimal.

Other drivers that use drm_gem_get_pages seem to use DMA map/unmap
on the shmem backed buffer (this is from where drm_gem_get_pages
allocates the pages) and this is an obvious approach as the buffer needs
to be shared with real HW for DMA - please correct me if my understanding
here is wrong.


I have created a patch which implements DMA mapping [1] and this

does solve artifacts problem for me.

Is this the right way to go?



This is the part I missed in my implementation as I don't really have a
HW device which needs DMA, but a backend running in a different Xen 
domain.


Thus, as the buffer is backed with cachable pages the backend may see

artifacts on its side.


I am looking for some advices on what would be the best option to
make sure dumb buffers are not flushed every page flip and still
the memory remains coherent to the backend. I have implemented a
DMA map/unmap of the shmem pages on GEM object creation/destruction
and this does solve the problem, but as the backend is not really
a DMA device this is a bit misleading.

Is there any other (more?) suitable/preferable way(s) to achieve the 
same?


Thank you,
Oleksandr


Thank you,

Oleksandr

[1] https://patchwork.freedesktop.org/series/53069/


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [alsa-devel] [PATCH v2 3/3] ALSA: xen-front: Use Xen common shared buffer implementation

[Oleksandr Andrushchenko]

On 12/3/18 11:36 AM, Takashi Iwai wrote:

On Fri, 30 Nov 2018 08:42:05 +0100,
Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

Use page directory based shared buffer implementation
now available as common code for Xen frontend drivers.

Signed-off-by: Oleksandr Andrushchenko 

For the sound bits,
   Reviewed-by: Takashi Iwai 

Thank you


I suppose the whole patchset will be taken through xen or other tree?

Xen tree


thanks,

Takashi


Thank you,

Oleksandr


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 1/3] xen: Introduce shared buffer helpers for page directory...

[Oleksandr Andrushchenko]

Changes in the series since v1:

- no changes to patch 1 (Xen shared code)

- ALSA: fix comment from Takashi (NULL stream->buffer)

- DRM: removed display buffer flush workaround

(proper fix is on review [1])

Thank you,

Oleksandr

On 11/30/18 9:42 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

based frontends. Currently the frontends which implement
similar code for sharing big buffers between frontend and
backend are para-virtualized DRM and sound drivers.
Both define the same way to share grant references of a
data buffer with the corresponding backend with little
differences.

Move shared code into a helper module, so there is a single
implementation of the same functionality for all.

This patch introduces code which is used by sound and display
frontend drivers without functional changes with the intention
to remove shared code from the corresponding drivers.

Signed-off-by: Oleksandr Andrushchenko 
---
  drivers/xen/Kconfig |   3 +
  drivers/xen/Makefile|   1 +
  drivers/xen/xen-front-pgdir-shbuf.c | 553 
  include/xen/xen-front-pgdir-shbuf.h |  89 +
  4 files changed, 646 insertions(+)
  create mode 100644 drivers/xen/xen-front-pgdir-shbuf.c
  create mode 100644 include/xen/xen-front-pgdir-shbuf.h

diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
index 815b9e9bb975..838b66a9a0e7 100644
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -340,4 +340,7 @@ config XEN_SYMS
  config XEN_HAVE_VPMU
 bool
  
+config XEN_FRONT_PGDIR_SHBUF

+   tristate
+
  endmenu
diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile
index 3e542f60f29f..c48927a58e10 100644
--- a/drivers/xen/Makefile
+++ b/drivers/xen/Makefile
@@ -44,3 +44,4 @@ xen-gntdev-y  := gntdev.o
  xen-gntdev-$(CONFIG_XEN_GNTDEV_DMABUF)+= gntdev-dmabuf.o
  xen-gntalloc-y:= gntalloc.o
  xen-privcmd-y := privcmd.o privcmd-buf.o
+obj-$(CONFIG_XEN_FRONT_PGDIR_SHBUF)+= xen-front-pgdir-shbuf.o
diff --git a/drivers/xen/xen-front-pgdir-shbuf.c 
b/drivers/xen/xen-front-pgdir-shbuf.c
new file mode 100644
index ..48a658dc7ccf
--- /dev/null
+++ b/drivers/xen/xen-front-pgdir-shbuf.c
@@ -0,0 +1,553 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+
+/*
+ * Xen frontend/backend page directory based shared buffer
+ * helper module.
+ *
+ * Copyright (C) 2018 EPAM Systems Inc.
+ *
+ * Author: Oleksandr Andrushchenko 
+ */
+
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+
+#ifndef GRANT_INVALID_REF
+/*
+ * FIXME: usage of grant reference 0 as invalid grant reference:
+ * grant reference 0 is valid, but never exposed to a PV driver,
+ * because of the fact it is already in use/reserved by the PV console.
+ */
+#define GRANT_INVALID_REF  0
+#endif
+
+/**
+ * This structure represents the structure of a shared page
+ * that contains grant references to the pages of the shared
+ * buffer. This structure is common to many Xen para-virtualized
+ * protocols at include/xen/interface/io/
+ */
+struct xen_page_directory {
+   grant_ref_t gref_dir_next_page;
+   grant_ref_t gref[1]; /* Variable length */
+};
+
+/**
+ * Shared buffer ops which are differently implemented
+ * depending on the allocation mode, e.g. if the buffer
+ * is allocated by the corresponding backend or frontend.
+ * Some of the operations.
+ */
+struct xen_front_pgdir_shbuf_ops {
+   /*
+* Calculate number of grefs required to handle this buffer,
+* e.g. if grefs are required for page directory only or the buffer
+* pages as well.
+*/
+   void (*calc_num_grefs)(struct xen_front_pgdir_shbuf *buf);
+
+   /* Fill page directory according to para-virtual display protocol. */
+   void (*fill_page_dir)(struct xen_front_pgdir_shbuf *buf);
+
+   /* Claim grant references for the pages of the buffer. */
+   int (*grant_refs_for_buffer)(struct xen_front_pgdir_shbuf *buf,
+grant_ref_t *priv_gref_head, int gref_idx);
+
+   /* Map grant references of the buffer. */
+   int (*map)(struct xen_front_pgdir_shbuf *buf);
+
+   /* Unmap grant references of the buffer. */
+   int (*unmap)(struct xen_front_pgdir_shbuf *buf);
+};
+
+/**
+ * Get granted reference to the very first page of the
+ * page directory. Usually this is passed to the backend,
+ * so it can find/fill the grant references to the buffer's
+ * pages.
+ *
+ * \param buf shared buffer which page directory is of interest.
+ * \return granted reference to the very first page of the
+ * page directory.
+ */
+grant_ref_t
+xen_front_pgdir_shbuf_get_dir_start(struct xen_front_pgdir_shbuf *buf)
+{
+   if (!buf->grefs)
+   return GRANT_INVALID_REF;
+
+   return buf->grefs[0];
+}
+EXPORT_SYMBOL_GPL(xen_front_pgdir_shbuf_get_dir_start);
+
+/**
+ * Ma

[Xen-devel] [PATCH v2 1/3] xen: Introduce shared buffer helpers for page directory...

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

based frontends. Currently the frontends which implement
similar code for sharing big buffers between frontend and
backend are para-virtualized DRM and sound drivers.
Both define the same way to share grant references of a
data buffer with the corresponding backend with little
differences.

Move shared code into a helper module, so there is a single
implementation of the same functionality for all.

This patch introduces code which is used by sound and display
frontend drivers without functional changes with the intention
to remove shared code from the corresponding drivers.

Signed-off-by: Oleksandr Andrushchenko 
---
 drivers/xen/Kconfig |   3 +
 drivers/xen/Makefile|   1 +
 drivers/xen/xen-front-pgdir-shbuf.c | 553 
 include/xen/xen-front-pgdir-shbuf.h |  89 +
 4 files changed, 646 insertions(+)
 create mode 100644 drivers/xen/xen-front-pgdir-shbuf.c
 create mode 100644 include/xen/xen-front-pgdir-shbuf.h

diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
index 815b9e9bb975..838b66a9a0e7 100644
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -340,4 +340,7 @@ config XEN_SYMS
 config XEN_HAVE_VPMU
bool
 
+config XEN_FRONT_PGDIR_SHBUF
+   tristate
+
 endmenu
diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile
index 3e542f60f29f..c48927a58e10 100644
--- a/drivers/xen/Makefile
+++ b/drivers/xen/Makefile
@@ -44,3 +44,4 @@ xen-gntdev-y  := gntdev.o
 xen-gntdev-$(CONFIG_XEN_GNTDEV_DMABUF) += gntdev-dmabuf.o
 xen-gntalloc-y := gntalloc.o
 xen-privcmd-y  := privcmd.o privcmd-buf.o
+obj-$(CONFIG_XEN_FRONT_PGDIR_SHBUF)+= xen-front-pgdir-shbuf.o
diff --git a/drivers/xen/xen-front-pgdir-shbuf.c 
b/drivers/xen/xen-front-pgdir-shbuf.c
new file mode 100644
index ..48a658dc7ccf
--- /dev/null
+++ b/drivers/xen/xen-front-pgdir-shbuf.c
@@ -0,0 +1,553 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+
+/*
+ * Xen frontend/backend page directory based shared buffer
+ * helper module.
+ *
+ * Copyright (C) 2018 EPAM Systems Inc.
+ *
+ * Author: Oleksandr Andrushchenko 
+ */
+
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+
+#ifndef GRANT_INVALID_REF
+/*
+ * FIXME: usage of grant reference 0 as invalid grant reference:
+ * grant reference 0 is valid, but never exposed to a PV driver,
+ * because of the fact it is already in use/reserved by the PV console.
+ */
+#define GRANT_INVALID_REF  0
+#endif
+
+/**
+ * This structure represents the structure of a shared page
+ * that contains grant references to the pages of the shared
+ * buffer. This structure is common to many Xen para-virtualized
+ * protocols at include/xen/interface/io/
+ */
+struct xen_page_directory {
+   grant_ref_t gref_dir_next_page;
+   grant_ref_t gref[1]; /* Variable length */
+};
+
+/**
+ * Shared buffer ops which are differently implemented
+ * depending on the allocation mode, e.g. if the buffer
+ * is allocated by the corresponding backend or frontend.
+ * Some of the operations.
+ */
+struct xen_front_pgdir_shbuf_ops {
+   /*
+* Calculate number of grefs required to handle this buffer,
+* e.g. if grefs are required for page directory only or the buffer
+* pages as well.
+*/
+   void (*calc_num_grefs)(struct xen_front_pgdir_shbuf *buf);
+
+   /* Fill page directory according to para-virtual display protocol. */
+   void (*fill_page_dir)(struct xen_front_pgdir_shbuf *buf);
+
+   /* Claim grant references for the pages of the buffer. */
+   int (*grant_refs_for_buffer)(struct xen_front_pgdir_shbuf *buf,
+grant_ref_t *priv_gref_head, int gref_idx);
+
+   /* Map grant references of the buffer. */
+   int (*map)(struct xen_front_pgdir_shbuf *buf);
+
+   /* Unmap grant references of the buffer. */
+   int (*unmap)(struct xen_front_pgdir_shbuf *buf);
+};
+
+/**
+ * Get granted reference to the very first page of the
+ * page directory. Usually this is passed to the backend,
+ * so it can find/fill the grant references to the buffer's
+ * pages.
+ *
+ * \param buf shared buffer which page directory is of interest.
+ * \return granted reference to the very first page of the
+ * page directory.
+ */
+grant_ref_t
+xen_front_pgdir_shbuf_get_dir_start(struct xen_front_pgdir_shbuf *buf)
+{
+   if (!buf->grefs)
+   return GRANT_INVALID_REF;
+
+   return buf->grefs[0];
+}
+EXPORT_SYMBOL_GPL(xen_front_pgdir_shbuf_get_dir_start);
+
+/**
+ * Map granted references of the shared buffer.
+ *
+ * Depending on the shared buffer mode of allocation
+ * (be_alloc flag) this can either do nothing (for buffers
+ * shared by the frontend itself) or map the provided granted
+ * references onto the backing storage (buf->pages).
+ *
+ * \param buf shared buffer whi

[Xen-devel] [PATCH v2 3/3] ALSA: xen-front: Use Xen common shared buffer implementation

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Use page directory based shared buffer implementation
now available as common code for Xen frontend drivers.

Signed-off-by: Oleksandr Andrushchenko 
---
 sound/xen/Kconfig   |   1 +
 sound/xen/Makefile  |   1 -
 sound/xen/xen_snd_front.c   |   7 +-
 sound/xen/xen_snd_front.h   |   4 +-
 sound/xen/xen_snd_front_alsa.c  | 102 +
 sound/xen/xen_snd_front_shbuf.c | 194 
 sound/xen/xen_snd_front_shbuf.h |  36 --
 7 files changed, 84 insertions(+), 261 deletions(-)
 delete mode 100644 sound/xen/xen_snd_front_shbuf.c
 delete mode 100644 sound/xen/xen_snd_front_shbuf.h

diff --git a/sound/xen/Kconfig b/sound/xen/Kconfig
index 4f1fceea82d2..e4d7beb4df1c 100644
--- a/sound/xen/Kconfig
+++ b/sound/xen/Kconfig
@@ -5,6 +5,7 @@ config SND_XEN_FRONTEND
depends on XEN
select SND_PCM
select XEN_XENBUS_FRONTEND
+   select XEN_FRONT_PGDIR_SHBUF
help
  Choose this option if you want to enable a para-virtualized
  frontend sound driver for Xen guest OSes.
diff --git a/sound/xen/Makefile b/sound/xen/Makefile
index 1e6470ecc2f2..24031775b715 100644
--- a/sound/xen/Makefile
+++ b/sound/xen/Makefile
@@ -3,7 +3,6 @@
 snd_xen_front-objs := xen_snd_front.o \
  xen_snd_front_cfg.o \
  xen_snd_front_evtchnl.o \
- xen_snd_front_shbuf.o \
  xen_snd_front_alsa.o
 
 obj-$(CONFIG_SND_XEN_FRONTEND) += snd_xen_front.o
diff --git a/sound/xen/xen_snd_front.c b/sound/xen/xen_snd_front.c
index b089b13b5160..a9e5c2cd7698 100644
--- a/sound/xen/xen_snd_front.c
+++ b/sound/xen/xen_snd_front.c
@@ -16,12 +16,12 @@
 #include 
 #include 
 
+#include 
 #include 
 
 #include "xen_snd_front.h"
 #include "xen_snd_front_alsa.h"
 #include "xen_snd_front_evtchnl.h"
-#include "xen_snd_front_shbuf.h"
 
 static struct xensnd_req *
 be_stream_prepare_req(struct xen_snd_front_evtchnl *evtchnl, u8 operation)
@@ -82,7 +82,7 @@ int xen_snd_front_stream_query_hw_param(struct 
xen_snd_front_evtchnl *evtchnl,
 }
 
 int xen_snd_front_stream_prepare(struct xen_snd_front_evtchnl *evtchnl,
-struct xen_snd_front_shbuf *sh_buf,
+struct xen_front_pgdir_shbuf *shbuf,
 u8 format, unsigned int channels,
 unsigned int rate, u32 buffer_sz,
 u32 period_sz)
@@ -99,7 +99,8 @@ int xen_snd_front_stream_prepare(struct xen_snd_front_evtchnl 
*evtchnl,
req->op.open.pcm_rate = rate;
req->op.open.buffer_sz = buffer_sz;
req->op.open.period_sz = period_sz;
-   req->op.open.gref_directory = xen_snd_front_shbuf_get_dir_start(sh_buf);
+   req->op.open.gref_directory =
+   xen_front_pgdir_shbuf_get_dir_start(shbuf);
mutex_unlock(>ring_io_lock);
 
ret = be_stream_do_io(evtchnl);
diff --git a/sound/xen/xen_snd_front.h b/sound/xen/xen_snd_front.h
index a2ea2463bcc5..05611f113b94 100644
--- a/sound/xen/xen_snd_front.h
+++ b/sound/xen/xen_snd_front.h
@@ -16,7 +16,7 @@
 struct xen_snd_front_card_info;
 struct xen_snd_front_evtchnl;
 struct xen_snd_front_evtchnl_pair;
-struct xen_snd_front_shbuf;
+struct xen_front_pgdir_shbuf;
 struct xensnd_query_hw_param;
 
 struct xen_snd_front_info {
@@ -35,7 +35,7 @@ int xen_snd_front_stream_query_hw_param(struct 
xen_snd_front_evtchnl *evtchnl,
struct xensnd_query_hw_param 
*hw_param_resp);
 
 int xen_snd_front_stream_prepare(struct xen_snd_front_evtchnl *evtchnl,
-struct xen_snd_front_shbuf *sh_buf,
+struct xen_front_pgdir_shbuf *shbuf,
 u8 format, unsigned int channels,
 unsigned int rate, u32 buffer_sz,
 u32 period_sz);
diff --git a/sound/xen/xen_snd_front_alsa.c b/sound/xen/xen_snd_front_alsa.c
index 129180e17db1..a17ae45ec634 100644
--- a/sound/xen/xen_snd_front_alsa.c
+++ b/sound/xen/xen_snd_front_alsa.c
@@ -15,17 +15,24 @@
 #include 
 
 #include 
+#include 
 
 #include "xen_snd_front.h"
 #include "xen_snd_front_alsa.h"
 #include "xen_snd_front_cfg.h"
 #include "xen_snd_front_evtchnl.h"
-#include "xen_snd_front_shbuf.h"
 
 struct xen_snd_front_pcm_stream_info {
struct xen_snd_front_info *front_info;
struct xen_snd_front_evtchnl_pair *evt_pair;
-   struct xen_snd_front_shbuf sh_buf;
+
+   /* This is the shared buffer with its backing storage. */
+   struct xen_front_pgdir_shbuf shbuf;
+   u8 *buffer;
+   size_t buffer_sz;
+   int num_pages;
+   struct page **pages;
+
int index;
 
bool is_open;
@@ -214,12 +221,20 @@ static void stream_clear(

[Xen-devel] [PATCH v2 2/3] drm/xen-front: Use Xen common shared buffer implementation

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Use page directory based shared buffer implementation
now available as common code for Xen frontend drivers.

Remove flushing of shared buffer on page flip as this
workaround needs a proper fix.

Signed-off-by: Oleksandr Andrushchenko 
---
 drivers/gpu/drm/xen/Kconfig   |   1 +
 drivers/gpu/drm/xen/Makefile  |   1 -
 drivers/gpu/drm/xen/xen_drm_front.c   |  65 ++--
 drivers/gpu/drm/xen/xen_drm_front_gem.c   |   1 -
 drivers/gpu/drm/xen/xen_drm_front_shbuf.c | 414 --
 drivers/gpu/drm/xen/xen_drm_front_shbuf.h |  64 
 6 files changed, 26 insertions(+), 520 deletions(-)
 delete mode 100644 drivers/gpu/drm/xen/xen_drm_front_shbuf.c
 delete mode 100644 drivers/gpu/drm/xen/xen_drm_front_shbuf.h

diff --git a/drivers/gpu/drm/xen/Kconfig b/drivers/gpu/drm/xen/Kconfig
index 4cca160782ab..f969d486855d 100644
--- a/drivers/gpu/drm/xen/Kconfig
+++ b/drivers/gpu/drm/xen/Kconfig
@@ -12,6 +12,7 @@ config DRM_XEN_FRONTEND
select DRM_KMS_HELPER
select VIDEOMODE_HELPERS
select XEN_XENBUS_FRONTEND
+   select XEN_FRONT_PGDIR_SHBUF
help
  Choose this option if you want to enable a para-virtualized
  frontend DRM/KMS driver for Xen guest OSes.
diff --git a/drivers/gpu/drm/xen/Makefile b/drivers/gpu/drm/xen/Makefile
index 712afff5ffc3..825905f67faa 100644
--- a/drivers/gpu/drm/xen/Makefile
+++ b/drivers/gpu/drm/xen/Makefile
@@ -4,7 +4,6 @@ drm_xen_front-objs := xen_drm_front.o \
  xen_drm_front_kms.o \
  xen_drm_front_conn.o \
  xen_drm_front_evtchnl.o \
- xen_drm_front_shbuf.o \
  xen_drm_front_cfg.o \
  xen_drm_front_gem.o
 
diff --git a/drivers/gpu/drm/xen/xen_drm_front.c 
b/drivers/gpu/drm/xen/xen_drm_front.c
index 6b6d5ab82ec3..4d3d36fc3a5d 100644
--- a/drivers/gpu/drm/xen/xen_drm_front.c
+++ b/drivers/gpu/drm/xen/xen_drm_front.c
@@ -19,6 +19,7 @@
 #include 
 #include 
 
+#include 
 #include 
 
 #include "xen_drm_front.h"
@@ -26,28 +27,20 @@
 #include "xen_drm_front_evtchnl.h"
 #include "xen_drm_front_gem.h"
 #include "xen_drm_front_kms.h"
-#include "xen_drm_front_shbuf.h"
 
 struct xen_drm_front_dbuf {
struct list_head list;
u64 dbuf_cookie;
u64 fb_cookie;
-   struct xen_drm_front_shbuf *shbuf;
+
+   struct xen_front_pgdir_shbuf shbuf;
 };
 
-static int dbuf_add_to_list(struct xen_drm_front_info *front_info,
-   struct xen_drm_front_shbuf *shbuf, u64 dbuf_cookie)
+static void dbuf_add_to_list(struct xen_drm_front_info *front_info,
+struct xen_drm_front_dbuf *dbuf, u64 dbuf_cookie)
 {
-   struct xen_drm_front_dbuf *dbuf;
-
-   dbuf = kzalloc(sizeof(*dbuf), GFP_KERNEL);
-   if (!dbuf)
-   return -ENOMEM;
-
dbuf->dbuf_cookie = dbuf_cookie;
-   dbuf->shbuf = shbuf;
list_add(>list, _info->dbuf_list);
-   return 0;
 }
 
 static struct xen_drm_front_dbuf *dbuf_get(struct list_head *dbuf_list,
@@ -62,15 +55,6 @@ static struct xen_drm_front_dbuf *dbuf_get(struct list_head 
*dbuf_list,
return NULL;
 }
 
-static void dbuf_flush_fb(struct list_head *dbuf_list, u64 fb_cookie)
-{
-   struct xen_drm_front_dbuf *buf, *q;
-
-   list_for_each_entry_safe(buf, q, dbuf_list, list)
-   if (buf->fb_cookie == fb_cookie)
-   xen_drm_front_shbuf_flush(buf->shbuf);
-}
-
 static void dbuf_free(struct list_head *dbuf_list, u64 dbuf_cookie)
 {
struct xen_drm_front_dbuf *buf, *q;
@@ -78,8 +62,8 @@ static void dbuf_free(struct list_head *dbuf_list, u64 
dbuf_cookie)
list_for_each_entry_safe(buf, q, dbuf_list, list)
if (buf->dbuf_cookie == dbuf_cookie) {
list_del(>list);
-   xen_drm_front_shbuf_unmap(buf->shbuf);
-   xen_drm_front_shbuf_free(buf->shbuf);
+   xen_front_pgdir_shbuf_unmap(>shbuf);
+   xen_front_pgdir_shbuf_free(>shbuf);
kfree(buf);
break;
}
@@ -91,8 +75,8 @@ static void dbuf_free_all(struct list_head *dbuf_list)
 
list_for_each_entry_safe(buf, q, dbuf_list, list) {
list_del(>list);
-   xen_drm_front_shbuf_unmap(buf->shbuf);
-   xen_drm_front_shbuf_free(buf->shbuf);
+   xen_front_pgdir_shbuf_unmap(>shbuf);
+   xen_front_pgdir_shbuf_free(>shbuf);
kfree(buf);
}
 }
@@ -171,9 +155,9 @@ int xen_drm_front_dbuf_create(struct xen_drm_front_info 
*front_info,
  u32 bpp, u64 size, struct page **pages)
 {
struct xen_drm_front_evtchnl *evtchnl;
-   struct xen_drm_front_shbuf *shbuf;
+   stru

Re: [Xen-devel] [PATCH 1/3] xen: Introduce shared buffer helpers for page directory...

[Oleksandr Andrushchenko]

On 11/30/18 8:50 AM, Juergen Gross wrote:

On 29/11/2018 12:22, Oleksandr Andrushchenko wrote:

ping

On 11/22/18 12:02 PM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

based frontends. Currently the frontends which implement
similar code for sharing big buffers between frontend and
backend are para-virtualized DRM and sound drivers.
Both define the same way to share grant references of a
data buffer with the corresponding backend with little
differences.

Move shared code into a helper module, so there is a single
implementation of the same functionality for all.

Signed-off-by: Oleksandr Andrushchenko 

In general I'm fine with this approach.

With the concerns raised for one of the other patches I wanted to wait
for V2 of the series.

Ah, I waited for any comments before rolling v2 out ;)

  Or won't the resulting change require a
modification of this patch?


This patch won't change, it is only DRM related

The concern for the DRM patch is already resolved

and the corresponding patch is on review now [1]



It would be nice if you could point out in the commit message whether
you are doing code movement (with some renames) only, or if there are
any functional changes involved (and which ones).

Sure, this is pure code movement, no functional changes

  This would make the
review much easier and less time consuming.


Juergen


Thank you,

Oleksandr

[1] https://lkml.org/lkml/2018/11/27/811


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/3] xen: Introduce shared buffer helpers for page directory...

[Oleksandr Andrushchenko]

ping

On 11/22/18 12:02 PM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

based frontends. Currently the frontends which implement
similar code for sharing big buffers between frontend and
backend are para-virtualized DRM and sound drivers.
Both define the same way to share grant references of a
data buffer with the corresponding backend with little
differences.

Move shared code into a helper module, so there is a single
implementation of the same functionality for all.

Signed-off-by: Oleksandr Andrushchenko 
---
  drivers/xen/Kconfig |   3 +
  drivers/xen/Makefile|   1 +
  drivers/xen/xen-front-pgdir-shbuf.c | 553 
  include/xen/xen-front-pgdir-shbuf.h |  89 +
  4 files changed, 646 insertions(+)
  create mode 100644 drivers/xen/xen-front-pgdir-shbuf.c
  create mode 100644 include/xen/xen-front-pgdir-shbuf.h

diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
index 815b9e9bb975..838b66a9a0e7 100644
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -340,4 +340,7 @@ config XEN_SYMS
  config XEN_HAVE_VPMU
 bool
  
+config XEN_FRONT_PGDIR_SHBUF

+   tristate
+
  endmenu
diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile
index 3e542f60f29f..c48927a58e10 100644
--- a/drivers/xen/Makefile
+++ b/drivers/xen/Makefile
@@ -44,3 +44,4 @@ xen-gntdev-y  := gntdev.o
  xen-gntdev-$(CONFIG_XEN_GNTDEV_DMABUF)+= gntdev-dmabuf.o
  xen-gntalloc-y:= gntalloc.o
  xen-privcmd-y := privcmd.o privcmd-buf.o
+obj-$(CONFIG_XEN_FRONT_PGDIR_SHBUF)+= xen-front-pgdir-shbuf.o
diff --git a/drivers/xen/xen-front-pgdir-shbuf.c 
b/drivers/xen/xen-front-pgdir-shbuf.c
new file mode 100644
index ..48a658dc7ccf
--- /dev/null
+++ b/drivers/xen/xen-front-pgdir-shbuf.c
@@ -0,0 +1,553 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+
+/*
+ * Xen frontend/backend page directory based shared buffer
+ * helper module.
+ *
+ * Copyright (C) 2018 EPAM Systems Inc.
+ *
+ * Author: Oleksandr Andrushchenko 
+ */
+
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+
+#ifndef GRANT_INVALID_REF
+/*
+ * FIXME: usage of grant reference 0 as invalid grant reference:
+ * grant reference 0 is valid, but never exposed to a PV driver,
+ * because of the fact it is already in use/reserved by the PV console.
+ */
+#define GRANT_INVALID_REF  0
+#endif
+
+/**
+ * This structure represents the structure of a shared page
+ * that contains grant references to the pages of the shared
+ * buffer. This structure is common to many Xen para-virtualized
+ * protocols at include/xen/interface/io/
+ */
+struct xen_page_directory {
+   grant_ref_t gref_dir_next_page;
+   grant_ref_t gref[1]; /* Variable length */
+};
+
+/**
+ * Shared buffer ops which are differently implemented
+ * depending on the allocation mode, e.g. if the buffer
+ * is allocated by the corresponding backend or frontend.
+ * Some of the operations.
+ */
+struct xen_front_pgdir_shbuf_ops {
+   /*
+* Calculate number of grefs required to handle this buffer,
+* e.g. if grefs are required for page directory only or the buffer
+* pages as well.
+*/
+   void (*calc_num_grefs)(struct xen_front_pgdir_shbuf *buf);
+
+   /* Fill page directory according to para-virtual display protocol. */
+   void (*fill_page_dir)(struct xen_front_pgdir_shbuf *buf);
+
+   /* Claim grant references for the pages of the buffer. */
+   int (*grant_refs_for_buffer)(struct xen_front_pgdir_shbuf *buf,
+grant_ref_t *priv_gref_head, int gref_idx);
+
+   /* Map grant references of the buffer. */
+   int (*map)(struct xen_front_pgdir_shbuf *buf);
+
+   /* Unmap grant references of the buffer. */
+   int (*unmap)(struct xen_front_pgdir_shbuf *buf);
+};
+
+/**
+ * Get granted reference to the very first page of the
+ * page directory. Usually this is passed to the backend,
+ * so it can find/fill the grant references to the buffer's
+ * pages.
+ *
+ * \param buf shared buffer which page directory is of interest.
+ * \return granted reference to the very first page of the
+ * page directory.
+ */
+grant_ref_t
+xen_front_pgdir_shbuf_get_dir_start(struct xen_front_pgdir_shbuf *buf)
+{
+   if (!buf->grefs)
+   return GRANT_INVALID_REF;
+
+   return buf->grefs[0];
+}
+EXPORT_SYMBOL_GPL(xen_front_pgdir_shbuf_get_dir_start);
+
+/**
+ * Map granted references of the shared buffer.
+ *
+ * Depending on the shared buffer mode of allocation
+ * (be_alloc flag) this can either do nothing (for buffers
+ * shared by the frontend itself) or map the provided granted
+ * references onto the backing storage (buf->pages).
+ *
+ * \param buf shared buffer which grants to be maped.
+ * \return zero on success or a negative number on failure.

[Xen-devel] [PATCH] drm/xen-front: Make shmem backed display buffer coherent

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

When GEM backing storage is allocated with drm_gem_get_pages
the backing pages may be cached, thus making it possible that
the backend sees only partial content of the buffer which may
lead to screen artifacts. Make sure that the frontend's
memory is coherent and the backend always sees correct display
buffer content.

Fixes: c575b7eeb89f ("drm/xen-front: Add support for Xen PV display frontend")

Signed-off-by: Oleksandr Andrushchenko 
---
 drivers/gpu/drm/xen/xen_drm_front_gem.c | 62 +++--
 1 file changed, 48 insertions(+), 14 deletions(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index 47ff019d3aef..c592735e49d2 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -33,8 +33,11 @@ struct xen_gem_object {
/* set for buffers allocated by the backend */
bool be_alloc;
 
-   /* this is for imported PRIME buffer */
-   struct sg_table *sgt_imported;
+   /*
+* this is for imported PRIME buffer or the one allocated via
+* drm_gem_get_pages.
+*/
+   struct sg_table *sgt;
 };
 
 static inline struct xen_gem_object *
@@ -77,10 +80,21 @@ static struct xen_gem_object *gem_create_obj(struct 
drm_device *dev,
return xen_obj;
 }
 
+struct sg_table *xen_drm_front_gem_get_sg_table(struct drm_gem_object *gem_obj)
+{
+   struct xen_gem_object *xen_obj = to_xen_gem_obj(gem_obj);
+
+   if (!xen_obj->pages)
+   return ERR_PTR(-ENOMEM);
+
+   return drm_prime_pages_to_sg(xen_obj->pages, xen_obj->num_pages);
+}
+
 static struct xen_gem_object *gem_create(struct drm_device *dev, size_t size)
 {
struct xen_drm_front_drm_info *drm_info = dev->dev_private;
struct xen_gem_object *xen_obj;
+   struct address_space *mapping;
int ret;
 
size = round_up(size, PAGE_SIZE);
@@ -113,10 +127,14 @@ static struct xen_gem_object *gem_create(struct 
drm_device *dev, size_t size)
xen_obj->be_alloc = true;
return xen_obj;
}
+
/*
 * need to allocate backing pages now, so we can share those
 * with the backend
 */
+   mapping = xen_obj->base.filp->f_mapping;
+   mapping_set_gfp_mask(mapping, GFP_USER | __GFP_DMA32);
+
xen_obj->num_pages = DIV_ROUND_UP(size, PAGE_SIZE);
xen_obj->pages = drm_gem_get_pages(_obj->base);
if (IS_ERR_OR_NULL(xen_obj->pages)) {
@@ -125,8 +143,27 @@ static struct xen_gem_object *gem_create(struct drm_device 
*dev, size_t size)
goto fail;
}
 
+   xen_obj->sgt = xen_drm_front_gem_get_sg_table(_obj->base);
+   if (IS_ERR_OR_NULL(xen_obj->sgt)){
+   ret = PTR_ERR(xen_obj->sgt);
+   xen_obj->sgt = NULL;
+   goto fail_put_pages;
+   }
+
+   if (!dma_map_sg(dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents,
+   DMA_BIDIRECTIONAL)) {
+   ret = -EFAULT;
+   goto fail_free_sgt;
+   }
+
return xen_obj;
 
+fail_free_sgt:
+   sg_free_table(xen_obj->sgt);
+   xen_obj->sgt = NULL;
+fail_put_pages:
+   drm_gem_put_pages(_obj->base, xen_obj->pages, true, false);
+   xen_obj->pages = NULL;
 fail:
DRM_ERROR("Failed to allocate buffer with size %zu\n", size);
return ERR_PTR(ret);
@@ -149,7 +186,7 @@ void xen_drm_front_gem_free_object_unlocked(struct 
drm_gem_object *gem_obj)
struct xen_gem_object *xen_obj = to_xen_gem_obj(gem_obj);
 
if (xen_obj->base.import_attach) {
-   drm_prime_gem_destroy(_obj->base, xen_obj->sgt_imported);
+   drm_prime_gem_destroy(_obj->base, xen_obj->sgt);
gem_free_pages_array(xen_obj);
} else {
if (xen_obj->pages) {
@@ -158,6 +195,13 @@ void xen_drm_front_gem_free_object_unlocked(struct 
drm_gem_object *gem_obj)
xen_obj->pages);
gem_free_pages_array(xen_obj);
} else {
+   if (xen_obj->sgt) {
+   dma_unmap_sg(xen_obj->base.dev->dev,
+xen_obj->sgt->sgl,
+xen_obj->sgt->nents,
+DMA_BIDIRECTIONAL);
+   sg_free_table(xen_obj->sgt);
+   }
drm_gem_put_pages(_obj->base,
  xen_obj->pages, true, false);
}
@@ -174,16 +218,6 @@ struct page **xen_drm_front_gem_get_pages

[Xen-devel] drm_gem_get_pages and proper flushing/coherency

[Oleksandr Andrushchenko]

Hello, all!

My driver (Xen para-virtualized frontend) in some scenarios uses
drm_gem_get_pages for allocating backing storage for dumb buffers.
There are use-cases which showed some artifacts on the screen
(modetest, other) which were worked around by flushing pages of the
buffer on page flip with drm_clflush_pages. But, the problem here
is that drm_clflush_pages is not available on ARM platforms (it is a NOP)
and doing flushes on every page flip seems to be non-optimal.

Other drivers that use drm_gem_get_pages seem to use DMA map/unmap
on the shmem backed buffer (this is from where drm_gem_get_pages
allocates the pages) and this is an obvious approach as the buffer needs
to be shared with real HW for DMA - please correct me if my understanding
here is wrong.

This is the part I missed in my implementation as I don't really have a
HW device which needs DMA, but a backend running in a different Xen domain.

Thus, as the buffer is backed with cachable pages the backend may see

artifacts on its side.


I am looking for some advices on what would be the best option to
make sure dumb buffers are not flushed every page flip and still
the memory remains coherent to the backend. I have implemented a
DMA map/unmap of the shmem pages on GEM object creation/destruction
and this does solve the problem, but as the backend is not really
a DMA device this is a bit misleading.

Is there any other (more?) suitable/preferable way(s) to achieve the same?

Thank you,
Oleksandr


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 2/3] drm/xen-front: Use Xen common shared buffer implementation

[Oleksandr Andrushchenko]

Hi, Daniel!

On 11/22/18 4:33 PM, Daniel Vetter wrote:

On Thu, Nov 22, 2018 at 12:02:29PM +0200, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

Use page directory based shared buffer implementation
now available as common code for Xen frontend drivers.

Signed-off-by: Oleksandr Andrushchenko 
---
  drivers/gpu/drm/xen/Kconfig   |   1 +
  drivers/gpu/drm/xen/Makefile  |   1 -
  drivers/gpu/drm/xen/xen_drm_front.c   |  60 ++--
  drivers/gpu/drm/xen/xen_drm_front_gem.c   |   1 -
  drivers/gpu/drm/xen/xen_drm_front_shbuf.c | 414 --
  drivers/gpu/drm/xen/xen_drm_front_shbuf.h |  64 
  6 files changed, 30 insertions(+), 511 deletions(-)
  delete mode 100644 drivers/gpu/drm/xen/xen_drm_front_shbuf.c
  delete mode 100644 drivers/gpu/drm/xen/xen_drm_front_shbuf.h

diff --git a/drivers/gpu/drm/xen/Kconfig b/drivers/gpu/drm/xen/Kconfig
index 4cca160782ab..f969d486855d 100644
--- a/drivers/gpu/drm/xen/Kconfig
+++ b/drivers/gpu/drm/xen/Kconfig
@@ -12,6 +12,7 @@ config DRM_XEN_FRONTEND
select DRM_KMS_HELPER
select VIDEOMODE_HELPERS
select XEN_XENBUS_FRONTEND
+   select XEN_FRONT_PGDIR_SHBUF
help
  Choose this option if you want to enable a para-virtualized
  frontend DRM/KMS driver for Xen guest OSes.
diff --git a/drivers/gpu/drm/xen/Makefile b/drivers/gpu/drm/xen/Makefile
index 712afff5ffc3..825905f67faa 100644
--- a/drivers/gpu/drm/xen/Makefile
+++ b/drivers/gpu/drm/xen/Makefile
@@ -4,7 +4,6 @@ drm_xen_front-objs := xen_drm_front.o \
  xen_drm_front_kms.o \
  xen_drm_front_conn.o \
  xen_drm_front_evtchnl.o \
- xen_drm_front_shbuf.o \
  xen_drm_front_cfg.o \
  xen_drm_front_gem.o
  
diff --git a/drivers/gpu/drm/xen/xen_drm_front.c b/drivers/gpu/drm/xen/xen_drm_front.c

index 6b6d5ab82ec3..9597544fecc1 100644
--- a/drivers/gpu/drm/xen/xen_drm_front.c
+++ b/drivers/gpu/drm/xen/xen_drm_front.c
@@ -19,6 +19,7 @@
  #include 
  #include 
  
+#include 

  #include 
  
  #include "xen_drm_front.h"

@@ -26,28 +27,20 @@
  #include "xen_drm_front_evtchnl.h"
  #include "xen_drm_front_gem.h"
  #include "xen_drm_front_kms.h"
-#include "xen_drm_front_shbuf.h"
  
  struct xen_drm_front_dbuf {

struct list_head list;
u64 dbuf_cookie;
u64 fb_cookie;
-   struct xen_drm_front_shbuf *shbuf;
+
+   struct xen_front_pgdir_shbuf shbuf;
  };
  
-static int dbuf_add_to_list(struct xen_drm_front_info *front_info,

-   struct xen_drm_front_shbuf *shbuf, u64 dbuf_cookie)
+static void dbuf_add_to_list(struct xen_drm_front_info *front_info,
+struct xen_drm_front_dbuf *dbuf, u64 dbuf_cookie)
  {
-   struct xen_drm_front_dbuf *dbuf;
-
-   dbuf = kzalloc(sizeof(*dbuf), GFP_KERNEL);
-   if (!dbuf)
-   return -ENOMEM;
-
dbuf->dbuf_cookie = dbuf_cookie;
-   dbuf->shbuf = shbuf;
list_add(>list, _info->dbuf_list);
-   return 0;
  }
  
  static struct xen_drm_front_dbuf *dbuf_get(struct list_head *dbuf_list,

@@ -64,11 +57,14 @@ static struct xen_drm_front_dbuf *dbuf_get(struct list_head 
*dbuf_list,
  
  static void dbuf_flush_fb(struct list_head *dbuf_list, u64 fb_cookie)

  {
+#if IS_ENABLED(CONFIG_X86)
struct xen_drm_front_dbuf *buf, *q;
  
  	list_for_each_entry_safe(buf, q, dbuf_list, list)

if (buf->fb_cookie == fb_cookie)
-   xen_drm_front_shbuf_flush(buf->shbuf);
+   drm_clflush_pages(buf->shbuf.pages,
+ buf->shbuf.num_pages);
+#endif

Why do we need to clflush here only on x86? Feels fairly fishy, but I
think we've discussed this problem for long time with the original
submission already.


First of all sorry for the late response: it took me quite some time

to dig deeper into the flushing issue and better understand the

root cause of this. At the moment my understanding is that this

flushing just hides the real problem and must be removed totally.

So, in v2 of this patch I will remove dbuf_flush_fb.

I am about to start a dedicated discussion on dri-devel with this respect

as I am still concerned about the way this can be solved: I have a suspect

and couple of solutions/workarounds, but I do also need some advise

from DRI community on this.



Anyway, I'm all for code duplication removal, so if the Xen folks are
happy with patch 1, this one here has my ack. Might also be best to merge
all three through the Xen tree. Fallback would be xen folks sending a
topic pull request with these 3 patches to drm-misc and takashi's sound
tree.
-Daniel


  }
  
  static void dbuf_free(struct list_head *dbuf_list, u64 dbuf_cookie)

@@ -78,8 +74,8 @@ static void dbuf_free(struct list_head *dbuf_list, 

Re: [Xen-devel] [alsa-devel] [PATCH 3/3] ALSA: xen-front: Use Xen common shared buffer implementation

[Oleksandr Andrushchenko]

On 11/22/18 5:47 PM, Takashi Iwai wrote:

On Thu, 22 Nov 2018 11:02:30 +0100,
Oleksandr Andrushchenko wrote:

@@ -214,12 +221,19 @@ static void stream_clear(struct 
xen_snd_front_pcm_stream_info *stream)
stream->out_frames = 0;
atomic_set(>hw_ptr, 0);
xen_snd_front_evtchnl_pair_clear(stream->evt_pair);
-   xen_snd_front_shbuf_clear(>sh_buf);
+   memset(>shbuf, 0, sizeof(stream->shbuf));
+   stream->buffer = NULL;
+   stream->buffer_sz = 0;
+   stream->pages = NULL;
+   stream->num_pages = 0;
  }
  
  static void stream_free(struct xen_snd_front_pcm_stream_info *stream)

  {
-   xen_snd_front_shbuf_free(>sh_buf);
+   xen_front_pgdir_shbuf_unmap(>shbuf);
+   xen_front_pgdir_shbuf_free(>shbuf);
+   free_pages_exact(stream->buffer, stream->buffer_sz);
+   kfree(stream->pages);
stream_clear(stream);
  }
  
@@ -421,10 +435,34 @@ static int alsa_close(struct snd_pcm_substream *substream)

return 0;
  }
  
+static int shbuf_setup_backstore(struct xen_snd_front_pcm_stream_info *stream,

+size_t buffer_sz)
+{
+   int i;
+
+   stream->buffer_sz = buffer_sz;
+   stream->buffer = alloc_pages_exact(stream->buffer_sz, GFP_KERNEL);
+   if (!stream->buffer)
+   return -ENOMEM;

This keeps the NULL stream->buffer, and then the caller goes to the
error path via stream_free() which will lead to an Oops due to the
unconditional call of free_pages_exact().


You are absolutely right, will fix

Thank you for catching this




thanks,

Takashi


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 2/3] drm/xen-front: Use Xen common shared buffer implementation

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Use page directory based shared buffer implementation
now available as common code for Xen frontend drivers.

Signed-off-by: Oleksandr Andrushchenko 
---
 drivers/gpu/drm/xen/Kconfig   |   1 +
 drivers/gpu/drm/xen/Makefile  |   1 -
 drivers/gpu/drm/xen/xen_drm_front.c   |  60 ++--
 drivers/gpu/drm/xen/xen_drm_front_gem.c   |   1 -
 drivers/gpu/drm/xen/xen_drm_front_shbuf.c | 414 --
 drivers/gpu/drm/xen/xen_drm_front_shbuf.h |  64 
 6 files changed, 30 insertions(+), 511 deletions(-)
 delete mode 100644 drivers/gpu/drm/xen/xen_drm_front_shbuf.c
 delete mode 100644 drivers/gpu/drm/xen/xen_drm_front_shbuf.h

diff --git a/drivers/gpu/drm/xen/Kconfig b/drivers/gpu/drm/xen/Kconfig
index 4cca160782ab..f969d486855d 100644
--- a/drivers/gpu/drm/xen/Kconfig
+++ b/drivers/gpu/drm/xen/Kconfig
@@ -12,6 +12,7 @@ config DRM_XEN_FRONTEND
select DRM_KMS_HELPER
select VIDEOMODE_HELPERS
select XEN_XENBUS_FRONTEND
+   select XEN_FRONT_PGDIR_SHBUF
help
  Choose this option if you want to enable a para-virtualized
  frontend DRM/KMS driver for Xen guest OSes.
diff --git a/drivers/gpu/drm/xen/Makefile b/drivers/gpu/drm/xen/Makefile
index 712afff5ffc3..825905f67faa 100644
--- a/drivers/gpu/drm/xen/Makefile
+++ b/drivers/gpu/drm/xen/Makefile
@@ -4,7 +4,6 @@ drm_xen_front-objs := xen_drm_front.o \
  xen_drm_front_kms.o \
  xen_drm_front_conn.o \
  xen_drm_front_evtchnl.o \
- xen_drm_front_shbuf.o \
  xen_drm_front_cfg.o \
  xen_drm_front_gem.o
 
diff --git a/drivers/gpu/drm/xen/xen_drm_front.c 
b/drivers/gpu/drm/xen/xen_drm_front.c
index 6b6d5ab82ec3..9597544fecc1 100644
--- a/drivers/gpu/drm/xen/xen_drm_front.c
+++ b/drivers/gpu/drm/xen/xen_drm_front.c
@@ -19,6 +19,7 @@
 #include 
 #include 
 
+#include 
 #include 
 
 #include "xen_drm_front.h"
@@ -26,28 +27,20 @@
 #include "xen_drm_front_evtchnl.h"
 #include "xen_drm_front_gem.h"
 #include "xen_drm_front_kms.h"
-#include "xen_drm_front_shbuf.h"
 
 struct xen_drm_front_dbuf {
struct list_head list;
u64 dbuf_cookie;
u64 fb_cookie;
-   struct xen_drm_front_shbuf *shbuf;
+
+   struct xen_front_pgdir_shbuf shbuf;
 };
 
-static int dbuf_add_to_list(struct xen_drm_front_info *front_info,
-   struct xen_drm_front_shbuf *shbuf, u64 dbuf_cookie)
+static void dbuf_add_to_list(struct xen_drm_front_info *front_info,
+struct xen_drm_front_dbuf *dbuf, u64 dbuf_cookie)
 {
-   struct xen_drm_front_dbuf *dbuf;
-
-   dbuf = kzalloc(sizeof(*dbuf), GFP_KERNEL);
-   if (!dbuf)
-   return -ENOMEM;
-
dbuf->dbuf_cookie = dbuf_cookie;
-   dbuf->shbuf = shbuf;
list_add(>list, _info->dbuf_list);
-   return 0;
 }
 
 static struct xen_drm_front_dbuf *dbuf_get(struct list_head *dbuf_list,
@@ -64,11 +57,14 @@ static struct xen_drm_front_dbuf *dbuf_get(struct list_head 
*dbuf_list,
 
 static void dbuf_flush_fb(struct list_head *dbuf_list, u64 fb_cookie)
 {
+#if IS_ENABLED(CONFIG_X86)
struct xen_drm_front_dbuf *buf, *q;
 
list_for_each_entry_safe(buf, q, dbuf_list, list)
if (buf->fb_cookie == fb_cookie)
-   xen_drm_front_shbuf_flush(buf->shbuf);
+   drm_clflush_pages(buf->shbuf.pages,
+ buf->shbuf.num_pages);
+#endif
 }
 
 static void dbuf_free(struct list_head *dbuf_list, u64 dbuf_cookie)
@@ -78,8 +74,8 @@ static void dbuf_free(struct list_head *dbuf_list, u64 
dbuf_cookie)
list_for_each_entry_safe(buf, q, dbuf_list, list)
if (buf->dbuf_cookie == dbuf_cookie) {
list_del(>list);
-   xen_drm_front_shbuf_unmap(buf->shbuf);
-   xen_drm_front_shbuf_free(buf->shbuf);
+   xen_front_pgdir_shbuf_unmap(>shbuf);
+   xen_front_pgdir_shbuf_free(>shbuf);
kfree(buf);
break;
}
@@ -91,8 +87,8 @@ static void dbuf_free_all(struct list_head *dbuf_list)
 
list_for_each_entry_safe(buf, q, dbuf_list, list) {
list_del(>list);
-   xen_drm_front_shbuf_unmap(buf->shbuf);
-   xen_drm_front_shbuf_free(buf->shbuf);
+   xen_front_pgdir_shbuf_unmap(>shbuf);
+   xen_front_pgdir_shbuf_free(>shbuf);
kfree(buf);
}
 }
@@ -171,9 +167,9 @@ int xen_drm_front_dbuf_create(struct xen_drm_front_info 
*front_info,
  u32 bpp, u64 size, struct page **pages)
 {
struct xen_drm_front_evtchnl *evtchnl;
-   struct xen_drm_fr

[Xen-devel] [PATCH 3/3] ALSA: xen-front: Use Xen common shared buffer implementation

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Use page directory based shared buffer implementation
now available as common code for Xen frontend drivers.

Signed-off-by: Oleksandr Andrushchenko 
---
 sound/xen/Kconfig   |   1 +
 sound/xen/Makefile  |   1 -
 sound/xen/xen_snd_front.c   |   7 +-
 sound/xen/xen_snd_front.h   |   4 +-
 sound/xen/xen_snd_front_alsa.c  | 101 +
 sound/xen/xen_snd_front_shbuf.c | 194 
 sound/xen/xen_snd_front_shbuf.h |  36 --
 7 files changed, 83 insertions(+), 261 deletions(-)
 delete mode 100644 sound/xen/xen_snd_front_shbuf.c
 delete mode 100644 sound/xen/xen_snd_front_shbuf.h

diff --git a/sound/xen/Kconfig b/sound/xen/Kconfig
index 4f1fceea82d2..e4d7beb4df1c 100644
--- a/sound/xen/Kconfig
+++ b/sound/xen/Kconfig
@@ -5,6 +5,7 @@ config SND_XEN_FRONTEND
depends on XEN
select SND_PCM
select XEN_XENBUS_FRONTEND
+   select XEN_FRONT_PGDIR_SHBUF
help
  Choose this option if you want to enable a para-virtualized
  frontend sound driver for Xen guest OSes.
diff --git a/sound/xen/Makefile b/sound/xen/Makefile
index 1e6470ecc2f2..24031775b715 100644
--- a/sound/xen/Makefile
+++ b/sound/xen/Makefile
@@ -3,7 +3,6 @@
 snd_xen_front-objs := xen_snd_front.o \
  xen_snd_front_cfg.o \
  xen_snd_front_evtchnl.o \
- xen_snd_front_shbuf.o \
  xen_snd_front_alsa.o
 
 obj-$(CONFIG_SND_XEN_FRONTEND) += snd_xen_front.o
diff --git a/sound/xen/xen_snd_front.c b/sound/xen/xen_snd_front.c
index b089b13b5160..a9e5c2cd7698 100644
--- a/sound/xen/xen_snd_front.c
+++ b/sound/xen/xen_snd_front.c
@@ -16,12 +16,12 @@
 #include 
 #include 
 
+#include 
 #include 
 
 #include "xen_snd_front.h"
 #include "xen_snd_front_alsa.h"
 #include "xen_snd_front_evtchnl.h"
-#include "xen_snd_front_shbuf.h"
 
 static struct xensnd_req *
 be_stream_prepare_req(struct xen_snd_front_evtchnl *evtchnl, u8 operation)
@@ -82,7 +82,7 @@ int xen_snd_front_stream_query_hw_param(struct 
xen_snd_front_evtchnl *evtchnl,
 }
 
 int xen_snd_front_stream_prepare(struct xen_snd_front_evtchnl *evtchnl,
-struct xen_snd_front_shbuf *sh_buf,
+struct xen_front_pgdir_shbuf *shbuf,
 u8 format, unsigned int channels,
 unsigned int rate, u32 buffer_sz,
 u32 period_sz)
@@ -99,7 +99,8 @@ int xen_snd_front_stream_prepare(struct xen_snd_front_evtchnl 
*evtchnl,
req->op.open.pcm_rate = rate;
req->op.open.buffer_sz = buffer_sz;
req->op.open.period_sz = period_sz;
-   req->op.open.gref_directory = xen_snd_front_shbuf_get_dir_start(sh_buf);
+   req->op.open.gref_directory =
+   xen_front_pgdir_shbuf_get_dir_start(shbuf);
mutex_unlock(>ring_io_lock);
 
ret = be_stream_do_io(evtchnl);
diff --git a/sound/xen/xen_snd_front.h b/sound/xen/xen_snd_front.h
index a2ea2463bcc5..05611f113b94 100644
--- a/sound/xen/xen_snd_front.h
+++ b/sound/xen/xen_snd_front.h
@@ -16,7 +16,7 @@
 struct xen_snd_front_card_info;
 struct xen_snd_front_evtchnl;
 struct xen_snd_front_evtchnl_pair;
-struct xen_snd_front_shbuf;
+struct xen_front_pgdir_shbuf;
 struct xensnd_query_hw_param;
 
 struct xen_snd_front_info {
@@ -35,7 +35,7 @@ int xen_snd_front_stream_query_hw_param(struct 
xen_snd_front_evtchnl *evtchnl,
struct xensnd_query_hw_param 
*hw_param_resp);
 
 int xen_snd_front_stream_prepare(struct xen_snd_front_evtchnl *evtchnl,
-struct xen_snd_front_shbuf *sh_buf,
+struct xen_front_pgdir_shbuf *shbuf,
 u8 format, unsigned int channels,
 unsigned int rate, u32 buffer_sz,
 u32 period_sz);
diff --git a/sound/xen/xen_snd_front_alsa.c b/sound/xen/xen_snd_front_alsa.c
index 129180e17db1..5b874a7014e0 100644
--- a/sound/xen/xen_snd_front_alsa.c
+++ b/sound/xen/xen_snd_front_alsa.c
@@ -15,17 +15,24 @@
 #include 
 
 #include 
+#include 
 
 #include "xen_snd_front.h"
 #include "xen_snd_front_alsa.h"
 #include "xen_snd_front_cfg.h"
 #include "xen_snd_front_evtchnl.h"
-#include "xen_snd_front_shbuf.h"
 
 struct xen_snd_front_pcm_stream_info {
struct xen_snd_front_info *front_info;
struct xen_snd_front_evtchnl_pair *evt_pair;
-   struct xen_snd_front_shbuf sh_buf;
+
+   /* This is the shared buffer with its backing storage. */
+   struct xen_front_pgdir_shbuf shbuf;
+   u8 *buffer;
+   size_t buffer_sz;
+   int num_pages;
+   struct page **pages;
+
int index;
 
bool is_open;
@@ -214,12 +221,19 @@ static void stream_clear(

[Xen-devel] [PATCH 1/3] xen: Introduce shared buffer helpers for page directory...

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

based frontends. Currently the frontends which implement
similar code for sharing big buffers between frontend and
backend are para-virtualized DRM and sound drivers.
Both define the same way to share grant references of a
data buffer with the corresponding backend with little
differences.

Move shared code into a helper module, so there is a single
implementation of the same functionality for all.

Signed-off-by: Oleksandr Andrushchenko 
---
 drivers/xen/Kconfig |   3 +
 drivers/xen/Makefile|   1 +
 drivers/xen/xen-front-pgdir-shbuf.c | 553 
 include/xen/xen-front-pgdir-shbuf.h |  89 +
 4 files changed, 646 insertions(+)
 create mode 100644 drivers/xen/xen-front-pgdir-shbuf.c
 create mode 100644 include/xen/xen-front-pgdir-shbuf.h

diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
index 815b9e9bb975..838b66a9a0e7 100644
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -340,4 +340,7 @@ config XEN_SYMS
 config XEN_HAVE_VPMU
bool
 
+config XEN_FRONT_PGDIR_SHBUF
+   tristate
+
 endmenu
diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile
index 3e542f60f29f..c48927a58e10 100644
--- a/drivers/xen/Makefile
+++ b/drivers/xen/Makefile
@@ -44,3 +44,4 @@ xen-gntdev-y  := gntdev.o
 xen-gntdev-$(CONFIG_XEN_GNTDEV_DMABUF) += gntdev-dmabuf.o
 xen-gntalloc-y := gntalloc.o
 xen-privcmd-y  := privcmd.o privcmd-buf.o
+obj-$(CONFIG_XEN_FRONT_PGDIR_SHBUF)+= xen-front-pgdir-shbuf.o
diff --git a/drivers/xen/xen-front-pgdir-shbuf.c 
b/drivers/xen/xen-front-pgdir-shbuf.c
new file mode 100644
index ..48a658dc7ccf
--- /dev/null
+++ b/drivers/xen/xen-front-pgdir-shbuf.c
@@ -0,0 +1,553 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+
+/*
+ * Xen frontend/backend page directory based shared buffer
+ * helper module.
+ *
+ * Copyright (C) 2018 EPAM Systems Inc.
+ *
+ * Author: Oleksandr Andrushchenko 
+ */
+
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+
+#ifndef GRANT_INVALID_REF
+/*
+ * FIXME: usage of grant reference 0 as invalid grant reference:
+ * grant reference 0 is valid, but never exposed to a PV driver,
+ * because of the fact it is already in use/reserved by the PV console.
+ */
+#define GRANT_INVALID_REF  0
+#endif
+
+/**
+ * This structure represents the structure of a shared page
+ * that contains grant references to the pages of the shared
+ * buffer. This structure is common to many Xen para-virtualized
+ * protocols at include/xen/interface/io/
+ */
+struct xen_page_directory {
+   grant_ref_t gref_dir_next_page;
+   grant_ref_t gref[1]; /* Variable length */
+};
+
+/**
+ * Shared buffer ops which are differently implemented
+ * depending on the allocation mode, e.g. if the buffer
+ * is allocated by the corresponding backend or frontend.
+ * Some of the operations.
+ */
+struct xen_front_pgdir_shbuf_ops {
+   /*
+* Calculate number of grefs required to handle this buffer,
+* e.g. if grefs are required for page directory only or the buffer
+* pages as well.
+*/
+   void (*calc_num_grefs)(struct xen_front_pgdir_shbuf *buf);
+
+   /* Fill page directory according to para-virtual display protocol. */
+   void (*fill_page_dir)(struct xen_front_pgdir_shbuf *buf);
+
+   /* Claim grant references for the pages of the buffer. */
+   int (*grant_refs_for_buffer)(struct xen_front_pgdir_shbuf *buf,
+grant_ref_t *priv_gref_head, int gref_idx);
+
+   /* Map grant references of the buffer. */
+   int (*map)(struct xen_front_pgdir_shbuf *buf);
+
+   /* Unmap grant references of the buffer. */
+   int (*unmap)(struct xen_front_pgdir_shbuf *buf);
+};
+
+/**
+ * Get granted reference to the very first page of the
+ * page directory. Usually this is passed to the backend,
+ * so it can find/fill the grant references to the buffer's
+ * pages.
+ *
+ * \param buf shared buffer which page directory is of interest.
+ * \return granted reference to the very first page of the
+ * page directory.
+ */
+grant_ref_t
+xen_front_pgdir_shbuf_get_dir_start(struct xen_front_pgdir_shbuf *buf)
+{
+   if (!buf->grefs)
+   return GRANT_INVALID_REF;
+
+   return buf->grefs[0];
+}
+EXPORT_SYMBOL_GPL(xen_front_pgdir_shbuf_get_dir_start);
+
+/**
+ * Map granted references of the shared buffer.
+ *
+ * Depending on the shared buffer mode of allocation
+ * (be_alloc flag) this can either do nothing (for buffers
+ * shared by the frontend itself) or map the provided granted
+ * references onto the backing storage (buf->pages).
+ *
+ * \param buf shared buffer which grants to be maped.
+ * \return zero on success or a negative number on failure.
+ */
+int xen_front_pgdir_shbuf_map(struct xen_front_pgdir_shbuf *buf)
+{
+   if (buf->ops &

Re: [Xen-devel] [PATCH 5/9] drm/xen/xen_drm_front_gem.c: Convert to use vm_insert_range

[Oleksandr Andrushchenko]

On 11/19/18 12:42 PM, Souptick Joarder wrote:

On Mon, Nov 19, 2018 at 3:22 PM Oleksandr Andrushchenko
 wrote:

On 11/15/18 5:49 PM, Souptick Joarder wrote:

Convert to use vm_insert_range() to map range of kernel
memory to user vma.

Signed-off-by: Souptick Joarder 
Reviewed-by: Matthew Wilcox 
---
   drivers/gpu/drm/xen/xen_drm_front_gem.c | 20 ++--
   1 file changed, 6 insertions(+), 14 deletions(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index 47ff019..a3eade6 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -225,8 +225,7 @@ struct drm_gem_object *
   static int gem_mmap_obj(struct xen_gem_object *xen_obj,
   struct vm_area_struct *vma)
   {
- unsigned long addr = vma->vm_start;
- int i;
+ int err;

I would love to keep ret, not err

Sure, will add it in v2.
But I think, err is more appropriate here.


I used "ret" throughout the driver, so this is just to remain consistent:

grep -rnw err drivers/gpu/drm/xen/ | wc -l
0
grep -rnw ret drivers/gpu/drm/xen/ | wc -l
204


   /*
* clear the VM_PFNMAP flag that was set by drm_gem_mmap(), and set the
@@ -247,18 +246,11 @@ static int gem_mmap_obj(struct xen_gem_object *xen_obj,
* FIXME: as we insert all the pages now then no .fault handler must
* be called, so don't provide one
*/
- for (i = 0; i < xen_obj->num_pages; i++) {
- int ret;
-
- ret = vm_insert_page(vma, addr, xen_obj->pages[i]);
- if (ret < 0) {
- DRM_ERROR("Failed to insert pages into vma: %d\n", ret);
- return ret;
- }
-
- addr += PAGE_SIZE;
- }
- return 0;
+ err = vm_insert_range(vma, vma->vm_start, xen_obj->pages,
+ xen_obj->num_pages);
+ if (err < 0)
+ DRM_ERROR("Failed to insert pages into vma: %d\n", err);
+ return err;
   }

   int xen_drm_front_gem_mmap(struct file *filp, struct vm_area_struct *vma)

With the above fixed,

Reviewed-by: Oleksandr Andrushchenko 



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 5/9] drm/xen/xen_drm_front_gem.c: Convert to use vm_insert_range

[Oleksandr Andrushchenko]

On 11/15/18 5:49 PM, Souptick Joarder wrote:

Convert to use vm_insert_range() to map range of kernel
memory to user vma.

Signed-off-by: Souptick Joarder 
Reviewed-by: Matthew Wilcox 
---
  drivers/gpu/drm/xen/xen_drm_front_gem.c | 20 ++--
  1 file changed, 6 insertions(+), 14 deletions(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index 47ff019..a3eade6 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -225,8 +225,7 @@ struct drm_gem_object *
  static int gem_mmap_obj(struct xen_gem_object *xen_obj,
struct vm_area_struct *vma)
  {
-   unsigned long addr = vma->vm_start;
-   int i;
+   int err;

I would love to keep ret, not err
  
  	/*

 * clear the VM_PFNMAP flag that was set by drm_gem_mmap(), and set the
@@ -247,18 +246,11 @@ static int gem_mmap_obj(struct xen_gem_object *xen_obj,
 * FIXME: as we insert all the pages now then no .fault handler must
 * be called, so don't provide one
 */
-   for (i = 0; i < xen_obj->num_pages; i++) {
-   int ret;
-
-   ret = vm_insert_page(vma, addr, xen_obj->pages[i]);
-   if (ret < 0) {
-   DRM_ERROR("Failed to insert pages into vma: %d\n", ret);
-   return ret;
-   }
-
-   addr += PAGE_SIZE;
-   }
-   return 0;
+   err = vm_insert_range(vma, vma->vm_start, xen_obj->pages,
+   xen_obj->num_pages);
+   if (err < 0)
+   DRM_ERROR("Failed to insert pages into vma: %d\n", err);
+   return err;
  }
  
  int xen_drm_front_gem_mmap(struct file *filp, struct vm_area_struct *vma)


With the above fixed,

Reviewed-by: Oleksandr Andrushchenko 


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5 21/25] xen: support console_switching between Dom0 and DomUs on ARM

[Oleksandr Andrushchenko]

On 10/23/2018 05:03 AM, Stefano Stabellini wrote:

Today Ctrl-AAA is used to switch between Xen and Dom0. Extend the
mechanism to allow for switching between Xen, Dom0, and any of the
initial DomU created from Xen alongside Dom0 out of information provided
via device tree.

Rename xen_rx to console_rx to match the new behavior.

Clarify existing comment about "notify the guest", making it clear that
it is only about the hardware domain.

Signed-off-by: Stefano Stabellini 
CC: andrew.coop...@citrix.com
CC: george.dun...@eu.citrix.com
CC: ian.jack...@eu.citrix.com
CC: jbeul...@suse.com
CC: konrad.w...@oracle.com
CC: t...@xen.org
CC: wei.l...@citrix.com
---
Changes in v5:
- move patch earlier and disable code that calls vpl011_rx_char_xen (not
   defined yet)
- improve comments
- replace ifs with a switch
- code style

Changes in v4:
- handle console_rx == 0 in console_input_domain
- use rcu_lock_by_domain instead of get_domain_by_id
- handle the case where the returned domain is NULL
- send_global_virq(VIRQ_CONSOLE) only when chars are for Dom0
- make console_rx unsigned int
- fix comment
- code readability improvement
- fix opt_conswitch[1] == 'x' case
- move console_input_domain to next patch

Changes in v3:
- only call vpl011 functions ifdef CONFIG_SBSA_VUART_CONSOLE
- add blank line and spaces
- remove xen_rx from print messages
- rename xen_rx to console_rx
- keep switch_serial_input() at boot
- add better comments
- fix existing comment
- add warning if no guests console/uart is available
- add console_input_domain function

Changes in v2:
- only call vpl011_rx_char if the vpl011 has been initialized
---
  xen/drivers/char/console.c | 79 ++
  1 file changed, 65 insertions(+), 14 deletions(-)

diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index 3b75f7a..75172e7 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -31,10 +31,13 @@
  #include 
  #include 
  #include 
+#include 
  
  #ifdef CONFIG_X86

  #include 
  #include 
+#else
+#include 
  #endif
  
  /* console: comma-separated list of console outputs. */

@@ -391,31 +394,79 @@ static void dump_console_ring_key(unsigned char key)
  free_xenheap_pages(buf, order);
  }
  
-/* CTRL- switches input direction between Xen and DOM0. */

+/*
+ * CTRL- changes input direction, rotating among Xen, Dom0,
+ * and DomUs.
+ */
  #define switch_code (opt_conswitch[0]-'a'+1)
-static int __read_mostly xen_rx = 1; /* FALSE => input passed to domain 0. */
+/*
+ * console_rx=0 => input to xen
+ * console_rx=1 => input to dom0
+ * console_rx=N => input to dom(N-1)
+ */
+static unsigned int __read_mostly console_rx = 0;
  
  static void switch_serial_input(void)

  {
-static char *input_str[2] = { "DOM0", "Xen" };
-xen_rx = !xen_rx;
-printk("*** Serial input -> %s", input_str[xen_rx]);
+if ( console_rx++ == max_init_domid + 1 )
+console_rx = 0;
+
+if ( console_rx == 0 )
+printk("*** Serial input to Xen");
+else
+printk("*** Serial input to DOM%d", console_rx - 1);
+
  if ( switch_code )
-printk(" (type 'CTRL-%c' three times to switch input to %s)",
-   opt_conswitch[0], input_str[!xen_rx]);
+printk(" (type 'CTRL-%c' three times to switch input)",
+   opt_conswitch[0]);
  printk("\n");
  }
  
  static void __serial_rx(char c, struct cpu_user_regs *regs)

  {
-if ( xen_rx )
+switch ( console_rx )
+{
+case 0:
  return handle_keypress(c, regs);
+case 1:
+{
+/*
+ * Deliver input to the hardware domain buffer, unless it is
+ * already full.
+ */
+if ( (serial_rx_prod - serial_rx_cons) != SERIAL_RX_SIZE )
+serial_rx_ring[SERIAL_RX_MASK(serial_rx_prod++)] = c;
+
+/*
+ * Always notify the hardware domain: prevents receive path from
+ * getting stuck.
+ */
+send_global_virq(VIRQ_CONSOLE);
+break;
+}

+ * console_rx=0 => input to xen
+ * console_rx=1 => input to dom0
+ * console_rx=N => input to dom(N-1)
So, why do you only handle case 0/1?

+#if 0

Do you need this #if 0?

+default:
+{
+struct domain *d = rcu_lock_domain_by_any_id(console_rx - 1);
+
+/*
+ * If we have a properly initialized vpl011 console for the
+ * domain, without a full PV ring to Dom0 (in that case input
+ * comes from the PV ring), then send the character to it.
+ */
+if ( d != NULL &&
+ !d->arch.vpl011.backend_in_domain &&
+ d->arch.vpl011.backend.xen != NULL )
+vpl011_rx_char_xen(d, c);
+else
+printk("Cannot send chars to Dom%d: no UART available\n",
+   console_rx - 1);
  
-/* Deliver input to guest buffer, unless it is already full. */

-if ( (serial_rx_prod-serial_rx_cons) != SERIAL_RX_SIZE )
-

Re: [Xen-devel] [PATCH v5 23/25] xen/vpl011: buffer out chars when the backend is xen

[Oleksandr Andrushchenko]

On 10/23/2018 05:03 AM, Stefano Stabellini wrote:

To avoid mixing the output of different domains on the console, buffer
the output chars and print line by line. Unless the domain has input
from the serial, in which case we want to print char by char for a
smooth user experience.

The size of SBSA_UART_OUT_BUF_SIZE is arbitrary, choose the same size
as VUART_BUT_SIZE used in vuart.c.

s/VUART_BUT_SIZE/VUART_BUF_SIZE


Export a function named console_input_domain() to allow others to know
which domains has input at a given time.

Signed-off-by: Stefano Stabellini 
CC: andrew.coop...@citrix.com
CC: george.dun...@eu.citrix.com
CC: ian.jack...@eu.citrix.com
CC: jbeul...@suse.com
CC: konrad.w...@oracle.com
CC: t...@xen.org
CC: wei.l...@citrix.com
---
XXX: merge this patch with "xen/arm: Allow vpl011 to be used by DomU" on
  commit

Changes in v5:
- use rcu_lock in console_input_domain
- rcu_unlock at the end of vpl011_write_data_xen
- add a comment on top of console_input_domain as a reminder that the
   caller needs to rcu_unlock

Changes in v4:
- make SBSA_UART_OUT_BUF_SIZE the same size of VUART_BUT_SIZE
- rearrange the code to be clearer input and != input cases
- code style
- add \n when printing the out buffer because is full
- don't print prefix for input domain
---
  xen/arch/arm/vpl011.c| 36 +---
  xen/drivers/char/console.c   |  8 
  xen/include/asm-arm/vpl011.h |  4 
  xen/include/xen/console.h|  2 ++
  4 files changed, 47 insertions(+), 3 deletions(-)

diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
index 131507e..f7db18b 100644
--- a/xen/arch/arm/vpl011.c
+++ b/xen/arch/arm/vpl011.c
@@ -28,6 +28,7 @@
  #include 
  #include 
  #include 
+#include 
  #include 
  #include 
  #include 
@@ -85,18 +86,47 @@ static void vpl011_write_data_xen(struct domain *d, uint8_t 
data)
  {
  unsigned long flags;
  struct vpl011 *vpl011 = >arch.vpl011;
+struct vpl011_xen_backend *intf = vpl011->backend.xen;
+struct domain *input = console_input_domain();
  
  VPL011_LOCK(d, flags);
  
-printk("%c", data);

-if (data == '\n')
-printk("DOM%u: ", d->domain_id);
+intf->out[intf->out_prod++] = data;
+if ( d == input )
+{
+if ( intf->out_prod == 1 )
+{
+printk("%c", data);
+intf->out_prod = 0;
+}
+else
+{
+if ( data != '\n' )
+intf->out[intf->out_prod++] = '\n';
+intf->out[intf->out_prod++] = '\0';
+printk("%s", intf->out);
+intf->out_prod = 0;
+}
+}
+else
+{
+if ( intf->out_prod == SBSA_UART_OUT_BUF_SIZE - 2 ||
+ data == '\n' )
+{
+if ( data != '\n' )
+intf->out[intf->out_prod++] = '\n';
+intf->out[intf->out_prod++] = '\0';
+printk("DOM%u: %s", d->domain_id, intf->out);
+intf->out_prod = 0;
+}
+}
  
  vpl011->uartris |= TXI;

  vpl011->uartfr &= ~TXFE;
  vpl011_update_interrupt_status(d);
  
  VPL011_UNLOCK(d, flags);

+rcu_unlock_domain(input);

input can be NULL. Although it won't hurt with the existing
code base things could change any time soon...

  }
  
  /*

diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index 990c51c..eb1cc1b 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -406,6 +406,14 @@ static void dump_console_ring_key(unsigned char key)
   */
  static unsigned int __read_mostly console_rx = 0;
  
+/* Make sure to rcu_unlock_domain after use */

+struct domain *console_input_domain(void)
+{
+if ( console_rx == 0 )
+return NULL;
+return rcu_lock_domain_by_id(console_rx - 1);
+}
+
  static void switch_serial_input(void)
  {
  if ( console_rx++ == max_init_domid + 1 )
diff --git a/xen/include/asm-arm/vpl011.h b/xen/include/asm-arm/vpl011.h
index 5eb6d25..ab6fd79 100644
--- a/xen/include/asm-arm/vpl011.h
+++ b/xen/include/asm-arm/vpl011.h
@@ -30,9 +30,13 @@
  #define VPL011_UNLOCK(d,flags) spin_unlock_irqrestore(&(d)->arch.vpl011.lock, 
flags)
  
  #define SBSA_UART_FIFO_SIZE 32

+/* Same size as VUART_BUT_SIZE, used in vuart.c */
+#define SBSA_UART_OUT_BUF_SIZE 128
  struct vpl011_xen_backend {
  char in[SBSA_UART_FIFO_SIZE];
+char out[SBSA_UART_OUT_BUF_SIZE];
  XENCONS_RING_IDX in_cons, in_prod;
+XENCONS_RING_IDX out_prod;
  };
  
  struct vpl011 {

diff --git a/xen/include/xen/console.h b/xen/include/xen/console.h
index 70c9911..c5a85c8 100644
--- a/xen/include/xen/console.h
+++ b/xen/include/xen/console.h
@@ -31,6 +31,8 @@ void console_end_sync(void);
  void console_start_log_everything(void);
  void console_end_log_everything(void);
  
+struct domain *console_input_domain(void);

+
  /*
   * Steal output from the console. Returns +ve identifier, else -ve error.
   * Takes the handle of the serial line to steal, and steal callback 

Re: [Xen-devel] [PATCH v5 0/6] Allow setting up shared memory areas between VMs from xl config files

[Oleksandr Andrushchenko]

On 10/24/2018 05:24 PM, Julien Grall wrote:



On 10/24/18 3:15 PM, Oleksandr Andrushchenko wrote:

Hi, Stefano!

I'm just curious if the below is on your TODO list [1]:

3. Future Directions:


Allow users to optionally share *mfn-contiguous* pages.


What is the use case for that?

No real use-case at the moment, but we might have one: for instance,
sharing some memory which is used by device firmware and domains.
But in this case iomem could probably help...


Cheers,




___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5 0/6] Allow setting up shared memory areas between VMs from xl config files

[Oleksandr Andrushchenko]

Hi, Stefano!

I'm just curious if the below is on your TODO list [1]:

3. Future Directions:


Allow users to optionally share *mfn-contiguous* pages.

Thank you,
Oleksandr

On 05/25/2018 03:55 AM, Stefano Stabellini wrote:

Hi,

This series implements a new xl config entry. Users can use the new
config entry to statically setup shared memory areas among VMs that
don't have grant table support so that they could communicate with each
other through the static shared memory areas.

It was originally developed by Zhongze, I am just updating the last few
issued that were address during the last round of reviews in January. I
tested the feature on ARM and works fine.

Cheers,

Stefano


Zhongze Liu (6):
   xen: xsm: flask: introduce XENMAPSPACE_gmfn_share for memory sharing
   libxl: introduce a new structure to represent static shared memory 
regions
   libxl: support mapping static shared memory areas during domain creation
   libxl: support unmapping static shared memory areas during domain 
destruction
   libxl:xl: add parsing code to parse "libxl_static_sshm" from xl config 
files
   docs: documentation about static shared memory regions

  docs/man/xl-static-shm-configuration.pod.5 | 257 +++
  docs/man/xl.cfg.pod.5.in   |   8 +
  docs/misc/xenstore-paths.markdown  |  47 +++
  tools/flask/policy/modules/xen.if  |   2 +
  tools/libxl/Makefile   |   4 +-
  tools/libxl/libxl.h|   6 +
  tools/libxl/libxl_arch.h   |   6 +
  tools/libxl/libxl_arm.c|  15 +
  tools/libxl/libxl_create.c |  27 ++
  tools/libxl/libxl_domain.c |   5 +
  tools/libxl/libxl_internal.h   |  16 +
  tools/libxl/libxl_sshm.c   | 512 +
  tools/libxl/libxl_types.idl|  32 +-
  tools/libxl/libxl_x86.c|  19 ++
  tools/libxl/libxlu_sshm.c  | 207 
  tools/libxl/libxlutil.h|   6 +
  tools/xl/xl_parse.c|  25 +-
  xen/arch/arm/mm.c  |   7 +-
  xen/include/public/memory.h|   8 +
  xen/include/xsm/dummy.h|  15 +
  xen/include/xsm/xsm.h  |   6 +
  xen/xsm/dummy.c|   1 +
  xen/xsm/flask/hooks.c  |  12 +
  xen/xsm/flask/policy/access_vectors|   5 +
  24 files changed, 1242 insertions(+), 6 deletions(-)
  create mode 100644 docs/man/xl-static-shm-configuration.pod.5
  create mode 100644 tools/libxl/libxl_sshm.c
  create mode 100644 tools/libxl/libxlu_sshm.c

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[1] https://lists.xen.org/archives/html/xen-devel/2017-08/msg03242.html

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v8 7/8] xen/arm: export shared memory regions as reserved-memory on device tree

[Oleksandr Andrushchenko]

On 10/09/2018 02:37 AM, Stefano Stabellini wrote:

Shared memory regions need to be advertised to the guest. Fortunately, a
device tree binding for special memory regions already exist:
reserved-memory.

Add a reserved-memory node for each shared memory region, for both
masters and slaves.

Signed-off-by: Stefano Stabellini 
---
Changes in v8:
- code style
- id is added to device tree

Changes in v7:
- change node name to xen-shmem
- add compatible property
- add id property
---
  tools/libxl/libxl_arch.h |  2 +-
  tools/libxl/libxl_arm.c  | 61 +---
  tools/libxl/libxl_dom.c  |  2 +-
  tools/libxl/libxl_x86.c  |  2 +-
  4 files changed, 61 insertions(+), 6 deletions(-)

diff --git a/tools/libxl/libxl_arch.h b/tools/libxl/libxl_arch.h
index 63c26cc..417e710 100644
--- a/tools/libxl/libxl_arch.h
+++ b/tools/libxl/libxl_arch.h
@@ -36,7 +36,7 @@ int libxl__arch_domain_create(libxl__gc *gc, 
libxl_domain_config *d_config,
  /* setup arch specific hardware description, i.e. DTB on ARM */
  _hidden
  int libxl__arch_domain_init_hw_description(libxl__gc *gc,
-   libxl_domain_build_info *info,
+   libxl_domain_config *d_config,
 libxl__domain_build_state *state,
 struct xc_dom_image *dom);
  /* finalize arch specific hardware description. */
diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
index 054ad58..c1624c0 100644
--- a/tools/libxl/libxl_arm.c
+++ b/tools/libxl/libxl_arm.c
@@ -436,6 +436,58 @@ static int make_memory_nodes(libxl__gc *gc, void *fdt,
  return 0;
  }
  
+static int make_reserved_nodes(libxl__gc *gc, void *fdt,

+   libxl_domain_config *d_config)
+{
+int res, i;
+const char *name;
+
+if (d_config->num_sshms == 0)
+return 0;
+
+res = fdt_begin_node(fdt, "reserved-memory");
+if (res) return res;
+
+res = fdt_property_cell(fdt, "#address-cells", GUEST_ROOT_ADDRESS_CELLS);
+if (res) return res;
+
+res = fdt_property_cell(fdt, "#size-cells", GUEST_ROOT_SIZE_CELLS);
+if (res) return res;
+
+res = fdt_property(fdt, "ranges", NULL, 0);
+if (res) return res;
+
+for (i = 0; i < d_config->num_sshms; i++) {
+uint64_t start = d_config->sshms[i].begin;
+
+if (d_config->sshms[i].role == LIBXL_SSHM_ROLE_SLAVE)
+start += d_config->sshms[i].offset;
+name = GCSPRINTF("xen-shmem@%"PRIx64, start);
+
+res = fdt_begin_node(fdt, name);
+if (res) return res;
+
+res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
+GUEST_ROOT_SIZE_CELLS, 1, start,
+d_config->sshms[i].size);
+if (res) return res;
+
+res = fdt_property_compat(gc, fdt, 1, "xen,shared-memory");

Shouldn't this be "shared-memory-v1" as you document it in [1]?

+if (res) return res;
+
+res = fdt_property_string(fdt, "id", d_config->sshms[i].id);
+if (res) return res;
+
+res = fdt_end_node(fdt);
+if (res) return res;
+}
+
+res = fdt_end_node(fdt);
+if (res) return res;
+
+return 0;
+}
+
  static int make_gicv2_node(libxl__gc *gc, void *fdt,
 uint64_t gicd_base, uint64_t gicd_size,
 uint64_t gicc_base, uint64_t gicc_size)
@@ -811,10 +863,11 @@ static int copy_partial_fdt(libxl__gc *gc, void *fdt, 
void *pfdt)
  
  #define FDT_MAX_SIZE (1<<20)
  
-static int libxl__prepare_dtb(libxl__gc *gc, libxl_domain_build_info *info,

+static int libxl__prepare_dtb(libxl__gc *gc, libxl_domain_config *d_config,
libxl__domain_build_state *state,
struct xc_dom_image *dom)
  {
+libxl_domain_build_info *info = _config->b_info;
  void *fdt = NULL;
  void *pfdt = NULL;
  int rc, res;
@@ -897,6 +950,7 @@ next_resize:
  FDT( make_psci_node(gc, fdt) );
  
  FDT( make_memory_nodes(gc, fdt, dom) );

+FDT( make_reserved_nodes(gc, fdt, d_config) );
  
  switch (info->arch_arm.gic_version) {

  case LIBXL_GIC_VERSION_V2:
@@ -946,12 +1000,13 @@ out:
  }
  
  int libxl__arch_domain_init_hw_description(libxl__gc *gc,

-   libxl_domain_build_info *info,
+   libxl_domain_config *d_config,
 libxl__domain_build_state *state,
 struct xc_dom_image *dom)
  {
  int rc;
  uint64_t val;
+libxl_domain_build_info *info = _config->b_info;
  
  if (info->type != LIBXL_DOMAIN_TYPE_PVH) {

  LOG(ERROR, "Unsupported Arm guest type %s",
@@ -971,7 +1026,7 @@ int libxl__arch_domain_init_hw_description(libxl__gc *gc,
  if (rc)
  return 

Re: [Xen-devel] [PATCH v8 4/8] libxl: support unmapping static shared memory areas during domain destruction

[Oleksandr Andrushchenko]

On 10/09/2018 02:37 AM, Stefano Stabellini wrote:

From: Zhongze Liu 

Author: Zhongze Liu 

Add libxl__sshm_del to unmap static shared memory areas mapped by
libxl__sshm_add during domain creation. The unmapping process is:

* For a master: decrease the refcount of the sshm region, if the refcount
   reaches 0, cleanup the whole sshm path.

* For a slave:
   1) unmap the shared pages, and cleanup related xs entries. If the
  system works normally, all the shared pages will be unmapped, so there
  won't be page leaks. In case of errors, the unmapping process will go
  on and unmap all the other pages that can be unmapped, so the other
  pages won't be leaked, either.
   2) Decrease the refcount of the sshm region, if the refcount reaches
  0, cleanup the whole sshm path.

This is for the proposal "Allow setting up shared memory areas between VMs
from xl config file" (see [1]).

[1] https://lists.xen.org/archives/html/xen-devel/2017-08/msg03242.html

Signed-off-by: Zhongze Liu 
Signed-off-by: Stefano Stabellini 

Cc: Ian Jackson 
Cc: Wei Liu 
Cc: Stefano Stabellini 
Cc: Julien Grall 
Cc: xen-de...@lists.xen.org

---
Changes in v5:
- fix typos
- add comments
- cannot move unmap before xenstore transaction because it needs to
   retrieve begin/size values from xenstore
---
  tools/libxl/libxl_domain.c   |   5 ++
  tools/libxl/libxl_internal.h |   2 +
  tools/libxl/libxl_sshm.c | 107 +++
  3 files changed, 114 insertions(+)

diff --git a/tools/libxl/libxl_domain.c b/tools/libxl/libxl_domain.c
index 3377bba..3f7ffa6 100644
--- a/tools/libxl/libxl_domain.c
+++ b/tools/libxl/libxl_domain.c
@@ -1060,6 +1060,11 @@ void libxl__destroy_domid(libxl__egc *egc, 
libxl__destroy_domid_state *dis)
  goto out;
  }
  
+rc = libxl__sshm_del(gc, domid);

+if (rc) {
+LOGD(ERROR, domid, "Deleting static shm failed.");
+}

Do you need brackets here?

+
  if (libxl__device_pci_destroy_all(gc, domid) < 0)
  LOGD(ERROR, domid, "Pci shutdown failed");
  rc = xc_domain_pause(ctx->xch, domid);
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 6f31a3d..e86d356 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -4442,6 +4442,8 @@ static inline const char *libxl__qemu_qmp_path(libxl__gc 
*gc, int domid)
  _hidden int libxl__sshm_add(libxl__gc *gc, uint32_t domid,
  libxl_static_shm *sshm, int len);
  
+_hidden int libxl__sshm_del(libxl__gc *gc, uint32_t domid);

+
  _hidden int libxl__sshm_check_overlap(libxl__gc *gc, uint32_t domid,
libxl_static_shm *sshms, int len);
  _hidden int libxl__sshm_setdefault(libxl__gc *gc, uint32_t domid,
diff --git a/tools/libxl/libxl_sshm.c b/tools/libxl/libxl_sshm.c
index f61b80c..9672056 100644
--- a/tools/libxl/libxl_sshm.c
+++ b/tools/libxl/libxl_sshm.c
@@ -94,6 +94,113 @@ int libxl__sshm_check_overlap(libxl__gc *gc, uint32_t domid,
  return 0;
  }
  
+/*

+ * Decrease the refcount of an sshm. When refcount reaches 0,
+ * clean up the whole sshm path.
+ * Xenstore operations are done within the same transaction.
+ */
+static void libxl__sshm_decref(libxl__gc *gc, xs_transaction_t xt,
+   const char *sshm_path)
+{
+int count;
+const char *count_path, *count_string;
+
+count_path = GCSPRINTF("%s/usercnt", sshm_path);
+if (libxl__xs_read_checked(gc, xt, count_path, _string))
+return;
+count = atoi(count_string);
+
+if (--count == 0) {
+libxl__xs_path_cleanup(gc, xt, sshm_path);
+return;
+}
+
+count_string = GCSPRINTF("%d", count);
+libxl__xs_write_checked(gc, xt, count_path, count_string);
+
+return;
+}
+
+static void libxl__sshm_do_unmap(libxl__gc *gc, uint32_t domid, const char *id,
+ uint64_t begin, uint64_t size)
+{
+uint64_t end;
+begin >>= XC_PAGE_SHIFT;
+size >>= XC_PAGE_SHIFT;
+end = begin + size;
+for (; begin < end; ++begin) {
+if (xc_domain_remove_from_physmap(CTX->xch, domid, begin)) {
+SSHM_ERROR(domid, id,
+   "unable to unmap shared page at 0x%"PRIx64".",
+   begin);
+}
+}
+}
+
+static void libxl__sshm_del_slave(libxl__gc *gc, xs_transaction_t xt,
+  uint32_t domid, const char *id, bool isretry)

isretry is not used, please remove

+{
+const char *slave_path, *begin_str, *size_str;
+uint64_t begin, size;
+
+slave_path = GCSPRINTF("%s/slaves/%"PRIu32, SSHM_PATH(id), domid);
+
+begin_str = libxl__xs_read(gc, xt, GCSPRINTF("%s/begin", slave_path));
+size_str = libxl__xs_read(gc, xt, GCSPRINTF("%s/size", slave_path));
+begin = strtoull(begin_str, NULL, 16);
+size = strtoull(size_str, NULL, 16);
+
+libxl__sshm_do_unmap(gc, domid, id, begin, size);
+libxl__xs_path_cleanup(gc, xt, 

Re: [Xen-devel] Xen PV: Sample new PV driver for buffer sharing between domains

[Oleksandr Andrushchenko]

On 09/27/2018 01:16 PM, Juergen Gross wrote:

On 27/09/2018 12:07, Oleksandr Andrushchenko wrote:

Hi,
On 09/27/2018 12:39 PM, Lars Kurth wrote:

Adding a few people who have recently been working on PV drivers, as
well as Julien
Lars


On 27 Sep 2018, at 06:44, Omkar Bolla
mailto:omkar.bo...@pathpartnertech.com>> wrote:

Hi,

I am using Debian as Domain-0 and Debian as Domain-U on Hikey960
platform(ARMv8) and using Xen-4.8 stable release. Here I want to
create a PV frontend and backend to share memory between Domain-0 and
Domain-U.



I used below link to create frontend and backend,
https://fnordig.de/2016/12/02/xen-a-backend-frontend-driver-example/

The link above has another link to github [1] with 2 chapters. And it
looks like you have
already modified the sources ("mydevice" -> "vdevb" at least).
So, what are the sources you are using?

You could probably take a look at the relatively small vkbd frontend
driver [2]
to get some hints.

But I am facing below errors while adding device vdevb to xenstore.
Below errors I am getting from xenbus_switch_state().
vdevb vdevb-0: failed to write error node for device/vdevb/0 (13
writing new state)

Error 13 is EACCESS. I guess the access rights of the Xenstore nodes
are not sufficient to write the needed entries.

Did you modify Xen tools (xl/libxl) for adding the new device type?
If not you need to setup the Xenstore nodes manually.

There is a script [1] which comes with the example implementation,
so I believe Omkar uses it with "mydevice" -> "vdevb" change



Juergen
[1] 
https://github.com/badboy/xen-split-driver-example/blob/master/chapter02/activate.sh


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Xen PV: Sample new PV driver for buffer sharing between domains

[Oleksandr Andrushchenko]

Hi,
On 09/27/2018 12:39 PM, Lars Kurth wrote:
Adding a few people who have recently been working on PV drivers, as 
well as Julien

Lars

On 27 Sep 2018, at 06:44, Omkar Bolla 
> wrote:


Hi,

I am using Debian as Domain-0 and Debian as Domain-U on Hikey960 
platform(ARMv8) and using Xen-4.8 stable release. Here I want to 
create a PV frontend and backend to share memory between Domain-0 and 
Domain-U.




I used below link to create frontend and backend,
https://fnordig.de/2016/12/02/xen-a-backend-frontend-driver-example/
The link above has another link to github [1] with 2 chapters. And it 
looks like you have

already modified the sources ("mydevice" -> "vdevb" at least).
So, what are the sources you are using?

You could probably take a look at the relatively small vkbd frontend 
driver [2]

to get some hints.


But I am facing below errors while adding device vdevb to xenstore.
Below errors I am getting from xenbus_switch_state().
vdevb vdevb-0: failed to write error node for device/vdevb/0 (13 
writing new state)


If the sources are known then we would need the full scenario which 
leads to the failure.
Could you please also add some debug logs into every function of the 
driver so we see what

and when happens on both backend and frontend sides?

Please suggest me, How to create PV drivers.

I would go with any existing driver in the kernel as an example

Thanks,
Omkar B

This message contains confidential information and is intended only 
for the individual(s) named.If you are not the intended recipient, 
you are notified that disclosing, copying, distributing or taking any 
action in reliance on the contents of this mail and attached file/s 
is strictly prohibited. Please notify the sender immediately and 
delete this e-mail from your system. E-mail transmission cannot be 
guaranteed to be secured or error-free as information could be 
intercepted, corrupted, lost, destroyed, arrive late or incomplete, 
or contain viruses. The sender therefore does not accept liability 
for any errors or omissions in the contents of this message, which 
arise as a result of e-mail transmission.


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org 
https://lists.xenproject.org/mailman/listinfo/xen-devel




___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[1] https://github.com/badboy/xen-split-driver-example
[2] 
https://elixir.bootlin.com/linux/latest/source/drivers/input/misc/xen-kbdfront.c
[3] 
https://github.com/badboy/xen-split-driver-example/blob/master/chapter02/activate.sh


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] ALSA: xen-front: Refine indentations and constify snd_pcm_ops

[Oleksandr Andrushchenko]

On 09/20/2018 12:47 AM, Nick Simonov wrote:

snd_pcm_ops are not supposed to change. So mark the
non-const structs as const. Also, refine indentation
to ncrease readability.

Signed-off-by: Nick Simonov 
---
  sound/xen/xen_snd_front_alsa.c | 46 +-
  1 file changed, 23 insertions(+), 23 deletions(-)

diff --git a/sound/xen/xen_snd_front_alsa.c b/sound/xen/xen_snd_front_alsa.c
index 129180e..2cbd967 100644
--- a/sound/xen/xen_snd_front_alsa.c
+++ b/sound/xen/xen_snd_front_alsa.c
@@ -637,31 +637,31 @@ static int alsa_pb_fill_silence(struct snd_pcm_substream 
*substream,
   * to know when the buffer can be transferred to the backend.
   */
  
-static struct snd_pcm_ops snd_drv_alsa_playback_ops = {

-   .open = alsa_open,
-   .close = alsa_close,
-   .ioctl = snd_pcm_lib_ioctl,
-   .hw_params = alsa_hw_params,
-   .hw_free = alsa_hw_free,
-   .prepare = alsa_prepare,
-   .trigger = alsa_trigger,
-   .pointer = alsa_pointer,
-   .copy_user = alsa_pb_copy_user,
-   .copy_kernel = alsa_pb_copy_kernel,
-   .fill_silence = alsa_pb_fill_silence,
+static const struct snd_pcm_ops snd_drv_alsa_playback_ops = {
+   .open   = alsa_open,
+   .close  = alsa_close,
+   .ioctl  = snd_pcm_lib_ioctl,
+   .hw_params  = alsa_hw_params,
+   .hw_free= alsa_hw_free,
+   .prepare= alsa_prepare,
+   .trigger= alsa_trigger,
+   .pointer= alsa_pointer,
+   .copy_user  = alsa_pb_copy_user,
+   .copy_kernel= alsa_pb_copy_kernel,
+   .fill_silence   = alsa_pb_fill_silence,
  };
  
-static struct snd_pcm_ops snd_drv_alsa_capture_ops = {

-   .open = alsa_open,
-   .close = alsa_close,
-   .ioctl = snd_pcm_lib_ioctl,
-   .hw_params = alsa_hw_params,
-   .hw_free = alsa_hw_free,
-   .prepare = alsa_prepare,
-   .trigger = alsa_trigger,
-   .pointer = alsa_pointer,
-   .copy_user = alsa_cap_copy_user,
-   .copy_kernel = alsa_cap_copy_kernel,
+static const struct snd_pcm_ops snd_drv_alsa_capture_ops = {
+   .open   = alsa_open,
+   .close  = alsa_close,
+   .ioctl  = snd_pcm_lib_ioctl,
+   .hw_params  = alsa_hw_params,
+   .hw_free= alsa_hw_free,
+   .prepare= alsa_prepare,
+   .trigger= alsa_trigger,
+   .pointer= alsa_pointer,
+   .copy_user  = alsa_cap_copy_user,
+   .copy_kernel= alsa_cap_copy_kernel,
  };
  
  static int new_pcm_instance(struct xen_snd_front_card_info *card_info,

Thank you for the patch,
Reviewed-by: Oleksandr Andrushchenko 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

On 09/12/2018 05:38 PM, Hans Verkuil wrote:

On 09/12/18 16:15, Oleksandr Andrushchenko wrote:

On 09/12/2018 04:38 PM, Hans Verkuil wrote:

On 09/12/18 15:02, Oleksandr Andrushchenko wrote:

On 09/12/2018 03:25 PM, Hans Verkuil wrote:

+ * formats
+ *  Values: 
+ *
+ *  Formats are organized as a set of directories one per each
+ *  supported pixel format. The name of the directory is the
+ *  corresponding FOURCC string label. The next level of
+ *  the directory under  represents supported resolutions.

So how will this work for a pixelformat like V4L2_PIX_FMT_ARGB555X?

As mentioned before, we display such formats as '-BE', i.e. char[7].

ok, then I'll change this to char[7] and put a note on big-endian:

 *  If format represents a big-endian FOURCC code, then "-BE"
 *  suffix must be added, case insensitive.

Since the fourcc is case-sensitive, I'd keep -BE case sensitive as well.
Your decision, though.

hm, I'm a little bit confused here... One of the previous comments was...

   >> + *  Formats are organized as a set of directories one per each
   >> + *  supported pixel format. The name of the directory is an
upper case
   >> + *  string of the corresponding FOURCC string label. The next
level of
   >> + *  the directory under  represents supported resolutions.

   >Lower-case characters are also use in pixelformats, so I'd just keep
this as-is.
   >
   >In addition it is common to set bit 31 of the fourcc to 1 if the format is
   >big-endian (see v4l2_fourcc_be macro). When v4l utilities print this
format we
   >add a -BE suffix, so V4L2_PIX_FMT_ARGB555X becomes "AR15-BE". You
might want to
   >keep that convention.

So, finally, I'll put upper case constraint here for fourcc and "-BE"?
Did I miss something here?

Easiest is to look at videodev2.h. Let me take two examples:

#define V4L2_PIX_FMT_ARGB555 v4l2_fourcc('A', 'R', '1', '5') /* 16  
ARGB-1-5-5-5  */
#define V4L2_PIX_FMT_ARGB555X v4l2_fourcc_be('A', 'R', '1', '5') /* 16  
ARGB-5-5-5 BE */

The fourcc macros are defined as:

#define v4l2_fourcc(a, b, c, d)\
  ((__u32)(a) | ((__u32)(b) << 8) | ((__u32)(c) << 16) | ((__u32)(d) << 
24))
#define v4l2_fourcc_be(a, b, c, d)  (v4l2_fourcc(a, b, c, d) | (1 << 31))

The characters can be any printable character, but currently we only use
a-z, A-Z, 0-9 and space (' ').

For big-endian formats we also set bit 31 (i.e. bit 7 of the last character)
to indicate this.

In our v4l2 utilities we use this function when we want to print a fourcc:

std::string fcc2s(__u32 val)
{
  std::string s;

  s += val & 0x7f;
  s += (val >> 8) & 0x7f;
  s += (val >> 16) & 0x7f;
  s += (val >> 24) & 0x7f;
  if (val & (1 << 31))
  s += "-BE";
  return s;
}

So the four characters (with bit 7 masked out) and the -BE suffix
if bit 7 was set for the fourth character.

So for your protocol, if you want to specify the fourcc, then I
assume dealing with characters with bit 7 set is a pain as well,
and in that case you are better off using the same scheme that we
do.

And to match the formats, applications should remember that the
string is case-sensitive, so 'abcd' != 'Abcd'.

Ah, got it, so I'll just make sure that "-BE" part
is case sensitive, e.g. remove "case insensitive":

   * formats
   *  Values: 
   *
   *  Formats are organized as a set of directories one per each
   *  supported pixel format. The name of the directory is the
   *  corresponding FOURCC string label. The next level of
   *  the directory under  represents supported resolutions.
   *  If format represents a big-endian FOURCC code, then "-BE"
   *  suffix must be added.

Hmm. How about: "If the format represents a big-endian variant of a little
endian format, then the "-BE" suffix must be added. E.g. 'AR15' vs 'AR15-BE'."

Sounds good, will use this, thank you

BTW, using bit 31 for this is a complete V4L2-specific invention.


Note that there can be spaces:

#define V4L2_PIX_FMT_Y16 v4l2_fourcc('Y', '1', '6', ' ') /* 16  Greyscale   
  */
#define V4L2_PIX_FMT_Y16_BE  v4l2_fourcc_be('Y', '1', '6', ' ') /* 16  
Greyscale BE  */

So that would be:

/local/domain/1/device/vcamera/0/formats/Y16 /1200x720/frame-rates = "15/2"

and:

/local/domain/1/device/vcamera/0/formats/Y16 -BE/1200x720/frame-rates = "15/2"

Not sure if that will fly for you.

Currently if there are spaced, then they are at the end, but I don't think
we can guarantee that for all time.

This is a problem as I cannot have spaces...
So, I can only put something like:

   *  If FOURCC string label has spaces then those are only allowed to
   *  be at the end of the label and must 

Re: [Xen-devel] [PATCH v2 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

On 09/12/2018 04:38 PM, Hans Verkuil wrote:

On 09/12/18 15:02, Oleksandr Andrushchenko wrote:

On 09/12/2018 03:25 PM, Hans Verkuil wrote:

+ * formats
+ *  Values: 
+ *
+ *  Formats are organized as a set of directories one per each
+ *  supported pixel format. The name of the directory is the
+ *  corresponding FOURCC string label. The next level of
+ *  the directory under  represents supported resolutions.

So how will this work for a pixelformat like V4L2_PIX_FMT_ARGB555X?

As mentioned before, we display such formats as '-BE', i.e. char[7].

ok, then I'll change this to char[7] and put a note on big-endian:

*  If format represents a big-endian FOURCC code, then "-BE"
*  suffix must be added, case insensitive.

Since the fourcc is case-sensitive, I'd keep -BE case sensitive as well.
Your decision, though.

hm, I'm a little bit confused here... One of the previous comments was...

  >> + *  Formats are organized as a set of directories one per each
  >> + *  supported pixel format. The name of the directory is an
upper case
  >> + *  string of the corresponding FOURCC string label. The next
level of
  >> + *  the directory under  represents supported resolutions.

  >Lower-case characters are also use in pixelformats, so I'd just keep
this as-is.
  >
  >In addition it is common to set bit 31 of the fourcc to 1 if the format is
  >big-endian (see v4l2_fourcc_be macro). When v4l utilities print this
format we
  >add a -BE suffix, so V4L2_PIX_FMT_ARGB555X becomes "AR15-BE". You
might want to
  >keep that convention.

So, finally, I'll put upper case constraint here for fourcc and "-BE"?
Did I miss something here?

Easiest is to look at videodev2.h. Let me take two examples:

#define V4L2_PIX_FMT_ARGB555 v4l2_fourcc('A', 'R', '1', '5') /* 16  
ARGB-1-5-5-5  */
#define V4L2_PIX_FMT_ARGB555X v4l2_fourcc_be('A', 'R', '1', '5') /* 16  
ARGB-5-5-5 BE */

The fourcc macros are defined as:

#define v4l2_fourcc(a, b, c, d)\
 ((__u32)(a) | ((__u32)(b) << 8) | ((__u32)(c) << 16) | ((__u32)(d) << 
24))
#define v4l2_fourcc_be(a, b, c, d)  (v4l2_fourcc(a, b, c, d) | (1 << 31))

The characters can be any printable character, but currently we only use
a-z, A-Z, 0-9 and space (' ').

For big-endian formats we also set bit 31 (i.e. bit 7 of the last character)
to indicate this.

In our v4l2 utilities we use this function when we want to print a fourcc:

std::string fcc2s(__u32 val)
{
 std::string s;

 s += val & 0x7f;
 s += (val >> 8) & 0x7f;
 s += (val >> 16) & 0x7f;
 s += (val >> 24) & 0x7f;
 if (val & (1 << 31))
 s += "-BE";
 return s;
}

So the four characters (with bit 7 masked out) and the -BE suffix
if bit 7 was set for the fourth character.

So for your protocol, if you want to specify the fourcc, then I
assume dealing with characters with bit 7 set is a pain as well,
and in that case you are better off using the same scheme that we
do.

And to match the formats, applications should remember that the
string is case-sensitive, so 'abcd' != 'Abcd'.

Ah, got it, so I'll just make sure that "-BE" part
is case sensitive, e.g. remove "case insensitive":

 * formats
 *  Values: 
 *
 *  Formats are organized as a set of directories one per each
 *  supported pixel format. The name of the directory is the
 *  corresponding FOURCC string label. The next level of
 *  the directory under  represents supported resolutions.
 *  If format represents a big-endian FOURCC code, then "-BE"
 *  suffix must be added.



Note that there can be spaces:

#define V4L2_PIX_FMT_Y16 v4l2_fourcc('Y', '1', '6', ' ') /* 16  Greyscale   
  */
#define V4L2_PIX_FMT_Y16_BE  v4l2_fourcc_be('Y', '1', '6', ' ') /* 16  
Greyscale BE  */

So that would be:

/local/domain/1/device/vcamera/0/formats/Y16 /1200x720/frame-rates = "15/2"

and:

/local/domain/1/device/vcamera/0/formats/Y16 -BE/1200x720/frame-rates = "15/2"

Not sure if that will fly for you.

Currently if there are spaced, then they are at the end, but I don't think
we can guarantee that for all time.

This is a problem as I cannot have spaces...
So, I can only put something like:

 *  If FOURCC string label has spaces then those are only allowed to
 *  be at the end of the label and must be trimmed.




I assume the pixelformats you use here are based on the V4L2_PIX_FMT_ fourccs?

Note that there is no real standard for fourcc values, so if you want to
support a Windows backend as well, then you'll need mappings from whatever
Windows uses to the V4L2 fourccs.

The V4L2_PIX_FMT_ fourccs are entirely V4L2 specific.

So you have to define here whose fourccs you are using.

I thought that [1] defines a

Re: [Xen-devel] [PATCH v2 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

On 09/12/2018 03:25 PM, Hans Verkuil wrote:

On 09/12/18 12:09, Oleksandr Andrushchenko wrote:

Hi, Hans!

Thank you for valuable comments and valid concerns!

On 09/12/2018 10:52 AM, Hans Verkuil wrote:

On 09/11/2018 10:29 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
   - pixel formats
   - resolutions
   - frame rates
2. Support basic camera controls:
   - contrast
   - brightness
   - hue
   - saturation
3. Support streaming control
4. Support zero-copying use-cases

Signed-off-by: Oleksandr Andrushchenko 
---
   xen/include/public/io/cameraif.h | 1263 ++
   1 file changed, 1263 insertions(+)
   create mode 100644 xen/include/public/io/cameraif.h

diff --git a/xen/include/public/io/cameraif.h b/xen/include/public/io/cameraif.h
new file mode 100644
index ..38b9b3741e75
--- /dev/null
+++ b/xen/include/public/io/cameraif.h
@@ -0,0 +1,1263 @@
+/**
+ * cameraif.h
+ *
+ * Unified camera device I/O interface for Xen guest OSes.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright (C) 2018 EPAM Systems Inc.
+ *
+ * Author: Oleksandr Andrushchenko 
+ */
+
+#ifndef __XEN_PUBLIC_IO_CAMERAIF_H__
+#define __XEN_PUBLIC_IO_CAMERAIF_H__
+
+#include "ring.h"
+#include "../grant_table.h"
+
+/*
+ **
+ *   Protocol version
+ **
+ */
+#define XENCAMERA_PROTOCOL_VERSION "1"
+
+/*
+ **
+ *  Feature and Parameter Negotiation
+ **
+ *
+ * Front->back notifications: when enqueuing a new request, sending a
+ * notification can be made conditional on xencamera_req (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Backends must set
+ * xencamera_req appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
+ *
+ * Back->front notifications: when enqueuing a new response, sending a
+ * notification can be made conditional on xencamera_resp (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Frontends must set
+ * xencamera_resp appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
+ *
+ * The two halves of a para-virtual camera driver utilize nodes within
+ * XenStore to communicate capabilities and to negotiate operating parameters.
+ * This section enumerates these nodes which reside in the respective front and
+ * backend portions of XenStore, following the XenBus convention.
+ *
+ * All data in XenStore is stored as strings. Nodes specifying numeric
+ * values are encoded in decimal. Integer value ranges listed below are
+ * expressed as fixed sized integer types capable of storing the conversion
+ * of a properly formatted node string, without loss of information.
+ *
+ **
+ *Example configuration
+ **
+ *
+ * This is an example of backend and frontend configuration:
+ *
+ *- Backend ---
+ *
+ * /local/domain/0/backend/vcamera/1/0/frontend-id = "1"
+ 

Re: [Xen-devel] [PATCH v2 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

On 09/12/2018 12:16 PM, Hans Verkuil wrote:

On 09/11/18 10:29, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
  - pixel formats
  - resolutions
  - frame rates
2. Support basic camera controls:
  - contrast
  - brightness
  - hue
  - saturation
3. Support streaming control
4. Support zero-copying use-cases

Signed-off-by: Oleksandr Andrushchenko 
---
  xen/include/public/io/cameraif.h | 1263 ++
  1 file changed, 1263 insertions(+)
  create mode 100644 xen/include/public/io/cameraif.h

diff --git a/xen/include/public/io/cameraif.h b/xen/include/public/io/cameraif.h
new file mode 100644
index ..38b9b3741e75
--- /dev/null
+++ b/xen/include/public/io/cameraif.h
@@ -0,0 +1,1263 @@
+/**
+ * cameraif.h
+ *
+ * Unified camera device I/O interface for Xen guest OSes.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright (C) 2018 EPAM Systems Inc.
+ *
+ * Author: Oleksandr Andrushchenko 
+ */
+
+#ifndef __XEN_PUBLIC_IO_CAMERAIF_H__
+#define __XEN_PUBLIC_IO_CAMERAIF_H__
+
+#include "ring.h"
+#include "../grant_table.h"
+
+/*
+ **
+ *   Protocol version
+ **
+ */
+#define XENCAMERA_PROTOCOL_VERSION "1"
+
+/*
+ **
+ *  Feature and Parameter Negotiation
+ **
+ *
+ * Front->back notifications: when enqueuing a new request, sending a
+ * notification can be made conditional on xencamera_req (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Backends must set
+ * xencamera_req appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
+ *
+ * Back->front notifications: when enqueuing a new response, sending a
+ * notification can be made conditional on xencamera_resp (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Frontends must set
+ * xencamera_resp appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
+ *
+ * The two halves of a para-virtual camera driver utilize nodes within
+ * XenStore to communicate capabilities and to negotiate operating parameters.
+ * This section enumerates these nodes which reside in the respective front and
+ * backend portions of XenStore, following the XenBus convention.
+ *
+ * All data in XenStore is stored as strings. Nodes specifying numeric
+ * values are encoded in decimal. Integer value ranges listed below are
+ * expressed as fixed sized integer types capable of storing the conversion
+ * of a properly formatted node string, without loss of information.
+ *
+ **
+ *Example configuration
+ **
+ *
+ * This is an example of backend and frontend configuration:
+ *
+ *- Backend ---
+ *
+ * /local/domain/0/backend/vcamera/1/0/frontend-id = "1"
+ * /local/domain/0/backend/vcamera/1/0/frontend = 
"/local/domain/1/device/vcamera/0"
+ * /local/domain/0/backend/vcamera/1/0/state = "4"
+ * /local/domain/0/

Re: [Xen-devel] [PATCH v2 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

Hi, Hans!

Thank you for valuable comments and valid concerns!

On 09/12/2018 10:52 AM, Hans Verkuil wrote:

On 09/11/2018 10:29 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
  - pixel formats
  - resolutions
  - frame rates
2. Support basic camera controls:
  - contrast
  - brightness
  - hue
  - saturation
3. Support streaming control
4. Support zero-copying use-cases

Signed-off-by: Oleksandr Andrushchenko 
---
  xen/include/public/io/cameraif.h | 1263 ++
  1 file changed, 1263 insertions(+)
  create mode 100644 xen/include/public/io/cameraif.h

diff --git a/xen/include/public/io/cameraif.h b/xen/include/public/io/cameraif.h
new file mode 100644
index ..38b9b3741e75
--- /dev/null
+++ b/xen/include/public/io/cameraif.h
@@ -0,0 +1,1263 @@
+/**
+ * cameraif.h
+ *
+ * Unified camera device I/O interface for Xen guest OSes.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright (C) 2018 EPAM Systems Inc.
+ *
+ * Author: Oleksandr Andrushchenko 
+ */
+
+#ifndef __XEN_PUBLIC_IO_CAMERAIF_H__
+#define __XEN_PUBLIC_IO_CAMERAIF_H__
+
+#include "ring.h"
+#include "../grant_table.h"
+
+/*
+ **
+ *   Protocol version
+ **
+ */
+#define XENCAMERA_PROTOCOL_VERSION "1"
+
+/*
+ **
+ *  Feature and Parameter Negotiation
+ **
+ *
+ * Front->back notifications: when enqueuing a new request, sending a
+ * notification can be made conditional on xencamera_req (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Backends must set
+ * xencamera_req appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
+ *
+ * Back->front notifications: when enqueuing a new response, sending a
+ * notification can be made conditional on xencamera_resp (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Frontends must set
+ * xencamera_resp appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
+ *
+ * The two halves of a para-virtual camera driver utilize nodes within
+ * XenStore to communicate capabilities and to negotiate operating parameters.
+ * This section enumerates these nodes which reside in the respective front and
+ * backend portions of XenStore, following the XenBus convention.
+ *
+ * All data in XenStore is stored as strings. Nodes specifying numeric
+ * values are encoded in decimal. Integer value ranges listed below are
+ * expressed as fixed sized integer types capable of storing the conversion
+ * of a properly formatted node string, without loss of information.
+ *
+ **
+ *Example configuration
+ **
+ *
+ * This is an example of backend and frontend configuration:
+ *
+ *- Backend ---
+ *
+ * /local/domain/0/backend/vcamera/1/0/frontend-id = "1"
+ * /local/domain/0/backend/vcamera/1/0/frontend = 
"/local/domain/1/device/vcamera/0"
+ * /local/doma

[Xen-devel] [PATCH v2 0/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Hello!

At the moment Xen [1] already supports some virtual multimedia
features [2] such as virtual display, sound. It supports keyboards,
pointers and multi-touch devices all allowing Xen to be used in
automotive appliances, In-Vehicle Infotainment (IVI) systems
and many more.

This work adds a new Xen para-virtualized protocol for a virtual
camera device which extends multimedia capabilities of Xen even
farther: video conferencing, IVI, high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
 - pixel formats
 - resolutions
 - frame rates
2. Support basic camera controls:
 - contrast
 - brightness
 - hue
 - saturation
3. Support streaming control
4. Support zero-copying use-cases

I hope that Xen and V4L and other communities could give their
valuable feedback on this work, so I can update the protocol
to better fit any additional requirements I might have missed.

I would like to thank Hans Verkuil  for valuable
comments and help.

Thank you,
Oleksandr Andrushchenko

Changes since v1:
=

1. Added XenStore entries:
 - frame-rates
2. Do not require the FOURCC code in XenStore to be upper case only
3. Added/changed command set:
 - configuration get/set
 - buffer queue/dequeue
 - control get
4. Added control flags, e.g. read-only etc.
5. Added colorspace configuration support, relevant constants
6. Added events:
 - configuration change
 - control change
7. Changed control values to 64-bit
8. Added sequence number to frame avail event
9. Coding style cleanup

[1] https://www.xenproject.org/
[2] https://xenbits.xen.org/gitweb/?p=xen.git;a=tree;f=xen/include/public/io

Oleksandr Andrushchenko (1):
  cameraif: add ABI for para-virtual camera

 xen/include/public/io/cameraif.h | 1263 ++
 1 file changed, 1263 insertions(+)
 create mode 100644 xen/include/public/io/cameraif.h

-- 
2.18.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

On 09/11/2018 10:52 AM, Hans Verkuil wrote:

On 09/11/18 09:14, Oleksandr Andrushchenko wrote:

On 09/11/2018 10:04 AM, Hans Verkuil wrote:

On 09/11/2018 08:52 AM, Oleksandr Andrushchenko wrote:

Hi, Hans!

On 09/10/2018 03:26 PM, Hans Verkuil wrote:

On 09/10/2018 01:49 PM, Oleksandr Andrushchenko wrote:

On 09/10/2018 02:09 PM, Hans Verkuil wrote:

On 09/10/2018 11:52 AM, Oleksandr Andrushchenko wrote:

On 09/10/2018 12:04 PM, Hans Verkuil wrote:

On 09/10/2018 10:24 AM, Oleksandr Andrushchenko wrote:

On 09/10/2018 10:53 AM, Hans Verkuil wrote:

Hi Oleksandr,

On 09/10/2018 09:16 AM, Oleksandr Andrushchenko wrote:




I suspect that you likely will want to support such sources eventually, so
it pays to design this with that in mind.

Again, I think that this is the backend to hide these
use-cases from the frontend.

I'm not sure you can: say you are playing a bluray connected to the system
with HDMI, then if there is a resolution change, what do you do? You can tear
everything down and build it up again, or you can just tell frontends that
something changed and that they have to look at the new vcamera configuration.

The latter seems to be more sensible to me. It is really not much that you
need to do: all you really need is an event signalling that something changed.
In V4L2 that's the V4L2_EVENT_SOURCE_CHANGE.

well, this complicates things a lot as I'll have to
re-allocate buffers - right?

Right. Different resolutions means different sized buffers and usually lots of
changes throughout the whole video pipeline, which in this case can even
go into multiple VMs.

One additional thing to keep in mind for the future: V4L2_EVENT_SOURCE_CHANGE
has a flags field that tells userspace what changed. Right now that is just the
resolution, but in the future you can expect flags for cases where just the
colorspace information changes, but not the resolution.

Which reminds me of two important missing pieces of information in your 
protocol:

1) You need to communicate the colorspace data:

- colorspace
- xfer_func
- ycbcr_enc/hsv_enc (unlikely you ever want to support HSV pixelformats, so I
   think you can ignore hsv_enc)
- quantization

See 
https://hverkuil.home.xs4all.nl/spec/uapi/v4l/pixfmt-v4l2.html#c.v4l2_pix_format
and the links to the colorspace sections in the V4L2 spec for details).

This information is part of the format, it is reported by the driver.

I'll take a look and think what can be put and how into the protocol,
do you think I'll have to implement all the above for
this stage?

Yes. Without it VMs will have no way of knowing how to reproduce the right 
colors.
They don't *have* to use this information, but it should be there. For cameras
this isn't all that important, for SDTV/HDTV sources this becomes more relevant
(esp. the quantization and ycbcr_enc information) and for sources with 
BT.2020/HDR
formats this is critical.

ok, then I'll add the following to the set_config request/response:

uint32_t colorspace;
uint32_t xfer_func;
uint32_t ycbcr_enc;
uint32_t quantization;

Yet another question here: are the above (color space, xfer etc.) and
display aspect ratio defined per pixel_format or per pixel_format +
resolution?

If per pixel_format then

.../vcamera/1/formats/YUYV/display-aspect-ratio = "59/58"

or if per resolution

.../vcamera/1/formats/YUYV/640x480/display-aspect-ratio = "59/58"

They are totally independent of resolution or pixelformat, with the
exception of ycbcr_enc which is of course ignored for RGB pixelformats.

They are set by the driver, never by the application.

For HDMI sources these values can change depending on what source is
connected, so they are not fixed and you need to query them whenever
a new source is connected. In fact, then can change midstream, but we
do not have good support for that at the moment.

Ah, great, then I'll define colorspace, xfer_func, quantization
and display aspect ratio as part of virtual camera device configuration
(as vcamera represents a single source) and ycbcr_enc as a part
of pixel format configuration (one ycbcr_enc per each
pixel format)

Does this sound ok?

Uh, no :-)

ycbcr_enc is not tied to specific pixel formats. The Y'CbCr encoding tells
you how the Y'CbCr values were derived from the R'G'B' values. So this only
makes sense if you are in fact receiving Y'CbCr pixels, otherwise you just
ignore it.

It's up to you what value to assign to ycbcr_enc in that case: V4L2 doesn't
have any hard requirements for that AFAIK, although it will most likely be
set to 0 (V4L2_YCBCR_ENC_DEFAULT).

Thank you for the explanation

Regards,

Hans

Thank you,
Oleksandr

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

On 09/11/2018 10:04 AM, Hans Verkuil wrote:

On 09/11/2018 08:52 AM, Oleksandr Andrushchenko wrote:

Hi, Hans!

On 09/10/2018 03:26 PM, Hans Verkuil wrote:

On 09/10/2018 01:49 PM, Oleksandr Andrushchenko wrote:

On 09/10/2018 02:09 PM, Hans Verkuil wrote:

On 09/10/2018 11:52 AM, Oleksandr Andrushchenko wrote:

On 09/10/2018 12:04 PM, Hans Verkuil wrote:

On 09/10/2018 10:24 AM, Oleksandr Andrushchenko wrote:

On 09/10/2018 10:53 AM, Hans Verkuil wrote:

Hi Oleksandr,

On 09/10/2018 09:16 AM, Oleksandr Andrushchenko wrote:




I suspect that you likely will want to support such sources eventually, so
it pays to design this with that in mind.

Again, I think that this is the backend to hide these
use-cases from the frontend.

I'm not sure you can: say you are playing a bluray connected to the system
with HDMI, then if there is a resolution change, what do you do? You can tear
everything down and build it up again, or you can just tell frontends that
something changed and that they have to look at the new vcamera configuration.

The latter seems to be more sensible to me. It is really not much that you
need to do: all you really need is an event signalling that something changed.
In V4L2 that's the V4L2_EVENT_SOURCE_CHANGE.

well, this complicates things a lot as I'll have to
re-allocate buffers - right?

Right. Different resolutions means different sized buffers and usually lots of
changes throughout the whole video pipeline, which in this case can even
go into multiple VMs.

One additional thing to keep in mind for the future: V4L2_EVENT_SOURCE_CHANGE
has a flags field that tells userspace what changed. Right now that is just the
resolution, but in the future you can expect flags for cases where just the
colorspace information changes, but not the resolution.

Which reminds me of two important missing pieces of information in your 
protocol:

1) You need to communicate the colorspace data:

- colorspace
- xfer_func
- ycbcr_enc/hsv_enc (unlikely you ever want to support HSV pixelformats, so I
  think you can ignore hsv_enc)
- quantization

See 
https://hverkuil.home.xs4all.nl/spec/uapi/v4l/pixfmt-v4l2.html#c.v4l2_pix_format
and the links to the colorspace sections in the V4L2 spec for details).

This information is part of the format, it is reported by the driver.

I'll take a look and think what can be put and how into the protocol,
do you think I'll have to implement all the above for
this stage?

Yes. Without it VMs will have no way of knowing how to reproduce the right 
colors.
They don't *have* to use this information, but it should be there. For cameras
this isn't all that important, for SDTV/HDTV sources this becomes more relevant
(esp. the quantization and ycbcr_enc information) and for sources with 
BT.2020/HDR
formats this is critical.

ok, then I'll add the following to the set_config request/response:

   uint32_t colorspace;
   uint32_t xfer_func;
   uint32_t ycbcr_enc;
   uint32_t quantization;

Yet another question here: are the above (color space, xfer etc.) and
display aspect ratio defined per pixel_format or per pixel_format +
resolution?

If per pixel_format then

.../vcamera/1/formats/YUYV/display-aspect-ratio = "59/58"

or if per resolution

.../vcamera/1/formats/YUYV/640x480/display-aspect-ratio = "59/58"

They are totally independent of resolution or pixelformat, with the
exception of ycbcr_enc which is of course ignored for RGB pixelformats.

They are set by the driver, never by the application.

For HDMI sources these values can change depending on what source is
connected, so they are not fixed and you need to query them whenever
a new source is connected. In fact, then can change midstream, but we
do not have good support for that at the moment.

Ah, great, then I'll define colorspace, xfer_func, quantization
and display aspect ratio as part of virtual camera device configuration
(as vcamera represents a single source) and ycbcr_enc as a part
of pixel format configuration (one ycbcr_enc per each
pixel format)

Does this sound ok?


Regards,

Hans

Thank you,
Oleksandr

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

Hi, Hans!

On 09/10/2018 03:26 PM, Hans Verkuil wrote:

On 09/10/2018 01:49 PM, Oleksandr Andrushchenko wrote:

On 09/10/2018 02:09 PM, Hans Verkuil wrote:

On 09/10/2018 11:52 AM, Oleksandr Andrushchenko wrote:

On 09/10/2018 12:04 PM, Hans Verkuil wrote:

On 09/10/2018 10:24 AM, Oleksandr Andrushchenko wrote:

On 09/10/2018 10:53 AM, Hans Verkuil wrote:

Hi Oleksandr,

On 09/10/2018 09:16 AM, Oleksandr Andrushchenko wrote:




I suspect that you likely will want to support such sources eventually, so
it pays to design this with that in mind.

Again, I think that this is the backend to hide these
use-cases from the frontend.

I'm not sure you can: say you are playing a bluray connected to the system
with HDMI, then if there is a resolution change, what do you do? You can tear
everything down and build it up again, or you can just tell frontends that
something changed and that they have to look at the new vcamera configuration.

The latter seems to be more sensible to me. It is really not much that you
need to do: all you really need is an event signalling that something changed.
In V4L2 that's the V4L2_EVENT_SOURCE_CHANGE.

well, this complicates things a lot as I'll have to
re-allocate buffers - right?

Right. Different resolutions means different sized buffers and usually lots of
changes throughout the whole video pipeline, which in this case can even
go into multiple VMs.

One additional thing to keep in mind for the future: V4L2_EVENT_SOURCE_CHANGE
has a flags field that tells userspace what changed. Right now that is just the
resolution, but in the future you can expect flags for cases where just the
colorspace information changes, but not the resolution.

Which reminds me of two important missing pieces of information in your 
protocol:

1) You need to communicate the colorspace data:

- colorspace
- xfer_func
- ycbcr_enc/hsv_enc (unlikely you ever want to support HSV pixelformats, so I
 think you can ignore hsv_enc)
- quantization

See 
https://hverkuil.home.xs4all.nl/spec/uapi/v4l/pixfmt-v4l2.html#c.v4l2_pix_format
and the links to the colorspace sections in the V4L2 spec for details).

This information is part of the format, it is reported by the driver.

I'll take a look and think what can be put and how into the protocol,
do you think I'll have to implement all the above for
this stage?

Yes. Without it VMs will have no way of knowing how to reproduce the right 
colors.
They don't *have* to use this information, but it should be there. For cameras
this isn't all that important, for SDTV/HDTV sources this becomes more relevant
(esp. the quantization and ycbcr_enc information) and for sources with 
BT.2020/HDR
formats this is critical.

ok, then I'll add the following to the set_config request/response:

  uint32_t colorspace;
  uint32_t xfer_func;
  uint32_t ycbcr_enc;
  uint32_t quantization;

Yet another question here: are the above (color space, xfer etc.) and
display aspect ratio defined per pixel_format or per pixel_format + 
resolution?


If per pixel_format then

.../vcamera/1/formats/YUYV/display-aspect-ratio = "59/58"

or if per resolution

.../vcamera/1/formats/YUYV/640x480/display-aspect-ratio = "59/58"



With this respect, I will need to put some OS agnostic constants
into the protocol, so if backend and frontend are not Linux/V4L2
based they can still talk to each other.
I see that V4L2 already defines constants for the above: [1], [2], [3], [4].

Do you think I can define the same replacing V4L2_ prefix
with XENCAMERA_, e.g. V4L2_XFER_FUNC_SRGB -> XENCAMERA_XFER_FUNC_SRGB?

Yes.


Do I need to define all those or there can be some subset of the
above for my simpler use-case?

Most of these defines directly map to standards. I would skip the following
defines:

V4L2_COLORSPACE_DEFAULT (not applicable)
V4L2_COLORSPACE_470_SYSTEM_*  (rarely used, if received by the HW the Xen 
backend
should map this to V4L2_COLORSPACE_SMPTE170M)
V4L2_COLORSPACE_JPEG (historical V4L2 artifact, see here how to map:
 
https://hverkuil.home.xs4all.nl/spec/uapi/v4l/colorspaces-details.html#col-jpeg)

V4L2_COLORSPACE_SMPTE240M (rarely used, map to V4L2_COLORSPACE_SMPTE170M if 
seen in backend)

V4L2_XFER_FUNC_SMPTE240M (rarely used, map to V4L2_XFER_FUNC_709)

V4L2_YCBCR_ENC_SMPTE240M (rarely used, map to V4L2_YCBCR_ENC_709)

While V4L2 allows 0 (DEFAULT) values for xfer_func, ycbcr_enc and quantization, 
and
provides macros to map default values to the actual values (for legacy reasons),
the Xen backend should always fill this in explicitly, using those same mapping
macros (see e.g. V4L2_MAP_XFER_FUNC_DEFAULT).

The V4L2 spec has extensive information on colorspaces (sections 2.14-2.17).


The vivid driver can actually reproduce all combinations, so that's a good 
driver
to test this with.

You mean I can use it on backend side instead of real HW camera and
test all the configurations possible/those of interest?

Right.

Regard

Re: [Xen-devel] [PATCH 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

On 09/10/2018 03:26 PM, Hans Verkuil wrote:

On 09/10/2018 01:49 PM, Oleksandr Andrushchenko wrote:

On 09/10/2018 02:09 PM, Hans Verkuil wrote:

On 09/10/2018 11:52 AM, Oleksandr Andrushchenko wrote:

On 09/10/2018 12:04 PM, Hans Verkuil wrote:

On 09/10/2018 10:24 AM, Oleksandr Andrushchenko wrote:

On 09/10/2018 10:53 AM, Hans Verkuil wrote:

Hi Oleksandr,

On 09/10/2018 09:16 AM, Oleksandr Andrushchenko wrote:




I suspect that you likely will want to support such sources eventually, so
it pays to design this with that in mind.

Again, I think that this is the backend to hide these
use-cases from the frontend.

I'm not sure you can: say you are playing a bluray connected to the system
with HDMI, then if there is a resolution change, what do you do? You can tear
everything down and build it up again, or you can just tell frontends that
something changed and that they have to look at the new vcamera configuration.

The latter seems to be more sensible to me. It is really not much that you
need to do: all you really need is an event signalling that something changed.
In V4L2 that's the V4L2_EVENT_SOURCE_CHANGE.

well, this complicates things a lot as I'll have to
re-allocate buffers - right?

Right. Different resolutions means different sized buffers and usually lots of
changes throughout the whole video pipeline, which in this case can even
go into multiple VMs.

One additional thing to keep in mind for the future: V4L2_EVENT_SOURCE_CHANGE
has a flags field that tells userspace what changed. Right now that is just the
resolution, but in the future you can expect flags for cases where just the
colorspace information changes, but not the resolution.

Which reminds me of two important missing pieces of information in your 
protocol:

1) You need to communicate the colorspace data:

- colorspace
- xfer_func
- ycbcr_enc/hsv_enc (unlikely you ever want to support HSV pixelformats, so I
 think you can ignore hsv_enc)
- quantization

See 
https://hverkuil.home.xs4all.nl/spec/uapi/v4l/pixfmt-v4l2.html#c.v4l2_pix_format
and the links to the colorspace sections in the V4L2 spec for details).

This information is part of the format, it is reported by the driver.

I'll take a look and think what can be put and how into the protocol,
do you think I'll have to implement all the above for
this stage?

Yes. Without it VMs will have no way of knowing how to reproduce the right 
colors.
They don't *have* to use this information, but it should be there. For cameras
this isn't all that important, for SDTV/HDTV sources this becomes more relevant
(esp. the quantization and ycbcr_enc information) and for sources with 
BT.2020/HDR
formats this is critical.

ok, then I'll add the following to the set_config request/response:

  uint32_t colorspace;
  uint32_t xfer_func;
  uint32_t ycbcr_enc;
  uint32_t quantization;

With this respect, I will need to put some OS agnostic constants
into the protocol, so if backend and frontend are not Linux/V4L2
based they can still talk to each other.
I see that V4L2 already defines constants for the above: [1], [2], [3], [4].

Do you think I can define the same replacing V4L2_ prefix
with XENCAMERA_, e.g. V4L2_XFER_FUNC_SRGB -> XENCAMERA_XFER_FUNC_SRGB?

Yes.


Do I need to define all those or there can be some subset of the
above for my simpler use-case?

Most of these defines directly map to standards. I would skip the following
defines:

V4L2_COLORSPACE_DEFAULT (not applicable)
V4L2_COLORSPACE_470_SYSTEM_*  (rarely used, if received by the HW the Xen 
backend
should map this to V4L2_COLORSPACE_SMPTE170M)
V4L2_COLORSPACE_JPEG (historical V4L2 artifact, see here how to map:
 
https://hverkuil.home.xs4all.nl/spec/uapi/v4l/colorspaces-details.html#col-jpeg)

V4L2_COLORSPACE_SMPTE240M (rarely used, map to V4L2_COLORSPACE_SMPTE170M if 
seen in backend)

V4L2_XFER_FUNC_SMPTE240M (rarely used, map to V4L2_XFER_FUNC_709)

V4L2_YCBCR_ENC_SMPTE240M (rarely used, map to V4L2_YCBCR_ENC_709)

While V4L2 allows 0 (DEFAULT) values for xfer_func, ycbcr_enc and quantization, 
and
provides macros to map default values to the actual values (for legacy reasons),
the Xen backend should always fill this in explicitly, using those same mapping
macros (see e.g. V4L2_MAP_XFER_FUNC_DEFAULT).

The V4L2 spec has extensive information on colorspaces (sections 2.14-2.17).


Thank you for such a detailed explanation!
I'll define the constants as agreed above.


The vivid driver can actually reproduce all combinations, so that's a good 
driver
to test this with.

You mean I can use it on backend side instead of real HW camera and
test all the configurations possible/those of interest?

Right.

Regards,

Hans

It seems that the number of changes discussed are begging
for the v2 of the protocol to be published ;)

Thank you,
Oleksandr

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.

Re: [Xen-devel] [PATCH 0/1] cameraif: Add ABI for para-virtualized

[Oleksandr Andrushchenko]

Hi, Laurent!

On 09/10/2018 03:48 PM, Laurent Pinchart wrote:

Hi Oleksandr,

Thank you for the patch.

On Tuesday, 31 July 2018 12:31:41 EEST Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

Hello!

At the moment Xen [1] already supports some virtual multimedia
features [2] such as virtual display, sound. It supports keyboards,
pointers and multi-touch devices all allowing Xen to be used in
automotive appliances, In-Vehicle Infotainment (IVI) systems
and many more.

This work adds a new Xen para-virtualized protocol for a virtual
camera device which extends multimedia capabilities of Xen even
farther: video conferencing, IVI, high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
  - pixel formats
  - resolutions
  - frame rates
2. Support basic camera controls:
  - contrast
  - brightness
  - hue
  - saturation
3. Support streaming control
4. Support zero-copying use-cases

I hope that Xen and V4L and other communities could give their
valuable feedback on this work, so I can update the protocol
to better fit any additional requirements I might have missed.

I'll start with a question : what are the expected use cases ?

The very basic use-case is to share a capture stream produced
by a single HW camera to multiple VMs for different
purposes: In-Vehicle Infotainment, high definition maps etc.
all running in different (dedicated) VMs at the same time

  The ones listed
above sound like they would better be solved by passing the corresponding
device(s) to the guest.

With the above use-case I cannot tell how passing the
corresponding *single* device can serve *multiple* VMs.
Could you please elaborate more on the solution you see?



[1] https://www.xenproject.org/
[2] https://xenbits.xen.org/gitweb/?p=xen.git;a=tree;f=xen/include/public/io

Oleksandr Andrushchenko (1):
   cameraif: add ABI for para-virtual camera

  xen/include/public/io/cameraif.h | 981 +++
  1 file changed, 981 insertions(+)
  create mode 100644 xen/include/public/io/cameraif.h

Thank you,
Oleksandr

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

On 09/10/2018 02:09 PM, Hans Verkuil wrote:

On 09/10/2018 11:52 AM, Oleksandr Andrushchenko wrote:

On 09/10/2018 12:04 PM, Hans Verkuil wrote:

On 09/10/2018 10:24 AM, Oleksandr Andrushchenko wrote:

On 09/10/2018 10:53 AM, Hans Verkuil wrote:

Hi Oleksandr,

On 09/10/2018 09:16 AM, Oleksandr Andrushchenko wrote:




I suspect that you likely will want to support such sources eventually, so
it pays to design this with that in mind.

Again, I think that this is the backend to hide these
use-cases from the frontend.

I'm not sure you can: say you are playing a bluray connected to the system
with HDMI, then if there is a resolution change, what do you do? You can tear
everything down and build it up again, or you can just tell frontends that
something changed and that they have to look at the new vcamera configuration.

The latter seems to be more sensible to me. It is really not much that you
need to do: all you really need is an event signalling that something changed.
In V4L2 that's the V4L2_EVENT_SOURCE_CHANGE.

well, this complicates things a lot as I'll have to
re-allocate buffers - right?

Right. Different resolutions means different sized buffers and usually lots of
changes throughout the whole video pipeline, which in this case can even
go into multiple VMs.

One additional thing to keep in mind for the future: V4L2_EVENT_SOURCE_CHANGE
has a flags field that tells userspace what changed. Right now that is just the
resolution, but in the future you can expect flags for cases where just the
colorspace information changes, but not the resolution.

Which reminds me of two important missing pieces of information in your 
protocol:

1) You need to communicate the colorspace data:

- colorspace
- xfer_func
- ycbcr_enc/hsv_enc (unlikely you ever want to support HSV pixelformats, so I
think you can ignore hsv_enc)
- quantization

See 
https://hverkuil.home.xs4all.nl/spec/uapi/v4l/pixfmt-v4l2.html#c.v4l2_pix_format
and the links to the colorspace sections in the V4L2 spec for details).

This information is part of the format, it is reported by the driver.

I'll take a look and think what can be put and how into the protocol,
do you think I'll have to implement all the above for
this stage?

Yes. Without it VMs will have no way of knowing how to reproduce the right 
colors.
They don't *have* to use this information, but it should be there. For cameras
this isn't all that important, for SDTV/HDTV sources this becomes more relevant
(esp. the quantization and ycbcr_enc information) and for sources with 
BT.2020/HDR
formats this is critical.

ok, then I'll add the following to the set_config request/response:

    uint32_t colorspace;
    uint32_t xfer_func;
    uint32_t ycbcr_enc;
    uint32_t quantization;

With this respect, I will need to put some OS agnostic constants
into the protocol, so if backend and frontend are not Linux/V4L2
based they can still talk to each other.
I see that V4L2 already defines constants for the above: [1], [2], [3], [4].

Do you think I can define the same replacing V4L2_ prefix
with XENCAMERA_, e.g. V4L2_XFER_FUNC_SRGB -> XENCAMERA_XFER_FUNC_SRGB?

Do I need to define all those or there can be some subset of the
above for my simpler use-case?


The vivid driver can actually reproduce all combinations, so that's a good 
driver
to test this with.

You mean I can use it on backend side instead of real HW camera and
test all the configurations possible/those of interest?

2) If you support interlaced formats and V4L2_FIELD_ALTERNATE (i.e.
 each buffer contains a single field), then you need to be able to tell
 userspace whether the dequeued buffer contains a top or bottom field.

I think at the first stage we can assume that interlaced
formats are not supported and add such support later if need be.

Frankly I consider that a smart move :-) Interlaced formats are awful...

You just have to keep this in mind if you ever have to add support for this.

Agreed



Also, what to do with dropped frames/fields: V4L2 has a sequence counter and
timestamp that can help detecting that. You probably need something similar.

Ok, this can be reported as part of XENCAMERA_EVT_FRAME_AVAIL event

But anyways, I can add
#define XENCAMERA_EVT_CFG_CHANGE   0x01
in the protocol, so we can address this use-case




1. set format command:
 * pixel_format - uint32_t, pixel format to be used, FOURCC code.
 * width - uint32_t, width in pixels.
 * height - uint32_t, height in pixels.

2. Set frame rate command:
 + * frame_rate_numer - uint32_t, numerator of the frame rate.
 + * frame_rate_denom - uint32_t, denominator of the frame rate.

3. Set/request num bufs:
 * num_bufs - uint8_t, desired number of buffers to be used.

I like this much better. 1+2 could be combined, but 3 should definitely remain
separate.

ok, then 1+2 combined + 3 separate.
Do you think we can still name 1+2 as "set_format" or "set_config"
will fit better?

set_format is 

Re: [Xen-devel] [PATCH 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

On 09/10/2018 12:04 PM, Hans Verkuil wrote:

On 09/10/2018 10:24 AM, Oleksandr Andrushchenko wrote:

On 09/10/2018 10:53 AM, Hans Verkuil wrote:

Hi Oleksandr,

On 09/10/2018 09:16 AM, Oleksandr Andrushchenko wrote:




I suspect that you likely will want to support such sources eventually, so
it pays to design this with that in mind.

Again, I think that this is the backend to hide these
use-cases from the frontend.

I'm not sure you can: say you are playing a bluray connected to the system
with HDMI, then if there is a resolution change, what do you do? You can tear
everything down and build it up again, or you can just tell frontends that
something changed and that they have to look at the new vcamera configuration.

The latter seems to be more sensible to me. It is really not much that you
need to do: all you really need is an event signalling that something changed.
In V4L2 that's the V4L2_EVENT_SOURCE_CHANGE.

well, this complicates things a lot as I'll have to
re-allocate buffers - right?

Right. Different resolutions means different sized buffers and usually lots of
changes throughout the whole video pipeline, which in this case can even
go into multiple VMs.

One additional thing to keep in mind for the future: V4L2_EVENT_SOURCE_CHANGE
has a flags field that tells userspace what changed. Right now that is just the
resolution, but in the future you can expect flags for cases where just the
colorspace information changes, but not the resolution.

Which reminds me of two important missing pieces of information in your 
protocol:

1) You need to communicate the colorspace data:

- colorspace
- xfer_func
- ycbcr_enc/hsv_enc (unlikely you ever want to support HSV pixelformats, so I
   think you can ignore hsv_enc)
- quantization

See 
https://hverkuil.home.xs4all.nl/spec/uapi/v4l/pixfmt-v4l2.html#c.v4l2_pix_format
and the links to the colorspace sections in the V4L2 spec for details).

This information is part of the format, it is reported by the driver.

I'll take a look and think what can be put and how into the protocol,
do you think I'll have to implement all the above for
this stage?



2) If you support interlaced formats and V4L2_FIELD_ALTERNATE (i.e.
each buffer contains a single field), then you need to be able to tell
userspace whether the dequeued buffer contains a top or bottom field.

I think at the first stage we can assume that interlaced
formats are not supported and add such support later if need be.


Also, what to do with dropped frames/fields: V4L2 has a sequence counter and
timestamp that can help detecting that. You probably need something similar.

Ok, this can be reported as part of XENCAMERA_EVT_FRAME_AVAIL event



But anyways, I can add
#define XENCAMERA_EVT_CFG_CHANGE   0x01
in the protocol, so we can address this use-case




1. set format command:
* pixel_format - uint32_t, pixel format to be used, FOURCC code.
* width - uint32_t, width in pixels.
* height - uint32_t, height in pixels.

2. Set frame rate command:
+ * frame_rate_numer - uint32_t, numerator of the frame rate.
+ * frame_rate_denom - uint32_t, denominator of the frame rate.

3. Set/request num bufs:
* num_bufs - uint8_t, desired number of buffers to be used.

I like this much better. 1+2 could be combined, but 3 should definitely remain
separate.

ok, then 1+2 combined + 3 separate.
Do you think we can still name 1+2 as "set_format" or "set_config"
will fit better?

set_format is closer to S_FMT as used in V4L2, so I have a slight preference
for that, but it is really up to you.

I'll probably stick to SET_CONFIG here



+ *
+ * See response format for this request.
+ *
+ * Notes:
+ *  - frontend must check the corresponding response in order to see
+ *if the values reported back by the backend do match the desired ones
+ *and can be accepted.
+ *  - frontend may send multiple XENCAMERA_OP_SET_CONFIG requests before
+ *sending XENCAMERA_OP_STREAM_START request to update or tune the
+ *configuration.
+ */
+struct xencamera_config {
+uint32_t pixel_format;
+uint32_t width;
+uint32_t height;
+uint32_t frame_rate_nom;
+uint32_t frame_rate_denom;
+uint8_t num_bufs;
+};
+
+/*
+ * Request buffer details - request camera buffer's memory layout.
+ * detailed description:
+ * 01 2   3octet
+ * +++++
+ * |   id|_GET_BUF_DETAILS|   reserved | 4
+ * +++++
+ * |  reserved | 8
+ * +++++
+ * |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/|
+ * +++++
+ * |  reserved 

Re: [Xen-devel] [PATCH 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

On 09/10/2018 11:14 AM, Hans Verkuil wrote:

On 09/10/2018 07:59 AM, Oleksandr Andrushchenko wrote:

Hi, Hans!

On 09/09/2018 01:42 PM, Hans Verkuil wrote:

On 09/04/2018 08:56 AM, Oleksandr Andrushchenko wrote:

On 09/03/2018 06:25 PM, Hans Verkuil wrote:

Hi Oleksandr,

On 09/03/2018 12:16 PM, Oleksandr Andrushchenko wrote:

On 08/21/2018 08:54 AM, Oleksandr Andrushchenko wrote:

On 08/14/2018 11:30 AM, Juergen Gross wrote:

On 31/07/18 11:31, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
 - pixel formats
 - resolutions
 - frame rates
2. Support basic camera controls:
 - contrast
 - brightness
 - hue
 - saturation
3. Support streaming control
4. Support zero-copying use-cases

Signed-off-by: Oleksandr Andrushchenko


Some style issues below...

Will fix all the below, thank you!

I would like to draw some attention of the Linux/V4L community to this
protocol as the plan is that once it is accepted for Xen we plan to
upstream a Linux camera front-end kernel driver which will be based
on this work and will be a V4L2 device driver (this is why I have sent
this patch not only to Xen, but to the corresponding Linux mailing list
as well)

ping

Sorry, this got buried in my mailbox, I only came across it today. I'll try
to review this this week, if not, just ping me again.

Thank you for your time

I had one high-level question, though:

What types of hardware do you intend to target? This initial version targets
(very) simple webcams, but what about HDMI or SDTV receivers? Or hardware
codecs? Or complex embedded video pipelines?

In other words, where are you planning to draw the line?

Even with just simple cameras there is a difference between regular UVC
webcams and cameras used with embedded systems: for the latter you often
need to provide more control w.r.t. white-balancing etc., things that a
UVC webcam will generally do for you in the webcam's firmware.

The use-cases we want to implement are mostly in automotive/embedded domain,
so there are many performance restrictions apply.
We are not targeting virtualizing very complex hardware and have no
intention
to make a 1:1 mapping of the real hardware: for that one can pass-through
a real HW device to a virtual machine (VM). The goal is to share a single
camera device to multiple virtual machines, no codecs, receivers etc.

Controlling the same HW device from different VMs doesn't look feasible:
what if the same control is set to different values from different VMs?

You can do this, actually: in V4L2 you can get an event when another process
changes a control, and update your own GUI/internal state accordingly.

So in this case if one VM changes a control, an event is sent to all others
that the control has changed value.

Well, technically this can be done by introducing one more
event for such a notification. But, from system partitioning
POV, I am still not convinced this should be done: I would prefer
that a single VM owns such a control and even which control and which
VM is decided while configuring the whole system.
So, I would like to keep it as is.

Well, I am not convinced you can avoid this: some controls are set by drivers
when something happens and so applications will subscribe to the control and
wait for changes in their values. While this is for more advanced use-cases
(certainly more than what you propose today), I would suggest that it is wise
to at least think on how this can be added in the future.

Controls and control events are a key part of V4L2.


ok, then I'll add
#define XENCAMERA_EVT_CTRL_CHANGE  0x02


Of course, this can be achieved if the corresponding backend can
post-process
original camera image with GPU, for example, thus applying different filters
for different VMs effectively emulating camera controls.
But this requires additional CPU/GPU power which we try to avoid.

System partitioning (camera and controls assignment) is done at
configuration
time (remember we are in automotive/embedded world, so most of the time
the set
of VMs requiring cameras is known at this stage and the configuration
remains
static at run-time). So, when para-virtualized (PV) approach is used then we
only implement very basic controls (those found in the protocol), so one can
assign set of controls (all or some) to one of the VMs (main or mission
critical
VM or whatever) allowing that VM to adjusts those for all VMs at once.
For other
VMs think of it as firmware implemented adjustment. And the backend still
controls the rest of the controls of the real HW camera you mention.

Just

Re: [Xen-devel] [PATCH 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

On 09/10/2018 10:53 AM, Hans Verkuil wrote:

Hi Oleksandr,

On 09/10/2018 09:16 AM, Oleksandr Andrushchenko wrote:

Hi, Hans!

On 09/09/2018 01:31 PM, Hans Verkuil wrote:

Hi Oleksandr,

Sorry for the delay in reviewing, I missed this patch until you pinged me, and
I was very busy after that as well.

I do appreciate you spending time on this!

On 07/31/2018 11:31 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
   - pixel formats
   - resolutions
   - frame rates
2. Support basic camera controls:
   - contrast
   - brightness
   - hue
   - saturation
3. Support streaming control
4. Support zero-copying use-cases

Signed-off-by: Oleksandr Andrushchenko 
---
   xen/include/public/io/cameraif.h | 981 +++
   1 file changed, 981 insertions(+)
   create mode 100644 xen/include/public/io/cameraif.h

diff --git a/xen/include/public/io/cameraif.h b/xen/include/public/io/cameraif.h
new file mode 100644
index ..bdc6a1262fcf
--- /dev/null
+++ b/xen/include/public/io/cameraif.h
@@ -0,0 +1,981 @@
+/**
+ * cameraif.h
+ *
+ * Unified camera device I/O interface for Xen guest OSes.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright (C) 2018 EPAM Systems Inc.
+ *
+ * Author: Oleksandr Andrushchenko 
+ */

Use SPDX tag instead of copying the license text.

This is yet a Xen header which belongs to Xen project and
all the rest of the protocols have the same license header.
If Xen community decides to use SPDX then I'll definitely follow.

Ah, yes, I was reviewing this as a kernel header, I hadn't realized
that this isn't a kernel header. Since it isn't a kernel header, you
can disregard my comments about style and naming conventions, since
you have your own.


Konrad, do you think this is the right time for such a move?

+
+#ifndef __XEN_PUBLIC_IO_CAMERAIF_H__
+#define __XEN_PUBLIC_IO_CAMERAIF_H__
+
+#include "ring.h"
+#include "../grant_table.h"
+
+/*
+ **
+ *   Protocol version
+ **
+ */
+#define XENCAMERA_PROTOCOL_VERSION "1"
+
+/*
+ **
+ *  Feature and Parameter Negotiation
+ **
+ *
+ * Front->back notifications: when enqueuing a new request, sending a
+ * notification can be made conditional on xencamera_req (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Backends must set
+ * xencamera_req appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
+ *
+ * Back->front notifications: when enqueuing a new response, sending a
+ * notification can be made conditional on xencamera_resp (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Frontends must set
+ * xencamera_resp appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
+ *
+ * The two halves of a para-virtual camera driver utilize nodes within
+ * XenStore to communicate capabilities and to negotiate operating parameters.
+ * This section enumerates these nodes which reside in the respective front and
+ * backend portions of XenStore, following the XenBus convention.
+ *
+ * All data in XenStore is stored as strings. Nodes specifying numeric

Re: [Xen-devel] [PATCH 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

Hi, Hans!

On 09/09/2018 01:31 PM, Hans Verkuil wrote:

Hi Oleksandr,

Sorry for the delay in reviewing, I missed this patch until you pinged me, and
I was very busy after that as well.

I do appreciate you spending time on this!


On 07/31/2018 11:31 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
  - pixel formats
  - resolutions
  - frame rates
2. Support basic camera controls:
  - contrast
  - brightness
  - hue
  - saturation
3. Support streaming control
4. Support zero-copying use-cases

Signed-off-by: Oleksandr Andrushchenko 
---
  xen/include/public/io/cameraif.h | 981 +++
  1 file changed, 981 insertions(+)
  create mode 100644 xen/include/public/io/cameraif.h

diff --git a/xen/include/public/io/cameraif.h b/xen/include/public/io/cameraif.h
new file mode 100644
index ..bdc6a1262fcf
--- /dev/null
+++ b/xen/include/public/io/cameraif.h
@@ -0,0 +1,981 @@
+/**
+ * cameraif.h
+ *
+ * Unified camera device I/O interface for Xen guest OSes.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright (C) 2018 EPAM Systems Inc.
+ *
+ * Author: Oleksandr Andrushchenko 
+ */

Use SPDX tag instead of copying the license text.

This is yet a Xen header which belongs to Xen project and
all the rest of the protocols have the same license header.
If Xen community decides to use SPDX then I'll definitely follow.

Konrad, do you think this is the right time for such a move?



+
+#ifndef __XEN_PUBLIC_IO_CAMERAIF_H__
+#define __XEN_PUBLIC_IO_CAMERAIF_H__
+
+#include "ring.h"
+#include "../grant_table.h"
+
+/*
+ **
+ *   Protocol version
+ **
+ */
+#define XENCAMERA_PROTOCOL_VERSION "1"
+
+/*
+ **
+ *  Feature and Parameter Negotiation
+ **
+ *
+ * Front->back notifications: when enqueuing a new request, sending a
+ * notification can be made conditional on xencamera_req (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Backends must set
+ * xencamera_req appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
+ *
+ * Back->front notifications: when enqueuing a new response, sending a
+ * notification can be made conditional on xencamera_resp (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Frontends must set
+ * xencamera_resp appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
+ *
+ * The two halves of a para-virtual camera driver utilize nodes within
+ * XenStore to communicate capabilities and to negotiate operating parameters.
+ * This section enumerates these nodes which reside in the respective front and
+ * backend portions of XenStore, following the XenBus convention.
+ *
+ * All data in XenStore is stored as strings. Nodes specifying numeric
+ * values are encoded in decimal. Integer value ranges listed below are
+ * expressed as fixed sized integer types capable of storing the conversion
+ * of a properly formatted node string, without loss of information.
+ *
+ ***

Re: [Xen-devel] [PATCH 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

Hi, Hans!

On 09/09/2018 01:42 PM, Hans Verkuil wrote:

On 09/04/2018 08:56 AM, Oleksandr Andrushchenko wrote:

On 09/03/2018 06:25 PM, Hans Verkuil wrote:

Hi Oleksandr,

On 09/03/2018 12:16 PM, Oleksandr Andrushchenko wrote:

On 08/21/2018 08:54 AM, Oleksandr Andrushchenko wrote:

On 08/14/2018 11:30 AM, Juergen Gross wrote:

On 31/07/18 11:31, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
- pixel formats
- resolutions
- frame rates
2. Support basic camera controls:
- contrast
- brightness
- hue
- saturation
3. Support streaming control
4. Support zero-copying use-cases

Signed-off-by: Oleksandr Andrushchenko


Some style issues below...

Will fix all the below, thank you!

I would like to draw some attention of the Linux/V4L community to this
protocol as the plan is that once it is accepted for Xen we plan to
upstream a Linux camera front-end kernel driver which will be based
on this work and will be a V4L2 device driver (this is why I have sent
this patch not only to Xen, but to the corresponding Linux mailing list
as well)

ping

Sorry, this got buried in my mailbox, I only came across it today. I'll try
to review this this week, if not, just ping me again.

Thank you for your time

I had one high-level question, though:

What types of hardware do you intend to target? This initial version targets
(very) simple webcams, but what about HDMI or SDTV receivers? Or hardware
codecs? Or complex embedded video pipelines?

In other words, where are you planning to draw the line?

Even with just simple cameras there is a difference between regular UVC
webcams and cameras used with embedded systems: for the latter you often
need to provide more control w.r.t. white-balancing etc., things that a
UVC webcam will generally do for you in the webcam's firmware.

The use-cases we want to implement are mostly in automotive/embedded domain,
so there are many performance restrictions apply.
We are not targeting virtualizing very complex hardware and have no
intention
to make a 1:1 mapping of the real hardware: for that one can pass-through
a real HW device to a virtual machine (VM). The goal is to share a single
camera device to multiple virtual machines, no codecs, receivers etc.

Controlling the same HW device from different VMs doesn't look feasible:
what if the same control is set to different values from different VMs?

You can do this, actually: in V4L2 you can get an event when another process
changes a control, and update your own GUI/internal state accordingly.

So in this case if one VM changes a control, an event is sent to all others
that the control has changed value.

Well, technically this can be done by introducing one more
event for such a notification. But, from system partitioning
POV, I am still not convinced this should be done: I would prefer
that a single VM owns such a control and even which control and which
VM is decided while configuring the whole system.
So, I would like to keep it as is.



Of course, this can be achieved if the corresponding backend can
post-process
original camera image with GPU, for example, thus applying different filters
for different VMs effectively emulating camera controls.
But this requires additional CPU/GPU power which we try to avoid.

System partitioning (camera and controls assignment) is done at
configuration
time (remember we are in automotive/embedded world, so most of the time
the set
of VMs requiring cameras is known at this stage and the configuration
remains
static at run-time). So, when para-virtualized (PV) approach is used then we
only implement very basic controls (those found in the protocol), so one can
assign set of controls (all or some) to one of the VMs (main or mission
critical
VM or whatever) allowing that VM to adjusts those for all VMs at once.
For other
VMs think of it as firmware implemented adjustment. And the backend still
controls the rest of the controls of the real HW camera you mention.

Just an example of automotive use-case (we can imagine many more):
1. Driver Domain - owns real camera HW and runs the camera backend.
 Uses camera output for mission critical tasks, e.g. parking assistance.
2. In-Vehicle Infotainment domain - uses PV camera for infotainment
purposes,
 e.g. taking pictures while in motion.
3. Navigation domain - uses PV camera for high definition maps

Hope, this helps understanding the possible uses of the proposed
protocol, its
intention and restrictions.

Right, so in this scenario you probably do not want hotpluggable
sources in the Driver Domain. So

Re: [Xen-devel] [PATCH 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

On 09/03/2018 06:25 PM, Hans Verkuil wrote:

Hi Oleksandr,

On 09/03/2018 12:16 PM, Oleksandr Andrushchenko wrote:

On 08/21/2018 08:54 AM, Oleksandr Andrushchenko wrote:

On 08/14/2018 11:30 AM, Juergen Gross wrote:

On 31/07/18 11:31, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
   - pixel formats
   - resolutions
   - frame rates
2. Support basic camera controls:
   - contrast
   - brightness
   - hue
   - saturation
3. Support streaming control
4. Support zero-copying use-cases

Signed-off-by: Oleksandr Andrushchenko


Some style issues below...

Will fix all the below, thank you!

I would like to draw some attention of the Linux/V4L community to this
protocol as the plan is that once it is accepted for Xen we plan to
upstream a Linux camera front-end kernel driver which will be based
on this work and will be a V4L2 device driver (this is why I have sent
this patch not only to Xen, but to the corresponding Linux mailing list
as well)

ping

Sorry, this got buried in my mailbox, I only came across it today. I'll try
to review this this week, if not, just ping me again.

Thank you for your time


I had one high-level question, though:

What types of hardware do you intend to target? This initial version targets
(very) simple webcams, but what about HDMI or SDTV receivers? Or hardware
codecs? Or complex embedded video pipelines?

In other words, where are you planning to draw the line?

Even with just simple cameras there is a difference between regular UVC
webcams and cameras used with embedded systems: for the latter you often
need to provide more control w.r.t. white-balancing etc., things that a
UVC webcam will generally do for you in the webcam's firmware.

The use-cases we want to implement are mostly in automotive/embedded domain,
so there are many performance restrictions apply.
We are not targeting virtualizing very complex hardware and have no 
intention

to make a 1:1 mapping of the real hardware: for that one can pass-through
a real HW device to a virtual machine (VM). The goal is to share a single
camera device to multiple virtual machines, no codecs, receivers etc.

Controlling the same HW device from different VMs doesn't look feasible:
what if the same control is set to different values from different VMs?
Of course, this can be achieved if the corresponding backend can 
post-process

original camera image with GPU, for example, thus applying different filters
for different VMs effectively emulating camera controls.
But this requires additional CPU/GPU power which we try to avoid.

System partitioning (camera and controls assignment) is done at 
configuration
time (remember we are in automotive/embedded world, so most of the time 
the set
of VMs requiring cameras is known at this stage and the configuration 
remains

static at run-time). So, when para-virtualized (PV) approach is used then we
only implement very basic controls (those found in the protocol), so one can
assign set of controls (all or some) to one of the VMs (main or mission 
critical
VM or whatever) allowing that VM to adjusts those for all VMs at once. 
For other

VMs think of it as firmware implemented adjustment. And the backend still
controls the rest of the controls of the real HW camera you mention.

Just an example of automotive use-case (we can imagine many more):
1. Driver Domain - owns real camera HW and runs the camera backend.
   Uses camera output for mission critical tasks, e.g. parking assistance.
2. In-Vehicle Infotainment domain - uses PV camera for infotainment 
purposes,

   e.g. taking pictures while in motion.
3. Navigation domain - uses PV camera for high definition maps

Hope, this helps understanding the possible uses of the proposed 
protocol, its

intention and restrictions.


Regards,

Hans

Thank you,
Oleksandr

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

On 08/21/2018 08:54 AM, Oleksandr Andrushchenko wrote:

On 08/14/2018 11:30 AM, Juergen Gross wrote:

On 31/07/18 11:31, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
  - pixel formats
  - resolutions
  - frame rates
2. Support basic camera controls:
  - contrast
  - brightness
  - hue
  - saturation
3. Support streaming control
4. Support zero-copying use-cases

Signed-off-by: Oleksandr Andrushchenko 


Some style issues below...

Will fix all the below, thank you!

I would like to draw some attention of the Linux/V4L community to this
protocol as the plan is that once it is accepted for Xen we plan to
upstream a Linux camera front-end kernel driver which will be based
on this work and will be a V4L2 device driver (this is why I have sent
this patch not only to Xen, but to the corresponding Linux mailing list
as well)

ping



---
  xen/include/public/io/cameraif.h | 981 
+++

  1 file changed, 981 insertions(+)
  create mode 100644 xen/include/public/io/cameraif.h

diff --git a/xen/include/public/io/cameraif.h 
b/xen/include/public/io/cameraif.h

new file mode 100644
index ..bdc6a1262fcf
--- /dev/null
+++ b/xen/include/public/io/cameraif.h
+struct xencamera_config {
+    uint32_t pixel_format;
+    uint32_t width;
+    uint32_t height;
+    uint32_t frame_rate_nom;
+    uint32_t frame_rate_denom;
+    uint8_t num_bufs;

Add explicit padding?


+};
+struct xencamera_req {
+    uint16_t id;
+    uint8_t operation;
+    uint8_t reserved[5];
+    union {
+    struct xencamera_config config;
+    struct xencamera_buf_create_req buf_create;
+    struct xencamera_buf_destroy_req buf_destroy;
+    struct xencamera_set_ctrl_req set_ctrl;

No tabs, please.


+    uint8_t reserved[56];
+    } req;
+};
+
+struct xencamera_resp {
+    uint16_t id;
+    uint8_t operation;
+    uint8_t reserved;
+    int32_t status;
+    union {
+    struct xencamera_config config;
+    struct xencamera_buf_details_resp buf_details;
+    struct xencamera_get_ctrl_details_resp ctrl_details;

Tab again.


+    uint8_t reserved1[56];
+    } resp;
+};


Juergen

Thank you,
Oleksandr



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2] libgnttab: Add support for Linux dma-buf

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Add support for Linux grant device driver extension which allows
converting existing dma-buf's into an array of grant references
and vise versa. This is only implemented for Linux as other OSes
have no Linux dma-buf support.

Bump gnttab library minor version to 3.

Signed-off-by: Oleksandr Andrushchenko 
---
 tools/include/xen-sys/Linux/gntdev.h  | 106 +++
 tools/libs/gnttab/Makefile|   2 +-
 tools/libs/gnttab/gnttab_core.c   |  26 ++
 tools/libs/gnttab/gnttab_unimp.c  |  26 ++
 tools/libs/gnttab/include/xengnttab.h |  61 ++
 tools/libs/gnttab/libxengnttab.map|   8 ++
 tools/libs/gnttab/linux.c | 117 ++
 tools/libs/gnttab/minios.c|  26 ++
 tools/libs/gnttab/private.h   |  13 +++
 9 files changed, 384 insertions(+), 1 deletion(-)

diff --git a/tools/include/xen-sys/Linux/gntdev.h 
b/tools/include/xen-sys/Linux/gntdev.h
index 0ca07c92b21c..d16076044c71 100644
--- a/tools/include/xen-sys/Linux/gntdev.h
+++ b/tools/include/xen-sys/Linux/gntdev.h
@@ -4,6 +4,7 @@
  * Interface to /dev/xen/gntdev.
  * 
  * Copyright (c) 2007, D G Murray
+ * Copyright (c) 2018, Oleksandr Andrushchenko, EPAM Systems Inc.
  * 
  * This program is free software; you can redistribute it and/or
  * modify it under the terms of the GNU General Public License version 2
@@ -168,4 +169,109 @@ struct ioctl_gntdev_grant_copy {
 struct ioctl_gntdev_grant_copy_segment *segments;
 };
 
+/*
+ * Flags to be used while requesting memory mapping's backing storage
+ * to be allocated with DMA API.
+ */
+
+/*
+ * The buffer is backed with memory allocated with dma_alloc_wc.
+ */
+#define GNTDEV_DMA_FLAG_WC (1 << 0)
+
+/*
+ * The buffer is backed with memory allocated with dma_alloc_coherent.
+ */
+#define GNTDEV_DMA_FLAG_COHERENT   (1 << 1)
+
+/*
+ * Create a dma-buf [1] from grant references @refs of count @count provided
+ * by the foreign domain @domid with flags @flags.
+ *
+ * By default dma-buf is backed by system memory pages, but by providing
+ * one of the GNTDEV_DMA_FLAG_XXX flags it can also be created as
+ * a DMA write-combine or coherent buffer, e.g. allocated with dma_alloc_wc/
+ * dma_alloc_coherent.
+ *
+ * Returns 0 if dma-buf was successfully created and the corresponding
+ * dma-buf's file descriptor is returned in @fd.
+ *
+ * [1] 
https://elixir.bootlin.com/linux/latest/source/Documentation/driver-api/dma-buf.rst
+ */
+
+#define IOCTL_GNTDEV_DMABUF_EXP_FROM_REFS \
+_IOC(_IOC_NONE, 'G', 9, \
+ sizeof(struct ioctl_gntdev_dmabuf_exp_from_refs))
+struct ioctl_gntdev_dmabuf_exp_from_refs {
+/* IN parameters. */
+/* Specific options for this dma-buf: see GNTDEV_DMABUF_FLAG_XXX. */
+uint32_t flags;
+/* Number of grant references in @refs array. */
+uint32_t count;
+/* OUT parameters. */
+/* File descriptor of the dma-buf. */
+uint32_t fd;
+/* The domain ID of the grant references to be mapped. */
+uint32_t domid;
+/* Variable IN parameter. */
+/* Array of grant references of size @count. */
+uint32_t refs[1];
+};
+
+/*
+ * This will block until the dma-buf with the file descriptor @fd is
+ * released. This is only valid for buffers created with
+ * IOCTL_GNTDEV_DMABUF_EXP_FROM_REFS.
+ *
+ * If withing @wait_to_ms milliseconds the buffer is not released
+ * then -ETIMEDOUT error is returned.
+ * If the buffer with file descriptor @fd does not exist or has already
+ * been released, then -ENOENT is returned. For valid file descriptors
+ * this must not be treated as error.
+ */
+#define IOCTL_GNTDEV_DMABUF_EXP_WAIT_RELEASED \
+_IOC(_IOC_NONE, 'G', 10, \
+ sizeof(struct ioctl_gntdev_dmabuf_exp_wait_released))
+struct ioctl_gntdev_dmabuf_exp_wait_released {
+/* IN parameters */
+uint32_t fd;
+uint32_t wait_to_ms;
+};
+
+/*
+ * Import a dma-buf with file descriptor @fd and export granted references
+ * to the pages of that dma-buf into array @refs of size @count.
+ */
+#define IOCTL_GNTDEV_DMABUF_IMP_TO_REFS \
+_IOC(_IOC_NONE, 'G', 11, \
+ sizeof(struct ioctl_gntdev_dmabuf_imp_to_refs))
+struct ioctl_gntdev_dmabuf_imp_to_refs {
+/* IN parameters. */
+/* File descriptor of the dma-buf. */
+uint32_t fd;
+/* Number of grant references in @refs array. */
+uint32_t count;
+/* The domain ID for which references to be granted. */
+uint32_t domid;
+/* Reserved - must be zero. */
+uint32_t reserved;
+/* OUT parameters. */
+/* Array of grant references of size @count. */
+uint32_t refs[1];
+};
+
+/*
+ * This will close all references to an imported buffer, so it can be
+ * released by the owner. This is only valid for buffers created with
+ * IOCTL_GNTDEV_DMABUF_IMP_TO_REFS.
+ */
+#define IOCTL_GNTDEV_DMABUF_IMP_RELEASE \
+_IOC(_IOC_NONE, 'G', 12, \
+ sizeof(struct ioctl_gntdev_dmabuf_imp_relea

Re: [Xen-devel] [PATCH 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

On 08/14/2018 11:30 AM, Juergen Gross wrote:

On 31/07/18 11:31, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
  - pixel formats
  - resolutions
  - frame rates
2. Support basic camera controls:
  - contrast
  - brightness
  - hue
  - saturation
3. Support streaming control
4. Support zero-copying use-cases

Signed-off-by: Oleksandr Andrushchenko 

Some style issues below...

Will fix all the below, thank you!

I would like to draw some attention of the Linux/V4L community to this
protocol as the plan is that once it is accepted for Xen we plan to
upstream a Linux camera front-end kernel driver which will be based
on this work and will be a V4L2 device driver (this is why I have sent
this patch not only to Xen, but to the corresponding Linux mailing list
as well)


---
  xen/include/public/io/cameraif.h | 981 +++
  1 file changed, 981 insertions(+)
  create mode 100644 xen/include/public/io/cameraif.h

diff --git a/xen/include/public/io/cameraif.h b/xen/include/public/io/cameraif.h
new file mode 100644
index ..bdc6a1262fcf
--- /dev/null
+++ b/xen/include/public/io/cameraif.h
+struct xencamera_config {
+uint32_t pixel_format;
+uint32_t width;
+uint32_t height;
+uint32_t frame_rate_nom;
+uint32_t frame_rate_denom;
+uint8_t num_bufs;

Add explicit padding?


+};
+struct xencamera_req {
+uint16_t id;
+uint8_t operation;
+uint8_t reserved[5];
+union {
+struct xencamera_config config;
+struct xencamera_buf_create_req buf_create;
+   struct xencamera_buf_destroy_req buf_destroy;
+   struct xencamera_set_ctrl_req set_ctrl;

No tabs, please.


+uint8_t reserved[56];
+} req;
+};
+
+struct xencamera_resp {
+uint16_t id;
+uint8_t operation;
+uint8_t reserved;
+int32_t status;
+union {
+struct xencamera_config config;
+struct xencamera_buf_details_resp buf_details;
+   struct xencamera_get_ctrl_details_resp ctrl_details;

Tab again.


+uint8_t reserved1[56];
+} resp;
+};


Juergen

Thank you,
Oleksandr

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] libgnttab: Add support for Linux dma-buf

[Oleksandr Andrushchenko]

On 08/20/2018 05:40 PM, Wei Liu wrote:

On Mon, Jul 23, 2018 at 03:27:25PM +0300, Oleksandr Andrushchenko wrote:
[...]

diff --git a/tools/libs/gnttab/linux.c b/tools/libs/gnttab/linux.c
index 8347ddd3d9cf..9765146f7eb6 100644
--- a/tools/libs/gnttab/linux.c
+++ b/tools/libs/gnttab/linux.c
@@ -1,5 +1,6 @@
  /*
   * Copyright (c) 2007-2008, D G Murray 
+ * Copyright (c) 2018, Oleksandr Andrushchenko, EPAM Systems Inc.
   *
   * This library is free software; you can redistribute it and/or
   * modify it under the terms of the GNU Lesser General Public
@@ -309,6 +310,118 @@ int osdep_gnttab_grant_copy(xengnttab_handle *xgt,
  return rc;
  }
  
+int osdep_gnttab_dmabuf_exp_from_refs(xengnttab_handle *xgt, uint32_t domid,

+  uint32_t flags, uint32_t count,
+  const uint32_t *refs,
+  uint32_t *dmabuf_fd)
+{
+struct ioctl_gntdev_dmabuf_exp_from_refs *from_refs;

You can set from_refs to NULL here, then ...


+int rc = 0;
+
+if ( !count )
+{
+errno = EINVAL;
+return -1;

 rc = -1;
 goto out;

in all exit paths.

I don't like using two error handling styles. So please either change to
use goto only or delete the only goto in this function.


+}
+
+from_refs = malloc(sizeof(*from_refs) +
+   (count - 1) * sizeof(from_refs->refs[0]));
+if ( !from_refs )
+{
+errno = ENOMEM;
+return -1;
+}
+
+from_refs->flags = flags;
+from_refs->count = count;
+from_refs->domid = domid;
+
+memcpy(from_refs->refs, refs, count * sizeof(from_refs->refs[0]));
+
+if ( (rc = ioctl(xgt->fd, IOCTL_GNTDEV_DMABUF_EXP_FROM_REFS, from_refs)) )
+{
+GTERROR(xgt->logger, "ioctl DMABUF_EXP_FROM_REFS failed");
+goto out;
+}
+
+*dmabuf_fd = from_refs->fd;
+
+out:
+free(from_refs);
+return rc;
+}
+
+int osdep_gnttab_dmabuf_exp_wait_released(xengnttab_handle *xgt,
+  uint32_t fd, uint32_t wait_to_ms)
+{
+struct ioctl_gntdev_dmabuf_exp_wait_released wait;
+int rc;
+
+wait.fd = fd;
+wait.wait_to_ms = wait_to_ms;
+
+if ( (rc = ioctl(xgt->fd, IOCTL_GNTDEV_DMABUF_EXP_WAIT_RELEASED, )) ) 
{

{ should be on a new line.


+if ( errno == ENOENT ) {

Ditto.


+/* The buffer may have already been released. */
+errno = 0;
+rc = 0;
+} else
+GTERROR(xgt->logger, "ioctl DMABUF_EXP_WAIT_RELEASED failed");
+}
+
+return rc;
+}
+
+int osdep_gnttab_dmabuf_imp_to_refs(xengnttab_handle *xgt, uint32_t domid,
+uint32_t fd, uint32_t count, uint32_t 
*refs)
+{
+struct ioctl_gntdev_dmabuf_imp_to_refs *to_refs;
+int rc = 0;
+
+if ( !count )
+{
+errno = EINVAL;
+return -1;

Same comments on error handling apply to this function too.

The rest looks fine.

Thank you,
I'll fix the above and send v1

Wei.



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] libgnttab: Add support for Linux dma-buf

[Oleksandr Andrushchenko]

ping

On 07/23/2018 03:27 PM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

Add support for Linux grant device driver extension which allows
converting existing dma-buf's into an array of grant references
and vise versa. This is only implemented for Linux as other OSes
have no Linux dma-buf support.

Bump gnttab library minor version to 3.

Signed-off-by: Oleksandr Andrushchenko 
---
  tools/include/xen-sys/Linux/gntdev.h  | 106 
  tools/libs/gnttab/Makefile|   2 +-
  tools/libs/gnttab/gnttab_core.c   |  26 ++
  tools/libs/gnttab/gnttab_unimp.c  |  26 ++
  tools/libs/gnttab/include/xengnttab.h |  61 ++
  tools/libs/gnttab/libxengnttab.map|   8 ++
  tools/libs/gnttab/linux.c | 113 ++
  tools/libs/gnttab/minios.c|  26 ++
  tools/libs/gnttab/private.h   |  13 +++
  9 files changed, 380 insertions(+), 1 deletion(-)

diff --git a/tools/include/xen-sys/Linux/gntdev.h 
b/tools/include/xen-sys/Linux/gntdev.h
index 0ca07c92b21c..d16076044c71 100644
--- a/tools/include/xen-sys/Linux/gntdev.h
+++ b/tools/include/xen-sys/Linux/gntdev.h
@@ -4,6 +4,7 @@
   * Interface to /dev/xen/gntdev.
   *
   * Copyright (c) 2007, D G Murray
+ * Copyright (c) 2018, Oleksandr Andrushchenko, EPAM Systems Inc.
   *
   * This program is free software; you can redistribute it and/or
   * modify it under the terms of the GNU General Public License version 2
@@ -168,4 +169,109 @@ struct ioctl_gntdev_grant_copy {
  struct ioctl_gntdev_grant_copy_segment *segments;
  };
  
+/*

+ * Flags to be used while requesting memory mapping's backing storage
+ * to be allocated with DMA API.
+ */
+
+/*
+ * The buffer is backed with memory allocated with dma_alloc_wc.
+ */
+#define GNTDEV_DMA_FLAG_WC (1 << 0)
+
+/*
+ * The buffer is backed with memory allocated with dma_alloc_coherent.
+ */
+#define GNTDEV_DMA_FLAG_COHERENT   (1 << 1)
+
+/*
+ * Create a dma-buf [1] from grant references @refs of count @count provided
+ * by the foreign domain @domid with flags @flags.
+ *
+ * By default dma-buf is backed by system memory pages, but by providing
+ * one of the GNTDEV_DMA_FLAG_XXX flags it can also be created as
+ * a DMA write-combine or coherent buffer, e.g. allocated with dma_alloc_wc/
+ * dma_alloc_coherent.
+ *
+ * Returns 0 if dma-buf was successfully created and the corresponding
+ * dma-buf's file descriptor is returned in @fd.
+ *
+ * [1] 
https://elixir.bootlin.com/linux/latest/source/Documentation/driver-api/dma-buf.rst
+ */
+
+#define IOCTL_GNTDEV_DMABUF_EXP_FROM_REFS \
+_IOC(_IOC_NONE, 'G', 9, \
+ sizeof(struct ioctl_gntdev_dmabuf_exp_from_refs))
+struct ioctl_gntdev_dmabuf_exp_from_refs {
+/* IN parameters. */
+/* Specific options for this dma-buf: see GNTDEV_DMABUF_FLAG_XXX. */
+uint32_t flags;
+/* Number of grant references in @refs array. */
+uint32_t count;
+/* OUT parameters. */
+/* File descriptor of the dma-buf. */
+uint32_t fd;
+/* The domain ID of the grant references to be mapped. */
+uint32_t domid;
+/* Variable IN parameter. */
+/* Array of grant references of size @count. */
+uint32_t refs[1];
+};
+
+/*
+ * This will block until the dma-buf with the file descriptor @fd is
+ * released. This is only valid for buffers created with
+ * IOCTL_GNTDEV_DMABUF_EXP_FROM_REFS.
+ *
+ * If withing @wait_to_ms milliseconds the buffer is not released
+ * then -ETIMEDOUT error is returned.
+ * If the buffer with file descriptor @fd does not exist or has already
+ * been released, then -ENOENT is returned. For valid file descriptors
+ * this must not be treated as error.
+ */
+#define IOCTL_GNTDEV_DMABUF_EXP_WAIT_RELEASED \
+_IOC(_IOC_NONE, 'G', 10, \
+ sizeof(struct ioctl_gntdev_dmabuf_exp_wait_released))
+struct ioctl_gntdev_dmabuf_exp_wait_released {
+/* IN parameters */
+uint32_t fd;
+uint32_t wait_to_ms;
+};
+
+/*
+ * Import a dma-buf with file descriptor @fd and export granted references
+ * to the pages of that dma-buf into array @refs of size @count.
+ */
+#define IOCTL_GNTDEV_DMABUF_IMP_TO_REFS \
+_IOC(_IOC_NONE, 'G', 11, \
+ sizeof(struct ioctl_gntdev_dmabuf_imp_to_refs))
+struct ioctl_gntdev_dmabuf_imp_to_refs {
+/* IN parameters. */
+/* File descriptor of the dma-buf. */
+uint32_t fd;
+/* Number of grant references in @refs array. */
+uint32_t count;
+/* The domain ID for which references to be granted. */
+uint32_t domid;
+/* Reserved - must be zero. */
+uint32_t reserved;
+/* OUT parameters. */
+/* Array of grant references of size @count. */
+uint32_t refs[1];
+};
+
+/*
+ * This will close all references to an imported buffer, so it can be
+ * released by the owner. This is only valid for buffers created with
+ * IOCTL_GNTDEV_DMABUF_IMP_TO_REFS.
+ */
+#define IOCTL_GNTDEV_DMABUF_IMP_RELEASE \
+_IOC(_IOC_N

[Xen-devel] [PATCH 0/1] cameraif: Add ABI for para-virtualized

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Hello!

At the moment Xen [1] already supports some virtual multimedia
features [2] such as virtual display, sound. It supports keyboards,
pointers and multi-touch devices all allowing Xen to be used in
automotive appliances, In-Vehicle Infotainment (IVI) systems
and many more.

This work adds a new Xen para-virtualized protocol for a virtual
camera device which extends multimedia capabilities of Xen even
farther: video conferencing, IVI, high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
 - pixel formats
 - resolutions
 - frame rates
2. Support basic camera controls:
 - contrast
 - brightness
 - hue
 - saturation
3. Support streaming control
4. Support zero-copying use-cases

I hope that Xen and V4L and other communities could give their
valuable feedback on this work, so I can update the protocol
to better fit any additional requirements I might have missed.

Thank you,
Oleksandr Andrushchenko

[1] https://www.xenproject.org/
[2] https://xenbits.xen.org/gitweb/?p=xen.git;a=tree;f=xen/include/public/io

Oleksandr Andrushchenko (1):
  cameraif: add ABI for para-virtual camera

 xen/include/public/io/cameraif.h | 981 +++
 1 file changed, 981 insertions(+)
 create mode 100644 xen/include/public/io/cameraif.h

-- 
2.18.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 1/1] cameraif: add ABI for para-virtual camera

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
 - pixel formats
 - resolutions
 - frame rates
2. Support basic camera controls:
 - contrast
 - brightness
 - hue
 - saturation
3. Support streaming control
4. Support zero-copying use-cases

Signed-off-by: Oleksandr Andrushchenko 
---
 xen/include/public/io/cameraif.h | 981 +++
 1 file changed, 981 insertions(+)
 create mode 100644 xen/include/public/io/cameraif.h

diff --git a/xen/include/public/io/cameraif.h b/xen/include/public/io/cameraif.h
new file mode 100644
index ..bdc6a1262fcf
--- /dev/null
+++ b/xen/include/public/io/cameraif.h
@@ -0,0 +1,981 @@
+/**
+ * cameraif.h
+ *
+ * Unified camera device I/O interface for Xen guest OSes.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright (C) 2018 EPAM Systems Inc.
+ *
+ * Author: Oleksandr Andrushchenko 
+ */
+
+#ifndef __XEN_PUBLIC_IO_CAMERAIF_H__
+#define __XEN_PUBLIC_IO_CAMERAIF_H__
+
+#include "ring.h"
+#include "../grant_table.h"
+
+/*
+ **
+ *   Protocol version
+ **
+ */
+#define XENCAMERA_PROTOCOL_VERSION "1"
+
+/*
+ **
+ *  Feature and Parameter Negotiation
+ **
+ *
+ * Front->back notifications: when enqueuing a new request, sending a
+ * notification can be made conditional on xencamera_req (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Backends must set
+ * xencamera_req appropriately (e.g., using RING_FINAL_CHECK_FOR_REQUESTS()).
+ *
+ * Back->front notifications: when enqueuing a new response, sending a
+ * notification can be made conditional on xencamera_resp (i.e., the generic
+ * hold-off mechanism provided by the ring macros). Frontends must set
+ * xencamera_resp appropriately (e.g., using RING_FINAL_CHECK_FOR_RESPONSES()).
+ *
+ * The two halves of a para-virtual camera driver utilize nodes within
+ * XenStore to communicate capabilities and to negotiate operating parameters.
+ * This section enumerates these nodes which reside in the respective front and
+ * backend portions of XenStore, following the XenBus convention.
+ *
+ * All data in XenStore is stored as strings. Nodes specifying numeric
+ * values are encoded in decimal. Integer value ranges listed below are
+ * expressed as fixed sized integer types capable of storing the conversion
+ * of a properly formatted node string, without loss of information.
+ *
+ **
+ *Example configuration
+ **
+ *
+ * This is an example of backend and frontend configuration:
+ *
+ *- Backend ---
+ *
+ * /local/domain/0/backend/vcamera/1/0/frontend-id = "1"
+ * /local/domain/0/backend/vcamera/1/0/frontend = 
"/local/domain/1/device/vcamera/0"
+ * /local/domain/0/backend/vcamera/1/0/state = "4"
+ * /local/domain/0/backend/vcamera/1/0/versions = "1,2"
+ *
+ *- Frontend --

Re: [Xen-devel] linux-next: build failure after merge of the xen-tip tree

[Oleksandr Andrushchenko]

From: Stephen Rothwell 
Date: Mon, 30 Jul 2018 15:08:21 +1000
Subject: [PATCH] xen/gntdev: fix up for attach callback API change

Signed-off-by: Stephen Rothwell 

Thank you for your patch,
Reviewed-by: Oleksandr Andrushchenko 


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] libgnttab: Add support for Linux dma-buf

[Oleksandr Andrushchenko]

On 07/25/2018 12:39 PM, Wei Liu wrote:

On Mon, Jul 23, 2018 at 03:27:25PM +0300, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

Add support for Linux grant device driver extension which allows
converting existing dma-buf's into an array of grant references
and vise versa. This is only implemented for Linux as other OSes
have no Linux dma-buf support.

Bump gnttab library minor version to 3.

Signed-off-by: Oleksandr Andrushchenko 

Has the Linux side been merged yet?

It is now merged [1] in Xen tip kernel tree

Wei.
[1] 
https://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git/log/?h=linux-next


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 10/10] ALSA: xen: Use standard pcm_format_to_bits() for ALSA format bits

[Oleksandr Andrushchenko]

Hello, Takashi!

Date: Wed, 25 Jul 2018 23:19:45 +0200
From: Takashi Iwai 
To: alsa-de...@alsa-project.org
Subject: [alsa-devel] [PATCH 10/10] ALSA: xen: Use standard
pcm_format_to_bits() for ALSA format bits
Message-ID: <20180725211945.19222-11-ti...@suse.de>

The open codes with the bit shift in xen_snd_front_alsa.c give sparse
warnings as the PCM format type is with __bitwise.
There is already a standard macro to get the format bits, so let's use
it instead.

This fixes sparse warnings like:
   sound/xen/xen_snd_front_alsa.c:191:47: warning: restricted snd_pcm_format_t 
degrades to integer

Signed-off-by: Takashi Iwai 
---
  sound/xen/xen_snd_front_alsa.c | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/sound/xen/xen_snd_front_alsa.c b/sound/xen/xen_snd_front_alsa.c
index 5a2bd70a2fa1..129180e17db1 100644
--- a/sound/xen/xen_snd_front_alsa.c
+++ b/sound/xen/xen_snd_front_alsa.c
@@ -188,7 +188,7 @@ static u64 to_sndif_formats_mask(u64 alsa_formats)
  
  	mask = 0;

for (i = 0; i < ARRAY_SIZE(ALSA_SNDIF_FORMATS); i++)
-   if (1 << ALSA_SNDIF_FORMATS[i].alsa & alsa_formats)
+   if (pcm_format_to_bits(ALSA_SNDIF_FORMATS[i].alsa) & 
alsa_formats)
mask |= 1 << ALSA_SNDIF_FORMATS[i].sndif;
  
  	return mask;

@@ -202,7 +202,7 @@ static u64 to_alsa_formats_mask(u64 sndif_formats)
mask = 0;
for (i = 0; i < ARRAY_SIZE(ALSA_SNDIF_FORMATS); i++)
if (1 << ALSA_SNDIF_FORMATS[i].sndif & sndif_formats)
-   mask |= 1 << ALSA_SNDIF_FORMATS[i].alsa;
+   mask |= pcm_format_to_bits(ALSA_SNDIF_FORMATS[i].alsa);
  
  	return mask;

  }

Thank you for your patch,
Reviewed-by: Oleksandr Andrushchenko 

Thank you,
Oleksandr

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] libgnttab: Add support for Linux dma-buf

[Oleksandr Andrushchenko]

On 07/25/2018 12:53 PM, Wei Liu wrote:

On Wed, Jul 25, 2018 at 12:49:29PM +0300, Oleksandr Andrushchenko wrote:

On 07/25/2018 12:39 PM, Wei Liu wrote:

On Mon, Jul 23, 2018 at 03:27:25PM +0300, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

Add support for Linux grant device driver extension which allows
converting existing dma-buf's into an array of grant references
and vise versa. This is only implemented for Linux as other OSes
have no Linux dma-buf support.

Bump gnttab library minor version to 3.

Signed-off-by: Oleksandr Andrushchenko 

Has the Linux side been merged yet?

It is going to be merged into Boris' staging for now
and then eventually move into Xen tip's for-linus (Boris can
tell more on that: Linux is already at rc6, so I think this
can happen in the nearest future).


OK. So the interfaces on Linux's side are agreed.

Yes



Given that Xen and Linux parts are actually independent,
I decided to start upstreaming libgnttab changes now, allowing
me to mature the code and address comments

 From my PoV this patch provides a bunch of wrappers around Linux ioctls
so I would like to be sure the underlying interfaces are agreed before
reviewing.

Fair enough,
Boris could you please estimate when the corresponding Linux
changes have a chance to get into Xen tip Linux tree?

Wei.

Thank you,
Oleksandr

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] libgnttab: Add support for Linux dma-buf

[Oleksandr Andrushchenko]

On 07/25/2018 12:39 PM, Wei Liu wrote:

On Mon, Jul 23, 2018 at 03:27:25PM +0300, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

Add support for Linux grant device driver extension which allows
converting existing dma-buf's into an array of grant references
and vise versa. This is only implemented for Linux as other OSes
have no Linux dma-buf support.

Bump gnttab library minor version to 3.

Signed-off-by: Oleksandr Andrushchenko 

Has the Linux side been merged yet?

It is going to be merged into Boris' staging for now
and then eventually move into Xen tip's for-linus (Boris can
tell more on that: Linux is already at rc6, so I think this
can happen in the nearest future).

Given that Xen and Linux parts are actually independent,
I decided to start upstreaming libgnttab changes now, allowing
me to mature the code and address comments

Wei.

Thank you,
Oleksandr

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel