[PATCH 04/10] [WORKAROUND] xen/arm: Update hwdom's p2m to trap ECAM space

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Host bridge controller's ECAM space is mapped into Domain-0's p2m,
thus it is not possible to trap the same for vPCI via MMIO handlers.
For this to work we need to unmap those mappings in p2m.

TODO (Julien): It would be best if we avoid the map/unmap operation.
So, maybe we want to introduce another way to avoid the mapping.
Maybe by changing the type of the controller to "PCI_HOSTCONTROLLER"
and checking if this is a PCI hostcontroller avoid the mapping.

Signed-off-by: Oleksandr Andrushchenko 
---
 xen/arch/arm/domain_build.c | 10 +-
 xen/arch/arm/pci/pci-host-common.c  | 15 +++
 xen/arch/arm/pci/pci-host-generic.c | 28 
 xen/include/asm-arm/pci.h   |  2 ++
 4 files changed, 54 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 1f83f9048146..3f696d2a6672 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2566,7 +2566,15 @@ int __init construct_dom0(struct domain *d)
 if ( rc < 0 )
 return rc;
 
-return construct_domain(d, );
+rc = construct_domain(d, );
+if ( rc < 0 )
+return rc;
+
+#ifdef CONFIG_HAS_PCI
+if ( has_vpci(d) )
+rc = pci_host_bridge_update_mappings(d);
+#endif
+return rc;
 }
 
 /*
diff --git a/xen/arch/arm/pci/pci-host-common.c 
b/xen/arch/arm/pci/pci-host-common.c
index b81184d34980..b6c4d7b636b1 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -235,6 +235,21 @@ int pci_host_iterate_bridges(struct domain *d,
 }
 return 0;
 }
+
+static int pci_host_bridge_update_mapping(struct domain *d,
+  struct pci_host_bridge *bridge)
+{
+if ( !bridge->ops->update_mappings )
+return 0;
+
+return bridge->ops->update_mappings(d, bridge);
+}
+
+int pci_host_bridge_update_mappings(struct domain *d)
+{
+return pci_host_iterate_bridges(d, pci_host_bridge_update_mapping);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/pci/pci-host-generic.c 
b/xen/arch/arm/pci/pci-host-generic.c
index 469df3da0116..772c53c881bc 100644
--- a/xen/arch/arm/pci/pci-host-generic.c
+++ b/xen/arch/arm/pci/pci-host-generic.c
@@ -21,6 +21,8 @@
 #include 
 #include 
 #include 
+#include 
+#include 
 #include 
 
 /*
@@ -85,6 +87,31 @@ int pci_ecam_config_read(struct pci_host_bridge *bridge, 
uint32_t sbdf,
 return 0;
 }
 
+/*
+ * TODO: This is called late on domain creation to mangle p2m if needed:
+ * for ECAM host controller for mmio region traps to work for Domain-0
+ * we need to unmap those mappings in p2m.
+ * This is WIP:
+ * julieng: I think it would be best if we avoid the map/unmap operation.
+ * So maybe we want to introduce another way to avoid the mapping.
+ * Maybe by changing the type of the controller to "PCI_HOSTCONTROLLER"
+ * and check if this is a PCI hostcontroller avoid the mapping.
+ */
+static int pci_ecam_update_mappings(struct domain *d,
+struct pci_host_bridge *bridge)
+{
+struct pci_config_window *cfg = bridge->sysdata;
+int ret;
+
+/* Only for control domain which owns this PCI host bridge. */
+if ( !is_control_domain(d) )
+return 0;
+
+ret = unmap_regions_p2mt(d, gaddr_to_gfn(cfg->phys_addr),
+ cfg->size >> PAGE_SHIFT, INVALID_MFN);
+return ret;
+}
+
 static int pci_ecam_register_mmio_handler(struct domain *d,
   struct pci_host_bridge *bridge,
   const struct mmio_handler_ops *ops)
@@ -101,6 +128,7 @@ struct pci_ecam_ops pci_generic_ecam_ops = {
 .pci_ops= {
 .read  = pci_ecam_config_read,
 .write = pci_ecam_config_write,
+.update_mappings   = pci_ecam_update_mappings,
 .register_mmio_handler = pci_ecam_register_mmio_handler,
 }
 };
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index e3a02429b8d4..d94e8a6628de 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -65,6 +65,7 @@ struct pci_ops {
 uint32_t sbdf, int where, int size, u32 *val);
 int (*write)(struct pci_host_bridge *bridge,
 uint32_t sbdf, int where, int size, u32 val);
+int (*update_mappings)(struct domain *d, struct pci_host_bridge *bridge);
 int (*register_mmio_handler)(struct domain *d,
  struct pci_host_bridge *bridge,
  const struct mmio_handler_ops *ops);
@@ -108,6 +109,7 @@ bool dt_pci_parse_bus_range(struct dt_device_node *dev,
 int pci_host_iterate_bridges(struct domain *d,
  int (*clb)(struct domain *d,
 struct pci_host_bridge *bridge));
+int pci_

[PATCH 05/10] xen/arm: Process pending vPCI map/unmap operations

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

vPCI may map and unmap PCI device memory (BARs) being passed through which
may take a lot of time. For this those operations may be deferred to be
performed later, so that they can be safely preempted.
Run the corresponding vPCI code while switching a vCPU.

Signed-off-by: Oleksandr Andrushchenko 
---
 xen/arch/arm/traps.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 8f40d0e0b6b1..1c54dc0cdd51 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -33,6 +33,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -2253,6 +2254,11 @@ static void check_for_vcpu_work(void)
 {
 struct vcpu *v = current;
 
+local_irq_enable();
+if ( has_vpci(v->domain) && vpci_process_pending(v) )
+raise_softirq(SCHEDULE_SOFTIRQ);
+local_irq_disable();
+
 if ( likely(!v->arch.need_flush_to_ram) )
 return;
 
-- 
2.17.1

[PATCH 09/10] vpci/rcar: Implement vPCI.update_bar_header callback

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Update hardware domain's BAR header as R-Car Gen3 is a non-ECAM host
controller, so vPCI MMIO handlers do not work for it in hwdom.

Signed-off-by: Oleksandr Andrushchenko 
---
 xen/arch/arm/pci/pci-host-rcar-gen3.c | 69 +++
 1 file changed, 69 insertions(+)

diff --git a/xen/arch/arm/pci/pci-host-rcar-gen3.c 
b/xen/arch/arm/pci/pci-host-rcar-gen3.c
index ec14bb29a38b..353ac2bfd6e6 100644
--- a/xen/arch/arm/pci/pci-host-rcar-gen3.c
+++ b/xen/arch/arm/pci/pci-host-rcar-gen3.c
@@ -23,6 +23,7 @@
 #include 
 #include 
 #include 
+#include 
 
 /* Error values that may be returned by PCI functions */
 #define PCIBIOS_SUCCESSFUL 0x00
@@ -307,12 +308,80 @@ int pci_rcar_gen3_config_write(struct pci_host_bridge 
*bridge, uint32_t _sbdf,
 return ret;
 }
 
+static void pci_rcar_gen3_hwbar_init(const struct pci_dev *pdev,
+ struct vpci_header *header)
+
+{
+static bool once = true;
+struct vpci_bar *bars = header->bars;
+unsigned int num_bars;
+int i;
+
+/* Run only once. */
+if (!once)
+return;
+once = false;
+
+printk("\n\n  %s ---\n", __func__);
+switch ( pci_conf_read8(pdev->sbdf, PCI_HEADER_TYPE) & 0x7f )
+{
+case PCI_HEADER_TYPE_NORMAL:
+num_bars = PCI_HEADER_NORMAL_NR_BARS;
+break;
+
+case PCI_HEADER_TYPE_BRIDGE:
+num_bars = PCI_HEADER_BRIDGE_NR_BARS;
+break;
+
+default:
+return;
+}
+
+for ( i = 0; i < num_bars; i++ )
+{
+uint8_t reg = PCI_BASE_ADDRESS_0 + i * 4;
+
+if ( bars[i].type == VPCI_BAR_MEM64_HI )
+{
+/*
+ * Skip hi part of the 64-bit register: it is read
+ * together with the lower part.
+ */
+continue;
+}
+
+if ( bars[i].type == VPCI_BAR_IO )
+{
+/* Skip IO. */
+continue;
+}
+
+if ( bars[i].type == VPCI_BAR_MEM64_LO )
+{
+/* Read both hi and lo parts of the 64-bit BAR. */
+bars[i].addr =
+(uint64_t)pci_conf_read32(pdev->sbdf, reg + 4) << 32 |
+pci_conf_read32(pdev->sbdf, reg);
+}
+else if ( bars[i].type == VPCI_BAR_MEM32 )
+{
+bars[i].addr = pci_conf_read32(pdev->sbdf, reg);
+}
+else
+{
+/* Expansion ROM? */
+continue;
+}
+}
+}
+
 /* R-Car Gen3 ops */
 static struct pci_ecam_ops pci_rcar_gen3_ops = {
 .bus_shift  = 20, /* FIXME: this is not used by RCar */
 .pci_ops= {
 .read   = pci_rcar_gen3_config_read,
 .write  = pci_rcar_gen3_config_write,
+.update_bar_header = pci_rcar_gen3_hwbar_init,
 }
 };
 
-- 
2.17.1

[PATCH 06/10] vpci: Make every domain handle its own BARs

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

At the moment there is an identity mapping between how a guest sees its
BARs and how they are programmed into guest domain's p2m. This is not
going to work as guest domains have their own view on the BARs.
Extend existing vPCI BAR handling to allow every domain to have its own
view of the BARs: only hardware domain sees physical memory addresses in
this case and for the rest those are emulated, including logic required
for the guests to detect memory sizes and properties.

While emulating BAR access for the guests create a link between the
virtual BAR address and physical one: use full memory address while
creating range sets used to map/unmap corresponding address spaces and
exploit the fact that PCI BAR value doesn't use 8 lower bits of the
memory address. Use those bits to pass physical BAR's index, so we can
build/remove proper p2m mappings.

Signed-off-by: Oleksandr Andrushchenko 
---
 xen/drivers/vpci/header.c | 276 ++
 xen/drivers/vpci/vpci.c   |   1 +
 xen/include/xen/vpci.h|  24 ++--
 3 files changed, 265 insertions(+), 36 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index f74f728884c0..7dc7c70e24f2 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -31,14 +31,87 @@
 struct map_data {
 struct domain *d;
 bool map;
+struct pci_dev *pdev;
 };
 
+static struct vpci_header *get_vpci_header(struct domain *d,
+   const struct pci_dev *pdev);
+
+static struct vpci_header *get_hwdom_vpci_header(const struct pci_dev *pdev)
+{
+if ( unlikely(list_empty(>vpci->headers)) )
+return get_vpci_header(hardware_domain, pdev);
+
+/* hwdom's header is always the very first entry. */
+return list_first_entry(>vpci->headers, struct vpci_header, node);
+}
+
+static struct vpci_header *get_vpci_header(struct domain *d,
+   const struct pci_dev *pdev)
+{
+struct list_head *prev;
+struct vpci_header *header;
+struct vpci *vpci = pdev->vpci;
+
+list_for_each( prev, >headers )
+{
+struct vpci_header *this = list_entry(prev, struct vpci_header, node);
+
+if ( this->domain_id == d->domain_id )
+return this;
+}
+printk(XENLOG_DEBUG "--" \
+   "Adding new vPCI BAR headers for domain %d: " PRI_pci" \n",
+   d->domain_id, pdev->sbdf.seg, pdev->sbdf.bus,
+   pdev->sbdf.dev, pdev->sbdf.fn);
+header = xzalloc(struct vpci_header);
+if ( !header )
+{
+printk(XENLOG_ERR
+   "Failed to add new vPCI BAR headers for domain %d: " PRI_pci" 
\n",
+   d->domain_id, pdev->sbdf.seg, pdev->sbdf.bus,
+   pdev->sbdf.dev, pdev->sbdf.fn);
+return NULL;
+}
+
+if ( !is_hardware_domain(d) )
+{
+struct vpci_header *hwdom_header = get_hwdom_vpci_header(pdev);
+
+/* Make a copy of the hwdom's BARs as the initial state for vBARs. */
+memcpy(header, hwdom_header, sizeof(*header));
+}
+
+header->domain_id = d->domain_id;
+list_add_tail(>node, >headers);
+return header;
+}
+
+static struct vpci_bar *get_vpci_bar(struct domain *d,
+ const struct pci_dev *pdev,
+ int bar_idx)
+{
+struct vpci_header *vheader;
+
+vheader = get_vpci_header(d, pdev);
+if ( !vheader )
+return NULL;
+
+return >bars[bar_idx];
+}
+
 static int map_range(unsigned long s, unsigned long e, void *data,
  unsigned long *c)
 {
 const struct map_data *map = data;
-int rc;
-
+unsigned long mfn;
+int rc, bar_idx;
+struct vpci_header *header = get_hwdom_vpci_header(map->pdev);
+
+bar_idx = s & ~PCI_BASE_ADDRESS_MEM_MASK;
+s = PFN_DOWN(s);
+e = PFN_DOWN(e);
+mfn = _mfn(PFN_DOWN(header->bars[bar_idx].addr));
 for ( ; ; )
 {
 unsigned long size = e - s + 1;
@@ -52,11 +125,15 @@ static int map_range(unsigned long s, unsigned long e, 
void *data,
  * - {un}map_mmio_regions doesn't support preemption.
  */
 
-rc = map->map ? map_mmio_regions(map->d, _gfn(s), size, _mfn(s))
-  : unmap_mmio_regions(map->d, _gfn(s), size, _mfn(s));
+rc = map->map ? map_mmio_regions(map->d, _gfn(s), size, mfn)
+  : unmap_mmio_regions(map->d, _gfn(s), size, mfn);
 if ( rc == 0 )
 {
-*c += size;
+/*
+ * Range set is not expressed in frame numbers and the size
+ * is the number of frames, so update accordingly.
+ */
+*c += size << PAGE_SHIFT;
 break;
 }
 if (

[PATCH 07/10] xen/arm: Do not hardcode phycial PCI device addresses

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

As vPCI now takes care of the proper p2m mappings for PCI devices there
is no more need to hardcode anything.

Signed-off-by: Oleksandr Andrushchenko 
---
 xen/include/public/arch-arm.h | 9 +
 1 file changed, 1 insertion(+), 8 deletions(-)

diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 2411ac9f7b0a..59baf1014fe3 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -444,15 +444,8 @@ typedef uint64_t xen_callback_t;
 #define GUEST_VPCI_MEM_CPU_ADDR   xen_mk_ullong(0x0402)
 #define GUEST_VPCI_IO_CPU_ADDRxen_mk_ullong(0xC0200800)
 
-/*
- * This is hardcoded values for the real PCI physical addresses.
- * This will be removed once we read the real PCI-PCIe physical
- * addresses form the config space and map to the guest memory map
- * when assigning the device to guest via VPCI.
- *
- */
 #define GUEST_VPCI_PREFETCH_MEM_PCI_ADDR  xen_mk_ullong(0x40)
-#define GUEST_VPCI_MEM_PCI_ADDR   xen_mk_ullong(0x5000)
+#define GUEST_VPCI_MEM_PCI_ADDR   xen_mk_ullong(0x0402)
 #define GUEST_VPCI_IO_PCI_ADDRxen_mk_ullong(0x)
 
 #define GUEST_VPCI_PREFETCH_MEM_SIZE  xen_mk_ullong(0x1)
-- 
2.17.1

[PATCH 03/10] xen/arm: Setup MMIO range trap handlers for hardware domain

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

In order vPCI to work it needs all access to PCI configuration space
access to be synchronized among all entities, e.g. hardware domain and
guests. For that implement PCI host bridge specific callbacks to
propelry setup those ranges depending on host bridge implementation.

This callback is optional and may not be used by non-ECAM host bridges.

Signed-off-by: Oleksandr Andrushchenko 
---
 xen/arch/arm/pci/pci-host-common.c  | 16 
 xen/arch/arm/pci/pci-host-generic.c | 15 +--
 xen/arch/arm/vpci.c | 16 +++-
 xen/include/asm-arm/pci.h   |  7 +++
 4 files changed, 51 insertions(+), 3 deletions(-)

diff --git a/xen/arch/arm/pci/pci-host-common.c 
b/xen/arch/arm/pci/pci-host-common.c
index b011c7eff3c8..b81184d34980 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -219,6 +219,22 @@ struct device *pci_find_host_bridge_device(struct device 
*dev)
 }
 return dt_to_dev(bridge->dt_node);
 }
+
+int pci_host_iterate_bridges(struct domain *d,
+ int (*clb)(struct domain *d,
+struct pci_host_bridge *bridge))
+{
+struct pci_host_bridge *bridge;
+int err;
+
+list_for_each_entry( bridge, _host_bridges, node )
+{
+err = clb(d, bridge);
+if ( err )
+return err;
+}
+return 0;
+}
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/pci/pci-host-generic.c 
b/xen/arch/arm/pci/pci-host-generic.c
index 54dd123e95c7..469df3da0116 100644
--- a/xen/arch/arm/pci/pci-host-generic.c
+++ b/xen/arch/arm/pci/pci-host-generic.c
@@ -85,12 +85,23 @@ int pci_ecam_config_read(struct pci_host_bridge *bridge, 
uint32_t sbdf,
 return 0;
 }
 
+static int pci_ecam_register_mmio_handler(struct domain *d,
+  struct pci_host_bridge *bridge,
+  const struct mmio_handler_ops *ops)
+{
+struct pci_config_window *cfg = bridge->sysdata;
+
+register_mmio_handler(d, ops, cfg->phys_addr, cfg->size, NULL);
+return 0;
+}
+
 /* ECAM ops */
 struct pci_ecam_ops pci_generic_ecam_ops = {
 .bus_shift  = 20,
 .pci_ops= {
-.read   = pci_ecam_config_read,
-.write  = pci_ecam_config_write,
+.read  = pci_ecam_config_read,
+.write = pci_ecam_config_write,
+.register_mmio_handler = pci_ecam_register_mmio_handler,
 }
 };
 
diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index 49e473ab0d10..2b9bf34c8fe6 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -80,11 +80,25 @@ static const struct mmio_handler_ops vpci_mmio_handler = {
 .write = vpci_mmio_write,
 };
 
+static int vpci_setup_mmio_handler(struct domain *d,
+   struct pci_host_bridge *bridge)
+{
+if ( bridge->ops->register_mmio_handler )
+return bridge->ops->register_mmio_handler(d, bridge,
+  _mmio_handler);
+return 0;
+}
+
+
 int domain_vpci_init(struct domain *d)
 {
-if ( !has_vpci(d) || is_hardware_domain(d) )
+if ( !has_vpci(d) )
 return 0;
 
+if ( is_hardware_domain(d) )
+return pci_host_iterate_bridges(d, vpci_setup_mmio_handler);
+
+/* Guest domains use what is programmed in their device tree. */
 register_mmio_handler(d, _mmio_handler,
 GUEST_VPCI_ECAM_BASE,GUEST_VPCI_ECAM_SIZE,NULL);
 
diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h
index ba23178f67ab..e3a02429b8d4 100644
--- a/xen/include/asm-arm/pci.h
+++ b/xen/include/asm-arm/pci.h
@@ -27,6 +27,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #ifdef CONFIG_ARM_PCI
 
@@ -64,6 +65,9 @@ struct pci_ops {
 uint32_t sbdf, int where, int size, u32 *val);
 int (*write)(struct pci_host_bridge *bridge,
 uint32_t sbdf, int where, int size, u32 val);
+int (*register_mmio_handler)(struct domain *d,
+ struct pci_host_bridge *bridge,
+ const struct mmio_handler_ops *ops);
 };
 
 /*
@@ -101,6 +105,9 @@ void pci_init(void);
 bool dt_pci_parse_bus_range(struct dt_device_node *dev,
 struct pci_config_window *cfg);
 
+int pci_host_iterate_bridges(struct domain *d,
+ int (*clb)(struct domain *d,
+struct pci_host_bridge *bridge));
 #else   /*!CONFIG_ARM_PCI*/
 struct arch_pci_dev { };
 static inline void  pci_init(void) { }
-- 
2.17.1

[PATCH 02/10] arm/pci: Maintain PCI assignable list

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

The original code depends on pciback to manage assignable device list.
The functionality which is implemented by the pciback and the toolstack
and which is relevant/missing/needed for ARM:

1. pciback is used as a database for assignable PCI devices, e.g. xl
   pci-assignable-{add|remove|list} manipulates that list. So, whenever the
   toolstack needs to know which PCI devices can be passed through it reads
   that from the relevant sysfs entries of the pciback.

2. pciback is used to hold the unbound PCI devices, e.g. when passing through
   a PCI device it needs to be unbound from the relevant device driver and bound
   to pciback (strictly speaking it is not required that the device is bound to
   pciback, but pciback is again used as a database of the passed through PCI
   devices, so we can re-bind the devices back to their original drivers when
   guest domain shuts down)

1. As ARM doesn't use pciback implement the above with additional sysctls:
 - XEN_SYSCTL_pci_device_set_assigned
 - XEN_SYSCTL_pci_device_get_assigned
 - XEN_SYSCTL_pci_device_enum_assigned
2. Extend struct pci_dev to hold assignment state.

Signed-off-by: Oleksandr Andrushchenko 
---
 tools/libxc/include/xenctrl.h |   9 +++
 tools/libxc/xc_domain.c   |   1 +
 tools/libxc/xc_misc.c |  46 +++
 tools/libxl/Makefile  |   4 ++
 tools/libxl/libxl_pci.c   | 105 --
 xen/arch/arm/sysctl.c |  66 -
 xen/drivers/passthrough/pci.c |  93 ++
 xen/include/public/sysctl.h   |  40 +
 xen/include/xen/pci.h |  12 
 9 files changed, 370 insertions(+), 6 deletions(-)

diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 4c89b7294c4f..77029013da7d 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2652,6 +2652,15 @@ int xc_livepatch_replace(xc_interface *xch, char *name, 
uint32_t timeout, uint32
 int xc_domain_cacheflush(xc_interface *xch, uint32_t domid,
  xen_pfn_t start_pfn, xen_pfn_t nr_pfns);
 
+typedef xen_sysctl_pci_device_enum_assigned_t xc_pci_device_enum_assigned_t;
+
+int xc_pci_device_set_assigned(xc_interface *xch, uint32_t machine_sbdf,
+   bool assigned);
+int xc_pci_device_get_assigned(xc_interface *xch, uint32_t machine_sbdf);
+
+int xc_pci_device_enum_assigned(xc_interface *xch,
+xc_pci_device_enum_assigned_t *e);
+
 /* Compat shims */
 #include "xenctrl_compat.h"
 
diff --git a/tools/libxc/xc_domain.c b/tools/libxc/xc_domain.c
index 71829c2bce3e..d515191e9243 100644
--- a/tools/libxc/xc_domain.c
+++ b/tools/libxc/xc_domain.c
@@ -2321,6 +2321,7 @@ int xc_domain_soft_reset(xc_interface *xch,
 domctl.domain = domid;
 return do_domctl(xch, );
 }
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libxc/xc_misc.c b/tools/libxc/xc_misc.c
index 3820394413a9..d439c4ba1019 100644
--- a/tools/libxc/xc_misc.c
+++ b/tools/libxc/xc_misc.c
@@ -988,6 +988,52 @@ int xc_livepatch_replace(xc_interface *xch, char *name, 
uint32_t timeout, uint32
 return _xc_livepatch_action(xch, name, LIVEPATCH_ACTION_REPLACE, timeout, 
flags);
 }
 
+int xc_pci_device_set_assigned(
+xc_interface *xch,
+uint32_t machine_sbdf,
+bool assigned)
+{
+DECLARE_SYSCTL;
+
+sysctl.cmd = XEN_SYSCTL_pci_device_set_assigned;
+sysctl.u.pci_set_assigned.machine_sbdf = machine_sbdf;
+sysctl.u.pci_set_assigned.assigned = assigned;
+
+return do_sysctl(xch, );
+}
+
+int xc_pci_device_get_assigned(
+xc_interface *xch,
+uint32_t machine_sbdf)
+{
+DECLARE_SYSCTL;
+
+sysctl.cmd = XEN_SYSCTL_pci_device_get_assigned;
+sysctl.u.pci_get_assigned.machine_sbdf = machine_sbdf;
+
+return do_sysctl(xch, );
+}
+
+int xc_pci_device_enum_assigned(xc_interface *xch,
+xc_pci_device_enum_assigned_t *e)
+{
+int ret;
+DECLARE_SYSCTL;
+
+sysctl.cmd = XEN_SYSCTL_pci_device_enum_assigned;
+sysctl.u.pci_enum_assigned.idx = e->idx;
+sysctl.u.pci_enum_assigned.report_not_assigned = e->report_not_assigned;
+ret = do_sysctl(xch, );
+if ( ret )
+errno = EINVAL;
+else
+{
+e->domain = sysctl.u.pci_enum_assigned.domain;
+e->machine_sbdf = sysctl.u.pci_enum_assigned.machine_sbdf;
+}
+return ret;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
index f3806aafcb4e..6f76ba35aec7 100644
--- a/tools/libxl/Makefile
+++ b/tools/libxl/Makefile
@@ -130,6 +130,10 @@ endif
 
 LIBXL_LIBS += -lyajl
 
+ifeq ($(CONFIG_X86),y)
+CFALGS += -DCONFIG_PCIBACK
+endif
+
 ifeq ($(CONFIG_ARM),y)
 CFALGS += -DCONFIG_ARM
 endif
diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c
index b93cf976642b..41f89b8aae10 100644
--- a/tools/libxl/libxl_pci.c
+++ b/tools/libxl/libxl_

[PATCH 01/10] pci/pvh: Allow PCI toolstack code run with PVH domains on ARM

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

According to https://wiki.xenproject.org/wiki/Linux_PVH:

Items not supported by PVH
 - PCI pass through (as of Xen 4.10)

Allow running PCI remove code on ARM and do not assert for PVH domains.

Signed-off-by: Oleksandr Andrushchenko 
---
 tools/libxl/Makefile| 4 
 tools/libxl/libxl_pci.c | 4 +++-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
index 241da7fff6f4..f3806aafcb4e 100644
--- a/tools/libxl/Makefile
+++ b/tools/libxl/Makefile
@@ -130,6 +130,10 @@ endif
 
 LIBXL_LIBS += -lyajl
 
+ifeq ($(CONFIG_ARM),y)
+CFALGS += -DCONFIG_ARM
+endif
+
 LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o libxl_pci.o \
libxl_dom.o libxl_exec.o libxl_xshelp.o libxl_device.o \
libxl_internal.o libxl_utils.o libxl_uuid.o \
diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c
index bc5843b13701..b93cf976642b 100644
--- a/tools/libxl/libxl_pci.c
+++ b/tools/libxl/libxl_pci.c
@@ -1915,8 +1915,10 @@ static void do_pci_remove(libxl__egc *egc, uint32_t 
domid,
 goto out_fail;
 }
 } else {
+/* PCI passthrough can also run on ARM PVH */
+#ifndef CONFIG_ARM
 assert(type == LIBXL_DOMAIN_TYPE_PV);
-
+#endif
 char *sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/resource", 
pcidev->domain,
  pcidev->bus, pcidev->dev, pcidev->func);
 FILE *f = fopen(sysfs_path, "r");
-- 
2.17.1

[PATCH 00/10] [RFC] ARM PCI passthrough configuration and vPCI

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Hello, all!

This is an RFC and an attempt to understand the best way to progress with ARM
PCI passthrough configuration. This includes, but not limited to, configuration
of assignable PCI devices, (legacy IRQs, MSI/MSI-X are not yet supported), MMIO
etc.

This is based on the original RFC from ARM [1] and bits of the possible
configuration approaches were discussed before [2]. So, I tried to implement
something, so we can discuss it in more detail. (Rahul, Bertrand: if you are
interested we can discuss all this in detail, so we can use this as a part of
ARM PCI passthrough solution).

This is all work in progress, so without having some other minor patches one
won’t be able to run that, but still the patches show the direction, which
should be fine for the RFC. Those interested in the full working example I have
created a branch [3], but please note that this was fully tested on R-Car Gen3
platform only which has a non-ECAM PCI host controller and only partially it
was tested on QEMU (without running guest domains, Domain-0 only).

In this RFC I only submit some patches which I would like to get community's
view on.  I will highlight some of them below and the rest is documented in
their commit messages:

1. [WORKAROUND] xen/arm: Update hwdom's p2m to trap ECAM space

This is a workaround to be able to trap ECAM address space in Domain-0 which is
normally not possible because PCI host bridge is mapped into Domain-0, so there
is a need in some special handling of such devices. I have discussed this with
Julien on IRC, but haven't implemented anything of production quality yet.

2. arm/pci: Maintain PCI assignable list

This patch needs a decision on pci-back use. As of now what is not covered is
the assignment of legacy IRQs (MMIOs and MSIs are handled by Xen without the
toolstack's help - am I right here?). MMIOs are assigned by vPCI code.  We
discussed [2] a possibility to run a "limited" version of the pci-back driver
for ARM, but I’d like to bring back that discussion here as it seems that only
some bits of the pci-back may now be used, so profit of having pci-back
in the picture is not clear.

3. vpci: Make every domain handle its own BARs

This is a big change in vPCI code which allows non-identity mappings for guest
domains. This also handles MMIO configuration for the guests without using the
toolstack which does the same via reading PCI bus sysfs entries in Domain-0.
(Thank you Roger for making lots of thing clear for me.) This implements PCI
headers per domain.

4. vpci/arm: Allow updating BAR's header for non-ECAM bridges

This allows non-ECAM bridges, which are not trapped by vPCI for Domain-0/hwdom,
to update vPCI's view of the real values of the BARs. The assumption here is
that Domain-0/hwdom won't relocate BARs which is usually the case.


5. Some code is for R-Car Gen3 which is not ECAM compatible. It is good for the
demostration of where generic ARM PCI framwork should be changed to support
such controllers.

Thank you,
Oleksandr Andrushchenko

P.S. I would like to thank Roger, Juilien and Jan for their attention
and time.

[1] https://www.mail-archive.com/xen-devel@lists.xenproject.org/msg77422.html
[2] https://www.mail-archive.com/xen-devel@lists.xenproject.org/msg84452.html
[3] https://github.com/andr2000/xen/tree/vpci_rfc

Oleksandr Andrushchenko (10):
  pci/pvh: Allow PCI toolstack code run with PVH domains on ARM
  arm/pci: Maintain PCI assignable list
  xen/arm: Setup MMIO range trap handlers for hardware domain
  [WORKAROUND] xen/arm: Update hwdom's p2m to trap ECAM space
  xen/arm: Process pending vPCI map/unmap operations
  vpci: Make every domain handle its own BARs
  xen/arm: Do not hardcode phycial PCI device addresses
  vpci/arm: Allow updating BAR's header for non-ECAM bridges
  vpci/rcar: Implement vPCI.update_bar_header callback
  [HACK] vpci/rcar: Make vPCI know DomD is hardware domain

 tools/libxc/include/xenctrl.h |   9 +
 tools/libxc/xc_domain.c   |   1 +
 tools/libxc/xc_misc.c |  46 
 tools/libxl/Makefile  |   8 +
 tools/libxl/libxl_pci.c   | 109 +-
 xen/arch/arm/domain_build.c   |  10 +-
 xen/arch/arm/pci/pci-host-common.c|  44 
 xen/arch/arm/pci/pci-host-generic.c   |  43 +++-
 xen/arch/arm/pci/pci-host-rcar-gen3.c |  69 ++
 xen/arch/arm/sysctl.c |  66 +-
 xen/arch/arm/traps.c  |   6 +
 xen/arch/arm/vpci.c   |  16 +-
 xen/drivers/passthrough/pci.c |  93 +
 xen/drivers/vpci/header.c | 289 +++---
 xen/drivers/vpci/vpci.c   |   1 +
 xen/include/asm-arm/pci.h |  17 ++
 xen/include/public/arch-arm.h |   9 +-
 xen/include/public/sysctl.h   |  40 
 xen/include/xen/pci.h |  12 ++
 xen/include/xen/vpci.h|  24 ++-
 20 files changed, 857 insertions(+), 55

Re: [XEN PATCH v1] xen/arm : Add support for SMMUv3 driver

[Oleksandr Andrushchenko]

Hello, Rahul!

On 11/6/20 4:41 PM, Rahul Singh wrote:

Hello Oleksandr,


On 6 Nov 2020, at 2:22 pm, Oleksandr Andrushchenko 
 wrote:

Hi, Rahul!

On 11/6/20 3:58 PM, Rahul Singh wrote:

Hello Oleksandr,


On 6 Nov 2020, at 1:00 pm, Oleksandr Andrushchenko 
 wrote:

Hello, Rahul!

On 11/6/20 2:48 PM, Rahul Singh wrote:

Hello Oleksandr,


On 2 Nov 2020, at 10:12 am, Oleksandr Andrushchenko 
 wrote:

Hi,

On 11/2/20 11:55 AM, Bertrand Marquis wrote:

Hi,


On 2 Nov 2020, at 05:55, Oleksandr Andrushchenko  wrote:

Hi, Julien!

On 10/30/20 7:18 PM, Julien Grall wrote:

Hi Oleksandr,

On 30/10/2020 10:44, Oleksandr Andrushchenko wrote:

On 10/20/20 6:25 PM, Rahul Singh wrote:

Add support for ARM architected SMMUv3 implementations. It is based on
the Linux SMMUv3 driver.

Major differences between the Linux driver are as follows:
1. Only Stage-2 translation is supported as compared to the Linux driver
  that supports both Stage-1 and Stage-2 translations.

First of all thank you for the efforts!

I tried the patch with QEMU and would like to know if my understanding correct

that this combination will not work as of now:

(XEN) SMMUv3: /smmuv3@905: SMMUv3: DT value = eventq
(XEN) Data Abort Trap. Syndrome=0x1940010
(XEN) Walking Hypervisor VA 0x40031000 on CPU0 via TTBR 0xb8469000
(XEN) 0TH[0x0] = 0xb8468f7f

[snip]

If this is expected then is there any plan to make QEMU work as well?

I see [1] says that "Only stage 1 and AArch64 PTW are supported." on QEMU side.

Just for clarication, you are trying to boot Xen on QEMU, right?

Exactly

You might be able to use the stage-1 page-tables to isolate each device in Xen. 
However, I don't think you will be able to share the P2M because the 
page-tables layout between stage-1 and stage-2 is different.

So, it is even more work then

Overall it would make more sense to spend some time adding proper support in 
Qemu then trying to modify the driver to support Qemu right now.


We are interested in QEMU/SMMUv3 as a flexible platform for PCI passthrough

implementation, so it could allow testing different setups and configurations 
with QEMU.

I would recommend to get the SMMU supporting supporting stage-2 page-tables.

You mean in QEMU?

See before.


Regardless that, I think Xen should be able to say the SMMU is not supported 
rather than crashing.

Yes, that would be nice

Fully agree and we will look into that.

Anything you could share so that we could quickly reproduce your setup would be 
more then great.

Nothing special,

qemu/aarch64-softmmu/qemu-system-aarch64 -machine type=virt -machine 
virt,gic-version=2 \

-machine virtualization=true -cpu cortex-a57 -smp 4 -m 2048 -nic 
user,hostfwd=tcp:127.0.0.1:-:22 \

-nographic -serial mon:stdio [..snip..]

I also set iommu to smmuv3 in my tests, QEMU emulator version 4.2.1

I just checked and confirmed that QEMU is booting with XEN SMMUv3 patch and XEN 
is able to say SMMU translation is not supported. As XEN supports Stage-2 
translation and QEMU supports Stage-1 only.


(XEN) SMMUv3: /smmuv3@905: SMMUv3: DT value = eventq
(XEN) SMMUv3: /smmuv3@905: IDR0.COHACC overridden by FW configuration 
(false)
(XEN) SMMUv3: /smmuv3@905: no translation support!
(XEN) I/O virtualisation disabled

Only difference I observed is that you have to add option "-machine 
virt,iommu=smmuv3 “ when launching the QEMU.

I do use the option

I used "-machine virt,iommu=smmuv3 “  option while creating the virt-dtb and 
while launching the QEMU.
I also observed the same error what you observed if I am not using the 
"-machine virt,iommu=smmuv3 “ options when launching the QEMU so I thought this 
might be case for you also but anyways you have use the options it might be other 
issue.

Hm, probably that was on my side as now I can see:

(XEN) SMMUv3: /smmuv3@905: SMMUv3: DT value = eventq
(XEN) SMMUv3: /smmuv3@905: IDR0.COHACC overridden by FW configuration 
(false)
(XEN) SMMUv3: /smmuv3@905: no translation support!
(XEN) I/O virtualisation disabled
(XEN)
(XEN) 
(XEN) Panic on CPU 0:
(XEN) Couldn't configure correctly all the IOMMUs.
(XEN) 
(XEN)
(XEN) Manual reset required ('noreboot' specified)

So, sorry for the noise, I might have misconfigured something it seems

When you say "Xen is booting" do you mean you see the same panic?

Yes I observe the same.

We have to decide now if for SMMUv3 there is no translation support do we have 
to print the logs and move forward  or as above return error to iommu_setup 
that will cal panic().

I would allow Xen to boot further


Regards,
Rahul


Thank you,

Oleksandr


Please let me know if it also works for you.

Well, I should have reported that earlier that I do not use the staging Xen at 
the moment,

it is 4.14.0. So, can this be a problem with that Xen version?

I don’t think so this is the problem with the XEN version.

An

Re: [XEN PATCH v1] xen/arm : Add support for SMMUv3 driver

[Oleksandr Andrushchenko]

Hi, Rahul!

On 11/6/20 3:58 PM, Rahul Singh wrote:
> Hello Oleksandr,
>
>> On 6 Nov 2020, at 1:00 pm, Oleksandr Andrushchenko 
>>  wrote:
>>
>> Hello, Rahul!
>>
>> On 11/6/20 2:48 PM, Rahul Singh wrote:
>>> Hello Oleksandr,
>>>
>>>> On 2 Nov 2020, at 10:12 am, Oleksandr Andrushchenko 
>>>>  wrote:
>>>>
>>>> Hi,
>>>>
>>>> On 11/2/20 11:55 AM, Bertrand Marquis wrote:
>>>>> Hi,
>>>>>
>>>>>> On 2 Nov 2020, at 05:55, Oleksandr Andrushchenko  
>>>>>> wrote:
>>>>>>
>>>>>> Hi, Julien!
>>>>>>
>>>>>> On 10/30/20 7:18 PM, Julien Grall wrote:
>>>>>>> Hi Oleksandr,
>>>>>>>
>>>>>>> On 30/10/2020 10:44, Oleksandr Andrushchenko wrote:
>>>>>>>> On 10/20/20 6:25 PM, Rahul Singh wrote:
>>>>>>>>> Add support for ARM architected SMMUv3 implementations. It is based on
>>>>>>>>> the Linux SMMUv3 driver.
>>>>>>>>>
>>>>>>>>> Major differences between the Linux driver are as follows:
>>>>>>>>> 1. Only Stage-2 translation is supported as compared to the Linux 
>>>>>>>>> driver
>>>>>>>>>   that supports both Stage-1 and Stage-2 translations.
>>>>>>>> First of all thank you for the efforts!
>>>>>>>>
>>>>>>>> I tried the patch with QEMU and would like to know if my understanding 
>>>>>>>> correct
>>>>>>>>
>>>>>>>> that this combination will not work as of now:
>>>>>>>>
>>>>>>>> (XEN) SMMUv3: /smmuv3@905: SMMUv3: DT value = eventq
>>>>>>>> (XEN) Data Abort Trap. Syndrome=0x1940010
>>>>>>>> (XEN) Walking Hypervisor VA 0x40031000 on CPU0 via TTBR 
>>>>>>>> 0xb8469000
>>>>>>>> (XEN) 0TH[0x0] = 0xb8468f7f
>>>>>>>>
>>>>>>>> [snip]
>>>>>>>>
>>>>>>>> If this is expected then is there any plan to make QEMU work as well?
>>>>>>>>
>>>>>>>> I see [1] says that "Only stage 1 and AArch64 PTW are supported." on 
>>>>>>>> QEMU side.
>>>>>>> Just for clarication, you are trying to boot Xen on QEMU, right?
>>>>>> Exactly
>>>>>>> You might be able to use the stage-1 page-tables to isolate each device 
>>>>>>> in Xen. However, I don't think you will be able to share the P2M 
>>>>>>> because the page-tables layout between stage-1 and stage-2 is different.
>>>>>> So, it is even more work then
>>>>> Overall it would make more sense to spend some time adding proper support 
>>>>> in Qemu then trying to modify the driver to support Qemu right now.
>>>>>
>>>>>>>> We are interested in QEMU/SMMUv3 as a flexible platform for PCI 
>>>>>>>> passthrough
>>>>>>>>
>>>>>>>> implementation, so it could allow testing different setups and 
>>>>>>>> configurations with QEMU.
>>>>>>> I would recommend to get the SMMU supporting supporting stage-2 
>>>>>>> page-tables.
>>>>>> You mean in QEMU?
>>>>> See before.
>>>>>
>>>>>>> Regardless that, I think Xen should be able to say the SMMU is not 
>>>>>>> supported rather than crashing.
>>>>>> Yes, that would be nice
>>>>> Fully agree and we will look into that.
>>>>>
>>>>> Anything you could share so that we could quickly reproduce your setup 
>>>>> would be more then great.
>>>> Nothing special,
>>>>
>>>> qemu/aarch64-softmmu/qemu-system-aarch64 -machine type=virt -machine 
>>>> virt,gic-version=2 \
>>>>
>>>> -machine virtualization=true -cpu cortex-a57 -smp 4 -m 2048 -nic 
>>>> user,hostfwd=tcp:127.0.0.1:-:22 \
>>>>
>>>> -nographic -serial mon:stdio [..snip..]
>>>>
>>>> I also set iommu to smmuv3 in my tests, QEMU emulator version 4.2.1
>>> I just checked and confirmed that QEMU is

Re: [XEN PATCH v1] xen/arm : Add support for SMMUv3 driver

[Oleksandr Andrushchenko]

Hello, Rahul!

On 11/6/20 2:48 PM, Rahul Singh wrote:
> Hello Oleksandr,
>
>> On 2 Nov 2020, at 10:12 am, Oleksandr Andrushchenko 
>>  wrote:
>>
>> Hi,
>>
>> On 11/2/20 11:55 AM, Bertrand Marquis wrote:
>>> Hi,
>>>
>>>> On 2 Nov 2020, at 05:55, Oleksandr Andrushchenko  
>>>> wrote:
>>>>
>>>> Hi, Julien!
>>>>
>>>> On 10/30/20 7:18 PM, Julien Grall wrote:
>>>>> Hi Oleksandr,
>>>>>
>>>>> On 30/10/2020 10:44, Oleksandr Andrushchenko wrote:
>>>>>> On 10/20/20 6:25 PM, Rahul Singh wrote:
>>>>>>> Add support for ARM architected SMMUv3 implementations. It is based on
>>>>>>> the Linux SMMUv3 driver.
>>>>>>>
>>>>>>> Major differences between the Linux driver are as follows:
>>>>>>> 1. Only Stage-2 translation is supported as compared to the Linux driver
>>>>>>>   that supports both Stage-1 and Stage-2 translations.
>>>>>> First of all thank you for the efforts!
>>>>>>
>>>>>> I tried the patch with QEMU and would like to know if my understanding 
>>>>>> correct
>>>>>>
>>>>>> that this combination will not work as of now:
>>>>>>
>>>>>> (XEN) SMMUv3: /smmuv3@905: SMMUv3: DT value = eventq
>>>>>> (XEN) Data Abort Trap. Syndrome=0x1940010
>>>>>> (XEN) Walking Hypervisor VA 0x40031000 on CPU0 via TTBR 
>>>>>> 0xb8469000
>>>>>> (XEN) 0TH[0x0] = 0xb8468f7f
>>>>>>
>>>>>> [snip]
>>>>>>
>>>>>> If this is expected then is there any plan to make QEMU work as well?
>>>>>>
>>>>>> I see [1] says that "Only stage 1 and AArch64 PTW are supported." on 
>>>>>> QEMU side.
>>>>> Just for clarication, you are trying to boot Xen on QEMU, right?
>>>> Exactly
>>>>> You might be able to use the stage-1 page-tables to isolate each device 
>>>>> in Xen. However, I don't think you will be able to share the P2M because 
>>>>> the page-tables layout between stage-1 and stage-2 is different.
>>>> So, it is even more work then
>>> Overall it would make more sense to spend some time adding proper support 
>>> in Qemu then trying to modify the driver to support Qemu right now.
>>>
>>>>>> We are interested in QEMU/SMMUv3 as a flexible platform for PCI 
>>>>>> passthrough
>>>>>>
>>>>>> implementation, so it could allow testing different setups and 
>>>>>> configurations with QEMU.
>>>>> I would recommend to get the SMMU supporting supporting stage-2 
>>>>> page-tables.
>>>> You mean in QEMU?
>>> See before.
>>>
>>>>> Regardless that, I think Xen should be able to say the SMMU is not 
>>>>> supported rather than crashing.
>>>> Yes, that would be nice
>>> Fully agree and we will look into that.
>>>
>>> Anything you could share so that we could quickly reproduce your setup 
>>> would be more then great.
>> Nothing special,
>>
>> qemu/aarch64-softmmu/qemu-system-aarch64 -machine type=virt -machine 
>> virt,gic-version=2 \
>>
>> -machine virtualization=true -cpu cortex-a57 -smp 4 -m 2048 -nic 
>> user,hostfwd=tcp:127.0.0.1:-:22 \
>>
>> -nographic -serial mon:stdio [..snip..]
>>
>> I also set iommu to smmuv3 in my tests, QEMU emulator version 4.2.1
> I just checked and confirmed that QEMU is booting with XEN SMMUv3 patch and 
> XEN is able to say SMMU translation is not supported. As XEN supports Stage-2 
> translation and QEMU supports Stage-1 only.
>
>
> (XEN) SMMUv3: /smmuv3@905: SMMUv3: DT value = eventq
> (XEN) SMMUv3: /smmuv3@905: IDR0.COHACC overridden by FW configuration 
> (false)
> (XEN) SMMUv3: /smmuv3@905: no translation support!
> (XEN) I/O virtualisation disabled
>
> Only difference I observed is that you have to add option "-machine 
> virt,iommu=smmuv3 “ when launching the QEMU.
I do use the option
>
> Please let me know if it also works for you.

Well, I should have reported that earlier that I do not use the staging Xen at 
the moment,

it is 4.14.0. So, can this be a problem with that Xen version?

Anyways, if it works with the staging then everything looks ok

Thank you,

Oleksandr

>   
>>> Regards
>>> Bertrand
>>>
>>>>> Cheers,
>>>>>
>>>> Thank you,
>>>>
>>>> Oleksandr

Re: [XEN PATCH v1] xen/arm : Add support for SMMUv3 driver

[Oleksandr Andrushchenko]

Hi,

On 11/2/20 11:55 AM, Bertrand Marquis wrote:
> Hi,
>
>> On 2 Nov 2020, at 05:55, Oleksandr Andrushchenko  wrote:
>>
>> Hi, Julien!
>>
>> On 10/30/20 7:18 PM, Julien Grall wrote:
>>> Hi Oleksandr,
>>>
>>> On 30/10/2020 10:44, Oleksandr Andrushchenko wrote:
>>>> On 10/20/20 6:25 PM, Rahul Singh wrote:
>>>>> Add support for ARM architected SMMUv3 implementations. It is based on
>>>>> the Linux SMMUv3 driver.
>>>>>
>>>>> Major differences between the Linux driver are as follows:
>>>>> 1. Only Stage-2 translation is supported as compared to the Linux driver
>>>>>   that supports both Stage-1 and Stage-2 translations.
>>>> First of all thank you for the efforts!
>>>>
>>>> I tried the patch with QEMU and would like to know if my understanding 
>>>> correct
>>>>
>>>> that this combination will not work as of now:
>>>>
>>>> (XEN) SMMUv3: /smmuv3@905: SMMUv3: DT value = eventq
>>>> (XEN) Data Abort Trap. Syndrome=0x1940010
>>>> (XEN) Walking Hypervisor VA 0x40031000 on CPU0 via TTBR 0xb8469000
>>>> (XEN) 0TH[0x0] = 0xb8468f7f
>>>>
>>>> [snip]
>>>>
>>>> If this is expected then is there any plan to make QEMU work as well?
>>>>
>>>> I see [1] says that "Only stage 1 and AArch64 PTW are supported." on QEMU 
>>>> side.
>>> Just for clarication, you are trying to boot Xen on QEMU, right?
>> Exactly
>>> You might be able to use the stage-1 page-tables to isolate each device in 
>>> Xen. However, I don't think you will be able to share the P2M because the 
>>> page-tables layout between stage-1 and stage-2 is different.
>> So, it is even more work then
> Overall it would make more sense to spend some time adding proper support in 
> Qemu then trying to modify the driver to support Qemu right now.
>
>>>>
>>>> We are interested in QEMU/SMMUv3 as a flexible platform for PCI passthrough
>>>>
>>>> implementation, so it could allow testing different setups and 
>>>> configurations with QEMU.
>>> I would recommend to get the SMMU supporting supporting stage-2 page-tables.
>> You mean in QEMU?
> See before.
>
>>> Regardless that, I think Xen should be able to say the SMMU is not 
>>> supported rather than crashing.
>> Yes, that would be nice
> Fully agree and we will look into that.
>
> Anything you could share so that we could quickly reproduce your setup would 
> be more then great.

Nothing special,

qemu/aarch64-softmmu/qemu-system-aarch64 -machine type=virt -machine 
virt,gic-version=2 \

-machine virtualization=true -cpu cortex-a57 -smp 4 -m 2048 -nic 
user,hostfwd=tcp:127.0.0.1:-:22 \

-nographic -serial mon:stdio [..snip..]

I also set iommu to smmuv3 in my tests, QEMU emulator version 4.2.1

>
> Regards
> Bertrand
>
>>> Cheers,
>>>
>> Thank you,
>>
>> Oleksandr

Re: [XEN PATCH v1] xen/arm : Add support for SMMUv3 driver

[Oleksandr Andrushchenko]

Hi, Julien!

On 10/30/20 7:18 PM, Julien Grall wrote:

Hi Oleksandr,

On 30/10/2020 10:44, Oleksandr Andrushchenko wrote:

On 10/20/20 6:25 PM, Rahul Singh wrote:

Add support for ARM architected SMMUv3 implementations. It is based on
the Linux SMMUv3 driver.

Major differences between the Linux driver are as follows:
1. Only Stage-2 translation is supported as compared to the Linux driver
 that supports both Stage-1 and Stage-2 translations.


First of all thank you for the efforts!

I tried the patch with QEMU and would like to know if my understanding correct

that this combination will not work as of now:

(XEN) SMMUv3: /smmuv3@905: SMMUv3: DT value = eventq
(XEN) Data Abort Trap. Syndrome=0x1940010
(XEN) Walking Hypervisor VA 0x40031000 on CPU0 via TTBR 0xb8469000
(XEN) 0TH[0x0] = 0xb8468f7f

[snip]

If this is expected then is there any plan to make QEMU work as well?

I see [1] says that "Only stage 1 and AArch64 PTW are supported." on QEMU side.


Just for clarication, you are trying to boot Xen on QEMU, right?

Exactly


You might be able to use the stage-1 page-tables to isolate each device in Xen. 
However, I don't think you will be able to share the P2M because the 
page-tables layout between stage-1 and stage-2 is different.

So, it is even more work then





We are interested in QEMU/SMMUv3 as a flexible platform for PCI passthrough

implementation, so it could allow testing different setups and configurations 
with QEMU.


I would recommend to get the SMMU supporting supporting stage-2 page-tables.

You mean in QEMU?


Regardless that, I think Xen should be able to say the SMMU is not supported 
rather than crashing.

Yes, that would be nice


Cheers,


Thank you,

Oleksandr

Re: [XEN PATCH v1] xen/arm : Add support for SMMUv3 driver

[Oleksandr Andrushchenko]

Hi,

On 10/30/20 6:08 PM, Bertrand Marquis wrote:
> Hi Oleksandr,
>
>> On 30 Oct 2020, at 15:02, Oleksandr Andrushchenko 
>>  wrote:
>>
>> Hi,
>>
>> On 10/30/20 4:47 PM, Rahul Singh wrote:
>>> Hello Oleksandr,
>>>
>>>> On 30 Oct 2020, at 10:44 am, Oleksandr Andrushchenko 
>>>>  wrote:
>>>>
>>>> Hi, Rahul!
>>>>
>>>> On 10/20/20 6:25 PM, Rahul Singh wrote:
>>>>> Add support for ARM architected SMMUv3 implementations. It is based on
>>>>> the Linux SMMUv3 driver.
>>>>>
>>>>> Major differences between the Linux driver are as follows:
>>>>> 1. Only Stage-2 translation is supported as compared to the Linux driver
>>>>> that supports both Stage-1 and Stage-2 translations.
>>>> First of all thank you for the efforts!
>>>>
>>>> I tried the patch with QEMU and would like to know if my understanding 
>>>> correct
>>>>
>>>> that this combination will not work as of now:
>>>>
>>>> (XEN) SMMUv3: /smmuv3@905: SMMUv3: DT value = eventq
>>> I have limited knowledge about QEMU internals.As what I see from the logs, 
>>> fault is occurred at early driver initialisation when SMMU driver is trying 
>>> to probe the HW.
>>>
>>>> (XEN) Data Abort Trap. Syndrome=0x1940010
>>>> (XEN) Walking Hypervisor VA 0x40031000 on CPU0 via TTBR 0xb8469000
>>>> (XEN) 0TH[0x0] = 0xb8468f7f
>>>>
>>>> [snip]
>>>>
>>>> If this is expected then is there any plan to make QEMU work as well?
>>>>
>>>> I see [1] says that "Only stage 1 and AArch64 PTW are supported." on QEMU 
>>>> side.
>>> Yes as of now only Stage-2 is supported in XEN.If we have any requirement 
>>> or use case that depends on Stage-1 translation we can support that also in 
>>> XEN.
>> The use case is below: PCI passthrough and various configurations including 
>> SR-IOV which is possible with QEMU...
> This is currently not in the list of configuration supported or that we have 
> planned on our side to support.
>
> But we would be more then happy to review any changes to enable this :-)

Fair enough

Unfortunately we do not have any HW with SMMUv3 at the moment, so not sure we 
can

invest time in this at the moment

>
> Regards
> Bertrand

Thank you,

Oleksandr

>
>>>> We are interested in QEMU/SMMUv3 as a flexible platform for PCI passthrough
>>>>
>>>> implementation, so it could allow testing different setups and 
>>>> configurations with QEMU.
>>>>
>>>>
>>>> Thank you in advance,
>>>>
>>>> Oleksandr
>>>>
>>>> [1] 
>>>> https://urldefense.com/v3/__https://patchwork.ozlabs.org/project/qemu-devel/cover/1524665762-31355-1-git-send-email-eric.au...@redhat.com/__;!!GF_29dbcQIUBPA!h-EaE0OnSbXtLBSwIS311alDl7pn8sH7sihgIYqilM5-r-8kCH6USNNlLB3xhbzc6eczUOrhcw$
>>>>  [patchwork[.]ozlabs[.]org]
>>> Regards,
>>> Rahul
>> Thank you,
>>
>> Oleksandr
>

Re: [XEN PATCH v1] xen/arm : Add support for SMMUv3 driver

[Oleksandr Andrushchenko]

Hi,

On 10/30/20 4:47 PM, Rahul Singh wrote:
> Hello Oleksandr,
>
>> On 30 Oct 2020, at 10:44 am, Oleksandr Andrushchenko 
>>  wrote:
>>
>> Hi, Rahul!
>>
>> On 10/20/20 6:25 PM, Rahul Singh wrote:
>>> Add support for ARM architected SMMUv3 implementations. It is based on
>>> the Linux SMMUv3 driver.
>>>
>>> Major differences between the Linux driver are as follows:
>>> 1. Only Stage-2 translation is supported as compared to the Linux driver
>>> that supports both Stage-1 and Stage-2 translations.
>> First of all thank you for the efforts!
>>
>> I tried the patch with QEMU and would like to know if my understanding 
>> correct
>>
>> that this combination will not work as of now:
>>
>> (XEN) SMMUv3: /smmuv3@905: SMMUv3: DT value = eventq
> I have limited knowledge about QEMU internals.As what I see from the logs, 
> fault is occurred at early driver initialisation when SMMU driver is trying 
> to probe the HW.
>
>> (XEN) Data Abort Trap. Syndrome=0x1940010
>> (XEN) Walking Hypervisor VA 0x40031000 on CPU0 via TTBR 0xb8469000
>> (XEN) 0TH[0x0] = 0xb8468f7f
>>
>> [snip]
>>
>> If this is expected then is there any plan to make QEMU work as well?
>>
>> I see [1] says that "Only stage 1 and AArch64 PTW are supported." on QEMU 
>> side.
> Yes as of now only Stage-2 is supported in XEN.If we have any requirement or 
> use case that depends on Stage-1 translation we can support that also in XEN.
The use case is below: PCI passthrough and various configurations including 
SR-IOV which is possible with QEMU...
>
>>
>> We are interested in QEMU/SMMUv3 as a flexible platform for PCI passthrough
>>
>> implementation, so it could allow testing different setups and 
>> configurations with QEMU.
>>
>>
>> Thank you in advance,
>>
>> Oleksandr
>>
>> [1] 
>> https://urldefense.com/v3/__https://patchwork.ozlabs.org/project/qemu-devel/cover/1524665762-31355-1-git-send-email-eric.au...@redhat.com/__;!!GF_29dbcQIUBPA!h-EaE0OnSbXtLBSwIS311alDl7pn8sH7sihgIYqilM5-r-8kCH6USNNlLB3xhbzc6eczUOrhcw$
>>  [patchwork[.]ozlabs[.]org]
> Regards,
> Rahul

Thank you,

Oleksandr

Re: [XEN PATCH v1] xen/arm : Add support for SMMUv3 driver

[Oleksandr Andrushchenko]

Hi, Rahul!

On 10/20/20 6:25 PM, Rahul Singh wrote:
> Add support for ARM architected SMMUv3 implementations. It is based on
> the Linux SMMUv3 driver.
>
> Major differences between the Linux driver are as follows:
> 1. Only Stage-2 translation is supported as compared to the Linux driver
> that supports both Stage-1 and Stage-2 translations.

First of all thank you for the efforts!

I tried the patch with QEMU and would like to know if my understanding correct

that this combination will not work as of now:

(XEN) SMMUv3: /smmuv3@905: SMMUv3: DT value = eventq
(XEN) Data Abort Trap. Syndrome=0x1940010
(XEN) Walking Hypervisor VA 0x40031000 on CPU0 via TTBR 0xb8469000
(XEN) 0TH[0x0] = 0xb8468f7f

[snip]

If this is expected then is there any plan to make QEMU work as well?

I see [1] says that "Only stage 1 and AArch64 PTW are supported." on QEMU side.


We are interested in QEMU/SMMUv3 as a flexible platform for PCI passthrough

implementation, so it could allow testing different setups and configurations 
with QEMU.


Thank you in advance,

Oleksandr

[1] 
https://patchwork.ozlabs.org/project/qemu-devel/cover/1524665762-31355-1-git-send-email-eric.au...@redhat.com/

Re: ARM/PCI passthrough: libxl_pci, sysfs and pciback questions

[Oleksandr Andrushchenko]

On 10/27/20 7:18 PM, Jan Beulich wrote:
> On 27.10.2020 16:52, Oleksandr Andrushchenko wrote:
>> On 10/27/20 2:55 PM, Roger Pau Monné wrote:
>>> On Tue, Oct 27, 2020 at 09:59:05AM +, Oleksandr Andrushchenko wrote:
>>>> 5. An alternative route for 3-4 could be to store that data in 
>>>> XenStore, e.g.
>>>>    MMIOs, IRQ, bind sysfs path etc. This would require more code on 
>>>> Xen side to
>>>>    access XenStore and won’t work if MMIOs/IRQs are passed via device 
>>>> tree/ACPI
>>>>    tables by the bootloaders.
>>> As above, I think I need more context to understand what and why you
>>> need to save such information.
>> Well, with pciback absence we loose a "database" which holds all the 
>> knowledge
>>
>> about which devices are assigned, bound etc.
> What hasn't become clear to me (sorry if I've overlooked it) is
> why some form of pciback is not an option on Arm.
Yes, it is probably possible to run pciback even without running

pcifront instances in guests and only use that functionality

which is needed for the toolstack. We can even have it as is without

modifications given that pcifront won't run and that part of the pciback

related to PCI config space, MSI etc. won't simply be used, but still

present in the pciback driver. We can try that (pciback is x86

only in the kernel).

> Where it would
> need to run in your split hardware-domain / Dom0 setup (if I got
> that right in the first place) would be a secondary question.

This actually becomes a problem if we think about hwdom != Dom0:

Dom0/toolstack wants to read PCI bus sysfs and it also wants to access

pciback's sysfs entries. So, for Dom0's toolstack to read sysfs in this scenario

we need a bridge between Dom0 and that hwdom to access both PCI

subsystem and pciback's sysfs: this could be implemented as a back-front pair

with a ring and event channel as PV drivers do. This approach of course will

require the toolstack to work in two modes: local sysfs/pciback and remote ones.

In the remote access model the toolstack will need to create a connection to

the hwdom each time it runs and requires sysfs data which should be acceptable.

It can also be possible to have the toolstack always use the remote model even

if it runs locally which will make the toolstack's code support a single model 
for all

the use-cases.

(Never thought if it is possible to run both backend and frontend in the same 
VM though).

> Jan

Thank you,

Oleksandr

Re: ARM/PCI passthrough: libxl_pci, sysfs and pciback questions

[Oleksandr Andrushchenko]

Hello, Roger!

On 10/27/20 2:55 PM, Roger Pau Monné wrote:
> On Tue, Oct 27, 2020 at 09:59:05AM +0000, Oleksandr Andrushchenko wrote:
>> Hello, all!
>>
>> While working on PCI passthrough on ARM (partial RFC was published by ARM
>> earlier this year) I tried to implement some related changes in the 
>> toolstack.
>> One of the obstacles for ARM is PCI backend’s related code presence: ARM is
>> going to fully emulate an ECAM host bridge in Xen, so no PCI backend/frontend
>> pair is going to be used.
>>
>> If my understanding correct the functionality which is implemented by the
>> pciback and toolstack and which is relevant/needed for ARM:
>>
>>    1. pciback is used as a database for assignable PCI devices, e.g. xl
>>       pci-assignable-{add|remove|list} manipulates that list. So, whenever 
>> the
>>       toolstack needs to know which PCI devices can be passed through it 
>> reads
>>       that from the relevant sysfs entries of the pciback.
>>
>>    2. pciback is used to hold the unbound PCI devices, e.g. when passing 
>> through a
>>       PCI device it needs to be unbound from the relevant device driver and 
>> bound
>>       to pciback (strictly speaking it is not required that the device is 
>> bound to
>>       pciback, but pciback is again used as a database of the passed through 
>> PCI
>>       devices, so we can re-bind the devices back to their original drivers 
>> when
>>       guest domain shuts down)
>>
>>    3. toolstack depends on Domain-0 for discovering PCI device resources 
>> which are
>>       then permitted for the guest domain, e.g MMIO ranges, IRQs. are read 
>> from
>>       the sysfs
>>
>>    4. toolstack is responsible for resetting PCI devices being passed 
>> through via
>>       sysfs/reset of the Domain-0’s PCI bus subsystem
>>
>>    5. toolstack is responsible for the devices are passed with all relevant
>>       functions, e.g. so for multifunction devices all the functions are 
>> passed to
>>       a domain and no partial passthrough is done
>>
>>    6. toolstack cares about SR-IOV devices (am I correct here?)
> I'm not sure I fully understand what this means. Toolstack cares about
> SR-IOV as it cares about other PCI devices, but the SR-IOV
> functionality is managed by the (dom0) kernel.
Yes, you are right. Please ignore #6
>
>>
>> I have implemented a really dirty POC for that which I would need to clean up
>> before showing, but before that I would like to get some feedback and advice 
>> on
>> how to proceed with the above. I suggest we:
>>
>>    1. Move all pciback related code (which seems to become x86 code only) 
>> into a
>>       dedicated file, something like tools/libxl/libxl_pci_x86.c
>>
>>    2. Make the functionality now provided by pciback architecture dependent, 
>> so
>>       tools/libxl/libxl_pci.c delegates actual assignable device list 
>> handling to
>>       that arch code and uses some sort of “ops”, e.g.
>>       arch->ops.get_all_assignable, arch->ops.add_assignable etc. (This can 
>> also
>>       be done with “#ifdef CONFIG_PCIBACK”, but seems to be not cute). 
>> Introduce
>>       tools/libxl/libxl_pci_arm.c to provide ARM implementation.
> To be fair this is arch and OS dependent, since it's currently based
> on sysfs which is Linux specific. So it should really be
> libxl_pci_linux_x86.c or similar.
This is true, but do we really have any other implementation yet?
>
>>    3. ARM only: As we do not have pciback on ARM we need to have some 
>> storage for
>>       assignable device list: move that into Xen by extending struct pci_dev 
>> with
>>       “bool assigned” and providing sysctls for manipulating that, e.g.
>>       XEN_SYSCTL_pci_device_{set|get}_assigned,
>>       XEN_SYSCTL_pci_device_enum_assigned (to enumerate/get the list of
>>       assigned/not-assigned PCI devices). Can this also be interesting for 
>> x86? At
>>       the moment it seems that x86 does rely on pciback presence, so 
>> probably this
>>       change might not be interesting for x86 world, but may allow stripping
>>       pciback functionality a bit and making the code common to both ARM and 
>> x86.
> How are you going to perform the device reset then? Will you assign
> the device to dom0 after removing it from the guest so that dom0 can
> perform the reset? You will need to use logic currently present in
> pciback to do so IIRC.
>
> It doesn't seem like a bad approach, but there are more consequences
> than just how assig

ARM/PCI passthrough: libxl_pci, sysfs and pciback questions

[Oleksandr Andrushchenko]

Hello, all!

While working on PCI passthrough on ARM (partial RFC was published by ARM
earlier this year) I tried to implement some related changes in the toolstack.
One of the obstacles for ARM is PCI backend’s related code presence: ARM is
going to fully emulate an ECAM host bridge in Xen, so no PCI backend/frontend
pair is going to be used.

If my understanding correct the functionality which is implemented by the
pciback and toolstack and which is relevant/needed for ARM:

  1. pciback is used as a database for assignable PCI devices, e.g. xl
     pci-assignable-{add|remove|list} manipulates that list. So, whenever the
     toolstack needs to know which PCI devices can be passed through it reads
     that from the relevant sysfs entries of the pciback.

  2. pciback is used to hold the unbound PCI devices, e.g. when passing through 
a
     PCI device it needs to be unbound from the relevant device driver and bound
     to pciback (strictly speaking it is not required that the device is bound 
to
     pciback, but pciback is again used as a database of the passed through PCI
     devices, so we can re-bind the devices back to their original drivers when
     guest domain shuts down)

  3. toolstack depends on Domain-0 for discovering PCI device resources which 
are
     then permitted for the guest domain, e.g MMIO ranges, IRQs. are read from
     the sysfs

  4. toolstack is responsible for resetting PCI devices being passed through via
     sysfs/reset of the Domain-0’s PCI bus subsystem

  5. toolstack is responsible for the devices are passed with all relevant
     functions, e.g. so for multifunction devices all the functions are passed 
to
     a domain and no partial passthrough is done

  6. toolstack cares about SR-IOV devices (am I correct here?)


I have implemented a really dirty POC for that which I would need to clean up
before showing, but before that I would like to get some feedback and advice on
how to proceed with the above. I suggest we:

  1. Move all pciback related code (which seems to become x86 code only) into a
     dedicated file, something like tools/libxl/libxl_pci_x86.c

  2. Make the functionality now provided by pciback architecture dependent, so
     tools/libxl/libxl_pci.c delegates actual assignable device list handling to
     that arch code and uses some sort of “ops”, e.g.
     arch->ops.get_all_assignable, arch->ops.add_assignable etc. (This can also
     be done with “#ifdef CONFIG_PCIBACK”, but seems to be not cute). Introduce
     tools/libxl/libxl_pci_arm.c to provide ARM implementation.

  3. ARM only: As we do not have pciback on ARM we need to have some storage for
     assignable device list: move that into Xen by extending struct pci_dev with
     “bool assigned” and providing sysctls for manipulating that, e.g.
     XEN_SYSCTL_pci_device_{set|get}_assigned,
     XEN_SYSCTL_pci_device_enum_assigned (to enumerate/get the list of
     assigned/not-assigned PCI devices). Can this also be interesting for x86? 
At
     the moment it seems that x86 does rely on pciback presence, so probably 
this
     change might not be interesting for x86 world, but may allow stripping
     pciback functionality a bit and making the code common to both ARM and x86.

  4. ARM only: It is not clear how to handle re-binding of the PCI driver on
     guest shutdown: we need to store the sysfs path of the original driver the
     device was bound to. Do we also want to store that in struct pci_dev?

  5. An alternative route for 3-4 could be to store that data in XenStore, e.g.
     MMIOs, IRQ, bind sysfs path etc. This would require more code on Xen side 
to
     access XenStore and won’t work if MMIOs/IRQs are passed via device 
tree/ACPI
     tables by the bootloaders.


Another big question is with respect to Domain-0 and PCI bus sysfs use. The
existing code for querying PCI device resources/IRQs and resetting those via
sysfs of Domain-0 is more than OK if Domain-0 is present and owns PCI HW. But,
there are at least two cases when this is not going to work on ARM: Dom0less
setups and when there is a hardware domain owning PCI devices.

In our case we have a dedicated guest which is a sort of hardware domain (driver
domain DomD) which owns all the hardware of the platform, so we are interested
in implementing something that fits our design as well: DomD/hardware domain
makes it not possible to access the relevant PCI bus sysfs entries from Domain-0
as those live in DomD/hwdom. This is also true for Dom0less setups as there is
no entity that can provide the same.

For that reason in my POC I have introduced the following: extended struct
pci_dev to hold an array of PCI device’s MMIO ranges and IRQ:

  1. Provide internal API for accessing the array of MMIO ranges and IRQ. This
     can be used in both Dom0less and Domain-0 setups to manipulate the relevant
     data. The actual data can be read from a device tree/ACPI tables if
     enumeration is done by bootloaders.

  2. For 

Re: [bug report] ALSA: xen-front: Use Xen common shared buffer implementation

[Oleksandr Andrushchenko]

Hello, Dan!

On 10/21/20 1:50 PM, Dan Carpenter wrote:
> Hello Oleksandr Andrushchenko,
>
> The patch 58f9d806d16a: "ALSA: xen-front: Use Xen common shared
> buffer implementation" from Nov 30, 2018, leads to the following
> static checker warning:
>
>  sound/xen/xen_snd_front_alsa.c:495 alsa_hw_params()
>  warn: 'stream->shbuf.directory' double freed
>  sound/xen/xen_snd_front_alsa.c:495 alsa_hw_params()
>  warn: 'stream->shbuf.grefs' double freed
>
> sound/xen/xen_snd_front_alsa.c
> 461  static int alsa_hw_params(struct snd_pcm_substream *substream,
> 462struct snd_pcm_hw_params *params)
> 463  {
> 464  struct xen_snd_front_pcm_stream_info *stream = 
> stream_get(substream);
> 465  struct xen_snd_front_info *front_info = stream->front_info;
> 466  struct xen_front_pgdir_shbuf_cfg buf_cfg;
> 467  int ret;
> 468
> 469  /*
> 470   * This callback may be called multiple times,
> 471   * so free the previously allocated shared buffer if any.
> 472   */
> 473  stream_free(stream);
>  ^^^
> This is freed here.
>
> 474  ret = shbuf_setup_backstore(stream, 
> params_buffer_bytes(params));
> 475  if (ret < 0)
> 476  goto fail;
>  ^^
> This leads to some double frees.  Probably more double frees than Smatch
> is detecting.
>
> 477
> 478  memset(_cfg, 0, sizeof(buf_cfg));
> 479  buf_cfg.xb_dev = front_info->xb_dev;
> 480  buf_cfg.pgdir = >shbuf;
> 481  buf_cfg.num_pages = stream->num_pages;
> 482  buf_cfg.pages = stream->pages;
> 483
> 484  ret = xen_front_pgdir_shbuf_alloc(_cfg);
>  
> This is where "stream->shbuf.directory" is re-allocated on the success
> path.
>
> 485  if (ret < 0)
> 486  goto fail;
> 487
> 488  ret = xen_front_pgdir_shbuf_map(>shbuf);
> 489  if (ret < 0)
> 490  goto fail;
> 491
> 492  return 0;
> 493
> 494  fail:
> 495  stream_free(stream);
>  
> Double free.
>
> 496  dev_err(_info->xb_dev->dev,
> 497  "Failed to allocate buffers for stream with index 
> %d\n",
> 498  stream->index);
> 499  return ret;
> 500  }
>
> regards,
> dan carpenter

Thank you for reporting this,

I'll try to look at it closely and prepare a patch.

Thank you,

Oleksandr

Re: [PATCH 2/2] libgnttab: Add support for Linux dma-buf offset

[Oleksandr Andrushchenko]

Hi,

On 9/28/20 6:20 PM, Ian Jackson wrote:
> Oleksandr Andrushchenko writes ("[PATCH 2/2] libgnttab: Add support for Linux 
> dma-buf offset"):
>> From: Oleksandr Andrushchenko 
>>
>> Add version 2 of the dma-buf ioctls which adds data_ofs parameter.
>>
>> dma-buf is backed by a scatter-gather table and has offset parameter
>> which tells where the actual data starts. Relevant ioctls are extended
>> to support that offset:
>>- when dma-buf is created (exported) from grant references then
>>  data_ofs is used to set the offset field in the scatter list
>>  of the new dma-buf
>>- when dma-buf is imported and grant references provided then
>>  data_ofs is used to report that offset to user-space
> Thanks.  I'm not a DMA expert, but I think this is probably going in
> roughly the right direction.  I will probably want a review from a DMA
> expert too, but let me get on with my questions:
>
> When you say "the protocol changes are already accepted" I think you
> mean the Linux ioctl changes ?  If not, what *do* you mean ?

I mean that the relevant protocol changes are already part of both Xen [1]

and Linux trees [2]. What is missing is ioctl implementation in the kernel and

its support in Xen' tools. This is why I have marked the patch as RFC in order

to get some view on the matter from Xen community. Once we agree on the

naming, structure etc. I'll send patches for both Xen and Linux

>
>> +/*
>> + * Version 2 of the ioctls adds @data_ofs parameter.
>> + *
>> + * dma-buf is backed by a scatter-gather table and has offset
>> + * parameter which tells where the actual data starts.
>> + * Relevant ioctls are extended to support that offset:
>> + *   - when dma-buf is created (exported) from grant references then
>> + * @data_ofs is used to set the offset field in the scatter list
>> + * of the new dma-buf
>> + *   - when dma-buf is imported and grant references are provided then
>> + * @data_ofs is used to report that offset to user-space
>> + */
>> +#define IOCTL_GNTDEV_DMABUF_EXP_FROM_REFS_V2 \
>> +_IOC(_IOC_NONE, 'G', 13, \
> I think this was copied from a Linux header file ?  If so please quote
> the precise file and revision in the commit message.
This is not upstream yet, please see explanation above
>And be sure to
> copy the copyright informtaion appropriately.
>
>> +int osdep_gnttab_dmabuf_exp_from_refs_v2(xengnttab_handle *xgt, uint32_t 
>> domid,
>> + uint32_t flags, uint32_t count,
>> + const uint32_t *refs,
>> + uint32_t *dmabuf_fd, uint32_t 
>> data_ofs)
>> +{
>> +abort();
> I'm pretty sure this is wrong.

First of all, Linux dma-bufs are only supported on Linux, so neither FreeBSD 
nor Mini-OS

will have that. If you are referring to "abort()" here, so I am just aligning 
to what previously

was there, e.g. all non-relevant dma-buf OS specifics were implemented like 
that.

>
> This leads me to ask about compatibility, both across versions of the
> various components, and API compatibility across different platforms.
>
> libxengnttab is supposed to have a stable API and ABI.  This means
> that old programs should work with the new library - which I think you
> have achieved.
Yes
>
> But I think it also means that it should work with new programs, and
> the new library, on old kernels.  What is your compatibility story
> here ?  What is the intended mode of use by an application ?

Well, this is a tough story. If we have new software and new library, but old

kernel it means that the offset we are trying to get with the new ioctl will be

unavailable to that new software. In most cases we can use offset of 0, but some

platforms (iMX8) use offset of 64. So, we can workaround that for most(?) 
platforms

by reporting offset 0, but some platforms will fail. I am not sure if this is 
good to state that

this combination of software (as described above) "will mostly work" or just let

the system fail at run-time, by letting Linux return ENOTSUPP for the new ioctl.

By fail I mean that the display backend may decide if to use the previous 
version of the ioctl

without the offset field.

>
> And the same application code should be useable, so far as possible,
> across different plaatforms that support Xen.
>
> What fallback would be possible for application do if the v2 function
> is not available ?  I think that fallback action needs to be
> selectable at runtime, to support new userspace on old kernels.

Well, as I said before, for the platforms with offset 0 we are "fine" ignoring 
th

Re: u-boot vs. uefi as boot loaders on ARM

[Oleksandr Andrushchenko]

On 8/20/20 1:50 PM, Julien Grall wrote:
> Hi Roman,
>
> On 16/08/2020 21:45, Roman Shaposhnik wrote:
>> On Sun, Aug 16, 2020 at 7:54 AM Julien Grall  wrote:
>>> On 15/08/2020 21:43, Roman Shaposhnik wrote:
 Hi!
>>>
>>> Hi,
>>>
 with the recent excellent work by Anastasiia committed to the u-boot's
 main line, we now have two different ways of bringing ARM DomUs.

 Is there any chance someone can educate the general public on pros
 and cons of both approaches?

 In Project EVE we're still using uefi on ARM (to stay closer to the more
 "ARM in the cloud" use case) but perhaps the situation now is more
 nuanced?
>>>
>>> UEFI is just standard, so I am guessing you are referring to
>>> Tianocore/EDK2. am I correct?
>>
>> Yes, but I was actually referring to both in a way (I should've been
>> clearer tho).
>> To be more explicit my question was around trying to compare a "standardized"
>> way of botting a generic DomU on ARM (and that standard is UEFI with one
>> particular implementation that works out of the box with Xen being TC/EDK2) 
>> with
>> a more ad-hoc u-boot style of booting.
>>
>>> Recent version of U-boot are also able to partially UEFI. This means you
>>> could easily use GRUB with U-boot.
>>
>> Yup -- which complicated things even more. And it is funny you should mention
>> it, since we actually started with TC/EDK2 for RaspberryPi4 as a board
>> bootloader,
>> but quickly switched to u-boot with UEFI shim layer, since it was much 
>> smaller,
>> better supported (still?) and gave us all we needed to boot Xen on RPi4 as a
>> UEFI payload.
>>
>>>  From my understanding, U-boot is just a bootloader. Therefore it will
>>> not provide runtime services (such as date & time).
>>
>> It actually does provide some of that (see below)
>
> Cool! Although, it looks mostly related to the environment variable though.
>
>>
>>> Furthermore, the
>>> interface is less user friendly, you will have to know the memory layout
>>> in order to load binaries.
>>>
>>> On the other hand, Tianocore/EDK2 is very similar to what non-embedded
>>> may be used to. It will not require you to know your memory layout. But
>>> this comes at the cost of a more complex bootloader to debug.
>>
>> That's literally the crux of my question -- trying to understand what use 
>> cases
>> either one of them is meant for. Especially given that this shim layer is now
>> quite capable:
>> https://github.com/ARM-software/u-boot/blob/master/doc/README.uefi#L127
>
> While I can see major differences when using either on baremetal (you have 
> better control on the Device-Tree with U-boot), it is much less clear in a 
> guest. Maybe Anastasiia can explain why they decided to add support in 
> U-boot? :).

Well, there are many SoC vendors provide u-boot as their boot loader,

so it was natural for us to add pvblock to it (Renesas, Xilinx, iMX, RPi, you 
name it).

So this is the only reason I guess
> Cheers,
>
Regards,

Oleksandr

Re: [PATCH v2 2/5] drm/xen-front: Fix misused IS_ERR_OR_NULL checks

[Oleksandr Andrushchenko]

Hi,

On 8/20/20 2:56 AM, Sasha Levin wrote:
> Hi
>
> [This is an automated email]
>
> This commit has been processed because it contains a "Fixes:" tag
> fixing commit: c575b7eeb89f ("drm/xen-front: Add support for Xen PV display 
> frontend").
>
> The bot has tested the following trees: v5.8.1, v5.7.15, v5.4.58, v4.19.139.
>
> v5.8.1: Build OK!
> v5.7.15: Build OK!
> v5.4.58: Failed to apply! Possible dependencies:
>  4c1cb04e0e7a ("drm/xen: fix passing zero to 'PTR_ERR' warning")
>  93adc0c2cb72 ("drm/xen: Simplify fb_create")
>
> v4.19.139: Failed to apply! Possible dependencies:
>  4c1cb04e0e7a ("drm/xen: fix passing zero to 'PTR_ERR' warning")
>  93adc0c2cb72 ("drm/xen: Simplify fb_create")
>
>
> NOTE: The patch will not be queued to stable trees until it is upstream.
>
> How should we proceed with this patch?
>
This is because of commit 4c1cb04e0e7ac4ba1ef5457929ef9b5671d9eed3

was not CCed to stable. So, if we want the patch to be applied to older stable

kernels we also need this patch as well.

Thank you,

Oleksandr

Re: [PATCH v2 0/5] Fixes and improvements for Xen pvdrm

[Oleksandr Andrushchenko]

On 8/13/20 6:13 PM, Jürgen Groß wrote:
> On 13.08.20 17:10, Oleksandr Andrushchenko wrote:
>>
>> On 8/13/20 6:02 PM, Jürgen Groß wrote:
>>> On 13.08.20 08:21, Oleksandr Andrushchenko wrote:
>>>> From: Oleksandr Andrushchenko 
>>>
>>> Series pushed to:
>>>
>>> xen/tip.git for-linus-5.9
>>>
>> The top patch has strange title though:
>>
>> "Subject: [PATCH v2 5/5] drm/xen-front: Pass dumb buffer data offset to the 
>> backend"
>
> Oh, indeed. I had to repair it manually as it seems some mail system
> (probably on my end) mangled it a little bit.
>
> Will repair it.
>
Now everything is great,

Thank you!

>
> Juergen

Re: [PATCH v2 0/5] Fixes and improvements for Xen pvdrm

[Oleksandr Andrushchenko]

On 8/13/20 6:02 PM, Jürgen Groß wrote:
> On 13.08.20 08:21, Oleksandr Andrushchenko wrote:
>> From: Oleksandr Andrushchenko 
>
> Series pushed to:
>
> xen/tip.git for-linus-5.9
>
The top patch has strange title though:

"Subject: [PATCH v2 5/5] drm/xen-front: Pass dumb buffer data offset to the 
backend"

>
> Juergen

Thank you,

Oleksandr

Re: [PATCH 18/20] drm/xen: Introduce GEM object functions

[Oleksandr Andrushchenko]

Hi,

On 8/13/20 11:36 AM, Thomas Zimmermann wrote:
> GEM object functions deprecate several similar callback interfaces in
> struct drm_driver. This patch replaces the per-driver callbacks with
> per-instance callbacks in xen. The only exception is gem_prime_mmap,
> which is non-trivial to convert.
>
> Signed-off-by: Thomas Zimmermann 
> ---
>   drivers/gpu/drm/xen/xen_drm_front.c | 12 +---
>   drivers/gpu/drm/xen/xen_drm_front.h |  2 ++
>   drivers/gpu/drm/xen/xen_drm_front_gem.c | 15 +++
>   3 files changed, 18 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/gpu/drm/xen/xen_drm_front.c 
> b/drivers/gpu/drm/xen/xen_drm_front.c
> index 3e660fb111b3..bd9af1875af1 100644
> --- a/drivers/gpu/drm/xen/xen_drm_front.c
> +++ b/drivers/gpu/drm/xen/xen_drm_front.c
> @@ -433,7 +433,7 @@ static int xen_drm_drv_dumb_create(struct drm_file *filp,
>   return ret;
>   }
>   
> -static void xen_drm_drv_free_object_unlocked(struct drm_gem_object *obj)
> +void xen_drm_drv_free_object_unlocked(struct drm_gem_object *obj)

Can we please have naming consistent and name it as

xen_drm_front_drv_free_object_unlocked or any other name if this seems to be 
too long,

but starting with xen_drm_front_ as the rest of exported functions?

With this,

Acked-by: Oleksandr Andrushchenko 

Thank you,

Oleksandr

>   {
>   struct xen_drm_front_drm_info *drm_info = obj->dev->dev_private;
>   int idx;
> @@ -481,22 +481,12 @@ static const struct file_operations xen_drm_dev_fops = {
>   .mmap   = xen_drm_front_gem_mmap,
>   };
>   
> -static const struct vm_operations_struct xen_drm_drv_vm_ops = {
> - .open   = drm_gem_vm_open,
> - .close  = drm_gem_vm_close,
> -};
> -
>   static struct drm_driver xen_drm_driver = {
>   .driver_features   = DRIVER_GEM | DRIVER_MODESET | 
> DRIVER_ATOMIC,
>   .release   = xen_drm_drv_release,
> - .gem_vm_ops= _drm_drv_vm_ops,
> - .gem_free_object_unlocked  = xen_drm_drv_free_object_unlocked,
>   .prime_handle_to_fd= drm_gem_prime_handle_to_fd,
>   .prime_fd_to_handle= drm_gem_prime_fd_to_handle,
>   .gem_prime_import_sg_table = xen_drm_front_gem_import_sg_table,
> - .gem_prime_get_sg_table= xen_drm_front_gem_get_sg_table,
> - .gem_prime_vmap= xen_drm_front_gem_prime_vmap,
> - .gem_prime_vunmap  = xen_drm_front_gem_prime_vunmap,
>   .gem_prime_mmap= xen_drm_front_gem_prime_mmap,
>   .dumb_create   = xen_drm_drv_dumb_create,
>   .fops  = _drm_dev_fops,
> diff --git a/drivers/gpu/drm/xen/xen_drm_front.h 
> b/drivers/gpu/drm/xen/xen_drm_front.h
> index f92c258350ca..93e60c1db550 100644
> --- a/drivers/gpu/drm/xen/xen_drm_front.h
> +++ b/drivers/gpu/drm/xen/xen_drm_front.h
> @@ -160,4 +160,6 @@ int xen_drm_front_page_flip(struct xen_drm_front_info 
> *front_info,
>   void xen_drm_front_on_frame_done(struct xen_drm_front_info *front_info,
>int conn_idx, u64 fb_cookie);
>   
> +void xen_drm_drv_free_object_unlocked(struct drm_gem_object *obj);
> +
>   #endif /* __XEN_DRM_FRONT_H_ */
> diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
> b/drivers/gpu/drm/xen/xen_drm_front_gem.c
> index f0b85e094111..7b315c08bcfc 100644
> --- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
> +++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
> @@ -56,6 +56,19 @@ static void gem_free_pages_array(struct xen_gem_object 
> *xen_obj)
>   xen_obj->pages = NULL;
>   }
>   
> +static const struct vm_operations_struct xen_drm_drv_vm_ops = {
> + .open   = drm_gem_vm_open,
> + .close  = drm_gem_vm_close,
> +};
> +
> +static const struct drm_gem_object_funcs xen_drm_front_gem_object_funcs = {
> + .free = xen_drm_drv_free_object_unlocked,
> + .get_sg_table = xen_drm_front_gem_get_sg_table,
> + .vmap = xen_drm_front_gem_prime_vmap,
> + .vunmap = xen_drm_front_gem_prime_vunmap,
> + .vm_ops = _drm_drv_vm_ops,
> +};
> +
>   static struct xen_gem_object *gem_create_obj(struct drm_device *dev,
>size_t size)
>   {
> @@ -66,6 +79,8 @@ static struct xen_gem_object *gem_create_obj(struct 
> drm_device *dev,
>   if (!xen_obj)
>   return ERR_PTR(-ENOMEM);
>   
> + xen_obj->base.funcs = _drm_front_gem_object_funcs;
> +
>   ret = drm_gem_object_init(dev, _obj->base, size);
>   if (ret < 0) {
>   kfree(xen_obj);

Re: [PATCH v2 0/5] Fixes and improvements for Xen pvdrm

[Oleksandr Andrushchenko]

On 8/13/20 10:05 AM, Jürgen Groß wrote:
> On 13.08.20 08:32, Oleksandr Andrushchenko wrote:
>> Juergen, Boris,
>>
>> can we please merge these via Xen Linux tree as I have collected enough 
>> Ack/R-b?
>>
>> The series has DRM patches, but those anyway are Xen related, so I think
>>
>> this should be fine from DRI point of view.
>
> Yes, fine with me.
Great, thank you
>
>
> Juergen
>
>>
>> Thank you,
>>
>> Oleksandr
>>
>> On 8/13/20 9:21 AM, Oleksandr Andrushchenko wrote:
>>> From: Oleksandr Andrushchenko 
>>>
>>> Hello,
>>>
>>> This series contains an assorted set of fixes and improvements for
>>> the Xen para-virtualized display driver and grant device driver which
>>> I have collected over the last couple of months:
>>>
>>> 1. Minor fixes to grant device driver and drm/xen-front.
>>>
>>> 2. New format (YUYV) added to the list of the PV DRM supported formats
>>> which allows the driver to be used in zero-copying use-cases when
>>> a camera device is the source of the dma-bufs.
>>>
>>> 3. Synchronization with the latest para-virtualized protocol definition
>>> in Xen [1].
>>>
>>> 4. SGT offset is now propagated to the backend: while importing a dmabuf
>>> it is possible that the data of the buffer is put with offset which is
>>> indicated by the SGT offset. This is needed for some GPUs which have
>>> non-zero offset.
>>>
>>> Thank you,
>>> Oleksandr Andrushchenko
>>>
>>> [1] 
>>> https://urldefense.com/v3/__https://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=c27a184225eab54d20435c8cab5ad0ef384dc2c0__;!!GF_29dbcQIUBPA!iAHOdk4M167VNM1AypMGVmyKJu-iqC9e5cXyu6N595Np3iyIZDnZl0MIBX3IROJSD1GSMX_GfQ$
>>>  [xenbits[.]xen[.]org]
>>>
>>> Changes since v1:
>>> =
>>>
>>> 1. Removed patch which adds EDID to PV DRM as it needs more time for review:
>>> "5. Version 2 of the Xen displif protocol adds XENDISPL_OP_GET_EDID
>>> request which allows frontends to request EDID structure per
>>> connector. This request is optional and if not supported by the
>>> backend then visible area is still defined by the relevant
>>> XenStore's "resolution" property.
>>> If backend provides EDID with XENDISPL_OP_GET_EDID request then
>>> its values must take precedence over the resolutions defined in
>>> XenStore."
>>> I will send this as a dedicated patch.
>>>
>>> 2. Added missing CC stable for the patches with fixes
>>>
>>> Oleksandr Andrushchenko (5):
>>>     xen/gntdev: Fix dmabuf import with non-zero sgt offset
>>>     drm/xen-front: Fix misused IS_ERR_OR_NULL checks
>>>     drm/xen-front: Add YUYV to supported formats
>>>     xen: Sync up with the canonical protocol definition in Xen
>>>     drm/xen-front: Pass dumb buffer data offset to the backend
>>>
>>>    drivers/gpu/drm/xen/xen_drm_front.c  | 10 +--
>>>    drivers/gpu/drm/xen/xen_drm_front.h  |  2 +-
>>>    drivers/gpu/drm/xen/xen_drm_front_conn.c |  1 +
>>>    drivers/gpu/drm/xen/xen_drm_front_gem.c  | 11 +--
>>>    drivers/gpu/drm/xen/xen_drm_front_kms.c  |  2 +-
>>>    drivers/xen/gntdev-dmabuf.c  |  8 +++
>>>    include/xen/interface/io/displif.h   | 91 +++-
>>>    7 files changed, 111 insertions(+), 14 deletions(-)
>>>
>>
>

Re: [PATCH v2 0/5] Fixes and improvements for Xen pvdrm

[Oleksandr Andrushchenko]

Juergen, Boris,

can we please merge these via Xen Linux tree as I have collected enough Ack/R-b?

The series has DRM patches, but those anyway are Xen related, so I think

this should be fine from DRI point of view.

Thank you,

Oleksandr

On 8/13/20 9:21 AM, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko 
>
> Hello,
>
> This series contains an assorted set of fixes and improvements for
> the Xen para-virtualized display driver and grant device driver which
> I have collected over the last couple of months:
>
> 1. Minor fixes to grant device driver and drm/xen-front.
>
> 2. New format (YUYV) added to the list of the PV DRM supported formats
> which allows the driver to be used in zero-copying use-cases when
> a camera device is the source of the dma-bufs.
>
> 3. Synchronization with the latest para-virtualized protocol definition
> in Xen [1].
>
> 4. SGT offset is now propagated to the backend: while importing a dmabuf
> it is possible that the data of the buffer is put with offset which is
> indicated by the SGT offset. This is needed for some GPUs which have
> non-zero offset.
>
> Thank you,
> Oleksandr Andrushchenko
>
> [1] 
> https://urldefense.com/v3/__https://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=c27a184225eab54d20435c8cab5ad0ef384dc2c0__;!!GF_29dbcQIUBPA!iAHOdk4M167VNM1AypMGVmyKJu-iqC9e5cXyu6N595Np3iyIZDnZl0MIBX3IROJSD1GSMX_GfQ$
>  [xenbits[.]xen[.]org]
>
> Changes since v1:
> =
>
> 1. Removed patch which adds EDID to PV DRM as it needs more time for review:
> "5. Version 2 of the Xen displif protocol adds XENDISPL_OP_GET_EDID
> request which allows frontends to request EDID structure per
> connector. This request is optional and if not supported by the
> backend then visible area is still defined by the relevant
> XenStore's "resolution" property.
> If backend provides EDID with XENDISPL_OP_GET_EDID request then
> its values must take precedence over the resolutions defined in
> XenStore."
> I will send this as a dedicated patch.
>
> 2. Added missing CC stable for the patches with fixes
>
> Oleksandr Andrushchenko (5):
>xen/gntdev: Fix dmabuf import with non-zero sgt offset
>drm/xen-front: Fix misused IS_ERR_OR_NULL checks
>drm/xen-front: Add YUYV to supported formats
>xen: Sync up with the canonical protocol definition in Xen
>drm/xen-front: Pass dumb buffer data offset to the backend
>
>   drivers/gpu/drm/xen/xen_drm_front.c  | 10 +--
>   drivers/gpu/drm/xen/xen_drm_front.h  |  2 +-
>   drivers/gpu/drm/xen/xen_drm_front_conn.c |  1 +
>   drivers/gpu/drm/xen/xen_drm_front_gem.c  | 11 +--
>   drivers/gpu/drm/xen/xen_drm_front_kms.c  |  2 +-
>   drivers/xen/gntdev-dmabuf.c  |  8 +++
>   include/xen/interface/io/displif.h   | 91 +++-
>   7 files changed, 111 insertions(+), 14 deletions(-)
>

[PATCH v2 5/5] drm/xen-front: Pass dumb buffer data offset to the backend

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

While importing a dmabuf it is possible that the data of the buffer
is put with offset which is indicated by the SGT offset.
Respect the offset value and forward it to the backend.

Signed-off-by: Oleksandr Andrushchenko 
Acked-by: Noralf Trønnes 
---
 drivers/gpu/drm/xen/xen_drm_front.c | 6 --
 drivers/gpu/drm/xen/xen_drm_front.h | 2 +-
 drivers/gpu/drm/xen/xen_drm_front_gem.c | 3 ++-
 3 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front.c 
b/drivers/gpu/drm/xen/xen_drm_front.c
index 51818e76facd..3dd56794593a 100644
--- a/drivers/gpu/drm/xen/xen_drm_front.c
+++ b/drivers/gpu/drm/xen/xen_drm_front.c
@@ -157,7 +157,8 @@ int xen_drm_front_mode_set(struct 
xen_drm_front_drm_pipeline *pipeline,
 
 int xen_drm_front_dbuf_create(struct xen_drm_front_info *front_info,
  u64 dbuf_cookie, u32 width, u32 height,
- u32 bpp, u64 size, struct page **pages)
+ u32 bpp, u64 size, u32 offset,
+ struct page **pages)
 {
struct xen_drm_front_evtchnl *evtchnl;
struct xen_drm_front_dbuf *dbuf;
@@ -194,6 +195,7 @@ int xen_drm_front_dbuf_create(struct xen_drm_front_info 
*front_info,
req->op.dbuf_create.gref_directory =
xen_front_pgdir_shbuf_get_dir_start(>shbuf);
req->op.dbuf_create.buffer_sz = size;
+   req->op.dbuf_create.data_ofs = offset;
req->op.dbuf_create.dbuf_cookie = dbuf_cookie;
req->op.dbuf_create.width = width;
req->op.dbuf_create.height = height;
@@ -408,7 +410,7 @@ static int xen_drm_drv_dumb_create(struct drm_file *filp,
ret = xen_drm_front_dbuf_create(drm_info->front_info,
xen_drm_front_dbuf_to_cookie(obj),
args->width, args->height, args->bpp,
-   args->size,
+   args->size, 0,
xen_drm_front_gem_get_pages(obj));
if (ret)
goto fail_backend;
diff --git a/drivers/gpu/drm/xen/xen_drm_front.h 
b/drivers/gpu/drm/xen/xen_drm_front.h
index f92c258350ca..54486d89650e 100644
--- a/drivers/gpu/drm/xen/xen_drm_front.h
+++ b/drivers/gpu/drm/xen/xen_drm_front.h
@@ -145,7 +145,7 @@ int xen_drm_front_mode_set(struct 
xen_drm_front_drm_pipeline *pipeline,
 
 int xen_drm_front_dbuf_create(struct xen_drm_front_info *front_info,
  u64 dbuf_cookie, u32 width, u32 height,
- u32 bpp, u64 size, struct page **pages);
+ u32 bpp, u64 size, u32 offset, struct page 
**pages);
 
 int xen_drm_front_fb_attach(struct xen_drm_front_info *front_info,
u64 dbuf_cookie, u64 fb_cookie, u32 width,
diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index 4ec8a49241e1..39ff95b75357 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -210,7 +210,8 @@ xen_drm_front_gem_import_sg_table(struct drm_device *dev,
 
ret = xen_drm_front_dbuf_create(drm_info->front_info,

xen_drm_front_dbuf_to_cookie(_obj->base),
-   0, 0, 0, size, xen_obj->pages);
+   0, 0, 0, size, sgt->sgl->offset,
+   xen_obj->pages);
if (ret < 0)
return ERR_PTR(ret);
 
-- 
2.17.1

[PATCH v2 3/5] drm/xen-front: Add YUYV to supported formats

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Add YUYV to supported formats, so the frontend can work with the
formats used by cameras and other HW.

Signed-off-by: Oleksandr Andrushchenko 
Acked-by: Noralf Trønnes 
---
 drivers/gpu/drm/xen/xen_drm_front_conn.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/gpu/drm/xen/xen_drm_front_conn.c 
b/drivers/gpu/drm/xen/xen_drm_front_conn.c
index 459702fa990e..44f1f70c0aed 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_conn.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_conn.c
@@ -33,6 +33,7 @@ static const u32 plane_formats[] = {
DRM_FORMAT_ARGB,
DRM_FORMAT_XRGB1555,
DRM_FORMAT_ARGB1555,
+   DRM_FORMAT_YUYV,
 };
 
 const u32 *xen_drm_front_conn_get_formats(int *format_count)
-- 
2.17.1

[PATCH v2 4/5] xen: Sync up with the canonical protocol definition in Xen

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

This is the sync up with the canonical definition of the
display protocol in Xen.

1. Add protocol version as an integer

Version string, which is in fact an integer, is hard to handle in the
code that supports different protocol versions. To simplify that
also add the version as an integer.

2. Pass buffer offset with XENDISPL_OP_DBUF_CREATE

There are cases when display data buffer is created with non-zero
offset to the data start. Handle such cases and provide that offset
while creating a display buffer.

3. Add XENDISPL_OP_GET_EDID command

Add an optional request for reading Extended Display Identification
Data (EDID) structure which allows better configuration of the
display connectors over the configuration set in XenStore.
With this change connectors may have multiple resolutions defined
with respect to detailed timing definitions and additional properties
normally provided by displays.

If this request is not supported by the backend then visible area
is defined by the relevant XenStore's "resolution" property.

If backend provides extended display identification data (EDID) with
XENDISPL_OP_GET_EDID request then EDID values must take precedence
over the resolutions defined in XenStore.

4. Bump protocol version to 2.

Signed-off-by: Oleksandr Andrushchenko 
Reviewed-by: Juergen Gross 
---
 include/xen/interface/io/displif.h | 91 +-
 1 file changed, 88 insertions(+), 3 deletions(-)

diff --git a/include/xen/interface/io/displif.h 
b/include/xen/interface/io/displif.h
index fdc279dc4a88..d43ca0361f86 100644
--- a/include/xen/interface/io/displif.h
+++ b/include/xen/interface/io/displif.h
@@ -38,7 +38,8 @@
  *   Protocol version
  **
  */
-#define XENDISPL_PROTOCOL_VERSION  "1"
+#define XENDISPL_PROTOCOL_VERSION  "2"
+#define XENDISPL_PROTOCOL_VERSION_INT   2
 
 /*
  **
@@ -202,6 +203,9 @@
  *  Width and height of the connector in pixels separated by
  *  XENDISPL_RESOLUTION_SEPARATOR. This defines visible area of the
  *  display.
+ *  If backend provides extended display identification data (EDID) with
+ *  XENDISPL_OP_GET_EDID request then EDID values must take precedence
+ *  over the resolutions defined here.
  *
  *-- Connector Request Transport Parameters ---
  *
@@ -349,6 +353,8 @@
 #define XENDISPL_OP_FB_DETACH  0x13
 #define XENDISPL_OP_SET_CONFIG 0x14
 #define XENDISPL_OP_PG_FLIP0x15
+/* The below command is available in protocol version 2 and above. */
+#define XENDISPL_OP_GET_EDID   0x16
 
 /*
  **
@@ -377,6 +383,10 @@
 #define XENDISPL_FIELD_BE_ALLOC"be-alloc"
 #define XENDISPL_FIELD_UNIQUE_ID   "unique-id"
 
+#define XENDISPL_EDID_BLOCK_SIZE   128
+#define XENDISPL_EDID_BLOCK_COUNT  256
+#define XENDISPL_EDID_MAX_SIZE (XENDISPL_EDID_BLOCK_SIZE * 
XENDISPL_EDID_BLOCK_COUNT)
+
 /*
  **
  *  STATUS RETURN CODES
@@ -451,7 +461,9 @@
  * +++++
  * |   gref_directory  | 40
  * +++++
- * | reserved  | 44
+ * | data_ofs  | 44
+ * +++++
+ * | reserved  | 48
  * +++++
  * |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/|
  * +++++
@@ -494,6 +506,7 @@
  *   buffer size (buffer_sz) exceeds what can be addressed by this single page,
  *   then reference to the next page must be supplied (see gref_dir_next_page
  *   below)
+ * data_ofs - uint32_t, offset of the data in the buffer, octets
  */
 
 #define XENDISPL_DBUF_FLG_REQ_ALLOC(1 << 0)
@@ -506,6 +519,7 @@ struct xendispl_dbuf_create_req {
uint32_t buffer_sz;
uint32_t flags;
grant_ref_t gref_directory;
+   uint32_t data_ofs;
 };
 
 /*
@@ -731,6 +745,44 @@ struct xendispl_page_flip_req {
uint64_t fb_cookie;
 };
 
+/*
+ * Request EDID - request EDID describing current connector:
+ * 01 2   3octet
+ * +++++
+ * |  

[PATCH v2 0/5] Fixes and improvements for Xen pvdrm

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Hello,

This series contains an assorted set of fixes and improvements for
the Xen para-virtualized display driver and grant device driver which
I have collected over the last couple of months:

1. Minor fixes to grant device driver and drm/xen-front.

2. New format (YUYV) added to the list of the PV DRM supported formats
which allows the driver to be used in zero-copying use-cases when
a camera device is the source of the dma-bufs.

3. Synchronization with the latest para-virtualized protocol definition
in Xen [1].

4. SGT offset is now propagated to the backend: while importing a dmabuf
it is possible that the data of the buffer is put with offset which is
indicated by the SGT offset. This is needed for some GPUs which have
non-zero offset.

Thank you,
Oleksandr Andrushchenko

[1] 
https://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=c27a184225eab54d20435c8cab5ad0ef384dc2c0

Changes since v1:
=

1. Removed patch which adds EDID to PV DRM as it needs more time for review:
"5. Version 2 of the Xen displif protocol adds XENDISPL_OP_GET_EDID
request which allows frontends to request EDID structure per
connector. This request is optional and if not supported by the
backend then visible area is still defined by the relevant
XenStore's "resolution" property.
If backend provides EDID with XENDISPL_OP_GET_EDID request then
its values must take precedence over the resolutions defined in
XenStore."
I will send this as a dedicated patch.

2. Added missing CC stable for the patches with fixes

Oleksandr Andrushchenko (5):
  xen/gntdev: Fix dmabuf import with non-zero sgt offset
  drm/xen-front: Fix misused IS_ERR_OR_NULL checks
  drm/xen-front: Add YUYV to supported formats
  xen: Sync up with the canonical protocol definition in Xen
  drm/xen-front: Pass dumb buffer data offset to the backend

 drivers/gpu/drm/xen/xen_drm_front.c  | 10 +--
 drivers/gpu/drm/xen/xen_drm_front.h  |  2 +-
 drivers/gpu/drm/xen/xen_drm_front_conn.c |  1 +
 drivers/gpu/drm/xen/xen_drm_front_gem.c  | 11 +--
 drivers/gpu/drm/xen/xen_drm_front_kms.c  |  2 +-
 drivers/xen/gntdev-dmabuf.c  |  8 +++
 include/xen/interface/io/displif.h   | 91 +++-
 7 files changed, 111 insertions(+), 14 deletions(-)

-- 
2.17.1

[PATCH v2 2/5] drm/xen-front: Fix misused IS_ERR_OR_NULL checks

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

The patch c575b7eeb89f: "drm/xen-front: Add support for Xen PV
display frontend" from Apr 3, 2018, leads to the following static
checker warning:

drivers/gpu/drm/xen/xen_drm_front_gem.c:140 xen_drm_front_gem_create()
warn: passing zero to 'ERR_CAST'

drivers/gpu/drm/xen/xen_drm_front_gem.c
   133  struct drm_gem_object *xen_drm_front_gem_create(struct drm_device *dev,
   134  size_t size)
   135  {
   136  struct xen_gem_object *xen_obj;
   137
   138  xen_obj = gem_create(dev, size);
   139  if (IS_ERR_OR_NULL(xen_obj))
   140  return ERR_CAST(xen_obj);

Fix this and the rest of misused places with IS_ERR_OR_NULL in the
driver.

Fixes:  c575b7eeb89f: "drm/xen-front: Add support for Xen PV display frontend"

Signed-off-by: Oleksandr Andrushchenko 
Reported-by: Dan Carpenter 
Reviewed-by: Dan Carpenter 
Cc: 
---
 drivers/gpu/drm/xen/xen_drm_front.c | 4 ++--
 drivers/gpu/drm/xen/xen_drm_front_gem.c | 8 
 drivers/gpu/drm/xen/xen_drm_front_kms.c | 2 +-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front.c 
b/drivers/gpu/drm/xen/xen_drm_front.c
index 1fd458e877ca..51818e76facd 100644
--- a/drivers/gpu/drm/xen/xen_drm_front.c
+++ b/drivers/gpu/drm/xen/xen_drm_front.c
@@ -400,8 +400,8 @@ static int xen_drm_drv_dumb_create(struct drm_file *filp,
args->size = args->pitch * args->height;
 
obj = xen_drm_front_gem_create(dev, args->size);
-   if (IS_ERR_OR_NULL(obj)) {
-   ret = PTR_ERR_OR_ZERO(obj);
+   if (IS_ERR(obj)) {
+   ret = PTR_ERR(obj);
goto fail;
}
 
diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index f0b85e094111..4ec8a49241e1 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -83,7 +83,7 @@ static struct xen_gem_object *gem_create(struct drm_device 
*dev, size_t size)
 
size = round_up(size, PAGE_SIZE);
xen_obj = gem_create_obj(dev, size);
-   if (IS_ERR_OR_NULL(xen_obj))
+   if (IS_ERR(xen_obj))
return xen_obj;
 
if (drm_info->front_info->cfg.be_alloc) {
@@ -117,7 +117,7 @@ static struct xen_gem_object *gem_create(struct drm_device 
*dev, size_t size)
 */
xen_obj->num_pages = DIV_ROUND_UP(size, PAGE_SIZE);
xen_obj->pages = drm_gem_get_pages(_obj->base);
-   if (IS_ERR_OR_NULL(xen_obj->pages)) {
+   if (IS_ERR(xen_obj->pages)) {
ret = PTR_ERR(xen_obj->pages);
xen_obj->pages = NULL;
goto fail;
@@ -136,7 +136,7 @@ struct drm_gem_object *xen_drm_front_gem_create(struct 
drm_device *dev,
struct xen_gem_object *xen_obj;
 
xen_obj = gem_create(dev, size);
-   if (IS_ERR_OR_NULL(xen_obj))
+   if (IS_ERR(xen_obj))
return ERR_CAST(xen_obj);
 
return _obj->base;
@@ -194,7 +194,7 @@ xen_drm_front_gem_import_sg_table(struct drm_device *dev,
 
size = attach->dmabuf->size;
xen_obj = gem_create_obj(dev, size);
-   if (IS_ERR_OR_NULL(xen_obj))
+   if (IS_ERR(xen_obj))
return ERR_CAST(xen_obj);
 
ret = gem_alloc_pages_array(xen_obj, size);
diff --git a/drivers/gpu/drm/xen/xen_drm_front_kms.c 
b/drivers/gpu/drm/xen/xen_drm_front_kms.c
index 78096bbcd226..ef11b1e4de39 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_kms.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_kms.c
@@ -60,7 +60,7 @@ fb_create(struct drm_device *dev, struct drm_file *filp,
int ret;
 
fb = drm_gem_fb_create_with_funcs(dev, filp, mode_cmd, _funcs);
-   if (IS_ERR_OR_NULL(fb))
+   if (IS_ERR(fb))
return fb;
 
gem_obj = fb->obj[0];
-- 
2.17.1

[PATCH v2 1/5] xen/gntdev: Fix dmabuf import with non-zero sgt offset

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

It is possible that the scatter-gather table during dmabuf import has
non-zero offset of the data, but user-space doesn't expect that.
Fix this by failing the import, so user-space doesn't access wrong data.

Fixes: bf8dc55b1358 ("xen/gntdev: Implement dma-buf import functionality")

Signed-off-by: Oleksandr Andrushchenko 
Acked-by: Juergen Gross 
Cc: 
---
 drivers/xen/gntdev-dmabuf.c | 8 
 1 file changed, 8 insertions(+)

diff --git a/drivers/xen/gntdev-dmabuf.c b/drivers/xen/gntdev-dmabuf.c
index 75d3bb948bf3..b1b6eebafd5d 100644
--- a/drivers/xen/gntdev-dmabuf.c
+++ b/drivers/xen/gntdev-dmabuf.c
@@ -613,6 +613,14 @@ dmabuf_imp_to_refs(struct gntdev_dmabuf_priv *priv, struct 
device *dev,
goto fail_detach;
}
 
+   /* Check that we have zero offset. */
+   if (sgt->sgl->offset) {
+   ret = ERR_PTR(-EINVAL);
+   pr_debug("DMA buffer has %d bytes offset, user-space expects 
0\n",
+sgt->sgl->offset);
+   goto fail_unmap;
+   }
+
/* Check number of pages that imported buffer has. */
if (attach->dmabuf->size != gntdev_dmabuf->nr_pages << PAGE_SHIFT) {
ret = ERR_PTR(-EINVAL);
-- 
2.17.1

Re: [PATCH 2/6] drm/xen-front: Fix misused IS_ERR_OR_NULL checks

[Oleksandr Andrushchenko]

On 8/4/20 9:12 AM, Jürgen Groß wrote:
> On 31.07.20 14:51, Oleksandr Andrushchenko wrote:
>> From: Oleksandr Andrushchenko 
>>
>> The patch c575b7eeb89f: "drm/xen-front: Add support for Xen PV
>> display frontend" from Apr 3, 2018, leads to the following static
>> checker warning:
>>
>> drivers/gpu/drm/xen/xen_drm_front_gem.c:140 xen_drm_front_gem_create()
>> warn: passing zero to 'ERR_CAST'
>>
>> drivers/gpu/drm/xen/xen_drm_front_gem.c
>>     133  struct drm_gem_object *xen_drm_front_gem_create(struct drm_device 
>> *dev,
>>     134  size_t size)
>>     135  {
>>     136  struct xen_gem_object *xen_obj;
>>     137
>>     138  xen_obj = gem_create(dev, size);
>>     139  if (IS_ERR_OR_NULL(xen_obj))
>>     140  return ERR_CAST(xen_obj);
>>
>> Fix this and the rest of misused places with IS_ERR_OR_NULL in the
>> driver.
>>
>> Fixes:  c575b7eeb89f: "drm/xen-front: Add support for Xen PV display 
>> frontend"
>
> Again forgot to Cc stable?

I was just not sure if these minor fixes need to go the stable, but I will add

Thank you

>
>
> Juergen

Re: [PATCH 1/6] xen/gntdev: Fix dmabuf import with non-zero sgt offset

[Oleksandr Andrushchenko]

On 8/4/20 9:11 AM, Jürgen Groß wrote:
> On 31.07.20 14:51, Oleksandr Andrushchenko wrote:
>> From: Oleksandr Andrushchenko 
>>
>> It is possible that the scatter-gather table during dmabuf import has
>> non-zero offset of the data, but user-space doesn't expect that.
>> Fix this by failing the import, so user-space doesn't access wrong data.
>>
>> Fixes: 37ccb44d0b00 ("xen/gntdev: Implement dma-buf import functionality")
>
> I can't find this commit in the tree. Did you mean bf8dc55b1358?
I'll double-check, thank you
>
> And don't you want to Cc stable for this patch, too?
Hm, yes, sounds reasonable
>
>>
>> Signed-off-by: Oleksandr Andrushchenko 
>
> Acked-by: Juergen Gross 
>
>
> Juergen

[PATCH 1/6] xen/gntdev: Fix dmabuf import with non-zero sgt offset

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

It is possible that the scatter-gather table during dmabuf import has
non-zero offset of the data, but user-space doesn't expect that.
Fix this by failing the import, so user-space doesn't access wrong data.

Fixes: 37ccb44d0b00 ("xen/gntdev: Implement dma-buf import functionality")

Signed-off-by: Oleksandr Andrushchenko 
---
 drivers/xen/gntdev-dmabuf.c | 8 
 1 file changed, 8 insertions(+)

diff --git a/drivers/xen/gntdev-dmabuf.c b/drivers/xen/gntdev-dmabuf.c
index 75d3bb948bf3..b1b6eebafd5d 100644
--- a/drivers/xen/gntdev-dmabuf.c
+++ b/drivers/xen/gntdev-dmabuf.c
@@ -613,6 +613,14 @@ dmabuf_imp_to_refs(struct gntdev_dmabuf_priv *priv, struct 
device *dev,
goto fail_detach;
}
 
+   /* Check that we have zero offset. */
+   if (sgt->sgl->offset) {
+   ret = ERR_PTR(-EINVAL);
+   pr_debug("DMA buffer has %d bytes offset, user-space expects 
0\n",
+sgt->sgl->offset);
+   goto fail_unmap;
+   }
+
/* Check number of pages that imported buffer has. */
if (attach->dmabuf->size != gntdev_dmabuf->nr_pages << PAGE_SHIFT) {
ret = ERR_PTR(-EINVAL);
-- 
2.17.1

[PATCH 5/6] drm/xen-front: Pass dumb buffer data offset to the backend

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

While importing a dmabuf it is possible that the data of the buffer
is put with offset which is indicated by the SGT offset.
Respect the offset value and forward it to the backend.

Signed-off-by: Oleksandr Andrushchenko 
---
 drivers/gpu/drm/xen/xen_drm_front.c | 6 --
 drivers/gpu/drm/xen/xen_drm_front.h | 2 +-
 drivers/gpu/drm/xen/xen_drm_front_gem.c | 3 ++-
 3 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front.c 
b/drivers/gpu/drm/xen/xen_drm_front.c
index 88db2726e8ce..013c9e0e412c 100644
--- a/drivers/gpu/drm/xen/xen_drm_front.c
+++ b/drivers/gpu/drm/xen/xen_drm_front.c
@@ -157,7 +157,8 @@ int xen_drm_front_mode_set(struct 
xen_drm_front_drm_pipeline *pipeline,
 
 int xen_drm_front_dbuf_create(struct xen_drm_front_info *front_info,
  u64 dbuf_cookie, u32 width, u32 height,
- u32 bpp, u64 size, struct page **pages)
+ u32 bpp, u64 size, u32 offset,
+ struct page **pages)
 {
struct xen_drm_front_evtchnl *evtchnl;
struct xen_drm_front_dbuf *dbuf;
@@ -194,6 +195,7 @@ int xen_drm_front_dbuf_create(struct xen_drm_front_info 
*front_info,
req->op.dbuf_create.gref_directory =
xen_front_pgdir_shbuf_get_dir_start(>shbuf);
req->op.dbuf_create.buffer_sz = size;
+   req->op.dbuf_create.data_ofs = offset;
req->op.dbuf_create.dbuf_cookie = dbuf_cookie;
req->op.dbuf_create.width = width;
req->op.dbuf_create.height = height;
@@ -408,7 +410,7 @@ static int xen_drm_drv_dumb_create(struct drm_file *filp,
ret = xen_drm_front_dbuf_create(drm_info->front_info,
xen_drm_front_dbuf_to_cookie(obj),
args->width, args->height, args->bpp,
-   args->size,
+   args->size, 0,
xen_drm_front_gem_get_pages(obj));
if (ret)
goto fail_backend;
diff --git a/drivers/gpu/drm/xen/xen_drm_front.h 
b/drivers/gpu/drm/xen/xen_drm_front.h
index f92c258350ca..54486d89650e 100644
--- a/drivers/gpu/drm/xen/xen_drm_front.h
+++ b/drivers/gpu/drm/xen/xen_drm_front.h
@@ -145,7 +145,7 @@ int xen_drm_front_mode_set(struct 
xen_drm_front_drm_pipeline *pipeline,
 
 int xen_drm_front_dbuf_create(struct xen_drm_front_info *front_info,
  u64 dbuf_cookie, u32 width, u32 height,
- u32 bpp, u64 size, struct page **pages);
+ u32 bpp, u64 size, u32 offset, struct page 
**pages);
 
 int xen_drm_front_fb_attach(struct xen_drm_front_info *front_info,
u64 dbuf_cookie, u64 fb_cookie, u32 width,
diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index 4ec8a49241e1..39ff95b75357 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -210,7 +210,8 @@ xen_drm_front_gem_import_sg_table(struct drm_device *dev,
 
ret = xen_drm_front_dbuf_create(drm_info->front_info,

xen_drm_front_dbuf_to_cookie(_obj->base),
-   0, 0, 0, size, xen_obj->pages);
+   0, 0, 0, size, sgt->sgl->offset,
+   xen_obj->pages);
if (ret < 0)
return ERR_PTR(ret);
 
-- 
2.17.1

[PATCH 6/6] drm/xen-front: Add support for EDID based configuration

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Version 2 of the Xen displif protocol adds XENDISPL_OP_GET_EDID
request which allows frontends to request EDID structure per
connector. This request is optional and if not supported by the
backend then visible area is still defined by the relevant
XenStore's "resolution" property.
If backend provides EDID with XENDISPL_OP_GET_EDID request then
its values must take precedence over the resolutions defined in
XenStore.

Signed-off-by: Oleksandr Andrushchenko 
---
 drivers/gpu/drm/xen/xen_drm_front.c | 62 
 drivers/gpu/drm/xen/xen_drm_front.h |  9 ++-
 drivers/gpu/drm/xen/xen_drm_front_cfg.c | 82 +
 drivers/gpu/drm/xen/xen_drm_front_cfg.h |  7 ++
 drivers/gpu/drm/xen/xen_drm_front_conn.c| 26 ++-
 drivers/gpu/drm/xen/xen_drm_front_evtchnl.c |  3 +
 drivers/gpu/drm/xen/xen_drm_front_evtchnl.h |  3 +
 drivers/gpu/drm/xen/xen_drm_front_kms.c |  5 ++
 8 files changed, 194 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front.c 
b/drivers/gpu/drm/xen/xen_drm_front.c
index 013c9e0e412c..cc5981bdbfb3 100644
--- a/drivers/gpu/drm/xen/xen_drm_front.c
+++ b/drivers/gpu/drm/xen/xen_drm_front.c
@@ -381,6 +381,59 @@ void xen_drm_front_on_frame_done(struct xen_drm_front_info 
*front_info,
fb_cookie);
 }
 
+int xen_drm_front_get_edid(struct xen_drm_front_info *front_info,
+  int conn_idx, struct page **pages,
+  u32 buffer_sz, u32 *edid_sz)
+{
+   struct xen_drm_front_evtchnl *evtchnl;
+   struct xen_front_pgdir_shbuf_cfg buf_cfg;
+   struct xen_front_pgdir_shbuf shbuf;
+   struct xendispl_req *req;
+   unsigned long flags;
+   int ret;
+
+   if (unlikely(conn_idx >= front_info->num_evt_pairs))
+   return -EINVAL;
+
+   memset(_cfg, 0, sizeof(buf_cfg));
+   buf_cfg.xb_dev = front_info->xb_dev;
+   buf_cfg.num_pages = DIV_ROUND_UP(buffer_sz, PAGE_SIZE);
+   buf_cfg.pages = pages;
+   buf_cfg.pgdir = 
+   buf_cfg.be_alloc = false;
+
+   ret = xen_front_pgdir_shbuf_alloc(_cfg);
+   if (ret < 0)
+   return ret;
+
+   evtchnl = _info->evt_pairs[conn_idx].req;
+
+   mutex_lock(>u.req.req_io_lock);
+
+   spin_lock_irqsave(_info->io_lock, flags);
+   req = be_prepare_req(evtchnl, XENDISPL_OP_GET_EDID);
+   req->op.get_edid.gref_directory =
+   xen_front_pgdir_shbuf_get_dir_start();
+   req->op.get_edid.buffer_sz = buffer_sz;
+
+   ret = be_stream_do_io(evtchnl, req);
+   spin_unlock_irqrestore(_info->io_lock, flags);
+
+   if (ret < 0)
+   goto fail;
+
+   ret = be_stream_wait_io(evtchnl);
+   if (ret < 0)
+   goto fail;
+
+   *edid_sz = evtchnl->u.req.resp.get_edid.edid_sz;
+
+fail:
+   mutex_unlock(>u.req.req_io_lock);
+   xen_front_pgdir_shbuf_free();
+   return ret;
+}
+
 static int xen_drm_drv_dumb_create(struct drm_file *filp,
   struct drm_device *dev,
   struct drm_mode_create_dumb *args)
@@ -466,6 +519,7 @@ static void xen_drm_drv_release(struct drm_device *dev)
xenbus_switch_state(front_info->xb_dev,
XenbusStateInitialising);
 
+   xen_drm_front_cfg_free(front_info, _info->cfg);
kfree(drm_info);
 }
 
@@ -562,6 +616,7 @@ static int xen_drm_drv_init(struct xen_drm_front_info 
*front_info)
drm_mode_config_cleanup(drm_dev);
drm_dev_put(drm_dev);
 fail:
+   xen_drm_front_cfg_free(front_info, _info->cfg);
kfree(drm_info);
return ret;
 }
@@ -622,7 +677,14 @@ static int displback_initwait(struct xen_drm_front_info 
*front_info)
 
 static int displback_connect(struct xen_drm_front_info *front_info)
 {
+   int ret;
+
xen_drm_front_evtchnl_set_state(front_info, EVTCHNL_STATE_CONNECTED);
+
+   /* We are all set to read additional configuration from the backend. */
+   ret = xen_drm_front_cfg_tail(front_info, _info->cfg);
+   if (ret < 0)
+   return ret;
return xen_drm_drv_init(front_info);
 }
 
diff --git a/drivers/gpu/drm/xen/xen_drm_front.h 
b/drivers/gpu/drm/xen/xen_drm_front.h
index 54486d89650e..be0c982f4d82 100644
--- a/drivers/gpu/drm/xen/xen_drm_front.h
+++ b/drivers/gpu/drm/xen/xen_drm_front.h
@@ -112,9 +112,12 @@ struct xen_drm_front_drm_pipeline {
struct drm_simple_display_pipe pipe;
 
struct drm_connector conn;
-   /* These are only for connector mode checking */
+   /* These are only for connector mode checking if no EDID present */
int width, height;
 
+   /* Is not NULL if EDID is used for connector configuration. */
+   struct edid *edid;
+
struct drm_pending_vblank_event *pending_event;
 
 

[PATCH 3/6] drm/xen-front: Add YUYV to supported formats

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Add YUYV to supported formats, so the frontend can work with the
formats used by cameras and other HW.

Signed-off-by: Oleksandr Andrushchenko 
---
 drivers/gpu/drm/xen/xen_drm_front_conn.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/gpu/drm/xen/xen_drm_front_conn.c 
b/drivers/gpu/drm/xen/xen_drm_front_conn.c
index 459702fa990e..44f1f70c0aed 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_conn.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_conn.c
@@ -33,6 +33,7 @@ static const u32 plane_formats[] = {
DRM_FORMAT_ARGB,
DRM_FORMAT_XRGB1555,
DRM_FORMAT_ARGB1555,
+   DRM_FORMAT_YUYV,
 };
 
 const u32 *xen_drm_front_conn_get_formats(int *format_count)
-- 
2.17.1

[PATCH 4/6] xen: Sync up with the canonical protocol definition in Xen

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

This is the sync up with the canonical definition of the
display protocol in Xen.

1. Add protocol version as an integer

Version string, which is in fact an integer, is hard to handle in the
code that supports different protocol versions. To simplify that
also add the version as an integer.

2. Pass buffer offset with XENDISPL_OP_DBUF_CREATE

There are cases when display data buffer is created with non-zero
offset to the data start. Handle such cases and provide that offset
while creating a display buffer.

3. Add XENDISPL_OP_GET_EDID command

Add an optional request for reading Extended Display Identification
Data (EDID) structure which allows better configuration of the
display connectors over the configuration set in XenStore.
With this change connectors may have multiple resolutions defined
with respect to detailed timing definitions and additional properties
normally provided by displays.

If this request is not supported by the backend then visible area
is defined by the relevant XenStore's "resolution" property.

If backend provides extended display identification data (EDID) with
XENDISPL_OP_GET_EDID request then EDID values must take precedence
over the resolutions defined in XenStore.

4. Bump protocol version to 2.

Signed-off-by: Oleksandr Andrushchenko 
---
 include/xen/interface/io/displif.h | 91 +-
 1 file changed, 88 insertions(+), 3 deletions(-)

diff --git a/include/xen/interface/io/displif.h 
b/include/xen/interface/io/displif.h
index fdc279dc4a88..c2d900186883 100644
--- a/include/xen/interface/io/displif.h
+++ b/include/xen/interface/io/displif.h
@@ -38,7 +38,8 @@
  *   Protocol version
  **
  */
-#define XENDISPL_PROTOCOL_VERSION  "1"
+#define XENDISPL_PROTOCOL_VERSION  "2"
+#define XENDISPL_PROTOCOL_VERSION_INT   2
 
 /*
  **
@@ -202,6 +203,9 @@
  *  Width and height of the connector in pixels separated by
  *  XENDISPL_RESOLUTION_SEPARATOR. This defines visible area of the
  *  display.
+ *  If backend provides extended display identification data (EDID) with
+ *  XENDISPL_OP_GET_EDID request then EDID values must take precedence
+ *  over the resolutions defined here.
  *
  *-- Connector Request Transport Parameters ---
  *
@@ -349,6 +353,8 @@
 #define XENDISPL_OP_FB_DETACH  0x13
 #define XENDISPL_OP_SET_CONFIG 0x14
 #define XENDISPL_OP_PG_FLIP0x15
+/* The below command is available in protocol version 2 and above. */
+#define XENDISPL_OP_GET_EDID   0x16
 
 /*
  **
@@ -377,6 +383,10 @@
 #define XENDISPL_FIELD_BE_ALLOC"be-alloc"
 #define XENDISPL_FIELD_UNIQUE_ID   "unique-id"
 
+#define XENDISPL_EDID_BLOCK_SIZE   128
+#define XENDISPL_EDID_BLOCK_COUNT  256
+#define XENDISPL_EDID_MAX_SIZE (XENDISPL_EDID_BLOCK_SIZE * 
XENDISPL_EDID_BLOCK_COUNT)
+
 /*
  **
  *  STATUS RETURN CODES
@@ -451,7 +461,9 @@
  * +++++
  * |   gref_directory  | 40
  * +++++
- * | reserved  | 44
+ * | data_ofs  | 44
+ * +++++
+ * | reserved  | 48
  * +++++
  * |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/|
  * +++++
@@ -494,6 +506,7 @@
  *   buffer size (buffer_sz) exceeds what can be addressed by this single page,
  *   then reference to the next page must be supplied (see gref_dir_next_page
  *   below)
+ * data_ofs - uint32_t, offset of the data in the buffer, octets
  */
 
 #define XENDISPL_DBUF_FLG_REQ_ALLOC(1 << 0)
@@ -506,6 +519,7 @@ struct xendispl_dbuf_create_req {
uint32_t buffer_sz;
uint32_t flags;
grant_ref_t gref_directory;
+   uint32_t data_ofs;
 };
 
 /*
@@ -731,6 +745,44 @@ struct xendispl_page_flip_req {
uint64_t fb_cookie;
 };
 
+/*
+ * Request EDID - request EDID describing current connector:
+ * 01 2   3octet
+ * +++++
+ * |  

[PATCH 0/6] Fixes and improvements for Xen pvdrm

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Hello,

This series contains an assorted set of fixes and improvements for
the Xen para-virtualized display driver and grant device driver which
I have collected over the last couple of months:

1. Minor fixes to grant device driver and drm/xen-front.

2. New format (YUYV) added to the list of the PV DRM supported formats
which allows the driver to be used in zero-copying use-cases when
a camera device is the source of the dma-bufs.

3. Synchronization with the latest para-virtualized protocol definition
in Xen [1].

4. SGT offset is now propagated to the backend: while importing a dmabuf
it is possible that the data of the buffer is put with offset which is
indicated by the SGT offset. This is needed for some GPUs which have
non-zero offset.

5. Version 2 of the Xen displif protocol adds XENDISPL_OP_GET_EDID
request which allows frontends to request EDID structure per
connector. This request is optional and if not supported by the
backend then visible area is still defined by the relevant
XenStore's "resolution" property.
If backend provides EDID with XENDISPL_OP_GET_EDID request then
its values must take precedence over the resolutions defined in
XenStore.

Thank you,
Oleksandr Andrushchenko

[1] 
https://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=c27a184225eab54d20435c8cab5ad0ef384dc2c0

Oleksandr Andrushchenko (6):
  xen/gntdev: Fix dmabuf import with non-zero sgt offset
  drm/xen-front: Fix misused IS_ERR_OR_NULL checks
  drm/xen-front: Add YUYV to supported formats
  xen: Sync up with the canonical protocol definition in Xen
  drm/xen-front: Pass dumb buffer data offset to the backend
  drm/xen-front: Add support for EDID based configuration

 drivers/gpu/drm/xen/xen_drm_front.c | 72 +++-
 drivers/gpu/drm/xen/xen_drm_front.h | 11 ++-
 drivers/gpu/drm/xen/xen_drm_front_cfg.c | 82 +++
 drivers/gpu/drm/xen/xen_drm_front_cfg.h |  7 ++
 drivers/gpu/drm/xen/xen_drm_front_conn.c| 27 +-
 drivers/gpu/drm/xen/xen_drm_front_evtchnl.c |  3 +
 drivers/gpu/drm/xen/xen_drm_front_evtchnl.h |  3 +
 drivers/gpu/drm/xen/xen_drm_front_gem.c | 11 +--
 drivers/gpu/drm/xen/xen_drm_front_kms.c |  7 +-
 drivers/xen/gntdev-dmabuf.c |  8 ++
 include/xen/interface/io/displif.h  | 91 -
 11 files changed, 305 insertions(+), 17 deletions(-)

-- 
2.17.1

[PATCH 2/6] drm/xen-front: Fix misused IS_ERR_OR_NULL checks

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

The patch c575b7eeb89f: "drm/xen-front: Add support for Xen PV
display frontend" from Apr 3, 2018, leads to the following static
checker warning:

drivers/gpu/drm/xen/xen_drm_front_gem.c:140 xen_drm_front_gem_create()
warn: passing zero to 'ERR_CAST'

drivers/gpu/drm/xen/xen_drm_front_gem.c
   133  struct drm_gem_object *xen_drm_front_gem_create(struct drm_device *dev,
   134  size_t size)
   135  {
   136  struct xen_gem_object *xen_obj;
   137
   138  xen_obj = gem_create(dev, size);
   139  if (IS_ERR_OR_NULL(xen_obj))
   140  return ERR_CAST(xen_obj);

Fix this and the rest of misused places with IS_ERR_OR_NULL in the
driver.

Fixes:  c575b7eeb89f: "drm/xen-front: Add support for Xen PV display frontend"

Signed-off-by: Oleksandr Andrushchenko 
Reported-by: Dan Carpenter 
---
 drivers/gpu/drm/xen/xen_drm_front.c | 4 ++--
 drivers/gpu/drm/xen/xen_drm_front_gem.c | 8 
 drivers/gpu/drm/xen/xen_drm_front_kms.c | 2 +-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front.c 
b/drivers/gpu/drm/xen/xen_drm_front.c
index 3e660fb111b3..88db2726e8ce 100644
--- a/drivers/gpu/drm/xen/xen_drm_front.c
+++ b/drivers/gpu/drm/xen/xen_drm_front.c
@@ -400,8 +400,8 @@ static int xen_drm_drv_dumb_create(struct drm_file *filp,
args->size = args->pitch * args->height;
 
obj = xen_drm_front_gem_create(dev, args->size);
-   if (IS_ERR_OR_NULL(obj)) {
-   ret = PTR_ERR_OR_ZERO(obj);
+   if (IS_ERR(obj)) {
+   ret = PTR_ERR(obj);
goto fail;
}
 
diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index f0b85e094111..4ec8a49241e1 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -83,7 +83,7 @@ static struct xen_gem_object *gem_create(struct drm_device 
*dev, size_t size)
 
size = round_up(size, PAGE_SIZE);
xen_obj = gem_create_obj(dev, size);
-   if (IS_ERR_OR_NULL(xen_obj))
+   if (IS_ERR(xen_obj))
return xen_obj;
 
if (drm_info->front_info->cfg.be_alloc) {
@@ -117,7 +117,7 @@ static struct xen_gem_object *gem_create(struct drm_device 
*dev, size_t size)
 */
xen_obj->num_pages = DIV_ROUND_UP(size, PAGE_SIZE);
xen_obj->pages = drm_gem_get_pages(_obj->base);
-   if (IS_ERR_OR_NULL(xen_obj->pages)) {
+   if (IS_ERR(xen_obj->pages)) {
ret = PTR_ERR(xen_obj->pages);
xen_obj->pages = NULL;
goto fail;
@@ -136,7 +136,7 @@ struct drm_gem_object *xen_drm_front_gem_create(struct 
drm_device *dev,
struct xen_gem_object *xen_obj;
 
xen_obj = gem_create(dev, size);
-   if (IS_ERR_OR_NULL(xen_obj))
+   if (IS_ERR(xen_obj))
return ERR_CAST(xen_obj);
 
return _obj->base;
@@ -194,7 +194,7 @@ xen_drm_front_gem_import_sg_table(struct drm_device *dev,
 
size = attach->dmabuf->size;
xen_obj = gem_create_obj(dev, size);
-   if (IS_ERR_OR_NULL(xen_obj))
+   if (IS_ERR(xen_obj))
return ERR_CAST(xen_obj);
 
ret = gem_alloc_pages_array(xen_obj, size);
diff --git a/drivers/gpu/drm/xen/xen_drm_front_kms.c 
b/drivers/gpu/drm/xen/xen_drm_front_kms.c
index 78096bbcd226..ef11b1e4de39 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_kms.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_kms.c
@@ -60,7 +60,7 @@ fb_create(struct drm_device *dev, struct drm_file *filp,
int ret;
 
fb = drm_gem_fb_create_with_funcs(dev, filp, mode_cmd, _funcs);
-   if (IS_ERR_OR_NULL(fb))
+   if (IS_ERR(fb))
return fb;
 
gem_obj = fb->obj[0];
-- 
2.17.1

Re: [PATCH 2/2] libgnttab: Add support for Linux dma-buf offset

[Oleksandr Andrushchenko]

Hello, Ian, Wei!

Initially I have sent this patch as a part of the series for the display 
protocol changes,

but later decided to split. At the moment the protocol changes are already

accepted, but this part is still missing feedback and is still wanted.

I would really appreciate if you could have a look at the change below.

Thank you in advance,

Oleksandr Andrushchenko

On 5/20/20 12:04 PM, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko 
>
> Add version 2 of the dma-buf ioctls which adds data_ofs parameter.
>
> dma-buf is backed by a scatter-gather table and has offset parameter
> which tells where the actual data starts. Relevant ioctls are extended
> to support that offset:
>- when dma-buf is created (exported) from grant references then
>  data_ofs is used to set the offset field in the scatter list
>  of the new dma-buf
>- when dma-buf is imported and grant references provided then
>  data_ofs is used to report that offset to user-space
>
> Signed-off-by: Oleksandr Andrushchenko 
> ---
>   tools/include/xen-sys/Linux/gntdev.h  | 53 +
>   tools/libs/gnttab/Makefile|  2 +-
>   tools/libs/gnttab/freebsd.c   | 15 +
>   tools/libs/gnttab/gnttab_core.c   | 17 ++
>   tools/libs/gnttab/include/xengnttab.h | 13 
>   tools/libs/gnttab/libxengnttab.map|  6 ++
>   tools/libs/gnttab/linux.c | 86 +++
>   tools/libs/gnttab/minios.c| 15 +
>   tools/libs/gnttab/private.h   |  9 +++
>   9 files changed, 215 insertions(+), 1 deletion(-)
>
> diff --git a/tools/include/xen-sys/Linux/gntdev.h 
> b/tools/include/xen-sys/Linux/gntdev.h
> index d16076044c71..0c43393cbee5 100644
> --- a/tools/include/xen-sys/Linux/gntdev.h
> +++ b/tools/include/xen-sys/Linux/gntdev.h
> @@ -274,4 +274,57 @@ struct ioctl_gntdev_dmabuf_imp_release {
>   uint32_t reserved;
>   };
>   
> +/*
> + * Version 2 of the ioctls adds @data_ofs parameter.
> + *
> + * dma-buf is backed by a scatter-gather table and has offset
> + * parameter which tells where the actual data starts.
> + * Relevant ioctls are extended to support that offset:
> + *   - when dma-buf is created (exported) from grant references then
> + * @data_ofs is used to set the offset field in the scatter list
> + * of the new dma-buf
> + *   - when dma-buf is imported and grant references are provided then
> + * @data_ofs is used to report that offset to user-space
> + */
> +#define IOCTL_GNTDEV_DMABUF_EXP_FROM_REFS_V2 \
> +_IOC(_IOC_NONE, 'G', 13, \
> + sizeof(struct ioctl_gntdev_dmabuf_exp_from_refs_v2))
> +struct ioctl_gntdev_dmabuf_exp_from_refs_v2 {
> +/* IN parameters. */
> +/* Specific options for this dma-buf: see GNTDEV_DMA_FLAG_XXX. */
> +uint32_t flags;
> +/* Number of grant references in @refs array. */
> +uint32_t count;
> +/* Offset of the data in the dma-buf. */
> +uint32_t data_ofs;
> +/* OUT parameters. */
> +/* File descriptor of the dma-buf. */
> +uint32_t fd;
> +/* The domain ID of the grant references to be mapped. */
> +uint32_t domid;
> +/* Variable IN parameter. */
> +/* Array of grant references of size @count. */
> +uint32_t refs[1];
> +};
> +
> +#define IOCTL_GNTDEV_DMABUF_IMP_TO_REFS_V2 \
> +_IOC(_IOC_NONE, 'G', 14, \
> + sizeof(struct ioctl_gntdev_dmabuf_imp_to_refs_v2))
> +struct ioctl_gntdev_dmabuf_imp_to_refs_v2 {
> +/* IN parameters. */
> +/* File descriptor of the dma-buf. */
> +uint32_t fd;
> +/* Number of grant references in @refs array. */
> +uint32_t count;
> +/* The domain ID for which references to be granted. */
> +uint32_t domid;
> +/* Reserved - must be zero. */
> +uint32_t reserved;
> +/* OUT parameters. */
> +/* Offset of the data in the dma-buf. */
> +uint32_t data_ofs;
> +/* Array of grant references of size @count. */
> +uint32_t refs[1];
> +};
> +
>   #endif /* __LINUX_PUBLIC_GNTDEV_H__ */
> diff --git a/tools/libs/gnttab/Makefile b/tools/libs/gnttab/Makefile
> index 2da8fbbb7f6f..5ee2d965214f 100644
> --- a/tools/libs/gnttab/Makefile
> +++ b/tools/libs/gnttab/Makefile
> @@ -2,7 +2,7 @@ XEN_ROOT = $(CURDIR)/../../..
>   include $(XEN_ROOT)/tools/Rules.mk
>   
>   MAJOR= 1
> -MINOR= 2
> +MINOR= 3
>   LIBNAME  := gnttab
>   USELIBS  := toollog toolcore
>   
> diff --git a/tools/libs/gnttab/freebsd.c b/tools/libs/gnttab/freebsd.c
> index 886b588303a0..baf0f60aa4d3 100644
> --- a/tools/libs/gnttab/freebsd.c
> +++ b/tools/libs/gnttab/freebsd.c
> @@ -319,6 +319,14 @@ int osdep_gnt

Re: [PATCH v2] xen/displif: Protocol version 2

[Oleksandr Andrushchenko]

Ping

On 7/1/20 10:19 AM, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko 
>
> 1. Add protocol version as an integer
>
> Version string, which is in fact an integer, is hard to handle in the
> code that supports different protocol versions. To simplify that
> also add the version as an integer.
>
> 2. Pass buffer offset with XENDISPL_OP_DBUF_CREATE
>
> There are cases when display data buffer is created with non-zero
> offset to the data start. Handle such cases and provide that offset
> while creating a display buffer.
>
> 3. Add XENDISPL_OP_GET_EDID command
>
> Add an optional request for reading Extended Display Identification
> Data (EDID) structure which allows better configuration of the
> display connectors over the configuration set in XenStore.
> With this change connectors may have multiple resolutions defined
> with respect to detailed timing definitions and additional properties
> normally provided by displays.
>
> If this request is not supported by the backend then visible area
> is defined by the relevant XenStore's "resolution" property.
>
> If backend provides extended display identification data (EDID) with
> XENDISPL_OP_GET_EDID request then EDID values must take precedence
> over the resolutions defined in XenStore.
>
> 4. Bump protocol version to 2.
>
> Signed-off-by: Oleksandr Andrushchenko 
> ---
>   xen/include/public/io/displif.h | 91 +++--
>   1 file changed, 88 insertions(+), 3 deletions(-)
>
> diff --git a/xen/include/public/io/displif.h b/xen/include/public/io/displif.h
> index cc5de9cb1f35..0055895510f7 100644
> --- a/xen/include/public/io/displif.h
> +++ b/xen/include/public/io/displif.h
> @@ -38,7 +38,8 @@
>*   Protocol version
>
> **
>*/
> -#define XENDISPL_PROTOCOL_VERSION "1"
> +#define XENDISPL_PROTOCOL_VERSION "2"
> +#define XENDISPL_PROTOCOL_VERSION_INT  2
>   
>   /*
>
> **
> @@ -202,6 +203,9 @@
>*  Width and height of the connector in pixels separated by
>*  XENDISPL_RESOLUTION_SEPARATOR. This defines visible area of the
>*  display.
> + *  If backend provides extended display identification data (EDID) with
> + *  XENDISPL_OP_GET_EDID request then EDID values must take precedence
> + *  over the resolutions defined here.
>*
>*-- Connector Request Transport Parameters 
> ---
>*
> @@ -349,6 +353,8 @@
>   #define XENDISPL_OP_FB_DETACH 0x13
>   #define XENDISPL_OP_SET_CONFIG0x14
>   #define XENDISPL_OP_PG_FLIP   0x15
> +/* The below command is available in protocol version 2 and above. */
> +#define XENDISPL_OP_GET_EDID  0x16
>   
>   /*
>
> **
> @@ -377,6 +383,10 @@
>   #define XENDISPL_FIELD_BE_ALLOC   "be-alloc"
>   #define XENDISPL_FIELD_UNIQUE_ID  "unique-id"
>   
> +#define XENDISPL_EDID_BLOCK_SIZE  128
> +#define XENDISPL_EDID_BLOCK_COUNT 256
> +#define XENDISPL_EDID_MAX_SIZE(XENDISPL_EDID_BLOCK_SIZE * 
> XENDISPL_EDID_BLOCK_COUNT)
> +
>   /*
>
> **
>*  STATUS RETURN CODES
> @@ -451,7 +461,9 @@
>* +++++
>* |   gref_directory  | 40
>* +++++
> - * | reserved  | 44
> + * | data_ofs  | 44
> + * +++++
> + * | reserved  | 48
>* +++++
>* |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/|
>* +++++
> @@ -494,6 +506,7 @@
>*   buffer size (buffer_sz) exceeds what can be addressed by this single 
> page,
>*   then reference to the next page must be supplied (see 
> gref_dir_next_page
>*   below)
> + * data_ofs - uint32_t, offset of the data in the buffer, octets
>*/
>   
>   #define XENDISPL_DBUF_FLG_REQ_ALLOC   (1 << 0)
> @@ -506,6 +519,7 @@

Re: [RFC PATCH v1 4/4] arm/libxl: Emulated PCI device tree node in libxl

[Oleksandr Andrushchenko]

On 7/24/20 2:39 AM, Stefano Stabellini wrote:
> On Thu, 23 Jul 2020, Rahul Singh wrote:
>> libxl will create an emulated PCI device tree node in the
>> device tree to enable the guest OS to discover the virtual
>> PCI during guest boot.
>>
>> We introduced the new config option [vpci="ecam"] for guests.
>> When this config option is enabled in a guest configuration,
>> a PCI device tree node will be created in the guest device tree.
>>
>> A new area has been reserved in the arm guest physical map at
>> which the VPCI bus is declared in the device tree (reg and ranges
>> parameters of the node).
>>
>> Change-Id: I47d39cbe8184de2226f174644df9790ecc610ccd
> Same question
>
>
>> Signed-off-by: Rahul Singh 
>> ---
>>   tools/libxl/libxl_arm.c   | 200 ++
>>   tools/libxl/libxl_types.idl   |   6 +
>>   tools/xl/xl_parse.c   |   7 ++
>>   xen/include/public/arch-arm.h |  28 +
>>   4 files changed, 241 insertions(+)
>>
>> diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
>> index 34f8a29056..84568e9dc9 100644
>> --- a/tools/libxl/libxl_arm.c
>> +++ b/tools/libxl/libxl_arm.c
>> @@ -268,6 +268,130 @@ static int fdt_property_regs(libxl__gc *gc, void *fdt,
>>   return fdt_property(fdt, "reg", regs, sizeof(regs));
>>   }
>>   
>> +static int fdt_property_vpci_bus_range(libxl__gc *gc, void *fdt,
>> +unsigned num_cells, ...)
>> +{
>> +uint32_t bus_range[num_cells];
>> +be32 *cells = _range[0];
>> +int i;
>> +va_list ap;
>> +uint32_t arg;
>> +
>> +va_start(ap, num_cells);
>> +for (i = 0 ; i < num_cells; i++) {
>> +arg = va_arg(ap, uint32_t);
>> +set_cell(, 1, arg);
>> +}
>> +va_end(ap);
>> +
>> +return fdt_property(fdt, "bus-range", bus_range, sizeof(bus_range));
>> +}
>> +
>> +static int fdt_property_vpci_interrupt_map_mask(libxl__gc *gc, void *fdt,
>> +unsigned num_cells, ...)
>> +{
>> +uint32_t interrupt_map_mask[num_cells];
>> +be32 *cells = _map_mask[0];
>> +int i;
>> +va_list ap;
>> +uint32_t arg;
>> +
>> +va_start(ap, num_cells);
>> +for (i = 0 ; i < num_cells; i++) {
>> +arg = va_arg(ap, uint32_t);
>> +set_cell(, 1, arg);
>> +}
>> +va_end(ap);
>> +
>> +return fdt_property(fdt, "interrupt-map-mask", interrupt_map_mask,
>> +sizeof(interrupt_map_mask));
>> +}
>> +
>> +static int fdt_property_vpci_ranges(libxl__gc *gc, void *fdt,
>> +unsigned vpci_addr_cells,
>> +unsigned cpu_addr_cells,
>> +unsigned vpci_size_cells,
>> +unsigned num_regs, ...)
>> +{
>> +uint32_t 
>> regs[num_regs*(vpci_addr_cells+cpu_addr_cells+vpci_size_cells)];
>> +be32 *cells = [0];
>> +int i;
>> +va_list ap;
>> +uint64_t arg;
>> +
>> +va_start(ap, num_regs);
>> +for (i = 0 ; i < num_regs; i++) {
>> +/* Set the memory bit field */
>> +arg = va_arg(ap, uint64_t);
>> +set_cell(, 1, arg);
>> +
>> +/* Set the vpci bus address */
>> +arg = vpci_addr_cells ? va_arg(ap, uint64_t) : 0;
>> +set_cell(, 2 , arg);
>> +
>> +/* Set the cpu bus address where vpci address is mapped */
>> +arg = cpu_addr_cells ? va_arg(ap, uint64_t) : 0;
>> +set_cell(, cpu_addr_cells, arg);
>> +
>> +/* Set the vpci size requested */
>> +arg = vpci_size_cells ? va_arg(ap, uint64_t) : 0;
>> +set_cell(, vpci_size_cells,arg);
>> +}
>> +va_end(ap);
>> +
>> +return fdt_property(fdt, "ranges", regs, sizeof(regs));
>> +}
>> +
>> +static int fdt_property_vpci_interrupt_map(libxl__gc *gc, void *fdt,
>> +unsigned child_unit_addr_cells,
>> +unsigned child_interrupt_specifier_cells,
>> +unsigned parent_unit_addr_cells,
>> +unsigned parent_interrupt_specifier_cells,
>> +unsigned num_regs, ...)
>> +{
>> +uint32_t interrupt_map[num_regs * (child_unit_addr_cells +
>> +child_interrupt_specifier_cells + parent_unit_addr_cells
>> ++ parent_interrupt_specifier_cells + 1)];
>> +be32 *cells = _map[0];
>> +int i,j;
>> +va_list ap;
>> +uint64_t arg;
>> +
>> +va_start(ap, num_regs);
>> +for (i = 0 ; i < num_regs; i++) {
>> +/* Set the child unit address*/
>> +for (j = 0 ; j < child_unit_addr_cells; j++) {
>> +arg = va_arg(ap, uint32_t);
>> +set_cell(, 1 , arg);
>> +}
>> +
>> +/* Set the child interrupt specifier*/
>> +for (j = 0 ; j < child_interrupt_specifier_cells ; j++) {
>> +arg = va_arg(ap, uint32_t);
>> +set_cell(, 1 , arg);
>> +}
>> +
>> +/* Set the interrupt-parent*/
>> +set_cell(, 1 , GUEST_PHANDLE_GIC);
>> +
>> +/* Set the parent unit address*/
>> +for (j = 0 ; j < parent_unit_addr_cells; j++) {
>> +arg = va_arg(ap, uint32_t);
>> +set_cell(, 1 , arg);
>> +}
>> +

Re: [RFC PATCH v1 2/4] xen/arm: Discovering PCI devices and add the PCI devices in XEN.

[Oleksandr Andrushchenko]

On 7/23/20 11:44 PM, Stefano Stabellini wrote:
> On Thu, 23 Jul 2020, Rahul Singh wrote:
>> Hardware domain is in charge of doing the PCI enumeration and will
>> discover the PCI devices and then will communicate to XEN via hyper
>> call PHYSDEVOP_pci_device_add to add the PCI devices in XEN.
>>
>> Change-Id: Ie87e19741689503b4b62da911c8dc2ee318584ac
> Same question about Change-Id
>
>
>> Signed-off-by: Rahul Singh 
>> ---
>>   xen/arch/arm/physdev.c | 42 +++---
>>   1 file changed, 39 insertions(+), 3 deletions(-)
>>
>> diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c
>> index e91355fe22..274720f98a 100644
>> --- a/xen/arch/arm/physdev.c
>> +++ b/xen/arch/arm/physdev.c
>> @@ -9,12 +9,48 @@
>>   #include 
>>   #include 
>>   #include 
>> -
>> +#include 
>> +#include 
>>   
>>   int do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>>   {
>> -gdprintk(XENLOG_DEBUG, "PHYSDEVOP cmd=%d: not implemented\n", cmd);
>> -return -ENOSYS;
>> +int ret = 0;
>> +
>> +switch ( cmd )
>> +{
>> +#ifdef CONFIG_HAS_PCI

In the cover letter you were saying "we are not enabling the HAS_PCI and 
HAS_VPCI flags for ARM".

Is this still valid?

>> +case PHYSDEVOP_pci_device_add:
>> +{
>> +struct physdev_pci_device_add add;
>> +struct pci_dev_info pdev_info;
>> +nodeid_t node = NUMA_NO_NODE;
>> +
>> +ret = -EFAULT;
>> +if ( copy_from_guest(, arg, 1) != 0 )
>> +break;
>> +
>> +pdev_info.is_extfn = !!(add.flags & XEN_PCI_DEV_EXTFN);
>> +if ( add.flags & XEN_PCI_DEV_VIRTFN )
>> +{
>> +pdev_info.is_virtfn = 1;
>> +pdev_info.physfn.bus = add.physfn.bus;
>> +pdev_info.physfn.devfn = add.physfn.devfn;
>> +}
>> +else
>> +pdev_info.is_virtfn = 0;
>> +
>> +ret = pci_add_device(add.seg, add.bus, add.devfn,
>> +_info, node);
>> +
>> +break;
>> +}
>> +#endif
>> +default:
>> +gdprintk(XENLOG_DEBUG, "PHYSDEVOP cmd=%d: not implemented\n", 
>> cmd);
>> +ret = -ENOSYS;
>> +}
> I think we should make the implementation common between arm and x86 by
> creating xen/common/physdev.c:do_physdev_op as a shared entry point for
> PHYSDEVOP hypercalls implementations. See for instance:
>
> xen/common/sysctl.c:do_sysctl
>
> and
>
> xen/arch/arm/sysctl.c:arch_do_sysctl
> xen/arch/x86/sysctl.c:arch_do_sysctl
>
>
> Jan, Andrew, Roger, any opinions?
>
>
I think we can also have a look at [1] by Julien. That implementation,

IMO, had some thoughts on making Arm/x86 code common where possible


[1] 
https://xenbits.xen.org/gitweb/?p=people/julieng/xen-unstable.git;a=shortlog;h=refs/heads/dev-pci

Re: [RFC PATCH v1 1/4] arm/pci: PCI setup and PCI host bridge discovery within XEN on ARM.

[Oleksandr Andrushchenko]

On 7/24/20 2:38 AM, Stefano Stabellini wrote:
> + Jan, Andrew, Roger
>
> Please have a look at my comment on whether we should share the MMCFG
> code below, feel free to ignore the rest :-)
>
>
> On Thu, 23 Jul 2020, Rahul Singh wrote:
>> XEN during boot will read the PCI device tree node “reg” property
>> and will map the PCI config space to the XEN memory.
>>
>> XEN will read the “linux, pci-domain” property from the device tree
>> node and configure the host bridge segment number accordingly.
>>
>> As of now "pci-host-ecam-generic" compatible board is supported.
>>
>> Change-Id: If32f7748b7dc89dd37114dc502943222a2a36c49
> What is this Change-Id property?
Gerrit ;)
>
>
>> Signed-off-by: Rahul Singh 
>> ---
>>   xen/arch/arm/Kconfig|   7 +
>>   xen/arch/arm/Makefile   |   1 +
>>   xen/arch/arm/pci/Makefile   |   4 +
>>   xen/arch/arm/pci/pci-access.c   | 101 ++
>>   xen/arch/arm/pci/pci-host-common.c  | 198 
>>   xen/arch/arm/pci/pci-host-generic.c | 131 ++
>>   xen/arch/arm/pci/pci.c  | 112 
>>   xen/arch/arm/setup.c|   2 +
>>   xen/include/asm-arm/device.h|   7 +-
>>   xen/include/asm-arm/pci.h   |  97 +-
>>   10 files changed, 654 insertions(+), 6 deletions(-)
>>   create mode 100644 xen/arch/arm/pci/Makefile
>>   create mode 100644 xen/arch/arm/pci/pci-access.c
>>   create mode 100644 xen/arch/arm/pci/pci-host-common.c
>>   create mode 100644 xen/arch/arm/pci/pci-host-generic.c
>>   create mode 100644 xen/arch/arm/pci/pci.c
>>
>> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
>> index 2777388265..ee13339ae9 100644
>> --- a/xen/arch/arm/Kconfig
>> +++ b/xen/arch/arm/Kconfig
>> @@ -31,6 +31,13 @@ menu "Architecture Features"
>>   
>>   source "arch/Kconfig"
>>   
>> +config ARM_PCI
>> +bool "PCI Passthrough Support"
>> +depends on ARM_64
>> +---help---
>> +
>> +  PCI passthrough support for Xen on ARM64.
>> +
>>   config ACPI
>>  bool "ACPI (Advanced Configuration and Power Interface) Support" if 
>> EXPERT
>>  depends on ARM_64
>> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
>> index 7e82b2178c..345cb83eed 100644
>> --- a/xen/arch/arm/Makefile
>> +++ b/xen/arch/arm/Makefile
>> @@ -6,6 +6,7 @@ ifneq ($(CONFIG_NO_PLAT),y)
>>   obj-y += platforms/
>>   endif
>>   obj-$(CONFIG_TEE) += tee/
>> +obj-$(CONFIG_ARM_PCI) += pci/
>>   
>>   obj-$(CONFIG_HAS_ALTERNATIVE) += alternative.o
>>   obj-y += bootfdt.init.o
>> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
>> new file mode 100644
>> index 00..358508b787
>> --- /dev/null
>> +++ b/xen/arch/arm/pci/Makefile
>> @@ -0,0 +1,4 @@
>> +obj-y += pci.o
>> +obj-y += pci-host-generic.o
>> +obj-y += pci-host-common.o
>> +obj-y += pci-access.o
>> diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
>> new file mode 100644
>> index 00..c53ef58336
>> --- /dev/null
>> +++ b/xen/arch/arm/pci/pci-access.c
>> @@ -0,0 +1,101 @@
>> +/*
>> + * Copyright (C) 2020 Arm Ltd.
I think SPDX will fit better in any new code.
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * along with this program.  If not, see .
>> + */
>> +
>> +#include 
>> +#include 
>> +#include 
>> +#include 
>> +
>> +static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg,
>> +unsigned int len)
>> +{
>> +int rc;
>> +uint32_t val = GENMASK(0, len * 8);
>> +
>> +struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, 
>> sbdf.bus);
>> +
>> +if ( unlikely(!bridge) )
>> +{
>> +printk(XENLOG_ERR "Unable to find bridge for "PRI_pci"\n",
>> +sbdf.seg, sbdf.bus, sbdf.dev, sbdf.fn);
>> +return val;
>> +}
>> +
>> +if ( unlikely(!bridge->ops->read) )
>> +return val;
>> +
>> +rc = bridge->ops->read(bridge, (uint32_t) sbdf.sbdf, reg, len, );
>> +if ( rc )
>> +printk(XENLOG_ERR "Failed to read reg %#x len %u for "PRI_pci"\n",
>> +reg, len, sbdf.seg, sbdf.bus, sbdf.dev, sbdf.fn);
>> +
>> +return val;
>> +}
>> +
>> +static void pci_config_write(pci_sbdf_t sbdf, unsigned int reg,
>> +unsigned int len, uint32_t val)
>> +{
>> +int rc;
>> +struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, 
>> sbdf.bus);
>> +
>> +if ( unlikely(!bridge) )
>> +{
>> +  

Re: RFC: PCI devices passthrough on Arm design proposal

[Oleksandr Andrushchenko]

On 7/17/20 2:26 PM, Julien Grall wrote:
>
>
> On 17/07/2020 08:41, Oleksandr Andrushchenko wrote:
>>>> We need to come up with something similar for dom0less too. It could be
>>>> exactly the same thing (a list of BDFs as strings as a device tree
>>>> property) or something else if we can come up with a better idea.
>>> Fully agree.
>>> Maybe a tree topology could allow more possibilities (like giving BAR 
>>> values) in the future.
>>>>
>>>>> # Emulated PCI device tree node in libxl:
>>>>>
>>>>> Libxl is creating a virtual PCI device tree node in the device tree to 
>>>>> enable the guest OS to discover the virtual PCI during guest boot. We 
>>>>> introduced the new config option [vpci="pci_ecam"] for guests. When this 
>>>>> config option is enabled in a guest configuration, a PCI device tree node 
>>>>> will be created in the guest device tree.
>>>>>
>>>>> A new area has been reserved in the arm guest physical map at which the 
>>>>> VPCI bus is declared in the device tree (reg and ranges parameters of the 
>>>>> node). A trap handler for the PCI ECAM access from guest has been 
>>>>> registered at the defined address and redirects requests to the VPCI 
>>>>> driver in Xen.
>>>>>
>>>>> Limitation:
>>>>> * Only one PCI device tree node is supported as of now.
>>>> I think vpci="pci_ecam" should be optional: if pci=[ "PCI_SPEC_STRING",
>>>> ...] is specififed, then vpci="pci_ecam" is implied.
>>>>
>>>> vpci="pci_ecam" is only useful one day in the future when we want to be
>>>> able to emulate other non-ecam host bridges. For now we could even skip
>>>> it.
>>> This would create a problem if xl is used to add a PCI device as we need 
>>> the PCI node to be in the DTB when the guest is created.
>>> I agree this is not needed but removing it might create more complexity in 
>>> the code.
>>
>> I would suggest we have it from day 0 as there are plenty of HW available 
>> which is not ECAM.
>>
>> Having vpci allows other bridges to be supported
>
> So I can understand why you would want to have a driver for non-ECAM host PCI 
> controller. However, why would you want to emulate a non-ECAM PCI controller 
> to a guest?
Indeed. No need to emulate non-ECAM
>
> Cheers,
>

Re: RFC: PCI devices passthrough on Arm design proposal

[Oleksandr Andrushchenko]

On 7/17/20 11:10 AM, Jan Beulich wrote:

On 16.07.2020 19:10, Rahul Singh wrote:

# Discovering PCI devices:

PCI-PCIe enumeration is a process of detecting devices connected to its host. 
It is the responsibility of the hardware domain or boot firmware to do the PCI 
enumeration and configure the BAR, PCI capabilities, and MSI/MSI-X 
configuration.

PCI-PCIe enumeration in XEN is not feasible for the configuration part as it 
would require a lot of code inside Xen which would require a lot of 
maintenance. Added to this many platforms require some quirks in that part of 
the PCI code which would greatly improve Xen complexity. Once hardware domain 
enumerates the device then it will communicate to XEN via the below hypercall.

#define PHYSDEVOP_pci_device_add25
struct physdev_pci_device_add {
 uint16_t seg;
 uint8_t bus;
 uint8_t devfn;
 uint32_t flags;
 struct {
uint8_t bus;
uint8_t devfn;
 } physfn;
 /*
 * Optional parameters array.
 * First element ([0]) is PXM domain associated with the device (if * 
XEN_PCI_DEV_PXM is set)
 */
 uint32_t optarr[XEN_FLEX_ARRAY_DIM];
 };

As the hypercall argument has the PCI segment number, XEN will access the PCI 
config space based on this segment number and find the host-bridge 
corresponding to this segment number. At this stage host bridge is fully 
initialized so there will be no issue to access the config space.

XEN will add the PCI devices in the linked list maintain in XEN using the 
function pci_add_device(). XEN will be aware of all the PCI devices on the 
system and all the device will be added to the hardware domain.

Have you had any thoughts about Dom0 re-arranging the bus numbering?
This is, afaict, a still open issue on x86 as well.


This can get even trickier as we may have PCI enumerated at boot time

by the firmware and then Dom0 may perform the enumeration differently.

So, Xen needs to be aware of what is going to be used as the source of the

enumeration data and be ready to re-build its internal structures in order

to be aligned with that entity: e.g. compare Dom0 and Dom0less use-cases




Limitations:
* When PCI devices are added to XEN, MSI capability is not initialized inside 
XEN and not supported as of now.

I think this is a pretty severe limitation, as modern devices tend to
not support pin based interrupts anymore.


# Emulated PCI device tree node in libxl:

Libxl is creating a virtual PCI device tree node in the device tree to enable the guest 
OS to discover the virtual PCI during guest boot. We introduced the new config option 
[vpci="pci_ecam"] for guests. When this config option is enabled in a guest 
configuration, a PCI device tree node will be created in the guest device tree.

I support Stefano's suggestion for this to be an optional thing, i.e.
there to be no need for it when there are PCI devices assigned to the
guest anyway. I also wonder about the pci_ prefix here - isn't
vpci="ecam" as unambiguous?


A new area has been reserved in the arm guest physical map at which the VPCI 
bus is declared in the device tree (reg and ranges parameters of the node). A 
trap handler for the PCI ECAM access from guest has been registered at the 
defined address and redirects requests to the VPCI driver in Xen.

Limitation:
* Only one PCI device tree node is supported as of now.

BAR value and IOMEM mapping:

Linux guest will do the PCI enumeration based on the area reserved for ECAM and 
IOMEM ranges in the VPCI device tree node. Once PCI device is assigned to 
the guest, XEN will map the guest PCI IOMEM region to the real physical IOMEM 
region only for the assigned devices.

As of now we have not modified the existing VPCI code to map the guest PCI 
IOMEM region to the real physical IOMEM region. We used the existing guest 
“iomem” config option to map the region.
For example:
Guest reserved IOMEM region:  0x0402
Real physical IOMEM region:0x5000
IOMEM size:128MB
iomem config will be:   iomem = ["0x5,0x8000@0x4020"]

This surely is planned to go away before the code hits upstream? The
ranges really should be read out of the BARs, as I see the
"limitations" section further down suggests, but it's not clear
whether "limitations" are items that you plan to take care of before
submitting your code for review.

Jan

Re: RFC: PCI devices passthrough on Arm design proposal

[Oleksandr Andrushchenko]

On 7/17/20 9:53 AM, Bertrand Marquis wrote:
>
>> On 16 Jul 2020, at 22:51, Stefano Stabellini  wrote:
>>
>> On Thu, 16 Jul 2020, Rahul Singh wrote:
>>> Hello All,
>>>
>>> Following up on discussion on PCI Passthrough support on ARM that we had at 
>>> the XEN summit, we are submitting a Review For Comment and a design 
>>> proposal for PCI passthrough support on ARM. Feel free to give your 
>>> feedback.
>>>
>>> The followings describe the high-level design proposal of the PCI 
>>> passthrough support and how the different modules within the system 
>>> interacts with each other to assign a particular PCI device to the guest.
>> I think the proposal is good and I only have a couple of thoughts to
>> share below.
>>
>>
>>> # Title:
>>>
>>> PCI devices passthrough on Arm design proposal
>>>
>>> # Problem statement:
>>>
>>> On ARM there in no support to assign a PCI device to a guest. PCI device 
>>> passthrough capability allows guests to have full access to some PCI 
>>> devices. PCI device passthrough allows PCI devices to appear and behave as 
>>> if they were physically attached to the guest operating system and provide 
>>> full isolation of the PCI devices.
>>>
>>> Goal of this work is to also support Dom0Less configuration so the PCI 
>>> backend/frontend drivers used on x86 shall not be used on Arm. It will use 
>>> the existing VPCI concept from X86 and implement the virtual PCI bus 
>>> through IO emulation​ such that only assigned devices are visible​ to the 
>>> guest and guest can use the standard PCI driver.
>>>
>>> Only Dom0 and Xen will have access to the real PCI bus,

So, in this case how is the access serialization going to work?

I mean that if both Xen and Dom0 are about to access the bus at the same time?

There was a discussion on the same before [1] and IMO it was not decided on

how to deal with that.

>>> ​ guest will have a direct access to the assigned device itself​. IOMEM 
>>> memory will be mapped to the guest ​and interrupt will be redirected to the 
>>> guest. SMMU has to be configured correctly to have DMA transaction.
>>>
>>> ## Current state: Draft version
>>>
>>> # Proposer(s): Rahul Singh, Bertrand Marquis
>>>
>>> # Proposal:
>>>
>>> This section will describe the different subsystem to support the PCI 
>>> device passthrough and how these subsystems interact with each other to 
>>> assign a device to the guest.
>>>
>>> # PCI Terminology:
>>>
>>> Host Bridge: Host bridge allows the PCI devices to talk to the rest of the 
>>> computer.
>>> ECAM: ECAM (Enhanced Configuration Access Mechanism) is a mechanism 
>>> developed to allow PCIe to access configuration space. The space available 
>>> per function is 4KB.
>>>
>>> # Discovering PCI Host Bridge in XEN:
>>>
>>> In order to support the PCI passthrough XEN should be aware of all the PCI 
>>> host bridges available on the system and should be able to access the PCI 
>>> configuration space. ECAM configuration access is supported as of now. XEN 
>>> during boot will read the PCI device tree node “reg” property and will map 
>>> the ECAM space to the XEN memory using the “ioremap_nocache ()” function.
>>>
>>> If there are more than one segment on the system, XEN will read the “linux, 
>>> pci-domain” property from the device tree node and configure  the host 
>>> bridge segment number accordingly. All the PCI device tree nodes should 
>>> have the “linux,pci-domain” property so that there will be no conflicts. 
>>> During hardware domain boot Linux will also use the same “linux,pci-domain” 
>>> property and assign the domain number to the host bridge.
>>>
>>> When Dom0 tries to access the PCI config space of the device, XEN will find 
>>> the corresponding host bridge based on segment number and access the 
>>> corresponding config space assigned to that bridge.
>>>
>>> Limitation:
>>> * Only PCI ECAM configuration space access is supported.

This is really the limitation which we have to think of now as there are lots of

HW w/o ECAM support and not providing a way to use PCI(e) on those boards

would render them useless wrt PCI. I don't suggest to have some real code for

that, but I would suggest we design some interfaces from day 0.

At the same time I do understand that supporting non-ECAM bridges is a pain

>>> * Device tree binding is supported as of now, ACPI is not supported.
>>> * Need to port the PCI host bridge access code to XEN to access the 
>>> configuration space (generic one works but lots of platforms will required 
>>> some specific code or quirks).
>>>
>>> # Discovering PCI devices:
>>>
>>> PCI-PCIe enumeration is a process of detecting devices connected to its 
>>> host. It is the responsibility of the hardware domain or boot firmware to 
>>> do the PCI enumeration and configure
Great, so we assume here that the bootloader can do the enumeration and 
configuration...
>>>   the BAR, PCI capabilities, and MSI/MSI-X configuration.
>>>
>>> PCI-PCIe enumeration in XEN is not feasible for the configuration 

Re: [PATCH v2] xen/displif: Protocol version 2

[Oleksandr Andrushchenko]

Hi, Paul!

On 7/2/20 11:42 AM, Jürgen Groß wrote:
> On 02.07.20 09:59, Oleksandr Andrushchenko wrote:
>>
>> On 7/2/20 10:20 AM, Jürgen Groß wrote:
>>> On 01.07.20 14:07, Oleksandr Andrushchenko wrote:
>>>> On 7/1/20 1:46 PM, Jürgen Groß wrote:
>>>>> On 01.07.20 09:19, Oleksandr Andrushchenko wrote:
>>>>>> From: Oleksandr Andrushchenko 
>>>>>>
>>>>>> 1. Add protocol version as an integer
>>>>>>
>>>>>> Version string, which is in fact an integer, is hard to handle in the
>>>>>> code that supports different protocol versions. To simplify that
>>>>>> also add the version as an integer.
>>>>>>
>>>>>> 2. Pass buffer offset with XENDISPL_OP_DBUF_CREATE
>>>>>>
>>>>>> There are cases when display data buffer is created with non-zero
>>>>>> offset to the data start. Handle such cases and provide that offset
>>>>>> while creating a display buffer.
>>>>>>
>>>>>> 3. Add XENDISPL_OP_GET_EDID command
>>>>>>
>>>>>> Add an optional request for reading Extended Display Identification
>>>>>> Data (EDID) structure which allows better configuration of the
>>>>>> display connectors over the configuration set in XenStore.
>>>>>> With this change connectors may have multiple resolutions defined
>>>>>> with respect to detailed timing definitions and additional properties
>>>>>> normally provided by displays.
>>>>>>
>>>>>> If this request is not supported by the backend then visible area
>>>>>> is defined by the relevant XenStore's "resolution" property.
>>>>>>
>>>>>> If backend provides extended display identification data (EDID) with
>>>>>> XENDISPL_OP_GET_EDID request then EDID values must take precedence
>>>>>> over the resolutions defined in XenStore.
>>>>>>
>>>>>> 4. Bump protocol version to 2.
>>>>>>
>>>>>> Signed-off-by: Oleksandr Andrushchenko 
>>>>>
>>>>> Reviewed-by: Juergen Gross 
>>>>
>>>> Thank you, do you want me to prepare the same for the kernel so
>>>>
>>>> you have it at hand when the time comes?
>>>
>>> It should be added to the kernel only when really needed (i.e. a user of
>>> the new functionality is showing up).
>>
>> We have a patch for that which adds EDID to the existing PV DRM frontend,
>>
>> so while upstreaming those changes I will also include changes to the 
>> protocol
>>
>> in the kernel series: for that we need the header in Xen tree first, right?
>
> Yes.
>
Is it possible that we have this change in the release please?

This is not used by any piece of code in Xen, so it won't hurt anything.

But I need this change in so I can proceed with the Linux kernel part:

we would like to upstream the relevant EDID change to the display

frontend driver and without the header in Xen tree we are stuck

Thank you in advance,

Oleksandr

>
> Juergen

Re: [BUG] Xen build for RCAR failing

[Oleksandr Andrushchenko]

On 7/7/20 10:58 AM, Manikandan Chockalingam (RBEI/ECF3) wrote:
>
> Hello Team,
>
> I am trying to build xen hypervisor for RCAR and following the 
> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Salvator-X
>  steps.
>
> But am facing the following issues
>
> 1.SRC_URI="http://v3.sk/~lkundrak/dev86/archive/Dev86src-${PV}.tar.gz in 
> recipes-extended/dev86/dev86_0.16.20.bb is not accesible
>
> *Modification 
> done:*SRC_URI=https://src.fedoraproject.org/lookaside/extras/dev86/Dev86src-0.16.20.tar.gz/567cf460d132f9d8775dd95f9208e49a/Dev86src-${PV}.tar.gz
>  
> 
>
You can try what we use [1]. And the issue you are facing is rather Yocto 
related, not R-Car specific, IMO

[1] 
https://github.com/xen-troops/meta-xt-prod-devel/blob/master/recipes-domd/domd-image-weston/files/meta-xt-prod-extra/recipes-extended/dev86/dev86_%25.bbappend

Re: [PATCH v2] xen/displif: Protocol version 2

[Oleksandr Andrushchenko]

On 7/2/20 10:20 AM, Jürgen Groß wrote:
> On 01.07.20 14:07, Oleksandr Andrushchenko wrote:
>> On 7/1/20 1:46 PM, Jürgen Groß wrote:
>>> On 01.07.20 09:19, Oleksandr Andrushchenko wrote:
>>>> From: Oleksandr Andrushchenko 
>>>>
>>>> 1. Add protocol version as an integer
>>>>
>>>> Version string, which is in fact an integer, is hard to handle in the
>>>> code that supports different protocol versions. To simplify that
>>>> also add the version as an integer.
>>>>
>>>> 2. Pass buffer offset with XENDISPL_OP_DBUF_CREATE
>>>>
>>>> There are cases when display data buffer is created with non-zero
>>>> offset to the data start. Handle such cases and provide that offset
>>>> while creating a display buffer.
>>>>
>>>> 3. Add XENDISPL_OP_GET_EDID command
>>>>
>>>> Add an optional request for reading Extended Display Identification
>>>> Data (EDID) structure which allows better configuration of the
>>>> display connectors over the configuration set in XenStore.
>>>> With this change connectors may have multiple resolutions defined
>>>> with respect to detailed timing definitions and additional properties
>>>> normally provided by displays.
>>>>
>>>> If this request is not supported by the backend then visible area
>>>> is defined by the relevant XenStore's "resolution" property.
>>>>
>>>> If backend provides extended display identification data (EDID) with
>>>> XENDISPL_OP_GET_EDID request then EDID values must take precedence
>>>> over the resolutions defined in XenStore.
>>>>
>>>> 4. Bump protocol version to 2.
>>>>
>>>> Signed-off-by: Oleksandr Andrushchenko 
>>>
>>> Reviewed-by: Juergen Gross 
>>
>> Thank you, do you want me to prepare the same for the kernel so
>>
>> you have it at hand when the time comes?
>
> It should be added to the kernel only when really needed (i.e. a user of
> the new functionality is showing up).

We have a patch for that which adds EDID to the existing PV DRM frontend,

so while upstreaming those changes I will also include changes to the protocol

in the kernel series: for that we need the header in Xen tree first, right?

>
>
> Juergen

Thank you,

Oleksandr

Re: [PATCH v2] xen/displif: Protocol version 2

[Oleksandr Andrushchenko]

On 7/1/20 1:46 PM, Jürgen Groß wrote:
> On 01.07.20 09:19, Oleksandr Andrushchenko wrote:
>> From: Oleksandr Andrushchenko 
>>
>> 1. Add protocol version as an integer
>>
>> Version string, which is in fact an integer, is hard to handle in the
>> code that supports different protocol versions. To simplify that
>> also add the version as an integer.
>>
>> 2. Pass buffer offset with XENDISPL_OP_DBUF_CREATE
>>
>> There are cases when display data buffer is created with non-zero
>> offset to the data start. Handle such cases and provide that offset
>> while creating a display buffer.
>>
>> 3. Add XENDISPL_OP_GET_EDID command
>>
>> Add an optional request for reading Extended Display Identification
>> Data (EDID) structure which allows better configuration of the
>> display connectors over the configuration set in XenStore.
>> With this change connectors may have multiple resolutions defined
>> with respect to detailed timing definitions and additional properties
>> normally provided by displays.
>>
>> If this request is not supported by the backend then visible area
>> is defined by the relevant XenStore's "resolution" property.
>>
>> If backend provides extended display identification data (EDID) with
>> XENDISPL_OP_GET_EDID request then EDID values must take precedence
>> over the resolutions defined in XenStore.
>>
>> 4. Bump protocol version to 2.
>>
>> Signed-off-by: Oleksandr Andrushchenko 
>
> Reviewed-by: Juergen Gross 

Thank you, do you want me to prepare the same for the kernel so

you have it at hand when the time comes?

>
>
> Juergen

[PATCH v2] xen/displif: Protocol version 2

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

1. Add protocol version as an integer

Version string, which is in fact an integer, is hard to handle in the
code that supports different protocol versions. To simplify that
also add the version as an integer.

2. Pass buffer offset with XENDISPL_OP_DBUF_CREATE

There are cases when display data buffer is created with non-zero
offset to the data start. Handle such cases and provide that offset
while creating a display buffer.

3. Add XENDISPL_OP_GET_EDID command

Add an optional request for reading Extended Display Identification
Data (EDID) structure which allows better configuration of the
display connectors over the configuration set in XenStore.
With this change connectors may have multiple resolutions defined
with respect to detailed timing definitions and additional properties
normally provided by displays.

If this request is not supported by the backend then visible area
is defined by the relevant XenStore's "resolution" property.

If backend provides extended display identification data (EDID) with
XENDISPL_OP_GET_EDID request then EDID values must take precedence
over the resolutions defined in XenStore.

4. Bump protocol version to 2.

Signed-off-by: Oleksandr Andrushchenko 
---
 xen/include/public/io/displif.h | 91 +++--
 1 file changed, 88 insertions(+), 3 deletions(-)

diff --git a/xen/include/public/io/displif.h b/xen/include/public/io/displif.h
index cc5de9cb1f35..0055895510f7 100644
--- a/xen/include/public/io/displif.h
+++ b/xen/include/public/io/displif.h
@@ -38,7 +38,8 @@
  *   Protocol version
  **
  */
-#define XENDISPL_PROTOCOL_VERSION "1"
+#define XENDISPL_PROTOCOL_VERSION "2"
+#define XENDISPL_PROTOCOL_VERSION_INT  2
 
 /*
  **
@@ -202,6 +203,9 @@
  *  Width and height of the connector in pixels separated by
  *  XENDISPL_RESOLUTION_SEPARATOR. This defines visible area of the
  *  display.
+ *  If backend provides extended display identification data (EDID) with
+ *  XENDISPL_OP_GET_EDID request then EDID values must take precedence
+ *  over the resolutions defined here.
  *
  *-- Connector Request Transport Parameters ---
  *
@@ -349,6 +353,8 @@
 #define XENDISPL_OP_FB_DETACH 0x13
 #define XENDISPL_OP_SET_CONFIG0x14
 #define XENDISPL_OP_PG_FLIP   0x15
+/* The below command is available in protocol version 2 and above. */
+#define XENDISPL_OP_GET_EDID  0x16
 
 /*
  **
@@ -377,6 +383,10 @@
 #define XENDISPL_FIELD_BE_ALLOC   "be-alloc"
 #define XENDISPL_FIELD_UNIQUE_ID  "unique-id"
 
+#define XENDISPL_EDID_BLOCK_SIZE  128
+#define XENDISPL_EDID_BLOCK_COUNT 256
+#define XENDISPL_EDID_MAX_SIZE(XENDISPL_EDID_BLOCK_SIZE * 
XENDISPL_EDID_BLOCK_COUNT)
+
 /*
  **
  *  STATUS RETURN CODES
@@ -451,7 +461,9 @@
  * +++++
  * |   gref_directory  | 40
  * +++++
- * | reserved  | 44
+ * | data_ofs  | 44
+ * +++++
+ * | reserved  | 48
  * +++++
  * |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/|
  * +++++
@@ -494,6 +506,7 @@
  *   buffer size (buffer_sz) exceeds what can be addressed by this single page,
  *   then reference to the next page must be supplied (see gref_dir_next_page
  *   below)
+ * data_ofs - uint32_t, offset of the data in the buffer, octets
  */
 
 #define XENDISPL_DBUF_FLG_REQ_ALLOC   (1 << 0)
@@ -506,6 +519,7 @@ struct xendispl_dbuf_create_req {
 uint32_t buffer_sz;
 uint32_t flags;
 grant_ref_t gref_directory;
+uint32_t data_ofs;
 };
 
 /*
@@ -731,6 +745,44 @@ struct xendispl_page_flip_req {
 uint64_t fb_cookie;
 };
 
+/*
+ * Request EDID - request EDID describing current connector:
+ * 01 2   3octet
+ * +++++
+ * |   id| _OP_GET_EDID   |   reserved | 4
+ * +++++
+ * |  

Re: [PATCH 2/2] libgnttab: Add support for Linux dma-buf offset

[Oleksandr Andrushchenko]

Ian, Wei, would you mind looking at the below please?

Thank you in advance,

Oleksandr

On 5/20/20 12:04 PM, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko 
>
> Add version 2 of the dma-buf ioctls which adds data_ofs parameter.
>
> dma-buf is backed by a scatter-gather table and has offset parameter
> which tells where the actual data starts. Relevant ioctls are extended
> to support that offset:
>- when dma-buf is created (exported) from grant references then
>  data_ofs is used to set the offset field in the scatter list
>  of the new dma-buf
>- when dma-buf is imported and grant references provided then
>  data_ofs is used to report that offset to user-space
>
> Signed-off-by: Oleksandr Andrushchenko 
> ---
>   tools/include/xen-sys/Linux/gntdev.h  | 53 +
>   tools/libs/gnttab/Makefile|  2 +-
>   tools/libs/gnttab/freebsd.c   | 15 +
>   tools/libs/gnttab/gnttab_core.c   | 17 ++
>   tools/libs/gnttab/include/xengnttab.h | 13 
>   tools/libs/gnttab/libxengnttab.map|  6 ++
>   tools/libs/gnttab/linux.c | 86 +++
>   tools/libs/gnttab/minios.c| 15 +
>   tools/libs/gnttab/private.h   |  9 +++
>   9 files changed, 215 insertions(+), 1 deletion(-)
>
> diff --git a/tools/include/xen-sys/Linux/gntdev.h 
> b/tools/include/xen-sys/Linux/gntdev.h
> index d16076044c71..0c43393cbee5 100644
> --- a/tools/include/xen-sys/Linux/gntdev.h
> +++ b/tools/include/xen-sys/Linux/gntdev.h
> @@ -274,4 +274,57 @@ struct ioctl_gntdev_dmabuf_imp_release {
>   uint32_t reserved;
>   };
>   
> +/*
> + * Version 2 of the ioctls adds @data_ofs parameter.
> + *
> + * dma-buf is backed by a scatter-gather table and has offset
> + * parameter which tells where the actual data starts.
> + * Relevant ioctls are extended to support that offset:
> + *   - when dma-buf is created (exported) from grant references then
> + * @data_ofs is used to set the offset field in the scatter list
> + * of the new dma-buf
> + *   - when dma-buf is imported and grant references are provided then
> + * @data_ofs is used to report that offset to user-space
> + */
> +#define IOCTL_GNTDEV_DMABUF_EXP_FROM_REFS_V2 \
> +_IOC(_IOC_NONE, 'G', 13, \
> + sizeof(struct ioctl_gntdev_dmabuf_exp_from_refs_v2))
> +struct ioctl_gntdev_dmabuf_exp_from_refs_v2 {
> +/* IN parameters. */
> +/* Specific options for this dma-buf: see GNTDEV_DMA_FLAG_XXX. */
> +uint32_t flags;
> +/* Number of grant references in @refs array. */
> +uint32_t count;
> +/* Offset of the data in the dma-buf. */
> +uint32_t data_ofs;
> +/* OUT parameters. */
> +/* File descriptor of the dma-buf. */
> +uint32_t fd;
> +/* The domain ID of the grant references to be mapped. */
> +uint32_t domid;
> +/* Variable IN parameter. */
> +/* Array of grant references of size @count. */
> +uint32_t refs[1];
> +};
> +
> +#define IOCTL_GNTDEV_DMABUF_IMP_TO_REFS_V2 \
> +_IOC(_IOC_NONE, 'G', 14, \
> + sizeof(struct ioctl_gntdev_dmabuf_imp_to_refs_v2))
> +struct ioctl_gntdev_dmabuf_imp_to_refs_v2 {
> +/* IN parameters. */
> +/* File descriptor of the dma-buf. */
> +uint32_t fd;
> +/* Number of grant references in @refs array. */
> +uint32_t count;
> +/* The domain ID for which references to be granted. */
> +uint32_t domid;
> +/* Reserved - must be zero. */
> +uint32_t reserved;
> +/* OUT parameters. */
> +/* Offset of the data in the dma-buf. */
> +uint32_t data_ofs;
> +/* Array of grant references of size @count. */
> +uint32_t refs[1];
> +};
> +
>   #endif /* __LINUX_PUBLIC_GNTDEV_H__ */
> diff --git a/tools/libs/gnttab/Makefile b/tools/libs/gnttab/Makefile
> index 2da8fbbb7f6f..5ee2d965214f 100644
> --- a/tools/libs/gnttab/Makefile
> +++ b/tools/libs/gnttab/Makefile
> @@ -2,7 +2,7 @@ XEN_ROOT = $(CURDIR)/../../..
>   include $(XEN_ROOT)/tools/Rules.mk
>   
>   MAJOR= 1
> -MINOR= 2
> +MINOR= 3
>   LIBNAME  := gnttab
>   USELIBS  := toollog toolcore
>   
> diff --git a/tools/libs/gnttab/freebsd.c b/tools/libs/gnttab/freebsd.c
> index 886b588303a0..baf0f60aa4d3 100644
> --- a/tools/libs/gnttab/freebsd.c
> +++ b/tools/libs/gnttab/freebsd.c
> @@ -319,6 +319,14 @@ int osdep_gnttab_dmabuf_exp_from_refs(xengnttab_handle 
> *xgt, uint32_t domid,
>   abort();
>   }
>   
> +int osdep_gnttab_dmabuf_exp_from_refs_v2(xengnttab_handle *xgt, uint32_t 
> domid,
> + uint32_t flags, uint32_t count,
> +

Re: [PATCH 1/2] xen/displif: Protocol version 2

[Oleksandr Andrushchenko]

On 6/30/20 10:30 AM, Jürgen Groß wrote:
> On 30.06.20 09:09, Oleksandr Andrushchenko wrote:
>> On 6/30/20 10:03 AM, Jürgen Groß wrote:
>>> On 30.06.20 08:13, Oleksandr Andrushchenko wrote:
>>>> On 6/29/20 10:02 AM, Jürgen Groß wrote:
>>>>> On 20.05.20 11:04, Oleksandr Andrushchenko wrote:
>>>>>> From: Oleksandr Andrushchenko 
>>>>>>
>>>>>> 1. Change protocol version from string to integer
>>>>>>
>>>>>> Version string, which is in fact an integer, is hard to handle in the
>>>>>> code that supports different protocol versions. To simplify that
>>>>>> make the version an integer.
>>>>>>
>>>>>> 2. Pass buffer offset with XENDISPL_OP_DBUF_CREATE
>>>>>>
>>>>>> There are cases when display data buffer is created with non-zero
>>>>>> offset to the data start. Handle such cases and provide that offset
>>>>>> while creating a display buffer.
>>>>>>
>>>>>> 3. Add XENDISPL_OP_GET_EDID command
>>>>>>
>>>>>> Add an optional request for reading Extended Display Identification
>>>>>> Data (EDID) structure which allows better configuration of the
>>>>>> display connectors over the configuration set in XenStore.
>>>>>> With this change connectors may have multiple resolutions defined
>>>>>> with respect to detailed timing definitions and additional properties
>>>>>> normally provided by displays.
>>>>>>
>>>>>> If this request is not supported by the backend then visible area
>>>>>> is defined by the relevant XenStore's "resolution" property.
>>>>>>
>>>>>> If backend provides extended display identification data (EDID) with
>>>>>> XENDISPL_OP_GET_EDID request then EDID values must take precedence
>>>>>> over the resolutions defined in XenStore.
>>>>>>
>>>>>> 4. Bump protocol version to 2.
>>>>>>
>>>>>> Signed-off-by: Oleksandr Andrushchenko 
>>>>>> ---
>>>>>>    xen/include/public/io/displif.h | 83 +++--
>>>>>>    1 file changed, 80 insertions(+), 3 deletions(-)
>>>>>>
>>>>>> diff --git a/xen/include/public/io/displif.h 
>>>>>> b/xen/include/public/io/displif.h
>>>>>> index cc5de9cb1f35..4d43ba5078c8 100644
>>>>>> --- a/xen/include/public/io/displif.h
>>>>>> +++ b/xen/include/public/io/displif.h
>>>>>> @@ -38,7 +38,7 @@
>>>>>>     *   Protocol version
>>>>>> **
>>>>>>     */
>>>>>> -#define XENDISPL_PROTOCOL_VERSION "1"
>>>>>> +#define XENDISPL_PROTOCOL_VERSION 2
>>>>>
>>>>> This is not very nice regarding compatibility.
>>>>>
>>>>> Can't you add a new macro for the integer value?
>>>>
>>>> We can leave it as is, e.g. define XENDISPL_PROTOCOL_VERSION as "2",
>>>>
>>>> so we do not break the existing users. Then if every user of the header has
>>>>
>>>> its local copy (this is what we now use in the display backend), then that
>>>> local copy can be changed in a way suitable for the concrete user, e.g. "2"
>>>>
>>>> redefined to 2. This cannot be done (?) for the Linux Kernel though.
>>>>
>>>> Or we can have
>>>>
>>>> #define XENDISPL_PROTOCOL_VERSION "2"
>>>>
>>>> #define XENDISPL_PROTOCOL_VERSION_INT  2
>>>>
>>>> Jurgen, what's your preference here?
>>>
>>> I would prefer the latter, as it avoids the need to modify the header
>>> when copying it to a local project.
>>>
>> Ok, I'll have 2 definitions then
>>
>> Anything else (beside the comments on new functionality) I can add/change
>>
>> before sending v2 of the patch?
>
> I would have said so. :-)

Thank you,

the series also has a patch for libgnttab. Do you want me to send the protocol 
patch

separately or should we wait for Ian and Wei to review? These changes are 
related,

thus I sent then togheter

>
>
> Juergen

Thank you,

Oleksandr

Re: [PATCH 1/2] xen/displif: Protocol version 2

[Oleksandr Andrushchenko]

On 6/30/20 10:03 AM, Jürgen Groß wrote:

On 30.06.20 08:13, Oleksandr Andrushchenko wrote:

On 6/29/20 10:02 AM, Jürgen Groß wrote:

On 20.05.20 11:04, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

1. Change protocol version from string to integer

Version string, which is in fact an integer, is hard to handle in the
code that supports different protocol versions. To simplify that
make the version an integer.

2. Pass buffer offset with XENDISPL_OP_DBUF_CREATE

There are cases when display data buffer is created with non-zero
offset to the data start. Handle such cases and provide that offset
while creating a display buffer.

3. Add XENDISPL_OP_GET_EDID command

Add an optional request for reading Extended Display Identification
Data (EDID) structure which allows better configuration of the
display connectors over the configuration set in XenStore.
With this change connectors may have multiple resolutions defined
with respect to detailed timing definitions and additional properties
normally provided by displays.

If this request is not supported by the backend then visible area
is defined by the relevant XenStore's "resolution" property.

If backend provides extended display identification data (EDID) with
XENDISPL_OP_GET_EDID request then EDID values must take precedence
over the resolutions defined in XenStore.

4. Bump protocol version to 2.

Signed-off-by: Oleksandr Andrushchenko 
---
   xen/include/public/io/displif.h | 83 +++--
   1 file changed, 80 insertions(+), 3 deletions(-)

diff --git a/xen/include/public/io/displif.h b/xen/include/public/io/displif.h
index cc5de9cb1f35..4d43ba5078c8 100644
--- a/xen/include/public/io/displif.h
+++ b/xen/include/public/io/displif.h
@@ -38,7 +38,7 @@
    *   Protocol version
**
    */
-#define XENDISPL_PROTOCOL_VERSION "1"
+#define XENDISPL_PROTOCOL_VERSION 2


This is not very nice regarding compatibility.

Can't you add a new macro for the integer value?


We can leave it as is, e.g. define XENDISPL_PROTOCOL_VERSION as "2",

so we do not break the existing users. Then if every user of the header has

its local copy (this is what we now use in the display backend), then that
local copy can be changed in a way suitable for the concrete user, e.g. "2"

redefined to 2. This cannot be done (?) for the Linux Kernel though.

Or we can have

#define XENDISPL_PROTOCOL_VERSION "2"

#define XENDISPL_PROTOCOL_VERSION_INT  2

Jurgen, what's your preference here?


I would prefer the latter, as it avoids the need to modify the header
when copying it to a local project.


Ok, I'll have 2 definitions then

Anything else (beside the comments on new functionality) I can add/change

before sending v2 of the patch?



Juergen

Re: [PATCH 1/2] xen/displif: Protocol version 2

[Oleksandr Andrushchenko]

On 6/29/20 10:02 AM, Jürgen Groß wrote:
> On 20.05.20 11:04, Oleksandr Andrushchenko wrote:
>> From: Oleksandr Andrushchenko 
>>
>> 1. Change protocol version from string to integer
>>
>> Version string, which is in fact an integer, is hard to handle in the
>> code that supports different protocol versions. To simplify that
>> make the version an integer.
>>
>> 2. Pass buffer offset with XENDISPL_OP_DBUF_CREATE
>>
>> There are cases when display data buffer is created with non-zero
>> offset to the data start. Handle such cases and provide that offset
>> while creating a display buffer.
>>
>> 3. Add XENDISPL_OP_GET_EDID command
>>
>> Add an optional request for reading Extended Display Identification
>> Data (EDID) structure which allows better configuration of the
>> display connectors over the configuration set in XenStore.
>> With this change connectors may have multiple resolutions defined
>> with respect to detailed timing definitions and additional properties
>> normally provided by displays.
>>
>> If this request is not supported by the backend then visible area
>> is defined by the relevant XenStore's "resolution" property.
>>
>> If backend provides extended display identification data (EDID) with
>> XENDISPL_OP_GET_EDID request then EDID values must take precedence
>> over the resolutions defined in XenStore.
>>
>> 4. Bump protocol version to 2.
>>
>> Signed-off-by: Oleksandr Andrushchenko 
>> ---
>>   xen/include/public/io/displif.h | 83 +++--
>>   1 file changed, 80 insertions(+), 3 deletions(-)
>>
>> diff --git a/xen/include/public/io/displif.h 
>> b/xen/include/public/io/displif.h
>> index cc5de9cb1f35..4d43ba5078c8 100644
>> --- a/xen/include/public/io/displif.h
>> +++ b/xen/include/public/io/displif.h
>> @@ -38,7 +38,7 @@
>>    *   Protocol version
>> **
>>    */
>> -#define XENDISPL_PROTOCOL_VERSION "1"
>> +#define XENDISPL_PROTOCOL_VERSION 2
>
> This is not very nice regarding compatibility.
>
> Can't you add a new macro for the integer value?

We can leave it as is, e.g. define XENDISPL_PROTOCOL_VERSION as "2",

so we do not break the existing users. Then if every user of the header has

its local copy (this is what we now use in the display backend), then that
local copy can be changed in a way suitable for the concrete user, e.g. "2"

redefined to 2. This cannot be done (?) for the Linux Kernel though.

Or we can have

#define XENDISPL_PROTOCOL_VERSION "2"

#define XENDISPL_PROTOCOL_VERSION_INT  2

Jurgen, what's your preference here?

>
> And please add comments further down which additions are related to
> the new version.
I will after the review is done and other comments are fixed
>
>
> Juergen

Thank you,

Oleksandr

Re: UEFI support in ARM DomUs

[Oleksandr Andrushchenko]

On 6/24/20 8:05 PM, Stefano Stabellini wrote:
> On Wed, 24 Jun 2020, Oleksandr Andrushchenko wrote:
>> On 6/23/20 8:31 AM, Oleksandr Andrushchenko wrote:
>>> On 6/23/20 4:20 AM, Stefano Stabellini wrote:
>>>> On Mon, 22 Jun 2020, Julien Grall wrote:
>>>>>>>> For the first part (__XEN_INTERFACE_VERSION__) I think we can provide 
>>>>>>>> it
>>>>>>>> via
>>>>>>>>
>>>>>>>> CFLAGS or something. This can also be done for the location of Xen
>>>>>>>> headers.
>>>>>>> __XEN_INTERFACE_VERSION__ should work through the CFLAGS. An alternative
>>>>>>> would be to allow the user to specify through the Kconfig.
>>>>>> You mean specifying via Kconfig something like "0x00040d00"?
>>>>> Possibly yes.
>>>>>
>>>>>> And what about the headers? How will we provide their location if we 
>>>>>> decide
>>>>>> not to include those
>>>>>>
>>>>>> in the tree?
>>>>> I would do through Kconfig as well.
>>>> If we specify the external location of the Xen headers via Kconfig, it
>>>> seems to me that we should be able to detect the interface version
>>>> automatically from any Makefile as part of the build. No need to ask the
>>>> user.
>>>>
>>>> However, if Oleksandr is thinking of using the Xen headers for the
>>>> hypercalls definitions, then I think we might not need external headers
>>>> at all because that is a stable interface, as Julien wrote. We could
>>>> just define our own few headers for just what you need like Linux does.
>>> This is a good idea: I'll try to get the minimal set of headers from Linux
>>>
>>> instead of Xen as those seem to be well prepared for such a use-case. This
>>>
>>> way we'll have headers in U-boot tree and guarantee that we have a minimal
>>>
>>> subset which is easier to maintain. I'll keep you updated on the progress
>> We've managed to strip the headers and remove __XEN__ and the rest 
>> definitions
>>
>> we were talking about. So, these are now the minimal required set of headers
>>
>> that allows U-boot to build serial and block drivers. Please take a look at 
>> [1]
>>
>> Pull request for comments is at [2]
> I think this is the right approach. There is no build-dependency on Xen
> anymore, is that correct?
No dependency

Re: UEFI support in ARM DomUs

[Oleksandr Andrushchenko]

On 6/24/20 10:26 AM, Peng Fan wrote:
>> Subject: Re: UEFI support in ARM DomUs
>>
>>
>> On 6/24/20 10:07 AM, Peng Fan wrote:
>>>> Subject: Re: UEFI support in ARM DomUs
>>>>
>>>>
>>>> On 6/23/20 8:31 AM, Oleksandr Andrushchenko wrote:
>>>>> On 6/23/20 4:20 AM, Stefano Stabellini wrote:
>>>>>> On Mon, 22 Jun 2020, Julien Grall wrote:
>>>>>>>>>> For the first part (__XEN_INTERFACE_VERSION__) I think we can
>>>>>>>>>> provide it via
>>>>>>>>>>
>>>>>>>>>> CFLAGS or something. This can also be done for the location of
>>>>>>>>>> Xen headers.
>>>>>>>>> __XEN_INTERFACE_VERSION__ should work through the CFLAGS.
>> An
>>>>>>>>> alternative would be to allow the user to specify through the
>> Kconfig.
>>>>>>>> You mean specifying via Kconfig something like "0x00040d00"?
>>>>>>> Possibly yes.
>>>>>>>
>>>>>>>> And what about the headers? How will we provide their location if
>>>>>>>> we decide not to include those
>>>>>>>>
>>>>>>>> in the tree?
>>>>>>> I would do through Kconfig as well.
>>>>>> If we specify the external location of the Xen headers via Kconfig,
>>>>>> it seems to me that we should be able to detect the interface
>>>>>> version automatically from any Makefile as part of the build. No
>>>>>> need to ask the user.
>>>>>>
>>>>>> However, if Oleksandr is thinking of using the Xen headers for the
>>>>>> hypercalls definitions, then I think we might not need external
>>>>>> headers at all because that is a stable interface, as Julien wrote.
>>>>>> We could just define our own few headers for just what you need
>>>>>> like Linux
>>>> does.
>>>>> This is a good idea: I'll try to get the minimal set of headers from
>>>>> Linux
>>>>>
>>>>> instead of Xen as those seem to be well prepared for such a use-case.
>>>>> This
>>>>>
>>>>> way we'll have headers in U-boot tree and guarantee that we have a
>>>>> minimal
>>>>>
>>>>> subset which is easier to maintain. I'll keep you updated on the
>>>>> progress
>>>> We've managed to strip the headers and remove __XEN__ and the rest
>>>> definitions
>>>>
>>>> we were talking about. So, these are now the minimal required set of
>>>> headers
>>>>
>>>> that allows U-boot to build serial and block drivers. Please take a
>>>> look at [1]
>>>>
>>>> Pull request for comments is at [2]
>>> The U-Boot new merge window will be open in 2020/7/1, so I'd suggest
>>> the patchset goes to U-Boot mail list for discussion if you wanna the
>>> patches gonna merged soon.
>> We definitely want the patches to be upstreamed and merged, but before
>> that
>>
>> we also want to make sure that Xen community is happy with what we
>> upstream
>>
>> I believe we resolved most of the concerns such as headers, PIE etc, so this
>> can
>>
>> be done.
>>
>> BTW, Peng, did you have a chance to try the pvblock driver yet?
> Not yet. I could have time to give a test next Monday. I think you not
> enable SPL, right?

No, we decided that we can run with U-boot proper, so we can have more

flexibility comparing to what we have in SPL. It seems that was the right

approach: you can jump to Linux or you can jump to the U-boot provided

by Android anyway, whatever your setup

>   So android dual bootloader feature not enabled.

I think this step will depend on the use-case you have: at the moment we have

a base build capable of booting Linux kernel, but this can easily be extended 
with

specific defconfig to build in boota command or whatever else is required.

> But SPL mostly not have MMU enabled..
>
> Regards,
> Peng.
>
>>> Regards,
>>> Peng.
>>>
>>>>>> If you can do that, I think it would be better because we decouple
>>>>>> the UBoot build from the Xen build completely. We don't even need
>>>>>> the Xen tree checked out to build UBoot. It would be a huge
>>>>>> advantage because it makes it far easier to build-test

Re: UEFI support in ARM DomUs

[Oleksandr Andrushchenko]

On 6/24/20 10:07 AM, Peng Fan wrote:
>> Subject: Re: UEFI support in ARM DomUs
>>
>>
>> On 6/23/20 8:31 AM, Oleksandr Andrushchenko wrote:
>>> On 6/23/20 4:20 AM, Stefano Stabellini wrote:
>>>> On Mon, 22 Jun 2020, Julien Grall wrote:
>>>>>>>> For the first part (__XEN_INTERFACE_VERSION__) I think we can
>>>>>>>> provide it via
>>>>>>>>
>>>>>>>> CFLAGS or something. This can also be done for the location of
>>>>>>>> Xen headers.
>>>>>>> __XEN_INTERFACE_VERSION__ should work through the CFLAGS. An
>>>>>>> alternative would be to allow the user to specify through the Kconfig.
>>>>>> You mean specifying via Kconfig something like "0x00040d00"?
>>>>> Possibly yes.
>>>>>
>>>>>> And what about the headers? How will we provide their location if
>>>>>> we decide not to include those
>>>>>>
>>>>>> in the tree?
>>>>> I would do through Kconfig as well.
>>>> If we specify the external location of the Xen headers via Kconfig,
>>>> it seems to me that we should be able to detect the interface version
>>>> automatically from any Makefile as part of the build. No need to ask
>>>> the user.
>>>>
>>>> However, if Oleksandr is thinking of using the Xen headers for the
>>>> hypercalls definitions, then I think we might not need external
>>>> headers at all because that is a stable interface, as Julien wrote.
>>>> We could just define our own few headers for just what you need like Linux
>> does.
>>> This is a good idea: I'll try to get the minimal set of headers from
>>> Linux
>>>
>>> instead of Xen as those seem to be well prepared for such a use-case.
>>> This
>>>
>>> way we'll have headers in U-boot tree and guarantee that we have a
>>> minimal
>>>
>>> subset which is easier to maintain. I'll keep you updated on the
>>> progress
>> We've managed to strip the headers and remove __XEN__ and the rest
>> definitions
>>
>> we were talking about. So, these are now the minimal required set of headers
>>
>> that allows U-boot to build serial and block drivers. Please take a look at 
>> [1]
>>
>> Pull request for comments is at [2]
> The U-Boot new merge window will be open in 2020/7/1, so I'd suggest
> the patchset goes to U-Boot mail list for discussion if you wanna the patches
> gonna merged soon.

We definitely want the patches to be upstreamed and merged, but before that

we also want to make sure that Xen community is happy with what we upstream

I believe we resolved most of the concerns such as headers, PIE etc, so this can

be done.

BTW, Peng, did you have a chance to try the pvblock driver yet?

>
> Regards,
> Peng.
>
>>>> If you can do that, I think it would be better because we decouple
>>>> the UBoot build from the Xen build completely. We don't even need the
>>>> Xen tree checked out to build UBoot. It would be a huge advantage
>>>> because it makes it far easier to build-test changes for others in
>>>> the community that don't know about Xen and also it becomes far
>>>> easier to integrate into any build system.
>> [1]
>> https://urldefense.com/v3/__https://eur01.safelinks.protection.outlook.com/?url=https*3A*2F*2Fgithub__;JSUl!!GF_29dbcQIUBPA!mwib3un6-vYBG68zMfurW6-0MuuER5tNmJtOitDpViICNkX0lhig8iPHmZokuM-BLQYpeKYAGQ$
>>  .
>> com%2Fandr2000%2Fu-boot%2Ftree%2Fpvblock_upstream_v1data=0
>> 2%7C01%7Cpeng.fan%40nxp.com%7C407d8af24a36483fbdce08d81805ed88
>> %7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637285761021975
>> 164sdata=5vWfBbLScICXPZWU%2BU3b7DyONcgxT8iICsxrwUbORZY%
>> 3Dreserved=0
>>
>> [2]
>> https://urldefense.com/v3/__https://eur01.safelinks.protection.outlook.com/?url=https*3A*2F*2Fgithub__;JSUl!!GF_29dbcQIUBPA!mwib3un6-vYBG68zMfurW6-0MuuER5tNmJtOitDpViICNkX0lhig8iPHmZokuM-BLQYpeKYAGQ$
>>  .
>> com%2Fxen-troops%2Fu-boot%2Fpull%2F2data=02%7C01%7Cpeng.fa
>> n%40nxp.com%7C407d8af24a36483fbdce08d81805ed88%7C686ea1d3bc2b4
>> c6fa92cd99c5c301635%7C0%7C0%7C637285761021975164sdata=%2
>> FmXheEvKssLjjaFKsHBBbqh%2B72jH3uQnE7cpN0J3k8I%3Dreserved=0

Re: UEFI support in ARM DomUs

[Oleksandr Andrushchenko]

On 6/23/20 8:31 AM, Oleksandr Andrushchenko wrote:
>
> On 6/23/20 4:20 AM, Stefano Stabellini wrote:
>> On Mon, 22 Jun 2020, Julien Grall wrote:
>>>>>> For the first part (__XEN_INTERFACE_VERSION__) I think we can provide it
>>>>>> via
>>>>>>
>>>>>> CFLAGS or something. This can also be done for the location of Xen
>>>>>> headers.
>>>>> __XEN_INTERFACE_VERSION__ should work through the CFLAGS. An alternative
>>>>> would be to allow the user to specify through the Kconfig.
>>>> You mean specifying via Kconfig something like "0x00040d00"?
>>> Possibly yes.
>>>
>>>> And what about the headers? How will we provide their location if we decide
>>>> not to include those
>>>>
>>>> in the tree?
>>> I would do through Kconfig as well.
>> If we specify the external location of the Xen headers via Kconfig, it
>> seems to me that we should be able to detect the interface version
>> automatically from any Makefile as part of the build. No need to ask the
>> user.
>>
>> However, if Oleksandr is thinking of using the Xen headers for the
>> hypercalls definitions, then I think we might not need external headers
>> at all because that is a stable interface, as Julien wrote. We could
>> just define our own few headers for just what you need like Linux does.
>
> This is a good idea: I'll try to get the minimal set of headers from Linux
>
> instead of Xen as those seem to be well prepared for such a use-case. This
>
> way we'll have headers in U-boot tree and guarantee that we have a minimal
>
> subset which is easier to maintain. I'll keep you updated on the progress

We've managed to strip the headers and remove __XEN__ and the rest definitions

we were talking about. So, these are now the minimal required set of headers

that allows U-boot to build serial and block drivers. Please take a look at [1]

Pull request for comments is at [2]

>
>>
>> If you can do that, I think it would be better because we decouple the
>> UBoot build from the Xen build completely. We don't even need the Xen
>> tree checked out to build UBoot. It would be a huge advantage because it
>> makes it far easier to build-test changes for others in the community
>> that don't know about Xen and also it becomes far easier to integrate
>> into any build system.

[1] https://github.com/andr2000/u-boot/tree/pvblock_upstream_v1

[2] https://github.com/xen-troops/u-boot/pull/2

Re: UEFI support in ARM DomUs

[Oleksandr Andrushchenko]

On 6/23/20 4:20 AM, Stefano Stabellini wrote:
> On Mon, 22 Jun 2020, Julien Grall wrote:
> For the first part (__XEN_INTERFACE_VERSION__) I think we can provide it
> via
>
> CFLAGS or something. This can also be done for the location of Xen
> headers.
 __XEN_INTERFACE_VERSION__ should work through the CFLAGS. An alternative
 would be to allow the user to specify through the Kconfig.
>>> You mean specifying via Kconfig something like "0x00040d00"?
>> Possibly yes.
>>
>>> And what about the headers? How will we provide their location if we decide
>>> not to include those
>>>
>>> in the tree?
>> I would do through Kconfig as well.
> If we specify the external location of the Xen headers via Kconfig, it
> seems to me that we should be able to detect the interface version
> automatically from any Makefile as part of the build. No need to ask the
> user.
>
> However, if Oleksandr is thinking of using the Xen headers for the
> hypercalls definitions, then I think we might not need external headers
> at all because that is a stable interface, as Julien wrote. We could
> just define our own few headers for just what you need like Linux does.

This is a good idea: I'll try to get the minimal set of headers from Linux

instead of Xen as those seem to be well prepared for such a use-case. This

way we'll have headers in U-boot tree and guarantee that we have a minimal

subset which is easier to maintain. I'll keep you updated on the progress

>
> If you can do that, I think it would be better because we decouple the
> UBoot build from the Xen build completely. We don't even need the Xen
> tree checked out to build UBoot. It would be a huge advantage because it
> makes it far easier to build-test changes for others in the community
> that don't know about Xen and also it becomes far easier to integrate
> into any build system.

Re: UEFI support in ARM DomUs

[Oleksandr Andrushchenko]

On 6/22/20 5:27 PM, Julien Grall wrote:
> Hi Oleksandr,
>
> On 22/06/2020 15:04, Oleksandr Andrushchenko wrote:
>> On 6/19/20 11:02 PM, Stefano Stabellini wrote:
>>> On Thu, 18 Jun 2020, Julien Grall wrote:
>> ifeq ($(CONFIG_XEN),y)
>> arch-y += -D__XEN_INTERFACE_VERSION__=0x00040d00
>> endif
>>
>> and we also have Xen 4.13 headers in the U-boot tree.
>
> Sorry if this was already asked before. Why do you need to specify 
> __XEN_INTERFACE_VERSION__?

This is because of include/xen/interface/xen-compat.h:

#if defined(__XEN__) || defined(__XEN_TOOLS__)

/* Xen is built with matching headers and implements the latest interface. */
#define __XEN_INTERFACE_VERSION__ __XEN_LATEST_INTERFACE_VERSION__
#elif !defined(__XEN_INTERFACE_VERSION__)
/* Guests which do not specify a version get the legacy interface. */
#define __XEN_INTERFACE_VERSION__ 0x
#endif

So, one needs to specify the version (QEMU does that via its configure script 
after

some tests)

>
>>
>> For the first part (__XEN_INTERFACE_VERSION__) I think we can provide it via
>>
>> CFLAGS or something. This can also be done for the location of Xen headers.
>
> __XEN_INTERFACE_VERSION__ should work through the CFLAGS. An alternative 
> would be to allow the user to specify through the Kconfig.

You mean specifying via Kconfig something like "0x00040d00"?

And what about the headers? How will we provide their location if we decide not 
to include those

in the tree?

> Cheers,
>

Re: UEFI support in ARM DomUs

[Oleksandr Andrushchenko]

On 6/19/20 11:02 PM, Stefano Stabellini wrote:
> On Thu, 18 Jun 2020, Julien Grall wrote:
>>> Copy/pasting here from my comment on the pull request plus additional
>>> thoughts.
>>>
>>> Uboot is one of those embedded projects that typically assumes that all
>>> the information about the platform is available at *build time*. It is
>>> meant to be built tailored for a specific version of a specific board. A
>>> Uboot binary is not expected to be "portable" across different versions
>>> of the platform or different platforms. In other words, it is not
>>> expected to be portable across Xen versions.
>> Can you define "version" here? Do you refer to the difference in terms
>> of memory?
>   
> Yes, I meant any meaningful differences in any of the external
> interfaces that end up impacting the UBoot implementation. A primary
> example would be the memory addresses for instance.
>
>
>>> This is a different model meant for different use-cases. I don't think
>>> it is a problem "philosophically" to let Uboot know about Xen details at
>>> build time. Yes, that is not what we did historically but it is very
>>> much in the spirit of Uboot.
>> TBH, I don't particularly mind that U-boot is built against a specific
>> version of Xen. I am more concerned about the long term implication if
>> we endorse it
>> and then try to change the memory layout in depth.
> I'll provide more information below.
>
>
>>> But of course the least Uboot depends on these details the better
>>> because it will be more flexible, but it could very well be that we
>>> won't be able to reach the point where the binary works on any version
>>> like we did with Tianocore. The two projects work differently.
>> Can we have a list of things U-boot require to know at compile time?
>>
>> In particular, I would like to understand if it would be sufficient to
>> only be aware of the first bank. If it is, then my preference would be
>> to standardize that bit of the layout.
> That would be my preference too, and it was great to read that it looks
> like it can be done. Yay!
>
>
>>> That said, I think Julien is right that we need to be careful on how we
>>> expose these information to Uboot, i.e. defining __XEN__ to build Uboot
>>> doesn't seem like a good idea because we enable definitions that were
>>> never meant to be used outside of a Xen build. Also, it wouldn't be easy
>>> to know exactly which definitions are actively used by Uboot and which
>>> ones aren't.
>>>
>>> If we are going to make Uboot dependent on version-specific information
>>> of the Xen interface, it would be better to be very clear about which
>>> definitions we are using. So that one day if we need to change them, we
>>> can find them easily.
>>>
>>> So I think it would be better to introduce a set of defines with the
>>> minimum amount of information required by Uboot statically. That way,
>>> at least we have a single place where to find all the version-specific
>>> definitions that Uboot is using.
>> I am not sure what you are suggesting. Can you expand a bit more?
>>
>>> We can also manage versioning of the
>>> Xen interface (like we do in QEMU) if we have to.
>> Can you provide more details about the compatibility? I am quite
>> interested in the part where you would have to deal with an older QEMU
>> version built against a new Xen?
> Sure let me expand a bit more. I'll switch "hat" to "QEMU (or UBoot)
> contributor" for the sake of this discussion.
>
> Xen Project offers certain stability guarantees on some interfaces and
> not others. Let's say that for any reason you have to or want to use one
> of the unstable interfaces in QEMU/UBoot/Whatever. Then it becomes your
> responsibility as QEMU developer to maintain compatibility in QEMU
> itself.
>
> First I'd add code to detect the version of the interface. The detection
> is based on the Xen headers/libraries provided. In QEMU it is done in
> the "configure" script. It sets a preprocessor #define to the version
> of the interface (e.g. CONFIG_XEN_CTRL_INTERFACE_VERSION == 41100).

I looked at QEMU's configure script and I'm afraid we can't have the

same flexibility in U-boot: we don't have configure script, pkg-config

is not widely used etc. So, for now we have hardcoded:

ifeq ($(CONFIG_XEN),y)
arch-y += -D__XEN_INTERFACE_VERSION__=0x00040d00
endif

and we also have Xen 4.13 headers in the U-boot tree.

For the first part (__XEN_INTERFACE_VERSION__) I think we can provide it via

CFLAGS or something. This can also be done for the location of Xen headers.

We don't have strong opinion here, so would love to hear from Xen community on 
this

Current WIP with the fixes requested by Julien are at [1]: can be and will be 
force pushed

as we are still working on it, but can give you an impression of what we have 
as of now

>
> Then you can have preprocessors checks in one of the headers such as:
>
> #if CONFIG_XEN_CTRL_INTERFACE_VERSION < 40701
>  #define xenevtchn_open(l, f) xc_evtchn_open(l, f);
> #else
>  XXX

Re: UEFI support in ARM DomUs

[Oleksandr Andrushchenko]

On 6/19/20 4:29 PM, Oleksandr Andrushchenko wrote:
> On 6/19/20 4:15 PM, Julien Grall wrote:
>>
>>
>> On 19/06/2020 14:06, Oleksandr Andrushchenko wrote:
>>>
>>> On 6/19/20 3:59 PM, Julien Grall wrote:
>>>> Hi,
>>>>
>>>> On 19/06/2020 13:51, Oleksandr Andrushchenko wrote:
>>>>> On 6/19/20 3:47 PM, Julien Grall wrote:
>>>>>> They will not be available from the fdt, but you can retrieve them with 
>>>>>> an hypervisor call (see HVM_PARAM_STORE_PFN, HVM_PARAM_CONSOLE_PFN).
>>>>> Yes, and it used in the relevant pieces of code (hyp calls)
>>>>>> One question though, why do you need to map them in advance? Couldn't 
>>>>>> you map them on demand?
>>>>>
>>>>> Well, we need to at least estimate the pg_table size so we can reserve 
>>>>> and allocate memory later,
>>>>
>>>> Oh, so U-boot doesn't support runtime page-table table allocation. Is that 
>>>> right?
>>> As per my understanding no, we provide a memory map and the tables are 
>>> allocated beforehand
>>
>> Ok :(.
>>
>>>>
>>>>>
>>>>> so I have to provide memory range from either by coding a constant or 
>>>>> looking into the devtree at
>>>>>
>>>>> hypervisor { reg = <>; }. It is a bit tricky though
>>>>
>>>> Looking for a node in the device-tree shouldn't be too difficult given 
>>>> that you have fdt_* available.
>>>>
>>>> However, please not that  doesn't refer to the guest magic pages. 
>>>> Instead, it provides a region you can use for mapping the grant-table 
>>>> frames
>>>
>>> Indeed, this is in my case 0x3800, but the magic is at 0x3900
>>>
>>> So, I need the memory range set up beforehand, but I can't as there is no 
>>> cute way to get that.
>>>
>>> Of course, I can issue a hyp call to get HVM_PARAM_CONSOLE_PFN and use it 
>>> as the base address,
>>>
>>> but this smells like a hack. I can call other HVM_PARAM_ to get their pfns 
>>> and set up the memory regions,
>>>
>>> but this looks a bit weird.
>>
>> Why is it weird? In general, you only want to map exactly what you need. Not 
>> less, not more.
>>
>> In your situation, you only care about two pages, the console page and the 
>> xenstore page. The rest shouldn't be mapped.
> Ok, so I'll try get pfns for HVM_PARAM_CONSOLE_PFN + XENSTORE_PFN_OFFSET via 
> hyp call and map those
>>
>>> I need that constant badly ;)
>>
>> No you don't ;).

We have managed to make use of the relevant hypercalls to get the PFNs, but for 
that

we need to maintain the caches as this happens (the calls) when MMU is not yet

setup and is in the process of.

>>
>> Cheers,
>>
> Thanks for helping with this

Re: UEFI support in ARM DomUs

[Oleksandr Andrushchenko]

On 6/19/20 4:16 PM, Peng Fan wrote:

Subject: Re: UEFI support in ARM DomUs


On 6/19/20 3:59 PM, Julien Grall wrote:

Hi,

On 19/06/2020 13:51, Oleksandr Andrushchenko wrote:

On 6/19/20 3:47 PM, Julien Grall wrote:

They will not be available from the fdt, but you can retrieve them with an

hypervisor call (see HVM_PARAM_STORE_PFN,
HVM_PARAM_CONSOLE_PFN).

Yes, and it used in the relevant pieces of code (hyp calls)

One question though, why do you need to map them in advance?

Couldn't you map them on demand?

Well, we need to at least estimate the pg_table size so we can reserve and

allocate memory later,

Oh, so U-boot doesn't support runtime page-table table allocation. Is that

right?
As per my understanding no, we provide a memory map and the tables are
allocated beforehand

You are right.

ok, so then we only have a choice of probing the range via hyp calls

Regards,
Peng.

so I have to provide memory range from either by coding a constant or

looking into the devtree at

hypervisor { reg = <>; }. It is a bit tricky though

Looking for a node in the device-tree shouldn't be too difficult given that you

have fdt_* available.

However, please not that  doesn't refer to the guest magic pages.

Instead, it provides a region you can use for mapping the grant-table frames

Indeed, this is in my case 0x3800, but the magic is at 0x3900

So, I need the memory range set up beforehand, but I can't as there is no cute
way to get that.

Of course, I can issue a hyp call to get HVM_PARAM_CONSOLE_PFN and use it
as the base address,

but this smells like a hack. I can call other HVM_PARAM_ to get their pfns and
set up the memory regions,

but this looks a bit weird. I need that constant badly ;)


Cheers,

Re: UEFI support in ARM DomUs

[Oleksandr Andrushchenko]

On 6/19/20 4:15 PM, Julien Grall wrote:



On 19/06/2020 14:06, Oleksandr Andrushchenko wrote:


On 6/19/20 3:59 PM, Julien Grall wrote:

Hi,

On 19/06/2020 13:51, Oleksandr Andrushchenko wrote:

On 6/19/20 3:47 PM, Julien Grall wrote:

They will not be available from the fdt, but you can retrieve them with an 
hypervisor call (see HVM_PARAM_STORE_PFN, HVM_PARAM_CONSOLE_PFN).

Yes, and it used in the relevant pieces of code (hyp calls)

One question though, why do you need to map them in advance? Couldn't you map 
them on demand?


Well, we need to at least estimate the pg_table size so we can reserve and 
allocate memory later,


Oh, so U-boot doesn't support runtime page-table table allocation. Is that 
right?

As per my understanding no, we provide a memory map and the tables are 
allocated beforehand


Ok :(.





so I have to provide memory range from either by coding a constant or looking 
into the devtree at

hypervisor { reg = <>; }. It is a bit tricky though


Looking for a node in the device-tree shouldn't be too difficult given that you 
have fdt_* available.

However, please not that  doesn't refer to the guest magic pages. Instead, 
it provides a region you can use for mapping the grant-table frames


Indeed, this is in my case 0x3800, but the magic is at 0x3900

So, I need the memory range set up beforehand, but I can't as there is no cute 
way to get that.

Of course, I can issue a hyp call to get HVM_PARAM_CONSOLE_PFN and use it as 
the base address,

but this smells like a hack. I can call other HVM_PARAM_ to get their pfns and 
set up the memory regions,

but this looks a bit weird.


Why is it weird? In general, you only want to map exactly what you need. Not 
less, not more.

In your situation, you only care about two pages, the console page and the 
xenstore page. The rest shouldn't be mapped.

Ok, so I'll try get pfns for HVM_PARAM_CONSOLE_PFN + XENSTORE_PFN_OFFSET via 
hyp call and map those



I need that constant badly ;)


No you don't ;).

Cheers,


Thanks for helping with this

Re: UEFI support in ARM DomUs

[Oleksandr Andrushchenko]

On 6/19/20 3:59 PM, Julien Grall wrote:
> Hi,
>
> On 19/06/2020 13:51, Oleksandr Andrushchenko wrote:
>> On 6/19/20 3:47 PM, Julien Grall wrote:
>>> They will not be available from the fdt, but you can retrieve them with an 
>>> hypervisor call (see HVM_PARAM_STORE_PFN, HVM_PARAM_CONSOLE_PFN).
>> Yes, and it used in the relevant pieces of code (hyp calls)
>>> One question though, why do you need to map them in advance? Couldn't you 
>>> map them on demand?
>>
>> Well, we need to at least estimate the pg_table size so we can reserve and 
>> allocate memory later,
>
> Oh, so U-boot doesn't support runtime page-table table allocation. Is that 
> right?
As per my understanding no, we provide a memory map and the tables are 
allocated beforehand
>
>>
>> so I have to provide memory range from either by coding a constant or 
>> looking into the devtree at
>>
>> hypervisor { reg = <>; }. It is a bit tricky though
>
> Looking for a node in the device-tree shouldn't be too difficult given that 
> you have fdt_* available.
>
> However, please not that  doesn't refer to the guest magic pages. 
> Instead, it provides a region you can use for mapping the grant-table frames

Indeed, this is in my case 0x3800, but the magic is at 0x3900

So, I need the memory range set up beforehand, but I can't as there is no cute 
way to get that.

Of course, I can issue a hyp call to get HVM_PARAM_CONSOLE_PFN and use it as 
the base address,

but this smells like a hack. I can call other HVM_PARAM_ to get their pfns and 
set up the memory regions,

but this looks a bit weird. I need that constant badly ;)

>
> Cheers,
>

Re: UEFI support in ARM DomUs

[Oleksandr Andrushchenko]

On 6/19/20 3:47 PM, Julien Grall wrote:
> Hi Oleksandr,
>
> On 19/06/2020 13:32, Oleksandr Andrushchenko wrote:
>>
>> On 6/19/20 1:49 AM, Julien Grall wrote:
>>> On Thu, 18 Jun 2020 at 23:00, Stefano Stabellini  
>>> wrote:
>>>> On Thu, 18 Jun 2020, Julien Grall wrote:
>>>>> On 18/06/2020 06:22, Oleksandr Andrushchenko wrote:
>>>>>> On 6/4/20 6:31 PM, Stefano Stabellini wrote:
>>>>>>> On Thu, 4 Jun 2020, Oleksandr Andrushchenko wrote:
>>>>>>>> On 6/4/20 4:57 AM, Peng Fan wrote:
>>>>>>>>> Grall ;
>>>>>>>>>> Nataliya Korovkina 
>>>>>>>>>> Subject: UEFI support in ARM DomUs
>>>>>>>>> We have made U-Boot run inside XEN DomU, but just only PV console
>>>>>>>>> part,
>>>>>>>>> not implement other frontend drivers currently. Would this help for
>>>>>>>>> your
>>>>>>>>> case if enable EFI in U-Boot?
>>>>>>>> Well, we have a working PV block implementation on top of that on iMX8
>>>>>>>>
>>>>>>>> platform, mostly ported from mini-os. Currently we are finalizing the
>>>>>>>> work
>>>>>>>>
>>>>>>>> and cleaning up (it's going to take a week or so hopefully). Then, we
>>>>>>>> we'll post
>>>>>>>>
>>>>>>>> it on our public github. We are also thinking about upstreaming the
>>>>>>>> work, but it may
>>>>>>>>
>>>>>>>> take quite some time if the whole idea fits u-boot's view on such an
>>>>>>>> extension at all.
>>>>>>> Yes please to both of you! :-)
>>>>>>>
>>>>>>> In the meantime, while we wait for those changes to go upstream in
>>>>>>> uboot, could you please post a branch on github and a link on this email
>>>>>>> thread?
>>>>>> It took a bit more time than we expected, but here we go [1]:
>>>>>>
>>>>>> this is in form of a pull-request as we would love to hear from the
>>>>>>
>>>>>> community and it is easier to discuss the code (please leave comments 
>>>>>> there)
>>>>>>
>>>>>> 1. There is code originating from MiniOS and work done by Peng, so we
>>>>>>
>>>>>> would like to ask the respective copyright owners to raise their hands 
>>>>>> and
>>>>> Not everyone are closely watching xen-devel. So if you want to find out 
>>>>> who
>>>>> are the copyright owners, then your best solution is to go through the 
>>>>> git log
>>>>> and CC the authors.
>>>> That is true. But why do you want to contact them? It doesn't look like
>>>> there would be any licensing issues.
>>>   From the sentence, I wasn't entirely sure whether he wanted to contact
>>> the copyright owner for crediting them in the headers.
>>>
>>>>>> 5. We use -D__XEN__ to access some of the hidden defines we need such as
>>>>>>
>>>>>> GUEST_RAM0_BASE and the friends as there is no other way but manually
>>>>>> defining the
>>>>>>
>>>>>> same which is not cute.
>>>>> I have replied to this in the pull request. But I want to bring the
>>>>> conversation here to have a wider discussion.
>>>>>
>>>>> For a first, __XEN__ should really only be defined by the hypervisor and 
>>>>> not
>>>>> used by the guest. This is used to gate non-stable ABI (such as the 
>>>>> tools) and
>>>>> anything behind it hasn't been vetted to work in other build configuration
>>>>> that the one used by Xen.
>>>>>
>>>>> In general, we expect the guest to discover everything for the device-tree
>>>>> because the memory layout is not stable (we want to be able to reshuffle 
>>>>> as we
>>>>> add more features).
>>>>>
>>>>> I know that EDK2/Tianocore can be built once and work on every Xen
>>>>> configuration. It would be ideal that U-boot follow the same. If it is 
>>>>> really
>>>>> not possible, then we should exp

Re: UEFI support in ARM DomUs

[Oleksandr Andrushchenko]

On 6/19/20 1:49 AM, Julien Grall wrote:
> On Thu, 18 Jun 2020 at 23:00, Stefano Stabellini  
> wrote:
>> On Thu, 18 Jun 2020, Julien Grall wrote:
>>> On 18/06/2020 06:22, Oleksandr Andrushchenko wrote:
>>>> On 6/4/20 6:31 PM, Stefano Stabellini wrote:
>>>>> On Thu, 4 Jun 2020, Oleksandr Andrushchenko wrote:
>>>>>> On 6/4/20 4:57 AM, Peng Fan wrote:
>>>>>>> Grall ;
>>>>>>>> Nataliya Korovkina 
>>>>>>>> Subject: UEFI support in ARM DomUs
>>>>>>> We have made U-Boot run inside XEN DomU, but just only PV console
>>>>>>> part,
>>>>>>> not implement other frontend drivers currently. Would this help for
>>>>>>> your
>>>>>>> case if enable EFI in U-Boot?
>>>>>> Well, we have a working PV block implementation on top of that on iMX8
>>>>>>
>>>>>> platform, mostly ported from mini-os. Currently we are finalizing the
>>>>>> work
>>>>>>
>>>>>> and cleaning up (it's going to take a week or so hopefully). Then, we
>>>>>> we'll post
>>>>>>
>>>>>> it on our public github. We are also thinking about upstreaming the
>>>>>> work, but it may
>>>>>>
>>>>>> take quite some time if the whole idea fits u-boot's view on such an
>>>>>> extension at all.
>>>>> Yes please to both of you! :-)
>>>>>
>>>>> In the meantime, while we wait for those changes to go upstream in
>>>>> uboot, could you please post a branch on github and a link on this email
>>>>> thread?
>>>> It took a bit more time than we expected, but here we go [1]:
>>>>
>>>> this is in form of a pull-request as we would love to hear from the
>>>>
>>>> community and it is easier to discuss the code (please leave comments 
>>>> there)
>>>>
>>>> 1. There is code originating from MiniOS and work done by Peng, so we
>>>>
>>>> would like to ask the respective copyright owners to raise their hands and
>>> Not everyone are closely watching xen-devel. So if you want to find out who
>>> are the copyright owners, then your best solution is to go through the git 
>>> log
>>> and CC the authors.
>> That is true. But why do you want to contact them? It doesn't look like
>> there would be any licensing issues.
>  From the sentence, I wasn't entirely sure whether he wanted to contact
> the copyright owner for crediting them in the headers.
>
>>>> 5. We use -D__XEN__ to access some of the hidden defines we need such as
>>>>
>>>> GUEST_RAM0_BASE and the friends as there is no other way but manually
>>>> defining the
>>>>
>>>> same which is not cute.
>>> I have replied to this in the pull request. But I want to bring the
>>> conversation here to have a wider discussion.
>>>
>>> For a first, __XEN__ should really only be defined by the hypervisor and not
>>> used by the guest. This is used to gate non-stable ABI (such as the tools) 
>>> and
>>> anything behind it hasn't been vetted to work in other build configuration
>>> that the one used by Xen.
>>>
>>> In general, we expect the guest to discover everything for the device-tree
>>> because the memory layout is not stable (we want to be able to reshuffle as 
>>> we
>>> add more features).
>>>
>>> I know that EDK2/Tianocore can be built once and work on every Xen
>>> configuration. It would be ideal that U-boot follow the same. If it is 
>>> really
>>> not possible, then we should explore a path that is preventing to define
>>> __XEN__.
>>>
>>> How much does U-boot expect to know about the memory layout? Does it require
>>> to know all the memory banks? Or would it be sufficient for it to know the
>>> start address of the first bank and the minimum of RAM?
>> Copy/pasting here from my comment on the pull request plus additional
>> thoughts.
>>
>> Uboot is one of those embedded projects that typically assumes that all
>> the information about the platform is available at *build time*. It is
>> meant to be built tailored for a specific version of a specific board. A
>> Uboot binary is not expected to be "portable" across different versions
>> of the platform or different platforms. In other wo

Re: UEFI support in ARM DomUs

[Oleksandr Andrushchenko]

On 6/18/20 5:50 PM, Julien Grall wrote:
>
>
> On 18/06/2020 06:22, Oleksandr Andrushchenko wrote:
>>
>> On 6/4/20 6:31 PM, Stefano Stabellini wrote:
>>> On Thu, 4 Jun 2020, Oleksandr Andrushchenko wrote:
>>>> On 6/4/20 4:57 AM, Peng Fan wrote:
>>>>> Grall ;
>>>>>> Nataliya Korovkina 
>>>>>> Subject: UEFI support in ARM DomUs
>>>>> We have made U-Boot run inside XEN DomU, but just only PV console part,
>>>>> not implement other frontend drivers currently. Would this help for your
>>>>> case if enable EFI in U-Boot?
>>>> Well, we have a working PV block implementation on top of that on iMX8
>>>>
>>>> platform, mostly ported from mini-os. Currently we are finalizing the work
>>>>
>>>> and cleaning up (it's going to take a week or so hopefully). Then, we 
>>>> we'll post
>>>>
>>>> it on our public github. We are also thinking about upstreaming the work, 
>>>> but it may
>>>>
>>>> take quite some time if the whole idea fits u-boot's view on such an 
>>>> extension at all.
>>> Yes please to both of you! :-)
>>>
>>> In the meantime, while we wait for those changes to go upstream in
>>> uboot, could you please post a branch on github and a link on this email
>>> thread?
>>
>> It took a bit more time than we expected, but here we go [1]:
>>
>> this is in form of a pull-request as we would love to hear from the
>>
>> community and it is easier to discuss the code (please leave comments there)
>>
>> 1. There is code originating from MiniOS and work done by Peng, so we
>>
>> would like to ask the respective copyright owners to raise their hands and
>
> Not everyone are closely watching xen-devel. So if you want to find out who 
> are the copyright owners, then your best solution is to go through the git 
> log and CC the authors.

We didn't want to contact the respective authors and copyright owners (some of 
those

are dated 2005 or so). This is to stress that we are trying hard to put all the 
copyrights

and not miss anyone: there is no intention to put our copyright at some 
inappropriate

place and remove someones else.

>
>>
>> let us *fix inappropriate licensing* if any.
>>
>> 2. Please note, the series has a HACK to move the RAM base as it is expected 
>> by
>>
>> our test platform (iMX8), so others will need to remove or modify that.
>>
>> 3. There is a limitation already noted by Peng that we do not have serial 
>> output
>>
>> until MMU is setup, so we have introduced xen_early_printk helper which 
>> always
>>
>> works, so you can use that for early stage debugging.
>>
>> 4. Minimal memory size supported is ~129M because of dtb placement by Xen 
>> tools
>
> Hmmm... Why? What's wrong with booting a guest with just 64MB?

Because of the bug in U-boot [1]: it tries to read beyond the physical memory 
if guest

has less than 128M+ as only then Xen tools put the fdt at 128M and not at the 
RAM end.

>
>>
>> 5. We use -D__XEN__ to access some of the hidden defines we need such as
>>
>> GUEST_RAM0_BASE and the friends as there is no other way but manually 
>> defining the
>>
>> same which is not cute.
>
> I have replied to this in the pull request. But I want to bring the 
> conversation here to have a wider discussion.
>
> For a first, __XEN__ should really only be defined by the hypervisor and not 
> used by the guest. This is used to gate non-stable ABI (such as the tools) 
> and anything behind it hasn't been vetted to work in other build 
> configuration that the one used by Xen.
>
> In general, we expect the guest to discover everything for the device-tree 
> because the memory layout is not stable (we want to be able to reshuffle as 
> we add more features).
>
> I know that EDK2/Tianocore can be built once and work on every Xen 
> configuration. It would be ideal that U-boot follow the same. If it is really 
> not possible, then we should explore a path that is preventing to define 
> __XEN__.
>
> How much does U-boot expect to know about the memory layout? Does it require 
> to know all the memory banks? Or would it be sufficient for it to know the 
> start address of the first bank and the minimum of RAM?
>
> Cheers,
>
[1] 
https://github.com/xen-troops/u-boot/pull/1/commits/9c639bd514961e70cfb2ebed9d8badb7b22d21ab

Re: UEFI support in ARM DomUs

[Oleksandr Andrushchenko]

On 6/4/20 6:31 PM, Stefano Stabellini wrote:
> On Thu, 4 Jun 2020, Oleksandr Andrushchenko wrote:
>> On 6/4/20 4:57 AM, Peng Fan wrote:
>>> Grall ;
>>>> Nataliya Korovkina 
>>>> Subject: UEFI support in ARM DomUs
>>> We have made U-Boot run inside XEN DomU, but just only PV console part,
>>> not implement other frontend drivers currently. Would this help for your
>>> case if enable EFI in U-Boot?
>> Well, we have a working PV block implementation on top of that on iMX8
>>
>> platform, mostly ported from mini-os. Currently we are finalizing the work
>>
>> and cleaning up (it's going to take a week or so hopefully). Then, we we'll 
>> post
>>
>> it on our public github. We are also thinking about upstreaming the work, 
>> but it may
>>
>> take quite some time if the whole idea fits u-boot's view on such an 
>> extension at all.
> Yes please to both of you! :-)
>
> In the meantime, while we wait for those changes to go upstream in
> uboot, could you please post a branch on github and a link on this email
> thread?

It took a bit more time than we expected, but here we go [1]:

this is in form of a pull-request as we would love to hear from the

community and it is easier to discuss the code (please leave comments there)

1. There is code originating from MiniOS and work done by Peng, so we

would like to ask the respective copyright owners to raise their hands and

let us *fix inappropriate licensing* if any.

2. Please note, the series has a HACK to move the RAM base as it is expected by

our test platform (iMX8), so others will need to remove or modify that.

3. There is a limitation already noted by Peng that we do not have serial output

until MMU is setup, so we have introduced xen_early_printk helper which always

works, so you can use that for early stage debugging.

4. Minimal memory size supported is ~129M because of dtb placement by Xen tools

5. We use -D__XEN__ to access some of the hidden defines we need such as

GUEST_RAM0_BASE and the friends as there is no other way but manually defining 
the

same which is not cute.

Have fun,

Anastasiia, Oleksandr

>
> Maybe we should have a wikipage on wiki.xenproject.org about
> work-in-progress uboot items.
>
>
>
>
>>> Regards,
>>> Peng.
>>>
>>>> Hi!
>>>>
>>>> with a lot of help from Stefano, we're getting RPi4 support in Project EVE
>>>> pretty much on par between KVM and Xen.
>>>>
>>>> One big area that still remains is supporting UEFI boot sequence for DomUs.
>>>> With KVM, given the qemu virt device model this is as simple as using 
>>>> either
>>>> stock UEFI build for arm or even U-Boot EFI emulation environment and
>>>> passing it via -bios option.
>>>>
>>>> Obviously with Xen on ARM we don't have the device model so my
>>>> understanding is that the easiest way we can support it would be to port
>>>> UEFI's OvmfPkg/OvmfXen target to ARM (it seems to be currently exclusively
>>>> X64).
>>>>
>>>> So here's my first question: if there's anybody on this list who had a 
>>>> hand in
>>>> implementing OvmfPkg/OvmfXen can you please share your thoughts on how
>>>> much work that port may be (or whether it is even feasible -- I may 
>>>> totally be
>>>> missing something really obvious here).
>>>>
>>>> And as long as I've got your attention: two more questions:
>>>>  1.. compared to the above, would porting pvgrub to ARM be any
>>>>  easier or more difficult?
>>>>
>>>>  2. same question for teaching u-boot about PV calls.
>>>>
>>>> Thanks,
>>>> Roman.
>>>>
>>>> P.S. Oh and I guess between:
>>>>  0. OvmfPkg/OvmfXen on ARM64
>>>>  1. pvgrub on ARM64
>>>>  2. u-boot/EFI emulation with PV calls backend I didn't miss any other
>>>> obvious way of making UEFI-aware VM images to boot on Xen ARM64 DomU,
>>>> right?
[1] https://github.com/xen-troops/u-boot/pull/1

Re: HYPERVISOR_console_io + CONSOLEIO_write needs COPY_flush_dcache

[Oleksandr Andrushchenko]

On 6/10/20 3:50 PM, Julien Grall wrote:
> On 10/06/2020 09:13, Oleksandr Andrushchenko wrote:
>> Hello,
>
> Hi,
>
>> While adding support for para-virtualized block device in u-boot
>>
>> I faced an issue with HYPERVISOR_console_io + CONSOLEIO_write.
>>
>> I tried to use the hypercall during MMU setup stage while enabling data cache
>>
>> on arm64 platform (e.g. data cache is not yet enabled) and saw in guest's 
>> console
>>
>> either old data or some sort of garbage. Printing constant strings, just 
>> like mini-os
>>
>> does on boot, works as expected. 
>
> Per the hypercall ABI (see include/public/arch-arm.h) all the buffers must 
> reside in memory which is mapped as Normal Inner Write-Back Inner-Shareable.
>
> You are passing non-cacheable memory, so the behavior you see is expected.
This is what we come up with as well
>
>> So, looking at the Xen code [1] it seems
>>
>> that we should copy guest's buffer with COPY_flush_dcache set in this case
>
> COPY_flush_dcache is only meant to be used when copy to guest memory to make 
> sure the data reached the point of coherency in your system. It is not meant 
> to be used when copying from guest.
Understood and agree
>
>> as (usually?) this hypercall is used in guest's code which doesn't have any
>> other means to print yet, e.g. early printk case.
>
> I have been using it after the MMU is setup 
The thing is that in u-boot a lot of things happen before the MMU setup...
> but before the console is properly setup by the guest (there is a lot that 
> can happen in Linux between the two). In my case, the flush is not necessary 
> and would be wrong.
... and without the flush you lose the ability to debug that code.
>
> In general, the cache is never off on Arm64 because you may have system cache 
> or, in the cache of virtualization, cacheable mapping in the hypervisor (all 
> the memory is mapped).
>
> When updating data with MMU/D-cache off the normal approach is:
>    1) Remove any dirty lines from the cache, otherwise they may get evicted 
> while updating the memory and override any value written with MMU off.
>    2) Update the memory
>    3) Remove any lines that may have been speculated so when you turn onthe 
> MMU and D-cache, U-boot can obverse the correct data.
>
> Step 1 is only necessary if you are going to write outside of the loaded 
> Image (BSS is not part of it).
>
> You most likely already have similar steps in U-boot for other part of the 
> memory access with MMU/D-Cache off. So I think U-boot is the best place to 
> invalidate the cache before issuing the hypercall.

Yes, I have invalidated the buffers and everything works like a charm now, 
thanks

Probably my use-case should be somewhere documented, so another time someone 
steps into similar issues it is explained.

>
> Cheers,
>
Thank you for helping,

Oleksandr

HYPERVISOR_console_io + CONSOLEIO_write needs COPY_flush_dcache

[Oleksandr Andrushchenko]

Hello,

While adding support for para-virtualized block device in u-boot

I faced an issue with HYPERVISOR_console_io + CONSOLEIO_write.

I tried to use the hypercall during MMU setup stage while enabling data cache

on arm64 platform (e.g. data cache is not yet enabled) and saw in guest's 
console

either old data or some sort of garbage. Printing constant strings, just like 
mini-os

does on boot, works as expected. So, looking at the Xen code [1] it seems

that we should copy guest's buffer with COPY_flush_dcache set in this case

as (usually?) this hypercall is used in guest's code which doesn't have any

other means to print yet, e.g. early printk case.

If my understanding of the issue correct then I can probably prepare a patch.

Thank you,

Oleksandr

[1] 
https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/drivers/char/console.c;h=56e24821b2f8d2b9bfb99d2acb721b06277834ce;hb=refs/heads/master#l597

Re: UEFI support in ARM DomUs

[Oleksandr Andrushchenko]

On 6/4/20 4:57 AM, Peng Fan wrote:
> Grall ;
>> Nataliya Korovkina 
>> Subject: UEFI support in ARM DomUs
> We have made U-Boot run inside XEN DomU, but just only PV console part,
> not implement other frontend drivers currently. Would this help for your
> case if enable EFI in U-Boot?

Well, we have a working PV block implementation on top of that on iMX8

platform, mostly ported from mini-os. Currently we are finalizing the work

and cleaning up (it's going to take a week or so hopefully). Then, we we'll post

it on our public github. We are also thinking about upstreaming the work, but 
it may

take quite some time if the whole idea fits u-boot's view on such an extension 
at all.

Regards,

Oleksandr

> Regards,
> Peng.
>
>> Hi!
>>
>> with a lot of help from Stefano, we're getting RPi4 support in Project EVE
>> pretty much on par between KVM and Xen.
>>
>> One big area that still remains is supporting UEFI boot sequence for DomUs.
>> With KVM, given the qemu virt device model this is as simple as using either
>> stock UEFI build for arm or even U-Boot EFI emulation environment and
>> passing it via -bios option.
>>
>> Obviously with Xen on ARM we don't have the device model so my
>> understanding is that the easiest way we can support it would be to port
>> UEFI's OvmfPkg/OvmfXen target to ARM (it seems to be currently exclusively
>> X64).
>>
>> So here's my first question: if there's anybody on this list who had a hand 
>> in
>> implementing OvmfPkg/OvmfXen can you please share your thoughts on how
>> much work that port may be (or whether it is even feasible -- I may totally 
>> be
>> missing something really obvious here).
>>
>> And as long as I've got your attention: two more questions:
>> 1.. compared to the above, would porting pvgrub to ARM be any
>> easier or more difficult?
>>
>> 2. same question for teaching u-boot about PV calls.
>>
>> Thanks,
>> Roman.
>>
>> P.S. Oh and I guess between:
>> 0. OvmfPkg/OvmfXen on ARM64
>> 1. pvgrub on ARM64
>> 2. u-boot/EFI emulation with PV calls backend I didn't miss any other
>> obvious way of making UEFI-aware VM images to boot on Xen ARM64 DomU,
>> right?

Re: [PATCH 0/2] displif: Protocol version 2

[Oleksandr Andrushchenko]

ping

On 5/20/20 12:04 PM, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko 
>
> Hello all,
>
> this series extends the existing displif protocol with new
> request and adds additional parameter to the exiting one.
> It also provides support for the new parameter in libgnttab
> via ioctl. The relevant changes in the backend can be found at [1]
> and the frontend at [2].
>
> List of changes:
>
> 1. Change protocol version from string to integer
>
> Version string, which is in fact an integer, is hard to handle in the
> code that supports different protocol versions. To simplify that
> make the version an integer.
>
> 2. Pass buffer offset with XENDISPL_OP_DBUF_CREATE
>
> There are cases when display data buffer is created with non-zero
> offset to the data start. Handle such cases and provide that offset
> while creating a display buffer. Add the corresponding filed to the
> protocol and handle it in libgnttab.
> This change is required for bringing virtual display on iMX8
> platform which uses offset of 64 bytes for the buffers allocated
> on GPU side and then imported into PV DRM frontend. Otherwise the
> final picture looks shifted.
>
> 3. Add XENDISPL_OP_GET_EDID command
>
> Add an optional request for reading Extended Display Identification
> Data (EDID) structure which allows better configuration of the
> display connectors over the configuration set in XenStore.
> With this change connectors may have multiple resolutions defined
> with respect to detailed timing definitions and additional properties
> normally provided by displays.
>
> If this request is not supported by the backend then visible area
> is defined by the relevant XenStore's "resolution" property.
>
> If backend provides extended display identification data (EDID) with
> XENDISPL_OP_GET_EDID request then EDID values must take precedence
> over the resolutions defined in XenStore.
>
> 4. Bump protocol version to 2.
>
> Open questions and notes on the changes:
>
> 1. gnttab minor version change from 2 to 3
> As per my understanding it is required to bump the version when
> I add the new functionality, but I would like to hear from the
> maintainers on that.
>
> 2. gnttab and version 2 of the ioctls
> Because we add an additional parameter (data offset) and the structures
> used to pass ioctl arguments cannot be extended (there are no enough
> reserved fields), so there are to ways to solve that:
> - break the existing API and add data_ofs field into the existing
> structures
> - create a copy of the ioctl (v2) with the additional parameter.
>
> It seems to be easier to go the first way, but this breaks things,
> so I decided to introduce v2 of the same ioctl which behaves gracefully
> with respect to the existing users, but adds some amount of
> copy-paste code (I was trying to minimize copy-paste so it is easier
> to maintain, but the result looked ugly to me because of lots of
> "if (protocol v1)" constructs.
>
> Please note that struct ioctl_gntdev_dmabuf_imp_to_refs, e.g.
> version 1 of the ioctl, has uint32_t reserved field which can be
> used for the data offset, but its counterpart 
> (ioctl_gntdev_dmabuf_exp_from_refs)
> doesn't have any, so it seems not to be aligned to only introduce
> version 2 of the ioctl for the later.
>
> The other question is if to keep that reserved field in
> ioctl_gntdev_dmabuf_imp_to_refs_v2 structure or drop it.
>
> 3. displif protocol version string to int conversion
> The existing protocol defines its version as a string "1"
> which is ok, but makes it not so easy to be directly used by
> C/C++ preprocessor which would like to see an integer for constructs
> like "#if XENDISPL_PROTOCOL_VERSION > 2"
>
> At the same time this change may break the existing users of the protocol
> which still expect version as a string. I tried something like
>
> #define STR_HELPER(x) #x
> #define STR(x) STR_HELPER(x)
>
> #define XENDISPL_PROTOCOL_VERSION_INT 1
> #define XENDISPL_PROTOCOL_VERSION STR(XENDISPL_PROTOCOL_VERSION_INT)
>
> but not sure if this is the right and good way to solve the issue,
> so in this series I have deliberately changed the protocol version to
> int.
> Other possible way could be that every user of the header has its local
> copy (this is what we now use in the display backend). This way the
> local copy can be changed in a way suitable for the concrete user.
> This cannot be done (?) for the Linux Kernel though.
>
> Thank you,
> Oleksandr
>
> [1] https://github.com/xen-troops/displ_be
> [2] https://github.com/xen-troops/linux/pull/87
>
> Oleksandr Andrushchenko (2):
>xen/displif: Protocol version 2
&g

Re: [PATCH v4] public/io/netif.h: add a new extra type for XDP

[Oleksandr Andrushchenko]

On 5/22/20 12:33 PM, Denis Kirjanov wrote:
> On 5/22/20, Oleksandr Andrushchenko  wrote:
>> On 5/22/20 12:17 PM, Denis Kirjanov wrote:
>>> On 5/22/20, Oleksandr Andrushchenko 
>>> wrote:
>>>> On 5/18/20 6:04 PM, Denis Kirjanov wrote:
>>>>> The patch adds a new extra type to be able to diffirentiate
>>>>> between RX responses on xen-netfront side with the adjusted offset
>>>>> required for XDP processing.
>>>>>
>>>>> The offset value from a guest is passed via xenstore.
>>>>>
>>>>> Signed-off-by: Denis Kirjanov 
>>>>> ---
>>>>> v4:
>>>>> - updated the commit and documenation
>>>>>
>>>>> v3:
>>>>> - updated the commit message
>>>>>
>>>>> v2:
>>>>> - added documentation
>>>>> - fixed padding for netif_extra_info
>>>>> ---
>>>>> ---
>>>>> xen/include/public/io/netif.h | 18 +-
>>>>> 1 file changed, 17 insertions(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/xen/include/public/io/netif.h
>>>>> b/xen/include/public/io/netif.h
>>>>> index 9fcf91a..a92bf04 100644
>>>>> --- a/xen/include/public/io/netif.h
>>>>> +++ b/xen/include/public/io/netif.h
>>>>> @@ -161,6 +161,17 @@
>>>>>  */
>>>>>
>>>>> /*
>>>>> + * "xdp-headroom" is used to request that extra space is added
>>>>> + * for XDP processing.  The value is measured in bytes and passed by
>>>> not sure that we should use word "bytes" here as the rest of the
>>>> protocol (mostly)
>>>>
>>>> talks about octets. It is somewhat mixed here, no strong opinion
>>> sure, but since the public header mixes it I've decided to use that word.
>>>
>>>
>>>>> + * the frontend to be consistent between both ends.
>>>>> + * If the value is greater than zero that means that
>>>>> + * an RX response is going to be passed to an XDP program for
>>>>> processing.
>>>>> + *
>>>>> + * "feature-xdp-headroom" is set to "1" by the netback side like other
>>>>> features
>>>>> + * so a guest can check if an XDP program can be processed.
>>>>> + */
>>>>> +
>>>>> +/*
>>>>>  * Control ring
>>>>>  * 
>>>>>  *
>>>>> @@ -985,7 +996,8 @@ typedef struct netif_tx_request netif_tx_request_t;
>>>>> #define XEN_NETIF_EXTRA_TYPE_MCAST_ADD (2)  /* u.mcast */
>>>>> #define XEN_NETIF_EXTRA_TYPE_MCAST_DEL (3)  /* u.mcast */
>>>>> #define XEN_NETIF_EXTRA_TYPE_HASH  (4)  /* u.hash */
>>>>> -#define XEN_NETIF_EXTRA_TYPE_MAX   (5)
>>>>> +#define XEN_NETIF_EXTRA_TYPE_XDP   (5)  /* u.xdp */
>>>>> +#define XEN_NETIF_EXTRA_TYPE_MAX   (6)
>>>>>
>>>>> /* netif_extra_info_t flags. */
>>>>> #define _XEN_NETIF_EXTRA_FLAG_MORE (0)
>>>>> @@ -1018,6 +1030,10 @@ struct netif_extra_info {
>>>>> uint8_t algorithm;
>>>>> uint8_t value[4];
>>>>> } hash;
>>>>> +struct {
>>>>> +uint16_t headroom;
>>>> why do you need "pad" field here?
>>> To state that we have a fixed size available.
>> Well, I would expect "reserved" or something in that case and "pad" in case
>>
>> there are other fields following (see gso above).
> it can be consistent with other names like pad at then end of the structure.
>
> If it really matters I can change it, no problem.
My point is that IMO it is not required at all, but this is up to 
maintainers to decide
>
>> But here I think "pad" is not required, just like mcast doesn't add any
> because it's already 6-bytes long
you are right
>
>>>>> +uint16_t pad[2]
>>>>> +} xdp;
>>>>> uint16_t pad[3];
>>>>> } u;
>>>>> };

Re: [PATCH v4] public/io/netif.h: add a new extra type for XDP

[Oleksandr Andrushchenko]

On 5/22/20 12:17 PM, Denis Kirjanov wrote:
> On 5/22/20, Oleksandr Andrushchenko  wrote:
>> On 5/18/20 6:04 PM, Denis Kirjanov wrote:
>>> The patch adds a new extra type to be able to diffirentiate
>>> between RX responses on xen-netfront side with the adjusted offset
>>> required for XDP processing.
>>>
>>> The offset value from a guest is passed via xenstore.
>>>
>>> Signed-off-by: Denis Kirjanov 
>>> ---
>>> v4:
>>> - updated the commit and documenation
>>>
>>> v3:
>>> - updated the commit message
>>>
>>> v2:
>>> - added documentation
>>> - fixed padding for netif_extra_info
>>> ---
>>> ---
>>>xen/include/public/io/netif.h | 18 +-
>>>1 file changed, 17 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/xen/include/public/io/netif.h
>>> b/xen/include/public/io/netif.h
>>> index 9fcf91a..a92bf04 100644
>>> --- a/xen/include/public/io/netif.h
>>> +++ b/xen/include/public/io/netif.h
>>> @@ -161,6 +161,17 @@
>>> */
>>>
>>>/*
>>> + * "xdp-headroom" is used to request that extra space is added
>>> + * for XDP processing.  The value is measured in bytes and passed by
>> not sure that we should use word "bytes" here as the rest of the
>> protocol (mostly)
>>
>> talks about octets. It is somewhat mixed here, no strong opinion
> sure, but since the public header mixes it I've decided to use that word.
>
>
>>> + * the frontend to be consistent between both ends.
>>> + * If the value is greater than zero that means that
>>> + * an RX response is going to be passed to an XDP program for
>>> processing.
>>> + *
>>> + * "feature-xdp-headroom" is set to "1" by the netback side like other
>>> features
>>> + * so a guest can check if an XDP program can be processed.
>>> + */
>>> +
>>> +/*
>>> * Control ring
>>> * 
>>> *
>>> @@ -985,7 +996,8 @@ typedef struct netif_tx_request netif_tx_request_t;
>>>#define XEN_NETIF_EXTRA_TYPE_MCAST_ADD (2)  /* u.mcast */
>>>#define XEN_NETIF_EXTRA_TYPE_MCAST_DEL (3)  /* u.mcast */
>>>#define XEN_NETIF_EXTRA_TYPE_HASH  (4)  /* u.hash */
>>> -#define XEN_NETIF_EXTRA_TYPE_MAX   (5)
>>> +#define XEN_NETIF_EXTRA_TYPE_XDP   (5)  /* u.xdp */
>>> +#define XEN_NETIF_EXTRA_TYPE_MAX   (6)
>>>
>>>/* netif_extra_info_t flags. */
>>>#define _XEN_NETIF_EXTRA_FLAG_MORE (0)
>>> @@ -1018,6 +1030,10 @@ struct netif_extra_info {
>>>uint8_t algorithm;
>>>uint8_t value[4];
>>>} hash;
>>> +struct {
>>> +uint16_t headroom;
>> why do you need "pad" field here?
> To state that we have a fixed size available.

Well, I would expect "reserved" or something in that case and "pad" in case

there are other fields following (see gso above).

But here I think "pad" is not required, just like mcast doesn't add any

>
>>> +uint16_t pad[2]
>>> +} xdp;
>>>uint16_t pad[3];
>>>} u;
>>>};

Re: [PATCH v4] public/io/netif.h: add a new extra type for XDP

[Oleksandr Andrushchenko]

On 5/18/20 6:04 PM, Denis Kirjanov wrote:
> The patch adds a new extra type to be able to diffirentiate
> between RX responses on xen-netfront side with the adjusted offset
> required for XDP processing.
>
> The offset value from a guest is passed via xenstore.
>
> Signed-off-by: Denis Kirjanov 
> ---
> v4:
> - updated the commit and documenation
>
> v3:
> - updated the commit message
>
> v2:
> - added documentation
> - fixed padding for netif_extra_info
> ---
> ---
>   xen/include/public/io/netif.h | 18 +-
>   1 file changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/xen/include/public/io/netif.h b/xen/include/public/io/netif.h
> index 9fcf91a..a92bf04 100644
> --- a/xen/include/public/io/netif.h
> +++ b/xen/include/public/io/netif.h
> @@ -161,6 +161,17 @@
>*/
>   
>   /*
> + * "xdp-headroom" is used to request that extra space is added
> + * for XDP processing.  The value is measured in bytes and passed by

not sure that we should use word "bytes" here as the rest of the 
protocol (mostly)

talks about octets. It is somewhat mixed here, no strong opinion

> + * the frontend to be consistent between both ends.
> + * If the value is greater than zero that means that
> + * an RX response is going to be passed to an XDP program for processing.
> + *
> + * "feature-xdp-headroom" is set to "1" by the netback side like other 
> features
> + * so a guest can check if an XDP program can be processed.
> + */
> +
> +/*
>* Control ring
>* 
>*
> @@ -985,7 +996,8 @@ typedef struct netif_tx_request netif_tx_request_t;
>   #define XEN_NETIF_EXTRA_TYPE_MCAST_ADD (2)  /* u.mcast */
>   #define XEN_NETIF_EXTRA_TYPE_MCAST_DEL (3)  /* u.mcast */
>   #define XEN_NETIF_EXTRA_TYPE_HASH  (4)  /* u.hash */
> -#define XEN_NETIF_EXTRA_TYPE_MAX   (5)
> +#define XEN_NETIF_EXTRA_TYPE_XDP   (5)  /* u.xdp */
> +#define XEN_NETIF_EXTRA_TYPE_MAX   (6)
>   
>   /* netif_extra_info_t flags. */
>   #define _XEN_NETIF_EXTRA_FLAG_MORE (0)
> @@ -1018,6 +1030,10 @@ struct netif_extra_info {
>   uint8_t algorithm;
>   uint8_t value[4];
>   } hash;
> +struct {
> +uint16_t headroom;
why do you need "pad" field here?
> +uint16_t pad[2]
> +} xdp;
>   uint16_t pad[3];
>   } u;
>   };

[PATCH 0/2] displif: Protocol version 2

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Hello all,

this series extends the existing displif protocol with new
request and adds additional parameter to the exiting one.
It also provides support for the new parameter in libgnttab
via ioctl. The relevant changes in the backend can be found at [1]
and the frontend at [2].

List of changes:

1. Change protocol version from string to integer

Version string, which is in fact an integer, is hard to handle in the
code that supports different protocol versions. To simplify that
make the version an integer.

2. Pass buffer offset with XENDISPL_OP_DBUF_CREATE

There are cases when display data buffer is created with non-zero
offset to the data start. Handle such cases and provide that offset
while creating a display buffer. Add the corresponding filed to the
protocol and handle it in libgnttab.
This change is required for bringing virtual display on iMX8
platform which uses offset of 64 bytes for the buffers allocated
on GPU side and then imported into PV DRM frontend. Otherwise the
final picture looks shifted.

3. Add XENDISPL_OP_GET_EDID command

Add an optional request for reading Extended Display Identification
Data (EDID) structure which allows better configuration of the
display connectors over the configuration set in XenStore.
With this change connectors may have multiple resolutions defined
with respect to detailed timing definitions and additional properties
normally provided by displays.

If this request is not supported by the backend then visible area
is defined by the relevant XenStore's "resolution" property.

If backend provides extended display identification data (EDID) with
XENDISPL_OP_GET_EDID request then EDID values must take precedence
over the resolutions defined in XenStore.

4. Bump protocol version to 2.

Open questions and notes on the changes:

1. gnttab minor version change from 2 to 3
As per my understanding it is required to bump the version when
I add the new functionality, but I would like to hear from the
maintainers on that.

2. gnttab and version 2 of the ioctls
Because we add an additional parameter (data offset) and the structures
used to pass ioctl arguments cannot be extended (there are no enough
reserved fields), so there are to ways to solve that:
- break the existing API and add data_ofs field into the existing
structures
- create a copy of the ioctl (v2) with the additional parameter.

It seems to be easier to go the first way, but this breaks things,
so I decided to introduce v2 of the same ioctl which behaves gracefully
with respect to the existing users, but adds some amount of
copy-paste code (I was trying to minimize copy-paste so it is easier
to maintain, but the result looked ugly to me because of lots of
"if (protocol v1)" constructs.

Please note that struct ioctl_gntdev_dmabuf_imp_to_refs, e.g.
version 1 of the ioctl, has uint32_t reserved field which can be
used for the data offset, but its counterpart 
(ioctl_gntdev_dmabuf_exp_from_refs)
doesn't have any, so it seems not to be aligned to only introduce
version 2 of the ioctl for the later.

The other question is if to keep that reserved field in
ioctl_gntdev_dmabuf_imp_to_refs_v2 structure or drop it.

3. displif protocol version string to int conversion
The existing protocol defines its version as a string "1"
which is ok, but makes it not so easy to be directly used by
C/C++ preprocessor which would like to see an integer for constructs
like "#if XENDISPL_PROTOCOL_VERSION > 2"

At the same time this change may break the existing users of the protocol
which still expect version as a string. I tried something like

#define STR_HELPER(x) #x
#define STR(x) STR_HELPER(x)

#define XENDISPL_PROTOCOL_VERSION_INT 1
#define XENDISPL_PROTOCOL_VERSION STR(XENDISPL_PROTOCOL_VERSION_INT)

but not sure if this is the right and good way to solve the issue,
so in this series I have deliberately changed the protocol version to
int.
Other possible way could be that every user of the header has its local
copy (this is what we now use in the display backend). This way the
local copy can be changed in a way suitable for the concrete user.
This cannot be done (?) for the Linux Kernel though.

Thank you,
Oleksandr

[1] https://github.com/xen-troops/displ_be
[2] https://github.com/xen-troops/linux/pull/87

Oleksandr Andrushchenko (2):
  xen/displif: Protocol version 2
  libgnttab: Add support for Linux dma-buf offset

 tools/include/xen-sys/Linux/gntdev.h  | 53 +
 tools/libs/gnttab/Makefile|  2 +-
 tools/libs/gnttab/freebsd.c   | 15 +
 tools/libs/gnttab/gnttab_core.c   | 17 ++
 tools/libs/gnttab/include/xengnttab.h | 13 
 tools/libs/gnttab/libxengnttab.map|  6 ++
 tools/libs/gnttab/linux.c | 86 +++
 tools/libs/gnttab/minios.c| 15 +
 tools/libs/gnttab/private.h   |  9 +++
 xen/include/public/io/displif.h   | 83 +

[PATCH 1/2] xen/displif: Protocol version 2

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

1. Change protocol version from string to integer

Version string, which is in fact an integer, is hard to handle in the
code that supports different protocol versions. To simplify that
make the version an integer.

2. Pass buffer offset with XENDISPL_OP_DBUF_CREATE

There are cases when display data buffer is created with non-zero
offset to the data start. Handle such cases and provide that offset
while creating a display buffer.

3. Add XENDISPL_OP_GET_EDID command

Add an optional request for reading Extended Display Identification
Data (EDID) structure which allows better configuration of the
display connectors over the configuration set in XenStore.
With this change connectors may have multiple resolutions defined
with respect to detailed timing definitions and additional properties
normally provided by displays.

If this request is not supported by the backend then visible area
is defined by the relevant XenStore's "resolution" property.

If backend provides extended display identification data (EDID) with
XENDISPL_OP_GET_EDID request then EDID values must take precedence
over the resolutions defined in XenStore.

4. Bump protocol version to 2.

Signed-off-by: Oleksandr Andrushchenko 
---
 xen/include/public/io/displif.h | 83 +++--
 1 file changed, 80 insertions(+), 3 deletions(-)

diff --git a/xen/include/public/io/displif.h b/xen/include/public/io/displif.h
index cc5de9cb1f35..4d43ba5078c8 100644
--- a/xen/include/public/io/displif.h
+++ b/xen/include/public/io/displif.h
@@ -38,7 +38,7 @@
  *   Protocol version
  **
  */
-#define XENDISPL_PROTOCOL_VERSION "1"
+#define XENDISPL_PROTOCOL_VERSION 2
 
 /*
  **
@@ -202,6 +202,9 @@
  *  Width and height of the connector in pixels separated by
  *  XENDISPL_RESOLUTION_SEPARATOR. This defines visible area of the
  *  display.
+ *  If backend provides extended display identification data (EDID) with
+ *  XENDISPL_OP_GET_EDID request then EDID values must take precedence
+ *  over the resolutions defined here.
  *
  *-- Connector Request Transport Parameters ---
  *
@@ -349,6 +352,7 @@
 #define XENDISPL_OP_FB_DETACH 0x13
 #define XENDISPL_OP_SET_CONFIG0x14
 #define XENDISPL_OP_PG_FLIP   0x15
+#define XENDISPL_OP_GET_EDID  0x16
 
 /*
  **
@@ -377,6 +381,10 @@
 #define XENDISPL_FIELD_BE_ALLOC   "be-alloc"
 #define XENDISPL_FIELD_UNIQUE_ID  "unique-id"
 
+#define XENDISPL_EDID_BLOCK_SIZE  128
+#define XENDISPL_EDID_BLOCK_COUNT 256
+#define XENDISPL_EDID_MAX_SIZE(XENDISPL_EDID_BLOCK_SIZE * 
XENDISPL_EDID_BLOCK_COUNT)
+
 /*
  **
  *  STATUS RETURN CODES
@@ -451,7 +459,9 @@
  * +++++
  * |   gref_directory  | 40
  * +++++
- * | reserved  | 44
+ * | data_ofs  | 44
+ * +++++
+ * | reserved  | 48
  * +++++
  * |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/|
  * +++++
@@ -494,6 +504,7 @@
  *   buffer size (buffer_sz) exceeds what can be addressed by this single page,
  *   then reference to the next page must be supplied (see gref_dir_next_page
  *   below)
+ * data_ofs - uint32_t, offset of the data in the buffer, octets
  */
 
 #define XENDISPL_DBUF_FLG_REQ_ALLOC   (1 << 0)
@@ -506,6 +517,7 @@ struct xendispl_dbuf_create_req {
 uint32_t buffer_sz;
 uint32_t flags;
 grant_ref_t gref_directory;
+uint32_t data_ofs;
 };
 
 /*
@@ -731,6 +743,42 @@ struct xendispl_page_flip_req {
 uint64_t fb_cookie;
 };
 
+/*
+ * Request EDID - request EDID describing current connector:
+ * 01 2   3octet
+ * +++++
+ * |   id| _OP_GET_EDID   |   reserved | 4
+ * +++++
+ * | buffer_sz | 8
+ * +++++

[PATCH 2/2] libgnttab: Add support for Linux dma-buf offset

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Add version 2 of the dma-buf ioctls which adds data_ofs parameter.

dma-buf is backed by a scatter-gather table and has offset parameter
which tells where the actual data starts. Relevant ioctls are extended
to support that offset:
  - when dma-buf is created (exported) from grant references then
data_ofs is used to set the offset field in the scatter list
of the new dma-buf
  - when dma-buf is imported and grant references provided then
data_ofs is used to report that offset to user-space

Signed-off-by: Oleksandr Andrushchenko 
---
 tools/include/xen-sys/Linux/gntdev.h  | 53 +
 tools/libs/gnttab/Makefile|  2 +-
 tools/libs/gnttab/freebsd.c   | 15 +
 tools/libs/gnttab/gnttab_core.c   | 17 ++
 tools/libs/gnttab/include/xengnttab.h | 13 
 tools/libs/gnttab/libxengnttab.map|  6 ++
 tools/libs/gnttab/linux.c | 86 +++
 tools/libs/gnttab/minios.c| 15 +
 tools/libs/gnttab/private.h   |  9 +++
 9 files changed, 215 insertions(+), 1 deletion(-)

diff --git a/tools/include/xen-sys/Linux/gntdev.h 
b/tools/include/xen-sys/Linux/gntdev.h
index d16076044c71..0c43393cbee5 100644
--- a/tools/include/xen-sys/Linux/gntdev.h
+++ b/tools/include/xen-sys/Linux/gntdev.h
@@ -274,4 +274,57 @@ struct ioctl_gntdev_dmabuf_imp_release {
 uint32_t reserved;
 };
 
+/*
+ * Version 2 of the ioctls adds @data_ofs parameter.
+ *
+ * dma-buf is backed by a scatter-gather table and has offset
+ * parameter which tells where the actual data starts.
+ * Relevant ioctls are extended to support that offset:
+ *   - when dma-buf is created (exported) from grant references then
+ * @data_ofs is used to set the offset field in the scatter list
+ * of the new dma-buf
+ *   - when dma-buf is imported and grant references are provided then
+ * @data_ofs is used to report that offset to user-space
+ */
+#define IOCTL_GNTDEV_DMABUF_EXP_FROM_REFS_V2 \
+_IOC(_IOC_NONE, 'G', 13, \
+ sizeof(struct ioctl_gntdev_dmabuf_exp_from_refs_v2))
+struct ioctl_gntdev_dmabuf_exp_from_refs_v2 {
+/* IN parameters. */
+/* Specific options for this dma-buf: see GNTDEV_DMA_FLAG_XXX. */
+uint32_t flags;
+/* Number of grant references in @refs array. */
+uint32_t count;
+/* Offset of the data in the dma-buf. */
+uint32_t data_ofs;
+/* OUT parameters. */
+/* File descriptor of the dma-buf. */
+uint32_t fd;
+/* The domain ID of the grant references to be mapped. */
+uint32_t domid;
+/* Variable IN parameter. */
+/* Array of grant references of size @count. */
+uint32_t refs[1];
+};
+
+#define IOCTL_GNTDEV_DMABUF_IMP_TO_REFS_V2 \
+_IOC(_IOC_NONE, 'G', 14, \
+ sizeof(struct ioctl_gntdev_dmabuf_imp_to_refs_v2))
+struct ioctl_gntdev_dmabuf_imp_to_refs_v2 {
+/* IN parameters. */
+/* File descriptor of the dma-buf. */
+uint32_t fd;
+/* Number of grant references in @refs array. */
+uint32_t count;
+/* The domain ID for which references to be granted. */
+uint32_t domid;
+/* Reserved - must be zero. */
+uint32_t reserved;
+/* OUT parameters. */
+/* Offset of the data in the dma-buf. */
+uint32_t data_ofs;
+/* Array of grant references of size @count. */
+uint32_t refs[1];
+};
+
 #endif /* __LINUX_PUBLIC_GNTDEV_H__ */
diff --git a/tools/libs/gnttab/Makefile b/tools/libs/gnttab/Makefile
index 2da8fbbb7f6f..5ee2d965214f 100644
--- a/tools/libs/gnttab/Makefile
+++ b/tools/libs/gnttab/Makefile
@@ -2,7 +2,7 @@ XEN_ROOT = $(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR= 1
-MINOR= 2
+MINOR= 3
 LIBNAME  := gnttab
 USELIBS  := toollog toolcore
 
diff --git a/tools/libs/gnttab/freebsd.c b/tools/libs/gnttab/freebsd.c
index 886b588303a0..baf0f60aa4d3 100644
--- a/tools/libs/gnttab/freebsd.c
+++ b/tools/libs/gnttab/freebsd.c
@@ -319,6 +319,14 @@ int osdep_gnttab_dmabuf_exp_from_refs(xengnttab_handle 
*xgt, uint32_t domid,
 abort();
 }
 
+int osdep_gnttab_dmabuf_exp_from_refs_v2(xengnttab_handle *xgt, uint32_t domid,
+ uint32_t flags, uint32_t count,
+ const uint32_t *refs,
+ uint32_t *dmabuf_fd, uint32_t 
data_ofs)
+{
+abort();
+}
+
 int osdep_gnttab_dmabuf_exp_wait_released(xengnttab_handle *xgt,
   uint32_t fd, uint32_t wait_to_ms)
 {
@@ -331,6 +339,13 @@ int osdep_gnttab_dmabuf_imp_to_refs(xengnttab_handle *xgt, 
uint32_t domid,
 abort();
 }
 
+int osdep_gnttab_dmabuf_imp_to_refs_v2(xengnttab_handle *xgt, uint32_t domid,
+   uint32_t fd, uint32_t count,
+   uint32_t *refs, uint32_t *data_ofs)
+{
+abort();
+}
+
 int osdep_gnttab_dmabuf_imp_release(xengnttab_handle *xgt, uint32_t fd)
 {
 abort();
diff --git

Re: [bug report] drm/xen-front: Add support for Xen PV display frontend

[Oleksandr Andrushchenko]

On 4/21/20 2:51 PM, Dan Carpenter wrote:
> It turns out there aren't that many of these in xen.
>
> $ grep IS_ERR_OR_NULL drivers/gpu/drm/xen/ -Rn
> drivers/gpu/drm/xen/xen_drm_front_kms.c:63: if (IS_ERR_OR_NULL(fb))
> drivers/gpu/drm/xen/xen_drm_front_gem.c:86: if (IS_ERR_OR_NULL(xen_obj))
> drivers/gpu/drm/xen/xen_drm_front_gem.c:120:if 
> (IS_ERR_OR_NULL(xen_obj->pages)) {
> drivers/gpu/drm/xen/xen_drm_front_gem.c:139:if (IS_ERR_OR_NULL(xen_obj))
> drivers/gpu/drm/xen/xen_drm_front_gem.c:197:if (IS_ERR_OR_NULL(xen_obj))
> drivers/gpu/drm/xen/xen_drm_front.c:403:if (IS_ERR_OR_NULL(obj)) {
>
> They're all wrong, because if the pointer was really NULL then it's
> not handled correctly and would eventually lead to a runtime failure.

It seems that you were right and I can simply change all IS_ERR_OR_NULL 
to just IS_ERR

I am planning a series of patches later on, so I'll include this fix as well

>
> Normally Smatch is smart enough to know that the pointer isn't NULL so
> it doesn't generate a warning but yesterday I introduced a bug in Smatch
> by mistake.  It's fixed now.
>
> regards,
> dan carpenter
>
Thank you,

Oleksandr

Re: [bug report] drm/xen-front: Add support for Xen PV display frontend

[Oleksandr Andrushchenko]

On 4/21/20 14:51, Dan Carpenter wrote:
> It turns out there aren't that many of these in xen.
>
> $ grep IS_ERR_OR_NULL drivers/gpu/drm/xen/ -Rn
> drivers/gpu/drm/xen/xen_drm_front_kms.c:63: if (IS_ERR_OR_NULL(fb))
> drivers/gpu/drm/xen/xen_drm_front_gem.c:86: if (IS_ERR_OR_NULL(xen_obj))
> drivers/gpu/drm/xen/xen_drm_front_gem.c:120:if 
> (IS_ERR_OR_NULL(xen_obj->pages)) {
> drivers/gpu/drm/xen/xen_drm_front_gem.c:139:if (IS_ERR_OR_NULL(xen_obj))
> drivers/gpu/drm/xen/xen_drm_front_gem.c:197:if (IS_ERR_OR_NULL(xen_obj))
> drivers/gpu/drm/xen/xen_drm_front.c:403:if (IS_ERR_OR_NULL(obj)) {
>
> They're all wrong, because if the pointer was really NULL then it's
> not handled correctly and would eventually lead to a runtime failure.
>
> Normally Smatch is smart enough to know that the pointer isn't NULL so
> it doesn't generate a warning but yesterday I introduced a bug in Smatch
> by mistake.  It's fixed now.
>
> regards,
> dan carpenter
>
Thank you,

I'll have a look at those

Re: [bug report] drm/xen-front: Add support for Xen PV display frontend

[Oleksandr Andrushchenko]

On 4/21/20 13:45, Dan Carpenter wrote:
> Hi Kernel Janitors,
Hi
>
> Here is another idea that someone could work on, fixing the
> IS_ERR_OR_NULL() checks in the xen driver.
>
> The patch c575b7eeb89f: "drm/xen-front: Add support for Xen PV
> display frontend" from Apr 3, 2018, leads to the following static
> checker warning:
>
>   drivers/gpu/drm/xen/xen_drm_front_gem.c:140 xen_drm_front_gem_create()
>   warn: passing zero to 'ERR_CAST'
>
> drivers/gpu/drm/xen/xen_drm_front_gem.c
> 133  struct drm_gem_object *xen_drm_front_gem_create(struct drm_device 
> *dev,
> 134  size_t size)
> 135  {
> 136  struct xen_gem_object *xen_obj;
> 137
> 138  xen_obj = gem_create(dev, size);
> 139  if (IS_ERR_OR_NULL(xen_obj))
> 140  return ERR_CAST(xen_obj);
>
> This warning is old and it's actually the result of a bug I had in my
> devel version of Smatch yesterday.  xen_obj can't really be NULL, but
> if it were then the caller would return success which would probably
> create some issues.
>
> When a function returns both error pointers and NULL then NULL is a
> special case of success.  Like say you have:  "p = start_feature();".
> If there is an allocation failure, then the code should return -ENOMEM
> and the whole thing should fail.  But if the feature is optional and
> the user has disabled it in the config then we return NULL and the
> caller has to be able to accept that.  There are a lot of these
> IS_ERR_OR_NULL() checks in the xen driver...
>
> The one here is clearly buggy because returning NULL would lead to a
> run time failure.  All these IS_ERR_OR_NULL() should be checked and
> probably changed to just IS_ERR().
>
> This sort of change is probably change is probably easiest if you build
> the Smatch DB and you can check if Smatch thinks the functions return
> NULL.
>
> ~/smatch/smatch_data/db/smdb.py return_states gem_create | grep INTERNAL
> drivers/gpu/drm/xen/xen_drm_front_gem.c | gem_create | 58 |  (-4095)-(-1) |   
>INTERNAL |  -1 |  | struct xen_gem_object*(*)(struct 
> drm_device*, ulong) |
> drivers/gpu/drm/xen/xen_drm_front_gem.c | gem_create | 62 | 
> 4065035897849303040 |  INTERNAL |  -1 |  | struct 
> xen_gem_object*(*)(struct drm_device*, ulong) |
> drivers/gpu/drm/xen/xen_drm_front_gem.c | gem_create | 66 | 
> 4065035897849303040 |  INTERNAL |  -1 |  | struct 
> xen_gem_object*(*)(struct drm_device*, ulong) |
> drivers/gpu/drm/xen/xen_drm_front_gem.c | gem_create | 68 | 0,(-4095)-(-1) |  
> INTERNAL |  -1 |  | struct xen_gem_object*(*)(struct 
> drm_device*, ulong) |
>
> Smatch thinks that gem_create() sometimes returns NULL or error pointers
> but that's because of a bug in the unreleased version of Smatch and
> because gem_create() uses IS_ERR_OR_NULL().
>
> 141
> 142  return _obj->base;
> 143  }
>
> regards,
> dan carpenter

Thank you for the report, I will try to find some time to look into this 
and come up with fixes.

Could you please (probably off-list) instruct me or give any pointers to 
how to reproduce

the results of the analyzer shown above?

Thank you,

Oleksandr

Re: [Xen-devel] PV DRM doesn't work without auto_translated_physmap feature in Dom0

[Oleksandr Andrushchenko]

Hi,

On 4/19/20 14:26, Santucco wrote:
> Hello,
> I have found a source of the problem.
> In displ_be,  BaseDump copies to the drm buffer with a size from 
> i915 drm driver, but this size a bit more than a size of my frontend 
> display buffer. I have made a quick and dirty fix, a copy a line of my 
> display buffer to a middle of a line of the drm display buffer. Patch 
> is attached.

Thank you for the patch and your efforts to fix the issue.

Could you please make a pull request to [1], so we can continue there?

Thank you in advance,

Oleksandr

> Best regards,
> Alexander
>
> Четверг, 6 февраля 2020, 11:20 +03:00 от Oleksandr Andrushchenko
> :
> On 2/5/20 8:59 PM, Santucco wrote:
> > Hello,
> > Ok, I  commented out the memcpy call and run the test.
> > displ_be hasn’t crached, I have seen FLIP events in the log.
> > But there hasn’t been the black screen, just a blink effect every
> > couple of seconds.
> > Logs are attached.
> Ok, so I believe that frontend - backend (displ_be) communication
> is ok
> and there is nothing to do there.
>
> Next, I would start debugging the following in Xen:
> (XEN) mm.c:2223:d2v0 Bad L1 flags 80
> and have a look at [1]. Probably, someone on Xen x86 side can tell
> if this could be related to the flags at [2].
>
> > Best regards,
> > Alexander
> >
> > Среда, 5 февраля 2020, 9:31 +03:00 от Oleksandr Andrushchenko
> >  >:
> > On 2/4/20 10:28 AM, Santucco wrote:
> > > Hello,
> > > displ_be was compiled without zero-copy support early.
> > > I have tried with the recompiled dom0 kernel, a result is the
> same.
> > > Logs and configs (+displ_be’s CMakeCache.txt ) are attached.
> > Ok, yet another test to localize the problem.
> > Could you please remove memcpy from
> > #1  0x55e5a1f28bec in Drm::DumbDrm::copy
> (this=0x7f9338000e00) at
> >
> /home/santucco/tmp/xen-troops/displ_be/src/displayBackend/drm/Dumb.cpp:149
> > and just memset the destination with 0 or whatever.
> >
> > I expect that system won't crash, nothing will be shown (black
> > screen), but
> > displ_be will show page flip events in its logs.
> > > Best regards,
> > > Alexander
> > >
> > > Понедельник, 3 февраля 2020, 10:36 +03:00 от Oleksandr
> > > Andrushchenko  
> > >:
> > >
> > >
> > > On 2/1/20 4:39 PM, Santucco wrote:
> > > > Hello again,
> > > > I have not yet made to work my drm client, so I have tried
> to run
> > > > linux like a domU (to see how it should work), it doesn’t
> work too
> > > > — displ_be catches SIGSEGV:
> > > >
> > > > #0  0x7f4afed1c161 in ?? () from /lib64/libc.so.6
> > > > #1  0x55723b9c5bec in Drm::DumbDrm::copy
> > > (this=0x7f4adc000e00) at
> > > >
> > >
> >
> /home/santucco/tmp/xen-troops/displ_be/src/displayBackend/drm/Dumb.cpp:149
> > > > #2  0x55723b9a8f51 in BuffersStorage::getFrameBufferAndCopy
> > > > (this=0x7f4ae00010e0, fbCookie=18446612682295083264) at
> > > >
> > >
> >
> 
> /home/santucco/tmp/xen-troops/displ_be/src/displayBackend/BuffersStorage.cpp:165
> > > > It tries to copy to mBuffer with non-accessible address.
> > > > For the moment I see a strange offset for mmap call of
> > > /dev/drm/card0
> > > > in the strace log — 0x1. Is that normal?
> > > > Any direction of which to dig will be very helpful.
> > > > Configuration details:
> > > > Xen 4.12.1 Dom0: Linux 4.20.17-gentoo #13 SMP Sat Dec 28
> > > 11:12:24 MSK
> > > > 2019 x86_64 Intel(R) Celeron(R) CPU N3050 @ 1.60GHz GenuineIntel
> > > GNU/Linux
> > > > DomU: Linux 4.20.17-gentoo
> > > > last xen-troops/libxenbe and xen-troops/displ_be
> > > > Logs (dmesg, xl dmesg, displ_be, strace log of displ_be), a
> > > backtrace
> > > > of gdb and kernel configs are attached.
> > > > Thanks in advance.
> > > Could you please try Dom0 kernel WITHOUT the options below:
> > > CONFIG_XEN_GNTDEV_DMABUF=y
> > > CONFIG_XEN_GRANT_DMA_ALLOC=y
> > >
> > > Then, just to make sure, did you build displ_be without zero-copy
>

Re: [Xen-devel] [PATCH] drm/xen: fix passing zero to 'PTR_ERR' warning

[Oleksandr Andrushchenko]

On 3/30/20 12:59, Ding Xiang wrote:

Fix a static code checker warning:
 drivers/gpu/drm/xen/xen_drm_front.c:404 xen_drm_drv_dumb_create()
 warn: passing zero to 'PTR_ERR'

Signed-off-by: Ding Xiang 

Reviewed-by: Oleksandr Andrushchenko 

---
  drivers/gpu/drm/xen/xen_drm_front.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front.c 
b/drivers/gpu/drm/xen/xen_drm_front.c
index 4be49c1..3741420 100644
--- a/drivers/gpu/drm/xen/xen_drm_front.c
+++ b/drivers/gpu/drm/xen/xen_drm_front.c
@@ -401,7 +401,7 @@ static int xen_drm_drv_dumb_create(struct drm_file *filp,
  
  	obj = xen_drm_front_gem_create(dev, args->size);

if (IS_ERR_OR_NULL(obj)) {
-   ret = PTR_ERR(obj);
+   ret = PTR_ERR_OR_ZERO(obj);
goto fail;
}
  

Re: [Xen-devel] [PATCH 04/52] drm: Set final_kfree in drm_dev_alloc

[Oleksandr Andrushchenko]

On 2/19/20 12:20 PM, Daniel Vetter wrote:
> I also did a full review of all callers, and only the xen driver
> forgot to call drm_dev_put in the failure path. Fix that up too.
>
> v2: I noticed that xen has a drm_driver.release hook, and uses
> drm_dev_alloc(). We need to remove the kfree from
> xen_drm_drv_release().
>
> bochs also has a release hook, but leaked the drm_device ever since
>
> commit 0a6659bdc5e8221da99eebb176fd9591435e38de
> Author: Gerd Hoffmann 
> Date:   Tue Dec 17 18:04:46 2013 +0100
>
>  drm/bochs: new driver
>
> This patch here fixes that leak.
>
> Same for virtio, started leaking with
>
> commit b1df3a2b24a917f8853d43fe9683c0e360d2c33a
> Author: Gerd Hoffmann 
> Date:   Tue Feb 11 14:58:04 2020 +0100
>
>  drm/virtio: add drm_driver.release callback.
>
> Cc: Gerd Hoffmann 
> Cc: Oleksandr Andrushchenko 
> Cc: xen-devel@lists.xenproject.org
>
> Signed-off-by: Daniel Vetter 
Thank you,
Reviewed-by: Oleksandr Andrushchenko 
> Cc: Maarten Lankhorst 
> Cc: Maxime Ripard 
> Cc: Thomas Zimmermann 
> Cc: David Airlie 
> Cc: Daniel Vetter 
> Cc: Oleksandr Andrushchenko 
> Cc: xen-devel@lists.xenproject.org
> ---
>   drivers/gpu/drm/drm_drv.c   | 3 +++
>   drivers/gpu/drm/xen/xen_drm_front.c | 2 +-
>   2 files changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/drm_drv.c b/drivers/gpu/drm/drm_drv.c
> index 3e5627d6eba6..9e62e28bbc62 100644
> --- a/drivers/gpu/drm/drm_drv.c
> +++ b/drivers/gpu/drm/drm_drv.c
> @@ -39,6 +39,7 @@
>   #include 
>   #include 
>   #include 
> +#include 
>   #include 
>   #include 
>   
> @@ -819,6 +820,8 @@ struct drm_device *drm_dev_alloc(struct drm_driver 
> *driver,
>   return ERR_PTR(ret);
>   }
>   
> + drmm_add_final_kfree(dev, dev);
> +
>   return dev;
>   }
>   EXPORT_SYMBOL(drm_dev_alloc);
> diff --git a/drivers/gpu/drm/xen/xen_drm_front.c 
> b/drivers/gpu/drm/xen/xen_drm_front.c
> index 4be49c1aef51..d22b5da38935 100644
> --- a/drivers/gpu/drm/xen/xen_drm_front.c
> +++ b/drivers/gpu/drm/xen/xen_drm_front.c
> @@ -461,7 +461,6 @@ static void xen_drm_drv_release(struct drm_device *dev)
>   drm_mode_config_cleanup(dev);
>   
>   drm_dev_fini(dev);
> - kfree(dev);
>   
>   if (front_info->cfg.be_alloc)
>   xenbus_switch_state(front_info->xb_dev,
> @@ -561,6 +560,7 @@ static int xen_drm_drv_init(struct xen_drm_front_info 
> *front_info)
>   fail_modeset:
>   drm_kms_helper_poll_fini(drm_dev);
>   drm_mode_config_cleanup(drm_dev);
> + drm_dev_put(drm_dev);
>   fail:
>   kfree(drm_info);
>   return ret;
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] PV DRM doesn't work without auto_translated_physmap feature in Dom0

[Oleksandr Andrushchenko]

On 2/5/20 8:59 PM, Santucco wrote:
> Hello,
> Ok, I  commented out the memcpy call and run the test.
> displ_be hasn’t crached, I have seen FLIP events in the log.
> But there hasn’t been the black screen, just a blink effect every 
> couple of seconds.
> Logs are attached.
Ok, so I believe that frontend - backend (displ_be) communication is ok
and there is nothing to do there.

Next, I would start debugging the following in Xen:
(XEN) mm.c:2223:d2v0 Bad L1 flags 80
and have a look at [1]. Probably, someone on Xen x86 side can tell
if this could be related to the flags at [2].

> Best regards,
> Alexander
>
> Среда, 5 февраля 2020, 9:31 +03:00 от Oleksandr Andrushchenko
> :
> On 2/4/20 10:28 AM, Santucco wrote:
> > Hello,
> > displ_be was compiled without zero-copy support early.
> > I have tried with the recompiled dom0 kernel, a result is the same.
> > Logs and configs (+displ_be’s CMakeCache.txt ) are attached.
> Ok, yet another test to localize the problem.
> Could you please remove memcpy from
> #1  0x55e5a1f28bec in Drm::DumbDrm::copy (this=0x7f9338000e00) at
> /home/santucco/tmp/xen-troops/displ_be/src/displayBackend/drm/Dumb.cpp:149
> and just memset the destination with 0 or whatever.
>
> I expect that system won't crash, nothing will be shown (black
> screen), but
> displ_be will show page flip events in its logs.
> > Best regards,
> > Alexander
> >
> > Понедельник, 3 февраля 2020, 10:36 +03:00 от Oleksandr
> > Andrushchenko  >:
> >
> >
> > On 2/1/20 4:39 PM, Santucco wrote:
> > > Hello again,
> > > I have not yet made to work my drm client, so I have tried to run
> > > linux like a domU (to see how it should work), it doesn’t work too
> > > — displ_be catches SIGSEGV:
> > >
> > > #0  0x7f4afed1c161 in ?? () from /lib64/libc.so.6
> > > #1  0x55723b9c5bec in Drm::DumbDrm::copy
> > (this=0x7f4adc000e00) at
> > >
> >
> /home/santucco/tmp/xen-troops/displ_be/src/displayBackend/drm/Dumb.cpp:149
> > > #2  0x55723b9a8f51 in BuffersStorage::getFrameBufferAndCopy
> > > (this=0x7f4ae00010e0, fbCookie=18446612682295083264) at
> > >
> >
> 
> /home/santucco/tmp/xen-troops/displ_be/src/displayBackend/BuffersStorage.cpp:165
> > > It tries to copy to mBuffer with non-accessible address.
> > > For the moment I see a strange offset for mmap call of
> > /dev/drm/card0
> > > in the strace log — 0x1. Is that normal?
> > > Any direction of which to dig will be very helpful.
> > > Configuration details:
> > > Xen 4.12.1 Dom0: Linux 4.20.17-gentoo #13 SMP Sat Dec 28
> > 11:12:24 MSK
> > > 2019 x86_64 Intel(R) Celeron(R) CPU N3050 @ 1.60GHz GenuineIntel
> > GNU/Linux
> > > DomU: Linux 4.20.17-gentoo
> > > last xen-troops/libxenbe and xen-troops/displ_be
> > > Logs (dmesg, xl dmesg, displ_be, strace log of displ_be), a
> > backtrace
> > > of gdb and kernel configs are attached.
> > > Thanks in advance.
> > Could you please try Dom0 kernel WITHOUT the options below:
> > CONFIG_XEN_GNTDEV_DMABUF=y
> > CONFIG_XEN_GRANT_DMA_ALLOC=y
> >
> > Then, just to make sure, did you build displ_be without zero-copy
> > support?
> >
> > > On 1/8/20 5:38 PM, Santucco wrote:
> > > > Thank you very much for all your answers.
> > > >
> > > > Среда, 8 января 2020, 10:54 +03:00 от Oleksandr Andrushchenko
> > > >  
> > > > >:
> > > > On 1/6/20 10:38 AM, Jürgen Groß wrote:
> > > > > On 06.01.20 08:56, Santucco wrote:
> > > > >> Hello,
> > > > >>
> > > > >> I’m trying to use vdispl interface from PV OS, it doesn’t
> work.
> > > > >> Configuration details:
> > > > >>  Xen 4.12.1
> > > > >>  Dom0: Linux 4.20.17-gentoo #13 SMP Sat Dec 28
> 11:12:24 MSK
> > > > 2019
> > > > >> x86_64 Intel(R) Celeron(R) CPU N3050 @ 1.60GHz GenuineIntel
> > > > GNU/Linux
> > > > >>  DomU: x86 Plan9, PV
> > > > >> displ_be as a backend for vdispl and vkb
> > > > >>
> > > > >> when VM starts, displ_be reports about an error:
> > 

Re: [Xen-devel] PV DRM doesn't work without auto_translated_physmap feature in Dom0

[Oleksandr Andrushchenko]

On 2/4/20 10:28 AM, Santucco wrote:
> Hello,
> displ_be was compiled without zero-copy support early.
> I have tried with the recompiled dom0 kernel, a result is the same.
> Logs and configs (+displ_be’s CMakeCache.txt ) are attached.
Ok, yet another test to localize the problem.
Could you please remove memcpy from
#1  0x55e5a1f28bec in Drm::DumbDrm::copy (this=0x7f9338000e00) at 
/home/santucco/tmp/xen-troops/displ_be/src/displayBackend/drm/Dumb.cpp:149
and just memset the destination with 0 or whatever.

I expect that system won't crash, nothing will be shown (black screen), but
displ_be will show page flip events in its logs.
> Best regards,
> Alexander
>
> Понедельник, 3 февраля 2020, 10:36 +03:00 от Oleksandr
> Andrushchenko :
>
>
> On 2/1/20 4:39 PM, Santucco wrote:
> > Hello again,
> > I have not yet made to work my drm client, so I have tried to run
> > linux like a domU (to see how it should work), it doesn’t work too
> > — displ_be catches SIGSEGV:
> >
> > #0  0x7f4afed1c161 in ?? () from /lib64/libc.so.6
> > #1  0x55723b9c5bec in Drm::DumbDrm::copy
> (this=0x7f4adc000e00) at
> >
> /home/santucco/tmp/xen-troops/displ_be/src/displayBackend/drm/Dumb.cpp:149
> > #2  0x55723b9a8f51 in BuffersStorage::getFrameBufferAndCopy
> > (this=0x7f4ae00010e0, fbCookie=18446612682295083264) at
> >
> 
> /home/santucco/tmp/xen-troops/displ_be/src/displayBackend/BuffersStorage.cpp:165
> > It tries to copy to mBuffer with non-accessible address.
> > For the moment I see a strange offset for mmap call of
> /dev/drm/card0
> > in the strace log — 0x1. Is that normal?
> > Any direction of which to dig will be very helpful.
> > Configuration details:
> > Xen 4.12.1 Dom0: Linux 4.20.17-gentoo #13 SMP Sat Dec 28
> 11:12:24 MSK
> > 2019 x86_64 Intel(R) Celeron(R) CPU N3050 @ 1.60GHz GenuineIntel
> GNU/Linux
> > DomU: Linux 4.20.17-gentoo
> > last xen-troops/libxenbe and xen-troops/displ_be
> > Logs (dmesg, xl dmesg, displ_be, strace log of displ_be), a
> backtrace
> > of gdb and kernel configs are attached.
> > Thanks in advance.
> Could you please try Dom0 kernel WITHOUT the options below:
> CONFIG_XEN_GNTDEV_DMABUF=y
> CONFIG_XEN_GRANT_DMA_ALLOC=y
>
> Then, just to make sure, did you build displ_be without zero-copy
> support?
>
> > On 1/8/20 5:38 PM, Santucco wrote:
> > > Thank you very much for all your answers.
> > >
> > > Среда, 8 января 2020, 10:54 +03:00 от Oleksandr Andrushchenko
> > >  > > >:
> > > On 1/6/20 10:38 AM, Jürgen Groß wrote:
> > > > On 06.01.20 08:56, Santucco wrote:
> > > >> Hello,
> > > >>
> > > >> I’m trying to use vdispl interface from PV OS, it doesn’t work.
> > > >> Configuration details:
> > > >>  Xen 4.12.1
> > > >>  Dom0: Linux 4.20.17-gentoo #13 SMP Sat Dec 28 11:12:24 MSK
> > > 2019
> > > >> x86_64 Intel(R) Celeron(R) CPU N3050 @ 1.60GHz GenuineIntel
> > > GNU/Linux
> > > >>  DomU: x86 Plan9, PV
> > > >> displ_be as a backend for vdispl and vkb
> > > >>
> > > >> when VM starts, displ_be reports about an error:
> > > >> gnttab: error: ioctl DMABUF_EXP_FROM_REFS failed: Invalid
> argument
> > > >> (displ_be.log:221)
> > > >>
> > > >> related Dom0 output is:
> > > >> [ 191.579278] Cannot provide dma-buf: use_ptemode 1
> > > >> (dmesg.create.log:123)
> > > >
> > > > This seems to be a limitation of the xen dma-buf driver. It was
> > > written
> > > > for being used on ARM initially where PV is not available.
> > > This is true and we never tried/targeted PV domains with this
> > > implementation,
> > > so if there is a need for that someone has to take a look on the
> > > proper
> > > implementation for PV…
> > >
> > > Have I got your right and there is no the proper
> implementation :-)?
> > There is no
> > >
> > > >
> > > > CC-ing Oleksandr Andrushchenko who is the author of that
> driver. He
> > > > should be able to tell us what would be needed to enable PV
> dom0.
> > 

Re: [Xen-devel] PV DRM doesn't work without auto_translated_physmap feature in Dom0

[Oleksandr Andrushchenko]

On 2/1/20 4:39 PM, Santucco wrote:
> Hello again,
> I have not yet made to work my drm client, so I have tried to run 
> linux like a domU (to see how it should work), it doesn’t work too 
> — displ_be catches SIGSEGV:
>
> #0  0x7f4afed1c161 in ?? () from /lib64/libc.so.6
> #1  0x55723b9c5bec in Drm::DumbDrm::copy (this=0x7f4adc000e00) at 
> /home/santucco/tmp/xen-troops/displ_be/src/displayBackend/drm/Dumb.cpp:149
> #2  0x55723b9a8f51 in BuffersStorage::getFrameBufferAndCopy 
> (this=0x7f4ae00010e0, fbCookie=18446612682295083264) at 
> /home/santucco/tmp/xen-troops/displ_be/src/displayBackend/BuffersStorage.cpp:165
> It tries to copy to mBuffer with non-accessible address.
> For the moment I see a strange offset for mmap call of /dev/drm/card0 
> in the strace log — 0x1. Is that normal?
> Any direction of which to dig will be very helpful.
> Configuration details:
> Xen 4.12.1 Dom0: Linux 4.20.17-gentoo #13 SMP Sat Dec 28 11:12:24 MSK 
> 2019 x86_64 Intel(R) Celeron(R) CPU N3050 @ 1.60GHz GenuineIntel GNU/Linux
> DomU: Linux 4.20.17-gentoo
> last xen-troops/libxenbe and xen-troops/displ_be
> Logs (dmesg, xl dmesg, displ_be, strace log of displ_be), a backtrace 
> of gdb and kernel configs are attached.
> Thanks in advance.
Could you please try Dom0 kernel WITHOUT the options below:
CONFIG_XEN_GNTDEV_DMABUF=y
CONFIG_XEN_GRANT_DMA_ALLOC=y

Then, just to make sure, did you build displ_be without zero-copy support?

> On 1/8/20 5:38 PM, Santucco wrote:
> > Thank you very much for all your answers.
> >
> > Среда, 8 января 2020, 10:54 +03:00 от Oleksandr Andrushchenko
> >  > >:
> > On 1/6/20 10:38 AM, Jürgen Groß wrote:
> > > On 06.01.20 08:56, Santucco wrote:
> > >> Hello,
> > >>
> > >> I’m trying to use vdispl interface from PV OS, it doesn’t work.
> > >> Configuration details:
> > >>  Xen 4.12.1
> > >>  Dom0: Linux 4.20.17-gentoo #13 SMP Sat Dec 28 11:12:24 MSK
> > 2019
> > >> x86_64 Intel(R) Celeron(R) CPU N3050 @ 1.60GHz GenuineIntel
> > GNU/Linux
> > >>  DomU: x86 Plan9, PV
> > >>  displ_be as a backend for vdispl and vkb
> > >>
> > >> when VM starts, displ_be reports about an error:
> > >> gnttab: error: ioctl DMABUF_EXP_FROM_REFS failed: Invalid argument
> > >> (displ_be.log:221)
> > >>
> > >> related Dom0 output is:
> > >> [  191.579278] Cannot provide dma-buf: use_ptemode 1
> > >> (dmesg.create.log:123)
> > >
> > > This seems to be a limitation of the xen dma-buf driver. It was
> > written
> > > for being used on ARM initially where PV is not available.
> > This is true and we never tried/targeted PV domains with this
> > implementation,
> > so if there is a need for that someone has to take a look on the
> > proper
> > implementation for PV…
> >
> > Have I got your right and there is no the proper implementation :-)?
> There is no
> >
> > >
> > > CC-ing Oleksandr Andrushchenko who is the author of that driver. He
> > > should be able to tell us what would be needed to enable PV dom0.
> > >
> > > Depending on your use case it might be possible to use PVH dom0, but
> > > support for this mode is "experimental" only and some features
> > are not
> > > yet working.
> > >
> > Well, one of the workarounds possible is to drop zero-copying use-case
> > (this is why display backend tries to create dmu-bufs from grants
> > passed
> > by the guest domain and fails because of "Cannot provide dma-buf:
> > use_ptemode 1")
> > So, in this case display backend will do memory copying for the
> > incoming
> > frames
> > and won't touch DMABUF_EXP_FROM_REFS ioctl.
> > To do so just disable zero-copying while building the backend [1]
> >
> > Thanks, I have just tried the workaround.  The backend has failed
> > in an other place not corresponding with dma_buf.
> > Anyway it is enough to continue debugging  my frontend implementation.
> > Do you know how big is performance penalty in comparison with
> > the zero-copy variant?
> Well, it solely depends on your setup, so I cannot tell what
> would the numbers be in your case. Comparing to what I have doesn't
> make any sense to me: one should compare apples to apples
> > Does it make a sense if I make a dedicated HVM domain with linux only
> > for the purpose of vdispl and vkbd backends? Is there a hope this
> > approach will work?
> You can try if this approach fits your design and requirements
> 

Re: [Xen-devel] [PATCH v5 15/15] drm/xen: Explicitly disable automatic sending of vblank event

[Oleksandr Andrushchenko]

On 1/29/20 2:05 PM, Thomas Zimmermann wrote:
> The atomic helpers automatically send out fake VBLANK events if no
> vblanking has been initialized. This would apply to xen, but xen has
> its own vblank logic. To avoid interfering with the atomic helpers,
> disable automatic vblank events explicitly.
>
> v5:
>   * update comment
> v4:
>   * separate commit from core vblank changes
>
> Signed-off-by: Thomas Zimmermann 
> Acked-by: Gerd Hoffmann 
> Reviewed-by: Daniel Vetter 
Thank you for your work,
Reviewed-by: Oleksandr Andrushchenko 
> ---
>   drivers/gpu/drm/xen/xen_drm_front_kms.c | 19 +++
>   1 file changed, 19 insertions(+)
>
> diff --git a/drivers/gpu/drm/xen/xen_drm_front_kms.c 
> b/drivers/gpu/drm/xen/xen_drm_front_kms.c
> index 4f34c5208180..78096bbcd226 100644
> --- a/drivers/gpu/drm/xen/xen_drm_front_kms.c
> +++ b/drivers/gpu/drm/xen/xen_drm_front_kms.c
> @@ -220,6 +220,24 @@ static bool display_send_page_flip(struct 
> drm_simple_display_pipe *pipe,
>   return false;
>   }
>   
> +static int display_check(struct drm_simple_display_pipe *pipe,
> +  struct drm_plane_state *plane_state,
> +  struct drm_crtc_state *crtc_state)
> +{
> + /*
> +  * Xen doesn't initialize vblanking via drm_vblank_init(), so
> +  * DRM helpers assume that it doesn't handle vblanking and start
> +  * sending out fake VBLANK events automatically.
> +  *
> +  * As xen contains it's own logic for sending out VBLANK events
> +  * in send_pending_event(), disable no_vblank (i.e., the xen
> +  * driver has vblanking support).
> +  */
> + crtc_state->no_vblank = false;
> +
> + return 0;
> +}
> +
>   static void display_update(struct drm_simple_display_pipe *pipe,
>  struct drm_plane_state *old_plane_state)
>   {
> @@ -284,6 +302,7 @@ static const struct drm_simple_display_pipe_funcs 
> display_funcs = {
>   .enable = display_enable,
>   .disable = display_disable,
>   .prepare_fb = drm_gem_fb_simple_display_pipe_prepare_fb,
> + .check = display_check,
>   .update = display_update,
>   };
>   
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 15/15] drm/xen: Explicitly disable automatic sending of vblank event

[Oleksandr Andrushchenko]

On 1/27/20 1:59 PM, Thomas Zimmermann wrote:
> Hi
>
> Am 27.01.20 um 10:53 schrieb Oleksandr Andrushchenko:
>> Sorry for jumping in late
>>
>> On 1/23/20 11:21 AM, Thomas Zimmermann wrote:
>>> The atomic helpers automatically send out fake VBLANK events if no
>>> vblanking has been initialized. This would apply to xen, but xen has
>>> its own vblank logic. To avoid interfering with the atomic helpers,
>>> disable automatic vblank events explictly.
>>>
>>> v4:
>>> * separate commit from core vblank changes
>>>
>>> Signed-off-by: Thomas Zimmermann 
>>> Acked-by: Gerd Hoffmann 
>> Reviewed-by: Oleksandr Andrushchenko 
>>
>>> ---
>>>drivers/gpu/drm/xen/xen_drm_front_kms.c | 13 +
>>>1 file changed, 13 insertions(+)
>>>
>>> diff --git a/drivers/gpu/drm/xen/xen_drm_front_kms.c 
>>> b/drivers/gpu/drm/xen/xen_drm_front_kms.c
>>> index 4f34c5208180..efde4561836f 100644
>>> --- a/drivers/gpu/drm/xen/xen_drm_front_kms.c
>>> +++ b/drivers/gpu/drm/xen/xen_drm_front_kms.c
>>> @@ -220,6 +220,18 @@ static bool display_send_page_flip(struct 
>>> drm_simple_display_pipe *pipe,
>>> return false;
>>>}
>>>
>>> +static int display_check(struct drm_simple_display_pipe *pipe,
>>> +struct drm_plane_state *plane_state,
>>> +struct drm_crtc_state *crtc_state)
>>> +{
>>> +   /* Make sure that DRM helpers don't send VBLANK events
>> Could you please put the comment on a separate line?
> You mean to add an empty line between comment and code?
>
Just like
/*
  * Make sure...
>>> +* automatically. Xen has it's own logic to do so.
>>> +*/
>>> +   crtc_state->no_vblank = false;
>> And it is still confusing, e.g. comment says
>> "Make sure that DRM helpers don't send VBLANK"
>> and we set "no_vblank" flag to false...
> I'll rephrase and add some more context.
Thank you
>
> Best regards
> Thomas
>
>>> +
>>> +   return 0;
>>> +}
>>> +
>>>static void display_update(struct drm_simple_display_pipe *pipe,
>>>struct drm_plane_state *old_plane_state)
>>>{
>>> @@ -284,6 +296,7 @@ static const struct drm_simple_display_pipe_funcs 
>>> display_funcs = {
>>> .enable = display_enable,
>>> .disable = display_disable,
>>> .prepare_fb = drm_gem_fb_simple_display_pipe_prepare_fb,
>>> +   .check = display_check,
>>> .update = display_update,
>>>};
>>>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 15/15] drm/xen: Explicitly disable automatic sending of vblank event

[Oleksandr Andrushchenko]

Sorry for jumping in late

On 1/23/20 11:21 AM, Thomas Zimmermann wrote:
> The atomic helpers automatically send out fake VBLANK events if no
> vblanking has been initialized. This would apply to xen, but xen has
> its own vblank logic. To avoid interfering with the atomic helpers,
> disable automatic vblank events explictly.
>
> v4:
>   * separate commit from core vblank changes
>
> Signed-off-by: Thomas Zimmermann 
> Acked-by: Gerd Hoffmann 
Reviewed-by: Oleksandr Andrushchenko 

> ---
>   drivers/gpu/drm/xen/xen_drm_front_kms.c | 13 +
>   1 file changed, 13 insertions(+)
>
> diff --git a/drivers/gpu/drm/xen/xen_drm_front_kms.c 
> b/drivers/gpu/drm/xen/xen_drm_front_kms.c
> index 4f34c5208180..efde4561836f 100644
> --- a/drivers/gpu/drm/xen/xen_drm_front_kms.c
> +++ b/drivers/gpu/drm/xen/xen_drm_front_kms.c
> @@ -220,6 +220,18 @@ static bool display_send_page_flip(struct 
> drm_simple_display_pipe *pipe,
>   return false;
>   }
>   
> +static int display_check(struct drm_simple_display_pipe *pipe,
> +  struct drm_plane_state *plane_state,
> +  struct drm_crtc_state *crtc_state)
> +{
> + /* Make sure that DRM helpers don't send VBLANK events
Could you please put the comment on a separate line?
> +  * automatically. Xen has it's own logic to do so.
> +  */
> + crtc_state->no_vblank = false;
And it is still confusing, e.g. comment says
"Make sure that DRM helpers don't send VBLANK"
and we set "no_vblank" flag to false...
> +
> + return 0;
> +}
> +
>   static void display_update(struct drm_simple_display_pipe *pipe,
>  struct drm_plane_state *old_plane_state)
>   {
> @@ -284,6 +296,7 @@ static const struct drm_simple_display_pipe_funcs 
> display_funcs = {
>   .enable = display_enable,
>   .disable = display_disable,
>   .prepare_fb = drm_gem_fb_simple_display_pipe_prepare_fb,
> + .check = display_check,
>   .update = display_update,
>   };
>   
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] PV DRM doesn't work without auto_translated_physmap feature in Dom0

[Oleksandr Andrushchenko]

On 1/8/20 5:38 PM, Santucco wrote:
> Thank you very much for all your answers.
>
> Среда, 8 января 2020, 10:54 +03:00 от Oleksandr Andrushchenko
>  >:
> On 1/6/20 10:38 AM, Jürgen Groß wrote:
> > On 06.01.20 08:56, Santucco wrote:
> >> Hello,
> >>
> >> I’m trying to use vdispl interface from PV OS, it doesn’t work.
> >> Configuration details:
> >>  Xen 4.12.1
> >>  Dom0: Linux 4.20.17-gentoo #13 SMP Sat Dec 28 11:12:24 MSK
> 2019
> >> x86_64 Intel(R) Celeron(R) CPU N3050 @ 1.60GHz GenuineIntel
> GNU/Linux
> >>  DomU: x86 Plan9, PV
> >>  displ_be as a backend for vdispl and vkb
> >>
> >> when VM starts, displ_be reports about an error:
> >> gnttab: error: ioctl DMABUF_EXP_FROM_REFS failed: Invalid argument
> >> (displ_be.log:221)
> >>
> >> related Dom0 output is:
> >> [  191.579278] Cannot provide dma-buf: use_ptemode 1
> >> (dmesg.create.log:123)
> >
> > This seems to be a limitation of the xen dma-buf driver. It was
> written
> > for being used on ARM initially where PV is not available.
> This is true and we never tried/targeted PV domains with this
> implementation,
> so if there is a need for that someone has to take a look on the
> proper
> implementation for PV…
>
> Have I got your right and there is no the proper implementation :-)?
There is no
>
> >
> > CC-ing Oleksandr Andrushchenko who is the author of that driver. He
> > should be able to tell us what would be needed to enable PV dom0.
> >
> > Depending on your use case it might be possible to use PVH dom0, but
> > support for this mode is "experimental" only and some features
> are not
> > yet working.
> >
> Well, one of the workarounds possible is to drop zero-copying use-case
> (this is why display backend tries to create dmu-bufs from grants
> passed
> by the guest domain and fails because of "Cannot provide dma-buf:
> use_ptemode 1")
> So, in this case display backend will do memory copying for the
> incoming
> frames
> and won't touch DMABUF_EXP_FROM_REFS ioctl.
> To do so just disable zero-copying while building the backend [1]
>
> Thanks, I have just tried the workaround.  The backend has failed 
> in an other place not corresponding with dma_buf.
> Anyway it is enough to continue debugging  my frontend implementation.
> Do you know how big is performance penalty in comparison with 
> the zero-copy variant?
Well, it solely depends on your setup, so I cannot tell what
would the numbers be in your case. Comparing to what I have doesn't
make any sense to me: one should compare apples to apples
> Does it make a sense if I make a dedicated HVM domain with linux only 
> for the purpose of vdispl and vkbd backends? Is there a hope this 
> approach will work?
You can try if this approach fits your design and requirements
>
> >
> > Juergen
> >
> [1]
> https://github.com/xen-troops/displ_be/blob/master/CMakeLists.txt#L12
> 
> <https://urldefense.com/v3/__https://github.com/xen-troops/displ_be/blob/master/CMakeLists.txt*L12__;Iw!!GF_29dbcQIUBPA!mz3gn1wQMX2DXeNuAV-1_dI7nxFYYZOgdPiJNSFMesCz9lAzOKlwVPlddbxbcLmUO44NOy0TFA$>
>
> Best regards,
>   Alexander Sychev
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] PV DRM doesn't work without auto_translated_physmap feature in Dom0

[Oleksandr Andrushchenko]

On 1/6/20 10:38 AM, Jürgen Groß wrote:
> On 06.01.20 08:56, Santucco wrote:
>> Hello,
>>
>> I’m trying to use vdispl interface from PV OS, it doesn’t work.
>> Configuration details:
>>  Xen 4.12.1
>>  Dom0: Linux 4.20.17-gentoo #13 SMP Sat Dec 28 11:12:24 MSK 2019 
>> x86_64 Intel(R) Celeron(R) CPU N3050 @ 1.60GHz GenuineIntel GNU/Linux
>>  DomU: x86 Plan9, PV
>>  displ_be as a backend for vdispl and vkb
>>
>> when VM starts, displ_be reports about an error:
>> gnttab: error: ioctl DMABUF_EXP_FROM_REFS failed: Invalid argument 
>> (displ_be.log:221)
>>
>> related Dom0 output is:
>> [  191.579278] Cannot provide dma-buf: use_ptemode 1 
>> (dmesg.create.log:123)
>
> This seems to be a limitation of the xen dma-buf driver. It was written
> for being used on ARM initially where PV is not available.
This is true and we never tried/targeted PV domains with this 
implementation,
so if there is a need for that someone has to take a look on the proper
implementation for PV...
>
> CC-ing Oleksandr Andrushchenko who is the author of that driver. He
> should be able to tell us what would be needed to enable PV dom0.
>
> Depending on your use case it might be possible to use PVH dom0, but
> support for this mode is "experimental" only and some features are not
> yet working.
>
Well, one of the workarounds possible is to drop zero-copying use-case
(this is why display backend tries to create dmu-bufs from grants passed
by the guest domain and fails because of "Cannot provide dma-buf: 
use_ptemode 1")
So, in this case display backend will do memory copying for the incoming 
frames
and won't touch DMABUF_EXP_FROM_REFS ioctl.
To do so just disable zero-copying while building the backend [1]
>
> Juergen
>
[1] https://github.com/xen-troops/displ_be/blob/master/CMakeLists.txt#L12
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel