On Fri, Sep 23, 2016 at 03:27:07PM +0200, Samuel Thibault wrote: > Marek Marczykowski-Górecki, on Fri 23 Sep 2016 10:48:14 +0200, wrote: > > I'm still trying to get PCI passthrough working with stubdomain and > > without qemu in dom0 (even for just vfb/vkbd backends). How is this > > supposed to work? > > Just as I recall from my memory: > > > 1. Should xen-pcifront in the target domain be used (and consequently, > > should xen-pciback be set for it)? > > I guess that could work.
Could, or should? ;)
In the meantime, I've found this in xen-pcifront driver:
static int __init pcifront_init(void)
{
if (!xen_pv_domain() || xen_initial_domain())
return -ENODEV;
So, it looks like pcifront is not used in PV domain.
> > Currently xen-pciback is set for both
> > stubdomain and target domain, which presents a race condition and
> > xen-pciback refuses to setup one of them.
>
> Yes, that's expected, for the reason you say.
In the meantime I've tried to modify libxl to not setup pciback for
target domain. This, along with other modifications (see below) improved
the situation.
> * to summarize
> **************
>
> If running PV drivers in the guest, you set the pciback on the guest, be
> it run with stubdom or not.
> If running plain drivers in the guest,
> * when not using a stubdom you don't need to set a pciback.
> * when using a stubdom you need to set a pciback on the stubdom.
Thanks for the explanation!
> So the unfortunate thing is that when using stubdom, you'd have to set
> the pciback either on the guest (to run a PV driver in it), or on the
> stubdom (to run a plain driver in the guest, and let mini-os use PV to
> let qemu pass the board through)
I've tried only on Linux HVM guest and as noted above xen-pcifront
doesn't look to be involved. Is it any different on other OSes?
As for getting PCI passthrough working with stubdomain, for now I hit
those problems:
1. getdomaininfo not allowed from stubdomain (already fixed in master),
2. double setup of pciback (explained above) - for now I've disabled one
of them,
3. race condition between pcifront in mini-os and qemu.
Regarding the last one:
Toolstack during (stub)domain startup setup two things, mostly
asynchronously:
1. pciback/pcifront (through standard xenstore entries)
2. instruct qemu to attach device (by writing "pci-ins" to
device-model/xxx/command)
The later fails if pcifront is not initialized yet. For now I've moved
the second step to be really after the first one, including
synchronization through libxl__wait_for_backend call.
Is it ok? Or maybe it should be done on stubdomain side (handling
"pci-ins" should wait on pcifront)? I guess the same will apply to
qemu-xen case, or any other stubdomain, so probably better have it in
toolstack, right?
Work-in-progress patches attached. The result is that lspci inside the
guest finally list the device. No idea if it's really working yet.
--
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
From b8a34904411cc6e7a7abdc6296657ff5f3eb92e9 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?=
<marma...@invisiblethingslab.com>
Date: Thu, 22 Sep 2016 16:07:14 +0200
Subject: [PATCH 1/2] libxl: attach xen-pciback only to stubdomain
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Organization: Invisible Things Lab
Cc: Marek Marczykowski-Górecki <marma...@invisiblethingslab.com>
Signed-off-by: Marek Marczykowski-Górecki <marma...@invisiblethingslab.com>
---
tools/libxl/libxl_create.c | 2 +-
tools/libxl/libxl_pci.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index c6862b8..5625ef2 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -1429,7 +1429,7 @@ static void domcreate_attach_pci(libxl__egc *egc,
libxl__multidev *multidev,
}
}
- if (d_config->num_pcidevs > 0) {
+ if (d_config->num_pcidevs > 0 && d_config->c_info.type ==
LIBXL_DOMAIN_TYPE_PV) {
ret = libxl__create_pci_backend(gc, domid, d_config->pcidevs,
d_config->num_pcidevs);
if (ret < 0) {
diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c
index 19c597e..2942772 100644
--- a/tools/libxl/libxl_pci.c
+++ b/tools/libxl/libxl_pci.c
@@ -1001,7 +1001,7 @@ out:
}
}
- if (!starting)
+ if (!starting && !hvm)
rc = libxl__device_pci_add_xenstore(gc, domid, pcidev, starting);
else
rc = 0;
--
2.5.5
From 930398583d5a590c511e58057d000e1ed7defb82 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?=
<marma...@invisiblethingslab.com>
Date: Fri, 23 Sep 2016 16:13:37 +0200
Subject: [PATCH 2/2] libxl: attach PCI device to qemu only after setting
pciback/pcifront
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Organization: Invisible Things Lab
Cc: Marek Marczykowski-Górecki <marma...@invisiblethingslab.com>
When qemu is running in stubdomain, handling "pci-ins" command will fail
if pcifront is not initialized already. Fix this by sending such command
only after confirming that pciback/front is running.
Signed-off-by: Marek Marczykowski-Górecki <marma...@invisiblethingslab.com>
---
tools/libxl/libxl_pci.c | 50 ++++++++++++++++++++++++++-----------------------
1 file changed, 27 insertions(+), 23 deletions(-)
diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c
index 2942772..88f8f5b 100644
--- a/tools/libxl/libxl_pci.c
+++ b/tools/libxl/libxl_pci.c
@@ -891,34 +891,15 @@ static int do_pci_add(libxl__gc *gc, uint32_t domid,
libxl_device_pci *pcidev, i
libxl_ctx *ctx = libxl__gc_owner(gc);
libxl_domain_type type = libxl__domain_type(gc, domid);
char *sysfs_path;
+ char *be_path;
FILE *f;
unsigned long long start, end, flags, size;
- int irq, i, rc, hvm = 0;
+ int irq, i, rc, hvm = (type == LIBXL_DOMAIN_TYPE_HVM);
uint32_t flag = XEN_DOMCTL_DEV_RDM_RELAXED;
if (type == LIBXL_DOMAIN_TYPE_INVALID)
return ERROR_FAIL;
- if (type == LIBXL_DOMAIN_TYPE_HVM) {
- hvm = 1;
- if (libxl__wait_for_device_model_deprecated(gc, domid, "running",
- NULL, NULL, NULL) < 0) {
- return ERROR_FAIL;
- }
- switch (libxl__device_model_version_running(gc, domid)) {
- case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL:
- rc = qemu_pci_add_xenstore(gc, domid, pcidev);
- break;
- case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
- rc = libxl__qmp_pci_add(gc, domid, pcidev);
- break;
- default:
- return ERROR_INVAL;
- }
- if ( rc )
- return ERROR_FAIL;
- }
-
sysfs_path = libxl__sprintf(gc, SYSFS_PCI_DEV"/"PCI_BDF"/resource",
pcidev->domain,
pcidev->bus, pcidev->dev, pcidev->func);
f = fopen(sysfs_path, "r");
@@ -1001,10 +982,33 @@ out:
}
}
- if (!starting && !hvm)
+ if (!starting && !hvm) {
rc = libxl__device_pci_add_xenstore(gc, domid, pcidev, starting);
- else
+ be_path = libxl__sprintf(gc, "%s/backend/pci/%d/0",
libxl__xs_get_dompath(gc, 0), domid);
+ if (libxl__wait_for_backend(gc, be_path, GCSPRINTF("%d",
XenbusStateConnected)) < 0)
+ return ERROR_FAIL;
+ } else
rc = 0;
+
+ if (type == LIBXL_DOMAIN_TYPE_HVM) {
+ if (libxl__wait_for_device_model_deprecated(gc, domid, "running",
+ NULL, NULL, NULL) < 0) {
+ return ERROR_FAIL;
+ }
+ switch (libxl__device_model_version_running(gc, domid)) {
+ case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL:
+ rc = qemu_pci_add_xenstore(gc, domid, pcidev);
+ break;
+ case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
+ rc = libxl__qmp_pci_add(gc, domid, pcidev);
+ break;
+ default:
+ return ERROR_INVAL;
+ }
+ if ( rc )
+ return ERROR_FAIL;
+ }
+
return rc;
}
--
2.5.5
signature.asc
Description: PGP signature
_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
