RE: [PATCH linux-next v2 4/7] vdpa: Define vdpa mgmt device, ops and a netlink interface
> From: Jason Wang
> Sent: Monday, January 4, 2021 12:33 PM
>
> On 2021/1/4 上午11:31, Parav Pandit wrote:
> > To add one or more VDPA devices, define a management device which
> > allows adding or removing vdpa device. A management device defines set
> > of callbacks to manage vdpa devices.
> >
> > To begin with, it defines add and remove callbacks through which a
> > user defined vdpa device can be added or removed.
> >
> > A unique management device is identified by its unique handle
> > identified by management device name and optionally the bus name.
> >
> > Hence, introduce routine through which driver can register a
> > management device and its callback operations for adding and remove a
> > vdpa device.
> >
> > Introduce vdpa netlink socket family so that user can query management
> > device and its attributes.
> >
> > Example of show vdpa management device which allows creating vdpa
> > device of networking class (device id = 0x1) of virtio specification
> > 1.1 section 5.1.1.
> >
> > $ vdpa mgmtdev show
> > vdpasim_net:
> >supported_classes:
> > net
> >
> > Example of showing vdpa management device in JSON format.
> >
> > $ vdpa mgmtdev show -jp
> > {
> > "show": {
> > "vdpasim_net": {
> > "supported_classes": [ "net" ]
> > }
> > }
> > }
> >
> > Signed-off-by: Parav Pandit
> > Reviewed-by: Eli Cohen
> > Reviewed-by: Jason Wang
> > ---
> > Changelog:
> > v1->v2:
> > - rebased
> > - updated commit log example for management device name from
> > "vdpasim" to "vdpasim_net"
> > - removed device_id as net and block management devices are
> > separated
>
>
> So I wonder whether there could be a type of management devices that can
> deal with multiple types of virtio devices. If yes, we probably need to add
> device id back.
At this point mlx5 plan to support only net.
It is useful to see what type of vdpa device is supported by a management
device.
In future if a mgmt dev supports multiple types, user needs to choose desired
type.
I guess we can differ this optional type to future, when such mgmt. device
will/may be available.
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [PATCH linux-next v2 7/7] vdpa_sim_net: Add support for user supported devices
> From: Jason Wang
> Sent: Monday, January 4, 2021 12:35 PM
>
> On 2021/1/4 上午11:31, Parav Pandit wrote:
> > static int __init vdpasim_net_init(void)
> > {
> > int ret = 0;
> > @@ -176,6 +264,8 @@ static int __init vdpasim_net_init(void)
> >
> > if (default_device)
> > ret = vdpasim_net_default_dev_register();
> > + else
> > + ret = vdpasim_net_mgmtdev_init();
> > return ret;
> > }
> >
> > @@ -183,6 +273,8 @@ static void __exit vdpasim_net_exit(void)
> > {
> > if (default_device)
> > vdpasim_net_default_dev_unregister();
> > + else
> > + vdpasim_net_mgmtdev_cleanup();
> > }
> >
> > module_init(vdpasim_net_init);
> > -- 2.26.2
>
>
> I wonder what's the value of keeping the default device that is out of the
> control of management API.
I think we can remove it like how I did in the v1 version. And actual vendor
drivers like mlx5_vdpa will likely should do only user created devices.
I added only for backward compatibility purpose, but we can remove the default
simulated vdpa net device.
What do you recommend?
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v2 7/7] vdpa_sim_net: Add support for user supported devices
On 2021/1/4 上午11:31, Parav Pandit wrote:
static int __init vdpasim_net_init(void)
{
int ret = 0;
@@ -176,6 +264,8 @@ static int __init vdpasim_net_init(void)
if (default_device)
ret = vdpasim_net_default_dev_register();
+ else
+ ret = vdpasim_net_mgmtdev_init();
return ret;
}
@@ -183,6 +273,8 @@ static void __exit vdpasim_net_exit(void)
{
if (default_device)
vdpasim_net_default_dev_unregister();
+ else
+ vdpasim_net_mgmtdev_cleanup();
}
module_init(vdpasim_net_init);
-- 2.26.2
I wonder what's the value of keeping the default device that is out of
the control of management API.
Thanks
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v2 4/7] vdpa: Define vdpa mgmt device, ops and a netlink interface
On 2021/1/4 上午11:31, Parav Pandit wrote:
To add one or more VDPA devices, define a management device which
allows adding or removing vdpa device. A management device defines
set of callbacks to manage vdpa devices.
To begin with, it defines add and remove callbacks through which a user
defined vdpa device can be added or removed.
A unique management device is identified by its unique handle identified
by management device name and optionally the bus name.
Hence, introduce routine through which driver can register a
management device and its callback operations for adding and remove
a vdpa device.
Introduce vdpa netlink socket family so that user can query management
device and its attributes.
Example of show vdpa management device which allows creating vdpa device of
networking class (device id = 0x1) of virtio specification 1.1
section 5.1.1.
$ vdpa mgmtdev show
vdpasim_net:
supported_classes:
net
Example of showing vdpa management device in JSON format.
$ vdpa mgmtdev show -jp
{
"show": {
"vdpasim_net": {
"supported_classes": [ "net" ]
}
}
}
Signed-off-by: Parav Pandit
Reviewed-by: Eli Cohen
Reviewed-by: Jason Wang
---
Changelog:
v1->v2:
- rebased
- updated commit log example for management device name from
"vdpasim" to "vdpasim_net"
- removed device_id as net and block management devices are separated
So I wonder whether there could be a type of management devices that can
deal with multiple types of virtio devices. If yes, we probably need to
add device id back.
Thanks
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH linux-next v2 1/7] vdpa_sim_net: Make mac address array static
On 2021/1/4 上午11:31, Parav Pandit wrote: MAC address array is used only in vdpa_sim_net.c. Hence, keep it static. Signed-off-by: Parav Pandit --- Changelog: v1->v2: - new patch --- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c index c10b6981fdab..f0482427186b 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c @@ -33,7 +33,7 @@ static char *macaddr; module_param(macaddr, charp, 0); MODULE_PARM_DESC(macaddr, "Ethernet MAC address"); -u8 macaddr_buf[ETH_ALEN]; +static u8 macaddr_buf[ETH_ALEN]; static struct vdpasim *vdpasim_net_dev; Acked-by: Jason Wang ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH V2 00/19] vDPA driver for virtio-pci device
On 2021/1/3 下午8:37, Michael S. Tsirkin wrote: On Thu, Dec 31, 2020 at 11:52:14AM +0800, Jason Wang wrote: On 2020/12/4 下午12:03, Jason Wang wrote: Hi all: This series tries to implement a vDPA driver for virtio-pci device which will bridge between vDPA bus and virtio-pci device. This could be used for future feature prototyping and testing. Please review Changes from V2: - don't try to use devres for virtio-pci core - tweak the commit log - split the patches furtherly to ease the reviewing Changes from V1: - Split common codes from virito-pci and share it with vDPA driver - Use dynamic id in order to be less confusing with virtio-pci driver - No feature whitelist, supporting any features (mq, config etc) Thanks Michael, any comment for this series? It's needed for testing doorbell mapping and config interrupt support. Thanks I saw you got some comments back in december so was expecting another version. If you'd rather I reviewed this one, let me know. Right. So I've posted a new version that tries to address the previous comments. Please review that version. Thanks ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 19/19] vdpa: introduce virtio pci driver
This patch introduce a vDPA driver for virtio-pci device. It bridges
the virtio-pci control command to the vDPA bus. This will be used for
features prototyping and testing.
Note that get/restore virtqueue state is not supported which needs
extension on the virtio specification.
Signed-off-by: Jason Wang
---
drivers/vdpa/Kconfig | 6 +
drivers/vdpa/Makefile | 1 +
drivers/vdpa/virtio_pci/Makefile | 2 +
drivers/vdpa/virtio_pci/vp_vdpa.c | 456 ++
4 files changed, 465 insertions(+)
create mode 100644 drivers/vdpa/virtio_pci/Makefile
create mode 100644 drivers/vdpa/virtio_pci/vp_vdpa.c
diff --git a/drivers/vdpa/Kconfig b/drivers/vdpa/Kconfig
index 6caf539091e5..81c6a3520813 100644
--- a/drivers/vdpa/Kconfig
+++ b/drivers/vdpa/Kconfig
@@ -49,4 +49,10 @@ config MLX5_VDPA_NET
be executed by the hardware. It also supports a variety of stateless
offloads depending on the actual device used and firmware version.
+config VP_VDPA
+ tristate "Virtio PCI bridge vDPA driver"
+ depends on PCI_MSI && VIRTIO_PCI_MODERN
+ help
+ This kernel module bridges virtio PCI device to vDPA bus.
+
endif # VDPA
diff --git a/drivers/vdpa/Makefile b/drivers/vdpa/Makefile
index d160e9b63a66..67fe7f3d6943 100644
--- a/drivers/vdpa/Makefile
+++ b/drivers/vdpa/Makefile
@@ -3,3 +3,4 @@ obj-$(CONFIG_VDPA) += vdpa.o
obj-$(CONFIG_VDPA_SIM) += vdpa_sim/
obj-$(CONFIG_IFCVF)+= ifcvf/
obj-$(CONFIG_MLX5_VDPA) += mlx5/
+obj-$(CONFIG_VP_VDPA)+= virtio_pci/
diff --git a/drivers/vdpa/virtio_pci/Makefile b/drivers/vdpa/virtio_pci/Makefile
new file mode 100644
index ..231088d3af7d
--- /dev/null
+++ b/drivers/vdpa/virtio_pci/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_VP_VDPA) += vp_vdpa.o
diff --git a/drivers/vdpa/virtio_pci/vp_vdpa.c
b/drivers/vdpa/virtio_pci/vp_vdpa.c
new file mode 100644
index ..4eda926493d9
--- /dev/null
+++ b/drivers/vdpa/virtio_pci/vp_vdpa.c
@@ -0,0 +1,456 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * vDPA bridge driver for modern virtio-pci device
+ *
+ * Copyright (c) 2020, Red Hat Inc. All rights reserved.
+ * Author: Jason Wang
+ *
+ * Based on virtio_pci_modern.c.
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#define VP_VDPA_QUEUE_MAX 256
+#define VP_VDPA_DRIVER_NAME "vp_vdpa"
+#define VP_VDPA_NAME_SIZE 256
+
+struct vp_vring {
+ void __iomem *notify;
+ char msix_name[VP_VDPA_NAME_SIZE];
+ struct vdpa_callback cb;
+ int irq;
+};
+
+struct vp_vdpa {
+ struct vdpa_device vdpa;
+ struct virtio_pci_modern_device mdev;
+ struct vp_vring *vring;
+ struct vdpa_callback config_cb;
+ char msix_name[VP_VDPA_NAME_SIZE];
+ int config_irq;
+ int queues;
+ int vectors;
+};
+
+static struct vp_vdpa *vdpa_to_vp(struct vdpa_device *vdpa)
+{
+ return container_of(vdpa, struct vp_vdpa, vdpa);
+}
+
+static struct virtio_pci_modern_device *vdpa_to_mdev(struct vdpa_device *vdpa)
+{
+ struct vp_vdpa *vp_vdpa = vdpa_to_vp(vdpa);
+
+ return _vdpa->mdev;
+}
+
+static u64 vp_vdpa_get_features(struct vdpa_device *vdpa)
+{
+ struct virtio_pci_modern_device *mdev = vdpa_to_mdev(vdpa);
+
+ return vp_modern_get_features(mdev);
+}
+
+static int vp_vdpa_set_features(struct vdpa_device *vdpa, u64 features)
+{
+ struct virtio_pci_modern_device *mdev = vdpa_to_mdev(vdpa);
+
+ vp_modern_set_features(mdev, features);
+
+ return 0;
+}
+
+static u8 vp_vdpa_get_status(struct vdpa_device *vdpa)
+{
+ struct virtio_pci_modern_device *mdev = vdpa_to_mdev(vdpa);
+
+ return vp_modern_get_status(mdev);
+}
+
+static void vp_vdpa_free_irq(struct vp_vdpa *vp_vdpa)
+{
+ struct virtio_pci_modern_device *mdev = _vdpa->mdev;
+ struct pci_dev *pdev = mdev->pci_dev;
+ int i;
+
+ for (i = 0; i < vp_vdpa->queues; i++) {
+ if (vp_vdpa->vring[i].irq != VIRTIO_MSI_NO_VECTOR) {
+ vp_modern_queue_vector(mdev, i, VIRTIO_MSI_NO_VECTOR);
+ devm_free_irq(>dev, vp_vdpa->vring[i].irq,
+ _vdpa->vring[i]);
+ vp_vdpa->vring[i].irq = VIRTIO_MSI_NO_VECTOR;
+ }
+ }
+
+ if (vp_vdpa->config_irq != VIRTIO_MSI_NO_VECTOR) {
+ vp_modern_config_vector(mdev, VIRTIO_MSI_NO_VECTOR);
+ devm_free_irq(>dev, vp_vdpa->config_irq, vp_vdpa);
+ vp_vdpa->config_irq = VIRTIO_MSI_NO_VECTOR;
+ }
+
+ if (vp_vdpa->vectors) {
+ pci_free_irq_vectors(pdev);
+ vp_vdpa->vectors = 0;
+ }
+}
+
+static irqreturn_t vp_vdpa_vq_handler(int irq, void *arg)
+{
+ struct vp_vring *vring = arg;
+
+ if (vring->cb.callback)
+ return vring->cb.callback(vring->cb.private);
+
+ return
[PATCH V3 18/19] virtio_vdpa: don't warn when fail to disable vq
There's no guarantee that the device can disable a specific virtqueue through set_vq_ready(). One example is the modern virtio-pci device. So this patch removes the warning. Signed-off-by: Jason Wang --- drivers/virtio/virtio_vdpa.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c index 4a9ddb44b2a7..e28acf482e0c 100644 --- a/drivers/virtio/virtio_vdpa.c +++ b/drivers/virtio/virtio_vdpa.c @@ -225,9 +225,8 @@ static void virtio_vdpa_del_vq(struct virtqueue *vq) list_del(>node); spin_unlock_irqrestore(_dev->lock, flags); - /* Select and deactivate the queue */ + /* Select and deactivate the queue (best effort) */ ops->set_vq_ready(vdpa, index, 0); - WARN_ON(ops->get_vq_ready(vdpa, index)); vring_del_virtqueue(vq); -- 2.25.1 ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 17/19] vdpa: set the virtqueue num during register
This patch delay the queue number setting to vDPA device
registering. This allows us to probe the virtqueue numbers between
device allocation and registering.
Reviewed-by: Stefano Garzarella
Signed-off-by: Jason Wang
---
drivers/vdpa/ifcvf/ifcvf_main.c | 5 ++---
drivers/vdpa/mlx5/net/mlx5_vnet.c | 5 ++---
drivers/vdpa/vdpa.c | 8
drivers/vdpa/vdpa_sim/vdpa_sim.c | 4 ++--
include/linux/vdpa.h | 7 +++
5 files changed, 13 insertions(+), 16 deletions(-)
diff --git a/drivers/vdpa/ifcvf/ifcvf_main.c b/drivers/vdpa/ifcvf/ifcvf_main.c
index 8b4028556cb6..d65f3221d8ed 100644
--- a/drivers/vdpa/ifcvf/ifcvf_main.c
+++ b/drivers/vdpa/ifcvf/ifcvf_main.c
@@ -438,8 +438,7 @@ static int ifcvf_probe(struct pci_dev *pdev, const struct
pci_device_id *id)
}
adapter = vdpa_alloc_device(struct ifcvf_adapter, vdpa,
- dev, _vdpa_ops,
- IFCVF_MAX_QUEUE_PAIRS * 2);
+ dev, _vdpa_ops);
if (adapter == NULL) {
IFCVF_ERR(pdev, "Failed to allocate vDPA structure");
return -ENOMEM;
@@ -463,7 +462,7 @@ static int ifcvf_probe(struct pci_dev *pdev, const struct
pci_device_id *id)
for (i = 0; i < IFCVF_MAX_QUEUE_PAIRS * 2; i++)
vf->vring[i].irq = -EINVAL;
- ret = vdpa_register_device(>vdpa);
+ ret = vdpa_register_device(>vdpa, IFCVF_MAX_QUEUE_PAIRS * 2);
if (ret) {
IFCVF_ERR(pdev, "Failed to register ifcvf to vdpa bus");
goto err;
diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c
b/drivers/vdpa/mlx5/net/mlx5_vnet.c
index f1d54814db97..a1b9260bf04d 100644
--- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
+++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
@@ -1958,8 +1958,7 @@ static int mlx5v_probe(struct auxiliary_device *adev,
max_vqs = MLX5_CAP_DEV_VDPA_EMULATION(mdev, max_num_virtio_queues);
max_vqs = min_t(u32, max_vqs, MLX5_MAX_SUPPORTED_VQS);
- ndev = vdpa_alloc_device(struct mlx5_vdpa_net, mvdev.vdev,
mdev->device, _vdpa_ops,
-2 * mlx5_vdpa_max_qps(max_vqs));
+ ndev = vdpa_alloc_device(struct mlx5_vdpa_net, mvdev.vdev,
mdev->device, _vdpa_ops);
if (IS_ERR(ndev))
return PTR_ERR(ndev);
@@ -1986,7 +1985,7 @@ static int mlx5v_probe(struct auxiliary_device *adev,
if (err)
goto err_res;
- err = vdpa_register_device(>vdev);
+ err = vdpa_register_device(>vdev, 2 *
mlx5_vdpa_max_qps(max_vqs));
if (err)
goto err_reg;
diff --git a/drivers/vdpa/vdpa.c b/drivers/vdpa/vdpa.c
index a69ffc991e13..ba89238f9898 100644
--- a/drivers/vdpa/vdpa.c
+++ b/drivers/vdpa/vdpa.c
@@ -61,7 +61,6 @@ static void vdpa_release_dev(struct device *d)
* initialized but before registered.
* @parent: the parent device
* @config: the bus operations that is supported by this device
- * @nvqs: number of virtqueues supported by this device
* @size: size of the parent structure that contains private data
*
* Driver should use vdpa_alloc_device() wrapper macro instead of
@@ -72,7 +71,6 @@ static void vdpa_release_dev(struct device *d)
*/
struct vdpa_device *__vdpa_alloc_device(struct device *parent,
const struct vdpa_config_ops *config,
- int nvqs,
size_t size)
{
struct vdpa_device *vdev;
@@ -99,7 +97,6 @@ struct vdpa_device *__vdpa_alloc_device(struct device *parent,
vdev->index = err;
vdev->config = config;
vdev->features_valid = false;
- vdev->nvqs = nvqs;
err = dev_set_name(>dev, "vdpa%u", vdev->index);
if (err)
@@ -122,11 +119,14 @@ EXPORT_SYMBOL_GPL(__vdpa_alloc_device);
* vdpa_register_device - register a vDPA device
* Callers must have a succeed call of vdpa_alloc_device() before.
* @vdev: the vdpa device to be registered to vDPA bus
+ * @nvqs: number of virtqueues supported by this device
*
* Returns an error when fail to add to vDPA bus
*/
-int vdpa_register_device(struct vdpa_device *vdev)
+int vdpa_register_device(struct vdpa_device *vdev, int nvqs)
{
+ vdev->nvqs = nvqs;
+
return device_add(>dev);
}
EXPORT_SYMBOL_GPL(vdpa_register_device);
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
index 6a90fdb9cbfc..b129cb4dd013 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
@@ -357,7 +357,7 @@ static struct vdpasim *vdpasim_create(void)
else
ops = _net_config_ops;
- vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops,
VDPASIM_VQ_NUM);
+ vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops);
if (!vdpasim)
goto err_alloc;
@@ -393,7 +393,7 @@ static struct vdpasim *vdpasim_create(void)
[PATCH V3 16/19] virtio-pci: introduce modern device module
Signed-off-by: Jason Wang
---
drivers/virtio/Kconfig | 10 +-
drivers/virtio/Makefile| 1 +
drivers/virtio/virtio_pci_common.h | 27 +-
drivers/virtio/virtio_pci_modern.c | 617 -
drivers/virtio/virtio_pci_modern_dev.c | 599
include/linux/virtio_pci_modern.h | 111 +
6 files changed, 721 insertions(+), 644 deletions(-)
create mode 100644 drivers/virtio/virtio_pci_modern_dev.c
create mode 100644 include/linux/virtio_pci_modern.h
diff --git a/drivers/virtio/Kconfig b/drivers/virtio/Kconfig
index 7b41130d3f35..6b9b81f4b8c2 100644
--- a/drivers/virtio/Kconfig
+++ b/drivers/virtio/Kconfig
@@ -12,6 +12,14 @@ config ARCH_HAS_RESTRICTED_VIRTIO_MEMORY_ACCESS
This option is selected if the architecture may need to enforce
VIRTIO_F_ACCESS_PLATFORM
+config VIRTIO_PCI_MODERN
+ tristate "Modern Virtio PCI Device"
+ depends on PCI
+ help
+ Modern PCI device implementation. This module implements the
+ basic probe and control for devices which are based on modern
+ PCI device with possible vendor specific extensions.
+
menuconfig VIRTIO_MENU
bool "Virtio drivers"
default y
@@ -20,7 +28,7 @@ if VIRTIO_MENU
config VIRTIO_PCI
tristate "PCI driver for virtio devices"
- depends on PCI
+ depends on VIRTIO_PCI_MODERN
select VIRTIO
help
This driver provides support for virtio based paravirtual device
diff --git a/drivers/virtio/Makefile b/drivers/virtio/Makefile
index 591e6f72aa54..f097578aaa8f 100644
--- a/drivers/virtio/Makefile
+++ b/drivers/virtio/Makefile
@@ -1,5 +1,6 @@
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_VIRTIO) += virtio.o virtio_ring.o
+obj-$(CONFIG_VIRTIO_PCI_MODERN) += virtio_pci_modern_dev.o
obj-$(CONFIG_VIRTIO_MMIO) += virtio_mmio.o
obj-$(CONFIG_VIRTIO_PCI) += virtio_pci.o
virtio_pci-y := virtio_pci_modern.o virtio_pci_common.o
diff --git a/drivers/virtio/virtio_pci_common.h
b/drivers/virtio/virtio_pci_common.h
index f35ff5b6b467..beec047a8f8d 100644
--- a/drivers/virtio/virtio_pci_common.h
+++ b/drivers/virtio/virtio_pci_common.h
@@ -25,6 +25,7 @@
#include
#include
#include
+#include
#include
#include
@@ -39,32 +40,6 @@ struct virtio_pci_vq_info {
unsigned msix_vector;
};
-struct virtio_pci_modern_device {
- struct pci_dev *pci_dev;
-
- struct virtio_pci_common_cfg __iomem *common;
- /* Device-specific data (non-legacy mode) */
- void __iomem *device;
- /* Base of vq notifications (non-legacy mode). */
- void __iomem *notify_base;
- /* Where to read and clear interrupt */
- u8 __iomem *isr;
-
- /* So we can sanity-check accesses. */
- size_t notify_len;
- size_t device_len;
-
- /* Capability for when we need to map notifications per-vq. */
- int notify_map_cap;
-
- /* Multiply queue_notify_off by this value. (non-legacy mode). */
- u32 notify_offset_multiplier;
-
- int modern_bars;
-
- struct virtio_device_id id;
-};
-
/* Our device structure */
struct virtio_pci_device {
struct virtio_device vdev;
diff --git a/drivers/virtio/virtio_pci_modern.c
b/drivers/virtio/virtio_pci_modern.c
index a5e3a5e40323..fbd4ebc00eb6 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -19,158 +19,6 @@
#define VIRTIO_RING_NO_LEGACY
#include "virtio_pci_common.h"
-/*
- * Type-safe wrappers for io accesses.
- * Use these to enforce at compile time the following spec requirement:
- *
- * The driver MUST access each field using the “natural” access
- * method, i.e. 32-bit accesses for 32-bit fields, 16-bit accesses
- * for 16-bit fields and 8-bit accesses for 8-bit fields.
- */
-static inline u8 vp_ioread8(const u8 __iomem *addr)
-{
- return ioread8(addr);
-}
-static inline u16 vp_ioread16 (const __le16 __iomem *addr)
-{
- return ioread16(addr);
-}
-
-static inline u32 vp_ioread32(const __le32 __iomem *addr)
-{
- return ioread32(addr);
-}
-
-static inline void vp_iowrite8(u8 value, u8 __iomem *addr)
-{
- iowrite8(value, addr);
-}
-
-static inline void vp_iowrite16(u16 value, __le16 __iomem *addr)
-{
- iowrite16(value, addr);
-}
-
-static inline void vp_iowrite32(u32 value, __le32 __iomem *addr)
-{
- iowrite32(value, addr);
-}
-
-static void vp_iowrite64_twopart(u64 val,
-__le32 __iomem *lo, __le32 __iomem *hi)
-{
- vp_iowrite32((u32)val, lo);
- vp_iowrite32(val >> 32, hi);
-}
-
-/*
- * vp_modern_map_capability - map a part of virtio pci capability
- * @mdev: the modern virtio-pci device
- * @off: offset of the capability
- * @minlen: minimal length of the capability
- * @align: align requirement
- * @start: start from the capability
- * @size: map size
- * @len: the length that is actually mapped
- *
- * Returns the io address of
[PATCH V3 15/19] virito-pci-modern: rename map_capability() to vp_modern_map_capability()
To ease the split, map_capability() was renamed to
vp_modern_map_capability(). While at it, add the comments for the
arguments and switch to use virtio_pci_modern_device as the first
parameter.
Signed-off-by: Jason Wang
---
drivers/virtio/virtio_pci_modern.c | 46 +++---
1 file changed, 30 insertions(+), 16 deletions(-)
diff --git a/drivers/virtio/virtio_pci_modern.c
b/drivers/virtio/virtio_pci_modern.c
index 217573f2588d..a5e3a5e40323 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -63,12 +63,25 @@ static void vp_iowrite64_twopart(u64 val,
vp_iowrite32(val >> 32, hi);
}
-static void __iomem *map_capability(struct pci_dev *dev, int off,
- size_t minlen,
- u32 align,
- u32 start, u32 size,
- size_t *len)
+/*
+ * vp_modern_map_capability - map a part of virtio pci capability
+ * @mdev: the modern virtio-pci device
+ * @off: offset of the capability
+ * @minlen: minimal length of the capability
+ * @align: align requirement
+ * @start: start from the capability
+ * @size: map size
+ * @len: the length that is actually mapped
+ *
+ * Returns the io address of for the part of the capability
+ */
+void __iomem *vp_modern_map_capability(struct virtio_pci_modern_device *mdev,
int off,
+ size_t minlen,
+ u32 align,
+ u32 start, u32 size,
+ size_t *len)
{
+ struct pci_dev *dev = mdev->pci_dev;
u8 bar;
u32 offset, length;
void __iomem *p;
@@ -582,7 +595,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device
*vp_dev,
vq->priv = (void __force *)mdev->notify_base +
off * mdev->notify_offset_multiplier;
} else {
- vq->priv = (void __force *)map_capability(mdev->pci_dev,
+ vq->priv = (void __force *)vp_modern_map_capability(mdev,
mdev->notify_map_cap,
2, 2,
off *
mdev->notify_offset_multiplier, 2,
NULL);
@@ -956,15 +969,15 @@ static int vp_modern_probe(struct
virtio_pci_modern_device *mdev)
return err;
err = -EINVAL;
- mdev->common = map_capability(pci_dev, common,
+ mdev->common = vp_modern_map_capability(mdev, common,
sizeof(struct virtio_pci_common_cfg), 4,
0, sizeof(struct virtio_pci_common_cfg),
NULL);
if (!mdev->common)
goto err_map_common;
- mdev->isr = map_capability(pci_dev, isr, sizeof(u8), 1,
- 0, 1,
- NULL);
+ mdev->isr = vp_modern_map_capability(mdev, isr, sizeof(u8), 1,
+0, 1,
+NULL);
if (!mdev->isr)
goto err_map_isr;
@@ -989,9 +1002,10 @@ static int vp_modern_probe(struct
virtio_pci_modern_device *mdev)
* Otherwise, map each VQ individually later.
*/
if ((u64)notify_length + (notify_offset % PAGE_SIZE) <= PAGE_SIZE) {
- mdev->notify_base = map_capability(pci_dev, notify, 2, 2,
- 0, notify_length,
- >notify_len);
+ mdev->notify_base = vp_modern_map_capability(mdev, notify,
+2, 2,
+0, notify_length,
+>notify_len);
if (!mdev->notify_base)
goto err_map_notify;
} else {
@@ -1002,9 +1016,9 @@ static int vp_modern_probe(struct
virtio_pci_modern_device *mdev)
* is more than enough for all existing devices.
*/
if (device) {
- mdev->device = map_capability(pci_dev, device, 0, 4,
- 0, PAGE_SIZE,
- >device_len);
+ mdev->device = vp_modern_map_capability(mdev, device, 0, 4,
+ 0, PAGE_SIZE,
+ >device_len);
if (!mdev->device)
goto err_map_device;
}
--
2.25.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 14/19] virtio-pci-modern: introduce helper to get notification offset
This patch introduces help to get notification offset of modern device.
Signed-off-by: Jason Wang
---
drivers/virtio/virtio_pci_modern.c | 21 -
1 file changed, 16 insertions(+), 5 deletions(-)
diff --git a/drivers/virtio/virtio_pci_modern.c
b/drivers/virtio/virtio_pci_modern.c
index bccad1329871..217573f2588d 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -303,6 +303,21 @@ static u16 vp_modern_get_num_queues(struct
virtio_pci_modern_device *mdev)
return vp_ioread16(>common->num_queues);
}
+/*
+ * vp_modern_get_queue_notify_off - get notification offset for a virtqueue
+ * @mdev: the modern virtio-pci device
+ * @index: the queue index
+ *
+ * Returns the notification offset for a virtqueue
+ */
+static u16 vp_modern_get_queue_notify_off(struct virtio_pci_modern_device
*mdev,
+ u16 index)
+{
+ vp_iowrite16(index, >common->queue_select);
+
+ return vp_ioread16(>common->queue_notify_off);
+}
+
/* virtio config->finalize_features() implementation */
static int vp_finalize_features(struct virtio_device *vdev)
{
@@ -516,7 +531,6 @@ static struct virtqueue *setup_vq(struct virtio_pci_device
*vp_dev,
{
struct virtio_pci_modern_device *mdev = _dev->mdev;
- struct virtio_pci_common_cfg __iomem *cfg = mdev->common;
struct virtqueue *vq;
u16 num, off;
int err;
@@ -524,9 +538,6 @@ static struct virtqueue *setup_vq(struct virtio_pci_device
*vp_dev,
if (index >= vp_modern_get_num_queues(mdev))
return ERR_PTR(-ENOENT);
- /* Select the queue we're interested in */
- vp_iowrite16(index, >queue_select);
-
/* Check if queue is either not available or already active. */
num = vp_modern_get_queue_size(mdev, index);
if (!num || vp_modern_get_queue_enable(mdev, index))
@@ -538,7 +549,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device
*vp_dev,
}
/* get offset of notification word for this vq */
- off = vp_ioread16(>queue_notify_off);
+ off = vp_modern_get_queue_notify_off(mdev, index);
info->msix_vector = msix_vec;
--
2.25.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 13/19] virtio-pci-modern: introduce helper for getting queue nums
This patch introduces helper for getting queue num of modern device.
Signed-off-by: Jason Wang
---
drivers/virtio/virtio_pci_modern.c | 13 -
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/drivers/virtio/virtio_pci_modern.c
b/drivers/virtio/virtio_pci_modern.c
index 7a89226135af..bccad1329871 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -292,6 +292,17 @@ static u16 vp_modern_get_queue_size(struct
virtio_pci_modern_device *mdev,
}
+/*
+ * vp_modern_get_num_queues - get the number of virtqueues
+ * @mdev: the modern virtio-pci device
+ *
+ * Returns the number of virtqueues
+ */
+static u16 vp_modern_get_num_queues(struct virtio_pci_modern_device *mdev)
+{
+ return vp_ioread16(>common->num_queues);
+}
+
/* virtio config->finalize_features() implementation */
static int vp_finalize_features(struct virtio_device *vdev)
{
@@ -510,7 +521,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device
*vp_dev,
u16 num, off;
int err;
- if (index >= vp_ioread16(>num_queues))
+ if (index >= vp_modern_get_num_queues(mdev))
return ERR_PTR(-ENOENT);
/* Select the queue we're interested in */
--
2.25.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 12/19] virtio-pci-modern: introduce helper for setting/geting queue size
This patch introduces helper for setting/getting queue size for modern
device.
Signed-off-by: Jason Wang
---
drivers/virtio/virtio_pci_modern.c | 34 --
1 file changed, 32 insertions(+), 2 deletions(-)
diff --git a/drivers/virtio/virtio_pci_modern.c
b/drivers/virtio/virtio_pci_modern.c
index 0e62820b83ff..7a89226135af 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -262,6 +262,36 @@ static bool vp_modern_get_queue_enable(struct
virtio_pci_modern_device *mdev,
return vp_ioread16(>common->queue_enable);
}
+/*
+ * vp_modern_set_queue_size - set size for a virtqueue
+ * @mdev: the modern virtio-pci device
+ * @index: the queue index
+ * @size: the size of the virtqueue
+ */
+static void vp_modern_set_queue_size(struct virtio_pci_modern_device *mdev,
+u16 index, u16 size)
+{
+ vp_iowrite16(index, >common->queue_select);
+ vp_iowrite16(size, >common->queue_size);
+
+}
+
+/*
+ * vp_modern_get_queue_size - get size for a virtqueue
+ * @mdev: the modern virtio-pci device
+ * @index: the queue index
+ *
+ * Returns the size of the virtqueue
+ */
+static u16 vp_modern_get_queue_size(struct virtio_pci_modern_device *mdev,
+ u16 index)
+{
+ vp_iowrite16(index, >common->queue_select);
+
+ return vp_ioread16(>common->queue_size);
+
+}
+
/* virtio config->finalize_features() implementation */
static int vp_finalize_features(struct virtio_device *vdev)
{
@@ -487,7 +517,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device
*vp_dev,
vp_iowrite16(index, >queue_select);
/* Check if queue is either not available or already active. */
- num = vp_ioread16(>queue_size);
+ num = vp_modern_get_queue_size(mdev, index);
if (!num || vp_modern_get_queue_enable(mdev, index))
return ERR_PTR(-ENOENT);
@@ -510,7 +540,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device
*vp_dev,
return ERR_PTR(-ENOMEM);
/* activate the queue */
- vp_iowrite16(virtqueue_get_vring_size(vq), >queue_size);
+ vp_modern_set_queue_size(mdev, index, virtqueue_get_vring_size(vq));
vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
virtqueue_get_avail_addr(vq),
virtqueue_get_used_addr(vq));
--
2.25.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 11/19] virtio-pci-modern: introduce helper to set/get queue_enable
This patch introduces a helper to set/get queue_enable for modern device.
Signed-off-by: Jason Wang
---
drivers/virtio/virtio_pci_modern.c | 37 +-
1 file changed, 31 insertions(+), 6 deletions(-)
diff --git a/drivers/virtio/virtio_pci_modern.c
b/drivers/virtio/virtio_pci_modern.c
index 05b21e18f46c..0e62820b83ff 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -234,6 +234,34 @@ static void vp_modern_queue_address(struct
virtio_pci_modern_device *mdev,
>queue_used_hi);
}
+/*
+ * vp_modern_set_queue_enable - enable a virtqueue
+ * @mdev: the modern virtio-pci device
+ * @index: the queue index
+ * @enable: whether the virtqueue is enable or not
+ */
+static void vp_modern_set_queue_enable(struct virtio_pci_modern_device *mdev,
+ u16 index, bool enable)
+{
+ vp_iowrite16(index, >common->queue_select);
+ vp_iowrite16(enable, >common->queue_enable);
+}
+
+/*
+ * vp_modern_get_queue_enable - enable a virtqueue
+ * @mdev: the modern virtio-pci device
+ * @index: the queue index
+ *
+ * Returns whether a virtqueue is enabled or not
+ */
+static bool vp_modern_get_queue_enable(struct virtio_pci_modern_device *mdev,
+ u16 index)
+{
+ vp_iowrite16(index, >common->queue_select);
+
+ return vp_ioread16(>common->queue_enable);
+}
+
/* virtio config->finalize_features() implementation */
static int vp_finalize_features(struct virtio_device *vdev)
{
@@ -460,7 +488,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device
*vp_dev,
/* Check if queue is either not available or already active. */
num = vp_ioread16(>queue_size);
- if (!num || vp_ioread16(>queue_enable))
+ if (!num || vp_modern_get_queue_enable(mdev, index))
return ERR_PTR(-ENOENT);
if (num & (num - 1)) {
@@ -538,7 +566,6 @@ static int vp_modern_find_vqs(struct virtio_device *vdev,
unsigned nvqs,
struct irq_affinity *desc)
{
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
- struct virtio_pci_common_cfg __iomem *cfg = vp_dev->mdev.common;
struct virtqueue *vq;
int rc = vp_find_vqs(vdev, nvqs, vqs, callbacks, names, ctx, desc);
@@ -548,10 +575,8 @@ static int vp_modern_find_vqs(struct virtio_device *vdev,
unsigned nvqs,
/* Select and activate all queues. Has to be done last: once we do
* this, there's no way to go back except reset.
*/
- list_for_each_entry(vq, >vqs, list) {
- vp_iowrite16(vq->index, >queue_select);
- vp_iowrite16(1, >queue_enable);
- }
+ list_for_each_entry(vq, >vqs, list)
+ vp_modern_set_queue_enable(_dev->mdev, vq->index, true);
return 0;
}
--
2.25.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 10/19] virtio-pci-modern: introduce vp_modern_queue_address()
This patch introduce a helper to set virtqueue address for modern address.
Signed-off-by: Jason Wang
---
drivers/virtio/virtio_pci_modern.c | 33 --
1 file changed, 27 insertions(+), 6 deletions(-)
diff --git a/drivers/virtio/virtio_pci_modern.c
b/drivers/virtio/virtio_pci_modern.c
index 05cd409c0731..05b21e18f46c 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -210,6 +210,30 @@ static u16 vp_modern_queue_vector(struct
virtio_pci_modern_device *mdev,
return vp_ioread16(>queue_msix_vector);
}
+/*
+ * vp_modern_queue_address - set the virtqueue address
+ * @mdev: the modern virtio-pci device
+ * @index: the queue index
+ * @desc_addr: address of the descriptor area
+ * @driver_addr: address of the driver area
+ * @device_addr: address of the device area
+ */
+static void vp_modern_queue_address(struct virtio_pci_modern_device *mdev,
+ u16 index, u64 desc_addr, u64 driver_addr,
+ u64 device_addr)
+{
+ struct virtio_pci_common_cfg __iomem *cfg = mdev->common;
+
+ vp_iowrite16(index, >queue_select);
+
+ vp_iowrite64_twopart(desc_addr, >queue_desc_lo,
+>queue_desc_hi);
+ vp_iowrite64_twopart(driver_addr, >queue_avail_lo,
+>queue_avail_hi);
+ vp_iowrite64_twopart(device_addr, >queue_used_lo,
+>queue_used_hi);
+}
+
/* virtio config->finalize_features() implementation */
static int vp_finalize_features(struct virtio_device *vdev)
{
@@ -459,12 +483,9 @@ static struct virtqueue *setup_vq(struct virtio_pci_device
*vp_dev,
/* activate the queue */
vp_iowrite16(virtqueue_get_vring_size(vq), >queue_size);
- vp_iowrite64_twopart(virtqueue_get_desc_addr(vq),
->queue_desc_lo, >queue_desc_hi);
- vp_iowrite64_twopart(virtqueue_get_avail_addr(vq),
->queue_avail_lo, >queue_avail_hi);
- vp_iowrite64_twopart(virtqueue_get_used_addr(vq),
->queue_used_lo, >queue_used_hi);
+ vp_modern_queue_address(mdev, index, virtqueue_get_desc_addr(vq),
+ virtqueue_get_avail_addr(vq),
+ virtqueue_get_used_addr(vq));
if (mdev->notify_base) {
/* offset should not wrap */
--
2.25.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 09/19] virtio-pci-modern: introduce vp_modern_set_queue_vector()
This patch introduces a helper to set virtqueue MSI vector.
Signed-off-by: Jason Wang
---
drivers/virtio/virtio_pci_modern.c | 35 --
1 file changed, 23 insertions(+), 12 deletions(-)
diff --git a/drivers/virtio/virtio_pci_modern.c
b/drivers/virtio/virtio_pci_modern.c
index a128e5814045..05cd409c0731 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -191,6 +191,25 @@ static void vp_modern_set_features(struct
virtio_pci_modern_device *mdev,
vp_iowrite32(features >> 32, >guest_feature);
}
+/*
+ * vp_modern_queue_vector - set the MSIX vector for a specific virtqueue
+ * @mdev: the modern virtio-pci device
+ * @index: queue index
+ * @vector: the config vector
+ *
+ * Returns the config vector read from the device
+ */
+static u16 vp_modern_queue_vector(struct virtio_pci_modern_device *mdev,
+ u16 index, u16 vector)
+{
+ struct virtio_pci_common_cfg __iomem *cfg = mdev->common;
+
+ vp_iowrite16(index, >queue_select);
+ vp_iowrite16(vector, >queue_msix_vector);
+ /* Flush the write out to device */
+ return vp_ioread16(>queue_msix_vector);
+}
+
/* virtio config->finalize_features() implementation */
static int vp_finalize_features(struct virtio_device *vdev)
{
@@ -474,8 +493,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device
*vp_dev,
}
if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
- vp_iowrite16(msix_vec, >queue_msix_vector);
- msix_vec = vp_ioread16(>queue_msix_vector);
+ msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
if (msix_vec == VIRTIO_MSI_NO_VECTOR) {
err = -EBUSY;
goto err_assign_vector;
@@ -522,17 +540,10 @@ static void del_vq(struct virtio_pci_vq_info *info)
struct virtqueue *vq = info->vq;
struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
struct virtio_pci_modern_device *mdev = _dev->mdev;
- struct virtio_pci_common_cfg __iomem *cfg = mdev->common;
-
- vp_iowrite16(vq->index, >queue_select);
-
- if (vp_dev->msix_enabled) {
- vp_iowrite16(VIRTIO_MSI_NO_VECTOR,
->queue_msix_vector);
- /* Flush the write out to device */
- vp_ioread16(>queue_msix_vector);
- }
+ if (vp_dev->msix_enabled)
+ vp_modern_queue_vector(mdev, vq->index,
+ VIRTIO_MSI_NO_VECTOR);
if (!mdev->notify_base)
pci_iounmap(mdev->pci_dev, (void __force __iomem *)vq->priv);
--
2.25.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 08/19] virtio-pci-modern: introduce vp_modern_generation()
This patch introduces vp_modern_generation() to get device generation.
Signed-off-by: Jason Wang
---
drivers/virtio/virtio_pci_modern.c | 17 ++---
1 file changed, 14 insertions(+), 3 deletions(-)
diff --git a/drivers/virtio/virtio_pci_modern.c
b/drivers/virtio/virtio_pci_modern.c
index cb14fc334a9c..a128e5814045 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -289,15 +289,26 @@ static void vp_set(struct virtio_device *vdev, unsigned
offset,
}
}
-static u32 vp_generation(struct virtio_device *vdev)
+/*
+ * vp_modern_generation - get the device genreation
+ * @mdev: the modern virtio-pci device
+ *
+ * Returns the genreation read from device
+ */
+static u32 vp_modern_generation(struct virtio_pci_modern_device *mdev)
{
- struct virtio_pci_device *vp_dev = to_vp_device(vdev);
- struct virtio_pci_modern_device *mdev = _dev->mdev;
struct virtio_pci_common_cfg __iomem *cfg = mdev->common;
return vp_ioread8(>config_generation);
}
+static u32 vp_generation(struct virtio_device *vdev)
+{
+ struct virtio_pci_device *vp_dev = to_vp_device(vdev);
+
+ return vp_modern_generation(_dev->mdev);
+}
+
/*
* vp_modern_get_status - get the device status
* @mdev: the modern virtio-pci device
--
2.25.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 07/19] virtio-pci-modern: introduce helpers for setting and getting features
This patch introduces helpers for setting and getting features.
Signed-off-by: Jason Wang
---
drivers/virtio/virtio_pci_modern.c | 43 +++---
1 file changed, 33 insertions(+), 10 deletions(-)
diff --git a/drivers/virtio/virtio_pci_modern.c
b/drivers/virtio/virtio_pci_modern.c
index ccde0a41209a..cb14fc334a9c 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -137,12 +137,16 @@ static void __iomem *map_capability(struct pci_dev *dev,
int off,
return p;
}
-/* virtio config->get_features() implementation */
-static u64 vp_get_features(struct virtio_device *vdev)
+/*
+ * vp_modern_get_features - get features from device
+ * @mdev: the modern virtio-pci device
+ *
+ * Returns the features read from the device
+ */
+static u64 vp_modern_get_features(struct virtio_pci_modern_device *mdev)
{
- struct virtio_pci_device *vp_dev = to_vp_device(vdev);
- struct virtio_pci_modern_device *mdev = _dev->mdev;
struct virtio_pci_common_cfg __iomem *cfg = mdev->common;
+
u64 features;
vp_iowrite32(0, >device_feature_select);
@@ -153,6 +157,14 @@ static u64 vp_get_features(struct virtio_device *vdev)
return features;
}
+/* virtio config->get_features() implementation */
+static u64 vp_get_features(struct virtio_device *vdev)
+{
+ struct virtio_pci_device *vp_dev = to_vp_device(vdev);
+
+ return vp_modern_get_features(_dev->mdev);
+}
+
static void vp_transport_features(struct virtio_device *vdev, u64 features)
{
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
@@ -163,12 +175,26 @@ static void vp_transport_features(struct virtio_device
*vdev, u64 features)
__virtio_set_bit(vdev, VIRTIO_F_SR_IOV);
}
+/*
+ * vp_modern_set_features - set features to device
+ * @mdev: the modern virtio-pci device
+ * @features: the features set to device
+ */
+static void vp_modern_set_features(struct virtio_pci_modern_device *mdev,
+ u64 features)
+{
+ struct virtio_pci_common_cfg __iomem *cfg = mdev->common;
+
+ vp_iowrite32(0, >guest_feature_select);
+ vp_iowrite32((u32)features, >guest_feature);
+ vp_iowrite32(1, >guest_feature_select);
+ vp_iowrite32(features >> 32, >guest_feature);
+}
+
/* virtio config->finalize_features() implementation */
static int vp_finalize_features(struct virtio_device *vdev)
{
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
- struct virtio_pci_modern_device *mdev = _dev->mdev;
- struct virtio_pci_common_cfg __iomem *cfg = mdev->common;
u64 features = vdev->features;
/* Give virtio_ring a chance to accept features. */
@@ -183,10 +209,7 @@ static int vp_finalize_features(struct virtio_device *vdev)
return -EINVAL;
}
- vp_iowrite32(0, >guest_feature_select);
- vp_iowrite32((u32)vdev->features, >guest_feature);
- vp_iowrite32(1, >guest_feature_select);
- vp_iowrite32(vdev->features >> 32, >guest_feature);
+ vp_modern_set_features(_dev->mdev, vdev->features);
return 0;
}
--
2.25.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 06/19] virtio-pci-modern: introduce helpers for setting and getting status
This patch introduces helpers to allow set and get device status.
Signed-off-by: Jason Wang
---
drivers/virtio/virtio_pci_modern.c | 37 +++---
1 file changed, 29 insertions(+), 8 deletions(-)
diff --git a/drivers/virtio/virtio_pci_modern.c
b/drivers/virtio/virtio_pci_modern.c
index 2e37bfc89655..ccde0a41209a 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -275,41 +275,62 @@ static u32 vp_generation(struct virtio_device *vdev)
return vp_ioread8(>config_generation);
}
+/*
+ * vp_modern_get_status - get the device status
+ * @mdev: the modern virtio-pci device
+ *
+ * Returns the status read from device
+ */
+static u8 vp_modern_get_status(struct virtio_pci_modern_device *mdev)
+{
+ struct virtio_pci_common_cfg __iomem *cfg = mdev->common;
+
+ return vp_ioread8(>device_status);
+}
+
/* config->{get,set}_status() implementations */
static u8 vp_get_status(struct virtio_device *vdev)
{
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
- struct virtio_pci_modern_device *mdev = _dev->mdev;
+
+ return vp_modern_get_status(_dev->mdev);
+}
+
+/*
+ * vp_modern_set_status - set status to device
+ * @mdev: the modern virtio-pci device
+ * @status: the status set to device
+ */
+static void vp_modern_set_status(struct virtio_pci_modern_device *mdev,
+u8 status)
+{
struct virtio_pci_common_cfg __iomem *cfg = mdev->common;
- return vp_ioread8(>device_status);
+ vp_iowrite8(status, >device_status);
}
static void vp_set_status(struct virtio_device *vdev, u8 status)
{
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
- struct virtio_pci_modern_device *mdev = _dev->mdev;
- struct virtio_pci_common_cfg __iomem *cfg = mdev->common;
/* We should never be setting status to 0. */
BUG_ON(status == 0);
- vp_iowrite8(status, >device_status);
+ vp_modern_set_status(_dev->mdev, status);
}
static void vp_reset(struct virtio_device *vdev)
{
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
struct virtio_pci_modern_device *mdev = _dev->mdev;
- struct virtio_pci_common_cfg __iomem *cfg = mdev->common;
/* 0 status means a reset. */
- vp_iowrite8(0, >device_status);
+ vp_modern_set_status(mdev, 0);
/* After writing 0 to device_status, the driver MUST wait for a read of
* device_status to return 0 before reinitializing the device.
* This will flush out the status write, and flush in device writes,
* including MSI-X interrupts, if any.
*/
- while (vp_ioread8(>device_status))
+ while (vp_modern_get_status(mdev))
msleep(1);
/* Flush pending VQ/configuration callbacks. */
vp_synchronize_vectors(vdev);
--
2.25.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 05/19] virtio-pci-modern: introduce helper to set config vector
This patch introduces vp_modern_config_vector() for setting config
vector.
Signed-off-by: Jason Wang
---
drivers/virtio/virtio_pci_modern.c | 16 ++--
1 file changed, 14 insertions(+), 2 deletions(-)
diff --git a/drivers/virtio/virtio_pci_modern.c
b/drivers/virtio/virtio_pci_modern.c
index 4be9afad547e..2e37bfc89655 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -315,9 +315,16 @@ static void vp_reset(struct virtio_device *vdev)
vp_synchronize_vectors(vdev);
}
-static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
+/*
+ * vp_modern_config_vector - set the vector for config interrupt
+ * @mdev: the modern virtio-pci device
+ * @vector: the config vector
+ *
+ * Returns the config vector read from the device
+ */
+static u16 vp_modern_config_vector(struct virtio_pci_modern_device *mdev,
+ u16 vector)
{
- struct virtio_pci_modern_device *mdev = _dev->mdev;
struct virtio_pci_common_cfg __iomem *cfg = mdev->common;
/* Setup the vector used for configuration events */
@@ -327,6 +334,11 @@ static u16 vp_config_vector(struct virtio_pci_device
*vp_dev, u16 vector)
return vp_ioread16(>msix_config);
}
+static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
+{
+ return vp_modern_config_vector(_dev->mdev, vector);
+}
+
static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
struct virtio_pci_vq_info *info,
unsigned index,
--
2.25.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 04/19] virtio-pci-modern: introduce vp_modern_remove()
This patch introduces vp_modern_remove() doing device resources
cleanup to make it can be used.
Signed-off-by: Jason Wang
---
drivers/virtio/virtio_pci_modern.c | 14 --
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/drivers/virtio/virtio_pci_modern.c
b/drivers/virtio/virtio_pci_modern.c
index 5d2d2ae0dfdb..4be9afad547e 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -873,9 +873,12 @@ int virtio_pci_modern_probe(struct virtio_pci_device
*vp_dev)
return 0;
}
-void virtio_pci_modern_remove(struct virtio_pci_device *vp_dev)
+/*
+ * vp_modern_probe: remove and cleanup the modern virtio pci device
+ * @mdev: the modern virtio-pci device
+ */
+static void vp_modern_remove(struct virtio_pci_modern_device *mdev)
{
- struct virtio_pci_modern_device *mdev = _dev->mdev;
struct pci_dev *pci_dev = mdev->pci_dev;
if (mdev->device)
@@ -886,3 +889,10 @@ void virtio_pci_modern_remove(struct virtio_pci_device
*vp_dev)
pci_iounmap(pci_dev, mdev->common);
pci_release_selected_regions(pci_dev, mdev->modern_bars);
}
+
+void virtio_pci_modern_remove(struct virtio_pci_device *vp_dev)
+{
+ struct virtio_pci_modern_device *mdev = _dev->mdev;
+
+ vp_modern_remove(mdev);
+}
--
2.25.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 03/19] virtio-pci-modern: factor out modern device initialization logic
This patch factors out the modern device initialization logic into a
helper. Note that it still depends on the caller to enable pci device
which allows the caller to use e.g devres.
Signed-off-by: Jason Wang
---
drivers/virtio/virtio_pci_modern.c | 50 +-
1 file changed, 36 insertions(+), 14 deletions(-)
diff --git a/drivers/virtio/virtio_pci_modern.c
b/drivers/virtio/virtio_pci_modern.c
index 524490a94ca4..5d2d2ae0dfdb 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -703,11 +703,16 @@ static inline void check_offsets(void)
offsetof(struct virtio_pci_common_cfg, queue_used_hi));
}
-/* the PCI probing function */
-int virtio_pci_modern_probe(struct virtio_pci_device *vp_dev)
+/*
+ * vp_modern_probe: probe the modern virtio pci device, note that the
+ * caller is required to enable PCI device before calling this function.
+ * @mdev: the modern virtio-pci device
+ *
+ * Return 0 on succeed otherwise fail
+ */
+static int vp_modern_probe(struct virtio_pci_modern_device *mdev)
{
- struct virtio_pci_modern_device *mdev = _dev->mdev;
- struct pci_dev *pci_dev = vp_dev->pci_dev;
+ struct pci_dev *pci_dev = mdev->pci_dev;
int err, common, isr, notify, device;
u32 notify_length;
u32 notify_offset;
@@ -826,18 +831,8 @@ int virtio_pci_modern_probe(struct virtio_pci_device
*vp_dev)
>device_len);
if (!mdev->device)
goto err_map_device;
-
- vp_dev->vdev.config = _pci_config_ops;
- } else {
- vp_dev->vdev.config = _pci_config_nodev_ops;
}
- vp_dev->config_vector = vp_config_vector;
- vp_dev->setup_vq = setup_vq;
- vp_dev->del_vq = del_vq;
- vp_dev->isr = mdev->isr;
- vp_dev->vdev.id = mdev->id;
-
return 0;
err_map_device:
@@ -851,6 +846,33 @@ int virtio_pci_modern_probe(struct virtio_pci_device
*vp_dev)
return err;
}
+/* the PCI probing function */
+int virtio_pci_modern_probe(struct virtio_pci_device *vp_dev)
+{
+ struct virtio_pci_modern_device *mdev = _dev->mdev;
+ struct pci_dev *pci_dev = vp_dev->pci_dev;
+ int err;
+
+ mdev->pci_dev = pci_dev;
+
+ err = vp_modern_probe(mdev);
+ if (err)
+ return err;
+
+ if (mdev->device)
+ vp_dev->vdev.config = _pci_config_ops;
+ else
+ vp_dev->vdev.config = _pci_config_nodev_ops;
+
+ vp_dev->config_vector = vp_config_vector;
+ vp_dev->setup_vq = setup_vq;
+ vp_dev->del_vq = del_vq;
+ vp_dev->isr = mdev->isr;
+ vp_dev->vdev.id = mdev->id;
+
+ return 0;
+}
+
void virtio_pci_modern_remove(struct virtio_pci_device *vp_dev)
{
struct virtio_pci_modern_device *mdev = _dev->mdev;
--
2.25.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 02/19] virtio-pci: split out modern device
This patch splits out the virtio-pci modern device only attributes
into another structure. While at it, a dedicated probe method for
modern only attributes is introduced. This may help for split the
logic into a dedicated module.
Signed-off-by: Jason Wang
---
drivers/virtio/virtio_pci_common.h | 25 +++--
drivers/virtio/virtio_pci_modern.c | 159 -
2 files changed, 105 insertions(+), 79 deletions(-)
diff --git a/drivers/virtio/virtio_pci_common.h
b/drivers/virtio/virtio_pci_common.h
index b2f0eb4067cb..f35ff5b6b467 100644
--- a/drivers/virtio/virtio_pci_common.h
+++ b/drivers/virtio/virtio_pci_common.h
@@ -39,22 +39,16 @@ struct virtio_pci_vq_info {
unsigned msix_vector;
};
-/* Our device structure */
-struct virtio_pci_device {
- struct virtio_device vdev;
+struct virtio_pci_modern_device {
struct pci_dev *pci_dev;
- /* In legacy mode, these two point to within ->legacy. */
- /* Where to read and clear interrupt */
- u8 __iomem *isr;
-
- /* Modern only fields */
- /* The IO mapping for the PCI config space (non-legacy mode) */
struct virtio_pci_common_cfg __iomem *common;
/* Device-specific data (non-legacy mode) */
void __iomem *device;
/* Base of vq notifications (non-legacy mode). */
void __iomem *notify_base;
+ /* Where to read and clear interrupt */
+ u8 __iomem *isr;
/* So we can sanity-check accesses. */
size_t notify_len;
@@ -68,6 +62,19 @@ struct virtio_pci_device {
int modern_bars;
+ struct virtio_device_id id;
+};
+
+/* Our device structure */
+struct virtio_pci_device {
+ struct virtio_device vdev;
+ struct pci_dev *pci_dev;
+ struct virtio_pci_modern_device mdev;
+
+ /* In legacy mode, these two point to within ->legacy. */
+ /* Where to read and clear interrupt */
+ u8 __iomem *isr;
+
/* Legacy only field */
/* the IO mapping for the PCI config space */
void __iomem *ioaddr;
diff --git a/drivers/virtio/virtio_pci_modern.c
b/drivers/virtio/virtio_pci_modern.c
index df1481fd400c..524490a94ca4 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -141,7 +141,8 @@ static void __iomem *map_capability(struct pci_dev *dev,
int off,
static u64 vp_get_features(struct virtio_device *vdev)
{
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
- struct virtio_pci_common_cfg __iomem *cfg = vp_dev->common;
+ struct virtio_pci_modern_device *mdev = _dev->mdev;
+ struct virtio_pci_common_cfg __iomem *cfg = mdev->common;
u64 features;
vp_iowrite32(0, >device_feature_select);
@@ -166,7 +167,8 @@ static void vp_transport_features(struct virtio_device
*vdev, u64 features)
static int vp_finalize_features(struct virtio_device *vdev)
{
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
- struct virtio_pci_common_cfg __iomem *cfg = vp_dev->common;
+ struct virtio_pci_modern_device *mdev = _dev->mdev;
+ struct virtio_pci_common_cfg __iomem *cfg = mdev->common;
u64 features = vdev->features;
/* Give virtio_ring a chance to accept features. */
@@ -194,12 +196,13 @@ static void vp_get(struct virtio_device *vdev, unsigned
offset,
void *buf, unsigned len)
{
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
- void __iomem *device = vp_dev->device;
+ struct virtio_pci_modern_device *mdev = _dev->mdev;
+ void __iomem *device = mdev->device;
u8 b;
__le16 w;
__le32 l;
- BUG_ON(offset + len > vp_dev->device_len);
+ BUG_ON(offset + len > mdev->device_len);
switch (len) {
case 1:
@@ -231,12 +234,13 @@ static void vp_set(struct virtio_device *vdev, unsigned
offset,
const void *buf, unsigned len)
{
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
- void __iomem *device = vp_dev->device;
+ struct virtio_pci_modern_device *mdev = _dev->mdev;
+ void __iomem *device = mdev->device;
u8 b;
__le16 w;
__le32 l;
- BUG_ON(offset + len > vp_dev->device_len);
+ BUG_ON(offset + len > mdev->device_len);
switch (len) {
case 1:
@@ -265,7 +269,8 @@ static void vp_set(struct virtio_device *vdev, unsigned
offset,
static u32 vp_generation(struct virtio_device *vdev)
{
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
- struct virtio_pci_common_cfg __iomem *cfg = vp_dev->common;
+ struct virtio_pci_modern_device *mdev = _dev->mdev;
+ struct virtio_pci_common_cfg __iomem *cfg = mdev->common;
return vp_ioread8(>config_generation);
}
@@ -274,7 +279,8 @@ static u32 vp_generation(struct virtio_device *vdev)
static u8 vp_get_status(struct virtio_device *vdev)
{
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
-
[PATCH V3 00/19] vDPA driver for virtio-pci device
Hi all: This series tries to implement a vDPA driver for virtio-pci device which will bridge between vDPA bus and virtio-pci device. This could be used for future feature prototyping and testing. Please review Changes since V2: - tweak config prompt - switch from 'cb' to 'config_cb' for vp_vdpa config interrupt - use a macro for vp_vdpa msix name length Changes since V1: - don't try to use devres for virtio-pci core - tweak the commit log - split the patches furtherly to ease the reviewing Changes since RFC: - Split common codes from virito-pci and share it with vDPA driver - Use dynamic id in order to be less confusing with virtio-pci driver - No feature whitelist, supporting any features (mq, config etc) Thanks Jason Wang (19): virtio-pci: do not access iomem via struct virtio_pci_device directly virtio-pci: split out modern device virtio-pci-modern: factor out modern device initialization logic virtio-pci-modern: introduce vp_modern_remove() virtio-pci-modern: introduce helper to set config vector virtio-pci-modern: introduce helpers for setting and getting status virtio-pci-modern: introduce helpers for setting and getting features virtio-pci-modern: introduce vp_modern_generation() virtio-pci-modern: introduce vp_modern_set_queue_vector() virtio-pci-modern: introduce vp_modern_queue_address() virtio-pci-modern: introduce helper to set/get queue_enable virtio-pci-modern: introduce helper for setting/geting queue size virtio-pci-modern: introduce helper for getting queue nums virtio-pci-modern: introduce helper to get notification offset virito-pci-modern: rename map_capability() to vp_modern_map_capability() virtio-pci: introduce modern device module vdpa: set the virtqueue num during register virtio_vdpa: don't warn when fail to disable vq vdpa: introduce virtio pci driver drivers/vdpa/Kconfig | 6 + drivers/vdpa/Makefile | 1 + drivers/vdpa/ifcvf/ifcvf_main.c| 5 +- drivers/vdpa/mlx5/net/mlx5_vnet.c | 5 +- drivers/vdpa/vdpa.c| 8 +- drivers/vdpa/vdpa_sim/vdpa_sim.c | 4 +- drivers/vdpa/virtio_pci/Makefile | 2 + drivers/vdpa/virtio_pci/vp_vdpa.c | 456 +++ drivers/virtio/Kconfig | 10 +- drivers/virtio/Makefile| 1 + drivers/virtio/virtio_pci_common.h | 22 +- drivers/virtio/virtio_pci_modern.c | 506 +++-- drivers/virtio/virtio_pci_modern_dev.c | 599 + drivers/virtio/virtio_vdpa.c | 3 +- include/linux/vdpa.h | 7 +- include/linux/virtio_pci_modern.h | 111 + 16 files changed, 1274 insertions(+), 472 deletions(-) create mode 100644 drivers/vdpa/virtio_pci/Makefile create mode 100644 drivers/vdpa/virtio_pci/vp_vdpa.c create mode 100644 drivers/virtio/virtio_pci_modern_dev.c create mode 100644 include/linux/virtio_pci_modern.h -- 2.25.1 ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 01/19] virtio-pci: do not access iomem via struct virtio_pci_device directly
Instead of accessing iomem via struct virito_pci_device directly,
tweak to call the io accessors through the iomem structure. This will
ease the splitting of modern virtio device logic.
Signed-off-by: Jason Wang
---
drivers/virtio/virtio_pci_modern.c | 76 ++
1 file changed, 46 insertions(+), 30 deletions(-)
diff --git a/drivers/virtio/virtio_pci_modern.c
b/drivers/virtio/virtio_pci_modern.c
index 3d6ae5a5e252..df1481fd400c 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -141,12 +141,13 @@ static void __iomem *map_capability(struct pci_dev *dev,
int off,
static u64 vp_get_features(struct virtio_device *vdev)
{
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
+ struct virtio_pci_common_cfg __iomem *cfg = vp_dev->common;
u64 features;
- vp_iowrite32(0, _dev->common->device_feature_select);
- features = vp_ioread32(_dev->common->device_feature);
- vp_iowrite32(1, _dev->common->device_feature_select);
- features |= ((u64)vp_ioread32(_dev->common->device_feature) << 32);
+ vp_iowrite32(0, >device_feature_select);
+ features = vp_ioread32(>device_feature);
+ vp_iowrite32(1, >device_feature_select);
+ features |= ((u64)vp_ioread32(>device_feature) << 32);
return features;
}
@@ -165,6 +166,7 @@ static void vp_transport_features(struct virtio_device
*vdev, u64 features)
static int vp_finalize_features(struct virtio_device *vdev)
{
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
+ struct virtio_pci_common_cfg __iomem *cfg = vp_dev->common;
u64 features = vdev->features;
/* Give virtio_ring a chance to accept features. */
@@ -179,10 +181,10 @@ static int vp_finalize_features(struct virtio_device
*vdev)
return -EINVAL;
}
- vp_iowrite32(0, _dev->common->guest_feature_select);
- vp_iowrite32((u32)vdev->features, _dev->common->guest_feature);
- vp_iowrite32(1, _dev->common->guest_feature_select);
- vp_iowrite32(vdev->features >> 32, _dev->common->guest_feature);
+ vp_iowrite32(0, >guest_feature_select);
+ vp_iowrite32((u32)vdev->features, >guest_feature);
+ vp_iowrite32(1, >guest_feature_select);
+ vp_iowrite32(vdev->features >> 32, >guest_feature);
return 0;
}
@@ -192,6 +194,7 @@ static void vp_get(struct virtio_device *vdev, unsigned
offset,
void *buf, unsigned len)
{
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
+ void __iomem *device = vp_dev->device;
u8 b;
__le16 w;
__le32 l;
@@ -200,21 +203,21 @@ static void vp_get(struct virtio_device *vdev, unsigned
offset,
switch (len) {
case 1:
- b = ioread8(vp_dev->device + offset);
+ b = ioread8(device + offset);
memcpy(buf, , sizeof b);
break;
case 2:
- w = cpu_to_le16(ioread16(vp_dev->device + offset));
+ w = cpu_to_le16(ioread16(device + offset));
memcpy(buf, , sizeof w);
break;
case 4:
- l = cpu_to_le32(ioread32(vp_dev->device + offset));
+ l = cpu_to_le32(ioread32(device + offset));
memcpy(buf, , sizeof l);
break;
case 8:
- l = cpu_to_le32(ioread32(vp_dev->device + offset));
+ l = cpu_to_le32(ioread32(device + offset));
memcpy(buf, , sizeof l);
- l = cpu_to_le32(ioread32(vp_dev->device + offset + sizeof l));
+ l = cpu_to_le32(ioread32(device + offset + sizeof l));
memcpy(buf + sizeof l, , sizeof l);
break;
default:
@@ -228,6 +231,7 @@ static void vp_set(struct virtio_device *vdev, unsigned
offset,
const void *buf, unsigned len)
{
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
+ void __iomem *device = vp_dev->device;
u8 b;
__le16 w;
__le32 l;
@@ -237,21 +241,21 @@ static void vp_set(struct virtio_device *vdev, unsigned
offset,
switch (len) {
case 1:
memcpy(, buf, sizeof b);
- iowrite8(b, vp_dev->device + offset);
+ iowrite8(b, device + offset);
break;
case 2:
memcpy(, buf, sizeof w);
- iowrite16(le16_to_cpu(w), vp_dev->device + offset);
+ iowrite16(le16_to_cpu(w), device + offset);
break;
case 4:
memcpy(, buf, sizeof l);
- iowrite32(le32_to_cpu(l), vp_dev->device + offset);
+ iowrite32(le32_to_cpu(l), device + offset);
break;
case 8:
memcpy(, buf, sizeof l);
- iowrite32(le32_to_cpu(l), vp_dev->device + offset);
+ iowrite32(le32_to_cpu(l), device + offset);
[PATCH linux-next v2 6/7] vdpa: Enable user to query vdpa device info
Enable user to query vdpa device information.
$ vdpa dev add mgmtdev vdpasim_net name foo2
Show the newly created vdpa device by its name:
$ vdpa dev show foo2
foo2: type network mgmtdev vdpasim_net vendor_id 0 max_vqs 2 max_vq_size 256
$ vdpa dev show foo2 -jp
{
"dev": {
"foo2": {
"type": "network",
"mgmtdev": "vdpasim_net",
"vendor_id": 0,
"max_vqs": 2,
"max_vq_size": 256
}
}
}
Signed-off-by: Parav Pandit
Reviewed-by: Eli Cohen
Reviewed-by: Jason Wang
---
drivers/vdpa/vdpa.c | 131 ++
include/uapi/linux/vdpa.h | 5 ++
2 files changed, 136 insertions(+)
diff --git a/drivers/vdpa/vdpa.c b/drivers/vdpa/vdpa.c
index dca67e4d32e5..9700a0adcca0 100644
--- a/drivers/vdpa/vdpa.c
+++ b/drivers/vdpa/vdpa.c
@@ -478,6 +478,131 @@ static int vdpa_nl_cmd_dev_del_set_doit(struct sk_buff
*skb, struct genl_info *i
return err;
}
+static int
+vdpa_dev_fill(struct vdpa_device *vdev, struct sk_buff *msg, u32 portid, u32
seq,
+ int flags, struct netlink_ext_ack *extack)
+{
+ u16 max_vq_size;
+ u32 device_id;
+ u32 vendor_id;
+ void *hdr;
+ int err;
+
+ hdr = genlmsg_put(msg, portid, seq, _nl_family, flags,
VDPA_CMD_DEV_NEW);
+ if (!hdr)
+ return -EMSGSIZE;
+
+ err = vdpa_nl_mgmtdev_handle_fill(msg, vdev->mdev);
+ if (err)
+ goto msg_err;
+
+ device_id = vdev->config->get_device_id(vdev);
+ vendor_id = vdev->config->get_vendor_id(vdev);
+ max_vq_size = vdev->config->get_vq_num_max(vdev);
+
+ err = -EMSGSIZE;
+ if (nla_put_string(msg, VDPA_ATTR_DEV_NAME, dev_name(>dev)))
+ goto msg_err;
+ if (nla_put_u32(msg, VDPA_ATTR_DEV_ID, device_id))
+ goto msg_err;
+ if (nla_put_u32(msg, VDPA_ATTR_DEV_VENDOR_ID, vendor_id))
+ goto msg_err;
+ if (nla_put_u32(msg, VDPA_ATTR_DEV_MAX_VQS, vdev->nvqs))
+ goto msg_err;
+ if (nla_put_u16(msg, VDPA_ATTR_DEV_MAX_VQ_SIZE, max_vq_size))
+ goto msg_err;
+
+ genlmsg_end(msg, hdr);
+ return 0;
+
+msg_err:
+ genlmsg_cancel(msg, hdr);
+ return err;
+}
+
+static int vdpa_nl_cmd_dev_get_doit(struct sk_buff *skb, struct genl_info
*info)
+{
+ struct vdpa_device *vdev;
+ struct sk_buff *msg;
+ const char *devname;
+ struct device *dev;
+ int err;
+
+ if (!info->attrs[VDPA_ATTR_DEV_NAME])
+ return -EINVAL;
+ devname = nla_data(info->attrs[VDPA_ATTR_DEV_NAME]);
+ msg = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL);
+ if (!msg)
+ return -ENOMEM;
+
+ mutex_lock(_dev_mutex);
+ dev = bus_find_device(_bus, NULL, devname, vdpa_name_match);
+ if (!dev) {
+ mutex_unlock(_dev_mutex);
+ NL_SET_ERR_MSG_MOD(info->extack, "device not found");
+ return -ENODEV;
+ }
+ vdev = container_of(dev, struct vdpa_device, dev);
+ if (!vdev->mdev) {
+ mutex_unlock(_dev_mutex);
+ put_device(dev);
+ return -EINVAL;
+ }
+ err = vdpa_dev_fill(vdev, msg, info->snd_portid, info->snd_seq, 0,
info->extack);
+ if (!err)
+ err = genlmsg_reply(msg, info);
+ put_device(dev);
+ mutex_unlock(_dev_mutex);
+
+ if (err)
+ nlmsg_free(msg);
+ return err;
+}
+
+struct vdpa_dev_dump_info {
+ struct sk_buff *msg;
+ struct netlink_callback *cb;
+ int start_idx;
+ int idx;
+};
+
+static int vdpa_dev_dump(struct device *dev, void *data)
+{
+ struct vdpa_device *vdev = container_of(dev, struct vdpa_device, dev);
+ struct vdpa_dev_dump_info *info = data;
+ int err;
+
+ if (!vdev->mdev)
+ return 0;
+ if (info->idx < info->start_idx) {
+ info->idx++;
+ return 0;
+ }
+ err = vdpa_dev_fill(vdev, info->msg, NETLINK_CB(info->cb->skb).portid,
+ info->cb->nlh->nlmsg_seq, NLM_F_MULTI,
info->cb->extack);
+ if (err)
+ return err;
+
+ info->idx++;
+ return 0;
+}
+
+static int vdpa_nl_cmd_dev_get_dumpit(struct sk_buff *msg, struct
netlink_callback *cb)
+{
+ struct vdpa_dev_dump_info info;
+
+ info.msg = msg;
+ info.cb = cb;
+ info.start_idx = cb->args[0];
+ info.idx = 0;
+
+ mutex_lock(_dev_mutex);
+ bus_for_each_dev(_bus, NULL, , vdpa_dev_dump);
+ mutex_unlock(_dev_mutex);
+ cb->args[0] = info.idx;
+ return msg->len;
+}
+
static const struct nla_policy vdpa_nl_policy[VDPA_ATTR_MAX] = {
[VDPA_ATTR_MGMTDEV_BUS_NAME] = { .type = NLA_NUL_STRING },
[VDPA_ATTR_MGMTDEV_DEV_NAME] = { .type = NLA_STRING },
@@ -503,6 +628,12 @@ static const struct genl_ops vdpa_nl_ops[] = {
[PATCH linux-next v2 7/7] vdpa_sim_net: Add support for user supported devices
Enable user to create vdpasim net simulate devices.
Show vdpa management device that supports creating, deleting vdpa devices.
$ vdpa mgmtdev show
vdpasim_net:
supported_classes
net
$ vdpa mgmtdev show -jp
{
"show": {
"vdpasim_net": {
"supported_classes": {
"net"
}
}
}
Create a vdpa device of type networking named as "foo2" from
the management device vdpasim:
$ vdpa dev add mgmtdev vdpasim_net name foo2
Show the newly created vdpa device by its name:
$ vdpa dev show foo2
foo2: type network mgmtdev vdpasim_net vendor_id 0 max_vqs 2 max_vq_size 256
$ vdpa dev show foo2 -jp
{
"dev": {
"foo2": {
"type": "network",
"mgmtdev": "vdpasim_net",
"vendor_id": 0,
"max_vqs": 2,
"max_vq_size": 256
}
}
}
Delete the vdpa device after its use:
$ vdpa dev del foo2
Signed-off-by: Parav Pandit
Reviewed-by: Eli Cohen
Acked-by: Jason Wang
---
Changelog:
v1->v2:
- rebased
---
drivers/vdpa/vdpa_sim/vdpa_sim.c | 3 +-
drivers/vdpa/vdpa_sim/vdpa_sim.h | 2 +
drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 92
3 files changed, 96 insertions(+), 1 deletion(-)
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
index db1636a99ba4..d5942842432d 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
@@ -235,7 +235,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_dev_attr
*dev_attr)
ops = _config_ops;
vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops,
- dev_attr->nvqs, NULL);
+ dev_attr->nvqs, dev_attr->name);
if (!vdpasim)
goto err_alloc;
@@ -249,6 +249,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_dev_attr
*dev_attr)
if (dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64)))
goto err_iommu;
set_dma_ops(dev, _dma_ops);
+ vdpasim->vdpa.mdev = dev_attr->mgmt_dev;
vdpasim->config = kzalloc(dev_attr->config_size, GFP_KERNEL);
if (!vdpasim->config)
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.h b/drivers/vdpa/vdpa_sim/vdpa_sim.h
index b02142293d5b..6d75444f9948 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim.h
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim.h
@@ -33,6 +33,8 @@ struct vdpasim_virtqueue {
};
struct vdpasim_dev_attr {
+ struct vdpa_mgmt_dev *mgmt_dev;
+ const char *name;
u64 supported_features;
size_t config_size;
size_t buffer_size;
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
index 34155831538c..b795e02bdad0 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
@@ -162,6 +162,94 @@ static void vdpasim_net_default_dev_unregister(void)
vdpa_unregister_device(vdpa);
}
+static void vdpasim_net_mgmtdev_release(struct device *dev)
+{
+}
+
+static struct device vdpasim_net_mgmtdev = {
+ .init_name = "vdpasim_net",
+ .release = vdpasim_net_mgmtdev_release,
+};
+
+static int vdpasim_net_dev_add(struct vdpa_mgmt_dev *mdev, const char *name)
+{
+ struct vdpasim_dev_attr dev_attr = {};
+ struct vdpasim *simdev;
+ int ret;
+
+ dev_attr.mgmt_dev = mdev;
+ dev_attr.name = name;
+ dev_attr.id = VIRTIO_ID_NET;
+ dev_attr.supported_features = VDPASIM_NET_FEATURES;
+ dev_attr.nvqs = VDPASIM_NET_VQ_NUM;
+ dev_attr.config_size = sizeof(struct virtio_net_config);
+ dev_attr.get_config = vdpasim_net_get_config;
+ dev_attr.work_fn = vdpasim_net_work;
+ dev_attr.buffer_size = PAGE_SIZE;
+
+ simdev = vdpasim_create(_attr);
+ if (IS_ERR(simdev))
+ return PTR_ERR(simdev);
+
+ ret = _vdpa_register_device(>vdpa);
+ if (ret)
+ goto reg_err;
+
+ return 0;
+
+reg_err:
+ put_device(>vdpa.dev);
+ return ret;
+}
+
+static void vdpasim_net_dev_del(struct vdpa_mgmt_dev *mdev,
+ struct vdpa_device *dev)
+{
+ struct vdpasim *simdev = container_of(dev, struct vdpasim, vdpa);
+
+ _vdpa_unregister_device(>vdpa);
+}
+
+static const struct vdpa_mgmtdev_ops vdpasim_net_mgmtdev_ops = {
+ .dev_add = vdpasim_net_dev_add,
+ .dev_del = vdpasim_net_dev_del
+};
+
+static struct virtio_device_id id_table[] = {
+ { VIRTIO_ID_NET, VIRTIO_DEV_ANY_ID },
+ { 0 },
+};
+
+static struct vdpa_mgmt_dev mgmt_dev = {
+ .device = _net_mgmtdev,
+ .id_table = id_table,
+ .ops = _net_mgmtdev_ops,
+};
+
+static int vdpasim_net_mgmtdev_init(void)
+{
+ int ret;
+
+ ret = device_register(_net_mgmtdev);
+ if (ret)
+ return ret;
+
+ ret = vdpa_mgmtdev_register(_dev);
+ if (ret)
+ goto parent_err;
+ return 0;
+
+parent_err:
+
[PATCH linux-next v2 3/7] vdpa: Extend routine to accept vdpa device name
In a subsequent patch, when user initiated command creates a vdpa device,
the user chooses the name of the vdpa device.
To support it, extend the device allocation API to consider this name
specified by the caller driver.
Signed-off-by: Parav Pandit
Reviewed-by: Eli Cohen
Acked-by: Jason Wang
---
Changelog:
v1->v2:
- rebased
---
drivers/vdpa/ifcvf/ifcvf_main.c | 2 +-
drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +-
drivers/vdpa/vdpa.c | 36 +++
drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 +-
include/linux/vdpa.h | 7 +++---
5 files changed, 38 insertions(+), 11 deletions(-)
diff --git a/drivers/vdpa/ifcvf/ifcvf_main.c b/drivers/vdpa/ifcvf/ifcvf_main.c
index fa1af301cf55..7c8bbfcf6c3e 100644
--- a/drivers/vdpa/ifcvf/ifcvf_main.c
+++ b/drivers/vdpa/ifcvf/ifcvf_main.c
@@ -432,7 +432,7 @@ static int ifcvf_probe(struct pci_dev *pdev, const struct
pci_device_id *id)
adapter = vdpa_alloc_device(struct ifcvf_adapter, vdpa,
dev, _vdpa_ops,
- IFCVF_MAX_QUEUE_PAIRS * 2);
+ IFCVF_MAX_QUEUE_PAIRS * 2, NULL);
if (adapter == NULL) {
IFCVF_ERR(pdev, "Failed to allocate vDPA structure");
return -ENOMEM;
diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c
b/drivers/vdpa/mlx5/net/mlx5_vnet.c
index 81b932f72e10..5920290521cf 100644
--- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
+++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
@@ -1946,7 +1946,7 @@ void *mlx5_vdpa_add_dev(struct mlx5_core_dev *mdev)
max_vqs = min_t(u32, max_vqs, MLX5_MAX_SUPPORTED_VQS);
ndev = vdpa_alloc_device(struct mlx5_vdpa_net, mvdev.vdev,
mdev->device, _vdpa_ops,
-2 * mlx5_vdpa_max_qps(max_vqs));
+2 * mlx5_vdpa_max_qps(max_vqs), NULL);
if (IS_ERR(ndev))
return ndev;
diff --git a/drivers/vdpa/vdpa.c b/drivers/vdpa/vdpa.c
index c0825650c055..7414bbd9057c 100644
--- a/drivers/vdpa/vdpa.c
+++ b/drivers/vdpa/vdpa.c
@@ -12,6 +12,8 @@
#include
#include
+/* A global mutex that protects vdpa management device and device level
operations. */
+static DEFINE_MUTEX(vdpa_dev_mutex);
static DEFINE_IDA(vdpa_index_ida);
static int vdpa_dev_probe(struct device *d)
@@ -63,6 +65,7 @@ static void vdpa_release_dev(struct device *d)
* @config: the bus operations that is supported by this device
* @nvqs: number of virtqueues supported by this device
* @size: size of the parent structure that contains private data
+ * @name: name of the vdpa device; optional.
*
* Driver should use vdpa_alloc_device() wrapper macro instead of
* using this directly.
@@ -72,8 +75,7 @@ static void vdpa_release_dev(struct device *d)
*/
struct vdpa_device *__vdpa_alloc_device(struct device *parent,
const struct vdpa_config_ops *config,
- int nvqs,
- size_t size)
+ int nvqs, size_t size, const char *name)
{
struct vdpa_device *vdev;
int err = -EINVAL;
@@ -101,7 +103,10 @@ struct vdpa_device *__vdpa_alloc_device(struct device
*parent,
vdev->features_valid = false;
vdev->nvqs = nvqs;
- err = dev_set_name(>dev, "vdpa%u", vdev->index);
+ if (name)
+ err = dev_set_name(>dev, "%s", name);
+ else
+ err = dev_set_name(>dev, "vdpa%u", vdev->index);
if (err)
goto err_name;
@@ -118,6 +123,13 @@ struct vdpa_device *__vdpa_alloc_device(struct device
*parent,
}
EXPORT_SYMBOL_GPL(__vdpa_alloc_device);
+static int vdpa_name_match(struct device *dev, const void *data)
+{
+ struct vdpa_device *vdev = container_of(dev, struct vdpa_device, dev);
+
+ return (strcmp(dev_name(>dev), data) == 0);
+}
+
/**
* vdpa_register_device - register a vDPA device
* Callers must have a succeed call of vdpa_alloc_device() before.
@@ -127,7 +139,21 @@ EXPORT_SYMBOL_GPL(__vdpa_alloc_device);
*/
int vdpa_register_device(struct vdpa_device *vdev)
{
- return device_add(>dev);
+ struct device *dev;
+ int err;
+
+ mutex_lock(_dev_mutex);
+ dev = bus_find_device(_bus, NULL, dev_name(>dev),
vdpa_name_match);
+ if (dev) {
+ put_device(dev);
+ err = -EEXIST;
+ goto name_err;
+ }
+
+ err = device_add(>dev);
+name_err:
+ mutex_unlock(_dev_mutex);
+ return err;
}
EXPORT_SYMBOL_GPL(vdpa_register_device);
@@ -137,7 +163,9 @@ EXPORT_SYMBOL_GPL(vdpa_register_device);
*/
void vdpa_unregister_device(struct vdpa_device *vdev)
{
+ mutex_lock(_dev_mutex);
device_unregister(>dev);
+ mutex_unlock(_dev_mutex);
}
EXPORT_SYMBOL_GPL(vdpa_unregister_device);
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c
[PATCH linux-next v2 5/7] vdpa: Enable a user to add and delete a vdpa device
Add the ability to add and delete a vdpa device.
Examples:
Create a vdpa device of type network named "foo2" from
the management device vdpasim:
$ vdpa dev add mgmtdev vdpasim_net name foo2
Delete the vdpa device after its use:
$ vdpa dev del foo2
Signed-off-by: Parav Pandit
Reviewed-by: Eli Cohen
Reviewed-by: Jason Wang
---
Changelog:
v1->v2:
- using int return type for dev_add callback
- removed device_id (type) as current drivers only supports single type
---
drivers/vdpa/vdpa.c | 143 +++---
include/linux/vdpa.h | 6 ++
include/uapi/linux/vdpa.h | 4 ++
3 files changed, 143 insertions(+), 10 deletions(-)
diff --git a/drivers/vdpa/vdpa.c b/drivers/vdpa/vdpa.c
index 319d09709dfc..dca67e4d32e5 100644
--- a/drivers/vdpa/vdpa.c
+++ b/drivers/vdpa/vdpa.c
@@ -136,6 +136,37 @@ static int vdpa_name_match(struct device *dev, const void
*data)
return (strcmp(dev_name(>dev), data) == 0);
}
+static int __vdpa_register_device(struct vdpa_device *vdev)
+{
+ struct device *dev;
+
+ lockdep_assert_held(_dev_mutex);
+ dev = bus_find_device(_bus, NULL, dev_name(>dev),
vdpa_name_match);
+ if (dev) {
+ put_device(dev);
+ return -EEXIST;
+ }
+ return device_add(>dev);
+}
+
+/**
+ * _vdpa_register_device - register a vDPA device with vdpa lock held
+ * Caller must have a succeed call of vdpa_alloc_device() before.
+ * Caller must invoke this routine in the management device dev_add()
+ * callback after setting up valid mgmtdev for this vdpa device.
+ * @vdev: the vdpa device to be registered to vDPA bus
+ *
+ * Returns an error when fail to add device to vDPA bus
+ */
+int _vdpa_register_device(struct vdpa_device *vdev)
+{
+ if (!vdev->mdev)
+ return -EINVAL;
+
+ return __vdpa_register_device(vdev);
+}
+EXPORT_SYMBOL_GPL(_vdpa_register_device);
+
/**
* vdpa_register_device - register a vDPA device
* Callers must have a succeed call of vdpa_alloc_device() before.
@@ -145,24 +176,29 @@ static int vdpa_name_match(struct device *dev, const void
*data)
*/
int vdpa_register_device(struct vdpa_device *vdev)
{
- struct device *dev;
int err;
mutex_lock(_dev_mutex);
- dev = bus_find_device(_bus, NULL, dev_name(>dev),
vdpa_name_match);
- if (dev) {
- put_device(dev);
- err = -EEXIST;
- goto name_err;
- }
-
- err = device_add(>dev);
-name_err:
+ err = __vdpa_register_device(vdev);
mutex_unlock(_dev_mutex);
return err;
}
EXPORT_SYMBOL_GPL(vdpa_register_device);
+/**
+ * _vdpa_unregister_device - unregister a vDPA device
+ * Caller must invoke this routine as part of management device dev_del()
+ * callback.
+ * @vdev: the vdpa device to be unregisted from vDPA bus
+ */
+void _vdpa_unregister_device(struct vdpa_device *vdev)
+{
+ lockdep_assert_held(_dev_mutex);
+ WARN_ON(!vdev->mdev);
+ device_unregister(>dev);
+}
+EXPORT_SYMBOL_GPL(_vdpa_unregister_device);
+
/**
* vdpa_unregister_device - unregister a vDPA device
* @vdev: the vdpa device to be unregisted from vDPA bus
@@ -221,10 +257,25 @@ int vdpa_mgmtdev_register(struct vdpa_mgmt_dev *mdev)
}
EXPORT_SYMBOL_GPL(vdpa_mgmtdev_register);
+static int vdpa_match_remove(struct device *dev, void *data)
+{
+ struct vdpa_device *vdev = container_of(dev, struct vdpa_device, dev);
+ struct vdpa_mgmt_dev *mdev = vdev->mdev;
+
+ if (mdev == data)
+ mdev->ops->dev_del(mdev, vdev);
+ return 0;
+}
+
void vdpa_mgmtdev_unregister(struct vdpa_mgmt_dev *mdev)
{
mutex_lock(_dev_mutex);
+
list_del(>list);
+
+ /* Filter out all the entries belong to this management device and
delete it. */
+ bus_for_each_dev(_bus, NULL, mdev, vdpa_match_remove);
+
mutex_unlock(_dev_mutex);
}
EXPORT_SYMBOL_GPL(vdpa_mgmtdev_unregister);
@@ -368,9 +419,69 @@ vdpa_nl_cmd_mgmtdev_get_dumpit(struct sk_buff *msg, struct
netlink_callback *cb)
return msg->len;
}
+static int vdpa_nl_cmd_dev_add_set_doit(struct sk_buff *skb, struct genl_info
*info)
+{
+ struct vdpa_mgmt_dev *mdev;
+ const char *name;
+ int err = 0;
+
+ if (!info->attrs[VDPA_ATTR_DEV_NAME])
+ return -EINVAL;
+
+ name = nla_data(info->attrs[VDPA_ATTR_DEV_NAME]);
+
+ mutex_lock(_dev_mutex);
+ mdev = vdpa_mgmtdev_get_from_attr(info->attrs);
+ if (IS_ERR(mdev)) {
+ NL_SET_ERR_MSG_MOD(info->extack, "Fail to find the specified
management device");
+ err = PTR_ERR(mdev);
+ goto err;
+ }
+
+ err = mdev->ops->dev_add(mdev, name);
+err:
+ mutex_unlock(_dev_mutex);
+ return err;
+}
+
+static int vdpa_nl_cmd_dev_del_set_doit(struct sk_buff *skb, struct genl_info
*info)
+{
+ struct vdpa_mgmt_dev *mdev;
+ struct vdpa_device *vdev;
+
[PATCH linux-next v2 4/7] vdpa: Define vdpa mgmt device, ops and a netlink interface
To add one or more VDPA devices, define a management device which
allows adding or removing vdpa device. A management device defines
set of callbacks to manage vdpa devices.
To begin with, it defines add and remove callbacks through which a user
defined vdpa device can be added or removed.
A unique management device is identified by its unique handle identified
by management device name and optionally the bus name.
Hence, introduce routine through which driver can register a
management device and its callback operations for adding and remove
a vdpa device.
Introduce vdpa netlink socket family so that user can query management
device and its attributes.
Example of show vdpa management device which allows creating vdpa device of
networking class (device id = 0x1) of virtio specification 1.1
section 5.1.1.
$ vdpa mgmtdev show
vdpasim_net:
supported_classes:
net
Example of showing vdpa management device in JSON format.
$ vdpa mgmtdev show -jp
{
"show": {
"vdpasim_net": {
"supported_classes": [ "net" ]
}
}
}
Signed-off-by: Parav Pandit
Reviewed-by: Eli Cohen
Reviewed-by: Jason Wang
---
Changelog:
v1->v2:
- rebased
- updated commit log example for management device name from
"vdpasim" to "vdpasim_net"
- removed device_id as net and block management devices are separated
- dev_add() return type is changed from struct vdpa_device to int
---
drivers/vdpa/Kconfig | 1 +
drivers/vdpa/vdpa.c | 213 +-
include/linux/vdpa.h | 31 ++
include/uapi/linux/vdpa.h | 31 ++
4 files changed, 275 insertions(+), 1 deletion(-)
create mode 100644 include/uapi/linux/vdpa.h
diff --git a/drivers/vdpa/Kconfig b/drivers/vdpa/Kconfig
index 92a6396f8a73..ffd1e098bfd2 100644
--- a/drivers/vdpa/Kconfig
+++ b/drivers/vdpa/Kconfig
@@ -1,6 +1,7 @@
# SPDX-License-Identifier: GPL-2.0-only
menuconfig VDPA
tristate "vDPA drivers"
+ depends on NET
help
Enable this module to support vDPA device that uses a
datapath which complies with virtio specifications with
diff --git a/drivers/vdpa/vdpa.c b/drivers/vdpa/vdpa.c
index 7414bbd9057c..319d09709dfc 100644
--- a/drivers/vdpa/vdpa.c
+++ b/drivers/vdpa/vdpa.c
@@ -11,11 +11,17 @@
#include
#include
#include
+#include
+#include
+#include
+static LIST_HEAD(mdev_head);
/* A global mutex that protects vdpa management device and device level
operations. */
static DEFINE_MUTEX(vdpa_dev_mutex);
static DEFINE_IDA(vdpa_index_ida);
+static struct genl_family vdpa_nl_family;
+
static int vdpa_dev_probe(struct device *d)
{
struct vdpa_device *vdev = dev_to_vdpa(d);
@@ -195,13 +201,218 @@ void vdpa_unregister_driver(struct vdpa_driver *drv)
}
EXPORT_SYMBOL_GPL(vdpa_unregister_driver);
+/**
+ * vdpa_mgmtdev_register - register a vdpa management device
+ *
+ * @mdev: Pointer to vdpa management device
+ * vdpa_mgmtdev_register() register a vdpa management device which supports
+ * vdpa device management.
+ */
+int vdpa_mgmtdev_register(struct vdpa_mgmt_dev *mdev)
+{
+ if (!mdev->device || !mdev->ops || !mdev->ops->dev_add ||
!mdev->ops->dev_del)
+ return -EINVAL;
+
+ INIT_LIST_HEAD(>list);
+ mutex_lock(_dev_mutex);
+ list_add_tail(>list, _head);
+ mutex_unlock(_dev_mutex);
+ return 0;
+}
+EXPORT_SYMBOL_GPL(vdpa_mgmtdev_register);
+
+void vdpa_mgmtdev_unregister(struct vdpa_mgmt_dev *mdev)
+{
+ mutex_lock(_dev_mutex);
+ list_del(>list);
+ mutex_unlock(_dev_mutex);
+}
+EXPORT_SYMBOL_GPL(vdpa_mgmtdev_unregister);
+
+static bool mgmtdev_handle_match(const struct vdpa_mgmt_dev *mdev,
+const char *busname, const char *devname)
+{
+ /* Bus name is optional for simulated management device, so ignore the
+* device with bus if bus attribute is provided.
+*/
+ if ((busname && !mdev->device->bus) || (!busname && mdev->device->bus))
+ return false;
+
+ if (!busname && strcmp(dev_name(mdev->device), devname) == 0)
+ return true;
+
+ if (busname && (strcmp(mdev->device->bus->name, busname) == 0) &&
+ (strcmp(dev_name(mdev->device), devname) == 0))
+ return true;
+
+ return false;
+}
+
+static struct vdpa_mgmt_dev *vdpa_mgmtdev_get_from_attr(struct nlattr **attrs)
+{
+ struct vdpa_mgmt_dev *mdev;
+ const char *busname = NULL;
+ const char *devname;
+
+ if (!attrs[VDPA_ATTR_MGMTDEV_DEV_NAME])
+ return ERR_PTR(-EINVAL);
+ devname = nla_data(attrs[VDPA_ATTR_MGMTDEV_DEV_NAME]);
+ if (attrs[VDPA_ATTR_MGMTDEV_BUS_NAME])
+ busname = nla_data(attrs[VDPA_ATTR_MGMTDEV_BUS_NAME]);
+
+ list_for_each_entry(mdev, _head, list) {
+ if (mgmtdev_handle_match(mdev, busname, devname))
+ return mdev;
+ }
+ return ERR_PTR(-ENODEV);
+}
+
+static
[PATCH linux-next v2 1/7] vdpa_sim_net: Make mac address array static
MAC address array is used only in vdpa_sim_net.c. Hence, keep it static. Signed-off-by: Parav Pandit --- Changelog: v1->v2: - new patch --- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c index c10b6981fdab..f0482427186b 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c @@ -33,7 +33,7 @@ static char *macaddr; module_param(macaddr, charp, 0); MODULE_PARM_DESC(macaddr, "Ethernet MAC address"); -u8 macaddr_buf[ETH_ALEN]; +static u8 macaddr_buf[ETH_ALEN]; static struct vdpasim *vdpasim_net_dev; -- 2.26.2 ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH linux-next v2 0/7] Introduce vdpa management tool
This patchset covers user requirements for managing existing vdpa devices,
using a tool and its internal design notes for kernel drivers.
Background and user requirements:
--
(1) Currently VDPA device is created by driver when driver is loaded.
However, user should have a choice when to create or not create a vdpa device
for the underlying management device.
For example, mlx5 PCI VF and subfunction device supports multiple classes of
device such netdev, vdpa, rdma. Howevever it is not required to always created
vdpa device for such device.
(2) In another use case, a device may support creating one or multiple vdpa
device of same or different class such as net and block.
Creating vdpa devices at driver load time further limits this use case.
(3) A user should be able to monitor and query vdpa queue level or device level
statistics for a given vdpa device.
(4) A user should be able to query what class of vdpa devices are supported
by its management device.
(5) A user should be able to view supported features and negotiated features
of the vdpa device.
(6) A user should be able to create a vdpa device in vendor agnostic manner
using single tool.
Hence, it is required to have a tool through which user can create one or more
vdpa devices from a management device which addresses above user requirements.
Example devices:
+---+ +---+ +-+ ++ +---+
|vdpa dev 0 | |vdpa dev 1 | |rdma dev | |netdev | |vdpa dev 3 |
|type=net | |type=net | |mlx5_0 | |ens3f0 | |type=net |
++--+ +-+-+ +++ +-+--+ ++--+
| ||| |
| ||| |
++-+| +++ |+++
| mlx5++ |mlx5 +---+|mlx5 |
|pci vf 2 ||pci vf 4 ||pci sf 8 |
|03:00:2 ||03:00.4 ||mlx5_sf.8|
++-+++++++
| | |
| ++-+|
+--+mlx5 ++
|pci pf 0 |
|03:00.0 |
+--+
vdpa tool:
--
vdpa tool is a tool to create, delete vdpa devices from a management device.
It is a tool that enables user to query statistics, features and may be more
attributes in future.
vdpa tool command draft:
(a) List management devices which support creating vdpa devices.
It also shows which class types supported by this management device.
In below command example four management devices support vdpa device creation.
First is PCI VF whose bdf is 03.00:2.
Second is PCI VF whose name is 03:00.4.
Third is PCI SF whose name is mlx5_core.sf.8
$ vdpa mgmtdev list
vdpasim_net
supported_classes
net
pci/:03.00:0
supported_classes
net
pci/:03.00:4
supported_classes
net
auxiliary/mlx5_core.sf.8
supported_classes
net
(b) Now add a vdpa device of networking class and show the device.
$ vdpa dev add mgmtdev pci/:03.00:0 name foo0
$ vdpa dev show foo0
foo0: mgmtdev pci/:03.00:2 type network vendor_id 0 max_vqs 2 max_vq_size
256
(c) Show features of a vdpa device
$ vdpa dev features show foo0
supported
iommu platform
version 1
(d) Dump vdpa device statistics
$ vdpa dev stats show foo0
kickdoorbells 10
wqes 100
(e) Now delete a vdpa device previously created.
$ vdpa dev del foo0
vdpa tool support in this patchset:
---
vdpa tool is created to create, delete and query vdpa devices.
examples:
Show vdpa management device that supports creating, deleting vdpa devices.
$ vdpa mgmtdev show
vdpasim_net:
supported_classes
net
$ vdpa mgmtdev show -jp
{
"show": {
"vdpasim_net": {
"supported_classes": {
"net"
}
}
}
Create a vdpa device of type networking named as "foo2" from the
management device vdpasim:
$ vdpa dev add mgmtdev vdpasim type net name foo2
Show the newly created vdpa device by its name:
$ vdpa dev show foo2
foo2: type network mgmtdev vdpasim vendor_id 0 max_vqs 2 max_vq_size 256
$ vdpa dev show foo2 -jp
{
"dev": {
"foo2": {
"type": "network",
"mgmtdev": "vdpasim_net",
"vendor_id": 0,
"max_vqs": 2,
"max_vq_size": 256
}
}
}
Delete the vdpa device after its use:
$ vdpa dev del foo2
vdpa tool support by kernel:
vdpa tool user interface will be supported by existing vdpa kernel framework,
i.e. drivers/vdpa/vdpa.c It services user command through a netlink interface.
Each management device registers supported callback operations with vdpa
subsystem through which vdpa device(s)
[PATCH linux-next v2 2/7] vdpa_sim_net: Add module param to disable default vdpa net device
To support creating multiple vdpa devices and to allow user to
manage them, add a knob to disable a default vdpa net device.
Signed-off-by: Parav Pandit
---
Changelog:
v1->v2:
- new patch
---
drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 41
1 file changed, 29 insertions(+), 12 deletions(-)
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
index f0482427186b..34155831538c 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
@@ -33,6 +33,10 @@ static char *macaddr;
module_param(macaddr, charp, 0);
MODULE_PARM_DESC(macaddr, "Ethernet MAC address");
+static bool default_device = true;
+module_param(default_device, bool, 0);
+MODULE_PARM_DESC(default_device, "Support single default VDPA device");
+
static u8 macaddr_buf[ETH_ALEN];
static struct vdpasim *vdpasim_net_dev;
@@ -120,21 +124,11 @@ static void vdpasim_net_get_config(struct vdpasim
*vdpasim, void *config)
memcpy(net_config->mac, macaddr_buf, ETH_ALEN);
}
-static int __init vdpasim_net_init(void)
+static int vdpasim_net_default_dev_register(void)
{
struct vdpasim_dev_attr dev_attr = {};
int ret;
- if (macaddr) {
- mac_pton(macaddr, macaddr_buf);
- if (!is_valid_ether_addr(macaddr_buf)) {
- ret = -EADDRNOTAVAIL;
- goto out;
- }
- } else {
- eth_random_addr(macaddr_buf);
- }
-
dev_attr.id = VIRTIO_ID_NET;
dev_attr.supported_features = VDPASIM_NET_FEATURES;
dev_attr.nvqs = VDPASIM_NET_VQ_NUM;
@@ -161,13 +155,36 @@ static int __init vdpasim_net_init(void)
return ret;
}
-static void __exit vdpasim_net_exit(void)
+static void vdpasim_net_default_dev_unregister(void)
{
struct vdpa_device *vdpa = _net_dev->vdpa;
vdpa_unregister_device(vdpa);
}
+static int __init vdpasim_net_init(void)
+{
+ int ret = 0;
+
+ if (macaddr) {
+ mac_pton(macaddr, macaddr_buf);
+ if (!is_valid_ether_addr(macaddr_buf))
+ return -EADDRNOTAVAIL;
+ } else {
+ eth_random_addr(macaddr_buf);
+ }
+
+ if (default_device)
+ ret = vdpasim_net_default_dev_register();
+ return ret;
+}
+
+static void __exit vdpasim_net_exit(void)
+{
+ if (default_device)
+ vdpasim_net_default_dev_unregister();
+}
+
module_init(vdpasim_net_init);
module_exit(vdpasim_net_exit);
--
2.26.2
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH V2 19/19] vdpa: introduce virtio pci driver
On 2020/12/4 下午11:20, Stefano Garzarella wrote:
+#define VP_VDPA_QUEUE_MAX 256
+#define VP_VDPA_DRIVER_NAME "vp_vdpa"
+
+struct vp_vring {
+ void __iomem *notify;
+ char msix_name[256];
Can we use a macro for the msix_name size, since we use 256 in
multiple places?
Yes, will switch to use a macro.
+ struct vdpa_callback cb;
+ int irq;
+};
+
+struct vp_vdpa {
+ struct vdpa_device vdpa;
+ struct virtio_pci_modern_device mdev;
+ struct vp_vring *vring;
+ struct vdpa_callback cb;
^
It is not relevant, but 'config_cb' is maybe clearer to read.
Will change to config_cb.
The rest looks good.
Thanks
Thanks,
Stefano
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH V2 16/19] virtio-pci: introduce modern device module
On 2020/12/5 上午1:10, Randy Dunlap wrote: Hi Jason-- On 12/3/20 8:03 PM, Jason Wang wrote: diff --git a/drivers/virtio/Kconfig b/drivers/virtio/Kconfig index 7b41130d3f35..d1a6bd2a975f 100644 --- a/drivers/virtio/Kconfig +++ b/drivers/virtio/Kconfig @@ -12,6 +12,14 @@ config ARCH_HAS_RESTRICTED_VIRTIO_MEMORY_ACCESS This option is selected if the architecture may need to enforce VIRTIO_F_ACCESS_PLATFORM +config VIRTIO_PCI_MODERN + tristate "Modern Virtio PCI Device" + depends on PCI + help + Modern PCI device implementation. This module implement the implements + basic probe and control for devices which is based on modern are + PCI device with possible vendor specific extensions. devices + cheers. All fixed. Thanks ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH V2 00/19] vDPA driver for virtio-pci device
On Thu, Dec 31, 2020 at 11:52:14AM +0800, Jason Wang wrote: > > On 2020/12/4 下午12:03, Jason Wang wrote: > > Hi all: > > > > This series tries to implement a vDPA driver for virtio-pci device > > which will bridge between vDPA bus and virtio-pci device. > > > > This could be used for future feature prototyping and testing. > > > > Please review > > > > Changes from V2: > > > > - don't try to use devres for virtio-pci core > > - tweak the commit log > > - split the patches furtherly to ease the reviewing > > > > Changes from V1: > > > > - Split common codes from virito-pci and share it with vDPA driver > > - Use dynamic id in order to be less confusing with virtio-pci driver > > - No feature whitelist, supporting any features (mq, config etc) > > > > Thanks > > > Michael, any comment for this series? > > It's needed for testing doorbell mapping and config interrupt support. > > Thanks I saw you got some comments back in december so was expecting another version. If you'd rather I reviewed this one, let me know. -- MST ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
