Hi,
I have been working on the PM driver implementation for a while now, and
I feel I need to share the current work status with you. No point
continuing working in a given direction if it is already be know to be wrong.
In short, this patch series define the ODP-driver interface (ODP south
interface) and implementes an uggly driver for X540 ixgbe board, to test it.
The Implementation is not complete: Release resource at pktio_close() is
not fully implemeneted nor tested. The driver itself requires much more work
This implements pci-vfio only at this stage. (requires iommu).
The Main 2 problems are:
1) getting contigous physical memory and its physical address from user space:
This is not really needed for vfio, but if user-io is supported in
the future, it will be required. DPDK seems to have an strange try and error
approach testing if pages are consecutive... None of this is really needed
for vfio, but anything else will be redundant if user-io is to be supported.
2) Thread vs Process implementation of ODP tasks and ressource sharing.
At lof of work is needed there. including for existing pktio, I am afraid.
Thanks for you comments,
Christophe.
Here follows a few comments on the included patches:
0002-api-introducing-the-driver-api-definition-file.patch:
patch 2 introduce the driver API: The application API (north) is defined by
the files included by odp.h whereas the driver API is defined by the
files included by odp_driver.h (south interface)
Though this method has the flexibility to enable API overlapping
(i.e. some file can be included by both files), this API separation method
comes with 1 drawbacks:
-The granularity of the API difference is at include file level:
for instance odp_shm_get_dma_map() introduced in patch 4 should
not be visible by an ODP app, and it will be here. having different
includes files for each interface can fix this but seems heavy...
-There is also an issue with the documentation but Mike
seemed to have solutions available...:-)
0005-linux-generic-shmem-retrieving-DMA-map-desriptor.patch
odp_shm_get_dma_map() at this stage returns the user virtual address
as DMA address. This will only work with tasks belonging to a single
thread and using pci-vfio only (where one can freely choose iova addresses)
0006-linux-generic-pool-retrieving-DMA-map-desriptor.patch
patch 6. As for patch5, the virtual user space address in used as DMA
address, with the same consequence.
0009-linux-generic-PCI-using-pci-vfio.patch
patch 9 (PCI using pci-vfio). The scope of a pci device is the process: PCI
devices use file descriptors whose scope is the process. Trying to open
many time the same device from different process will be rejected by pci_vfio
(vfio group busy). Multithread access to the same device is also rejected
at the moment: Allowing it would require thread-safe driver with mutex cost.
Not sure whether we realy want that as most HW seems to have mutliqueue
support (opening more that ine queue for a device seems a better future
solution.). Seems there is job remaining in this area, for all pktio...
Also, in patch 9, odp_pci_vfio.c is largely inspired by DPDK eal_pci_vfio.c.
Not sure what to write there regarding the license.
Resource release not fully implemented yet
0014-driver-api-registration-and-probing-interface.patch
At this stage the probing function expects the PCI address in the pktio
device name. This could be hidden behind some kind of pci enumeration
in the future.
0022-drivers-ixgbe-copying-base-files.patch
Warnings have been disabled so that these files can remain unchanged
(They are common to all ixgbe drivers). I think it should be so.
checkpatch fails for the same reason.
0026-drivers-ixgbe-adding-basic-driver.patch
This is still really an uggly hack used to proove the interface.
checkpatch fails: it should be fixed.
0030-validation-pktio-skipping-all-tests-but-one.patch
This just shows what I needed to hack into the pktio test to be able to
loopback a packet: Because resource deallocation is still imcomplete, more
than one test cannot be run consecutively
Christophe Milard (31):
api: dma: defining the dma region descriptor
api: introducing the driver api definition file
linux-generix: dma: implementation
api: shared_memory: getting DMA descriptor
linux-generic: shmem: retrieving DMA map desriptor
linux-generic: pool: retrieving DMA map desriptor
driver api: adding pci interface
linux-generic: defining PCI api
linux-generic: PCI using pci-vfio
linux-generic: generic PCI implementation
defining the location for ODP drivers
driver api: nic segments and segment pool
linux-generic: nic driver segment implementation
driver api: registration and probing interface
linux-generic: driver registration implementation
linux-generic: driver: nic device alloc and free
linux-generic: nic pktio
linux-generic: nic pktio: adding tx and rx
linux-generic: nic pktio: adding mac_get
drivers: defining the common Makefile.inc
drivers: defining the drivers logging macros.
drivers: ixgbe: copying base files
drivers: ixgbe: fixing base/ixgbe_osdep.h for ODP
drivers: ixgbe: building base files
drivers: ixgbe: adding logging macros
drivers: ixgbe: adding basic driver
validation: possibility to inactive preconded test
validation: pktio: increasing number of rx pkts
validation: pktio: increasing the wait delay
validation: pktio: skipping all tests but one
validation: pktio: setup for nic pktio
Makefile.am | 6 +-
configure.ac | 16 +
drivers/Makefile.am | 10 +
drivers/Makefile.inc | 6 +
drivers/README | 1 +
drivers/driver_init.c | 13 +
drivers/driver_init.h | 7 +
drivers/driver_log.h | 26 +
drivers/ixgbe/Makefile.am | 7 +
drivers/ixgbe/base/Makefile.am | 21 +
drivers/ixgbe/base/README | 61 +
drivers/ixgbe/base/ixgbe_82598.c | 1436 ++++++
drivers/ixgbe/base/ixgbe_82598.h | 52 +
drivers/ixgbe/base/ixgbe_82599.c | 2580 ++++++++++
drivers/ixgbe/base/ixgbe_82599.h | 64 +
drivers/ixgbe/base/ixgbe_api.c | 1643 +++++++
drivers/ixgbe/base/ixgbe_api.h | 220 +
drivers/ixgbe/base/ixgbe_common.c | 5143 ++++++++++++++++++++
drivers/ixgbe/base/ixgbe_common.h | 188 +
drivers/ixgbe/base/ixgbe_dcb.c | 714 +++
drivers/ixgbe/base/ixgbe_dcb.h | 174 +
drivers/ixgbe/base/ixgbe_dcb_82598.c | 360 ++
drivers/ixgbe/base/ixgbe_dcb_82598.h | 99 +
drivers/ixgbe/base/ixgbe_dcb_82599.c | 593 +++
drivers/ixgbe/base/ixgbe_dcb_82599.h | 153 +
drivers/ixgbe/base/ixgbe_mbx.c | 789 +++
drivers/ixgbe/base/ixgbe_mbx.h | 150 +
drivers/ixgbe/base/ixgbe_osdep.h | 168 +
drivers/ixgbe/base/ixgbe_phy.c | 2738 +++++++++++
drivers/ixgbe/base/ixgbe_phy.h | 207 +
drivers/ixgbe/base/ixgbe_type.h | 4033 +++++++++++++++
drivers/ixgbe/base/ixgbe_vf.c | 723 +++
drivers/ixgbe/base/ixgbe_vf.h | 140 +
drivers/ixgbe/base/ixgbe_x540.c | 1012 ++++
drivers/ixgbe/base/ixgbe_x540.h | 66 +
drivers/ixgbe/base/ixgbe_x550.c | 3209 ++++++++++++
drivers/ixgbe/base/ixgbe_x550.h | 108 +
drivers/ixgbe/ixgbe_common.h | 23 +
drivers/ixgbe/ixgbe_logs.h | 67 +
drivers/ixgbe/ixgbe_main.c | 520 ++
drivers/ixgbe/ixgbe_main.h | 7 +
drivers/ixgbe/ixgbe_rxtx.c | 1279 +++++
drivers/ixgbe/ixgbe_rxtx.h | 225 +
drivers/ixgbe/ixgbe_supported_dev.h | 119 +
drivers/m4/configure.m4 | 5 +
include/odp.h | 2 +-
include/odp/api/dma.h | 102 +
include/odp/api/nic.h | 274 ++
include/odp/api/pci.h | 131 +
include/odp/api/shared_memory.h | 14 +
include/odp_driver.h | 36 +
platform/linux-generic/Makefile.am | 26 +
platform/linux-generic/include/odp/dma.h | 36 +
platform/linux-generic/include/odp/nic.h | 41 +
platform/linux-generic/include/odp/pci.h | 36 +
.../linux-generic/include/odp/plat/dma_types.h | 42 +
.../linux-generic/include/odp/plat/nic_types.h | 46 +
.../linux-generic/include/odp/plat/pci_types.h | 40 +
.../linux-generic/include/odp/plat/pool_types.h | 3 +-
platform/linux-generic/include/odp_dma_internal.h | 68 +
platform/linux-generic/include/odp_internal.h | 2 +
platform/linux-generic/include/odp_nic_internal.h | 64 +
.../linux-generic/include/odp_packet_io_internal.h | 1 +
platform/linux-generic/include/odp_pci_internal.h | 103 +
.../linux-generic/include/odp_pci_vfio_internal.h | 92 +
platform/linux-generic/include/odp_pool_internal.h | 4 +
platform/linux-generic/odp_dma.c | 78 +
platform/linux-generic/odp_init.c | 5 +
platform/linux-generic/odp_nic.c | 342 ++
platform/linux-generic/odp_pci.c | 190 +
platform/linux-generic/odp_pci_vfio.c | 873 ++++
platform/linux-generic/odp_pool.c | 59 +
platform/linux-generic/odp_shared_memory.c | 26 +
platform/linux-generic/pktio/io_ops.c | 3 +
platform/linux-generic/pktio/nic.c | 108 +
platform/linux-generic/test/pktio/pktio_setup_nic | 105 +
test/validation/common/odp_cunit_common.h | 2 +-
test/validation/pktio/pktio.c | 64 +-
78 files changed, 32164 insertions(+), 35 deletions(-)
create mode 100644 drivers/Makefile.am
create mode 100644 drivers/Makefile.inc
create mode 100644 drivers/README
create mode 100644 drivers/driver_init.c
create mode 100644 drivers/driver_init.h
create mode 100644 drivers/driver_log.h
create mode 100644 drivers/ixgbe/Makefile.am
create mode 100644 drivers/ixgbe/base/Makefile.am
create mode 100644 drivers/ixgbe/base/README
create mode 100644 drivers/ixgbe/base/ixgbe_82598.c
create mode 100644 drivers/ixgbe/base/ixgbe_82598.h
create mode 100644 drivers/ixgbe/base/ixgbe_82599.c
create mode 100644 drivers/ixgbe/base/ixgbe_82599.h
create mode 100644 drivers/ixgbe/base/ixgbe_api.c
create mode 100644 drivers/ixgbe/base/ixgbe_api.h
create mode 100644 drivers/ixgbe/base/ixgbe_common.c
create mode 100644 drivers/ixgbe/base/ixgbe_common.h
create mode 100644 drivers/ixgbe/base/ixgbe_dcb.c
create mode 100644 drivers/ixgbe/base/ixgbe_dcb.h
create mode 100644 drivers/ixgbe/base/ixgbe_dcb_82598.c
create mode 100644 drivers/ixgbe/base/ixgbe_dcb_82598.h
create mode 100644 drivers/ixgbe/base/ixgbe_dcb_82599.c
create mode 100644 drivers/ixgbe/base/ixgbe_dcb_82599.h
create mode 100644 drivers/ixgbe/base/ixgbe_mbx.c
create mode 100644 drivers/ixgbe/base/ixgbe_mbx.h
create mode 100644 drivers/ixgbe/base/ixgbe_osdep.h
create mode 100644 drivers/ixgbe/base/ixgbe_phy.c
create mode 100644 drivers/ixgbe/base/ixgbe_phy.h
create mode 100644 drivers/ixgbe/base/ixgbe_type.h
create mode 100644 drivers/ixgbe/base/ixgbe_vf.c
create mode 100644 drivers/ixgbe/base/ixgbe_vf.h
create mode 100644 drivers/ixgbe/base/ixgbe_x540.c
create mode 100644 drivers/ixgbe/base/ixgbe_x540.h
create mode 100644 drivers/ixgbe/base/ixgbe_x550.c
create mode 100644 drivers/ixgbe/base/ixgbe_x550.h
create mode 100644 drivers/ixgbe/ixgbe_common.h
create mode 100644 drivers/ixgbe/ixgbe_logs.h
create mode 100644 drivers/ixgbe/ixgbe_main.c
create mode 100644 drivers/ixgbe/ixgbe_main.h
create mode 100644 drivers/ixgbe/ixgbe_rxtx.c
create mode 100644 drivers/ixgbe/ixgbe_rxtx.h
create mode 100644 drivers/ixgbe/ixgbe_supported_dev.h
create mode 100644 drivers/m4/configure.m4
create mode 100644 include/odp/api/dma.h
create mode 100644 include/odp/api/nic.h
create mode 100644 include/odp/api/pci.h
create mode 100644 include/odp_driver.h
create mode 100644 platform/linux-generic/include/odp/dma.h
create mode 100644 platform/linux-generic/include/odp/nic.h
create mode 100644 platform/linux-generic/include/odp/pci.h
create mode 100644 platform/linux-generic/include/odp/plat/dma_types.h
create mode 100644 platform/linux-generic/include/odp/plat/nic_types.h
create mode 100644 platform/linux-generic/include/odp/plat/pci_types.h
create mode 100644 platform/linux-generic/include/odp_dma_internal.h
create mode 100644 platform/linux-generic/include/odp_nic_internal.h
create mode 100644 platform/linux-generic/include/odp_pci_internal.h
create mode 100644 platform/linux-generic/include/odp_pci_vfio_internal.h
create mode 100644 platform/linux-generic/odp_dma.c
create mode 100644 platform/linux-generic/odp_nic.c
create mode 100644 platform/linux-generic/odp_pci.c
create mode 100644 platform/linux-generic/odp_pci_vfio.c
create mode 100644 platform/linux-generic/pktio/nic.c
create mode 100755 platform/linux-generic/test/pktio/pktio_setup_nic
--
2.1.4
_______________________________________________
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp
