From: Kiran Patchala <[email protected]>

Add a new poll-mode driver for the Synopsys DesignWare Cores Enhanced
Quality-of-Service (DWC EQoS / GMAC4, "stmmac") Ethernet MAC present.

The device is enumerated by the platform bus and bound to vfio-platform.
The kernel recovery driver handles clock and PHY reset before the PMD
takes over.

Driver features:
 - One TX queue and one RX queue on DMA channel 0; single-segment mbufs
   only; no TSO, PTP, or secondary process support.
 - Non-coherent DMA: DC CVAC/CIVAC cache maintenance on arm64 for
   descriptor and payload lines; no-op stubs on other architectures.
 - Ethernet ops: configure, start, stop, close, link_update,
   promiscuous enable/disable, mac_addr_set, stats_get, stats_reset,
   dev_infos_get, rx/tx_queue_setup, rxq/txq_info_get.

Limitations:
 - Single DMA channel (queue 0 only); no multi-queue support.
 - Single-segment mbufs; scatter-gather, TSO and jumbo frames are not
   supported.
 - Tested on arm64 only.

Testing:
  dpdk-testpmd: txonly / rxonly / io forward at 1 Gb/s link

Co-developed-by: Mohan Degalwade <[email protected]>
Signed-off-by: Mohan Degalwade <[email protected]>
Signed-off-by: Kiran Patchala <[email protected]>
---
 doc/guides/nics/index.rst          |    1 +
 doc/guides/nics/stmmac.rst         |  304 +++++++++
 drivers/net/meson.build            |    1 +
 drivers/net/stmmac/meson.build     |   15 +
 drivers/net/stmmac/stmmac.h        |   95 +++
 drivers/net/stmmac/stmmac_ethdev.c | 1007 ++++++++++++++++++++++++++++
 drivers/net/stmmac/stmmac_logs.h   |   25 +
 drivers/net/stmmac/stmmac_regs.h   |  302 +++++++++
 drivers/net/stmmac/stmmac_rxtx.c   |  416 ++++++++++++
 9 files changed, 2166 insertions(+)
 create mode 100644 doc/guides/nics/stmmac.rst
 create mode 100644 drivers/net/stmmac/meson.build
 create mode 100644 drivers/net/stmmac/stmmac.h
 create mode 100644 drivers/net/stmmac/stmmac_ethdev.c
 create mode 100644 drivers/net/stmmac/stmmac_logs.h
 create mode 100644 drivers/net/stmmac/stmmac_regs.h
 create mode 100644 drivers/net/stmmac/stmmac_rxtx.c

diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index e20be478f8..b5ca9bddb5 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -68,6 +68,7 @@ Network Interface Controller Drivers
     rnp
     sfc_efx
     softnic
+       stmmac
     sxe2
     tap
     thunderx
diff --git a/doc/guides/nics/stmmac.rst b/doc/guides/nics/stmmac.rst
new file mode 100644
index 0000000000..3cb9cc5af0
--- /dev/null
+++ b/doc/guides/nics/stmmac.rst
@@ -0,0 +1,304 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright 2026 Qualcomm Technologies, Inc.
+
+Stmmac Poll Mode Driver
+=======================
+
+The stmmac PMD (**librte_net_stmmac**) provides poll mode driver
+support for the Synopsys DesignWare Cores Enhanced Quality-of-Service
+(DWC EQoS / GMAC4) Ethernet MAC present on Qualcomm SoCs such as the
+IQ9075 (QCS9075) found on the Shikra EVK platform.
+
+More information on the Qualcomm Shikra platform can be found at
+`Qualcomm Developer Network <https://www.qualcomm.com/developer>`_.
+
+Synopsys DWC EQoS (GMAC4, "stmmac")
+-------------------------------------
+
+This section provides an overview of the Synopsys DWC EQoS MAC
+architecture and how it is integrated into DPDK.
+
+Contents summary
+
+- DWC EQoS overview
+- Driver architecture overview
+- Device configuration and devargs
+
+.. _stmmac_overview:
+
+DWC EQoS Overview
+~~~~~~~~~~~~~~~~~~
+
+The Synopsys DesignWare Cores Enhanced Quality-of-Service (DWC EQoS)
+is a fully integrated 10/100/1000 Mb/s Ethernet MAC core. The "stmmac"
+name follows the Linux kernel driver family
+(``drivers/net/ethernet/stmicro/stmmac``).
+
+The MAC core revision used on the Qualcomm Shikra EVK is GMAC4 v5.30a.
+
+DWC EQoS includes:
+
+- MAC core (GMAC4): frame transmission, reception, MDIO management,
+  address filtering, and hardware checksum offload
+- MTL (MAC Transaction Layer): configurable TX/RX FIFOs with
+  store-and-forward mode; FIFO sizes are read from HW_FEATURE1 at probe
+- DMA engine: AXI system bus with fixed burst, mixed burst, AAL and
+  EAME (>4 GiB IOVA support); per-channel burst (PBL 32, PBLx8, OSP)
+- RGMII IO macro: Qualcomm-specific SerDes block for RGMII-ID timing
+
+The Network and packet I/O component:
+
+- One DMA channel (channel 0) with one TX queue and one RX queue.
+  The PMD operates DMA channel 0 exclusively.
+- The DMA is not cache-coherent on the Shikra platform. The PMD
+  performs explicit DC CVAC/CIVAC cache maintenance on arm64 for both
+  descriptor lines and packet payload. No-op stubs are provided for
+  other architectures.
+
+Stmmac DPDK - Poll Mode Driver Overview
+-----------------------------------------
+
+This section provides an overview of the stmmac PMD for DPDK.
+
+The PMD is probed by the DPDK platform bus and uses ``vfio-platform``
+for MMIO access and DMA mapping. The kernel recovery driver must be
+loaded first to perform clock enable and PHY reset; the PMD then takes
+over and must not be unloaded while the PMD is running.
+
+.. code-block:: console
+
+                            +-------------------+
+                            |  DPDK Application |
+                            +--------+----------+
+                                     |
+                            +--------+----------+
+                            |  stmmac PMD       |
+                            | (librte_net_stmmac)|
+                            +----+----------+---+
+                                 |          |
+                       +---------+--+   +---+---------+
+                       | TX Burst   |   | RX Burst    |
+                       | (DMA ch 0) |   | (DMA ch 0)  |
+                       +---------+--+   +---+---------+
+                                 |          |
+              ===================|==========|================ HARDWARE
+                            +----+----------+---+
+                            |  DWC EQoS (GMAC4) |
+                            |  MAC / MTL / DMA   |
+                            +--------+----------+
+                                     |
+                            +--------+----------+
+                            |  RGMII IO Macro   |
+                            +--------+----------+
+                                     |
+                            +--------+----------+
+                            |  DP83867 PHY      |
+                            +-------------------+
+
+Platform Bus Driver
+~~~~~~~~~~~~~~~~~~~
+
+The stmmac PMD registers as an ``rte_platform_driver``. The platform bus
+scans the kernel sysfs tree for a device whose driver alias matches
+``vfio-platform``. Two MMIO resources are consumed:
+
+- ``resource[0]``: MAC/MTL/DMA register block (``stmmaceth`` in DT)
+- ``resource[1]``: RGMII IO macro register block (``rgmii`` in DT)
+
+The kernel ``vfio-platform`` driver must be bound to the stmmac device
+before launching a DPDK application.
+
+Stmmac NIC Driver (PMD)
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The stmmac PMD implements the standard DPDK ethdev API for the DWC EQoS
+MAC. It directly programs the MAC, MTL and DMA register blocks over
+the MMIO mapping provided by vfio-platform.
+
+- On probe, the PMD reads devargs, maps MMIO resources, sets RGMII IO
+  macro timing, programs DP83867 RGMII internal delays via MMD indirect
+  access, and registers the ethdev.
+- On ``dev_start``, the PMD enables the RGMII functional clock,
+  performs a DMA soft reset, programs AXI sysbus and per-channel burst
+  settings, sets up TX and RX descriptor rings, initialises the MAC
+  and MTL, and starts DMA.
+- Link state is polled from the DP83867 PHY using IEEE 802.3 clause-22
+  MDIO. BMSR is double-read to clear the latched link bit; live speed
+  and duplex come from PHYSTS.
+
+Features
+^^^^^^^^
+
+Features of the stmmac PMD are:
+
+- Single TX queue and single RX queue on DMA channel 0
+- IPv4, TCP and UDP hardware checksum offload (TX and RX)
+- Promiscuous mode
+- MAC address set
+- Link state update via DP83867 PHY MDIO (clause-22)
+- Per-queue packet and byte counters (stats_get / stats_reset)
+- Configurable ring sizes (64..1024 descriptors, multiples of 4)
+- Non-coherent DMA support (arm64 DC CVAC/CIVAC cache maintenance)
+- 36-bit DMA addressing (EAME enabled, mbufs above 4 GiB supported)
+
+Allowing & Blocking
+-------------------
+
+To block a stmmac device from being used by DPDK, the following EAL
+argument can be used:
+
+.. code-block:: console
+
+   <dpdk app> <EAL args> -b "platform:<device_name>" -- ...
+   e.g. -b "platform:a000000.ethernet"
+
+Supported Platforms
+-------------------
+
+- Qualcomm IQ9075 (QCS9075) — Shikra EVK (arm64)
+
+Prerequisites
+-------------
+
+- Follow the DPDK :doc:`/linux_gsg/index` to set up the basic DPDK
+  environment.
+- An arm64 cross-compilation toolchain is required to build for the
+  Shikra EVK target from an x86 host.
+- The kernel ``vfio-platform`` and ``vfio_iommu_type1`` modules must be
+  loaded on the target before launching a DPDK application.
+- The Qualcomm stmmac recovery kernel driver must be loaded first to
+  enable clocks and reset the PHY. The PMD does not perform these steps.
+
+.. note::
+
+   The stmmac PMD is supported on Linux only. It requires the platform
+   bus (``rte_bus_platform``) and ``vfio-platform`` kernel support.
+
+Configuration
+-------------
+
+Devargs
+~~~~~~~
+
+The stmmac PMD accepts the following device arguments:
+
+.. list-table:: stmmac devargs
+   :header-rows: 1
+   :widths: 20 10 70
+
+   * - Parameter
+     - Default
+     - Description
+   * - ``has_emac_ge_3``
+     - ``1``
+     - DMA/MTL channel stride override. Set to ``1`` for EMAC >= 3.0
+       platforms (Shikra IQ9075), which use a 0x1000 channel stride
+       (base 0x8100 for DMA, 0x8000 for MTL). Set to ``0`` for
+       generic dwmac4 layout (base 0x1100, stride 0x80).
+   * - ``mdio_addr``
+     - ``7``
+     - PHY MDIO address. The DP83867 on the Shikra EVK is at address 7.
+       Override if the PHY is at a different address.
+
+Example devarg usage:
+
+.. code-block:: console
+
+   --dev "platform:a000000.ethernet,has_emac_ge_3=1,mdio_addr=7"
+
+Driver Compilation and Testing
+-------------------------------
+
+Refer to the document :doc:`build_and_test` for details.
+
+#. Cross-compile for arm64 (from an x86 host):
+
+   .. code-block:: console
+
+      meson setup build-arm64 \
+          --cross-file config/arm/arm64_armv8_linux_gcc \
+          -Dplatform=generic
+      ninja -C build-arm64
+
+#. Bind the device to ``vfio-platform`` on the target:
+
+   .. code-block:: console
+
+      # Load required kernel modules
+      modprobe vfio
+      modprobe vfio-platform
+      modprobe vfio_iommu_type1
+
+      # Identify the device name from sysfs
+      ls /sys/bus/platform/devices/ | grep ethernet
+      # e.g. a000000.ethernet
+
+      # Bind to vfio-platform
+      echo vfio-platform > 
/sys/bus/platform/devices/a000000.ethernet/driver_override
+      echo a000000.ethernet > /sys/bus/platform/drivers/vfio-platform/bind
+
+      # Verify the bind succeeded
+      ls /sys/bus/platform/devices/a000000.ethernet/driver
+      # Should show: .../vfio-platform
+
+#. Running testpmd on the Shikra EVK:
+
+   .. code-block:: console
+
+      ./<build_dir>/app/dpdk-testpmd \
+          -l 0-3 -n 1 --vdev="" \
+          -- -i --portmask=0x1 --nb-cores=1 --no-flush-rx
+      .....
+      EAL: Registered [platform] bus.
+      EAL: Detected 8 lcore(s)
+      .....
+      Configuring Port 0 (socket 0)
+      Port 0: 02:02:02:02:02:02
+      .....
+      Checking link statuses...
+      Port 0 Link Up - speed 1000 Mbps - full-duplex
+      Done
+      testpmd>
+
+   * Use devarg ``has_emac_ge_3=0`` for non-Shikra platforms with the
+     generic dwmac4 DMA channel layout.
+   * Use devarg ``mdio_addr=<n>`` if the PHY MDIO address differs from
+     the Shikra default of 7.
+
+Limitations
+-----------
+
+Single Queue
+~~~~~~~~~~~~
+
+The stmmac PMD supports one TX queue and one RX queue only (DMA
+channel 0). Multi-queue configurations are not supported in this
+release.
+
+Single-segment Mbufs
+~~~~~~~~~~~~~~~~~~~~~
+
+Only single-segment mbufs are supported. Scatter-gather, TSO, jumbo
+frames, and multi-segment packet transmission are not supported. The
+application must ensure that ``nb_segs == 1`` for every transmitted
+mbuf.
+
+No Interrupt Support
+~~~~~~~~~~~~~~~~~~~~~
+
+The PMD is polling-only. No interrupt-driven TX completion or RX
+notification is supported. The platform bus does not expose an interrupt
+to the PMD.
+
+No Secondary Process Support
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Multi-process operation where I/O is performed by a secondary DPDK
+process is not supported.
+
+Platform Requirement
+~~~~~~~~~~~~~~~~~~~~~
+
+The stmmac PMD is supported on Linux on arm64 only. It has been tested
+on the Qualcomm Shikra EVK (IQ9075 / QCS9075 SoC). Operation on other
+platforms is not guaranteed.
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 4e8ccb945f..4d8549c4f9 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -58,6 +58,7 @@ drivers = [
         'rnp',
         'sfc',
         'softnic',
+        'stmmac',
         'sxe2',
         'tap',
         'thunderx',
diff --git a/drivers/net/stmmac/meson.build b/drivers/net/stmmac/meson.build
new file mode 100644
index 0000000000..fb3ed0e162
--- /dev/null
+++ b/drivers/net/stmmac/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2026 Qualcomm Technologies, Inc.
+
+if not is_linux
+    build = false
+    reason = 'only supported on linux'
+    subdir_done()
+endif
+
+deps += ['bus_platform']
+
+sources = files(
+        'stmmac_ethdev.c',
+        'stmmac_rxtx.c',
+)
diff --git a/drivers/net/stmmac/stmmac.h b/drivers/net/stmmac/stmmac.h
new file mode 100644
index 0000000000..ccfc9309c3
--- /dev/null
+++ b/drivers/net/stmmac/stmmac.h
@@ -0,0 +1,95 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2026 Qualcomm Technologies, Inc.
+ */
+
+#ifndef _STMMAC_H_
+#define _STMMAC_H_
+
+#include <stdint.h>
+
+#include <rte_ether.h>
+#include <rte_ethdev.h>
+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+
+#include "stmmac_regs.h"
+
+#define STMMAC_MAX_Q                   1
+
+#define STMMAC_DEFAULT_RING_SIZE       256
+#define STMMAC_MAX_RING_SIZE           1024    /* Linux DMA_MAX_RX/TX_SIZE */
+
+/*
+ * Descriptor overlay: raw 32-bit words, interpreted through the bit macros
+ * in stmmac_regs.h, as the kernel driver does for the GMAC4 normal descriptor.
+ */
+struct stmmac_desc {
+       uint32_t des[4];
+};
+
+struct stmmac_tx_queue {
+       struct stmmac_desc      *tx_ring;
+       uint64_t                tx_ring_iova;
+       struct rte_mbuf         **sw_ring;
+       uint16_t                nb_desc;
+       uint16_t                tx_tail;        /* next slot to fill */
+       uint16_t                tx_free;        /* next slot to reclaim */
+       uint16_t                queue_id;
+       uint16_t                port_id;
+       void                    *chan_base;     /* mac_base + DMA chan(0) 
offset */
+       const struct rte_memzone *mz;
+       uint64_t                tx_packets;
+       uint64_t                tx_bytes;
+       uint64_t                tx_errors;      /* TDES3 ES seen on reclaim */
+       uint64_t                tx_ring_full;   /* burst stopped: no free slot 
*/
+};
+
+struct stmmac_rx_queue {
+       struct stmmac_desc      *rx_ring;
+       uint64_t                rx_ring_iova;
+       struct rte_mbuf         **sw_ring;
+       uint16_t                nb_desc;
+       uint16_t                rx_tail;        /* next slot to check */
+       uint16_t                rx_refill;      /* next slot to refill */
+       uint16_t                queue_id;
+       uint16_t                port_id;
+       struct rte_mempool      *mb_pool;
+       void                    *chan_base;     /* mac_base + DMA chan(0) 
offset */
+       const struct rte_memzone *mz;
+       uint64_t                rx_packets;
+       uint64_t                rx_bytes;
+       uint64_t                rx_errors;
+       uint64_t                rx_nombuf;
+};
+
+struct stmmac_priv {
+       void                    *mac_base;      /* resource[0]: MAC/MTL/DMA 
block */
+       void                    *rgmii_base;    /* resource[1]: RGMII IO macro 
(NULL if absent) */
+       struct stmmac_chan_addrs chan_addrs;
+
+       struct stmmac_tx_queue  *txq[STMMAC_MAX_Q];
+       struct stmmac_rx_queue  *rxq[STMMAC_MAX_Q];
+
+       struct rte_ether_addr   mac_addr;
+
+       uint16_t                mdio_addr;      /* PHY MDIO address, from 
devarg */
+       int                     has_emac_ge_3;  /* DMA/MTL channel stride 
override */
+
+       int                     link_up;
+       uint32_t                link_speed;
+
+       struct rte_eth_stats    stats;
+};
+
+/* stmmac_rxtx.c */
+uint16_t stmmac_rx_pkt_burst_nc(void *rxq, struct rte_mbuf **rx_pkts,
+                               uint16_t nb_pkts);
+uint16_t stmmac_tx_pkt_burst_nc(void *txq, struct rte_mbuf **tx_pkts,
+                               uint16_t nb_pkts);
+
+uint16_t stmmac_mdio_read(void *mac_base, uint16_t mdio_addr,
+                         uint16_t phyreg);
+void stmmac_mdio_write(void *mac_base, uint16_t mdio_addr,
+                      uint16_t phyreg, uint16_t phydata);
+
+#endif /* _STMMAC_H_ */
diff --git a/drivers/net/stmmac/stmmac_ethdev.c 
b/drivers/net/stmmac/stmmac_ethdev.c
new file mode 100644
index 0000000000..ecbca764a5
--- /dev/null
+++ b/drivers/net/stmmac/stmmac_ethdev.c
@@ -0,0 +1,1007 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2026 Qualcomm Technologies, Inc.
+ *
+ * DPDK poll-mode driver for the Synopsys DWC EQoS (GMAC4, "stmmac") MAC.
+ *
+ * Scope:
+ *   - DMA channel 0 with one TX and one RX queue.
+ *   - Polling only: the platform bus has no interrupt support, so TX and RX
+ *     completion and link state are polled.
+ *   - Single-segment mbufs only; no TSO, PTP, or secondary process support.
+ *   - Link state from the DP83867 PHY (BMSR and PHYSTS), using generic
+ *     IEEE 802.3 clause-22 MDIO.
+ *   - The RGMII IO macro and the DP83867 RGMII delays are configured here.
+ *     The kernel recovery driver handles the clocks and the PHY reset.
+ */
+
+#include <stdlib.h>
+#include <string.h>
+
+#include <bus_platform_driver.h>
+#include <ethdev_driver.h>
+#include <rte_common.h>
+#include <rte_cycles.h>
+#include <rte_ethdev.h>
+#include <rte_io.h>
+#include <rte_kvargs.h>
+#include <rte_malloc.h>
+
+#include "stmmac.h"
+#include "stmmac_regs.h"
+#include "stmmac_logs.h"
+
+RTE_LOG_REGISTER_DEFAULT(stmmac_logtype_pmd, NOTICE);
+
+#define STMMAC_DRIVER_NAME             net_stmmac
+
+#define STMMAC_DEVARG_HAS_EMAC_GE_3    "has_emac_ge_3"
+#define STMMAC_DEVARG_MDIO_ADDR                "mdio_addr"
+
+/* Shikra EVK PHY (DP83867) MDIO address */
+#define STMMAC_DEFAULT_MDIO_ADDR       7
+
+#define STMMAC_DMA_RESET_POLL_US       1000
+#define STMMAC_DMA_RESET_POLLS         1000    /* 1 s in total, as 
dwmac4_dma_reset() */
+
+#define STMMAC_DMA_PBL                 32
+#define STMMAC_DMA_PBL_SHIFT           16
+#define STMMAC_DMA_PBL_MASK            (0x3fu << STMMAC_DMA_PBL_SHIFT)
+
+/* MTL RX/TX queue FIFO size fields (see stmmac_mtl_init) */
+#define STMMAC_ACTIVE_QUEUES           1
+
+/* Qualcomm RGMII IO macro and SDCC registers (offsets within the rgmii 
resource) */
+#define RGMII_IO_MACRO_CONFIG          0x00
+#define RGMII_IO_MACRO_CONFIG2         0x1c
+#define SDCC_HC_REG_DLL_CONFIG         0x04
+#define SDCC_USR_CTL                   0x18
+
+#define RGMII_CONFIG_FUNC_CLK_EN       RTE_BIT32(30)
+#define RGMII_CFG_DDR_MODE             RTE_BIT32(0)
+#define RGMII_CFG_PROG_SWAP            RTE_BIT32(1)
+#define RGMII_CFG_LOOPBACK_EN          RTE_BIT32(2)
+#define RGMII_CFG_BYPASS_TX_ID_EN      RTE_BIT32(3)
+#define RGMII_CFG_POS_NEG_DATA_SEL     RTE_BIT32(23)
+#define RGMII_CFG2_TX_TO_RX_LOOPBACK   RTE_BIT32(13)
+#define RGMII_CFG2_RSVD_CONFIG15       (0x7fffu << 17)
+#define RGMII_CFG2_RX_PROG_SWAP                RTE_BIT32(7)
+#define RGMII_CFG2_DATA_DIV_CLK_SEL    RTE_BIT32(6)
+#define RGMII_CFG2_TX_PHASE_SHIFT_EN   RTE_BIT32(5)
+#define SDCC_DLL_CONFIG_PDN            RTE_BIT32(29)
+#define SDCC_USR_CTL_DDR_BYPASS                RTE_BIT32(30)
+
+/* DP83867 RGMII internal delays, set through the MMD indirect registers */
+#define MII_PHYSID1                    0x02
+#define MII_PHYSID2                    0x03
+#define MII_REGCR                      0x0d
+#define MII_ADDAR                      0x0e
+#define MII_REGCR_ADDR                 0x001f
+#define MII_REGCR_DATA                 0x401f
+#define DP83867_RGMIICTL               0x0032
+#define DP83867_RGMIIDCTL              0x0086
+#define DP83867_RGMII_TX_CLK_DELAY_EN  RTE_BIT32(1)
+#define DP83867_RGMII_RX_CLK_DELAY_EN  RTE_BIT32(0)
+#define DP83867_RGMIIDCTL_2_00_NS      0x7
+#define DP83867_PHYCTRL                        0x10
+#define DP83867_PHY_ID                 0x2000a231
+
+static const char * const stmmac_valid_args[] = {
+       STMMAC_DEVARG_HAS_EMAC_GE_3,
+       STMMAC_DEVARG_MDIO_ADDR,
+       NULL,
+};
+
+static uint64_t dev_rx_offloads_sup = RTE_ETH_RX_OFFLOAD_IPV4_CKSUM;
+static uint64_t dev_tx_offloads_sup = RTE_ETH_TX_OFFLOAD_IPV4_CKSUM |
+                                     RTE_ETH_TX_OFFLOAD_TCP_CKSUM |
+                                     RTE_ETH_TX_OFFLOAD_UDP_CKSUM;
+
+static int
+stmmac_kvarg_handler(const char *key, const char *value, void *opaque)
+{
+       unsigned long v;
+       char *end;
+
+       RTE_SET_USED(key);
+
+       v = strtoul(value, &end, 0);
+       if (*end != '\0')
+               return -EINVAL;
+
+       *(uint32_t *)opaque = (uint32_t)v;
+       return 0;
+}
+
+static void
+stmmac_parse_devargs(struct rte_platform_device *pdev, struct stmmac_priv 
*priv)
+{
+       struct rte_kvargs *kvlist;
+       uint32_t v;
+
+       priv->has_emac_ge_3 = 1;
+       priv->mdio_addr = STMMAC_DEFAULT_MDIO_ADDR;
+
+       if (pdev->device.devargs == NULL ||
+           pdev->device.devargs->args == NULL)
+               return;
+
+       kvlist = rte_kvargs_parse(pdev->device.devargs->args,
+                                 stmmac_valid_args);
+       if (kvlist == NULL)
+               return;
+
+       if (rte_kvargs_count(kvlist, STMMAC_DEVARG_HAS_EMAC_GE_3) > 0) {
+               v = 0;
+               if (rte_kvargs_process(kvlist, STMMAC_DEVARG_HAS_EMAC_GE_3,
+                                      stmmac_kvarg_handler, &v) == 0)
+                       priv->has_emac_ge_3 = (int)v;
+       }
+
+       if (rte_kvargs_count(kvlist, STMMAC_DEVARG_MDIO_ADDR) > 0) {
+               v = 0;
+               if (rte_kvargs_process(kvlist, STMMAC_DEVARG_MDIO_ADDR,
+                                      stmmac_kvarg_handler, &v) == 0)
+                       priv->mdio_addr = (uint16_t)v;
+       }
+
+       rte_kvargs_free(kvlist);
+}
+
+/*
+ * Keep the RGMII IO macro clock enabled. Without FUNC_CLK_EN the DMA reset
+ * can time out because the PHY clock is missing.
+ */
+static void
+stmmac_rgmii_func_clk_en(struct stmmac_priv *priv)
+{
+       void *reg;
+
+       if (priv->rgmii_base == NULL) {
+               STMMAC_PMD_WARN("RGMII region not mapped, FUNC_CLK_EN not set");
+               return;
+       }
+       reg = (uint8_t *)priv->rgmii_base + RGMII_IO_MACRO_CONFIG;
+       rte_write32(rte_read32(reg) | RGMII_CONFIG_FUNC_CLK_EN, reg);
+}
+
+static void
+stmmac_rgmii_rmw(struct stmmac_priv *priv, uint32_t off, uint32_t clr, 
uint32_t set)
+{
+       void *reg = (uint8_t *)priv->rgmii_base + off;
+
+       rte_write32((rte_read32(reg) & ~clr) | set, reg);
+}
+
+/*
+ * RGMII IO macro setup for rgmii-id at 1 Gb/s, as the kernel does at link-up.
+ * The PHY adds the RGMII delays, so the MAC-side DLL is powered down and the
+ * DDR path is bypassed. TX-to-RX loopback is disabled so RX data comes from
+ * the PHY.
+ */
+static void
+stmmac_rgmii_id_1g_init(struct stmmac_priv *priv)
+{
+       if (priv->rgmii_base == NULL)
+               return;
+
+       stmmac_rgmii_rmw(priv, SDCC_HC_REG_DLL_CONFIG, 0, SDCC_DLL_CONFIG_PDN);
+       stmmac_rgmii_rmw(priv, SDCC_USR_CTL, 0, SDCC_USR_CTL_DDR_BYPASS);
+
+       stmmac_rgmii_rmw(priv, RGMII_IO_MACRO_CONFIG2, 
RGMII_CFG2_TX_TO_RX_LOOPBACK, 0);
+       stmmac_rgmii_rmw(priv, RGMII_IO_MACRO_CONFIG,
+                        RGMII_CFG_POS_NEG_DATA_SEL | RGMII_CFG_PROG_SWAP |
+                        RGMII_CFG_LOOPBACK_EN,
+                        RGMII_CFG_DDR_MODE | RGMII_CFG_BYPASS_TX_ID_EN);
+       stmmac_rgmii_rmw(priv, RGMII_IO_MACRO_CONFIG2,
+                        RGMII_CFG2_DATA_DIV_CLK_SEL | 
RGMII_CFG2_TX_PHASE_SHIFT_EN |
+                        RGMII_CFG2_RSVD_CONFIG15,
+                        RGMII_CFG2_RX_PROG_SWAP);
+}
+
+static uint16_t
+stmmac_phy_mmd_read(struct stmmac_priv *priv, uint16_t reg)
+{
+       stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_REGCR, 
MII_REGCR_ADDR);
+       stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_ADDAR, reg);
+       stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_REGCR, 
MII_REGCR_DATA);
+       return stmmac_mdio_read(priv->mac_base, priv->mdio_addr, MII_ADDAR);
+}
+
+static void
+stmmac_phy_mmd_write(struct stmmac_priv *priv, uint16_t reg, uint16_t val)
+{
+       stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_REGCR, 
MII_REGCR_ADDR);
+       stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_ADDAR, reg);
+       stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_REGCR, 
MII_REGCR_DATA);
+       stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_ADDAR, val);
+}
+
+/*
+ * Set the DP83867 RGMII internal delays to 2.00 ns on both TX and RX. The
+ * kernel does the same for phy-mode "rgmii-id". The MAC does not add these
+ * delays, so without them RX frames are sampled off the data eye.
+ */
+static void
+stmmac_phy_rgmii_id_init(struct stmmac_priv *priv)
+{
+       uint16_t id1, id2, ctl;
+
+       id1 = stmmac_mdio_read(priv->mac_base, priv->mdio_addr, MII_PHYSID1);
+       id2 = stmmac_mdio_read(priv->mac_base, priv->mdio_addr, MII_PHYSID2);
+       if (id1 == 0xffff || id2 == 0xffff) {
+               STMMAC_PMD_WARN("PHY not responding on MDIO, RGMII delays not 
set");
+               return;
+       }
+       if (((uint32_t)id1 << 16 | id2) != DP83867_PHY_ID)
+               STMMAC_PMD_WARN("unexpected PHY ID 0x%04x%04x", id1, id2);
+
+       ctl = stmmac_phy_mmd_read(priv, DP83867_RGMIICTL);
+       stmmac_phy_mmd_write(priv, DP83867_RGMIICTL,
+                            ctl | DP83867_RGMII_TX_CLK_DELAY_EN |
+                            DP83867_RGMII_RX_CLK_DELAY_EN);
+       stmmac_phy_mmd_write(priv, DP83867_RGMIIDCTL,
+                            DP83867_RGMIIDCTL_2_00_NS |
+                            (DP83867_RGMIIDCTL_2_00_NS << 4));
+}
+
+/*
+ * DMA soft reset, as dwmac4_dma_reset(): poll the SWR bit for up to 1 s.
+ */
+static int
+stmmac_dma_reset(void *mac_base)
+{
+       void *reg = (uint8_t *)mac_base + DMA_BUS_MODE;
+       unsigned int i;
+
+       rte_write32(rte_read32(reg) | DMA_BUS_MODE_SFT_RESET, reg);
+
+       for (i = 0; i < STMMAC_DMA_RESET_POLLS; i++) {
+               if (!(rte_read32(reg) & DMA_BUS_MODE_SFT_RESET))
+                       return 0;
+               rte_delay_us(STMMAC_DMA_RESET_POLL_US);
+       }
+
+       STMMAC_PMD_ERR("DMA soft reset timed out");
+       return -ETIMEDOUT;
+}
+
+/*
+ * DMA system and AXI bus configuration. Fixed burst, mixed burst and AAL are
+ * enabled, with BLEN16 and the maximum outstanding requests. EAME is set
+ * because the rings and mbufs are above 4 GiB.
+ */
+static void
+stmmac_dma_sysbus_init(void *mac_base)
+{
+       void *sysbus_reg = (uint8_t *)mac_base + DMA_SYS_BUS_MODE;
+       void *axi_reg = (uint8_t *)mac_base + DMA_AXI_BUS_MODE;
+       uint32_t value;
+
+       value = rte_read32(sysbus_reg);
+       value |= DMA_SYS_BUS_FB | DMA_SYS_BUS_MB | DMA_SYS_BUS_AAL;
+       value |= DMA_SYS_BUS_EAME;
+       rte_write32(value, sysbus_reg);
+
+       value = rte_read32(axi_reg);
+       value &= ~(DMA_AXI_WR_OSR_LMT | DMA_AXI_RD_OSR_LMT);
+       value |= (DMA_AXI_OSR_MAX << DMA_AXI_WR_OSR_LMT_SHIFT) & 
DMA_AXI_WR_OSR_LMT;
+       value |= (DMA_AXI_OSR_MAX << DMA_AXI_RD_OSR_LMT_SHIFT) & 
DMA_AXI_RD_OSR_LMT;
+       value |= DMA_AXI_BLEN16;
+       rte_write32(value, axi_reg);
+}
+
+/*
+ * Per-channel DMA burst settings: PBL 32, PBLx8 on, and OSP on TX. The DMA is
+ * too slow to keep up with the wire if these stay at the reset values.
+ */
+static void
+stmmac_dma_chan_cfg(void *chan_base)
+{
+       void *ch_ctrl = (uint8_t *)chan_base + DMA_CHAN_OFF_CONTROL;
+       void *tx_ctrl = (uint8_t *)chan_base + DMA_CHAN_OFF_TX_CONTROL;
+       void *rx_ctrl = (uint8_t *)chan_base + DMA_CHAN_OFF_RX_CONTROL;
+       uint32_t v;
+
+       v = rte_read32(ch_ctrl);
+       rte_write32(v | DMA_CONTROL_PBLX8, ch_ctrl);
+
+       v = rte_read32(tx_ctrl);
+       v &= ~STMMAC_DMA_PBL_MASK;
+       v |= ((uint32_t)STMMAC_DMA_PBL << STMMAC_DMA_PBL_SHIFT) | 
DMA_CHAN_TX_CONTROL_OSP;
+       rte_write32(v, tx_ctrl);
+
+       v = rte_read32(rx_ctrl);
+       v &= ~STMMAC_DMA_PBL_MASK;
+       v |= (uint32_t)STMMAC_DMA_PBL << STMMAC_DMA_PBL_SHIFT;
+       rte_write32(v, rx_ctrl);
+}
+
+static int
+stmmac_tx_ring_setup(struct stmmac_tx_queue *txq)
+{
+       uint8_t *chan = txq->chan_base;
+       unsigned int i;
+
+       for (i = 0; i < txq->nb_desc; i++)
+               memset(&txq->tx_ring[i], 0, sizeof(struct stmmac_desc));
+       /* the non-coherent DMA must see the zeroed ring */
+       stmmac_dcache_clean(txq->tx_ring, (size_t)txq->nb_desc * sizeof(struct 
stmmac_desc));
+
+       rte_write32((uint32_t)(txq->tx_ring_iova >> 32), chan + 
DMA_CHAN_OFF_TX_BASE_ADDR_HI);
+       rte_write32((uint32_t)txq->tx_ring_iova, chan + 
DMA_CHAN_OFF_TX_BASE_ADDR);
+       /* the ring length register holds nb_desc - 1 */
+       rte_write32(txq->nb_desc - 1, chan + DMA_CHAN_OFF_TX_RING_LEN);
+       rte_write32((uint32_t)txq->tx_ring_iova, chan + 
DMA_CHAN_OFF_TX_END_ADDR);
+
+       return 0;
+}
+
+static int
+stmmac_rx_ring_setup(struct stmmac_rx_queue *rxq)
+{
+       uint8_t *chan = rxq->chan_base;
+       unsigned int i;
+       uint64_t tail;
+       uint32_t bufsz, v;
+
+       for (i = 0; i < rxq->nb_desc; i++) {
+               struct rte_mbuf *mb = rte_pktmbuf_alloc(rxq->mb_pool);
+               uint64_t iova;
+
+               if (mb == NULL) {
+                       STMMAC_PMD_ERR("failed to allocate RX mbuf %u", i);
+                       return -ENOMEM;
+               }
+
+               iova = rte_pktmbuf_iova(mb);
+               rxq->sw_ring[i] = mb;
+               /* drop any dirty lines over the buffer before the DMA writes 
it */
+               stmmac_dcache_flush(rte_pktmbuf_mtod(mb, void *),
+                                   rte_pktmbuf_data_room_size(rxq->mb_pool) -
+                                   RTE_PKTMBUF_HEADROOM);
+               rxq->rx_ring[i].des[0] = (uint32_t)iova;
+               rxq->rx_ring[i].des[1] = (uint32_t)(iova >> 32);
+               rxq->rx_ring[i].des[2] = 0;
+               rxq->rx_ring[i].des[3] = RDES3_OWN | RDES3_BUFFER1_VALID_ADDR |
+                                        RDES3_INT_ON_COMPLETION_EN;
+       }
+       /* the non-coherent DMA must see every OWN=1 descriptor before RXTAIL 
is set */
+       stmmac_dcache_clean(rxq->rx_ring, (size_t)rxq->nb_desc * sizeof(struct 
stmmac_desc));
+
+       /* RX buffer size, which must be non-zero and a multiple of 4 */
+       bufsz = (rte_pktmbuf_data_room_size(rxq->mb_pool) - 
RTE_PKTMBUF_HEADROOM) & ~3u;
+       v = rte_read32(chan + DMA_CHAN_OFF_RX_CONTROL);
+       v = (v & ~DMA_CONTROL_RBSZ_MASK) | ((bufsz << DMA_CONTROL_RBSZ_SHIFT) &
+                                           DMA_CONTROL_RBSZ_MASK);
+       rte_write32(v, chan + DMA_CHAN_OFF_RX_CONTROL);
+
+       rte_write32((uint32_t)(rxq->rx_ring_iova >> 32), chan + 
DMA_CHAN_OFF_RX_BASE_ADDR_HI);
+       rte_write32((uint32_t)rxq->rx_ring_iova, chan + 
DMA_CHAN_OFF_RX_BASE_ADDR);
+       /* the ring length register holds nb_desc - 1 */
+       rte_write32(rxq->nb_desc - 1, chan + DMA_CHAN_OFF_RX_RING_LEN);
+       tail = rxq->rx_ring_iova + (uint64_t)rxq->nb_desc * sizeof(struct 
stmmac_desc);
+       rte_write32((uint32_t)tail, chan + DMA_CHAN_OFF_RX_END_ADDR);
+
+       return 0;
+}
+
+/*
+ * MTL operating mode for the single active queue. Store-and-forward is
+ * enabled, and the queue FIFO sizes (RQS/TQS) are programmed from
+ * HW_FEATURE1. The reset value of 256 bytes is too small for 
store-and-forward.
+ * Queue 0 owns the whole FIFO and maps to DMA channel 0. Flow control is not
+ * enabled.
+ */
+static void
+stmmac_mtl_init(void *mac_base, const struct stmmac_chan_addrs *addrs)
+{
+       void *tx_op = (uint8_t *)mac_base + MTL_CHAN_TX_OP_MODE(addrs, 0);
+       void *rx_op = (uint8_t *)mac_base + MTL_CHAN_RX_OP_MODE(addrs, 0);
+       void *rxq_map = (uint8_t *)mac_base + MTL_RXQ_DMA_MAP0;
+       uint32_t hwf1 = rte_read32((uint8_t *)mac_base + GMAC_HW_FEATURE1);
+       uint32_t rx_fifo = 128u << (hwf1 & GMAC_HWF1_RXFIFOSIZE_MASK);
+       uint32_t tx_fifo = 128u << ((hwf1 >> GMAC_HWF1_TXFIFOSIZE_SHIFT) &
+                                   GMAC_HWF1_TXFIFOSIZE_MASK);
+       uint32_t rqs = rx_fifo / STMMAC_ACTIVE_QUEUES >= 256 ?
+                      rx_fifo / STMMAC_ACTIVE_QUEUES / 256 - 1 : 0;
+       uint32_t tqs = tx_fifo / STMMAC_ACTIVE_QUEUES >= 256 ?
+                      tx_fifo / STMMAC_ACTIVE_QUEUES / 256 - 1 : 0;
+       uint32_t value;
+
+       if (rqs > (MTL_OP_MODE_RQS_MASK >> MTL_OP_MODE_RQS_SHIFT))
+               rqs = MTL_OP_MODE_RQS_MASK >> MTL_OP_MODE_RQS_SHIFT;
+       if (tqs > (MTL_OP_MODE_TQS_MASK >> MTL_OP_MODE_TQS_SHIFT))
+               tqs = MTL_OP_MODE_TQS_MASK >> MTL_OP_MODE_TQS_SHIFT;
+
+       value = rte_read32(tx_op);
+       value &= ~MTL_OP_MODE_TQS_MASK;
+       value |= MTL_OP_MODE_TSF | MTL_OP_MODE_TXQEN | (tqs << 
MTL_OP_MODE_TQS_SHIFT);
+       rte_write32(value, tx_op);
+
+       value = rte_read32(rx_op);
+       value &= ~MTL_OP_MODE_RQS_MASK;
+       value |= MTL_OP_MODE_RSF | (rqs << MTL_OP_MODE_RQS_SHIFT);
+       rte_write32(value, rx_op);
+
+       /* RX queue 0 maps to DMA channel 0 */
+       value = rte_read32(rxq_map);
+       value &= ~0xfu;
+       rte_write32(value, rxq_map);
+}
+
+static void
+stmmac_mac_init(void *mac_base, struct rte_ether_addr *mac_addr)
+{
+       void *config_reg = (uint8_t *)mac_base + GMAC_CONFIG;
+       uint32_t value;
+       uint32_t addr_lo, addr_hi;
+
+       addr_lo = mac_addr->addr_bytes[0] | (mac_addr->addr_bytes[1] << 8) |
+                 (mac_addr->addr_bytes[2] << 16) | (mac_addr->addr_bytes[3] << 
24);
+       addr_hi = mac_addr->addr_bytes[4] | (mac_addr->addr_bytes[5] << 8);
+
+       rte_write32(addr_lo, (uint8_t *)mac_base + GMAC_ADDR_LOW(0));
+       rte_write32(addr_hi, (uint8_t *)mac_base + GMAC_ADDR_HIGH(0));
+
+       value = rte_read32(config_reg);
+       value |= GMAC_CONFIG_IPC | GMAC_CONFIG_ACS;
+       value |= GMAC_CONFIG_TE | GMAC_CONFIG_RE;
+       rte_write32(value, config_reg);
+}
+
+static void
+stmmac_dma_start(struct stmmac_priv *priv)
+{
+       uint8_t *chan = (uint8_t *)priv->mac_base + 
stmmac_dma_chan_base(&priv->chan_addrs, 0);
+       void *tx_ctrl = chan + DMA_CHAN_OFF_TX_CONTROL;
+       void *rx_ctrl = chan + DMA_CHAN_OFF_RX_CONTROL;
+       uint32_t value;
+
+       value = rte_read32(tx_ctrl);
+       value |= DMA_CONTROL_ST;
+       rte_write32(value, tx_ctrl);
+
+       value = rte_read32(rx_ctrl);
+       value &= ~DMA_CONTROL_RPF;
+       value |= DMA_CONTROL_SR;
+       rte_write32(value, rx_ctrl);
+}
+
+static void
+stmmac_dma_stop(struct stmmac_priv *priv)
+{
+       uint8_t *chan = (uint8_t *)priv->mac_base + 
stmmac_dma_chan_base(&priv->chan_addrs, 0);
+       void *tx_ctrl = chan + DMA_CHAN_OFF_TX_CONTROL;
+       void *rx_ctrl = chan + DMA_CHAN_OFF_RX_CONTROL;
+       uint32_t value;
+
+       value = rte_read32(tx_ctrl);
+       value &= ~DMA_CONTROL_ST;
+       rte_write32(value, tx_ctrl);
+
+       value = rte_read32(rx_ctrl);
+       value |= DMA_CONTROL_RPF;
+       value &= ~DMA_CONTROL_SR;
+       rte_write32(value, rx_ctrl);
+}
+
+static int
+stmmac_eth_configure(struct rte_eth_dev *dev)
+{
+       RTE_SET_USED(dev);
+       return 0;
+}
+
+static int
+stmmac_eth_start(struct rte_eth_dev *dev)
+{
+       struct stmmac_priv *priv = dev->data->dev_private;
+       int ret;
+
+       if (priv->rxq[0] == NULL || priv->txq[0] == NULL) {
+               STMMAC_PMD_ERR("queues not set up");
+               return -EINVAL;
+       }
+
+       /* the RGMII clock must run before the DMA reset */
+       stmmac_rgmii_func_clk_en(priv);
+       stmmac_rgmii_id_1g_init(priv);
+       stmmac_phy_rgmii_id_init(priv);
+
+       ret = stmmac_dma_reset(priv->mac_base);
+       if (ret != 0)
+               return ret;
+
+       stmmac_dma_sysbus_init(priv->mac_base);
+       stmmac_dma_chan_cfg((uint8_t *)priv->mac_base +
+                           stmmac_dma_chan_base(&priv->chan_addrs, 0));
+
+       ret = stmmac_tx_ring_setup(priv->txq[0]);
+       if (ret != 0)
+               return ret;
+
+       ret = stmmac_rx_ring_setup(priv->rxq[0]);
+       if (ret != 0)
+               return ret;
+
+       stmmac_mac_init(priv->mac_base, &priv->mac_addr);
+
+       /* enable RX queue 0 in generic (non-DCB) mode */
+       {
+               void *rxq_ctrl = (uint8_t *)priv->mac_base + GMAC_RXQ_CTRL0;
+               uint32_t v = rte_read32(rxq_ctrl);
+
+               rte_write32((v & ~0x3u) | 0x2u, rxq_ctrl);
+       }
+
+       stmmac_mtl_init(priv->mac_base, &priv->chan_addrs);
+       stmmac_dma_start(priv);
+
+       dev->rx_pkt_burst = stmmac_rx_pkt_burst_nc;
+       dev->tx_pkt_burst = stmmac_tx_pkt_burst_nc;
+
+       dev->data->dev_started = 1;
+       dev->data->rx_queue_state[0] = RTE_ETH_QUEUE_STATE_STARTED;
+       dev->data->tx_queue_state[0] = RTE_ETH_QUEUE_STATE_STARTED;
+
+       return 0;
+}
+
+static int
+stmmac_eth_stop(struct rte_eth_dev *dev)
+{
+       struct stmmac_priv *priv = dev->data->dev_private;
+
+       dev->data->dev_started = 0;
+       stmmac_dma_stop(priv);
+       dev->data->rx_queue_state[0] = RTE_ETH_QUEUE_STATE_STOPPED;
+       dev->data->tx_queue_state[0] = RTE_ETH_QUEUE_STATE_STOPPED;
+       return 0;
+}
+
+static void
+stmmac_queue_release(struct stmmac_priv *priv)
+{
+       if (priv->txq[0] != NULL) {
+               rte_free(priv->txq[0]->sw_ring);
+               rte_free(priv->txq[0]);
+               priv->txq[0] = NULL;
+       }
+       if (priv->rxq[0] != NULL) {
+               unsigned int i;
+
+               for (i = 0; i < priv->rxq[0]->nb_desc; i++)
+                       rte_pktmbuf_free(priv->rxq[0]->sw_ring[i]);
+               rte_free(priv->rxq[0]->sw_ring);
+               rte_free(priv->rxq[0]);
+               priv->rxq[0] = NULL;
+       }
+}
+
+static int
+stmmac_eth_close(struct rte_eth_dev *dev)
+{
+       struct stmmac_priv *priv = dev->data->dev_private;
+
+       stmmac_queue_release(priv);
+
+       return 0;
+}
+
+/*
+ * Link state. The DP83867 BMSR link bit is latched low, so BMSR is read twice
+ * to clear the latch. The live link state and the speed and duplex come from
+ * PHYSTS, which is not latched. BMSR is used only if PHYSTS does not respond.
+ */
+static int
+stmmac_eth_link_update(struct rte_eth_dev *dev, int wait_to_complete)
+{
+       struct stmmac_priv *priv = dev->data->dev_private;
+       struct rte_eth_link link;
+       uint16_t bmsr, physts;
+
+       RTE_SET_USED(wait_to_complete);
+
+       memset(&link, 0, sizeof(link));
+
+       /* the first read clears the latched link bit */
+       (void)stmmac_mdio_read(priv->mac_base, priv->mdio_addr, 
STMMAC_MII_BMSR);
+       bmsr = stmmac_mdio_read(priv->mac_base, priv->mdio_addr, 
STMMAC_MII_BMSR);
+       physts = stmmac_mdio_read(priv->mac_base, priv->mdio_addr, 
DP83867_PHYSTS);
+
+       if (physts != 0xffff)
+               priv->link_up = !!(physts & DP83867_PHYSTS_LINK_BIT);
+       else
+               priv->link_up = !!(bmsr & STMMAC_MII_BMSR_LSTATUS);
+       link.link_status = priv->link_up;
+
+       if (physts != 0xffff && priv->link_up) {
+               uint16_t spd = physts & DP83867_PHYSTS_SPEED_MASK;
+
+               if (spd == DP83867_PHYSTS_SPEED_1G)
+                       link.link_speed = RTE_ETH_SPEED_NUM_1G;
+               else if (spd == DP83867_PHYSTS_SPEED_100M)
+                       link.link_speed = RTE_ETH_SPEED_NUM_100M;
+               else
+                       link.link_speed = RTE_ETH_SPEED_NUM_10M;
+
+               link.link_duplex = (physts & DP83867_PHYSTS_DUPLEX_BIT) ?
+                       RTE_ETH_LINK_FULL_DUPLEX : RTE_ETH_LINK_HALF_DUPLEX;
+       } else {
+               link.link_speed = RTE_ETH_SPEED_NUM_1G;
+               link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+       }
+
+       link.link_autoneg = RTE_ETH_LINK_AUTONEG;
+
+       return rte_eth_linkstatus_set(dev, &link);
+}
+
+static int
+stmmac_promiscuous_enable(struct rte_eth_dev *dev)
+{
+       struct stmmac_priv *priv = dev->data->dev_private;
+       void *filter_reg = (uint8_t *)priv->mac_base + GMAC_PACKET_FILTER;
+
+       rte_write32(rte_read32(filter_reg) | GMAC_PACKET_FILTER_PR, filter_reg);
+
+       return 0;
+}
+
+static int
+stmmac_promiscuous_disable(struct rte_eth_dev *dev)
+{
+       struct stmmac_priv *priv = dev->data->dev_private;
+       void *filter_reg = (uint8_t *)priv->mac_base + GMAC_PACKET_FILTER;
+
+       rte_write32(rte_read32(filter_reg) & ~GMAC_PACKET_FILTER_PR, 
filter_reg);
+
+       return 0;
+}
+
+static int
+stmmac_mac_addr_set(struct rte_eth_dev *dev, struct rte_ether_addr *addr)
+{
+       struct stmmac_priv *priv = dev->data->dev_private;
+
+       rte_ether_addr_copy(addr, &priv->mac_addr);
+
+       rte_write32(addr->addr_bytes[0] | (addr->addr_bytes[1] << 8) |
+                   (addr->addr_bytes[2] << 16) | (addr->addr_bytes[3] << 24),
+                   (uint8_t *)priv->mac_base + GMAC_ADDR_LOW(0));
+       rte_write32(addr->addr_bytes[4] | (addr->addr_bytes[5] << 8),
+                   (uint8_t *)priv->mac_base + GMAC_ADDR_HIGH(0));
+
+       return 0;
+}
+
+static int
+stmmac_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats,
+                struct eth_queue_stats *qstats)
+{
+       struct stmmac_priv *priv = dev->data->dev_private;
+
+       RTE_SET_USED(qstats);
+       memcpy(stats, &priv->stats, sizeof(*stats));
+
+       /* the queue counters are updated by the burst functions */
+       if (priv->rxq[0] != NULL) {
+               stats->ipackets = priv->rxq[0]->rx_packets;
+               stats->ibytes = priv->rxq[0]->rx_bytes;
+               stats->ierrors = priv->rxq[0]->rx_errors;
+               stats->rx_nombuf = priv->rxq[0]->rx_nombuf;
+       }
+       if (priv->txq[0] != NULL) {
+               stats->opackets = priv->txq[0]->tx_packets;
+               stats->obytes = priv->txq[0]->tx_bytes;
+               stats->oerrors = priv->txq[0]->tx_errors;
+       }
+
+       return 0;
+}
+
+static int
+stmmac_stats_reset(struct rte_eth_dev *dev)
+{
+       struct stmmac_priv *priv = dev->data->dev_private;
+
+       memset(&priv->stats, 0, sizeof(priv->stats));
+       if (priv->rxq[0] != NULL) {
+               priv->rxq[0]->rx_packets = 0;
+               priv->rxq[0]->rx_bytes = 0;
+               priv->rxq[0]->rx_errors = 0;
+               priv->rxq[0]->rx_nombuf = 0;
+       }
+       if (priv->txq[0] != NULL) {
+               priv->txq[0]->tx_packets = 0;
+               priv->txq[0]->tx_bytes = 0;
+               priv->txq[0]->tx_errors = 0;
+               priv->txq[0]->tx_ring_full = 0;
+       }
+
+       return 0;
+}
+
+static int
+stmmac_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info 
*dev_info)
+{
+       RTE_SET_USED(dev);
+
+       dev_info->max_rx_queues = STMMAC_MAX_Q;
+       dev_info->max_tx_queues = STMMAC_MAX_Q;
+       dev_info->min_rx_bufsize = RTE_ETHER_MIN_LEN;
+       dev_info->max_rx_pktlen = RTE_ETHER_MAX_LEN;
+       dev_info->rx_offload_capa = dev_rx_offloads_sup;
+       dev_info->tx_offload_capa = dev_tx_offloads_sup;
+
+       /*
+        * Ring sizes follow the Linux limits (64..1024). Sizes must be a
+        * multiple of 4, one descriptor cache line. One segment per packet.
+        */
+       dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+               .nb_max = STMMAC_MAX_RING_SIZE,
+               .nb_min = 64,
+               .nb_align = 4,
+               .nb_seg_max = 1,
+               .nb_mtu_seg_max = 1,
+       };
+       dev_info->tx_desc_lim = dev_info->rx_desc_lim;
+       dev_info->default_rxportconf.ring_size = STMMAC_MAX_RING_SIZE;
+       dev_info->default_txportconf.ring_size = STMMAC_MAX_RING_SIZE;
+
+       return 0;
+}
+
+static int
+stmmac_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id,
+                     uint16_t nb_rx_desc, unsigned int socket_id,
+                     const struct rte_eth_rxconf *rx_conf,
+                     struct rte_mempool *mb_pool)
+{
+       struct stmmac_priv *priv = dev->data->dev_private;
+       struct stmmac_rx_queue *rxq;
+       const struct rte_memzone *mz;
+       size_t ring_size;
+
+       RTE_SET_USED(rx_conf);
+
+       if (rx_queue_id >= STMMAC_MAX_Q) {
+               STMMAC_PMD_ERR("invalid rx queue id %u, max %u",
+                              rx_queue_id, STMMAC_MAX_Q);
+               return -EINVAL;
+       }
+
+       if (nb_rx_desc == 0)
+               nb_rx_desc = STMMAC_DEFAULT_RING_SIZE;
+
+       rxq = rte_zmalloc_socket("stmmac_rxq", sizeof(*rxq),
+                                RTE_CACHE_LINE_SIZE, socket_id);
+       if (rxq == NULL)
+               return -ENOMEM;
+
+       ring_size = nb_rx_desc * sizeof(struct stmmac_desc);
+       mz = rte_eth_dma_zone_reserve(dev, "rx_ring", rx_queue_id, ring_size,
+                                     RTE_CACHE_LINE_SIZE, socket_id);
+       if (mz == NULL) {
+               rte_free(rxq);
+               return -ENOMEM;
+       }
+
+       rxq->sw_ring = rte_zmalloc_socket("stmmac_rx_sw_ring",
+                       nb_rx_desc * sizeof(struct rte_mbuf *),
+                       RTE_CACHE_LINE_SIZE, socket_id);
+       if (rxq->sw_ring == NULL) {
+               rte_free(rxq);
+               return -ENOMEM;
+       }
+
+       rxq->rx_ring = mz->addr;
+       rxq->rx_ring_iova = mz->iova;
+       rxq->nb_desc = nb_rx_desc;
+       rxq->rx_tail = 0;
+       rxq->rx_refill = 0;
+       rxq->queue_id = rx_queue_id;
+       rxq->port_id = dev->data->port_id;
+       rxq->mb_pool = mb_pool;
+       rxq->chan_base = (uint8_t *)priv->mac_base +
+               stmmac_dma_chan_base(&priv->chan_addrs, 0);
+       rxq->mz = mz;
+
+       priv->rxq[rx_queue_id] = rxq;
+       dev->data->rx_queues[rx_queue_id] = rxq;
+       return 0;
+}
+
+static int
+stmmac_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id,
+                     uint16_t nb_tx_desc, unsigned int socket_id,
+                     const struct rte_eth_txconf *tx_conf)
+{
+       struct stmmac_priv *priv = dev->data->dev_private;
+       struct stmmac_tx_queue *txq;
+       const struct rte_memzone *mz;
+       size_t ring_size;
+
+       RTE_SET_USED(tx_conf);
+
+       if (tx_queue_id >= STMMAC_MAX_Q) {
+               STMMAC_PMD_ERR("invalid tx queue id %u, max %u",
+                              tx_queue_id, STMMAC_MAX_Q);
+               return -EINVAL;
+       }
+
+       if (nb_tx_desc == 0)
+               nb_tx_desc = STMMAC_DEFAULT_RING_SIZE;
+       /* the TX burst writes whole descriptor cache lines: the ring must be a 
multiple of 4 */
+       if (nb_tx_desc % STMMAC_DESCS_PER_LINE) {
+               STMMAC_PMD_ERR("nb_tx_desc %u must be a multiple of %u",
+                              nb_tx_desc, STMMAC_DESCS_PER_LINE);
+               return -EINVAL;
+       }
+
+       txq = rte_zmalloc_socket("stmmac_txq", sizeof(*txq),
+                                RTE_CACHE_LINE_SIZE, socket_id);
+       if (txq == NULL)
+               return -ENOMEM;
+
+       ring_size = nb_tx_desc * sizeof(struct stmmac_desc);
+       mz = rte_eth_dma_zone_reserve(dev, "tx_ring", tx_queue_id, ring_size,
+                                     RTE_CACHE_LINE_SIZE, socket_id);
+       if (mz == NULL) {
+               rte_free(txq);
+               return -ENOMEM;
+       }
+
+       txq->sw_ring = rte_zmalloc_socket("stmmac_tx_sw_ring",
+                       nb_tx_desc * sizeof(struct rte_mbuf *),
+                       RTE_CACHE_LINE_SIZE, socket_id);
+       if (txq->sw_ring == NULL) {
+               rte_free(txq);
+               return -ENOMEM;
+       }
+
+       txq->tx_ring = mz->addr;
+       txq->tx_ring_iova = mz->iova;
+       txq->nb_desc = nb_tx_desc;
+       txq->tx_tail = 0;
+       txq->tx_free = 0;
+       txq->queue_id = tx_queue_id;
+       txq->port_id = dev->data->port_id;
+       txq->chan_base = (uint8_t *)priv->mac_base +
+               stmmac_dma_chan_base(&priv->chan_addrs, 0);
+       txq->mz = mz;
+
+       priv->txq[tx_queue_id] = txq;
+       dev->data->tx_queues[tx_queue_id] = txq;
+       return 0;
+}
+
+static void
+stmmac_rxq_info_get(struct rte_eth_dev *dev, uint16_t qid,
+                   struct rte_eth_rxq_info *qinfo)
+{
+       struct stmmac_rx_queue *rxq = dev->data->rx_queues[qid];
+
+       qinfo->mp = rxq->mb_pool;
+       qinfo->nb_desc = rxq->nb_desc;
+}
+
+static void
+stmmac_txq_info_get(struct rte_eth_dev *dev, uint16_t qid,
+                   struct rte_eth_txq_info *qinfo)
+{
+       struct stmmac_tx_queue *txq = dev->data->tx_queues[qid];
+
+       qinfo->nb_desc = txq->nb_desc;
+}
+
+static const struct eth_dev_ops stmmac_eth_ops = {
+       .dev_configure          = stmmac_eth_configure,
+       .dev_start              = stmmac_eth_start,
+       .dev_stop               = stmmac_eth_stop,
+       .dev_close              = stmmac_eth_close,
+       .link_update            = stmmac_eth_link_update,
+       .promiscuous_enable     = stmmac_promiscuous_enable,
+       .promiscuous_disable    = stmmac_promiscuous_disable,
+       .mac_addr_set           = stmmac_mac_addr_set,
+       .stats_get              = stmmac_stats_get,
+       .stats_reset            = stmmac_stats_reset,
+       .dev_infos_get          = stmmac_dev_infos_get,
+       .rx_queue_setup         = stmmac_rx_queue_setup,
+       .tx_queue_setup         = stmmac_tx_queue_setup,
+       .rxq_info_get           = stmmac_rxq_info_get,
+       .txq_info_get           = stmmac_txq_info_get,
+};
+
+static int
+stmmac_platform_probe(struct rte_platform_device *pdev)
+{
+       struct rte_eth_dev *dev;
+       struct stmmac_priv *priv;
+       unsigned int i;
+
+       if (pdev->num_resource < 1 || pdev->resource[0].mem.addr == NULL) {
+               STMMAC_PMD_ERR("probe failed: no MMIO resource[0] (MAC/DMA/MTL 
block)");
+               return -EINVAL;
+       }
+
+       dev = rte_eth_dev_allocate(pdev->device.name);
+       if (dev == NULL) {
+               STMMAC_PMD_ERR("probe failed: rte_eth_dev_allocate() returned 
NULL for %s",
+                              pdev->device.name);
+               return -ENOMEM;
+       }
+       dev->device = &pdev->device;
+
+       priv = rte_zmalloc_socket(NULL, sizeof(*priv), RTE_CACHE_LINE_SIZE,
+                                 dev->data->numa_node);
+       if (priv == NULL) {
+               STMMAC_PMD_ERR("probe failed: unable to allocate stmmac_priv 
for %s",
+                              pdev->device.name);
+               rte_eth_dev_release_port(dev);
+               return -ENOMEM;
+       }
+       dev->data->dev_private = priv;
+
+       priv->mac_base = pdev->resource[0].mem.addr;
+       /* DT reg-names = "stmmaceth", "rgmii": resource[1] is the RGMII IO 
macro */
+       if (pdev->num_resource > 1)
+               priv->rgmii_base = pdev->resource[1].mem.addr;
+
+       stmmac_parse_devargs(pdev, priv);
+
+       if (priv->has_emac_ge_3) {
+               struct stmmac_chan_addrs ge3 = STMMAC_CHAN_ADDRS_EMAC_GE3_INIT;
+
+               priv->chan_addrs = ge3;
+       } else {
+               struct stmmac_chan_addrs generic = 
STMMAC_CHAN_ADDRS_GENERIC_INIT;
+
+               priv->chan_addrs = generic;
+       }
+
+       dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+       if (dev->data->mac_addrs == NULL) {
+               STMMAC_PMD_ERR("probe failed: unable to allocate mac_addrs for 
%s",
+                              pdev->device.name);
+               rte_free(priv);
+               rte_eth_dev_release_port(dev);
+               return -ENOMEM;
+       }
+       for (i = 0; i < RTE_ETHER_ADDR_LEN; i++)
+               dev->data->mac_addrs[0].addr_bytes[i] = 0x02;
+       rte_ether_addr_copy(&dev->data->mac_addrs[0], &priv->mac_addr);
+
+       dev->dev_ops = &stmmac_eth_ops;
+
+       rte_eth_dev_probing_finish(dev);
+
+       return 0;
+}
+
+static int
+stmmac_platform_remove(struct rte_platform_device *pdev)
+{
+       struct rte_eth_dev *dev;
+
+       dev = rte_eth_dev_allocated(pdev->device.name);
+       if (dev == NULL)
+               return 0;
+
+       stmmac_eth_close(dev);
+       rte_free(dev->data->dev_private);
+       rte_eth_dev_release_port(dev);
+
+       return 0;
+}
+
+static struct rte_platform_driver stmmac_platform_drv = {
+       .probe = stmmac_platform_probe,
+       .remove = stmmac_platform_remove,
+};
+
+RTE_PMD_REGISTER_PLATFORM(STMMAC_DRIVER_NAME, stmmac_platform_drv);
+RTE_PMD_REGISTER_ALIAS(STMMAC_DRIVER_NAME, vfio-platform);
+RTE_PMD_REGISTER_PARAM_STRING(STMMAC_DRIVER_NAME,
+       STMMAC_DEVARG_HAS_EMAC_GE_3 "=<0|1> "
+       STMMAC_DEVARG_MDIO_ADDR "=<uint16>");
diff --git a/drivers/net/stmmac/stmmac_logs.h b/drivers/net/stmmac/stmmac_logs.h
new file mode 100644
index 0000000000..37db8e3ea9
--- /dev/null
+++ b/drivers/net/stmmac/stmmac_logs.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2026 Qualcomm Technologies, Inc.
+ */
+
+#ifndef _STMMAC_LOGS_H_
+#define _STMMAC_LOGS_H_
+
+#include <rte_log.h>
+
+extern int stmmac_logtype_pmd;
+#define RTE_LOGTYPE_STMMAC_NET stmmac_logtype_pmd
+
+#define STMMAC_PMD_LOG(level, ...) \
+       RTE_LOG_LINE_PREFIX(level, STMMAC_NET, "%s()", __func__, __VA_ARGS__)
+
+#define STMMAC_PMD_DEBUG(fmt, ...) \
+       STMMAC_PMD_LOG(DEBUG, fmt, ## __VA_ARGS__)
+#define STMMAC_PMD_ERR(fmt, ...) \
+       STMMAC_PMD_LOG(ERR, fmt, ## __VA_ARGS__)
+#define STMMAC_PMD_INFO(fmt, ...) \
+       STMMAC_PMD_LOG(INFO, fmt, ## __VA_ARGS__)
+#define STMMAC_PMD_WARN(fmt, ...) \
+       STMMAC_PMD_LOG(WARNING, fmt, ## __VA_ARGS__)
+
+#endif /* _STMMAC_LOGS_H_ */
diff --git a/drivers/net/stmmac/stmmac_regs.h b/drivers/net/stmmac/stmmac_regs.h
new file mode 100644
index 0000000000..2fac1d9f21
--- /dev/null
+++ b/drivers/net/stmmac/stmmac_regs.h
@@ -0,0 +1,302 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2026 Qualcomm Technologies, Inc.
+ */
+
+#ifndef _STMMAC_REGS_H_
+#define _STMMAC_REGS_H_
+
+#include <stdint.h>
+#include <rte_bitops.h>
+#include <rte_common.h>
+#include <rte_io.h>
+
+/*
+ * Register and bitfield definitions for the Synopsys DWC EQoS GMAC4 core
+ * ("stmmac" / "dwmac4" in the Linux driver family). Offsets and bit positions
+ * follow drivers/net/ethernet/stmicro/stmmac/{dwmac4.h, dwmac4_dma.h,
+ * dwmac4_descs.h, descs.h, stmmac_mdio.c, dwmac4_core.c}.
+ */
+
+/*
+ * Per-channel DMA/MTL register stride. The generic dwmac4 layout is used by
+ * default; platforms with has_emac_ge_3 use a 0x1000 stride.
+ */
+struct stmmac_chan_addrs {
+       uint32_t dma_chan_base;
+       uint32_t dma_chan_stride;
+       uint32_t mtl_chan_base;
+       uint32_t mtl_chan_stride;
+};
+
+/* ---------------------------------------------------------------- */
+/* MAC registers                                                     */
+/* ---------------------------------------------------------------- */
+
+#define GMAC_CONFIG                    0x00000000
+#define GMAC_PACKET_FILTER             0x00000008
+#define GMAC_HW_FEATURE1               0x00000120
+#define GMAC_MDIO_ADDR                 0x00000200
+#define GMAC_MDIO_DATA                 0x00000204
+#define GMAC_RXQ_CTRL0                 0x000000a0
+#define GMAC_ADDR_HIGH(reg)            (0x300 + (reg) * 8)
+#define GMAC_ADDR_LOW(reg)             (0x304 + (reg) * 8)
+
+/* GMAC_CONFIG bits */
+#define GMAC_CONFIG_IPC                        RTE_BIT32(27)
+#define GMAC_CONFIG_ACS                        RTE_BIT32(20)
+#define GMAC_CONFIG_TE                 RTE_BIT32(1)
+#define GMAC_CONFIG_RE                 RTE_BIT32(0)
+
+/* GMAC_PACKET_FILTER bits */
+#define GMAC_PACKET_FILTER_PR          RTE_BIT32(0)
+
+/* GMAC_HW_FEATURE1: RX/TX FIFO size encodings */
+#define GMAC_HWF1_RXFIFOSIZE_MASK      0x1fu           /* [4:0]  */
+#define GMAC_HWF1_TXFIFOSIZE_SHIFT     6               /* [10:6] */
+#define GMAC_HWF1_TXFIFOSIZE_MASK      0x1fu
+
+/* ---------------------------------------------------------------- */
+/* MTL registers                                                     */
+/* ---------------------------------------------------------------- */
+
+#define MTL_CHAN_BASE_ADDR             0x00000d00
+#define MTL_CHAN_BASE_OFFSET           0x40
+#define MTL_RXQ_DMA_MAP0               0x00000c30
+
+static inline uint32_t
+stmmac_mtl_chan_base(const struct stmmac_chan_addrs *addrs, uint32_t x)
+{
+       if (addrs != NULL)
+               return addrs->mtl_chan_base + x * addrs->mtl_chan_stride;
+       return MTL_CHAN_BASE_ADDR + x * MTL_CHAN_BASE_OFFSET;
+}
+
+#define MTL_CHAN_TX_OP_MODE(addrs, x) \
+       (stmmac_mtl_chan_base((addrs), (x)))
+#define MTL_CHAN_RX_OP_MODE(addrs, x) \
+       (stmmac_mtl_chan_base((addrs), (x)) + 0x30)
+
+#define MTL_OP_MODE_TSF                        RTE_BIT32(1)
+#define MTL_OP_MODE_TXQEN              RTE_BIT32(3)
+#define MTL_OP_MODE_TXQEN_MASK         (0x3u << 2)
+#define MTL_OP_MODE_RSF                        RTE_BIT32(5)
+#define MTL_OP_MODE_TQS_MASK           (0x1ffu << 16)
+#define MTL_OP_MODE_TQS_SHIFT          16
+#define MTL_OP_MODE_RQS_MASK           (0x3ffu << 20)
+#define MTL_OP_MODE_RQS_SHIFT          20
+
+/* ---------------------------------------------------------------- */
+/* DMA registers                                                     */
+/* ---------------------------------------------------------------- */
+
+#define DMA_BUS_MODE                   0x00001000
+#define DMA_SYS_BUS_MODE               0x00001004
+#define DMA_AXI_BUS_MODE               0x00001028
+
+#define DMA_BUS_MODE_SFT_RESET         RTE_BIT32(0)
+
+/* DMA_SYS_BUS_MODE bits */
+#define DMA_SYS_BUS_MB                 RTE_BIT32(14)
+#define DMA_SYS_BUS_AAL                        RTE_BIT32(12)
+#define DMA_SYS_BUS_EAME               RTE_BIT32(11)
+#define DMA_SYS_BUS_FB                 RTE_BIT32(0)
+
+/* DMA_AXI_BUS_MODE bits */
+#define DMA_AXI_WR_OSR_LMT             (0xfu << 24)
+#define DMA_AXI_WR_OSR_LMT_SHIFT       24
+#define DMA_AXI_RD_OSR_LMT             (0xfu << 16)
+#define DMA_AXI_RD_OSR_LMT_SHIFT       16
+#define DMA_AXI_BLEN16                 RTE_BIT32(3)
+#define DMA_AXI_OSR_MAX                        0xf
+
+/* Per-channel DMA registers */
+#define DMA_CHAN_BASE_ADDR             0x00001100
+#define DMA_CHAN_BASE_OFFSET           0x80
+
+static inline uint32_t
+stmmac_dma_chan_base(const struct stmmac_chan_addrs *addrs, uint32_t x)
+{
+       if (addrs != NULL)
+               return addrs->dma_chan_base + x * addrs->dma_chan_stride;
+       return DMA_CHAN_BASE_ADDR + x * DMA_CHAN_BASE_OFFSET;
+}
+
+/*
+ * Offsets relative to a resolved per-channel base (chan_base already holds
+ * mac_base + stmmac_dma_chan_base(addrs, 0)).
+ */
+#define DMA_CHAN_OFF_CONTROL           0x00
+#define DMA_CHAN_OFF_TX_CONTROL                0x04
+#define DMA_CHAN_OFF_RX_CONTROL                0x08
+#define DMA_CHAN_OFF_TX_BASE_ADDR_HI   0x10
+#define DMA_CHAN_OFF_TX_BASE_ADDR      0x14
+#define DMA_CHAN_OFF_RX_BASE_ADDR_HI   0x18
+#define DMA_CHAN_OFF_RX_BASE_ADDR      0x1c
+#define DMA_CHAN_OFF_RX_END_ADDR_HI    0x24
+#define DMA_CHAN_OFF_TX_END_ADDR       0x20
+#define DMA_CHAN_OFF_RX_END_ADDR       0x28
+#define DMA_CHAN_OFF_TX_RING_LEN       0x2c
+#define DMA_CHAN_OFF_RX_RING_LEN       0x30
+#define DMA_CHAN_OFF_CURTXD            0x44
+#define DMA_CHAN_OFF_STATUS            0x60
+
+#define DMA_CHAN_TX_CONTROL_OSP                RTE_BIT32(4)
+
+/* DMA_CHAN_CONTROL / TX / RX control bits */
+#define DMA_CONTROL_PBLX8              RTE_BIT32(16)
+#define DMA_CONTROL_ST                 RTE_BIT32(0)
+#define DMA_CONTROL_RPF                        RTE_BIT32(31)
+#define DMA_CONTROL_SR                 RTE_BIT32(0)
+#define DMA_CONTROL_PBL_SHIFT          16
+#define DMA_CONTROL_PBL_MASK           (0x3fu << DMA_CONTROL_PBL_SHIFT)
+#define DMA_CONTROL_RBSZ_SHIFT         1
+#define DMA_CONTROL_RBSZ_MASK          (0x3fffu << DMA_CONTROL_RBSZ_SHIFT)
+
+/* DMA_CHAN_STATUS bits */
+#define DMA_CHAN_STATUS_TI             RTE_BIT32(0)
+#define DMA_CHAN_STATUS_RI             RTE_BIT32(6)
+#define DMA_CHAN_STATUS_RBU            RTE_BIT32(7)
+
+/*
+ * The Shikra EMAC is not DMA-coherent: CPU writes to descriptors must be
+ * cleaned to the point of coherency before the DMA reads them, and lines must
+ * be invalidated before the CPU reads DMA write-back.
+ */
+#define STMMAC_CACHE_LINE              64
+/* 16-byte descriptors, four per cache line; the ring base is 64-byte aligned 
*/
+#define STMMAC_DESCS_PER_LINE          (STMMAC_CACHE_LINE / 16)
+
+/*
+ * Cache maintenance for a non-coherent DMA (arm64). Clean writes CPU-written
+ * lines back so the DMA reads them; flush also discards CPU copies before the
+ * CPU reads DMA write-back. dsb orders the maintenance before the DMA sees it.
+ */
+static inline void
+stmmac_dcache_clean(const void *addr, size_t len)
+{
+       uintptr_t p = (uintptr_t)addr & ~(uintptr_t)(STMMAC_CACHE_LINE - 1);
+       uintptr_t end = (uintptr_t)addr + len;
+
+       for (; p < end; p += STMMAC_CACHE_LINE)
+               asm volatile("dc cvac, %0" : : "r"(p) : "memory");
+       asm volatile("dsb sy" : : : "memory");
+}
+
+static inline void
+stmmac_dcache_flush(const void *addr, size_t len)
+{
+       uintptr_t p = (uintptr_t)addr & ~(uintptr_t)(STMMAC_CACHE_LINE - 1);
+       uintptr_t end = (uintptr_t)addr + len;
+
+       for (; p < end; p += STMMAC_CACHE_LINE)
+               asm volatile("dc civac, %0" : : "r"(p) : "memory");
+       asm volatile("dsb sy" : : : "memory");
+}
+
+static inline void
+stmmac_dcache_inval_line(const void *addr)
+{
+       asm volatile("dc civac, %0" : : "r"(addr) : "memory");
+       rte_io_rmb();
+}
+
+/* ---------------------------------------------------------------- */
+/* TX descriptor bits                                                */
+/* ---------------------------------------------------------------- */
+
+/* TDES2 (read format) */
+#define TDES2_BUFFER1_SIZE_MASK                0x3fffu
+#define TDES2_INTERRUPT_ON_COMPLETION  RTE_BIT32(31)
+
+/* TDES3 (read format) */
+#define TDES3_PACKET_SIZE_MASK         0x7fffu
+#define TDES3_CHECKSUM_INSERTION_MASK  (0x3u << 16)
+#define TDES3_CHECKSUM_INSERTION_SHIFT 16
+
+/* Insert IP header and pseudo-header checksums */
+#define TX_CIC_FULL                    3
+
+/* TDES3 (write-back and common) */
+#define TDES3_ERROR_SUMMARY            RTE_BIT32(15)
+#define TDES3_LAST_DESCRIPTOR          RTE_BIT32(28)
+#define TDES3_FIRST_DESCRIPTOR         RTE_BIT32(29)
+#define TDES3_CONTEXT_TYPE             RTE_BIT32(30)
+#define TDES3_OWN                      RTE_BIT32(31)
+
+/* ---------------------------------------------------------------- */
+/* RX descriptor bits                                                */
+/* ---------------------------------------------------------------- */
+
+/* RDES1 (write-back format) */
+#define RDES1_IP_HDR_ERROR             RTE_BIT32(3)
+#define RDES1_IPV4_HEADER              RTE_BIT32(4)
+#define RDES1_IPV6_HEADER              RTE_BIT32(5)
+#define RDES1_IP_CSUM_BYPASSED         RTE_BIT32(6)
+#define RDES1_IP_CSUM_ERROR            RTE_BIT32(7)
+
+/* RDES3 (write-back format) */
+#define RDES3_PACKET_SIZE_MASK         0x7fffu
+#define RDES3_ERROR_SUMMARY            RTE_BIT32(15)
+#define RDES3_OVERFLOW_ERROR           RTE_BIT32(21)
+#define RDES3_CRC_ERROR                        RTE_BIT32(24)
+#define RDES3_LAST_DESCRIPTOR          RTE_BIT32(28)
+#define RDES3_FIRST_DESCRIPTOR         RTE_BIT32(29)
+#define RDES3_CONTEXT_DESCRIPTOR       RTE_BIT32(30)
+
+/* RDES3 (read format) */
+#define RDES3_BUFFER1_VALID_ADDR       RTE_BIT32(24)
+#define RDES3_INT_ON_COMPLETION_EN     RTE_BIT32(30)
+#define RDES3_OWN                      RTE_BIT32(31)
+
+/* ---------------------------------------------------------------- */
+/* MDIO                                                              */
+/* ---------------------------------------------------------------- */
+
+#define MII_BUSY                       RTE_BIT32(0)
+#define MII_DATA_MASK                  0xffffu
+
+#define MII_GMAC4_GOC_SHIFT            2
+#define MII_GMAC4_WRITE                        (1 << MII_GMAC4_GOC_SHIFT)
+#define MII_GMAC4_READ                 (3 << MII_GMAC4_GOC_SHIFT)
+
+#define MII_ADDR_SHIFT                 21
+#define MII_ADDR_MASK                  (0x1fu << MII_ADDR_SHIFT)
+#define MII_REG_SHIFT                  16
+#define MII_REG_MASK                   (0x1fu << MII_REG_SHIFT)
+
+/* IEEE 802.3 clause-22 registers */
+#define STMMAC_MII_BMSR                        0x01
+#define STMMAC_MII_BMSR_LSTATUS                RTE_BIT32(2)
+
+/*
+ * DP83867 PHY registers (clause-22 direct access).
+ * PHYSTS bits: 15:14 speed (11 = 1000, 10 = 100, 01 = 10 Mb/s),
+ * 13 duplex (1 = full), 10 link (not latched).
+ */
+#define DP83867_PHYSTS                 0x11
+#define DP83867_PHYSTS_LINK_BIT                RTE_BIT32(10)
+#define DP83867_PHYSTS_SPEED_MASK      (RTE_BIT32(15) | RTE_BIT32(14))
+#define DP83867_PHYSTS_SPEED_1G                RTE_BIT32(15)
+#define DP83867_PHYSTS_SPEED_100M      RTE_BIT32(14)
+#define DP83867_PHYSTS_DUPLEX_BIT      RTE_BIT32(13)
+
+/* ---------------------------------------------------------------- */
+/* Channel stride presets                                            */
+/* ---------------------------------------------------------------- */
+
+#define STMMAC_CHAN_ADDRS_GENERIC_INIT {               \
+       .dma_chan_base   = DMA_CHAN_BASE_ADDR,          \
+       .dma_chan_stride = DMA_CHAN_BASE_OFFSET,        \
+       .mtl_chan_base   = MTL_CHAN_BASE_ADDR,          \
+       .mtl_chan_stride = MTL_CHAN_BASE_OFFSET,        \
+}
+
+/* has_emac_ge_3 layout (dwmac-qcom-ethqos.c emac_v4_0_0_data) */
+#define STMMAC_CHAN_ADDRS_EMAC_GE3_INIT {              \
+       .dma_chan_base   = 0x8100,                      \
+       .dma_chan_stride = 0x1000,                      \
+       .mtl_chan_base   = 0x8000,                      \
+       .mtl_chan_stride = 0x1000,                      \
+}
+
+#endif /* _STMMAC_REGS_H_ */
diff --git a/drivers/net/stmmac/stmmac_rxtx.c b/drivers/net/stmmac/stmmac_rxtx.c
new file mode 100644
index 0000000000..61f7249759
--- /dev/null
+++ b/drivers/net/stmmac/stmmac_rxtx.c
@@ -0,0 +1,416 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2026 Qualcomm Technologies, Inc.
+ */
+
+#include <rte_branch_prediction.h>
+#include <rte_common.h>
+#include <rte_cycles.h>
+#include <rte_io.h>
+#include <rte_mbuf.h>
+
+#include "stmmac.h"
+#include "stmmac_regs.h"
+#include "stmmac_logs.h"
+
+#define STMMAC_RX_BUNDLE       32
+#define STMMAC_TX_BUNDLE       32
+
+/* MDIO busy-poll bound, matching stmmac_mdio.c (10000 x 1 us) */
+#define STMMAC_MDIO_TIMEOUT_US         10000
+
+/* Index of the TX descriptor the DMA is working on: everything before it is 
sent. */
+static inline uint16_t
+stmmac_tx_dma_slot(struct stmmac_tx_queue *txq)
+{
+       uint32_t cur = rte_read32((uint8_t *)txq->chan_base + 
DMA_CHAN_OFF_CURTXD);
+
+       return (uint16_t)(((cur - (uint32_t)txq->tx_ring_iova) /
+                          sizeof(struct stmmac_desc)) % txq->nb_desc);
+}
+
+/*
+ * MDIO clause-22 read. Uses the GMAC4 opcode and the address/register field
+ * layout from dwmac4_core.c. The CSR clock divider is left at zero.
+ */
+uint16_t
+stmmac_mdio_read(void *mac_base, uint16_t mdio_addr, uint16_t phyreg)
+{
+       void *addr_reg = (uint8_t *)mac_base + GMAC_MDIO_ADDR;
+       void *data_reg = (uint8_t *)mac_base + GMAC_MDIO_DATA;
+       uint32_t value;
+       unsigned int timeout = STMMAC_MDIO_TIMEOUT_US;
+
+       while ((rte_read32(addr_reg) & MII_BUSY) && timeout > 0) {
+               rte_delay_us(1);
+               timeout--;
+       }
+       if (timeout == 0) {
+               STMMAC_PMD_ERR("MDIO busy timeout before read");
+               return 0xffff;
+       }
+
+       value = MII_GMAC4_READ | MII_BUSY;
+       value |= ((uint32_t)mdio_addr << MII_ADDR_SHIFT) & MII_ADDR_MASK;
+       value |= ((uint32_t)phyreg << MII_REG_SHIFT) & MII_REG_MASK;
+
+       rte_write32(value, addr_reg);
+
+       timeout = STMMAC_MDIO_TIMEOUT_US;
+       while ((rte_read32(addr_reg) & MII_BUSY) && timeout > 0) {
+               rte_delay_us(1);
+               timeout--;
+       }
+       if (timeout == 0) {
+               STMMAC_PMD_ERR("MDIO busy timeout after read");
+               return 0xffff;
+       }
+
+       return (uint16_t)(rte_read32(data_reg) & MII_DATA_MASK);
+}
+
+void
+stmmac_mdio_write(void *mac_base, uint16_t mdio_addr, uint16_t phyreg,
+                 uint16_t phydata)
+{
+       void *addr_reg = (uint8_t *)mac_base + GMAC_MDIO_ADDR;
+       void *data_reg = (uint8_t *)mac_base + GMAC_MDIO_DATA;
+       uint32_t value;
+       unsigned int timeout = STMMAC_MDIO_TIMEOUT_US;
+
+       while ((rte_read32(addr_reg) & MII_BUSY) && timeout > 0) {
+               rte_delay_us(1);
+               timeout--;
+       }
+       if (timeout == 0) {
+               STMMAC_PMD_ERR("MDIO busy timeout before write");
+               return;
+       }
+
+       rte_write32(phydata, data_reg);
+
+       value = MII_GMAC4_WRITE | MII_BUSY;
+       value |= ((uint32_t)mdio_addr << MII_ADDR_SHIFT) & MII_ADDR_MASK;
+       value |= ((uint32_t)phyreg << MII_REG_SHIFT) & MII_REG_MASK;
+
+       rte_write32(value, addr_reg);
+
+       timeout = STMMAC_MDIO_TIMEOUT_US;
+       while ((rte_read32(addr_reg) & MII_BUSY) && timeout > 0) {
+               rte_delay_us(1);
+               timeout--;
+       }
+       if (timeout == 0)
+               STMMAC_PMD_ERR("MDIO busy timeout after write");
+}
+
+/*
+ * Refill 'cleaned_cnt' RX descriptors starting at rx_refill. Mbufs are
+ * allocated in bulk, descriptors are written, and each cache line of
+ * descriptors is cleaned once. The DMA doorbell moves to the last descriptor
+ * handed back. Returns the number refilled; a short count means the mbuf pool
+ * ran out, and the remaining slots are retried on the next burst.
+ */
+static uint16_t
+stmmac_refill_rx_ring(struct stmmac_rx_queue *rxq, uint16_t cleaned_cnt)
+{
+       struct rte_mbuf *m[STMMAC_RX_BUNDLE];
+       struct stmmac_desc *grp_start = NULL;
+       uint16_t i = rxq->rx_refill;
+       uint16_t j;
+       uint16_t k = STMMAC_RX_BUNDLE;
+       uint16_t grp_cnt = 0;
+
+       for (j = 0; j < cleaned_cnt; j++) {
+               struct stmmac_desc *desc = &rxq->rx_ring[i];
+               struct rte_mbuf *mb;
+               uint64_t iova;
+
+               /* bulk allocation, sized to the descriptors still to refill */
+               if (k == STMMAC_RX_BUNDLE) {
+                       uint16_t m_cnt = RTE_MIN(cleaned_cnt - j, 
STMMAC_RX_BUNDLE);
+
+                       if (rte_pktmbuf_alloc_bulk(rxq->mb_pool, m, m_cnt)) {
+                               rxq->rx_nombuf++;
+                               break;
+                       }
+                       k = 0;
+               }
+               mb = m[k++];
+
+               rxq->sw_ring[i] = mb;
+               /* drop stale CPU lines over the buffer the DMA is about to 
write */
+               stmmac_dcache_flush(rte_pktmbuf_mtod(mb, void *),
+                                   rte_pktmbuf_data_room_size(rxq->mb_pool) -
+                                   RTE_PKTMBUF_HEADROOM);
+
+               /* full 36-bit buffer address: the mbufs live above 4 GiB */
+               iova = rte_pktmbuf_iova(mb);
+               desc->des[0] = (uint32_t)iova;
+               desc->des[1] = (uint32_t)(iova >> 32);
+               desc->des[2] = 0;
+               rte_io_wmb();
+               desc->des[3] = RDES3_OWN | RDES3_BUFFER1_VALID_ADDR |
+                              RDES3_INT_ON_COMPLETION_EN;
+
+               /* clean the descriptors one cache line at a time */
+               if (grp_cnt == 0)
+                       grp_start = desc;
+               grp_cnt++;
+               if (grp_cnt == STMMAC_DESCS_PER_LINE || i == rxq->nb_desc - 1) {
+                       stmmac_dcache_clean(grp_start, grp_cnt * sizeof(*desc));
+                       grp_cnt = 0;
+               }
+
+               i = (i + 1) % rxq->nb_desc;
+       }
+
+       /* clean the partial group left by the last descriptor */
+       if (grp_cnt)
+               stmmac_dcache_clean(grp_start, grp_cnt * sizeof(struct 
stmmac_desc));
+
+       rxq->rx_refill = i;
+
+       if (j > 0) {
+               uint16_t last = (i + rxq->nb_desc - 1) % rxq->nb_desc;
+               uint64_t tail_iova = rxq->rx_ring_iova +
+                       (uint64_t)last * sizeof(struct stmmac_desc);
+
+               /*
+                * Clear RBU if the DMA suspended on an unavailable descriptor. 
This
+                * is a write, not a status read: a register read stalls the 
core.
+                * The DMA only suspends after it has used every slot refilled 
so
+                * far, so the burst that refills again is where RBU is cleared.
+                */
+               rte_write32(DMA_CHAN_STATUS_RBU,
+                           (uint8_t *)rxq->chan_base + DMA_CHAN_OFF_STATUS);
+
+               /*
+                * The doorbell is the last descriptor handed back. Writing the
+                * refill index instead would make RXTAIL equal CURRXD, which 
the
+                * DMA treats as an empty ring. The 36-bit tail is written HI 
first.
+                */
+               rte_write32((uint32_t)(tail_iova >> 32),
+                           (uint8_t *)rxq->chan_base + 
DMA_CHAN_OFF_RX_END_ADDR_HI);
+               rte_io_wmb();
+               rte_write32((uint32_t)tail_iova,
+                           (uint8_t *)rxq->chan_base + 
DMA_CHAN_OFF_RX_END_ADDR);
+       }
+
+       return j;
+}
+
+uint16_t
+stmmac_rx_pkt_burst_nc(void *rx_queue, struct rte_mbuf **rx_pkts,
+                      uint16_t nb_pkts)
+{
+       struct stmmac_rx_queue *rxq = rx_queue;
+       uint16_t nb_rx = 0;
+       uint16_t start = rxq->rx_tail;
+       uint16_t tail = start;
+       uint16_t pending;
+
+       while (nb_rx < nb_pkts) {
+               struct stmmac_desc *desc = &rxq->rx_ring[tail];
+               struct rte_mbuf *rxm;
+               uint32_t des3;
+               uint32_t pkt_len;
+
+               /* slot is waiting for a refill (mbuf allocation failed) */
+               if (rxq->sw_ring[tail] == NULL)
+                       break;
+
+               /*
+                * Shikra is not DMA-coherent: invalidate the descriptor line 
once
+                * per cache line, at the start of the burst and at each line
+                * boundary. A stale OWN bit on a later slot only defers that 
frame
+                * to the next burst.
+                */
+               if (tail == start || tail % STMMAC_DESCS_PER_LINE == 0)
+                       stmmac_dcache_inval_line(desc);
+               des3 = rte_read32_relaxed(&desc->des[3]);
+
+               if (des3 & RDES3_OWN)
+                       break;
+
+               if (des3 & (RDES3_CONTEXT_DESCRIPTOR | RDES3_ERROR_SUMMARY)) {
+                       /* not a frame to deliver: free its buffer, refill 
allocates a new one */
+                       if (des3 & RDES3_ERROR_SUMMARY)
+                               rxq->rx_errors++;
+                       rte_pktmbuf_free(rxq->sw_ring[tail]);
+                       rxq->sw_ring[tail] = NULL;
+                       tail = (tail + 1) % rxq->nb_desc;
+                       continue;
+               }
+
+               rxm = rxq->sw_ring[tail];
+               rxq->sw_ring[tail] = NULL;
+               pkt_len = des3 & RDES3_PACKET_SIZE_MASK;
+
+               /* drop stale CPU lines over the received payload */
+               stmmac_dcache_flush((uint8_t *)rxm->buf_addr + 
RTE_PKTMBUF_HEADROOM,
+                                   pkt_len);
+
+               rxm->data_off = RTE_PKTMBUF_HEADROOM;
+               rxm->pkt_len = pkt_len;
+               rxm->data_len = pkt_len;
+               rxm->port = rxq->port_id;
+               rxm->nb_segs = 1;
+               rxm->next = NULL;
+               rxm->ol_flags = 0;
+
+               if (desc->des[1] & (RDES1_IPV4_HEADER | RDES1_IPV6_HEADER)) {
+                       if (desc->des[1] & RDES1_IP_CSUM_ERROR)
+                               rxm->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+                       else
+                               rxm->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+               }
+
+               rx_pkts[nb_rx++] = rxm;
+               rxq->rx_packets++;
+               rxq->rx_bytes += pkt_len;
+
+               tail = (tail + 1) % rxq->nb_desc;
+       }
+
+       rxq->rx_tail = tail;
+
+       /* refill what was consumed, plus any slots a failed allocation left 
pending */
+       pending = (tail + rxq->nb_desc - rxq->rx_refill) % rxq->nb_desc;
+       if (pending)
+               stmmac_refill_rx_ring(rxq, pending);
+
+       return nb_rx;
+}
+
+/*
+ * Reclaim sent TX frames in bulk, up to the descriptor the DMA is working on.
+ * Everything before that slot has been transmitted.
+ */
+static uint16_t
+stmmac_clean_tx_ring(struct stmmac_tx_queue *txq)
+{
+       struct rte_mbuf *m[STMMAC_TX_BUNDLE];
+       uint16_t hwci = stmmac_tx_dma_slot(txq);
+       uint16_t i = txq->tx_free;
+       uint16_t n = 0;
+       uint16_t total = 0;
+
+       while (i != hwci && txq->sw_ring[i] != NULL) {
+               if (i == txq->tx_free || i % STMMAC_DESCS_PER_LINE == 0)
+                       stmmac_dcache_inval_line(&txq->tx_ring[i]);
+               if (rte_read32_relaxed(&txq->tx_ring[i].des[3]) & 
TDES3_ERROR_SUMMARY)
+                       txq->tx_errors++;
+
+               m[n++] = txq->sw_ring[i];
+               txq->sw_ring[i] = NULL;
+               if (n == STMMAC_TX_BUNDLE) {
+                       rte_pktmbuf_free_bulk(m, n);
+                       total += n;
+                       n = 0;
+               }
+               i = (i + 1) % txq->nb_desc;
+       }
+
+       if (n) {
+               rte_pktmbuf_free_bulk(m, n);
+               total += n;
+       }
+       txq->tx_free = i;
+       return total;
+}
+
+/* Clean the descriptor lines that hold 'cnt' slots from 'first', once, before 
the doorbell */
+static void
+stmmac_tx_clean_range(struct stmmac_tx_queue *txq, uint16_t first, uint16_t 
cnt)
+{
+       uint16_t s = first - first % STMMAC_DESCS_PER_LINE;
+       uint16_t left = cnt + (first - s);
+
+       left = (left + STMMAC_DESCS_PER_LINE - 1) / STMMAC_DESCS_PER_LINE *
+               STMMAC_DESCS_PER_LINE;
+       if (left > txq->nb_desc)
+               left = txq->nb_desc;
+
+       while (left) {
+               uint16_t run = RTE_MIN((uint16_t)(txq->nb_desc - s), left);
+
+               stmmac_dcache_clean(&txq->tx_ring[s], run * sizeof(struct 
stmmac_desc));
+               s = (s + run) % txq->nb_desc;
+               left -= run;
+       }
+}
+
+uint16_t
+stmmac_tx_pkt_burst_nc(void *tx_queue, struct rte_mbuf **tx_pkts,
+                      uint16_t nb_pkts)
+{
+       struct stmmac_tx_queue *txq = tx_queue;
+       uint16_t nb_tx = 0;
+       uint16_t first = txq->tx_tail;
+       uint16_t tail = first;
+
+       stmmac_clean_tx_ring(txq);
+
+       /* drop the CPU copy of the first line before filling into it */
+       stmmac_dcache_inval_line(&txq->tx_ring[first - first % 
STMMAC_DESCS_PER_LINE]);
+
+       while (nb_tx < nb_pkts) {
+               struct stmmac_desc *desc = &txq->tx_ring[tail];
+               struct rte_mbuf *txm = tx_pkts[nb_tx];
+               uint32_t tdes3;
+               uint64_t iova;
+
+               if (unlikely(txm->nb_segs != 1)) {
+                       STMMAC_PMD_ERR("multi-segment mbuf not supported");
+                       break;
+               }
+
+               /* ring full: this slot still holds a frame the DMA has not 
finished */
+               if (txq->sw_ring[tail] != NULL) {
+                       txq->tx_ring_full++;
+                       break;
+               }
+
+               /* entering a new line: drop the CPU copy before writing into 
it */
+               if (tail != first && tail % STMMAC_DESCS_PER_LINE == 0)
+                       stmmac_dcache_inval_line(desc);
+
+               /* the CPU-written payload must reach memory before the DMA 
reads it */
+               stmmac_dcache_clean(rte_pktmbuf_mtod(txm, void *),
+                                   rte_pktmbuf_data_len(txm));
+
+               iova = rte_pktmbuf_iova(txm);
+               desc->des[0] = (uint32_t)iova;
+               desc->des[1] = (uint32_t)(iova >> 32);
+               desc->des[2] = rte_pktmbuf_data_len(txm) & 
TDES2_BUFFER1_SIZE_MASK;
+
+               tdes3 = rte_pktmbuf_pkt_len(txm) & TDES3_PACKET_SIZE_MASK;
+               tdes3 |= TDES3_FIRST_DESCRIPTOR | TDES3_LAST_DESCRIPTOR;
+               if (txm->ol_flags & (RTE_MBUF_F_TX_IP_CKSUM | 
RTE_MBUF_F_TX_L4_MASK))
+                       tdes3 |= (TX_CIC_FULL << 
TDES3_CHECKSUM_INSERTION_SHIFT);
+
+               txq->sw_ring[tail] = txm;
+
+               rte_io_wmb();
+               desc->des[3] = tdes3 | TDES3_OWN;
+
+               txq->tx_packets++;
+               txq->tx_bytes += rte_pktmbuf_pkt_len(txm);
+
+               tail = (tail + 1) % txq->nb_desc;
+               nb_tx++;
+       }
+
+       if (nb_tx > 0) {
+               uint64_t tail_iova = txq->tx_ring_iova +
+                       (uint64_t)tail * sizeof(struct stmmac_desc);
+
+               /* the written descriptor lines are cleaned once, then the 
doorbell moves */
+               stmmac_tx_clean_range(txq, first, nb_tx);
+               txq->tx_tail = tail;
+               rte_write32((uint32_t)tail_iova,
+                           (uint8_t *)txq->chan_base + 
DMA_CHAN_OFF_TX_END_ADDR);
+       }
+
+       return nb_tx;
+}
-- 
2.34.1

Reply via email to