Hi, > Subject: RE: DPDK doesn't work with iommu=pt > > Met the similar issue before. > VT-d enabled? If so you may need to contact HP to upgrade the BIOS or you may > disable VT-d and remove iommu=pt intel_iommu=on > if you don't need VF function.
we need VT-d and it's enabled. What we want to do is that use SR-IOV functionality and DPDK application concurrently on the same box. thanks, Hiroshi > > >-----Original Message----- > >From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Hiroshi Shimamoto > >Sent: Friday, September 26, 2014 5:14 PM > >To: dev at dpdk.org > >Cc: Hayato Momma > >Subject: [dpdk-dev] DPDK doesn't work with iommu=pt > > > >I encountered an issue that DPDK doesn't work with "iommu=pt > >intel_iommu=on" > >on HP ProLiant DL380p Gen8 server. I'm using the following environment; > > > > HW: ProLiant DL380p Gen8 > > CPU: E5-2697 v2 > > OS: RHEL7 > > kernel: kernel-3.10.0-123 and the latest kernel 3.17-rc6+ > > DPDK: v1.7.1-53-gce5abac > > NIC: 82599ES > > > >When boot with "iommu=pt intel_iommu=on", I got the below message and no > >packets are handled. > > > > [ 120.809611] dmar: DRHD: handling fault status reg 2 > > [ 120.809635] dmar: DMAR:[DMA Read] Request device [21:00.0] fault addr > >aa010000 > > DMAR:[fault reason 02] Present bit in context entry is clear > > > >How to reproduce; > >just run testpmd > ># ./testpmd -c 0xf -n 4 -- -i > > > >Configuring Port 0 (socket 0) > >PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7ffff54eafc0 > >hw_ring=0x7ffff4200000 dma_addr=0xaa000000 > >PMD: ixgbe_dev_tx_queue_setup(): Using full-featured tx code path > >PMD: ixgbe_dev_tx_queue_setup(): - txq_flags = 0 > >[IXGBE_SIMPLE_FLAGS=f01] > >PMD: ixgbe_dev_tx_queue_setup(): - tx_rs_thresh = 32 > >[RTE_PMD_IXGBE_TX_MAX_BURST=32] > >PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7ffff54ea740 > >hw_ring=0x7ffff4210000 dma_addr=0xaa010000 > >PMD: check_rx_burst_bulk_alloc_preconditions(): Rx Burst Bulk Alloc > >Preconditions: rxq->rx_free_thresh=0, RTE_PMD_IXGBE_RX_MAX_BURST=32 > >PMD: ixgbe_dev_rx_queue_setup(): Rx Burst Bulk Alloc Preconditions are not > >satisfied, Scattered Rx is requested, or > >RTE_LIBRTE_IXGBE_RX_ALLOW_BULK_ALLOC is not enabled (port=0, queue=0). > >PMD: check_rx_burst_bulk_alloc_preconditions(): Rx Burst Bulk Alloc > >Preconditions: rxq->rx_free_thresh=0, RTE_PMD_IXGBE_RX_MAX_BURST=32 > > > >testpmd> start > > io packet forwarding - CRC stripping disabled - packets/burst=32 > > nb forwarding cores=1 - nb forwarding ports=2 > > RX queues=1 - RX desc=128 - RX free threshold=0 > > RX threshold registers: pthresh=8 hthresh=8 wthresh=0 > > TX queues=1 - TX desc=512 - TX free threshold=0 > > TX threshold registers: pthresh=32 hthresh=0 wthresh=0 > > TX RS bit threshold=0 - TXQ flags=0x0 > > > > > >and ping from another box to this server. > ># ping6 -I eth2 ff02::1 > > > >I got the below error message and no packet is received. > >I couldn't see any increase RX/TX count in testpmt statistics > > > >testpmd> show port stats 0 > > > > ######################## NIC statistics for port 0 > >######################## > > RX-packets: 6 RX-missed: 0 RX-bytes: 732 > > RX-badcrc: 0 RX-badlen: 0 RX-errors: 0 > > RX-nombuf: 0 > > TX-packets: 0 TX-errors: 0 TX-bytes: 0 > > > >################################################################# > >########### > >testpmd> show port stats 0 > > > > ######################## NIC statistics for port 0 > >######################## > > RX-packets: 6 RX-missed: 0 RX-bytes: 732 > > RX-badcrc: 0 RX-badlen: 0 RX-errors: 0 > > RX-nombuf: 0 > > TX-packets: 0 TX-errors: 0 TX-bytes: 0 > > > >################################################################# > >########### > > > > > >The fault addr in error message must be RX DMA descriptor > > > >error message > > [ 120.809635] dmar: DMAR:[DMA Read] Request device [21:00.0] fault addr > >aa010000 > > > >log in testpmd > > PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7ffff54ea740 > >hw_ring=0x7ffff4210000 dma_addr=0xaa010000 > > > >I think the NIC received a packet in fifo and try to put into memory with > >DMA. > >Before starting DMA, the NIC get the target address from RX descriptors in > >RDBA register. > >But accessing RX descriptors failed in IOMMU unit and reported it to the > >kernel. > > > > DMAR:[fault reason 02] Present bit in context entry is clear > > > >The error message looks there is no valid entry in IOMMU. > > > >I think the following issue is very similar, but using Ubuntu14.04 couldn't > >fix in my > >case. > >http://thread.gmane.org/gmane.comp.networking.dpdk.devel/2281 > > > >I tried Ubuntu14.04.1 and got the below error. > > > > [ 199.710191] dmar: DRHD: handling fault status reg 2 > > [ 199.710896] dmar: DMAR:[DMA Read] Request device [21:00.0] fault addr > >7c24df000 > > [ 199.710896] DMAR:[fault reason 06] PTE Read access is not set > > > >Currently I could see this issue on HP ProLiant DL380p Gen8 only. > >Is there any idea? > >Has anyone noticed this issue? > > > >Note: we're thinking to use SR-IOV and DPDK app in the same box. > >The box has 2 NICs, one for SR-IOV and pass through to VM, one (no SR-IOV) > >for > >DPDK app in host. > > > >thanks, > >Hiroshi