Hi folks, I have been helping Shivaram off-list with this problem and just wanted to update this thread with the solution.
Basically the issue was with how hugepages were being preallocated by OVS for use with DPDK. I believe by default OVS sets other_config:dpdk-socket-mem to "1024,0". Which will give one 1GB hugepage to CPU0 and none to CPU1. The ports Shivaram was trying to assign to DPDK were connected physically via a PCI-E bus to CPU1 and since there are no hugepages preallocated there for DPDK then we can't add it. The fix is simple, to assign a hugepage to CPU1: $ sudo ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem="1024,1024" In OVS 2.7.0 unfortunately due to an issue (that is now fixed by commit ef6cca6fdc67f3cee21c6bb1c13c4ca7f8241314) you get a segfault rather than an error message telling you the problem. In OVS 2.7.90 however the issue becomes quite clear as there is a nice log message like such: 2017-03-29T00:19:57Z|00067|netdev_dpdk|ERR|Insufficient memory to create memory pool for netdev dpdk-p0, with MTU 1500 on socket 1 Unfortunately to those unfamiliar with DPDK and NUMA architectures this won't be very obvious. Potentially we could add some additional help to the DPDK documentation pages for OVS that explains the other_config options you may need to reconfigure on multi-CPU NUMA machines? Brad On 29 March 2017 at 10:02, Shivaram Mysore <shivaram.mys...@gmail.com> wrote: > Ok thanks. Will remember the same. Right now, I will not specify > anything as I have to get this to work first! > > On Tue, Mar 28, 2017 at 2:00 PM, Bodireddy, Bhanuprakash < > bhanuprakash.bodire...@intel.com> wrote: > >> >Question: >> > >> >I don't have any specific cpu cores associated with DPDK. Will this >> make any >> >difference? I also did not see this requirement as a MUST have in the >> >documentation. >> >> I assume your question is on PMD threads. If you don’t specify any thing >> explicitly >> PMD pmd thread shall be created and pinned to core 0. You can explicitly >> select the no. of pmd threads >> and the cores by explicitly setting the affinity mask using >> 'other_config:pmd-cpu-mask'. >> >> Based on the affinity mask, the PMD threads shall be spawned and the rx >> queues will be uniformly >> distributed among the PMD threads. >> >> Refer http://docs.openvswitch.org/en/latest/howto/dpdk/. >> >> Also note that 'dpdk-lcore-mask' should be used to set the dpdk lcore >> threads and vswitchd. >> >> - Bhanuprakash. >> >> > >> ># cat /proc/cmdline >> >BOOT_IMAGE=/boot/vmlinuz-4.10.0-14-generic root=/dev/mapper/intelsdn- >> >-vg-root ro quiet intel_iommu=on iommu=pt default_hugepagesz=1G >> >hugepagesz=1G hugepages=4 >> > >> > >> >Thanks >> > >> >/Shivaram >> > >> >On Tue, Mar 28, 2017 at 1:32 PM, Shivaram Mysore >> ><shivaram.mys...@gmail.com> wrote: >> >I was advised on this mailing list that VFIO drivers are better than >> IGB_UIO + >> >more forgiving [1]. Hence switched to the same. >> > >> >[1] http://dpdk-guide.gitlab.io/dpdk-guide/setup/binding.html >> > >> >/Shivaram >> > >> >On Tue, Mar 28, 2017 at 1:12 PM, Bodireddy, Bhanuprakash >> ><bhanuprakash.bodire...@intel.com> wrote: >> > >> >>any hints for me based on the info provided? >> > >> >Your BIOS options and bootcmd line options are appropriate and no issues >> >there. I wanted to check if you tried igb_uio drivers >> >but saw your post here https://mail.openvswitch.org/pipermail/ovs- >> >discuss/2017-March/043993.html, reporting a crash with UIO drivers. >> > >> >Is that the reason why you moved to VFIO? I cross checked your logs but >> >found nothing suspicious there. >> > >> >-Bhanuprakash. >> > >> >> >> >>Thanks >> >> >> >>On Tue, Mar 28, 2017 at 8:41 AM, Shivaram Mysore >> >><shivaram.mys...@gmail.com> wrote: >> >>Hi, >> >> >> >># dmesg | grep -e DMAR -e IOMMU >> >>[ 0.000000] ACPI: DMAR 0x00000000798AF340 000158 (v01 SUPERM SMCI-- >> >>MB 00000001 INTL 20091013) >> >>[ 0.000000] DMAR: IOMMU enabled >> >>[ 0.109987] DMAR: Host address width 46 >> >>[ 0.109988] DMAR: DRHD base: 0x000000fbffc000 flags: 0x0 >> >>[ 0.109997] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap >> >>d2078c106f0466 ecap f020de >> >>[ 0.109998] DMAR: DRHD base: 0x000000c7ffc000 flags: 0x1 >> >>[ 0.110002] DMAR: dmar1: reg_base_addr c7ffc000 ver 1:0 cap >> >>d2078c106f0466 ecap f020de >> >>[ 0.110003] DMAR: RMRR base: 0x0000007ba75000 end: 0x0000007ba84fff >> >>[ 0.110004] DMAR: ATSR flags: 0x0 >> >>[ 0.110005] DMAR: RHSA base: 0x000000c7ffc000 proximity domain: 0x0 >> >>[ 0.110006] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x1 >> >>[ 0.110008] DMAR-IR: IOAPIC id 3 under DRHD base 0xfbffc000 IOMMU 0 >> >>[ 0.110008] DMAR-IR: IOAPIC id 1 under DRHD base 0xc7ffc000 IOMMU 1 >> >>[ 0.110009] DMAR-IR: IOAPIC id 2 under DRHD base 0xc7ffc000 IOMMU 1 >> >>[ 0.110010] DMAR-IR: HPET id 0 under DRHD base 0xc7ffc000 >> >>[ 0.110011] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt >> out >> >bit. >> >>[ 0.110012] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the >> >>BIOS setting. >> >>[ 0.110902] DMAR-IR: Enabled IRQ remapping in xapic mode >> >>[ 1.687595] DMAR: dmar0: Using Queued invalidation >> >>[ 1.687608] DMAR: dmar1: Using Queued invalidation >> >>[ 1.688473] DMAR: Hardware identity mapping for device 0000:ff:08.0 >> >>[ 1.688475] DMAR: Hardware identity mapping for device 0000:ff:08.2 >> >>[ 1.688477] DMAR: Hardware identity mapping for device 0000:ff:08.3 >> >>[ 1.688478] DMAR: Hardware identity mapping for device 0000:ff:09.0 >> >>[ 1.688480] DMAR: Hardware identity mapping for device 0000:ff:09.2 >> >>[ 1.688482] DMAR: Hardware identity mapping for device 0000:ff:09.3 >> >>[ 1.688483] DMAR: Hardware identity mapping for device 0000:ff:0b.0 >> >>[ 1.688485] DMAR: Hardware identity mapping for device 0000:ff:0b.1 >> >>[ 1.688487] DMAR: Hardware identity mapping for device 0000:ff:0b.2 >> >>[ 1.688488] DMAR: Hardware identity mapping for device 0000:ff:0c.0 >> >>[ 1.688490] DMAR: Hardware identity mapping for device 0000:ff:0c.1 >> >>[ 1.688491] DMAR: Hardware identity mapping for device 0000:ff:0c.2 >> >>[ 1.688493] DMAR: Hardware identity mapping for device 0000:ff:0c.3 >> >>[ 1.688494] DMAR: Hardware identity mapping for device 0000:ff:0c.4 >> >>[ 1.688496] DMAR: Hardware identity mapping for device 0000:ff:0c.5 >> >>[ 1.688498] DMAR: Hardware identity mapping for device 0000:ff:0c.6 >> >>[ 1.688499] DMAR: Hardware identity mapping for device 0000:ff:0c.7 >> >>[ 1.688500] DMAR: Hardware identity mapping for device 0000:ff:0d.0 >> >>[ 1.688502] DMAR: Hardware identity mapping for device 0000:ff:0d.1 >> >>[ 1.688504] DMAR: Hardware identity mapping for device 0000:ff:0d.2 >> >>[ 1.688505] DMAR: Hardware identity mapping for device 0000:ff:0d.3 >> >>[ 1.688506] DMAR: Hardware identity mapping for device 0000:ff:0d.4 >> >>[ 1.688508] DMAR: Hardware identity mapping for device 0000:ff:0d.5 >> >>[ 1.688509] DMAR: Hardware identity mapping for device 0000:ff:0d.6 >> >>[ 1.688510] DMAR: Hardware identity mapping for device 0000:ff:0d.7 >> >>[ 1.688511] DMAR: Hardware identity mapping for device 0000:ff:0e.0 >> >>[ 1.688513] DMAR: Hardware identity mapping for device 0000:ff:0e.1 >> >>[ 1.688514] DMAR: Hardware identity mapping for device 0000:ff:0f.0 >> >>[ 1.688516] DMAR: Hardware identity mapping for device 0000:ff:0f.1 >> >>[ 1.688517] DMAR: Hardware identity mapping for device 0000:ff:0f.2 >> >>[ 1.688518] DMAR: Hardware identity mapping for device 0000:ff:0f.3 >> >>[ 1.688520] DMAR: Hardware identity mapping for device 0000:ff:0f.4 >> >>[ 1.688521] DMAR: Hardware identity mapping for device 0000:ff:0f.5 >> >>[ 1.688523] DMAR: Hardware identity mapping for device 0000:ff:0f.6 >> >>[ 1.688524] DMAR: Hardware identity mapping for device 0000:ff:10.0 >> >>[ 1.688526] DMAR: Hardware identity mapping for device 0000:ff:10.1 >> >>[ 1.688527] DMAR: Hardware identity mapping for device 0000:ff:10.5 >> >>[ 1.688529] DMAR: Hardware identity mapping for device 0000:ff:10.6 >> >>[ 1.688530] DMAR: Hardware identity mapping for device 0000:ff:10.7 >> >>[ 1.688532] DMAR: Hardware identity mapping for device 0000:ff:12.0 >> >>[ 1.688533] DMAR: Hardware identity mapping for device 0000:ff:12.1 >> >>[ 1.688534] DMAR: Hardware identity mapping for device 0000:ff:12.4 >> >>[ 1.688536] DMAR: Hardware identity mapping for device 0000:ff:12.5 >> >>[ 1.688537] DMAR: Hardware identity mapping for device 0000:ff:13.0 >> >>[ 1.688539] DMAR: Hardware identity mapping for device 0000:ff:13.1 >> >>[ 1.688541] DMAR: Hardware identity mapping for device 0000:ff:13.2 >> >>[ 1.688542] DMAR: Hardware identity mapping for device 0000:ff:13.3 >> >>[ 1.688543] DMAR: Hardware identity mapping for device 0000:ff:13.6 >> >>[ 1.688545] DMAR: Hardware identity mapping for device 0000:ff:13.7 >> >>[ 1.688546] DMAR: Hardware identity mapping for device 0000:ff:14.0 >> >>[ 1.688547] DMAR: Hardware identity mapping for device 0000:ff:14.1 >> >>[ 1.688549] DMAR: Hardware identity mapping for device 0000:ff:14.2 >> >>[ 1.688550] DMAR: Hardware identity mapping for device 0000:ff:14.3 >> >>[ 1.688552] DMAR: Hardware identity mapping for device 0000:ff:14.4 >> >>[ 1.688553] DMAR: Hardware identity mapping for device 0000:ff:14.5 >> >>[ 1.688559] DMAR: Hardware identity mapping for device 0000:ff:14.6 >> >>[ 1.688561] DMAR: Hardware identity mapping for device 0000:ff:14.7 >> >>[ 1.688562] DMAR: Hardware identity mapping for device 0000:ff:16.0 >> >>[ 1.688564] DMAR: Hardware identity mapping for device 0000:ff:16.1 >> >>[ 1.688565] DMAR: Hardware identity mapping for device 0000:ff:16.2 >> >>[ 1.688566] DMAR: Hardware identity mapping for device 0000:ff:16.3 >> >>[ 1.688568] DMAR: Hardware identity mapping for device 0000:ff:16.6 >> >>[ 1.688569] DMAR: Hardware identity mapping for device 0000:ff:16.7 >> >>[ 1.688571] DMAR: Hardware identity mapping for device 0000:ff:17.0 >> >>[ 1.688573] DMAR: Hardware identity mapping for device 0000:ff:17.1 >> >>[ 1.688574] DMAR: Hardware identity mapping for device 0000:ff:17.2 >> >>[ 1.688576] DMAR: Hardware identity mapping for device 0000:ff:17.3 >> >>[ 1.688577] DMAR: Hardware identity mapping for device 0000:ff:17.4 >> >>[ 1.688579] DMAR: Hardware identity mapping for device 0000:ff:17.5 >> >>[ 1.688581] DMAR: Hardware identity mapping for device 0000:ff:17.6 >> >>[ 1.688582] DMAR: Hardware identity mapping for device 0000:ff:17.7 >> >>[ 1.688584] DMAR: Hardware identity mapping for device 0000:ff:1e.0 >> >>[ 1.688586] DMAR: Hardware identity mapping for device 0000:ff:1e.1 >> >>[ 1.688588] DMAR: Hardware identity mapping for device 0000:ff:1e.2 >> >>[ 1.688590] DMAR: Hardware identity mapping for device 0000:ff:1e.3 >> >>[ 1.688592] DMAR: Hardware identity mapping for device 0000:ff:1e.4 >> >>[ 1.688594] DMAR: Hardware identity mapping for device 0000:ff:1f.0 >> >>[ 1.688596] DMAR: Hardware identity mapping for device 0000:ff:1f.2 >> >>[ 1.688610] DMAR: Hardware identity mapping for device 0000:7f:08.0 >> >>[ 1.688612] DMAR: Hardware identity mapping for device 0000:7f:08.2 >> >>[ 1.688614] DMAR: Hardware identity mapping for device 0000:7f:08.3 >> >>[ 1.688616] DMAR: Hardware identity mapping for device 0000:7f:09.0 >> >>[ 1.688617] DMAR: Hardware identity mapping for device 0000:7f:09.2 >> >>[ 1.688619] DMAR: Hardware identity mapping for device 0000:7f:09.3 >> >>[ 1.688621] DMAR: Hardware identity mapping for device 0000:7f:0b.0 >> >>[ 1.688623] DMAR: Hardware identity mapping for device 0000:7f:0b.1 >> >>[ 1.688625] DMAR: Hardware identity mapping for device 0000:7f:0b.2 >> >>[ 1.688627] DMAR: Hardware identity mapping for device 0000:7f:0c.0 >> >>[ 1.688629] DMAR: Hardware identity mapping for device 0000:7f:0c.1 >> >>[ 1.688631] DMAR: Hardware identity mapping for device 0000:7f:0c.2 >> >>[ 1.688632] DMAR: Hardware identity mapping for device 0000:7f:0c.3 >> >>[ 1.688634] DMAR: Hardware identity mapping for device 0000:7f:0c.4 >> >>[ 1.688636] DMAR: Hardware identity mapping for device 0000:7f:0c.5 >> >>[ 1.688637] DMAR: Hardware identity mapping for device 0000:7f:0c.6 >> >>[ 1.688639] DMAR: Hardware identity mapping for device 0000:7f:0c.7 >> >>[ 1.688641] DMAR: Hardware identity mapping for device 0000:7f:0d.0 >> >>[ 1.688642] DMAR: Hardware identity mapping for device 0000:7f:0d.1 >> >>[ 1.688644] DMAR: Hardware identity mapping for device 0000:7f:0d.2 >> >>[ 1.688646] DMAR: Hardware identity mapping for device 0000:7f:0d.3 >> >>[ 1.688647] DMAR: Hardware identity mapping for device 0000:7f:0d.4 >> >>[ 1.688649] DMAR: Hardware identity mapping for device 0000:7f:0d.5 >> >>[ 1.688650] DMAR: Hardware identity mapping for device 0000:7f:0d.6 >> >>[ 1.688652] DMAR: Hardware identity mapping for device 0000:7f:0d.7 >> >>[ 1.688653] DMAR: Hardware identity mapping for device 0000:7f:0e.0 >> >>[ 1.688655] DMAR: Hardware identity mapping for device 0000:7f:0e.1 >> >>[ 1.688657] DMAR: Hardware identity mapping for device 0000:7f:0f.0 >> >>[ 1.688658] DMAR: Hardware identity mapping for device 0000:7f:0f.1 >> >>[ 1.688660] DMAR: Hardware identity mapping for device 0000:7f:0f.2 >> >>[ 1.688661] DMAR: Hardware identity mapping for device 0000:7f:0f.3 >> >>[ 1.688663] DMAR: Hardware identity mapping for device 0000:7f:0f.4 >> >>[ 1.688664] DMAR: Hardware identity mapping for device 0000:7f:0f.5 >> >>[ 1.688666] DMAR: Hardware identity mapping for device 0000:7f:0f.6 >> >>[ 1.688668] DMAR: Hardware identity mapping for device 0000:7f:10.0 >> >>[ 1.688669] DMAR: Hardware identity mapping for device 0000:7f:10.1 >> >>[ 1.688671] DMAR: Hardware identity mapping for device 0000:7f:10.5 >> >>[ 1.688673] DMAR: Hardware identity mapping for device 0000:7f:10.6 >> >>[ 1.688674] DMAR: Hardware identity mapping for device 0000:7f:10.7 >> >>[ 1.688676] DMAR: Hardware identity mapping for device 0000:7f:12.0 >> >>[ 1.688678] DMAR: Hardware identity mapping for device 0000:7f:12.1 >> >>[ 1.688684] DMAR: Hardware identity mapping for device 0000:7f:12.4 >> >>[ 1.688686] DMAR: Hardware identity mapping for device 0000:7f:12.5 >> >>[ 1.688688] DMAR: Hardware identity mapping for device 0000:7f:13.0 >> >>[ 1.688689] DMAR: Hardware identity mapping for device 0000:7f:13.1 >> >>[ 1.688691] DMAR: Hardware identity mapping for device 0000:7f:13.2 >> >>[ 1.688692] DMAR: Hardware identity mapping for device 0000:7f:13.3 >> >>[ 1.688694] DMAR: Hardware identity mapping for device 0000:7f:13.6 >> >>[ 1.688696] DMAR: Hardware identity mapping for device 0000:7f:13.7 >> >>[ 1.688697] DMAR: Hardware identity mapping for device 0000:7f:14.0 >> >>[ 1.688699] DMAR: Hardware identity mapping for device 0000:7f:14.1 >> >>[ 1.688701] DMAR: Hardware identity mapping for device 0000:7f:14.2 >> >>[ 1.688703] DMAR: Hardware identity mapping for device 0000:7f:14.3 >> >>[ 1.688704] DMAR: Hardware identity mapping for device 0000:7f:14.4 >> >>[ 1.688706] DMAR: Hardware identity mapping for device 0000:7f:14.5 >> >>[ 1.688708] DMAR: Hardware identity mapping for device 0000:7f:14.6 >> >>[ 1.688710] DMAR: Hardware identity mapping for device 0000:7f:14.7 >> >>[ 1.688712] DMAR: Hardware identity mapping for device 0000:7f:16.0 >> >>[ 1.688714] DMAR: Hardware identity mapping for device 0000:7f:16.1 >> >>[ 1.688716] DMAR: Hardware identity mapping for device 0000:7f:16.2 >> >>[ 1.688718] DMAR: Hardware identity mapping for device 0000:7f:16.3 >> >>[ 1.688720] DMAR: Hardware identity mapping for device 0000:7f:16.6 >> >>[ 1.688722] DMAR: Hardware identity mapping for device 0000:7f:16.7 >> >>[ 1.688724] DMAR: Hardware identity mapping for device 0000:7f:17.0 >> >>[ 1.688726] DMAR: Hardware identity mapping for device 0000:7f:17.1 >> >>[ 1.688729] DMAR: Hardware identity mapping for device 0000:7f:17.2 >> >>[ 1.688731] DMAR: Hardware identity mapping for device 0000:7f:17.3 >> >>[ 1.688733] DMAR: Hardware identity mapping for device 0000:7f:17.4 >> >>[ 1.688735] DMAR: Hardware identity mapping for device 0000:7f:17.5 >> >>[ 1.688737] DMAR: Hardware identity mapping for device 0000:7f:17.6 >> >>[ 1.688739] DMAR: Hardware identity mapping for device 0000:7f:17.7 >> >>[ 1.688741] DMAR: Hardware identity mapping for device 0000:7f:1e.0 >> >>[ 1.688742] DMAR: Hardware identity mapping for device 0000:7f:1e.1 >> >>[ 1.688744] DMAR: Hardware identity mapping for device 0000:7f:1e.2 >> >>[ 1.688746] DMAR: Hardware identity mapping for device 0000:7f:1e.3 >> >>[ 1.688748] DMAR: Hardware identity mapping for device 0000:7f:1e.4 >> >>[ 1.688750] DMAR: Hardware identity mapping for device 0000:7f:1f.0 >> >>[ 1.688752] DMAR: Hardware identity mapping for device 0000:7f:1f.2 >> >>[ 1.688769] DMAR: Hardware identity mapping for device 0000:00:00.0 >> >>[ 1.688774] DMAR: Hardware identity mapping for device 0000:00:01.0 >> >>[ 1.688778] DMAR: Hardware identity mapping for device 0000:00:01.1 >> >>[ 1.688783] DMAR: Hardware identity mapping for device 0000:00:03.0 >> >>[ 1.688785] DMAR: Hardware identity mapping for device 0000:00:04.0 >> >>[ 1.688787] DMAR: Hardware identity mapping for device 0000:00:04.1 >> >>[ 1.688790] DMAR: Hardware identity mapping for device 0000:00:04.2 >> >>[ 1.688792] DMAR: Hardware identity mapping for device 0000:00:04.3 >> >>[ 1.688794] DMAR: Hardware identity mapping for device 0000:00:04.4 >> >>[ 1.688797] DMAR: Hardware identity mapping for device 0000:00:04.5 >> >>[ 1.688799] DMAR: Hardware identity mapping for device 0000:00:04.6 >> >>[ 1.688801] DMAR: Hardware identity mapping for device 0000:00:04.7 >> >>[ 1.688803] DMAR: Hardware identity mapping for device 0000:00:05.0 >> >>[ 1.688807] DMAR: Hardware identity mapping for device 0000:00:05.1 >> >>[ 1.688809] DMAR: Hardware identity mapping for device 0000:00:05.2 >> >>[ 1.688811] DMAR: Hardware identity mapping for device 0000:00:05.4 >> >>[ 1.688814] DMAR: Hardware identity mapping for device 0000:00:11.0 >> >>[ 1.688816] DMAR: Hardware identity mapping for device 0000:00:11.4 >> >>[ 1.688818] DMAR: Hardware identity mapping for device 0000:00:14.0 >> >>[ 1.688821] DMAR: Hardware identity mapping for device 0000:00:16.0 >> >>[ 1.688823] DMAR: Hardware identity mapping for device 0000:00:16.1 >> >>[ 1.688825] DMAR: Hardware identity mapping for device 0000:00:1a.0 >> >>[ 1.688828] DMAR: Hardware identity mapping for device 0000:00:1c.0 >> >>[ 1.688831] DMAR: Hardware identity mapping for device 0000:00:1c.2 >> >>[ 1.688833] DMAR: Hardware identity mapping for device 0000:00:1d.0 >> >>[ 1.688835] DMAR: Hardware identity mapping for device 0000:00:1f.0 >> >>[ 1.688837] DMAR: Hardware identity mapping for device 0000:00:1f.2 >> >>[ 1.688844] DMAR: Hardware identity mapping for device 0000:00:1f.3 >> >>[ 1.688846] DMAR: Hardware identity mapping for device 0000:00:1f.6 >> >>[ 1.688864] DMAR: Hardware identity mapping for device 0000:01:00.0 >> >>[ 1.688870] DMAR: Hardware identity mapping for device 0000:01:00.1 >> >>[ 1.688876] DMAR: Hardware identity mapping for device 0000:01:00.2 >> >>[ 1.688883] DMAR: Hardware identity mapping for device 0000:01:00.3 >> >>[ 1.688895] DMAR: Hardware identity mapping for device 0000:80:01.0 >> >>[ 1.688899] DMAR: Hardware identity mapping for device 0000:80:02.0 >> >>[ 1.688903] DMAR: Hardware identity mapping for device 0000:80:03.0 >> >>[ 1.688906] DMAR: Hardware identity mapping for device 0000:80:04.0 >> >>[ 1.688908] DMAR: Hardware identity mapping for device 0000:80:04.1 >> >>[ 1.688911] DMAR: Hardware identity mapping for device 0000:80:04.2 >> >>[ 1.688913] DMAR: Hardware identity mapping for device 0000:80:04.3 >> >>[ 1.688915] DMAR: Hardware identity mapping for device 0000:80:04.4 >> >>[ 1.688918] DMAR: Hardware identity mapping for device 0000:80:04.5 >> >>[ 1.688920] DMAR: Hardware identity mapping for device 0000:80:04.6 >> >>[ 1.688923] DMAR: Hardware identity mapping for device 0000:80:04.7 >> >>[ 1.688936] DMAR: Hardware identity mapping for device 0000:80:05.0 >> >>[ 1.688940] DMAR: Hardware identity mapping for device 0000:80:05.1 >> >>[ 1.688943] DMAR: Hardware identity mapping for device 0000:80:05.2 >> >>[ 1.688945] DMAR: Hardware identity mapping for device 0000:80:05.4 >> >>[ 1.688956] DMAR: Hardware identity mapping for device 0000:81:00.0 >> >>[ 1.688961] DMAR: Hardware identity mapping for device 0000:81:00.1 >> >>[ 1.688974] DMAR: Hardware identity mapping for device 0000:82:00.0 >> >>[ 1.688980] DMAR: Hardware identity mapping for device 0000:82:00.1 >> >>[ 1.688986] DMAR: Hardware identity mapping for device 0000:82:00.2 >> >>[ 1.688992] DMAR: Hardware identity mapping for device 0000:82:00.3 >> >>[ 1.689006] DMAR: Hardware identity mapping for device 0000:83:00.0 >> >>[ 1.689012] DMAR: Hardware identity mapping for device 0000:83:00.1 >> >>[ 1.689018] DMAR: Hardware identity mapping for device 0000:83:00.2 >> >>[ 1.689024] DMAR: Hardware identity mapping for device 0000:83:00.3 >> >>[ 1.689025] DMAR: Setting RMRR: >> >>[ 1.689026] DMAR: Ignoring identity map for HW passthrough device >> >>0000:00:14.0 [0x7ba75000 - 0x7ba84fff] >> >>[ 1.689027] DMAR: Ignoring identity map for HW passthrough device >> >>0000:00:1a.0 [0x7ba75000 - 0x7ba84fff] >> >>[ 1.689028] DMAR: Ignoring identity map for HW passthrough device >> >>0000:00:1d.0 [0x7ba75000 - 0x7ba84fff] >> >>[ 1.689036] DMAR: Prepare 0-16MiB unity mapping for LPC >> >>[ 1.689036] DMAR: Ignoring identity map for HW passthrough device >> >>0000:00:1f.0 [0x0 - 0xffffff] >> >>[ 1.689040] DMAR: Intel(R) Virtualization Technology for Directed I/O >> >>[ 2.299766] DMAR: 32bit 0000:00:11.4 uses non-identity mapping >> >>[ 2.301941] DMAR: 32bit 0000:00:1a.0 uses non-identity mapping >> >>[ 2.302679] DMAR: Setting identity map for device 0000:00:1a.0 >> [0x7ba75000 >> >- >> >>0x7ba84fff] >> >>[ 2.320910] DMAR: 32bit 0000:00:1d.0 uses non-identity mapping >> >>[ 2.321685] DMAR: Setting identity map for device 0000:00:1d.0 >> >[0x7ba75000 >> >>- 0x7ba84fff] >> >># >> >> >> >> >> >>On Tue, Mar 28, 2017 at 1:46 AM, Bodireddy, Bhanuprakash >> >><bhanuprakash.bodire...@intel.com> wrote: >> >>Hello shivaram, >> >> >> >>Darrell has already asked if VT-d is enabled? >> >>Your cmdline options are appropriate as you have iommu >> >>enabled(intel_iommu=on iommu=pt). >> >>can you also check the output of below mentioned command to see if VT-d >> >is >> >>enabled in BIOS? >> >> >> >>$ dmesg | grep -e DMAR -e IOMMU >> >> >> >>Regards, >> >>Bhanuprakash. >> >> >> >>From: ovs-discuss-boun...@openvswitch.org [mailto:ovs-discuss- >> >>boun...@openvswitch.org] On Behalf Of Shivaram Mysore >> >>Sent: Tuesday, March 28, 2017 6:25 AM >> >>To: Darrell Ball <db...@vmware.com> >> >>Cc: ovs-discuss@openvswitch.org >> >>Subject: Re: [ovs-discuss] Error attaching device to DPDK >> >> >> >># uname -r >> >>4.10.0-14-generic >> >> >> >># cat /proc/cmdline >> >>BOOT_IMAGE=/boot/vmlinuz-4.10.0-14-generic >> >root=/dev/mapper/intelsdn- >> >>-vg-root ro quiet intel_iommu=on iommu=pt default_hugepagesz=1G >> >>hugepagesz=1G hugepages=4 >> >> >> >># uname -a >> >>Linux intelsdn 4.10.0-14-generic #16-Ubuntu SMP Fri Mar 17 15:19:26 UTC >> >2017 >> >>x86_64 x86_64 x86_64 GNU/Linux >> >> >> >># cat /etc/*release >> >>DISTRIB_ID=Ubuntu >> >>DISTRIB_RELEASE=17.04 >> >>DISTRIB_CODENAME=zesty >> >>DISTRIB_DESCRIPTION="Ubuntu Zesty Zapus (development branch)" >> >>NAME="Ubuntu" >> >>VERSION="17.04 (Zesty Zapus)" >> >>ID=ubuntu >> >>ID_LIKE=debian >> >>PRETTY_NAME="Ubuntu Zesty Zapus (development branch)" >> >>VERSION_ID="17.04" >> >> >> >> >> >>Nothing in the syslog related >> >> >> >>On Mon, Mar 27, 2017 at 10:02 PM, Darrell Ball <db...@vmware.com> >> wrote: >> >>I see the pastebins now. >> >> >> >>What kernel version are you using ? >> >>VT-d enabled ? >> >>Anything related in syslog ? >> >> >> >>From: Shivaram Mysore <shivaram.mys...@gmail.com> >> >>Date: Monday, March 27, 2017 at 9:34 PM >> >>To: Darrell Ball <db...@vmware.com> >> >>Cc: "ovs-discuss@openvswitch.org" <ovs-discuss@openvswitch.org> >> >>Subject: Re: [ovs-discuss] Error attaching device to DPDK >> >> >> >>One more datapoint, please check the paste bin log in URL sent >> >earlier. Dpdk- >> >>devbind works and you can see results in the paste bin log >> >> >> >>/Shivaram >> >>::Sent from my mobile device:: >> >> >> >>On Mar 27, 2017, at 8:34 PM, Darrell Ball <db...@vmware.com> wrote: >> >>see inline below >> >> >> >>But a few questions: >> >>1) Are you using the recommended version of dpdk with OVS 2.7 – >> 16.11 ? >> >>2) Are you getting a crash still or is this post crash ? >> >>3) Was the previous crash still happening with 16.11 or only 17.02 >> ? >> >> >> >>Thanks >> >>Darrell >> >> >> >>From: <ovs-discuss-boun...@openvswitch.org> on behalf of Shivaram >> >>Mysore <shivaram.mys...@gmail.com> >> >>Date: Monday, March 27, 2017 at 7:45 PM >> >>To: "ovs-discuss@openvswitch.org" <ovs-discuss@openvswitch.org> >> >>Subject: [ovs-discuss] Error attaching device to DPDK >> >> >> >>Hello, >> >> >> >>I have installed OVS 2.7 running. When I do: >> >>1. # ovs-vsctl add-port ovs-ip64-br0 dpdk-p0 -- set Interface >> dpdk-p0 >> >>type=dpdk options:dpdk-devargs=0000:82:00.0 >> >>2. ovs-vsctl: Error detected while setting up 'dpdk-p0': Error >> attaching >> >>device '0000:82:00.0' to DPDK. See ovs-vswitchd log for details. >> >> >> >>Possibly, the port is not bound. >> >>Check using >> >>$DPDK_DIR/tools/dpdk-devbind.py –status >> >>and see the documentation for more detail >> >> >> >>and the log file shows: >> >>1. 2017-03-28T02:04:40.004Z|00064|netdev_dpdk|WARN|Error attaching >> >>device '0000:82:00.0' to DPDK >> >>2. 2017-03-28T02:04:40.004Z|00065|netdev|WARN|dpdk-p0: could >> >>not set configuration (Invalid argument) >> >> >> >>What would cause such a situation? >> >> >> >>Commands run: https://pastebin.com/0P754x0t >> >>Detailed OVS Log: https://pastebin.com/ctBhLuRG >> >> >> >> >> > >> >> > > _______________________________________________ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > >
_______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
