Re: [E1000-devel] when SR-IOV is used with Intel 82599 NIC, multicast packets are dropped and not delivered to VF(Virtual Function)
Hi Chen,
I have tried to answer your questions as best I can below. However I am
not the maintainer for the ixgbe SR-IOV functionality. It is Greg Rose
who is the maintainer so I have added him and the e1000-devel mailing
list to this email.
Thanks,
Alex
On 12/19/2012 07:08 AM, Rugang Chen wrote:
Hi Alexander Duyck,
We are building a Virtualization test bed with Intel 82599 NIC on Linux
Host and KVM is used. Following is a brief description:
*Host system:* Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic x86_64)
*NIC: *Intel 82599EB, ixgbe driver version is 3.11.33. See more details
in attachment lspci.log
*KVM: *QEMU emulator version 1.0 (qemu-kvm-1.0), Copyright (c) 2003-2008
Fabrice Bellard (run kvm --version)**
*Virtual Machines:*
· Two Virtual Machines are installed with a same Linux
kernel(2.6.26.5), and version of ixgbevf driver running on Virtual
Machines is 2.6.2
· There's a VF (Virtual Function) used as NIC in each Virtual
Machine, and the two VFs are of the same PF(Physical Function)
*Configuration:*
· Both PF (Intel 8259 NIC) and VF(ixgbevf NIC seen in Virtual
Machine) are set to promisc on multicast on allmulticast on with tool
iproute2
· Load ixgbe.ko in Host with parameter max_vfs=16
*Result: *
· When sending multicast packets out from one Virtual Machine A,
no packet is seen in another Virtual Machine B**
In our testing, multicast MAC address in packets is “01:00:5e:00:00:d2”
· On Virtual Machine, when we run ip maddr add
01:00:5e:00:00:d2 dev ethx(ethx = interface name seen in Virtual
Machine), we can debug that ixgbe_update_mc_addr_list_vf is called **
· Communication of unicast from A to B is OK**
We checked the ixgbe driver codes(version 3.11.33), and seen following:
(Function “*ixgbe_set_rx_mode*”, file “*ixgbe_main.c*”)
if (hw-mac.type !=ixgbe_mac_82598EB) {
vmolr |= IXGBE_READ_REG(hw, IXGBE_VMOLR(VMDQ_P(0)))
~(IXGBE_VMOLR_MPE|IXGBE_VMOLR_ROMPE|
IXGBE_VMOLR_ROPE);
IXGBE_WRITE_REG(hw, IXGBE_VMOLR(VMDQ_P(0)), vmolr);
}
We can see that“IXGBE_VMOLR_MPE” and“IXGBE_VMOLR_ROMPE” are disabled,
and this may explain why multicast doesn’t work.
Checking the previous version of file ixgbe_main.c shipped in kernel
source, we found that “IXGBE_VMOLR_MPE” and“IXGBE_VMOLR_ROMPE” are
disabled since the driver was firstly in the kernel.
So our question are:
· Why we disable “IXGBE_VMOLR_MPE” and“IXGBE_VMOLR_ROMPE”, and
only enable them for 82598EB in latest 3.11.33?
The code you are referencing above only affects the PF specifically.
The VMDQ_P(0) macro is accessing the offloads register for the PF itself.
The VMOLR_MPE bit would be undesirable as that enables receive of all
multicast packets. As far as the VMOLR_ROMPE bit it should be enabled
the first time a VF has a reset event. I believe you should see this in
ixgbe_sriov.c if you grep for VMOLR_ROMPE in the driver.
The 82598EB does not have a VMOLR register and does not support SR-IOV.
This is why it is excluded in the code you reference above.
· Multicast from VF to VF (or between VF and PF) is supported ?
Yes it is my understanding that this is supported.
· If we just enable “IXGBE_VMOLR_MPE” and“IXGBE_VMOLR_ROMPE” for
multicast on 82599EB, will there be any other influence ?
The problem with enabling these by default is that it would mean that
the PF would always be receiving multicast traffic even for groups that
it did not add.
Thanks for your patience to read the mail. Wait for your answer which
may be very helpful to us.
Best regards,
Rugang Chen
--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit
http://communities.intel.com/community/wired
Re: [E1000-devel] when SR-IOV is used with Intel 82599 NIC, multicast packets are dropped and not delivered to VF(Virtual Function)
We do not enable multicast promiscuous. The VF drivers do not support any form
of promiscuous receive behavior. If you want to enable multicast promiscuous
for VFs then you'll have to customize the driver for your own use and then
carry the patch that does so locally.
This command:
promisc on multicast on allmulticast on
has no effect on the VF driver.
Hope that helps, if you have further questions let me know.
Thanks,
- Greg
-Original Message-
From: Duyck, Alexander H
Sent: Wednesday, December 19, 2012 8:48 AM
To: Rugang Chen
Cc: Rose, Gregory V; E1000-devel@lists.sourceforge.net
Subject: Re: when SR-IOV is used with Intel 82599 NIC, multicast packets
are dropped and not delivered to VF(Virtual Function)
Hi Chen,
I have tried to answer your questions as best I can below. However I am
not the maintainer for the ixgbe SR-IOV functionality. It is Greg Rose
who is the maintainer so I have added him and the e1000-devel mailing list
to this email.
Thanks,
Alex
On 12/19/2012 07:08 AM, Rugang Chen wrote:
Hi Alexander Duyck,
We are building a Virtualization test bed with Intel 82599 NIC on
Linux Host and KVM is used. Following is a brief description:
*Host system:* Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic x86_64)
*NIC: *Intel 82599EB, ixgbe driver version is 3.11.33. See more
details in attachment lspci.log
*KVM: *QEMU emulator version 1.0 (qemu-kvm-1.0), Copyright (c)
2003-2008 Fabrice Bellard (run kvm --version)**
*Virtual Machines:*
* Two Virtual Machines are installed with a same Linux
kernel(2.6.26.5), and version of ixgbevf driver running on Virtual
Machines is 2.6.2
* There's a VF (Virtual Function) used as NIC in each Virtual
Machine, and the two VFs are of the same PF(Physical Function)
*Configuration:*
* Both PF (Intel 8259 NIC) and VF(ixgbevf NIC seen in Virtual
Machine) are set to promisc on multicast on allmulticast on with
tool iproute2
* Load ixgbe.ko in Host with parameter max_vfs=16
*Result: *
* When sending multicast packets out from one Virtual Machine A,
no packet is seen in another Virtual Machine B**
In our testing, multicast MAC address in packets is 01:00:5e:00:00:d2
* On Virtual Machine, when we run ip maddr add
01:00:5e:00:00:d2 dev ethx(ethx = interface name seen in Virtual
Machine), we can debug that ixgbe_update_mc_addr_list_vf is called
**
* Communication of unicast from A to B is OK**
We checked the ixgbe driver codes(version 3.11.33), and seen following:
(Function *ixgbe_set_rx_mode*, file *ixgbe_main.c*)
if (hw-mac.type !=ixgbe_mac_82598EB) {
vmolr |= IXGBE_READ_REG(hw, IXGBE_VMOLR(VMDQ_P(0)))
~(IXGBE_VMOLR_MPE|IXGBE_VMOLR_ROMPE|
IXGBE_VMOLR_ROPE);
IXGBE_WRITE_REG(hw, IXGBE_VMOLR(VMDQ_P(0)), vmolr);
}
We can see thatIXGBE_VMOLR_MPE andIXGBE_VMOLR_ROMPE are disabled,
and this may explain why multicast doesn't work.
Checking the previous version of file ixgbe_main.c shipped in kernel
source, we found that IXGBE_VMOLR_MPE andIXGBE_VMOLR_ROMPE are
disabled since the driver was firstly in the kernel.
So our question are:
* Why we disable IXGBE_VMOLR_MPE andIXGBE_VMOLR_ROMPE, and
only enable them for 82598EB in latest 3.11.33?
The code you are referencing above only affects the PF specifically.
The VMDQ_P(0) macro is accessing the offloads register for the PF itself.
The VMOLR_MPE bit would be undesirable as that enables receive of all
multicast packets. As far as the VMOLR_ROMPE bit it should be enabled the
first time a VF has a reset event. I believe you should see this in
ixgbe_sriov.c if you grep for VMOLR_ROMPE in the driver.
The 82598EB does not have a VMOLR register and does not support SR-IOV.
This is why it is excluded in the code you reference above.
* Multicast from VF to VF (or between VF and PF) is supported ?
Yes it is my understanding that this is supported.
* If we just enable IXGBE_VMOLR_MPE andIXGBE_VMOLR_ROMPE for
multicast on 82599EB, will there be any other influence ?
The problem with enabling these by default is that it would mean that the
PF would always be receiving multicast traffic even for groups that it did
not add.
Thanks for your patience to read the mail. Wait for your answer which
may be very helpful to us.
Best regards,
Rugang Chen
--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
E1000-devel
Re: [E1000-devel] epoll with ONESHOT possibly fails to deliver events
We (i.e. I together with my colleague Jason Wang, cc'ed) installed the latest stable kernel (3.7.1) and verified that the bug still occurs. The bug occurs when testing the program across a network link and when testing on the loopback interface. We also noticed that when testing across the network, the hardware irq affinity settings (in the /proc/irq/ files) affect how frequently the bug occurs. When we use the default irq settings, the bug does not seem to occur, whereas when we use settings suggested by Intel (with script set_irq_affinity.sh) the bug occurs more frequently. Cheers, Andi On Dec 18, 2012, at 12:27 PM, Andreas Voellmy andreas.voel...@yale.edu wrote: BTW, I simplified the test program a bit: I removed the loop that epoll_waits on the eventfd fd and reads from it (I also removed the epoll instance in that loop). The bug still occurs with this removed. Now the bug is triggered simply by adding the call to eventfd_write after processing each request. I pushed the update to my github project for the test program. -Andi On Dec 17, 2012, at 9:35 PM, Andreas Voellmy andreas.voel...@yale.edu wrote: On Dec 17, 2012, at 9:07 PM, Eric Wong normalper...@yhbt.net wrote: Andreas Voellmy andreas.voel...@yale.edu wrote: There were a couple of errors in the code when I posted my last message. I have fixed those. The epoll bug still occurs. Sorry I haven't gotten around to this. Can you reproduce this with fewer cores? (I only have 4 at most). I've been able to reproduce it on as few as 8 cores. I've never seen it occur with fewer than that. Have you tried the latest stable kernel version? No, I've only tried 3.4.0.0. Can you reproduce this over TCP loopback, or only across two machines? If the latter, it could also be a driver or firmware bug... Yes, it also occurs when I run the http request generator on the same machine on the loopback interface. -Andi -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel#174; Ethernet, visit http://communities.intel.com/community/wired
[E1000-devel] Patches for building ixgbe-3.11.33 on Linux 3.7.1
Dear list, We were trying to build ixgbe-3.11.33 on Linux 3.7.1 and encountered several problems. We fixed them and hope the following patch help your releasing of new version. --Junchang diff --git a/Makefile b/Makefile index 1e490fe..51c7ea9 100644 --- a/Makefile +++ b/Makefile @@ -88,6 +88,7 @@ endif VSP := $(KOBJ)/include/generated/utsrelease.h \ $(KOBJ)/include/linux/utsrelease.h \ $(KOBJ)/include/linux/version.h \ + $(KOBJ)/include/generated/uapi/linux/version.h \ /boot/vmlinuz.version.h # Config file Search Path diff --git a/ixgbe_main.c b/ixgbe_main.c index 809d1f9..ae106c9 100644 --- a/ixgbe_main.c +++ b/ixgbe_main.c @@ -9108,7 +9108,7 @@ static pci_ers_result_t ixgbe_io_error_detected(struct pci_dev *pdev, goto skip_bad_vf_detection; bdev = pdev-bus-self; - while (bdev (bdev-pcie_type != PCI_EXP_TYPE_ROOT_PORT)) + while (bdev (pci_pcie_type(bdev) != PCI_EXP_TYPE_ROOT_PORT)) bdev = bdev-bus-self; if (!bdev) -- ixgbe.patch Description: Binary data -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d___ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel#174; Ethernet, visit http://communities.intel.com/community/wired
Re: [E1000-devel] Patches for building ixgbe-3.11.33 on Linux 3.7.1
-Original Message- From: Junchang Wang [mailto:junchangw...@gmail.com] Sent: Wednesday, December 19, 2012 11:57 AM To: e1000-devel@lists.sourceforge.net Cc: Andreas Voellmy Subject: [E1000-devel] Patches for building ixgbe-3.11.33 on Linux 3.7.1 Dear list, We were trying to build ixgbe-3.11.33 on Linux 3.7.1 and encountered several problems. We fixed them and hope the following patch help your releasing of new version. Thanks for the patch, this was brought up before. We have a new version that will come out this week that will build on 3.7. Emil -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel#174; Ethernet, visit http://communities.intel.com/community/wired
