Re: [E1000-devel] dropped rx with i40e
Stefan, I've run into a glitch while in the process of configuring a setup for repo - we need to do some reconfig in our lab. I've put in the appropriate lab request and they're usually pretty good about quick response but I may not get the test running tonight. If not I'll get it started ASAP. Thanks for your patience. - Greg > -Original Message- > From: Rose, Gregory V [mailto:gregory.v.r...@intel.com] > Sent: Thursday, August 13, 2015 1:16 PM > To: Stefan Priebe; e1000-devel@lists.sourceforge.net > Subject: Re: [E1000-devel] dropped rx with i40e > > > -Original Message- > > From: Stefan Priebe [mailto:s.pri...@profihost.ag] > > Sent: Thursday, August 13, 2015 12:02 PM > > To: Rose, Gregory V; e1000-devel@lists.sourceforge.net > > Subject: Re: [E1000-devel] dropped rx with i40e > > > > Am 13.08.2015 um 20:59 schrieb Rose, Gregory V: > > > Thanks Stefan, I'll set up a test to replicate your traffic profile > > > as > > closely as possible and let it run overnight to see if I can repro and > > then update you tomorrow. > > > > > > It does seem that it has nothing to do with load so that makes it > > > even > > more curious. > > > > May it be related to jumbo frames? > > Could be - I'll be looking at that angle during my testing and evaluation. > > - Greg > > > > > Stefan > > > > > > > > - Greg > > > > > >> -Original Message- > > >> From: Stefan Priebe [mailto:s.pri...@profihost.ag] > > >> Sent: Thursday, August 13, 2015 11:53 AM > > >> To: Rose, Gregory V; e1000-devel@lists.sourceforge.net > > >> Subject: Re: [E1000-devel] dropped rx with i40e > > >> > > >> Hi, > > >> > > >> sorry for top posting. > > >> > > >> I will try to describe the workload as good as i can. > > >> > > >> Application is ceph storage (http://ceph.com/). > > >> > > >> Workload is TCP Only, Active/Active bond on both ports of the XL710 > > >> card and jumbo frames (MTU 9000). Traffic peak was 400MBit/s - So > > >> overall speed does not seem to matter. Also i can use iperf and get > > >> a constant speed of 9.8Gb/s in both directions without any rx drops. > > >> > > >> The drops don't occur regulary they just happen at a time X and > > >> then > > stop. > > >> After some hours it happens again. > > >> Stefan > > >> > > >> Am 13.08.2015 um 17:58 schrieb Rose, Gregory V: > > >>> My apologies but I've been unable to get back to this issue. > > >>> > > >>> After reviewing the thread I don't see anything about steps to > > >>> reproduce > > >> the problem. I understand that you're seeing dropped packets with > > >> the > > >> Xl710 with various versions of the i40e driver while the X520 with > > >> the ixgbe driver does not drop packets under the same load. > > >>> > > >>> I don't' see any description of the type of traffic load that is > > >>> causing > > >> the problem. That would help me to reproduce the issue. > > >>> > > >>> Keep in mind that dropped packets in and of itself is not a bug. > > >>> It may > > >> mean that the X520 and the ixgbe driver are more mature and have > > >> had more "tuning" and thus work better under the type of traffic > > >> load you have on your network. Thus it is important that we > > >> understand the type of traffic you're seeing on your network so > > >> that we can work on making the XL710 and i40e driver performance on > > >> par with the X520 and > > the ixgbe driver. > > >>> > > >>> One other thing. Below I notice this: > > >>> > > I tested this one: > > ethtool -C eth3 adaptive-rx off adaptive-tx off rx-usecs 2 > > tx-usecs > > 0 > > >>> > > >>> I believe that you would be better off using higher values. > > >>> Really low > > >> values mean the HW interrupt will fire more often - instead you > > >> should allow the soft IRQ polling to keep processing packets. > > >>> > > >>> - Greg > > >>> > > -Original Message- > > From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag] > > Sent: Thursday, August 13, 2015 5:41 AM > > To: Rose, Gregory V; e1000-devel@lists.sourceforge.net > > Subject: Re: [E1000-devel] dropped rx with i40e > > > > 1.3.12-k from net-next devel does not help either ;-( > > > > Should we open an intel support ticket? We really need a solution. > > > > Stefan > > > > Am 12.08.2015 um 10:29 schrieb Stefan Priebe - Profihost AG: > > > Might this be a memory allocation problem? It happens only after > > > some hours running and when the whole memory is filled with > > > linux fs > > >> cache. > > > > > > Is the i40e driver using kmalloc or vmalloc? > > > > > > Stefan > > > Am 11.08.2015 um 06:03 schrieb Stefan Priebe: > > >> One more thing to note. It mostly happens after around 8-24 > > >> hours and i could stop it again by rebooting the system/server. > > >> (can't prove > > >> it) > > >> > > >> Stefan > > >> Am 06.08.2015 um 22:59 schrieb Rose, Gregory V: > > >>> Thanks Stefan. I
Re: [E1000-devel] dropped rx with i40e
> -Original Message- > From: Stefan Priebe [mailto:s.pri...@profihost.ag] > Sent: Thursday, August 13, 2015 12:02 PM > To: Rose, Gregory V; e1000-devel@lists.sourceforge.net > Subject: Re: [E1000-devel] dropped rx with i40e > > Am 13.08.2015 um 20:59 schrieb Rose, Gregory V: > > Thanks Stefan, I'll set up a test to replicate your traffic profile as > closely as possible and let it run overnight to see if I can repro and > then update you tomorrow. > > > > It does seem that it has nothing to do with load so that makes it even > more curious. > > May it be related to jumbo frames? Could be - I'll be looking at that angle during my testing and evaluation. - Greg > > Stefan > > > > > - Greg > > > >> -Original Message- > >> From: Stefan Priebe [mailto:s.pri...@profihost.ag] > >> Sent: Thursday, August 13, 2015 11:53 AM > >> To: Rose, Gregory V; e1000-devel@lists.sourceforge.net > >> Subject: Re: [E1000-devel] dropped rx with i40e > >> > >> Hi, > >> > >> sorry for top posting. > >> > >> I will try to describe the workload as good as i can. > >> > >> Application is ceph storage (http://ceph.com/). > >> > >> Workload is TCP Only, Active/Active bond on both ports of the XL710 > >> card and jumbo frames (MTU 9000). Traffic peak was 400MBit/s - So > >> overall speed does not seem to matter. Also i can use iperf and get a > >> constant speed of 9.8Gb/s in both directions without any rx drops. > >> > >> The drops don't occur regulary they just happen at a time X and then > stop. > >> After some hours it happens again. > >> Stefan > >> > >> Am 13.08.2015 um 17:58 schrieb Rose, Gregory V: > >>> My apologies but I've been unable to get back to this issue. > >>> > >>> After reviewing the thread I don't see anything about steps to > >>> reproduce > >> the problem. I understand that you're seeing dropped packets with > >> the > >> Xl710 with various versions of the i40e driver while the X520 with > >> the ixgbe driver does not drop packets under the same load. > >>> > >>> I don't' see any description of the type of traffic load that is > >>> causing > >> the problem. That would help me to reproduce the issue. > >>> > >>> Keep in mind that dropped packets in and of itself is not a bug. It > >>> may > >> mean that the X520 and the ixgbe driver are more mature and have had > >> more "tuning" and thus work better under the type of traffic load you > >> have on your network. Thus it is important that we understand the > >> type of traffic you're seeing on your network so that we can work on > >> making the XL710 and i40e driver performance on par with the X520 and > the ixgbe driver. > >>> > >>> One other thing. Below I notice this: > >>> > I tested this one: > ethtool -C eth3 adaptive-rx off adaptive-tx off rx-usecs 2 tx-usecs > 0 > >>> > >>> I believe that you would be better off using higher values. Really > >>> low > >> values mean the HW interrupt will fire more often - instead you > >> should allow the soft IRQ polling to keep processing packets. > >>> > >>> - Greg > >>> > -Original Message- > From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag] > Sent: Thursday, August 13, 2015 5:41 AM > To: Rose, Gregory V; e1000-devel@lists.sourceforge.net > Subject: Re: [E1000-devel] dropped rx with i40e > > 1.3.12-k from net-next devel does not help either ;-( > > Should we open an intel support ticket? We really need a solution. > > Stefan > > Am 12.08.2015 um 10:29 schrieb Stefan Priebe - Profihost AG: > > Might this be a memory allocation problem? It happens only after > > some hours running and when the whole memory is filled with linux > > fs > >> cache. > > > > Is the i40e driver using kmalloc or vmalloc? > > > > Stefan > > Am 11.08.2015 um 06:03 schrieb Stefan Priebe: > >> One more thing to note. It mostly happens after around 8-24 hours > >> and i could stop it again by rebooting the system/server. (can't > >> prove > >> it) > >> > >> Stefan > >> Am 06.08.2015 um 22:59 schrieb Rose, Gregory V: > >>> Thanks Stefan. I think for now you've given us enough data to > >>> go on > >>> - I've got some research to do and then I'll get back to you. > >>> > >>> - Greg > >>> > -Original Message- > From: Stefan Priebe - Profihost AG > [mailto:s.pri...@profihost.ag] > Sent: Wednesday, August 05, 2015 11:32 PM > To: Rose, Gregory V; e1000-devel@lists.sourceforge.net > Subject: Re: [E1000-devel] dropped rx with i40e > > Am 06.08.2015 um 00:22 schrieb Rose, Gregory V: > > Stefan, > > > > Could you please send me the output of 'ethtool' and 'ethtool - > i' > > for > each i40e interface that is experiencing the dropped packets > issue? > > These are around 100 cards. So i won't post the output for a
Re: [E1000-devel] dropped rx with i40e
Thanks Stefan, I'll set up a test to replicate your traffic profile as closely as possible and let it run overnight to see if I can repro and then update you tomorrow. It does seem that it has nothing to do with load so that makes it even more curious. - Greg > -Original Message- > From: Stefan Priebe [mailto:s.pri...@profihost.ag] > Sent: Thursday, August 13, 2015 11:53 AM > To: Rose, Gregory V; e1000-devel@lists.sourceforge.net > Subject: Re: [E1000-devel] dropped rx with i40e > > Hi, > > sorry for top posting. > > I will try to describe the workload as good as i can. > > Application is ceph storage (http://ceph.com/). > > Workload is TCP Only, Active/Active bond on both ports of the XL710 card > and jumbo frames (MTU 9000). Traffic peak was 400MBit/s - So overall speed > does not seem to matter. Also i can use iperf and get a constant speed of > 9.8Gb/s in both directions without any rx drops. > > The drops don't occur regulary they just happen at a time X and then stop. > After some hours it happens again. > Stefan > > Am 13.08.2015 um 17:58 schrieb Rose, Gregory V: > > My apologies but I've been unable to get back to this issue. > > > > After reviewing the thread I don't see anything about steps to reproduce > the problem. I understand that you're seeing dropped packets with the > Xl710 with various versions of the i40e driver while the X520 with the > ixgbe driver does not drop packets under the same load. > > > > I don't' see any description of the type of traffic load that is causing > the problem. That would help me to reproduce the issue. > > > > Keep in mind that dropped packets in and of itself is not a bug. It may > mean that the X520 and the ixgbe driver are more mature and have had more > "tuning" and thus work better under the type of traffic load you have on > your network. Thus it is important that we understand the type of traffic > you're seeing on your network so that we can work on making the XL710 and > i40e driver performance on par with the X520 and the ixgbe driver. > > > > One other thing. Below I notice this: > > > >> I tested this one: > >> ethtool -C eth3 adaptive-rx off adaptive-tx off rx-usecs 2 tx-usecs 0 > > > > I believe that you would be better off using higher values. Really low > values mean the HW interrupt will fire more often - instead you should > allow the soft IRQ polling to keep processing packets. > > > > - Greg > > > >> -Original Message- > >> From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag] > >> Sent: Thursday, August 13, 2015 5:41 AM > >> To: Rose, Gregory V; e1000-devel@lists.sourceforge.net > >> Subject: Re: [E1000-devel] dropped rx with i40e > >> > >> 1.3.12-k from net-next devel does not help either ;-( > >> > >> Should we open an intel support ticket? We really need a solution. > >> > >> Stefan > >> > >> Am 12.08.2015 um 10:29 schrieb Stefan Priebe - Profihost AG: > >>> Might this be a memory allocation problem? It happens only after > >>> some hours running and when the whole memory is filled with linux fs > cache. > >>> > >>> Is the i40e driver using kmalloc or vmalloc? > >>> > >>> Stefan > >>> Am 11.08.2015 um 06:03 schrieb Stefan Priebe: > One more thing to note. It mostly happens after around 8-24 hours > and i could stop it again by rebooting the system/server. (can't > prove > it) > > Stefan > Am 06.08.2015 um 22:59 schrieb Rose, Gregory V: > > Thanks Stefan. I think for now you've given us enough data to go > > on > > - I've got some research to do and then I'll get back to you. > > > > - Greg > > > >> -Original Message- > >> From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag] > >> Sent: Wednesday, August 05, 2015 11:32 PM > >> To: Rose, Gregory V; e1000-devel@lists.sourceforge.net > >> Subject: Re: [E1000-devel] dropped rx with i40e > >> > >> Am 06.08.2015 um 00:22 schrieb Rose, Gregory V: > >>> Stefan, > >>> > >>> Could you please send me the output of 'ethtool' and 'ethtool -i' > >>> for > >> each i40e interface that is experiencing the dropped packets issue? > >> > >> These are around 100 cards. So i won't post the output for all of > >> them. > >> As they're all using the same driver and the same firmware - we > >> updated all of them i hope it's ok to post the output only from > >> one > >> of them. > >> > >> # ethtool eth2 > >> Settings for eth2: > >> Supported ports: [ FIBRE ] > >> Supported link modes: 1baseT/Full > >> Supported pause frame use: Symmetric > >> Supports auto-negotiation: No > >> Advertised link modes: Not reported > >> Advertised pause frame use: No > >> Advertised auto-negotiation: No > >> Speed: 1Mb/s > >> Duplex: Full > >> Port: Direct Attach Copper > >>
Re: [E1000-devel] dropped rx with i40e
Am 13.08.2015 um 20:59 schrieb Rose, Gregory V: > Thanks Stefan, I'll set up a test to replicate your traffic profile as > closely as possible and let it run overnight to see if I can repro and then > update you tomorrow. > > It does seem that it has nothing to do with load so that makes it even more > curious. May it be related to jumbo frames? Stefan > > - Greg > >> -Original Message- >> From: Stefan Priebe [mailto:s.pri...@profihost.ag] >> Sent: Thursday, August 13, 2015 11:53 AM >> To: Rose, Gregory V; e1000-devel@lists.sourceforge.net >> Subject: Re: [E1000-devel] dropped rx with i40e >> >> Hi, >> >> sorry for top posting. >> >> I will try to describe the workload as good as i can. >> >> Application is ceph storage (http://ceph.com/). >> >> Workload is TCP Only, Active/Active bond on both ports of the XL710 card >> and jumbo frames (MTU 9000). Traffic peak was 400MBit/s - So overall speed >> does not seem to matter. Also i can use iperf and get a constant speed of >> 9.8Gb/s in both directions without any rx drops. >> >> The drops don't occur regulary they just happen at a time X and then stop. >> After some hours it happens again. >> Stefan >> >> Am 13.08.2015 um 17:58 schrieb Rose, Gregory V: >>> My apologies but I've been unable to get back to this issue. >>> >>> After reviewing the thread I don't see anything about steps to reproduce >> the problem. I understand that you're seeing dropped packets with the >> Xl710 with various versions of the i40e driver while the X520 with the >> ixgbe driver does not drop packets under the same load. >>> >>> I don't' see any description of the type of traffic load that is causing >> the problem. That would help me to reproduce the issue. >>> >>> Keep in mind that dropped packets in and of itself is not a bug. It may >> mean that the X520 and the ixgbe driver are more mature and have had more >> "tuning" and thus work better under the type of traffic load you have on >> your network. Thus it is important that we understand the type of traffic >> you're seeing on your network so that we can work on making the XL710 and >> i40e driver performance on par with the X520 and the ixgbe driver. >>> >>> One other thing. Below I notice this: >>> I tested this one: ethtool -C eth3 adaptive-rx off adaptive-tx off rx-usecs 2 tx-usecs 0 >>> >>> I believe that you would be better off using higher values. Really low >> values mean the HW interrupt will fire more often - instead you should >> allow the soft IRQ polling to keep processing packets. >>> >>> - Greg >>> -Original Message- From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag] Sent: Thursday, August 13, 2015 5:41 AM To: Rose, Gregory V; e1000-devel@lists.sourceforge.net Subject: Re: [E1000-devel] dropped rx with i40e 1.3.12-k from net-next devel does not help either ;-( Should we open an intel support ticket? We really need a solution. Stefan Am 12.08.2015 um 10:29 schrieb Stefan Priebe - Profihost AG: > Might this be a memory allocation problem? It happens only after > some hours running and when the whole memory is filled with linux fs >> cache. > > Is the i40e driver using kmalloc or vmalloc? > > Stefan > Am 11.08.2015 um 06:03 schrieb Stefan Priebe: >> One more thing to note. It mostly happens after around 8-24 hours >> and i could stop it again by rebooting the system/server. (can't >> prove >> it) >> >> Stefan >> Am 06.08.2015 um 22:59 schrieb Rose, Gregory V: >>> Thanks Stefan. I think for now you've given us enough data to go >>> on >>> - I've got some research to do and then I'll get back to you. >>> >>> - Greg >>> -Original Message- From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag] Sent: Wednesday, August 05, 2015 11:32 PM To: Rose, Gregory V; e1000-devel@lists.sourceforge.net Subject: Re: [E1000-devel] dropped rx with i40e Am 06.08.2015 um 00:22 schrieb Rose, Gregory V: > Stefan, > > Could you please send me the output of 'ethtool' and 'ethtool -i' > for each i40e interface that is experiencing the dropped packets issue? These are around 100 cards. So i won't post the output for all of them. As they're all using the same driver and the same firmware - we updated all of them i hope it's ok to post the output only from one of them. # ethtool eth2 Settings for eth2: Supported ports: [ FIBRE ] Supported link modes: 1baseT/Full Supported pause frame use: Symmetric Supports auto-negotiation: No Advertised link modes: Not reported Advertised pause frame use: No Advertised a
Re: [E1000-devel] dropped rx with i40e
Hi, sorry for top posting. I will try to describe the workload as good as i can. Application is ceph storage (http://ceph.com/). Workload is TCP Only, Active/Active bond on both ports of the XL710 card and jumbo frames (MTU 9000). Traffic peak was 400MBit/s - So overall speed does not seem to matter. Also i can use iperf and get a constant speed of 9.8Gb/s in both directions without any rx drops. The drops don't occur regulary they just happen at a time X and then stop. After some hours it happens again. Stefan Am 13.08.2015 um 17:58 schrieb Rose, Gregory V: > My apologies but I've been unable to get back to this issue. > > After reviewing the thread I don't see anything about steps to reproduce the > problem. I understand that you're seeing dropped packets with the Xl710 with > various versions of the i40e driver while the X520 with the ixgbe driver does > not drop packets under the same load. > > I don't' see any description of the type of traffic load that is causing the > problem. That would help me to reproduce the issue. > > Keep in mind that dropped packets in and of itself is not a bug. It may mean > that the X520 and the ixgbe driver are more mature and have had more "tuning" > and thus work better under the type of traffic load you have on your network. > Thus it is important that we understand the type of traffic you're seeing on > your network so that we can work on making the XL710 and i40e driver > performance on par with the X520 and the ixgbe driver. > > One other thing. Below I notice this: > >> I tested this one: >> ethtool -C eth3 adaptive-rx off adaptive-tx off rx-usecs 2 tx-usecs 0 > > I believe that you would be better off using higher values. Really low > values mean the HW interrupt will fire more often - instead you should allow > the soft IRQ polling to keep processing packets. > > - Greg > >> -Original Message- >> From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag] >> Sent: Thursday, August 13, 2015 5:41 AM >> To: Rose, Gregory V; e1000-devel@lists.sourceforge.net >> Subject: Re: [E1000-devel] dropped rx with i40e >> >> 1.3.12-k from net-next devel does not help either ;-( >> >> Should we open an intel support ticket? We really need a solution. >> >> Stefan >> >> Am 12.08.2015 um 10:29 schrieb Stefan Priebe - Profihost AG: >>> Might this be a memory allocation problem? It happens only after some >>> hours running and when the whole memory is filled with linux fs cache. >>> >>> Is the i40e driver using kmalloc or vmalloc? >>> >>> Stefan >>> Am 11.08.2015 um 06:03 schrieb Stefan Priebe: One more thing to note. It mostly happens after around 8-24 hours and i could stop it again by rebooting the system/server. (can't prove it) Stefan Am 06.08.2015 um 22:59 schrieb Rose, Gregory V: > Thanks Stefan. I think for now you've given us enough data to go on > - I've got some research to do and then I'll get back to you. > > - Greg > >> -Original Message- >> From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag] >> Sent: Wednesday, August 05, 2015 11:32 PM >> To: Rose, Gregory V; e1000-devel@lists.sourceforge.net >> Subject: Re: [E1000-devel] dropped rx with i40e >> >> Am 06.08.2015 um 00:22 schrieb Rose, Gregory V: >>> Stefan, >>> >>> Could you please send me the output of 'ethtool' and 'ethtool -i' >>> for >> each i40e interface that is experiencing the dropped packets issue? >> >> These are around 100 cards. So i won't post the output for all of >> them. >> As they're all using the same driver and the same firmware - we >> updated all of them i hope it's ok to post the output only from one >> of them. >> >> # ethtool eth2 >> Settings for eth2: >> Supported ports: [ FIBRE ] >> Supported link modes: 1baseT/Full >> Supported pause frame use: Symmetric >> Supports auto-negotiation: No >> Advertised link modes: Not reported >> Advertised pause frame use: No >> Advertised auto-negotiation: No >> Speed: 1Mb/s >> Duplex: Full >> Port: Direct Attach Copper >> PHYAD: 0 >> Transceiver: external >> Auto-negotiation: off >> Supports Wake-on: g >> Wake-on: d >> Current message level: 0x000f (15) >> drv probe link timer >> Link detected: yes >> # ethtool -i eth2 >> driver: i40e >> version: 1.3.4-k >> firmware-version: f4.33.31377 a1.2 n4.42 e191b >> bus-info: :03:00.0 >> supports-statistics: yes >> supports-test: yes >> supports-eeprom-access: yes >> supports-register-dump: yes >> supports-priv-flags: yes >> >>>Also, the system log might help also - dmesg can get that.
Re: [E1000-devel] dropped rx with i40e
My apologies but I've been unable to get back to this issue. After reviewing the thread I don't see anything about steps to reproduce the problem. I understand that you're seeing dropped packets with the Xl710 with various versions of the i40e driver while the X520 with the ixgbe driver does not drop packets under the same load. I don't' see any description of the type of traffic load that is causing the problem. That would help me to reproduce the issue. Keep in mind that dropped packets in and of itself is not a bug. It may mean that the X520 and the ixgbe driver are more mature and have had more "tuning" and thus work better under the type of traffic load you have on your network. Thus it is important that we understand the type of traffic you're seeing on your network so that we can work on making the XL710 and i40e driver performance on par with the X520 and the ixgbe driver. One other thing. Below I notice this: > I tested this one: > ethtool -C eth3 adaptive-rx off adaptive-tx off rx-usecs 2 tx-usecs 0 I believe that you would be better off using higher values. Really low values mean the HW interrupt will fire more often - instead you should allow the soft IRQ polling to keep processing packets. - Greg > -Original Message- > From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag] > Sent: Thursday, August 13, 2015 5:41 AM > To: Rose, Gregory V; e1000-devel@lists.sourceforge.net > Subject: Re: [E1000-devel] dropped rx with i40e > > 1.3.12-k from net-next devel does not help either ;-( > > Should we open an intel support ticket? We really need a solution. > > Stefan > > Am 12.08.2015 um 10:29 schrieb Stefan Priebe - Profihost AG: > > Might this be a memory allocation problem? It happens only after some > > hours running and when the whole memory is filled with linux fs cache. > > > > Is the i40e driver using kmalloc or vmalloc? > > > > Stefan > > Am 11.08.2015 um 06:03 schrieb Stefan Priebe: > >> One more thing to note. It mostly happens after around 8-24 hours and > >> i could stop it again by rebooting the system/server. (can't prove > >> it) > >> > >> Stefan > >> Am 06.08.2015 um 22:59 schrieb Rose, Gregory V: > >>> Thanks Stefan. I think for now you've given us enough data to go on > >>> - I've got some research to do and then I'll get back to you. > >>> > >>> - Greg > >>> > -Original Message- > From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag] > Sent: Wednesday, August 05, 2015 11:32 PM > To: Rose, Gregory V; e1000-devel@lists.sourceforge.net > Subject: Re: [E1000-devel] dropped rx with i40e > > Am 06.08.2015 um 00:22 schrieb Rose, Gregory V: > > Stefan, > > > > Could you please send me the output of 'ethtool' and 'ethtool -i' > > for > each i40e interface that is experiencing the dropped packets issue? > > These are around 100 cards. So i won't post the output for all of > them. > As they're all using the same driver and the same firmware - we > updated all of them i hope it's ok to post the output only from one > of them. > > # ethtool eth2 > Settings for eth2: > Supported ports: [ FIBRE ] > Supported link modes: 1baseT/Full > Supported pause frame use: Symmetric > Supports auto-negotiation: No > Advertised link modes: Not reported > Advertised pause frame use: No > Advertised auto-negotiation: No > Speed: 1Mb/s > Duplex: Full > Port: Direct Attach Copper > PHYAD: 0 > Transceiver: external > Auto-negotiation: off > Supports Wake-on: g > Wake-on: d > Current message level: 0x000f (15) > drv probe link timer > Link detected: yes > # ethtool -i eth2 > driver: i40e > version: 1.3.4-k > firmware-version: f4.33.31377 a1.2 n4.42 e191b > bus-info: :03:00.0 > supports-statistics: yes > supports-test: yes > supports-eeprom-access: yes > supports-register-dump: yes > supports-priv-flags: yes > > > Also, the system log might help also - dmesg can get that. > > That'll > give me something to look at. > > As this one is pretty long. i pasted dmesg to pastebin: > http://pastebin.com/raw.php?i=7Tjp3eDT > > > By the way, have you tried using ethtool to turn adaptive RX and > > TX off > using ethtool to see if that has any impact on the dropped packets? > > I tested this one: > ethtool -C eth3 adaptive-rx off adaptive-tx off rx-usecs 2 tx-usecs > 0 > > but it has not helped. Still dropped rx packets. While a 2nd system > receiving the same load using ixgbe has no dropped packets. > > > That might be an easy test to run.
[E1000-devel] ❀ ¿Tu maceta favorita es la mejor de todas? ❀
Si no visualiza correctamente este E-Mail haga Click Aquí ( Link -> http://goto-2.net/track/click?u=web&p=36343234393a34303a33353a303a303a30&m=684073&s=4e3170d5bf0b7dab26573c9f1ed44baa ) | ( Link -> http://goto-2.net/track/click?u=forwardto&p=36343234393a34303a33353a303a303a30&m=684073&s=4e3170d5bf0b7dab26573c9f1ed44baa ) ( Link -> http://facebook.com/jardinetargentina ) ( Link -> https://apps.agorapulse.com/app/go/58602/54923 ) /* ¿Cual es la mejor maceta? ( Link -> https://apps.agorapulse.com/app/go/58602/54923 ) */ Tendremos 2 ganadores Tendremos 2 ganadores ( Link -> https://jardinet.com.ar/blog/como-cuidar-las-plantas-en-macetas-b15.html ) Ganador N° 1: la maceta más linda y mas votada por nuestros fans Ganador N° 2: la maceta mas linda votada por el equipo de Jardinet subí tu foto para ganar>> ( Link -> https://apps.agorapulse.com/app/go/58602/54923 ) Visitá nuestro Facebook para participar de más premios ( Link -> http://facebook.com/jardinetargentina ) ven a visitar nuestro BLOG EXPERTO BLOG EXPERTO ( Link -> https://jardinet.com.ar/blog.html ) Para más consejos para tu jardín Compartir ( Link -> http://goto-2.net/track/click?u=facebook&p=36343234393a34303a33353a303a303a30&m=684073&s=4e3170d5bf0b7dab26573c9f1ed44baa ) Twittear ( Link -> http://goto-2.net/track/click?u=twitter&p=36343234393a34303a33353a303a303a30&m=684073&s=4e3170d5bf0b7dab26573c9f1ed44baa ) +1 ( Link -> http://goto-2.net/track/click?u=googleplus&p=36343234393a34303a33353a303a303a30&m=684073&s=4e3170d5bf0b7dab26573c9f1ed44baa ) Compartir ( Link -> http://goto-2.net/track/click?u=linkedin&p=36343234393a34303a33353a303a303a30&m=684073&s=4e3170d5bf0b7dab26573c9f1ed44baa ) Reenvía a un amigo ( Link -> http://goto-2.net/track/click?u=forwardto&p=36343234393a34303a33353a303a303a30&m=684073&s=4e3170d5bf0b7dab26573c9f1ed44baa ) 18 cuotas sin interés con Mercado Pago Llevamos todo lo que necesite tu jardín a cualquier parte del país Los mejores descuentos y cientos de productos y ofertas Un nuevo concepto en productos para tu jardín 100% On-line 100% Seguro ( Link -> https://jardinet.com.ar/ ) /* JARDINET ( Link -> https://jardinet.com.ar/ ) */ JARDINET.COM.AR ( Link -> http://jardinet.com.ar ) 011 325 19686i...@jardinet.com.ar Agréganos a tu lista de contactos Información de Contacto ( Link -> http://goto-2.net/track/click?u=vcard&p=36343234393a34303a33353a303a303a30&m=684073&s=4e3170d5bf0b7dab26573c9f1ed44baa ) Para desuscribirse de nuestra lista haga ( Link -> http://goto-2.net/track/click?u=unsubscribe&p=36343234393a34303a33353a303a303a30&m=684073&s=4e3170d5bf0b7dab26573c9f1ed44baa ) -- ___ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
Re: [E1000-devel] dropped rx with i40e
1.3.12-k from net-next devel does not help either ;-( Should we open an intel support ticket? We really need a solution. Stefan Am 12.08.2015 um 10:29 schrieb Stefan Priebe - Profihost AG: > Might this be a memory allocation problem? It happens only after some > hours running and when the whole memory is filled with linux fs cache. > > Is the i40e driver using kmalloc or vmalloc? > > Stefan > Am 11.08.2015 um 06:03 schrieb Stefan Priebe: >> One more thing to note. It mostly happens after around 8-24 hours and i >> could stop it again by rebooting the system/server. (can't prove it) >> >> Stefan >> Am 06.08.2015 um 22:59 schrieb Rose, Gregory V: >>> Thanks Stefan. I think for now you've given us enough data to go on - >>> I've got some research to do and then I'll get back to you. >>> >>> - Greg >>> -Original Message- From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag] Sent: Wednesday, August 05, 2015 11:32 PM To: Rose, Gregory V; e1000-devel@lists.sourceforge.net Subject: Re: [E1000-devel] dropped rx with i40e Am 06.08.2015 um 00:22 schrieb Rose, Gregory V: > Stefan, > > Could you please send me the output of 'ethtool' and 'ethtool -i' for each i40e interface that is experiencing the dropped packets issue? These are around 100 cards. So i won't post the output for all of them. As they're all using the same driver and the same firmware - we updated all of them i hope it's ok to post the output only from one of them. # ethtool eth2 Settings for eth2: Supported ports: [ FIBRE ] Supported link modes: 1baseT/Full Supported pause frame use: Symmetric Supports auto-negotiation: No Advertised link modes: Not reported Advertised pause frame use: No Advertised auto-negotiation: No Speed: 1Mb/s Duplex: Full Port: Direct Attach Copper PHYAD: 0 Transceiver: external Auto-negotiation: off Supports Wake-on: g Wake-on: d Current message level: 0x000f (15) drv probe link timer Link detected: yes # ethtool -i eth2 driver: i40e version: 1.3.4-k firmware-version: f4.33.31377 a1.2 n4.42 e191b bus-info: :03:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: yes > Also, the system log might help also - dmesg can get that. That'll give me something to look at. As this one is pretty long. i pasted dmesg to pastebin: http://pastebin.com/raw.php?i=7Tjp3eDT > By the way, have you tried using ethtool to turn adaptive RX and TX off using ethtool to see if that has any impact on the dropped packets? I tested this one: ethtool -C eth3 adaptive-rx off adaptive-tx off rx-usecs 2 tx-usecs 0 but it has not helped. Still dropped rx packets. While a 2nd system receiving the same load using ixgbe has no dropped packets. > That might be an easy test to run. Thanks! Greets, Stefan > Thanks, > > - Greg > >> -Original Message- >> From: Stefan Priebe [mailto:s.pri...@profihost.ag] >> Sent: Wednesday, August 05, 2015 11:14 AM >> To: e1000-devel@lists.sourceforge.net >> Subject: Re: [E1000-devel] dropped rx with i40e >> >> >> Something i've noticed: >> ixgbe: >> Adaptive RX: off TX: off >> rx-usecs: 1 >> tx-usecs: 0 >> >> i40e: >> Adaptive RX: on TX: on >> rx-usecs: 62 >> tx-usecs: 122 >> >> Stefan >> >> Am 05.08.2015 um 09:02 schrieb Stefan Priebe - Profihost AG: >>> Hello list, >>> >>> we're using the intel X520 cards with the ixgbe driver since a long >>> time for our cloud infrastructure. We never had a problem with >>> dropped packets and everything was always fine. >>> >>> Since a year we started switching to the X710 cards as they're >>> better regarding their specs (lower power consumption, lower >>> latency, better price). >>> >>> We've around 100 X710 cards running now and we had a lot of trouble >>> with them. Back in 2014 there were a firmware bug, then there were >>> driver problems with bonding and so on. >>> >>> Now we have detected a new problem! We're seeing a lot of rx_dropped >>> packets on all X710 cards while all ixgbe based cards are working fine. >>> >>> I've tested the 1.2.48 driver als also the latest 1.3.4-k driver >>> from 4.2-rc5. >>> >>> Can anybody help? >>> >>> Greets, >>> Stefan >>> >> >> - >>
