In the case of multi-threaded OACTIVE, why not just do this:
upon queue completion:
* lock;
* do queue completion;
* if any frames were completed, clear queue busy;
* if (queue busy == clear && OACTIVE) clear OACTIVE;
* run queue completion;
* unlock.
upon queue transmit:
* lock;
* queue frame;
I'm testing something right now, what I've done is take all the OACTIVE
manipulation
and centralized it to the start routines (multiqueue or single), and I now
keep a queue
specific drained variable that gets set when you run to minimum
descriptors, and then
cleared in txeof. In start, particularly
Wait, but low resources which are stopping _what_ ?
The whole point behind OACTIVE is to say "oi, give me time to flush my
TX queue." I'll tell you when I'm ready.
So when you clear OACTIVE after a TX completion handler, you would
then call _start (or _start_locked) to reschedule some more frames
On 11/10/2011 3:39 AM, Adrian Chadd wrote:
There's no locking around the OACTIVE flag set/clear, right?
Is it possible that multiple TX threads are fiddling with OACTIVE and
then it's not being properly cleared and tx kicked?
Adrian
sorry! I forgot to cleanup the the last message ... here is t
On 11/10/2011 3:39 AM, Adrian Chadd wrote:
There's no locking around the OACTIVE flag set/clear, right?
Is it possible that multiple TX threads are fiddling with OACTIVE and
then it's not being properly cleared and tx kicked?
Adrian
If we check for OACTIVE periodically (for instance, in local_
BTW, the new delta on the driver is coming, I just ran into some issues
with the validation testing done in house and I've had to iron a few things
out.
I am going to implement Hooman's idea of a TX clean from local_timer,
that seems like a good idea.
The other thing I'm doing right now is reenab
Hmmm, that's an interesting point Adrian, I'll look at that more closely.
Jack
On Wed, Nov 9, 2011 at 4:09 PM, Adrian Chadd wrote:
> There's no locking around the OACTIVE flag set/clear, right?
> Is it possible that multiple TX threads are fiddling with OACTIVE and
> then it's not being proper
There's no locking around the OACTIVE flag set/clear, right?
Is it possible that multiple TX threads are fiddling with OACTIVE and
then it's not being properly cleared and tx kicked?
Adrian
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.o
On 11/8/2011 10:23 PM, Jason Wolfe wrote:
On Tue, Nov 8, 2011 at 10:21 AM, Hooman Fazaeli mailto:hoomanfaza...@gmail.com>> wrote:
I have allocated more time to the problem and guess I can explain what
your problem is.
With MSIX disabled, the driver uses fast interrupt handler (em_ir
On 11/8/2011 11:00 PM, Adrian Chadd wrote:
On 8 November 2011 09:21, Hooman Fazaeli wrote:
With MSIX enabled, the link task (em_handle_link) does _not_ triggers
_start when the link changes state from inactive to active (which it
should).
If if_snd quickly fills up during a temporary link loss
On 8 November 2011 09:21, Hooman Fazaeli wrote:
> With MSIX enabled, the link task (em_handle_link) does _not_ triggers
> _start when the link changes state from inactive to active (which it
> should).
> If if_snd quickly fills up during a temporary link loss, transmission is
> stopped forever an
On Tue, Nov 8, 2011 at 10:21 AM, Hooman Fazaeli wrote:
> I have allocated more time to the problem and guess I can explain what
> your problem is.
>
> With MSIX disabled, the driver uses fast interrupt handler (em_irq_fast)
> which calls rx/tx task and then checks for link status change. This
> i
On 11/7/2011 9:24 PM, Jason Wolfe wrote:
On Mon, Oct 31, 2011 at 1:13 AM, Hooman Fazaeli mailto:hoomanfaza...@gmail.com>> wrote:
Attached is a patch for if_em.c. It flushes interface queue when it is full
and link is not active. Please note that when this happens, drops are
increasing
On Mon, Oct 31, 2011 at 1:13 AM, Hooman Fazaeli wrote:
>
> Attached is a patch for if_em.c. It flushes interface queue when it is
> full
> and link is not active. Please note that when this happens, drops are
> increasing
> on interface and this will trigger your scripts as before. You need to
> c
On 10/31/2011 12:51 PM, Emil Muratov wrote:
On 31.10.2011 12:13, Hooman Fazaeli wrote:
Thanks for looking into this. I'd be happy to test any patch thrown my way, but keep in mind my issue is only tickled when MSI-X is enabled. My interfaces aren't bouncing, though it might be
possible so
On 31.10.2011 12:13, Hooman Fazaeli wrote:
Thanks for looking into this. I'd be happy to test any patch thrown
my way, but keep in mind my issue is only tickled when MSI-X is
enabled. My interfaces aren't bouncing, though it might be possible
some unique path in the MSI-X code is causing
On 10/31/2011 11:43 AM, Emil Muratov wrote:
You may try these settings and see if they help:
- hw.em.fc_setting=0 (in /boot/loader.conf)
- hw.em.rxd="4096" (in /boot/loader.conf)
- hw.em.txd="4096" (in /boot/loader.conf)
- Fix speed and duplex at both link sides. After doing that, confirm on
You may try these settings and see if they help:
- hw.em.fc_setting=0 (in /boot/loader.conf)
- hw.em.rxd="4096" (in /boot/loader.conf)
- hw.em.txd="4096" (in /boot/loader.conf)
- Fix speed and duplex at both link sides. After doing that, confirm
on the freebsd
box (with ifconfig) and the ot
On 10/31/2011 7:33 AM, Jason Wolfe wrote:
Thanks for looking into this. I'd be happy to test any patch thrown my way, but keep in mind my issue is only tickled when MSI-X is enabled. My interfaces aren't bouncing, though it might be
possible some unique path in the MSI-X code is causing a th
On Sun, Oct 30, 2011 at 9:03 PM, Jason Wolfe wrote:
...
>
>
> Jack is the delta your speaking to the 7.2.4 code? I did manage to get
> the code from Intel compiled with a couple minutes of work, but haven't
> loaded it up yet as I didn't see anything that caught my untrained eye in
> the diffs.
On Sun, Oct 30, 2011 at 1:57 AM, Hooman Fazaeli wrote:
>
> I finally managed to re-produce an affect similar to Jason's case. It
> may not be the exact same issue, but it is a serious problem and must
> be addressed.
>
> 1. Push out packet on em/igb with high rate.
> 2. Disconnect cable and wait f
On Sun, Oct 30, 2011 at 12:27:14PM +0330, Hooman Fazaeli wrote:
>
> I finally managed to re-produce an affect similar to Jason's case. It
> may not be the exact same issue, but it is a serious problem and must
> be addressed.
>
> 1. Push out packet on em/igb with high rate.
> 2. Disconnect cable
I have a delta to em that is coming, it has some changes that effect 82574
anyway and everyone
with issues should test it.
I'll give this issue some thought tomorrow.
Jack
On Sun, Oct 30, 2011 at 9:00 AM, Hooman Fazaeli wrote:
> On 10/30/2011 6:03 PM, Ryan Stone wrote:
>
>> On Sun, Oct 30, 201
On 10/30/2011 6:03 PM, Ryan Stone wrote:
On Sun, Oct 30, 2011 at 4:57 AM, Hooman Fazaeli wrote:
IMHO, this is not a driver issue and the real fix would be to change
IFQ_HANDOFF to call if_start when the queue is full.
I'm not sure that's the right approach. 99% of the time, calling
if_start w
On Sun, Oct 30, 2011 at 4:57 AM, Hooman Fazaeli wrote:
> IMHO, this is not a driver issue and the real fix would be to change
> IFQ_HANDOFF to call if_start when the queue is full.
I'm not sure that's the right approach. 99% of the time, calling
if_start when the queue is full will be a waste of
I finally managed to re-produce an affect similar to Jason's case. It
may not be the exact same issue, but it is a serious problem and must
be addressed.
1. Push out packet on em/igb with high rate.
2. Disconnect cable and wait for a few seconds. "netstat -ind" shows that
Drops are increasing
Hi,
On Fri, Oct 28, 2011 at 4:33 AM, Emil Muratov wrote:
>
Hi,
Can yan you pls post the output of these command _when_ the problem
happens?
uname -a
sysctl dev.em
netstat -ind
ifconfig
>>> Hi Hooman
>>>
>>> Here is what I've got when the script tr
Hi,
Can yan you pls post the output of these command _when_ the problem
happens?
uname -a
sysctl dev.em
netstat -ind
ifconfig
Hi Hooman
Here is what I've got when the script triggered just in time when the
interface was locked
11.10.26-23:39:10 ... interface em0 is down...
FreeBSD ion.ho
Hi,
On Thu, Oct 27, 2011 at 2:29 AM, Emil Muratov wrote:
>
>
>> Hi,
>>
>> Can yan you pls post the output of these command _when_ the problem
>> happens?
>>
>> uname -a
>> sysctl dev.em
>> netstat -ind
>> ifconfig
>>
>
> Hi Hooman
>
> Here is what I've got when the script triggered just in time w
Hi Hooman
Here is what I've got when the script triggered just in time when the
interface was locked
11.10.26-23:39:10 ... interface em0 is down...
FreeBSD ion.hotplug.ru 8.2-STABLE FreeBSD 8.2-STABLE #0: Thu Oct 20
20:20:25 MSD 2011 r...@epia.home
.lan:/usr/obj/usr/src/sys/ION6debu
Hello, Mike.
You wrote 7 октября 2011 г., 19:06:34:
> This sure sounds like the issue I was seeing with the 7.1.9 driver...
> However, it has been fixed for me by going to 7.2.3, which is in
> RELENG_8. Is it possible you have a couple of issues going on since you
> are using lagg as well
On 10/27/2011 9:59 AM, Emil Muratov wrote:
Hi Hooman
Here is what I've got when the script triggered just in time when the interface
was locked
11.10.26-23:39:10 ... interface em0 is down...
FreeBSD ion.hotplug.ru 8.2-STABLE FreeBSD 8.2-STABLE #0: Thu Oct 20 20:20:25
MSD 2011 r...@epia
Hi All,
I've got almost the same problem with intel 82574L based nic. My platform is
nvidia ion running Atom 1.6 and nic is an external PCI-express adapter.
Unlike Jason's case mine is always stuck in receiving traffic, it's Ierrs
increasing while Ipkts not. Thanks to Jason's script I can see th
Hi,
Can yan you pls post the output of these command _when_ the problem
happens?
uname -a
sysctl dev.em
netstat -ind
ifconfig
Hi Hooman
Here is what I've got when the script triggered just in time when the
interface was locked
11.10.26-23:39:10 ... interface em0 is down...
FreeBSD
On Wed, Oct 26, 2011 at 6:16 AM, Mike Tancsa wrote:
> On 10/26/2011 7:33 AM, Jason Wolfe wrote:
> > Hooman,
> >
> > I have run with dev.em.X.flow_control=0, which should have the same
> result
> > as hw.em.fc_setting=0, and net.inet.tcp.tso is also 0. I'm not sure the
> > remaining options would
Hooman,
I've added ifconfig to my collection script (as well as sysctl hw.em) that
will run at the time of the next wedge, the ifconfig lines in my email were
just to show the before and after as I've also modified the running options
on emX in this test group as you proposed.
Thank you for the d
Hi,
Can yan you pls post the output of these command _when_ the problem happens?
uname -a
sysctl dev.em
netstat -ind
ifconfig
I've got almost the same problem with intel 82574L based nic. My platform is nvidia ion running Atom 1.6 and nic is an external PCI-express adapter. Unlike Jason's case
Dear Jason
I was actually interested in "status:" and "flags=" line from ifconfig output
_at_ the time problem happens.
From your previous post, it is clear that when the driver stops transmitting,
output errors are zero and "Drops" increasing. This happens when the stack uses
IFQ_ENQUEUE to pu
On 26 October 2011 21:47, Emil Muratov wrote:
> Hi All,
> I've got almost the same problem with intel 82574L based nic. My platform is
> nvidia ion running Atom 1.6 and nic is an external PCI-express adapter.
> Unlike Jason's case mine is always stuck in receiving traffic, it's Ierrs
> increasing
Hooman,
I have run with dev.em.X.flow_control=0, which should have the same result
as hw.em.fc_setting=0, and net.inet.tcp.tso is also 0. I'm not sure the
remaining options would be able to produce the scenario I'm seeing, but I'm
open to giving it a try with no options on the interfaces. I'v
On 10/26/2011 7:33 AM, Jason Wolfe wrote:
> Hooman,
>
> I have run with dev.em.X.flow_control=0, which should have the same result
> as hw.em.fc_setting=0, and net.inet.tcp.tso is also 0. I'm not sure the
> remaining options would be able to produce the scenario I'm seeing, but I'm
> open to givi
Hooman,
I have run with dev.em.X.flow_control=0, which should have the same result
as hw.em.fc_setting=0, and net.inet.tcp.tso is also 0. I'm not sure the
remaining options would be able to produce the scenario I'm seeing, but I'm
open to giving it a try with no options on the interfaces. I've a
Hi Jason
Have you tried:
hw.em.fc_setting="0" (in loader.conf)
ifconfig emX -tso -lro -rxcsum -txcsum -vlanhwtag -wol
with MSIX and no multiqueue.
Advanced features has always been a source of problem.
It is worth a try and help to narrow down possibilities.
It would also be helpful if you pr
On Fri, Oct 7, 2011 at 2:14 PM, Jason Wolfe wrote:
> Bumping rx/tx descriptors to 2048 was actually for performance reasons and
> not to try to get around the issue. I did some fairly in depth testing and
> found under heavy load it performed the best with those settings.
>
> As mentioned on the
On Fri, Oct 7, 2011 at 2:14 PM, Jason Wolfe wrote:
> Bumping rx/tx descriptors to 2048 was actually for performance reasons and
> not to try to get around the issue. I did some fairly in depth testing and
> found under heavy load it performed the best with those settings.
>
> As mentioned on the
On Fri, Oct 7, 2011 at 12:55 PM, Arnaud Lacombe wrote:
> Hi,
>
> On Fri, Oct 7, 2011 at 3:24 PM, Mike Tancsa wrote:
> > When you disable MSI-X, you mean via hw.pci.enable_msix=0 across the
> > board, or you disable multi-queue for the NIC, so it uses just one
> > interrupt, rather than separate
On Fri, Oct 7, 2011 at 12:55 PM, Arnaud Lacombe wrote:
> Hi,
>
> On Fri, Oct 7, 2011 at 3:24 PM, Mike Tancsa wrote:
>
> When you disable MSI-X, you mean via hw.pci.enable_msix=0 across the
> > board, or you disable multi-queue for the NIC, so it uses just one
> > interrupt, rather than separate
On Fri, Oct 7, 2011 at 12:24 PM, Arnaud Lacombe wrote:
> Hi,
>
> On Fri, Oct 7, 2011 at 2:57 PM, Jason Wolfe wrote:
> > Jack,
> >
> > Entirely possible there are multiple moving pieces here, the only bit I
> know
> > for certain is it's related to the different operation when running with
> MSI
On 10/7/2011 4:26 PM, Arnaud Lacombe wrote:
>> em0: port 0x4040-0x405f mem
>> 0xb450-0xb451,0xb4525000-0xb4525fff irq 16 at device 25.0 on pci0
>> em0: Using an MSI interrupt
>> em0: [FILTER]
>> em0: Ethernet address: 00:15:17:ed:68:a5
>>
>> em1: port 0x2000-0x201f mem
>> 0xb410-0xb41
Hi,
On Fri, Oct 7, 2011 at 4:12 PM, Mike Tancsa wrote:
> On 10/7/2011 3:55 PM, Arnaud Lacombe wrote:
>>> When you disable MSI-X, you mean via hw.pci.enable_msix=0 across the
>>> board, or you disable multi-queue for the NIC, so it uses just one
>>> interrupt, rather than separate ones for xmit an
On 10/7/2011 3:55 PM, Arnaud Lacombe wrote:
>> When you disable MSI-X, you mean via hw.pci.enable_msix=0 across the
>> board, or you disable multi-queue for the NIC, so it uses just one
>> interrupt, rather than separate ones for xmit and recv ?
>>
> em(4)'s multiqueue is misleading. By default, wi
Hi,
On Fri, Oct 7, 2011 at 3:24 PM, Mike Tancsa wrote:
> On 10/7/2011 2:59 PM, Jason Wolfe wrote:
>> Mike,
>>
>> I had a large pool of servers running 7.2.3 with MSI-X enabled during my
>> testing, but it didn't resolve the issue. I just pulled back the
>> sys/dev/e1000 directory from 8-STABLE an
Hi,
On Fri, Oct 7, 2011 at 2:57 PM, Jason Wolfe wrote:
> Jack,
>
> Entirely possible there are multiple moving pieces here, the only bit I know
> for certain is it's related to the different operation when running with MSI
> vs MSI-X. Here is also my loader.conf for reference. I'm currently runni
On 10/7/2011 2:59 PM, Jason Wolfe wrote:
> Mike,
>
> I had a large pool of servers running 7.2.3 with MSI-X enabled during my
> testing, but it didn't resolve the issue. I just pulled back the
> sys/dev/e1000 directory from 8-STABLE and ran it on 8-RELEASE-p2 though, so
> if there were changes mad
Mike,
I had a large pool of servers running 7.2.3 with MSI-X enabled during my
testing, but it didn't resolve the issue. I just pulled back the
sys/dev/e1000 directory from 8-STABLE and ran it on 8-RELEASE-p2 though, so
if there were changes made outside of the actual driver code that helped I
may
Jack,
Entirely possible there are multiple moving pieces here, the only bit I know
for certain is it's related to the different operation when running with MSI
vs MSI-X. Here is also my loader.conf for reference. I'm currently running
the modular congestion control stuff with cubic in use, but the
On 10/6/2011 7:15 PM, Jason Wolfe wrote:
> I'm seeing the interface wedge on a good number of systems with Intel 82574L
> chips under FBSD8.2 _only when MSI-X is enabled_, running either 7.1.9 from
> 8.2-RELEASE or 7.2.3 from 8.2-STABLE. I have em0 and em1 in a lagg, but
> only one side would fail
There are plenty other users using the 574 with MSI-X without issue, so its
likely
more complicated than just that 1 variable. The issue discussed was like
last year,
it has to do with getting stuck in low power state, and my driver has code
that addresses
that. There was rigorous testing done at t
I'm seeing the interface wedge on a good number of systems with Intel 82574L
chips under FBSD8.2 _only when MSI-X is enabled_, running either 7.1.9 from
8.2-RELEASE or 7.2.3 from 8.2-STABLE. I have em0 and em1 in a lagg, but
only one side would fail, and a few systems that didn't have a lagg also
59 matches
Mail list logo
