Hi David,
On Wed, Oct 07, 2026 at 10:59:16AM +1000, David Gwynne wrote:
| > Yesterday (before the system slowed to a crawl and stopped
| > responding), this was at "266504 mbufs in use". I was surprised it
| > could use that many mbufs - is there no cap to that?
|
| yes, there is, which is why the system feels like it slowed down.
|
| the amount of memory that mbufs can use (rather than the number of
| mbufs) is limited by the kern.maxclusters sysctl. as the leak approaches
| that limit, you're effectively restricting the network stack to a small
| number of mbufs. instead of being able to run the network stack with
| dozens or hundreds of mbufs, it ends up only being able to push a
| handful through at a time, which is why it feels slow.
That makes a lot of sense. However, kern.maxclusters on my machine is
262144 whereas I had 266504 mbufs in use. Am I misunderstanding
things?
| > 2. Should an interface in UNKNOWN link state still receive packets?
| > 3. Should an interface in UNKNOWN link state still queue packets?
|
| this depends on the driver. what does link state mean for something like
| gif(4)?
Fair point...
| technically the interface link state is informational, ie, it's
| interesting for the operator of the system, but it's not used by the
| network stack. the stack only really cares if the interface is up or not
| (ie, ifconfig aq0 up), but it's up to the driver to do the right thing
| with a packet for the underlying transport, which is the hardware and
| ethernet link in aq(4)s.
Informational, but the operator has a hard time figuring this out.
`ifconfig` doesn't show it, there it's "UP" or not, and then it has
status "no carrier" or "active" or something. But I was surprised to
learn of this other link state that's not reflected in ifconfig
output.
| in my mind it's reasonable for a driver to queue packets when the link
| is down, the hardware should just complete tx processing with errors.
| aq hardware seems to be waiting for the link to appear to run the
| ring and make progress though.
I understand your point, but seems to me that there should be a limit
to the number of packets to queue. At this point in time I see:
08:14:49 aq0:0:txq:0 packets: 69748 packets
08:14:49 aq0:0:txq:1 packets: 3363 packets
08:14:49 aq0:0:txq:2 packets: 35566 packets
08:14:49 aq0:0:txq:3 packets: 2335 packets
08:14:49 aq0:0:txq:4 packets: 1754 packets
08:14:49 aq0:0:txq:5 packets: 4051 packets
08:14:49 aq0:0:txq:6 packets: 7631 packets
08:14:49 aq0:0:txq:7 packets: 1558 packets
total packets: 126006. That's more than 8x 2048.
free = tx->tx_cons + AQ_TXD_NUM - tx->tx_prod
Both tx_cons and tx_prod are in [0..2047], so if tx_prod has wrapped
but tx_cons hasn't yet (likely in the case where you're producing
packets but never consuming them because the link is down) suddenly
theres 2048 extra slots?
Then this happens later on:
if (used + AQ_TX_MAX_SEGMENTS + 1 >= free) {
ifq_set_oactive(ifq);
break;
}
But that condition isn't going to be true if free is more than 2048
because of the previous bit, right?
(caveat - i don't really know what i'm doing here, just trying to
follow the code but i'm not really good at this stuff)
| > 4. Should aq try to set its link state while bringing it up?
|
| yes. aq(4) should initialise the link state to DOWN since it's "load
| bearing" with LINK_STATE_IS_UP being used at the top of aq_start().
|
| my aq(4) is busy at the moment, so i can't test the diff below. i'm
| specifically interested in what the aq(4) link state looks like
| when a machine is booted from cold both with and without link.
|
| `systat if` is probably the easiest way to see it. with the current
| driver i'm guessing that you see "aq0 up" if the machine is
| booted without a working ethernet cable plugged in. after this diff
| it should be "up:D". if you boot with a working ethernet cable
| plugged in it should be "up:U".
You're right, it shows 'aq0 up'. Hopefully I'll be able
to test your diff later today. Will follow up when I do.
Thanks,
Paul
--
>++++++++[<++++++++++>-]<+++++++.>+++[<------>-]<.>+++[<+
+++++++++++>-]<.>++[<------------>-]<+.--------------.[-]
http://www.weirdnet.nl/
