Re: 'dropped due to full socket buffers' by SNMP
22.01.2021 20:26, Victor Gamov wrote:
What FreeBSD version do you use currently? Do you use IPv6 for UDP or
IPv4 only?
>>>
>>>
>>> FreeBSD 12.2-STABLE r366543 GENERIC amd64
>>>
>>> UDP-4 only
>>
>> In case of IPv4 UDP the counter "dropped due to full socket buffers"
>> is increased for incoming packets only. Therefore, the problem is in a code
>> processing incoming stream(s): either it locks for long time on something,
>> or it just has no enough raw CPU horsepower to deal with incoming stream.
>>
>> Look at "top -SHPI" CPU counters, if your CPU cores are loaded evenly;
>
> it's CPU E3-1270 v6 @ 3.80GHz with WCPU about 60% idle + 9%
> kernel{if_io_tqg_X) + 5% intr{swi1: netisr X). And many processes about 1%
> WCPU for multicast receive/resend (one for every multicast)
>
> Also "netstat -na -p udp" shows me zero or very small Recv-Q
>
>> if some of cores became overloaded sometimes. You should also draw per-cpu
>> load graphs.
>> (f.e. sysctl kern.cp_times)
>
> I have SNMP stats and it show me about 40% load
There is no standard MIBs to show per-core load. You get only an average via
standard SNMP MIBs
and average can be misleading and hide 100% load of some cores.
If you really have no overloaded cores, then this should be locking problem
that prevents processing code
to do its job timely.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: net.add_addr_allfibs=0 is ignored in loader.conf
On Sat, Jan 23, 2021 at 11:51:21AM +1100, Paul H wrote: > G'Day, > > I run to strange problem, I can't set net.add_addr_allfibs=0 via > loader.conf. It seems to be ignored. > After reboot net.add_addr_allfibs is set to 1. /etc/sysctl.conf -- Herbert ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
net.add_addr_allfibs=0 is ignored in loader.conf
G'Day, I run to strange problem, I can't set net.add_addr_allfibs=0 via loader.conf. It seems to be ignored. After reboot net.add_addr_allfibs is set to 1. # sysctl net.add_addr_allfibs net.add_addr_allfibs: 1 I can set it to 0 manually via sysctl. Did anyone seen similar behaviour on 12.2? It is nearly fresh install. # uname -a FreeBSD gateway 12.2-RELEASE-p1 FreeBSD 12.2-RELEASE-p1 GENERIC amd64 - # cat /boot/loader.conf comconsole_speed="115200" console="comconsole" amdtemp_load="YES" net.add_addr_allfibs=0 net.fibs=3 - # cat /etc/rc.conf hostname="gateway" ifconfig_igb0="DHCP" ifconfig_igb0_ipv6="inet6 accept_rtadv -no_radr" rtsold_enable="YES" ifconfig_igb1="inet 10.112.146.1/24 fib 1" local_unbound_enable="YES" sshd_enable="YES" ntpd_enable="YES" powerd_enable="YES" gateway_enable="YES" ipv6_gateway_enable="YES" -- Paul. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: 'dropped due to full socket buffers' by SNMP
On 22.01.2021 16:09, Eugene Grosbein wrote:
22.01.2021 19:28, Victor Gamov wrote:
On 22.01.2021 13:21, Eugene Grosbein wrote:
22.01.2021 17:02, Victor Gamov wrote:
No link overload: this host attached to network via 10G ix0, many
VLANs on this ix0 and some sender on every VLAN sends multicast
traffic to this host. Total input traffic about 1Gbit/s
What FreeBSD version do you use currently? Do you use IPv6 for UDP or
IPv4 only?
FreeBSD 12.2-STABLE r366543 GENERIC amd64
UDP-4 only
In case of IPv4 UDP the counter "dropped due to full socket buffers"
is increased for incoming packets only. Therefore, the problem is in a code
processing incoming stream(s): either it locks for long time on something,
or it just has no enough raw CPU horsepower to deal with incoming stream.
Look at "top -SHPI" CPU counters, if your CPU cores are loaded evenly;
it's CPU E3-1270 v6 @ 3.80GHz with WCPU about 60% idle + 9%
kernel{if_io_tqg_X) + 5% intr{swi1: netisr X). And many processes about
1% WCPU for multicast receive/resend (one for every multicast)
Also "netstat -na -p udp" shows me zero or very small Recv-Q
if some of cores became overloaded sometimes. You should also draw per-cpu load
graphs.
(f.e. sysctl kern.cp_times)
I have SNMP stats and it show me about 40% load
I have no visible problems with growing "dropped due to full socket
buffers" but I think it's not well when this counter increasing.
--
CU,
Victor Gamov
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: 'dropped due to full socket buffers' by SNMP
22.01.2021 19:28, Victor Gamov wrote: > On 22.01.2021 13:21, Eugene Grosbein wrote: >> 22.01.2021 17:02, Victor Gamov wrote: >> >>> No link overload: this host attached to network via 10G ix0, many >>> VLANs on this ix0 and some sender on every VLAN sends multicast >>> traffic to this host. Total input traffic about 1Gbit/s >> >> What FreeBSD version do you use currently? Do you use IPv6 for UDP or >> IPv4 only? > > > FreeBSD 12.2-STABLE r366543 GENERIC amd64 > > UDP-4 only In case of IPv4 UDP the counter "dropped due to full socket buffers" is increased for incoming packets only. Therefore, the problem is in a code processing incoming stream(s): either it locks for long time on something, or it just has no enough raw CPU horsepower to deal with incoming stream. Look at "top -SHPI" CPU counters, if your CPU cores are loaded evenly; if some of cores became overloaded sometimes. You should also draw per-cpu load graphs. (f.e. sysctl kern.cp_times) ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: 'dropped due to full socket buffers' by SNMP
On 22.01.2021 13:21, Eugene Grosbein wrote: 22.01.2021 17:02, Victor Gamov wrote: No link overload: this host attached to network via 10G ix0, many VLANs on this ix0 and some sender on every VLAN sends multicast traffic to this host. Total input traffic about 1Gbit/s What FreeBSD version do you use currently? Do you use IPv6 for UDP or IPv4 only? FreeBSD 12.2-STABLE r366543 GENERIC amd64 UDP-4 only -- CU, Victor Gamov ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: 'dropped due to full socket buffers' by SNMP
22.01.2021 17:02, Victor Gamov wrote: > No link overload: this host attached to network via 10G ix0, many VLANs on > this ix0 and some sender on every VLAN sends multicast traffic to this host. > Total input traffic about 1Gbit/s What FreeBSD version do you use currently? Do you use IPv6 for UDP or IPv4 only? ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: 'dropped due to full socket buffers' by SNMP
On 22.01.2021 12:52, Eugene Grosbein wrote: 22.01.2021 16:27, Victor Gamov wrote: I increase kern.ipc.maxsockbuf from 2097152 -> 2597152 -> 3145728 but netstat -sn -p udp | grep 'dropped due to full socket buffers' still show dropped packets. Then I increase net.inet.udp.recvspace 84160 -> 105200 but 'dropped due to full socket buffers' packets still here. Do I need to try to increase something else? Increase of various buffers may help against very short traffic bursts or limited time interface congestion. It won't help in case you have permanent (or nearly permanent) link overload. No link overload: this host attached to network via 10G ix0, many VLANs on this ix0 and some sender on every VLAN sends multicast traffic to this host. Total input traffic about 1Gbit/s -- CU, Victor Gamov ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: 'dropped due to full socket buffers' by SNMP
22.01.2021 16:27, Victor Gamov wrote: > I increase kern.ipc.maxsockbuf from 2097152 -> 2597152 -> 3145728 but > netstat -sn -p udp | grep 'dropped due to full socket buffers' still show > dropped packets. > > Then I increase net.inet.udp.recvspace 84160 -> 105200 but 'dropped due to > full socket buffers' packets still here. > > Do I need to try to increase something else? Increase of various buffers may help against very short traffic bursts or limited time interface congestion. It won't help in case you have permanent (or nearly permanent) link overload. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: 'dropped due to full socket buffers' by SNMP
Hi All On 05.01.2021 12:39, Victor Gamov wrote: Hi Eugene! Thanks for your responces. And Happy New Year for everyone! On 01.01.2021 03:19, Eugene Grosbein wrote: 30.12.2020 23:08, Victor Gamov wrote: As I understand hw.ix.flow_control=3 to allow flow-control for negotiation. Real PAUSE setting will be set during negotiation. At the moment of congestion. As I understand PAUSE feature negotiated during auto-negotiation process. If flow-control disabled on one side (switch for example) then other side (host) will not to use this feature too. Is it right? So where I can find active flow-control setting for host interface? Can't check for ix just now, but for em(4) there is sysctl dev.em.0.fc. It should be similar for ix. I have hw.ix.flow_control=3 (what does is it means ?) and dev.ix.0.fc=3 (and what does is it means?) maybe increase kern.ipc.maxsockbuf and then net.inet.udp.recvspace. Eugene, at first message you suppose Host-A (sender) "outgoing link for that UDP packets is congested" because this host shows non-zero "dropped due to full socket buffers". So is net.inet.udp.recvspace increasing on Host-B (mainly receiver) will be affected for this congestion? Can't tell in details without going deep into your setup :-) You can try it yourself and verify quickly. Or I need to try to increase both kern.ipc.maxsockbuf and net.inet.udp.recvspace on both hosts? Tune one that drops UDP. Also how I can check current sockbuf usage? netstat -xn Unfortunately it never shoes counters about UDP multicast traffic. I'll increase kern.ipc.maxsockbuf and net.inet.udp.recvspace at next week and write about results. I increase kern.ipc.maxsockbuf from 2097152 -> 2597152 -> 3145728 but netstat -sn -p udp | grep 'dropped due to full socket buffers' still show dropped packets. Then I increase net.inet.udp.recvspace 84160 -> 105200 but 'dropped due to full socket buffers' packets still here. Do I need to try to increase something else? -- CU, Victor Gamov ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
