Replying to my own thread as it was pointed out that I neglected to add some information.
OpenBSD 6.5 (GENERIC.MP) #7: Wed Nov 20 23:21:48 MST 2019 Native unbound (latest syspatch) Bge interfaces running on an LACP trunk with IPv4 and IPv6 addresses. Name Mtu Network Address Ipkts Ifail Opkts Ofail Colls bge0 1500 <Link> 44:a8:42:37:bb:b8 114258345 0 84203414 0 0 bge1 1500 <Link> 44:a8:42:37:bb:b8 57304058 0 84467834 0 0 bge2* 1500 <Link> 44:a8:42:37:bb:ba 0 0 0 0 0 bge3* 1500 <Link> 44:a8:42:37:bb:bb 0 0 0 0 0 enc0* 0 <Link> 0 0 0 0 0 trunk0 1500 <Link> 44:a8:42:37:bb:b8 171549799 0 168659644 12 0 trunk0 1500 fe80::%trunk0/64 fe80::601b:75c2:6b28:7276%trunk0 171549799 0 168659644 12 0 -Steve S. -----Original Message----- From: Steven Surdock Sent: Monday, December 2, 2019 1:34 PM To: misc@openbsd.org Subject: unbound network optimizations I'm running a pair of unbound resolvers and am attempting to optimize performance on them. This stemmed from noticing a couple of issues in the logs. Dec 2 11:26:52 ns1 unbound: [54230:5] error: recvfrom 26 failed: Host is down Dec 2 11:27:11 ns1 unbound: [54230:5] notice: sendto failed: Resource temporarily unavailable Dec 2 11:27:11 ns1 unbound: [54230:5] notice: remote address is 192.168.2.42 port 5088 I believed the first message is related to a dropped UDP request or subsequent response. 'netstat -p -u udp' shows "dropped due to full socket buffers". This was significantly reduced by increasing, net.inet.udp.recvspace=262144 net.inet.udp.sendspace=262144 Unfortunately, I'm still seeing a few UDP drops. Is there a danger in setting this is high? ns1$ netstat -s -p udp udp: 698584369 datagrams received 0 with incomplete header 0 with bad data length field 2508 with bad checksum 676259 with no checksum 86709458 input packets software-checksummed 706308843 output packets software-checksummed 641800 dropped due to no socket 0 broadcast/multicast datagrams dropped due to no socket 0 dropped due to missing IPsec protection 77324 dropped due to full socket buffers 697862737 delivered 706308952 datagrams output 698578008 missed PCB cache The second log message seems to stem from a dropped TCP request. There seems to be a significant number of these and I'm assuming they stem from "452447 SYN packets dropped due to queue or memory full" as the number of log message is in the same range as the number of dropped SYN packets. ns1$ netstat -s -p tcp tcp: 1856161 packets sent 359575 data packets (73608768 bytes) 27022 data packets (5076843 bytes) retransmitted 0 fast retransmitted packets 928517 ack-only packets (414664 delayed) 0 URG only packets 67 window probe packets 2217 window update packets 538808 control packets 271352 packets software-checksummed 2391157 packets received 739060 acks (for 71221089 bytes) 225691 duplicate acks 506 acks for unsent data 0 acks for old data 473441 packets (101441404 bytes) received in-sequence 111074 completely duplicate packets (75769595 bytes) 21701 old duplicate packets 3 packets with some duplicate data (112 bytes duplicated) 231945 out-of-order packets (88494422 bytes) 21 packets (0 bytes) of data after window 0 window probes 34417 window update packets 6771 packets received after close 52 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 0 discarded for missing IPsec protection 0 discarded due to memory shortage 231084 packets software-checksummed 0 bad/missing md5 checksums 0 good md5 checksums 213191 connection requests 156110 connection accepts 340472 connections established (including accepts) 369167 connections closed (including 14600 drops) 0 connections drained 14167 embryonic connections dropped 860911 segments updated rtt (of 838375 attempts) 40788 retransmit timeouts 3005 connections dropped by rexmit timeout 69 persist timeouts 6563 keepalive timeouts 0 keepalive probes sent 0 connections dropped by keepalive 12445 correct ACK header predictions 222843 correct data packet header predictions 828362 PCB cache misses 40214 dropped due to no socket 0 ECN connections accepted 0 ECE packets received 0 CWR packets received 9148 CE packets received 0 ECT packets sent 0 ECE packets sent 0 CWR packets sent cwr by fastrecovery: 385 cwr by timeout: 40788 cwr by ecn: 0 3161 bad connection attempts 452447 SYN packets dropped due to queue or memory full 161093 SYN cache entries added 0 hash collisions 156110 completed 0 aborted (no space to build PCB) 252 timed out 0 dropped due to overflow 0 dropped due to bucket overflow 4731 dropped due to RST 0 dropped due to ICMP unreachable 2809 SYN,ACKs retransmitted 913 duplicate SYNs received for entries already in the cache 0 SYNs dropped (no route or no space) 2 SYN cache seeds with new random 293 hash bucket array size in current SYN cache 0 entries in current SYN cache, limit is 10255 0 longest bucket length in current SYN cache, limit is 105 38907 uses of current SYN cache left 385 SACK recovery episodes 11 segment rexmits in SACK recovery episodes 3925 byte rexmits in SACK recovery episodes 461 SACK options received 52940 SACK options sent Based on the unbound docs I've tweaked some setting in unbound.conf. ns1$ more /var/unbound/etc/unbound.conf num-threads: 6 msg-cache-slabs: 4 rrset-cache-slabs: 4 infra-cache-slabs: 4 key-cache-slabs: 4 rrset-cache-size: 50m msg-cache-size: 128m outgoing-range: 8192 num-queries-per-thread: 4096 outgoing-num-tcp: 20 incoming-num-tcp: 20 Any insight on reducing the TCP "queue or memory full" issues would be appreciated. -Steve S.