On 7 February 2011 16:53, Jeff Ross <jr...@openvistas.net> wrote:
> Greetings,
>
> I found this explanation for the following OpenVPN error
> "write UDPv4: No buffer space available (code=55)"
>
> "No buffer space available is reported by the kernel network layer when
> thekernel's transmit buffer is full - i.e. the network connection is
running
> at full speed, and there's no space for more. This means that something is
> filling it, such as other OpenVPN requests, or simultaneous traffic. When
> this happens, the UDP packet is dropped, and the traffic is shaped to the
> size of the pipe it is travelling on."
>
> http://readlist.com/lists/lists.sourceforge.net/openvpn-users/0/4136.html
>
> I also got a very similar no buffer space available message while running
> traceroute earlier today.
>
> However, unlike the OP in the thread above, this is kicking my one VPN
user
> off every couple of minutes.
>
> I have plenty of ram (4GB--full dmesg below) in these identical and CARPed
> firewalls--is there a sysctl I can bump to give the kernel more transmit
> space?
>
> Thanks,
>
> Jeff Ross
>
> OpenBSD 4.9-beta (GENERIC.MP) #764: Thu Jan 27 13:19:27 MST 2011
> t...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP
> cpu0: Intel(R) Xeon(TM) CPU 3.20GHz ("GenuineIntel" 686-class) 3.21 GHz
> cpu0:
>
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS
> ,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR
> real mem = 3757522944 (3583MB)
> avail mem = 3685912576 (3515MB)
> mainbus0 at root
> bios0 at mainbus0: AT/286+ BIOS, date 03/29/05, BIOS32 rev. 0 @ 0xf0010,
> SMBIOS rev. 2.3 @ 0xfa380 (61 entries)
> bios0: vendor American Megatrends Inc. version "080010" date 03/29/2005
> bios0: SiMech R200
> acpi0 at bios0: rev 2
> acpi0: sleep states S0 S1 S4 S5
> acpi0: tables DSDT FACP APIC OEMB
> acpi0: wakeup devices PXHA(S4) PXHB(S4) EPA0(S4) EPA1(S4) EPB0(S4) EPB1(S4)
> EPC0(S4) P0P1(S4) MC97(S4) USB1(S1) U
> SB2(S1) EUSB(S4) PS2K(S4) PS2M(S4) P0PC(S4) SLPB(S4)
> acpitimer0 at acpi0: 3579545 Hz, 24 bits
> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
> cpu0 at mainbus0: apid 0 (boot processor)
> cpu0: apic clock running at 200MHz
> cpu1 at mainbus0: apid 6 (application processor)
> cpu1: Intel(R) Xeon(TM) CPU 3.20GHz ("GenuineIntel" 686-class) 3.21 GHz
> cpu1:
>
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS
> ,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR
> cpu2 at mainbus0: apid 1 (application processor)
> cpu2: Intel(R) Xeon(TM) CPU 3.20GHz ("GenuineIntel" 686-class) 3.21 GHz
> cpu2:
>
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS
> ,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR
> cpu3 at mainbus0: apid 7 (application processor)
> cpu3: Intel(R) Xeon(TM) CPU 3.20GHz ("GenuineIntel" 686-class) 3.21 GHz
> cpu3:
>
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS
> ,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR
> ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
> ioapic1 at mainbus0: apid 11 pa 0xfec10000, version 20, 24 pins
> ioapic1: misconfigured as apic 9, remapped to apid 11
> ioapic2 at mainbus0: apid 9 pa 0xfec80000, version 20, 24 pins
> ioapic2: misconfigured as apic 10, remapped to apid 9
> ioapic3 at mainbus0: apid 10 pa 0xfec80400, version 20, 24 pins
> ioapic3: misconfigured as apic 11, remapped to apid 10
> acpiprt0 at acpi0: bus 0 (PCI0)
> acpiprt1 at acpi0: bus 1 (EPA0)
> acpiprt2 at acpi0: bus 3 (PXHA)
> acpiprt3 at acpi0: bus 2 (PXHB)
> acpiprt4 at acpi0: bus 4 (EPA1)
> acpiprt5 at acpi0: bus 6 (P0P1)
> acpiprt6 at acpi0: bus 5 (P0PC)
> acpicpu0 at acpi0
> acpicpu1 at acpi0
> acpicpu2 at acpi0
> acpicpu3 at acpi0
> acpibtn0 at acpi0: PWRB
> acpibtn1 at acpi0: SLPB
> bios0: ROM list: 0xc0000/0x8000 0xc8000/0x800 0xc8800/0x1000 0xc9800/0x1000
> ipmi at mainbus0 not configured
> pci0 at mainbus0 bus 0: configuration mode 1 (bios)
> pchb0 at pci0 dev 0 function 0 "Intel E7320 Host" rev 0x0c
> ppb0 at pci0 dev 2 function 0 "Intel E7520 PCIE" rev 0x0c
> pci1 at ppb0 bus 1
> ppb1 at pci1 dev 0 function 0 "Intel PCIE-PCIE" rev 0x09
> pci2 at ppb1 bus 3
> mpi0 at pci2 dev 5 function 0 "Symbios Logic 53c1030" rev 0xc1: apic 9 int
2
> (irq 6)
> scsibus0 at mpi0: 16 targets, initiator 7
> ppb2 at pci1 dev 0 function 2 "Intel PCIE-PCIE" rev 0x09
> pci3 at ppb2 bus 2
> em0 at pci3 dev 1 function 0 "Intel PRO/1000MT (82546GB)" rev 0x03: apic 10
> int 0 (irq 10), address 00:1b:21:15:6
> a:82
> em1 at pci3 dev 1 function 1 "Intel PRO/1000MT (82546GB)" rev 0x03: apic 10
> int 1 (irq 11), address 00:1b:21:15:6
> a:83
> ppb3 at pci0 dev 3 function 0 "Intel E7520 PCIE" rev 0x0c
> pci4 at ppb3 bus 4
> ppb4 at pci0 dev 28 function 0 "Intel 6300ESB PCIX" rev 0x02
> pci5 at ppb4 bus 5
> em2 at pci5 dev 1 function 0 "Intel PRO/1000MT (82541GI)" rev 0x05: apic 11
> int 2 (irq 9), address 00:30:48:87:2e
> :b0
> em3 at pci5 dev 2 function 0 "Intel PRO/1000MT (82541GI)" rev 0x05: apic 11
> int 3 (irq 9), address 00:30:48:87:2e
> :b1
> uhci0 at pci0 dev 29 function 0 "Intel 6300ESB USB" rev 0x02: apic 8 int 16
> (irq 10)
> uhci1 at pci0 dev 29 function 1 "Intel 6300ESB USB" rev 0x02: apic 8 int 19
> (irq 7)
> "Intel 6300ESB WDT" rev 0x02 at pci0 dev 29 function 4 not configured
> "Intel 6300ESB APIC" rev 0x02 at pci0 dev 29 function 5 not configured
> ehci0 at pci0 dev 29 function 7 "Intel 6300ESB USB" rev 0x02: apic 8 int 23
> (irq 5)
> usb0 at ehci0: USB revision 2.0
> uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
> ppb5 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0x0a
> pci6 at ppb5 bus 6
> vga1 at pci6 dev 2 function 0 "ATI Rage XL" rev 0x27
> wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
> wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
> ichpcib0 at pci0 dev 31 function 0 "Intel 6300ESB LPC" rev 0x02
> pciide0 at pci0 dev 31 function 1 "Intel 6300ESB IDE" rev 0x02: DMA,
channel
> 0 configured to compatibility, chann
> el 1 configured to compatibility
> pciide0: channel 0 disabled (no drives)
> pciide0: channel 1 disabled (no drives)
> ichiic0 at pci0 dev 31 function 3 "Intel 6300ESB SMBus" rev 0x02: apic 8
int
> 17 (irq 0)
> iic0 at ichiic0
> lm1 at iic0 addr 0x2c: W83627HF
> lm2 at iic0 addr 0x2f: W83792D rev D
> spdmem0 at iic0 addr 0x51: 1GB DDR SDRAM registered ECC PC2700CL2.5
> spdmem1 at iic0 addr 0x53: 1GB DDR SDRAM registered ECC PC2700CL2.5
> spdmem2 at iic0 addr 0x55: 1GB DDR SDRAM registered ECC PC2700CL2.5
> spdmem3 at iic0 addr 0x56: 1GB DDR SDRAM registered ECC PC2700CL2.5
> usb1 at uhci0: USB revision 1.0
> uhub1 at usb1 "Intel UHCI root hub" rev 1.00/1.00 addr 1
> usb2 at uhci1: USB revision 1.0
> uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
> isa0 at ichpcib0
> isadma0 at isa0
> com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
> com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
> pckbc0 at isa0 port 0x60/5
> pckbd0 at pckbc0 (kbd slot)
> pckbc0: using irq 1 for kbd slot
> wskbd0 at pckbd0: console keyboard, using wsdisplay0
> pms0 at pckbc0 (aux slot)
> pckbc0: using irq 12 for aux slot
> wsmouse0 at pms0 mux 0
> pcppi0 at isa0 port 0x61
> spkr0 at pcppi0
> wbsio0 at isa0 port 0x2e/2: W83627HF rev 0x41
> lm3 at wbsio0 port 0x290/8: W83627HF
> lm1 detached
> npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
> mtrr: Pentium Pro MTRR support
> umass0 at uhub0 port 3 configuration 1 interface 0 "Imation Nano Pro" rev
> 2.00/1.00 addr 2
> umass0: using SCSI over Bulk-Only
> scsibus1 at umass0: 2 targets, initiator 0
> sd0 at scsibus1 targ 1 lun 0: <Imation, Nano Pro, PMAP> SCSI0 0/direct
> removable
> sd0: 3822MB, 512 bytes/sec, 7827456 sec total
> umass1 at uhub0 port 4 configuration 1 interface 0 "Imation Nano Pro" rev
> 2.00/1.00 addr 3
> umass1: using SCSI over Bulk-Only
> scsibus2 at umass1: 2 targets, initiator 0
> sd1 at scsibus2 targ 1 lun 0: <Imation, Nano Pro, PMAP> SCSI0 0/direct
> removable
> sd1: 3822MB, 512 bytes/sec, 7827456 sec total
> vscsi0 at root
> scsibus3 at vscsi0: 256 targets
> softraid0 at root
> root on sd0a swap on sd0b dump on sd0b
>
>
code 55 is ENOBUFS.
This is weird since the default UDP send buffer is 9kb:
u_int udp_sendspace = 9216; /* really max datagram size */
Coincidently I was looking this afternoon for what seems to be a bug,
it seems poll(2) and friends return a writable state for the socket
disrespecting SO_SNDLOWAT in UDP.
Although I don't think it is related to your problem, and I don't
believe OpenVPN can fill up 9k before something being sent to the
ethernet device.
You're probably hitting this in udp_usrreq.c:
/*
* Calculate data length and get a mbuf
* for UDP and IP headers.
*/
M_PREPEND(m, sizeof(struct udpiphdr), M_DONTWAIT);
if (m == NULL) {
error = ENOBUFS;
goto bail;
}
How is the load on the system ? Can you give us some netstat output ?
