Elantech Clickpad (pms0) stopped working on current
Synopsis: Elantech Clickpad (pms0) stopped working on current Category: kernel Environment: System : OpenBSD 5.7 Details : OpenBSD 5.7-current (GENERIC.MP) #1046: Fri Jun 5 13:25:01 MDT 2015 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP Architecture: OpenBSD.amd64 Machine : amd64 Description: When I try to use the touchpad on my noteboot the below messages are shown in xconsole and dmesg. The mouse pointer does not react anymore to movements on the touchpad. Affected devic: pms0: Elantech Clickpad, version 4, firmware 0x575f03 I first noticed this with the snapshot kernel from May 31. With the kernel from June 5 it's the same. The touchpad was working fine with the snapshot from May 23. pms0: not in sync yet, discard input (state 0) pms0: not in sync yet, discard input (state 0) pms0: unknown packet type 0x1 pms0: unknown packet type 0x1 pms0: unknown packet type 0x1 pms0: unknown packet type 0x1 pms0: unknown packet type 0x1 pms0: unknown packet type 0x1 pms0: not in sync yet, discard input (state 0) pms0: unknown packet type 0x3 pms0: unknown packet type 0x3 pms0: unknown packet type 0x3 pms0: unknown packet type 0x13 pms0: unknown packet type 0x3 pms0: unknown packet type 0x3 pms0: unknown packet type 0x3 pms0: unknown packet type 0x13 pms0: unknown packet type 0x3 pms0: not in sync yet, discard input (state 0) pms0: not in sync yet, discard input (state 0) pms0: not in sync yet, discard input (state 0) pms0: not in sync yet, discard input (state 0) pms0: not in sync yet, discard input (state 0) pms0: unknown packet type 0x18 pms0: unknown packet type 0x18 pms0: unknown packet type 0x18 pms0: unknown packet type 0x8 pms0: unknown packet type 0x18 pms0: unknown packet type 0x18 pms0: unknown packet type 0x8 pms0: unknown packet type 0x18 pms0: not in sync yet, discard input (state 0) pms0: not in sync yet, discard input (state 0) pms0: unknown packet type 0x5 pms0: unknown packet type 0x5 pms0: unknown packet type 0x5 pms0: unknown packet type 0x5 pms0: unknown packet type 0x5 pms0: unknown packet type 0x5 pms0: unknown packet type 0x5 pms0: unknown packet type 0x5 pms0: not in sync yet, discard input (state 0) pms0: not in sync yet, discard input (state 0) pms0: unknown packet type 0x15 pms0: unknown packet type 0x15 pms0: unknown packet type 0x1 pms0: unknown packet type 0x1 pms0: unknown packet type 0x1 pms0: not in sync yet, discard input (state 0) pms0: not in sync yet, discard input (state 0) pms0: not in sync yet, discard input (state 0) How-To-Repeat: Use an Elantech Clickpad, version 4, firmware 0x575f03 with snapshot from June 5. Fix: Workaround: use snapshot from May 23. dmesg: OpenBSD 5.7-current (GENERIC.MP) #1046: Fri Jun 5 13:25:01 MDT 2015 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 3881746432 (3701MB) avail mem = 3760226304 (3586MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xe0970 (63 entries) bios0: vendor Phoenix Technologies Ltd. version P00ACX date 04/26/2013 bios0: SAMSUNG ELECTRONICS CO., LTD. 900X3F acpi0 at bios0: rev 2 acpi0: sleep states S0 S3 S4 S5 acpi0: tables DSDT FACP SLIC SSDT ASF! HPET APIC MCFG FPDT SSDT SSDT UEFI MSDM UEFI UEFI acpi0: wakeup devices P0P1(S4) GLAN(S4) XHC_(S4) HDEF(S4) PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4) PXSX(S4) RP06(S4) PXSX(S4) RP07(S4) [...] acpitimer0 at acpi0: 3579545 Hz, 24 bits acpihpet0 at acpi0: 14318179 Hz acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Core(TM) i5-3337U CPU @ 1.80GHz, 1696.39 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT cpu0: 256KB 64b/line 8-way L2 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges cpu0: apic clock running at 99MHz cpu0: mwait min=64, max=64, C-substates=0.2.1.1.2, IBE cpu1 at mainbus0: apid 1 (application processor) cpu1: Intel(R) Core(TM) i5-3337U CPU @ 1.80GHz, 1696.15 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT cpu1: 256KB 64b/line 8-way L2 cache cpu1: smt 1, core 0, package 0 cpu2 at mainbus0: apid 2 (application processor) cpu2: Intel(R) Core(TM) i5-3337U CPU @ 1.80GHz, 1696.15 MHz cpu2:
Re: Elantech Clickpad (pms0) stopped working on current
On Sat, Jun 06, 2015 at 10:47:19AM +0200, Stefan Sperling wrote: On Sat, Jun 06, 2015 at 10:21:59AM +0200, Remi Locherer wrote: Synopsis: Elantech Clickpad (pms0) stopped working on current Category: kernel Environment: System : OpenBSD 5.7 Details : OpenBSD 5.7-current (GENERIC.MP) #1046: Fri Jun 5 13:25:01 MDT 2015 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP Architecture: OpenBSD.amd64 Machine : amd64 Description: When I try to use the touchpad on my noteboot the below messages are shown in xconsole and dmesg. The mouse pointer does not react anymore to movements on the touchpad. Affected devic: pms0: Elantech Clickpad, version 4, firmware 0x575f03 I first noticed this with the snapshot kernel from May 31. With the kernel from June 5 it's the same. The touchpad was working fine with the snapshot from May 23. This was probably caused by pms.c commit r1.61 which enabled touchpad support for newer elantech devices using firmware versions 0xX8. Your device (firmware version 0xX7) was not matched before but matches now because of the way the new matching check was written. Can you try this diff to see if it restores previous behaviour for your device? After applying this patch I can use the touchpad again. Thanks! Index: pms.c === RCS file: /cvs/src/sys/dev/pckbc/pms.c,v retrieving revision 1.61 diff -u -p -r1.61 pms.c --- pms.c 25 May 2015 15:04:26 - 1.61 +++ pms.c 6 Jun 2015 08:37:41 - @@ -1840,7 +1840,8 @@ elantech_get_hwinfo_v4(struct pms_softc if (synaptics_query(sc, ELANTECH_QUE_FW_VER, fw_version)) return (-1); - if (((fw_version 0x0f) 16) 6) + if (((fw_version 0x0f) 16) != 6 + (fw_version 0x0f) 16 != 8) return (-1); elantech-fw_version = fw_version;
system freeze: acpicpu setperf failed to alter frequency
Synopsis: system freeze: acpicpu setperf failed to alter frequency Category: kernel Environment: System : OpenBSD 5.7 Details : OpenBSD 5.7 (GENERIC) #0: Thu Apr 30 22:01:01 CEST 2015 r...@stable-57-amd64.mtier.org:/binpatchng/work-binpatch57-amd64/src/sys/arch/amd64/compile/GENERIC Architecture: OpenBSD.amd64 Machine : amd64 Description: After executing the command sysctl hw the server freezes without dropping me to ddb. With OpenBSD 5.6 sysctl hw was working fine on this hardware. It happens with 5.7 and -current mp kernels. sp kernels work as expected. Output from the serial console (with different kernel versions): OpenBSD 5.7 (GENERIC.MP) #0: Thu Apr 30 21:56:27 CEST 2015 r...@stable-57-amd64.mtier.org:/binpatchng/work-binpatch57-amd64/src/sys/arc h/amd64/compile/GENERIC.MP root@hurricane# sysctl hw hw.machine=amd64CPU3: acpicpu setperf failed to alter frequency hw.model=Intel -- root@hurricane# sysctl kern.version kern.version=OpenBSD 5.7 (GENERIC.MP) #881: Sun Mar 8 11:04:17 MDT 2015 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP root@hurricane# sysctl hw hw.machine=amd64 hCw.model=IntelPU1: acpicpu setperf failed to alter frequency (R) -- dtse9# sysctl kern.version kern.version=OpenBSD 5.7-current (GENERIC.MP) #1021: Thu May 28 19:38:27 MDT 201 5 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP dtse9# sysctl hw hw.machine=amd64 hw.model=Intel(R) Xeon(R) CPU E5440 @ 2.83GHz hw.ncpu=8 hw.byteorder=1234 hw.pagesize=4096 hw.disknames=sd I have access to a similar server: HP ProLiant DL360 G5 with only one quadcore cpu installed (Intel Xeon E5335). There this problem does not appear. How-To-Repeat: Install OpenBSD 5.7 amd64 on a HP ProLiant DL360 G5 with two Intel Xeon E5440 cpus. Then do sysctl hw. Workaround: Run an sp kernel instead of mp. dmesg: OpenBSD 5.7 (GENERIC) #0: Thu Apr 30 22:01:01 CEST 2015 r...@stable-57-amd64.mtier.org:/binpatchng/work-binpatch57-amd64/src/sys/arch/amd64/compile/GENERIC real mem = 25751203840 (24558MB) avail mem = 25061834752 (23900MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xee000 (69 entries) bios0: vendor HP version P58 date 05/02/2011 bios0: HP ProLiant DL360 G5 acpi0 at bios0: rev 2 acpi0: sleep states S0 S4 S5 acpi0: tables DSDT FACP SPCR MCFG HPET SPMI ERST APIC BERT HEST SSDT acpi0: wakeup devices PCI0(S5) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimcfg0 at acpi0 addr 0xe000, bus 0-255 acpihpet0 at acpi0: 14318179 Hz acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Xeon(R) CPU E5440 @ 2.83GHz, 2833.77 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,XSAVE,NXE,LONG,LAHF,PERF cpu0: 6MB 64b/line 16-way L2 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges cpu0: apic clock running at 333MHz cpu at mainbus0: not configured cpu at mainbus0: not configured cpu at mainbus0: not configured cpu at mainbus0: not configured cpu at mainbus0: not configured cpu at mainbus0: not configured cpu at mainbus0: not configured ioapic0 at mainbus0: apid 8 pa 0xfec0, version 20, 24 pins ioapic1 at mainbus0: apid 9 pa 0xfec8, version 20, 24 pins acpiprt0 at acpi0: bus 1 (IP2P) acpiprt1 at acpi0: bus 11 (IPE1) acpiprt2 at acpi0: bus 10 (IPE4) acpiprt3 at acpi0: bus 16 (P2P2) acpiprt4 at acpi0: bus 9 (PT02) acpiprt5 at acpi0: bus 6 (PT03) acpiprt6 at acpi0: bus 19 (PT04) acpiprt7 at acpi0: bus 3 (NB01) acpiprt8 at acpi0: bus 5 (NB02) acpiprt9 at acpi0: bus 0 (PCI0) acpicpu0 at acpi0: C3, C1, FVS, 2832, 2332, 1999 MHz acpitz0 at acpi0: critical temperature is 31 degC ipmi at mainbus0 not configured pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 Intel 5000P Host rev 0xb1 ppb0 at pci0 dev 2 function 0 Intel 5000 PCIE rev 0xb1 pci1 at ppb0 bus 9 ppb1 at pci1 dev 0 function 0 Intel 6321ESB PCIE rev 0x01 pci2 at ppb1 bus 10 ppb2 at pci2 dev 0 function 0 Intel 6321ESB PCIE rev 0x01 pci3 at ppb2 bus 11 ppb3 at pci2 dev 1 function 0 Intel 6321ESB PCIE rev 0x01 pci4 at ppb3 bus 14 ppb4 at pci2 dev 2 function 0 Intel 6321ESB PCIE rev 0x01 pci5 at ppb4 bus 15 ppb5 at pci1 dev 0 function 3 Intel 6321ESB PCIE-PCIX rev 0x01 pci6 at ppb5 bus 16 ppb6 at pci0 dev 3 function 0 Intel 5000 PCIE rev 0xb1 pci7 at ppb6 bus 6 ciss0 at pci7 dev 0 function 0 Hewlett-Packard Smart Array rev 0x04: apic 8 int 16 ciss0: 1 LD, HW rev 4, FW 7.24/7.24, 64bit fifo scsibus1 at ciss0: 1 targets sd0 at scsibus1 targ 0 lun 0: HP, LOGICAL VOLUME, 7.24 SCSI3 0/direct fixed sd0: 2861512MB, 512 bytes/sector, 5860378032 sectors ppb7 at pci0 dev 4 function 0 Intel 5000 PCIE x8 rev 0xb1 pci8 at ppb7 bus 19 ppb8 at
Re: Serving large files with httpd eats fscktons of memory.
On 6/6/15, Florian Obser flor...@openbsd.org wrote: On Sat, Jun 06, 2015 at 07:05:46PM +, Florian Obser wrote: On Fri, Jun 05, 2015 at 05:15:09PM -0500, Matthew Martin wrote: Synopsis: Serving large files with httpd eats fscktons of memory. Category: system Environment: System : OpenBSD 5.7 Details : OpenBSD 5.7 (GENERIC) #0: Thu Apr 30 22:01:01 CEST 2015 r...@stable-57-amd64.mtier.org:/binpatchng/work-binpatch57-amd64/src/sys/arch/amd64/compile/GENERIC Architecture: OpenBSD.amd64 Machine : amd64 Description: A couple of people concurrently downloading iso sized files can drag the server into using all of swap (there's only a gig) in a frighteningly short time. Revision 1.30 of http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.sbin/httpd/server_file.c had message Adjust the read/write watermarks according to the TCP send buffer. This fixes sending of large files. Previously, httpd was reading the input file too quickly and could run out of memory when filling the input buffer. That pretty well describes the situation I'm seeing now with 5.7 (-RELEASE + m:tier binpatches). It also happens with -CURRENT (as of Jun 5) httpd compiled and slapped in its place. How-To-Repeat: Something along the lines of hostA# dd if=/dev/zero of=/var/www/htdocs/test_512.dat bs=1M count=512 hostB$ ftp http://hostA/test_512.dat Fix: I've attempted various settings of 'tcp socket buffer' and shotgunning around the lines modified in 1.30 to no avail. Trying to limit its consumption via login.conf just makes it die when it hits the limit. the problem is that we have to bufferevents. clt-clt_bev which is handleing the accept(2)ed socket and clt-clt_srvbev which is handleing the fd of the served file. it goes like this: clt_bev: Here I am, brain the size of a planet, and they ask me to serve a file. clt_srvbev: Whee, I can read clt_sndbufsiz bytes from the file. Hello clt_bev, here, have some data. clt_bev: Ok, I can't send it right now, but I have a buffer, I'll put the data at the end. [last message repeated n times] clt_bev: Phew, I can send some data to the client, hopfully clt_bev stops to send me data soon. and so on... In the end the whole iso ends up in clt_bev. I'm not sure yet about the 32 and 256 magic numbers, they seem to it some kind of sweet spot on *my* system. Make them to small and everything slows down, make them to large and you waste memory. Please try this: this gets rid of a compiler warning: diff --git server.c server.c index ca67a47..8725e2c 100644 --- server.c +++ server.c @@ -704,7 +704,7 @@ server_input(struct client *clt) /* Adjust write watermark to the socket buffer output size */ bufferevent_setwatermark(clt-clt_bev, EV_WRITE, - clt-clt_sndbufsiz, 0); + 32 * clt-clt_sndbufsiz, 0); /* Read at most amount of data that fits in one fcgi record. */ bufferevent_setwatermark(clt-clt_bev, EV_READ, 0, FCGI_CONTENT_SIZE); @@ -729,6 +729,10 @@ server_write(struct bufferevent *bev, void *arg) goto done; bufferevent_enable(bev, EV_READ); + + if (clt-clt_srvbev !(clt-clt_srvbev-enabled EV_READ)) + bufferevent_enable(clt-clt_srvbev, EV_READ); + return; done: (*bev-errorcb)(bev, EVBUFFER_WRITE|EVBUFFER_EOF, bev-cbarg); @@ -769,6 +773,11 @@ server_read(struct bufferevent *bev, void *arg) goto fail; if (clt-clt_done) goto done; + + if (EVBUFFER_LENGTH(EVBUFFER_OUTPUT(clt-clt_bev)) (size_t) 256 * + clt-clt_sndbufsiz) + bufferevent_disable(bev, EV_READ); + return; done: (*bev-errorcb)(bev, EVBUFFER_READ|EVBUFFER_EOF, bev-cbarg); -- I'm not entirely sure you are real. Works great. $ sudo pkg_delete apache-httpd # :) Here's some meaningless benchmarks. Disregard grotesque CPU usage: It's a virtual machine under virtualbox running on a commodity craptop with a slow disk trying to read from one VM and write to another. And everything else because who cares about benchmarks. But as there doesn't seem to be anything else too horrific I'll throw it on the live site and maybe get some of the other recent changes stressed a bit too. If I hit anything else I'll double check on an entirely -CURRENT system. # -RELEASE + binpatch httpd: # ### After booting, before doing anything: ### load averages: 0.57, 0.24, 0.10test.vm.local 12:53:35 35 processes: 34 idle, 1 on processor 1 CPUs: 1.1% user, 0.0% nice, 1.5% system, 0.1% interrupt, 97.3% idle Memory: Real: 23M/73M act/tot Free: 165M Cache: 20M Swap: 0K/1024M PID USERNAME PRI NICE SIZE RES STATE WAIT TIMECPU COMMAND 1 root 100 516K 524K sleep wait 0:01 0.00% init 20593 admin 20 1168K 2140K sleep kqread0:00 0.00%
Re: Serving large files with httpd eats fscktons of memory.
On Fri, Jun 05, 2015 at 05:15:09PM -0500, Matthew Martin wrote: Synopsis:Serving large files with httpd eats fscktons of memory. Category:system Environment: System : OpenBSD 5.7 Details : OpenBSD 5.7 (GENERIC) #0: Thu Apr 30 22:01:01 CEST 2015 r...@stable-57-amd64.mtier.org:/binpatchng/work-binpatch57-amd64/src/sys/arch/amd64/compile/GENERIC Architecture: OpenBSD.amd64 Machine : amd64 Description: A couple of people concurrently downloading iso sized files can drag the server into using all of swap (there's only a gig) in a frighteningly short time. Revision 1.30 of http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.sbin/httpd/server_file.c had message Adjust the read/write watermarks according to the TCP send buffer. This fixes sending of large files. Previously, httpd was reading the input file too quickly and could run out of memory when filling the input buffer. That pretty well describes the situation I'm seeing now with 5.7 (-RELEASE + m:tier binpatches). It also happens with -CURRENT (as of Jun 5) httpd compiled and slapped in its place. How-To-Repeat: Something along the lines of hostA# dd if=/dev/zero of=/var/www/htdocs/test_512.dat bs=1M count=512 hostB$ ftp http://hostA/test_512.dat Fix: I've attempted various settings of 'tcp socket buffer' and shotgunning around the lines modified in 1.30 to no avail. Trying to limit its consumption via login.conf just makes it die when it hits the limit. the problem is that we have to bufferevents. clt-clt_bev which is handleing the accept(2)ed socket and clt-clt_srvbev which is handleing the fd of the served file. it goes like this: clt_bev: Here I am, brain the size of a planet, and they ask me to serve a file. clt_srvbev: Whee, I can read clt_sndbufsiz bytes from the file. Hello clt_bev, here, have some data. clt_bev: Ok, I can't send it right now, but I have a buffer, I'll put the data at the end. [last message repeated n times] clt_bev: Phew, I can send some data to the client, hopfully clt_bev stops to send me data soon. and so on... In the end the whole iso ends up in clt_bev. I'm not sure yet about the 32 and 256 magic numbers, they seem to it some kind of sweet spot on *my* system. Make them to small and everything slows down, make them to large and you waste memory. Please try this: diff --git server.c server.c index ca67a47..22b47b8 100644 --- server.c +++ server.c @@ -704,7 +704,7 @@ server_input(struct client *clt) /* Adjust write watermark to the socket buffer output size */ bufferevent_setwatermark(clt-clt_bev, EV_WRITE, - clt-clt_sndbufsiz, 0); + 32 * clt-clt_sndbufsiz, 0); /* Read at most amount of data that fits in one fcgi record. */ bufferevent_setwatermark(clt-clt_bev, EV_READ, 0, FCGI_CONTENT_SIZE); @@ -729,6 +729,10 @@ server_write(struct bufferevent *bev, void *arg) goto done; bufferevent_enable(bev, EV_READ); + + if (clt-clt_srvbev !(clt-clt_srvbev-enabled EV_READ)) + bufferevent_enable(clt-clt_srvbev, EV_READ); + return; done: (*bev-errorcb)(bev, EVBUFFER_WRITE|EVBUFFER_EOF, bev-cbarg); @@ -769,6 +773,11 @@ server_read(struct bufferevent *bev, void *arg) goto fail; if (clt-clt_done) goto done; + + if (EVBUFFER_LENGTH(EVBUFFER_OUTPUT(clt-clt_bev)) 256 * + clt-clt_sndbufsiz) + bufferevent_disable(bev, EV_READ); + return; done: (*bev-errorcb)(bev, EVBUFFER_READ|EVBUFFER_EOF, bev-cbarg); -- I'm not entirely sure you are real.
Re: Serving large files with httpd eats fscktons of memory.
On Sat, Jun 06, 2015 at 07:05:46PM +, Florian Obser wrote: On Fri, Jun 05, 2015 at 05:15:09PM -0500, Matthew Martin wrote: Synopsis: Serving large files with httpd eats fscktons of memory. Category: system Environment: System : OpenBSD 5.7 Details : OpenBSD 5.7 (GENERIC) #0: Thu Apr 30 22:01:01 CEST 2015 r...@stable-57-amd64.mtier.org:/binpatchng/work-binpatch57-amd64/src/sys/arch/amd64/compile/GENERIC Architecture: OpenBSD.amd64 Machine : amd64 Description: A couple of people concurrently downloading iso sized files can drag the server into using all of swap (there's only a gig) in a frighteningly short time. Revision 1.30 of http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.sbin/httpd/server_file.c had message Adjust the read/write watermarks according to the TCP send buffer. This fixes sending of large files. Previously, httpd was reading the input file too quickly and could run out of memory when filling the input buffer. That pretty well describes the situation I'm seeing now with 5.7 (-RELEASE + m:tier binpatches). It also happens with -CURRENT (as of Jun 5) httpd compiled and slapped in its place. How-To-Repeat: Something along the lines of hostA# dd if=/dev/zero of=/var/www/htdocs/test_512.dat bs=1M count=512 hostB$ ftp http://hostA/test_512.dat Fix: I've attempted various settings of 'tcp socket buffer' and shotgunning around the lines modified in 1.30 to no avail. Trying to limit its consumption via login.conf just makes it die when it hits the limit. the problem is that we have to bufferevents. clt-clt_bev which is handleing the accept(2)ed socket and clt-clt_srvbev which is handleing the fd of the served file. it goes like this: clt_bev: Here I am, brain the size of a planet, and they ask me to serve a file. clt_srvbev: Whee, I can read clt_sndbufsiz bytes from the file. Hello clt_bev, here, have some data. clt_bev: Ok, I can't send it right now, but I have a buffer, I'll put the data at the end. [last message repeated n times] clt_bev: Phew, I can send some data to the client, hopfully clt_bev stops to send me data soon. and so on... In the end the whole iso ends up in clt_bev. I'm not sure yet about the 32 and 256 magic numbers, they seem to it some kind of sweet spot on *my* system. Make them to small and everything slows down, make them to large and you waste memory. Please try this: this gets rid of a compiler warning: diff --git server.c server.c index ca67a47..8725e2c 100644 --- server.c +++ server.c @@ -704,7 +704,7 @@ server_input(struct client *clt) /* Adjust write watermark to the socket buffer output size */ bufferevent_setwatermark(clt-clt_bev, EV_WRITE, - clt-clt_sndbufsiz, 0); + 32 * clt-clt_sndbufsiz, 0); /* Read at most amount of data that fits in one fcgi record. */ bufferevent_setwatermark(clt-clt_bev, EV_READ, 0, FCGI_CONTENT_SIZE); @@ -729,6 +729,10 @@ server_write(struct bufferevent *bev, void *arg) goto done; bufferevent_enable(bev, EV_READ); + + if (clt-clt_srvbev !(clt-clt_srvbev-enabled EV_READ)) + bufferevent_enable(clt-clt_srvbev, EV_READ); + return; done: (*bev-errorcb)(bev, EVBUFFER_WRITE|EVBUFFER_EOF, bev-cbarg); @@ -769,6 +773,11 @@ server_read(struct bufferevent *bev, void *arg) goto fail; if (clt-clt_done) goto done; + + if (EVBUFFER_LENGTH(EVBUFFER_OUTPUT(clt-clt_bev)) (size_t) 256 * + clt-clt_sndbufsiz) + bufferevent_disable(bev, EV_READ); + return; done: (*bev-errorcb)(bev, EVBUFFER_READ|EVBUFFER_EOF, bev-cbarg); -- I'm not entirely sure you are real.
