https://bz.apache.org/bugzilla/show_bug.cgi?id=61616
Bug ID: 61616
Summary: mod_proxy_connect: stall and connection loss on
bi-directional traffic
Product: Apache httpd-2
Version: 2.4.25
Hardware: PC
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: mod_proxy_connect
Assignee: bugs@httpd.apache.org
Reporter: cars...@wolffcarsten.de
Target Milestone: ---
Created attachment 35421
--> https://bz.apache.org/bugzilla/attachment.cgi?id=35421&action=edit
default vhost with proxy_connect conf
When proxy_connect is used for taffic that is less of a request-response
nature, it can happen, that the send buffer on the backend socket gets full. In
that case, in libapr, when the writev() comes back with EAGAIN, it will block
in poll() on the backend socket with a 300-second timeout. No read from any of
the two sockets can happen anymore in that situation. Thus, depending on the
application protocol that is tunneled through CONNECT, the backend application
will then fillup the receive buffer at proxy_connect's backend socket and then
stall trying to send indefinitely. Now proxy_connect and the backend
application are in a deadlock.
A concrete setup to reproduce it (using debian stretch):
rsync -------------------------> apache ---------------> rsync --daemon
CONNECT localhost:873
I will attach configuration and a script to this report and can also provide a
qemu image, if desired.
In this setup the stall looks like this:
----------------------------- rsync ----------------------------------------
# RSYNC_PROXY=localhost:80 rsync -rcP rsync://localhost:873/testsrc/ /testdst/
[..]
5269
0 0% 0.00kB/s 0:00:00
[ .. 5 minute delay .. ]
rsync: connection unexpectedly closed (403374822 bytes received so far)
[receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(235)
[receiver=3.1.2]
rsync: connection unexpectedly closed (242925 bytes received so far)
[generator]
rsync error: error in rsync protocol data stream (code 12) at io.c(235)
[generator=3.1.2]
----------------------------- netstat --------------------------------------
# netstat -pnt | grep -v :22
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
PID/Program name
tcp6 0 0 ::1:43642 ::1:80
ESTABLISHED 21638/rsync
tcp6 169814 0 ::1:80 ::1:43642
ESTABLISHED 20628/apache2
tcp6 5974823 2608576 ::1:43798 ::1:873
ESTABLISHED 20628/apache2
tcp6 809935 2760413 ::1:873 ::1:43798
ESTABLISHED 21639/rsync
----------------------------- strace --------------------------------------
[..]
read(10,
"\24\372\342\1\16\255\223\375G\213k\251\215\360\357}\311\275\202\5jQ>\341>\370
jr\357,\4"..., 8000) = 8000 <---- fd 10 is client socket
writev(11, [{iov_base= [..] ., iov_len=7424}], 16) = -1 EAGAIN (Resource
temporarily unavailable) <---- fd 11 is backend socket
poll([{fd=11, events=POLLOUT}], 1, 300000)
<detached ...>
----------------------------- gdb -----------------------------------------
(gdb) bt
#0 0x00007f37a88c4690 in __poll_nocancel () at
../sysdeps/unix/syscall-template.S:84
#1 0x00007f37a8dc5088 in apr_poll () from
target:/usr/lib/x86_64-linux-gnu/libapr-1.so.0
#2 0x00005572f5f781cd in send_brigade_blocking (c=0x7f37a95ad9b0,
bytes_written=0x7f37a95ade68, bb=0x7f37a95a1068, s=0x7f37a95a6c28) at
core_filters.c:747
#3 ap_core_output_filter (f=0x7f37a95add88, new_bb=0x7f37a95a1068) at
core_filters.c:542
#4 0x00007f37a58fac77 in ap_proxy_transfer_between_connections
(r=r@entry=0x7f37a95a50a0, c_i=0x7f37a95ad290, c_o=0x7f37a95ad9b0,
bb_i=<optimized out>, bb_o=0x7f37a95a1068, name=0x7f37a56e39dd "client",
sent=0x0,
bsize=8192, after=1) at proxy_util.c:3836
#5 0x00007f37a56e31c3 in proxy_connect_handler (r=0x7f37a95a50a0,
worker=<optimized out>, conf=<optimized out>, url=<optimized out>,
proxyname=<optimized out>, proxyport=<optimized out>) at
mod_proxy_connect.c:421
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscr...@httpd.apache.org
For additional commands, e-mail: bugs-h...@httpd.apache.org
