Hi,
I have question about PING feature on mod_proxy_http. (I am not sure that this
is right mailing list).
When reverse proxy server does not received 100-Continue before PING timeout,
doest request should be replayed fully on next backend server?
If yes (request should be replyed on next backend server), i observed strange
behaviour of httpd when PING timeout occurs POST request is replayed (failover)
on next backend server but the Content-Length of the POST request has been set
with value 0 and not POST data are sent. Hereafter the POST request forwarded
during failover on next backend server:
POST /foo/form.html HTTP/1.1
Host: xxxx
User-Agent: Wget/1.14 (linux-gnu)
Accept: */*
Content-Type: application/x-www-form-urlencoded
Expect: 100-Continue, 100-Continue
X-Forwarded-For: 192.168.216.158, 192.168.216.158
X-Forwarded-Host: foo, foo
X-Forwarded-Server: foo, foo
Connection: Keep-Alive
Content-Length: 0
Original request forwarded one first backend server:
POST /foo/form.html HTTP/1.1
Host: xxxx
User-Agent: Wget/1.14 (linux-gnu)
Accept: */*
Content-Type: application/x-www-form-urlencoded
Expect: 100-Continue
X-Forwarded-For: 192.168.216.158
X-Forwarded-Host: foo
X-Forwarded-Server: foo
Connection: Keep-Alive
Content-Length: 7
foo=jjj
Is it normal that Content-Length equal zero and no data are sent during
failover?
This case occurs for exemple when TCP session has been etablished between
reverse proxy HTTPD and backend server, and backend does not respond
100-Continue fast enough.
>From my observations it seems that failover on POST request replay does not
>works correctly.
My load balancer configuration is:
<Proxy balancer://www01 nonce=none>
BalancerMember http://10.206.3.179:80 retry=90 ping=5 timeout=15
connectiontimeout=5 lbset=2 route=179 ttl=60
BalancerMember http://10.206.3.180:80 retry=90 ping=5 timeout=15
connectiontimeout=5 lbset=2 route=180 ttl=60
ProxySet lbmethod=bytraffic
</Proxy>
My reverse proxy OS is Centos 7 and my httpd version is:
Server version: Apache/2.4.6 (CentOS)
Server built: Nov 19 2015 21:43:13
To reproduce the case on backend server DROP HTTP stream (but allow TCP session
establishment):
sudo iptables -A INPUT -i eth0 -p tcp -d 10.206.3.179 --dport 80 ! --tcp-flags
SYN,ACK SYN -j DROP
Reverse proxy is 10.206.3.14
Backend 1 is 10.206.3.179
Backend 2 (failover) is 10.206.3.180
Thank you for your response and your help
Full trace:
21:39:19.679883 IP 10.206.3.146.50441 > 10.206.3.179.80: Flags [S], seq
2351030123, win 29120, options [mss 1456,sackOK,TS val 1677094581 ecr
0,nop,wscale 7], length 0
E..<..@.@.k.
...
.... .P.!.k......q............
c.n.........
21:39:20.682489 IP 10.206.3.146.50441 > 10.206.3.179.80: Flags [S], seq
2351030123, win 29120, options [mss 1456,sackOK,TS val 1677095584 ecr
0,nop,wscale 7], length 0
E..<..@.@.k.
...
.... .P.!.k......q............
c.r.........
21:39:20.683586 IP 10.206.3.179.80 > 10.206.3.146.50441: Flags [S.], seq
784451006, ack 2351030124, win 14440, options [mss 1456,sackOK,TS val
2787238342 ecr 1677095584,nop,wscale 7], length 0
E..<..@.@...
...
....P. .....!.l..8hn..........
.!..c.r.....
21:39:20.683623 IP 10.206.3.146.50441 > 10.206.3.179.80: Flags [.], ack 1, win
228, options [nop,nop,TS val 1677095585 ecr 2787238342], length 0
E..4..@.@.l.
...
.... .P.!.l...............
c.r..!..
21:39:20.683902 IP 10.206.3.146.50441 > 10.206.3.179.80: Flags [P.], seq 1:337,
ack 1, win 228, options [nop,nop,TS val 1677095585 ecr 2787238342], length 336
E.....@.@.j.
...
.... .P.!.l.........W.....
c.r..!..POST /foo/form.html HTTP/1.1
Host: foo
User-Agent: Wget/1.14 (linux-gnu)
Accept: */*
Content-Type: application/x-www-form-urlencoded
Expect: 100-Continue
X-Forwarded-For: 192.168.216.158
X-Forwarded-Host: foo
X-Forwarded-Server: foo
Connection: Keep-Alive
Content-Length: 7
foo=jjj
21:39:20.884469 IP 10.206.3.146.50441 > 10.206.3.179.80: Flags [P.], seq 1:337,
ack 1, win 228, options [nop,nop,TS val 1677095786 ecr 2787238342], length 336
E.....@.@.j.
...
.... .P.!.l.........W.....
c.sj.!..POST /foo/form.html HTTP/1.1
Host: foo
User-Agent: Wget/1.14 (linux-gnu)
Accept: */*
Content-Type: application/x-www-form-urlencoded
Expect: 100-Continue
X-Forwarded-For: 192.168.216.158
X-Forwarded-Host: foo
X-Forwarded-Server: foo
Connection: Keep-Alive
Content-Length: 7
foo=jjj
21:39:21.085467 IP 10.206.3.146.50441 > 10.206.3.179.80: Flags [P.], seq 1:337,
ack 1, win 228, options [nop,nop,TS val 1677095987 ecr 2787238342], length 336
E.....@.@.j.
...
.... .P.!.l.........W.....
c.t3.!..POST /foo/form.html HTTP/1.1
Host: foo
User-Agent: Wget/1.14 (linux-gnu)
Accept: */*
Content-Type: application/x-www-form-urlencoded
Expect: 100-Continue
X-Forwarded-For: 192.168.216.158
X-Forwarded-Host: foo
21:39:21.085467 IP 10.206.3.146.50441 > 10.206.3.179.80: Flags [P.], seq 1:337,
ack 1, win 228, options [nop,nop,TS val 1677095987 ecr 2787238342], length 336
E.....@.@.j.
...
.... .P.!.l.........W.....
c.t3.!..POST /foo/form.html HTTP/1.1
Host: foo
User-Agent: Wget/1.14 (linux-gnu)
Accept: */*
Content-Type: application/x-www-form-urlencoded
Expect: 100-Continue
X-Forwarded-For: 192.168.216.158
X-Forwarded-Host: foo
X-Forwarded-Server: foo
Connection: Keep-Alive
Content-Length: 7
foo=jjj
21:39:21.488518 IP 10.206.3.146.50441 > 10.206.3.179.80: Flags [P.], seq 1:337,
ack 1, win 228, options [nop,nop,TS val 1677096390 ecr 2787238342], length 336
E.....@.@.j.
...
.... .P.!.l.........W.....
c.u..!..POST /foo/form.html HTTP/1.1
Host: foo
User-Agent: Wget/1.14 (linux-gnu)
Accept: */*
Content-Type: application/x-www-form-urlencoded
Expect: 100-Continue
X-Forwarded-For: 192.168.216.158
X-Forwarded-Host: foo
X-Forwarded-Server: foo
Connection: Keep-Alive
Content-Length: 7
foo=jjj
21:39:21.882671 IP 10.206.3.179.80 > 10.206.3.146.50441: Flags [S.], seq
784451006, ack 2351030124, win 14440, options [mss 1456,sackOK,TS val
2787239542 ecr 1677095584,nop,wscale 7], length 0
E..<..@.@...
...
....P. .....!.l..8hj..........
.!.vc.r.....
21:39:21.882686 IP 10.206.3.146.50441 > 10.206.3.179.80: Flags [.], ack 1, win
228, options [nop,nop,TS val 1677096784 ecr 2787238342], length 0
E..4..@.@.k.
...
.... .P.!.................
c.wP.!..
21:39:22.294489 IP 10.206.3.146.50441 > 10.206.3.179.80: Flags [P.], seq 1:337,
ack 1, win 228, options [nop,nop,TS val 1677097196 ecr 2787238342], length 336
E.....@.@.j.
...
.... .P.!.l.........W.....
c.x..!..POST /foo/form.html HTTP/1.1
Host: foo
User-Agent: Wget/1.14 (linux-gnu)
Accept: */*
Content-Type: application/x-www-form-urlencoded
Expect: 100-Continue
X-Forwarded-For: 192.168.216.158
X-Forwarded-Host: foo
X-Forwarded-Server: foo
Connection: Keep-Alive
Content-Length: 7
foo=jjj
21:39:23.882741 IP 10.206.3.179.80 > 10.206.3.146.50441: Flags [S.], seq
784451006, ack 2351030124, win 14440, options [mss 1456,sackOK,TS val
2787241542 ecr 1677095584,nop,wscale 7], length 0
E..<..@.@...
...
....P. .....!.l..8hbD.........
.!.Fc.r.....
21:39:23.882773 IP 10.206.3.146.50441 > 10.206.3.179.80: Flags [.], ack 1, win
228, options [nop,nop,TS val 1677098784 ecr 2787238342], length 0
E..4..@.@.k.
...
.... .P.!.................
c.. .!..
21:39:23.906484 IP 10.206.3.146.50441 > 10.206.3.179.80: Flags [P.], seq 1:337,
ack 1, win 228, options [nop,nop,TS val 1677098808 ecr 2787238342], length 336
E.....@.@.j.
...
.... .P.!.l.........W.....
c..8.!..POST /foo/form.html HTTP/1.1
Host: foo
User-Agent: Wget/1.14 (linux-gnu)
Accept: */*
Content-Type: application/x-www-form-urlencoded
Expect: 100-Continue
X-Forwarded-For: 192.168.216.158
X-Forwarded-Host: foo
X-Forwarded-Server: foo
Connection: Keep-Alive
Content-Length: 7
foo=jjj
21:39:25.689162 IP 10.206.3.146.50441 > 10.206.3.179.80: Flags [F.], seq 337,
ack 1, win 228, options [nop,nop,TS val 1677100590 ecr 2787238342], length 0
E..4..@.@.k.
...
.... .P.!.................
c....!..
21:39:25.689403 IP 10.206.3.146.46621 > 10.206.3.180.80: Flags [S], seq
2060213667, win 29120, options [mss 1456,sackOK,TS val 1677100590 ecr
0,nop,wscale 7], length 0
E..<..@.@.BM
...
......Pz.].......q............
c...........
21:39:25.690007 IP 10.206.3.180.80 > 10.206.3.146.46621: Flags [S.], seq
2781467574, ack 2060213668, win 14440, options [mss 1456,sackOK,TS val
2787243349 ecr 1677100590,nop,wscale 7], length 0
E..<..@.@...
...
....P......z.]...8hd..........
.!.Uc.......
21:39:25.690036 IP 10.206.3.146.46621 > 10.206.3.180.80: Flags [.], ack 1, win
228, options [nop,nop,TS val 1677100591 ecr 2787243349], length 0
E..4..@.@.BT
...
......Pz.]................
c../.!.U
21:39:25.690152 IP 10.206.3.146.46621 > 10.206.3.180.80: Flags [P.], seq 1:399,
ack 1, win 228, options [nop,nop,TS val 1677100591 ecr 2787243349], length 398
E.....@.@.@.
...
......Pz.]................
c../.!.UPOST /foo/form.html HTTP/1.1
Host: foo
User-Agent: Wget/1.14 (linux-gnu)
Accept: */*
Content-Type: application/x-www-form-urlencoded
Expect: 100-Continue, 100-Continue
X-Forwarded-For: 192.168.216.158, 192.168.216.158
X-Forwarded-Host: foo, foo
X-Forwarded-Server: foo, foo
Connection: Keep-Alive
Content-Length: 0
21:39:25.690911 IP 10.206.3.180.80 > 10.206.3.146.46621: Flags [.], ack 399,
win 122, options [nop,nop,TS val 2787243350 ecr 1677100591], length 0
E..4&.@.@...
...
....P......z._2...z.......
.!.Vc../
21:39:25.691243 IP 10.206.3.180.80 > 10.206.3.146.46621: Flags [P.], seq 1:552,
ack 399, win 122, options [nop,nop,TS val 2787243350 ecr 1677100591], length 551
E..[&.@.@...
...
....P......z._2...z.X.....
.!.Vc../HTTP/1.1 417 Expectation Failed
Date: Mon, 30 May 2016 19:39:25 GMT
Server: Apache
Content-Length: 378
Connection: close
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>417 Expectation Failed</title>
</head><body>
<h1>Expectation Failed</h1>
<p>The expectation given in the Expect request-header
field could not be met by this server.
The client sent<pre>
Expect: 100-Continue, 100-Continue
</pre>
</p><p>Only the 100-continue expectation is supported.</p>
</body></html>
21:39:25.691257 IP 10.206.3.146.46621 > 10.206.3.180.80: Flags [.], ack 552,
win 237, options [nop,nop,TS val 1677100592 ecr 2787243350], length 0
E..4..@.@.BR
...
......Pz._2...............
c..0.!.V
21:39:25.691282 IP 10.206.3.180.80 > 10.206.3.146.46621: Flags [F.], seq 552,
ack 399, win 122, options [nop,nop,TS val 2787243350 ecr 1677100591], length 0
E..4&.@.@...
...
....P......z._2...z.......
.!.Vc../
21:39:25.691396 IP 10.206.3.146.46621 > 10.206.3.180.80: Flags [F.], seq 399,
ack 553, win 237, options [nop,nop,TS val 1677100592 ecr 2787243350], length 0
E..4..@.@.BQ
...
......Pz._2...............
c..0.!.V
21:39:25.692009 IP 10.206.3.180.80 > 10.206.3.146.46621: Flags [.], ack 400,
win 122, options [nop,nop,TS val 2787243351 ecr 1677100592], length 0
E..4&.@.@...
...
....P......z._3...z.......
.!.Wc..0
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org
