Thoughts welcomed on the following problem :
Centos 5.4, http 2.2.3, tomcat5-5.5.23-0jpp.7.el5_3.2,
java-1.6.0-openjdk-1.6.0.0-1.7.b09.el5
A large servlet request, POST'ed from client to apache, connected using
proxy_ajp to ajp://localhost:8009.
reassembled on server eth0 as 10766 bytes but only partially sent to tomcat. Th
eth0 line trace to http ( port 80 ) was split over 19 packets of 536 bytes each.
My initial thoughts were, that maxHttpHeaderSize=8192, was too low. Increasing
to 16384 did not resolve the issue.
So more wireshark line traces, ( one tcpdump across eth0 capturing the http
POST and a 2nd tcpdump across loopback capturing ajp connector traffic )
revealed, that apache via connector ajp delivered each packet realtime time to
tomcat, without waiting for all 10766 bytes to arrive, though the trace across
loopback on port 8009, revealed that tomcat starting the reply before all 10766
bytes had arrived.
tcpdump on eth0 confirmed, by reassembled tcp segment to contained the 10766
bytes from the browser client.
( Note : I have mangled URI SRV referer host headers )
Apache JServ Protocol v1.3
Magic: 1234
Length: 528
Code: (2) FORWARD REQUEST
Method: (4) POST
Version: HTTP/1.1
URI: /xxxx/yyyyy
RADDR: 192.168.252.68
RHOST:
SRV: xxxyyyy
PORT: 80
SSLP: 0
NHDR: 11
accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
application/x-shockwave-flash, application/vnd.ms-powerpoint,
application/vnd.ms-excel, application/msword, */*
referer: http://xxxxyyyy
accept-language: en-us
content-type: application/x-www-form-urlencoded
UA-CPU: x86
accept-encoding: gzip, deflate
user-agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
host: xxxyyyy
content-length: 10766
connection: Keep-Alive
Cache-Control: no-cache
My question is, why did tomcat start to send the RSP ( response ) after only
receiving the first data packet from ajp connector, around 500 bytes, when the
above states the content length is 10766 bytes. You can also see that
subsequent REQ ( request body ) data packets are still transmitted to tomcat,
though the response is already being returned.
Another option I thought of, was to buffer the whole 10766 bytes up on the
apache side, and then get the connector to pass the request across as one 'big'
packet. Is this possible to configure ? If so, then maxHttpHeaderSize will come
into play, though at the moment only many small packets are being sent across
in the fragmented request.
A stack dump in catalina.out can be seen below, where the input filter is
balking on the POSTed parameters, most likely because only 550 bytes of the
10766, have turned up for the input filter to process.
Apr 9, 2010 10:51:55 AM org.apache.catalina.connector.Request parseParameters
WARNING: Exception thrown whilst processing POSTed parameters
java.io.IOException: Socket read failed
at org.apache.coyote.ajp.AjpAprProcessor.read(AjpAprProcessor.java:1038)
at
org.apache.coyote.ajp.AjpAprProcessor.readMessage(AjpAprProcessor.java:1159)
at
org.apache.coyote.ajp.AjpAprProcessor.receive(AjpAprProcessor.java:1091)
at
org.apache.coyote.ajp.AjpAprProcessor.refillReadBuffer(AjpAprProcessor.java:1130)
at
org.apache.coyote.ajp.AjpAprProcessor.access$0(AjpAprProcessor.java:1115)
at
org.apache.coyote.ajp.AjpAprProcessor$SocketInputBuffer.doRead(AjpAprProcessor.java:1233)
at org.apache.coyote.Request.doRead(Request.java:419)
at
org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:265)
at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:403)
at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:280)
at
org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:193)
at org.apache.catalina.connector.Request.readPostBody(Request.java:2419)
at
org.apache.catalina.connector.Request.parseParameters(Request.java:2398)
at org.apache.catalina.connector.Request.getParameter(Request.java:1005)
at
org.apache.catalina.connector.RequestFacade.getParameter(RequestFacade.java:353)
at
com.xxxx.yyyyy.filters.RequestValidation.getParameter(RequestValidation.java:40)
at com.xxxx.yyyy.zzzzz.service(zzzzz.java:157)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at com.xxxx.yyyy.filters.RequestFilter.doFilter(RequestFilter.java:16)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
at
org.apache.coyote.ajp.AjpAprProcessor.process(AjpAprProcessor.java:444)
No. Time Source SrcPort Destination
DPort Protocol Info
731 10:24:51.406434 127.0.0.1 46704 127.0.0.1
8009 AJP13 0:REQ:POST /xxxx/yyyy HTTP/1.1
732 10:24:51.406452 127.0.0.1 46704 127.0.0.1
8009 AJP13 0:REQ:Body
733 10:24:51.408554 127.0.0.1 8009 127.0.0.1
46704 TCP 8009 > 46704 [ACK] Seq=273919 Ack=8041 Win=386 Len=0
TSV=192177833 TSER=192177830
734 10:24:51.408730 127.0.0.1 8009 127.0.0.1
46704 AJP13 0:RSP:GET BODY CHUNK
735 10:24:51.408737 127.0.0.1 46704 127.0.0.1
8009 TCP 46704 > 8009 [ACK] Seq=8041 Ack=273926 Win=2440 Len=0
TSV=192177833 TSER=192177833
742 10:24:51.460670 127.0.0.1 46704 127.0.0.1
8009 AJP13 0:REQ:Body
743 10:24:51.460728 127.0.0.1 8009 127.0.0.1
46704 AJP13 0:RSP:GET BODY CHUNK
767 10:24:51.500813 127.0.0.1 46704 127.0.0.1
8009 TCP 46704 > 8009 [ACK] Seq=8583 Ack=273933 Win=2441 Len=0
TSV=192177925 TSER=192177885
768 10:24:51.562298 127.0.0.1 8009 127.0.0.1
46704 AJP13 0:RSP:SEND BODY CHUNK
769 10:24:51.562316 127.0.0.1 46704 127.0.0.1
8009 TCP 46704 > 8009 [ACK] Seq=8583 Ack=282304 Win=2389 Len=0
TSV=192177986 TSER=192177986
770 10:24:51.562662 127.0.0.1 8009 127.0.0.1
46704 AJP13 0:RSP:SEND BODY CHUNK
771 10:24:51.562670 127.0.0.1 46704 127.0.0.1
8009 TCP 46704 > 8009 [ACK] Seq=8583 Ack=298688 Win=2291 Len=0
TSV=192177987 TSER=192177987
772 10:24:51.581234 127.0.0.1 8009 127.0.0.1
46704 AJP13 0:RSP:END RESPONSE
773 10:24:51.581243 127.0.0.1 46704 127.0.0.1
8009 TCP 46704 > 8009 [ACK] Seq=8583 Ack=307526 Win=2236 Len=0
TSV=192178005 TSER=192178005
783 10:24:51.700763 127.0.0.1 46704 127.0.0.1
8009 AJP13 0:REQ:Body
784 10:24:51.701186 127.0.0.1 8009 127.0.0.1
46704 AJP13 0:RSP:END RESPONSE
785 10:24:51.701194 127.0.0.1 46704 127.0.0.1
8009 TCP 46704 > 8009 [ACK] Seq=9125 Ack=307555 Win=2417 Len=0
TSV=192178125 TSER=192178125
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
