Re: memory leak in http codec
Luis Neves wrote: I find those - and ~ very weird, I doubt that the clients are sending those headers. My bad. As it turns out the clients are really sending that garbage. The main problem seems to be that the decoder gets confused when clients send Headers without values, e.g: *** GET /somepath HTTP/1.1 Accept: */* Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 Host: foo.bar.com Referer: Connection: Keep-Alive *** Notice the empty referer header. I worked around my problem by putting a limit in the size of every Collection that holds HTTP Headers and also an upper/lower limit in the length of every Header Name/Value. I still don't know how to properly fixed the decoder but for now this inelegant fix will do. As an aside I also tried the Grizzly http server as stopgap measure and although there are no OOM errors with misbehaved requests it completely freezes and stops serving requests... but that might just be because of my unfamiliarity with the creature. -- Luis Neves
Question regards AsyncWeb
Hi all, where's the current AsyncWeb code being hosted that compiles with MINA trunk? It be also awesome if somebody could give me some hints regards AsyncWeb and MINA 1.1.x. The latest release that's offered on safehaus.org seems to be built for older versions of MINA? Thanks James
Re: memory leak in http codec
Luis Neves wrote: Luis Neves wrote: I find those - and ~ very weird, I doubt that the clients are sending those headers. My bad. As it turns out the clients are really sending that garbage. The main problem seems to be that the decoder gets confused when clients send Headers without values, e.g: *** GET /somepath HTTP/1.1 Accept: */* Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 Host: foo.bar.com Referer: Connection: Keep-Alive *** Notice the empty referer header. I worked around my problem by putting a limit in the size of every Collection that holds HTTP Headers and also an upper/lower limit in the length of every Header Name/Value. I still don't know how to properly fixed the decoder but for now this inelegant fix will do. As an aside I also tried the Grizzly http server as stopgap measure and although there are no OOM errors with misbehaved requests it completely freezes and stops serving requests... but that might just be because of my unfamiliarity with the creature. Well, in any way, I think MINA should be able to handle faulty clients like these more gracefully. OOM isn't acceptable. Please file a JIRA issue and we'll look into it. /Niklas
Re: Question regards AsyncWeb
James, I believe it is here: https://svn.apache.org/repos/asf/mina/sandbox/asyncweb/ Jeff James Apfel wrote: Hi all, where's the current AsyncWeb code being hosted that compiles with MINA trunk? It be also awesome if somebody could give me some hints regards AsyncWeb and MINA 1.1.x. The latest release that's offered on safehaus.org seems to be built for older versions of MINA? Thanks James
[jira] Created: (DIRMINA-505) OOM errors when handling badly formed HTTP requests
OOM errors when handling badly formed HTTP requests --- Key: DIRMINA-505 URL: https://issues.apache.org/jira/browse/DIRMINA-505 Project: MINA Issue Type: Bug Components: Protocol - HTTP, Statemachine Affects Versions: 2.0.0-M2 Reporter: Luis Neves Badly formed HTTP Requests can make the HTTP decoder to cause OOM errors. The following request captured with the command tcpdump -nnASs 0 'dst port 80' is an example of such request (beware wrapping): ** 21:26:55.828483 IP 83.174.45.34.59872 213.13.146.84.80: S 3131042262:3131042262(0) win 65535 mss 1460,nop,wscale 0,nop,nop,timestamp 13682342 0 .T...P... 21:26:55.828606 IP 89.181.19.190.64449 213.13.146.84.80: . ack 4109384713 win 16407 .T...Py'[EMAIL PROTECTED] 21:26:55.837645 IP 83.174.45.34.59872 213.13.146.84.80: . ack 12756759 win 65535 nop,nop,timestamp 13682342 221364412 .T...P... 1... 21:26:55.838271 IP 82.155.88.187.3485 213.13.146.84.80: . ack 4268305021 win 17021 ..Pei*}P.B}.| 21:26:55.838317 IP 83.174.45.34.59872 213.13.146.84.80: P 3131042263:3131043023(760) ack 12756759 win 65535 nop,nop,timestamp 13682342 221364412 .T...P].. 1..GET /analytics.js HTTP/1.1 Host: wa.sl.pt Pragma: no-cache accept-language: pt ua-os: Windows CE (Smartphone) - Version 5.2 ua-color: color16 x-wap-profile: http://www.htcmms.com.tw/gen/Volans-1.0.xml; ua-voice: TRUE referer: http://auto.sapo.pt/vehicleDetails.aspx -- user-agent: HTC_S730 Mozilla/4.0 (compatible; MSIE 6.0; Windows CE; IEMobile 7.6) x-wsb-contextid: D51E281020EEAF0E accept: application/vnd.wap.mms-message;*/*,*/*;q=0.001 accept-charset: *;q=0.001 accept-encoding: gzip,deflate,*;q=0.001 Max-Forwards: 10 Connection: Keep-Alive X-BlueCoat-Via: 80B23F200A28D3DE ** Another source of problems are requests that have Header names but no Header values, e.g: ** 21:12:36.953721 IP 87.103.25.114.4160 213.13.146.84.80: . ack 3029163034 win 32224 [EMAIL PROTECTED] 21:12:37.060742 IP 89.26.250.104.4602 213.13.146.84.80: S 2765090470:2765090470(0) win 16384 mss 1360,nop,nop,sackOK [EMAIL PROTECTED] 21:12:37.082011 IP 89.26.250.104.4602 213.13.146.84.80: . ack 3407385009 win 17680 .T...PP.E..S 21:12:37.090524 IP 89.26.250.104.4602 213.13.146.84.80: P 2765090471:2765090506(35) ack 3407385009 win 17680 .T...PP.E.GET /robots.txt HTTP/1.0 Host: ** Notice the missing Host Header value. My local fix for these issues was to use bounded collections in HttpHeaderDecodingState to hold Header information and to add extra sanity checks for header values, a better solution will probably fix the issue at the state machine level. -- Luis Neves -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: memory leak in http codec
Niklas Therning wrote: Well, in any way, I think MINA should be able to handle faulty clients like these more gracefully. OOM isn't acceptable. I agree. Please file a JIRA issue and we'll look into it. /Niklas Done... see DIRMINA-505. -- Luis Neves
Re: memory leak in http codec
Niklas Therning wrote: Luis Neves wrote: I still don't know how to properly fixed the decoder but for now this inelegant fix will do. snip/ Well, in any way, I think MINA should be able to handle faulty clients like these more gracefully. OOM isn't acceptable. Please file a JIRA issue and we'll look into it. Just thinking that this problem is not specific to the HttpDecoder. It would be very valuable to be able to handle big chunks of data as streamed data instead as byte[], so that the decoder won't be swamped by huge packets stored in memory. A good strategy could be to stream to disk any packet above a certain size (say, 4 k), until the decoder has finished his work. That means we have to write a streamed decoder : it should not be such a complex task... We need it in ADS, because we can receive massive pieces of data (like pictures : some Mb), and we can't afford to store them in memory until the full load has been received. Note : we know that since months (years ?) but never found time to work on this subject ... :/ /Niklas -- -- cordialement, regards, Emmanuel Lécharny www.iktek.com directory.apache.org
