Taras, On Wed, Jul 25, 2012 at 4:52 PM, Taras <ox...@oxdef.info> wrote: > Andres, > > I have found repeatable scenario. Sometime w3af constructs and tries to > navigate URL like http://example.com?someparam=1 > > Pay attention on absent leading "/" in the path. urllib constructs following > HTTP request: > > GET ?dsfsdf HTTP/1.0 > Host: example.com > User-Agent: Python-urllib/1.17 > > And on such request some web servers answers nothing, some (e.g. nginx) - > without headers and we get statuline like '<html>\r\n'
Interesting! > But! In python >= 2.7 httplib adds this absent "/" at the beginning of the > path if it is absent and everything works fine :) All right, that's good news, > What we can do with it? At least fix xUrllib _incrementGlobalErrorCount() to > process correctly such errors. I see in the code [0]: Hmmm, but do we actually need to do something in the error handling? w3af is now supported under Python 2.7 (based on a recent change we made in dependency_check.py) and xUrllib uses urllib2 which uses -the fixed- httplib. So the error shouldn't happen anymore if the user is running w3af with py27. For users running in py26, maybe we have to back-port the httplib fix or something like that in order to avoid the issue in the first place. > > if type(error) is urllib2.URLError: > ... > if isinstance(reason_err, socket.error): > .... > elif isinstance(reason_err, httplib.HTTPException): > > # Here we catch: > # > # BadStatusLine, ResponseNotReady, CannotSendHeader, > > in same time it doesn't work for badstatusline exception because > type of this exception is httplib.BadStatusLine. Same for _send() method > > > >>> Sometime w3af finishes its work with BadStatusLine error. >>> Problem is I can't figure out stable steps to reproduce this case. >>> What is usually caused it and what we/w3af should do when it get strange >>> HTTP response which urllib can't parse? >> >> >> I've seen this issue a couple of times in w3af, AND also in other >> softwares [0][1], maybe the answers in those references can help you >> fix the issue. Last time I checked, it was because Apache was actually >> omitting the status line for *some* HTTP responses. Two options come >> to my mind: >> >> * If we identify which requests trigger those invalid responses, >> and if the requests are specific enough, we can accept not to send >> them? >> * Create a special treatment for those invalid, out of protocol, >> responses >> >> But before doing anything, it would be nice to understand why Apache >> sends something like that! >> > [0] > http://sourceforge.net/apps/trac/w3af/browser/trunk/core/data/url/xUrllib.py#L709 > > > > -- > Taras > http://oxdef.info > GPG: C8D1F510 -- Andrés Riancho Project Leader at w3af - http://w3af.org/ Web Application Attack and Audit Framework Twitter: @w3af GPG: 0x93C344F3 ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
