Andres,
I have found repeatable scenario. Sometime w3af constructs and tries to
navigate URL like http://example.com?someparam=1
Pay attention on absent leading "/" in the path. urllib constructs
following HTTP request:
GET ?dsfsdf HTTP/1.0
Host: example.com
User-Agent: Python-urllib/1.17
And on such request some web servers answers nothing, some (e.g. nginx)
- without headers and we get statuline like '<html>\r\n'
But! In python >= 2.7 httplib adds this absent "/" at the beginning of
the path if it is absent and everything works fine :)
What we can do with it? At least fix xUrllib
_incrementGlobalErrorCount() to process correctly such errors. I see in
the code [0]:
if type(error) is urllib2.URLError:
...
if isinstance(reason_err, socket.error):
....
elif isinstance(reason_err, httplib.HTTPException):
# Here we catch:
#
# BadStatusLine, ResponseNotReady, CannotSendHeader,
in same time it doesn't work for badstatusline exception because
type of this exception is httplib.BadStatusLine. Same for _send() method
>> Sometime w3af finishes its work with BadStatusLine error.
>> Problem is I can't figure out stable steps to reproduce this case.
>> What is usually caused it and what we/w3af should do when it get strange
>> HTTP response which urllib can't parse?
>
> I've seen this issue a couple of times in w3af, AND also in other
> softwares [0][1], maybe the answers in those references can help you
> fix the issue. Last time I checked, it was because Apache was actually
> omitting the status line for *some* HTTP responses. Two options come
> to my mind:
>
> * If we identify which requests trigger those invalid responses,
> and if the requests are specific enough, we can accept not to send
> them?
> * Create a special treatment for those invalid, out of protocol,
> responses
>
> But before doing anything, it would be nice to understand why Apache
> sends something like that!
>
[0]
http://sourceforge.net/apps/trac/w3af/browser/trunk/core/data/url/xUrllib.py#L709
--
Taras
http://oxdef.info
GPG: C8D1F510
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
W3af-develop mailing list
W3af-develop@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/w3af-develop
