Andres,
I have spent more time on this problem and here are some results.
w3af is now supported under Python 2.7 (based on a recent change we
made in dependency_check.py) and xUrllib uses urllib2 which uses -the
fixed- httplib. So the error shouldn't happen anymore if the user is
running w3af with py27.
For users running in py26, maybe we have to back-port the httplib fix
or something like that in order to avoid the issue in the first place.
Back-port fix from 2.7 for malformed URLs is in attachment. In same
time, imho, it is not good idea to ask users for patching Python's libs.
It will mean the ending of Python < 2.7 supporting. Do we have python
version usage statistics for w3af?
> Hmmm, but do we actually need to do something in the error handling?
Another result is...raising BadStatusLine with incorrect path in URL is
only one example. As I think, when we talk about web app fuzzer there is
also possible a huge number of another cases with strange responses from
web server. Currently when w3af gets such strange response (with some
urllib2.URLError exception) it tries to send request again N times
(maxRetrys). If on N attempt w3af also gets strange response it **raises
w3afMustStopOnUrlError** and **stops** the whole scan. Imho, it is too
severely because one request with e.g. 2 retries can stop the whole
scan. Here we need to do same thing as for unexpected errors - increment
global error count. We also need to make errtotal value configurable.
--
Taras
http://oxdef.info
GPG: C8D1F510
--- /usr/lib/python2.6/urllib.py 2010-12-27 02:10:16.000000000 +0300
+++ urllib.py 2012-07-26 11:21:25.000000000 +0400
@@ -1060,7 +1060,13 @@
_hostprog = re.compile('^//([^/?]*)(.*)$')
match = _hostprog.match(url)
- if match: return match.group(1, 2)
+ # Fix mailformed URL
+ if match:
+ host_port = match.group(1)
+ path = match.group(2)
+ if path and not path.startswith('/'):
+ path = '/' + path
+ return host_port, path
return None, url
_userprog = None
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
W3af-develop mailing list
W3af-develop@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/w3af-develop
