Andres, ping! :) >>>> Hmmm, but do we actually need to do something in the error handling? >> >> After some thinking, I came up with a solution that should solve the >> initial issue you pointed out: "the lack of /". For now and because >> I'm not sure if this fixes all issues (there might be more than one >> source of BadStatusLine) I commited it to my branch [0] , please >> review so I can merge into trunk. To test, please move this change to >> a local trunk, the threading2 branch isn't done yet. >> >> [0] http://sourceforge.net/apps/trac/w3af/changeset/5409 > > Great! Can confirm that it fixes issue with absent "/" :) > >>> Another result is...raising BadStatusLine with incorrect path in URL is only >>> one example. As I think, when we talk about web app fuzzer there is also >>> possible a huge number of another cases with strange responses from web >>> server. Currently when w3af gets such strange response (with some >>> urllib2.URLError exception) it tries to send request again N times >>> (maxRetrys). If on N attempt w3af also gets strange response it **raises >>> w3afMustStopOnUrlError** and **stops** the whole scan. Imho, it is too >>> severely because one request with e.g. 2 retries can stop the whole scan. >>> Here we need to do same thing as for unexpected errors - increment global >>> error count. We also need to make errtotal value configurable. >> >> Yes, that's an issue we have in the error handling that should be >> fixed, but I'm not sure if elevating the maxRetrys is the way. > > Why? What is difference between BadStatusLine exception and others? > Also I have gotten BadStatusLine exception in Python 2.7 with > **correct** URL but with HTTP response like: > > $ curl -v 'http://example.com/goto/?nh=1&path=<ScRIPT>a=/U7fZ/%0A1' > * About to connect() to example.com port 80 (#0) > * Trying 127.0.0.1... > * connected > * Connected to example.com (127.0.0.1) port 80 (#0) > > GET /goto/?nh=1&path=<ScRIPT>a=/U7fZ/%0A1 HTTP/1.1 > > User-Agent: curl/7.26.0 > > Host: example.com > > Accept: */* > > > * additional stuff not fine transfer.c:1037: 0 0 > * Empty reply from server > * Connection #0 to host example.com left intact > curl: (52) Empty reply from server > * Closing connection #0 > > And we can't fix such cases with correcting URL which are generated with > w3af. So what do you think about such patch? > > $ diff -u xUrllib.py.old xUrllib.py > --- xUrllib.py.old 2012-07-26 16:13:10.000000000 +0400 > +++ xUrllib.py 2012-07-26 16:13:25.000000000 +0400 > @@ -558,7 +558,11 @@ > traceback.format_exc()) > req._Request__original = original_url > # Then retry! > - return self._retry(req, e, cache) > + try: > + return self._retry(req, e, cache) > + except w3afMustStopOnUrlError, e: > + self._incrementGlobalErrorCount(e) > + return self._new_no_content_resp(original_url_inst, > log_it=True) > except KeyboardInterrupt: > # Correct control+c handling... > raise >
-- Taras http://oxdef.info GPG: C8D1F510 ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
