On Mon, Jul 30, 2012 at 5:27 PM, Taras <ox...@oxdef.info> wrote: > Andres, > > >>> Great! Can confirm that it fixes issue with absent "/" :) >> >> >> Perfect, could you please merge it to trunk for me? > > Yes, I will do it. > > >>> Also I have gotten BadStatusLine exception in Python 2.7 with **correct** >>> URL but with HTTP response like: >>> >>> $ curl -v 'http://example.com/goto/?nh=1&path=<ScRIPT>a=/U7fZ/%0A1' >>> * About to connect() to example.com port 80 (#0) >>> * Trying 127.0.0.1... >>> * connected >>> * Connected to example.com (127.0.0.1) port 80 (#0) >>>> >>>> GET /goto/?nh=1&path=<ScRIPT>a=/U7fZ/%0A1 HTTP/1.1 >>>> User-Agent: curl/7.26.0 >>>> Host: example.com >>>> Accept: */* >> >> >> Hmmm, that changes stuff, so the reason for BadStatusLine is not only >> the missing / . > > Exactly! From the server side we can get almost everything, e.g. empty > response. It also may be issue when WAF is used. > > >> Maybe the problem with that request you're sending now >> is that curl is not url-encoding the< and> ? Could you confirm with >> a pcap?. > > In this case it is mostly issue on server side because of special char: > %0A1 > > And we don't need to fix this in request generating part of w3af but we need > to fix error handing. w3ad should not falls down on the of **one** of such > cases. > > >> The bad thing is that in some cases we want to send those >> payloads without url-encoding. > > It is not bad thing because we are talking about web fuzzer. Not about > regular web browser. > > >> If the problem is around encoding those<>, we can fix it, but we >> don't want to because its a feature. > > Exactly! > > >>> So what do you think about such patch? >>> >>> >>> $ diff -u xUrllib.py.old xUrllib.py >>> --- xUrllib.py.old 2012-07-26 16:13:10.000000000 +0400 >>> +++ xUrllib.py 2012-07-26 16:13:25.000000000 +0400 >>> @@ -558,7 +558,11 @@ >>> traceback.format_exc()) >>> req._Request__original = original_url >>> # Then retry! >>> - return self._retry(req, e, cache) >>> + try: >>> + return self._retry(req, e, cache) >>> + except w3afMustStopOnUrlError, e: >>> + self._incrementGlobalErrorCount(e) >>> + return self._new_no_content_resp(original_url_inst, >>> log_it=True) >> >> >> So your fix simply "eliminates the w3afMustStopOnUrlError" and w3af >> will keep trying to scan a host even if it is down? > > **One** empty response (with BadStatusLine) doesn't mean that host is down. > But N such error could stop the scan.
Sure, but the same piece of code handles hosts being down AND BadStatusLine (and many other errors), so if we change the way it works for one without thinking about all the other cases, we're making a big mistake. Error handling in that piece of code should be done with LOTS of care and with deep understanding of what's going on lower and upper levels. I'm not even sure on how to fix the issue without analyzing it for 4 to 6 hours :( We need to fix this, but at this point I don't want to focus on that piece of code, I'm working very strongly on other sections that also require attention. If you want to work on it, which I would love, please take into account all cases. Regards, > -- > Taras > http://oxdef.info > GPG: C8D1F510 -- Andrés Riancho Project Leader at w3af - http://w3af.org/ Web Application Attack and Audit Framework Twitter: @w3af GPG: 0x93C344F3 ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
