Re: [W3af-develop] REST support
Andres, If you don't mind I begin to develop rewritten URLs [0] support with fuzzURLParts and urlRules options inside http-settings. I've made a branch for it [0]. [0] http://en.wikipedia.org/wiki/Rewrite_engine [1] https://w3af.svn.sourceforge.net/svnroot/w3af/branches/rewritten-urls/ 16.11.2011 19:33, Taras пишет: Andres, Andres, when I have suggested this feature in w3af I didn't mean *full* REST specification support. Today a lot of web applications (especially based on frameworks like Django or in the old way by Apache mod_rewrite module) uses REST-like URLs e.g.: http://example.com/foo/bar/123 In this URL we (not scanner) can see such parts as: * foo - controller name * bar - action name * 123 - parameter value From classic web spider point of view it looks like directory hierarchy - it is incorrect behavior! All these parts we need to fuzz! Agreed, so we don't want to support REST, we want to support mod_rewrite. Would that appreciation be correct? I wouldn't use term mod_rewrite as the name of the whole feature because mod_rewrite is simply one of REST URLs cases. In modern web applications which are based on frameworks like Django *internal URL processing* becomes more and more popular. mod_rewrite is web server based URL processing. What I suggest to implement is rules for such URLs. It can be done as http-settings file option called url-rules (name is not important): /top/users/%s/view/%d/ /controller/action/%d/ ... %s and %d are special tokens which can be used by w3af to determine fuzz points. I like the idea, but I would do it in a more configurable way. Right now we have the fuzzFileName setting in w3af, which enabled/disables what I explained in the initial email. In the future I would like to see the following options: * fuzzFileName (default: False) * fuzzDirectories (default: False) * url-rules (default: no filename with rules) With this, when a user wants to fuzz all the directories in all URLs he just enables fuzzDirectories and fuzzFileName. If he wants to have more control over which parts of the URL are actually fuzzed, he can disable the previous ones and set the url-rules himself. Hmmm, interesting idea. But let's call this option something like fuzzURLParts. fuzzDirectories can be misunderstood by user. For the rules file, what I recommend is that we support parsing of mod_rewrite and django rules (if possible) so that a developer can simply copy/paste those rules into a file and point w3af into it. Agree, plus Nginx config In your example you put something like %s and %d. Do we care if it's a string or a digit? Should the scanner change it's behavior based on that? I think it is not so important on the first iteration. We can consider all such tokens as strings. Regards, This email is just a conversation starter for defining how we're going to deal with REST urls. REST, as described in [0], has two important moving parts: 1- URLs that look nice (no parameters: /people/1/phones/23 ) 2- Heavy usage of HTTP methods like GET, POST, DELETE, PUT. The first question that I would ask myself is... do we want to support 1 and 2? Only 1? What is really needed by our users? If we only want to implement #1, it should be easy enough, since we already have something similar (see: mutantFileName.py). This mutant, together with the fuzzer.py (more specifically _createFileNameMutants) will behave like this: - Original URL: http://host.tld/foo/spam-eggs.jsp - Input strings: [ 'scriptalert(1)/script', 'ping localhost'] - Output URLs: * http://host.tld/foo/scriptalert(1)/script-eggs.jsp * http://host.tld/foo/spam-scriptalert(1)/script.jsp * http://host.tld/foo/ping%20localhost-eggs.jsp * http://host.tld/foo/spam-ping%20localhost.jsp As you can see, it will split the filename using any character that's not a letter and put the strings into those positions. If we change this from just the filename into the whole path, it should work and inject into each URL section. Please note that the current implementation only performs file name fuzzing if misc-settings fuzzFileName is enabled (which is off by default). Should we also think about this and potentially modify this to true? Regarding #2 , I don't see a reason for it not to work with w3af... but I could be mistaken. We should perform some tests to check if w3af parses and correctly sends requests associated with forms that use PUT, DELETE, etc. The meta-question here is... do we want w3af to send requests that will DELETE stuff? Ok... that's enough for a conversation starter :) What do you guys think? [0] http://microformats.org/wiki/rest/urls Regards, -- Taras http://oxdef.info -- Taras http://oxdef.info -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance,
Re: [W3af-develop] REST support
Taras, I don't mind :) You're more than welcome to create branches :) Regards, On Fri, Dec 2, 2011 at 12:33 PM, Taras ox...@oxdef.info wrote: Andres, If you don't mind I begin to develop rewritten URLs [0] support with fuzzURLParts and urlRules options inside http-settings. I've made a branch for it [0]. [0] http://en.wikipedia.org/wiki/Rewrite_engine [1] https://w3af.svn.sourceforge.net/svnroot/w3af/branches/rewritten-urls/ 16.11.2011 19:33, Taras пишет: Andres, Andres, when I have suggested this feature in w3af I didn't mean *full* REST specification support. Today a lot of web applications (especially based on frameworks like Django or in the old way by Apache mod_rewrite module) uses REST-like URLs e.g.: http://example.com/foo/bar/123 In this URL we (not scanner) can see such parts as: * foo - controller name * bar - action name * 123 - parameter value From classic web spider point of view it looks like directory hierarchy - it is incorrect behavior! All these parts we need to fuzz! Agreed, so we don't want to support REST, we want to support mod_rewrite. Would that appreciation be correct? I wouldn't use term mod_rewrite as the name of the whole feature because mod_rewrite is simply one of REST URLs cases. In modern web applications which are based on frameworks like Django *internal URL processing* becomes more and more popular. mod_rewrite is web server based URL processing. What I suggest to implement is rules for such URLs. It can be done as http-settings file option called url-rules (name is not important): /top/users/%s/view/%d/ /controller/action/%d/ ... %s and %d are special tokens which can be used by w3af to determine fuzz points. I like the idea, but I would do it in a more configurable way. Right now we have the fuzzFileName setting in w3af, which enabled/disables what I explained in the initial email. In the future I would like to see the following options: * fuzzFileName (default: False) * fuzzDirectories (default: False) * url-rules (default: no filename with rules) With this, when a user wants to fuzz all the directories in all URLs he just enables fuzzDirectories and fuzzFileName. If he wants to have more control over which parts of the URL are actually fuzzed, he can disable the previous ones and set the url-rules himself. Hmmm, interesting idea. But let's call this option something like fuzzURLParts. fuzzDirectories can be misunderstood by user. For the rules file, what I recommend is that we support parsing of mod_rewrite and django rules (if possible) so that a developer can simply copy/paste those rules into a file and point w3af into it. Agree, plus Nginx config In your example you put something like %s and %d. Do we care if it's a string or a digit? Should the scanner change it's behavior based on that? I think it is not so important on the first iteration. We can consider all such tokens as strings. Regards, This email is just a conversation starter for defining how we're going to deal with REST urls. REST, as described in [0], has two important moving parts: 1- URLs that look nice (no parameters: /people/1/phones/23 ) 2- Heavy usage of HTTP methods like GET, POST, DELETE, PUT. The first question that I would ask myself is... do we want to support 1 and 2? Only 1? What is really needed by our users? If we only want to implement #1, it should be easy enough, since we already have something similar (see: mutantFileName.py). This mutant, together with the fuzzer.py (more specifically _createFileNameMutants) will behave like this: - Original URL: http://host.tld/foo/spam-eggs.jsp - Input strings: [ 'scriptalert(1)/script', 'ping localhost'] - Output URLs: * http://host.tld/foo/scriptalert(1)/script-eggs.jsp * http://host.tld/foo/spam-scriptalert(1)/script.jsp * http://host.tld/foo/ping%20localhost-eggs.jsp * http://host.tld/foo/spam-ping%20localhost.jsp As you can see, it will split the filename using any character that's not a letter and put the strings into those positions. If we change this from just the filename into the whole path, it should work and inject into each URL section. Please note that the current implementation only performs file name fuzzing if misc-settings fuzzFileName is enabled (which is off by default). Should we also think about this and potentially modify this to true? Regarding #2 , I don't see a reason for it not to work with w3af... but I could be mistaken. We should perform some tests to check if w3af parses and correctly sends requests associated with forms that use PUT, DELETE, etc. The meta-question here is... do we want w3af to send requests that will DELETE stuff? Ok... that's enough for a conversation starter :) What do you guys think? [0] http://microformats.org/wiki/rest/urls Regards, -- Taras http://oxdef.info -- Taras http://oxdef.info -- Andrés Riancho Director of Web Security at
Re: [W3af-develop] REST support
Am 16.11.2011 15:37, schrieb Andres Riancho: Achim, On Tue, Nov 15, 2011 at 10:34 AM, Achim Hoffmann webse...@sic-sec.org wrote: Hi all, I fully agree with Taras. Question before I dig deeper: does w3af currently identify (correctly) which parts of the URL are the INFO_PATH (actually tartofdefence.com h/bar/123 part, see below)? What do you mean by INFO_PATH? Achim http://www.ietf.org/rfc/rfc3875 If you have an URL like: http://some.where/path/to/script/additional/info where /path/to/script is the script executed on the server, anything right to the scriptname and left to the ? is the PATH_INFO which would be /additional/info in my example. The usage of PATH_INFO and PATH_TRANSLATED was traditionaly a path (nomen est omen), but the script is free to use it as whatever it likes. In modern applications we often see it used as parameters passed to the script. Means that parameters ar in the PATH_INFO and not the QUERY_STRING (as Taras already explained). Achim -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d ___ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
Re: [W3af-develop] REST support
Taras, On Tue, Nov 15, 2011 at 10:25 AM, Taras ox...@oxdef.info wrote: Hi, all! Andres, when I have suggested this feature in w3af I didn't mean *full* REST specification support. Today a lot of web applications (especially based on frameworks like Django or in the old way by Apache mod_rewrite module) uses REST-like URLs e.g.: http://example.com/foo/bar/123 In this URL we (not scanner) can see such parts as: * foo - controller name * bar - action name * 123 - parameter value From classic web spider point of view it looks like directory hierarchy - it is incorrect behavior! All these parts we need to fuzz! Agreed, so we don't want to support REST, we want to support mod_rewrite. Would that appreciation be correct? What I suggest to implement is rules for such URLs. It can be done as http-settings file option called url-rules (name is not important): /top/users/%s/view/%d/ /controller/action/%d/ ... %s and %d are special tokens which can be used by w3af to determine fuzz points. I like the idea, but I would do it in a more configurable way. Right now we have the fuzzFileName setting in w3af, which enabled/disables what I explained in the initial email. In the future I would like to see the following options: * fuzzFileName (default: False) * fuzzDirectories (default: False) * url-rules (default: no filename with rules) With this, when a user wants to fuzz all the directories in all URLs he just enables fuzzDirectories and fuzzFileName. If he wants to have more control over which parts of the URL are actually fuzzed, he can disable the previous ones and set the url-rules himself. For the rules file, what I recommend is that we support parsing of mod_rewrite and django rules (if possible) so that a developer can simply copy/paste those rules into a file and point w3af into it. In your example you put something like %s and %d. Do we care if it's a string or a digit? Should the scanner change it's behavior based on that? Regards, This email is just a conversation starter for defining how we're going to deal with REST urls. REST, as described in [0], has two important moving parts: 1- URLs that look nice (no parameters: /people/1/phones/23 ) 2- Heavy usage of HTTP methods like GET, POST, DELETE, PUT. The first question that I would ask myself is... do we want to support 1 and 2? Only 1? What is really needed by our users? If we only want to implement #1, it should be easy enough, since we already have something similar (see: mutantFileName.py). This mutant, together with the fuzzer.py (more specifically _createFileNameMutants) will behave like this: - Original URL: http://host.tld/foo/spam-eggs.jsp - Input strings: [ 'scriptalert(1)/script', 'ping localhost'] - Output URLs: * http://host.tld/foo/scriptalert(1)/script-eggs.jsp * http://host.tld/foo/spam-scriptalert(1)/script.jsp * http://host.tld/foo/ping%20localhost-eggs.jsp * http://host.tld/foo/spam-ping%20localhost.jsp As you can see, it will split the filename using any character that's not a letter and put the strings into those positions. If we change this from just the filename into the whole path, it should work and inject into each URL section. Please note that the current implementation only performs file name fuzzing if misc-settings fuzzFileName is enabled (which is off by default). Should we also think about this and potentially modify this to true? Regarding #2 , I don't see a reason for it not to work with w3af... but I could be mistaken. We should perform some tests to check if w3af parses and correctly sends requests associated with forms that use PUT, DELETE, etc. The meta-question here is... do we want w3af to send requests that will DELETE stuff? Ok... that's enough for a conversation starter :) What do you guys think? [0] http://microformats.org/wiki/rest/urls Regards, -- Taras http://oxdef.info -- Andrés Riancho Director of Web Security at Rapid7 LLC Founder at Bonsai Information Security Project Leader at w3af -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
Re: [W3af-develop] REST support
Achim, On Tue, Nov 15, 2011 at 10:34 AM, Achim Hoffmann webse...@sic-sec.org wrote: Hi all, I fully agree with Taras. Question before I dig deeper: does w3af currently identify (correctly) which parts of the URL are the INFO_PATH (actually tartofdefence.com h/bar/123 part, see below)? What do you mean by INFO_PATH? Achim Am 15.11.2011 14:25, schrieb Taras: Hi, all! Andres, when I have suggested this feature in w3af I didn't mean *full* REST specification support. Today a lot of web applications (especially based on frameworks like Django or in the old way by Apache mod_rewrite module) uses REST-like URLs e.g.: http://example.com/foo/bar/123 In this URL we (not scanner) can see such parts as: * foo - controller name * bar - action name * 123 - parameter value From classic web spider point of view it looks like directory hierarchy - it is incorrect behavior! All these parts we need to fuzz! What I suggest to implement is rules for such URLs. It can be done as http-settings file option called url-rules (name is not important): /top/users/%s/view/%d/ /controller/action/%d/ ... %s and %d are special tokens which can be used by w3af to determine fuzz points. This email is just a conversation starter for defining how we're going to deal with REST urls. REST, as described in [0], has two important moving parts: 1- URLs that look nice (no parameters: /people/1/phones/23 ) 2- Heavy usage of HTTP methods like GET, POST, DELETE, PUT. The first question that I would ask myself is... do we want to support 1 and 2? Only 1? What is really needed by our users? If we only want to implement #1, it should be easy enough, since we already have something similar (see: mutantFileName.py). This mutant, together with the fuzzer.py (more specifically _createFileNameMutants) will behave like this: - Original URL: http://host.tld/foo/spam-eggs.jsp - Input strings: [ 'scriptalert(1)/script', 'ping localhost'] - Output URLs: * http://host.tld/foo/scriptalert(1)/script-eggs.jsp * http://host.tld/foo/spam-scriptalert(1)/script.jsp * http://host.tld/foo/ping%20localhost-eggs.jsp * http://host.tld/foo/spam-ping%20localhost.jsp As you can see, it will split the filename using any character that's not a letter and put the strings into those positions. If we change this from just the filename into the whole path, it should work and inject into each URL section. Please note that the current implementation only performs file name fuzzing if misc-settings fuzzFileName is enabled (which is off by default). Should we also think about this and potentially modify this to true? Regarding #2 , I don't see a reason for it not to work with w3af... but I could be mistaken. We should perform some tests to check if w3af parses and correctly sends requests associated with forms that use PUT, DELETE, etc. The meta-question here is... do we want w3af to send requests that will DELETE stuff? Ok... that's enough for a conversation starter :) What do you guys think? [0] http://microformats.org/wiki/rest/urls Regards, -- Andrés Riancho Director of Web Security at Rapid7 LLC Founder at Bonsai Information Security Project Leader at w3af -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
Re: [W3af-develop] REST support
Andres, Andres, when I have suggested this feature in w3af I didn't mean *full* REST specification support. Today a lot of web applications (especially based on frameworks like Django or in the old way by Apache mod_rewrite module) uses REST-like URLs e.g.: http://example.com/foo/bar/123 In this URL we (not scanner) can see such parts as: * foo - controller name * bar - action name * 123 - parameter value From classic web spider point of view it looks like directory hierarchy - it is incorrect behavior! All these parts we need to fuzz! Agreed, so we don't want to support REST, we want to support mod_rewrite. Would that appreciation be correct? I wouldn't use term mod_rewrite as the name of the whole feature because mod_rewrite is simply one of REST URLs cases. In modern web applications which are based on frameworks like Django *internal URL processing* becomes more and more popular. mod_rewrite is web server based URL processing. What I suggest to implement is rules for such URLs. It can be done as http-settings file option called url-rules (name is not important): /top/users/%s/view/%d/ /controller/action/%d/ ... %s and %d are special tokens which can be used by w3af to determine fuzz points. I like the idea, but I would do it in a more configurable way. Right now we have the fuzzFileName setting in w3af, which enabled/disables what I explained in the initial email. In the future I would like to see the following options: * fuzzFileName (default: False) * fuzzDirectories (default: False) * url-rules (default: no filename with rules) With this, when a user wants to fuzz all the directories in all URLs he just enables fuzzDirectories and fuzzFileName. If he wants to have more control over which parts of the URL are actually fuzzed, he can disable the previous ones and set the url-rules himself. Hmmm, interesting idea. But let's call this option something like fuzzURLParts. fuzzDirectories can be misunderstood by user. For the rules file, what I recommend is that we support parsing of mod_rewrite and django rules (if possible) so that a developer can simply copy/paste those rules into a file and point w3af into it. Agree, plus Nginx config In your example you put something like %s and %d. Do we care if it's a string or a digit? Should the scanner change it's behavior based on that? I think it is not so important on the first iteration. We can consider all such tokens as strings. Regards, This email is just a conversation starter for defining how we're going to deal with REST urls. REST, as described in [0], has two important moving parts: 1- URLs that look nice (no parameters: /people/1/phones/23 ) 2- Heavy usage of HTTP methods like GET, POST, DELETE, PUT. The first question that I would ask myself is... do we want to support 1 and 2? Only 1? What is really needed by our users? If we only want to implement #1, it should be easy enough, since we already have something similar (see: mutantFileName.py). This mutant, together with the fuzzer.py (more specifically _createFileNameMutants) will behave like this: - Original URL: http://host.tld/foo/spam-eggs.jsp - Input strings: [ 'scriptalert(1)/script', 'ping localhost'] - Output URLs: * http://host.tld/foo/scriptalert(1)/script-eggs.jsp * http://host.tld/foo/spam-scriptalert(1)/script.jsp * http://host.tld/foo/ping%20localhost-eggs.jsp * http://host.tld/foo/spam-ping%20localhost.jsp As you can see, it will split the filename using any character that's not a letter and put the strings into those positions. If we change this from just the filename into the whole path, it should work and inject into each URL section. Please note that the current implementation only performs file name fuzzing if misc-settings fuzzFileName is enabled (which is off by default). Should we also think about this and potentially modify this to true? Regarding #2 , I don't see a reason for it not to work with w3af... but I could be mistaken. We should perform some tests to check if w3af parses and correctly sends requests associated with forms that use PUT, DELETE, etc. The meta-question here is... do we want w3af to send requests that will DELETE stuff? Ok... that's enough for a conversation starter :) What do you guys think? [0] http://microformats.org/wiki/rest/urls Regards, -- Taras http://oxdef.info -- Taras http://oxdef.info -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d
Re: [W3af-develop] REST support
Taras, On Wed, Nov 16, 2011 at 12:33 PM, Taras ox...@oxdef.info wrote: Andres, Andres, when I have suggested this feature in w3af I didn't mean *full* REST specification support. Today a lot of web applications (especially based on frameworks like Django or in the old way by Apache mod_rewrite module) uses REST-like URLs e.g.: http://example.com/foo/bar/123 In this URL we (not scanner) can see such parts as: * foo - controller name * bar - action name * 123 - parameter value From classic web spider point of view it looks like directory hierarchy - it is incorrect behavior! All these parts we need to fuzz! Agreed, so we don't want to support REST, we want to support mod_rewrite. Would that appreciation be correct? I wouldn't use term mod_rewrite as the name of the whole feature because mod_rewrite is simply one of REST URLs cases. In modern web applications which are based on frameworks like Django *internal URL processing* becomes more and more popular. mod_rewrite is web server based URL processing. Agreed. What I suggest to implement is rules for such URLs. It can be done as http-settings file option called url-rules (name is not important): /top/users/%s/view/%d/ /controller/action/%d/ ... %s and %d are special tokens which can be used by w3af to determine fuzz points. I like the idea, but I would do it in a more configurable way. Right now we have the fuzzFileName setting in w3af, which enabled/disables what I explained in the initial email. In the future I would like to see the following options: * fuzzFileName (default: False) * fuzzDirectories (default: False) * url-rules (default: no filename with rules) With this, when a user wants to fuzz all the directories in all URLs he just enables fuzzDirectories and fuzzFileName. If he wants to have more control over which parts of the URL are actually fuzzed, he can disable the previous ones and set the url-rules himself. Hmmm, interesting idea. But let's call this option something like fuzzURLParts. fuzzDirectories can be misunderstood by user. Agreed on the name :) For the rules file, what I recommend is that we support parsing of mod_rewrite and django rules (if possible) so that a developer can simply copy/paste those rules into a file and point w3af into it. Agree, plus Nginx config Great! In your example you put something like %s and %d. Do we care if it's a string or a digit? Should the scanner change it's behavior based on that? I think it is not so important on the first iteration. We can consider all such tokens as strings. Exactly Regards, This email is just a conversation starter for defining how we're going to deal with REST urls. REST, as described in [0], has two important moving parts: 1- URLs that look nice (no parameters: /people/1/phones/23 ) 2- Heavy usage of HTTP methods like GET, POST, DELETE, PUT. The first question that I would ask myself is... do we want to support 1 and 2? Only 1? What is really needed by our users? If we only want to implement #1, it should be easy enough, since we already have something similar (see: mutantFileName.py). This mutant, together with the fuzzer.py (more specifically _createFileNameMutants) will behave like this: - Original URL: http://host.tld/foo/spam-eggs.jsp - Input strings: [ 'scriptalert(1)/script', 'ping localhost'] - Output URLs: * http://host.tld/foo/scriptalert(1)/script-eggs.jsp * http://host.tld/foo/spam-scriptalert(1)/script.jsp * http://host.tld/foo/ping%20localhost-eggs.jsp * http://host.tld/foo/spam-ping%20localhost.jsp As you can see, it will split the filename using any character that's not a letter and put the strings into those positions. If we change this from just the filename into the whole path, it should work and inject into each URL section. Please note that the current implementation only performs file name fuzzing if misc-settings fuzzFileName is enabled (which is off by default). Should we also think about this and potentially modify this to true? Regarding #2 , I don't see a reason for it not to work with w3af... but I could be mistaken. We should perform some tests to check if w3af parses and correctly sends requests associated with forms that use PUT, DELETE, etc. The meta-question here is... do we want w3af to send requests that will DELETE stuff? Ok... that's enough for a conversation starter :) What do you guys think? [0] http://microformats.org/wiki/rest/urls Regards, -- Taras http://oxdef.info -- Taras http://oxdef.info -- Andrés Riancho Director of Web Security at Rapid7 LLC Founder at Bonsai Information Security Project Leader at w3af -- All the data continuously generated in your IT infrastructure contains a
Re: [W3af-develop] REST support
Andres, - Regarding the discovery method, actually a REST API es pretty unique, maybe sending the Accept apliccation/json header and check for a positive answer is a good place to start. Well coded APIs accept the OPTIONS method to describe their behavior and their resources, vital info could be grabbed from there. When an authenticated schema is implemented is not that easy. Resources are very particular for each API, mapping them is not easy.asnavigating a directory schema on a webserver, usage of that resources are very special in each case, you can face things like curl -X GET http://api.servercloud.com/servers; and curl -X GET http://api.othercloud.com/?servers=* , of course someone is violating REST standards but i see this all the time. But we have to face that now everything is presented as a service to be consumed via an API . So i think this is a very nice to have feature on w3af Regards Lean On Tue, Nov 15, 2011 at 8:33 AM, Andres Riancho andres.rian...@gmail.comwrote: List, This email is just a conversation starter for defining how we're going to deal with REST urls. REST, as described in [0], has two important moving parts: 1- URLs that look nice (no parameters: /people/1/phones/23 ) 2- Heavy usage of HTTP methods like GET, POST, DELETE, PUT. The first question that I would ask myself is... do we want to support 1 and 2? Only 1? What is really needed by our users? If we only want to implement #1, it should be easy enough, since we already have something similar (see: mutantFileName.py). This mutant, together with the fuzzer.py (more specifically _createFileNameMutants) will behave like this: - Original URL: http://host.tld/foo/spam-eggs.jsp - Input strings: [ 'scriptalert(1)/script', 'ping localhost'] - Output URLs: * http://host.tld/foo/scriptalert(1)/script-eggs.jsp * http://host.tld/foo/spam-scriptalert(1)/script.jsp * http://host.tld/foo/ping%20localhost-eggs.jsp * http://host.tld/foo/spam-ping%20localhost.jsp As you can see, it will split the filename using any character that's not a letter and put the strings into those positions. If we change this from just the filename into the whole path, it should work and inject into each URL section. Please note that the current implementation only performs file name fuzzing if misc-settings fuzzFileName is enabled (which is off by default). Should we also think about this and potentially modify this to true? Regarding #2 , I don't see a reason for it not to work with w3af... but I could be mistaken. We should perform some tests to check if w3af parses and correctly sends requests associated with forms that use PUT, DELETE, etc. The meta-question here is... do we want w3af to send requests that will DELETE stuff? Ok... that's enough for a conversation starter :) What do you guys think? [0] http://microformats.org/wiki/rest/urls Regards, -- Andrés Riancho Director of Web Security at Rapid7 LLC Founder at Bonsai Information Security Project Leader at w3af -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1___ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
Re: [W3af-develop] REST support
Andrés, REST, as described in [0], has two important moving parts: ... 2- Heavy usage of HTTP methods like GET, POST, DELETE, PUT. IMHO testing and/or fuzzing HTTP methods is independent of REST. If fuzzing methods will be a feature, then there're more methods to be tested, like: get post JEFF CATS TRAC TRACK GET http:// (if not already implemented) GET%08/ GET%09/ OPTIONS * HTTP/1.0 OPTIONS / HTTP/1.0 -- GET / HTTP/1.0 Content-Length: 17 key=valuepar=val -- Note: being pedantic, this is not web application testing, but web or application server (capabilities) testing. Achim -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
Re: [W3af-develop] REST support
Hi, all! Andres, when I have suggested this feature in w3af I didn't mean *full* REST specification support. Today a lot of web applications (especially based on frameworks like Django or in the old way by Apache mod_rewrite module) uses REST-like URLs e.g.: http://example.com/foo/bar/123 In this URL we (not scanner) can see such parts as: * foo - controller name * bar - action name * 123 - parameter value From classic web spider point of view it looks like directory hierarchy - it is incorrect behavior! All these parts we need to fuzz! What I suggest to implement is rules for such URLs. It can be done as http-settings file option called url-rules (name is not important): /top/users/%s/view/%d/ /controller/action/%d/ ... %s and %d are special tokens which can be used by w3af to determine fuzz points. This email is just a conversation starter for defining how we're going to deal with REST urls. REST, as described in [0], has two important moving parts: 1- URLs that look nice (no parameters: /people/1/phones/23 ) 2- Heavy usage of HTTP methods like GET, POST, DELETE, PUT. The first question that I would ask myself is... do we want to support 1 and 2? Only 1? What is really needed by our users? If we only want to implement #1, it should be easy enough, since we already have something similar (see: mutantFileName.py). This mutant, together with the fuzzer.py (more specifically _createFileNameMutants) will behave like this: - Original URL: http://host.tld/foo/spam-eggs.jsp - Input strings: [ 'scriptalert(1)/script', 'ping localhost'] - Output URLs: * http://host.tld/foo/scriptalert(1)/script-eggs.jsp * http://host.tld/foo/spam-scriptalert(1)/script.jsp * http://host.tld/foo/ping%20localhost-eggs.jsp * http://host.tld/foo/spam-ping%20localhost.jsp As you can see, it will split the filename using any character that's not a letter and put the strings into those positions. If we change this from just the filename into the whole path, it should work and inject into each URL section. Please note that the current implementation only performs file name fuzzing if misc-settings fuzzFileName is enabled (which is off by default). Should we also think about this and potentially modify this to true? Regarding #2 , I don't see a reason for it not to work with w3af... but I could be mistaken. We should perform some tests to check if w3af parses and correctly sends requests associated with forms that use PUT, DELETE, etc. The meta-question here is... do we want w3af to send requests that will DELETE stuff? Ok... that's enough for a conversation starter :) What do you guys think? [0] http://microformats.org/wiki/rest/urls Regards, -- Taras http://oxdef.info -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
Re: [W3af-develop] REST support
Hi all, I fully agree with Taras. Question before I dig deeper: does w3af currently identify (correctly) which parts of the URL are the INFO_PATH (actually tartofdefence.com h/bar/123 part, see below)? Achim Am 15.11.2011 14:25, schrieb Taras: Hi, all! Andres, when I have suggested this feature in w3af I didn't mean *full* REST specification support. Today a lot of web applications (especially based on frameworks like Django or in the old way by Apache mod_rewrite module) uses REST-like URLs e.g.: http://example.com/foo/bar/123 In this URL we (not scanner) can see such parts as: * foo - controller name * bar - action name * 123 - parameter value From classic web spider point of view it looks like directory hierarchy - it is incorrect behavior! All these parts we need to fuzz! What I suggest to implement is rules for such URLs. It can be done as http-settings file option called url-rules (name is not important): /top/users/%s/view/%d/ /controller/action/%d/ ... %s and %d are special tokens which can be used by w3af to determine fuzz points. This email is just a conversation starter for defining how we're going to deal with REST urls. REST, as described in [0], has two important moving parts: 1- URLs that look nice (no parameters: /people/1/phones/23 ) 2- Heavy usage of HTTP methods like GET, POST, DELETE, PUT. The first question that I would ask myself is... do we want to support 1 and 2? Only 1? What is really needed by our users? If we only want to implement #1, it should be easy enough, since we already have something similar (see: mutantFileName.py). This mutant, together with the fuzzer.py (more specifically _createFileNameMutants) will behave like this: - Original URL: http://host.tld/foo/spam-eggs.jsp - Input strings: [ 'scriptalert(1)/script', 'ping localhost'] - Output URLs: * http://host.tld/foo/scriptalert(1)/script-eggs.jsp * http://host.tld/foo/spam-scriptalert(1)/script.jsp * http://host.tld/foo/ping%20localhost-eggs.jsp * http://host.tld/foo/spam-ping%20localhost.jsp As you can see, it will split the filename using any character that's not a letter and put the strings into those positions. If we change this from just the filename into the whole path, it should work and inject into each URL section. Please note that the current implementation only performs file name fuzzing if misc-settings fuzzFileName is enabled (which is off by default). Should we also think about this and potentially modify this to true? Regarding #2 , I don't see a reason for it not to work with w3af... but I could be mistaken. We should perform some tests to check if w3af parses and correctly sends requests associated with forms that use PUT, DELETE, etc. The meta-question here is... do we want w3af to send requests that will DELETE stuff? Ok... that's enough for a conversation starter :) What do you guys think? [0] http://microformats.org/wiki/rest/urls Regards, -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
