Re: [W3af-develop] w3af: Is there any API for the webSpider plugin?
seemanta, On Fri, May 29, 2009 at 10:09 AM, Andres Riancho wrote: > seemanta, > > On Fri, May 29, 2009 at 9:37 AM, seemanta patnaik > wrote: >> Hello Everybody, >> >> I have a requirement to crawl a website from my base existing >> software(program) and to push the output into an xml file. Please let me >> know if there is any API on w3af which can be called from my base program >> for crawling a website. > > Nop, > >> Alternatively I wrote the following batch script & called it from my base >> program (platform- w3af version 1.0-rc2 on Windows XP Professional OS):- >> >> plugins >> discovery webSpider >> output console, textFile, xmlFile >> output config console >> set verbose True >> back >> output config textFile >> set verbose True >> set fileName result2.txt >> back >> output config xmlFile >> set fileName result2.xml >> back >> back >> target >> set target >> back >> start >> >> On executing the above w3af script it was observed that both textFile >> (result2.txt) & xmlFile (result2.xml) are created. However result2.xml >> contains no data while textFile contains the required output. I have already >> tried various options but still facing the same problem. > > Are you waiting until the end of the w3af process? I think that the > xml file is created in memory, and then written all at the end. > >> >> >> Please let me if i am missing something in the script. > > I don't think so, Ahh, also... the xml only records vulnerabilities, (as far as I can tell from a small run). If you want it to record the found URLs, you have to modify the plugin code. Also an option, without modifying the code but not in XML is to enable the "exportFuzzableRequests" in "misc-settings". >> >> >> Thanks, >> >> Seemanta >> >> -- >> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT >> is a gathering of tech-side developers & brand creativity professionals. >> Meet >> the minds behind Google Creative Lab, Visual Complexity, Processing, & >> iPhoneDevCamp as they present alongside digital heavyweights like Barbarian >> Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com >> ___ >> W3af-develop mailing list >> W3af-develop@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/w3af-develop >> >> > > > > -- > Andrés Riancho > Founder, Bonsai - Information Security > http://www.bonsai-sec.com/ > http://w3af.sf.net/ > -- Andrés Riancho Founder, Bonsai - Information Security http://www.bonsai-sec.com/ http://w3af.sf.net/ -- Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com ___ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
Re: [W3af-develop] w3af: Is there any API for the webSpider plugin?
seemanta, On Fri, May 29, 2009 at 9:37 AM, seemanta patnaik wrote: > Hello Everybody, > > I have a requirement to crawl a website from my base existing > software(program) and to push the output into an xml file. Please let me > know if there is any API on w3af which can be called from my base program > for crawling a website. Nop, > Alternatively I wrote the following batch script & called it from my base > program (platform- w3af version 1.0-rc2 on Windows XP Professional OS):- > > plugins > discovery webSpider > output console, textFile, xmlFile > output config console > set verbose True > back > output config textFile > set verbose True > set fileName result2.txt > back > output config xmlFile > set fileName result2.xml > back > back > target > set target > back > start > > On executing the above w3af script it was observed that both textFile > (result2.txt) & xmlFile (result2.xml) are created. However result2.xml > contains no data while textFile contains the required output. I have already > tried various options but still facing the same problem. Are you waiting until the end of the w3af process? I think that the xml file is created in memory, and then written all at the end. > > > Please let me if i am missing something in the script. I don't think so, > > > Thanks, > > Seemanta > > -- > Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT > is a gathering of tech-side developers & brand creativity professionals. > Meet > the minds behind Google Creative Lab, Visual Complexity, Processing, & > iPhoneDevCamp as they present alongside digital heavyweights like Barbarian > Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com > ___ > W3af-develop mailing list > W3af-develop@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/w3af-develop > > -- Andrés Riancho Founder, Bonsai - Information Security http://www.bonsai-sec.com/ http://w3af.sf.net/ -- Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com ___ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
Re: [W3af-develop] Wordpress version discovery plugin
Ryan,
On Thu, May 28, 2009 at 10:11 PM, Ryan Dewhurst wrote:
> Im loooking into searching the response html of the index page for the
> following string:
>
>
> Ive tried with regular expressions and am unable to get it to work,
backbone sent you a solution,
> Ive read that re is bad for parsing HTML and that BeautifulSoup
> should be used.
>
> Does w3af already have BeautifulSoup in its dependency list?
Yes, it's in the dependency list, but we aren't using it "for that".
Long story short, please use the re =)
> Ryan
>
> P.S. Thanks for the advice backbone46, I'll have a look into that once
> Ive sorted this out.
>
>
> 2009/5/28 :
>> Sorry to bump in just like that in the discussion, about the meta tag that
>> displays
>> the WordPress version.
>>
>> Only since version 2.7 the generator function is in the core of WordPress,
>> on
>> earlier versions it was only in the theme.
>>
>> Just wanted to mention that. :)
>>
>> ---
>> http://insanesecurity.info
>>
>>
>> On Thu, May 28, 2009 at 10:53 PM, Ryan Dewhurst
>> wrote:
>>>
>>> Yes, I dont see why not. Should be easy enough tro implement.
>>>
>>> You mentioned during our email conversation that wordpress echos its
>>> version number in the page head. I managed to find an example of it.
>>> Your right I do have a security plugin installed which must have
>>> removed it from my blog.
>>>
>>> Here is an example:
>>>
>>>
>>>
>>> 2009/5/28 Andres Riancho :
>>> > Ryan,
>>> >
>>> > On Wed, May 27, 2009 at 10:18 PM, Andres Riancho
>>> > wrote:
>>> >> Ryan,
>>> >>
>>> >> On Wed, May 27, 2009 at 9:58 PM, Ryan Dewhurst
>>> >> wrote:
>>> >>> Hello,
>>> >>> Im new to mailing lists so im not sure if this will be sent there.
>>> >>
>>> >> It depends on the mailing list. This one is configured to accept
>>> >> attachments,
>>> >>
>>> >>> I'll have a look into intergrating the script into w3af over the next
>>> >>> couple of days and hopefully have a working version by the weekend.
>>> >>
>>> >> Excellent, if you need ANY help, just let us know.
>>> >>
>>> >>> The script is quite simple once you have the gathered the nesesary
>>> >>> data. I went through versions 2.2 to 2.7.1 and manually found client
>>> >>> side differences in most of them, I also used the official changelogs
>>> >>> to help identify them.
>>> >>
>>> >> Ohhh, you are the guy that wrote that blog post with the "diffs" of
>>> >> different wordpress release packages?
>>> >>
>>> >>> The client side differences are in files such as CSS, javascript and
>>> >>> HTML. Some versions did not have any differences apart from having
>>> >>> extra files, which can easliy be identified with HTTP response codes.
>>> >>>
>>> >>> It works as such...
>>> >>>
>>> >>> Starting from version 2.7.1 (latest), the script tries to find
>>> >>> something that 2.7 doesnt have, if it finds that something then the
>>> >>> script stops and echos the version number.
>>> >>>
>>> >>> If the script doesnt find the difference it moves onto identifying the
>>> >>> next version, i.e. does 2.7 have something the earlier version doesnt
>>> >>> have. and so on and so forth.
>>> >>
>>> >> Ok, makes sense.
>>> >>
>>> >> Some comments regarding your code:
>>> >>
>>> >> - w3af uses PEP-8, with among other things says 4-spaces for
>>> >> indentations. Your code has 1-space (?) indentations. Please correct
>>> >> that.
>>> >>
>>> >> - The code is pretty simple, but i think it could be done in a better
>>> >> way. Having that many functions (wp22 to wp271) doesn't seem to be a
>>> >> good option. Do you think that the code could be changed a little bit,
>>> >> and create a database (which can be easily updated) and then use that
>>> >> database to store the information? Example of the databse
>>> >>
>>> >> self._wp_fingerprint =
>>> >>
>>> >> [('/wp-includes/js/thickbox/thickbox.css','-ms-filter:'),('/wp-admin/css/farbtastic.css',
>>> >> 'farbtastic')]
>>> >>
>>> >> - Also, by default wordpress publishes the version number in every
>>> >> page head. Maybe it would be a good idea to parse that, and compare it
>>> >> with the result of the fingerprinting. What do you think?
>>> >
>>> > A good idea would be to have a first step, before all the version
>>> > specific checks, that verifies something that's true for all wordpress
>>> > installations (some X file has to be present) before even starting the
>>> > fingerprinting. Could this be done?
>>> >
>>> >> Cheers,
>>> >>
>>> >>> Ryan
>>> >>>
>>> >>>
>>> >>> 2009/5/28 Andres Riancho :
>>> Ryan,
>>>
>>> On Wed, May 27, 2009 at 5:07 PM, Ryan Dewhurst
>>> wrote:
>>> > Hello,
>>> > I have developed a python script that can detect the version of a
>>> > wordpress installation. I think it would fit well within w3af,
>>>
>>> Yes, it seems that it's something good to have in the framework.
>>>
>>> I have like a ton of questions about how it works, could you please
>>> send the script (as it is) to this mailing list for us to read it?
>>>
>>> >
[W3af-develop] w3af: Is there any API for the webSpider plugin?
Hello Everybody, I have a requirement to crawl a website from my base existing software(program) and to push the output into an xml file. Please let me know if there is any API on w3af which can be called from my base program for crawling a website. Alternatively I wrote the following batch script & called it from my base program (platform- w3af version 1.0-rc2 on Windows XP Professional OS):- plugins discovery webSpider output console, textFile, xmlFile output config console set verbose True back output config textFile set verbose True set fileName result2.txt back output config xmlFile set fileName result2.xml back back target set target back start On executing the above w3af script it was observed that both textFile (result2.txt) & xmlFile (result2.xml) are created. However result2.xml contains no data while textFile contains the required output. I have already tried various options but still facing the same problem. Please let me if i am missing something in the script. Thanks, Seemanta-- Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com ___ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
Re: [W3af-develop] Wordpress version discovery plugin
backbone,
On Thu, May 28, 2009 at 5:06 PM, wrote:
> Sorry to bump in just like that in the discussion, about the meta tag that
> displays
> the WordPress version.
>
> Only since version 2.7 the generator function is in the core of WordPress,
> on
> earlier versions it was only in the theme.
>
> Just wanted to mention that. :)
Thanks for the comment =)
> ---
> http://insanesecurity.info
>
>
> On Thu, May 28, 2009 at 10:53 PM, Ryan Dewhurst
> wrote:
>>
>> Yes, I dont see why not. Should be easy enough tro implement.
>>
>> You mentioned during our email conversation that wordpress echos its
>> version number in the page head. I managed to find an example of it.
>> Your right I do have a security plugin installed which must have
>> removed it from my blog.
>>
>> Here is an example:
>>
>>
>>
>> 2009/5/28 Andres Riancho :
>> > Ryan,
>> >
>> > On Wed, May 27, 2009 at 10:18 PM, Andres Riancho
>> > wrote:
>> >> Ryan,
>> >>
>> >> On Wed, May 27, 2009 at 9:58 PM, Ryan Dewhurst
>> >> wrote:
>> >>> Hello,
>> >>> Im new to mailing lists so im not sure if this will be sent there.
>> >>
>> >> It depends on the mailing list. This one is configured to accept
>> >> attachments,
>> >>
>> >>> I'll have a look into intergrating the script into w3af over the next
>> >>> couple of days and hopefully have a working version by the weekend.
>> >>
>> >> Excellent, if you need ANY help, just let us know.
>> >>
>> >>> The script is quite simple once you have the gathered the nesesary
>> >>> data. I went through versions 2.2 to 2.7.1 and manually found client
>> >>> side differences in most of them, I also used the official changelogs
>> >>> to help identify them.
>> >>
>> >> Ohhh, you are the guy that wrote that blog post with the "diffs" of
>> >> different wordpress release packages?
>> >>
>> >>> The client side differences are in files such as CSS, javascript and
>> >>> HTML. Some versions did not have any differences apart from having
>> >>> extra files, which can easliy be identified with HTTP response codes.
>> >>>
>> >>> It works as such...
>> >>>
>> >>> Starting from version 2.7.1 (latest), the script tries to find
>> >>> something that 2.7 doesnt have, if it finds that something then the
>> >>> script stops and echos the version number.
>> >>>
>> >>> If the script doesnt find the difference it moves onto identifying the
>> >>> next version, i.e. does 2.7 have something the earlier version doesnt
>> >>> have. and so on and so forth.
>> >>
>> >> Ok, makes sense.
>> >>
>> >> Some comments regarding your code:
>> >>
>> >> - w3af uses PEP-8, with among other things says 4-spaces for
>> >> indentations. Your code has 1-space (?) indentations. Please correct
>> >> that.
>> >>
>> >> - The code is pretty simple, but i think it could be done in a better
>> >> way. Having that many functions (wp22 to wp271) doesn't seem to be a
>> >> good option. Do you think that the code could be changed a little bit,
>> >> and create a database (which can be easily updated) and then use that
>> >> database to store the information? Example of the databse
>> >>
>> >> self._wp_fingerprint =
>> >>
>> >> [('/wp-includes/js/thickbox/thickbox.css','-ms-filter:'),('/wp-admin/css/farbtastic.css',
>> >> 'farbtastic')]
>> >>
>> >> - Also, by default wordpress publishes the version number in every
>> >> page head. Maybe it would be a good idea to parse that, and compare it
>> >> with the result of the fingerprinting. What do you think?
>> >
>> > A good idea would be to have a first step, before all the version
>> > specific checks, that verifies something that's true for all wordpress
>> > installations (some X file has to be present) before even starting the
>> > fingerprinting. Could this be done?
>> >
>> >> Cheers,
>> >>
>> >>> Ryan
>> >>>
>> >>>
>> >>> 2009/5/28 Andres Riancho :
>> Ryan,
>>
>> On Wed, May 27, 2009 at 5:07 PM, Ryan Dewhurst
>> wrote:
>> > Hello,
>> > I have developed a python script that can detect the version of a
>> > wordpress installation. I think it would fit well within w3af,
>>
>> Yes, it seems that it's something good to have in the framework.
>>
>> I have like a ton of questions about how it works, could you please
>> send the script (as it is) to this mailing list for us to read it?
>>
>> > the
>> > only problem being is that I have been unable to find a plugin
>> > development manual to be able to implement my script.
>>
>> There is no development manual :(
>>
>> For the type of feature that you want to add, the correct thing is to
>> use a discovery plugin. discovery plugins are simple, they follow
>> these rules:
>>
>> - the entry point is the discover method
>>
>> - the discover method takes a fuzzable request object as a parameter,
>> and returns a list of fuzzable requests
>> (fuzzable requests are representations of GET/POST requests, which
>> represent links, and forms)
>>
>
