https://github.com/andresriancho/w3af/blob/master/w3af/core/data/kb/kb_observer.py
https://github.com/andresriancho/w3af/blob/master/w3af/core/data/kb/knowledge_base.py#L513
On Wed, Apr 22, 2015 at 1:08 PM, Electric Mind <mai...@zensecurity.su> wrote:
> Hi, Andres!
>
> Check the knowledge base hooks, I would rather use that instead of
> waiting for the scan to finish. With the hooks you can give the users
> instant feedback by sending a new task to the broker containing the
> vulnerability. Then a worker would consume the task (with the vuln)
> and show it to the user.
>
> <<
>
> You mean just to override Plugin method `kb_append` for example ?
>
>
> Regards.
>
>
>> On 03 Apr 2015, at 19:08, Andres Riancho <andres.rian...@gmail.com> wrote:
>>
>> Well, yeah, that might be a problem. I recommend:
>>
>> * You should use different processes (prefork/processes should be ok)
>> * Make sure ALL your w3af imports are done INSIDE the celery task.
>> So, this won't work:
>>
>> from w3af import something
>>
>> class Scan(Task):
>> def run(...):
>> something()
>>
>> But this might/should:
>>
>> class Scan(Task):
>> def run(...):
>> from w3af import something
>> something()
>>
>>
>> I haven't played with celery+latest w3af, but I'm 100% sure that old
>> w3af versions (that didn't make use of multiprocessing) did work
>> inside celery tasks. Let me know how your implementation goes,
>>
>> On Fri, Apr 3, 2015 at 6:10 AM, Electric Mind <mai...@zensecurity.su> wrote:
>>> Hi, Andres.
>>>
>>> Could you please show code example how to start your multiply w3af core
>>> inside of celery processes / prefork.
>>> Example:
>>>
>>> celery -A scanner worker --concurrency 2 -l debug -P prefork.
>>> celery -A scanner worker --concurrency 2 -l debug -P processes
>>>
>>> I have troubles with celery threads in my previous code example, because
>>> they use global Knowledge base object which is shared during the celery
>>> worker live.
>>>
>>> Regards.
>>>
>>> On 25 Mar 2015, at 21:55, Andres Riancho <andres.rian...@gmail.com> wrote:
>>>
>>> Electric Mind,
>>>
>>> Please read inline,
>>>
>>> On Wed, Mar 25, 2015 at 3:49 PM, Electric Mind <mai...@zensecurity.su>
>>> wrote:
>>>
>>> Hello everyone!
>>>
>>> I’d like to start my w3af instances inside of the celery.
>>> Code example is below. What is the best practice for doing that ?
>>>
>>>
>>> First I would recommend you configure celery to:
>>> * Prefetch only one task
>>> * Run one task at the time
>>> * Run scan tasks in different processes
>>>
>>> Regards!
>>>
>>>
>>> scanner.py
>>> -----
>>>
>>> # -*- coding: utf-8 -*-
>>> # !/usr/bin/env python
>>>
>>> from __future__ import absolute_import
>>> from multiprocessing import cpu_count
>>> from w3af.core.controllers.w3afCore import w3afCore
>>> from w3af.plugins.tests.helper import create_target_option_list
>>> from w3af.core.data.parsers.url import URL
>>> from w3af.core.data.options.option_list import OptionList
>>> from w3af.core.data.options.opt_factory import opt_factory
>>> from w3af.core.data.kb import knowledge_base
>>> from w3af.core.data.kb.info_set import InfoSet
>>>
>>> from scanner.celery import app
>>>
>>>
>>> @app.task
>>> def start_scan(host):
>>> target_opts = create_target_option_list(URL(host))
>>> core = w3afCore()
>>> core.WORKER_THREADS = cpu_count() * 2
>>>
>>>
>>> Threads are not processes
>>>
>>> core.target.set_options(target_opts)
>>>
>>> core.plugins.set_plugins(
>>> [
>>> 'xss_protection_header',
>>> 'csp',
>>> 'strange_headers',
>>> 'click_jacking',
>>> 'content_type_options_header'
>>> ], 'grep')
>>>
>>> core.plugins.set_plugins(
>>> [
>>> 'allowed_methods',
>>> 'find_vhosts'
>>> ], 'infrastructure')
>>>
>>> core.plugins.set_plugins(
>>> [
>>> 'dir_file_bruter',
>>> 'robots_txt',
>>> 'ria_enumerator'
>>> ], 'crawl')
>>>
>>> core.plugins.set_plugins(
>>> [
>>> 'ssl',
>>> ], 'audit')
>>>
>>> core.plugins.set_plugins(
>>> [
>>> 'console'
>>> ], 'output')
>>> console_options = OptionList()
>>> console_options.add(opt_factory('verbose', True, 'desc', 'boolean'))
>>> core.plugins.set_plugin_options('output', 'console', console_options)
>>>
>>> core.plugins.init_plugins()
>>> core.start()
>>>
>>> result = {'information_disclosures': [], 'vulnerabilities': []}
>>>
>>> for information_disclosure in knowledge_base.kb.get_all_infos():
>>> if isinstance(information_disclosure, InfoSet):
>>>
>>> result['information_disclosures'].append("{}".format(information_disclosure.get_desc()))
>>> else:
>>>
>>> result['information_disclosures'].append("{}".format(information_disclosure))
>>>
>>> for vulnerability in knowledge_base.kb.get_all_vulns():
>>> result['vulnerabilities'].append("{}".format(vulnerability))
>>>
>>>
>>> Check the knowledge base hooks, I would rather use that instead of
>>> waiting for the scan to finish. With the hooks you can give the users
>>> instant feedback by sending a new task to the broker containing the
>>> vulnerability. Then a worker would consume the task (with the vuln)
>>> and show it to the user.
>>>
>>> return result
>>>
>>> ------------------------------------------------------------------------------
>>> Dive into the World of Parallel Programming The Go Parallel Website,
>>> sponsored
>>> by Intel and developed in partnership with Slashdot Media, is your hub for
>>> all
>>> things parallel software development, from weekly thought leadership blogs
>>> to
>>> news, videos, case studies, tutorials and more. Take a look and join the
>>> conversation now. http://goparallel.sourceforge.net/
>>> _______________________________________________
>>> W3af-develop mailing list
>>> W3af-develop@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>>>
>>>
>>>
>>>
>>> --
>>> Andrés Riancho
>>> Project Leader at w3af - http://w3af.org/
>>> Web Application Attack and Audit Framework
>>> Twitter: @w3af
>>> GPG: 0x93C344F3
>>>
>>>
>>
>>
>>
>> --
>> Andrés Riancho
>> Project Leader at w3af - http://w3af.org/
>> Web Application Attack and Audit Framework
>> Twitter: @w3af
>> GPG: 0x93C344F3
>
--
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
W3af-develop mailing list
W3af-develop@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/w3af-develop
