Javier,
On Fri, Jan 21, 2011 at 12:27 PM, Javier Andalia
<javier_anda...@rapid7.com> wrote:
> Andres Riancho wrote:
>>
>> Javier,
>>
>> When you finish implementing the auto-update feature in the
>> w3af-console, could you please send us an email with one file
>> containing all the changes you've made? It will be easier to review
>> your changes like that. Thanks!
>>
>> Regards,
>>
>
> File attached!
These are some comments about the code you sent:
- Please add a new section to the users' guide in order to explain how
this new feature works, how to use it, why its cool, where the
startup.conf file lives, etc.
- You're still missing the check in dependencyCheck for pysvn; or
something is being imported before running the dependencyCheck.py
module.
"""
ariancho@toreng-0557:/mnt/big-fs/w3af/branches/auto-upd$ sudo ./w3af_console
Traceback (most recent call last):
File "./w3af_console", line 126, in <module>
errCode = main()
File "./w3af_console", line 102, in main
from core.ui.consoleUi.consoleUi import consoleUi
"""
- Replace "Checking if a new version is available in our code repo.
Please wait..." with "Checking if a new version is available in our
SVN repository. Please wait..."
- Show a message that indicated that the user version is up to date!
If we keep it as is:
"""
ariancho@toreng-0557:/mnt/big-fs/w3af/branches/auto-upd$ sudo ./w3af_console
Checking if a new version is available in our code repo. Please wait...
w3af>>>
"""
The user doesn't know what happened.
- I run the same command (sudo ./w3af_console) twice, and both times
it said "Checking if a new version is available in our code repo.
Please wait..." shouldn't that be done once a day based on
startup.conf configuration?
- Are my tests going to work with this setting?
+REPO_URL = 'http://localhost/svn/w3af'
- Missed this TODO
+#### TODO: REMOVE ME BEFORE COMMIT!!! ##
+import sys
+sys.path.append('/home/jandalia/workspace2/w3af')
- Not sure if this is going to work on my system:
+LOCAL_PATH = '/home/user/w3af'
- Have you tried this code in a w3af install that doesn't have svn
metadata? (no .svn files). In the first roll-out of this release, some
users might not have the metadata.
- Add the d to encouraged "Child classes are encourage to"
- I prefer w3af in all cases, even at the start of a sentence (W3af
version manager class.)
- Please also add the second dependency check file (there is one for
the gtkui) to this section:
depcheck_fpath = 'core/controllers/misc/dependencyCheck.py'
- What if I'm adding an import that I know will work (import os at the
beginning of the file) instead of adding an import that I think will
fail? I would try to find " import"
+ newlineswithimport = \
+ [nl for nl in diff_str.split('\n') \
+ if nl.startswith('-') and nl.find('import') != -1]
- Lets try to use underscore for most things:
self._start_section = 'StartConfig'
- Real header!
+'''
+Created on Jan 10, 2011
+
+@author: jandalia
+'''
- I would move these lines:
self._w3af = core.controllers.w3afCore.w3afCore()
self._w3af.setPlugins(['console'], 'output')
Further below, in order to perform the update first and don't have any
issues with updates over modules that were already loaded.
- Use print:
log = om.out.console
- 100% sure that this works on all cases?
vmgr = VersionMgr(localpath=os.getcwd(), log=log)
What if w3af is in the PATH and the user is standing in "/tmp/"?
All in all, we have some details to polish, but the feature seems
to be almost finished for the console UI, congrats on a job well
executed :)
Regards,
> Thanks,
>
> --
> Javier Andalia - Python Developer
>
> Gorostiaga 2355 - Office 203
> Buenos Aires. Argentina
>
> Rapid7 Recipient of Highest Ranking in Vulnerability Management from Gartner
> and Forrester:
> http://www.rapid7.com/resources/gartner_marketscope.jsp
> http://www.rapid7.com/resources/forrester-wave.jsp
>
> Index: w3af_console
> ===================================================================
> --- w3af_console (revision 3894)
> +++ w3af_console (revision 3977)
> @@ -23,45 +23,80 @@
> except w3afException, w3:
> print 'Something went wrong, w3af failed to init the output manager.
> Exception: ', str(w3)
> sys.exit(-9)
> +
> +
> +usage_doc = '''
> +w3af - Web Application Attack and Audit Framework
> +
> +Usage:
> +
> + ./w3af_console -h
> + ./w3af_console -t
> + ./w3af_console [-s <script_file>]
> +
> +Options:
> +
> + -h or --help
> + Display this help message.
> +
> + -t or --test-all
> + Runs all test scripts containing an 'assert' sentence.
>
> + -s <script_file> or --script=<script_file>
> + Run <script_file> script.
>
> + -n or --no-update
> + No update check will be made when starting. This option takes
> + precedence over the 'auto-update' setting in 'startup.conf' file.
> +
> + -f or --force-update
> + An update check will be made when starting. This option takes
> + precedence over the 'auto-update' setting in 'startup.conf' file.
> +
> + -p <profile> or --profile=<profile>
> + Run with the selecte <profile>
> +
> +For more info visit http://w3af.sourceforge.net/
> +'''
> +
> def usage():
> - om.out.information('w3af - Web Application Attack and Audit Framework')
> - om.out.information('')
> - om.out.information('Options:')
> - om.out.information(' -h Print this help message.')
> - om.out.information(' -s <file> Execute a script file.')
> - om.out.information(' -p <profile> Run with the selected profile')
> - om.out.information('')
> - om.out.information('http://w3af.sourceforge.net/')
> + om.out.information(usage_doc)
>
> def main():
> try:
> - opts, args = getopt.getopt(sys.argv[1:], "p:i:hs:get", [] )
> - except getopt.GetoptError:
> + long_options = ['script=', 'help', 'test-all', 'no-update',
> + 'force-update', 'profile=']
> + opts, args = getopt.getopt(sys.argv[1:], "ehts:nfp:", long_options)
> + except getopt.GetoptError, e:
> # print help information and exit:
> usage()
> return -3
> scriptFile = None
> profile = None
> + doupdate = None
> +
> for o, a in opts:
> - if o in ( "-e" ):
> + if o == "-e":
> # easter egg
> import base64
> - om.out.information( base64.b64decode(
> 'R3JhY2lhcyBFdWdlIHBvciBiYW5jYXJtZSB0YW50YXMgaG9yYXMgZGUgZGVzYXJyb2xsbywgdGUgYW1vIGdvcmRhIQ=='
> ) )
> - if o in ( "-t" ):
> + om.out.information(
> base64.b64decode('R3JhY2lhcyBFdWdlIHBvciBiYW5jYXJtZSB0YW50YXMgaG9yYXMgZGUgZGVzYXJyb2xsbywgdGUgYW1vIGdvcmRhIQ=='))
> + if o in ('-t', '--test-all'):
> # Test all scripts that have an assert call
> from core.controllers.misc.w3afTest import w3afTest
> w3afTest()
> return 0
> - if o == "-s":
> + if o in ('-s', '--script'):
> scriptFile = a
> if o in ('-p', '--profile'):
> # selected profile
> profile = a
> - if o == "-h":
> + if o in ('-h', '--help'):
> usage()
> return 0
> + if o in ('-f', '--force-update'):
> + doupdate = True
> + elif o in ('-n', '--no-update'):
> + doupdate = False
>
> # console
> from core.ui.consoleUi.consoleUi import consoleUi
> @@ -77,13 +112,13 @@
> commandsToRun = []
> for line in fd:
> line = line.strip()
> - if line != '' and line[0] != '#': # if not a comment..
> + if line != '' and line[0] != '#': # if not a comment..
> commandsToRun.append( line )
> fd.close()
> elif profile is not None:
> commandsToRun = ["profiles use %s" % profile]
>
> - console = consoleUi(commands=commandsToRun)
> + console = consoleUi(commands=commandsToRun, do_upd=doupdate)
> console.sh()
>
>
> @@ -91,4 +126,3 @@
> errCode = main()
> backToCurrentDir()
> sys.exit(errCode)
> -
> Index: core/controllers/misc/dependencyCheck.py
> ===================================================================
> --- core/controllers/misc/dependencyCheck.py (revision 3894)
> +++ core/controllers/misc/dependencyCheck.py (revision 3977)
> @@ -22,7 +22,6 @@
>
> import core.controllers.outputManager as om
> import sys
> -import subprocess
>
> # Use w3af's 'extlib' modules first. By doing this we ensure that modules
> # in 'w3af/extlib/' are first imported over python installation
> 'site-packages/'
> @@ -95,5 +94,13 @@
> msg += ' - On Debian based distributions: apt-get install
> python-lxml'
> print msg
> sys.exit( 1 )
> +
> + try:
> + import pysvn
> + except:
> + msg = 'You have to install python pysvn lib. \n'
> + msg += ' - On Debian based distributions: apt-get install
> python-svn'
> + print msg
> + sys.exit( 1 )
>
>
> Index: core/controllers/auto_update/auto_update.py
> ===================================================================
> --- core/controllers/auto_update/auto_update.py (revision 0)
> +++ core/controllers/auto_update/auto_update.py (revision 3977)
> @@ -0,0 +1,657 @@
> +'''
> +auto_update.py
> +
> +Copyright 2011 Andres Riancho
> +
> +This file is part of w3af, w3af.sourceforge.net .
> +
> +w3af is free software; you can redistribute it and/or modify
> +it under the terms of the GNU General Public License as published by
> +the Free Software Foundation version 2 of the License.
> +
> +w3af is distributed in the hope that it will be useful,
> +but WITHOUT ANY WARRANTY; without even the implied warranty of
> +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +GNU General Public License for more details.
> +
> +You should have received a copy of the GNU General Public License
> +along with w3af; if not, write to the Free Software
> +Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +'''
> +
> +from __future__ import with_statement
> +from datetime import datetime, date, timedelta
> +import os
> +import time
> +import ConfigParser
> +import threading
> +
> +
> +class SVNError(Exception):
> + pass
> +
> +
> +class SVNUpdateError(SVNError):
> + pass
> +
> +
> +class SVNCommitError():
> + pass
> +
> +
> +class SVNClient(object):
> + '''
> + Typically an abstract class. Intended to define behaviour. Not to be
> + instantiated.
> + SVN implementations should extend from this class.
> + Implemented methods in child classes should potentially raise SVNError
> + exception (or descendant) when a condition error is found.
> + '''
> +
> + def __init__(self, localpath):
> + self._localpath = localpath
> + self._repourl = self._get_repourl()
> + # Action Locker!
> + self._actionlock = threading.RLock()
> +
> + def _get_repourl(self):
> + '''
> + Get repo's URL. To be implemented by subclasses.
> + '''
> + raise NotImplementedError
> +
> + def update(self):
> + '''
> + Update local repo to last revision.
> + '''
> + raise NotImplementedError
> +
> + def commit(self):
> + '''
> + Commit to remote repo changes in local repo.
> + '''
> + raise NotImplementedError
> +
> + def status(self, path=None):
> + '''
> + Return a SVNFilesList object.
> +
> + @param localpath: Path to get the status from. If None use
> project's
> + root.
> + '''
> + raise NotImplementedError
> +
> + def list(self, path_or_url=None):
> + '''
> + Return a SVNFilesList. Elements are tuples containing the path and
> + the status for all files in `path_or_url` at the provided revision.
> + '''
> + raise NotImplementedError
> +
> + def diff(self, localpath, rev=None):
> + '''
> + Return string with the differences between `rev` and HEAD revision
> for
> + `localpath`
> + '''
> + raise NotImplementedError
> +
> +
> +
> +import pysvn
> +
> +# Actions on files
> +FILE_UPD = 'UPD' # Updated
> +FILE_NEW = 'NEW' # New
> +FILE_DEL = 'DEL' # Removed
> +
> +wcna = pysvn.wc_notify_action
> +pysvn_action_translator = {
> + wcna.update_add: FILE_NEW,
> + wcna.update_delete: FILE_DEL,
> + wcna.update_update: FILE_UPD
> +}
> +
> +# Files statuses
> +ST_CONFLICT = 'C'
> +ST_NORMAL = 'N'
> +ST_UNVERSIONED = 'U'
> +ST_MODIFIED = 'M'
> +ST_UNKNOWN = '?'
> +
> +wcsk = pysvn.wc_status_kind
> +pysvn_status_translator = {
> + wcsk.conflicted: ST_CONFLICT,
> + wcsk.normal: ST_NORMAL,
> + wcsk.unversioned: ST_UNVERSIONED,
> + wcsk.modified: ST_MODIFIED
> +}
> +
> +
> +class W3afSVNClient(SVNClient):
> + '''
> + Our wrapper for pysvn.Client class.
> + '''
> +
> + UPD_ACTIONS = (wcna.update_add, wcna.update_delete, wcna.update_update)
> +
> + def __init__(self, localpath):
> + self._svnclient = pysvn.Client()
> + # Call parent's __init__
> + super(W3afSVNClient, self).__init__(localpath)
> + # Set callback function
> + self._svnclient.callback_notify = self._register
> + # Events occurred in current action
> + self._events = []
> +
> + def __getattribute__(self, name):
> + '''
> + Wrap all methods on order to be able to respond to Ctrl+C signals.
> + This implementation was added due to limitations in pysvn.
> + '''
> + def new_meth(*args, **kwargs):
> + def wrapped_meth(*args, **kwargs):
> + try:
> + self._result = meth(*args, **kwargs)
> + self._exc = None
> + except Exception, exc:
> + if isinstance(exc, pysvn.ClientError):
> + exc = SVNError(*exc.args)
> + self._exc = exc
> + # Run wrapped_meth in new thread.
> + th = threading.Thread(target=wrapped_meth, args=args,
> + kwargs=kwargs)
> + th.setDaemon(True)
> + try:
> + th.start()
> + while th.isAlive():
> + time.sleep(0.2)
> + try:
> + if self._exc:
> + raise self._exc
> + return self._result
> + finally:
> + self._exc = self._result = None
> + except KeyboardInterrupt:
> + raise
> +
> + attr = SVNClient.__getattribute__(self, name)
> + if callable(attr):
> + meth = attr
> + return new_meth
> + return attr
> +
> + @property
> + def URL(self):
> + return self._repourl
> +
> + def update(self):
> + with self._actionlock:
> + self._events = []
> + try:
> + pysvn_rev = self._svnclient.update(self._localpath)[0]
> + except pysvn.ClientError, ce:
> + raise SVNUpdateError(*ce.args)
> + else:
> + updfiles = self._filter_files(self.UPD_ACTIONS)
> + updfiles.rev = Revision(pysvn_rev.number, pysvn_rev.date)
> + return updfiles
> +
> + def status(self, localpath=None):
> + with self._actionlock:
> + path = localpath or self._localpath
> + entries = self._svnclient.status(path, recurse=False)
> + res = [(ent.path, pysvn_status_translator.get(ent.text_status,
> + ST_UNKNOWN)) for ent in entries]
> + return SVNFilesList(res)
> +
> + def list(self, path_or_url=None):
> + with self._actionlock:
> + if not path_or_url:
> + path_or_url = self._localpath
> + entries = self._svnclient.list(path_or_url, recurse=False)
> + res = [(ent.path, None) for ent, _ in entries]
> + return SVNFilesList(res)
> +
> + def diff(self, localpath, rev=None):
> + with self._actionlock:
> + path = os.path.join(self._localpath, localpath)
> + # If no rev is passed the compare to HEAD
> + if rev is None:
> + rev = pysvn.Revision(pysvn.opt_revision_kind.head)
> + tempfile = os.tempnam()
> + diff_str = self._svnclient.diff(tempfile, path, revision1=rev)
> + return diff_str
> +
> + def log(self, start_rev, end_rev):
> + '''
> + Return SVNLogList of log messages between `start_rev` and
> `end_rev`
> + revisions.
> +
> + @param start_rev: Revision object
> + @param end_rev: Revision object
> + '''
> + with self._actionlock:
> + # Expected by pysvn.Client.log method
> + pysvnstartrev = pysvn.Revision(pysvn.opt_revision_kind.number,
> + start_rev.number)
> + pysvnendrev = pysvn.Revision(pysvn.opt_revision_kind.number,
> + end_rev.number)
> + logs = (l.message for l in self._svnclient.log(
> + self._localpath,
> +
> revision_start=pysvnstartrev,
> + revision_end=pysvnendrev))
> + rev = end_rev if (end_rev.number > start_rev.number) else
> start_rev
> + return SVNLogList(logs, rev)
> +
> +
> + def _get_repourl(self):
> + '''
> + Get repo's URL.
> + '''
> + svninfo = self._get_svn_info(self._localpath)
> + return svninfo.URL
> +
> + def _get_svn_info(self, path_or_url):
> + try:
> + return self._svnclient.info2(path_or_url, recurse=False)[0][1]
> + except pysvn.ClientError, ce:
> + raise SVNUpdateError(*ce.args)
> +
> + def get_revision(self, local=True):
> + '''
> + Return Revision object.
> +
> + @param local: If true return local's revision data; otherwise use
> + repo's.
> + '''
> + path_or_url = self._localpath if local else self._repourl
> + _rev = self._get_svn_info(path_or_url).rev
> + return Revision(_rev.number, _rev.date)
> +
> + def _filter_files(self, filterbyactions=()):
> + '''
> + Filter... Return files-actions
> + @param filterby:
> + '''
> + files = SVNFilesList()
> + for ev in self._events:
> + action = ev['action']
> + if action in filterbyactions:
> + path = ev['path']
> + # We're not interested on reporting directories unless a
> + # 'delete' has been performed on them
> + if not os.path.isdir(path) or action == wcna.update_delete:
> + files.append(path, pysvn_action_translator[action])
> + return files
> +
> + def _register(self, event):
> + '''
> + Callback method. Registers all events taking place during this
> action.
> + '''
> + self._events.append(event)
> +
> +
> +class Revision(object):
> + '''
> + Our own class for revisions.
> + '''
> +
> + def __init__(self, number, date):
> + self._number = number
> + self._date = date
> +
> + def __eq__(self, rev):
> + return self._number == rev.number and \
> + self._date == rev.date
> +
> + def __lt__(self, rev):
> + return self._number < rev.number
> +
> + @property
> + def date(self):
> + return self._date
> +
> + @property
> + def number(self):
> + return self._number
> +
> +
> +# Limit of lines to SVNList types. To be used in __str__ method
> re-definition.
> +PRINT_LINES = 20
> +
> +class SVNList(list):
> +
> + '''
> + Wrapper for python list type. It may contain the number of the current
> + revision and do a custom list print. Child classes are encourage to
> + redefine the __str__ method.
> + '''
> +
> + def __init__(self, seq=(), rev=None):
> + '''
> + @param rev: Revision object
> + '''
> + list.__init__(self, seq)
> + self._rev = rev
> + self._sorted = True
> +
> + def _getrev(self):
> + return self._rev
> +
> + def _setrev(self, rev):
> + self._rev = rev
> +
> + # TODO: Cannot use *full* decorators as we're still on py2.5
> + rev = property(_getrev, _setrev)
> +
> + def __eq__(self, olist):
> + return list.__eq__(self, olist) and self._rev == olist.rev
> +
> +
> +class SVNFilesList(SVNList):
> + '''
> + Custom SVN files list holder.
> + '''
> +
> + def __init__(self, seq=(), rev=None):
> + SVNList.__init__(self, seq, rev)
> + self._sorted = True
> +
> + def append(self, path, status):
> + list.append(self, (path, status))
> + self._sorted = False
> +
> + def __str__(self):
> + # First sort by status
> + sortfunc = lambda x, y: cmp(x[1], y[1])
> + self.sort(cmp=sortfunc)
> + lines, rest = self[:PRINT_LINES], max(len(self) - PRINT_LINES, 0)
> + print_list = ['%s %s' % (f, s) for s, f in lines]
> + if rest:
> + print_list.append('and %d files more.' % rest)
> + if self._rev:
> + print_list.append('At revision %s.' % self._rev.number)
> + return os.linesep.join(print_list)
> +
> +
> +class SVNLogList(SVNList):
> + '''
> + Provides a custom way to print a SVN logs list.
> + '''
> + def __str__(self):
> + print_list = []
> + if self._rev:
> + print_list.append('Revision %s:' % self._rev.number)
> + lines, rest = self[:PRINT_LINES], max(len(self) - PRINT_LINES, 0)
> + print_list += ['%3d. %s' % (n + 1, ln) for n, ln in
> enumerate(lines)]
> + if rest:
> + print_list.append('and %d commit logs more.' % rest)
> + return os.linesep.join(print_list)
> +
> +
> +# Use this class to perform svn actions on code
> +SVNClientClass = W3afSVNClient
> +
> +
> +# Facade class. Intended to be used to to interact with the module
> +class VersionMgr(object): #TODO: Make it singleton?
> +
> + # Events constants
> + ON_UPDATE = 1
> + ON_CONFIRM_UPDATE = 2
> + ON_UPDATE_CHECK = 3
> + ON_COMMIT = 4
> +
> + def __init__(self, localpath, log):
> + '''
> + W3af version manager class. Handles the logic concerning the
> + automatic update/commit process of the code.
> +
> + @param localpath: Working directory
> + @param log: Default output function
> + '''
> + self._localpath = localpath
> +
> + self._log = log
> + self._client = SVNClientClass(localpath)
> + # Registered functions
> + self._reg_funcs = {}
> + # Startup configuration
> + self._start_cfg = StartUpConfig()
> +
> +
> + def update(self, force=False, askvalue=None, print_result=False,
> + show_log=False):
> + '''
> + Perform code update if necessary.
> +
> + @param askvalue: Callback function that will output the update
> + confirmation response.
> + @param print_result: If True print the result files using
> instance's
> + log function.
> + @param show_log: If True interact with the user through `askvalue`
> and
> + show a summary of the log messages.
> + '''
> + client = self._client
> + lrev = client.get_revision(local=True)
> + files = SVNFilesList(rev=lrev)
> +
> + if force or self._has_to_update():
> + self._notify(VersionMgr.ON_UPDATE)
> + rrev = client.get_revision(local=False)
> +
> + # If local rev is not lt repo's then we got nothing to update.
> + if not (lrev < rrev):
> + return files
> +
> + proceed_upd = True
> + # Call callback function
> + if askvalue:
> + proceed_upd = askvalue(\
> + 'Your current w3af installation is r%s. Do you want to ' \
> + 'update to r%s [y/N]? ' % (lrev.number, rrev.number))
> + proceed_upd = (proceed_upd.lower() == 'y')
> +
> + if proceed_upd:
> + msg = 'w3af is updating from the official SVN server...'
> + self._notify(VersionMgr.ON_UPDATE, msg)
> + # Find new deps.
> + newdeps = self._added_new_dependencies()
> + if newdeps:
> + msg = 'At least one new dependency (%s) was included in
> ' \
> + 'w3af. Please update manually.' % str(',
> '.join(newdeps))
> + self._notify(VersionMgr.ON_UPDATE, msg)
> + else:
> + # Finally do the update!
> + files = client.update()
> + # Now save today as last-update date and persist it.
> + self._start_cfg.last_upd = date.today()
> + self._start_cfg.save()
> +
> + # Before returning perform some interaction with the user if
> + # requested.
> + if print_result:
> + self._log(str(files))
> +
> + if show_log:
> + show_log = askvalue('Do you want to see a summary of the '
> \
> + 'new code commits log messages? [y/N]? ').lower() == 'y'
> + if show_log:
> + self._log(str(self._client.log(lrev, rrev)))
> + return files
> +
> + def status(self, path=None):
> + return self._client.status(path)
> +
> + def register(self, eventname, func, msg):
> + funcs = self._reg_funcs.setdefault(eventname, [])
> + funcs.append((func, msg))
> +
> + def _notify(self, event, msg=None):
> + '''
> + Call registered functions for event. If `msg` is not None then
> force
> + to call the registered functions with `msg`.
> + '''
> + for f, _msg in self._reg_funcs.get(event, []):
> + f(msg or _msg)
> +
> + def _added_new_dependencies(self):
> + '''
> + Return tuple with the dependencies added to extlib/ in the repo if
> + any. Basically it compares local dirs under extlib/ to those in the
> + repo as well as checks if at least a new sentence containing the
> + import keyword was added to the dependencyCheck.py file.
> + '''
> + #
> + # Check if a new directory was added to repo's extlib
> + #
> + client = self._client
> + ospath = os.path
> + join = ospath.join
> + # Find dirs in repo
> + repourl = self._client.URL + '/' + 'extlib'
> + # In repo we distinguish dirs from files by the dot (.) presence
> + repodirs = (ospath.basename(d) for d, _ in client.list(repourl)[1:]
> \
> + if ospath.basename(d).find('.') ==
> -1)
> + # Get local dirs
> + extliblocaldir = join(self._localpath, 'extlib')
> + extlibcontent = (join(extliblocaldir, f) for f in \
> + os.listdir(extliblocaldir))
> + localdirs = (ospath.basename(d) for d in extlibcontent \
> + if ospath.isdir(d))
> + # New dependencies
> + deps = tuple(set(repodirs).difference(localdirs))
> +
> + #
> + # Additional constraint: We should verify that at least an import
> + # sentence was added to the dependencyCheck.py file
> + #
> + if deps:
> + depcheck_fpath = 'core/controllers/misc/dependencyCheck.py'
> + diff_str = client.diff(depcheck_fpath)
> + # SVN shows HEAD rev's new lines preceeded by a '-' char.
> + newlineswithimport = \
> + [nl for nl in diff_str.split('\n') \
> + if nl.startswith('-') and nl.find('import') != -1]
> + # Ok, no import sentence was detected so no dep. was *really*
> + # added.
> + if not newlineswithimport:
> + deps = ()
> +
> + return deps
> +
> + def _has_to_update(self):
> + '''
> + Helper method that figures out if an update should be performed
> + according to the startup cfg file.
> + Some rules:
> + 1) IF auto_upd is False THEN return False
> + 2) IF last_upd == 'yesterday' and freq == 'D' THEN return True
> + 3) IF last_upd == 'two_days_ago' and freq == 'W' THEN return
> False.
> + @return: Boolean value.
> + '''
> + startcfg = self._start_cfg
> + # That's it!
> + if not startcfg.auto_upd:
> + return False
> + else:
> + freq = startcfg.freq
> + diff_days = max((date.today()-startcfg.last_upd).days, 0)
> +
> + if (freq == StartUpConfig.FREQ_DAILY and diff_days > 0) or \
> + (freq == StartUpConfig.FREQ_WEEKLY and diff_days > 6) or \
> + (freq == StartUpConfig.FREQ_MONTHLY and diff_days > 29):
> + return True
> + return False
> +
> +
> +from core.controllers.misc.homeDir import get_home_dir
> +
> +class StartUpConfig(object):
> + '''
> + Wrapper class for ConfigParser.ConfigParser.
> + Holds the configuration for the VersionMgr update/commit process
> + '''
> +
> + ISO_DATE_FMT = '%Y-%m-%d'
> + # Frequency constants
> + FREQ_DAILY = 'D' # [D]aily
> + FREQ_WEEKLY = 'W' # [W]eekly
> + FREQ_MONTHLY = 'M' # [M]onthly
> +
> + def __init__(self):
> +
> + self._start_cfg_file = os.path.join(get_home_dir(), 'startup.conf')
> + self._start_section = 'StartConfig'
> + defaults = {'auto-update': 'true', 'frequency': 'D',
> + 'last-update': 'None'}
> + self._config = ConfigParser.ConfigParser(defaults)
> + self._autoupd, self._freq, self._lastupd = self._load_cfg()
> +
> + ### PROPERTIES #
> +
> + def _get_last_upd(self):
> + '''
> + Getter method.
> + '''
> + return self._lastupd
> +
> + def _set_last_upd(self, datevalue):
> + '''
> + @param datevalue: datetime.date value
> + '''
> + self._lastupd = datevalue
> + self._config.set(self._start_section, 'last-update',
> + datevalue.isoformat())
> +
> + # TODO: Cannot use *full* decorators as we're still on py2.5
> + # Read/Write property
> + last_upd = property(_get_last_upd, _set_last_upd)
> +
> + @property
> + def freq(self):
> + return self._freq
> +
> + @property
> + def auto_upd(self):
> + return self._autoupd
> +
> + ### METHODS #
> +
> + def _load_cfg(self):
> + '''
> + Loads configuration from config file.
> + '''
> + config = self._config
> + startsection = self._start_section
> + if not config.has_section(startsection):
> + config.add_section(startsection)
> +
> + # Read from file
> + config.read(self._start_cfg_file)
> +
> + auto_upd = config.get(startsection, 'auto-update', raw=True)
> + boolvals = {'false': 0, 'off': 0, 'no': 0,
> + 'true': 1, 'on': 1, 'yes': 1}
> + auto_upd = bool(boolvals.get(auto_upd.lower(), False))
> +
> + freq = config.get(startsection, 'frequency', raw=True).upper()
> + if freq not in (StartUpConfig.FREQ_DAILY,
> StartUpConfig.FREQ_WEEKLY,
> + StartUpConfig.FREQ_MONTHLY):
> + freq = StartUpConfig.FREQ_DAILY
> +
> + lastupdstr = config.get(startsection, 'last-update',
> raw=True).upper()
> + # Try to parse it
> + try:
> + lastupd = datetime.strptime(lastupdstr,
> self.ISO_DATE_FMT).date()
> + except:
> + # Provide default value that enforces the update to happen
> + lastupd = date.today() - timedelta(days=31)
> + return (auto_upd, freq, lastupd)
> +
> + def save(self):
> + '''
> + Saves current values to cfg file
> + '''
> + with open(self._start_cfg_file, 'wb') as configfile:
> + self._config.write(configfile)
> Index: core/controllers/auto_update/tests/test_auto_update.py
> ===================================================================
> --- core/controllers/auto_update/tests/test_auto_update.py (revision 0)
> +++ core/controllers/auto_update/tests/test_auto_update.py (revision
> 3977)
> @@ -0,0 +1,193 @@
> +'''
> +Created on Jan 10, 2011
> +
> +@author: jandalia
> +'''
> +
> +#### TODO: REMOVE ME BEFORE COMMIT!!! ##
> +import sys
> +sys.path.append('/home/jandalia/workspace2/w3af')
> +########################################
> +
> +from pymock import PyMockTestCase, method, override, dontcare, set_count
> +
> +from core.controllers.auto_update.auto_update import W3afSVNClient,
> Revision, \
> + VersionMgr, SVNFilesList, StartUpConfig, FILE_UPD, FILE_NEW, FILE_DEL,
> \
> + ST_CONFLICT, ST_MODIFIED, ST_UNKNOWN
> +
> +del W3afSVNClient.__getattribute__
> +
> +REPO_URL = 'http://localhost/svn/w3af'
> +LOCAL_PATH = '/home/user/w3af'
> +
> +
> +def dummy(*args, **kwargs):
> + pass
> +
> +
> +class TestW3afSVNClient(PyMockTestCase):
> +
> + rev = Revision(112, None)
> + upd_files = SVNFilesList(
> + [('file1.txt', FILE_NEW),
> + ('file2.py', FILE_UPD),
> + ('file3.jpg', FILE_DEL)],
> + rev
> + )
> +
> + def setUp(self):
> + PyMockTestCase.setUp(self)
> +
> + W3afSVNClient._get_repourl = dummy
> + self.client = W3afSVNClient(LOCAL_PATH)
> + self.client._repourl = REPO_URL
> + self.client._svnclient = self.mock()
> +
> + def test_has_repourl(self):
> + self.assertTrue(self.client._repourl is not None)
> +
> + def test_has_svn_client(self):
> + self.assertTrue(self.client._svnclient is not None)
> +
> + def test_has_localpath(self):
> + self.assertTrue(self.client._localpath is not None)
> +
> + def test_upd(self):
> + client = self.client
> + method(client._svnclient,
> 'update').expects(LOCAL_PATH).returns([self.rev])
> + override(client, '_filter_files').expects(client.UPD_ACTIONS)
> + self.returns(self.upd_files)
> + ## Stop recording. Play!
> + self.replay()
> + self.assertEquals(self.upd_files, client.update())
> + ## Verify ##
> + self.verify()
> +
> + def test_upd_fail(self):
> + from pysvn import ClientError
> + from core.controllers.auto_update.auto_update import SVNUpdateError
> + client = self.client
> + method(client._svnclient, 'update').expects(LOCAL_PATH)
> + self.raises(ClientError('file locked'))
> + ## Stop recording. Play!
> + self.replay()
> + self.assertRaises(SVNUpdateError, client.update)
> + ## Verify ##
> + self.verify()
> +
> + def test_upd_conflict(self):
> + '''
> + Files in conflict exists after update.
> + '''
> + pass
> +
> + def test_upd_nothing_to_update(self):
> + '''No update to current copy was made. Tell the user. Presumably
> + the revision was incremented'''
> + pass
> +
> + def test_filter_files(self):
> + import pysvn
> + from pysvn import wc_notify_action as wcna
> + from pysvn import Revision
> + from core.controllers.auto_update.auto_update import os
> + client = self.client
> + override(os.path, 'isdir').expects(dontcare()).returns(False)
> + set_count(exactly=2)
> + ## Stop recording. Play!
> + self.replay()
> + # Call client's callback function several times
> + f1 = '/path/to/file/foo.py'
> + ev = {'action': wcna.update_delete,
> + 'error': None, 'mime_type': None, 'path': f1,
> + 'revision': Revision(pysvn.opt_revision_kind.number, 11)}
> + client._register(ev)
> +
> + f2 = '/path/to/file/foo2.py'
> + ev2 = {'action': wcna.update_update,
> + 'error': None, 'mime_type': None,
> + 'path': f2,
> + 'revision': Revision(pysvn.opt_revision_kind.number, 11)}
> + client._register(ev2)
> +
> + expected_res = SVNFilesList([(f1, FILE_DEL), (f2, FILE_UPD)])
> + self.assertEquals(expected_res,
> +
> client._filter_files(filterbyactions=W3afSVNClient.UPD_ACTIONS))
> + ## Verify ##
> + self.verify()
> +
> + def test_status(self):
> + from pysvn import wc_status_kind as wcsk
> + client = self.client
> + # Mock pysvnstatus objects
> + smock = self.mock()
> + smock.path
> + self.setReturn('/some/path/foo')
> + smock.text_status
> + self.setReturn(wcsk.modified)
> +
> + smock2 = self.mock()
> + smock2.path
> + self.setReturn('/some/path/foo2')
> + smock2.text_status
> + self.setReturn(wcsk.conflicted)
> +
> + smock3 = self.mock()
> + smock3.path
> + self.setReturn('/some/path/foo3')
> + smock3.text_status
> + self.setReturn('some_weird_status')
> +
> + status_files = [smock, smock2, smock3]
> + method(client._svnclient, 'status').expects(LOCAL_PATH,
> recurse=False)
> + self.returns(status_files)
> + ## Stop recording - Replay ##
> + self.replay()
> + expected_res = \
> + SVNFilesList([
> + ('/some/path/foo', ST_MODIFIED),
> + ('/some/path/foo2', ST_CONFLICT),
> + ('/some/path/foo3', ST_UNKNOWN)])
> + self.assertEquals(expected_res, client.status())
> + ## Verify ##
> + self.verify()
> +
> + def test_commit(self):
> + pass
> +
> +
> +class TestVersionMgr(PyMockTestCase):
> +
> + def setUp(self):
> + PyMockTestCase.setUp(self)
> + # Override auto_update module variable
> + import core.controllers.auto_update.auto_update as autoupdmod
> + autoupdmod.SVNClientClass = self.mock()
> + self.vmgr = VersionMgr(LOCAL_PATH, dummy)
> +
> + def test_has_to_update(self):
> +
> + vmgr = self.vmgr
> + start_cfg_mock = self.mock()
> + vmgr._start_cfg = start_cfg_mock
> +
> + # Test no auto-update
> + start_cfg_mock.auto_upd
> + self.setReturn(False)
> + self.replay()
> + self.assertFalse(vmgr._has_to_update())
> +
> + # Test [D]aily, [W]eekly and [M]onthly auto-update
> + import datetime
> + SC = StartUpConfig
> + for freq, diffdays in ((SC.FREQ_DAILY, 1), (SC.FREQ_WEEKLY, 8), \
> + (SC.FREQ_MONTHLY, 34)):
> + self.reset()
> + start_cfg_mock.auto_upd
> + self.setReturn(True)
> + start_cfg_mock.freq
> + self.setReturn(freq)
> + start_cfg_mock.last_upd
> + self.setReturn(datetime.date.today() -
> datetime.timedelta(days=diffdays))
> + self.replay()
> + self.assertTrue(vmgr._has_to_update())
> Index: core/controllers/auto_update/tests/__init__.py
> ===================================================================
> Index: core/controllers/auto_update/__init__.py
> ===================================================================
> --- core/controllers/auto_update/__init__.py (revision 0)
> +++ core/controllers/auto_update/__init__.py (revision 3977)
> @@ -0,0 +1,2 @@
> +from auto_update import *
> +__all__ = ['VersionMgr']
> \ No newline at end of file
> Index: core/ui/consoleUi/consoleUi.py
> ===================================================================
> --- core/ui/consoleUi/consoleUi.py (revision 3894)
> +++ core/ui/consoleUi/consoleUi.py (revision 3977)
> @@ -23,8 +23,11 @@
> import sys
> try:
> from shlex import *
> - import os.path
> + import os
> + import random
> import traceback
> +
> + from core.controllers.auto_update import VersionMgr, SVNError
> from core.ui.consoleUi.rootMenu import *
> from core.ui.consoleUi.callbackMenu import *
> from core.ui.consoleUi.util import *
> @@ -35,10 +38,10 @@
> import core.controllers.outputManager as om
> import core.controllers.miscSettings as miscSettings
> from core.controllers.w3afException import w3afException
> - import random
> except KeyboardInterrupt:
> sys.exit(0)
>
> +
> class consoleUi:
> '''
> This class represents the console.
> @@ -47,38 +50,57 @@
> @author Alexander Berezhnoy (alexander.berezhnoy |at| gmail.com)
> '''
>
> - def __init__(self, commands=[], parent=None):
> + def __init__(self, commands=[], parent=None, do_upd=None):
> self._commands = commands
> self._line = [] # the line which is being typed
> self._position = 0 # cursor position
> self._history = historyTable() # each menu has array of (array,
> positionInArray)
> self._trace = []
> + self._upd_avail = False
>
> - self._handlers = { '\t' : self._onTab, \
> - '\r' : self._onEnter, \
> - term.KEY_BACKSPACE : self._onBackspace, \
> - term.KEY_LEFT : self._onLeft, \
> - term.KEY_RIGHT : self._onRight, \
> - term.KEY_UP : self._onUp, \
> - term.KEY_DOWN : self._onDown, \
> - '^C' : self._backOrExit, \
> + self._handlers = {
> + '\t' : self._onTab,
> + '\r' : self._onEnter,
> + term.KEY_BACKSPACE : self._onBackspace,
> + term.KEY_LEFT : self._onLeft,
> + term.KEY_RIGHT : self._onRight,
> + term.KEY_UP : self._onUp,
> + term.KEY_DOWN : self._onDown,
> + '^C' : self._backOrExit,
> '^D' : self._backOrExit,
> '^L' : self._clearScreen,
> '^W' : self._delWord,
> '^H' : self._onBackspace,
> '^A' : self._toLineStart,
> - '^E' : self._toLineEnd }
> + '^E' : self._toLineEnd
> + }
>
> if parent:
> self.__initFromParent(parent)
> else:
> - self.__initRoot()
> + self.__initRoot(do_upd)
>
> -
> - def __initRoot(self):
> + def __initRoot(self, do_upd):
> + '''
> + Root menu init routine.
> + '''
> self._w3af = core.controllers.w3afCore.w3afCore()
> self._w3af.setPlugins(['console'], 'output')
> -
> +
> + if do_upd is not False:
> + log = om.out.console
> + vmgr = VersionMgr(localpath=os.getcwd(), log=log)
> + msg = 'Checking if a new version is available in our code repo.
> ' \
> + 'Please wait...'
> + vmgr.register(vmgr.ON_UPDATE, log, msg)
> + try:
> + vmgr.update(force=do_upd is True, askvalue=raw_input,
> + print_result=True, show_log=True)
> + except SVNError, e:
> + om.out.error('An error occured while updating:\n%s' %
> e.args)
> + except KeyboardInterrupt:
> + pass
> +
> def __initFromParent(self, parent):
> self._context = parent._context
> self._w3af = parent._w3af
> @@ -124,7 +146,6 @@
> def _executePending(self):
> while (self._commands):
> curCmd, self._commands = self._commands[0], self._commands[1:]
> -
> self._paste(curCmd)
> self._onEnter()
>
> Index: core/ui/consoleUi/menu.py
> ===================================================================
> --- core/ui/consoleUi/menu.py (revision 3894)
> +++ core/ui/consoleUi/menu.py (revision 3977)
> @@ -20,14 +20,12 @@
>
> '''
>
> -import traceback
> -
> -import core.data.kb.knowledgeBase as kb
> +from core.controllers.w3afException import w3afException
> from core.ui.consoleUi.util import *
> from core.ui.consoleUi.history import *
> from core.ui.consoleUi.help import *
> import core.controllers.outputManager as om
> -from core.controllers.w3afException import w3afException
> +import core.data.kb.knowledgeBase as kb
>
>
> class menu:
> @@ -86,7 +84,7 @@
>
> self._paramHandlers = {}
> for cmd in [c for c in dir(self) if c.startswith('_cmd_')]:
> - self._handlers[cmd[5:]] = getattr(self, cmd)
> + self._handlers[cmd[5:]] = getattr(self, cmd)
>
> for cmd in self._handlers.keys():
> try:
> @@ -219,7 +217,6 @@
> else:
> om.out.console( repr(res) )
>
> -
> def _cmd_assert(self, params):
> if not len(params):
> raise w3afException('Expression is expected')
>
>
--
Andrés Riancho
Director of Web Security at Rapid7 LLC
Founder at Bonsai Information Security
Project Leader at w3af
------------------------------------------------------------------------------
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires
February 28th, so secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsight-sfd2d
_______________________________________________
W3af-develop mailing list
W3af-develop@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/w3af-develop
