Re: Creating (rather) generic plugin framework?
Jean-Paul Calderone <[EMAIL PROTECTED]> writes: > You may want to look at a few existing Python plugin systems. To get you > started, here's a link to the Twisted plugin system documentation: > <http://twistedmatrix.com/users/warner/doc-latest/core/howto/plugin.html> Thanks both to Bruno and Jean-Paul! I think I'll go the twisted way (I try to avoid re-inventing the wheel here) -- # Edvard Majakari Software Engineer # PGP PUBLIC KEY available Soli Deo Gloria! "Debugging is twice as hard as writing the code in the firstplace. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." -- Brian W. Kernighan -- http://mail.python.org/mailman/listinfo/python-list
Creating (rather) generic plugin framework?
Hi,
My idea is to create a system working as follows: each module knows
path to plugin directory, and that directory contains modules which
may add hooks to some points in the code.
Inspired by http://www.python.org/pycon/2005/papers/7/pyconHooking.html
I would create a class like this:
class Printer:
def printit(self, msg):
stuff = self.beforePrintHook(msg)
if stuff:
msg = stuff
print msg
self.afterPrintHook(msg)
def beforePrintHook(self, msg): pass
def afterPrintHook(self, msg): pass
Now, in the spirit of py.test, I'd like API to be practically no API at all :)
moreover, deploying a plugin must consist simply of adding appropriate file to
plugins directory, nothing more, and removing it would uninstall it. The
plugin should be able to somehow result in all future invocations to
Printer.printit() call hooks specified in the plugin. Now, the plugin module
for class above /might/ be along the following lines (I'm thinking of stuff
here, so I don't know yet what would be the most appropriate way):
### a very simple plugin which uppercases all data fed to it.
extensions = {'Printer': 'PrinterHook'}
class PrinterHook:
def beforePrintHook(self, msg):
return msg.upper()
def afterPrintHook(self, msg):
print "Called afterPrintHook with msg %s" % msg
Now, I have a very rude (I think) implementation which has two methods, first
the one that loads plugin modules:
def find_plugins():
mods = [mod for mod in os.listdir(PLUGIN_DIR) if mod.endswith('.py')]
# for each module in plugin dir, import the module and setup hooks. Hooks
# must have equal method names in plugin module as in the original class.
for mod in mods:
name = os.path.splitext(mod)[0]
fobj, fpath, desc = imp.find_module(os.path.join(PLUGIN_DIR, name))
module = imp.load_module(name, fobj, fpath, desc)
set_hooks(module)
...then the other that is responsible for setting up hooks
def set_hooks(mod):
# mod.extensions has "base" class names as keys, (hook, priority) as
# values
for k, hook in mod.extensions.items():
# get class object
hook_cls = mod.__dict__[hook]
try:
base = globals()[k]
except KeyError:
print "No such class to insert hooks to:", k
else:
for item in base.__dict__:
if item.endswith('Hook'):
# Override original (no-op) hook method
# uhh.. kludgety kludge
base.__dict__[item] = hook_cls.__dict__[item]
now, my plugindemo.py just does as follows:
find_plugins()
p = Printer()
p.printit('Hello, world!')
which prints
$ python2.4 plugindemo.py
HELLO, WORLD!
Called afterPrintHook with msg HELLO, WORLD!
But hey, this has many downsides. First off, mechanism doesn't support
arbitrary namespaces. Here, class identifier in the plugin must be as it is
seen from the module which calls the plugin (not a problem for me, but could
be more elegant; probably a mapping between logical class identifiers and
actual class names, hmm?). Second, if one wants to use many hooks (with
priority for conflicts), it is not possible now; set_hooks always overrides
potentially existing hooks. And probably many other problems that are not
obvious to me, but for the simple purpose I have in mind, it seems to work.
This is the first plugin system in Python I'm writing, so I can be a way off
the correct path..
--
http://mail.python.org/mailman/listinfo/python-list
Re: Looking for info on Python's memory allocation
Steven D'Aprano <[EMAIL PROTECTED]> writes: > I'm discussing memory allocation techniques with somebody, and I'm trying to > find a quote from -- I think -- Tim Peters where he discusses the way Python > allocates memory when you append to lists. In basic terms, he says that every > time you try to append to a list that is already full, Python doubles the size > of the list. This wastes no more than 50% of the memory needed for that list, > but has various advantages -- and I'm damned if I can remember exactly what > those advantages were. That method of allocating memory is not used only in Python. The idea is that if you double the amount of memory always allocated, the interval between allocations grows exponentially (assuming storage usage grows in linear manner). Memory allocation is often very expensive, because the os works often has to seek for large enough free block to allocate for application. What is even more expensive is joining of free blocks which happens every now and then. I guess you could do the same by tripling memory usage every time you need more memory. This would reduce number of allocations needed even more. But the problem is you'd waste even more memory - 2/3 actually. So, doubling the size of chunks is used, and the technique is quite common. -- # Edvard Majakari Software Engineer # PGP PUBLIC KEY available Soli Deo Gloria! You shouldn't verb verbs. -- http://mail.python.org/mailman/listinfo/python-list
Re: What XML lib to use?
Kalle Anke <[EMAIL PROTECTED]> writes:
> I'm confused, I want to read/write XML files but I don't really understand
> what library to use.
>
> I've used DOM-based libraries in other languages, is PyXML the library to
> use?
It depends. Like there's no best car - "best" is very dependant on use of the
vehicle concerned in addition to personal preferences - there's no best XML
module either. Some seem very well in many respects, though :)
I recommend using EffBot's ElementTree. It's very simple to use (you get to do
stuff without thinking delicacies of parsing/generating), and it is
_fast_. Now let me repeat the last part - normally speed is of no concern
with the computers we have nowadays, but using eg. xml.minidom to process
files of size > 10 MB, your system might get very sluggish unless you are
quite careful in traversing the parse tree (and maybe even then).
Using a SAX / full-compliant DOM parser could be good for learning things,
though. As I said, depends a lot.
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: time.mktime problem
"McBooCzech" <[EMAIL PROTECTED]> writes:
> ===snip===
> Values 100-1899 are always illegal.
> .
> .
> strptime(string[, format])
> .
> .
> The default values used to fill in any missing data are:
> (1900, 1, 1, 0, 0, 0, 0, 1, -1)
> ===snip===
>
> BTW, check the following code:
>>>import datetime, time
>>>print time.gmtime(time.mktime((1900, 1, 1, 0, 0, 0, 0, 1, -1)))
> (1901, 12, 13, 20, 45, 52, 4, 347, 0)
>
> but (1900, 1, 1, 0, 0, 0, 0, 1, -1) is (IMHO) expected Hmmm. But I
> am just a newbie!!! :)
You are comparing apples and oranges here. You checked documentation of
strptime, and the problem is in the use of time.mktime().
The point: time.mktime() returns Epoch time (seconds since 1970) and you are
passing it a tuple which is (way before) 1970. There is no such thing as
negative epoch. It is like computing packaging day of milk which hasn't been
milked from the cow yet :)
I really wonder what version of Python you are running:
>>> import datetime, time
>>> print time.gmtime(time.mktime((1900, 1, 1, 0, 0, 0, 0, 1, -1)))
Traceback (most recent call last):
File "", line 1, in ?
OverflowError: mktime argument out of range
Python 2.3 and 2.4 both give the same error. As for the python version 2.2, no
datetime module was implemented.
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: time.mktime problem
"McBooCzech" <[EMAIL PROTECTED]> writes:
> Hi, on Linux (Fedora FC4) and Python 2.4.1
> I am trying to know the time delta in seconds between two times given
> in the HHMMSS format. My code looks like:
>
> import datetime, time
> ta1=(time.strptime('01', '%H%M%S'))
> ta2=(time.strptime('230344', '%H%M%S'))
> t1=time.mktime(ta1)
> t2=time.mktime(ta2)
> print t1, t2
>
> -2147483648.0 -2147483648.0
>
> I just can not figure out, why the t1 and t2 are the same?
Hm. You are trying to convert (1900, 1, 1, 0, 0, 1, 0, 1, -1) to epoch.
However, epochs start from 1970-01-01 00:00. So that at least is not right.
Hint... see what var ta1 is. With python2.3 you'll get overflow error, becuase
mktime argument is out of range.
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: HTML/text formatting question
"Dr. Who" <[EMAIL PROTECTED]> writes:
> This seems clunky and my next step was going to be to define generic
> functions which would generate the surrounding html tags only when
> passed the proper argument. I was wondering if there was a better way
> to do this with a standard Python library. It looked like formatter
> might but that it also might be too low-level.
You could use something like this:
class HTMLFormatter:
def __init__(self, tag, contents=None, **kwargs):
self.tag = tag
self._content = contents
self.attrs = dict()
self._set_attrs(kwargs)
def _set_attrs(self, attrs):
self.attrs = attrs
if '_class' in self.attrs:
self.attrs['class'] = self.attrs['_class']
del self.attrs['_class']
def set_content(self, contents, **kwargs):
"""
Set content of HTML element to contents.
>>> f = HTMLFormatter('a')
>>> f.set_content('cat', href='http://www.cat.org')
>>> str(f)
'http://www.cat.org";>cat'
>>> str(HTMLFormatter('td', 'cat'))
'cat'
>>> str(HTMLFormatter('p', 'kitty kit', _class='cat'))
'kitty kit'
>>> str(HTMLFormatter('br'))
''
"""
self._content = contents
if kwargs:
self._set_attrs(kwargs)
def set_attribute(self, attr, val):
"""Set/update attribute 'attr' to 'val'."""
self.attrs[attr] = val
def add_content(self, contents):
"""Add content to element.
>>> p = HTMLFormatter('p', 'name of the cat is ')
>>> p.add_content('meow')
>>> str(p)
'name of the cat is meow'
>>> p = HTMLFormatter('td')
>>> p.add_content('cat')
>>> str(p)
'cat'
"""
if self._content is None:
self._content = ''
self._content = "%s%s" % (self._content, str(contents))
def contents(self):
"""Get contents of object.
>>> p = HTMLFormatter('p', 'nice doggy dog')
>>> p.contents()
'nice doggy dog'
>>> p.add_content(HTMLFormatter('em', 'called wuff'))
>>> p.contents()
'nice doggy dogcalled wuff'
"""
return self._content
def __str__(self):
open_tag = '%s' % self.tag
if self.attrs:
attrs = self.attrs.items()
attrs.sort()
attrs_str = ' '.join(['%s="%s"' % (k, v) \
for k,v in attrs])
open_tag = '%s %s' % (self.tag, attrs_str)
if self._content is not None:
return '<%s>%s' % (open_tag, self._content, self.tag)
else:
return '<%s/>' % open_tag
Doctest strings show examples how to use it. For serious HTML building stuff
it needs fiddling with, but should be handy for tiny projects.
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: How can I import a py script by its absolute path name?
"J.Bijsterbosch" <[EMAIL PROTECTED]> writes: > Hmm, what you call special treatment comes from pythons deep underlying C > and C++ language heietidge I presume. A backslash in a C or C++ string means > the following character is a so called escape character, like \n represents > a newline and \r a return to the beginning of a line. > If you really want a backslash you need to type it twice like so \\. Has > nothing to do with Windows...;-)) Yes, I'm well aware of that. However, you can say that using '\' as a path separator needs special treatment, because it is conventionally treated as an escape character. Moreover, I wans't the one asking for information, I have privilidge to use real operating systems as a programming platform. Thanks for enthsiasm, though :) -- # Edvard Majakari Software Engineer # PGP PUBLIC KEY available Soli Deo Gloria! You shouldn't verb verbs. -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I import a py script by its absolute path name?
Thorsten Kampe <[EMAIL PROTECTED]> writes:
> "sys.path.append('c:\\xxx\\yyy')" or "sys.path.append('c:/xxx/yyy')"
Well, of course. As I said, it was untested :) I just copied the path string,
and didn't remember Windows uses path names which need special
treatment. One more reason to avoid inferior platforms :->
--
#!/usr/bin/perl -w
$h={23,69,28,'6e',2,64,3,76,7,20,13,61,8,'4d',24,73,10,'6a',12,'6b',21,68,14,
72,16,'2c',17,20,9,61,11,61,25,74,4,61,1,45,29,20,5,72,18,61,15,69,20,43,26,
69,19,20,6,64,27,61,22,72};$_=join'',map{chr hex $h->{$_}}sort{$a<=>$b}
keys%$h;m/(\w).*\s(\w+)/x;$_.=uc substr(crypt(join('',60,28,14,49),join'',
map{lc}($1,substr $2,4,1)),2,4)."\n"; print;
--
http://mail.python.org/mailman/listinfo/python-list
Re: How can I import a py script by its absolute path name?
could ildg <[EMAIL PROTECTED]> writes:
> I want to import c:\xxx\yyy\zzz.py into my programme,
> What should I do?
> Thank you~
import sys
sys.path.append('c:\xxx\yyy')
import zzz
(Untested, similar idiom would work in *nix systems, never programmed in
Windows)
However, I guess it is not very usual you should need to import stuff from
arbitrary locations. Consider publishing those modules in normal Python
include path (just see what ''print sys.path'' produces)
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: all possible combinations
John Machin <[EMAIL PROTECTED]> writes: >>>My list is not arbitrary. I'm looking for all 'combinations' as I >>>originally posted. Order does not matter to me... just all possibilities. >> That's good, since you only need combinations of "a", "b" and "c" the > "You keep using that word. I do not think it means what you think it means." Inconceivable! -- # Edvard Majakari Software Engineer # PGP PUBLIC KEY available Soli Deo Gloria! "Debugging is twice as hard as writing the code in the firstplace. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." -- Brian W. Kernighan -- http://mail.python.org/mailman/listinfo/python-list
Re: Should I use "if" or "try" (as a matter of speed)?
Peter Hansen <[EMAIL PROTECTED]> writes: >> "first make it work, then make it right, then make it fast" ... > The expression describes (most recently, if not originally) the practice in > Test-Driven Development (TDD) of making your code pass the test as quickly as > possible, without worrying about how nice it is. Ack(nowledged). > The "right" part doesn't refer to correctness, but to structure, style, > readability, and all those other nice things that an automated test can't > check. You aren't doing it "wrong" at first, just expediently. Yes, that I understood; if the first version worked, it had to be correct already. But as I said, if you want to make ideas easy to remember, you have to make them short enough, and you can probably assume the reader understands more than what is explicitly stated. I didn't know the expression originates from TDD, that puts it in a bit different light - and makes it more understandable IMO. > And it really does make sense, because at that early stage, you aren't even > absolutely certain that your test is entirely correct, so making your code a > paragon of elegance is a potential waste of time, ^^^ :-D Which is a seductive trap, that.. really, I mean, how many times you've polished a module so much that you would want to publish it in every single article you write about computing as an ideal example, one you care about and nurture like it was your own child (or your fancy-schmancy, model '74 V12-engine, chrome-plated, mean monster-of-a-vehicle car, if you are one of those types)? Then you report your progress to your superior and feel ashamed because the only thing you've worked with in last 3 weeks is that (my) precious(!) module.. hum. But I digress. > and distracting. Once you've been able to pass that test (and all the > others, since you have to make sure all previous tests still pass as well), > then and only then is it sensible > -- and required! -- to refactor the code to make it elegant, concise, clean, > etc. Yep. And thats one of the reasons I really like TDD and unit testing - you know when to stop working with a piece of code. When all the tests pass, stop. > Of course, your point about temptation is sound. Extreme Programming tries > to avoid that problem partly by pairing programmers together, and it is the > responsibility of both partners to encourage^H^H^H^H^H insist that the > refactor "make it right" stage must occur _now_, before we check the code > in. If you skip this step, you're failing to be an agile programmer, and > your code base will become a tar pit even more quickly than it would in a > traditional (non-agile) project... Yup. Too bad I've had the opportunity to work that way (pair programming) only few times, and even then it wasn't XP-style in any other way. It is too often considered waste of labour, I guess. -- # Edvard Majakari Software Engineer # PGP PUBLIC KEY available Soli Deo Gloria! "Debugging is twice as hard as writing the code in the firstplace. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." -- Brian W. Kernighan -- http://mail.python.org/mailman/listinfo/python-list
Re: Should I use "if" or "try" (as a matter of speed)?
Steven D'Aprano <[EMAIL PROTECTED]> writes:
> My opinion is, no, you don't need to be a C programmer, or an assembly
> programmer, or a hardware level physicist who understands NAND gates, but
> it is very useful to have some understanding of what is going on at the
> low-level implementation.
Yes, I fully agree: in the example presented, it is sufficient to understand
that string concatenation is (relatively) expensive. Yet I'd emphasize that
most often speed is improved by better algorithms, not by low-level
optimisations and language-specific features (if speed is even an issue, that
is).
> The way I see it, programmers need to be somewhat aware of the eventual
> optimization stage in their program, so as to avoid poor design choices
> from the start. But you can't always recognise poor design up front, so
> even more important is careful encapsulation and design, so you
> can make significant implementation changes without needing to throw away
> your work. (Well, that's the theory.)
So true, extra emphasis on encapsulation and independence. Even seasoned
professionals fail to create dazzling products at version 1.0. Good component
design is crucial because you eventually want to do major rewrites later.
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Reading network interface data in linux
Suppose one wants to fetch the following data from given network interface,
say, eth0:
>>> Ethinf('eth0').addr()
'192.168.1.42/24'
>>> Ethinf('eth0').route('default')
'192.168.1.1'
>>> Ethinf('eth0').duplex()
'full'
>>> Ethinf('eth0').speed()
100
Some statistics:
>>> Ethstat('eth0').rx_bytes()
14325235341223
>>> Ethstat('eth0').tx_bytes()
2513152423
One could implement modules by installing eg. ethtool and reading speed/duplex
information by parsing ethtool output, but it is ugly way to do it, prone to
errors, requires launching a process as well as installation of ethtool.
As for the byte counts, you could get all information ifconfig show by reading
/proc/net/dev (which is more nice to parse), but it seems like those counters
are 32-bit, and they wrap around quite quickly in a 1000 gbit network (~30
seconds).
I was wondering is there really no module implemented which already does those
things?
I've already implemented a proto by doing exactly as I told (parsing ethtool &
/proc/net/dev), but I strive for more elegant solution. I'm willing to join
the effort with someone else, if possible.
--
#!/usr/bin/perl -w
$h={23,69,28,'6e',2,64,3,76,7,20,13,61,8,'4d',24,73,10,'6a',12,'6b',21,68,14,
72,16,'2c',17,20,9,61,11,61,25,74,4,61,1,45,29,20,5,72,18,61,15,69,20,43,26,
69,19,20,6,64,27,61,22,72};$_=join'',map{chr hex $h->{$_}}sort{$a<=>$b}
keys%$h;m/(\w).*\s(\w+)/x;$_.=uc substr(crypt(join('',60,28,14,49),join'',
map{lc}($1,substr $2,4,1)),2,4)."\n"; print;
--
http://mail.python.org/mailman/listinfo/python-list
Re: removing list comprehensions in Python 3.0
Steven Bethard <[EMAIL PROTECTED]> writes: > $ python -m timeit "for x in (i for i in xrange(10)): y = x" > 10 loops, best of 3: 4.75 usec per loop Yowza! One of the features I really liked in Perl has shored Python island somewhere in the 2.4'ies, it seems[1]. Thanks for the tip! PS. In case it wasn't clear what I referred to, it was the ability to run given module as a script. Of course you could supply full path to timeit.py: $ python2.3 /usr/lib/python2.3/timeit.py \ "for x in [i for i in xrange(10)]: y = x" 10 loops, best of 3: 9.96 usec per loop But using -m makes it much more convenient. Footnotes: [1] Well, not exactly equal to -M in Perl, but close enough for timing stuff -- # Edvard Majakari Software Engineer # PGP PUBLIC KEY available Soli Deo Gloria! You shouldn't verb verbs. -- http://mail.python.org/mailman/listinfo/python-list
Re: Should I use "if" or "try" (as a matter of speed)?
Thorsten Kampe <[EMAIL PROTECTED]> writes:
> Speed considerations and benchmarking should come in after you wrote
> the program. "Premature optimisation is the root of all evil" and
> "first make it work, then make it right, then make it fast" (but only
> if it's not already fast enough) - common quotes not only with Python
> developers.
Just a minor note: regarding quote
"first make it work, then make it right, then make it fast"
Shouldn't one avoid doing it the wrong way from the very beginning? If you
make it "just work" the first time, you'll probably use the old code later on
because "functionality is already there" and temptatation to build on probably
relatively bad architecture can be too strong.
How about
First make it work (but avoid ad-hoc designs), then make it right, then make
it fast
Of course, such emphasis doesn't go well with classic idioms..
(yeah, programmer's block at the moment: I should clean up a 120+ -line
if-elif-elif-elif... else -block which tests a single variable and calls
different methods with variable number of parameters depending on the value of
the variable - guess I should apply command pattern or similar...)
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: Favorite non-python language trick?
Simon Brunning <[EMAIL PROTECTED]> writes:
> http://wiki.python.org/moin/PythonDecoratorLibrary?#head-de01988728ccdec415708f10928cc6feb022e7bb
Neat.
I guess about 75% about programming-related things classified as neat-o or
"convenient!" are already implemented by some Pythonista(s). Spoils all the
fun for reinventing the wheel, doesn't it. :)
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: Favorite non-python language trick?
Thomas Heller <[EMAIL PROTECTED]> writes:
> I don't see what's wrong with this code, and if one wanted, one could
> also implement a decorator which calls warnings.warn when the function
> is called:
>
> def c_buffer(init, size=None):
> "deprecated, use create_string_buffer instead"
> import warnings
> warnings.warn("c_buffer is deprecated, use create_string_buffer instead",
> DeprecationWarning, stacklevel=2)
> return create_string_buffer(init, size)
Well, nothing's wrong there, and the same could be done with Java
before. However, having a consistent deprecated string everywhere allows
easier eg. automatic finding of such methods from documentation.
Decorators also help here, but that requires version 2.3 or newer (which
usually isn't a problem, but can be)
Hey! I hadn't realized category parameter nor stacklevel in warnings module
(just used a few times, never read the doc because I didn't need to). Neat,
thanks.
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: Favorite non-python language trick?
(sorry, my NUA had lost the original article)
>
>> I'm curious -- what is everyone's favorite trick from a non-python
>> language? And -- why isn't it in Python?
Ability to tag some methods 'deprecated' as in Java (from 1.5
onwards?). However, Python interpreter doesn't have to do it: pydoc and
similar tools could detect, say, '@deprecated' in method comment string and
warn user about it.
Currently I just document deprecated methods, and if I feel like it, I also
add
def some_method_which_is_badly_named_or_just_plain_wrong(..)
"""docstring
This method is now deprecated. Use frob() instead.
"""
sys.stderr.write('warning: method
some_method_which_is_badly_named_or_just_plain_wrong is now deprecated')
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
You shouldn't verb verbs.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Programmers Contest: Fit pictures on a page
Dan Sommers <[EMAIL PROTECTED]> writes:
> There's some sort of irony or something in there about not writing the
> best genetic algorithm, but I can't quite put my finger on it.
+1 QOTW :)
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: Escaping commas within parens in CSV parsing?
[EMAIL PROTECTED] writes:
> I am trying to use the csv module to parse a column of values
> containing comma-delimited values with unusual escaping:
>
> AAA, BBB, CCC (some text, right here), DDD
>
> I want this to come back as:
>
> ["AAA", "BBB", "CCC (some text, right here)", "DDD"]
Quick and somewhat dirty: change your delimiter to a char that never exists in
fields (eg. null character '\0').
Example:
>>> s = 'AAA\0 BBB\0 CCC (some text, right here)\0 DDD'
>>> [f.strip() for f in s.split('\0')]
['AAA', 'BBB', 'CCC (some text, right here)', 'DDD']
But then you'd need to be certain there's no null character in the input
lines by checking it:
colsep = '\0'
for field in inputs:
if colsep in field:
raise IllegalCharException('invalid chars in field %s' % field)
If you need to stick with comma as a separator and the format is relatively
fixed, I'd probably use some parser module instead. Regular expressions are
nice too, but it is easy to make a mistake with those, and for non-trivial
stuff they tend to become write-only.
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: python install settings...
"jtan325" <[EMAIL PROTECTED]> writes:
> customize_compiler
> cc_cmd = cc + ' ' + opt
> TypeError: cannot concatenate 'str' and 'NoneType' objects
[...]
> upon closer inspection of Python's distutils sysconfig.py, is the error
> being caused by the ' ' in "cc_cmd = cc + ' ' + opt"? Any ideas on this
> new error? Are there packages/settings I need to take care of before i
> can use Python's distutils to install stuff?
The error message indicates that one of the operands on the right side is
None. ' ' is clearly a string constant (string with one whitespace), so the
problem must be either cc or opt - one of them is None.
Toying a little I realized Python emits different warnings for pairs
cc, opt = None, 'cat' and
cc, opt = 'cat', None:
>>> cc, opt = None, 'cat'
>>> cc_cmd = cc + ' ' + opt
Traceback (most recent call last):
File "", line 1, in ?
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
# note the different TypeError message below
>>> cc, opt = 'cat', None
>>> cc_cmd = cc + ' ' + opt
Traceback (most recent call last):
File "", line 1, in ?
TypeError: cannot concatenate 'str' and 'NoneType' objects
So, for some reason, variable opt is None.
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: Boss wants me to program
phil <[EMAIL PROTECTED]> writes:
> From 30 years of application development experience I will tell you
> NOT HUMBLY, that Python is easily the most productive, the most read-write
> and the most elegant of any of the above. Handsdown better than Java, the
> runner up in that group.
I don't want to start a flamewar here - but I would like to point out that not
only the language you use affects productivity, but also tools that support
working with the language affect a lot too. For example, I once thought I
would start writing any documents longer than say, two A4 pages with lout
(I've used LaTeX so far). However, realising how much more supporting tools,
add-ons and utilities LaTeX had, I stayed with that (moreover, some of the
other people already new LaTeX but didn't know lout).
Recently I participated in creating a visual FSM editor as Eclipse plugin. I
hadn't used Eclipse before, but seeing how easy it was to create tests,
refactor code (bicyclerepairman is not even close to features offered by
Eclipse) and use gazillion other tools designed to improve Java productivity
made me realise the language has a really, really great selection of tools and
utilities available.
Now, I earn my bread by coding Python and I do like coding in Python the most,
but sometimes I think I would have been better off with Java - not because of
the language, but because of the environment and sheer selection of tools
available.
Let me emphasize a little more. Even though Python itself is great, I think we
don't have quite yet tools that offer
* Industrial-grade reverse-engineering tool (ie. automatic UML diagram
generation out of code) which also supports creating classes/interfaces out
of UML diagrams, and modifies the other automatically when the other changes
* Automatic unit test case generation (pydev is going to this direction, I
think)
* Decent code coverage tools - and I don't mean statement coverage, but path
coverage or multi-condition coverage
Just see how many handy tools there are for Java if you use Eclipse:
http://eclipse-plugins.2y.net/eclipse/plugins.jsp
(Yes, I know that many of those plugins are not related to any language but
Eclipse and that some of the plugins are specifically Python related, but most
of the good stuff is for Java Development)
Pydev looks really promising, though. With Eclipse, I think it is a very good
alternative to commercial Python IDEs and could mature to the Other Way(TM)
for developing Python programs (the other is, of course, vi(m)/(X)Emacs)
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANN: PyDev 0.9.5 released
Dave Cook <[EMAIL PROTECTED]> writes:
>> PyDev - Python IDE (Python Development Enviroment for Eclipse) version
>> 0.9.5 has just been released.
>
> Does it work with the newly released Eclipse 3.1?
Seems to work for me (but I've only coded one smallish Python program with it)
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: Which kid's beginners programming - Python or Forth?
[EMAIL PROTECTED] writes:
> Ivan Van Laningham wrote:
> [...]
>>
>> Seriously, PostScript is a lot more fun to learn than Forth, and more
>> directly useful. Since the rewards are so immediate, a kid's attention
>> could be gained and kept pretty easily.
>
> PostScript is easy, but I'm afraid some technical details could get in
> the way of enjoyable exploration, e.g. font types or scaling.
> PostScript is also a single purpose language: it can print static
> graphics and with a slightly more complex setup it can display static
> graphics on the screen, period. No interactivity, no files, no network,
> no general computation or data structures.
PostScript is _not_ limited to static stuff, and it _does_ support
interactivity. See eg.
http://www.cs.technion.ac.il/~wagner/index_files/aaa.html
Of course, this is just academic fun(?). PostScript is mostly used for
printers, and as for yet, quite few papers support animated graphics :->
And yes, I also encourage to try Python.
> List comprehensions, however, *are* the basic control flow; loops are
> much more verbose and they should be used only when necessary.
Hm. My experience is that people find loops easier to understand - varies
somewhat, though. For some, 'more verbose' is 'more easy to understand'.
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
RRArray for Python?
I was wondering if there is already some 'official' module for round robin
arrays. Of course, it is relatively easy to implement your own if you need it,
but IMHO it feels like there should be one, because they are so
general-purpose.
Here's my implementation, if someone needs one. If there are enough people
supporting it's release, I could add it to some static web address.
"""
Round-robin array module
This module contains class for a round-robin array and associated round-robin
array database to contain long-time spanning sample values (cf. rrdtool).
Arrays can be arbitrary length, and default slot values can be defined by the
user.
Edvard Majakari <[EMAIL PROTECTED]>
"""
class RRArrayError(Exception): pass
class RRArray:
"""Simple round-robin class for arbitrary items"""
def __init__(self, slots, default=None):
"""Instantiate RRArray object
@param default: value to fill empty slots with
>>> r = RRArray(10) # creates a round-robin array of length 10
>>> len(r)
10
"""
self.default = default
self._slots = slots
self._array = [default]*slots
self._next = 0 # points to next free index in array
self._array_full = False
def clear(self):
"""Erase array by setting all values to default fill value"""
self._array = [self.default]*self._slots
self._next = 0
self._array_full = False
def contents(self):
"""Return contents of RRArray object as a list
so that most recent item is the last position.
>>> r = RRArray(3, 0)
>>> r.contents()
[0, 0, 0]
>>> r.insert(2)
>>> r.contents()
[0, 0, 2]
"""
retval = self.latest(self._slots)
retval.reverse()
return retval
def insert(self, item):
"""Insert an item to object
>>> r = RRArray(3)
>>> r.insert(42)
>>> r.most_recent() == 42 and r.used_slots() == 1
True
"""
self._array[self._next % self._slots] = item
if self._next == self._slots - 1:
self._array_full = True
self._next = 0 # wrap-around
else:
self._next += 1
def latest(self, count):
"""Return count most recent items
Note that it is possible to receive default values which were never
inserted, eg.
r = RRArray(5)
r.insert(2), r.latest(3) # would return [2, None, None]
>>> r = RRArray(3)
>>> r.insert(42)
>>> r.insert(5)
>>> r.insert(7)
>>> r.insert(11)
>>> r.latest(2)
[11, 7]
>>> r.contents()
[5, 7, 11]
"""
if count > self._slots:
err = "attempted to get %d items from rra of size %d"
raise RRArrayError(err % (count, self._slots))
latest_idx = self._latest_index()
head = self._array[0:latest_idx+1]
head.reverse()
if count >= len(head):
tail = self._array[latest_idx+1:self._slots]
tail.reverse()
head.extend(tail)
return head[0:count]
def most_recent(self):
"""Return most recent item inserted.
An IndexError is raised if no items have been inserted yet.
"""
if self._next == 0 and not self._array_full:
raise IndexError("no items inserted yet")
return self._array[(self._next - 1) % self._slots]
def used_slots(self):
"""Return number of used slots."""
if self._next < self._slots and not self._array_full:
return self._next
else:
return self._slots
### private and special methods
def __len__(self):
"""Return number of slots in the object."""
return self._slots
def _latest_index(self):
return (self._next - 1) % self._slots
def _test():
import doctest, rrarray
doctest.testmod(rrarray)
if __name__ == '__main__':
_test()
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
"Debugging is twice as hard as writing the code in the firstplace. Therefore,
if you write the code as cleverly as possible, you are, by definition,
not smart enough to debug it." -- Brian W. Kernighan
--
http://mail.python.org/mailman/listinfo/python-list
Re: how to accomplish dl progress bars?
"flamesrock" <[EMAIL PROTECTED]> writes:
> Certain web applications, everything from wget to downloader for X has
> this nifty feature I'd like to accomplish in python.
>
> Its the progress bar/time elapsed/time remaining scheme
>
> Filename | Progress| Speed (kB/s) | T/Elapsed | T/Remaining
>
> blah.bin |-->45% | 56.6 | 02:39 | 02:45
> file2.bz2 |--->33% | 35.0 | 05:22 | 11:24
> etc |>60% | 47.5 | 00:14 | 00:08
See python cookbook. The idea is simply to print out stuff and when updating,
erase previous characters eg. using '\r' ASCII code. Python Cookbook has a
progress bar implementation, you could use that as a basis for your code
(speed, t/elapsed and t/remaining are technically no different; if you can do
a progress bar, you can do those others as well).
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: Safe eval, or how to get list from string
[EMAIL PROTECTED] writes:
> I've to use ConfigParser.
>
> It returns values that are exactly in the config file, so get string
> variables like:
> int1 with quotes and characers: "42"
> this is easy to convert to int:
> realint = int(int1)
There's already a method getint() for that, you can just say
cfgparser.getint('section', 'int1')
> if there is list1 = "[ 'filea', 'fileb', ]", I can get at list by
> doing:
> reallist = eval(list1)
Are you sure you need to use ConfigParser for that? If you need to have
persistent Python data structures, I'd suggest marshalling data and using
(c)Pickle or similar.
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
"Debugging is twice as hard as writing the code in the firstplace. Therefore,
if you write the code as cleverly as possible, you are, by definition,
not smart enough to debug it." -- Brian W. Kernighan
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python features
"Fredrik Lundh" <[EMAIL PROTECTED]> writes:
> does that apply to sturgeon's law itself?
>
> (fwiw, this is of course why google displays 10 results on the
> first page. according to the law, one of them is always exactly
> what you want).
+ 1 QOTW :-)
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: Unique Elements in a List
Scott David Daniels <[EMAIL PROTECTED]> writes:
> Normally this is called a polynomial, rather than exponential increase.
> Exponential increases are typically of the form (C^N) (they are all
> equivalent).
> Polynomial times are hallways characterized by their largest exponent,
> So you never call something O(N^3 - N^2) Since, as N gets large enough,
> The N^2 term shrinks to non-existence.
Yup, you are of course, completely correct. I was thinking of "exponent here
is two" and mistakenly named in exponential.
my_text.replace('exponent','polynom'), there :)
Reminding of ignoring terms with smaller exponent was good, too.
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: Unique Elements in a List
[EMAIL PROTECTED] (Aahz) writes:
>>[x for x in data if data.count(x) == 1]
>>
>>suffice? it is also "stable" preserving order of items. Lemme demo:
>
> Only for small datasets -- this is an O(N^2) algorithm.
I realized that, but maybe I should've pointed it out too. For the OP if
he/she is unaware - notation O(N^2) (big O n squared) means the computing time
of the algorithm increases exponentially (where exponent is 2) relative to the
size of the input.
Eg. if the input size is i and it takes p seconds to compute it, then given
input size 10*i, the computing time would be 100*p. These notions can apply
for memory usage as well, but the problem in here is the computing time:
list.count() must iterate through the list each time, and as such the loop
[x for x in data if data.count(x) == 1]
iterates through each item in data (x for x in data), and for each item it
will again iterate through each item in data to see how many times it
occurred. If data contains 200 items, this idiom would iterate the structure
40 000 times. With today's computers one wouldn't notice it, unless each item
requires heavy processing (eg. launching a separate process per item
etc). However, if the length of the data can be thousands or even tens of
thousands, this idiom would become unusable. If data contained 75 000 items,
the loop would do 25 625 000 000 iterations, effectively bringing cpu to
halt..
So, generally one should avoid using exponential O(n^k) (where k > 1)
algorithms, unless faster O(n) or O(n*lg(n)) solution is either impossible (or
too complex, and inputs are known to be small etc).
Wikipedia has good resources and pointers to related things, see
http://en.wikipedia.org/wiki/Analysis_of_algorithms
--
#!/usr/bin/perl -w
$h={23,69,28,'6e',2,64,3,76,7,20,13,61,8,'4d',24,73,10,'6a',12,'6b',21,68,14,
72,16,'2c',17,20,9,61,11,61,25,74,4,61,1,45,29,20,5,72,18,61,15,69,20,43,26,
69,19,20,6,64,27,61,22,72};$_=join'',map{chr hex $h->{$_}}sort{$a<=>$b}
keys%$h;m/(\w).*\s(\w+)/x;$_.=uc substr(crypt(join('',60,28,14,49),join'',
map{lc}($1,substr $2,4,1)),2,4)."\n"; print;
--
http://mail.python.org/mailman/listinfo/python-list
Re: Module on Getting the Date & Time
Sara Khalatbari <[EMAIL PROTECTED]> writes:
Hi,
> Is there a Modules in Python that returns the time &
> date of today when ran?
You mean a method in a module? Yes, there's good 'ol time module, which will
probably provide what you need:
>>> import time
>>> time.time() # number of seconds (+ fract) elapsed after 01.01.1970
1115881981.3717401
>>> time.ctime() # current time in string format
'Thu May 12 10:13:20 2005'
>>> time.localtime() # current time as a tuple
(2005, 5, 12, 10, 13, 30, 3, 132, 1)
For details, see 'pydoc time'
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: Unique Elements in a List
"Michael J. Fromberger" <[EMAIL PROTECTED]> writes:
> One reasonable solution might be as follows:
>
> def unique_elts(seq):
> elts = {}
> for pos, elt in enumerate(seq):
> elts.setdefault(elt, []).append(pos)
>
> return [ (x, p[0]) for (x, p) in elts.iteritems()
> if len(p) == 1 ]
This is neat. Being lazy (the wrong way) I've never payed much attention why
would I need dict.setdefault() when I have things as dict.get(k, [def]). This
was a nice example of good use for setdefault() because it works like
dict.get() except it also sets the value if it didn't exist.
+1 IOTW (idiom of the week).
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: Unique Elements in a List
James Stroud <[EMAIL PROTECTED]> writes: > from sets import Set > > data = [0.1,0.5,0.6,0.4,0.1,0.5,0.6,0.9] > > [x for x in Set(data) if data.count(x) == 1] Um. ...I must have missed something, but I'll post nevertheless: wouldn't just [x for x in data if data.count(x) == 1] suffice? it is also "stable" preserving order of items. Lemme demo: >>> [x for x in Set(data) if data.count(x) == 1] [0.90002, 0.40002] >>> [x for x in data if data.count(x) == 1] [0.40002, 0.90002] Though I'll admit I also thought of Sets first, because I didn't remember there was such a nice method list.count(). -- # Edvard Majakari Software Engineer # PGP PUBLIC KEY available Soli Deo Gloria! One day, when he was naughty, Mr Bunnsy looked over the hedge into Farmer Fred's field and it was full of fresh green lettuces. Mr Bunnsy, however, was not full of lettuces. This did not seem fair. --Mr Bunnsy has an adventure -- http://mail.python.org/mailman/listinfo/python-list
Re: Design advice for unit test asserters
"Gary" <[EMAIL PROTECTED]> writes: Hi again, > First, thanks to both Kent and Edvard for useful comments. > I certainly need to consider whether it make sense to switch to py.test at > this time; its simplicity is attractive. For what it's worth, I've also coded a simple py.test skeleton generator, which does have some funky features like creating working tests using existing doctest strings in the module. For simple cases, tests work out of the box, but mostly I use it to save typing. Of course, you'd be better off using TDD, but the program can help you there too; it is able to create a module skeleton out of py.test suite, though it is not very mature yet. You can grab it from http://majakari.net/dl/pytestgen/ if you wish to try it. -- # Edvard Majakari Software Engineer # PGP PUBLIC KEY available Soli Deo Gloria! One day, when he was naughty, Mr Bunnsy looked over the hedge into Farmer Fred's field and it was full of fresh green lettuces. Mr Bunnsy, however, was not full of lettuces. This did not seem fair. --Mr Bunnsy has an adventure -- http://mail.python.org/mailman/listinfo/python-list
Re: Design advice for unit test asserters
"Gary" <[EMAIL PROTECTED]> writes:
[ py.test ad follows :) ]
> def test_SomeTest(...):
> ...
> self.AssertAllFilesExist(fileList)
>
> or
>
> def test_SomeTest(...):
> ...
> [ self.AssertFileExists(f) for f in fileList ]
I prefer the latter, because then you'd get error for the missing file -
otherwise you'd just know a file didn't exist (but not which, unless you
explicitly coded that in, no?)
with py.test you could just write:
def file_exists(fname):
assert os.path.isfile(fname)
def test_file_exists():
for f in fileList:
yield file_exists, f
that way you'd generate a separate test for each file in fileList, and easily
see which file would fail the test. :)
no comment for the rest of your posts though - I'm still learning how to
create good unit tests myself (and I also see you're using standard unit test
modules - I'm already seduced by the no-API py.test library...)
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: Creating module skeleton from unit tests
Fabio Zadrozny <[EMAIL PROTECTED]> writes:
> I think that the best approach I saw to this was in the Eclipse java
> ide... You can basically go on the declaration of
>
> self.obj = player.Player('Fred the Adventurer')
>
> press Ctrl+1 and it adds a suggestion to create the class Player.
>
> Then go to
>
> assert self.obj.name == 'Fred the Adventurer'
>
> press Ctrl+1 and it adds suggestion: Declare field name in class
> Player... and so on for methods... (true, you still have to go and press
> some Ctrl+1s, but that should be fairly easy, especially if you had some
> hints on what is missing... Python has a very dynamic nature, but most of it
> can still be done...
Yes, I know. Eclipse is an excellent Java IDE - I've already seen ''quick
assist'' at work and I envy those Eclipse users for having such a nice tool :)
PyDev for eclipse seems /very/ promising, though.
> I think that most Python IDEs are still not in the same level, but some day
> they might get there... Being the maintaner of PyDev (http://pydev.sf.net),
> I think it will get there someday, true, lots of work to make it happen,
> right now only few things in Ctrl+1 are available like that (still, some
> already are)... and that's the way things work... nothing's always perfect
> (but at least they evolve).
Thanks for the comments - and for your pydev plugin, too.
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
"Debugging is twice as hard as writing the code in the firstplace. Therefore,
if you write the code as cleverly as possible, you are, by definition,
not smart enough to debug it." -- Brian W. Kernighan
--
http://mail.python.org/mailman/listinfo/python-list
Re: Creating module skeleton from unit tests
Peter Maas <[EMAIL PROTECTED]> writes:
> I think this is too difficult, because there are many ways to write
> code (even skeletons) for a use case. An easier approach would
> be to write the skeleton manually, embed the test cases in the doc
> strings and generate the test code from the doc strings. If I
> remember correctly IBM has published something to generate unit
> tests from code. Python has a doctest module to support testing
> derived from doc strings. This can be combined with unit tests.
Yes - actually I channged my mind in somewhere in the article - I actually
don't want to create typical use-cases in the test bed, rather create a
skeleton which covers each method at least somehow (no code inside skeleton
would be needed).
> > The problem can be solved more easily if you design the module
>> skeleton first, then the tests and then the logic for the skeleton
>> - you would be creating tests before the code, but many people
> > wouldn't regard it as TDD then.
> You shouldn't care if your approach works for you.
Yes, you are right. I didn't select my words very well - I just meant that it
wouldn't be TDD then. Of course it might work, but I'd like to to it the TDD
way for now.
But thanks for the tip, I could see what IBM has done and then forget about
doing it automatically :)
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Creating module skeleton from unit tests
o follow).
What do you think? I know that probably the best way to go on is just do it
the old way (ie. code both the test and module by hand), or look for more
intelligent IDEs.
For what it's worth, I still use most recent XEmacs for coding because it is
just so handy in many areas - sometimes I miss Eclipse-like IDE and I have
even tried pydev for Eclipse (which is a good plugin, btw!), but to this day
I've went back to XEmacs every time because in the end I've always been
lacking something I couldn't find a decent substitute for.
Obligatory note: According to the XP folks, TDD is not as much about testing
as it is about design: "Test Driven *Development*" or maybe even "Test Driven
*Design*". The problem can be solved more easily if you design the module
skeleton first, then the tests and then the logic for the skeleton
- you would be creating tests before the code, but many people wouldn't regard
it as TDD then.
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: Unit testing - one test class/method, or test class/class
"John Roth" <[EMAIL PROTECTED]> writes:
> I tend to write one test class per class, but that's
> just the way I got started. My feeling is that the
> methods in a test class should tell a story if you
> read the names in the order they were written,
> so I'd split the tests for a class into several
> classes if they had different stories to tell.
Well, that's one of the things I forgot to mention. Often I use
TestSomeClassA:
# test for normal, legal inputs
TestSomeClassB:
# test for extreme/border cases, still legal inputs
TestSomeClassC:
# test cases with illegal input, eg. negative integer when only positive
# inputs make sense etc.
Just like you said, they tell a bit different story each, so it probably
makes sense to separate those.
>
> John Roth
>
> "Edvard Majakari" <[EMAIL PROTECTED]> wrote in message
> news:[EMAIL PROTECTED]
>>
>> Hi,
>>
>> I just found py.test[1] and converted a large unit test module to py.test
>> format (which is actually almost-no-format-at-all, but I won't get there
>> now). Having 348 test cases in the module and huge test classes, I started
>> to think about splitting classes. Basically you have at least three
>> obvious
>> choises, if you are going for consistency in your test modules:
>>
>> Choise a:
>>
>> Create a single test class for the whole module to be tested, whether it
>> contains multiple classes or not.
>>
>> ...I dont think this method deserves closer inspection. It's probably
>> rather
>> poor method to begin with. With py.test where no subclassing is required
>> (like in Python unittest, where you have to subclass unittest.TestCase)
>> you'd probably be better off with just writing a test method for each
>> class
>> and each class method in the module.
>>
>> Choise b:
>>
>> Create a test class for each class in the module, plus one class for any
>> non-class methods defined in the module.
>>
>> + Feels clean, because each test class is mapped to one class in the
>> module
>> + It is rather easy to find all tests for given class
>> + Relatively easy to create class skeleton automatically from test module
>> and the other way round
>>
>> - Test classes get huge easily
>> - Missing test methods are not very easy to find[2]
>> - A test method may depend on other tests in the same class
>>
>> Choise c:
>>
>> Create a test class for each non-class method and class method in the
>> tested
>> module.
>>
>> + Test classes are small, easy to find all tests for given method
>> + Helps in test isolation - having separate test class for single method
>> makes tested class less dependent of any other methods/classes
>> + Relatively easy to create test module from existing class (but then you
>> are not doing TDD!) but not vice versa
>>
>> - Large number of classes results in more overhead; more typing, probably
>> requires subclassing because of common test class setup methods etc.
>>
>> What do you think, any important points I'm missing?
>>
>> Footnotes:
>> [1] In reality, this is a secret plot to advertise py.test, see
>> http://codespeak.net/py/current/doc/test.html
>>
>> [2] However, this problem disappears if you start with writing your tests
>>first: with TDD, you don't have untested methods, because you start by
>>writing the tests first, and end up with a module that passes the tests
>>
>>
>> --
>> # Edvard Majakari Software Engineer
>> # PGP PUBLIC KEY availableSoli Deo Gloria!
>> One day, when he was naughty, Mr Bunnsy looked over the hedge into Farmer
>> Fred's field and it was full of fresh green lettuces. Mr Bunnsy, however,
>> was
>> not full of lettuces. This did not seem fair. --Mr Bunnsy has an
>> adventure
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Re: Unit testing - one test class/method, or test class/class
aurora <[EMAIL PROTECTED]> writes:
> What I really want to bring up is your might want to look at refactoring
> your module in the first place. 348 test cases for one module sounds like a
> large number. That reflects you have a fairly complex module to be tested
> to start with. Often the biggest benefit of doing automated unit testing is
> it forces the developers to modularize and decouple their code in order to
> make it testable. This action alone improve that code quality a lot. If
> breaking up the module make sense in your case, the test structure will
> follows.
Here I have to emphasize a little: of those 348 test cases, only ~30 or so
are real, hand-coded methods. Some of the tests are generated on the fly by
py.test. It is not as fancy as it sounds, though. All it does is
test_some_feature(self):
for foo, bar, expected in known_values:
yield self.foo_bar_equals, foo, bar, expected
def foo_bar_equals(self, foo, bar, expected):
assert some_feature(foo, bar) == expected
There are two methods. However, if known_values contains 100 tuples,
py.test generates 100 "test methods" on the fly. Of course you could just do
test_some_feature(self):
for foo, bar, expected in known_values:
assert some_feature(foo, bar) == expected
but then you wouldn't see so easily which values are tested when you use
verbose mode, that is. That's one of the (many) nice things in py.test I
like :)
However, being practical in testing is probably more worth than being
completely orthodox, on that I agree. That's why I seldom stick to strict
rules in doing tests, though being systematic helps alot, especially
regarding orthogonality. It doesn't help to test same features over and
over again. Eg. if I wrote a test for a dot in 2D space, I'd write tests for
dot on origo, on positive x-axis with y < 0 and y > 0, ditto for x and y
reversed, then same tests for negative x and y, and last for positive and
negative x and y with other being exactly zero. There's no point testing
other values; all other combinations fall to some of the categories
mentioned.
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";
--
http://mail.python.org/mailman/listinfo/python-list
Unit testing - one test class/method, or test class/class
Hi, I just found py.test[1] and converted a large unit test module to py.test format (which is actually almost-no-format-at-all, but I won't get there now). Having 348 test cases in the module and huge test classes, I started to think about splitting classes. Basically you have at least three obvious choises, if you are going for consistency in your test modules: Choise a: Create a single test class for the whole module to be tested, whether it contains multiple classes or not. ...I dont think this method deserves closer inspection. It's probably rather poor method to begin with. With py.test where no subclassing is required (like in Python unittest, where you have to subclass unittest.TestCase) you'd probably be better off with just writing a test method for each class and each class method in the module. Choise b: Create a test class for each class in the module, plus one class for any non-class methods defined in the module. + Feels clean, because each test class is mapped to one class in the module + It is rather easy to find all tests for given class + Relatively easy to create class skeleton automatically from test module and the other way round - Test classes get huge easily - Missing test methods are not very easy to find[2] - A test method may depend on other tests in the same class Choise c: Create a test class for each non-class method and class method in the tested module. + Test classes are small, easy to find all tests for given method + Helps in test isolation - having separate test class for single method makes tested class less dependent of any other methods/classes + Relatively easy to create test module from existing class (but then you are not doing TDD!) but not vice versa - Large number of classes results in more overhead; more typing, probably requires subclassing because of common test class setup methods etc. What do you think, any important points I'm missing? Footnotes: [1] In reality, this is a secret plot to advertise py.test, see http://codespeak.net/py/current/doc/test.html [2] However, this problem disappears if you start with writing your tests first: with TDD, you don't have untested methods, because you start by writing the tests first, and end up with a module that passes the tests -- # Edvard Majakari Software Engineer # PGP PUBLIC KEY available Soli Deo Gloria! One day, when he was naughty, Mr Bunnsy looked over the hedge into Farmer Fred's field and it was full of fresh green lettuces. Mr Bunnsy, however, was not full of lettuces. This did not seem fair. --Mr Bunnsy has an adventure -- http://mail.python.org/mailman/listinfo/python-list
Statement coverage tools revisited
;,
'-r': 'report',
+'-v:': 'visited-prefix=',
+'-u:': 'unvisited-prefix=',
'-x': 'execute',
}
short_opts = string.join(map(lambda o: o[1:], optmap.keys()), '')
@@ -178,6 +185,9 @@
execfile(sys.argv[0], __main__.__dict__)
if not args:
args = self.cexecuted.keys()
+
+self.visited_pfx = settings.get('visited-prefix=', '>')
+self.unvisited_pfx = settings.get('unvisited-prefix=', '!')
ignore_errors = settings.get('ignore-errors')
show_missing = settings.get('show-missing')
directory = settings.get('directory=')
@@ -193,8 +203,8 @@
sys.settrace(None)
def erase(self):
-global c
-c = {}
+global coverage_dict
+coverage_dict = {}
self.analysis_cache = {}
self.cexecuted = {}
if os.path.exists(self.cache):
@@ -213,8 +223,8 @@
# exists).
def restore(self):
-global c
-c = {}
+global coverage_dict
+coverage_dict = {}
self.cexecuted = {}
if not os.path.exists(self.cache):
return
@@ -252,13 +262,13 @@
# "executed" map.
def canonicalize_filenames(self):
-global c
-for filename, lineno in c.keys():
+global coverage_dict
+for filename, lineno in coverage_dict.keys():
f = self.canonical_filename(filename)
if not self.cexecuted.has_key(f):
self.cexecuted[f] = {}
self.cexecuted[f][lineno] = 1
-c = {}
+coverage_dict = {}
# morf_filename(morf). Return the filename for a module or file.
@@ -474,17 +484,17 @@
# Special logic for lines containing only
# 'else:'. See [GDR 2001-12-04b, 3.2].
if i >= len(statements) and j >= len(missing):
-dest.write('! ')
+dest.write(self.unvisited_pfx)
elif i >= len(statements) or j >= len(missing):
-dest.write('> ')
+dest.write(self.visited_pfx)
elif statements[i] == missing[j]:
- dest.write('! ')
+dest.write(self.unvisited_pfx)
else:
-dest.write('> ')
+dest.write(self.visited_pfx)
elif covered:
-dest.write('> ')
+dest.write(self.visited_pfx)
else:
-dest.write('! ')
+dest.write(self.unvisited_pfx)
dest.write(line)
source.close()
dest.close()
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
"Debugging is twice as hard as writing the code in the firstplace. Therefore,
if you write the code as cleverly as possible, you are, by definition,
not smart enough to debug it." -- Brian W. Kernighan
--
http://mail.python.org/mailman/listinfo/python-list
