unittest.assertRaise and keyword arguments?
Dear list, The syntax for using assertRaise is assertRaise(exception, function, para1, para2,...) However, I have a long list of arguments (>20) so I would like to test some of them using keyword arguments (use default for others). Is there a way to do this except for manually try...except? Thanks. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: unittest.assertRaise and keyword arguments?
Giovanni Bajo wrote: > You can pass keyword arguments to assertRaises without problems: > > self.assertRaises(ValueError, myfunc, arg1,arg2, arg3, arg4, abc=0, foo=1, > bar="hello") Well, I though abc=0 would be keyword arguments for assertRaisers and never tried it! > > Or you can always do something like: > > self.assertRaises(ValueError, lambda: myfunc(arg1,arg2, arg3, arg4, abc=0, > foo=1, bar="hello")) This is also useful. I now have nothing to complain against assertTaises. :-) Bo -- http://mail.python.org/mailman/listinfo/python-list
exposing C array to python namespace: NumPy and array module.
Dear list, I am writing a Python extension module that needs a way to expose pieces of a big C array to python. Currently, I am using NumPy like the following: PyObject* res = PyArray_FromDimsAndData(1, int*dim, PyArray_DOUBLE, char*buf); Users will get a Numeric Array object and can change its values (and actually change the underlying C array). This works fine. However, when I deliver my module, I find NumPy is unnecessarily large for this simple task. As a matter of fact, I had to build from source NumPy, ATLAS etc on Solaris, Linux, Mac and if a user would like to use my module, he has to do the same thing! Python's array module is built-in, easy to use, but *without* a FromLenAndData function! Even the buffer interface provides only 'get buffer' but no 'set buffer' functions. Could anyone tell me how I can create an array object from existing data? Some vague ideas might be used: 1. PyCObject (I do not really understand the manual), 2. copy and modify arraymodule.c to my project (doable at all? License issue?) 3. Create an array object and hack it. (no api to do this.) I would strongly suggest an arraymodule.h with Array_FromLenAndData. Many thanks in advance. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: exposing C array to python namespace: NumPy and array module.
Craig Ringer wrote: On Sat, 2005-01-01 at 08:18, Bo Peng wrote: Python's array module is built-in, easy to use, but *without* a FromLenAndData function! Even the buffer interface provides only 'get buffer' but no 'set buffer' functions. Could anyone tell me how I can create an array object from existing data? Python has no array objects in the core language, only lists. The distinction is important when discussing numarray etc, because Python lists and NumPy etc arrays are very different. Thank you very much for the detailed reply! Sorry if I was not clear enough. I was talking about the differece between python array module (http://docs.python.org/lib/module-array.html, Modules/arraymodule.c in the source tree) and NumPy array. They both use C-style memory block arrangement for efficient memory access. While NumPy has both, the array module is designed to be used purely in Python so there is no header file and no function to build an array from a pointer. One of the methods you suggested (creating a new type) already implemented in arraymodule.c. I am not sure if it is appropriate to add the file into my project and add a 'CreateFromLenAndBuf' function. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: exposing C array to python namespace: NumPy and array module.
Scott David Daniels wrote: Python's array module is not built to do this well. It can re-size the array, delete elements inside the array, and other things that don't work very well with C-managed data. I wrote "blocks and views" to overcome this problem. As always the case, this problem have been encountered and solved! Your code is exactly what I needed. However, I was too impatient to wait for your reply. :-) I have realized the problems of arraymodule.c and added a ob_ownmem member in the object structure. Existing operations have been modified (mostly unchanged) according to this flag and the new module works well. Thank everyone for the help. especially in thie new year's day. Bo -- http://mail.python.org/mailman/listinfo/python-list
compiler version mismatch when building ext_modules with MSVC
Dear list, I was using autoconf/automake etc to build my python extension. Since distutil seems to be easier, I have switched to this method. However, when I build the module under windows, I am told that compiler version mismatch... I am using msvc 7 and python was build by msvc 6.0. I am not sure why compiler version matters here since 1. if proceed by removing corresponding lines in distutil, the generated module works fine (as far as I can tell). 2. I am told that I can use minGW gcc ... Isn't mingw differ more with msvc6 than msvc7? 3. I was using msvc7 (automake etc, not distutil) and everything was fine. I was not even warned about potential problems. Since there are many different Python out there (Official, ActivePython, Enthough), do I have to figure out which compiler they use to build my module for them? Currently, my MSVC7 built module works fine with all of them. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
mixing SWIG generated and Python-level usertype?
Dear list,
My SWIG generated module (myModule) needs an array-like object (carray)
to work. Carray objects are created both internally (in C++ level) and
through Python so I have to load it when myModule initializes.
carray is modified from arraymodule.c and is quite simple:
static PyMethodDef a_methods[] =
{
{"carray", a_array, METH_VARARGS, a_array_doc},
{ /* sentinel */
NULL,NULL
}
};
Currently, I load a_methods directly using code (error checking ignored)
PyObject* mm = PyImport_AddModule("__main__");
PyObject* dict = PyModule_GetDict(mm);
PyObject* v = PyCFunction_New(a_methods, NULL);
PyDict_SetItemString(dict, a_methods->ml_name, v);
There are several problems with this approach:
1. use of __main__? carray can not be accessed directly within other
libraries. ('from myModule import *' DOES NOT import carray!) I tried to
use __builtins__ but it does not work for some reason out of my
understanding of Python. I am not sure how to add carray to myModule
dictionary.
2. No type object? I am not sure what is the purpose of ArrayType but
the usual init_module should be
m = Py_InitModule3("carray", a_methods, module_doc);
d = PyModule_GetDict(m);
PyDict_SetItemString(dict, "ArrayType", (PyObject *)&Arraytype);
When I add ArrayType to __main__ , access to ArrayType leads to a quick
core dump.
I do not feel comfortable with my current approach. Could anyone tell me
some better (more standard) way?
Many thanks in advance.
Bo
--
http://mail.python.org/mailman/listinfo/python-list
changing local namespace of a function
Dear list,
I have many dictionaries with the same set of keys and I would like to
write a function to calculate something based on these values. For
example, I have
a = {'x':1, 'y':2}
b = {'x':3, 'y':3}
def fun(dict):
dict['z'] = dict['x'] + dict['y']
fun(a) and fun(b) will set z in each dictionary as the sum of x and y.
My function and dictionaries are a lot more complicated than these so I
would like to set dict as the default namespace of fun. Is this
possible? The ideal code would be:
def fun(dict):
# set dict as local namespace
# locals() = dict?
z = x + y
Many thanks in advance.
Bo
--
http://mail.python.org/mailman/listinfo/python-list
Re: changing local namespace of a function
M.E.Farmer wrote:
def fun(d):
... __dict__ = d
... return __dict__
hth,
Does not work?
>>> a = { 'x':1, 'y':2}
>>> b = { 'x':2, 'y':9}
>>> def fun(d):
... __dict__ = d
... print locals()
... z = x + y
>>> fun(a)
{'__dict__': {'y': 2, 'x': 1}, 'd': {'y': 2, 'x': 1}}
Traceback (most recent call last):
File "", line 1, in ?
File "/usr/tmp/python-6377tax.py", line 4, in fun
NameError: global name 'x' is not defined
--
http://mail.python.org/mailman/listinfo/python-list
Re: changing local namespace of a function
Michael Spencer wrote:
As you no doubt have discovered from the docs and this group, that isn't
doable with CPython.
Too bad to know this.
>>> a = {'x':1, 'y':2}
>>> b = {'x':3, 'y':3}
...
>>> def funa(x,y, **kw):
... del kw #Careful of unwanted names in locals with this approach
... z = x + y
... return locals()
There are hundreds of items in the dictionary (that will be needed in
the calculation) so passing the whole dictionary is a lot better than
passing individual items.
Alternatively, you could use exec:
Yes. I thought of using exec or eval. If there are a dozen statements,
def fun(d):
exec 'z = x + y' in globals(), d
seems to be more readable than
def fun(d):
d['z'] = d['x'] + d['y']
But how severe will the performance penalty be?
Bo
--
http://mail.python.org/mailman/listinfo/python-list
Re: changing local namespace of a function
Jeff Shannon wrote: This sounds to me like you're trying to re-implement object orientation. I have no control over the big dictionaries. All I need to do is processing them in situ --- that is to say, go into each map and manipulate numbers. Parameter passing should be avoid whenever possible since involved number of items are huge. It is basically like exec 'statements' in d but a function with local dictionary d would be best. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: changing local namespace of a function
Thank all for your suggestions. I have tried all methods and compared
their performance.
>>> import profile
>>> a = {'x':1, 'y':2}
>>> N = 10
>>> # solution one: use dictionary directly
... def fun1(d):
... for i in xrange(0,N):
... d['z'] = d['x'] + d['y']
...
>>> # solution two: use exec
... def fun2(d):
... for i in xrange(0,N):
... exec 'z = x + y' in globals(), d
...
>>> # solution three: update local dictionary
... # Note that locals() is *not* d after update() so
... # z = x + y
... # does not set z in d
... def fun3(d):
... exec "locals().update(d)"
... for i in xrange(0,N):
... d['z'] = x + y
...
>>> # solution four: use dict wrapper
... # this makes code easier to write and read
... class wrapdict(object):
... """Lazy attribute access to dictionary keys. Will not access
... keys that are not valid attribute names!"""
... def __init__(self, mydict):
... object.__setattr__(self, "mydict",mydict)
... def __getattr__(self, attrname):
... return self.mydict[attrname]
... def __setattr__(self, attrname, value):
... self.mydict[attrname] = value
...
>>> # use wrapper
... def fun4(d):
... wd = wrapdict(d)
... for i in xrange(0,N):
... wd.z = wd.x + wd.y
...
>>> profile.run('fun1(a)')
3 function calls in 0.070 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
10.0000.0000.0600.060 :1(?)
10.0100.0100.0700.070 profile:0(fun1(a))
00.000 0.000 profile:0(profiler)
10.0600.0600.0600.060 python-4645vcY.py:2(fun1)
>>> profile.run('fun2(a)')
13 function calls (3 primitive calls) in 5.890 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
11/10.4400.0005.8905.890 :1(?)
10.0000.0005.8905.890 profile:0(fun2(a))
00.000 0.000 profile:0(profiler)
15.4505.4505.8905.890 python-46458me.py:2(fun2)
>>> profile.run('fun3(a)')
4 function calls (3 primitive calls) in 0.060 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
2/10.0000.0000.0600.060 :1(?)
10.0000.0000.0600.060 profile:0(fun3(a))
00.000 0.000 profile:0(profiler)
10.0600.0600.0600.060 python-4645Jxk.py:5(fun3)
>>> profile.run('fun4(a)')
34 function calls in 3.910 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
10.0000.0003.9103.910 :1(?)
10.0000.0003.9103.910 profile:0(fun4(a))
00.000 0.000 profile:0(profiler)
100.5300.0000.5300.000
python-4645W7q.py:10(__setattr__)
10.0000.0000.0000.000 python-4645W7q.py:6(__init__)
200.9600.0000.9600.000
python-4645W7q.py:8(__getattr__)
12.4202.4203.9103.910 python-4645jFx.py:1(fun4)
Exec is slow since compiling the string and calls to globals() use a lot
of time. The last one is most elegant but __getattr__ and __setattr__
are costly. The 'evil hack' solution is good since accessing x and y
takes no additional time.
I guess I will go with solution 3. It is evil but it is most close to my
original intention. It leads to most readable code (except for the first
line to do the magic and the last line to return result) and fastest
performance.
Thank again for everyone's help. I have learned a lot from the posts,
especially the wrapdict class.
Bo
--
http://mail.python.org/mailman/listinfo/python-list
Re: changing local namespace of a function
M.E.Farmer wrote: I really don't see your need. Maybe it is just my laziness. It is almost intolerable for me to write lines and lines of code like d['z'] = func(d['x']+d['y']+d['whatever']['as']+d[a][0] ) It is ugly, unreadable and error prone. If I have to use this code, I would write _z = func(_x + _y + _whatever['as'] + _a[0]) and use a perl script to generate the real code. (See, I am not lazy :-) Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: changing local namespace of a function
Maybe it is just my laziness. It is almost intolerable for me to write lines and lines of code like d['z'] = func(d['x']+d['y']+d['whatever']['as']+d[a][0] ) By the way, will 'with statement', like the one in pascal and many other languages, be a good addition to python? For example, with d do: z = x + y would be equivalent to d['z']=d['x']+d['y'] or d.z = d.x + d.y in some other cases. This would absolutely be the *best* solution to my problem. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: changing local namespace of a function
Nick Coghlan wrote: If you want to add more calculated properties to the data manipulator, simply define additional calculator methods, and define the attribute with make_prop. This has became really appealing You know, I have a deep root in C/C++ so performance is the king and hacking is part of my daily life. Time to change now. :) My original problem was that I need to manipulate and create dictionary items in situ and since there are so many "d['key']"s, I would like to make my code easier to read (and save some key strokes) by using "d.key" or simply "key", without sacrificing performance. OOP seemed to be irrelevant. But this on-demand calculation (quote: make first call to z automatic) is *really* good! I can calculate self._data.stat_a = self._data.stat_b + 1 without worrying about the status of stat_b and users can access any value in the dictionary, original or calculated, in a uniform and easier way! This makes my program instantly easier to use. I can not say enough thank you for this. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: changing local namespace of a function
Kent Johnson wrote:
You can part way there using keyword arguments. You just have to use
dictionary syntax for changing values in the dictionary:
>>> def f(d, x=None, y=None):
... d['z'] = x + y
...
>>> a = {'x':1, 'y':2}
>>> b = {'x':3, 'y':3}
>>>
>>> f(a, **a)
>>> a
{'y': 2, 'x': 1, 'z': 3}
>>> f(b, **b)
>>> b
{'y': 3, 'x': 3, 'z': 6}
This is not possible in my case since my dictionary have many more items
than just x and y. So, if there is are other items in the dictionary
>>> a = {'x':1, 'y':2, 'else':4}
>>> f(a,**a)
Traceback (most recent call last):
File "", line 1, in ?
File "/usr/tmp/python-10176RtT.py", line 1, in ?
f(a,**a)
TypeError: f() got an unexpected keyword argument 'else'
Bo
--
http://mail.python.org/mailman/listinfo/python-list
Re: changing local namespace of a function
Exec is slow since compiling the string and calls to globals() use a lot
of time. The last one is most elegant but __getattr__ and __setattr__
are costly. The 'evil hack' solution is good since accessing x and y
takes no additional time.
Previous comparison was not completely fair since I could pre-compile
fun2 and I used indirect __setattr__. Here is the new one:
>>> import profile
>>> a = {'x':1, 'y':2}
>>> N = 10
>>> # solution one: use dictionary directly
... def fun1(d):
... for i in xrange(0,N):
... d['z'] = d['x'] + d['y']
...
>>> # solution two: use exec
... def makeFunction(funcStr, name):
... code = compile(funcStr, name, 'exec')
... def f(d):
... exec code in d
... return f
...
>>> def fun2(d):
... myfun = makeFunction('z = x + y', 'myfun')
... for i in xrange(0,N):
... myfun(d)
... del d['__builtins__']
...
... # solution three: update local dictionary
>>> # Note that locals() is NOT d after update() so
... # z = x + y
... # does not set z in d
... def fun3(d):
... exec "locals().update(d)"
... for i in xrange(0,N):
... d['z'] = x + y
...
>>> # solution four: use dict wrapper
... # this makes code easier to write and read
... class wrapdict(object):
... """Lazy attribute access to dictionary keys. Will not access
... keys that are not valid attribute names!"""
... def __init__(self, mydict):
... self.__dict__ = mydict
...
... # use wrapper
>>> def fun4(d):
... wd = wrapdict(d)
... for i in xrange(0,N):
... wd.z = wd.x + wd.y
...
>>> profile.run('fun1(a)')
3 function calls in 0.060 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
10.0000.0000.0600.060 :1(?)
10.0000.0000.0600.060 profile:0(fun1(a))
00.000 0.000 profile:0(profiler)
10.0600.0600.0600.060 python-10176FWs.py:2(fun1)
>>> profile.run('fun2(a)')
24 function calls in 2.130 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
10.0000.0002.1302.130 :1(?)
100.5200.0000.5200.000 myfun:1(?)
10.0000.0002.1302.130 profile:0(fun2(a))
00.000 0.000 profile:0(profiler)
10.5900.5902.1302.130 python-10176EqB.py:1(fun2)
10.0000.0000.0000.000
python-10176Sgy.py:2(makeFunction)
101.0200.0001.5400.000 python-10176Sgy.py:4(f)
>>> profile.run('fun3(a)')
4 function calls (3 primitive calls) in 0.070 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
2/10.0000.0000.0700.070 :1(?)
10.0000.0000.0700.070 profile:0(fun3(a))
00.000 0.000 profile:0(profiler)
10.0700.0700.0700.070 python-10176R0H.py:4(fun3)
>>> profile.run('fun4(a)')
4 function calls in 0.100 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
10.0000.0000.1000.100 :1(?)
10.0000.0000.1000.100 profile:0(fun4(a))
00.000 0.000 profile:0(profiler)
10.0000.0000.0000.000
python-10176e-N.py:6(__init__)
10.1000.1000.1000.100 python-10176rIU.py:1(fun4)
Since
d['x'] is fast but cumbersome
exec "z=x+y' is still slow.
exec "locals().update(d)" is evil
d.x is elegant and only a little slower than d['x']
I am announcing the winner of the contest: dictwrap! (applause)
Bo
--
http://mail.python.org/mailman/listinfo/python-list
Re: changing local namespace of a function
Kent Johnson wrote:
Bo Peng wrote:
Exec is slow since compiling the string and calls to globals() use a
lot of time. The last one is most elegant but __getattr__ and
__setattr__ are costly. The 'evil hack' solution is good since
accessing x and y takes no additional time.
Previous comparison was not completely fair since I could pre-compile
fun2 and I used indirect __setattr__. Here is the new one:
>>> # solution two: use exec
... def makeFunction(funcStr, name):
... code = compile(funcStr, name, 'exec')
... def f(d):
... exec code in d
... return f
...
>>> def fun2(d):
... myfun = makeFunction('z = x + y', 'myfun')
... for i in xrange(0,N):
... myfun(d)
... del d['__builtins__']
You are still including the compile overhead in fun2. If you want to see
how fast the compiled code is you should take the definition of myfun
out of fun2:
myfun = makeFunction('z = x + y', 'myfun')
def fun2(d):
for i in xrange(0,N):
myfun(d)
del d['__builtins__']
Kent
I assumed that most of the time will be spent on N times execution of
myfunc.
>>> myfun = makeFunction('z = x + y', 'myfun')
>>> def fun2(d):
... # myfun = makeFunction('z = x + y', 'myfun')
... for i in xrange(0,N):
... myfun(d)
... del d['__builtins__']
>>> def fun21(d):
... myfun = makeFunction('z = x + y', 'myfun')
... for i in xrange(0,N):
... myfun(d)
... del d['__builtins__']
>>> profile.run('fun2(a)')
23 function calls in 1.930 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
10.0000.0001.9301.930 :1(?)
100.4600.0000.4600.000 myfun:1(?)
10.0000.0001.9301.930 profile:0(fun2(a))
00.000 0.000 profile:0(profiler)
100.9800.0001.4400.000 python-162616Io.py:4(f)
10.4900.4901.9301.930 python-16261Ud0.py:1(fun2)
>>> profile.run('fun21(a)')
24 function calls in 1.920 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
10.0000.0001.9201.920 :1(?)
100.3900.0000.3900.000 myfun:1(?)
10.0000.0001.9201.920 profile:0(fun21(a))
00.000 0.000 profile:0(profiler)
10.0000.0000.0000.000
python-162616Io.py:2(makeFunction)
100.9300.0001.3200.000 python-162616Io.py:4(f)
10.6000.6001.9201.920 python-16261huu.py:1(fun21)
--
http://mail.python.org/mailman/listinfo/python-list
Is there something similar to ?: operator (C/C++) in Python?
Hi, I need to pass a bunch of parameters conditionally. In C/C++, I can do func(cond1?a:b,cond2?c:d,.) In Python, I am using temporary variables like if cond1: para1 = a else: para1 = b # # a bunch of such if/else func(para1, para2,...) Is there an easier way to do this in Python? BTW, I do not want to use the following since all of a,b,c,d etc will be evaluated. def condValue(cond, val1,val2): if cond: return val1 else: return val2 func(condValue(cond1,a,b), condValue(cond2,c,d),...) Thanks. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there something similar to ?: operator (C/C++) in Python?
>>In Python, I am using temporary variables like >> >>if cond1: >> para1 = a >>else: >> para1 = b >> >># >># a bunch of such if/else >> >>func(para1, para2,...) > > > Yeah, that's how I would do it. How many of these things do you have? I have around 10 of them. Using these if/else, it will take 50 lines for a function call. It also bothers me to have 10 variables left in the namespace, in addition to finding 10 meaningful names. The FAQ provides two solutions, neither of them are elegant. I guess I will use if/else for reasability purpose. Thanks. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there something similar to ?: operator (C/C++) in Python?
Roy Smith wrote: > Can you give us some idea of what it is that you're trying to do? It pretty > unusual to see > a requirement like that. def func(type_of_obj1, type_of_obj2, .): callfunc( [ type_of_obj1 and obj1a() or obj1b(), type_of_obj2 and obj2a() or obj2b(), ]) callfunc can take arbitrary number of objects whose types are determined by type_of_obj1 etc. I was using a bunch of if/else to create objects and pass them to callfunc. Since type_of_obj1 etc are usually binary and obj1a() etc will never be false, the and/or solution does not look so bad in this case. Thanks. Bo -- http://mail.python.org/mailman/listinfo/python-list
How to obtain a 'interactive session' of a script?
Dear list, I have a long list of commands in the form of a script and would like to obtain a log file as if I enter the commands one by one. (The output will be used in a tutorial.) What would be the best way to do it? Copy and paste is not acceptable since I make frequent changes tot he script. Many thanks in advance. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: How to obtain a 'interactive session' of a script?
Thank you for the suggestions and code!
> import code
>
> SCRIPT = [line.rstrip() for line in open("myscript.py")]
>
> script = ""
> prompt = ">>>"
>
> for line in SCRIPT:
> print prompt, line
> script = script + line + "\n"
> co = code.compile_command(script, "", "exec")
> if co:
> # got a complete statement. execute it!
> exec co
> script = ""
> prompt = ">>>"
> else:
> prompt = "..."
>
This one fails at function definition.
def fun():
a=1
b=2 <--- not included.
Still trying other methods.
Bo
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to obtain a 'interactive session' of a script?
[EMAIL PROTECTED] wrote: > The 'code' module contains 'Utilities needed to emulate Python's interactive > interpreter.'. By subclassing code.InteractiveConsole and replacing the > raw_input method with one which reads from a file, I think you can get what > you > want. This method works fine with only one minor problem. It would stop (waiting for user input) at help(str) command. I will have to find a way to feed the program with'q' etc. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: How to obtain a 'interactive session' of a script?
Fredrik Lundh wrote: > replacing sys.stdin with something that isn't a TTY will fix this. This works like magic! Thank you! Bo -- http://mail.python.org/mailman/listinfo/python-list
Big latex subscript using python.sty and manual.cls
Dear list,
I am writing a manual for my python extension using Python (2.4) latex
classes. Everything seems to be fine except that the subscripts are as
big as normal text ( see page 55 of
http://bp6.stat.rice.edu:8080/simuPOP_doc/userGuide.pdf ). I doubt that
the following piece of code in python.sty is causing the problem but can
not fix it. (I get errors when commenting out this block of code.) Does
anyone know what is happening here?
Note that the subscripts are OK if I use other classes like report.cls.
Many thanks in advance.
Bo
% in python.sty
% This gets the underscores closer to the right width; the only change
% from standard LaTeX is the width specified.
\DeclareTextCommandDefault{\textunderscore}{%
\leavevmode [EMAIL PROTECTED]
% Underscore hack (only act like subscript operator if in math mode)
%
% The following is due to Mark Wooding (the old version didn't work with
% Latex 2e.
\DeclareRobustCommand\hackscore{%
\ifmmode_\else\textunderscore\fi%
}
\begingroup
\catcode`\_\active
\def\next{%
\AtBeginDocument{\catcode`\_\active\def_{\hackscore{}}}%
}
\expandafter\endgroup\next
%%%
--
http://mail.python.org/mailman/listinfo/python-list
default value for list access?
Dear list,
My program needs to do calculation based on a giving data structure
(like a sparse matrix) with lots of missing values (invalid
indices/keys of lists/dictionaries).
The programing is difficult in that I have to use a try...except block
for every item access. I have written a function
def getItem(item, idx, default):
try:
return item[idx]
except:
return default
Then I will have to write a[1]['a'][4] as
getItem( getItem( getItem(a,1,{}), 'a', []), 4, 0)
Is there any way to automatically return 0 when any exception is raised
for such data acceses? (automatic wrapping a[...] as
try:
a[...]
except:
0
)
Many thanks in advance.
Bo
--
http://mail.python.org/mailman/listinfo/python-list
Re: default value for list access?
To clearify the problem: The data is the count of something, for example a[90]=10. a may be a dictionary if the range is huge and a list when the range is reasonably small. In the dictionary case, I can use a.setdefault(80, 0) if key 80 does not exist. In the list case, I have to check the length of list before I access it (or use exception). This becomes troublesome when I need to do the following a lot. b = a[80][1] + a[80][2] + a[80][3] If I use the getItem function in my previous email, it will look like b = getItem(getItem(a, 80, []), 1, 0) + getItem(getItem(a, 80, []), 2, 0) + getItem(getItem(a, 80, []), 3, 0) What would be the best solution to handle this? Something like b = df( a[80][1], 0) + df( a[80][2], 0) + df( a[80][3], 0) would be reasonable but df can not be easily defined since the exception will be raised before function df is called. Something like b = df( a, [80, 1], 0) + df( a, [80,2], 0) + df( a, [80, 3], 0) would work if df is defined as def df(a, idx, default): try: for i in range(0, idx): a = a[ idx[i] ] return a except: return 0 but I am afraid that a for loop would bring serious performance problem. Thanks. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: default value for list access?
Sounds to me like the best solution is to simplify the implementation
and dispense with the list alternative. Why use a list if a dictionary
is suitable? Don't say performance: that's premature optimization.
Dictionaries already have what you need, apparently, with setdefault(),
so just use them and keep the code simple.
You are right that if I use all dictionaries, I can use
a.setdefault(4,{}).setdefault(2,0)
to access a[4][2]. Not too bad compared to the getItem function.
However, this is an scientific application. Data are stored internally
in C format and exposed to Python as python array (arraymodule). I guess
the best way is to modify the arraymodule interface and stop it from
raising an exception when index is out of range. (similar to Ruslan's
solution). But this means too much trouble for now.
I will use the df(a, idx, default) method and see if its performance is
acceptable.
Thanks.
Bo
--
http://mail.python.org/mailman/listinfo/python-list
parameter name conflict. How to solve?
Dear list, If you ask: why do you choose these names? The answer is: they need to be conformable with other functions, parameter names. I have a function that pretty much like: def output(output=''): print output and now in another function, I need to call output function, with again keyword parameter output def func(output=''): output(output=output) Naturally, I get 'str' object is not callable. Is there a way to tell func that the first output is actually a function? (like in C++, ::output(output) ) Thanks. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: parameter name conflict. How to solve?
Delaney, Timothy C (Timothy) wrote: Is this a style guide thing? Why not just: def func(output_param=''): output(output=output_param) This is exactly the problem. There are a bunch of other functions that use output='' parameter. Changing parameter name for this single function may cause confusion. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: parameter name conflict. How to solve?
Kent Johnson wrote: Bo Peng wrote: def func(output=''): output(output=output) Naturally, I get 'str' object is not callable. Is there a way to tell func that the first output is actually a function? (like in C++, ::output(output) ) You could use a default argument: def func(output='', output_fn=output): output_fn(output=output) Kent Thank everyone for the quick responses. All methods work in general. Since func() has to take the same parameter set as some other functins, I can not use func(output='', output_fn=output). "output_alias=output" etc are fine. Bo -- http://mail.python.org/mailman/listinfo/python-list
How to pass parameter when importing a module?
Dear list,
What I would like to do is something like:
In myModule.py ( a wrapper module for different versions of the module),
if lib == 'standard':
from myModule_std import *
elsif lib == 'optimized'
from myModule_op import *
but I do not know how to pass variable lib to myModule.py to achieve the
following effect:
>>> lib = 'standard'
>>> from myModule import * # actually import myModule_std
From what I have read, from does not take any parameter. Maybe I
should use environmental variables?
>>> os.putenv('lib', 'standard')
>>> from myModule import *
myModule.py
-
import os
lib = os.getenv('lib')
...
or use a separate module?
param.py
-
para = {}
def setParam(key, val):
para[key] = val
main session
>>> import param
>>> param.setParam('lib','standard')
>>> from myModule import *
in myModule.py
--
from param import para
try:
lib = para['lib']
except:
lib = 'standard'
...
Is there an established approach for this problem?
Many thanks in davance.
Bo
===
FULL STORY:
I have several modules all (SWIG) wrapped from the same C++ source code
but with different compiling flags. For example,
myModule_std (standard module)
myModule_op (optimized, without error checking)
...
These modules are put directly under /.../site-packages . To load a
module, I use, for example
from myModule_op import *
This works fine until I need to write some helper functions for
myModule_?? in another module myHelper.py since I do not know which
myModule is being used
from myModule?? import A,B
I find one solution
# find out which module is being used
import sys
if 'myModule_op' in sys.modules.keys():
from myModule_op import A,B
else:
from myModule_std import A,B
but not completely satisfied. Therefore, I am writing a 'wrapper' module
myModule that can load one of the myModule_?? modules according to user
supplied info.
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to pass parameter when importing a module?
Take a look at wxPython versioning:
http://wiki.wxpython.org/index.cgi/MultiVersionInstalls
The most simple usage looks like
import wxversion
wxversion.select("2.4")
import wx
Serge.
This is essentially my second method: using another module to set
parameter for myModule. Since wxPython uses this method, I suppose this
is the standard approach for this problem.
Thanks.
Bo
--
http://mail.python.org/mailman/listinfo/python-list
Testing the availability of a module
Dear list, Is there a better way than doing try: import aModule except: has_aModule = False else: has_aModule = True The main concern here is that loading aModule is unnecessary (and may take time). Many thanks in advance. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: Testing the availability of a module
Peter Hansen wrote: > Bo Peng wrote: > >> Is there a better way than doing >> >> try: >>import aModule >> except: >>has_aModule = False >> else: >>has_aModule = True >> >> The main concern here is that loading aModule is unnecessary (and may >> take time). > And if you aren't doing the above multiple times, then you *really* > shouldn't be worried about the time to load "aModule" This is used in a scenario like: if wxPython is available: prepare wx based GUI elif Tkinter is available: prepare tk based GUI else: command line only Since a user may prefer command line so wxPython/tkinter should not be actually loaded before the GUI function is called. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: Testing the availability of a module
> if the user happens to prefer the command line interface, why bother looking
> for a GUI ? sounds like you haven't really thought this through...
>
> here's an outline that matches your use case:
>
> if user wants command line:
> command line only
> else:
> if wxPython is available:
> prepare wx based GUI
> elif Tkinter is available:
> prepare tk based GUI
> else:
> command line only
This is exactly the logic I use,... Eliminating all unrelated stuff, I
have something like (thanks for the imp hint):
def wxGUI():
import wx
wx...
def tkGUI():
import Tkinter as tk
tk
def cml():
command line
def GUI():
if options['cml']:
cml()
else:
if imp.find_module('wx'):
wxGUI()
elif imp.find_module('Tkinter'):
tkGUI()
else:
cml()
After a user loads this module (without wx loaded automatically as
before), he can call cml() or GUI().
Bo
--
http://mail.python.org/mailman/listinfo/python-list
Weird memory consumption problem.
Dear list,
I spent the last 12 hours in catching this bug (?) and what I found out
is very difficult to explain:
Basically, I need to call a user-provided function many times, with a
tuple as parameter.
C/C++ side: (class A, constructed using a python function m_func)
// create in the constructor, used to hold the parameter
m_numArray = PyTuple_New(m_len);
...
// member function, will be called many times
void fun(){
// set the tuple
for( j=0; jhttp://mail.python.org/mailman/listinfo/python-list
Re: Weird memory consumption problem.
Steven D'Aprano wrote: > Sorry, are you saying that the code you posted does NOT have a memory > leak, but you want us to find the memory leak in your real code sight > unseen? valgrind does not detect anything so it does not look like memory leak. I just can not figure out why val[0], readonly access to a tuple element, can cause this kind of problem. Anyway, is there a way to list all or newly added python objects? I guess I can trace the ref_cnt of the tuple element manually. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: Weird memory consumption problem.
Bo Peng wrote: >> Sorry, are you saying that the code you posted does NOT have a memory >> leak, but you want us to find the memory leak in your real code sight >> unseen? Problem found. It is hidden in a utility function that converts the return value to a double. The refcnt of the middle result was not decreased. PyObject* res = PyNumber_Float(obj); val = PyFloat_AsDouble(res); // missing Py_DECREF(res); The problem is not that difficult to find, but it was 2am in the morning and I was misled by the different behavior of pyFun1 and pyFun2. Thanks, Bo -- http://mail.python.org/mailman/listinfo/python-list
Why does Rpy/R-plot work under PythonWin, but not under commandline/IDLE?
Dear list, I am using rpy, a python module to control statistical package R from python. Running the following commands >>> from rpy import * >>> r.plot(0) will pass command 'plot' to R and run it. I notice that the R-plot will not refresh (a window is created but the figure is not drawn) until the next plot command is executed, * when I use python (2.3, 2.4) command line or IDLE * NOT when I use pythonwin ( provided with pywin32 package) Since rpy+R works under python+pythonwin, I assume that this is not a problem with rpy or R. Does anyone know why commandline/IDLE cause this problem, while PythonWin does not? I am interested to know what I can do in commandline (or in rpy) to fix this problem. Regards, Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: Why does Rpy/R-plot work under PythonWin, but not under commandline/IDLE?
Bo Peng wrote: > Does anyone know why commandline/IDLE cause this > problem, while PythonWin does not? I am interested to know what I can do > in commandline (or in rpy) to fix this problem. Just note that R/plot works fine both from command window and a R-GUI. Bo -- http://mail.python.org/mailman/listinfo/python-list
How to distribute an additional DLL to site-packages?
Dear Python experts,
I am writing a python extension module that needs to link with a
third-party DLL. How can I copy this DLL to the site-packages directory
along with my extension modules? It seems that data_files parameter can
do this, but I do not know how to get the absolute destination
directory. After all, this directory is configurable.
I guess I need to determine /path/to/site-package for
setup(
...
ext_modules = [...]
data_files = [ ('/path/to/site-package', ['mydll.dll]) ]
)
using something like distutil.config.get_dest_dir().
Many thanks in advance.
Bo
--
http://mail.python.org/mailman/listinfo/python-list
Building python C++ extension modules using MS VC++ 2005?
Dear list, I have been using mingw to build a python extension module. I had to jump through a number of hooks like the _ctype problem caused by the use of msvcr71.dll, but the module was mostly usable. Things become complicated when I use more and more boost libraries and mingw can not work well with some of the modules. I am trying to switch to VC but I was suggested that I need to use the identical version of VC that is used to build python, which means VC7.1, 2003. However, microsoft seems to stop distributing this version of VC, and its website points me to VC expression 2005... (http://msdn2.microsoft.com/en-us/visualc/aa336490.aspx) Can I use the 2005 version to build python extension? What is the official or recommended way to build python extension module under windows? Many thank in advance. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: Building python C++ extension modules using MS VC++ 2005?
Sandra-24 wrote: > You can use 2005 to build extensions for Python 2.5. I've done this > with several extensions, both my own and others. I do not know if you > can use it for Python 2.4, so I won't advise you on that. I thought > Microsoft made its C/C++ compiler, version 7.1 (2003) freely available > as a command line tool. If you can't find it on their site, ask around, > I'm sure a lot of people have it. Probably you'll also find someone has > put it up somewhere if you search on google. Try 2005 first and see > what happens though. But how exactly should I do it? I installed VC8 and get the all-familiar error message: Python was built with Visual Stuidio 2003 Actually, I get a copy of VC6/2003 but this compiler can not compile boost 1.33.1/serialization so it can not be used to compile my extension module. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: How to distribute an additional DLL to site-packages?
> I am writing a python extension module that needs to link with a > third-party DLL. How can I copy this DLL to the site-packages directory > along with my extension modules? It seems that data_files parameter can > do this, but I do not know how to get the absolute destination > directory. After all, this directory is configurable. So nobody has ever done this??? Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: How to distribute an additional DLL to site-packages?
> Use the package_data option. setup(..., packages=['yyy'],
> package_data={'yyy':['xxx.dll']}, ...)
> (Distutils documentation may be arcane sometimes, but this is easily
> found at http://docs.python.org/dist/node12.html)
> Absolute dirs are almost never necesary, usually all distutils commands
> operate on relative paths (relative to location of setup.py for source
> files, relative to some meaningful directory for targets).
I read that page but I am not using package so things are installed to
site-packages directly. What should I put for 'yyy'?
Bo
--
http://mail.python.org/mailman/listinfo/python-list
Is there a way to profile underlying C++ code?
Dear list, I have a C++-SWIG-wrapped python module. The running time for one of the functions is pretty unpredictable so I would like to profile it. However, the python profiler does not seem to enter the compiled module (.so file). Is there a way to profile the C++ functions? Many thanks in advance. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a way to profile underlying C++ code?
Travis E. Oliphant wrote: > On Linux you can use oprofile (which is pretty nice and easy to use --- > no recompiling. Just start the profiler, run your code, and stop the > profiler). Thank you very much for the tip. This is a great tool. The source of the problem has been found: cache misses. Briefly, I have a long array and I need to choose numbers from it randomly. When the array is short, the performance is good. When the array is big, picking a number will cause a cache miss almost all the time. This results in almost doubled execution time. Problem found, but I am out of idea how to fix it since the nature of my algorithm is working randomly on large arrays. Cheers, Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: Start a python interactive shell from python.
I think I find what I need: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/355319 Bo -- http://mail.python.org/mailman/listinfo/python-list
Start a python interactive shell from python.
Dear list, This may sound strange but I need to start a python shell from python. The motivation is that I have a bunch of (numeric) python functions to provide to a user. The best way I can think of is packing my python module using py2exe, and because it is easiest to let him run the functions through python syntax, I would like to start a python shell after the module is loaded. Something like: def myfunc(a): pass if __name__ == '__namin__': ' myfunc has been defined ' START A PYTHON SHELL? so the user can type myfunc(0.5) and others. I can of course use raw_input and eval but I will lose the convenience of a real shell. Many thanks in advance. Bo -- http://mail.python.org/mailman/listinfo/python-list
Module written in C does not repond to Ctrl-C interruption.
Dear list, I have not done a thorough test, but it occurs to me that 1. python code can be interrupted by Ctrl-C. 2. A C module, if I add a main() function and run independently, can be interrupted by Ctrl-C. 3. If I load the C module in python and run, the program will not respond to Ctrl-C interruption. I have to kill python explicitly. If this is a known behavior or just a special case of mine? Any fix to it? I am using python 2.3.4 under Redhat EL4 with gcc 3.3.4. Many thanks in advance. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: Module written in C does not repond to Ctrl-C interruption.
Daniel Dittmar wrote: > > You could set up your own signal handler when entering the C extension. > This should abort the extension (tricky) and call the Python signal > handler. This can be done under linux using things in signal.h but I am not sure whether or not there is a portable way to do it (max, win32). Does anybody know a quick way under windows? Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: Suggestions for documentation generation?
kpd wrote: > Hello, > > I have written a C++ library that I've then wrapped with Pyrex. > Any suggestions to the best-in-class tool to create documentation for > the libraries? > > I would love to document things in one spot (could be the code) and > generate html and PDF from there. > > Doxygen (www.doxygen.org) looks to be about the best so far. > > I was facing the same problem and took the following approach: 1. write c++ code with doxygen-complaint comments 2. generate XML comments using doxygen 3. parse XML and generate Python docstring 4. insert the docstring into SWIG interface file I can send you the parser but I do not know if pyrex support python docstring. Bo -- http://mail.python.org/mailman/listinfo/python-list
control precision for str(obj) output?
Dear list, I have enjoyed the convenience to output any object with str(obj) for a while. However, I get long output for things like str([0.0002]) in my output (which bothers my users more than me though). I also do not understand why the following is happening: >>> str([0.0002]) '[0.00020001]' >>> str(0.0002) '0.0002' Is there any way to fix this, other than checking the type of everything and output with %.4f? Many thanks in advance. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: control precision for str(obj) output?
> The unfortunate side effect of this is to make str(list of floats) > ugly, but... Yeah, Nothing is perfect. I can control the damage as long as I know what is going on. I guess this would fit in a faq list quite well. I would have to thank Mike and Dan for your quick and detailed reply. As a matter of fact, this python list is the most friendly mailinglist I have ever joined. Thanks! Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: control precision for str(obj) output?
Dan Bishop wrote: > Andrew Dalke wrote: > >>Mike Meyer wrote: >> >>>Someone want to tell me the procedure for submitting FAQ entries, > > so I > >>>can do that for this? >> >>You mean more than what already exists at >> > > http://www.python.org/doc/faq/general.html#why-are-floating-point-calculations-so-inaccurate > >>which has a link to an even more detailed chapter in the tutorial at >> http://docs.python.org/tut/node16.html > > > Yes. Those pages explain why repr(0.2) != '0.2', but they don't > explain why str([x]) != '[%s]' % x . > I am new to Python but has experience with other languages. I am quite aware of 'why-are-floating-point-calculations-so-inaccurate'. I would not be surprised if >>> str[0.0002) 0.0002001 >>> str([0.0002]) [0.0002001] It was the difference between str([x]) and '[%s]' % x str(x) that confused me. Bo -- http://mail.python.org/mailman/listinfo/python-list
Python distutil: build libraries before modules
Dear list, My python modules depend on a few boost libraries. For convenience, I include the source code of these libraries with my distribution and treat them as standard source files. This results in huge cc.exe command lines and I get an error saying error: command 'cc' failed: Invalid argument under win32/mingw. (I am not quite sure this is the reason though.) How can I let setup.py build these boost libraries separately so that I can link them to my modules? This will also save some link time because all my python modules link to the same boost libraries. Many thanks in advance. Bo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python distutil: build libraries before modules
Bo Peng wrote: > How can I let setup.py build these boost libraries separately so that I > can link them to my modules? This will also save some link time because > all my python modules link to the same boost libraries. Although this is nowhere documented, I find that I can use from distutils.ccompiler import * def buildStaticLibrary(sourceFiles, libName, libDir): '''Build libraries to be linked to simuPOP modules''' # get a c compiler comp = new_compiler() objFiles = comp.compile(sourceFiles) comp.create_static_lib(objFiles, libName, libDir) to build the libraries Cheers, Bo -- http://mail.python.org/mailman/listinfo/python-list
