Re: tuple to string?
Robert Kern wrote:
>Francois De Serres wrote:
>
>
>
>>I'll pick ('%c' * len(t)) % t, for it's readability and the fact that
>>join() is on the deprec'd list.
>>
>>
>
>''.join() is certainly not deprecated. What made you think that?
>
>
>
this:
http://www.python.org/doc/2.4.1/lib/node110.html
but I now realize it's a different version of join() that was proposed
here...
thank you,
Francois
--
http://mail.python.org/mailman/listinfo/python-list
Re: tuple to string?
Francois De Serres wrote:
>hiho,
>
>what's the clean way to translate the tuple (0x73, 0x70, 0x61, 0x6D) to
>the string 'spam'?
>
>TIA,
>Francois
>
>
thanks to all!
I'll pick ('%c' * len(t)) % t, for it's readability and the fact that
join() is on the deprec'd list.
Francois
--
http://mail.python.org/mailman/listinfo/python-list
tuple to string?
hiho, what's the clean way to translate the tuple (0x73, 0x70, 0x61, 0x6D) to the string 'spam'? TIA, Francois -- http://mail.python.org/mailman/listinfo/python-list
Exception in callback => GPF?
Hiho,
When there's an unhandled exception in my
extension-module's-callback-into-Python-function-object, I get a GPF and
Python exits.
When the exception is being handled within the callback (hence in
Python), I get a very significant hiccup (1 to 5 seconds freeze).
Question: is there a specific way to deal with exceptions when they
happen inside such a callback? I searched the web with no avail. All
callbacks examples I've seen seem not to care about exceptions.
Maybe of importance: the callback is initiated within a native thread.
My C extension module is calling back into Python like this:
/* Python callback function object. */
static PyObject * my_callback = NULL;
/* C callback function passed to the implementation. */
static void external_callback(const MidiData * const data) {
if (my_callback && (my_callback != Py_None)) {
if (! data) {
PyErr_SetString(PyExc_IndexError, getLastErrorMessage());
} else {
PyObject * arglist = NULL;
PyObject * result = NULL;
PyGILState_STATE gil = PyGILState_Ensure();
arglist = Py_BuildValue("(i,i,s#)", data->deviceIndex,
data->timestamp, data->buffer, data->size);// 0, 0, "test", 4);//
result = PyEval_CallObject(my_callback, arglist);
Py_DECREF(arglist);
Py_DECREF(result);
PyGILState_Release(gil);
}
}
}
TIA,
Francois
--
http://mail.python.org/mailman/listinfo/python-list
Re: Documenting extension modules?
Simon Dahlbacka wrote: >Re: assigning a PyStr object to __doc__, take a look at Py_InitModule3, >which does that for you. > > > got it, thx. >Then you have the PyDoc_STRVAR macro in python.h that you might want to >use (see definition below). But as Robert already told you, you'll need >to provide the necessary information about i.e. parameters yourself in >the docstrings. > >/* Define macros for inline documentation. */ >#define PyDoc_VAR(name) static char name[] >#define PyDoc_STRVAR(name,str) PyDoc_VAR(name) = PyDoc_STR(str) >#ifdef WITH_DOC_STRINGS >#define PyDoc_STR(str) str >#else >#define PyDoc_STR(str) "" >#endif > > > beautiful. PS: The following snip from http://python.fyxm.net/peps/pep-0007.html was helpful to me in understanding usage of these macros: - Use the PyDoc_STR() or PyDoc_STRVAR() macro for docstrings to support building Python without docstrings (./configure --without-doc-strings). For C code that needs to support versions of Python older than 2.3, you can include this after including Python.h: #ifndef PyDoc_STR #define PyDoc_VAR(name) static char name[] #define PyDoc_STR(str) (str) #define PyDoc_STRVAR(name, str) PyDoc_VAR(name) = PyDoc_STR(str) #endif - The first line of each fuction docstring should be a "signature line" that gives a brief synopsis of the arguments and return value. For example: PyDoc_STRVAR(myfunction__doc__, "myfunction(name, value) -> bool\n\n\ Determine whether name and value make a valid pair."); Always include a blank line between the signature line and the text of the description. If the return value for the function is always None (because there is no meaningful return value), do not include the indication of the return type. - When writing multi-line docstrings, be sure to always use backslash continuations, as in the example above, or string literal concatenation: PyDoc_STRVAR(myfunction__doc__, "myfunction(name, value) -> bool\n\n" "Determine whether name and value make a valid pair."); Though some C compilers accept string literals without either: /* BAD -- don't do this! */ PyDoc_STRVAR(myfunction__doc__, "myfunction(name, value) -> bool\n\n Determine whether name and value make a valid pair."); not all do; the MSVC compiler is known to complain about this. -- http://mail.python.org/mailman/listinfo/python-list
Re: Documenting extension modules?
Robert Kern wrote:
>Francois De Serres wrote:
>
>
>>Hiho,
>>
>>I can't seem to find a proper way to document my extension module.
>>Following the C API doc:
>>
>>static PyMethodDef ioMethods[] = {
>>{"o_count", o_count, METH_VARARGS, "Return the count of available
>>MIDI outputs."},
>>
>>}
>>
>>lacks:
>>a) module level documentation
>>
>>
>
>In your init function, assign a PyStr object to __doc__.
>
>
>
>>b) function parameters
>>
>>
>
>Add that information to your docstring. I don't think that there is a
>way to get this information via inspect.
>
>
>
>>Also, I'd like to know if there's a typical format for the help string
>>(but in C), compatible with docstring's
>>"""short desription
>>
>>long description"""
>>
>>
>
>char *o_count__doc__;
>char *o_count__doc__ = "short description\n"\
>"\n"\
>"long description\n";
>
>
>
Mucho thankees Robert.
--
http://mail.python.org/mailman/listinfo/python-list
Documenting extension modules?
Hiho,
I can't seem to find a proper way to document my extension module.
Following the C API doc:
static PyMethodDef ioMethods[] = {
{"o_count", o_count, METH_VARARGS, "Return the count of available
MIDI outputs."},
}
lacks:
a) module level documentation
b) function parameters
Also, I'd like to know if there's a typical format for the help string
(but in C), compatible with docstring's
"""short desription
long description"""
Any pointers please?
TIA,
Francois
--
http://mail.python.org/mailman/listinfo/python-list
Re: Building a function call? (update)
Scott David Daniels wrote:
>Francois De Serres wrote:
>
>
>>Francois De Serres wrote:
>>
>>
>>>Having a string: "dothat"
>>>and a tuple: (x, y)
>>>1. What's the best way to build a function call like: dothat(x,y)?
>>>
>>>Assuming dothat is def'd in the same module,
>>>2. is: eval("dothat(x,y)", None, (('x', 100), ('y', 200)))
>>>the right way to have it executed?
>>>
>>>
>
>You do know that you could do something like:
> result = eval('dothat')(100, 200)
>
>That is, eval of the function name gives you a function.
>
>--Scott David Daniels
>[EMAIL PROTECTED]
>
>
No I did not know. And, that, is smart!
Many thanks,
F.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Building a function call? (update)
Duncan Booth wrote:
>Francois De Serres wrote:
>
>
>
>>Sorry, I was unclear about the fact that the args are formals. I'm
>>trying to do something like:
>>
>>func = "dothat"
>>args = ('x','y')
>>localcontext = ()
>>r = None
>>while r is None:
>>try:
>>r = eval("dothat(x,y)", None, localcontext) #how do I construct
>>"dothat(x,y)"? with 'dothat(%s,%s)' % args?
>>except NameError:
>>ensure_context(args) #find/compute the formals, eg: localcontext
>>= (('x', 100), ('y', 200))
>>
>>
>
>I think when your code ends up in as big a mess as this, you need to take a
>step back. Forget about asking us how you get at functions or arguments as
>strings, instead tell us what problem you are really trying to solve.
>
>I'm pretty sure that whatever your problem is, the solution won't involve
>using 'eval', and it almost certainly won't involve looking up variables
>or functions from their names.
>
>
Sure? okay, let me take the time to formulate a thorough blueprint of my
context, and I'll come back to you...
F.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Building a function call?
Steven D'Aprano wrote:
>On Wed, 13 Jul 2005 06:16:54 -0700, Robert Kern wrote:
>
>
>
>>Duncan Booth wrote:
>>
>>
>>>Francois De Serres wrote:
>>>
>>>
>>>
>>>>Having a string: "dothat"
>>>>and a tuple: (x, y)
>>>>1. What's the best way to build a function call like: dothat(x,y)?
>>>>
>>>>
>[snip]
>
>
>>>No, none of this is a good place to use eval.
>>>
>>>
>[snip]
>
>
>>> import otherModule
>>> vars(otherModule)[aString](*aTuple)
>>>
>>>
>>Ick! Please:
>>getattr(otherModule, aString)(*aTuple)
>>
>>
>
>
>Or, remember that functions are first class objects in Python. Instead of
>passing around the function name as a string, pass around a reference to
>the function itself. Something like this:
>
>
>def dothis(x,y):
>return x-y
>
>def dothat(x,y):
>return x+y
>
>somefunction = dothis
>somefunction(3, 2)
>=> returns 1
>
>somefunction = dothat
>somefunction(3, 2)
>=> returns 5
>
>allfunctions = [dothis, dothat]
>for func in allfunctions:
>print func(3, 2)
>=> prints 1 then 5
>
>If you want to collect user-supplied strings and use them to find a
>function, you could use eval (terribly risky and unsafe), or you could do
>something like this:
>
>funcnames = {}
>for func in allfunctions:
>funcnames[func.__name__] = func
>F = raw_input("Enter the name of a function: ")
>try:
>funcnames[F](3, 2)
>except KeyError:
>print "Function '%s' not found!" % F
>
>
>In my humble opinion, people muck about with eval, locals and globals far
>too often. It is unclear, hard to maintain, and frequently a security
>risk. These confusing, unsafe practices can usually be avoided by
>remembering that functions are first class objects just like ints, strings
>and lists.
>
>
>
>
>
I'm aware of the functions being objects, but I really need to work with
strings in that case.
Still, I was not aware that eval() was such a 'rogue', it's not said in
the manuals ;)
Many thanks!
F.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Building a function call? (update)
Francois De Serres wrote:
>Hiho,
>
>Having a string: "dothat"
>and a tuple: (x, y)
>1. What's the best way to build a function call like: dothat(x,y)?
>
>Assuming dothat is def'd in the same module,
>2. is: eval("dothat(x,y)", None, (('x', 100), ('y', 200)))
>the right way to have it executed?
>
>If dothat is def'd in another module:
>3. what would be the right way to initialize the globals to pass to eval ?
>
>
>TIA,
>Francois
>
>
>
>
Sorry, I was unclear about the fact that the args are formals. I'm
trying to do something like:
func = "dothat"
args = ('x','y')
localcontext = ()
r = None
while r is None:
try:
r = eval("dothat(x,y)", None, localcontext) #how do I construct
"dothat(x,y)"? with 'dothat(%s,%s)' % args?
except NameError:
ensure_context(args) #find/compute the formals, eg: localcontext
= (('x', 100), ('y', 200))
F.
--
http://mail.python.org/mailman/listinfo/python-list
Re: **kwargs?
Michael Hoffman wrote: >Peter Hansen wrote: > > >>Francois De Serres wrote: >> >> >> > > > >>>*args is documented in the Tutorial. I reckon **kwargs represents a >>>dictionary of arguments. But I don't quite get the semantics of **x. >>>Undefined length tuple of undefined length tuples? Are there other >>>practical use cases for ** (common enough please, I wish I was, but >>>I'm not a scientist). >>> >>> >>Where did you get "tuples of tuples" for **x ? >> >> > >I would guess it is confusion from languages where unary * means >dereference and ** means double dereference. > > That was precisely my mistake. >To the OP: I'm glad you have been reading the tutorial. If you have >further questions the reference manual is a good place to look: > >http://docs.python.org/ref/function.html > > Now I see, there's a '**' token, which is not the same as two adjacents '*' tokens. >Just as you can use a name other than self as the first argument to an >unbound method, you can call your *args and **kwargs *x and **y instead, >but they will still act just like *args and **kwargs. > >The stars are magic, not the names. > > Thanks mucho! F. -- http://mail.python.org/mailman/listinfo/python-list
Re: Building a function call?
Peter Hansen wrote:
>Roland Heiber wrote:
>
>
>>Not the best (not at all) but one way:
>>
>>def dothat(x,y):
>> print "Called with:", x, y
>>
>>c = (1,2)
>>
>>locals().get("dothat")(*c)
>>
>>
>
>As you say, not the best, but in fact not really advisable under any
>circumstances. locals() returns "module level" stuff only when executed
>_at_ module level (i.e. not inside a function). Otherwise it will
>return only the local variables of the frame its in.
>
>Better is this, since it works for the OP's requirements in either
>situation.
>
> globals()['dothat'](*c)
>
>-Peter
>
>
Actually that's part of my dilemma... I guess the function name is
moreoften in globals().
But the parameters in the tuple are
A) formals (that was not clear in my OP, sorry),
B) once bound by another function, I they'll become part of locals().
func = "dothat"
args = (x,y)
localcontext = ()
r = None
while r is None:
try:
r = eval("dothat(x,y)", None, localcontext) #how do I construct
"dothat(x,y)"?
except NameError:
ensure_context(x,y) #find/compute the formals, eg: localcontext
= (('x', 100), ('y', 200))
F.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Building a function call?
Roland Heiber wrote:
>Francois De Serres wrote:
>
>
>>Hiho,
>>
>>Having a string: "dothat"
>>and a tuple: (x, y)
>>1. What's the best way to build a function call like: dothat(x,y)?
>>
>>
>
>Not the best (not at all) but one way:
>
>
Still pretty interesting, thx.
>def dothat(x,y):
> print "Called with:", x, y
>
>c = (1,2)
>
>locals().get("dothat")(*c)
>
>
>Called with: 1 2
>
>HtH, Roland
>
>
--
http://mail.python.org/mailman/listinfo/python-list
**kwargs?
All your **kwargs are belong to us. *args is documented in the Tutorial. I reckon **kwargs represents a dictionary of arguments. But I don't quite get the semantics of **x. Undefined length tuple of undefined length tuples? Are there other practical use cases for ** (common enough please, I wish I was, but I'm not a scientist). TIA, Francois -- http://mail.python.org/mailman/listinfo/python-list
Building a function call?
Hiho,
Having a string: "dothat"
and a tuple: (x, y)
1. What's the best way to build a function call like: dothat(x,y)?
Assuming dothat is def'd in the same module,
2. is: eval("dothat(x,y)", None, (('x', 100), ('y', 200)))
the right way to have it executed?
If dothat is def'd in another module:
3. what would be the right way to initialize the globals to pass to eval ?
TIA,
Francois
--
http://mail.python.org/mailman/listinfo/python-list
The GIL, callbacks, and GPFs
Hello,
I'm chasing a GPF in the interpreter when running my extension module.
It's not very elaborated, but uses a system (threaded) callback, and
therefore the GIL.
Help would be mucho appreciated. Here's the rough picture:
win32_spam.c
/* code here is unit-tested OK */
typedef struct Bag {
char data[128];
site_t size;
} Bag;
typedef void (*PCallback)(const Bag * const bag);
Bag bag;
PCallback user_callback = NULL;
/* called by windoz */
static void CALLBACK win32_call_me(void) {
memcpy(bag.data, somestuff, 100);
bag.size = 100;
SPAWN_THREAD(user_callback, & bag);//pseudo-code
}
spam_module.c
-
/* most of the code here is pasted from doc */
static PyObject * my_callback = NULL;
static PyObject *
setCallback(PyObject *dummy, PyObject *args) {
PyObject *result = NULL;
PyObject *temp;
if (PyArg_ParseTuple(args, "O:miSetCallback", &temp)) {
if ((temp != Py_None) && (!PyCallable_Check(temp))) {
PyErr_SetString(PyExc_TypeError, "parameter must be callable");
return NULL;
}
Py_XINCREF(temp);
Py_XDECREF(my_callback);
my_callback = temp;
Py_INCREF(Py_None);
result = Py_None;
/* set the actual callback in win32_spam.c */
user_callback = & external_callback;
}
return result;
}
static void external_callback(const Bag * const bag) {
if (my_callback && (my_callback != Py_None)) {
PyObject * arglist = NULL;
PyObject * result = NULL;
arglist = Py_BuildValue("(s#)", bag->data, bag->size);
/* do it */
PyGILState_STATE gil = PyGILState_Ensure();
result = PyEval_CallObject(my_callback, arglist);
PyGILState_Release(gil);
Py_DECREF(arglist);
Py_DECREF(result);
}
}
blowup_spam1.py
-
# This throws a GPF on callback.
# Why, since the GIL is acquired by the callback?
import spam.py
def callback(stuff):
print stuff
if __name__ == '__main__':
spam.setCallback(callback)
raw_input()
blowup_spam2.py
-
# This throws a GPF as well
# Why, since we're using a threadsafe queue?
import spam
import Queue
q = Queue.Queue()
def callback(stuff):
q.put(stuff)
if __name__ == '__main__':
spam.setCallback(callback)
while True:
print q.get()
nice_spam.py
-
# This works
# Why, since the rest did not?
import spam
import Queue
import threading
q = Queue.Queue()
def callback(stuff):
q.put(stuff)
def job():
while True:
print q.get()
if __name__ == '__main__':
spam.setCallback(callback)
t = threading.Thread(job)
t.start()
raw_input()
Please point me to what I'm doing wrong...
TIA,
Francois
PS: I've already submitted my issue under "(Win32 API) callback to
Python, threading hiccups", but I guess it was poorly formulated.
--
http://mail.python.org/mailman/listinfo/python-list
Re: (Win32 API) callback to Python, threading hiccups
Christopher Subich wrote: >Francois De Serres wrote: > > >>- so, on callback, I create a new thread, after checking that the >>previous one has returned already (WaitOnSingleObject(mythread)) so we >>only have one thread involved. >> >> > >Uh... to me, this looks like a frighteningly inefficient way of doing >things. How about using a synchronous queue to post the data to a >processing thread? That way, you don't have to create an entierly new >thread each time you receive data in the callback. > > Thanks for the tip, and sorry for frightening you. Maybe you can help as well with my issue? Francois -- http://mail.python.org/mailman/listinfo/python-list
(Win32 API) callback to Python, threading hiccups
Hiho,
could somebody please enlighten me about the mechanics of C callbacks to
Python? My domain is more specifically callbacks from the win32 API, but
I'm not sure that's where the problem lies. Here's a description...
I want a callback-based MIDI input/processing, so PortMidi was not an
alternative. I have written a C extension module that links to the mmsys
MIDI API. I separated the win32-dependant code from the Python extension
code, so a) the module interface is system-neutral, b) the
implementation can be tested (re-used) outside of Python. So, that code
is tested OK, but it might be useful to sketch it's design:
- as you may know, the MIDI input on win32 is already managed thru a
callback mechanism; the driver calls back your program with data buffers
- yet, I don't call directly into Python from the callback, since some
restrictions apply on what system calls you can make from there, and I
don't know what Python's interpreter / scripts might call.
- so, on callback, I create a new thread, after checking that the
previous one has returned already (WaitOnSingleObject(mythread)) so we
only have one thread involved.
- this is this thread that calls the user callback, yet this callback
isn't yet a Python callable, we're still in native C code.
- on the side of the extension module now, I merely copied the callback
example from the Python/C API doc, and added GIL management around the call:
static PyObject * my_callback = NULL; //this gets fixed by a
setCallback() func
static void external_callback(const MidiData * const data) {
if (my_callback && (my_callback != Py_None)) {
if (! data) {
PyErr_SetString(PyExc_IndexError, getLastErrorMessage());
} else {
PyObject * arglist = NULL;
PyObject * result = NULL;
arglist = Py_BuildValue("(i,i,s#)", data->deviceIndex,
data->timestamp, data->buffer, data->size);// 0, 0, "test", 4);//
PyGILState_STATE gil = PyGILState_Ensure();
result = PyEval_CallObject(my_callback, arglist);
PyGILState_Release(gil);
Py_DECREF(arglist);
Py_DECREF(result);
}
}
}
- this one above is what is actually passed as callback to the 'native'
C part. So, here's what (I presume) happens:
1. the driver calls into my C code
2. my C code spawns a thread that calls into the extension
3. the extension calls into Python, after acquiring the GIL
Now, here's the hiccup:
inside a Python script, anytime a Python object is accessed by both the
(Python) callback and the main program, I get a GPF :/
You bet I tried to use locks, Queues and whatnot, but nothing will do,
it seems I get protection faults on accessing... thread exclusion objects.
Yet, there is a way where it works seamlessly: it's when the __main__
actually spawns a threading.Thread to access the shared object. Hence I
am lost.
This works:
def input_callback(msg):
midiio.send(msg) ##MIDI thru, a call into my extension that wraps
the implementation call with BEGIN_ALLOW_THREADS and END_...
__main__:
raw_input()
This results in GPF as soon as the callback is fired:
tape = Queue.Queue()
def input_callback(msg):
tape.put(msg)
__main__:
while True:
print tape.get()
This works like a charm:
tape = Queue.Queue()
def input_callback(msg):
tape.put(msg)
def job():
while True:
print tape.get()
__main__:
t = threading.Thread(target = job)
t.start()
raw_input()
Note that I also get a GPF in the later case if I stop the program with
KeyInterrupt, but I guess it's something I'll look into afterwards...
While I can think of hiding away the added threading within a wrapper
module, I won't sleep well untill I figure out what's really happening.
Could someone save my precious and already sparse sleeping time, by
pointing me to what I'm missing here?
WinXP SP1
Python 2.4.1
MinGW GCC 3.4.2
TIA,
Francois De Serres
--
http://mail.python.org/mailman/listinfo/python-list
