Re: distutils compiler flags for extension modules
Thanks! I'll try that. Sorry for replying so late - just didn't get to it. Christian -- http://mail.python.org/mailman/listinfo/python-list
distutils compiler flags for extension modules
Hoi, as I got no answers with the previous question (subject: disabling compiler flags in distutils), I thought I should ask the question in a different way: Is there an option to set the compiler flags for a C/C++ extension in distutils? There is the extra_compile_args-option in the Extension class, yet this offers only to give additional flags, but I'd like to have 'total' control about the compile args. Any hint? Thanks Christian -- http://mail.python.org/mailman/listinfo/python-list
disabling compiler flags in distutils
Hoi, I've written an extension in C++ which compiles wonderfully. However, I want to compile it using a setup script with distutils and distutils gives the compiler the unwanted (!) flag "-Wstrict-prototypes", which is a flag only understood when compiling C. The C++ compiler runs, but issues a warning I don't want to see. Is there a way to turn off that flag in a setup script (using distutils)? TIA Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: How to convert image into numpy.ndarray
On Wed, 25 Feb 2009 07:52:03 -0800, anti-suho wrote: > In scipy module, there is a function named misc.lena which can return an > array of numpy.ndarray type. If you use this array as parameter of > matplotlib.pyplot.imshow and then call the matplotlib.pyplot.imshow > function, an image will be shown. The shown image is generated by the > numpy.ndarray array. > > How to convert an arbitrary image into an array of numpy.ndarray type? Well, arbitrary ... But this may serve as a starting point: from scipy.misc import fromimage import Image #PIL my_array = fromimage(Image.open(_file_name_)) Of course, you should perform the appropriate error checking, too. ;-) HTH Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: glibc detected *** python: corrupted double-linked list
Hi, >> I have a problem using my software on my 64bit laptop, after an update >> of my system. The same code still runs on 32bit Intel, but on my laptop >> I provoke the crash in the title. The crash is caused - as narrowed >> down by me - by returning a static PyObject from a C-extension >> function. > > I think you got all the reference counts wrong, specially dummy1, dummy2 > and r. Might be a good point, but can you give me a hint where to look specifically? > > (BTW, when you know the size, it's better to use PyList_New(size) + > PyList_SET_ITEM instead of PyList_New(0) + PyList_Append) Just rewrote that section. Thank you. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: glibc detected *** python: corrupted double-linked list
Thanks David! It's still not debugged, but indeed: I get a bunch of warnings. And this already showed me that there are more potential problems than my first guess indicated. Alas, for my specific problem I cannot work with ints chars and doubles. I need to have unsigned longs at some points. Well, I'll sort it out. Cheers Christian -- http://mail.python.org/mailman/listinfo/python-list
glibc detected *** python: corrupted double-linked list
Hoi, I have a problem using my software on my 64bit laptop, after an update of my system. The same code still runs on 32bit Intel, but on my laptop I provoke the crash in the title. The crash is caused - as narrowed down by me - by returning a static PyObject from a C-extension function. Well, now I wondering what to do? A web search delivered no relevant bug reports about glibc on my system (the recent Ubuntu 8.10). Is there a source for validated glibc for x86_64? Oh, and I'm using Ubuntu's ready- made Python package, version 2.5.2.. Anyone experiences with such problems? Any ideas? Information missing? TIA Christian PS The extension module itself can be found here: http:// svn.origo.ethz.ch/viewvc/sas-rigid/src/calc.c?revision=209&view=markup But, as I said, the code is perhaps far from being perfect, but it runs on 32bit computers. The crash occurs at line 164 - "return py_pofr;". -- http://mail.python.org/mailman/listinfo/python-list
Re: 32 bit or 64 bit?
Peter Otten wrote: > > How did you determine that standard python floats are not good enough? > Everything beyond that is unlikely to be supported by the hardware and > will therefore introduce a speed penalty. > > Did you try gmpy? I would like to add: If Python's precision (or that of additional modules) won't do it for you, look around what other people in the physics community are using. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: problem with Py_BuildValue
Thank you so much - I was such an idiot (see below). >>> I see nothing wrong with your code so I'd say it is somewhere else (did >>> you snip any code between the end of the loop and the return?). > >>No. (Apart from freeing allocated memory.) > > I'm pretty sure we'll find something interesting here :) Still not. I was about to prove it and already uploaded the file, when I saw, what was really going wrong ... > > PyList_GetItem returns a borrowed reference so you shoud _not_ unref it > (this explains the refcnt -1 I think) This is THE crucial point. If I just delete the Py_CLEAR-line, everything is working smoothly and calling PyFloat_FromDouble-is working too. Again: Thanks a lot. 'Reference counting' won't become one of my hobbies ;-). Best, Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: 32 bit or 64 bit?
> I do need speed. Is there an option? Mind telling us what you *actually* want to achieve? (What do you want to calculate?) Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: problem with Py_BuildValue
Thank you. At least I can exclude another few error sources, now.
Cédric Lucantis wrote:
> I see nothing wrong with your code so I'd say it is somewhere else (did
> you snip any code between the end of the loop and the return?).
No. (Apart from freeing allocated memory.)
> I've never
> seen those 'refcnt' objects but a refcount of 0 sounds like you unrefed
> your objects one extra time by mistake. This would produce a segfault on
> unix, but maybe not on all platforms ?
Well, I am working on Linux. Python 2.5.1, gcc 4.1.3 . And I do not see
segfaults until I start working in Python with the return value of that
function, of course.
> You should check the return value
> of PyList_Append()
It is always 0, regardless of what I do.
> and if it doesn't help trace the content of your list
> after each iteration to see when the bad things happen (you can check the
> reference count of an object with obj->ob_refcnt).
Seems ok. What I did to check this was placing this after building the list:
for (i=0; i < limit; i++) {
dummy = PyList_GetItem(python_return_value, i);
printf("%f\n", PyFloat_AsDouble(dummy));
Py_CLEAR(dummy);
}
Which gives reasonable numbers.
>
> Finally note that in your case it would be much simpler and more efficient
> to use the float constructor directly:
>
> dummy = PyFloat_FromDouble(internal_list([i]))
I tried that (actually PyFloat_FromDouble(internal_list[i]) ): Same thing,
but now more like [, , etc.
(Note the -1.)
>
> PS: always use Py_CLEAR(dummy) instead of Py_DECREF(dummy); dummy=NULL;
> (though it doesn't really matter in this simple case - see
> http://docs.python.org/api/countingRefs.html)
Good idea! Since I require 2.4 for users anyway, there is no harm in
reducing the code.
Christian
--
http://mail.python.org/mailman/listinfo/python-list
problem with Py_BuildValue
Hi,
currently I have a problem understanding Py_BuildValue. I have this code:
static PyObject *function(PyObject *self, PyObject *args) {
PyObject * python_return_value = NULL;
PyObject * dummy = NULL;
double * internal_list;
/* converting to python representation */
for (i=0; i < limit; i++) {
dummy = Py_BuildValue("d", internal_list[i]);
if (!dummy) return NULL;
PyList_Append(python_return_value, dummy);
Py_DECREF(dummy); dummy = NULL;
}
return python_return_value
}
This doesn't work. What I see, when invoking the function "function()" in
Python is a list of refcounts, like: [, , ...]. However, if I change the Py_BuildValue-line to be
dummy = Py_BuildValue("i", (int)internal_list[i]);
I do get the 'right' integer return values. Point is that I really would
like to work with Python-floats afterwards.
Any idea where a pitfall might be here?
TIA
Christian
PS Oh, and I tried casting to float and explicitly to double, too. Same
result as without the casts.
--
http://mail.python.org/mailman/listinfo/python-list
Re: seg. fault with Py_BuildValue?
Ok now, I know where the error is: "y" actually contained refcounts. This, of course, is complete nonsense and causes the interpreter to crash at some point. Thanks to all of you: You helped at least to track down the problem. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: seg. fault with Py_BuildValue?
> Check if either x or y are NULL. They aren't. Just did an explicit check. -- http://mail.python.org/mailman/listinfo/python-list
Re: seg. fault with Py_BuildValue?
> What happens if you get rid of the ()? Can't see any difference. -- http://mail.python.org/mailman/listinfo/python-list
seg. fault with Py_BuildValue?
Hi
I'm having trouble with Py_BuildValue. I was able to pinpoint the following
statement as the one causing a seg. fault with my script:
static PyObject * funcname(PyObject *self, PyObject *args) {
...
return Py_BuildValue("(OO)", x, y);
}
where x & y are both of type PyObject.
Any suggestions here? Do I need to handle the output of Py_BuildValue
somehow before returning? How? Need to decref x & y before returning?
TIA
Christian
--
http://mail.python.org/mailman/listinfo/python-list
Re: extending Python - passing nested lists
> You didn't mention speed in your original post.
Sorry, perhaps I considered this self-evident - which it is, of course, not.
> What about using
> array.array? Unless I am mistaken, these are just a thin wrapper
> around normal C arrays.
The algorithm I want to implement requires several million floating point
operations. Neither the array-modules nor numpy's thin layer seem thin
enough for me. ;-)
> Anyway you could always convert your list
> into a c array, do lots and lots of fast calculations, then convert it
> back again to a list.
I guess I am too blind to see, but I couldn't discover a method description
like "double* PyList_toDouble". So, yes, my question is a C-API-newbie
question: What is the way to say "myCarray = SomePyMethod(InputPyList)"? Or
better, what should be here instead
static PyObject *_foo(PyObject *self, PyObject *args) {
double *v;
if (!PyArg_Parse(args, "(d)", &v))
return NULL;
to get a list as an array of doubles into 'v' (and back to Python)?
I did read the API-description, but still am lost at this point. I presume,
once I get to know the answer I'll bang my head on the table ... ;-)
Cheers
Christian
--
http://mail.python.org/mailman/listinfo/python-list
Re: extending Python - passing nested lists
Thanks. Point is that all such approaches would require lots(!) of calls to the Python API - a way by which I won't gain the desired speed. I've tried pyrex and the corresponding C-file is so convoluted with dummy variables, incrementing & decrementing references, and other stuff, that I want to try to write a C-function myself. My goal is not to avoid PyObjects* and the corresponding reference handling - apart from the head and end of the function. (I could use ctypes instead, but that again would obfuscate the API of my package a bit.) Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: extending Python - passing nested lists
Think, that I'm still at the wrong track. Point is that I cannot find any
examples and don't know where to start here.
Perhaps my problem boils down to two questions:
I'd like to pass lists (in some cases nested ones) from Python to C and
convert those Python-lists to C-arrays (e. g. of doubles). My second wish
is to return a C-array of longs to a Python list.
My approach so far:
static PyObject *_foo(PyObject *self, PyObject *args) {
double *v;
if (!PyArg_Parse(args, "(d)", &v))
return NULL;
// then I can't access v like v[1] ...
// then return *v
return with something like PyBuildValue (but didn't get so far)
}
Can somebody give me a hint here, please? Passing simple arguments to and
fro (e. g. single integer values) is no problem, but lists of unknown size?
TIA
Christian
--
http://mail.python.org/mailman/listinfo/python-list
Re: extending Python - passing nested lists
Mark Dickinson wrote: > Well, it's pretty clear: you misspelt "length" as "lenght". :) Well, that's not it ;-). (Damn copy & paste plague ...) > > PySequence_Fast doesn't return an array: it returns a PyObject---in > this case, a PyObject corresponding to a Python tuple. That's it. Thanks. Think I just need a different approach. I got completely on the wrong track here. Thanks Christian -- http://mail.python.org/mailman/listinfo/python-list
extending Python - passing nested lists
Hi,
I would like to write a C-extension function for an application of mine. For
this I need to pass a nested list (like: [[a, b, c], [d, e, f], ...], where
all letters are floats) to the C-function. Now, with the code I have the
compiler is complaining: "subscripted value is neither array nor pointer".
Can somebody tell me what's wrong?
Here a code snippet to reproduce this problem:
static PyObject *_foo(PyObject *self, PyObject *args) {
int i;
long lenght;
float ax, ay, az;
PyObject *dummy_list;
if (!PyArg_ParseTuple(args, "O", &dummy_list))
return NULL;
dummy_list = PySequence_Fast(dummy_list, "argument must be iterable");
lenght = PyObject_Length(dummy_list);
for (i=0; i < lenght; i++) {
// part which does not work:
ax = dummy_list[i][0];
ay = dummy_list[i][1];
az = dummy_list[i][2];
}
return 0;
}
TIA
Christian
--
http://mail.python.org/mailman/listinfo/python-list
Re: permuting over nested dicts?
Thanks everyone, I knew there must be a snippet somewhere, just couldn't find one! (Just for the sake of completeness: Order doesn't matter and I hope that with my data I won't reach the recursion depth limit.) Christian -- http://mail.python.org/mailman/listinfo/python-list
permuting over nested dicts?
Hoi,
I have the following data structure (of variable size actually, to make
things simple, just that one):
d = {'a': {'x':[1,2,3], 'y':[4,5,6]},
'b': {'x':[7,8,9], 'y':[10,11,12]}}
This can be read as a dict of possibilities: The entities 'a' and 'b' have
the parameters 'x' and 'y', each. And d['a']['x'] can be either 1 or 2 or
3. Does anybody know a convenient (and fast) way to permute over all
possible nested dicts like
{'a': {'x':1, 'y':4},
'b': {'x':7, 'y':10}}
and
{'a': {'x':2, 'y':4},
'b': {'x':7, 'y':10}}
and so forth?
Any link or snippet is appreciated.
TIA
Christian
--
http://mail.python.org/mailman/listinfo/python-list
Re: strange behavious of the logging module?
Thanks Peter and Vinay, I finally understood. And indeed, removing the pyc-file in questions solves the problem - at least temporarily. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: strange behavious of the logging module?
Peter Otten wrote: > Vinay Sajip wrote: > >> This is a known bug, and not specifically related to logging, though >> it sometimes manifests itself via logging: >> >> http://bugs.python.org/issue1180193 > > Yup, thanks for looking into it. > > Peter Thanks Vinay, but I am curious how issue 1180193 is related to the "bug" Peter and me described: I do not have any *.pyc file around. As I kept changing the file during development I used to call the program directly using a link in my PATH. Hence no *.pyc file got created at all. However, does anybody know a suggestion for a work-around? Christian -- http://mail.python.org/mailman/listinfo/python-list
problem writing setup script
Hi
I have the following file structure
MANIFEST
README
INSTALL -- all text
setup.py
src/
__init__.py
foo.py
for_ext.pyx
and this setup-script:
import sys
from distutils.core import setup
from distutils.extension import Extension
try:
from Pyrex.Distutils import build_ext
except ImportError:
__info__ = """
Please install Pyrex
(http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/)
before running this setup script.
"""
sys.exit(__info__)
setup(
name = "foo",
# snip
packages = ['foo'],
package_dir={'foo': 'src/'},
package_data = {'foo': ['README', 'INSTALL']},
ext_package='foo_ext',
ext_modules = [
Extension("foo_ext", ["src/foo_ext.pyx"])
],
cmdclass = {'build_ext' : build_ext}
)
Typing
sudo python setup.py
runs without warning, however, this results in a file
/usr/lib/python2.5/site-packages/foo/foo.py , which I only can import like
import foo.foo
and not
import foo
in order to access foo's namespace.
Furhtermore a file foo_ext.so will be created
in /usr/lib/python2.5/site-packages, but I would like to see it
in /usr/lib/python2.5/site-packages/foo, since importing foo_ext.so only
makes sense for foo.
Does anybody know how I should change my file structure or the setup-script
to achieve those two goals?
TIA
Christian
--
http://mail.python.org/mailman/listinfo/python-list
Re: strange behavious of the logging module?
Thanks Peter and Gabriel, I see, so I should provide the application with an additional handler for the file in order to gain maximum independence of the two handlers (console & file stream). >> Also, adding %(funcName)-8s to the formatter in basigConfig does not work >> for me: instead of the functions name the level in lower case gets >> inserted >> into the logfile. > > Looks like a bug... > Well, the documentation -- as far as I understand it ;-) -- says the behaviour should be different, you two basically agree ... think I should file a report. Thanks, Christian -- http://mail.python.org/mailman/listinfo/python-list
strange behavious of the logging module?
Hi,
having the following code:
import logging
logging.basicConfig(level=logging.ERROR,
format='%(levelname)-8s %(message)s',
filename='mc_rigid.log',
filemode='w')
# define a Handler which writes INFO messages or higher to the sys.stderr
console = logging.StreamHandler()
console.setLevel(logging.INFO)
# set a format which is simpler for console use
formatter = logging.Formatter('%(message)s')
# tell the handler to use this format
console.setFormatter(formatter)
# add the handler to the root logger
logging.getLogger('').addHandler(console)
I observe nothing printed on the console. However if I change the level
(just after the import) from logging.ERROR to logging.DEBUG I do see all
info lines on the console as intended.
Can anybody tell my mistake? I thought changing the level in basicConfig
would have no impact on the following StreamHandler instance.
Also, adding %(funcName)-8s to the formatter in basigConfig does not work
for me: instead of the functions name the level in lower case gets inserted
into the logfile.
My python:
Python 2.5.1 (r251:54863, May 2 2007, 16:56:35)
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
TIA
Christian
--
http://mail.python.org/mailman/listinfo/python-list
Re: permutations - fast & with low memory consumption?
Thanks Simon & Gerard! I will check those exampels out. Christian PS Of course, I did google - but apparently not creative enough. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is htmlGen still alive?
If starship does not get up anymore (give it a few hours) you might want to have a look here: http://www.python.org/ftp/python/contrib-09-Dec-1999/Network/ Else, I can send you a tarfile of version 2.1, too. As hg said, apart from a few deprecation warnings it's working fine for me (but I'm only using it for one internal site, where layout doesn't matter ...). Cheers Christian [EMAIL PROTECTED] wrote: > Does anybody know whether htmlGen, the Python-class library for > generating HTML, is still being maintained? Or from where it can be > downloaded? The Starship site where it used to be hosted is dead. > > Thanks for your help! > > Klaus Muller -- http://mail.python.org/mailman/listinfo/python-list
permutations - fast & with low memory consumption?
Hi,
I'd like to hack a function which returns all possible permutations as lists
(or tuples) of two from a given list. So far, I came up with this solution,
but it turned out to be too slow for the given problem, because the list
passed ("atomlist") can be some 1e5 items long:
def permute(atomlist, size = 2):
"""
returns a list of atoms grouped by two
"""
if not size or not atomlist:
return [atomlist[:0]]
else:
result = list()
for i in xrange(len(atomlist)):
pick = atomlist[i:i+1] # sequence slice
remainder = atomlist[:i] + atomlist[i+1:] # keep [:i] part
for x in __permute(remainder, size = size - 1):
result.append(pick + x)
return result
Does anybody know a solution which consumes less memory and is possibly
faster, perhaps using generator expressions? All my attempts so far failed.
Any help appreciated!
TIA
Christian
--
http://mail.python.org/mailman/listinfo/python-list
Re: excel in unix?
[EMAIL PROTECTED] wrote: > hi > is it possible to create excel files using python in Unix env? > if so, what module should i use? > thanks You might want to give pyExelerator a try: http://sourceforge.net/projects/pyexcelerator Remark: I had the problem of having a bunch of Data on my Linux machine and some students to teach, who didn't know anything of Linux. (And OpenOffice was not capable of doing what I wanted it to do, btw.) So I choosed pyExelerator in combination with PreExisting EXCEL-Sheets. Cheers, Christian -- http://mail.python.org/mailman/listinfo/python-list
