Re: Reading c struct via python
Looks like your e... Is that the key field? Yes, you are right 'e' is the key. And rest of them are data. AND the order of the items is o before i -- that doesn't seem to match your C struct definition. Sorry, I was testing the bdb and while doing that I noticed, c-struct order is not same as inserted bdb record . But forgot to fix the struct order before posting it here. (And also I didn't expect someone will dig this deeper on the binary string output :p . Thanks for your effort! ) s format in which you precode the length of the string in the format (10s is a 10 character string), and p format in which the first byte of the string is the length (0..255) of the rest of the string. The struct module doesn't handle C-type null terminated strings directly. I think we can use 'p' format. (Thus storing the string-length before actual string content). That should help us unpack easily. Thanks Dennis Lee Bieber, for the detailed info and step by step parsing of the output. It really helped. -- Wulfraed Dennis Lee Bieber AF6VN wlfr...@ix.netcom.comHTTP://wlfraed.home.netcom.com/ -- https://mail.python.org/mailman/listinfo/python-list -- Cheers, Lakshmipathi.G FOSS Programmer. www.giis.co.in -- https://mail.python.org/mailman/listinfo/python-list
Reading c struct via python
Hi -
We have C code which writes following struct into berkeley db (my_db.db).
struct my_info {
unsigned long int i, e;
int o;
char *f;
char *s;
};
How to read this via Python? Google search gave this code
---
$ cat pybsd2.py
from bsddb import db
fruitDB = db.DB()
fruitDB.open('my_db.db', None, db.DB_BTREE, db.DB_DIRTY_READ)
cursor = fruitDB.cursor()
rec = cursor.first()
while rec:
print rec
rec = cursor.next()
fruitDB.close()
---
While storing an entry (o=500,f=/home/laks/abcde,s=OSr,i=4668368
,e=1) and reading it back i get
$ python pybsd2.py
(\x10'\x00\x00\x00\x00\x00\x00,
\xf4\x01\x00\x00\xd0;G\x00\x00\x00\x00\x00\x10'\x00\x00\x00\x00\x00\x00/home/laks/abcde\x00OSr\x00)
And this http://docs.python.org/2/library/bsddb.html says bdb module is
removed recently.
Further Searching provides modules like cpickle,ctypes,struct - not sure
which is the right approach/way to proceed.
Should we proceed with bdb module and find out how to retrive integer from
db or use others like ctypes/(c)pickle/struct etc?
Thanks for any help.
Ps : If this question already answered and discussed often, please redirect
me to that thread.
--
Cheers,
Lakshmipathi.G
FOSS Programmer.
www.giis.co.in
--
https://mail.python.org/mailman/listinfo/python-list
C struct to Python
I am not sure how to proceed. I am writing a Python interface to a C library. The C library uses structures. I was looking at the struct module but struct.unpack only seems to deal with data that was packed using struct.pack or some other buffer. All I have is the struct itself, a pointer in C. Is there a way to unpack directly from a memory address? Right now on the C side of things I can create a buffer of the struct data like so... MyStruct ms; unsigned char buffer[sizeof(MyStruct) + 1]; memcpy(buffer, ms, sizeof(MyStruct)); return Py_BuildValue(s#, buffer, sizeof(MyStruct)); Then on the Python side I can unpack it using struct.unpack. I'm just wondering if I need to jump through these hoops of packing it on the C side or if I can do it directly from Python. Thanks, ~Eric -- http://mail.python.org/mailman/listinfo/python-list
Re: C struct to Python
On Tue, Nov 30, 2010 at 10:57 AM, Eric Frederich eric.freder...@gmail.com wrote: I am not sure how to proceed. I am writing a Python interface to a C library. The C library uses structures. I was looking at the struct module but struct.unpack only seems to deal with data that was packed using struct.pack or some other buffer. All I have is the struct itself, a pointer in C. Is there a way to unpack directly from a memory address? Right now on the C side of things I can create a buffer of the struct data like so... MyStruct ms; unsigned char buffer[sizeof(MyStruct) + 1]; memcpy(buffer, ms, sizeof(MyStruct)); return Py_BuildValue(s#, buffer, sizeof(MyStruct)); Then on the Python side I can unpack it using struct.unpack. I'm just wondering if I need to jump through these hoops of packing it on the C side or if I can do it directly from Python. Thanks, ~Eric ctypes[0] sounds like a possible solution, although if you're already writing a C extension it might be better practice to just write a Python object that wraps your C struct appropriately. If you're not wedded to the C extension, though, I've had very good luck writing C interfaces with with ctypes and a few useful decorators [1], [2]. Others prefer Cython[3], which I like for speed but which sometimes seems to get in my way when I'm trying to interface with existing code. There's a good, if somewhat dated, overview of a few other strategies here[4]. Geremy Condra [0]: http://docs.python.org/library/ctypes.html [1]: http://code.activestate.com/recipes/576734-c-struct-decorator/ [2]: http://code.activestate.com/recipes/576731/ [3]: http://www.cython.org/ [4]: http://www.suttoncourtenay.org.uk/duncan/accu/integratingpython.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Using C struct in Python
Dennis Lee Bieber wrote: On Wed, 07 Jun 2006 17:35:58 -0400, Sudheer Gupta [EMAIL PROTECTED] declaimed the following in comp.lang.python: print cp print cp.next There is a typo in this. Second statement was suppose to be cp = cp.next. I corrected it latter with the second email. What happens if you keep adding print cp.next statements (say 50 or more of them). Your error message might indicate that you ran out of cp.next /values/ (ie: the last valid item in the chain of cp's does not have a next link at all, rather than a next link that contains null) -- http://mail.python.org/mailman/listinfo/python-list
Using C struct in Python** corrected
Hi,
I am having trouble using C struct in python. Hope anyone can help me
out ...
Say, I have my C struct as
typedef struct call
{
struct call *next;
// .
} call_t;
I have a global variable, namely call_pool, which is of type call_t *
My python program:
cp = call_pool # no error doing this, means that call_pool is accessable
while cp:
print cp
cp = cp.next
This is giving me error: There is no member or method name c_next
Now, If I just do:
print cp
print cp.next
there is no problem. But I am seeing a difference in the way python is
looking at the struct:
print cp - (call_t*) 0xb0...
print cp.next - (struct call *) 0xb0...
Is python not intelligent enough to diagnose the next pointer ??
Responses appreciated.
Thanks
Sudheer
--
http://mail.python.org/mailman/listinfo/python-list
Using C struct in Python
Hi,
I am having trouble using C struct in python. Hope anyone can help me
out ...
Say, I have my C struct as
typedef struct call
{
struct call *next;
// .
} call_t;
I have a global variable, namely call_pool, which is of type call_t *
My python program:
cp = call_pool # no error doing this, means that call_pool is accessable
while cp:
print cp
print cp.next
This is giving me error: There is no member or method name c_next
Now, If I just do:
print cp
print cp.next
there is no problem. But I am seeing a difference in the way python is
looking at the struct:
print cp - (call_t*) 0xb0...
print cp.next - (struct call *) 0xb0...
Is python not intelligent enough to diagnose the next pointer ??
Responses appreciated.
Thanks
Sudheer
--
http://mail.python.org/mailman/listinfo/python-list
Re: Using C struct in Python** corrected
Sudheer Gupta wrote:
I am having trouble using C struct in python. Hope anyone can help me
out ...
Say, I have my C struct as
typedef struct call
{
struct call *next;
// .
} call_t;
I have a global variable, namely call_pool, which is of type call_t *
My python program:
cp = call_pool # no error doing this, means that call_pool is accessable
Without any glue code? That's weird. What Python implementation is this?
while cp:
print cp
cp = cp.next
This is giving me error: There is no member or method name c_next
Doesn't look like a Python exception to me. Cannot find any trace of
that message in the CPython sources either...
Now, If I just do:
print cp
print cp.next
there is no problem. But I am seeing a difference in the way python is
looking at the struct:
print cp - (call_t*) 0xb0...
print cp.next - (struct call *) 0xb0...
Is python not intelligent enough to diagnose the next pointer ??
I've written more C extensions for Python than most people, but I'm
definitely not intelligent enough to decipher your post. What on earth
are you talking about ? Are you sure you're posting to the right
newsgroup ?
/F
--
http://mail.python.org/mailman/listinfo/python-list
Re: Using C struct in Python
On 8/06/2006 7:35 AM, Sudheer Gupta wrote:
Hi,
Hi.
Your later correction doesn't clear up the confusion below. Quick
eye-balling revealed no difference. If you have to correct a minor typo
in a posting, please consider saying change X to Y instead of
reposting the whole thing.
I am having trouble using C struct in python. Hope anyone can help me
out ...
Say, I have my C struct as
typedef struct call
{
struct call *next;
// .
} call_t;
I have a global variable, namely call_pool, which is of type call_t *
You really need to explain what sort of glue you have between your
Python code and your C code.
1. Are you the author of the glue? If not, better ask the author.
2. Are you extending Python with a module written in C, or are you
embedding Python in a C program?
3. If extending, what's your glue? SWIG? something else? hand-crafted?
My python program:
cp = call_pool # no error doing this, means that call_pool is accessable
Acessible from where? How do you bind the name call_pool to an object?
while cp:
print cp
print cp.next
This is giving me error: There is no member or method name c_next
Not a Python error message. Must be coming from inside your extension
module.
Now, If I just do:
print cp
print cp.next
there is no problem.
Sorry, I don't understand. Above you said it was giving me error.
But I am seeing a difference in the way python is
looking at the struct:
print cp - (call_t*) 0xb0...
print cp.next - (struct call *) 0xb0...
If they are actual results from a Python print statement, then I can
only assume the extension module defines types which have (in effect)
str() and/or repr() methods which produce such output.
Python does not know that the C type of cp is (call_t*) and that of
cp.next is (struct call *). In fact it doesn't care, and it shouldn't
care. The extension module could be written in assembly language or APL
or even INTERCAL if it obeys the conventions, which don't include
exposing a C type for each object. Another way of looking at it: methods
in extension modules are mostly expected to behave like methods written
in Python.
I get the impression that you are using some superglue that parses C
declarations and writes (parts of) Python extension modules in C. Do you
think you could possibly tell us what the name of this superglue is?
Is python not intelligent enough to diagnose the next pointer ??
To the extent to which I can understand what your question means, the
answer is: It by design makes no attempt to be what you are calling
intelligent.
Python does only limited inspection of the tables of methods that an
extension type says it supports. It uses only the very basic
information: is the pointer to method X NULL?
I hope some of the above helps you amplify your question.
Cheers,
John
--
http://mail.python.org/mailman/listinfo/python-list
Re: Using C struct in Python
Hi,
Thanks a lot for your responses. It cleared up a lot for me !!
Its a superglue developed and used in house and cannot be revealed ..
sorry for that !!
I am only extending to the existing glue. Have contacted the author
regarding the problem.
Sorry for confusion
-Sudheer
John Machin wrote:
On 8/06/2006 7:35 AM, Sudheer Gupta wrote:
Hi,
Hi.
Your later correction doesn't clear up the confusion below. Quick
eye-balling revealed no difference. If you have to correct a minor
typo in a posting, please consider saying change X to Y instead of
reposting the whole thing.
I am having trouble using C struct in python. Hope anyone can help me
out ...
Say, I have my C struct as
typedef struct call
{
struct call *next;
// .
} call_t;
I have a global variable, namely call_pool, which is of type call_t *
You really need to explain what sort of glue you have between your
Python code and your C code.
1. Are you the author of the glue? If not, better ask the author.
2. Are you extending Python with a module written in C, or are you
embedding Python in a C program?
3. If extending, what's your glue? SWIG? something else? hand-crafted?
My python program:
cp = call_pool # no error doing this, means that call_pool is
accessable
Acessible from where? How do you bind the name call_pool to an object?
while cp:
print cp
print cp.next
This is giving me error: There is no member or method name c_next
Not a Python error message. Must be coming from inside your extension
module.
Now, If I just do:
print cp
print cp.next
there is no problem.
Sorry, I don't understand. Above you said it was giving me error.
But I am seeing a difference in the way python is looking at the struct:
print cp - (call_t*) 0xb0...
print cp.next - (struct call *) 0xb0...
If they are actual results from a Python print statement, then I can
only assume the extension module defines types which have (in effect)
str() and/or repr() methods which produce such output.
Python does not know that the C type of cp is (call_t*) and that of
cp.next is (struct call *). In fact it doesn't care, and it shouldn't
care. The extension module could be written in assembly language or
APL or even INTERCAL if it obeys the conventions, which don't include
exposing a C type for each object. Another way of looking at it:
methods in extension modules are mostly expected to behave like
methods written in Python.
I get the impression that you are using some superglue that parses C
declarations and writes (parts of) Python extension modules in C. Do
you think you could possibly tell us what the name of this superglue is?
Is python not intelligent enough to diagnose the next pointer ??
To the extent to which I can understand what your question means, the
answer is: It by design makes no attempt to be what you are calling
intelligent.
Python does only limited inspection of the tables of methods that an
extension type says it supports. It uses only the very basic
information: is the pointer to method X NULL?
I hope some of the above helps you amplify your question.
Cheers,
John
--
http://mail.python.org/mailman/listinfo/python-list
