[issue5710] ctypes should return composite types from callbacks

[Russell Keith-Magee]

Russell Keith-Magee  added the comment:

For those interested, we developed a workaround for this in Rubicon: 

https://github.com/pybee/rubicon-objc/pull/85/files

The fix involves using ctypes to access ctypes own internals, and build a 
modified version of the Structure data type that is able to perform a copy when 
used as returned value. 

Hopefully we'll be able to get this into the form of a patch for ctypes that is 
acceptable to Python core.

--
nosy: +freakboy3742

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5710] ctypes should return composite types from callbacks

[Pam McA'Nulty]

Pam McA'Nulty  added the comment:

oops - 
"In the example code, the 'result' variable is declared on the stack in the 
callback() function.  So it will be effectively when callback() returns." 
should be 
"In the example code, the 'result' variable is declared on the stack in the 
callback() function.  So it will be effectively DESTROYED when callback() 
returns."

I hate waking up at 4:15a.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5710] ctypes should return composite types from callbacks

[Pam McA'Nulty]

Pam McA'Nulty  added the comment:

In the example code, the 'result' variable is declared on the stack in the 
callback() function.  So it will be effectively when callback() returns.

Also the "result" variable in main() is never referenced.  A pointer main()'s 
"result" variable should be passed to callback() as an argument ("") and 
`callback(struct helloStruct *result)` should populate it (via `result->i = 
10;` etc)

```struct helloStruct callback () {
  struct helloStruct result;
  result.i = 10;
  result.f = 3.14159;
  return result;
}```

--
nosy: +Pam.McANulty

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5710] ctypes should return composite types from callbacks

[Albert Zeyer]

Albert Zeyer added the comment:

Any update here?

--
nosy: +Albert.Zeyer
versions: +Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5710] ctypes should return composite types from callbacks

[R. David Murray]

Changes by R. David Murray rdmur...@bitdance.com:


--
assignee: theller - 
versions: +Python 3.4 -Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5710
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5710] ctypes should return composite types from callbacks

[Mason Bially]

Mason Bially added the comment:

I agree with Amaury that this is purely a user side concern. While I think it's 
important to note the behaviour of ctypes in the case that Thomas describes, I 
believe it's more important to fully support the range of behaviours allowed by 
C function callbacks.

I see the use cases for complex return types that don't fall under the concerns 
raised by Thomas as the following:

* Returning a pointer to already existing memory.
* Returning complex value types (structs of ints, doubles, chars, ect). This is 
especially important from a compatibility standpoint for C libraries which 
expect such return types.

Because I need this for my current project I will work on writing a patch.

--
nosy: +Mason.Bially

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5710
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5710] ctypes should return composite types from callbacks

[Terry J. Reedy]

Changes by Terry J. Reedy tjre...@udel.edu:


--
stage:  - unit test needed
type:  - feature request
versions: +Python 3.2 -Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python 
3.1

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5710
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5710] ctypes should return composite types from callbacks

[Thomas Heller]

Thomas Heller thel...@ctypes.org added the comment:

There is a problem returning arbitrary complicated ctypes types from
callbacks.  Consider that a callback function returns a structure; the
structure itself may contain 'char *' field for example.  The callback
function creates the structure and fills in the fields.  The 'char *'
field contains a pointer to a nul-terminated string; the ctypes
structure object must keep the Python string alive as long as the
structure itself.  It does this via the private '_objects' instance
variable.  If the callback function now returns the structure, the
Python ctypes object usually will go out of scope and will be garbage
collected, together with its '_objects'.  So, the pointer(s) contained
in the structure will become invalid now, and even simple structure
fields will become invalid.

However, the callback function result will be used by some other code
and will surely crash.

In principle it should be possible to make code like this work; one
would have to maintain the returned objects elsewhere and clean them up
when they are no longer used anywhere; but I think this gets too
complicated.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5710
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5710] ctypes should return composite types from callbacks

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc amaur...@gmail.com added the comment:

But isn't this purely a user-side concern?

For example, if I want to use a function such as QBuffer::setBuffer in 
the Qt library:
http://doc.trolltech.com/4.4/qbuffer.html#setBuffer
I must keep a reference to the buffer as long as the QBuffer is alive, 
or expect a crash.

Returning a pointer from a function is always tough, even in C: 
everybody has already tried to return the address of a local variable...
the pointer must belong to some container that outlives the function 
call.
ctypes is not different in this aspect. The same precautions as in C 
apply. And with a warning note in the documentation, there seems to be 
no reason to limit the return type of a callback.

--
nosy: +amaury.forgeotdarc

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5710
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5710] ctypes should return composite types from callbacks

[Greg Holling]

New submission from Greg Holling gholl...@vni.com:

We have an application that calls a 3rd party library that returns a
structure (by value) from a callback.  I'm including some sample code
that duplicates the behavior.  The problem is that return types from
callbacks cannot be anything other than simple datatypes (c_int,
c_float, ..., c_void_p).  For other datatypes (STRUCTURE, POINTER, ...),
ctypes returns the following error:

invalid result type for callback function

The error message comes from callback.c, in function AllocFunctionCallback.

I think this may be a duplicate of issue #1574584.

I've tested this on Windows and linux; I'm including a Makefile and
source code that works on Windows/cygwin.

-- file: Makefile 

all: hello.dll hello.exe hello_dll.exe
clean: 
rm *.exe *.dll

hello.exe: hello.h hello.c hello_main.c
gcc hello.c hello_main.c -o hello.exe --save-temps

hello.dll: hello.h hello.c
gcc -mno-cygwin -shared hello.c -o hello.dll --save-temps

hello_dll.exe: hello.h hello.c
gcc hello_main.c -L. -lhello -o hello_main.exe

-- file: hello.h 
struct helloStruct {
  int i;
  float f;
  int i2;
  int i3;
};

float fxn (struct  helloStruct callback());

-- file: hello.c 

#include stdio.h 
#include hello.h

float fxn (struct helloStruct callback()) {
  struct  helloStruct result = callback();

  printf (i: %d\n, result.i);
  printf (f: %f\n, result.f);
  return result.f * result.i;
}

-- file: hello_main.c 

#include stdio.h 
#include hello.h

struct helloStruct callback();

int main (int argc, char **argv) {
  float f;
  struct helloStruct result;
  printf (Hello world\n);
  f = fxn (callback);
  printf (Callback result: %f\n, f);
}

struct helloStruct callback () {
  struct helloStruct result;
  result.i = 10;
  result.f = 3.14159;
  return result;
}

int int_callback () {
  return 42;
}

-- file: hello.py 

from ctypes import cdll, c_char, c_int, c_float, Structure, CFUNCTYPE,
POINTER, c_char_p

class helloStruct (Structure):
pass
helloStruct._fields_ = [
('i', c_int),
('f', c_float)]

def callback():
print (callback())
hs = helloStruct()
hs.i = 10
hs.f = 25.5
return hs

libc = cdll.msvcrt
#helloLib = libc.load_library(hello)
#helloLib = libc.hello
helloLib = cdll.hello

helloLib.fxn.restype = helloStruct
# It looks like only simple return types are supported for
# callback functions.  simple = c_int, c_float, ...
# Python bug # 1574584 - status: closed.
#Suggests posting to ctypes-users, but I don't see any recent activity.
#
TMP_FCN = CFUNCTYPE (POINTER(c_char))   # Error message
#TMP_FCN = CFUNCTYPE (c_char_p)  # Runs, but invalid result
#TMP_FCN = CFUNCTYPE (c_void_p)  # Runs, but invalid result
#TMP_FCN = CFUNCTYPE (c_int)  # Runs, but invalid result
#TMP_FCN = CFUNCTYPE (POINTER(c_int)) # Error message
#TMP_FCN = CFUNCTYPE (POINTER(helloStruct))   # Error message
#TMP_FCN = CFUNCTYPE (helloStruct)# Error message
callback_fcn = TMP_FCN (callback)
result = helloLib.fxn (callback_fcn)

# 2.5
#print result: , result
# 3.0
print (result: , result)

--
assignee: theller
components: ctypes
messages: 85654
nosy: gholling, theller
severity: normal
status: open
title: ctypes should return composite types from callbacks
versions: Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python 3.1

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5710
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com