[issue17310] Ctypes callbacks shows problem on Windows Python (64bit)

[Vinay Sajip]

Vinay Sajip added the comment:

I'm closing this as a duplicate of issue 20160 - it appears to be the same bug.

--
resolution:  -> duplicate
stage: test needed -> resolved
status: open -> closed
superseder:  -> broken ctypes calling convention on MSVC / 64-bit Windows 
(large structs)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17310] Ctypes callbacks shows problem on Windows Python (64bit)

[Patrick Stewart]

Patrick Stewart added the comment:

I've attached a patch with an extra fix to the duplicated issue 20160 
http://bugs.python.org/issue20160

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17310] Ctypes callbacks shows problem on Windows Python (64bit)

[Patrick Stewart]

Patrick Stewart added the comment:

This also fixes python 3.5

--
nosy: +Patrick Stewart

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17310] Ctypes callbacks shows problem on Windows Python (64bit)

[Vinay Sajip]

Vinay Sajip added the comment:

I note that the code for CFFI has a fix in ffi_prep_incoming_args_SYSV, as 
follows:

#ifdef _WIN64
  if (z  8)
{
  /* On Win64, if a single argument takes more than 8 bytes,
 then it is always passed by reference. */
  *p_argv = *((void**) argp);
  z = 8;
}
  else
#endif
  *p_argv = (void*) argp; /* The original code, works for 32-bit */

(Source: 
https://bitbucket.org/cffi/cffi/src/abc8ff5b2885b3d9f22bbb314a011b8dd63c9e14/c/libffi_msvc/ffi.c?at=default)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17310
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17310] Ctypes callbacks shows problem on Windows Python (64bit)

[Vinay Sajip]

Vinay Sajip added the comment:

Although Matt was able to work around his problem, this problem seems to still 
be present in 2.7 as well as all 3.x versions. I think I've found the cause of 
the problem: in 64-bit code, the calling conventions for passing structures by 
value are different. From this page:

https://msdn.microsoft.com/en-us/library/zthk2dkh(v=vs.90).aspx

I would point to Structs/unions of size 8, 16, 32, or 64 bits and __m64 will 
be passed as if they were integers of the same size. Structs/unions other than 
these sizes will be passed as a pointer to memory allocated by the caller. For 
these aggregate types passed as a pointer (including __m128), the 
caller-allocated temporary memory will be 16-byte aligned.

The code in ffi_prep_incoming_args_SYSV (see 
https://hg.python.org/cpython/file/a1b76c1c3be8/Modules/_ctypes/libffi_msvc/ffi.c#l368)
 assumes (see lines 399, 402) that all value parameters are always passed 
inline on the stack - which is plainly not as per the documentation I linked 
to, for 64-bit code.

--
nosy: +vinay.sajip

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17310
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17310] Ctypes callbacks shows problem on Windows Python (64bit)

[Vinay Sajip]

Vinay Sajip added the comment:

I can confirm that the CFFI patch works as expected on Python 2.7.10.

--
stage:  - test needed
versions: +Python 3.4, Python 3.5, Python 3.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17310
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17310] Ctypes callbacks shows problem on Windows Python (64bit)

[Matt Clarke]

Matt Clarke added the comment:

It seems that any argument greater than 8 bytes is automatically converted
to a references. Thus, changing to using ctypes.POINTER works. For example:

callback_t = ctypes.CFUNCTYPE(None, ctypes.POINTER(myst_args))

Quite a simple solution in the end. Is it worth documenting this on the
ctypes page?

Thanks for your help,

Matt

On 18 March 2013 15:19, Matt Clarke rep...@bugs.python.org wrote:


 Matt Clarke added the comment:

 Hi Amaury.

 They are both 12 bytes.

 Matt

 --

 ___
 Python tracker rep...@bugs.python.org
 http://bugs.python.org/issue17310
 ___


--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17310
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17310] Ctypes callbacks shows problem on Windows Python (64bit)

[Matt Clarke]

Matt Clarke added the comment:

Hi Amaury.

I have tried removing pack and using /Zp1, but it makes no difference.
The size of callback_t and the one in C are 8 bytes.

Thanks,

Matt

On 13 March 2013 13:19, Amaury Forgeot d'Arc rep...@bugs.python.org wrote:


 Amaury Forgeot d'Arc added the comment:

 Is _pack_ = 1 correct? Did you compile your C library with /Zp1 or
 similar?
 Also check that ctypes.sizeof(callback_t) matches the one given by the C
 compiler.

 --

 ___
 Python tracker rep...@bugs.python.org
 http://bugs.python.org/issue17310
 ___


--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17310
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17310] Ctypes callbacks shows problem on Windows Python (64bit)

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc added the comment:

Sorry, I asked the wrong question; callback_t is a function pointer, so 8 bytes 
is expected.
What is sizeof(myst_args) both in C and Python?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17310
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17310] Ctypes callbacks shows problem on Windows Python (64bit)

[Matt Clarke]

Matt Clarke added the comment:

Hi Amaury.

They are both 12 bytes.

Matt

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17310
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17310] Ctypes callbacks shows problem on Windows Python (64bit)

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc added the comment:

Is _pack_ = 1 correct? Did you compile your C library with /Zp1 or similar?
Also check that ctypes.sizeof(callback_t) matches the one given by the C 
compiler.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17310
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17310] Ctypes callbacks shows problem on Windows Python (64bit)

[Terry J. Reedy]

Changes by Terry J. Reedy tjre...@udel.edu:


--
nosy: +amaury.forgeotdarc, meador.inge

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17310
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17310] Ctypes callbacks shows problem on Windows Python (64bit)

[Matt Clarke]

New submission from Matt Clarke:

I have had an issue arise with ctypes callbacks with 64bit Python on Windows.
Note: everything works fine with 32bit Python on Windows and on 32bit and 64bit 
Linux.

I have created a simple example to illustrate the issue I have (see 
attachment), but the real-life issue occurs with using Python to interact with 
the EPICS control software (http://www.aps.anl.gov/epics/) used at many major 
scientific institutes.

Basically, if I have a C callback that takes a struct (by value) greater than 8 
bytes then the callback returns nonsense. 8 bytes or less works fine.

Stepping through with the Windows debugger, if appears that something goes 
amiss between the callback being called in C and the closure_fcn(ffi_cif *cif, 
void *resp, void **args, void *userdata) function in ctypes's callback.c file. 
Unfortunately, the debugger won't let me step in between those two points.

Looking at the memory I can see the original data in memory at some memory 
address, X, and a copy of the data at X+40 bytes, but the args in the 
closure_fcn points at X-40 bytes (which is junk).

Using 32bit Python the data copy is at X-40 bytes and the args pointer in the 
closure_fcn also points at this.

EPICS has some 64bit C/C++ clients that work fine using callbacks on Windows. 
Likewise, doing the same sort of thing as ctypes does with EPICS from C# using 
PInvoke works fine.
 
Any help would be much appreciated.

--
components: ctypes
files: code.txt
messages: 183156
nosy: Matt.Clarke
priority: normal
severity: normal
status: open
title: Ctypes callbacks shows problem on Windows Python (64bit)
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file29265/code.txt

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17310
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com