[issue42880] ctypes: variadic function call still doesn't work on Apple Silicon

[Ronald Oussoren]

Change by Ronald Oussoren :


--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42880] ctypes: variadic function call still doesn't work on Apple Silicon

[Ronald Oussoren]

Ronald Oussoren  added the comment:

I agree that this should be mentioned in the documentation for ctypes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42880] ctypes: variadic function call still doesn't work on Apple Silicon

[Ziqiao Kong]

Ziqiao Kong  added the comment:

I retry my cases with types specified and it works indeed. Maybe the documents 
should be updated for this case.

Thanks a lot!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42880] ctypes: variadic function call still doesn't work on Apple Silicon

[Ronald Oussoren]

Ronald Oussoren  added the comment:

AFAIK you should specify the types of the fixed arguments for variadic 
functions. The ABI for the M1 has a different calling convention of the 
variadic arguments, the ctypes implementation counts how many variadic 
arguments there are based on the specified number of fixed arguments.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42880] ctypes: variadic function call still doesn't work on Apple Silicon

[Ziqiao Kong]

Ziqiao Kong  added the comment:

Hello, we do a clean reinstallation of Big Sur 11.1. The problem still exists:

with brew-installed python3.9
```
qiling@kamino /tmp % DYLD_PRINT_LIBRARIES=1 python3.9 test_main.py
[...]
dyld: loaded: <52918C9B-7E0B-3852-AB19-F74846BCAAF8> 
/Users/qiling/brew/Cellar/python@3.9/3.9.1_6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload/_heapq.cpython-39-darwin.so
dyld: loaded:  
/Users/qiling/brew/Cellar/python@3.9/3.9.1_6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload/_ctypes.cpython-39-darwin.so
dyld: loaded:  /usr/lib/libffi.dylib
dyld: loaded: <656F7BE5-2BA6-3FE9-BD31-77ED659B6194> 
/Users/qiling/brew/Cellar/python@3.9/3.9.1_6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload/_struct.cpython-39-darwin.so
dyld: loaded:  ./main.dylib
50900232
qiling@kamino /tmp % DYLD_PRINT_LIBRARIES=1 python3.9 test_main.py
[...]
dyld: loaded: <52918C9B-7E0B-3852-AB19-F74846BCAAF8> 
/Users/qiling/brew/Cellar/python@3.9/3.9.1_6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload/_heapq.cpython-39-darwin.so
dyld: loaded:  
/Users/qiling/brew/Cellar/python@3.9/3.9.1_6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload/_ctypes.cpython-39-darwin.so
dyld: loaded:  /usr/lib/libffi.dylib
dyld: loaded: <656F7BE5-2BA6-3FE9-BD31-77ED659B6194> 
/Users/qiling/brew/Cellar/python@3.9/3.9.1_6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload/_struct.cpython-39-darwin.so
dyld: loaded:  ./main.dylib
86961416
```

even the output differs between different runs...

uname -a

```
Darwin kamino.lan 20.2.0 Darwin Kernel Version 20.2.0: Wed Dec  2 20:40:21 PST 
2020; root:xnu-7195.60.75~1/RELEASE_ARM64_T8101 arm64
```

btw, the python 3.10.0a4 built from sources:

```
qiling@kamino cpython-3.10.0a4 % DYLD_PRINT_LIBRARIES=1 ./python.exe 
test_main.py
dyld: loaded: <483C8366-9BE1-3A03-B41A-4A526F6A6F91> 
/Users/qiling/cpython-3.10.0a4/./python.exe
[...]
dyld: loaded: <97E08C95-0A13-3030-AC8D-3330CB10AD0B> 
/Users/qiling/cpython-3.10.0a4/build/lib.macosx-11.1-arm64-3.10/_ctypes.cpython-310-darwin.so
dyld: loaded:  /usr/lib/libffi.dylib
dyld: loaded: <9CA1E334-8C96-362A-B21F-9A2C43ACF58D> 
/Users/qiling/cpython-3.10.0a4/build/lib.macosx-11.1-arm64-3.10/_struct.cpython-310-darwin.so
dyld: loaded:  ./main.dylib
3
qiling@kamino cpython-3.10.0a4 % DYLD_PRINT_LIBRARIES=1 ./python.exe 
test_main.py
dyld: loaded: <483C8366-9BE1-3A03-B41A-4A526F6A6F91> 
/Users/qiling/cpython-3.10.0a4/./python.exe
[...]
dyld: loaded: <97E08C95-0A13-3030-AC8D-3330CB10AD0B> 
/Users/qiling/cpython-3.10.0a4/build/lib.macosx-11.1-arm64-3.10/_ctypes.cpython-310-darwin.so
dyld: loaded:  /usr/lib/libffi.dylib
dyld: loaded: <9CA1E334-8C96-362A-B21F-9A2C43ACF58D> 
/Users/qiling/cpython-3.10.0a4/build/lib.macosx-11.1-arm64-3.10/_struct.cpython-310-darwin.so
dyld: loaded:  ./main.dylib
3
qiling@kamino cpython-3.10.0a4 %
```

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42880] ctypes: variadic function call still doesn't work on Apple Silicon

[Ziqiao Kong]

Ziqiao Kong  added the comment:

Sorry that our test machine failed to boot due to some firmware problem when 
upgrading to 11.1 yesterday. I will re-run my tests after the upgrade gets done.

Also, I'm sure the last output is "3" in my first message.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42880] ctypes: variadic function call still doesn't work on Apple Silicon

[Ned Deily]

Ned Deily  added the comment:

> Seems that we are getting the same libffi.dylib.

I suppose it is still possible that another libffi is being found first. To be 
absolutely sure, you could ask dyld to print the loaded libraries:

$ DYLD_PRINT_LIBRARIES=1 python3.9 test_main.py
[...]
dyld: loaded: <003A027D-9CE3-3794-A319-88495844662D> 
/usr/lib/system/libxpc.dylib
dyld: loaded: <7FBC5290-B2B3-3312-B69A-77378C539520> 
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload/_ctypes.cpython-39-darwin.so
dyld: loaded: <943473A5-A82B-3E21-8EB3-0BBDF5E605AA> /usr/lib/libffi.dylib
dyld: loaded: <58910956-4F2F-363A-80E5-D5E1C71DD83E> 
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload/_struct.cpython-39-darwin.so
dyld: loaded: <3444F392-DB43-3AEF-8596-2BFFBBD8EC0C> ./main.dylib
34

But, more importantly, the uname -a output for my M1 is:

Darwin ankl.local 20.2.0 Darwin Kernel Version 20.2.0: Wed Dec  2 20:40:21 PST 
2020; root:xnu-7195.60.75~1/RELEASE_ARM64_T8101 x86_64

Perhaps you have not updated to macOS 11.1 yet?
 
By the way, in your original message's last example, is the output really "3"?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42880] ctypes: variadic function call still doesn't work on Apple Silicon

[Ziqiao Kong]

Ziqiao Kong  added the comment:

BTW, uname for the M1 MBP.

```
kabeor@kamino cpython-3.10.0a4 % uname -a
Darwin kamino.lan 20.1.0 Darwin Kernel Version 20.1.0: Sat Oct 31 00:07:10 PDT 
2020; root:xnu-7195.50.7~2/RELEASE_ARM64_T8101 arm6464
```

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42880] ctypes: variadic function call still doesn't work on Apple Silicon

[Ziqiao Kong]

Ziqiao Kong  added the comment:

Seems that we are getting the same libffi.dylib.

kabeor@kamino cpython-3.10.0a4 % otool -L $(./python.exe -c 'import 
_ctypes;print(_ctypes.__file__)')
/Users/kabeor/cpython-3.10.0a4/build/lib.macosx-11.0-arm64-3.10/_ctypes.cpython-310-darwin.so:
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 
27.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current 
version 1292.60.1)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42880] ctypes: variadic function call still doesn't work on Apple Silicon

[Ned Deily]

Ned Deily  added the comment:

Yes, otool should work.  Something like:

$ otool -L $(python3.9 -c 'import _ctypes;print(_ctypes.__file__)')
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload/_ctypes.cpython-39-darwin.so:
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 
27.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current 
version 1292.0.0)

/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation 
(compatibility version 150.0.0, current version 1770.106.0)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42880] ctypes: variadic function call still doesn't work on Apple Silicon

[Ziqiao Kong]

Ziqiao Kong  added the comment:

Thanks for your reply. What shall I do to confirm the version of libffi? By 
otool?


From: report=bugs.python@roundup.psfhosted.org 
 on behalf of Ned Deily 

Sent: Monday, January 11, 2021 2:17:53 AM
To: ziqiaok...@gmail.com 
Subject: [issue42880] ctypes: variadic function call still doesn't work on 
Apple Silicon

Ned Deily  added the comment:

(I also get *34* when building v3.10.0a4 from source on the M1 with 11.1 and 
Xcode 12.3.)

--

___
Python tracker 
<https://bugs.python.org/issue42880>
___

--

___
Python tracker 
<https://bugs.python.org/issue42880>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42880] ctypes: variadic function call still doesn't work on Apple Silicon

[Ned Deily]

Ned Deily  added the comment:

(I also get *34* when building v3.10.0a4 from source on the M1 with 11.1 and 
Xcode 12.3.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42880] ctypes: variadic function call still doesn't work on Apple Silicon

[Ned Deily]

Ned Deily  added the comment:

I tried your test programs on a M1 Mac running macOS 11.1 with the latest macOS 
universal2 Pythons available from python.org 
(https://www.python.org/downloads/), 3.9.1 and 3.10.0a4, and both produced the 
same (correct) result as in your 3.8 example.  Are you sure the brew Pythons 
are using the macOS 11 system libffi and not their own or older version?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42880] ctypes: variadic function call still doesn't work on Apple Silicon

[Ned Deily]

Change by Ned Deily :


--
components: +macOS
nosy: +ned.deily, ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42880] ctypes: variadic function call still doesn't work on Apple Silicon

[Ziqiao Kong]

New submission from Ziqiao Kong :

Hello! Thanks for your contribution on porting python to native Apple Silicon. 
I have noticed that there is an issue https://bugs.python.org/issue41100 and 
corresponding Github PR https://github.com/python/cpython/pull/22855 stating 
that variadic functions ABI has been corrected. However, during our test, it 
still doesn't work on Apple Silicon with brew-installed python 3.9.1 and python 
3..10.0a4 built from sources. The details are as follows:

A x86_64 Mojave Macmini with python 3.8:
```
/tmp $ python3 --version
Python 3.8.6
/tmp $ uname -a
Darwin *** 18.7.0 Darwin Kernel Version 18.7.0: Mon Apr 27 20:09:39 PDT 2020; 
root:xnu-4903.278.35~1/RELEASE_X86_64 x86_64
/tmp $ cat main.c
#include 
#include 
#include 
void test(uint64_t end, ...){
  int a;
  va_list valist;
  va_start(valist, end);
  a = va_arg(valist, int);
  va_end(valist);
  printf("%d\n", a);
  return;
}
/tmp $ cc -shared -g main.c -o ./main.dylib
/tmp $ cat test_main.py
from ctypes import *

d = CDLL("./main.dylib")
d.test.restype = None
d.test(c_uint64(12), c_int(34))
/tmp $ python3 test_main.py
34
/tmp $
```

An M1 Macbook Pro with brew-installed python 3.9.1
```
kabeor@kamino /tmp % python3 --version
Python 3.9.1
kabeor@kamino /tmp % cat main.c
#include 
#include 
#include 
void test(uint64_t end, ...){
  int a;
  va_list valist;
  va_start(valist, end);
  a = va_arg(valist, int);
  va_end(valist);
  printf("%d\n", a);
  return;
}
kabeor@kamino /tmp % cc -shared -g main.c -o ./main.dylib
kabeor@kamino /tmp % cat test_main.py
from ctypes import *

d = CDLL("./main.dylib")
d.test.restype = None
d.test(c_uint64(12), c_int(34))
kabeor@kamino /tmp % python3 test_main.py
48144104
kabeor@kamino /tmp %
```

An M1 Macbook Pro with python 3.10.0a4 built from Github release tarball
```
kabeor@kamino cpython-3.10.0a4 % ./python.exe --version
Python 3.10.0a4
kabeor@kamino cpython-3.10.0a4 % cp /tmp/main.c ./
kabeor@kamino cpython-3.10.0a4 % cp /tmp/test_main.py ./
kabeor@kamino cpython-3.10.0a4 % ./python.exe --version
Python 3.10.0a4
kabeor@kamino cpython-3.10.0a4 % cat ./main.c
#include 
#include 
#include 
void test(uint64_t end, ...){
  int a;
  va_list valist;
  va_start(valist, end);
  a = va_arg(valist, int);
  va_end(valist);
  printf("%d\n", a);
  return;
}
kabeor@kamino cpython-3.10.0a4 % cc -shared -g main.c -o ./main.dylib
kabeor@kamino cpython-3.10.0a4 % cat test_main.py
from ctypes import *

d = CDLL("./main.dylib")
d.test.restype = None
d.test(c_uint64(12), c_int(34))
kabeor@kamino cpython-3.10.0a4 % ./python.exe test_main.py
3
kabeor@kamino cpython-3.10.0a4 %
```

Thanks in advance!

--
components: ctypes
messages: 384756
nosy: lawrence-danna-apple, lazymio
priority: normal
severity: normal
status: open
title: ctypes: variadic function call still doesn't work on Apple Silicon
versions: Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com