[issue43816] Missing 'extern "C"' for _Py_ctype_table

[STINNER Victor]

STINNER Victor  added the comment:

Stéphane Wirtel: "In fact, _Py_ctype_table is limited to the internal parts of 
the interpreter. So in this case, this one could not be used in an external 
tool. You can read: https://docs.python.org/3/c-api/stable.html;

You're right that pyctype.h is excluded from the limited C API. But almost all 
C extensions use the "regular" C API which includes header files in 
Include/cpython/.

The internal C API is something different: header files in Include/internal/.

I know that it's complicated, that why a README was written :-)
https://github.com/python/cpython/blob/master/Include/README.rst

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[STINNER Victor]

STINNER Victor  added the comment:

The workaround for this issue is to put extern "C" around #include , 
or around the header file which includes .

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[STINNER Victor]

STINNER Victor  added the comment:

It's now fixed. Thanks Andrew V. Jones for the fix.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 15ad30d88fead718b7eeff8c54454b82753d520e by Miss Islington (bot) 
in branch '3.9':
bpo-43816: Add extern "C" to Include/cpython/pyctype.h (GH-25365) (GH-25387)
https://github.com/python/cpython/commit/15ad30d88fead718b7eeff8c54454b82753d520e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[miss-islington]

miss-islington  added the comment:


New changeset 47a894d338f436ea8f513f1ad2cc7e1b086e99cc by Miss Islington (bot) 
in branch '3.8':
bpo-43816: Add extern "C" to Include/cpython/pyctype.h (GH-25365)
https://github.com/python/cpython/commit/47a894d338f436ea8f513f1ad2cc7e1b086e99cc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[miss-islington]

Change by miss-islington :


--
pull_requests: +24120
pull_request: https://github.com/python/cpython/pull/25387

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 5.0 -> 6.0
pull_requests: +24119
pull_request: https://github.com/python/cpython/pull/25386

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 54db51c9114ac49030832f5134979ca866ffd21c by Andrew V. Jones in 
branch 'master':
bpo-43816: Add extern "C" to Include/cpython/pyctype.h (GH-25365)
https://github.com/python/cpython/commit/54db51c9114ac49030832f5134979ca866ffd21c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[Stefan Behnel]

Stefan Behnel  added the comment:

The macros were moved to "Includes/cpython/", not to "Includes/internal/". That 
suggests to me that they should use "extern C", so that C++ code that wants to 
make use of CPython internals can use them.

Basically, the way I see it, *all* header files in "Includes/" and 
"Includes/cpython/" should work with C++ code and thus have an "extern C". Only 
the header files in "Includes/internal/" should not have them.

Regardless, I've removed the macro usage from Cython so that the current state 
doesn't hurt our users.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[STINNER Victor]

STINNER Victor  added the comment:

See https://github.com/python/cpython/blob/master/Include/README.rst for the 
organization of the C API.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[Andrew V. Jones]

Andrew V. Jones  added the comment:

> I think the fact that they've been moved to Include/cpython means that user
> code shouldn't be using them.
>

I think it is fine to say that they shouldn't be used, but then we get this 
from Victor's blog:

> It was decided that internal header files must not be included implicitly by
> the generic #include , but included explicitly. 
>

So, is it the case that we have two issues here:

1) Cython is using stuff it shouldn't (I can do a PR against Cython)

2) Python.h is exposing more than it should (so, if Python "core" wants 
something from pyctype.h, it should be explicitly including pyctype.h and not 
getting via Python.h)

?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[Eric V. Smith]

Eric V. Smith  added the comment:

These files were probably added as part of str.format() or short float repr. I 
think the fact that they've been moved to Include/cpython means that user code 
shouldn't be using them. See https://bugs.python.org/issue35134 and 
https://vstinner.github.io/split-include-directory-python38.html. Probably best 
to ask Victor.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[Stefan Behnel]

Stefan Behnel  added the comment:

These macros are a sort-of documented part of the C-API. At least, they were 
mentioned in a What's New document:

https://docs.python.org/3/whatsnew/2.7.html?highlight=py_isspace#build-and-c-api-changes

They were added here, for the Py2.7 release:

https://github.com/python/cpython/commit/8374981fb4d781d5ddbca313656bd3f32b49cef4

It looks to me like the header file should use "extern C".
@Eric, do you agree?

--
nosy: +eric.smith, scoder
type:  -> compile error
versions: +Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[Andrew V. Jones]

Andrew V. Jones  added the comment:

> 1) The Cython-generated code uses `Py_ISSPACE` (and not `_Py_ctype_table`), 
> but the expansion of the macro `Py_ISSPACE` then adds `_Py_ctype_table` to 
> the user's code
>

I wrote this up as a Cython bug here (just to see if the Cython team consider 
this "their" bug): https://github.com/cython/cython/issues/4111

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
components: +C API

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[Andrew V. Jones]

Andrew V. Jones  added the comment:

> I am happy to re-assign this as a Cython bug, but the fact it is fixed with 
> an `extern "C"` in Python.h, really makes it feel like it is a Python-proper 
> issue and not a "user" issue.
>

Just to extend on this:

1) The Cython-generated code uses `Py_ISSPACE` (and not `_Py_ctype_table`), 
but the expansion of the macro `Py_ISSPACE` then adds `_Py_ctype_table` to the 
user's code

2) The "user-fix" is to wrap `#include ` in `extern "C"` -- 
however, given other parts of Python.h already do this, it seems extraneous to 
expect a C++ user to wrap Python.h in this way

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[Andrew V. Jones]

Andrew V. Jones  added the comment:

> In fact, _Py_ctype_table is limited to the internal parts of the interpreter. 
> So in this case, this one could not be used in an external tool.
>

Hmm, so why is this "exposed" by the "world-facing" `Python.h` file?

I should say: we found this bug via Cython; and it was Cython that was 
accessing/referring to `_Py_ctype_table` -- our Cythonated code pulls in C++ 
headers, so we need to compile these files as C++.

I am happy to re-assign this as a Cython bug, but the fact it is fixed with an 
`extern "C"` in Python.h, really makes it feel like it is a Python-proper issue 
and not a "user" issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

In fact, _Py_ctype_table is limited to the internal parts of the interpreter. 
So in this case, this one could not be used in an external tool.

You can read: https://docs.python.org/3/c-api/stable.html

I am not sure that you correctly use the API.

--
nosy: +matrixise

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[Andrew V. Jones]

Change by Andrew V. Jones :


--
keywords: +patch
pull_requests: +24099
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/25365

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43816] Missing 'extern "C"' for _Py_ctype_table

[Andrew V. Jones]

New submission from Andrew V. Jones :

With Python 3.9.4, and when compiling with Visual Studio 2019, we have noticed 
that the variable `_Py_ctype_table` is *not* scoped with in an `extern "C"` 
block, and where the Python library (`python39.lib`) *has* been compiled with a 
C compiler.

This causes an issue when trying to refer to `_Py_ctype_table` from a C++ file, 
as the compiler tries to name-mangle the _use_ of `_Py_ctype_table`, but the 
linker cannot then tie the mangled name to non-mangled named from  
`python39.lib`.

Example:

```
#include "Python.h"
int main() { return _Py_ctype_table[0]; }
```

Compilation:

```
cl.exe /Fe:test.exe /TP /I include test.cpp /link libs/python39.lib
Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29336 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

test.cpp
Microsoft (R) Incremental Linker Version 14.28.29336.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:test.exe
libs/python39.lib
test.obj
test.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) 
unsigned int const * const _Py_ctype_table" (__imp_?_Py_ctype_table@@3QBIB) 
referenced in function main
test.exe : fatal error LNK1120: 1 unresolved externals
```

With `cl.exe`:

```
cl.exe /Bv  
Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29336 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

Compiler Passes:
 Z:\home\avj\visual_studio\MSVC\14.28.29333\bin\HostX64\x64\cl.exe:
Version 19.28.29336.0
 Z:\home\avj\visual_studio\MSVC\14.28.29333\bin\HostX64\x64\c1.dll:
Version 19.28.29336.0
 Z:\home\avj\visual_studio\MSVC\14.28.29333\bin\HostX64\x64\c1xx.dll:  
Version 19.28.29336.0
 Z:\home\avj\visual_studio\MSVC\14.28.29333\bin\HostX64\x64\c2.dll:
Version 19.28.29336.0
 Z:\home\avj\visual_studio\MSVC\14.28.29333\bin\HostX64\x64\c1xx.dll:  
Version 19.28.29336.0
 Z:\home\avj\visual_studio\MSVC\14.28.29333\bin\HostX64\x64\link.exe:  
Version 14.28.29336.0
 Z:\home\avj\visual_studio\MSVC\14.28.29333\bin\HostX64\x64\mspdb140.dll:  
Version 14.28.29336.0
 Z:\home\avj\visual_studio\MSVC\14.28.29333\bin\HostX64\x64\1033\clui.dll: 
Version 19.28.29336.0
```

A naïve check of Python.h (e126547c07) seems to suggest that:

* There are 82 includes

* 64 of these contain `extern "C"`

* 8 do not contain `extern "C"`

* The remaining 10 are either system includes or pyconfig.h

For the 8 that *do not* contain `extern "C"`, none of these use `PyAPI_DATA`. 
This leads me to believe that it is an oversight that `pyctype.h` does not have 
`extern "C"`

--
messages: 390855
nosy: andrewvaughanj
priority: normal
severity: normal
status: open
title: Missing 'extern "C"' for _Py_ctype_table
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com