[issue23606] ctypes.util.find_library("c") no longer makes sense

[Alex Gaynor]

Alex Gaynor added the comment:

Yeah, this got me (happy to explain what I was trying to do in more detail, if 
it'd be helpful), took me longer to understand why my tests passed on 
{26,27,33,34} but failed on 35 since the public "what's changed" docs page is 
where I went to.

Ultimately I discovered the root cause when I started reading the 
find_library() source code, and found this issue :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library("c") no longer makes sense

[Steve Dower]

Steve Dower added the comment:

Noted.

Did this bite you somehow? Is there something else that should be added/changed?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library("c") no longer makes sense

[Alex Gaynor]

Alex Gaynor added the comment:

An FYI for the future, it would have been very helpful if this had been 
documented in the whats-changed file for 3.5.

--
nosy: +alex

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library("c") no longer makes sense

[Eryk Sun]

Eryk Sun added the comment:

I occasionally come across code snippets on Stack Overflow, and projects such 
as win-unicode-console (IIRC), that use ctypes to work with C stdio FILE 
streams (sometimes for dubious reasons, such as a DLL API that uses FILE 
streams). Maybe the _ctypes extension module could provide void pointers for 
the current C stdin, stdout, and stderr -- as well as stdio functions such as 
fflush, fopen, and freopen. This is already done with _ctypes._memmove_addr and 
_ctypes._memset_addr. However, getting the current standard stream pointers 
would need to use a callable or descriptor. 

> it'd be good content for a "Best practices" section 

The tutorial itself is outdated in places and doesn't promote best practices. 
For example, it assigns a ValidHandle function to 
windll.kernel32.GetModuleHandleA.restype, which would affect every module that 
uses windll.kernel32. Also, this ValidHandle example is bogus, as is every 
example in the tutorial that uses GetModuleHandle without setting restype to a 
pointer type such as c_void_p. It's truncating 64-bit pointers to 32-bit int 
values. You just need to try a DLL that loads at a high address:

>>> kernel32.GetModuleHandleA(b'advapi32')
-27590656
>>> def ValidHandle(value):
... if value == 0:
... raise WinError()
... return value
...
>>> kernel32.GetModuleHandleA.restype = ValidHandle
>>> kernel32.GetModuleHandleA(b'advapi32')
-27590656

>>> hex(kernel32.GetModuleHandleA(b'advapi32') & 2**32-1)
'0xfe5b'
>>> kernel32.GetModuleHandleA.restype = c_void_p 
>>> hex(kernel32.GetModuleHandleA(b'advapi32'))  
'0x7fefe5b'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library("c") no longer makes sense

[Steve Dower]

Steve Dower added the comment:

That's a bit too much of a tangent for me to note in-place in the docs, but 
it'd be good content for a "Best practices" section (I seem to recall we had 
one somewhere, that basically started with "avoid ctypes if possible"...)

Given we've heard no feedback about this change (well, I've heard no feedback - 
if anyone else has please let me know) I'm going to consider this closed.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library("c") no longer makes sense

[Roundup Robot]

Roundup Robot added the comment:

New changeset f9dc71b566fb by Steve Dower in branch '3.5':
Issue #23606: Adds note to ctypes documentation regarding cdll.msvcrt.
https://hg.python.org/cpython/rev/f9dc71b566fb

New changeset 6d84fe4d8cb0 by Steve Dower in branch 'default':
Issue #23606: Adds note to ctypes documentation regarding cdll.msvcrt.
https://hg.python.org/cpython/rev/6d84fe4d8cb0

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library("c") no longer makes sense

[Eryk Sun]

Eryk Sun added the comment:

If the examples continue to use printf, then msvcrt.dll is the best option. The 
universal CRT exports a single Swiss-Army-knife function, 
__stdio_common_vfprintf, which requires 5 parameters, including a va_list for 
the variadic argument list. That's not appropriate for a tutorial. We just need 
a docs update to warn that msvcrt.dll has its own set of file descriptors, 
heap, and thread-locale state.

> Does it mean `cdll.msvcrt` is not the standard way to access 
> the C symbols anymore?

On a tangent, cdll.msvcrt shouldn't be recommended anywhere in the ctypes docs. 
All code that uses cdll.msvcrt shares the same restype, argtypes, and errcheck 
prototypes for each function pointer, which is an unholy mess. The docs should 
recommend CDLL('mscvrt') or CDLL('msvcrt', use_errno=True). Every module or 
package should use its own private instance of CDLL.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library("c") no longer makes sense

[Steve Dower]

Steve Dower added the comment:

The msvcrt module? I don't think so.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library("c") no longer makes sense

[Antoine Pitrou]

Antoine Pitrou added the comment:

> Strictly there's nothing incorrect about the docs, and `cdll.msvcrt` is no 
> more incorrect than it has been since Python 2.4 or so (whenever we  stopped 
> using VC6). It will find the DLL and you can call the functions, but it isn't 
> the same DLL as the exports in the `msvcrt` module will call.

Is there a documentation of what functions are exposed through it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library("c") no longer makes sense

[Antoine Pitrou]

Antoine Pitrou added the comment:

Does it mean `cdll.msvcrt` is not the standard way to access the C symbols 
anymore? Could you update the docs to reflect the current guidelines?
(https://docs.python.org/3/library/ctypes.html)

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library("c") no longer makes sense

[Steve Dower]

Steve Dower added the comment:

Strictly there's nothing incorrect about the docs, and `cdll.msvcrt` is no more 
incorrect than it has been since Python 2.4 or so (whenever we  stopped using 
VC6). It will find the DLL and you can call the functions, but it isn't the 
same DLL as the exports in the `msvcrt` module will call.

Might be worth a note pointing this out, but it's probably better off with a 
different example. I'll try and think of something, but I'm fairly distracted 
at the moment and would appreciate suggestions. Otherwise I'll get there 
eventually.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library("c") no longer makes sense

[eryksun]

eryksun added the comment:

> some code that uses it will need updating (due to API changes 
> since VC9/VC10)

For example, I discovered that the stdio exports don't include printf, etc. 
Using __stdio_common_vfprintf to implement printf relies on calling 
__acrt_iob_func to get stdout and (in general) dynamically building a va_list 
argument list. Of course, relying on either operation is a bad idea.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library(c) no longer makes sense

[Steve Dower]

Steve Dower added the comment:

I've been ignoring this because I wasn't assigned...

Here's the options. If we make it load ucrtbase.dll directly (which does still 
have named exports, and also uses the API schema on Windows 10):
* some code that uses it will need updating (due to API changes since VC9/VC10)
* code may need to change depending on OS updated (which could change the 
ucrtbase exports)
* much existing code using this will probably work

If we leave it as is:
* all existing uses will obviously fail, nothing subtle
* users will either substitute the name themselves, find an equivalent stdlib 
function, or use a supported Windows API

Having just written that out, I still think not supporting it is best, but I do 
need to write it up in the porting info still. Any other thoughts?

--
assignee:  - steve.dower

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23606
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library(c) no longer makes sense

[Christoph Gohlke]

Changes by Christoph Gohlke cgoh...@uci.edu:


--
nosy: +cgohlke

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23606
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library(c) no longer makes sense

[Steve Dower]

Steve Dower added the comment:

Pretty much, except the entry point DLL version won't increment unless there's 
a breaking change to the API. So you have to know where it's from, but (AIUI) 
the l1-0-0 file will always be available. At some point it may turn into a 
wrapper rather than a redirect, but that doesn't really matter.

I'm not sure there's any better way to provide access to the functions other 
than adding them to msvcrtmodule. It's more work, but far more friendly. I hope 
we have a pretty low bar for adding functions to that module.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23606
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library(c) no longer makes sense

[eryksun]

eryksun added the comment:

Say I need to use ctypes to call _wsopen_s to open a file without write 
sharing. If I read you correctly, you're saying I'll need to know it's exported 
by api-ms-win-crt-stdio-l1-1-0.dll? Does the 'l1-1-0' suffix reflect a version 
number that will be incremented over time? If so, how about adding an API that 
maps 'stdio' to the version that Python is linked against?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23606
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library(c) no longer makes sense

[Roundup Robot]

Roundup Robot added the comment:

New changeset fabbe6093567 by Steve Dower in branch 'default':
Issue #23606: Temporarily suppress test for CRT name.
https://hg.python.org/cpython/rev/fabbe6093567

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23606
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library(c) no longer makes sense

[Roundup Robot]

Roundup Robot added the comment:

New changeset 86c9ef950288 by Steve Dower in branch 'default':
Issue #23606: Disable ctypes.util.find_library(c) on Windows so tests are 
skipped while we figure out how best to approach the CRT change
https://hg.python.org/cpython/rev/86c9ef950288

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23606
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library(c) no longer makes sense

[Steve Dower]

Steve Dower added the comment:

That change should get the buildbots passing again, as it will now skip those 
tests. The appcrt%d return value was blatantly incorrect anyway.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23606
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library(c) no longer makes sense

[Steve Dower]

Steve Dower added the comment:

Good question, I'm not sure. Right now, the api-* DLLs use forwarding rather 
than apisetschema, but that could change. Since they support back to XP though, 
I suspect it probably won't the api-* DLLs will stay as they are with ucrtbase 
exporting ordinals rather than names.

ucrtbased is probably going to stay as it is. When you install a debug runtime 
you're voluntarily giving up well-defined versioning. You'll only have that 
file installed if you have VS 2015 or later though - it won't be part of the 
updates for everyone.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23606
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library(c) no longer makes sense

[eryksun]

eryksun added the comment:

Will manual forwarding eventually be replaced by the loader's ApiSetMap (in the 
process PEB), i.e. will ucrtbase join ntdll, sechost, kernelbase, and kernel32 
in apisetschema.dll? In this case will the CRT API sets only exist in winsxs, 
i.e. will they no longer be hard linked in System32? 

What about debug builds? python35_d.dll links directly to ucrtbased.dll:

C:\Program Files\Python35dumpbin /dependents python35_d.dll | find /i 
.dll
Dump of file python35_d.dll
WS2_32.dll
KERNEL32.dll
ADVAPI32.dll
VCRUNTIME140D.dll
ucrtbased.dll

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23606
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library(c) no longer makes sense

[eryksun]

eryksun added the comment:

The api-ms-win-crt-* DLLs forward their exports to ucrtbase.dll, which 
currently uses named exports. When is it planned to remove the named exports?

 crt = CDLL('ucrtbase')
 filesystem = CDLL('api-ms-win-crt-filesystem-l1-1-0')  
 math = CDLL('api-ms-win-crt-math-l1-1-0')

 c_void_p.from_buffer(crt._stat64)
c_void_p(8791677890384)
 c_void_p.from_buffer(filesystem._stat64)
c_void_p(8791677890384)

 c_void_p.from_buffer(crt.fabs)  
c_void_p(8791678417616)
 c_void_p.from_buffer(math.fabs)
c_void_p(8791678417616)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23606
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library(c) no longer makes sense

[Steve Dower]

Steve Dower added the comment:

AIUI, by the time Windows 10 or Visual Studio 2015 releases (since it's now a 
Windows component, it's technically on a different schedule, even though the 
main developer is still working against Visual Studio's schedule) and probably 
sooner (VS 2015 RC is most likely).

It's always possible that they'll decide not to remove them, but the current 
plan is that loading ucrtbase.dll directly will not be helpful, since it would 
prevent Windows from making breaking changes to the API that the api-*.dll 
files would mask.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23606
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library(c) no longer makes sense

[Steve Dower]

Steve Dower added the comment:

That was my original thought, but it's going to lose all of its named exports 
and effectively become useless for this.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23606
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library(c) no longer makes sense

[Steve Dower]

New submission from Steve Dower:

With the changes to the CRT on Windows, it no longer makes any sense to call 
find_library(c) or (m) as there is no single C Runtime DLL. The new 
structure has a grouped and layered approach that is better for versioning, so 
we now link to api-ms-win-crt-*-l1-1-0.dll where * is something like 
filesystem, heap, locale, math, string, etc. and the l1-1-0 part is 
a version.

I don't know what the intended purpose of this function is, so I can't suggest 
a good replacement other than very fast deprecation in 3.4 and raising an error 
in 3.5. Any better suggestions?

--
components: ctypes
messages: 237510
nosy: amaury.forgeotdarc, belopolsky, meador.inge, steve.dower, tim.golden, 
zach.ware
priority: normal
severity: normal
status: open
title: ctypes.util.find_library(c) no longer makes sense
type: behavior
versions: Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23606
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library(c) no longer makes sense

[eryksun]

eryksun added the comment:

Shouldn't find_library(c) return ucrtbase.dll or ucrtbased.dll (debug)? 

Introducing the Universal CRT
http://blogs.msdn.com/b/vcblog/archive/2015/03/03/introducing-the-universal-crt.aspx

--
nosy: +eryksun

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23606
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com