[issue25655] Python errors related to failures loading DLL's lack information

[Steve Dower]

Change by Steve Dower :


--
versions: +Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[Steve Dower]

Steve Dower  added the comment:

> Steve, the PR that you pushed has the wrong error and error message. 

Ah whoops. I'll reopen and hopefully someone can fix it up.

> The advice to use dumpbin is fine and works well in simple cases. 

It's only meant as a starting point. It's possible to brute force your way 
through more complex cases, but to describe anything more advanced we'd be 
writing a tutorial in our docs, which is not appropriate.

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[miss-islington]

miss-islington  added the comment:


New changeset f22f874a66d6ddb32fa74ad4325199db7e4c25fd by Miss Skeleton (bot) 
in branch '3.9':
bpo-25655: Improve Win DLL loading failures doc (GH-22372)
https://github.com/python/cpython/commit/f22f874a66d6ddb32fa74ad4325199db7e4c25fd


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[Eryk Sun]

Eryk Sun  added the comment:

Steve, the PR that you pushed has the wrong error and error message. I told 
Philippe in msg377335 that ctypes raises FileNotFoundError with no error code. 
For example:

>>> try: ctypes.CDLL('spam')
... except OSError as e: err = e
...
>>> err
FileNotFoundError("Could not find module 'spam' (or one of its 
dependencies). Try using the full path with constructor syntax.")
>>> err.winerror is None
True

The advice to use dumpbin is fine and works well in simple cases. I wouldn't 
use it generally since recursiveley parsing through the dependency graph of 
every dependent DLL could be tedious.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[miss-islington]

miss-islington  added the comment:


New changeset 5d8bc65ba5be5742b3a4cc470dfd990512bdaa93 by Miss Skeleton (bot) 
in branch '3.8':
bpo-25655: Improve Win DLL loading failures doc (GH-22372)
https://github.com/python/cpython/commit/5d8bc65ba5be5742b3a4cc470dfd990512bdaa93


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[Steve Dower]

Steve Dower  added the comment:

Thanks for the PR!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[miss-islington]

Change by miss-islington :


--
pull_requests: +21827
pull_request: https://github.com/python/cpython/pull/22895

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 8.0 -> 9.0
pull_requests: +21826
pull_request: https://github.com/python/cpython/pull/22894

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[Steve Dower]

Steve Dower  added the comment:


New changeset b6f2fc90409e291822166d74ce7402e0ef4dba91 by Philippe Ombredanne 
in branch 'master':
bpo-25655: Improve Win DLL loading failures doc (GH-22372)
https://github.com/python/cpython/commit/b6f2fc90409e291822166d74ce7402e0ef4dba91


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[Steve Dower]

Steve Dower  added the comment:

Thanks for doing the search :)

 > - 
https://docs.python.org/3/faq/windows.html#is-a-pyd-file-the-same-as-a-dll

Probably not here.

 > - https://docs.python.org/3/using/windows.html#finding-modules

Perhaps it is best to put a new section here like what you posted above 
(but more generic for ctypes and imports), and then link to it from the 
other places?

 > - https://docs.python.org/3/library/os.html?#os.add_dll_directory

e.g. "This function may be used to work around  errors"

 > - https://docs.python.org/3/extending/windows.html

e.g. "If your extension module relies on any DLLs other than those 
included with Windows or CPython, you will need to include them or else 
users may receive 'module not found' errors. See  for more 
details."

(Some of that text may already be there, been a while since I read that 
one.)

 > Also AFAIK there is no Windows Sphinx tag beyond .. availability::

Yeah, I think I was thinking of a different project. But if it's all in 
Windows-specific sections anyway, and pointing towards the Windows doc 
page, then it won't matter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[Philippe Ombredanne]

Philippe Ombredanne  added the comment:

So the other locations to add docs would be petinetially

- https://docs.python.org/3/library/os.html?#os.add_dll_directory
- https://docs.python.org/3/extending/windows.html
- https://docs.python.org/3/faq/windows.html#is-a-pyd-file-the-same-as-a-dll
- https://docs.python.org/3/using/windows.html#finding-modules

Which ones would be the best?

Also AFAIK there is no Windows Sphinx tag beyond .. availability::

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[Steve Dower]

Steve Dower  added the comment:

> would you see this as a note in 
https://docs.python.org/3/library/ctypes.html?highlight=ctypes#loading-shared-libraries

Haven't looked at the PR, but it probably needs to be somewhere in the 
import docs as well, to do with native extension modules. That's where 
most people run into this. And in general the solution is either going 
to involve moving/renaming files or calling os.add_dll_directory, so a 
link to the latter may also be useful.

I think we've got a Sphinx tag for platform-specific information? If we 
do, it should use that. (Unless I'm just thinking of the "API 
availability" tag rather than a "Note" style box.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[Philippe Ombredanne]

Change by Philippe Ombredanne :


--
keywords: +patch
pull_requests: +21411
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/22372

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[Philippe Ombredanne]

Philippe Ombredanne  added the comment:

> I wouldn't refuse a docs PR to add a short section pointing to
> this page and explaining its relevance: 
> https://docs.microsoft.com/cpp/build/reference/dependents

Steve,
would you see this as a note in 
https://docs.python.org/3/library/ctypes.html?highlight=ctypes#loading-shared-libraries

What about something like this?

class ctypes.CDLL
.

Note: On Windows a call to CDLL(name) may fail even if the DLL name exists when 
a dependent DLL of this DLL is found. This will lead to an OSErrror error with 
the message "[WinError 126] The specified module could not be found".

This error message does not contains the name of the missing DLL because the 
Windows API does not return this information making this error hard to diagnose.

To resolve this error and determine which DLL is missing, you need to find the 
list of dependent DLLs using Windows debugging and tracing tools.

See https://docs.microsoft.com/cpp/build/reference/dependents for some 
explanations.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[Steve Dower]

Steve Dower  added the comment:

> IMO, the most direct way to resolve the problem is by enabling "loader snaps" 
> for python.exe via gflags and attaching a native debugger to the process ...

This is indeed the best way to go about solving it, so you can see why we don't 
put it in an error message or take responsibility for documenting the process. 
It's not for the faint-hearted :)

Also, the recommended releases of WinDBG (from the Microsoft Store) no longer 
include gflags, though I believe once you're in the debugger it will just break 
at the point where the DLL can't be loaded and it's "simple" to get its 
expected name.

I wouldn't refuse a docs PR to add a short section pointing to this page and 
explaining its relevance: 
https://docs.microsoft.com/cpp/build/reference/dependents

I *would* stop short of writing a whole tutorial on how to do it. That's a 
great topic for someone's blog, and will likely get better SEO and social 
attention from not being in the docs.

--
versions: +Python 3.10 -Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[Eryk Sun]

Eryk Sun  added the comment:

> " OSError: [WinError 126] The specified module could not be found" is 
> raised when calling ctypes.CDLL(dll_path) even when this "dll_path" 
> exists... because the error comes from another DLL.

That's the old error. bpo-36085 changed it to FileNotFoundError, with the 
message "Could not find module '%.500S'. Try using the full path with 
constructor syntax." bpo-39393 modified the message to "Could not find module 
'%.500S' (or one of its dependencies). Try using the full path with constructor 
syntax."

IMO, the most direct way to resolve the problem is by enabling "loader snaps" 
for python.exe via gflags and attaching a native debugger to the process. The 
loader outputs debug strings that show the computed DLL search path (from 
LdrpComputeLazyDllPath), each attempt to resolve the dependent DLL to a 
directory in the search path (via LdrpResolveDllName), and the final result 
from loader's work queue (from LdrpProcessWork), which includes the dependent 
DLL that caused loading to fail and the parent module (DLL or EXE) that depends 
on it.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[Philippe Ombredanne]

Philippe Ombredanne  added the comment:

Eric Smith, you wrote:

> My understanding is that Windows doesn't tell you which DLL is missing. I 
> think the best we could do is append something to the error message saying 
> "or one its dependencies".

If we have such an error message, this means the main DLL exists: the original 
path passed to ctypes exists and is a valid DLL otherwise the message would be 
different. 

So I think that this is always a direct or indirect dependency of that primary 
DLL that would be missing and we could be explicit in the error message.

We could also provide some hints in the error message on how to research the 
issue may be?

--
nosy:  -altendky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[Kyle Altendorf]

Change by Kyle Altendorf :


--
nosy: +altendky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[Philippe Ombredanne]

Philippe Ombredanne  added the comment:

>From https://bugs.python.org/issue41836 closed as a dupe of this:

When the dependency of a DLL is missing (at least on Windows) the error " 
OSError: [WinError 126] The specified module could not be found" is raised when 
calling ctypes.CDLL(dll_path) even when this "dll_path" exists... because the 
error comes from another DLL.

These errors are really hard to diagnose because the path of the missing DLL is 
not returned in the exception message. Returning it would help fixing these 
kind of errors quickly.

Researching errors such as this one 
https://github.com/nexB/scancode-toolkit/issues/2236 wastes quite a bit of time 
and would be made a non issue if we had the path in the error message.


and this reply from Eric Smith: https://bugs.python.org/msg377324

> Author: Eric V. Smith (eric.smith) * (Python committer)   Date: 
> 2020-09-22 14:13

> My understanding is that Windows doesn't tell you which DLL is missing. I 
> think the best we could do is append something to the error message saying 
> "or one its dependencies".

--
nosy: +pombredanne

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[Florian Bruhin]

Change by Florian Bruhin :


--
nosy: +The Compiler

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[Steve Dower]

Steve Dower added the comment:

I don't know that we can necessarily provide correct information for those 
errors as it depends where they actually fail. If we're simply passing on an 
error from the loader, then there's very little we can do.

We may be able to make an educated guess based on our context, but there's a 
fairly high chance we'll guess wrong in some cases and make them more difficult 
to debug (unless you know that the error is only a guess, in which case you're 
back to procmon or Dependency Walker).

You're right that all versions are affected, however I think improving these 
diagnostics is only within scope for 3.5 and 3.6 at this stage.

--
versions: +Python 3.5, Python 3.6 -Python 2.7, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25655] Python errors related to failures loading DLL's lack information

[never-eat-yellow-snow]

New submission from never-eat-yellow-snow:

Currently you get errors like this:

ImportError: DLL load failed: The specified procedure could not be found.
ImportError: DLL load failed: The specified module could not be found.

It would be nice to include more information, at least the name of the dll 
which could not be loaded. Maybe also the name of the (dependent) dll which 
could not be found.

Currently, I use ProcessMonitor to debug which dll could not be found, because 
the error message is lacking important information.

Note: I tagged the two versions I use, but probably all python versions are 
affected by this issue.

--
components: Windows
messages: 254838
nosy: never-eat-yellow-snow, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Python errors related to failures loading DLL's lack information
type: enhancement
versions: Python 2.7, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com