[issue42839] SourceFileLoader does not (fully) accept path-like objects

[Petr Viktorin]

Petr Viktorin  added the comment:

I just filed the slightly more general bpo-45127.

--
nosy: +petr.viktorin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42839] SourceFileLoader does not (fully) accept path-like objects

[Filipe Laíns]

Change by Filipe Laíns :


--
nosy: +FFY00

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42839] SourceFileLoader does not (fully) accept path-like objects

[Guido van Rossum]

Guido van Rossum  added the comment:

If Brett's theory is right, compile() has a bug though -- it shouldn't plug a 
non-string into the code object. Or possibly the code object constructor should 
reject such objects (or extract the string).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42839] SourceFileLoader does not (fully) accept path-like objects

[Brett Cannon]

Brett Cannon  added the comment:

importlib is probably not os.PathLike-clean due to its bootstrapping 
restrictions of not getting to use anything implemented in Python from 'os' 
(i.e. if it isn't being managed in posixmodule.c then it probably won't work).

If you follow the traceback it's trying to marshal a code object for the 
eventual .pyc file and failing 
(https://github.com/python/cpython/blob/faf49573963921033c608b4d2f398309d9f0d2b5/Lib/importlib/_bootstrap_external.py#L604).
 The real question is why is any unmarshallable object getting passed in the 
first place since that object is the code object that compile() returned.

Best guess? The compile() function is being given the path-like object (via 
https://github.com/python/cpython/blob/faf49573963921033c608b4d2f398309d9f0d2b5/Lib/importlib/_bootstrap_external.py#L848)
 and it's blindly setting it on the code object itself, and then marhsal fails 
since it can't handle pathlib.Path. If my hunch is right, then the possible 
solutions are:

- Don't do that 
- Make compile() aware of path-like objects
- Have importlib explicitly work around compile()'s shortcoming by doing the 
right thing for path-like objects before passing in the the 'path' argument.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42839] SourceFileLoader does not (fully) accept path-like objects

[Guido van Rossum]

Guido van Rossum  added the comment:

Maybe Brett can help?

--
nosy: +brett.cannon, gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42839] SourceFileLoader does not (fully) accept path-like objects

[favonia]

New submission from favonia :

If one uses the loader created by 
importlib.machinery.SourceFileLoader(module_name, path) in a meta path finder, 
where path is not a str, then the following error would happen at the moment 
the module is being imported. Note that, the error would not occur if the 
corresponding bytecode cache (__pycache__/*.pyc) is present.

  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "", line 1030, in _gcd_import
  File "", line 1007, in _find_and_load
  File "", line 972, in _find_and_load_unlocked
  File "", line 228, in _call_with_frames_removed
  File "", line 1030, in _gcd_import
  File "", line 1007, in _find_and_load
  File "", line 986, in _find_and_load_unlocked
  File "", line 680, in _load_unlocked
  File "", line 786, in exec_module
  File "", line 932, in get_code
  File "", line 604, in 
_code_to_timestamp_pyc
ValueError: unmarshallable object

Here is an example that could trigger the error. The package unipath is not 
very important except that unipath.Path is not marshallable. (Any library with 
a fancy, unmarshallable path-like object would work.) The choice of 'local' and 
the use of 'os.getcwd()' are also irrelevant. The actual production code is 
quite complicated, using different path-like objects and convoluted logic. I 
made up this example from scratch.

import os
import sys
import importlib
import importlib.abc
import importlib.util
from unipath import Path # just an example

class DummyFinder(importlib.abc.MetaPathFinder):
def __init__(self):
self.cwd = os.getcwd()
def find_spec(self, fullname, path, target=None):
if fullname == 'local':
initpath = os.path.join(self.cwd, '__init__.py')
if os.path.isfile(initpath):
loader = importlib.machinery.SourceFileLoader(fullname, 
Path(initpath)) # some non-str Path-like object here
return importlib.util.spec_from_file_location(fullname, 
initpath,
loader = loader, submodule_search_locations=[self.cwd])
else:
return None

sys.meta_path.append(DummyFinder())
importlib.import_module("local.test2")

If this is indeed a bug, there might be other classes and functions in 
importlib that share the same problem.

--
components: Library (Lib)
messages: 384504
nosy: favonia
priority: normal
severity: normal
status: open
title: SourceFileLoader does not (fully) accept path-like objects
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com