[issue21736] Add __file__ attribute to frozen modules

[Eric Snow]

Eric Snow  added the comment:

I've opened the following issues related to frozen stdlib modules:

https://bugs.python.org/issue45652
https://bugs.python.org/issue45658
https://bugs.python.org/issue45659

Again, I'm leaving this issue open to deal with the broader question of frozen 
modules outside the stdlib, where the solution isn't so obvious.  The question 
of targeting the FileLoader (or SourceLoader) ABC should probably be handled 
separately.  (Note that bpo-45020 has some related discussion.)

--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

Weird.  PR 28655 is merged on GH, but still shows open on this bpo ticket.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Gregory P. Smith]

Gregory P. Smith  added the comment:

That appears to have caused https://bugs.python.org/issue45506

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Eric Snow]

Eric Snow  added the comment:

I've merged the change to add __file__ for frozen stdlib modules (when 
possible) and to set __path__ appropriately.  There are still a number of 
things to address though:

* set co_filename (for tracebacks)
* implement FrozenImporter.get_filename()
* implement FrozenImporter.get_source()

And then there's the question of supporting __file__, etc. for custom frozen 
modules.  Presumably all those things are still covered by this issue so I'm 
leaving it open.

--
stage: patch review -> needs patch
versions: +Python 3.11 -Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Eric Snow]

Eric Snow  added the comment:


New changeset 79cf20e48d0b5d69d9fac2a0204b5ac2c366066a by Eric Snow in branch 
'main':
bpo-21736: Set __file__ on frozen stdlib modules. (gh-28656)
https://github.com/python/cpython/commit/79cf20e48d0b5d69d9fac2a0204b5ac2c366066a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +27022
pull_request: https://github.com/python/cpython/pull/28656

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +27021
pull_request: https://github.com/python/cpython/pull/28655

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Guido van Rossum]

Guido van Rossum  added the comment:

> Note that the filename information is already available in the
> code object's co_filename attribute.

Hm, in the latest (3.11) main that's not true -- co_filename is set to 
something like "". And that's about right, since the filename 
contained in the frozen data can at best reflect the filename at the time the 
freeze script ran, which is not necessarily the same as the filename when the 
user runs the Python binary.

I don't think we should try to set __file__ or any other attributes on frozen 
modules (at least not for the small set of frozen modules we're contemplating 
for 3.11).  User who need the file can run their Python 3.11 binary with 
-Xfrozen_modules=off, to disable the use of frozen modules (other than the 
three importlib bootstrap files).

Tools that freeze the entire stdlib or 3rd party code may have to make a  
different choice, but that's not our problem (yet).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +26748
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28335

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Guido van Rossum]

Change by Guido van Rossum :


--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 15.06.2014 01:53, Nick Coghlan wrote:
 
 Can we just drop __file__ and set the origin for frozen modules to
 something that includes the original file name?

This wouldn't really help, because too much code out there uses
the __file__ attribute and assumes it's always available.

Note that the filename information is already available in the
code object's co_filename attribute.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21736
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Eric Snow]

Eric Snow added the comment:

__file__ is the filename from which the module *was* loaded (the inspect doc 
[1] should probably reflect that [2]).  The import machinery only uses the 
module's __spec__ (origin, etc.).  __file__ is set strictly as informational 
(and for backward-compatibility).

Per the language reference [3], __file__ may be omitted when it does not have 
semantic meaning.  It also says Ultimately, the loader is what makes use of 
__file__, but that hasn't been accurate since PEP 451 landed. [4]  Notably, 
though, for now the module __repr__() *does* use __file__ if it is available 
(and the loader doesn't implement module_repr).  The counterpart of __file__ 
within a module's spec is __spec__.origin.  The two should stay in sync.  In 
the case of frozen modules origin is set to frozen.

Giving __file__ to frozen modules is inaccurate.  The file probably won't be 
there and the module certainly wasn't loaded from that location.

Stdlib modules should not rely on all module's having __file__.  Removing 
__file__ from frozen modules was a change in 3.4 and I'd consider it a 3.4 bug 
if any stdlib module relied on it.  For that matter, I'd consider it a bug if a 
module relied on all modules having __file__ or even __file__ being set to an 
actual filename.

Would it be inappropriate to set an additional informational attribute on 
frozen modules to indicate the original path?  Something like 
__frozen_filename__.  Then you wouldn't need to rely on __code__.co_filename.

p.s. Searching for __file__ in the docs [5] illustrates its prevalence.

[1] https://docs.python.org/3/library/inspect.html#types-and-members
[2] issue #21760
[3] https://docs.python.org/3/reference/import.html#__file__
[4] issue #21761
[5] https://docs.python.org/3/search.html?q=__file__

--
nosy: +brett.cannon, eric.snow, ncoghlan

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21736
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Nick Coghlan]

Nick Coghlan added the comment:

Can we just drop __file__ and set the origin for frozen modules to
something that includes the original file name?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21736
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Marc-Andre Lemburg]

New submission from Marc-Andre Lemburg:

The missing __file__ attribute on frozen modules causes lots of issues with the 
stdlib (see e.g. Issue21709 and the stdlib test suite) and other tools that 
expect this attribute to always be present.

The attached patch for 3.4.1 adds this attribute to all frozen modules and 
resolves most issues. It cannot resolve the problem of not necessarily finding 
the directories/files listed in those attributes, but at least makes it 
possible to continue using code that only uses the attribute for error 
reporting.

--
components: Interpreter Core, Library (Lib)
messages: 220362
nosy: lemburg
priority: normal
severity: normal
status: open
title: Add __file__ attribute to frozen modules
versions: Python 3.4, Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21736
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Marc-Andre Lemburg]

Changes by Marc-Andre Lemburg m...@egenix.com:


--
keywords: +patch
Added file: http://bugs.python.org/file35595/__file__-for-frozen-modules.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21736
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Barry A. Warsaw]

Changes by Barry A. Warsaw ba...@python.org:


--
nosy: +barry

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21736
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

I'm -0 on this patch.  I can understand that in some sense, frozen modules do 
semantically have an associated file, but OTOH, once they're frozen the 
connection to their file is broken.  Also, I think anything that assumes 
__file__ exists is simply broken and should be fixed.  There are other cases 
than frozen modules where a module would have no reasonable value for __file__ 
and thus shouldn't have one.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21736
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 12.06.2014 18:35, Barry A. Warsaw wrote:
 
 I'm -0 on this patch.  I can understand that in some sense, frozen modules do 
 semantically have an associated file, but OTOH, once they're frozen the 
 connection to their file is broken.  Also, I think anything that assumes 
 __file__ exists is simply broken and should be fixed.  There are other cases 
 than frozen modules where a module would have no reasonable value for 
 __file__ and thus shouldn't have one.

This one falls into the practicality beats purity category. Of
course, the __file__ attribute doesn't always makes sense as
file path, but it does serve an information purpose.

We're doing this in eGenix PyRun to get 3rd party code working
(including parts of the Python stdlib :-)). Not doing so
would simply lead to the whole freezing approach pretty much
useless, since so much code uses the attribute without checking
or providing a fallback solution.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21736
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21736] Add __file__ attribute to frozen modules

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

PBP might be reasonably used to justify it for the frozen case.  I just don't 
want to use that as a wedge to define __file__ in *all* cases, even when no 
reasonable file name exists.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21736
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com