On 9/27/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote:
Yeah, how to get the proper information to the data importers is going to be the trick.
I will think about it, but I am still trying to get the original question of how bad the C code is compared to rewriting import in Python from people. =)
-Brett
At 05:26 PM 9/27/2006 -0700, Brett Cannon wrote:
>Ah, OK. So for importing 'email', the zipimporter would call the .pyc
>importer and it would ask the zipimporter, "can you get me email.pyc?" and
>if it said no it would move on to asking the .py importer for email.py, etc.
Yes, exactly.
>That's fine. Just thinking about how the current situation sucks for NFS
>but how caching just isn't done. But obvoiusly this could change.
Well, with this design, you can have a CachingFilesystemImporter as your
storage mechanism to speed things up.
>> >>Of course, to fully support .pyc timestamp checking and writeback, you'd
>> >>need some sort of "stat" or "getmtime" feature on the parent importer, as
>> >>well as perhaps an optional "save_data" method. These would be extensions
>> >>to PEP 302, but welcome ones.
>> >
>> >Could pass the string representing the location of where the string came
>> >from. That would allow for the required stat calls for .pyc files as
>> >needed without having to implement methods just for this one use case.
>>
>>Huh? In order to know if a .pyc is up to date, you need the st_mtime of
>>the .py file. That can't be done in the parent importer without giving it
>>format knowledge, which goes against the point of the exercise.
>
>Sorry, thought .pyc files based whether they needed to be recompiled based
>on the stat info on the .py and .pyc file, not on data stored from within
>the .pyc .
It's not just that (although I believe it's also the case that there is a
timestamp inside .pyc), it's that to do the check in the parent importer,
the parent importer would have to know that there is such a thing as
.py-and-.pyc. The whole point of this design is that the parent importer
doesn't have to know *anything* about filename extensions OR how those
files are formatted internally. In this scheme, adding more child
importers is sufficient to add all the special handling needed for
.py/.pyc-style schemes.
Of course, for maximum flexibility, you might want get_stream() and
get_file() methods optionally available, since a .so loader really needs a
file, and .pyc might want to read in two stages. But the child importers
can be defensively coded so as to be able to live with only a
parent.get_data(), if necessary, and do the enhanced behaviors only if
stat() or get_stream() or write_data() etc. attributes are available on the
parent.
Yeah, how to get the proper information to the data importers is going to be the trick.
If we get some standards for these additional attributes, we can document
them as standard PEP 302 extensions.
The format importer mechanism might want to have something like
'sys.import_formats' as a list of importer classes (or factories). Parent
(storage) importer classes would then create instances to use.
If you add a new format importer to sys.import_formats, you would of course
need to clear sys.path_importer_cache, so that the individual importers are
rebuilt on the next import, and thus they will create new child importer
chains.
Yeah, that pretty much ought to do it.
I will think about it, but I am still trying to get the original question of how bad the C code is compared to rewriting import in Python from people. =)
-Brett
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
