On Jan 30, 2005, at 5:50 PM, Martin v. Löwis wrote:
Tony Meyer wrote:The main question (to steal Thomas's words) is whether the library modules
should be written to help the freeze tools - if the answer is 'yes', then
I'll submit the above as a patch for 2.5.
The answer to this question certainly is "yes, if possible". In this specific case, I wonder whether the backwards compatibility is still required in the first place. According to PEP 291, Greg Smith and Barry Warsaw decide on this, so I think they would need to comment first because any patch can be integrated. If they comment that 2.1 compatibility is still desirable, your patch would be fine (I guess); if they say that the compatibility requirement can be dropped for 2.5, I suggest that the entire exec statement is removed, along with the conditional clause.
py2app <http://undefined.org/python/#py2app> handles this situation by using a much richer way to analyze module dependencies, in that it can use hooks (called "recipes") to trigger arbitrary behavior when the responsible recipe sees that a certain module is in the dependency graph. This is actually necessary to do the right thing in the case of extensions and modules that are not friendly with bytecode analysis. Though there are not many of these in the standard library, a few common packages such as PIL have a real need for this. Also, since modulegraph uses a graph data structure it is much better suited to pruning the dependency graph. For example, pydoc imports Tkinter to support an obscure feature, but this is almost never desired in the context of an application freeze tool. py2app ships with a recipe that automatically breaks the edge between pydoc and Tkinter <http://svn.red-bean.com/bob/py2app/trunk/src/py2app/recipes/ docutils.py>, so if Tkinter is not explicitly included or used by anything else in the dependency graph, it is correctly excluded from the resultant application bundle.
In order to correctly cover the Python API, I needed to ALWAYS include: unicodedata, warnings, encodings, and weakref because they can be used by the implementation of Python itself without any "import" hints (which, if py2exe also did this, would've probably solved Tony's issue with bsddb). Also, I did an analysis of the Python standard library and I discovered that the following (hopefully rather complete) list of implicit dependencies (from <http://svn.red-bean.com/bob/py2app/trunk/src/modulegraph/ find_modules.py>):
{
# imports done from builtin modules in C code (untrackable by modulegraph)
"time": ["_strptime"],
"datetime": ["time"],
"MacOS": ["macresource"],
"cPickle": ["copy_reg", "cStringIO"],
"parser": ["copy_reg"],
"codecs": ["encodings"],
"cStringIO": ["copy_reg"],
"_sre": ["copy", "string", "sre"],
"zipimport": ["zlib"],
# mactoolboxglue can do a bunch more of these
# that are far harder to predict, these should be tracked
# manually for now.
# this isn't C, but it uses __import__
"anydbm": ["dbhash", "gdbm", "dbm", "dumbdbm", "whichdb"],
}I would like to write a PEP for modulegraph as a replacement for modulefinder at some point, but I can't budget the time for it right now. The current implementation is also largely untested on other platforms. I hear it has been used by a Twisted developer to create Windows NT services using py2exe (augmenting py2exe's rather simple dependency resolution mechanism), however I'm not sure if that is in public svn or not. If the authors of the other freeze tools are interested, they can feel free to use modulegraph from py2app -- it is cross-platform code under MIT license, but I can dual-license if necessary (however I think it should be compatible with cx_freeze, py2exe, and Python itself). The API is purposefully different than modulefinder, but it is close enough such that most of the work involved is just removing unnecessary kludges.
-bob
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
