On Wed, Feb 23, 2011 at 10:57 AM, Jason Grout <jason-s...@creativetrax.com> wrote: > On 2/23/11 12:28 PM, William Stein wrote: >> >> At lunch yesterday Robert Bradshaw made the interesting suggestion to >> read the docs for importlib >> (http://docs.python.org/dev/library/importlib.html) and write a >> customized import hook, so that every time during Sage startup that a >> module is imported, the import is done from a single big in-memory zip >> file instead of done using the filesystem. If this can be made to >> work, it would be a huge win for slow filesystems. The basic problem >> is that some filesystems are fast but have huge*latency*. > > Is it a big win primarily because the zip file contents can be read in and > cached by us? I'm just trying to understand it better.
Which would you rather do on a high latency filesystem: (1) Read/stat 20,000 little files, or (2) Read exactly one 40MB file. > Is this the same idea as Jar files in java? I don't know. > You mean like http://docs.python.org/library/zipimport.html ? Cool. I don't know for a fact that Robert Bradshaw's suggestion will be a big win, since nobody has tried this yet. But I'm optimistic. The idea would be to make a zip archive of $SAGE_ROOT/local/lib/python/site-packages (say), and do *all* imports using that massive zip archive. -- William -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
