For Test/Main, I have the user's ~/.bash_profile set $PYTHON_EGG_CACHE on a per-node basis. This could also be done per-node and per-pty to ensure uniqueness per job.
--nate On Sep 18, 2012, at 11:24 AM, James Taylor wrote: > Interesting. If I'm reading this correctly the problem is happening > inside pkg_resources? (galaxy.eggs unzips eggs, but I think it does so > on install [fetch_eggs] time not run time which would avoid this). If > so this would seem to be a locking bug in pkg_resources. Dannon, we > could put a guard around the imports in extract_dataset_part.py as an > (overly aggressive and hacky) fix. > > -- jt > > > On Tue, Sep 18, 2012 at 10:37 AM, Jorrit Boekel > <jorrit.boe...@scilifelab.se> wrote: >> - which lead to unzipping .so libraries from python eggs into the nodes' >> /home/galaxy/.python-eggs >> - this runs into lib/pkg_resources.py and its _bypass_ensure_directory >> method that creates the temporary dir for the egg unzip >> - since there are 8 processes on the node, sometimes this method tries to >> mkdir a directory that was just made by the previous process after the >> isdir. > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
