I'm having another issue freshening up my easybuild environment.
My cluster is not directly Internet connected and compute nodes do not
have any Internet access at all. In the past I've been able to build
easybuild based programs once I've externally fetched the sources and
placed them in a source tree.
With 2.8.1 (and I think starting with 2.7.0) I'm now unable to rebuild
easybuild itself into its final place. There seems to be at least one
failing connection attempt in the easybuild build process itself.
Hand transcribed output from the error log:
== testing...
== installimg...
== FAILED: Installation ended unsuccessfully...300 chars: cmd "
/usr/bin/python setup.py install --prefix....
Hand transcribed portion of the --keep-log log file:
ERROR cmd " /usr/bin/python setup.py install --prefix....
This is based on vsc.install.shared_setup 0.10.6
...
Searching for vsc-install>=0.10.1
Reading http://pypi.python.org/simple/vsc-install/
WARN: Download error: [errno -2] name or service not found.....
On my external VM I see the following file/directory being fetched:
GET "http://pypi.python.org/simple/vsc-install/"; "Python-urllib/2.6
distribute/0.6.10"
It only seems to read the directory and not download any actual files.
On the non-Internet connected system this seems to be the cause of the
installation abort.
Is vsc-install required? It doesn't actually seem to download it. I
thought it was actually packaged in easybuild itself. I think there
have been issues in the past with vsc-base requirements. I don't see
a CentOS RPM that provides this functionality (but may have missed
it).
I have tried placing the current vsc-install source tarball in my
sources directory at various places that seem likely but haven't found
a path which works (v/, v/vsc-install/, in a python modules
directory). Is there somewhere I can drop the source file to work
around this problem?
Please excuse the vagueness of this report. Getting machine readable
data out from my build system is very awkward at the present time.
========
For reference (in the past) I have had reasonable success with build
on a non-Internet connected system with the following process (again
waving hands a little bit since I'm not on my actual build system at
the moment):
I build a scratch KVM Based virtual machine with Internet access. I
can then download and execute the easybuild bootstrap script.
Alternatively, I can copy over an already installed easybuild
installation and use it.
I can then use the --stop=fetch (along with the occasional --robot and
--force options) to get easybuild to download most source distribution
files.
I always use a constant --sourcepath option so I accumulate a cache of
downloaded sources which I can transfer to the build system.
This works for most of the sources, however there are a couple of
exceptions where this doesn't work (I think these actually apply even
if you are Internet connected):
- Certain files require a manual download, possibly accepting a
license agreement through a web interface. In some cases finding the
download link is difficult and easybuild could help by giving an
accurate starting link (some do, some are missing or wrong).
- Occasionally a file distribution location has changed and the .eb
files needs to be updated (or the source file manually downloaded).
- Source distribution files "disappear". I keep a long term cache of
all source distribution files I've previously downloaded.
- Other than the above, I don't think I've run into anything which
actually requires Internet access at build time if you have all the
source tarballs. Anything needing Internet access at build/install
time is not supplying complete sources and I consider broken.
Even with having source tarballs, you may not have the actual sources.
A disturbing number of distribution tarballs are now coming with
precompiled binaries and not true sources.
Thanks,
Stuart
--
I've never been lost; I was once bewildered for three days, but never lost!
-- Daniel Boone
