I too felt the same Nick but I don't have root privileges on the cluster,
unfortunately. Are there any alternatives?
On 27 June 2014 08:04, Nick Pentreath nick.pentre...@gmail.com wrote:
I've not tried this - but numpy is a tricky and complex package with many
dependencies on Fortran/C
Would deploying virtualenv on each directory on the cluster be viable?
The dependencies would get tricky but I think this is the sort of
situation it's built for.
On 6/27/14, 11:06 AM, Avishek Saha wrote:
I too felt the same Nick but I don't have root privileges on the
cluster, unfortunately.
I suppose along those lines, there's also Anaconda:
https://store.continuum.io/cshop/anaconda/
On 6/27/14, 11:13 AM, Nick Pentreath wrote:
Hadoopy uses http://www.pyinstaller.org/ to package things up into an
executable that should be runnable without root privileges. It says it
support numpy
Hi all,
Instead of installing numpy in each worker node, is it possible to
ship numpy (via --py-files option maybe) while invoking the
spark-submit?
Thanks,
Avishek
