Re: Problems running modified spark version on ec2 cluster

2014-08-04 Thread Matt Forbes
After rummaging through the worker instances I noticed they were using the assembly jar (which I hadn't noticed before). Now instead of using the core and mllib jars individually, I'm just overwriting the assembly jar in the master and using spark-ec2/copy-dir. For posterity, my run script is: MAS

Problems running modified spark version on ec2 cluster

2014-08-04 Thread Matt Forbes
I'm trying to run a forked version of mllib where I am experimenting with a boosted trees implementation. Here is what I've tried, but can't seem to get working properly: *Directory layout:* src/spark-dev (spark github fork) pom.xml - I've tried changing the version to 1.2 arbitrarily in core