Thanks to Aniket’s work there is two new options to the EMR install script for
Spark. See
https://github.com/awslabs/emr-bootstrap-actions/blob/master/spark/README.md
The “-a” option can be used to bump the spark-assembly to the front of the
classpath.
-Christopher
From: Aniket Bhatnagar
Meanwhile, I have submitted a pull request (
https://github.com/awslabs/emr-bootstrap-actions/pull/37) that allows users
to place their jars ahead of all other jars in spark classpath. This should
serve as a temporary workaround for all class conflicts.
Thanks,
Aniket
On Mon Jan 05 2015 at
Just an update on this - I found that the script by Amazon was the culprit
- not exactly sure why. When I installed Spark manually onto the EMR (and
did the manual configuration of all the EMR stuff), it worked fine.
On Mon, Dec 22, 2014 at 11:37 AM, Adam Gilmore dragoncu...@gmail.com
wrote:
Can you confirm your emr version? Could it be because of the classpath
entries for emrfs? You might face issues with using S3 without them.
Thanks,
Aniket
On Mon, Jan 5, 2015, 11:16 AM Adam Gilmore dragoncu...@gmail.com wrote:
Just an update on this - I found that the script by Amazon was the
