Hi, Trying to use an external package with the following command line and it doesn't work:
./spark-shell --packages harsha2010:magellan:1.0.3-s_2.10 Please find the package info here: http://spark-packages.org/package/harsha2010/magellan The error related to one of the dependencies was not found successfully: [NOT FOUND ] org.codehaus.jackson#jackson-core-asl;1.9.12!jackson-core-asl.jar (1ms) And the weird part is that Spark or Ivy didn't actually try to look it up online, but simply reported error when it didn't find the package in my local m2 repository. Please see the complete error message below and help me out. MacBook-Pro-2:bin jeffhuang$ ./spark-shell --packages harsha2010:magellan:1.0.3-s_2.10 Ivy Default Cache set to: /Users/jeffhuang/.ivy2/cache The jars for the packages stored in: /Users/jeffhuang/.ivy2/jars :: loading settings :: url = jar:file:/Users/jeffhuang/learning/spark/spark-1.5.1-bin-hadoop2.6/lib/spark-assembly-1.5.1-hadoop2.6.0.jar!/org/apache/ivy/core/settings/ivysettings.xml harsha2010#magellan added as a dependency :: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0 confs: [default] found harsha2010#magellan;1.0.3-s_2.10 in spark-packages found commons-io#commons-io;2.4 in local-m2-cache found com.esri.geometry#esri-geometry-api;1.2.1 in central found org.json#json;20090211 in local-m2-cache found org.codehaus.jackson#jackson-core-asl;1.9.12 in local-m2-cache :: resolution report :: resolve 282ms :: artifacts dl 9ms :: modules in use: com.esri.geometry#esri-geometry-api;1.2.1 from central in [default] commons-io#commons-io;2.4 from local-m2-cache in [default] harsha2010#magellan;1.0.3-s_2.10 from spark-packages in [default] org.codehaus.jackson#jackson-core-asl;1.9.12 from local-m2-cache in [default] org.json#json;20090211 from local-m2-cache in [default] --------------------------------------------------------------------- | | modules || artifacts | | conf | number| search|dwnlded|evicted|| number|dwnlded| --------------------------------------------------------------------- | default | 5 | 0 | 0 | 0 || 5 | 0 | --------------------------------------------------------------------- :: problems summary :: :::: WARNINGS [NOT FOUND ] org.codehaus.jackson#jackson-core-asl;1.9.12!jackson-core-asl.jar (1ms) ==== local-m2-cache: tried file:/Users/jeffhuang/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.9.12/jackson-core-asl-1.9.12.jar :::::::::::::::::::::::::::::::::::::::::::::: :: FAILED DOWNLOADS :: :: ^ see resolution messages for details ^ :: :::::::::::::::::::::::::::::::::::::::::::::: :: org.codehaus.jackson#jackson-core-asl;1.9.12!jackson-core-asl.jar :::::::::::::::::::::::::::::::::::::::::::::: :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS Exception in thread "main" java.lang.RuntimeException: [download failed: org.codehaus.jackson#jackson-core-asl;1.9.12!jackson-core-asl.jar] at org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1009) at org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:286) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:153) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) -- Data Bean - A Big Data Solution Provider in Australia.
