Thanks Divya for helping me. It does connect to the internet.
And I even tried to use a local artifactory repository and it didn't work either. $ ./spark-shell --packages harsha2010:magellan:1.0.3-s_2.10 --repositories http://localhost:8081/artifactory/libs-release Ivy Default Cache set to: /Users/jeffhuang/.ivy2/cache The jars for the packages stored in: /Users/jeffhuang/.ivy2/jars http://localhost:8081/artifactory/libs-release added as a remote repository with the name: repo-1 :: loading settings :: url = jar:file:/Users/jeffhuang/learning/spark/spark-1.5.1-bin-hadoop2.6/lib/spark-assembly-1.5.1-hadoop2.6.0.jar!/org/apache/ivy/core/settings/ivysettings.xml harsha2010#magellan added as a dependency :: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0 confs: [default] found harsha2010#magellan;1.0.3-s_2.10 in spark-packages found commons-io#commons-io;2.4 in local-m2-cache found com.esri.geometry#esri-geometry-api;1.2.1 in central found org.json#json;20090211 in local-m2-cache found org.codehaus.jackson#jackson-core-asl;1.9.12 in local-m2-cache :: resolution report :: resolve 282ms :: artifacts dl 10ms :: modules in use: com.esri.geometry#esri-geometry-api;1.2.1 from central in [default] commons-io#commons-io;2.4 from local-m2-cache in [default] harsha2010#magellan;1.0.3-s_2.10 from spark-packages in [default] org.codehaus.jackson#jackson-core-asl;1.9.12 from local-m2-cache in [default] org.json#json;20090211 from local-m2-cache in [default] On Fri, Feb 5, 2016 at 5:01 PM, Divya Gehlot <divya.htco...@gmail.com> wrote: > Is your machine connected to internet ? > > > On 5 February 2016 at 12:00, Jeff - Data Bean Australia < > databean...@gmail.com> wrote: > >> Hi, >> >> Trying to use an external package with the following command line and it >> doesn't work: >> >> ./spark-shell --packages harsha2010:magellan:1.0.3-s_2.10 >> >> Please find the package info here: >> http://spark-packages.org/package/harsha2010/magellan >> >> The error related to one of the dependencies was not found successfully: >> >> [NOT FOUND ] >> org.codehaus.jackson#jackson-core-asl;1.9.12!jackson-core-asl.jar (1ms) >> >> And the weird part is that Spark or Ivy didn't actually try to look it up >> online, but simply reported error when it didn't find the package in my >> local m2 repository. Please see the complete error message below and help >> me out. >> >> MacBook-Pro-2:bin jeffhuang$ ./spark-shell --packages >> harsha2010:magellan:1.0.3-s_2.10 >> Ivy Default Cache set to: /Users/jeffhuang/.ivy2/cache >> The jars for the packages stored in: /Users/jeffhuang/.ivy2/jars >> :: loading settings :: url = >> jar:file:/Users/jeffhuang/learning/spark/spark-1.5.1-bin-hadoop2.6/lib/spark-assembly-1.5.1-hadoop2.6.0.jar!/org/apache/ivy/core/settings/ivysettings.xml >> harsha2010#magellan added as a dependency >> :: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0 >> confs: [default] >> found harsha2010#magellan;1.0.3-s_2.10 in spark-packages >> found commons-io#commons-io;2.4 in local-m2-cache >> found com.esri.geometry#esri-geometry-api;1.2.1 in central >> found org.json#json;20090211 in local-m2-cache >> found org.codehaus.jackson#jackson-core-asl;1.9.12 in local-m2-cache >> :: resolution report :: resolve 282ms :: artifacts dl 9ms >> :: modules in use: >> com.esri.geometry#esri-geometry-api;1.2.1 from central in [default] >> commons-io#commons-io;2.4 from local-m2-cache in [default] >> harsha2010#magellan;1.0.3-s_2.10 from spark-packages in [default] >> org.codehaus.jackson#jackson-core-asl;1.9.12 from local-m2-cache in >> [default] >> org.json#json;20090211 from local-m2-cache in [default] >> --------------------------------------------------------------------- >> | | modules || artifacts | >> | conf | number| search|dwnlded|evicted|| number|dwnlded| >> --------------------------------------------------------------------- >> | default | 5 | 0 | 0 | 0 || 5 | 0 | >> --------------------------------------------------------------------- >> >> :: problems summary :: >> :::: WARNINGS >> [NOT FOUND ] >> org.codehaus.jackson#jackson-core-asl;1.9.12!jackson-core-asl.jar (1ms) >> >> ==== local-m2-cache: tried >> >> >> >> file:/Users/jeffhuang/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.9.12/jackson-core-asl-1.9.12.jar >> >> :::::::::::::::::::::::::::::::::::::::::::::: >> >> :: FAILED DOWNLOADS :: >> >> :: ^ see resolution messages for details ^ :: >> >> :::::::::::::::::::::::::::::::::::::::::::::: >> >> :: org.codehaus.jackson#jackson-core-asl;1.9.12!jackson-core-asl.jar >> >> :::::::::::::::::::::::::::::::::::::::::::::: >> >> >> >> :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS >> Exception in thread "main" java.lang.RuntimeException: [download failed: >> org.codehaus.jackson#jackson-core-asl;1.9.12!jackson-core-asl.jar] >> at >> org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1009) >> at >> org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:286) >> at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:153) >> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120) >> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) >> >> -- >> Data Bean - A Big Data Solution Provider in Australia. >> > > -- Data Bean - A Big Data Solution Provider in Australia.
