Thanks Sean. Your are right! Yes it works when replacing the bintray repo with
repos.spark-packages.org.
On Wednesday, May 19, 2021, 02:03:14 PM EDT, Sean Owen <[email protected]>
wrote:
I think it's because the bintray repo has gone away. Did you see the recent
email about the new repo for these packages?
On Wed, May 19, 2021 at 12:42 PM Wensheng Deng <[email protected]>
wrote:
Hi experts:
I tried the example as shown on this page, and it is not working for
me:https://spark-packages.org/package/graphframes/graphframes
Please advise how to proceed. I also tried to unzip the zip file, ran 'sbt
assembly', and got an error of 'sbt-spark-package;0.2.6: not found'. Is there a
solution to this?
$ spark-shell --packages graphframes:graphframes:0.8.1-spark2.4-s_2.11Ivy
Default Cache set to: /Users/wsd57/.ivy2/cacheThe jars for the packages stored
in: /Users/wsd57/.ivy2/jars:: loading settings :: url =
jar:file:/Users/wsd57/spark-2.4.0-bin-hadoop2.6/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xmlgraphframes#graphframes
added as a dependency:: resolving dependencies ::
org.apache.spark#spark-submit-parent-eb9df503-f763-49b3-83e9-5afef8b0311d;1.0
confs: [default]:: resolution report :: resolve 1429ms :: artifacts dl 0ms ::
modules in use:
--------------------------------------------------------------------- |
| modules || artifacts | | conf
| number| search|dwnlded|evicted|| number|dwnlded|
--------------------------------------------------------------------- |
default | 1 | 0 | 0 | 0 || 0 | 0 |
---------------------------------------------------------------------
:: problems summary :::::: WARNINGS module not found:
graphframes#graphframes;0.8.1-spark2.4-s_2.11
==== local-m2-cache: tried
file:/Users/wsd57/.m2/repository/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.pom
-- artifact graphframes#graphframes;0.8.1-spark2.4-s_2.11!graphframes.jar:
file:/Users/wsd57/.m2/repository/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.jar
==== local-ivy-cache: tried
/Users/wsd57/.ivy2/local/graphframes/graphframes/0.8.1-spark2.4-s_2.11/ivys/ivy.xml
-- artifact graphframes#graphframes;0.8.1-spark2.4-s_2.11!graphframes.jar:
/Users/wsd57/.ivy2/local/graphframes/graphframes/0.8.1-spark2.4-s_2.11/jars/graphframes.jar
==== central: tried
https://repo1.maven.org/maven2/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.pom
-- artifact
graphframes#graphframes;0.8.1-spark2.4-s_2.11!graphframes.jar:
https://repo1.maven.org/maven2/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.jar
==== spark-packages: tried
http://dl.bintray.com/spark-packages/maven/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.pom
-- artifact
graphframes#graphframes;0.8.1-spark2.4-s_2.11!graphframes.jar:
http://dl.bintray.com/spark-packages/maven/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.jar
::::::::::::::::::::::::::::::::::::::::::::::
:: UNRESOLVED DEPENDENCIES ::
::::::::::::::::::::::::::::::::::::::::::::::
:: graphframes#graphframes;0.8.1-spark2.4-s_2.11: not found
::::::::::::::::::::::::::::::::::::::::::::::
:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILSException in thread
"main" java.lang.RuntimeException: [unresolved dependency:
graphframes#graphframes;0.8.1-spark2.4-s_2.11: not found] at
org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1306)
at
org.apache.spark.deploy.DependencyUtils$.resolveMavenDependencies(DependencyUtils.scala:54)
at
org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:315)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:143) at
org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933) at
org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Thank you!
RegardsW. Deng
