[jira] [Comment Edited] (SPARK-18778) Fix the Scala classpath in the spark-shell
[
https://issues.apache.org/jira/browse/SPARK-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368442#comment-16368442
]
Shaun Jackman edited comment on SPARK-18778 at 2/18/18 5:25 AM:
I have this issue as well. spark-shell fails with Java 9. Is there a work
around?
{{❯❯❯ spark-shell 'sc.parallelize(1 to 1000).count()'}}
{{Failed to initialize compiler: object java.lang.Object in compiler mirror not
found.}}{{** Note that as of 2.8 scala does not assume use of the java
classpath.}}{{** For the old behavior pass -usejavacp to scala, or if using a
Settings}}{{** object programmatically, settings.usejavacp.value = true.}}
was (Author: sjackman):
I have this issue as well. spark-shell fails with Java 9. Is there a work
around?
{{❯❯❯ spark-shell 'sc.parallelize(1 to 1000).count()'}}{{Failed to initialize
compiler: object java.lang.Object in compiler mirror not found.}}{{** Note that
as of 2.8 scala does not assume use of the java classpath.}}{{** For the old
behavior pass -usejavacp to scala, or if using a Settings}}{{** object
programmatically, settings.usejavacp.value = true.}}
> Fix the Scala classpath in the spark-shell
> --
>
> Key: SPARK-18778
> URL: https://issues.apache.org/jira/browse/SPARK-18778
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
>Affects Versions: 1.6.1, 2.0.2
>Reporter: DjvuLee
>Priority: Major
>
> Failed to initialize compiler: object scala.runtime in compiler mirror not
> found.
> ** Note that as of 2.8 scala does not assume use of the java classpath.
> ** For the old behavior pass -usejavacp to scala, or if using a Settings
> ** object programatically, settings.usejavacp.value = true.
> Exception in thread "main" java.lang.AssertionError: assertion failed: null
> at scala.Predef$.assert(Predef.scala:179)
> at
> org.apache.spark.repl.SparkIMain.initializeSynchronous(SparkIMain.scala:247)
> at
> org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:990)
> at
> org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
> at
> org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
> at
> scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
> at
> org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
> at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)
> at org.apache.spark.repl.Main$.main(Main.scala:31)
> at org.apache.spark.repl.Main.main(Main.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
> at
> org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
> at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-18778) Fix the Scala classpath in the spark-shell
[ https://issues.apache.org/jira/browse/SPARK-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731403#comment-15731403 ] DjvuLee edited comment on SPARK-18778 at 12/8/16 7:46 AM: -- When I just run the ./bin/spark-shell under our environment, the spark-shell occurs the above error. I can fix it by pass the -usejavacp directly to the spark-shell, like running ./bin/spark-shell -usejavacp My environment is jdk1.8.0_91, and we do not install the scala. the OS is Debian 4.6.4. was (Author: djvulee): When I just run the ./bin/spark-shell under our environment, the spark-shell occurs the above error. I can fix it by pass the -usejavacp directly to the spark-shell, like running ./bin/spark-shell -usejavacp My environment is jdk1.8.0_91, and we do not install the scala. the -Dscala.usejavacp=true in bin/spark-shell seems not work. > Fix the Scala classpath in the spark-shell > -- > > Key: SPARK-18778 > URL: https://issues.apache.org/jira/browse/SPARK-18778 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 1.6.1, 2.0.2 >Reporter: DjvuLee > > Failed to initialize compiler: object scala.runtime in compiler mirror not > found. > ** Note that as of 2.8 scala does not assume use of the java classpath. > ** For the old behavior pass -usejavacp to scala, or if using a Settings > ** object programatically, settings.usejavacp.value = true. > Exception in thread "main" java.lang.AssertionError: assertion failed: null > at scala.Predef$.assert(Predef.scala:179) > at > org.apache.spark.repl.SparkIMain.initializeSynchronous(SparkIMain.scala:247) > at > org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:990) > at > org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) > at > org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) > at > scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135) > at > org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945) > at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059) > at org.apache.spark.repl.Main$.main(Main.scala:31) > at org.apache.spark.repl.Main.main(Main.scala) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) > at > org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) > at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-18778) Fix the Scala classpath in the spark-shell
[ https://issues.apache.org/jira/browse/SPARK-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731431#comment-15731431 ] DjvuLee edited comment on SPARK-18778 at 12/8/16 7:38 AM: -- I give a fix in the https://github.com/apache/spark/pull/16210. This seems a little wired, since the -Dscala.usejavacp=true is try to fix this.[https://issues.apache.org/jira/browse/SPARK-4161] was (Author: djvulee): I give a fix in the https://github.com/apache/spark/pull/16210. This seems a little wired, since the -Dscala.usejavacp=true is try to fix this. > Fix the Scala classpath in the spark-shell > -- > > Key: SPARK-18778 > URL: https://issues.apache.org/jira/browse/SPARK-18778 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 1.6.1, 2.0.2 >Reporter: DjvuLee > > Failed to initialize compiler: object scala.runtime in compiler mirror not > found. > ** Note that as of 2.8 scala does not assume use of the java classpath. > ** For the old behavior pass -usejavacp to scala, or if using a Settings > ** object programatically, settings.usejavacp.value = true. > Exception in thread "main" java.lang.AssertionError: assertion failed: null > at scala.Predef$.assert(Predef.scala:179) > at > org.apache.spark.repl.SparkIMain.initializeSynchronous(SparkIMain.scala:247) > at > org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:990) > at > org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) > at > org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) > at > scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135) > at > org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945) > at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059) > at org.apache.spark.repl.Main$.main(Main.scala:31) > at org.apache.spark.repl.Main.main(Main.scala) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) > at > org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) > at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-18778) Fix the Scala classpath in the spark-shell
[ https://issues.apache.org/jira/browse/SPARK-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731431#comment-15731431 ] DjvuLee edited comment on SPARK-18778 at 12/8/16 7:38 AM: -- I give a fix in the https://github.com/apache/spark/pull/16210. This seems a little wired, since the -Dscala.usejavacp=true is try to fix this.[https://issues.apache.org/jira/browse/SPARK-4161] was (Author: djvulee): I give a fix in the https://github.com/apache/spark/pull/16210. This seems a little wired, since the -Dscala.usejavacp=true is try to fix this.[https://issues.apache.org/jira/browse/SPARK-4161] > Fix the Scala classpath in the spark-shell > -- > > Key: SPARK-18778 > URL: https://issues.apache.org/jira/browse/SPARK-18778 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 1.6.1, 2.0.2 >Reporter: DjvuLee > > Failed to initialize compiler: object scala.runtime in compiler mirror not > found. > ** Note that as of 2.8 scala does not assume use of the java classpath. > ** For the old behavior pass -usejavacp to scala, or if using a Settings > ** object programatically, settings.usejavacp.value = true. > Exception in thread "main" java.lang.AssertionError: assertion failed: null > at scala.Predef$.assert(Predef.scala:179) > at > org.apache.spark.repl.SparkIMain.initializeSynchronous(SparkIMain.scala:247) > at > org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:990) > at > org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) > at > org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) > at > scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135) > at > org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945) > at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059) > at org.apache.spark.repl.Main$.main(Main.scala:31) > at org.apache.spark.repl.Main.main(Main.scala) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) > at > org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) > at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-18778) Fix the Scala classpath in the spark-shell
[ https://issues.apache.org/jira/browse/SPARK-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731431#comment-15731431 ] DjvuLee edited comment on SPARK-18778 at 12/8/16 7:35 AM: -- I give a fix in the https://github.com/apache/spark/pull/16210. This seems a little wired, since the -Dscala.usejavacp=true is try to fix this. was (Author: djvulee): I give a fix in the https://github.com/apache/spark/pull/16210 > Fix the Scala classpath in the spark-shell > -- > > Key: SPARK-18778 > URL: https://issues.apache.org/jira/browse/SPARK-18778 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 1.6.1, 2.0.2 >Reporter: DjvuLee > > Failed to initialize compiler: object scala.runtime in compiler mirror not > found. > ** Note that as of 2.8 scala does not assume use of the java classpath. > ** For the old behavior pass -usejavacp to scala, or if using a Settings > ** object programatically, settings.usejavacp.value = true. > Exception in thread "main" java.lang.AssertionError: assertion failed: null > at scala.Predef$.assert(Predef.scala:179) > at > org.apache.spark.repl.SparkIMain.initializeSynchronous(SparkIMain.scala:247) > at > org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:990) > at > org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) > at > org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) > at > scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135) > at > org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945) > at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059) > at org.apache.spark.repl.Main$.main(Main.scala:31) > at org.apache.spark.repl.Main.main(Main.scala) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) > at > org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) > at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-18778) Fix the Scala classpath in the spark-shell
[ https://issues.apache.org/jira/browse/SPARK-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15731403#comment-15731403 ] DjvuLee edited comment on SPARK-18778 at 12/8/16 7:25 AM: -- When I just run the ./bin/spark-shell under our environment, the spark-shell occurs the above error. I can fix it by pass the -usejavacp directly to the spark-shell, like running ./bin/spark-shell -usejavacp My environment is jdk1.8.0_91, and we do not install the scala. the -Dscala.usejavacp=true in bin/spark-shell seems not work. was (Author: djvulee): When I just run the ./bin/spark-shell under our environment, the spark-shell occurs the above error. I can fix it by pass the -usejavacp directly to the spark-shell, like running ./bin/spark-sell -usejavacp My environment is jdk1.8.0_91, and we do not install the scala. the -Dscala.usejavacp=true in bin/spark-shell seems not work. > Fix the Scala classpath in the spark-shell > -- > > Key: SPARK-18778 > URL: https://issues.apache.org/jira/browse/SPARK-18778 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 1.6.1, 2.0.2 >Reporter: DjvuLee > > Failed to initialize compiler: object scala.runtime in compiler mirror not > found. > ** Note that as of 2.8 scala does not assume use of the java classpath. > ** For the old behavior pass -usejavacp to scala, or if using a Settings > ** object programatically, settings.usejavacp.value = true. > Exception in thread "main" java.lang.AssertionError: assertion failed: null > at scala.Predef$.assert(Predef.scala:179) > at > org.apache.spark.repl.SparkIMain.initializeSynchronous(SparkIMain.scala:247) > at > org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:990) > at > org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) > at > org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) > at > scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135) > at > org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945) > at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059) > at org.apache.spark.repl.Main$.main(Main.scala:31) > at org.apache.spark.repl.Main.main(Main.scala) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) > at > org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) > at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
