Hi Thanks for reply, but unfortunately did not work. I am getting same error.
sshuser@ed0-svochd:~/azure-spark-docdb-test$ spark-shell --jars azure-documentdb-spark-0.0.3-SNAPSHOT.jar,azure-documentdb-1.10.0.jar SPARK_MAJOR_VERSION is set to 2, using Spark2 Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). [init] error: error while loading <root>, Error accessing /home/sshuser/azure-spark-docdb-test/azure-documentdb-spark-0.0.3-SNAPSHOT.jar Failed to initialize compiler: object java.lang.Object in compiler mirror not found. ** Note that as of 2.8 scala does not assume use of the java classpath. ** For the old behavior pass -usejavacp to scala, or if using a Settings ** object programmatically, settings.usejavacp.value = true. Failed to initialize compiler: object java.lang.Object in compiler mirror not found. ** Note that as of 2.8 scala does not assume use of the java classpath. ** For the old behavior pass -usejavacp to scala, or if using a Settings ** object programmatically, settings.usejavacp.value = true. Exception in thread "main" java.lang.NullPointerException at scala.reflect.internal.SymbolTable.exitingPhase(SymbolTable.scala:256) at scala.tools.nsc.interpreter.IMain$Request.x$20$lzycompute(IMain.scala:896) at scala.tools.nsc.interpreter.IMain$Request.x$20(IMain.scala:895) at scala.tools.nsc.interpreter.IMain$Request.headerPreamble$lzycompute(IMain.scala:895) at scala.tools.nsc.interpreter.IMain$Request.headerPreamble(IMain.scala:895) at scala.tools.nsc.interpreter.IMain$Request$Wrapper.preamble(IMain.scala:918) at scala.tools.nsc.interpreter.IMain$CodeAssembler$$anonfun$apply$23.apply(IMain.scala:1337) at scala.tools.nsc.interpreter.IMain$CodeAssembler$$anonfun$apply$23.apply(IMain.scala:1336) at scala.tools.nsc.util.package$.stringFromWriter(package.scala:64) at scala.tools.nsc.interpreter.IMain$CodeAssembler$class.apply(IMain.scala:1336) at scala.tools.nsc.interpreter.IMain$Request$Wrapper.apply(IMain.scala:908) at scala.tools.nsc.interpreter.IMain$Request.compile$lzycompute(IMain.scala:1002) at scala.tools.nsc.interpreter.IMain$Request.compile(IMain.scala:997) at scala.tools.nsc.interpreter.IMain.compile(IMain.scala:579) at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:567) at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565) at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807) at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681) at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395) at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38) at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37) at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37) at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214) at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37) at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:94) at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920) at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909) at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909) at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97) at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909) at org.apache.spark.repl.Main$.doMain(Main.scala:68) at org.apache.spark.repl.Main$.main(Main.scala:51) at org.apache.spark.repl.Main.main(Main.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:736) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:210) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) sshuser@ed0-svochd:~/azure-spark-docdb-test$ On Mon, May 8, 2017 at 11:50 PM, Denny Lee <denny.g....@gmail.com> wrote: > This appears to be an issue with the Spark to DocumentDB connector, > specifically version 0.0.1. Could you run the 0.0.3 version of the jar and > see if you're still getting the same error? i.e. > > spark-shell --master yarn --jars azure-documentdb-spark-0.0.3- > SNAPSHOT.jar,azure-documentdb-1.10.0.jar > > > On Mon, May 8, 2017 at 5:01 AM ayan guha <guha.a...@gmail.com> wrote: > >> Hi >> >> I am facing an issue while trying to use azure-document-db connector from >> Microsoft. Instructions/Github >> <https://github.com/Azure/azure-documentdb-spark/wiki/Azure-DocumentDB-Spark-Connector-User-Guide> >> . >> >> Error while trying to add jar in spark-shell: >> >> spark-shell --jars azure-documentdb-spark-0.0.1. >> jar,azure-documentdb-1.9.6.jar >> SPARK_MAJOR_VERSION is set to 2, using Spark2 >> Setting default log level to "WARN". >> To adjust logging level use sc.setLogLevel(newLevel). >> [init] error: error while loading <root>, Error accessing >> /home/sshuser/azure-spark-docdb-test/v1/azure-documentdb-spark-0.0.1.jar >> >> Failed to initialize compiler: object java.lang.Object in compiler mirror >> not found. >> ** Note that as of 2.8 scala does not assume use of the java classpath. >> ** For the old behavior pass -usejavacp to scala, or if using a Settings >> ** object programmatically, settings.usejavacp.value = true. >> >> Failed to initialize compiler: object java.lang.Object in compiler mirror >> not found. >> ** Note that as of 2.8 scala does not assume use of the java classpath. >> ** For the old behavior pass -usejavacp to scala, or if using a Settings >> ** object programmatically, settings.usejavacp.value = true. >> Exception in thread "main" java.lang.NullPointerException >> at scala.reflect.internal.SymbolTable.exitingPhase( >> SymbolTable.scala:256) >> at scala.tools.nsc.interpreter.IMain$Request.x$20$lzycompute( >> IMain.scala:896) >> at scala.tools.nsc.interpreter.IMain$Request.x$20(IMain. >> scala:895) >> at scala.tools.nsc.interpreter.IMain$Request.headerPreamble$ >> lzycompute(IMain.scala:895) >> at scala.tools.nsc.interpreter.IMain$Request.headerPreamble( >> IMain.scala:895) >> at scala.tools.nsc.interpreter.IMain$Request$Wrapper. >> preamble(IMain.scala:918) >> at scala.tools.nsc.interpreter.IMain$CodeAssembler$$anonfun$ >> apply$23.apply(IMain.scala:1337) >> at scala.tools.nsc.interpreter.IMain$CodeAssembler$$anonfun$ >> apply$23.apply(IMain.scala:1336) >> at scala.tools.nsc.util.package$.stringFromWriter(package. >> scala:64) >> at scala.tools.nsc.interpreter.IMain$CodeAssembler$class. >> apply(IMain.scala:1336) >> at scala.tools.nsc.interpreter.IMain$Request$Wrapper.apply( >> IMain.scala:908) >> at scala.tools.nsc.interpreter.IMain$Request.compile$ >> lzycompute(IMain.scala:1002) >> at scala.tools.nsc.interpreter.IMain$Request.compile(IMain. >> scala:997) >> at scala.tools.nsc.interpreter.IMain.compile(IMain.scala:579) >> at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:567) >> at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565) >> at scala.tools.nsc.interpreter.ILoop.interpretStartingWith( >> ILoop.scala:807) >> at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681) >> at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395) >> at org.apache.spark.repl.SparkILoop$$anonfun$ >> initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38) >> at org.apache.spark.repl.SparkILoop$$anonfun$ >> initializeSpark$1.apply(SparkILoop.scala:37) >> at org.apache.spark.repl.SparkILoop$$anonfun$ >> initializeSpark$1.apply(SparkILoop.scala:37) >> at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain. >> scala:214) >> at org.apache.spark.repl.SparkILoop.initializeSpark( >> SparkILoop.scala:37) >> at org.apache.spark.repl.SparkILoop.loadFiles( >> SparkILoop.scala:94) >> at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1. >> apply$mcZ$sp(ILoop.scala:920) >> at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1. >> apply(ILoop.scala:909) >> at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1. >> apply(ILoop.scala:909) >> at scala.reflect.internal.util.ScalaClassLoader$. >> savingContextLoader(ScalaClassLoader.scala:97) >> at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909) >> at org.apache.spark.repl.Main$.doMain(Main.scala:68) >> at org.apache.spark.repl.Main$.main(Main.scala:51) >> at org.apache.spark.repl.Main.main(Main.scala) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at sun.reflect.NativeMethodAccessorImpl.invoke( >> NativeMethodAccessorImpl.java:62) >> at sun.reflect.DelegatingMethodAccessorImpl.invoke( >> DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:498) >> at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$ >> deploy$SparkSubmit$$runMain(SparkSubmit.scala:736) >> at org.apache.spark.deploy.SparkSubmit$.doRunMain$1( >> SparkSubmit.scala:185) >> at org.apache.spark.deploy.SparkSubmit$.submit( >> SparkSubmit.scala:210) >> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit. >> scala:124) >> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) >> sshuser@ed0-svochd:~/azure-spark-docdb-test/v1$ >> >> I think I am missing some basic configuration here or there is classpath >> related issue. Can anyone help? >> >> Additional info: >> Environment: HDInsight 3.5, based on HDP 2.5 >> >> sshuser@ed0-svochd:~/azure-spark-docdb-test/v1$ echo $JAVA_HOME >> /usr/lib/jvm/java-8-openjdk-amd64 >> >> sshuser@ed0-svochd:~/azure-spark-docdb-test/v1$ echo $SPARK_HOME >> /usr/hdp/current/spark2-client >> >> sshuser@ed0-svochd:~/azure-spark-docdb-test/v1$ java -version >> openjdk version "1.8.0_121" >> OpenJDK Runtime Environment (build 1.8.0_121-8u121-b13-0ubuntu1. >> 16.04.2-b13) >> OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode) >> >> sshuser@ed0-svochd:~/azure-spark-docdb-test/v1$ uname -a >> Linux ed0-svochd 4.4.0-72-generic #93-Ubuntu SMP Fri Mar 31 14:07:41 UTC >> 2017 x86_64 x86_64 x86_64 GNU/Linux >> sshuser@ed0-svochd:~/azure-spark-docdb-test/v1$ >> >> -- >> Best Regards, >> Ayan Guha >> > -- Best Regards, Ayan Guha