Hi All,
I am trying to create a hive context in a scala prog as follows in eclipse:
Note -- i have added the maven dependency for spark -core , hive , and sql.
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext
import org.apache.spark.rdd.RDD.rddToPairRDDFunctions
object DataExp {
def main(args: Array[String]) = {
val conf = new SparkConf().setAppName("DataExp").setMaster("local")
val sc = new SparkContext(conf)
* val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc)*
}
}
I get the the following *errors*: @ line of hiveContext above in the prog
1 --- Error in Scala compiler: bad symbolic reference. A signature in
HiveContext.class refers to term ui in package
org.apache.spark.sql.execution which is not available. It may be completely
missing from the current classpath, or the version on the classpath might
be incompatible with the version used when compiling HiveContext.class.
spark Unknown Scala Problem
2 --- SBT builder crashed while compiling. The error message is 'bad
symbolic reference. A signature in HiveContext.class refers to term ui in
package org.apache.spark.sql.execution which is not available. It may be
completely missing from the current classpath, or the version on the
classpath might be incompatible with the version used when compiling
HiveContext.class.'. Check Error Log for details. spark Unknown Scala
Problem
3 --- while compiling:
/Users/sunitisingh/sparktest/spark/src/main/scala/com/sparktest/spark/DataExp.scala
during phase: erasure library version: version 2.10.6
compiler version: version 2.10.6 reconstructed args: -javabootclasspath
/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/System/Library/Java/Extensions/AppleScriptEngine.jar:/System/Library/Java/Extensions/dns_sd.jar:/System/Library/Java/Extensions/j3daudio.jar:/System/Library/Java/Extensions/j3dcore.jar:/System/Library/Java/Extensions/j3dutils.jar:/System/Library/Java/Extensions/jai_codec.jar:/System/Library/Java/Extensions/jai_core.jar:/System/Library/Java/Extensions/mlibwrapper_jai.jar:/System/Library/Java/Extensions/MRJToolkit.jar:/System/Library/Java/Extensions/vecmath.jar
-classpath
/Users/sunitisingh/sparktest/spark/target/classes:/Users/sunitisingh/sparktest/spark/target/test-classes:/Users/sunitisingh/.m2/repository/org/apache/spark/spark-core_2.10/1.6.0/spark-core_2.10-1.6.0.jar:/Users/sunitisingh/.m2/repository/org/apache/avro/avro-mapred/1.7.7/avro-mapred-1.7.7-hadoop2.jar:/Users/sunitisingh/.m2/repository/org/apache/avro/avro-ipc/1.7.7/avro-ipc-1.7.7.jar:/Users/sunitisingh/.m2/repository/org/apache/avro/avro-ipc/1.7.7/avro-ipc-1.7.7-tests.jar:/Users/sunitisingh/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar:/Users/sunitisingh/.m2/repository/com/twitter/chill_2.10/0.5.0/chill_2.10-0.5.0.jar:/Users/sunitisingh/.m2/repository/com/esotericsoftware/kryo/kryo/2.21/kryo-2.21.jar:/Users/sunitisingh/.m2/repository/com/esotericsoftware/reflectasm/reflectasm/1.07/reflectasm-1.07-shaded.jar:/Users/sunitisingh/.m2/repository/com/esotericsoftware/minlog/minlog/1.2/minlog-1.2.jar:/Users/sunitisingh/.m2/repository/org/objenesis/objenesis/1.2/objenesis-1.2.jar:/Users/sunitisingh/.m2/repository/com/twitter/chill-java/0.5.0/chill-java-0.5.0.jar:/Users/sunitisingh/.m2/repository/org/apache/xbean/xbean-asm5-shaded/4.4/xbean-asm5-shaded-4.4.jar:/Users/sunitisingh/.m2/repository/org/apache/hadoop/hadoop-client/2.2.0/hadoop-client-2.2.0.jar:/Users/sunitisingh/.m2/repository/org/apache/hadoop/hadoop-common/2.2.0/hadoop-common-2.2.0.jar:/Users/sunitisingh/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar:/Users/sunitisingh/.m2/repository/xmlenc/xmlenc/0.52/xmlenc-0.52.jar:/Users/sunitisingh/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/Users/sunitisingh/.m2/repository/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar:/Users/sunitisingh/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/Users/sunitisingh/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar:/Users/sunitisingh/.m2/repository/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar:/Users/sunitisingh/.m2/repository/org/apache/hadoop/hadoop-auth/2.2.0/hadoop-auth-2.2.0.jar:/Users/sunitisingh/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.2.0/hadoop-hdfs-2.2.0.jar:/Users/sunitisingh/.m2/repository/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar:/Users/sunitisingh/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-app/2.2.0/hadoop-mapreduce-client-app-2.2.0.jar:/Users/sunitisingh/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-common/2.2.0/hadoop-mapreduce-client-common-2.2.0.jar:/Users/sunitisingh/.m2/repository/org/apache/hadoop/hadoop-yarn-client/2.2.0/hadoop-yarn-client-2.2.0.jar:/Users/sunitisingh/.m2/repository/com/google/inject/guice/3.0/guice-3.0.jar:/Users/sunitisingh/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar:/Users/sunitisingh/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar:/Users/sunitisingh/.m2/repository/com/sun/jersey/jersey-test-framework/jersey-test-framework-grizzly2/1.9/jersey-test-framework-grizzly2-1.9.jar:/Users/sunitisingh/.m2/repository/com/sun/jersey/jersey-test-framework/jersey-test-framework-core/1.9/jersey-test-framework-core-1.9.jar:/Users/sunitisingh/.m2/repository/javax/servlet/javax.servlet-api/3.0.1/javax.servlet-api-3.0.1.jar:/Users/sunitisingh/.m2/repository/com/sun/jersey/jersey-client/1.9/jersey-client-1.9.jar:/Users/sunitisingh/.m2/repository/com/sun/jersey/jersey-grizzly2/1.9/jersey-grizzly2-1.9.jar:/Users/sunitisingh/.m2/repository/org/glassfish/grizzly/grizzly-http/2.1.2/grizzly-http-2.1.2.jar:/Users/sunitisingh/.m2/repository/org/glassfish/grizzly/grizzly-framework/2.1.2/grizzly-framework-2.1.2.jar:/Users/sunitisingh/.m2/repository/org/glassfish/gmbal/gmbal-api-only/3.0.0-b023/gmbal-api-only-3.0.0-b023.jar:/Users/sunitisingh/.m2/repository/org/glassfish/external/management-api/3.0.0-b012/management-api-3.0.0-b012.jar:/Users/sunitisingh/.m2/repository/org/glassfish/grizzly/grizzly-http-server/2.1.2/grizzly-http-server-2.1.2.jar:/Users/sunitisingh/.m2/repository/org/glassfish/grizzly/grizzly-rcm/2.1.2/grizzly-rcm-2.1.2.jar:/Users/sunitisingh/.m2/repository/org/glassfish/grizzly/grizzly-http-servlet/2.1.2/grizzly-http-servlet-2.1.2.jar:/Users/sunitisingh/.m2/repository/org/glassfish/javax.servlet/3.1/javax.servlet-3.1.jar:/Users/sunitisingh/.m2/repository/com/sun/jersey/jersey-json/1.9/jersey-json-1.9.jar:/Users/sunitisingh/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/Users/sunitisingh/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar:/Users/sunitisingh/.m2/repository/javax/xml/bind/jaxb-api/2.2.2/jaxb-api-2.2.2.jar:/Users/sunitisingh/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar:/Users/sunitisingh/.m2/repository/org/codehaus/jackson/jackson-jaxrs/1.8.3/jackson-jaxrs-1.8.3.jar:/Users/sunitisingh/.m2/repository/org/codehaus/jackson/jackson-xc/1.8.3/jackson-xc-1.8.3.jar:/Users/sunitisingh/.m2/repository/com/sun/jersey/contribs/jersey-guice/1.9/jersey-guice-1.9.jar:/Users/sunitisingh/.m2/repository/org/apache/hadoop/hadoop-yarn-server-common/2.2.0/hadoop-yarn-server-common-2.2.0.jar:/Users/sunitisingh/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-shuffle/2.2.0/hadoop-mapreduce-client-shuffle-2.2.0.jar:/Users/sunitisingh/.m2/repository/org/apache/hadoop/hadoop-yarn-api/2.2.0/hadoop-yarn-api-2.2.0.jar:/Users/sunitisingh/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/2.2.0/hadoop-mapreduce-client-core-2.2.0.jar:/Users/sunitisingh/.m2/repository/org/apache/hadoop/hadoop-yarn-common/2.2.0/hadoop-yarn-common-2.2.0.jar:/Users/sunitisingh/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.2.0/hadoop-mapreduce-client-jobclient-2.2.0.jar:/Users/sunitisingh/.m2/repository/org/apache/hadoop/hadoop-annotations/2.2.0/hadoop-annotations-2.2.0.jar:/Users/sunitisingh/.m2/repository/org/apache/spark/spark-launcher_2.10/1.6.0/spark-launcher_2.10-1.6.0.jar:/Users/sunitisingh/.m2/repository/org/apache/spark/spark-network-common_2.10/1.6.0/spark-network-common_2.10-1.6.0.jar:/Users/sunitisingh/.m2/repository/org/apache/spark/spark-network-shuffle_2.10/1.6.0/spark-network-shuffle_2.10-1.6.0.jar:/Users/sunitisingh/.m2/repository/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar:/Users/sunitisingh/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.4.4/jackson-annotations-2.4.4.jar:/Users/sunitisingh/.m2/repository/org/apache/spark/spark-unsafe_2.10/1.6.0/spark-unsafe_2.10-1.6.0.jar:/Users/sunitisingh/.m2/repository/net/java/dev/jets3t/jets3t/0.7.1/jets3t-0.7.1.jar:/Users/sunitisingh/.m2/repository/org/apache/curator/curator-recipes/2.4.0/curator-recipes-2.4.0.jar:/Users/sunitisingh/.m2/repository/org/apache/curator/curator-framework/2.4.0/curator-framework-2.4.0.jar:/Users/sunitisingh/.m2/repository/org/apache/curator/curator-client/2.4.0/curator-client-2.4.0.jar:/Users/sunitisingh/.m2/repository/org/apache/zookeeper/zookeeper/3.4.5/zookeeper-3.4.5.jar:/Users/sunitisingh/.m2/repository/com/google/guava/guava/14.0.1/guava-14.0.1.jar:/Users/sunitisingh/.m2/repository/org/eclipse/jetty/orbit/javax.servlet/3.0.0.v201112011016/javax.servlet-3.0.0.v201112011016.jar:/Users/sunitisingh/.m2/repository/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar:/Users/sunitisingh/.m2/repository/org/apache/commons/commons-math3/3.4.1/commons-math3-3.4.1.jar:/Users/sunitisingh/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/Users/sunitisingh/.m2/repository/org/slf4j/slf4j-api/1.7.10/slf4j-api-1.7.10.jar:/Users/sunitisingh/.m2/repository/org/slf4j/jul-to-slf4j/1.7.10/jul-to-slf4j-1.7.10.jar:/Users/sunitisingh/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.10/jcl-over-slf4j-1.7.10.jar:/Users/sunitisingh/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/Users/sunitisingh/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar:/Users/sunitisingh/.m2/repository/com/ning/compress-lzf/1.0.3/compress-lzf-1.0.3.jar:/Users/sunitisingh/.m2/repository/org/xerial/snappy/snappy-java/1.1.2/snappy-java-1.1.2.jar:/Users/sunitisingh/.m2/repository/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.jar:/Users/sunitisingh/.m2/repository/org/roaringbitmap/RoaringBitmap/0.5.11/RoaringBitmap-0.5.11.jar:/Users/sunitisingh/.m2/repository/commons-net/commons-net/2.2/commons-net-2.2.jar:/Users/sunitisingh/.m2/repository/com/typesafe/akka/akka-remote_2.10/2.3.11/akka-remote_2.10-2.3.11.jar:/Users/sunitisingh/.m2/repository/com/typesafe/akka/akka-actor_2.10/2.3.11/akka-actor_2.10-2.3.11.jar:/Users/sunitisingh/.m2/repository/com/typesafe/config/1.2.1/config-1.2.1.jar:/Users/sunitisingh/.m2/repository/io/netty/netty/3.8.0.Final/netty-3.8.0.Final.jar:/Users/sunitisingh/.m2/repository/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar:/Users/sunitisingh/.m2/repository/org/uncommons/maths/uncommons-maths/1.2.2a/uncommons-maths-1.2.2a.jar:/Users/sunitisingh/.m2/repository/com/typesafe/akka/akka-slf4j_2.10/2.3.11/akka-slf4j_2.10-2.3.11.jar:/Users/sunitisingh/.m2/repository/org/json4s/json4s-jackson_2.10/3.2.10/json4s-jackson_2.10-3.2.10.jar:/Users/sunitisingh/.m2/repository/org/json4s/json4s-core_2.10/3.2.10/json4s-core_2.10-3.2.10.jar:/Users/sunitisingh/.m2/repository/org/json4s/json4s-ast_2.10/3.2.10/json4s-ast_2.10-3.2.10.jar:/Users/sunitisingh/.m2/repository/org/scala-lang/scalap/2.10.0/scalap-2.10.0.jar:/Users/sunitisingh/.m2/repository/com/sun/jersey/jersey-server/1.9/jersey-server-1.9.jar:/Users/sunitisingh/.m2/repository/asm/asm/3.1/asm-3.1.jar:/Users/sunitisingh/.m2/repository/com/sun/jersey/jersey-core/1.9/jersey-core-1.9.jar:/Users/sunitisingh/.m2/repository/org/apache/mesos/mesos/0.21.1/mesos-0.21.1-shaded-protobuf.jar:/Users/sunitisingh/.m2/repository/io/netty/netty-all/4.0.29.Final/netty-all-4.0.29.Final.jar:/Users/sunitisingh/.m2/repository/com/clearspring/analytics/stream/2.7.0/stream-2.7.0.jar:/Users/sunitisingh/.m2/repository/io/dropwizard/metrics/metrics-core/3.1.2/metrics-core-3.1.2.jar:/Users/sunitisingh/.m2/repository/io/dropwizard/metrics/metrics-jvm/3.1.2/metrics-jvm-3.1.2.jar:/Users/sunitisingh/.m2/repository/io/dropwizard/metrics/metrics-json/3.1.2/metrics-json-3.1.2.jar:/Users/sunitisingh/.m2/repository/io/dropwizard/metrics/metrics-graphite/3.1.2/metrics-graphite-3.1.2.jar:/Users/sunitisingh/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.4.4/jackson-databind-2.4.4.jar:/Users/sunitisingh/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.4.4/jackson-core-2.4.4.jar:/Users/sunitisingh/.m2/repository/com/fasterxml/jackson/module/jackson-module-scala_2.10/2.4.4/jackson-module-scala_2.10-2.4.4.jar:/Users/sunitisingh/.m2/repository/org/scala-lang/scala-reflect/2.10.4/scala-reflect-2.10.4.jar:/Users/sunitisingh/.m2/repository/com/thoughtworks/paranamer/paranamer/2.6/paranamer-2.6.jar:/Users/sunitisingh/.m2/repository/org/apache/ivy/ivy/2.4.0/ivy-2.4.0.jar:/Users/sunitisingh/.m2/repository/oro/oro/2.0.8/oro-2.0.8.jar:/Users/sunitisingh/.m2/repository/org/tachyonproject/tachyon-client/0.8.2/tachyon-client-0.8.2.jar:/Users/sunitisingh/.m2/repository/commons-lang/commons-lang/2.4/commons-lang-2.4.jar:/Users/sunitisingh/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar:/Users/sunitisingh/.m2/repository/org/tachyonproject/tachyon-underfs-hdfs/0.8.2/tachyon-underfs-hdfs-0.8.2.jar:/Users/sunitisingh/.m2/repository/org/tachyonproject/tachyon-underfs-s3/0.8.2/tachyon-underfs-s3-0.8.2.jar:/Users/sunitisingh/.m2/repository/org/tachyonproject/tachyon-underfs-local/0.8.2/tachyon-underfs-local-0.8.2.jar:/Users/sunitisingh/.m2/repository/net/razorvine/pyrolite/4.9/pyrolite-4.9.jar:/Users/sunitisingh/.m2/repository/net/sf/py4j/py4j/0.9/py4j-0.9.jar:/Users/sunitisingh/.m2/repository/org/spark-project/spark/unused/1.0.0/unused-1.0.0.jar:/Users/sunitisingh/.m2/repository/org/apache/spark/spark-sql_2.10/1.2.0/spark-sql_2.10-1.2.0.jar:/Users/sunitisingh/.m2/repository/org/apache/spark/spark-catalyst_2.10/1.2.0/spark-catalyst_2.10-1.2.0.jar:/Users/sunitisingh/.m2/repository/org/scala-lang/scala-compiler/2.10.4/scala-compiler-2.10.4.jar:/Users/sunitisingh/.m2/repository/org/scalamacros/quasiquotes_2.10/2.0.1/quasiquotes_2.10-2.0.1.jar:/Users/sunitisingh/.m2/repository/com/twitter/parquet-column/1.6.0rc3/parquet-column-1.6.0rc3.jar:/Users/sunitisingh/.m2/repository/com/twitter/parquet-common/1.6.0rc3/parquet-common-1.6.0rc3.jar:/Users/sunitisingh/.m2/repository/com/twitter/parquet-encoding/1.6.0rc3/parquet-encoding-1.6.0rc3.jar:/Users/sunitisingh/.m2/repository/com/twitter/parquet-generator/1.6.0rc3/parquet-generator-1.6.0rc3.jar:/Users/sunitisingh/.m2/repository/com/twitter/parquet-hadoop/1.6.0rc3/parquet-hadoop-1.6.0rc3.jar:/Users/sunitisingh/.m2/repository/com/twitter/parquet-format/2.2.0-rc1/parquet-format-2.2.0-rc1.jar:/Users/sunitisingh/.m2/repository/com/twitter/parquet-jackson/1.6.0rc3/parquet-jackson-1.6.0rc3.jar:/Users/sunitisingh/.m2/repository/org/apache/spark/spark-hive_2.10/1.6.0/spark-hive_2.10-1.6.0.jar:/Users/sunitisingh/.m2/repository/com/twitter/parquet-hadoop-bundle/1.6.0/parquet-hadoop-bundle-1.6.0.jar:/Users/sunitisingh/.m2/repository/org/spark-project/hive/hive-exec/1.2.1.spark/hive-exec-1.2.1.spark.jar:/Users/sunitisingh/.m2/repository/javolution/javolution/5.5.1/javolution-5.5.1.jar:/Users/sunitisingh/.m2/repository/log4j/apache-log4j-extras/1.2.17/apache-log4j-extras-1.2.17.jar:/Users/sunitisingh/.m2/repository/org/antlr/antlr-runtime/3.4/antlr-runtime-3.4.jar:/Users/sunitisingh/.m2/repository/org/antlr/stringtemplate/3.2.1/stringtemplate-3.2.1.jar:/Users/sunitisingh/.m2/repository/antlr/antlr/2.7.7/antlr-2.7.7.jar:/Users/sunitisingh/.m2/repository/org/antlr/ST4/4.0.4/ST4-4.0.4.jar:/Users/sunitisingh/.m2/repository/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar:/Users/sunitisingh/.m2/repository/org/tukaani/xz/1.0/xz-1.0.jar:/Users/sunitisingh/.m2/repository/org/codehaus/groovy/groovy-all/2.1.6/groovy-all-2.1.6.jar:/Users/sunitisingh/.m2/repository/com/googlecode/javaewah/JavaEWAH/0.3.2/JavaEWAH-0.3.2.jar:/Users/sunitisingh/.m2/repository/org/iq80/snappy/snappy/0.2/snappy-0.2.jar:/Users/sunitisingh/.m2/repository/org/json/json/20090211/json-20090211.jar:/Users/sunitisingh/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/Users/sunitisingh/.m2/repository/net/sf/opencsv/opencsv/2.3/opencsv-2.3.jar:/Users/sunitisingh/.m2/repository/jline/jline/2.12/jline-2.12.jar:/Users/sunitisingh/.m2/repository/org/spark-project/hive/hive-metastore/1.2.1.spark/hive-metastore-1.2.1.spark.jar:/Users/sunitisingh/.m2/repository/com/jolbox/bonecp/0.8.0.RELEASE/bonecp-0.8.0.RELEASE.jar:/Users/sunitisingh/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/Users/sunitisingh/.m2/repository/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.jar:/Users/sunitisingh/.m2/repository/org/apache/derby/derby/
10.10.2.0/derby-10.10.2.0.jar:/Users/sunitisingh/.m2/repository/org/datanucleus/datanucleus-api-jdo/3.2.6/datanucleus-api-jdo-3.2.6.jar:/Users/sunitisingh/.m2/repository/org/datanucleus/datanucleus-rdbms/3.2.9/datanucleus-rdbms-3.2.9.jar:/Users/sunitisingh/.m2/repository/commons-pool/commons-pool/1.5.4/commons-pool-1.5.4.jar:/Users/sunitisingh/.m2/repository/commons-dbcp/commons-dbcp/1.4/commons-dbcp-1.4.jar:/Users/sunitisingh/.m2/repository/javax/jdo/jdo-api/3.0.1/jdo-api-3.0.1.jar:/Users/sunitisingh/.m2/repository/javax/transaction/jta/1.1/jta-1.1.jar:/Users/sunitisingh/.m2/repository/org/apache/avro/avro/1.7.7/avro-1.7.7.jar:/Users/sunitisingh/.m2/repository/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar:/Users/sunitisingh/.m2/repository/org/apache/calcite/calcite-avatica/1.2.0-incubating/calcite-avatica-1.2.0-incubating.jar:/Users/sunitisingh/.m2/repository/org/apache/calcite/calcite-core/1.2.0-incubating/calcite-core-1.2.0-incubating.jar:/Users/sunitisingh/.m2/repository/org/apache/calcite/calcite-linq4j/1.2.0-incubating/calcite-linq4j-1.2.0-incubating.jar:/Users/sunitisingh/.m2/repository/net/hydromatic/eigenbase-properties/1.1.5/eigenbase-properties-1.1.5.jar:/Users/sunitisingh/.m2/repository/org/codehaus/janino/commons-compiler/2.7.6/commons-compiler-2.7.6.jar:/Users/sunitisingh/.m2/repository/org/apache/httpcomponents/httpclient/4.3.2/httpclient-4.3.2.jar:/Users/sunitisingh/.m2/repository/org/apache/httpcomponents/httpcore/4.3.1/httpcore-4.3.1.jar:/Users/sunitisingh/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar:/Users/sunitisingh/.m2/repository/commons-codec/commons-codec/1.10/commons-codec-1.10.jar:/Users/sunitisingh/.m2/repository/joda-time/joda-time/2.9/joda-time-2.9.jar:/Users/sunitisingh/.m2/repository/org/jodd/jodd-core/3.5.2/jodd-core-3.5.2.jar:/Users/sunitisingh/.m2/repository/org/datanucleus/datanucleus-core/3.2.10/datanucleus-core-3.2.10.jar:/Users/sunitisingh/.m2/repository/org/apache/thrift/libthrift/0.9.2/libthrift-0.9.2.jar:/Users/sunitisingh/.m2/repository/org/apache/thrift/libfb303/0.9.2/libfb303-0.9.2.jar
-bootclasspath
/Users/sunitisingh/.m2/repository/org/scala-lang/scala-library/2.10.5/scala-library-2.10.5.jar
-javaextdirs last tree to typer: TypeTree(class BoxedUnit)
symbol: class BoxedUnit in package runtime (flags: ) symbol
definition: class BoxedUnit extends tpe:
runtime.BoxedUnit symbol owners: class BoxedUnit -> package runtime
context owners: value hiveContext -> method main -> object DataExp ->
package spark == Enclosing template or block == DefDef( // def main(args:
Array[String]): Unit in object DataExp <method> "main" [] // 1
parameter list ValDef( // args: Array[String] <param> "args"
<tpt> // tree.tpe=Array[String] <empty> ) <tpt> //
tree.tpe=runtime.BoxedUnit Block( // tree.tpe=Unit // 3 statements
ValDef( // val conf: org.apache.spark.SparkConf <triedcooking>
"conf" <tpt> // tree.tpe=org.apache.spark.SparkConf Apply( //
def setMaster(master: String): org.apache.spark.SparkConf in class
SparkConf new
org.apache.spark.SparkConf().setAppName("DataExp")."setMaster" // def
setMaster(master: String): org.apache.spark.SparkConf in class SparkConf
"local" ) ) ValDef( // val sc:
org.apache.spark.SparkContext <triedcooking> "sc" <tpt>
// tree.tpe=org.apache.spark.SparkContext Apply( // def
<init>(config: org.apache.spark.SparkConf): org.apache.spark.SparkContext
in class SparkContext new org.apache.spark.SparkContext."<init>" //
def <init>(config: org.apache.spark.SparkConf):
org.apache.spark.SparkContext in class SparkContext "conf" // val
conf: org.apache.spark.SparkConf ) ) ValDef( // val
hiveContext: org.apache.spark.sql.hive.HiveContext 0
"hiveContext" <tpt> // tree.t spark Unknown Scala Problem
*Maven dependency* --
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.6.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.10</artifactId>
<version>1.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_2.10</artifactId>
<version>1.6.0</version>
</dependency>
I am not able to understand what is missing.
Regards,
Suniti
