I have built many projects using IntellJ, maven using prior version of Spark. If anyone has a successful project with Kafka, Spark 2.0.1, Cassandra, Please share the pom.xml file. Thanks -S
On Thu, Nov 3, 2016 at 10:03 PM, Hyukjin Kwon <gurwls...@gmail.com> wrote: > Hi Shyla, > > there is the documentation for setting up IDE - https://cwiki.apache.org/ > confluence/display/SPARK/Useful+Developer+Tools# > UsefulDeveloperTools-IDESetup > > I hope this is helpful. > > > 2016-11-04 9:10 GMT+09:00 shyla deshpande <deshpandesh...@gmail.com>: > >> Hello Everyone, >> >> I just installed Spark 2.0.1, spark shell works fine. >> >> Was able to run some simple programs from the Spark Shell, but find it >> hard to make the same program work when using IntelliJ. >> I am getting the following error. >> >> Exception in thread "main" java.lang.NoSuchMethodError: >> scala.Predef$.$scope()Lscala/xml/TopScope$; >> at org.apache.spark.ui.jobs.StagePage.<init>(StagePage.scala:44) >> at org.apache.spark.ui.jobs.StagesTab.<init>(StagesTab.scala:34) >> at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:62) >> at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:215) >> at org.apache.spark.ui.SparkUI$.createLiveUI(SparkUI.scala:157) >> at org.apache.spark.SparkContext.<init>(SparkContext.scala:440) >> at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2275) >> at org.apache.spark.sql.SparkSession$Builder$$anonfun$8.apply( >> SparkSession.scala:831) >> at org.apache.spark.sql.SparkSession$Builder$$anonfun$8.apply( >> SparkSession.scala:823) >> at scala.Option.getOrElse(Option.scala:121) >> at org.apache.spark.sql.SparkSession$Builder.getOrCreate( >> SparkSession.scala:823) >> at SparkSessionTest.DataSetWordCount$.main(DataSetWordCount.scala:15) >> at SparkSessionTest.DataSetWordCount.main(DataSetWordCount.scala) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce >> ssorImpl.java:62) >> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe >> thodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:498) >> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147) >> >> Thanks >> -Shyla >> >> >> >
