I added `libraryDependencies += "org.apache.spark" % "spark-sql_2.11" %
"1.3.1"` to `build.sbt` but the error remains. Do I need to import modules
other than `import org.apache.spark.sql.{ Row, SQLContext }`?On Tue, May 12, 2015 at 5:56 PM Olivier Girardot <ssab...@gmail.com> wrote: > toDF is part of spark SQL so you need Spark SQL dependency + import > sqlContext.implicits._ to get the toDF method. > > Regards, > > Olivier. > > Le mar. 12 mai 2015 à 11:36, SLiZn Liu <sliznmail...@gmail.com> a écrit : > >> Hi User Group, >> >> I’m trying to reproduce the example on Spark SQL Programming Guide >> <https://spark.apache.org/docs/latest/sql-programming-guide.html#inferring-the-schema-using-reflection>, >> and got a compile error when packaging with sbt: >> >> [error] myfile.scala:30: value toDF is not a member of >> org.apache.spark.rdd.RDD[Person] >> [error] val people = >> sc.textFile("examples/src/main/resources/people.txt").map(_.split(",")).map(p >> => Person(p(0), p(1).trim.toInt)).toDF() >> [error] >> ^ >> [error] one error found >> [error] (compile:compileIncremental) Compilation failed >> [error] Total time: 3 s, completed May 12, 2015 4:11:53 PM >> >> I double checked my code includes import sqlContext.implicits._ after >> reading this post >> <https://mail-archives.apache.org/mod_mbox/spark-user/201503.mbox/%3c1426522113299-22083.p...@n3.nabble.com%3E> >> on spark mailing list, even tried to use toDF("col1", "col2") suggested >> by Xiangrui Meng in that post and got the same error. >> >> The Spark version is specified in build.sbt file as follows: >> >> scalaVersion := "2.11.6" >> libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "1.3.1" % >> "provided" >> libraryDependencies += "org.apache.spark" % "spark-mllib_2.11" % "1.3.1" >> >> Anyone have ideas the cause of this error? >> >> REGARDS, >> Todd Leo >> >> >
