I have this line which works in the spark interactive console but it fails in Intellij
Using Spark 2.1.1 in both cases:
Exception in thread "main" java.lang.RuntimeException: Multiple sources found
for csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat,
com.databricks.spark.csv.DefaultSource15), please specify the fully qualified
class name.
source:
val ie8df =
sqlContext.read.schema(SomeSchema).option("mode","dropmalformed").option("sep",
"\t").format("csv").load(somepath)
