Hello, I try to magage NA in this dataset. I import my dataset with the com.databricks.spark.csv package
When I do this: allyears2k.na.drop() I have no result. Can you help me please ? Regards, ------------------- The dataset ------------------------- dataset: https://s3.amazonaws.com/h2o-airlines-unpacked/allyears2k.csv ------------------- The code ------------------------- // Prepare environment import sys.process._ val sqlContext = new org.apache.spark.sql.SQLContext(sc) import sqlContext.implicits._ val allyears2k = sqlContext.read.format("com.databricks.spark.csv").option("header", "true").load("/home/clark/allyears2k.csv") allyears2k.registerTempTable("allyears2k") val rv = allyears2k.na.drop()
