hi,
Are there setup instructions on the website for
spark-3.0.0-preview2-bin-hadoop2.7I can run same program for hdfs format
val textFile = sc.textFile("hdfs://...")
val counts = textFile.flatMap(line => line.split(" "))
.map(word => (word, 1))
.reduceByKey(_ + _)
counts.saveAsTextFile("hdfs://...")
val textFile = sc.textFile("/data/README.md")val counts = textFile.flatMap(line
=> line.split(" ")) .map(word => (word, 1))
.reduceByKey(_ + _)counts.saveAsTextFile("/data/wordcount")
textFile: org.apache.spark.rdd.RDD[String] = /data/README.md
MapPartitionsRDD[23] at textFile at <console>:28counts:
org.apache.spark.rdd.RDD[(String, Int)] = ShuffledRDD[26] at reduceByKey at
<console>:31
br
Jane
