Is it sufficient to set "spark.hadoop.validateOutputSpecs" to false?
http://spark.apache.org/docs/latest/configuration.html - Patrick On Wed, Dec 24, 2014 at 10:52 PM, Shao, Saisai <saisai.s...@intel.com> wrote: > Hi, > > > > We have such requirements to save RDD output to HDFS with saveAsTextFile > like API, but need to overwrite the data if existed. I'm not sure if current > Spark support such kind of operations, or I need to check this manually? > > > > There's a thread in mailing list discussed about this > (http://apache-spark-user-list.1001560.n3.nabble.com/How-can-I-make-Spark-1-0-saveAsTextFile-to-overwrite-existing-file-td6696.html), > I'm not sure this feature is enabled or not, or with some configurations? > > > > Appreciate your suggestions. > > > > Thanks a lot > > Jerry --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
