Re: Writing all values for same key to one file

2016-08-05 Thread colzer
In my opinion,"Append to a file" maybe is not good idea. By using `MultipleTextOutputFormat`, you can append all values for a given key to a directory for example: class RDDMultipleTextOutputFormat extends MultipleTextOutputFormat[Any, Any] { override def generateFileNameForKeyValue(ke

Re: Writing all values for same key to one file

2016-08-04 Thread colzer
for rdd, you can use `saveAsHadoopFile` with a Custom `MultipleOutputFormat` -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Writing-all-values-for-same-key-to-one-file-tp27455p27483.html Sent from the Apache Spark User List mailing list archive at Nabble.co