I would not count on order preserving nature of the operations, because it is not guranteed. I would assign some order to the sentences and sort at the end before write back
On Fri, 27 Jan 2017 at 10:59 pm, Md. Rezaul Karim < rezaul.ka...@insight-centre.org> wrote: > Some operations like map, filter, flatMap and coalesce (with > shuffle=false) usually preserve the order. However, sortBy, reduceBy, > partitionBy, join etc. do not. > > Regards, > _________________________________ > *Md. Rezaul Karim*, BSc, MSc > PhD Researcher, INSIGHT Centre for Data Analytics > National University of Ireland, Galway > IDA Business Park, Dangan, Galway, Ireland > Web: http://www.reza-analytics.eu/index.html > <http://139.59.184.114/index.html> > > On 27 January 2017 at 09:44, Soheila S. <soheila...@gmail.com> wrote: > > Hi All, > I read a test file using sparkContext.textfile(filename) and assign it to > an RDD and process the RDD (replace some words) and finally write it to > a text file using rdd.saveAsTextFile(output). > Is there any way to be sure the order of the sentences will not be > changed? I need to have the same text with some corrected words. > > thanks! > > Soheila > > > -- Best Regards, Ayan Guha
