Re: Can Spark Dataframes preserve order when joining?

Hi, Most of join strategies do not preserve the orderings of input dfs (sort-merge joins only hold the ordering of a left input df). So, as said earlier, you need to explicitly sort them if you want ordered outputs. // maropu On Wed, Jun 29, 2016 at 3:38 PM, Mich Talebzadeh

Re: Can Spark Dataframes preserve order when joining?

Hi, Well I would not assume anything myself. If you want to order it do it explicitly. Let us take a simple case by creating three DFs based on existing tables val s = HiveContext.table("sales").select("AMOUNT_SOLD","TIME_ID","CHANNEL_ID") val c =

Can Spark Dataframes preserve order when joining?

If it’s not too much trouble, could I get some pointers/help on this? (see link) http://stackoverflow.com/questions/38085801/can-dataframe-joins-in-spark-preserve-order -also, as a side question, do