RE: Applying a limit after orderBy of big dataframe hangs spark

Hi thanks for the assistance, 1. Standalone 2. df.orderBy(field).limit(5000).write.parquet(...) From: Mich Talebzadeh [mailto:mich.talebza...@gmail.com] Sent: Friday, August 05, 2016 4:33 PM To: Ellafi, Saif A. Cc: user @spark Subject: Re: Applying a limit after orderBy of big

Re: Applying a limit after orderBy of big dataframe hangs spark

Hi, 1. What scheduling are you using standalone, yarn etc? 2. How arte you limiting the df output? HTH Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw