Does the filter under consideration operate on sorted column(s) ? Cheers
> On Jul 7, 2016, at 2:25 AM, tan shai <[email protected]> wrote: > > Hi, > > I have a sorted dataframe, I need to optimize the filter operations. > How does Spark performs filter operations on sorted dataframe? > > It is scanning all the data? > > Many thanks. --------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
