Hi Yeikel,

I can not copy anything from the system.
But I have seen explain output.

It was doing sortMergeJoin for all tables.
There are 10 tables , all of them doing left outer join.

Out of 10 tables, 1 table is of 50MB and second table is of 200MB. Rest are
big tables.

Also the data is in Avro form.

I am using spark 2.2

I suspect broadcast can help , not sure because broadcast works for 10MB
sized smaller tables

Thanks
Rajat

On Wed, 17 Apr 2019, 23:53 Yeikel <em...@yeikel.com wrote:

> Can you share the output of df.explain() ?
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>
>

Reply via email to