Hi Yeikel, I can not copy anything from the system. But I have seen explain output.
It was doing sortMergeJoin for all tables. There are 10 tables , all of them doing left outer join. Out of 10 tables, 1 table is of 50MB and second table is of 200MB. Rest are big tables. Also the data is in Avro form. I am using spark 2.2 I suspect broadcast can help , not sure because broadcast works for 10MB sized smaller tables Thanks Rajat On Wed, 17 Apr 2019, 23:53 Yeikel <em...@yeikel.com wrote: > Can you share the output of df.explain() ? > > > > -- > Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
