increased the
> threshold (4Gb) for the same then also it did not work. and the df3 is
> somewhat greater than 2gb.
>
> Trying by removing broadcast as well.. Job is running since 1 hour. Will
> let you know.
>
>
> Thanks
> Sachit
>
> On Wed, 7 Oct 2020, 00:41 Eve Liao
How many rows does df3 have? Broadcast joins are a great way to append data
stored in relatively *small* single source of truth data files to large
DataFrames. DataFrames up to 2GB can be broadcasted so a data file with
tens or even hundreds of thousands of rows is a broadcast candidate. Your
