Hello guys, I am launching through code (client mode) a Spark program to run in Hadoop. If I execute on the dataset methods of the likes of show() and count() or collectAsList() (that are displayed in the Spark UI) after performing heavy transformations on the columns then the mentioned methods will cause the execution to freeze on Hadoop and that independently of the dataset size (intriguing issue for small size datasets!). Any idea what could be causing this type of issue? To note that if I execute collectAsList on the dataset at the beginning of the program (before performing the transformations on the columns) then the method yields results correctly.
Thanks. Regards
