Re: An alternative logic to collaborative filtering works fine but we are facing run time issues in executing the job

Hi Balakumar Two things. One - It seems like your cluster is running out of memory and then eventually out of disc , likely while materializing the dataframe to write (what's the volume of data created by the join?) Two - Your job is running in local mode, and is able to utilize just the master

An alternative logic to collaborative filtering works fine but we are facing run time issues in executing the job

Hi , While running the following spark code in the cluster with following configuration it is spread into 3 job Id's CLUSTER CONFIGURATION 3 NODE CLUSTER NODE 1 - 64GB 16CORES NODE 2 - 64GB 16CORES NODE 3 - 64GB 16CORES At Job Id 2 job is stuck at the stage 51 of 254 and then it starts