That kind of issue SparkUI and DAG visualization always helpful.
https://databricks.com/blog/2015/06/22/understanding-your-spark-application-through-visualization.html Regards, Vaquar khan On Fri, Dec 16, 2016 at 11:10 AM, Vikas K. <vikas.re...@gmail.com> wrote: > Unsubscribe. > > On Fri, Dec 16, 2016 at 9:21 PM, KhajaAsmath Mohammed < > mdkhajaasm...@gmail.com> wrote: > >> Hi, >> >> I am facing an issue with join operation on dataframe. My job is running >> for very long time( > 2 hrs ) without any result. can someone help me on >> how to resolve. >> >> I tried re-partition with 13 but no luck. >> >> >> val results_dataframe = sqlContext.sql("select gt.*,ct.* from >> PredictTempTable pt,ClusterTempTable ct,GamificationTempTable gt where >> gt.vin=pt.vin and pt.cluster=ct.cluster") >> //val results_dataframe_partitioned=results_dataframe.coalesce(numPartitions) >> val results_dataframe_partitioned=results_dataframe.repartition(13) >> >> [image: Inline image 1] >> >> Thanks, >> Asmath >> > > -- Regards, Vaquar Khan +1 -224-436-0783 IT Architect / Lead Consultant Greater Chicago
