I did not check the console because once the job starts I cannot run anything else and have to force shutdown the system. I commented parts of codes and I tested. I doubt it is because of union. So, I want to change it to something else and see if the problem persists.
Thank you On Mon, Feb 2, 2015 at 11:53 AM, Jerry Lam <chiling...@gmail.com> wrote: > Hi Deep, > > How do you know the cluster is not responsive because of "Union"? > Did you check the spark web console? > > Best Regards, > > Jerry > > > On Mon, Feb 2, 2015 at 1:21 AM, Deep Pradhan <pradhandeep1...@gmail.com> > wrote: > >> The cluster hangs. >> >> On Mon, Feb 2, 2015 at 11:25 AM, Jerry Lam <chiling...@gmail.com> wrote: >> >>> Hi Deep, >>> >>> what do you mean by stuck? >>> >>> Jerry >>> >>> >>> On Mon, Feb 2, 2015 at 12:44 AM, Deep Pradhan <pradhandeep1...@gmail.com >>> > wrote: >>> >>>> Hi, >>>> Is there any better operation than Union. I am using union and the >>>> cluster is getting stuck with a large data set. >>>> >>>> Thank you >>>> >>> >>> >> >
