Hi Deep, What is your configuration and what is the size of the 2 data sets?
Thanks Arush On Mon, Feb 2, 2015 at 11:56 AM, Deep Pradhan <pradhandeep1...@gmail.com> wrote: > I did not check the console because once the job starts I cannot run > anything else and have to force shutdown the system. I commented parts of > codes and I tested. I doubt it is because of union. So, I want to change it > to something else and see if the problem persists. > > Thank you > > On Mon, Feb 2, 2015 at 11:53 AM, Jerry Lam <chiling...@gmail.com> wrote: > >> Hi Deep, >> >> How do you know the cluster is not responsive because of "Union"? >> Did you check the spark web console? >> >> Best Regards, >> >> Jerry >> >> >> On Mon, Feb 2, 2015 at 1:21 AM, Deep Pradhan <pradhandeep1...@gmail.com> >> wrote: >> >>> The cluster hangs. >>> >>> On Mon, Feb 2, 2015 at 11:25 AM, Jerry Lam <chiling...@gmail.com> wrote: >>> >>>> Hi Deep, >>>> >>>> what do you mean by stuck? >>>> >>>> Jerry >>>> >>>> >>>> On Mon, Feb 2, 2015 at 12:44 AM, Deep Pradhan < >>>> pradhandeep1...@gmail.com> wrote: >>>> >>>>> Hi, >>>>> Is there any better operation than Union. I am using union and the >>>>> cluster is getting stuck with a large data set. >>>>> >>>>> Thank you >>>>> >>>> >>>> >>> >> > -- [image: Sigmoid Analytics] <http://htmlsig.com/www.sigmoidanalytics.com> *Arush Kharbanda* || Technical Teamlead ar...@sigmoidanalytics.com || www.sigmoidanalytics.com
