Hi, Putting this back in User email list…
Ok, So its not inter-job communication, but taking the output of Job 1 and using it as input to job 2. (Chaining of jobs.) So you don’t need anything fancy… > On Apr 18, 2016, at 1:47 PM, Soumitra Johri <soumitra.siddha...@gmail.com> > wrote: > > Yes , I want to chain spark applications. > On Mon, Apr 18, 2016 at 4:46 PM Michael Segel <msegel_had...@hotmail.com > <mailto:msegel_had...@hotmail.com>> wrote: > Yes, but I’m confused. Are you chaining your spark jobs? So you run job one > and its output is the input to job 2? > >> On Apr 18, 2016, at 1:21 PM, Soumitra Johri <soumitra.siddha...@gmail.com >> <mailto:soumitra.siddha...@gmail.com>> wrote: >> >> By Akka you mean the Akka Actors ? >> >> I am sending basically sending counts which are further aggregated on the >> second application. >> >> >> Warm Regards >> Soumitra >> >> On Mon, Apr 18, 2016 at 3:54 PM, Michael Segel <msegel_had...@hotmail.com >> <mailto:msegel_had...@hotmail.com>> wrote: >> have you thought about Akka? >> >> What are you trying to send? Why do you want them to talk to one another? >> >> > On Apr 18, 2016, at 12:04 PM, Soumitra Johri <soumitra.siddha...@gmail.com >> > <mailto:soumitra.siddha...@gmail.com>> wrote: >> > >> > Hi, >> > >> > I have two applications : App1 and App2. >> > On a single cluster I have to spawn 5 instances os App1 and 1 instance of >> > App2. >> > >> > What would be the best way to send data from the 5 App1 instances to the >> > single App2 instance ? >> > >> > Right now I am using Kafka to send data from one spark application to the >> > spark application but the setup doesn't seem right and I hope there is a >> > better way to do this. >> > >> > Warm Regards >> > Soumitra >> >> >
