Re: inter spark application communication

Hi, Putting this back in User email list… Ok, So its not inter-job communication, but taking the output of Job 1 and using it as input to job 2. (Chaining of jobs.) So you don’t need anything fancy… > On Apr 18, 2016, at 1:47 PM, Soumitra Johri > wrote:

Re: inter spark application communication

have you thought about Akka? What are you trying to send? Why do you want them to talk to one another? > On Apr 18, 2016, at 12:04 PM, Soumitra Johri > wrote: > > Hi, > > I have two applications : App1 and App2. > On a single cluster I have to spawn 5

inter spark application communication

Hi, I have two applications : App1 and App2. On a single cluster I have to spawn 5 instances os App1 and 1 instance of App2. What would be the best way to send data from the 5 App1 instances to the single App2 instance ? Right now I am using Kafka to send data from one spark application to the