Disha: Please consider these resources: https://groups.google.com/forum/#!forum/akka-user https://groups.google.com/forum/#!forum/akka-dev
On Sun, Dec 27, 2015 at 5:15 AM, Dean Wampler <deanwamp...@gmail.com> wrote: > As Reynold said, you can still use Akka with Spark, but now it's more like > using any third-party library that isn't already a Spark dependency (at > least once the current Akka dependency is fully removed). > > Dean Wampler, Ph.D. > Author: Programming Scala, 2nd Edition > <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) > Typesafe <http://typesafe.com> > @deanwampler <http://twitter.com/deanwampler> > http://polyglotprogramming.com > > On Sun, Dec 27, 2015 at 4:06 AM, Disha Shrivastava <dishu....@gmail.com> > wrote: > >> Hi All, >> >> I need an Akka like framework to implement model parallelism in neural >> networks, an architecture similar to that given in the link >> http://alexminnaar.com/implementing-the-distbelief-deep-neural-network-training-framework-with-akka.html. >> I need to divide a big neural network ( which can't fit into the memory of >> one machine) layer by layer and do message passing across actors which are >> distributed across different worker machines. I found Akka to be most >> suitable for the job. >> >> Please suggest if it can be done by any other suitable frameworks. >> >> Regards, >> Disha >> >> On Sun, Dec 27, 2015 at 1:04 PM, Reynold Xin <r...@databricks.com> wrote: >> >>> We are just removing Spark's dependency on Akka. It has nothing to do >>> with whether user applications can use Akka or not. As a matter of fact, by >>> removing the Akka dependency from Spark, it becomes easier for user >>> applications to use Akka, because there is no more dependency conflict. >>> >>> For more information, see >>> https://issues.apache.org/jira/browse/SPARK-5293 >>> >>> On Sat, Dec 26, 2015 at 9:31 PM, Soumya Simanta < >>> soumya.sima...@gmail.com> wrote: >>> >>>> >>>> >>>> Any rationale for removing Akka from Spark ? Also, what is the >>>> replacement ? >>>> >>>> Thanks >>>> >>>> On Dec 27, 2015, at 8:31 AM, Dean Wampler <deanwamp...@gmail.com> >>>> wrote: >>>> >>>> Note that Akka is being removed from Spark. Even if it weren't, I would >>>> consider keeping Akka processes separate from Spark processes, so you can >>>> monitor, debug, and scale them independently. So consider streaming data >>>> from Akka to Spark Streaming or go the other way, from Spark to Akka >>>> Streams. >>>> >>>> dean >>>> >>>> Dean Wampler, Ph.D. >>>> Author: Programming Scala, 2nd Edition >>>> <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) >>>> Typesafe <http://typesafe.com> >>>> @deanwampler <http://twitter.com/deanwampler> >>>> http://polyglotprogramming.com >>>> >>>> On Sat, Dec 26, 2015 at 12:54 PM, Ted Yu <yuzhih...@gmail.com> wrote: >>>> >>>>> Do you mind sharing your use case ? >>>>> >>>>> It may be possible to use a different approach than Akka. >>>>> >>>>> Cheers >>>>> >>>>> On Sat, Dec 26, 2015 at 10:08 AM, Disha Shrivastava < >>>>> dishu....@gmail.com> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I wanted to know how to use Akka framework with Spark starting from >>>>>> basics. I saw online that Spark uses Akka framework but I am not really >>>>>> sure if I can define Actors and use it in Spark. >>>>>> >>>>>> Also, how to integrate Akka with Spark as in how will I know how many >>>>>> Akka actors are running on each of my worker machines? Can I control >>>>>> that? >>>>>> >>>>>> Please help. The only useful resource which I could find online was >>>>>> Akka with Spark Streaming which was also not very clear. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Disha >>>>>> >>>>> >>>>> >>>> >>> >> >
