I'm interested in processing graphs with Spark. I'm able to do sbt/sbt assembly and the process exits with success. I can find:
-bash-3.2$ find . | grep jar$ | grep -assembly- ./core/target/spark-core-assembly-0.7.3cdh3u3.jar ./streaming/target/spark-streaming-assembly-0.7.3cdh3u3.jar I add these to my local Nexus repo, include them in my pom.xml, and build/run distributed jobs just fine. Now, how do I code up iterative graph algorithms? "sbt/sbt assembly" didn't make a Bagel assembly. Also, I can't find example code. The code here: https://github.com/mesos/spark/wiki/Bagel-Programming-Guide has lots of @serializable depreciation warnings and I can't find the ./run script they are taking about at the bottom of this page: http://spark-project.org/docs/latest/bagel-programming-guide.html I get the impression that Bagel may be discontinued. I also noticed some ads for something called "GraphX" on the BDAS website. Should I be using GraphX instead? On Wed, Aug 7, 2013 at 7:05 PM, Ryan Compton <[email protected]> wrote: > I'm interested in processing graphs with Spark. I'm able to do > sbt/sbt assembly and the process exits with success. I can find: > > -bash-3.2$ find . | grep jar$ | grep -assembly- > ./core/target/spark-core-assembly-0.7.3cdh3u3.jar > ./streaming/target/spark-streaming-assembly-0.7.3cdh3u3.jar > > I add these to my local Nexus repo, include them in my pom.xml, and > build/run distributed jobs just fine. > > Now, how do I code up iterative graph algorithms? "sbt/sbt assembly" > didn't make a Bagel assembly. > > Also, I can't find example code. The code here: > https://github.com/mesos/spark/wiki/Bagel-Programming-Guide has lots > of @serializable depreciation warnings and I can't find the ./run > script they are taking about at the bottom of this page: > http://spark-project.org/docs/latest/bagel-programming-guide.html > > I get the impression that Bagel may be discontinued. Is there > something else I can use for graphs? > > On Wed, Aug 7, 2013 at 7:02 PM, Ryan Compton <[email protected]> wrote: >> I'm interesting in processing graphs with Spark. I'm able to do >> sbt/sbt assembly and the process exits with success. I can find: >> >> -bash-3.2$ find . | grep jar$ | grep -assembly- >> ./core/target/spark-core-assembly-0.7.3cdh3u3.jar >> ./streaming/target/spark-streaming-assembly-0.7.3cdh3u3.jar >> >> I add these to my local Nexus repo, include them in my pom.xml, and >> build/run distributed jobs just fine. >> >> Now, how do I code up iterative graph algorithms? "sbt/sbt assembly" >> didn't make a Bagel assembly. >> >> Also, I can't find example code. The code here: >> https://github.com/mesos/spark/wiki/Bagel-Programming-Guide has lots >> of @serializable depreciation warnings and I can't find the ./run >> script they are taking about at the bottom of this page: >> http://spark-project.org/docs/latest/bagel-programming-guide.html >> >> I get the impression that Bagel may be discontinued. Is there >> something else I can use for graphs?
