spark-0.7.3/bagel/target or spark-0.7.3/bagel/bagel/target ?
On Wed, Aug 7, 2013 at 9:17 PM, Matei Zaharia <[email protected]> wrote: > Hi Ryan, > > To use Bagel, you can just add the spark-bagel.jar built within the > bagel/target directory, in addition to the spark-core-assembly. It doesn't > have other dependencies beyond Spark Core. > > GraphX is an upcoming project to do richer types of graph processing, but > it's fine to use Bagel for now. We may eventually update the Bagel API to run > on top of GraphX. > > Matei > > On Aug 7, 2013, at 8:54 PM, Ryan Compton <[email protected]> wrote: > >> I'm interested in processing graphs with Spark. I'm able to do sbt/sbt >> assembly and the process exits with success. I can find: >> >> -bash-3.2$ find . | grep jar$ | grep -assembly- >> ./core/target/spark-core-assembly-0.7.3cdh3u3.jar >> ./streaming/target/spark-streaming-assembly-0.7.3cdh3u3.jar >> >> I add these to my local Nexus repo, include them in my pom.xml, and >> build/run distributed jobs just fine. >> >> Now, how do I code up iterative graph algorithms? "sbt/sbt assembly" >> didn't make a Bagel assembly. >> >> Also, I can't find example code. The code here: >> https://github.com/mesos/spark/wiki/Bagel-Programming-Guide has lots >> of @serializable depreciation warnings and I can't find the ./run >> script they are taking about at the bottom of this page: >> http://spark-project.org/docs/latest/bagel-programming-guide.html >> >> I get the impression that Bagel may be discontinued. I also noticed >> some ads for something called "GraphX" on the BDAS website. Should I >> be using GraphX instead? >> >> On Wed, Aug 7, 2013 at 7:05 PM, Ryan Compton <[email protected]> wrote: >>> I'm interested in processing graphs with Spark. I'm able to do >>> sbt/sbt assembly and the process exits with success. I can find: >>> >>> -bash-3.2$ find . | grep jar$ | grep -assembly- >>> ./core/target/spark-core-assembly-0.7.3cdh3u3.jar >>> ./streaming/target/spark-streaming-assembly-0.7.3cdh3u3.jar >>> >>> I add these to my local Nexus repo, include them in my pom.xml, and >>> build/run distributed jobs just fine. >>> >>> Now, how do I code up iterative graph algorithms? "sbt/sbt assembly" >>> didn't make a Bagel assembly. >>> >>> Also, I can't find example code. The code here: >>> https://github.com/mesos/spark/wiki/Bagel-Programming-Guide has lots >>> of @serializable depreciation warnings and I can't find the ./run >>> script they are taking about at the bottom of this page: >>> http://spark-project.org/docs/latest/bagel-programming-guide.html >>> >>> I get the impression that Bagel may be discontinued. Is there >>> something else I can use for graphs? >>> >>> On Wed, Aug 7, 2013 at 7:02 PM, Ryan Compton <[email protected]> wrote: >>>> I'm interesting in processing graphs with Spark. I'm able to do >>>> sbt/sbt assembly and the process exits with success. I can find: >>>> >>>> -bash-3.2$ find . | grep jar$ | grep -assembly- >>>> ./core/target/spark-core-assembly-0.7.3cdh3u3.jar >>>> ./streaming/target/spark-streaming-assembly-0.7.3cdh3u3.jar >>>> >>>> I add these to my local Nexus repo, include them in my pom.xml, and >>>> build/run distributed jobs just fine. >>>> >>>> Now, how do I code up iterative graph algorithms? "sbt/sbt assembly" >>>> didn't make a Bagel assembly. >>>> >>>> Also, I can't find example code. The code here: >>>> https://github.com/mesos/spark/wiki/Bagel-Programming-Guide has lots >>>> of @serializable depreciation warnings and I can't find the ./run >>>> script they are taking about at the bottom of this page: >>>> http://spark-project.org/docs/latest/bagel-programming-guide.html >>>> >>>> I get the impression that Bagel may be discontinued. Is there >>>> something else I can use for graphs? >
