Spark and RDF

Hi guys, I'm analyzing the possibility to use Spark to analyze RDF files and define reusable Shark operators on them (custom filtering, transforming, aggregating, etc). Is that possible? Any hint? Best, Flavio

Re: Spark and RDF

For RDF, may GraphX be particularly approriated? aℕdy ℙetrella about.me/noootsab [image: aℕdy ℙetrella on about.me] On Thu, Jun 19, 2014 at 4:49 PM, Flavio Pompermaier wrote: > Hi guys, > > I'm analyzing the possibility to use Spark to analyze RDF files and define

Re: Spark and RDF

You are looking to create Shark operators for RDF? Since Shark backend is shifting to SparkSQL it would be slightly hard but much better effort would be to shift Gremlin to Spark (though a much beefier one :) ) Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi

Re: Spark and RDF

Maybe some SPARQL features in Shark, then ? aℕdy ℙetrella about.me/noootsab [image: aℕdy ℙetrella on about.me] On Fri, Jun 20, 2014 at 9:45 PM, Mayur Rustagi wrote: > You are looking to create Shark operators for RDF? Since Shark backend is > shifting to SparkSQL i

Re: Spark and RDF

or a seperate RDD for sparql operations ala SchemaRDD .. operators for sparql can be defined thr.. not a bad idea :) Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi On Fri, Jun 20, 2014 at 3:56 PM, andy petrella wrote: > M

Re: Spark and RDF

yep, would be cool. Even though sparql has its drawbacks (vs cypher vs gremlin I mean), however still cool for semantic thingies and c°. aℕdy ℙetrella about.me/noootsab [image: aℕdy ℙetrella on about.me] On Fri, Jun 20, 2014 at 10:03 PM, Mayur Rustagi wrote: > or