Re: Implementing TinkerPop on top of GraphX

2015-01-16 Thread Kushal Datta
an API definition outside of Spark. Thanks, -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Implementing-TinkerPop-on-top-of-GraphX-tp9169p10126.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com

Re: Implementing TinkerPop on top of GraphX

2015-01-16 Thread Kyle Ellrott
://apache-spark-developers-list.1001551.n3.nabble.com/Implementing-TinkerPop-on-top-of-GraphX-tp9169p10126.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev

Re: Implementing TinkerPop on top of GraphX

2015-01-16 Thread Kushal Datta
/incubator/TinkerPopProposal which might alleviate concerns about having an API definition outside of Spark. Thanks, -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Implementing-TinkerPop-on-top-of-GraphX-tp9169p10126.html Sent from

Re: Implementing TinkerPop on top of GraphX

2015-01-16 Thread Kushal Datta
: http://apache-spark-developers-list.1001551.n3.nabble.com/Implementing-TinkerPop-on-top-of-GraphX-tp9169p10126.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev

Re: Implementing TinkerPop on top of GraphX

2015-01-15 Thread David Robinson
/TinkerPopProposal which might alleviate concerns about having an API definition outside of Spark. Thanks, -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Implementing-TinkerPop-on-top-of-GraphX-tp9169p10126.html Sent from the Apache Spark Developers List

Re: Implementing TinkerPop on top of GraphX

2014-11-20 Thread Kushal Datta
Subject: Re: Implementing TinkerPop on top of GraphX I still have to dig into the Tinkerpop3 internals (I started my work long before it had been released), but I can say that to get the Tinerpop2 Gremlin pipeline to work in the GraphX was a bit of a hack. The whole Tinkerpop2 Gremlin design

Re: Implementing TinkerPop on top of GraphX

2014-11-18 Thread Kyle Ellrott
Subject: Re: Implementing TinkerPop on top of GraphX Before we dive into the implementation details, what are the high level thoughts on Gremlin/GraphX? Scala already provides the procedural way to query graphs in GraphX today. So, today I can run g.vertices().filter().join() queries as OLAP

Re: Implementing TinkerPop on top of GraphX

2014-11-07 Thread Kyle Ellrott
...@thinkaurelius.com Subject: Re: Implementing TinkerPop on top of GraphX Before we dive into the implementation details, what are the high level thoughts on Gremlin/GraphX? Scala already provides the procedural way to query graphs in GraphX today. So, today I can run g.vertices().filter().join() queries

Re: Implementing TinkerPop on top of GraphX

2014-11-07 Thread York, Brennon
Datta kushal.da...@gmail.commailto:kushal.da...@gmail.com, dev@spark.apache.orgmailto:dev@spark.apache.org dev@spark.apache.orgmailto:dev@spark.apache.org, Matthias Broecheler matth...@thinkaurelius.commailto:matth...@thinkaurelius.com Subject: Re: Implementing TinkerPop on top of GraphX Who here

Re: Implementing TinkerPop on top of GraphX

2014-11-07 Thread Kushal Datta
: Implementing TinkerPop on top of GraphX I still have to dig into the Tinkerpop3 internals (I started my work long before it had been released), but I can say that to get the Tinerpop2 Gremlin pipeline to work in the GraphX was a bit of a hack. The whole Tinkerpop2 Gremlin design was based around

Implementing TinkerPop on top of GraphX

2014-11-06 Thread York, Brennon
All, was wondering if there had been any discussion around this topic yet? TinkerPop https://github.com/tinkerpop is a great abstraction for graph databases and has been implemented across various graph database backends / gaining traction. Has anyone thought about integrating the TinkerPop

Re: Implementing TinkerPop on top of GraphX

2014-11-06 Thread Reynold Xin
cc Matthias In the past we talked with Matthias and there were some discussions about this. On Thu, Nov 6, 2014 at 11:34 AM, York, Brennon brennon.y...@capitalone.com wrote: All, was wondering if there had been any discussion around this topic yet? TinkerPop https://github.com/tinkerpop is a

Re: Implementing TinkerPop on top of GraphX

2014-11-06 Thread Kyle Ellrott
I've taken a crack at implementing the TinkerPop Blueprints API in GraphX ( https://github.com/kellrott/sparkgraph ). I've also implemented portions of the Gremlin Search Language and a Parquet based graph store. I've been working out finalize some code details and putting together better code

Re: Implementing TinkerPop on top of GraphX

2014-11-06 Thread andy petrella
Great stuffs! I've got some thoughts about that, and I was wondering if it would be first interesting to have something like for spark-core (let's say): 0/ Core API offering basic (or advanced → HeLP) primitives 1/ catalyst optimizer for a text base system (SPARQL, Cypher, custom SQL3, whatnot) 2/

Re: Implementing TinkerPop on top of GraphX

2014-11-06 Thread Kyle Ellrott
I still have to dig into the Tinkerpop3 internals (I started my work long before it had been released), but I can say that to get the Tinerpop2 Gremlin pipeline to work in the GraphX was a bit of a hack. The whole Tinkerpop2 Gremlin design was based around streaming pipes of data, rather then

Re: Implementing TinkerPop on top of GraphX

2014-11-06 Thread York, Brennon
brennon.y...@capitalone.commailto:brennon.y...@capitalone.com, dev@spark.apache.orgmailto:dev@spark.apache.org dev@spark.apache.orgmailto:dev@spark.apache.org, Matthias Broecheler matth...@thinkaurelius.commailto:matth...@thinkaurelius.com Subject: Re: Implementing TinkerPop on top of GraphX I still

Re: Implementing TinkerPop on top of GraphX

2014-11-06 Thread Kyle Ellrott
brennon.y...@capitalone.com, dev@spark.apache.org dev@spark.apache.org, Matthias Broecheler matth...@thinkaurelius.com Subject: Re: Implementing TinkerPop on top of GraphX I still have to dig into the Tinkerpop3 internals (I started my work long before it had been released), but I can say that to get

Re: Implementing TinkerPop on top of GraphX

2014-11-06 Thread Kushal Datta
Date: Thursday, November 6, 2014 at 12:10 PM To: Kushal Datta kushal.da...@gmail.com Cc: Reynold Xin r...@databricks.com, York, Brennon brennon.y...@capitalone.com, dev@spark.apache.org dev@spark.apache.org, Matthias Broecheler matth...@thinkaurelius.com Subject: Re: Implementing TinkerPop on top

Re: Implementing TinkerPop on top of GraphX

2014-11-06 Thread York, Brennon
...@databricks.commailto:r...@databricks.com, dev@spark.apache.orgmailto:dev@spark.apache.org dev@spark.apache.orgmailto:dev@spark.apache.org, Matthias Broecheler matth...@thinkaurelius.commailto:matth...@thinkaurelius.com Subject: Re: Implementing TinkerPop on top of GraphX Before we dive into the implementation

Re: Implementing TinkerPop on top of GraphX

2014-11-06 Thread Kyle Ellrott
...@thinkaurelius.com Subject: Re: Implementing TinkerPop on top of GraphX Before we dive into the implementation details, what are the high level thoughts on Gremlin/GraphX? Scala already provides the procedural way to query graphs in GraphX today. So, today I can run g.vertices().filter

Re: Implementing TinkerPop on top of GraphX

2014-11-06 Thread Reynold Xin
...@databricks.com, dev@spark.apache.org dev@spark.apache.org, Matthias Broecheler matth...@thinkaurelius.com Subject: Re: Implementing TinkerPop on top of GraphX Before we dive into the implementation details, what are the high level thoughts on Gremlin/GraphX? Scala already provides the procedural way