> + I'll come back to this in the points below, but as a big fat general question: how will we deal with transactions?
Bullseye. I'd add that assuming that TinkerGraph would be the "one true TinkerPop graph" for gremlin-x testing, it needs to support transactions which it doesn't do currently. Or introduce a new reference implementation graph that does. On Wed, Nov 30, 2016 at 4:58 PM, Stephen Mallette <spmalle...@gmail.com> wrote: > Added back "dev".... > > I think we've tried a lot of things for TinkerPop 3 and that there are too > many options for application development as well as an over-exposed API, so > I like this direction, though a fair bit of complexity lies in making it > happen. Before getting to that complexity, I don't really think of this as > a "new idea". To me the description of gremlin-x is really just a GLV for > Java. I mean - consider that gremlin-python (something we classify as a > GLV) is basically "everything a python dev needs to build applications - > nothing more/less" and that's pretty much what we want for java in > gremlin-x. gremlin-x would basically be "everything a java dev needs to > build applications - nothing more/less". So I'm not seeing much distinction > there (nor the need for naming it "gremlin-x" really - seems to be as > simple as "gremlin-java" to be in line with other GLVs we will have). > > Back to the "complexity" - and I think the questions in my mind that > follow apply with or without this gremlin-x idea (they are general issues > that just are faced by "remote" and GLVs): > > + I'll come back to this in the points below, but as a big fat general > question: how will we deal with transactions? > + What will be the nature of Graph.io() - in one sense i'd like to see > that as something hidden as it is the source of a fair bit of confusion > (i.e. someone trying to load a multi GB graphson file) but on the other > hand it's convenient for small graphs - that story could be made better. > Maybe Graph.io() being a part of "Graph" is hidden and only used by the > test suite to load data into graph providers implementations for testing > purposes. Of course, we will need our story in order for loading data in > different formats if that isn't available (which I guess we need to do > anyway). > + Can remoting Gremlin bytecode cover everything that users currently do > in embedded/local mode? The drawback with a remote traversal is that it is > bound to a single transaction and you therefore lose some efficiency with > certain graphs databases as logic with three traversals that might span a > single transaction, for example, will be executed less efficiently as three > different transaction refreshing a transaction cache each time. Maybe the > server protocol needs to be session based? Maybe, going back to the first > bullet i had, the transaction model needs to change so that its not bound > to a single thread by default? > + Related to the last point, is there an easy way to blend client code > with server code? Meaning, if i have a body of complex logic right now, my > only option is to submit a script. I think the ScriptEngine stuff should > really just be used for those situations where you need a lambda. That's a > nice, simple, easy-to-explain rule, whereas "do it if there is complex > logic" is a bit more subjective (and potentially troublesome). How can GLVs > get complex logic on the "server" (let's say the logic is written in java > and available to gremlin server) so that it could be executed as part of > some bytecode in any GLV? > + It would be neat to see what kind of "server" changes we could make to > optimize for bytecode. Right now we sorta tied bytecode execution into the > same Gremlin Server execution pipeline as ScriptEngine processing. I think > we could probably do better on that. > > Not sure there are any immediate answers to any of the above, but it's > what I've been thinking about with respect to TinkerPop 3.3.0 lately. > > > > On Wed, Nov 30, 2016 at 1:11 PM, pieter-gmail <pieter.mar...@gmail.com> > wrote: > >> Ok, if Sqlg just implements its own RemoteConnection and no need for an >> additional server in the architecture then I am a lot less nervous about >> gremlin-x. >> >> As an aside, Sqlg has its own peculiarities and optimization outside the >> standard TinkerPop interface. Most providers will have such features. If >> the graph is not exposed directly there will need to be some way to call >> custom features. No idea how now, but still. >> >> E.g. for Sqlg the most used such feature is >> SqlgGraph.tx().batchMode(batchModeType) but there are others. Index >> management will be another. Without being able to access these features >> too many benefits of choosing a particular provider will be lost. >> >> Cheers >> Pieter >> >> On 30/11/2016 19:31, Marko Rodriguez wrote: >> > Hi, >> > >> >> I generally recommend the exact opposite. Neo4j shines when embedded. >> > >> > And thats fine. There would simply be a EmbeddedRemoteConnection which >> > is basically just a proxy that does nothing. No need for server stuff. >> > >> >> Sqlg already works of databases that is designed for remote access. >> > >> > And yes, thats where a full fledged RemoteConnection (like >> > GremlinServer’s) is needed for marshaling data over a network. >> > >> >> Wrapping that with yet another server is an unnecessary complication. >> > >> > Its not about GremlinServer so much as its about RemoteConnection. If >> > Sqlg has their own RemoteConnection implementation, then it uses that. >> > Likewise, Neo4jServer could provide its own RemoteConnection >> > implementation and thus, no need for GremlinServer. >> > >> >> (Not having used it I do not know the usability or performance impact, >> >> however being another layer it can only degrade performance and >> >> complexity). >> > >> > For embedded, its an implementation that just returns the traversal as >> > was submitted. >> > For remote, its as it is now. You have to send the bytecode over the >> > network and stream back results. >> > >> >> Further because Sqlg already works of a remote database it supports >> many >> >> jvm's directly accessing the same graph. I imagine other providers >> could >> >> have the same feature. >> > >> > This is fine with RemoteConnection. What is “not fine” (the reason for >> > gremlin-x) is to make it so users can’t talk with the Graph object. >> > That is a Java thing and should not be exposed to users. More >> > specifically, that is a Gremlin traversal machine <-> graph system >> > provider interface. Users only interact with a RemoteConnection >> > implementation. >> > >> >> So gremlin-x is fine but it is not the obvious best choice for all >> >> architectures. >> > >> > I was shootin’ this idea around DSEGraph people and Bob was like: >> > “pfff…where was this idea 4 years ago. Everyone is already addicting >> > to sending a Gremlin-Groovy script over the network and getting a >> > ResultSet back.” >> > >> >> All the above is for happy java devs. >> > >> > Again, the purpose is the constrain Java developers too. Things like >> > this should never happen: >> > >> > v1.property(‘name’).value() >> > v1.edges(OUT).next() >> > cluster.submit(“1+2”) >> > graph = GraphFactory.open(...) >> > graph.io <http://graph.io>(gryo()).readGraph(‘data/blah.gryo’) >> > >> > >> > >> > It should be: >> > >> > g.V(1).values(‘name’) >> > g.V(1).outE().next() >> > g.inject(1,2).sum() >> > g.withRemote(config) >> > // hmmm… don’t know about the last one. >> > >> > >> > See ya, >> > Marko. >> > >> > >> >> Cheers >> >> Pieter >> >> >> >> On 30/11/2016 16:06, Marko Rodriguez wrote: >> >>> Hello, >> >>> >> >>> I think TinkerPop has too many ways of doing things. For instance: >> >>> >> >>> 1. Users can use graph traversal or interact with vertex/edge/etc. >> >>> Java objects directly. >> >>> 2. Users can interact with the graph system embedded (via the Graph >> >>> object) or remotely (via RemoteConnection). >> >>> 3. Users can submit Gremlin string scripts or submit Gremlin >> >>> bytecode to GremlinServer (along with other REST/etc. type >> >>> capabilities possible). >> >>> 4. Users can use GraphML, GraphSON, or Gryo for serialization. >> >>> 5. Users can use Gremlin-Groovy, Gremlin-Java, Gremlin-Python, >> >>> Gremlin-Scala, etc. >> >>> 6. Users can use Titan, Neo4j, OrientDB, etc. >> >>> >> >>> I think we should create a module called gremlin-x/ which is the >> >>> most simplified, concise, recommended way to use TinkerPop. In >> >>> correspondence to the itemization above: >> >>> >> >>> 1. Users can only use graph traversal. All returned elements are >> >>> ReferenceElements. >> >>> 2. Users can only interact with the graph remotely (where remote may >> >>> just be a localhost connection — in-memory proxy (e.g. for >> >>> TinkerGraph)). >> >>> 3. Users can only submit Gremlin bytecode to GremlinServer (thus, >> >>> only RemoteConnection can be used). No more security issues, no more >> >>> uninterruptible scripts, etc. >> >>> 4. Users can only use GraphSON as its language agnostic. >> >>> 5. Users can use any Gremlin language variant as long is generates >> >>> Gremlin bytecode. >> >>> 6. Users can use any graph system provider as long as it has a >> >>> RemoteConnection exposed (where the provider can simply leverage >> >>> GremlinServer). >> >>> >> >>> Finally, we should create a separate documentation called >> >>> gremlin-x.asciidoc which is small and discusses the rationale for >> >>> the constrained specification and references the core reference docs >> >>> accordingly. For instance, no need to have all the step examples >> >>> re-written. >> >>> >> >>> I believe that this will make it easier for users to say the >> following: >> >>> >> >>> “Okay, I get it. Open a RemoteConnection from a GraphTraversalSource >> >>> (g.withRemote(...)) and spawn traversals and get back results. Thats >> >>> it?! Easy peasy lemon squeezy.” >> >>> >> >>> I believe that this will make it easier for system developers to say >> >>> the following: >> >>> >> >>> “We can cover 95% of use cases by simply exposing a >> >>> RemoteConnection. Thats it?! Easy peasy lemon squeezy." >> >>> >> >>> In general, this model looks at TinkerPop as a “remote database >> >>> system” where users submit “queries” and get back “results.” That is >> >>> it. Nothing Java-specific, nothing with executing arbitrary scripts >> >>> remotely, nothing with REST, nothing with embedded (at least from a >> >>> look-and-feel perspective), etc. >> >>> >> >>> Thoughts?, >> >>> Marko. >> >>> >> >>> http://markorodriguez.com >> >>> >> >>> >> >>> >> >> >> >> -- >> >> You received this message because you are subscribed to the Google >> >> Groups "Gremlin-users" group. >> >> To unsubscribe from this group and stop receiving emails from it, >> >> send an email to gremlin-users+unsubscr...@googlegroups.com >> >> <mailto:gremlin-users+unsubscr...@googlegroups.com>. >> >> To view this discussion on the web visit >> >> https://groups.google.com/d/msgid/gremlin-users/6f8790d7-9fa >> 8-99c8-b553-1793c42f9c04%40gmail.com. >> >> For more options, visit https://groups.google.com/d/optout. >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups "Gremlin-users" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an email to gremlin-users+unsubscr...@googlegroups.com >> > <mailto:gremlin-users+unsubscr...@googlegroups.com>. >> > To view this discussion on the web visit >> > https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF >> 8-4683-8C1F-E01A3C4F7815%40gmail.com >> > <https://groups.google.com/d/msgid/gremlin-users/50D34645-5A >> F8-4683-8C1F-E01A3C4F7815%40gmail.com?utm_medium=email&utm_source=footer >> >. >> > For more options, visit https://groups.google.com/d/optout. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Gremlin-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to gremlin-users+unsubscr...@googlegroups.com. >> To view this discussion on the web visit https://groups.google.com/d/ms >> gid/gremlin-users/1b1639d8-4c1d-ed65-5218-8c8786b76c97%40gmail.com. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "Gremlin-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to gremlin-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/gremlin-users/CAA-H4390oFKxksw1bxDSO7Ej83NBes6Na > BD_EBG77wD6QC786Q%40mail.gmail.com > <https://groups.google.com/d/msgid/gremlin-users/CAA-H4390oFKxksw1bxDSO7Ej83NBes6NaBD_EBG77wD6QC786Q%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. >
