The use of an index for injection was just for purpose of demonstrating
things quickly. You could maybe use an identity step or something as a
placeholder and then inject/replace on that step instance. It's not
perfect - just thinking out loud. There is a ticket out there to make
TraversalStrategy writing "better". Ultimately, anything we do in that
regard would probably help this problem.
On Thu, Apr 21, 2016 at 10:32 AM, Dylan Millikin <dylan.milli...@gmail.com>
wrote:
> Ohh, I wasn't aware of that. It looks like TraversalHelper has a bunch of
> really useful stuff for these scenarios.
> You are indeed grasping the problem.
> Looking at the signatures there are still grey areas and things that aren't
> really usable (like having to know the index of the injection point) and
> I'm not really sure how this translates to sub traversals like:
>
> g.V().hasLabel("project").union(out('dev')/** inject here **/,
> out('company'))
> yet still work with:
> g.V().hasLabel("person")/** inject here **/.out('friend').out('company');
>
> Is there perhaps a way of including a flag somewhere and have that server
> as the index id ?
>
> Anyways this is definitely what I'm talking about. I'm guessing that there
> may be some conflicts from the fact that method signatures aren't
> necessarily kept in gremlin language variants. Mostly around changing
> existing steps, but if there's a way to get this to work in broader use
> cases it would definitely fit the bill.
>
> Cheers for that. I'll definitely dig a bit deeper around this.
>
> On Thu, Apr 21, 2016 at 10:18 AM, Stephen Mallette <spmalle...@gmail.com>
> wrote:
>
> > I'm not sure what you mean by:
> >
> > > Why is this not an option: It boils down to the fact that you still
> > can't modify
> > a traversal "after the fact".
> >
> > You can modify the Traversal up to the point where you iterate it and
> > TraversalStrategy are applied.
> >
> > gremlin> base = g.V().hasLabel('person').out();[]
> > gremlin> filter = __.has('age',29)
> > gremlin> TraversalHelper.insertTraversal(1,filter,base);[]
> > gremlin> base.toString()
> > ==>[GraphStep(vertex,[]), HasStep([~label.eq(person)]),
> > HasStep([age.eq(29)]), VertexStep(OUT,vertex)]
> > gremlin> base
> > ==>v[3]
> > ==>v[2]
> > ==>v[4]
> >
> > Note how I injected an anonymous Traversal (i.e. the "filter") into the
> > middle of a "base" Traversal. This is a little known function that is
> > probably familiar to TraversalStrategy developers only, but it exists. I
> > suppose more advanced forms of Gremlin Language Variants should/could
> have
> > this kind of capability - I guess the point is that it's possible and
> > arguably better than string manipulation. You might not agree, but am I
> at
> > least grasping the problem you're trying to express?
> >
> >
> > On Thu, Apr 21, 2016 at 9:49 AM, Dylan Millikin <
> dylan.milli...@gmail.com>
> > wrote:
> >
> > > Essentially what is required is to be able to "build" traversals with a
> > > complete disregard for the order of the steps you're adding. Think of
> it
> > as
> > > adding entries to a match(). It doesn't matter what order you add those
> > > entries in.
> > > This is something that is very "declarative" in nature but is a
> > requirement
> > > when developing complex applications that need modularity. You also
> need
> > > this in an imperative setting as you may want control on how the data
> is
> > > fetched.
> > >
> > > On Thu, Apr 21, 2016 at 9:39 AM, Dylan Millikin <
> > dylan.milli...@gmail.com>
> > > wrote:
> > >
> > > > Yeah there are some more complex situations. You actually left out
> the
> > > > final out() in your example. For instance just keeping things simple,
> > > > your base query could be required to be reused in many places. So the
> > > > following is not an option:
> > > >
> > > > base = g.V().hasLabel('person');[]
> > > > /** apply some filters like you did **/
> > > > base.out('company')
> > > >
> > > > Why is this not an option: It boils down to the fact that you still
> > can't
> > > > modify a traversal "after the fact".
> > > > Your reusable base is not complete without the out() step. Therefore,
> > > > using this model you can't store the base query and test it
> separately.
> > > You
> > > > also have to append the out step everywhere you use this base, which
> in
> > > > addition to creating a lot of duplicate code can also not be an
> option
> > > when
> > > > queries are generated automatically. Sometimes you don't even really
> > know
> > > > what the base query is, you just know that it contains certain
> > "injection
> > > > points".
> > > > To give you a few examples, these following base queries all have the
> > > same
> > > > injection point that allows them to be edited with the same filters:
> > > >
> > > > g.V().hasLabel("person")/** inject here **/.out('company');
> > > > g.V().hasLabel("person")/** inject here
> > **/.out('friend').out('company');
> > > > g.V().has("person", "name", "marko").out('friend')/** inject here
> > > > **/.out('company');
> > > >
> > > > These are all valid. My filters apply to "people" but the traversals
> > > > leading to and from these people can be anything. And
> programmatically
> > I
> > > > have no way of knowing what the traversal is. This gets increasingly
> > > > complexe with injection points in sub-traversals, or when injecting
> > full
> > > > traversals with their own injection points, and so on.
> > > >
> > > > Also your example doesn't cover changing existing steps.
> > > >
> > > > I don't know if that sheds anymore light.
> > > >
> > > > On Thu, Apr 21, 2016 at 9:14 AM, Stephen Mallette <
> > spmalle...@gmail.com>
> > > > wrote:
> > > >
> > > >> So "query building" is where you hold string representations of
> steps
> > or
> > > >> groups of steps and then have some logic that concatenates them
> > > together.
> > > >> I
> > > >> guess my question is why are "strings" a requirement for that? It
> > seems
> > > >> like you could do the same query builder stuff with an actual
> > Traversal
> > > >> object in whatever Gremlin Language Variant you were working with.
> How
> > > is
> > > >> string concatenation different than this:
> > > >>
> > > >> gremlin> graph = TinkerFactory.createModern()
> > > >> ==>tinkergraph[vertices:6 edges:6]
> > > >> gremlin> g = graph.traversal()
> > > >> ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
> > > >> gremlin> base = g.V().hasLabel('person');[]
> > > >> gremlin> addFilters = { age, name, t ->
> > > >> gremlin> t = age > 0 ? t.has('age',age) : t
> > > >> gremlin> !name.isEmpty() ? t.has('name',name) : t
> > > >> gremlin> }
> > > >> ==>groovysh_evaluate$_run_closure1@503d56b5
> > > >> gremlin> traversal = addFilters(29,'',base);[]
> > > >> gremlin> traversal.toString()
> > > >> ==>[GraphStep(vertex,[]), HasStep([~label.eq(person)]),
> > > >> HasStep([age.eq(29)])]
> > > >> gremlin> traversal
> > > >> ==>v[1]
> > > >>
> > > >> What about DSLs? Maybe filtering logic drops behind a custom step
> > > specific
> > > >> to the domain:
> > > >>
> > > >> g.V().personsWith(29, '')
> > > >>
> > > >> which would basically compile to the same thing as the gremlin
> output
> > > >> above:
> > > >>
> > > >> [GraphStep(vertex,[]), HasStep([~label.eq(person)]),
> > > >> HasStep([age.eq(29)])]
> > > >>
> > > >> Is there some more complex aspect "query building" that can't be
> > handled
> > > >> this way (I know I took a fairly simple example)?
> > > >>
> > > >>
> > > >>
> > > >> On Wed, Apr 20, 2016 at 7:54 PM, Dylan Millikin <
> > > dylan.milli...@gmail.com
> > > >> >
> > > >> wrote:
> > > >>
> > > >> > I've approached this a few times in the past though never really
> in
> > > >> depth.
> > > >> > The idea is that you want separation of logic in your queries,
> > mostly
> > > >> for
> > > >> > maintenance, testing and overall convenience. This takes the form
> of
> > > >> > partial/incomplete traversals that have no start nor end.
> > > >> > One of the simplest applications would be to have a base query and
> > > apply
> > > >> > filters to this query. For example we can imagine a recap page of
> > > >> > everyone's companies:
> > > >> >
> > > >> > base query : g.V().has(label, "person").out('company')
> > > >> >
> > > >> > To this you have a set of filters that are partial traversals and
> > > allow
> > > >> > users to better refine their search:
> > > >> >
> > > >> > older than 30 filter: has("age", gt(30))
> > > >> > male filter: has("gender", "male")
> > > >> >
> > > >> > Now depending on user input you'll want to apply either or both of
> > the
> > > >> > above to "person" in order to obtain something along the lines of:
> > > >> >
> > > >> > g.V().has(label, "person").has("age", gt(30)).has("gender",
> > > >> > "male").out('company')
> > > >> >
> > > >> > Of course there are plenty of ways of doing the above depending on
> > > >> > requirements and complexity. Such as where() or
> > as("a")....select("a")
> > > >> > etc..
> > > >> >
> > > >> > These are simple examples where steps are appended in various
> places
> > > but
> > > >> > you can imagine the same with traversal manipulation such as
> turning
> > > >> > out('company') into out('company', 'organization') where the step
> > gets
> > > >> > altered. This would work the same way if you used
> > > union(out('company'))
> > > >> > instead. You would have to alter the union step like this :
> > > >> > union(out('company'),
> > > >> > out('organization'))
> > > >> >
> > > >> > All of the above are relatively simple but picture having a lot of
> > > very
> > > >> > complex traversal "filters" that sometimes have dependencies on
> > other
> > > >> > filters.
> > > >> > All this is doable but it currently requires looking at the
> gremlin
> > > >> > building process as a query building one rather than native
> support
> > > for
> > > >> the
> > > >> > gremlin language.
> > > >> >
> > > >> > Does that make any sense?
> > > >> >
> > > >> > On Wed, Apr 20, 2016 at 7:15 PM, Stephen Mallette <
> > > spmalle...@gmail.com
> > > >> >
> > > >> > wrote:
> > > >> >
> > > >> > > > It's imperative that you have the ability to group multiline
> > > >> scripts
> > > >> > > into a single query or the reactivity of your applications will
> > > >> greatly
> > > >> > > suffer.
> > > >> > >
> > > >> > > A fair point and something we might yet address as part of all
> > this
> > > >> > > thinking. RemoteGraph is pretty new and it demonstrated a
> critical
> > > >> aspect
> > > >> > > of communications with Gremlin Server. Now we need to think
> about
> > > how
> > > >> to
> > > >> > > improve upon it.
> > > >> > >
> > > >> > > > Namely having the ability to edit/modify/add steps to an
> already
> > > >> > defined
> > > >> > > traversal.
> > > >> > >
> > > >> > > I don't quite follow that point...could you please elaborate?
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > On Wed, Apr 20, 2016 at 4:15 PM, Dylan Millikin <
> > > >> > dylan.milli...@gmail.com>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Nice post,
> > > >> > > >
> > > >> > > > I'm going to jump straight to the end. I think that one of the
> > > >> problems
> > > >> > > of
> > > >> > > > doing an abstraction similar to remoteGraph is that in reality
> > the
> > > >> > > overhead
> > > >> > > > of communicating with the server is too big to make this
> viable.
> > > >> It's
> > > >> > > > imperative that you have the ability to group multiline
> scripts
> > > >> into a
> > > >> > > > single query or the reactivity of your applications will
> greatly
> > > >> > suffer.
> > > >> > > >
> > > >> > > > The Gremlin-java base also has a few limitations that some
> query
> > > >> > builders
> > > >> > > > try to fix (which can only be done if you abandon the idea of
> a
> > > >> natural
> > > >> > > > gremlin language variant in favor of a query builder). Namely
> > > having
> > > >> > the
> > > >> > > > ability to edit/modify/add steps to an already defined
> > traversal.
> > > >> > Though
> > > >> > > in
> > > >> > > > time it might be nice to have these be part of the original
> > > >> > > implementation.
> > > >> > > >
> > > >> > > > I personally love the idea of gremlin language variants. I
> just
> > > >> don't
> > > >> > > think
> > > >> > > > they're production value is any good without some extended
> > > >> > functionality
> > > >> > > > (beyond what gremlin-java currently is).
> > > >> > > >
> > > >> > > > On Wed, Apr 20, 2016 at 3:23 PM, Stephen Mallette <
> > > >> > spmalle...@gmail.com>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > This thread on Gremlin Language Variants has been very
> > > >> interesting:
> > > >> > > > >
> > > >> > > > > https://pony-poc.apache.org/thread.html/Zcazrw7k442xcwc
> > > >> > > > >
> > > >> > > > > I think that this work goes a long way to address two issues
> > > I've
> > > >> > been
> > > >> > > > > concerned about:
> > > >> > > > >
> > > >> > > > > 1. Greater consistency in how different languages do Gremlin
> > > >> > > > > 2. Less fragmentation in terms of libraries and how they
> work
> > so
> > > >> that
> > > >> > > > users
> > > >> > > > > aren't confused with how to get started (though I don't
> think
> > > the
> > > >> > goal
> > > >> > > > here
> > > >> > > > > is to restrict choices or slow down innovation)
> > > >> > > > >
> > > >> > > > > One of the first things we should probably do is start
> > thinking
> > > in
> > > >> > > terms
> > > >> > > > of
> > > >> > > > > the types of libraries that are built on TinkerPop (outside
> of
> > > >> those
> > > >> > > > things
> > > >> > > > > that are Graph Systems) and those are listed here currently:
> > > >> > > > >
> > > >> > > > > http://tinkerpop.apache.org/#graph-libraries
> > > >> > > > >
> > > >> > > > > Marko mentioned to me that he saw the libraries we listed
> here
> > > >> > breaking
> > > >> > > > > into three categories:
> > > >> > > > >
> > > >> > > > > 1. Gremlin Language Variants - which the other thread
> > > demonstrates
> > > >> > > quite
> > > >> > > > > nicely
> > > >> > > > > 2. Gremlin Drivers - the Gremlin Server protocol
> > > implementations -
> > > >> > > those
> > > >> > > > > things that send traversals to Gremlin Server and get back
> > > >> results.
> > > >> > > > > 3. OGM and others - I say "others" because there might be
> > > plugins
> > > >> and
> > > >> > > > other
> > > >> > > > > similar odds and ends
> > > >> > > > >
> > > >> > > > > I like Marko's category system here and I think that having
> > > these
> > > >> > kinds
> > > >> > > > of
> > > >> > > > > categories will help folks organize their libraries to fit
> > into
> > > >> one
> > > >> > of
> > > >> > > > > these spaces and make it easier for users to know what they
> > need
> > > >> to
> > > >> > get
> > > >> > > > in
> > > >> > > > > order to start doing TinkerPop in their language.
> > > >> > > > >
> > > >> > > > > Anyway, the category thing is just setting the stage for
> this
> > > big
> > > >> > > > > bombshell. I think TinkerPop should consider maintaining
> the
> > > >> Gremlin
> > > >> > > > > Language Variants.
> > > >> > > > >
> > > >> > > > > Heresy! right?
> > > >> > > > >
> > > >> > > > > Well, I think it's the best way to achieve consistency
> across
> > > >> > > languages.
> > > >> > > > > Under this model, TinkerPop provides the base language
> variant
> > > and
> > > >> > > people
> > > >> > > > > can choose to extend upon it, but the base stays tied to our
> > > >> > archetype
> > > >> > > of
> > > >> > > > > Java and we end up with a much more clear story for
> virtually
> > > any
> > > >> > > > > programming language.
> > > >> > > > >
> > > >> > > > > So how do we do this? Slowly and deliberately. We should
> look
> > to
> > > >> only
> > > >> > > > > include language variants where we:
> > > >> > > > >
> > > >> > > > > + have good automation in place (like what Marko did for
> > > Python),
> > > >> > > > > + some competence on the committer list in that language
> > > >> > > > > + a nice testing framework that operates in our standard
> > > >> > build/release
> > > >> > > > > process.
> > > >> > > > >
> > > >> > > > > That's setting a high bar, but if we don't keep it high, I
> > think
> > > >> we
> > > >> > > will
> > > >> > > > be
> > > >> > > > > left unable to properly support and maintain what we hang
> out
> > > >> there.
> > > >> > > > >
> > > >> > > > > I'd also like to express that we should not be looking to
> > > maintain
> > > >> > > > language
> > > >> > > > > drivers. I think that should remain a third-party community
> > > effort
> > > >> > just
> > > >> > > > > like Graph Systems. In other words, we remain a repository
> for
> > > >> > > reference
> > > >> > > > > implementations for everything else. Why? Because, as it
> sits
> > > >> right
> > > >> > > now,
> > > >> > > > > just based on the level of effort for what Marko did with
> > > Python,
> > > >> > > > > maintaining a "base" Gremlin Language Variants shouldn't be
> > > hard.
> > > >> We
> > > >> > > > won't
> > > >> > > > > be building tons of add-on capabilities to the base
> variants -
> > > >> they
> > > >> > > will
> > > >> > > > > pretty much just stay on par with the java archetype.
> Drivers
> > > on
> > > >> the
> > > >> > > > other
> > > >> > > > > hand have lots of implementation details, with many
> different
> > > >> > > > technologies
> > > >> > > > > that could be used, etc. They have similar complexity to
> > Graph
> > > >> > System
> > > >> > > > > implementations in many ways. I also think that the drivers
> > can
> > > >> > afford
> > > >> > > to
> > > >> > > > > have different APIs and approaches without being detrimental
> > to
> > > >> the
> > > >> > > > > community.
> > > >> > > > >
> > > >> > > > > If gremlin-js-driver wants to do:
> > > >> > > > >
> > > >> > > > > client.submit("g.V()")
> > > >> > > > >
> > > >> > > > > and gremlin-python-driver wants to do:
> > > >> > > > >
> > > >> > > > > client.send("g.V()")
> > > >> > > > >
> > > >> > > > > that's not a big deal.
> > > >> > > > >
> > > >> > > > > The last point that I'll make is that I think Gremlin
> Language
> > > >> > > Variants,
> > > >> > > > > that don't operate on the JVM (e.g. Jython) and use Gremlin
> > > >> Server,
> > > >> > > > should
> > > >> > > > > have some abstraction that is similar to RemoteGraph.
> > > RemoteGraph
> > > >> > > > exposes
> > > >> > > > > a DriverConnection interface that is currently implemented
> by
> > > >> > > > > gremlin-driver. The DriverConnection is responsible for
> > > sending a
> > > >> > > > > traversal to the server and returning results. It would be
> > nice
> > > if
> > > >> > the
> > > >> > > > > language variants had a similar interface that the various
> > > >> community
> > > >> > > > > drivers could implement. In that way, the user never has to
> do
> > > any
> > > >> > form
> > > >> > > > of:
> > > >> > > > >
> > > >> > > > > client.submit(someGremlinString)
> > > >> > > > >
> > > >> > > > > in any language. We really need to try to make that pattern
> go
> > > >> away
> > > >> > > > across
> > > >> > > > > the TinkerPop community.
> > > >> > > > >
> > > >> > > > > So - that's was a long email. Looking forward to hearing
> some
> > > >> > > discussion
> > > >> > > > on
> > > >> > > > > this.
> > > >> > > > >
> > > >> > > > > Stephen
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
