> Perhaps this is also the time to think about custom steps and some way for
providers to inject custom steps into the traversal.
I'm not completelyclear on how custom steps, DSLs, etc should be properly
implemented. I think it would be important to have an approach for that
under this model and get it documented.
On Fri, Dec 2, 2016 at 6:49 AM, pieter-gmail <pieter.mar...@gmail.com>
wrote:
> Hi,
>
> Perhaps this is also the time to think about custom steps and some way
> for providers to inject custom steps into the traversal.
> Currently the steps are all defined on `GraphTraversal`. Maybe providers
> can extend the interface with their functionality and if
> `graph.traversal()` returns the extended interface the custom steps will
> be available.
>
> e.g.
>
> graph.traversal().streamVertex(...)
>
> Or if the traversal becomes session aware then perhaps,
>
> GraphTraversal gt = graph.traversal();
> gt.bulkModeOn();
> for 1 .. 1 000 000
> gt.addV().next();
>
> gt.V().specialRecipe().next();
> gt.commit();
>
> Just some thoughts
>
> Cheers
> Pieter
>
> On 01/12/2016 22:57, Marko Rodriguez wrote:
> > Ah. Yes. I concur Pieter.
> >
> > Then I think we need to get smart about “what data?” We sort of
> > already do with HaltedTraverserStrategy but this is really specific to
> > internal computing and GremlinServer. We could go deeper into this
> > path with:
> >
> > g.withDetachment(…)
> > - Detach.reference // just ids
> > - Detach.reduced // ids, labels, keys, values (basically, what
> > you need for toString())
> > - Detach.rich // property data included
> > - Detached.full // edge data included (basically, StarGraph)
> >
> >
> > Next, GraphSON would have to specify a “subtype” to the various
> > g:Element types.
> >
> > { @type:”g:Vertex", @detach:”reference", @value: { id:1 } }
> >
> >
> > Then, we could add a method to Element.
> >
> > Element.detachment() -> Detach enum
> >
> >
> > This way, users can always know what is available to them. Finally for
> > cleanliness:
> >
> > Element.detach(Detach) -> Element
> > Element.attach(Graph) -> Element (always a Detach.full element)
> >
> >
> > Just spit-ballin’,
> > Marko.
> >
> > http://markorodriguez.com
> >
> >
> >
> >> On Dec 1, 2016, at 12:28 PM, pieter-gmail <pieter.mar...@gmail.com
> >> <mailto:pieter.mar...@gmail.com>> wrote:
> >>
> >> Hi,
> >>
> >> "So with ReferenceElements, latency will be less too because it takes
> >> less time to construct the ReferenceVertex than it does to construct a
> >> DetachedVertex. Imagine a vertex with 100 properties and meta
> >> properties. ?!"
> >>
> >> But ReferencedElement does not have the properties so more round trips
> >> are needed increasing latency. One of the first things to make Sqlg at
> >> all usable was to make sure that a Vertex contains all of its
> >> properties. Else at least one more call is needed per Vertex. Its a
> >> latency killer. For those mostly few cases where the Vertex is so fat
> >> that it is slow to load and only a few properties are needed then
> >> g.V().hasLabel("label").values("property1", "property2") is used. So to
> >> my mind ReferencedElement increases latency and does not decreases it.
> >>
> >> Using ReferencedElement which is hardly an Element at all, after all it
> >> throws exceptions on almost all of its own interface, the user has to
> >> get the properties manually and then is back in a world of Map and Lists
> >> of Maps.
> >>
> >> A refactor of much existing code will need to toss the Vertex notion all
> >> together and replace it with Maps and Lists of Maps. Almost like writing
> >> an application in pure JDBC code with thousands of lines iterating
> >> through ResultSets mapping things back and forth. Unless I am missing
> >> something this change seems huge.
> >>
> >> I get that all this is important for non java devs but it be a pity if
> >> their problems becomes java devs problems.
> >>
> >> Cheers
> >> Pieter
> >>
> >>
> >> On 01/12/2016 20:38, Marko Rodriguez wrote:
> >>> Hi,
> >>>
> >>> *PIETER REPLIES:*
> >>>
> >>>> One of the first reasons I came to graphs, Neo4j and then TinkerPop
> way
> >>>> back was precisely because of the direct access to Node/Vertex. The
> >>>> user
> >>>> treats it like any other object, not a remote connection. It is the
> >>>> embedded nature that makes life so easy. In a way it was like having a
> >>>> simplistic Hibernate as the core api. 99% of queries we write is to
> >>>> retrieve vertices. Not Maps and Lists of something. TinkerPop's own
> >>>> test
> >>>> suite applies this type of thinking. Querying/modifying Elements and
> >>>> asserting them. Vertex and Edge abound as first class citizens.
> >>>
> >>> So Graph/Vertex/Edge/VertexProperty/Property will still exist for
> >>> users as objects in the respective GLV language, it is just they are
> >>> not “attached” and “rich.”
> >>>
> >>> For instance, in Gremlin-Python, you have:
> >>>
> >>> v = g.V().next()
> >>> v.id
> >>>
> >>> A ReferenceVertex contains the id of the vertex so you can always
> >>> “re-attach” it to the source.
> >>>
> >>> g.V(v).out()
> >>>
> >>>
> >>>> Graph, Vertex and Edge is the primary abstraction that users deal
> with.
> >>>> Having the direct representation of this is very very nice.
> >>>> It makes user code easy and readable. You know you are dealing
> >>>> with the
> >>>> "Person/Address/Dog/This/That" entity/label as opposed to just a
> >>>> decontextualized bunch of data, Maps and Lists. If
> Vertex/Edge/Property
> >>>> were to disappear I'd say it would be the first call of duty to write
> a
> >>>> baby hibernate to bring the property model back in again into
> >>>> userspace.
> >>>
> >>> Again, the abstraction is still there, but just ALWAYS in a detached
> >>> form.
> >>>
> >>>>
> >>>> Regarding jdbc, this kinda makes the point. Sqlg and Hibernate and
> many
> >>>> many other tools exists precisely so that users do not need to use
> JDBC
> >>>> with endless hardcoded strings guiding the application. Making
> >>>> TinkerPop
> >>>> more like JDBC is not an obvious plus point.
> >>>
> >>> So, RemoteConnection differs from JDBC in that its not a fat string,
> >>> but RemoteConnection.submit(Bytecode). Thus, you still work at the
> >>> GraphTraversal level in every GLV.
> >>>
> >>>> A ReferencedElement is also no good as the problem I experience is
> >>>> latency not bandwidth.
> >>>
> >>> So with ReferenceElements, latency will be less too because it takes
> >>> less time to construct the ReferenceVertex than it does to construct a
> >>> DetachedVertex. Imagine a vertex with 100 properties and meta
> >>> properties. ?!
> >>>
> >>>> I reckon the experience and usage of TinkerPop is rather different for
> >>>> java and non java people and perhaps even java folks. Hopefully I
> >>>> am not
> >>>> the only one who have made such heavy happy use of the TinkerPop
> >>>> property meta model and would be sad to see it go.
> >>>>
> >>>> Cheers
> >>>> Pieter
> >>>>
> >>>
> >>>
> >>> *ROBERT REPLIES:*
> >>>
> >>>> I agree the focus should be on the Connection (being separate from
> >>>> Graph) and Traversal. I wouldn't constrain it to "RemoteConnection",
> >>>> just Connection or GraphConnection. Perhaps there's an
> >>>> EmbeddedConnection and a RemoteConnection or maybe it's URI-oriented
> >>>> similar to how JDBC does it. In either case, the behavior of Remote
> >>>> and Embedded is the same which is what I think we're striving for.
> >>>
> >>> Yes. Good point. Just Connection.
> >>>
> >>>> I would also like to see Transactions be Connection-oriented. With
> >>>> the right API, it could hook into JTA and be able to take advantage
> >>>> of various annotations for marking transaction boundaries.
> >>>
> >>> g = g.openTx()
> >>> g.V().out().out()
> >>> g.addV()
> >>> g.V(1).addE().to(2)
> >>> g.closeTx();
> >>>
> >>>
> >>> ??? This way, its all about GraphTraversalSource/GraphTraversal. That
> >>> is truly the “connection” where the Connection implementation is just
> >>> provider/machine specific shuffling of Bytecode in and Traversers out.
> >>>
> >>>> Are there features of a lambda that couldn't be replaced by a more
> >>>> feature-rich gremlin?
> >>>> g.V().out('knows').map{it.get().value('name') + ' is the friend
> name'}
> >>>> g.V().out('knows').map(lambda(concat(__.it.get().value('name'), ' is
> >>>> the friend name’))
> >>>
> >>> So we currently have the concept of g:Lambda and this allows for
> >>> lambdas to be used remotely.
> >>>
> >>> g.V().map(function(“it.get().label()”)) // Gremlin-Java traversal
> >>> with a Gremlin-Groovy lambda.
> >>>
> >>>
> >>> The crappy thing is that the lambda is always a String.
> >>>
> >>>> Reference-only makes total sense. This works really well especially
> >>>> with a local cache or for use cases where most of the data is stored
> >>>> in a separate database. I think it would lend itself nicely to lazy
> >>>> loading. When you need values there are options for that as well
> >>>> (properties/values/valueMap). One of the problems with 'attached'
> >>>> elements is you don't know what the implementation does. So
> >>>> potentially every get or set property call is going to the database
> >>>> and you don't realize it. That can hurt performance and have
> >>>> unintended consequences.
> >>>
> >>> Dude, I’ve been saying this forever. DetachedXXX is a bad idea for the
> >>> reasons you have stipulated. Just imagine:
> >>>
> >>> g.V(1).out(‘knows')
> >>>
> >>>
> >>> The GraphSON return is every vertex 1 knows and all its properties and
> >>> meta properties?!?! If you wanted that data too you would have queried
> >>> for it.
> >>>
> >>> Marko.
> >>> --
> >>> You received this message because you are subscribed to the Google
> >>> Groups "Gremlin-users" group.
> >>> To unsubscribe from this group and stop receiving emails from it, send
> >>> an email to gremlin-users+unsubscr...@googlegroups.com
> >>> <mailto:gremlin-users+unsubscr...@googlegroups.com>
> >>> <mailto:gremlin-users+unsubscr...@googlegroups.com>.
> >>> To view this discussion on the web visit
> >>> https://groups.google.com/d/msgid/gremlin-users/7CBD403D-
> 4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com
> >>> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-
> 4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer
> >.
> >>> For more options, visit https://groups.google.com/d/optout.
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> >> Groups "Gremlin-users" group.
> >> To unsubscribe from this group and stop receiving emails from it,
> >> send an email to gremlin-users+unsubscr...@googlegroups.com
> >> <mailto:gremlin-users+unsubscr...@googlegroups.com>.
> >> To view this discussion on the web visit
> >> https://groups.google.com/d/msgid/gremlin-users/79132fdd-
> f67f-5c3c-f8e3-87ab80f3c6f9%40gmail.com.
> >> For more options, visit https://groups.google.com/d/optout.
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Gremlin-users" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> > an email to gremlin-users+unsubscr...@googlegroups.com
> > <mailto:gremlin-users+unsubscr...@googlegroups.com>.
> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/gremlin-users/F51985DD-
> 7F40-45D7-BA18-FEC81E130099%40gmail.com
> > <https://groups.google.com/d/msgid/gremlin-users/F51985DD-
> 7F40-45D7-BA18-FEC81E130099%40gmail.com?utm_medium=email&utm_source=footer
> >.
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to gremlin-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/gremlin-users/7dc4ff30-0730-46ef-befc-ae8d2bf36e8d%40gmail.com.
> For more options, visit https://groups.google.com/d/optout.
>
