Hi, Perhaps this is also the time to think about custom steps and some way for providers to inject custom steps into the traversal. Currently the steps are all defined on `GraphTraversal`. Maybe providers can extend the interface with their functionality and if `graph.traversal()` returns the extended interface the custom steps will be available.
e.g. graph.traversal().streamVertex(...) Or if the traversal becomes session aware then perhaps, GraphTraversal gt = graph.traversal(); gt.bulkModeOn(); for 1 .. 1 000 000 gt.addV().next(); gt.V().specialRecipe().next(); gt.commit(); Just some thoughts Cheers Pieter On 01/12/2016 22:57, Marko Rodriguez wrote: > Ah. Yes. I concur Pieter. > > Then I think we need to get smart about “what data?” We sort of > already do with HaltedTraverserStrategy but this is really specific to > internal computing and GremlinServer. We could go deeper into this > path with: > > g.withDetachment(…) > - Detach.reference // just ids > - Detach.reduced // ids, labels, keys, values (basically, what > you need for toString()) > - Detach.rich // property data included > - Detached.full // edge data included (basically, StarGraph) > > > Next, GraphSON would have to specify a “subtype” to the various > g:Element types. > > { @type:”g:Vertex", @detach:”reference", @value: { id:1 } } > > > Then, we could add a method to Element. > > Element.detachment() -> Detach enum > > > This way, users can always know what is available to them. Finally for > cleanliness: > > Element.detach(Detach) -> Element > Element.attach(Graph) -> Element (always a Detach.full element) > > > Just spit-ballin’, > Marko. > > http://markorodriguez.com > > > >> On Dec 1, 2016, at 12:28 PM, pieter-gmail <pieter.mar...@gmail.com >> <mailto:pieter.mar...@gmail.com>> wrote: >> >> Hi, >> >> "So with ReferenceElements, latency will be less too because it takes >> less time to construct the ReferenceVertex than it does to construct a >> DetachedVertex. Imagine a vertex with 100 properties and meta >> properties. ?!" >> >> But ReferencedElement does not have the properties so more round trips >> are needed increasing latency. One of the first things to make Sqlg at >> all usable was to make sure that a Vertex contains all of its >> properties. Else at least one more call is needed per Vertex. Its a >> latency killer. For those mostly few cases where the Vertex is so fat >> that it is slow to load and only a few properties are needed then >> g.V().hasLabel("label").values("property1", "property2") is used. So to >> my mind ReferencedElement increases latency and does not decreases it. >> >> Using ReferencedElement which is hardly an Element at all, after all it >> throws exceptions on almost all of its own interface, the user has to >> get the properties manually and then is back in a world of Map and Lists >> of Maps. >> >> A refactor of much existing code will need to toss the Vertex notion all >> together and replace it with Maps and Lists of Maps. Almost like writing >> an application in pure JDBC code with thousands of lines iterating >> through ResultSets mapping things back and forth. Unless I am missing >> something this change seems huge. >> >> I get that all this is important for non java devs but it be a pity if >> their problems becomes java devs problems. >> >> Cheers >> Pieter >> >> >> On 01/12/2016 20:38, Marko Rodriguez wrote: >>> Hi, >>> >>> *PIETER REPLIES:* >>> >>>> One of the first reasons I came to graphs, Neo4j and then TinkerPop way >>>> back was precisely because of the direct access to Node/Vertex. The >>>> user >>>> treats it like any other object, not a remote connection. It is the >>>> embedded nature that makes life so easy. In a way it was like having a >>>> simplistic Hibernate as the core api. 99% of queries we write is to >>>> retrieve vertices. Not Maps and Lists of something. TinkerPop's own >>>> test >>>> suite applies this type of thinking. Querying/modifying Elements and >>>> asserting them. Vertex and Edge abound as first class citizens. >>> >>> So Graph/Vertex/Edge/VertexProperty/Property will still exist for >>> users as objects in the respective GLV language, it is just they are >>> not “attached” and “rich.” >>> >>> For instance, in Gremlin-Python, you have: >>> >>> v = g.V().next() >>> v.id >>> >>> A ReferenceVertex contains the id of the vertex so you can always >>> “re-attach” it to the source. >>> >>> g.V(v).out() >>> >>> >>>> Graph, Vertex and Edge is the primary abstraction that users deal with. >>>> Having the direct representation of this is very very nice. >>>> It makes user code easy and readable. You know you are dealing >>>> with the >>>> "Person/Address/Dog/This/That" entity/label as opposed to just a >>>> decontextualized bunch of data, Maps and Lists. If Vertex/Edge/Property >>>> were to disappear I'd say it would be the first call of duty to write a >>>> baby hibernate to bring the property model back in again into >>>> userspace. >>> >>> Again, the abstraction is still there, but just ALWAYS in a detached >>> form. >>> >>>> >>>> Regarding jdbc, this kinda makes the point. Sqlg and Hibernate and many >>>> many other tools exists precisely so that users do not need to use JDBC >>>> with endless hardcoded strings guiding the application. Making >>>> TinkerPop >>>> more like JDBC is not an obvious plus point. >>> >>> So, RemoteConnection differs from JDBC in that its not a fat string, >>> but RemoteConnection.submit(Bytecode). Thus, you still work at the >>> GraphTraversal level in every GLV. >>> >>>> A ReferencedElement is also no good as the problem I experience is >>>> latency not bandwidth. >>> >>> So with ReferenceElements, latency will be less too because it takes >>> less time to construct the ReferenceVertex than it does to construct a >>> DetachedVertex. Imagine a vertex with 100 properties and meta >>> properties. ?! >>> >>>> I reckon the experience and usage of TinkerPop is rather different for >>>> java and non java people and perhaps even java folks. Hopefully I >>>> am not >>>> the only one who have made such heavy happy use of the TinkerPop >>>> property meta model and would be sad to see it go. >>>> >>>> Cheers >>>> Pieter >>>> >>> >>> >>> *ROBERT REPLIES:* >>> >>>> I agree the focus should be on the Connection (being separate from >>>> Graph) and Traversal. I wouldn't constrain it to "RemoteConnection", >>>> just Connection or GraphConnection. Perhaps there's an >>>> EmbeddedConnection and a RemoteConnection or maybe it's URI-oriented >>>> similar to how JDBC does it. In either case, the behavior of Remote >>>> and Embedded is the same which is what I think we're striving for. >>> >>> Yes. Good point. Just Connection. >>> >>>> I would also like to see Transactions be Connection-oriented. With >>>> the right API, it could hook into JTA and be able to take advantage >>>> of various annotations for marking transaction boundaries. >>> >>> g = g.openTx() >>> g.V().out().out() >>> g.addV() >>> g.V(1).addE().to(2) >>> g.closeTx(); >>> >>> >>> ??? This way, its all about GraphTraversalSource/GraphTraversal. That >>> is truly the “connection” where the Connection implementation is just >>> provider/machine specific shuffling of Bytecode in and Traversers out. >>> >>>> Are there features of a lambda that couldn't be replaced by a more >>>> feature-rich gremlin? >>>> g.V().out('knows').map{it.get().value('name') + ' is the friend name'} >>>> g.V().out('knows').map(lambda(concat(__.it.get().value('name'), ' is >>>> the friend name’)) >>> >>> So we currently have the concept of g:Lambda and this allows for >>> lambdas to be used remotely. >>> >>> g.V().map(function(“it.get().label()”)) // Gremlin-Java traversal >>> with a Gremlin-Groovy lambda. >>> >>> >>> The crappy thing is that the lambda is always a String. >>> >>>> Reference-only makes total sense. This works really well especially >>>> with a local cache or for use cases where most of the data is stored >>>> in a separate database. I think it would lend itself nicely to lazy >>>> loading. When you need values there are options for that as well >>>> (properties/values/valueMap). One of the problems with 'attached' >>>> elements is you don't know what the implementation does. So >>>> potentially every get or set property call is going to the database >>>> and you don't realize it. That can hurt performance and have >>>> unintended consequences. >>> >>> Dude, I’ve been saying this forever. DetachedXXX is a bad idea for the >>> reasons you have stipulated. Just imagine: >>> >>> g.V(1).out(‘knows') >>> >>> >>> The GraphSON return is every vertex 1 knows and all its properties and >>> meta properties?!?! If you wanted that data too you would have queried >>> for it. >>> >>> Marko. >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Gremlin-users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to gremlin-users+unsubscr...@googlegroups.com >>> <mailto:gremlin-users+unsubscr...@googlegroups.com> >>> <mailto:gremlin-users+unsubscr...@googlegroups.com>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com >>> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer>. >>> For more options, visit https://groups.google.com/d/optout. >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Gremlin-users" group. >> To unsubscribe from this group and stop receiving emails from it, >> send an email to gremlin-users+unsubscr...@googlegroups.com >> <mailto:gremlin-users+unsubscr...@googlegroups.com>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/gremlin-users/79132fdd-f67f-5c3c-f8e3-87ab80f3c6f9%40gmail.com. >> For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google > Groups "Gremlin-users" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to gremlin-users+unsubscr...@googlegroups.com > <mailto:gremlin-users+unsubscr...@googlegroups.com>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/gremlin-users/F51985DD-7F40-45D7-BA18-FEC81E130099%40gmail.com > <https://groups.google.com/d/msgid/gremlin-users/F51985DD-7F40-45D7-BA18-FEC81E130099%40gmail.com?utm_medium=email&utm_source=footer>. > For more options, visit https://groups.google.com/d/optout.