RE: io and graphson-v3
This v3 looks like a <3 heart where the top broke off. Sad. -Original Message- From: Stephen Mallette [mailto:spmalle...@gmail.com] Sent: Wednesday, September 06, 2017 1:45 PM To: dev@tinkerpop.apache.org Subject: Re: io and graphson-v3 never mind - i see what's messed up On Wed, Sep 6, 2017 at 1:43 PM, Stephen Mallettewrote: > Which tests in particular are failing for SerializationTest? all of them? > > On Wed, Sep 6, 2017 at 1:35 PM, pieter gmail > wrote: > >> Hi, >> >> Pulled TINKERPOP-1767 branch, changed SqlgGraph's io method and ran >> the tests. >> >> All the io tests are passing. >> Only SerializationTest fails for the same reason. It too needs the >> version specified. I did that locally and then all tests passes. >> >> Thanks >> Pieter >> >> On 06/09/2017 18:09, Stephen Mallette wrote: >> >>> Pieter, I created this issue: >>> >>> https://issues.apache.org/jira/browse/TINKERPOP-1767 >>> >>> and made an effort to try to figure a way to fix it: >>> >>> https://github.com/apache/tinkerpop/tree/TINKERPOP-1767 >>> >>> Note the change to TinkerGraph and its io() method. I suppose you >>> could do something similar to get the right registry in play? could >>> you have a look and see if what i did helps? if that works then i'll >>> issue a PR and we can get it reviewed/merged. >>> >>> >>> On Tue, Sep 5, 2017 at 12:10 PM, pieter gmail >>> >>> wrote: >>> >>> Ok, at present there is only one SimpleModule, the default. I can >>> make it v2 or v3 but not both. Lets say I make the SimpleModule support V2. Then when calling IoEdgeTest for {"graphson-v3", true, true, (Function ) g -> g.io (IoCore.graphson()).reader ().mapper(g.io(GraphSONIo.build(GraphSONVersion.V3_0)). mapper().create()).create(), (Function ) g -> g.io (IoCore.graphson()).writer ().mapper(g.io(GraphSONIo.build(GraphSONVersion.V3_0)). mapper().create()).create()}, then the deserializers that run are for V2, static class SchemaTableIdJacksonDeserializerV2d0 extends AbstractObjectDeserializer { SchemaTableIdJacksonDeserializerV2d0() { super(SchemaTable.class); } @Override public SchemaTable createObject(final Map data) { return SchemaTable.of((String)data.get("schema"), (String) data.get("table")); } } when createObject fires the map data has V3 elements in it like @type and @value whilst its expecting "schema" and "table" If we make the SimpleModule support V3 then graphson-v2 will fail. {"graphson-v2", false, false, (Function ) g -> g.io (IoCore.graphson()).reader ().mapper(g.io(GraphSONIo.build(GraphSONVersion.V2_0)). mapper().typeInfo(TypeInfo.NO_TYPES).create()).create(), (Function ) g -> g.io (IoCore.graphson()).writer ().mapper(g.io(GraphSONIo.build(GraphSONVersion.V2_0)). mapper().typeInfo(TypeInfo.NO_TYPES).create()).create()}, Now the deserializers are for V3. static class SchemaTableJacksonDeserializerV3d0 extends StdDeserializer { public SchemaTableJacksonDeserializerV3d0() { super(RecordId.class); } @Override public SchemaTable deserialize(final JsonParser jsonParser, final DeserializationContext deserializationContext) throws IOException, JsonProcessingException { final Map data = deserializationContext.readValue(jsonParser, Map.class); return SchemaTable.of((String)data.get("schema"), (String) data.get("table")); } @Override public boolean isCachable() { return true; } } This does not fire at all. Eventually I get a detached edge with an id that is a map. It never deserialized. So basically it only works if the SimpleModule version, i.e. serialize/deserialize code matches up with the version. Sqlg serializes RecordId to, static class RecordIdJacksonSerializerV3d0 extends StdScalarSerializer { public RecordIdJacksonSerializerV3d0() { super(RecordId.class); } @Override public void serialize(final RecordId recordId, final JsonGenerator jsonGenerator, final SerializerProvider serializerProvider) throws IOException, JsonGenerationException { final Map m = new HashMap<>(); m.put("schemaTable", recordId.getSchemaTable());
RE: Requirements for listing as a vendor
Hi Robert, Sorry for the delayed response. Here's the verbiage we would like to use. Pitney Bowes Spectrum Data Hub Module<http://www.pitneybowes.com/us/customer-information-management/data-integration-management/spectrum-data-hub-module.html> – Uses Gremlin OLTP to query Neo4j-powered MDM graph DB. Do you have room for a logo? If another size is needed, I can provide one. [http://bizwire.tekgroup.com/media/95/20150114105005209pb_hor_4cp_grd_pos.jpg]<http://bizwire.tekgroup.com/media/95/20150114105005209pb_hor_4cp_grd_pos.jpg> Thanks, -Paul From: Robert Dale [mailto:robd...@gmail.com] Sent: Thursday, April 20, 2017 3:38 PM To: dev@tinkerpop.apache.org; Paul A. Jackson <paul.jack...@pb.com> Subject: Re: Requirements for listing as a vendor Paul, We have come up with a format for the Powered By listing. It can be "Company Product" or just "Product name" link and then some optional summary text. The max length of the whole line is 100 chars. For example, Pitney Bowes Spectrum Data Hub Module<http://www.pitneybowes.com/us/customer-information-management/data-integration-management/spectrum-data-hub-module.html> - uses Gremlin in some short way or Spectrum Data Hub Module<http://www.pitneybowes.com/us/customer-information-management/data-integration-management/spectrum-data-hub-module.html> - uses Gremlin in much bigger and longer and uncut ways Let us know what you would like on there. -- Robert Dale
RE: [DISCUSS] ASF Board Draft Report - April 2017
Is this a forum for the "Powered By" concept? Thanks, -Paul -Original Message- From: Stephen Mallette [mailto:spmalle...@gmail.com] Sent: Tuesday, April 04, 2017 1:42 PM To: dev@tinkerpop.apache.org Subject: [DISCUSS] ASF Board Draft Report - April 2017 Here's a draft of the board report for April. Please let me know if you think there's anything else we might add. Note that there is one outstanding issue that might make it on here related to PMC membership - we'll see if that shakes out before this needs to be submitted to the board. ## Description: Apache TinkerPop is a graph computing framework for both graph databases (OLTP) and graph analytic systems (OLAP). ## Activity: The TinkerPop PMC voted positively to add a new committer, but that person declined when invited, preferring instead to continue to contribute as they have been. TinkerPop released versions 3.1.6 and 3.2.5. New development has started on the follow-on releases to those versions and work for the new major line of 3.3.0 continues. The wider TinkerPop community saw some growth with the annoucement of the release of ChronoGraph[1], a versioned graph database that implements the TinkerPop interfaces. Also of note, JanusGraph[2] (formerly Titan) is under development at the Linux Foundation. JanusGraph continues in the steps of Titan in also implementing the TinkerPop interfaces. There were also a number of talks given related to TinkerPop and the Gremlin language. Jason Plurad, a PMC member, gave one such talk[3]. ## Issues: There are no issues requiring board attention at this time. ## Releases: - 3.1.6 (February 8, 2017) - 3.2.4 (February 8, 2017) ## PMC/Committer: - Last PMC addition was Jason Plurad - August 2016 - Last committer addition was Robert Dale - October 2016 ## Links [1] https://github.com/MartinHaeusler/chronos/ [2] http://janusgraph.org/ [3] https://www.youtube.com/watch?v=7kTqqvBzL60
RE: Requirements for listing as a vendor
This sounds great. At the same time, though, I was looking at other providers and judging from architecture diagrams (eg. https://grakn.ai/pages/documentation/the-fundamentals/grakn-architecture.html) some look similar to ours. It may be the case that our web site is directed more towards business types than developers, is available through direct sales only, etc, rather than architectural differences? -Paul -Original Message- From: Marko Rodriguez [mailto:okramma...@gmail.com] Sent: Thursday, March 16, 2017 4:13 PM To: dev@tinkerpop.apache.org Subject: Re: Requirements for listing as a vendor I say we add a “Powered By” section to the website and list “leveraging systems” as I wouldn’t call this a “TinkerPop Provider” like I would a more “low level” graph database system…. If we open up “Powered By” it would allow us to show people the types of products, projects, nick nacks and doo dads that are out there leveraging TinkerPizzzop. Marko. > On Mar 16, 2017, at 1:56 PM, Paul A. Jackson <paul.jack...@pb.com> wrote: > > I agree that the last bullet is not met. We bundle the Tinkerpop stack, so > the idea of version compatibility doesn't make sense. Instead, we offer a > platform that builds an ecosystem around the graph (ETL, fuzzy matching, rich > analysis client, etc). One feature is graph query. The users can supply their > own Gremlin queries, or they can use our graphical query builder, which > generates Gremlin. > > Anywho, what do you think? I concur we do not fit in the provider bucket. But > we have been around for 5 years. The offering is pretty mature at this point. > > -Paul > > > > -Original Message- > From: Robert Dale [mailto:robd...@gmail.com] > Sent: Thursday, March 16, 2017 1:50 PM > To: dev@tinkerpop.apache.org > Subject: Re: Requirements for listing as a vendor > > You should start here: http://tinkerpop.apache.org/policy.html > > It's not obvious to me that the last 3 bullets are met. > > Robert Dale > > On Thu, Mar 16, 2017 at 12:56 PM, Paul A. Jackson > <paul.jack...@pb.com> > wrote: > >> Hi, >> >> What are the requirements for being lists as a Gremlin provider? We >> (Pitney Bowes) have an integrated product offering called Spectrum >> Data Hub (you'll also see it listed as part of the Spectrum MDM >> solution). The product is its own application server and we embed >> Gremlin on top of embedded Neo4j using our own "blueprints" adapter. >> So, access to the graph is from our own apps, rather that via a Gremlin >> console or server. >> >> I don't know if this makes us a special case or not. Can you let us >> know if we qualify and/or if there's anything we would need to do so? >> >> Here's a link to the product literature: >> http://www.pitneybowes.com/us/ >> customer-information-management/data-integration- >> management/spectrum-data-hub-module.html >> >> Thanks, >> -Paul >> >> >> >> > > >
RE: Requirements for listing as a vendor
I agree that the last bullet is not met. We bundle the Tinkerpop stack, so the idea of version compatibility doesn't make sense. Instead, we offer a platform that builds an ecosystem around the graph (ETL, fuzzy matching, rich analysis client, etc). One feature is graph query. The users can supply their own Gremlin queries, or they can use our graphical query builder, which generates Gremlin. Anywho, what do you think? I concur we do not fit in the provider bucket. But we have been around for 5 years. The offering is pretty mature at this point. -Paul -Original Message- From: Robert Dale [mailto:robd...@gmail.com] Sent: Thursday, March 16, 2017 1:50 PM To: dev@tinkerpop.apache.org Subject: Re: Requirements for listing as a vendor You should start here: http://tinkerpop.apache.org/policy.html It's not obvious to me that the last 3 bullets are met. Robert Dale On Thu, Mar 16, 2017 at 12:56 PM, Paul A. Jackson <paul.jack...@pb.com> wrote: > Hi, > > What are the requirements for being lists as a Gremlin provider? We > (Pitney Bowes) have an integrated product offering called Spectrum > Data Hub (you'll also see it listed as part of the Spectrum MDM > solution). The product is its own application server and we embed > Gremlin on top of embedded Neo4j using our own "blueprints" adapter. > So, access to the graph is from our own apps, rather that via a Gremlin > console or server. > > I don't know if this makes us a special case or not. Can you let us > know if we qualify and/or if there's anything we would need to do so? > > Here's a link to the product literature: > http://www.pitneybowes.com/us/ > customer-information-management/data-integration- > management/spectrum-data-hub-module.html > > Thanks, > -Paul > > > >
Requirements for listing as a vendor
Hi, What are the requirements for being lists as a Gremlin provider? We (Pitney Bowes) have an integrated product offering called Spectrum Data Hub (you'll also see it listed as part of the Spectrum MDM solution). The product is its own application server and we embed Gremlin on top of embedded Neo4j using our own "blueprints" adapter. So, access to the graph is from our own apps, rather that via a Gremlin console or server. I don't know if this makes us a special case or not. Can you let us know if we qualify and/or if there's anything we would need to do so? Here's a link to the product literature: http://www.pitneybowes.com/us/customer-information-management/data-integration-management/spectrum-data-hub-module.html Thanks, -Paul
RE: [GitHub] tinkerpop issue #567: TINKERPOP-1644 Improve script compilation syncronisati...
This code is using the double-checked locking idiom, which does not work in Java: Class clazz = getScriptClassFromMap(script); if (clazz != null) { return clazz; } synchronized (this) { clazz = getScriptClassFromMap(script); if (clazz != null) { return clazz; } See https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html I don't know the details well enough to judge whether the failure modes will be harmful in this case. I think it's best avoided or replaced with one of the suggestions in the link above. -Paul -Original Message- From: spmallette [mailto:g...@git.apache.org] Sent: Tuesday, March 07, 2017 6:25 AM To: dev@tinkerpop.apache.org Subject: [GitHub] tinkerpop issue #567: TINKERPOP-1644 Improve script compilation syncronisati... Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/567 I will clean up a few code formatting things on merge, but this looks good to me. All tests pass with `docker/build.sh -t -n -i` VOTE +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
RE: [DISCUSS] P.not and __.not
If you'll forgive "thinking out loud," is there any way to create a wrapper-not that implements both interfaces and just works? -Original Message- From: Stephen Mallette [mailto:spmalle...@gmail.com] Sent: Friday, February 24, 2017 2:35 PM To: dev@tinkerpop.apache.org Subject: [DISCUSS] P.not and __.not P.not and __.not unfortunately tangle with each other when using static imports. I think we allowed the console to dictate to us that P.not is the lucky one that gets to be used without its qualifying prefix. I'm not sure there was any conscious decision to do it that way. Indeed, I think i would prefer getting __.not over P.not. I also think that the behavior is sort of random that we get P.not rather than __.not (for reasons I can go into in more detail if anyone cares). Anyway, I'd like to resolve this issue in 3.3.0. I think that immediately, I can introduce the breaking change to the console that explicitly imports __.not rather than P.not - this will remove randomness. In the longer term we can deprecate P.not and either drop it all together or rename it. I'm not sure how strongly folks feel about usage of P.not so I guess I'll just open it up for discussion as to what the long term fix should be here. If we don't develop any real consensus here for the longer term option I will just create an issue in JIRA and it can be dealt with later. I'm mostly interested in getting a short-term solution in place to solve some problems I'm facing right now. Thanks, Stephen
RE: [VOTE] TinkerPop 3.2.4 Release
I'm looking for some insight for when 3.2.4 will release. It looks like voting closes tomorrow. Is that on schedule, and if so what's a typical ETA for binaries to be posted? I don't know if my vote counts, but FTW! VOTE: +1 Thanks, -Paul -Original Message- From: Stephen Mallette [mailto:spmalle...@gmail.com] Sent: Friday, February 10, 2017 7:03 AM To: dev@tinkerpop.apache.org Subject: Re: [VOTE] TinkerPop 3.2.4 Release validate-distribution.sh was good for me - thanks for doing all the work on this one Jason. nice job VOTE: +1 On Fri, Feb 10, 2017 at 1:22 AM, Ted Wilmeswrote: > Release and docs looks good to me. > > Validating source distribution > > * downloading Apache TinkerPop 3.2.4 (apache-tinkerpop-3.2.4-src.zip)... > OK > * validating signatures and checksums ... > * PGP signature ... OK > * MD5 checksum ... OK > * SHA1 checksum ... OK > * unzipping Apache TinkerPop 3.2.4 ... OK > * building project ... OK > > > VOTE: +1 > > Thanks, > Ted > > On Thu, Feb 9, 2017 at 12:32 PM, pieter-gmail > > wrote: > > > Ok, however I did test on the 3.2.4-SNAPSHOT immediately after > > Jason's email on the 2/2/2017 and those changes were not there. > > They are there now but there was a SNAPSHOT release on the > > 08/02/2017 so things changed. > > Anyhow that might just be some SNAPSHOT confusion thing. > > > > Next time I'll pull the code and build it manually to make sure. > > > > Thanks > > Pieter > > > > > > > > On 09/02/2017 22:27, Marko Rodriguez wrote: > > > Hi, > > > > > >> The significant change is > > >> ffe1b4c Marko A. Rodriguez on 2016/11/15 > > >> at > > 12:44 AM > > > Ah yea. Thats different from what I thought you had issue with — > > has-containers and arrays/collections handling. > > > > > >> I don't have a problem with the changes just that is quite a > > >> refactor > on > > >> my side as it changes the structure of the HasSteps and its > > >> HasContainers. I had made some assumptions around how HasSteps > > >> and HasContainers look when optimizing. > > > Yea, it “folds left” now so you don’t have to walk over has()-chains. > > > > > >> The change is quite old but somehow it was not present in the > > >> 3.2.4-SNAPSHOT I tested on it when the code freeze announcement > > >> was made. Not sure what happened there but alas its not giving me > > >> enough time to get things working again. > > > I would say don’t wait till releases to test Sqlg. In principle, > > > you > > should VOTE on every PR by building and testing the changes against Sqlg. > > That is where you can make a huge contribution. > > > > > >> So I am not voting negative just requesting a weekend, if > > >> possible, to get through the refactor. > > > Again, PR awareness, not release awareness. > > > > > > Marko. > > > > > > http://markorodriguez.com > > > > > > > > >> Cheers > > >> Pieter > > >> > > >> > > >> > > >> > > >> > > >> "added TraversalHelper.addHasContainer() on 2016/11/15 > > >> > > >> On 09/02/2017 22:11, Stephen Mallette wrote: > > >>> Unless I'm missing something, HasStep hasn't changed in 4 months > > >>> and HasContainer hasn't changed in 3 months. The only update > > >>> that went in > > after > > >>> 2/2 that I can think of that would have any bearing for graph > > providers who > > >>> tested before/after that date would be the AutoCloseable stuff: > > >>> > > >>> https://github.com/apache/tinkerpop/pull/548 > > >>> > > >>> Your problems aren't related to that are they? Can you provide > > >>> some synopsis of where the problems lie? > > >>> > > >>> > > >>> > > >>> On Thu, Feb 9, 2017 at 2:47 PM, pieter-gmail < > pieter.mar...@gmail.com> > > >>> wrote: > > >>> > > Hi, > > > > Some issues regarding the release process. > > > > On the 2/2/2017 the 3.2.4-SNAPSHOT was released. > > I then started testing and found almost no issues. > > > > However yesterday when the VOTE mail came I found many issues > > on > > 3.2.4. > > To understand the confusion I tested again on 3.2.4-SNAPSHOT > > and > found > > the same new issues. > > I then checked the 3.2.4-SNAPSHOT timestamp and it changed to > > 08/02/2017 > > > > Not sure what happened there as I can not, nor is it worth it, > > check > > an > > old SNAPSHOT version's binary. > > The relevant code changes are quite old (December 2016) but it > > may > > have > > been done on a separate branch and rebasing may loose the > > merging information. Not sure about this though. > > > > This only leaves the 72 hours to catch up. > > > > Even though it is a minor release there has been significant > > changes > > to > > HasStep and HasContainer which breaks the heart (mine at least) > > of implementors optimization code. > > > > Basically to get any value from a vote I for one will need more > time. > > > > So far I have
RE: [GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced CloseableIterator
I am git illiterate. I committed to my repo. It looks like that's all I have to do. Travis build tells me to hang tight. It knows the commit is related to an open pull request and your repo knows to pull it in automatically? That's cool, if true. -Paul -Original Message- From: Paul A. Jackson [mailto:paul.jack...@pb.com] Sent: Friday, January 27, 2017 2:24 PM To: dev@tinkerpop.apache.org Subject: RE: [GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced CloseableIterator Hi Stephen, I think there are two parts to this, and I may have responded to the wrong part. I can easily and cleanly make those subclasses implement AutoCloseable and remove that from FlatMapStep. Implementing AutoCloseable is needed for the second half of the problem, where a traversal is closed before it is read to completion. This is easily delegated to the subclasses. There's the first half of the problem dealing with closing iterators as FlatMapStep finishes with them, but before it loses its reference to them. This is very hard to remove from FlatMapStep without duplicating all the code in the subclasses. If you consent I'd like to update the pull request that removes AutoCloseable from FlatMapStep but keeps the closing of iterators on the fly in FlatMapStep. I'd can also move that try/catch stuff to a helper method (in CloseableInterface?). That keeps FlatMapStep very clean. Thoughts? public abstract class FlatMapStep<S, E> extends AbstractStep<S, E> { private Traverser.Admin head = null; protected Iterator iterator = EmptyIterator.instance(); // Made protected public FlatMapStep(final Traversal.Admin traversal) { super(traversal); } @Override protected Traverser.Admin processNextStart() { while (true) { if (this.iterator.hasNext()) { return this.head.split(this.iterator.next(), this); } else { this.head = this.starts.next(); closeIterator();// Added this.iterator = this.flatMap(this.head); } } } protected abstract Iterator flatMap(final Traverser.Admin traverser); @Override public void reset() { super.reset(); closeIterator();// Added this.iterator = EmptyIterator.instance(); } void closeIterator() { CloseableIterator.closeIterator(this.iterator); } } Thanks, -Paul -Original Message- From: Paul A. Jackson [mailto:paul.jack...@pb.com] Sent: Friday, January 27, 2017 1:55 PM To: dev@tinkerpop.apache.org Subject: RE: [GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced CloseableIterator I can do this, and I can see why you are suggesting it, but I didn't originally because it means overriding all the methods from FlatMapStep into these subclasses. The way it's currently designed all the subclasses need to do is implement flatMap(). FlatMapStep has private access to iterator and head. It knows when it is done with an iterator, that when it's done with an iterator that it should replace it with an EmptyIterator, that if it's not done it needs to call head.split with the next element, etc. To implement what you suggest, correct me if I'm wrong, all that private knowledge needs to be duplicated in the subclasses. If feels like a bad design because if any of those behaviors change in FlatMapStep we need to remember to go into the subclasses and mirror the changes there. Maybe I should move the try/catch stuff to a helper method so that all FlatMapStep needs to do is execute one line when it's done with an iterator? Should I submit parallel pull request so you can compare/contrast? -Paul -Original Message- From: spmallette [mailto:g...@git.apache.org] Sent: Friday, January 27, 2017 1:23 PM To: dev@tinkerpop.apache.org Subject: [GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced CloseableIterator Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/548 I'm pretty sure @okram was affirming that `VertexStep`, `EdgeVertexStep` and `PropertiesStep` should implement `AutoCloseable` (I don't think that there are others) rather than a blanket change of just applying it to `FlatMapStep`. As the logic is re-used, perhaps you could do an `ElementStep` that extends the `FlatMapStep` and implements `AutoCloseable`. Then have those three steps extend `ElementStep` which would contain the close logic that you have. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
RE: [GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced CloseableIterator
Hi Stephen, I think there are two parts to this, and I may have responded to the wrong part. I can easily and cleanly make those subclasses implement AutoCloseable and remove that from FlatMapStep. Implementing AutoCloseable is needed for the second half of the problem, where a traversal is closed before it is read to completion. This is easily delegated to the subclasses. There's the first half of the problem dealing with closing iterators as FlatMapStep finishes with them, but before it loses its reference to them. This is very hard to remove from FlatMapStep without duplicating all the code in the subclasses. If you consent I'd like to update the pull request that removes AutoCloseable from FlatMapStep but keeps the closing of iterators on the fly in FlatMapStep. I'd can also move that try/catch stuff to a helper method (in CloseableInterface?). That keeps FlatMapStep very clean. Thoughts? public abstract class FlatMapStep<S, E> extends AbstractStep<S, E> { private Traverser.Admin head = null; protected Iterator iterator = EmptyIterator.instance(); // Made protected public FlatMapStep(final Traversal.Admin traversal) { super(traversal); } @Override protected Traverser.Admin processNextStart() { while (true) { if (this.iterator.hasNext()) { return this.head.split(this.iterator.next(), this); } else { this.head = this.starts.next(); closeIterator();// Added this.iterator = this.flatMap(this.head); } } } protected abstract Iterator flatMap(final Traverser.Admin traverser); @Override public void reset() { super.reset(); closeIterator();// Added this.iterator = EmptyIterator.instance(); } void closeIterator() { CloseableIterator.closeIterator(this.iterator); } } Thanks, -Paul -Original Message- From: Paul A. Jackson [mailto:paul.jack...@pb.com] Sent: Friday, January 27, 2017 1:55 PM To: dev@tinkerpop.apache.org Subject: RE: [GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced CloseableIterator I can do this, and I can see why you are suggesting it, but I didn't originally because it means overriding all the methods from FlatMapStep into these subclasses. The way it's currently designed all the subclasses need to do is implement flatMap(). FlatMapStep has private access to iterator and head. It knows when it is done with an iterator, that when it's done with an iterator that it should replace it with an EmptyIterator, that if it's not done it needs to call head.split with the next element, etc. To implement what you suggest, correct me if I'm wrong, all that private knowledge needs to be duplicated in the subclasses. If feels like a bad design because if any of those behaviors change in FlatMapStep we need to remember to go into the subclasses and mirror the changes there. Maybe I should move the try/catch stuff to a helper method so that all FlatMapStep needs to do is execute one line when it's done with an iterator? Should I submit parallel pull request so you can compare/contrast? -Paul -Original Message- From: spmallette [mailto:g...@git.apache.org] Sent: Friday, January 27, 2017 1:23 PM To: dev@tinkerpop.apache.org Subject: [GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced CloseableIterator Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/548 I'm pretty sure @okram was affirming that `VertexStep`, `EdgeVertexStep` and `PropertiesStep` should implement `AutoCloseable` (I don't think that there are others) rather than a blanket change of just applying it to `FlatMapStep`. As the logic is re-used, perhaps you could do an `ElementStep` that extends the `FlatMapStep` and implements `AutoCloseable`. Then have those three steps extend `ElementStep` which would contain the close logic that you have. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
RE: [DISCUSS] Release 3.2.4 and 3.1.6
OK, that's done, as you can no doubt see. Hoping this can make it into 3.2.4. Thanks. -Paul -Original Message- From: Stephen Mallette [mailto:spmalle...@gmail.com] Sent: Wednesday, January 25, 2017 6:13 PM To: dev@tinkerpop.apache.org Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6 sure - please put that in the prefix of the PR title so it links back to that original jira ticket On Wed, Jan 25, 2017 at 5:17 PM, Paul A. Jackson <paul.jack...@pb.com> wrote: > OK, low-risk PR coming for OLTP part. Should I reuse TINKERPOP-1589? > > Thanks, > -Paul > > -Original Message- > From: Stephen Mallette [mailto:spmalle...@gmail.com] > Sent: Wednesday, January 25, 2017 5:07 PM > To: dev@tinkerpop.apache.org > Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6 > > Being as close as we are to our code freeze/test week, I'd say that > big, complex or otherwise risky changes probably won't collect too > many +1 reviews at this point. If the OLTP improvement is > small/concise (low risk), it could be considered for inclusion in 3.2.4. > > On Wed, Jan 25, 2017 at 5:00 PM, Paul A. Jackson <paul.jack...@pb.com> > wrote: > > > I have something that fixes OLTP. I haven't worked with OLAP and it > > looks like the changes for this will be extensive, touching > > IteratorUtils and so on. > > > > Would you be interested in a PR for just the OLTP part? > > > > > > -Original Message- > > From: Stephen Mallette [mailto:spmalle...@gmail.com] > > Sent: Wednesday, January 25, 2017 3:58 PM > > To: dev@tinkerpop.apache.org > > Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6 > > > > in his case, it should go to tp32. > > > > On Wed, Jan 25, 2017 at 3:56 PM, Paul A. Jackson > > <paul.jack...@pb.com> > > wrote: > > > > > For what branch should a pull request be submitted? > > > > > > > > > -Original Message- > > > From: Stephen Mallette [mailto:spmalle...@gmail.com] > > > Sent: Wednesday, January 25, 2017 3:41 PM > > > To: dev@tinkerpop.apache.org > > > Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6 > > > > > > I went with the most obvious implementation place for > CloseableIterator. > > > If you see other spots where you could make an argument that it > > > would make sense to add it then feel free to offer a pull request > > > and we can get it reviewed. I didn't look into your VertexStep > > > suggestion too deeply, but a quick review seems to have me > > > thinking that it would make > > sense to do that. > > > Basically anywhere that a step interacts with the structure API > > > seems like it would be a candidate for CloseableIterator to be > > > implemented as it is in GraphStep. > > > > > > On Wed, Jan 25, 2017 at 3:30 PM, Paul A. Jackson > > > <paul.jack...@pb.com> > > > wrote: > > > > > > > So, I modified my code to work with CloseableIterator. I was > > > > hoping this would be honored in more places than it is. > > > > > > > > Where it does work is if the user of a traversal calls > > > > traversal.close() all the steps will get closed, including the > > > > typically > > > first GraphStep. > > > > GraphStep in turn checks whether the iterator that was provided > > > > by iteratorSupplier implements CloseableIterator and if so, > > > > closes it, and this is good. > > > > > > > > What I was hoping, in addition, though, was when > > > > VertexStep.flatMap() (or anything else) calls Vertex.vertices() > > > > or > > > > Vertex.edges() that before it finishes with the iterator it also > > > > make the same check for CloseableIterator and call close(). > > > > > > > > Thoughts? > > > > > > > > -Paul > > > > > > > > -Original Message- > > > > From: Paul A. Jackson [mailto:paul.jack...@pb.com] > > > > Sent: Tuesday, January 24, 2017 3:01 PM > > > > To: dev@tinkerpop.apache.org > > > > Subject: RE: [DISCUSS] Release 3.2.4 and 3.1.6 > > > > > > > > Great. I'll try it out. > > > > > > > > -Paul > > > > > > > > > > > > -Original Message- > > > > From: Stephen Mallette [mailto:spmalle...@gmail.com] > > > > Sent: Tuesday, January 24, 2017 2:54 PM > > > > To: dev@tinkerpop.apache.org > > > > Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6 > > > > &
RE: [DISCUSS] Release 3.2.4 and 3.1.6
OK, low-risk PR coming for OLTP part. Should I reuse TINKERPOP-1589? Thanks, -Paul -Original Message- From: Stephen Mallette [mailto:spmalle...@gmail.com] Sent: Wednesday, January 25, 2017 5:07 PM To: dev@tinkerpop.apache.org Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6 Being as close as we are to our code freeze/test week, I'd say that big, complex or otherwise risky changes probably won't collect too many +1 reviews at this point. If the OLTP improvement is small/concise (low risk), it could be considered for inclusion in 3.2.4. On Wed, Jan 25, 2017 at 5:00 PM, Paul A. Jackson <paul.jack...@pb.com> wrote: > I have something that fixes OLTP. I haven't worked with OLAP and it > looks like the changes for this will be extensive, touching > IteratorUtils and so on. > > Would you be interested in a PR for just the OLTP part? > > > -Original Message- > From: Stephen Mallette [mailto:spmalle...@gmail.com] > Sent: Wednesday, January 25, 2017 3:58 PM > To: dev@tinkerpop.apache.org > Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6 > > in his case, it should go to tp32. > > On Wed, Jan 25, 2017 at 3:56 PM, Paul A. Jackson <paul.jack...@pb.com> > wrote: > > > For what branch should a pull request be submitted? > > > > > > -Original Message- > > From: Stephen Mallette [mailto:spmalle...@gmail.com] > > Sent: Wednesday, January 25, 2017 3:41 PM > > To: dev@tinkerpop.apache.org > > Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6 > > > > I went with the most obvious implementation place for CloseableIterator. > > If you see other spots where you could make an argument that it > > would make sense to add it then feel free to offer a pull request > > and we can get it reviewed. I didn't look into your VertexStep > > suggestion too deeply, but a quick review seems to have me thinking > > that it would make > sense to do that. > > Basically anywhere that a step interacts with the structure API > > seems like it would be a candidate for CloseableIterator to be > > implemented as it is in GraphStep. > > > > On Wed, Jan 25, 2017 at 3:30 PM, Paul A. Jackson > > <paul.jack...@pb.com> > > wrote: > > > > > So, I modified my code to work with CloseableIterator. I was > > > hoping this would be honored in more places than it is. > > > > > > Where it does work is if the user of a traversal calls > > > traversal.close() all the steps will get closed, including the > > > typically > > first GraphStep. > > > GraphStep in turn checks whether the iterator that was provided by > > > iteratorSupplier implements CloseableIterator and if so, closes > > > it, and this is good. > > > > > > What I was hoping, in addition, though, was when > > > VertexStep.flatMap() (or anything else) calls Vertex.vertices() or > > > Vertex.edges() that before it finishes with the iterator it also > > > make the same check for CloseableIterator and call close(). > > > > > > Thoughts? > > > > > > -Paul > > > > > > -Original Message- > > > From: Paul A. Jackson [mailto:paul.jack...@pb.com] > > > Sent: Tuesday, January 24, 2017 3:01 PM > > > To: dev@tinkerpop.apache.org > > > Subject: RE: [DISCUSS] Release 3.2.4 and 3.1.6 > > > > > > Great. I'll try it out. > > > > > > -Paul > > > > > > > > > -Original Message- > > > From: Stephen Mallette [mailto:spmalle...@gmail.com] > > > Sent: Tuesday, January 24, 2017 2:54 PM > > > To: dev@tinkerpop.apache.org > > > Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6 > > > > > > no - it's in 3.2.4 and merged forward to 3.3.0: > > > > > > https://github.com/apache/tinkerpop/blob/e3889bf2401b42c3afbc85eab > > > c2 fb c ebf2588974/gremlin-core/src/main/java/org/apache/ > > > tinkerpop/gremlin/structure/util/CloseableIterator.java > > > > > > On Tue, Jan 24, 2017 at 2:52 PM, Paul A. Jackson > > > <paul.jack...@pb.com> > > > wrote: > > > > > > > Is CloseableIterator only in the 3.3 branch? > > > > > > > > -Paul > > > > > > > > > > > > -Original Message- > > > > From: Stephen Mallette [mailto:spmalle...@gmail.com] > > > > Sent: Monday, January 23, 2017 9:58 AM > > > > To: dev@tinkerpop.apache.org > > > > Subject: [DISCUSS] Release 3.2.4 and 3.1.6 > > > > > > > > It's been a while since we've had a release (Octob
RE: [DISCUSS] Release 3.2.4 and 3.1.6
I have something that fixes OLTP. I haven't worked with OLAP and it looks like the changes for this will be extensive, touching IteratorUtils and so on. Would you be interested in a PR for just the OLTP part? -Original Message- From: Stephen Mallette [mailto:spmalle...@gmail.com] Sent: Wednesday, January 25, 2017 3:58 PM To: dev@tinkerpop.apache.org Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6 in his case, it should go to tp32. On Wed, Jan 25, 2017 at 3:56 PM, Paul A. Jackson <paul.jack...@pb.com> wrote: > For what branch should a pull request be submitted? > > > -Original Message- > From: Stephen Mallette [mailto:spmalle...@gmail.com] > Sent: Wednesday, January 25, 2017 3:41 PM > To: dev@tinkerpop.apache.org > Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6 > > I went with the most obvious implementation place for CloseableIterator. > If you see other spots where you could make an argument that it would > make sense to add it then feel free to offer a pull request and we can > get it reviewed. I didn't look into your VertexStep suggestion too > deeply, but a quick review seems to have me thinking that it would make sense > to do that. > Basically anywhere that a step interacts with the structure API seems > like it would be a candidate for CloseableIterator to be implemented > as it is in GraphStep. > > On Wed, Jan 25, 2017 at 3:30 PM, Paul A. Jackson <paul.jack...@pb.com> > wrote: > > > So, I modified my code to work with CloseableIterator. I was hoping > > this would be honored in more places than it is. > > > > Where it does work is if the user of a traversal calls > > traversal.close() all the steps will get closed, including the > > typically > first GraphStep. > > GraphStep in turn checks whether the iterator that was provided by > > iteratorSupplier implements CloseableIterator and if so, closes it, > > and this is good. > > > > What I was hoping, in addition, though, was when > > VertexStep.flatMap() (or anything else) calls Vertex.vertices() or > > Vertex.edges() that before it finishes with the iterator it also > > make the same check for CloseableIterator and call close(). > > > > Thoughts? > > > > -Paul > > > > -Original Message- > > From: Paul A. Jackson [mailto:paul.jack...@pb.com] > > Sent: Tuesday, January 24, 2017 3:01 PM > > To: dev@tinkerpop.apache.org > > Subject: RE: [DISCUSS] Release 3.2.4 and 3.1.6 > > > > Great. I'll try it out. > > > > -Paul > > > > > > -Original Message- > > From: Stephen Mallette [mailto:spmalle...@gmail.com] > > Sent: Tuesday, January 24, 2017 2:54 PM > > To: dev@tinkerpop.apache.org > > Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6 > > > > no - it's in 3.2.4 and merged forward to 3.3.0: > > > > https://github.com/apache/tinkerpop/blob/e3889bf2401b42c3afbc85eabc2 > > fb c ebf2588974/gremlin-core/src/main/java/org/apache/ > > tinkerpop/gremlin/structure/util/CloseableIterator.java > > > > On Tue, Jan 24, 2017 at 2:52 PM, Paul A. Jackson > > <paul.jack...@pb.com> > > wrote: > > > > > Is CloseableIterator only in the 3.3 branch? > > > > > > -Paul > > > > > > > > > -Original Message- > > > From: Stephen Mallette [mailto:spmalle...@gmail.com] > > > Sent: Monday, January 23, 2017 9:58 AM > > > To: dev@tinkerpop.apache.org > > > Subject: [DISCUSS] Release 3.2.4 and 3.1.6 > > > > > > It's been a while since we've had a release (October 2016) and > > > given the importance of the recent critical security fix from > > > Groovy 2.4.8 I think it's worth getting some new versions out > > > there. I'm not sure what everyone is working on or has concerns > > > about, but after the PRs that are out there for tp32 get merged > > > (especially https://github.com/apache/ > > > tinkerpop/pull/541) > > > I don't really have anything else critical for those versions. > > > Please call out any issues that might be important for this > > > release on this > > thread. > > > > > > I don't think we should worry about doing a milestone release of > > > 3.3.0 > > yet. > > > I'd like to see some more change go into that branch before we do > > > that, but if others feel differently and would like to offer an > > > argument I'd be open to the idea. > > > > > > I propose we focus on a release of 3.1.6 and 3.2.4 in two weeks > > > time with the code freeze going into place at end of day friday of > > > this week (January 27, 2017). If there are no objections in the > > > next three days (Thursday, January 26, 2017, 10:00am), let's > > > assume lazy consensus and move forward with that plan. > > > > > > Assuming we do move forward with a release, are there any > > > volunteers for release manager? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
RE: [DISCUSS] Release 3.2.4 and 3.1.6
For what branch should a pull request be submitted? -Original Message- From: Stephen Mallette [mailto:spmalle...@gmail.com] Sent: Wednesday, January 25, 2017 3:41 PM To: dev@tinkerpop.apache.org Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6 I went with the most obvious implementation place for CloseableIterator. If you see other spots where you could make an argument that it would make sense to add it then feel free to offer a pull request and we can get it reviewed. I didn't look into your VertexStep suggestion too deeply, but a quick review seems to have me thinking that it would make sense to do that. Basically anywhere that a step interacts with the structure API seems like it would be a candidate for CloseableIterator to be implemented as it is in GraphStep. On Wed, Jan 25, 2017 at 3:30 PM, Paul A. Jackson <paul.jack...@pb.com> wrote: > So, I modified my code to work with CloseableIterator. I was hoping > this would be honored in more places than it is. > > Where it does work is if the user of a traversal calls > traversal.close() all the steps will get closed, including the typically > first GraphStep. > GraphStep in turn checks whether the iterator that was provided by > iteratorSupplier implements CloseableIterator and if so, closes it, > and this is good. > > What I was hoping, in addition, though, was when VertexStep.flatMap() > (or anything else) calls Vertex.vertices() or Vertex.edges() that > before it finishes with the iterator it also make the same check for > CloseableIterator and call close(). > > Thoughts? > > -Paul > > -Original Message- > From: Paul A. Jackson [mailto:paul.jack...@pb.com] > Sent: Tuesday, January 24, 2017 3:01 PM > To: dev@tinkerpop.apache.org > Subject: RE: [DISCUSS] Release 3.2.4 and 3.1.6 > > Great. I'll try it out. > > -Paul > > > -Original Message- > From: Stephen Mallette [mailto:spmalle...@gmail.com] > Sent: Tuesday, January 24, 2017 2:54 PM > To: dev@tinkerpop.apache.org > Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6 > > no - it's in 3.2.4 and merged forward to 3.3.0: > > https://github.com/apache/tinkerpop/blob/e3889bf2401b42c3afbc85eabc2fb > c ebf2588974/gremlin-core/src/main/java/org/apache/ > tinkerpop/gremlin/structure/util/CloseableIterator.java > > On Tue, Jan 24, 2017 at 2:52 PM, Paul A. Jackson <paul.jack...@pb.com> > wrote: > > > Is CloseableIterator only in the 3.3 branch? > > > > -Paul > > > > > > -Original Message- > > From: Stephen Mallette [mailto:spmalle...@gmail.com] > > Sent: Monday, January 23, 2017 9:58 AM > > To: dev@tinkerpop.apache.org > > Subject: [DISCUSS] Release 3.2.4 and 3.1.6 > > > > It's been a while since we've had a release (October 2016) and given > > the importance of the recent critical security fix from Groovy 2.4.8 > > I think it's worth getting some new versions out there. I'm not sure > > what everyone is working on or has concerns about, but after the PRs > > that are out there for tp32 get merged (especially > > https://github.com/apache/ > > tinkerpop/pull/541) > > I don't really have anything else critical for those versions. > > Please call out any issues that might be important for this release > > on this > thread. > > > > I don't think we should worry about doing a milestone release of > > 3.3.0 > yet. > > I'd like to see some more change go into that branch before we do > > that, but if others feel differently and would like to offer an > > argument I'd be open to the idea. > > > > I propose we focus on a release of 3.1.6 and 3.2.4 in two weeks time > > with the code freeze going into place at end of day friday of this > > week (January 27, 2017). If there are no objections in the next > > three days (Thursday, January 26, 2017, 10:00am), let's assume lazy > > consensus and move forward with that plan. > > > > Assuming we do move forward with a release, are there any volunteers > > for release manager? > > > > > > > > > > > > > > >
RE: [DISCUSS] Release 3.2.4 and 3.1.6
So, I modified my code to work with CloseableIterator. I was hoping this would be honored in more places than it is. Where it does work is if the user of a traversal calls traversal.close() all the steps will get closed, including the typically first GraphStep. GraphStep in turn checks whether the iterator that was provided by iteratorSupplier implements CloseableIterator and if so, closes it, and this is good. What I was hoping, in addition, though, was when VertexStep.flatMap() (or anything else) calls Vertex.vertices() or Vertex.edges() that before it finishes with the iterator it also make the same check for CloseableIterator and call close(). Thoughts? -Paul -Original Message- From: Paul A. Jackson [mailto:paul.jack...@pb.com] Sent: Tuesday, January 24, 2017 3:01 PM To: dev@tinkerpop.apache.org Subject: RE: [DISCUSS] Release 3.2.4 and 3.1.6 Great. I'll try it out. -Paul -Original Message- From: Stephen Mallette [mailto:spmalle...@gmail.com] Sent: Tuesday, January 24, 2017 2:54 PM To: dev@tinkerpop.apache.org Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6 no - it's in 3.2.4 and merged forward to 3.3.0: https://github.com/apache/tinkerpop/blob/e3889bf2401b42c3afbc85eabc2fbcebf2588974/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/CloseableIterator.java On Tue, Jan 24, 2017 at 2:52 PM, Paul A. Jackson <paul.jack...@pb.com> wrote: > Is CloseableIterator only in the 3.3 branch? > > -Paul > > > -Original Message- > From: Stephen Mallette [mailto:spmalle...@gmail.com] > Sent: Monday, January 23, 2017 9:58 AM > To: dev@tinkerpop.apache.org > Subject: [DISCUSS] Release 3.2.4 and 3.1.6 > > It's been a while since we've had a release (October 2016) and given > the importance of the recent critical security fix from Groovy 2.4.8 I > think it's worth getting some new versions out there. I'm not sure > what everyone is working on or has concerns about, but after the PRs > that are out there for tp32 get merged (especially > https://github.com/apache/ > tinkerpop/pull/541) > I don't really have anything else critical for those versions. Please > call out any issues that might be important for this release on this thread. > > I don't think we should worry about doing a milestone release of 3.3.0 yet. > I'd like to see some more change go into that branch before we do > that, but if others feel differently and would like to offer an > argument I'd be open to the idea. > > I propose we focus on a release of 3.1.6 and 3.2.4 in two weeks time > with the code freeze going into place at end of day friday of this > week (January 27, 2017). If there are no objections in the next three > days (Thursday, January 26, 2017, 10:00am), let's assume lazy > consensus and move forward with that plan. > > Assuming we do move forward with a release, are there any volunteers > for release manager? > > > >
RE: [DISCUSS] Release 3.2.4 and 3.1.6
Is CloseableIterator only in the 3.3 branch? -Paul -Original Message- From: Stephen Mallette [mailto:spmalle...@gmail.com] Sent: Monday, January 23, 2017 9:58 AM To: dev@tinkerpop.apache.org Subject: [DISCUSS] Release 3.2.4 and 3.1.6 It's been a while since we've had a release (October 2016) and given the importance of the recent critical security fix from Groovy 2.4.8 I think it's worth getting some new versions out there. I'm not sure what everyone is working on or has concerns about, but after the PRs that are out there for tp32 get merged (especially https://github.com/apache/tinkerpop/pull/541) I don't really have anything else critical for those versions. Please call out any issues that might be important for this release on this thread. I don't think we should worry about doing a milestone release of 3.3.0 yet. I'd like to see some more change go into that branch before we do that, but if others feel differently and would like to offer an argument I'd be open to the idea. I propose we focus on a release of 3.1.6 and 3.2.4 in two weeks time with the code freeze going into place at end of day friday of this week (January 27, 2017). If there are no objections in the next three days (Thursday, January 26, 2017, 10:00am), let's assume lazy consensus and move forward with that plan. Assuming we do move forward with a release, are there any volunteers for release manager?
RE: [TinkerPop] gremlin-x
I have 2 cents to throw in FWIW. We use Neo4j and support Gremlin as a query language. Sometimes it makes sense to preload properties when you load a vertex if you are in a concurrent environment. Say, you find a vertex via query lookup, if you read the actual property lazily it's possible for that property to change or disappear (Neo4j has READ_COMMITTED isolation level). But, if you are in a context where others cannot write you can benefit from loading properties lazily. For us, this knowledge and the related decisions can all be done below the Gremlin layer. -Paul -Original Message- From: pieter-gmail [mailto:pieter.mar...@gmail.com] Sent: Tuesday, December 06, 2016 10:43 AM To: gremlin-us...@googlegroups.com Cc: dev@tinkerpop.apache.org Subject: Re: [TinkerPop] gremlin-x Hi, Now that we have established our disagreement lets continue... "You prefer your model to be that of the underlying graph (following that logic, you would use Hibernate to map to Table objects?) and I prefer using application domain models." When working with TinkerPop I certainly prefer that its model is that of the underlying meta model i.e. having the property graph model as a first class construct. For modeling Cat/Dog/Person/Organization/Molecule... I also, like you use application models as an abstraction on top of TinkerPop. Hibernate on top off TinkerPop as opposed to JDBC if you will. "You prefer your query to return the underlying graph model and I prefer it to return any data." True, however I am totally fine with g.V().this().that().values(...); for the any data query. "You prefer your query to always return all properties and I prefer it to always return only selected properties." This is partially true and indeed the default that I and I'd say the TinkerPop test suites have assumed. There are countless gremlins that return GraphTraversalthat happily iterate the traversal accessing its properties assuming that they are right there and not another db hit. It has bitten me though with some fat vertices making it not such a sensible default anymore. And when loading vertices just to create edges, then loading the properties are somewhat silly. "You prefer your objects to be proxies to the underlying datastore (I think this blurs the lines between being a graph provider and gremlin client) and I prefer my objects to be detached with load/store being explicit." True. My programming model assumes this to be the case and again so does the TinkerPop test suite since day one. I'd argue it makes for eloquent code. No need to worry about attaching and detaching and reattaching with transaction semantics getting confused. You read and write within the same transaction boundaries with the Vertex/Edge being bound to the transaction. Very similar to you previous mail where you suggested binding the interaction to a connection, except for the object we bind to to is the transaction. Pretty much a one to one between transaction and connection for most ACID databases. "In the end, it sounds like you want gremlin to be an object-graph mapper in the graph model and I prefer a layered approach where gremlin is a simple query language of which an object-graph mapper, in any domain model, could be built on top (like so many other query languages)." Not so true. Object Graph Mappers will use TinkerPop as the graph layer to model their domain on top off. TinkerPop responsibility is to model its own model. The property graph model. One can think of Vertex/Edge/Property as a baby hardcoded meta ORM but maybe its not a useful analogy. All that said I think we are not all that far apart as the way I read it our primary disagreement is the properties. Lets take GraphTraversal . If its a ReferencedVertex then every property access will be another round trip. This seems unacceptable to me. If it returns a Vertex with all its properties pre-loaded then its also unacceptable. (Even if its the current default that I assume) So perhaps what Marko suggested g.withDetachment(…) - Detach.reference // just ids - Detach.reduced // ids, labels, keys, values (basically, what you need for toString()) - Detach.rich // property data included - Detached.full // edge data included (basically, StarGraph) will makes us both happy? Cheers Pieter On 05/12/2016 17:06, Robert Dale wrote: > Clearly we have different use cases. > > You prefer your model to be that of the underlying graph (following > that logic, you would use Hibernate to map to Table objects?) and I > prefer using application domain models. > > You prefer your query to return the underlying graph model and I > prefer it to return any data. > > You prefer your query to always return all properties and I prefer it > to always return only selected properties. > > You prefer your objects to be proxies to the underlying datastore (I > think this blurs the lines between being a