[jira] [Created] (TINKERPOP-2290) Javascript GLV connection refused error handling
Ted Wilmes created TINKERPOP-2290: - Summary: Javascript GLV connection refused error handling Key: TINKERPOP-2290 URL: https://issues.apache.org/jira/browse/TINKERPOP-2290 Project: TinkerPop Issue Type: Bug Components: javascript Affects Versions: 3.4.1 Reporter: Ted Wilmes I believe the Javascript driver is emitting connection refused errors in a manner that makes it difficult for users to add in retry and other error handling logic. My Javascript knowledge is minimal so apologies if the below examples are non-idiomatic Javascript. Here's an example of the error: {code:java} UNCAUGHT: { Error: connect ECONNREFUSED 127.0.0.1:8182 at Object._errnoException (util.js:1022:11) at _exceptionWithHostPort (util.js:1044:20) at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1198:14) code: 'ECONNREFUSED', errno: 'ECONNREFUSED', syscall: 'connect', address: '127.0.0.1', port: 8182 } UNCAUGHT: TypeError: Cannot read property 'aborted' of null at ClientRequest._req.on (/home/twilmes/repos/nodetest/node_modules/ws/lib/WebSocket.js:644:19) at emitOne (events.js:116:13) at ClientRequest.emit (events.js:211:7) at Socket.socketCloseListener (_http_client.js:363:9) at emitOne (events.js:121:20) at Socket.emit (events.js:211:7) at TCP._handle.close [as _onclose] (net.js:567:12) {code} I did some looking at that UNCAUGHT TypeError is a separate issue and appears to be fixed in a newer version of the WS package that is being used. To reproduce this problem, you can either start an application up and point it at a non-existant TP enabled DB (JanusGraph in my case) or turn the graph off while the application is running. After this is done, an error is logged but the only way I'm able to catch it is to do something like this: {code:java} process.on('uncaughtException', function (err) { console.log("UNCAUGHT: ", err); });{code} Wrapping my queries and connection code in try/catch blocks does not catch anything. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Closed] (TINKERPOP-1973) Emit predicate ignored after final loop of a RepeatStep with times or until
[ https://issues.apache.org/jira/browse/TINKERPOP-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes closed TINKERPOP-1973. - Resolution: Won't Fix Closing as it is the expected behavior. Thanks for the examples [~dkuppitz] > Emit predicate ignored after final loop of a RepeatStep with times or until > --- > > Key: TINKERPOP-1973 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1973 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.3.3, 3.2.9 >Reporter: Ted Wilmes >Assignee: Ted Wilmes >Priority: Major > > When combined with a {{times}} or {{until}}, the {{emit}} predicate is not > applied to the output of the last loop of the {{RepeatStep}}. > {code} > gremlin> g = TinkerFactory.createModern().traversal() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] > gremlin> g.V().repeat(out()).emit(has('name', 'lop')).values('name') > ==>lop > ==>lop > ==>lop > ==>lop > gremlin> g.V().repeat(out()).times(2).emit(has('name', 'lop')).values('name') > ==>lop > ==>ripple > ==>lop > ==>lop > ==>lop > gremlin> > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TINKERPOP-1973) Emit predicate ignored after final loop of a RepeatStep with times or until
[ https://issues.apache.org/jira/browse/TINKERPOP-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16484040#comment-16484040 ] Ted Wilmes commented on TINKERPOP-1973: --- Ok, that make sense in the context of the injector distinction. So for my contrived example, the key would be to include the same predicate after the emit. That'll work for most cases I think as long as you aren't relying on the loop counter as part of your emit logic since (I think) the loop count is no longer available after the traverser has exited the repeat. The reason this came up was I was writing a traversal that traversed up a tree from the leaves a set number of levels, storing the intermediate results in a subgraph. For a smaller subset of those levels, It needed to execute another repeat traversal so I was using an {{emit(loops().is(lt(3))).repeat()}}. The issue was that repeat was executed for levels less than 3 and then the last level exiting the repeat. > Emit predicate ignored after final loop of a RepeatStep with times or until > --- > > Key: TINKERPOP-1973 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1973 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.3.3, 3.2.9 >Reporter: Ted Wilmes >Assignee: Ted Wilmes >Priority: Major > > When combined with a {{times}} or {{until}}, the {{emit}} predicate is not > applied to the output of the last loop of the {{RepeatStep}}. > {code} > gremlin> g = TinkerFactory.createModern().traversal() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] > gremlin> g.V().repeat(out()).emit(has('name', 'lop')).values('name') > ==>lop > ==>lop > ==>lop > ==>lop > gremlin> g.V().repeat(out()).times(2).emit(has('name', 'lop')).values('name') > ==>lop > ==>ripple > ==>lop > ==>lop > ==>lop > gremlin> > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TINKERPOP-1973) Emit predicate ignored after final loop of a RepeatStep with times or until
[ https://issues.apache.org/jira/browse/TINKERPOP-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482859#comment-16482859 ] Ted Wilmes commented on TINKERPOP-1973: --- [~dkuppitz] this behavior seemed like a bug to me but I wanted to double check with you to see if you thought there was a purpose for it. > Emit predicate ignored after final loop of a RepeatStep with times or until > --- > > Key: TINKERPOP-1973 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1973 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.3.3, 3.2.9 >Reporter: Ted Wilmes >Assignee: Ted Wilmes >Priority: Major > > When combined with a {{times}} or {{until}}, the {{emit}} predicate is not > applied to the output of the last loop of the {{RepeatStep}}. > {code} > gremlin> g = TinkerFactory.createModern().traversal() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] > gremlin> g.V().repeat(out()).emit(has('name', 'lop')).values('name') > ==>lop > ==>lop > ==>lop > ==>lop > gremlin> g.V().repeat(out()).times(2).emit(has('name', 'lop')).values('name') > ==>lop > ==>ripple > ==>lop > ==>lop > ==>lop > gremlin> > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (TINKERPOP-1973) Emit predicate ignored after final loop of a RepeatStep with times or until
[ https://issues.apache.org/jira/browse/TINKERPOP-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes updated TINKERPOP-1973: -- Description: When combined with a {{times}} or {{until}}, the {{emit}} predicate is not applied to the output of the last loop of the {{RepeatStep}}. {code} gremlin> g = TinkerFactory.createModern().traversal() ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] gremlin> g.V().repeat(out()).emit(has('name', 'lop')).values('name') ==>lop ==>lop ==>lop ==>lop gremlin> g.V().repeat(out()).times(2).emit(has('name', 'lop')).values('name') ==>lop ==>ripple ==>lop ==>lop ==>lop gremlin> {code} was: When combined with a `times` or `until`, the `emit` predicate is not applied to the output of the last loop of the `RepeatStep`. {code:groovy} gremlin> g = TinkerFactory.createModern().traversal() ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] gremlin> g.V().repeat(out()).emit(has('name', 'lop')).values('name') ==>lop ==>lop ==>lop ==>lop gremlin> g.V().repeat(out()).times(2).emit(has('name', 'lop')).values('name') ==>lop ==>ripple ==>lop ==>lop ==>lop gremlin> {code} > Emit predicate ignored after final loop of a RepeatStep with times or until > --- > > Key: TINKERPOP-1973 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1973 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.3.3, 3.2.9 >Reporter: Ted Wilmes >Assignee: Ted Wilmes >Priority: Major > > When combined with a {{times}} or {{until}}, the {{emit}} predicate is not > applied to the output of the last loop of the {{RepeatStep}}. > {code} > gremlin> g = TinkerFactory.createModern().traversal() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] > gremlin> g.V().repeat(out()).emit(has('name', 'lop')).values('name') > ==>lop > ==>lop > ==>lop > ==>lop > gremlin> g.V().repeat(out()).times(2).emit(has('name', 'lop')).values('name') > ==>lop > ==>ripple > ==>lop > ==>lop > ==>lop > gremlin> > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (TINKERPOP-1973) Emit predicate ignored after final loop of a RepeatStep with times or until
[ https://issues.apache.org/jira/browse/TINKERPOP-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes updated TINKERPOP-1973: -- Description: When combined with a `times` or `until`, the `emit` predicate is not applied to the output of the last loop of the `RepeatStep`. {code:groovy} gremlin> g = TinkerFactory.createModern().traversal() ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] gremlin> g.V().repeat(out()).emit(has('name', 'lop')).values('name') ==>lop ==>lop ==>lop ==>lop gremlin> g.V().repeat(out()).times(2).emit(has('name', 'lop')).values('name') ==>lop ==>ripple ==>lop ==>lop ==>lop gremlin> {code} was: When combined with a `times` or `until`, the `emit` predicate is not applied to the output of the last loop of the `RepeatStep`. ``` gremlin> g = TinkerFactory.createModern().traversal() ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] gremlin> g.V().repeat(out()).emit(has('name', 'lop')).values('name') ==>lop ==>lop ==>lop ==>lop gremlin> g.V().repeat(out()).times(2).emit(has('name', 'lop')).values('name') ==>lop ==>ripple ==>lop ==>lop ==>lop gremlin> ``` > Emit predicate ignored after final loop of a RepeatStep with times or until > --- > > Key: TINKERPOP-1973 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1973 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.3.3, 3.2.9 >Reporter: Ted Wilmes >Assignee: Ted Wilmes >Priority: Major > > When combined with a `times` or `until`, the `emit` predicate is not applied > to the output of the last loop of the `RepeatStep`. > {code:groovy} > gremlin> g = TinkerFactory.createModern().traversal() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] > gremlin> g.V().repeat(out()).emit(has('name', 'lop')).values('name') > ==>lop > ==>lop > ==>lop > ==>lop > gremlin> g.V().repeat(out()).times(2).emit(has('name', 'lop')).values('name') > ==>lop > ==>ripple > ==>lop > ==>lop > ==>lop > gremlin> > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (TINKERPOP-1973) Emit predicate ignored after final loop of a RepeatStep with times or until
Ted Wilmes created TINKERPOP-1973: - Summary: Emit predicate ignored after final loop of a RepeatStep with times or until Key: TINKERPOP-1973 URL: https://issues.apache.org/jira/browse/TINKERPOP-1973 Project: TinkerPop Issue Type: Bug Components: process Affects Versions: 3.2.9, 3.3.3 Reporter: Ted Wilmes Assignee: Ted Wilmes When combined with a `times` or `until`, the `emit` predicate is not applied to the output of the last loop of the `RepeatStep`. ``` gremlin> g = TinkerFactory.createModern().traversal() ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] gremlin> g.V().repeat(out()).emit(has('name', 'lop')).values('name') ==>lop ==>lop ==>lop ==>lop gremlin> g.V().repeat(out()).times(2).emit(has('name', 'lop')).values('name') ==>lop ==>ripple ==>lop ==>lop ==>lop gremlin> ``` -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (TINKERPOP-1925) Don't apply PathRetractionStrategy recursively
[ https://issues.apache.org/jira/browse/TINKERPOP-1925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes reassigned TINKERPOP-1925: - Assignee: Ted Wilmes > Don't apply PathRetractionStrategy recursively > -- > > Key: TINKERPOP-1925 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1925 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.2.7 >Reporter: Daniel Kuppitz >Assignee: Ted Wilmes >Priority: Major > > {{PathRetractionStrategy}} should not be applied recursively, if possible. > This is considered an anti-pattern and just recently it caused a lot of > trouble. This commit fixed the recent issue: > https://github.com/apache/tinkerpop/commit/b8eb8a02f1675a1b22b86ccc54d22453c65f406f > {{PathRetractionStrategy}} collects requirements of child traversals, before > other strategies have a chance to mutate them. When other strategies get to > the child traversals, mutate them and thus add or remove requirements, a few > things can go crazy, especially because traverser requirements get cached > when they're first requested. > Since I do not fully understand the {{PathRetractionStrategy}} code, my fix > was actually a workaround. As you can see in the commit linked above, I > implemented the same anti-pattern in {{SubgraphStrategy}} and > {{IncidentToAdjacentStrategy}}; these are the 2 strategies that can > potentially change the traverser requirements. > It would be nice to revert these changes and instead make > {{PathRetractionStrategy}} work like all others. However, I'm not sure if > that's even possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (TINKERPOP-1810) Add Lambda.binaryOperator and Lambda.unaryOperator
Ted Wilmes created TINKERPOP-1810: - Summary: Add Lambda.binaryOperator and Lambda.unaryOperator Key: TINKERPOP-1810 URL: https://issues.apache.org/jira/browse/TINKERPOP-1810 Project: TinkerPop Issue Type: Improvement Components: process Affects Versions: 3.2.6 Reporter: Ted Wilmes While working with sack and lambdas, I found that I could not specify a split ({{UnaryOperator}}) or merge ({{BinaryOperator}}) operator remotely using the {{Lambda}} class helper methods. The sack supplier worked without issue because we already have {{Lambda.supplier}}. I propose we add {{Lambda.unaryOperator}} and {{Lambda.binaryOperator}}. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (TINKERPOP-1650) PathRetractionStrategy makes Match steps unsolvable
[ https://issues.apache.org/jira/browse/TINKERPOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes resolved TINKERPOP-1650. --- Resolution: Fixed Fix Version/s: 3.3.1 > PathRetractionStrategy makes Match steps unsolvable > --- > > Key: TINKERPOP-1650 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1650 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.3, 3.2.4 >Reporter: Branden Moore >Assignee: Ted Wilmes > Fix For: 3.2.7, 3.3.1 > > > The `PathRetractionStrategy` can make certain Match() steps "Unsolvable". > This (nonsensical) example demonstrates the issue: > {code} > gremlin> g = graph.traversal() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] > gremlin> > g.V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > The provided match pattern is unsolvable: [[MatchStartStep(a), > VertexStep(OUT,[created],vertex), MatchEndStep(sw)], [MatchStartStep(sw), > HasStep([lang.eq(java)]), MatchEndStep(java)], [MatchStartStep(sw), > WherePredicateStep(neq(a)), MatchEndStep]] > Type ':help' or ':h' for help. > Display stack trace? [yN]n > gremlin> > {code} > If we remove the `PathRetractionStrategy`, or use `g.withPath()` the match > step is solvable, and works fine. > {code} > g.withoutStrategies(PathRetractionStrategy).V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > ==>v[1] > ==>v[4] > ==>v[4] > ==>v[6] > gremlin> > g.withPath().V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > ==>v[1] > ==>v[4] > ==>v[4] > ==>v[6] > gremlin> > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (TINKERPOP-1650) PathRetractionStrategy makes Match steps unsolvable
[ https://issues.apache.org/jira/browse/TINKERPOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes closed TINKERPOP-1650. - > PathRetractionStrategy makes Match steps unsolvable > --- > > Key: TINKERPOP-1650 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1650 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.3, 3.2.4 >Reporter: Branden Moore >Assignee: Ted Wilmes > Fix For: 3.2.7, 3.3.1 > > > The `PathRetractionStrategy` can make certain Match() steps "Unsolvable". > This (nonsensical) example demonstrates the issue: > {code} > gremlin> g = graph.traversal() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] > gremlin> > g.V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > The provided match pattern is unsolvable: [[MatchStartStep(a), > VertexStep(OUT,[created],vertex), MatchEndStep(sw)], [MatchStartStep(sw), > HasStep([lang.eq(java)]), MatchEndStep(java)], [MatchStartStep(sw), > WherePredicateStep(neq(a)), MatchEndStep]] > Type ':help' or ':h' for help. > Display stack trace? [yN]n > gremlin> > {code} > If we remove the `PathRetractionStrategy`, or use `g.withPath()` the match > step is solvable, and works fine. > {code} > g.withoutStrategies(PathRetractionStrategy).V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > ==>v[1] > ==>v[4] > ==>v[4] > ==>v[6] > gremlin> > g.withPath().V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > ==>v[1] > ==>v[4] > ==>v[4] > ==>v[6] > gremlin> > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (TINKERPOP-1650) PathRetractionStrategy makes Match steps unsolvable
[ https://issues.apache.org/jira/browse/TINKERPOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes reopened TINKERPOP-1650: --- > PathRetractionStrategy makes Match steps unsolvable > --- > > Key: TINKERPOP-1650 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1650 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.3, 3.2.4 >Reporter: Branden Moore >Assignee: Ted Wilmes > Fix For: 3.2.7, 3.3.1 > > > The `PathRetractionStrategy` can make certain Match() steps "Unsolvable". > This (nonsensical) example demonstrates the issue: > {code} > gremlin> g = graph.traversal() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] > gremlin> > g.V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > The provided match pattern is unsolvable: [[MatchStartStep(a), > VertexStep(OUT,[created],vertex), MatchEndStep(sw)], [MatchStartStep(sw), > HasStep([lang.eq(java)]), MatchEndStep(java)], [MatchStartStep(sw), > WherePredicateStep(neq(a)), MatchEndStep]] > Type ':help' or ':h' for help. > Display stack trace? [yN]n > gremlin> > {code} > If we remove the `PathRetractionStrategy`, or use `g.withPath()` the match > step is solvable, and works fine. > {code} > g.withoutStrategies(PathRetractionStrategy).V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > ==>v[1] > ==>v[4] > ==>v[4] > ==>v[6] > gremlin> > g.withPath().V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > ==>v[1] > ==>v[4] > ==>v[4] > ==>v[6] > gremlin> > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (TINKERPOP-1650) PathRetractionStrategy makes Match steps unsolvable
[ https://issues.apache.org/jira/browse/TINKERPOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes closed TINKERPOP-1650. - > PathRetractionStrategy makes Match steps unsolvable > --- > > Key: TINKERPOP-1650 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1650 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.3, 3.2.4 >Reporter: Branden Moore >Assignee: Ted Wilmes > Fix For: 3.2.7 > > > The `PathRetractionStrategy` can make certain Match() steps "Unsolvable". > This (nonsensical) example demonstrates the issue: > {code} > gremlin> g = graph.traversal() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] > gremlin> > g.V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > The provided match pattern is unsolvable: [[MatchStartStep(a), > VertexStep(OUT,[created],vertex), MatchEndStep(sw)], [MatchStartStep(sw), > HasStep([lang.eq(java)]), MatchEndStep(java)], [MatchStartStep(sw), > WherePredicateStep(neq(a)), MatchEndStep]] > Type ':help' or ':h' for help. > Display stack trace? [yN]n > gremlin> > {code} > If we remove the `PathRetractionStrategy`, or use `g.withPath()` the match > step is solvable, and works fine. > {code} > g.withoutStrategies(PathRetractionStrategy).V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > ==>v[1] > ==>v[4] > ==>v[4] > ==>v[6] > gremlin> > g.withPath().V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > ==>v[1] > ==>v[4] > ==>v[4] > ==>v[6] > gremlin> > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (TINKERPOP-1650) PathRetractionStrategy makes Match steps unsolvable
[ https://issues.apache.org/jira/browse/TINKERPOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes resolved TINKERPOP-1650. --- Resolution: Fixed Fix Version/s: 3.2.7 > PathRetractionStrategy makes Match steps unsolvable > --- > > Key: TINKERPOP-1650 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1650 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.3, 3.2.4 >Reporter: Branden Moore >Assignee: Ted Wilmes > Fix For: 3.2.7 > > > The `PathRetractionStrategy` can make certain Match() steps "Unsolvable". > This (nonsensical) example demonstrates the issue: > {code} > gremlin> g = graph.traversal() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] > gremlin> > g.V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > The provided match pattern is unsolvable: [[MatchStartStep(a), > VertexStep(OUT,[created],vertex), MatchEndStep(sw)], [MatchStartStep(sw), > HasStep([lang.eq(java)]), MatchEndStep(java)], [MatchStartStep(sw), > WherePredicateStep(neq(a)), MatchEndStep]] > Type ':help' or ':h' for help. > Display stack trace? [yN]n > gremlin> > {code} > If we remove the `PathRetractionStrategy`, or use `g.withPath()` the match > step is solvable, and works fine. > {code} > g.withoutStrategies(PathRetractionStrategy).V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > ==>v[1] > ==>v[4] > ==>v[4] > ==>v[6] > gremlin> > g.withPath().V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > ==>v[1] > ==>v[4] > ==>v[4] > ==>v[6] > gremlin> > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (TINKERPOP-1781) Traversal admin addStep does not update bytecode
Ted Wilmes created TINKERPOP-1781: - Summary: Traversal admin addStep does not update bytecode Key: TINKERPOP-1781 URL: https://issues.apache.org/jira/browse/TINKERPOP-1781 Project: TinkerPop Issue Type: Improvement Components: process Affects Versions: 3.2.6, 3.3.0 Reporter: Ted Wilmes Right now bytecode is constructed at traversal construction time and is not updated post-construction when traversal rewriting happens via strategies. This works fine in most cases because the traversal is locked post-strategy application and will be executed, not serialized out for execution somewhere else. There are issues when trying to apply client side, pre-serialization traversal rewriting (a kind of client-side traversal strategy outside of the usual strategy flow). Since a direct call to {{addStep}} circumvents the bytecode construction, the bytecode diverges from the rewritten traversal. {code} gremlin> traversal = g.V().out();null ==>null gremlin> traversal.getSteps() ==>GraphStep(vertex,[]) ==>VertexStep(OUT,vertex) gremlin> traversal.getBytecode() ==>[[], [V(), out()]] gremlin> TraversalHelper.insertAfterStep(new CountGlobalStep(traversal), traversal.getSteps()[1], traversal.asAdmin()) ==>null gremlin> traversal.getSteps() ==>GraphStep(vertex,[]) ==>VertexStep(OUT,vertex) ==>CountGlobalStep gremlin> traversal.getBytecode() ==>[[], [V(), out()]] {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (TINKERPOP-1287) StarGraph has an overdose of Stream usage.
[ https://issues.apache.org/jira/browse/TINKERPOP-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes closed TINKERPOP-1287. - Resolution: Not A Problem Profiling did not show stream usage making a significant impact on performance in this case. The use of `intern` mentioned in the comments was taken care of by TINKERPOP-1680. > StarGraph has an overdose of Stream usage. > -- > > Key: TINKERPOP-1287 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1287 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop, structure >Affects Versions: 3.2.0-incubating, 3.1.2-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > Fix For: 3.2.7 > > Attachments: g.V.out.out.count.svg, stage0.svg, stage1.svg, stage2.svg > > > {{StarGraph}} is loaded with {{Stream}}-usage. Gutting streams from > TinkerGraph made it much faster. It would be good if we did the same thing > for {{StarGraph}}. > This can go into tp31/ and upmerge to master/. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TINKERPOP-1287) StarGraph has an overdose of Stream usage.
[ https://issues.apache.org/jira/browse/TINKERPOP-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16163135#comment-16163135 ] Ted Wilmes commented on TINKERPOP-1287: --- Not at this point [~spmallette], especially since the {{intern}} was taken care of. I'll go ahead and close it. > StarGraph has an overdose of Stream usage. > -- > > Key: TINKERPOP-1287 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1287 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop, structure >Affects Versions: 3.2.0-incubating, 3.1.2-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > Fix For: 3.2.7 > > Attachments: g.V.out.out.count.svg, stage0.svg, stage1.svg, stage2.svg > > > {{StarGraph}} is loaded with {{Stream}}-usage. Gutting streams from > TinkerGraph made it much faster. It would be good if we did the same thing > for {{StarGraph}}. > This can go into tp31/ and upmerge to master/. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (TINKERPOP-1727) Bytecode object shallow copied when traversals are cloned
[ https://issues.apache.org/jira/browse/TINKERPOP-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes closed TINKERPOP-1727. - Resolution: Fixed Fix Version/s: 3.2.6 3.3.0 > Bytecode object shallow copied when traversals are cloned > - > > Key: TINKERPOP-1727 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1727 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.3.0, 3.2.5 >Reporter: Ted Wilmes >Assignee: Ted Wilmes > Fix For: 3.3.0, 3.2.6 > > > The {{bytecode}} member of traversal objects are only shallow copied when a > traversal is cloned. This causes issues because any updates to the clone will > be propagated to cloned traversal. > {code} > gremlin> g = TinkerGraph.open().traversal() > ==>graphtraversalsource[tinkergraph[vertices:0 edges:0], standard] > gremlin> traversal = g.V().out();null > ==>null > gremlin> clonedTraversal = traversal.asAdmin().clone();null > ==>null > gremlin> clonedTraversal.out().out();null > ==>null > gremlin> traversal.getBytecode() > ==>[[], [V(), out(), out(), out()]] > gremlin> clonedTraversal.getBytecode() > ==>[[], [V(), out(), out(), out()]] > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (TINKERPOP-1727) Bytecode object shallow copied when traversals are cloned
Ted Wilmes created TINKERPOP-1727: - Summary: Bytecode object shallow copied when traversals are cloned Key: TINKERPOP-1727 URL: https://issues.apache.org/jira/browse/TINKERPOP-1727 Project: TinkerPop Issue Type: Bug Components: process Affects Versions: 3.2.5, 3.3.0 Reporter: Ted Wilmes Assignee: Ted Wilmes The {{bytecode}} member of traversal objects are only shallow copied when a traversal is cloned. This causes issues because any updates to the clone will be propagated to cloned traversal. {code} gremlin> g = TinkerGraph.open().traversal() ==>graphtraversalsource[tinkergraph[vertices:0 edges:0], standard] gremlin> traversal = g.V().out();null ==>null gremlin> clonedTraversal = traversal.asAdmin().clone();null ==>null gremlin> clonedTraversal.out().out();null ==>null gremlin> traversal.getBytecode() ==>[[], [V(), out(), out(), out()]] gremlin> clonedTraversal.getBytecode() ==>[[], [V(), out(), out(), out()]] {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TINKERPOP-1650) PathRetractionStrategy makes Match steps unsolvable
[ https://issues.apache.org/jira/browse/TINKERPOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051943#comment-16051943 ] Ted Wilmes commented on TINKERPOP-1650: --- Yes, I still plan on fixing this one. > PathRetractionStrategy makes Match steps unsolvable > --- > > Key: TINKERPOP-1650 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1650 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.3, 3.2.4 >Reporter: Branden Moore >Assignee: Ted Wilmes > > The `PathRetractionStrategy` can make certain Match() steps "Unsolvable". > This (nonsensical) example demonstrates the issue: > {code} > gremlin> g = graph.traversal() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] > gremlin> > g.V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > The provided match pattern is unsolvable: [[MatchStartStep(a), > VertexStep(OUT,[created],vertex), MatchEndStep(sw)], [MatchStartStep(sw), > HasStep([lang.eq(java)]), MatchEndStep(java)], [MatchStartStep(sw), > WherePredicateStep(neq(a)), MatchEndStep]] > Type ':help' or ':h' for help. > Display stack trace? [yN]n > gremlin> > {code} > If we remove the `PathRetractionStrategy`, or use `g.withPath()` the match > step is solvable, and works fine. > {code} > g.withoutStrategies(PathRetractionStrategy).V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > ==>v[1] > ==>v[4] > ==>v[4] > ==>v[6] > gremlin> > g.withPath().V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > ==>v[1] > ==>v[4] > ==>v[4] > ==>v[6] > gremlin> > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (TINKERPOP-1652) Disable PathRetractionStrategy strategy if VertexProgramStep has LABELLED_PATH requirement
[ https://issues.apache.org/jira/browse/TINKERPOP-1652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes resolved TINKERPOP-1652. --- Resolution: Fixed Fix Version/s: 3.2.5 > Disable PathRetractionStrategy strategy if VertexProgramStep has > LABELLED_PATH requirement > -- > > Key: TINKERPOP-1652 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1652 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.4 >Reporter: Ted Wilmes >Assignee: Ted Wilmes > Fix For: 3.2.5 > > > [~dkuppitz] notified me that there is an issue where > {{PathRetractionStrategy}} is incorrectly dropping path elements when a > {{VertexProgramStep}} is present that has a {{LABELLED_PATH}} requirement. > We already handle the case where path is required but this issue will also > disable the strategy if a vertex program is present that requires > {{LABELLED_PATH}}. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (TINKERPOP-1652) Disable PathRetractionStrategy strategy if VertexProgramStep has LABELLED_PATH requirement
[ https://issues.apache.org/jira/browse/TINKERPOP-1652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes closed TINKERPOP-1652. - > Disable PathRetractionStrategy strategy if VertexProgramStep has > LABELLED_PATH requirement > -- > > Key: TINKERPOP-1652 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1652 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.4 >Reporter: Ted Wilmes >Assignee: Ted Wilmes > Fix For: 3.2.5 > > > [~dkuppitz] notified me that there is an issue where > {{PathRetractionStrategy}} is incorrectly dropping path elements when a > {{VertexProgramStep}} is present that has a {{LABELLED_PATH}} requirement. > We already handle the case where path is required but this issue will also > disable the strategy if a vertex program is present that requires > {{LABELLED_PATH}}. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TINKERPOP-1652) Disable PathRetractionStrategy strategy if VertexProgramStep has LABELLED_PATH requirement
[ https://issues.apache.org/jira/browse/TINKERPOP-1652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940784#comment-15940784 ] Ted Wilmes commented on TINKERPOP-1652: --- Thanks for the test, I was thinking this wouldn't fix that. That'll be next on the list. > Disable PathRetractionStrategy strategy if VertexProgramStep has > LABELLED_PATH requirement > -- > > Key: TINKERPOP-1652 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1652 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.4 >Reporter: Ted Wilmes >Assignee: Ted Wilmes > > [~dkuppitz] notified me that there is an issue where > {{PathRetractionStrategy}} is incorrectly dropping path elements when a > {{VertexProgramStep}} is present that has a {{LABELLED_PATH}} requirement. > We already handle the case where path is required but this issue will also > disable the strategy if a vertex program is present that requires > {{LABELLED_PATH}}. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (TINKERPOP-1652) Disable PathRetractionStrategy strategy if VertexProgramStep has LABELLED_PATH requirement
Ted Wilmes created TINKERPOP-1652: - Summary: Disable PathRetractionStrategy strategy if VertexProgramStep has LABELLED_PATH requirement Key: TINKERPOP-1652 URL: https://issues.apache.org/jira/browse/TINKERPOP-1652 Project: TinkerPop Issue Type: Bug Reporter: Ted Wilmes Assignee: Ted Wilmes [~dkuppitz] notified me that there is an issue where {{PathRetractionStrategy}} is incorrectly dropping path elements when a {{VertexProgramStep}} is present that has a {{LABELLED_PATH}} requirement. We already handle the case where path is required but this issue will also disable the strategy if a vertex program is present that requires {{LABELLED_PATH}}. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (TINKERPOP-1650) PathRetractionStrategy makes Match steps unsolvable
[ https://issues.apache.org/jira/browse/TINKERPOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes reassigned TINKERPOP-1650: - Assignee: Ted Wilmes > PathRetractionStrategy makes Match steps unsolvable > --- > > Key: TINKERPOP-1650 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1650 > Project: TinkerPop > Issue Type: Bug >Affects Versions: 3.2.3, 3.2.4 >Reporter: Branden Moore >Assignee: Ted Wilmes > > The `PathRetractionStrategy` can make certain Match() steps "Unsolvable". > This (nonsensical) example demonstrates the issue: > {code} > gremlin> g = graph.traversal() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] > gremlin> > g.V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > The provided match pattern is unsolvable: [[MatchStartStep(a), > VertexStep(OUT,[created],vertex), MatchEndStep(sw)], [MatchStartStep(sw), > HasStep([lang.eq(java)]), MatchEndStep(java)], [MatchStartStep(sw), > WherePredicateStep(neq(a)), MatchEndStep]] > Type ':help' or ':h' for help. > Display stack trace? [yN]n > gremlin> > {code} > If we remove the `PathRetractionStrategy`, or use `g.withPath()` the match > step is solvable, and works fine. > {code} > g.withoutStrategies(PathRetractionStrategy).V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > ==>v[1] > ==>v[4] > ==>v[4] > ==>v[6] > gremlin> > g.withPath().V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > ==>v[1] > ==>v[4] > ==>v[4] > ==>v[6] > gremlin> > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TINKERPOP-1650) PathRetractionStrategy makes Match steps unsolvable
[ https://issues.apache.org/jira/browse/TINKERPOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15926897#comment-15926897 ] Ted Wilmes commented on TINKERPOP-1650: --- Thanks for the steps to reproduce [~bjmoor]. I'll take a look. > PathRetractionStrategy makes Match steps unsolvable > --- > > Key: TINKERPOP-1650 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1650 > Project: TinkerPop > Issue Type: Bug >Affects Versions: 3.2.3, 3.2.4 >Reporter: Branden Moore > > The `PathRetractionStrategy` can make certain Match() steps "Unsolvable". > This (nonsensical) example demonstrates the issue: > {code} > gremlin> g = graph.traversal() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] > gremlin> > g.V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > The provided match pattern is unsolvable: [[MatchStartStep(a), > VertexStep(OUT,[created],vertex), MatchEndStep(sw)], [MatchStartStep(sw), > HasStep([lang.eq(java)]), MatchEndStep(java)], [MatchStartStep(sw), > WherePredicateStep(neq(a)), MatchEndStep]] > Type ':help' or ':h' for help. > Display stack trace? [yN]n > gremlin> > {code} > If we remove the `PathRetractionStrategy`, or use `g.withPath()` the match > step is solvable, and works fine. > {code} > g.withoutStrategies(PathRetractionStrategy).V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > ==>v[1] > ==>v[4] > ==>v[4] > ==>v[6] > gremlin> > g.withPath().V().hasLabel("person").as("perp").match(__.as("a").out("created").as("sw"), > __.as("sw").has("lang", "java").as("java")).where("sw", > neq("a")).select("perp") > ==>v[1] > ==>v[4] > ==>v[4] > ==>v[6] > gremlin> > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (TINKERPOP-1620) Choose with count branch traversal type coercion
Ted Wilmes created TINKERPOP-1620: - Summary: Choose with count branch traversal type coercion Key: TINKERPOP-1620 URL: https://issues.apache.org/jira/browse/TINKERPOP-1620 Project: TinkerPop Issue Type: Bug Components: process Affects Versions: 3.2.3 Reporter: Ted Wilmes Assignee: Ted Wilmes A {{ChooseStep}} with a {{count}} branching traversal is currently sensitive to whether or not the {{option}} keys are {{long}} or {{int}}. This results in the following behavior: {code} gremlin> g = TinkerFactory.createModern().traversal() ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] gremlin> g.V().choose(bothE().count()).option(1, constant(1)).option(3, constant(3)) gremlin> g.V().choose(bothE().count()).option(1l, constant(1)).option(3l, constant(3)) ==>3 ==>1 ==>3 ==>3 ==>1 ==>1 {code} The branch traversal results if, numeric, should be evaluated against the options using {{NumberHelper}} so as to be consistent with other steps. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] (TINKERPOP-1597) PathRetractionStrategy messing up certain traversals
[ https://issues.apache.org/jira/browse/TINKERPOP-1597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes closed TINKERPOP-1597. - Resolution: Fixed A `PathRetractionStrategy` test has been added to verify the nested `dedup` does not drop labels. The new `OptionalTest` test method provided by [~dkuppitz] is affected by TINKERPOP-1619 so it was not merged as part of this ticket. > PathRetractionStrategy messing up certain traversals > > > Key: TINKERPOP-1597 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1597 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.3.0 >Reporter: Daniel Kuppitz >Assignee: Ted Wilmes > Fix For: 3.3.0 > > > There are certain types of traversals that do not work when > {{PathRetractionStrategy}} is included. This ticket discusses one specific > case that is only reproducible in {{3.3.0-SNAPSHOT}} / current {{master/}}. > Here's the traversal and the expected result: > {noformat} > gremlin> > g.withoutStrategies(PathRetractionStrategy).V(1,2).as("a").optional(bothE().dedup().as("b")).choose(select("b").select("a","b")., > project("a").by(select("a"))) > ==>[a:v[2]] > ==>[a:v[1],b:e[9][1-created->3]] > ==>[a:v[1],b:e[7][1-knows->2]] > ==>[a:v[1],b:e[8][1-knows->4]] > {noformat} > With {{PathRetractionStrategy}} included we get: > {noformat} > gremlin> > g.V(1,2).as("a").optional(bothE().dedup().as("b")).choose(select("b"), > select("a","b"), project("a").by(select("a"))) > ==>[a:v[2]] > {noformat} > What's interesting is, that the following 2 traversals compile to the same > final traversal (according to {{.explain()}}): > {noformat} > g.withoutStrategies(PathRetractionStrategy).V(1,2).as("a").optional(bothE().dedup().as("b")).choose(select("b").barrier(2500), > select("a","b").barrier(2500), project("a").by(select("a").barrier(2500))) > g.V(1,2).as("a").optional(bothE().dedup().as("b")).choose(select("b"), > select("a","b"), project("a").by(select("a"))) > {noformat} > ...but the one w/ {{PathRetractionStrategy}} still returns the wrong result, > while the other traversal returns the expected result. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] (TINKERPOP-1619) TinkerGraphComputer worker count affects OptionalStep query results
[ https://issues.apache.org/jira/browse/TINKERPOP-1619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847131#comment-15847131 ] Ted Wilmes commented on TINKERPOP-1619: --- Ok, that makes sense. I was wondering if when I bumped the worker count to 4 the duplicate edges were no longer being processed by the same worker and the dedup wasn't global. Kuppitz had a variation of this query as a test case for TINKERPOP-1597 so I'll go ahead and close that one out now. > TinkerGraphComputer worker count affects OptionalStep query results > --- > > Key: TINKERPOP-1619 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1619 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.3.0 >Reporter: Ted Wilmes > > I found that when I bump the worker count up to 4 on my local box, the dedup > nested in the optional appears to not be handled correctly. > {code} > gremlin> g = TinkerFactory.createModern().traversal() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] > gremlin> g.withComputer(Computer.compute().workers(3)).V(1, > 2).optional(bothE().dedup()) > ==>e[9][1-created->3] > ==>e[7][1-knows->2] > ==>e[8][1-knows->4] > ==>v[2] > gremlin> g.withComputer(Computer.compute().workers(4)).V(1, > 2).optional(bothE().dedup()) > ==>e[9][1-created->3] > ==>e[7][1-knows->2] > ==>e[7][1-knows->2] > ==>e[8][1-knows->4] > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] (TINKERPOP-1619) TinkerGraphComputer worker count affects OptionalStep query results
[ https://issues.apache.org/jira/browse/TINKERPOP-1619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847108#comment-15847108 ] Ted Wilmes commented on TINKERPOP-1619: --- That's a good question, {{tp32}} throws this exception with that query: {code} gremlin> g = TinkerFactory.createModern().traversal() gremlin> g.withComputer(Computer.compute().workers(3)).V(1, 2).optional(bothE().dedup()) [33/1896] java.lang.RuntimeException: java.lang.IllegalStateException: java.lang.IllegalArgumentException: The provided key is not a memory compute key: 1.2.1(1.1.0(2.0.0())) Type ':help' or ':h' for help. Display stack trace? [yN]y java.lang.IllegalStateException: java.lang.RuntimeException: java.lang.IllegalStateException: java.lang.IllegalArgumentException: The provided key is not a memory compute key: 1.2.1(1.1.0(2.0.0())) at org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.VertexProgramStep.processNextStart(VertexProgramStep.java:88) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50) at org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.ComputerResultStep.processNextStart(ComputerResultStep.java:68) at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:184) at org.apache.tinkerpop.gremlin.console.Console$_closure3.doCall(Console.groovy:237) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1027) at org.codehaus.groovy.tools.shell.Groovysh.setLastResult(Groovysh.groovy:447) at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:191) at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.super$3$execute(GremlinGroovysh.groovy) at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1218) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132) at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.execute(GremlinGroovysh.groovy:72) at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:122) at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:95) at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1218) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152) at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:124) at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:59) at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[jira] (TINKERPOP-1619) TinkerGraphComputer worker count affects OptionalStep query results
Ted Wilmes created TINKERPOP-1619: - Summary: TinkerGraphComputer worker count affects OptionalStep query results Key: TINKERPOP-1619 URL: https://issues.apache.org/jira/browse/TINKERPOP-1619 Project: TinkerPop Issue Type: Bug Components: process Affects Versions: 3.3.0 Reporter: Ted Wilmes I found that when I bump the worker count up to 4 on my local box, the dedup nested in the optional appears to not be handled correctly. {code} gremlin> g = TinkerFactory.createModern().traversal() ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] gremlin> g.withComputer(Computer.compute().workers(3)).V(1, 2).optional(bothE().dedup()) ==>e[9][1-created->3] ==>e[7][1-knows->2] ==>e[8][1-knows->4] ==>v[2] gremlin> g.withComputer(Computer.compute().workers(4)).V(1, 2).optional(bothE().dedup()) ==>e[9][1-created->3] ==>e[7][1-knows->2] ==>e[7][1-knows->2] ==>e[8][1-knows->4] {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (TINKERPOP-1583) PathRetractionStrategy retracts keys that are actually needed
[ https://issues.apache.org/jira/browse/TINKERPOP-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes closed TINKERPOP-1583. - Resolution: Fixed Fix Version/s: 3.2.4 > PathRetractionStrategy retracts keys that are actually needed > - > > Key: TINKERPOP-1583 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1583 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.3 >Reporter: Geoff Reedy >Assignee: Ted Wilmes > Fix For: 3.2.4 > > > We've seen this specifically for labels used in the until modulator of repeat > but I suspect it happens for other modulators as well. Here's a test case: > {code} > graph = TinkerGraph.open() > g = graph.traversal() > g.addV().as("first").repeat(addE("next").to(addV()).inV()).times(5).addE("next").to(select("first")).iterate() > g.V().limit(1).as('z').out().repeat(store('seen').out().where(without('seen'))).until(where(eq('z'))) > {code} > complains there is no z-key > I tired to fix it myself and submit a pull request but I found the > implementation of PathRetractionStrategy confusing. > One thing I noticed is that it seems the set of labels a step needs present > in order to work properly is determined external to the steps and that code > includes a lot of type-tests. If that logic were pushed down into the step > implementations I think fixing the repeat case would be easier and it would > be possible for extension steps to work properly with this strategy > (currently it seems they can't because of the closed-world assumption > inherent in the type-casing). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (TINKERPOP-1433) Add steps to dev docs to help committers get their keys in order
[ https://issues.apache.org/jira/browse/TINKERPOP-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes closed TINKERPOP-1433. - Resolution: Fixed Fix Version/s: 3.2.4 3.3.0 > Add steps to dev docs to help committers get their keys in order > > > Key: TINKERPOP-1433 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1433 > Project: TinkerPop > Issue Type: Improvement > Components: documentation >Affects Versions: 3.1.4 >Reporter: Ted Wilmes >Assignee: Ted Wilmes > Fix For: 3.3.0, 3.2.4 > > > I'll add some steps to help committers navigate the Apache PGP docs and will > include any TinkerPop specific steps (update validate-distribution.sh). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TINKERPOP-1597) PathRetractionStrategy messing up certain traversals
[ https://issues.apache.org/jira/browse/TINKERPOP-1597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15821928#comment-15821928 ] Ted Wilmes edited comment on TINKERPOP-1597 at 1/13/17 4:02 PM: The {{dedup}} step is incorrectly dropping labels because there is a bug related to child traversals. I'll clean up my quick and dirty test fix and submit a PR. was (Author: twilmes): The {{dedup}} step is incorrectly dropping labels because their is a bug related to child traversals. I'll clean up my quick and dirty test fix and submit a PR. > PathRetractionStrategy messing up certain traversals > > > Key: TINKERPOP-1597 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1597 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.3.0 >Reporter: Daniel Kuppitz >Assignee: Ted Wilmes > > There are certain types of traversals that do not work when > {{PathRetractionStrategy}} is included. This ticket discusses one specific > case that is only reproducible in {{3.3.0-SNAPSHOT}} / current {{master/}}. > Here's the traversal and the expected result: > {noformat} > gremlin> > g.withoutStrategies(PathRetractionStrategy).V(1,2).as("a").optional(bothE().dedup().as("b")).choose(select("b").select("a","b")., > project("a").by(select("a"))) > ==>[a:v[2]] > ==>[a:v[1],b:e[9][1-created->3]] > ==>[a:v[1],b:e[7][1-knows->2]] > ==>[a:v[1],b:e[8][1-knows->4]] > {noformat} > With {{PathRetractionStrategy}} included we get: > {noformat} > gremlin> > g.V(1,2).as("a").optional(bothE().dedup().as("b")).choose(select("b"), > select("a","b"), project("a").by(select("a"))) > ==>[a:v[2]] > {noformat} > What's interesting is, that the following 2 traversals compile to the same > final traversal (according to {{.explain()}}): > {noformat} > g.withoutStrategies(PathRetractionStrategy).V(1,2).as("a").optional(bothE().dedup().as("b")).choose(select("b").barrier(2500), > select("a","b").barrier(2500), project("a").by(select("a").barrier(2500))) > g.V(1,2).as("a").optional(bothE().dedup().as("b")).choose(select("b"), > select("a","b"), project("a").by(select("a"))) > {noformat} > ...but the one w/ {{PathRetractionStrategy}} still returns the wrong result, > while the other traversal returns the expected result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1597) PathRetractionStrategy messing up certain traversals
[ https://issues.apache.org/jira/browse/TINKERPOP-1597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15821928#comment-15821928 ] Ted Wilmes commented on TINKERPOP-1597: --- The {{dedup}} step is incorrectly dropping labels because their is a bug related to child traversals. I'll clean up my quick and dirty test fix and submit a PR. > PathRetractionStrategy messing up certain traversals > > > Key: TINKERPOP-1597 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1597 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.3.0 >Reporter: Daniel Kuppitz >Assignee: Ted Wilmes > > There are certain types of traversals that do not work when > {{PathRetractionStrategy}} is included. This ticket discusses one specific > case that is only reproducible in {{3.3.0-SNAPSHOT}} / current {{master/}}. > Here's the traversal and the expected result: > {noformat} > gremlin> > g.withoutStrategies(PathRetractionStrategy).V(1,2).as("a").optional(bothE().dedup().as("b")).choose(select("b").select("a","b")., > project("a").by(select("a"))) > ==>[a:v[2]] > ==>[a:v[1],b:e[9][1-created->3]] > ==>[a:v[1],b:e[7][1-knows->2]] > ==>[a:v[1],b:e[8][1-knows->4]] > {noformat} > With {{PathRetractionStrategy}} included we get: > {noformat} > gremlin> > g.V(1,2).as("a").optional(bothE().dedup().as("b")).choose(select("b"), > select("a","b"), project("a").by(select("a"))) > ==>[a:v[2]] > {noformat} > What's interesting is, that the following 2 traversals compile to the same > final traversal (according to {{.explain()}}): > {noformat} > g.withoutStrategies(PathRetractionStrategy).V(1,2).as("a").optional(bothE().dedup().as("b")).choose(select("b").barrier(2500), > select("a","b").barrier(2500), project("a").by(select("a").barrier(2500))) > g.V(1,2).as("a").optional(bothE().dedup().as("b")).choose(select("b"), > select("a","b"), project("a").by(select("a"))) > {noformat} > ...but the one w/ {{PathRetractionStrategy}} still returns the wrong result, > while the other traversal returns the expected result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TINKERPOP-1597) PathRetractionStrategy messing up certain traversals
[ https://issues.apache.org/jira/browse/TINKERPOP-1597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes reassigned TINKERPOP-1597: - Assignee: Ted Wilmes > PathRetractionStrategy messing up certain traversals > > > Key: TINKERPOP-1597 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1597 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.3.0 >Reporter: Daniel Kuppitz >Assignee: Ted Wilmes > > There are certain types of traversals that do not work when > {{PathRetractionStrategy}} is included. This ticket discusses one specific > case that is only reproducible in {{3.3.0-SNAPSHOT}} / current {{master/}}. > Here's the traversal and the expected result: > {noformat} > gremlin> > g.withoutStrategies(PathRetractionStrategy).V(1,2).as("a").optional(bothE().dedup().as("b")).choose(select("b").select("a","b")., > project("a").by(select("a"))) > ==>[a:v[2]] > ==>[a:v[1],b:e[9][1-created->3]] > ==>[a:v[1],b:e[7][1-knows->2]] > ==>[a:v[1],b:e[8][1-knows->4]] > {noformat} > With {{PathRetractionStrategy}} included we get: > {noformat} > gremlin> > g.V(1,2).as("a").optional(bothE().dedup().as("b")).choose(select("b"), > select("a","b"), project("a").by(select("a"))) > ==>[a:v[2]] > {noformat} > What's interesting is, that the following 2 traversals compile to the same > final traversal (according to {{.explain()}}): > {noformat} > g.withoutStrategies(PathRetractionStrategy).V(1,2).as("a").optional(bothE().dedup().as("b")).choose(select("b").barrier(2500), > select("a","b").barrier(2500), project("a").by(select("a").barrier(2500))) > g.V(1,2).as("a").optional(bothE().dedup().as("b")).choose(select("b"), > select("a","b"), project("a").by(select("a"))) > {noformat} > ...but the one w/ {{PathRetractionStrategy}} still returns the wrong result, > while the other traversal returns the expected result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TINKERPOP-1596) TinkerGraphComputer does not shutdown ComputerService Executor
Ted Wilmes created TINKERPOP-1596: - Summary: TinkerGraphComputer does not shutdown ComputerService Executor Key: TINKERPOP-1596 URL: https://issues.apache.org/jira/browse/TINKERPOP-1596 Project: TinkerPop Issue Type: Bug Components: tinkergraph Affects Versions: 3.2.3 Reporter: Ted Wilmes Assignee: Ted Wilmes I haven't seen any issues in normal usage, but while benchmarking `TinkerGraphComputer` with JMH, I noticed the forked JVM was hanging and had to be forcibly killed by JMH. {code} Thread[TinkerGraphComputer-boss,5,main] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) {code} The {{TinkerWorkerPool}} is being closed but not the {{ComputerService}} which causes the hang. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (TINKERPOP-1583) PathRetractionStrategy retracts keys that are actually needed
[ https://issues.apache.org/jira/browse/TINKERPOP-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on TINKERPOP-1583 started by Ted Wilmes. - > PathRetractionStrategy retracts keys that are actually needed > - > > Key: TINKERPOP-1583 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1583 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.3 >Reporter: Geoff Reedy >Assignee: Ted Wilmes > > We've seen this specifically for labels used in the until modulator of repeat > but I suspect it happens for other modulators as well. Here's a test case: > {code} > graph = TinkerGraph.open() > g = graph.traversal() > g.addV().as("first").repeat(addE("next").to(addV()).inV()).times(5).addE("next").to(select("first")).iterate() > g.V().limit(1).as('z').out().repeat(store('seen').out().where(without('seen'))).until(where(eq('z'))) > {code} > complains there is no z-key > I tired to fix it myself and submit a pull request but I found the > implementation of PathRetractionStrategy confusing. > One thing I noticed is that it seems the set of labels a step needs present > in order to work properly is determined external to the steps and that code > includes a lot of type-tests. If that logic were pushed down into the step > implementations I think fixing the repeat case would be easier and it would > be possible for extension steps to work properly with this strategy > (currently it seems they can't because of the closed-world assumption > inherent in the type-casing). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1583) PathRetractionStrategy retracts keys that are actually needed
[ https://issues.apache.org/jira/browse/TINKERPOP-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15730335#comment-15730335 ] Ted Wilmes commented on TINKERPOP-1583: --- Thanks for reporting this [~gereedy]. I'll take a look. > PathRetractionStrategy retracts keys that are actually needed > - > > Key: TINKERPOP-1583 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1583 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.3 >Reporter: Geoff Reedy > > We've seen this specifically for labels used in the until modulator of repeat > but I suspect it happens for other modulators as well. Here's a test case: > {code} > graph = TinkerGraph.open() > g = graph.traversal() > g.addV().as("first").repeat(addE("next").to(addV()).inV()).times(5).addE("next").to(select("first")).iterate() > g.V().limit(1).as('z').out().repeat(store('seen').out().where(without('seen'))).until(where(eq('z'))) > {code} > complains there is no z-key > I tired to fix it myself and submit a pull request but I found the > implementation of PathRetractionStrategy confusing. > One thing I noticed is that it seems the set of labels a step needs present > in order to work properly is determined external to the steps and that code > includes a lot of type-tests. If that logic were pushed down into the step > implementations I think fixing the repeat case would be easier and it would > be possible for extension steps to work properly with this strategy > (currently it seems they can't because of the closed-world assumption > inherent in the type-casing). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TINKERPOP-1583) PathRetractionStrategy retracts keys that are actually needed
[ https://issues.apache.org/jira/browse/TINKERPOP-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes reassigned TINKERPOP-1583: - Assignee: Ted Wilmes > PathRetractionStrategy retracts keys that are actually needed > - > > Key: TINKERPOP-1583 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1583 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.3 >Reporter: Geoff Reedy >Assignee: Ted Wilmes > > We've seen this specifically for labels used in the until modulator of repeat > but I suspect it happens for other modulators as well. Here's a test case: > {code} > graph = TinkerGraph.open() > g = graph.traversal() > g.addV().as("first").repeat(addE("next").to(addV()).inV()).times(5).addE("next").to(select("first")).iterate() > g.V().limit(1).as('z').out().repeat(store('seen').out().where(without('seen'))).until(where(eq('z'))) > {code} > complains there is no z-key > I tired to fix it myself and submit a pull request but I found the > implementation of PathRetractionStrategy confusing. > One thing I noticed is that it seems the set of labels a step needs present > in order to work properly is determined external to the steps and that code > includes a lot of type-tests. If that logic were pushed down into the step > implementations I think fixing the repeat case would be easier and it would > be possible for extension steps to work properly with this strategy > (currently it seems they can't because of the closed-world assumption > inherent in the type-casing). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (TINKERPOP-1548) Traversals can complete before interrupted in TraversalInterruptionComputerTest
[ https://issues.apache.org/jira/browse/TINKERPOP-1548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes closed TINKERPOP-1548. - Resolution: Fixed Fix Version/s: 3.2.4 > Traversals can complete before interrupted in > TraversalInterruptionComputerTest > --- > > Key: TINKERPOP-1548 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1548 > Project: TinkerPop > Issue Type: Bug > Components: test-suite >Affects Versions: 3.2.3 >Reporter: Ted Wilmes >Assignee: Ted Wilmes > Fix For: 3.2.4 > > > TinkerGraphComputer {{TraversalInterruptionComputerTest}} fails for me fairly > consistently. It appears that the test traversal is completing before > {{t.interrupt}} on occasion which causes the test to fail because the > {{TraversalInterruptedException}} is never thrown. I can reproduce this > failure more consistently when running TinkerGraphComputer with 1 worker. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (TINKERPOP-1548) Traversals can complete before interrupted in TraversalInterruptionComputerTest
[ https://issues.apache.org/jira/browse/TINKERPOP-1548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on TINKERPOP-1548 started by Ted Wilmes. - > Traversals can complete before interrupted in > TraversalInterruptionComputerTest > --- > > Key: TINKERPOP-1548 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1548 > Project: TinkerPop > Issue Type: Bug > Components: test-suite >Affects Versions: 3.2.3 >Reporter: Ted Wilmes >Assignee: Ted Wilmes > > TinkerGraphComputer {{TraversalInterruptionComputerTest}} fails for me fairly > consistently. It appears that the test traversal is completing before > {{t.interrupt}} on occasion which causes the test to fail because the > {{TraversalInterruptedException}} is never thrown. I can reproduce this > failure more consistently when running TinkerGraphComputer with 1 worker. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (TINKERPOP-965) Optimize TraversalStrategy application
[ https://issues.apache.org/jira/browse/TINKERPOP-965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes closed TINKERPOP-965. Resolution: Won't Fix Further profiling has shown that the new long pole in the tent is the PathRetractionStrategy. Strategy application is still running pretty quickly so we're in pretty good shape but I'd like to make it faster. Due to the complexity of the PathRetractionStrategy I will not risk introducing issues with a refactor for this release but will target performance improvements for 3.3. > Optimize TraversalStrategy application > -- > > Key: TINKERPOP-965 > URL: https://issues.apache.org/jira/browse/TINKERPOP-965 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.1.0-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > > From the work of [~twilmes], it appears that we can gain a lot by making > {{Traversal.applyStrategies()}} as fast as possible. This is going to be > especially important as we transition people away from using > "Blueprints-methods" (e.g. {{Graph.addVertex()}} and {{Vertex.addEdge()}}) to > "Gremlin-methods" (e.g. {{g.addV()}} and > {{g.V().as('a').out().addE('knows').from('a')}}). > Obviously the "Blueprints-methods" will always be faster given that they are > direct calls to the underlying provider. However, they should not be so > ridiculously faster that people won't use the "Gremlin-methods." Furthermore, > given [~dkuppitz]'s recent work with {{BulkLoaderVertexProgram}} and soon > TINKERPOP3-942, the "Gremlin-methods" will be the only way to get these > features. > In general, the work should go beyond the {{Mutation}}-steps and make it so > that all traversals compile as fast as possible. The first step is to realize > where things are slow and knock those out firsthopefully we realize some > tricks! > Related to: TINKERPOP3-957 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TINKERPOP-1548) Traversals can complete before interrupted in TraversalInterruptionComputerTest
Ted Wilmes created TINKERPOP-1548: - Summary: Traversals can complete before interrupted in TraversalInterruptionComputerTest Key: TINKERPOP-1548 URL: https://issues.apache.org/jira/browse/TINKERPOP-1548 Project: TinkerPop Issue Type: Bug Components: test-suite Affects Versions: 3.2.3 Reporter: Ted Wilmes Assignee: Ted Wilmes TinkerGraphComputer {{TraversalInterruptionComputerTest}} fails for me fairly consistently. It appears that the test traversal is completing before {{t.interrupt}} on occasion which causes the test to fail because the {{TraversalInterruptedException}} is never thrown. I can reproduce this failure more consistently when running TinkerGraphComputer with 1 worker. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (TINKERPOP-1468) GraphTraversal.optional throws a NPE if no traversal is provided
[ https://issues.apache.org/jira/browse/TINKERPOP-1468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes closed TINKERPOP-1468. - Resolution: Won't Fix Assignee: Ted Wilmes Since this is related to Groovy and a 0 arg method call (see [here|https://objectpartners.com/2015/09/02/groovy-gotcha-passing-zero-arguments-to-a-method-that-expects-one/] for details) we are closing this as "won't fix". > GraphTraversal.optional throws a NPE if no traversal is provided > > > Key: TINKERPOP-1468 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1468 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.2 >Reporter: Ted Wilmes >Assignee: Ted Wilmes > Labels: trivial > > {{GraphTraversal.optional}} throws a NPE if no traversal is provided. We > should probably throw a friendlier exception with instructions to provide a > traversal as an argument. > {code} > gremlin> g.V().optional() > java.lang.NullPointerException > Type ':help' or ':h' for help. > Display stack trace? [yN]y > java.lang.NullPointerException > at > org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal.optional(GraphTraversal.java:1316) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1468) GraphTraversal.optional throws a NPE if no traversal is provided
[ https://issues.apache.org/jira/browse/TINKERPOP-1468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15611913#comment-15611913 ] Ted Wilmes commented on TINKERPOP-1468: --- I haven't kept up as much with the GLV development, but thinking through that case where at some future point, I would think many folks would not be running an embedded graph db and instead would be using a client that would be passing Gremlin byte code, which would then be reconstructed on the server side, we'd probably avoid this issue anyway or maybe catch it with byte code level validation? Regardless, I'm okay with closing it "won't fix". > GraphTraversal.optional throws a NPE if no traversal is provided > > > Key: TINKERPOP-1468 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1468 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.2 >Reporter: Ted Wilmes > Labels: trivial > > {{GraphTraversal.optional}} throws a NPE if no traversal is provided. We > should probably throw a friendlier exception with instructions to provide a > traversal as an argument. > {code} > gremlin> g.V().optional() > java.lang.NullPointerException > Type ':help' or ':h' for help. > Display stack trace? [yN]y > java.lang.NullPointerException > at > org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal.optional(GraphTraversal.java:1316) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-965) Optimize TraversalStrategy application
[ https://issues.apache.org/jira/browse/TINKERPOP-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565513#comment-15565513 ] Ted Wilmes commented on TINKERPOP-965: -- Further profiling has shown that the new long pole in the tent is the {{PathRetractionStrategy}}. Strategy application is still running pretty quickly so we're in pretty good shape but I'd like to make it faster. Due to the complexity of the {{PathRetractionStrategy}} I will not risk introducing issues with a refactor for this release but will target performance improvements for 3.3. > Optimize TraversalStrategy application > -- > > Key: TINKERPOP-965 > URL: https://issues.apache.org/jira/browse/TINKERPOP-965 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.1.0-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > > From the work of [~twilmes], it appears that we can gain a lot by making > {{Traversal.applyStrategies()}} as fast as possible. This is going to be > especially important as we transition people away from using > "Blueprints-methods" (e.g. {{Graph.addVertex()}} and {{Vertex.addEdge()}}) to > "Gremlin-methods" (e.g. {{g.addV()}} and > {{g.V().as('a').out().addE('knows').from('a')}}). > Obviously the "Blueprints-methods" will always be faster given that they are > direct calls to the underlying provider. However, they should not be so > ridiculously faster that people won't use the "Gremlin-methods." Furthermore, > given [~dkuppitz]'s recent work with {{BulkLoaderVertexProgram}} and soon > TINKERPOP3-942, the "Gremlin-methods" will be the only way to get these > features. > In general, the work should go beyond the {{Mutation}}-steps and make it so > that all traversals compile as fast as possible. The first step is to realize > where things are slow and knock those out firsthopefully we realize some > tricks! > Related to: TINKERPOP3-957 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TINKERPOP-1287) StarGraph has an overdose of Stream usage.
[ https://issues.apache.org/jira/browse/TINKERPOP-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes updated TINKERPOP-1287: -- Attachment: g.V.out.out.count.svg > StarGraph has an overdose of Stream usage. > -- > > Key: TINKERPOP-1287 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1287 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop, structure >Affects Versions: 3.2.0-incubating, 3.1.2-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > Fix For: 3.2.3 > > Attachments: g.V.out.out.count.svg, stage0.svg, stage1.svg, stage2.svg > > > {{StarGraph}} is loaded with {{Stream}}-usage. Gutting streams from > TinkerGraph made it much faster. It would be good if we did the same thing > for {{StarGraph}}. > This can go into tp31/ and upmerge to master/. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1287) StarGraph has an overdose of Stream usage.
[ https://issues.apache.org/jira/browse/TINKERPOP-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565503#comment-15565503 ] Ted Wilmes commented on TINKERPOP-1287: --- I have done further comparison between master and a stream free {{StarGraph}} and determined there is no appreciable difference in runtime. The {{String.intern()}} calls are showing up and since we have removed all of the other uses of {{intern}} I think it's still worth taking these out and I'll submit a PR to do so. We could take the {{stream}} calls out even though I can't demonstrate an effect on runtime performance, but I'm very hesitant to do that at this point. I have attached profiling information gathered from running a {{g.V().out().out().count()}} on {{SparkGraphComputer}} (single node) against a graph with 1 million vertices and 10 million edges. Large portions of time are spent deserializing and then I need to do some further digging to confirm, but I believe a not insignificant amount of time is being spent on {{NoSuchElementException}} handling. I have seen similar overhead using {{TinkerGraphComputer}}. > StarGraph has an overdose of Stream usage. > -- > > Key: TINKERPOP-1287 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1287 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop, structure >Affects Versions: 3.2.0-incubating, 3.1.2-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > Fix For: 3.2.3 > > Attachments: stage0.svg, stage1.svg, stage2.svg > > > {{StarGraph}} is loaded with {{Stream}}-usage. Gutting streams from > TinkerGraph made it much faster. It would be good if we did the same thing > for {{StarGraph}}. > This can go into tp31/ and upmerge to master/. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1433) Add steps to dev docs to help committers get their keys in order
[ https://issues.apache.org/jira/browse/TINKERPOP-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15552512#comment-15552512 ] Ted Wilmes commented on TINKERPOP-1433: --- Yes, I need to do this and then incorporate a few other things that I ran into last release. I'll dust the cobwebs off during the prep for 3.1.5. > Add steps to dev docs to help committers get their keys in order > > > Key: TINKERPOP-1433 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1433 > Project: TinkerPop > Issue Type: Improvement > Components: documentation >Affects Versions: 3.1.4 >Reporter: Ted Wilmes >Assignee: Ted Wilmes > > I'll add some steps to help committers navigate the Apache PGP docs and will > include any TinkerPop specific steps (update validate-distribution.sh). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1404) Path/label optimization
[ https://issues.apache.org/jira/browse/TINKERPOP-1404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529426#comment-15529426 ] Ted Wilmes commented on TINKERPOP-1404: --- Apologies for not looking yet [~pietermartin]. Please go ahead and open a PR and I'll take a look this week. > Path/label optimization > --- > > Key: TINKERPOP-1404 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1404 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.2.1 >Reporter: pieter martin >Assignee: pieter martin > Labels: performance > > Currently path queries do a lot of label collection copying. This has a > significant impact on performance. > As the labels are known and set on the traverser when {{Traverser.split(r, > step)}} is called there is no need to call {{Traverser.addLabels}} again in > {{AbstractStep}} > Also seeing as {{AbstractStep.getLabels()}} returns an {{UnmodifyableSet}} > the step's labels can be used directly in the traverser. There is no need to > make a copy of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-965) Optimize TraversalStrategy application
[ https://issues.apache.org/jira/browse/TINKERPOP-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15527359#comment-15527359 ] Ted Wilmes commented on TINKERPOP-965: -- I've made some initial profiling progress but will be doing another round with the strategy additions and updates. Providing I can make some worthwhile performance improvements, I'll be targeting this for inclusion in the next release. > Optimize TraversalStrategy application > -- > > Key: TINKERPOP-965 > URL: https://issues.apache.org/jira/browse/TINKERPOP-965 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.1.0-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > > From the work of [~twilmes], it appears that we can gain a lot by making > {{Traversal.applyStrategies()}} as fast as possible. This is going to be > especially important as we transition people away from using > "Blueprints-methods" (e.g. {{Graph.addVertex()}} and {{Vertex.addEdge()}}) to > "Gremlin-methods" (e.g. {{g.addV()}} and > {{g.V().as('a').out().addE('knows').from('a')}}). > Obviously the "Blueprints-methods" will always be faster given that they are > direct calls to the underlying provider. However, they should not be so > ridiculously faster that people won't use the "Gremlin-methods." Furthermore, > given [~dkuppitz]'s recent work with {{BulkLoaderVertexProgram}} and soon > TINKERPOP3-942, the "Gremlin-methods" will be the only way to get these > features. > In general, the work should go beyond the {{Mutation}}-steps and make it so > that all traversals compile as fast as possible. The first step is to realize > where things are slow and knock those out firsthopefully we realize some > tricks! > Related to: TINKERPOP3-957 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TINKERPOP-1468) GraphTraversal.optional throws a NPE if no traversal is provided
Ted Wilmes created TINKERPOP-1468: - Summary: GraphTraversal.optional throws a NPE if no traversal is provided Key: TINKERPOP-1468 URL: https://issues.apache.org/jira/browse/TINKERPOP-1468 Project: TinkerPop Issue Type: Bug Components: process Affects Versions: 3.2.2 Reporter: Ted Wilmes {{GraphTraversal.optional}} throws a NPE if no traversal is provided. We should probably throw a friendlier exception with instructions to provide a traversal as an argument. {code} gremlin> g.V().optional() java.lang.NullPointerException Type ':help' or ':h' for help. Display stack trace? [yN]y java.lang.NullPointerException at org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal.optional(GraphTraversal.java:1316) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TINKERPOP-1433) Add steps to dev docs to help committers get their keys in order
Ted Wilmes created TINKERPOP-1433: - Summary: Add steps to dev docs to help committers get their keys in order Key: TINKERPOP-1433 URL: https://issues.apache.org/jira/browse/TINKERPOP-1433 Project: TinkerPop Issue Type: Improvement Components: documentation Affects Versions: 3.1.4 Reporter: Ted Wilmes Assignee: Ted Wilmes I'll add some steps to help committers navigate the Apache PGP docs and will include any TinkerPop specific steps (update validate-distribution.sh). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1415) Vertex programs that rely on getting path parameters can't work
[ https://issues.apache.org/jira/browse/TINKERPOP-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436838#comment-15436838 ] Ted Wilmes commented on TINKERPOP-1415: --- It's all coming back to me know, thanks [~okram]. [~bryncooke] in light of TINKERPOP-1280 and in the meantime, the availability of {{withPath}} would you be okay with closing this ticket out? > Vertex programs that rely on getting path parameters can't work > --- > > Key: TINKERPOP-1415 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1415 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.1 >Reporter: Bryn Cooke > > Recently some of our vertex programs have started failing. > They rely of being able to retrieve an element out of the path. > However, whereas this used to work now the path element isn't there anymore. > The root cause is that path retraction doesn't know that the vertex program > is interested in the path and therefore culls out parameter. > I can understand why it does this, but there seems to be no way of exposing > the vertex program path requirements such that PathRetractionStrategy can > pick them up and do the right thing. > I tried hacking together a solution by making ProgramVertexProgramStep > implement Scoping. This did fix the issue, but I am not confident that this > is the way to go. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (TINKERPOP-1415) Vertex programs that rely on getting path parameters can't work
[ https://issues.apache.org/jira/browse/TINKERPOP-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes updated TINKERPOP-1415: -- Comment: was deleted (was: Hi Bryn, do things work if you use {{withPath}} by chance like: {{g.withPath().V().program(...)}}? [~okram] do you think it would make sense to add a {{getRequirements}} to {{VertexProgram}} that would be bubbled up to the strategies via the {{ProgramVertexProgramStep}}?) > Vertex programs that rely on getting path parameters can't work > --- > > Key: TINKERPOP-1415 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1415 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.1 >Reporter: Bryn Cooke > > Recently some of our vertex programs have started failing. > They rely of being able to retrieve an element out of the path. > However, whereas this used to work now the path element isn't there anymore. > The root cause is that path retraction doesn't know that the vertex program > is interested in the path and therefore culls out parameter. > I can understand why it does this, but there seems to be no way of exposing > the vertex program path requirements such that PathRetractionStrategy can > pick them up and do the right thing. > I tried hacking together a solution by making ProgramVertexProgramStep > implement Scoping. This did fix the issue, but I am not confident that this > is the way to go. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1415) Vertex programs that rely on getting path parameters can't work
[ https://issues.apache.org/jira/browse/TINKERPOP-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15435078#comment-15435078 ] Ted Wilmes commented on TINKERPOP-1415: --- Hi Bryn, do things work if you use {{withPath}} by chance like: {{g.withPath().V().program(...)}}? [~okram] do you think it would make sense to add a {{getRequirements}} to {{VertexProgram}} that would be bubbled up to the strategies via the {{ProgramVertexProgramStep}}? > Vertex programs that rely on getting path parameters can't work > --- > > Key: TINKERPOP-1415 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1415 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.1 >Reporter: Bryn Cooke > > Recently some of our vertex programs have started failing. > They rely of being able to retrieve an element out of the path. > However, whereas this used to work now the path element isn't there anymore. > The root cause is that path retraction doesn't know that the vertex program > is interested in the path and therefore culls out parameter. > I can understand why it does this, but there seems to be no way of exposing > the vertex program path requirements such that PathRetractionStrategy can > pick them up and do the right thing. > I tried hacking together a solution by making ProgramVertexProgramStep > implement Scoping. This did fix the issue, but I am not confident that this > is the way to go. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1415) Vertex programs that rely on getting path parameters can't work
[ https://issues.apache.org/jira/browse/TINKERPOP-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15435076#comment-15435076 ] Ted Wilmes commented on TINKERPOP-1415: --- Hi Bryn, do things work if you use {{withPath}} by chance like: {{g.withPath().V().program(...)}}? [~okram] do you think it would make sense to add a {{getRequirements}} to {{VertexProgram}} that would be bubbled up to the strategies via the {{ProgramVertexProgramStep}}? > Vertex programs that rely on getting path parameters can't work > --- > > Key: TINKERPOP-1415 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1415 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.1 >Reporter: Bryn Cooke > > Recently some of our vertex programs have started failing. > They rely of being able to retrieve an element out of the path. > However, whereas this used to work now the path element isn't there anymore. > The root cause is that path retraction doesn't know that the vertex program > is interested in the path and therefore culls out parameter. > I can understand why it does this, but there seems to be no way of exposing > the vertex program path requirements such that PathRetractionStrategy can > pick them up and do the right thing. > I tried hacking together a solution by making ProgramVertexProgramStep > implement Scoping. This did fix the issue, but I am not confident that this > is the way to go. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1287) StarGraph has an overdose of Stream usage.
[ https://issues.apache.org/jira/browse/TINKERPOP-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424532#comment-15424532 ] Ted Wilmes commented on TINKERPOP-1287: --- I've completed the stream conversion on this one and been doing some testing. Unfortunately this is one of those cases where though the JMH micro benchmarks show large improvements across the board on the individual method calls, when I run full SparkGraphComputer OLAP ad-hoc queries on larger datasets, the improvements are lost in the noise of other areas that are really driving the response times. I still have some more testing to do to make sure my results are good and when I complete, I'll report back. > StarGraph has an overdose of Stream usage. > -- > > Key: TINKERPOP-1287 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1287 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop, structure >Affects Versions: 3.2.0-incubating, 3.1.2-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > Fix For: 3.2.2 > > Attachments: stage0.svg, stage1.svg, stage2.svg > > > {{StarGraph}} is loaded with {{Stream}}-usage. Gutting streams from > TinkerGraph made it much faster. It would be good if we did the same thing > for {{StarGraph}}. > This can go into tp31/ and upmerge to master/. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (TINKERPOP-1405) profile() doesn't like withPath()
[ https://issues.apache.org/jira/browse/TINKERPOP-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes closed TINKERPOP-1405. - > profile() doesn't like withPath() > - > > Key: TINKERPOP-1405 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1405 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.1 >Reporter: Daniel Kuppitz >Assignee: Ted Wilmes > Fix For: 3.2.2 > > > {{profile()}} fails when used in conjunction with {{withPath()}}: > {code} > gremlin> g.withPath().V().profile() > When specified, the profile()-Step must be the last step or followed only by > the cap()-step. > Display stack trace? [yN] > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TINKERPOP-1405) profile() doesn't like withPath()
[ https://issues.apache.org/jira/browse/TINKERPOP-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes resolved TINKERPOP-1405. --- Resolution: Fixed > profile() doesn't like withPath() > - > > Key: TINKERPOP-1405 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1405 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.1 >Reporter: Daniel Kuppitz >Assignee: Ted Wilmes > Fix For: 3.2.2 > > > {{profile()}} fails when used in conjunction with {{withPath()}}: > {code} > gremlin> g.withPath().V().profile() > When specified, the profile()-Step must be the last step or followed only by > the cap()-step. > Display stack trace? [yN] > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (TINKERPOP-1405) profile() doesn't like withPath()
[ https://issues.apache.org/jira/browse/TINKERPOP-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on TINKERPOP-1405 started by Ted Wilmes. - > profile() doesn't like withPath() > - > > Key: TINKERPOP-1405 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1405 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.1 >Reporter: Daniel Kuppitz >Assignee: Ted Wilmes > Fix For: 3.2.2 > > > {{profile()}} fails when used in conjunction with {{withPath()}}: > {code} > gremlin> g.withPath().V().profile() > When specified, the profile()-Step must be the last step or followed only by > the cap()-step. > Display stack trace? [yN] > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1405) profile() doesn't like withPath()
[ https://issues.apache.org/jira/browse/TINKERPOP-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421587#comment-15421587 ] Ted Wilmes commented on TINKERPOP-1405: --- I was wondering if the {{PathRetractionStrategy}} was causing issues with this so I took a look and it looks like it's actually having an issue because the {{withPath}} adds a {{RequirementsStep}} to the tail end of the traversal which breaks some of the {{StandardVerfificationStrategy}} logic. Looks like a straightforward fix so I can put a PR in shortly if something isn't already in the works. > profile() doesn't like withPath() > - > > Key: TINKERPOP-1405 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1405 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.1 >Reporter: Daniel Kuppitz > Fix For: 3.2.2 > > > {{profile()}} fails when used in conjunction with {{withPath()}}: > {code} > gremlin> g.withPath().V().profile() > When specified, the profile()-Step must be the last step or followed only by > the cap()-step. > Display stack trace? [yN] > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (TINKERPOP-1254) Support dropping traverser path information when it is no longer needed.
[ https://issues.apache.org/jira/browse/TINKERPOP-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes closed TINKERPOP-1254. - Resolution: Fixed > Support dropping traverser path information when it is no longer needed. > > > Key: TINKERPOP-1254 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1254 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.1.1-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > Fix For: 3.2.1 > > > The most expensive traversals (especially in OLAP) are those that can not be > "bulked." There are various reasons why two traversers at the same object can > not be bulked, but the primary reason is {{PATH}} or {{LABELED_PATH}}. That > is, when the history of the traverser is required, the probability of two > traversers having the same history is low. > A key to making traversals more efficient is to do as a much as possible to > remove historic information from a traverser so it can get bulked. How does > one do this? > {code} > g.V.as('a').out().as('b').out().where(neq('a').and().neq('b')).both().name > {code} > The {{LABELED_PATH}} of "a" and "b" are required up to the {{where()}} and at > which point, at {{both()}}, they are no longer required. It would be smart to > support: > {code} > traverser.dropLabels(Set) > traverser.dropPath() > {code} > We would then, via a {{TraversalOptimizationStrategy}} insert a step between > {{where()}} and {{both()}} called {{PathPruneStep}} which would be a > {{SideEffectStep}}. The strategy would know which labels were no longer > needed (via forward lookahead) and then do: > {code} > public class PathPruneStep { > final Set dropLabels = ... > final boolean dropPath = ... > public void sideEffect(final Traverser traverser) { > final Traverser start = this.starts.next(); > if(this.dropPath) start.dropPath(); > else start.dropLabels(labels); > } > } > {code} > Again, the more we can prune historic path data no longer needed, the higher > the probability of bulking. Think about this in terms of {{match()}}. > {code} > g.V().match( > a.out.b, > b.out.c, > c.neq.a, > c.out.b, > ).select("a") > {code} > All we need is "a" at the end. Thus, once a pattern has been passed and no > future patterns require that label, drop it! > This idea is related to TINKERPOP-331, but I don't think we should deal with > manipulating the species. Thus, I think 331 is too "low level." -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TINKERPOP-1254) Support dropping traverser path information when it is no longer needed.
[ https://issues.apache.org/jira/browse/TINKERPOP-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Wilmes updated TINKERPOP-1254: -- Fix Version/s: 3.2.1 > Support dropping traverser path information when it is no longer needed. > > > Key: TINKERPOP-1254 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1254 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.1.1-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > Fix For: 3.2.1 > > > The most expensive traversals (especially in OLAP) are those that can not be > "bulked." There are various reasons why two traversers at the same object can > not be bulked, but the primary reason is {{PATH}} or {{LABELED_PATH}}. That > is, when the history of the traverser is required, the probability of two > traversers having the same history is low. > A key to making traversals more efficient is to do as a much as possible to > remove historic information from a traverser so it can get bulked. How does > one do this? > {code} > g.V.as('a').out().as('b').out().where(neq('a').and().neq('b')).both().name > {code} > The {{LABELED_PATH}} of "a" and "b" are required up to the {{where()}} and at > which point, at {{both()}}, they are no longer required. It would be smart to > support: > {code} > traverser.dropLabels(Set) > traverser.dropPath() > {code} > We would then, via a {{TraversalOptimizationStrategy}} insert a step between > {{where()}} and {{both()}} called {{PathPruneStep}} which would be a > {{SideEffectStep}}. The strategy would know which labels were no longer > needed (via forward lookahead) and then do: > {code} > public class PathPruneStep { > final Set dropLabels = ... > final boolean dropPath = ... > public void sideEffect(final Traverser traverser) { > final Traverser start = this.starts.next(); > if(this.dropPath) start.dropPath(); > else start.dropLabels(labels); > } > } > {code} > Again, the more we can prune historic path data no longer needed, the higher > the probability of bulking. Think about this in terms of {{match()}}. > {code} > g.V().match( > a.out.b, > b.out.c, > c.neq.a, > c.out.b, > ).select("a") > {code} > All we need is "a" at the end. Thus, once a pattern has been passed and no > future patterns require that label, drop it! > This idea is related to TINKERPOP-331, but I don't think we should deal with > manipulating the species. Thus, I think 331 is too "low level." -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1254) Support dropping traverser path information when it is no longer needed.
[ https://issues.apache.org/jira/browse/TINKERPOP-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345136#comment-15345136 ] Ted Wilmes commented on TINKERPOP-1254: --- Thanks much for the review. I will be updating {{MatchStep}} step to do the dynamic label dropping at runtime. I'll get that taken care of, along with general code cleanup, ensure the unit tests all look good, and then do a round of benchmarking and include the results with the PR. > Support dropping traverser path information when it is no longer needed. > > > Key: TINKERPOP-1254 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1254 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.1.1-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > > The most expensive traversals (especially in OLAP) are those that can not be > "bulked." There are various reasons why two traversers at the same object can > not be bulked, but the primary reason is {{PATH}} or {{LABELED_PATH}}. That > is, when the history of the traverser is required, the probability of two > traversers having the same history is low. > A key to making traversals more efficient is to do as a much as possible to > remove historic information from a traverser so it can get bulked. How does > one do this? > {code} > g.V.as('a').out().as('b').out().where(neq('a').and().neq('b')).both().name > {code} > The {{LABELED_PATH}} of "a" and "b" are required up to the {{where()}} and at > which point, at {{both()}}, they are no longer required. It would be smart to > support: > {code} > traverser.dropLabels(Set) > traverser.dropPath() > {code} > We would then, via a {{TraversalOptimizationStrategy}} insert a step between > {{where()}} and {{both()}} called {{PathPruneStep}} which would be a > {{SideEffectStep}}. The strategy would know which labels were no longer > needed (via forward lookahead) and then do: > {code} > public class PathPruneStep { > final Set dropLabels = ... > final boolean dropPath = ... > public void sideEffect(final Traverser traverser) { > final Traverser start = this.starts.next(); > if(this.dropPath) start.dropPath(); > else start.dropLabels(labels); > } > } > {code} > Again, the more we can prune historic path data no longer needed, the higher > the probability of bulking. Think about this in terms of {{match()}}. > {code} > g.V().match( > a.out.b, > b.out.c, > c.neq.a, > c.out.b, > ).select("a") > {code} > All we need is "a" at the end. Thus, once a pattern has been passed and no > future patterns require that label, drop it! > This idea is related to TINKERPOP-331, but I don't think we should deal with > manipulating the species. Thus, I think 331 is too "low level." -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TINKERPOP-1254) Support dropping traverser path information when it is no longer needed.
[ https://issues.apache.org/jira/browse/TINKERPOP-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15334745#comment-15334745 ] Ted Wilmes edited comment on TINKERPOP-1254 at 6/16/16 10:23 PM: - [~okram] I pushed a very, very rough version. Not ready for PR yet as I need to cleanup code and add more tests. Having said that, if you're curious and want to take a look at it in its current state, I'd appreciate any feedback. I think I've captured the basic ideas. Diff can be seen at https://github.com/apache/tinkerpop/compare/master...TINKERPOP-1254 I was wondering what your thoughts were on unit testing the strategy application. It's different than the current strategy tests because I'm not rewriting the traversal, but instead setting the keep labels on certain steps. My simple idea was to just add a {{getKeepLabels}} to the {{PathProcessor}} interface so I could introspect for my unit tests, but thought you might have a better idea. was (Author: twilmes): @okram I pushed a very, very rough version. Not ready for PR yet as I need to cleanup code and add more tests. Having said that, if you're curious and want to take a look at it in its current state, I'd appreciate any feedback. I think I've captured the basic ideas. Diff can be seen at https://github.com/apache/tinkerpop/compare/master...TINKERPOP-1254 I was wondering what your thoughts were on unit testing the strategy application. It's different than the current strategy tests because I'm not rewriting the traversal, but instead setting the keep labels on certain steps. My simple idea was to just add a {{getKeepLabels}} to the {{PathProcessor}} interface so I could introspect for my unit tests, but thought you might have a better idea. > Support dropping traverser path information when it is no longer needed. > > > Key: TINKERPOP-1254 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1254 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.1.1-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > > The most expensive traversals (especially in OLAP) are those that can not be > "bulked." There are various reasons why two traversers at the same object can > not be bulked, but the primary reason is {{PATH}} or {{LABELED_PATH}}. That > is, when the history of the traverser is required, the probability of two > traversers having the same history is low. > A key to making traversals more efficient is to do as a much as possible to > remove historic information from a traverser so it can get bulked. How does > one do this? > {code} > g.V.as('a').out().as('b').out().where(neq('a').and().neq('b')).both().name > {code} > The {{LABELED_PATH}} of "a" and "b" are required up to the {{where()}} and at > which point, at {{both()}}, they are no longer required. It would be smart to > support: > {code} > traverser.dropLabels(Set) > traverser.dropPath() > {code} > We would then, via a {{TraversalOptimizationStrategy}} insert a step between > {{where()}} and {{both()}} called {{PathPruneStep}} which would be a > {{SideEffectStep}}. The strategy would know which labels were no longer > needed (via forward lookahead) and then do: > {code} > public class PathPruneStep { > final Set dropLabels = ... > final boolean dropPath = ... > public void sideEffect(final Traverser traverser) { > final Traverser start = this.starts.next(); > if(this.dropPath) start.dropPath(); > else start.dropLabels(labels); > } > } > {code} > Again, the more we can prune historic path data no longer needed, the higher > the probability of bulking. Think about this in terms of {{match()}}. > {code} > g.V().match( > a.out.b, > b.out.c, > c.neq.a, > c.out.b, > ).select("a") > {code} > All we need is "a" at the end. Thus, once a pattern has been passed and no > future patterns require that label, drop it! > This idea is related to TINKERPOP-331, but I don't think we should deal with > manipulating the species. Thus, I think 331 is too "low level." -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1254) Support dropping traverser path information when it is no longer needed.
[ https://issues.apache.org/jira/browse/TINKERPOP-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15334745#comment-15334745 ] Ted Wilmes commented on TINKERPOP-1254: --- @okram I pushed a very, very rough version. Not ready for PR yet as I need to cleanup code and add more tests. Having said that, if you're curious and want to take a look at it in its current state, I'd appreciate any feedback. I think I've captured the basic ideas. Diff can be seen at https://github.com/apache/tinkerpop/compare/master...TINKERPOP-1254 I was wondering what your thoughts were on unit testing the strategy application. It's different than the current strategy tests because I'm not rewriting the traversal, but instead setting the keep labels on certain steps. My simple idea was to just add a {{getKeepLabels}} to the {{PathProcessor}} interface so I could introspect for my unit tests, but thought you might have a better idea. > Support dropping traverser path information when it is no longer needed. > > > Key: TINKERPOP-1254 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1254 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.1.1-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > > The most expensive traversals (especially in OLAP) are those that can not be > "bulked." There are various reasons why two traversers at the same object can > not be bulked, but the primary reason is {{PATH}} or {{LABELED_PATH}}. That > is, when the history of the traverser is required, the probability of two > traversers having the same history is low. > A key to making traversals more efficient is to do as a much as possible to > remove historic information from a traverser so it can get bulked. How does > one do this? > {code} > g.V.as('a').out().as('b').out().where(neq('a').and().neq('b')).both().name > {code} > The {{LABELED_PATH}} of "a" and "b" are required up to the {{where()}} and at > which point, at {{both()}}, they are no longer required. It would be smart to > support: > {code} > traverser.dropLabels(Set) > traverser.dropPath() > {code} > We would then, via a {{TraversalOptimizationStrategy}} insert a step between > {{where()}} and {{both()}} called {{PathPruneStep}} which would be a > {{SideEffectStep}}. The strategy would know which labels were no longer > needed (via forward lookahead) and then do: > {code} > public class PathPruneStep { > final Set dropLabels = ... > final boolean dropPath = ... > public void sideEffect(final Traverser traverser) { > final Traverser start = this.starts.next(); > if(this.dropPath) start.dropPath(); > else start.dropLabels(labels); > } > } > {code} > Again, the more we can prune historic path data no longer needed, the higher > the probability of bulking. Think about this in terms of {{match()}}. > {code} > g.V().match( > a.out.b, > b.out.c, > c.neq.a, > c.out.b, > ).select("a") > {code} > All we need is "a" at the end. Thus, once a pattern has been passed and no > future patterns require that label, drop it! > This idea is related to TINKERPOP-331, but I don't think we should deal with > manipulating the species. Thus, I think 331 is too "low level." -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1254) Support dropping traverser path information when it is no longer needed.
[ https://issues.apache.org/jira/browse/TINKERPOP-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318584#comment-15318584 ] Ted Wilmes commented on TINKERPOP-1254: --- I'm getting close to ready for a review. I ended up switching over to keeping vs. dropping the labels. Drop works out well if a traversal is totally flat but becomes problematic when you're dealing with steps that take nested traversals. Currently I'm finishing up some fixes related to errors while running on graph computer, and then I'll push so we have something concrete to discuss. > Support dropping traverser path information when it is no longer needed. > > > Key: TINKERPOP-1254 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1254 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.1.1-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > > The most expensive traversals (especially in OLAP) are those that can not be > "bulked." There are various reasons why two traversers at the same object can > not be bulked, but the primary reason is {{PATH}} or {{LABELED_PATH}}. That > is, when the history of the traverser is required, the probability of two > traversers having the same history is low. > A key to making traversals more efficient is to do as a much as possible to > remove historic information from a traverser so it can get bulked. How does > one do this? > {code} > g.V.as('a').out().as('b').out().where(neq('a').and().neq('b')).both().name > {code} > The {{LABELED_PATH}} of "a" and "b" are required up to the {{where()}} and at > which point, at {{both()}}, they are no longer required. It would be smart to > support: > {code} > traverser.dropLabels(Set) > traverser.dropPath() > {code} > We would then, via a {{TraversalOptimizationStrategy}} insert a step between > {{where()}} and {{both()}} called {{PathPruneStep}} which would be a > {{SideEffectStep}}. The strategy would know which labels were no longer > needed (via forward lookahead) and then do: > {code} > public class PathPruneStep { > final Set dropLabels = ... > final boolean dropPath = ... > public void sideEffect(final Traverser traverser) { > final Traverser start = this.starts.next(); > if(this.dropPath) start.dropPath(); > else start.dropLabels(labels); > } > } > {code} > Again, the more we can prune historic path data no longer needed, the higher > the probability of bulking. Think about this in terms of {{match()}}. > {code} > g.V().match( > a.out.b, > b.out.c, > c.neq.a, > c.out.b, > ).select("a") > {code} > All we need is "a" at the end. Thus, once a pattern has been passed and no > future patterns require that label, drop it! > This idea is related to TINKERPOP-331, but I don't think we should deal with > manipulating the species. Thus, I think 331 is too "low level." -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1254) Support dropping traverser path information when it is no longer needed.
[ https://issues.apache.org/jira/browse/TINKERPOP-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308767#comment-15308767 ] Ted Wilmes commented on TINKERPOP-1254: --- I like it. I'll follow this approach and, like you mentioned, see if I can just update {{PathProcessor}} with a {{setDropLabels}} and update {{MatchStep}} to implement it. Thanks for the thoughts on the approach. > Support dropping traverser path information when it is no longer needed. > > > Key: TINKERPOP-1254 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1254 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.1.1-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > > The most expensive traversals (especially in OLAP) are those that can not be > "bulked." There are various reasons why two traversers at the same object can > not be bulked, but the primary reason is {{PATH}} or {{LABELED_PATH}}. That > is, when the history of the traverser is required, the probability of two > traversers having the same history is low. > A key to making traversals more efficient is to do as a much as possible to > remove historic information from a traverser so it can get bulked. How does > one do this? > {code} > g.V.as('a').out().as('b').out().where(neq('a').and().neq('b')).both().name > {code} > The {{LABELED_PATH}} of "a" and "b" are required up to the {{where()}} and at > which point, at {{both()}}, they are no longer required. It would be smart to > support: > {code} > traverser.dropLabels(Set) > traverser.dropPath() > {code} > We would then, via a {{TraversalOptimizationStrategy}} insert a step between > {{where()}} and {{both()}} called {{PathPruneStep}} which would be a > {{SideEffectStep}}. The strategy would know which labels were no longer > needed (via forward lookahead) and then do: > {code} > public class PathPruneStep { > final Set dropLabels = ... > final boolean dropPath = ... > public void sideEffect(final Traverser traverser) { > final Traverser start = this.starts.next(); > if(this.dropPath) start.dropPath(); > else start.dropLabels(labels); > } > } > {code} > Again, the more we can prune historic path data no longer needed, the higher > the probability of bulking. Think about this in terms of {{match()}}. > {code} > g.V().match( > a.out.b, > b.out.c, > c.neq.a, > c.out.b, > ).select("a") > {code} > All we need is "a" at the end. Thus, once a pattern has been passed and no > future patterns require that label, drop it! > This idea is related to TINKERPOP-331, but I don't think we should deal with > manipulating the species. Thus, I think 331 is too "low level." -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1254) Support dropping traverser path information when it is no longer needed.
[ https://issues.apache.org/jira/browse/TINKERPOP-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308248#comment-15308248 ] Ted Wilmes commented on TINKERPOP-1254: --- Not sure if this makes sense, but I'm in the thick of this right now and taking an approach where I build a dependency tree for the match traversals. Contrived example here to illustrate: {code} match( __.as('a').out().as('b'), (1) __.as('b').out().as('c'), (2) __.as('c').out().as('d'), (3) __.as('b').out().as('e')(4) ).select('d', 'e').dropLabels('d', 'e') {code} Traversal (2) and (4) depend on (1), Traversal (3) depends on (2). The results of (3) and (4) are referenced in the {{select}}. This info is used to insert the following dropLabel steps: {code} match( __.as('a').out().as('b'), __.as('b').dropLabels('b').out().as('c'), __.as('c').dropLabels('c').out().as('d') __.as('b').dropLabels('b').out().as('e') ).select('d', 'e').dropLabels('d', 'e') {code} My thinking is even though the order execution is determined at runtime, we can figure out beforehand which match traversals depend on each other and insert the drop steps accordingly. Does this make sense or have I made some incorrect assumptions on dependencies between match traversals? > Support dropping traverser path information when it is no longer needed. > > > Key: TINKERPOP-1254 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1254 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.1.1-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > > The most expensive traversals (especially in OLAP) are those that can not be > "bulked." There are various reasons why two traversers at the same object can > not be bulked, but the primary reason is {{PATH}} or {{LABELED_PATH}}. That > is, when the history of the traverser is required, the probability of two > traversers having the same history is low. > A key to making traversals more efficient is to do as a much as possible to > remove historic information from a traverser so it can get bulked. How does > one do this? > {code} > g.V.as('a').out().as('b').out().where(neq('a').and().neq('b')).both().name > {code} > The {{LABELED_PATH}} of "a" and "b" are required up to the {{where()}} and at > which point, at {{both()}}, they are no longer required. It would be smart to > support: > {code} > traverser.dropLabels(Set) > traverser.dropPath() > {code} > We would then, via a {{TraversalOptimizationStrategy}} insert a step between > {{where()}} and {{both()}} called {{PathPruneStep}} which would be a > {{SideEffectStep}}. The strategy would know which labels were no longer > needed (via forward lookahead) and then do: > {code} > public class PathPruneStep { > final Set dropLabels = ... > final boolean dropPath = ... > public void sideEffect(final Traverser traverser) { > final Traverser start = this.starts.next(); > if(this.dropPath) start.dropPath(); > else start.dropLabels(labels); > } > } > {code} > Again, the more we can prune historic path data no longer needed, the higher > the probability of bulking. Think about this in terms of {{match()}}. > {code} > g.V().match( > a.out.b, > b.out.c, > c.neq.a, > c.out.b, > ).select("a") > {code} > All we need is "a" at the end. Thus, once a pattern has been passed and no > future patterns require that label, drop it! > This idea is related to TINKERPOP-331, but I don't think we should deal with > manipulating the species. Thus, I think 331 is too "low level." -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1254) Support dropping traverser path information when it is no longer needed.
[ https://issues.apache.org/jira/browse/TINKERPOP-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307845#comment-15307845 ] Ted Wilmes commented on TINKERPOP-1254: --- I'm pretty close to PR-ready. Hopefully have something pushed later this week and ready for an in-depth review. I need to make a few final updates to the strategy and I also need to do some further testing to make sure my retract on the {{ImmutablePath}} doesn't lead to performance degradation as the number of Immutable paths temporarily increases. > Support dropping traverser path information when it is no longer needed. > > > Key: TINKERPOP-1254 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1254 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.1.1-incubating >Reporter: Marko A. Rodriguez >Assignee: Ted Wilmes > > The most expensive traversals (especially in OLAP) are those that can not be > "bulked." There are various reasons why two traversers at the same object can > not be bulked, but the primary reason is {{PATH}} or {{LABELED_PATH}}. That > is, when the history of the traverser is required, the probability of two > traversers having the same history is low. > A key to making traversals more efficient is to do as a much as possible to > remove historic information from a traverser so it can get bulked. How does > one do this? > {code} > g.V.as('a').out().as('b').out().where(neq('a').and().neq('b')).both().name > {code} > The {{LABELED_PATH}} of "a" and "b" are required up to the {{where()}} and at > which point, at {{both()}}, they are no longer required. It would be smart to > support: > {code} > traverser.dropLabels(Set) > traverser.dropPath() > {code} > We would then, via a {{TraversalOptimizationStrategy}} insert a step between > {{where()}} and {{both()}} called {{PathPruneStep}} which would be a > {{SideEffectStep}}. The strategy would know which labels were no longer > needed (via forward lookahead) and then do: > {code} > public class PathPruneStep { > final Set dropLabels = ... > final boolean dropPath = ... > public void sideEffect(final Traverser traverser) { > final Traverser start = this.starts.next(); > if(this.dropPath) start.dropPath(); > else start.dropLabels(labels); > } > } > {code} > Again, the more we can prune historic path data no longer needed, the higher > the probability of bulking. Think about this in terms of {{match()}}. > {code} > g.V().match( > a.out.b, > b.out.c, > c.neq.a, > c.out.b, > ).select("a") > {code} > All we need is "a" at the end. Thus, once a pattern has been passed and no > future patterns require that label, drop it! > This idea is related to TINKERPOP-331, but I don't think we should deal with > manipulating the species. Thus, I think 331 is too "low level." -- This message was sent by Atlassian JIRA (v6.3.4#6332)