[GitHub] tinkerpop pull request #783: Initial commit for sparql-gremlin changes
GitHub user harsh9t opened a pull request: https://github.com/apache/tinkerpop/pull/783 Initial commit for sparql-gremlin changes Summary: * added sparql-gremlin folder for adding the sparql-gremlin plugin 0.2 to the tinkerpop project TODOs: * add a readme * add test cases * standardize API to communicate with tinkerpop You can merge this pull request into a Git repository by running: $ git pull https://github.com/harsh9t/tinkerpop sparql-gremlin Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tinkerpop/pull/783.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #783 commit 9584373f822f5cbcc3a70dd94d324b2a3d414af3 Author: harsh9t Date: 2018-01-18T00:33:29Z Initial commit for sparql-gremlin changes ---
[jira] [Commented] (TINKERPOP-1489) Provide a Javascript Gremlin Language Variant
[ https://issues.apache.org/jira/browse/TINKERPOP-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16329671#comment-16329671 ] ASF GitHub Bot commented on TINKERPOP-1489: --- Github user jbmusso commented on the issue: https://github.com/apache/tinkerpop/pull/695 Finally found some time. Wew. Well, this PR is very well crafted - well done! My comments are only minor: * About `utils.toPromise` - if I understand you well, you want a dual callback/promise API for most async functions? * In `graph-traversal.js`, most methods of `GraphTraversalSource` and `GraphTraversal` and function attached to `statics` could maybe be dynamically created from an array of method names and dynamically added to these classes/object. I don’t know what would be the performance implication of this, but I don’t think there should be any unless V8 really can’t figure out what’s going on when parsing that file. Hopefully it's smart and figures out that the class is not changing. That’d lower the file size and help maintainability a lot. * ES6, most likely friendlier for more recent versions of V8 and supported by Node.js v4+ (see [Node green](http://node.green)): * `arguments` is deprecated and is replaced by `...args` for variadic functions * most `function` keywords could be replaced by arrow functions (lexical scoping and/or concise syntax). I tend to keep `function` for top-level functions, and use fat arrows everywhere even when `this` binding isn't needed (ie. callback w/o `this`) * `array.splice(0)` could be replaced by `const copy = [...original]` * `func.apply(null, arguments)` could be replaced by `func(...args)` when first argument `this` value is indeed meant to be `null` * `package.json`: `./node_modules/.bin` is added to the `$PATH` by `npm` or `yarn`, so we could just use `"test": "mocha test/unit test/integration -t 5000"`. Yay npm! I can fork and push 4 distinct commits for this if you want, so this can be cherry-picked. A more major update would be to author in ES6/7/8 and add a transpilation step, so all runtime could use code that they can optimize best. Using babel with [babel-preset-env](https://www.npmjs.com/package/babel-preset-env) combined with [postinstall-build](https://www.npmjs.com/package/postinstall-build) is an option. This will ensure that latest versions of Node.js use mostly non-transpiled code, while older versions automatically transpile what is strictly needed. That would make things more performant for latest versions of Node.js, since V8 optimizes a lot for new syntax, while still making the GLV compatible for older versions of Node.js. The nice thing is that the build step is automatically handled at install time by npm, so no extra maven coding should be required. I think such approach could be added in future releases and is out of scope today. Also, I'm ok to transfer/donate the "gremlin" package name to Apache TinkerPop so this can be published under this name. > Provide a Javascript Gremlin Language Variant > - > > Key: TINKERPOP-1489 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1489 > Project: TinkerPop > Issue Type: Improvement > Components: javascript >Affects Versions: 3.2.5 >Reporter: Jorge Bay >Priority: Major > > It would be nice to have a Javascript Gremlin Language Variant that could > work with any ES5 runtime, specially the ones that support > [CommonJs|http://requirejs.org/docs/commonjs.html], like Node.js. > Nashorn, the engine shipped with JDK 8+, does not implement CommonJs but > provides [additional > extensions|https://wiki.openjdk.java.net/display/Nashorn/Nashorn+extensions] > making modular JavaScript possible. Nashorn should be supported in order to > run glv tests under the same infrastructure (JDK8). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] tinkerpop issue #695: TINKERPOP-1489 JavaScript GLV
Github user jbmusso commented on the issue: https://github.com/apache/tinkerpop/pull/695 Finally found some time. Wew. Well, this PR is very well crafted - well done! My comments are only minor: * About `utils.toPromise` - if I understand you well, you want a dual callback/promise API for most async functions? * In `graph-traversal.js`, most methods of `GraphTraversalSource` and `GraphTraversal` and function attached to `statics` could maybe be dynamically created from an array of method names and dynamically added to these classes/object. I donât know what would be the performance implication of this, but I donât think there should be any unless V8 really canât figure out whatâs going on when parsing that file. Hopefully it's smart and figures out that the class is not changing. Thatâd lower the file size and help maintainability a lot. * ES6, most likely friendlier for more recent versions of V8 and supported by Node.js v4+ (see [Node green](http://node.green)): * `arguments` is deprecated and is replaced by `...args` for variadic functions * most `function` keywords could be replaced by arrow functions (lexical scoping and/or concise syntax). I tend to keep `function` for top-level functions, and use fat arrows everywhere even when `this` binding isn't needed (ie. callback w/o `this`) * `array.splice(0)` could be replaced by `const copy = [...original]` * `func.apply(null, arguments)` could be replaced by `func(...args)` when first argument `this` value is indeed meant to be `null` * `package.json`: `./node_modules/.bin` is added to the `$PATH` by `npm` or `yarn`, so we could just use `"test": "mocha test/unit test/integration -t 5000"`. Yay npm! I can fork and push 4 distinct commits for this if you want, so this can be cherry-picked. A more major update would be to author in ES6/7/8 and add a transpilation step, so all runtime could use code that they can optimize best. Using babel with [babel-preset-env](https://www.npmjs.com/package/babel-preset-env) combined with [postinstall-build](https://www.npmjs.com/package/postinstall-build) is an option. This will ensure that latest versions of Node.js use mostly non-transpiled code, while older versions automatically transpile what is strictly needed. That would make things more performant for latest versions of Node.js, since V8 optimizes a lot for new syntax, while still making the GLV compatible for older versions of Node.js. The nice thing is that the build step is automatically handled at install time by npm, so no extra maven coding should be required. I think such approach could be added in future releases and is out of scope today. Also, I'm ok to transfer/donate the "gremlin" package name to Apache TinkerPop so this can be published under this name. ---
[GitHub] tinkerpop pull request #782: Initial commit for sparql-gremlin changes
GitHub user harsh9t opened a pull request: https://github.com/apache/tinkerpop/pull/782 Initial commit for sparql-gremlin changes Summary: - added sparql-gremlin folder for adding the sparql-gremlin plugin 0.2 to the tinkerpop project TODOs: - add a readme - add test cases - standardize API to communicate with tinkerpop You can merge this pull request into a Git repository by running: $ git pull https://github.com/harsh9t/tinkerpop sparql-gremlin Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tinkerpop/pull/782.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #782 commit 3fdab38743e22b6d3e535c1f86c62e17a11d9aa4 Author: harsh9t Date: 2017-12-14T00:12:46Z cleaning phase 2 revised commit 97d90ca2625b4dada399793c55ff2b5ddfe57696 Author: harsh9t Date: 2018-01-17T22:50:14Z removed bsbm graph file ---
[jira] [Commented] (TINKERPOP-1861) VertexProgram create with varargs for Graphs
[
https://issues.apache.org/jira/browse/TINKERPOP-1861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16329154#comment-16329154
]
ASF GitHub Bot commented on TINKERPOP-1861:
---
Github user PBGraff commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/772#discussion_r162138049
--- Diff:
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/VertexProgram.java
---
@@ -219,13 +234,28 @@ public default void workerIterationEnd(final Memory
memory) {
* @paramThe vertex program type
* @return the newly constructed vertex program
*/
+@Deprecated
--- End diff --
Done in new commit. Will push with other changes.
> VertexProgram create with varargs for Graphs
>
>
> Key: TINKERPOP-1861
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1861
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
>Affects Versions: 3.2.7
>Reporter: Philip Graff
>Priority: Minor
>
> VertexProgram.Builder.create(Graph) can be modified to
> VertexProgram.Builder.create(Graph...) so that passing in zero or many graphs
> is naturally handled. The current state of passing in null when no graph is
> needed is bad practice.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] tinkerpop pull request #772: TINKERPOP-1861 Modify VertexProgram Builder to ...
Github user PBGraff commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/772#discussion_r162138049
--- Diff:
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/VertexProgram.java
---
@@ -219,13 +234,28 @@ public default void workerIterationEnd(final Memory
memory) {
* @paramThe vertex program type
* @return the newly constructed vertex program
*/
+@Deprecated
--- End diff --
Done in new commit. Will push with other changes.
---
[jira] [Commented] (TINKERPOP-1489) Provide a Javascript Gremlin Language Variant
[ https://issues.apache.org/jira/browse/TINKERPOP-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16328873#comment-16328873 ] ASF GitHub Bot commented on TINKERPOP-1489: --- Github user jorgebay commented on the issue: https://github.com/apache/tinkerpop/pull/695 Thanks @dkuppitz for looking into the maven issue! I'll rebase it and add a `g:T` deserializer. > Provide a Javascript Gremlin Language Variant > - > > Key: TINKERPOP-1489 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1489 > Project: TinkerPop > Issue Type: Improvement > Components: javascript >Affects Versions: 3.2.5 >Reporter: Jorge Bay >Priority: Major > > It would be nice to have a Javascript Gremlin Language Variant that could > work with any ES5 runtime, specially the ones that support > [CommonJs|http://requirejs.org/docs/commonjs.html], like Node.js. > Nashorn, the engine shipped with JDK 8+, does not implement CommonJs but > provides [additional > extensions|https://wiki.openjdk.java.net/display/Nashorn/Nashorn+extensions] > making modular JavaScript possible. Nashorn should be supported in order to > run glv tests under the same infrastructure (JDK8). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] tinkerpop issue #695: TINKERPOP-1489 JavaScript GLV
Github user jorgebay commented on the issue: https://github.com/apache/tinkerpop/pull/695 Thanks @dkuppitz for looking into the maven issue! I'll rebase it and add a `g:T` deserializer. ---
[jira] [Commented] (TINKERPOP-1870) n^2 synchronious operation in OLAP WorkerExecutor.execute() method
[
https://issues.apache.org/jira/browse/TINKERPOP-1870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16328828#comment-16328828
]
ASF GitHub Bot commented on TINKERPOP-1870:
---
Github user artem-aliev commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/778#discussion_r162065650
--- Diff:
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/gryo/GryoVersion.java
---
@@ -305,6 +307,7 @@ public String getVersion() {
add(GryoTypeReg.of(RangeGlobalStep.RangeBiOperator.class,
114));
add(GryoTypeReg.of(OrderGlobalStep.OrderBiOperator.class, 118,
new JavaSerializer()));
add(GryoTypeReg.of(ProfileStep.ProfileBiOperator.class, 119));
+add(GryoTypeReg.of(VertexTraverserSet.class, 173));
--- End diff --
master PR: https://github.com/apache/tinkerpop/pull/781
I have make both tp32 and tp33 ids the same. so I skipped 171,172 in tp32
> n^2 synchronious operation in OLAP WorkerExecutor.execute() method
> --
>
> Key: TINKERPOP-1870
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1870
> Project: TinkerPop
> Issue Type: Improvement
>Affects Versions: 3.2.7, 3.3.1
>Reporter: Artem Aliev
>Priority: Major
> Attachments: findTraverser1.png, findTraverser2.png,
> findTraverserFixed.png
>
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/WorkerExecutor.java#L80-L93]
> This block of code iterates over all remote traverses to select one related
> to the current vertex and remove it. This operation is repeated for the next
> vertex and so one. For following example query it means n^2 operations (n is
> number of vertices). All of them in sync block. multi core spark executor
> will do this operations serial.
> {code}
> g.V().emit().repeat(both().dedup()).count().next()
> {code}
> See jvisualvm screenshot.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] tinkerpop pull request #778: TINKERPOP-1870: Extends TraverserSet to have Ve...
Github user artem-aliev commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/778#discussion_r162065650
--- Diff:
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/gryo/GryoVersion.java
---
@@ -305,6 +307,7 @@ public String getVersion() {
add(GryoTypeReg.of(RangeGlobalStep.RangeBiOperator.class,
114));
add(GryoTypeReg.of(OrderGlobalStep.OrderBiOperator.class, 118,
new JavaSerializer()));
add(GryoTypeReg.of(ProfileStep.ProfileBiOperator.class, 119));
+add(GryoTypeReg.of(VertexTraverserSet.class, 173));
--- End diff --
master PR: https://github.com/apache/tinkerpop/pull/781
I have make both tp32 and tp33 ids the same. so I skipped 171,172 in tp32
---
[GitHub] tinkerpop pull request #781: Tinkerpop 1870 master
GitHub user artem-aliev opened a pull request: https://github.com/apache/tinkerpop/pull/781 Tinkerpop 1870 master You can merge this pull request into a Git repository by running: $ git pull https://github.com/artem-aliev/tinkerpop TINKERPOP-1870-master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tinkerpop/pull/781.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #781 commit a6ac037f0582187a47186264e59a1e42a14a7d1f Author: artemaliev Date: 2018-01-15T15:23:49Z TINKERPOP-1870: Extends TraverserSet to have Vertex index for remote traversers That replaces linear search in reversal traversal interator with hashtable lookup. commit ff05f07f082c87c788827143fc0d64ec38073005 Author: artemaliev Date: 2018-01-17T14:17:47Z Merge branch 'TINKERPOP-1870' into TINKERPOP-1870-master ---
[jira] [Commented] (TINKERPOP-1872) Apply edgeFunction in SparkMessenger
[ https://issues.apache.org/jira/browse/TINKERPOP-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16328637#comment-16328637 ] ASF GitHub Bot commented on TINKERPOP-1872: --- Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/779 > my IDE is not ok with it. please see: http://tinkerpop.apache.org/docs/current/dev/developer/#_ide_setup_with_intellij > So, should I make this change only on tp32 branch, and leave the master branch to you guys? as this does not look like a complicated change, you can just issue the single PR to tp32 and we can handle the merged to master. thanks > Apply edgeFunction in SparkMessenger > > > Key: TINKERPOP-1872 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1872 > Project: TinkerPop > Issue Type: Bug > Components: hadoop >Affects Versions: 3.2.7 >Reporter: stephen mallette >Priority: Major > > It seem the edgeFunction defined in Local MessageScope will not be applied > anywhere when you are using SparkGraphComputer. It will cause a difference > in result with what you will have when you are using TinkerGraphComputer. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] tinkerpop issue #779: TINKERPOP-1872 Apply edgeFunction in SparkMessenger
Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/779 > my IDE is not ok with it. please see: http://tinkerpop.apache.org/docs/current/dev/developer/#_ide_setup_with_intellij > So, should I make this change only on tp32 branch, and leave the master branch to you guys? as this does not look like a complicated change, you can just issue the single PR to tp32 and we can handle the merged to master. thanks ---
[jira] [Commented] (TINKERPOP-1872) Apply edgeFunction in SparkMessenger
[ https://issues.apache.org/jira/browse/TINKERPOP-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16328588#comment-16328588 ] ASF GitHub Bot commented on TINKERPOP-1872: --- Github user Stargazer4869 commented on the issue: https://github.com/apache/tinkerpop/pull/779 As for which branch should I apply this code changes to. This issue does occur on the tp32 branch too. So, should I make this change only on tp32 branch, and leave the master branch to you guys? > Apply edgeFunction in SparkMessenger > > > Key: TINKERPOP-1872 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1872 > Project: TinkerPop > Issue Type: Bug > Components: hadoop >Affects Versions: 3.2.7 >Reporter: stephen mallette >Priority: Major > > It seem the edgeFunction defined in Local MessageScope will not be applied > anywhere when you are using SparkGraphComputer. It will cause a difference > in result with what you will have when you are using TinkerGraphComputer. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] tinkerpop issue #779: TINKERPOP-1872 Apply edgeFunction in SparkMessenger
Github user Stargazer4869 commented on the issue: https://github.com/apache/tinkerpop/pull/779 As for which branch should I apply this code changes to. This issue does occur on the tp32 branch too. So, should I make this change only on tp32 branch, and leave the master branch to you guys? ---
[jira] [Commented] (TINKERPOP-1872) Apply edgeFunction in SparkMessenger
[ https://issues.apache.org/jira/browse/TINKERPOP-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16328578#comment-16328578 ] ASF GitHub Bot commented on TINKERPOP-1872: --- Github user Stargazer4869 commented on the issue: https://github.com/apache/tinkerpop/pull/779 Still working on the test case. A little question here, although maven can generate a jar file for the module gremlin-shade, my IDE is not ok with it. It can find dependencies which should provided by this module, such as: import org.apache.tinkerpop.shaded.kryo.Kryo; Any way I can deal with this? It will be great help. My IDE is IntelliJ, if there is a solutions for Eclipse, it will be fine too. > Apply edgeFunction in SparkMessenger > > > Key: TINKERPOP-1872 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1872 > Project: TinkerPop > Issue Type: Bug > Components: hadoop >Affects Versions: 3.2.7 >Reporter: stephen mallette >Priority: Major > > It seem the edgeFunction defined in Local MessageScope will not be applied > anywhere when you are using SparkGraphComputer. It will cause a difference > in result with what you will have when you are using TinkerGraphComputer. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] tinkerpop issue #779: TINKERPOP-1872 Apply edgeFunction in SparkMessenger
Github user Stargazer4869 commented on the issue: https://github.com/apache/tinkerpop/pull/779 Still working on the test case. A little question here, although maven can generate a jar file for the module gremlin-shade, my IDE is not ok with it. It can find dependencies which should provided by this module, such as: import org.apache.tinkerpop.shaded.kryo.Kryo; Any way I can deal with this? It will be great help. My IDE is IntelliJ, if there is a solutions for Eclipse, it will be fine too. ---
