GitHub user spmallette opened a pull request:
https://github.com/apache/tinkerpop/pull/893
TINKERPOP-1996 io()
https://issues.apache.org/jira/browse/TINKERPOP-1996
Introduces `io()` with full support for GLVs and both OLAP and OLTP.
There's a bit too much to type here to fully describe this - please take a look
at the documentation in the PR as it goes into the details.
All tests pass with `docker/build.sh -t -n -i`
VOTE +1
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/tinkerpop TINKERPOP-1996
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/tinkerpop/pull/893.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #893
----
commit ec1d05f74fa875e4a07699dc89c3d1956aab586f
Author: Stephen Mallette <spmva@...>
Date: 2018-06-29T19:04:17Z
TINKERPOP-1996 Introduce read() and write() steps
commit 0785090f67ff88a834f77c7181a69594049c65ea
Author: Stephen Mallette <spmva@...>
Date: 2018-07-10T16:21:52Z
TINKERPOP-1996 Have the basics of OLAP read()/write() steps working
This is still fairly skeletal at this point. Just trying to make sure
things work properly before building read()/write() out fully.
commit ff2773a5ef1eff379cb5f3b809d4677ff64a076d
Author: Stephen Mallette <spmva@...>
Date: 2018-07-10T16:55:06Z
TINKERPOP-1996 Minor refactoring of Reading/Writing and javadoc
commit d99909c4e19fcb3ec3b8a8c94eb054a4332ad219
Author: Stephen Mallette <spmva@...>
Date: 2018-07-10T18:53:52Z
TINKERPOP-1996 read()/write() api changes for return type
No more weird Map status return for read() and write(). Both just work like
a terminator and self iterate to return nothing.
commit 767d65b9d54f1774b13cc5a61db711268f213ff1
Author: Stephen Mallette <spmva@...>
Date: 2018-07-11T14:20:39Z
TINKERPOP-1996 Made read()/write() terminator steps
Without this approach the with() operator couldn't be used because the
traversal would already be iterated on the call to read() and write(). In this
way read() and write() are both terminators and modulators at the same time.
commit d181563d24a401ac6550e06d86d78b12a8f16f51
Author: Stephen Mallette <spmva@...>
Date: 2018-07-11T14:36:11Z
TINKERPOP-1996 Added some javadoc and code formatting
commit 13e552b289b5266fcb418baaf2ae90626f6055c4
Author: Stephen Mallette <spmva@...>
Date: 2018-07-11T19:40:39Z
TINKERPOP-1996 Added with() options for io()
Included GraphReader and GraphWriter detection and added tests
commit be9db8de6a9d2abb5222b7ab5b9326049060e85a
Author: Stephen Mallette <spmva@...>
Date: 2018-07-12T12:17:56Z
TINKERPOP-1996 none() doesn't need to be removed in HadoopIoStrategy
Not sure why this was there in the first place. Removing it not allows
Hadoop integration tests to pass, but seems to have no real effect on existing
operations.
commit 328737a371f8a2040d02f9c2dbb06d049ce3c881
Author: Stephen Mallette <spmva@...>
Date: 2018-07-12T15:19:10Z
TINKERPOP-1996 Added IO to imports and javadoc fixes
commit ae796378e07925f9385f3ec65c10022b59aab8b5
Author: Stephen Mallette <spmva@...>
Date: 2018-07-12T18:33:03Z
TINKERPOP-1996 Deprecated Graph.io() and related infrastructure.
commit 9e4da0149247a50277e2a468b0becf892426ce2e
Author: Stephen Mallette <spmva@...>
Date: 2018-07-13T15:37:02Z
TINKERPOP-1996 Fixed a bad method call for Configuring steps
commit bd275a7ffa4f0d04634c830aa4f7577375c7944c
Author: Stephen Mallette <spmva@...>
Date: 2018-07-13T15:37:21Z
TINKERPOP-1996 Removed OptOuts for read()/write() tests
Not necessary because existing checks ignore these. For read() you can't
write to a HadoopGraph directly (i.e. create vertices/edges) and for write()
(and technically read()) it is ignored as it requires a GraphComputer to work.
commit f148e9331a945e0f4f707ea937b610e5902701c7
Author: Stephen Mallette <spmva@...>
Date: 2018-07-13T19:17:17Z
TINKERPOP-1996 Got read/write() tests running for OLAP
Introduced new Graph.Features to provider better separation between graph
mutation features and graph loading features - they are two different things as
demonstrated by io(). Fixed HadoopIoStep/Strategy so that they properly handle
the different input/output format types expected.
commit 576649fd5456f6390bf9481d01438a7e78db083e
Author: Stephen Mallette <spmva@...>
Date: 2018-07-13T19:22:43Z
TINKERPOP-1996 Updated changelog
commit 62175c228b77bdbda96c11015f2974828df8f3aa
Author: Stephen Mallette <spmva@...>
Date: 2018-07-13T21:31:46Z
TINKERPOP-1996 Added docs for io()
Killed all the old IO documentation that utilized the GraphReader/Writer
classes directly as well as the Graph.io() method that is now deprecated.
commit 6d05805ada657bcb3f50a60aa0c313c29d4611bb
Author: Stephen Mallette <spmva@...>
Date: 2018-07-14T10:23:54Z
TINKERPOP-1996 Moved IoStep implementations to sideEffect package
These steps really aren't quite sideEffects and not quite map steps either
but they seem to fit better as sideEffect. meh
commit 5bf19e2d35aa2f050ecca0a7b2190553b0422720
Author: Stephen Mallette <spmva@...>
Date: 2018-07-16T13:29:01Z
TINKERPOP-1996 Removed use of graph.io() in docs
commit 8187016886b8b699ff107c9b6a7dfe95deb7e4a1
Author: Stephen Mallette <spmva@...>
Date: 2018-07-16T16:10:02Z
TINKERPOP-1996 Used g.io() in tests by default
commit f8e3b8a1df1639405a29f261a9d7da147e88b356
Author: Stephen Mallette <spmva@...>
Date: 2018-07-16T18:47:47Z
TINKERPOP-1996 Removed unecessary enum
commit 048ea21c0adf9297233d06222e75c64dbe1fa1ca
Author: Stephen Mallette <spmva@...>
Date: 2018-07-19T17:30:02Z
TINKERPOP-1996 Added support/testing for io() in GLVs
Had to revert to using iterate() and stop read/write() from terminating the
traversal. Kinda stinks, but we rely on iterate() quite heavily and for
remoting allowing read()/write() to terminate means that the traversal will
execute during traversal construction in the translator (which is early and
potentially bad).
commit a580b6fda232ce6dd035e4c261a53d0f0dc69f83
Author: Stephen Mallette <spmva@...>
Date: 2018-07-19T18:08:25Z
TINKERPOP-1996 Enabled feature coverage checks for GLV tests on
read()/write()
commit ae3b149789b8a9779d0549cee8a6ca2e2febbd2b
Author: Stephen Mallette <spmva@...>
Date: 2018-07-19T18:24:15Z
TINKERPOP-1996 Added iterate() to read()/write() steps in docs
commit ff71c6abee0b39d7ee95128c3d64906daad96a76
Author: Stephen Mallette <spmva@...>
Date: 2018-07-19T20:13:57Z
TINKERPOP-1996 Added support for setting IoRegistries using with()
IORegistry instances are important because they feed serializer information
to the Reader/Writer instances. Of all the configuration options that one
seemed like the most important to make possible using with().
commit 94233970f8dc7c5d7b8fdc4b9e7bdbf0ff905c25
Author: Stephen Mallette <spmva@...>
Date: 2018-07-20T10:31:42Z
TINKERPOP-1996 Fixed bad test assertions after last body of changes.
commit 51dc82122af6b8d8c783de58a0451b8b2071c051
Author: Stephen Mallette <spmva@...>
Date: 2018-07-20T11:15:07Z
TINKERPOP-1996 Added some docs around IO.registry
commit 4d979cf8dc4482d574191a76a0c60e281f887563
Author: Stephen Mallette <spmva@...>
Date: 2018-07-20T11:35:48Z
TINKERPOP-1996 Pass configurations from with() through to Hadoop
This will allow users to override or add to the Hadoop/Spark/OLAP
configuration as needed
commit c97d747f4b05cef42eb0e53adae50fcdee083a2e
Author: Stephen Mallette <spmva@...>
Date: 2018-07-20T14:30:00Z
TINKERPOP-1996 Fixed up general strategy application around io()
The GraphComputer was not being set properly in the HadoopIoStep and
therefore executions of OLAP runs would not work even if
withComputer(SparkGraphComputer) was set. It only worked if the
gremlin.hadoop.defaultGraphComputer property was set which was weird.
commit 8fd3bf21efac092f619254245ba83614327c4dcb
Author: Stephen Mallette <spmva@...>
Date: 2018-07-20T14:47:20Z
TINKERPOP-1996 Undeprecated some Io related classes
These classes still have use as part of IoRegistry which is still in use
and I don't see a clear way to get rid of that easily. We'd have to change the
whole system for serialization configuration to accomplish that so I guess this
stuff stays for now.
commit ded7c187480a68b6f7be2d0cb777461f57a2871c
Author: Stephen Mallette <spmva@...>
Date: 2018-07-20T15:20:03Z
TINKERPOP-1996 Added upgrade docs
commit e6e4413e46594ae8f6d408de4e3d3bd85c228f53
Author: Stephen Mallette <spmva@...>
Date: 2018-07-20T15:47:24Z
TINKERPOP-1996 Verification strategy to prevent io() from misuse
----
---
