Here are a couple ideas :
Simple : If we go the docker way, generate a test result recap file with
testIds that correspond to TinkerPop enforced tests. That way the test suit
can determine if a test was not performed or if a test failed. The testIds
would work something along the lines of :
#1 : establish connection
#2 : test text/script for g.V()
#3 : test bytecode for g.V()
#4 : etc..
So your resulting recap file could be as simple as the above with
true/false markers next to the IDs. This means we still need to code the
tests in each language but at least there's good quality control.
Complexe: Allow GLVs to reverse Bytecode into their language.
[
[V, 1]
[outE, 'created']
[inV]
[repeat, [
[out, 'created']
[in, 'created']
]
[times, 5]
[valueMap, 'name', 'age']
]
would become
graph.traversal().V(1).outE('created').inV().repeat(out('created').in('created')).times(5).valueMap('name',
'age') in gremlin-groovy
$graph->traversal()->
V(1)->outE('created')->inV()->repeat($graph->traversal()->out('created')->in('created'))->times(5)->valueMap('name',
'age') in gremlin-php
etc...
This would have the advantage of allowing users to seamlessly translate
gremlin-x to their own language or any other (imagine documentation where
you can let users switch to their language dynamically or generate Cypher
from your glv, etc.). So long as lambdas aren't involved of course.
This could also allow us to make some TestProcessor for gremlin-server that
would submit code to exec to the clients and then test the request
messages. In which case all GLV test cases would be covered in one spot and
submitted to the GLVs. This has the downside of not really allowing lower
level driver testing though.
Crazy/impossible : Do the above but turn Bytecode into a language agnostic
language. Run any code you want in any language ;p just sayin'
On Tue, Aug 16, 2016 at 9:45 AM, Stephen Mallette <spmalle...@gmail.com>
wrote:
> Yeah - (1) is stinky. The more I think about it the more I don't think
> there is value in validating gremlin-python over Jython, if Jython is not
> up to date and, perhaps more importantly, not the environment users will
> run their code in. Doesn't seem to have much value in that case. Perhaps
> for 3.2.2 we don't have to try to tackle this testing issue completely.
>
> On Tue, Aug 16, 2016 at 9:31 AM, Dylan Millikin <dylan.milli...@gmail.com>
> wrote:
>
> > For 1) maybe now is a good time to look into using docker for testing?
> That
> > would have it's own issues in the fact that testing would have to be
> > written directly in python so you would loose reusability for future GLVs
> > though.
> >
> > I'm not sure how to build a test suit that would either work across all
> > languages or enforce behaviors in all GLVs. I know this came up before I
> > just can't seem to find those issues. Might not be searching properly.
> >
> >
> > On Mon, Aug 15, 2016 at 4:29 PM, Stephen Mallette <spmalle...@gmail.com>
> > wrote:
> >
> > > After some review and tweaks, this PR from David just merged which got
> > the
> > > Python RemoteConnection going on TINKERPOP-1278:
> > >
> > > https://github.com/apache/tinkerpop/pull/379
> > >
> > > Still some work to do here:
> > >
> > > 1. Proper testing for Python's RemoteConnection implementation - ran
> into
> > > problems getting everything running under jython which is sorta out of
> > date
> > > with the latest python stuff. still trying to figure out what we want
> to
> > do
> > > there with respect to testing. basically we get some dependency errors
> > > around Python's fcntl, possibly related to: http://bugs.jython.org/
> > > issue1943
> > > - not completely sure what to do about that. On one hand we're using
> > Jython
> > > to test client side operations in the JVM environment so that we can
> get
> > > some re-use out of the full gremlin test suite (that's the good side).
> On
> > > the downside, no one will likely execute their gremlin-python code in
> > > Jython, we get bound to python 2.x where Jython appears largely stuck
> and
> > > we can't really work our way out of that bug anyway. Anyway, Jython
> > still
> > > remains useful on the server where it will let us process python
> lambdas
> > > embedded in traversals - so even though you'd be bound to jython and
> > python
> > > 2.x for those lambdas that's still pretty neat. Solving this testing
> > issue
> > > is a top priority I think.
> > >
> > > 2. Making an actual "driver". Dave's PR focused on getting a Python
> > > RemoteConnection built, but it would be nice if the code that sent the
> > > request to the server was extracted a bit so that users could
> > instantiate a
> > > low-level driver to submit raw scripts and custom request messages. In
> > that
> > > way it would be in line with how gremlin-driver works for Java and we'd
> > > have some consistency for our first two languages in this department.
> > >
> > > 3. Returning side-effects. I documented the new TraversalOpProcessor in
> > the
> > > TINKERPOP-1278 branch ( https://github.com/apache/
> > > tinkerpop/blob/TINKERPOP-1278/docs/src/dev/provider/index.
> > > asciidoc#traversal-opprocessor ) and it describes how Traversal
> > > side-effects are treated. We need to get that piece working
> > >
> > > 4. I sense that GraphSON 2.0 is going to be important for this branch.
> We
> > > really need embedded types and the GraphSON 1.0 ones are clunky. I'd
> hate
> > > to take us down a path of trying to build GLVs on the clunkiness only
> to
> > > have that clunkiness removed in the very near future.
> > >
> > > If anyone has any thoughts on any of this, especially on the issues we
> > face
> > > with item 1, please feel free to participate.
> > >
> > >
> > >
> > > On Sat, Aug 6, 2016 at 10:52 AM, Stephen Mallette <
> spmalle...@gmail.com>
> > > wrote:
> > >
> > > > Coming from TinkerPop 2.x with lots of repos, I think we'd want to
> try
> > to
> > > > avoid that (even though this situation is a little different). Before
> > we
> > > go
> > > > that route I think it would be helpful to understand what the
> > challenges
> > > > are in integrating a particular build environment into TinkerPop's.
> For
> > > > example, I imagine that a C# GLV would present some "interesting"
> > issues
> > > to
> > > > overcome. Perhaps you already have PHP in mind as a "problem".
> Anyway,
> > I
> > > > just think that we should deal with those problems as we identify
> them
> > > and
> > > > focus on the less troublesome to integrate languages first. I'm sure
> > > that
> > > > we can figure out something with Apache when/if the time comes that
> the
> > > > build complexity consumes us. Maybe docker can help solve some of our
> > > > build/test environment problems?.
> > > >
> > > > On Sat, Aug 6, 2016 at 10:28 AM, Dylan Millikin <
> > > dylan.milli...@gmail.com>
> > > > wrote:
> > > >
> > > >> That makes perfect sense Stephen. Quick question regarding the last
> > > point
> > > >> (testing environments and projects). Do you know what apache
> policies
> > > are
> > > >> in regards to maintaining multiple repos for "official"
> plugins/GLVs?
> > > This
> > > >> might make streamlining testing and updating the package in
> dependency
> > > >> management systems easier (it's actually mandatory for php
> composer).
> > > >> If you don't I can always bring it up with infra.
> > > >>
> > > >> On Sat, Aug 6, 2016 at 6:47 AM, Stephen Mallette <
> > spmalle...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Let me first address this piece:
> > > >> >
> > > >> > > My understanding around the driver needing modifications is -I'm
> > > >> > guessing- mostly to accommodate for the updates in TINKERPOP-1278
> > > rather
> > > >> > than from drivers needing to be specific for GLVs. If not let me
> > know.
> > > >> >
> > > >> > The modifications are mostly to trim down the size of
> gremlinclient
> > > and
> > > >> > simplify. gremlinclient allowed for pluggable webscocket
> > > implementations
> > > >> > and other stuff that adds some complexity/code we could be better
> > > >> without
> > > >> > in TinkerPop. There are some specific updates needed for full
> > support
> > > of
> > > >> > GLVs. (i.e. collecting side-effects which i think can remain
> > optional)
> > > >> but
> > > >> > (I think) David's work can already submit bytecode (if not, then
> > we'll
> > > >> need
> > > >> > to change that too, but that's pretty trivial.
> > > >> >
> > > >> > As for the rest, which largely boils down to "consider depending
> on
> > > >> > third-party drivers":
> > > >> >
> > > >> > I should have included some reasoning around your suggestion in my
> > > >> original
> > > >> > post as this was a position in my mind that was considered. The
> > > problem
> > > >> > with depending on third-party projects are the downsides you laid
> > out
> > > >> and I
> > > >> > think those are more than we should bear. I would also add to that
> > > list
> > > >> > that the dependency is cyclical. The third party project depends
> on
> > us
> > > >> then
> > > >> > TinkerPop depends back on them. I don't imagine that this will
> > > >> necessarily
> > > >> > cause a "compilation" problem but it doesn't seem quite
> > natural/right.
> > > >> >
> > > >> > > Would it not be better to have GLV's simply depend on third
> party
> > > >> > projects when they exist and are decently up to date)?
> > > >> >
> > > >> > The part in parenthesis is part of the problem in the long run.
> Code
> > > >> gets
> > > >> > stale, people move on. It's easy to write a gremlin-driver on the
> > > >> weekend,
> > > >> > but to build a driver on-par with the performance/features of the
> > java
> > > >> > driver - that's a difference story. Note that neo4j i think
> > discovered
> > > >> this
> > > >> > which is why they in-housed their drivers for bolt.
> > > >> >
> > > >> > > If features are lacking the community can provide patches, etc.
> > > >> >
> > > >> > We could try and I would think most driver developers would be
> > > amenable
> > > >> to
> > > >> > that, however some might not. Also the direction of an external
> > > project,
> > > >> > might not match the direction TinkerPop wants. Apache governance
> > > doesn't
> > > >> > extend to third-party projects so in the worst case, if we hit a
> > wall
> > > on
> > > >> > some technical issue that the owner of the project doesn't agree
> > with,
> > > >> we'd
> > > >> > be left with "find something else". In that case, TinkerPop has
> > > already
> > > >> > spent a lot of time sending patches to that third-party and our
> time
> > > >> would
> > > >> > have be wasted.
> > > >> >
> > > >> > > If a project dies we can always switch it out for another one.
> > > >> >
> > > >> > I don't know if it's that simple. Again, we'd (TinkerPop)
> presumably
> > > >> > invested time getting a project on par with the java driver in
> terms
> > > of
> > > >> > features and performance. So first of all, is there another
> project
> > to
> > > >> > switch to (right now there's not many second options for different
> > > >> > languages)? And second, how much effort will there be to re-invest
> > in
> > > a
> > > >> > different driver to get it consistent with the
> features/performance
> > of
> > > >> the
> > > >> > java driver?
> > > >> >
> > > >> > > I just think that as we get more GLVs, providing CI with the
> > correct
> > > >> > environments
> > > >> > to run various lower level driver tests in various languages is
> > going
> > > >> to be
> > > >> > a headache.
> > > >> >
> > > >> > Agreed. I worried about that too. However, this wont' be a fast
> > > process.
> > > >> > We're not on a tear to build as many GLVs as possible. We're
> trying
> > to
> > > >> get
> > > >> > the model right with gremlin-python, set a high bar for the build
> > > >> > automation needed, and then look at how to apply that elsewhere.
> If
> > we
> > > >> can
> > > >> > attain that same high bar, then we include a GLV. If we can't,
> then
> > we
> > > >> wait
> > > >> > until we can figure out how to. At least that's how I think about
> > it.
> > > >> >
> > > >> > I'm not sure if any of that changes your position, but I believe
> the
> > > >> risks
> > > >> > of depending on third-party drivers is a bit too high - at least
> > > higher
> > > >> > than us having the extra code and build automation problems of
> > owning
> > > >> the
> > > >> > drivers ourselves.
> > > >> >
> > > >> >
> > > >> > On Sat, Aug 6, 2016 at 1:26 AM, Dylan Millikin <
> > > >> dylan.milli...@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> > > If I may suggest an alternative. Most (all?) languages have
> > > dependency
> > > >> > > management.
> > > >> > > Would it not be better to have GLV's simply depend on third
> party
> > > >> > projects
> > > >> > > (when they exist and are decently up to date)?
> > > >> > > These dependencies can be pulled during testing. If features are
> > > >> lacking
> > > >> > > the community can provide patches, etc. If a project dies we can
> > > >> always
> > > >> > > switch it out for another one.
> > > >> > > When it comes to users, they simply download their GLV and go.
> > > Ideally
> > > >> > the
> > > >> > > GLV would exist as a package in their dependency management
> system
> > > of
> > > >> > > choice. (as a side note apt-get/yum install gremlin-server would
> > be
> > > >> > > amazing)
> > > >> > >
> > > >> > > My understanding around the driver needing modifications is -I'm
> > > >> > guessing-
> > > >> > > mostly to accommodate for the updates in TINKERPOP-1278 rather
> > than
> > > >> from
> > > >> > > drivers needing to be specific for GLVs. If not let me know.
> > > >> > >
> > > >> > > This obviously comes with it's own downsides. If a project dies
> > and
> > > >> > there's
> > > >> > > no replacement we basically have to fall back to your proposal
> by
> > > >> forking
> > > >> > > and maintaining it ourselves. I also don't know how licensing
> > would
> > > >> work
> > > >> > in
> > > >> > > this case, but I'm guessing a license change could be akin to
> the
> > > >> project
> > > >> > > dying in regards to TP's dependance on it.
> > > >> > >
> > > >> > > I just think that as we get more GLVs, providing CI with the
> > correct
> > > >> > > environments to run various lower level driver tests in various
> > > >> languages
> > > >> > > is going to be a headache. It would be much better to keep them
> as
> > > >> > separate
> > > >> > > projects.
> > > >> > >
> > > >> > > If I completely missed the original intent of the thread also do
> > let
> > > >> me
> > > >> > > know.
> > > >> > >
> > > >> > >
> > > >> > > On Fri, Aug 5, 2016 at 4:57 PM, Stephen Mallette <
> > > >> spmalle...@gmail.com>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > I mentioned this thread to David Brown and asked if he thought
> > > that
> > > >> > > > gremlinclient would be a good fit for this issue. He said that
> > he
> > > >> > thought
> > > >> > > > it would be with some modifications. He's agreed to simplify
> > > >> > gremliclient
> > > >> > > > with the intent of making it the driver for TinkerPop's
> > > >> gremlin-python.
> > > >> > > > Assuming no objections, he said that he could look to have a
> > pull
> > > >> > request
> > > >> > > > to TINKERPOP-1278 by end of next week.
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > > On Thu, Aug 4, 2016 at 1:39 PM, Stephen Mallette <
> > > >> spmalle...@gmail.com
> > > >> > >
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > TinkerPop has long held to the idea that "drivers" would be
> a
> > > bit
> > > >> > like
> > > >> > > > > graph implementations. TinkerPop builds reference
> > implementation
> > > >> of a
> > > >> > > > graph
> > > >> > > > > and third-parties maintain "other" implementations
> separately.
> > > The
> > > >> > same
> > > >> > > > > idea has been held for "drivers" in that we have our
> reference
> > > >> > > > > implementation of the Java driver and third-parties produce
> > > >> > > > implementations
> > > >> > > > > for other languages.
> > > >> > > > >
> > > >> > > > > GLVs sorta changed things though. Now we have a model where
> we
> > > >> > maintain
> > > >> > > > > GLVs of other languages in TinkerPop itself. While somewhat
> > > risky
> > > >> to
> > > >> > > > > include the added code/complexity, maintaining the GLV in
> > > >> TinkerPop
> > > >> > > has a
> > > >> > > > > lot of good advantages for users and for growing our
> > community.
> > > We
> > > >> > also
> > > >> > > > > have come up with ways to manage the risk, by establishing a
> > > >> really
> > > >> > > nice
> > > >> > > > > initial reference implementation of gremlin-python which
> sets
> > a
> > > >> high
> > > >> > > bar
> > > >> > > > > for new GLVs to come in (i.e. hooked into tinkerpop test
> > suite,
> > > >> > > automated
> > > >> > > > > build mixed with maven, non-native JVM language focused,
> > etc.).
> > > >> > > > >
> > > >> > > > > Of course, by introducing GLVs to TinkerPop though, we now
> > have
> > > >> the
> > > >> > > issue
> > > >> > > > > of conflict with TinkerPop only having a single "reference
> > > >> > > > implementation"
> > > >> > > > > for drivers because without a driver the GLV is pretty
> > useless.
> > > >> We've
> > > >> > > > > mitigated that in gremlin-python so far with a simple REST
> > > >> > > implementation
> > > >> > > > > of a driver, but as we advance the capabilities of GLVs that
> > > >> approach
> > > >> > > > won't
> > > >> > > > > be good for the long term.
> > > >> > > > >
> > > >> > > > > So - it seems like we (TinkerPop) should have a driver to go
> > > with
> > > >> > each
> > > >> > > > > GLV? To me, the upsides are big for users:
> > > >> > > > >
> > > >> > > > > 1. Users don't have to hunt for drivers elsewhere - they
> just
> > > grab
> > > >> > > their
> > > >> > > > > GLV and go. I suppose there is still opportunity for
> > third-party
> > > >> > > drivers
> > > >> > > > to
> > > >> > > > > be built, but at least users have an immediate easy option
> > > within
> > > >> the
> > > >> > > > > Apache TinkerPop.
> > > >> > > > > 2. Our docs for GLV usage are completely in Apache TinkerPop
> > and
> > > >> are
> > > >> > > > > "production ready" (i.e. not REST based then go switch to
> > > >> something
> > > >> > > > > third-party)
> > > >> > > > >
> > > >> > > > > There are some downsides which basically boil down to "Mo'
> > code
> > > >> Mo'
> > > >> > > > > problems" but basically:
> > > >> > > > >
> > > >> > > > > 1. I believe that it will be expected that we maintain some
> > > >> parity in
> > > >> > > > > terms of features and performance with the drivers. We can't
> > > have
> > > >> the
> > > >> > > > java
> > > >> > > > > driver be 10x over the python driver
> > > >> > > > > 2. The GLV code base becomes more complex. A high performing
> > > >> driver
> > > >> > > will
> > > >> > > > > not be trivial.
> > > >> > > > > 3. A whole class of downsides related to GLVs in general
> where
> > > we
> > > >> > don't
> > > >> > > > > have the necessary language expertise to maintain it -
> though
> > -
> > > >> > again,
> > > >> > > > > mitigated by the high bar we set with our reference
> > > implementation
> > > >> > for
> > > >> > > > > gremlin-python. Also, rather than a negative, one could also
> > see
> > > >> this
> > > >> > > as
> > > >> > > > an
> > > >> > > > > opportunity to grow the TinkerPop community.
> > > >> > > > >
> > > >> > > > > Ultimately, I think all of this is outweighed by the simple
> > fact
> > > >> that
> > > >> > > we
> > > >> > > > > need a driver in Apache TinkerPop to fully test GLVs.
> Without
> > > that
> > > >> > > we're
> > > >> > > > > kinda of stuck ourselves.
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
