[Discuss] Versioning for Gremlin

[divijvaidya13]

Hi all,

I would like to understand the thoughts of the community and future direction 
of development for versioning of Gremlin.

Ideally, if a server supports multiple major versions of TP (let’s say TP3 & 
TP4), the client (GLV) should have the capability to issue a query pertaining 
to different versions. It could probably be achieved by using different 
serialization format for different version but IMO, query language should not 
be coupled with serialziation formats. A better way to achieve it would be to 
introduce version field in the byte code message itself.

What do you think?

[jira] [Commented] (TINKERPOP-1595) Go through TraversalVertexProgram with a profile and optimize.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457022#comment-16457022
 ] 

ASF GitHub Bot commented on TINKERPOP-1595:
---

Github user dkuppitz commented on the issue:

https://github.com/apache/tinkerpop/pull/860
  
VOTE: +1


> Go through TraversalVertexProgram with a profile and optimize.
> --
>
> Key: TINKERPOP-1595
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1595
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: process
>Affects Versions: 3.2.3
>Reporter: Marko A. Rodriguez
>Assignee: stephen mallette
>Priority: Minor
>
> Work with [~twilmes] on profiling {{TraversalVertexProgram}}. When I was 
> doing the {{DedupGlobalStep}} work with [~dkuppitz], I noticed various 
> nick-nack optimizations. My profiling skills are poor, so if [~twilmes] could 
> identify long running sections of the code, that would be huge.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] tinkerpop issue #860: TINKERPOP-1595 Optimize TraversalVertexProgram

[dkuppitz]

Github user dkuppitz commented on the issue:

https://github.com/apache/tinkerpop/pull/860
  
VOTE: +1


---

[jira] [Commented] (TINKERPOP-1956) Deprecate Order incr/decr for asc/desc

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457015#comment-16457015
 ] 

ASF GitHub Bot commented on TINKERPOP-1956:
---

Github user dkuppitz commented on the issue:

https://github.com/apache/tinkerpop/pull/861
  
VOTE: +1


> Deprecate Order incr/decr for asc/desc
> --
>
> Key: TINKERPOP-1956
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1956
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: process
>Affects Versions: 3.3.2
>Reporter: stephen mallette
>Assignee: stephen mallette
>Priority: Minor
>  Labels: deprecation
>
> {{Order}} uses {{incr}} and {{decr}} which aren't terribly standard sounding 
> names for query languages where folks are more used to seeing {{asc}} and 
> {{desc}}. Deprecate the old terms in favor of these new ones. Since {{incr}} 
> and {{decr}} are in wide usage now, we might not be in a rush to remove them 
> completely any time soon - but that's a separate discussion.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] tinkerpop issue #861: TINKERPOP-1956 Deprecated Order.incr and Order.decr

[dkuppitz]

Github user dkuppitz commented on the issue:

https://github.com/apache/tinkerpop/pull/861
  
VOTE: +1


---

[jira] [Closed] (TINKERPOP-1497) Add BalancedBarrierStrategy to protect from OME

[stephen mallette (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stephen mallette closed TINKERPOP-1497.
---
Resolution: Won't Do

I randomly brought this one up with kuppitz while perusing old issues and he 
mentioned that he wasn't able to recall a situation where we'd hit OME due to 
barriers. I can't recall such a complaint either. If we haven't seen that yet, 
it's probably safe to not worry about that issue until it arises and if it 
does, we might question why that traversal wasn't using OLAP.

> Add BalancedBarrierStrategy to protect from OME
> ---
>
> Key: TINKERPOP-1497
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1497
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: process
>Affects Versions: 3.2.2
>Reporter: Marko A. Rodriguez
>Priority: Major
>
> I'm not quite sure of the need for this feature, but will add it regardless 
> so we can think on it more.
> There are various strategies that introduce "lazy barriers" into the 
> traversal.
> * {{LazyBarrierStrategy}}
> * {{RepeatUnrollStrategy}}
> * {{PathRetractionStrategy}}
> Right now, the size of these barriers are hardcoded as "private static ints" 
> inside of each respective strategy. The problem is that with all these 
> barriers, we might get OME if all barriers are "full" (and we have many 
> barriers).
> {{BalancedBarrierStrategy}} (a {{FinalizationStrategy}}) would have a "max 
> total barrier size" and then would divide that barrier size amongst all the 
> barriers (in global children) in the traversal. Perhaps the division can be 
> biased by the use of filters, the depth in the traversal of the barrier, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (TINKERPOP-1956) Deprecate Order incr/decr for asc/desc

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456958#comment-16456958
 ] 

ASF GitHub Bot commented on TINKERPOP-1956:
---

GitHub user spmallette opened a pull request:

https://github.com/apache/tinkerpop/pull/861

TINKERPOP-1956 Deprecated Order.incr and Order.decr

https://issues.apache.org/jira/browse/TINKERPOP-1595

Replaced by `Order.asc` and `Order.desc` respectively. Left a number of 
uses of `decr`/`incr` so as to ensure ongoing compatibility during deprecation.

Feel free to VOTE, but this won't merge until after we open 3.3.4 for 
development.

All tests pass with `docker/build.sh -t -n -i`

VOTE +1

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/tinkerpop TINKERPOP-1956

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/tinkerpop/pull/861.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #861


commit a937287d983c179668af92696f6e43e2151568bf
Author: Stephen Mallette 
Date:   2018-04-27T10:22:13Z

TINKERPOP-1956 Deprecated Order.incr and Order.decr

Replaced by Order.asc and Order.desc respectively. Left a number of uses of 
decr/incr so as to ensure ongoing compatibility during deprecation.




> Deprecate Order incr/decr for asc/desc
> --
>
> Key: TINKERPOP-1956
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1956
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: process
>Affects Versions: 3.3.2
>Reporter: stephen mallette
>Assignee: stephen mallette
>Priority: Minor
>  Labels: deprecation
>
> {{Order}} uses {{incr}} and {{decr}} which aren't terribly standard sounding 
> names for query languages where folks are more used to seeing {{asc}} and 
> {{desc}}. Deprecate the old terms in favor of these new ones. Since {{incr}} 
> and {{decr}} are in wide usage now, we might not be in a rush to remove them 
> completely any time soon - but that's a separate discussion.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] tinkerpop pull request #861: TINKERPOP-1956 Deprecated Order.incr and Order....

[spmallette]

GitHub user spmallette opened a pull request:

https://github.com/apache/tinkerpop/pull/861

TINKERPOP-1956 Deprecated Order.incr and Order.decr

https://issues.apache.org/jira/browse/TINKERPOP-1595

Replaced by `Order.asc` and `Order.desc` respectively. Left a number of 
uses of `decr`/`incr` so as to ensure ongoing compatibility during deprecation.

Feel free to VOTE, but this won't merge until after we open 3.3.4 for 
development.

All tests pass with `docker/build.sh -t -n -i`

VOTE +1

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/tinkerpop TINKERPOP-1956

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/tinkerpop/pull/861.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #861


commit a937287d983c179668af92696f6e43e2151568bf
Author: Stephen Mallette 
Date:   2018-04-27T10:22:13Z

TINKERPOP-1956 Deprecated Order.incr and Order.decr

Replaced by Order.asc and Order.desc respectively. Left a number of uses of 
decr/incr so as to ensure ongoing compatibility during deprecation.




---

[jira] [Updated] (TINKERPOP-1958) TinkerGraphCountStrategy can return wrong counts

[Daniel Kuppitz (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Kuppitz updated TINKERPOP-1958:
--
Affects Version/s: 3.2.8

> TinkerGraphCountStrategy can return wrong counts
> 
>
> Key: TINKERPOP-1958
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1958
> Project: TinkerPop
>  Issue Type: Bug
>  Components: process
>Affects Versions: 3.2.8, 3.3.2
>Reporter: Daniel Kuppitz
>Assignee: Daniel Kuppitz
>Priority: Major
>
> {noformat}
> gremlin> g = TinkerFactory.createModern().traversal()
> ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
> gremlin> g.V().map(out())
> ==>v[3]
> ==>v[5]
> ==>v[3]
> gremlin> g.V().map(out()).count()
> ==>6
> gremlin> g.V().select("a")
> gremlin> g.V().select("a").count()
> ==>6
> {noformat}
> Not tested in 3.2.x, but it's probably the same issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (TINKERPOP-1595) Go through TraversalVertexProgram with a profile and optimize.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456938#comment-16456938
 ] 

ASF GitHub Bot commented on TINKERPOP-1595:
---

GitHub user spmallette opened a pull request:

https://github.com/apache/tinkerpop/pull/860

TINKERPOP-1595 Optimize TraversalVertexProgram

https://issues.apache.org/jira/browse/TINKERPOP-1595

I did as the issue requested and optimized `TraversalVertexProgram`. 
Definitely some hotspots in there that have been removed. I'd say that the main 
fixes involved the direct lookup of `clone()` in `MemoryComputeKey` rather than 
cycling all methods and the use of Base64 encoding instead of CSV style 
string/regex derser for traversals in OLAP.

Feel free to VOTE but this change will not be merged until we open 
development for 3.2.10.

All tests pass with `docker/build.sh -t -n -i`

VOTE +1

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/tinkerpop TINKERPOP-1595

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/tinkerpop/pull/860.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #860


commit 172dfa5ea6e5dca52ee9e593da5eb054940c7fe7
Author: Stephen Mallette 
Date:   2018-04-24T11:59:58Z

TINKERPOP-1595 Removed usage of deprecated graph computer method

commit 344d1b24445fb332b88ecce9af8020f91046c3e0
Author: Stephen Mallette 
Date:   2018-04-24T12:00:48Z

TINKERPOP-1595 Removed stream() usage

commit 91f078a79de8ef4445a00e568cce1bff09a7095d
Author: Stephen Mallette 
Date:   2018-04-24T12:02:31Z

TINKERPOP-1595 Removed stream() usage

commit 1d149c6d56421bacecdd53ab0c223f386bc999db
Author: Stephen Mallette 
Date:   2018-04-24T12:03:28Z

TINKERPOP-1595 Refactored reflective lookups for clone()

Rather than iterate all methods to find clone() just call getMethod() and 
call that if found. Nothing seems to be exercising this bit of code in our test 
suite. I forced it by adding clone() temporarily to RangeBiOperator and it 
worked without issue. Not sure how else to test this.

commit e2eebf4c5590fd20c8184bd0162c717614575034
Author: Stephen Mallette 
Date:   2018-04-24T12:06:56Z

TINKERPOP-1595 Changed vertex program serialization to base64 encoded

This approach shows to be faster than writing the bytes as an array of 
string values and deserializing with regex based parsing.

commit f2119bce29615792a235e139047089bbc8c39df2
Author: Stephen Mallette 
Date:   2018-04-24T12:10:12Z

TINKERPOP-1595 Updated changelog

commit aef6896bda2fea7708dc16951bb8652a1b488e93
Author: Stephen Mallette 
Date:   2018-04-27T14:31:15Z

TINKERPOP-1595 Cleaned up exception handling for giraph

Some really specific exception handling of deserialization errors in Giraph 
were causing tests to hang. Corrected those problems and commented heavily.

commit 074ab1e5dca147acb6e05c4c1b4fff9f7e69d8b8
Author: Stephen Mallette 
Date:   2018-04-27T14:33:58Z

TINKERPOP-1595 Fixed up changelog as this moved to 3.2.10




> Go through TraversalVertexProgram with a profile and optimize.
> --
>
> Key: TINKERPOP-1595
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1595
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: process
>Affects Versions: 3.2.3
>Reporter: Marko A. Rodriguez
>Assignee: stephen mallette
>Priority: Minor
>
> Work with [~twilmes] on profiling {{TraversalVertexProgram}}. When I was 
> doing the {{DedupGlobalStep}} work with [~dkuppitz], I noticed various 
> nick-nack optimizations. My profiling skills are poor, so if [~twilmes] could 
> identify long running sections of the code, that would be huge.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (TINKERPOP-1958) TinkerGraphCountStrategy can return wrong counts

[Daniel Kuppitz (JIRA)]

Daniel Kuppitz created TINKERPOP-1958:
-

 Summary: TinkerGraphCountStrategy can return wrong counts
 Key: TINKERPOP-1958
 URL: https://issues.apache.org/jira/browse/TINKERPOP-1958
 Project: TinkerPop
  Issue Type: Bug
  Components: process
Affects Versions: 3.3.2
Reporter: Daniel Kuppitz
Assignee: Daniel Kuppitz


{noformat}
gremlin> g = TinkerFactory.createModern().traversal()
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
gremlin> g.V().map(out())
==>v[3]
==>v[5]
==>v[3]
gremlin> g.V().map(out()).count()
==>6
gremlin> g.V().select("a")
gremlin> g.V().select("a").count()
==>6
{noformat}

Not tested in 3.2.x, but it's probably the same issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (TINKERPOP-1933) gremlin-python maximum recursion depth exceeded on large responses

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456924#comment-16456924
 ] 

ASF GitHub Bot commented on TINKERPOP-1933:
---

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/854
  
i dunno - that's sorta why i left it alone.  if you can't think of any way 
to test it well, a nice long explanatory comment with the JIRA referenced where 
you made the change might be enough to scare off another dev from coming along 
and trying to change that.


> gremlin-python maximum recursion depth exceeded on large responses
> --
>
> Key: TINKERPOP-1933
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1933
> Project: TinkerPop
>  Issue Type: Bug
>  Components: python
>Affects Versions: 3.3.1
>Reporter: Branden Moore
>Assignee: David M. Brown
>Priority: Major
>
> gremlin-python fails to deserialize large lists returned from the server, 
> giving the error:
>  
> {{>>> x = g.V().limit(65000).toList()}}
> {{Traceback (most recent call last):}}
> {{  File "/usr/lib/python2.7/logging/__init__.py", line 861, in emit}}
> {{Traceback (most recent call last):}}
> {{  File "", line 1, in }}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/process/traversal.py",
>  line 52, in toList}}
> {{    return list(iter(self))}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/process/traversal.py",
>  line 70, in next}}
> {{    return self.__next__()}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/process/traversal.py",
>  line 43, in __next__}}
> {{    self.traversal_strategies.apply_strategies(self)}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/process/traversal.py",
>  line 352, in apply_strategies}}
> {{    traversal_strategy.apply(traversal)}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/driver/remote_connection.py",
>  line 143, in apply}}
> {{    remote_traversal = self.remote_connection.submit(traversal.bytecode)}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/driver/driver_remote_connection.py",
>  line 54, in submit}}
> {{    results = result_set.all().result()}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/concurrent/futures/_base.py",
>  line 462, in result}}
> {{    return self.__get_result()}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/concurrent/futures/_base.py",
>  line 414, in __get_result}}
> {{    raise exception_type, self._exception, self._traceback}}
> {{RuntimeError: maximum recursion depth exceeded while calling a Python 
> object}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] tinkerpop issue #854: TINKERPOP-1933 gremlin-python maximum recursion depth ...

[spmallette]

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/854
  
i dunno - that's sorta why i left it alone.  if you can't think of any way 
to test it well, a nice long explanatory comment with the JIRA referenced where 
you made the change might be enough to scare off another dev from coming along 
and trying to change that.


---

Re: [DISCUSS] Depth First Repeat step

[Stephen Mallette]

It seems like we have general agreement on the easy things, that is:

1. this is a change for 3.4.0/master and
2. we're all for a DFS option

but we still have the hard part of having to come to consensus on how it's
used/implemented. The quick summary of this thread in that regard goes
something like this: We currently have this PR that introduces DFS, but
does so as a configuration per repeat() step. From what I gather a good
many of us seem to find that approach undesirable for one or more of the
following reasons:

1. The use of order() seems misplaced purely from a usability/API
perspective
2. The approach seems to be at odds with how everything else works given
barrier() and strategies
3. The approach seems to be at odds with our current mixed mode of DFS/BFS

I think that we can see those issues resolve themselves with something
Kuppitz mentioned to me: repeat() should be DFS by default where barrier()
will change that behavior as required. That change would yield the
following approaches:

Full BFS: manually add `barrier()`'s
Mixed mode: Default, let strategies do their thing OR remove strategies and
manually add your own barrier()
Full DFS: execute `.withoutStrategies(Lazy...)`

Futherrmore, we probably should have some form of verification strategy
that ensures all BFS or all DFS so that users can't get tricked along the
way. It's not enough to just remove LazyBarrierStrategy to get DFS if
another strategy comes along and throws in a barrier().

So if all that sounds good from a usability perspective, then we get all
three modes that we want using existing traversal semantics which removes
the three concerns I've summarized from this thread. We also get Keith's
desire to have control over which part of a traversal is BFS/DFS if users
want that capability because they can do a manual Mixed Mode and add their
own barrier() to control the flow. For Pieter (or any graph provider)
nothing really changes and there is opportunity to control flow with
strategies as usual.

I haven't really focused much on what's involved in adapting the current
work in the PR to this approach as I more wanted to find the common ground
among all the people who commented on the thread. If we agree that this is
a nice way to go, then we can think more about "how" it could happen.

Keith, I saw you mention earlier that:

>  The barrier step that Daniel described doesn’t currently work since
there’s basically booleans in the RepeatStep on whether or not to stash the
starts to make the RepeatStep depth first.

I presume that would be some source of technical derailment to this
approach.




On Tue, Apr 24, 2018 at 3:05 PM, Keith Lohnes  wrote:

> Yeah, that's what I meant. The steps inside are replaced with some
> JanusGraph stuff.
>
> Cheers,
> Keith
>
>
> On Tue, Apr 24, 2018 at 1:52 PM pieter gmail 
> wrote:
>
> > Nah, that looks to me like the RepeatStep survived. Just the nested
> > VertexStep that got replaced with JanusgraphVertexStep.
> > Good for them, first prize is not replacing anything.
> >
> > Cheers
> > Pieter
> >
> > On 24/04/2018 19:50, Keith Lohnes wrote:
> > > It looks like it,
> > > `g.V().has("foo", "bar").repeat(out()).emit().explain()`
> > >
> > > yields
> > >
> > > `[JanusGraphStep([],[foo.eq(bar)]),
> > > RepeatStep([JanusGraphVertexStep(OUT,vertex),
> > > RepeatEndStep],until(false),emit(true))]`
> > >
> > >
> > >
> > > On Tue, Apr 24, 2018 at 12:12 PM pieter gmail  >
> > > wrote:
> > >
> > >> Hi,
> > >>
> > >> Sqlg completely replaces TinkerPop's RepeatStep. The idea being that
> > >> with g.V().repeat(out()).times(x) only x round trips to the db is
> needed
> > >> regardless of the size of the graph. Each time it will go to the db
> with
> > >> the full set of the previous step's incoming starts.
> > >>
> > >> But yeah TinkerPop's implementation is always the starting point so
> I'll
> > >> definitely have a look at how you have implemented DFS.
> > >>
> > >> BTW, does Janus graph use TinkerPop's default RepeatStep as is with no
> > >> optimization strategies?
> > >>
> > >> Cheers
> > >> Pieter
> > >>
> > >> On 24/04/2018 16:33, Keith Lohnes wrote:
> > >>> Pieter,
> > >>>
> > >>> If you take a look at https://github.com/apache/tinkerpop/pull/838
> DFS
> > >> is
> > >>> implemented as a modification to BFS. It's taking the starts that
> come
> > in
> > >>> from a BFS and stashing them to be processed later. I haven't seen a
> > big
> > >>> performance difference on JanusGraph; At least for the queries that
> > I've
> > >>> been running with it. I'm not terribly familiar with Sqlg, but I
> wonder
> > >> if
> > >>> in the context of how DFS is implemented there, it may be less of a
> > >>> concern.
> > >>>
> > >>> Cheers,
> > >>> Keith
> > >>>
> > >>> On Thu, Apr 19, 2018 at 12:46 PM pieter gmail <
> pieter.mar...@gmail.com
> > >
> > >>> wrote:
> > >>>
> >  Hi,
> > 
> >  Not really sure either what 'global' means 

[jira] [Commented] (TINKERPOP-1933) gremlin-python maximum recursion depth exceeded on large responses

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456794#comment-16456794
 ] 

ASF GitHub Bot commented on TINKERPOP-1933:
---

Github user davebshow commented on the issue:

https://github.com/apache/tinkerpop/pull/854
  
I'm happy to write a test. Is there an established way to to test large 
streaming responses? Or any tips on how to write one?


> gremlin-python maximum recursion depth exceeded on large responses
> --
>
> Key: TINKERPOP-1933
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1933
> Project: TinkerPop
>  Issue Type: Bug
>  Components: python
>Affects Versions: 3.3.1
>Reporter: Branden Moore
>Assignee: David M. Brown
>Priority: Major
>
> gremlin-python fails to deserialize large lists returned from the server, 
> giving the error:
>  
> {{>>> x = g.V().limit(65000).toList()}}
> {{Traceback (most recent call last):}}
> {{  File "/usr/lib/python2.7/logging/__init__.py", line 861, in emit}}
> {{Traceback (most recent call last):}}
> {{  File "", line 1, in }}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/process/traversal.py",
>  line 52, in toList}}
> {{    return list(iter(self))}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/process/traversal.py",
>  line 70, in next}}
> {{    return self.__next__()}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/process/traversal.py",
>  line 43, in __next__}}
> {{    self.traversal_strategies.apply_strategies(self)}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/process/traversal.py",
>  line 352, in apply_strategies}}
> {{    traversal_strategy.apply(traversal)}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/driver/remote_connection.py",
>  line 143, in apply}}
> {{    remote_traversal = self.remote_connection.submit(traversal.bytecode)}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/driver/driver_remote_connection.py",
>  line 54, in submit}}
> {{    results = result_set.all().result()}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/concurrent/futures/_base.py",
>  line 462, in result}}
> {{    return self.__get_result()}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/concurrent/futures/_base.py",
>  line 414, in __get_result}}
> {{    raise exception_type, self._exception, self._traceback}}
> {{RuntimeError: maximum recursion depth exceeded while calling a Python 
> object}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (TINKERPOP-1942) Binary serialization format

[David M. Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456772#comment-16456772
 ] 

David M. Brown commented on TINKERPOP-1942:
---

I haven't reviewed it closely, but I am pro-binary serialization format. I will 
happily implement a C/Python parser/serializer.

> Binary serialization format
> ---
>
> Key: TINKERPOP-1942
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1942
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Reporter: Jorge Bay
>Priority: Major
>
> We should provide a binary serialization format designed to reduce 
> serialization overhead and minimizing the size of the payload that is 
> transmitted over the wire.
> It could be implemented in a very similar way as Kryo support but with 
> interoperability in mind and ultimately we could fade Gryo out, as now with 
> the GLVs it doesn't have a role to play.
> The main benefit would be the performance improvement, making serialization 
> and deserialization processing time negligible on both the server and the 
> client.
> Background: 
> https://lists.apache.org/thread.html/13e70235591853801bab16ed457ee4f56f3dfe2d1c5817c34a036408@%3Cdev.tinkerpop.apache.org%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (TINKERPOP-1942) Binary serialization format

[stephen mallette (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456746#comment-16456746
 ] 

stephen mallette commented on TINKERPOP-1942:
-

If it replaces Gryo then this issue and the related issues bound to it should 
be considered: TINKERPOP-1346

> Binary serialization format
> ---
>
> Key: TINKERPOP-1942
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1942
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Reporter: Jorge Bay
>Priority: Major
>
> We should provide a binary serialization format designed to reduce 
> serialization overhead and minimizing the size of the payload that is 
> transmitted over the wire.
> It could be implemented in a very similar way as Kryo support but with 
> interoperability in mind and ultimately we could fade Gryo out, as now with 
> the GLVs it doesn't have a role to play.
> The main benefit would be the performance improvement, making serialization 
> and deserialization processing time negligible on both the server and the 
> client.
> Background: 
> https://lists.apache.org/thread.html/13e70235591853801bab16ed457ee4f56f3dfe2d1c5817c34a036408@%3Cdev.tinkerpop.apache.org%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (TINKERPOP-1933) gremlin-python maximum recursion depth exceeded on large responses

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456561#comment-16456561
 ] 

ASF GitHub Bot commented on TINKERPOP-1933:
---

Github user dkuppitz commented on the issue:

https://github.com/apache/tinkerpop/pull/854
  
VOTE: +1

But I do think a simple test would be nice. The code screams "make me 
recursive" and a future contributor, not aware of this issue, might try to 
change it back and not realize that it introduces a bug.


> gremlin-python maximum recursion depth exceeded on large responses
> --
>
> Key: TINKERPOP-1933
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1933
> Project: TinkerPop
>  Issue Type: Bug
>  Components: python
>Affects Versions: 3.3.1
>Reporter: Branden Moore
>Assignee: David M. Brown
>Priority: Major
>
> gremlin-python fails to deserialize large lists returned from the server, 
> giving the error:
>  
> {{>>> x = g.V().limit(65000).toList()}}
> {{Traceback (most recent call last):}}
> {{  File "/usr/lib/python2.7/logging/__init__.py", line 861, in emit}}
> {{Traceback (most recent call last):}}
> {{  File "", line 1, in }}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/process/traversal.py",
>  line 52, in toList}}
> {{    return list(iter(self))}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/process/traversal.py",
>  line 70, in next}}
> {{    return self.__next__()}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/process/traversal.py",
>  line 43, in __next__}}
> {{    self.traversal_strategies.apply_strategies(self)}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/process/traversal.py",
>  line 352, in apply_strategies}}
> {{    traversal_strategy.apply(traversal)}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/driver/remote_connection.py",
>  line 143, in apply}}
> {{    remote_traversal = self.remote_connection.submit(traversal.bytecode)}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/gremlin_python/driver/driver_remote_connection.py",
>  line 54, in submit}}
> {{    results = result_set.all().result()}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/concurrent/futures/_base.py",
>  line 462, in result}}
> {{    return self.__get_result()}}
> {{  File 
> "/ascldap/users/bjmoor/.local/lib/python2.7/site-packages/concurrent/futures/_base.py",
>  line 414, in __get_result}}
> {{    raise exception_type, self._exception, self._traceback}}
> {{RuntimeError: maximum recursion depth exceeded while calling a Python 
> object}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] tinkerpop issue #854: TINKERPOP-1933 gremlin-python maximum recursion depth ...

[dkuppitz]

Github user dkuppitz commented on the issue:

https://github.com/apache/tinkerpop/pull/854
  
VOTE: +1

But I do think a simple test would be nice. The code screams "make me 
recursive" and a future contributor, not aware of this issue, might try to 
change it back and not realize that it introduces a bug.


---

[jira] [Updated] (TINKERPOP-1595) Go through TraversalVertexProgram with a profile and optimize.

[stephen mallette (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stephen mallette updated TINKERPOP-1595:

Priority: Minor  (was: Major)

> Go through TraversalVertexProgram with a profile and optimize.
> --
>
> Key: TINKERPOP-1595
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1595
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: process
>Affects Versions: 3.2.3
>Reporter: Marko A. Rodriguez
>Assignee: stephen mallette
>Priority: Minor
>
> Work with [~twilmes] on profiling {{TraversalVertexProgram}}. When I was 
> doing the {{DedupGlobalStep}} work with [~dkuppitz], I noticed various 
> nick-nack optimizations. My profiling skills are poor, so if [~twilmes] could 
> identify long running sections of the code, that would be huge.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (TINKERPOP-1595) Go through TraversalVertexProgram with a profile and optimize.

[stephen mallette (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stephen mallette updated TINKERPOP-1595:

Fix Version/s: (was: 3.2.9)
   (was: 3.3.3)
   (was: 3.4.0)

> Go through TraversalVertexProgram with a profile and optimize.
> --
>
> Key: TINKERPOP-1595
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1595
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: process
>Affects Versions: 3.2.3
>Reporter: Marko A. Rodriguez
>Assignee: stephen mallette
>Priority: Major
>
> Work with [~twilmes] on profiling {{TraversalVertexProgram}}. When I was 
> doing the {{DedupGlobalStep}} work with [~dkuppitz], I noticed various 
> nick-nack optimizations. My profiling skills are poor, so if [~twilmes] could 
> identify long running sections of the code, that would be huge.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [DISCUSS] Consider patch release 3.2.9/3.3.3

[Stephen Mallette]

Thanks for taking 3.2.9 Robert. 3.3.3 is still up for grabs if anyone else
wants to take that one.

We still need to merge these three PRs:

https://github.com/apache/tinkerpop/pull/854 - needs a VOTE still before
merge
https://github.com/apache/tinkerpop/pull/858 and
https://github.com/apache/tinkerpop/pull/852 - Jorge is off this week, but
I assume he will merge early next week

I don't think we need to worry about any others.

Jorge and maybe Jean-Baptiste, I've noticed more than a few people having
issues with gremlin-javascript - the two things of concern:

1. documentation
2. how to send scripts

I seem to sense that a lot of the confusion that led to item 1 is addressed
in TINKERPOP-1944 as well as the issue that CosmosDB doesn't support
bytecode which then leads to item 2 because you need to send scripts there.
On top of all that we also didn't get things clear as to what happened in
the npm gremlin repository when we basically deprecated out the
"unofficial" gremlin-javascript that JB maintained, flubbed the
documentation on which npm repo to use, etc..

Can either of you speak to how 3.2.9/3.3.3 (and whatever else we might have
done) will better address those issues now that we have a second chance at
getting this all "right"? I want the groovy issue fixed, but I'd like to
polish up gremlin-javascript more as well and if we need extra time to do
that, I think we should take that time and not rush to release.

Thanks!

On Thu, Apr 26, 2018 at 8:18 AM, Robert Dale  wrote:

> I'll take 3.2.9
>
> Robert Dale
>
> On Thu, Apr 26, 2018 at 7:35 AM, Stephen Mallette 
> wrote:
>
> > Seems like we're all on board with getting a quick turnaround release out
> > the door. At this point the groovy fix is now in place and Jorge has both
> > TINKERPOP-1943/1944 prepared. Unless there are other major issues anyone
> > wants to bring up, we should consider a code freeze starting this friday
> > and plan for a release the week of May 7.
> >
> > Any volunteers to help with release management duties?
> >
> >
> >
> > On Mon, Apr 23, 2018 at 9:19 AM, Jorge Bay Gondra <
> > jorgebaygon...@gmail.com>
> > wrote:
> >
> > > sgtm, I'm also +1 on releasing patch releases asap to fix it.
> > >
> > > Regarding gremlin-javascript, we could include TINKERPOP-1944 (I just
> > > submitted a pull request) but I'm not sure we can include
> TINKERPOP-1943,
> > > the implementation is trivial but it will require some changes to make
> > the
> > > test suite pass with GraphSON3 (Map instances vs Objects).
> > >
> > >
> > > 2018-04-23 15:00 GMT+02:00 Robert Dale :
> > >
> > > > +1
> > > >
> > > > Robert Dale
> > > >
> > > > On Mon, Apr 23, 2018 at 8:44 AM, Stephen Mallette <
> > spmalle...@gmail.com>
> > > > wrote:
> > > >
> > > > > Based on:
> > > > >
> > > > > https://lists.apache.org/thread.html/
> de4ec9054fd7a88cbd42163a0b1050
> > > > > 19aae52a7e930b09a9bd4c971d@%3Cdev.tinkerpop.apache.org%3E
> > > > >
> > > > > We have a pretty serious bug that I can't seem to find a workaround
> > > for.
> > > > > The only workaround I can think of would be to replace this:
> > > > >
> > > > > Lambda.function("it.get()")
> > > > >
> > > > > with
> > > > >
> > > > > new OneArgLambda("it.get()", "gremlin-groovy")
> > > > >
> > > > > Not great as existing code using Lambda will break and none of the
> > docs
> > > > > make any reference to direct usage of the various "Lambda"
> > > constructors.
> > > > > So, short of someone suggesting one very quickly, I think we need
> to
> > > > > release a patch.
> > > > >
> > > > > Any concerns with doing that? Also, assuming we proceed with the
> > patch
> > > as
> > > > > quickly as possible, can anyone think of any critical items that
> need
> > > to
> > > > be
> > > > > done for 3.2.9/3.3.3?
> > > > >
> > > > > As a side note, this is somewhat good for gremlin-javascript as we
> > > sorta
> > > > > flubbed the documentation a bit and missed some
> odds/ends.perhaps
> > > > this
> > > > > allows us to get that patched up as well sooner than expected?
> > > > >
> > > >
> > >
> >
>

[jira] [Closed] (TINKERPOP-1195) Client and ResultSet API changes

[stephen mallette (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stephen mallette closed TINKERPOP-1195.
---
Resolution: Won't Do

Don't think I will change this now - the preferred method for sending requests 
now is over bytecode so this API will become less and less exposed as time goes 
on. I don't think there's much point to changing it now.

> Client and ResultSet API changes
> 
>
> Key: TINKERPOP-1195
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1195
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: driver
>Affects Versions: 3.1.1-incubating
>Reporter: stephen mallette
>Assignee: stephen mallette
>Priority: Major
>  Labels: breaking
>
> Given discussion on a couple different threads:
> https://lists.apache.org/thread.html/Z424dvlo67k5np4
> https://lists.apache.org/thread.html/Z70ku3nckrkkgiy
> the idea to introduce a breaking change to the {{Client}} and {{Result}} API 
> seems to be established.  The basic changes involve:
> 1. Remove the return of {{CompletableFuture}} from a {{ResultSet}} thus 
> allowing it to block for the {{some()}} and {{all()}} methods.
> 2. Change {{Client.submit()}} to return {{List}} rather than return a 
> {{ResultSet}} - that method will basically block until all results are 
> returned and then unwrap the {{List}} and return that.
> Unfortunately, there doesn't appear to be a way to make it so that these 
> changes are non-breaking.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (TINKERPOP-1957) Static Groovy script compilation requires long literals in predicates

[stephen mallette (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stephen mallette closed TINKERPOP-1957.
---
Resolution: Not A Problem

I can't isolate exactly why there is a discrepancy in behavior between the two 
versions, but I sense that the behavior you are seeing is the correct one in 
3.3.2 (which i still see in 3.3.3-SNAPSHOT). Your child traversal of 
{{outE().count()}} returns a {{Long}} which means that the {{is()}} is going to 
expect either {{P}} or {{Object}} given the available signatures. Your 
traversal gives it a {{P}} and Groovy makes a best guess between the 
two options and ends up with a generics problem. Note that you can either 
specify the {{Long}} to get by this issue or cast to {{P}}:

{code}
g.V().where(outE().count().is((P) lt(50)))
{code}

which I guess forces Groovy down the right path. Again, I'm not clear what the 
discrepancy is between the two versions, but my guess is either/both of the 
following:

1. There is some method signature difference between 3.2.x and 3.3.x (that i 
can't seem to locate) that subtly allows Groovy to be happy to type check in 
your favor for 3.3.2. 
2. I've known Groovy's approach to method invocation to be somewhat 
non-deterministic (seemingly) in the ScriptEngine. 

Anyway, my general thinking is that when faced with these kinds of situations 
with type checking I'd tend to assume that a more strict interpretation from 
Groovy is the "correct" application of that function. Also, issues like this 
would likely not be issues with TinkerPop as we rely completely on Groovy for 
this functionality.

> Static Groovy script compilation requires long literals in predicates
> -
>
> Key: TINKERPOP-1957
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1957
> Project: TinkerPop
>  Issue Type: Bug
>  Components: groovy
>Affects Versions: 3.3.2
> Environment: GremlinGroovyScriptEngine with 
> GroovyCompilerGremlinPlugin and COMPILE_STATIC
>Reporter: Konstantin Müller
>Priority: Major
>
> When using the GremlinGroovyScriptEngine together with 
> GroovyCompilerGremlinPlugin and compilation option COMPILE_STATIC it is 
> required to use long literals instead of integer literals in predicates. For 
> example the following query fails during compilation:
> {noformat}
> g.V().where(outE().count().is(lt(50))){noformat}
> This gives the following error message:
> {noformat}
> Script1.groovy: 1: [Static type checking] - Cannot call 
> org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal 
>  java.lang.Long>#is(org.apache.tinkerpop.gremlin.process.traversal.P 
> ) with arguments 
> [org.apache.tinkerpop.gremlin.process.traversal.P ]
>  @ line 1, column 13.
>    g.V().where(outE().count().is(lt(50)))
>    ^
> Script1.groovy: 1: [Static type checking] - Cannot find matching method 
> org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal#where(java.lang.Object).
>  Please check if the declared type is correct and if the method exists.
>  @ line 1, column 1.
>    g.V().where(outE().count().is(lt(50))){noformat}
> Changing the query to use long literals makes the query work:
> {noformat}
> g.V().where(outE().count().is(lt(50L))){noformat}
> The original query works with version 3.2.8 or if COMPILE_STATIC is NOT used.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (TINKERPOP-1957) Static Groovy script compilation requires long literals in predicates

[JIRA]

Konstantin Müller created TINKERPOP-1957:


 Summary: Static Groovy script compilation requires long literals 
in predicates
 Key: TINKERPOP-1957
 URL: https://issues.apache.org/jira/browse/TINKERPOP-1957
 Project: TinkerPop
  Issue Type: Bug
  Components: groovy
Affects Versions: 3.3.2
 Environment: GremlinGroovyScriptEngine with 
GroovyCompilerGremlinPlugin and COMPILE_STATIC
Reporter: Konstantin Müller


When using the GremlinGroovyScriptEngine together with 
GroovyCompilerGremlinPlugin and compilation option COMPILE_STATIC it is 
required to use long literals instead of integer literals in predicates. For 
example the following query fails during compilation:
{noformat}
g.V().where(outE().count().is(lt(50))){noformat}
This gives the following error message:
{noformat}
Script1.groovy: 1: [Static type checking] - Cannot call 
org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal 
#is(org.apache.tinkerpop.gremlin.process.traversal.P 
) with arguments 
[org.apache.tinkerpop.gremlin.process.traversal.P ]
 @ line 1, column 13.
   g.V().where(outE().count().is(lt(50)))
   ^

Script1.groovy: 1: [Static type checking] - Cannot find matching method 
org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal#where(java.lang.Object).
 Please check if the declared type is correct and if the method exists.
 @ line 1, column 1.
   g.V().where(outE().count().is(lt(50))){noformat}
Changing the query to use long literals makes the query work:
{noformat}
g.V().where(outE().count().is(lt(50L))){noformat}
The original query works with version 3.2.8 or if COMPILE_STATIC is NOT used.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

AW: .NET NamingConversions Template

[Florian Hockmann]

Oh yes, looks like I forgot to remove the template when I transformed the enums 
into classes for Gremlin.Net. I just removed the NamingConversions.cs file and 
deleted the lines in the groovy script that use the template but forgot the 
template itself [1].

I'll push a CTR commit that also removes the template.

[1] commit: 
https://github.com/apache/tinkerpop/pull/815/commits/dcf3da3917a9626d432ef26089bb05c6a15158ed


-Ursprüngliche Nachricht-
Von: Stephen Mallette  
Gesendet: Freitag, 27. April 2018 01:29
An: dev@tinkerpop.apache.org
Betreff: .NET NamingConversions Template

I just noticed that the NamingConversions.template

https://github.com/apache/tinkerpop/blob/2c373064021c9df5cf02906e73bdb9f37d6c9d6c/gremlin-dotnet/glv/NamingConversions.template

Doesn't appear to be referenced in the groovy script for some reason:

https://github.com/apache/tinkerpop/blob/2c373064021c9df5cf02906e73bdb9f37d6c9d6c/gremlin-dotnet/glv/generate.groovy

Was that an oversight of some sort?