[GitHub] tinkerpop pull request #728: Do not strong-freeze dependencies

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/tinkerpop/pull/728


---

[GitHub] tinkerpop issue #728: Do not strong-freeze dependencies

[jorgebay]

Github user jorgebay commented on the issue:

https://github.com/apache/tinkerpop/pull/728
  
VOTE +1


---

Re: [DISCUSS] GLV Test Suite

[Stephen Mallette]

Just a quick update on this item - it continues

Went down a bad path earlier in the week and ended up shelving a lot of
work - ended up just coded into a corner. Anyway, I've not really changed
much in the implementation, but I've still not reached a point where the
addition of a new .feature files comes without meeting some new type of
assertion that has to be dealt with. This ends up slowing down progress on
porting over the tests. Please feel free to peruse the latest changes on
the branch and let me know if there's any feedback.



On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette 
wrote:

> I did some pretty heavy refactoring to the python test logic (altered a
> bit by some revision of the gherkin feature file language) and the result
> is a much more simplified test logic file:
>
> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> gremlin-python/src/main/jython/radish/feature_steps.py
>
> About 120 lines of code (down from about 170). If you include the test
> logic "setup" file:
>
> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> gremlin-python/src/main/jython/radish/terrain.py
>
> we end up with about 250 lines of test logic total (take out the
> comment/license and we're probably well under 200 total - not too bad). I
> think I'm closing in on the end to infrastructure building here so the
> basic framework is getting close to final at this point I believe. I'll
> keep scanning the tests looking for other types of assertions that I've not
> yet covered, but it's getting pretty solid I think. Hopefully, there won't
> need to be too many more lines of code needed to express the test logic as
> I like how things are looking right now.
>
>
>
>
> On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> jorgebaygon...@gmail.com> wrote:
>
>> Great progress! I like how you avoided using ids, even if it adds some
>> complexity to the transformation required.
>>
>> The Python step definitions are still quite simple, I think it would be
>> mostly the same on the rest of the languages.
>>
>> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette 
>> wrote:
>>
>> > Just another update on progress with the test suite. The language of the
>> > .feature files is getting slightly more complex as I try to translate
>> more
>> > and more of our Java process suite tests into the language of the
>> gherkin
>> > files. You can see here where I've added the ability to include
>> parameters:
>> >
>> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > gremlin-test/features/filter/Has.feature#L33
>> >
>> > I also came up with a method of asserting edges (didn't want to rely on
>> ids
>> > as it makes the gherkin harder to read, plus i didn't want to assume
>> > TinkerGraph identifiers in case these tests were every used with some
>> other
>> > graph database that didn't use longs):
>> >
>> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > gremlin-test/features/map/Vertex.feature#L105-L111
>> >
>> > for all those additions (and others) the logic required by the GLV to
>> > process these tests has stayed surprisingly simple:
>> >
>> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > gremlin-python/src/main/jython/radish/feature_steps.py
>> >
>> > There's a fair bit of regex/string manipulation involved there, but it's
>> > processing strings from the feature file so that's the nature of it I
>> > suppose. I think I'm of the mind that I want to port all of the tests to
>> > feature files, so I wrote this unit test to help validate that none were
>> > missed:
>> >
>> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
>> > structure/FeatureCoverageTest.java
>> >
>> > I don't know that we have to get to 100% porting right away, but I think
>> > that once we have these gherkin files written they not only become the
>> > basis for our current GLV testing work, but we might be able to simply
>> use
>> > them for testing the Java stuff as well - that would rid us of having
>> the
>> > test code duplication. It also sets us up with a portable body of tests
>> > that can be re-used in TinkerPop 4.x.
>> >
>> > I'm open to suggestions if anyone has any.
>> >
>> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
>> > jorgebaygon...@gmail.com
>> > > wrote:
>> >
>> > > I was able to build a proof of concept for a Gherkin-based test
>> runner in
>> > > C#, that takes the proposed count and select features
>> > > > > > gremlin-test/features/map>
>> > > and runs them using C# step definitions.
>> > >
>> > > It uses the Gherkin parser > herkin-dotnet>
>> > > from
>> > > cucumber, there isn't a release of the parser with .NET Core support
>> so
>> > > I've
>> > > asked them to release one
>> > >  (there is no
>> > > limitation from their source files). If they are not 

[jira] [Commented] (TINKERPOP-1786) Recipe and missing manifest items for Spark on Yarn

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16202457#comment-16202457
 ] 

ASF GitHub Bot commented on TINKERPOP-1786:
---

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/721
  
There are two aspects to CHANGELOG - (1) developer notes on what was done 
and (2) JIRA tickets. You are really just responsible for (1) - it is typically 
a more descriptive explanation of everything that was added/modified. It is 
also usually more readable to a user, who may be perusing it (as opposed to the 
sometimes terse JIRA ticket titles).

The JIRA tickets get added on release by the release manager, so you don't 
really need to reference JIRA tickets at all in the CHANGELOG yourself. So to 
fix up your PR you just need to remove the `Improvements` header and the JIRA 
ticket below it.

As for your PR to master and 3.3.x, you don't have to adjust 
CHANGELOGjust leave your entries on 3.2.x - we assume that any fixes to 
3.2.x fold are inclusive to 3.3.x and make reference to that in the CHANGELOG 
so that we don't duplicate the entries. Let me know if what I have written here 
isn't clear for some reason.


> Recipe and missing manifest items for Spark on Yarn
> ---
>
> Key: TINKERPOP-1786
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1786
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: hadoop
>Affects Versions: 3.3.0, 3.1.8, 3.2.6
> Environment: gremlin-console
>Reporter: Marc de Lignie
>Priority: Minor
> Fix For: 3.2.7, 3.3.1
>
>
> Thorough documentation for running OLAP queries on Spark on Yarn has been 
> missing, keeping some users from getting the benefits of this nice feature of 
> the Tinkerpop stack and resulting in a significant number of questions on the 
> gremlin users list.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] tinkerpop issue #721: TINKERPOP-1786 Recipe and missing manifest items for S...

[spmallette]

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/721
  
There are two aspects to CHANGELOG - (1) developer notes on what was done 
and (2) JIRA tickets. You are really just responsible for (1) - it is typically 
a more descriptive explanation of everything that was added/modified. It is 
also usually more readable to a user, who may be perusing it (as opposed to the 
sometimes terse JIRA ticket titles).

The JIRA tickets get added on release by the release manager, so you don't 
really need to reference JIRA tickets at all in the CHANGELOG yourself. So to 
fix up your PR you just need to remove the `Improvements` header and the JIRA 
ticket below it.

As for your PR to master and 3.3.x, you don't have to adjust 
CHANGELOGjust leave your entries on 3.2.x - we assume that any fixes to 
3.2.x fold are inclusive to 3.3.x and make reference to that in the CHANGELOG 
so that we don't duplicate the entries. Let me know if what I have written here 
isn't clear for some reason.


---

[jira] [Commented] (TINKERPOP-1786) Recipe and missing manifest items for Spark on Yarn

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16202429#comment-16202429
 ] 

ASF GitHub Bot commented on TINKERPOP-1786:
---

Github user vtslab commented on the issue:

https://github.com/apache/tinkerpop/pull/721
  
Hi @spmallette,
No problem, but it is unclear to me whether JIRA will also add items to the 
list then, apart from adding the section headings. Where will my two change 
items appear, for which item(s) should I add the JIRA issue number? 
And for the TP3.3 line (the other PR), should the two new items be in the 
3.2 section, the 3.3 section, in both or in none?
Cheers,Marc


> Recipe and missing manifest items for Spark on Yarn
> ---
>
> Key: TINKERPOP-1786
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1786
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: hadoop
>Affects Versions: 3.3.0, 3.1.8, 3.2.6
> Environment: gremlin-console
>Reporter: Marc de Lignie
>Priority: Minor
> Fix For: 3.2.7, 3.3.1
>
>
> Thorough documentation for running OLAP queries on Spark on Yarn has been 
> missing, keeping some users from getting the benefits of this nice feature of 
> the Tinkerpop stack and resulting in a significant number of questions on the 
> gremlin users list.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] tinkerpop issue #721: TINKERPOP-1786 Recipe and missing manifest items for S...

[vtslab]

Github user vtslab commented on the issue:

https://github.com/apache/tinkerpop/pull/721
  
Hi @spmallette,
No problem, but it is unclear to me whether JIRA will also add items to the 
list then, apart from adding the section headings. Where will my two change 
items appear, for which item(s) should I add the JIRA issue number? 
And for the TP3.3 line (the other PR), should the two new items be in the 
3.2 section, the 3.3 section, in both or in none?
Cheers,Marc


---

[jira] [Commented] (TINKERPOP-1632) Create a set of default functions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16202317#comment-16202317
 ] 

ASF GitHub Bot commented on TINKERPOP-1632:
---

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/729
  
just pushed a fix for the double issueit was easy - pretty much as 
@FlorianHockmann deduced. re-running docker build now.


> Create a set of default functions
> -
>
> Key: TINKERPOP-1632
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1632
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: process
>Affects Versions: 3.3.0
>Reporter: Daniel Kuppitz
>
> We already have a a set of default BiFunctions / operators, that are not 
> treated as lambdas. We should also have a bunch of simple functions, that can 
> then be used in {{map()}} or {{sack()}} and that can be serialized as 
> bytecode. For example:
> {noformat}
> ...map(sqrt)
> ...map(log)
> ...sack(sigmoid) // compute sigmoid of the current sack and update the 
> sack
> ...sack(cos).by("x") // compute the cosine of the current element's "x" and 
> assign / overwrite the current sack value
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] tinkerpop issue #729: TINKERPOP-1632: Create a set of default functions

[spmallette]

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/729
  
just pushed a fix for the double issueit was easy - pretty much as 
@FlorianHockmann deduced. re-running docker build now.


---

[jira] [Closed] (TINKERPOP-1761) GremlinExecutor: Timeout future not cancelled on successful script evaluation

[stephen mallette (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stephen mallette closed TINKERPOP-1761.
---
   Resolution: Fixed
 Assignee: stephen mallette
Fix Version/s: 3.3.1
   3.2.7

> GremlinExecutor: Timeout future not cancelled on successful script evaluation
> -
>
> Key: TINKERPOP-1761
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1761
> Project: TinkerPop
>  Issue Type: Bug
>  Components: groovy
>Affects Versions: 3.2.6
>Reporter: Konstantin Müller
>Assignee: stephen mallette
> Fix For: 3.2.7, 3.3.1
>
>
> This bug seems to be introduced by TINKERPOP-1714. The timeout future in 
> GremlinExecutor.eval() is not cancelled when the evaluation future is 
> completed successfully before the timeout (lines 304ff). This was done 
> correctly before the changes introduced by TINKERPOP-1714. Code to reproduce:
> {code:java}
> try (GremlinExecutor executor = 
> GremlinExecutor.build().scriptEvaluationTimeout(15_000).create()) {
>   executor.eval("1+1").get();
> } catch (Exception e) {
>   // NOOP
> }
> {code}
> When the code leaves the try-block it calls GremlinExecutor.close() which 
> will clean up executorService and scheduledExecutorService, but because the 
> timeout future is not cancelled this will hang until the future is finished 
> (here 15s).
> Workaround: Provide an own ExecutorService when constructing a 
> GremlinExecutor instance and cancel all tasks when the script evaluation is 
> finished successfully.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (TINKERPOP-1761) GremlinExecutor: Timeout future not cancelled on successful script evaluation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16202256#comment-16202256
 ] 

ASF GitHub Bot commented on TINKERPOP-1761:
---

Github user asfgit closed the pull request at:

https://github.com/apache/tinkerpop/pull/709


> GremlinExecutor: Timeout future not cancelled on successful script evaluation
> -
>
> Key: TINKERPOP-1761
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1761
> Project: TinkerPop
>  Issue Type: Bug
>  Components: groovy
>Affects Versions: 3.2.6
>Reporter: Konstantin Müller
>
> This bug seems to be introduced by TINKERPOP-1714. The timeout future in 
> GremlinExecutor.eval() is not cancelled when the evaluation future is 
> completed successfully before the timeout (lines 304ff). This was done 
> correctly before the changes introduced by TINKERPOP-1714. Code to reproduce:
> {code:java}
> try (GremlinExecutor executor = 
> GremlinExecutor.build().scriptEvaluationTimeout(15_000).create()) {
>   executor.eval("1+1").get();
> } catch (Exception e) {
>   // NOOP
> }
> {code}
> When the code leaves the try-block it calls GremlinExecutor.close() which 
> will clean up executorService and scheduledExecutorService, but because the 
> timeout future is not cancelled this will hang until the future is finished 
> (here 15s).
> Workaround: Provide an own ExecutorService when constructing a 
> GremlinExecutor instance and cancel all tasks when the script evaluation is 
> finished successfully.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] tinkerpop pull request #709: TINKERPOP-1761: Cancel script evaluation timeou...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/tinkerpop/pull/709


---

[jira] [Commented] (TINKERPOP-1786) Recipe and missing manifest items for Spark on Yarn

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16202249#comment-16202249
 ] 

ASF GitHub Bot commented on TINKERPOP-1786:
---

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/721
  
Added a minor note about CHANGELOG, but pending that little fix:

VOTE +1


> Recipe and missing manifest items for Spark on Yarn
> ---
>
> Key: TINKERPOP-1786
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1786
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: hadoop
>Affects Versions: 3.3.0, 3.1.8, 3.2.6
> Environment: gremlin-console
>Reporter: Marc de Lignie
>Priority: Minor
> Fix For: 3.2.7, 3.3.1
>
>
> Thorough documentation for running OLAP queries on Spark on Yarn has been 
> missing, keeping some users from getting the benefits of this nice feature of 
> the Tinkerpop stack and resulting in a significant number of questions on the 
> gremlin users list.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] tinkerpop issue #721: TINKERPOP-1786 Recipe and missing manifest items for S...

[spmallette]

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/721
  
Added a minor note about CHANGELOG, but pending that little fix:

VOTE +1


---

[GitHub] tinkerpop pull request #721: TINKERPOP-1786 Recipe and missing manifest item...

[spmallette]

Github user spmallette commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/721#discussion_r144345569
  
--- Diff: CHANGELOG.asciidoc ---
@@ -43,7 +43,13 @@ 
image::https://raw.githubusercontent.com/apache/tinkerpop/master/docs/static/ima
 * Fixed a bug in `Neo4jGremlinPlugin` that prevented it from loading 
properly in the `GremlinPythonScriptEngine`.
 * Fixed a bug in `ComputerVerificationStrategy` where child traversals 
were being analyzed prior to compilation.
 * Fixed a bug that prevented Gremlin from ordering lists and streams made 
of mixed number types.
-* Fixed a bug where `keepLabels` were being corrupted because a defensive 
copy was not being made when they were being set by `PathRetractionStrategy`. 
+* Fixed a bug where `keepLabels` were being corrupted because a defensive 
copy was not being made when they were being set by `PathRetractionStrategy`.
+* Added a recipe for OLAP traversals with Spark on Yarn
+
+Improvements
--- End diff --

No need to add the "Improvements" section. That gets added on release and 
we generate that output from JIRA.


---

[jira] [Commented] (TINKERPOP-1786) Recipe and missing manifest items for Spark on Yarn

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16202246#comment-16202246
 ] 

ASF GitHub Bot commented on TINKERPOP-1786:
---

Github user spmallette commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/721#discussion_r144345569
  
--- Diff: CHANGELOG.asciidoc ---
@@ -43,7 +43,13 @@ 
image::https://raw.githubusercontent.com/apache/tinkerpop/master/docs/static/ima
 * Fixed a bug in `Neo4jGremlinPlugin` that prevented it from loading 
properly in the `GremlinPythonScriptEngine`.
 * Fixed a bug in `ComputerVerificationStrategy` where child traversals 
were being analyzed prior to compilation.
 * Fixed a bug that prevented Gremlin from ordering lists and streams made 
of mixed number types.
-* Fixed a bug where `keepLabels` were being corrupted because a defensive 
copy was not being made when they were being set by `PathRetractionStrategy`. 
+* Fixed a bug where `keepLabels` were being corrupted because a defensive 
copy was not being made when they were being set by `PathRetractionStrategy`.
+* Added a recipe for OLAP traversals with Spark on Yarn
+
+Improvements
--- End diff --

No need to add the "Improvements" section. That gets added on release and 
we generate that output from JIRA.


> Recipe and missing manifest items for Spark on Yarn
> ---
>
> Key: TINKERPOP-1786
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1786
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: hadoop
>Affects Versions: 3.3.0, 3.1.8, 3.2.6
> Environment: gremlin-console
>Reporter: Marc de Lignie
>Priority: Minor
> Fix For: 3.2.7, 3.3.1
>
>
> Thorough documentation for running OLAP queries on Spark on Yarn has been 
> missing, keeping some users from getting the benefits of this nice feature of 
> the Tinkerpop stack and resulting in a significant number of questions on the 
> gremlin users list.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: ASF Board Draft Report - October 2017

[Stephen Mallette]

I've posted the Report to the board for this period...as there were no
additional comments, it went in as I have it written here.

On Mon, Oct 9, 2017 at 7:36 AM, Stephen Mallette 
wrote:

> Here is the draft of the July 2017 board report - please let me know if
> there is anything else to add.
>
> 
> ---
>
> ## Description:
> Apache TinkerPop is a graph computing framework for both graph databases
> (OLTP) and graph analytic systems (OLAP).
>
> ## Activity:
> TinkerPop released versions 3.1.8, 3.2.6, and 3.3.0, where 3.3.0
> represented
> a major new release with a number of new features. As part of this release
> cycle, we also offered a release candidate for a .NET Gremlin Language
> Variant[2], which should further expose TinkerPop beyond the Java
> ecosystem.
> Release 3.1.8 will be the last release on the 3.1.x line, leaving the
> community to focus efforts on maintaining 3.2.x and 3.3.x areas of
> development.
>
> In the wider TinkerPop community, it was announced that Apache S2Graph[1]
> implemented support for TinkerPop as part of their recent release. The
> community was also pleased to see that Apache TinkerPop had won a 2017
> Bossie Award in the "Best Databases and Analytics Tools" category[3] which
> covered InfoWorld's picks for the "best open source software for
> large-scale
> search, SQL, NoSQL, and streaming analytics".
>
> ## Issues:
> There are no issues requiring board attention at this time.
>
> ## Releases:
> - 3.1.8 (August 20, 2017)
> - 3.2.6 (August 20, 2017)
> - 3.3.0 (August 20, 2017)
>
> ## PMC/Committer:
>
> - Last PMC addition was Robert Dale - April 2017
> - Last committer addition was Florian Hockmann/Jorge Bay Gondra - July 2017
>
> ## Links
>
> [1] https://s2graph.apache.org/
> [2] https://s.apache.org/FTtC
> [3] https://s.apache.org/dK4t
>