[jira] [Commented] (TINKERPOP-1254) Support dropping traverser path information when it is no longer needed.

2016-07-10 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370018#comment-15370018
 ] 

ASF GitHub Bot commented on TINKERPOP-1254:
---

Github user twilmes commented on the issue:

https://github.com/apache/tinkerpop/pull/358
  
That's excellent!  Integration tests almost ran all the way through for me, 
got an odd failure during the Archetype - Server, noted below.  I'm going to 
rerun anyway with the latest commit and using docker this time.

```
[INFO] 

[INFO] Reactor Summary:
[INFO]
[INFO] Apache TinkerPop ... SUCCESS [  
9.540 s]
[INFO] Apache TinkerPop :: Gremlin Shaded . SUCCESS [  
3.640 s]
[INFO] Apache TinkerPop :: Gremlin Core ... SUCCESS [01:27 
min]
[INFO] Apache TinkerPop :: Gremlin Test ... SUCCESS [ 
11.518 s]
[INFO] Apache TinkerPop :: Gremlin Groovy . SUCCESS [ 
49.133 s]
[INFO] Apache TinkerPop :: Gremlin Groovy Test  SUCCESS [  
8.280 s]
[INFO] Apache TinkerPop :: TinkerGraph Gremlin  SUCCESS [03:40 
min]
[INFO] Apache TinkerPop :: Gremlin Benchmark .. SUCCESS [  
4.660 s]
[INFO] Apache TinkerPop :: Hadoop Gremlin . SUCCESS [06:14 
min]
[INFO] Apache TinkerPop :: Spark Gremlin .. SUCCESS [19:37 
min]
[INFO] Apache TinkerPop :: Giraph Gremlin . SUCCESS [  
02:54 h]
[INFO] Apache TinkerPop :: Neo4j Gremlin .. SUCCESS [  
4.160 s]
[INFO] Apache TinkerPop :: Gremlin Driver . SUCCESS [ 
11.340 s]
[INFO] Apache TinkerPop :: Gremlin Server . SUCCESS [12:58 
min]
[INFO] Apache TinkerPop :: Gremlin Console  SUCCESS [01:36 
min]
[INFO] Apache TinkerPop :: Gremlin Archetype .. SUCCESS [  
0.151 s]
[INFO] Apache TinkerPop :: Archetype - TinkerGraph  FAILURE [  
0.374 s]
[INFO] Apache TinkerPop :: Archetype - Server . SKIPPED
[INFO] 

[INFO] BUILD FAILURE
[INFO] 

[INFO] Total time: 03:41 h
[INFO] Finished at: 2016-07-10T15:06:49-05:00
[INFO] Final Memory: 66M/727M
[INFO] 

[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project gremlin-archetype-tinkergraph: Execution default-test of goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test failed: There was an 
error in the forke
d process
[ERROR] java.lang.NoClassDefFoundError: 
projects/standard/project/gremlin-archetype-tinkergraph/target/test-classes/com/test/example/AppTest
 (wrong name: com/test/example/AppTest)
[ERROR] at java.lang.ClassLoader.defineClass1(Native Method)
[ERROR] at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
[ERROR] at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
```


> Support dropping traverser path information when it is no longer needed.
> 
>
> Key: TINKERPOP-1254
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1254
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: process
>Affects Versions: 3.1.1-incubating
>Reporter: Marko A. Rodriguez
>Assignee: Ted Wilmes
>
> The most expensive traversals (especially in OLAP) are those that can not be 
> "bulked." There are various reasons why two traversers at the same object can 
> not be bulked, but the primary reason is {{PATH}} or {{LABELED_PATH}}. That 
> is, when the history of the traverser is required, the probability of two 
> traversers having the same history is low.
> A key to making traversals more efficient is to do as a much as possible to 
> remove historic information from a traverser so it can get bulked. How does 
> one do this? 
> {code}
> g.V.as('a').out().as('b').out().where(neq('a').and().neq('b')).both().name
> {code}
> The {{LABELED_PATH}} of "a" and "b" are required up to the {{where()}} and at 
> which point, at {{both()}}, they are no longer required. It would be smart to 
> support:
> {code}
> traverser.dropLabels(Set)
> traverser.dropPath()
> {code}
> We would then, via a {{TraversalOptimizationStrategy}} insert a step between 
> {{where()}} and {{both()}} called {{PathPruneStep}} which would be a 
> {{SideEffectStep}}. The strategy would know which labels were no longer 
> needed (via forward lookahead) and then do:
> {code}
> public class

Re: [DISCUSS] ASF Board Draft Report - July 2016

2016-07-10 Thread Ted Wilmes 
Looks good, thanks Stephen.

--Ted

On Fri, Jul 8, 2016 at 7:12 AM, Stephen Mallette 
wrote:

> ## Description:
>  Apache TinkerPop is a graph computing framework for both graph databases
>  (OLTP) and graph analytic systems (OLAP).
>
>  It was reported last month that TinkerPop had still not completely
> transferred
>  from incubator infrastructure. Since that time, Apache Infrastructure has
>  completed the transfer and TinkerPop is now fully on TLP infrastructure.
>
>  As alluded to in the previous report, two releases (3.1.3 and 3.2.1) are
> expected
>  to be up for vote by the community during the week of July 18th. Major
>  development continues to focus on opening TinkerPop to non-JVM programming
>  languages through the concept of Gremlin Language Variants[1]. TinkerPop
> is
>  currently focused on Python as the first of these languages to support.
>
> ## Issues:
>   There are no issues requiring board attention at this time.
>
> ## Releases:
>  - 3.1.2 (April 8, 2016)
>  - 3.2.0 (April 8, 2016)
>
> ## PMC/Committer:
>
>  - Last PMC addition was Dylan Millikin - May 2016
>  - Last committer addition was Michael Pollmeier - April 2016
>
> ## Links
>
> [1]
>
> http://tinkerpop.apache.org/docs/3.2.1-SNAPSHOT/tutorials/gremlin-language-variants/
>

[jira] [Commented] (TINKERPOP-1254) Support dropping traverser path information when it is no longer needed.

2016-07-10 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15369701#comment-15369701
 ] 

ASF GitHub Bot commented on TINKERPOP-1254:
---

Github user twilmes commented on the issue:

https://github.com/apache/tinkerpop/pull/358
  
I made a small update to `ReferencePath` to create new label Sets when a 
patch is detached.  This had been causing issues where the first set of labels 
for a path where being shared across `MutablePaths` after detachment.  A label 
would be removed from one, and therefore all of the traversers that had that 
label `Set` in their path, would be affected.

The `PathProcessors` are now respecting keepLabels null and labels are not 
dropped if `PrunePathStrategy` is not applied.

**PrunePathStrategy on**
```
g.V().match(
as("a").in("sungBy").as("b"),
as("a").in("sungBy").as("c"),
as("b").out("writtenBy").as("d"),
as("c").out("writtenBy").as("e"),
as("d").has("name", "George_Harrison"),
as("e").has("name", "Bob_Marley")).select("a").count().profile()

Traversal Metrics
Step   Count  
Traversers   Time (ms)% Dur

=
GraphStep(vertex,[]) 808
 808  44.21799.97
MatchStep(AND,[[MatchStartStep(a), ProfileStep,... 1
   1   0.004 0.01
  MatchStartStep(a)  808
 808  43.517
  VertexStep(IN,[sungBy],vertex) 501
 499  20.323
  MatchEndStep(b) (profiling ignored)   
   0.000
  MatchStartStep(a)2
   2   0.006
  VertexStep(IN,[sungBy],vertex) 156
 156   2.129
  MatchEndStep(c) (profiling ignored)   
   0.000
  MatchStartStep(b)  501
 499   5.126
  VertexStep(OUT,[writtenBy],vertex) 509
 504   3.423
  MatchEndStep(d) (profiling ignored)   
   0.000
  MatchStartStep(c)  156
 156   1.083
  VertexStep(OUT,[writtenBy],vertex) 157
 157   1.029
  MatchEndStep(e) (profiling ignored)   
   0.000
  MatchStartStep(d)  509
 266   1.685
  HasStep([name.eq(George_Harrison)])  2
   2   0.002
  MatchEndStep (profiling ignored)  
   0.000
  MatchStartStep(e)  157
  57   0.391
  HasStep([name.eq(Bob_Marley)])   1
   1   0.001
  MatchEndStep (profiling ignored)  
   0.000
SelectOneStep(a)   1
   1   0.003 0.01
CountGlobalStep1
   1   0.003 0.01
>TOTAL -
   -  44.228-
```
**PrunePathStrategy off**
```
Traversal Metrics
Step   Count  
Traversers   Time (ms)% Dur

=
GraphStep(vertex,[]) 808
 808   7.56599.84
MatchStep(AND,[[MatchStartStep(a), ProfileStep,... 1
   1   0.007 0.10
  MatchStartStep(a)  808
 808   5.726
  VertexStep(IN,[sungBy],vertex) 501
 499   9.532
  MatchEndStep(b) (profiling ignored)   
   0.000
  MatchStartStep(a)2
   2   0.007
  VertexStep(IN,[sungBy],vertex) 156
 156   1.803

[GitHub] tinkerpop issue #358: TINKERPOP-1254 Support dropping traverser path informa...

2016-07-10 Thread twilmes 
Github user twilmes commented on the issue:

https://github.com/apache/tinkerpop/pull/358
  
I made a small update to `ReferencePath` to create new label Sets when a 
patch is detached.  This had been causing issues where the first set of labels 
for a path where being shared across `MutablePaths` after detachment.  A label 
would be removed from one, and therefore all of the traversers that had that 
label `Set` in their path, would be affected.

The `PathProcessors` are now respecting keepLabels null and labels are not 
dropped if `PrunePathStrategy` is not applied.

**PrunePathStrategy on**
```
g.V().match(
as("a").in("sungBy").as("b"),
as("a").in("sungBy").as("c"),
as("b").out("writtenBy").as("d"),
as("c").out("writtenBy").as("e"),
as("d").has("name", "George_Harrison"),
as("e").has("name", "Bob_Marley")).select("a").count().profile()

Traversal Metrics
Step   Count  
Traversers   Time (ms)% Dur

=
GraphStep(vertex,[]) 808
 808  44.21799.97
MatchStep(AND,[[MatchStartStep(a), ProfileStep,... 1
   1   0.004 0.01
  MatchStartStep(a)  808
 808  43.517
  VertexStep(IN,[sungBy],vertex) 501
 499  20.323
  MatchEndStep(b) (profiling ignored)   
   0.000
  MatchStartStep(a)2
   2   0.006
  VertexStep(IN,[sungBy],vertex) 156
 156   2.129
  MatchEndStep(c) (profiling ignored)   
   0.000
  MatchStartStep(b)  501
 499   5.126
  VertexStep(OUT,[writtenBy],vertex) 509
 504   3.423
  MatchEndStep(d) (profiling ignored)   
   0.000
  MatchStartStep(c)  156
 156   1.083
  VertexStep(OUT,[writtenBy],vertex) 157
 157   1.029
  MatchEndStep(e) (profiling ignored)   
   0.000
  MatchStartStep(d)  509
 266   1.685
  HasStep([name.eq(George_Harrison)])  2
   2   0.002
  MatchEndStep (profiling ignored)  
   0.000
  MatchStartStep(e)  157
  57   0.391
  HasStep([name.eq(Bob_Marley)])   1
   1   0.001
  MatchEndStep (profiling ignored)  
   0.000
SelectOneStep(a)   1
   1   0.003 0.01
CountGlobalStep1
   1   0.003 0.01
>TOTAL -
   -  44.228-
```
**PrunePathStrategy off**
```
Traversal Metrics
Step   Count  
Traversers   Time (ms)% Dur

=
GraphStep(vertex,[]) 808
 808   7.56599.84
MatchStep(AND,[[MatchStartStep(a), ProfileStep,... 1
   1   0.007 0.10
  MatchStartStep(a)  808
 808   5.726
  VertexStep(IN,[sungBy],vertex) 501
 499   9.532
  MatchEndStep(b) (profiling ignored)   
   0.000
  MatchStartStep(a)2
   2   0.007
  VertexStep(IN,[sungBy],vertex) 156
 156   1.803
  MatchEndStep(c) (profiling ignored)   
   0.000
  MatchStartStep(b)  501
 499   3.221
  VertexStep(OUT,[writtenBy],vertex)

[jira] [Commented] (TINKERPOP-1360) intermittent error in spark-gremlin integration test

2016-07-10 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15369653#comment-15369653
 ] 

ASF GitHub Bot commented on TINKERPOP-1360:
---

GitHub user pluradj opened a pull request:

https://github.com/apache/tinkerpop/pull/361

TINKERPOP-1360: compare vertex property ids as Long

https://issues.apache.org/jira/browse/TINKERPOP-1360

Verified with:

```
mvn clean install && mvn verify -q -DskipIntegrationTests=false -pl 
spark-gremlin
```

And via Docker
```
docker/build.sh -t -i -n
```

@spmallette @dkuppitz Please try out multiple times in your environments 
since the error only happens 1 out of 4 scenarios (based on 2 different 
`RANDOM.nextBooolean()` calls). I manually forced each scenario in by setting 
specific boolean values in `HadoopGraphProvider` and `SparkHadoopGraphProvider`.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/tinkerpop TINKERPOP-1360

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/tinkerpop/pull/361.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #361


commit 30677963d4d5b74322796fb990161a94b43d00b1
Author: Jason Plurad 
Date:   2016-07-10T13:53:56Z

compare vertex property ids as Long




> intermittent error in spark-gremlin integration test
> 
>
> Key: TINKERPOP-1360
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1360
> Project: TinkerPop
>  Issue Type: Bug
>  Components: test-suite
>Affects Versions: 3.1.2-incubating
>Reporter: Jason Plurad
>Assignee: Jason Plurad
>Priority: Minor
>
> This error intermittently comes up during the Spark integration tests
> {noformat}
> Tests run: 457, Failures: 1, Errors: 0, Skipped: 150, Time elapsed: 110.068 
> sec <<< FAILURE! - in 
> org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputerProcessIntegrateTest
> g_V_hasXageX_propertiesXnameX(org.apache.tinkerpop.gremlin.process.traversal.step.map.PropertiesTest$Traversals)
>   Time elapsed: 0.185 sec  <<< FAILURE!
> java.lang.AssertionError: expected: java.lang.Integer<6> but was: 
> java.lang.Long<6>
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.failNotEquals(Assert.java:834)
>   at org.junit.Assert.assertEquals(Assert.java:118)
>   at org.junit.Assert.assertEquals(Assert.java:144)
>   at 
> org.apache.tinkerpop.gremlin.process.traversal.step.map.PropertiesTest.g_V_hasXageX_propertiesXnameX(PropertiesTest.java:103)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:497)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
>   at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>   at 
> org.apache.tinkerpop.gremlin.process.GremlinProcessRunner.runChild(GremlinProcessRunner.java:53)
>   at 
> org.apache.tinkerpop.gremlin.process.GremlinProcessRunner.runChild(GremlinProcessRunner.java:37)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at org.junit.runners.Suite.runChild(Suite.java:128)
>   at 
> org.apache.tinkerpop.gremlin.AbstractGremlinSuite.runChild(AbstractGremlinSuite.java:212)
>   at 
> org.apache.tinkerpop.gremlin.AbstractGremlinSuite.runChild(AbstractGremlinSuite.java:50)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at

[GitHub] tinkerpop pull request #361: TINKERPOP-1360: compare vertex property ids as ...

2016-07-10 Thread pluradj 
GitHub user pluradj opened a pull request:

https://github.com/apache/tinkerpop/pull/361

TINKERPOP-1360: compare vertex property ids as Long

https://issues.apache.org/jira/browse/TINKERPOP-1360

Verified with:

```
mvn clean install && mvn verify -q -DskipIntegrationTests=false -pl 
spark-gremlin
```

And via Docker
```
docker/build.sh -t -i -n
```

@spmallette @dkuppitz Please try out multiple times in your environments 
since the error only happens 1 out of 4 scenarios (based on 2 different 
`RANDOM.nextBooolean()` calls). I manually forced each scenario in by setting 
specific boolean values in `HadoopGraphProvider` and `SparkHadoopGraphProvider`.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/tinkerpop TINKERPOP-1360

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/tinkerpop/pull/361.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #361


commit 30677963d4d5b74322796fb990161a94b43d00b1
Author: Jason Plurad 
Date:   2016-07-10T13:53:56Z

compare vertex property ids as Long




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (TINKERPOP-1365) Eliminate Random.nextBoolean() usage in tests

2016-07-10 Thread Jason Plurad (JIRA) 
Jason Plurad created TINKERPOP-1365:
---

 Summary: Eliminate Random.nextBoolean() usage in tests
 Key: TINKERPOP-1365
 URL: https://issues.apache.org/jira/browse/TINKERPOP-1365
 Project: TinkerPop
  Issue Type: Improvement
  Components: test-suite
Affects Versions: 3.1.2-incubating, 3.2.0-incubating
Reporter: Jason Plurad


HadoopGraphProvider and SparkHadoopGraphProvider use RANDOM.nextBoolean() in an 
attempt to get coverage over different load scenarios. In practice, this ends 
up causing nondeterministic execution of the test suite, which can mask real 
errors lingering in the code or test code.

See https://issues.apache.org/jira/browse/TINKERPOP-1360




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)