[GitHub] tinkerpop pull request #729: TINKERPOP-1632: Create a set of default functio...

2017-10-13 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/tinkerpop/pull/729


---

[GitHub] tinkerpop pull request #729: TINKERPOP-1632: Create a set of default functio...

2017-10-09 Thread spmallette 
Github user spmallette commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/729#discussion_r143518649
  
--- Diff: docs/src/reference/the-traversal.asciidoc ---
@@ -1207,6 +1207,50 @@ system to leverage the filter for index lookups.
 IMPORTANT: A `where()`-step is a filter and thus, variables within a 
`where()` clause are not globally bound to the
 path of the traverser in `match()`. As such, `where()`-steps in `match()` 
are used for filtering, not binding.
 
+[[math-step]]
+=== Math Step
+
+The `math()`-step (*math*) enables scientific calculator functionality 
within Gremlin. This step deviates from the common
+function composition and nesting formalisms to provide an easy to read 
string-based math processor. Variables within the
+equation map to scopes in Gremlin -- e.g. path labels, side-effects, or 
incoming map keys. This step supports
+`by()`-modulation where the `by()`-modulators are applied in the order in 
which the variables are first referenced
+within the equation. Note that the reserved variable `_` refers to the 
current numeric traverser object incoming to the
+`math()`-step.
+
+[gremlin-groovy,modern]
+
+g.V().as('a').out('knows').as('b').math('a + b').by('age')
+g.V().as('a').out('created').as('b').
+  math('b + a').
+by(both().count().math('_ + 100')).
+by('age')
+g.withSideEffect('x',10).V().values('age').math('_ / x')

+g.withSack(1).V(1).repeat(sack(sum).by(constant(1))).times(10).emit().sack().math('sin
 _')
+
+
+The operators supported by the calculator include: `*`, `+`, `\`, `^`, and 
`%`.
+Furthermore, the following built in functions are provided:
+
+* `abs`: absolute value
+* `acos`: arc cosine
+* `asin`: arc sine
+* `atan`: arc tangent
+* `cbrt`: cubic root
+* `ceil`: nearest upper integer
+* `cos`: cosine
+* `cosh`: hyperbolic cosine
+* `exp`: euler's number raised to the power (`e^x`)
+* `floor`: nearest lower integer
+* `log`: logarithmus naturalis (base e)
+* `log10`: logarithm (base 10)
+* `log2`: logarithm (base 2)
+* `sin`: sine
+* `sinh`: hyperbolic sine
+* `sqrt`: square root
+* `tan`: tangent
+* `tanh`: hyperbolic tangent
+* `signum`: signum function
+
--- End diff --

please reference the ticket number here.


---

[GitHub] tinkerpop pull request #729: TINKERPOP-1632: Create a set of default functio...

2017-10-05 Thread twilmes 
Github user twilmes commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/729#discussion_r142936026
  
--- Diff: 
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/MathStep.java
 ---
@@ -0,0 +1,171 @@
+/*
+ *  Licensed to the Apache Software Foundation (ASF) under one
+ *  or more contributor license agreements.  See the NOTICE file
+ *  distributed with this work for additional information
+ *  regarding copyright ownership.  The ASF licenses this file
+ *  to you under the Apache License, Version 2.0 (the
+ *  "License"); you may not use this file except in compliance
+ *  with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing,
+ *  software distributed under the License is distributed on an
+ *  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ *  KIND, either express or implied.  See the License for the
+ *  specific language governing permissions and limitations
+ *  under the License.
+ */
+
+package org.apache.tinkerpop.gremlin.process.traversal.step.map;
+
+import net.objecthunter.exp4j.Expression;
+import net.objecthunter.exp4j.ExpressionBuilder;
+import org.apache.tinkerpop.gremlin.process.traversal.Pop;
+import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
+import org.apache.tinkerpop.gremlin.process.traversal.Traverser;
+import org.apache.tinkerpop.gremlin.process.traversal.step.ByModulating;
+import org.apache.tinkerpop.gremlin.process.traversal.step.PathProcessor;
+import org.apache.tinkerpop.gremlin.process.traversal.step.Scoping;
+import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
+import 
org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement;
+import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalRing;
+import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalUtil;
+import org.apache.tinkerpop.gremlin.structure.util.StringFactory;
+
+import java.util.HashSet;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+/**
+ * @author Marko A. Rodriguez (http://markorodriguez.com)
+ */
+public final class MathStep extends MapStep implements 
ByModulating, TraversalParent, Scoping, PathProcessor {
+
+private static final String CURRENT = "_";
+private final String equation;
+private final Set variables;
+private TraversalRing traversalRing = new 
TraversalRing<>();
+private Set keepLabels;
+
+public MathStep(final Traversal.Admin traversal, final String 
equation) {
+super(traversal);
+this.equation = equation;
+this.variables = MathStep.getVariables(this.equation);
+
+}
+
+@Override
+protected Traverser.Admin processNextStart() {
+return 
PathProcessor.processTraverserPathLabels(super.processNextStart(), 
this.keepLabels);
+}
+
+@Override
+protected Double map(final Traverser.Admin traverser) {
+final Expression expression = new ExpressionBuilder(this.equation)
--- End diff --

Yes, good point. I was trying to think of a way to do it with a 
`ThreadLocal` but can't think of one. I think I found a way around the 
threading issue that at least circumvents the repeated expression parsing. We 
can't get away from the extra object creation but the `Expression` constructor 
can take an `Expression` as input. That creates a copy with the tokens already 
pulled out so no parsing is needed. A golden copy of the expression could be 
created in the constructor and then used to seed the method-level expressions.

I ran this quick little test against grateful-dead.kryo. It's a nonsensical 
query, but I wanted to see how much of a difference parsing made.
`clockWithResult(2){g.V().hasLabel("song").math("_ * 100 + 200 / 
300").by('performances').limit(1000).toList()}`

original: 0.87 ms
`new Expression(expression)`: 0.38 ms


---

[GitHub] tinkerpop pull request #729: TINKERPOP-1632: Create a set of default functio...

2017-10-04 Thread twilmes 
Github user twilmes commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/729#discussion_r142801689
  
--- Diff: 
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/MathStep.java
 ---
@@ -0,0 +1,171 @@
+/*
+ *  Licensed to the Apache Software Foundation (ASF) under one
+ *  or more contributor license agreements.  See the NOTICE file
+ *  distributed with this work for additional information
+ *  regarding copyright ownership.  The ASF licenses this file
+ *  to you under the Apache License, Version 2.0 (the
+ *  "License"); you may not use this file except in compliance
+ *  with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing,
+ *  software distributed under the License is distributed on an
+ *  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ *  KIND, either express or implied.  See the License for the
+ *  specific language governing permissions and limitations
+ *  under the License.
+ */
+
+package org.apache.tinkerpop.gremlin.process.traversal.step.map;
+
+import net.objecthunter.exp4j.Expression;
+import net.objecthunter.exp4j.ExpressionBuilder;
+import org.apache.tinkerpop.gremlin.process.traversal.Pop;
+import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
+import org.apache.tinkerpop.gremlin.process.traversal.Traverser;
+import org.apache.tinkerpop.gremlin.process.traversal.step.ByModulating;
+import org.apache.tinkerpop.gremlin.process.traversal.step.PathProcessor;
+import org.apache.tinkerpop.gremlin.process.traversal.step.Scoping;
+import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
+import 
org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement;
+import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalRing;
+import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalUtil;
+import org.apache.tinkerpop.gremlin.structure.util.StringFactory;
+
+import java.util.HashSet;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+/**
+ * @author Marko A. Rodriguez (http://markorodriguez.com)
+ */
+public final class MathStep extends MapStep implements 
ByModulating, TraversalParent, Scoping, PathProcessor {
+
+private static final String CURRENT = "_";
+private final String equation;
+private final Set variables;
+private TraversalRing traversalRing = new 
TraversalRing<>();
+private Set keepLabels;
+
+public MathStep(final Traversal.Admin traversal, final String 
equation) {
+super(traversal);
+this.equation = equation;
+this.variables = MathStep.getVariables(this.equation);
+
+}
+
+@Override
+protected Traverser.Admin processNextStart() {
+return 
PathProcessor.processTraverserPathLabels(super.processNextStart(), 
this.keepLabels);
+}
+
+@Override
+protected Double map(final Traverser.Admin traverser) {
+final Expression expression = new ExpressionBuilder(this.equation)
--- End diff --

Looking at the exp4j source, I think an `Expression`  can be reused. If so, 
this could be moved up into the constructor to save the repeated initialization 
cost.


---

[GitHub] tinkerpop pull request #729: TINKERPOP-1632: Create a set of default functio...

2017-10-04 Thread okram 
GitHub user okram opened a pull request:

https://github.com/apache/tinkerpop/pull/729

TINKERPOP-1632: Create a set of default functions

https://issues.apache.org/jira/browse/TINKERPOP-1632

This is an implementation of lambda-less math capabilities in Gremlin 
leveraging the String-based calculator model proposed by @twilmes. This model 
deviates from Gremlin's standard function composition and nesting model to 
provide a easy to read "math processor" that leverages Gremlin scopes for data 
access (e.g. path data, map keys, and side-effects). The feature is 
encapsulated in `MathStep` which implements `PathProcessor` and `Scoping`. 
Furthermore, `by()`-modulation is supported. `by()`-modulators are applied in 
the order in which the variable is first used in the equation. Thus:

```
g.V().as("a").out("knows").by("b").
  math("b + a").
by(out().count()).
by("age")
```

In the above: `a` -> `age` and `b` -> `out().count()`. More complex 
examples are provided below:

```
gremlin> g = TinkerFactory.createModern().traversal()
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
gremlin> g.V().as('a').out('knows').as('b').math('a + b').by('age')
==>56.0
==>61.0
gremlin> g.V().as('a').out('created').as('b').
..1>   math('b + a').
..2> by(both().count().math('_ + 100')).
..3> by('age')
==>132.0
==>133.0
==>135.0
==>138.0
gremlin> g.withSideEffect('x',10).V().values('age').math('_ / x')
==>2.9
==>2.7
==>3.2
==>3.5
gremlin> 
g.withSack(1).V(1).repeat(sack(sum).by(constant(1))).times(10).emit().sack().math('sin
 _')
==>0.9092974268256817
==>0.1411200080598672
==>-0.7568024953079282
==>-0.9589242746631385
==>-0.27941549819892586
==>0.6569865987187891
==>0.9893582466233818
==>0.4121184852417566
==>-0.5440211108893698
==>-0.902065507035
```

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/tinkerpop TINKERPOP-1632

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/tinkerpop/pull/729.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #729


commit 6f1a1f7028f285c6e45b7e7596320710b91114c5
Author: Marko A. Rodriguez 
Date:   2017-10-03T23:37:10Z

first non-tested implementation of math()-step. It uses the @twilmes model 
which parses a String representation of the expression.

commit 624a8d706e150cc15dc2412d66d3ac9a61b90682
Author: Marko A. Rodriguez 
Date:   2017-10-04T16:27:48Z

I wrote a small parser that is able to extract variables from an exp4j 
equation. So far, it is pretty durable. This is a much cleaner way of 
determining variables than via label, side-effect, and map analysis. However, 
this means that the by()-modulations are with respects to the order in which 
the variables are contained in the equation.

commit c36493b6d66406b0d6b50e6c292e6d43216b4c4c
Author: Marko A. Rodriguez 
Date:   2017-10-04T16:42:06Z

added MathTest to test math() using step labels, side-effects, and implicit 
current -- and with various uses of by()-modulation. This is a really cool step.

commit dfddccaaa85898e046e6b7a5bd81da3d94777d7d
Author: Marko A. Rodriguez 
Date:   2017-10-04T17:01:53Z

added MathStepTest to test hashCode() and the custom variable finder. Found 
a couple of problems in my parser and I fixed them. Forgot to make MathStep a 
PathProcessor so that OLAP is smart about data access. Both OLTP and OLAP tests 
pass.

commit 7217823c3ea73f2d32ffa975e417dcd49d40cbdd
Author: Marko A. Rodriguez 
Date:   2017-10-04T17:30:55Z

added math()-step to the reference docs and updated CHANGELOG and upgrade 
docs.




---