[GitHub] tinkerpop pull request #729: TINKERPOP-1632: Create a set of default functio...
Github user asfgit closed the pull request at: https://github.com/apache/tinkerpop/pull/729 ---
[GitHub] tinkerpop pull request #729: TINKERPOP-1632: Create a set of default functio...
Github user spmallette commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/729#discussion_r143518649
--- Diff: docs/src/reference/the-traversal.asciidoc ---
@@ -1207,6 +1207,50 @@ system to leverage the filter for index lookups.
IMPORTANT: A `where()`-step is a filter and thus, variables within a
`where()` clause are not globally bound to the
path of the traverser in `match()`. As such, `where()`-steps in `match()`
are used for filtering, not binding.
+[[math-step]]
+=== Math Step
+
+The `math()`-step (*math*) enables scientific calculator functionality
within Gremlin. This step deviates from the common
+function composition and nesting formalisms to provide an easy to read
string-based math processor. Variables within the
+equation map to scopes in Gremlin -- e.g. path labels, side-effects, or
incoming map keys. This step supports
+`by()`-modulation where the `by()`-modulators are applied in the order in
which the variables are first referenced
+within the equation. Note that the reserved variable `_` refers to the
current numeric traverser object incoming to the
+`math()`-step.
+
+[gremlin-groovy,modern]
+
+g.V().as('a').out('knows').as('b').math('a + b').by('age')
+g.V().as('a').out('created').as('b').
+ math('b + a').
+by(both().count().math('_ + 100')).
+by('age')
+g.withSideEffect('x',10).V().values('age').math('_ / x')
+g.withSack(1).V(1).repeat(sack(sum).by(constant(1))).times(10).emit().sack().math('sin
_')
+
+
+The operators supported by the calculator include: `*`, `+`, `\`, `^`, and
`%`.
+Furthermore, the following built in functions are provided:
+
+* `abs`: absolute value
+* `acos`: arc cosine
+* `asin`: arc sine
+* `atan`: arc tangent
+* `cbrt`: cubic root
+* `ceil`: nearest upper integer
+* `cos`: cosine
+* `cosh`: hyperbolic cosine
+* `exp`: euler's number raised to the power (`e^x`)
+* `floor`: nearest lower integer
+* `log`: logarithmus naturalis (base e)
+* `log10`: logarithm (base 10)
+* `log2`: logarithm (base 2)
+* `sin`: sine
+* `sinh`: hyperbolic sine
+* `sqrt`: square root
+* `tan`: tangent
+* `tanh`: hyperbolic tangent
+* `signum`: signum function
+
--- End diff --
please reference the ticket number here.
---
[GitHub] tinkerpop pull request #729: TINKERPOP-1632: Create a set of default functio...
Github user twilmes commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/729#discussion_r142936026
--- Diff:
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/MathStep.java
---
@@ -0,0 +1,171 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.tinkerpop.gremlin.process.traversal.step.map;
+
+import net.objecthunter.exp4j.Expression;
+import net.objecthunter.exp4j.ExpressionBuilder;
+import org.apache.tinkerpop.gremlin.process.traversal.Pop;
+import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
+import org.apache.tinkerpop.gremlin.process.traversal.Traverser;
+import org.apache.tinkerpop.gremlin.process.traversal.step.ByModulating;
+import org.apache.tinkerpop.gremlin.process.traversal.step.PathProcessor;
+import org.apache.tinkerpop.gremlin.process.traversal.step.Scoping;
+import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
+import
org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement;
+import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalRing;
+import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalUtil;
+import org.apache.tinkerpop.gremlin.structure.util.StringFactory;
+
+import java.util.HashSet;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+/**
+ * @author Marko A. Rodriguez (http://markorodriguez.com)
+ */
+public final class MathStep extends MapStep implements
ByModulating, TraversalParent, Scoping, PathProcessor {
+
+private static final String CURRENT = "_";
+private final String equation;
+private final Set variables;
+private TraversalRing traversalRing = new
TraversalRing<>();
+private Set keepLabels;
+
+public MathStep(final Traversal.Admin traversal, final String
equation) {
+super(traversal);
+this.equation = equation;
+this.variables = MathStep.getVariables(this.equation);
+
+}
+
+@Override
+protected Traverser.Admin processNextStart() {
+return
PathProcessor.processTraverserPathLabels(super.processNextStart(),
this.keepLabels);
+}
+
+@Override
+protected Double map(final Traverser.Admin traverser) {
+final Expression expression = new ExpressionBuilder(this.equation)
--- End diff --
Yes, good point. I was trying to think of a way to do it with a
`ThreadLocal` but can't think of one. I think I found a way around the
threading issue that at least circumvents the repeated expression parsing. We
can't get away from the extra object creation but the `Expression` constructor
can take an `Expression` as input. That creates a copy with the tokens already
pulled out so no parsing is needed. A golden copy of the expression could be
created in the constructor and then used to seed the method-level expressions.
I ran this quick little test against grateful-dead.kryo. It's a nonsensical
query, but I wanted to see how much of a difference parsing made.
`clockWithResult(2){g.V().hasLabel("song").math("_ * 100 + 200 /
300").by('performances').limit(1000).toList()}`
original: 0.87 ms
`new Expression(expression)`: 0.38 ms
---
[GitHub] tinkerpop pull request #729: TINKERPOP-1632: Create a set of default functio...
Github user okram commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/729#discussion_r142803300
--- Diff:
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/MathStep.java
---
@@ -0,0 +1,171 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.tinkerpop.gremlin.process.traversal.step.map;
+
+import net.objecthunter.exp4j.Expression;
+import net.objecthunter.exp4j.ExpressionBuilder;
+import org.apache.tinkerpop.gremlin.process.traversal.Pop;
+import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
+import org.apache.tinkerpop.gremlin.process.traversal.Traverser;
+import org.apache.tinkerpop.gremlin.process.traversal.step.ByModulating;
+import org.apache.tinkerpop.gremlin.process.traversal.step.PathProcessor;
+import org.apache.tinkerpop.gremlin.process.traversal.step.Scoping;
+import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
+import
org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement;
+import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalRing;
+import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalUtil;
+import org.apache.tinkerpop.gremlin.structure.util.StringFactory;
+
+import java.util.HashSet;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+/**
+ * @author Marko A. Rodriguez (http://markorodriguez.com)
+ */
+public final class MathStep extends MapStep implements
ByModulating, TraversalParent, Scoping, PathProcessor {
+
+private static final String CURRENT = "_";
+private final String equation;
+private final Set variables;
+private TraversalRing traversalRing = new
TraversalRing<>();
+private Set keepLabels;
+
+public MathStep(final Traversal.Admin traversal, final String
equation) {
+super(traversal);
+this.equation = equation;
+this.variables = MathStep.getVariables(this.equation);
+
+}
+
+@Override
+protected Traverser.Admin processNextStart() {
+return
PathProcessor.processTraverserPathLabels(super.processNextStart(),
this.keepLabels);
+}
+
+@Override
+protected Double map(final Traverser.Admin traverser) {
+final Expression expression = new ExpressionBuilder(this.equation)
--- End diff --
Not with threading. Thus, OLAP and any other execution engine that uses
threads will get inconsistent results.
---
[GitHub] tinkerpop pull request #729: TINKERPOP-1632: Create a set of default functio...
Github user twilmes commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/729#discussion_r142801689
--- Diff:
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/MathStep.java
---
@@ -0,0 +1,171 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.tinkerpop.gremlin.process.traversal.step.map;
+
+import net.objecthunter.exp4j.Expression;
+import net.objecthunter.exp4j.ExpressionBuilder;
+import org.apache.tinkerpop.gremlin.process.traversal.Pop;
+import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
+import org.apache.tinkerpop.gremlin.process.traversal.Traverser;
+import org.apache.tinkerpop.gremlin.process.traversal.step.ByModulating;
+import org.apache.tinkerpop.gremlin.process.traversal.step.PathProcessor;
+import org.apache.tinkerpop.gremlin.process.traversal.step.Scoping;
+import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent;
+import
org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement;
+import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalRing;
+import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalUtil;
+import org.apache.tinkerpop.gremlin.structure.util.StringFactory;
+
+import java.util.HashSet;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+/**
+ * @author Marko A. Rodriguez (http://markorodriguez.com)
+ */
+public final class MathStep extends MapStep implements
ByModulating, TraversalParent, Scoping, PathProcessor {
+
+private static final String CURRENT = "_";
+private final String equation;
+private final Set variables;
+private TraversalRing traversalRing = new
TraversalRing<>();
+private Set keepLabels;
+
+public MathStep(final Traversal.Admin traversal, final String
equation) {
+super(traversal);
+this.equation = equation;
+this.variables = MathStep.getVariables(this.equation);
+
+}
+
+@Override
+protected Traverser.Admin processNextStart() {
+return
PathProcessor.processTraverserPathLabels(super.processNextStart(),
this.keepLabels);
+}
+
+@Override
+protected Double map(final Traverser.Admin traverser) {
+final Expression expression = new ExpressionBuilder(this.equation)
--- End diff --
Looking at the exp4j source, I think an `Expression` can be reused. If so,
this could be moved up into the constructor to save the repeated initialization
cost.
---
[GitHub] tinkerpop pull request #729: TINKERPOP-1632: Create a set of default functio...
GitHub user okram opened a pull request:
https://github.com/apache/tinkerpop/pull/729
TINKERPOP-1632: Create a set of default functions
https://issues.apache.org/jira/browse/TINKERPOP-1632
This is an implementation of lambda-less math capabilities in Gremlin
leveraging the String-based calculator model proposed by @twilmes. This model
deviates from Gremlin's standard function composition and nesting model to
provide a easy to read "math processor" that leverages Gremlin scopes for data
access (e.g. path data, map keys, and side-effects). The feature is
encapsulated in `MathStep` which implements `PathProcessor` and `Scoping`.
Furthermore, `by()`-modulation is supported. `by()`-modulators are applied in
the order in which the variable is first used in the equation. Thus:
```
g.V().as("a").out("knows").by("b").
math("b + a").
by(out().count()).
by("age")
```
In the above: `a` -> `age` and `b` -> `out().count()`. More complex
examples are provided below:
```
gremlin> g = TinkerFactory.createModern().traversal()
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
gremlin> g.V().as('a').out('knows').as('b').math('a + b').by('age')
==>56.0
==>61.0
gremlin> g.V().as('a').out('created').as('b').
..1> math('b + a').
..2> by(both().count().math('_ + 100')).
..3> by('age')
==>132.0
==>133.0
==>135.0
==>138.0
gremlin> g.withSideEffect('x',10).V().values('age').math('_ / x')
==>2.9
==>2.7
==>3.2
==>3.5
gremlin>
g.withSack(1).V(1).repeat(sack(sum).by(constant(1))).times(10).emit().sack().math('sin
_')
==>0.9092974268256817
==>0.1411200080598672
==>-0.7568024953079282
==>-0.9589242746631385
==>-0.27941549819892586
==>0.6569865987187891
==>0.9893582466233818
==>0.4121184852417566
==>-0.5440211108893698
==>-0.902065507035
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/tinkerpop TINKERPOP-1632
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/tinkerpop/pull/729.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #729
commit 6f1a1f7028f285c6e45b7e7596320710b91114c5
Author: Marko A. Rodriguez
Date: 2017-10-03T23:37:10Z
first non-tested implementation of math()-step. It uses the @twilmes model
which parses a String representation of the expression.
commit 624a8d706e150cc15dc2412d66d3ac9a61b90682
Author: Marko A. Rodriguez
Date: 2017-10-04T16:27:48Z
I wrote a small parser that is able to extract variables from an exp4j
equation. So far, it is pretty durable. This is a much cleaner way of
determining variables than via label, side-effect, and map analysis. However,
this means that the by()-modulations are with respects to the order in which
the variables are contained in the equation.
commit c36493b6d66406b0d6b50e6c292e6d43216b4c4c
Author: Marko A. Rodriguez
Date: 2017-10-04T16:42:06Z
added MathTest to test math() using step labels, side-effects, and implicit
current -- and with various uses of by()-modulation. This is a really cool step.
commit dfddccaaa85898e046e6b7a5bd81da3d94777d7d
Author: Marko A. Rodriguez
Date: 2017-10-04T17:01:53Z
added MathStepTest to test hashCode() and the custom variable finder. Found
a couple of problems in my parser and I fixed them. Forgot to make MathStep a
PathProcessor so that OLAP is smart about data access. Both OLTP and OLAP tests
pass.
commit 7217823c3ea73f2d32ffa975e417dcd49d40cbdd
Author: Marko A. Rodriguez
Date: 2017-10-04T17:30:55Z
added math()-step to the reference docs and updated CHANGELOG and upgrade
docs.
---
