[
https://issues.apache.org/jira/browse/TINKERPOP-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17757636#comment-17757636
]
Cole Greer commented on TINKERPOP-2946:
---------------------------------------
One flaky test identified by the new TinkerShuffleGraph in the PR is the
[shortest path feature
test|https://github.com/apache/tinkerpop/blob/master/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/test/features/integrated/Paths.feature].
This test will occasionally produce duplicate results when run on a graph
which does not guarantee ordering.
Failures produce the following error:
{code:java}
java.lang.AssertionError:
Expected :30
Actual :32
<Click to see difference>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:633)
at
org.apache.tinkerpop.gremlin.features.StepDefinition.theResultShouldBeUnordered(StepDefinition.java:314)
at ✽.the result should be
unordered(classpath:org/apache/tinkerpop/gremlin/test/features/integrated/Paths.feature:52)
{code}
Running it through the shuffle graph many times, I've seen 3 situations arise:
1. The results are correct
2. The results contain duplicate:
==> marko
==> vadas
3. The results contain duplicate:
==> lop
==> peter
Running it always seems to produce one of those 3 situations with seemingly
equal probability. It is produced simply by shuffling the iterators returned by
Graph.vertices() and Graph.edges()
Here are all of the failed result sets I've seen from that test in shuffle
graph:
//marko and vadas
[[ripple, josh, lop], [josh, marko, vadas], [josh, lop], [peter, lop, marko],
[ripple, josh, marko], [josh, marko], [marko, lop], [lop, marko], [josh, lop,
peter], [peter, lop, josh], [vadas, marko], [ripple, josh], [marko], [marko],
[josh], [ripple], [josh, ripple], [peter, lop], [vadas, marko, josh], [lop,
josh, ripple], [marko, josh], [lop, marko, vadas], [peter], [lop], [vadas],
[vadas], [marko, josh, ripple], [marko, vadas], [vadas, marko, lop], [lop,
peter], [lop, josh], [marko, lop, peter]]
// lop and peter
[[josh, marko, vadas], [ripple, josh, lop], [josh, lop], [peter, lop, marko],
[ripple, josh, marko], [josh, marko], [marko, lop], [lop, marko], [peter, lop,
josh], [josh, lop, peter], [vadas, marko], [ripple, josh], [josh], [ripple],
[marko], [peter, lop], [josh, ripple], [vadas, marko, josh], [lop, josh,
ripple], [marko, josh], [lop, marko, vadas], [lop], [lop], [vadas], [peter],
[peter], [marko, josh, ripple], [marko, vadas], [vadas, marko, lop], [lop,
peter], [lop, josh], [marko, lop, peter]]
//marko and vadas
[[ripple, josh, lop], [josh, marko, vadas], [josh, lop], [peter, lop, marko],
[ripple, josh, marko], [josh, marko], [lop, marko], [marko, lop], [peter, lop,
josh], [josh, lop, peter], [vadas, marko], [ripple, josh], [marko], [marko],
[josh], [ripple], [peter, lop], [josh, ripple], [vadas, marko, josh], [lop,
josh, ripple], [marko, josh], [lop, marko, vadas], [peter], [lop], [vadas],
[vadas], [marko, josh, ripple], [marko, vadas], [vadas, marko, lop], [lop,
peter], [lop, josh], [marko, lop, peter]]
//lop and peter
[[josh, marko, vadas], [ripple, josh, lop], [josh, lop], [peter, lop, marko],
[ripple, josh, marko], [josh, marko], [lop, marko], [marko, lop], [peter, lop,
josh], [josh, lop, peter], [ripple, josh], [vadas, marko], [josh], [ripple],
[marko], [peter, lop], [josh, ripple], [lop, josh, ripple], [vadas, marko,
josh], [marko, josh], [lop, marko, vadas], [lop], [lop], [vadas], [peter],
[peter], [marko, josh, ripple], [marko, vadas], [vadas, marko, lop], [lop,
peter], [lop, josh], [marko, lop, peter]]
> Resolve ordering issues in gherkin tests
> ----------------------------------------
>
> Key: TINKERPOP-2946
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2946
> Project: TinkerPop
> Issue Type: Improvement
> Components: test-suite
> Affects Versions: 3.6.2
> Reporter: Stephen Mallette
> Priority: Blocker
>
> Gremlin doesn't enforce order, yet many of the gherkin tests carry that
> expectation.
> 1. Develop a mechanism to ensure that tests don't come with that expectation.
> 2. Fix tests that have this assumption and improve test language and
> assertion logic as needed.
> One way to do (1) would be to provide a configuration to TinkerGraph that
> would make it behave more like most graph databases and purposefully
> randomize result streams. Perhaps this configuration could be used as part of
> a integration test to ensure a regular {{mvn clean install}} works
> deterministically?? Doing (1) should yield information on what needs to be
> done with (2). Some of (2) may be test fixes, but I think assertion logic
> around containers may need to be improved. When asserting a {{List}} result
> for example we often want to say that it contains "all items" rather than
> "all items in the specified order". We have that sort of gherkin language at
> the overall result level but not at the individual one.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
