https://issues.apache.org/jira/browse/TINKERPOP-2126

So, apparently, Spark 2.4.0 has some background logging going on, where it 
takes objects out of memory and dumps their `toString()` into the log. I don't 
think we can control when Spark is going to do that, so we have to work around 
it by making our `toString()` methods thread-safe or handle each 
`ConcurrentModificationException`. Guaranteeing thread-safety was easy for 
`TraverserSet` but turned out to be tricky (impossible?) for `ObjectWritable` 
as it's totally unclear where the objects are coming from and/or where they get 
modified while Spark is trying to log them.

I don't think we can make any arbitrary object thread-safe, but we can easily 
catch the `ConcurrentModificationException` and retry until we succeed. `while 
(true)` looks a bit scary, but I don't think that this can ever become an 
infinite loop.

`docker/build.sh -t -i -n` passed (as well as the external provider tests that 
consistently triggered the concurrency exceptions).

VOTE +1

[ Full content available at: https://github.com/apache/tinkerpop/pull/1044 ]
This message was relayed via gitbox.apache.org for dev@tinkerpop.apache.org

Reply via email to