[
https://issues.apache.org/jira/browse/TINKERPOP-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16750434#comment-16750434
]
ASF GitHub Bot commented on TINKERPOP-2126:
-------------------------------------------
dkuppitz commented on pull request #1044: TINKERPOP-2126 Fixed concurrency
issue in `ObjectWritable::toString()`.
URL: https://github.com/apache/tinkerpop/pull/1044
https://issues.apache.org/jira/browse/TINKERPOP-2126
So, apparently, Spark 2.4.0 has some background logging going on, where it
takes objects out of memory and dumps their `toString()` into the log. I don't
think we can control when Spark is going to do that, so we have to work around
it by making our `toString()` methods thread-safe or handle each
`ConcurrentModificationException`. Guaranteeing thread-safety was easy for
`TraverserSet` but turned out to be tricky (impossible?) for `ObjectWritable`
as it's totally unclear where the objects are coming from and/or where they get
modified while Spark is trying to log them.
I don't think we can make any arbitrary object thread-safe, but we can
easily catch the `ConcurrentModificationException` and retry until we succeed.
`while (true)` looks a bit scary, but I don't think that this can ever become
an infinite loop.
`docker/build.sh -t -i -n` passed (as well as the external provider tests
that consistently triggered the concurrency exceptions).
VOTE +1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> toString() methods not thread-safe
> ----------------------------------
>
> Key: TINKERPOP-2126
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2126
> Project: TinkerPop
> Issue Type: Bug
> Components: process
> Affects Versions: 3.3.5
> Reporter: Daniel Kuppitz
> Assignee: Daniel Kuppitz
> Priority: Major
> Fix For: 3.3.6, 3.4.1
>
>
> {{TraverserSet::toString()}} is not thread-safe as it uses an iterator on the
> internal map, which could concurrently be modified (most likely in OLAP
> environments).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
