[
https://issues.apache.org/jira/browse/TINKERPOP-3087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Bennington updated TINKERPOP-3087:
----------------------------------------
Description:
Ocasionally, I get the following panic in the gremlin-go driver:
{noformat}
10:22:39.895 <successfully added a vertex to janusgraph>
10:22:44.487 panic: runtime error: invalid memory address or nil pointer
dereference
10:22:44.487 [signal SIGSEGV: segmentation violation code=0x1 addr=0x78
pc=0x11fdd16]
10:22:44.487
10:22:44.487 goroutine 275 [running]:
10:22:44.487
github.com/apache/tinkerpop/gremlin-go/v3/driver.(*gremlinServerWSProtocol).responseHandler(0xc0006be6c0,
0xc0001dfc60, {{0x9, 0xf6, 0x33, 0x7f, 0x19, 0x16, 0x4e, 0xc9, ...}, ...})
10:22:44.487
/go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3@v3.7.1/driver/protocol.go:116
+0x7d6
10:22:44.487
github.com/apache/tinkerpop/gremlin-go/v3/driver.(*gremlinServerWSProtocol).readLoop(0xc0006be6c0,
0xc0001dfc60, 0xc0001dfca0)
10:22:44.487
/go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3@v3.7.1/driver/protocol.go:82
+0x272
10:22:44.487 created by
github.com/apache/tinkerpop/gremlin-go/v3/driver.newGremlinServerWSProtocol in
goroutine 272
10:22:44.487
/go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3@v3.7.1/driver/protocol.go:197
+0x23a{noformat}
I can't reliably reproduce, it's never happened locally. It's occurred just
after doing some work in the logs above, but it's also happened after a period
of doing nothing.
First glance looks like it's a nil being returned from the `resultSets`
synchronizedMap, something getting in and removing it, which should only happen
when the `channelResultSet` itself is closed? Seems like a race condition to me.
I think the sensible way to handle this, if it is what I suspect (I've no proof
yet), would be to
* Store a reference to the ResultSet when first loading it in responseHandler
* Make `channelResultSet` discard modifications after close OR
* responseHandler needs to aquire the `channelMutex` somehow for the duration
of modifications (I think I prefer this, it's more obvious to what's happening)
I did try and get the project tests to run, but it's been _years_ since I've
had to use maven and I can't seem to find a good idiots guide to working with
the project from a non java POV - I need to know how to build the docker images
the tests require. If someone could help me get started I'd be more than happy
to contribute a PR (although given the nature, a reliable test may be hard to
write).
As it stands, I'm probably going to modify my fork and mod replace, see if a
fix would actually work in my deployments.
was:
Ocasionally, I get the following panic in the gremlin-go driver:
```
{color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}39.895{color}{color:#000000}
<{color}{color:#001080}successfully{color}{color:#000000}
{color}{color:#001080}added{color}{color:#000000}
{color}{color:#001080}a{color}{color:#000000}
{color}{color:#001080}vertex{color}{color:#000000}
{color}{color:#001080}to{color}{color:#000000}
{color}{color:#001080}janusgraph{color}{color:#000000}>{color}
{color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000}
{color}{color:#001080}panic{color}{color:#000000}:
{color}{color:#001080}runtime{color}{color:#000000}
{color}{color:#267f99}error{color}{color:#000000}:
{color}{color:#001080}invalid{color}{color:#000000}
{color}{color:#001080}memory{color}{color:#000000}
{color}{color:#001080}address{color}{color:#000000}
{color}{color:#001080}or{color}{color:#000000}
{color}{color:#0000ff}nil{color}{color:#000000}
{color}{color:#001080}pointer{color}{color:#000000}
{color}{color:#001080}dereference{color}
{color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000}
[{color}{color:#001080}signal{color}{color:#000000}
{color}{color:#001080}SIGSEGV{color}{color:#000000}:
{color}{color:#001080}segmentation{color}{color:#000000}
{color}{color:#001080}violation{color}{color:#000000}
{color}{color:#001080}code{color}{color:#000000}={color}{color:#098658}0x1{color}{color:#000000}
{color}{color:#001080}addr{color}{color:#000000}={color}{color:#098658}0x78{color}{color:#000000}
{color}{color:#001080}pc{color}{color:#000000}={color}{color:#098658}0x11fdd16{color}{color:#000000}]{color}
{color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000}
{color}
{color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000}
{color}{color:#001080}goroutine{color}{color:#000000}
{color}{color:#098658}275{color}{color:#000000}
[{color}{color:#001080}running{color}{color:#000000}]:{color}
{color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000}
{color}{color:#001080}github{color}{color:#000000}.{color}{color:#001080}com{color}{color:#000000}/{color}{color:#001080}apache{color}{color:#000000}/{color}{color:#001080}tinkerpop{color}{color:#000000}/{color}{color:#001080}gremlin{color}{color:#000000}-{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}v3{color}{color:#000000}/{color}{color:#001080}driver{color}{color:#000000}.(*{color}{color:#267f99}gremlinServerWSProtocol{color}{color:#000000}).{color}{color:#795e26}responseHandler{color}{color:#000000}({color}{color:#098658}0xc0006be6c0{color}{color:#000000},
{color}{color:#098658}0xc0001dfc60{color}{color:#000000},
{{{color}{color:#098658}0x9{color}{color:#000000},
{color}{color:#098658}0xf6{color}{color:#000000},
{color}{color:#098658}0x33{color}{color:#000000},
{color}{color:#098658}0x7f{color}{color:#000000},
{color}{color:#098658}0x19{color}{color:#000000},
{color}{color:#098658}0x16{color}{color:#000000},
{color}{color:#098658}0x4e{color}{color:#000000},
{color}{color:#098658}0xc9{color}{color:#000000}, ...}, ...}){color}
{color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000}
/{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}pkg{color}{color:#000000}/{color}{color:#001080}mod{color}{color:#000000}/{color}{color:#001080}github{color}{color:#000000}.{color}{color:#001080}com{color}{color:#000000}/{color}{color:#001080}apache{color}{color:#000000}/{color}{color:#001080}tinkerpop{color}{color:#000000}/{color}{color:#001080}gremlin{color}{color:#000000}-{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}v3{color}{color:#000000}@{color}{color:#001080}v3{color}{color:#000000}.{color}{color:#098658}7.1{color}{color:#000000}/{color}{color:#001080}driver{color}{color:#000000}/{color}{color:#001080}protocol{color}{color:#000000}.{color}{color:#af00db}go{color}{color:#000000}:{color}{color:#098658}116{color}{color:#000000}
+{color}{color:#098658}0x7d6{color}
{color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000}
{color}{color:#001080}github{color}{color:#000000}.{color}{color:#001080}com{color}{color:#000000}/{color}{color:#001080}apache{color}{color:#000000}/{color}{color:#001080}tinkerpop{color}{color:#000000}/{color}{color:#001080}gremlin{color}{color:#000000}-{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}v3{color}{color:#000000}/{color}{color:#001080}driver{color}{color:#000000}.(*{color}{color:#267f99}gremlinServerWSProtocol{color}{color:#000000}).{color}{color:#795e26}readLoop{color}{color:#000000}({color}{color:#098658}0xc0006be6c0{color}{color:#000000},
{color}{color:#098658}0xc0001dfc60{color}{color:#000000},
{color}{color:#098658}0xc0001dfca0{color}{color:#000000}){color}
{color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000}
/{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}pkg{color}{color:#000000}/{color}{color:#001080}mod{color}{color:#000000}/{color}{color:#001080}github{color}{color:#000000}.{color}{color:#001080}com{color}{color:#000000}/{color}{color:#001080}apache{color}{color:#000000}/{color}{color:#001080}tinkerpop{color}{color:#000000}/{color}{color:#001080}gremlin{color}{color:#000000}-{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}v3{color}{color:#000000}@{color}{color:#001080}v3{color}{color:#000000}.{color}{color:#098658}7.1{color}{color:#000000}/{color}{color:#001080}driver{color}{color:#000000}/{color}{color:#001080}protocol{color}{color:#000000}.{color}{color:#af00db}go{color}{color:#000000}:{color}{color:#098658}82{color}{color:#000000}
+{color}{color:#098658}0x272{color}
{color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000}
{color}{color:#001080}created{color}{color:#000000}
{color}{color:#001080}by{color}{color:#000000}
{color}{color:#001080}github{color}{color:#000000}.{color}{color:#001080}com{color}{color:#000000}/{color}{color:#001080}apache{color}{color:#000000}/{color}{color:#001080}tinkerpop{color}{color:#000000}/{color}{color:#001080}gremlin{color}{color:#000000}-{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}v3{color}{color:#000000}/{color}{color:#001080}driver{color}{color:#000000}.{color}{color:#001080}newGremlinServerWSProtocol{color}{color:#000000}
{color}{color:#001080}in{color}{color:#000000}
{color}{color:#001080}goroutine{color}{color:#000000}
{color}{color:#098658}272{color}
{color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000}
/{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}pkg{color}{color:#000000}/{color}{color:#001080}mod{color}{color:#000000}/{color}{color:#001080}github{color}{color:#000000}.{color}{color:#001080}com{color}{color:#000000}/{color}{color:#001080}apache{color}{color:#000000}/{color}{color:#001080}tinkerpop{color}{color:#000000}/{color}{color:#001080}gremlin{color}{color:#000000}-{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}v3{color}{color:#000000}@{color}{color:#001080}v3{color}{color:#000000}.{color}{color:#098658}7.1{color}{color:#000000}/{color}{color:#001080}driver{color}{color:#000000}/{color}{color:#001080}protocol{color}{color:#000000}.{color}{color:#af00db}go{color}{color:#000000}:{color}{color:#098658}197{color}{color:#000000}
+{color}{color:#098658}0x23a{color}
```
I can't reliably reproduce, it's never happened locally. It's occurred just
after doing some work in the logs above, but it's also happened after a period
of doing nothing.
First glance looks like it's a nil being returned from the `resultSets`
synchronizedMap, something getting in and removing it, which should only happen
when the `channelResultSet` itself is closed? Seems like a race condition to me.
I think the sensible way to handle this, if it is what I suspect (I've no proof
yet), would be to
* Store a reference to the ResultSet when first loading it in responseHandler
* Make `channelResultSet` discard modifications after close OR
* responseHandler needs to aquire the `channelMutex` somehow for the duration
of modifications (I think I prefer this, it's more obvious to what's happening)
I did try and get the project tests to run, but it's been _years_ since I've
had to use maven and I can't seem to find a good idiots guide to working with
the project from a non java POV - I need to know how to build the docker images
the tests require. If someone could help me get started I'd be more than happy
to contribute a PR (although given the nature, a reliable test may be hard to
write).
As it stands, I'm probably going to modify my fork and mod replace, see if a
fix would actually work in my deployments.
> Panic in gremlin-go driver responseHandler
> ------------------------------------------
>
> Key: TINKERPOP-3087
> URL: https://issues.apache.org/jira/browse/TINKERPOP-3087
> Project: TinkerPop
> Issue Type: Bug
> Components: go
> Affects Versions: 3.7.1
> Environment: Ubuntu 20.04, x86_64, AWS EC2 image. Think it's 2c2g,
> can't recall off the top of my head.
> Compiled _without_ CGO.
> Reporter: David Bennington
> Priority: Minor
>
> Ocasionally, I get the following panic in the gremlin-go driver:
>
> {noformat}
> 10:22:39.895 <successfully added a vertex to janusgraph>
> 10:22:44.487 panic: runtime error: invalid memory address or nil pointer
> dereference
> 10:22:44.487 [signal SIGSEGV: segmentation violation code=0x1 addr=0x78
> pc=0x11fdd16]
> 10:22:44.487
> 10:22:44.487 goroutine 275 [running]:
> 10:22:44.487
> github.com/apache/tinkerpop/gremlin-go/v3/driver.(*gremlinServerWSProtocol).responseHandler(0xc0006be6c0,
> 0xc0001dfc60, {{0x9, 0xf6, 0x33, 0x7f, 0x19, 0x16, 0x4e, 0xc9, ...}, ...})
> 10:22:44.487
> /go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3@v3.7.1/driver/protocol.go:116
> +0x7d6
> 10:22:44.487
> github.com/apache/tinkerpop/gremlin-go/v3/driver.(*gremlinServerWSProtocol).readLoop(0xc0006be6c0,
> 0xc0001dfc60, 0xc0001dfca0)
> 10:22:44.487
> /go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3@v3.7.1/driver/protocol.go:82
> +0x272
> 10:22:44.487 created by
> github.com/apache/tinkerpop/gremlin-go/v3/driver.newGremlinServerWSProtocol
> in goroutine 272
> 10:22:44.487
> /go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3@v3.7.1/driver/protocol.go:197
> +0x23a{noformat}
> I can't reliably reproduce, it's never happened locally. It's occurred just
> after doing some work in the logs above, but it's also happened after a
> period of doing nothing.
>
> First glance looks like it's a nil being returned from the `resultSets`
> synchronizedMap, something getting in and removing it, which should only
> happen when the `channelResultSet` itself is closed? Seems like a race
> condition to me.
> I think the sensible way to handle this, if it is what I suspect (I've no
> proof yet), would be to
> * Store a reference to the ResultSet when first loading it in responseHandler
> * Make `channelResultSet` discard modifications after close OR
> * responseHandler needs to aquire the `channelMutex` somehow for the
> duration of modifications (I think I prefer this, it's more obvious to what's
> happening)
> I did try and get the project tests to run, but it's been _years_ since I've
> had to use maven and I can't seem to find a good idiots guide to working with
> the project from a non java POV - I need to know how to build the docker
> images the tests require. If someone could help me get started I'd be more
> than happy to contribute a PR (although given the nature, a reliable test may
> be hard to write).
> As it stands, I'm probably going to modify my fork and mod replace, see if a
> fix would actually work in my deployments.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
