[ https://issues.apache.org/jira/browse/TINKERPOP-3087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Bennington updated TINKERPOP-3087: ---------------------------------------- Description: Ocasionally, I get the following panic in the gremlin-go driver: {noformat} 10:22:39.895 <successfully added a vertex to janusgraph> 10:22:44.487 panic: runtime error: invalid memory address or nil pointer dereference 10:22:44.487 [signal SIGSEGV: segmentation violation code=0x1 addr=0x78 pc=0x11fdd16] 10:22:44.487 10:22:44.487 goroutine 275 [running]: 10:22:44.487 github.com/apache/tinkerpop/gremlin-go/v3/driver.(*gremlinServerWSProtocol).responseHandler(0xc0006be6c0, 0xc0001dfc60, {{0x9, 0xf6, 0x33, 0x7f, 0x19, 0x16, 0x4e, 0xc9, ...}, ...}) 10:22:44.487 /go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3@v3.7.1/driver/protocol.go:116 +0x7d6 10:22:44.487 github.com/apache/tinkerpop/gremlin-go/v3/driver.(*gremlinServerWSProtocol).readLoop(0xc0006be6c0, 0xc0001dfc60, 0xc0001dfca0) 10:22:44.487 /go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3@v3.7.1/driver/protocol.go:82 +0x272 10:22:44.487 created by github.com/apache/tinkerpop/gremlin-go/v3/driver.newGremlinServerWSProtocol in goroutine 272 10:22:44.487 /go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3@v3.7.1/driver/protocol.go:197 +0x23a{noformat} I can't reliably reproduce, it's never happened locally. It's occurred just after doing some work in the logs above, but it's also happened after a period of doing nothing. First glance looks like it's a nil being returned from the `resultSets` synchronizedMap, something getting in and removing it, which should only happen when the `channelResultSet` itself is closed? Seems like a race condition to me. I think the sensible way to handle this, if it is what I suspect (I've no proof yet), would be to * Store a reference to the ResultSet when first loading it in responseHandler * Make `channelResultSet` discard modifications after close OR * responseHandler needs to aquire the `channelMutex` somehow for the duration of modifications (I think I prefer this, it's more obvious to what's happening) I did try and get the project tests to run, but it's been _years_ since I've had to use maven and I can't seem to find a good idiots guide to working with the project from a non java POV - I need to know how to build the docker images the tests require. If someone could help me get started I'd be more than happy to contribute a PR (although given the nature, a reliable test may be hard to write). As it stands, I'm probably going to modify my fork and mod replace, see if a fix would actually work in my deployments. was: Ocasionally, I get the following panic in the gremlin-go driver: ``` {color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}39.895{color}{color:#000000} <{color}{color:#001080}successfully{color}{color:#000000} {color}{color:#001080}added{color}{color:#000000} {color}{color:#001080}a{color}{color:#000000} {color}{color:#001080}vertex{color}{color:#000000} {color}{color:#001080}to{color}{color:#000000} {color}{color:#001080}janusgraph{color}{color:#000000}>{color} {color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000} {color}{color:#001080}panic{color}{color:#000000}: {color}{color:#001080}runtime{color}{color:#000000} {color}{color:#267f99}error{color}{color:#000000}: {color}{color:#001080}invalid{color}{color:#000000} {color}{color:#001080}memory{color}{color:#000000} {color}{color:#001080}address{color}{color:#000000} {color}{color:#001080}or{color}{color:#000000} {color}{color:#0000ff}nil{color}{color:#000000} {color}{color:#001080}pointer{color}{color:#000000} {color}{color:#001080}dereference{color} {color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000} [{color}{color:#001080}signal{color}{color:#000000} {color}{color:#001080}SIGSEGV{color}{color:#000000}: {color}{color:#001080}segmentation{color}{color:#000000} {color}{color:#001080}violation{color}{color:#000000} {color}{color:#001080}code{color}{color:#000000}={color}{color:#098658}0x1{color}{color:#000000} {color}{color:#001080}addr{color}{color:#000000}={color}{color:#098658}0x78{color}{color:#000000} {color}{color:#001080}pc{color}{color:#000000}={color}{color:#098658}0x11fdd16{color}{color:#000000}]{color} {color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000} {color} {color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000} {color}{color:#001080}goroutine{color}{color:#000000} {color}{color:#098658}275{color}{color:#000000} [{color}{color:#001080}running{color}{color:#000000}]:{color} {color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000} {color}{color:#001080}github{color}{color:#000000}.{color}{color:#001080}com{color}{color:#000000}/{color}{color:#001080}apache{color}{color:#000000}/{color}{color:#001080}tinkerpop{color}{color:#000000}/{color}{color:#001080}gremlin{color}{color:#000000}-{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}v3{color}{color:#000000}/{color}{color:#001080}driver{color}{color:#000000}.(*{color}{color:#267f99}gremlinServerWSProtocol{color}{color:#000000}).{color}{color:#795e26}responseHandler{color}{color:#000000}({color}{color:#098658}0xc0006be6c0{color}{color:#000000}, {color}{color:#098658}0xc0001dfc60{color}{color:#000000}, {{{color}{color:#098658}0x9{color}{color:#000000}, {color}{color:#098658}0xf6{color}{color:#000000}, {color}{color:#098658}0x33{color}{color:#000000}, {color}{color:#098658}0x7f{color}{color:#000000}, {color}{color:#098658}0x19{color}{color:#000000}, {color}{color:#098658}0x16{color}{color:#000000}, {color}{color:#098658}0x4e{color}{color:#000000}, {color}{color:#098658}0xc9{color}{color:#000000}, ...}, ...}){color} {color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000} /{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}pkg{color}{color:#000000}/{color}{color:#001080}mod{color}{color:#000000}/{color}{color:#001080}github{color}{color:#000000}.{color}{color:#001080}com{color}{color:#000000}/{color}{color:#001080}apache{color}{color:#000000}/{color}{color:#001080}tinkerpop{color}{color:#000000}/{color}{color:#001080}gremlin{color}{color:#000000}-{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}v3{color}{color:#000000}@{color}{color:#001080}v3{color}{color:#000000}.{color}{color:#098658}7.1{color}{color:#000000}/{color}{color:#001080}driver{color}{color:#000000}/{color}{color:#001080}protocol{color}{color:#000000}.{color}{color:#af00db}go{color}{color:#000000}:{color}{color:#098658}116{color}{color:#000000} +{color}{color:#098658}0x7d6{color} {color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000} {color}{color:#001080}github{color}{color:#000000}.{color}{color:#001080}com{color}{color:#000000}/{color}{color:#001080}apache{color}{color:#000000}/{color}{color:#001080}tinkerpop{color}{color:#000000}/{color}{color:#001080}gremlin{color}{color:#000000}-{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}v3{color}{color:#000000}/{color}{color:#001080}driver{color}{color:#000000}.(*{color}{color:#267f99}gremlinServerWSProtocol{color}{color:#000000}).{color}{color:#795e26}readLoop{color}{color:#000000}({color}{color:#098658}0xc0006be6c0{color}{color:#000000}, {color}{color:#098658}0xc0001dfc60{color}{color:#000000}, {color}{color:#098658}0xc0001dfca0{color}{color:#000000}){color} {color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000} /{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}pkg{color}{color:#000000}/{color}{color:#001080}mod{color}{color:#000000}/{color}{color:#001080}github{color}{color:#000000}.{color}{color:#001080}com{color}{color:#000000}/{color}{color:#001080}apache{color}{color:#000000}/{color}{color:#001080}tinkerpop{color}{color:#000000}/{color}{color:#001080}gremlin{color}{color:#000000}-{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}v3{color}{color:#000000}@{color}{color:#001080}v3{color}{color:#000000}.{color}{color:#098658}7.1{color}{color:#000000}/{color}{color:#001080}driver{color}{color:#000000}/{color}{color:#001080}protocol{color}{color:#000000}.{color}{color:#af00db}go{color}{color:#000000}:{color}{color:#098658}82{color}{color:#000000} +{color}{color:#098658}0x272{color} {color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000} {color}{color:#001080}created{color}{color:#000000} {color}{color:#001080}by{color}{color:#000000} {color}{color:#001080}github{color}{color:#000000}.{color}{color:#001080}com{color}{color:#000000}/{color}{color:#001080}apache{color}{color:#000000}/{color}{color:#001080}tinkerpop{color}{color:#000000}/{color}{color:#001080}gremlin{color}{color:#000000}-{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}v3{color}{color:#000000}/{color}{color:#001080}driver{color}{color:#000000}.{color}{color:#001080}newGremlinServerWSProtocol{color}{color:#000000} {color}{color:#001080}in{color}{color:#000000} {color}{color:#001080}goroutine{color}{color:#000000} {color}{color:#098658}272{color} {color:#098658}10{color}{color:#000000}:{color}{color:#098658}22{color}{color:#000000}:{color}{color:#098658}44.487{color}{color:#000000} /{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}pkg{color}{color:#000000}/{color}{color:#001080}mod{color}{color:#000000}/{color}{color:#001080}github{color}{color:#000000}.{color}{color:#001080}com{color}{color:#000000}/{color}{color:#001080}apache{color}{color:#000000}/{color}{color:#001080}tinkerpop{color}{color:#000000}/{color}{color:#001080}gremlin{color}{color:#000000}-{color}{color:#af00db}go{color}{color:#000000}/{color}{color:#001080}v3{color}{color:#000000}@{color}{color:#001080}v3{color}{color:#000000}.{color}{color:#098658}7.1{color}{color:#000000}/{color}{color:#001080}driver{color}{color:#000000}/{color}{color:#001080}protocol{color}{color:#000000}.{color}{color:#af00db}go{color}{color:#000000}:{color}{color:#098658}197{color}{color:#000000} +{color}{color:#098658}0x23a{color} ``` I can't reliably reproduce, it's never happened locally. It's occurred just after doing some work in the logs above, but it's also happened after a period of doing nothing. First glance looks like it's a nil being returned from the `resultSets` synchronizedMap, something getting in and removing it, which should only happen when the `channelResultSet` itself is closed? Seems like a race condition to me. I think the sensible way to handle this, if it is what I suspect (I've no proof yet), would be to * Store a reference to the ResultSet when first loading it in responseHandler * Make `channelResultSet` discard modifications after close OR * responseHandler needs to aquire the `channelMutex` somehow for the duration of modifications (I think I prefer this, it's more obvious to what's happening) I did try and get the project tests to run, but it's been _years_ since I've had to use maven and I can't seem to find a good idiots guide to working with the project from a non java POV - I need to know how to build the docker images the tests require. If someone could help me get started I'd be more than happy to contribute a PR (although given the nature, a reliable test may be hard to write). As it stands, I'm probably going to modify my fork and mod replace, see if a fix would actually work in my deployments. > Panic in gremlin-go driver responseHandler > ------------------------------------------ > > Key: TINKERPOP-3087 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3087 > Project: TinkerPop > Issue Type: Bug > Components: go > Affects Versions: 3.7.1 > Environment: Ubuntu 20.04, x86_64, AWS EC2 image. Think it's 2c2g, > can't recall off the top of my head. > Compiled _without_ CGO. > Reporter: David Bennington > Priority: Minor > > Ocasionally, I get the following panic in the gremlin-go driver: > > {noformat} > 10:22:39.895 <successfully added a vertex to janusgraph> > 10:22:44.487 panic: runtime error: invalid memory address or nil pointer > dereference > 10:22:44.487 [signal SIGSEGV: segmentation violation code=0x1 addr=0x78 > pc=0x11fdd16] > 10:22:44.487 > 10:22:44.487 goroutine 275 [running]: > 10:22:44.487 > github.com/apache/tinkerpop/gremlin-go/v3/driver.(*gremlinServerWSProtocol).responseHandler(0xc0006be6c0, > 0xc0001dfc60, {{0x9, 0xf6, 0x33, 0x7f, 0x19, 0x16, 0x4e, 0xc9, ...}, ...}) > 10:22:44.487 > /go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3@v3.7.1/driver/protocol.go:116 > +0x7d6 > 10:22:44.487 > github.com/apache/tinkerpop/gremlin-go/v3/driver.(*gremlinServerWSProtocol).readLoop(0xc0006be6c0, > 0xc0001dfc60, 0xc0001dfca0) > 10:22:44.487 > /go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3@v3.7.1/driver/protocol.go:82 > +0x272 > 10:22:44.487 created by > github.com/apache/tinkerpop/gremlin-go/v3/driver.newGremlinServerWSProtocol > in goroutine 272 > 10:22:44.487 > /go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3@v3.7.1/driver/protocol.go:197 > +0x23a{noformat} > I can't reliably reproduce, it's never happened locally. It's occurred just > after doing some work in the logs above, but it's also happened after a > period of doing nothing. > > First glance looks like it's a nil being returned from the `resultSets` > synchronizedMap, something getting in and removing it, which should only > happen when the `channelResultSet` itself is closed? Seems like a race > condition to me. > I think the sensible way to handle this, if it is what I suspect (I've no > proof yet), would be to > * Store a reference to the ResultSet when first loading it in responseHandler > * Make `channelResultSet` discard modifications after close OR > * responseHandler needs to aquire the `channelMutex` somehow for the > duration of modifications (I think I prefer this, it's more obvious to what's > happening) > I did try and get the project tests to run, but it's been _years_ since I've > had to use maven and I can't seem to find a good idiots guide to working with > the project from a non java POV - I need to know how to build the docker > images the tests require. If someone could help me get started I'd be more > than happy to contribute a PR (although given the nature, a reliable test may > be hard to write). > As it stands, I'm probably going to modify my fork and mod replace, see if a > fix would actually work in my deployments. -- This message was sent by Atlassian Jira (v8.20.10#820010)