[jira] [Updated] (CASSANDRA-10175) cassandra-stress should be tolerant when a remote node shutdown
[ https://issues.apache.org/jira/browse/CASSANDRA-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-10175: -- Resolution: Fixed Status: Resolved (was: Open) This seems to not happen anymore after CASSANDRA-12585 was introduced. I tried to turn off a node and JMX metrics just stopped to be collected. After I started that node, it just continued to report them. There was try-catch added here (1) which catches errors which happen here (2). (1) https://github.com/apache/cassandra/blob/trunk/tools/stress/src/org/apache/cassandra/stress/report/StressMetrics.java#L237-L244 (2) https://github.com/apache/cassandra/blob/trunk/tools/stress/src/org/apache/cassandra/stress/util/JmxCollector.java#L112 > cassandra-stress should be tolerant when a remote node shutdown > > > Key: CASSANDRA-10175 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10175 > Project: Cassandra > Issue Type: Improvement > Components: Tool/stress >Reporter: Alan Boudreault >Assignee: Stefan Miklosovic >Priority: Normal > Labels: stress > Fix For: 5.x > > > Currently, if we start a stress session with 3 nodes and shutdown one node, > stress will crash. It is caused by the JMX connection lost on the node, which > is use to collect some gc stats IIRC. > backtrace: https://gist.github.com/aboudreault/6cd82bb0acc681992414 > Stress should handle that jmx connection lost in a better way so the session > could continue. Ideally, it should try to *reconnect* to JMX if the node is > back online? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-10175) cassandra-stress should be tolerant when a remote node shutdown
[ https://issues.apache.org/jira/browse/CASSANDRA-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-10175: -- Fix Version/s: (was: 5.x) > cassandra-stress should be tolerant when a remote node shutdown > > > Key: CASSANDRA-10175 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10175 > Project: Cassandra > Issue Type: Improvement > Components: Tool/stress >Reporter: Alan Boudreault >Assignee: Stefan Miklosovic >Priority: Normal > Labels: stress > > Currently, if we start a stress session with 3 nodes and shutdown one node, > stress will crash. It is caused by the JMX connection lost on the node, which > is use to collect some gc stats IIRC. > backtrace: https://gist.github.com/aboudreault/6cd82bb0acc681992414 > Stress should handle that jmx connection lost in a better way so the session > could continue. Ideally, it should try to *reconnect* to JMX if the node is > back online? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-10175) cassandra-stress should be tolerant when a remote node shutdown
[ https://issues.apache.org/jira/browse/CASSANDRA-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-10175: - Component/s: Stress > cassandra-stress should be tolerant when a remote node shutdown > > > Key: CASSANDRA-10175 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10175 > Project: Cassandra > Issue Type: Improvement > Components: Stress >Reporter: Alan Boudreault >Priority: Major > Labels: stress > Fix For: 4.x > > > Currently, if we start a stress session with 3 nodes and shutdown one node, > stress will crash. It is caused by the JMX connection lost on the node, which > is use to collect some gc stats IIRC. > backtrace: https://gist.github.com/aboudreault/6cd82bb0acc681992414 > Stress should handle that jmx connection lost in a better way so the session > could continue. Ideally, it should try to *reconnect* to JMX if the node is > back online? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-10175) cassandra-stress should be tolerant when a remote node shutdown
[ https://issues.apache.org/jira/browse/CASSANDRA-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Hanna updated CASSANDRA-10175: - Labels: stress (was: ) > cassandra-stress should be tolerant when a remote node shutdown > > > Key: CASSANDRA-10175 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10175 > Project: Cassandra > Issue Type: Improvement >Reporter: Alan Boudreault > Labels: stress > Fix For: 3.x > > > Currently, if we start a stress session with 3 nodes and shutdown one node, > stress will crash. It is caused by the JMX connection lost on the node, which > is use to collect some gc stats IIRC. > backtrace: https://gist.github.com/aboudreault/6cd82bb0acc681992414 > Stress should handle that jmx connection lost in a better way so the session > could continue. Ideally, it should try to *reconnect* to JMX if the node is > back online? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10175) cassandra-stress should be tolerant when a remote node shutdown
[ https://issues.apache.org/jira/browse/CASSANDRA-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-10175: -- Issue Type: Improvement (was: Bug) > cassandra-stress should be tolerant when a remote node shutdown > > > Key: CASSANDRA-10175 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10175 > Project: Cassandra > Issue Type: Improvement >Reporter: Alan Boudreault > Fix For: 3.x > > > Currently, if we start a stress session with 3 nodes and shutdown one node, > stress will crash. It is caused by the JMX connection lost on the node, which > is use to collect some gc stats IIRC. > backtrace: https://gist.github.com/aboudreault/6cd82bb0acc681992414 > Stress should handle that jmx connection lost in a better way so the session > could continue. Ideally, it should try to *reconnect* to JMX if the node is > back online? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10175) cassandra-stress should be tolerant when a remote node shutdown
[ https://issues.apache.org/jira/browse/CASSANDRA-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-10175: - Assignee: (was: Benedict) > cassandra-stress should be tolerant when a remote node shutdown > > > Key: CASSANDRA-10175 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10175 > Project: Cassandra > Issue Type: Bug >Reporter: Alan Boudreault > Fix For: 3.x > > > Currently, if we start a stress session with 3 nodes and shutdown one node, > stress will crash. It is caused by the JMX connection lost on the node, which > is use to collect some gc stats IIRC. > backtrace: https://gist.github.com/aboudreault/6cd82bb0acc681992414 > Stress should handle that jmx connection lost in a better way so the session > could continue. Ideally, it should try to *reconnect* to JMX if the node is > back online? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10175) cassandra-stress should be tolerant when a remote node shutdown
[ https://issues.apache.org/jira/browse/CASSANDRA-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-10175: --- Priority: Major (was: Minor) cassandra-stress should be tolerant when a remote node shutdown Key: CASSANDRA-10175 URL: https://issues.apache.org/jira/browse/CASSANDRA-10175 Project: Cassandra Issue Type: Improvement Reporter: Alan Boudreault Fix For: 3.x Currently, if we start a stress session with 3 nodes and shutdown one node, stress will crash. It is caused by the JMX connection lost on the node, which is use to collect some gc stats IIRC. backtrace: https://gist.github.com/aboudreault/6cd82bb0acc681992414 Stress should handle that jmx connection lost in a better way so the session could continue. Ideally, it should try to *reconnect* to JMX if the node is back online? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10175) cassandra-stress should be tolerant when a remote node shutdown
[ https://issues.apache.org/jira/browse/CASSANDRA-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-10175: --- Assignee: Benedict cassandra-stress should be tolerant when a remote node shutdown Key: CASSANDRA-10175 URL: https://issues.apache.org/jira/browse/CASSANDRA-10175 Project: Cassandra Issue Type: Bug Reporter: Alan Boudreault Assignee: Benedict Fix For: 3.x Currently, if we start a stress session with 3 nodes and shutdown one node, stress will crash. It is caused by the JMX connection lost on the node, which is use to collect some gc stats IIRC. backtrace: https://gist.github.com/aboudreault/6cd82bb0acc681992414 Stress should handle that jmx connection lost in a better way so the session could continue. Ideally, it should try to *reconnect* to JMX if the node is back online? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10175) cassandra-stress should be tolerant when a remote node shutdown
[ https://issues.apache.org/jira/browse/CASSANDRA-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-10175: --- Issue Type: Bug (was: Improvement) cassandra-stress should be tolerant when a remote node shutdown Key: CASSANDRA-10175 URL: https://issues.apache.org/jira/browse/CASSANDRA-10175 Project: Cassandra Issue Type: Bug Reporter: Alan Boudreault Fix For: 3.x Currently, if we start a stress session with 3 nodes and shutdown one node, stress will crash. It is caused by the JMX connection lost on the node, which is use to collect some gc stats IIRC. backtrace: https://gist.github.com/aboudreault/6cd82bb0acc681992414 Stress should handle that jmx connection lost in a better way so the session could continue. Ideally, it should try to *reconnect* to JMX if the node is back online? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
