[jira] [Commented] (CASSANDRA-12195) dtest failure in upgrade_tests.upgrade_through_versions_test.TestUpgrade_current_2_2_x_To_indev_3_0_x.rolling_upgrade_with_internode_ssl_test
[
https://issues.apache.org/jira/browse/CASSANDRA-12195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15402690#comment-15402690
]
Russ Hatch commented on CASSANDRA-12195:
With this (or similar) test moved now to the dtest large job, I don't see any
recent failure. In any case, if it recurs I think we'll get better debug
output. I think this is safe to resolve for now and can be reopened later if it
happens again.
> dtest failure in
> upgrade_tests.upgrade_through_versions_test.TestUpgrade_current_2_2_x_To_indev_3_0_x.rolling_upgrade_with_internode_ssl_test
> -
>
> Key: CASSANDRA-12195
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12195
> Project: Cassandra
> Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
> Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log,
> node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/upgrade_tests-all/59/testReport/upgrade_tests.upgrade_through_versions_test/TestUpgrade_current_2_2_x_To_indev_3_0_x/rolling_upgrade_with_internode_ssl_test
> Failed on CassCI build upgrade_tests-all #59
> {code}
> Stacktrace
> File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
> File
> "/home/automaton/cassandra-dtest/upgrade_tests/upgrade_through_versions_test.py",
> line 295, in rolling_upgrade_with_internode_ssl_test
> self.upgrade_scenario(rolling=True, internode_ssl=True)
> File
> "/home/automaton/cassandra-dtest/upgrade_tests/upgrade_through_versions_test.py",
> line 352, in upgrade_scenario
> self._check_on_subprocs(self.subprocs)
> File
> "/home/automaton/cassandra-dtest/upgrade_tests/upgrade_through_versions_test.py",
> line 409, in _check_on_subprocs
> raise RuntimeError(message)
> "A subprocess has terminated early. Subprocess statuses: Process-13
> (is_alive: False), Process-14 (is_alive: True), Process-15 (is_alive: True),
> Process-16 (is_alive: True), attempting to terminate remaining subprocesses
> now.
> {code}
> node2_debug.log is too large to attach.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12195) dtest failure in upgrade_tests.upgrade_through_versions_test.TestUpgrade_current_2_2_x_To_indev_3_0_x.rolling_upgrade_with_internode_ssl_test
[
https://issues.apache.org/jira/browse/CASSANDRA-12195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382548#comment-15382548
]
Philip Thompson commented on CASSANDRA-12195:
-
[~rhatch], with no errors in the logs, do we have any idea why these
subprocesses may have aborted?
> dtest failure in
> upgrade_tests.upgrade_through_versions_test.TestUpgrade_current_2_2_x_To_indev_3_0_x.rolling_upgrade_with_internode_ssl_test
> -
>
> Key: CASSANDRA-12195
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12195
> Project: Cassandra
> Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
> Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log,
> node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/upgrade_tests-all/59/testReport/upgrade_tests.upgrade_through_versions_test/TestUpgrade_current_2_2_x_To_indev_3_0_x/rolling_upgrade_with_internode_ssl_test
> Failed on CassCI build upgrade_tests-all #59
> {code}
> Stacktrace
> File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
> File
> "/home/automaton/cassandra-dtest/upgrade_tests/upgrade_through_versions_test.py",
> line 295, in rolling_upgrade_with_internode_ssl_test
> self.upgrade_scenario(rolling=True, internode_ssl=True)
> File
> "/home/automaton/cassandra-dtest/upgrade_tests/upgrade_through_versions_test.py",
> line 352, in upgrade_scenario
> self._check_on_subprocs(self.subprocs)
> File
> "/home/automaton/cassandra-dtest/upgrade_tests/upgrade_through_versions_test.py",
> line 409, in _check_on_subprocs
> raise RuntimeError(message)
> "A subprocess has terminated early. Subprocess statuses: Process-13
> (is_alive: False), Process-14 (is_alive: True), Process-15 (is_alive: True),
> Process-16 (is_alive: True), attempting to terminate remaining subprocesses
> now.
> {code}
> node2_debug.log is too large to attach.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12195) dtest failure in upgrade_tests.upgrade_through_versions_test.TestUpgrade_current_2_2_x_To_indev_3_0_x.rolling_upgrade_with_internode_ssl_test
[
https://issues.apache.org/jira/browse/CASSANDRA-12195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382568#comment-15382568
]
Russ Hatch commented on CASSANDRA-12195:
Most commonly this is an assertion error killing the process (unhandled
exception), but that should in theory be visible on stderr/stdout.
My work to transition upgrade tests over to regular test jobs will include
giving the subprocesses descriptive names, so at the very least in the future
we'll know what process died.
> dtest failure in
> upgrade_tests.upgrade_through_versions_test.TestUpgrade_current_2_2_x_To_indev_3_0_x.rolling_upgrade_with_internode_ssl_test
> -
>
> Key: CASSANDRA-12195
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12195
> Project: Cassandra
> Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
> Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log,
> node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/upgrade_tests-all/59/testReport/upgrade_tests.upgrade_through_versions_test/TestUpgrade_current_2_2_x_To_indev_3_0_x/rolling_upgrade_with_internode_ssl_test
> Failed on CassCI build upgrade_tests-all #59
> {code}
> Stacktrace
> File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
> File
> "/home/automaton/cassandra-dtest/upgrade_tests/upgrade_through_versions_test.py",
> line 295, in rolling_upgrade_with_internode_ssl_test
> self.upgrade_scenario(rolling=True, internode_ssl=True)
> File
> "/home/automaton/cassandra-dtest/upgrade_tests/upgrade_through_versions_test.py",
> line 352, in upgrade_scenario
> self._check_on_subprocs(self.subprocs)
> File
> "/home/automaton/cassandra-dtest/upgrade_tests/upgrade_through_versions_test.py",
> line 409, in _check_on_subprocs
> raise RuntimeError(message)
> "A subprocess has terminated early. Subprocess statuses: Process-13
> (is_alive: False), Process-14 (is_alive: True), Process-15 (is_alive: True),
> Process-16 (is_alive: True), attempting to terminate remaining subprocesses
> now.
> {code}
> node2_debug.log is too large to attach.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
