[
https://issues.apache.org/jira/browse/CASSANDRA-16841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17414028#comment-17414028
]
Ruslan Fomkin commented on CASSANDRA-16841:
-------------------------------------------
[~adelapena] It is actually an old issue, which became visible with my patch.
Basically importing a class brings all the tests from the imported class to be
part of this file. Inheriting the imported class also brings the same file but
as part of the child class, which is marked as upgrade test. As result there
are always two sets of test methods in such case. Here is an experiment, which
confirms this:
{code}
pytest --cassandra-dir=../cassandra/ upgrade_tests/bootstrap_upgrade_test.py
--collect-only --execute-upgrade-tests
...
collected 57 items
<Module 'upgrade_tests/bootstrap_upgrade_test.py'>
<Class 'TestBootstrap'>
<Instance '()'>
<Function 'test_simple_bootstrap_with_ssl'>
<Function 'test_simple_bootstrap'>
<Function 'test_bootstrap_on_write_survey'>
<Function 'test_simple_bootstrap_small_keepalive_period'>
<Function 'test_simple_bootstrap_nodata'>
<Function 'test_schema_removed_nodes'>
<Function 'test_read_from_bootstrapped_node'>
<Function 'test_bootstrap_waits_for_streaming_to_finish'>
<Function
'test_consistent_range_movement_true_with_replica_down_should_fail'>
<Function
'test_consistent_range_movement_false_with_replica_down_should_succeed'>
<Function 'test_consistent_range_movement_true_with_rf1_should_fail'>
<Function 'test_consistent_range_movement_false_with_rf1_should_succeed'>
<Function 'test_rf_gt_nodes_multidc_should_succeed'>
<Function 'test_resumable_bootstrap'>
<Function 'test_bootstrap_with_reset_bootstrap_state'>
<Function 'test_manual_bootstrap'>
<Function 'test_local_quorum_bootstrap'>
<Function 'test_shutdown_wiped_node_cannot_join'>
<Function 'test_killed_wiped_node_cannot_join'>
<Function 'test_decommissioned_wiped_node_can_join'>
<Function 'test_decommissioned_wiped_node_can_gossip_to_single_seed'>
<Function 'test_failed_bootstrap_wiped_node_can_join'>
<Function
'test_node_cannot_join_as_hibernating_node_without_replace_address'>
<Function 'test_simultaneous_bootstrap'>
<Function 'test_cleanup'>
<Function 'test_bootstrap_binary_disabled'>
<Function 'test_invalid_host_id'>
<Function 'test_host_id_override'>
<Class 'TestBootstrapUpgrade'>
<Instance '()'>
<Function 'test_simple_bootstrap_with_ssl'>
<Function 'test_simple_bootstrap'>
<Function 'test_bootstrap_on_write_survey'>
<Function 'test_simple_bootstrap_small_keepalive_period'>
<Function 'test_simple_bootstrap_nodata'>
<Function 'test_schema_removed_nodes'>
<Function 'test_read_from_bootstrapped_node'>
<Function 'test_bootstrap_waits_for_streaming_to_finish'>
<Function
'test_consistent_range_movement_true_with_replica_down_should_fail'>
<Function
'test_consistent_range_movement_false_with_replica_down_should_succeed'>
<Function 'test_consistent_range_movement_true_with_rf1_should_fail'>
<Function 'test_consistent_range_movement_false_with_rf1_should_succeed'>
<Function 'test_rf_gt_nodes_multidc_should_succeed'>
<Function 'test_resumable_bootstrap'>
<Function 'test_bootstrap_with_reset_bootstrap_state'>
<Function 'test_manual_bootstrap'>
<Function 'test_local_quorum_bootstrap'>
<Function 'test_shutdown_wiped_node_cannot_join'>
<Function 'test_killed_wiped_node_cannot_join'>
<Function 'test_decommissioned_wiped_node_can_join'>
<Function 'test_decommissioned_wiped_node_can_gossip_to_single_seed'>
<Function 'test_failed_bootstrap_wiped_node_can_join'>
<Function
'test_node_cannot_join_as_hibernating_node_without_replace_address'>
<Function 'test_simultaneous_bootstrap'>
<Function 'test_cleanup'>
<Function 'test_bootstrap_binary_disabled'>
<Function 'test_invalid_host_id'>
<Function 'test_host_id_override'>
<Function 'test_simple_bootstrap_mixed_versions'>
{code}
To my understanding this side effect of importing with the double execution of
tests is not intentional. Before the change the same set of imported tests
were executed two time as upgrade tests and once as non-upgrade as part of the
original file. After this patch the upgrade tests are executed once, while the
non-upgrade tests are executed twice: one as part of the original file and one
as imported.
I currently have no good idea how to fix it and remove double execution.
> Unexpectedly ignored dtests
> ---------------------------
>
> Key: CASSANDRA-16841
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16841
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
> Reporter: Ruslan Fomkin
> Assignee: Ruslan Fomkin
> Priority: Normal
> Attachments: collected-dtests-diffs.txt
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> An issue, which I was hit:
> When one class in a dtest file is marked as resource intensive, then all
> tests in all classes are treated as resource intensive. For example,
> [repair_tests/repair_test.py|https://github.com/apache/cassandra-dtest/blob/trunk/repair_tests/repair_test.py]
> contains three classes and the last class is marked as resource intensive:
> {code:java}
> @pytest.mark.resource_intensive
> class TestRepairDataSystemTable(Tester):
> {code}
> So if I try to run an unmarked class:
> {code:java}
> pytest --cassandra-dir=../cassandra repair_tests/repair_test.py::TestRepair
> --collect-only --skip-resource-intensive-tests
> {code}
> then all tests are ignored
> {code:java}
> collected 36 items / 36 deselected
> {code}
> This is because a test is treated to be marked if any class in the same file
> has the mark. This bug was introduced in the fix of CASS-16399. Before only
> upgrade tests had such behaviour, i.e., if a class is marked as upgrade test,
> then all tests are upgrade test in the file.
>
> This bug, for example, means that if the same file contains one class marked
> with vnodes and another class with no_vnodes, then no tests will be executed
> in the file.
> I also noticed another issue that If a test run is executed with the argument
> {{-only-resource-intensive-tests}} and there is no sufficient resources for
> resource intensive tests, then no tests were executed. Thus it was necessary
> to provide {{-force-resource-intensive-tests}} in addition.
> Suggestions for the solutions:
> # Require to mark each class and remove the special case of upgrade tests.
> This will simplify the implementation and might be more obvious for new
> comers.
> # Treat {{-only-resource-intensive-tests}} in the same way as
> {{-force-resource-intensive-tests}}, so it will be enough to just specify it
> even with no sufficient resources.
> *Update:* comments were provided to keep only the first suggestion and do not
> implement the second suggestion.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
