[
https://issues.apache.org/jira/browse/CASSANDRA-16841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17411355#comment-17411355
]
Ekaterina Dimitrova edited comment on CASSANDRA-16841 at 9/7/21, 4:48 PM:
--------------------------------------------------------------------------
I am inclined to not change the behavior as this will lead to a lot of noise in
CI environments. I would say we need to have a warning why tests were skipped
but then I think there is already an option that can be run in order to get the
reason for skipping tests in pytest. Quick check showed me it is not used at
least in Circle CI environment.
I would suggest finding a way probably to add a warning why the tests didn't
work in a separate ticket. For the record, I just did a quick check whether the
pytest option _-rxXS_ will produce the warning but it didn't. It works nicely
with the mark for cassandra version. For example:
{code:java}
pytest --cassandra-dir=../cassandra
repair_tests/repair_test.py::TestRepair::test_parent_repair_session_cleanup
-rxXS
============================================================================================================
test session starts
=============================================================================================================
platform darwin -- Python 3.8.10, pytest-3.6.4, py-1.10.0, pluggy-0.7.1
rootdir: ../cassandra-dtest-d, inifile: pytest.ini
plugins: timeout-1.4.2, repeat-0.9.1, flaky-3.7.0
timeout: 900.0s
timeout method: signal
timeout func_only: False
collected 1 item
repair_tests/repair_test.py s
[100%]
==========================================================================================================
short test summary info
===========================================================================================================
SKIP [1] ../cassandra-dtest-d/conftest.py:461: 3.11.12 < 4.0
{code}
Note: I had to remove the --collect-only flag to see the skip reason.
was (Author: e.dimitrova):
I am inclined to not change the behavior as this will lead to a lot of noise in
CI environments. I would say we need to have a warning why tests were skipped
but then I think there is already an option that can be run in order to get the
reason for skipping tests in pytest. Quick check showed me it is not used at
least in Circle CI environment.
I would suggest finding a way probably to add a warning why the tests didn't
work in a separate ticket. For the record, I just did a quick check whether the
pytest option _-rxXS_ will produce the warning but it didn't. It works nicely
with the mark for cassandra version. For example:
{code:java}
pytest --cassandra-dir=../cassandra
repair_tests/repair_test.py::TestRepair::test_parent_repair_session_cleanup
-rxXS
============================================================================================================
test session starts
=============================================================================================================
platform darwin -- Python 3.8.10, pytest-3.6.4, py-1.10.0, pluggy-0.7.1
rootdir: ../cassandra-dtest-d, inifile: pytest.ini
plugins: timeout-1.4.2, repeat-0.9.1, flaky-3.7.0
timeout: 900.0s
timeout method: signal
timeout func_only: False
collected 1 item
repair_tests/repair_test.py s
[100%]
==========================================================================================================
short test summary info
===========================================================================================================
SKIP [1] ../cassandra-dtest-d/conftest.py:461: 3.11.12 < 4.0
{code}
Note: I had to remove the --collect-only flag to see the skip reason.
> Unexpectedly ignored dtests
> ---------------------------
>
> Key: CASSANDRA-16841
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16841
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
> Reporter: Ruslan Fomkin
> Assignee: Ruslan Fomkin
> Priority: Normal
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> An issue, which I was hit:
> When one class in a dtest file is marked as resource intensive, then all
> tests in all classes are treated as resource intensive. For example,
> [repair_tests/repair_test.py|https://github.com/apache/cassandra-dtest/blob/trunk/repair_tests/repair_test.py]
> contains three classes and the last class is marked as resource intensive:
> {code:java}
> @pytest.mark.resource_intensive
> class TestRepairDataSystemTable(Tester):
> {code}
> So if I try to run an unmarked class:
> {code:java}
> pytest --cassandra-dir=../cassandra repair_tests/repair_test.py::TestRepair
> --collect-only --skip-resource-intensive-tests
> {code}
> then all tests are ignored
> {code:java}
> collected 36 items / 36 deselected
> {code}
> This is because a test is treated to be marked if any class in the same file
> has the mark. This bug was introduced in the fix of CASS-16399. Before only
> upgrade tests had such behaviour, i.e., if a class is marked as upgrade test,
> then all tests are upgrade test in the file.
>
> This bug, for example, means that if the same file contains one class marked
> with vnodes and another class with no_vnodes, then no tests will be executed
> in the file.
> I also noticed another issue that If a test run is executed with the argument
> {{-only-resource-intensive-tests}} and there is no sufficient resources for
> resource intensive tests, then no tests were executed. Thus it was necessary
> to provide {{-force-resource-intensive-tests}} in addition.
> Suggestions for the solutions:
> # Require to mark each class and remove the special case of upgrade tests.
> This will simplify the implementation and might be more obvious for new
> comers.
> # Treat {{-only-resource-intensive-tests}} in the same way as
> {{-force-resource-intensive-tests}}, so it will be enough to just specify it
> even with no sufficient resources.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
