mumrah commented on code in PR #18170:
URL: https://github.com/apache/kafka/pull/18170#discussion_r1884719244
##########
.github/scripts/develocity_reports.py:
##########
@@ -729,35 +729,39 @@ def get_cleared_tests(self, project: str, results:
List[TestResult],
def print_summary(problematic_tests: Dict[str, Dict], flaky_regressions:
Dict[str, Dict]):
"""Print a summary of the most problematic tests at the top of the
report"""
- print("\nSummary of Most Problematic Tests")
- print("=" * 50)
+ print("\n## Summary of Most Problematic Tests")
# Combine and sort all test cases by failure rate
all_problem_cases = []
# Process problematic quarantined tests
- for class_name, details in problematic_tests.items():
+ if len(problematic_tests) > 0:
+ print(f"Found {len(problematic_tests)} tests that have been
quarantined for a while and are still flaky.")
+ for full_class_name, details in problematic_tests.items():
for test_case in details['test_cases']:
total_runs = test_case.outcome_distribution.total
+ method_name = test_case.name.split('.')[-1]
if total_runs > 0:
failure_rate = (test_case.outcome_distribution.failed +
test_case.outcome_distribution.flaky) /
total_runs
all_problem_cases.append({
- 'class': class_name,
- 'method': test_case.name.split('.')[-1],
+ 'class': full_class_name,
+ 'method': method_name,
Review Comment:
Looks like Gradle is not capturing the parameterized arguments in the test
case name. In Develocity, we just see `testSelectorPollReadSize(Args)[1]`:
https://ge.apache.org/scans/tests?search.rootProjectNames=kafka&search.tags=github%2Ctrunk&search.timeZoneId=America%2FNew_York&tests.container=org.apache.kafka.common.network.SslTransportLayerTest&tests.sortField=FLAKY&tests.test=testSelectorPollReadSize(Args)%5B1%5D
This might be something we can tweak in our Gradle/JUnit config.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
