shibe2655 opened a new pull request, #59372: URL: https://github.com/apache/spark/pull/59372
### What changes were proposed in this pull request? Add `test_command_iterator_skips_disallowed_accumulator` to `test_client.py`. The test uses `MockService` to return a `__python_accumulator__` observed metric with two values: 1. A metric that contains `collections.defaultdict`, which is not in the allowlist of `specialAccumulatorSer`. 2. A valid V2 profiler update. The test checks that: - One warning is logged, and the message includes `collections.defaultdict`. - Only the valid update is passed to `_profiler_collector._update`. ### Why are the changes needed? SPARK-59508 added an `except pickle.UnpicklingError` branch in `SparkConnectClient._execute_and_fetch_as_iterator`. This branch is not covered by any test. The new test checks that a rejected metric does not fail the query and does not stop the processing of the metrics after it. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Added `test_command_iterator_skips_disallowed_accumulator`. All 45 tests in `pyspark.sql.tests.connect.client.test_client` passed. Ruff check and format passed. I also confirmed that the new test fails when `continue` is changed to `break`, or when the `logger.warning` call is removed. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code 2.1.284 (Claude Opus 5.5), OpenAI Codex 0.159.2 (GPT-6.1 Sol) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
