brijrajk opened a new pull request, #59142: URL: https://github.com/apache/spark/pull/59142
### What changes were proposed in this pull request? `_read_descriptor_set_file()` in `pyspark.sql.protobuf.functions` previously used a bare `open()` call with a `TODO` comment to raise a proper exception. This PR resolves the TODO: - `FileNotFoundError` now raises `PySparkRuntimeError` with error class `PROTOBUF_DESCRIPTOR_FILE_NOT_FOUND` (existing condition, also added by PR #56174). - All other `OSError` subclasses (`PermissionError`, `IsADirectoryError`, etc.) now raise `PySparkRuntimeError` with a new error class `PROTOBUF_DESCRIPTOR_FILE_READ_FAILED`. The Connect module's `_read_descriptor_set_file` previously duplicated the bare `open()` call. It now delegates to the classic module's implementation, so both error mappings are inherited automatically with no duplication. ### Why are the changes needed? Bare `OSError` exceptions expose Python internals and OS-specific wording directly to users. Structured `PySparkRuntimeError` instances carry a stable error class (`SQLSTATE 42K0G`) and named message parameters, which are easier to catch programmatically and display consistently across classic and Connect modes. The TODO comment in the original code explicitly called for this improvement. ### Does this PR introduce _any_ user-facing change? Yes. Previously, passing a non-existent or unreadable descriptor file path caused a raw `FileNotFoundError` or other `OSError` to propagate to the caller. After this change, a `PySparkRuntimeError` is raised instead with a structured error class and message: - **Before:** `FileNotFoundError: [Errno 2] No such file or directory: '/path/to/file.desc'` - **After:** `PySparkRuntimeError: [PROTOBUF_DESCRIPTOR_FILE_NOT_FOUND] Error reading Protobuf descriptor file at path: /path/to/file.desc.` The same applies for `PermissionError` and other `OSError` subclasses, which now raise `PROTOBUF_DESCRIPTOR_FILE_READ_FAILED` instead. ### How was this patch tested? New test suite `python/pyspark/sql/tests/test_protobuf.py` covers: - `test_read_descriptor_set_file_not_found` — verifies `PROTOBUF_DESCRIPTOR_FILE_NOT_FOUND` for a non-existent path on the classic module. - `test_read_descriptor_set_file_read_error` — verifies `PROTOBUF_DESCRIPTOR_FILE_READ_FAILED` when a directory path is passed (raises `IsADirectoryError`, an `OSError` subclass, regardless of privilege level). - `test_descriptor_file_not_found_connect` — same not-found check via `connect.from_protobuf` / `connect.to_protobuf` (skipped when Connect deps absent). - `test_descriptor_file_read_error_connect` — same read-error check via Connect functions. Run locally: ``` python/run-tests --testnames pyspark.sql.tests.test_protobuf ``` All 4 tests pass (Connect tests skipped when dependencies not installed). ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Sonnet 4.6 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
