dbtsai opened a new pull request, #124: URL: https://github.com/apache/spark-connect-rust/pull/124
### What changes were proposed in this pull request? Add `DataFrame.__dir__` and `_ipython_key_completions_` in the PyO3 binding, matching Python Connect. `dir(df)` returns sorted, deduplicated class attributes plus all column names. Key completion returns column names in schema order, including duplicates and non-identifiers. Both methods propagate schema errors and release the GIL during analysis. Class inspection avoids recursive instance `dir` calls. ### Why are the changes needed? For a DataFrame with column `c0`, master omits `c0` from `dir(df)` and raises `AttributeError` for `_ipython_key_completions_()`. This prevents interactive column discovery and bracket completion. [SPARK-60681](https://issues.apache.org/jira/browse/SPARK-60681) ### Does this PR introduce _any_ user-facing change? Yes. Interactive completion includes DataFrame columns while preserving existing methods. This is based directly on master and is independent of schema caching in #120: each completion call still performs schema analysis. ### How was this patch tested? - Added 13 separate-process mock gRPC tests covering ordinary/non-identifier/Unicode/empty/duplicate names, method collisions, fresh result lists, schema errors and successful retries. All pass against the locked release wheel. - Checked the same name and error behavior against frozen Python Connect; the new test fails on the unchanged Rust baseline as expected. - Rust: 392 client library tests and 95 core library tests pass; full client test compilation passes with default and `wasm-udf` features; formatting and diff checks pass. Workspace-wide compilation requires an unavailable `wasm32-unknown-unknown` target for examples, so that initial failure is retained locally. - Existing Python offline suite: 274 tests pass on candidate and baseline using `pytest --rootdir=python --import-mode=importlib`. - Unchanged frozen loopback benchmark, five fresh rotating paired samples per client/case across 18 cases: candidate and Python each pass 90/90; baseline passes all 40 guard samples and fails the 50 completion samples. All ten inspection cases are fixed; guards cover schema, columns, named-column access, select, withColumn and missing-column errors. No speedup is claimed for formerly failing cases. Measured completion loops issue one schema RPC per call (20 for the repeated fixture). No live Spark server or JVM was used. Original setup failures, raw measurements, commands, source/wheel hashes and dependency freezes are retained in campaign artifacts. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (GPT-6) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
