dongjoon-hyun opened a new pull request, #606:
URL: https://github.com/apache/spark-connect-swift/pull/606
### What changes were proposed in this pull request?
This PR fixes `DataFrame.collect()` to return `struct` columns as nested
`Row`s (like Scala) instead of a `String`.
- `collect()` converts each `struct` field recursively with the same logic
as top-level columns.
- `Row.toString()` and `==` support nested `Row`s.
- `BinaryArray.asString` falls back to hex for non-UTF-8 bytes instead of
trapping.
### Why are the changes needed?
`collect()` rendered a `struct` via `BinaryArray.asString`, which
force-unwrapped `String(data:encoding: .utf8)`. This crashes on non-UTF-8
bytes, e.g. `VARIANT` and `GEOMETRY`/`GEOGRAPHY` columns. Scala keeps these as
bytes inside a `Row`.
### Does this PR introduce _any_ user-facing change?
Yes. A `struct` column from `collect()` is now a `Row` instead of a `String`
(e.g. `"{1,x}"` becomes a `Row` with `a = 1` and `b = "x"`). `VARIANT` and
`GEOMETRY`/`GEOGRAPHY` columns no longer crash.
### How was this patch tested?
Added `ArrowArrayTests`, `RowTests.nestedRow`,
`DataFrameTests.collectStruct`, and `DataFrameTests.collectVariant`. Verified
against an `apache/spark:4.2.0` Connect server.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Sonnet 5.5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
