dongjoon-hyun opened a new pull request, #604:
URL: https://github.com/apache/spark-connect-swift/pull/604
### What changes were proposed in this pull request?
This PR aims to fix `DataFrame.na.fill`, `DataFrame.na.replace`, and
`DataFrame.stat.sampleBy` to not crash on unsupported value types. The public
API stays non-throwing.
- Support `Int8` and `Int16` in `na.fill` and `na.replace`.
- Convert any other type via `ExpressionLiteral.init(_:)`, so `Decimal`,
`Date`, etc. are sent as literals.
- Types that cannot be converted are sent as a string literal of their
description instead of `as! String`.
### Why are the changes needed?
The `default` branches used `value as! String`, which traps and crashes the
client process:
```swift
df.na.fill(["a": Int8(1)]) // crash
df.stat.sampleBy("key", [Int16(1): 0.5]) // crash
```
### Does this PR introduce _any_ user-facing change?
Yes, it fixes a bug. These calls used to crash the process. Now they build a
plan, and values the server does not accept are reported by the Spark Connect
server. The API signatures do not change.
### How was this patch tested?
Pass the CIs with the newly added test cases in `DataFrameInternalTests`.
They only inspect the generated plan, so they do not need a running server.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Sonnet 5.5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
