dongjoon-hyun opened a new pull request, #585:
URL: https://github.com/apache/spark-connect-swift/pull/585
### What changes were proposed in this pull request?
This PR aims to fix `DataFrame.hint` so that it no longer crashes on
out-of-range integers or unsupported parameter types. The public API stays
non-throwing.
`SparkConnectClient.getHint` now converts hint parameters as follows:
- `Int64` and `Int` values within the `Int32` range are still sent as
integer literals, because Spark's `ResolveHints` expects an `IntegerLiteral`
for partition numbers. Values outside that range are sent as long literals, so
the Spark Connect server reports the error. This matches how PySpark infers
integer literals.
- Any other type goes through the existing `ExpressionLiteral.init(_:)`
conversion, so `Double`, `Float`, `Decimal` and so on are now supported.
- Types that cannot be converted are sent as a string literal of their
description instead of being force-cast with `as! String`.
### Why are the changes needed?
`getHint` used `Int32(value)` and `$0 as! String`, which trap and crash the
whole client process. For example:
```swift
df.hint("rebalance", 3_000_000_000) // crash: Int32 overflow
df.hint("x", 1.5) // crash: Could not cast value of type
'Double' to 'String'
```
### Does this PR introduce _any_ user-facing change?
Yes, it fixes a bug. These calls used to crash the process. Now they build a
valid plan, and invalid hints are reported by the Spark Connect server instead.
The API signature does not change.
### How was this patch tested?
Pass the CIs with the newly added test cases
(`SparkConnectClientTests.getHintWithIntegers` and
`SparkConnectClientTests.getHintWithOtherTypes`).
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5.5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
