dongjoon-hyun opened a new pull request, #526: URL: https://github.com/apache/spark-connect-swift/pull/526
### What changes were proposed in this pull request? This PR aims to support 16 `VARIANT` functions (24 overloads). | Function | Since | Signature | | --- | --- | --- | | `parse_json` | 4.0.0 | `(Column)` | | `try_parse_json` | 4.0.0 | `(Column)` | | `to_variant_object` | 4.0.0 | `(Column)` | | `is_variant_null` | 4.0.0 | `(Column)` | | `variant_get` | 4.0.0 | `(Column, String \| Column, String)` | | `try_variant_get` | 4.0.0 | `(Column, String \| Column, String)` | | `schema_of_variant` | 4.0.0 | `(Column)` | | `schema_of_variant_agg` | 4.0.0 | `(Column)` | | `is_valid_variant` | 4.2.0 | `(Column)` | | `variant_insert` | 4.3.0 | `(Column, String \| Column, Column)` | | `try_variant_insert` | 4.3.0 | `(Column, String \| Column, Column)` | | `variant_set` | 4.3.0 | `(Column, String \| Column, Column, Bool = true)` | | `try_variant_set` | 4.3.0 | `(Column, String \| Column, Column, Bool = true)` | | `variant_array_append` | 4.3.0 | `(Column, String \| Column, Column)` | | `try_variant_array_append` | 4.3.0 | `(Column, String \| Column, Column)` | | `variant_strip_nulls` | 4.3.0 | `(Column, Bool = true)` | The functions live in a new `VariantFunctions.swift`, except `schema_of_variant_agg` which is an aggregate function and is placed in `AggregateFunctions.swift`. Where the upstream `path` argument is `Union[Column, str]`, both a `String` and a `Column` overload are provided, following the existing `schema_of_json` precedent. Like PySpark's Spark Connect client, the `createIfMissing` and `includeArrays` flags are always sent as literal arguments. `variant_delete` is out of scope because it is introduced in Spark 5.0.0. ### Why are the changes needed? To improve API coverage. `DataType` already models `.variant`, but no `VARIANT` function was available in this client. ### Does this PR introduce _any_ user-facing change? No, this is a new addition to the API. ### How was this patch tested? Pass the CIs with the newly added test suite, `VariantFunctionsTests`. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
