grundprinzip opened a new pull request, #162:
URL: https://github.com/apache/spark-connect-go/pull/162
### What changes were proposed in this pull request?
This PR adds JSON serialization capability to the Row interface by
implementing a `ToJsonString()` method. The
implementation includes:
- Added `ToJsonString()` method to the Row interface that converts Row
data to a JSON string representation
- Implemented comprehensive `convertToJsonValue()` function that handles
all Spark data types including:
- Basic types (bool, string, integers, floats)
- Binary data (base64 encoded)
- Decimal types (decimal128/256 as string representations)
- Arrow timestamp and date types (RFC3339 and ISO date formats)
- Arrays and nested structures with recursive conversion
- Maps with string keys (validates non-string keys return errors)
- Custom types using reflection fallback for underlying basic types
- Added extensive test suite with 238 lines covering all supported data
types and error scenarios
### Why are the changes needed?
This feature enables users to easily serialize Row data to JSON format,
which is essential for:
- Data export and interoperability with other systems
- Debugging and logging Row contents in human-readable format
- Integration with REST APIs and web services that expect JSON
- Data analysis workflows that require JSON output
Currently, there's no built-in way to convert Row data to JSON, forcing
users to manually iterate through fields
and handle type conversions.
### Does this PR introduce _any_ user-facing change?
Yes. This PR adds a new public method `ToJsonString() (string, error)` to
the Row interface. Users can now call
this method on any Row instance to get a JSON string representation:
```go
row := // ... get a Row from DataFrame operations
jsonStr, err := row.ToJsonString()
if err != nil {
// handle conversion error
}
// jsonStr contains: {"field1": "value1", "field2": 42, ...}
```
### How was this patch tested?
Added comprehensive unit tests in spark/sql/types/row_json_test.go
covering:
- Basic data types: strings, integers, floats, booleans, nil values
- Binary data: byte arrays converted to base64 encoding
- Decimal types: decimal128 and decimal256 number representations
- Temporal types: Arrow timestamps, Date32, Date64, and Go time.Time
- Collection types: arrays with recursive element conversion
- Map types: both map[string]any and map[any]any with string key validation
- Nested structures: complex combinations of arrays and maps
- Error cases: invalid map keys and conversion failures
- JSON validity: all outputs are verified to be valid JSON through
round-trip parsing
Tests ensure both successful conversions produce expected JSON and error
cases properly return meaningful error
messages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
