SEZ9 commented on issue #9746: URL: https://github.com/apache/seatunnel/issues/9746#issuecomment-5755194211
Thanks @goutamadwant for picking this up and opening the Spark PR — preserving nested-array schemas and values, including empty arrays and null map elements, is exactly the kind of coverage this issue needs. @dybyte, no worries about the bandwidth — thanks for flagging it early so nobody was blocked. Since the original issue body is empty and the scope was never written down, a couple of questions so we can track this to completion: 1. **Scope of the Spark PR.** Does it also cover the MAP-nesting combinations this issue is meant to address, e.g. `ARRAY<MAP<...>>` and `MAP<STRING, ARRAY<ROW>>`, or is it limited to nested arrays for now? Either is fine, but please state it explicitly in the PR description so we know what remains. 2. **Flink portion.** This issue covers both the Flink and Spark translation layers. Are you planning a follow-up Flink PR, or should we leave that part open for someone else? 3. **Tests.** It would help reviewers if the PR includes round-trip tests (schema and value conversion, both directions) for the nested type combinations above, including the empty-array and null-element edge cases you mentioned. Once I know the answers to 1 and 2, I'll update the issue description with the concrete scope so we can close it out cleanly when both halves land. <!-- streview-comment:1213 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
