sdf-jkl commented on PR #22908: URL: https://github.com/apache/datafusion/pull/22908#issuecomment-5502047869
> When I used datafusion-variant s in a test program, I also found I wanted to mimic spark's : operator @Samyak2 was working on it here - https://github.com/datafusion-contrib/datafusion-variant/issues/26 > So maybe what we need is some high level treatment "here is how to integrate datafusion-variant" (library level) and then here is how a user would use it We talked about it here - #21301. Maybe another discussion/mailing list is needed. datafusion-comet could be the biggest downstream consumer. It has an EPIC for native Variant support - https://github.com/apache/datafusion-comet/issues/5438. Since Spark-compatible Variant semantics were also an original goal of datafusion-variant, there appears to be substantial overlap. Coordinating now could help both efforts converge on shared semantics, kernels, and representation boundaries rather than implementing them independently. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
