phillipleblanc commented on PR #2204: URL: https://github.com/apache/datafusion-ballista/pull/2204#issuecomment-5186676665
I think having the per-format implementations of `RangeShuffleWriter` and `RangeShuffleReader` that own both the data + index makes sense to me. Its unlikely we'll want to add a bunch of formats to ballista, and this would open the door for us to have our Vortex shuffle implementation out of our fork and fully contained in our own project code. As long as the traits `RangeShuffleWriter` and `RangeShuffleReader` are public and there is some way for us to register our implementation of it then this makes sense to me! Also RE: the breaking change - I checked and this won't be a problem for us. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
