avantgardnerio commented on PR #2204: URL: https://github.com/apache/datafusion-ballista/pull/2204#issuecomment-5115245522
@phillipleblanc I'd be curious to hear how this works with your Vortex shuffles. I cloned a few of your repos, but couldn't find them. I also considered implementing this functionality in the flight server (binary search on read), but it seems like both Corlogix and SpiceAI flush shuffles to s3 in case an executor fails? This PR solves that problem with a (round-trip-heavy) pure S3 range-request answer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
