unikdahal commented on issue #6523: URL: https://github.com/apache/datafusion-comet/issues/6523#issuecomment-5955012040
@andygrove from what I understand, there isn’t currently a way to run the native Comet -> Celeborn shuffle with a stock released OSS Celeborn client. With current 0.6.x / 0.7.x releases, Comet can still work with Celeborn through the existing Spark/Celeborn shuffle path, but the new native writer/reader path is rejected by the completion-tracking compatibility checks. I think the part that would help most is having a clear guide or compatibility contract describing what a Celeborn client needs to provide for the native path to be considered supported and correct end-to-end. @pingzh @sunchao, is that a fair understanding? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
