unikdahal commented on issue #6523: URL: https://github.com/apache/datafusion-comet/issues/6523#issuecomment-5947924905
Thanks @pingzh, maintaining a patched Celeborn client on our side is completely fine for us. The part I’m still trying to understand is what exactly a compatible client needs to provide. From Apache main I can see the structural checks around the completion-tracking fields, but it’s not clear to me whether satisfying those checks alone is enough, or whether there are additional requirements around retries, cancellation, callback ownership, transport buffer release, lifecycle, etc. Right now it feels like there isn’t a fully reproducible path for someone outside the original fork to exercise the native implementation end-to-end without reverse-engineering those assumptions from the Comet internals. Would it be possible to share the patched 0.6.1 diff/branch you used, or even just outline the minimum Celeborn-side compatibility contract / set of changes the native path expects? I’m happy to maintain a 0.7.x fork and validate it against a real Spark 3.5 cluster. I mainly want to make sure I’m implementing the intended semantics rather than just making the admission check pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
