ulysses-you opened a new issue, #5383: URL: https://github.com/apache/kyuubi/issues/5383
### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before creating - [X] I have searched in the [task list](https://github.com/orgs/apache/projects/296) and found no similar tasks. ### Mentor - [X] I have sufficient knowledge and experience of this task, and I volunteer to be the mentor of this task to guide contributors to complete the task. ### Skill requirements - Knowledge about Spark Connect - Knowledge about Kyuubi architecture - Knowledge about protobuf - Knowledge about grpc - Knowledge about thrift ### Background and Goals Make Kyuubi server compatible with Spark Connect protocol, so that people can use Spark Connect client to connect to Kyuubi Server. ### Implementation steps 1. Add a new Spark Connect frontend 1.1 Add basic gRpc server as frontend 1.2 Compatible with Spark Connect protocol, see https://github.com/apache/spark/blob/master/connector/connect/common/src/main/protobuf/spark/connect/base.proto 1.3 Support ExecutePlan 1.4 Support AnalyzePlan 1.5 Support Config 1.6 Support AddArtifacts 1.7 Support ArtifactsStatus 1.8 Support Interrupt 1.9 Support ReattachExecute 1.10 Support ReleaseExecute 1.11 Serialize the protobuf based request to thrift 1.12 Deserialize thrift response to protobuf based 2. Add a new Spark Connect backend 2.1 Imprort Sprak-Connect-Server and rewrite SparkConnectService using Thrift https://github.com/apache/spark/blob/master/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectServer.scala 2.2 Compatible with response from backend to frontend to make 1.4 work 3. Add IT 4. Add docs ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
