Background & Motivation What do we need to do
* Will we add a new module? maybe. * Will we add new APIs? Yes. * Will we add new feature? Yes. Why should we do that * Are there any problems of our current project? a. Remoting module is too complicated to maintain, gRPC makes it easier to establish a robust communication layer, the current remoting module would be simplified radically. b. gRPC has been the de-facto standard in CloudNative, service mesh would be easily applied if gRPC is enabled. c. The private protocol of RocketMQ depends on the FastJson, it is difficult to adapt for other language. On the other side, since the pop consumer has been merged, we could implement new SDK based on gRPC and pop, which is easier to develop and maintain. Chinese Version: a. Remoting 模块对于长期的维护而言过于复杂了，我们可以使用 gRPC 更轻松地建立起一个健壮的通信层，这会使得现有的 remoting 模块从根本上得到简化。 b. gRPC 目前已经是云原生时代的事实标准，使用 gRPC 可以使得我们天然获取一些云原生的能力，比如 Service Mesh。 c. 目前 RocketMQ 的私有协议强烈依赖 FastJson，多语言的适配将会变得困难。 从另外一个角度来说，鉴于 pop 消费者已经被合并，我们可以基于 gRPC 和 pop 实现新的 SDK，新的 SDK 将会更加易于开发和维护。 Goals * What problem is this proposal designed to solve? Support gRPC's protocol, simplify current communication layer oof RocketMQ, make it easier to adapt for other language, which is not limited to CPP/GO/C#/GO。 Chinese Version: 支持 gRPC 协议，简化 RocketMQ 现有的通信层，复用 gRPC 的能力，简化多语言适配成本，不限于 CPP/GO/C#/GO。 * To what degree should we solve the problem? This RIP must guarantee below point: 1. Compatibility: Both of gRPC and RemotingCommand should be supported. 2. High performance: This implementation does not affects latency and throughput. Chinese Version: 新方案需要保证两点： 1. 兼容性：同时支持 gRPC 和 RemotingCommand 协议，不影响现有功能。 2. 高性能：基于 gRPC 的实现不影响整理的延时和吞吐量。 Non-Goals * What problem is this proposal NOT designed to solve? Nothing specific. * Are there any limits of this proposal? Nothing specific. Changes Architecture Current broker processor and client. [https://intranetproxy.alipay.com/skylark/lark/0/2021/png/200096/1623142547507-128b85f5-98f4-4568-85f8-28ef32982b7c.png] Proposed gRPC processor and client. [https://intranetproxy.alipay.com/skylark/lark/0/2021/png/200096/1623142552491-a7f58ac0-cd7d-4ddd-936e-fb296b667196.png] Broker would provide a protocol negotiate procedure to distinguish RemotingCommand from gRPC protocol. two kinds or processor in broker would re-use the same port to serve for RPC from different SDK. Chinese Version: broker 本身提供协议协商机制用于区分 RemotingCommnad 和 gRPC 协议，broker 针对 gRPC 和 RemotingCommand 提供不同的 processor 为各自的 SDK 服务。 Interface Design/Change * Method signature changes Nothing specific. * Method behavior changes Nothing specific. * CLI command changes Nothing specific. * Log format or content changes Nothing specific. Compatibility, Deprecation, and Migration Plan * Are backward and forward compatibility taken into consideration? Broker support processor of RemotingCommand and gRPC simultaneously, so there are one compatibility situations: If user migrates from original SDK to gRPC SDK in push mode, the re-balance policy should make sure that it would not cause repeated consumption for a lot of messages. * Are there deprecated APIs? Nothing specific. * How do we do migration? Nothing specific. Implementation Outline We will implement the proposed changes by 4 phases. Phase 1 1. Provides gRPC protocol definition(IDL) Phase 2 1. Implement gRPC processor of broker. 2. Implement protocol negotiation of two kinds of protocol(gRPC and RemotingCommand) Phase 3 1. Implement new JAVA and CPP native SDK based on gRPC Phase 4 1. Implement native SDK base on gRPC for other language. Rejected Alternatives How does alternatives solve the issue you proposed? Thrift? not so much impact as gRPC in community. Pros and Cons of alternatives Nothing specific. Why should we reject above alternatives
