Deeply thanks for your suggestions, consider our system mainly interacts within interior system, in this point, there is no need to cover cross-language serialization. If cross-language is excluded, in my opinion, ProtoBuf not the best choice. And have I fully understand what you express or are there any sides I have not take into account? Looking forward more suggestions from you , Thx.
Best wishes！ CalvinKirs On 01/5/2021 22:51，Sheng Wu<[email protected]> wrote： As a developer's perspective(not mentor). When you work into the global level integration at the cloud native field, you will notice how grpc(protobuf) widely used :) If your plans are around this, I would like to suggest you considering more than just performance. Some of you should know what I think about the performance :P Sheng Wu 吴晟 Twitter, wusheng1108 lidong dai <[email protected]> 于2021年1月5日周二 下午10:39写道： hi, It seems like Kryo is more popular besides ProtoBuf, so I suggest Kryo Best Regards --------------- DolphinScheduler(Incubator) PPMC Lidong Dai [email protected] --------------- On Mon, Jan 4, 2021 at 2:39 PM CalvinKirs <[email protected]> wrote: hi, everyone. Before I mentioned the communication serialization scheme, I conducted JMH Test on protobuf, kryo and protostuff. This is the result @WarmUp(iterations = 10, time = 2) @measurement(iterations = 10, time = 2) @BenchmarkMode(Mode.Throughput) @OutputTimeUnit(TimeUnit.SECONDS) @State(Scope.Benchmark) Benchmark Mode Cnt Score Error Units SerializerBenchMark.Kryo thrpt 20 171204.238 ± 27941.211 ops/s SerializerBenchMark.ProtoBuf thrpt 20 180551.996 ± 6657.504 ops/s SerializerBenchMark.ProtoStuff thrpt 20 221247.510 ± 3620.933 ops/s 大家可以下载我分支的代码来进行测试： https://github.com/CalvinKirs/incubator-dolphinscheduler/tree/jmh_ser/dolphinscheduler-microbench Note: Kryo (I turned on the function of class registration) When the data volume is too small, the efficiency of ProtoStuff is 3-5 times that of Kryo. When the data volume is too large, ProtoStuff still performs well enough. I prefer ProtoStuff, the serialization scheme is basically becoming mature, so the community release will be slower. (ProtoStuff released three versions in 2020, so I think there is no problem with community activity.) At the same time, it is extremely efficient, easy to use, and performs very well in serialization speed and serialized volume. ProtoBuf is very good in cross-language, but we may not need cross-language support later. At the same time, because of the need to write proto files, I think this has a certain learning cost for new developers. Therefore it is not recommended. What do you think? I want to hear suggestions from the community. Best wishes！ CalvinKirs
