GitHub user funky-eyes edited a discussion: Discussion of seata-server and proxy compatibility programs
议题背景： 在通过钉钉群，github issue，社区收到了不少由于seata-server与4层代理不兼容的反馈，由于seata-server二阶段需要找到正确的channel进行下发，故由于通过了代理连接后，比如代理负载均衡策略有轮询，背后2台server。 tm->proxy->server1 rm->proxy->server2 tm决议时提交到server1，server1下发二阶段提交时，没有rm的连接，导致无法下发。当然如果相同resourceid的rm节点数量较多可能可以缓解该问题，但是只是治标不治本的方法。 故我提出以下几种解决方案，希望跟社区探讨下如何彻底解决以上问题，因为2.x版本上的目的是让seata更加好用，所以应该考虑到一些场景的兼容性问题或者说是适配情况。 1. 通过xid中的ip直接建立一个绕过代理的连接，保证了tm决议时，对应的tc上有相关rm的连接 缺点：1. 当begin时的server宕机后，tm决议时的server可能改变，故二阶段可能依然无法下发。 2. 当情况1发生的时候，rm可能无法创建相关连接，会导致事务失败。3. doconnect时是否需要维护这种传递式的连接地址。 4. 很有可能tc的ip是一个无法通过直接连接的ip，所以才有了代理，很可能直接连接ip是无法连接通过。 2. 通过开放raft集群方式与其他存储模式结合解决该问题，将server搭建成一个raft集群，在file为注册中心实现中增加一个通过http获取服务端集群元数据的功能，将lb背后server的真实ip地址列表全部获取，进行连接。将代理作为一个获取元数据的作用即可，实际begin，registry，commit等动作是直接连接至tc侧。相关类似能力可以参考kafka等 缺点：与方案一的缺点4相同问题，但是可以参考kafka的`advertised.listeners` 配置项，该项允许堆外暴露的ip被自定义为一个可被对方连接的ip，比如server的ip无法直接连接，但是可以连接对应nat后的地址，所以可以暴露一个nat后的地址至元数据中，提供给client。而且kafka该配置可以填写多个，会自动根据client连接的ip给于对应的元数据，比如 ``` advertised.listeners=PLAINTEXT://192.168.1.123:9092,PLAINTEXT2://nat地址:9093 listener.security.protocol.map=PLAINTEXT2:PLAINTEXT,PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT ``` 当客户端通过nat地址:9093这个连接地址连接时，获取到的server地址就是nat后的地址，保证了客户端跟服务端的网络连通性。 Topic background: In the DingTalk group, github issue, the community received a lot of feedback due to the incompatibility of the seata-server with the 4-layer proxy. Since the seata-server second stage needs to find the correct channel for delivery, it is necessary to pass the proxy connection. For example, the proxy load balancing strategy has Polling, and there are 2 servers behind it. Tm- > proxy- > server1 Rm- > proxy- > server2 When tm decides to commit to server1, when server1 issues a two-stage commit, there is no rm connection, so it cannot be sent. Of course, if the number of rm nodes with the same resourceid is large, the problem may be alleviated, but it only treats the symptoms, not the underlying causes. Therefore, I propose the following solutions, hoping to discuss with the community how to solve the above problems completely, because the purpose of the 2.x version is to make seata easier to use, so we should consider the compatibility issues or adaptations of some scenarios. 1. Establish a connection to bypass the proxy directly through the ip in the xid to ensure that when the tm resolution is made, there is a connection to the relevant rm on the corresponding tc Disadvantages: 1. When the server crashes at the beginning, the server at the time of tm resolution may change, so the second stage may still not be delivered. 2. When situation 1 occurs, rm may not be able to create related connections, resulting in transaction failure. 3. Do you need to maintain this kind of transitive connection address when doconnect? 4. It is very likely that the ip of tc is an ip that cannot be directly connected, so there is a proxy. It is very likely that the direct connection to the ip cannot be connected. 2. To solve this problem by combining it with other storage models through an open raft clustering approach. build the server into a raft cluster, add a function to obtain server level cluster metadata through http in the file as registry implementation, and obtain all the real IP address lists of the server behind the lb and connect. The proxy can be used as a function to obtain metadata. In fact, begin, registry, commit and other actions are directly connected to the tc side. For related similar capabilities, please refer to kafka, etc. Disadvantages: The same problem as the disadvantage 4 of scheme 1, but you can refer to the'advertised.listeners' configuration item of kafka, which allows the exposed ip outside the heap to be customized as an ip that can be connected to the other party. For example, the ip of the server cannot be directly connected, but it can be connected to the address corresponding to nat, so a nat address can be exposed to the metadata and provided to the client. And kafka can fill in multiple configurations, and will automatically give the corresponding metadata according to the ip connected by the client, such as ``` Advertised.listeners = PLAINTEXT://192.168.1.123:9092, PLAINTEXT2://nat Address: 9093 listener.security.protocol.map = PLAINTEXT2: PLAINTEXT, PLAINTEXT: PLAINTEXT, CONTROLLER: PLAINTEXT ``` When the client end connects through the nat address: 9093, the server address obtained is the address after nat, which ensures the network connectivity between the client end and the server end. GitHub link: https://github.com/apache/incubator-seata/discussions/6279 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
