[
https://issues.apache.org/jira/browse/LIVY-735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Saisai Shao resolved LIVY-735.
------------------------------
Fix Version/s: 0.8.0
Assignee: runzhiwang
Resolution: Fixed
Issue resolved by pull request 268
https://github.com/apache/incubator-livy/pull/268
> Fix RPC Channel Closed When Multi Clients Connect to One Driver
> ----------------------------------------------------------------
>
> Key: LIVY-735
> URL: https://issues.apache.org/jira/browse/LIVY-735
> Project: Livy
> Issue Type: Sub-task
> Reporter: Yiheng Wang
> Assignee: runzhiwang
> Priority: Major
> Fix For: 0.8.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Currently, the driver tries to support communicating with multi-clients, by
> registering each client at
> https://github.com/apache/incubator-livy/blob/master/rsc/src/main/java/org/apache/livy/rsc/driver/RSCDriver.java#L220.
> But actually, if multi-clients connect to one driver, the rpc channel will
> close, the reason are as follows.
> 1. In every communication, client sends two packages to driver: header\{type,
> id}, and payload at
> https://github.com/apache/incubator-livy/blob/master/rsc/src/main/java/org/apache/livy/rsc/rpc/RpcDispatcher.java#L144.
> 2. If client1 sends header1, payload1, and client2 sends header2, payload2 at
> the same time.
> The driver receives the package in the order: header1, header2, payload1,
> payload2.
> 3. When driver receives header1, driver assigns lastHeader at
> https://github.com/apache/incubator-livy/blob/master/rsc/src/main/java/org/apache/livy/rsc/rpc/RpcDispatcher.java#L73.
> 4. Then driver receives header2, driver process it as a payload at
> https://github.com/apache/incubator-livy/blob/master/rsc/src/main/java/org/apache/livy/rsc/rpc/RpcDispatcher.java#L78
> which cause exception and rpc channel closed.
> In the muti-active HA mode, the design doc is at:
> https://docs.google.com/document/d/1bD3qYZpw14_NuCcSGUOfqQ0pqvSbCQsOLFuZp26Ohjc/edit?usp=sharing,
> the session is allocated among servers by consistent hashing. If a new livy
> joins, some session will be migrated from old livy to new livy. If the
> session client in new livy connect to driver before stoping session client in
> old livy, then two session clients will both connect to driver, and rpc
> channel close. In this case, it's hard to ensure only one client connect to
> one driver at any time. So it's better to support multi-clients connect to
> one driver, which has no side effects.
> How to fix:
> 1. Move the code of processing client message from `RpcDispatcher` to each
> `Rpc`.
> 2. Each `Rpc` registers itself to `channelRpc` in RpcDispatcher.
> 3. `RpcDispatcher` dispatches each message to `Rpc` according to
> `ctx.channel()`.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
