[
https://issues.apache.org/jira/browse/HBASE-12597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229559#comment-14229559
]
Jurriaan Mous commented on HBASE-12597:
---------------------------------------
bq. I was just trying to warn you off the implementation to instead work
against the Connection (rather than HCI) if possible. The new Interfaces are
kinda sweet. Would be grand if your new rpc Interface could be in the same
style.
Since the RpcClient changes proposed here would probably land on branch-1
and/or master I think I need to make a branch myself to support the new Hbase
version. I will take your recommendation and base it on the new Api. :)
bq. Can you say more on the above? Why would you use hbase implementation
connection pooling and not Netty's (I'm not sure – I'm guessing it can do this)
While Netty pools the connections (called channels) for the outside world in a
connection pool, inside the application depends on how you want to recycle
connections. I based it on the ConnectionId inside RpcClient so channels get
mapped in the same way as in Hbase itself.
ConnectionId takes care of the grouping which is on User ticket, protobuf
servicename, the region server address and the rpc timeout. This is pretty
HBase specific. The map it would end up in is PoolMap<RpcClient.ConnectionId,
AsyncRpcChannel> connections. And the client uses the map to give users an
existing or a new connection.
bq. Do you have a proposal for rpcclient interface?
Will look into it. I would probably extract whatever is used in the
HConnectionImplementation into an interface.
bq. There is our use of the protobuf Service Interfaces taking rpc controllers
etc. We just took it on because it was there and the pb code generation did all
the stubbing for us. Just saying we are amenable to change long as the basic
protocol stays the same, for now.
The protobuf Service interface made it easy to make a comparable
implementation. Although I could see that it could be made more optimal since
like the Hbase implementation I also needed to create a new async
PayloadCarryingRpcController to pass information around the limiting interface.
Although it was not that big of a problem. I would recommend not to make big
changes in it without taking care of an async implementation. But I don't know
if it is worth it to create new implementations for each Service.
bq. That'd be pretty great. Thinking on it, I suppose it should be ok doing it
as netty channel. You just have to have our idiosyncratic header stuff up front
on the connection which I believe netty will let you just insert. Thats great.
I handle those directly after channel creation. If channel compression is added
there needs to be a new header defining the compression defined. We need
something of a header in which the client announces in the connect header which
compressions it supports and the server replying with a header which
compression it uses.
As said I will return with a first suggestion on the RpcClient interface.
> Add RpcClient interface and enable changing of RpcClient implementation
> -----------------------------------------------------------------------
>
> Key: HBASE-12597
> URL: https://issues.apache.org/jira/browse/HBASE-12597
> Project: HBase
> Issue Type: Improvement
> Components: Client
> Reporter: Jurriaan Mous
>
> Currently HConnectionImplementation works with the included RpcClient which
> is a direct implementation and not defined by an interface.
> It would be great to be able to swap out the default RpcClient with another
> implementation which can also be controlled by the default
> HConnectionImplementation.
> Suggested changes:
> - Create a RpcClient interface which defines all the ways
> HConnectionImplementation interacts with an RPC client. Like getting a
> blocking protobuf service interface or closing the client.
> - Define which RpcClient implementation to construct by setting a
> configuration variable which defaults to the current RpcClient.
> - Possibly create an abstract RpcClient class to only load all the basic Rpc
> layer configurations to be used in an implementation.
> Why? It enables experimentation with RpcClients which could enable new
> features or could be more performant than the included client.
> I created a new RpcClient implementation based on Netty which can also be
> called asynchronously. It would be great to also be able to use this
> RpcClient in all the default ways and tests to see if there are any issues
> with it.
> https://github.com/jurmous/async-hbase-client/
> https://github.com/jurmous/async-hbase-client/blob/master/src/main/java/org/apache/hadoop/hbase/ipc/AsyncRpcClient.java
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
