[
https://issues.apache.org/jira/browse/HBASE-10169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13863800#comment-13863800
]
Andrew Purtell commented on HBASE-10169:
----------------------------------------
I could see how we might some new APIs, but how about:
1. New client API. The porcelain revealed to the user. This should represent
invoking a coprocessor call on more than one region on a regionserver. Using a
List<R> type here would be ok.
2. New wire level API. The client should group regions involved in a the
processing of #1, create a request object per-regionserver containing multiple
CoprocessorServiceRequest instances (one per region), and dispatch them.
{code}
message BatchCoprocessorServiceRequest {
repeated CoprocessorServiceRequest = 1;
}
{code}
And the response:
{code}
message CoprocessorServiceResponseOrException {
required CoprocessorServiceResponse response = 1;
// If the operation failed, this exception is set
optional NameBytesPair exception = 2;
}
message BatchCoprocessorServiceResponse {
repeated CoprocessorServiceResponseOrException = 1;
}
{code}
3. New server API. RegionServer support for receiving the request of #2,
dispatching them individually, tracking their execution, and returning the
combined response. Aggregation of the server side responses should be done
separately as HBASE-5762. For now, wait for all of the invocations in the RS to
complete and send each response back.
If this goes well, perhaps we can deprecate the old API and message types and
switch to this one as the preferred way to execute coprocessors, so I wouldn't
worry about having similar client APIs for single coprocessor call and batched
coprocessor calls at this time.
> Batch coprocessor
> -----------------
>
> Key: HBASE-10169
> URL: https://issues.apache.org/jira/browse/HBASE-10169
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
> Affects Versions: 0.99.0
> Reporter: Jingcheng Du
> Assignee: Jingcheng Du
> Attachments: Batch Coprocessor Design Document.docx, HBASE-10169.patch
>
>
> This is designed to improve the coprocessor invocation in the client side.
> Currently the coprocessor invocation is to send a call to each region. If
> there’s one region server, and 100 regions are located in this server, each
> coprocessor invocation will send 100 calls, each call uses a single thread in
> the client side. The threads will run out soon when the coprocessor
> invocations are heavy.
> In this design, all the calls to the same region server will be grouped into
> one in a single coprocessor invocation. This call will be spread into each
> region in the server side, and the results will be merged ahead in the server
> side before being returned to the client.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
