[
https://issues.apache.org/jira/browse/HBASE-15638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-15638:
--------------------------
Description:
We need to change our protobuf. Currently it is pb2.5.0. As is, protobufs
expect all buffers to be on-heap byte arrays. It does not have facility for
dealing in ByteBuffers and off-heap ByteBuffers in particular. This fact
frustrates the off-heaping-of-the-write-path project as
marshalling/unmarshalling of protobufs involves a copy on-heap first.
So, we need to patch our protobuf so it supports off-heap ByteBuffers. To
ensure we pick up the patched protobuf always, we need to relocate/shade our
protobuf and adjust all protobuf references accordingly.
Given as we have protobufs in our public facing API, Coprocessor Endpoints --
which use protobuf Service to describe new API -- a blind relocation/shading of
com.google.protobuf.* will break our API for CoProcessor EndPoints (CPEP) in
particular. For example, in the Table Interface, to invoke a method on a
registered CPEP, we have:
{{<T extends com.google.protobuf.Service,R> Map<byte[],R> coprocessorService(
Class<T> service, byte[] startKey, byte[] endKey,
org.apache.hadoop.hbase.client.coprocessor.Batch.Call<T,R>
callable)
throws com.google.protobuf.ServiceException, Throwable}}
This issue is how we intend to shade protobuf for hbase-2.0.0 while preserving
our API as is so CPEPs continue to work on the new hbase.
was:
We need to change our protobuf. Currently it is pb2.5.0. As is, protobufs
expect all buffers to be on-heap byte arrays. It does not have facility for
dealing in ByteBuffers and off-heap ByteBuffers in particular. This fact
frustrates the off-heaping-of-the-write-path project as
marshalling/unmarshalling of protobufs involves a copy on-heap first.
So, we need to patch our protobuf so it supports off-heap ByteBuffers. To
ensure we pick up the patched protobuf always, we need to relocate/shade our
protobuf and adjust all protobuf references accordingly.
Given as we have protobufs in our public facing API, Coprocessor Endpoints --
which use protobuf Service to describe new API -- a blind relocation/shading of
com.google.protobuf.* will break our API for CoProcessor EndPoints (CPEP) in
particular. For example, in the Table Interface, to invoke a method on a
registered CPEP, we have:
<T extends com.google.protobuf.Service,R> Map<byte[],R> coprocessorService(
Class<T> service, byte[] startKey, byte[] endKey,
org.apache.hadoop.hbase.client.coprocessor.Batch.Call<T,R>
callable)
throws com.google.protobuf.ServiceException, Throwable
This issue is how we intend to shade protobuf for hbase-2.0.0 while preserving
our API as is so CPEPs continue to work on the new hbase.
> Shade protobuf
> --------------
>
> Key: HBASE-15638
> URL: https://issues.apache.org/jira/browse/HBASE-15638
> Project: HBase
> Issue Type: Bug
> Components: Protobufs
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Attachments: 15638v2.patch, HBASE-15638.master.001.patch,
> HBASE-15638.master.002.patch, HBASE-15638.master.003 (1).patch,
> HBASE-15638.master.003 (1).patch, HBASE-15638.master.003 (1).patch,
> HBASE-15638.master.003.patch, HBASE-15638.master.003.patch,
> HBASE-15638.master.004.patch, HBASE-15638.master.005.patch,
> HBASE-15638.master.006.patch, HBASE-15638.master.007.patch,
> HBASE-15638.master.007.patch, HBASE-15638.master.008.patch,
> HBASE-15638.master.009.patch, as.far.as.server.patch
>
>
> We need to change our protobuf. Currently it is pb2.5.0. As is, protobufs
> expect all buffers to be on-heap byte arrays. It does not have facility for
> dealing in ByteBuffers and off-heap ByteBuffers in particular. This fact
> frustrates the off-heaping-of-the-write-path project as
> marshalling/unmarshalling of protobufs involves a copy on-heap first.
> So, we need to patch our protobuf so it supports off-heap ByteBuffers. To
> ensure we pick up the patched protobuf always, we need to relocate/shade our
> protobuf and adjust all protobuf references accordingly.
> Given as we have protobufs in our public facing API, Coprocessor Endpoints --
> which use protobuf Service to describe new API -- a blind relocation/shading
> of com.google.protobuf.* will break our API for CoProcessor EndPoints (CPEP)
> in particular. For example, in the Table Interface, to invoke a method on a
> registered CPEP, we have:
> {{<T extends com.google.protobuf.Service,R> Map<byte[],R> coprocessorService(
> Class<T> service, byte[] startKey, byte[] endKey,
> org.apache.hadoop.hbase.client.coprocessor.Batch.Call<T,R>
> callable)
> throws com.google.protobuf.ServiceException, Throwable}}
> This issue is how we intend to shade protobuf for hbase-2.0.0 while
> preserving our API as is so CPEPs continue to work on the new hbase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
