[
https://issues.apache.org/jira/browse/HIVE-4388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13739078#comment-13739078
]
stack commented on HIVE-4388:
-----------------------------
First, thanks for working on this you all.
Protbuf'ing the Result and its content will have the advantage of making the
serialization evolvable but it will likely slow the marshall/unmarshalling by
an order of magnitude (protobufing the Result and its KeyValue/Cell content
means that for every KeyValue, we'll take each piece of the KV -- the row, the
column family, the qualifier, the value -- and we'll *copy* each into a pb
ByteString; we'll then compose the pb Result of all these pb atoms -- more
copies).
Over in hbase land, we don't put the data (the actual KeyValues) into pbs; we
have a sidecar mechanism where pb is more used describing the metadata and we
put the KeyValue data into blocks of cells that we send after the metadata pb
when passing data over rpc, etc. These cell blocks can be variously composed
w/ compression, encoding, etc.
What is going on in this patch as is, is the old style serialization where we
just write out the KeyValue backing byte array. This ResultWritable is
versioned. We could just change the version when we want ResultWritable to do
serializations other than the default, current serialization, rather than pb it
all?
Just a suggestion.
> HBase tests fail against Hadoop 2
> ---------------------------------
>
> Key: HIVE-4388
> URL: https://issues.apache.org/jira/browse/HIVE-4388
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
> Reporter: Gunther Hagleitner
> Assignee: Brock Noland
> Attachments: HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch,
> HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388-wip.txt
>
>
> Currently we're building by default against 0.92. When you run against hadoop
> 2 (-Dhadoop.mr.rev=23) builds fail because of: HBASE-5963.
> HIVE-3861 upgrades the version of hbase used. This will get you past the
> problem in HBASE-5963 (which was fixed in 0.94.1) but fails with: HBASE-6396.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
