[
https://issues.apache.org/jira/browse/KUDU-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211417#comment-17211417
]
ASF subversion and git services commented on KUDU-3201:
-------------------------------------------------------
Commit 6b20440f4c51a6b69c1382db51139bf8d3467b05 in kudu's branch
refs/heads/master from Grant Henke
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=6b20440 ]
KUDU-3201: [hms] Support gzipped HMS notifications (GzipJSONMessageEncoder)
This patch adds support for HMS notification messages encoded
via the GzipJSONMessageEncoder that was added in
HIVE-20679. It does this by decoding the base64 encoding
and decompressing the messages sent with the `gzip(json-2.0)`
format.
Because HIVE-20679 is only in Hive 4 which is unreleased,
I created a static encoded message to test the gzip functionality.
I also manually tested this functionality on a real cluster that has
HIVE-20679.
While testing this patch I found and fixed a bug in our zlib.cc
Uncompress method. Without the fix zlib would
return a BUF_ERROR when the decompressed data fits
in the output buffer on the first call to `inflate`.
For reference I used the following post to help understand
the expected usage of `inflate`:
https://zlib.net/zlib_how.html
Change-Id: I0f94e2446255e1fafb1dac9bf9ce23b81d6c0960
Reviewed-on: http://gerrit.cloudera.org:8080/16565
Tested-by: Grant Henke <granthe...@apache.org>
Reviewed-by: Andrew Wong <aw...@cloudera.com>
> Support gzipped HMS notifications (GzipJSONMessageEncoder)
> ----------------------------------------------------------
>
> Key: KUDU-3201
> URL: https://issues.apache.org/jira/browse/KUDU-3201
> Project: Kudu
> Issue Type: Improvement
> Components: hms
> Affects Versions: 1.13.0
> Reporter: Grant Henke
> Assignee: Grant Henke
> Priority: Major
>
> In HIVE-20679 a new message encoder was added to Hive that gzips and based64
> encodes notification messages. In HIVE-21307 the default encoder was changed
> to use the gzip encoder, GzipJSONMessageEncoder.
> Though Hive 4 is not released, failures as a result of this new message
> format were reported in IMPALA-8751. We should add support for the gzip
> format to prevent future HMS sync issues.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
