[
https://issues.apache.org/jira/browse/IMPALA-7934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758590#comment-16758590
]
ASF subversion and git services commented on IMPALA-7934:
---------------------------------------------------------
Commit b0942296ab5f24660473abc218d45978fc402d81 in impala's branch
refs/heads/master from Fredy Wijaya
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b094229 ]
IMPALA-7934: Switch to java.util.Base64 implementation
It is shown that java.util.Base64 implementation seems to have better
performance compared to Apache Commons Codec's Base64 implementation,
which can benefit operations, such as incremental stats. This patch
switches the implementation of Base64 from Apache Commons
Codec to java.util.Base64 implementation.
This is the JMH benchmark result comparing java.util.Base64 vs Commons
Codec's Base64:
Result "base64.Base64Benchmark.javaBase64":
31.149 ±(99.9%) 1.567 ms/op [Average]
(min, avg, max) = (27.564, 31.149, 34.675), stdev = 2.091
CI (99.9%): [29.583, 32.716] (assumes normal distribution)
Result "base64.Base64Benchmark.codecBase64":
65.921 ±(99.9%) 4.762 ms/op [Average]
(min, avg, max) = (58.072, 65.921, 80.470), stdev = 6.357
CI (99.9%): [61.159, 70.683] (assumes normal distribution)
Benchmark Mode Cnt Score Error Units
Base64Benchmark.javaBase64 avgt 25 31.149 ± 1.567 ms/op
Base64Benchmark.codecBase64 avgt 25 65.921 ± 4.762 ms/op
Testing:
- Ran all FE tests
- Created a table with incremental stats without a patch and read it
with the patch
Change-Id: I2d43d4a4f073a800d963ce4c77f21c9efa8471ac
Reviewed-on: http://gerrit.cloudera.org:8080/12250
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> Switch to using Java 8's Base64 impl for incremental stats encoding
> -------------------------------------------------------------------
>
> Key: IMPALA-7934
> URL: https://issues.apache.org/jira/browse/IMPALA-7934
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 3.1.0
> Reporter: bharath v
> Assignee: Fredy Wijaya
> Priority: Major
> Labels: ramp-up
> Fix For: Impala 3.2.0
>
> Attachments: base64.png
>
>
> Incremental stats are compressed and Base64 encoded before they are chunked
> and written to the HMS' partition parameters map. When they are read back, we
> need to Base64 decode and decompress.
> For certain incremental stats heavy tables, we noticed that a significant
> amount of time is spent in these base64 classes (see the attached image for
> the stack. Unfortunately, I don't have the text version of it).
> Java 8 comes with its own Base64 implementation and that has shown much
> better perf results [1] compared to apache codec's impl. So consider
> switching to Java 8's base64 impl.
> [1] http://java-performance.info/base64-encoding-and-decoding-performance/
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
