Surya Hebbar has uploaded this change for review. (
http://gerrit.cloudera.org:8080/22599
Change subject: IMPALA-13795: Support serving webUI metadata with gzip
compression
......................................................................
IMPALA-13795: Support serving webUI metadata with gzip compression
This patch adds support for serving all the webUI metadata with gzip
content-encoding.
For large JSONs and text profiles, Impala's webUI renderings maybe
hindered by the user's network bandwidth.
As the browser's native gzip decompression is very fast e.g. 300-400MB/s,
combining it with a faster compression level(i.e. gzip Z_BEST_SPEED) in
backend results in significant increases in speed i.e. faster load times.
During compression, instead of multiple reallocations, existing string data
is reinterpreted to reduce memory usage.
Added compression level to THdfsCompression::GZIP / GzipCompressor.
Testing:
* Stability:
-> Ran tests/webserver/test_web_pages.py::TestWebPage, where Python's
'requests' library includes compression headers by default
i.e. Accept-Encoding: gzip.
-> Manually tested webUI with browser
-> Fetched webUI pages without compression using curl
* Performance:
Approximate improvements for a TPC-DS 14 query ran locally with 3 nodes
with defaults
-> JSON profile : 4.53MB to 428.94KB
8mbps throttled network: 8s to 2s
Without throttling / Raw local: 421ms to 421ms
-> Text profile : 1.24MB to 219KB
8mbps throttled network: 1.3s to 281ms
Without throttling / Raw local: 281ms to 281ms
Change-Id: I431088a30337bbef2c8d6e16dd15fb6572db0f15
---
M be/src/util/codec.cc
M be/src/util/compress.cc
M be/src/util/compress.h
M be/src/util/webserver.cc
4 files changed, 50 insertions(+), 13 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/22599/1
--
To view, visit http://gerrit.cloudera.org:8080/22599
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I431088a30337bbef2c8d6e16dd15fb6572db0f15
Gerrit-Change-Number: 22599
Gerrit-PatchSet: 1
Gerrit-Owner: Surya Hebbar <[email protected]>
