Tim Armstrong has uploaded a new patch set (#8). Change subject: IMPALA-4118: extract encryption utils from BufferedBlockMgr ......................................................................
IMPALA-4118: extract encryption utils from BufferedBlockMgr As groundwork for IMPALA-4118, extract encryption and integrity verification routines into a separate module that can be used by the new BufferPool. Simplify the logic in BufferedBlockMgr by not distinguishing between the integrity check and encryption options, which are controlled by the same command line flag anyway. This patch changes how the OpenSSL RNG is seeded. I consulted with the original author of the code (Michael Yoder), who suggested that the new approach would be appropriate: "I'd pick whatever implementation is easiest for you. You could add entropy once per query, or once every X keys (100 keys seems reasonable), or once every "convenient place". 4kb is probably overkill, you could use 512b for example." Testing: Added some unit tests for the utilities. We already have coverage of the BufferedBlockMgr with encryption in buffered-block-mgr-test. Also reduce the number of iterations in the buffered-block-mgr-test to save some testing time. Change-Id: Ibebe431f69c3c569cbff68171beaa32ef2ab03b6 --- M be/src/benchmarks/bswap-benchmark.cc M be/src/bufferpool/buffer-pool-test.cc M be/src/bufferpool/buffer-pool.cc M be/src/bufferpool/reservation-tracker-test.cc M be/src/bufferpool/reservation-tracker.cc M be/src/codegen/codegen-symbol-emitter.cc M be/src/common/init.cc M be/src/common/names.h M be/src/exec/catalog-op-executor.cc M be/src/exec/hdfs-parquet-scanner.cc M be/src/exec/kudu-scan-node-test.cc M be/src/exec/kudu-scan-node.cc M be/src/exec/kudu-scanner.cc M be/src/exec/parquet-column-readers.cc M be/src/exec/parquet-metadata-utils.cc M be/src/exprs/bit-byte-functions.cc M be/src/exprs/operators-ir.cc M be/src/exprs/timezone_db.cc M be/src/runtime/buffered-block-mgr-test.cc M be/src/runtime/buffered-block-mgr.cc M be/src/runtime/buffered-block-mgr.h M be/src/runtime/data-stream-sender.cc M be/src/runtime/hdfs-fs-cache-test.cc M be/src/scheduling/request-pool-service.cc M be/src/util/CMakeLists.txt M be/src/util/cpu-info.cc M be/src/util/hdr-histogram.cc A be/src/util/openssl-util-test.cc A be/src/util/openssl-util.cc A be/src/util/openssl-util.h M be/src/util/redactor-config-parser-test.cc M be/src/util/redactor.cc 32 files changed, 503 insertions(+), 265 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/4389/8 -- To view, visit http://gerrit.cloudera.org:8080/4389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibebe431f69c3c569cbff68171beaa32ef2ab03b6 Gerrit-PatchSet: 8 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com> Gerrit-Reviewer: Michael Ho <k...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>