Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4118: extract encryption utils from BufferedBlockMgr ......................................................................
IMPALA-4118: extract encryption utils from BufferedBlockMgr As groundwork for IMPALA-4118, extract encryption and integrity verification routines into a separate module that can be used by the new BufferPool. Simplify the logic in BufferedBlockMgr by not distinguishing between the integrity check and encryption options, which are controlled by the same command line flag anyway. This patch changes how the OpenSSL RNG is seeded. I consulted with the original author of the code (Michael Yoder), who suggested that the new approach would be appropriate: "I'd pick whatever implementation is easiest for you. You could add entropy once per query, or once every X keys (100 keys seems reasonable), or once every "convenient place". 4kb is probably overkill, you could use 512b for example." Testing: Added some unit tests for the utilities. We already have coverage of the BufferedBlockMgr with encryption in buffered-block-mgr-test. Also reduce the number of iterations in the buffered-block-mgr-test to save some testing time. Change-Id: Ibebe431f69c3c569cbff68171beaa32ef2ab03b6 Reviewed-on: http://gerrit.cloudera.org:8080/4389 Reviewed-by: Tim Armstrong <tarmstr...@cloudera.com> Tested-by: Internal Jenkins --- M be/src/benchmarks/bswap-benchmark.cc M be/src/bufferpool/buffer-pool-test.cc M be/src/bufferpool/buffer-pool.cc M be/src/bufferpool/reservation-tracker-test.cc M be/src/bufferpool/reservation-tracker.cc M be/src/codegen/codegen-symbol-emitter.cc M be/src/common/init.cc M be/src/common/names.h M be/src/exec/catalog-op-executor.cc M be/src/exec/hdfs-parquet-scanner.cc M be/src/exec/kudu-scan-node-test.cc M be/src/exec/kudu-scan-node.cc M be/src/exec/kudu-scanner.cc M be/src/exec/parquet-column-readers.cc M be/src/exec/parquet-metadata-utils.cc M be/src/exprs/bit-byte-functions.cc M be/src/exprs/operators-ir.cc M be/src/exprs/timezone_db.cc M be/src/runtime/buffered-block-mgr-test.cc M be/src/runtime/buffered-block-mgr.cc M be/src/runtime/buffered-block-mgr.h M be/src/runtime/data-stream-sender.cc M be/src/runtime/hdfs-fs-cache-test.cc M be/src/scheduling/request-pool-service.cc M be/src/util/CMakeLists.txt M be/src/util/cpu-info.cc M be/src/util/hdr-histogram.cc A be/src/util/openssl-util-test.cc A be/src/util/openssl-util.cc A be/src/util/openssl-util.h M be/src/util/redactor-config-parser-test.cc M be/src/util/redactor.cc 32 files changed, 505 insertions(+), 266 deletions(-) Approvals: Internal Jenkins: Verified Tim Armstrong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/4389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ibebe431f69c3c569cbff68171beaa32ef2ab03b6 Gerrit-PatchSet: 13 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho <k...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>