Hello Adar Dembo, Kudu Jenkins,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/5565
to review the following change.
Change subject: KUDU-1812. Redact pretty-printed sensitive user data
......................................................................
KUDU-1812. Redact pretty-printed sensitive user data
This commit introduces a framework in logging.h to enable sensitive user
data to be redacted in the Kudu server, and to a lesser extent, in the
C++ client.
- The 'log_redact_user_data' gflag is added which controls whether row
and predicate data is redacted from log, error and status messages. The
flag defaults to true in the Kudu server binaries order to provide a
safe default for Kudu users. The flag is explicitly set to false in
tests and the 'kudu' CLI tool in order to aid debugging.
- There is a new thread-local boolean which indicates whether ToString
functions should redact user data in the current thread. This defaults
to 'true', but it is always also combined with the
'log_redact_user_data' flag to determine whether redaction will actually
happen.
- A utility macro KUDU_DISABLE_REDACTION(...) and RAII equivalent
ScopedDisableRedaction can disable redaction while evaluating a
particular expression or scope, useful in contexts such as the web UI or
tools where we don't want to redact.
- A macro KUDU_REDACT(expr) replaces its argument with '<redacted>' if
redaction is enabled.
- ToString and equivalent calls have been changed to consult
log_redact_user_data and the TLS flag to determine whether to perform
redaction, where appropriate. Some ToString calls specifically disable
redaction based on the type of data being stringified; this behavior
has been explicitly called out in the header documentation, but this
isn't expected to be a source of security issues in the future since
in all such cases the data is not considered sensitive. A handful of
ToString methods in our public API disable redaction in order to
retain the same behavior; these methods should never be used for
internal Kudu logging without manual redaction, and their doxygen has
been updated to indicate this.
- Redaction in the C++ client uses the same mechanism as in the server,
but the 'log_redact_user_data' is permanently set to 'true'. We can
add a public API option to allow applications to turn off redaction in
the future if it proves to be necessary.
The advantage of a TLS flag in addition to 'log_redact_user_data' is
that we're now able to apply KUDU_REDACT() at a very low level (the
stringification functions in types.cc which are used for stringifying
all user data). This means that we are by-default redacted anywhere that
stringifies a row, rather than having to look for all cases that may
lead to this stringification. Instead, we only have to find the places
that explicitly want to disable redaction, which should be the exception
rather than the rule.
Redaction of raw buffers and protobuf messages will be included in a
follow up commit.
Change-Id: I2b87a3065280116bb8af6f26f072dafdfd1ee077
Reviewed-on: http://gerrit.cloudera.org:8080/5555
Tested-by: Kudu Jenkins
Reviewed-by: Adar Dembo <a...@cloudera.com>
---
M src/kudu/client/batcher.cc
M src/kudu/client/client.cc
M src/kudu/client/client.h
M src/kudu/client/meta_cache.h
M src/kudu/client/scanner-internal.h
M src/kudu/client/session-internal.cc
M src/kudu/client/write_op.h
M src/kudu/common/column_predicate-test.cc
M src/kudu/common/partial_row.cc
M src/kudu/common/partial_row.h
M src/kudu/common/partition-test.cc
M src/kudu/common/partition.cc
M src/kudu/common/partition.h
M src/kudu/common/partition_pruner-test.cc
M src/kudu/common/partition_pruner.h
M src/kudu/common/row_operations.cc
M src/kudu/common/row_operations.h
M src/kudu/common/scan_spec-test.cc
M src/kudu/common/schema-test.cc
M src/kudu/common/types-test.cc
M src/kudu/common/types.cc
M src/kudu/integration-tests/external_mini_cluster.cc
M src/kudu/server/webserver.cc
M src/kudu/tablet/transactions/write_transaction.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tools/tool_main.cc
M src/kudu/util/flag_tags-test.cc
M src/kudu/util/flags-test.cc
M src/kudu/util/logging-test.cc
M src/kudu/util/logging.cc
M src/kudu/util/logging.h
M src/kudu/util/test_util.cc
32 files changed, 349 insertions(+), 78 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/65/5565/1
--
To view, visit http://gerrit.cloudera.org:8080/5565
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2b87a3065280116bb8af6f26f072dafdfd1ee077
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: branch-1.2.x
Gerrit-Owner: Dan Burkert <danburk...@apache.org>
Gerrit-Reviewer: Adar Dembo <a...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
