----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59828/ -----------------------------------------------------------
Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21181 https://issues.apache.org/jira/browse/AMBARI-21181 Repository: ambari Description ------- Have a new mapper called map_anonymize which accepts a pattern containing <hide> elements. After the mapping if the pattern is found in the field, than the parts matching the <hide> elements are replaced with a configurable hiding character ( default 'X' ). Example: pattern: the secret is <hide> / <hide> which is confidential log message: Bla bla bla the secret is SECRET1 / SECRET2 which is confidential bla bla after the mapping: Bla bla bla the secret is XXXXXXX / XXXXXXX which is confidential bla bla This patch also fixes the checkpoint file writing, which wasn't atomic. The checkpoint files are now written under a temporary name first, than atomically renamed to the proper name, thus making it impossible to have a half written file. Diffs ----- ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapAnonymizeDescriptor.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapDateDescriptor.java f88435f ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapFieldDescriptor.java db086c5 ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapFieldValueDescriptor.java cf37e62 ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapAnonymizeDescriptorImpl.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/PostMapValuesAdapter.java 32aded8 ambari-logsearch/ambari-logsearch-logfeeder/docs/postMapValues.md 906359a ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/AbstractInputFile.java 2bc4e68 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java fba596d ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputManager.java 8c76785 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputS3File.java 4bf162b ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/mapper/MapperAnonymize.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/FileUtil.java ffd6cec ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/alias_config.json e2ed625 ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/mapper/MapperAnonymizeTest.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/test/resources/log4j.xml e641018 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapDate.java dcb0393 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapField.java b18439c ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapFieldAnonymize.java PRE-CREATION ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerPostMapValuesListDeserializer.java 18744e2 ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-hst.json 4ccef74 ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-zookeeper.json d3685a4 Diff: https://reviews.apache.org/r/59828/diff/1/ Testing ------- Tested on local cluster, and by the new unit tests added. Thanks, Miklos Gergely