hansva commented on code in PR #9590:
URL: https://github.com/apache/hop/pull/9590#discussion_r4640728224
##########
plugins/transforms/textchunker/src/main/java/org/apache/hop/pipeline/transforms/chunker/document/metadata/HopMetadataJsonParser.java:
##########
@@ -190,10 +197,14 @@ private static JsonNode redactSecrets(JsonNode node) {
while (names.hasNext()) {
fields.add(names.next());
}
+ boolean secretEntry = isSecretEntry(object, fields);
for (String field : fields) {
JsonNode value = object.get(field);
if (value != null && value.isValueNode()) {
String redacted = SecretRedaction.redact(field, value.asText());
+ if (secretEntry && SecretRedaction.isEntryValueKey(field) &&
!value.asText().isEmpty()) {
Review Comment:
Nit: `value.asText()` on a `NullNode` returns `"null"`, so
`{"name":"DB_PASSWORD","value":null}` is shown as `**redacted**`, which
suggests a secret is set when it isn't. Booleans and numbers under
secret-looking keys get the same treatment, and so does the new top-level path
above (around line 124), e.g. `"usePassword": false`. Skipping `value.isNull()`
and booleans would keep those meaningful.
##########
plugins/transforms/textchunker/src/main/java/org/apache/hop/pipeline/transforms/chunker/document/hopxml/HopXmlSupport.java:
##########
@@ -356,16 +363,21 @@ private static void serializeElement(Element element,
StringBuilder sb, int dept
}
String text = SecretRedaction.redact(tag, textOnly.toString().trim());
+ if (secretEntryValue && !text.isEmpty()) {
Review Comment:
Nit: a value that is only a variable reference, e.g.
`<name>DB_PASSWORD</name><value>${PROD_DB_PASSWORD}</value>`, is redacted too.
That hides which variable supplies the credential, which isn't a secret and is
useful to find. Consider keeping values that match `^\$\{[^}]+\}$`, here, in
the JSON path and in `formatParameters`.
##########
plugins/transforms/textchunker/src/main/java/org/apache/hop/pipeline/transforms/chunker/document/metadata/HopMetadataJsonParser.java:
##########
@@ -190,10 +197,14 @@ private static JsonNode redactSecrets(JsonNode node) {
while (names.hasNext()) {
fields.add(names.next());
}
+ boolean secretEntry = isSecretEntry(object, fields);
for (String field : fields) {
JsonNode value = object.get(field);
if (value != null && value.isValueNode()) {
Review Comment:
When `secretEntry` is true but the entry's value is an array or object, this
goes to the `else` branch and `redactSecrets(value)`. For an array of strings
nothing gets redacted, because there's no field name to match. For example,
`{"name":"API_TOKENS","value":["tok-1","tok-2"]}` puts both tokens into the
chunk text. The XML side redacts the whole subtree in this case
(`secretEntryValue`). Could you either replace a container value with
`REDACTED` when `secretEntry && isEntryValueKey(field)`, or pass a "redact
everything below" flag down the recursion like the XML side? A test for this
case would be good too.
##########
plugins/transforms/textchunker/src/main/java/org/apache/hop/pipeline/transforms/chunker/document/hopxml/HopXmlSupport.java:
##########
@@ -356,16 +363,21 @@ private static void serializeElement(Element element,
StringBuilder sb, int dept
}
String text = SecretRedaction.redact(tag, textOnly.toString().trim());
+ if (secretEntryValue && !text.isEmpty()) {
+ text = SecretRedaction.REDACTED;
+ }
if (childElements.isEmpty()) {
if (!text.isEmpty() && !SKIP_TAGS.contains(tag)) {
indent(sb, depth).append(tag).append(": ").append(text).append('\n');
}
return;
}
+ // The root is the transform or action itself: its <name> is not the name
of an entry.
+ boolean secretEntry = depth > 0 && isSecretEntry(childElements);
Review Comment:
Nit: this marks any nested element with a
`<name>`/`<variable>`/`<variable_name>` child containing a marker as a secret
entry. Ordinary field definitions match too, so a Generate rows / Add constants
/ Text file input field named `token_count` or `password_changed_at` loses its
`nullif`/default value (`nullif: **redacted**`). Could this be limited to the
real entry containers (parameters, variable mappings, Set variables fields)? If
not, please document the over-redaction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]