Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package nelm for openSUSE:Factory checked in at 2025-07-14 10:50:49 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/nelm (Old) and /work/SRC/openSUSE:Factory/.nelm.new.7373 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "nelm" Mon Jul 14 10:50:49 2025 rev:13 rq:1292303 version:1.8.0 Changes: -------- --- /work/SRC/openSUSE:Factory/nelm/nelm.changes 2025-07-11 21:31:50.709336168 +0200 +++ /work/SRC/openSUSE:Factory/.nelm.new.7373/nelm.changes 2025-07-14 10:55:59.575305225 +0200 @@ -1,0 +2,11 @@ +Sat Jul 12 07:15:22 UTC 2025 - Johannes Kastl <opensuse_buildserv...@ojkastl.de> + +- Update to version 1.8.0: + * Features + - werf.io/sensitive-paths annotation and + WERF_FEAT_FIELD_SENSITIVE featgate (#364) (e3f9798) + * Bug Fixes + - leaking goroutines during tracking (1c1be03) + - logs from libraries still showed by default (c6b3928) + +------------------------------------------------------------------- Old: ---- nelm-1.7.2.obscpio New: ---- nelm-1.8.0.obscpio ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ nelm.spec ++++++ --- /var/tmp/diff_new_pack.AkZE1F/_old 2025-07-14 10:56:00.591347300 +0200 +++ /var/tmp/diff_new_pack.AkZE1F/_new 2025-07-14 10:56:00.595347466 +0200 @@ -17,7 +17,7 @@ Name: nelm -Version: 1.7.2 +Version: 1.8.0 Release: 0 Summary: Helm 3 alternative License: Apache-2.0 ++++++ _service ++++++ --- /var/tmp/diff_new_pack.AkZE1F/_old 2025-07-14 10:56:00.635349123 +0200 +++ /var/tmp/diff_new_pack.AkZE1F/_new 2025-07-14 10:56:00.639349288 +0200 @@ -3,7 +3,7 @@ <param name="url">https://github.com/werf/nelm</param> <param name="scm">git</param> <param name="exclude">.git</param> - <param name="revision">v1.7.2</param> + <param name="revision">v1.8.0</param> <param name="versionformat">@PARENT_TAG@</param> <param name="versionrewrite-pattern">v(.*)</param> <param name="changesgenerate">enable</param> ++++++ _servicedata ++++++ --- /var/tmp/diff_new_pack.AkZE1F/_old 2025-07-14 10:56:00.663350282 +0200 +++ /var/tmp/diff_new_pack.AkZE1F/_new 2025-07-14 10:56:00.667350448 +0200 @@ -1,6 +1,6 @@ <servicedata> <service name="tar_scm"> <param name="url">https://github.com/werf/nelm</param> - <param name="changesrevision">50af53676602ce74ed9e6f0bd4a4df62bae061ad</param></service></servicedata> + <param name="changesrevision">c40b3aaefe124b57d9d43b0cccc0f6f58bd77646</param></service></servicedata> (No newline at EOF) ++++++ nelm-1.7.2.obscpio -> nelm-1.8.0.obscpio ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/nelm-1.7.2/CHANGELOG.md new/nelm-1.8.0/CHANGELOG.md --- old/nelm-1.7.2/CHANGELOG.md 2025-07-10 15:34:42.000000000 +0200 +++ new/nelm-1.8.0/CHANGELOG.md 2025-07-11 16:18:26.000000000 +0200 @@ -1,5 +1,18 @@ # Changelog +## [1.8.0](https://www.github.com/werf/nelm/compare/v1.7.2...v1.8.0) (2025-07-11) + + +### Features + +* werf.io/sensitive-paths annotation and WERF_FEAT_FIELD_SENSITIVE featgate ([#364](https://www.github.com/werf/nelm/issues/364)) ([e3f9798](https://www.github.com/werf/nelm/commit/e3f97984dbb8dc3a13e186284f49b72efc9943f4)) + + +### Bug Fixes + +* leaking goroutines during tracking ([1c1be03](https://www.github.com/werf/nelm/commit/1c1be031e43311e015be06fc4ed07c46ec785fe2)) +* logs from libraries still showed by default ([c6b3928](https://www.github.com/werf/nelm/commit/c6b39287b0c132b324b7d9ff26b43d769dc6bce9)) + ### [1.7.2](https://www.github.com/werf/nelm/compare/v1.7.1...v1.7.2) (2025-07-10) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/nelm-1.7.2/README.md new/nelm-1.8.0/README.md --- old/nelm-1.7.2/README.md 2025-07-10 15:34:42.000000000 +0200 +++ new/nelm-1.8.0/README.md 2025-07-11 16:18:26.000000000 +0200 @@ -38,6 +38,7 @@ - [Annotation `<id>.external-dependency.werf.io/resource`](#annotation-idexternal-dependencywerfioresource) - [Annotation `<id>.external-dependency.werf.io/name`](#annotation-idexternal-dependencywerfioname) - [Annotation `werf.io/sensitive`](#annotation-werfiosensitive) + - [Annotation `werf.io/sensitive-paths`](#annotation-werfiosensitive-paths) - [Annotation `werf.io/track-termination-mode`](#annotation-werfiotrack-termination-mode) - [Annotation `werf.io/fail-mode`](#annotation-werfiofail-mode) - [Annotation `werf.io/failures-allowed-per-replica`](#annotation-werfiofailures-allowed-per-replica) @@ -427,6 +428,21 @@ Don't show diffs for the resource. +The behavior of this annotation depends on the `NELM_FEAT_FIELD_SENSITIVE` feature gate: +- **Without feature gate (default):** Hides the entire resource content +- **With feature gate:** Redacts only common sensitive fields (`data.*`, `stringData.*`) instead of hiding the entire resource + +#### Annotation `werf.io/sensitive-paths` + +Format: `JSONPath1,JSONPath2,...` \ +Example: `werf.io/sensitive-paths: "$.spec.template.spec.containers[*].env[*].value,$.data.*"` + +Allows fine-grained control over which specific fields should be redacted in diffs using JSONPath expressions. Multiple paths can be specified as a comma-separated list. + +This provides precise control over sensitive data redaction, allowing you to hide only specific sensitive fields (like passwords, API keys, etc.) rather than the entire resource, making diffs more useful while still protecting sensitive information. + +*Annotation precedence:* `werf.io/sensitive-paths` has highest priority, over `werf.io/sensitive: "true"` + #### Annotation `werf.io/track-termination-mode` Format: `WaitUntilResourceReady|NonBlocking` \ @@ -600,6 +616,21 @@ Every few seconds print stack traces of all goroutines. Useful for debugging purposes. +#### Env variable `NELM_FEAT_FIELD_SENSITIVE` + +Example: +```shell +export NELM_FEAT_FIELD_SENSITIVE=true +nelm release plan install -n myproject -r myproject +``` + +Changes the behavior of the `werf.io/sensitive` annotation and default Secret handling: + +- **Without feature gate (default):** `werf.io/sensitive: "true"` and Secrets without annotations hide the entire resource content +- **With feature gate:** `werf.io/sensitive: "true"` and Secrets without annotations hide only `data.*` and `stringData.*` fields + +Note: The `werf.io/sensitive-paths` annotation works regardless of this feature gate setting. + ### More information For more information, see [Helm docs](https://helm.sh/docs/) and [werf docs](https://werf.io/docs/v2/usage/deploy/overview.html). diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/nelm-1.7.2/go.mod new/nelm-1.8.0/go.mod --- old/nelm-1.7.2/go.mod 2025-07-10 15:34:42.000000000 +0200 +++ new/nelm-1.8.0/go.mod 2025-07-11 16:18:26.000000000 +0200 @@ -24,6 +24,7 @@ github.com/jellydator/ttlcache/v3 v3.1.1 github.com/looplab/fsm v1.0.2 github.com/moby/term v0.5.0 + github.com/ohler55/ojg v1.26.7 github.com/onsi/ginkgo/v2 v2.20.1 github.com/onsi/gomega v1.36.0 github.com/pkg/errors v0.9.1 @@ -32,11 +33,12 @@ github.com/sourcegraph/conc v0.3.0 github.com/spf13/cobra v1.8.0 github.com/spf13/pflag v1.0.5 + github.com/stretchr/testify v1.10.0 github.com/tidwall/sjson v1.2.5 github.com/wI2L/jsondiff v0.5.0 github.com/werf/3p-helm v0.0.0-20250609150428-130783e0dc18 github.com/werf/common-go v0.0.0-20250520111308-b0eda28dde0d - github.com/werf/kubedog v0.13.1-0.20250710125425-f736fad4b7b7 + github.com/werf/kubedog v0.13.1-0.20250710181210-b4a5a7f76b11 github.com/werf/lockgate v0.1.1 github.com/werf/logboek v0.6.1 github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e @@ -135,6 +137,7 @@ github.com/opencontainers/image-spec v1.1.0 // indirect github.com/peterbourgon/diskv v2.0.1+incompatible // indirect github.com/pjbgf/sha1cd v0.3.0 // indirect + github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect github.com/prometheus/client_golang v1.19.0 // indirect github.com/prometheus/client_model v0.6.0 // indirect github.com/prometheus/common v0.48.0 // indirect diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/nelm-1.7.2/go.sum new/nelm-1.8.0/go.sum --- old/nelm-1.7.2/go.sum 2025-07-10 15:34:42.000000000 +0200 +++ new/nelm-1.8.0/go.sum 2025-07-11 16:18:26.000000000 +0200 @@ -316,6 +316,8 @@ github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f h1:y5//uYreIhSUg3J1GEMiLbxo1LJaP8RfCpH6pymGZus= github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+o7JKHSa8/e818NopupXU1YMK5fe1lsApnBw= +github.com/ohler55/ojg v1.26.7 h1:yZLS2xlZF/qk5LHM4LFhxxTDyMgZl+46Z6p7wQm8KAU= +github.com/ohler55/ojg v1.26.7/go.mod h1:/Y5dGWkekv9ocnUixuETqiL58f+5pAsUfg5P8e7Pa2o= github.com/onsi/ginkgo/v2 v2.20.1 h1:YlVIbqct+ZmnEph770q9Q7NVAz4wwIiVNahee6JyUzo= github.com/onsi/ginkgo/v2 v2.20.1/go.mod h1:lG9ey2Z29hR41WMVthyJBGUBcBhGOtoPF2VFMvBXFCI= github.com/onsi/gomega v1.36.0 h1:Pb12RlruUtj4XUuPUqeEWc6j5DkVVVA49Uf6YLfC95Y= @@ -417,8 +419,8 @@ github.com/werf/3p-helm v0.0.0-20250609150428-130783e0dc18/go.mod h1:KDjmOsjFiOmj0fB0+q+0gGvlejPMjTgckLC59bX0BLg= github.com/werf/common-go v0.0.0-20250520111308-b0eda28dde0d h1:nVN0E4lQdToFiPty19uwj5TF+bCI/kAp5LLG4stWdO4= github.com/werf/common-go v0.0.0-20250520111308-b0eda28dde0d/go.mod h1:taKDUxKmGfqNOlVx1O0ad5vdV4duKexTLO7Rch9HfeA= -github.com/werf/kubedog v0.13.1-0.20250710125425-f736fad4b7b7 h1:HtmEewxpEkGRFMOInKPIflRO7HRSqMBEZ0EjTQ8sX4k= -github.com/werf/kubedog v0.13.1-0.20250710125425-f736fad4b7b7/go.mod h1:Y6pesrIN5uhFKqmHnHSoeW4jmVyZlWPFWv5SjB0rUPg= +github.com/werf/kubedog v0.13.1-0.20250710181210-b4a5a7f76b11 h1:9aZ8CjaczcO6Ez9T25DSPBE5EtmeBFyKdbwmvwBcvjM= +github.com/werf/kubedog v0.13.1-0.20250710181210-b4a5a7f76b11/go.mod h1:Y6pesrIN5uhFKqmHnHSoeW4jmVyZlWPFWv5SjB0rUPg= github.com/werf/lockgate v0.1.1 h1:S400JFYjtWfE4i4LY9FA8zx0fMdfui9DPrBiTciCrx4= github.com/werf/lockgate v0.1.1/go.mod h1:0yIFSLq9ausy6ejNxF5uUBf/Ib6daMAfXuCaTMZJzIE= github.com/werf/logboek v0.6.1 h1:oEe6FkmlKg0z0n80oZjLplj6sXcBeLleCkjfOOZEL2g= diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/nelm-1.7.2/internal/plan/calculate_planned_changes.go new/nelm-1.8.0/internal/plan/calculate_planned_changes.go --- old/nelm-1.7.2/internal/plan/calculate_planned_changes.go 2025-07-10 15:34:42.000000000 +0200 +++ new/nelm-1.8.0/internal/plan/calculate_planned_changes.go 2025-07-11 16:18:26.000000000 +0200 @@ -122,7 +122,7 @@ func hookResourcesChanges(infos []*info.DeployableHookResourceInfo, prevRelFailed bool, releaseName, releaseNamespace string) (changes []any, present bool) { for _, info := range infos { isCrd := util.IsCRDFromGK(info.ResourceID.GroupVersionKind().GroupKind()) - isSensitive := resource.IsSensitive(info.ResourceID.GroupVersionKind().GroupKind(), info.Resource().Unstructured().GetAnnotations()) + sensitiveInfo := resource.GetSensitiveInfo(info.ResourceID.GroupVersionKind().GroupKind(), info.Resource().Unstructured().GetAnnotations()) create := info.ShouldCreate() recreate := info.ShouldRecreate() update := info.ShouldUpdate() @@ -134,8 +134,13 @@ var uDiff string if isCrd { uDiff = HiddenInsignificantOutput - } else if isSensitive { - uDiff = HiddenSensitiveOutput + } else if sensitiveInfo.IsSensitive { + if len(sensitiveInfo.SensitivePaths) == 1 && sensitiveInfo.SensitivePaths[0] == resource.HideAll { + uDiff = HiddenSensitiveOutput + } else { + redactedResource := resource.RedactSensitiveData(info.Resource().Unstructured(), sensitiveInfo.SensitivePaths) + uDiff = lo.Must(util.ColoredUnifiedDiff("", diffableResource(redactedResource))) + } } else { uDiff = lo.Must(util.ColoredUnifiedDiff("", diffableResource(info.Resource().Unstructured()))) } @@ -150,8 +155,13 @@ var uDiff string if isCrd { uDiff = HiddenInsignificantOutput - } else if isSensitive { - uDiff = HiddenSensitiveOutput + } else if sensitiveInfo.IsSensitive { + if len(sensitiveInfo.SensitivePaths) == 1 && sensitiveInfo.SensitivePaths[0] == resource.HideAll { + uDiff = HiddenSensitiveOutput + } else { + redactedResource := resource.RedactSensitiveData(info.Resource().Unstructured(), sensitiveInfo.SensitivePaths) + uDiff = lo.Must(util.ColoredUnifiedDiff("", diffableResource(redactedResource))) + } } else { uDiff = lo.Must(util.ColoredUnifiedDiff("", diffableResource(info.Resource().Unstructured()))) } @@ -164,14 +174,28 @@ }) } else if update { var uDiff string - if ud, nonEmpty := util.ColoredUnifiedDiff(diffableResource(info.LiveResource().Unstructured()), diffableResource(info.DryApplyResource().Unstructured())); nonEmpty { - if isSensitive { - uDiff = HiddenSensitiveChanges + if sensitiveInfo.IsSensitive { + if len(sensitiveInfo.SensitivePaths) == 1 && sensitiveInfo.SensitivePaths[0] == resource.HideAll { + if _, nonEmpty := util.ColoredUnifiedDiff(diffableResource(info.LiveResource().Unstructured()), diffableResource(info.DryApplyResource().Unstructured())); nonEmpty { + uDiff = HiddenSensitiveChanges + } else { + uDiff = HiddenInsignificantChanges + } } else { - uDiff = ud + redactedLive := resource.RedactSensitiveData(info.LiveResource().Unstructured(), sensitiveInfo.SensitivePaths) + redactedNew := resource.RedactSensitiveData(info.DryApplyResource().Unstructured(), sensitiveInfo.SensitivePaths) + if ud, nonEmpty := util.ColoredUnifiedDiff(diffableResource(redactedLive), diffableResource(redactedNew)); nonEmpty { + uDiff = ud + } else { + uDiff = HiddenInsignificantChanges + } } } else { - uDiff = HiddenInsignificantChanges + if ud, nonEmpty := util.ColoredUnifiedDiff(diffableResource(info.LiveResource().Unstructured()), diffableResource(info.DryApplyResource().Unstructured())); nonEmpty { + uDiff = ud + } else { + uDiff = HiddenInsignificantChanges + } } changes = append(changes, &UpdatedResourceChange{ @@ -184,8 +208,13 @@ var uDiff string if isCrd { uDiff = HiddenInsignificantOutput - } else if isSensitive { - uDiff = HiddenSensitiveOutput + } else if sensitiveInfo.IsSensitive { + if len(sensitiveInfo.SensitivePaths) == 1 && sensitiveInfo.SensitivePaths[0] == resource.HideAll { + uDiff = HiddenSensitiveOutput + } else { + redactedResource := resource.RedactSensitiveData(info.Resource().Unstructured(), sensitiveInfo.SensitivePaths) + uDiff = lo.Must(util.ColoredUnifiedDiff("", diffableResource(redactedResource))) + } } else { uDiff = lo.Must(util.ColoredUnifiedDiff("", diffableResource(info.Resource().Unstructured()))) } @@ -205,7 +234,7 @@ func generalResourcesChanges(infos []*info.DeployableGeneralResourceInfo, prevRelFailed bool, releaseName, releaseNamespace string) (changes []any, present bool) { for _, info := range infos { isCrd := util.IsCRDFromGK(info.ResourceID.GroupVersionKind().GroupKind()) - isSensitive := resource.IsSensitive(info.ResourceID.GroupVersionKind().GroupKind(), info.Resource().Unstructured().GetAnnotations()) + sensitiveInfo := resource.GetSensitiveInfo(info.ResourceID.GroupVersionKind().GroupKind(), info.Resource().Unstructured().GetAnnotations()) create := info.ShouldCreate() recreate := info.ShouldRecreate() update := info.ShouldUpdate() @@ -217,8 +246,13 @@ var uDiff string if isCrd { uDiff = HiddenInsignificantOutput - } else if isSensitive { - uDiff = HiddenSensitiveOutput + } else if sensitiveInfo.IsSensitive { + if len(sensitiveInfo.SensitivePaths) == 1 && sensitiveInfo.SensitivePaths[0] == resource.HideAll { + uDiff = HiddenSensitiveOutput + } else { + redactedResource := resource.RedactSensitiveData(info.Resource().Unstructured(), sensitiveInfo.SensitivePaths) + uDiff = lo.Must(util.ColoredUnifiedDiff("", diffableResource(redactedResource))) + } } else { uDiff = lo.Must(util.ColoredUnifiedDiff("", diffableResource(info.Resource().Unstructured()))) } @@ -233,8 +267,13 @@ var uDiff string if isCrd { uDiff = HiddenInsignificantOutput - } else if isSensitive { - uDiff = HiddenSensitiveOutput + } else if sensitiveInfo.IsSensitive { + if len(sensitiveInfo.SensitivePaths) == 1 && sensitiveInfo.SensitivePaths[0] == resource.HideAll { + uDiff = HiddenSensitiveOutput + } else { + redactedResource := resource.RedactSensitiveData(info.Resource().Unstructured(), sensitiveInfo.SensitivePaths) + uDiff = lo.Must(util.ColoredUnifiedDiff("", diffableResource(redactedResource))) + } } else { uDiff = lo.Must(util.ColoredUnifiedDiff("", diffableResource(info.Resource().Unstructured()))) } @@ -247,14 +286,28 @@ }) } else if update { var uDiff string - if ud, nonEmpty := util.ColoredUnifiedDiff(diffableResource(info.LiveResource().Unstructured()), diffableResource(info.DryApplyResource().Unstructured())); nonEmpty { - if isSensitive { - uDiff = HiddenSensitiveChanges + if sensitiveInfo.IsSensitive { + if len(sensitiveInfo.SensitivePaths) == 1 && sensitiveInfo.SensitivePaths[0] == resource.HideAll { + if _, nonEmpty := util.ColoredUnifiedDiff(diffableResource(info.LiveResource().Unstructured()), diffableResource(info.DryApplyResource().Unstructured())); nonEmpty { + uDiff = HiddenSensitiveChanges + } else { + uDiff = HiddenInsignificantChanges + } } else { - uDiff = ud + redactedLive := resource.RedactSensitiveData(info.LiveResource().Unstructured(), sensitiveInfo.SensitivePaths) + redactedNew := resource.RedactSensitiveData(info.DryApplyResource().Unstructured(), sensitiveInfo.SensitivePaths) + if ud, nonEmpty := util.ColoredUnifiedDiff(diffableResource(redactedLive), diffableResource(redactedNew)); nonEmpty { + uDiff = ud + } else { + uDiff = HiddenInsignificantChanges + } } } else { - uDiff = HiddenInsignificantChanges + if ud, nonEmpty := util.ColoredUnifiedDiff(diffableResource(info.LiveResource().Unstructured()), diffableResource(info.DryApplyResource().Unstructured())); nonEmpty { + uDiff = ud + } else { + uDiff = HiddenInsignificantChanges + } } changes = append(changes, &UpdatedResourceChange{ @@ -267,8 +320,13 @@ var uDiff string if isCrd { uDiff = HiddenInsignificantOutput - } else if isSensitive { - uDiff = HiddenSensitiveOutput + } else if sensitiveInfo.IsSensitive { + if len(sensitiveInfo.SensitivePaths) == 1 && sensitiveInfo.SensitivePaths[0] == resource.HideAll { + uDiff = HiddenSensitiveOutput + } else { + redactedResource := resource.RedactSensitiveData(info.Resource().Unstructured(), sensitiveInfo.SensitivePaths) + uDiff = lo.Must(util.ColoredUnifiedDiff("", diffableResource(redactedResource))) + } } else { uDiff = lo.Must(util.ColoredUnifiedDiff("", diffableResource(info.Resource().Unstructured()))) } @@ -288,15 +346,20 @@ func prevReleaseGeneralResourcesChanges(infos []*info.DeployablePrevReleaseGeneralResourceInfo, curReleaseExistResourcesUIDs []types.UID, releaseName, releaseNamespace string, deployType common.DeployType) (changes []any, present bool) { for _, info := range infos { isCrd := util.IsCRDFromGK(info.ResourceID.GroupVersionKind().GroupKind()) - isSensitive := resource.IsSensitive(info.ResourceID.GroupVersionKind().GroupKind(), info.Resource().Unstructured().GetAnnotations()) + sensitiveInfo := resource.GetSensitiveInfo(info.ResourceID.GroupVersionKind().GroupKind(), info.Resource().Unstructured().GetAnnotations()) delete := info.ShouldDelete(curReleaseExistResourcesUIDs, releaseName, releaseNamespace, deployType) if delete { var uDiff string if isCrd { uDiff = HiddenInsignificantOutput - } else if isSensitive { - uDiff = HiddenSensitiveOutput + } else if sensitiveInfo.IsSensitive { + if len(sensitiveInfo.SensitivePaths) == 1 && sensitiveInfo.SensitivePaths[0] == resource.HideAll { + uDiff = HiddenSensitiveOutput + } else { + redactedResource := resource.RedactSensitiveData(info.LiveResource().Unstructured(), sensitiveInfo.SensitivePaths) + uDiff = lo.Must(util.ColoredUnifiedDiff(diffableResource(redactedResource), "")) + } } else { uDiff = lo.Must(util.ColoredUnifiedDiff(diffableResource(info.LiveResource().Unstructured()), "")) } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/nelm-1.7.2/internal/resource/common.go new/nelm-1.8.0/internal/resource/common.go --- old/nelm-1.7.2/internal/resource/common.go 2025-07-10 15:34:42.000000000 +0200 +++ new/nelm-1.8.0/internal/resource/common.go 2025-07-11 16:18:26.000000000 +0200 @@ -9,8 +9,9 @@ "strings" "time" + "github.com/ohler55/ojg/jp" "github.com/samber/lo" - "k8s.io/api/core/v1" + v1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" @@ -165,8 +166,10 @@ ) var ( - annotationKeyHumanSensitive = "werf.io/sensitive" - annotationKeyPatternSensitive = regexp.MustCompile(`^werf.io/sensitive$`) + annotationKeyHumanSensitive = "werf.io/sensitive" + annotationKeyPatternSensitive = regexp.MustCompile(`^werf.io/sensitive$`) + annotationKeyHumanSensitivePaths = "werf.io/sensitive-paths" + annotationKeyPatternSensitivePaths = regexp.MustCompile(`^werf.io/sensitive-paths$`) ) func validateHook(res *unstructured.Unstructured) error { @@ -686,6 +689,26 @@ } } + if key, value, found := FindAnnotationOrLabelByKeyPattern(unstruct.GetAnnotations(), annotationKeyPatternSensitivePaths); found { + if value == "" { + return fmt.Errorf("invalid value %q for annotation %q, expected non-empty comma-separated list of JSONPath strings", value, key) + } + + paths := ParseSensitivePaths(value) + if len(paths) == 0 { + return fmt.Errorf("invalid value %q for annotation %q, expected non-empty comma-separated list of JSONPath strings", value, key) + } + for _, path := range paths { + if strings.TrimSpace(path) == "" { + return fmt.Errorf("invalid value %q for annotation %q, JSONPath cannot be empty", value, key) + } + + if _, err := jp.ParseString(path); err != nil { + return fmt.Errorf("invalid JSONPath expression %q in annotation %q: %v", path, key, err) + } + } + } + return nil } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/nelm-1.7.2/internal/resource/sensitive.go new/nelm-1.8.0/internal/resource/sensitive.go --- old/nelm-1.7.2/internal/resource/sensitive.go 1970-01-01 01:00:00.000000000 +0100 +++ new/nelm-1.8.0/internal/resource/sensitive.go 2025-07-11 16:18:26.000000000 +0200 @@ -0,0 +1,163 @@ +package resource + +import ( + "crypto/sha256" + "encoding/json" + "fmt" + "strconv" + "strings" + + "github.com/ohler55/ojg/jp" + "github.com/samber/lo" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime/schema" + + "github.com/werf/nelm/pkg/featgate" +) + +const ( + HideAll = "$$HIDE_ALL$$" +) + +type SensitiveInfo struct { + IsSensitive bool + SensitivePaths []string +} + +func IsSensitive(groupKind schema.GroupKind, annotations map[string]string) bool { + info := GetSensitiveInfo(groupKind, annotations) + return info.IsSensitive +} + +func GetSensitiveInfo(groupKind schema.GroupKind, annotations map[string]string) SensitiveInfo { + // Check for werf.io/sensitive-paths (comma-separated) + if _, value, found := FindAnnotationOrLabelByKeyPattern(annotations, annotationKeyPatternSensitivePaths); found { + paths := ParseSensitivePaths(value) + if len(paths) > 0 { + return SensitiveInfo{IsSensitive: true, SensitivePaths: paths} + } + } + + useNewBehavior := featgate.FeatGateFieldSensitive.Enabled() || featgate.FeatGatePreviewV2.Enabled() + + // Check for werf.io/sensitive annotation + if _, value, found := FindAnnotationOrLabelByKeyPattern(annotations, annotationKeyPatternSensitive); found { + sensitive := lo.Must(strconv.ParseBool(value)) + if sensitive { + if useNewBehavior { + // V2 behavior: only hide data.* and stringData.* + return SensitiveInfo{IsSensitive: true, SensitivePaths: []string{"data.*", "stringData.*"}} + } else { + // V1 behavior: hide everything + return SensitiveInfo{IsSensitive: true, SensitivePaths: []string{HideAll}} + } + } else { + return SensitiveInfo{IsSensitive: false, SensitivePaths: nil} + } + } + + // Default behavior for Secrets + if groupKind == (schema.GroupKind{Group: "", Kind: "Secret"}) { + if useNewBehavior { + // V2 behavior: only hide data.* and stringData.* + return SensitiveInfo{IsSensitive: true, SensitivePaths: []string{"data.*", "stringData.*"}} + } else { + // V1 behavior: hide everything + return SensitiveInfo{IsSensitive: true, SensitivePaths: []string{HideAll}} + } + } + + return SensitiveInfo{IsSensitive: false, SensitivePaths: nil} +} + +func ParseSensitivePaths(value string) []string { + if strings.TrimSpace(value) == "" { + return nil + } + + var paths []string + var current strings.Builder + escaped := false + + for _, r := range value { + if escaped { + current.WriteRune(r) + escaped = false + } else if r == '\\' { + escaped = true + } else if r == ',' { + if path := strings.TrimSpace(current.String()); path != "" { + paths = append(paths, path) + } + current.Reset() + } else { + current.WriteRune(r) + } + } + + if path := strings.TrimSpace(current.String()); path != "" { + paths = append(paths, path) + } + + return paths +} + +func RedactSensitiveData(unstruct *unstructured.Unstructured, sensitivePaths []string) *unstructured.Unstructured { + copy := unstruct.DeepCopy() + + if len(sensitivePaths) == 0 { + return copy + } + + return redactSensitiveData(copy, sensitivePaths) +} + +func redactSensitiveData(unstruct *unstructured.Unstructured, sensitivePaths []string) *unstructured.Unstructured { + for _, pathExpr := range sensitivePaths { + if pathExpr == HideAll { + return &unstructured.Unstructured{Object: map[string]interface{}{ + "apiVersion": unstruct.GetAPIVersion(), + "kind": unstruct.GetKind(), + "metadata": map[string]interface{}{ + "name": unstruct.GetName(), + "namespace": unstruct.GetNamespace(), + }, + }} + } + + x := lo.Must(jp.ParseString(pathExpr)) + redactAtJSONPath(unstruct.Object, &x) + } + + return unstruct +} + +func redactAtJSONPath(obj map[string]interface{}, jsonPath *jp.Expr) { + jsonPath.MustModify(obj, func(element interface{}) (interface{}, bool) { + return createSensitiveReplacement(element), true + }) +} + +func createSensitiveReplacement(value interface{}) interface{} { + switch v := value.(type) { + case string: + hash := fmt.Sprintf("%x", sha256.Sum256([]byte(v)))[:12] + return fmt.Sprintf("SENSITIVE (%d bytes, %s)", len(v), hash) + case []byte: + hash := fmt.Sprintf("%x", sha256.Sum256(v))[:12] + return fmt.Sprintf("SENSITIVE (%d bytes, %s)", len(v), hash) + case []interface{}: + jsonData, _ := json.Marshal(v) + hash := fmt.Sprintf("%x", sha256.Sum256(jsonData))[:12] + return fmt.Sprintf("SENSITIVE (%d entries, %s)", len(v), hash) + case map[string]interface{}: + jsonData, _ := json.Marshal(v) + hash := fmt.Sprintf("%x", sha256.Sum256(jsonData))[:12] + return fmt.Sprintf("SENSITIVE (%d entries, %s)", len(v), hash) + default: + // For other types, convert to string and hash + str := fmt.Sprintf("%v", v) + hash := fmt.Sprintf("%x", sha256.Sum256([]byte(str)))[:12] + return fmt.Sprintf("SENSITIVE (%d bytes, %s)", len(str), hash) + } +} diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/nelm-1.7.2/internal/resource/util.go new/nelm-1.8.0/internal/resource/util.go --- old/nelm-1.7.2/internal/resource/util.go 2025-07-10 15:34:42.000000000 +0200 +++ new/nelm-1.8.0/internal/resource/util.go 2025-07-11 16:18:26.000000000 +0200 @@ -2,30 +2,12 @@ import ( "regexp" - "strconv" "strings" "github.com/samber/lo" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" - "k8s.io/apimachinery/pkg/runtime/schema" ) -func IsSensitive(groupKind schema.GroupKind, annotations map[string]string) bool { - if _, value, found := FindAnnotationOrLabelByKeyPattern(annotations, annotationKeyPatternSensitive); found { - sensitive := lo.Must(strconv.ParseBool(value)) - - if sensitive { - return true - } - } - - if groupKind == (schema.GroupKind{Group: "", Kind: "Secret"}) { - return true - } - - return false -} - func IsHook(annotations map[string]string) bool { _, _, found := FindAnnotationOrLabelByKeyPattern(annotations, annotationKeyPatternHook) return found diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/nelm-1.7.2/pkg/action/util.go new/nelm-1.8.0/pkg/action/util.go --- old/nelm-1.7.2/pkg/action/util.go 2025-07-10 15:34:42.000000000 +0200 +++ new/nelm-1.8.0/pkg/action/util.go 2025-07-11 16:18:26.000000000 +0200 @@ -2,6 +2,7 @@ import ( "context" + "flag" "fmt" "io" "io/ioutil" @@ -41,15 +42,21 @@ case SilentLogLevel, ErrorLogLevel, WarningLogLevel, InfoLogLevel: stdlog.SetOutput(io.Discard) - klog.SetOutputBySeverity("FATAL", ioutil.Discard) - klog.SetOutputBySeverity("ERROR", ioutil.Discard) - klog.SetOutputBySeverity("WARNING", ioutil.Discard) - klog.SetOutputBySeverity("INFO", ioutil.Discard) - - klogv2.SetOutputBySeverity("FATAL", ioutil.Discard) - klogv2.SetOutputBySeverity("ERROR", ioutil.Discard) - klogv2.SetOutputBySeverity("WARNING", ioutil.Discard) - klogv2.SetOutputBySeverity("INFO", ioutil.Discard) + klog.SetOutput(io.Discard) + // From: https://github.com/kubernetes/klog/issues/87#issuecomment-1671820147 + klogFlags := &flag.FlagSet{} + klog.InitFlags(klogFlags) + klogFlags.Set("logtostderr", "false") + klogFlags.Set("alsologtostderr", "false") + klogFlags.Set("stderrthreshold", "4") + + klogv2.SetOutput(io.Discard) + // From: https://github.com/kubernetes/klog/issues/87#issuecomment-1671820147 + klogV2Flags := &flag.FlagSet{} + klogv2.InitFlags(klogV2Flags) + klogV2Flags.Set("logtostderr", "false") + klogV2Flags.Set("alsologtostderr", "false") + klogV2Flags.Set("stderrthreshold", "4") logrus.SetOutput(ioutil.Discard) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/nelm-1.7.2/pkg/featgate/feat.go new/nelm-1.8.0/pkg/featgate/feat.go --- old/nelm-1.7.2/pkg/featgate/feat.go 2025-07-10 15:34:42.000000000 +0200 +++ new/nelm-1.8.0/pkg/featgate/feat.go 2025-07-11 16:18:26.000000000 +0200 @@ -32,6 +32,11 @@ `Use the new "release uninstall" command implementation (not fully backwards compatible)`, ) + FeatGateFieldSensitive = NewFeatGate( + "field-sensitive", + `Enable JSONPath-based selective sensitive field redaction`, + ) + FeatGatePreviewV2 = NewFeatGate( "preview-v2", `Active all feature gates that will be enabled by default in Nelm v2`, diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/nelm-1.7.2/test/sensitive_test.go new/nelm-1.8.0/test/sensitive_test.go --- old/nelm-1.7.2/test/sensitive_test.go 1970-01-01 01:00:00.000000000 +0100 +++ new/nelm-1.8.0/test/sensitive_test.go 2025-07-11 16:18:26.000000000 +0200 @@ -0,0 +1,929 @@ +package test + +import ( + "os" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime/schema" + + "github.com/werf/nelm/internal/resource" + "github.com/werf/nelm/pkg/featgate" +) + +func TestGetSensitiveInfo(t *testing.T) { + // Save original env and restore after test + originalEnv := os.Getenv(featgate.FeatGateFieldSensitive.EnvVarName()) + defer func() { + if originalEnv != "" { + os.Setenv(featgate.FeatGateFieldSensitive.EnvVarName(), originalEnv) + } else { + os.Unsetenv(featgate.FeatGateFieldSensitive.EnvVarName()) + } + }() + + tests := []struct { + name string + enableFeature bool + groupKind schema.GroupKind + annotations map[string]string + expected resource.SensitiveInfo + }{ + { + name: "regular resource not sensitive", + enableFeature: true, + groupKind: schema.GroupKind{Group: "apps", Kind: "Deployment"}, + annotations: map[string]string{}, + expected: resource.SensitiveInfo{IsSensitive: false, SensitivePaths: nil}, + }, + { + name: "secret resource automatically sensitive - legacy behavior", + enableFeature: false, + groupKind: schema.GroupKind{Group: "", Kind: "Secret"}, + annotations: map[string]string{}, + expected: resource.SensitiveInfo{IsSensitive: true, SensitivePaths: []string{"$$HIDE_ALL$$"}}, + }, + { + name: "secret resource with annotation - legacy behavior", + enableFeature: false, + groupKind: schema.GroupKind{Group: "", Kind: "Secret"}, + annotations: map[string]string{ + "werf.io/sensitive": "false", + }, + expected: resource.SensitiveInfo{IsSensitive: false, SensitivePaths: nil}, + }, + { + name: "secret with sensitive annotation set to false", + enableFeature: true, + groupKind: schema.GroupKind{Group: "", Kind: "Secret"}, + annotations: map[string]string{ + "werf.io/sensitive": "false", + }, + expected: resource.SensitiveInfo{IsSensitive: false, SensitivePaths: nil}, + }, + { + name: "secret resource automatically sensitive - new behavior", + enableFeature: true, + groupKind: schema.GroupKind{Group: "", Kind: "Secret"}, + annotations: map[string]string{}, + expected: resource.SensitiveInfo{IsSensitive: true, SensitivePaths: []string{"data.*", "stringData.*"}}, + }, + { + name: "resource with sensitive annotation set to true - legacy behavior", + enableFeature: false, + groupKind: schema.GroupKind{Group: "apps", Kind: "Deployment"}, + annotations: map[string]string{ + "werf.io/sensitive": "true", + }, + expected: resource.SensitiveInfo{IsSensitive: true, SensitivePaths: []string{resource.HideAll}}, + }, + { + name: "resource with sensitive annotation set to true - new behavior", + enableFeature: true, + groupKind: schema.GroupKind{Group: "apps", Kind: "Deployment"}, + annotations: map[string]string{ + "werf.io/sensitive": "true", + }, + expected: resource.SensitiveInfo{IsSensitive: true, SensitivePaths: []string{"data.*", "stringData.*"}}, + }, + { + name: "resource with comma-separated sensitive-paths annotation", + enableFeature: true, + groupKind: schema.GroupKind{Group: "apps", Kind: "Deployment"}, + annotations: map[string]string{ + "werf.io/sensitive-paths": "spec.template.spec.containers.*.env.*.value,data.password", + }, + expected: resource.SensitiveInfo{IsSensitive: true, SensitivePaths: []string{"spec.template.spec.containers.*.env.*.value", "data.password"}}, + }, + { + name: "resource with escaped comma in sensitive-paths", + enableFeature: true, + groupKind: schema.GroupKind{Group: "apps", Kind: "Deployment"}, + annotations: map[string]string{ + "werf.io/sensitive-paths": "data.field\\,with\\,commas,spec.other", + }, + expected: resource.SensitiveInfo{IsSensitive: true, SensitivePaths: []string{"data.field,with,commas", "spec.other"}}, + }, + { + name: "resource with both sensitive and sensitive-paths annotations - sensitive path precedence in v2", + enableFeature: true, + groupKind: schema.GroupKind{Group: "apps", Kind: "Deployment"}, + annotations: map[string]string{ + "werf.io/sensitive": "true", + "werf.io/sensitive-paths": "data.password", + }, + expected: resource.SensitiveInfo{IsSensitive: true, SensitivePaths: []string{"data.password"}}, + }, + { + name: "resource with empty sensitive-paths annotation", + enableFeature: true, + groupKind: schema.GroupKind{Group: "apps", Kind: "Deployment"}, + annotations: map[string]string{ + "werf.io/sensitive-paths": "", + }, + expected: resource.SensitiveInfo{IsSensitive: false, SensitivePaths: nil}, + }, + { + name: "resource with sensitive-paths annotation - feature flag disabled", + enableFeature: false, + groupKind: schema.GroupKind{Group: "v1", Kind: "ConfigMap"}, + annotations: map[string]string{ + "werf.io/sensitive-paths": "$.data[*]", + }, + expected: resource.SensitiveInfo{IsSensitive: true, SensitivePaths: []string{"$.data[*]"}}, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + // Set feature gate + if tt.enableFeature { + os.Setenv(featgate.FeatGateFieldSensitive.EnvVarName(), "true") + } else { + os.Unsetenv(featgate.FeatGateFieldSensitive.EnvVarName()) + } + + result := resource.GetSensitiveInfo(tt.groupKind, tt.annotations) + + assert.Equal(t, tt.expected, result, "behavior should match expected") + }) + } +} + +func TestParseSensitivePaths(t *testing.T) { + tests := []struct { + name string + input string + expected []string + }{ + { + name: "empty string", + input: "", + expected: nil, + }, + { + name: "single path", + input: "data.password", + expected: []string{"data.password"}, + }, + { + name: "multiple paths", + input: "data.password,spec.template.spec.containers.*.env.*.value", + expected: []string{"data.password", "spec.template.spec.containers.*.env.*.value"}, + }, + { + name: "paths with spaces", + input: " data.password , spec.template ", + expected: []string{"data.password", "spec.template"}, + }, + { + name: "escaped commas", + input: "data.field\\,with\\,commas,spec.other", + expected: []string{"data.field,with,commas", "spec.other"}, + }, + { + name: "multiple escaped commas", + input: "data.a\\,b\\,c,spec.d\\,e,metadata.f", + expected: []string{"data.a,b,c", "spec.d,e", "metadata.f"}, + }, + { + name: "trailing comma", + input: "data.password,spec.template,", + expected: []string{"data.password", "spec.template"}, + }, + { + name: "empty segments", + input: "data.password,,spec.template", + expected: []string{"data.password", "spec.template"}, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + result := resource.ParseSensitivePaths(tt.input) + assert.Equal(t, tt.expected, result) + }) + } +} + +func TestRedactSensitiveData(t *testing.T) { + // Save original env and restore after test + originalEnv := os.Getenv(featgate.FeatGateFieldSensitive.EnvVarName()) + defer func() { + if originalEnv != "" { + os.Setenv(featgate.FeatGateFieldSensitive.EnvVarName(), originalEnv) + } else { + os.Unsetenv(featgate.FeatGateFieldSensitive.EnvVarName()) + } + }() + + tests := []struct { + name string + enableFeature bool + input *unstructured.Unstructured + sensitivePaths []string + checkFunc func(t *testing.T, result *unstructured.Unstructured) + }{ + { + name: "bug: sensitive-paths ignored when feature flag disabled", + enableFeature: false, + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "apiVersion": "v1", + "kind": "ConfigMap", + "metadata": map[string]interface{}{ + "name": "test-config", + "namespace": "default", + }, + "data": map[string]interface{}{ + "key1": "sensitive-value-1", + "key2": "sensitive-value-2", + }, + }, + }, + sensitivePaths: []string{"data.*"}, + checkFunc: func(t *testing.T, result *unstructured.Unstructured) { + // The bug is that when feature flag is disabled, the entire data section + // is removed instead of redacting only the specified sensitive paths + data, found, err := unstructured.NestedMap(result.Object, "data") + require.NoError(t, err) + + if !found { + t.Errorf("data section was completely removed instead of being redacted") + } else { + // If data exists, it should be redacted + for key, value := range data { + valueStr, ok := value.(string) + if ok && !strings.Contains(valueStr, "SENSITIVE") { + t.Errorf("Expected data.%s to be redacted but got: %s", key, valueStr) + } + } + } + }, + }, + { + name: "no sensitive paths", + enableFeature: true, + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "apiVersion": "v1", + "kind": "Secret", + "metadata": map[string]interface{}{ + "name": "test-secret", + "namespace": "default", + }, + "data": map[string]interface{}{ + "username": "dXNlcm5hbWU=", + "password": "cGFzc3dvcmQ=", + }, + }, + }, + sensitivePaths: []string{}, + checkFunc: func(t *testing.T, result *unstructured.Unstructured) { + data := result.Object["data"].(map[string]interface{}) + assert.Equal(t, "dXNlcm5hbWU=", data["username"]) + assert.Equal(t, "cGFzc3dvcmQ=", data["password"]) + }, + }, + { + name: "hide all with feature gate", + enableFeature: true, + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "apiVersion": "v1", + "kind": "Secret", + "metadata": map[string]interface{}{ + "name": "test-secret", + "namespace": "default", + }, + "data": map[string]interface{}{ + "username": "dXNlcm5hbWU=", + "password": "cGFzc3dvcmQ=", + }, + }, + }, + sensitivePaths: []string{resource.HideAll}, + checkFunc: func(t *testing.T, result *unstructured.Unstructured) { + assert.Equal(t, "v1", result.Object["apiVersion"]) + assert.Equal(t, "Secret", result.Object["kind"]) + metadata := result.Object["metadata"].(map[string]interface{}) + assert.Equal(t, "test-secret", metadata["name"]) + assert.Equal(t, "default", metadata["namespace"]) + assert.NotContains(t, result.Object, "data") + }, + }, + { + name: "redact data fields with wildcard - new behavior", + enableFeature: true, + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "apiVersion": "v1", + "kind": "Secret", + "metadata": map[string]interface{}{ + "name": "test-secret", + "namespace": "default", + }, + "data": map[string]interface{}{ + "username": "dXNlcm5hbWU=", + "password": "cGFzc3dvcmQ=", + }, + "type": "Opaque", + }, + }, + sensitivePaths: []string{"data.*"}, + checkFunc: func(t *testing.T, result *unstructured.Unstructured) { + data := result.Object["data"].(map[string]interface{}) + usernameVal := data["username"].(string) + passwordVal := data["password"].(string) + + // Check that values are replaced with SENSITIVE format + assert.Contains(t, usernameVal, "SENSITIVE") + assert.Contains(t, usernameVal, "12 bytes") + assert.Contains(t, passwordVal, "SENSITIVE") + assert.Contains(t, passwordVal, "12 bytes") + + // Check that type field is preserved + assert.Equal(t, "Opaque", result.Object["type"]) + }, + }, + { + name: "redact specific field", + enableFeature: true, + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "apiVersion": "v1", + "kind": "Secret", + "metadata": map[string]interface{}{ + "name": "test-secret", + "namespace": "default", + }, + "data": map[string]interface{}{ + "username": "dXNlcm5hbWU=", + "password": "cGFzc3dvcmQ=", + }, + }, + }, + sensitivePaths: []string{"data.password"}, + checkFunc: func(t *testing.T, result *unstructured.Unstructured) { + data := result.Object["data"].(map[string]interface{}) + + // Username should be unchanged + assert.Equal(t, "dXNlcm5hbWU=", data["username"]) + + // Password should be redacted + passwordVal := data["password"].(string) + assert.Contains(t, passwordVal, "SENSITIVE") + assert.Contains(t, passwordVal, "12 bytes") + }, + }, + { + name: "type change handling - string to slice", + enableFeature: true, + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "apiVersion": "apps/v1", + "kind": "Deployment", + "spec": map[string]interface{}{ + "template": map[string]interface{}{ + "spec": map[string]interface{}{ + "containers": []interface{}{ + map[string]interface{}{ + "name": "app", + "env": []interface{}{ + map[string]interface{}{ + "name": "CONFIG", + "value": []interface{}{"item1", "item2", "item3"}, + }, + }, + }, + }, + }, + }, + }, + }, + }, + sensitivePaths: []string{"spec.template.spec.containers[0].env[0].value"}, + checkFunc: func(t *testing.T, result *unstructured.Unstructured) { + spec := result.Object["spec"].(map[string]interface{}) + template := spec["template"].(map[string]interface{}) + templateSpec := template["spec"].(map[string]interface{}) + containers := templateSpec["containers"].([]interface{}) + container := containers[0].(map[string]interface{}) + env := container["env"].([]interface{}) + envVar := env[0].(map[string]interface{}) + + valueStr := envVar["value"].(string) + assert.Contains(t, valueStr, "SENSITIVE") + assert.Contains(t, valueStr, "entries") + }, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + // Set feature gate + if tt.enableFeature { + os.Setenv(featgate.FeatGateFieldSensitive.EnvVarName(), "true") + } else { + os.Unsetenv(featgate.FeatGateFieldSensitive.EnvVarName()) + } + + result := resource.RedactSensitiveData(tt.input, tt.sensitivePaths) + + // Ensure original object is not modified + assert.NotSame(t, tt.input, result, "Original object should not be modified") + + tt.checkFunc(t, result) + }) + } +} + +func TestSHA256HashingConsistency(t *testing.T) { + // Enable feature gate + originalEnv := os.Getenv(featgate.FeatGateFieldSensitive.EnvVarName()) + defer func() { + if originalEnv != "" { + os.Setenv(featgate.FeatGateFieldSensitive.EnvVarName(), originalEnv) + } else { + os.Unsetenv(featgate.FeatGateFieldSensitive.EnvVarName()) + } + }() + os.Setenv(featgate.FeatGateFieldSensitive.EnvVarName(), "true") + + input1 := &unstructured.Unstructured{ + Object: map[string]interface{}{ + "data": map[string]interface{}{ + "password": "secret123", + }, + }, + } + + input2 := &unstructured.Unstructured{ + Object: map[string]interface{}{ + "data": map[string]interface{}{ + "password": "secret123", + }, + }, + } + + result1 := resource.RedactSensitiveData(input1, []string{"data.password"}) + result2 := resource.RedactSensitiveData(input2, []string{"data.password"}) + + data1 := result1.Object["data"].(map[string]interface{}) + data2 := result2.Object["data"].(map[string]interface{}) + + // Same input should produce same hash + assert.Equal(t, data1["password"], data2["password"], "Same input should produce same redacted output") + + // Different inputs should produce different hashes + input3 := &unstructured.Unstructured{ + Object: map[string]interface{}{ + "data": map[string]interface{}{ + "password": "different123", + }, + }, + } + + result3 := resource.RedactSensitiveData(input3, []string{"data.password"}) + data3 := result3.Object["data"].(map[string]interface{}) + + assert.NotEqual(t, data1["password"], data3["password"], "Different inputs should produce different redacted outputs") +} + +func TestRedactAtJSONPath(t *testing.T) { + // Enable feature gate + originalEnv := os.Getenv(featgate.FeatGateFieldSensitive.EnvVarName()) + defer func() { + if originalEnv != "" { + os.Setenv(featgate.FeatGateFieldSensitive.EnvVarName(), originalEnv) + } else { + os.Unsetenv(featgate.FeatGateFieldSensitive.EnvVarName()) + } + }() + os.Setenv(featgate.FeatGateFieldSensitive.EnvVarName(), "true") + + tests := []struct { + name string + input *unstructured.Unstructured + sensitivePaths []string + checkFunc func(t *testing.T, result *unstructured.Unstructured) + }{ + { + name: "nested object paths", + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "apiVersion": "v1", + "kind": "ConfigMap", + "data": map[string]interface{}{ + "config.yaml": "password: secret123\nuser: admin", + "app.conf": "db_password=mysecret", + }, + "metadata": map[string]interface{}{ + "name": "test-config", + }, + }, + }, + sensitivePaths: []string{"data['config.yaml']"}, + checkFunc: func(t *testing.T, result *unstructured.Unstructured) { + data := result.Object["data"].(map[string]interface{}) + configYaml := data["config.yaml"].(string) + appConf := data["app.conf"].(string) + + assert.Contains(t, configYaml, "SENSITIVE") + assert.Contains(t, configYaml, "bytes") + assert.Equal(t, "db_password=mysecret", appConf) // unchanged + }, + }, + { + name: "array elements with wildcard", + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "spec": map[string]interface{}{ + "containers": []interface{}{ + map[string]interface{}{ + "name": "app1", + "image": "nginx:latest", + "env": []interface{}{ + map[string]interface{}{ + "name": "PASSWORD", + "value": "secret123", + }, + map[string]interface{}{ + "name": "USER", + "value": "admin", + }, + }, + }, + map[string]interface{}{ + "name": "app2", + "image": "alpine:latest", + "env": []interface{}{ + map[string]interface{}{ + "name": "DB_PASSWORD", + "value": "dbsecret456", + }, + }, + }, + }, + }, + }, + }, + sensitivePaths: []string{"spec.containers.*.env.*.value"}, + checkFunc: func(t *testing.T, result *unstructured.Unstructured) { + spec := result.Object["spec"].(map[string]interface{}) + containers := spec["containers"].([]interface{}) + + // Check first container + container1 := containers[0].(map[string]interface{}) + env1 := container1["env"].([]interface{}) + env1_0 := env1[0].(map[string]interface{}) + env1_1 := env1[1].(map[string]interface{}) + + assert.Contains(t, env1_0["value"].(string), "SENSITIVE") + assert.Contains(t, env1_1["value"].(string), "SENSITIVE") + + // Check second container + container2 := containers[1].(map[string]interface{}) + env2 := container2["env"].([]interface{}) + env2_0 := env2[0].(map[string]interface{}) + + assert.Contains(t, env2_0["value"].(string), "SENSITIVE") + + // Verify names are unchanged + assert.Equal(t, "PASSWORD", env1_0["name"]) + assert.Equal(t, "USER", env1_1["name"]) + assert.Equal(t, "DB_PASSWORD", env2_0["name"]) + }, + }, + { + name: "specific array index", + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "data": map[string]interface{}{ + "items": []interface{}{ + "public_item", + "secret_item", + "another_public_item", + }, + }, + }, + }, + sensitivePaths: []string{"data.items[1]"}, + checkFunc: func(t *testing.T, result *unstructured.Unstructured) { + data := result.Object["data"].(map[string]interface{}) + items := data["items"].([]interface{}) + + assert.Equal(t, "public_item", items[0]) + assert.Contains(t, items[1].(string), "SENSITIVE") + assert.Equal(t, "another_public_item", items[2]) + }, + }, + { + name: "complex nested structure", + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "spec": map[string]interface{}{ + "template": map[string]interface{}{ + "spec": map[string]interface{}{ + "volumes": []interface{}{ + map[string]interface{}{ + "name": "config-volume", + "configMap": map[string]interface{}{ + "name": "my-config", + }, + }, + map[string]interface{}{ + "name": "secret-volume", + "secret": map[string]interface{}{ + "secretName": "my-secret", + "items": []interface{}{ + map[string]interface{}{ + "key": "password", + "path": "db/password", + }, + map[string]interface{}{ + "key": "username", + "path": "db/username", + }, + }, + }, + }, + }, + }, + }, + }, + }, + }, + sensitivePaths: []string{"spec.template.spec.volumes[1].secret.items.*.key"}, + checkFunc: func(t *testing.T, result *unstructured.Unstructured) { + spec := result.Object["spec"].(map[string]interface{}) + template := spec["template"].(map[string]interface{}) + templateSpec := template["spec"].(map[string]interface{}) + volumes := templateSpec["volumes"].([]interface{}) + + // First volume should be unchanged + volume0 := volumes[0].(map[string]interface{}) + assert.Equal(t, "config-volume", volume0["name"]) + + // Second volume's secret items keys should be redacted + volume1 := volumes[1].(map[string]interface{}) + secret := volume1["secret"].(map[string]interface{}) + items := secret["items"].([]interface{}) + + item0 := items[0].(map[string]interface{}) + item1 := items[1].(map[string]interface{}) + + assert.Contains(t, item0["key"].(string), "SENSITIVE") + assert.Contains(t, item1["key"].(string), "SENSITIVE") + + // Paths should remain unchanged + assert.Equal(t, "db/password", item0["path"]) + assert.Equal(t, "db/username", item1["path"]) + }, + }, + { + name: "mixed data types", + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "data": map[string]interface{}{ + "stringValue": "sensitive_string", + "intValue": "42", + "boolValue": "true", + "arrayValue": []interface{}{"item1", "item2", "item3"}, + "objectValue": map[string]interface{}{ + "nested": "nested_value", + }, + }, + }, + }, + sensitivePaths: []string{"data.*"}, + checkFunc: func(t *testing.T, result *unstructured.Unstructured) { + data := result.Object["data"].(map[string]interface{}) + + // All values should be redacted with appropriate SENSITIVE format + for key, value := range data { + valueStr := value.(string) + assert.Contains(t, valueStr, "SENSITIVE", "Key %s should be redacted", key) + + switch key { + case "stringValue": + assert.Contains(t, valueStr, "16 bytes") + case "intValue": + assert.Contains(t, valueStr, "2 bytes") // "42" + case "boolValue": + assert.Contains(t, valueStr, "4 bytes") // "true" + case "arrayValue": + assert.Contains(t, valueStr, "3 entries") + case "objectValue": + assert.Contains(t, valueStr, "1 entries") + } + } + }, + }, + { + name: "recursive descent pattern", + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "level1": map[string]interface{}{ + "password": "secret1", + "level2": map[string]interface{}{ + "password": "secret2", + "level3": map[string]interface{}{ + "password": "secret3", + "other": "public", + }, + }, + }, + "password": "root_secret", + }, + }, + sensitivePaths: []string{"$..password"}, + checkFunc: func(t *testing.T, result *unstructured.Unstructured) { + // Root level password + rootPassword := result.Object["password"].(string) + assert.Contains(t, rootPassword, "SENSITIVE") + + // Level 1 password + level1 := result.Object["level1"].(map[string]interface{}) + level1Password := level1["password"].(string) + assert.Contains(t, level1Password, "SENSITIVE") + + // Level 2 password + level2 := level1["level2"].(map[string]interface{}) + level2Password := level2["password"].(string) + assert.Contains(t, level2Password, "SENSITIVE") + + // Level 3 password + level3 := level2["level3"].(map[string]interface{}) + level3Password := level3["password"].(string) + assert.Contains(t, level3Password, "SENSITIVE") + + // Other field should be unchanged + assert.Equal(t, "public", level3["other"]) + }, + }, + { + name: "multiple separate paths", + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "spec": map[string]interface{}{ + "database": map[string]interface{}{ + "password": "db_secret", + "host": "localhost", + }, + "redis": map[string]interface{}{ + "auth": "redis_secret", + "port": "6379", + }, + }, + "data": map[string]interface{}{ + "api_key": "api_secret", + "config": "public_config", + }, + }, + }, + sensitivePaths: []string{"spec.database.password", "spec.redis.auth", "data.api_key"}, + checkFunc: func(t *testing.T, result *unstructured.Unstructured) { + spec := result.Object["spec"].(map[string]interface{}) + database := spec["database"].(map[string]interface{}) + redis := spec["redis"].(map[string]interface{}) + data := result.Object["data"].(map[string]interface{}) + + // Sensitive fields should be redacted + assert.Contains(t, database["password"].(string), "SENSITIVE") + assert.Contains(t, redis["auth"].(string), "SENSITIVE") + assert.Contains(t, data["api_key"].(string), "SENSITIVE") + + // Non-sensitive fields should remain unchanged + assert.Equal(t, "localhost", database["host"]) + assert.Equal(t, "6379", redis["port"]) + assert.Equal(t, "public_config", data["config"]) + }, + }, + { + name: "specific array indices", + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "items": []interface{}{ + "item0", + "item1_secret", + "item2_secret", + "item3_secret", + "item4", + }, + }, + }, + sensitivePaths: []string{"items[1]", "items[2]", "items[3]"}, + checkFunc: func(t *testing.T, result *unstructured.Unstructured) { + items := result.Object["items"].([]interface{}) + + // Items 0 and 4 should be unchanged + assert.Equal(t, "item0", items[0]) + assert.Equal(t, "item4", items[4]) + + // Items 1, 2, 3 should be redacted + for i := 1; i <= 3; i++ { + assert.Contains(t, items[i].(string), "SENSITIVE") + } + }, + }, + { + name: "empty and nil values", + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "data": map[string]interface{}{ + "empty_string": "", + "nil_value": nil, + "empty_array": []interface{}{}, + "empty_object": map[string]interface{}{}, + }, + }, + }, + sensitivePaths: []string{"data.*"}, + checkFunc: func(t *testing.T, result *unstructured.Unstructured) { + data := result.Object["data"].(map[string]interface{}) + + // All values should be redacted, even empty ones + for key, value := range data { + if key == "nil_value" { + // nil values get converted to string representation + assert.Contains(t, value.(string), "SENSITIVE") + assert.Contains(t, value.(string), "bytes") + } else { + valueStr := value.(string) + assert.Contains(t, valueStr, "SENSITIVE", "Key %s should be redacted", key) + } + } + }, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + result := resource.RedactSensitiveData(tt.input, tt.sensitivePaths) + + // Ensure original object is not modified + assert.NotSame(t, tt.input, result, "Original object should not be modified") + + tt.checkFunc(t, result) + }) + } +} + +func TestRedactSensitiveDataEdgeCases(t *testing.T) { + // Enable feature gate + originalEnv := os.Getenv(featgate.FeatGateFieldSensitive.EnvVarName()) + defer func() { + if originalEnv != "" { + os.Setenv(featgate.FeatGateFieldSensitive.EnvVarName(), originalEnv) + } else { + os.Unsetenv(featgate.FeatGateFieldSensitive.EnvVarName()) + } + }() + os.Setenv(featgate.FeatGateFieldSensitive.EnvVarName(), "true") + + tests := []struct { + name string + input *unstructured.Unstructured + sensitivePaths []string + expectNoChange bool + }{ + { + name: "non-existent path should not error", + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "data": map[string]interface{}{ + "password": "secret123", + }, + }, + }, + sensitivePaths: []string{"nonexistent.field"}, + expectNoChange: true, + }, + { + name: "path to non-existent array index", + input: &unstructured.Unstructured{ + Object: map[string]interface{}{ + "items": []interface{}{"item1", "item2"}, + }, + }, + sensitivePaths: []string{"items[10]"}, + expectNoChange: true, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + originalData := tt.input.DeepCopy() + result := resource.RedactSensitiveData(tt.input, tt.sensitivePaths) + + if tt.expectNoChange { + // Compare the data sections to verify no changes + assert.Equal(t, originalData.Object, result.Object, "Data should remain unchanged for invalid paths") + } + }) + } +} diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/nelm-1.7.2/trdl_channels.yaml new/nelm-1.8.0/trdl_channels.yaml --- old/nelm-1.7.2/trdl_channels.yaml 2025-07-10 15:34:42.000000000 +0200 +++ new/nelm-1.8.0/trdl_channels.yaml 2025-07-11 16:18:26.000000000 +0200 @@ -2,7 +2,7 @@ - name: "1" channels: - name: alpha - version: 1.7.1 + version: 1.7.2 - name: beta version: 1.7.0 - name: ea ++++++ nelm.obsinfo ++++++ --- /var/tmp/diff_new_pack.AkZE1F/_old 2025-07-14 10:56:00.963362706 +0200 +++ /var/tmp/diff_new_pack.AkZE1F/_new 2025-07-14 10:56:00.967362871 +0200 @@ -1,5 +1,5 @@ name: nelm -version: 1.7.2 -mtime: 1752154482 -commit: 50af53676602ce74ed9e6f0bd4a4df62bae061ad +version: 1.8.0 +mtime: 1752243506 +commit: c40b3aaefe124b57d9d43b0cccc0f6f58bd77646 ++++++ vendor.tar.gz ++++++ /work/SRC/openSUSE:Factory/nelm/vendor.tar.gz /work/SRC/openSUSE:Factory/.nelm.new.7373/vendor.tar.gz differ: char 12, line 1