bamaer opened a new issue, #9500: URL: https://github.com/apache/hop/issues/9500
### What would you like to happen? Five gaps found while using `hop lint` output as audit evidence (Hop 2.20.0). Each part can be a separate PR that references this issue. - [ ] 1. SARIF declares every enabled rule, and the run's invocation - [ ] 2. SARIF `shortDescription` of native verify rules (bug) - [ ] 3. A ServiceLoader rule pack cannot declare overrides (bug) - [ ] 4. SEC-002 and SEC-003 miss secrets that Hop stores as passwords - [ ] 5. Option conditions for map fields; connection `url`, transform `category`, workflow `actionsWithoutFailurePath` #### 1. SARIF: declare every enabled rule, and the invocation `LintReportWriter.renderSarif` builds `tool.driver.rules` from the findings only (`LintReportWriter.java:151-189`), and writes no `invocations`. A rule that ran and found nothing looks the same as one that was disabled or not installed. Example: a two-transform pipeline declares 3 rules in SARIF; `hop lint --list-rules` shows 10 enabled. Done when a run with no findings still declares every enabled rule (matching `--list-rules`), `runs[0].invocations` has `executionSuccessful` and the configuration (config file, severity, baseline, pack ids and versions), and the Run Linter action's SARIF does the same. #### 2. Bug: native verify rules are described by the first transform that reported them A native remark's rule name is its source transform (`LintCheckResultAdapter.java:148-149`), and `renderSarif` writes the first finding's rule name as `shortDescription.text` (`LintReportWriter.java:171-172`). Two Table output transforms without a connection, "Load customers" and "Load orders": rule `CONNECTION_NOT_ASSIGNED` gets the description `Load customers`, and code-scanning tools label every finding with it. Done when no `shortDescription` comes from a transform or action name (`HOP-CHECK` uses its name in `hop-lint-core.yml`), with a test. #### 3. Bug: a ServiceLoader rule pack cannot declare overrides `AbstractYamlRulePack` does not implement `getOverrides()` (`AbstractYamlRulePack.java:23-65`), so the `pack.overrides:` in its YAML is ignored and `RuleRegistry.mergePack` refuses the pack's version of a core rule as an undeclared collision. `FileYamlRulePack` returns them correctly (`FileYamlRulePack.java:55-56`). Done when an `AbstractYamlRulePack` subclass that declares `overrides:` replaces the named rules, with a unit test next to the `FileYamlRulePack` one. #### 4. SEC-002 and SEC-003 miss secrets that Hop stores as passwords The rules match field names only (`CustomRuleExecutor.java:1128-1168`). Not matched, although stored with `password = true`: `sasKey` (Azure Event Hubs Writer and Listener), `privateKey` (Beam Snowflake Input and Output), `keyFilePass` (FTP Delete action). Also the Event Hubs Listener's `storageConnectionString`, which holds the storage account key. Proposed: also report any String field stored with `password = true` that holds something other than a variable expression, and add `privateKey` and `connectionString` to `fieldPatterns`. Beam Kinesis marks `streamName` as a password and would start reporting; check that annotation in the same change. #### 5. Option conditions and fields for TLS and failure-path rules - `HAS_OPTION` / `OPTION_MATCHES` for map fields such as a connection's `attributes` (accepting `sslmode` as well as `EXTRA_OPTION_POSTGRESQL.sslmode`). Today only a regular expression over the map's `toString()` works, which breaks on values with `,` or `}`. - `url` on database connections: the URL as Hop builds it, variables unresolved. - `category` on transforms (Input, Output, ...), language-independent. - `actionsWithoutFailurePath` on workflows: actions other than Start with an outgoing hop and none followed on failure. - Optional: `appliesTo` on a connection rule is compared with the metadata key `rdbms`, so `appliesTo: [POSTGRESQL]` silently matches nothing. Compare with the database type, or report it as a configuration error. Example: "PostgreSQL without TLS" today needs three `allOf` clauses with negative-lookahead regexes over `attributes` and `manualUrl`; with `OPTION_MATCHES` it is one clause: `conditionValue: "sslmode=(require|verify-ca|verify-full)"`. ### Issue Priority Priority: 2 ### Issue Component Component: CLI, Component: Other -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
