det101 opened a new pull request, #18329: URL: https://github.com/apache/dolphinscheduler/pull/18329
<!--Thanks very much for contributing to Apache DolphinScheduler, we are happy that you want to help us improve DolphinScheduler! --> ## Was this PR generated or assisted by AI? YES, The code was generated with AI assistance. <!--(Please answer YES or NO. If YES, please specify which parts were generated or assisted by AI)--> ## Purpose of the pull request This PR implements [DSIP-105 / #17937](https://github.com/apache/dolphinscheduler/issues/17937): support for sensitive workflow variables (global and local parameters). Users can mark a parameter as sensitive=true. The system then: masks values as ****** in API/UI responses encrypts sensitive values before persisting to DB (reusing PasswordUtils and datasource.encryption.enable) merges ****** on save/update to keep the existing secret decrypts sensitive values at runtime so tasks receive real values in prepareParamsMap masks sensitive values in task stdout logs (task-scoped dynamic masking with cleanup after task finish) Closes #17937. <!--(For example: This pull request adds checkstyle plugin).--> ## Brief change log - Add Property.sensitive flag (default false, backward compatible) - Add PropertySensitiveUtils for masking and ****** placeholder merge (no crypto dependency) - Add SensitivePropertyUtils (API) and MasterSensitivePropertyUtils (Master) for encrypt/decrypt and API masking - Mask sensitive values in workflow definition/instance/task query APIs - Encrypt on create/update; merge ****** before persist - Decrypt in RunWorkflowCommandHandler, TaskExecutionContextFactory, and CuringParamsServiceImpl - Register/clear dynamic log mask patterns in PhysicalTaskExecutor - Stop logging full TaskExecutorDispatchRequest in Worker dispatch logs (avoid leaking prepareParamsMap) - UI: add sensitive checkbox for global/local params; show ****** in variable views - Tests: unit tests + SensitiveWorkflowVariableAPITest E2E <!--*(for example:)* - *Add maven-checkstyle-plugin to root pom.xml* --> ## Verify this pull request <!--*(Please pick either of the following options)*--> This change added tests and can be verified as follows: ### Unit tests ``` ./mvnw -pl dolphinscheduler-task-plugin/dolphinscheduler-task-api test \ -Dtest=PropertySensitiveUtilsTest,SensitiveDataConverterTest -Djacoco.skip=true ./mvnw -pl dolphinscheduler-api test \ -Dtest=SensitivePropertyUtilsTest,WorkflowDefinitionServiceTest#testCreateWorkflowDefinitionShouldEncryptSensitiveParamsBeforeSaving,WorkflowDefinitionServiceTest#testUpdateWorkflowDefinitionShouldKeepOldSensitiveValueWhenPlaceholderSubmitted,WorkflowDefinitionServiceTest#testViewVariablesShouldMaskSensitiveGlobalAndLocalParams,WorkflowDefinitionServiceTest#testQueryWorkflowDefinitionByCodeShouldMaskSensitiveParams,WorkflowDefinitionServiceTest#testQueryWorkflowDefinitionListPagingShouldMaskSensitiveGlobalParams,WorkflowInstanceServiceTest#testViewVariablesShouldMaskSensitiveGlobalAndLocalParams -Djacoco.skip=true ./mvnw -pl dolphinscheduler-service test \ -Dtest=CuringParamsServiceImplTest#testParamParsingPreparationShouldKeepGlobalValueWhenSensitiveCommandParamIsPlaceholder -Djacoco.skip=true ``` ### API integration tests (standalone) ``` mvn -f dolphinscheduler-api-test/pom.xml -pl dolphinscheduler-api-test-core install -DskipTests -Djacoco.skip=true mvn -f dolphinscheduler-api-test/pom.xml -pl dolphinscheduler-api-test-case \ test -Dtest=SensitiveWorkflowVariableAPITest -Dlocal=true -Djacoco.skip=true ``` ### Manual verification 1. Create a workflow with a sensitive global parameter. 2. Query the definition — value shows ******, not plaintext. 3. Update with ****** unchanged — run again, task still succeeds with the original secret. 4. Check task log — sensitive output is masked; subsequent non-sensitive tasks do not retain stale masks. > ## Pull Request Notice [Pull Request Notice](https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md) If your pull request contains incompatible change, you should also add it to `docs/docs/en/guide/upgrade/incompatible.md` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
