bamaer opened a new pull request, #9471: URL: https://github.com/apache/hop/pull/9471
Item 3 of #9323, on top of #9457. Mask fields gets a "Data catalog" box: a data catalog, a dataset and rules that map a tag or glossary term to a masking pattern. Every incoming field the catalog labels with that tag or term is masked with the rule's pattern. Rows in the Fields table still win. - The transform asks the catalog when it starts and logs which fields it masks. When the catalog can't answer, the dataset is missing, or a tag holds a variable that isn't set, the transform doesn't start: it never guesses. - Pipelines without tag rules behave as before. The metadata injection keys of the Fields table are unchanged; the tag rules use `tag_rule_tag` and `tag_rule_pattern`. - Engine: `ISchemaCapability`; `CatalogDataset` moves from the Check data catalog action to the engine so both can use it (same serialization keys). - DataHub reads tags and terms from the ingested schema and from the edits made in the DataHub UI. A term matches by name, URN or glossary path. Only fields present in DataHub 1.0 to 1.8 are queried. - Docs: Mask fields, Data catalog, dev manual. - Tests: unit tests, plus `integration-tests/datacatalog` case 0002 with a golden data set. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
