bamaer opened a new pull request, #9471:
URL: https://github.com/apache/hop/pull/9471

   Item 3 of #9323, on top of #9457.
   
   Mask fields gets a "Data catalog" box: a data catalog, a dataset and rules 
that map a tag or glossary term to a masking pattern. Every incoming field the 
catalog labels with that tag or term is masked with the rule's pattern. Rows in 
the Fields table still win.
   
   - The transform asks the catalog when it starts and logs which fields it 
masks. When the catalog can't answer, the dataset is missing, or a tag holds a 
variable that isn't set, the transform doesn't start: it never guesses.
   - Pipelines without tag rules behave as before. The metadata injection keys 
of the Fields table are unchanged; the tag rules use `tag_rule_tag` and 
`tag_rule_pattern`.
   - Engine: `ISchemaCapability`; `CatalogDataset` moves from the Check data 
catalog action to the engine so both can use it (same serialization keys).
   - DataHub reads tags and terms from the ingested schema and from the edits 
made in the DataHub UI. A term matches by name, URN or glossary path. Only 
fields present in DataHub 1.0 to 1.8 are queried.
   - Docs: Mask fields, Data catalog, dev manual.
   - Tests: unit tests, plus `integration-tests/datacatalog` case 0002 with a 
golden data set.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to