kacpermuda commented on PR #71312: URL: https://github.com/apache/airflow/pull/71312#issuecomment-5763245255
Thanks for the ping, still not convinced I have all the answers, but let's try to move this forward: > Should hook scope be limited to per-asset controls, i.e. reject rules combining it with emit etc. during validation (like the existing operator + emit_dag_events check)? Yes, I think we can remove such combinations now, we can always allow them later. Let's stick to what makes the most sense now. > And would you want hook_lineage: false allowed at hook scope as a nicer spelling of "drop everything from this hook"? I think hook_lineage and emit would mean the same? We can probably allow it, but I think it's ok to also clearly mark that `emit` is the only valid way and reject the other one. Up to you. > What should exclude_datasets patterns match against? Hook assets have Airflow URIs pre-translation while operator lineage is OL datasets (namespace + name), so should I match the translated OL identity everywhere so one pattern behaves the same across sources? I think we can do namespace + name here? Some URIs do not get translated anyway, so this way we'll be dropping only those that survived the translation. > Should it also filter manually annotated inlets/outlets, or only extractor and hook lineage? Hmm, good question. I think if it's easy we can apply it to all to be consistent. > For tier resolution, is replace-rather-than-merge (most specific matching rule wins per asset, so a task rule can narrow or clear a global list) the behaviour you'd expect? I think it's fine, as long as we honor the `freeze` of the global rule and do not replace then. We need to clearly mark this behavior in the docs, and maybe also log some info when this happens, just for clearer debugging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
