dabla commented on PR #61535: URL: https://github.com/apache/airflow/pull/61535#issuecomment-3896608097
> > I'm just wondering, not sure if possible, but lineage is a cross cutting concern, so would a decorator not be a viable solution for this? > > Yes, lineage as a cross-cutting concern is a common concept, and there are multiple ways to implement it. In this case of Hook Level Lineage, we can send assets directly via `add_input_asset` and `add_output_asset`, but each schema has its own asset factory and different kwargs to create asset uri. We also use `add_extra` for things like tracking SQL executions - but since different databases can expose different arguments, the tracking isn’t uniform. > > A decorator could work, but I think the current approach is clearer and more explicit. Given that this helper is limited to SQL-based hooks, there’s no strong need for a global decorator in my opinion. > > If you have some example of how it could look like so that it's easier to implement or easier to read lmk, I'm open to changing the approach, I just feel like there is not much to gain from decorator here - it'd be the same code, just implemented differently, and with explicit call we have full control over when we execute it - after cursor execution, before connection closed, and f.e. just once for `insert_rows` when cursor is called multiple times. Thank you for your reply and explanation. Indeed it’s not as easy as it sounds, I would certainly not change the current implementation as I like the work you did on this PR. This was just a thought of mine that maybe we should think of in the future to possibly make it more available out of the. I wasn’t already thinking on all hooks in general but purely on the one based on the DBApiHook, which would be a challenge already. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
