Re: [DISCUSS] common.compat provider (WAS: Common.util provider?)

2024-06-13 Thread Jarek Potiuk
> The only known problem with that idea is that the common code has to live "forever" - as long as someone can use the older providers (or older Airflow version). Here Jakub is right - it's only 'older providers" not "older airlfow" - the provider will have "apache-airflow>=2.7.0" so older

Re: [DISCUSS] common.compat provider (WAS: Common.util provider?)

2024-06-10 Thread Amogh Desai
I like the idea of having a common.compat provider too. (common.util was actually kinda confusing) * when providers get >= airflow 2.10 - we change them to import from > `airflow.openlineage` rather than from "airflow.providers.common.compat". > I am not so sure why this is the case. Can you

Re: [DISCUSS] common.compat provider (WAS: Common.util provider?)

2024-06-10 Thread Jakub DardziƄski
As the author of https://github.com/apache/airflow/pull/39530 I love the idea. * when providers get >= airflow 2.10 - we change them to import from > `airflow.openlineage` rather than from "airflow.providers.common.compat". > What's the reasoning behind that? How would Airflow core release impact

Re: [DISCUSS] common.compat provider (WAS: Common.util provider?)

2024-06-10 Thread Maciej Obuchowski
I think it's a good solution. The only known problem with that idea is that the common code has to live "forever" - as long as someone can use the older providers (or older Airflow version). The solution would be to introduce some explicit deprecation or versioning for provider dependencies - but

Re: [DISCUSS] common.compat provider (WAS: Common.util provider?)

2024-06-08 Thread Jarek Potiuk
I have an idea about that one, and probably that one will fulfill the "polyfill" approach discussed earlier. I think we should not name the provider "common.util" but "common.compat" - because all the code that we need to put there is really about keeping compatibility. For example look here