Re: [DISCUSS] Common.util provider?

2024-02-21 Thread Tzu-ping Chung
It would help in the sense mentioned in previous posts, yes. But one thing I want to point out is, for the provider to actually be helpful, it must be treated a bit differently from normal providers, but more like a separate third-party dependency. Specifically, the provider should not have a

Re: [DISCUSS] Considering trying out uv for our CI workflows

2024-02-21 Thread Amogh Desai
I agree with Niko here. If someone is willing to give it a try, we should enable it experimentally and give it a stint for a couple of weeks. If we see significant results, we can adopt it. Thanks & Regards, Amogh Desai On Thu, Feb 22, 2024 at 3:32 AM Oliveira, Niko wrote: > The Astral folks

RE: [DISCUSS] Common.util provider?

2024-02-21 Thread Scheffler Jens (XC-AS/EAE-ADA-T)
@Uranusjr would this help as a pilot in your AIP-60 code to parse and validate URIs for datasets? Mit freundlichen Grüßen / Best regards Jens Scheffler Alliance: Enabler - Tech Lead (XC-AS/EAE-ADA-T) Robert Bosch GmbH | Hessbruehlstraße 21 | 70565 Stuttgart-Vaihingen | GERMANY | www.bosch.com

Re: [DISCUSS] Common.util provider?

2024-02-21 Thread Jarek Potiuk
Yep. It could work with symbolic links. Tested it and with flit - both wheel and sdist packaged code such symbolically linked file is dereferenced and copy of the file is added there. It could be a nice way of doing it. Maybe then worth trying next time if someone has a need? J On Thu, Feb 22,

RE: [DISCUSS] Common.util provider?

2024-02-21 Thread Scheffler Jens (XC-AS/EAE-ADA-T)
>>> As of additional dependency complexity between providers actually the additional dependency I think creates more problems than the benefit… would be cool if there would be an option to „inline“ common code from a single place but keep individual providers fully independent… >Well, we

Re: [DISCUSS] Considering trying out uv for our CI workflows

2024-02-21 Thread Oliveira, Niko
The Astral folks also seem very focused on it being a drop-in/compliant replacement for pip. So I think it's definitely worth dropping it in and seeing if we get the expected performance improvements. If tests still pass and user facing constraints and install instructions remain unchanged I

Re: [DISCUSS] Common.util provider?

2024-02-21 Thread Jarek Potiuk
> if we have a common piece then we are locking all depending providers (potentially) together if common code changes Yes, coupling in this case is the drawback of this solution. You can't have cake and eat it too and in this case you trade DRY with coupling. > As of additional dependency

Re: [DISCUSS] Common.util provider?

2024-02-21 Thread Scheffler Jens (XC-AS/EAE-ADA-T)
Hi Jarek, At reviewing the PR from uranusjr for AIP-60 I also had the feeling that a lot of very similar code is repeated in all the providers. But during review yesterday I dropped the ides because if we have a common piece then we are locking all depending providers (potentially) together if

[DISCUSS] Common.util provider?

2024-02-21 Thread Jarek Potiuk
Hello everyone, How do we feel about introducing a common.util provider? I know it's not been the original idea behind providers, but - after testing common.sql and now also having common.io, seems like more and more we would like to extract some common code that we would like providers to use,

Re: [EXTERNAL] Re: [VOTE] [RESULT] New Airflow Community Provider: Teradata

2024-02-21 Thread K Mallam, Sunil
We understand, Jarek. We’ll look forward to your notification about the next wave with Teradata provider’s release. Also, we’ll make sure we plan our enhancements and releases based on the standard release process going forward. Thanks, Sunil K Mallam Staff Product Manager – 3rd Party

Re: [EXTERNAL] Re: [VOTE] [RESULT] New Airflow Community Provider: Teradata

2024-02-21 Thread Jarek Potiuk
> I’ve asked Elad about this and maybe I’ll ask here too, is there a possibility of an ad-hoc release this week? IMHO. Unless there is a good reason for it (for the community) then we do not do that. This is part of the price you pay for being a "community" provider, that it has to fit in the

RE: Re: [DISCUSS] Deprecate cli options in Airflow Configurations and airflow.api.client

2024-02-21 Thread Brianna
On 2024/02/19 10:34:44 Bolke de Bruin wrote: > with regards to the api client: the intention was indeed to remove direct > access to the DB, which I think is still a valid thing to do. In my opinion > the underlying question is what are we going to do with direct DB access > from the cli. Is that

Re: [EXTERNAL] Re: [VOTE] [RESULT] New Airflow Community Provider: Teradata

2024-02-21 Thread K Mallam, Sunil
Thanks for your response, Jarek. We’ll keep a watch on the system tests dashboard and make sure we run system tests for every release. Jared: They are released roughly every-two-weeks. Currently we are in the middle of RC2 from the previous wave, so the next wave for the Teradata provider is ~