Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-18 Thread Jarek Potiuk
Fine for me. Deprecating might be deferred, I have no problem with that :). On Thu, Aug 13, 2020 at 11:08 PM Kaxil Naik wrote: > I feel deprecating without having a replacement will not aid anything. > > "deprecations" are marked just to give users a warning and I feel we have > enough warnings

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread Kaxil Naik
I feel deprecating without having a replacement will not aid anything. "deprecations" are marked just to give users a warning and I feel we have enough warnings for the user in the API endpoint itself where we say it is "experimental". https://airflow.readthedocs.io/en/stable/rest-api-ref.html sa

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread Jarek Potiuk
Yep I agree removal (especially now in 1.10) is not a good idea.That's fine not to remove it in 2.0 but disable by default. But I still think we should mark the whole API deprecated NOW (in 1.10.12). It would be great to have at least one release in 1.10. where we explicitly mark it as deprecated

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread Ry Walker
Yes I agree the new API would be made default, and experimental would be disabled by default. Before the new API goes live, we should allow minor improvements to the Experimental API (for example https://github.com/apache/airflow/pull/10308, which addresses to some degree the security hole in the

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread Kamil Breguła
I agree. We do not have to completely delete the experimental API in Airflow 2.0, but I think it is worth turning off by default so that the user has to make a conscious decision that they want to use the API, which provides a limited level of security- no permission control, all authorized users

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread QP Hou
Yeah, i am also curious to know more about the reason why we want to nuke the experimental api code soon instead of just marking it as deprecated. As for getting more insights into remote mode cli usage, would it make sense to make it part of the airflow user survey? Thanks, QP Hou On Thu, Aug

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread Ry Walker
I would think we would deprecate the old API once we say the new API is “ready to go” - and leave it in place a while as users transition to new API. Why is there an urgency to remove it from codebase? On Thu, Aug 13, 2020 at 5:46 AM Ash Berlin-Taylor wrote: > Removing the experimental is a fund

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread Ash Berlin-Taylor
Removing the experimental is a fundamental breaking change to users' workflows, and so we should remove it before 2.0. -ash On 13 August 2020 10:14:02 BST, Jarek Potiuk wrote: >And I think we should make the whole experimenta API deprecated in >1.10.12 >possibly ? > >On Thu, Aug 13, 2020 at 11:

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread Jarek Potiuk
And I think we should make the whole experimenta API deprecated in 1.10.12 possibly ? On Thu, Aug 13, 2020 at 11:12 AM Jarek Potiuk wrote: > I think no matter what, Maybe we should simply make it deprecated in the > upcoming (today?) release of 1.10.12 ? Then we can decide if in - > potential -

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread Jarek Potiuk
I think no matter what, Maybe we should simply make it deprecated in the upcoming (today?) release of 1.10.12 ? Then we can decide if in - potential - 1.10.13 we remove it or leave it as it is. On Wed, Aug 12, 2020 at 6:37 PM Kamil Breguła wrote: > I started this thread mainly to discuss what w

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-12 Thread Kamil Breguła
I started this thread mainly to discuss what we want to do with this remote mode prior to the Airflow 2.0 release. This is mainly due to the fact that he is using an experimental API which will be deprecated. In my opinion, we have several solutions. a) Delete this mode as unused and not supported

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-11 Thread QP Hou
I think it's best to divide the discussion into two separate topics. First one is to replace the existing json_client with the new to be created official Airflow Python Client backed by the new RESTful API. This IMHO is a must have considering we are to deprecate experimental API going forward. T

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-11 Thread Ash Berlin-Taylor
-1 from me without a firm plan how we will replace it. I see keeping it and extending to use the new API would ensure that everything the CLI can do locally (i.e. when airflow webserver isn't up yet, with the ) also works over the API with the exception of db utilities. -ash On 11 August 2020

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-11 Thread QP Hou
+1 for replacing the existing remote mode client with the open api based client. IMO, we don't really have other options here because the experimental API will be deprecated in the future. For OpenAPI based Airflow REST clients, the current plan is to maintain all the code gen automation within th

[2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-11 Thread Kamil Breguła
Hello, I think we should remove remote mode in CLI and in-core API Client (airflow.api.client package). Here is docs about remote mode: https://airflow.readthedocs.io/en/latest/usage-cli.html#set-up-connection-to-a-remote-airflow-instance Since these features were introduced, it has never been ac