Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread Kaxil Naik
I feel deprecating without having a replacement will not aid anything. "deprecations" are marked just to give users a warning and I feel we have enough warnings for the user in the API endpoint itself where we say it is "experimental". https://airflow.readthedocs.io/en/stable/rest-api-ref.html sa

Re: [PROPOSAL][AIP-36 DAG Versioning]

2020-08-13 Thread Jarek Potiuk
I fully agree with the "user" not having to know any of the "wheel' details. Similarly as they do not have to know python interpreter or the underlying libc library details. This all should be hidden from the users. I think the wheels API that we might have there, does not have to be user-facing.

Re: Simplifying the KubernetesExecutor

2020-08-13 Thread Jarek Potiuk
+1 for pod mutation in config. It's not a "yamly" thing - it's a python code to run, so it's quite appropriate to have it in config. On Wed, Aug 12, 2020 at 9:43 PM Daniel Imberman wrote: > I think pod_mutation is important as an “admin override”. E.g. there might > be a label that is required b

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread Jarek Potiuk
Yep I agree removal (especially now in 1.10) is not a good idea.That's fine not to remove it in 2.0 but disable by default. But I still think we should mark the whole API deprecated NOW (in 1.10.12). It would be great to have at least one release in 1.10. where we explicitly mark it as deprecated

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread Ry Walker
Yes I agree the new API would be made default, and experimental would be disabled by default. Before the new API goes live, we should allow minor improvements to the Experimental API (for example https://github.com/apache/airflow/pull/10308, which addresses to some degree the security hole in the

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread Kamil Breguła
I agree. We do not have to completely delete the experimental API in Airflow 2.0, but I think it is worth turning off by default so that the user has to make a conscious decision that they want to use the API, which provides a limited level of security- no permission control, all authorized users

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread QP Hou
Yeah, i am also curious to know more about the reason why we want to nuke the experimental api code soon instead of just marking it as deprecated. As for getting more insights into remote mode cli usage, would it make sense to make it part of the airflow user survey? Thanks, QP Hou On Thu, Aug

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread Ry Walker
I would think we would deprecate the old API once we say the new API is “ready to go” - and leave it in place a while as users transition to new API. Why is there an urgency to remove it from codebase? On Thu, Aug 13, 2020 at 5:46 AM Ash Berlin-Taylor wrote: > Removing the experimental is a fund

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread Ash Berlin-Taylor
Removing the experimental is a fundamental breaking change to users' workflows, and so we should remove it before 2.0. -ash On 13 August 2020 10:14:02 BST, Jarek Potiuk wrote: >And I think we should make the whole experimenta API deprecated in >1.10.12 >possibly ? > >On Thu, Aug 13, 2020 at 11:

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread Jarek Potiuk
And I think we should make the whole experimenta API deprecated in 1.10.12 possibly ? On Thu, Aug 13, 2020 at 11:12 AM Jarek Potiuk wrote: > I think no matter what, Maybe we should simply make it deprecated in the > upcoming (today?) release of 1.10.12 ? Then we can decide if in - > potential -

Re: [2.0 spring cleaning] Remove remote mode in Airflow CLI and in-core API Client

2020-08-13 Thread Jarek Potiuk
I think no matter what, Maybe we should simply make it deprecated in the upcoming (today?) release of 1.10.12 ? Then we can decide if in - potential - 1.10.13 we remove it or leave it as it is. On Wed, Aug 12, 2020 at 6:37 PM Kamil Breguła wrote: > I started this thread mainly to discuss what w