Internally, we have blocked people from using SubDags due to its slowness and sometimes instability. I would vote for adding TaskGroups earliest as possible + remove documentation for SubDags (users can go to old versions of docs to find that) + add a notice of deprecation on new TaskGroup documentation page + Add deprecation warning on import/usage SubDag.
I would vote for not having to wait until 3.X for deprecating it as it's difficult to use and adds complexity that may not be needed. Instead schedule removal at 2.1 or so. This is the take from our side that we don't have SubDags, would be curious on people usages of SubDags and to check if there's any use case that does not fit TaskGroups. Gerard Casas Saez Twitter | Cortex | @casassaez <http://twitter.com/casassaez> On Fri, Sep 18, 2020 at 6:34 AM Kaxil Naik <kaxiln...@apache.org> wrote: > Hi all, > > One of the things we discussed on Monday's Airflow 2.0 Dev call was around > *TaskGroups* - a new concept introduced by > https://github.com/apache/airflow/pull/10153 (AIP-34 > < > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-34+TaskGroup%3A+A+UI+task+grouping+concept+as+an+alternative+to+SubDagOperator > >) > from > Qian Yu. > > There was a point raised in the meeting that we should deprecate SubDags in > favour of TaskGroups since TaskGroups are UI-only concept and do not impact > scheduling decisions or have known limitations around Concurrency / Pools > limits like SubDags. > > Having thought. a little bit more after the meeting, I think we should > hold-off on Deprecating Subdags in 2.0. We can wait until 2.2-2.3 and see > how users feel about TaskGroup and we, the developers would also know more > about limitations of it (if any). > > We can then issue a Deprecation Warning and remove SubDags eventually in > Airflow 3.0. Until then both can live in the codebase and we document the > TaskGroups better. > > What do you all think? > > Meeting Notes from that call are at: > > https://cwiki.apache.org/confluence/display/AIRFLOW/Meeting+Notes#MeetingNotes-#4:14Sep2020 > <https://cwiki.apache.org/confluence/display/AIRFLOW/Meeting+Notes#MeetingNotes-%234:14Sep2020> > < > https://cwiki.apache.org/confluence/display/AIRFLOW/Meeting+Notes#MeetingNotes-%234:14Sep2020 > > > > Regards, > Kaxil >
