Re: [AIP-34] Rewrite SubDagOperator

2020-06-13 Thread Kaxil Naik
Thanks, Bin, The proposal looks good. As long as we can maintain the "zoom-in/zoom-out" *Grouping feature in the UI too I am +1 to this idea. The ">>" idea can happen separately I feel, doesn't need to be a part of this AIP. Also just an FYI: https://github.com/apache/airflow/pull/7685 i.e. previ

Re: [AIP-34] Rewrite SubDagOperator

2020-06-13 Thread Daniel Imberman
Yes that is exactly the thought. Treat a Subdag as a task that can “zoom in” on sub-tasks that are running. I really think this makes the most sense. The idea of running entire schedulers in a worker is kind of insane/opens up a lot of potential for complexity (e.g. if someone is using the Kube