Thank you all for your responses. Dennis, With respect to your concern on the "hard data - numbers" to support the assertion of the audience. I agree that we don't have the hard numbers at this time to support this assertion and I also understand your concern that there is additional burden to an already stretched community to maintain this. I will attempt to get some more data on numbers by running some experiments before bringing the more "maintenance heavy" elements back to the dev list.
Ping and others, I will also draft up a proposal for tagging features / issues with "production focus" vs. "dev focus" labels. I think we last reviewed the labels holistically and updated them right after the 2.0 release and a lot of time has passed since then. We can then use these to automate (as far as possible) the release note generation process, without putting undue tasks on the release managers. As always, any feedback is welcome! On Wed, Jul 13, 2022 at 9:07 PM Ping Zhang <pin...@umich.edu> wrote: > Hi Vikram, > > Great thoughts on this. > > I agree with this: > >> I am concerned that we are overwhelming our audience segment (1) with the >> work and configurations around running Airflow at scale. > > > I would like to see a better separation on 1) how to set up an airflow > cluster in production, how to monitor the health of an airflow cluster 2) > how to write better airflow DAGs. > > I also like the idea of segmenting the release notes for different groups. > > > Thanks, > > Ping > > > On Thu, Jun 30, 2022 at 8:50 PM Vikram Koka <vik...@astronomer.io.invalid> > wrote: > >> Hi everyone, >> >> As I have been looking through the recent AIPs, development features, and >> mailing list discussions, it struck me that we have effectively three >> different audiences here for Airflow. >> >> 1. Individuals and small teams using Airflow for their purpose, >> 2. Enterprises managing Airflow for large teams of data engineers and >> data scientists, and >> 3. Service providers making "Airflow as a service" available for many >> customers, either external or internal. >> >> Why does this even matter? Let me elaborate below: >> >> - Clearly, a lot of "data practitioners", people who are primarily >> focused on creating pipelines and working with data are spread across all >> three audiences above. >> - However, "Airflow administrators" i.e. people who are focused on >> running Airflow for data practitioners, especially at scale are primarily >> in the audiences (2) and (3) above. >> - It is my observation that a lot of work being done right now in >> Airflow such as multi-tenancy (but not limited to it), is focused on >> Airflow administration. >> - I am concerned that we are overwhelming our audience segment (1) >> with the work and configurations around running Airflow at scale. >> >> >> If this is true, I would like to propose that we segment our Airflow >> configurations, our packaging including our docs, and even our release >> notes to make it easier for our audience (1), who is almost certainly the >> largest block of our Airflow user community. >> >> I would like the opinions of the community on this topic. >> >> Best regards, >> Vikram >> >>
