Agree with Kaxil. Last run is.good. Actually if we are looking for Airflow 3, the target for 'readu for migration' will be just '0 everywhere' . Having last run and getting totals and seeing it goes to 0 steadily should be the 'ready for migration' target for anyone running Airflow 3.
pt., 14 cze 2024, 22:36 użytkownik Kaxil Naik <[email protected]> napisał: > Yeah, it is 30s by default; storing it in DB would be useless, apart from > the most recent run, in my opinion. > > On Fri, 14 Jun 2024 at 15:24, Constance Martineau > <[email protected]> wrote: > > > I love the idea. If we were to store it in the DB, would we keep a > history, > > or only the latest stats from the most recent dag parsing loop? DAG > parsing > > by default is every 30s right? > > > > On Fri, Jun 14, 2024 at 6:53 AM Jarek Potiuk <[email protected]> wrote: > > > > > > I think we still need to enable the ability for DAGs at parse time to > > > access Variables. > > > > > > It's actually DISCOURAGED ;) to access Variables at parse time - though > > we > > > have an experimental feature to make it efficient and we discussed > > whether > > > to treat it as "good practice" and there was strong opposition to that > > .... > > > So I am not actually sure what will be our position on it. > > > > > > On Fri, Jun 14, 2024 at 12:50 PM Ash Berlin-Taylor <[email protected]> > > wrote: > > > > > > > > > > > > > > > > On 14 Jun 2024, at 10:22, Jarek Potiuk <[email protected]> wrote: > > > > > > > > > > think in the future of Airflow 3 where > > > > > we will have task isolation, having `0` for all the DAGs will be a > > > > > prerequisite for switching to "task isolation" mode and this could > be > > > > > actually verified in a migration tool. > > > > > > > > I think we still need to enable the ability for DAGs at parse time to > > > > access Variables. > > > > > > > > Or at least I am not proposing we remove that ability. (I wouldn’t be > > > > against it, but I was planning on continuing to support that for now) > > > > > > > > -ash > > > > > >
