I wanted to add one topic to the discussion: How viable it is to expect migration from 1.10 DB to 2.0 will working and whether we should recommend as a community to start Airflow Metadata from scratch, vs. supporting 1.10.13 -> 2.0 DB migration as a fully supported option.
We've been discussing it in the past meeting but I do not think we reached (and did not capture) any finl conclusion (and we might have different assumptions here). The https://github.com/apache/airflow/blob/master/UPGRADING_TO_2.0.md has one mention of "airflow db upgrade" ("As part of running db upgrade, existing permissions will be migrated for you."). But the DB migration step is not included in the document and I do not know of any efforts of testing a large-scale migration of existing Airflow 1.10 DB to 2.0. I know from past experiences that depending on databases and the amount of data in the DB such migrations can take hours or in extreme cases even days (depending on how optimized the migration scripts are). I think we need to discuss and capture at least community recommendations on that and make it clear in the migration docs. J. On Mon, Nov 30, 2020 at 3:33 PM Kaxil Naik <[email protected]> wrote: > Hi all, > > I would like to remind whoever is interested in helping in Airflow 2.0 > Release process that we will have our weekly developer call today: > > *Airflow 2.0 Dev Call Calendar Link*: > https://calendar.google.com/calendar/embed?src=c_dhdh3bdjc42c4ngtnpg7ovcs9g%40group.calendar.google.com&ctz=Europe%2FLondon > *Time*: 5:30-6:30 PM GMT > *Zoom Link*: > https://astronomer.zoom.us/j/93263978170?pwd=Ym9wMUJUNkc5cnJ0VnNabVRtaVBvZz09 > > > Some useful links: > > - Planning Doc & Meeting Notes: > https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0+-+Planning > - 2.0 Milestone: https://github.com/apache/airflow/milestone/3 > - Issue Tracker: https://github.com/apache/airflow/issues/10152 > > > Last Meeting call was short, the meeting notes for which is published at > https://cwiki.apache.org/confluence/display/AIRFLOW/Meeting+Notes#MeetingNotes-#14:23Nov2020 > <https://cwiki.apache.org/confluence/display/AIRFLOW/Meeting+Notes#MeetingNotes-%2314:23Nov2020>. > Airflow 1.10.13 and Upgrade-Check 1.0.0 were released on 25 Nov. > > 1.10.14 should hopefully be released this week, that fixes a bug in 1.10.13 > > Regards, > Kaxil > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
