Just an update to today's planning. I decided to push a little harder and I managed to complete the "good enough" version of provider packages split - ready for the Alpha tomorrow. I thoroughly tested it - including automated installation and import tests in CI as well as manual testing and releasing an example package to test PyPI https://test.pypi.org/project/apache-airflow-providers-zendesk/
The PR is here: https://github.com/apache/airflow/pull/11487 J. On Sun, Oct 11, 2020 at 9:32 PM Jarek Potiuk <jarek.pot...@polidea.com> wrote: > Hello Everyone, > > I have some proposal :). Since we are very close to releasing Alpha (keep > my fingers crossed). I thought that we can get a much better overview of > what is still left (at least for the people who are observing it closely), > by looking at the Github Issues/Milestones. rather than the Wiki document. > I am a big proponent of keeping everything in GitHub Issues (and some > recent changes in GitHub Projects/Issues make it really useful) > > I tried to organize the areas that I am involved in and created > several milestones in GitHub to reflect the planning. For one split the > "Package Providers" into separate issues and created a project for it: > https://github.com/apache/airflow/projects/5 - it has a number of issues > that can be addressed later, not holding the Alpha release - I will try to > address them till tomorrow "2.0 meeting". > > I've added a few milestones to reflect the planning: > https://github.com/apache/airflow/milestones and I reviewed my own issues > and assigned them as I think it makes sense (some to alpha1, some to > alpha2, some to beta and some only needed once we release final 2.0). > > I think that might help with the planning and execution of 2.0 in a > low-stress and high-motivation way. And we could discuss them tomorrow at > the 2.0 meeting quickly if we prepare and pre-assign the issues. > > Maybe others could do the same with their issues - to assign them to the > right milestone? > > This way we might use the 2.0 meeting time effectively and then decisions > on when we release which version might be more "data-driven" than > "wish-driven" :). > > J > > > > On Tue, Oct 6, 2020 at 12:30 PM Kaxil Naik <kaxiln...@apache.org> wrote: > >> Hi all, >> >> I have updated our meeting notes document to summarize the discussion >> from our dev call for Airflow 2.0 yesterday. >> >> Thank you all who joined the call. >> >> *Doc Link*: >> https://cwiki.apache.org/confluence/display/AIRFLOW/Meeting+Notes#MeetingNotes-#7:5Oct2020 >> <https://cwiki.apache.org/confluence/display/AIRFLOW/Meeting+Notes#MeetingNotes-%237:5Oct2020> >> >> To all those who attended, can you please double-check and add if I have >> missed anything? >> To all those who didn't join, if you disagree to anything in >> the Summary please voice your opinion. >> >> Also please let me know if someone wants to include an item in >> Next call's Agenda. >> >> Including the Summary here too (might potentially break formatting): >> >> *Key Decisions* >> >> - *Scheduler HA* >> - >> >> *Progress* >> - MySQL 8 tests have been added >> - Adoption of orphaned tasks have been added to Kubernetes and >> to the new CeleryKubernetesExecutor (PR >> <https://github.com/apache/airflow/pull/10901>) >> - All (but one) Tests are passing >> - Instructions to run the PR with Multiple Schedulers using >> docker-compose have been added (link to the comment >> >> <https://github.com/apache/airflow/pull/10956#issuecomment-702997850> >> ) >> - Received some reviews on Monday -- Should be addressed by >> today/tomorrow >> - All Big/Critical Concerns have been addressed (escape hatch for >> locks, MySQL 8 tests, Moving callbacks to DagFileProcessor) >> - Room to improve code readability which isn't a blocker and would >> be addressed in follow-up PR >> - Ash is in talks with Kevin to test his Giant DAG to verify >> Scheduler Performance to address one of Kevin Yang's concern >> - There would be a separate PR for retrying transactions to avoid >> deadlocks as suggested by MySQL guide on handling deadlocks >> >> <https://dev.mysql.com/doc/refman/5.7/en/innodb-deadlocks-handling.html> >> - *API* >> - Kamil and Omair are hoping to complete work on the Task Instance >> Endpoint PR <https://github.com/apache/airflow/pull/9597> by the >> end of this week >> - James is working to map existing permissions to the UI, hoping >> to complete by end of the week too. Link to PRs >> <https://github.com/apache/airflow/pulls/jhtimmins> >> - *Backport Providers* >> - Backport packages 2020.10.5 will be released >> without cncf.kubernetes. We will create a separate release for >> cncf.kubernetes that will address the issues Kamil found in RCs. Link >> to VOTE thread >> >> <https://lists.apache.org/thread.html/r4dfc18d2376601667efe7dfaf0bf1c07b840f9bcd9f6311d48cb3c64%40%3Cdev.airflow.apache.org%3E> >> . >> - *Airflow 1.10.13 Release* >> - Will be released as per the planned timeline (Airflow 2.0 - >> Planning >> >> <https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0+-+Planning>) >> i.e. Last Week of October. >> - *Airflow 2.0 Alpha* >> - Revisit on Thursday based on the progress with Scheduler HA & >> API whether we would be able to release first alpha this Friday. If >> not we >> will update timelines accordingly. >> - *Other small items*: >> - Kaxil to create Github Issues for the feedback received on the >> Documentation Improvement Thread and create threads asking for VOTES >> for >> previously discussed items like removing pickles. >> - Vikram will write up, share and all can review on what we mean >> by ALPHA and BETAs. >> >> >> Regards, >> Kaxil >> > > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/> > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
