I have some concern that this change
https://github.com/apache/incubator-airflow/pull/1939
[AIRFLOW-679] may be having issues because we are seeing lots of double triggers
of tasks and tasks being killed as a result.
On Wed, Feb 22, 2017 4:35 PM, Dan Davydov dan.davy...@airbnb.com.INVALID wrote:
Bumping the thread so another user can comment.
On Wed, Feb 22, 2017 at 3:12 PM, Maxime Beauchemin <
maximebeauche...@gmail.com> wrote:
> What I meant to ask is "how much engineering effort it takes to bake a
> single RC?", I guess it depends on how much git-fu is necessary plus some
> overhead cost of doing the series of actions/commands/emails/jira.
>
> I can volunteer for 1.8.1 (hopefully I can get do it along another Airbnb
> engineer/volunteer to tag along) and will try to document/automate
> everything I can as I go through the process. The goal of 1.8.1 could be to
> basically package 1.8.0 + Dan's bugfix, and for Airbnb to get familiar with
> the process.
>
> It'd be great if you can dump your whole process on the wiki, and we'll
> improve it on this next pass.
>
> Thanks again for the mountain of work that went into packaging this
> release.
>
> Max
>
> On Wed, Feb 22, 2017 at 2:44 PM, Bolke de Bruin <bdbr...@gmail.com> wrote:
>
> > I thought you volunteered to baby sit 1.8.1 Chris ;-)?
> >
> > Sent from my iPhone
> >
> > > On 22 Feb 2017, at 23:31, Chris Riccomini <criccom...@apache.org>
> wrote:
> > >
> > > I'm +1 for doing a 1.8.1 fast follow-on
> > >
> > > On Wed, Feb 22, 2017 at 2:26 PM, Maxime Beauchemin <
> > > maximebeauche...@gmail.com> wrote:
> > >
> > >> Our database may have edge cases that could be associated with running
> > any
> > >> previous version that may or may not have been part of an official
> > release.
> > >>
> > >> Let's see if anyone else reports the issue. If no one does, one option
> > is
> > >> to release 1.8.0 as is with a comment in the release notes, and have a
> > >> future official minor apache release 1.8.1 that would fix these minor
> > >> issues that are not deal breaker.
> > >>
> > >> @bolke, I'm curious, how long does it take you to go through one
> release
> > >> cycle? Oh, and do you have a documented step by step process for
> > releasing?
> > >> I'd like to add the Pypi part to this doc and add committers that are
> > >> interested to have rights on the project on Pypi.
> > >>
> > >> Max
> > >>
> > >>> On Wed, Feb 22, 2017 at 2:00 PM, Bolke de Bruin <bdbr...@gmail.com>
> > wrote:
> > >>>
> > >>> So it is a database integrity issue? Afaik a start_date should always
> > be
> > >>> set for a DagRun (create_dagrun) does so I didn't check the code
> > though.
> > >>>
> > >>> Sent from my iPhone
> > >>>
> > >>>> On 22 Feb 2017, at 22:19, Dan Davydov <dan.davy...@airbnb.com.
> > INVALID>
> > >>> wrote:
> > >>>>
> > >>>> Should clarify this occurs when a dagrun does not have a start date,
> > >> not
> > >>> a
> > >>>> dag (which makes it even less likely to happen). I don't think this
> is
> > >> a
> > >>>> blocker for releasing.
> > >>>>
> > >>>>> On Wed, Feb 22, 2017 at 1:15 PM, Dan Davydov <
> dan.davy...@airbnb.com
> > >
> > >>> wrote:
> > >>>>>
> > >>>>> I rolled this out in our prod and the webservers failed to load due
> > to
> > >>>>> this commit:
> > >>>>>
> > >>>>> [AIRFLOW-510] Filter Paused Dags, show Last Run & Trigger Dag
> > >>>>> 7c94d81c390881643f94d5e3d7d6fb351a445b72
> > >>>>>
> > >>>>> This fixed it:
> > >>>>> - </a> <span id="statuses_info"
> > >>>>> class="glyphicon glyphicon-info-sign" aria-hidden="true"
> title="Start
> > >>> Date:
> > >>>>> {{last_run.start_date.strftime('%Y-%m-%d %H:%M')}}"></span>
> > >>>>> + </a> <span id="statuses_info"
> > >>>>> class="glyphicon glyphicon-info-sign" aria-hidden="true"></span>
> > >>>>>
> > >>>>> This is caused by assuming that all DAGs have start dates set, so a
> > >>> broken
> > >>>>> DAG will take down the whole UI. Not sure if we want to make this a
> > >>> blocker
> > >>>>> for the release or not, I'm guessing for most deployments this
> would
> > >>> occur
> > >>>>> pretty rarely. I'll submit a PR to fix it soon.
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On Tue, Feb 21, 2017 at 9:49 AM, Chris Riccomini <
> > >> criccom...@apache.org
> > >>>>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Ack that the vote has already passed, but belated +1 (binding)
> > >>>>>>
> > >>>>>> On Tue, Feb 21, 2017 at 7:42 AM, Bolke de Bruin <
> bdbr...@gmail.com>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> IPMC Voting can be found here:
> > >>>>>>>
> > >>>>>>> http://mail-archives.apache.org/mod_mbox/incubator-general/
> > >>>>>> 201702.mbox/%
> > >>>>>>> 3c676bdc9f-1b55-4469-92a7-9ff309ad0...@gmail.com%3e <
> > >>>>>>> http://mail-archives.apache.org/mod_mbox/incubator-general/
> > >>>>>> 201702.mbox/%
> > >>>>>>> 3c676bdc9f-1b55-4469-92a7-9ff309ad0...@gmail.com%3E>
> > >>>>>>>
> > >>>>>>> Kind regards,
> > >>>>>>> Bolke
> > >>>>>>>
> > >>>>>>>> On 21 Feb 2017, at 08:20, Bolke de Bruin <bdbr...@gmail.com>
> > >> wrote:
> > >>>>>>>>
> > >>>>>>>> Hello,
> > >>>>>>>>
> > >>>>>>>> Apache Airflow (incubating) 1.8.0 (based on RC4) has been
> > accepted.
> > >>>>>>>>
> > >>>>>>>> 9 “+1” votes received:
> > >>>>>>>>
> > >>>>>>>> - Maxime Beauchemin (binding)
> > >>>>>>>> - Arthur Wiedmer (binding)
> > >>>>>>>> - Dan Davydov (binding)
> > >>>>>>>> - Jeremiah Lowin (binding)
> > >>>>>>>> - Siddharth Anand (binding)
> > >>>>>>>> - Alex van Boxel (binding)
> > >>>>>>>> - Bolke de Bruin (binding)
> > >>>>>>>>
> > >>>>>>>> - Jayesh Senjaliya (non-binding)
> > >>>>>>>> - Yi (non-binding)
> > >>>>>>>>
> > >>>>>>>> Vote thread (start):
> > >>>>>>>> http://mail-archives.apache.org/mod_mbox/incubator-
> > >>>>>>> airflow-dev/201702.mbox/%3cD360D9BE-C358-42A1-9188-
> > >>>>>>> 6c92c31a2...@gmail.com%3e <http://mail-archives.apache.
> > >>>>>>> org/mod_mbox/incubator-airflow-dev/201702.mbox/%3C7EB7B6D6-
> > >>>>>> 092E-48D2-AA0F-
> > >>>>>>> 15f44376a...@gmail.com%3E>
> > >>>>>>>>
> > >>>>>>>> Next steps:
> > >>>>>>>> 1) will start the voting process at the IPMC mailinglist. I do
> > >> expect
> > >>>>>>> some changes to be required mostly in documentation maybe a
> license
> > >>> here
> > >>>>>>> and there. So, we might end up with changes to stable. As long as
> > >>> these
> > >>>>>> are
> > >>>>>>> not (significant) code changes I will not re-raise the vote.
> > >>>>>>>> 2) Only after the positive voting on the IPMC and finalisation I
> > >> will
> > >>>>>>> rebrand the RC to Release.
> > >>>>>>>> 3) I will upload it to the incubator release page, then the tar
> > >> ball
> > >>>>>>> needs to propagate to the mirrors.
> > >>>>>>>> 4) Update the website (can someone volunteer please?)
> > >>>>>>>> 5) Finally, I will ask Maxime to upload it to pypi. It seems we
> > can
> > >>>>>> keep
> > >>>>>>> the apache branding as lib cloud is doing this as well (
> > >>>>>>> https://libcloud.apache.org/downloads.html#pypi-package <
> > >>>>>>> https://libcloud.apache.org/downloads.html#pypi-package>).
> > >>>>>>>>
> > >>>>>>>> Jippie!
> > >>>>>>>>
> > >>>>>>>> Bolke
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>
> > >>
> >
>
