I have asked Sid to create and Jira and to make it reproducible. Nevertheless, I do not consider it a blocker as a workaround exists and it is relatively small in scope (while slightly annoying I understand that).
Let’s get 1.8 out and do bug fixes in 1.8.1. More bugs will inevitably pop up :). - Bolke > On 15 Mar 2017, at 09:04, Chris Riccomini <criccom...@apache.org> wrote: > > Has anyone been able to reproduce Sid's issue? > > On Tue, Mar 14, 2017 at 11:17 PM, Bolke de Bruin <bdbr...@gmail.com> wrote: > >> That is not an airflow error, but a Kerberos error. Try executing the >> kinit command on the command line by yourself. >> >> Bolke >> >> Sent from my iPhone >> >>> On 14 Mar 2017, at 23:11, Ruslan Dautkhanov <dautkha...@gmail.com> >> wrote: >>> >>> `airflow kerberos` is broken in 1.8-rc5 >>> https://issues.apache.org/jira/browse/AIRFLOW-987 >>> Hopefully fix can be part of the 1.8 release. >>> >>> >>> >>> -- >>> Ruslan Dautkhanov >>> >>>> On Tue, Mar 14, 2017 at 6:19 PM, siddharth anand <san...@apache.org> >> wrote: >>>> >>>> FYI, >>>> I've just hit a major bug in the release candidate related to "clear >> task" >>>> behavior. >>>> >>>> I've been running airflow in both stage and prod since yesterday on rc5 >> and >>>> have reproduced this in both environments. I will file a JIRA for this >>>> tonight, but wanted to send a note over email as well. >>>> >>>> In my example, I have a 2 task DAG. For a given DAG run that has >> completed >>>> successfully, if I >>>> 1) clear task2 (leaf task in this case), the previously-successful DAG >> Run >>>> goes back to Running, requeues, and executes the task successfully. The >> DAG >>>> Run the returns from Running to Success. >>>> 2) clear task1 (root task in this case), the previously-successful DAG >> Run >>>> goes back to Running, DOES NOT requeue or execute the task at all. The >> DAG >>>> Run the returns from Running to Success though it never ran the task. >>>> >>>> 1) is expected and previous behavior. 2) is a regression. >>>> >>>> The only workaround is to use the CLI to run the task cleared. Here are >>>> some images : >>>> *After Clearing the Tasks* >>>> https://www.dropbox.com/s/wmuxt0krwx6wurr/Screenshot% >>>> 202017-03-14%2014.09.34.png?dl=0 >>>> >>>> *After DAG Runs return to Success* >>>> https://www.dropbox.com/s/qop933rzgdzchpd/Screenshot% >>>> 202017-03-14%2014.09.49.png?dl=0 >>>> >>>> This is a major regression because it will force everyone to use the CLI >>>> for things that they would normally use the UI for. >>>> >>>> -s >>>> >>>> >>>> -s >>>> >>>> >>>>> On Tue, Mar 14, 2017 at 1:32 PM, Daniel Huang <dxhu...@gmail.com> >> wrote: >>>>> >>>>> +1 (non-binding)! >>>>> >>>>> On Tue, Mar 14, 2017 at 11:35 AM, siddharth anand <san...@apache.org> >>>>> wrote: >>>>> >>>>>> +1 (binding) >>>>>> >>>>>> >>>>>> On Tue, Mar 14, 2017 at 8:42 AM, Maxime Beauchemin < >>>>>> maximebeauche...@gmail.com> wrote: >>>>>> >>>>>>> +1 (binding) >>>>>>> >>>>>>> On Tue, Mar 14, 2017 at 3:59 AM, Alex Van Boxel <a...@vanboxel.be> >>>>>> wrote: >>>>>>> >>>>>>>> +1 (binding) >>>>>>>> >>>>>>>> Note: we had to revert all our ONE_SUCCESS with ALL_SUCCESS trigger >>>>>> rules >>>>>>>> where the parent nodes where joining with a SKIP. But I can of >>>> should >>>>>>> have >>>>>>>> known this was coming. Apart of that I had a successful run last >>>>> night. >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Mar 14, 2017 at 1:37 AM siddharth anand <san...@apache.org >>>>> >>>>>>> wrote: >>>>>>>> >>>>>>>> I'm going to deploy this to staging now. Fab work Bolke! >>>>>>>> -s >>>>>>>> >>>>>>>> On Mon, Mar 13, 2017 at 2:16 PM, Dan Davydov < >>>> dan.davy...@airbnb.com >>>>> . >>>>>>>> invalid >>>>>>>>> wrote: >>>>>>>> >>>>>>>>> I'll test this on staging as soon as I get a chance (the testing >>>> is >>>>>>>>> non-blocking on the rc5). Bolke very much in particular :). >>>>>>>>> >>>>>>>>> On Mon, Mar 13, 2017 at 10:46 AM, Jeremiah Lowin < >>>>> jlo...@apache.org> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> +1 (binding) extremely impressed by the work and diligence all >>>>>>>>> contributors >>>>>>>>>> have put in to getting these blockers fixed, Bolke in >>>> particular. >>>>>>>>>> >>>>>>>>>> On Mon, Mar 13, 2017 at 1:07 AM Arthur Wiedmer < >>>>> art...@apache.org> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> +1 (binding) >>>>>>>>>>> >>>>>>>>>>> Thanks again for steering us through Bolke. >>>>>>>>>>> >>>>>>>>>>> Best, >>>>>>>>>>> Arthur >>>>>>>>>>> >>>>>>>>>>> On Sun, Mar 12, 2017 at 9:59 PM, Bolke de Bruin < >>>>>> bdbr...@gmail.com >>>>>>>> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Dear All, >>>>>>>>>>>> >>>>>>>>>>>> Finally, I have been able to make the FIFTH RELEASE >>>> CANDIDATE >>>>>> of >>>>>>>>>> Airflow >>>>>>>>>>>> 1.8.0 available at: https://dist.apache.org/repos/ >>>>>>>>>>>> dist/dev/incubator/airflow/ <https://dist.apache.org/ >>>>>>>>>>>> repos/dist/dev/incubator/airflow/> , public keys are >>>>> available >>>>>>> at >>>>>>>>>>>> https://dist.apache.org/repos/dist/release/incubator/ >>>>> airflow/ >>>>>> < >>>>>>>>>>>> https://dist.apache.org/repos/dist/release/incubator/ >>>>> airflow/> >>>>>> . >>>>>>>> It >>>>>>>>> is >>>>>>>>>>>> tagged with a local version “apache.incubating” so it >>>> allows >>>>>>>>> upgrading >>>>>>>>>>> from >>>>>>>>>>>> earlier releases. >>>>>>>>>>>> >>>>>>>>>>>> Issues fixed since rc4: >>>>>>>>>>>> >>>>>>>>>>>> [AIRFLOW-900] Double trigger should not kill original task >>>>>>> instance >>>>>>>>>>>> [AIRFLOW-900] Fixes bugs in LocalTaskJob for double run >>>>>>> protection >>>>>>>>>>>> [AIRFLOW-932] Do not mark tasks removed when backfilling >>>>>>>>>>>> [AIRFLOW-961] run onkill when SIGTERMed >>>>>>>>>>>> [AIRFLOW-910] Use parallel task execution for backfills >>>>>>>>>>>> [AIRFLOW-967] Wrap strings in native for py2 ldap >>>>> compatibility >>>>>>>>>>>> [AIRFLOW-941] Use defined parameters for psycopg2 >>>>>>>>>>>> [AIRFLOW-719] Prevent DAGs from ending prematurely >>>>>>>>>>>> [AIRFLOW-938] Use test for True in task_stats queries >>>>>>>>>>>> [AIRFLOW-937] Improve performance of task_stats >>>>>>>>>>>> [AIRFLOW-933] use ast.literal_eval rather eval because >>>>>>>>> ast.literal_eval >>>>>>>>>>>> does not execute input. >>>>>>>>>>>> [AIRFLOW-919] Running tasks with no start date shouldn't >>>>> break >>>>>> a >>>>>>>> DAGs >>>>>>>>>> UI >>>>>>>>>>>> [AIRFLOW-897] Prevent dagruns from failing with unfinished >>>>>> tasks >>>>>>>>>>>> [AIRFLOW-861] make pickle_info endpoint be login_required >>>>>>>>>>>> [AIRFLOW-853] use utf8 encoding for stdout line decode >>>>>>>>>>>> [AIRFLOW-856] Make sure execution date is set for local >>>>> client >>>>>>>>>>>> [AIRFLOW-830][AIRFLOW-829][AIRFLOW-88] Reduce Travis log >>>>>>> verbosity >>>>>>>>>>>> [AIRFLOW-794] Access DAGS_FOLDER and SQL_ALCHEMY_CONN >>>>>> exclusively >>>>>>>>> from >>>>>>>>>>>> settings >>>>>>>>>>>> [AIRFLOW-694] Fix config behaviour for empty envvar >>>>>>>>>>>> [AIRFLOW-365] Set dag.fileloc explicitly and use for Code >>>>> view >>>>>>>>>>>> [AIRFLOW-931] Do not set QUEUED in TaskInstances >>>>>>>>>>>> [AIRFLOW-899] Tasks in SCHEDULED state should be white in >>>> the >>>>>> UI >>>>>>>>>> instead >>>>>>>>>>>> of black >>>>>>>>>>>> [AIRFLOW-895] Address Apache release incompliancies >>>>>>>>>>>> [AIRFLOW-893][AIRFLOW-510] Fix crashing webservers when a >>>>>> dagrun >>>>>>>> has >>>>>>>>> no >>>>>>>>>>>> start date >>>>>>>>>>>> [AIRFLOW-793] Enable compressed loading in S3ToHiveTransfer >>>>>>>>>>>> [AIRFLOW-863] Example DAGs should have recent start dates >>>>>>>>>>>> [AIRFLOW-869] Refactor mark success functionality >>>>>>>>>>>> [AIRFLOW-856] Make sure execution date is set for local >>>>> client >>>>>>>>>>>> [AIRFLOW-814] Fix Presto*CheckOperator.__init__ >>>>>>>>>>>> [AIRFLOW-844] Fix cgroups directory creation >>>>>>>>>>>> >>>>>>>>>>>> No known issues anymore. >>>>>>>>>>>> >>>>>>>>>>>> I would also like to raise a VOTE for releasing 1.8.0 based >>>>> on >>>>>>>>> release >>>>>>>>>>>> candidate 5, i.e. just renaming release candidate 5 to >>>> 1.8.0 >>>>>>>> release. >>>>>>>>>>>> >>>>>>>>>>>> Please respond to this email by: >>>>>>>>>>>> >>>>>>>>>>>> +1,0,-1 with *binding* if you are a PMC member or >>>>> *non-binding* >>>>>>> if >>>>>>>>> you >>>>>>>>>>> are >>>>>>>>>>>> not. >>>>>>>>>>>> >>>>>>>>>>>> Thanks! >>>>>>>>>>>> Bolke >>>>>>>>>>>> >>>>>>>>>>>> My VOTE: +1 (binding) >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> _/ >>>>>>>> _/ Alex Van Boxel >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>
