The scheduler should clear import errors for DAGfiles that no longer exist or DAGs that have fixed the import errors. Take a look at https://github.com/apache/incubator-airflow/blob/master/airflow/jobs.py#L761 and where it is called.
If it's not working there might be a bug. On Fri, Aug 10, 2018 at 10:19 AM Ben Laird <br.la...@gmail.com> wrote: > What is the desired behavior then? If I fix an error in a DAG, there should > be a way to clear the state in the error table. Since these errors pop up > in the webserver logs, and restarting the webserver with no error signals > that the DAGs are healthy, should restarting the webserver clear the error > table? Or have a CLI command as a fallback? > > > On Thu, Aug 9, 2018 at 5:35 PM, Alex Guziel <alex.guz...@airbnb.com > .invalid> > wrote: > > > IIRC the scheduler sets these messages in the error table in the db. > > > > On Thu, Aug 9, 2018 at 2:13 PM, Ben Laird <br.la...@gmail.com> wrote: > > > > > The messages persist even after restarting the webserver. I've verified > > > with other airflow users in the office that they'd have to manually > > delete > > > records from the 'import_error' table. > > > > > > When you say 'sync your DAGs', what do you mean exactly? When we fix a > > DAG, > > > we'd normally kill the webserver process, push a zip containing our dag > > > directory (with the fixed code), unzip and restart the webserver. > > > > > > Thanks > > > > > > On Thu, Aug 9, 2018 at 4:43 PM, Taylor Edmiston <tedmis...@gmail.com> > > > wrote: > > > > > > > Yeah, you definitely shouldn't need to do a resetdb for that. > > > > > > > > Did you try restarting the webserver? > > > > > > > > How do you sync your DAGs to the webserver? Is it possible the fixed > > DAG > > > > didn't get synced there? > > > > > > > > For me, IIRC, the error stops persisting once the DAG is fixed and > > > synced. > > > > > > > > *Taylor Edmiston* > > > > Blog <https://blog.tedmiston.com/> | CV > > > > <https://stackoverflow.com/cv/taylor> | LinkedIn > > > > <https://www.linkedin.com/in/tedmiston/> | AngelList > > > > <https://angel.co/taylor> | Stack Overflow > > > > <https://stackoverflow.com/users/149428/taylor-edmiston> > > > > > > > > > > > > On Thu, Aug 9, 2018 at 3:35 PM, Ben Laird <br.la...@gmail.com> > wrote: > > > > > > > > > Hello - > > > > > > > > > > I've noticed this several times and not sure what the solution is. > > If I > > > > > have a DAG error at some point, I'll see message in the webserver > > that > > > > says > > > > > "Broken DAG: [Error]". However, after fixing the code, restarting > the > > > > > webserver, etc, the error persists. After closing it out, it will > > just > > > > pop > > > > > up again after reloading. > > > > > > > > > > The only way I was able to delete was by doing a `airflow resetdb`. > > I'd > > > > > like to avoid manually deleting records from the DB, as now in prod > > we > > > > > cannot just kill the DB state. > > > > > > > > > > Any suggestions? > > > > > > > > > > Thanks, > > > > > Ben Laird > > > > > > > > > > > > > > >
