Hi All, I am having a production issue running only one instance of DAG at a time. If the DAG is running one instance, a 2nd instance does not kick off. But if any of the task fails in the active DAG instance, the DAG gets marked failed but a 2nd instance kicks off after 5 mins(5 mins scheduled time for DAG.
Please help. Regards Sandeep On Mon, Sep 28, 2020 at 1:18 PM Tavares Forby <[email protected]> wrote: > > > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > I am having a few issues with Airflow and task instances greater than > 750. I am getting one consistent error and one error that happens random > (understand, it's technically not random). > > > > > > > > > > > > Consistent error: > > > > > [2020-09-25 12:28:01,703] {scheduler_job.py:237} > > WARNING - Killing PID 119970 [2020-09-25 12:29:17,110] > {scheduler_job.py:237} WARNING - Killing PID 121013 [2020-09-25 > 12:29:17,110] {scheduler_job.py:237} WARNING - Killing PID 121013 > [2020-09-25 12:30:12,171] {scheduler_job.py:237} WARNING - Killing PID > > 123243 > > > > > > > > > > > > > > Random error: > > > > > [2020-09-27 > > 19:37:25,127] {scheduler_job.py:771} INFO - Examining DAG run <DagRun > tutorial_large_design_debug7 @ 2020-09-28 02:37:24+00:00: > manual__2020-09-28T02:37:24+00:00, externally triggered: True> [2020-09-27 > 19:37:26,749] {logging_mixin.py:112} INFO - [2020-09-27 > > 19:37:26,749] {dagrun.py:408} INFO - (MySQLdb. > > *exceptions.IntegrityError) (1062, "Duplicate entry > 'echo__a*-tutorial_large_design_debug7-2020- > 09-28 02:37:24.000000' for key 'PRIMARY'") > > [SQL: INSERT INTO task_instance (task_id, dag_id, execution_date, > start_date, end_date, duration, state, try_number, max_tries, hostname, > unixname, job_id, pool, pool_slots, queue, priority_weight, operator, qu > eued_dttm, pid, executor_config) VALUES (%s, > > %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, > %s)] > > > > > > > > > > > > > > > > > > > > > Please > > help! thanks! > > > > > > >
