[jira] [Comment Edited] (AIRFLOW-168) schedule_interval @once scheduling dag atleast twice
[ https://issues.apache.org/jira/browse/AIRFLOW-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301539#comment-15301539 ] Bolke de Bruin edited comment on AIRFLOW-168 at 5/26/16 10:30 AM: -- The double scheduling is indeed a bug on master, also with the updated scheduler from 124, that I will need to fix. It is also a bug in the current scheduler but does not show as tasks are not eagerly created. was (Author: bolke): The double scheduling is indeed a bug on master, also with the updated scheduler from 124, that I will need to fix. > schedule_interval @once scheduling dag atleast twice > > > Key: AIRFLOW-168 > URL: https://issues.apache.org/jira/browse/AIRFLOW-168 > Project: Apache Airflow > Issue Type: Bug > Components: scheduler >Affects Versions: Airflow 1.7.1.2 >Reporter: Sumit Maheshwari > Attachments: Screen Shot 2016-05-24 at 9.51.50 PM.png, > screenshot-1.png > > > I was looking at example_xcom example and found that it got scheduled twice. > Ones at the start_time and ones at the current time. To be correct I tried > multiple times (by reloading db) and its same. > I am on airflow master, using sequential executor with sqlite3. Though it > works as expected on a prod env which is running v1.7 with celery workers and > mysql backend. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (AIRFLOW-168) schedule_interval @once scheduling dag atleast twice
[
https://issues.apache.org/jira/browse/AIRFLOW-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301664#comment-15301664
]
Bolke de Bruin edited comment on AIRFLOW-168 at 5/26/16 7:20 AM:
-
The issue with double dagruns is due to this line:
{code}
# don't ever schedule prior to the dag's start_date
if dag.start_date:
next_run_date = dag.start_date if not next_run_date else
max(next_run_date, dag.start_date)
{code}
which doesn't check if it was scheduled before. Will have a fix shortly.
was (Author: bolke):
The issue with double dagruns is due to this line:
# don't ever schedule prior to the dag's start_date
if dag.start_date:
next_run_date = dag.start_date if not next_run_date else
max(next_run_date, dag.start_date)
which doesn't check if it was scheduled before. Will have a fix shortly.
> schedule_interval @once scheduling dag atleast twice
>
>
> Key: AIRFLOW-168
> URL: https://issues.apache.org/jira/browse/AIRFLOW-168
> Project: Apache Airflow
> Issue Type: Bug
> Components: scheduler
>Affects Versions: Airflow 1.7.1.2
>Reporter: Sumit Maheshwari
> Attachments: Screen Shot 2016-05-24 at 9.51.50 PM.png,
> screenshot-1.png
>
>
> I was looking at example_xcom example and found that it got scheduled twice.
> Ones at the start_time and ones at the current time. To be correct I tried
> multiple times (by reloading db) and its same.
> I am on airflow master, using sequential executor with sqlite3. Though it
> works as expected on a prod env which is running v1.7 with celery workers and
> mysql backend.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (AIRFLOW-168) schedule_interval @once scheduling dag atleast twice
[ https://issues.apache.org/jira/browse/AIRFLOW-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301539#comment-15301539 ] Bolke de Bruin edited comment on AIRFLOW-168 at 5/26/16 5:08 AM: - The double scheduling is indeed a bug on master, also with the updated scheduler from 124, that I will need to fix. was (Author: bolke): The double scheduling is indeed a bug, also with the updated scheduler from 124, that I will need to fix. > schedule_interval @once scheduling dag atleast twice > > > Key: AIRFLOW-168 > URL: https://issues.apache.org/jira/browse/AIRFLOW-168 > Project: Apache Airflow > Issue Type: Bug > Components: scheduler >Affects Versions: Airflow 1.7.1.2 >Reporter: Sumit Maheshwari > Attachments: Screen Shot 2016-05-24 at 9.51.50 PM.png, > screenshot-1.png > > > I was looking at example_xcom example and found that it got scheduled twice. > Ones at the start_time and ones at the current time. To be correct I tried > multiple times (by reloading db) and its same. > I am on airflow master, using sequential executor with sqlite3. Though it > works as expected on a prod env which is running v1.7 with celery workers and > mysql backend. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (AIRFLOW-168) schedule_interval @once scheduling dag atleast twice
[ https://issues.apache.org/jira/browse/AIRFLOW-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301507#comment-15301507 ] Bolke de Bruin edited comment on AIRFLOW-168 at 5/26/16 4:44 AM: - Yes sorry I mentioned this on gitter. With master deadlock detection is broken due to the eager creation and the scheduler will not check for existing task_instances before creation, hence the constraint error. It will need the follow up patch from AIRFLOW-128. Black is indeed the color (basically undefined) for tasksinstances that are created but have not been picked up by the scheduler. was (Author: bolke): Yes sorry I mentioned this on gitter. With master deadlock detection is broken due to the eager creation. It will need the follow up patch from AIRFLOW-128. Black is indeed the color (basically undefined) > schedule_interval @once scheduling dag atleast twice > > > Key: AIRFLOW-168 > URL: https://issues.apache.org/jira/browse/AIRFLOW-168 > Project: Apache Airflow > Issue Type: Bug > Components: scheduler >Affects Versions: Airflow 1.7.1.2 >Reporter: Sumit Maheshwari > Attachments: Screen Shot 2016-05-24 at 9.51.50 PM.png, > screenshot-1.png > > > I was looking at example_xcom example and found that it got scheduled twice. > Ones at the start_time and ones at the current time. To be correct I tried > multiple times (by reloading db) and its same. > I am on airflow master, using sequential executor with sqlite3. Though it > works as expected on a prod env which is running v1.7 with celery workers and > mysql backend. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
