[jira] [Updated] (AIRFLOW-1141) Remove DAG.crawl_for_tasks method

2017-04-24 Thread Jeeyoung Kim (JIRA) 

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-1141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeeyoung Kim updated AIRFLOW-1141:
--
Description: 
Currently {{DAG}} class has the method:

{noformat}
def crawl_for_tasks(objects):
"""
Typically called at the end of a script by passing globals() as a
parameter. This allows to not explicitly add every single task to the
dag explicitly.
"""
raise NotImplementedError("")
{noformat}

which is not called or tested from anywhere:
{noformat}
$ git grep crawl_for_tasks
airflow/models.py:def crawl_for_tasks(objects):
{noformat}
 I suggest removing this method as it doesn't do anything and there's no 
documentation related to it. Moreover, if one creates a subclass of DAG, pylint 
returns the following warning: {{Method 'crawl_for_tasks' is abstract in class 
'DAG' but is not overridden (abstract-method)}}

  was:
Currently {{DAG}} class has the method:

{noformat}
def crawl_for_tasks(objects):
"""
Typically called at the end of a script by passing globals() as a
parameter. This allows to not explicitly add every single task to the
dag explicitly.
"""
raise NotImplementedError("")
{noformat}

which is not called or tested from anywhere. I suggest removing this method as 
it doesn't do anything and there's no documentation related to it. Moreover, if 
one creates a subclass of DAG, pylint returns the following warning: {{Method 
'crawl_for_tasks' is abstract in class 'DAG' but is not overridden 
(abstract-method)}}


> Remove DAG.crawl_for_tasks method
> -
>
> Key: AIRFLOW-1141
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1141
> Project: Apache Airflow
>  Issue Type: Task
>Reporter: Jeeyoung Kim
>Priority: Trivial
>
> Currently {{DAG}} class has the method:
> {noformat}
> def crawl_for_tasks(objects):
> """
> Typically called at the end of a script by passing globals() as a
> parameter. This allows to not explicitly add every single task to the
> dag explicitly.
> """
> raise NotImplementedError("")
> {noformat}
> which is not called or tested from anywhere:
> {noformat}
> $ git grep crawl_for_tasks
> airflow/models.py:def crawl_for_tasks(objects):
> {noformat}
>  I suggest removing this method as it doesn't do anything and there's no 
> documentation related to it. Moreover, if one creates a subclass of DAG, 
> pylint returns the following warning: {{Method 'crawl_for_tasks' is abstract 
> in class 'DAG' but is not overridden (abstract-method)}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (AIRFLOW-1141) Remove DAG.crawl_for_tasks method

2017-04-24 Thread Jeeyoung Kim (JIRA) 
Jeeyoung Kim created AIRFLOW-1141:
-

 Summary: Remove DAG.crawl_for_tasks method
 Key: AIRFLOW-1141
 URL: https://issues.apache.org/jira/browse/AIRFLOW-1141
 Project: Apache Airflow
  Issue Type: Task
Reporter: Jeeyoung Kim
Priority: Trivial


Currently {{DAG}} class has the method:

{noformat}
def crawl_for_tasks(objects):
"""
Typically called at the end of a script by passing globals() as a
parameter. This allows to not explicitly add every single task to the
dag explicitly.
"""
raise NotImplementedError("")
{noformat}

which is not called or tested from anywhere. I suggest removing this method as 
it doesn't do anything and there's no documentation related to it. Moreover, if 
one creates a subclass of DAG, pylint returns the following warning: {{Method 
'crawl_for_tasks' is abstract in class 'DAG' but is not overridden 
(abstract-method)}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (AIRFLOW-1017) get_task_instance should return None instead of throw an exception for non-existent TIs

2017-03-24 Thread Jeeyoung Kim (JIRA) 

[ 
https://issues.apache.org/jira/browse/AIRFLOW-1017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941325#comment-15941325
 ] 

Jeeyoung Kim commented on AIRFLOW-1017:
---

I just ran into this issue. Thanks for resolving!

> get_task_instance should return None instead of throw an exception for 
> non-existent TIs
> ---
>
> Key: AIRFLOW-1017
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1017
> Project: Apache Airflow
>  Issue Type: Bug
>Reporter: Dan Davydov
>Assignee: Dan Davydov
>Priority: Critical
>
> We were seeing errors in our scheduler like the following due to this issue:
>   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 
> 2796, in one
> raise orm_exc.NoResultFound("No row was found for one()")



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)