[jira] [Created] (AIRFLOW-4058) Rename test files
Peter van 't Hof created AIRFLOW-4058: - Summary: Rename test files Key: AIRFLOW-4058 URL: https://issues.apache.org/jira/browse/AIRFLOW-4058 Project: Apache Airflow Issue Type: Improvement Components: tests Reporter: Peter van 't Hof Assignee: Peter van 't Hof Currently we have `tests/core.py` and `tests/models.py`. It's better to rename this to `tests/test_core.py` and `tests/test_models.py`. This is follow more the python conventions. This also offers better integration with ide's. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-3850) Remove DagBag from TaskInstanceModelView
[ https://issues.apache.org/jira/browse/AIRFLOW-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-3850 started by Peter van 't Hof. - > Remove DagBag from TaskInstanceModelView > > > Key: AIRFLOW-3850 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3850 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-3835) Remove DagBag from /task
[ https://issues.apache.org/jira/browse/AIRFLOW-3835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-3835 started by Peter van 't Hof. - > Remove DagBag from /task > > > Key: AIRFLOW-3835 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3835 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-3835) Remove DagBag from /task
[ https://issues.apache.org/jira/browse/AIRFLOW-3835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof reassigned AIRFLOW-3835: - Assignee: Julian de Ruiter (was: Peter van 't Hof) > Remove DagBag from /task > > > Key: AIRFLOW-3835 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3835 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Julian de Ruiter >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-4047) Remove DagBag from /paused
[ https://issues.apache.org/jira/browse/AIRFLOW-4047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-4047 started by Peter van 't Hof. - > Remove DagBag from /paused > -- > > Key: AIRFLOW-4047 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4047 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4047) Remove DagBag from /paused
Peter van 't Hof created AIRFLOW-4047: - Summary: Remove DagBag from /paused Key: AIRFLOW-4047 URL: https://issues.apache.org/jira/browse/AIRFLOW-4047 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof Assignee: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-3833) Remove DagBag from /get_logs_with_metadata
[ https://issues.apache.org/jira/browse/AIRFLOW-3833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof reassigned AIRFLOW-3833: - Assignee: Julian de Ruiter (was: Peter van 't Hof) > Remove DagBag from /get_logs_with_metadata > -- > > Key: AIRFLOW-3833 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3833 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Julian de Ruiter >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-3833) Remove DagBag from /get_logs_with_metadata
[ https://issues.apache.org/jira/browse/AIRFLOW-3833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-3833 started by Peter van 't Hof. - > Remove DagBag from /get_logs_with_metadata > -- > > Key: AIRFLOW-3833 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3833 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-3843) Remove DagBag from /tries
[ https://issues.apache.org/jira/browse/AIRFLOW-3843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-3843 started by Peter van 't Hof. - > Remove DagBag from /tries > - > > Key: AIRFLOW-3843 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3843 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-3832) Remove DagBag from /rendered
[ https://issues.apache.org/jira/browse/AIRFLOW-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof reassigned AIRFLOW-3832: - Assignee: Julian de Ruiter (was: Peter van 't Hof) > Remove DagBag from /rendered > > > Key: AIRFLOW-3832 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3832 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Julian de Ruiter >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-3846) Remove DagBag from /gantt
[ https://issues.apache.org/jira/browse/AIRFLOW-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-3846 started by Peter van 't Hof. - > Remove DagBag from /gantt > - > > Key: AIRFLOW-3846 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3846 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-3846) Remove DagBag from /gantt
[ https://issues.apache.org/jira/browse/AIRFLOW-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof reassigned AIRFLOW-3846: - Assignee: Julian de Ruiter (was: Peter van 't Hof) > Remove DagBag from /gantt > - > > Key: AIRFLOW-3846 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3846 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Julian de Ruiter >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-3844) Remove DagBag from /landing_times
[ https://issues.apache.org/jira/browse/AIRFLOW-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof reassigned AIRFLOW-3844: - Assignee: Julian de Ruiter (was: Peter van 't Hof) > Remove DagBag from /landing_times > - > > Key: AIRFLOW-3844 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3844 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Julian de Ruiter >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-3844) Remove DagBag from /landing_times
[ https://issues.apache.org/jira/browse/AIRFLOW-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-3844 started by Peter van 't Hof. - > Remove DagBag from /landing_times > - > > Key: AIRFLOW-3844 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3844 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-3832) Remove DagBag from /rendered
[ https://issues.apache.org/jira/browse/AIRFLOW-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-3832 started by Peter van 't Hof. - > Remove DagBag from /rendered > > > Key: AIRFLOW-3832 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3832 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-3837) Remove DagBag from /blocked
[ https://issues.apache.org/jira/browse/AIRFLOW-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-3837 started by Peter van 't Hof. - > Remove DagBag from /blocked > --- > > Key: AIRFLOW-3837 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3837 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-3837) Remove DagBag from /blocked
[ https://issues.apache.org/jira/browse/AIRFLOW-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof reassigned AIRFLOW-3837: - Assignee: Bas Harenslak (was: Peter van 't Hof) > Remove DagBag from /blocked > --- > > Key: AIRFLOW-3837 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3837 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Bas Harenslak >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-3842) Remove DagBag from /duration
[ https://issues.apache.org/jira/browse/AIRFLOW-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof reassigned AIRFLOW-3842: - Assignee: Julian de Ruiter (was: Peter van 't Hof) > Remove DagBag from /duration > > > Key: AIRFLOW-3842 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3842 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Julian de Ruiter >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-3836) Remove DagBag from /run
[ https://issues.apache.org/jira/browse/AIRFLOW-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-3836 started by Peter van 't Hof. - > Remove DagBag from /run > --- > > Key: AIRFLOW-3836 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3836 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-3836) Remove DagBag from /run
[ https://issues.apache.org/jira/browse/AIRFLOW-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof reassigned AIRFLOW-3836: - Assignee: Bas Harenslak (was: Peter van 't Hof) > Remove DagBag from /run > --- > > Key: AIRFLOW-3836 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3836 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Bas Harenslak >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-4035) Remove DagBag from /graph
[ https://issues.apache.org/jira/browse/AIRFLOW-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-4035 started by Peter van 't Hof. - > Remove DagBag from /graph > - > > Key: AIRFLOW-4035 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4035 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4035) Remove DagBag from /graph
Peter van 't Hof created AIRFLOW-4035: - Summary: Remove DagBag from /graph Key: AIRFLOW-4035 URL: https://issues.apache.org/jira/browse/AIRFLOW-4035 Project: Apache Airflow Issue Type: Improvement Reporter: Peter van 't Hof Assignee: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-3831) Remove DagBag from /pickle_info
[ https://issues.apache.org/jira/browse/AIRFLOW-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-3831 started by Peter van 't Hof. - > Remove DagBag from /pickle_info > --- > > Key: AIRFLOW-3831 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3831 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-3831) Remove DagBag from /pickle_info
[ https://issues.apache.org/jira/browse/AIRFLOW-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof reassigned AIRFLOW-3831: - Assignee: Bas Harenslak (was: Peter van 't Hof) > Remove DagBag from /pickle_info > --- > > Key: AIRFLOW-3831 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3831 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Bas Harenslak >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-3834) Remove DagBag from /log
[ https://issues.apache.org/jira/browse/AIRFLOW-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-3834 started by Peter van 't Hof. - > Remove DagBag from /log > --- > > Key: AIRFLOW-3834 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3834 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4027) Make experimental api tests more stateless
Peter van 't Hof created AIRFLOW-4027: - Summary: Make experimental api tests more stateless Key: AIRFLOW-4027 URL: https://issues.apache.org/jira/browse/AIRFLOW-4027 Project: Apache Airflow Issue Type: Improvement Components: api, tests Reporter: Peter van 't Hof Assignee: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4024) Improve local client tests
Peter van 't Hof created AIRFLOW-4024: - Summary: Improve local client tests Key: AIRFLOW-4024 URL: https://issues.apache.org/jira/browse/AIRFLOW-4024 Project: Apache Airflow Issue Type: Improvement Reporter: Peter van 't Hof Assignee: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4016) Make job tests more stateless
Peter van 't Hof created AIRFLOW-4016: - Summary: Make job tests more stateless Key: AIRFLOW-4016 URL: https://issues.apache.org/jira/browse/AIRFLOW-4016 Project: Apache Airflow Issue Type: Bug Components: tests Reporter: Peter van 't Hof Assignee: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-3830) Remove DagBag from /dag_details
[ https://issues.apache.org/jira/browse/AIRFLOW-3830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof reassigned AIRFLOW-3830: - Assignee: Peter van 't Hof > Remove DagBag from /dag_details > --- > > Key: AIRFLOW-3830 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3830 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3906) Npm package are not in the docker file
Peter van 't Hof created AIRFLOW-3906: - Summary: Npm package are not in the docker file Key: AIRFLOW-3906 URL: https://issues.apache.org/jira/browse/AIRFLOW-3906 Project: Apache Airflow Issue Type: Bug Components: docker Reporter: Peter van 't Hof Assignee: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-3904) Adding Foreign Key's
[ https://issues.apache.org/jira/browse/AIRFLOW-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof reassigned AIRFLOW-3904: - Assignee: (was: Peter van 't Hof) > Adding Foreign Key's > > > Key: AIRFLOW-3904 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3904 > Project: Apache Airflow > Issue Type: Bug > Components: database >Reporter: Peter van 't Hof >Priority: Major > Fix For: 2.0.0 > > Attachments: airflow.pdf > > > Currently the database has almost no relationships. We should try to link all > tables where possible. > [^airflow.pdf] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3904) Adding Foreign Key's
Peter van 't Hof created AIRFLOW-3904: - Summary: Adding Foreign Key's Key: AIRFLOW-3904 URL: https://issues.apache.org/jira/browse/AIRFLOW-3904 Project: Apache Airflow Issue Type: Bug Components: database Reporter: Peter van 't Hof Assignee: Peter van 't Hof Fix For: 2.0.0 Attachments: airflow.pdf Currently the database has almost no relationships. We should try to link all tables where possible. [^airflow.pdf] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-3562) [AIP-12] Remove DagBag dependency inside the webserver
[ https://issues.apache.org/jira/browse/AIRFLOW-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof updated AIRFLOW-3562: -- Summary: [AIP-12] Remove DagBag dependency inside the webserver (was: Remove DagBag dependency inside the webserver) > [AIP-12] Remove DagBag dependency inside the webserver > -- > > Key: AIRFLOW-3562 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3562 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Affects Versions: 1.10.0, 1.10.1 >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Critical > Fix For: 2.0.0 > > > Currently the webserver does depend on the dag directory and reparsing the > dag files. Because this happens with gunicorn each process has there own > DagBag. The means the webserver is statefull and depending which process is > assigned a request the information can be changed. If all information will > come from the database this will not happen. The only process that should add > dag/dagruns is the scheduler. > The webserver should have enough on only the database. Currently not all > information is already in the database. This could be fixable, already > started this in AIRFLOW-3561. > I do notice this might give some conflicts with the brace change that is > coming. Any suggestion how to approach this? > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-3839) Remove DagBag from _mark_dagrun_state_as_success
[ https://issues.apache.org/jira/browse/AIRFLOW-3839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof updated AIRFLOW-3839: -- Summary: Remove DagBag from _mark_dagrun_state_as_success (was: Remove DagBNag from _mark_dagrun_state_as_success) > Remove DagBag from _mark_dagrun_state_as_success > > > Key: AIRFLOW-3839 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3839 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-3585) [AIP-12] Adding Dag edges to the database
[ https://issues.apache.org/jira/browse/AIRFLOW-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof updated AIRFLOW-3585: -- Summary: [AIP-12] Adding Dag edges to the database (was: Adding Dag edges to the database) > [AIP-12] Adding Dag edges to the database > - > > Key: AIRFLOW-3585 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3585 > Project: Apache Airflow > Issue Type: Improvement > Components: database >Affects Versions: 1.10.1 >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > Fix For: 2.0.0 > > > This is part of AIRFLOW-3562. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-3884) Doc checker does accept warnings
[ https://issues.apache.org/jira/browse/AIRFLOW-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof resolved AIRFLOW-3884. --- Resolution: Fixed Fix Version/s: 2.0.0 > Doc checker does accept warnings > > > Key: AIRFLOW-3884 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3884 > Project: Apache Airflow > Issue Type: Bug > Components: Documentation >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3884) Doc checker does accept warnings
Peter van 't Hof created AIRFLOW-3884: - Summary: Doc checker does accept warnings Key: AIRFLOW-3884 URL: https://issues.apache.org/jira/browse/AIRFLOW-3884 Project: Apache Airflow Issue Type: Bug Components: Documentation Reporter: Peter van 't Hof Assignee: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3847) Remove DagBag from /object/task_instances
Peter van 't Hof created AIRFLOW-3847: - Summary: Remove DagBag from /object/task_instances Key: AIRFLOW-3847 URL: https://issues.apache.org/jira/browse/AIRFLOW-3847 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3849) Remove DagBag from action_set_success
Peter van 't Hof created AIRFLOW-3849: - Summary: Remove DagBag from action_set_success Key: AIRFLOW-3849 URL: https://issues.apache.org/jira/browse/AIRFLOW-3849 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3840) Remove DagBag from _mark_task_instance_state
Peter van 't Hof created AIRFLOW-3840: - Summary: Remove DagBag from _mark_task_instance_state Key: AIRFLOW-3840 URL: https://issues.apache.org/jira/browse/AIRFLOW-3840 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3843) Remove DagBag from /tries
Peter van 't Hof created AIRFLOW-3843: - Summary: Remove DagBag from /tries Key: AIRFLOW-3843 URL: https://issues.apache.org/jira/browse/AIRFLOW-3843 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3850) Remove DagBag from TaskInstanceModelView
Peter van 't Hof created AIRFLOW-3850: - Summary: Remove DagBag from TaskInstanceModelView Key: AIRFLOW-3850 URL: https://issues.apache.org/jira/browse/AIRFLOW-3850 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3848) Remove DagBag from action_set_failed
Peter van 't Hof created AIRFLOW-3848: - Summary: Remove DagBag from action_set_failed Key: AIRFLOW-3848 URL: https://issues.apache.org/jira/browse/AIRFLOW-3848 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3846) Remove DagBag from /gantt
Peter van 't Hof created AIRFLOW-3846: - Summary: Remove DagBag from /gantt Key: AIRFLOW-3846 URL: https://issues.apache.org/jira/browse/AIRFLOW-3846 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3845) Remove DagBag from /refresh and /refresh_all
Peter van 't Hof created AIRFLOW-3845: - Summary: Remove DagBag from /refresh and /refresh_all Key: AIRFLOW-3845 URL: https://issues.apache.org/jira/browse/AIRFLOW-3845 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3844) Remove DagBag from /landing_times
Peter van 't Hof created AIRFLOW-3844: - Summary: Remove DagBag from /landing_times Key: AIRFLOW-3844 URL: https://issues.apache.org/jira/browse/AIRFLOW-3844 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3842) Remove DagBag from /duration
Peter van 't Hof created AIRFLOW-3842: - Summary: Remove DagBag from /duration Key: AIRFLOW-3842 URL: https://issues.apache.org/jira/browse/AIRFLOW-3842 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3841) Remove DagBag from /tree
Peter van 't Hof created AIRFLOW-3841: - Summary: Remove DagBag from /tree Key: AIRFLOW-3841 URL: https://issues.apache.org/jira/browse/AIRFLOW-3841 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3839) Remove DagBNag from _mark_dagrun_state_as_success
Peter van 't Hof created AIRFLOW-3839: - Summary: Remove DagBNag from _mark_dagrun_state_as_success Key: AIRFLOW-3839 URL: https://issues.apache.org/jira/browse/AIRFLOW-3839 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3838) Remove DagBag from _mark_dagrun_state_as_failed
Peter van 't Hof created AIRFLOW-3838: - Summary: Remove DagBag from _mark_dagrun_state_as_failed Key: AIRFLOW-3838 URL: https://issues.apache.org/jira/browse/AIRFLOW-3838 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3837) Remove DagBag from /blocked
Peter van 't Hof created AIRFLOW-3837: - Summary: Remove DagBag from /blocked Key: AIRFLOW-3837 URL: https://issues.apache.org/jira/browse/AIRFLOW-3837 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3836) Remove DagBag from /run
Peter van 't Hof created AIRFLOW-3836: - Summary: Remove DagBag from /run Key: AIRFLOW-3836 URL: https://issues.apache.org/jira/browse/AIRFLOW-3836 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3835) Remove DagBag from /task
Peter van 't Hof created AIRFLOW-3835: - Summary: Remove DagBag from /task Key: AIRFLOW-3835 URL: https://issues.apache.org/jira/browse/AIRFLOW-3835 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3830) Remove dag from /dag_details
Peter van 't Hof created AIRFLOW-3830: - Summary: Remove dag from /dag_details Key: AIRFLOW-3830 URL: https://issues.apache.org/jira/browse/AIRFLOW-3830 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3834) Remove DagBag from /log
Peter van 't Hof created AIRFLOW-3834: - Summary: Remove DagBag from /log Key: AIRFLOW-3834 URL: https://issues.apache.org/jira/browse/AIRFLOW-3834 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3833) Remove DagBag from /get_logs_with_metadata
Peter van 't Hof created AIRFLOW-3833: - Summary: Remove DagBag from /get_logs_with_metadata Key: AIRFLOW-3833 URL: https://issues.apache.org/jira/browse/AIRFLOW-3833 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3832) Remove DagBag from /rendered
Peter van 't Hof created AIRFLOW-3832: - Summary: Remove DagBag from /rendered Key: AIRFLOW-3832 URL: https://issues.apache.org/jira/browse/AIRFLOW-3832 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3831) Remove DagBag from /pickle_info
Peter van 't Hof created AIRFLOW-3831: - Summary: Remove DagBag from /pickle_info Key: AIRFLOW-3831 URL: https://issues.apache.org/jira/browse/AIRFLOW-3831 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-3830) Remove DagBag from /dag_details
[ https://issues.apache.org/jira/browse/AIRFLOW-3830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof updated AIRFLOW-3830: -- Summary: Remove DagBag from /dag_details (was: Remove dag from /dag_details) > Remove DagBag from /dag_details > --- > > Key: AIRFLOW-3830 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3830 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-2619) UI is inconsistent with actual data - each refresh shows diffrent data
[ https://issues.apache.org/jira/browse/AIRFLOW-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof reassigned AIRFLOW-2619: - Assignee: Peter van 't Hof > UI is inconsistent with actual data - each refresh shows diffrent data > -- > > Key: AIRFLOW-2619 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2619 > Project: Apache Airflow > Issue Type: Bug > Components: core, ui >Affects Versions: 1.9.0 >Reporter: jack >Assignee: Peter van 't Hof >Priority: Major > Labels: usability > Attachments: BUG1.PNG, bug2.PNG > > > When I reresh my UI everytime it shows different info. > I assume this happens because one of the DAGs (the one that sometimes appear > and sometimes not) isn't defined properly but still - The UI should show it > and indicate the issue rather than sometimes show it and sometimes not. > > See two attached screen shots. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3735) Separated parse dag method
Peter van 't Hof created AIRFLOW-3735: - Summary: Separated parse dag method Key: AIRFLOW-3735 URL: https://issues.apache.org/jira/browse/AIRFLOW-3735 Project: Apache Airflow Issue Type: Improvement Components: core Reporter: Peter van 't Hof Assignee: Peter van 't Hof Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3685) Move licence header check to pre-test stage
Peter van 't Hof created AIRFLOW-3685: - Summary: Move licence header check to pre-test stage Key: AIRFLOW-3685 URL: https://issues.apache.org/jira/browse/AIRFLOW-3685 Project: Apache Airflow Issue Type: Improvement Components: travis Reporter: Peter van 't Hof Assignee: Peter van 't Hof To prevent useless travis resources I move the check to a pre-test stage. Also early detection of header errors. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3674) Adding documentation on official docker images
Peter van 't Hof created AIRFLOW-3674: - Summary: Adding documentation on official docker images Key: AIRFLOW-3674 URL: https://issues.apache.org/jira/browse/AIRFLOW-3674 Project: Apache Airflow Issue Type: Improvement Components: docker Reporter: Peter van 't Hof Assignee: Peter van 't Hof -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3673) Adding automated docker docker push of latest build
Peter van 't Hof created AIRFLOW-3673: - Summary: Adding automated docker docker push of latest build Key: AIRFLOW-3673 URL: https://issues.apache.org/jira/browse/AIRFLOW-3673 Project: Apache Airflow Issue Type: New Feature Components: docker Reporter: Peter van 't Hof Assignee: Peter van 't Hof Together with [~Fokko] we will make an general/official docker images that publish automatic. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3588) Using edges in database from the database
Peter van 't Hof created AIRFLOW-3588: - Summary: Using edges in database from the database Key: AIRFLOW-3588 URL: https://issues.apache.org/jira/browse/AIRFLOW-3588 Project: Apache Airflow Issue Type: Improvement Components: webserver Affects Versions: 1.10.1 Reporter: Peter van 't Hof Assignee: Peter van 't Hof Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-3585) Adding Dag edges to the database
[ https://issues.apache.org/jira/browse/AIRFLOW-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-3585 started by Peter van 't Hof. - > Adding Dag edges to the database > > > Key: AIRFLOW-3585 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3585 > Project: Apache Airflow > Issue Type: Improvement > Components: database >Affects Versions: 1.10.1 >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > Fix For: 2.0.0 > > > This is part of AIRFLOW-3562. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3562) Remove DagBag dependency inside the webserver
[ https://issues.apache.org/jira/browse/AIRFLOW-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730040#comment-16730040 ] Peter van 't Hof commented on AIRFLOW-3562: --- Rendering code only needs the dag directory, so just the files. The object DagBag itself is not required for that. > Remove DagBag dependency inside the webserver > - > > Key: AIRFLOW-3562 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3562 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Affects Versions: 1.10.0, 1.10.1 >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Critical > Fix For: 2.0.0 > > > Currently the webserver does depend on the dag directory and reparsing the > dag files. Because this happens with gunicorn each process has there own > DagBag. The means the webserver is statefull and depending which process is > assigned a request the information can be changed. If all information will > come from the database this will not happen. The only process that should add > dag/dagruns is the scheduler. > The webserver should have enough on only the database. Currently not all > information is already in the database. This could be fixable, already > started this in AIRFLOW-3561. > I do notice this might give some conflicts with the brace change that is > coming. Any suggestion how to approach this? > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-3530) No re-parsing of dag files
[ https://issues.apache.org/jira/browse/AIRFLOW-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-3530 started by Peter van 't Hof. - > No re-parsing of dag files > -- > > Key: AIRFLOW-3530 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3530 > Project: Apache Airflow > Issue Type: Improvement > Components: DAG, scheduler >Affects Versions: 1.10.0, 1.10.1 >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > Fix For: 1.10.2, 2.0.0 > > > Currently the scheduler does parse the dag files each heartbeat. With this > change dags will only be read again when a change is detected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-3575) Split jobs.py into a package
[ https://issues.apache.org/jira/browse/AIRFLOW-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-3575 started by Peter van 't Hof. - > Split jobs.py into a package > > > Key: AIRFLOW-3575 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3575 > Project: Apache Airflow > Issue Type: Improvement >Affects Versions: 1.10.1 >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3573) Remove dagstat table
Peter van 't Hof created AIRFLOW-3573: - Summary: Remove dagstat table Key: AIRFLOW-3573 URL: https://issues.apache.org/jira/browse/AIRFLOW-3573 Project: Apache Airflow Issue Type: Improvement Affects Versions: 1.10.1 Reporter: Peter van 't Hof Assignee: Peter van 't Hof Fix For: 2.0.0 Removing dagstat table, no need anymore after AIRFLOW-3561 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-3561) Improve some views
[ https://issues.apache.org/jira/browse/AIRFLOW-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter van 't Hof resolved AIRFLOW-3561. --- Resolution: Fixed Fix Version/s: 2.0.0 Merged > Improve some views > -- > > Key: AIRFLOW-3561 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3561 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Minor > Fix For: 2.0.0 > > > Some views does interaction with the dag bag while is not needed for the > query. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3562) Remove DagBag dependency inside the webserver
Peter van 't Hof created AIRFLOW-3562: - Summary: Remove DagBag dependency inside the webserver Key: AIRFLOW-3562 URL: https://issues.apache.org/jira/browse/AIRFLOW-3562 Project: Apache Airflow Issue Type: Improvement Components: webserver Affects Versions: 1.10.1, 1.10.0 Reporter: Peter van 't Hof Currently the webserver does depend on the dag directory and reparsing the dag files. Because this happens with gunicorn each process has there own DagBag. The means the webserver is statefull and depending which process is assigned a request the information can be changed. If all information will come from the database this will not happen. The only process that should add dag/dagruns is the scheduler. The webserver should have enough on only the database. Currently not all information is already in the database. This could be fixable, already started this in AIRFLOW-3561. I do notice this might give some conflicts with the brace change that is coming. Any suggestion how to approach this? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
