[GitHub] [airflow] codecov-io edited a comment on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-22 Thread GitBox
codecov-io edited a comment on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-576523641 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7191?src=pr=h1) Report > Merging

[GitHub] [airflow] mik-laj edited a comment on issue #7502: [AIRFLOW-6881][depends on AIRFLOW-6869][WIP] Bulk fetch DAGRun for create_dag_run

2020-02-22 Thread GitBox
mik-laj edited a comment on issue #7502: [AIRFLOW-6881][depends on AIRFLOW-6869][WIP] Bulk fetch DAGRun for create_dag_run URL: https://github.com/apache/airflow/pull/7502#issuecomment-589997296 ```python from datetime import timedelta from airflow.models import DAG from

[GitHub] [airflow] codecov-io commented on issue #7510: [AIRFLOW-6887][WIP] Do not check the state of fresh DAGRun

2020-02-22 Thread GitBox
codecov-io commented on issue #7510: [AIRFLOW-6887][WIP] Do not check the state of fresh DAGRun URL: https://github.com/apache/airflow/pull/7510#issuecomment-590034803 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7510?src=pr=h1) Report > Merging

[GitHub] [airflow] potiuk commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-22 Thread GitBox
potiuk commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-590033385 one transient error - but I restarted it. Looks good !

[GitHub] [airflow] potiuk commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-22 Thread GitBox
potiuk commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-590033396 :crossed_fingers: This is an automated message from the

[jira] [Commented] (AIRFLOW-6887) Do not check the state of fresh DAGRun

2020-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042809#comment-17042809 ] ASF GitHub Bot commented on AIRFLOW-6887: - mik-laj commented on pull request #7510:

[GitHub] [airflow] mik-laj opened a new pull request #7510: [AIRFLOW-6887][WIP] Do not check the state of fresh DAGRun

2020-02-22 Thread GitBox
mik-laj opened a new pull request #7510: [AIRFLOW-6887][WIP] Do not check the state of fresh DAGRun URL: https://github.com/apache/airflow/pull/7510 When I have following DAG File: ```python from datetime import timedelta from airflow.models import DAG from

[jira] [Created] (AIRFLOW-6887) Do not check the state of fresh DAGRun

2020-02-22 Thread Kamil Bregula (Jira)
Kamil Bregula created AIRFLOW-6887: -- Summary: Do not check the state of fresh DAGRun Key: AIRFLOW-6887 URL: https://issues.apache.org/jira/browse/AIRFLOW-6887 Project: Apache Airflow Issue

[GitHub] [airflow] codecov-io edited a comment on issue #7477: [AIRFLOW-6857][WIP] Bulk sync DAGs

2020-02-22 Thread GitBox
codecov-io edited a comment on issue #7477: [AIRFLOW-6857][WIP] Bulk sync DAGs URL: https://github.com/apache/airflow/pull/7477#issuecomment-589466612 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7477?src=pr=h1) Report > Merging

[GitHub] [airflow] vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-22 Thread GitBox
vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-590029643 Alright, rebase is just done! I think it's likely something transient, because the rebase just had changes to UPDATING.md

[GitHub] [airflow] codecov-io edited a comment on issue #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-22 Thread GitBox
codecov-io edited a comment on issue #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#issuecomment-584201606 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7391?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-22 Thread GitBox
codecov-io edited a comment on issue #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#issuecomment-584201606 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7391?src=pr=h1) Report > Merging

[GitHub] [airflow] stale[bot] closed pull request #6795: Adjust the MASTER_URL of spark-submit in SparkSubmitHook

2020-02-22 Thread GitBox
stale[bot] closed pull request #6795: Adjust the MASTER_URL of spark-submit in SparkSubmitHook URL: https://github.com/apache/airflow/pull/6795 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] potiuk merged pull request #7509: [AIRFLOW-XXXX] Fix grammar in UPDATING.md

2020-02-22 Thread GitBox
potiuk merged pull request #7509: [AIRFLOW-] Fix grammar in UPDATING.md URL: https://github.com/apache/airflow/pull/7509 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] potiuk commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-22 Thread GitBox
potiuk commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-590026699 It could be a transient error or some temporary problem with master (despite of the protections we have in place it breaks

[GitHub] [airflow] potiuk commented on issue #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-22 Thread GitBox
potiuk commented on issue #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#issuecomment-590026357 All fixed @kaxil This is an automated message from the Apache Git Service.

[GitHub] [airflow] potiuk commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-22 Thread GitBox
potiuk commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#discussion_r382963742 ## File path: backport_packages/setup_backport_packages.py ## @@ -0,0 +1,251 @@ +# +# Licensed to the

[GitHub] [airflow] potiuk commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-22 Thread GitBox
potiuk commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#discussion_r382963732 ## File path: backport_packages/setup_backport_packages.py ## @@ -0,0 +1,251 @@ +# +# Licensed to the

[GitHub] [airflow] potiuk commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-22 Thread GitBox
potiuk commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#discussion_r382963718 ## File path: backport_packages/setup_backport_packages.py ## @@ -0,0 +1,251 @@ +# +# Licensed to the

[GitHub] [airflow] potiuk commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-22 Thread GitBox
potiuk commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#discussion_r382963626 ## File path: CONTRIBUTING.rst ## @@ -826,3 +826,43 @@ Resources & Links - `Airflow’s official

[GitHub] [airflow] potiuk commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-22 Thread GitBox
potiuk commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#discussion_r382963632 ## File path: scripts/ci/pre_commit_setup_cfg_file.sh ## @@ -0,0 +1,44 @@ +#!/usr/bin/env bash +#

[GitHub] [airflow] potiuk commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
potiuk commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382963463 ## File path: airflow/utils/helpers.py ## @@ -30,6 +30,8 @@ from

[GitHub] [airflow] potiuk commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
potiuk commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382963207 ## File path: airflow/jobs/scheduler_job.py ## @@ -1018,7 +1024,7 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
potiuk commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382963088 ## File path: airflow/jobs/backfill_job.py ## @@ -203,18 +204,18 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
potiuk commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382963240 ## File path: airflow/jobs/scheduler_job.py ## @@ -1162,8 +1170,8 @@ def

[GitHub] [airflow] yuqian90 commented on a change in pull request #7470: [AIRFLOW-6834] Fix flaky test_scheduler_job by sorting TaskInstance

2020-02-22 Thread GitBox
yuqian90 commented on a change in pull request #7470: [AIRFLOW-6834] Fix flaky test_scheduler_job by sorting TaskInstance URL: https://github.com/apache/airflow/pull/7470#discussion_r382962334 ## File path: airflow/models/dagrun.py ## @@ -201,7 +201,7 @@ def

[GitHub] [airflow] codecov-io edited a comment on issue #7470: [AIRFLOW-6834] Fix flaky test_scheduler_job by sorting TaskInstance

2020-02-22 Thread GitBox
codecov-io edited a comment on issue #7470: [AIRFLOW-6834] Fix flaky test_scheduler_job by sorting TaskInstance URL: https://github.com/apache/airflow/pull/7470#issuecomment-588602591 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7470?src=pr=h1) Report > Merging

[GitHub] [airflow] kaxil commented on a change in pull request #7123: [AIRFLOW-6062] Executor would only delete workers in its own namespace

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7123: [AIRFLOW-6062] Executor would only delete workers in its own namespace URL: https://github.com/apache/airflow/pull/7123#discussion_r382954805 ## File path: airflow/executors/kubernetes_executor.py ## @@ -344,27 +348,33 @@

[GitHub] [airflow] kaxil commented on a change in pull request #7123: [AIRFLOW-6062] Executor would only delete workers in its own namespace

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7123: [AIRFLOW-6062] Executor would only delete workers in its own namespace URL: https://github.com/apache/airflow/pull/7123#discussion_r382954798 ## File path: airflow/executors/kubernetes_executor.py ## @@ -344,27 +348,33 @@

[GitHub] [airflow] kaxil commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#discussion_r382954379 ## File path: backport_packages/setup_backport_packages.py ## @@ -0,0 +1,251 @@ +# +# Licensed to the

[GitHub] [airflow] kaxil commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#discussion_r382954365 ## File path: backport_packages/setup_backport_packages.py ## @@ -0,0 +1,251 @@ +# +# Licensed to the

[GitHub] [airflow] kaxil commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#discussion_r382954347 ## File path: backport_packages/setup_backport_packages.py ## @@ -0,0 +1,251 @@ +# +# Licensed to the

[GitHub] [airflow] kaxil commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#discussion_r382954231 ## File path: scripts/ci/pre_commit_setup_cfg_file.sh ## @@ -0,0 +1,44 @@ +#!/usr/bin/env bash +# Licensed

[GitHub] [airflow] kaxil commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#discussion_r382953918 ## File path: CONTRIBUTING.rst ## @@ -826,3 +826,43 @@ Resources & Links - `Airflow’s official

[GitHub] [airflow] codecov-io commented on issue #7490: [AIRFLOW-6860] Default ignore_first_depends_on_past to True

2020-02-22 Thread GitBox
codecov-io commented on issue #7490: [AIRFLOW-6860] Default ignore_first_depends_on_past to True URL: https://github.com/apache/airflow/pull/7490#issuecomment-590014447 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7490?src=pr=h1) Report > Merging

[GitHub] [airflow] kaxil commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382953272 ## File path: airflow/jobs/backfill_job.py ## @@ -203,18 +204,18 @@ def

[GitHub] [airflow] vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-22 Thread GitBox
vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-590012811 @potiuk for the recent tests, I'm seeing a redirect (302) for an address that (used to be?) 200? I don't see any singularity

[jira] [Commented] (AIRFLOW-6204) Add GCP system tests helper

2020-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042756#comment-17042756 ] ASF GitHub Bot commented on AIRFLOW-6204: - stale[bot] commented on pull request #6994:

[GitHub] [airflow] stale[bot] closed pull request #6994: [AIRFLOW-6204][POC] Remove separate GCP system tests helper

2020-02-22 Thread GitBox
stale[bot] closed pull request #6994: [AIRFLOW-6204][POC] Remove separate GCP system tests helper URL: https://github.com/apache/airflow/pull/6994 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success URL: https://github.com/apache/airflow/pull/7507#discussion_r382946447 ## File path: tests/executors/test_kubernetes_executor.py ## @@ -270,12 +270,27 @@ def

[GitHub] [airflow] kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success URL: https://github.com/apache/airflow/pull/7507#discussion_r382946377 ## File path: airflow/executors/kubernetes_executor.py ## @@ -848,10 +850,13 @@ def sync(self) -> None:

[GitHub] [airflow] kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success URL: https://github.com/apache/airflow/pull/7507#discussion_r382946433 ## File path: tests/executors/test_kubernetes_executor.py ## @@ -270,12 +270,27 @@ def

[GitHub] [airflow] kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success URL: https://github.com/apache/airflow/pull/7507#discussion_r382946408 ## File path: airflow/executors/kubernetes_executor.py ## @@ -848,10 +850,13 @@ def sync(self) -> None:

[GitHub] [airflow] kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success URL: https://github.com/apache/airflow/pull/7507#discussion_r382946377 ## File path: airflow/executors/kubernetes_executor.py ## @@ -848,10 +850,13 @@ def sync(self) -> None:

[GitHub] [airflow] kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success URL: https://github.com/apache/airflow/pull/7507#discussion_r382946358 ## File path: airflow/executors/kubernetes_executor.py ## @@ -96,6 +96,8 @@ def __init__(self): # pylint:

[GitHub] [airflow] kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success URL: https://github.com/apache/airflow/pull/7507#discussion_r382946358 ## File path: airflow/executors/kubernetes_executor.py ## @@ -96,6 +96,8 @@ def __init__(self): # pylint:

[GitHub] [airflow] kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success URL: https://github.com/apache/airflow/pull/7507#discussion_r382946207 ## File path: airflow/executors/kubernetes_executor.py ## @@ -848,10 +850,13 @@ def sync(self) -> None:

[GitHub] [airflow] kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success URL: https://github.com/apache/airflow/pull/7507#discussion_r382946219 ## File path: airflow/executors/kubernetes_executor.py ## @@ -848,10 +850,13 @@ def sync(self) -> None:

[GitHub] [airflow] kaxil opened a new pull request #7509: [AIRFLOW-XXXX] Fix grammar in UPDATING.md

2020-02-22 Thread GitBox
kaxil opened a new pull request #7509: [AIRFLOW-] Fix grammar in UPDATING.md URL: https://github.com/apache/airflow/pull/7509 "supersede" to "superseded" --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the

[GitHub] [airflow] kaxil commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances URL: https://github.com/apache/airflow/pull/7489#discussion_r382944080 ## File path: airflow/jobs/scheduler_job.py ## @@ -699,23 +727,30 @@ def

[GitHub] [airflow] codecov-io commented on issue #7508: [AIRFLOW-6886] Cleanup the SageMakerTrainingOperator result

2020-02-22 Thread GitBox
codecov-io commented on issue #7508: [AIRFLOW-6886] Cleanup the SageMakerTrainingOperator result URL: https://github.com/apache/airflow/pull/7508#issuecomment-590001224 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7508?src=pr=h1) Report > Merging

[GitHub] [airflow] stale[bot] commented on issue #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection

2020-02-22 Thread GitBox
stale[bot] commented on issue #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection URL: https://github.com/apache/airflow/pull/7075#issuecomment-59164 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382941957 ## File path: airflow/models/sqla_models.py ## @@ -0,0 +1,13 @@ +"""Load

[GitHub] [airflow] mik-laj commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances

2020-02-22 Thread GitBox
mik-laj commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances URL: https://github.com/apache/airflow/pull/7489#discussion_r382941484 ## File path: airflow/jobs/scheduler_job.py ## @@ -699,23 +727,30 @@ def

[GitHub] [airflow] kaxil commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances URL: https://github.com/apache/airflow/pull/7489#discussion_r382941092 ## File path: airflow/models/dagrun.py ## @@ -125,14 +125,16 @@ def

[GitHub] [airflow] kaxil commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances

2020-02-22 Thread GitBox
kaxil commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances URL: https://github.com/apache/airflow/pull/7489#discussion_r382940926 ## File path: airflow/jobs/scheduler_job.py ## @@ -699,23 +727,30 @@ def

[GitHub] [airflow] mik-laj commented on issue #7502: [AIRFLOW-6881][depends on AIRFLOW-6869][WIP] Bulk fetch DAGRun for create_dag_run

2020-02-22 Thread GitBox
mik-laj commented on issue #7502: [AIRFLOW-6881][depends on AIRFLOW-6869][WIP] Bulk fetch DAGRun for create_dag_run URL: https://github.com/apache/airflow/pull/7502#issuecomment-589997296 ``` from datetime import timedelta from airflow.models import DAG from

[GitHub] [airflow] mik-laj commented on issue #7502: [AIRFLOW-6881][depends on AIRFLOW-6869][WIP] Bulk fetch DAGRun for create_dag_run

2020-02-22 Thread GitBox
mik-laj commented on issue #7502: [AIRFLOW-6881][depends on AIRFLOW-6869][WIP] Bulk fetch DAGRun for create_dag_run URL: https://github.com/apache/airflow/pull/7502#issuecomment-589994462 @tooptoop4 The number of tasks does not affect this, but I will also check this case and give values

[GitHub] [airflow] mik-laj commented on issue #7501: [AIRFLOW-6880][WIP] Do not check task instances for new DAGRuns

2020-02-22 Thread GitBox
mik-laj commented on issue #7501: [AIRFLOW-6880][WIP] Do not check task instances for new DAGRuns URL: https://github.com/apache/airflow/pull/7501#issuecomment-589994328 It won't work, but we need to add tests to detect these situations. TaskInstance can have several related DAGRuns.

[GitHub] [airflow] potiuk commented on a change in pull request #7506: [AIRFLOW-6877] [WIP] check cross providers dependencies depends on [AIRFLOW-6663]

2020-02-22 Thread GitBox
potiuk commented on a change in pull request #7506: [AIRFLOW-6877] [WIP] check cross providers dependencies depends on [AIRFLOW-6663] URL: https://github.com/apache/airflow/pull/7506#discussion_r382938276 ## File path: airflow/providers/amazon/aws/operators/s3_to_redshift.py

[GitHub] [airflow] potiuk commented on a change in pull request #7506: [AIRFLOW-6877] [WIP] check cross providers dependencies depends on [AIRFLOW-6663]

2020-02-22 Thread GitBox
potiuk commented on a change in pull request #7506: [AIRFLOW-6877] [WIP] check cross providers dependencies depends on [AIRFLOW-6663] URL: https://github.com/apache/airflow/pull/7506#discussion_r382938156 ## File path: airflow/providers/google/cloud/operators/sftp_to_gcs.py

[GitHub] [airflow] potiuk commented on a change in pull request #7506: [AIRFLOW-6877] [WIP] check cross providers dependencies depends on [AIRFLOW-6663]

2020-02-22 Thread GitBox
potiuk commented on a change in pull request #7506: [AIRFLOW-6877] [WIP] check cross providers dependencies depends on [AIRFLOW-6663] URL: https://github.com/apache/airflow/pull/7506#discussion_r382937981 ## File path:

[GitHub] [airflow] tooptoop4 commented on issue #7502: [AIRFLOW-6881][depends on AIRFLOW-6869][WIP] Bulk fetch DAGRun for create_dag_run

2020-02-22 Thread GitBox
tooptoop4 commented on issue #7502: [AIRFLOW-6881][depends on AIRFLOW-6869][WIP] Bulk fetch DAGRun for create_dag_run URL: https://github.com/apache/airflow/pull/7502#issuecomment-589992285 can u get perf timings before/after for dags with 500 tasks in them? with these types of changes

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7506: [AIRFLOW-6877] [WIP] check cross providers dependencies depends on [AIRFLOW-6663]

2020-02-22 Thread GitBox
nuclearpinguin commented on a change in pull request #7506: [AIRFLOW-6877] [WIP] check cross providers dependencies depends on [AIRFLOW-6663] URL: https://github.com/apache/airflow/pull/7506#discussion_r382935850 ## File path:

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7506: [AIRFLOW-6877] [WIP] check cross providers dependencies depends on [AIRFLOW-6663]

2020-02-22 Thread GitBox
nuclearpinguin commented on a change in pull request #7506: [AIRFLOW-6877] [WIP] check cross providers dependencies depends on [AIRFLOW-6663] URL: https://github.com/apache/airflow/pull/7506#discussion_r382935743 ## File path: airflow/providers/google/cloud/operators/sftp_to_gcs.py

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7506: [AIRFLOW-6877] [WIP] check cross providers dependencies depends on [AIRFLOW-6663]

2020-02-22 Thread GitBox
nuclearpinguin commented on a change in pull request #7506: [AIRFLOW-6877] [WIP] check cross providers dependencies depends on [AIRFLOW-6663] URL: https://github.com/apache/airflow/pull/7506#discussion_r382935600 ## File path:

[jira] [Commented] (AIRFLOW-6886) Cleanup the SageMakerTrainingOperator result

2020-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042690#comment-17042690 ] ASF GitHub Bot commented on AIRFLOW-6886: - BasPH commented on pull request #7508:

[GitHub] [airflow] BasPH opened a new pull request #7508: [AIRFLOW-6886] Cleanup the SageMakerTrainingOperator result

2020-02-22 Thread GitBox
BasPH opened a new pull request #7508: [AIRFLOW-6886] Cleanup the SageMakerTrainingOperator result URL: https://github.com/apache/airflow/pull/7508 The SageMakerTrainingOperator returns the response of `hook.describe_training_job()` wrapped in a dict with key `Training`. I don't see it

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success

2020-02-22 Thread GitBox
nuclearpinguin commented on a change in pull request #7507: [AIRFLOW-6885] Delete worker on success URL: https://github.com/apache/airflow/pull/7507#discussion_r382935382 ## File path: airflow/executors/kubernetes_executor.py ## @@ -848,10 +850,13 @@ def sync(self) ->

[jira] [Updated] (AIRFLOW-6886) Cleanup the SageMakerTrainingOperator result

2020-02-22 Thread Bas Harenslak (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bas Harenslak updated AIRFLOW-6886: --- Issue Type: Improvement (was: Bug) > Cleanup the SageMakerTrainingOperator result >

[jira] [Assigned] (AIRFLOW-6886) Cleanup the SageMakerTrainingOperator result

2020-02-22 Thread Bas Harenslak (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bas Harenslak reassigned AIRFLOW-6886: -- Assignee: Bas Harenslak > Cleanup the SageMakerTrainingOperator result >

[jira] [Created] (AIRFLOW-6886) Cleanup the SageMakerTrainingOperator result

2020-02-22 Thread Bas Harenslak (Jira)
Bas Harenslak created AIRFLOW-6886: -- Summary: Cleanup the SageMakerTrainingOperator result Key: AIRFLOW-6886 URL: https://issues.apache.org/jira/browse/AIRFLOW-6886 Project: Apache Airflow

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382932914 ## File path: airflow/models/sqla_models.py ## @@ -0,0 +1,13 @@ +"""Load

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382933161 ## File path: airflow/jobs/scheduler_job.py ## @@ -1018,7 +1024,7 @@ def

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382933152 ## File path: airflow/jobs/scheduler_job.py ## @@ -29,7 +29,7 @@ from

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382933120 ## File path: airflow/jobs/local_task_job.py ## @@ -57,6 +62,7 @@ def

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382933042 ## File path: airflow/utils/db.py ## @@ -23,16 +23,18 @@ from airflow

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382932981 ## File path: airflow/utils/helpers.py ## @@ -30,6 +30,8 @@ from

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382933009 ## File path: airflow/utils/db.py ## @@ -23,16 +23,18 @@ from airflow

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
nuclearpinguin commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382932914 ## File path: airflow/models/sqla_models.py ## @@ -0,0 +1,13 @@ +"""Load

[jira] [Updated] (AIRFLOW-6885) Add option to only delete KubernetesExecutor pods on successful task completion

2020-02-22 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman updated AIRFLOW-6885: - Fix Version/s: 1.10.10 > Add option to only delete KubernetesExecutor pods on

[GitHub] [airflow] dimberman opened a new pull request #7507: Users now have the option to only delete worker pods when they are su…

2020-02-22 Thread GitBox
dimberman opened a new pull request #7507: Users now have the option to only delete worker pods when they are su… URL: https://github.com/apache/airflow/pull/7507 …ccessful --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make

[jira] [Created] (AIRFLOW-6885) Add option to only delete KubernetesExecutor pods on successful task completion

2020-02-22 Thread Daniel Imberman (Jira)
Daniel Imberman created AIRFLOW-6885: Summary: Add option to only delete KubernetesExecutor pods on successful task completion Key: AIRFLOW-6885 URL: https://issues.apache.org/jira/browse/AIRFLOW-6885

[jira] [Assigned] (AIRFLOW-6815) Dag Graph View Javascript Error

2020-02-22 Thread Andrew (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew reassigned AIRFLOW-6815: --- Assignee: Andrew > Dag Graph View Javascript Error > --- > >

[jira] [Commented] (AIRFLOW-6815) Dag Graph View Javascript Error

2020-02-22 Thread Andrew (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042652#comment-17042652 ] Andrew commented on AIRFLOW-6815: - Hey I'm unable to recreate this issue on my machine. Which version

[GitHub] [airflow] potiuk commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
potiuk commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382928307 ## File path: airflow/models/sqla_models.py ## @@ -0,0 +1,13 @@ +"""Load

[GitHub] [airflow] potiuk commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
potiuk commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382928208 ## File path: airflow/models/sqla_models.py ## @@ -0,0 +1,13 @@ +"""Load

[GitHub] [airflow] potiuk commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
potiuk commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382928208 ## File path: airflow/models/sqla_models.py ## @@ -0,0 +1,13 @@ +"""Load

[GitHub] [airflow] potiuk commented on issue #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-22 Thread GitBox
potiuk commented on issue #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#issuecomment-589979050 All warnings and Travis fixed with this one - and it is really nicer and more separated. @ashb @mik-laj @nuclearpinguin @kaxil . I would love

[GitHub] [airflow] potiuk commented on issue #7506: [AIRFLOW-6877] [WIP] check cross providers dependencies depends on [AIRFLOW-6663]

2020-02-22 Thread GitBox
potiuk commented on issue #7506: [AIRFLOW-6877] [WIP] check cross providers dependencies depends on [AIRFLOW-6663] URL: https://github.com/apache/airflow/pull/7506#issuecomment-589978777 @ashb @feluelle @Fokko @mik-laj @nuclearpinguin @kaxil @BasPH -> I think this one is really important

[GitHub] [airflow] ashb commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
ashb commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382927501 ## File path: airflow/models/sqla_models.py ## @@ -0,0 +1,13 @@ +"""Load SQLAlchemy

[GitHub] [airflow] ashb commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
ashb commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382927501 ## File path: airflow/models/sqla_models.py ## @@ -0,0 +1,13 @@ +"""Load SQLAlchemy

[GitHub] [airflow] ashb commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
ashb commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382927403 ## File path: airflow/models/sqla_models.py ## @@ -0,0 +1,13 @@ +"""Load SQLAlchemy

[GitHub] [airflow] ashb commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
ashb commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382927352 ## File path: airflow/models/sqla_models.py ## @@ -0,0 +1,13 @@ +"""Load SQLAlchemy

[GitHub] [airflow] ashb commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
ashb commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382927101 ## File path: airflow/jobs/scheduler_job.py ## @@ -1607,16 +1615,14 @@ def

[GitHub] [airflow] ashb commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
ashb commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382927081 ## File path: airflow/jobs/scheduler_job.py ## @@ -1584,12 +1593,11 @@ def

[GitHub] [airflow] ashb commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
ashb commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382927022 ## File path: airflow/jobs/scheduler_job.py ## @@ -1018,7 +1024,7 @@ def

[GitHub] [airflow] ashb commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible

2020-02-22 Thread GitBox
ashb commented on a change in pull request #7484: [AIRFLOW-6864][depends on 6863] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#discussion_r382926937 ## File path: airflow/jobs/scheduler_job.py ## @@ -29,7 +29,7 @@ from datetime

[jira] [Commented] (AIRFLOW-6877) Add backport packages dependency warnings

2020-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042641#comment-17042641 ] ASF GitHub Bot commented on AIRFLOW-6877: - potiuk commented on pull request #7506:

[GitHub] [airflow] potiuk opened a new pull request #7506: [AIRFLOW-6877] check cross providers dependencies

2020-02-22 Thread GitBox
potiuk opened a new pull request #7506: [AIRFLOW-6877] check cross providers dependencies URL: https://github.com/apache/airflow/pull/7506 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before

  1   2   3   >