[GitHub] [airflow] Fokko commented on a change in pull request #7502: [AIRFLOW-6881][depends on AIRFLOW-6869][WIP] Bulk fetch DAGRun for create_dag_run

2020-02-24 Thread GitBox
Fokko commented on a change in pull request #7502: [AIRFLOW-6881][depends on AIRFLOW-6869][WIP] Bulk fetch DAGRun for create_dag_run URL: https://github.com/apache/airflow/pull/7502#discussion_r383708347 ## File path: airflow/models/dagrun.py ## @@ -125,14 +125,16 @@ def

[GitHub] [airflow] Fokko commented on a change in pull request #7502: [AIRFLOW-6881][depends on AIRFLOW-6869][WIP] Bulk fetch DAGRun for create_dag_run

2020-02-24 Thread GitBox
Fokko commented on a change in pull request #7502: [AIRFLOW-6881][depends on AIRFLOW-6869][WIP] Bulk fetch DAGRun for create_dag_run URL: https://github.com/apache/airflow/pull/7502#discussion_r383708424 ## File path: airflow/models/dagrun.py ## @@ -125,14 +125,16 @@ def

[GitHub] [airflow] Fokko commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances

2020-02-24 Thread GitBox
Fokko commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances URL: https://github.com/apache/airflow/pull/7489#discussion_r383706497 ## File path: airflow/jobs/scheduler_job.py ## @@ -684,11 +687,36 @@ def

[GitHub] [airflow] Fokko commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances

2020-02-24 Thread GitBox
Fokko commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances URL: https://github.com/apache/airflow/pull/7489#discussion_r383706008 ## File path: airflow/jobs/scheduler_job.py ## @@ -631,16 +632,16 @@ def

[GitHub] [airflow] Fokko commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances

2020-02-24 Thread GitBox
Fokko commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances URL: https://github.com/apache/airflow/pull/7489#discussion_r383705945 ## File path: airflow/jobs/scheduler_job.py ## @@ -631,16 +632,16 @@ def

[GitHub] [airflow] Fokko commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances

2020-02-24 Thread GitBox
Fokko commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances URL: https://github.com/apache/airflow/pull/7489#discussion_r383706748 ## File path: airflow/models/dagrun.py ## @@ -125,14 +125,16 @@ def

[GitHub] [airflow] Fokko commented on a change in pull request #7481: [AIRFLOW-6862][WIP] Do not check the freshness of fresh DAG

2020-02-24 Thread GitBox
Fokko commented on a change in pull request #7481: [AIRFLOW-6862][WIP] Do not check the freshness of fresh DAG URL: https://github.com/apache/airflow/pull/7481#discussion_r383705120 ## File path: airflow/jobs/scheduler_job.py ## @@ -668,16 +668,14 @@ def

[GitHub] [airflow] Fokko commented on a change in pull request #7481: [AIRFLOW-6862][WIP] Do not check the freshness of fresh DAG

2020-02-24 Thread GitBox
Fokko commented on a change in pull request #7481: [AIRFLOW-6862][WIP] Do not check the freshness of fresh DAG URL: https://github.com/apache/airflow/pull/7481#discussion_r383705449 ## File path: tests/jobs/test_scheduler_job.py ## @@ -1010,7 +1010,7 @@ def

[GitHub] [airflow] mik-laj commented on a change in pull request #7476: [AIRFLOW-6856][depends on AIRFLOW-6862][WIP] Bulk fetch paused_dag_ids

2020-02-24 Thread GitBox
mik-laj commented on a change in pull request #7476: [AIRFLOW-6856][depends on AIRFLOW-6862][WIP] Bulk fetch paused_dag_ids URL: https://github.com/apache/airflow/pull/7476#discussion_r383696220 ## File path: airflow/jobs/scheduler_job.py ## @@ -672,33 +672,24 @@ def

[GitHub] [airflow] mik-laj commented on a change in pull request #7476: [AIRFLOW-6856][depends on AIRFLOW-6862][WIP] Bulk fetch paused_dag_ids

2020-02-24 Thread GitBox
mik-laj commented on a change in pull request #7476: [AIRFLOW-6856][depends on AIRFLOW-6862][WIP] Bulk fetch paused_dag_ids URL: https://github.com/apache/airflow/pull/7476#discussion_r383695112 ## File path: airflow/jobs/scheduler_job.py ## @@ -672,33 +672,24 @@ def

[GitHub] [airflow] mik-laj commented on a change in pull request #7476: [AIRFLOW-6856][depends on AIRFLOW-6862][WIP] Bulk fetch paused_dag_ids

2020-02-24 Thread GitBox
mik-laj commented on a change in pull request #7476: [AIRFLOW-6856][depends on AIRFLOW-6862][WIP] Bulk fetch paused_dag_ids URL: https://github.com/apache/airflow/pull/7476#discussion_r383695112 ## File path: airflow/jobs/scheduler_job.py ## @@ -672,33 +672,24 @@ def

[jira] [Updated] (AIRFLOW-6912) Airflow unable to run concurrent ssh tasks (up to 27) in a single dag

2020-02-24 Thread Nguyen Lam Phuc (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nguyen Lam Phuc updated AIRFLOW-6912: - Description: *Current working Airflow version:* 1.10.7 *Environment:* Kubernetes

[jira] [Updated] (AIRFLOW-6912) Airflow unable to run concurrent ssh tasks (up to 27) in a single dag

2020-02-24 Thread Nguyen Lam Phuc (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nguyen Lam Phuc updated AIRFLOW-6912: - Description: *Current working Airflow version:* 1.10.7 *Environment:* Kubernetes

[jira] [Created] (AIRFLOW-6912) Airflow unable to run concurrent ssh tasks (up to 27) in a single dag

2020-02-24 Thread Nguyen Lam Phuc (Jira)
Nguyen Lam Phuc created AIRFLOW-6912: Summary: Airflow unable to run concurrent ssh tasks (up to 27) in a single dag Key: AIRFLOW-6912 URL: https://issues.apache.org/jira/browse/AIRFLOW-6912

[jira] [Assigned] (AIRFLOW-5485) scheduler_job: replace == None with is_() comparison

2020-02-24 Thread Saurabh Dhupar (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saurabh Dhupar reassigned AIRFLOW-5485: --- Assignee: Saurabh Dhupar > scheduler_job: replace == None with is_() comparison >

[GitHub] [airflow] Fokko commented on a change in pull request #7476: [AIRFLOW-6856][depends on AIRFLOW-6862][WIP] Bulk fetch paused_dag_ids

2020-02-24 Thread GitBox
Fokko commented on a change in pull request #7476: [AIRFLOW-6856][depends on AIRFLOW-6862][WIP] Bulk fetch paused_dag_ids URL: https://github.com/apache/airflow/pull/7476#discussion_r383692759 ## File path: airflow/jobs/scheduler_job.py ## @@ -672,33 +672,24 @@ def

[GitHub] [airflow] codecov-io commented on issue #7527: [AIRFLOW-6907] Simplify SchedulerJob

2020-02-24 Thread GitBox
codecov-io commented on issue #7527: [AIRFLOW-6907] Simplify SchedulerJob URL: https://github.com/apache/airflow/pull/7527#issuecomment-590717059 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7527?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base

[jira] [Commented] (AIRFLOW-5178) AWS Batch Operator improvement to support batch job parameters

2020-02-24 Thread Saurabh Dhupar (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044167#comment-17044167 ] Saurabh Dhupar commented on AIRFLOW-5178: - Looks like someone already worked on it - 

[jira] [Assigned] (AIRFLOW-5178) AWS Batch Operator improvement to support batch job parameters

2020-02-24 Thread Saurabh Dhupar (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saurabh Dhupar reassigned AIRFLOW-5178: --- Assignee: Saurabh Dhupar > AWS Batch Operator improvement to support batch job

[GitHub] [airflow] baolsen commented on issue #7443: [AIRFLOW-6824] EMRAddStepsOperator problem with multi-step XCom

2020-02-24 Thread GitBox
baolsen commented on issue #7443: [AIRFLOW-6824] EMRAddStepsOperator problem with multi-step XCom URL: https://github.com/apache/airflow/pull/7443#issuecomment-590687083 Hi @mik-laj please review and merge :) This is an

[GitHub] [airflow] sarah-johnson commented on a change in pull request #7428: [AIRFLOW-4526] Poll k8s logs to avoid problem with kubernetes logs --follow

2020-02-24 Thread GitBox
sarah-johnson commented on a change in pull request #7428: [AIRFLOW-4526] Poll k8s logs to avoid problem with kubernetes logs --follow URL: https://github.com/apache/airflow/pull/7428#discussion_r383651163 ## File path: airflow/kubernetes/pod_launcher.py ## @@ -162,22

[GitHub] [airflow] sarah-johnson commented on a change in pull request #7428: [AIRFLOW-4526] Poll k8s logs to avoid problem with kubernetes logs --follow

2020-02-24 Thread GitBox
sarah-johnson commented on a change in pull request #7428: [AIRFLOW-4526] Poll k8s logs to avoid problem with kubernetes logs --follow URL: https://github.com/apache/airflow/pull/7428#discussion_r383650742 ## File path: airflow/kubernetes/pod_launcher.py ## @@ -162,22

[jira] [Created] (AIRFLOW-6911) Airflow worker with Kube Executor doesn't open worker_log_server_port

2020-02-24 Thread Sourabh Bajaj (Jira)
Sourabh Bajaj created AIRFLOW-6911: -- Summary: Airflow worker with Kube Executor doesn't open worker_log_server_port Key: AIRFLOW-6911 URL: https://issues.apache.org/jira/browse/AIRFLOW-6911 Project:

[GitHub] [airflow] codecov-io commented on issue #7531: [AIRFLOW-6910] Fix kill_zombie method call

2020-02-24 Thread GitBox
codecov-io commented on issue #7531: [AIRFLOW-6910] Fix kill_zombie method call URL: https://github.com/apache/airflow/pull/7531#issuecomment-590645698 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7531?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base

[GitHub] [airflow] kaxil opened a new pull request #7532: [AIRFLOW-XXXX] Fix outdated doc on settings.policy

2020-02-24 Thread GitBox
kaxil opened a new pull request #7532: [AIRFLOW-] Fix outdated doc on settings.policy URL: https://github.com/apache/airflow/pull/7532 The policy is for **task** and not **task_instance**. Original commit that introduced this feature:

[jira] [Commented] (AIRFLOW-6384) Custom Spark Livy Operator

2020-02-24 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044018#comment-17044018 ] Kaxil Naik commented on AIRFLOW-6384: - This was added in

[GitHub] [airflow] KevinYang21 commented on issue #7269: [AIRFLOW-6651] Add Redis Heartbeat option

2020-02-24 Thread GitBox
KevinYang21 commented on issue #7269: [AIRFLOW-6651] Add Redis Heartbeat option URL: https://github.com/apache/airflow/pull/7269#issuecomment-590627923 @ashb Good question. I don't have an exact performance A/B test result number for this change( we're still building that framework). But

[jira] [Commented] (AIRFLOW-6910) Fix kill_zombie method call

2020-02-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044010#comment-17044010 ] ASF GitHub Bot commented on AIRFLOW-6910: - mik-laj commented on pull request #7531:

[GitHub] [airflow] mik-laj opened a new pull request #7531: [AIRFLOW-6910] Fix kill_zombie method call

2020-02-24 Thread GitBox
mik-laj opened a new pull request #7531: [AIRFLOW-6910] Fix kill_zombie method call URL: https://github.com/apache/airflow/pull/7531 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before creating PR:

[jira] [Created] (AIRFLOW-6910) Fix kill_zombie method call

2020-02-24 Thread Kamil Bregula (Jira)
Kamil Bregula created AIRFLOW-6910: -- Summary: Fix kill_zombie method call Key: AIRFLOW-6910 URL: https://issues.apache.org/jira/browse/AIRFLOW-6910 Project: Apache Airflow Issue Type: New

[GitHub] [airflow] yuqian90 commented on a change in pull request #7464: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation

2020-02-24 Thread GitBox
yuqian90 commented on a change in pull request #7464: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation URL: https://github.com/apache/airflow/pull/7464#discussion_r383601716 ## File path: airflow/operators/python.py ## @@ -167,6 +167,34 @@ def

[jira] [Assigned] (AIRFLOW-6797) Create policy hooks for DAGs

2020-02-24 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik reassigned AIRFLOW-6797: --- Assignee: Kaxil Naik > Create policy hooks for DAGs > > >

[GitHub] [airflow] kaxil opened a new pull request #7530: [AIRFLOW-XXXX] Update docs on starting Kubernetes tests

2020-02-24 Thread GitBox
kaxil opened a new pull request #7530: [AIRFLOW-] Update docs on starting Kubernetes tests URL: https://github.com/apache/airflow/pull/7530 Update docs on starting Kubernetes tests --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg)

[jira] [Commented] (AIRFLOW-6909) Prepare backport packages on post-test stage

2020-02-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043999#comment-17043999 ] ASF GitHub Bot commented on AIRFLOW-6909: - mik-laj commented on pull request #7529:

[GitHub] [airflow] mik-laj opened a new pull request #7529: [AIRFLOW-6909] Prepare backport packages on post-test stage

2020-02-24 Thread GitBox
mik-laj opened a new pull request #7529: [AIRFLOW-6909] Prepare backport packages on post-test stage URL: https://github.com/apache/airflow/pull/7529 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below

[jira] [Created] (AIRFLOW-6909) Prepare backport packages on post-test stage

2020-02-24 Thread Kamil Bregula (Jira)
Kamil Bregula created AIRFLOW-6909: -- Summary: Prepare backport packages on post-test stage Key: AIRFLOW-6909 URL: https://issues.apache.org/jira/browse/AIRFLOW-6909 Project: Apache Airflow

[jira] [Commented] (AIRFLOW-6908) Lazy load AirflowException

2020-02-24 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043997#comment-17043997 ] ASF subversion and git services commented on AIRFLOW-6908: -- Commit

[jira] [Commented] (AIRFLOW-6908) Lazy load AirflowException

2020-02-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043996#comment-17043996 ] ASF GitHub Bot commented on AIRFLOW-6908: - mik-laj commented on pull request #7528:

[jira] [Closed] (AIRFLOW-6908) Lazy load AirflowException

2020-02-24 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula closed AIRFLOW-6908. -- Fix Version/s: 2.0.0 Resolution: Fixed > Lazy load AirflowException >

[GitHub] [airflow] mik-laj merged pull request #7528: [AIRFLOW-6908] Lazy load AirflowException

2020-02-24 Thread GitBox
mik-laj merged pull request #7528: [AIRFLOW-6908] Lazy load AirflowException URL: https://github.com/apache/airflow/pull/7528 This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Commented] (AIRFLOW-6908) Lazy load AirflowException

2020-02-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043994#comment-17043994 ] ASF GitHub Bot commented on AIRFLOW-6908: - mik-laj commented on pull request #7528:

[GitHub] [airflow] mik-laj opened a new pull request #7528: [AIRFLOW-6908] Lazy load AirflowException

2020-02-24 Thread GitBox
mik-laj opened a new pull request #7528: [AIRFLOW-6908] Lazy load AirflowException URL: https://github.com/apache/airflow/pull/7528 It is also often imported and can damage a lot of DAGs. For example, we have a broken master now. --- Issue link: WILL BE INSERTED BY

[jira] [Created] (AIRFLOW-6908) Lazy load AirflowException

2020-02-24 Thread Kamil Bregula (Jira)
Kamil Bregula created AIRFLOW-6908: -- Summary: Lazy load AirflowException Key: AIRFLOW-6908 URL: https://issues.apache.org/jira/browse/AIRFLOW-6908 Project: Apache Airflow Issue Type: New

[GitHub] [airflow] codecov-io edited a comment on issue #7523: [AIRFLOW-6843] Add delete_option_kwargs to delete_namespaced_pod

2020-02-24 Thread GitBox
codecov-io edited a comment on issue #7523: [AIRFLOW-6843] Add delete_option_kwargs to delete_namespaced_pod URL: https://github.com/apache/airflow/pull/7523#issuecomment-590610951 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7523?src=pr=h1) Report > :exclamation: No coverage

[GitHub] [airflow] kaxil commented on issue #7507: [AIRFLOW-6885] Delete worker on success

2020-02-24 Thread GitBox
kaxil commented on issue #7507: [AIRFLOW-6885] Delete worker on success URL: https://github.com/apache/airflow/pull/7507#issuecomment-590610930 Travis failed: ``` tests/executors/test_kubernetes_executor.py F ___ TestKubernetesExecutor.test_change_state_failed

[GitHub] [airflow] codecov-io commented on issue #7523: [AIRFLOW-6843] Add delete_option_kwargs to delete_namespaced_pod

2020-02-24 Thread GitBox
codecov-io commented on issue #7523: [AIRFLOW-6843] Add delete_option_kwargs to delete_namespaced_pod URL: https://github.com/apache/airflow/pull/7523#issuecomment-590610951 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7523?src=pr=h1) Report > :exclamation: No coverage

[jira] [Resolved] (AIRFLOW-6593) Google Cloud Stackdriver Alerting Operators

2020-02-24 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula resolved AIRFLOW-6593. Fix Version/s: 2.0.0 Resolution: Fixed > Google Cloud Stackdriver Alerting

[jira] [Commented] (AIRFLOW-6593) Google Cloud Stackdriver Alerting Operators

2020-02-24 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043962#comment-17043962 ] ASF subversion and git services commented on AIRFLOW-6593: -- Commit

[jira] [Commented] (AIRFLOW-6593) Google Cloud Stackdriver Alerting Operators

2020-02-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043961#comment-17043961 ] ASF GitHub Bot commented on AIRFLOW-6593: - mik-laj commented on pull request #7322:

[GitHub] [airflow] mik-laj merged pull request #7322: [AIRFLOW-6593] Add GCP Stackdriver Alerting Hooks and Operators

2020-02-24 Thread GitBox
mik-laj merged pull request #7322: [AIRFLOW-6593] Add GCP Stackdriver Alerting Hooks and Operators URL: https://github.com/apache/airflow/pull/7322 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] mik-laj commented on a change in pull request #7477: [AIRFLOW-6857][WIP] Bulk sync DAGs

2020-02-24 Thread GitBox
mik-laj commented on a change in pull request #7477: [AIRFLOW-6857][WIP] Bulk sync DAGs URL: https://github.com/apache/airflow/pull/7477#discussion_r383561313 ## File path: airflow/models/dag.py ## @@ -1455,63 +1455,92 @@ def create_dagrun(self, return run +

[GitHub] [airflow] kaxil commented on a change in pull request #7523: [AIRFLOW-6843] Add delete_option_kwargs to delete_namespaced_pod

2020-02-24 Thread GitBox
kaxil commented on a change in pull request #7523: [AIRFLOW-6843] Add delete_option_kwargs to delete_namespaced_pod URL: https://github.com/apache/airflow/pull/7523#discussion_r383567643 ## File path: airflow/config_templates/default_airflow.cfg ## @@ -980,6 +980,14 @@

[jira] [Commented] (AIRFLOW-6906) Flaky test_kubernetes_executor.py

2020-02-24 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043947#comment-17043947 ] ASF subversion and git services commented on AIRFLOW-6906: -- Commit

[jira] [Commented] (AIRFLOW-6906) Flaky test_kubernetes_executor.py

2020-02-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043945#comment-17043945 ] ASF GitHub Bot commented on AIRFLOW-6906: - potiuk commented on pull request #7525:

[jira] [Resolved] (AIRFLOW-6906) Flaky test_kubernetes_executor.py

2020-02-24 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6906. --- Fix Version/s: 2.0.0 Resolution: Fixed > Flaky test_kubernetes_executor.py >

[GitHub] [airflow] potiuk merged pull request #7525: [AIRFLOW-6906] Fix flaky tests in kubernetes tests

2020-02-24 Thread GitBox
potiuk merged pull request #7525: [AIRFLOW-6906] Fix flaky tests in kubernetes tests URL: https://github.com/apache/airflow/pull/7525 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] houqp commented on issue #7492: [AIRFLOW-6871] optimize tree view for large DAGs

2020-02-24 Thread GitBox
houqp commented on issue #7492: [AIRFLOW-6871] optimize tree view for large DAGs URL: https://github.com/apache/airflow/pull/7492#issuecomment-590590775 @ashb all builds are passing now, ready for another round of review.

[GitHub] [airflow] mik-laj commented on a change in pull request #7477: [AIRFLOW-6857][WIP] Bulk sync DAGs

2020-02-24 Thread GitBox
mik-laj commented on a change in pull request #7477: [AIRFLOW-6857][WIP] Bulk sync DAGs URL: https://github.com/apache/airflow/pull/7477#discussion_r383561313 ## File path: airflow/models/dag.py ## @@ -1455,63 +1455,92 @@ def create_dagrun(self, return run +

[jira] [Commented] (AIRFLOW-6907) Simplify SchedulerJob

2020-02-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043939#comment-17043939 ] ASF GitHub Bot commented on AIRFLOW-6907: - mik-laj commented on pull request #7527:

[GitHub] [airflow] mik-laj opened a new pull request #7527: [AIRFLOW-6907][WIP] Simplify SchedulerJob

2020-02-24 Thread GitBox
mik-laj opened a new pull request #7527: [AIRFLOW-6907][WIP] Simplify SchedulerJob URL: https://github.com/apache/airflow/pull/7527 If we realize that some methods always get one constant value, then their logic can be simplified. --- Issue link: WILL BE INSERTED BY

[GitHub] [airflow] codecov-io edited a comment on issue #7492: [AIRFLOW-6871] optimize tree view for large DAGs

2020-02-24 Thread GitBox
codecov-io edited a comment on issue #7492: [AIRFLOW-6871] optimize tree view for large DAGs URL: https://github.com/apache/airflow/pull/7492#issuecomment-589921201 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7492?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io commented on issue #7525: [AIRFLOW-6906] Fix flaky tests in kubernetes tests

2020-02-24 Thread GitBox
codecov-io commented on issue #7525: [AIRFLOW-6906] Fix flaky tests in kubernetes tests URL: https://github.com/apache/airflow/pull/7525#issuecomment-590586789 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7525?src=pr=h1) Report > Merging

[jira] [Created] (AIRFLOW-6907) Simplify SchedulerJob

2020-02-24 Thread Kamil Bregula (Jira)
Kamil Bregula created AIRFLOW-6907: -- Summary: Simplify SchedulerJob Key: AIRFLOW-6907 URL: https://issues.apache.org/jira/browse/AIRFLOW-6907 Project: Apache Airflow Issue Type: New Feature

[GitHub] [airflow] mik-laj commented on a change in pull request #7477: [AIRFLOW-6857][WIP] Bulk sync DAGs

2020-02-24 Thread GitBox
mik-laj commented on a change in pull request #7477: [AIRFLOW-6857][WIP] Bulk sync DAGs URL: https://github.com/apache/airflow/pull/7477#discussion_r383546400 ## File path: airflow/models/dag.py ## @@ -1455,63 +1455,92 @@ def create_dagrun(self, return run +

[GitHub] [airflow] saguziel commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances

2020-02-24 Thread GitBox
saguziel commented on a change in pull request #7489: [AIRFLOW-6869][WIP] Bulk fetch DAGRuns for _process_task_instances URL: https://github.com/apache/airflow/pull/7489#discussion_r383544228 ## File path: airflow/jobs/scheduler_job.py ## @@ -699,23 +727,30 @@ def

[GitHub] [airflow] kaxil edited a comment on issue #7119: [AIRFLOW-5840] Add operator extra link to external task sensor

2020-02-24 Thread GitBox
kaxil edited a comment on issue #7119: [AIRFLOW-5840] Add operator extra link to external task sensor URL: https://github.com/apache/airflow/pull/7119#issuecomment-590573136 Thanks @rconroy293 , this looks good. Can you rebase to the latest master and resolve the conflicts please.

[GitHub] [airflow] kaxil commented on issue #7119: [AIRFLOW-5840] Add operator extra link to external task sensor

2020-02-24 Thread GitBox
kaxil commented on issue #7119: [AIRFLOW-5840] Add operator extra link to external task sensor URL: https://github.com/apache/airflow/pull/7119#issuecomment-590573136 Thanks @rconroy293 , this looks good. Can you resolve the conflicts please

[GitHub] [airflow] saguziel commented on a change in pull request #7502: [AIRFLOW-6881][depends on AIRFLOW-6869][WIP] Bulk fetch DAGRun for create_dag_run

2020-02-24 Thread GitBox
saguziel commented on a change in pull request #7502: [AIRFLOW-6881][depends on AIRFLOW-6869][WIP] Bulk fetch DAGRun for create_dag_run URL: https://github.com/apache/airflow/pull/7502#discussion_r383542743 ## File path: airflow/jobs/scheduler_job.py ## @@ -699,23 +734,30

[GitHub] [airflow] ashb commented on a change in pull request #7523: [AIRFLOW-6843] Add delete_option_kwargs to delete_namespaced_pod

2020-02-24 Thread GitBox
ashb commented on a change in pull request #7523: [AIRFLOW-6843] Add delete_option_kwargs to delete_namespaced_pod URL: https://github.com/apache/airflow/pull/7523#discussion_r383542166 ## File path: airflow/config_templates/default_airflow.cfg ## @@ -980,6 +980,14 @@

[GitHub] [airflow] saguziel commented on a change in pull request #7370: [AIRFLOW-6590] Use batch db operations in jobs

2020-02-24 Thread GitBox
saguziel commented on a change in pull request #7370: [AIRFLOW-6590] Use batch db operations in jobs URL: https://github.com/apache/airflow/pull/7370#discussion_r383539224 ## File path: airflow/jobs/scheduler_job.py ## @@ -1270,20 +1276,17 @@ def

[GitHub] [airflow] ashb edited a comment on issue #7269: [AIRFLOW-6651] Add Redis Heartbeat option

2020-02-24 Thread GitBox
ashb edited a comment on issue #7269: [AIRFLOW-6651] Add Redis Heartbeat option URL: https://github.com/apache/airflow/pull/7269#issuecomment-590563447 How much db load does this actually save? This makes the code a lot more complex so... Can we see performance test numbers?

[GitHub] [airflow] ashb commented on a change in pull request #7269: [AIRFLOW-6651] Add Redis Heartbeat option

2020-02-24 Thread GitBox
ashb commented on a change in pull request #7269: [AIRFLOW-6651] Add Redis Heartbeat option URL: https://github.com/apache/airflow/pull/7269#discussion_r383534431 ## File path: airflow/jobs/base_job.py ## @@ -308,3 +316,24 @@ def query(result, items):

[GitHub] [airflow] ashb commented on a change in pull request #7269: [AIRFLOW-6651] Add Redis Heartbeat option

2020-02-24 Thread GitBox
ashb commented on a change in pull request #7269: [AIRFLOW-6651] Add Redis Heartbeat option URL: https://github.com/apache/airflow/pull/7269#discussion_r383535134 ## File path: airflow/jobs/local_task_job.py ## @@ -161,3 +166,55 @@ def heartbeat_callback(self,

[GitHub] [airflow] ashb commented on a change in pull request #7269: [AIRFLOW-6651] Add Redis Heartbeat option

2020-02-24 Thread GitBox
ashb commented on a change in pull request #7269: [AIRFLOW-6651] Add Redis Heartbeat option URL: https://github.com/apache/airflow/pull/7269#discussion_r383533275 ## File path: airflow/jobs/base_job.py ## @@ -66,23 +76,23 @@ class BaseJob(Base, LoggingMixin):

[GitHub] [airflow] ashb commented on issue #7269: [AIRFLOW-6651] Add Redis Heartbeat option

2020-02-24 Thread GitBox
ashb commented on issue #7269: [AIRFLOW-6651] Add Redis Heartbeat option URL: https://github.com/apache/airflow/pull/7269#issuecomment-590563447 How much db load does this actually save? This makes the code a lot more complex so...

[jira] [Commented] (AIRFLOW-6905) Update pinwheel.svg

2020-02-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043886#comment-17043886 ] ASF GitHub Bot commented on AIRFLOW-6905: - nuclearpinguin commented on pull request #7524:

[jira] [Commented] (AIRFLOW-6905) Update pinwheel.svg

2020-02-24 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043887#comment-17043887 ] ASF subversion and git services commented on AIRFLOW-6905: -- Commit

[jira] [Resolved] (AIRFLOW-6905) Update pinwheel.svg

2020-02-24 Thread Tomasz Urbaszek (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomasz Urbaszek resolved AIRFLOW-6905. -- Fix Version/s: 1.10.10 Resolution: Done > Update pinwheel.svg >

[GitHub] [airflow] nuclearpinguin merged pull request #7524: [AIRFLOW-6905] Update pin.svg with new pinwheel

2020-02-24 Thread GitBox
nuclearpinguin merged pull request #7524: [AIRFLOW-6905] Update pin.svg with new pinwheel URL: https://github.com/apache/airflow/pull/7524 This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Resolved] (AIRFLOW-6838) Introduce real subcommands for Breeze

2020-02-24 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6838. --- Fix Version/s: 1.10.10 Resolution: Fixed > Introduce real subcommands for Breeze >

[GitHub] [airflow] saguziel commented on a change in pull request #7477: [AIRFLOW-6857][WIP] Bulk sync DAGs

2020-02-24 Thread GitBox
saguziel commented on a change in pull request #7477: [AIRFLOW-6857][WIP] Bulk sync DAGs URL: https://github.com/apache/airflow/pull/7477#discussion_r383527240 ## File path: airflow/models/dag.py ## @@ -1455,63 +1455,92 @@ def create_dagrun(self, return run +

[jira] [Updated] (AIRFLOW-6838) Introduce real subcommands for Breeze

2020-02-24 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6838: -- Affects Version/s: 2.0.0 > Introduce real subcommands for Breeze >

[jira] [Commented] (AIRFLOW-6838) Introduce real subcommands for Breeze

2020-02-24 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043878#comment-17043878 ] ASF subversion and git services commented on AIRFLOW-6838: -- Commit

[jira] [Commented] (AIRFLOW-6838) Introduce real subcommands for Breeze

2020-02-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043877#comment-17043877 ] ASF GitHub Bot commented on AIRFLOW-6838: - potiuk commented on pull request #7515:

[GitHub] [airflow] potiuk merged pull request #7515: [AIRFLOW-6838] Add subcommands to Breeze

2020-02-24 Thread GitBox
potiuk merged pull request #7515: [AIRFLOW-6838] Add subcommands to Breeze URL: https://github.com/apache/airflow/pull/7515 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] saguziel commented on a change in pull request #7269: [AIRFLOW-6651] Add Redis Heartbeat option

2020-02-24 Thread GitBox
saguziel commented on a change in pull request #7269: [AIRFLOW-6651] Add Redis Heartbeat option URL: https://github.com/apache/airflow/pull/7269#discussion_r383526519 ## File path: airflow/jobs/base_job.py ## @@ -102,7 +112,16 @@ def most_recent_job(cls, session=None) ->

[GitHub] [airflow] abdulbasitds edited a comment on issue #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration

2020-02-24 Thread GitBox
abdulbasitds edited a comment on issue #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration URL: https://github.com/apache/airflow/pull/6007#issuecomment-590549904 @feluelle I have done point 1,2 and 3, can you please verify if it was the change you asked? Also, do you have an

[GitHub] [airflow] abdulbasitds edited a comment on issue #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration

2020-02-24 Thread GitBox
abdulbasitds edited a comment on issue #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration URL: https://github.com/apache/airflow/pull/6007#issuecomment-590549904 @feluelle I have done point 1,2 and 3, can you please verify if it was the change you asked?

[GitHub] [airflow] kaxil commented on a change in pull request #7407: [AIRFLOW-6786] Add KafkaConsumerHook, KafkaProduerHook and KafkaSensor

2020-02-24 Thread GitBox
kaxil commented on a change in pull request #7407: [AIRFLOW-6786] Add KafkaConsumerHook, KafkaProduerHook and KafkaSensor URL: https://github.com/apache/airflow/pull/7407#discussion_r383521933 ## File path: airflow/contrib/hooks/kafka_consumer_hook.py ## @@ -0,0 +1,71 @@

[GitHub] [airflow] kaxil commented on a change in pull request #7407: [AIRFLOW-6786] Add KafkaConsumerHook, KafkaProduerHook and KafkaSensor

2020-02-24 Thread GitBox
kaxil commented on a change in pull request #7407: [AIRFLOW-6786] Add KafkaConsumerHook, KafkaProduerHook and KafkaSensor URL: https://github.com/apache/airflow/pull/7407#discussion_r383521743 ## File path: airflow/contrib/hooks/kafka_consumer_hook.py ## @@ -0,0 +1,71 @@

[GitHub] [airflow] codecov-io commented on issue #7515: [AIRFLOW-6838] Add subcommands to Breeze

2020-02-24 Thread GitBox
codecov-io commented on issue #7515: [AIRFLOW-6838] Add subcommands to Breeze URL: https://github.com/apache/airflow/pull/7515#issuecomment-590554262 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7515?src=pr=h1) Report > Merging

[GitHub] [airflow] kaxil commented on issue #7422: [AIRFLOW-6809] Test for presto operators

2020-02-24 Thread GitBox
kaxil commented on issue #7422: [AIRFLOW-6809] Test for presto operators URL: https://github.com/apache/airflow/pull/7422#issuecomment-590553700 Flake8 errors: ``` tests/providers/apache/hive/operators/test_hive.py:156:14: F821 undefined name 'HdfsSensor'

[GitHub] [airflow] kaxil commented on a change in pull request #7269: [AIRFLOW-6651] Add Redis Heartbeat option

2020-02-24 Thread GitBox
kaxil commented on a change in pull request #7269: [AIRFLOW-6651] Add Redis Heartbeat option URL: https://github.com/apache/airflow/pull/7269#discussion_r383517536 ## File path: airflow/jobs/base_job.py ## @@ -102,7 +112,16 @@ def most_recent_job(cls, session=None) ->

[GitHub] [airflow] kaxil commented on a change in pull request #7269: [AIRFLOW-6651] Add Redis Heartbeat option

2020-02-24 Thread GitBox
kaxil commented on a change in pull request #7269: [AIRFLOW-6651] Add Redis Heartbeat option URL: https://github.com/apache/airflow/pull/7269#discussion_r383516404 ## File path: airflow/jobs/base_job.py ## @@ -21,6 +21,8 @@ from time import sleep from typing import

[GitHub] [airflow] abdulbasitds commented on issue #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration

2020-02-24 Thread GitBox
abdulbasitds commented on issue #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration URL: https://github.com/apache/airflow/pull/6007#issuecomment-590549904 @feluelle I have done point 1 and 3, can you please verify if it was the change you asked? For point 2, I am not able to

[GitHub] [airflow] kaxil commented on a change in pull request #7464: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation

2020-02-24 Thread GitBox
kaxil commented on a change in pull request #7464: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation URL: https://github.com/apache/airflow/pull/7464#discussion_r383512767 ## File path: airflow/operators/python.py ## @@ -167,6 +167,34 @@ def

[GitHub] [airflow] kaxil commented on a change in pull request #7464: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation

2020-02-24 Thread GitBox
kaxil commented on a change in pull request #7464: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation URL: https://github.com/apache/airflow/pull/7464#discussion_r383511142 ## File path: airflow/operators/python.py ## @@ -167,6 +167,34 @@ def

[GitHub] [airflow] kaxil commented on issue #7484: [AIRFLOW-6864] Make airflow/jobs pylint compatible

2020-02-24 Thread GitBox
kaxil commented on issue #7484: [AIRFLOW-6864] Make airflow/jobs pylint compatible URL: https://github.com/apache/airflow/pull/7484#issuecomment-590544060 There are merge conflicts This is an automated message from the

[GitHub] [airflow] potiuk commented on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2020-02-24 Thread GitBox
potiuk commented on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-590542381 rebase? This is an automated message

[jira] [Commented] (AIRFLOW-6873) mypyd is failing again with new type errors

2020-02-24 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043854#comment-17043854 ] ASF subversion and git services commented on AIRFLOW-6873: -- Commit

  1   2   3   4   >