[jira] [Assigned] (AIRFLOW-6250) on_failure_callback does not know the task_id when handle_failure() is called without passing context

2019-12-12 Thread Qian Yu (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qian Yu reassigned AIRFLOW-6250: Assignee: Qian Yu > on_failure_callback does not know the task_id when handle_failure() is calle

[jira] [Created] (AIRFLOW-6251) add config for max tasks per dag and max num dags to prevent scheduler denial of service

2019-12-12 Thread t oo (Jira)
t oo created AIRFLOW-6251: - Summary: add config for max tasks per dag and max num dags to prevent scheduler denial of service Key: AIRFLOW-6251 URL: https://issues.apache.org/jira/browse/AIRFLOW-6251 Project:

[jira] [Created] (AIRFLOW-6250) on_failure_callback does not know the task_id when handle_failure() is called without passing context

2019-12-12 Thread Qian Yu (Jira)
Qian Yu created AIRFLOW-6250: Summary: on_failure_callback does not know the task_id when handle_failure() is called without passing context Key: AIRFLOW-6250 URL: https://issues.apache.org/jira/browse/AIRFLOW-6250

[jira] [Updated] (AIRFLOW-6249) Spark driver status polling for standalone, YARN, Mesos and K8s

2019-12-12 Thread Albertus Kelvin (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Albertus Kelvin updated AIRFLOW-6249: - Description: I think the driver status tracker should also be enabled for yarn, mesos &

[GitHub] [airflow] baolsen commented on a change in pull request #6773: [AIRFLOW-6038] AWS DataSync example_dags added

2019-12-12 Thread GitBox
baolsen commented on a change in pull request #6773: [AIRFLOW-6038] AWS DataSync example_dags added URL: https://github.com/apache/airflow/pull/6773#discussion_r357501076 ## File path: docs/howto/operator/amazon/aws/datasync.rst ## @@ -0,0 +1,155 @@ + .. Licensed to the Ap

[jira] [Updated] (AIRFLOW-6249) Spark driver status polling for standalone, YARN, Mesos and K8s

2019-12-12 Thread Albertus Kelvin (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Albertus Kelvin updated AIRFLOW-6249: - Description: I think the driver status tracker should also be enabled for yarn, mesos &

[jira] [Updated] (AIRFLOW-6249) Spark driver status polling for standalone, YARN, Mesos and K8s

2019-12-12 Thread Albertus Kelvin (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Albertus Kelvin updated AIRFLOW-6249: - Description: I think the driver status tracker should also be enabled for yarn, mesos &

[GitHub] [airflow] baolsen commented on a change in pull request #6773: [AIRFLOW-6038] AWS DataSync example_dags added

2019-12-12 Thread GitBox
baolsen commented on a change in pull request #6773: [AIRFLOW-6038] AWS DataSync example_dags added URL: https://github.com/apache/airflow/pull/6773#discussion_r357501204 ## File path: airflow/providers/amazon/aws/example_dags/example_datasync_simple.py ## @@ -0,0 +1,75 @@

[GitHub] [airflow] baolsen commented on a change in pull request #6773: [AIRFLOW-6038] AWS DataSync example_dags added

2019-12-12 Thread GitBox
baolsen commented on a change in pull request #6773: [AIRFLOW-6038] AWS DataSync example_dags added URL: https://github.com/apache/airflow/pull/6773#discussion_r357500994 ## File path: airflow/providers/amazon/aws/operators/datasync.py ## @@ -124,379 +184,192 @@ def get_ho

[GitHub] [airflow] baolsen commented on a change in pull request #6773: [AIRFLOW-6038] AWS DataSync example_dags added

2019-12-12 Thread GitBox
baolsen commented on a change in pull request #6773: [AIRFLOW-6038] AWS DataSync example_dags added URL: https://github.com/apache/airflow/pull/6773#discussion_r357500939 ## File path: airflow/providers/amazon/aws/operators/datasync.py ## @@ -62,60 +80,102 @@ class AWSData

[GitHub] [airflow] baolsen commented on a change in pull request #6726: [AIRFLOW-6170] BranchPythonOperator does not do XCom push of returned value

2019-12-12 Thread GitBox
baolsen commented on a change in pull request #6726: [AIRFLOW-6170] BranchPythonOperator does not do XCom push of returned value URL: https://github.com/apache/airflow/pull/6726#discussion_r357500643 ## File path: UPDATING.md ## @@ -46,6 +46,10 @@ assists users migrating t

[jira] [Created] (AIRFLOW-6249) Spark driver status polling for standalone, YARN, Mesos and K8s

2019-12-12 Thread Albertus Kelvin (Jira)
Albertus Kelvin created AIRFLOW-6249: Summary: Spark driver status polling for standalone, YARN, Mesos and K8s Key: AIRFLOW-6249 URL: https://issues.apache.org/jira/browse/AIRFLOW-6249 Project: Ap

[GitHub] [airflow] maxirus commented on issue #6643: [AIRFLOW-6040] Fix KubernetesJobWatcher Read time out error

2019-12-12 Thread GitBox
maxirus commented on issue #6643: [AIRFLOW-6040] Fix KubernetesJobWatcher Read time out error URL: https://github.com/apache/airflow/pull/6643#issuecomment-565297542 I can implement it like this but do we really want the timeout to always be `1` for any integer >0? ---

[GitHub] [airflow] codecov-io commented on issue #6810: [AIRFLOW-6243]Optimize NamedHivePartitionSensor poke

2019-12-12 Thread GitBox
codecov-io commented on issue #6810: [AIRFLOW-6243]Optimize NamedHivePartitionSensor poke URL: https://github.com/apache/airflow/pull/6810#issuecomment-565288214 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6810?src=pr&el=h1) Report > Merging [#6810](https://codecov.io/gh/apac

[GitHub] [airflow] codecov-io edited a comment on issue #6810: [AIRFLOW-6243]Optimize NamedHivePartitionSensor poke

2019-12-12 Thread GitBox
codecov-io edited a comment on issue #6810: [AIRFLOW-6243]Optimize NamedHivePartitionSensor poke URL: https://github.com/apache/airflow/pull/6810#issuecomment-565288214 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6810?src=pr&el=h1) Report > Merging [#6810](https://codecov.io/

[jira] [Created] (AIRFLOW-6248) Should host variable in Connection class contain scheme?

2019-12-12 Thread xifeng (Jira)
xifeng created AIRFLOW-6248: --- Summary: Should host variable in Connection class contain scheme? Key: AIRFLOW-6248 URL: https://issues.apache.org/jira/browse/AIRFLOW-6248 Project: Apache Airflow Iss

[jira] [Commented] (AIRFLOW-6245) Allow custom waiters for AWS batch jobs

2019-12-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995275#comment-16995275 ] ASF GitHub Bot commented on AIRFLOW-6245: - darrenleeweber commented on pull req

[GitHub] [airflow] darrenleeweber opened a new pull request #6811: [WIP][AIRFLOW-6245] Add custom waiters for AWS batch jobs

2019-12-12 Thread GitBox
darrenleeweber opened a new pull request #6811: [WIP][AIRFLOW-6245] Add custom waiters for AWS batch jobs URL: https://github.com/apache/airflow/pull/6811 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow Jira](https://issues.

[jira] [Commented] (AIRFLOW-6243) Optimize NamedHivePartitionSensor to reduce Hive metastore poke load

2019-12-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995253#comment-16995253 ] ASF GitHub Bot commented on AIRFLOW-6243: - YingboWang commented on pull request

[GitHub] [airflow] YingboWang opened a new pull request #6810: [AIRFLOW-6243]Optimize NamedHivePartitionSensor poke

2019-12-12 Thread GitBox
YingboWang opened a new pull request #6810: [AIRFLOW-6243]Optimize NamedHivePartitionSensor poke URL: https://github.com/apache/airflow/pull/6810 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow Jira](https://issues.apache.or

[jira] [Commented] (AIRFLOW-5660) Scheduler becomes unresponsive when processing large DAGs on kubernetes.

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995250#comment-16995250 ] ASF subversion and git services commented on AIRFLOW-5660: -- Co

[jira] [Commented] (AIRFLOW-6241) Fix typo in airflow/gcp/operator/dataflow.py

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995249#comment-16995249 ] ASF subversion and git services commented on AIRFLOW-6241: -- Co

[GitHub] [airflow] codecov-io edited a comment on issue #6776: [AIRFLOW-6220] BigQuery - remove redundant hook tests after renaming

2019-12-12 Thread GitBox
codecov-io edited a comment on issue #6776: [AIRFLOW-6220] BigQuery - remove redundant hook tests after renaming URL: https://github.com/apache/airflow/pull/6776#issuecomment-564985782 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6776?src=pr&el=h1) Report > Merging [#6776](htt

[GitHub] [airflow] codecov-io edited a comment on issue #6776: [AIRFLOW-6220] BigQuery - remove redundant hook tests after renaming

2019-12-12 Thread GitBox
codecov-io edited a comment on issue #6776: [AIRFLOW-6220] BigQuery - remove redundant hook tests after renaming URL: https://github.com/apache/airflow/pull/6776#issuecomment-564985782 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6776?src=pr&el=h1) Report > Merging [#6776](htt

[jira] [Updated] (AIRFLOW-5808) DAG timezones given as datetime.timezone cause odd errors.

2019-12-12 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik updated AIRFLOW-5808: Fix Version/s: (was: 1.10.7) > DAG timezones given as datetime.timezone cause odd errors. > -

[jira] [Reopened] (AIRFLOW-5808) DAG timezones given as datetime.timezone cause odd errors.

2019-12-12 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik reopened AIRFLOW-5808: - > DAG timezones given as datetime.timezone cause odd errors. >

[jira] [Closed] (AIRFLOW-5808) DAG timezones given as datetime.timezone cause odd errors.

2019-12-12 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik closed AIRFLOW-5808. --- Resolution: Duplicate > DAG timezones given as datetime.timezone cause odd errors. > --

[jira] [Work started] (AIRFLOW-6247) Fix sort order in Alembic Migration template

2019-12-12 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6247 started by Kaxil Naik. --- > Fix sort order in Alembic Migration template > ---

[jira] [Commented] (AIRFLOW-6171) airflow ignore file with .* located in a subdirectory ignores dags in other dirs

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995241#comment-16995241 ] ASF subversion and git services commented on AIRFLOW-6171: -- Co

[GitHub] [airflow] mik-laj merged pull request #6359: [AIRFLOW-XXX] Change instances "Google cloud storage" to "Google Cloud Storage".

2019-12-12 Thread GitBox
mik-laj merged pull request #6359: [AIRFLOW-XXX] Change instances "Google cloud storage" to "Google Cloud Storage". URL: https://github.com/apache/airflow/pull/6359 This is an automated message from the Apache Git Service. T

[jira] [Commented] (AIRFLOW-6247) Fix sort order in Alembic Migration template

2019-12-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995230#comment-16995230 ] ASF GitHub Bot commented on AIRFLOW-6247: - kaxil commented on pull request #680

[GitHub] [airflow] kaxil opened a new pull request #6809: [AIRFLOW-6247] Fix sort order in Alembic Migration template

2019-12-12 Thread GitBox
kaxil opened a new pull request #6809: [AIRFLOW-6247] Fix sort order in Alembic Migration template URL: https://github.com/apache/airflow/pull/6809 Make sure you have checked _all_ steps below. ### Jira - [x] https://issues.apache.org/jira/browse/AIRFLOW-6247 ### Descri

[GitHub] [airflow] dgorelik commented on issue #6359: [AIRFLOW-XXX] Change instances "Google cloud storage" to "Google Cloud Storage".

2019-12-12 Thread GitBox
dgorelik commented on issue #6359: [AIRFLOW-XXX] Change instances "Google cloud storage" to "Google Cloud Storage". URL: https://github.com/apache/airflow/pull/6359#issuecomment-565245896 Thansk for wrapping this up! This is

[jira] [Created] (AIRFLOW-6247) Fix sort order in Alembic Migration template

2019-12-12 Thread Kaxil Naik (Jira)
Kaxil Naik created AIRFLOW-6247: --- Summary: Fix sort order in Alembic Migration template Key: AIRFLOW-6247 URL: https://issues.apache.org/jira/browse/AIRFLOW-6247 Project: Apache Airflow Issue T

[jira] [Commented] (AIRFLOW-6241) Fix typo in airflow/gcp/operator/dataflow.py

2019-12-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995226#comment-16995226 ] ASF GitHub Bot commented on AIRFLOW-6241: - kaxil commented on pull request #680

[jira] [Commented] (AIRFLOW-6241) Fix typo in airflow/gcp/operator/dataflow.py

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995228#comment-16995228 ] ASF subversion and git services commented on AIRFLOW-6241: -- Co

[jira] [Resolved] (AIRFLOW-6241) Fix typo in airflow/gcp/operator/dataflow.py

2019-12-12 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik resolved AIRFLOW-6241. - Fix Version/s: 1.10.7 2.0.0 Resolution: Fixed > Fix typo in airflow/g

[GitHub] [airflow] kaxil merged pull request #6806: [AIRFLOW-6241] - Fix typo in airflow/gcp/operator/dataflow.py

2019-12-12 Thread GitBox
kaxil merged pull request #6806: [AIRFLOW-6241] - Fix typo in airflow/gcp/operator/dataflow.py URL: https://github.com/apache/airflow/pull/6806 This is an automated message from the Apache Git Service. To respond to the mess

[jira] [Commented] (AIRFLOW-6246) MySqlToGoogleCloudStorageOperator: Null value of Bytes type conversion fails

2019-12-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995222#comment-16995222 ] ASF GitHub Bot commented on AIRFLOW-6246: - stoynov96 commented on pull request

[GitHub] [airflow] stoynov96 opened a new pull request #6808: [AIRFLOW-6246] Bytes conversion when value is None now doesn't fail

2019-12-12 Thread GitBox
stoynov96 opened a new pull request #6808: [AIRFLOW-6246] Bytes conversion when value is None now doesn't fail URL: https://github.com/apache/airflow/pull/6808 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [AIRFLOW-6246](https://is

[jira] [Created] (AIRFLOW-6246) MySqlToGoogleCloudStorageOperator: Null value of Bytes type conversion fails

2019-12-12 Thread Nikolay Stoynov (Jira)
Nikolay Stoynov created AIRFLOW-6246: Summary: MySqlToGoogleCloudStorageOperator: Null value of Bytes type conversion fails Key: AIRFLOW-6246 URL: https://issues.apache.org/jira/browse/AIRFLOW-6246

[GitHub] [airflow] codecov-io edited a comment on issue #6359: [AIRFLOW-XXX] Change instances "Google cloud storage" to "Google Cloud Storage".

2019-12-12 Thread GitBox
codecov-io edited a comment on issue #6359: [AIRFLOW-XXX] Change instances "Google cloud storage" to "Google Cloud Storage". URL: https://github.com/apache/airflow/pull/6359#issuecomment-565239559 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6359?src=pr&el=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6359: [AIRFLOW-XXX] Change instances "Google cloud storage" to "Google Cloud Storage".

2019-12-12 Thread GitBox
codecov-io edited a comment on issue #6359: [AIRFLOW-XXX] Change instances "Google cloud storage" to "Google Cloud Storage". URL: https://github.com/apache/airflow/pull/6359#issuecomment-565239559 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6359?src=pr&el=h1) Report > Merging

[GitHub] [airflow] codecov-io commented on issue #6359: [AIRFLOW-XXX] Change instances "Google cloud storage" to "Google Cloud Storage".

2019-12-12 Thread GitBox
codecov-io commented on issue #6359: [AIRFLOW-XXX] Change instances "Google cloud storage" to "Google Cloud Storage". URL: https://github.com/apache/airflow/pull/6359#issuecomment-565239559 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6359?src=pr&el=h1) Report > Merging [#6359

[GitHub] [airflow] davlum edited a comment on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-12-12 Thread GitBox
davlum edited a comment on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#issuecomment-565236466 Thanks @ashb for a very thorough review on a complex PR. I'll try to address each of your comments. The a

[GitHub] [airflow] kaxil commented on issue #6794: [AIRFLOW-6231] Display DAG run conf in the graph view

2019-12-12 Thread GitBox
kaxil commented on issue #6794: [AIRFLOW-6231] Display DAG run conf in the graph view URL: https://github.com/apache/airflow/pull/6794#issuecomment-565236892 > > I like that 👍 But I am wondering if it would look better below the Graph View or between the Base Date row and the Graph View.

[GitHub] [airflow] davlum commented on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-12-12 Thread GitBox
davlum commented on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#issuecomment-565236466 Thanks @ashb for a very thorough review on a complex PR. I'll try to address each of your comments. The api chan

[GitHub] [airflow] davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-12-12 Thread GitBox
davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r357424985 ## File path: airflow/kubernetes/worker_configuration.py ## @@ -362,24 +366,1

[GitHub] [airflow] codecov-io edited a comment on issue #6794: [AIRFLOW-6231] Display DAG run conf in the graph view

2019-12-12 Thread GitBox
codecov-io edited a comment on issue #6794: [AIRFLOW-6231] Display DAG run conf in the graph view URL: https://github.com/apache/airflow/pull/6794#issuecomment-564805241 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6794?src=pr&el=h1) Report > Merging [#6794](https://codecov.io

[GitHub] [airflow] codecov-io edited a comment on issue #6794: [AIRFLOW-6231] Display DAG run conf in the graph view

2019-12-12 Thread GitBox
codecov-io edited a comment on issue #6794: [AIRFLOW-6231] Display DAG run conf in the graph view URL: https://github.com/apache/airflow/pull/6794#issuecomment-564805241 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6794?src=pr&el=h1) Report > Merging [#6794](https://codecov.io

[GitHub] [airflow] davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-12-12 Thread GitBox
davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r357420456 ## File path: airflow/executors/kubernetes_executor.py ## @@ -402,22 +405,30

[GitHub] [airflow] davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-12-12 Thread GitBox
davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r357419658 ## File path: airflow/kubernetes/worker_configuration.py ## @@ -389,11 +386,1

[jira] [Assigned] (AIRFLOW-6245) Allow custom waiters for AWS batch jobs

2019-12-12 Thread Darren Weber (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darren Weber reassigned AIRFLOW-6245: - Assignee: Darren Weber > Allow custom waiters for AWS batch jobs > ---

[jira] [Work started] (AIRFLOW-6245) Allow custom waiters for AWS batch jobs

2019-12-12 Thread Darren Weber (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6245 started by Darren Weber. - > Allow custom waiters for AWS batch jobs >

[GitHub] [airflow] davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-12-12 Thread GitBox
davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r357419658 ## File path: airflow/kubernetes/worker_configuration.py ## @@ -389,11 +386,1

[GitHub] [airflow] davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-12-12 Thread GitBox
davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r357416853 ## File path: airflow/executors/kubernetes_executor.py ## @@ -372,7 +374,8 @@

[jira] [Resolved] (AIRFLOW-6211) Document using conda virtualenv for development

2019-12-12 Thread Darren Weber (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darren Weber resolved AIRFLOW-6211. --- Resolution: Fixed > Document using conda virtualenv for development > -

[jira] [Created] (AIRFLOW-6245) Allow custom waiters for AWS batch jobs

2019-12-12 Thread Darren Weber (Jira)
Darren Weber created AIRFLOW-6245: - Summary: Allow custom waiters for AWS batch jobs Key: AIRFLOW-6245 URL: https://issues.apache.org/jira/browse/AIRFLOW-6245 Project: Apache Airflow Issue Ty

[GitHub] [airflow] codecov-io edited a comment on issue #6608: [AIRFLOW-5780] Move AWS Glue from contrib to providers

2019-12-12 Thread GitBox
codecov-io edited a comment on issue #6608: [AIRFLOW-5780] Move AWS Glue from contrib to providers URL: https://github.com/apache/airflow/pull/6608#issuecomment-555726762 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6608?src=pr&el=h1) Report > Merging [#6608](https://codecov.i

[GitHub] [airflow] codecov-io edited a comment on issue #6608: [AIRFLOW-5780] Move AWS Glue from contrib to providers

2019-12-12 Thread GitBox
codecov-io edited a comment on issue #6608: [AIRFLOW-5780] Move AWS Glue from contrib to providers URL: https://github.com/apache/airflow/pull/6608#issuecomment-555726762 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6608?src=pr&el=h1) Report > Merging [#6608](https://codecov.i

[jira] [Updated] (AIRFLOW-6206) Adapt AWS batch operator to AIP-21

2019-12-12 Thread Darren Weber (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darren Weber updated AIRFLOW-6206: -- Fix Version/s: 1.10.7 2.0.0 Description: Migrate AWS batch operator t

[jira] [Work started] (AIRFLOW-6206) Adapt AWS batch operator to AIP-21

2019-12-12 Thread Darren Weber (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6206 started by Darren Weber. - > Adapt AWS batch operator to AIP-21 > -- >

[GitHub] [airflow] davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-12-12 Thread GitBox
davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r357401311 ## File path: tests/integration/kubernetes/dep.yaml ## @@ -0,0 +1,33 @@ +# Li

[GitHub] [airflow] davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-12-12 Thread GitBox
davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r357399220 ## File path: airflow/kubernetes/pod_generator.py ## @@ -298,51 +361,171 @@ d

[GitHub] [airflow] davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-12-12 Thread GitBox
davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r357398414 ## File path: airflow/executors/kubernetes_executor.py ## @@ -402,22 +405,30

[GitHub] [airflow] davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-12-12 Thread GitBox
davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r357397543 ## File path: airflow/kubernetes/pod_generator.py ## @@ -298,51 +361,171 @@ d

[GitHub] [airflow] davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-12-12 Thread GitBox
davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r357397226 ## File path: airflow/kubernetes/pod_generator.py ## @@ -298,51 +361,171 @@ d

[GitHub] [airflow] marinajpereira commented on issue #6196: [AIRFLOW-3152] Kubernetes Pod Operator should support init containers.

2019-12-12 Thread GitBox
marinajpereira commented on issue #6196: [AIRFLOW-3152] Kubernetes Pod Operator should support init containers. URL: https://github.com/apache/airflow/pull/6196#issuecomment-565207648 > Hi @marinajpereira, apologies I hadn't seen this PR. Could you rebase? Would be glad to PR an merge afte

[jira] [Commented] (AIRFLOW-6181) Add InProcessExecutor

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995140#comment-16995140 ] ASF subversion and git services commented on AIRFLOW-6181: -- Co

[jira] [Commented] (AIRFLOW-6181) Add InProcessExecutor

2019-12-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995139#comment-16995139 ] ASF GitHub Bot commented on AIRFLOW-6181: - mik-laj commented on pull request #6

[GitHub] [airflow] mik-laj merged pull request #6740: [AIRFLOW-6181] Add InProcessExecutor

2019-12-12 Thread GitBox
mik-laj merged pull request #6740: [AIRFLOW-6181] Add InProcessExecutor URL: https://github.com/apache/airflow/pull/6740 This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[GitHub] [airflow] codecov-io edited a comment on issue #6806: [AIRFLOW-6241] - Fix typo in airflow/gcp/operator/dataflow.py

2019-12-12 Thread GitBox
codecov-io edited a comment on issue #6806: [AIRFLOW-6241] - Fix typo in airflow/gcp/operator/dataflow.py URL: https://github.com/apache/airflow/pull/6806#issuecomment-565197337 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6806?src=pr&el=h1) Report > :exclamation: No coverage u

[GitHub] [airflow] codecov-io commented on issue #6806: [AIRFLOW-6241] - Fix typo in airflow/gcp/operator/dataflow.py

2019-12-12 Thread GitBox
codecov-io commented on issue #6806: [AIRFLOW-6241] - Fix typo in airflow/gcp/operator/dataflow.py URL: https://github.com/apache/airflow/pull/6806#issuecomment-565197337 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6806?src=pr&el=h1) Report > :exclamation: No coverage uploaded

[jira] [Created] (AIRFLOW-6244) spark operator fails tasks for kubernetes jobs, eventhough the job is successful.

2019-12-12 Thread Mike Prior (Jira)
Mike Prior created AIRFLOW-6244: --- Summary: spark operator fails tasks for kubernetes jobs, eventhough the job is successful. Key: AIRFLOW-6244 URL: https://issues.apache.org/jira/browse/AIRFLOW-6244 Pro

[jira] [Commented] (AIRFLOW-6211) Document using conda virtualenv for development

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995123#comment-16995123 ] ASF subversion and git services commented on AIRFLOW-6211: -- Co

[jira] [Commented] (AIRFLOW-6226) warning.catch_warning should not be used in our code

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995124#comment-16995124 ] ASF subversion and git services commented on AIRFLOW-6226: -- Co

[jira] [Commented] (AIRFLOW-6191) Adjust pytest verbosity in CI and local environment

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995121#comment-16995121 ] ASF subversion and git services commented on AIRFLOW-6191: -- Co

[jira] [Commented] (AIRFLOW-6018) Display task instance in table during backfilling

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995119#comment-16995119 ] ASF subversion and git services commented on AIRFLOW-6018: -- Co

[jira] [Commented] (AIRFLOW-6216) Allow pytests to be run without "tests"

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995122#comment-16995122 ] ASF subversion and git services commented on AIRFLOW-6216: -- Co

[jira] [Commented] (AIRFLOW-6018) Display task instance in table during backfilling

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995120#comment-16995120 ] ASF subversion and git services commented on AIRFLOW-6018: -- Co

[jira] [Commented] (AIRFLOW-6189) Reduce the maximum test duration to 8 minutes

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995118#comment-16995118 ] ASF subversion and git services commented on AIRFLOW-6189: -- Co

[jira] [Commented] (AIRFLOW-6211) Document using conda virtualenv for development

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995115#comment-16995115 ] ASF subversion and git services commented on AIRFLOW-6211: -- Co

[jira] [Commented] (AIRFLOW-6216) Allow pytests to be run without "tests"

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995114#comment-16995114 ] ASF subversion and git services commented on AIRFLOW-6216: -- Co

[GitHub] [airflow] ashb commented on a change in pull request #6792: [AIRFLOW-5930] Use cached-SQL query building for hot-path queries

2019-12-12 Thread GitBox
ashb commented on a change in pull request #6792: [AIRFLOW-5930] Use cached-SQL query building for hot-path queries URL: https://github.com/apache/airflow/pull/6792#discussion_r357375764 ## File path: airflow/jobs/scheduler_job.py ## @@ -1290,7 +1256,7 @@ def _enqueue_task

[jira] [Commented] (AIRFLOW-6018) Display task instance in table during backfilling

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995111#comment-16995111 ] ASF subversion and git services commented on AIRFLOW-6018: -- Co

[jira] [Commented] (AIRFLOW-6018) Display task instance in table during backfilling

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995112#comment-16995112 ] ASF subversion and git services commented on AIRFLOW-6018: -- Co

[jira] [Commented] (AIRFLOW-6226) warning.catch_warning should not be used in our code

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995116#comment-16995116 ] ASF subversion and git services commented on AIRFLOW-6226: -- Co

[jira] [Commented] (AIRFLOW-6191) Adjust pytest verbosity in CI and local environment

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995113#comment-16995113 ] ASF subversion and git services commented on AIRFLOW-6191: -- Co

[jira] [Commented] (AIRFLOW-6211) Document using conda virtualenv for development

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995108#comment-16995108 ] ASF subversion and git services commented on AIRFLOW-6211: -- Co

[jira] [Commented] (AIRFLOW-6189) Reduce the maximum test duration to 8 minutes

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995110#comment-16995110 ] ASF subversion and git services commented on AIRFLOW-6189: -- Co

[jira] [Commented] (AIRFLOW-6226) warning.catch_warning should not be used in our code

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995109#comment-16995109 ] ASF subversion and git services commented on AIRFLOW-6226: -- Co

[jira] [Commented] (AIRFLOW-6018) Display task instance in table during backfilling

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995103#comment-16995103 ] ASF subversion and git services commented on AIRFLOW-6018: -- Co

[jira] [Commented] (AIRFLOW-6189) Reduce the maximum test duration to 8 minutes

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995102#comment-16995102 ] ASF subversion and git services commented on AIRFLOW-6189: -- Co

[jira] [Commented] (AIRFLOW-6018) Display task instance in table during backfilling

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995104#comment-16995104 ] ASF subversion and git services commented on AIRFLOW-6018: -- Co

[jira] [Commented] (AIRFLOW-6191) Adjust pytest verbosity in CI and local environment

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995105#comment-16995105 ] ASF subversion and git services commented on AIRFLOW-6191: -- Co

[jira] [Commented] (AIRFLOW-6216) Allow pytests to be run without "tests"

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995106#comment-16995106 ] ASF subversion and git services commented on AIRFLOW-6216: -- Co

[jira] [Commented] (AIRFLOW-6018) Display task instance in table during backfilling

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995096#comment-16995096 ] ASF subversion and git services commented on AIRFLOW-6018: -- Co

[jira] [Commented] (AIRFLOW-6226) warning.catch_warning should not be used in our code

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995100#comment-16995100 ] ASF subversion and git services commented on AIRFLOW-6226: -- Co

[jira] [Commented] (AIRFLOW-6211) Document using conda virtualenv for development

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995099#comment-16995099 ] ASF subversion and git services commented on AIRFLOW-6211: -- Co

[jira] [Commented] (AIRFLOW-6189) Reduce the maximum test duration to 8 minutes

2019-12-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995094#comment-16995094 ] ASF subversion and git services commented on AIRFLOW-6189: -- Co

  1   2   3   4   >