[jira] [Commented] (AIRFLOW-5328) EmrCreateJobFlowOperator success when EMR isn't ready

2019-10-30 Thread Juan Ramos Fuentes (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963655#comment-16963655 ] Juan Ramos Fuentes commented on AIRFLOW-5328: - +1 for this! emr_job_flow_sensor will only

[GitHub] [airflow] codecov-io edited a comment on issue #6474: [AIRFLOW-5787] Moving AWS SQS to /providers/aws.

2019-10-30 Thread GitBox
codecov-io edited a comment on issue #6474: [AIRFLOW-5787] Moving AWS SQS to /providers/aws. URL: https://github.com/apache/airflow/pull/6474#issuecomment-548214250 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6474?src=pr=h1) Report > Merging

[jira] [Assigned] (AIRFLOW-5790) Migrate AwsHook to /providers/aws

2019-10-30 Thread Ishan Rastogi (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Rastogi reassigned AIRFLOW-5790: -- Assignee: Ishan Rastogi > Migrate AwsHook to /providers/aws >

[GitHub] [airflow] codecov-io commented on issue #6474: [AIRFLOW-5787] Moving AWS SQS to /providers/aws.

2019-10-30 Thread GitBox
codecov-io commented on issue #6474: [AIRFLOW-5787] Moving AWS SQS to /providers/aws. URL: https://github.com/apache/airflow/pull/6474#issuecomment-548214250 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6474?src=pr=h1) Report > Merging

[GitHub] [airflow] adityav commented on issue #6340: [AIRFLOW-5660] Attempt to find the task in DB from Kubernetes pod labels

2019-10-30 Thread GitBox
adityav commented on issue #6340: [AIRFLOW-5660] Attempt to find the task in DB from Kubernetes pod labels URL: https://github.com/apache/airflow/pull/6340#issuecomment-548208947 > > Only dag_id / task_id are being stored in the labels. The execution date isn't being stored there so it

[GitHub] [airflow] adityav edited a comment on issue #6340: [AIRFLOW-5660] Attempt to find the task in DB from Kubernetes pod labels

2019-10-30 Thread GitBox
adityav edited a comment on issue #6340: [AIRFLOW-5660] Attempt to find the task in DB from Kubernetes pod labels URL: https://github.com/apache/airflow/pull/6340#issuecomment-548208947 > > Only dag_id / task_id are being stored in the labels. The execution date isn't being stored there

[jira] [Updated] (AIRFLOW-5820) Long delay between individual tasks in a large backfill

2019-10-30 Thread Eric Nichols (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Nichols updated AIRFLOW-5820: -- Description: I am new to Airflow. I made a simple task in a trivial DAG. It takes 0.004

[jira] [Updated] (AIRFLOW-5820) Long delay between individual tasks in a large backfill

2019-10-30 Thread Eric Nichols (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Nichols updated AIRFLOW-5820: -- Description: I am new to Airflow. I made a simple task in a trivial DAG. It takes 0.004

[jira] [Created] (AIRFLOW-5820) Long delay between individual tasks in a large backfill

2019-10-30 Thread Eric Nichols (Jira)
Eric Nichols created AIRFLOW-5820: - Summary: Long delay between individual tasks in a large backfill Key: AIRFLOW-5820 URL: https://issues.apache.org/jira/browse/AIRFLOW-5820 Project: Apache Airflow

[jira] [Resolved] (AIRFLOW-5729) Make InputDataConfig in SageMakerTrainingOperator optional

2019-10-30 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik resolved AIRFLOW-5729. - Fix Version/s: 2.0.0 Resolution: Fixed > Make InputDataConfig in

[GitHub] [airflow] ZxMYS opened a new pull request #6478: [AIRFLOW-XXXX] Expose SqlAlchemy's connect_args and make it configurable

2019-10-30 Thread GitBox
ZxMYS opened a new pull request #6478: [AIRFLOW-] Expose SqlAlchemy's connect_args and make it configurable URL: https://github.com/apache/airflow/pull/6478 #6226 # Jira - [ ] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues

[GitHub] [airflow] kaxil opened a new pull request #6477: [AIRFLOW-XXX] FIX typo in Breeze.rst

2019-10-30 Thread GitBox
kaxil opened a new pull request #6477: [AIRFLOW-XXX] FIX typo in Breeze.rst URL: https://github.com/apache/airflow/pull/6477 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

[jira] [Closed] (AIRFLOW-5752) flakey slack build message test

2019-10-30 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik closed AIRFLOW-5752. --- Resolution: Duplicate > flakey slack build message test > --- > >

[jira] [Resolved] (AIRFLOW-5740) Fix Transient failure in Slack test

2019-10-30 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik resolved AIRFLOW-5740. - Resolution: Fixed > Fix Transient failure in Slack test > ---

[jira] [Resolved] (AIRFLOW-5750) Licence check not checked for non-executable .sh files

2019-10-30 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik resolved AIRFLOW-5750. - Fix Version/s: 1.10.6 Resolution: Fixed > Licence check not checked for non-executable

[jira] [Commented] (AIRFLOW-5743) Move Gogle PubSub to providers package

2019-10-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963506#comment-16963506 ] ASF GitHub Bot commented on AIRFLOW-5743: - ratb3rt commented on pull request #6476:

[GitHub] [airflow] ratb3rt opened a new pull request #6476: [AIRFLOW-5743] Move Google PubSub to providers package

2019-10-30 Thread GitBox
ratb3rt opened a new pull request #6476: [AIRFLOW-5743] Move Google PubSub to providers package URL: https://github.com/apache/airflow/pull/6476 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

[GitHub] [airflow-site] mik-laj commented on a change in pull request #103: [depends on #96] Search meetups

2019-10-30 Thread GitBox
mik-laj commented on a change in pull request #103: [depends on #96] Search meetups URL: https://github.com/apache/airflow-site/pull/103#discussion_r340890654 ## File path: landing-pages/src/js/meetupsList.js ## @@ -0,0 +1,118 @@ +/** + * Licensed to the Apache Software

[jira] [Assigned] (AIRFLOW-5743) Move Gogle PubSub to providers package

2019-10-30 Thread Rich Dean (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Dean reassigned AIRFLOW-5743: -- Assignee: Rich Dean > Move Gogle PubSub to providers package >

[GitHub] [airflow] codecov-io edited a comment on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-10-30 Thread GitBox
codecov-io edited a comment on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#issuecomment-537654940 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6230?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-10-30 Thread GitBox
codecov-io edited a comment on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#issuecomment-537654940 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6230?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-5774) PapermillOperator fails to render parameters

2019-10-30 Thread Manny Cato (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963475#comment-16963475 ] Manny Cato commented on AIRFLOW-5774: - I see this has been closed due to being a duplicate, but

[jira] [Commented] (AIRFLOW-5741) Move Cloud Natural Language to providers

2019-10-30 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963470#comment-16963470 ] ASF subversion and git services commented on AIRFLOW-5741: -- Commit

[jira] [Resolved] (AIRFLOW-5741) Move Cloud Natural Language to providers

2019-10-30 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula resolved AIRFLOW-5741. Fix Version/s: 2.0.0 Resolution: Fixed > Move Cloud Natural Language to providers

[GitHub] [airflow] kaxil commented on issue #6396: [AIRFLOW-5726] Delete table as file name in RedshiftToS3Transfer

2019-10-30 Thread GitBox
kaxil commented on issue #6396: [AIRFLOW-5726] Delete table as file name in RedshiftToS3Transfer URL: https://github.com/apache/airflow/pull/6396#issuecomment-548132277 add one here: https://github.com/apache/airflow/blob/master/tests/operators/test_redshift_to_s3_operator.py

[GitHub] [airflow] mik-laj commented on issue #6421: [AIRFLOW-5741] Move Cloud Natural Language to providers

2019-10-30 Thread GitBox
mik-laj commented on issue #6421: [AIRFLOW-5741] Move Cloud Natural Language to providers URL: https://github.com/apache/airflow/pull/6421#issuecomment-548132431 Thank you very much. I look forward to your next contributions. ;-D

[jira] [Commented] (AIRFLOW-5741) Move Cloud Natural Language to providers

2019-10-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963469#comment-16963469 ] ASF GitHub Bot commented on AIRFLOW-5741: - mik-laj commented on pull request #6421:

[GitHub] [airflow] mik-laj merged pull request #6421: [AIRFLOW-5741] Move Cloud Natural Language to providers

2019-10-30 Thread GitBox
mik-laj merged pull request #6421: [AIRFLOW-5741] Move Cloud Natural Language to providers URL: https://github.com/apache/airflow/pull/6421 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] kaxil commented on issue #6396: [AIRFLOW-5726] Delete table as file name in RedshiftToS3Transfer

2019-10-30 Thread GitBox
kaxil commented on issue #6396: [AIRFLOW-5726] Delete table as file name in RedshiftToS3Transfer URL: https://github.com/apache/airflow/pull/6396#issuecomment-548131975 ![image](https://user-images.githubusercontent.com/8811558/67902414-07a27e80-fb61-11e9-9b75-cdf0618386f1.png) We have

[GitHub] [airflow] ratb3rt commented on issue #6421: [AIRFLOW-5741] Move Cloud Natural Language to providers

2019-10-30 Thread GitBox
ratb3rt commented on issue #6421: [AIRFLOW-5741] Move Cloud Natural Language to providers URL: https://github.com/apache/airflow/pull/6421#issuecomment-548127224 Rebased, and Travis is still happy. Nice. This is an automated

[GitHub] [airflow] codecov-io edited a comment on issue #6421: [AIRFLOW-5741] Move Cloud Natural Language to providers

2019-10-30 Thread GitBox
codecov-io edited a comment on issue #6421: [AIRFLOW-5741] Move Cloud Natural Language to providers URL: https://github.com/apache/airflow/pull/6421#issuecomment-546633480 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6421?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6421: [AIRFLOW-5741] Move Cloud Natural Language to providers

2019-10-30 Thread GitBox
codecov-io edited a comment on issue #6421: [AIRFLOW-5741] Move Cloud Natural Language to providers URL: https://github.com/apache/airflow/pull/6421#issuecomment-546633480 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6421?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-5818) Very bad webserver performance when defining many dags with many operators

2019-10-30 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963455#comment-16963455 ] Ash Berlin-Taylor commented on AIRFLOW-5818: I think we have fixed this in a different way

[GitHub] [airflow] ashb commented on a change in pull request #6473: [AIRFLOW-5819] Update AWSBatchOperator default value

2019-10-30 Thread GitBox
ashb commented on a change in pull request #6473: [AIRFLOW-5819] Update AWSBatchOperator default value URL: https://github.com/apache/airflow/pull/6473#discussion_r340871679 ## File path: airflow/contrib/operators/awsbatch_operator.py ## @@ -83,7 +83,7 @@ class

[GitHub] [airflow] ashb commented on issue #6462: [AIRFLOW-4708] Make tests/security pylint compatible.

2019-10-30 Thread GitBox
ashb commented on issue #6462: [AIRFLOW-4708] Make tests/security pylint compatible. URL: https://github.com/apache/airflow/pull/6462#issuecomment-548122169 You should be removing the files from scripts/ci/pylint_todo.txt when you make these changes.

[GitHub] [airflow] ashb commented on a change in pull request #6461: [AIRFLOW-5804] Batch the xcom pull operation

2019-10-30 Thread GitBox
ashb commented on a change in pull request #6461: [AIRFLOW-5804] Batch the xcom pull operation URL: https://github.com/apache/airflow/pull/6461#discussion_r340864756 ## File path: airflow/models/xcom.py ## @@ -124,85 +122,70 @@ def set( @classmethod

[GitHub] [airflow] ashb commented on a change in pull request #6241: [AIRFLOW-3014] Increase connection password length to match model

2019-10-30 Thread GitBox
ashb commented on a change in pull request #6241: [AIRFLOW-3014] Increase connection password length to match model URL: https://github.com/apache/airflow/pull/6241#discussion_r340862444 ## File path: airflow/migrations/versions/37c6b55d15a9_merge_migration_heads.py ## @@

[GitHub] [airflow] kimpers commented on issue #6241: [AIRFLOW-3014] Increase connection password length to match model

2019-10-30 Thread GitBox
kimpers commented on issue #6241: [AIRFLOW-3014] Increase connection password length to match model URL: https://github.com/apache/airflow/pull/6241#issuecomment-548113932 @mik-laj any update on this? I can rebase and update it if needed. Would be really helpful for us to get this change

[GitHub] [airflow] codecov-io commented on issue #6475: [AIRFLOW-XXX]Fix typo in "airflow dags list_runs"

2019-10-30 Thread GitBox
codecov-io commented on issue #6475: [AIRFLOW-XXX]Fix typo in "airflow dags list_runs" URL: https://github.com/apache/airflow/pull/6475#issuecomment-548112681 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6475?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6475: [AIRFLOW-XXX]Fix typo in "airflow dags list_runs"

2019-10-30 Thread GitBox
codecov-io edited a comment on issue #6475: [AIRFLOW-XXX]Fix typo in "airflow dags list_runs" URL: https://github.com/apache/airflow/pull/6475#issuecomment-548112681 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6475?src=pr=h1) Report > Merging

[jira] [Updated] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?

2019-10-30 Thread Vince (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vince updated AIRFLOW-4470: --- Attachment: image-2019-10-30-16-25-14-436.png > RBAC Github Enterprise OAuth provider callback URL? >

[GitHub] [airflow] codecov-io edited a comment on issue #6466: [AIRFLOW-5811] add metric for externally killed task count

2019-10-30 Thread GitBox
codecov-io edited a comment on issue #6466: [AIRFLOW-5811] add metric for externally killed task count URL: https://github.com/apache/airflow/pull/6466#issuecomment-548097115 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6466?src=pr=h1) Report > Merging

[GitHub] [airflow] potiuk commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE]

2019-10-30 Thread GitBox
potiuk commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE] URL: https://github.com/apache/airflow/pull/5659#issuecomment-548097035 Yeah. Snakebite-py3 is not working indeed. I think the only way to go is to implement new Hadoop operators base don another

[jira] [Commented] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?

2019-10-30 Thread Vince (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963393#comment-16963393 ] Vince commented on AIRFLOW-4470: !image-2019-10-30-16-25-14-436.png! > RBAC Github Enterprise OAuth

[GitHub] [airflow] codecov-io commented on issue #6466: [AIRFLOW-5811] add metric for externally killed task count

2019-10-30 Thread GitBox
codecov-io commented on issue #6466: [AIRFLOW-5811] add metric for externally killed task count URL: https://github.com/apache/airflow/pull/6466#issuecomment-548097115 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6466?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?

2019-10-30 Thread Vince (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963388#comment-16963388 ] Vince commented on AIRFLOW-4470: Hi [~Nidhi94_] - to clarify, the callback URL is something you specify

[GitHub] [airflow] KyleJamesWalker commented on issue #4931: [AIRFLOW-4109] Added support for passing the airflow fernet key to th…

2019-10-30 Thread GitBox
KyleJamesWalker commented on issue #4931: [AIRFLOW-4109] Added support for passing the airflow fernet key to th… URL: https://github.com/apache/airflow/pull/4931#issuecomment-548086338 In that case shouldn't `AIRFLOW__CORE__SQL_ALCHEMY_CONN` be removed also?

[GitHub] [airflow] ashb commented on issue #4931: [AIRFLOW-4109] Added support for passing the airflow fernet key to th…

2019-10-30 Thread GitBox
ashb commented on issue #4931: [AIRFLOW-4109] Added support for passing the airflow fernet key to th… URL: https://github.com/apache/airflow/pull/4931#issuecomment-548082230 @dacohen Yeah, maybe that makes sense, except that include the Fernet key "in-the-plain" in the pod defintion, and

[jira] [Assigned] (AIRFLOW-905) Support querying dag_run state by run_id

2019-10-30 Thread Tanay Tummalapalli (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanay Tummalapalli reassigned AIRFLOW-905: -- Assignee: Tanay Tummalapalli (was: Shashi Kant Sharms) > Support querying

[GitHub] [airflow] dacohen commented on issue #4931: [AIRFLOW-4109] Added support for passing the airflow fernet key to th…

2019-10-30 Thread GitBox
dacohen commented on issue #4931: [AIRFLOW-4109] Added support for passing the airflow fernet key to th… URL: https://github.com/apache/airflow/pull/4931#issuecomment-548074625 @ashb, per my previous comment in this thread, what's the reason for requiring end users to pass the Fernet key

[GitHub] [airflow] ttanay commented on issue #6475: [AIRFLOW-XXX]Fix typo in "airflow dags list_runs"

2019-10-30 Thread GitBox
ttanay commented on issue #6475: [AIRFLOW-XXX]Fix typo in "airflow dags list_runs" URL: https://github.com/apache/airflow/pull/6475#issuecomment-548070506 R: @jmcarp This is an automated message from the Apache Git Service.

[GitHub] [airflow] ttanay opened a new pull request #6475: Fix typo in "airflow dags list_runs"

2019-10-30 Thread GitBox
ttanay opened a new pull request #6475: Fix typo in "airflow dags list_runs" URL: https://github.com/apache/airflow/pull/6475 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

[GitHub] [airflow] ishan4488 opened a new pull request #6474: [AIRFLOW-5787] Moving AWS SQS to /providers/aws.

2019-10-30 Thread GitBox
ishan4488 opened a new pull request #6474: [AIRFLOW-5787] Moving AWS SQS to /providers/aws. URL: https://github.com/apache/airflow/pull/6474 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

[jira] [Commented] (AIRFLOW-5787) Migrate AWS SQS to /providers/aws [AIP-21]

2019-10-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963340#comment-16963340 ] ASF GitHub Bot commented on AIRFLOW-5787: - ishan4488 commented on pull request #6474:

[GitHub] [airflow] codecov-io commented on issue #6473: [AIRFLOW-5819] Update AWSBatchOperator default value

2019-10-30 Thread GitBox
codecov-io commented on issue #6473: [AIRFLOW-5819] Update AWSBatchOperator default value URL: https://github.com/apache/airflow/pull/6473#issuecomment-548057865 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6473?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-5525) Extra TaskTemplate and ContainerSpec arguments in SwarmOperator

2019-10-30 Thread Akshesh Doshi (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963310#comment-16963310 ] Akshesh Doshi commented on AIRFLOW-5525: Considering my previous comment, I believe that

[jira] [Commented] (AIRFLOW-5525) Extra TaskTemplate and ContainerSpec arguments in SwarmOperator

2019-10-30 Thread Akshesh Doshi (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963306#comment-16963306 ] Akshesh Doshi commented on AIRFLOW-5525: This task basically requires to add further parameters

[jira] [Commented] (AIRFLOW-5810) Using `-int` with `airflow run` with a kubernetesexecutor does not pass -int to container run command

2019-10-30 Thread Francesco Macagno (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963299#comment-16963299 ] Francesco Macagno commented on AIRFLOW-5810: airflow run; Example:  `airflow run k8s-test

[GitHub] [airflow] KyleJamesWalker commented on issue #4931: [AIRFLOW-4109] Added support for passing the airflow fernet key to th…

2019-10-30 Thread GitBox
KyleJamesWalker commented on issue #4931: [AIRFLOW-4109] Added support for passing the airflow fernet key to th… URL: https://github.com/apache/airflow/pull/4931#issuecomment-548045257 @ashb It works as expected in `1.10.6` thanks! So with 1.10.6 we can do: ```

[GitHub] [airflow] kaxil commented on a change in pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency

2019-10-30 Thread GitBox
kaxil commented on a change in pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency URL: https://github.com/apache/airflow/pull/6470#discussion_r340763142 ## File path: airflow/gcp/hooks/bigquery.py ## @@ -332,19 +332,13 @@ def

[GitHub] [airflow] kaxil commented on a change in pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency

2019-10-30 Thread GitBox
kaxil commented on a change in pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency URL: https://github.com/apache/airflow/pull/6470#discussion_r340763301 ## File path: airflow/gcp/hooks/bigquery.py ## @@ -332,19 +332,13 @@ def

[GitHub] [airflow] yegeniy commented on issue #6281: Run batches of (self-terminating) EMR JobFlows [AIRFLOW-5763]

2019-10-30 Thread GitBox
yegeniy commented on issue #6281: Run batches of (self-terminating) EMR JobFlows [AIRFLOW-5763] URL: https://github.com/apache/airflow/pull/6281#issuecomment-548026133 > couldn't this be implemented as multiple EmrCreateJobFlowOperators and some sensors at the end Yes, one of the

[GitHub] [airflow] elukey commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE]

2019-10-30 Thread GitBox
elukey commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE] URL: https://github.com/apache/airflow/pull/5659#issuecomment-548017600 @potiuk hi! We met at the Apachecon, I am wondering if I can help with anything. What is the current status? From what I can see

[GitHub] [airflow] blcksrx commented on issue #6467: [AIRFLOW-5814] Implementing Presto hook tests

2019-10-30 Thread GitBox
blcksrx commented on issue #6467: [AIRFLOW-5814] Implementing Presto hook tests URL: https://github.com/apache/airflow/pull/6467#issuecomment-548016785 > @bslakter that's ur call I think but I think it makes sense to add it in the same PR. Okay! I will rename my PR and also the Jira

[jira] [Commented] (AIRFLOW-5787) Migrate AWS SQS to /providers/aws [AIP-21]

2019-10-30 Thread Ishan Rastogi (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963237#comment-16963237 ] Ishan Rastogi commented on AIRFLOW-5787: Adding changes related to SQS Hook, Operator and

[GitHub] [airflow] dstandish commented on a change in pull request #5787: [AIRFLOW-5172] Add choice of interval edge scheduling

2019-10-30 Thread GitBox
dstandish commented on a change in pull request #5787: [AIRFLOW-5172] Add choice of interval edge scheduling URL: https://github.com/apache/airflow/pull/5787#discussion_r340744386 ## File path: airflow/jobs/scheduler_job.py ## @@ -648,7 +648,7 @@ def create_dag_run(self,

[GitHub] [airflow] dstandish commented on a change in pull request #5787: [AIRFLOW-5172] Add choice of interval edge scheduling

2019-10-30 Thread GitBox
dstandish commented on a change in pull request #5787: [AIRFLOW-5172] Add choice of interval edge scheduling URL: https://github.com/apache/airflow/pull/5787#discussion_r340744386 ## File path: airflow/jobs/scheduler_job.py ## @@ -648,7 +648,7 @@ def create_dag_run(self,

[GitHub] [airflow] dstandish commented on a change in pull request #5787: [AIRFLOW-5172] Add choice of interval edge scheduling

2019-10-30 Thread GitBox
dstandish commented on a change in pull request #5787: [AIRFLOW-5172] Add choice of interval edge scheduling URL: https://github.com/apache/airflow/pull/5787#discussion_r340744386 ## File path: airflow/jobs/scheduler_job.py ## @@ -648,7 +648,7 @@ def create_dag_run(self,

[jira] [Commented] (AIRFLOW-5819) AWSBatchOperator has invalid default value for array_properties

2019-10-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963225#comment-16963225 ] ASF GitHub Bot commented on AIRFLOW-5819: - domantasjurkus commented on pull request #6473:

[GitHub] [airflow] domantasjurkus opened a new pull request #6473: [AIRFLOW-5819] Update AWSBatchOperator default value

2019-10-30 Thread GitBox
domantasjurkus opened a new pull request #6473: [AIRFLOW-5819] Update AWSBatchOperator default value URL: https://github.com/apache/airflow/pull/6473 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

[GitHub] [airflow] OmerJog commented on issue #6467: [AIRFLOW-5814] Implementing Presto hook tests

2019-10-30 Thread GitBox
OmerJog commented on issue #6467: [AIRFLOW-5814] Implementing Presto hook tests URL: https://github.com/apache/airflow/pull/6467#issuecomment-548009699 @bslakter that's ur call I think but I think it makes sense to add it in the same PR.

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency

2019-10-30 Thread GitBox
nuclearpinguin commented on a change in pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency URL: https://github.com/apache/airflow/pull/6470#discussion_r340741140 ## File path: airflow/gcp/hooks/bigquery.py ## @@ -332,19 +332,13 @@ def

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency

2019-10-30 Thread GitBox
nuclearpinguin commented on a change in pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency URL: https://github.com/apache/airflow/pull/6470#discussion_r340740308 ## File path: airflow/gcp/hooks/bigquery.py ## @@ -332,19 +332,13 @@ def

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency

2019-10-30 Thread GitBox
nuclearpinguin commented on a change in pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency URL: https://github.com/apache/airflow/pull/6470#discussion_r340740540 ## File path: airflow/gcp/hooks/bigquery.py ## @@ -332,19 +332,13 @@ def

[jira] [Commented] (AIRFLOW-5819) AWSBatchOperator has invalid default value for array_properties

2019-10-30 Thread Dom (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963217#comment-16963217 ] Dom commented on AIRFLOW-5819: -- I am not entirely sure why this issue is not capture by the existing test

[jira] [Comment Edited] (AIRFLOW-5819) AWSBatchOperator has invalid default value for array_properties

2019-10-30 Thread Dom (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963217#comment-16963217 ] Dom edited comment on AIRFLOW-5819 at 10/30/19 4:55 PM: I am not entirely sure

[jira] [Closed] (AIRFLOW-5813) Move AWS SQS Hook to /providers/aws/hooks

2019-10-30 Thread Ishan Rastogi (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Rastogi closed AIRFLOW-5813. -- Resolution: Fixed > Move AWS SQS Hook to /providers/aws/hooks >

[jira] [Created] (AIRFLOW-5819) AWSBatchOperator has invalid default value for array_properties

2019-10-30 Thread Dom (Jira)
Dom created AIRFLOW-5819: Summary: AWSBatchOperator has invalid default value for array_properties Key: AIRFLOW-5819 URL: https://issues.apache.org/jira/browse/AIRFLOW-5819 Project: Apache Airflow

[jira] [Closed] (AIRFLOW-5812) Move AWS SQS Sensor to /providers/aws/sensors

2019-10-30 Thread Ishan Rastogi (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Rastogi closed AIRFLOW-5812. -- Resolution: Fixed > Move AWS SQS Sensor to /providers/aws/sensors >

[jira] [Commented] (AIRFLOW-5812) Move AWS SQS Sensor to /providers/aws/sensors

2019-10-30 Thread Ishan Rastogi (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963215#comment-16963215 ] Ishan Rastogi commented on AIRFLOW-5812: Changes related to this ticket have been merged in

[jira] [Commented] (AIRFLOW-5813) Move AWS SQS Hook to /providers/aws/hooks

2019-10-30 Thread Ishan Rastogi (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963216#comment-16963216 ] Ishan Rastogi commented on AIRFLOW-5813: Changes related to this ticket have been merged in

[GitHub] [airflow] feng-tao commented on issue #6199: [AIRFLOW-5562] Skip grant single DAG permissions for Admin role.

2019-10-30 Thread GitBox
feng-tao commented on issue #6199: [AIRFLOW-5562] Skip grant single DAG permissions for Admin role. URL: https://github.com/apache/airflow/pull/6199#issuecomment-548004761 thanks @potiuk , sorry for the hiccup, will keep that in mind.

[GitHub] [airflow] kaxil commented on a change in pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency

2019-10-30 Thread GitBox
kaxil commented on a change in pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency URL: https://github.com/apache/airflow/pull/6470#discussion_r340729445 ## File path: airflow/gcp/hooks/bigquery.py ## @@ -332,19 +332,13 @@ def

[GitHub] [airflow] kaxil commented on a change in pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency

2019-10-30 Thread GitBox
kaxil commented on a change in pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency URL: https://github.com/apache/airflow/pull/6470#discussion_r340728709 ## File path: airflow/gcp/hooks/bigquery.py ## @@ -332,19 +332,13 @@ def

[GitHub] [airflow] blcksrx commented on issue #6467: [AIRFLOW-5814] Implementing Presto hook tests

2019-10-30 Thread GitBox
blcksrx commented on issue #6467: [AIRFLOW-5814] Implementing Presto hook tests URL: https://github.com/apache/airflow/pull/6467#issuecomment-547966446 > can you also add tests for the connection? > Similar to what we have for other dbs >

[GitHub] [airflow] codecov-io edited a comment on issue #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency

2019-10-30 Thread GitBox
codecov-io edited a comment on issue #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency URL: https://github.com/apache/airflow/pull/6470#issuecomment-547949655 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6470?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency

2019-10-30 Thread GitBox
codecov-io edited a comment on issue #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency URL: https://github.com/apache/airflow/pull/6470#issuecomment-547949655 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6470?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io commented on issue #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency

2019-10-30 Thread GitBox
codecov-io commented on issue #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency URL: https://github.com/apache/airflow/pull/6470#issuecomment-547949655 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6470?src=pr=h1) Report > Merging

[GitHub] [airflow] OmerJog commented on issue #6467: [AIRFLOW-5814] Implementing Presto hook tests

2019-10-30 Thread GitBox
OmerJog commented on issue #6467: [AIRFLOW-5814] Implementing Presto hook tests URL: https://github.com/apache/airflow/pull/6467#issuecomment-547940777 can you also add tests for the connection? Similar to what we have for other dbs

[GitHub] [airflow] nuclearpinguin opened a new pull request #6472: [AIRFLOW-YYYY][POC] Running tests with pytest

2019-10-30 Thread GitBox
nuclearpinguin opened a new pull request #6472: [AIRFLOW-][POC] Running tests with pytest URL: https://github.com/apache/airflow/pull/6472 This commit is a base for proof of concept for using pytest for runing Airflow's test suite. Make sure you have checked _all_ steps

[GitHub] [airflow] nuclearpinguin opened a new pull request #6471: [AIRFLOW-YYYY][PoC] Running tests with pytest

2019-10-30 Thread GitBox
nuclearpinguin opened a new pull request #6471: [AIRFLOW-][PoC] Running tests with pytest URL: https://github.com/apache/airflow/pull/6471 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

[GitHub] [airflow] nuclearpinguin closed pull request #6471: [AIRFLOW-YYYY][PoC] Running tests with pytest

2019-10-30 Thread GitBox
nuclearpinguin closed pull request #6471: [AIRFLOW-][PoC] Running tests with pytest URL: https://github.com/apache/airflow/pull/6471 This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Created] (AIRFLOW-5818) Very bad webserver performance when defining many dags with many operators

2019-10-30 Thread Gary Harpaz (Jira)
Gary Harpaz created AIRFLOW-5818: Summary: Very bad webserver performance when defining many dags with many operators Key: AIRFLOW-5818 URL: https://issues.apache.org/jira/browse/AIRFLOW-5818

[GitHub] [airflow] codecov-io commented on issue #6467: [AIRFLOW-5814] Implementing Presto hook tests

2019-10-30 Thread GitBox
codecov-io commented on issue #6467: [AIRFLOW-5814] Implementing Presto hook tests URL: https://github.com/apache/airflow/pull/6467#issuecomment-547924590 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6467?src=pr=h1) Report > Merging

[GitHub] [airflow] Fokko commented on a change in pull request #6461: [AIRFLOW-5804] Batch the xcom pull operation

2019-10-30 Thread GitBox
Fokko commented on a change in pull request #6461: [AIRFLOW-5804] Batch the xcom pull operation URL: https://github.com/apache/airflow/pull/6461#discussion_r340618741 ## File path: airflow/models/xcom.py ## @@ -124,85 +122,71 @@ def set( @classmethod

[GitHub] [airflow] iroddis commented on a change in pull request #5787: [AIRFLOW-5172] Add choice of interval edge scheduling

2019-10-30 Thread GitBox
iroddis commented on a change in pull request #5787: [AIRFLOW-5172] Add choice of interval edge scheduling URL: https://github.com/apache/airflow/pull/5787#discussion_r340614930 ## File path: airflow/jobs/scheduler_job.py ## @@ -648,7 +648,7 @@ def create_dag_run(self,

[GitHub] [airflow] nuclearpinguin commented on issue #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency

2019-10-30 Thread GitBox
nuclearpinguin commented on issue #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency URL: https://github.com/apache/airflow/pull/6470#issuecomment-547902311 @TobKed can you take a look? This is an automated message

[jira] [Commented] (AIRFLOW-5817) Improve BigQuery operators idempotency

2019-10-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963015#comment-16963015 ] ASF GitHub Bot commented on AIRFLOW-5817: - nuclearpinguin commented on pull request #6470:

[GitHub] [airflow] Fokko commented on a change in pull request #6461: [AIRFLOW-5804] Batch the xcom pull operation

2019-10-30 Thread GitBox
Fokko commented on a change in pull request #6461: [AIRFLOW-5804] Batch the xcom pull operation URL: https://github.com/apache/airflow/pull/6461#discussion_r340611565 ## File path: airflow/models/taskinstance.py ## @@ -1376,17 +1375,20 @@ def xcom_pull( if dag_id

[GitHub] [airflow] nuclearpinguin opened a new pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency

2019-10-30 Thread GitBox
nuclearpinguin opened a new pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency URL: https://github.com/apache/airflow/pull/6470 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

  1   2   >