[GitHub] [airflow] yuqian90 commented on a change in pull request #14640: Allow ExternalTaskSensor to wait for taskgroup

2021-04-16 Thread GitBox
yuqian90 commented on a change in pull request #14640: URL: https://github.com/apache/airflow/pull/14640#discussion_r614600073 ## File path: airflow/models/dag.py ## @@ -745,10 +745,18 @@ def tasks(self, val): def task_ids(self) -> List[str]: return list(self.task

[GitHub] [airflow] yuqian90 commented on a change in pull request #15382: Fix dag.clear() to set multiple dags to running when necessary

2021-04-16 Thread GitBox
yuqian90 commented on a change in pull request #15382: URL: https://github.com/apache/airflow/pull/15382#discussion_r614626032 ## File path: airflow/models/taskinstance.py ## @@ -205,19 +206,25 @@ def clear_task_instances( for job in session.query(BaseJob).filter(Base

[GitHub] [airflow] quoc-t-le commented on issue #15396: Airflow 2.0.1: Dag - Subdag - Scheduler issues

2021-04-16 Thread GitBox
quoc-t-le commented on issue #15396: URL: https://github.com/apache/airflow/issues/15396#issuecomment-820982551 Just read about TaskGroups and was able to convert subdags overand i can confirm that the encoding for sql to gcs is fixed for exporting format json that contains emojis also

[GitHub] [airflow] quoc-t-le closed issue #15396: Airflow 2.0.1: Dag - Subdag - Scheduler issues

2021-04-16 Thread GitBox
quoc-t-le closed issue #15396: URL: https://github.com/apache/airflow/issues/15396 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] r12habh commented on issue #14231: Quick start for Airflow on Mac OS

2021-04-16 Thread GitBox
r12habh commented on issue #14231: URL: https://github.com/apache/airflow/issues/14231#issuecomment-820984778 @mik-laj It would be great if new contributors who use macOS can get an installation guide for airflow 2.0.1dev like this https://medium.com/@rafaelbottega/from-zero-to-apac

[GitHub] [airflow] r12habh edited a comment on issue #14231: Quick start for Airflow on Mac OS

2021-04-16 Thread GitBox
r12habh edited a comment on issue #14231: URL: https://github.com/apache/airflow/issues/14231#issuecomment-820984778 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For que

[airflow] branch v2-0-test updated (20d4c38 -> a46e809)

2021-04-16 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch v2-0-test in repository https://gitbox.apache.org/repos/asf/airflow.git. discard 20d4c38 Test! new 0e0dc73 Bugfix: ``TypeError`` when Serializing & sorting iterables (#15395) new 3e3e4

[airflow] 04/04: Fixes pushing constraints (#15243)

2021-04-16 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v2-0-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit a46e809731241caa7dba5e983e8677ca7e539e79 Author: Jarek Potiuk AuthorDate: Wed Apr 7 12:22:21 2021 +0200 Fixes push

[airflow] 03/04: Do not remove 'full-tests-needed' when approval missing (#15175)

2021-04-16 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v2-0-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 3369e2f6d275083f5b1b01df9d3299104f9340b8 Author: Jarek Potiuk AuthorDate: Wed Apr 14 13:00:22 2021 +0200 Do not re

[airflow] 01/04: Bugfix: ``TypeError`` when Serializing & sorting iterables (#15395)

2021-04-16 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v2-0-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 0e0dc73c0b511598414f8d89e595efd09e5087ad Author: Kaxil Naik AuthorDate: Fri Apr 16 03:39:26 2021 +0100 Bugfix: ``T

[airflow] 02/04: Don't try to push the python build image when building on release branches (#15394)

2021-04-16 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v2-0-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 3e3e4507e4b3aed9a1d3a95035ce6d2937279a09 Author: Ash Berlin-Taylor AuthorDate: Fri Apr 16 07:57:16 2021 +0100 Don'

[GitHub] [airflow] itroulli opened a new pull request #15400: Adds a test for the description field in variable

2021-04-16 Thread GitBox
itroulli opened a new pull request #15400: URL: https://github.com/apache/airflow/pull/15400 Related: #15194 In continuation of PR #15194, I've added a test for the description field of the variable. Thank you very much @kaxil, @xinbinhuang for all the tips & hints. Let me

[GitHub] [airflow] ashb commented on a change in pull request #15398: WIP: Uses the resource and action naming scheme.

2021-04-16 Thread GitBox
ashb commented on a change in pull request #15398: URL: https://github.com/apache/airflow/pull/15398#discussion_r614661550 ## File path: airflow/hooks/base.py ## @@ -18,14 +18,12 @@ """Base class for all hooks""" import logging import warnings -from typing import TYPE_CHECKI

[GitHub] [airflow] ashb commented on pull request #12822: Removing redundant calls to session.commit()

2021-04-16 Thread GitBox
ashb commented on pull request #12822: URL: https://github.com/apache/airflow/pull/12822#issuecomment-821017415 @SamWheating Would you like one of us to finish of this PR for you? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [airflow] alexInhert opened a new issue #15404: Improve KubernetesPodOperator error message for env_vars

2021-04-16 Thread GitBox
alexInhert opened a new issue #15404: URL: https://github.com/apache/airflow/issues/15404 passing str to `env_vars` gives the following error: ![Screen Shot 2021-04-16 at 12 21 22](https://user-images.githubusercontent.com/62940116/115003269-51466c80-9eae-11eb-95c4-b54ab66f28d1.png)

[GitHub] [airflow] boring-cyborg[bot] commented on issue #15405: AUTH_OAUTH: Access is Denied for: can_list on: CustomUserOAuthModelView

2021-04-16 Thread GitBox
boring-cyborg[bot] commented on issue #15405: URL: https://github.com/apache/airflow/issues/15405#issuecomment-821055812 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] johnskopis opened a new issue #15405: AUTH_OAUTH: Access is Denied for: can_list on: CustomUserOAuthModelView

2021-04-16 Thread GitBox
johnskopis opened a new issue #15405: URL: https://github.com/apache/airflow/issues/15405 **Apache Airflow version**: 2.1.0.dev0 (f94effeab1a943a7fad19845c10a2b3ea2731fce) ``` alabaster==0.7.12 alembic==1.5.8 -e git://github.com/apache/airflow@f94effeab1a943a7fad19845c10a2b3ea2

[GitHub] [airflow] itroulli commented on a change in pull request #15194: Adds description field in variable (#12413)

2021-04-16 Thread GitBox
itroulli commented on a change in pull request #15194: URL: https://github.com/apache/airflow/pull/15194#discussion_r614724786 ## File path: airflow/models/variable.py ## @@ -47,12 +47,14 @@ class Variable(Base, LoggingMixin): id = Column(Integer, primary_key=True) ke

[GitHub] [airflow] SamWheating commented on pull request #12822: Removing redundant calls to session.commit()

2021-04-16 Thread GitBox
SamWheating commented on pull request #12822: URL: https://github.com/apache/airflow/pull/12822#issuecomment-821091046 I'm on it - sorry for taking so long, this one keeps falling of my radar somehow. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] kaxil merged pull request #15400: Adds a test for the description field in variable

2021-04-16 Thread GitBox
kaxil merged pull request #15400: URL: https://github.com/apache/airflow/pull/15400 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] kaxil commented on pull request #15400: Adds a test for the description field in variable

2021-04-16 Thread GitBox
kaxil commented on pull request #15400: URL: https://github.com/apache/airflow/pull/15400#issuecomment-821107671 Thanks @itroulli 🎉 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[airflow] branch master updated (f94effe -> e7c642b)

2021-04-16 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from f94effe Don't try to push the python build image when building on release branches (#15394) add e7c642b A

[GitHub] [airflow] mik-laj commented on a change in pull request #15385: Added table to view providers in airflow ui under admin tab

2021-04-16 Thread GitBox
mik-laj commented on a change in pull request #15385: URL: https://github.com/apache/airflow/pull/15385#discussion_r614773231 ## File path: airflow/www/views.py ## @@ -71,6 +71,7 @@ set_dag_run_state_to_failed, set_dag_run_state_to_success, ) +from airflow.cli.comman

[GitHub] [airflow] mik-laj commented on a change in pull request #15398: WIP: Uses the resource and action naming scheme.

2021-04-16 Thread GitBox
mik-laj commented on a change in pull request #15398: URL: https://github.com/apache/airflow/pull/15398#discussion_r614775012 ## File path: airflow/hooks/base.py ## @@ -18,14 +18,12 @@ """Base class for all hooks""" import logging import warnings -from typing import TYPE_CHE

[GitHub] [airflow] thejens commented on a change in pull request #15367: Implement BigQuery Table Schema Patch Operator

2021-04-16 Thread GitBox
thejens commented on a change in pull request #15367: URL: https://github.com/apache/airflow/pull/15367#discussion_r614775355 ## File path: airflow/providers/google/cloud/operators/bigquery.py ## @@ -2039,6 +2039,149 @@ def execute(self, context) -> None: ) +class

[GitHub] [airflow] thejens commented on a change in pull request #15367: Implement BigQuery Table Schema Patch Operator

2021-04-16 Thread GitBox
thejens commented on a change in pull request #15367: URL: https://github.com/apache/airflow/pull/15367#discussion_r614776461 ## File path: airflow/providers/google/cloud/operators/bigquery.py ## @@ -2039,6 +2039,149 @@ def execute(self, context) -> None: ) +class

[GitHub] [airflow] thejens commented on a change in pull request #15367: Implement BigQuery Table Schema Patch Operator

2021-04-16 Thread GitBox
thejens commented on a change in pull request #15367: URL: https://github.com/apache/airflow/pull/15367#discussion_r614778960 ## File path: airflow/providers/google/cloud/operators/bigquery.py ## @@ -2039,6 +2039,149 @@ def execute(self, context) -> None: ) +class

[GitHub] [airflow] izhangzhihao commented on issue #15319: [Remote Logging] Support MinIO & Aliyun OSS & other storage

2021-04-16 Thread GitBox
izhangzhihao commented on issue #15319: URL: https://github.com/apache/airflow/issues/15319#issuecomment-821189306 > @izhangzhihao Currently Its possible to use S3 compatible object storage. you have to create an S3 connection in airflow with these extra args: > > ``` > { >

[GitHub] [airflow] ryanahamilton commented on pull request #15385: Added table to view providers in airflow ui under admin tab

2021-04-16 Thread GitBox
ryanahamilton commented on pull request #15385: URL: https://github.com/apache/airflow/pull/15385#issuecomment-821205579 Is there a way that we could parse the descriptions to convert the URLs (that most of them contain) to actual links? -- This is an automated message from the Apache Gi

[GitHub] [airflow] github-actions[bot] commented on pull request #15141: Share app instance between Kerberos tests

2021-04-16 Thread GitBox
github-actions[bot] commented on pull request #15141: URL: https://github.com/apache/airflow/pull/15141#issuecomment-821206131 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not m

[GitHub] [airflow] xinbinhuang commented on pull request #15400: Adds a test for the description field in variable

2021-04-16 Thread GitBox
xinbinhuang commented on pull request #15400: URL: https://github.com/apache/airflow/pull/15400#issuecomment-821208549 Thank you @itroulli ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [airflow] uranusjr commented on pull request #15141: Share app instance between Kerberos tests

2021-04-16 Thread GitBox
uranusjr commented on pull request #15141: URL: https://github.com/apache/airflow/pull/15141#issuecomment-821208750 No since I haven't figured out a way to set up Kerberos to run these tests locally 😟 -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] jhtimmins commented on a change in pull request #15158: Permission guard templated field rendering

2021-04-16 Thread GitBox
jhtimmins commented on a change in pull request #15158: URL: https://github.com/apache/airflow/pull/15158#discussion_r614876105 ## File path: airflow/security/permissions.py ## @@ -32,6 +32,7 @@ RESOURCE_JOB = "Jobs" RESOURCE_POOL = "Pools" RESOURCE_PLUGIN = "Plugins" +RESOU

[GitHub] [airflow] eladkal closed issue #15381: Delete DAG crashes

2021-04-16 Thread GitBox
eladkal closed issue #15381: URL: https://github.com/apache/airflow/issues/15381 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please co

[GitHub] [airflow] eladkal commented on issue #15381: Delete DAG crashes

2021-04-16 Thread GitBox
eladkal commented on issue #15381: URL: https://github.com/apache/airflow/issues/15381#issuecomment-82121 Duplicate of https://github.com/apache/airflow/issues/15354 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] jhtimmins commented on a change in pull request #9973: Improve compatibility with mssql

2021-04-16 Thread GitBox
jhtimmins commented on a change in pull request #9973: URL: https://github.com/apache/airflow/pull/9973#discussion_r614881677 ## File path: airflow/www/security.py ## @@ -347,19 +347,23 @@ def can_read_dag(self, dag_id, user=None) -> bool: if not user: use

[GitHub] [airflow] marcosmarxm commented on a change in pull request #15367: Implement BigQuery Table Schema Patch Operator

2021-04-16 Thread GitBox
marcosmarxm commented on a change in pull request #15367: URL: https://github.com/apache/airflow/pull/15367#discussion_r614882725 ## File path: airflow/providers/google/cloud/operators/bigquery.py ## @@ -2039,6 +2039,155 @@ def execute(self, context) -> None: ) +cl

[GitHub] [airflow] kaxil commented on issue #14422: on_failure_callback does not seem to fire on pod deletion/eviction

2021-04-16 Thread GitBox
kaxil commented on issue #14422: URL: https://github.com/apache/airflow/issues/14422#issuecomment-821222484 @houqp This is what I had suggested on call with @ephraimbuddy few days back to add `self.handle_task_exit(1)` ```diff def signal_handler(signum, frame): """Sett

[GitHub] [airflow] ashb merged pull request #15141: Share app instance between Kerberos tests

2021-04-16 Thread GitBox
ashb merged pull request #15141: URL: https://github.com/apache/airflow/pull/15141 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] ashb commented on pull request #15141: Share app instance between Kerberos tests

2021-04-16 Thread GitBox
ashb commented on pull request #15141: URL: https://github.com/apache/airflow/pull/15141#issuecomment-821223370 `./breeze -i kerberos` should give you a docker env with kerberos set up I think. From CI on this branch: ``` 2021-04-01T20:33:12.0023083Z 18.85s setup tests/api/

[airflow] branch master updated (e7c642b -> 54edbaa)

2021-04-16 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from e7c642b Adds a test for the description field in variable (#15400) add 54edbaa Share app instance between Kerbe

[GitHub] [airflow] ashb edited a comment on pull request #15141: Share app instance between Kerberos tests

2021-04-16 Thread GitBox
ashb edited a comment on pull request #15141: URL: https://github.com/apache/airflow/pull/15141#issuecomment-821223370 `./breeze -i kerberos` should give you a docker env with kerberos set up I think. From CI on this branch: ``` 18.85s setup tests/api/auth/backend/test_kerb

[GitHub] [airflow] kaxil edited a comment on pull request #14152: Helm RBAC Best Practices

2021-04-16 Thread GitBox
kaxil edited a comment on pull request #14152: URL: https://github.com/apache/airflow/pull/14152#issuecomment-818314692 > @ashb, @mik-laj, @kaxil This has been rebased with master again. Let me know if there's anything that needs to be addressed. I will take a look in coming days, th

[GitHub] [airflow] marcosmarxm commented on a change in pull request #15393: Add Connection Documentation for Popular Providers

2021-04-16 Thread GitBox
marcosmarxm commented on a change in pull request #15393: URL: https://github.com/apache/airflow/pull/15393#discussion_r614892162 ## File path: docs/apache-airflow-providers-mongo/connections/mongo.rst ## @@ -0,0 +1,82 @@ +.. Licensed to the Apache Software Foundation (ASF) und

[GitHub] [airflow] jhtimmins commented on a change in pull request #15398: WIP: Uses the resource and action naming scheme.

2021-04-16 Thread GitBox
jhtimmins commented on a change in pull request #15398: URL: https://github.com/apache/airflow/pull/15398#discussion_r614896546 ## File path: airflow/hooks/base.py ## @@ -18,14 +18,12 @@ """Base class for all hooks""" import logging import warnings -from typing import TYPE_C

[GitHub] [airflow] ephraimbuddy commented on issue #14422: on_failure_callback does not seem to fire on pod deletion/eviction

2021-04-16 Thread GitBox
ephraimbuddy commented on issue #14422: URL: https://github.com/apache/airflow/issues/14422#issuecomment-821230437 @kaxil , Yes. We both tried it and it didn't work because the task was still in the RUNNING state. I have tried marking the task as Failed when AirflowException is raised and

[GitHub] [airflow] kaxil edited a comment on issue #14422: on_failure_callback does not seem to fire on pod deletion/eviction

2021-04-16 Thread GitBox
kaxil edited a comment on issue #14422: URL: https://github.com/apache/airflow/issues/14422#issuecomment-821222484 @houqp This is what I had suggested on call with @ephraimbuddy few days back to add `self.handle_task_exit(1)` ```diff def signal_handler(signum, frame):

[GitHub] [airflow] aneesh-joseph commented on a change in pull request #9973: Improve compatibility with mssql

2021-04-16 Thread GitBox
aneesh-joseph commented on a change in pull request #9973: URL: https://github.com/apache/airflow/pull/9973#discussion_r614906460 ## File path: airflow/www/security.py ## @@ -347,19 +347,23 @@ def can_read_dag(self, dag_id, user=None) -> bool: if not user:

[GitHub] [airflow] uranusjr commented on pull request #15141: Share app instance between Kerberos tests

2021-04-16 Thread GitBox
uranusjr commented on pull request #15141: URL: https://github.com/apache/airflow/pull/15141#issuecomment-821251521 Thanks, I'll learn that. The gain is actually much more significant than I anticipated! -- This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [airflow] minnieshi commented on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

2021-04-16 Thread GitBox
minnieshi commented on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-821258217 **My situation** - kubenetes - celery executor - dag is correct as it works in other environment. - pool (default_pool), 32 slots, 0 used slots, queued slots =1) -

[GitHub] [airflow] thejens commented on a change in pull request #15367: Implement BigQuery Table Schema Patch Operator

2021-04-16 Thread GitBox
thejens commented on a change in pull request #15367: URL: https://github.com/apache/airflow/pull/15367#discussion_r614928939 ## File path: airflow/providers/google/cloud/operators/bigquery.py ## @@ -2039,6 +2039,149 @@ def execute(self, context) -> None: ) +class

[GitHub] [airflow] thejens commented on a change in pull request #15367: Implement BigQuery Table Schema Patch Operator

2021-04-16 Thread GitBox
thejens commented on a change in pull request #15367: URL: https://github.com/apache/airflow/pull/15367#discussion_r614929411 ## File path: airflow/providers/google/cloud/operators/bigquery.py ## @@ -2039,6 +2039,155 @@ def execute(self, context) -> None: ) +class

[airflow] branch constraints-2-0 updated: Updating constraints. Build id:754833502

2021-04-16 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-2-0 in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-2-0 by this push: new 4ce3900 Updating constra

[GitHub] [airflow] thejens commented on a change in pull request #15367: Implement BigQuery Table Schema Patch Operator

2021-04-16 Thread GitBox
thejens commented on a change in pull request #15367: URL: https://github.com/apache/airflow/pull/15367#discussion_r614930112 ## File path: airflow/providers/google/cloud/operators/bigquery.py ## @@ -2039,6 +2039,155 @@ def execute(self, context) -> None: ) +class

[GitHub] [airflow] minnieshi edited a comment on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

2021-04-16 Thread GitBox
minnieshi edited a comment on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-821258217 **My situation** - kubenetes - celery executor - only 1 dag is 'on', the rest 20 dags are 'off' - dag is correct as it works in other environment. - pool (de

[airflow] branch master updated (54edbaa -> adbab36)

2021-04-16 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 54edbaa Share app instance between Kerberos tests (#15141) add adbab36 Add changelog for what will become 2.0.2

[GitHub] [airflow] sunkickr commented on a change in pull request #15393: Add Connection Documentation for Popular Providers

2021-04-16 Thread GitBox
sunkickr commented on a change in pull request #15393: URL: https://github.com/apache/airflow/pull/15393#discussion_r614938268 ## File path: airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py ## @@ -34,8 +34,9 @@ class SparkKubernetesOperator(BaseOperator): :t

[GitHub] [airflow] ashb merged pull request #15380: Add changelog for what will become 2.0.2

2021-04-16 Thread GitBox
ashb merged pull request #15380: URL: https://github.com/apache/airflow/pull/15380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[airflow] branch v2-0-test updated (a46e809 -> 62b5835)

2021-04-16 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch v2-0-test in repository https://gitbox.apache.org/repos/asf/airflow.git. from a46e809 Fixes pushing constraints (#15243) add 62b5835 Add changelog for what will become 2.0.2 (#15380) No

[GitHub] [airflow] ashb opened a new pull request #15406: Prepare

2021-04-16 Thread GitBox
ashb opened a new pull request #15406: URL: https://github.com/apache/airflow/pull/15406 **DO NOT MERGE** For approval only. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [airflow] github-actions[bot] commented on pull request #15367: Implement BigQuery Table Schema Patch Operator

2021-04-16 Thread GitBox
github-actions[bot] commented on pull request #15367: URL: https://github.com/apache/airflow/pull/15367#issuecomment-821268825 [The Workflow run](https://github.com/apache/airflow/actions/runs/756152896) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build

[GitHub] [airflow] minnieshi edited a comment on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

2021-04-16 Thread GitBox
minnieshi edited a comment on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-821258217 **My situation** - kubenetes - celery executor - only 1 dag is 'on', the rest 20 dags are 'off' - dag is correct as it works in other environment. - pool (de

[GitHub] [airflow] thejens commented on a change in pull request #15367: Implement BigQuery Table Schema Patch Operator

2021-04-16 Thread GitBox
thejens commented on a change in pull request #15367: URL: https://github.com/apache/airflow/pull/15367#discussion_r614945105 ## File path: airflow/providers/google/cloud/operators/bigquery.py ## @@ -2039,6 +2039,155 @@ def execute(self, context) -> None: ) +class

[GitHub] [airflow] minnieshi edited a comment on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

2021-04-16 Thread GitBox
minnieshi edited a comment on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-821258217 **My situation** - kubenetes - airflow 1.10.14 - celery executor - only 1 dag is 'on', the rest 20 dags are 'off' - dag is correct as it works in other enviro

[airflow] branch v2-0-test updated (62b5835 -> e494306)

2021-04-16 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch v2-0-test in repository https://gitbox.apache.org/repos/asf/airflow.git. from 62b5835 Add changelog for what will become 2.0.2 (#15380) add e494306 Update version to 2.0.2 No new revisio

[GitHub] [airflow] xinbinhuang commented on pull request #13796: Fix S3ToFTPOperator

2021-04-16 Thread GitBox
xinbinhuang commented on pull request #13796: URL: https://github.com/apache/airflow/pull/13796#issuecomment-821279946 @JavierLopezT can you fix the pylint error thanks :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [airflow] minnieshi edited a comment on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

2021-04-16 Thread GitBox
minnieshi edited a comment on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-821258217 **My situation** - kubenetes - airflow 1.10.14 - celery executor - only 1 dag is 'on', the rest 20 dags are 'off' - dag is correct as it works in other enviro

[GitHub] [airflow] minnieshi edited a comment on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

2021-04-16 Thread GitBox
minnieshi edited a comment on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-821258217 **My situation** - kubenetes - airflow 1.10.14 - celery executor - only 1 dag is 'on', the rest 20 dags are 'off' - dag is correct as it works in other enviro

[GitHub] [airflow] github-actions[bot] commented on pull request #15406: Prepare 2.0.2

2021-04-16 Thread GitBox
github-actions[bot] commented on pull request #15406: URL: https://github.com/apache/airflow/pull/15406#issuecomment-821282239 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

[GitHub] [airflow] ashb merged pull request #15406: Prepare 2.0.2

2021-04-16 Thread GitBox
ashb merged pull request #15406: URL: https://github.com/apache/airflow/pull/15406 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[airflow] branch v2-0-stable updated (beb8af5 -> e494306)

2021-04-16 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch v2-0-stable in repository https://gitbox.apache.org/repos/asf/airflow.git. from beb8af5 Update Changelog for 2.0.1rc2 add 417d7cb Fix spellings add 94ae38f Restores flexible install

[airflow] annotated tag constraints-2.0.2rc1 updated (4ce3900 -> 7c73284)

2021-04-16 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to annotated tag constraints-2.0.2rc1 in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag constraints-2.0.2rc1 was modified! *** from 4ce3900 (commit) to 7c73284 (tag)

[GitHub] [airflow] SalmonTimo commented on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

2021-04-16 Thread GitBox
SalmonTimo commented on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-821292708 @minnieshi Do you have access to logs from the worker tasks? If so, can you include any errors there? @lukas-at-harren Well done! The second part of your proposed solution

[GitHub] [airflow] jedcunningham commented on issue #15399: Not scheduling since there are (negative number) open slots in pool

2021-04-16 Thread GitBox
jedcunningham commented on issue #15399: URL: https://github.com/apache/airflow/issues/15399#issuecomment-821294376 Yes, `QUEUED` should be included, this might help: https://airflow.apache.org/docs/apache-airflow/stable/concepts.html#task-lifecycle Basically, `QUEUED` means t

[GitHub] [airflow] houqp commented on issue #14422: on_failure_callback does not seem to fire on pod deletion/eviction

2021-04-16 Thread GitBox
houqp commented on issue #14422: URL: https://github.com/apache/airflow/issues/14422#issuecomment-821296305 Interesting, I was expecting the second SIGTERM would have resulted in the task subprocess to set its own state in through `handle_failure` because `self.on_kill` calls `self.task_ru

[GitHub] [airflow] houqp edited a comment on issue #14422: on_failure_callback does not seem to fire on pod deletion/eviction

2021-04-16 Thread GitBox
houqp edited a comment on issue #14422: URL: https://github.com/apache/airflow/issues/14422#issuecomment-821296305 Interesting, I was expecting the second SIGTERM would have resulted in the task subprocess to set its own state in through `handle_failure` because `self.on_kill` calls `self.

svn commit: r47124 - /dev/airflow/2.0.2rc1/

2021-04-16 Thread ash
Author: ash Date: Fri Apr 16 16:35:08 2021 New Revision: 47124 Log: Add artefacts from Airflow 2.0.02rc1 Added: dev/airflow/2.0.2rc1/ dev/airflow/2.0.2rc1/apache-airflow-2.0.2rc1-bin.tar.gz (with props) dev/airflow/2.0.2rc1/apache-airflow-2.0.2rc1-bin.tar.gz.asc (with props) d

[GitHub] [airflow] Dr-Denzy commented on issue #13918: KubernetesPodOperator with pod_template_file = No Metadata & Wrong Pod Name

2021-04-16 Thread GitBox
Dr-Denzy commented on issue #13918: URL: https://github.com/apache/airflow/issues/13918#issuecomment-821305671 Using airflow 2.0.0 ... kubectl description of the `privileged-pod` gotten did not show task metadata (dag_id, task_id, etc). However, `airflow 2.0.1` with `apache-airflow-prov

[GitHub] [airflow] quoc-t-le opened a new issue #15407: ShortCircuitOperator to short circuit downstream of TaskGroup only

2021-04-16 Thread GitBox
quoc-t-le opened a new issue #15407: URL: https://github.com/apache/airflow/issues/15407 **Description** I am trying to convert a SubDag to a TaskGroup which uses the ShortCircuitOperator. Originally, the short circuit will trip and move on to next SubDag. I cannot get the same be

[airflow] annotated tag 2.0.2rc1 updated (e494306 -> d28a705)

2021-04-16 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to annotated tag 2.0.2rc1 in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag 2.0.2rc1 was modified! *** from e494306 (commit) to d28a705 (tag) tagging e494306fb01f3a

[GitHub] [airflow] sunkickr opened a new pull request #15408: Add Connection Documentation to more Providers

2021-04-16 Thread GitBox
sunkickr opened a new pull request #15408: URL: https://github.com/apache/airflow/pull/15408 This PR adds and updates documentation for connecting to some popular providers. It also adds links to this documentation in the doc strings of modules that use each connection. Documentation for t

[GitHub] [airflow] houqp edited a comment on issue #14422: on_failure_callback does not seem to fire on pod deletion/eviction

2021-04-16 Thread GitBox
houqp edited a comment on issue #14422: URL: https://github.com/apache/airflow/issues/14422#issuecomment-821296305 Interesting, I was expecting the second SIGTERM would have resulted in the task subprocess to set its own state in through `handle_failure` because `self.on_kill` calls `self.

[GitHub] [airflow] blcksrx commented on issue #15332: SftpSensor w/ possibility to use RegEx or fnmatch

2021-04-16 Thread GitBox
blcksrx commented on issue #15332: URL: https://github.com/apache/airflow/issues/15332#issuecomment-821319755 it sounds for *nix OS that provides shell. it's convenient to use wildcards like this: ``` hook.get_conn().execute("ls PATH/*.csv") ``` but it is too raw and not useable

[GitHub] [airflow] kaxil merged pull request #15352: Update azure connection documentation

2021-04-16 Thread GitBox
kaxil merged pull request #15352: URL: https://github.com/apache/airflow/pull/15352 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[airflow] branch master updated (adbab36 -> cb1344b)

2021-04-16 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from adbab36 Add changelog for what will become 2.0.2 (#15380) add cb1344b Update azure connection documentatio

[GitHub] [airflow] quoc-t-le commented on issue #15355: MysqlHook Utf8mb4

2021-04-16 Thread GitBox
quoc-t-le commented on issue #15355: URL: https://github.com/apache/airflow/issues/15355#issuecomment-821324755 I was able to get Airflow 2 running...the problem is fixed there. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [airflow] sosso commented on issue #14205: Scheduler "deadlocks" itself when max_active_runs_per_dag is reached by up_for_retry tasks

2021-04-16 Thread GitBox
sosso commented on issue #14205: URL: https://github.com/apache/airflow/issues/14205#issuecomment-821326846 +1 for us on this issue as well, I think? Very strangely, we see the most recent run for a DAG have its *run* be set to 'running', but the only task in the DAG be a clear success:

[GitHub] [airflow] sosso edited a comment on issue #14205: Scheduler "deadlocks" itself when max_active_runs_per_dag is reached by up_for_retry tasks

2021-04-16 Thread GitBox
sosso edited a comment on issue #14205: URL: https://github.com/apache/airflow/issues/14205#issuecomment-821326846 +1 for us on this issue as well, I think? Very strangely, we see the most recent run for a DAG have its *run* be set to 'running', but the only task in the DAG be a clear suc

[GitHub] [airflow] eladkal commented on issue #15407: ShortCircuitOperator to short circuit downstream of TaskGroup only

2021-04-16 Thread GitBox
eladkal commented on issue #15407: URL: https://github.com/apache/airflow/issues/15407#issuecomment-821334673 `ShortCircuitOperator` cascades through everything by design https://github.com/apache/airflow/issues/7858 Since TaskGroup is more of a UI feature this is expected. However s

[GitHub] [airflow] eladkal edited a comment on issue #15407: ShortCircuitOperator to short circuit downstream of TaskGroup only

2021-04-16 Thread GitBox
eladkal edited a comment on issue #15407: URL: https://github.com/apache/airflow/issues/15407#issuecomment-821334673 `ShortCircuitOperator` cascades through everything by design https://github.com/apache/airflow/issues/7858 Since TaskGroup is more of a UI feature this is expected. Ho

[GitHub] [airflow] eladkal commented on a change in pull request #15408: Add Connection Documentation to more Providers

2021-04-16 Thread GitBox
eladkal commented on a change in pull request #15408: URL: https://github.com/apache/airflow/pull/15408#discussion_r615024146 ## File path: docs/apache-airflow-providers-snowflake/connections/snowflake.rst ## @@ -0,0 +1,76 @@ +.. Licensed to the Apache Software Foundation (ASF)

[GitHub] [airflow] eladkal edited a comment on issue #15407: ShortCircuitOperator to short circuit downstream of TaskGroup only

2021-04-16 Thread GitBox
eladkal edited a comment on issue #15407: URL: https://github.com/apache/airflow/issues/15407#issuecomment-821334673 `ShortCircuitOperator` cascades through everything by design https://github.com/apache/airflow/issues/7858 Since TaskGroup is more of a UI feature this is expected. Ho

[GitHub] [airflow] eladkal commented on a change in pull request #15408: Add Connection Documentation to more Providers

2021-04-16 Thread GitBox
eladkal commented on a change in pull request #15408: URL: https://github.com/apache/airflow/pull/15408#discussion_r615024146 ## File path: docs/apache-airflow-providers-snowflake/connections/snowflake.rst ## @@ -0,0 +1,76 @@ +.. Licensed to the Apache Software Foundation (ASF)

[GitHub] [airflow] eladkal commented on a change in pull request #15408: Add Connection Documentation to more Providers

2021-04-16 Thread GitBox
eladkal commented on a change in pull request #15408: URL: https://github.com/apache/airflow/pull/15408#discussion_r615024146 ## File path: docs/apache-airflow-providers-snowflake/connections/snowflake.rst ## @@ -0,0 +1,76 @@ +.. Licensed to the Apache Software Foundation (ASF)

[GitHub] [airflow] minnieshi commented on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

2021-04-16 Thread GitBox
minnieshi commented on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-821356514 @SalmonTimo i have access to the workers, as well as the dag logs folder, which is saved in the file share (PVC), can be viewed via azure storage explorer. The environment is

[GitHub] [airflow] minnieshi edited a comment on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

2021-04-16 Thread GitBox
minnieshi edited a comment on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-821356514 @SalmonTimo i have access to the workers, as well as the dag logs folder, which is saved in the file share (PVC), can be viewed via azure storage explorer. The environ

[GitHub] [airflow] minnieshi edited a comment on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

2021-04-16 Thread GitBox
minnieshi edited a comment on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-821356514 @SalmonTimo i have access to the workers, as well as the dag logs folder, which is saved in the file share (PVC), can be viewed via azure storage explorer. The environ

[GitHub] [airflow] minnieshi edited a comment on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

2021-04-16 Thread GitBox
minnieshi edited a comment on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-821356514 @SalmonTimo i have access to the workers, as well as the dag logs folder, which is saved in the file share (PVC), can be viewed via azure storage explorer. The environ

[GitHub] [airflow] minnieshi edited a comment on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

2021-04-16 Thread GitBox
minnieshi edited a comment on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-821356514 @SalmonTimo i have access to the workers, as well as the dag logs folder (which is saved in the file share -PVC, and can be viewed via azure storage explorer). ps.

[GitHub] [airflow] sosso edited a comment on issue #14205: Scheduler "deadlocks" itself when max_active_runs_per_dag is reached by up_for_retry tasks

2021-04-16 Thread GitBox
sosso edited a comment on issue #14205: URL: https://github.com/apache/airflow/issues/14205#issuecomment-821326846 +1 for us on this issue as well, I think? Very strangely, we see the most recent run for a DAG have its *run* be set to 'running', but the only task in the DAG be a clear suc

  1   2   >