[GitHub] [airflow] riccardo-salamanna commented on a change in pull request #21401: added envFrom statement to the flower deployment

[GitBox]

riccardo-salamanna commented on a change in pull request #21401:
URL: https://github.com/apache/airflow/pull/21401#discussion_r801349808



##
File path: chart/templates/flower/flower-deployment.yaml
##
@@ -128,6 +128,8 @@ spec:
   name: {{ template "flower_secret" . }}
   key: basicAuth
 {{- end }}
+  envFrom:
+  {{- include "custom_airflow_environment_from" . | default "\n  []" | 
indent 10 }}

Review comment:
   Thanks for answering and pointing those things out. i **think** it 
should be better now. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] avkirilishin edited a comment on pull request #21214: Removes the next_dagrun_create_after reset

[GitBox]

avkirilishin edited a comment on pull request #21214:
URL: https://github.com/apache/airflow/pull/21214#issuecomment-1032303165


   Can we add `test_runs_are_created_after_max_active_runs_was_reached` to 
#21413 from this merge request?
   
https://github.com/apache/airflow/blob/039ba528b870e3935780d99f02370d816feb1b61/tests/jobs/test_scheduler_job.py#L2927-L2961
   
   I will remove `min_active_runs_check_interval` from it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] avkirilishin edited a comment on pull request #21214: Removes the next_dagrun_create_after reset

[GitBox]

avkirilishin edited a comment on pull request #21214:
URL: https://github.com/apache/airflow/pull/21214#issuecomment-1032303165


   Can we add `test_runs_are_created_after_max_active_runs_was_reached` to 
#21413 from this merge request?
   
https://github.com/apache/airflow/blob/039ba528b870e3935780d99f02370d816feb1b61/tests/jobs/test_scheduler_job.py#L2927-L2961


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] avkirilishin commented on pull request #21214: Removes the next_dagrun_create_after reset

[GitBox]

avkirilishin commented on pull request #21214:
URL: https://github.com/apache/airflow/pull/21214#issuecomment-1032303165


   Can we add `test_runs_are_created_after_max_active_runs_was_reached` to 
#21413 from this merge request?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ephraimbuddy commented on pull request #21214: Removes the next_dagrun_create_after reset

[GitBox]

ephraimbuddy commented on pull request #21214:
URL: https://github.com/apache/airflow/pull/21214#issuecomment-1032286602


   I have added the PR: https://github.com/apache/airflow/pull/21413


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ephraimbuddy opened a new pull request #21413: Fix max_active_runs=1 not scheduling runs when min_file_process_interval is high

[GitBox]

ephraimbuddy opened a new pull request #21413:
URL: https://github.com/apache/airflow/pull/21413


   When we make a query to get the total active runs, it returns the dagrun 
being
   examined as running. Therefore we subtract the dagrun being examined from
   active runs.
   
   There's a bug in this query `dag.get_num_active_runs` but for now we should 
get this fixed this way
   
   closes: https://github.com/apache/airflow/issues/21083
   closes: https://github.com/apache/airflow/issues/19901
   
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Acehaidrey commented on pull request #20733: Add Audit Log View to Dag View

[GitBox]

Acehaidrey commented on pull request #20733:
URL: https://github.com/apache/airflow/pull/20733#issuecomment-1032262412


   I have tried everything I can to get the docker-compose quick start test to 
complete successfully. I see other PRs also failing similarly. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] BenGatewood opened a new issue #21412: v3.5.0 airflow.providers.microsoft.azure.operators.cosmos not running

[GitBox]

BenGatewood opened a new issue #21412:
URL: https://github.com/apache/airflow/issues/21412


   ### Apache Airflow version
   
   2.2.3 (latest released)
   
   ### What happened
   
   Submitting this on advice from the community Slack: Attempting to use the 
v3.5.0 `AzureCosmosInsertDocumentOperator` operator fails with an attribute 
error: `AttributeError: 'CosmosClient' object has no attribute 'QueryDatabases'`
   
   ### What you expected to happen
   
   Expected behaviour is that the document is upserted correctly.  I've traced 
through the source and `does_database_exist()` seems to call `QueryDatabases()` 
 on the result of `self.get_conn()`. Thing is `get_conn()` (AFAICT) returns an 
actual MS/AZ `CosmosClient` which definitely does not have a `QueryDatabases()` 
method (it's `query_databases()`)
   
   ### How to reproduce
   
   From what I can see, any attempt to use this operator on airflow 2.2.3 will 
fail in this way
   
   ### Operating System
   
   Ubuntu 18.04.5 LTS
   
   ### Versions of Apache Airflow Providers
   
   azure-batch==12.0.0
   azure-common==1.1.28
   azure-core==1.22.0
   azure-cosmos==4.2.0
   azure-datalake-store==0.0.52
   azure-identity==1.7.1
   azure-keyvault==4.1.0
   azure-keyvault-certificates==4.3.0
   azure-keyvault-keys==4.4.0
   azure-keyvault-secrets==4.3.0
   azure-kusto-data==0.0.45
   azure-mgmt-containerinstance==1.5.0
   azure-mgmt-core==1.3.0
   azure-mgmt-datafactory==1.1.0
   azure-mgmt-datalake-nspkg==3.0.1
   azure-mgmt-datalake-store==0.5.0
   azure-mgmt-nspkg==3.0.2
   azure-mgmt-resource==20.1.0
   azure-nspkg==3.0.2
   azure-storage-blob==12.8.1
   azure-storage-common==2.1.0
   azure-storage-file==2.1.0
   msrestazure==0.6.4
   
   ### Deployment
   
   Virtualenv installation
   
   ### Deployment details
   
   Clean standalone install I am using for evaluating airflow for our 
environment
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #21412: v3.5.0 airflow.providers.microsoft.azure.operators.cosmos not running

[GitBox]

boring-cyborg[bot] commented on issue #21412:
URL: https://github.com/apache/airflow/issues/21412#issuecomment-1032239971


   Thanks for opening your first issue here! Be sure to follow the issue 
template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] josh-fell opened a new pull request #21411: Update Zendesk example DAG to use TaskFlow API

[GitBox]

josh-fell opened a new pull request #21411:
URL: https://github.com/apache/airflow/pull/21411


   Related: #9415
   
   Staying consistent with previous work that was done to convert example DAGs 
to the functional style of authoring, this PR updates the example DAG for 
Zendesk to use the TaskFlow API.
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (471e368 -> 34d63fa)

[kamilbregula]

This is an automated email from the ASF dual-hosted git repository.

kamilbregula pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git.


from 471e368  [Oracle] Oracle Hook - automatically set current_schema when 
defined in Connection (#19084)
 add 34d63fa  Fix BigQueryDataTransferServiceHook.get_transfer_run() 
request parameter (#21293)

No new revisions were added by this update.

Summary of changes:
 airflow/providers/google/cloud/hooks/bigquery_dts.py|  2 +-
 tests/providers/google/cloud/hooks/test_bigquery_dts.py | 12 
 2 files changed, 13 insertions(+), 1 deletion(-)

[GitHub] [airflow] mik-laj merged pull request #21293: Fix BigQueryDataTransferServiceHook.get_transfer_run() request parameter

[GitBox]

mik-laj merged pull request #21293:
URL: https://github.com/apache/airflow/pull/21293


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #21293: Fix BigQueryDataTransferServiceHook.get_transfer_run() request parameter

[GitBox]

boring-cyborg[bot] commented on pull request #21293:
URL: https://github.com/apache/airflow/pull/21293#issuecomment-1032194740


   Awesome work, congrats on your first merged pull request!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] danmactough commented on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

[GitBox]

danmactough commented on issue #13542:
URL: https://github.com/apache/airflow/issues/13542#issuecomment-1032153919


   > Thre is also 2.2.2 as of recently. Can you please upgrade and check it 
there @danmactough ?
   
   Wow @potiuk I totally missed that update! Huge news! I'll check that out and 
see if it helps.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] zmac12 opened a new pull request #21408: Remove extra 'of'

[GitBox]

zmac12 opened a new pull request #21408:
URL: https://github.com/apache/airflow/pull/21408


   Just a double 'of', removes the second one.
   
   
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #21408: Remove extra 'of'

[GitBox]

boring-cyborg[bot] commented on pull request #21408:
URL: https://github.com/apache/airflow/pull/21408#issuecomment-1032152939


   Congratulations on your first Pull Request and welcome to the Apache Airflow 
community! If you have any issues or are unsure about any anything please check 
our Contribution Guide 
(https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
   Here are some useful points:
   - Pay attention to the quality of your code (flake8, mypy and type 
annotations). Our [pre-commits]( 
https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks)
 will help you with that.
   - In case of a new feature add useful documentation (in docstrings or in 
`docs/` directory). Adding a new operator? Check this short 
[guide](https://github.com/apache/airflow/blob/main/docs/apache-airflow/howto/custom-operator.rst)
 Consider adding an example DAG that shows how users should use it.
   - Consider using [Breeze 
environment](https://github.com/apache/airflow/blob/main/BREEZE.rst) for 
testing locally, it’s a heavy docker but it ships with a working Airflow and a 
lot of integrations.
   - Be patient and persistent. It might take some time to get a review or get 
the final approval from Committers.
   - Please follow [ASF Code of 
Conduct](https://www.apache.org/foundation/policies/conduct) for all 
communication including (but not limited to) comments on Pull Requests, Mailing 
list and Slack.
   - Be sure to read the [Airflow Coding style]( 
https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#coding-style-and-best-practices).
   Apache Airflow is a community-driven project and together we are making it 
better .
   In case of doubts contact the developers at:
   Mailing List: d...@airflow.apache.org
   Slack: https://s.apache.org/airflow-slack
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] pingzh commented on pull request #21296: Fix relationship join bug in FAB/SecurityManager with SQLA 1.4

[GitBox]

pingzh commented on pull request #21296:
URL: https://github.com/apache/airflow/pull/21296#issuecomment-1032131134


   could you please also put more context information about the relationship 
join bug that this pr tried to fix?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] pingzh edited a comment on issue #21407: AssertionError: Dependency rule tried to blank-out primary key column 'ab_permission.id' on instance '

[GitBox]

pingzh edited a comment on issue #21407:
URL: https://github.com/apache/airflow/issues/21407#issuecomment-1032125080


   i think it should be related to this PR 
https://github.com/apache/airflow/pull/21296
   
   after I reseted to a commit before that PR. it worked.
   
   How to reproduce:
   
   1. drop my mysql database
   2. pull latest main branch
   3. `airflow db init`
   4. `airflow webserver` --> will fail
   5. `git reset --hard e1a2717dae4614dbd3523bb1baacdecbdeadfcac` # reset 
commit back to that pr.
   6. `airflow webserver` --> fails again
   7. `git reset --hard 40d3a76a9bce2360b951f2e990cba571c5f51a76` # 1 commit 
before that pr
   8. `airflow webserver` --> works


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] pingzh commented on pull request #21296: Fix relationship join bug in FAB/SecurityManager with SQLA 1.4

[GitBox]

pingzh commented on pull request #21296:
URL: https://github.com/apache/airflow/pull/21296#issuecomment-1032125611


   @ashb , looks like this pr has caused this issue 
https://github.com/apache/airflow/issues/21407
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] pingzh commented on issue #21407: AssertionError: Dependency rule tried to blank-out primary key column 'ab_permission.id' on instance '

[GitBox]

pingzh commented on issue #21407:
URL: https://github.com/apache/airflow/issues/21407#issuecomment-1032125080


   i think it should be related to this PR 
https://github.com/apache/airflow/pull/21296
   
   after I reseted to a commit before that PR. it worked.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kbaruri commented on issue #14320: Testing tasks with pytest: sqlalchemy.orm.exc.NoResultFound raised on a task's first .run()

[GitBox]

kbaruri commented on issue #14320:
URL: https://github.com/apache/airflow/issues/14320#issuecomment-1032123233


   +1 encountered this issue while running a similar test using the 
`python:3.7` docker image.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch constraints-2-2 updated: Updating constraints. Build id:1809570159

[github-bot]

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch constraints-2-2
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/constraints-2-2 by this push:
 new 2e47d4d  Updating constraints. Build id:1809570159
2e47d4d is described below

commit 2e47d4d3cfd03b66de66237a616b1492fca031c7
Author: Automated GitHub Actions commit 
AuthorDate: Tue Feb 8 01:07:26 2022 +

Updating constraints. Build id:1809570159

This update in constraints is automatically committed by the CI 
'constraints-push' step based on
HEAD of 'refs/heads/v2-2-test' in 'apache/airflow'
with commit sha b92cf65f0300bbac305f19980384110283eba4d5.

All tests passed in this build so we determined we can push the updated 
constraints.

See 
https://github.com/apache/airflow/blob/main/README.md#installing-from-pypi for 
details.
---
 constraints-3.6.txt  | 28 ++--
 constraints-3.7.txt  | 30 +++---
 constraints-3.8.txt  | 30 +++---
 constraints-3.9.txt  | 30 +++---
 constraints-no-providers-3.6.txt | 10 +-
 constraints-no-providers-3.7.txt |  8 
 constraints-no-providers-3.8.txt | 10 +-
 constraints-no-providers-3.9.txt | 10 +-
 constraints-source-providers-3.6.txt | 28 ++--
 constraints-source-providers-3.7.txt | 30 +++---
 constraints-source-providers-3.8.txt | 30 +++---
 constraints-source-providers-3.9.txt | 30 +++---
 12 files changed, 137 insertions(+), 137 deletions(-)

diff --git a/constraints-3.6.txt b/constraints-3.6.txt
index ae688b8..d9dc0a6 100644
--- a/constraints-3.6.txt
+++ b/constraints-3.6.txt
@@ -1,5 +1,5 @@
 #
-# This constraints file was automatically generated on 2022-02-03T23:43:44Z
+# This constraints file was automatically generated on 2022-02-08T01:03:09Z
 # via "eager-upgrade" mechanism of PIP. For the "v2-2-test" branch of Airflow.
 # This variant of constraints install uses the HEAD of the branch version for 
'apache-airflow' but installs
 # the providers from PIP-released packages at the moment of the constraint 
generation.
@@ -51,7 +51,7 @@ aiohttp==3.8.1
 aiosignal==1.2.0
 alabaster==0.7.12
 alembic==1.7.6
-aliyun-python-sdk-core==2.13.35
+aliyun-python-sdk-core==2.13.36
 aliyun-python-sdk-kms==2.15.0
 amqp==5.0.9
 analytics-python==1.4.0
@@ -281,7 +281,7 @@ google-cloud-videointelligence==1.16.1
 google-cloud-vision==1.0.0
 google-cloud-workflows==1.5.0
 google-crc32c==1.3.0
-google-resumable-media==2.1.0
+google-resumable-media==2.2.0
 googleapis-common-protos==1.54.0
 graphviz==0.19.1
 greenlet==1.1.2
@@ -293,7 +293,7 @@ gunicorn==20.1.0
 h11==0.12.0
 hdfs==2.6.0
 hmsclient==0.1.1
-httpcore==0.14.6
+httpcore==0.14.7
 httplib2==0.19.1
 httpx==0.22.0
 humanize==3.14.0
@@ -380,7 +380,7 @@ parameterized==0.8.1
 paramiko==2.9.2
 parso==0.7.1
 pathspec==0.9.0
-pbr==5.8.0
+pbr==5.8.1
 pdpyras==4.4.0
 pendulum==2.1.2
 pep562==1.1
@@ -398,8 +398,8 @@ pre-commit==2.17.0
 presto-python-client==0.7.0
 prison==0.2.1
 prometheus-client==0.13.1
-prompt-toolkit==3.0.26
-proto-plus==1.19.9
+prompt-toolkit==3.0.27
+proto-plus==1.20.0
 protobuf==3.19.4
 psutil==5.9.0
 psycopg2-binary==2.9.3
@@ -414,8 +414,8 @@ pyasn1==0.4.8
 pycodestyle==2.8.0
 pycountry==22.1.10
 pycparser==2.21
-pycryptodome==3.14.0
-pycryptodomex==3.14.0
+pycryptodome==3.14.1
+pycryptodomex==3.14.1
 pydata-google-auth==1.3.0
 pydot==1.4.2
 pydruid==0.6.2
@@ -435,7 +435,7 @@ pyspnego==0.3.1
 pytest-asyncio==0.16.0
 pytest-cov==3.0.0
 pytest-forked==1.4.0
-pytest-httpx==0.19.0
+pytest-httpx==0.20.0
 pytest-instafail==0.4.2
 pytest-rerunfailures==9.1.1
 pytest-timeouts==1.2.1
@@ -459,7 +459,7 @@ pyzmq==22.3.0
 qds-sdk==1.16.1
 readme-renderer==32.0
 redis==3.5.3
-redshift-connector==2.0.903
+redshift-connector==2.0.904
 requests-file==1.5.1
 requests-kerberos==0.14.0
 requests-mock==1.9.3
@@ -482,7 +482,7 @@ sentry-sdk==1.5.4
 setproctitle==1.2.2
 simple-salesforce==1.11.5
 six==1.16.0
-slack-sdk==3.13.0
+slack-sdk==3.14.0
 smbprotocol==1.9.0
 smmap==5.0.0
 snakebite-py3==3.0.5
@@ -495,7 +495,7 @@ soupsieve==2.3.1
 sphinx-airflow-theme==0.0.6
 sphinx-argparse==0.3.1
 sphinx-autoapi==1.8.4
-sphinx-copybutton==0.4.0
+sphinx-copybutton==0.5.0
 sphinx-jinja==1.4.0
 sphinx-rtd-theme==1.0.0
 sphinxcontrib-applehelp==1.0.2
@@ -543,7 +543,7 @@ uritemplate==3.0.1
 urllib3==1.26.8
 vertica-python==1.0.3
 vine==5.0.0
-virtualenv==20.13.0
+virtualenv==20.13.1
 volatile==2.1.0
 watchtower==2.0.1
 wcwidth==0.2.5
diff --git a/constraints-3.7.txt b/constraints-3.7.txt
index 4e57c65..2718c9f 100644
--- a/constraints-3.7.txt
+++ b/constraints-3.7.txt
@@ -1,5 +1,5 @@
 #
-# This constraints file was automatically generated on 

[GitHub] [airflow] pingzh opened a new issue #21407: AssertionError: Dependency rule tried to blank-out primary key column 'ab_permission.id' on instance '

[GitBox]

pingzh opened a new issue #21407:
URL: https://github.com/apache/airflow/issues/21407


   ### Apache Airflow version
   
   main (development)
   
   ### What happened
   
   airflow webserver cannot start due to error
   
   ```
   AssertionError: Dependency rule tried to blank-out primary key column 
'ab_permission.id' on instance '
   ```
   
   ### What you expected to happen
   
   airflow webserver cannot start due to error
   
   ### How to reproduce
   
   Error in CI:
   
https://github.com/apache/airflow/runs/5101435727?check_suite_focus=true#step:7:2424
   
   Error from my local:
   
   i have the latest airflow main commit: 
`471e368eacbcae1eedf9b7e1cb4290c385396ea9`
   
   python version: `3.9.7`
   Flask related pkg:
   
   ```
   ❯ pip list | grep Flas
   Flask   1.1.4
   Flask-AppBuilder3.4.4
   Flask-Babel 2.0.0
   Flask-Caching   1.10.1
   Flask-JWT-Extended  3.25.1
   Flask-Login 0.4.1
   Flask-OpenID1.3.0
   Flask-SQLAlchemy2.5.1
   Flask-WTF   0.14.3
   ```
   
   database: mysql
   
   mysql version:
   
   ```
   ❯ mysql -V
   mysql  Ver 8.0.28 for macos12.0 on arm64 (Homebrew)
   ```
   
   
   the aiflow.cfg and webserver conf are auto generated, i did not change 
except `sql_alchemy_conn`
   
   
   ### Operating System
   
   Apple M1 Max, version: 12.2
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Other
   
   ### Deployment details
   
   local dev
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] blag opened a new pull request #21406: Tweak origin parameter

[GitBox]

blag opened a new pull request #21406:
URL: https://github.com/apache/airflow/pull/21406


   
   
   This is a simpler alternative to #21310.
   
   The `origin` parameter is used to take users back to a previous page they 
were on. This PR fixes the way this parameter was handled.
   
   The tiny [`html`](https://docs.python.org/3/library/html.html) library was 
added in Python 3.2 and expanded in Python 3.4.
   
   I changed the `` to a normal ``, since it 
is more appropriate. The 
[commit](https://github.com/apache/airflow/pull/11195/files#diff-c7993103f49b0ad44fbe04d22a72f17989dd8908efd88c2ee92ce47ae21dc3c4R37)
 that introduced the `` behavior was part of the [2.0 UI 
refresh PR](https://github.com/apache/airflow/pull/11195).
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] dstandish commented on a change in pull request #21398: Add pre-commit check for docstring param types

[GitBox]

dstandish commented on a change in pull request #21398:
URL: https://github.com/apache/airflow/pull/21398#discussion_r801178851



##
File path: scripts/ci/pre_commit/pre_commit_docstring_param_type.py
##
@@ -0,0 +1,58 @@
+#!/usr/bin/env python3
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+import re
+import sys
+from pathlib import Path
+
+from rich.console import Console
+
+if __name__ not in ("__main__", "__mp_main__"):
+raise SystemExit(
+"This file is intended to be executed as an executable program. You 
cannot use it as a module."
+f"To run this script, run the ./{__file__} command [FILE] ..."
+)
+
+
+console = Console(color_system="standard", width=200)
+
+
+def _check_file(file: Path) -> list:
+content = file.read_text()
+return re.findall(r' +\:type .+?\:', content)

Review comment:
   see description  here 
https://github.com/apache/airflow/pull/21014#issue-1110403553




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] dstandish commented on a change in pull request #21398: Add pre-commit check for docstring param types

[GitBox]

dstandish commented on a change in pull request #21398:
URL: https://github.com/apache/airflow/pull/21398#discussion_r801174509



##
File path: scripts/ci/pre_commit/pre_commit_docstring_param_type.py
##
@@ -0,0 +1,58 @@
+#!/usr/bin/env python3
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+import re
+import sys
+from pathlib import Path
+
+from rich.console import Console
+
+if __name__ not in ("__main__", "__mp_main__"):
+raise SystemExit(
+"This file is intended to be executed as an executable program. You 
cannot use it as a module."
+f"To run this script, run the ./{__file__} command [FILE] ..."
+)
+
+
+console = Console(color_system="standard", width=200)
+
+
+def _check_file(file: Path) -> list:
+content = file.read_text()
+return re.findall(r' +\:type .+?\:', content)

Review comment:
   Apparently we can't do rtype at this time because not supported by sphinx
   
   I'll try and link relevant pr comment later




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] 05/07: Fix docs link for smart sensor deprecation (#21394)

[jedcunningham]

This is an automated email from the ASF dual-hosted git repository.

jedcunningham pushed a commit to branch v2-2-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit 8971e0ff4283febfdb8a292e978ee1247f583f0d
Author: Jed Cunningham <66968678+jedcunning...@users.noreply.github.com>
AuthorDate: Mon Feb 7 08:55:20 2022 -0700

Fix docs link for smart sensor deprecation (#21394)

We are releasing the deprecation in version 2.2.4, not 2.3.0 like
originally planned.

(cherry picked from commit 3a780380d8f5d50ffc876c326e70ee0eee033c0d)
---
 UPDATING.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/UPDATING.md b/UPDATING.md
index a75b2d6..2ed4aac 100644
--- a/UPDATING.md
+++ b/UPDATING.md
@@ -87,7 +87,7 @@ https://developers.google.com/style/inclusive-documentation
 
 Smart sensors, an "early access" feature added in Airflow 2, are now 
deprecated and will be removed in Airflow 2.4.0. They have been superseded by 
Deferable Operators, added in Airflow 2.2.0.
 
-See [Migrating to Deferrable 
Operators](https://airflow.apache.org/docs/apache-airflow/2.3.0/concepts/smart-sensors.html#migrating-to-deferrable-operators)
 for details on how to migrate.
+See [Migrating to Deferrable 
Operators](https://airflow.apache.org/docs/apache-airflow/2.2.4/concepts/smart-sensors.html#migrating-to-deferrable-operators)
 for details on how to migrate.
 
 ## Airflow 2.2.3
 

[airflow] 06/07: Avoid deadlock when rescheduling task (#21362)

[jedcunningham]

This is an automated email from the ASF dual-hosted git repository.

jedcunningham pushed a commit to branch v2-2-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit e2f3e94f054bc7217378aa743d185e5e508468ea
Author: Jarek Potiuk 
AuthorDate: Mon Feb 7 20:12:05 2022 +0100

Avoid deadlock when rescheduling task (#21362)

The scheduler job performs scheduling after locking the "scheduled"
DagRun row for writing. This should prevent from modifying DagRun
and related task instances by another scheduler or "mini-scheduler"
run after task is completed.

However there is apparently one more case where the DagRun is being
locked by "Task" processes - namely when task throws
AirflowRescheduleException. In this case a new "TaskReschedule"
entity is inserted into the database and it also performs lock
on the DagRun (because TaskReschedule has "DagRun" relationship.

This PR modifies handling the AirflowRescheduleException to obtain the
very same DagRun lock before it attempts to insert TaskReschedule
entity.

Seems that TaskReschedule is the only one that has this relationship
so likely all the misterious SchedulerJob deadlock cases we
experienced might be explained (and fixed) by this one.

It is likely that this one:

* Fixes: #16982
* Fixes: #19957

(cherry picked from commit 6d110b565a505505351d1ff19592626fb24e4516)
---
 airflow/models/taskinstance.py | 15 ++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/airflow/models/taskinstance.py b/airflow/models/taskinstance.py
index ec34156..2dcc923 100644
--- a/airflow/models/taskinstance.py
+++ b/airflow/models/taskinstance.py
@@ -93,7 +93,7 @@ from airflow.utils.operator_helpers import 
context_to_airflow_vars
 from airflow.utils.platform import getuser
 from airflow.utils.retries import run_with_db_retries
 from airflow.utils.session import create_session, provide_session
-from airflow.utils.sqlalchemy import ExtendedJSON, UtcDateTime
+from airflow.utils.sqlalchemy import ExtendedJSON, UtcDateTime, with_row_locks
 from airflow.utils.state import DagRunState, State
 from airflow.utils.timeout import timeout
 
@@ -1657,11 +1657,24 @@ class TaskInstance(Base, LoggingMixin):
 # Don't record reschedule request in test mode
 if test_mode:
 return
+
+from airflow.models.dagrun import DagRun  # Avoid circular import
+
 self.refresh_from_db(session)
 
 self.end_date = timezone.utcnow()
 self.set_duration()
 
+# Lock DAG run to be sure not to get into a deadlock situation when 
trying to insert
+# TaskReschedule which apparently also creates lock on corresponding 
DagRun entity
+with_row_locks(
+session.query(DagRun).filter_by(
+dag_id=self.dag_id,
+run_id=self.run_id,
+),
+session=session,
+).one()
+
 # Log reschedule request
 session.add(
 TaskReschedule(

[airflow] 07/07: Add possibility to create user in the Remote User mode (#19963)

[jedcunningham]

This is an automated email from the ASF dual-hosted git repository.

jedcunningham pushed a commit to branch v2-2-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit b92cf65f0300bbac305f19980384110283eba4d5
Author: Łukasz Wyszomirski 
AuthorDate: Fri Jan 28 06:18:05 2022 +0100

Add possibility to create user in the Remote User mode (#19963)

(cherry picked from commit cdd9ea66208e3d70d1cf2a34530ba69bc3c58a50)
---
 airflow/www/views.py | 25 +
 1 file changed, 25 insertions(+)

diff --git a/airflow/www/views.py b/airflow/www/views.py
index f2642a7..2ed2a67 100644
--- a/airflow/www/views.py
+++ b/airflow/www/views.py
@@ -4731,3 +4731,28 @@ class CustomUserOIDModelView(MultiResourceUserMixin, 
UserOIDModelView):
 
 class CustomUserRemoteUserModelView(MultiResourceUserMixin, 
UserRemoteUserModelView):
 """Customize permission names for FAB's builtin UserRemoteUserModelView."""
+
+_class_permission_name = permissions.RESOURCE_USER
+
+class_permission_name_mapping = {
+'userinfoedit': permissions.RESOURCE_MY_PROFILE,
+'userinfo': permissions.RESOURCE_MY_PROFILE,
+}
+
+method_permission_name = {
+'add': 'create',
+'userinfo': 'read',
+'download': 'read',
+'show': 'read',
+'list': 'read',
+'edit': 'edit',
+'userinfoedit': 'edit',
+'delete': 'delete',
+}
+
+base_permissions = [
+permissions.ACTION_CAN_CREATE,
+permissions.ACTION_CAN_READ,
+permissions.ACTION_CAN_EDIT,
+permissions.ACTION_CAN_DELETE,
+]

[airflow] 03/07: Filter celery stuck task query to exclude completed tasks (#21335)

[jedcunningham]

This is an automated email from the ASF dual-hosted git repository.

jedcunningham pushed a commit to branch v2-2-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit 317dd298e02f54156f362f8d39307e575106b5b5
Author: Ephraim Anierobi 
AuthorDate: Mon Feb 7 10:43:51 2022 +0100

Filter celery stuck task query to exclude completed tasks (#21335)

On testing #19769, it was reported that there was a spike in CPU usage 
https://github.com/apache/airflow/pull/19769#issuecomment-1029755436
Hopefully, this will fix it

(cherry picked from commit a49224fa7ce45e9765c0d752edc30430e0d3ce14)
---
 airflow/executors/celery_executor.py| 7 ++-
 tests/executors/test_celery_executor.py | 8 ++--
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/airflow/executors/celery_executor.py 
b/airflow/executors/celery_executor.py
index 8daced6..9185f1b 100644
--- a/airflow/executors/celery_executor.py
+++ b/airflow/executors/celery_executor.py
@@ -404,7 +404,12 @@ class CeleryExecutor(BaseExecutor):
 session_ = app.backend.ResultSession()
 task_cls = getattr(app.backend, "task_cls", TaskDb)
 with session_cleanup(session_):
-celery_task_ids = [t.task_id for t in 
session_.query(task_cls.task_id).all()]
+celery_task_ids = [
+t.task_id
+for t in session_.query(task_cls.task_id)
+.filter(~task_cls.status.in_([celery_states.SUCCESS, 
celery_states.FAILURE]))
+.all()
+]
 self.log.debug("Checking for stuck queued tasks")
 
 max_allowed_time = utcnow() - self.task_adoption_timeout
diff --git a/tests/executors/test_celery_executor.py 
b/tests/executors/test_celery_executor.py
index 5632f7d..f057440 100644
--- a/tests/executors/test_celery_executor.py
+++ b/tests/executors/test_celery_executor.py
@@ -42,7 +42,7 @@ from parameterized import parameterized
 from airflow.configuration import conf
 from airflow.exceptions import AirflowException, AirflowTaskTimeout
 from airflow.executors import celery_executor
-from airflow.executors.celery_executor import BulkStateFetcher
+from airflow.executors.celery_executor import BulkStateFetcher, CeleryExecutor
 from airflow.models.baseoperator import BaseOperator
 from airflow.models.dag import DAG
 from airflow.models.taskinstance import SimpleTaskInstance, TaskInstance, 
TaskInstanceKey
@@ -532,6 +532,7 @@ class TestCeleryExecutor:
 assert ti.state == state
 
 @mock.patch("celery.backends.database.DatabaseBackend.ResultSession")
+@mock.patch.object(CeleryExecutor, "update_all_task_states")
 @pytest.mark.backend("mysql", "postgres")
 @freeze_time("2020-01-01")
 @pytest.mark.parametrize(
@@ -543,6 +544,7 @@ class TestCeleryExecutor:
 )
 def 
test_the_check_interval_to_clear_stuck_queued_task_is_correct_for_db_query(
 self,
+mock_update_all_task_states,
 mock_result_session,
 task_id,
 state,
@@ -557,7 +559,9 @@ class TestCeleryExecutor:
 mock_backend = DatabaseBackend(app=celery_executor.app, 
url="sqlite3://")
 with 
mock.patch('airflow.executors.celery_executor.Celery.backend', mock_backend):
 mock_session = mock_backend.ResultSession.return_value
-mock_session.query.return_value.all.return_value = 
[result_obj("SUCCESS", task_id)]
+
mock_session.query.return_value.filter.return_value.all.return_value = [
+result_obj("SUCCESS", task_id)
+]
 
 last_check_time = time.time() - 302  # should clear ti state
 

[airflow] 04/07: Update example DAGs (#21372)

[jedcunningham]

This is an automated email from the ASF dual-hosted git repository.

jedcunningham pushed a commit to branch v2-2-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit 99abbfc873d3df070fff5926d69c31bfad462359
Author: Jed Cunningham <66968678+jedcunning...@users.noreply.github.com>
AuthorDate: Mon Feb 7 11:24:31 2022 -0700

Update example DAGs (#21372)

(cherry picked from commit 7a38ec2ad3b3bd6fda5e1ee9fe9e644ccb8b4c12)
---
 .../example_dags/example_passing_params_via_test_command.py   | 11 ++-
 airflow/example_dags/tutorial.py  |  2 --
 docs/apache-airflow/tutorial.rst  |  9 ++---
 tests/cli/commands/test_task_command.py   |  1 -
 4 files changed, 8 insertions(+), 15 deletions(-)

diff --git a/airflow/example_dags/example_passing_params_via_test_command.py 
b/airflow/example_dags/example_passing_params_via_test_command.py
index e3f04c4..d4781af 100644
--- a/airflow/example_dags/example_passing_params_via_test_command.py
+++ b/airflow/example_dags/example_passing_params_via_test_command.py
@@ -68,17 +68,18 @@ with DAG(
 ) as dag:
 run_this = my_py_command(params={"miff": "agg"})
 
-my_templated_command = dedent(
+my_command = dedent(
+"""
+echo "'foo' was passed in via Airflow CLI Test command with value 
'$FOO'"
+echo "'miff' was passed in via BashOperator with value '$MIFF'"
 """
-echo " 'foo was passed in via Airflow CLI Test command with value {{ 
params.foo }} "
-echo " 'miff was passed in via BashOperator with value {{ params.miff 
}} "
-"""
 )
 
 also_run_this = BashOperator(
 task_id='also_run_this',
-bash_command=my_templated_command,
+bash_command=my_command,
 params={"miff": "agg"},
+env={"FOO": "{{ params.foo }}", "MIFF": "{{ params.miff }}"},
 )
 
 env_var_test_task = print_env_vars()
diff --git a/airflow/example_dags/tutorial.py b/airflow/example_dags/tutorial.py
index 1049772..ff2bd2f 100644
--- a/airflow/example_dags/tutorial.py
+++ b/airflow/example_dags/tutorial.py
@@ -109,7 +109,6 @@ with DAG(
 {% for i in range(5) %}
 echo "{{ ds }}"
 echo "{{ macros.ds_add(ds, 7)}}"
-echo "{{ params.my_param }}"
 {% endfor %}
 """
 )
@@ -118,7 +117,6 @@ with DAG(
 task_id='templated',
 depends_on_past=False,
 bash_command=templated_command,
-params={'my_param': 'Parameter I passed in'},
 )
 # [END jinja_template]
 
diff --git a/docs/apache-airflow/tutorial.rst b/docs/apache-airflow/tutorial.rst
index 1c32e78..0034b2c 100644
--- a/docs/apache-airflow/tutorial.rst
+++ b/docs/apache-airflow/tutorial.rst
@@ -151,13 +151,8 @@ stamp").
 :end-before: [END jinja_template]
 
 Notice that the ``templated_command`` contains code logic in ``{% %}`` blocks,
-references parameters like ``{{ ds }}``, calls a function as in
-``{{ macros.ds_add(ds, 7)}}``, and references a user-defined parameter
-in ``{{ params.my_param }}``.
-
-The ``params`` hook in ``BaseOperator`` allows you to pass a dictionary of
-parameters and/or objects to your templates. Please take the time
-to understand how the parameter ``my_param`` makes it through to the template.
+references parameters like ``{{ ds }}``, and calls a function as in
+``{{ macros.ds_add(ds, 7)}}``.
 
 Files can also be passed to the ``bash_command`` argument, like
 ``bash_command='templated_command.sh'``, where the file location is relative to
diff --git a/tests/cli/commands/test_task_command.py 
b/tests/cli/commands/test_task_command.py
index 201af16..76c6cdb 100644
--- a/tests/cli/commands/test_task_command.py
+++ b/tests/cli/commands/test_task_command.py
@@ -263,7 +263,6 @@ class TestCliTasks(unittest.TestCase):
 
 assert 'echo "2016-01-01"' in output
 assert 'echo "2016-01-08"' in output
-assert 'echo "Parameter I passed in"' in output
 
 def test_cli_run_when_pickle_and_dag_cli_method_selected(self):
 """

[airflow] 02/07: Update error docs to include before_send option (#21275)

[jedcunningham]

This is an automated email from the ASF dual-hosted git repository.

jedcunningham pushed a commit to branch v2-2-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit e11b90cd5800f1071bdee311b8f4cbb5e8d4b611
Author: Abhijeet Prasad 
AuthorDate: Thu Feb 3 18:15:21 2022 -0500

Update error docs to include before_send option (#21275)

https://github.com/apache/airflow/pull/18261 Added support for the 
`before_send` option when initializing the Sentry SDK in airflow. This patch 
updates the documentation to reflect this change.
(cherry picked from commit b38391e2f91760e64576723c876341f532a6ee2d)
---
 docs/apache-airflow/logging-monitoring/errors.rst | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/docs/apache-airflow/logging-monitoring/errors.rst 
b/docs/apache-airflow/logging-monitoring/errors.rst
index 9f4256a..30bb66d 100644
--- a/docs/apache-airflow/logging-monitoring/errors.rst
+++ b/docs/apache-airflow/logging-monitoring/errors.rst
@@ -42,8 +42,14 @@ Add your ``SENTRY_DSN`` to your configuration file e.g. 
``airflow.cfg`` in ``[se
 .. note::
 If this value is not provided, the SDK will try to read it from the 
``SENTRY_DSN`` environment variable.
 
-You can supply `additional configuration options 
`__ based on the 
Python platform via ``[sentry]`` section.
-Unsupported options: ``integrations``, ``in_app_include``, ``in_app_exclude``, 
``ignore_errors``, ``before_breadcrumb``, ``before_send``, ``transport``.
+The ``before_send`` option can be used to modify or drop events before they 
are sent to Sentry. To set this option, provide a dotted path to a before_send 
function that the sentry SDK should be configured to use.
+
+.. code-block:: ini
+
+[sentry]
+before_send = path.to.my.sentry.before_send
+
+You can supply `additional configuration options 
`__ based on the 
Python platform via ``[sentry]`` section. Unsupported options: 
``integrations``, ``in_app_include``, ``in_app_exclude``, ``ignore_errors``, 
``before_breadcrumb``, ``transport``.
 
 Tags
 -

[airflow] 01/07: Fix the incorrect scheduling time for the first run of dag (#21011)

[jedcunningham]

This is an automated email from the ASF dual-hosted git repository.

jedcunningham pushed a commit to branch v2-2-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit a04ead208da620040b718151c0800bb2ff59bbc4
Author: wano <55907021+wan...@users.noreply.github.com>
AuthorDate: Mon Feb 7 02:02:57 2022 +0800

Fix the incorrect scheduling time for the first run of dag (#21011)

When Catchup_by_default is set to false and start_date in the DAG is the
previous day, the first schedule time for this DAG may be incorrect

Co-authored-by: wanlce 
(cherry picked from commit 0bcca55f4881bacc3fbe86f69e71981f5552b398)
---
 airflow/timetables/interval.py  |  2 +-
 tests/timetables/test_interval_timetable.py | 21 +
 2 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/airflow/timetables/interval.py b/airflow/timetables/interval.py
index d669cb6..01fac3a 100644
--- a/airflow/timetables/interval.py
+++ b/airflow/timetables/interval.py
@@ -218,7 +218,7 @@ class CronDataIntervalTimetable(_DataIntervalTimetable):
 raise AssertionError("next schedule shouldn't be earlier")
 if earliest is None:
 return new_start
-return max(new_start, earliest)
+return max(new_start, self._align(earliest))
 
 def infer_manual_data_interval(self, *, run_after: DateTime) -> 
DataInterval:
 # Get the last complete period before run_after, e.g. if a DAG run is
diff --git a/tests/timetables/test_interval_timetable.py 
b/tests/timetables/test_interval_timetable.py
index 842cc1f2..fe09e0c 100644
--- a/tests/timetables/test_interval_timetable.py
+++ b/tests/timetables/test_interval_timetable.py
@@ -35,11 +35,32 @@ PREV_DATA_INTERVAL_END = START_DATE + 
datetime.timedelta(days=1)
 PREV_DATA_INTERVAL = DataInterval(start=PREV_DATA_INTERVAL_START, 
end=PREV_DATA_INTERVAL_END)
 
 CURRENT_TIME = pendulum.DateTime(2021, 9, 7, tzinfo=TIMEZONE)
+YESTERDAY = CURRENT_TIME - datetime.timedelta(days=1)
 
 HOURLY_CRON_TIMETABLE = CronDataIntervalTimetable("@hourly", TIMEZONE)
 HOURLY_TIMEDELTA_TIMETABLE = 
DeltaDataIntervalTimetable(datetime.timedelta(hours=1))
 HOURLY_RELATIVEDELTA_TIMETABLE = 
DeltaDataIntervalTimetable(dateutil.relativedelta.relativedelta(hours=1))
 
+CRON_TIMETABLE = CronDataIntervalTimetable("30 16 * * *", TIMEZONE)
+DELTA_FROM_MIDNIGHT = datetime.timedelta(minutes=30, hours=16)
+
+
+@pytest.mark.parametrize(
+"last_automated_data_interval",
+[pytest.param(None, id="first-run"), pytest.param(PREV_DATA_INTERVAL, 
id="subsequent")],
+)
+@freezegun.freeze_time(CURRENT_TIME)
+def test_no_catchup_first_starts_at_current_time(
+last_automated_data_interval: Optional[DataInterval],
+) -> None:
+"""If ``catchup=False`` and start_date is a day before"""
+next_info = CRON_TIMETABLE.next_dagrun_info(
+last_automated_data_interval=last_automated_data_interval,
+restriction=TimeRestriction(earliest=YESTERDAY, latest=None, 
catchup=False),
+)
+expected_start = YESTERDAY + DELTA_FROM_MIDNIGHT
+assert next_info == DagRunInfo.interval(start=expected_start, 
end=CURRENT_TIME + DELTA_FROM_MIDNIGHT)
+
 
 @pytest.mark.parametrize(
 "timetable",

[airflow] branch v2-2-test updated (d4000e8 -> b92cf65)

[jedcunningham]

This is an automated email from the ASF dual-hosted git repository.

jedcunningham pushed a change to branch v2-2-test
in repository https://gitbox.apache.org/repos/asf/airflow.git.


from d4000e8  Update stat_name_handler documentation (#21298)
 new a04ead2  Fix the incorrect scheduling time for the first run of dag 
(#21011)
 new e11b90c  Update error docs to include before_send option (#21275)
 new 317dd29  Filter celery stuck task query to exclude completed tasks 
(#21335)
 new 99abbfc  Update example DAGs (#21372)
 new 8971e0f  Fix docs link for smart sensor deprecation (#21394)
 new e2f3e94  Avoid deadlock when rescheduling task (#21362)
 new b92cf65  Add possibility to create user in the Remote User mode 
(#19963)

The 7 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 UPDATING.md|  2 +-
 .../example_passing_params_via_test_command.py | 11 +-
 airflow/example_dags/tutorial.py   |  2 --
 airflow/executors/celery_executor.py   |  7 +-
 airflow/models/taskinstance.py | 15 -
 airflow/timetables/interval.py |  2 +-
 airflow/www/views.py   | 25 ++
 docs/apache-airflow/logging-monitoring/errors.rst  | 10 +++--
 docs/apache-airflow/tutorial.rst   |  9 ++--
 tests/cli/commands/test_task_command.py|  1 -
 tests/executors/test_celery_executor.py|  8 +--
 tests/timetables/test_interval_timetable.py| 21 ++
 12 files changed, 90 insertions(+), 23 deletions(-)

[GitHub] [airflow] github-actions[bot] closed pull request #14640: Allow ExternalTaskSensor to wait for taskgroup

[GitBox]

github-actions[bot] closed pull request #14640:
URL: https://github.com/apache/airflow/pull/14640


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] aa3pankaj commented on a change in pull request #21342: Add 'Show record' option for variables

[GitBox]

aa3pankaj commented on a change in pull request #21342:
URL: https://github.com/apache/airflow/pull/21342#discussion_r801157071



##
File path: airflow/www/widgets.py
##
@@ -72,3 +72,8 @@ class BS3TextAreaROWidget(BS3TextAreaFieldWidget):
 def __call__(self, field, **kwargs):
 kwargs['readonly'] = 'true'
 return super().__call__(field, **kwargs)
+
+class AirflowModelShowWidget(RenderTemplateWidget):
+"""Airflow model show widget"""
+
+template = 'airflow/model_show.html'

Review comment:
   Done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kazanzhy edited a comment on issue #21125: apache-airflow-providers-trino issue when I use "run" method, query is not performed

[GitBox]

kazanzhy edited a comment on issue #21125:
URL: https://github.com/apache/airflow/issues/21125#issuecomment-1032050659


   Seems the passing handler is accidentally missed.
   Talking about `fetchAll()`, did you try the `trino_hook.get_records`?
   It returns `cursor.fetchall()`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kazanzhy commented on issue #21125: apache-airflow-providers-trino issue when I use "run" method, query is not performed

[GitBox]

kazanzhy commented on issue #21125:
URL: https://github.com/apache/airflow/issues/21125#issuecomment-1032050659


   Seems the passing handler is accidentally missed.
   Talking about `fetchAll(), did you tried the `trino_hook.get_records`?
   It returns `cursor.fetchall()`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] pingzh commented on pull request #18675: New Tree View

[GitBox]

pingzh commented on pull request #18675:
URL: https://github.com/apache/airflow/pull/18675#issuecomment-1032032585


   @potiuk yep, adding the upstream/downstream to this new Tree view also works 
well. I think it is very important for users to easily see upstream/downstream 
in the Tree view when they do operations for their task runs.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kazanzhy commented on issue #21388: Optionally raise an error if source file does not exist in GCSToGCSOperator

[GitBox]

kazanzhy commented on issue #21388:
URL: https://github.com/apache/airflow/issues/21388#issuecomment-1032029707


   Sounds interesting. It's pretty similar to pandas functions parameter - 
`errors={'ignore', 'raise', 'coerce'}`. But I see that there are a lot of 
different operators and sensors in Airflow that help you build complex 
pipelines with very simple blocks.
   
   I also implemented similar logic to RDS Operators with optional errors in 
the first iteration but after I decided to keep Operators as straightforward 
and simple as possible without any logic under the hood. 
   In this case, `GCSObjectExistenceSensor` might be useful. If something goes 
wrong you see it in the UI.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] pingzh commented on a change in pull request #21332: Add option to compress dag data

[GitBox]

pingzh commented on a change in pull request #21332:
URL: https://github.com/apache/airflow/pull/21332#discussion_r801092172



##
File path: airflow/models/serialized_dag.py
##
@@ -20,11 +20,12 @@
 
 import hashlib
 import logging
+import zlib

Review comment:
   although `bz2` can achieve smaller size of compress data, but we noticed 
that the `zlib` is faster than `bz2` when  compressing and decompressing data 
with the same level of compression (zlib with default level 6, also set 
compresslevel as 6 for bz2).
   
   test with python3.7.9, `raw json data size`: `514MB`, `bz2` compressed: 
`14M` and `zlib` compressed: `44M`.
   
   ```
   ~$ python test_bz2.py
   bz2 took 89.2322297607
   bz2 decompress took 15.989518880844116
   
   
   ~$ python test_zlib.py
   zlib compress took 8.829093933105469
   zlib decompress took 1.1581130027770996
   
   ```
   
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] jedcunningham commented on a change in pull request #21222: Helm: Fix elasticsearch URL in secret when username/password fields are empty,

[GitBox]

jedcunningham commented on a change in pull request #21222:
URL: https://github.com/apache/airflow/pull/21222#discussion_r801116690



##
File path: chart/tests/test_elasticsearch_secret.py
##
@@ -132,3 +132,39 @@ def 
test_should_generate_secret_with_specified_schemes(self, scheme):
 )
 
 assert f"{scheme}://username:password@elastichostname:9200" == 
connection
+
+def test_url_generated_when_user_is_empty(self):
+connection = self._get_connection(
+{
+"elasticsearch": {
+"enabled": True,
+"connection": {"pass": "password", "host": 
"elastichostname", "port": 8080},
+}
+}
+)
+
+assert "http://elastichostname:8080; == connection
+
+def test_url_generated_when_password_is_empty(self):
+connection = self._get_connection(
+{
+"elasticsearch": {
+"enabled": True,
+"connection": {"user": "admin", "host": "elastichostname", 
"port": 8080},
+}
+}
+)
+
+assert "http://elastichostname:8080; == connection
+
+def test_url_generated_with_valid_user_password(self):
+connection = self._get_connection(
+{
+"elasticsearch": {
+"enabled": True,
+"connection": {"user": "admin", "pass": "pass", "host": 
"elastichostname", "port": 8080},
+}
+}
+)
+
+assert "http://admin:pass@elastichostname:8080; == connection

Review comment:
   I was kinda thinking something like this (untested, but might help?):
   
   ```
   @parameterized([
   ({}, ""),
   ({"user": "admin", ""),
   ({"pass": "password", ""),
   ({"user": "admin", "pass": "password", "admin:password"),
   ])
   def test_something(self, extra_conn_kwargs, expected_userinfo):
   connection = self._get_connection({
  "elasticsearch": {
   "enabled": True,
   "connection": {
   "host": "elastichostname",
   "port": 8080,
   **extra_conn_kwargs
}
   }
   })
   
   assert f"http://{expected_userinfo}@elastichostname:8080; == connection
   
   
   I think not being a tuple might be your issue?
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] mik-laj commented on a change in pull request #21398: Add pre-commit check for docstring param types

[GitBox]

mik-laj commented on a change in pull request #21398:
URL: https://github.com/apache/airflow/pull/21398#discussion_r801110671



##
File path: scripts/ci/pre_commit/pre_commit_docstring_param_type.py
##
@@ -0,0 +1,58 @@
+#!/usr/bin/env python3
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+import re
+import sys
+from pathlib import Path
+
+from rich.console import Console
+
+if __name__ not in ("__main__", "__mp_main__"):
+raise SystemExit(
+"This file is intended to be executed as an executable program. You 
cannot use it as a module."
+f"To run this script, run the ./{__file__} command [FILE] ..."
+)
+
+
+console = Console(color_system="standard", width=200)
+
+
+def _check_file(file: Path) -> list:
+content = file.read_text()
+return re.findall(r' +\:type .+?\:', content)

Review comment:
   Can you add a check for `rtype` (return type) also?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] subkanthi commented on a change in pull request #21222: Helm: Fix elasticsearch URL in secret when username/password fields are empty,

[GitBox]

subkanthi commented on a change in pull request #21222:
URL: https://github.com/apache/airflow/pull/21222#discussion_r801109387



##
File path: chart/tests/test_elasticsearch_secret.py
##
@@ -132,3 +132,39 @@ def 
test_should_generate_secret_with_specified_schemes(self, scheme):
 )
 
 assert f"{scheme}://username:password@elastichostname:9200" == 
connection
+
+def test_url_generated_when_user_is_empty(self):
+connection = self._get_connection(
+{
+"elasticsearch": {
+"enabled": True,
+"connection": {"pass": "password", "host": 
"elastichostname", "port": 8080},
+}
+}
+)
+
+assert "http://elastichostname:8080; == connection
+
+def test_url_generated_when_password_is_empty(self):
+connection = self._get_connection(
+{
+"elasticsearch": {
+"enabled": True,
+"connection": {"user": "admin", "host": "elastichostname", 
"port": 8080},
+}
+}
+)
+
+assert "http://elastichostname:8080; == connection
+
+def test_url_generated_with_valid_user_password(self):
+connection = self._get_connection(
+{
+"elasticsearch": {
+"enabled": True,
+"connection": {"user": "admin", "pass": "pass", "host": 
"elastichostname", "port": 8080},
+}
+}
+)
+
+assert "http://admin:pass@elastichostname:8080; == connection

Review comment:
   Thanks, Im actually having problems passing JSON in parametrized.expand, 
trying to figure out, for some reason the parameter only has the first key.
   
   ``` @parameterized.expand(
   [
   # When user is empty
   
   ({"enabled": True, "connection" : {"pass": "password", "host": 
"elastichostname", "port": 8080}}),
   # When pass is empty
   # ({"connection": {"user": "admin", "host": "elastichostname", 
"port": 8080}}),
   # # When user and pass are empty
   # ({"connection": {"host": "elastichostname", "port": 8080}}),
   ],
   )```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] subkanthi commented on a change in pull request #21222: Helm: Fix elasticsearch URL in secret when username/password fields are empty,

[GitBox]

subkanthi commented on a change in pull request #21222:
URL: https://github.com/apache/airflow/pull/21222#discussion_r801109387



##
File path: chart/tests/test_elasticsearch_secret.py
##
@@ -132,3 +132,39 @@ def 
test_should_generate_secret_with_specified_schemes(self, scheme):
 )
 
 assert f"{scheme}://username:password@elastichostname:9200" == 
connection
+
+def test_url_generated_when_user_is_empty(self):
+connection = self._get_connection(
+{
+"elasticsearch": {
+"enabled": True,
+"connection": {"pass": "password", "host": 
"elastichostname", "port": 8080},
+}
+}
+)
+
+assert "http://elastichostname:8080; == connection
+
+def test_url_generated_when_password_is_empty(self):
+connection = self._get_connection(
+{
+"elasticsearch": {
+"enabled": True,
+"connection": {"user": "admin", "host": "elastichostname", 
"port": 8080},
+}
+}
+)
+
+assert "http://elastichostname:8080; == connection
+
+def test_url_generated_with_valid_user_password(self):
+connection = self._get_connection(
+{
+"elasticsearch": {
+"enabled": True,
+"connection": {"user": "admin", "pass": "pass", "host": 
"elastichostname", "port": 8080},
+}
+}
+)
+
+assert "http://admin:pass@elastichostname:8080; == connection

Review comment:
   Thanks, Im actually having problems passing JSON in parametrized.expand
   
   ``` @parameterized.expand(
   [
   # When user is empty
   
   ({"enabled": True, "connection" : {"pass": "password", "host": 
"elastichostname", "port": 8080}}),
   # When pass is empty
   # ({"connection": {"user": "admin", "host": "elastichostname", 
"port": 8080}}),
   # # When user and pass are empty
   # ({"connection": {"host": "elastichostname", "port": 8080}}),
   ],
   )```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] avkirilishin opened a new pull request #21404: EdgeModifier refactoring

[GitBox]

avkirilishin opened a new pull request #21404:
URL: https://github.com/apache/airflow/pull/21404


   closes: #17469
   
   | Dag | Before | After |
   | - | - | - |
   | simple_label | https://user-images.githubusercontent.com/54231417/152872227-3857a811-1c7d-4efc-9e06-180024f2245d.png;>
 | https://user-images.githubusercontent.com/54231417/152867781-6de7552b-a9fa-4564-a2a7-0db9529ae155.png;>
 |
   | simple_label_reversed | https://user-images.githubusercontent.com/54231417/152872356-a8c3447e-2b2d-41be-8938-a716d6226b03.png;>
 | https://user-images.githubusercontent.com/54231417/152867702-b61ef838-bd1b-4160-8cae-a514ce029cab.png;>
 |
   | complex_label | https://user-images.githubusercontent.com/54231417/152872508-e78eebf0-c0ca-4960-ba2b-a1454c9b4aca.png;>
 | https://user-images.githubusercontent.com/54231417/152865925-5b25d205-591b-4065-9127-eb15aa44d360.png;>
 |
   
   Tested on dags: 
   
[test_label_refactoring.py.zip](https://github.com/avkirilishin/airflow/files/8018631/test_label_refactoring.py.zip)
   
   
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a change in pull request #21403: Update `template_fields_renderers` using new SQL lexers to "sql" renderer

[GitBox]

potiuk commented on a change in pull request #21403:
URL: https://github.com/apache/airflow/pull/21403#discussion_r801100375



##
File path: airflow/providers/amazon/aws/operators/redshift_sql.py
##
@@ -44,7 +44,8 @@ class RedshiftSQLOperator(BaseOperator):
 
 template_fields: Sequence[str] = ('sql',)
 template_ext: Sequence[str] = ('.sql',)
-template_fields_renderers = {"sql": "postgresql"}
+# TODO: Update ``sql`` renderer to "postgresql" when the lexer is 
available in Airflow 2.3.0.
+template_fields_renderers = {"sql": "sql"}

Review comment:
   something like that should work :)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a change in pull request #21403: Update `template_fields_renderers` using new SQL lexers to "sql" renderer

[GitBox]

potiuk commented on a change in pull request #21403:
URL: https://github.com/apache/airflow/pull/21403#discussion_r801100269



##
File path: airflow/providers/amazon/aws/operators/redshift_sql.py
##
@@ -44,7 +44,8 @@ class RedshiftSQLOperator(BaseOperator):
 
 template_fields: Sequence[str] = ('sql',)
 template_ext: Sequence[str] = ('.sql',)
-template_fields_renderers = {"sql": "postgresql"}
+# TODO: Update ``sql`` renderer to "postgresql" when the lexer is 
available in Airflow 2.3.0.
+template_fields_renderers = {"sql": "sql"}

Review comment:
   ```suggestion
   template_fields_renderers = {"sql": "postgresql"} if  'postgresql' in 
get_attr_renderer() else {"sql": "sql"}
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] josh-fell commented on a change in pull request #21403: Update `template_fields_renderers` using new SQL lexers to "sql" renderer

[GitBox]

josh-fell commented on a change in pull request #21403:
URL: https://github.com/apache/airflow/pull/21403#discussion_r801100278



##
File path: airflow/providers/amazon/aws/operators/redshift_sql.py
##
@@ -44,7 +44,8 @@ class RedshiftSQLOperator(BaseOperator):
 
 template_fields: Sequence[str] = ('sql',)
 template_ext: Sequence[str] = ('.sql',)
-template_fields_renderers = {"sql": "postgresql"}
+# TODO: Update ``sql`` renderer to "postgresql" when the lexer is 
available in Airflow 2.3.0.

Review comment:
   Ah you're right. I'll update.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a change in pull request #21403: Update `template_fields_renderers` using new SQL lexers to "sql" renderer

[GitBox]

potiuk commented on a change in pull request #21403:
URL: https://github.com/apache/airflow/pull/21403#discussion_r801098384



##
File path: airflow/providers/amazon/aws/operators/redshift_sql.py
##
@@ -44,7 +44,8 @@ class RedshiftSQLOperator(BaseOperator):
 
 template_fields: Sequence[str] = ('sql',)
 template_ext: Sequence[str] = ('.sql',)
-template_fields_renderers = {"sql": "postgresql"}
+# TODO: Update ``sql`` renderer to "postgresql" when the lexer is 
available in Airflow 2.3.0.

Review comment:
   I  think it would be better to dynamically check the renderers available 
in this case. Otherwise, even if we release 2.3.0, we will not be able to 
change it as the providers are 2.1.0+ 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] pingzh commented on a change in pull request #21332: Add option to compress dag data

[GitBox]

pingzh commented on a change in pull request #21332:
URL: https://github.com/apache/airflow/pull/21332#discussion_r801092172



##
File path: airflow/models/serialized_dag.py
##
@@ -20,11 +20,12 @@
 
 import hashlib
 import logging
+import zlib

Review comment:
   although `bz2` can achieve smaller size of compress data, but we noticed 
that the `zlib` is faster than `bz2` when  compressing and decompressing data 
with the same level of compression (zlib with default level 6, also set 
compresslevel as 6 for bz2).
   
   test with python3.7.9, `raw json data size`: `514MB`, `bz2` compressed: 
`14M` and `zlib` compressed: `44M`.
   
   ```
   ping_zhang@i-001e445e8c0ed963f:~$ python test_bz2.py
   bz2 took 89.2322297607
   bz2 decompress took 15.989518880844116
   
   
   ping_zhang@i-001e445e8c0ed963f:~$ python test_zlib.py
   zlib compress took 8.829093933105469
   zlib decompress took 1.1581130027770996
   
   ```
   
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] josh-fell commented on pull request #21403: Update `template_fields_renderers` using new SQL lexers to "sql" renderer

[GitBox]

josh-fell commented on pull request #21403:
URL: https://github.com/apache/airflow/pull/21403#issuecomment-1031968398


   CC @potiuk 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] josh-fell opened a new pull request #21403: Update `template_fields_renderers` using new SQL lexers to "sql" renderer

[GitBox]

josh-fell opened a new pull request #21403:
URL: https://github.com/apache/airflow/pull/21403


   Related: #21237 #21348
   
   New SQL lexers were added as part of #21237, however, these new lexers won't 
be available for use as `template_fields_renderers` in operators until Airflow 
2.3. This PR updates operators using these lexers to the "sql" renderer for now.
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kurtqq edited a comment on issue #21252: Distinguish between marking failure/success by user and by task

[GitBox]

kurtqq edited a comment on issue #21252:
URL: https://github.com/apache/airflow/issues/21252#issuecomment-1031958805


   > There's an enum of task states 
[here](https://github.com/apache/airflow/blob/1d170f899bcc87110e55192517270ec89d511ca8/airflow/utils/state.py#L26),
 which we could add to, but that would imply changing the marking mechanism to 
e.g. `MARKED_SUCCESS` or `MARKED_FAILED` instead of `SUCCESS` or `FAILED`. 
Which would then break anything that depends on the task instances being one of 
the latter two states. That sounds like a lot bigger of a change than I am 
ready for...
   
   The concept of adding explicit status per each possible action is dangerous. 
When start this you will never hear the end of it.
   To answer of the question who did something should be stored in logs. If 
needed it should be displayed from the logs.
   When you know who did it you also get the answer about what he did because 
if a user marked task as failed clearly this is done manually so you get this 
way more information without adding more statuses.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kurtqq commented on issue #21252: Distinguish between marking failure/success by user and by task

[GitBox]

kurtqq commented on issue #21252:
URL: https://github.com/apache/airflow/issues/21252#issuecomment-1031958805


   > There's an enum of task states 
[here](https://github.com/apache/airflow/blob/1d170f899bcc87110e55192517270ec89d511ca8/airflow/utils/state.py#L26),
 which we could add to, but that would imply changing the marking mechanism to 
e.g. `MARKED_SUCCESS` or `MARKED_FAILED` instead of `SUCCESS` or `FAILED`. 
Which would then break anything that depends on the task instances being one of 
the latter two states. That sounds like a lot bigger of a change than I am 
ready for...
   
   The concept of adding explicit status per each possible action is dangerous. 
When start this you will never hear the end of it.
   To answer of the question who did something should be stored in logs. If 
needed it should be displayed from the logs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch constraints-main updated: Updating constraints. Build id:1808732466

[github-bot]

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch constraints-main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/constraints-main by this push:
 new e029fd0  Updating constraints. Build id:1808732466
e029fd0 is described below

commit e029fd09d85bed3f13afc330dc3f838a5d9aef8e
Author: Automated GitHub Actions commit 
AuthorDate: Mon Feb 7 21:21:15 2022 +

Updating constraints. Build id:1808732466

This update in constraints is automatically committed by the CI 
'constraints-push' step based on
HEAD of 'refs/heads/main' in 'apache/airflow'
with commit sha 471e368eacbcae1eedf9b7e1cb4290c385396ea9.

All tests passed in this build so we determined we can push the updated 
constraints.

See 
https://github.com/apache/airflow/blob/main/README.md#installing-from-pypi for 
details.
---
 constraints-3.7.txt  | 2 +-
 constraints-3.8.txt  | 2 +-
 constraints-3.9.txt  | 2 +-
 constraints-no-providers-3.7.txt | 2 +-
 constraints-no-providers-3.8.txt | 2 +-
 constraints-no-providers-3.9.txt | 2 +-
 constraints-source-providers-3.7.txt | 8 
 constraints-source-providers-3.8.txt | 8 
 constraints-source-providers-3.9.txt | 8 
 9 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/constraints-3.7.txt b/constraints-3.7.txt
index 9a213eb..5caf4fa 100644
--- a/constraints-3.7.txt
+++ b/constraints-3.7.txt
@@ -1,5 +1,5 @@
 #
-# This constraints file was automatically generated on 2022-02-07T19:00:48Z
+# This constraints file was automatically generated on 2022-02-07T21:17:57Z
 # via "eager-upgrade" mechanism of PIP. For the "main" branch of Airflow.
 # This variant of constraints install uses the HEAD of the branch version for 
'apache-airflow' but installs
 # the providers from PIP-released packages at the moment of the constraint 
generation.
diff --git a/constraints-3.8.txt b/constraints-3.8.txt
index b0cfc62..effa4aa 100644
--- a/constraints-3.8.txt
+++ b/constraints-3.8.txt
@@ -1,5 +1,5 @@
 #
-# This constraints file was automatically generated on 2022-02-07T19:00:45Z
+# This constraints file was automatically generated on 2022-02-07T21:17:51Z
 # via "eager-upgrade" mechanism of PIP. For the "main" branch of Airflow.
 # This variant of constraints install uses the HEAD of the branch version for 
'apache-airflow' but installs
 # the providers from PIP-released packages at the moment of the constraint 
generation.
diff --git a/constraints-3.9.txt b/constraints-3.9.txt
index 89cf512..74d4157 100644
--- a/constraints-3.9.txt
+++ b/constraints-3.9.txt
@@ -1,5 +1,5 @@
 #
-# This constraints file was automatically generated on 2022-02-07T19:00:45Z
+# This constraints file was automatically generated on 2022-02-07T21:17:49Z
 # via "eager-upgrade" mechanism of PIP. For the "main" branch of Airflow.
 # This variant of constraints install uses the HEAD of the branch version for 
'apache-airflow' but installs
 # the providers from PIP-released packages at the moment of the constraint 
generation.
diff --git a/constraints-no-providers-3.7.txt b/constraints-no-providers-3.7.txt
index 97ce45a..b07dd60 100644
--- a/constraints-no-providers-3.7.txt
+++ b/constraints-no-providers-3.7.txt
@@ -1,5 +1,5 @@
 #
-# This constraints file was automatically generated on 2022-02-07T19:04:13Z
+# This constraints file was automatically generated on 2022-02-07T21:21:08Z
 # via "eager-upgrade" mechanism of PIP. For the "main" branch of Airflow.
 # This variant of constraints install just the 'bare' 'apache-airflow' package 
build from the HEAD of
 # the branch, without installing any of the providers.
diff --git a/constraints-no-providers-3.8.txt b/constraints-no-providers-3.8.txt
index 98a1e96..9fcc84b 100644
--- a/constraints-no-providers-3.8.txt
+++ b/constraints-no-providers-3.8.txt
@@ -1,5 +1,5 @@
 #
-# This constraints file was automatically generated on 2022-02-07T19:04:13Z
+# This constraints file was automatically generated on 2022-02-07T21:21:06Z
 # via "eager-upgrade" mechanism of PIP. For the "main" branch of Airflow.
 # This variant of constraints install just the 'bare' 'apache-airflow' package 
build from the HEAD of
 # the branch, without installing any of the providers.
diff --git a/constraints-no-providers-3.9.txt b/constraints-no-providers-3.9.txt
index bf47353..fc9c8f8 100644
--- a/constraints-no-providers-3.9.txt
+++ b/constraints-no-providers-3.9.txt
@@ -1,5 +1,5 @@
 #
-# This constraints file was automatically generated on 2022-02-07T19:04:11Z
+# This constraints file was automatically generated on 2022-02-07T21:21:05Z
 # via "eager-upgrade" mechanism of PIP. For the "main" branch of Airflow.
 # This variant of constraints install just the 'bare' 'apache-airflow' package 
build from the HEAD of
 # the branch, without installing any of the providers.
diff --git 

[GitHub] [airflow] jedcunningham commented on a change in pull request #21401: added envFrom statement to the flower deployment

[GitBox]

jedcunningham commented on a change in pull request #21401:
URL: https://github.com/apache/airflow/pull/21401#discussion_r801061676



##
File path: chart/templates/flower/flower-deployment.yaml
##
@@ -128,6 +128,8 @@ spec:
   name: {{ template "flower_secret" . }}
   key: basicAuth
 {{- end }}
+  envFrom:
+  {{- include "custom_airflow_environment_from" . | default "\n  []" | 
indent 10 }}

Review comment:
   This can't go in between stuff in `env`. Either move it above or below 
the whole section.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #20733: Add Audit Log View to Dag View

[GitBox]

github-actions[bot] commented on pull request #20733:
URL: https://github.com/apache/airflow/pull/20733#issuecomment-1031925818


   The PR most likely needs to run full matrix of tests because it modifies 
parts of the core of Airflow. However, committers might decide to merge it 
quickly and take the risk. If they don't merge it quickly - please rebase it to 
the latest main at your convenience, or amend the last commit of the PR, and 
push it with --force-with-lease.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] raphaelauv commented on a change in pull request #21391: Optionally raise an error if source file does not exist in GCSToGCSOperator

[GitBox]

raphaelauv commented on a change in pull request #21391:
URL: https://github.com/apache/airflow/pull/21391#discussion_r801042926



##
File path: airflow/providers/google/cloud/transfers/gcs_to_gcs.py
##
@@ -313,6 +318,14 @@ def _copy_source_without_wildcard(self, hook, prefix):
 self._copy_single_object(
 hook=hook, source_object=prefix, 
destination_object=self.destination_object
 )
+else:
+msg = (
+f'{prefix} does not exist in bucket {self.source_bucket}'
+)
+self.log.warning(msg)

Review comment:
   maybe move the log inside the if, so current users do not see logs with 
WARNING level




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] raphaelauv commented on a change in pull request #21391: Optionally raise an error if source file does not exist in GCSToGCSOperator

[GitBox]

raphaelauv commented on a change in pull request #21391:
URL: https://github.com/apache/airflow/pull/21391#discussion_r801042926



##
File path: airflow/providers/google/cloud/transfers/gcs_to_gcs.py
##
@@ -313,6 +318,14 @@ def _copy_source_without_wildcard(self, hook, prefix):
 self._copy_single_object(
 hook=hook, source_object=prefix, 
destination_object=self.destination_object
 )
+else:
+msg = (
+f'{prefix} does not exist in bucket {self.source_bucket}'
+)
+self.log.warning(msg)

Review comment:
   maybe move the log inside the if, so current users do not log with 
WARNING level




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #21402: Fix type annotations re task decorator map

[GitBox]

github-actions[bot] commented on pull request #21402:
URL: https://github.com/apache/airflow/pull/21402#issuecomment-1031898327


   The PR most likely needs to run full matrix of tests because it modifies 
parts of the core of Airflow. However, committers might decide to merge it 
quickly and take the risk. If they don't merge it quickly - please rebase it to 
the latest main at your convenience, or amend the last commit of the PR, and 
push it with --force-with-lease.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated: [Oracle] Oracle Hook - automatically set current_schema when defined in Connection (#19084)

[potiuk]

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
 new 471e368  [Oracle] Oracle Hook - automatically set current_schema when 
defined in Connection (#19084)
471e368 is described below

commit 471e368eacbcae1eedf9b7e1cb4290c385396ea9
Author: mehmax <84371809+meh...@users.noreply.github.com>
AuthorDate: Mon Feb 7 21:37:51 2022 +0100

[Oracle] Oracle Hook - automatically set current_schema when defined in 
Connection (#19084)
---
 airflow/providers/oracle/hooks/oracle.py| 20 ++--
 tests/providers/oracle/hooks/test_oracle.py |  4 
 2 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/airflow/providers/oracle/hooks/oracle.py 
b/airflow/providers/oracle/hooks/oracle.py
index 95b5fdc..9ebed94 100644
--- a/airflow/providers/oracle/hooks/oracle.py
+++ b/airflow/providers/oracle/hooks/oracle.py
@@ -16,6 +16,7 @@
 # specific language governing permissions and limitations
 # under the License.
 
+import warnings
 from datetime import datetime
 from typing import Dict, List, Optional, Union
 
@@ -87,6 +88,7 @@ class OracleHook(DbApiHook):
 conn_config = {'user': conn.login, 'password': conn.password}
 sid = conn.extra_dejson.get('sid')
 mod = conn.extra_dejson.get('module')
+schema = conn.schema
 
 service_name = conn.extra_dejson.get('service_name')
 port = conn.port if conn.port else 1521
@@ -100,8 +102,16 @@ class OracleHook(DbApiHook):
 dsn = conn.host
 if conn.port is not None:
 dsn += ":" + str(conn.port)
-if service_name or conn.schema:
-dsn += "/" + (service_name or conn.schema)
+if service_name:
+dsn += "/" + service_name
+elif conn.schema:
+warnings.warn(
+"""Using conn.schema to pass the Oracle Service Name 
is deprecated.
+Please use conn.extra.service_name instead.""",
+DeprecationWarning,
+stacklevel=2,
+)
+dsn += "/" + conn.schema
 conn_config['dsn'] = dsn
 
 if 'encoding' in conn.extra_dejson:
@@ -146,6 +156,12 @@ class OracleHook(DbApiHook):
 if mod is not None:
 conn.module = mod
 
+# if Connection.schema is defined, set schema after connecting 
successfully
+# cannot be part of conn_config
+# 
https://cx-oracle.readthedocs.io/en/latest/api_manual/connection.html?highlight=schema#Connection.current_schema
+if schema is not None:
+conn.current_schema = schema
+
 return conn
 
 def insert_rows(
diff --git a/tests/providers/oracle/hooks/test_oracle.py 
b/tests/providers/oracle/hooks/test_oracle.py
index 3eae248..9217fec 100644
--- a/tests/providers/oracle/hooks/test_oracle.py
+++ b/tests/providers/oracle/hooks/test_oracle.py
@@ -176,6 +176,10 @@ class TestOracleHookConn(unittest.TestCase):
 assert args == ()
 assert kwargs['purity'] == purity.get(pur)
 
+@mock.patch('airflow.providers.oracle.hooks.oracle.cx_Oracle.connect')
+def test_set_current_schema(self, mock_connect):
+assert self.db_hook.get_conn().current_schema == self.connection.schema
+
 
 @unittest.skipIf(cx_Oracle is None, 'cx_Oracle package not present')
 class TestOracleHook(unittest.TestCase):

[GitHub] [airflow] ashb commented on a change in pull request #21402: Fix type annotations re task decorator map

[GitBox]

ashb commented on a change in pull request #21402:
URL: https://github.com/apache/airflow/pull/21402#discussion_r801038280



##
File path: airflow/decorators/__init__.pyi
##
@@ -226,4 +226,5 @@ class TaskDecoratorCollection:
 """
 # [END decorator_signature]
 
-task: TaskDecoratorCollection
+T = TypeVar("T", bound=TaskDecoratorCollection)
+task: Callable[[Function], T]

Review comment:
   How is this different from this?
   
   ```python
   
   task: Callable[[Function], TaskDecoratorCollection]
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk merged pull request #19084: [Oracle] Oracle Hook - automatically set current_schema when defined in Connection

[GitBox]

potiuk merged pull request #19084:
URL: https://github.com/apache/airflow/pull/19084


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #19084: [Oracle] Oracle Hook - automatically set current_schema when defined in Connection

[GitBox]

boring-cyborg[bot] commented on pull request #19084:
URL: https://github.com/apache/airflow/pull/19084#issuecomment-1031897507


   Awesome work, congrats on your first merged pull request!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk closed issue #18664: [Oracle] Oracle Hook - make it possible to define a schema in the connection parameters

[GitBox]

potiuk closed issue #18664:
URL: https://github.com/apache/airflow/issues/18664


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #19084: [Oracle] Oracle Hook - automatically set current_schema when defined in Connection

[GitBox]

potiuk commented on pull request #19084:
URL: https://github.com/apache/airflow/pull/19084#issuecomment-1031897290


   Just a docker-compose failure which we will fix shorlty.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #19084: [Oracle] Oracle Hook - automatically set current_schema when defined in Connection

[GitBox]

github-actions[bot] commented on pull request #19084:
URL: https://github.com/apache/airflow/pull/19084#issuecomment-1031897211


   The PR is likely OK to be merged with just subset of tests for default 
Python and Database versions without running the full matrix of tests, because 
it does not modify the core of Airflow. If the committers decide that the full 
tests matrix is needed, they will add the label 'full tests needed'. Then you 
should rebase to the latest main or amend the last commit of the PR, and push 
it with --force-with-lease.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ashb commented on a change in pull request #21342: Add 'Show record' option for variables

[GitBox]

ashb commented on a change in pull request #21342:
URL: https://github.com/apache/airflow/pull/21342#discussion_r801037088



##
File path: airflow/www/widgets.py
##
@@ -72,3 +72,8 @@ class BS3TextAreaROWidget(BS3TextAreaFieldWidget):
 def __call__(self, field, **kwargs):
 kwargs['readonly'] = 'true'
 return super().__call__(field, **kwargs)
+
+class AirflowModelShowWidget(RenderTemplateWidget):
+"""Airflow model show widget"""
+
+template = 'airflow/model_show.html'

Review comment:
   This template is specific to just Variables show then? If so we should 
name it as such.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #21397: Stale DAG Deactivation in DAG Processor is extremely hard on the database in environments with many DAGs

[GitBox]

potiuk commented on issue #21397:
URL: https://github.com/apache/airflow/issues/21397#issuecomment-1031891355


   > Oh thanks Mysql.
   
   A lot.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #21357: Fix incorrect datetime details (DagRun views)

[GitBox]

github-actions[bot] commented on pull request #21357:
URL: https://github.com/apache/airflow/pull/21357#issuecomment-1031891302


   The PR is likely OK to be merged with just subset of tests for default 
Python and Database versions without running the full matrix of tests, because 
it does not modify the core of Airflow. If the committers decide that the full 
tests matrix is needed, they will add the label 'full tests needed'. Then you 
should rebase to the latest main or amend the last commit of the PR, and push 
it with --force-with-lease.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] riccardo-salamanna opened a new pull request #21401: added envFrom statement to the flower deployment

[GitBox]

riccardo-salamanna opened a new pull request #21401:
URL: https://github.com/apache/airflow/pull/21401


   Added `envFrom` statement to the flower deployement in the helm chart
   ---
   The flower deployement is missing the envFrom statement. This causes that if 
you use the `extraEnvFrom` in the value files,  they are not deployed on all 
the pods.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #21401: added envFrom statement to the flower deployment

[GitBox]

boring-cyborg[bot] commented on pull request #21401:
URL: https://github.com/apache/airflow/pull/21401#issuecomment-1031883639


   Congratulations on your first Pull Request and welcome to the Apache Airflow 
community! If you have any issues or are unsure about any anything please check 
our Contribution Guide 
(https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
   Here are some useful points:
   - Pay attention to the quality of your code (flake8, mypy and type 
annotations). Our [pre-commits]( 
https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks)
 will help you with that.
   - In case of a new feature add useful documentation (in docstrings or in 
`docs/` directory). Adding a new operator? Check this short 
[guide](https://github.com/apache/airflow/blob/main/docs/apache-airflow/howto/custom-operator.rst)
 Consider adding an example DAG that shows how users should use it.
   - Consider using [Breeze 
environment](https://github.com/apache/airflow/blob/main/BREEZE.rst) for 
testing locally, it’s a heavy docker but it ships with a working Airflow and a 
lot of integrations.
   - Be patient and persistent. It might take some time to get a review or get 
the final approval from Committers.
   - Please follow [ASF Code of 
Conduct](https://www.apache.org/foundation/policies/conduct) for all 
communication including (but not limited to) comments on Pull Requests, Mailing 
list and Slack.
   - Be sure to read the [Airflow Coding style]( 
https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#coding-style-and-best-practices).
   Apache Airflow is a community-driven project and together we are making it 
better .
   In case of doubts contact the developers at:
   Mailing List: d...@airflow.apache.org
   Slack: https://s.apache.org/airflow-slack
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a change in pull request #20642: Added Hook for Amazon RDS

[GitBox]

potiuk commented on a change in pull request #20642:
URL: https://github.com/apache/airflow/pull/20642#discussion_r795033020



##
File path: setup.py
##
@@ -194,6 +194,7 @@ def write_version(filename: str = os.path.join(*[my_dir, 
"airflow", "git_version
 'redshift_connector~=2.0.888',
 'sqlalchemy_redshift~=0.8.6',
 pandas_requirement,
+'boto3_stubs[rds]~=1.20.32',

Review comment:
   Yep. Lets leave it unpinned. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] NilsJPWerner commented on issue #21336: Override the dag run_id from within the ui

[GitBox]

NilsJPWerner commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1031881493


   I understand now. I got the indication from @ashb's tag that a PR for this 
issue would likely be accepted so I will look into what it would take to make 
the change. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #21336: Override the dag run_id from within the ui

[GitBox]

potiuk commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1031879570


   > In the 
[CONTRIBUTING.rst](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#submit-feedback)
 there is a section about creating feature requests, and the issue page has a 
dedicated button for feature requests, so I believed that posting a feature 
request was the way to go about these things.
   
   I did not say it's WRONG to add feature request, I just said you do not need 
it if you want open a PR :) . It was not a complaint - just a statement :D .


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] NilsJPWerner edited a comment on issue #21336: Override the dag run_id from within the ui

[GitBox]

NilsJPWerner edited a comment on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1031876060


   > Or pick someone's interest and convince them to implement it. Or mark it 
as a good first issue and hope someone will pick it as their first issue if 
this is an easy fix.
   
   That was my goal when I opened the feature request. I could potentially 
attempt the PR myself, but I am not familiar with the code base so would 
probably need a bit of direction. I definitely wouldn't want to create a pr if 
I didn't have an indication of whether it would be acceptable to the community, 
so an issue seems like a good way to gauge that.
   
   In the 
[CONTRIBUTING.rst](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#submit-feedback
 ) there is a section about creating feature requests, and the issue page has a 
dedicated button for feature requests, so I believed that posting a feature 
request was the way to go about these things.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] NilsJPWerner commented on issue #21336: Override the dag run_id from within the ui

[GitBox]

NilsJPWerner commented on issue #21336:
URL: https://github.com/apache/airflow/issues/21336#issuecomment-1031876060


   > Or pick someone's interest and convince them to implement it. Or mark it 
as a good first issue and hope someone will pick it as their first issue if 
this is an easy fix.
   
   That was my goal when I opened the feature request. I could potentially 
attempt the PR myself, but I am not familiar with the code base so would 
probably need a bit of direction.
   
   In the 
[CONTRIBUTING.rst](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#submit-feedback
 ) there is a section about creating feature requests, and the issue page has a 
dedicated button for feature requests, so I believed that posting a feature 
request was the way to go about these things.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] blag commented on pull request #21167: Add a session backend to store session data in the database

[GitBox]

blag commented on pull request #21167:
URL: https://github.com/apache/airflow/pull/21167#issuecomment-1031870760


   @dstandish That's a good call, but it's looking less and less likely that 
our changes are going to be palatable to the maintainer of Flask-Session. If 
the changes we need won't ever be merged into Flask-Session, then our modified 
vendored code just becomes...our code, with an additional copyright line/header.
   
   A "middle path" of this would be to start maintaining our own fork of 
Flask-Session, merge in the changes that we need, maybe merge in a few more 
unmerged PRs from Flask-Session, and have our fork become the friendlier (than 
Flask-Session) session package for Flask. But I don't want to sign the company 
up for that responsibility without some consensus and blessing, and I don't 
really want to maintain my own personal fork of Flask-Session specifically for 
Airflow.
   
   I'm happy to do whatever it takes to get this merged, but I will need some 
guidance on how to get there.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kazanzhy opened a new pull request #20642: Added Hook for Amazon RDS

[GitBox]

kazanzhy opened a new pull request #20642:
URL: https://github.com/apache/airflow/pull/20642


   I had two different implementations of `RDSHook`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk closed pull request #20642: Added Hook for Amazon RDS

[GitBox]

potiuk closed pull request #20642:
URL: https://github.com/apache/airflow/pull/20642


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] pingzh commented on a change in pull request #21332: Add option to compress dag data

[GitBox]

pingzh commented on a change in pull request #21332:
URL: https://github.com/apache/airflow/pull/21332#discussion_r801008951



##
File path: tests/models/test_serialized_dag.py
##
@@ -63,60 +72,68 @@ def _write_example_dags(self):
 return example_dags
 
 def test_write_dag(self):
-"""DAGs can be written into database."""
-example_dags = self._write_example_dags()
+"""DAGs can be written into database when no comprehension"""
+with mock.patch(
+'airflow.models.serialized_dag.COMPRESS_SERIALIZED_DAGS', 
self.compress_serialized_dags
+):

Review comment:
   Good call. thanks

##
File path: airflow/config_templates/config.yml
##
@@ -368,6 +368,13 @@
   type: string
   example: ~
   default: "30"
+- name: compress_serialized_dags
+  description: |
+If True, serialized DAGs are compressed before writing to DB

Review comment:
   Good call




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #21400: Docker operator fails

[GitBox]

potiuk commented on issue #21400:
URL: https://github.com/apache/airflow/issues/21400#issuecomment-1031866151


   @ferruzzi @raphaelauv  -> would you have time to take a look at that, or 
should I look first thing tomorrow in the morning :D ? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk opened a new issue #21400: Docker operator fails

[GitBox]

potiuk opened a new issue #21400:
URL: https://github.com/apache/airflow/issues/21400


   ### Apache Airflow version
   
   main (development)
   
   ### What happened
   
   As discussed in #21348
   
   cc: @ferruzzi @raphaelauv 
   
   Seems that main docker operator is buggy even aftter 
https://github.com/apache/airflow/pull/21175 
   
   with `do_xcom_push=False` it fail
   
   ```python
   DockerOperator(
   task_id='docker_op_tester',
   dag=dag,
   api_version='auto',
   docker_url="unix://var/run/docker.sock",
   command='/bin/echo tata',
   image='centos:latest',
   network_mode='bridge',
   mount_tmp_dir=False,
   do_xcom_push=True,
   )
   ```
   fail
   
   with 
   
   ```log
   [2022-02-05, 22:38:18 UTC] {taskinstance.py:1259} INFO - Executing 
 on 2022-02-03 07:00:00+00:00
   [2022-02-05, 22:38:18 UTC] {standard_task_runner.py:52} INFO - Started 
process 325 to run task
   [2022-02-05, 22:38:18 UTC] {standard_task_runner.py:76} INFO - Running: 
['airflow', 'tasks', 'run', 'docker_dag', 'docker_op_tester', 
'scheduled__2022-02-03T07:00:00+00:00', '--job-id', '34', '--raw', '--subdir', 
'DAGS_FOLDER/docker_dag.py', '--cfg-path', '/tmp/tmpqd592sdh', '--error-file', 
'/tmp/tmp3e4mxwk4']
   [2022-02-05, 22:38:18 UTC] {standard_task_runner.py:77} INFO - Job 34: 
Subtask docker_op_tester
   [2022-02-05, 22:38:18 UTC] {logging_mixin.py:109} INFO - Running 
 on host 1b2f3575c860
   [2022-02-05, 22:38:18 UTC] {taskinstance.py:1424} INFO - Exporting the 
following env vars:
   AIRFLOW_CTX_DAG_EMAIL=airf...@example.com
   AIRFLOW_CTX_DAG_OWNER=airflow
   AIRFLOW_CTX_DAG_ID=docker_dag
   AIRFLOW_CTX_TASK_ID=docker_op_tester
   AIRFLOW_CTX_EXECUTION_DATE=2022-02-03T07:00:00+00:00
   AIRFLOW_CTX_DAG_RUN_ID=scheduled__2022-02-03T07:00:00+00:00
   [2022-02-05, 22:38:18 UTC] {docker.py:227} INFO - Starting docker container 
from image centos:latest
   [2022-02-05, 22:38:19 UTC] {docker.py:289} INFO - tata
   [2022-02-05, 22:38:19 UTC] {xcom.py:333} ERROR - Could not serialize the 
XCom value into JSON. If you are using pickle instead of JSON for XCom, then 
you need to enable pickle support for XCom in your airflow config.
   [2022-02-05, 22:38:19 UTC] {taskinstance.py:1700} ERROR - Task failed with 
exception
   Traceback (most recent call last):
 File 
"/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 
1329, in _run_raw_task
   self._execute_task_with_callbacks(context)
 File 
"/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 
1455, in _execute_task_with_callbacks
   result = self._execute_task(context, self.task)
 File 
"/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 
1514, in _execute_task
   self.xcom_push(key=XCOM_RETURN_KEY, value=result)
 File "/usr/local/lib/python3.9/site-packages/airflow/utils/session.py", 
line 70, in wrapper
   return func(*args, session=session, **kwargs)
 File 
"/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 
2135, in xcom_push
   XCom.set(
 File "/usr/local/lib/python3.9/site-packages/airflow/utils/session.py", 
line 67, in wrapper
   return func(*args, **kwargs)
 File "/usr/local/lib/python3.9/site-packages/airflow/models/xcom.py", line 
100, in set
   value = XCom.serialize_value(value)
 File "/usr/local/lib/python3.9/site-packages/airflow/models/xcom.py", line 
331, in serialize_value
   return json.dumps(value).encode('UTF-8')
 File "/usr/local/lib/python3.9/json/__init__.py", line 231, in dumps
   return _default_encoder.encode(obj)
 File "/usr/local/lib/python3.9/json/encoder.py", line 199, in encode
   chunks = self.iterencode(o, _one_shot=True)
 File "/usr/local/lib/python3.9/json/encoder.py", line 257, in iterencode
   return _iterencode(o, 0)
 File "/usr/local/lib/python3.9/json/encoder.py", line 179, in default
   raise TypeError(f'Object of type {o.__class__.__name__} '
   TypeError: Object of type CancellableStream is not JSON serializable
   [2022-02-05, 22:38:19 UTC] {taskinstance.py:1267} INFO - Marking task as 
FAILED. dag_id=docker_dag, task_id=docker_op_tester, 
execution_date=20220203T07, start_date=20220205T223818, 
end_date=20220205T223819
   [2022-02-05, 22:38:19 UTC] {standard_task_runner.py:89} ERROR - Failed to 
execute job 34 for task docker_op_tester
   Traceback (most recent call last):
 File 
"/usr/local/lib/python3.9/site-packages/airflow/task/task_runner/standard_task_runner.py",
 line 85, in _start_by_fork
   args.func(args, dag=self.dag)
 File "/usr/local/lib/python3.9/site-packages/airflow/cli/cli_parser.py", 
line 48, in command
   return func(*args, **kwargs)
 File "/usr/local/lib/python3.9/site-packages/airflow/utils/cli.py", line 
92, in wrapper
   return f(*args, **kwargs)
 File 
"/usr/local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", 

[GitHub] [airflow] potiuk commented on pull request #21393: Add how-to guide for WebHDFS operators

[GitBox]

potiuk commented on pull request #21393:
URL: https://github.com/apache/airflow/pull/21393#issuecomment-1031854918


   That's the whole error: it expects that providers.yaml links to the 
webhdfs.rst and hdfs.rst rather than to index.rst
   
   ```
   Validate providers.yaml 
filesFailed
   - hook id: provider-yamls
   - exit code: 1
   
   Checking integration duplicates
   Checking completeness of list of {sensors, hooks, operators}
-- {sensors, hooks, operators} - Expected modules(Left): Current 
Modules(Right)
   Checking for duplicates in list of {sensors, hooks, operators}
   Checking completeness of list of transfers
-- Expected transfers modules(Left): Current transfers Modules(Right)
   Checking for duplicates in list of transfers
   Checking connection classes belong to package
-- Checking providers: present in code(left), mentioned in 
/home/runner/work/airflow/airflow/.github/ISSUE_TEMPLATE/airflow_providers_bug_report.yml
 (right)
   Checking doc files
-- Checking document urls: expected(left), current(right)
   -- Items in the left set but not the right:
  '/docs/apache-airflow-providers-apache-hdfs/operators/hdfs.rst'
  '/docs/apache-airflow-providers-apache-hdfs/operators/webhdfs.rst'
   -- Items in the right set but not the left:
  '/docs/apache-airflow-providers-apache-hdfs/operators/index.rst'
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk edited a comment on issue #21392: .airflowignore does not use 'glob' as .gitignore for example

[GitBox]

potiuk edited a comment on issue #21392:
URL: https://github.com/apache/airflow/issues/21392#issuecomment-1031851652


   > Yeah, if we wanted to change the format of this file to more closely match 
what people expect (which I agree is confusing) -- i.e. `.gitignore` we'd 
either have to change the filename, or add a `# airflow-ignore-v2` comment as 
the first line or something.
   
   The comment would be also not good, because people would not know they have 
to add it :( . Same story as now.
   
   I think we are a bit stuck with regexp until ~2.3.0~ 3.0.0 (then we could 
change it and add migration check with smart detection and even conversion 
suggestions  - that would be kinda doable I think.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk edited a comment on issue #21392: .airflowignore does not use 'glob' as .gitignore for example

[GitBox]

potiuk edited a comment on issue #21392:
URL: https://github.com/apache/airflow/issues/21392#issuecomment-1031851652


   > Yeah, if we wanted to change the format of this file to more closely match 
what people expect (which I agree is confusing) -- i.e. `.gitignore` we'd 
either have to change the filename, or add a `# airflow-ignore-v2` comment as 
the first line or something.
   
   The comment would be also not good, because people would not know they have 
to add it :( . Same story as now.
   
   I think we are a bit stuck with regexp until 3.0.0 (then we could change it 
and add migration check with smart detection and even conversion suggestions  - 
that would be kinda doable I think.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #21392: .airflowignore does not use 'glob' as .gitignore for example

[GitBox]

potiuk commented on issue #21392:
URL: https://github.com/apache/airflow/issues/21392#issuecomment-1031851652


   > Yeah, if we wanted to change the format of this file to more closely match 
what people expect (which I agree is confusing) -- i.e. `.gitignore` we'd 
either have to change the filename, or add a `# airflow-ignore-v2` comment as 
the first line or something.
   
   The comment would be also not good, because people would not know they have 
to add it :( . Same story as now.
   
   I think we are a bit stuck with regexp until 2.3.0 (then we could change it 
and add migration check with smart detection and even conversion suggestions  - 
that would be kinda doable I think.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Melodie97 commented on a change in pull request #21346: Update docs/readme.rst

[GitBox]

Melodie97 commented on a change in pull request #21346:
URL: https://github.com/apache/airflow/pull/21346#discussion_r801001745



##
File path: docs/README.rst
##
@@ -74,7 +74,24 @@ Once you have built the documentation run the following 
command from the root di
 
 docs/start_doc_server.sh
 
-Then, view your docs at ``localhost:8000``
+You need to have python installed to run the command, if you get an error 
message -- python: command not found
+It is possible that you don't have python installed or added to PATH.
+Visit link  to install python and add 
to PATH

Review comment:
   ok




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Melodie97 commented on pull request #21393: Add how-to guide for WebHDFS operators

[GitBox]

Melodie97 commented on pull request #21393:
URL: https://github.com/apache/airflow/pull/21393#issuecomment-1031850660


   > Some checks failing
   
   this is what I'm seeing locally in pre-commit, I don't know what that means
   
   
![airdoc](https://user-images.githubusercontent.com/78446940/152860291-a2f326f5-283b-4858-936d-8f2bc69e055e.PNG)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] bbovenzi commented on a change in pull request #20733: Add Audit Log View to Dag View

[GitBox]

bbovenzi commented on a change in pull request #20733:
URL: https://github.com/apache/airflow/pull/20733#discussion_r801000678



##
File path: airflow/config_templates/default_airflow.cfg
##
@@ -670,6 +670,17 @@ auto_refresh_interval = 3
 # Boolean for displaying warning for publicly viewable deployment
 warn_deployment_exposure = True
 
+# Comma separated string of view events to exclude from dag audit view.
+# All other events will be added minus the ones passed here.
+# The audit logs in the db will not be affected by this parameter.
+audit_view_excluded_events = 
gantt,landing_times,tries,duration,calendar,graph,tree

Review comment:
   ```suggestion
   audit_view_excluded_events = 
gantt,landing_times,tries,duration,calendar,graph,tree,tree_data
   ```

##
File path: airflow/config_templates/config.yml
##
@@ -1334,6 +1334,24 @@
   type: boolean
   example: ~
   default: "True"
+- name: audit_view_excluded_events
+  description: |
+Comma separated string of view events to exclude from dag audit view.
+All other events will be added minus the ones passed here.
+The audit logs in the db will not be affected by this parameter.
+  version_added: 2.3.0
+  type: string
+  example: ~
+  default: "gantt,landing_times,tries,duration,calendar,graph,tree"

Review comment:
   ```suggestion
 default: 
"gantt,landing_times,tries,duration,calendar,graph,tree,tree_data"
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] dstandish commented on a change in pull request #21398: Add pre-commit check for docstring param types

[GitBox]

dstandish commented on a change in pull request #21398:
URL: https://github.com/apache/airflow/pull/21398#discussion_r800994542



##
File path: scripts/ci/pre_commit/pre_commit_docstring_param_type.py
##
@@ -0,0 +1,64 @@
+#!/usr/bin/env python3
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+import re
+import sys
+from pathlib import Path
+
+from rich.console import Console
+
+if __name__ not in ("__main__", "__mp_main__"):
+raise SystemExit(
+"This file is intended to be executed as an executable program. You 
cannot use it as a module."
+f"To run this script, run the ./{__file__} command [FILE] ..."
+)
+
+
+console = Console(color_system="standard", width=200)
+
+
+def check_file(the_file: Path) -> list:
+"""Returns number of wrong checkout instructions in the workflow file"""

Review comment:
   thanks




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk closed issue #21348: Status of testing Providers that were prepared on February 05, 2022

[GitBox]

potiuk closed issue #21348:
URL: https://github.com/apache/airflow/issues/21348


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ashb commented on pull request #21342: Add 'Show record' option for variables

[GitBox]

ashb commented on pull request #21342:
URL: https://github.com/apache/airflow/pull/21342#issuecomment-1031839888


   `white-space: pre-wrap;` is the CSS style you are looking for I suspect. 
Make sure it only gets applied to Variables though, not all Show pages.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #21348: Status of testing Providers that were prepared on February 05, 2022

[GitBox]

potiuk commented on issue #21348:
URL: https://github.com/apache/airflow/issues/21348#issuecomment-1031840031


   As discussed above ^^ I will cancel the whole vote/release and will make an 
RC2 as soon as we release the conditonally working SQL renderer. Thanks 
@josh-fell for spotting this and raising it! 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #21348: Status of testing Providers that were prepared on February 05, 2022

[GitBox]

potiuk commented on issue #21348:
URL: https://github.com/apache/airflow/issues/21348#issuecomment-1031838519


   Yeah. It's easy enough to fix and I agree it's better.
   
   I think - due to the the number of those affected, it makes more sense to 
cancel the whole release and re-release all providers as RC2 with that fix to 
not mess with two releases. there are no super-urgent changes in this wave.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ashb commented on issue #21392: .airflowignore does not use 'glob' as .gitignore for example

[GitBox]

ashb commented on issue #21392:
URL: https://github.com/apache/airflow/issues/21392#issuecomment-1031836603


   Yeah, if we wanted to change the format of this file to more closely match 
what people expect (which I agree is confusing) -- i.e. `.gitignore` we'd 
either have to change the filename, or add a `# airflow-ignore-v2` comment as 
the first line or something.
   
   (Of the two I'd prefer a new name, but I don't know what I'd call the file)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] jedcunningham commented on issue #21348: Status of testing Providers that were prepared on February 05, 2022

[GitBox]

jedcunningham commented on issue #21348:
URL: https://github.com/apache/airflow/issues/21348#issuecomment-1031835368


   I'd rather not include it for 2.2.4 since, as you said, it's a new feature. 
Doing it conditionally, until the next lower bound bump, I think makes sense.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk edited a comment on issue #21348: Status of testing Providers that were prepared on February 05, 2022

[GitBox]

potiuk edited a comment on issue #21348:
URL: https://github.com/apache/airflow/issues/21348#issuecomment-1031832322


   The second option is to indeed drop all the "21237" providers an release 
them in RC2 with some "conditional" code that will check if the lexer is there. 
After thinking a bit I think that would be much "cleaner" solution and I am 
leaning towards this option.
   
   smth like:
   ```
   template_fields_renderer = {//}.update({'sql': 'sql'} if 'sql' in 
get_attr_renderer() else {})
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] tlskr commented on issue #21395: Airflow Not Automatically Sending Errors to Sentry

[GitBox]

tlskr commented on issue #21395:
URL: https://github.com/apache/airflow/issues/21395#issuecomment-1031833167


   We're using `sentry-sdk`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org