[GitHub] [airflow] ephraimbuddy opened a new pull request, #23928: Clarify manual merging of PR in release doc
ephraimbuddy opened a new pull request, #23928: URL: https://github.com/apache/airflow/pull/23928 It was not clear to me what this really means -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] jacoDentsu commented on issue #10026: Duplicate tasks invoked for a single task_id when manually invoked task details modal.
jacoDentsu commented on issue #10026: URL: https://github.com/apache/airflow/issues/10026#issuecomment-1138244396 A new version 3.2 and still no fix... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #23927: fix: Databricks Provider Dependency #23917
github-actions[bot] commented on PR #23927: URL: https://github.com/apache/airflow/pull/23927#issuecomment-1138246743 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #23928: Clarify manual merging of PR in release doc
github-actions[bot] commented on PR #23928: URL: https://github.com/apache/airflow/pull/23928#issuecomment-1138248355 The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide that full test matrix is needed and add the 'full tests needed' label. Then you should rebase it to the latest main or amend the last commit of the PR, and push it with --force-with-lease. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #23760: OOME when using GCSObjectsWithPrefixExistenceSensor on bucket with many objects
uranusjr commented on issue #23760: URL: https://github.com/apache/airflow/issues/23760#issuecomment-1138257659 Since this is a provider package, the general rule is “try to keep compatibility, but break it if you must”. From your comment it seems like a break is necessary? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #23911: Add DAG "Skipped" status in case of "startDate" in the future while manually triggering a DAGRun
uranusjr commented on issue #23911: URL: https://github.com/apache/airflow/issues/23911#issuecomment-1138262737 Skipped sounds wrong because a skipped task means it _won’t_ run, and it’d be confusing if a skipped DAG run means otherwise. I wonder if we can simply set the state to None. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] wojsamjan closed pull request #23865: Cloud Storage assets & StorageLink update
wojsamjan closed pull request #23865: Cloud Storage assets & StorageLink update URL: https://github.com/apache/airflow/pull/23865 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] tanelk opened a new pull request, #23929: Clear next method when clearing TIs
tanelk opened a new pull request, #23929:
URL: https://github.com/apache/airflow/pull/23929
Clearing TIs did not allways completly clear deferred TIs.
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
for more information.
In case of fundamental code change, Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragement file, named `{pr_number}.significant.rst`, in
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] norm commented on a diff in pull request #23510: Fix permission issue for dag that has dot in name
norm commented on code in PR #23510:
URL: https://github.com/apache/airflow/pull/23510#discussion_r882478211
##
airflow/security/permissions.py:
##
@@ -66,14 +66,16 @@
DAG_ACTIONS = {ACTION_CAN_READ, ACTION_CAN_EDIT, ACTION_CAN_DELETE}
-def resource_name_for_dag(dag_id):
+def resource_name_for_dag(dag_id, is_subdag=False):
"""Returns the resource name for a DAG id."""
if dag_id == RESOURCE_DAG:
return dag_id
if dag_id.startswith(RESOURCE_DAG_PREFIX):
return dag_id
-# To account for SubDags
-root_dag_id = dag_id.split(".")[0]
-return f"{RESOURCE_DAG_PREFIX}{root_dag_id}"
+if is_subdag:
+# To account for SubDags
Review Comment:
I don't see this comment change actually in the code?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] norm commented on a diff in pull request #23510: Fix permission issue for dag that has dot in name
norm commented on code in PR #23510:
URL: https://github.com/apache/airflow/pull/23510#discussion_r882483645
##
airflow/security/permissions.py:
##
@@ -66,14 +66,16 @@
DAG_ACTIONS = {ACTION_CAN_READ, ACTION_CAN_EDIT, ACTION_CAN_DELETE}
-def resource_name_for_dag(dag_id):
+def resource_name_for_dag(dag_id, is_subdag=False):
"""Returns the resource name for a DAG id."""
if dag_id == RESOURCE_DAG:
return dag_id
if dag_id.startswith(RESOURCE_DAG_PREFIX):
return dag_id
-# To account for SubDags
-root_dag_id = dag_id.split(".")[0]
-return f"{RESOURCE_DAG_PREFIX}{root_dag_id}"
+if is_subdag:
+# To account for SubDags
+root_dag_id = dag_id.split(".", 1)[0]
Review Comment:
Is it possible to have a subdag of a dag with a dot in the name? eg
`data.munge` with a subdag making it `data.munge.subdag`? If so, this won't
return the right value for `root_dag_id`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] MaksYermak opened a new pull request, #23930: Add key_secret_project_id parameter which specifies a project with KeyFile
MaksYermak opened a new pull request, #23930:
URL: https://github.com/apache/airflow/pull/23930
In Airflow we have ability to work with Google Secret Manager, but I noticed
that Airflow doesn't have ability to take a project_id from which we want to
take this Secrets. In this PR I have added this ability.
Co-authored-by: Maksim Yermakou
[maks...@google.com](mailto:maks...@google.com)
Co-authored-by: Wojciech Januszek
[janus...@google.com](mailto:janus...@google.com)
Co-authored-by: Lukasz Wyszomirski
[wyszomir...@google.com](mailto:wyszomir...@google.com)
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
for more information.
In case of fundamental code change, Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragement file, named `{pr_number}.significant.rst`, in
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] koconder commented on pull request #23927: fix: Databricks Provider Dependency #23917
koconder commented on PR #23927: URL: https://github.com/apache/airflow/pull/23927#issuecomment-1138385288 @uranusjr the 1-2 random test failures seem totally unrelated to the provider code, is there some other issue with tests or is this transient? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk opened a new pull request, #23932: Fix failure for None task summaries
potiuk opened a new pull request, #23932:
URL: https://github.com/apache/airflow/pull/23932
Alternative fix to #23908 - with added typing.
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
for more information.
In case of fundamental code change, Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragement file, named `{pr_number}.significant.rst`, in
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #23924: Only return non-None task summaries for grid view
potiuk commented on code in PR #23924:
URL: https://github.com/apache/airflow/pull/23924#discussion_r882555795
##
airflow/www/views.py:
##
@@ -258,7 +258,11 @@ def task_group_to_grid(task_item_or_group, dag, dag_runs,
session):
if isinstance(task_item_or_group, AbstractOperator):
return {
'id': task_item_or_group.task_id,
-'instances': [wwwutils.get_task_summary(dr, task_item_or_group,
session) for dr in dag_runs],
+'instances': [
+ts
+for ts in (wwwutils.get_task_summary(dr, task_item_or_group,
session) for dr in dag_runs)
+if ts is not None
+],
Review Comment:
Alternative fix here: https://github.com/apache/airflow/pull/23932 - I think
we shoudl slowly start add typing also to views.py as this precisely the kind
of errors that would have been prevented if we had typing here in the first
place.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #23932: Fix failure for None task summaries
potiuk commented on code in PR #23932: URL: https://github.com/apache/airflow/pull/23932#discussion_r882569629 ## airflow/models/abstractoperator.py: ## @@ -101,6 +101,7 @@ class AbstractOperator(LoggingMixin, DAGNode): owner: str task_id: str +is_mapped: ClassVar[bool] Review Comment: @uranusjr -> maybe there is another way, but I believe the lack of "is_mapped" in AbstractOperator was a bit wrong (and it would prevent adding the proper typing in task_group_to_grid ad AbstractOperator missed `is_mapped` field. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #23932: Fix failure for None task summaries
potiuk commented on code in PR #23932: URL: https://github.com/apache/airflow/pull/23932#discussion_r882569629 ## airflow/models/abstractoperator.py: ## @@ -101,6 +101,7 @@ class AbstractOperator(LoggingMixin, DAGNode): owner: str task_id: str +is_mapped: ClassVar[bool] Review Comment: @uranusjr -> maybe there is another way, but I believe the lack of "is_mapped" in AbstractOperator was a bit wrong (and it would prevent adding the proper typing in task_group_to_grid ad AbstractOperator missed `is_mapped` field). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] mhenc commented on pull request #23886: Remove redundant register exit signals in `dag-processor` command
mhenc commented on PR #23886: URL: https://github.com/apache/airflow/pull/23886#issuecomment-1138438419 Thank you for taking care of that! LGTM from me, but we need to wait for @potiuk to approve and merge it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] henriqueribeiro commented on pull request #23887: Remove hack from BigQuery DTS hook
henriqueribeiro commented on PR #23887: URL: https://github.com/apache/airflow/pull/23887#issuecomment-1138442863 > Simply add a line to the `google` entry in `setup.py`. thanks a lot @uranusjr. I just pushed that change -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] bbovenzi commented on a diff in pull request #23924: Only return non-None task summaries for grid view
bbovenzi commented on code in PR #23924:
URL: https://github.com/apache/airflow/pull/23924#discussion_r882589037
##
airflow/www/views.py:
##
@@ -258,7 +258,11 @@ def task_group_to_grid(task_item_or_group, dag, dag_runs,
session):
if isinstance(task_item_or_group, AbstractOperator):
return {
'id': task_item_or_group.task_id,
-'instances': [wwwutils.get_task_summary(dr, task_item_or_group,
session) for dr in dag_runs],
+'instances': [
+ts
+for ts in (wwwutils.get_task_summary(dr, task_item_or_group,
session) for dr in dag_runs)
+if ts is not None
+],
Review Comment:
More type checking is always nice. I've been meaning to add typescript to
the UI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] matan-michaely opened a new issue, #23933: Failed to load Grid view when adding tasks to already exist dag
matan-michaely opened a new issue, #23933: URL: https://github.com/apache/airflow/issues/23933 ### Apache Airflow version 2.3.1 (latest released) ### What happened I have a DAG with several dag_runs which rendered fine in the Grid view. After adding new task Grid view failed to load getting blank view . There is an error in browser console. `TypeError: Cannot read properties of null (reading 'runId')` This Error comes from `grid.jsx` ### What you think should happen instead dag should be rendered and new task state in previous dag run should be `no_status` ### How to reproduce As described above create DAG with one task (you can use for example `BashOperator` with `echo` command) and run it. After you have at least one dag run add new task to your dag and try to reload Grid view ### Operating System Debian GNU/Linux ### Versions of Apache Airflow Providers ``` apache-airflow-providers-amazon==3.4.0 apache-airflow-providers-celery==2.1.4 apache-airflow-providers-cncf-kubernetes==4.0.2 apache-airflow-providers-docker==2.7.0 apache-airflow-providers-elasticsearch==3.0.3 apache-airflow-providers-ftp==2.1.2 apache-airflow-providers-google==7.0.0 apache-airflow-providers-grpc==2.0.4 apache-airflow-providers-hashicorp==2.2.0 apache-airflow-providers-http==2.1.2 apache-airflow-providers-imap==2.2.3 apache-airflow-providers-microsoft-azure==3.9.0 apache-airflow-providers-mysql==2.2.3 apache-airflow-providers-odbc==2.0.4 apache-airflow-providers-postgres==4.1.0 apache-airflow-providers-redis==2.0.4 apache-airflow-providers-sendgrid==2.0.4 apache-airflow-providers-sftp==2.6.0 apache-airflow-providers-slack==4.2.3 apache-airflow-providers-sqlite==2.1.3 apache-airflow-providers-ssh==2.4.4 ``` ### Deployment Official Apache Airflow Helm Chart ### Deployment details _No response_ ### Anything else _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #23933: Failed to load Grid view when adding tasks to already exist dag
boring-cyborg[bot] commented on issue #23933: URL: https://github.com/apache/airflow/issues/23933#issuecomment-1138445641 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] bbovenzi commented on pull request #23924: Only return non-None task summaries for grid view
bbovenzi commented on PR #23924: URL: https://github.com/apache/airflow/pull/23924#issuecomment-1138452020 I'll try out this and Jarek's solution in a few hours. I'll also open a PR to make sure the UI handles this issue better. We should definitely show a blank task instance in the grid if it doesn't exist in a dag run but does in others. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] pierrejeambrun opened a new pull request, #23934: Fix grid details button truncated and legend row wrapping
pierrejeambrun opened a new pull request, #23934:
URL: https://github.com/apache/airflow/pull/23934
Hello,
Really small PR to suggest small tweaks in the grid view UI.
Just noticed that the `details button` was truncated on my side when the
detail panel is hidden. Looking closer, it has an absolute position, and
appears after the Grid component in the dom, making it hard to align with the
AutoRefresh. Changed that to make it part of a common flex container with the
`AutoRefresh`.
Also took the opportunity to add wrapping to the `LegendRow` for a more
responsive UI.
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
for more information.
In case of fundamental code change, Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragement file, named `{pr_number}.significant.rst`, in
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] yfyg opened a new issue, #23935: No option to set blocksize when retrieving a file in ftphook
yfyg opened a new issue, #23935:
URL: https://github.com/apache/airflow/issues/23935
### Apache Airflow version
2.0.0
### What happened
using ftphook, im trying to download a file in chunks but the deafult
blocksize is 8192 and cannot be changed.
retrieve_file code is calling conn.retrbinary(f'RETR {remote_file_name}',
callback) but no blocksize is passed while this function is declared:
def retrbinary(self, cmd, callback, blocksize=8192, rest=None):
### What you think should happen instead
allow passing a blocksize
### How to reproduce
_No response_
### Operating System
gcp
### Versions of Apache Airflow Providers
_No response_
### Deployment
Docker-Compose
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #23935: No option to set blocksize when retrieving a file in ftphook
boring-cyborg[bot] commented on issue #23935: URL: https://github.com/apache/airflow/issues/23935#issuecomment-1138488713 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow-site] ephraimbuddy commented on pull request #602: Announce Apache Airflow 2.3.1
ephraimbuddy commented on PR #602: URL: https://github.com/apache/airflow-site/pull/602#issuecomment-1138506662 cc: @potiuk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow-site] potiuk merged pull request #602: Announce Apache Airflow 2.3.1
potiuk merged PR #602: URL: https://github.com/apache/airflow-site/pull/602 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[airflow-site] branch main updated: Announce Apache Airflow 2.3.1 (#602)
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/main by this push: new 4a5611859 Announce Apache Airflow 2.3.1 (#602) 4a5611859 is described below commit 4a5611859e103388eb165218be83edf6e090a0d1 Author: Ephraim Anierobi AuthorDate: Thu May 26 13:15:55 2022 +0100 Announce Apache Airflow 2.3.1 (#602) --- landing-pages/site/content/en/announcements/_index.md | 9 + 1 file changed, 9 insertions(+) diff --git a/landing-pages/site/content/en/announcements/_index.md b/landing-pages/site/content/en/announcements/_index.md index c63ce656b..0a2a7172f 100644 --- a/landing-pages/site/content/en/announcements/_index.md +++ b/landing-pages/site/content/en/announcements/_index.md @@ -13,6 +13,15 @@ menu: **Note:** Follow [@ApacheAirflow](https://twitter.com/ApacheAirflow) on Twitter for the latest news and announcements! +# May 25, 2022 + +We’ve just released Apache **Airflow 2.3.1**. + +📦 PyPI: https://pypi.org/project/apache-airflow/2.3.1/ \ +📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.3.1 \ +🛠️ Changelog: https://airflow.apache.org/docs/apache-airflow/2.3.1/release_notes.html \ +🪶 Sources: https://airflow.apache.org/docs/apache-airflow/2.3.1/installation/installing-from-sources.html + # May 20, 2022 We've just released Apache **Airflow Helm chart 1.6.0**.
[airflow-site] branch gh-pages updated: Deploying to gh-pages from @ 4a5611859e103388eb165218be83edf6e090a0d1 🚀
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/gh-pages by this push: new 04584892c Deploying to gh-pages from @ 4a5611859e103388eb165218be83edf6e090a0d1 🚀 04584892c is described below commit 04584892c982b4733fd1ddd8722e5ac0384ff900 Author: potiuk AuthorDate: Thu May 26 12:24:38 2022 + Deploying to gh-pages from @ 4a5611859e103388eb165218be83edf6e090a0d1 🚀 --- announcements/index.html | 9 ++ blog/airflow-1.10.10/index.html| 4 +- blog/airflow-1.10.12/index.html| 4 +- blog/airflow-1.10.8-1.10.9/index.html | 4 +- blog/airflow-2.2.0/index.html | 4 +- blog/airflow-2.3.0/index.html | 4 +- blog/airflow-survey-2020/index.html| 4 +- blog/airflow-survey/index.html | 4 +- blog/airflow-two-point-oh-is-here/index.html | 4 +- blog/airflow_summit_2021/index.html| 4 +- blog/airflow_summit_2022/index.html| 4 +- blog/announcing-new-website/index.html | 4 +- blog/apache-airflow-for-newcomers/index.html | 4 +- .../index.html | 4 +- .../index.html | 4 +- .../index.html | 4 +- .../index.html | 4 +- .../index.html | 4 +- .../index.html | 4 +- index.html | 32 search/index.html | 4 +- sitemap.xml| 96 +++--- use-cases/adobe/index.html | 4 +- use-cases/adyen/index.html | 4 +- use-cases/big-fish-games/index.html| 4 +- use-cases/dish/index.html | 4 +- use-cases/experity/index.html | 4 +- use-cases/onefootball/index.html | 4 +- use-cases/plarium-krasnodar/index.html | 4 +- use-cases/seniorlink/index.html| 4 +- use-cases/sift/index.html | 4 +- 31 files changed, 129 insertions(+), 120 deletions(-) diff --git a/announcements/index.html b/announcements/index.html index b7f75ec39..5ce19a133 100644 --- a/announcements/index.html +++ b/announcements/index.html @@ -464,6 +464,15 @@ Note: Follow https://twitter.com/ApacheAirflow"; target="_blank">@ApacheAirflow on Twitter for the latest news and announcements! +May 25, 2022 + +We’ve just released Apache Airflow 2.3.1. + +📦 PyPI: https://pypi.org/project/apache-airflow/2.3.1/"; target="_blank">https://pypi.org/project/apache-airflow/2.3.1/ +📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.3.1"; target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.1 +🛠️ Changelog: https://airflow.apache.org/docs/apache-airflow/2.3.1/release_notes.html"; target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.1/release_notes.html +🪶 Sources: https://airflow.apache.org/docs/apache-airflow/2.3.1/installation/installing-from-sources.html"; target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.1/installation/installing-from-sources.html + May 20, 2022 We’ve just released Apache Airflow Helm chart 1.6.0. diff --git a/blog/airflow-1.10.10/index.html b/blog/airflow-1.10.10/index.html index 30cbf3d71..3aa054520 100644 --- a/blog/airflow-1.10.10/index.html +++ b/blog/airflow-1.10.10/index.html @@ -36,13 +36,13 @@ - + - + diff --git a/blog/airflow-1.10.12/index.html b/blog/airflow-1.10.12/index.html index 133793770..06e7e2253 100644 --- a/blog/airflow-1.10.12/index.html +++ b/blog/airflow-1.10.12/index.html @@ -36,13 +36,13 @@ - + - + diff --git a/blog/airflow-1.10.8-1.10.9/index.html b/blog/airflow-1.10.8-1.10.9/index.html index 249e661aa..d80502654 100644 --- a/blog/airflow-1.10.8-1.10.9/index.html +++ b/blog/airflow-1.10.8-1.10.9/index.html @@ -36,13 +36,13 @@ - + - + diff --git a/blog/airflow-2.2.0/index.html b/blog/airflow-2.2.0/index.html index a719f4a21..4aeb68a8c 100644 --- a/blog/airflow-2.2.0/index.html +++ b/blog/airflow-2.2.0/index.html @@ -36,13 +36,13 @@ - + - + diff --git a/blog/airflow-2.3.0/index.html b/blog/airflow-2.3.0/index.html index df2a8961a..e1768960a 100644 --- a/blog/airflow-2.3.0/index.html +++ b/blog/airflow-2.3.0/index.html @@ -36,13 +36,13 @@ - + - + diff --git a/blog/airflow-survey-2020/index.html b/blog/airflow-survey-2020/index.html index 67409d4e3..d1537e78a 100644 --- a/blog/airflow-survey-2020/index.html +++ b/bl
[GitHub] [airflow] mfridrikhson commented on issue #13692: task failed with a null hostname
mfridrikhson commented on issue #13692: URL: https://github.com/apache/airflow/issues/13692#issuecomment-1138530242 We had the same issue with retrieving task logs but from what I've been able to find out it looks like it is not related to any connection issues with the actual log provider. Our specific case of this issue was the following: we got a task failure and the task instance was marked `failed` in the DAG view. When you go to the task view you could see 2 log tabs: first with the `Failed to fetch log file` message and second with a successful run and all the logs. I went to check the log folder and what I found out is that there was only logs for the second tab (task try):  _(Ignore the 3rd log file - it's from a later rerun after the incident I'm describing had happened)_ The actual cause why the UI renders such an error (at least in our case) is that it renders `max_try_number` number of tabs and queries the contents for each one by its index. It happened to be that there is no file for tab #1 and thus it shown this error message. So my next guess was that for some reason task instance's `try_number` got increased one extra time. And that's why I find @doowhtron's [comment](https://github.com/apache/airflow/issues/13692#issuecomment-800085669) important here - maybe some kind of race condition happens which causes the task to execute twice or something. Some notes about our case and environment: - The issue happens intermittently (usually everything works fine) - It is not happening to some specific task or operator (both sensors and default tasks had this issue) - It is not happening specifically when the cluster is experiencing high load or the opposite, so I don't think it has something to do with the performance - We use Airflow v2.2.4 - We store logs on S3 - We run a **single** scheduler -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23920: Update environments documentation links.
potiuk commented on PR #23920: URL: https://github.com/apache/airflow/pull/23920#issuecomment-1138531345 A LOT ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #14529: Clear the entire TaskGroup section
potiuk commented on issue #14529: URL: https://github.com/apache/airflow/issues/14529#issuecomment-1138546544 > Hey guys, what's going on with that task? I need this solution. Please, do it :) It's not complex I know it! Just do it! If you know it's easy @romj4k3 - feel free to provide PR. As simple as that. You can become one of the more than 2000 contributors. It's super easy way to repay the debt you have for using free software. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow-site] potiuk merged pull request #603: Fix 2.3.1 release date
potiuk merged PR #603: URL: https://github.com/apache/airflow-site/pull/603 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[airflow-site] branch main updated: Fix 2.3.1 release date (#603)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
The following commit(s) were added to refs/heads/main by this push:
new abc7d8f8a Fix 2.3.1 release date (#603)
abc7d8f8a is described below
commit abc7d8f8ab23eb1483d1199c35367b7128c6e2a7
Author: Jed Cunningham <66968678+jedcunning...@users.noreply.github.com>
AuthorDate: Thu May 26 07:09:38 2022 -0600
Fix 2.3.1 release date (#603)
---
.../apache-airflow/2.3.1/cli-and-env-variables-ref.html | 16
docs-archive/apache-airflow/2.3.1/release_notes.html | 12 ++--
2 files changed, 14 insertions(+), 14 deletions(-)
diff --git a/docs-archive/apache-airflow/2.3.1/cli-and-env-variables-ref.html
b/docs-archive/apache-airflow/2.3.1/cli-and-env-variables-ref.html
index 66a44b21e..9d6750a9f 100644
--- a/docs-archive/apache-airflow/2.3.1/cli-and-env-variables-ref.html
+++ b/docs-archive/apache-airflow/2.3.1/cli-and-env-variables-ref.html
@@ -1869,14 +1869,14 @@ development and testing.
list¶
List options for the configuration
-airflow config list [-h] [--color {auto,off,on [...]
+airflow config list [-h] [--color {auto,on,off [...]
Named Arguments¶
--color
-Possible choices: auto, off, on
+Possible choices: auto, on, off
Do emit colored output (default: auto)
Default: “auto”
@@ -1960,7 +1960,7 @@ development and testing.
delete¶
Delete a connection
-airflow connections delete [-h] [--color {auto,off,airflow connections delete [-h] [--color {auto,on,
@@ -1975,7 +1975,7 @@ development and testing.
Named Arguments¶
--color
-Possible choices: auto, off, on
+Possible choices: auto, on, off
Do emit colored output (default: auto)
Default: “auto”
@@ -2028,7 +2028,7 @@ When exporting in env file format, you
control whether URI format o
get¶
Get a connection
-airflow connections get [-h] [--color {auto,off,airflow connections get [-h] [--color {auto,on, [...]
[-o table, json, yaml, plain] [-v]
conn_id
@@ -2045,7 +2045,7 @@ When exporting in env file format, you
control whether URI format o
Named Arguments¶
--color
-Possible choices: auto, off, on
+Possible choices: auto, on, off
Do emit colored output (default: auto)
Default: “auto”
@@ -3331,7 +3331,7 @@ To check if the local scheduler is still working
properly, run:
get¶
Get detailed information about a provider
-airflow providers get [-h] [--color {auto,off,o [...]
+airflow providers get [-h] [--color {auto,on,of [...]
[-o table, json, yaml, plain] [-v]
provider_name
@@ -3348,7 +3348,7 @@ To check if the local scheduler is still working
properly, run:
Named Arguments¶
--color
-Possible choices: auto, off, on
+Possible choices: auto, on, off
Do emit colored output (default: auto)
Default: “auto”
diff --git a/docs-archive/apache-airflow/2.3.1/release_notes.html
b/docs-archive/apache-airflow/2.3.1/release_notes.html
index 43af62a7d..b2e0cb1b2 100644
--- a/docs-archive/apache-airflow/2.3.1/release_notes.html
+++ b/docs-archive/apache-airflow/2.3.1/release_notes.html
@@ -353,7 +353,7 @@
Modules Management
Release Policies
Release Notes
-Airflow 2.3.1 (2022-05-24)
+Airflow 2.3.1 (2022-05-25)
Significant Changes
Bug
Fixes
Doc only changes
@@ -1188,7 +1188,7 @@
Modules Management
Release Policies
Release Notes
-Airflow 2.3.1 (2022-05-24)
+Airflow 2.3.1 (2022-05-25)
Significant Changes
Bug
Fixes
Doc only changes
@@ -1981,7 +1981,7 @@
Apache Airflow Releases
-Airflow 2.3.1 (2022-05-24)
+Airflow 2.3.1 (2022-05-25)
Airflow 2.3.0 (2022-04-30)
Airflow 2.2.5, (2022-04-04)
Airflow 2.2.4, (2022-02-22)
@@ -2021,8 +2021,8 @@
Airflow 1.7.1 (2016-05-19)
-
-Airflow 2.3.1 (2022-05-24)¶
+
+Airflow 2.3.1 (2022-05-25)¶
Significant Changes¶
No significant changes.
@@ -12149,7 +12149,7 @@ It was not confirmed, but a workaround was found by
changing the default back to
Release Notes
-Airflow
2.3.1 (2022-05-24)
+Airflow
2.3.1 (2022-05-25)
Significant
Changes
Bug Fixes
Doc only
changes
[GitHub] [airflow] potiuk commented on a diff in pull request #23841: Make CI and PROD image builds consistent
potiuk commented on code in PR #23841:
URL: https://github.com/apache/airflow/pull/23841#discussion_r882653595
##
.github/workflows/ci.yml:
##
@@ -309,8 +309,6 @@ jobs:
with:
python-version: ${{ needs.build-info.outputs.defaultPythonVersion }}
if: needs.build-info.outputs.inWorkflowBuild == 'true'
- - run: ./scripts/ci/install_breeze.sh
-if: needs.build-info.outputs.inWorkflowBuild == 'true'
- name: "Retrieve DEFAULTS from the _initialization.sh"
Review Comment:
Absolutely. There is no bad side effect of it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #23841: Make CI and PROD image builds consistent
github-actions[bot] commented on PR #23841: URL: https://github.com/apache/airflow/pull/23841#issuecomment-1138561616 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #23841: Make CI and PROD image builds consistent
potiuk merged PR #23841: URL: https://github.com/apache/airflow/pull/23841 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[airflow] branch main updated: Make CI and PROD image builds consistent (#23841)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 30262b1fe3 Make CI and PROD image builds consistent (#23841)
30262b1fe3 is described below
commit 30262b1fe3b095168c9b4fe5a78255048d2f410f
Author: Jed Cunningham <66968678+jedcunning...@users.noreply.github.com>
AuthorDate: Thu May 26 07:13:15 2022 -0600
Make CI and PROD image builds consistent (#23841)
Simple refactoring to make the jobs more consistent.
---
.github/workflows/ci.yml | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 6281f09e68..a49f185d39 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -309,8 +309,6 @@ jobs:
with:
python-version: ${{ needs.build-info.outputs.defaultPythonVersion }}
if: needs.build-info.outputs.inWorkflowBuild == 'true'
- - run: ./scripts/ci/install_breeze.sh
-if: needs.build-info.outputs.inWorkflowBuild == 'true'
- name: "Retrieve DEFAULTS from the _initialization.sh"
# We cannot "source" the script here because that would be a security
problem (we cannot run
# any code that comes from the sources coming from the PR. Therefore
we extract the
@@ -328,6 +326,8 @@ jobs:
awk 'BEGIN{FS="="} {print $3}' | sed s'/["}]//g')
echo "DEBIAN_VERSION=${DEBIAN_VERSION}" >> $GITHUB_ENV
if: needs.build-info.outputs.inWorkflowBuild == 'true'
+ - run: ./scripts/ci/install_breeze.sh
+if: needs.build-info.outputs.inWorkflowBuild == 'true'
- name: "Free space"
run: breeze free-space
if: needs.build-info.outputs.inWorkflowBuild == 'true'
@@ -392,7 +392,7 @@ jobs:
awk 'BEGIN{FS="="} {print $3}' | sed s'/["}]//g')
echo "DEFAULT_CONSTRAINTS_BRANCH=${DEFAULT_CONSTRAINTS_BRANCH}" >>
$GITHUB_ENV
DEBIAN_VERSION=$(grep "export DEBIAN_VERSION"
scripts/ci/libraries/_initialization.sh | \
-cut -d "=" -f 3 | sed s'/["}]//g')
+awk 'BEGIN{FS="="} {print $3}' | sed s'/["}]//g')
echo "DEBIAN_VERSION=${DEBIAN_VERSION}" >> $GITHUB_ENV
if: needs.build-info.outputs.inWorkflowBuild == 'true'
- run: ./scripts/ci/install_breeze.sh
[GitHub] [airflow] potiuk merged pull request #23923: Alphabetizes two tables
potiuk merged PR #23923: URL: https://github.com/apache/airflow/pull/23923 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[airflow] branch main updated: Alphabetizes two tables (#23923)
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new ef3f5301f3 Alphabetizes two tables (#23923) ef3f5301f3 is described below commit ef3f5301f3f04b6718b2aeb722deffd9c075fcde Author: Marty Jackson AuthorDate: Thu May 26 06:14:34 2022 -0700 Alphabetizes two tables (#23923) The rest of the page has consistently alphabetized tables. This commit fixes three `extras` that were not alphabetized. --- docs/apache-airflow/extra-packages-ref.rst | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/apache-airflow/extra-packages-ref.rst b/docs/apache-airflow/extra-packages-ref.rst index e281a45e85..cefa4a67bf 100644 --- a/docs/apache-airflow/extra-packages-ref.rst +++ b/docs/apache-airflow/extra-packages-ref.rst @@ -218,6 +218,8 @@ Those are extras that add dependencies needed for integration with other softwar +-+-+---+ | extra | install command | enables | +=+=+===+ +| arangodb| ``pip install 'apache-airflow[arangodb]'`` | ArangoDB operators, sensors and hook | ++-+-+---+ | docker | ``pip install 'apache-airflow[docker]'``| Docker hooks and operators| +-+-+---+ | elasticsearch | ``pip install 'apache-airflow[elasticsearch]'`` | Elasticsearch hooks and Log Handler | @@ -256,8 +258,6 @@ Those are extras that add dependencies needed for integration with other softwar +-+-+---+ | trino | ``pip install 'apache-airflow[trino]'`` | All Trino related operators & hooks | +-+-+---+ -| arangodb| ``pip install 'apache-airflow[arangodb]'`` | ArangoDB operators, sensors and hook | -+-+-+---+ Other extras @@ -278,6 +278,10 @@ Those are extras that provide support for integration with external systems via +-+-+--+--+ | jdbc| ``pip install 'apache-airflow[jdbc]'`` | JDBC hooks and operators | | +-+-+--+--+ +| microsoft.psrp | ``pip install 'apache-airflow[microsoft.psrp]'``| PSRP hooks and operators | | ++-+-+--+--+ +| microsoft.winrm | ``pip install 'apache-airflow[microsoft.winrm]'`` | WinRM hooks and operators| | ++-+-+--+--+ | papermill | ``pip install 'apache-airflow[papermill]'`` | Papermill hooks and operators| | +-+-+--+--+ | sftp| ``pip install 'apache-airflow[sftp]'`` | SFTP hooks, operators and sensors| | @@ -286,10 +290,6 @@ Those are extras that provide support for integration with external systems via +-+-+--+--+ | ssh | ``pip install 'apache-airflow[ssh]'`` | SSH hooks and operators | | +-+-+--+--+ -| microsoft.psrp | ``pip install 'apache-airflow[microsoft.psrp]'``| PSRP hooks and operators | | -+-+-+--+--+
[GitHub] [airflow] josh-fell commented on issue #23935: No option to set blocksize when retrieving a file in ftphook
josh-fell commented on issue #23935: URL: https://github.com/apache/airflow/issues/23935#issuecomment-1138565463 @yfyg It seems like you know what the feature implementation should be and can easily test it too. Do you want to submit a PR for this? Airflow has over 2K contributors and we love when new people come onboard! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[airflow-site] branch gh-pages updated: Deploying to gh-pages from @ abc7d8f8ab23eb1483d1199c35367b7128c6e2a7 🚀
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/gh-pages by this push: new 9c35b830c Deploying to gh-pages from @ abc7d8f8ab23eb1483d1199c35367b7128c6e2a7 🚀 9c35b830c is described below commit 9c35b830c56ae7bb8724c1a931797be0ba84e56d Author: potiuk AuthorDate: Thu May 26 13:18:23 2022 + Deploying to gh-pages from @ abc7d8f8ab23eb1483d1199c35367b7128c6e2a7 🚀 --- blog/airflow-1.10.10/index.html| 4 +- blog/airflow-1.10.12/index.html| 4 +- blog/airflow-1.10.8-1.10.9/index.html | 4 +- blog/airflow-2.2.0/index.html | 4 +- blog/airflow-2.3.0/index.html | 4 +- blog/airflow-survey-2020/index.html| 4 +- blog/airflow-survey/index.html | 4 +- blog/airflow-two-point-oh-is-here/index.html | 4 +- blog/airflow_summit_2021/index.html| 4 +- blog/airflow_summit_2022/index.html| 4 +- blog/announcing-new-website/index.html | 4 +- blog/apache-airflow-for-newcomers/index.html | 4 +- .../index.html | 4 +- .../index.html | 4 +- .../index.html | 4 +- .../index.html | 4 +- .../index.html | 4 +- .../index.html | 4 +- .../2.3.1/cli-and-env-variables-ref.html | 16 ++-- docs/apache-airflow/2.3.1/release_notes.html | 12 +-- .../stable/cli-and-env-variables-ref.html | 16 ++-- docs/apache-airflow/stable/release_notes.html | 12 +-- index.html | 30 +++ search/index.html | 4 +- sitemap.xml| 96 +++--- use-cases/adobe/index.html | 4 +- use-cases/adyen/index.html | 4 +- use-cases/big-fish-games/index.html| 4 +- use-cases/dish/index.html | 4 +- use-cases/experity/index.html | 4 +- use-cases/onefootball/index.html | 4 +- use-cases/plarium-krasnodar/index.html | 4 +- use-cases/seniorlink/index.html| 4 +- use-cases/sift/index.html | 4 +- 34 files changed, 147 insertions(+), 147 deletions(-) diff --git a/blog/airflow-1.10.10/index.html b/blog/airflow-1.10.10/index.html index 3aa054520..a6fc0791f 100644 --- a/blog/airflow-1.10.10/index.html +++ b/blog/airflow-1.10.10/index.html @@ -36,13 +36,13 @@ - + - + diff --git a/blog/airflow-1.10.12/index.html b/blog/airflow-1.10.12/index.html index 06e7e2253..098c88a00 100644 --- a/blog/airflow-1.10.12/index.html +++ b/blog/airflow-1.10.12/index.html @@ -36,13 +36,13 @@ - + - + diff --git a/blog/airflow-1.10.8-1.10.9/index.html b/blog/airflow-1.10.8-1.10.9/index.html index d80502654..3d75a19c7 100644 --- a/blog/airflow-1.10.8-1.10.9/index.html +++ b/blog/airflow-1.10.8-1.10.9/index.html @@ -36,13 +36,13 @@ - + - + diff --git a/blog/airflow-2.2.0/index.html b/blog/airflow-2.2.0/index.html index 4aeb68a8c..8f327d406 100644 --- a/blog/airflow-2.2.0/index.html +++ b/blog/airflow-2.2.0/index.html @@ -36,13 +36,13 @@ - + - + diff --git a/blog/airflow-2.3.0/index.html b/blog/airflow-2.3.0/index.html index e1768960a..260c65fdc 100644 --- a/blog/airflow-2.3.0/index.html +++ b/blog/airflow-2.3.0/index.html @@ -36,13 +36,13 @@ - + - + diff --git a/blog/airflow-survey-2020/index.html b/blog/airflow-survey-2020/index.html index d1537e78a..fa07f19fb 100644 --- a/blog/airflow-survey-2020/index.html +++ b/blog/airflow-survey-2020/index.html @@ -36,13 +36,13 @@ - + - + diff --git a/blog/airflow-survey/index.html b/blog/airflow-survey/index.html index af6b67768..ae7264445 100644 --- a/blog/airflow-survey/index.html +++ b/blog/airflow-survey/index.html @@ -36,13 +36,13 @@ - + - + diff --git a/blog/airflow-two-point-oh-is-here/index.html b/blog/airflow-two-point-oh-is-here/index.html index 014cba79c..6d5a901f4 100644 --- a/blog/airflow-two-point-oh-is-here/index.html +++ b/blog/airflow-two-point-oh-is-here/index.html @@ -36,13 +36,13 @@ - + - + diff --git a/blog/airflow_summit_2021/index.html b/blog/airflow_summit_2021/index.html index e89e0beef..43250f135 100644 --- a/blog/airflow_summit_2021/index.html +++ b/blog/airflow_summit_2021/index.html @@ -36,13 +36,13 @@ - + - + diff --git a/blog/airflow_summit_2022/index.html b/blog/airf
[GitHub] [airflow] potiuk commented on issue #15933: Consider splitting Google Provider
potiuk commented on issue #15933: URL: https://github.com/apache/airflow/issues/15933#issuecomment-1138568255 @rodrigo-fss -> Plyvel is not a requirement for Google Provider any more - it's optional dependency - you do not have to install it. But in any case you need to upgrade to 2.1+ as google providers are already not compatible with 2.0.* line so you are using a really old version of those and even if we split providers, you will have to upgrade anyway to use that. So upgreding to Airflow 2.1+ (Ideallly 2.3+) is the RIGHT solution to your problem, not waiting for splitting the provider. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15770: google provider requires leveldb installed in system
potiuk commented on issue #15770: URL: https://github.com/apache/airflow/issues/15770#issuecomment-1138571580 Yeah. As mentioned in #15933 the right solution to your problem is to upgrade to Airflow 2.1+. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23927: fix: Databricks Provider Dependency #23917
potiuk commented on PR #23927: URL: https://github.com/apache/airflow/pull/23927#issuecomment-1138574833 The problem is that we currently have some newly released dependencies breaking main (and any of the PRs that change requirements). We will need to fix it first (I am going to take a look at it shortly - but since we have Airflow Summit this week - please a little patience :). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23924: Only return non-None task summaries for grid view
potiuk commented on PR #23924: URL: https://github.com/apache/airflow/pull/23924#issuecomment-1138594827 > I'll try out this and Jarek's solution in a few hours. I'll also open a PR to make sure the UI handles this issue better. We should definitely show a blank task instance in the grid if it doesn't exist in a dag run but does in others. Oh yeah. Absolutelye. Typescript is s much better than plain javascript. One thing that it makes easier - is for anyone to contribute more fixes in an easy way. (once we have a built-in automation to transpile it, which is super-easy with yarn etc). I think lack of typing support, autocomplete and verification makes it really difficult for "newcomers" to reason about their changes and it's so much easier to contribute small fixes there. You somehow loose that when you get familiar, when you put yourselve in the shoes of new contributor - it's a world of difference. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #23924: Only return non-None task summaries for grid view
potiuk commented on code in PR #23924:
URL: https://github.com/apache/airflow/pull/23924#discussion_r882686726
##
airflow/www/views.py:
##
@@ -258,7 +258,11 @@ def task_group_to_grid(task_item_or_group, dag, dag_runs,
session):
if isinstance(task_item_or_group, AbstractOperator):
return {
'id': task_item_or_group.task_id,
-'instances': [wwwutils.get_task_summary(dr, task_item_or_group,
session) for dr in dag_runs],
+'instances': [
+ts
+for ts in (wwwutils.get_task_summary(dr, task_item_or_group,
session) for dr in dag_runs)
+if ts is not None
+],
Review Comment:
> More type checking is always nice. I've been meaning to add typescript to
the UI.
Oh yeah. Absolutely. Typescript is s much better than plain javascript.
One thing that it makes easier - is for anyone to contribute more fixes in an
easy way. (once we have a built-in automation to transpile it, which is
super-easy with yarn etc). I think lack of typing support, autocomplete and
verification makes it really difficult for "newcomers" to reason about their
changes and it's so much easier to contribute small fixes there.
You somehow loose that when you get familiar, when you put yourselve in the
shoes of new contributor - it's a world of difference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] tirkarthi commented on issue #23925: Airflow 2.3.1 : "CryptographyDeprecationWarning: Blowfish has been deprecated" message on gunicorn 20.1.0 startup
tirkarthi commented on issue #23925: URL: https://github.com/apache/airflow/issues/23925#issuecomment-1138618369 Related issue https://github.com/paramiko/paramiko/issues/2038 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #23925: Airflow 2.3.1 : "CryptographyDeprecationWarning: Blowfish has been deprecated" message on gunicorn 20.1.0 startup
potiuk commented on issue #23925: URL: https://github.com/apache/airflow/issues/23925#issuecomment-1138621639 I wil take a look at that one shortly (i.e. next week due to Summit). It's not clear for me what blocks paramiko from upgrading to the latest version - even in main). Stay tuned. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] abhishekbhakat commented on issue #23361: Scheduler crashes with psycopg2.errors.DeadlockDetected exception
abhishekbhakat commented on issue #23361:
URL: https://github.com/apache/airflow/issues/23361#issuecomment-1138646896
> The deadlock issue is not the cause of the tasks being set to skipped --
the deadlock occurs when trying to.
>
> @abhishekbhakat what error do you get with that? The log you included
there doesn't show any error.
@ashb The log file provided has that error in around line no. 5810
Attaching the error message below:
```
[[34m2022-04-29 12:25:18,883[0m] {[34mscheduler_job.py:[0m753} ERROR[0m
- Exception when executing SchedulerJob._run_scheduler_loop[0m
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py",
line 1685, in _execute_context
self.dialect.do_executemany(
File
"/usr/local/lib/python3.9/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py",
line 917, in do_executemany
self._psycopg2_extras().execute_batch(
File "/usr/local/lib/python3.9/site-packages/psycopg2/extras.py", line
1187, in execute_batch
cur.execute(b";".join(sqls))
psycopg2.errors.DeadlockDetected: deadlock detected
DETAIL: Process 29749 waits for ShareLock on transaction 62344; blocked by
process 31075.
Process 31075 waits for ShareLock on transaction 62338; blocked by process
29749.
HINT: See server log for query details.
CONTEXT: while updating tuple (4,23) in relation "task_instance"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File
"/usr/local/lib/python3.9/site-packages/airflow/jobs/scheduler_job.py", line
736, in _execute
self._run_scheduler_loop()
File
"/usr/local/lib/python3.9/site-packages/airflow/jobs/scheduler_job.py", line
824, in _run_scheduler_loop
num_queued_tis = self._do_scheduling(session)
File
"/usr/local/lib/python3.9/site-packages/airflow/jobs/scheduler_job.py", line
906, in _do_scheduling
callback_to_run = self._schedule_dag_run(dag_run, session)
File
"/usr/local/lib/python3.9/site-packages/airflow/jobs/scheduler_job.py", line
1146, in _schedule_dag_run
self._verify_integrity_if_dag_changed(dag_run=dag_run, session=session)
File "/usr/local/lib/python3.9/site-packages/airflow/utils/session.py",
line 68, in wrapper
return func(*args, **kwargs)
File
"/usr/local/lib/python3.9/site-packages/airflow/jobs/scheduler_job.py", line
1176, in _verify_integrity_if_dag_changed
dag_run.verify_integrity(session=session)
File "/usr/local/lib/python3.9/site-packages/airflow/utils/session.py",
line 68, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/airflow/models/dagrun.py",
line 944, in verify_integrity
session.flush()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py",
line 3255, in flush
self._flush(objects)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py",
line 3395, in _flush
transaction.rollback(_capture_exception=True)
File
"/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line
70, in __exit__
compat.raise_(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py",
line 211, in raise_
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py",
line 3355, in _flush
flush_context.execute()
File
"/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line
453, in execute
rec.execute(self)
File
"/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line
627, in execute
util.preloaded.orm_persistence.save_obj(
File
"/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line
234, in save_obj
_emit_update_statements(
File
"/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line
998, in _emit_update_statements
c = connection._execute_20(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py",
line 1520, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/sql/elements.py",
line 313, in _execute_on_connection
return connection._execute_clauseelement(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py",
line 1389, in _execute_clauseelement
ret = self._execute_context(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py",
line 1748, in _execute_context
self._handle_dbapi_exception(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py",
line 1929, in _handle_dbapi_exception
util.raise_(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py",
line 211, in raise_
raise exception
File "/usr/local
[GitHub] [airflow] dstandish merged pull request #23676: Use "remote" pod when patching KPO pod as "checked"
dstandish merged PR #23676: URL: https://github.com/apache/airflow/pull/23676 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[airflow] branch main updated: Use "remote" pod when patching KPO pod as "checked" (#23676)
This is an automated email from the ASF dual-hosted git repository.
dstandish pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 6bbe015905 Use "remote" pod when patching KPO pod as "checked" (#23676)
6bbe015905 is described below
commit 6bbe015905bd2709e621455d9f71a78b374d1337
Author: Daniel Standish <15932138+dstand...@users.noreply.github.com>
AuthorDate: Thu May 26 07:40:19 2022 -0700
Use "remote" pod when patching KPO pod as "checked" (#23676)
When patching as "checked", we have to use the current version of the pod
otherwise we may get an error when trying to patch it, e.g.:
```
Operation cannot be fulfilled on pods
\"test-kubernetes-pod-db9eedb7885c40099dd40cd4edc62415\": the object has been
modified; please apply your changes to the latest version and try again"
```
This error would not cause a failure of the task, since errors in `cleanup`
are suppressed. However, it would fail to patch.
I believe one scenario when the pod may be updated is when retrieving xcom,
since the sidecar is terminated after extracting the value.
Concerning some changes in the tests re the "already_checked" label, it was
added to a few "expected pods" recently, when we changed it to patch even in
the case of a successful pod.
Since we are changing the "patch" code to patch with the latest read on the
pod that we have (i.e. using the `remote_pod` variable), and no longer the pod
object stored on `k.pod`, the label no longer shows up in those tests (that's
because in k.pod isn't actually a read of the remote pod, but just happens to
get mutated in the patch function before it is used to actually patch the pod).
Further, since the `remote_pod` is a local variable, we can't observe it in
tests. So we have to read the pod using k8s api. _But_, our "find pod"
function excludes "already checked" pods! So we have to make this configurable.
So, now we have a proper integration test for the "already_checked"
behavior (there was already a unit test).
---
.../cncf/kubernetes/operators/kubernetes_pod.py| 14 --
kubernetes_tests/test_kubernetes_pod_operator.py | 55 --
.../test_kubernetes_pod_operator_backcompat.py | 1 -
3 files changed, 61 insertions(+), 9 deletions(-)
diff --git a/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py
b/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py
index ee55d34ae1..6e46fdbb16 100644
--- a/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py
+++ b/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py
@@ -330,9 +330,9 @@ class KubernetesPodOperator(BaseOperator):
kwargs.update(in_cluster=self.in_cluster)
return kube_client.get_kube_client(**kwargs)
-def find_pod(self, namespace, context) -> Optional[k8s.V1Pod]:
+def find_pod(self, namespace, context, *, exclude_checked=True) ->
Optional[k8s.V1Pod]:
"""Returns an already-running pod for this task instance if one
exists."""
-label_selector = self._build_find_pod_label_selector(context)
+label_selector = self._build_find_pod_label_selector(context,
exclude_checked=exclude_checked)
pod_list = self.client.list_namespaced_pod(
namespace=namespace,
label_selector=label_selector,
@@ -412,7 +412,7 @@ class KubernetesPodOperator(BaseOperator):
pod_phase = remote_pod.status.phase if hasattr(remote_pod, 'status')
else None
if not self.is_delete_operator_pod:
with _suppress(Exception):
-self.patch_already_checked(pod)
+self.patch_already_checked(remote_pod)
if pod_phase != PodPhase.SUCCEEDED:
if self.log_events_on_failure:
with _suppress(Exception):
@@ -436,10 +436,14 @@ class KubernetesPodOperator(BaseOperator):
else:
self.log.info("skipping deleting pod: %s", pod.metadata.name)
-def _build_find_pod_label_selector(self, context: Optional[dict] = None)
-> str:
+def _build_find_pod_label_selector(self, context: Optional[dict] = None,
*, exclude_checked=True) -> str:
labels = self._get_ti_pod_labels(context, include_try_number=False)
label_strings = [f'{label_id}={label}' for label_id, label in
sorted(labels.items())]
-return ','.join(label_strings) +
f',{self.POD_CHECKED_KEY}!=True,!airflow-worker'
+labels_value = ','.join(label_strings)
+if exclude_checked:
+labels_value += f',{self.POD_CHECKED_KEY}!=True'
+labels_value += ',!airflow-worker'
+return labels_value
def _set_name(self, name):
if name is None:
diff --git a/kubernetes_tests/test_kubernetes_pod_operator.py
b/kubernetes_tests/test_kubernetes_pod_operator.py
index 25c5bab763..56705c7a9d 100644
[GitHub] [airflow] eladkal commented on pull request #23927: fix: Databricks Provider Dependency #23917
eladkal commented on PR #23927: URL: https://github.com/apache/airflow/pull/23927#issuecomment-1138651784 > I am going to take a look at it shortly main is broken by moto they already fixed the issue https://github.com/spulec/moto/pull/5165 but didn't release 3.1.11 yet -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] igorborgest opened a new pull request, #23938: Update INSTALL_PROVIDERS_FROM_SOURCES instructions.
igorborgest opened a new pull request, #23938: URL: https://github.com/apache/airflow/pull/23938 Hi! I'm still digging into the docs 🤓 I think there is something inaccurate about the `INSTALL_PROVIDERS_FROM_SOURCES` explanation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] danielbeach commented on issue #22317: Make pause DAG its own role separate from edit DAG
danielbeach commented on issue #22317: URL: https://github.com/apache/airflow/issues/22317#issuecomment-1138654415 @eladkal nope. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] bbovenzi opened a new pull request, #23939: Prevent UI from crashing if grid task instances are null
bbovenzi opened a new pull request, #23939:
URL: https://github.com/apache/airflow/pull/23939
UI-side fix with an additional test.
Fixes: #23908
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
for more information.
In case of fundamental code change, Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragement file, named `{pr_number}.significant.rst`, in
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] Dark-Knight11 commented on issue #9941: Extra links for Google Cloud operators
Dark-Knight11 commented on issue #9941: URL: https://github.com/apache/airflow/issues/9941#issuecomment-1138660510 Hi, I would like to add operators for Compute Engine service. Can you please guide me on it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #23940: Fix broken main
github-actions[bot] commented on PR #23940: URL: https://github.com/apache/airflow/pull/23940#issuecomment-1138661225 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] ephraimbuddy merged pull request #23928: Clarify manual merging of PR in release doc
ephraimbuddy merged PR #23928: URL: https://github.com/apache/airflow/pull/23928 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[airflow] branch main updated: Clarify manual merging of PR in release doc (#23928)
This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new bfe51ea64a Clarify manual merging of PR in release doc (#23928)
bfe51ea64a is described below
commit bfe51ea64a6957587c33baf32916b205dea91a45
Author: Ephraim Anierobi
AuthorDate: Thu May 26 15:52:10 2022 +0100
Clarify manual merging of PR in release doc (#23928)
It was not clear to me what this really means
---
dev/README_RELEASE_AIRFLOW.md | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/dev/README_RELEASE_AIRFLOW.md b/dev/README_RELEASE_AIRFLOW.md
index ebe94e6abb..6c050ebd4a 100644
--- a/dev/README_RELEASE_AIRFLOW.md
+++ b/dev/README_RELEASE_AIRFLOW.md
@@ -224,7 +224,12 @@ The Release Candidate artifacts we vote upon should be the
exact ones we vote ag
- Update the `REVISION_HEADS_MAP` at airflow/utils/db.py to include the
revision head of the release even if there are no migrations.
- Commit the version change.
-- PR from the 'test' branch to the 'stable' branch, and manually merge it once
approved.
+- PR from the 'test' branch to the 'stable' branch, and manually merge it once
approved. Here's how to manually merge the PR:
+
+```shell script
+git merge --ff-only v${VERSION_BRANCH}-test
+```
+
- Check out the 'stable' branch
```shell script
[GitHub] [airflow] bbovenzi commented on pull request #23924: Only return non-None task summaries for grid view
bbovenzi commented on PR #23924: URL: https://github.com/apache/airflow/pull/23924#issuecomment-1138657866 UI fix: https://github.com/apache/airflow/pull/23939 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] eladkal opened a new pull request, #23940: Fix broken main
eladkal opened a new pull request, #23940:
URL: https://github.com/apache/airflow/pull/23940
main breaks with
```
Traceback:
/usr/local/lib/python3.7/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
tests/providers/amazon/aws/hooks/test_cloud_formation.py:31: in
class TestCloudFormationHook(unittest.TestCase):
tests/providers/amazon/aws/hooks/test_cloud_formation.py:67: in
TestCloudFormationHook
@mock_cloudformation
/usr/local/lib/python3.7/site-packages/moto/__init__.py:30: in f
module = importlib.import_module(module_name, "moto")
/usr/local/lib/python3.7/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
/usr/local/lib/python3.7/site-packages/moto/cloudformation/__init__.py:1:
in
from .models import cloudformation_backends
/usr/local/lib/python3.7/site-packages/moto/cloudformation/models.py:18:
in
from .parsing import ResourceMap, OutputMap
/usr/local/lib/python3.7/site-packages/moto/cloudformation/parsing.py:17:
in
from moto.apigateway import models # noqa # pylint: disable=all
/usr/local/lib/python3.7/site-packages/moto/apigateway/__init__.py:1: in
from .models import apigateway_backends
/usr/local/lib/python3.7/site-packages/moto/apigateway/models.py:9: in
from openapi_spec_validator import validate_spec
E ModuleNotFoundError: No module named 'openapi_spec_validator'
```
Fix is already in placed in moto https://github.com/spulec/moto/pull/5165
but version 3.1.11 wasn't released yet
Example for CI fail:
https://github.com/apache/airflow/runs/6605589497?check_suite_focus=true#step:9:5685
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
for more information.
In case of fundamental code change, Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragement file, named `{pr_number}.significant.rst`, in
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #23939: Prevent UI from crashing if grid task instances are null
github-actions[bot] commented on PR #23939: URL: https://github.com/apache/airflow/pull/23939#issuecomment-1138670292 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23940: Fix broken main
potiuk commented on PR #23940: URL: https://github.com/apache/airflow/pull/23940#issuecomment-1138670683 🤞 ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23940: Fix broken main
potiuk commented on PR #23940: URL: https://github.com/apache/airflow/pull/23940#issuecomment-1138671877 At the very least there will be LESS errors. I believe there could be two breaking dependencies breaking current main. Luckily it only impacts PRs that change setup.*. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[airflow-site] branch flip_wording created (now a885299ec)
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch flip_wording in repository https://gitbox.apache.org/repos/asf/airflow-site.git at a885299ec Change "Changelog" to "Release Notes" for 2.3 This branch includes the following new commits: new a885299ec Change "Changelog" to "Release Notes" for 2.3 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[airflow-site] 01/01: Change "Changelog" to "Release Notes" for 2.3
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch flip_wording in repository https://gitbox.apache.org/repos/asf/airflow-site.git commit a885299ec7290a5911d124b2ae50d55809fe517e Author: Jed Cunningham AuthorDate: Thu May 26 09:14:50 2022 -0600 Change "Changelog" to "Release Notes" for 2.3 --- landing-pages/site/content/en/announcements/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/landing-pages/site/content/en/announcements/_index.md b/landing-pages/site/content/en/announcements/_index.md index 0a2a7172f..b51f298b2 100644 --- a/landing-pages/site/content/en/announcements/_index.md +++ b/landing-pages/site/content/en/announcements/_index.md @@ -19,7 +19,7 @@ We’ve just released Apache **Airflow 2.3.1**. 📦 PyPI: https://pypi.org/project/apache-airflow/2.3.1/ \ 📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.3.1 \ -🛠️ Changelog: https://airflow.apache.org/docs/apache-airflow/2.3.1/release_notes.html \ +🛠️ Release Notes: https://airflow.apache.org/docs/apache-airflow/2.3.1/release_notes.html \ 🪶 Sources: https://airflow.apache.org/docs/apache-airflow/2.3.1/installation/installing-from-sources.html # May 20, 2022 @@ -37,7 +37,7 @@ We’ve just released Apache **Airflow 2.3.0**. You can read more in the [What's 📦 PyPI: https://pypi.org/project/apache-airflow/2.3.0/ \ 📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.3.0 \ -🛠️ Changelog: https://airflow.apache.org/docs/apache-airflow/2.3.0/release_notes.html \ +🛠️ Release Notes: https://airflow.apache.org/docs/apache-airflow/2.3.0/release_notes.html \ 🪶 Sources: https://airflow.apache.org/docs/apache-airflow/2.3.0/installation/installing-from-sources.html # April 04, 2022
[GitHub] [airflow-site] jedcunningham opened a new pull request, #604: Change "Changelog" to "Release Notes" for 2.3
jedcunningham opened a new pull request, #604: URL: https://github.com/apache/airflow-site/pull/604 Sorry, I didn't notice this earlier (and had a pending suggestion on the last PR 🤦♂️). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23940: Fix broken main
potiuk commented on PR #23940: URL: https://github.com/apache/airflow/pull/23940#issuecomment-1138695126 Yeah. As I thought. There is a static check failing with new cosmos library 4.3.0 released 2 days ago. But this one should be easy to fix. Let me take that one :) https://pypi.org/project/azure-cosmos/#history -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] pierrejeambrun commented on pull request #23934: Fix grid details button truncated and legend row wrapping
pierrejeambrun commented on PR #23934: URL: https://github.com/apache/airflow/pull/23934#issuecomment-1138697001 @bbovenzi thank you for the quick review. I took your suggestion and moved all the actions to a common unique row. Let me know if you still see issues with the margin. It looks like this:   I also fixed a small alignement issue on the AutoRefresh component: before  after  Let me know what you think, Best :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk opened a new pull request, #23941: Add typing to Azure Cosmos Client Hook
potiuk opened a new pull request, #23941:
URL: https://github.com/apache/airflow/pull/23941
New release of Azure Cosmos library has added typing information
and it broke main builds with mypy verification.
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
for more information.
In case of fundamental code change, Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragement file, named `{pr_number}.significant.rst`, in
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] jedcunningham commented on a diff in pull request #23939: Prevent UI from crashing if grid task instances are null
jedcunningham commented on code in PR #23939:
URL: https://github.com/apache/airflow/pull/23939#discussion_r882799076
##
airflow/www/static/js/grid/renderTaskRows.test.jsx:
##
@@ -73,32 +73,43 @@ const mockGroup = {
instances: [],
};
+const nullMockGroup = {
+ id: null,
+ label: null,
+ children: [
+{
+ extraLinks: [],
+ id: 'group_1',
+ label: 'group_1',
+ instances: [null],
+ children: [
+{
+ id: 'group_1.task_1',
+ label: 'group_1.task_1',
+ extraLinks: [],
+ instances: [null],
+},
+ ],
+},
+ ],
+ instances: [null],
+};
+
describe('Test renderTaskRows', () => {
+ test('Renders name and task instance', () => {
+const task = mockGroup;
+
+const { queryByTestId, getByText } = render(
+ <>{renderTaskRows({ task, dagRunIds: ['run1'] })},
+ { wrapper: TableWrapper },
+);
+
+expect(getByText('group_1')).toBeInTheDocument();
+expect(queryByTestId('task-instance')).toBeDefined();
+expect(queryByTestId('blank-task')).toBeNull();
+ });
+
test('Still renders names if there are no instances', () => {
-global.gridData = {
- groups: {
-id: null,
-label: null,
-children: [
- {
-extraLinks: [],
-id: 'group_1',
-label: 'group_1',
-instances: [],
-children: [
- {
-id: 'group_1.task_1',
-label: 'group_1.task_1',
-extraLinks: [],
-instances: [],
- },
-],
- },
-],
-instances: [],
- },
- dagRuns: [],
-};
Review Comment:
If we remove this chunk, we aren't actually testing with `instances=[]` any
longer, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23927: fix: Databricks Provider Dependency #23917
potiuk commented on PR #23927: URL: https://github.com/apache/airflow/pull/23927#issuecomment-1138700453 And fix for static check failure here: https://github.com/apache/airflow/pull/23941 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23940: Fix broken main
potiuk commented on PR #23940: URL: https://github.com/apache/airflow/pull/23940#issuecomment-1138700850 Fix in https://github.com/apache/airflow/pull/23941 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] bbovenzi commented on a diff in pull request #23939: Prevent UI from crashing if grid task instances are null
bbovenzi commented on code in PR #23939:
URL: https://github.com/apache/airflow/pull/23939#discussion_r882803462
##
airflow/www/static/js/grid/renderTaskRows.test.jsx:
##
@@ -73,32 +73,43 @@ const mockGroup = {
instances: [],
};
+const nullMockGroup = {
+ id: null,
+ label: null,
+ children: [
+{
+ extraLinks: [],
+ id: 'group_1',
+ label: 'group_1',
+ instances: [null],
+ children: [
+{
+ id: 'group_1.task_1',
+ label: 'group_1.task_1',
+ extraLinks: [],
+ instances: [null],
+},
+ ],
+},
+ ],
+ instances: [null],
+};
+
describe('Test renderTaskRows', () => {
+ test('Renders name and task instance', () => {
+const task = mockGroup;
+
+const { queryByTestId, getByText } = render(
+ <>{renderTaskRows({ task, dagRunIds: ['run1'] })},
+ { wrapper: TableWrapper },
+);
+
+expect(getByText('group_1')).toBeInTheDocument();
+expect(queryByTestId('task-instance')).toBeDefined();
+expect(queryByTestId('blank-task')).toBeNull();
+ });
+
test('Still renders names if there are no instances', () => {
-global.gridData = {
- groups: {
-id: null,
-label: null,
-children: [
- {
-extraLinks: [],
-id: 'group_1',
-label: 'group_1',
-instances: [],
-children: [
- {
-id: 'group_1.task_1',
-label: 'group_1.task_1',
-extraLinks: [],
-instances: [],
- },
-],
- },
-],
-instances: [],
- },
- dagRuns: [],
-};
Review Comment:
oh yes, fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23940: Fix broken main
potiuk commented on PR #23940: URL: https://github.com/apache/airflow/pull/23940#issuecomment-1138706094 Looks good. Tests are passing. Merging. Once the static checks passes in #23941 we can merge that one too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #23940: Fix broken main
potiuk merged PR #23940: URL: https://github.com/apache/airflow/pull/23940 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #23941: Add typing to Azure Cosmos Client Hook
github-actions[bot] commented on PR #23941: URL: https://github.com/apache/airflow/pull/23941#issuecomment-1138705977 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[airflow] branch main updated (bfe51ea64a -> 86d40a95c4)
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from bfe51ea64a Clarify manual merging of PR in release doc (#23928) add 86d40a95c4 Fix broken main (#23940) No new revisions were added by this update. Summary of changes: setup.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
[GitHub] [airflow] bbovenzi commented on issue #23933: Failed to load Grid view when adding tasks to already exist dag
bbovenzi commented on issue #23933: URL: https://github.com/apache/airflow/issues/23933#issuecomment-1138710901 Duplicate of https://github.com/apache/airflow/discussions/23908 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] bbovenzi closed issue #23933: Failed to load Grid view when adding tasks to already exist dag
bbovenzi closed issue #23933: Failed to load Grid view when adding tasks to already exist dag URL: https://github.com/apache/airflow/issues/23933 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #23938: Update INSTALL_PROVIDERS_FROM_SOURCES instructions.
potiuk merged PR #23938: URL: https://github.com/apache/airflow/pull/23938 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[airflow] branch main updated (86d40a95c4 -> f7c4804ecf)
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 86d40a95c4 Fix broken main (#23940) add f7c4804ecf Update INSTALL_PROVIDERS_FROM_SOURCES instructions. (#23938) No new revisions were added by this update. Summary of changes: CONTRIBUTING.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[GitHub] [airflow-site] potiuk merged pull request #604: Change "Changelog" to "Release Notes" for 2.3
potiuk merged PR #604: URL: https://github.com/apache/airflow-site/pull/604 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[airflow-site] branch main updated: Change "Changelog" to "Release Notes" for 2.3 (#604)
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/main by this push: new 24f5b053e Change "Changelog" to "Release Notes" for 2.3 (#604) 24f5b053e is described below commit 24f5b053ecfeb5d20e69731bd30f2a0be946a911 Author: Jed Cunningham <66968678+jedcunning...@users.noreply.github.com> AuthorDate: Thu May 26 09:48:58 2022 -0600 Change "Changelog" to "Release Notes" for 2.3 (#604) --- landing-pages/site/content/en/announcements/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/landing-pages/site/content/en/announcements/_index.md b/landing-pages/site/content/en/announcements/_index.md index 0a2a7172f..b51f298b2 100644 --- a/landing-pages/site/content/en/announcements/_index.md +++ b/landing-pages/site/content/en/announcements/_index.md @@ -19,7 +19,7 @@ We’ve just released Apache **Airflow 2.3.1**. 📦 PyPI: https://pypi.org/project/apache-airflow/2.3.1/ \ 📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.3.1 \ -🛠️ Changelog: https://airflow.apache.org/docs/apache-airflow/2.3.1/release_notes.html \ +🛠️ Release Notes: https://airflow.apache.org/docs/apache-airflow/2.3.1/release_notes.html \ 🪶 Sources: https://airflow.apache.org/docs/apache-airflow/2.3.1/installation/installing-from-sources.html # May 20, 2022 @@ -37,7 +37,7 @@ We’ve just released Apache **Airflow 2.3.0**. You can read more in the [What's 📦 PyPI: https://pypi.org/project/apache-airflow/2.3.0/ \ 📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.3.0 \ -🛠️ Changelog: https://airflow.apache.org/docs/apache-airflow/2.3.0/release_notes.html \ +🛠️ Release Notes: https://airflow.apache.org/docs/apache-airflow/2.3.0/release_notes.html \ 🪶 Sources: https://airflow.apache.org/docs/apache-airflow/2.3.0/installation/installing-from-sources.html # April 04, 2022
[GitHub] [airflow] bbovenzi commented on pull request #23934: Fix grid details button truncated and legend row wrapping
bbovenzi commented on PR #23934: URL: https://github.com/apache/airflow/pull/23934#issuecomment-1138723523 This makes the left grid side a bit compact if you're filtering upstream on a DAG with task groups. I'm in favor of changing the "Hide details panel" button to an icon, moving the text to the button's title, and setting a min-width to the grid component. https://user-images.githubusercontent.com/4600967/170524612-b38f3eca-7edf-4365-95db-07a183d202ab.png";> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #23928: Clarify manual merging of PR in release doc
potiuk commented on code in PR #23928: URL: https://github.com/apache/airflow/pull/23928#discussion_r882824735 ## dev/README_RELEASE_AIRFLOW.md: ## @@ -224,7 +224,12 @@ The Release Candidate artifacts we vote upon should be the exact ones we vote ag - Update the `REVISION_HEADS_MAP` at airflow/utils/db.py to include the revision head of the release even if there are no migrations. - Commit the version change. -- PR from the 'test' branch to the 'stable' branch, and manually merge it once approved. +- PR from the 'test' branch to the 'stable' branch, and manually merge it once approved. Here's how to manually merge the PR: Review Comment: I think it should be after checking out stable actually @ephraimbuddy -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[airflow-site] branch gh-pages updated: Deploying to gh-pages from @ 24f5b053ecfeb5d20e69731bd30f2a0be946a911 🚀
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/gh-pages by this push: new 07bebae92 Deploying to gh-pages from @ 24f5b053ecfeb5d20e69731bd30f2a0be946a911 🚀 07bebae92 is described below commit 07bebae92c8c9af77b100c0552f0228eafe311d7 Author: potiuk AuthorDate: Thu May 26 15:57:07 2022 + Deploying to gh-pages from @ 24f5b053ecfeb5d20e69731bd30f2a0be946a911 🚀 --- announcements/index.html | 4 +- blog/airflow-1.10.10/index.html| 4 +- blog/airflow-1.10.12/index.html| 4 +- blog/airflow-1.10.8-1.10.9/index.html | 4 +- blog/airflow-2.2.0/index.html | 4 +- blog/airflow-2.3.0/index.html | 4 +- blog/airflow-survey-2020/index.html| 4 +- blog/airflow-survey/index.html | 4 +- blog/airflow-two-point-oh-is-here/index.html | 4 +- blog/airflow_summit_2021/index.html| 4 +- blog/airflow_summit_2022/index.html| 4 +- blog/announcing-new-website/index.html | 4 +- blog/apache-airflow-for-newcomers/index.html | 4 +- .../index.html | 4 +- .../index.html | 4 +- .../index.html | 4 +- .../index.html | 4 +- .../index.html | 4 +- .../index.html | 4 +- index.html | 32 search/index.html | 4 +- sitemap.xml| 96 +++--- use-cases/adobe/index.html | 4 +- use-cases/adyen/index.html | 4 +- use-cases/big-fish-games/index.html| 4 +- use-cases/dish/index.html | 4 +- use-cases/experity/index.html | 4 +- use-cases/onefootball/index.html | 4 +- use-cases/plarium-krasnodar/index.html | 4 +- use-cases/seniorlink/index.html| 4 +- use-cases/sift/index.html | 4 +- 31 files changed, 122 insertions(+), 122 deletions(-) diff --git a/announcements/index.html b/announcements/index.html index 5ce19a133..3bbefd89f 100644 --- a/announcements/index.html +++ b/announcements/index.html @@ -470,7 +470,7 @@ 📦 PyPI: https://pypi.org/project/apache-airflow/2.3.1/"; target="_blank">https://pypi.org/project/apache-airflow/2.3.1/ 📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.3.1"; target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.1 -🛠️ Changelog: https://airflow.apache.org/docs/apache-airflow/2.3.1/release_notes.html"; target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.1/release_notes.html +🛠️ Release Notes: https://airflow.apache.org/docs/apache-airflow/2.3.1/release_notes.html"; target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.1/release_notes.html 🪶 Sources: https://airflow.apache.org/docs/apache-airflow/2.3.1/installation/installing-from-sources.html"; target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.1/installation/installing-from-sources.html May 20, 2022 @@ -488,7 +488,7 @@ 📦 PyPI: https://pypi.org/project/apache-airflow/2.3.0/"; target="_blank">https://pypi.org/project/apache-airflow/2.3.0/ 📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.3.0"; target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.0 -🛠️ Changelog: https://airflow.apache.org/docs/apache-airflow/2.3.0/release_notes.html"; target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.0/release_notes.html +🛠️ Release Notes: https://airflow.apache.org/docs/apache-airflow/2.3.0/release_notes.html"; target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.0/release_notes.html 🪶 Sources: https://airflow.apache.org/docs/apache-airflow/2.3.0/installation/installing-from-sources.html"; target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.0/installation/installing-from-sources.html April 04, 2022 diff --git a/blog/airflow-1.10.10/index.html b/blog/airflow-1.10.10/index.html index a6fc0791f..c86f3df0a 100644 --- a/blog/airflow-1.10.10/index.html +++ b/blog/airflow-1.10.10/index.html @@ -36,13 +36,13 @@ - + - + diff --git a/blog/airflow-1.10.12/index.html b/blog/airflow-1.10.12/index.html index 098c88a00..c5000f5b9 100644 --- a/blog/airflow-1.10.12/index.html +++ b/blog/airflow-1.10.12/index.html @@ -36,13 +36,13 @@ - + - + diff --git a/blog/airflow-1.10.8-1.10.9/index.html b/blog/airflow-1.10.8-1.10.9/ind
[GitHub] [airflow] bbovenzi closed issue #23273: Test Connection button in connections
bbovenzi closed issue #23273: Test Connection button in connections URL: https://github.com/apache/airflow/issues/23273 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] ZouhirSarrar opened a new issue, #23942: Trigger_rule don't work while using task decorator
ZouhirSarrar opened a new issue, #23942:
URL: https://github.com/apache/airflow/issues/23942
### Apache Airflow version
2.3.0
### What happened
Hello!
I'm using taskflow api for my dag and was trying to run the last task even
if the previous task failed, I used `@task(trigger_rule=TriggerRule.ALL_DONE)`
but i still got the ending task failed too.
https://user-images.githubusercontent.com/38496723/170526480-131e6ed4-e2a0-4711-9328-2326f94885e3.png";>
The UI confirms that I declared the trigger_rule "all_done"
https://user-images.githubusercontent.com/38496723/170526766-72ba0eb7-2525-4d60-a8c3-ab3dd700d88b.png";>
### What you think should happen instead
_No response_
### How to reproduce
This is the whole code
`import logging
import pendulum
from airflow.decorators import task, dag
from airflow.utils.trigger_rule import TriggerRule
@dag(
schedule_interval=None,
start_date=pendulum.datetime(2021, 5, 10, tz="UTC"),
catchup=False,
)
def taskflow_trigger():
@task()
def first_task():
return "first_task"
@task
def second_task(value):
return "second_task"
@task
def task_to_fail(value):
data = {"test": 1}
val = data["not_here"]
return val
@task(trigger_rule=TriggerRule.ALL_DONE)
def end(value):
return "yes"
op_1 = first_task()
op_3 = task_to_fail(op_1)
op_2 = second_task(op_1)
end([op_3, op_2])
taskflow_trigger = taskflow_trigger()
`
### Operating System
"Debian GNU/Linux 11 (bullseye)
### Versions of Apache Airflow Providers
_No response_
### Deployment
Docker-Compose
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #23942: Trigger_rule don't work while using task decorator
boring-cyborg[bot] commented on issue #23942: URL: https://github.com/apache/airflow/issues/23942#issuecomment-1138732941 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk opened a new pull request, #23943: Disable rebase workflow
potiuk opened a new pull request, #23943:
URL: https://github.com/apache/airflow/pull/23943
The change of the release workflow in #23928 removed the reason
why we should have rebase workflow possible. We only needed to
do rebase when we merged test branch into stable branch and
since we are doing it manually, there is no more reeason to
have it in the GitHub UI.
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
for more information.
In case of fundamental code change, Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragement file, named `{pr_number}.significant.rst`, in
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #23928: Clarify manual merging of PR in release doc
potiuk commented on code in PR #23928: URL: https://github.com/apache/airflow/pull/23928#discussion_r882836130 ## dev/README_RELEASE_AIRFLOW.md: ## @@ -224,7 +224,12 @@ The Release Candidate artifacts we vote upon should be the exact ones we vote ag - Update the `REVISION_HEADS_MAP` at airflow/utils/db.py to include the revision head of the release even if there are no migrations. - Commit the version change. -- PR from the 'test' branch to the 'stable' branch, and manually merge it once approved. +- PR from the 'test' branch to the 'stable' branch, and manually merge it once approved. Here's how to manually merge the PR: Review Comment: I fixed it in https://github.com/apache/airflow/pull/23943 together with disabling rebase workflow from GitHub UI as it is dangerous to have it enabled (and the only reason I believe we had it was to merge test into stable). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #23941: Add typing to Azure Cosmos Client Hook
potiuk merged PR #23941: URL: https://github.com/apache/airflow/pull/23941 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[airflow] branch main updated (f7c4804ecf -> 3393647aa6)
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from f7c4804ecf Update INSTALL_PROVIDERS_FROM_SOURCES instructions. (#23938) add 3393647aa6 Add typing to Azure Cosmos Client Hook (#23941) No new revisions were added by this update. Summary of changes: airflow/providers/microsoft/azure/hooks/cosmos.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[GitHub] [airflow] potiuk commented on pull request #23941: Add typing to Azure Cosmos Client Hook
potiuk commented on PR #23941: URL: https://github.com/apache/airflow/pull/23941#issuecomment-1138736552 Looks good. Merging! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23927: fix: Databricks Provider Dependency #23917
potiuk commented on PR #23927: URL: https://github.com/apache/airflow/pull/23927#issuecomment-1138736998 Rebased it to re-run the build after both fixes are merged :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #23886: Remove redundant register exit signals in `dag-processor` command
potiuk merged PR #23886: URL: https://github.com/apache/airflow/pull/23886 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[airflow] branch main updated (3393647aa6 -> db890df38f)
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 3393647aa6 Add typing to Azure Cosmos Client Hook (#23941) add db890df38f Remove redundant register exit signals in `dag-processor` command (#23886) No new revisions were added by this update. Summary of changes: airflow/cli/commands/dag_processor_command.py | 2 -- 1 file changed, 2 deletions(-)
[GitHub] [airflow] github-actions[bot] commented on pull request #23943: Disable rebase workflow
github-actions[bot] commented on PR #23943: URL: https://github.com/apache/airflow/pull/23943#issuecomment-1138738094 The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide that full test matrix is needed and add the 'full tests needed' label. Then you should rebase it to the latest main or amend the last commit of the PR, and push it with --force-with-lease. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
