Re: [I] UI: New trigger DAG with config form doesn’t allow valid floats and empty strings [airflow]

2024-07-29 Thread via GitHub
github-actions[bot] commented on issue #31930: URL: https://github.com/apache/airflow/issues/31930#issuecomment-2255093221 This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activit

Re: [I] Auto-refresh turn off before updating status of MAPPED tasks [airflow]

2024-07-29 Thread via GitHub
github-actions[bot] commented on issue #31870: URL: https://github.com/apache/airflow/issues/31870#issuecomment-2255093256 This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activit

Re: [I] "trigger end" being spuriously logged from triggerer_job_runner at level 100 [airflow]

2024-07-29 Thread via GitHub
github-actions[bot] commented on issue #31799: URL: https://github.com/apache/airflow/issues/31799#issuecomment-2255093293 This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activit

Re: [I] Status of testing Providers that were prepared on July 28, 2024 [airflow]

2024-07-29 Thread via GitHub
JDarDagran commented on issue #41080: URL: https://github.com/apache/airflow/issues/41080#issuecomment-2255114497 https://github.com/apache/airflow/pull/39530 adds `apache-airflow-providers-common-compat>=1.1.0` dependency to amazon and google providers. You can't install new packages witho

[PR] Improve logging in _register_dataset_changes [airflow]

2024-07-29 Thread via GitHub
uranusjr opened a new pull request, #41089: URL: https://github.com/apache/airflow/pull/41089 Logging should be smart enough to render rich objects directly, so we don't need to format the object string ourselves. This makes the log messages a little longer, but should be be acceptable.

Re: [PR] Remove Database Isolation feature flag and run DB isolation tests [airflow]

2024-07-29 Thread via GitHub
aritra24 commented on PR #41067: URL: https://github.com/apache/airflow/pull/41067#issuecomment-2255235803 I'm looking at some of the test_python failures, should have something by EoD -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] Task with "wait_for_downstream" doesn't wait, because start_date is mishandled [airflow]

2024-07-29 Thread via GitHub
omkar-foss commented on issue #40841: URL: https://github.com/apache/airflow/issues/40841#issuecomment-2255245212 Hey @shai-ikko, good to hear from you. Apologies on my delayed response here. So in [this PR](https://github.com/apache/airflow/pull/40963), the check I've added - to chec

(airflow) branch main updated: Improve logging in _register_dataset_changes (#41089)

2024-07-29 Thread weilee
This is an automated email from the ASF dual-hosted git repository. weilee pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new c0cc614b2d Improve logging in _register_dataset_cha

Re: [PR] Improve logging in _register_dataset_changes [airflow]

2024-07-29 Thread via GitHub
Lee-W merged PR #41089: URL: https://github.com/apache/airflow/pull/41089 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.ap

Re: [I] KubernetesExecutor does not save logs [airflow]

2024-07-29 Thread via GitHub
omkar-foss commented on issue #40861: URL: https://github.com/apache/airflow/issues/40861#issuecomment-2255301921 >At the same time, Celery Executor saves logs both in persistence storage and in s3 This issue could be similar to (but not the same) https://github.com/apache/airflow/is

Re: [I] Status of testing Providers that were prepared on July 28, 2024 [airflow]

2024-07-29 Thread via GitHub
potiuk commented on issue #41080: URL: https://github.com/apache/airflow/issues/41080#issuecomment-2255360235 > https://github.com/apache/airflow/pull/39530 adds apache-airflow-providers-common-compat>=1.1.0 dependency to amazon and google providers. You can't install new packages without

Re: [PR] Remove Database Isolation feature flag and run DB isolation tests [airflow]

2024-07-29 Thread via GitHub
potiuk commented on PR #41067: URL: https://github.com/apache/airflow/pull/41067#issuecomment-2255376446 I am **very little** available today - but if somoene wants to take a stab at some of those - feel free, I will try to look them up. For now **some** tests are hanging - I will have to t

Re: [PR] Microsoft Power BI operator to refresh the dataset [airflow]

2024-07-29 Thread via GitHub
ambika-garg commented on PR #40356: URL: https://github.com/apache/airflow/pull/40356#issuecomment-2255388782 I added some more tests, to ensure full code coverage, I'm pasting the codecov html report below. https://github.com/user-attachments/assets/f03ba7be-edc6-45d7-952f-e1eba4210038"

[PR] set "has_outlet_datasets" to true if "dataset alias" exists [airflow]

2024-07-29 Thread via GitHub
Lee-W opened a new pull request, #41091: URL: https://github.com/apache/airflow/pull/41091 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull

[PR] Fix cloudsql-query system tests [airflow]

2024-07-29 Thread via GitHub
moiseenkov opened a new pull request, #41092: URL: https://github.com/apache/airflow/pull/41092 - Fix system tests for CloudSQL - Fix CloudSQLDatabaseHook -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[I] Add `execute_tasks_new_python_interpreter` argument to `PythonOperator` [airflow]

2024-07-29 Thread via GitHub
pedro-cf opened a new issue, #41093: URL: https://github.com/apache/airflow/issues/41093 ### Description Currently from my knowledge it is only possible to force all python tasks to execute in a new interpreter globally via this setting: - https://airflow.apache.org/docs/

Re: [PR] Make HookLineageCollector group datasets [airflow]

2024-07-29 Thread via GitHub
mobuchowski merged PR #41034: URL: https://github.com/apache/airflow/pull/41034 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airf

(airflow) branch main updated: Make HookLineageCollector group datasets by (#41034)

2024-07-29 Thread mobuchowski
This is an automated email from the ASF dual-hosted git repository. mobuchowski pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 072f3b95ad Make HookLineageCollector group dat

Re: [PR] set "has_outlet_datasets" to true if "dataset alias" exists [airflow]

2024-07-29 Thread via GitHub
Lee-W commented on PR #41091: URL: https://github.com/apache/airflow/pull/41091#issuecomment-2255541106 It's also possible a DAG with `DatasetAlias` doesn't create a dataset event. The UI will show https://github.com/user-attachments/assets/efa72fda-98c0-4ee1-96ac-e8e383cf47d9";>

Re: [PR] Fix MRO in operators without `__init__` [airflow]

2024-07-29 Thread via GitHub
grihabor commented on PR #41086: URL: https://github.com/apache/airflow/pull/41086#issuecomment-2255584797 Sure. Is this the correct place to put the test? https://github.com/apache/airflow/pull/41086/commits/4aa7491b8df517d8fa10ad5a73a42ff5f6173b78 I added the example I encountered i

Re: [I] SnowflakeSqlApiOperator's Snowflake Legacy Url Authentication Issue [airflow]

2024-07-29 Thread via GitHub
shubham-arora-18 closed issue #40885: SnowflakeSqlApiOperator's Snowflake Legacy Url Authentication Issue URL: https://github.com/apache/airflow/issues/40885 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [I] SnowflakeSqlApiOperator's Snowflake Legacy Url Authentication Issue [airflow]

2024-07-29 Thread via GitHub
shubham-arora-18 commented on issue #40885: URL: https://github.com/apache/airflow/issues/40885#issuecomment-2255655913 Yes, I just checked it and it is fixed. Closing the issue, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [I] Status of testing Providers that were prepared on July 28, 2024 [airflow]

2024-07-29 Thread via GitHub
gopidesupavan commented on issue #41080: URL: https://github.com/apache/airflow/issues/41080#issuecomment-2255689553 Yes it is failing to install package. https://github.com/user-attachments/assets/d71fd653-127e-4d9d-b754-51fc489f9b75";> Once it updated i will test these #40976 , #4

Re: [PR] mypy version bump 1.11.0 [airflow]

2024-07-29 Thread via GitHub
dirrao closed pull request #39850: mypy version bump 1.11.0 URL: https://github.com/apache/airflow/pull/39850 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

[PR] mypy version bump 1.11.0 [airflow]

2024-07-29 Thread via GitHub
dirrao opened a new pull request, #39850: URL: https://github.com/apache/airflow/pull/39850 mypy version bump 1.11.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

[PR] openlineage: adjust DefaultExtractor's on_failure detection for airflow 2.10 fix [airflow]

2024-07-29 Thread via GitHub
kacpermuda opened a new pull request, #41094: URL: https://github.com/apache/airflow/pull/41094 A followup to #41053, adjusting OpenLineage provider to work well after we change the place of the listener hook manager being called. Should only be merged after #41053 is

Re: [I] Status of testing Providers that were prepared on July 28, 2024 [airflow]

2024-07-29 Thread via GitHub
dirrao commented on issue #41080: URL: https://github.com/apache/airflow/issues/41080#issuecomment-2255804496 [#41020)](https://github.com/apache/airflow/pull/41020): [@dirrao](https://github.com/dirrao) working as expected. -- This is an automated message from the Apache Git Service. To

Re: [I] Kubernetes Executor leaving pods in a "Running" state after a task is completed [airflow]

2024-07-29 Thread via GitHub
ahipp13 commented on issue #40957: URL: https://github.com/apache/airflow/issues/40957#issuecomment-2255846755 To update, I went to just one scheduler and it still didn't fix the issue.. Anybody have any debugging tips on what logs I should look at to see why it is sometimes not updating th

[PR] Add dataset alias unique constraint and remove wrong dataset alias removing logic [airflow]

2024-07-29 Thread via GitHub
Lee-W opened a new pull request, #41097: URL: https://github.com/apache/airflow/pull/41097 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull

Re: [PR] Airflow 3: processor_poll_interval deprecated parameter usage removal [airflow]

2024-07-29 Thread via GitHub
kaxil commented on PR #41096: URL: https://github.com/apache/airflow/pull/41096#issuecomment-2255973036 Airflow main branch is still for Airflow 2.10 :) Let's wait until the 2.10.0rc1 is created before we start removing deprecations -- This is an automated message from the Apache Git Serv

svn commit: r70604 - in /release/airflow/clients/python: 2.9.0/ 2.9.1/

2024-07-29 Thread ephraimanierobi
Author: ephraimanierobi Date: Mon Jul 29 13:52:38 2024 New Revision: 70604 Log: Release Apache Airflow Python Client 2.9.1 from 2.9.1rc1 Added: release/airflow/clients/python/2.9.1/ release/airflow/clients/python/2.9.1/apache_airflow_client-2.9.1-py3-none-any.whl - copied unchanged

[I] XCom is cleared when a task resumes from deferral [airflow]

2024-07-29 Thread via GitHub
Sanketp1997 opened a new issue, #41098: URL: https://github.com/apache/airflow/issues/41098 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.8.4+astro.3 ### What happened? A tas

Re: [I] Task with "wait_for_downstream" doesn't wait, because start_date is mishandled [airflow]

2024-07-29 Thread via GitHub
shai-ikko commented on issue #40841: URL: https://github.com/apache/airflow/issues/40841#issuecomment-2256026832 Hi, I've looked a little more into documentation and code. Since I think the `start_date`/`execution_date` comparison is utterly wrong, I've tried just commenting it out. I

(airflow) annotated tag python-client-2.9.1 updated (9cb7c47ff2 -> 37ee0fb2bc)

2024-07-29 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to annotated tag python-client-2.9.1 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag python-client-2.9.1 was modified! *** from 9cb7c47ff2 (commit) to 37e

(airflow-client-python) annotated tag 2.9.1 updated (35ee071 -> 440ef3e)

2024-07-29 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to annotated tag 2.9.1 in repository https://gitbox.apache.org/repos/asf/airflow-client-python.git *** WARNING: tag 2.9.1 was modified! *** from 35ee071 (commit) to 440ef3e (tag) tag

(airflow) branch constraints-main updated: Updating constraints. Github run id:10145290446

2024-07-29 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 990cbbb6b8 Updating con

[PR] Task outlet dataset alias reference [airflow]

2024-07-29 Thread via GitHub
Lee-W opened a new pull request, #41099: URL: https://github.com/apache/airflow/pull/41099 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull

[PR] Minor updates to the release guides [airflow]

2024-07-29 Thread via GitHub
ephraimbuddy opened a new pull request, #41100: URL: https://github.com/apache/airflow/pull/41100 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [PR] Minor updates to the release guides [airflow]

2024-07-29 Thread via GitHub
ephraimbuddy commented on code in PR #41100: URL: https://github.com/apache/airflow/pull/41100#discussion_r1695340371 ## dev/README_RELEASE_PYTHON_CLIENT.md: ## @@ -561,10 +564,10 @@ done # Remove old release cd .. svn rm ${PREVIOUS_VERSION} -svn commit -m "Release Apache Air

Re: [PR] Minor updates to the release guides [airflow]

2024-07-29 Thread via GitHub
ephraimbuddy commented on code in PR #41100: URL: https://github.com/apache/airflow/pull/41100#discussion_r1695342822 ## dev/breeze/src/airflow_breeze/commands/minor_release_command.py: ## @@ -69,7 +69,7 @@ def commit_changes(version_branch): if DRY_RUN: co

Re: [PR] Add dataset alias unique constraint and remove wrong dataset alias removing logic [airflow]

2024-07-29 Thread via GitHub
ephraimbuddy commented on code in PR #41097: URL: https://github.com/apache/airflow/pull/41097#discussion_r1695359509 ## airflow/migrations/versions/0151_2_10_0_dataset_alias_unique_constraint.py: ## @@ -0,0 +1,49 @@ +# +# Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] Remove Database Isolation feature flag and run DB isolation tests [airflow]

2024-07-29 Thread via GitHub
vincbeck commented on PR #41067: URL: https://github.com/apache/airflow/pull/41067#issuecomment-2256129598 I'll try to pick some tests when I can, possibly around the end of the week -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] Add dataset alias unique constraint and remove wrong dataset alias removing logic [airflow]

2024-07-29 Thread via GitHub
Lee-W commented on code in PR #41097: URL: https://github.com/apache/airflow/pull/41097#discussion_r1695383138 ## airflow/migrations/versions/0151_2_10_0_dataset_alias_unique_constraint.py: ## @@ -0,0 +1,49 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# o

Re: [I] Tasks are stuck in scheduled state when AIRFLOW__CORE__PARALLELISM env is set to 0 in Kubernetes Executor [airflow]

2024-07-29 Thread via GitHub
o-nikolas commented on issue #41055: URL: https://github.com/apache/airflow/issues/41055#issuecomment-2256264249 > Hi @o-nikolas , in the base executor, it's described that setting the parallelism to '0' means infinite parallelism. See : > > https://github.com/apache/airflow/blob/c0c

Re: [I] Tasks are stuck in scheduled state when AIRFLOW__CORE__PARALLELISM env is set to 0 in Kubernetes Executor [airflow]

2024-07-29 Thread via GitHub
o-nikolas commented on issue #41055: URL: https://github.com/apache/airflow/issues/41055#issuecomment-2256269128 All that too say though, I'm happy to disagree and commit :slightly_smiling_face: I can add the infinity to the scheduler code, but suppose that I'd at least ask that someone her

[I] DAG triggering without all Datasets being triggered [airflow]

2024-07-29 Thread via GitHub
slords opened a new issue, #41101: URL: https://github.com/apache/airflow/issues/41101 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.9.2 ### What happened? We have an airflow

Re: [I] DAG triggering without all Datasets being triggered [airflow]

2024-07-29 Thread via GitHub
boring-cyborg[bot] commented on issue #41101: URL: https://github.com/apache/airflow/issues/41101#issuecomment-2256299348 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [I] DAG triggering without all Datasets being triggered [airflow]

2024-07-29 Thread via GitHub
slords commented on issue #41101: URL: https://github.com/apache/airflow/issues/41101#issuecomment-2256308540 https://github.com/user-attachments/assets/b43422ed-fb3a-4c84-8947-a212236c9ff9";> https://github.com/user-attachments/assets/27094fd0-d260-42dd-a5fd-58654648644b";> https://git

Re: [PR] Minor updates to the release guides [airflow]

2024-07-29 Thread via GitHub
kaxil commented on code in PR #41100: URL: https://github.com/apache/airflow/pull/41100#discussion_r1695484216 ## dev/README_RELEASE_AIRFLOW.md: ## @@ -315,11 +315,11 @@ The Release Candidate artifacts we vote upon should be the exact ones we vote ag - Create issue in github

[PR] Check aliases when determining task.has_outlet_datasets [airflow]

2024-07-29 Thread via GitHub
bbovenzi opened a new pull request, #41102: URL: https://github.com/apache/airflow/pull/41102 Check for `DatasetAlias` outlets when determining if a task `has_dataset_outlets`. Fixes a bug where some dataset events were not rendering for a Task Instance in the UI --- **^ Add me

Re: [PR] Check aliases when determining task.has_outlet_datasets [airflow]

2024-07-29 Thread via GitHub
bbovenzi closed pull request #41102: Check aliases when determining task.has_outlet_datasets URL: https://github.com/apache/airflow/pull/41102 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] AIP-64: Add task instance history list endpoint [airflow]

2024-07-29 Thread via GitHub
ephraimbuddy merged PR #40988: URL: https://github.com/apache/airflow/pull/40988 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@air

(airflow) branch main updated: fix(view): set "has_outlet_datasets" to true if "dataset alias" exists (#41091)

2024-07-29 Thread bbovenzi
This is an automated email from the ASF dual-hosted git repository. bbovenzi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 36b92345b9 fix(view): set "has_outlet_datasets" t

Re: [PR] Check aliases when determining task.has_outlet_datasets [airflow]

2024-07-29 Thread via GitHub
bbovenzi commented on PR #41102: URL: https://github.com/apache/airflow/pull/41102#issuecomment-2256384801 Duplicate of https://github.com/apache/airflow/pull/41091 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

(airflow) branch main updated: AIP-64: Add task instance history list endpoint (#40988)

2024-07-29 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 815d1122cf AIP-64: Add task instance histo

Re: [PR] set "has_outlet_datasets" to true if "dataset alias" exists [airflow]

2024-07-29 Thread via GitHub
bbovenzi merged PR #41091: URL: https://github.com/apache/airflow/pull/41091 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

Re: [PR] Minor updates to the release guides [airflow]

2024-07-29 Thread via GitHub
ephraimbuddy commented on code in PR #41100: URL: https://github.com/apache/airflow/pull/41100#discussion_r1695543004 ## dev/README_RELEASE_AIRFLOW.md: ## @@ -315,11 +315,11 @@ The Release Candidate artifacts we vote upon should be the exact ones we vote ag - Create issue in

[PR] Fix `EmrServerlessJobSensor` [airflow]

2024-07-29 Thread via GitHub
vincbeck opened a new pull request, #41103: URL: https://github.com/apache/airflow/pull/41103 If `deferrable` is `True`, it does not mean the operator should wait. This is configured by `wait_for_completion`. --- **^ Add meaningful description above**

Re: [PR] Fix `EmrServerlessJobSensor` [airflow]

2024-07-29 Thread via GitHub
shahar1 commented on code in PR #41103: URL: https://github.com/apache/airflow/pull/41103#discussion_r1695717877 ## airflow/providers/amazon/aws/operators/emr.py: ## @@ -1382,30 +1382,30 @@ def execute(self, context: Context, event: dict[str, Any] | None = None) -> str

Re: [I] duplicate key value violates unique constraint serialized_dag_pkey for dynamic dag generation [airflow]

2024-07-29 Thread via GitHub
rafidka commented on issue #40082: URL: https://github.com/apache/airflow/issues/40082#issuecomment-2256698918 We created this PR https://github.com/aws/amazon-mwaa-docker-images/pull/125 to fix the issue in Amazon MWAA. However, I believe a fix is also needed on Airflow side. If the databa

[PR] Bugfix/41067 Fix `test_base_aws.py` [airflow]

2024-07-29 Thread via GitHub
vincbeck opened a new pull request, #41104: URL: https://github.com/apache/airflow/pull/41104 Related: #41067 Fix `test_base_aws.py` to make it work when DB isolation is on. Note, tests are still failing but for another reason. As explained in #41082, there is currently a bug in `ref

Re: [PR] Remove Database Isolation feature flag and run DB isolation tests [airflow]

2024-07-29 Thread via GitHub
vincbeck commented on PR #41067: URL: https://github.com/apache/airflow/pull/41067#issuecomment-2256761468 I tried to run `tests/datasets/test_dataset.py` and it seems all tests are passing? -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Fix `EmrServerlessStartJobOperator` [airflow]

2024-07-29 Thread via GitHub
vincbeck commented on code in PR #41103: URL: https://github.com/apache/airflow/pull/41103#discussion_r1695786320 ## airflow/providers/amazon/aws/operators/emr.py: ## @@ -1382,30 +1382,30 @@ def execute(self, context: Context, event: dict[str, Any] | None = None) -> str

(airflow) branch main updated: Fix `EmrServerlessStartJobOperator` (#41103)

2024-07-29 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 97c4fdce71 Fix `EmrServerlessStartJobOperator` (#

Re: [PR] Fix `EmrServerlessStartJobOperator` [airflow]

2024-07-29 Thread via GitHub
vincbeck merged PR #41103: URL: https://github.com/apache/airflow/pull/41103 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

Re: [I] Tasks that are running when a DAG run reaches its timeout should optionally support `on_killed` [airflow]

2024-07-29 Thread via GitHub
eladkal commented on issue #41036: URL: https://github.com/apache/airflow/issues/41036#issuecomment-2256815880 Was something changed since my comment in https://github.com/apache/airflow/issues/30264#issuecomment-1482028871 ? If not then I don't see what is the task here? If tasks mark

Re: [PR] fix tests/core/test_sentry.py for internal API tests [airflow]

2024-07-29 Thread via GitHub
jscheffl merged PR #41084: URL: https://github.com/apache/airflow/pull/41084 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

(airflow) branch main updated: fix tests/core/test_sentry.py for internal API tests (#41084)

2024-07-29 Thread jscheffl
This is an automated email from the ASF dual-hosted git repository. jscheffl pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new c07b69b676 fix tests/core/test_sentry.py for inte

Re: [I] `airflow.Param` fails to get the default value of `array`. [airflow]

2024-07-29 Thread via GitHub
jscheffl commented on issue #41019: URL: https://github.com/apache/airflow/issues/41019#issuecomment-2256834438 Can you please describe a bit more in detail what "So I can't run DAG" means? Does it not start or do you get a specific error message? What I can tell is if the array field

[PR] Bugfix/41067 Fix `base_auth_manager` [airflow]

2024-07-29 Thread via GitHub
vincbeck opened a new pull request, #41105: URL: https://github.com/apache/airflow/pull/41105 The test `test_security_manager_return_default_security_manager` is failing when DB isolation is on. I actually refactored the test to make it a non DB test instead. That fixed the issue as well.

[I] Incompatible snowflake-connector-python & keyring versions in constraints file [airflow]

2024-07-29 Thread via GitHub
erikogan opened a new issue, #41106: URL: https://github.com/apache/airflow/issues/41106 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.9.2 ### What happened? I am investigati

Re: [I] Tasks are stuck in scheduled state when AIRFLOW__CORE__PARALLELISM env is set to 0 in Kubernetes Executor [airflow]

2024-07-29 Thread via GitHub
ephraimbuddy commented on issue #41055: URL: https://github.com/apache/airflow/issues/41055#issuecomment-2257018162 > All that too say though, I'm happy to disagree and commit πŸ™‚ I can add the infinity to the scheduler code, but suppose that I'd at least ask that someone here who is arguing

[PR] Re-add the ability to set parallelism to unlimited [airflow]

2024-07-29 Thread via GitHub
o-nikolas opened a new pull request, #41107: URL: https://github.com/apache/airflow/pull/41107 This feature was/is a bit half baked and undocumented and got dropped during development of multiple executor configuration. This PR re-adds that feature, and also adds some tests and documentatio

Re: [PR] Re-add the ability to set parallelism to unlimited [airflow]

2024-07-29 Thread via GitHub
o-nikolas commented on code in PR #41107: URL: https://github.com/apache/airflow/pull/41107#discussion_r1695976549 ## airflow/executors/base_executor.py: ## @@ -288,7 +288,8 @@ def _emit_metrics(self, open_slots, num_running_tasks, num_queued_tasks): self.log.debug("%s

Re: [I] Tasks are stuck in scheduled state when AIRFLOW__CORE__PARALLELISM env is set to 0 in Kubernetes Executor [airflow]

2024-07-29 Thread via GitHub
o-nikolas commented on issue #41055: URL: https://github.com/apache/airflow/issues/41055#issuecomment-2257029091 @ephraimbuddy fair enough, just posted a PR here: #41107 :slightly_smiling_face: -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] Bad map_index in the log view of a mapped task [airflow]

2024-07-29 Thread via GitHub
atomic77 commented on issue #40655: URL: https://github.com/apache/airflow/issues/40655#issuecomment-2257029405 I was able to reproduce this locally for `example_dynamic_task_mapping` against v2.9.2; if no one else has spoken for this issue I would like to give it a try. -- This is an a

(airflow) branch main updated: Bugfix/41067 Fix `test_base_aws.py` (#41104)

2024-07-29 Thread jscheffl
This is an automated email from the ASF dual-hosted git repository. jscheffl pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 6e2634a9eb Bugfix/41067 Fix `test_base_aws.py` (#

(airflow) branch main updated: Bugfix/41067 Fix `base_auth_manager` (#41105)

2024-07-29 Thread jscheffl
This is an automated email from the ASF dual-hosted git repository. jscheffl pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 010b507f29 Bugfix/41067 Fix `base_auth_manager` (

Re: [PR] Bugfix/41067 Fix `test_base_aws.py` [airflow]

2024-07-29 Thread via GitHub
jscheffl merged PR #41104: URL: https://github.com/apache/airflow/pull/41104 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

Re: [PR] Bugfix/41067 Fix `base_auth_manager` [airflow]

2024-07-29 Thread via GitHub
jscheffl merged PR #41105: URL: https://github.com/apache/airflow/pull/41105 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

[PR] typo doc unit_tests [airflow]

2024-07-29 Thread via GitHub
raphaelauv opened a new pull request, #41108: URL: https://github.com/apache/airflow/pull/41108 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Fix BashOperator on Windows [airflow]

2024-07-29 Thread via GitHub
uranusjr commented on code in PR #41073: URL: https://github.com/apache/airflow/pull/41073#discussion_r1696015087 ## airflow/hooks/subprocess.py: ## @@ -61,41 +64,68 @@ def run_command( or stdout """ self.log.info("Tmp dir root location: %s", gette

Re: [I] Consolidate "Serialization" code [airflow]

2024-07-29 Thread via GitHub
gyli commented on issue #40974: URL: https://github.com/apache/airflow/issues/40974#issuecomment-2257204204 Yeah I can start from https://github.com/apache/airflow/issues/31213 first. Note that by doing so, internal_api_call has to temporarily call either old serializer or `serde` de

Re: [I] soft_fail | operator is skipped in all cases and not only "data" related fail [airflow]

2024-07-29 Thread via GitHub
raphaelauv commented on issue #40787: URL: https://github.com/apache/airflow/issues/40787#issuecomment-2257218781 @potiuk could you please re-open the issue , since PR was reverted , thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [I] `airflow.Param` fails to get the default value of `array`. [airflow]

2024-07-29 Thread via GitHub
phi-friday commented on issue #41019: URL: https://github.com/apache/airflow/issues/41019#issuecomment-2257221667 > Can you please describe a bit more in detail what "So I can't run DAG" means? Does it not start or do you get a specific error message? > > What I can tell is if the arr

Re: [I] `airflow.Param` fails to get the default value of `array`. [airflow]

2024-07-29 Thread via GitHub
phi-friday commented on issue #41019: URL: https://github.com/apache/airflow/issues/41019#issuecomment-2257239855 Also, in `2.8.4`, it works like this ![스크란샷 2024-07-30 α„‹α…©α„Œα…₯ᆫ 9 22 32](https://github.com/user-attachments/assets/d48a078a-f73b-4e00-acdf-bd0a1dab80ce) In `2.9.3`,

Re: [PR] Fix BashOperator on Windows [airflow]

2024-07-29 Thread via GitHub
SledgeHammer01 commented on code in PR #41073: URL: https://github.com/apache/airflow/pull/41073#discussion_r1696195558 ## airflow/hooks/subprocess.py: ## @@ -61,41 +64,68 @@ def run_command( or stdout """ self.log.info("Tmp dir root location: %s",

[PR] Fix ApiException handling when adopting completed pods [airflow]

2024-07-29 Thread via GitHub
jedcunningham opened a new pull request, #41109: URL: https://github.com/apache/airflow/pull/41109 When trying to complete adopted pods, if we encounter an ApiException, we should assume we were unable to adopt the pod meaning we shouldn't add that to our `running` set. If we do, `running`

[PR] Upgrade istallers after pip release [airflow]

2024-07-29 Thread via GitHub
potiuk opened a new pull request, #41110: URL: https://github.com/apache/airflow/pull/41110 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pul

Re: [PR] Minor updates to the release guides [airflow]

2024-07-29 Thread via GitHub
potiuk commented on code in PR #41100: URL: https://github.com/apache/airflow/pull/41100#discussion_r1696207157 ## dev/README_RELEASE_PYTHON_CLIENT.md: ## @@ -561,10 +564,10 @@ done # Remove old release cd .. svn rm ${PREVIOUS_VERSION} -svn commit -m "Release Apache Airflow P

Re: [PR] Minor updates to the release guides [airflow]

2024-07-29 Thread via GitHub
potiuk commented on code in PR #41100: URL: https://github.com/apache/airflow/pull/41100#discussion_r1696208126 ## dev/README_RELEASE_PYTHON_CLIENT.md: ## @@ -561,10 +564,10 @@ done # Remove old release cd .. svn rm ${PREVIOUS_VERSION} -svn commit -m "Release Apache Airflow P

Re: [PR] Bugfix/41067 Fix `test_base_aws.py` [airflow]

2024-07-29 Thread via GitHub
potiuk commented on PR #41104: URL: https://github.com/apache/airflow/pull/41104#issuecomment-2257373792 Nice. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [PR] Bugfix/41067 Fix `base_auth_manager` [airflow]

2024-07-29 Thread via GitHub
potiuk commented on PR #41105: URL: https://github.com/apache/airflow/pull/41105#issuecomment-2257374272 Cool. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [I] Consolidate "Serialization" code [airflow]

2024-07-29 Thread via GitHub
potiuk commented on issue #40974: URL: https://github.com/apache/airflow/issues/40974#issuecomment-2257378098 I am not sure we allow DAG or Operator to be passed over the internal-api. I **think** it should not be the case and we should likely change it. I will review it shortly as well.

Re: [I] Consolidate "Serialization" code [airflow]

2024-07-29 Thread via GitHub
gyli commented on issue #40974: URL: https://github.com/apache/airflow/issues/40974#issuecomment-2257383222 Here are some internal API endpoints I found with DAG/operator arguments: https://github.com/apache/airflow/blob/6684481c67f6a21a72e7f1512b450a433c5313b5/airflow/cli/commands/task_c

(airflow) branch main updated: Fix cloudsql-query system tests (#41092)

2024-07-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 8f616551ca Fix cloudsql-query system tests (#41092)

Re: [PR] Fix cloudsql-query system tests [airflow]

2024-07-29 Thread via GitHub
potiuk merged PR #41092: URL: https://github.com/apache/airflow/pull/41092 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [I] Add `execute_tasks_new_python_interpreter` argument to `PythonOperator` [airflow]

2024-07-29 Thread via GitHub
potiuk commented on issue #41093: URL: https://github.com/apache/airflow/issues/41093#issuecomment-2257386874 Marked it as a good first issue for anyone to take - but feel free to propose a PR as well. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] typo doc unit_tests [airflow]

2024-07-29 Thread via GitHub
jedcunningham merged PR #41108: URL: https://github.com/apache/airflow/pull/41108 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

(airflow) branch main updated: Fix typo in contributing doc about skipping unit_tests (#41108)

2024-07-29 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 8a1b84dc54 Fix typo in contributing doc abou

  1   2   >