[PR] Chart: Helm chart 1.15.0 release notes [airflow]

2024-07-19 Thread via GitHub
jedcunningham opened a new pull request, #40898: URL: https://github.com/apache/airflow/pull/40898 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Microsoft Power BI operator to refresh the dataset [airflow]

2024-07-19 Thread via GitHub
ambika-garg commented on PR #40356: URL: https://github.com/apache/airflow/pull/40356#issuecomment-2240904521 Hey @dabla, thank you so much for reviewing the PR. I deeply appreciate it! I had a quick question related to the comment you added about using more of deferrable mechanism, do

Re: [I] migrate from `dill` to `cloudpickle` for advanced serialization [airflow]

2024-07-19 Thread via GitHub
gyli commented on issue #7870: URL: https://github.com/apache/airflow/issues/7870#issuecomment-2240891259 With https://github.com/apache/airflow/pull/39270 completed, argument `serializer` was added to support choosing from serializers. Looks like this issue can be closed. -- This is an

Re: [I] Release Airflow 3.0 [airflow]

2024-07-19 Thread via GitHub
gyli commented on issue #39593: URL: https://github.com/apache/airflow/issues/39593#issuecomment-2240842745 Hi @kaxil, I would love to take Consolidate "Serialization" code, could you please create an issue for this one? -- This is an automated message from the Apache Git Service. To

Re: [PR] Format DAG run count [airflow]

2024-07-19 Thread via GitHub
github-actions[bot] commented on PR #39684: URL: https://github.com/apache/airflow/pull/39684#issuecomment-2240787628 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [PR] fixes serialize dag problem when `TriggerDagRunOperator` has XComArg value for `trigger_dag_id` argument [airflow]

2024-07-19 Thread via GitHub
nyoungstudios commented on code in PR #39772: URL: https://github.com/apache/airflow/pull/39772#discussion_r1685164164 ## airflow/models/xcom_arg.py: ## @@ -255,6 +255,9 @@ def __getitem__(self, item: str) -> XComArg: raise ValueError(f"XComArg only supports str

Re: [PR] Add support for basic authentication on internal API client [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40897: URL: https://github.com/apache/airflow/pull/40897#discussion_r1685103512 ## airflow/api_internal/internal_api_call.py: ## @@ -68,21 +73,31 @@ def get_internal_api_endpoint(): InternalApiConfig._init_values() return

Re: [PR] Add support for basic authentication on internal API client [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40897: URL: https://github.com/apache/airflow/pull/40897#discussion_r1685103512 ## airflow/api_internal/internal_api_call.py: ## @@ -68,21 +73,31 @@ def get_internal_api_endpoint(): InternalApiConfig._init_values() return

Re: [PR] Add support for basic authentication on internal API client [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40897: URL: https://github.com/apache/airflow/pull/40897#discussion_r1685096045 ## airflow/api_internal/internal_api_call.py: ## @@ -68,21 +73,31 @@ def get_internal_api_endpoint(): InternalApiConfig._init_values() return

Re: [PR] Use clearer event naming for UI task instance actions [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40895: URL: https://github.com/apache/airflow/pull/40895#discussion_r1685090146 ## airflow/www/views.py: ## @@ -2373,7 +2373,7 @@ def _clear_dag_tis( @expose("/clear", methods=["POST"]) @auth.has_access_dag("PUT",

[PR] Add support for basic authentication on internal API client [airflow]

2024-07-19 Thread via GitHub
jscheffl opened a new pull request, #40897: URL: https://github.com/apache/airflow/pull/40897 During the implementation of authentication protection for AIP-69 I realized that Internal API does not carry support for authentication access. This PR adds support on the client side with

Re: [PR] Use clearer event naming for UI task instance actions [airflow]

2024-07-19 Thread via GitHub
dstandish commented on PR #40895: URL: https://github.com/apache/airflow/pull/40895#issuecomment-2240278993 any other concern @bbovenzi ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] AIP-44 make database isolation mode work in Breeze [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40894: URL: https://github.com/apache/airflow/pull/40894#discussion_r1685020469 ## airflow/utils/types.py: ## @@ -54,7 +54,7 @@ class AttributeRemoved: """ def __getattr__(self, item): -raise RuntimeError("Attribute was

Re: [PR] AIP-44 make database isolation mode work in Breeze [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40894: URL: https://github.com/apache/airflow/pull/40894#discussion_r1685019082 ## airflow/serialization/pydantic/taskinstance.py: ## @@ -458,9 +458,9 @@ def schedule_downstream_tasks(self, session: Session | None = None, max_tis_per_

Re: [PR] [mothership pr] Run a full celery task with internal API [airflow]

2024-07-19 Thread via GitHub
dstandish closed pull request #37851: [mothership pr] Run a full celery task with internal API URL: https://github.com/apache/airflow/pull/37851 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [mothership pr] Run a full celery task with internal API [airflow]

2024-07-19 Thread via GitHub
dstandish commented on PR #37851: URL: https://github.com/apache/airflow/pull/37851#issuecomment-2240234586 the job of this PR is done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] AIP-44 make database isolation mode work in Breeze [airflow]

2024-07-19 Thread via GitHub
potiuk commented on code in PR #40894: URL: https://github.com/apache/airflow/pull/40894#discussion_r1685004350 ## airflow/serialization/pydantic/taskinstance.py: ## @@ -458,9 +458,9 @@ def schedule_downstream_tasks(self, session: Session | None = None, max_tis_per_

Re: [PR] AIP-44 make database isolation mode work in Breeze [airflow]

2024-07-19 Thread via GitHub
potiuk commented on code in PR #40894: URL: https://github.com/apache/airflow/pull/40894#discussion_r1684995935 ## airflow/utils/types.py: ## @@ -54,7 +54,7 @@ class AttributeRemoved: """ def __getattr__(self, item): -raise RuntimeError("Attribute was

Re: [PR] AIP-44 make database isolation mode work in Breeze [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40894: URL: https://github.com/apache/airflow/pull/40894#discussion_r1684979063 ## airflow/utils/types.py: ## @@ -54,7 +54,7 @@ class AttributeRemoved: """ def __getattr__(self, item): -raise RuntimeError("Attribute was

Re: [PR] AIP-44 make database isolation mode work in Breeze [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40894: URL: https://github.com/apache/airflow/pull/40894#discussion_r1684975951 ## airflow/serialization/pydantic/taskinstance.py: ## @@ -458,9 +458,9 @@ def schedule_downstream_tasks(self, session: Session | None = None, max_tis_per_

Re: [I] Add links to example dags in use params section [airflow]

2024-07-19 Thread via GitHub
AChowdhury1211 commented on issue #40811: URL: https://github.com/apache/airflow/issues/40811#issuecomment-2240103154 I'm a bit confused if "example_params_ui_tutorial" is correct or not? instead seems more appropriate. > To pre-populate values in the form when publishing a link to the

Re: [I] Add links to example dags in use params section [airflow]

2024-07-19 Thread via GitHub
AChowdhury1211 commented on issue #40811: URL: https://github.com/apache/airflow/issues/40811#issuecomment-2240101197 @amolsr If you are planning on opening a PR add a link to this line as well: > If you want to provide special formatting or links you need to use the Param attribute

Re: [I] map_index_template not compute when task marked success manually [airflow]

2024-07-19 Thread via GitHub
karenbraganz commented on issue #39092: URL: https://github.com/apache/airflow/issues/39092#issuecomment-2240098883 I got the error referenced in my PR after adding my rendering logic to set_task_instance_state() in models/dag.py. I just tried adding the logic to _set_state() (from

Re: [PR] AIP-44 make database isolation mode work in Breeze [airflow]

2024-07-19 Thread via GitHub
potiuk commented on code in PR #40894: URL: https://github.com/apache/airflow/pull/40894#discussion_r1684966256 ## airflow/serialization/pydantic/taskinstance.py: ## @@ -458,9 +458,9 @@ def schedule_downstream_tasks(self, session: Session | None = None, max_tis_per_

Re: [PR] fixes: breeze svg not working in the developer tasks doc [airflow]

2024-07-19 Thread via GitHub
boring-cyborg[bot] commented on PR #40892: URL: https://github.com/apache/airflow/pull/40892#issuecomment-2240088342 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional

(airflow) branch main updated: Modified file 03_developer_tasks to solve the breeze_down.svg not found problem in issue #40890 (#40892)

2024-07-19 Thread shahar
This is an automated email from the ASF dual-hosted git repository. shahar pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new dbd2e4e0ac Modified file 03_developer_tasks to

Re: [PR] fixes: breeze svg not working in the developer tasks doc [airflow]

2024-07-19 Thread via GitHub
shahar1 merged PR #40892: URL: https://github.com/apache/airflow/pull/40892 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] AIP-44 make database isolation mode work in Breeze [airflow]

2024-07-19 Thread via GitHub
jscheffl commented on code in PR #40894: URL: https://github.com/apache/airflow/pull/40894#discussion_r1684954042 ## airflow/serialization/pydantic/taskinstance.py: ## @@ -458,9 +458,9 @@ def schedule_downstream_tasks(self, session: Session | None = None, max_tis_per_

Re: [PR] Introduce Amazon Kinesis Analytics V2 (Managed Service for Apache Flink application) [airflow]

2024-07-19 Thread via GitHub
gopidesupavan commented on code in PR #40765: URL: https://github.com/apache/airflow/pull/40765#discussion_r1684951239 ## airflow/providers/amazon/aws/sensors/kinesis_analytics.py: ## @@ -0,0 +1,242 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

Re: [PR] Move AWS Managed Service for Apache Flink sensor states to Hook [airflow]

2024-07-19 Thread via GitHub
gopidesupavan commented on PR #40896: URL: https://github.com/apache/airflow/pull/40896#issuecomment-2240068724 cc @vincbeck -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[PR] Move AWS Managed Service for Apache Flink sensor states to Hook [airflow]

2024-07-19 Thread via GitHub
gopidesupavan opened a new pull request, #40896: URL: https://github.com/apache/airflow/pull/40896 Moving sensor states to Hook. --- **^ Add meaningful description above** Read the **[Pull Request

Re: [PR] Adding Code Coverage webserver code and Fix TypeError and AttributeError in Views.py [airflow]

2024-07-19 Thread via GitHub
andyjianzhou commented on code in PR #40254: URL: https://github.com/apache/airflow/pull/40254#discussion_r1684914602 ## tests/models/test_xcom_arg_map.py: ## @@ -188,6 +191,44 @@ def does_not_work_with_c(v): ] +def test_task_map_from_task_instance_xcom(): Review

Re: [PR] Replace usages of task context logger with the log table [airflow]

2024-07-19 Thread via GitHub
dstandish merged PR #40867: URL: https://github.com/apache/airflow/pull/40867 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] Fix backfilling CLI command where DAG files are missing [airflow]

2024-07-19 Thread via GitHub
shahar1 commented on issue #40525: URL: https://github.com/apache/airflow/issues/40525#issuecomment-2240007841 > @shahar1 Mind if I take-over?  Be my guest :) Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

(airflow) branch main updated (e7e83254db -> f684a58822)

2024-07-19 Thread dstandish
This is an automated email from the ASF dual-hosted git repository. dstandish pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from e7e83254db Add pre-commit to check attrs of TI and TIHistory (#40846) add f684a58822 Replace usages of task

Re: [PR] Replace usages of task context logger with the log table [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1684909173 ## airflow/jobs/scheduler_job_runner.py: ## @@ -1066,6 +1068,14 @@ def _run_scheduler_loop(self) -> None: num_finished_events +=

Re: [I] Fix backfilling CLI command where DAG files are missing [airflow]

2024-07-19 Thread via GitHub
boushphong commented on issue #40525: URL: https://github.com/apache/airflow/issues/40525#issuecomment-2240005279 @shahar1 Mind if I take-over?  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] duplicate key value violates unique constraint serialized_dag_pkey for dynamic dag generation [airflow]

2024-07-19 Thread via GitHub
greg-seesaw commented on issue #40082: URL: https://github.com/apache/airflow/issues/40082#issuecomment-2239964163 Funny enough, this error started happening to us after we upgraded MWAA from 2.8.1 to 2.9.2 on Wednesday. I'm not sure what's happening exactly either, but putting some

Re: [PR] Replace usages of task context logger with the log table [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1684882967 ## airflow/jobs/scheduler_job_runner.py: ## @@ -1066,6 +1068,14 @@ def _run_scheduler_loop(self) -> None: num_finished_events +=

Re: [PR] Use clearer event naming for UI task instance actions [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40895: URL: https://github.com/apache/airflow/pull/40895#discussion_r1684879219 ## airflow/www/views.py: ## @@ -2373,7 +2373,7 @@ def _clear_dag_tis( @expose("/clear", methods=["POST"]) @auth.has_access_dag("PUT",

Re: [PR] Use clearer event naming for UI task instance actions [airflow]

2024-07-19 Thread via GitHub
bbovenzi commented on code in PR #40895: URL: https://github.com/apache/airflow/pull/40895#discussion_r1684876951 ## airflow/www/views.py: ## @@ -2373,7 +2373,7 @@ def _clear_dag_tis( @expose("/clear", methods=["POST"]) @auth.has_access_dag("PUT",

Re: [PR] Use clearer event naming for UI task instance actions [airflow]

2024-07-19 Thread via GitHub
bbovenzi commented on code in PR #40895: URL: https://github.com/apache/airflow/pull/40895#discussion_r1684876951 ## airflow/www/views.py: ## @@ -2373,7 +2373,7 @@ def _clear_dag_tis( @expose("/clear", methods=["POST"]) @auth.has_access_dag("PUT",

Re: [PR] Adding Code Coverage webserver code and Fix TypeError and AttributeError in Views.py [airflow]

2024-07-19 Thread via GitHub
andyjianzhou commented on code in PR #40254: URL: https://github.com/apache/airflow/pull/40254#discussion_r1684866921 ## tests/models/test_xcom_arg_map.py: ## @@ -188,6 +191,44 @@ def does_not_work_with_c(v): ] +def test_task_map_from_task_instance_xcom(): Review

Re: [PR] Adding Code Coverage webserver code and Fix TypeError and AttributeError in Views.py [airflow]

2024-07-19 Thread via GitHub
andyjianzhou commented on code in PR #40254: URL: https://github.com/apache/airflow/pull/40254#discussion_r1684863244 ## tests/models/test_xcom_arg_map.py: ## @@ -188,6 +191,44 @@ def does_not_work_with_c(v): ] +def test_task_map_from_task_instance_xcom(): Review

Re: [PR] Use clearer event naming for UI task instance actions [airflow]

2024-07-19 Thread via GitHub
ephraimbuddy commented on code in PR #40895: URL: https://github.com/apache/airflow/pull/40895#discussion_r1684854833 ## airflow/www/views.py: ## @@ -2373,7 +2373,7 @@ def _clear_dag_tis( @expose("/clear", methods=["POST"]) @auth.has_access_dag("PUT",

Re: [PR] Use clearer event naming for UI task instance actions [airflow]

2024-07-19 Thread via GitHub
ephraimbuddy commented on code in PR #40895: URL: https://github.com/apache/airflow/pull/40895#discussion_r1684852475 ## airflow/www/views.py: ## @@ -2373,7 +2373,7 @@ def _clear_dag_tis( @expose("/clear", methods=["POST"]) @auth.has_access_dag("PUT",

Re: [PR] Adding Code Coverage webserver code and Fix TypeError and AttributeError in Views.py [airflow]

2024-07-19 Thread via GitHub
andyjianzhou commented on code in PR #40254: URL: https://github.com/apache/airflow/pull/40254#discussion_r1684814837 ## tests/www/views/test_views_tasks.py: ## @@ -66,7 +66,7 @@ def reset_dagruns(): @pytest.fixture(autouse=True) -def init_dagruns(app, reset_dagruns):

Re: [I] Support DEBUG logging by component [airflow]

2024-07-19 Thread via GitHub
RNHTTR closed issue #40840: Support DEBUG logging by component URL: https://github.com/apache/airflow/issues/40840 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Replace usages of task context logger with the log table [airflow]

2024-07-19 Thread via GitHub
vincbeck commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1684839047 ## airflow/executors/base_executor.py: ## @@ -139,6 +148,10 @@ def __repr__(self): def start(self): # pragma: no cover """Executors may need to get

Re: [PR] Use clearer event naming for UI task instance actions [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40895: URL: https://github.com/apache/airflow/pull/40895#discussion_r1684834538 ## airflow/www/views.py: ## @@ -2373,7 +2373,7 @@ def _clear_dag_tis( @expose("/clear", methods=["POST"]) @auth.has_access_dag("PUT",

Re: [PR] Use clearer event naming for UI task instance actions [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40895: URL: https://github.com/apache/airflow/pull/40895#discussion_r1684829105 ## airflow/www/views.py: ## @@ -2373,7 +2373,7 @@ def _clear_dag_tis( @expose("/clear", methods=["POST"]) @auth.has_access_dag("PUT",

Re: [PR] Use clearer event naming for UI task instance actions [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40895: URL: https://github.com/apache/airflow/pull/40895#discussion_r1684826368 ## airflow/www/views.py: ## @@ -2373,7 +2373,7 @@ def _clear_dag_tis( @expose("/clear", methods=["POST"]) @auth.has_access_dag("PUT",

Re: [PR] Log info about task instance clearing/marking as succcess/failure [airflow]

2024-07-19 Thread via GitHub
dstandish commented on PR #40656: URL: https://github.com/apache/airflow/pull/40656#issuecomment-2239892930 ok so, it turns out, this information is already more or less available in the UI via the audit log. recently, i added try_number to the table and added try_number and map_index to

Re: [PR] Log info about task instance clearing/marking as succcess/failure [airflow]

2024-07-19 Thread via GitHub
dstandish closed pull request #40656: Log info about task instance clearing/marking as succcess/failure URL: https://github.com/apache/airflow/pull/40656 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Use clearer event naming for UI task instance actions [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40895: URL: https://github.com/apache/airflow/pull/40895#discussion_r1684827112 ## airflow/www/views.py: ## @@ -2373,7 +2373,7 @@ def _clear_dag_tis( @expose("/clear", methods=["POST"]) @auth.has_access_dag("PUT",

Re: [PR] Use clearer event naming for UI task instance actions [airflow]

2024-07-19 Thread via GitHub
hussein-awala commented on code in PR #40895: URL: https://github.com/apache/airflow/pull/40895#discussion_r1684825784 ## airflow/www/views.py: ## @@ -2373,7 +2373,7 @@ def _clear_dag_tis( @expose("/clear", methods=["POST"]) @auth.has_access_dag("PUT",

Re: [PR] Use clearer event naming for UI task instance actions [airflow]

2024-07-19 Thread via GitHub
bbovenzi commented on code in PR #40895: URL: https://github.com/apache/airflow/pull/40895#discussion_r1684817007 ## airflow/www/views.py: ## @@ -2373,7 +2373,7 @@ def _clear_dag_tis( @expose("/clear", methods=["POST"]) @auth.has_access_dag("PUT",

Re: [PR] Adding Code Coverage webserver code and Fix TypeError and AttributeError in Views.py [airflow]

2024-07-19 Thread via GitHub
andyjianzhou commented on code in PR #40254: URL: https://github.com/apache/airflow/pull/40254#discussion_r1684814837 ## tests/www/views/test_views_tasks.py: ## @@ -66,7 +66,7 @@ def reset_dagruns(): @pytest.fixture(autouse=True) -def init_dagruns(app, reset_dagruns):

[PR] Use clearer event naming for UI task instance actions [airflow]

2024-07-19 Thread via GitHub
dstandish opened a new pull request, #40895: URL: https://github.com/apache/airflow/pull/40895 This makes it a little clearer to the user what actually happened. Here is an example https://github.com/user-attachments/assets/21f144a1-17ea-4ca9-88cc-3fc30add3605;> See there it

Re: [PR] Replace usages of task context logger with the log table [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1684801332 ## airflow/executors/base_executor.py: ## @@ -139,6 +148,10 @@ def __repr__(self): def start(self): # pragma: no cover """Executors may need to get

Re: [PR] Replace usages of task context logger with the log table [airflow]

2024-07-19 Thread via GitHub
ephraimbuddy commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1684755334 ## airflow/jobs/scheduler_job_runner.py: ## @@ -1066,6 +1068,14 @@ def _run_scheduler_loop(self) -> None: num_finished_events +=

Re: [PR] Replace usages of task context logger with the log table [airflow]

2024-07-19 Thread via GitHub
ephraimbuddy commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1684744011 ## airflow/jobs/scheduler_job_runner.py: ## @@ -1066,6 +1068,14 @@ def _run_scheduler_loop(self) -> None: num_finished_events +=

Re: [PR] retry for failure in the queue [airflow]

2024-07-19 Thread via GitHub
RNHTTR commented on code in PR #39398: URL: https://github.com/apache/airflow/pull/39398#discussion_r1684729030 ## airflow/settings.py: ## @@ -98,6 +98,7 @@ STATE_COLORS = { "deferred": "mediumpurple", "failed": "red", +"failed_in_queue": "reddishorange", Review

Re: [PR] AIP-44 make database isolation mode work in Breeze [airflow]

2024-07-19 Thread via GitHub
vincbeck commented on code in PR #40894: URL: https://github.com/apache/airflow/pull/40894#discussion_r1684707710 ## airflow/__main__.py: ## @@ -55,6 +55,17 @@ def main(): conf = write_default_airflow_configuration_if_needed() if args.subcommand in

Re: [PR] Enable ending the task directly from the triggerer without going into the worker. [airflow]

2024-07-19 Thread via GitHub
sunank200 commented on code in PR #40084: URL: https://github.com/apache/airflow/pull/40084#discussion_r1684696476 ## airflow/triggers/temporal.py: ## @@ -84,7 +95,11 @@ class TimeDeltaTrigger(DateTimeTrigger): While this is its own distinct class here, it will serialise

Re: [PR] Enable ending the task directly from the triggerer without going into the worker. [airflow]

2024-07-19 Thread via GitHub
sunank200 commented on code in PR #40084: URL: https://github.com/apache/airflow/pull/40084#discussion_r1684695820 ## airflow/triggers/base.py: ## @@ -137,3 +150,96 @@ def __eq__(self, other): if isinstance(other, TriggerEvent): return other.payload ==

Re: [PR] Enable ending the task directly from the triggerer without going into the worker. [airflow]

2024-07-19 Thread via GitHub
sunank200 commented on code in PR #40084: URL: https://github.com/apache/airflow/pull/40084#discussion_r1684695310 ## airflow/triggers/base.py: ## @@ -137,3 +150,96 @@ def __eq__(self, other): if isinstance(other, TriggerEvent): return other.payload ==

Re: [PR] Enable ending the task directly from the triggerer without going into the worker. [airflow]

2024-07-19 Thread via GitHub
sunank200 commented on code in PR #40084: URL: https://github.com/apache/airflow/pull/40084#discussion_r1684694349 ## airflow/triggers/base.py: ## @@ -137,3 +150,96 @@ def __eq__(self, other): if isinstance(other, TriggerEvent): return other.payload ==

Re: [PR] Introduce Amazon Kinesis Analytics V2 (Managed Service for Apache Flink application) [airflow]

2024-07-19 Thread via GitHub
eladkal commented on code in PR #40765: URL: https://github.com/apache/airflow/pull/40765#discussion_r1684691796 ## airflow/providers/amazon/aws/sensors/kinesis_analytics.py: ## @@ -0,0 +1,242 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

Re: [PR] Enable ending the task directly from the triggerer without going into the worker. [airflow]

2024-07-19 Thread via GitHub
sunank200 commented on code in PR #40084: URL: https://github.com/apache/airflow/pull/40084#discussion_r1684690172 ## airflow/sensors/time_sensor.py: ## @@ -73,12 +75,11 @@ def __init__(self, *, target_time: datetime.time, **kwargs) -> None: self.target_datetime =

Re: [PR] Enable ending the task directly from the triggerer without going into the worker. [airflow]

2024-07-19 Thread via GitHub
sunank200 commented on code in PR #40084: URL: https://github.com/apache/airflow/pull/40084#discussion_r1684689712 ## airflow/sensors/time_delta.py: ## @@ -59,28 +59,34 @@ class TimeDeltaSensorAsync(TimeDeltaSensor): Will defers itself to avoid taking up a worker slot

Re: [PR] Enable ending the task directly from the triggerer without going into the worker. [airflow]

2024-07-19 Thread via GitHub
sunank200 commented on code in PR #40084: URL: https://github.com/apache/airflow/pull/40084#discussion_r1684688122 ## airflow/sensors/time_delta.py: ## @@ -59,28 +59,34 @@ class TimeDeltaSensorAsync(TimeDeltaSensor): Will defers itself to avoid taking up a worker slot

Re: [PR] AIP-44 make database isolation mode work in Breeze [airflow]

2024-07-19 Thread via GitHub
potiuk commented on PR #40894: URL: https://github.com/apache/airflow/pull/40894#issuecomment-2239702991 ``` breeze start-airflow --database-isolation --executor CeleryExecutor --load-default-connections --load-example-dags ``` -- This is an automated message from the Apache

Re: [PR] AIP-44 make database isolation mode work in Breeze [airflow]

2024-07-19 Thread via GitHub
potiuk commented on PR #40894: URL: https://github.com/apache/airflow/pull/40894#issuecomment-2239701745 I run quite many DAGs in "DB isolation mode" and with those changes, it looks like celery worker nicely runs them (and it's not even very slow it looks). For now "mini scheduler" is

[PR] AIP-44 make database isolation mode work in Breeze [airflow]

2024-07-19 Thread via GitHub
potiuk opened a new pull request, #40894: URL: https://github.com/apache/airflow/pull/40894 With this PR, it is possible to get a working "DB isolation" working solution with celery executor. From my tests it works comparably fast to the non-DB isolation executor. Things changed

Re: [PR] Add start execution from triggerer support to dynamic task mapping [airflow]

2024-07-19 Thread via GitHub
Lee-W commented on code in PR #39912: URL: https://github.com/apache/airflow/pull/39912#discussion_r1684623009 ## airflow/models/mappedoperator.py: ## @@ -729,6 +731,67 @@ def _get_unmap_kwargs(self, mapped_kwargs: Mapping[str, Any], *, strict: bool) - "params":

Re: [PR] Add start execution from triggerer support to dynamic task mapping [airflow]

2024-07-19 Thread via GitHub
Lee-W commented on code in PR #39912: URL: https://github.com/apache/airflow/pull/39912#discussion_r1684620020 ## airflow/models/dagrun.py: ## @@ -1543,11 +1543,26 @@ def schedule_tis( and not ti.task.outlets ):

Re: [PR] Adding Code Coverage webserver code and Fix TypeError and AttributeError in Views.py [airflow]

2024-07-19 Thread via GitHub
aritra24 commented on code in PR #40254: URL: https://github.com/apache/airflow/pull/40254#discussion_r1684610001 ## tests/www/views/test_views_tasks.py: ## @@ -66,7 +66,7 @@ def reset_dagruns(): @pytest.fixture(autouse=True) -def init_dagruns(app, reset_dagruns): Review

Re: [PR] Replace usages of task context logger with the log table [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1684533304 ## airflow/jobs/scheduler_job_runner.py: ## @@ -1066,6 +1068,14 @@ def _run_scheduler_loop(self) -> None: num_finished_events +=

Re: [PR] Enable ending the task directly from the triggerer without going into the worker. [airflow]

2024-07-19 Thread via GitHub
sunank200 commented on code in PR #40084: URL: https://github.com/apache/airflow/pull/40084#discussion_r1684595916 ## docs/apache-airflow/authoring-and-scheduling/deferring.rst: ## @@ -213,6 +213,70 @@ This is particularly useful when deferring is the only thing the

Re: [PR] Enable ending the task directly from the triggerer without going into the worker. [airflow]

2024-07-19 Thread via GitHub
sunank200 commented on code in PR #40084: URL: https://github.com/apache/airflow/pull/40084#discussion_r1684595489 ## docs/apache-airflow/authoring-and-scheduling/deferring.rst: ## @@ -213,6 +213,70 @@ This is particularly useful when deferring is the only thing the

Re: [PR] Ability to add custom facet in OpenLineage events [airflow]

2024-07-19 Thread via GitHub
anandhimurali commented on code in PR #38982: URL: https://github.com/apache/airflow/pull/38982#discussion_r1684583435 ## tests/system/providers/google/cloud/datapipelines/resources/__init__.py: ## Review Comment: This was added by the pre-commit hook. I tried again now by

Re: [PR] fixes: breeze svg not working in the developer tasks doc [airflow]

2024-07-19 Thread via GitHub
Lee-W commented on code in PR #40892: URL: https://github.com/apache/airflow/pull/40892#discussion_r1684565711 ## dev/breeze/doc/03_developer_tasks.rst: ## @@ -434,7 +434,7 @@ Then, next time when you start Breeze, it will have the data pre-populated. These are all available

Re: [PR] fixes: breeze svg not working in the developer tasks doc [airflow]

2024-07-19 Thread via GitHub
Lee-W commented on code in PR #40892: URL: https://github.com/apache/airflow/pull/40892#discussion_r1684565711 ## dev/breeze/doc/03_developer_tasks.rst: ## @@ -434,7 +434,7 @@ Then, next time when you start Breeze, it will have the data pre-populated. These are all available

Re: [PR] Enable ending the task directly from the triggerer without going into the worker. [airflow]

2024-07-19 Thread via GitHub
Lee-W commented on code in PR #40084: URL: https://github.com/apache/airflow/pull/40084#discussion_r1684537451 ## airflow/sensors/time_delta.py: ## @@ -59,28 +59,34 @@ class TimeDeltaSensorAsync(TimeDeltaSensor): Will defers itself to avoid taking up a worker slot while it

Re: [PR] Enable ending the task directly from the triggerer without going into the worker. [airflow]

2024-07-19 Thread via GitHub
sunank200 commented on code in PR #40084: URL: https://github.com/apache/airflow/pull/40084#discussion_r1684549574 ## airflow/models/taskinstance.py: ## @@ -715,6 +716,13 @@ def _execute_task(task_instance: TaskInstance | TaskInstancePydantic, context: C if not

Re: [PR] fixes: breeze svg not working in the developer tasks doc #40890 [airflow]

2024-07-19 Thread via GitHub
boring-cyborg[bot] commented on PR #40892: URL: https://github.com/apache/airflow/pull/40892#issuecomment-2239446916 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[PR] fixes: breeze svg not working in the developer tasks doc #40890 [airflow]

2024-07-19 Thread via GitHub
AChowdhury1211 opened a new pull request, #40892: URL: https://github.com/apache/airflow/pull/40892 Fixes: breeze_down.svg not working in the developer_tasks.rst doc #40890 --- **^ Add meaningful description above** Read the **[Pull Request

Re: [PR] Enable ending the task directly from the triggerer without going into the worker. [airflow]

2024-07-19 Thread via GitHub
Lee-W commented on code in PR #40084: URL: https://github.com/apache/airflow/pull/40084#discussion_r1684536248 ## airflow/sensors/time_delta.py: ## @@ -59,28 +59,34 @@ class TimeDeltaSensorAsync(TimeDeltaSensor): Will defers itself to avoid taking up a worker slot while it

Re: [PR] Enable ending the task directly from the triggerer without going into the worker. [airflow]

2024-07-19 Thread via GitHub
Lee-W commented on code in PR #40084: URL: https://github.com/apache/airflow/pull/40084#discussion_r1684527807 ## airflow/models/taskinstance.py: ## @@ -715,6 +716,13 @@ def _execute_task(task_instance: TaskInstance | TaskInstancePydantic, context: C if not

Re: [PR] Replace usages of task context logger with the log table [airflow]

2024-07-19 Thread via GitHub
dstandish commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1684533304 ## airflow/jobs/scheduler_job_runner.py: ## @@ -1066,6 +1068,14 @@ def _run_scheduler_loop(self) -> None: num_finished_events +=

(airflow) branch main updated (e774d08773 -> e7e83254db)

2024-07-19 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from e774d08773 Update Glue Data Quality system test sample data (#40870) add e7e83254db Add pre-commit to

Re: [PR] Add pre-commit to check attrs of TI and TIHistory [airflow]

2024-07-19 Thread via GitHub
ephraimbuddy merged PR #40846: URL: https://github.com/apache/airflow/pull/40846 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Add task duration plot across dagruns [airflow]

2024-07-19 Thread via GitHub
bbovenzi commented on code in PR #40755: URL: https://github.com/apache/airflow/pull/40755#discussion_r1684510081 ## airflow/www/static/js/dag/details/task/AllTaskDuration.tsx: ## @@ -0,0 +1,160 @@ +/*! + * Licensed to the Apache Software Foundation (ASF) under one + * or more

(airflow) branch main updated: Update Glue Data Quality system test sample data (#40870)

2024-07-19 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new e774d08773 Update Glue Data Quality system test

Re: [PR] Update Glue Data Quality system test sample data [airflow]

2024-07-19 Thread via GitHub
vincbeck merged PR #40870: URL: https://github.com/apache/airflow/pull/40870 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] cohere 5.0.0 support [airflow]

2024-07-19 Thread via GitHub
github-actions[bot] commented on PR #38465: URL: https://github.com/apache/airflow/pull/38465#issuecomment-2239319321 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [PR] Fix overriden base container name not used to fetch logs on KPO trigger_reentry [airflow]

2024-07-19 Thread via GitHub
peloyeje commented on PR #40835: URL: https://github.com/apache/airflow/pull/40835#issuecomment-2239316177 Static checks fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] HttpTrigger does not use the new `json` param in the HttpAsyncHook [airflow]

2024-07-19 Thread via GitHub
mperesson commented on issue #40889: URL: https://github.com/apache/airflow/issues/40889#issuecomment-2239312916 Looks like this PR https://github.com/apache/airflow/pull/38512 would solve the issue. -- This is an automated message from the Apache Git Service. To respond to the message,

[PR] fix: always defer once more after log fetching to ensure pod completion is handled [airflow]

2024-07-19 Thread via GitHub
peloyeje opened a new pull request, #40891: URL: https://github.com/apache/airflow/pull/40891 Fix based on a real world issue seen in production where a failing pod does not make the associated task fail Running theory: then a pod fails while in

  1   2   >