Re: [PR] Replace usages of task context logger with the log table [airflow]

dstandish merged PR #40867: URL: https://github.com/apache/airflow/pull/40867 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [PR] Replace usages of task context logger with the log table [airflow]

dstandish commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1684909173 ## airflow/jobs/scheduler_job_runner.py: ## @@ -1066,6 +1068,14 @@ def _run_scheduler_loop(self) -> None: num_finished_events += self._proc

Re: [PR] Replace usages of task context logger with the log table [airflow]

dstandish commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1684882967 ## airflow/jobs/scheduler_job_runner.py: ## @@ -1066,6 +1068,14 @@ def _run_scheduler_loop(self) -> None: num_finished_events += self._proc

Re: [PR] Replace usages of task context logger with the log table [airflow]

vincbeck commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1684839047 ## airflow/executors/base_executor.py: ## @@ -139,6 +148,10 @@ def __repr__(self): def start(self): # pragma: no cover """Executors may need to get th

Re: [PR] Replace usages of task context logger with the log table [airflow]

dstandish commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1684801332 ## airflow/executors/base_executor.py: ## @@ -139,6 +148,10 @@ def __repr__(self): def start(self): # pragma: no cover """Executors may need to get t

Re: [PR] Replace usages of task context logger with the log table [airflow]

ephraimbuddy commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1684755334 ## airflow/jobs/scheduler_job_runner.py: ## @@ -1066,6 +1068,14 @@ def _run_scheduler_loop(self) -> None: num_finished_events += self._p

Re: [PR] Replace usages of task context logger with the log table [airflow]

ephraimbuddy commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1684744011 ## airflow/jobs/scheduler_job_runner.py: ## @@ -1066,6 +1068,14 @@ def _run_scheduler_loop(self) -> None: num_finished_events += self._p

Re: [PR] Replace usages of task context logger with the log table [airflow]

dstandish commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1684533304 ## airflow/jobs/scheduler_job_runner.py: ## @@ -1066,6 +1068,14 @@ def _run_scheduler_loop(self) -> None: num_finished_events += self._proc

Re: [PR] Replace usages of task context logger with the log table [airflow]

dstandish commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1684533304 ## airflow/jobs/scheduler_job_runner.py: ## @@ -1066,6 +1068,14 @@ def _run_scheduler_loop(self) -> None: num_finished_events += self._proc

Re: [PR] Replace usages of task context logger with the log table [airflow]

potiuk closed pull request #40867: Replace usages of task context logger with the log table URL: https://github.com/apache/airflow/pull/40867 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Replace usages of task context logger with the log table [airflow]

ephraimbuddy commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1683821442 ## airflow/jobs/scheduler_job_runner.py: ## @@ -1066,6 +1068,14 @@ def _run_scheduler_loop(self) -> None: num_finished_events += self._p

Re: [PR] Replace usages of task context logger with the log table [airflow]

vincbeck commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1683417281 ## airflow/providers/amazon/aws/executors/batch/batch_executor.py: ## @@ -292,12 +292,13 @@ def attempt_submit_jobs(self): if failure_reason:

Re: [PR] Replace usages of task context logger with the log table [airflow]

vincbeck commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1683413318 ## airflow/providers/amazon/aws/executors/ecs/ecs_executor.py: ## @@ -385,18 +385,25 @@ def attempt_task_runs(self): ) self

Re: [PR] Replace usages of task context logger with the log table [airflow]

dstandish commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1683412728 ## airflow/providers/amazon/aws/executors/batch/batch_executor.py: ## @@ -292,12 +292,13 @@ def attempt_submit_jobs(self): if failure_reason:

Re: [PR] Replace usages of task context logger with the log table [airflow]

vincbeck commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1683410837 ## airflow/providers/amazon/aws/executors/batch/batch_executor.py: ## @@ -292,12 +292,13 @@ def attempt_submit_jobs(self): if failure_reason:

Re: [PR] Replace usages of task context logger with the log table [airflow]

dstandish commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1683408716 ## airflow/providers/amazon/aws/executors/batch/batch_executor.py: ## @@ -292,12 +292,13 @@ def attempt_submit_jobs(self): if failure_reason:

Re: [PR] Replace usages of task context logger with the log table [airflow]

dstandish commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1683409487 ## airflow/providers/amazon/aws/executors/ecs/ecs_executor.py: ## @@ -385,18 +385,25 @@ def attempt_task_runs(self): ) sel

Re: [PR] Replace usages of task context logger with the log table [airflow]

dstandish commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1683407367 ## airflow/providers/amazon/aws/executors/ecs/ecs_executor.py: ## @@ -385,18 +385,25 @@ def attempt_task_runs(self): ) sel

Re: [PR] Replace usages of task context logger with the log table [airflow]

vincbeck commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1683343804 ## airflow/providers/amazon/aws/executors/ecs/ecs_executor.py: ## @@ -385,18 +385,25 @@ def attempt_task_runs(self): ) self

Re: [PR] Replace usages of task context logger with the log table [airflow]

vincbeck commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1683341652 ## airflow/providers/amazon/aws/executors/batch/batch_executor.py: ## @@ -292,12 +292,13 @@ def attempt_submit_jobs(self): if failure_reason:

Re: [PR] Replace usages of task context logger with the log table [airflow]

dstandish commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1683336091 ## airflow/providers/amazon/aws/executors/ecs/ecs_executor.py: ## @@ -385,18 +385,25 @@ def attempt_task_runs(self): ) sel

Re: [PR] Replace usages of task context logger with the log table [airflow]

dstandish commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1683336784 ## airflow/providers/amazon/aws/executors/batch/batch_executor.py: ## @@ -292,12 +292,13 @@ def attempt_submit_jobs(self): if failure_reason:

Re: [PR] Replace usages of task context logger with the log table [airflow]

vincbeck commented on code in PR #40867: URL: https://github.com/apache/airflow/pull/40867#discussion_r1683310982 ## airflow/jobs/scheduler_job_runner.py: ## @@ -1649,11 +1658,15 @@ def _fail_tasks_stuck_in_queued(self, session: Session = NEW_SESSION) -> None:

Re: [PR] Replace usages of task context logger with the log table [airflow]

vincbeck commented on PR #40867: URL: https://github.com/apache/airflow/pull/40867#issuecomment-2236821219 Some tests need to be updated though -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[PR] Replace usages of task context logger with the log table [airflow]

dstandish opened a new pull request, #40867: URL: https://github.com/apache/airflow/pull/40867 Shipping messages to TI logs, which unfortunately in many cases means uploading to s3, is inefficient, and not really something the scheduler should be doing. Instead we can use an old but