o-nikolas opened a new pull request, #38514:
URL: https://github.com/apache/airflow/pull/38514
This PR delivers the first set of changes to the scheduler job to transform
it to support hybrid executors. It covers starting and ending the executors as
well as debug dump.
o-nikolas commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1540104792
##
airflow/jobs/job.py:
##
@@ -104,12 +105,13 @@ class Job(Base, LoggingMixin):
Only makes sense for SchedulerJob and BackfillJob instances.
"""
-de
uranusjr commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1547079296
##
airflow/jobs/scheduler_job_runner.py:
##
@@ -819,19 +821,20 @@ def _execute(self) -> int | None:
)
try:
-self.job.executor.job
o-nikolas commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1548204315
##
airflow/jobs/scheduler_job_runner.py:
##
@@ -819,19 +821,20 @@ def _execute(self) -> int | None:
)
try:
-self.job.executor.jo
dstandish commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1548637266
##
airflow/jobs/job.py:
##
@@ -104,12 +105,13 @@ class Job(Base, LoggingMixin):
Only makes sense for SchedulerJob and BackfillJob instances.
"""
-de
dstandish commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1548638498
##
airflow/jobs/scheduler_job_runner.py:
##
@@ -270,8 +270,10 @@ def _debug_dump(self, signum: int, frame: FrameType |
None) -> None:
self.log.info("%s\
dstandish commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1548638986
##
airflow/jobs/scheduler_job_runner.py:
##
@@ -270,8 +270,10 @@ def _debug_dump(self, signum: int, frame: FrameType |
None) -> None:
self.log.info("%s\
dstandish commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1548639831
##
airflow/jobs/scheduler_job_runner.py:
##
@@ -819,19 +821,20 @@ def _execute(self) -> int | None:
)
try:
-self.job.executor.jo
o-nikolas commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1548690491
##
airflow/jobs/job.py:
##
@@ -104,12 +105,13 @@ class Job(Base, LoggingMixin):
Only makes sense for SchedulerJob and BackfillJob instances.
"""
-de
o-nikolas commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1548693252
##
airflow/jobs/scheduler_job_runner.py:
##
@@ -270,8 +270,10 @@ def _debug_dump(self, signum: int, frame: FrameType |
None) -> None:
self.log.info("%s\
o-nikolas commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1548690491
##
airflow/jobs/job.py:
##
@@ -104,12 +105,13 @@ class Job(Base, LoggingMixin):
Only makes sense for SchedulerJob and BackfillJob instances.
"""
-de
o-nikolas commented on PR #38514:
URL: https://github.com/apache/airflow/pull/38514#issuecomment-2033219581
> on executor end... i looked at the way executors have implemented end... i
was surprised to see that they wait for all tasks to finish. given that we have
re-adoption logic, i wonde
o-nikolas commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1548704690
##
airflow/jobs/scheduler_job_runner.py:
##
@@ -819,19 +821,20 @@ def _execute(self) -> int | None:
)
try:
-self.job.executor.jo
uranusjr commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1548745509
##
airflow/jobs/job.py:
##
@@ -104,12 +105,13 @@ class Job(Base, LoggingMixin):
Only makes sense for SchedulerJob and BackfillJob instances.
"""
-def
dstandish commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1549952673
##
airflow/jobs/job.py:
##
@@ -104,12 +105,13 @@ class Job(Base, LoggingMixin):
Only makes sense for SchedulerJob and BackfillJob instances.
"""
-de
dstandish commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1550099456
##
airflow/jobs/scheduler_job_runner.py:
##
@@ -270,8 +270,10 @@ def _debug_dump(self, signum: int, frame: FrameType |
None) -> None:
self.log.info("%s\
o-nikolas commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1552549164
##
airflow/jobs/scheduler_job_runner.py:
##
@@ -270,8 +270,10 @@ def _debug_dump(self, signum: int, frame: FrameType |
None) -> None:
self.log.info("%s\
dstandish commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1552594898
##
airflow/jobs/scheduler_job_runner.py:
##
@@ -270,8 +270,10 @@ def _debug_dump(self, signum: int, frame: FrameType |
None) -> None:
self.log.info("%s\
o-nikolas commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1554018702
##
airflow/jobs/scheduler_job_runner.py:
##
@@ -819,19 +821,20 @@ def _execute(self) -> int | None:
)
try:
-self.job.executor.jo
potiuk commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1555067360
##
airflow/jobs/job.py:
##
@@ -104,12 +105,13 @@ class Job(Base, LoggingMixin):
Only makes sense for SchedulerJob and BackfillJob instances.
"""
-def _
uranusjr commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1555276527
##
airflow/jobs/job.py:
##
@@ -104,12 +106,13 @@ class Job(Base, LoggingMixin):
Only makes sense for SchedulerJob and BackfillJob instances.
"""
-def
o-nikolas commented on code in PR #38514:
URL: https://github.com/apache/airflow/pull/38514#discussion_r1556103085
##
airflow/jobs/job.py:
##
@@ -104,12 +106,13 @@ class Job(Base, LoggingMixin):
Only makes sense for SchedulerJob and BackfillJob instances.
"""
-de
o-nikolas merged PR #38514:
URL: https://github.com/apache/airflow/pull/38514
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflo
23 matches
Mail list logo