kosteev commented on code in PR #27155:
URL: https://github.com/apache/airflow/pull/27155#discussion_r1004255873
##########
airflow/jobs/local_task_job.py:
##########
@@ -162,6 +162,7 @@ def handle_task_exit(self, return_code: int) -> None:
# Without setting this, heartbeat may get us
self.terminating = True
self.log.info("Task exited with return code %s", return_code)
+
Stats.incr(f'ti.raw_task_return_code.{self.dag_id}.{self.task_instance.task_id}.{return_code}')
Review Comment:
Not sure about how is better design here, but I thought for a moment that
maybe good to have encapsulated the logic of emitting this metric inside
taskinstance.py module (as other metrics relate to task instance).
##########
tests/jobs/test_local_task_job.py:
##########
@@ -374,6 +374,29 @@ def test_localtaskjob_double_trigger(self):
session.close()
+ @patch.object(StandardTaskRunner, 'return_code')
+ @mock.patch('airflow.jobs.scheduler_job.Stats.incr')
Review Comment:
It is good to have autospec=True parameter set for mocked objects, that will
enforce method signature checks by unittest.mock library.
Although, I am not sure if this guideline is followed in the repo.
##########
tests/jobs/test_local_task_job.py:
##########
@@ -374,6 +374,29 @@ def test_localtaskjob_double_trigger(self):
session.close()
+ @patch.object(StandardTaskRunner, 'return_code')
+ @mock.patch('airflow.jobs.scheduler_job.Stats.incr')
+ def test_raw_task_return_code_metric(self, mock_stats_incr,
mock_return_code, create_dummy_dag):
+
+ _, task = create_dummy_dag('test_localtaskjob_double_trigger')
+ mock_stats_incr.reset_mock()
Review Comment:
Is it really needed (reset_mock)? This is just beginning of the method.
##########
airflow/jobs/local_task_job.py:
##########
@@ -162,6 +162,7 @@ def handle_task_exit(self, return_code: int) -> None:
# Without setting this, heartbeat may get us
self.terminating = True
self.log.info("Task exited with return code %s", return_code)
+
Stats.incr(f'ti.raw_task_return_code.{self.dag_id}.{self.task_instance.task_id}.{return_code}')
Review Comment:
Or maybe metric should be more like
`local_task_job.task_exit.{dag_id}.{task_id}.{return_code}`. WDYT?
##########
tests/jobs/test_local_task_job.py:
##########
@@ -374,6 +374,29 @@ def test_localtaskjob_double_trigger(self):
session.close()
+ @patch.object(StandardTaskRunner, 'return_code')
+ @mock.patch('airflow.jobs.scheduler_job.Stats.incr')
+ def test_raw_task_return_code_metric(self, mock_stats_incr,
mock_return_code, create_dummy_dag):
+
+ _, task = create_dummy_dag('test_localtaskjob_double_trigger')
+ mock_stats_incr.reset_mock()
+
+ ti_run = TaskInstance(task=task, execution_date=DEFAULT_DATE)
+ ti_run.refresh_from_db()
+ job1 = LocalTaskJob(task_instance=ti_run,
executor=SequentialExecutor())
+
+ mock_return_code.side_effect = [None, -9, None]
+
+ with timeout(10):
+ job1.run()
+
+ mock_stats_incr.assert_has_calls(
+ [
+
mock.call('ti.raw_task_return_code.test_localtaskjob_double_trigger.op1.-9'),
+ ],
+ any_order=True,
Review Comment:
You set "any_order=True", but only one call is expected, looks strange?
Maybe good idea to actually have multiple calls expected from different
tasks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
