dheerajturaga commented on PR #65833: URL: https://github.com/apache/airflow/pull/65833#issuecomment-4320852363
> Okay, signalling is better but I do not see an effect. At least folliowing
my "integration test" Dag also a failed task ("plan_to_fail") still is reported
in "success":
>
> <img alt="Image" width="1283" height="541"
src="https://private-user-images.githubusercontent.com/95105677/583785573-c8aa1ed0-aeb2-452e-ba13-9194ee139a74.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NzcxNjExMzIsIm5iZiI6MTc3NzE2MDgzMiwicGF0aCI6Ii85NTEwNTY3Ny81ODM3ODU1NzMtYzhhYTFlZDAtYWViMi00NTJlLWJhMTMtOTE5NGVlMTM5YTc0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjA0MjUlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYwNDI1VDIzNDcxMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTI5YWI3NWEyODg0YTczYmMzMzhkZTg0MmNkYmFjNWRkNjZlZDQyMjk0ZjI0NmMxZWYxMmUwMWFhYzdkZmI3Y2MmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JnJlc3BvbnNlLWNvbnRlbnQtdHlwZT1pbWFnZSUyRnBuZyJ9.PMXBK6_TJ5bkGpcgTy7xLUvfQXHN3QnghiUG1keWCF4";>
> So as I also see this as an improvement, can you tell where you see an
effect in reporting the exit status?
What you are reporting seems to be a different bug altogether. when a user
task raises, task_runner.main() catches it, sends TaskState(FAILED) to the
supervisor, and returns normally. _fork_main then exits the task-runner
subprocess with code 0. supervise_task returns exit_code=0. edge3's
_run_job_via_supervisor only looks at the return value, so Job.is_success is
True and it calls jobs_set_state(..., SUCCESS)
this was the original error that I saw and was trying to fix
```
[TIMESTAMP] {worker.py:226} ERROR - Task execution failed
Traceback (most recent call last):
File
".../airflow/venv/lib/python3.11/site-packages/httpx/_transports/default.py",
line 101, in map_httpcore_exceptions
yield
File
".../airflow/venv/lib/python3.11/site-packages/httpx/_transports/default.py",
line 250, in handle_request
resp = self._pool.handle_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
".../airflow/venv/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py",
line 256, in handle_request
raise exc from None
File
".../airflow/venv/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py",
line 236, in handle_request
response = connection.handle_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File
".../airflow/venv/lib/python3.11/site-packages/httpcore/_sync/connection.py",
line 101, in handle_request
raise exc
File
".../airflow/venv/lib/python3.11/site-packages/httpcore/_sync/connection.py",
line 78, in handle_request
stream = self._connect(request)
^^^^^^^^^^^^^^^^^^^^^^
File
".../airflow/venv/lib/python3.11/site-packages/httpcore/_sync/connection.py",
line 124, in _connect
stream = self._network_backend.connect_tcp(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
".../airflow/venv/lib/python3.11/site-packages/httpcore/_backends/sync.py",
line 207, in connect_tcp
with map_exceptions(exc_map):
File ".../python3.11/contextlib.py", line 155, in __exit__
self.gen.throw(typ, value, traceback)
File
".../airflow/venv/lib/python3.11/site-packages/httpcore/_exceptions.py", line
14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ConnectError: [Errno 111] Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File
".../airflow/venv/lib/python3.11/site-packages/airflow/providers/edge3/cli/worker.py",
line 214, in _run_job_via_supervisor
supervise(
File
".../airflow/venv/lib/python3.11/site-packages/airflow/sdk/execution_time/supervisor.py",
line 1993, in supervise
exit_code = process.wait()
^^^^^^^^^^^^^^
File
".../airflow/venv/lib/python3.11/site-packages/airflow/sdk/execution_time/supervisor.py",
line 1004, in wait
self.update_task_state_if_needed()
File
".../airflow/venv/lib/python3.11/site-packages/airflow/sdk/execution_time/supervisor.py",
line 1018, in update_task_state_if_needed
self.client.task_instances.finish(
File
".../airflow/venv/lib/python3.11/site-packages/airflow/sdk/api/client.py", line
226, in finish
self.client.patch(f"task-instances/{id}/state",
content=body.model_dump_json())
File ".../airflow/venv/lib/python3.11/site-packages/httpx/_client.py",
line 1218, in patch
return self.request(
^^^^^^^^^^^^^
File
".../airflow/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 331,
in wrapped_f
return copy(f, *args, **kw)
^^^^^^^^^^^^^^^^^^^^
File
".../airflow/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 470,
in __call__
do = self.iter(retry_state=retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
".../airflow/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 371,
in iter
result = action(retry_state)
^^^^^^^^^^^^^^^^^^^
File
".../airflow/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 413,
in exc_check
raise retry_exc.reraise()
^^^^^^^^^^^^^^^^^^^
File
".../airflow/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 184,
in reraise
raise self.last_attempt.result()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File ".../python3.11/concurrent/futures/_base.py", line 401, in
__get_result
raise self._exception
File
".../airflow/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 473,
in __call__
result = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File
".../airflow/venv/lib/python3.11/site-packages/airflow/sdk/api/client.py", line
887, in request
return super().request(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../airflow/venv/lib/python3.11/site-packages/httpx/_client.py",
line 825, in request
return self.send(request, auth=auth,
follow_redirects=follow_redirects)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../airflow/venv/lib/python3.11/site-packages/httpx/_client.py",
line 914, in send
response = self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../airflow/venv/lib/python3.11/site-packages/httpx/_client.py",
line 942, in _send_handling_auth
response = self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../airflow/venv/lib/python3.11/site-packages/httpx/_client.py",
line 979, in _send_handling_redirects
response = self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../airflow/venv/lib/python3.11/site-packages/httpx/_client.py",
line 1014, in _send_single_request
response = transport.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
".../airflow/venv/lib/python3.11/site-packages/httpx/_transports/default.py",
line 249, in handle_request
with map_httpcore_exceptions():
File ".../python3.11/contextlib.py", line 155, in __exit__
self.gen.throw(typ, value, traceback)
File
".../airflow/venv/lib/python3.11/site-packages/httpx/_transports/default.py",
line 118, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectError: [Errno 111] Connection refused
[TIMESTAMP] {worker.py:166} INFO - SIGTERM received. Sending SIGTERM to
all jobs and quit
[TIMESTAMP] {worker.py:166} INFO - SIGTERM received. Sending SIGTERM to
all jobs and quit
[TIMESTAMP] {worker.py:166} INFO - SIGTERM received. Sending SIGTERM to
all jobs and quit
[TIMESTAMP] {worker.py:166} INFO - SIGTERM received. Sending SIGTERM to
all jobs and quit
[TIMESTAMP] {worker.py:166} INFO - SIGTERM received. Sending SIGTERM to
all jobs and quit
[TIMESTAMP] {worker.py:166} INFO - SIGTERM received. Sending SIGTERM to
all jobs and quit
[TIMESTAMP] {worker.py:166} INFO - SIGTERM received. Sending SIGTERM to
all jobs and quit
[TIMESTAMP] {worker.py:166} INFO - SIGTERM received. Sending SIGTERM to
all jobs and quit
[TIMESTAMP] {worker.py:166} INFO - SIGTERM received. Sending SIGTERM to
all jobs and quit
[TIMESTAMP] {worker.py:166} INFO - SIGTERM received. Sending SIGTERM to
all jobs and quit
[TIMESTAMP] {worker.py:372} INFO - Job completed: dag_id=<redacted_dag>
task_id=<redacted_task> run_id=manual__<redacted_timestamp> map_index=-1
try_number=4
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
![https://private-user-images.githubusercontent.com/95105677/583785573-c8aa1ed0-aeb2-452e-ba13-9194ee139a74.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NzcxNjExMzIsIm5iZiI6MTc3NzE2MDgzMiwicGF0aCI6Ii85NTEwNTY3Ny81ODM3ODU1NzMtYzhhYTFlZDAtYWViMi00NTJlLWJhMTMtOTE5NGVlMTM5YTc0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjA0MjUlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYwNDI1VDIzNDcxMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTI5YWI3NWEyODg0YTczYmMzMzhkZTg0MmNkYmFjNWRkNjZlZDQyMjk0ZjI0NmMxZWYxMmUwMWFhYzdkZmI3Y2MmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JnJlc3BvbnNlLWNvbnRlbnQtdHlwZT1pbWFnZSUyRnBuZyJ9.PMXBK6_TJ5bkGpcgTy7xLUvfQXHN3QnghiUG1keWCF4"](https://private-user-images.githubusercontent.com/95105677/583785573-c8aa1ed0-aeb2-452e-ba13-9194ee139a74.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NzcxNjExMzIsIm5iZiI6MTc3NzE2MDgzMiwicGF0aCI6Ii85NTEwNTY3Ny81ODM3ODU1NzMtYzhhYTFlZDAtYWViMi00NTJlLWJhMTMtOTE5NGVlMTM5YTc0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjA0MjUlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYwNDI1VDIzNDcxMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTI5YWI3NWEyODg0YTczYmMzMzhkZTg0MmNkYmFjNWRkNjZlZDQyMjk0ZjI0NmMxZWYxMmUwMWFhYzdkZmI3Y2MmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JnJlc3BvbnNlLWNvbnRlbnQtdHlwZT1pbWFnZSUyRnBuZyJ9.PMXBK6_TJ5bkGpcgTy7xLUvfQXHN3QnghiUG1keWCF4"){kind=link}