The GitHub Actions job "Tests AMD" on airflow.git/v3-1-test has failed. Run started by GitHub user kaxil (triggered by kaxil).
Head commit for run: 1d9e6c0fc96bb6b507481b4d721c7f0c350d6981 / Kaxil Naik <[email protected]> Fix scheduler ``serve_logs`` subprocess file descriptor errors (#55443) Remove workers parameter from `uvicorn.run()` in `serve_logs` to fix file descriptor errors when scheduler starts serve_logs as a subprocess. The original implementation used `workers=2` (copied from Gunicorn) but this caused multiprocessing issues in containerized environments. Also implemented lazy app loading via `get_app()` function for better initialization order and architectural consistency with main API. The primary fix addresses: OSError: [Errno 9] Bad file descriptor. Secondary improvement ensures proper initialization timing. This regression was introduced in [#52581](https://github.com/apache/airflow/pull/52581) when serve_logs was refactored from Flask to FastAPI. --- Without this fix, we see following error when running `airflow standalone` fresh in an isolated container: Error 1: ``` scheduler | Traceback (most recent call last): scheduler | File "/.venv/bin/airflow", line 10, in <module> scheduler | sys.exit(main()) scheduler | File "/.venv/lib/python3.10/site-packages/airflow/__main__.py", line 55, in main scheduler | args.func(args) scheduler | File "/.venv/lib/python3.10/site-packages/airflow/cli/cli_config.py", line 49, in command scheduler | return func(*args, **kwargs) scheduler | File "/.venv/lib/python3.10/site-packages/airflow/utils/cli.py", line 114, in wrapper scheduler | return f(*args, **kwargs) scheduler | File "/.venv/lib/python3.10/site-packages/airflow/utils/providers_configuration_loader.py", line 54, in wrapped_function scheduler | return func(*args, **kwargs) scheduler | File "/.venv/lib/python3.10/site-packages/airflow/cli/commands/scheduler_command.py", line 52, in scheduler scheduler | run_command_with_daemon_option( scheduler | File "/.venv/lib/python3.10/site-packages/airflow/cli/commands/daemon_utils.py", line 86, in run_command_with_daemon_option scheduler | callback() scheduler | File "/.venv/lib/python3.10/site-packages/airflow/cli/commands/scheduler_command.py", line 55, in <lambda> scheduler | callback=lambda: _run_scheduler_job(args), scheduler | File "/.venv/lib/python3.10/site-packages/airflow/cli/commands/scheduler_command.py", line 42, in _run_scheduler_job scheduler | with _serve_logs(args.skip_serve_logs), _serve_health_check(enable_health_check): scheduler | File "/usr/local/lib/python3.10/contextlib.py", line 135, in __enter__ scheduler | return next(self.gen) scheduler | File "/.venv/lib/python3.10/site-packages/airflow/cli/commands/scheduler_command.py", line 62, in _serve_logs scheduler | from airflow.utils.serve_logs import serve_logs scheduler | File "/.venv/lib/python3.10/site-packages/airflow/utils/serve_logs/__init__.py", line 20, in <module> scheduler | from airflow.utils.serve_logs.log_server import create_app scheduler | File "/.venv/lib/python3.10/site-packages/airflow/utils/serve_logs/log_server.py", line 160, in <module> scheduler | app = create_app() scheduler | File "/.venv/lib/python3.10/site-packages/airflow/utils/serve_logs/log_server.py", line 153, in create_app scheduler | JWTAuthStaticFiles(directory=log_directory, html=False), scheduler | File "/.venv/lib/python3.10/site-packages/airflow/utils/serve_logs/log_server.py", line 49, in __init__ scheduler | super().__init__(*args, **kwargs) scheduler | File "/.venv/lib/python3.10/site-packages/starlette/staticfiles.py", line 56, in __init__ scheduler | raise RuntimeError(f"Directory '{directory}' does not exist") scheduler | RuntimeError: Directory '/root/airflow/logs' does not exist ``` Error 2: This was because we had harcoded 2 workers! but this isn't needed ``` scheduler | Process SpawnProcess-1:53: scheduler | Traceback (most recent call last): scheduler | File "/usr/local/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap scheduler | self.run() scheduler | File "/usr/local/lib/python3.10/multiprocessing/process.py", line 108, in run scheduler | self._target(*self._args, **self._kwargs) scheduler | File "/.venv/lib/python3.10/site-packages/uvicorn/_subprocess.py", line 73, in subprocess_started scheduler | sys.stdin = os.fdopen(stdin_fileno) # pragma: full coverage scheduler | File "/usr/local/lib/python3.10/os.py", line 1030, in fdopen scheduler | return io.open(fd, mode, buffering, encoding, *args, **kwargs) scheduler | OSError: [Errno 9] Bad file descriptor scheduler | Process SpawnProcess-1:54: scheduler | Traceback (most recent call last): scheduler | File "/usr/local/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap scheduler | self.run() scheduler | File "/usr/local/lib/python3.10/multiprocessing/process.py", line 108, in run scheduler | self._target(*self._args, **self._kwargs) scheduler | File "/.venv/lib/python3.10/site-packages/uvicorn/_subprocess.py", line 73, in subprocess_started scheduler | sys.stdin = os.fdopen(stdin_fileno) # pragma: full coverage scheduler | File "/usr/local/lib/python3.10/os.py", line 1030, in fdopen scheduler | return io.open(fd, mode, buffering, encoding, *args, **kwargs) scheduler | OSError: [Errno 9] Bad file descriptor triggerer | Process SpawnProcess-1:53: triggerer | Traceback (most recent call last): triggerer | File "/usr/local/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap triggerer | self.run() triggerer | File "/usr/local/lib/python3.10/multiprocessing/process.py", line 108, in run triggerer | self._target(*self._args, **self._kwargs) triggerer | File "/.venv/lib/python3.10/site-packages/uvicorn/_subprocess.py", line 73, in subprocess_started triggerer | sys.stdin = os.fdopen(stdin_fileno) # pragma: full coverage triggerer | File "/usr/local/lib/python3.10/os.py", line 1030, in fdopen triggerer | return io.open(fd, mode, buffering, encoding, *args, **kwargs) triggerer | OSError: [Errno 9] Bad file descriptor ``` Report URL: https://github.com/apache/airflow/actions/runs/17597165789 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
