GrumpyCat51 opened a new issue, #49897:
URL: https://github.com/apache/airflow/issues/49897
### Official Helm Chart version
1.16.0 (latest released)
### Apache Airflow version
3.0.0
### Kubernetes Version
Minikube v1.35.0
### Helm Chart configuration
```
executor: "CeleryKubernetesExecutor"
```
### Docker Image customizations
_No response_
### What happened
When selecting the CeleryKubernetesOperator as specified in the comment just
above the `executor` setting, both the scheduler as well as the celery worker
don't start as the executor cannot be loaded properly:
```
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/module_loading.py",
line 35, in import_string
module_path, class_name = dotted_path.rsplit(".", 1)
^^^^^^^^^^^^^^^^^^^^^^^
ValueError: not enough values to unpack (expected 2, got 1)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 8, in <module>
sys.exit(main())
^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/__main__.py", line
55, in main
args.func(args)
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/cli_config.py",
line 48, in command
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/cli.py", line
111, in wrapper
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/providers_configuration_loader.py",
line 55, in wrapped_function
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/scheduler_command.py",
line 52, in scheduler
run_command_with_daemon_option(
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/daemon_utils.py",
line 86, in run_command_with_daemon_option
callback()
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/scheduler_command.py",
line 55, in <lambda>
callback=lambda: _run_scheduler_job(args),
^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/scheduler_command.py",
line 42, in _run_scheduler_job
with _serve_logs(args.skip_serve_logs),
_serve_health_check(enable_health_check):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/contextlib.py", line 137, in __enter__
return next(self.gen)
^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/scheduler_command.py",
line 65, in _serve_logs
executor_class, _ = ExecutorLoader.import_default_executor_cls()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/executors/executor_loader.py",
line 299, in import_default_executor_cls
executor, source = cls.import_executor_cls(executor_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/executors/executor_loader.py",
line 289, in import_executor_cls
return import_string(executor_name.module_path),
executor_name.connector_source
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/module_loading.py",
line 37, in import_string
raise ImportError(f"{dotted_path} doesn't look like a module path")
ImportError: CeleryKubernetesExecutor doesn't look like a module path
```
In version 2.10.5, the mapping of executors was set here:
https://github.com/apache/airflow/blob/2.10.5/airflow/executors/executor_loader.py#L63-L74
In version 3.0.0, this is severly reduced to
https://github.com/apache/airflow/blob/3.0.0/airflow-core/src/airflow/executors/executor_loader.py#L56-L61
I'd be happy to create a PR for this, but I'm not quite sure how it was
intended to look like. Should the `CeleryKubernetesExecutor` be dropped in
favor of specifying both executors individually, are they simply missing from
in version 3.0.0, or should the logic figure out that
`CeleryKubernetesExecutor` should split into `CeleryExecutor` and
`KubernetesExecutor`?
### What you think should happen instead
_No response_
### How to reproduce
Run the helm chart with `executor: "CeleryKubernetesExecutor"`
### Anything else
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
