dilnazanlid commented on code in PR #73118:
URL: https://github.com/apache/airflow/pull/73118#discussion_r4004041274
##########
task-sdk/src/airflow/sdk/importers/python_importer.py:
##########
@@ -78,41 +135,43 @@ def can_handle(self, definition: DagDefinition | str |
Path) -> bool:
def list_dag_definitions(
self,
bundle: BaseDagBundle,
- *,
- safe_mode: bool = True,
- ) -> Iterator[DagDefinition]:
- """List Python DAG definitions in a bundle matching supported
extensions."""
- yield from find_file_dag_definitions(bundle.path,
self.supported_extensions, safe_mode=safe_mode)
+ ) -> Iterator[FileDagDefinition | ZipFileDagDefinition]:
+ """
+ List Python DAG files in a bundle matching supported extensions.
+
+ This does not look for members inside zip archives. Definition
discovery
+ of zip archives is done inside :class:`..zip_importer.ZipImporter`
+ instead.
+ """
+ if bundle.path.is_dir():
+ yield from find_file_dag_definitions(bundle.path,
self.supported_extensions)
Review Comment:
I am not sure here, but the `list_dag_definitions` method is said to return
`FileDagDefinition | ZipFileDagDefinition`, but `find_file_dag_definitions`
method in `base.py` only returns `FileDagDefinition` now.
Before, `find_file_dag_definitions` could return all `DagDefinition`s, but
now it will return nothing for python files inside the zip (and overall for all
zip files as the `find_file_dag_definitions` is used in the `ZipImporter`'s
`list_dag_definitions` too).
Should `ZipFileDagDefinition` be added into the `find_file_dag_definitions`
as return type? Or maybe make `ZipFileDagDefinition` subclass of
`FileDagDefinition` but I am not sure about it, up to you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
