AutomationDev85 commented on code in PR #65957:
URL: https://github.com/apache/airflow/pull/65957#discussion_r3166802620
##########
airflow-core/src/airflow/config_templates/airflow_local_settings.py:
##########
@@ -159,7 +159,12 @@ def _default_conn_name_from(mod_path, hook_name):
"logging/remote_task_handler_kwargs must be a JSON object (a
python dict), we got "
f"{type(remote_task_handler_kwargs)}"
)
- _handler_kwargs = cast("dict[str, Any]", remote_task_handler_kwargs)
+ _all_kwargs = cast("dict[str, Any]", remote_task_handler_kwargs)
+ # Params accepted by FileTaskHandler.__init__ (excluding base_log_folder,
set separately).
+ # Everything else is forwarded to the RemoteLogIO constructor.
+ _FILE_HANDLER_PARAMS = frozenset({"max_bytes", "backup_count", "delay"})
Review Comment:
Hard-coding these parameters is not a good approach. Different task handlers
may require different parameter sets, so the configuration should be handled in
a more generic way. The parameter selection should depend on the configured
task handler rather than being fixed in the code.
##########
airflow-core/src/airflow/config_templates/airflow_local_settings.py:
##########
@@ -174,10 +179,10 @@ def _default_conn_name_from(mod_path, hook_name):
"remote_base": remote_base_log_folder,
"delete_local_copy": delete_local_copy,
}
- | _handler_kwargs,
+ | _io_kwargs,
)
)
- _handler_kwargs = {}
+ _io_kwargs = {}
Review Comment:
All these _io_kwargs = {} lines can be removed as the variable is not used
any more in code below.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
