matejmurin01 opened a new issue, #37200: URL: https://github.com/apache/airflow/issues/37200
### Apache Airflow version 2.8.1 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? As per [pathlib documentation](https://docs.python.org/3/library/pathlib.html#pathlib.Path.mkdir): `Create a new directory at this given path. If mode is given, it is combined with the process’ umask value to determine the file mode and access flags. If the path already exists, [FileExistsError](https://docs.python.org/3/library/exceptions.html#FileExistsError) is raised. If parents is true, any missing parents of this path are created as needed; they are created with the default permissions without taking mode into account (mimicking the POSIX mkdir -p command).` Only the leaf folder is created with the specified permisions. Therefore, if logging structure is specified and contains other than leaf folders (such as the default one), the permissions to not get propagated, resulting in a **permission denied** error if, for example, a celery worker running on a different machine (while in the same group with same GID) tries to write to a directory already created by a task created by a worker on a different machine. ### What you think should happen instead? _No response_ ### How to reproduce 1. Use file logging in airflow. 2. Use the default formatting of log files. 3. Create a worker on a remote machine (in my case via celery on redis) with its specific queue. 4. Run any task on this queue. Two things can happen: a.) If this task is the first in the DAG, other workers won't be able to write into logs of this attempt b.) If this task is not the first in the DAG, then the logs of this task won't be able to be written. ### Operating System Debian GNU/Linux 11 (bullseye) ### Versions of Apache Airflow Providers _No response_ ### Deployment Virtualenv installation ### Deployment details _No response_ ### Anything else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
