jvstein commented on code in PR #68304:
URL: https://github.com/apache/airflow/pull/68304#discussion_r3396540047
##########
airflow-core/src/airflow/config_templates/config.yml:
##########
@@ -779,6 +779,18 @@ logging:
type: string
example: ~
default: "False"
+ object_store_write_mode:
Review Comment:
I reviewed the other remote loggers and found these issues regarding one or
both aspects of these PRs.
- `gcs_task_handler.py` and `wasb_task_handler.py` both have the same
download/append problem as `s3_task_handler.py`
- `oss_task_handler.py` appends the full local log file using a byte based
append call; on a shared log filesystem it duplicates logs, but is unlikely to
cause memory issues for the worker
- `hdfs_task_handler.py` uploads the local file with no append logic, so it
would lose logs if a shared logging volume wasn't used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
