he-gong-sardine commented on PR #36545: URL: https://github.com/apache/airflow/pull/36545#issuecomment-1913606077
> > * High Memory (Over 100MB): It is recommended to use Google's upload_from_file method, which is more robust and includes a retry feature. This method is preferable when sufficient memory is available. > > > * Low Memory: Lower the stream_chunk_size and employ the custom method for streaming. This approach transfers smaller chunks of data, which is advantageous for systems with limited memory. > > Great initiative. I believe compute memory (100MB) is not a problem anymore with the latest computing technology. So, I don't think we need to implement it for the low-memory use case. thanks. I removed low-memory use case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
