[ https://issues.apache.org/jira/browse/AIRAVATA-3679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17647250#comment-17647250 ]
ASF subversion and git services commented on AIRAVATA-3679: ----------------------------------------------------------- Commit 4726a828753ae0ea2be154af22f7e14a85a3baa3 in airavata-django-portal's branch refs/heads/dependabot/npm_and_yarn/django_airavata/apps/admin/moment-2.29.4 from Marcus Christie [ https://gitbox.apache.org/repos/asf?p=airavata-django-portal.git;h=4726a828 ] AIRAVATA-3679 Upgrade to sdk 1.7.0 > directory download error: Zip file size exceeds max of 1073741824 bytes > ----------------------------------------------------------------------- > > Key: AIRAVATA-3679 > URL: https://issues.apache.org/jira/browse/AIRAVATA-3679 > Project: Airavata > Issue Type: Bug > Components: Django Portal > Reporter: Marcus Christie > Assignee: Marcus Christie > Priority: Major > > When a directory is downloaded in the Django portal, the directory is first > converted to a zip file and then that zip file is returned in the response. > Because the zip file has to be created first, as a temporary file, there is a > limit on how large it can be so it doesn't overfill the temporary storage > space (usually the same as the root partition). To prevent a disk space > overfill there is a cap on directory downloads of 1GB. > Whenever a directory zip download file goes above 1GB, the *Zip file size > exceeds max of 1073741824 bytes* error is generated. > Initially I thought I could do a check first if the directory is larger than > 1GB and then disallow download. But a directory might be larger than 1GB and > yet compress to much less than 1GB. It's generally hard to know in advance > how large a zipped directory will be. > Another approach is to eliminate the temporary file creation and just stream > the zip file. That is, write the zip file directly to the HTTP response > instead of having a temporary file intermediary. This is light on memory > usage and disk usage and also benefits the user by starting the download > sooner. So I'm going to explore this option. -- This message was sent by Atlassian Jira (v8.20.10#820010)