All depends on whether;
- you have enough memory
- you are going to enforce the file size
- you have appropriate resource limitations in place on the server
- you are going to enforce the maximum number of processing requests
etc etc...
Personally, if all those conditions were met, I'd probably do it in memory
to avoid the unnecessary disk IO overhead.. but it all depends on what you
need :)
On Tue, Nov 2, 2010 at 1:00 PM, wawa wawawa wrote:
> Hi,
>
> I am writing an app to "apply some rainbows and unicorns" to a bunch
> of uploaded XML files (which can be uploaded as a compressed tar or
> zip file). Celery and rabbitmq will be used for the asynchronous tasks
> with jQuery (maybe with dajaxproject) to get updates on progress to
> the browser.
>
> I'm at the point where I'm passing the uploaded files to my offline
> tasks and using cStringIO with both the tarfile and zipfile modules to
> handle the whole compressed files as strings and then saving the
> decompressed XML files in an array of strings to be later manhandled
> by cElementTree.
>
> There is a closed audience for this app and uncompressed filesizes
> will never be more than 3 or 4 MB (mostly they'll under 100KB). I
> don't expect this to be used by more than a handful of people per day
> (but it will save _alot_ of man hours).
>
> Is it madness to try to do this all in memory?
>
> Should I write out the file contents to the filesystem first using
> "for i in uploadedfile.chunks()"?
>
> Does anyone know of any open Django apps that do anything similar
> (upload a file, get presented with the results) so I can have a look?
>
> Thanks in advance
>
> W
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To post to this group, send email to django-us...@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.
>
>
--
Cal Leeming
Operational Security & Support Team
*Out of Hours: *+44 (07534) 971120 | *Support Tickets: *
supp...@simplicitymedialtd.co.uk
*Fax: *+44 (02476) 578987 | *Email: *cal.leem...@simplicitymedialtd.co.uk
*IM: *AIM / ICQ / MSN / Skype (available upon request)
Simplicity Media Ltd. All rights reserved.
Registered company number 7143564
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
