Thanks for the responses, guys.
Regarding my deployment, I've been doing my development on my local machine
using manage.py. I'll give this a go in WSGI and see what happens.
Graham, I totally agree that there are ways to get around my problem outside
of trying to send HTTP headers before the message body. A simple way would
be to give the user a "please wait" page while the tarball is generated.
I'm mostly just lazy and don't want to do that, haha. That said, thanks for
the suggestion.
Rick, the issue with your suggestion is that I need to actually create the
tarball, which takes time. I'm trying to avoid having the user wait while
the tarball is created by giving them the download dialogue before
generation happens. Using your BigTarFileWrapper assumes that the tarball
has been created already, at least from what I can tell.
So again, I'll give this a go in WSGI and see what happens. In the
meantime, I suppose the empty gzipped file gets the job done.
Thanks again for all the responses. Go Django!
Alex
On Wed, Apr 15, 2009 at 8:36 AM, Rick Wagner wrote:
>
>
>
> On Apr 14, 6:55 pm, Graham Dumpleton
> wrote:
> > On Apr 15, 7:49 am, Alex Loddengaard wrote:
> >
> >
> >
> > > I've found several messages on this list discussing ways to send large
> files
> > > in a HttpResponse. One can use FileWrapper, or one can use a generator
> and
> > > yield chunks of the large file. What about the case when the large
> file is
> > > generated at HTTP request time? In this case, it would be annoying to
> have
> > > the user wait for the page to generate the large file and then stream
> the
> > > file. Instead we would want a way to start the HTTP response (so that
> the
> > > user gets the download dialogue), generate the large file, and then
> stream
> > > the file. Let's take the following example:
> >
> > > def create_tarball():
> >
> > > > path = create_some_big_tarball()
> >
> > > > chunk = None
> > > > fh = open(path, 'r')
> > > > while True:
> > > > chunk = fh.read(1024 * 128)
> > > > if chunk == '':
> > > > break
> > > > yield chunk
> >
> > > > def sample_view(request):
> > > > response = HttpResponse(create_tarball(),
> > > > mimetype='application/x-compressed')
> > > > response['Content-Disposition'] =
> "attachment;filename=mytarball.tar.gz"
> >
> > > The above example nearly accomplishes what we want, but it doesn't
> start the
> > > HTTP response before the tarball is created, hence making the user wait
> a
> > > long time before the download dialogue box shows up. Let's try
> something
> > > like this (notice the addition of a noop yield):
> >
> > > def create_tarball():
> >
> > > yield '' # noop to send the HTTP headers
> >
> > > > path = create_some_big_tarball()
> >
> > > > chunk = None
> > > > fh = open(path, 'r')
> > > > while True:
> > > > chunk = fh.read(1024 * 128)
> > > > if chunk == '':
> > > > break
> > > > yield chunk
> >
> > > > def sample_view(request):
> > > > response = HttpResponse(create_tarball(),
> > > > mimetype='application/x-compressed')
> > > > response['Content-Disposition'] =
> "attachment;filename=mytarball.tar.gz"
> >
> > > The issue with the above example is that the "yield ''" seems to be
> > > ignored. HTTP headers are not sent before the tarball is created.
> > > Similarly, "yield ' '" and "yield None" don't work, because they
> corrupt the
> > > tarball (HttpResponse calls str() on the iterable items given to the
> > > HttpResponse constructor). As a temporary solution, we're writing an
> empty
> > > gzip file in the first yield. Our large tarball is gzipped, and since
> gzip
> > > files can be concatenated to one and other, our hack seems to be
> working.
> > > In the above example, replace the first "yield ''" with:
> >
> > > noop = StringIO.StringIO()
> >
> > > > empty = gzip.GzipFile(mode='w', fileobj=noop)
> > > > empty.write("")
> > > > empty.close()
> > > > yield noop.getvalue()
> >
> > &g
