Re: [Trac] Uploading files for Download
wrote on 2015-Jan-07 at 08:15 PM:>
File "/usr/lib/python2.7/rfc822.py", line 155, in readheaders
> line = self.fp.readline()
>File "/usr/lib/python2.7/site-packages/trac/web/modpython_frontend.py",
> line 52, in readline
> return self.req.readline(size)
> MemoryError
It seems that to be a mod_python issue. I believe mod_wsgi doesn't
have the issue.
Try to switch mod_wsgi. Otherwise, please try the following patch
(sorry, untested).
diff --git a/trac/web/modpython_frontend.py b/trac/web/modpython_frontend.py
index 60daa89..bddc0ae 100644
--- a/trac/web/modpython_frontend.py
+++ b/trac/web/modpython_frontend.py
@@ -39,6 +39,8 @@ from trac.web.wsgi import WSGIGateway, _ErrorsWrapper
class InputWrapper(object):
+CHUNK_SIZE = 8192
+
def __init__(self, req):
self.req = req
@@ -49,7 +51,17 @@ class InputWrapper(object):
return self.req.read(size)
def readline(self, size=-1):
-return self.req.readline(size)
+if size >= 0:
+return self.req.readline(size)
+else:
+# avoid MemoryError when an uploaded file is huge
+buf = []
+while True:
+line = self.req.readline(self.CHUNK_SIZE)
+buf.append(line)
+if not line or line.endswith('\n'):
+break
+return ''.join(buf)
def readlines(self, hint=-1):
return self.req.readlines(hint)
--
Jun Omae (大前 潤)
--
You received this message because you are subscribed to the Google Groups "Trac
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to trac-users+unsubscr...@googlegroups.com.
To post to this group, send email to trac-users@googlegroups.com.
Visit this group at http://groups.google.com/group/trac-users.
For more options, visit https://groups.google.com/d/optout.
Re: [Trac] Uploading files for Download
No idea. But since it is most likely using the http protocol for this, I would not be surprised if all data is in memory buffers at some point. I am trying to see how best to set it up so I can update a file outside trac and then just update the trac database with the new information on the file. I would hope then the same problem would not occur when sending back a big file. If so, then I am going to have a problem... On Tue, Jan 13, 2015 at 10:14 PM, Steffen Hoffmann wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 07.01.2015 12:15, Roger Oberholtzer wrote: >> line 52, in readline >> return self.req.readline(size) >> MemoryError > > I guess that you'll need quite a bit more RAM than the biggest file you > try to upload on that machine. But this shouldn't be the problem in your > case. > > May it be, that Python simply has not permission (process memory quota > or similar restriction) to reserve enough memory for buffering the > received data stream in memory? > > Steffen Hoffmann > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.12 (GNU/Linux) > > iEYEARECAAYFAlS1irQACgkQ31DJeiZFuHfmqgCg5r1qtFmJxL1dTK69hEfNLdoh > +LoAnRWS05u63vQcWw9ihqNCF+ZL7mdV > =mErl > -END PGP SIGNATURE- > > -- > You received this message because you are subscribed to the Google Groups > "Trac Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to trac-users+unsubscr...@googlegroups.com. > To post to this group, send email to trac-users@googlegroups.com. > Visit this group at http://groups.google.com/group/trac-users. > For more options, visit https://groups.google.com/d/optout. -- Roger Oberholtzer -- You received this message because you are subscribed to the Google Groups "Trac Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscr...@googlegroups.com. To post to this group, send email to trac-users@googlegroups.com. Visit this group at http://groups.google.com/group/trac-users. For more options, visit https://groups.google.com/d/optout.
Re: [Trac] Uploading files for Download
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07.01.2015 12:15, Roger Oberholtzer wrote: > line 52, in readline > return self.req.readline(size) > MemoryError I guess that you'll need quite a bit more RAM than the biggest file you try to upload on that machine. But this shouldn't be the problem in your case. May it be, that Python simply has not permission (process memory quota or similar restriction) to reserve enough memory for buffering the received data stream in memory? Steffen Hoffmann -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlS1irQACgkQ31DJeiZFuHfmqgCg5r1qtFmJxL1dTK69hEfNLdoh +LoAnRWS05u63vQcWw9ihqNCF+ZL7mdV =mErl -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "Trac Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscr...@googlegroups.com. To post to this group, send email to trac-users@googlegroups.com. Visit this group at http://groups.google.com/group/trac-users. For more options, visit https://groups.google.com/d/optout.
[Trac] Uploading files for Download
I am trying to upload a file into trac that will be accessed via the
DownloadsPlugin (http://trac-hacks.org/wiki/DownloadsPlugin). The
files in question are 750MB. When I try to upload them, I get the
error below. In the [downloads] section of my trac.ini, I have:
max_size = 8
I do have files that are bigger than 450 MB. But the 750 MB files are
problematic. I know that when the max_size value was too small, I got
a different error. So I am suspecting something outside of Trac. But I
am not sure. Trac is run via apache on an openSUSE Linux server.
Oops…
Trac detected an internal error:
MemoryError:
Python Traceback
Most recent call last:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/trac/web/main.py", line 512,
in _dispatch_request
dispatcher.dispatch(req)
File "/usr/lib/python2.7/site-packages/trac/web/main.py", line 186,
in dispatch
self._pre_process_request(req, chosen_handler)
File "/usr/lib/python2.7/site-packages/trac/web/main.py", line 333,
in _pre_process_request
chosen_handler = filter_.pre_process_request(req, chosen_handler)
File
"/usr/lib/python2.7/site-packages/TracDateField-1.1.0_r0-py2.7.egg/datefield/filter.py",
line 164, in pre_process_request
if req.args.get('type') == 'date':
File "/usr/lib/python2.7/site-packages/trac/web/api.py", line 353,
in __getattr__
value = self.callbacks[name](self)
File "/usr/lib/python2.7/site-packages/trac/web/api.py", line 336, in
'args': lambda req: arg_list_to_args(req.arg_list),
File "/usr/lib/python2.7/site-packages/trac/web/api.py", line 353,
in __getattr__
value = self.callbacks[name](self)
File "/usr/lib/python2.7/site-packages/trac/web/api.py", line 694,
in _parse_arg_list
fs = _FieldStorage(fp, environ=self.environ, keep_blank_values=True)
File "/usr/lib/python2.7/cgi.py", line 508, in __init__
self.read_multi(environ, keep_blank_values, strict_parsing)
File "/usr/lib/python2.7/site-packages/trac/web/api.py", line 206,
in read_multi
cgi.FieldStorage.read_multi(self, *args, **kwargs)
File "/usr/lib/python2.7/cgi.py", line 635, in read_multi
headers = rfc822.Message(self.fp)
File "/usr/lib/python2.7/rfc822.py", line 108, in __init__
self.readheaders()
File "/usr/lib/python2.7/rfc822.py", line 155, in readheaders
line = self.fp.readline()
File "/usr/lib/python2.7/site-packages/trac/web/modpython_frontend.py",
line 52, in readline
return self.req.readline(size)
MemoryError
--
Roger Oberholtzer
--
You received this message because you are subscribed to the Google Groups "Trac
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to trac-users+unsubscr...@googlegroups.com.
To post to this group, send email to trac-users@googlegroups.com.
Visit this group at http://groups.google.com/group/trac-users.
For more options, visit https://groups.google.com/d/optout.
