still, I think web2py fetches the entire set in memory . did you inspect how much memory in rocket it takes ?
On Tuesday, October 8, 2013 12:01:02 AM UTC+2, viniciusban wrote: > > Michele, I'm using the admin feature to export as csv. > > Niphlod, records are not big. With rocket it works nice. > > On Mon, Oct 7, 2013 at 6:17 PM, Michele Comitini > <michele....@gmail.com <javascript:>> wrote: > > There is not enough information to say much. > > if possible try to stream directly rather than buffering data first in a > > variable, this should make things faster and lighter. > > > > > > 2013/10/7 Niphlod <nip...@gmail.com <javascript:>> > >> > >> did you try upping the memory limits ? > >> limit-as, reload-on-as, reload-on-rss > >> > >> > >> PS: dumping 50k records in a csv is definitely not something for a > >> webserver (timeout, anyone ?) > >> > >> On Monday, October 7, 2013 10:46:34 PM UTC+2, viniciusban wrote: > >>> > >>> Hi guys. > >>> > >>> I'm experiencing a problem I think is not web2py fault, but uwsgi or > >>> nginx configuration. > >>> > >>> I'm running web2py inside a Vagrant box (Virtualbox) with nginx. > >>> > >>> I have a table with more than 50k lines and when I try to export as > >>> csv in admin, the generated file doesn't have all records. Each time > >>> it generates a > >>> different number of rows. > >>> > >>> > >>> Reading logs, I found something: > >>> > >>> in /var/log/nginx/error.log: > >>> 2013/10/07 20:25:15 [crit] 955#0: *291 open() > >>> "/var/lib/nginx/uwsgi/9/00/0000000009" failed (13: Permission denied) > >>> while reading upstream, client: 10.11.12.1, server: wsl, request: "GET > >>> /wsl/appadmin/csv/db?query=db.publicacao.id%3E0 HTTP/1.1", upstream: > >>> "uwsgi://unix:///tmp/web2py.socket:", host: "10.11.12.13", referrer: > >>> "https://10.11.12.13/wsl/appadmin/select/db?query=db.publicacao.id%3E0"; > > >>> > >>> > >>> in /var/log/uwsgi/uwsgi.log: > >>> SIGPIPE: writing to a closed pipe/socket/fd (probably the client > >>> disconnected) on request > >>> /wsl/appadmin/csv/db?query=db.publicacao.id%3E0 (ip 10.11.12.1) !!! > >>> uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 296] > >>> IOError: write error > >>> [pid: 2168|app: 0|req: 53/153] 10.11.12.1 () {46 vars in 1015 bytes} > >>> [Mon Oct 7 20:25:06 2013] GET > >>> /wsl/appadmin/csv/db?query=db.publicacao.id%3E0 => generated 0 bytes > >>> in 9064 msecs (HTTP/1.1 200) 7 headers in 371 bytes (4 switches on > >>> core 0) > >>> ...The work of process 2168 is done. Seeya! > >>> worker 4 killed successfully (pid: 2168) > >>> Respawned uWSGI worker 4 (new pid: 2326) > >>> mapping worker 4 to CPUs: 1 > >>> > >>> > >>> > >>> > >>> What is happening? It seems somehow the socket connection is being > >>> lost. Do you have an idea to fix that? > >>> > >>> BTW, I'm using webp2y v2.4.6 on Ubuntu 12.04 server and it was > configured > >>> by > >>> setup-web2py-nginx-uwsgi-ubuntu.sh distributed along with web2py. > >>> > >>> My /etc/uwsgi/web2py.ini file is: > >>> [uwsgi] > >>> socket = /tmp/web2py.socket > >>> pythonpath = /home/vagrant/web2py/ > >>> mount = /=wsgihandler:application > >>> processes = 4 > >>> master = true > >>> harakiri = 60 > >>> reload-mercy = 8 > >>> cpu-affinity = 1 > >>> stats = /tmp/stats.socket > >>> max-requests = 2000 > >>> limit-as = 512 > >>> reload-on-as = 256 > >>> reload-on-rss = 192 > >>> uid = vagrant > >>> gid = vagrant > >>> cron = 0 0 -1 -1 -1 python /home/vagrant/web2py/web2py.py -Q -S > >>> welcome -M -R scripts/sessions2trash.py -A -o > >>> no-orphans = true > >>> > >>> > >>> > >>> Any help? > >> > >> -- > >> Resources: > >> - http://web2py.com > >> - http://web2py.com/book (Documentation) > >> - http://github.com/web2py/web2py (Source code) > >> - https://code.google.com/p/web2py/issues/list (Report Issues) > >> --- > >> You received this message because you are subscribed to the Google > Groups > >> "web2py-users" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an > >> email to web2py+un...@googlegroups.com <javascript:>. > >> For more options, visit https://groups.google.com/groups/opt_out. > > > > > > -- > > Resources: > > - http://web2py.com > > - http://web2py.com/book (Documentation) > > - http://github.com/web2py/web2py (Source code) > > - https://code.google.com/p/web2py/issues/list (Report Issues) > > --- > > You received this message because you are subscribed to the Google > Groups > > "web2py-users" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to web2py+un...@googlegroups.com <javascript:>. > > For more options, visit https://groups.google.com/groups/opt_out. > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
