[web2py] Re: rocket performance issues

[mdipierro]

It is easy to fix things when problems are clearly reported. ;-)

On Jul 29, 4:38 pm, Michael Ellis  wrote:
> Seems to be.  I updated, restarted, and restored the colon after the
> comment.  No errors when I load the page.  Thanks for fixing it so quickly!
>
> On Thu, Jul 29, 2010 at 5:21 PM, mdipierro  wrote:
> > Can you check if this issue is solved in trunk?
>
> > Massimo
>
> > On Jul 29, 7:55 am, Michael Ellis  wrote:
> > > Has anything changed at tip in the view parser? Looks like there may
> > > be a problem now with colons at the end of comment lines. I have a
> > > view with the following that was working fine before I updated this
> > > morning.
>
> > > {{
> > > ## On entry, vars returned by controller are:
> > > ## problem, c, s, sc, wsc, problemid, currenthash, nparticipants
> > > bulbicon = URL(r=request, c="static", f="lightbulb32.png")
> > > gaugeicon = URL(r=request, c='static', f="gauge.png")
>
> > > ...
>
> > > *After updating it started issuing tickets about "Missing pass in
> > > view" and the ticket listing showed an unexpected indentation. *
>
> > > *
> > > *
>
> > > ## On entry, vars returned by controller are:
> > >     ## problem, c, s, sc, wsc, problemid, currenthash, nparticipants
> > >     bulbicon = URL (r=request
> > > , c="static", f="lightbulb32.png")
> > >     gaugeicon = URL (r=request
> > > , c='static', f="gauge.png")
>
> > > ...
>
> > > Removing the colon at the end of the comment fixed it.
>
> > > *
> > > *
>
> > > On Thu, Jul 29, 2010 at 8:19 AM, Michael Ellis <
> > michael.f.el...@gmail.com>wrote:
>
> > > > Testing under way...
>
> > > > (Iceberg, I'm running OS X 10.6.4)
>
> > > > Cheers,
> > > > Mike
>
> > > > On Thu, Jul 29, 2010 at 2:57 AM, mdipierro  > >wrote:
>
> > > >> Mike,
>
> > > >> I modified the expire_sessions.py in trunk and added a
> > > >> try:...execept... If the problem was that a file was locked and the
> > > >> cron task got stuck, it may be solved now. Can you test it?
>
> > > >> Massimo
>
> > > >> On Jul 28, 4:10 pm, MikeEllis  wrote:
> > > >> > Update: (Summarizes a couple of emails exchanged with Massimo)
>
> > > >> > Yesterday I found that my instance of web2py appeared be "leaking"
> > > >> > processes at about 1 per hour.  Massimo suggested running with -N to
> > > >> > disable cron.  I started two instances this way -- one under winpdb
> > > >> > and another directly.  They've been running almost 24 hours now and
> > > >> > neither has added any processes.  As noted earlier, my app has
> > nothing
> > > >> > in crontab. The admin and examples apps have the crontabs that are
> > "as
> > > >> > shipped" in v1.85.
>
> > > >> > Cheers,
> > > >> > Mike
>
> > > >> > On Jul 27, 12:17 pm, Jonathan Lundell  wrote:
>
> > > >> > > On Jul 27, 2010, at 9:01 AM, Michael Ellis wrote:
>
> > > >> > > > Thanks for the info about sloth. I may try it later.  It was
> > pretty
> > > >> easy to get what I needed with ipython:
>
> > > >> > > > procs = !ps ax | grep "web2py"
> > > >> > > > pids = [r[0] for r in procs.fields()[0:-2]]
> > > >> > > > for p in pids:
> > > >> > > >      print p
> > > >> > > >      !lsof | grep $p | wc
>
> > > >> > > Also: sudo launchctl limit maxfiles
>
> > > >> > > I see 256 by default. The second limit is the "hard limit", beyond
> > > >> which you can't configure (unlimited in this case).
>
> >http://serverfault.com/questions/15564/where-are-the-default-ulimits-.
> > > >> ..

Re: [web2py] Re: rocket performance issues

[Michael Ellis]

Seems to be.  I updated, restarted, and restored the colon after the
comment.  No errors when I load the page.  Thanks for fixing it so quickly!



On Thu, Jul 29, 2010 at 5:21 PM, mdipierro  wrote:

> Can you check if this issue is solved in trunk?
>
> Massimo
>
> On Jul 29, 7:55 am, Michael Ellis  wrote:
> > Has anything changed at tip in the view parser? Looks like there may
> > be a problem now with colons at the end of comment lines. I have a
> > view with the following that was working fine before I updated this
> > morning.
> >
> > {{
> > ## On entry, vars returned by controller are:
> > ## problem, c, s, sc, wsc, problemid, currenthash, nparticipants
> > bulbicon = URL(r=request, c="static", f="lightbulb32.png")
> > gaugeicon = URL(r=request, c='static', f="gauge.png")
> >
> > ...
> >
> > *After updating it started issuing tickets about "Missing pass in
> > view" and the ticket listing showed an unexpected indentation. *
> >
> > *
> > *
> >
> > ## On entry, vars returned by controller are:
> > ## problem, c, s, sc, wsc, problemid, currenthash, nparticipants
> > bulbicon = URL (r=request
> > , c="static", f="lightbulb32.png")
> > gaugeicon = URL (r=request
> > , c='static', f="gauge.png")
> >
> > ...
> >
> > Removing the colon at the end of the comment fixed it.
> >
> > *
> > *
> >
> > On Thu, Jul 29, 2010 at 8:19 AM, Michael Ellis <
> michael.f.el...@gmail.com>wrote:
> >
> > > Testing under way...
> >
> > > (Iceberg, I'm running OS X 10.6.4)
> >
> > > Cheers,
> > > Mike
> >
> > > On Thu, Jul 29, 2010 at 2:57 AM, mdipierro  >wrote:
> >
> > >> Mike,
> >
> > >> I modified the expire_sessions.py in trunk and added a
> > >> try:...execept... If the problem was that a file was locked and the
> > >> cron task got stuck, it may be solved now. Can you test it?
> >
> > >> Massimo
> >
> > >> On Jul 28, 4:10 pm, MikeEllis  wrote:
> > >> > Update: (Summarizes a couple of emails exchanged with Massimo)
> >
> > >> > Yesterday I found that my instance of web2py appeared be "leaking"
> > >> > processes at about 1 per hour.  Massimo suggested running with -N to
> > >> > disable cron.  I started two instances this way -- one under winpdb
> > >> > and another directly.  They've been running almost 24 hours now and
> > >> > neither has added any processes.  As noted earlier, my app has
> nothing
> > >> > in crontab. The admin and examples apps have the crontabs that are
> "as
> > >> > shipped" in v1.85.
> >
> > >> > Cheers,
> > >> > Mike
> >
> > >> > On Jul 27, 12:17 pm, Jonathan Lundell  wrote:
> >
> > >> > > On Jul 27, 2010, at 9:01 AM, Michael Ellis wrote:
> >
> > >> > > > Thanks for the info about sloth. I may try it later.  It was
> pretty
> > >> easy to get what I needed with ipython:
> >
> > >> > > > procs = !ps ax | grep "web2py"
> > >> > > > pids = [r[0] for r in procs.fields()[0:-2]]
> > >> > > > for p in pids:
> > >> > > >  print p
> > >> > > >  !lsof | grep $p | wc
> >
> > >> > > Also: sudo launchctl limit maxfiles
> >
> > >> > > I see 256 by default. The second limit is the "hard limit", beyond
> > >> which you can't configure (unlimited in this case).
> >
> > >> > >
> http://serverfault.com/questions/15564/where-are-the-default-ulimits-.
> > >> ..
>

[web2py] Re: rocket performance issues

[mdipierro]

Can you check if this issue is solved in trunk?

Massimo

On Jul 29, 7:55 am, Michael Ellis  wrote:
> Has anything changed at tip in the view parser? Looks like there may
> be a problem now with colons at the end of comment lines. I have a
> view with the following that was working fine before I updated this
> morning.
>
> {{
> ## On entry, vars returned by controller are:
> ## problem, c, s, sc, wsc, problemid, currenthash, nparticipants
> bulbicon = URL(r=request, c="static", f="lightbulb32.png")
> gaugeicon = URL(r=request, c='static', f="gauge.png")
>
> ...
>
> *After updating it started issuing tickets about "Missing pass in
> view" and the ticket listing showed an unexpected indentation. *
>
> *
> *
>
> ## On entry, vars returned by controller are:
>     ## problem, c, s, sc, wsc, problemid, currenthash, nparticipants
>     bulbicon = URL (r=request
> , c="static", f="lightbulb32.png")
>     gaugeicon = URL (r=request
> , c='static', f="gauge.png")
>
> ...
>
> Removing the colon at the end of the comment fixed it.
>
> *
> *
>
> On Thu, Jul 29, 2010 at 8:19 AM, Michael Ellis 
> wrote:
>
> > Testing under way...
>
> > (Iceberg, I'm running OS X 10.6.4)
>
> > Cheers,
> > Mike
>
> > On Thu, Jul 29, 2010 at 2:57 AM, mdipierro wrote:
>
> >> Mike,
>
> >> I modified the expire_sessions.py in trunk and added a
> >> try:...execept... If the problem was that a file was locked and the
> >> cron task got stuck, it may be solved now. Can you test it?
>
> >> Massimo
>
> >> On Jul 28, 4:10 pm, MikeEllis  wrote:
> >> > Update: (Summarizes a couple of emails exchanged with Massimo)
>
> >> > Yesterday I found that my instance of web2py appeared be "leaking"
> >> > processes at about 1 per hour.  Massimo suggested running with -N to
> >> > disable cron.  I started two instances this way -- one under winpdb
> >> > and another directly.  They've been running almost 24 hours now and
> >> > neither has added any processes.  As noted earlier, my app has nothing
> >> > in crontab. The admin and examples apps have the crontabs that are "as
> >> > shipped" in v1.85.
>
> >> > Cheers,
> >> > Mike
>
> >> > On Jul 27, 12:17 pm, Jonathan Lundell  wrote:
>
> >> > > On Jul 27, 2010, at 9:01 AM, Michael Ellis wrote:
>
> >> > > > Thanks for the info about sloth. I may try it later.  It was pretty
> >> easy to get what I needed with ipython:
>
> >> > > > procs = !ps ax | grep "web2py"
> >> > > > pids = [r[0] for r in procs.fields()[0:-2]]
> >> > > > for p in pids:
> >> > > >      print p
> >> > > >      !lsof | grep $p | wc
>
> >> > > Also: sudo launchctl limit maxfiles
>
> >> > > I see 256 by default. The second limit is the "hard limit", beyond
> >> which you can't configure (unlimited in this case).
>
> >> > >http://serverfault.com/questions/15564/where-are-the-default-ulimits-.
> >> ..

[web2py] Re: rocket performance issues

[mdipierro]

good point. This runs as a daemon therefore there is no stdout by
definition. I will change it into logging.

On Jul 29, 11:19 am, Jonathan Lundell  wrote:
> On Jul 28, 2010, at 11:57 PM, mdipierro wrote:
>
> > I modified the expire_sessions.py in trunk and added a
> > try:...execept... If the problem was that a file was locked and the
> > cron task got stuck, it may be solved now. Can you test it?
>
> In this patch:
>
> @@ -6,8 +6,14 @@
>  now=time.time()
>  for file in os.listdir(path):
>     filename=os.path.join(path,file)
> -   t=os.stat(filename)[stat.ST_MTIME]
> -   if os.path.isfile(filename) and now-t>EXPIRATION_MINUTES*60 \
> -          and file.startswith(('1','2','3','4','5','6','7','8','9')):
> -      os.unlink(filename)
> -
> +   try:
> +      t=os.stat(filename)[stat.ST_MTIME]
> +      if os.path.isfile(filename) and now-t>EXPIRATION_MINUTES*60 \
> +             and file.startswith(('1','2','3','4','5','6','7','8','9')):
> +         try:
> +            os.unlink(filename)
> +         except:
> +            print 'failure to unlink', filename
> +   except:
> +      print 'failure to stat', filename
> +        
>
> you might want to print "failed to unlink", and the errno or its string 
> expansion would be handy
>
> But a question: where does this print end up? What's stdout in this situation?

Re: [web2py] Re: rocket performance issues

[Jonathan Lundell]

On Jul 28, 2010, at 11:57 PM, mdipierro wrote:

> I modified the expire_sessions.py in trunk and added a
> try:...execept... If the problem was that a file was locked and the
> cron task got stuck, it may be solved now. Can you test it?

In this patch:

@@ -6,8 +6,14 @@
 now=time.time()
 for file in os.listdir(path):
filename=os.path.join(path,file)
-   t=os.stat(filename)[stat.ST_MTIME]
-   if os.path.isfile(filename) and now-t>EXPIRATION_MINUTES*60 \
-  and file.startswith(('1','2','3','4','5','6','7','8','9')):
-  os.unlink(filename)
-
+   try:
+  t=os.stat(filename)[stat.ST_MTIME]
+  if os.path.isfile(filename) and now-t>EXPIRATION_MINUTES*60 \
+ and file.startswith(('1','2','3','4','5','6','7','8','9')):
+ try:
+os.unlink(filename)
+ except:
+print 'failure to unlink', filename
+   except:
+  print 'failure to stat', filename
+ 

you might want to print "failed to unlink", and the errno or its string 
expansion would be handy

But a question: where does this print end up? What's stdout in this situation?

Re: [web2py] Re: rocket performance issues

[Michael Ellis]

without



On Thu, Jul 29, 2010 at 11:38 AM, mdipierro  wrote:

> with or without -N
>
> On Jul 29, 10:27 am, Michael Ellis  wrote:
> > No process leakage in instances I've been running since this morning.
>  Open
> > file count appears stable, too.
> > Cheers,
> > Mike
> >
> > On Thu, Jul 29, 2010 at 9:22 AM, mdipierro 
> wrote:
> > > we have a new template parser. My guess is that it is not ignoring
> > > the : at the end of {{# comments.
> >
> > > On Jul 29, 7:55 am, Michael Ellis  wrote:
> > > > Has anything changed at tip in the view parser? Looks like there may
> > > > be a problem now with colons at the end of comment lines. I have a
> > > > view with the following that was working fine before I updated this
> > > > morning.
> >
> > > > {{
> > > > ## On entry, vars returned by controller are:
> > > > ## problem, c, s, sc, wsc, problemid, currenthash, nparticipants
> > > > bulbicon = URL(r=request, c="static", f="lightbulb32.png")
> > > > gaugeicon = URL(r=request, c='static', f="gauge.png")
> >
> > > > ...
> >
> > > > *After updating it started issuing tickets about "Missing pass in
> > > > view" and the ticket listing showed an unexpected indentation. *
> >
> > > > *
> > > > *
> >
> > > > ## On entry, vars returned by controller are:
> > > > ## problem, c, s, sc, wsc, problemid, currenthash, nparticipants
> > > > bulbicon = URL (r=request
> > > > , c="static", f="lightbulb32.png")
> > > > gaugeicon = URL (r=request
> > > > , c='static', f="gauge.png")
> >
> > > > ...
> >
> > > > Removing the colon at the end of the comment fixed it.
> >
> > > > *
> > > > *
> >
> > > > On Thu, Jul 29, 2010 at 8:19 AM, Michael Ellis <
> > > michael.f.el...@gmail.com>wrote:
> >
> > > > > Testing under way...
> >
> > > > > (Iceberg, I'm running OS X 10.6.4)
> >
> > > > > Cheers,
> > > > > Mike
> >
> > > > > On Thu, Jul 29, 2010 at 2:57 AM, mdipierro <
> mdipie...@cs.depaul.edu
> > > >wrote:
> >
> > > > >> Mike,
> >
> > > > >> I modified the expire_sessions.py in trunk and added a
> > > > >> try:...execept... If the problem was that a file was locked and
> the
> > > > >> cron task got stuck, it may be solved now. Can you test it?
> >
> > > > >> Massimo
> >
> > > > >> On Jul 28, 4:10 pm, MikeEllis  wrote:
> > > > >> > Update: (Summarizes a couple of emails exchanged with Massimo)
> >
> > > > >> > Yesterday I found that my instance of web2py appeared be
> "leaking"
> > > > >> > processes at about 1 per hour.  Massimo suggested running with
> -N to
> > > > >> > disable cron.  I started two instances this way -- one under
> winpdb
> > > > >> > and another directly.  They've been running almost 24 hours now
> and
> > > > >> > neither has added any processes.  As noted earlier, my app has
> > > nothing
> > > > >> > in crontab. The admin and examples apps have the crontabs that
> are
> > > "as
> > > > >> > shipped" in v1.85.
> >
> > > > >> > Cheers,
> > > > >> > Mike
> >
> > > > >> > On Jul 27, 12:17 pm, Jonathan Lundell 
> wrote:
> >
> > > > >> > > On Jul 27, 2010, at 9:01 AM, Michael Ellis wrote:
> >
> > > > >> > > > Thanks for the info about sloth. I may try it later.  It was
> > > pretty
> > > > >> easy to get what I needed with ipython:
> >
> > > > >> > > > procs = !ps ax | grep "web2py"
> > > > >> > > > pids = [r[0] for r in procs.fields()[0:-2]]
> > > > >> > > > for p in pids:
> > > > >> > > >  print p
> > > > >> > > >  !lsof | grep $p | wc
> >
> > > > >> > > Also: sudo launchctl limit maxfiles
> >
> > > > >> > > I see 256 by default. The second limit is the "hard limit",
> beyond
> > > > >> which you can't configure (unlimited in this case).
> >
> > >http://serverfault.com/questions/15564/where-are-the-default-ulimits-.
> > > > >> ..
>

[web2py] Re: rocket performance issues

[mdipierro]

with or without -N

On Jul 29, 10:27 am, Michael Ellis  wrote:
> No process leakage in instances I've been running since this morning.  Open
> file count appears stable, too.
> Cheers,
> Mike
>
> On Thu, Jul 29, 2010 at 9:22 AM, mdipierro  wrote:
> > we have a new template parser. My guess is that it is not ignoring
> > the : at the end of {{# comments.
>
> > On Jul 29, 7:55 am, Michael Ellis  wrote:
> > > Has anything changed at tip in the view parser? Looks like there may
> > > be a problem now with colons at the end of comment lines. I have a
> > > view with the following that was working fine before I updated this
> > > morning.
>
> > > {{
> > > ## On entry, vars returned by controller are:
> > > ## problem, c, s, sc, wsc, problemid, currenthash, nparticipants
> > > bulbicon = URL(r=request, c="static", f="lightbulb32.png")
> > > gaugeicon = URL(r=request, c='static', f="gauge.png")
>
> > > ...
>
> > > *After updating it started issuing tickets about "Missing pass in
> > > view" and the ticket listing showed an unexpected indentation. *
>
> > > *
> > > *
>
> > > ## On entry, vars returned by controller are:
> > >     ## problem, c, s, sc, wsc, problemid, currenthash, nparticipants
> > >     bulbicon = URL (r=request
> > > , c="static", f="lightbulb32.png")
> > >     gaugeicon = URL (r=request
> > > , c='static', f="gauge.png")
>
> > > ...
>
> > > Removing the colon at the end of the comment fixed it.
>
> > > *
> > > *
>
> > > On Thu, Jul 29, 2010 at 8:19 AM, Michael Ellis <
> > michael.f.el...@gmail.com>wrote:
>
> > > > Testing under way...
>
> > > > (Iceberg, I'm running OS X 10.6.4)
>
> > > > Cheers,
> > > > Mike
>
> > > > On Thu, Jul 29, 2010 at 2:57 AM, mdipierro  > >wrote:
>
> > > >> Mike,
>
> > > >> I modified the expire_sessions.py in trunk and added a
> > > >> try:...execept... If the problem was that a file was locked and the
> > > >> cron task got stuck, it may be solved now. Can you test it?
>
> > > >> Massimo
>
> > > >> On Jul 28, 4:10 pm, MikeEllis  wrote:
> > > >> > Update: (Summarizes a couple of emails exchanged with Massimo)
>
> > > >> > Yesterday I found that my instance of web2py appeared be "leaking"
> > > >> > processes at about 1 per hour.  Massimo suggested running with -N to
> > > >> > disable cron.  I started two instances this way -- one under winpdb
> > > >> > and another directly.  They've been running almost 24 hours now and
> > > >> > neither has added any processes.  As noted earlier, my app has
> > nothing
> > > >> > in crontab. The admin and examples apps have the crontabs that are
> > "as
> > > >> > shipped" in v1.85.
>
> > > >> > Cheers,
> > > >> > Mike
>
> > > >> > On Jul 27, 12:17 pm, Jonathan Lundell  wrote:
>
> > > >> > > On Jul 27, 2010, at 9:01 AM, Michael Ellis wrote:
>
> > > >> > > > Thanks for the info about sloth. I may try it later.  It was
> > pretty
> > > >> easy to get what I needed with ipython:
>
> > > >> > > > procs = !ps ax | grep "web2py"
> > > >> > > > pids = [r[0] for r in procs.fields()[0:-2]]
> > > >> > > > for p in pids:
> > > >> > > >      print p
> > > >> > > >      !lsof | grep $p | wc
>
> > > >> > > Also: sudo launchctl limit maxfiles
>
> > > >> > > I see 256 by default. The second limit is the "hard limit", beyond
> > > >> which you can't configure (unlimited in this case).
>
> >http://serverfault.com/questions/15564/where-are-the-default-ulimits-.
> > > >> ..

Re: [web2py] Re: rocket performance issues

[Michael Ellis]

No process leakage in instances I've been running since this morning.  Open
file count appears stable, too.
Cheers,
Mike


On Thu, Jul 29, 2010 at 9:22 AM, mdipierro  wrote:

> we have a new template parser. My guess is that it is not ignoring
> the : at the end of {{# comments.
>
> On Jul 29, 7:55 am, Michael Ellis  wrote:
> > Has anything changed at tip in the view parser? Looks like there may
> > be a problem now with colons at the end of comment lines. I have a
> > view with the following that was working fine before I updated this
> > morning.
> >
> > {{
> > ## On entry, vars returned by controller are:
> > ## problem, c, s, sc, wsc, problemid, currenthash, nparticipants
> > bulbicon = URL(r=request, c="static", f="lightbulb32.png")
> > gaugeicon = URL(r=request, c='static', f="gauge.png")
> >
> > ...
> >
> > *After updating it started issuing tickets about "Missing pass in
> > view" and the ticket listing showed an unexpected indentation. *
> >
> > *
> > *
> >
> > ## On entry, vars returned by controller are:
> > ## problem, c, s, sc, wsc, problemid, currenthash, nparticipants
> > bulbicon = URL (r=request
> > , c="static", f="lightbulb32.png")
> > gaugeicon = URL (r=request
> > , c='static', f="gauge.png")
> >
> > ...
> >
> > Removing the colon at the end of the comment fixed it.
> >
> > *
> > *
> >
> > On Thu, Jul 29, 2010 at 8:19 AM, Michael Ellis <
> michael.f.el...@gmail.com>wrote:
> >
> > > Testing under way...
> >
> > > (Iceberg, I'm running OS X 10.6.4)
> >
> > > Cheers,
> > > Mike
> >
> > > On Thu, Jul 29, 2010 at 2:57 AM, mdipierro  >wrote:
> >
> > >> Mike,
> >
> > >> I modified the expire_sessions.py in trunk and added a
> > >> try:...execept... If the problem was that a file was locked and the
> > >> cron task got stuck, it may be solved now. Can you test it?
> >
> > >> Massimo
> >
> > >> On Jul 28, 4:10 pm, MikeEllis  wrote:
> > >> > Update: (Summarizes a couple of emails exchanged with Massimo)
> >
> > >> > Yesterday I found that my instance of web2py appeared be "leaking"
> > >> > processes at about 1 per hour.  Massimo suggested running with -N to
> > >> > disable cron.  I started two instances this way -- one under winpdb
> > >> > and another directly.  They've been running almost 24 hours now and
> > >> > neither has added any processes.  As noted earlier, my app has
> nothing
> > >> > in crontab. The admin and examples apps have the crontabs that are
> "as
> > >> > shipped" in v1.85.
> >
> > >> > Cheers,
> > >> > Mike
> >
> > >> > On Jul 27, 12:17 pm, Jonathan Lundell  wrote:
> >
> > >> > > On Jul 27, 2010, at 9:01 AM, Michael Ellis wrote:
> >
> > >> > > > Thanks for the info about sloth. I may try it later.  It was
> pretty
> > >> easy to get what I needed with ipython:
> >
> > >> > > > procs = !ps ax | grep "web2py"
> > >> > > > pids = [r[0] for r in procs.fields()[0:-2]]
> > >> > > > for p in pids:
> > >> > > >  print p
> > >> > > >  !lsof | grep $p | wc
> >
> > >> > > Also: sudo launchctl limit maxfiles
> >
> > >> > > I see 256 by default. The second limit is the "hard limit", beyond
> > >> which you can't configure (unlimited in this case).
> >
> > >> > >
> http://serverfault.com/questions/15564/where-are-the-default-ulimits-.
> > >> ..
>

[web2py] Re: rocket performance issues

[mdipierro]

we have a new template parser. My guess is that it is not ignoring
the : at the end of {{# comments.

On Jul 29, 7:55 am, Michael Ellis  wrote:
> Has anything changed at tip in the view parser? Looks like there may
> be a problem now with colons at the end of comment lines. I have a
> view with the following that was working fine before I updated this
> morning.
>
> {{
> ## On entry, vars returned by controller are:
> ## problem, c, s, sc, wsc, problemid, currenthash, nparticipants
> bulbicon = URL(r=request, c="static", f="lightbulb32.png")
> gaugeicon = URL(r=request, c='static', f="gauge.png")
>
> ...
>
> *After updating it started issuing tickets about "Missing pass in
> view" and the ticket listing showed an unexpected indentation. *
>
> *
> *
>
> ## On entry, vars returned by controller are:
>     ## problem, c, s, sc, wsc, problemid, currenthash, nparticipants
>     bulbicon = URL (r=request
> , c="static", f="lightbulb32.png")
>     gaugeicon = URL (r=request
> , c='static', f="gauge.png")
>
> ...
>
> Removing the colon at the end of the comment fixed it.
>
> *
> *
>
> On Thu, Jul 29, 2010 at 8:19 AM, Michael Ellis 
> wrote:
>
> > Testing under way...
>
> > (Iceberg, I'm running OS X 10.6.4)
>
> > Cheers,
> > Mike
>
> > On Thu, Jul 29, 2010 at 2:57 AM, mdipierro wrote:
>
> >> Mike,
>
> >> I modified the expire_sessions.py in trunk and added a
> >> try:...execept... If the problem was that a file was locked and the
> >> cron task got stuck, it may be solved now. Can you test it?
>
> >> Massimo
>
> >> On Jul 28, 4:10 pm, MikeEllis  wrote:
> >> > Update: (Summarizes a couple of emails exchanged with Massimo)
>
> >> > Yesterday I found that my instance of web2py appeared be "leaking"
> >> > processes at about 1 per hour.  Massimo suggested running with -N to
> >> > disable cron.  I started two instances this way -- one under winpdb
> >> > and another directly.  They've been running almost 24 hours now and
> >> > neither has added any processes.  As noted earlier, my app has nothing
> >> > in crontab. The admin and examples apps have the crontabs that are "as
> >> > shipped" in v1.85.
>
> >> > Cheers,
> >> > Mike
>
> >> > On Jul 27, 12:17 pm, Jonathan Lundell  wrote:
>
> >> > > On Jul 27, 2010, at 9:01 AM, Michael Ellis wrote:
>
> >> > > > Thanks for the info about sloth. I may try it later.  It was pretty
> >> easy to get what I needed with ipython:
>
> >> > > > procs = !ps ax | grep "web2py"
> >> > > > pids = [r[0] for r in procs.fields()[0:-2]]
> >> > > > for p in pids:
> >> > > >      print p
> >> > > >      !lsof | grep $p | wc
>
> >> > > Also: sudo launchctl limit maxfiles
>
> >> > > I see 256 by default. The second limit is the "hard limit", beyond
> >> which you can't configure (unlimited in this case).
>
> >> > >http://serverfault.com/questions/15564/where-are-the-default-ulimits-.
> >> ..

Re: [web2py] Re: rocket performance issues

[Michael Ellis]

Has anything changed at tip in the view parser? Looks like there may
be a problem now with colons at the end of comment lines. I have a
view with the following that was working fine before I updated this
morning.

{{
## On entry, vars returned by controller are:
## problem, c, s, sc, wsc, problemid, currenthash, nparticipants
bulbicon = URL(r=request, c="static", f="lightbulb32.png")
gaugeicon = URL(r=request, c='static', f="gauge.png")

...

*After updating it started issuing tickets about "Missing pass in
view" and the ticket listing showed an unexpected indentation. *

*
*

## On entry, vars returned by controller are:
## problem, c, s, sc, wsc, problemid, currenthash, nparticipants
bulbicon = URL (r=request
, c="static", f="lightbulb32.png")
gaugeicon = URL (r=request
, c='static', f="gauge.png")

...

Removing the colon at the end of the comment fixed it.

*
*



On Thu, Jul 29, 2010 at 8:19 AM, Michael Ellis wrote:

> Testing under way...
>
> (Iceberg, I'm running OS X 10.6.4)
>
> Cheers,
> Mike
>
>
>
> On Thu, Jul 29, 2010 at 2:57 AM, mdipierro wrote:
>
>> Mike,
>>
>> I modified the expire_sessions.py in trunk and added a
>> try:...execept... If the problem was that a file was locked and the
>> cron task got stuck, it may be solved now. Can you test it?
>>
>> Massimo
>>
>> On Jul 28, 4:10 pm, MikeEllis  wrote:
>> > Update: (Summarizes a couple of emails exchanged with Massimo)
>> >
>> > Yesterday I found that my instance of web2py appeared be "leaking"
>> > processes at about 1 per hour.  Massimo suggested running with -N to
>> > disable cron.  I started two instances this way -- one under winpdb
>> > and another directly.  They've been running almost 24 hours now and
>> > neither has added any processes.  As noted earlier, my app has nothing
>> > in crontab. The admin and examples apps have the crontabs that are "as
>> > shipped" in v1.85.
>> >
>> > Cheers,
>> > Mike
>> >
>> > On Jul 27, 12:17 pm, Jonathan Lundell  wrote:
>> >
>> > > On Jul 27, 2010, at 9:01 AM, Michael Ellis wrote:
>> >
>> > > > Thanks for the info about sloth. I may try it later.  It was pretty
>> easy to get what I needed with ipython:
>> >
>> > > > procs = !ps ax | grep "web2py"
>> > > > pids = [r[0] for r in procs.fields()[0:-2]]
>> > > > for p in pids:
>> > > >  print p
>> > > >  !lsof | grep $p | wc
>> >
>> > > Also: sudo launchctl limit maxfiles
>> >
>> > > I see 256 by default. The second limit is the "hard limit", beyond
>> which you can't configure (unlimited in this case).
>> >
>> > >http://serverfault.com/questions/15564/where-are-the-default-ulimits-.
>> ..
>>
>
>

Re: [web2py] Re: rocket performance issues

[Michael Ellis]

Testing under way...

(Iceberg, I'm running OS X 10.6.4)

Cheers,
Mike


On Thu, Jul 29, 2010 at 2:57 AM, mdipierro  wrote:

> Mike,
>
> I modified the expire_sessions.py in trunk and added a
> try:...execept... If the problem was that a file was locked and the
> cron task got stuck, it may be solved now. Can you test it?
>
> Massimo
>
> On Jul 28, 4:10 pm, MikeEllis  wrote:
> > Update: (Summarizes a couple of emails exchanged with Massimo)
> >
> > Yesterday I found that my instance of web2py appeared be "leaking"
> > processes at about 1 per hour.  Massimo suggested running with -N to
> > disable cron.  I started two instances this way -- one under winpdb
> > and another directly.  They've been running almost 24 hours now and
> > neither has added any processes.  As noted earlier, my app has nothing
> > in crontab. The admin and examples apps have the crontabs that are "as
> > shipped" in v1.85.
> >
> > Cheers,
> > Mike
> >
> > On Jul 27, 12:17 pm, Jonathan Lundell  wrote:
> >
> > > On Jul 27, 2010, at 9:01 AM, Michael Ellis wrote:
> >
> > > > Thanks for the info about sloth. I may try it later.  It was pretty
> easy to get what I needed with ipython:
> >
> > > > procs = !ps ax | grep "web2py"
> > > > pids = [r[0] for r in procs.fields()[0:-2]]
> > > > for p in pids:
> > > >  print p
> > > >  !lsof | grep $p | wc
> >
> > > Also: sudo launchctl limit maxfiles
> >
> > > I see 256 by default. The second limit is the "hard limit", beyond
> which you can't configure (unlimited in this case).
> >
> > >http://serverfault.com/questions/15564/where-are-the-default-ulimits-.
> ..
>

[web2py] Re: rocket performance issues

[mdipierro]

Mike,

I modified the expire_sessions.py in trunk and added a
try:...execept... If the problem was that a file was locked and the
cron task got stuck, it may be solved now. Can you test it?

Massimo

On Jul 28, 4:10 pm, MikeEllis  wrote:
> Update: (Summarizes a couple of emails exchanged with Massimo)
>
> Yesterday I found that my instance of web2py appeared be "leaking"
> processes at about 1 per hour.  Massimo suggested running with -N to
> disable cron.  I started two instances this way -- one under winpdb
> and another directly.  They've been running almost 24 hours now and
> neither has added any processes.  As noted earlier, my app has nothing
> in crontab. The admin and examples apps have the crontabs that are "as
> shipped" in v1.85.
>
> Cheers,
> Mike
>
> On Jul 27, 12:17 pm, Jonathan Lundell  wrote:
>
> > On Jul 27, 2010, at 9:01 AM, Michael Ellis wrote:
>
> > > Thanks for the info about sloth. I may try it later.  It was pretty easy 
> > > to get what I needed with ipython:
>
> > > procs = !ps ax | grep "web2py"
> > > pids = [r[0] for r in procs.fields()[0:-2]]
> > > for p in pids:
> > >      print p
> > >      !lsof | grep $p | wc
>
> > Also: sudo launchctl limit maxfiles
>
> > I see 256 by default. The second limit is the "hard limit", beyond which 
> > you can't configure (unlimited in this case).
>
> >http://serverfault.com/questions/15564/where-are-the-default-ulimits-...

[web2py] Re: rocket performance issues

[Iceberg]

The one "leak" process per hour coincident is "coincidently" matching
the one run per hour of admin/cron/expire_sessions.py

That really raises my concern, cause I just gonna use cron as a trick
to touch my app's cache frequently. :-/

By the way, which os is Mike Ellis using? I did not observe same
problem on web2py 1.81.5, binary package, on Windows XP. No increasing
process, just the file handlers may slightly increased by one or two
per minute. But, file handlers will become stable after a total 20-or-
so increase, within half an hour. So perhaps situation is not that
bad. (Tips: to shorten the test time, I change the crontab time gap to
minimal one minute.)

Regards,
Iceberg

On Jul 29, 6:15am, mdipierro  wrote:
> There clearly is a problem with the cron processes.
>
> Massimo
>
> On Jul 28, 4:10 pm, MikeEllis  wrote:
>
>
>
> > Update: (Summarizes a couple of emails exchanged with Massimo)
>
> > Yesterday I found that my instance of web2py appeared be "leaking"
> > processes at about 1 per hour.  Massimo suggested running with -N to
> > disable cron.  I started two instances this way -- one under winpdb
> > and another directly.  They've been running almost 24 hours now and
> > neither has added any processes.  As noted earlier, my app has nothing
> > in crontab. The admin and examples apps have the crontabs that are "as
> > shipped" in v1.85.
>
> > Cheers,
> > Mike
>
> > On Jul 27, 12:17 pm, Jonathan Lundell  wrote:
>
> > > On Jul 27, 2010, at 9:01 AM, Michael Ellis wrote:
>
> > > > Thanks for the info about sloth. I may try it later.  It was pretty 
> > > > easy to get what I needed with ipython:
>
> > > > procs = !ps ax | grep "web2py"
> > > > pids = [r[0] for r in procs.fields()[0:-2]]
> > > > for p in pids:
> > > >      print p
> > > >      !lsof | grep $p | wc
>
> > > Also: sudo launchctl limit maxfiles
>
> > > I see 256 by default. The second limit is the "hard limit", beyond which 
> > > you can't configure (unlimited in this case).

[web2py] Re: rocket performance issues

[mdipierro]

There clearly is a problem with the cron processes.

Massimo

On Jul 28, 4:10 pm, MikeEllis  wrote:
> Update: (Summarizes a couple of emails exchanged with Massimo)
>
> Yesterday I found that my instance of web2py appeared be "leaking"
> processes at about 1 per hour.  Massimo suggested running with -N to
> disable cron.  I started two instances this way -- one under winpdb
> and another directly.  They've been running almost 24 hours now and
> neither has added any processes.  As noted earlier, my app has nothing
> in crontab. The admin and examples apps have the crontabs that are "as
> shipped" in v1.85.
>
> Cheers,
> Mike
>
> On Jul 27, 12:17 pm, Jonathan Lundell  wrote:
>
> > On Jul 27, 2010, at 9:01 AM, Michael Ellis wrote:
>
> > > Thanks for the info about sloth. I may try it later.  It was pretty easy 
> > > to get what I needed with ipython:
>
> > > procs = !ps ax | grep "web2py"
> > > pids = [r[0] for r in procs.fields()[0:-2]]
> > > for p in pids:
> > >      print p
> > >      !lsof | grep $p | wc
>
> > Also: sudo launchctl limit maxfiles
>
> > I see 256 by default. The second limit is the "hard limit", beyond which 
> > you can't configure (unlimited in this case).
>
> >http://serverfault.com/questions/15564/where-are-the-default-ulimits-...

[web2py] Re: rocket performance issues

[MikeEllis]

Update: (Summarizes a couple of emails exchanged with Massimo)

Yesterday I found that my instance of web2py appeared be "leaking"
processes at about 1 per hour.  Massimo suggested running with -N to
disable cron.  I started two instances this way -- one under winpdb
and another directly.  They've been running almost 24 hours now and
neither has added any processes.  As noted earlier, my app has nothing
in crontab. The admin and examples apps have the crontabs that are "as
shipped" in v1.85.

Cheers,
Mike


On Jul 27, 12:17 pm, Jonathan Lundell  wrote:
> On Jul 27, 2010, at 9:01 AM, Michael Ellis wrote:
>
> > Thanks for the info about sloth. I may try it later.  It was pretty easy to 
> > get what I needed with ipython:
>
> > procs = !ps ax | grep "web2py"
> > pids = [r[0] for r in procs.fields()[0:-2]]
> > for p in pids:
> >      print p
> >      !lsof | grep $p | wc
>
> Also: sudo launchctl limit maxfiles
>
> I see 256 by default. The second limit is the "hard limit", beyond which you 
> can't configure (unlimited in this case).
>
> http://serverfault.com/questions/15564/where-are-the-default-ulimits-...

Re: [web2py] Re: rocket performance issues

[Jonathan Lundell]

On Jul 27, 2010, at 9:01 AM, Michael Ellis wrote:

> Thanks for the info about sloth. I may try it later.  It was pretty easy to 
> get what I needed with ipython:
> 
> procs = !ps ax | grep "web2py"
> pids = [r[0] for r in procs.fields()[0:-2]]
> for p in pids:
>  print p
>  !lsof | grep $p | wc
> 

Also: sudo launchctl limit maxfiles

I see 256 by default. The second limit is the "hard limit", beyond which you 
can't configure (unlimited in this case).

http://serverfault.com/questions/15564/where-are-the-default-ulimits-specified-on-os-x-10-5

Re: [web2py] Re: rocket performance issues

[Michael Ellis]

Sorry, what I sent was very large (~750K each). Didn't realize GoogleGroups
would reduce it  to a thumbnail.  I'll try to forward the original message
directly to you ...



On Tue, Jul 27, 2010 at 12:07 PM, mdipierro  wrote:

> The picture is too small. Anything interesting in there?
>
> On Jul 27, 11:01 am, Michael Ellis  wrote:
> > Thanks for the info about sloth. I may try it later.  It was pretty easy
> to
> > get what I needed with ipython:
> >
> > procs = !ps ax | grep "web2py"
> > pids = [r[0] for r in procs.fields()[0:-2]]
> > for p in pids:
> >  print p
> >  !lsof | grep $p | wc
> >
> > On Tue, Jul 27, 2010 at 11:50 AM, Jonathan Lundell  >wrote:
> >
> > > On Jul 27, 2010, at 8:23 AM, Jonathan Lundell wrote:
> >
> > > On Jul 27, 2010, at 5:33 AM, Michael Ellis wrote:
> >
> > > This morning I found several OSError reports about "Too many open
> files" in
> > > a web2py development server that's been running locally for several
> days.
> > >  The app code isn't doing any explicit file i/o so I don't know what's
> going
> > > on,  but here are the tracebacks in case someone else is seeing
> anything
> > > similar.
> >
> > > The first thing to do is to use lsof (try: man lsof) to see what's
> being
> > > opened.
> >
> > > I just noticed a graphical interface to lsof that looks pretty nice:
> >
> > > Sloth 1.5 - Displays all open files in use by an app. (Free)<
> http://feedproxy.google.com/~r/macupdate/~3/q-i76jy2bk8/sloth>
> > > 7/27/10 5:09 AM
> > > MacUpdate 
> > > Sloth 1.5
> > > 
> > > *Sloth* is a program that displays a list of all open files in use by
> all
> > > the applications your user account is running on the system—this list
> > > includes IP and Unix sockets, character devices and directories, and is
> > > generated by the 'lsof' command. This list is presented along with the
> names
> > > of applications using the file and their process IDs. This makes it
> easy to
> > > monitor what applications are 'locking' which files. Sloth is a
> graphical
> > > interface front-end to 'lsof', a powerful unix tool.
> >
> > > WHAT'S NEW
> > > Version 1.4:
> >
> > >- Column sorting, column rearrangement
> > >- Copy/Drag and drop
> > >- Several bug fixes
> > >- Various interface refinements
> >
> > > REQUIREMENTS
> > > Mac OS X 10.4 or later.
> >
> > > PRICE
> > > Free
> >
> > > DEVELOPER
> > > Sveinbjorn Thordarson 
> >
> > > DOWNLOADS
> > > 6080
> >
> > > DOWNLOAD NOW (110 K)<
> http://www.macupdate.com/download.php/14230%20sloth.zip>
> >
> > > More information 
> >
> >
> >
> >  14230_sm.png
> > 68KViewDownload

[web2py] Re: rocket performance issues

[mdipierro]

The picture is too small. Anything interesting in there?

On Jul 27, 11:01 am, Michael Ellis  wrote:
> Thanks for the info about sloth. I may try it later.  It was pretty easy to
> get what I needed with ipython:
>
> procs = !ps ax | grep "web2py"
> pids = [r[0] for r in procs.fields()[0:-2]]
> for p in pids:
>      print p
>      !lsof | grep $p | wc
>
> On Tue, Jul 27, 2010 at 11:50 AM, Jonathan Lundell wrote:
>
> > On Jul 27, 2010, at 8:23 AM, Jonathan Lundell wrote:
>
> > On Jul 27, 2010, at 5:33 AM, Michael Ellis wrote:
>
> > This morning I found several OSError reports about "Too many open files" in
> > a web2py development server that's been running locally for several days.
> >  The app code isn't doing any explicit file i/o so I don't know what's going
> > on,  but here are the tracebacks in case someone else is seeing anything
> > similar.
>
> > The first thing to do is to use lsof (try: man lsof) to see what's being
> > opened.
>
> > I just noticed a graphical interface to lsof that looks pretty nice:
>
> > Sloth 1.5 - Displays all open files in use by an app. 
> > (Free)
> > 7/27/10 5:09 AM
> > MacUpdate 
> > Sloth 1.5
> > 
> > *Sloth* is a program that displays a list of all open files in use by all
> > the applications your user account is running on the system—this list
> > includes IP and Unix sockets, character devices and directories, and is
> > generated by the 'lsof' command. This list is presented along with the names
> > of applications using the file and their process IDs. This makes it easy to
> > monitor what applications are 'locking' which files. Sloth is a graphical
> > interface front-end to 'lsof', a powerful unix tool.
>
> > WHAT'S NEW
> > Version 1.4:
>
> >    - Column sorting, column rearrangement
> >    - Copy/Drag and drop
> >    - Several bug fixes
> >    - Various interface refinements
>
> > REQUIREMENTS
> > Mac OS X 10.4 or later.
>
> > PRICE
> > Free
>
> > DEVELOPER
> > Sveinbjorn Thordarson 
>
> > DOWNLOADS
> > 6080
>
> > DOWNLOAD NOW (110 
> > K)
>
> > More information 
>
>
>
>  14230_sm.png
> 68KViewDownload

Re: [web2py] Re: rocket performance issues

[Michael Ellis]

Thanks, Jonathan. That's a good suggestion.

I last restarted the app a couple of hours ago.  ps reports 4 processes that
match 'web2py'.  Grepping for each pid in lsof shows nothing extraordinary.
 Each process has 80-90 files open. The vast majority are libraries (.so).
 There are a few chr files and about 10 pipes associated with each pid.

I'll check again in a few hours and see if the numbers are increasing.

Cheers,
Mike


On Tue, Jul 27, 2010 at 11:23 AM, Jonathan Lundell wrote:

> On Jul 27, 2010, at 5:33 AM, Michael Ellis wrote:
>
> > This morning I found several OSError reports about "Too many open files"
> in a web2py development server that's been running locally for several days.
>  The app code isn't doing any explicit file i/o so I don't know what's going
> on,  but here are the tracebacks in case someone else is seeing anything
> similar.
>
> The first thing to do is to use lsof (try: man lsof) to see what's being
> opened.

Re: [web2py] Re: rocket performance issues

[Jonathan Lundell]

On Jul 27, 2010, at 5:33 AM, Michael Ellis wrote:

> This morning I found several OSError reports about "Too many open files" in a 
> web2py development server that's been running locally for several days.  The 
> app code isn't doing any explicit file i/o so I don't know what's going on,  
> but here are the tracebacks in case someone else is seeing anything similar.

The first thing to do is to use lsof (try: man lsof) to see what's being opened.

Re: [web2py] Re: rocket performance issues

[Michael Ellis]

ok, I'll give that  a shot in a few minutes if I don't get a fork break by
then.  BTW, I verified that cron/crontab is empty in my app.  However it's
not empty in admin and examples:

$ cat admin/cron/crontab
10 * * * * root **applications/admin/cron/expire_sessions.py

$ cat examples/cron/crontab
#0-59/1 * * * * root python /path/to/python/script
#*/1 * * * * root *applications/admin/cron/expire_sessions.py
#*/1*   *   *   *   root *applications/admin/cron/my magic script.py
#*/1*   *   *   *   root **applications/admin/cron/nodb.py
#*/1*   *   *   *   root /tmp/my\ script.py 1 2 3
#*/1*   *   *   *   root "/tmp/my script.py" 456
#...@reboot root *default/preload

Cheers,
Mike


On Tue, Jul 27, 2010 at 8:57 AM, mdipierro  wrote:

> Yet is being called. It is on by default. Try set cron=False at the
> bottom of web2py.py
>
> On Jul 27, 7:49 am, Michael Ellis  wrote:
> > Not using cron.
> >
> > On Tue, Jul 27, 2010 at 8:43 AM, mdipierro 
> wrote:
> > > Ignore my previous email I see you are using 2.6 and the problem
> > > is with cron. I think the problem is a cron process that does not end
> > > and keeps restarting. Are you using cron?
> >
> > > On Jul 27, 7:33 am, Michael Ellis  wrote:
> > > > Not sure if this is related to Rocket or whether a new topic is
> needed.
> >
> > > > This morning I found several OSError reports about "Too many open
> files"
> > > in
> > > > a web2py development server that's been running locally for several
> days.
> > > >  The app code isn't doing any explicit file i/o so I don't know
> what's
> > > going
> > > > on,  but here are the tracebacks in case someone else is seeing
> anything
> > > > similar.
> >
> > > > Mike
> >
> > > > Exception in thread Thread-89:
> > > > Traceback (most recent call last):
> > > >   File
> >
> > >
> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/threading.py",
> > > > line 522, in __bootstrap_inner
> > > > self.run()
> > > >   File "/Users/mellis/w2ptip/gluon/newcron.py", line 206, in run
> > > > shell=self.shell)
> > > >   File
> >
> > >
> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py",
> > > > line 595, in __init__
> > > > errread, errwrite)
> > > >   File
> >
> > >
> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py",
> > > > line 1018, in _execute_child
> > > > errpipe_read, errpipe_write = os.pipe()
> > > > OSError: [Errno 24] Too many open files
> >
> > > > ERROR:Rocket.Errors.ThreadPool:Traceback (most recent call last):
> > > >   File "/Users/mellis/w2ptip/gluon/rocket.py", line 302, in start
> > > > sock = l.accept()
> > > >   File
> >
> > >
> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/socket.py",
> > > > line 195, in accept
> > > > sock, addr = self._sock.accept()
> > > > error: [Errno 24] Too many open files
> >
> > > > ERROR:root:corrupted file:
> > > > /Users/mellis/w2ptip/applications/init/cache/cache.shelve
> > > > ERROR:root:Traceback (most recent call last):
> > > >   File "/Users/mellis/w2ptip/gluon/restricted.py", line 178, in
> > > restricted
> > > > exec ccode in environment
> > > >   File "/Users/mellis/w2ptip/applications/init/models/db.py", line
> 35, in
> > > > 
> > > > auth.define_tables() # creates all needed
> > > tables
> > > >   File "/Users/mellis/w2ptip/gluon/tools.py", line 1106, in
> define_tables
> > > > format='%(first_name)s %(last_name)s (%(id)s)')
> > > >   File "/Users/mellis/w2ptip/gluon/sql.py", line 1309, in
> define_table
> > > >   File "/Users/mellis/w2ptip/gluon/sql.py", line 1715, in _create
> > > > IOError: [Errno 24] Too many open files:
> > > > '/Users/mellis/w2ptip/applications/init/databases/sql.log'
> >
> > > > On Tue, Jul 27, 2010 at 2:54 AM, Rahul 
> wrote:
> > > > > Thanks everyone, for getting this issue resolved...
> > > > > Web2py rocks!
> > > > > Cheers, Rahul
> >
> > > > > On Jul 24, 6:25 pm, Phyo Arkar  wrote:
> > > > > > Yes , Since from start of web2py , Massimo and Us never recommend
> to
> > > use
> > > > > > built-in web2py server at production. They are mainly for
> development
> > > > > > purpose.
> > > > > > Since Very start of the project , since Cherrypy , Massimo always
> > > Suggest
> > > > > to
> > > > > > use Apache/Cherokee/LighHTTPD over fcgi/wsgi/uwsgi or mod_python
> in
> > > > > Serious
> > > > > > Production Server.
> >
> > > > > > Rocket tho tries quite hard to achieve production level
> Performance
> > > with
> > > > > all
> > > > > > Cool Comet/Threading Stuff . It still Quite Young .Lets give it a
> > > chance.
> >
> > > > > > On Sat, Jul 24, 2010 at 7:39 PM, Scott 
> > > wrote:
> > > > > > > Please allow me to preface my comments: I have nothing against
> > > Rocket;
> > > > > > > my opinions come from years of experience with Java EE
> deployments.
> >
> > > > > > > I think raising the max_threads to 1024 is a good idea.
>  However,
> > > my
> > > > > 

[web2py] Re: rocket performance issues

[mdipierro]

Yet is being called. It is on by default. Try set cron=False at the
bottom of web2py.py

On Jul 27, 7:49 am, Michael Ellis  wrote:
> Not using cron.
>
> On Tue, Jul 27, 2010 at 8:43 AM, mdipierro  wrote:
> > Ignore my previous email I see you are using 2.6 and the problem
> > is with cron. I think the problem is a cron process that does not end
> > and keeps restarting. Are you using cron?
>
> > On Jul 27, 7:33 am, Michael Ellis  wrote:
> > > Not sure if this is related to Rocket or whether a new topic is needed.
>
> > > This morning I found several OSError reports about "Too many open files"
> > in
> > > a web2py development server that's been running locally for several days.
> > >  The app code isn't doing any explicit file i/o so I don't know what's
> > going
> > > on,  but here are the tracebacks in case someone else is seeing anything
> > > similar.
>
> > > Mike
>
> > > Exception in thread Thread-89:
> > > Traceback (most recent call last):
> > >   File
>
> > "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/threading.py",
> > > line 522, in __bootstrap_inner
> > >     self.run()
> > >   File "/Users/mellis/w2ptip/gluon/newcron.py", line 206, in run
> > >     shell=self.shell)
> > >   File
>
> > "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py",
> > > line 595, in __init__
> > >     errread, errwrite)
> > >   File
>
> > "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py",
> > > line 1018, in _execute_child
> > >     errpipe_read, errpipe_write = os.pipe()
> > > OSError: [Errno 24] Too many open files
>
> > > ERROR:Rocket.Errors.ThreadPool:Traceback (most recent call last):
> > >   File "/Users/mellis/w2ptip/gluon/rocket.py", line 302, in start
> > >     sock = l.accept()
> > >   File
>
> > "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/socket.py",
> > > line 195, in accept
> > >     sock, addr = self._sock.accept()
> > > error: [Errno 24] Too many open files
>
> > > ERROR:root:corrupted file:
> > > /Users/mellis/w2ptip/applications/init/cache/cache.shelve
> > > ERROR:root:Traceback (most recent call last):
> > >   File "/Users/mellis/w2ptip/gluon/restricted.py", line 178, in
> > restricted
> > >     exec ccode in environment
> > >   File "/Users/mellis/w2ptip/applications/init/models/db.py", line 35, in
> > > 
> > >     auth.define_tables()                         # creates all needed
> > tables
> > >   File "/Users/mellis/w2ptip/gluon/tools.py", line 1106, in define_tables
> > >     format='%(first_name)s %(last_name)s (%(id)s)')
> > >   File "/Users/mellis/w2ptip/gluon/sql.py", line 1309, in define_table
> > >   File "/Users/mellis/w2ptip/gluon/sql.py", line 1715, in _create
> > > IOError: [Errno 24] Too many open files:
> > > '/Users/mellis/w2ptip/applications/init/databases/sql.log'
>
> > > On Tue, Jul 27, 2010 at 2:54 AM, Rahul  wrote:
> > > > Thanks everyone, for getting this issue resolved...
> > > > Web2py rocks!
> > > > Cheers, Rahul
>
> > > > On Jul 24, 6:25 pm, Phyo Arkar  wrote:
> > > > > Yes , Since from start of web2py , Massimo and Us never recommend to
> > use
> > > > > built-in web2py server at production. They are mainly for development
> > > > > purpose.
> > > > > Since Very start of the project , since Cherrypy , Massimo always
> > Suggest
> > > > to
> > > > > use Apache/Cherokee/LighHTTPD over fcgi/wsgi/uwsgi or mod_python in
> > > > Serious
> > > > > Production Server.
>
> > > > > Rocket tho tries quite hard to achieve production level Performance
> > with
> > > > all
> > > > > Cool Comet/Threading Stuff . It still Quite Young .Lets give it a
> > chance.
>
> > > > > On Sat, Jul 24, 2010 at 7:39 PM, Scott 
> > wrote:
> > > > > > Please allow me to preface my comments: I have nothing against
> > Rocket;
> > > > > > my opinions come from years of experience with Java EE deployments.
>
> > > > > > I think raising the max_threads to 1024 is a good idea.  However,
> > my
> > > > > > opinion is that Rocket alone should not be used for a production
> > > > > > deployment; much as I would not use the built-in Web server in
> > JBoss,
> > > > > > WebLogic, Geronimo, etc. as the front door.  My suggestion for
> > > > > > production would be to use an Apache front-end into Rocket.  Apache
> > is
> > > > > > more battle-hardened in this area, and it's a lot easier to handle
> > DoS
> > > > > > attacks through modules such as mod_evasive.  There are numerous
> > other
> > > > > > benefits too, such as easily enabling gzip compression and allowing
> > > > > > you a better security model through Defense in Depth... but I
> > digress.
>
> > > > > > On Jul 23, 5:41 pm, mdipierro  wrote:
> > > > > > > On a second thought this open the door to more sever denial of
> > > > service
> > > > > > > attacks than caused by the original problem. How about, until
> > there
> > > > is
> > > > > > > a better under understanding and solution, we just increase
> > > > > > > max_threads from

Re: [web2py] Re: rocket performance issues

[Michael Ellis]

I think I can test the cron hypothesis.  I'm running web2py under winpdb.
Normally, I set the fork mode to "parent, auto" so that it doesn't break
when a child process starts. If I change it to manual, it should break next
time a child process is spawned.  What should I look for when that happens?



On Tue, Jul 27, 2010 at 8:49 AM, Michael Ellis wrote:

> Not using cron.
>
>
>
>
> On Tue, Jul 27, 2010 at 8:43 AM, mdipierro wrote:
>
>> Ignore my previous email I see you are using 2.6 and the problem
>> is with cron. I think the problem is a cron process that does not end
>> and keeps restarting. Are you using cron?
>>
>> On Jul 27, 7:33 am, Michael Ellis  wrote:
>> > Not sure if this is related to Rocket or whether a new topic is needed.
>> >
>> > This morning I found several OSError reports about "Too many open files"
>> in
>> > a web2py development server that's been running locally for several
>> days.
>> >  The app code isn't doing any explicit file i/o so I don't know what's
>> going
>> > on,  but here are the tracebacks in case someone else is seeing anything
>> > similar.
>> >
>> > Mike
>> >
>> > Exception in thread Thread-89:
>> > Traceback (most recent call last):
>> >   File
>> >
>> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/threading.py",
>> > line 522, in __bootstrap_inner
>> > self.run()
>> >   File "/Users/mellis/w2ptip/gluon/newcron.py", line 206, in run
>> > shell=self.shell)
>> >   File
>> >
>> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py",
>> > line 595, in __init__
>> > errread, errwrite)
>> >   File
>> >
>> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py",
>> > line 1018, in _execute_child
>> > errpipe_read, errpipe_write = os.pipe()
>> > OSError: [Errno 24] Too many open files
>> >
>> > ERROR:Rocket.Errors.ThreadPool:Traceback (most recent call last):
>> >   File "/Users/mellis/w2ptip/gluon/rocket.py", line 302, in start
>> > sock = l.accept()
>> >   File
>> >
>> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/socket.py",
>> > line 195, in accept
>> > sock, addr = self._sock.accept()
>> > error: [Errno 24] Too many open files
>> >
>> > ERROR:root:corrupted file:
>> > /Users/mellis/w2ptip/applications/init/cache/cache.shelve
>> > ERROR:root:Traceback (most recent call last):
>> >   File "/Users/mellis/w2ptip/gluon/restricted.py", line 178, in
>> restricted
>> > exec ccode in environment
>> >   File "/Users/mellis/w2ptip/applications/init/models/db.py", line 35,
>> in
>> > 
>> > auth.define_tables() # creates all needed
>> tables
>> >   File "/Users/mellis/w2ptip/gluon/tools.py", line 1106, in
>> define_tables
>> > format='%(first_name)s %(last_name)s (%(id)s)')
>> >   File "/Users/mellis/w2ptip/gluon/sql.py", line 1309, in define_table
>> >   File "/Users/mellis/w2ptip/gluon/sql.py", line 1715, in _create
>> > IOError: [Errno 24] Too many open files:
>> > '/Users/mellis/w2ptip/applications/init/databases/sql.log'
>> >
>> > On Tue, Jul 27, 2010 at 2:54 AM, Rahul  wrote:
>> > > Thanks everyone, for getting this issue resolved...
>> > > Web2py rocks!
>> > > Cheers, Rahul
>> >
>> > > On Jul 24, 6:25 pm, Phyo Arkar  wrote:
>> > > > Yes , Since from start of web2py , Massimo and Us never recommend to
>> use
>> > > > built-in web2py server at production. They are mainly for
>> development
>> > > > purpose.
>> > > > Since Very start of the project , since Cherrypy , Massimo always
>> Suggest
>> > > to
>> > > > use Apache/Cherokee/LighHTTPD over fcgi/wsgi/uwsgi or mod_python in
>> > > Serious
>> > > > Production Server.
>> >
>> > > > Rocket tho tries quite hard to achieve production level Performance
>> with
>> > > all
>> > > > Cool Comet/Threading Stuff . It still Quite Young .Lets give it a
>> chance.
>> >
>> > > > On Sat, Jul 24, 2010 at 7:39 PM, Scott 
>> wrote:
>> > > > > Please allow me to preface my comments: I have nothing against
>> Rocket;
>> > > > > my opinions come from years of experience with Java EE
>> deployments.
>> >
>> > > > > I think raising the max_threads to 1024 is a good idea.  However,
>> my
>> > > > > opinion is that Rocket alone should not be used for a production
>> > > > > deployment; much as I would not use the built-in Web server in
>> JBoss,
>> > > > > WebLogic, Geronimo, etc. as the front door.  My suggestion for
>> > > > > production would be to use an Apache front-end into Rocket.
>>  Apache is
>> > > > > more battle-hardened in this area, and it's a lot easier to handle
>> DoS
>> > > > > attacks through modules such as mod_evasive.  There are numerous
>> other
>> > > > > benefits too, such as easily enabling gzip compression and
>> allowing
>> > > > > you a better security model through Defense in Depth... but I
>> digress.
>> >
>> > > > > On Jul 23, 5:41 pm, mdipierro  wrote:
>> > > > > > On a second thought this open the door to more sever denial of
>> > > service
>

Re: [web2py] Re: rocket performance issues

[Michael Ellis]

Not using cron.




On Tue, Jul 27, 2010 at 8:43 AM, mdipierro  wrote:

> Ignore my previous email I see you are using 2.6 and the problem
> is with cron. I think the problem is a cron process that does not end
> and keeps restarting. Are you using cron?
>
> On Jul 27, 7:33 am, Michael Ellis  wrote:
> > Not sure if this is related to Rocket or whether a new topic is needed.
> >
> > This morning I found several OSError reports about "Too many open files"
> in
> > a web2py development server that's been running locally for several days.
> >  The app code isn't doing any explicit file i/o so I don't know what's
> going
> > on,  but here are the tracebacks in case someone else is seeing anything
> > similar.
> >
> > Mike
> >
> > Exception in thread Thread-89:
> > Traceback (most recent call last):
> >   File
> >
> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/threading.py",
> > line 522, in __bootstrap_inner
> > self.run()
> >   File "/Users/mellis/w2ptip/gluon/newcron.py", line 206, in run
> > shell=self.shell)
> >   File
> >
> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py",
> > line 595, in __init__
> > errread, errwrite)
> >   File
> >
> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py",
> > line 1018, in _execute_child
> > errpipe_read, errpipe_write = os.pipe()
> > OSError: [Errno 24] Too many open files
> >
> > ERROR:Rocket.Errors.ThreadPool:Traceback (most recent call last):
> >   File "/Users/mellis/w2ptip/gluon/rocket.py", line 302, in start
> > sock = l.accept()
> >   File
> >
> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/socket.py",
> > line 195, in accept
> > sock, addr = self._sock.accept()
> > error: [Errno 24] Too many open files
> >
> > ERROR:root:corrupted file:
> > /Users/mellis/w2ptip/applications/init/cache/cache.shelve
> > ERROR:root:Traceback (most recent call last):
> >   File "/Users/mellis/w2ptip/gluon/restricted.py", line 178, in
> restricted
> > exec ccode in environment
> >   File "/Users/mellis/w2ptip/applications/init/models/db.py", line 35, in
> > 
> > auth.define_tables() # creates all needed
> tables
> >   File "/Users/mellis/w2ptip/gluon/tools.py", line 1106, in define_tables
> > format='%(first_name)s %(last_name)s (%(id)s)')
> >   File "/Users/mellis/w2ptip/gluon/sql.py", line 1309, in define_table
> >   File "/Users/mellis/w2ptip/gluon/sql.py", line 1715, in _create
> > IOError: [Errno 24] Too many open files:
> > '/Users/mellis/w2ptip/applications/init/databases/sql.log'
> >
> > On Tue, Jul 27, 2010 at 2:54 AM, Rahul  wrote:
> > > Thanks everyone, for getting this issue resolved...
> > > Web2py rocks!
> > > Cheers, Rahul
> >
> > > On Jul 24, 6:25 pm, Phyo Arkar  wrote:
> > > > Yes , Since from start of web2py , Massimo and Us never recommend to
> use
> > > > built-in web2py server at production. They are mainly for development
> > > > purpose.
> > > > Since Very start of the project , since Cherrypy , Massimo always
> Suggest
> > > to
> > > > use Apache/Cherokee/LighHTTPD over fcgi/wsgi/uwsgi or mod_python in
> > > Serious
> > > > Production Server.
> >
> > > > Rocket tho tries quite hard to achieve production level Performance
> with
> > > all
> > > > Cool Comet/Threading Stuff . It still Quite Young .Lets give it a
> chance.
> >
> > > > On Sat, Jul 24, 2010 at 7:39 PM, Scott 
> wrote:
> > > > > Please allow me to preface my comments: I have nothing against
> Rocket;
> > > > > my opinions come from years of experience with Java EE deployments.
> >
> > > > > I think raising the max_threads to 1024 is a good idea.  However,
> my
> > > > > opinion is that Rocket alone should not be used for a production
> > > > > deployment; much as I would not use the built-in Web server in
> JBoss,
> > > > > WebLogic, Geronimo, etc. as the front door.  My suggestion for
> > > > > production would be to use an Apache front-end into Rocket.  Apache
> is
> > > > > more battle-hardened in this area, and it's a lot easier to handle
> DoS
> > > > > attacks through modules such as mod_evasive.  There are numerous
> other
> > > > > benefits too, such as easily enabling gzip compression and allowing
> > > > > you a better security model through Defense in Depth... but I
> digress.
> >
> > > > > On Jul 23, 5:41 pm, mdipierro  wrote:
> > > > > > On a second thought this open the door to more sever denial of
> > > service
> > > > > > attacks than caused by the original problem. How about, until
> there
> > > is
> > > > > > a better under understanding and solution, we just increase
> > > > > > max_threads from the original 128 to 1024.
> >
> > > > > > On Jul 22, 11:27 am, Timbo  wrote:
> >
> > > > > > > Try one quick change for me please...rocketis constructed
> around
> > > line
> > > > > > > 655 in main.py
> >
> > > > > > > Add a parameter to the constructor call(s):  max_threads=0
> >
> > > > > > > Please l

[web2py] Re: rocket performance issues

[mdipierro]

Ignore my previous email I see you are using 2.6 and the problem
is with cron. I think the problem is a cron process that does not end
and keeps restarting. Are you using cron?

On Jul 27, 7:33 am, Michael Ellis  wrote:
> Not sure if this is related to Rocket or whether a new topic is needed.
>
> This morning I found several OSError reports about "Too many open files" in
> a web2py development server that's been running locally for several days.
>  The app code isn't doing any explicit file i/o so I don't know what's going
> on,  but here are the tracebacks in case someone else is seeing anything
> similar.
>
> Mike
>
> Exception in thread Thread-89:
> Traceback (most recent call last):
>   File
> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/threading.py",
> line 522, in __bootstrap_inner
>     self.run()
>   File "/Users/mellis/w2ptip/gluon/newcron.py", line 206, in run
>     shell=self.shell)
>   File
> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py",
> line 595, in __init__
>     errread, errwrite)
>   File
> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py",
> line 1018, in _execute_child
>     errpipe_read, errpipe_write = os.pipe()
> OSError: [Errno 24] Too many open files
>
> ERROR:Rocket.Errors.ThreadPool:Traceback (most recent call last):
>   File "/Users/mellis/w2ptip/gluon/rocket.py", line 302, in start
>     sock = l.accept()
>   File
> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/socket.py",
> line 195, in accept
>     sock, addr = self._sock.accept()
> error: [Errno 24] Too many open files
>
> ERROR:root:corrupted file:
> /Users/mellis/w2ptip/applications/init/cache/cache.shelve
> ERROR:root:Traceback (most recent call last):
>   File "/Users/mellis/w2ptip/gluon/restricted.py", line 178, in restricted
>     exec ccode in environment
>   File "/Users/mellis/w2ptip/applications/init/models/db.py", line 35, in
> 
>     auth.define_tables()                         # creates all needed tables
>   File "/Users/mellis/w2ptip/gluon/tools.py", line 1106, in define_tables
>     format='%(first_name)s %(last_name)s (%(id)s)')
>   File "/Users/mellis/w2ptip/gluon/sql.py", line 1309, in define_table
>   File "/Users/mellis/w2ptip/gluon/sql.py", line 1715, in _create
> IOError: [Errno 24] Too many open files:
> '/Users/mellis/w2ptip/applications/init/databases/sql.log'
>
> On Tue, Jul 27, 2010 at 2:54 AM, Rahul  wrote:
> > Thanks everyone, for getting this issue resolved...
> > Web2py rocks!
> > Cheers, Rahul
>
> > On Jul 24, 6:25 pm, Phyo Arkar  wrote:
> > > Yes , Since from start of web2py , Massimo and Us never recommend to use
> > > built-in web2py server at production. They are mainly for development
> > > purpose.
> > > Since Very start of the project , since Cherrypy , Massimo always Suggest
> > to
> > > use Apache/Cherokee/LighHTTPD over fcgi/wsgi/uwsgi or mod_python in
> > Serious
> > > Production Server.
>
> > > Rocket tho tries quite hard to achieve production level Performance with
> > all
> > > Cool Comet/Threading Stuff . It still Quite Young .Lets give it a chance.
>
> > > On Sat, Jul 24, 2010 at 7:39 PM, Scott  wrote:
> > > > Please allow me to preface my comments: I have nothing against Rocket;
> > > > my opinions come from years of experience with Java EE deployments.
>
> > > > I think raising the max_threads to 1024 is a good idea.  However, my
> > > > opinion is that Rocket alone should not be used for a production
> > > > deployment; much as I would not use the built-in Web server in JBoss,
> > > > WebLogic, Geronimo, etc. as the front door.  My suggestion for
> > > > production would be to use an Apache front-end into Rocket.  Apache is
> > > > more battle-hardened in this area, and it's a lot easier to handle DoS
> > > > attacks through modules such as mod_evasive.  There are numerous other
> > > > benefits too, such as easily enabling gzip compression and allowing
> > > > you a better security model through Defense in Depth... but I digress.
>
> > > > On Jul 23, 5:41 pm, mdipierro  wrote:
> > > > > On a second thought this open the door to more sever denial of
> > service
> > > > > attacks than caused by the original problem. How about, until there
> > is
> > > > > a better under understanding and solution, we just increase
> > > > > max_threads from the original 128 to 1024.
>
> > > > > On Jul 22, 11:27 am, Timbo  wrote:
>
> > > > > > Try one quick change for me please...rocketis constructed around
> > line
> > > > > > 655 in main.py
>
> > > > > > Add a parameter to the constructor call(s):  max_threads=0
>
> > > > > > Please let me know if that affects the problem.
>
> > > > > > -tim
>
> > > > > > On Jul 22, 10:34 am, mdipierro  wrote:
>
> > > > > > > I can reproduce the problem. I did on localhost with two
> > different
> > > > > > > browsers.
> > > > > > > Using firebug I can see it takes 25seconds to download base.css
> > (the
> > > > > > > probl

[web2py] Re: rocket performance issues

[mdipierro]

Are you opening any files explicitely in your app? Which web2py
version and which Python version are you using?

On Jul 27, 7:33 am, Michael Ellis  wrote:
> Not sure if this is related to Rocket or whether a new topic is needed.
>
> This morning I found several OSError reports about "Too many open files" in
> a web2py development server that's been running locally for several days.
>  The app code isn't doing any explicit file i/o so I don't know what's going
> on,  but here are the tracebacks in case someone else is seeing anything
> similar.
>
> Mike
>
> Exception in thread Thread-89:
> Traceback (most recent call last):
>   File
> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/threading.py",
> line 522, in __bootstrap_inner
>     self.run()
>   File "/Users/mellis/w2ptip/gluon/newcron.py", line 206, in run
>     shell=self.shell)
>   File
> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py",
> line 595, in __init__
>     errread, errwrite)
>   File
> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py",
> line 1018, in _execute_child
>     errpipe_read, errpipe_write = os.pipe()
> OSError: [Errno 24] Too many open files
>
> ERROR:Rocket.Errors.ThreadPool:Traceback (most recent call last):
>   File "/Users/mellis/w2ptip/gluon/rocket.py", line 302, in start
>     sock = l.accept()
>   File
> "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/socket.py",
> line 195, in accept
>     sock, addr = self._sock.accept()
> error: [Errno 24] Too many open files
>
> ERROR:root:corrupted file:
> /Users/mellis/w2ptip/applications/init/cache/cache.shelve
> ERROR:root:Traceback (most recent call last):
>   File "/Users/mellis/w2ptip/gluon/restricted.py", line 178, in restricted
>     exec ccode in environment
>   File "/Users/mellis/w2ptip/applications/init/models/db.py", line 35, in
> 
>     auth.define_tables()                         # creates all needed tables
>   File "/Users/mellis/w2ptip/gluon/tools.py", line 1106, in define_tables
>     format='%(first_name)s %(last_name)s (%(id)s)')
>   File "/Users/mellis/w2ptip/gluon/sql.py", line 1309, in define_table
>   File "/Users/mellis/w2ptip/gluon/sql.py", line 1715, in _create
> IOError: [Errno 24] Too many open files:
> '/Users/mellis/w2ptip/applications/init/databases/sql.log'
>
> On Tue, Jul 27, 2010 at 2:54 AM, Rahul  wrote:
> > Thanks everyone, for getting this issue resolved...
> > Web2py rocks!
> > Cheers, Rahul
>
> > On Jul 24, 6:25 pm, Phyo Arkar  wrote:
> > > Yes , Since from start of web2py , Massimo and Us never recommend to use
> > > built-in web2py server at production. They are mainly for development
> > > purpose.
> > > Since Very start of the project , since Cherrypy , Massimo always Suggest
> > to
> > > use Apache/Cherokee/LighHTTPD over fcgi/wsgi/uwsgi or mod_python in
> > Serious
> > > Production Server.
>
> > > Rocket tho tries quite hard to achieve production level Performance with
> > all
> > > Cool Comet/Threading Stuff . It still Quite Young .Lets give it a chance.
>
> > > On Sat, Jul 24, 2010 at 7:39 PM, Scott  wrote:
> > > > Please allow me to preface my comments: I have nothing against Rocket;
> > > > my opinions come from years of experience with Java EE deployments.
>
> > > > I think raising the max_threads to 1024 is a good idea.  However, my
> > > > opinion is that Rocket alone should not be used for a production
> > > > deployment; much as I would not use the built-in Web server in JBoss,
> > > > WebLogic, Geronimo, etc. as the front door.  My suggestion for
> > > > production would be to use an Apache front-end into Rocket.  Apache is
> > > > more battle-hardened in this area, and it's a lot easier to handle DoS
> > > > attacks through modules such as mod_evasive.  There are numerous other
> > > > benefits too, such as easily enabling gzip compression and allowing
> > > > you a better security model through Defense in Depth... but I digress.
>
> > > > On Jul 23, 5:41 pm, mdipierro  wrote:
> > > > > On a second thought this open the door to more sever denial of
> > service
> > > > > attacks than caused by the original problem. How about, until there
> > is
> > > > > a better under understanding and solution, we just increase
> > > > > max_threads from the original 128 to 1024.
>
> > > > > On Jul 22, 11:27 am, Timbo  wrote:
>
> > > > > > Try one quick change for me please...rocketis constructed around
> > line
> > > > > > 655 in main.py
>
> > > > > > Add a parameter to the constructor call(s):  max_threads=0
>
> > > > > > Please let me know if that affects the problem.
>
> > > > > > -tim
>
> > > > > > On Jul 22, 10:34 am, mdipierro  wrote:
>
> > > > > > > I can reproduce the problem. I did on localhost with two
> > different
> > > > > > > browsers.
> > > > > > > Using firebug I can see it takes 25seconds to download base.css
> > (the
> > > > > > > problem is not always with the same file).
> > > > > > > While I did the t

Re: [web2py] Re: rocket performance issues

[Michael Ellis]

Not sure if this is related to Rocket or whether a new topic is needed.

This morning I found several OSError reports about "Too many open files" in
a web2py development server that's been running locally for several days.
 The app code isn't doing any explicit file i/o so I don't know what's going
on,  but here are the tracebacks in case someone else is seeing anything
similar.

Mike

Exception in thread Thread-89:
Traceback (most recent call last):
  File
"/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/threading.py",
line 522, in __bootstrap_inner
self.run()
  File "/Users/mellis/w2ptip/gluon/newcron.py", line 206, in run
shell=self.shell)
  File
"/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py",
line 595, in __init__
errread, errwrite)
  File
"/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py",
line 1018, in _execute_child
errpipe_read, errpipe_write = os.pipe()
OSError: [Errno 24] Too many open files

ERROR:Rocket.Errors.ThreadPool:Traceback (most recent call last):
  File "/Users/mellis/w2ptip/gluon/rocket.py", line 302, in start
sock = l.accept()
  File
"/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/socket.py",
line 195, in accept
sock, addr = self._sock.accept()
error: [Errno 24] Too many open files

ERROR:root:corrupted file:
/Users/mellis/w2ptip/applications/init/cache/cache.shelve
ERROR:root:Traceback (most recent call last):
  File "/Users/mellis/w2ptip/gluon/restricted.py", line 178, in restricted
exec ccode in environment
  File "/Users/mellis/w2ptip/applications/init/models/db.py", line 35, in

auth.define_tables() # creates all needed tables
  File "/Users/mellis/w2ptip/gluon/tools.py", line 1106, in define_tables
format='%(first_name)s %(last_name)s (%(id)s)')
  File "/Users/mellis/w2ptip/gluon/sql.py", line 1309, in define_table
  File "/Users/mellis/w2ptip/gluon/sql.py", line 1715, in _create
IOError: [Errno 24] Too many open files:
'/Users/mellis/w2ptip/applications/init/databases/sql.log'



On Tue, Jul 27, 2010 at 2:54 AM, Rahul  wrote:

> Thanks everyone, for getting this issue resolved...
> Web2py rocks!
> Cheers, Rahul
>
> On Jul 24, 6:25 pm, Phyo Arkar  wrote:
> > Yes , Since from start of web2py , Massimo and Us never recommend to use
> > built-in web2py server at production. They are mainly for development
> > purpose.
> > Since Very start of the project , since Cherrypy , Massimo always Suggest
> to
> > use Apache/Cherokee/LighHTTPD over fcgi/wsgi/uwsgi or mod_python in
> Serious
> > Production Server.
> >
> > Rocket tho tries quite hard to achieve production level Performance with
> all
> > Cool Comet/Threading Stuff . It still Quite Young .Lets give it a chance.
> >
> > On Sat, Jul 24, 2010 at 7:39 PM, Scott  wrote:
> > > Please allow me to preface my comments: I have nothing against Rocket;
> > > my opinions come from years of experience with Java EE deployments.
> >
> > > I think raising the max_threads to 1024 is a good idea.  However, my
> > > opinion is that Rocket alone should not be used for a production
> > > deployment; much as I would not use the built-in Web server in JBoss,
> > > WebLogic, Geronimo, etc. as the front door.  My suggestion for
> > > production would be to use an Apache front-end into Rocket.  Apache is
> > > more battle-hardened in this area, and it's a lot easier to handle DoS
> > > attacks through modules such as mod_evasive.  There are numerous other
> > > benefits too, such as easily enabling gzip compression and allowing
> > > you a better security model through Defense in Depth... but I digress.
> >
> > > On Jul 23, 5:41 pm, mdipierro  wrote:
> > > > On a second thought this open the door to more sever denial of
> service
> > > > attacks than caused by the original problem. How about, until there
> is
> > > > a better under understanding and solution, we just increase
> > > > max_threads from the original 128 to 1024.
> >
> > > > On Jul 22, 11:27 am, Timbo  wrote:
> >
> > > > > Try one quick change for me please...rocketis constructed around
> line
> > > > > 655 in main.py
> >
> > > > > Add a parameter to the constructor call(s):  max_threads=0
> >
> > > > > Please let me know if that affects the problem.
> >
> > > > > -tim
> >
> > > > > On Jul 22, 10:34 am, mdipierro  wrote:
> >
> > > > > > I can reproduce the problem. I did on localhost with two
> different
> > > > > > browsers.
> > > > > > Using firebug I can see it takes 25seconds to download base.css
> (the
> > > > > > problem is not always with the same file).
> > > > > > While I did the test, I also monitored httpserver.log and I find
> that
> > > > > > it NEVER takes more than 1.2ms serve base.css.
> > > > > > This is what the log shows:
> >
> > > > > > 
> > > > > > 127.0.0.1, 2010-07-22 10:16:38, GET,
> /michealellistest/static/images/
> > > > > > header.png, HTTP/1.1, 304, 0.000563
> > > > > > 127.0.0.1

[web2py] Re: rocket performance issues

[Rahul]

Thanks everyone, for getting this issue resolved...
Web2py rocks!
Cheers, Rahul

On Jul 24, 6:25 pm, Phyo Arkar  wrote:
> Yes , Since from start of web2py , Massimo and Us never recommend to use
> built-in web2py server at production. They are mainly for development
> purpose.
> Since Very start of the project , since Cherrypy , Massimo always Suggest to
> use Apache/Cherokee/LighHTTPD over fcgi/wsgi/uwsgi or mod_python in Serious
> Production Server.
>
> Rocket tho tries quite hard to achieve production level Performance with all
> Cool Comet/Threading Stuff . It still Quite Young .Lets give it a chance.
>
> On Sat, Jul 24, 2010 at 7:39 PM, Scott  wrote:
> > Please allow me to preface my comments: I have nothing against Rocket;
> > my opinions come from years of experience with Java EE deployments.
>
> > I think raising the max_threads to 1024 is a good idea.  However, my
> > opinion is that Rocket alone should not be used for a production
> > deployment; much as I would not use the built-in Web server in JBoss,
> > WebLogic, Geronimo, etc. as the front door.  My suggestion for
> > production would be to use an Apache front-end into Rocket.  Apache is
> > more battle-hardened in this area, and it's a lot easier to handle DoS
> > attacks through modules such as mod_evasive.  There are numerous other
> > benefits too, such as easily enabling gzip compression and allowing
> > you a better security model through Defense in Depth... but I digress.
>
> > On Jul 23, 5:41 pm, mdipierro  wrote:
> > > On a second thought this open the door to more sever denial of service
> > > attacks than caused by the original problem. How about, until there is
> > > a better under understanding and solution, we just increase
> > > max_threads from the original 128 to 1024.
>
> > > On Jul 22, 11:27 am, Timbo  wrote:
>
> > > > Try one quick change for me please...rocketis constructed around line
> > > > 655 in main.py
>
> > > > Add a parameter to the constructor call(s):  max_threads=0
>
> > > > Please let me know if that affects the problem.
>
> > > > -tim
>
> > > > On Jul 22, 10:34 am, mdipierro  wrote:
>
> > > > > I can reproduce the problem. I did on localhost with two different
> > > > > browsers.
> > > > > Using firebug I can see it takes 25seconds to download base.css (the
> > > > > problem is not always with the same file).
> > > > > While I did the test, I also monitored httpserver.log and I find that
> > > > > it NEVER takes more than 1.2ms serve base.css.
> > > > > This is what the log shows:
>
> > > > > 
> > > > > 127.0.0.1, 2010-07-22 10:16:38, GET, /michealellistest/static/images/
> > > > > header.png, HTTP/1.1, 304, 0.000563
> > > > > 127.0.0.1, 2010-07-22 10:16:38, GET, /favicon.ico, HTTP/1.1, 400,
> > > > > 0.000631
> > > > > 127.0.0.1, 2010-07-22 10:16:55, GET, /michealellistest/static/
> > > > > base.css, HTTP/1.1, 304, 0.000791    locks firefox for 25secs
> > > > > 
> > > > > 127.0.0.1, 2010-07-22 10:22:42, GET, /michealellistest/static/
> > > > > jquery.timers-1.2.js, HTTP/1.1, 304, 0.000552
> > > > > 127.0.0.1, 2010-07-22 10:22:42, GET, /favicon.ico, HTTP/1.1, 400,
> > > > > 0.000497
> > > > > 127.0.0.1, 2010-07-22 10:23:02, GET, /michealellistest/static/
> > > > > superfish.js, HTTP/1.1, 304, 0.000914    locks chrome for 25secs
>
> > > > > Do you see the time gaps?
>
> > > > > There is a clear pattern. Under heavy load a request that results in
> > a
> > > > > HTTP 400 error locksRocket.
>
> > > > > Notice that the logging is done by a wsgi application that calls
> > > > > web2py wsgibase, i.e it time how long it takes web2py to receive the
> > > > > request and send the response. The extra time must be spent inside
> > the
> > > > > web server.
>
> > > > > It is also important that the times showed in the logs are the actual
> > > > > time when the data is being written in the logs. You can see firefox
> > > > > waiting for base.css, the server waiting to log base.css and nothing
> > > > > else is being printed during the wait, signifying that web2py is not
> > > > > running any request.
>
> > > > > We need Tim! This is a problem.
>
> > > > > Massimo
>
> > > > > On Jul 22, 9:22 am, Michael Ellis  wrote:
>
> > > > > > I've isolated the problem but absolutely do not understand it.  I
> > can
> > > > > > reproduce it with a two-line change to web2py_ajax.html.   Will
> > someone with
> > > > > > the time and equipment please attempt to replicate  this as a
> > sanity check?
>
> > > > > > Here's how:
>
> > > > > > In the welcome app's web2py_ajax.html, insert the following after
> > line 3.
>
> > response.files.insert(3,URL(r=request,c='static',f='jquery.sparkline.js'))
>
> > response.files.insert(4,URL(r=request,c='static',f='jquery.timers-1.2.js'))
>
> > > > > > Copy the attached js files into welcome/static.  They should be the
> > same as
> > > > > > the versions available online.
>
> > > > > > To reproduce the problem, serve web2py on your LAN.  Open the
> > welcome home
> > > > > > page on

Re: [web2py] Re: rocket performance issues

[Phyo Arkar]

Yes , Since from start of web2py , Massimo and Us never recommend to use
built-in web2py server at production. They are mainly for development
purpose.
Since Very start of the project , since Cherrypy , Massimo always Suggest to
use Apache/Cherokee/LighHTTPD over fcgi/wsgi/uwsgi or mod_python in Serious
Production Server.

Rocket tho tries quite hard to achieve production level Performance with all
Cool Comet/Threading Stuff . It still Quite Young .Lets give it a chance.

On Sat, Jul 24, 2010 at 7:39 PM, Scott  wrote:

> Please allow me to preface my comments: I have nothing against Rocket;
> my opinions come from years of experience with Java EE deployments.
>
> I think raising the max_threads to 1024 is a good idea.  However, my
> opinion is that Rocket alone should not be used for a production
> deployment; much as I would not use the built-in Web server in JBoss,
> WebLogic, Geronimo, etc. as the front door.  My suggestion for
> production would be to use an Apache front-end into Rocket.  Apache is
> more battle-hardened in this area, and it's a lot easier to handle DoS
> attacks through modules such as mod_evasive.  There are numerous other
> benefits too, such as easily enabling gzip compression and allowing
> you a better security model through Defense in Depth... but I digress.
>
> On Jul 23, 5:41 pm, mdipierro  wrote:
> > On a second thought this open the door to more sever denial of service
> > attacks than caused by the original problem. How about, until there is
> > a better under understanding and solution, we just increase
> > max_threads from the original 128 to 1024.
> >
> > On Jul 22, 11:27 am, Timbo  wrote:
> >
> >
> >
> >
> >
> >
> >
> > > Try one quick change for me please...rocketis constructed around line
> > > 655 in main.py
> >
> > > Add a parameter to the constructor call(s):  max_threads=0
> >
> > > Please let me know if that affects the problem.
> >
> > > -tim
> >
> > > On Jul 22, 10:34 am, mdipierro  wrote:
> >
> > > > I can reproduce the problem. I did on localhost with two different
> > > > browsers.
> > > > Using firebug I can see it takes 25seconds to download base.css (the
> > > > problem is not always with the same file).
> > > > While I did the test, I also monitored httpserver.log and I find that
> > > > it NEVER takes more than 1.2ms serve base.css.
> > > > This is what the log shows:
> >
> > > > 
> > > > 127.0.0.1, 2010-07-22 10:16:38, GET, /michealellistest/static/images/
> > > > header.png, HTTP/1.1, 304, 0.000563
> > > > 127.0.0.1, 2010-07-22 10:16:38, GET, /favicon.ico, HTTP/1.1, 400,
> > > > 0.000631
> > > > 127.0.0.1, 2010-07-22 10:16:55, GET, /michealellistest/static/
> > > > base.css, HTTP/1.1, 304, 0.000791    locks firefox for 25secs
> > > > 
> > > > 127.0.0.1, 2010-07-22 10:22:42, GET, /michealellistest/static/
> > > > jquery.timers-1.2.js, HTTP/1.1, 304, 0.000552
> > > > 127.0.0.1, 2010-07-22 10:22:42, GET, /favicon.ico, HTTP/1.1, 400,
> > > > 0.000497
> > > > 127.0.0.1, 2010-07-22 10:23:02, GET, /michealellistest/static/
> > > > superfish.js, HTTP/1.1, 304, 0.000914    locks chrome for 25secs
> >
> > > > Do you see the time gaps?
> >
> > > > There is a clear pattern. Under heavy load a request that results in
> a
> > > > HTTP 400 error locksRocket.
> >
> > > > Notice that the logging is done by a wsgi application that calls
> > > > web2py wsgibase, i.e it time how long it takes web2py to receive the
> > > > request and send the response. The extra time must be spent inside
> the
> > > > web server.
> >
> > > > It is also important that the times showed in the logs are the actual
> > > > time when the data is being written in the logs. You can see firefox
> > > > waiting for base.css, the server waiting to log base.css and nothing
> > > > else is being printed during the wait, signifying that web2py is not
> > > > running any request.
> >
> > > > We need Tim! This is a problem.
> >
> > > > Massimo
> >
> > > > On Jul 22, 9:22 am, Michael Ellis  wrote:
> >
> > > > > I've isolated the problem but absolutely do not understand it.  I
> can
> > > > > reproduce it with a two-line change to web2py_ajax.html.   Will
> someone with
> > > > > the time and equipment please attempt to replicate  this as a
> sanity check?
> >
> > > > > Here's how:
> >
> > > > > In the welcome app's web2py_ajax.html, insert the following after
> line 3.
> > > > >
> response.files.insert(3,URL(r=request,c='static',f='jquery.sparkline.js'))
> > > > >
> response.files.insert(4,URL(r=request,c='static',f='jquery.timers-1.2.js'))
> >
> > > > > Copy the attached js files into welcome/static.  They should be the
> same as
> > > > > the versions available online.
> >
> > > > > To reproduce the problem, serve web2py on your LAN.  Open the
> welcome home
> > > > > page on two different machines.  One of them can be on the server.
>  Briskly
> > > > > reload the page 10 or more times on either machine then try to
> reload on the
> > > > > other.  In my setup, the delay is reliably 25

[web2py] Re: rocket performance issues

[Scott]

Please allow me to preface my comments: I have nothing against Rocket;
my opinions come from years of experience with Java EE deployments.

I think raising the max_threads to 1024 is a good idea.  However, my
opinion is that Rocket alone should not be used for a production
deployment; much as I would not use the built-in Web server in JBoss,
WebLogic, Geronimo, etc. as the front door.  My suggestion for
production would be to use an Apache front-end into Rocket.  Apache is
more battle-hardened in this area, and it's a lot easier to handle DoS
attacks through modules such as mod_evasive.  There are numerous other
benefits too, such as easily enabling gzip compression and allowing
you a better security model through Defense in Depth... but I digress.

On Jul 23, 5:41 pm, mdipierro  wrote:
> On a second thought this open the door to more sever denial of service
> attacks than caused by the original problem. How about, until there is
> a better under understanding and solution, we just increase
> max_threads from the original 128 to 1024.
>
> On Jul 22, 11:27 am, Timbo  wrote:
>
>
>
>
>
>
>
> > Try one quick change for me please...rocketis constructed around line
> > 655 in main.py
>
> > Add a parameter to the constructor call(s):  max_threads=0
>
> > Please let me know if that affects the problem.
>
> > -tim
>
> > On Jul 22, 10:34 am, mdipierro  wrote:
>
> > > I can reproduce the problem. I did on localhost with two different
> > > browsers.
> > > Using firebug I can see it takes 25seconds to download base.css (the
> > > problem is not always with the same file).
> > > While I did the test, I also monitored httpserver.log and I find that
> > > it NEVER takes more than 1.2ms serve base.css.
> > > This is what the log shows:
>
> > > 
> > > 127.0.0.1, 2010-07-22 10:16:38, GET, /michealellistest/static/images/
> > > header.png, HTTP/1.1, 304, 0.000563
> > > 127.0.0.1, 2010-07-22 10:16:38, GET, /favicon.ico, HTTP/1.1, 400,
> > > 0.000631
> > > 127.0.0.1, 2010-07-22 10:16:55, GET, /michealellistest/static/
> > > base.css, HTTP/1.1, 304, 0.000791    locks firefox for 25secs
> > > 
> > > 127.0.0.1, 2010-07-22 10:22:42, GET, /michealellistest/static/
> > > jquery.timers-1.2.js, HTTP/1.1, 304, 0.000552
> > > 127.0.0.1, 2010-07-22 10:22:42, GET, /favicon.ico, HTTP/1.1, 400,
> > > 0.000497
> > > 127.0.0.1, 2010-07-22 10:23:02, GET, /michealellistest/static/
> > > superfish.js, HTTP/1.1, 304, 0.000914    locks chrome for 25secs
>
> > > Do you see the time gaps?
>
> > > There is a clear pattern. Under heavy load a request that results in a
> > > HTTP 400 error locksRocket.
>
> > > Notice that the logging is done by a wsgi application that calls
> > > web2py wsgibase, i.e it time how long it takes web2py to receive the
> > > request and send the response. The extra time must be spent inside the
> > > web server.
>
> > > It is also important that the times showed in the logs are the actual
> > > time when the data is being written in the logs. You can see firefox
> > > waiting for base.css, the server waiting to log base.css and nothing
> > > else is being printed during the wait, signifying that web2py is not
> > > running any request.
>
> > > We need Tim! This is a problem.
>
> > > Massimo
>
> > > On Jul 22, 9:22 am, Michael Ellis  wrote:
>
> > > > I've isolated the problem but absolutely do not understand it.  I can
> > > > reproduce it with a two-line change to web2py_ajax.html.   Will someone 
> > > > with
> > > > the time and equipment please attempt to replicate  this as a sanity 
> > > > check?
>
> > > > Here's how:
>
> > > > In the welcome app's web2py_ajax.html, insert the following after line 
> > > > 3.
> > > > response.files.insert(3,URL(r=request,c='static',f='jquery.sparkline.js'))
> > > > response.files.insert(4,URL(r=request,c='static',f='jquery.timers-1.2.js'))
>
> > > > Copy the attached js files into welcome/static.  They should be the 
> > > > same as
> > > > the versions available online.
>
> > > > To reproduce the problem, serve web2py on your LAN.  Open the welcome 
> > > > home
> > > > page on two different machines.  One of them can be on the server.  
> > > > Briskly
> > > > reload the page 10 or more times on either machine then try to reload 
> > > > on the
> > > > other.  In my setup, the delay is reliably 25 seconds from the time I 
> > > > make
> > > > the last click on the first machine.
>
> > > > I'm able to reproduce this in FF, Chrome, and Safari using the latest 
> > > > web2py
> > > > from trunk.  Haven't tried any other browsers yet.  As noted previously 
> > > > both
> > > > machines are MacBooks running Snow Leopard.
>
> > > > Mike
>
> > > >  jquery.timers-1.2.js
> > > > 4KViewDownload
>
> > > >  jquery.sparkline.js
> > > > 62KViewDownload

[web2py] Re: rocket performance issues

[mdipierro]

On a second thought this open the door to more sever denial of service
attacks than caused by the original problem. How about, until there is
a better under understanding and solution, we just increase
max_threads from the original 128 to 1024.

On Jul 22, 11:27 am, Timbo  wrote:
> Try one quick change for me please...rocketis constructed around line
> 655 in main.py
>
> Add a parameter to the constructor call(s):  max_threads=0
>
> Please let me know if that affects the problem.
>
> -tim
>
> On Jul 22, 10:34 am, mdipierro  wrote:
>
> > I can reproduce the problem. I did on localhost with two different
> > browsers.
> > Using firebug I can see it takes 25seconds to download base.css (the
> > problem is not always with the same file).
> > While I did the test, I also monitored httpserver.log and I find that
> > it NEVER takes more than 1.2ms serve base.css.
> > This is what the log shows:
>
> > 
> > 127.0.0.1, 2010-07-22 10:16:38, GET, /michealellistest/static/images/
> > header.png, HTTP/1.1, 304, 0.000563
> > 127.0.0.1, 2010-07-22 10:16:38, GET, /favicon.ico, HTTP/1.1, 400,
> > 0.000631
> > 127.0.0.1, 2010-07-22 10:16:55, GET, /michealellistest/static/
> > base.css, HTTP/1.1, 304, 0.000791    locks firefox for 25secs
> > 
> > 127.0.0.1, 2010-07-22 10:22:42, GET, /michealellistest/static/
> > jquery.timers-1.2.js, HTTP/1.1, 304, 0.000552
> > 127.0.0.1, 2010-07-22 10:22:42, GET, /favicon.ico, HTTP/1.1, 400,
> > 0.000497
> > 127.0.0.1, 2010-07-22 10:23:02, GET, /michealellistest/static/
> > superfish.js, HTTP/1.1, 304, 0.000914    locks chrome for 25secs
>
> > Do you see the time gaps?
>
> > There is a clear pattern. Under heavy load a request that results in a
> > HTTP 400 error locksRocket.
>
> > Notice that the logging is done by a wsgi application that calls
> > web2py wsgibase, i.e it time how long it takes web2py to receive the
> > request and send the response. The extra time must be spent inside the
> > web server.
>
> > It is also important that the times showed in the logs are the actual
> > time when the data is being written in the logs. You can see firefox
> > waiting for base.css, the server waiting to log base.css and nothing
> > else is being printed during the wait, signifying that web2py is not
> > running any request.
>
> > We need Tim! This is a problem.
>
> > Massimo
>
> > On Jul 22, 9:22 am, Michael Ellis  wrote:
>
> > > I've isolated the problem but absolutely do not understand it.  I can
> > > reproduce it with a two-line change to web2py_ajax.html.   Will someone 
> > > with
> > > the time and equipment please attempt to replicate  this as a sanity 
> > > check?
>
> > > Here's how:
>
> > > In the welcome app's web2py_ajax.html, insert the following after line 3.
> > > response.files.insert(3,URL(r=request,c='static',f='jquery.sparkline.js'))
> > > response.files.insert(4,URL(r=request,c='static',f='jquery.timers-1.2.js'))
>
> > > Copy the attached js files into welcome/static.  They should be the same 
> > > as
> > > the versions available online.
>
> > > To reproduce the problem, serve web2py on your LAN.  Open the welcome home
> > > page on two different machines.  One of them can be on the server.  
> > > Briskly
> > > reload the page 10 or more times on either machine then try to reload on 
> > > the
> > > other.  In my setup, the delay is reliably 25 seconds from the time I make
> > > the last click on the first machine.
>
> > > I'm able to reproduce this in FF, Chrome, and Safari using the latest 
> > > web2py
> > > from trunk.  Haven't tried any other browsers yet.  As noted previously 
> > > both
> > > machines are MacBooks running Snow Leopard.
>
> > > Mike
>
> > >  jquery.timers-1.2.js
> > > 4KViewDownload
>
> > >  jquery.sparkline.js
> > > 62KViewDownload

Re: [web2py] Re: rocket performance issues

[Michele Comitini]

Yes that could be really a small problem with Rocket's threaded
implementation, not the GIL, which usually shows up
on higher concurrency loads.  Anyway that would be a point for Guido... :D


ciao,
mic

2010/7/23 mdipierro 

> I am not convinced this is a GIL problem. Why the lock after the 400
> errors only?
>
> On Jul 23, 9:18 am, Michele Comitini 
> wrote:
> > Because of the GIL threads on CPython are problematic more than one could
> > think.  Googling "GIL python" gives an idea...
> > Not to start the war threading vs forking model in general, but the GIL
> will
> > not go away from van Rossum's
> > python implementation (no not even in 3.x), he rather suggests to stay
> out
> > of threading (and he means that in general).
> >
> > This is better clarified in this Guido's mail:
> >
> > http://mail.python.org/pipermail/python-3000/2007-May/007414.html
> >
> > ciao,
> > mic
> >
> > 2010/7/22 Timbo 
> >
> > > OK, so it's a thread-starvation issue.  Let this be the interim fix
> > > for web2py.  I've got several busy days ahead of me so I'll put out a
> > > real fix for Rocket in the coming days.
> >
> > > -tim
> >
> > > On Jul 22, 11:39 am, Michael Ellis  wrote:
> > > > Much improved with suggested change.  Latencies for .js/.css files
> > > reduced
> > > > to 100-150 msec.  Thanks Tim!
> >
> > > > Cheers,
> > > > Mike
> >
> > > > On Thu, Jul 22, 2010 at 12:27 PM, Timbo 
> > > wrote:
> > > > > Try one quick change for me please...rocket is constructed around
> line
> > > > > 655 in main.py
> >
> > > > > Add a parameter to the constructor call(s):  max_threads=0
> >
> > > > > Please let me know if that affects the problem.
> >
> > > > > -tim
> >
> > > > > On Jul 22, 10:34 am, mdipierro  wrote:
> > > > > > I can reproduce the problem. I did on localhost with two
> different
> > > > > > browsers.
> > > > > > Using firebug I can see it takes 25seconds to download base.css
> (the
> > > > > > problem is not always with the same file).
> > > > > > While I did the test, I also monitored httpserver.log and I find
> that
> > > > > > it NEVER takes more than 1.2ms serve base.css.
> > > > > > This is what the log shows:
> >
> > > > > > 
> > > > > > 127.0.0.1, 2010-07-22 10:16:38, GET,
> /michealellistest/static/images/
> > > > > > header.png, HTTP/1.1, 304, 0.000563
> > > > > > 127.0.0.1, 2010-07-22 10:16:38, GET, /favicon.ico, HTTP/1.1, 400,
> > > > > > 0.000631
> > > > > > 127.0.0.1, 2010-07-22 10:16:55, GET, /michealellistest/static/
> > > > > > base.css, HTTP/1.1, 304, 0.000791    locks firefox for 25secs
> > > > > > 
> > > > > > 127.0.0.1, 2010-07-22 10:22:42, GET, /michealellistest/static/
> > > > > > jquery.timers-1.2.js, HTTP/1.1, 304, 0.000552
> > > > > > 127.0.0.1, 2010-07-22 10:22:42, GET, /favicon.ico, HTTP/1.1, 400,
> > > > > > 0.000497
> > > > > > 127.0.0.1, 2010-07-22 10:23:02, GET, /michealellistest/static/
> > > > > > superfish.js, HTTP/1.1, 304, 0.000914    locks chrome for
> 25secs
> >
> > > > > > Do you see the time gaps?
> >
> > > > > > There is a clear pattern. Under heavy load a request that results
> in
> > > a
> > > > > > HTTP 400 error locks Rocket.
> >
> > > > > > Notice that the logging is done by a wsgi application that calls
> > > > > > web2py wsgibase, i.e it time how long it takes web2py to receive
> the
> > > > > > request and send the response. The extra time must be spent
> inside
> > > the
> > > > > > web server.
> >
> > > > > > It is also important that the times showed in the logs are the
> actual
> > > > > > time when the data is being written in the logs. You can see
> firefox
> > > > > > waiting for base.css, the server waiting to log base.css and
> nothing
> > > > > > else is being printed during the wait, signifying that web2py is
> not
> > > > > > running any request.
> >
> > > > > > We need Tim! This is a problem.
> >
> > > > > > Massimo
> >
> > > > > > On Jul 22, 9:22 am, Michael Ellis 
> wrote:
> >
> > > > > > > I've isolated the problem but absolutely do not understand it.
>  I
> > > can
> > > > > > > reproduce it with a two-line change to web2py_ajax.html.   Will
> > > someone
> > > > > with
> > > > > > > the time and equipment please attempt to replicate  this as a
> > > sanity
> > > > > check?
> >
> > > > > > > Here's how:
> >
> > > > > > > In the welcome app's web2py_ajax.html, insert the following
> after
> > > line
> > > > > 3.
> >
> > >
> response.files.insert(3,URL(r=request,c='static',f='jquery.sparkline.js'))
> >
> > >
> response.files.insert(4,URL(r=request,c='static',f='jquery.timers-1.2.js'))
> >
> > > > > > > Copy the attached js files into welcome/static.  They should be
> the
> > > > > same as
> > > > > > > the versions available online.
> >
> > > > > > > To reproduce the problem, serve web2py on your LAN.  Open the
> > > welcome
> > > > > home
> > > > > > > page on two different machines.  One of them can be on the
> server.
> > > > >  Briskly
> > > > > > > reload the page 10 or more times on either machine then try to
> > > reload
> > 

[web2py] Re: rocket performance issues

[mdipierro]

I am not convinced this is a GIL problem. Why the lock after the 400
errors only?

On Jul 23, 9:18 am, Michele Comitini 
wrote:
> Because of the GIL threads on CPython are problematic more than one could
> think.  Googling "GIL python" gives an idea...
> Not to start the war threading vs forking model in general, but the GIL will
> not go away from van Rossum's
> python implementation (no not even in 3.x), he rather suggests to stay out
> of threading (and he means that in general).
>
> This is better clarified in this Guido's mail:
>
> http://mail.python.org/pipermail/python-3000/2007-May/007414.html
>
> ciao,
> mic
>
> 2010/7/22 Timbo 
>
> > OK, so it's a thread-starvation issue.  Let this be the interim fix
> > for web2py.  I've got several busy days ahead of me so I'll put out a
> > real fix for Rocket in the coming days.
>
> > -tim
>
> > On Jul 22, 11:39 am, Michael Ellis  wrote:
> > > Much improved with suggested change.  Latencies for .js/.css files
> > reduced
> > > to 100-150 msec.  Thanks Tim!
>
> > > Cheers,
> > > Mike
>
> > > On Thu, Jul 22, 2010 at 12:27 PM, Timbo 
> > wrote:
> > > > Try one quick change for me please...rocket is constructed around line
> > > > 655 in main.py
>
> > > > Add a parameter to the constructor call(s):  max_threads=0
>
> > > > Please let me know if that affects the problem.
>
> > > > -tim
>
> > > > On Jul 22, 10:34 am, mdipierro  wrote:
> > > > > I can reproduce the problem. I did on localhost with two different
> > > > > browsers.
> > > > > Using firebug I can see it takes 25seconds to download base.css (the
> > > > > problem is not always with the same file).
> > > > > While I did the test, I also monitored httpserver.log and I find that
> > > > > it NEVER takes more than 1.2ms serve base.css.
> > > > > This is what the log shows:
>
> > > > > 
> > > > > 127.0.0.1, 2010-07-22 10:16:38, GET, /michealellistest/static/images/
> > > > > header.png, HTTP/1.1, 304, 0.000563
> > > > > 127.0.0.1, 2010-07-22 10:16:38, GET, /favicon.ico, HTTP/1.1, 400,
> > > > > 0.000631
> > > > > 127.0.0.1, 2010-07-22 10:16:55, GET, /michealellistest/static/
> > > > > base.css, HTTP/1.1, 304, 0.000791    locks firefox for 25secs
> > > > > 
> > > > > 127.0.0.1, 2010-07-22 10:22:42, GET, /michealellistest/static/
> > > > > jquery.timers-1.2.js, HTTP/1.1, 304, 0.000552
> > > > > 127.0.0.1, 2010-07-22 10:22:42, GET, /favicon.ico, HTTP/1.1, 400,
> > > > > 0.000497
> > > > > 127.0.0.1, 2010-07-22 10:23:02, GET, /michealellistest/static/
> > > > > superfish.js, HTTP/1.1, 304, 0.000914    locks chrome for 25secs
>
> > > > > Do you see the time gaps?
>
> > > > > There is a clear pattern. Under heavy load a request that results in
> > a
> > > > > HTTP 400 error locks Rocket.
>
> > > > > Notice that the logging is done by a wsgi application that calls
> > > > > web2py wsgibase, i.e it time how long it takes web2py to receive the
> > > > > request and send the response. The extra time must be spent inside
> > the
> > > > > web server.
>
> > > > > It is also important that the times showed in the logs are the actual
> > > > > time when the data is being written in the logs. You can see firefox
> > > > > waiting for base.css, the server waiting to log base.css and nothing
> > > > > else is being printed during the wait, signifying that web2py is not
> > > > > running any request.
>
> > > > > We need Tim! This is a problem.
>
> > > > > Massimo
>
> > > > > On Jul 22, 9:22 am, Michael Ellis  wrote:
>
> > > > > > I've isolated the problem but absolutely do not understand it.  I
> > can
> > > > > > reproduce it with a two-line change to web2py_ajax.html.   Will
> > someone
> > > > with
> > > > > > the time and equipment please attempt to replicate  this as a
> > sanity
> > > > check?
>
> > > > > > Here's how:
>
> > > > > > In the welcome app's web2py_ajax.html, insert the following after
> > line
> > > > 3.
>
> > response.files.insert(3,URL(r=request,c='static',f='jquery.sparkline.js'))
>
> > response.files.insert(4,URL(r=request,c='static',f='jquery.timers-1.2.js'))
>
> > > > > > Copy the attached js files into welcome/static.  They should be the
> > > > same as
> > > > > > the versions available online.
>
> > > > > > To reproduce the problem, serve web2py on your LAN.  Open the
> > welcome
> > > > home
> > > > > > page on two different machines.  One of them can be on the server.
> > > >  Briskly
> > > > > > reload the page 10 or more times on either machine then try to
> > reload
> > > > on the
> > > > > > other.  In my setup, the delay is reliably 25 seconds from the time
> > I
> > > > make
> > > > > > the last click on the first machine.
>
> > > > > > I'm able to reproduce this in FF, Chrome, and Safari using the
> > latest
> > > > web2py
> > > > > > from trunk.  Haven't tried any other browsers yet.  As noted
> > previously
> > > > both
> > > > > > machines are MacBooks running Snow Leopard.
>
> > > > > > Mike
>
> > > > > >  jquery.timers-1.2.js
> > > > > > 4KViewDownload
>
> > > > > >  j

Re: [web2py] Re: rocket performance issues

[Michele Comitini]

Because of the GIL threads on CPython are problematic more than one could
think.  Googling "GIL python" gives an idea...
Not to start the war threading vs forking model in general, but the GIL will
not go away from van Rossum's
python implementation (no not even in 3.x), he rather suggests to stay out
of threading (and he means that in general).

This is better clarified in this Guido's mail:

http://mail.python.org/pipermail/python-3000/2007-May/007414.html


ciao,
mic

2010/7/22 Timbo 

> OK, so it's a thread-starvation issue.  Let this be the interim fix
> for web2py.  I've got several busy days ahead of me so I'll put out a
> real fix for Rocket in the coming days.
>
> -tim
>
> On Jul 22, 11:39 am, Michael Ellis  wrote:
> > Much improved with suggested change.  Latencies for .js/.css files
> reduced
> > to 100-150 msec.  Thanks Tim!
> >
> > Cheers,
> > Mike
> >
> >
> >
> > On Thu, Jul 22, 2010 at 12:27 PM, Timbo 
> wrote:
> > > Try one quick change for me please...rocket is constructed around line
> > > 655 in main.py
> >
> > > Add a parameter to the constructor call(s):  max_threads=0
> >
> > > Please let me know if that affects the problem.
> >
> > > -tim
> >
> > > On Jul 22, 10:34 am, mdipierro  wrote:
> > > > I can reproduce the problem. I did on localhost with two different
> > > > browsers.
> > > > Using firebug I can see it takes 25seconds to download base.css (the
> > > > problem is not always with the same file).
> > > > While I did the test, I also monitored httpserver.log and I find that
> > > > it NEVER takes more than 1.2ms serve base.css.
> > > > This is what the log shows:
> >
> > > > 
> > > > 127.0.0.1, 2010-07-22 10:16:38, GET, /michealellistest/static/images/
> > > > header.png, HTTP/1.1, 304, 0.000563
> > > > 127.0.0.1, 2010-07-22 10:16:38, GET, /favicon.ico, HTTP/1.1, 400,
> > > > 0.000631
> > > > 127.0.0.1, 2010-07-22 10:16:55, GET, /michealellistest/static/
> > > > base.css, HTTP/1.1, 304, 0.000791    locks firefox for 25secs
> > > > 
> > > > 127.0.0.1, 2010-07-22 10:22:42, GET, /michealellistest/static/
> > > > jquery.timers-1.2.js, HTTP/1.1, 304, 0.000552
> > > > 127.0.0.1, 2010-07-22 10:22:42, GET, /favicon.ico, HTTP/1.1, 400,
> > > > 0.000497
> > > > 127.0.0.1, 2010-07-22 10:23:02, GET, /michealellistest/static/
> > > > superfish.js, HTTP/1.1, 304, 0.000914    locks chrome for 25secs
> >
> > > > Do you see the time gaps?
> >
> > > > There is a clear pattern. Under heavy load a request that results in
> a
> > > > HTTP 400 error locks Rocket.
> >
> > > > Notice that the logging is done by a wsgi application that calls
> > > > web2py wsgibase, i.e it time how long it takes web2py to receive the
> > > > request and send the response. The extra time must be spent inside
> the
> > > > web server.
> >
> > > > It is also important that the times showed in the logs are the actual
> > > > time when the data is being written in the logs. You can see firefox
> > > > waiting for base.css, the server waiting to log base.css and nothing
> > > > else is being printed during the wait, signifying that web2py is not
> > > > running any request.
> >
> > > > We need Tim! This is a problem.
> >
> > > > Massimo
> >
> > > > On Jul 22, 9:22 am, Michael Ellis  wrote:
> >
> > > > > I've isolated the problem but absolutely do not understand it.  I
> can
> > > > > reproduce it with a two-line change to web2py_ajax.html.   Will
> someone
> > > with
> > > > > the time and equipment please attempt to replicate  this as a
> sanity
> > > check?
> >
> > > > > Here's how:
> >
> > > > > In the welcome app's web2py_ajax.html, insert the following after
> line
> > > 3.
> >
> > >
> response.files.insert(3,URL(r=request,c='static',f='jquery.sparkline.js'))
> >
> > >
> response.files.insert(4,URL(r=request,c='static',f='jquery.timers-1.2.js'))
> >
> > > > > Copy the attached js files into welcome/static.  They should be the
> > > same as
> > > > > the versions available online.
> >
> > > > > To reproduce the problem, serve web2py on your LAN.  Open the
> welcome
> > > home
> > > > > page on two different machines.  One of them can be on the server.
> > >  Briskly
> > > > > reload the page 10 or more times on either machine then try to
> reload
> > > on the
> > > > > other.  In my setup, the delay is reliably 25 seconds from the time
> I
> > > make
> > > > > the last click on the first machine.
> >
> > > > > I'm able to reproduce this in FF, Chrome, and Safari using the
> latest
> > > web2py
> > > > > from trunk.  Haven't tried any other browsers yet.  As noted
> previously
> > > both
> > > > > machines are MacBooks running Snow Leopard.
> >
> > > > > Mike
> >
> > > > >  jquery.timers-1.2.js
> > > > > 4KViewDownload
> >
> > > > >  jquery.sparkline.js
> > > > > 62KViewDownload
>

[web2py] Re: rocket performance issues

[mdipierro]

Good idea. In trunk.

On Jul 23, 6:47 am, Iceberg  wrote:
> Nice to have partial progress in 1.81.5 anyway. Congratulations. Even
> so, how comes an HTTP 400 error can lock Rocket? �...@_@
>
> And if that is somehow really the cause, we need to prevent 400 error
> from /favicon.ico  It is simple, add this lines into welcome
> scaffold's layout.html
>    href="{{=URL(request.application,'static','favicon.ico')}}"
> type="image/vnd.microsoft.icon">
>
> and supply a default welcome/static/favicon.ico also.
>
> On Jul 22, 11:34pm, mdipierro  wrote:
>
> > I can reproduce the problem. I did on localhost with two different
> > browsers.
> > Using firebug I can see it takes 25seconds to download base.css (the
> > problem is not always with the same file).
> > While I did the test, I also monitored httpserver.log and I find that
> > it NEVER takes more than 1.2ms serve base.css.
> > This is what the log shows:
>
> > 
> > 127.0.0.1, 2010-07-22 10:16:38, GET, /michealellistest/static/images/
> > header.png, HTTP/1.1, 304, 0.000563
> > 127.0.0.1, 2010-07-22 10:16:38, GET, /favicon.ico, HTTP/1.1, 400,
> > 0.000631
> > 127.0.0.1, 2010-07-22 10:16:55, GET, /michealellistest/static/
> > base.css, HTTP/1.1, 304, 0.000791    locks firefox for 25secs
> > 
> > 127.0.0.1, 2010-07-22 10:22:42, GET, /michealellistest/static/
> > jquery.timers-1.2.js, HTTP/1.1, 304, 0.000552
> > 127.0.0.1, 2010-07-22 10:22:42, GET, /favicon.ico, HTTP/1.1, 400,
> > 0.000497
> > 127.0.0.1, 2010-07-22 10:23:02, GET, /michealellistest/static/
> > superfish.js, HTTP/1.1, 304, 0.000914    locks chrome for 25secs
>
> > Do you see the time gaps?
>
> > There is a clear pattern. Under heavy load a request that results in a
> > HTTP 400 error locks Rocket.
>
> > Notice that the logging is done by a wsgi application that calls
> > web2py wsgibase, i.e it time how long it takes web2py to receive the
> > request and send the response. The extra time must be spent inside the
> > web server.
>
> > It is also important that the times showed in the logs are the actual
> > time when the data is being written in the logs. You can see firefox
> > waiting for base.css, the server waiting to log base.css and nothing
> > else is being printed during the wait, signifying that web2py is not
> > running any request.
>
> > We need Tim! This is a problem.
>
> > Massimo
>
> > On Jul 22, 9:22 am, Michael Ellis  wrote:
>
> > > I've isolated the problem but absolutely do not understand it.  I can
> > > reproduce it with a two-line change to web2py_ajax.html.   Will someone 
> > > with
> > > the time and equipment please attempt to replicate  this as a sanity 
> > > check?
>
> > > Here's how:
>
> > > In the welcome app's web2py_ajax.html, insert the following after line 3.
> > > response.files.insert(3,URL(r=request,c='static',f='jquery.sparkline.js'))
> > > response.files.insert(4,URL(r=request,c='static',f='jquery.timers-1.2.js'))
>
> > > Copy the attached js files into welcome/static.  They should be the same 
> > > as
> > > the versions available online.
>
> > > To reproduce the problem, serve web2py on your LAN.  Open the welcome home
> > > page on two different machines.  One of them can be on the server.  
> > > Briskly
> > > reload the page 10 or more times on either machine then try to reload on 
> > > the
> > > other.  In my setup, the delay is reliably 25 seconds from the time I make
> > > the last click on the first machine.
>
> > > I'm able to reproduce this in FF, Chrome, and Safari using the latest 
> > > web2py
> > > from trunk.  Haven't tried any other browsers yet.  As noted previously 
> > > both
> > > machines are MacBooks running Snow Leopard.
>
> > > Mike
>
> > >  jquery.timers-1.2.js
> > > 4KViewDownload
>
> > >  jquery.sparkline.js
> > > 62KViewDownload

[web2py] Re: rocket performance issues

[Iceberg]

Nice to have partial progress in 1.81.5 anyway. Congratulations. Even
so, how comes an HTTP 400 error can lock Rocket?  @_@

And if that is somehow really the cause, we need to prevent 400 error
from /favicon.ico  It is simple, add this lines into welcome
scaffold's layout.html
  

and supply a default welcome/static/favicon.ico also.


On Jul 22, 11:34pm, mdipierro  wrote:
> I can reproduce the problem. I did on localhost with two different
> browsers.
> Using firebug I can see it takes 25seconds to download base.css (the
> problem is not always with the same file).
> While I did the test, I also monitored httpserver.log and I find that
> it NEVER takes more than 1.2ms serve base.css.
> This is what the log shows:
>
> 
> 127.0.0.1, 2010-07-22 10:16:38, GET, /michealellistest/static/images/
> header.png, HTTP/1.1, 304, 0.000563
> 127.0.0.1, 2010-07-22 10:16:38, GET, /favicon.ico, HTTP/1.1, 400,
> 0.000631
> 127.0.0.1, 2010-07-22 10:16:55, GET, /michealellistest/static/
> base.css, HTTP/1.1, 304, 0.000791    locks firefox for 25secs
> 
> 127.0.0.1, 2010-07-22 10:22:42, GET, /michealellistest/static/
> jquery.timers-1.2.js, HTTP/1.1, 304, 0.000552
> 127.0.0.1, 2010-07-22 10:22:42, GET, /favicon.ico, HTTP/1.1, 400,
> 0.000497
> 127.0.0.1, 2010-07-22 10:23:02, GET, /michealellistest/static/
> superfish.js, HTTP/1.1, 304, 0.000914    locks chrome for 25secs
>
> Do you see the time gaps?
>
> There is a clear pattern. Under heavy load a request that results in a
> HTTP 400 error locks Rocket.
>
> Notice that the logging is done by a wsgi application that calls
> web2py wsgibase, i.e it time how long it takes web2py to receive the
> request and send the response. The extra time must be spent inside the
> web server.
>
> It is also important that the times showed in the logs are the actual
> time when the data is being written in the logs. You can see firefox
> waiting for base.css, the server waiting to log base.css and nothing
> else is being printed during the wait, signifying that web2py is not
> running any request.
>
> We need Tim! This is a problem.
>
> Massimo
>
> On Jul 22, 9:22 am, Michael Ellis  wrote:
>
>
>
> > I've isolated the problem but absolutely do not understand it.  I can
> > reproduce it with a two-line change to web2py_ajax.html.   Will someone with
> > the time and equipment please attempt to replicate  this as a sanity check?
>
> > Here's how:
>
> > In the welcome app's web2py_ajax.html, insert the following after line 3.
> > response.files.insert(3,URL(r=request,c='static',f='jquery.sparkline.js'))
> > response.files.insert(4,URL(r=request,c='static',f='jquery.timers-1.2.js'))
>
> > Copy the attached js files into welcome/static.  They should be the same as
> > the versions available online.
>
> > To reproduce the problem, serve web2py on your LAN.  Open the welcome home
> > page on two different machines.  One of them can be on the server.  Briskly
> > reload the page 10 or more times on either machine then try to reload on the
> > other.  In my setup, the delay is reliably 25 seconds from the time I make
> > the last click on the first machine.
>
> > I'm able to reproduce this in FF, Chrome, and Safari using the latest web2py
> > from trunk.  Haven't tried any other browsers yet.  As noted previously both
> > machines are MacBooks running Snow Leopard.
>
> > Mike
>
> >  jquery.timers-1.2.js
> > 4KViewDownload
>
> >  jquery.sparkline.js
> > 62KViewDownload

[web2py] Re: rocket performance issues

[mdipierro]

This is now in 1.81.5

On Jul 22, 11:18 pm, Phyo Arkar  wrote:
> Thats very good news. I will be trying out soon!
>
> On 7/22/10, Timbo  wrote:
>
> > OK, so it's a thread-starvation issue.  Let this be the interim fix
> > for web2py.  I've got several busy days ahead of me so I'll put out a
> > real fix for Rocket in the coming days.
>
> > -tim
>
> > On Jul 22, 11:39 am, Michael Ellis  wrote:
> >> Much improved with suggested change.  Latencies for .js/.css files reduced
> >> to 100-150 msec.  Thanks Tim!
>
> >> Cheers,
> >> Mike
>
> >> On Thu, Jul 22, 2010 at 12:27 PM, Timbo  wrote:
> >> > Try one quick change for me please...rocket is constructed around line
> >> > 655 in main.py
>
> >> > Add a parameter to the constructor call(s):  max_threads=0
>
> >> > Please let me know if that affects the problem.
>
> >> > -tim
>
> >> > On Jul 22, 10:34 am, mdipierro  wrote:
> >> > > I can reproduce the problem. I did on localhost with two different
> >> > > browsers.
> >> > > Using firebug I can see it takes 25seconds to download base.css (the
> >> > > problem is not always with the same file).
> >> > > While I did the test, I also monitored httpserver.log and I find that
> >> > > it NEVER takes more than 1.2ms serve base.css.
> >> > > This is what the log shows:
>
> >> > > 
> >> > > 127.0.0.1, 2010-07-22 10:16:38, GET, /michealellistest/static/images/
> >> > > header.png, HTTP/1.1, 304, 0.000563
> >> > > 127.0.0.1, 2010-07-22 10:16:38, GET, /favicon.ico, HTTP/1.1, 400,
> >> > > 0.000631
> >> > > 127.0.0.1, 2010-07-22 10:16:55, GET, /michealellistest/static/
> >> > > base.css, HTTP/1.1, 304, 0.000791    locks firefox for 25secs
> >> > > 
> >> > > 127.0.0.1, 2010-07-22 10:22:42, GET, /michealellistest/static/
> >> > > jquery.timers-1.2.js, HTTP/1.1, 304, 0.000552
> >> > > 127.0.0.1, 2010-07-22 10:22:42, GET, /favicon.ico, HTTP/1.1, 400,
> >> > > 0.000497
> >> > > 127.0.0.1, 2010-07-22 10:23:02, GET, /michealellistest/static/
> >> > > superfish.js, HTTP/1.1, 304, 0.000914    locks chrome for 25secs
>
> >> > > Do you see the time gaps?
>
> >> > > There is a clear pattern. Under heavy load a request that results in a
> >> > > HTTP 400 error locks Rocket.
>
> >> > > Notice that the logging is done by a wsgi application that calls
> >> > > web2py wsgibase, i.e it time how long it takes web2py to receive the
> >> > > request and send the response. The extra time must be spent inside the
> >> > > web server.
>
> >> > > It is also important that the times showed in the logs are the actual
> >> > > time when the data is being written in the logs. You can see firefox
> >> > > waiting for base.css, the server waiting to log base.css and nothing
> >> > > else is being printed during the wait, signifying that web2py is not
> >> > > running any request.
>
> >> > > We need Tim! This is a problem.
>
> >> > > Massimo
>
> >> > > On Jul 22, 9:22 am, Michael Ellis  wrote:
>
> >> > > > I've isolated the problem but absolutely do not understand it.  I
> >> > > > can
> >> > > > reproduce it with a two-line change to web2py_ajax.html.   Will
> >> > > > someone
> >> > with
> >> > > > the time and equipment please attempt to replicate  this as a sanity
> >> > check?
>
> >> > > > Here's how:
>
> >> > > > In the welcome app's web2py_ajax.html, insert the following after
> >> > > > line
> >> > 3.
>
> >> > response.files.insert(3,URL(r=request,c='static',f='jquery.sparkline.js'))
>
> >> > response.files.insert(4,URL(r=request,c='static',f='jquery.timers-1.2.js'))
>
> >> > > > Copy the attached js files into welcome/static.  They should be the
> >> > same as
> >> > > > the versions available online.
>
> >> > > > To reproduce the problem, serve web2py on your LAN.  Open the
> >> > > > welcome
> >> > home
> >> > > > page on two different machines.  One of them can be on the server.
> >> >  Briskly
> >> > > > reload the page 10 or more times on either machine then try to
> >> > > > reload
> >> > on the
> >> > > > other.  In my setup, the delay is reliably 25 seconds from the time
> >> > > > I
> >> > make
> >> > > > the last click on the first machine.
>
> >> > > > I'm able to reproduce this in FF, Chrome, and Safari using the
> >> > > > latest
> >> > web2py
> >> > > > from trunk.  Haven't tried any other browsers yet.  As noted
> >> > > > previously
> >> > both
> >> > > > machines are MacBooks running Snow Leopard.
>
> >> > > > Mike
>
> >> > > >  jquery.timers-1.2.js
> >> > > > 4KViewDownload
>
> >> > > >  jquery.sparkline.js
> >> > > > 62KViewDownload

Re: [web2py] Re: rocket performance issues

[Phyo Arkar]

Thats very good news. I will be trying out soon!

On 7/22/10, Timbo  wrote:
> OK, so it's a thread-starvation issue.  Let this be the interim fix
> for web2py.  I've got several busy days ahead of me so I'll put out a
> real fix for Rocket in the coming days.
>
> -tim
>
> On Jul 22, 11:39 am, Michael Ellis  wrote:
>> Much improved with suggested change.  Latencies for .js/.css files reduced
>> to 100-150 msec.  Thanks Tim!
>>
>> Cheers,
>> Mike
>>
>>
>>
>> On Thu, Jul 22, 2010 at 12:27 PM, Timbo  wrote:
>> > Try one quick change for me please...rocket is constructed around line
>> > 655 in main.py
>>
>> > Add a parameter to the constructor call(s):  max_threads=0
>>
>> > Please let me know if that affects the problem.
>>
>> > -tim
>>
>> > On Jul 22, 10:34 am, mdipierro  wrote:
>> > > I can reproduce the problem. I did on localhost with two different
>> > > browsers.
>> > > Using firebug I can see it takes 25seconds to download base.css (the
>> > > problem is not always with the same file).
>> > > While I did the test, I also monitored httpserver.log and I find that
>> > > it NEVER takes more than 1.2ms serve base.css.
>> > > This is what the log shows:
>>
>> > > 
>> > > 127.0.0.1, 2010-07-22 10:16:38, GET, /michealellistest/static/images/
>> > > header.png, HTTP/1.1, 304, 0.000563
>> > > 127.0.0.1, 2010-07-22 10:16:38, GET, /favicon.ico, HTTP/1.1, 400,
>> > > 0.000631
>> > > 127.0.0.1, 2010-07-22 10:16:55, GET, /michealellistest/static/
>> > > base.css, HTTP/1.1, 304, 0.000791    locks firefox for 25secs
>> > > 
>> > > 127.0.0.1, 2010-07-22 10:22:42, GET, /michealellistest/static/
>> > > jquery.timers-1.2.js, HTTP/1.1, 304, 0.000552
>> > > 127.0.0.1, 2010-07-22 10:22:42, GET, /favicon.ico, HTTP/1.1, 400,
>> > > 0.000497
>> > > 127.0.0.1, 2010-07-22 10:23:02, GET, /michealellistest/static/
>> > > superfish.js, HTTP/1.1, 304, 0.000914    locks chrome for 25secs
>>
>> > > Do you see the time gaps?
>>
>> > > There is a clear pattern. Under heavy load a request that results in a
>> > > HTTP 400 error locks Rocket.
>>
>> > > Notice that the logging is done by a wsgi application that calls
>> > > web2py wsgibase, i.e it time how long it takes web2py to receive the
>> > > request and send the response. The extra time must be spent inside the
>> > > web server.
>>
>> > > It is also important that the times showed in the logs are the actual
>> > > time when the data is being written in the logs. You can see firefox
>> > > waiting for base.css, the server waiting to log base.css and nothing
>> > > else is being printed during the wait, signifying that web2py is not
>> > > running any request.
>>
>> > > We need Tim! This is a problem.
>>
>> > > Massimo
>>
>> > > On Jul 22, 9:22 am, Michael Ellis  wrote:
>>
>> > > > I've isolated the problem but absolutely do not understand it.  I
>> > > > can
>> > > > reproduce it with a two-line change to web2py_ajax.html.   Will
>> > > > someone
>> > with
>> > > > the time and equipment please attempt to replicate  this as a sanity
>> > check?
>>
>> > > > Here's how:
>>
>> > > > In the welcome app's web2py_ajax.html, insert the following after
>> > > > line
>> > 3.
>>
>> > response.files.insert(3,URL(r=request,c='static',f='jquery.sparkline.js'))
>>
>> > response.files.insert(4,URL(r=request,c='static',f='jquery.timers-1.2.js'))
>>
>> > > > Copy the attached js files into welcome/static.  They should be the
>> > same as
>> > > > the versions available online.
>>
>> > > > To reproduce the problem, serve web2py on your LAN.  Open the
>> > > > welcome
>> > home
>> > > > page on two different machines.  One of them can be on the server.
>> >  Briskly
>> > > > reload the page 10 or more times on either machine then try to
>> > > > reload
>> > on the
>> > > > other.  In my setup, the delay is reliably 25 seconds from the time
>> > > > I
>> > make
>> > > > the last click on the first machine.
>>
>> > > > I'm able to reproduce this in FF, Chrome, and Safari using the
>> > > > latest
>> > web2py
>> > > > from trunk.  Haven't tried any other browsers yet.  As noted
>> > > > previously
>> > both
>> > > > machines are MacBooks running Snow Leopard.
>>
>> > > > Mike
>>
>> > > >  jquery.timers-1.2.js
>> > > > 4KViewDownload
>>
>> > > >  jquery.sparkline.js
>> > > > 62KViewDownload

[web2py] Re: rocket performance issues

[Timbo]

OK, so it's a thread-starvation issue.  Let this be the interim fix
for web2py.  I've got several busy days ahead of me so I'll put out a
real fix for Rocket in the coming days.

-tim

On Jul 22, 11:39 am, Michael Ellis  wrote:
> Much improved with suggested change.  Latencies for .js/.css files reduced
> to 100-150 msec.  Thanks Tim!
>
> Cheers,
> Mike
>
>
>
> On Thu, Jul 22, 2010 at 12:27 PM, Timbo  wrote:
> > Try one quick change for me please...rocket is constructed around line
> > 655 in main.py
>
> > Add a parameter to the constructor call(s):  max_threads=0
>
> > Please let me know if that affects the problem.
>
> > -tim
>
> > On Jul 22, 10:34 am, mdipierro  wrote:
> > > I can reproduce the problem. I did on localhost with two different
> > > browsers.
> > > Using firebug I can see it takes 25seconds to download base.css (the
> > > problem is not always with the same file).
> > > While I did the test, I also monitored httpserver.log and I find that
> > > it NEVER takes more than 1.2ms serve base.css.
> > > This is what the log shows:
>
> > > 
> > > 127.0.0.1, 2010-07-22 10:16:38, GET, /michealellistest/static/images/
> > > header.png, HTTP/1.1, 304, 0.000563
> > > 127.0.0.1, 2010-07-22 10:16:38, GET, /favicon.ico, HTTP/1.1, 400,
> > > 0.000631
> > > 127.0.0.1, 2010-07-22 10:16:55, GET, /michealellistest/static/
> > > base.css, HTTP/1.1, 304, 0.000791    locks firefox for 25secs
> > > 
> > > 127.0.0.1, 2010-07-22 10:22:42, GET, /michealellistest/static/
> > > jquery.timers-1.2.js, HTTP/1.1, 304, 0.000552
> > > 127.0.0.1, 2010-07-22 10:22:42, GET, /favicon.ico, HTTP/1.1, 400,
> > > 0.000497
> > > 127.0.0.1, 2010-07-22 10:23:02, GET, /michealellistest/static/
> > > superfish.js, HTTP/1.1, 304, 0.000914    locks chrome for 25secs
>
> > > Do you see the time gaps?
>
> > > There is a clear pattern. Under heavy load a request that results in a
> > > HTTP 400 error locks Rocket.
>
> > > Notice that the logging is done by a wsgi application that calls
> > > web2py wsgibase, i.e it time how long it takes web2py to receive the
> > > request and send the response. The extra time must be spent inside the
> > > web server.
>
> > > It is also important that the times showed in the logs are the actual
> > > time when the data is being written in the logs. You can see firefox
> > > waiting for base.css, the server waiting to log base.css and nothing
> > > else is being printed during the wait, signifying that web2py is not
> > > running any request.
>
> > > We need Tim! This is a problem.
>
> > > Massimo
>
> > > On Jul 22, 9:22 am, Michael Ellis  wrote:
>
> > > > I've isolated the problem but absolutely do not understand it.  I can
> > > > reproduce it with a two-line change to web2py_ajax.html.   Will someone
> > with
> > > > the time and equipment please attempt to replicate  this as a sanity
> > check?
>
> > > > Here's how:
>
> > > > In the welcome app's web2py_ajax.html, insert the following after line
> > 3.
>
> > response.files.insert(3,URL(r=request,c='static',f='jquery.sparkline.js'))
>
> > response.files.insert(4,URL(r=request,c='static',f='jquery.timers-1.2.js'))
>
> > > > Copy the attached js files into welcome/static.  They should be the
> > same as
> > > > the versions available online.
>
> > > > To reproduce the problem, serve web2py on your LAN.  Open the welcome
> > home
> > > > page on two different machines.  One of them can be on the server.
> >  Briskly
> > > > reload the page 10 or more times on either machine then try to reload
> > on the
> > > > other.  In my setup, the delay is reliably 25 seconds from the time I
> > make
> > > > the last click on the first machine.
>
> > > > I'm able to reproduce this in FF, Chrome, and Safari using the latest
> > web2py
> > > > from trunk.  Haven't tried any other browsers yet.  As noted previously
> > both
> > > > machines are MacBooks running Snow Leopard.
>
> > > > Mike
>
> > > >  jquery.timers-1.2.js
> > > > 4KViewDownload
>
> > > >  jquery.sparkline.js
> > > > 62KViewDownload

Re: [web2py] Re: rocket performance issues

[Michael Ellis]

Much improved with suggested change.  Latencies for .js/.css files reduced
to 100-150 msec.  Thanks Tim!

Cheers,
Mike


On Thu, Jul 22, 2010 at 12:27 PM, Timbo  wrote:

> Try one quick change for me please...rocket is constructed around line
> 655 in main.py
>
> Add a parameter to the constructor call(s):  max_threads=0
>
> Please let me know if that affects the problem.
>
> -tim
>
> On Jul 22, 10:34 am, mdipierro  wrote:
> > I can reproduce the problem. I did on localhost with two different
> > browsers.
> > Using firebug I can see it takes 25seconds to download base.css (the
> > problem is not always with the same file).
> > While I did the test, I also monitored httpserver.log and I find that
> > it NEVER takes more than 1.2ms serve base.css.
> > This is what the log shows:
> >
> > 
> > 127.0.0.1, 2010-07-22 10:16:38, GET, /michealellistest/static/images/
> > header.png, HTTP/1.1, 304, 0.000563
> > 127.0.0.1, 2010-07-22 10:16:38, GET, /favicon.ico, HTTP/1.1, 400,
> > 0.000631
> > 127.0.0.1, 2010-07-22 10:16:55, GET, /michealellistest/static/
> > base.css, HTTP/1.1, 304, 0.000791    locks firefox for 25secs
> > 
> > 127.0.0.1, 2010-07-22 10:22:42, GET, /michealellistest/static/
> > jquery.timers-1.2.js, HTTP/1.1, 304, 0.000552
> > 127.0.0.1, 2010-07-22 10:22:42, GET, /favicon.ico, HTTP/1.1, 400,
> > 0.000497
> > 127.0.0.1, 2010-07-22 10:23:02, GET, /michealellistest/static/
> > superfish.js, HTTP/1.1, 304, 0.000914    locks chrome for 25secs
> >
> > Do you see the time gaps?
> >
> > There is a clear pattern. Under heavy load a request that results in a
> > HTTP 400 error locks Rocket.
> >
> > Notice that the logging is done by a wsgi application that calls
> > web2py wsgibase, i.e it time how long it takes web2py to receive the
> > request and send the response. The extra time must be spent inside the
> > web server.
> >
> > It is also important that the times showed in the logs are the actual
> > time when the data is being written in the logs. You can see firefox
> > waiting for base.css, the server waiting to log base.css and nothing
> > else is being printed during the wait, signifying that web2py is not
> > running any request.
> >
> > We need Tim! This is a problem.
> >
> > Massimo
> >
> > On Jul 22, 9:22 am, Michael Ellis  wrote:
> >
> >
> >
> > > I've isolated the problem but absolutely do not understand it.  I can
> > > reproduce it with a two-line change to web2py_ajax.html.   Will someone
> with
> > > the time and equipment please attempt to replicate  this as a sanity
> check?
> >
> > > Here's how:
> >
> > > In the welcome app's web2py_ajax.html, insert the following after line
> 3.
> > >
> response.files.insert(3,URL(r=request,c='static',f='jquery.sparkline.js'))
> > >
> response.files.insert(4,URL(r=request,c='static',f='jquery.timers-1.2.js'))
> >
> > > Copy the attached js files into welcome/static.  They should be the
> same as
> > > the versions available online.
> >
> > > To reproduce the problem, serve web2py on your LAN.  Open the welcome
> home
> > > page on two different machines.  One of them can be on the server.
>  Briskly
> > > reload the page 10 or more times on either machine then try to reload
> on the
> > > other.  In my setup, the delay is reliably 25 seconds from the time I
> make
> > > the last click on the first machine.
> >
> > > I'm able to reproduce this in FF, Chrome, and Safari using the latest
> web2py
> > > from trunk.  Haven't tried any other browsers yet.  As noted previously
> both
> > > machines are MacBooks running Snow Leopard.
> >
> > > Mike
> >
> > >  jquery.timers-1.2.js
> > > 4KViewDownload
> >
> > >  jquery.sparkline.js
> > > 62KViewDownload
>

[web2py] Re: rocket performance issues

[Timbo]

Try one quick change for me please...rocket is constructed around line
655 in main.py

Add a parameter to the constructor call(s):  max_threads=0

Please let me know if that affects the problem.

-tim

On Jul 22, 10:34 am, mdipierro  wrote:
> I can reproduce the problem. I did on localhost with two different
> browsers.
> Using firebug I can see it takes 25seconds to download base.css (the
> problem is not always with the same file).
> While I did the test, I also monitored httpserver.log and I find that
> it NEVER takes more than 1.2ms serve base.css.
> This is what the log shows:
>
> 
> 127.0.0.1, 2010-07-22 10:16:38, GET, /michealellistest/static/images/
> header.png, HTTP/1.1, 304, 0.000563
> 127.0.0.1, 2010-07-22 10:16:38, GET, /favicon.ico, HTTP/1.1, 400,
> 0.000631
> 127.0.0.1, 2010-07-22 10:16:55, GET, /michealellistest/static/
> base.css, HTTP/1.1, 304, 0.000791    locks firefox for 25secs
> 
> 127.0.0.1, 2010-07-22 10:22:42, GET, /michealellistest/static/
> jquery.timers-1.2.js, HTTP/1.1, 304, 0.000552
> 127.0.0.1, 2010-07-22 10:22:42, GET, /favicon.ico, HTTP/1.1, 400,
> 0.000497
> 127.0.0.1, 2010-07-22 10:23:02, GET, /michealellistest/static/
> superfish.js, HTTP/1.1, 304, 0.000914    locks chrome for 25secs
>
> Do you see the time gaps?
>
> There is a clear pattern. Under heavy load a request that results in a
> HTTP 400 error locks Rocket.
>
> Notice that the logging is done by a wsgi application that calls
> web2py wsgibase, i.e it time how long it takes web2py to receive the
> request and send the response. The extra time must be spent inside the
> web server.
>
> It is also important that the times showed in the logs are the actual
> time when the data is being written in the logs. You can see firefox
> waiting for base.css, the server waiting to log base.css and nothing
> else is being printed during the wait, signifying that web2py is not
> running any request.
>
> We need Tim! This is a problem.
>
> Massimo
>
> On Jul 22, 9:22 am, Michael Ellis  wrote:
>
>
>
> > I've isolated the problem but absolutely do not understand it.  I can
> > reproduce it with a two-line change to web2py_ajax.html.   Will someone with
> > the time and equipment please attempt to replicate  this as a sanity check?
>
> > Here's how:
>
> > In the welcome app's web2py_ajax.html, insert the following after line 3.
> > response.files.insert(3,URL(r=request,c='static',f='jquery.sparkline.js'))
> > response.files.insert(4,URL(r=request,c='static',f='jquery.timers-1.2.js'))
>
> > Copy the attached js files into welcome/static.  They should be the same as
> > the versions available online.
>
> > To reproduce the problem, serve web2py on your LAN.  Open the welcome home
> > page on two different machines.  One of them can be on the server.  Briskly
> > reload the page 10 or more times on either machine then try to reload on the
> > other.  In my setup, the delay is reliably 25 seconds from the time I make
> > the last click on the first machine.
>
> > I'm able to reproduce this in FF, Chrome, and Safari using the latest web2py
> > from trunk.  Haven't tried any other browsers yet.  As noted previously both
> > machines are MacBooks running Snow Leopard.
>
> > Mike
>
> >  jquery.timers-1.2.js
> > 4KViewDownload
>
> >  jquery.sparkline.js
> > 62KViewDownload

[web2py] Re: rocket performance issues

[mdipierro]

I can reproduce the problem. I did on localhost with two different
browsers.
Using firebug I can see it takes 25seconds to download base.css (the
problem is not always with the same file).
While I did the test, I also monitored httpserver.log and I find that
it NEVER takes more than 1.2ms serve base.css.
This is what the log shows:


127.0.0.1, 2010-07-22 10:16:38, GET, /michealellistest/static/images/
header.png, HTTP/1.1, 304, 0.000563
127.0.0.1, 2010-07-22 10:16:38, GET, /favicon.ico, HTTP/1.1, 400,
0.000631
127.0.0.1, 2010-07-22 10:16:55, GET, /michealellistest/static/
base.css, HTTP/1.1, 304, 0.000791    locks firefox for 25secs

127.0.0.1, 2010-07-22 10:22:42, GET, /michealellistest/static/
jquery.timers-1.2.js, HTTP/1.1, 304, 0.000552
127.0.0.1, 2010-07-22 10:22:42, GET, /favicon.ico, HTTP/1.1, 400,
0.000497
127.0.0.1, 2010-07-22 10:23:02, GET, /michealellistest/static/
superfish.js, HTTP/1.1, 304, 0.000914    locks chrome for 25secs

Do you see the time gaps?

There is a clear pattern. Under heavy load a request that results in a
HTTP 400 error locks Rocket.

Notice that the logging is done by a wsgi application that calls
web2py wsgibase, i.e it time how long it takes web2py to receive the
request and send the response. The extra time must be spent inside the
web server.

It is also important that the times showed in the logs are the actual
time when the data is being written in the logs. You can see firefox
waiting for base.css, the server waiting to log base.css and nothing
else is being printed during the wait, signifying that web2py is not
running any request.

We need Tim! This is a problem.

Massimo



On Jul 22, 9:22 am, Michael Ellis  wrote:
> I've isolated the problem but absolutely do not understand it.  I can
> reproduce it with a two-line change to web2py_ajax.html.   Will someone with
> the time and equipment please attempt to replicate  this as a sanity check?
>
> Here's how:
>
> In the welcome app's web2py_ajax.html, insert the following after line 3.
> response.files.insert(3,URL(r=request,c='static',f='jquery.sparkline.js'))
> response.files.insert(4,URL(r=request,c='static',f='jquery.timers-1.2.js'))
>
> Copy the attached js files into welcome/static.  They should be the same as
> the versions available online.
>
> To reproduce the problem, serve web2py on your LAN.  Open the welcome home
> page on two different machines.  One of them can be on the server.  Briskly
> reload the page 10 or more times on either machine then try to reload on the
> other.  In my setup, the delay is reliably 25 seconds from the time I make
> the last click on the first machine.
>
> I'm able to reproduce this in FF, Chrome, and Safari using the latest web2py
> from trunk.  Haven't tried any other browsers yet.  As noted previously both
> machines are MacBooks running Snow Leopard.
>
> Mike
>
>  jquery.timers-1.2.js
> 4KViewDownload
>
>  jquery.sparkline.js
> 62KViewDownload

Re: [web2py] Re: rocket performance issues

[Jonathan Lundell]

On Jul 22, 2010, at 6:55 AM, MikeEllis wrote:

> Doesn't fix the symptom

Ah. Well, negative results are still results. 

> 
> On Jul 21, 8:54 pm, Jonathan Lundell  wrote:
>> On Jul 21, 2010, at 2:03 PM, mdipierro wrote:
>> 
>>> The fact that you are having the problem with static files is giving
>>> me an idea for a test.
>>> Can you please try the web2py I just posted in trunk?
>> 
>> That seems like a nice optimization, but (assuming it fixes the symptom) 
>> does it explain the 20-25-second stalls?
>> 
>> 
>> 
>> 
>> 
>>> Massimo
>> 
>>> On Jul 21, 2:08 pm, MikeEllis  wrote:
 One more detail.  As you would hope, it's generally not the case that
 both browsers stall at the same time.  While stalled on either laptop
 I can keep reloading on the other.
>> 
 On Jul 21, 3:03 pm, MikeEllis  wrote:
>> 
> I tried changing to cron=False in web2py.py.  No impact on problem.
> Same two-laptop setup as described previous.  Happens even on  pages
> that contain very little.  Chrome DevTools shows same problem as
> before:  One or more the .js/.css/.png files are delayed by 20 to 25
> seconds latency.
>> 
> The problem is also unaffected by whether I run web2py under winpdb or
> directly from the command line.   If I interrupt during the latency
> while running under winpdb,  I typically see
>> 
> * MainThread in rocket.py start() on line 301
> * Thread1 in threading.py wait() called by Queue.py get() called by
> rocket.py run() line 405
> * Remaining 10 threads all in socket.py readline() called by rocket.py
> read_request_line() line 825
>> 
> I'd send you the full stack traces but winpdb doesn't seem to support
> copying selections. (argggh!)
>> 
> Hope this is useful.
> Mike
>> 
> On Jul 21, 1:01 pm, Kuba Kucharski  wrote:
>> 
>> yes, but I am on the GPRS connection now, I am not able to test it so
>> the results be reliable, I will try to do that asap

[web2py] Re: rocket performance issues

[MikeEllis]

Doesn't fix the symptom

On Jul 21, 8:54 pm, Jonathan Lundell  wrote:
> On Jul 21, 2010, at 2:03 PM, mdipierro wrote:
>
> > The fact that you are having the problem with static files is giving
> > me an idea for a test.
> > Can you please try the web2py I just posted in trunk?
>
> That seems like a nice optimization, but (assuming it fixes the symptom) does 
> it explain the 20-25-second stalls?
>
>
>
>
>
> > Massimo
>
> > On Jul 21, 2:08 pm, MikeEllis  wrote:
> >> One more detail.  As you would hope, it's generally not the case that
> >> both browsers stall at the same time.  While stalled on either laptop
> >> I can keep reloading on the other.
>
> >> On Jul 21, 3:03 pm, MikeEllis  wrote:
>
> >>> I tried changing to cron=False in web2py.py.  No impact on problem.
> >>> Same two-laptop setup as described previous.  Happens even on  pages
> >>> that contain very little.  Chrome DevTools shows same problem as
> >>> before:  One or more the .js/.css/.png files are delayed by 20 to 25
> >>> seconds latency.
>
> >>> The problem is also unaffected by whether I run web2py under winpdb or
> >>> directly from the command line.   If I interrupt during the latency
> >>> while running under winpdb,  I typically see
>
> >>> * MainThread in rocket.py start() on line 301
> >>> * Thread1 in threading.py wait() called by Queue.py get() called by
> >>> rocket.py run() line 405
> >>> * Remaining 10 threads all in socket.py readline() called by rocket.py
> >>> read_request_line() line 825
>
> >>> I'd send you the full stack traces but winpdb doesn't seem to support
> >>> copying selections. (argggh!)
>
> >>> Hope this is useful.
> >>> Mike
>
> >>> On Jul 21, 1:01 pm, Kuba Kucharski  wrote:
>
>  yes, but I am on the GPRS connection now, I am not able to test it so
>  the results be reliable, I will try to do that asap

Re: [web2py] Re: rocket performance issues

[Jonathan Lundell]

On Jul 21, 2010, at 2:03 PM, mdipierro wrote:

> The fact that you are having the problem with static files is giving
> me an idea for a test.
> Can you please try the web2py I just posted in trunk?

That seems like a nice optimization, but (assuming it fixes the symptom) does 
it explain the 20-25-second stalls?

> 
> Massimo
> 
> On Jul 21, 2:08 pm, MikeEllis  wrote:
>> One more detail.  As you would hope, it's generally not the case that
>> both browsers stall at the same time.  While stalled on either laptop
>> I can keep reloading on the other.
>> 
>> On Jul 21, 3:03 pm, MikeEllis  wrote:
>> 
>>> I tried changing to cron=False in web2py.py.  No impact on problem.
>>> Same two-laptop setup as described previous.  Happens even on  pages
>>> that contain very little.  Chrome DevTools shows same problem as
>>> before:  One or more the .js/.css/.png files are delayed by 20 to 25
>>> seconds latency.
>> 
>>> The problem is also unaffected by whether I run web2py under winpdb or
>>> directly from the command line.   If I interrupt during the latency
>>> while running under winpdb,  I typically see
>> 
>>> * MainThread in rocket.py start() on line 301
>>> * Thread1 in threading.py wait() called by Queue.py get() called by
>>> rocket.py run() line 405
>>> * Remaining 10 threads all in socket.py readline() called by rocket.py
>>> read_request_line() line 825
>> 
>>> I'd send you the full stack traces but winpdb doesn't seem to support
>>> copying selections. (argggh!)
>> 
>>> Hope this is useful.
>>> Mike
>> 
>>> On Jul 21, 1:01 pm, Kuba Kucharski  wrote:
>> 
 yes, but I am on the GPRS connection now, I am not able to test it so
 the results be reliable, I will try to do that asap

[web2py] Re: rocket performance issues

[MikeEllis]

Alas, no change  with

Version 1.81.4 (2010-07-17 16:14:44)

One more little observation: In general, I can load a page from
either, but not both,  of the browsers repeatedly and not get the
problem. It happens with alternation, as discussed previously, but I
now notice that I seem to be able to cause it by reloading on either
laptop 10-15 times and then it will happen first time I click the
other.

Here's the simplest controller and view I've been able to reproduce it
with:

def index():
"""
example action using the internationalization operator T and flash
rendered by views/default/index.html or views/generic.html
"""
response.flash = T('Welcome to PeerTool')
return dict(message=T('Hello World'))




{{extend 'layout.html'}}

{{try:}}{{=H2(message)}}{{except:}}{{=BEAUTIFY(response._vars)}}
{{pass}}
PeerTool is an easy-to-use web application for collaborative problem
solving. To use it, you and your group need to:

Register and confirm your identity by email
Watch the short tutorial screencast below
Start collaborating!




The db.py is only a little more complicated.  Sets up logging, auth,
local_imports a module, and defines a couple of tables.

Hope this is useful,
Mike

On Jul 21, 5:03 pm, mdipierro  wrote:
> The fact that you are having the problem with static files is giving
> me an idea for a test.
> Can you please try the web2py I just posted in trunk?
>
> Massimo
>
> On Jul 21, 2:08 pm, MikeEllis  wrote:
>
>
>
> > One more detail.  As you would hope, it's generally not the case that
> > both browsers stall at the same time.  While stalled on either laptop
> > I can keep reloading on the other.
>
> > On Jul 21, 3:03 pm, MikeEllis  wrote:
>
> > > I tried changing to cron=False in web2py.py.  No impact on problem.
> > > Same two-laptop setup as described previous.  Happens even on  pages
> > > that contain very little.  Chrome DevTools shows same problem as
> > > before:  One or more the .js/.css/.png files are delayed by 20 to 25
> > > seconds latency.
>
> > > The problem is also unaffected by whether I run web2py under winpdb or
> > > directly from the command line.   If I interrupt during the latency
> > > while running under winpdb,  I typically see
>
> > > * MainThread in rocket.py start() on line 301
> > > * Thread1 in threading.py wait() called by Queue.py get() called by
> > > rocket.py run() line 405
> > > * Remaining 10 threads all in socket.py readline() called by rocket.py
> > > read_request_line() line 825
>
> > > I'd send you the full stack traces but winpdb doesn't seem to support
> > > copying selections. (argggh!)
>
> > > Hope this is useful.
> > > Mike
>
> > > On Jul 21, 1:01 pm, Kuba Kucharski  wrote:
>
> > > > yes, but I am on the GPRS connection now, I am not able to test it so
> > > > the results be reliable, I will try to do that asap

[web2py] Re: rocket performance issues

[mdipierro]

The fact that you are having the problem with static files is giving
me an idea for a test.
Can you please try the web2py I just posted in trunk?

Massimo

On Jul 21, 2:08 pm, MikeEllis  wrote:
> One more detail.  As you would hope, it's generally not the case that
> both browsers stall at the same time.  While stalled on either laptop
> I can keep reloading on the other.
>
> On Jul 21, 3:03 pm, MikeEllis  wrote:
>
> > I tried changing to cron=False in web2py.py.  No impact on problem.
> > Same two-laptop setup as described previous.  Happens even on  pages
> > that contain very little.  Chrome DevTools shows same problem as
> > before:  One or more the .js/.css/.png files are delayed by 20 to 25
> > seconds latency.
>
> > The problem is also unaffected by whether I run web2py under winpdb or
> > directly from the command line.   If I interrupt during the latency
> > while running under winpdb,  I typically see
>
> > * MainThread in rocket.py start() on line 301
> > * Thread1 in threading.py wait() called by Queue.py get() called by
> > rocket.py run() line 405
> > * Remaining 10 threads all in socket.py readline() called by rocket.py
> > read_request_line() line 825
>
> > I'd send you the full stack traces but winpdb doesn't seem to support
> > copying selections. (argggh!)
>
> > Hope this is useful.
> > Mike
>
> > On Jul 21, 1:01 pm, Kuba Kucharski  wrote:
>
> > > yes, but I am on the GPRS connection now, I am not able to test it so
> > > the results be reliable, I will try to do that asap

[web2py] Re: rocket performance issues

[MikeEllis]

One more detail.  As you would hope, it's generally not the case that
both browsers stall at the same time.  While stalled on either laptop
I can keep reloading on the other.

On Jul 21, 3:03 pm, MikeEllis  wrote:
> I tried changing to cron=False in web2py.py.  No impact on problem.
> Same two-laptop setup as described previous.  Happens even on  pages
> that contain very little.  Chrome DevTools shows same problem as
> before:  One or more the .js/.css/.png files are delayed by 20 to 25
> seconds latency.
>
> The problem is also unaffected by whether I run web2py under winpdb or
> directly from the command line.   If I interrupt during the latency
> while running under winpdb,  I typically see
>
> * MainThread in rocket.py start() on line 301
> * Thread1 in threading.py wait() called by Queue.py get() called by
> rocket.py run() line 405
> * Remaining 10 threads all in socket.py readline() called by rocket.py
> read_request_line() line 825
>
> I'd send you the full stack traces but winpdb doesn't seem to support
> copying selections. (argggh!)
>
> Hope this is useful.
> Mike
>
> On Jul 21, 1:01 pm, Kuba Kucharski  wrote:
>
>
>
> > yes, but I am on the GPRS connection now, I am not able to test it so
> > the results be reliable, I will try to do that asap

[web2py] Re: rocket performance issues

[MikeEllis]

I tried changing to cron=False in web2py.py.  No impact on problem.
Same two-laptop setup as described previous.  Happens even on  pages
that contain very little.  Chrome DevTools shows same problem as
before:  One or more the .js/.css/.png files are delayed by 20 to 25
seconds latency.

The problem is also unaffected by whether I run web2py under winpdb or
directly from the command line.   If I interrupt during the latency
while running under winpdb,  I typically see

* MainThread in rocket.py start() on line 301
* Thread1 in threading.py wait() called by Queue.py get() called by
rocket.py run() line 405
* Remaining 10 threads all in socket.py readline() called by rocket.py
read_request_line() line 825

I'd send you the full stack traces but winpdb doesn't seem to support
copying selections. (argggh!)

Hope this is useful.
Mike



On Jul 21, 1:01 pm, Kuba Kucharski  wrote:
> yes, but I am on the GPRS connection now, I am not able to test it so
> the results be reliable, I will try to do that asap

Re: [web2py] Re: rocket performance issues

[Kuba Kucharski]

yes, but I am on the GPRS connection now, I am not able to test it so
the results be reliable, I will try to do that asap

[web2py] Re: rocket performance issues

[mdipierro]

Kuba, please try set cron=False in web2py.py. Does it make things
better?

Massimo

On Jul 21, 10:33 am, Kuba Kucharski  wrote:
> "sometimes, once every few hits, page loads significantly slower
> weirdest thing is when you re-click the link it loads instantly, when
> you left it working to load on itself, it is slow.. like 4 to 8
> seconds"
>
> some people report more, 20 seconds, 1 minute.
>
> Tim, can you reproduce this? Have you tried MikeEllis instructions of
> reproducing?
>
> --
> Kuba

Re: [web2py] Re: rocket performance

[Timothy Farrell]

This is the first I'd heard of x-sendfile so I looked it up.  It seems 
to work to off-load work from the application server to the web-server 
for serving large files.


In theory, there would be no real benefit from this in Rocket since 
Rocket and web2py run in the same process.  Serving a file from disk is 
hardly a CPU-intensive process for a threaded system.  The only benefit 
would be to allow the GC to collect some of the request environment 
stuff earlier.  I can't see this having a measurable benefit.


For Apache and LightTPd, it makes a more sense when the app-server runs 
in a different process.


-tim

On 4/22/2010 11:11 AM, Thadeus Burgess wrote:

I know I have been talking about x-sendfile alot lately

Is there any reason that rocket should support it?

--
Thadeus





On Tue, Apr 20, 2010 at 4:06 PM, mdipierro  wrote:
   

ok. will fix this soon.

On Apr 20, 4:03 pm, Timothy Farrell  wrote:
 

Ton's of references in epydoc.  Those will be there as long as
wsgiserver.py is in the distro.  Otherwise, run epydoc and sphinx.

Other than that, theres:
\examples\views\default\index.html:line 35
- change link tohttp://launchpad.net/rocket

\examples\views\default\who.html:line 77
- remove the reference so long as wsgiserver is not the default
- add "Rocket Web Server developed by Timothy Farrell."

On 4/20/2010 9:21 AM, mdipierro wrote:

   

I made a mistake. Left references to cherry in some of the pages in
applications/example. Could you help me locate them and suggest
changes?
 
   

Massimo
 
   

On Apr 20, 8:11 am, Timothy Farrellwrote:
 
   

Thank you.  It's always good to see the fruits of one's labor come out.
   
   

On 4/19/2010 3:57 PM, elffikk wrote:
   
   

a simple test loading one by one the same link, leaving concurrency
and advanced testing to Tim and others :)
just wanted to feel the improvement, and I have to say Tim did his job
very well
 
   

web2py 1.76.5  vs 1.77.1 load test
 
   

10 requests:
1.76.5  - 0.3016 seconds
1.77.1 -  0.2736 seconds
 
   

100 requests:
1.76.5  - 1.7729 seconds
1.77.1 -  1.4585 seconds
 
   

1000 requests:
1.76.5  - 16.6493 seconds
1.77.1 -  13.4033 seconds
 
   

and here is the great test script :)
 
   

import sys, urllib
n = int((sys.argv[1:] or [10])[0])
s = 'http://127.0.0.1:8001/'#s = 'http://127.0.0.1:8002/'
for i in xrange(n):
 urllib.urlopen(s).read()
 
   

--
Subscription settings:http://groups.google.com/group/web2py/subscribe?hl=en
   
 

Re: [web2py] Re: rocket performance

[Thadeus Burgess]

I know I have been talking about x-sendfile alot lately

Is there any reason that rocket should support it?

--
Thadeus





On Tue, Apr 20, 2010 at 4:06 PM, mdipierro  wrote:
> ok. will fix this soon.
>
> On Apr 20, 4:03 pm, Timothy Farrell  wrote:
>> Ton's of references in epydoc.  Those will be there as long as
>> wsgiserver.py is in the distro.  Otherwise, run epydoc and sphinx.
>>
>> Other than that, theres:
>> \examples\views\default\index.html:line 35
>> - change link tohttp://launchpad.net/rocket
>>
>> \examples\views\default\who.html:line 77
>> - remove the reference so long as wsgiserver is not the default
>> - add "Rocket Web Server developed by Timothy Farrell."
>>
>> On 4/20/2010 9:21 AM, mdipierro wrote:
>>
>> > I made a mistake. Left references to cherry in some of the pages in
>> > applications/example. Could you help me locate them and suggest
>> > changes?
>>
>> > Massimo
>>
>> > On Apr 20, 8:11 am, Timothy Farrell  wrote:
>>
>> >> Thank you.  It's always good to see the fruits of one's labor come out.
>>
>> >> On 4/19/2010 3:57 PM, elffikk wrote:
>>
>> >>>    a simple test loading one by one the same link, leaving concurrency
>> >>> and advanced testing to Tim and others :)
>> >>> just wanted to feel the improvement, and I have to say Tim did his job
>> >>> very well
>>
>> >>> web2py 1.76.5  vs 1.77.1 load test
>>
>> >>> 10 requests:
>> >>> 1.76.5  - 0.3016 seconds
>> >>> 1.77.1 -  0.2736 seconds
>>
>> >>> 100 requests:
>> >>> 1.76.5  - 1.7729 seconds
>> >>> 1.77.1 -  1.4585 seconds
>>
>> >>> 1000 requests:
>> >>> 1.76.5  - 16.6493 seconds
>> >>> 1.77.1 -  13.4033 seconds
>>
>> >>> and here is the great test script :)
>>
>> >>> import sys, urllib
>> >>> n = int((sys.argv[1:] or [10])[0])
>> >>> s = 'http://127.0.0.1:8001/'#s = 'http://127.0.0.1:8002/'
>> >>> for i in xrange(n):
>> >>>     urllib.urlopen(s).read()
>>
>> >> --
>> >> Subscription 
>> >> settings:http://groups.google.com/group/web2py/subscribe?hl=en
>

[web2py] Re: rocket performance

[mdipierro]

ok. will fix this soon.

On Apr 20, 4:03 pm, Timothy Farrell  wrote:
> Ton's of references in epydoc.  Those will be there as long as
> wsgiserver.py is in the distro.  Otherwise, run epydoc and sphinx.
>
> Other than that, theres:
> \examples\views\default\index.html:line 35
> - change link tohttp://launchpad.net/rocket
>
> \examples\views\default\who.html:line 77
> - remove the reference so long as wsgiserver is not the default
> - add "Rocket Web Server developed by Timothy Farrell."
>
> On 4/20/2010 9:21 AM, mdipierro wrote:
>
> > I made a mistake. Left references to cherry in some of the pages in
> > applications/example. Could you help me locate them and suggest
> > changes?
>
> > Massimo
>
> > On Apr 20, 8:11 am, Timothy Farrell  wrote:
>
> >> Thank you.  It's always good to see the fruits of one's labor come out.
>
> >> On 4/19/2010 3:57 PM, elffikk wrote:
>
> >>>    a simple test loading one by one the same link, leaving concurrency
> >>> and advanced testing to Tim and others :)
> >>> just wanted to feel the improvement, and I have to say Tim did his job
> >>> very well
>
> >>> web2py 1.76.5  vs 1.77.1 load test
>
> >>> 10 requests:
> >>> 1.76.5  - 0.3016 seconds
> >>> 1.77.1 -  0.2736 seconds
>
> >>> 100 requests:
> >>> 1.76.5  - 1.7729 seconds
> >>> 1.77.1 -  1.4585 seconds
>
> >>> 1000 requests:
> >>> 1.76.5  - 16.6493 seconds
> >>> 1.77.1 -  13.4033 seconds
>
> >>> and here is the great test script :)
>
> >>> import sys, urllib
> >>> n = int((sys.argv[1:] or [10])[0])
> >>> s = 'http://127.0.0.1:8001/'#s = 'http://127.0.0.1:8002/'
> >>> for i in xrange(n):
> >>>     urllib.urlopen(s).read()
>
> >> --
> >> Subscription settings:http://groups.google.com/group/web2py/subscribe?hl=en

Re: [web2py] Re: rocket performance

[Timothy Farrell]

Ton's of references in epydoc.  Those will be there as long as 
wsgiserver.py is in the distro.  Otherwise, run epydoc and sphinx.


Other than that, theres:
\examples\views\default\index.html:line 35
- change link to http://launchpad.net/rocket

\examples\views\default\who.html:line 77
- remove the reference so long as wsgiserver is not the default
- add "Rocket Web Server developed by Timothy Farrell."


On 4/20/2010 9:21 AM, mdipierro wrote:

I made a mistake. Left references to cherry in some of the pages in
applications/example. Could you help me locate them and suggest
changes?

Massimo

On Apr 20, 8:11 am, Timothy Farrell  wrote:
   

Thank you.  It's always good to see the fruits of one's labor come out.

On 4/19/2010 3:57 PM, elffikk wrote:



 

   a simple test loading one by one the same link, leaving concurrency
and advanced testing to Tim and others :)
just wanted to feel the improvement, and I have to say Tim did his job
very well
   
 

web2py 1.76.5  vs 1.77.1 load test
   
 

10 requests:
1.76.5  - 0.3016 seconds
1.77.1 -  0.2736 seconds
   
 

100 requests:
1.76.5  - 1.7729 seconds
1.77.1 -  1.4585 seconds
   
 

1000 requests:
1.76.5  - 16.6493 seconds
1.77.1 -  13.4033 seconds
   
 

and here is the great test script :)
   
 

import sys, urllib
n = int((sys.argv[1:] or [10])[0])
s = 'http://127.0.0.1:8001/'# s = 'http://127.0.0.1:8002/'
for i in xrange(n):
urllib.urlopen(s).read()
   

--
Subscription settings:http://groups.google.com/group/web2py/subscribe?hl=en
 

[web2py] Re: rocket performance

[mdipierro]

I made a mistake. Left references to cherry in some of the pages in
applications/example. Could you help me locate them and suggest
changes?

Massimo

On Apr 20, 8:11 am, Timothy Farrell  wrote:
> Thank you.  It's always good to see the fruits of one's labor come out.
>
> On 4/19/2010 3:57 PM, elffikk wrote:
>
>
>
> >   a simple test loading one by one the same link, leaving concurrency
> > and advanced testing to Tim and others :)
> > just wanted to feel the improvement, and I have to say Tim did his job
> > very well
>
> > web2py 1.76.5  vs 1.77.1 load test
>
> > 10 requests:
> > 1.76.5  - 0.3016 seconds
> > 1.77.1 -  0.2736 seconds
>
> > 100 requests:
> > 1.76.5  - 1.7729 seconds
> > 1.77.1 -  1.4585 seconds
>
> > 1000 requests:
> > 1.76.5  - 16.6493 seconds
> > 1.77.1 -  13.4033 seconds
>
> > and here is the great test script :)
>
> > import sys, urllib
> > n = int((sys.argv[1:] or [10])[0])
> > s = 'http://127.0.0.1:8001/'# s = 'http://127.0.0.1:8002/'
> > for i in xrange(n):
> >    urllib.urlopen(s).read()
>
> --
> Subscription settings:http://groups.google.com/group/web2py/subscribe?hl=en

[web2py] Re: rocket performance

[mdipierro]

:-)

On Apr 19, 3:57 pm, elffikk  wrote:
>  a simple test loading one by one the same link, leaving concurrency
> and advanced testing to Tim and others :)
> just wanted to feel the improvement, and I have to say Tim did his job
> very well
>
> web2py 1.76.5  vs 1.77.1 load test
>
> 10 requests:
> 1.76.5  - 0.3016 seconds
> 1.77.1 -  0.2736 seconds
>
> 100 requests:
> 1.76.5  - 1.7729 seconds
> 1.77.1 -  1.4585 seconds
>
> 1000 requests:
> 1.76.5  - 16.6493 seconds
> 1.77.1 -  13.4033 seconds
>
> and here is the great test script :)
>
> import sys, urllib
> n = int((sys.argv[1:] or [10])[0])
> s = 'http://127.0.0.1:8001/'# s = 'http://127.0.0.1:8002/'
> for i in xrange(n):
>         urllib.urlopen(s).read()
>
> --
> Subscription settings:http://groups.google.com/group/web2py/subscribe?hl=en