Re: [web2py] Re: Server slow
So it rather fast not slow :=)
But the edited_by column is not showing a name, it shows a number.
But that is not very intresting as I don´t need a list like that, it was
just a test. I´m trying to optimize some reporting functions. I´ll have
to start from an other angle.
Kenneth
Now I understand.
The problem is here:
Field('edited_by', db[user_table], required=True),
When you do {{=orders}} is uses a default representation for
edited_by. The field contains a user id and the default representation
is the user name. So for each record it has to do a database lookup
(980 of them).
If you want to represent users by name, you should either cache them.
Something like this:
db.orders.edited_by.represent = lambda id: cache.ram('user:
%i'%id,lambda:db.auth_user(id).first_name,3600)
or turn your query into a join.
Massimo
On Dec 26, 11:37 pm, Kenneth Lundström
wrote:
What part of the model would you like to see?
This?
db.define_table('orders',
db.Field('name', 'string', length=40, requires=IS_NOT_EMPTY()),
db.Field('district', db.district, required=True),
db.Field('edited_by', db[user_table], required=True),
db.Field('edited_date', 'datetime', required=True),
db.Field('reward_selected', 'integer', default=0),
db.Field('members', 'integer', default=0),
db.Field('memb_no', 'string', default=0),
db.Field('locked', 'integer', default=False, required=True),
db.Field('cow_id', 'string', default=''),
migrate=settings.migrate)
So far there was no view defined for this controller. If I define a
empty one it takes like 2-3 seconds to show a empty page, controller
changed to dict(orders=orders).
If I define the view to be {{=orders}} it takes again 40 seconds to show.
The strange thing is if I change the view to
{{
for order in orders:
=XML(order)
pass
}}
it only takes about 2-3 seconds to show all orders. Not as nice as
=orders but very strange anyhow.
Kenneth
I have never seen this before. Can I see the model?
massimo
On Dec 25, 3:51 pm, Kenneth Lundstr�m
wrote:
Is this reproducible?
This happens all the time, if I do it ten times in a row every time it takes
38-39 seconds. Even with lynx on the server itself it takes about 40 seconds.
If I use:
t0=time.time()
orders = db(db.orders.id> 0).select()
logging.info('time to fetch %s' % (time.time()-t0))
The time is 1.75 - 1.90 seconds to fetch the data from database
I change the controller to this
sql = db(db.orders.id> 0)._select()
t0=time.time()
db.executesql(sql)
logging.info('time to fetch %s' % (time.time()-t0))
orders=[]
now I get times like 0.38 - 0.39 seconds
If I instead of db.orders.id< 490) to get half the rows all times drops to
half, even 40 seconds is now 20 seconds.
Just testing I tried with:
t0=time.time()
orders = db(db.orders.id> 0).select()
orders1 = db(db.orders.id> 0).select()
orders2 = db(db.orders.id> 0).select()
orders3 = db(db.orders.id> 0).select()
orders4 = db(db.orders.id> 0).select()
logging.info('time to fetch %s' % (time.time()-t0))
return orders
This takes about, 5x1,8s (to fetch data) + 35 seconds = 45 seconds
I tried with a different table that contains about 1250 rows, 6 columns. It
takes 1.2-1.3 seconds to fetch the data, but then over 60 seconds to display it.
Kenneth
Re: [web2py] Re: Server slow
What part of the model would you like to see?
This?
db.define_table('orders',
db.Field('name', 'string', length=40, requires=IS_NOT_EMPTY()),
db.Field('district', db.district, required=True),
db.Field('edited_by', db[user_table], required=True),
db.Field('edited_date', 'datetime', required=True),
db.Field('reward_selected', 'integer', default=0),
db.Field('members', 'integer', default=0),
db.Field('memb_no', 'string', default=0),
db.Field('locked', 'integer', default=False, required=True),
db.Field('cow_id', 'string', default=''),
migrate=settings.migrate)
So far there was no view defined for this controller. If I define a
empty one it takes like 2-3 seconds to show a empty page, controller
changed to dict(orders=orders).
If I define the view to be {{=orders}} it takes again 40 seconds to show.
The strange thing is if I change the view to
{{
for order in orders:
=XML(order)
pass
}}
it only takes about 2-3 seconds to show all orders. Not as nice as
=orders but very strange anyhow.
Kenneth
I have never seen this before. Can I see the model?
massimo
On Dec 25, 3:51 pm, Kenneth Lundström
wrote:
Is this reproducible?
This happens all the time, if I do it ten times in a row every time it takes
38-39 seconds. Even with lynx on the server itself it takes about 40 seconds.
If I use:
t0=time.time()
orders = db(db.orders.id>0).select()
logging.info('time to fetch %s' % (time.time()-t0))
The time is 1.75 - 1.90 seconds to fetch the data from database
I change the controller to this
sql = db(db.orders.id>0)._select()
t0=time.time()
db.executesql(sql)
logging.info('time to fetch %s' % (time.time()-t0))
orders=[]
now I get times like 0.38 - 0.39 seconds
If I instead of db.orders.id<490) to get half the rows all times drops to
half, even 40 seconds is now 20 seconds.
Just testing I tried with:
t0=time.time()
orders = db(db.orders.id>0).select()
orders1 = db(db.orders.id>0).select()
orders2 = db(db.orders.id>0).select()
orders3 = db(db.orders.id>0).select()
orders4 = db(db.orders.id>0).select()
logging.info('time to fetch %s' % (time.time()-t0))
return orders
This takes about, 5x1,8s (to fetch data) + 35 seconds = 45 seconds
I tried with a different table that contains about 1250 rows, 6 columns. It
takes 1.2-1.3 seconds to fetch the data, but then over 60 seconds to display it.
Kenneth
Re: [web2py] Re: Server slow
Is this reproducible?
This happens all the time, if I do it ten times in a row every time it takes
38-39 seconds. Even with lynx on the server itself it takes about 40 seconds.
If I use:
t0=time.time()
orders = db(db.orders.id> 0).select()
logging.info('time to fetch %s' % (time.time()-t0))
The time is 1.75 - 1.90 seconds to fetch the data from database
I change the controller to this
sql = db(db.orders.id> 0)._select()
t0=time.time()
db.executesql(sql)
logging.info('time to fetch %s' % (time.time()-t0))
orders=[]
now I get times like 0.38 - 0.39 seconds
If I instead of db.orders.id< 490) to get half the rows all times drops to
half, even 40 seconds is now 20 seconds.
Just testing I tried with:
t0=time.time()
orders = db(db.orders.id> 0).select()
orders1 = db(db.orders.id> 0).select()
orders2 = db(db.orders.id> 0).select()
orders3 = db(db.orders.id> 0).select()
orders4 = db(db.orders.id> 0).select()
logging.info('time to fetch %s' % (time.time()-t0))
return orders
This takes about, 5x1,8s (to fetch data) + 35 seconds = 45 seconds
I tried with a different table that contains about 1250 rows, 6 columns. It
takes 1.2-1.3 seconds to fetch the data, but then over 60 seconds to display it.
Kenneth
Re: [web2py] Re: Server slow
980 rows, 10 columns of data,
38 sec.
MySQL server is on the same server, Apache server is not local, but the
delay is allways 38-39 seconds.
Kenneth
No idea. How much data.
What db? is it local? Probably something is wrong anyway.
On Dec 24, 5:55 pm, Kenneth Lundström
wrote:
I finally had time to find out a little about loggin.conf.
In my logs/web2py.log I get this line:
2010-12-25 01:42:30,097 - root - INFO - time to fetch 1.81220602989
So it takes under 2 seconds to fetch the data and the rest 37 to show
it? How come?
Kenneth
Were is this logged, I could not find anything in any logs?
Kenneth
try
import time, logging
def testing():
t0=time.time()
orders = db(db.orders.id>0).select()
logging.info('time to fetch %s' % (time.time()-t0))
return orders
so you can isolate the problem and see if it is in fetching or
somewhere else (for example session locking).
On Dec 2, 4:21 pm, Kenneth Lundstr m
wrote:
When you test is that the only active connection to the db?
I tested it on an application running on the test instance. There could
have been lite activity one production site.
But when I tested loading the same function five times in about 10
minutes I allways 39 seconds to load the data.
Kenneth
2010/12/2 Kenneth Lundstr m:
Please tell us more about the setup. Are the three instances behind
running on the same server?
Yes.
why three? What do they do?
One is production, one is testing and last one is development. I
had two
instances running on a virtual server before and it worked fine.
This is not normal but I suspect the problem is with database.
Did you set DAL(...,pool_size=10)?
No, but I tried but no change.
If I try to select more rows:
def testing():
orders = db(db.orders.id> 0).select()
return orders
It takes 80 seconds to return 1608 rows.
If I change the return to return len(troops) it only takes 1,5
seconds.
I have no view defined, just trying out the database.
Kenneth
On Dec 2, 5:13 am, Kenneth Lundstr m
wrote:
Hello list,
need some help finding out why my server is so slow.
The server is with a Intel Celeron 2.66 GHz CPU, 4 GB of memory
CentOS 5.5 64-bit, Apache 2.2.3, MySQL 5.0.77, mod_wsgi, Web2py
1.89.5
The server is dedicated to web2py, there is three instances of
web2py
running.
I have migrate=False, sessions on disc.
If I try the following code
def testing():
customers = db(db.customer.id>0).select()
return customers
takes 39 seconds, 5 times in a row, to return 979 rows.
If I put db.customer<100 it takes 4,5 seconds to return 87
rows.
Is it just me or are this a bit long times?
Kenneth
Re: [web2py] Re: Server slow
I finally had time to find out a little about loggin.conf.
In my logs/web2py.log I get this line:
2010-12-25 01:42:30,097 - root - INFO - time to fetch 1.81220602989
So it takes under 2 seconds to fetch the data and the rest 37 to show
it? How come?
Kenneth
Were is this logged, I could not find anything in any logs?
Kenneth
try
import time, logging
def testing():
t0=time.time()
orders = db(db.orders.id> 0).select()
logging.info('time to fetch %s' % (time.time()-t0))
return orders
so you can isolate the problem and see if it is in fetching or
somewhere else (for example session locking).
On Dec 2, 4:21 pm, Kenneth Lundström
wrote:
> When you test is that the only active connection to the db?
I tested it on an application running on the test instance. There could
have been lite activity one production site.
But when I tested loading the same function five times in about 10
minutes I allways 39 seconds to load the data.
Kenneth
2010/12/2 Kenneth Lundström:
Please tell us more about the setup. Are the three instances behind
running on the same server?
Yes.
why three? What do they do?
One is production, one is testing and last one is development. I
had two
instances running on a virtual server before and it worked fine.
This is not normal but I suspect the problem is with database.
Did you set DAL(...,pool_size=10)?
No, but I tried but no change.
If I try to select more rows:
def testing():
orders = db(db.orders.id>0).select()
return orders
It takes 80 seconds to return 1608 rows.
If I change the return to return len(troops) it only takes 1,5
seconds.
I have no view defined, just trying out the database.
Kenneth
On Dec 2, 5:13 am, Kenneth Lundström
wrote:
Hello list,
need some help finding out why my server is so slow.
The server is with a Intel Celeron 2.66 GHz CPU, 4 GB of memory
CentOS 5.5 64-bit, Apache 2.2.3, MySQL 5.0.77, mod_wsgi, Web2py
1.89.5
The server is dedicated to web2py, there is three instances of
web2py
running.
I have migrate=False, sessions on disc.
If I try the following code
def testing():
customers = db(db.customer.id> 0).select()
return customers
takes 39 seconds, 5 times in a row, to return 979 rows.
If I put db.customer< 100 it takes 4,5 seconds to return 87
rows.
Is it just me or are this a bit long times?
Kenneth
Re: [web2py] Re: Server slow
Were is this logged, I could not find anything in any logs?
Kenneth
try
import time, logging
def testing():
t0=time.time()
orders = db(db.orders.id> 0).select()
logging.info('time to fetch %s' % (time.time()-t0))
return orders
so you can isolate the problem and see if it is in fetching or
somewhere else (for example session locking).
On Dec 2, 4:21 pm, Kenneth Lundström
wrote:
> When you test is that the only active connection to the db?
I tested it on an application running on the test instance. There could
have been lite activity one production site.
But when I tested loading the same function five times in about 10
minutes I allways 39 seconds to load the data.
Kenneth
2010/12/2 Kenneth Lundström:
Please tell us more about the setup. Are the three instances behind
running on the same server?
Yes.
why three? What do they do?
One is production, one is testing and last one is development. I had two
instances running on a virtual server before and it worked fine.
This is not normal but I suspect the problem is with database.
Did you set DAL(...,pool_size=10)?
No, but I tried but no change.
If I try to select more rows:
def testing():
orders = db(db.orders.id>0).select()
return orders
It takes 80 seconds to return 1608 rows.
If I change the return to return len(troops) it only takes 1,5 seconds.
I have no view defined, just trying out the database.
Kenneth
On Dec 2, 5:13 am, Kenneth Lundström
wrote:
Hello list,
need some help finding out why my server is so slow.
The server is with a Intel Celeron 2.66 GHz CPU, 4 GB of memory
CentOS 5.5 64-bit, Apache 2.2.3, MySQL 5.0.77, mod_wsgi, Web2py 1.89.5
The server is dedicated to web2py, there is three instances of web2py
running.
I have migrate=False, sessions on disc.
If I try the following code
def testing():
customers = db(db.customer.id> 0).select()
return customers
takes 39 seconds, 5 times in a row, to return 979 rows.
If I put db.customer< 100 it takes 4,5 seconds to return 87 rows.
Is it just me or are this a bit long times?
Kenneth
Re: [web2py] Re: Server slow
> When you test is that the only active connection to the db? I tested it on an application running on the test instance. There could have been lite activity one production site. But when I tested loading the same function five times in about 10 minutes I allways 39 seconds to load the data. Kenneth 2010/12/2 Kenneth Lundström: Please tell us more about the setup. Are the three instances behind running on the same server? Yes. why three? What do they do? One is production, one is testing and last one is development. I had two instances running on a virtual server before and it worked fine. This is not normal but I suspect the problem is with database. Did you set DAL(...,pool_size=10)? No, but I tried but no change. If I try to select more rows: def testing(): orders = db(db.orders.id> 0).select() return orders It takes 80 seconds to return 1608 rows. If I change the return to return len(troops) it only takes 1,5 seconds. I have no view defined, just trying out the database. Kenneth On Dec 2, 5:13 am, Kenneth Lundström wrote: Hello list, need some help finding out why my server is so slow. The server is with a Intel Celeron 2.66 GHz CPU, 4 GB of memory CentOS 5.5 64-bit, Apache 2.2.3, MySQL 5.0.77, mod_wsgi, Web2py 1.89.5 The server is dedicated to web2py, there is three instances of web2py running. I have migrate=False, sessions on disc. If I try the following code def testing(): customers = db(db.customer.id>0).select() return customers takes 39 seconds, 5 times in a row, to return 979 rows. If I put db.customer<100 it takes 4,5 seconds to return 87 rows. Is it just me or are this a bit long times? Kenneth
Re: [web2py] Re: Server slow
Kenneth, When you test is that the only active connection to the db? 2010/12/2 Kenneth Lundström : >> Please tell us more about the setup. Are the three instances behind >> running on the same server? > > Yes. > >> why three? What do they do? > > One is production, one is testing and last one is development. I had two > instances running on a virtual server before and it worked fine. > >> This is not normal but I suspect the problem is with database. >> Did you set DAL(...,pool_size=10)? > > No, but I tried but no change. > > If I try to select more rows: > def testing(): > orders = db(db.orders.id > 0).select() > return orders > > It takes 80 seconds to return 1608 rows. > > If I change the return to return len(troops) it only takes 1,5 seconds. > > I have no view defined, just trying out the database. > > > Kenneth > > >> >> On Dec 2, 5:13 am, Kenneth Lundström >> wrote: >>> >>> Hello list, >>> >>> need some help finding out why my server is so slow. >>> >>> The server is with a Intel Celeron 2.66 GHz CPU, 4 GB of memory >>> CentOS 5.5 64-bit, Apache 2.2.3, MySQL 5.0.77, mod_wsgi, Web2py 1.89.5 >>> The server is dedicated to web2py, there is three instances of web2py >>> running. >>> >>> I have migrate=False, sessions on disc. >>> >>> If I try the following code >>> >>> def testing(): >>> customers = db(db.customer.id> 0).select() >>> return customers >>> >>> takes 39 seconds, 5 times in a row, to return 979 rows. >>> >>> If I put db.customer< 100 it takes 4,5 seconds to return 87 rows. >>> >>> Is it just me or are this a bit long times? >>> >>> Kenneth > >
Re: [web2py] Re: Server slow
> Please tell us more about the setup. Are the three instances behind running on the same server? Yes. > why three? What do they do? One is production, one is testing and last one is development. I had two instances running on a virtual server before and it worked fine. > This is not normal but I suspect the problem is with database. > Did you set DAL(...,pool_size=10)? No, but I tried but no change. If I try to select more rows: def testing(): orders = db(db.orders.id > 0).select() return orders It takes 80 seconds to return 1608 rows. If I change the return to return len(troops) it only takes 1,5 seconds. I have no view defined, just trying out the database. Kenneth On Dec 2, 5:13 am, Kenneth Lundström wrote: Hello list, need some help finding out why my server is so slow. The server is with a Intel Celeron 2.66 GHz CPU, 4 GB of memory CentOS 5.5 64-bit, Apache 2.2.3, MySQL 5.0.77, mod_wsgi, Web2py 1.89.5 The server is dedicated to web2py, there is three instances of web2py running. I have migrate=False, sessions on disc. If I try the following code def testing(): customers = db(db.customer.id> 0).select() return customers takes 39 seconds, 5 times in a row, to return 979 rows. If I put db.customer< 100 it takes 4,5 seconds to return 87 rows. Is it just me or are this a bit long times? Kenneth
