Re: [google-appengine] High Replication - Large CPU spikes without Datastore Access

2011-02-08 Thread Wim den Ouden
I'm tuning for the hr.
update entities are fast
entities with 3 properties add, delete 149ms 101cpu_ms 84api_cpu_ms
(tested over more days)
entities with 25 properties
http://code.google.com/p/relat/source/browse/trunk/relat/e-comm/contactmodel.py
add, delete 1825cpu_ms 1807api_cpu_ms (tested over more days)
it also depend on what properties
http://code.google.com/p/relat/source/browse/trunk/relat/e-comm/journmodel.py
2160cpu_ms 2142api_cpu_ms (tested over more days)
i have taken out the properties which are not necessary at the moment.
(into new kinds)
All cud actions are done by taskqueque
gr
wim

2011/2/8 Simon Knott knott.si...@gmail.com:
 I've already posted this on the Java GAE group, but maybe someone has
 seen this issue on a Python-hosted app.

 I'm carrying out some testing of my app against the HR-store, instead
 of the M/S datastore, and I've noticed that occasionally I'll get a
 request which has a massive CPU spike.  These spikes consume
 ~30-40seconds of CPU time, when my usual request times are ~500ms.
 I'd assumed that something must be fundamentally wrong with my
 application against the HR datastore, but I've now had three requests
 to AppStats which have the same problem and one of them was an
 image!

 For example:

 /appstats/static/app_engine_logo_sm.gif   200   131ms    31811cpu_ms
 --- Image?!
 /appstats/stats   200 536ms         32375cpu_ms
 /appstats/stats 200 169ms         23878cpu_ms


 Yesterday I had the request below in my log.  It didn't touch the
 datastore, had 6 calls to MemCache and according to AppStats had a
 grand total time of 125ms.

 /user/get 200 156ms     42020cpu_ms

 What could cause these large CPU spikes?  They aren't loading
 requests, as I'm correctly getting those marked in my logs and they
 are in the usual region of 2-3s duration. I also don't experience them
 on the same app against the M/S datastore.  My initial thoughts were
 that AppStats was potentially the cause of the problem, until I saw
 the request for a static image with massive CPU usage!

 --
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.





-- 
gr
Wim den Ouden
Custom applications, based on Google App Engine https://e-comm1.appspot.com/
Free open source E-commerce/E-bookkeeping/E-business framework (web)
apps, http://code.google.com/p/relat/

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



Re: [google-appengine] High Replication - Large CPU spikes without Datastore Access

2011-02-08 Thread Simon Knott
Just to emphasise, I'm seeing these spikes on requests which don't hit the 
datastore at all, and I don't see the same spikes on the same app with the 
datastore set to M/S.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



Re: [google-appengine] High Replication - Large CPU spikes without Datastore Access

2011-02-08 Thread Wim den Ouden
Hi Simon,
I'm glad your mention the subject, to get a spead gain i moved out all
the transaction surrounding where possible before testing the hr
datastore.
But it seems optimized for transactions because it is much faster in a
transaction. After putting all put actions in a transaction the kind
which gives me the biggest spikes (journ) is now 755cpu_ms
718api_cpu_ms (1/3 remaining cpu_ms within a transaction, without
a transaction is much, much more cpu_ms intensive) wow i'm glad. Gona
change everything back!!! (journ has about 10 indexes that is also
part of the speed difference)
I'm using python with jquery on the client, so a lot cpu usage is on
the client, i don't see other difference in use (spikes)
gr
wim

2011/2/8 Simon Knott knott.si...@gmail.com:
 Just to emphasise, I'm seeing these spikes on requests which don't hit the
 datastore at all, and I don't see the same spikes on the same app with the
 datastore set to M/S.

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




-- 
gr
Wim den Ouden
Custom applications, based on Google App Engine https://e-comm1.appspot.com/
Free open source E-commerce/E-bookkeeping/E-business framework (web)
apps, http://code.google.com/p/relat/

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.