I do not understand. When you compare Django vs web2p[y performance, you seem to be comparing the MCycles for /api/1.0/position<https://appengine.google.com/logs?version_id=production.359553040742132058&app_id=s%7Egps-holdsat-hrd&filter_type=labels&filter=path%3A%22%2Fapi%2F1.0%2Fposition%22&severity_level_override=1&view=Search> (in case of Django) with /main/queues/position<https://appengine.google.com/logs?version_id=testing.360875204440667710&app_id=s%7Eapache-felipemeirelles&filter_type=labels&filter=path%3A%22%2Fmain%2Fqueues%2Fposition%22&severity_level_override=1&view=Search> (in the case of web2py). Shouldn't you be comparing with /main/api/position<https://appengine.google.com/logs?version_id=testing.360875204440667710&app_id=s%7Eapache-felipemeirelles&filter_type=labels&filter=path%3A%22%2Fmain%2Fapi%2Fposition%22&severity_level_override=1&view=Search>? Or perhaps the names are reversed?
Do they perform the same DB IO? Do they both store sessions in DB (web2py does)? It is likely Django uses simplejson binary while web2py (for portability) only ships with simplejson in pure python. This may be another performance loss if you use json a lot. We need to fix this at the web2py level. Massimo On Wednesday, 8 August 2012 08:06:05 UTC-5, Felipe Meirelles wrote: > > Well, after 24h running the new model less code I think I can post some > data: > > First of all I'll explain my application archtecture: > > I have a cloud server gattering information from around 150 gps on some > asserts. Each assert sends information every 30s, and some sends additional > reading information, wich give me a total arround 150k request/day. This > cloud server is integrated with appengine trought a REST API, speaching > Json to each other. > > Before the model less refactor I had the following status: > *Current Load [image: > help]<https://developers.google.com/appengine/kb/general#currentload> > *URI Req/SeccurrentRequestslast 6 hrsRuntime MCycleslast hrAvg Latency > /main/queues/position<https://appengine.google.com/logs?version_id=testing.360875204440667710&app_id=s%7Eapache-felipemeirelles&filter_type=labels&filter=path%3A%22%2Fmain%2Fqueues%2Fposition%22&severity_level_override=1&view=Search> > 1.248.78K3511200ms > /main/api/position<https://appengine.google.com/logs?version_id=testing.360875204440667710&app_id=s%7Eapache-felipemeirelles&filter_type=labels&filter=path%3A%22%2Fmain%2Fapi%2Fposition%22&severity_level_override=1&view=Search> > 0.944.37K267600 ms > As you can see, I have controllers, the /api/ receives a call from the > cloud server and enqueues a process request on appengine (this is a very > simple controller, no data processing at all), and the /queues/ wich > actualy process the data, make some inserts/updates and handle the memcache. > > After the refactor: > > *Current Load [image: > help]<https://developers.google.com/appengine/kb/general#currentload> > *URI Req/SeccurrentRequestslast 6 hrsRuntime MCycleslast hrAvg Latency > /main/queues/position<https://appengine.google.com/logs?version_id=testing.360875204440667710&app_id=s%7Eapache-felipemeirelles&filter_type=labels&filter=path%3A%22%2Fmain%2Fqueues%2Fposition%22&severity_level_override=1&view=Search> > 1.149.34K180404 ms > /main/api/position<https://appengine.google.com/logs?version_id=testing.360875204440667710&app_id=s%7Eapache-felipemeirelles&filter_type=labels&filter=path%3A%22%2Fmain%2Fapi%2Fposition%22&severity_level_override=1&view=Search> > 0.944.80K78139 ms > Im only posting the position URL becouse its the most complex and most > hitted. > > In comparsion, the Django project (same logic, I have it still runing > while the web2py one is not ready): > > *Current Load [image: > help]<https://developers.google.com/appengine/kb/general#currentload> > *URI Req/SeccurrentRequestslast 6 hrsRuntime MCycleslast hrAvg Latencylast > hr > /api/1.0/position<https://appengine.google.com/logs?version_id=production.359553040742132058&app_id=s%7Egps-holdsat-hrd&filter_type=labels&filter=path%3A%22%2Fapi%2F1.0%2Fposition%22&severity_level_override=1&view=Search> > 0.850.13K92406 ms > On the django version I didnt had the queue yet, so this one URL proccess > the request at all. > > Final toughts are, after a model less aproch, I reached half of the > performance from django (still, some parts of the code are sightly > diferent, and a lot more Json conversion and memcache is used in favour of > reducing datastore access, wich can explain the loss on processing > performance). Also, after the model less aproch, compared to web2py it > self, I got a huge performance gain, and expect a solid cost reduction > (I'll post the cost reduction latter, when appengine consolidates the > billing). > > Thanks you all for sharing your toughts. > > On Tuesday, August 7, 2012 6:56:15 PM UTC-3, Massimo Di Pierro wrote: >> >> Please share you benchmarks when you think you are done. >> >> On Tuesday, 7 August 2012 14:29:51 UTC-5, Felipe Meirelles wrote: >>> >>> Well, I'm still in the middle of tests but all points to a huge drop on >>> cpu usage using a model less aproch. I've made, in a few days ill post >>> numbers, but the cpu_ms is arround half of it was. >>> >>> I've set all db.define_table() in separete models, and just include them >>> where i use them. Also, all the configs from 0.py were split in files, and >>> are just called when I need them. >>> >>> Thanks for the help! >>> >>> On Tue, Aug 7, 2012 at 4:25 PM, Anthony <abas...@gmail.com> wrote: >>> >>>> The app I'm developing is highly dependent on auth. >>>>> >>>>> Does it mean that any request that uses auth won't be faster than >>>>> 200ms on GAE? >>>>> >>>> >>>> Note, if you're just checking for login, I don't think you need to call >>>> auth.define_tables() (that should only be needed for registration, login, >>>> and checking permissions/membership). Also, make sure to use >>>> auth.define_tables(migrate=False) to avoid migrations on the auth tables >>>> in >>>> production. >>>> >>>> Anthony >>>> >>>> -- >>>> >>>> >>>> >>>> >>> >>> >>> >>> -- >>> Att, >>> Felipe Meirelles. >>> >>> --
