[web2py] Re: GAE Caching and Views

[howesc]

Constantine, do you mind sharing what and how you are serializing things 
with protocol buffers?  perhaps we can adapt what you are doing into the 
web2py GAE caching core.

JT - thanks for the nice writeup.  i'm working on optimizing some web2py 
apps for customers now, and should be using your technique where i can.

[web2py] Re: GAE Caching and Views

[Constantine Vasil]

>not picklable

there is a way to serialize with protocol buffers.  I am using it in my 
code but it better be implemented in the web2py core as a 'gae' dependency.

[web2py] Re: GAE Caching and Views

[johntynan]

Thanks everyone for your excellent help!

I posted an account of enabling caching with web2py on GAE in a blog
post here:

http://opensourcebroadcasting.blogspot.com/2011/11/web2py-caching-and-google-app-engine.html

All the best, JT

Re: [web2py] Re: GAE Caching and Views

[Bruno Rocha]

On Tue, Nov 22, 2011 at 8:34 PM, Massimo Di Pierro <
massimo.dipie...@gmail.com> wrote:

> . I have never tried it on GAE


I have a select cache on my app, works locally, but fails on GAE. It does
not raise any exception but does not cache anything, and I only realized it
does not works when I see my "clean cache" function raising an error.

My code is:
https://github.com/rochacbruno/Movuca/blob/master/modules/config.py#L41

I have not tried other cache methods for GAE.


-- 

Bruno Rocha
[http://rochacbruno.com.br]

[web2py] Re: GAE Caching and Views

[howesc]

hmmm, i've never known GAE to cache selects.  at one point i thought the 
book said as much, now it simply says:

"The results of a select are complex, un-pickleable objects; they cannot be 
stored in a session and cannot be cached in any other way than the one 
explained here."

and i believe that un-pickleable means not memcacheable.

reading dal.py, select() in the GAEAdaptor does not seem to do anything 
with cache.  (maybe i'm mis-reading it)

[web2py] Re: GAE Caching and Views

[Massimo Di Pierro]

select(cache) does not serielizes Rows but the dictionary returned by
the select. I have never tried it on GAE. For sure you cannot cache in
run but it should be possible to cache.memcache. If that does not work
it is because the datastore returns a type that is not serializible.

On Nov 22, 3:26 pm, howesc  wrote:
> actually, unless something changed on on GAE, select caching does not work
> because the rows object is not picklable.
>
> the @cache decorator caches the dict that you return to the view, not the
> rendered view.  if something in that dict is not picklable (say a rows
> object) the cache will fail.  anthony's first suggestion should work in all
> cases.

[web2py] Re: GAE Caching and Views

[howesc]

actually, unless something changed on on GAE, select caching does not work 
because the rows object is not picklable.

the @cache decorator caches the dict that you return to the view, not the 
rendered view.  if something in that dict is not picklable (say a rows 
object) the cache will fail.  anthony's first suggestion should work in all 
cases.

[web2py] Re: GAE Caching and Views

[Anthony]

How about:

return response.render(dict(pledges=pledges))

An alternative is just to cache the select instead of the whole view:

pledges=db((db.pledge.segment==segment_id) &
(db.pledge.read == False)).select(orderby=~db.pledge.created_on,
cache=(cache.ram, 5))

In that case, you would remove the @cache decorator from the function.

See http://www.web2py.com/book/default/chapter/06#Caching-Selects.

Anthony

On Tuesday, November 22, 2011 10:25:22 AM UTC-5, johntynan wrote:
>
> I am using the following decorator:
>
> @cache(request.env.path_info, time_expire=5, cache_model=cache.ram)
>
> To cache some db queries, however, when I go to deploy this on Google
> App Engine, I receive a "PicklingError: Can't pickle  'function'>: attribute lookup __builtin__.function failed" You can see
> the traceback here:
>
> https://gist.github.com/1384892
>
> I have a feeling the error has to do with caching a for loop in a
> view.  You can see the view here:
>
>
> http://code.google.com/p/pledgedrivetracker/source/browse/pledgedrives/views/refresh/thank_yous.html?spec=svnf8cd17ed0367521b8bd726c48ac829e920207c11&r=f8cd17ed0367521b8bd726c48ac829e920207c11
>
> You can also see the "thank_yous" function here:
>
>
> http://code.google.com/p/pledgedrivetracker/source/browse/pledgedrives/controllers/refresh.py?spec=svnf8cd17ed0367521b8bd726c48ac829e920207c11&r=f8cd17ed0367521b8bd726c48ac829e920207c11#74
>
> I think the solution lies with creating the view within the function
> using return response.render() as described in the web2py book here:
>
> http://www.web2py.com/book/default/chapter/04?search=cache+view
>
> but I'm at a loss for how to do this.  Does anyone have any clear
> examples for using response.render and an html template.
>
> Thanks!
>
>

[web2py] Re: GAE caching

[howesc]

actually i would recommend that this be removed from web2py.  Ikai Lan (app 
engine system developer at google) says they can detect those dummy requests 
and punish apps that do that (was chatting with him at a conference last 
fall).  i would instead pay the modest $9 a month for "always on".

or the other solution is to just get enough users that you have constant 
traffic. ;)

cfh