[google-appengine] Re: jinja2 environment loading on each request to /
On Mar 27, 5:38 am, gops wrote: > that cookbook code was written by me. sorry for that...but > cookbook does not allow to re-edit once you post it. I thought that as well, but it turns out you can edit your recipes. After you login, click on the "Edit My Recipes" link at the top right corner. (Yeah, they really should add an edit link right on the recipe page.) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: jinja2 environment loading on each request to /
that cookbook code was written by me. sorry for that...but
cookbook does not allow to re-edit once you post it.
I actually use , multiple loader for productin and testing,
the production uses memcache -- which not allow frequent changes to
template. { you have to return the expire date function in order to do
that } ,
so i use filesystemloader for testing purpose
as the marshal module is not available , i did an ugly hack and remove
the marshal code from bcccache ( conditionally via debug ) and
replace with my own string routine ... it is working good so far... my
response time are normally along 200-300ms with two-three datastore
read/write. i will paste it once i figured out how to do it without
touching the jinja2 code.
nonetheless , i think , jinja2 itself caching the template bytecode if
the instance is not restarted. so , jinja2 is good for a high traffic
site.
On Mar 13, 6:47 pm, peterh wrote:
> this is what i use:
> ===
> from jinja2 import Environment, FunctionLoader , TemplateNotFound
> from google.appengine.api import memcache
> import os,logging
> VERSION = os.environ['CURRENT_VERSION_ID']
>
> logging.warn ("COLD START")
> def jinja2_template_loader(templatename):
> templatepath = os.path.abspath(os.curdir+'/
> template/'+templatename)
> template = memcache.get(templatepath+VERSION)
> logging.debug( "..INSIDE JINJA")
> if template is None:
> try:
> logging.debug( "..READ")
> template = file(templatepath).read()
> memcache.set(templatepath+VERSION,template,time=500)
> except:
> template = None
> return template
> SERVER = os.environ.get('SERVER_SOFTWARE','').lower()
> if SERVER.startswith('devel'):
> logging.debug("...DEVEL JINJA")
> load_this = Environment(cache_size=0,loader = FunctionLoader
> (jinja2_template_loader))
> else:
> load_this = Environment(loader= FunctionLoader
> (jinja2_template_loader))
>
> def render(template_name,context):
> template = load_this.get_template(template_name)
> content = template.render(context)
> return content
>
> =
>
> if your app handlers are cached (ie via "if __name__ == '__main__':main
> ()") this should be imported only once per server instance. It also
> caches templates both in memcache (for 8 minutes) and in instance
> memory
>
> (in case of local testing, templates are not cached in memory, so you
> can make changes and see the changes immediately)
>
> HTH
> Peter
>
> On Mar 13, 7:34 am, Jarek Zgoda wrote:
>
> > If you move Environment creation routine to another module it will be
> > cached by Google, but only for few seconds (2-3?), so if your
> > application does not get enough traffic you'd gain nothing from this.
>
> > In my attempts to speedup things I discovered that only loader can be
> > cached, but it has to be invalidated upon each new deployment so its
> > key has to have application version added (this is how I did key
> > versioning in my caching tools). Unfortunately, the gain from this
> > caching approaches 0.
>
> > And the last thing: Armin Ronacher, the guy behind Jinja, is aware of
> > the problem and has some ideas on how to improve things on GAE,
> > hopefully including template bytecode caching (this would be really
> > super-nice!).
>
> > On 13 Mar, 00:38, pedepy wrote:
>
> > > hey so i use jinja2 templates because they are better [no citation
> > > needed], but from what i understand, loading the jinja2 environment
> > > can be quite an expensive task. Through logging, I have found out that
> > > each request to / loads the environment over and over.. I have those
> > > routines in a seperate python script (not the one that app.yaml
> > > fowards requests to for /).
>
> > > anything im missing here ? I think this could be adding also an extra
> > > 1000 cpu ms on every requests ...
>
> > > (by the way, i have tried the cookbook's memcache technique and it
> > > only works locally, it fails on the google servers..)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: jinja2 environment loading on each request to /
How would a beginner implement this, Peter H.? Only loading it once
every eight minutes sounds great to me, and I have no idea where to
put your code.
On Mar 13, 9:47 am, peterh wrote:
> this is what i use:
> ===
> fromjinja2import Environment, FunctionLoader , TemplateNotFound
> from google.appengine.api import memcache
> import os,logging
> VERSION = os.environ['CURRENT_VERSION_ID']
>
> logging.warn ("COLD START")
> def jinja2_template_loader(templatename):
> templatepath = os.path.abspath(os.curdir+'/
> template/'+templatename)
> template = memcache.get(templatepath+VERSION)
> logging.debug( "..INSIDE JINJA")
> if template is None:
> try:
> logging.debug( "..READ")
> template = file(templatepath).read()
> memcache.set(templatepath+VERSION,template,time=500)
> except:
> template = None
> return template
> SERVER = os.environ.get('SERVER_SOFTWARE','').lower()
> if SERVER.startswith('devel'):
> logging.debug("...DEVEL JINJA")
> load_this = Environment(cache_size=0,loader = FunctionLoader
> (jinja2_template_loader))
> else:
> load_this = Environment(loader= FunctionLoader
> (jinja2_template_loader))
>
> def render(template_name,context):
> template = load_this.get_template(template_name)
> content = template.render(context)
> return content
>
> =
>
> if your app handlers are cached (ie via "if __name__ == '__main__':main
> ()") this should be imported only once per server instance. It also
> caches templates both in memcache (for 8 minutes) and in instance
> memory
>
> (in case of local testing, templates are not cached in memory, so you
> can make changes and see the changes immediately)
>
> HTH
> Peter
>
> On Mar 13, 7:34 am, Jarek Zgoda wrote:
>
>
>
> > If you move Environment creation routine to another module it will be
> > cached by Google, but only for few seconds (2-3?), so if your
> > application does not get enough traffic you'd gain nothing from this.
>
> > In my attempts to speedup things I discovered that only loader can be
> > cached, but it has to be invalidated upon each new deployment so its
> > key has to have application version added (this is how I did key
> > versioning in my caching tools). Unfortunately, the gain from this
> > caching approaches 0.
>
> > And the last thing: Armin Ronacher, the guy behind Jinja, is aware of
> > the problem and has some ideas on how to improve things on GAE,
> > hopefully including template bytecode caching (this would be really
> > super-nice!).
>
> > On 13 Mar, 00:38, pedepy wrote:
>
> > > hey so i usejinja2templates because they are better [no citation
> > > needed], but from what i understand, loading thejinja2environment
> > > can be quite an expensive task. Through logging, I have found out that
> > > each request to / loads the environment over and over.. I have those
> > > routines in a seperate python script (not the one that app.yaml
> > > fowards requests to for /).
>
> > > anything im missing here ? I think this could be adding also an extra
> > > 1000 cpu ms on every requests ...
>
> > > (by the way, i have tried the cookbook's memcache technique and it
> > > only works locally, it fails on the google servers..)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: jinja2 environment loading on each request to /
this is what i use:
===
from jinja2 import Environment, FunctionLoader , TemplateNotFound
from google.appengine.api import memcache
import os,logging
VERSION = os.environ['CURRENT_VERSION_ID']
logging.warn ("COLD START")
def jinja2_template_loader(templatename):
templatepath = os.path.abspath(os.curdir+'/
template/'+templatename)
template = memcache.get(templatepath+VERSION)
logging.debug( "..INSIDE JINJA")
if template is None:
try:
logging.debug( "..READ")
template = file(templatepath).read()
memcache.set(templatepath+VERSION,template,time=500)
except:
template = None
return template
SERVER = os.environ.get('SERVER_SOFTWARE','').lower()
if SERVER.startswith('devel'):
logging.debug("...DEVEL JINJA")
load_this = Environment(cache_size=0,loader = FunctionLoader
(jinja2_template_loader))
else:
load_this = Environment(loader= FunctionLoader
(jinja2_template_loader))
def render(template_name,context):
template = load_this.get_template(template_name)
content = template.render(context)
return content
=
if your app handlers are cached (ie via "if __name__ == '__main__':main
()") this should be imported only once per server instance. It also
caches templates both in memcache (for 8 minutes) and in instance
memory
(in case of local testing, templates are not cached in memory, so you
can make changes and see the changes immediately)
HTH
Peter
On Mar 13, 7:34 am, Jarek Zgoda wrote:
> If you move Environment creation routine to another module it will be
> cached by Google, but only for few seconds (2-3?), so if your
> application does not get enough traffic you'd gain nothing from this.
>
> In my attempts to speedup things I discovered that only loader can be
> cached, but it has to be invalidated upon each new deployment so its
> key has to have application version added (this is how I did key
> versioning in my caching tools). Unfortunately, the gain from this
> caching approaches 0.
>
> And the last thing: Armin Ronacher, the guy behind Jinja, is aware of
> the problem and has some ideas on how to improve things on GAE,
> hopefully including template bytecode caching (this would be really
> super-nice!).
>
> On 13 Mar, 00:38, pedepy wrote:
>
> > hey so i use jinja2 templates because they are better [no citation
> > needed], but from what i understand, loading the jinja2 environment
> > can be quite an expensive task. Through logging, I have found out that
> > each request to / loads the environment over and over.. I have those
> > routines in a seperate python script (not the one that app.yaml
> > fowards requests to for /).
>
> > anything im missing here ? I think this could be adding also an extra
> > 1000 cpu ms on every requests ...
>
> > (by the way, i have tried the cookbook's memcache technique and it
> > only works locally, it fails on the google servers..)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: jinja2 environment loading on each request to /
If you move Environment creation routine to another module it will be cached by Google, but only for few seconds (2-3?), so if your application does not get enough traffic you'd gain nothing from this. In my attempts to speedup things I discovered that only loader can be cached, but it has to be invalidated upon each new deployment so its key has to have application version added (this is how I did key versioning in my caching tools). Unfortunately, the gain from this caching approaches 0. And the last thing: Armin Ronacher, the guy behind Jinja, is aware of the problem and has some ideas on how to improve things on GAE, hopefully including template bytecode caching (this would be really super-nice!). On 13 Mar, 00:38, pedepy wrote: > hey so i use jinja2 templates because they are better [no citation > needed], but from what i understand, loading the jinja2 environment > can be quite an expensive task. Through logging, I have found out that > each request to / loads the environment over and over.. I have those > routines in a seperate python script (not the one that app.yaml > fowards requests to for /). > > anything im missing here ? I think this could be adding also an extra > 1000 cpu ms on every requests ... > > (by the way, i have tried the cookbook's memcache technique and it > only works locally, it fails on the google servers..) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: jinja2 environment loading on each request to /
HI One thing I noticed when hacking together a zope3 stack, was that any z3 setup/loading of zcml would execute on every request if the setup call was part of my main. By putting all of the setup into a seperate import and not calling the setup explicitly would mean it would be cached. Rgds T On Mar 13, 8:38 am, pedepy wrote: > hey so i use jinja2 templates because they are better [no citation > needed], but from what i understand, loading the jinja2 environment > can be quite an expensive task. Through logging, I have found out that > each request to / loads the environment over and over.. I have those > routines in a seperate python script (not the one that app.yaml > fowards requests to for /). > > anything im missing here ? I think this could be adding also an extra > 1000 cpu ms on every requests ... > > (by the way, i have tried the cookbook's memcache technique and it > only works locally, it fails on the google servers..) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: jinja2 environment loading on each request to /
I've been trying to figure this out too... See also this thread: http://stackoverflow.com/questions/618827/optimizing-jinja2-environment-creation On Mar 12, 7:38 pm, pedepy wrote: > hey so i use jinja2 templates because they are better [no citation > needed], but from what i understand, loading the jinja2 environment > can be quite an expensive task. Through logging, I have found out that > each request to / loads the environment over and over.. I have those > routines in a seperate python script (not the one that app.yaml > fowards requests to for /). > > anything im missing here ? I think this could be adding also an extra > 1000 cpu ms on every requests ... > > (by the way, i have tried the cookbook's memcache technique and it > only works locally, it fails on the google servers..) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
