Re: [Zope-dev] RAMcache and container vs. context
Stefan H. Holek wrote at 2005-5-27 10:59 +0100: A TALES expression may be prohibitively expensive in any case, no matter how simple it is kept. Please make sure to do some comparative profiling. Cache keys are recomputed on every call of the script, AFAICS. The thought of doing this in restricted Python makes my skin crawl. But TALES expressions need not necessarily be restricted (see my TrustedExecutables). Though, I did not think about the security inplications when we would use unrestricted TALES expressions for a cache manager. It would probably be a bad idea. -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RAMcache and container vs. context
A TALES expression may be prohibitively expensive in any case, no matter how simple it is kept. Please make sure to do some comparative profiling. Cache keys are recomputed on every call of the script, AFAICS. The thought of doing this in restricted Python makes my skin crawl. Stefan On 26. Mai 2005, at 09:12, Chris Withers wrote: Paul Winkler wrote: I kinda wish the RAMCache manager allowed other things than REQUEST variables for differentiation. Arbitrary TALES expressions would be nice. Then we could get rid of this hard-coded gunk and get the same effect by having context/getPhysicalPath as one of the expressions. But of course then you'd have the opportunity to kill your cache performance by making the expressions too expensive. Ah well, TANSTAAFL and all that. I'll probably just drop the idea. I still think it's worth it. Crippling everyone on the basis that stupid people can hurt themselves is not good logic in my books ;-) If you could chuck this as a feature request in the collector and assign it to me, I'd love to work on it if I ever find the time... cheers, Chris -- Software Engineering is Programming when you can't. --E. W. Dijkstra -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RAMcache and container vs. context
Paul Winkler wrote: I kinda wish the RAMCache manager allowed other things than REQUEST variables for differentiation. Arbitrary TALES expressions would be nice. Then we could get rid of this hard-coded gunk and get the same effect by having context/getPhysicalPath as one of the expressions. But of course then you'd have the opportunity to kill your cache performance by making the expressions too expensive. Ah well, TANSTAAFL and all that. I'll probably just drop the idea. I still think it's worth it. Crippling everyone on the basis that stupid people can hurt themselves is not good logic in my books ;-) If you could chuck this as a feature request in the collector and assign it to me, I'd love to work on it if I ever find the time... cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RAMcache and container vs. context
On Thu, May 26, 2005 at 09:12:30AM +0100, Chris Withers wrote: If you could chuck this as a feature request in the collector and assign it to me, I'd love to work on it if I ever find the time... Done. http://www.zope.org/Collectors/Zope/1795 -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RAMcache and container vs. context
Sorry for comming late into the fray :-)
Em Qui, 2005-04-21 às 19:46 -0400, Paul Winkler escreveu:
On Fri, Apr 22, 2005 at 12:27:18AM +0200, Stefan H. Holek wrote:
Note that aq_parent() gives you the URL parent, not the container. I
see no way around that as the return value of a script may well depend
on its context.
Yes, it may, agreed. Thanks much for pointing out the relevant
code, at least now I understand what's happening.
But I still would like to argue against this behavior:
There *is* an easy alternative, and that's to put one or more
of the many location-related request variables into the
cache manager's configuration.
This alternative won't help you when:
* the pythonscript is sensitive to the context AND
* it is called w/ 2 or more different contexts on the same request
An artificial example I can come up w/ from the top of my head:
Suppose you have a folder w/ an index_html that displays information
about it's subfolders that is calculated by an expensive script and you
want to cache the results of this script, you'd go:
ul
li tal:repeat=subfolder python: here.objectValues('Folder')
Folder span tal:content=subfolder/title_or_id contains
span tal:content=subfolder/expensively_calculate_shruberries
shruberries.
/li
/ul
Then you'd go and RAMCache expensively_calculate_shruberries
But if we implement the change you're suggesting, then this page would
list the same number of shruberries of the first subfolder for all of
them.
-1 from me
For a related annoyance, see:
http://www.zope.org/Collectors/CMF/343
This annoyance is indeed related, but the proper fix is for
FSPythonScript to have a ZCacheable_manage page that takes into account
the fact that it's usually part of a portal_skins setup and deal with
it. Alternativelly, portal_skins should provide the functionality of
expiring RAMCached subitems.
--
Leonardo Rochael Almeida [EMAIL PROTECTED]
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RAMcache and container vs. context
On Mon, Apr 25, 2005 at 03:19:42PM -0300, Leonardo Rochael Almeida wrote:
Sorry for comming late into the fray :-)
(snip)
This alternative won't help you when:
* the pythonscript is sensitive to the context AND
* it is called w/ 2 or more different contexts on the same request
An artificial example I can come up w/ from the top of my head:
Suppose you have a folder w/ an index_html that displays information
about it's subfolders that is calculated by an expensive script and you
want to cache the results of this script, you'd go:
ul
li tal:repeat=subfolder python: here.objectValues('Folder')
Folder span tal:content=subfolder/title_or_id contains
span tal:content=subfolder/expensively_calculate_shruberries
shruberries.
/li
/ul
Then you'd go and RAMCache expensively_calculate_shruberries
But if we implement the change you're suggesting, then this page would
list the same number of shruberries of the first subfolder for all of
them.
-1 from me
OK, and now that you've said all that, -1 from me as well;
I think the idiom you describe is very common.
That's why I like to discuss things on the list before filing
collector items :-)
I kinda wish the RAMCache manager allowed other things than
REQUEST variables for differentiation. Arbitrary TALES expressions
would be nice. Then we could get rid of this hard-coded gunk and
get the same effect by having context/getPhysicalPath as one
of the expressions.
But of course then you'd have the opportunity to kill your
cache performance by making the expressions too expensive.
Ah well, TANSTAAFL and all that. I'll probably just drop the idea.
For a related annoyance, see:
http://www.zope.org/Collectors/CMF/343
This annoyance is indeed related, but the proper fix is for
FSPythonScript to have a ZCacheable_manage page that takes into account
the fact that it's usually part of a portal_skins setup and deal with
it. Alternativelly, portal_skins should provide the functionality of
expiring RAMCached subitems.
Of those, I'd prefer the latter approach. Added this as a comment
to #343.
Thanks for the valuable input!
--
Paul Winkler
http://www.slinkp.com
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] RAMcache and container vs. context
I never noticed this before today, but apparently RAMCacheManager is sensitive to whether cached items are acquired from the context or from the container. (Zope 2.7.3 but afaict it should behave the same in any recent version). Is that intended behavior or is this a bug? Let's say I have a CMF site at http://foo:8080/TheSite. And a template at /TheSite/portal_skins/custom/template1 which looks in part like like: p tal:content=context/expensive_script.../p Let's say that expensive_script is also in portal_skins/custom and is associated with a RAMCacheManager. Let's say that there are *no* request variables configured for this RAMCacheManager, which would seem to indicate that we consider all calls to expensive_script equivalent regardless of the request. If I visit http://foo:8080/TheSite/page1, which page makes use of this template, and I reload several times, and look at the RAMCacheManager statistics view, I see several hits for /TheSite/expensive_script. Every time I reload, the hit count goes up by one. So far so good. But if I then visit another page, http://foo:8080/TheSite/page2, which uses the same template, I get a miss and the entry count goes up by one. Reloading then gives me a hit, presumably from the new entry. If I browse around the site, the entry count increments for every new page I hit. Remember that I have nothing configured in the REQUEST Variables field. So the script should be the same regardless of the request right? This was puzzling me until, on a hunch, I looked at the calling template and changed it from this: p tal:content=context/expensive_script.../p ...to this: p tal:content=container/expensive_script.../p And presto, I now get only one entry, and every page on the site gives me a hit from that one entry. Is this really intended behavior? It's not obvious to me from the code why this is happening (or why it might be desirable). The cache manager stores and looks up items by their getPhysicalPath(), so I don't see why context vs. container is relevant. If I'd wanted the URL to be relevant, I would have put that in the REQUEST Variables configuration. This could be a performance hit for the unwary ... aside from all the unnecessary misses, we'd be using RAM on the order of N * M * S where N is number of cached items and M is number of pages in the site, and S is average size of return value. We only should be using N * S. -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RAMcache and container vs. context
This is due to how Python Scripts compute their cache keys. The
relevant snippet from PythonScript._exec() is:
asgns = self.getBindingAssignments()
name_context = asgns.getAssignedName('name_context', None)
if name_context:
keyset[name_context] = aq_parent(self).getPhysicalPath()
Note that aq_parent() gives you the URL parent, not the container. I
see no way around that as the return value of a script may well depend
on its context.
HTH,
Stefan
On 21. Apr 2005, at 22:26, Paul Winkler wrote:
It's not obvious to me from the code why this is happening
(or why it might be desirable). The cache manager stores and looks
up items by their getPhysicalPath(), so I don't see why context vs.
container is relevant. If I'd wanted the URL to be relevant,
I would have put that in the REQUEST Variables configuration.
--
Software Engineering is Programming when you can't. --E. W. Dijkstra
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RAMcache and container vs. context
On Fri, Apr 22, 2005 at 12:27:18AM +0200, Stefan H. Holek wrote:
This is due to how Python Scripts compute their cache keys. The
relevant snippet from PythonScript._exec() is:
asgns = self.getBindingAssignments()
name_context = asgns.getAssignedName('name_context', None)
if name_context:
keyset[name_context] = aq_parent(self).getPhysicalPath()
Note that aq_parent() gives you the URL parent, not the container. I
see no way around that as the return value of a script may well depend
on its context.
Yes, it may, agreed. Thanks much for pointing out the relevant
code, at least now I understand what's happening.
But I still would like to argue against this behavior:
There *is* an easy alternative, and that's to put one or more
of the many location-related request variables into the
cache manager's configuration.
This has two advantages: it's more explicit, and control
is given to the site admin rather than hardcoding it.
The disadvantages I see to using my current
container workaround are:
* The templates are client code. Having to change potentially
multiple templates just to affect cacheing behavior of one script
violates the Don't Repeat Yourself principle.
* It's too surprising.
When reviewing code, the difference between context and container
brings many things to mind. Cache hit/miss ratio is not one of them.
When I stumbled on the workaround, if I hadn't bothered to
ask here but had merely committed the change to my templates
and moved on, I would have been programming by coincidence
(see The Pragmatic Programmer).
I'll probably remember it now, but it seems like kind of an
obscure guru-level thing for a template author to have to know,
especially if they are not even a developer and only know how to
write some simple TALES expressions.
* It's an implicit implementation detail. The difference in caching
behavior between context and container is AFAIK not documented
anywhere, not even in the source code (unless you argue that
the semantics of the lines you posted above count as documentation).
Anyway, I have no guarantee that future versions of zope
won't silently break it.
* There are other reasons I might be forced to use context.
For example, if my site offers two CMF skins, each of which
provides some scripts with the same names, I might have common
templates that need to acquire the script from the current
skin and thus may not be able to find it via container.
admittedly there are hacky workarounds in such situations,
e.g. have trivial wrapper scripts live near the templates.
For a related annoyance, see:
http://www.zope.org/Collectors/CMF/343
--
Paul Winkler
http://www.slinkp.com
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
