Re: [Zope-CMF] Re: CachingPolicyManager improvements
On 2 Sep 2005, at 23:49, Geoff Davis wrote: On Fri, 02 Sep 2005 23:08:37 +0100, Jens Vagelpohl wrote: IMHO there is at least one problematic item (well, it's not for the trunk, but for the 1.5/1.4 branches), which is the dependency on ZopeTestCase. The way the code works right now you immediately prevent any tests from running on platforms that do not have ZopeTestCase installed - it blows up the testrunner. That needs to degrade a bit more gracefully in the absence of ZopeTestCase. So you would be OK if the tests just failed if ZopeTestCase were not installed? IMHO they should not fail, they just should not be run. People with a "normal" CMF 1.5-based setup and without ZopeTestCase should not see failing tests because another third-party package is not installed. That's confusing. jens ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements
On 5 Sep 2005, at 11:45, Florent Guillaume wrote: However this kind of default fixture actually is quite heavy, and makes you write tests that are hardly "unit" anymore. They are still good integration tests, but I've found that using PortalTestCase or something like that is often extremely costly in terms of time spent running the tests. That's very true. In the project that I'm working on we happen to use Plone, and almost all tests are using PloneTestCase (well, something subclassed from it). The tests are painfully slow. The best situation is a happy medium where you're only ever allowed to use the 100 ton fixture if you *really really really* need to and you cannot do it in a cheaper fixture. jens ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements
yuppie wrote at 2005-9-5 14:06 +0200:
> ...
>PortalTestCase is in the wrong layer. It makes assumptions about the way
>CMF works. Changes to the CMF might break PortalTestCase and create a
>dependency on a new Zope release. (We saw that problem already with the
>_refreshSkinData changes in CMF 1.5)
It is not better to fix such problems at a single place ("PortalTestCase")
rather than in lots of individual tests (that happen to make similar
assumptions)?
--
Dieter
___
Zope-CMF maillist - Zope-CMF@lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf
See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements
On Monday 05 September 2005 09:27 am, Geoff Davis wrote: > On Mon, 05 Sep 2005 18:15:41 +0200, Florent Guillaume wrote: > >>> If an FSPageTemplate is associated with a Caching Policy and that > >>> Caching Policy has 304s explicitly enabled, a series of checks take > >>> place. If there is an If-Modified-Since header, the server checks the > >>> modification time associated with the template via the Caching Policy. > >>> If there is an If-None-Match header and the Caching Policy defines an > >>> ETag function, the ETag is checked. If all the checks pass, the server > >>> returns a 304 status and stops without rendering the page. The server > >>> does less computation, and less data goes over the wire, so it's a > >>> double win. > >> > >> If I'm not mistaken, this will only be useful for ZPTs that provide > >> unchanged content over time ? Like css or scripts or invariant > >> resources, i.e. not useful for the usual case of views ? > > If you write your ETags in an appropriate way, this works beautifully for > views. Consider an ETag that consists of a string containing (1) the > content object's modification date, (2) the user name for the currently > authenticated user, and (3) the current time rounded to the nearest > hour. If a user gets this content while logged in, the ETag will match > (1) when s/he is logged in, (2) if the content has not changed, and (3) if > the cached copy is less than an hour old. You get cached, personalized > content that is guaranteed not to be stale and has an expiration time > that you control. Very good stuff. Except for any dynamic portlets on the page, but that's what ESI is for I guess. AJAX loaded content areas (which themselves could be cached with an ETag) would help here as well. Alec ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements
> On 6 Sep 2005, at 15:24, Alexander Limi wrote: >> On Tue, 06 Sep 2005 12:21:07 +0200, Florent Guillaume >> <[EMAIL PROTECTED]> wrote: >>> Alexander Limi wrote: And the biggest etc: RAMCache does not know how to handle anything but 7-bit ASCII data. >>> >>> What? There's no such limitation. RAMCacheManager caches arbitrary >>> python objects in memory. Unless you're thinking of the fact that >>> it doesn't cache REQUEST headers, but that's not its goal. >>> >> >> Try caching a content object in UTF-8 with it, and you'll see what I >> mean (yes, this is not what it's designed for, I know :). Ah, I think I see what you mean: In the suggested case, since the cached object returns only a fragment of XHTML, the encoding would be "out-of-band" data, which RAMCacheManager does not support. You're right, that can't work. -PW ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements
Florent Guillaume wrote at 2005-9-6 15:30 +0200:
> ...
>Of course if you directly serve content objects from the cache, which
>is explicitely unsupported and not recommended...
Where did you find this information ("*explicitely* unsupported").
--
Dieter
___
Zope-CMF maillist - Zope-CMF@lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf
See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements
On 6 Sep 2005, at 20:05, Dieter Maurer wrote:
Florent Guillaume wrote at 2005-9-6 15:30 +0200:
...
Of course if you directly serve content objects from the cache, which
is explicitely unsupported and not recommended...
Where did you find this information ("*explicitely* unsupported").
When you configure a RAMCacheManager, the dtml page says:
The RAM Cache Manager allows you to cache the result of
calling expensive objects, such as Python Scripts and External
Methods, in memory. Because it does *not* cache HTTP headers,
caching full web pages is generally not advised.
(Added by myself last november because I'd been bitten by it.)
There's also some text in the online help about caching complete web
pages being a bad idea.
Florent
--
Florent Guillaume, Nuxeo (Paris, France) CTO, Director of R&D
+33 1 40 33 71 59 http://nuxeo.com [EMAIL PROTECTED]
___
Zope-CMF maillist - Zope-CMF@lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf
See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements
Florent wrote: > There's also some text in the online help about caching complete web > pages being a bad idea. And in the docstring of the source too. -PW ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements
This is correct for the default configuration. I deemed CMF site creation fast enough at that time. In PloneTestCase, for example, we instantiate a single portal at import time and reuse it for every test. In the end it is a simple matter of overriding getPortal (which is a hook that exists for this very purpose). Note that I plan to reevaluate PortalTestCase/CMFTestCase in the light of CMF 1.5+ now that my Plone workload has eased a bit. Stefan On 8. Sep 2005, at 14:30, Florent Guillaume wrote: ZopeTestCase is ok. It's PortalTestCase and the like that are slow, because they instantiate a full portal. Florent -- Anything that happens, happens. --Douglas Adams ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements
PortalTestCase is actually only intended as a base class for CMFTestCase/PloneTestCase/CPSTestCase/whatever, not for direct use. It provides some default behavior, because, for example, the requirement to refresh the skin cache is pretty hard to figure out. The ZopeTestCase philosophy is about convenience and "plumbing included". I am committed to keeping things up-to-date. Please make me aware of breakage, I am going to address everything that might come up. Stefan On 5. Sep 2005, at 14:06, yuppie wrote: PortalTestCase is in the wrong layer. It makes assumptions about the way CMF works. Changes to the CMF might break PortalTestCase and create a dependency on a new Zope release. (We saw that problem already with the _refreshSkinData changes in CMF 1.5) -- Anything that happens, happens. --Douglas Adams ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements
Well, actually it is not correct ;-). PortalTestCase does *not* provide a portal object but expects subclasses to take care of that part of the fixture. Stefan On 9. Sep 2005, at 10:34, Stefan H. Holek wrote: This is correct for the default configuration. -- Anything that happens, happens. --Douglas Adams ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements
> Hi Stefan!
>
>
> Stefan H. Holek wrote:
>> Well, you are supposed to use a subclass anyway, and in that subclass
>> you can fix everything, even _refreshSkinData.
>
> Sure there are ways to work around the problems. But nevertheless Zope
> should not contain code that requires knowledge about the CMF.
Yes! Maybe PortalTestCase.py should be moved into CMFTestCase?
Certainly by zope 2.9 I think we should not ship PortalTestCase with
Zope. Maybe in the next 2.8 release, ZopeTestCase/PortalTestCase.py
could give a deprecation warning at import time, like:
warnings.warn("ZopeTestCase will no longer include PortalTestCase as of "
"Zope 2.9. Tests should be migrated to use CMFTestCase "
"instead.", DeprecationWarning)
or whatever the policy will be.
>> Maybe I should make
>> PortalTestCase truly abstract to emphasize the point.
>> NotImplementedError anyone? :-)
>
> What ever helps to convince Sidnei and Geoff not to check in tests that
> use PortalTestCase directly ;)
Indeed, this point is not clear in the PortalTestCase docs.
I (perhaps foolishly) wrote a bunch of tests like so:
class MyTestCase(ZopeTestCase.PortalTestCase):
def getPortal(self):
# see PortalTestCase. This must be defined to create the portal.
name = ZopeTestCase.portal_name
manage_addCMFSite(self.app, name)
return self.app[name]
...
Which, perhaps by luck, worked just fine in fifty tests in two products;
but failed (BadRequest because the test fixture stuck around between tests)
in the product I'm working on this week. I never did figure out what
was significantly different about that one. I banged my head on fiddling
with transactions, sandboxes, beforeTearDown(), beforeClose(), etc.
for many hours before a suggestion led me to try CMFTestCase and only
create the portal once. Problem solved.
-PW
___
Zope-CMF maillist - Zope-CMF@lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf
See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements
Tres Seaver wrote: Note that the goal with unit tests is to use the *smallest possible fixture*; using PortalTestCase when you could equally well use RequestTest reduces the usefulness of the test: - it runs slower (the "nibbled to death by ducks" effect, particularly when re-running the tests frequently, as we all should). - it may mask undesirable dependencies (CMFCore is not supposed to depend on CMFDefault, for instance, even in testing). - it *may* make the tests themselves obscure, because the actual behavior being tested is tied up in lots of "extraneous" bookkeeping. Hear, hear! (to all of the above) I spent a lot of time long ago making sure CMF had UNIT as opposed to functional tests. PortalPloneTestCase doesn't even try, thanks to the massive dependencies that Plone introduces, so you end up trying to run functional tests all the time which is just too slow. Ask Jens about the project we both work on that now has about 700 tests base on PloneTestCase and how pitifully slow it is, and how we spend a lot of time tracking down problems caused by the huge overhead to having to get Plone fixtures set up "just right" so that nothing explodes to often... cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements checked in to svn
On 8 Sep 2005, at 11:08, yuppie wrote: - Please make sure your checkins show up on the CMF-checkins list. Don't know if Tres can fix that for you or if you've got to register for that list. I was wondering why I did not get any mail... Geoff, can you double- check that the email address you have in your zope.org membership is correct and then subscribe to the cmf-checkins list? That should make any further check-in messages appear for us. PS Those of you who raised concerns about ZopeTestCase causing test problems in Zope 2.7 might want to take a look at why test_z3interfaces is broken when you run the tests with zopectl test. 1.) Why "those of you who raised concerns about ZopeTestCase"? I raised a concern that Zope 2.7 does not come with ZopeTestCase, so the test that uses it should not raise an error or failure condition. There should not be any failures when you download a vanilla Zope 2.7.x and CMF 1.5.x and run the unit tests, unless there is a real software bug. jens ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements checked in to svn
On 8 Sep 2005, at 16:48, Geoff Davis wrote: bin/zopectl test --dir Products/CMFCore/tests I get a bunch of errors like the following: == ERROR: test_z3interfaces (CMFCore.tests.test_ActionInformation.ActionInfoTests) -- Traceback (most recent call last): File "/home/zope/plonedev/Products/CMFCore/tests/ test_ActionInformation.py", line 60, in test_z3interfaces from Products.CMFCore.interfaces import IActionInfo ImportError: cannot import name IActionInfo I don't see IActionInfo defined anywhere in interfaces, so I don't think it's just me. All those tests run fine for me using your branch and a vanilla Zope 2.7.6. I'd make a wild guess and say it's your setup. jens ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements checked in to svn
| The z3interfaces tests are based on the assumption that Five is | available if zope.interface is available. Five creates IActionInfo | dynamically on startup. | | That's obviously not true in your setup. Looking again at these tests it | would be more robust to include the interface imports in the try/except | ImportError statement. For the records, I had other tests failing for the same reason in PAS. It's common to have zope.interface but not five if you install twisted in Ubuntu it has a dependency on the python-zope-interface package or something. -- Sidnei da Silva Enfold Systems, LLC. http://enfoldsystems.com ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements checked in to svn
On 9 Sep 2005, at 15:10, yuppie wrote: Geoff Davis wrote: On Fri, 09 Sep 2005 11:33:53 +0200, yuppie wrote: Are you aware of the fact that test_Template304Handling.py depends on PortalTestCase? Do you plan to change that before merging your branch? Yes, I am aware of that fact. No, I am not planning to change it -- it works just fine. I am sure that if the CMF changes as substantially as you are worried about, there will be numerous things that will need to change. I'm not sure that it is time well spent trying to fix things that may or may not happen at some unspecified point in the future. However, if you feel sufficiently strongly about it, the code is in SVN and you are more than welcome to work on it. I'm still -1 on merging the branch as it is and Stefan confirmed that PortalTestCase should not be used directly: http:// mail.zope.org/pipermail/zope-cmf/2005-September/022932.html How about a compromise: I'll spend a little time tomorrow rewriting that test module so it does not use ZopeTestCase at all. The changes on this branch are good and valuable, and a final decision and implementation of the extended testing fixtures problem will probably take a while... jens ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements checked in to svn
On Fri, Sep 09, 2005 at 04:33:48PM +0100, Jens Vagelpohl wrote: | >I'm still -1 on merging the branch as it is and Stefan confirmed | >that PortalTestCase should not be used directly: http:// | >mail.zope.org/pipermail/zope-cmf/2005-September/022932.html | | How about a compromise: I'll spend a little time tomorrow rewriting | that test module so it does not use ZopeTestCase at all. | | The changes on this branch are good and valuable, and a final | decision and implementation of the extended testing fixtures problem | will probably take a while... I compromise in backporting your changes to 1.4 branch. -- Sidnei da Silva Enfold Systems, LLC. http://enfoldsystems.com ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: CachingPolicyManager improvements checked in to svn
Here's the result of my refactoring/rewriting for the tests in question: http://svn.zope.org/CMF/branches/geoffd-cachingpolicymanager-branch/? rev=38439&view=rev Geoff, as the one with the most "domain knowledge" as far as the code changes go, could you check the tests (and especially the comments I have added) to make sure they make sense? Flames welcome from everyone else as well, of course ;) jens ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Sane checkin mail delivery (was: Re: [Zope-CMF] Re: CachingPolicyManager improvements checked in to svn)
Jens Vagelpohl wrote at 2005-9-8 11:41 +0100: > >On 8 Sep 2005, at 11:08, yuppie wrote: > ... >Geoff, can you double- >check that the email address you have in your zope.org membership is >correct and then subscribe to the cmf-checkins list? That should make >any further check-in messages appear for us. I have seen such a wish several times. Would it not be more sane to fix the checkin mail delivery such that it does not require the checking in person to have a subscription? Someone making occational changes may not be interested to be informed about all the modifications going on. -- Dieter ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: Sane checkin mail delivery (was: Re: [Zope-CMF] Re: CachingPolicyManager improvements checked in to svn)
Dieter Maurer wrote: > Someone making occational changes may not be interested > to be informed about all the modifications going on. +1. -PW ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
