Re: lastModificationDate misuse? - old thread - progress
Carsten Ziegeler wrote:
> The "S-xml-1_=NOP Validity" is appended by the xml serializer, so
> in fact the cached entry with the "S-xml-1_=NOP Validity" is for
> a complete pipeline with serializer. I'm still wondering
> why this is the value for prev. For which internal pipeline
> does this happen?
Any/every pipeline that builds up the xsp.
I created a very simple pipeline (see below) to test and I got the same
problem, ie "S-xml-1_=NOP Validity" appearing in the prev. I cannot find
the code appends the "S-xml-1_=NOP Validity" as follows:
prev: {G-file--7461341140568259924_=TimeStamp Validity[1060277818000],
T-xslt-2097470703455153761_=TimeStamp Validity[1062494248000], S-xml-1_=NOP
Validity}
curr: {G-file--7461341140568259924_=TimeStamp Validity[1060277818000],
T-xslt-2097470703455153761_=TimeStamp Validity[1062494248000]}
>From sitemap
my serialiser is: org.apache.cocoon.serialization.XMLSerializer
my serverpages generator is:
org.apache.cocoon.generation.ServerPagesGenerator
>From .xconf
my event-pipeline
class="org.apache.cocoon.components.pipeline.CachingEventPipeline
Re: lastModificationDate misuse? - old thread - progress
Carsten Ziegeler wrote:
> > Inspecting the contents of the currValidity and prevValidity Maps I find
> > that they are equivalent excepting that the prevValidity contains an
entry
> > "S-xml-1_=NOP Validity" so the test fails and code is
> > re-generated.
>
> Huh! That's strange, now as far as I remember the code, the prevValidity
> should
> not contain "S-xml-1_=NOP Validity", as this is the validity for the
> serializer
> which is not used for internal pipeline calls. Hmm.
I assume you mean src="cocoon://foo" calls and not internal-only pipelines,
ie one with internal-only="true".
I have a fairly deep pipeline underlying the generated XSP "file" from which
.java is generated. Maybe this depth exposes a problem in discarding
NOPCacheValidity for internal pipeline calls. Can you see from the following
which class is adding the "S-xml-1_=NOP Validity", I will then have a poke
around/debug.
Below are some details of the case causing the problems:
create-categories.xsp - which has the "S-xml-1_=NOP Validity" in the
prevValidity is:
{
aggregation of {
aggregation of {tableDefs/categories.xml,
appDef/categories/UISpecifics.xml and appDef/terminologyMappings.xml}
transformed with stylesheets/buildUIStructure.xsl
and
appDef/categories/aCustomisation.xml
}
transformed with stylesheets/applyCustomisation.xsl
}
transformed with stylesheets/create.xsl
This is what validities look like: (equal besides the "S-xml-1_=NOP
Validity")
prev: {G-file-7747944243312186483_=TimeStamp Validity[1062058603973],
T-xslt--3690387136403106359_=Aggregated Validity[TimeStamp
Validity[106198276]:TimeStamp Validity[1061976654000]], S-xml-1_=NOP
Validity}
curr: {G-file-7747944243312186483_=TimeStamp Validity[1062058603973],
T-xslt--3690387136403106359_=Aggregated Validity[TimeStamp
Validity[106198276]:TimeStamp Validity[1061976654000]]}
Relevant part of the sitemap is as follows:
Re: lastModificationDate misuse? - old thread - progress
Joerg Heinicke wrote
> Bug was fixed some days ago:
> http://nagoya.apache.org/bugzilla/show_bug.cgi?id=14348. Carsten
> requested for testers of the patch, so if you can test the behaviour
> with the current Cocoon 2.0 CVS it will be good.
> > If XSP is the result of a pipeline then the ServerPages Generator can't
> > work out its currency and randomly re-generates and compiles. To make
> > I have inspected the code of 2.0.4 and it is still there, ie code is
> > same as when reported in 2001. Surely a typical use of cocoon is to
> > build XSP using XSLT?
I have tested (with cocoon-2.0_20030812221701) and believe the problem has
only been partially solved. Instead of the prior situation where the .java &
.class were re-generated from XSP on a random basis - they now are always
regenerated. This is at least consistent - but very wasteful on resources &
latency in requests.
I have investigated the source and found that the test as to whether or not
regeneration takes place is:
!currValidity.equals(prevValidity)
Inspecting the contents of the currValidity and prevValidity Maps I find
that they are equivalent excepting that the prevValidity contains an entry
"S-xml-1_=NOP Validity" so the test fails and code is re-generated. However,
it appears that the "NOP Validity" is simply indicating an always valid
condition - so code should not be re-generated.
I have added some code to strip the NOPCacheValidity entries and do a
comparison and my problem is solved - ie code is only regenerated when the
cached objects are out-of-date.
I have pasted the diff of my patch at the end of this message. I think the
bug http://nagoya.apache.org/bugzilla/show_bug.cgi?id=14348 should be
re-opened. Your views Joerg , Carsten?
John Williams
Suggested patch to SiteMapSource.java:
@@ -56,12 +56,15 @@
import java.io.InputStream;
import java.net.MalformedURLException;
import java.util.Map;
+import java.util.HashMap;
+import java.util.Iterator;
import org.apache.avalon.framework.component.ComponentException;
import org.apache.avalon.framework.component.ComponentManager;
import org.apache.cocoon.ProcessingException;
import org.apache.cocoon.Processor;
import org.apache.cocoon.caching.PipelineCacheKey;
+import org.apache.cocoon.caching.NOPCacheValidity;
import org.apache.cocoon.components.CocoonComponentManager;
import org.apache.cocoon.components.EnvironmentStack;
import org.apache.cocoon.components.pipeline.CacheableEventPipeline;
@@ -322,7 +325,7 @@
Map currValidity = cep.generateValidity(this.environment);
- if (prevValidity == null || !currValidity.equals(prevValidity)) {
+ if (prevValidity == null || !equalsIgnoreNOPCacheValidities(currValidity,
prevValidity)) {
// validity changed, update cached validity, timestamp and last modified
this.lastModificationDate = System.currentTimeMillis();
obj = new Object[] {new Long (this.lastModificationDate), currValidity};
@@ -365,6 +368,29 @@
this.needsRefresh = false;
}
+ static boolean equalsIgnoreNOPCacheValidities (Map aMapWithNOPValidities,
Map anotherMapWithNOPValidities) {
+ return
stripNOPValidities(aMapWithNOPValidities).toString().equals(stripNOPValiditi
es(anotherMapWithNOPValidities).toString());
+ }
+
+
+ static Map stripNOPValidities(Map withNOPValidities) {
+ if (withNOPValidities == null) return null;
+ else {
+ if (withNOPValidities.containsValue(NOPCacheValidity.CACHE_VALIDITY)) {
+ Map retVal = new HashMap();
+ Iterator iter = withNOPValidities.entrySet().iterator();
+ for (; iter.hasNext(); ) {
+ Map.Entry entry = (Map.Entry)iter.next();
+ if (!entry.getValue().equals(NOPCacheValidity.CACHE_VALIDITY))
+ retVal.put(entry.getKey(), entry.getValue());
+ }
+ return retVal;
+ }
+ else return withNOPValidities;
+ }
+ }
+
+
/**
* Return a new InputSource object
*/
lastModificationDate misuse? - old thread - progress
I have been mystified by the symptoms of the
problem raised by Stuart Roebuck some time ago on cocoon-dev and I can't see
if/when it has been fixed or a workaround/hack avail:
If XSP is the result of a pipeline then the
ServerPages Generator can't work out its currency and randomly re-generates and
compiles. To make matters worse if the XSP has the code necessary for resultant
objects to be cached then the code generated and compiled is not even used, ie a
cached object is used instead.
I have inspected the code of 2.0.4 and it is still there, ie code is
same as when reported in 2001. Surely a typical use of cocoon is to build
XSP using XSLT?
John
Stuart Roebuck wrote:>>On Tuesday, August 14, 2001, at
04:04 pm, Carsten Ziegeler wrote:> In
SitemapSource.java are the lines:>>> // the
event pipeline is cacheable // now calculate a last
modification date String hashKey = pck.toString() +
validity.toString(); this.lastModificationDate =
HashUtil.hash(hashKey);>> this is used in
ProgramGeneratorImpl.java:>>>
/* * FIXME: It's the program (not the instance) that
must * be queried for
changes!!! */ if (programInstance
!= null &&
programInstance.modifiedSince(source.getLastModified()))
{ // Release the
component.
release(programInstance);>> consequently, any use
of the ServerPages Generator which takes an input>>> from within
the sitemap (e.g. using a "cocoon:/" URI) will result>>> in (on
a>>> practically random basis) certain XSP (et. al) pages being
regenerated>>> over and over again, and will potentially prevent
updated XSP (et. al)>>> from being properly
regenerated.> Hm, that is true. The problem is that I
don't saw any other way to>> implement the lastModificationDate for
the SitemapSource than hashing>> the key.>> Usually this
modification date is used in the caching algorithm>> and is there only
tested against equal. So there this hack works. Any
solution for this?>>For the caching, I presume there is a mapping
of hashKeys to >serializedResults. If this is true, then couldn't
these also map to a >date-stamp of when the result was cached and have
lastModificationDate = >lastCachedDate?
