RE: caching proposals: SQLTransformer and Request
I just skimmed through the mailing list to find ideas of how to cache a pipeline starting with a request generator and later on passing data through the SQLTransformer. Did you or anybody else follow up on this idea? NB: Caching of the request generator would probably also need to cache request parameters passed not part as part of the URL. As usually only some request parameters are used in the pipeline, the sitemap element might list the request parameters to consider when generating cache key: map:generator type=request map:parameter name=request-data-used value=input1 map:parameter name=request-data-used value=input2 ... - A couple of things I'd like to do with Cocoon caching; let me know if this is crazy. 1. Add caching to the request generator. Many of my pipelines are transformations based upon the request, and since requestGenerator currently does not support caching, it means those transformations are always re-done (and often there is some sql at the end of the pipe which is really slow). I'd like to hash (or MD5?) the request string and use that as the cache key so that if I get the same one the pipeline knows it can skip over everything. 2. Add caching to the SqlTransformer. I know this sounds weird, but hear me out. Our database is modified infrequently, so usually returns the same data. There is a datestamp in a special table which indicates when the last time the database was updated. The SQL Transformer would remember the time of the last query. I would have a new parameter to the sql transformer to indicate when the data is dirty. map:transform type=sql map:parameter name=last-update-time value=cocoon:/lastUpdateTime.xml/ /map:transform LastUpdateTime looks like: Date20021005144321/Date (Or whatever the xml date format is, I forget). SQLGenerator would resolve cocoon:/lastUpdateTime.xml. I would then have a pipeline for lastUpdateTime.xml which would build it by querying my special table (but if you wanted, you could use some other mechanism to build it). SQLGenerator would compare the two dates and re-run the sql if it needed to. How does this sound? Steve - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
AW: All hope lost: Can FragmentExtractor serve SVG images to IE (socket write error) ?
TK To my 2.0 times I also had the problem. The problem was gone since I use TK cocoon 2.1. I guess it has something to do with pipeline caching, but TK have never tested. Torsten, thanks for the quick feedback. My understanding was that 2.1 is still alpha. Do you have any system in production, yet? We are quite close to release date... Best regards, Christian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
AW: All hope lost: Can FragmentExtractor serve SVG images to IE (socket write error) ?
All of this works fine when requesting the page using Netscape Navigator (valid page and images, no errors in log file). As soon as I use the Microsoft Internet Explorer to request the same page, the HTML page is returned, but the svg image has not been served properly. According to the log file there was a java.net.SocketException: Software caused connection abort: socket write error (details included below), I remember that other software (Squeak) had a somewhat similar problem which was solved by changing something in the networking layer. The browsers were different regarding opening new connections. I think Netscape created one connection to download one file after another while MSIE created a connection for every single file on a webpage. Something like that. It took some time to fix that problem... Thank you, Andreas. I am afraid this is way too deep for my java skills. You don't have any source code files/ changes by any chance? Best regards, Christian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
All hope lost: Can FragmentExtractor serve SVG images to IE (socket write error) ?
Hi, has anybody successfully served svg images (not svg rendered as JPG or PNG, this just works great!) to the IE using the FragmentGenerator? I would really appreciate any information or code snippets about sitemap setup and versions used (OS, JDK, Tomcat, Cocoon, Xalan). After trying to fix what I though was a simple problem for 1.5 weeks I have got no ideas on how to investigate anymore: I am trying to serve a generated svg image as an svg file. The two pipelines are basically: FileGenerator XSLT-Transformer to turn xml data into a xml document containing Custom-Transformer for calling GraphViz to create SVG elements in xml document XSLT-Transformer FragmentExtractor XSLT-Tranformer HTML-Serializer FragmentGenerator SVGXML-Serializer All of this works fine when requesting the page using Netscape Navigator (valid page and images, no errors in log file). As soon as I use the Microsoft Internet Explorer to request the same page, the HTML page is returned, but the svg image has not been served properly. According to the log file there was a java.net.SocketException: Software caused connection abort: socket write error (details included below), in certain configurations I am receiving a NPE in DOMStreamer.stream instead (details included below). As discussed in other threads (http://marc.theaimsgroup.com/?l=xml-cocoon-devm=101994024218728w=2, http://marc.theaimsgroup.com/?l=xml-cocoon-usersm=103907382827236w=2), this looks like IE requesting the svg twice and aborting the first request causing the trouble. Is the FragmentExtractor able to handle this, if the serializer is not JPG or PNG? I am using WinXP and have tried various configurations: -cocoon (2.0.3, 2.0.4), -Tomcat (4.1.24, 4.1.18) -JDK (1.4.0_02, 1.4.1_02) -XALAN (2.3.1, 2.4.1, 2.5.1) and SAXON (6.5), -incremental processing turned on and off. Any hints, ideas on further research very much appreciated. Thanks in advance, Christian - SOCKET WRITE ERROR - FATAL_E (2003-05-28) 00:04.34:399 [core.xslt-processor] (/eti/etimdb/imsds-graph/e920f-f56ba6b144--8000.svg) Thread-10/TraxErrorHandler: java.net.SocketException: Software caused connection abort: socket write error javax.xml.transform.TransformerException: java.net.SocketException: Software caused connection abort: socket write error at org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.j ava:690) at org.apache.xalan.templates.ElemApplyTemplates.transformSelectedNodes(Ele mApplyTemplates.java:423) at org.apache.xalan.templates.ElemApplyTemplates.execute(ElemApplyTemplates .java:226) at org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(Trans formerImpl.java:2182) at org.apache.xalan.transformer.TransformerImpl.applyTemplateToNode(Transfo rmerImpl.java:2008) at org.apache.xalan.transformer.TransformerImpl.transformNode(TransformerIm pl.java:1171) at org.apache.xalan.transformer.TransformerImpl.run(TransformerImpl.java:31 35) at java.lang.Thread.run(Thread.java:536) Caused by: java.net.SocketException: Software caused connection abort: socket write error at org.apache.xalan.serialize.SerializerToXML.outputLineSep(SerializerToXML .java:199) at org.apache.xalan.serialize.SerializerToXML.indent(SerializerToXML.java:2 308) at org.apache.xalan.serialize.SerializerToHTML.startElement(SerializerToHTM L.java:630) at org.apache.xalan.transformer.TransformerIdentityImpl.startElement(Transf ormerIdentityImpl.java:1017) at org.apache.cocoon.xml.AbstractXMLPipe.startElement(AbstractXMLPipe.java: 130) at org.apache.cocoon.serialization.AbstractTextSerializer$NamespaceAsAttrib utes.startElement(AbstractTextSerializer.java:486) at org.apache.cocoon.xml.AbstractXMLPipe.startElement(AbstractXMLPipe.java: 130) at org.apache.xalan.transformer.ResultTreeHandler.flushElem(ResultTreeHandl er.java:856) at org.apache.xalan.transformer.ResultTreeHandler.flushPending(ResultTreeHa ndler.java:931) at org.apache.xalan.transformer.ResultTreeHandler.endElement(ResultTreeHand ler.java:306) at org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.j ava:684) ... 7 more - java.net.SocketException: Software caused connection abort: socket write error at org.apache.xalan.serialize.SerializerToXML.outputLineSep(SerializerToXML .java:199) at org.apache.xalan.serialize.SerializerToXML.indent(SerializerToXML.java:2 308) at org.apache.xalan.serialize.SerializerToHTML.startElement(SerializerToHTM L.java:630) at org.apache.xalan.transformer.TransformerIdentityImpl.startElement(Transf ormerIdentityImpl.java:1017) at org.apache.cocoon.xml.AbstractXMLPipe.startElement(AbstractXMLPipe.java: 130) at
Re: ServerPageAction: XMLFragment reuse in XSL transformer
Sorry for the delay! Thanks for your answer, Peter. I must admit that I did not compare the performance between the aggregation and the transformer approach. The aggregation is just one additional component in the pipeline, that has to process the big document. Apart from that the necessarykind of processing of the big document requires the XSL-transformer to build an entire DOMDocument before doing the transformations, I guess. So what I have done is I have written a Transformer for handling expandable data like the directory tree in the Windows' Explorer. It requires all input elements to have a unique id attribute. According to the tree status stored in the session, the transformer will only forward the visible tags and drop everything else. It also supports URL commands for setting upthe status tree (only root element is visible or all parent elements of one particular one are visible),expanding and collapsing child elements. Later transformers can then easily do thepresentation of the visible part of the tree and setup URL links e.g. for expanding leaf nodes futhermore. In most cases all SAX processing is done on-the-fly without intermediate storage of all SAX element. The quality of the code is probably not very high, as I am not working with Cocoon for very long, but at least its documented. Do you think this transformer could be useful for other people? And do you know to whom I should give it? Thank you and best regards, Christian - Original Message - From: Hunsberger, Peter To: '[EMAIL PROTECTED]' Sent: Monday, October 21, 2002 2:56 PM Subject: RE: ServerPageAction: XMLFragment reuse in XSL transformer That should work. I don't know ifit is much (any?) more efficient thanusing aggregation... -Original Message-From: Christian Kurz [mailto:[EMAIL PROTECTED]]Sent: Saturday, October 19, 2002 4:53 AMTo: [EMAIL PROTECTED]Subject: Re: ServerPageAction: XMLFragment reuse in XSL transformer I'll try to implement a pipeline similar to this: FileGenerator - "MyFilterTransformer" - other steps The FileGenerator will read the big document and is cacheable. MyFilterTransformer will use the XMLFragment from the session to forward only a small portion of the events from the generator, following your idea. MyFilterTransformer is the first component in the pipeline, which is NOT cacheable, as it returns different data on every request. To my understanding cocoon's caching strategy should cache the pipeline as far as possible. In this particular case only including the FileGenerator: "The keys of all cacheable components are chained, and together they build the cache key. The request is processed, and the document is built. The cache stores the result of the _last_ component, indicating cacheablility. The next time this document is requested, the key is built, and the cached content is fetched from the cache. Next, the cache asks all components of the event pipeline, if their input has changed since the time the content was cached. For example, the generator checks this by looking at the last modification date of the xml document, the xslt transformer checksthe date of the stylesheet, and so on. Only if all state that the content is still valid it is used from the cache. Otherwise, the document is generated from scratch. So the event pipeline tries to cache as much of the XML processing pipeline as possible." from New Riders, Cocoon: Building XML Applications by Matthew Langham and Carsten Ziegeler, p. 182. - Original Message - From: Hunsberger, Peter To: '[EMAIL PROTECTED]' Sent: Friday, October 18, 2002 4:23 PM Subject: RE: ServerPageAction: XMLFragment reuse in XSL transformer Well, obviously, it something changes on every request then, by definition, it isn't cacheable. However, I believe, with aggregation, only the components that are marked as invalidated in the cache are rebuilt on a new request; the rest of the data stays in it's respective cache. This does mean that the aggregated document is rebuilt each time (I have no idea what the impact of that would be). But the "large" document source would stay not be retrieved from scratch. Perhaps using the document function as Olivier would avoid this. At one point document did notcheck cache validity at all and you would only get a newer version of the data if the main data was invalidated in cache. I think this has been fixed in 2.0.3. The problem is how to get the URI resolve for the document function to fund the data in session. Someone else will have to tell you how to do
Re: Convert XMLFragment to DOM node (solved)
The problem was, that I hadn't sent the startDocument and endDocument events. - Original Message - From: Christian Kurz To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Monday, October 21, 2002 12:09 AM Subject: Re: Convert XMLFragment to DOM node I am experiencing the same problem. Have you been able to solve this problem? Any hints? Thanks in advance, Christian
Re: Convert XMLFragment to DOM node
I am experiencing the same problem. Have you been able to solve this problem? Any hints? Thanks in advance, Christian
Re: ServerPageAction: XMLFragment reuse in XSL transformer
Salut Olivier,
thank you for your help! That sounds like a quick and easy solution. I'd
just need to use url-encoded session-ids to pass the session id to the
cocoon:/ request and in order to do so, I need to get it from the request
object. I'll see whether this is easily possible.
Thanks,
Christian
- Original Message -
From: ROSSEL Olivier [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Friday, October 18, 2002 8:49 AM
Subject: RE: ServerPageAction: XMLFragment reuse in XSL transformer
May be you can try to import a XML fragment via the document() XSL
function.
Calling a cocoon:/ URL, you will get the XML fragment in a variable inside
your XSL.
Neat and very efficient !!!
-Message d'origine-
De: Christian Kurz [mailto:crkurz;gmx.de]
Date: vendredi 18 octobre 2002 08:42
À: [EMAIL PROTECTED]
Objet: Re: ServerPageAction: XMLFragment reuse in XSL transformer
Thank you very much for the quick feed-back! The idea sounds great and is
a
lot cleaner, than fiddling something in some XSL extension.
I am not sure about the cachaebility: the XMLFragment specifying, which
nodes to filter from the big input document, changes everytime, so Cocoon
would need to parse the source file on every request (, if my
understanding
is right).
If I'd slidely change your approach to implementing the same approach into
a
transformer component. This transformer component will not be cacheable,
but
at least the generator in front of it would be.
Thanks again,
Christian
BTW, thanks also for the code snippet. It helps a lot, as soon as it comes
to thinks like the ObjectModel, I start feeling uncomfortable.
- Original Message -
From: Hunsberger, Peter mailto:Peter.Hunsberger;stjude.org
To: '[EMAIL PROTECTED]' mailto:'[EMAIL PROTECTED]'
Sent: Thursday, October 17, 2002 6:46 PM
Subject: RE: ServerPageAction: XMLFragment reuse in XSL transformer
There's probably about half a dozen ways to do this. Perhaps one of the
simplest is just to create your own caching generator and use aggregation
(with any other XML you may need) in the pipeline.
In the generator you'll need to implement the setup method to see the
objectModel, something like the following:
private gunk mySessionData = null;
public void setup( SourceResolver resolver, Map objectModel, String src,
Parameters parms )
throws ProcessingException, SAXException, IOException
{
if (mySessionData == null ) {
super.setup( resolver, objectModel, src, parms );
Request request =
(Request)ObjectModelHelper.getRequest(objectModel);
Session session = request.getSession(false);
if (session != null) {
// save a pointer to your session data for use in the generate
method
mySessionData =
}
}
}
Now in your generate method just pick up whatever data hangs off of
mySessionData and away you go
-Original Message-
From: Christian Kurz [mailto:crkurz;gmx.de]
Sent: Thursday, October 17, 2002 11:26 AM
To: [EMAIL PROTECTED]
Subject: ServerPageAction: XMLFragment reuse in XSL transformer
Hello cocoon-users,
I need to generate some tiny XML elements (XMLFragment) within a
ServerPageAction and I would like to use this XMLFragment later on in an
XSL
transformer, that is fed by an xml generator. The XMLFragment captured in
the ServerPageAction is basically saying, which nodes are to be returned
from the big input document.
From some other message in this group I have understood, that passing
objects is only possible through session or request objects, but not
through
sitemap variables. I don't like to use a request generator as the starting
point of the pipeline, as I'd loose cacheability at a very early step in
the
pipeline. With a quite big xml input document, this does not seem a good
idea to me.
So I am currently struggling how to get a piece of XML, that is attached
to
a session or request object, into the xsl transformer. Has anybody tried
this before e.g. using an XSL extension?
Any help or hints appreciated!
Thank you in advance,
Christian
This e-mail is intended only for the above addressee. It may contain
privileged information. If you are not the addressee you must not copy,
distribute, disclose or use any of the information in it. If you have
received it in error please delete it and immediately notify the sender.
Security Notice: all e-mail, sent to or from this address, may be
accessed by someone other than the recipient, for system management and
security reasons. This access is controlled under Regulation of
Investigatory Powers Act 2000, Lawful Business Practises.
-
Please check that your question has not already been answered in the
FAQ before posting. http://xml.apache.org/cocoon/faq/index.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail
Re: ServerPageAction: XMLFragment reuse in XSL transformer
I'll try to implement a pipeline similar to
this:
FileGenerator -
"MyFilterTransformer" - other steps
The FileGenerator will read the big document and
is cacheable.
MyFilterTransformer will use the XMLFragment from
the session to forward only a small portion of the events from the generator,
following your idea. MyFilterTransformer is the first component in the pipeline,
which is NOT cacheable, as it returns different data on every
request.
To my understanding cocoon's caching strategy
should cache the pipeline as far as possible. In this particular case only
including the FileGenerator:
"The keys of all cacheable components are
chained, and together they build the cache key. The request is processed, and
the document is built. The cache stores the result of the _last_ component,
indicating cacheablility. The next time this document is requested, the key is
built, and the cached content is fetched from the cache.
Next, the cache asks all components of the event
pipeline, if their input has changed since the time the content was cached. For
example, the generator checks this by looking at the last modification date of
the xml document, the xslt transformer checksthe date of the stylesheet,
and so on. Only if all state that the content is still valid it is used from the
cache. Otherwise, the document is generated from scratch. So the event pipeline
tries to cache as much of the XML processing pipeline as possible."
from New Riders, Cocoon: Building XML
Applications by Matthew Langham and Carsten Ziegeler, p. 182.
- Original Message -
From:
Hunsberger, Peter
To: '[EMAIL PROTECTED]'
Sent: Friday, October 18, 2002 4:23
PM
Subject: RE: ServerPageAction:
XMLFragment reuse in XSL transformer
Well, obviously, it something changes on every request then, by
definition, it isn't cacheable. However, I believe, with aggregation,
only the components that are marked as invalidated in the cache are rebuilt on
a new request; the rest of the data stays in it's respective
cache. This does mean that the aggregated document is rebuilt each
time (I have no idea what the impact of that would be). But the "large"
document source would stay not be retrieved from scratch.
Perhaps using the document function as Olivier would avoid this.
At one point document did notcheck cache validity at all and you would
only get a newer version of the data if the main data was invalidated in
cache. I think this has been fixed in 2.0.3. The problem is how to get
the URI resolve for the document function to fund the data in session.
Someone else will have to tell you how to do that (and if it's even
possible)...
-Original
Message-From: Christian Kurz
[mailto:[EMAIL PROTECTED]]Sent: Friday, October 18, 2002 1:42
AMTo: [EMAIL PROTECTED]Subject: Re:
ServerPageAction: XMLFragment reuse in XSL transformer
Thank you very much for the quick feed-back!
The idea sounds great and is a lot cleaner,
than fiddling something in someXSL extension.
I am not sure about the cachaebility: the
XMLFragment specifying, which nodes to filter from the big input document,
changes everytime, so Cocoon would need to parse the source file on every
request (, if my understanding is right).
If I'd slidely change your approach to
implementing the same approach into a transformer component. This
transformer component will not be cacheable, but at least the generator in
front of it would be.
Thanks again,
Christian
BTW, thanks also for the code snippet. It helps
a lot, as soon as it comes to thinks like the ObjectModel, I start feeling
uncomfortable.
- Original Message -
From:
Hunsberger, Peter
To: '[EMAIL PROTECTED]'
Sent: Thursday, October 17, 2002 6:46
PM
Subject: RE: ServerPageAction:
XMLFragment reuse in XSL transformer
There's probably about half a dozen ways to do this. Perhaps
one of the simplest is just to create your own caching generator and use
aggregation (with any other XML you may need)in the
pipeline.
In the generator you'll need to implement the setup method to see
the objectModel, something like the following:
private gunk mySessionData = null;
public void setup( SourceResolver resolver, Map objectModel, String
src, Parameters parms ) throws
ProcessingException, SAXException,
IOException {
if (mySessionData == null )
{
super.setup(
resolver, objectModel, src, parms );
Request
request =
(Request)ObjectModelHelper.getRequest(objectModel);
Session session =
request.getSession(false);
if (session != null) {
// save a pointer to your sessio
Re: svg2jpeg error?
I had the same problem, I think it was related to running Batik on a headless system. There is more information at http://xml.apache.org/cocoon/installing/index.html, section UNIX with X server - Original Message - From: Harry J. Foxwell [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Saturday, October 19, 2002 11:44 PM Subject: svg2jpeg error? Running jakarta-tomcat-4.0.4, Solaris 9 (SPARC), Cocoon 2.03. when transforming xml doc to svg jpeg, svg2jpeg is not working, getting error in log file: java.lang.NoClassDefFoundError at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:130) at java.awt.GraphicsEnvironment.getLocalGraphicsEnvironment(GraphicsEnvironment .java:62) at sun.awt.font.FontResolver.init(FontResolver.java:46) at sun.awt.font.FontResolver.getInstance(FontResolver.java:150) at java.awt.font.TextLine.getFontAtCurrentPos(TextLine.java:984) at java.awt.font.TextLayout.standardInit(TextLayout.java:721) at java.awt.font.TextLayout.init(TextLayout.java:613) etc... any hints? (standalone xalan conversion of xml doc to svg produces apparently okay svg doc which displays correctly under Mozilla) Thanks! - Please check that your question has not already been answered in the FAQ before posting. http://xml.apache.org/cocoon/faq/index.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Please check that your question has not already been answered in the FAQ before posting. http://xml.apache.org/cocoon/faq/index.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: ServerPageAction: XMLFragment reuse in XSL transformer
Thank you very much for the quick feed-back!
The idea sounds great and is a lot cleaner, than
fiddling something in someXSL extension.
I am not sure about the cachaebility: the
XMLFragment specifying, which nodes to filter from the big input document,
changes everytime, so Cocoon would need to parse the source file on every
request (, if my understanding is right).
If I'd slidely change your approach to implementing
the same approach into a transformer component. This transformer component will
not be cacheable, but at least the generator in front of it would
be.
Thanks again,
Christian
BTW, thanks also for the code snippet. It helps a
lot, as soon as it comes to thinks like the ObjectModel, I start feeling
uncomfortable.
- Original Message -
From:
Hunsberger, Peter
To: '[EMAIL PROTECTED]'
Sent: Thursday, October 17, 2002 6:46
PM
Subject: RE: ServerPageAction:
XMLFragment reuse in XSL transformer
There's probably about half a dozen ways to do this. Perhaps one
of the simplest is just to create your own caching generator and use
aggregation (with any other XML you may need)in the
pipeline.
In
the generator you'll need to implement the setup method to see the
objectModel, something like the following:
private gunk mySessionData = null;
public void setup( SourceResolver resolver, Map objectModel, String
src, Parameters parms ) throws
ProcessingException, SAXException,
IOException {
if (mySessionData == null )
{
super.setup(
resolver, objectModel, src, parms );
Request request
=
(Request)ObjectModelHelper.getRequest(objectModel);
Session session =
request.getSession(false);
if (session != null) {
//
save a pointer to your session data for use in the generate
method
mySessionData =
}
} }
Now
in your generate method just pick up whatever data hangs off of
"mySessionData" and away you go
-Original Message-From:
Christian Kurz [mailto:[EMAIL PROTECTED]]Sent: Thursday, October 17,
2002 11:26 AMTo: [EMAIL PROTECTED]Subject:
ServerPageAction: XMLFragment reuse in XSL transformerHello cocoon-users,
I need to generate some tiny XML elements
(XMLFragment) within a ServerPageAction and I would like to use this
XMLFragment later on in an XSL transformer, that is fed by an xml generator.
The XMLFragment captured in the ServerPageAction is
basically saying, which nodes are to be returned from the big input
document.
From some other message in this group I have
understood, that passing objects is only possible through session or request
objects, but not through sitemap variables. I don't like to use a request
generator as the starting point of the pipeline, as I'd loose cacheability
at a very early step in the pipeline.With a quite bigxml input
document,this does not seem a good idea to me.
So I am currently struggling how to get a piece
of XML, that isattached to a session or request object, into the xsl
transformer. Has anybody tried this before e.g. using an XSL
extension?
Any help or hints appreciated!
Thank you in advance,
Christian
