Hi Thorsten, is this working for you locally? I noticed the Forrestbot failure just now and I'm also failing locally (with a NPE) on a dispatcher sample site. Was thinking it might be related but don't see anything obvious in here...
--tim On Mon, May 17, 2010 at 1:42 PM, <thors...@apache.org> wrote: > Author: thorsten > Date: Mon May 17 17:42:05 2010 > New Revision: 945269 > > URL: http://svn.apache.org/viewvc?rev=945269&view=rev > Log: > FOR-1194 Fixing utf-8 compability by forcing to use UTF-8 in every step > > Modified: > > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap > > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispatcher-sitemapcomponents.xconf > > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java > > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.java > > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTransformer.java > > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapperTransformer.java > > Modified: > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap > URL: > http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap?rev=945269&r1=945268&r2=945269&view=diff > ============================================================================== > --- > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap > (original) > +++ > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap > Mon May 17 17:42:05 2010 > @@ -22,7 +22,7 @@ xmlns:map="http://apache.org/cocoon/site > <map:pipeline id="lm"> > <map:match pattern="locationmap.xml"> > <map:generate src="locationmap.xml" /> > - <map:serialize type="xml" /> > + <map:serialize/> > </map:match> > </map:pipeline> > <map:pipeline id="dispatcher"> > @@ -61,7 +61,7 @@ xmlns:map="http://apache.org/cocoon/site > </map:transform> > <map:transform src="lm://hooks-to-fo.xsl" /> > <map:transform src="lm://strip-dispatcher-remains-fo.xsl" /> > - <map:serialize type="xml" /> > + <map:serialize/> > </map:match> > <map:match pattern="**.prepare.dispatcher.css"> > <map:generate src="lm://resolve.structurer.{1}" type="jx"> > @@ -93,11 +93,11 @@ xmlns:map="http://apache.org/cocoon/site > <map:act type="locale"> > <map:match pattern="resolve.structurer.**"> > <map:generate src="lm://resolve.structurer.{1}" /> > - <map:serialize type="xml" /> > + <map:serialize/> > </map:match> > <map:match pattern="resolve.contract.*.**"> > <map:generate src="{lm:resolve.contract.{1}.{2}}" /> > - <map:serialize type="xml" /> > + <map:serialize/> > </map:match> > <map:match pattern="prepare.contract.*.**"> > <map:generate src="{lm:resolve.contract.{1}.{2}}" /> > @@ -105,7 +105,7 @@ xmlns:map="http://apache.org/cocoon/site > <map:transform type="i18n"> > <map:parameter name="locale" value="{../locale}" /> > </map:transform> > - <map:serialize type="xml" /> > + <map:serialize/> > </map:match> > </map:act> > </map:pipeline> > @@ -116,7 +116,7 @@ xmlns:map="http://apache.org/cocoon/site > <map:match pattern="prepare.panels.**"> > <map:generate src="{lm:resolve.panels.{1}}" /> > <map:transform src="{lm:root-strip.xsl}" /> > - <map:serialize type="xml" /> > + <map:serialize/> > </map:match> > </map:pipeline> > <map:pipeline> > @@ -126,7 +126,7 @@ xmlns:map="http://apache.org/cocoon/site > <map:parameter name="path" value="{1}.html" /> > <map:parameter name="theme" value="{global:dispatcher.theme}" /> > </map:transform> > - <map:serialize type="xml" /> > + <map:serialize/> > </map:match> > </map:pipeline> > <map:pipeline id="resources"> > > Modified: > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispatcher-sitemapcomponents.xconf > URL: > http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispatcher-sitemapcomponents.xconf?rev=945269&r1=945268&r2=945269&view=diff > ============================================================================== > --- > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispatcher-sitemapcomponents.xconf > (original) > +++ > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispatcher-sitemapcomponents.xconf > Mon May 17 17:42:05 2010 > @@ -28,7 +28,7 @@ limitations under the License. > <cache-at-startup>true</cache-at-startup> > </map:transformer> > </map:transformers> > - <map:serializers default="xhtml"> > + <map:serializers default="xml-utf8"> > <map:serializer logger="sitemap.serializer.xhtml" > mime-type="text/html; charset=UTF-8" name="xhtml" ppool-max="64" > src="org.apache.cocoon.serialization.XMLSerializer"> > @@ -57,5 +57,9 @@ limitations under the License. > <encoding>UTF-8</encoding> > <indent>true</indent> > </map:serializer> > - </map:serializers> > + <map:serializer name="xml-utf8" mime-type="text/xml" > + src="org.apache.cocoon.serialization.XMLSerializer"> > + <encoding>UTF-8</encoding> > + </map:serializer> > + </map:serializers> > </map:components> > > Modified: > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java > URL: > http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java?rev=945269&r1=945268&r2=945269&view=diff > ============================================================================== > --- > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java > (original) > +++ > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java > Mon May 17 17:42:05 2010 > @@ -21,6 +21,8 @@ import java.io.BufferedOutputStream; > import java.io.ByteArrayInputStream; > import java.io.ByteArrayOutputStream; > import java.io.InputStream; > +import java.io.OutputStreamWriter; > +import java.io.UnsupportedEncodingException; > import java.util.Map; > > import javax.xml.stream.XMLStreamException; > @@ -105,17 +107,23 @@ public class XSLContract extends Loggabl > } > } > ByteArrayOutputStream out = new ByteArrayOutputStream(); > + byte[] bytes = null; > + String utf8 = ""; > // create a StreamResult and use it for the transformation > - Result streamResult = new StreamResult(new BufferedOutputStream(out)); > try { > - helper.transform(dataStream,streamResult); > + OutputStreamWriter writer = new OutputStreamWriter(out,"UTF-8"); > + Result streamResult = new StreamResult(writer); > + helper.transform(dataStream,streamResult); > + utf8 = out.toString("UTF-8"); > + log.debug(utf8); > + bytes = utf8.getBytes("utf-8"); > } catch (Exception e) { > String message = "Could not invoke the transformation for " > + "the contract \""+name+"\". "+"\n"+ e; > throw new ContractException(message); > } > - log.debug(out.toString()); > - return new BufferedInputStream(new > ByteArrayInputStream(out.toByteArray())); > + > + return new BufferedInputStream(new ByteArrayInputStream(bytes)); > } > > /* (non-Javadoc) > > Modified: > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.java > URL: > http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.java?rev=945269&r1=945268&r2=945269&view=diff > ============================================================================== > --- > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.java > (original) > +++ > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.java > Mon May 17 17:42:05 2010 > @@ -20,6 +20,7 @@ import java.io.BufferedInputStream; > import java.io.ByteArrayInputStream; > import java.io.IOException; > import java.io.InputStream; > +import java.io.InputStreamReader; > import java.io.PrintWriter; > import java.util.Iterator; > import java.util.Map; > @@ -124,6 +125,7 @@ public class XSLContractHelper extends L > SAXException, IOException { > // prepare transformation > transformer = transFact.newTransformer(xslSource); > + transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8"); > transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes"); > transformer.setOutputProperty(OutputKeys.INDENT, "yes"); > transformer.setOutputProperty(OutputKeys.METHOD, "xml"); > @@ -220,8 +222,8 @@ public class XSLContractHelper extends L > } > // Setting the necessary attributes of the contract > // xsl > - contract.setXslSource(new StreamSource(new ByteArrayInputStream(handler > - .getBytes()))); > + byte[] bytes = handler.getBytes(); > + contract.setXslSource(new StreamSource(new ByteArrayInputStream(bytes))); > /* DEBUG_CODE: > * The following is useful to see what is going on > * > @@ -254,7 +256,9 @@ public class XSLContractHelper extends L > throws ContractException { > //Source dataSource = new StreamSource(dataStream); > try { > - SAXSource saxSource = new SAXSource(xmlReader,new > InputSource(dataStream)); > + InputSource inputSource = new InputSource(new > InputStreamReader(dataStream, "UTF-8")); > + inputSource.setEncoding("UTF-8"); > + SAXSource saxSource = new SAXSource(xmlReader,inputSource); > transformer.transform(saxSource, streamResult); > } catch (Exception e) { > String message = "The xsl transformation has thrown an exception. for " > > Modified: > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTransformer.java > URL: > http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTransformer.java?rev=945269&r1=945268&r2=945269&view=diff > ============================================================================== > --- > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTransformer.java > (original) > +++ > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTransformer.java > Mon May 17 17:42:05 2010 > @@ -20,6 +20,7 @@ import java.io.BufferedInputStream; > import java.io.File; > import java.io.IOException; > import java.io.InputStream; > +import java.io.InputStreamReader; > import java.io.Serializable; > import java.io.StringReader; > import java.util.HashMap; > @@ -490,7 +491,7 @@ public class DispatcherTransformer exten > throw new SAXException(e); > } > if (null != property) { > - localParams.put(currentProperty, property.getBytes()); > + localParams.put(currentProperty, property.getBytes("UTF-8")); > } > } > } > @@ -695,7 +696,7 @@ public class DispatcherTransformer exten > root.serialize(out); > } > > - InputSource is = new InputSource(new StringReader(out.toString())); > + InputSource is = new InputSource(new > StringReader(out.toString("UTF-8"))); > // adding the result to the consumer > parser.parse(is, super.xmlConsumer); > } catch (Exception e) { > @@ -1099,8 +1100,9 @@ public class DispatcherTransformer exten > */ > private static InputSource getInputSource(final Source source) > throws IOException, SourceException { > - final InputSource newObject = new InputSource(source.getInputStream()); > - newObject.setSystemId(source.getURI()); > - return newObject; > + final InputSource inputSource = new InputSource(new > InputStreamReader(source.getInputStream(), "UTF-8")); > + inputSource.setEncoding("UTF-8"); > + inputSource.setSystemId(source.getURI()); > + return inputSource; > } > } > > Modified: > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapperTransformer.java > URL: > http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapperTransformer.java?rev=945269&r1=945268&r2=945269&view=diff > ============================================================================== > --- > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapperTransformer.java > (original) > +++ > forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapperTransformer.java > Mon May 17 17:42:05 2010 > @@ -374,7 +374,7 @@ public class DispatcherWrapperTransforme > } > // get the result of the structurer as stream > InputStream result = structurer.execute(new BufferedInputStream( > - new ByteArrayInputStream(document.getBytes())), requestedFormat); > + new ByteArrayInputStream(document.getBytes("UTF-8"))), > requestedFormat); > // requesting a parser > parser = (SAXParser) manager.lookup(SAXParser.ROLE); > // adding the result to the consumer > > >
