Re: Very large Generator file
Hi Jeff, > > Thanks Huib, > I'm afraid my document is very interconnected. I'm coming to realize that > xslt has pretty much a hard limit on the size of files you can process with > it, i.e. if it can't fit into memory and you need the full power of xslt > you're pretty much out of luck. What about going to a 64 bit machine and > just allocating a huge amount of ram to Tomcat? Can this been done with > tomcat/cocoon? It works until XSLT needs more than the available RAM. I'd try STX (Joost) first, it works very well. The interconnectedness of your document might spoil it though. You have another option: store the XML in a database (BaseX or eXist for instance) and use XQuery. My guess is that could improve things as I imagine it is not necessary to have the entire document in memory since it is already addressable 'in the database'. YMMV. And then there's "The XML technology that they don't want you to know about" option (http://vtd-xml.sourceforge.net/). I haven't used it, perhaps because I didn't want to know about it ;-), but it is claimed to be able to parse and process (using XPath) 256GB documents. Huib. > > Van: "Steven D. Majewski" > > [1] You might look at using Joost/STX which does streaming transforms. > It's not XSLT -- it's an xslt-like transformation language designed for one > pass processing, > so STXPath is more restricted than XPath. > > http://joost.sourceforge.net/
RE: Very large Generator file
Hi Steve, Yeah, I was looking at the Saxon solution earlier. The restriction that you can't use predicates (e.g. //eleme...@name=$refname]) I'm afraid may be a deal breaker for us. Jeff > -Original Message- > From: Steven D. Majewski [mailto:sd...@virginia.edu] > Sent: Tuesday, May 11, 2010 9:23 AM > To: users@cocoon.apache.org > Subject: Re: Very large Generator file > > > [1] You might look at using Joost/STX which does streaming transforms. > It's not XSLT -- it's an xslt-like transformation language designed > for one pass processing, > so STXPath is more restricted than XPath. > > http://joost.sourceforge.net/ > > We've used it within cocoon to extract data which is then may be > transformed in a 2nd stage. > We've run into a few bugs, but we're not using the latest current > version. > ( I think there were some incompatibilities with the cocoon 2.1.* stx > transformer and >the newer joost libraries, but I haven't looked at this in quite a > while. ) > > > [2] I believe Saxon can do some transforms in a streaming mode. I > think this extension > requires one of the paid licensed versions of Saxon. I haven't tried > this myself. > > http://saxonica.blogharbor.com/blog/_archives/2006/7/4/2084120.ht > ml > > -- Steve Majewski > > > On May 11, 2010, at 4:51 AM, Huib Verweij wrote: > > > Hi Jeff, > > > > You wrote: "I guess I was under the impression that SAX based xsl > > transformers didn't use much memory regardless file size, but > > perhaps that's not true?" > > > > It's true. However, when you do a XSLT transformation the entire XML > > document needs to be available to the XSLT processor, because you > > can access any node in the document, e.g. "/very/large/path/to/some/ > > dark/corner/of/the/XML/node". > > > > I'm not sure about your use-case so I don't know if it will help, > > but you could try using the MultiFragmentTraxTransformer from the > > cocooncomponents project on Google code. It reduces memory > > consumption drastically by allowing you to specify a fragment in the > > XML that you want transformed. If you have a very large document > > that looks like this: > > > > > > > > > > > > > > > > > > and all you want to do is transform the elements then the > > MultiFragmentTraxTransformer is your friend. > > > > > - > To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org > For additional commands, e-mail: users-h...@cocoon.apache.org - To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org For additional commands, e-mail: users-h...@cocoon.apache.org
Re: Very large Generator file
[1] You might look at using Joost/STX which does streaming transforms. It's not XSLT -- it's an xslt-like transformation language designed for one pass processing, so STXPath is more restricted than XPath. http://joost.sourceforge.net/ We've used it within cocoon to extract data which is then may be transformed in a 2nd stage. We've run into a few bugs, but we're not using the latest current version. ( I think there were some incompatibilities with the cocoon 2.1.* stx transformer and the newer joost libraries, but I haven't looked at this in quite a while. ) [2] I believe Saxon can do some transforms in a streaming mode. I think this extension requires one of the paid licensed versions of Saxon. I haven't tried this myself. http://saxonica.blogharbor.com/blog/_archives/2006/7/4/2084120.html -- Steve Majewski On May 11, 2010, at 4:51 AM, Huib Verweij wrote: Hi Jeff, You wrote: "I guess I was under the impression that SAX based xsl transformers didn't use much memory regardless file size, but perhaps that's not true?" It's true. However, when you do a XSLT transformation the entire XML document needs to be available to the XSLT processor, because you can access any node in the document, e.g. "/very/large/path/to/some/ dark/corner/of/the/XML/node". I'm not sure about your use-case so I don't know if it will help, but you could try using the MultiFragmentTraxTransformer from the cocooncomponents project on Google code. It reduces memory consumption drastically by allowing you to specify a fragment in the XML that you want transformed. If you have a very large document that looks like this: and all you want to do is transform the elements then the MultiFragmentTraxTransformer is your friend. - To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org For additional commands, e-mail: users-h...@cocoon.apache.org
Re: Very large Generator file
On Tue, May 11, 2010 at 07:56:19AM -0500, Schmitz, Jeffrey A wrote: > Thanks Huib, >I'm afraid my document is very interconnected. I'm coming to realize that > xslt has pretty much a hard limit on the size of files you can process with > it, i.e. if it can't fit into memory and you need the full power of xslt > you're pretty much out of luck. What about going to a 64 bit machine and > just allocating a huge amount of ram to Tomcat? Can this been done with > tomcat/cocoon? It can be done with a 64-bit JVM. Tomcat won't know and neither will Cocoon. -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Balance your desire for bells and whistles with the reality that only a little more than 2 percent of world population has broadband. -- Ledford and Tyler, _Google Analytics 2.0_ pgpzEqtBX6xps.pgp Description: PGP signature
RE: Very large Generator file
Thanks Huib, I'm afraid my document is very interconnected. I'm coming to realize that xslt has pretty much a hard limit on the size of files you can process with it, i.e. if it can't fit into memory and you need the full power of xslt you're pretty much out of luck. What about going to a 64 bit machine and just allocating a huge amount of ram to Tomcat? Can this been done with tomcat/cocoon? > -Original Message- > From: Huib Verweij [mailto:h...@home.nl] > Sent: Tuesday, May 11, 2010 3:51 AM > To: users@cocoon.apache.org > Cc: Huib Verweij > Subject: Re: Very large Generator file > > Hi Jeff, > > You wrote: "I guess I was under the impression that SAX based xsl > transformers didn't use much memory regardless file size, but perhaps > that's not true?" > > It's true. However, when you do a XSLT transformation the entire XML > document needs to be available to the XSLT processor, because you can > access any node in the document, e.g. > "/very/large/path/to/some/dark/corner/of/the/XML/node". > > I'm not sure about your use-case so I don't know if it will help, but > you could try using the MultiFragmentTraxTransformer from the > cocooncomponents project on Google code. It reduces memory consumption > drastically by allowing you to specify a fragment in the XML that you > want transformed. If you have a very large document that looks like > this: > > > > > > > > > and all you want to do is transform the elements then the > MultiFragmentTraxTransformer is your friend. > > Huib. > - > To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org > For additional commands, e-mail: users-h...@cocoon.apache.org - To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org For additional commands, e-mail: users-h...@cocoon.apache.org
Re: Very large Generator file
Hi Jeff, You wrote: "I guess I was under the impression that SAX based xsl transformers didn't use much memory regardless file size, but perhaps that's not true?" It's true. However, when you do a XSLT transformation the entire XML document needs to be available to the XSLT processor, because you can access any node in the document, e.g. "/very/large/path/to/some/dark/corner/of/the/XML/node". I'm not sure about your use-case so I don't know if it will help, but you could try using the MultiFragmentTraxTransformer from the cocooncomponents project on Google code. It reduces memory consumption drastically by allowing you to specify a fragment in the XML that you want transformed. If you have a very large document that looks like this: and all you want to do is transform the elements then the MultiFragmentTraxTransformer is your friend. Huib. - To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org For additional commands, e-mail: users-h...@cocoon.apache.org
RE: Very large Generator file
Thanks Jos, I have tried moving the tfactory stuff into my specialized cocoon transformer (com.boeing.transform.SPARQLinTrans), and I've also tried processing the file in a stand-alone java program with 1 Gig of memory assigned to it, and in both cases I'm still running out of heap space (see exception stack below). So, the current problem must lie in the xsl transform libraries I'm using (i.e. not really a cocoon problem at this point). I know there must be techniques for doing xsl on very large files, do you have any pointers or ideas for this problem in general? Thanks! Jeff java.lang.OutOfMemoryError: Java heap space net.sf.saxon.tinytree.TinyTree.ensureNodeCapacity(TinyTree.java:215) net.sf.saxon.tinytree.TinyTree.addNode(TinyTree.java:309) net.sf.saxon.tinytree.TinyBuilder.characters(TinyBuilder.java:306) net.sf.saxon.event.ReceivingContentHandler.flush(ReceivingContentHandler.java:544) net.sf.saxon.event.ReceivingContentHandler.startElement(ReceivingContentHandler.java:275) org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown Source) org.apache.xerces.parsers.AbstractXMLDocumentParser.emptyElement(Unknown Source) org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown Source) org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source) org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source) org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) org.apache.xerces.parsers.XMLParser.parse(Unknown Source) org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source) net.sf.saxon.event.Sender.sendSAXSource(Sender.java:354) net.sf.saxon.event.Sender.send(Sender.java:184) net.sf.saxon.event.Sender.send(Sender.java:49) net.sf.saxon.Controller.transform(Controller.java:1550) com.boeing.transform.SPARQLinTrans.adapt(SPARQLinTrans.java:143) Of course this is still a part of the Tomcat process. > -Original Message- > From: Jos Snellings [mailto:jos.snelli...@pandora.be] > Sent: Monday, May 10, 2010 10:04 AM > To: users@cocoon.apache.org > Subject: RE: Very large Generator file > > Yes, Jeff, my guess is that the tfactory stuff will accomplish the job > for you without complaining about memory shortage! Your tomcat instance > may have other things around increasing the memory footprint all the > time. > > > On Mon, 2010-05-10 at 09:43 -0500, Schmitz, Jeffrey A wrote: > > Yes, the resulting XHTML file is not for online use. > > > > And yes, I've given max memory to the tomcat instance. So, will > having my specialized transformer do the xsl transform directly using a > SAX transformer not work either? e.g. something like: > > > >TransformerFactory tfactory = TransformerFactory.newInstance(); > >Transformer transformer = tfactory.newTransformer(new > StreamSource( > > xslID)); > >transformer.transform(new StreamSource(sourceFile), new > StreamResult( > > new File(modelFile))); > > > > > > I guess I was under the impression that SAX based xsl transformers > didn't use much memory regardless file size, but perhaps that's not > true? > > > > If that's not going to work, would moving to a 64 bit machine so I > can give more memory to Tomcat be an option (is there a 64 bit Tomcat)? > > > > > If this is the case a short java program can make that xhtml file > for > > > you. > > I'm not exactly sure what you're suggesting here. Do you mean for > the tomcat/cocoon process to exec a separate java program that will do > the xsl transform on the large file? Won't any java program come up > against the 2 Gig max memory limitation? > > > > Thanks, > > Jeff > > > > > -Original Message- > > > From: Jos Snellings [mailto:jos.snelli...@pandora.be] > > > Sent: Monday, May 10, 2010 9:30 AM > > > To: users@cocoon.apache.org > > > Subject: RE: Very large Generator file > > > > > > Hi Jeff, > > > > > > If you have been trying giving all the available memory to your > tomcat > > > instance (JVM options) you are at the end of you tether for the > current > > > solutions, unless somebody has a different idea. > > > > > > I understood from your previous mail that the resulting xhtml is > not > > > for > > > on-line us
RE: Very large Generator file
Yes, Jeff, my guess is that the tfactory stuff will accomplish the job for you without complaining about memory shortage! Your tomcat instance may have other things around increasing the memory footprint all the time. On Mon, 2010-05-10 at 09:43 -0500, Schmitz, Jeffrey A wrote: > Yes, the resulting XHTML file is not for online use. > > And yes, I've given max memory to the tomcat instance. So, will having my > specialized transformer do the xsl transform directly using a SAX transformer > not work either? e.g. something like: > >TransformerFactory tfactory = TransformerFactory.newInstance(); >Transformer transformer = tfactory.newTransformer(new StreamSource( > xslID)); >transformer.transform(new StreamSource(sourceFile), new StreamResult( > new File(modelFile))); > > > I guess I was under the impression that SAX based xsl transformers didn't use > much memory regardless file size, but perhaps that's not true? > > If that's not going to work, would moving to a 64 bit machine so I can give > more memory to Tomcat be an option (is there a 64 bit Tomcat)? > > > If this is the case a short java program can make that xhtml file for > > you. > I'm not exactly sure what you're suggesting here. Do you mean for the > tomcat/cocoon process to exec a separate java program that will do the xsl > transform on the large file? Won't any java program come up against the 2 > Gig max memory limitation? > > Thanks, > Jeff > > > -Original Message- > > From: Jos Snellings [mailto:jos.snelli...@pandora.be] > > Sent: Monday, May 10, 2010 9:30 AM > > To: users@cocoon.apache.org > > Subject: RE: Very large Generator file > > > > Hi Jeff, > > > > If you have been trying giving all the available memory to your tomcat > > instance (JVM options) you are at the end of you tether for the current > > solutions, unless somebody has a different idea. > > > > I understood from your previous mail that the resulting xhtml is not > > for > > on-line use, it is a sort of report/summary. > > If this is the case a short java program can make that xhtml file for > > you. > > > > Cheers, > > Jos > > > > > > On Mon, 2010-05-10 at 09:17 -0500, Schmitz, Jeffrey A wrote: > > > Hi Jos, > > >I've already written my own Transformer (extends > > AbstractSAXTransformer) that currently takes the output of the default > > cocoon xsl transformer (via the sitemap) and performs subsequent, non- > > xsl transforms, writes the results to file and generates the returned > > web-page > > > e.g. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'm currently heading in the direction of just adding the initial xsl > > transformation as part of my own Transformer's duties (I think your > > option 3) so that it can hopefully handle the larger files, but I was > > wondering if there might be a way to easily keep things as they are and > > continue using the default cocoon xsl tranform as the first step. > > Sounds like there's not? > > > > > > > > > > -Original Message- > > > > From: Jos Snellings [mailto:jos.snelli...@pandora.be] > > > > Sent: Monday, May 10, 2010 8:39 AM > > > > To: users@cocoon.apache.org > > > > Subject: RE: Very large Generator file > > > > > > > > A SAX pipeline is part of cocoon. > > > > Can you explain briefly what you want to do? > > > > Consider using a cocoon-3 pipeline. > > > > Consider using a SAX transformation: > > > > - read xml > > > > - do something with it (transforming, for instance) > > > > - write output to a file > > > > > > > > Jos > > > > > > > > On Mon, 2010-05-10 at 08:23 -0500, Schmitz, Jeffrey A wrote: > > > > > Correct, it's not being turned into a web page. Is a SAX > > pipeline > > > > part of Cocoon? I'm still on Cocoon 2.1, but I've never heard that > > > > term specifically, I thought the default generator did use SAX. > > > > > > > > > > > -Original Message- > > > > > > From: Jos Snellings [mailto:jos.snelli...@pandora.be] > > > > >
RE: Very large Generator file
Yes, the resulting XHTML file is not for online use. And yes, I've given max memory to the tomcat instance. So, will having my specialized transformer do the xsl transform directly using a SAX transformer not work either? e.g. something like: TransformerFactory tfactory = TransformerFactory.newInstance(); Transformer transformer = tfactory.newTransformer(new StreamSource( xslID)); transformer.transform(new StreamSource(sourceFile), new StreamResult( new File(modelFile))); I guess I was under the impression that SAX based xsl transformers didn't use much memory regardless file size, but perhaps that's not true? If that's not going to work, would moving to a 64 bit machine so I can give more memory to Tomcat be an option (is there a 64 bit Tomcat)? > If this is the case a short java program can make that xhtml file for > you. I'm not exactly sure what you're suggesting here. Do you mean for the tomcat/cocoon process to exec a separate java program that will do the xsl transform on the large file? Won't any java program come up against the 2 Gig max memory limitation? Thanks, Jeff > -Original Message- > From: Jos Snellings [mailto:jos.snelli...@pandora.be] > Sent: Monday, May 10, 2010 9:30 AM > To: users@cocoon.apache.org > Subject: RE: Very large Generator file > > Hi Jeff, > > If you have been trying giving all the available memory to your tomcat > instance (JVM options) you are at the end of you tether for the current > solutions, unless somebody has a different idea. > > I understood from your previous mail that the resulting xhtml is not > for > on-line use, it is a sort of report/summary. > If this is the case a short java program can make that xhtml file for > you. > > Cheers, > Jos > > > On Mon, 2010-05-10 at 09:17 -0500, Schmitz, Jeffrey A wrote: > > Hi Jos, > >I've already written my own Transformer (extends > AbstractSAXTransformer) that currently takes the output of the default > cocoon xsl transformer (via the sitemap) and performs subsequent, non- > xsl transforms, writes the results to file and generates the returned > web-page > > e.g. > > > > > > > > > > > > > > > > > > > > > > > > > > I'm currently heading in the direction of just adding the initial xsl > transformation as part of my own Transformer's duties (I think your > option 3) so that it can hopefully handle the larger files, but I was > wondering if there might be a way to easily keep things as they are and > continue using the default cocoon xsl tranform as the first step. > Sounds like there's not? > > > > > > > -Original Message- > > > From: Jos Snellings [mailto:jos.snelli...@pandora.be] > > > Sent: Monday, May 10, 2010 8:39 AM > > > To: users@cocoon.apache.org > > > Subject: RE: Very large Generator file > > > > > > A SAX pipeline is part of cocoon. > > > Can you explain briefly what you want to do? > > > Consider using a cocoon-3 pipeline. > > > Consider using a SAX transformation: > > > - read xml > > > - do something with it (transforming, for instance) > > > - write output to a file > > > > > > Jos > > > > > > On Mon, 2010-05-10 at 08:23 -0500, Schmitz, Jeffrey A wrote: > > > > Correct, it's not being turned into a web page. Is a SAX > pipeline > > > part of Cocoon? I'm still on Cocoon 2.1, but I've never heard that > > > term specifically, I thought the default generator did use SAX. > > > > > > > > > -Original Message- > > > > > From: Jos Snellings [mailto:jos.snelli...@pandora.be] > > > > > Sent: Saturday, May 08, 2010 12:06 AM > > > > > To: users@cocoon.apache.org > > > > > Subject: Re: Very large Generator file > > > > > > > > > > Hi Jeff, > > > > > > > > > > That depends a great deal on what you want to do exactly with > the > > > large > > > > > file. I guess you do not want to transform it into one huge web > > > page? > > > > > In general, if you have to deal with lengthy inputs, a SAX > pipeline > > > is > > > > > the way to go. > > > > > > > > > > Jos > > > > > > > > > > On Fri, 2010-05-07 at 18:14 -0500, Schmitz, Jeffrey A wrote:
RE: Very large Generator file
Hi Jeff, If you have been trying giving all the available memory to your tomcat instance (JVM options) you are at the end of you tether for the current solutions, unless somebody has a different idea. I understood from your previous mail that the resulting xhtml is not for on-line use, it is a sort of report/summary. If this is the case a short java program can make that xhtml file for you. Cheers, Jos On Mon, 2010-05-10 at 09:17 -0500, Schmitz, Jeffrey A wrote: > Hi Jos, >I've already written my own Transformer (extends AbstractSAXTransformer) > that currently takes the output of the default cocoon xsl transformer (via > the sitemap) and performs subsequent, non-xsl transforms, writes the results > to file and generates the returned web-page > e.g. > > > > > > > > > > > > > I'm currently heading in the direction of just adding the initial xsl > transformation as part of my own Transformer's duties (I think your option 3) > so that it can hopefully handle the larger files, but I was wondering if > there might be a way to easily keep things as they are and continue using the > default cocoon xsl tranform as the first step. Sounds like there's not? > > > > -Original Message- > > From: Jos Snellings [mailto:jos.snelli...@pandora.be] > > Sent: Monday, May 10, 2010 8:39 AM > > To: users@cocoon.apache.org > > Subject: RE: Very large Generator file > > > > A SAX pipeline is part of cocoon. > > Can you explain briefly what you want to do? > > Consider using a cocoon-3 pipeline. > > Consider using a SAX transformation: > > - read xml > > - do something with it (transforming, for instance) > > - write output to a file > > > > Jos > > > > On Mon, 2010-05-10 at 08:23 -0500, Schmitz, Jeffrey A wrote: > > > Correct, it's not being turned into a web page. Is a SAX pipeline > > part of Cocoon? I'm still on Cocoon 2.1, but I've never heard that > > term specifically, I thought the default generator did use SAX. > > > > > > > -Original Message- > > > > From: Jos Snellings [mailto:jos.snelli...@pandora.be] > > > > Sent: Saturday, May 08, 2010 12:06 AM > > > > To: users@cocoon.apache.org > > > > Subject: Re: Very large Generator file > > > > > > > > Hi Jeff, > > > > > > > > That depends a great deal on what you want to do exactly with the > > large > > > > file. I guess you do not want to transform it into one huge web > > page? > > > > In general, if you have to deal with lengthy inputs, a SAX pipeline > > is > > > > the way to go. > > > > > > > > Jos > > > > > > > > On Fri, 2010-05-07 at 18:14 -0500, Schmitz, Jeffrey A wrote: > > > > > Hello, > > > > > > > > > >I was wondering if there’s a way to handle very large files as > > > > > inputs to the default cocoon generator? As it is, I am getting a > > > > > memory heap error when trying to process an input file because > > it’s > > > > so > > > > > large. Am I going to have to process it in some other manner, or > > is > > > > > there a way to make Cocoon perform the transform for me? > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > Jeff > > > > > > > > > > > > > > > > > > > > > > > > > > --- > > -- > > > > To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org > > > > For additional commands, e-mail: users-h...@cocoon.apache.org > > > > > > > > > > > - > > To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org > > For additional commands, e-mail: users-h...@cocoon.apache.org > - To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org For additional commands, e-mail: users-h...@cocoon.apache.org
RE: Very large Generator file
Hi Jos, I've already written my own Transformer (extends AbstractSAXTransformer) that currently takes the output of the default cocoon xsl transformer (via the sitemap) and performs subsequent, non-xsl transforms, writes the results to file and generates the returned web-page e.g. I'm currently heading in the direction of just adding the initial xsl transformation as part of my own Transformer's duties (I think your option 3) so that it can hopefully handle the larger files, but I was wondering if there might be a way to easily keep things as they are and continue using the default cocoon xsl tranform as the first step. Sounds like there's not? > -Original Message- > From: Jos Snellings [mailto:jos.snelli...@pandora.be] > Sent: Monday, May 10, 2010 8:39 AM > To: users@cocoon.apache.org > Subject: RE: Very large Generator file > > A SAX pipeline is part of cocoon. > Can you explain briefly what you want to do? > Consider using a cocoon-3 pipeline. > Consider using a SAX transformation: > - read xml > - do something with it (transforming, for instance) > - write output to a file > > Jos > > On Mon, 2010-05-10 at 08:23 -0500, Schmitz, Jeffrey A wrote: > > Correct, it's not being turned into a web page. Is a SAX pipeline > part of Cocoon? I'm still on Cocoon 2.1, but I've never heard that > term specifically, I thought the default generator did use SAX. > > > > > -Original Message- > > > From: Jos Snellings [mailto:jos.snelli...@pandora.be] > > > Sent: Saturday, May 08, 2010 12:06 AM > > > To: users@cocoon.apache.org > > > Subject: Re: Very large Generator file > > > > > > Hi Jeff, > > > > > > That depends a great deal on what you want to do exactly with the > large > > > file. I guess you do not want to transform it into one huge web > page? > > > In general, if you have to deal with lengthy inputs, a SAX pipeline > is > > > the way to go. > > > > > > Jos > > > > > > On Fri, 2010-05-07 at 18:14 -0500, Schmitz, Jeffrey A wrote: > > > > Hello, > > > > > > > >I was wondering if there’s a way to handle very large files as > > > > inputs to the default cocoon generator? As it is, I am getting a > > > > memory heap error when trying to process an input file because > it’s > > > so > > > > large. Am I going to have to process it in some other manner, or > is > > > > there a way to make Cocoon perform the transform for me? > > > > > > > > > > > > > > > > Thanks, > > > > Jeff > > > > > > > > > > > > > > > > > > > > --- > -- > > > To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org > > > For additional commands, e-mail: users-h...@cocoon.apache.org > > > > > > - > To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org > For additional commands, e-mail: users-h...@cocoon.apache.org
RE: Very large Generator file
A SAX pipeline is part of cocoon. Can you explain briefly what you want to do? Consider using a cocoon-3 pipeline. Consider using a SAX transformation: - read xml - do something with it (transforming, for instance) - write output to a file Jos On Mon, 2010-05-10 at 08:23 -0500, Schmitz, Jeffrey A wrote: > Correct, it's not being turned into a web page. Is a SAX pipeline part of > Cocoon? I'm still on Cocoon 2.1, but I've never heard that term > specifically, I thought the default generator did use SAX. > > > -Original Message- > > From: Jos Snellings [mailto:jos.snelli...@pandora.be] > > Sent: Saturday, May 08, 2010 12:06 AM > > To: users@cocoon.apache.org > > Subject: Re: Very large Generator file > > > > Hi Jeff, > > > > That depends a great deal on what you want to do exactly with the large > > file. I guess you do not want to transform it into one huge web page? > > In general, if you have to deal with lengthy inputs, a SAX pipeline is > > the way to go. > > > > Jos > > > > On Fri, 2010-05-07 at 18:14 -0500, Schmitz, Jeffrey A wrote: > > > Hello, > > > > > >I was wondering if there’s a way to handle very large files as > > > inputs to the default cocoon generator? As it is, I am getting a > > > memory heap error when trying to process an input file because it’s > > so > > > large. Am I going to have to process it in some other manner, or is > > > there a way to make Cocoon perform the transform for me? > > > > > > > > > > > > Thanks, > > > Jeff > > > > > > > > > > > > > > - > > To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org > > For additional commands, e-mail: users-h...@cocoon.apache.org > - To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org For additional commands, e-mail: users-h...@cocoon.apache.org
RE: Very large Generator file
Correct, it's not being turned into a web page. Is a SAX pipeline part of Cocoon? I'm still on Cocoon 2.1, but I've never heard that term specifically, I thought the default generator did use SAX. > -Original Message- > From: Jos Snellings [mailto:jos.snelli...@pandora.be] > Sent: Saturday, May 08, 2010 12:06 AM > To: users@cocoon.apache.org > Subject: Re: Very large Generator file > > Hi Jeff, > > That depends a great deal on what you want to do exactly with the large > file. I guess you do not want to transform it into one huge web page? > In general, if you have to deal with lengthy inputs, a SAX pipeline is > the way to go. > > Jos > > On Fri, 2010-05-07 at 18:14 -0500, Schmitz, Jeffrey A wrote: > > Hello, > > > >I was wondering if there’s a way to handle very large files as > > inputs to the default cocoon generator? As it is, I am getting a > > memory heap error when trying to process an input file because it’s > so > > large. Am I going to have to process it in some other manner, or is > > there a way to make Cocoon perform the transform for me? > > > > > > > > Thanks, > > Jeff > > > > > > > > - > To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org > For additional commands, e-mail: users-h...@cocoon.apache.org
Re: Very large Generator file
Hi Jeff, That depends a great deal on what you want to do exactly with the large file. I guess you do not want to transform it into one huge web page? In general, if you have to deal with lengthy inputs, a SAX pipeline is the way to go. Jos On Fri, 2010-05-07 at 18:14 -0500, Schmitz, Jeffrey A wrote: > Hello, > >I was wondering if there’s a way to handle very large files as > inputs to the default cocoon generator? As it is, I am getting a > memory heap error when trying to process an input file because it’s so > large. Am I going to have to process it in some other manner, or is > there a way to make Cocoon perform the transform for me? > > > > Thanks, > Jeff > > - To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org For additional commands, e-mail: users-h...@cocoon.apache.org
