Re: Large xml's in cocoon
Upayavira wrote: Switching to Jetty was the best thing we did - Tomcat fell over all the time. Jetty has just stayed up. I find Tomcat 5 to be infinitely more stable and performant than Tomcat 4, so I can live with that for the moment. If you ever decide to switch, let me know and I'll send you config samples. We are in the process of ordering a couple new servers, so we might switch when we install them. But it's more probable that I switch jobs before we switch application servers ;-). Thank you anyway. Ugo
Re: Large xml's in cocoon
Ugo Cei wrote: Pier Fumagalli wrote: Oh, on a side note, Ugo, I'd seriously switch your development/production environments... I'm not going into performance details (I don't want to raise a can of worms), but at least on my tests Tomcat is using 3x the memory that Jetty is using per single request, so it's more likely for you to have OOMEs on Tomcat rather than Jetty. I couldn't agree more with this, Pier, but until my boss gets me a new engineer, I'm not going to learn how to properly configure Jetty for vhosts, authentication, logging and HTTPS and switch over. I don't want to do this job anymore, so they stay where they are ;-). Whilst I can undestand the sentiment of not wanting to do sysadmin work, I would say that Jetty is _far_ easier to configure than Tomcat. It is in fact just far simpler alround. Vhosts are easy, as is authentication realms (I can send you config samples). For logging, I'm not that sure (we just redirect Jetty's console to a file). HTTPs listeners are available in Jetty's default config file, including examples of how to refer to your keystore. Switching to Jetty was the best thing we did - Tomcat fell over all the time. Jetty has just stayed up. If you ever decide to switch, let me know and I'll send you config samples. Regards, Upayavira
Re: Large xml's in cocoon
Pier Fumagalli wrote: Oh, on a side note, Ugo, I'd seriously switch your development/production environments... I'm not going into performance details (I don't want to raise a can of worms), but at least on my tests Tomcat is using 3x the memory that Jetty is using per single request, so it's more likely for you to have OOMEs on Tomcat rather than Jetty. I couldn't agree more with this, Pier, but until my boss gets me a new engineer, I'm not going to learn how to properly configure Jetty for vhosts, authentication, logging and HTTPS and switch over. I don't want to do this job anymore, so they stay where they are ;-). Ugo
Re: Large xml's in cocoon
On 28 May 2004, at 15:34, Ugo Cei wrote:
Stefano Mazzocchi wrote:
what if I'm using jetty?
I don't know. We're currently using the bundled Jetty for development
and Tomcat for staging/production. But my point still remains: it's
better to crash and burn rather than try to survive if you get an
OOMException.
Yeah, the problem is that you never "crash and burn" on OOMEs. The JVM
doesn't itself exit.
You could (for instance) do something like:
try {
...
} catch (OutOfMemoryError e) {
System.exit(0);
}
I've seen a lot of OOMEs happening just because the machine was simply
overloaded, and in those situation doing a System.exit() was more
harmful than simply letting that thread die.
When a thread (a request thread, for instance) dies because of an OOME,
is _definitely_ going to free up a couple of objects, and having the
JVM that bit of millisecond to re-sort itself out and clean up some
memory after thread death...
Oh, on a side note, Ugo, I'd seriously switch your
development/production environments... I'm not going into performance
details (I don't want to raise a can of worms), but at least on my
tests Tomcat is using 3x the memory that Jetty is using per single
request, so it's more likely for you to have OOMEs on Tomcat rather
than Jetty.
Pier
smime.p7s
Description: S/MIME cryptographic signature
Re: Large xml's in cocoon
On 28 May 2004, at 15:28, Stefano Mazzocchi wrote:
Antonio Gallardo wrote:
Stefano Mazzocchi dijo:
Hmmm, Anna is right to report the problem though. I mean, why in
hell an
OutOfMemoryException get swollen like that?
Not sure. I don't think we are swollen the exception in this case.
But I
am not the best knowledge to tell that. ;-)
What I can said is: we have an application running since Cocoon 2.0.3
and
has being updated on each Cocoon version to 2.1..5 and it works fine.
We
render some reports to excel, html and pdf with more than 5,000 rows.
I did experience the same "white page" error before and I feel that
was an OutOfMemoryException due to the JPEG thumbnail code (I was
creating 20 thumbnails at the same time from 4 megapixel images... I
guess the JVM heap was going insane).
So, let's use heuristics here: did anybody ever see an
OutOfMemoryException in an cocoon error page? I haven't.
Did you ever get a white page in cocoon that went away after one or
two reloads? I did.
thoughts?
Ste, it's easy... I noticed several times that in OOMEs the stacktraces
were all wrong as well... And I started to ask myself why...
Any time you do a catch (Throwable e) which (in theory) should also
catch the OOME, you'll create an object, even maybe a silly one, like
an exception wrapping the OOME, but, hey, you're out of memory, so,
ANYTHING you do with that exception, apart from probably
e.printStackTrace() will inevitably trigger another OOME...
I got around the problem by doing something like this:
try {
} catch (OutOfMemoryException exception) {
System.gc();
Thread.currentThread().sleep(1000);
// handle the OOME
}
but it doesn't always work...
I solved the problem by always piping my java standard output and error
to a file, because being handled in the C code, that exception falls
back to the bottom and (at the end) is dumped on your console...
But (for example) Tomcat's default startup scripts (or Jetty's for that
matter) completely ignore the console (and that's why I always use my
own builds of everything, most of the code out there is simply not
ready "as is" to be deployed on production).
I mean, don't get me wrong, it's all very well written code, but until
you don't hit a problem like a OOME on a real server, well, noone seems
to care! :-P
Pier
smime.p7s
Description: S/MIME cryptographic signature
Re: Large xml's in cocoon
Bruno Dumon wrote:
On Fri, 2004-05-28 at 16:28, Stefano Mazzocchi wrote:
Antonio Gallardo wrote:
Stefano Mazzocchi dijo:
Hmmm, Anna is right to report the problem though. I mean, why in hell an
OutOfMemoryException get swollen like that?
Not sure. I don't think we are swollen the exception in this case. But I
am not the best knowledge to tell that. ;-)
What I can said is: we have an application running since Cocoon 2.0.3 and
has being updated on each Cocoon version to 2.1..5 and it works fine. We
render some reports to excel, html and pdf with more than 5,000 rows.
I did experience the same "white page" error before and I feel that was
an OutOfMemoryException due to the JPEG thumbnail code (I was creating
20 thumbnails at the same time from 4 megapixel images... I guess the
JVM heap was going insane).
So, let's use heuristics here: did anybody ever see an
OutOfMemoryException in an cocoon error page? I haven't.
Did you ever get a white page in cocoon that went away after one or two
reloads? I did.
thoughts?
It is because it is an OutOfMemoryError, not an Exception. Currently
Errors are not catched by Cocoon.
There has been a discussion thread about this in the past, I don't
remember if any consensus was reached about what how we want to treat
Errors.
IIRC, we found that OutOfMemoryError was special and could be catched by
Cocoon, as it can potentially occur in the processing of pipelines that
normally succeed, either will smaller documents or less concurrent users.
Other errors (NoClassDefFound, StackOverflow etc) should not be handled
by Cocoon
Sylvain
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
Re: Large xml's in cocoon
On 28.05.2004 16:42, Bruno Dumon wrote: It is because it is an OutOfMemoryError, not an Exception. Currently Errors are not catched by Cocoon. There has been a discussion thread about this in the past, I don't remember if any consensus was reached about what how we want to treat Errors. There are also samples about exception and error handling and Antonio mentioned immediately before the release that the error results in a blank page. AFAIK that was the consensus of that discussion thread. Joerg
Re: Large xml's in cocoon
On Fri, 2004-05-28 at 16:28, Stefano Mazzocchi wrote: > Antonio Gallardo wrote: > > > Stefano Mazzocchi dijo: > > > >>Hmmm, Anna is right to report the problem though. I mean, why in hell an > >>OutOfMemoryException get swollen like that? > > > > > > Not sure. I don't think we are swollen the exception in this case. But I > > am not the best knowledge to tell that. ;-) > > > > What I can said is: we have an application running since Cocoon 2.0.3 and > > has being updated on each Cocoon version to 2.1..5 and it works fine. We > > render some reports to excel, html and pdf with more than 5,000 rows. > > I did experience the same "white page" error before and I feel that was > an OutOfMemoryException due to the JPEG thumbnail code (I was creating > 20 thumbnails at the same time from 4 megapixel images... I guess the > JVM heap was going insane). > > So, let's use heuristics here: did anybody ever see an > OutOfMemoryException in an cocoon error page? I haven't. > > Did you ever get a white page in cocoon that went away after one or two > reloads? I did. > > thoughts? It is because it is an OutOfMemoryError, not an Exception. Currently Errors are not catched by Cocoon. There has been a discussion thread about this in the past, I don't remember if any consensus was reached about what how we want to treat Errors. -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: Large xml's in cocoon
Stefano Mazzocchi wrote: what if I'm using jetty? I don't know. We're currently using the bundled Jetty for development and Tomcat for staging/production. But my point still remains: it's better to crash and burn rather than try to survive if you get an OOMException. Ugo
Re: Large xml's in cocoon
Ugo Cei wrote: Il giorno 28/mag/04, alle 03:26, Stefano Mazzocchi ha scritto: Hmmm, Anna is right to report the problem though. I mean, why in hell an OutOfMemoryException get swollen like that? Swollen? I think you mean "swallowed" here. eheh, right :-) [maybe my mind thought of "swollen" because of too much memory used ;-)] Anyway, I think it has to do more with Tomcat than with Cocoon. It happens (particularly when you frequently reload applications) that Tomcat outputs a blank page and logs a simple "java.lang.OutOfMemoryException" without a stacktrace to catalina.out. I can understand this, what else can you do when you're out of memory? You're not guaranteed to be able to log a stacktrace or send anything to the client. Actually, I'd rather wish Tomcat crashed in this circumstance, so we might at least restart it automatically. what if I'm using jetty? -- Stefano. smime.p7s Description: S/MIME Cryptographic Signature
Re: Large xml's in cocoon
Antonio Gallardo wrote: Stefano Mazzocchi dijo: Hmmm, Anna is right to report the problem though. I mean, why in hell an OutOfMemoryException get swollen like that? Not sure. I don't think we are swollen the exception in this case. But I am not the best knowledge to tell that. ;-) What I can said is: we have an application running since Cocoon 2.0.3 and has being updated on each Cocoon version to 2.1..5 and it works fine. We render some reports to excel, html and pdf with more than 5,000 rows. I did experience the same "white page" error before and I feel that was an OutOfMemoryException due to the JPEG thumbnail code (I was creating 20 thumbnails at the same time from 4 megapixel images... I guess the JVM heap was going insane). So, let's use heuristics here: did anybody ever see an OutOfMemoryException in an cocoon error page? I haven't. Did you ever get a white page in cocoon that went away after one or two reloads? I did. thoughts? -- Stefano. smime.p7s Description: S/MIME Cryptographic Signature
Re: Large xml's in cocoon
Il giorno 28/mag/04, alle 03:26, Stefano Mazzocchi ha scritto: Hmmm, Anna is right to report the problem though. I mean, why in hell an OutOfMemoryException get swollen like that? Swollen? I think you mean "swallowed" here. Anyway, I think it has to do more with Tomcat than with Cocoon. It happens (particularly when you frequently reload applications) that Tomcat outputs a blank page and logs a simple "java.lang.OutOfMemoryException" without a stacktrace to catalina.out. I can understand this, what else can you do when you're out of memory? You're not guaranteed to be able to log a stacktrace or send anything to the client. Actually, I'd rather wish Tomcat crashed in this circumstance, so we might at least restart it automatically. Ugo -- Ugo Cei - http://beblogging.com/
RE: Large xml's in cocoon
Hi Guys, I have experienced similar problems. My work around was: 1. Use the latest version of Xalan (there is a bug fix which is of similar nature to this, although I can't recall what its number is.) 2. Tweak the buffer size settings for your pipeline. I think number 2 is probably a little superstitious ;) But do try it. Corin -Original Message- From: Antonio Gallardo [mailto:[EMAIL PROTECTED] Sent: Friday, 28 May 2004 2:29 p.m. To: [EMAIL PROTECTED] Subject: Re: Large xml's in cocoon Stefano Mazzocchi dijo: > Hmmm, Anna is right to report the problem though. I mean, why in hell > an OutOfMemoryException get swollen like that? Not sure. I don't think we are swollen the exception in this case. But I am not the best knowledge to tell that. ;-) What I can said is: we have an application running since Cocoon 2.0.3 and has being updated on each Cocoon version to 2.1..5 and it works fine. We render some reports to excel, html and pdf with more than 5,000 rows. Best Regards, Antonio Gallardo CAUTION: This e-mail and any attachment(s) contains information that is intended to be read only by the named recipient(s). It may contain information that is confidential, proprietary or the subject of legal privilege. This information is not to be used by any other person and/or organisation. If you are not the intended recipient, please advise us immediately and delete this e-mail from your system. Do not use any information contained in it. For more information on the Television New Zealand Group, visit us online at http://www.tvnz.co.nz
Re: Large xml's in cocoon
Stefano Mazzocchi dijo: > Hmmm, Anna is right to report the problem though. I mean, why in hell an > OutOfMemoryException get swollen like that? Not sure. I don't think we are swollen the exception in this case. But I am not the best knowledge to tell that. ;-) What I can said is: we have an application running since Cocoon 2.0.3 and has being updated on each Cocoon version to 2.1..5 and it works fine. We render some reports to excel, html and pdf with more than 5,000 rows. Best Regards, Antonio Gallardo
Re: Large xml's in cocoon
Antonio Gallardo wrote: Anna Bikkina dijo: Hi, I have a xml file that contains database rows. If the number of rows are greater than 5000 then cocoon fails to tranform them to html and display them. It gives a blank page with no errors. Has anyone experienced this before? Is there a way I can make cocoon perform better with big xml.? Hi: Cocoon can manage it. Please check your catalina.out to see if there is a "Out Of Memory error" If this is the case, then you need to allow your servlet container (Tomcat, jetty, etc.) use more memory. I managed to transform some files with more than 10,000 rows. If you don't have any error in catalina.out, then there is no error and maybe the browser is taking a lot of time to render the page: If we generate a big HTML file with a lot of rows. It generates a lot of tags and the browse need to calculate how to render this huge table and that is the cause of the apparently long delay in the response. BTW, you can see the Cocoon response time in /WEB-INF/logs/access.log file. The rest of the time is used by the browser to render the page. Hmmm, Anna is right to report the problem though. I mean, why in hell an OutOfMemoryException get swollen like that? -- Stefano. smime.p7s Description: S/MIME Cryptographic Signature
Re: Large xml's in cocoon
On Thu, 2004-05-27 at 16:36, Anna Bikkina wrote: > Hi, > > I have a xml file that contains database rows. If the number of rows are > greater than 5000 then cocoon fails to tranform them to html and display > them. It gives a blank page with no errors. > > Has anyone experienced this before? Is there a way I can make cocoon perform > better with big xml.? > > Any suggestions,inputs are welcome. > Besides the suggestions already given, it is recommendable to disable complete output buffering by specifying the outputBufferSize parameter to the pipeline in the sitemap. See Cocoon's default sitemap for more info on this. -- Bruno Dumon http://outerthought.org/ Outerthought - Open Source, Java & XML Competence Support Center [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: Large xml's in cocoon
On 27 May 2004, at 17:34, Greg Weinger wrote: Hi, The way your stylesheet is written can also create OutOfMemoryErrors. If your xpaths require that the entire document be held in memory before it is written, then you can run out of memory quickly. This is, alas, still the behaviour of every XSLT engine out there. It's not a full DOM which gets built, but still it's a table/tree representation of the entire input document - so you need memory to handle large input documents. Some XSLT engines do however start writing out output before the transformation is completed, which gives slightly better response times, but just masks the problem. Or it must be that recent Xalan/Saxon iterations finally tackled this. In your templates try to use absolute xpaths (/document/path/to/node) instead of inexact (//node). This is of course a good XSLT practice. -- Steven Noelshttp://outerthought.org/ Outerthought - Open Source Java & XMLAn Orixo Member Read my weblog athttp://blogs.cocoondev.org/stevenn/ stevenn at outerthought.orgstevenn at apache.org
Re: Large xml's in cocoon
Hi, The way your stylesheet is written can also create OutOfMemoryErrors. If your xpaths require that the entire document be held in memory before it is written, then you can run out of memory quickly. In your templates try to use absolute xpaths (/document/path/to/node) instead of inexact (//node). Hope that helps, Greg Antonio Gallardo wrote: Anna Bikkina dijo: Hi, I have a xml file that contains database rows. If the number of rows are greater than 5000 then cocoon fails to tranform them to html and display them. It gives a blank page with no errors. Has anyone experienced this before? Is there a way I can make cocoon perform better with big xml.? Hi: Cocoon can manage it. Please check your catalina.out to see if there is a "Out Of Memory error" If this is the case, then you need to allow your servlet container (Tomcat, jetty, etc.) use more memory. I managed to transform some files with more than 10,000 rows. If you don't have any error in catalina.out, then there is no error and maybe the browser is taking a lot of time to render the page: If we generate a big HTML file with a lot of rows. It generates a lot of tags and the browse need to calculate how to render this huge table and that is the cause of the apparently long delay in the response. BTW, you can see the Cocoon response time in /WEB-INF/logs/access.log file. The rest of the time is used by the browser to render the page. Hope this help. Best Regards, Antonio Gallardo.
Re: Large xml's in cocoon
Anna Bikkina dijo: > Hi, > > I have a xml file that contains database rows. If the number of rows are > greater than 5000 then cocoon fails to tranform them to html and display > them. It gives a blank page with no errors. > > Has anyone experienced this before? Is there a way I can make cocoon > perform > better with big xml.? Hi: Cocoon can manage it. Please check your catalina.out to see if there is a "Out Of Memory error" If this is the case, then you need to allow your servlet container (Tomcat, jetty, etc.) use more memory. I managed to transform some files with more than 10,000 rows. If you don't have any error in catalina.out, then there is no error and maybe the browser is taking a lot of time to render the page: If we generate a big HTML file with a lot of rows. It generates a lot of tags and the browse need to calculate how to render this huge table and that is the cause of the apparently long delay in the response. BTW, you can see the Cocoon response time in /WEB-INF/logs/access.log file. The rest of the time is used by the browser to render the page. Hope this help. Best Regards, Antonio Gallardo.
Re: Large xml's in cocoon
On 27 May 2004, at 15:36, Anna Bikkina wrote: Hi, I have a xml file that contains database rows. If the number of rows are greater than 5000 then cocoon fails to tranform them to html and display them. It gives a blank page with no errors. Has anyone experienced this before? Is there a way I can make cocoon perform better with big xml.? Any suggestions,inputs are welcome. Thanks, Anna. Check your standard error / standard output for OutOfMemoryException, and run cocoon with greater Java heap sizes (-Xms512m -Xmx512m) Pier smime.p7s Description: S/MIME cryptographic signature
Large xml's in cocoon
Hi, I have a xml file that contains database rows. If the number of rows are greater than 5000 then cocoon fails to tranform them to html and display them. It gives a blank page with no errors. Has anyone experienced this before? Is there a way I can make cocoon perform better with big xml.? Any suggestions,inputs are welcome. Thanks, Anna.
