too many open files
hi,from time to time my cocoon get this error message:Too Many Open FilesSometimes after refresh it continues work, but sometimes it needs to be restarted.Is there any way to fix this issue?
Too many open files
I'm getting this error in apparently random circumstances on http://publish.ucc.ie/researchprofiles/ using Cocoon 2.1.11 under Tomcat5 on RHEL5. This application is reprocessing some HTML feeds from another server in order to present the information in a different layout, and do some extensive cleaning-up of the data. Most of the time it works just fine. The full stacktrace says: > org.apache.cocoon.ProcessingException: Failed to process pipeline > at - file:///var/www/xml/profiles/sitemap.xmap:74:18 > at - file:///var/www/xml/profiles/sitemap.xmap:69:36 > at - > file:///var/www/xml/profiles/sitemap.xmap:68:17 The line numbers vary according to the page type (and thus which piece of the pipeline). All the templates generate the page layout by retrieving a HTML page via Tidy, and slotting different pieces of data into different places: a fairly typical mashup. In the process, the XSLT will open five internal pipelines via document() calls to retrieve data, each of which opens one or two other document()s and two local (static) XML lookup tables (very small). > at - > file:///usr/share/tomcat5/webapps/ROOT/sitemap.xmap:1115:57 > at > org.apache.cocoon.ProcessingException.throwLocated(ProcessingException.java:145) Further down the stacktrace, it says: > Caused by: java.net.SocketException: Too many open files > at java.net.Socket.createImpl(Socket.java:388) > at java.net.Socket.connect(Socket.java:517) > at java.net.Socket.connect(Socket.java:469) > at sun.net.NetworkClient.doConnect(NetworkClient.java:157) > at sun.net.www.http.HttpClient.openServer(HttpClient.java:394) In my ignorance of the internals of Cocoon and Tomcat and Java, is it running out of file handles when making a HTTP call? The server is lightly loaded, but each page does open a significant number of document()s: is there a setting that will increase the number of open files allowed? Is there any way to get it to report *what* it was trying to open (the URI) when it failed? ///Peter - To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org For additional commands, e-mail: users-h...@cocoon.apache.org
Re: too many open files
On 7/3/06, Bokluci Bokluci <[EMAIL PROTECTED]> wrote: ...from time to time my cocoon get this error message: Too Many Open Files... We need more info to help, what kind of app is this, what components of Cocoon and what external components are you using? -Bertrand - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: too many open files
No special components, just cocoon started with cocoon servlet. What do you mean by "what kind of app is this?" -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Bertrand Delacretaz Sent: Monday, July 03, 2006 9:35 PM To: users@cocoon.apache.org Subject: Re: too many open files On 7/3/06, Bokluci Bokluci <[EMAIL PROTECTED]> wrote: > ...from time to time my cocoon get this error message: > > Too Many Open Files... We need more info to help, what kind of app is this, what components of Cocoon and what external components are you using? -Bertrand - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: too many open files
On 7/4/06, Bokluci <[EMAIL PROTECTED]> wrote: ...What do you mean by "what kind of app is this?"... The question is, what are your application patterns, i.e. do you use many XSLT transforms, or the SQLTransformer, often call external processes, use a lot of Flow and/or CForms, etc.. -Bertrand - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: too many open files
> ...What do you mean by "what kind of app is this?"... The question is, what are your application patterns, i.e. do you use many XSLT transforms, or the SQLTransformer, often call external processes, use a lot of Flow and/or CForms, etc.. And i would add: running with Tomcat under Linux with a non-privileged user? (see /etc/security/limits.conf) Cayetano - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: too many open files
I heard something about a Java bug that seemed to cause these kinds of errors. What version of Java are you using and what happens if you update to the latest version (if you are not yet using it?) Kind regards, Geert > Drs. G.P.H. Josten Consultant Daidalos BV Source of Innovation Hoekeindsehof 1-4 2665 JZ Bleiswijk Tel.: +31 (0) 10 850 1200 Fax: +31 (0) 10 850 1199 www.daidalos.nl De informatie - verzonden in of met dit emailbericht - is afkomstig van Daidalos BV en is uitsluitend bestemd voor de geadresseerde. Indien u dit bericht onbedoeld hebt ontvangen, verzoeken wij u het te verwijderen. Aan dit bericht kunnen geen rechten worden ontleend. > Van: Bokluci Bokluci [mailto:[EMAIL PROTECTED] > Verzonden: maandag 3 juli 2006 19:01 > Aan: users@cocoon.apache.org > Onderwerp: too many open files > > hi, > > from time to time my cocoon get this error message: > > Too Many Open Files > > Sometimes after refresh it continues work, but sometimes it > needs to be restarted. > Is there any way to fix this issue? > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: too many open files
Hi Bokluci, In case you're running Cocoon on Linux, these are my experiences with the "too many open files" issue. I used this in the past to raise the number of open files, which might not be the actual solution to your problem, but at least it will give you some pointers for where to look at. * login as root user* check 'ulimit -n', this will show you the current limit of open files* edit /etc/security/limits.conf:** soft nofile 8192** hard nofile 65535* edit /etc/pam.d/sshd:** session required /lib/security/pam_limits.so* 'ulimit -n 8192' (set limit to 8192)* 'service sshd restart' (or /etc/init.d/sshd restart)* logoff* logon again and check 'ulimit -n', this will show you the new values The maximum number of open files is limited to the capabilities of the filesystem in use. You can find the max number for *all processes combined* by doing 'cat /proc/sys/fs/file-max'. It's normally somewhere between 100.000 and 500.000.If you set the hard limit in limits.conf to high, you might block the filesystem which makes the server unreliable and even block SSH shells!! 8192 is a common setting. Good luck, Kind regards,Arjé CahnHippo Oosteinde 111017WT AmsterdamThe NetherlandsTel +31 (0)20 5224466-[EMAIL PROTECTED] / www.hippo.nl / [EMAIL PROTECTED]-- From: Bokluci Bokluci [mailto:[EMAIL PROTECTED] Posted At: maandag 3 juli 2006 19:01Posted To: Cocoon User ListConversation: too many open filesSubject: too many open fileshi,from time to time my cocoon get this error message:Too Many Open FilesSometimes after refresh it continues work, but sometimes it needs to be restarted.Is there any way to fix this issue?
Re: too many open files
On 7/4/06, Geert Josten <[EMAIL PROTECTED]> wrote: ...I heard something about a Java bug that seemed to cause these kinds of errors... Which might be: using the Process object and not explicitely closing its channels, see http://codeconsult.ch/bertrand/archives/000456.html -Bertrand - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: too many open files
As a matter of a fact I'm not using cocoon with combination with tomcat, should I? Are there any drawbacks if I use it alone with the standard bundled server? -Original Message- From: Cayetano [mailto:[EMAIL PROTECTED] Sent: Tuesday, July 04, 2006 8:13 AM To: users@cocoon.apache.org Subject: Re: too many open files > > ...What do you mean by "what kind of app is this?"... > > The question is, what are your application patterns, i.e. do you use > many XSLT transforms, or the SQLTransformer, often call external > processes, use a lot of Flow and/or CForms, etc.. > And i would add: running with Tomcat under Linux with a non-privileged user? (see /etc/security/limits.conf) Cayetano - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: too many open files
On 7/4/06, Bokluci <[EMAIL PROTECTED]> wrote: ...Are there any drawbacks if I use it alone with the standard bundled server?... Not necessarily, it's not optimized for very high loads but runs just fine on several of my production systems, with up to 9 months uptime until now. -Bertrand - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: too many open files
How high should be "very high load"? over 10 000 hits a day? Are there any drawbacks running cocoon as non privileged user? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Bertrand Delacretaz Sent: Tuesday, July 04, 2006 1:34 PM To: users@cocoon.apache.org Subject: Re: too many open files On 7/4/06, Bokluci <[EMAIL PROTECTED]> wrote: > ...Are there any drawbacks if I use it alone with the standard > bundled server?... Not necessarily, it's not optimized for very high loads but runs just fine on several of my production systems, with up to 9 months uptime until now. -Bertrand - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: too many open files
On 7/4/06, Bokluci <[EMAIL PROTECTED]> wrote: How high should be "very high load"? over 10 000 hits a day? Impossible to say, it all depends on how the replies to these hits are generated, and your traffic patterns. You need to measure the performance with simulated or actual load to make decisions about this. 10'000 hits a day (86'400 seconds) can be as low as 0.12 per second if evenly spread, which is not much at all. OTOH if these hits come within 2 minutes...you get the idea. ...Are there any drawbacks running cocoon as non privileged user?... Not that I know, quite the contrary, except that it cannot listen on port 80 then. But it's a good idea to put an httpd / reverse proxy in front anyway. -Bertrand - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
error: Too Many Open Files
>From Time to time cocoon shows "Too many open files" error. I've already set ulimit to ulimit -n 8192 , but it still shows from time to time this error... Any suggestions? Thanks in advance. Vik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
"Too many open files" error
I am running a single Cocoon app on the server, and am the only one testing it... despite this, I get a: cause: java.io.FileNotFoundException: /usr/local/jakarta-tomcat-4.1.31/webapps/cocoon/myapp/myfile.xml (Too many open files) error when I try and load CForm-based page. The server is Linux-based, but on my test machine (Windows) I did not notice this problem. Any ideas what the cause could be or, more to the point, how to prevent this kind of problem? (eg. if I have 20 users who are going to be opening files, this implies the app will come to a standstill in the first few seconds!) Thanks Derek - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Too many open files
Hello, Le 16/12/2010 13:58, Peter Flynn a écrit : >> Caused by: java.net.SocketException: Too many open files >> at java.net.Socket.createImpl(Socket.java:388) >> at java.net.Socket.connect(Socket.java:517) >> at java.net.Socket.connect(Socket.java:469) >> at sun.net.NetworkClient.doConnect(NetworkClient.java:157) >> at sun.net.www.http.HttpClient.openServer(HttpClient.java:394) > > In my ignorance of the internals of Cocoon and Tomcat and Java, is it > running out of file handles when making a HTTP call? > > The server is lightly loaded, but each page does open a significant > number of document()s: is there a setting that will increase the number > of open files allowed? We already have seen this issue on some cocoon applications ; most of the time, that was due to intensive use of xpath "document" function with. We observe that using saxon as xslt processor genrerally solved that issue. Most of these issues has been resolved for us doing that ; some are always present but I did not have time to further investigate yet. > > Is there any way to get it to report *what* it was trying to open (the > URI) when it failed? > > ///Peter > Hope that could help you. Regards, -- Johan Cwiklinski AJLSM - To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org For additional commands, e-mail: users-h...@cocoon.apache.org
Re: Too many open files
On 16/12/10 13:03, Johan Cwiklinski wrote: > Hello, > > Le 16/12/2010 13:58, Peter Flynn a écrit : >>> Caused by: java.net.SocketException: Too many open files >>> at java.net.Socket.createImpl(Socket.java:388) >>> at java.net.Socket.connect(Socket.java:517) >>> at java.net.Socket.connect(Socket.java:469) >>> at sun.net.NetworkClient.doConnect(NetworkClient.java:157) >>> at sun.net.www.http.HttpClient.openServer(HttpClient.java:394) >> >> In my ignorance of the internals of Cocoon and Tomcat and Java, is it >> running out of file handles when making a HTTP call? >> >> The server is lightly loaded, but each page does open a significant >> number of document()s: is there a setting that will increase the number >> of open files allowed? > > We already have seen this issue on some cocoon applications ; most of > the time, that was due to intensive use of xpath "document" function > with. We observe that using saxon as xslt processor genrerally solved > that issue. I have had that on my TODO list for some time. About time to do it...thanks. ///Peter - To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org For additional commands, e-mail: users-h...@cocoon.apache.org
Re: Too many open files
Hi all, if you're in a Cocoon environment, it's always a good idea to avoid the document function. Cocoon offers so many better ways like map:aggregate or the CInclude transformer preserving the cache functionalities. Greetings Andreas From: Johan Cwiklinski To: users@cocoon.apache.org Sent: Thu, December 16, 2010 2:03:25 PM Subject: Re: Too many open files Hello, Le 16/12/2010 13:58, Peter Flynn a écrit : >> Caused by: java.net.SocketException: Too many open files >> at java.net.Socket.createImpl(Socket.java:388) >> at java.net.Socket.connect(Socket.java:517) >> at java.net.Socket.connect(Socket.java:469) >> at sun.net.NetworkClient.doConnect(NetworkClient.java:157) >> at sun.net.www.http.HttpClient.openServer(HttpClient.java:394) > > In my ignorance of the internals of Cocoon and Tomcat and Java, is it > running out of file handles when making a HTTP call? > > The server is lightly loaded, but each page does open a significant > number of document()s: is there a setting that will increase the number > of open files allowed? We already have seen this issue on some cocoon applications ; most of the time, that was due to intensive use of xpath "document" function with. We observe that using saxon as xslt processor genrerally solved that issue. Most of these issues has been resolved for us doing that ; some are always present but I did not have time to further investigate yet. > > Is there any way to get it to report *what* it was trying to open (the > URI) when it failed? > > ///Peter > Hope that could help you. Regards, -- Johan Cwiklinski AJLSM - To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org For additional commands, e-mail: users-h...@cocoon.apache.org
Re: Too many open files
On 16/12/10 14:39, Andreas Kuehne wrote: > Hi all, > > if you're in a Cocoon environment, it's always a good idea to avoid the > document function. Cocoon offers so many better ways like map:aggregate > or the CInclude transformer preserving the cache functionalities. Thank you, that sounds like a useful tip, although I wasn't aware of either of them. The problem is that the XSLT I am using also needs to run standalone (ie outside Cocoon) for other reasons, so I was trying to avoid too much Cocoon dependency. ///Peter - To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org For additional commands, e-mail: users-h...@cocoon.apache.org
Re: Too many open files
I hit this problem not too long ago and decided the o/s default limit of 1024 was too low for what we were trying to do. For me, on Redhat, the solution was to edit /etc/security/limits.conf tomcatuser hard nofile 65535 tomcatuser soft nofile 65535 --- On Thu, 12/16/10, Peter Flynn wrote: > From: Peter Flynn > Subject: Re: Too many open files > To: users@cocoon.apache.org > Date: Thursday, December 16, 2010, 4:00 PM > On 16/12/10 14:39, Andreas Kuehne > wrote: > > Hi all, > > > > if you're in a Cocoon environment, it's always a good > idea to avoid the > > document function. Cocoon offers so many better ways > like map:aggregate > > or the CInclude transformer preserving the cache > functionalities. > > Thank you, that sounds like a useful tip, although I wasn't > aware of > either of them. The problem is that the XSLT I am using > also needs to > run standalone (ie outside Cocoon) for other reasons, so I > was trying to > avoid too much Cocoon dependency. > > ///Peter > > - > To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org > For additional commands, e-mail: users-h...@cocoon.apache.org > > - To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org For additional commands, e-mail: users-h...@cocoon.apache.org
RE: Too many open files
Yes, had that too in the past, ulimit -H -n 65535 cured the issue. Laurent • This email and any files transmitted with it are CONFIDENTIAL and intended solely for the use of the individual or entity to which they are addressed. • Any unauthorized copying, disclosure, or distribution of the material within this email is strictly forbidden. • Any views or opinions presented within this e-mail are solely those of the author and do not necessarily represent those of Odyssey Financial Technologies SA unless otherwise specifically stated. • An electronic message is not binding on its sender. Any message referring to a binding engagement must be confirmed in writing and duly signed. • If you have received this email in error, please notify the sender immediately and delete the original.
Re: Too many open files
On 16/12/10 16:49, john muth wrote: > > I hit this problem not too long ago and decided the o/s default limit of 1024 > was too low for what we were trying to do. > > For me, on Redhat, the solution was to edit /etc/security/limits.conf > > tomcatuser hard nofile 65535 > tomcatuser soft nofile 65535 On 17/12/10 08:41, Laurent Medioni wrote: > Yes, had that too in the past, ulimit -H -n 65535 cured the issue. Thanks, I'll try. And presumably reboot the server. ///Peter - To unsubscribe, e-mail: users-unsubscr...@cocoon.apache.org For additional commands, e-mail: users-h...@cocoon.apache.org
Re: error: Too Many Open Files
On 06.11.2006 23:19, Bokluci wrote: From Time to time cocoon shows "Too many open files" error. I've already set ulimit to ulimit -n 8192 , but it still shows from time to time this error... Any suggestions? I'm not aware of any "resource leak" in Cocoon itself. Are you doing anything special? Opening and handling many files or anything like that? For example, are you releasing all resolved sources you obtained via a SourceResolver. Jörg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: error: Too Many Open Files
I'm doing big searches trough lot of files (3-400)... I'm using simply "document" function, is there any release function I'm missing? Vik -Original Message- From: Joerg Heinicke [mailto:[EMAIL PROTECTED] Sent: Monday, November 06, 2006 11:45 PM To: users@cocoon.apache.org Subject: Re: error: Too Many Open Files On 06.11.2006 23:19, Bokluci wrote: >>From Time to time cocoon shows "Too many open files" error. > I've already set ulimit to > > ulimit -n 8192 > > , but it still shows from time to time this error... > Any suggestions? I'm not aware of any "resource leak" in Cocoon itself. Are you doing anything special? Opening and handling many files or anything like that? For example, are you releasing all resolved sources you obtained via a SourceResolver. Jörg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: error: Too Many Open Files
On 07.11.2006 00:03, Bokluci wrote: I'm doing big searches trough lot of files (3-400)... I'm using simply "document" function, is there any release function I'm missing? This means document function in XSLT? Or something different? Can you describe your technical concept a bit? Maybe this points to a problem with document in XSLT - which is rarely used in Cocoon environments. Jörg -Original Message- From: Joerg Heinicke [mailto:[EMAIL PROTECTED] Sent: Monday, November 06, 2006 11:45 PM To: users@cocoon.apache.org Subject: Re: error: Too Many Open Files On 06.11.2006 23:19, Bokluci wrote: >From Time to time cocoon shows "Too many open files" error. I've already set ulimit to ulimit -n 8192 , but it still shows from time to time this error... Any suggestions? I'm not aware of any "resource leak" in Cocoon itself. Are you doing anything special? Opening and handling many files or anything like that? For example, are you releasing all resolved sources you obtained via a SourceResolver. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: error: Too Many Open Files
Exactly. It looks for previous and next file in a file with xml tree, then looks for special info in these files. I'm not very good with xslt, but as far as I know the only sitable function for my case is document function... Or I'm wrong? Why document is not used in cocoon env? Which function is used instead? Vik -Original Message- From: Joerg Heinicke [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 07, 2006 12:13 AM To: users@cocoon.apache.org Subject: Re: error: Too Many Open Files On 07.11.2006 00:03, Bokluci wrote: > I'm doing big searches trough lot of files (3-400)... I'm using simply > "document" function, is there any release function I'm missing? This means document function in XSLT? Or something different? Can you describe your technical concept a bit? Maybe this points to a problem with document in XSLT - which is rarely used in Cocoon environments. Jörg > -Original Message- > From: Joerg Heinicke [mailto:[EMAIL PROTECTED] > Sent: Monday, November 06, 2006 11:45 PM > To: users@cocoon.apache.org > Subject: Re: error: Too Many Open Files > > On 06.11.2006 23:19, Bokluci wrote: >> >From Time to time cocoon shows "Too many open files" error. >> I've already set ulimit to >> >> ulimit -n 8192 >> >> , but it still shows from time to time this error... >> Any suggestions? > > I'm not aware of any "resource leak" in Cocoon itself. Are you doing > anything special? Opening and handling many files or anything like that? > For example, are you releasing all resolved sources you obtained via a > SourceResolver. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: error: Too Many Open Files
Hi, i've seen this happen in combination with lucene quite often. Might that be your problem? Regards, Jeroen Reijn Joerg Heinicke wrote: On 06.11.2006 23:19, Bokluci wrote: From Time to time cocoon shows "Too many open files" error. I've already set ulimit to ulimit -n 8192 , but it still shows from time to time this error... Any suggestions? I'm not aware of any "resource leak" in Cocoon itself. Are you doing anything special? Opening and handling many files or anything like that? For example, are you releasing all resolved sources you obtained via a SourceResolver. Jörg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: error: Too Many Open Files
No, it's not my case... does cocoon not automatically close files? -Original Message- From: Jeroen Reijn [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 07, 2006 12:30 AM To: users@cocoon.apache.org Subject: Re: error: Too Many Open Files Hi, i've seen this happen in combination with lucene quite often. Might that be your problem? Regards, Jeroen Reijn Joerg Heinicke wrote: > On 06.11.2006 23:19, Bokluci wrote: >>> From Time to time cocoon shows "Too many open files" error. >> I've already set ulimit to >> >> ulimit -n 8192 >> >> , but it still shows from time to time this error... >> Any suggestions? > > I'm not aware of any "resource leak" in Cocoon itself. Are you doing > anything special? Opening and handling many files or anything like that? > For example, are you releasing all resolved sources you obtained via a > SourceResolver. > > Jörg > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: error: Too Many Open Files
Yes cocoon should automatically close the files, unless you use the document function a lot. Can't you generate the XML to search in from cocoon instead of using the document function? Reijn Bokluci wrote: No, it's not my case... does cocoon not automatically close files? -Original Message- From: Jeroen Reijn [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 07, 2006 12:30 AM To: users@cocoon.apache.org Subject: Re: error: Too Many Open Files Hi, i've seen this happen in combination with lucene quite often. Might that be your problem? Regards, Jeroen Reijn Joerg Heinicke wrote: On 06.11.2006 23:19, Bokluci wrote: From Time to time cocoon shows "Too many open files" error. I've already set ulimit to ulimit -n 8192 , but it still shows from time to time this error... Any suggestions? I'm not aware of any "resource leak" in Cocoon itself. Are you doing anything special? Opening and handling many files or anything like that? For example, are you releasing all resolved sources you obtained via a SourceResolver. Jörg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: error: Too Many Open Files
No... those are xml files with a lot of info in them... can't be automatically generated... How does cocoon make difference between using document function "a lot" and not so much... -Original Message- From: Jeroen Reijn [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 07, 2006 12:38 AM To: users@cocoon.apache.org Subject: Re: error: Too Many Open Files Yes cocoon should automatically close the files, unless you use the document function a lot. Can't you generate the XML to search in from cocoon instead of using the document function? Reijn Bokluci wrote: > No, it's not my case... does cocoon not automatically close files? > > -Original Message- > From: Jeroen Reijn [mailto:[EMAIL PROTECTED] > Sent: Tuesday, November 07, 2006 12:30 AM > To: users@cocoon.apache.org > Subject: Re: error: Too Many Open Files > > Hi, > > i've seen this happen in combination with lucene quite often. Might that > be your problem? > > Regards, > > Jeroen Reijn > > Joerg Heinicke wrote: >> On 06.11.2006 23:19, Bokluci wrote: >>>> From Time to time cocoon shows "Too many open files" error. >>> I've already set ulimit to >>> >>> ulimit -n 8192 >>> >>> , but it still shows from time to time this error... >>> Any suggestions? >> I'm not aware of any "resource leak" in Cocoon itself. Are you doing >> anything special? Opening and handling many files or anything like that? >> For example, are you releasing all resolved sources you obtained via a >> SourceResolver. >> >> Jörg >> >> - >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: error: Too Many Open Files
I've heard of a java bug that caused this message. What Java version are you using and what happens if you upgrade to the latest? (if possible) Kind regards, Geert > Drs. G.P.H. Josten Consultant Daidalos BV Source of Innovation Hoekeindsehof 1-4 2665 JZ Bleiswijk Tel.: +31 (0) 10 850 1200 Fax: +31 (0) 10 850 1199 www.daidalos.nl KvK 27164984 De informatie - verzonden in of met dit emailbericht - is afkomstig van Daidalos BV en is uitsluitend bestemd voor de geadresseerde. Indien u dit bericht onbedoeld hebt ontvangen, verzoeken wij u het te verwijderen. Aan dit bericht kunnen geen rechten worden ontleend. > Van: Bokluci [mailto:[EMAIL PROTECTED] > Verzonden: maandag 6 november 2006 23:19 > Aan: users@cocoon.apache.org > Onderwerp: error: Too Many Open Files > > > From Time to time cocoon shows "Too many open files" error. > I've already set ulimit to > > ulimit -n 8192 > > , but it still shows from time to time this error... > Any suggestions? > > Thanks in advance. > Vik > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: "Too many open files" error
> I am running a single Cocoon app on the server, and > am the only one testing it... despite this, I get a: > > cause: java.io.FileNotFoundException: > /usr/local/jakarta-tomcat-4.1.31/webapps/cocoon/myapp/myfile.xml (Too > many open files) > > error when I try and load CForm-based page. > > The server is Linux-based, but on my test machine > (Windows) I did not notice this problem. Check ulimit -n on the linux box - you may need to increase this setting in the shell script which launches Tomcat (if you can). jon This electronic message contains information which may be privileged and confidential. The information is intended to be for the use of the individual(s) or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this electronic message in error, please notify us by telephone on 0131 476 6000 and delete the material from your computer. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
No lucene, but "Too many open files"
Hi there! I'm running Cocoon 2.1.4 within Tomcat 5.0.19 on Linux. I currently face the problem that cocoon stops to work with various exceptions (i.e. FileNotFoundException), all stating "too many open files". I tracked down the problem to the use of the xpath function 'document($phrase_list)' in one of my XSLT stylesheets. (I have to look-up some additional data in a separate XML file while transforming.) It seems that cocoon opens the referenced XML file every time document() is called, resulting in the creation of a new file descriptor for this file. A new file descriptor is even created if the same file within the same transformation has already been opened. And even worse these files stay open -- for a while at least. Normally there wouldn't be a problem, but if requests come in to frequently the limit of 1024 open files is reached quite soon. And after that Cocoon will not recover any more... Is there a possibility to configure that these files shall be closes earlier? Or is the access via document() handled entirely by the XSL-Transformer (i.e. Xalan)? Are there any other suggestions how to implement looking-up data within other (quite large) XML files while transforming. Thanks in advance. Daniel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: No lucene, but "Too many open files"
did you check bug 31116 ? There are a few suggestions as to what could cause this. HTH Jorg Huber, Daniel wrote: Hi there! I'm running Cocoon 2.1.4 within Tomcat 5.0.19 on Linux. I currently face the problem that cocoon stops to work with various exceptions (i.e. FileNotFoundException), all stating "too many open files". I tracked down the problem to the use of the xpath function 'document($phrase_list)' in one of my XSLT stylesheets. (I have to look-up some additional data in a separate XML file while transforming.) It seems that cocoon opens the referenced XML file every time document() is called, resulting in the creation of a new file descriptor for this file. A new file descriptor is even created if the same file within the same transformation has already been opened. And even worse these files stay open -- for a while at least. Normally there wouldn't be a problem, but if requests come in to frequently the limit of 1024 open files is reached quite soon. And after that Cocoon will not recover any more... Is there a possibility to configure that these files shall be closes earlier? Or is the access via document() handled entirely by the XSL-Transformer (i.e. Xalan)? Are there any other suggestions how to implement looking-up data within other (quite large) XML files while transforming. Thanks in advance. Daniel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: No lucene, but "Too many open files"
> > I currently face the problem that cocoon stops to work with various > > exceptions (i.e. FileNotFoundException), all stating "too many open > > files". > > [...] > > It seems that cocoon opens the referenced XML file every > > time document() is called, resulting in the creation of a > > new file descriptor for this file. A new file descriptor > > is even created if the same file within the same > > transformation has already been opened. And even worse > > these files stay open -- for a while at least. > did you check bug 31116 ? There are a few suggestions as to > what could cause this. Now ... yes. Thank you for the hint. But even though the bug has status 'resolved', I can't see any suggestions which solve the root of the problem. INHO only the symptoms are addressed. Tuning the garbage collector won't be a *real* solution. It just hides the problem some kind. As a work-around using aggregation instead of document() could work, although this would be a bit awkward when including XML files just for looking-up some values, when the included tree won't be output to the resulting document... Another strange thing is, that not all XML files I use for look-ups stay open. I have one other file which I access only once within a transformation. It never shows up in the list of open files. The other will show up about 30 times just after processing a single request! Regards, Daniel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: No lucene, but "Too many open files"
> ["too many open files" in conjunction with xpath function document()] Another weired part of the puzzle is this posting (I finally found): http://issues.apache.org/eyebrowse/[EMAIL PROTECTED] che.org&msgId=729039 It describes quite the same problem. As solution upgrading to Xalan 2.5.1 and Xerces 2.4.0 was described. In my Cocoon 2.1.4 there are already more recent version included (Xalan 2.5.2 and Xerces 2.6.1). It is not very likly that they re-implemented an old bug!? So is it really a problem of Xalan... - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: No lucene, but "Too many open files"
Huber, Daniel wrote: ["too many open files" in conjunction with xpath function document()] Another weired part of the puzzle is this posting (I finally found): http://issues.apache.org/eyebrowse/[EMAIL PROTECTED] che.org&msgId=729039 It describes quite the same problem. As solution upgrading to Xalan 2.5.1 and Xerces 2.4.0 was described. In my Cocoon 2.1.4 there are already more recent version included (Xalan 2.5.2 and Xerces 2.6.1). It is not very likly that they re-implemented an old bug!? So is it really a problem of Xalan... Have you checked their mailinglist? There must be a workaround for a problem of this magnitude. Jorg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: No lucene, but "Too many open files"
Huber, Daniel wrote: Hi there! I'm running Cocoon 2.1.4 within Tomcat 5.0.19 on Linux. I currently face the problem that cocoon stops to work with various exceptions (i.e. FileNotFoundException), all stating "too many open files". Hi Daniel, What flavour/version of linux are you using? We had a similar problem on some older versions of redhat, but found that for some reason it didn't occur on our boxes running redhat WS 3.0. We installed the newer OS on the offending boxes and haven't had the problem since. It's not an ideal solution for your problem, but might be something worth looking at. HTH, Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: No lucene, but "Too many open files"
> What flavour/version of linux are you using? We had a similar > problem on some older versions of redhat, but found that for > some reason it didn't occur on our boxes running redhat WS 3.0. > We installed the newer OS on the offending boxes and haven't > had the problem since. It's not an ideal solution for your > problem, but might be something worth looking at. Currently it's some kind of SuSE (2.4.20-64GB-SMP). But it is definitely not an issue of the operation system. I can run the whole system on Windows, where I see the same dangling file handles. The only cause why the problem does not get visible on Windows is that there is no default-limit of 1024 for open files per process. The effect you experienced might be due to other default-settings (ulimit) on the other distribution you used. I decided to do two things now: First, I will assure that there is no older Xalan/Xerces version in the classpath. Second if the problem can't be fixed with Xalan, I will try to replace the usage of xpath's document() function with Cocoon's Xinclude transformer where possible. Thank you for your hints. Regards, Daniel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: No lucene, but "Too many open files"
Hi I have had a similar problem running on a Linux box. I tried changing the open files limit (ulimit -n) from 1024 to 2048 but it seemed to make no difference at all. Mandrake Linux 9.1 Java build 1.4.2_03-b02 Apache 2.0.47 Tomcat 4.1.27-2 Cocoon 2.1.5.1 xalan 2.5.2 xerces 2.6.1 open file size is 1024 I have run the same site on an Apple G5: Darwin Kernel Version 7.5.0 (BSD Unix) Java build 1.4.2_05-141.3 Apache 2.0.52 Tomcat 4.1.24 Cocoon 2.1.5.1 xalan 2.5.2 xerces 2.6.1 open file size is 256 There is absolutely no problem with it running here. I have run it for hours at high levels of stress and there were no errors reported at all. Very curious. Forcing the Linux based system to use endorsed jar files for the latest xalan/xerces instead of the Java supplied ones (which seemed to be the same version anyway) did not give any noticeable improvement. Unfortunately I cannot use aggregation as I too am taking variable file names into document(...) calls in the XSL. I can only suggest that replacing the XSLT transformer that requires variable document calls with a special Java transformer may remove the problem. Hugh F-R Dr Hugh S. Field-Richards QinetiQ, St Andrew's Road, Malvern, Worcs, WR14 3PS, UK Tel: ++1684 895075 Fax: ++1684 896113 Email: [EMAIL PROTECTED] The views expressed above area are entirely those of the writer and do not represent the views, policy or understanding of any other person or official body. On 20 Sep 2004, at 10:29, Huber, Daniel wrote: What flavour/version of linux are you using? We had a similar problem on some older versions of redhat, but found that for some reason it didn't occur on our boxes running redhat WS 3.0. We installed the newer OS on the offending boxes and haven't had the problem since. It's not an ideal solution for your problem, but might be something worth looking at. Currently it's some kind of SuSE (2.4.20-64GB-SMP). But it is definitely not an issue of the operation system. I can run the whole system on Windows, where I see the same dangling file handles. The only cause why the problem does not get visible on Windows is that there is no default-limit of 1024 for open files per process. The effect you experienced might be due to other default-settings (ulimit) on the other distribution you used. I decided to do two things now: First, I will assure that there is no older Xalan/Xerces version in the classpath. Second if the problem can't be fixed with Xalan, I will try to replace the usage of xpath's document() function with Cocoon's Xinclude transformer where possible. Thank you for your hints. Regards, Daniel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Too many open files (was PDF Image Caching Problem)
2008/5/5 Lehtonen, Mika <[EMAIL PROTECTED]>: > Actually there might be another problem (or bug) lying around if you > produce htmls' with 'noncaching' option. Cocoon seems to be leaving files > open when using html-serializer with noncaching option in your pipeline. And > eventually your application can crash because you exceed the maximum open > files that your OS tolerates. (Cocoon 2.1.11 / Centos 4.6) Really? Aha... It's not the cocoon.xconf that's left open is it? And could this affect earlier versions too? I mailed the dev list about a week ago (subject "Excessive file handles on cocoon.xconf", see http://www.nabble.com/Excessive-file-handles-on-cocoon.xconf-tp16973172p17000464.html) because we were encountering "Too many open files" errors after migrating a 2.1.7 app from Websphere 5.0 on Windows to 6.0 on Linux. I'm wondering if this could be the reason. Do the files get closed eventually? We were getting these errors with the limit set to 1024, but after a few minutes things would start working again for a while. Also, after increasing the limit to 8000 we haven't seen the error since. I assume this is because it's now high enough for the responsible component(s) to get destroyed, garbage collected, and basically cleaned up properly before things get that bad. Andy -- http://pseudoq.sourceforge.net/ Open source java Sudoku application - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Too many open files (was PDF Image Caching Problem)
Hi Andy, it sounds like you are having slightly different malfunction than me. If I recall right, the files that were left open, were xmls' including some content that should have been fetched into the page. There were hundreds of tags in those xml's and fetching every single one of them opened the xml again and left it open. I can't recall for sure if they were eventually released but clicking browser's refresh button did that. Switching caching on changed the situation so that the files were opened only per first request and at the next time for the same request, no files were opened. mika Andy Stevens kirjoitti: 2008/5/5 Lehtonen, Mika <[EMAIL PROTECTED]>: Actually there might be another problem (or bug) lying around if you produce htmls' with 'noncaching' option. Cocoon seems to be leaving files open when using html-serializer with noncaching option in your pipeline. And eventually your application can crash because you exceed the maximum open files that your OS tolerates. (Cocoon 2.1.11 / Centos 4.6) Really? Aha... It's not the cocoon.xconf that's left open is it? And could this affect earlier versions too? I mailed the dev list about a week ago (subject "Excessive file handles on cocoon.xconf", see http://www.nabble.com/Excessive-file-handles-on-cocoon.xconf-tp16973172p17000464.html) because we were encountering "Too many open files" errors after migrating a 2.1.7 app from Websphere 5.0 on Windows to 6.0 on Linux. I'm wondering if this could be the reason. Do the files get closed eventually? We were getting these errors with the limit set to 1024, but after a few minutes things would start working again for a while. Also, after increasing the limit to 8000 we haven't seen the error since. I assume this is because it's now high enough for the responsible component(s) to get destroyed, garbage collected, and basically cleaned up properly before things get that bad. Andy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
