[us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-19 Thread Ben Welsh
Hello, I'm managing a pgsql --> django --> mod_python --> apache machine that serves no media, but handles the postgres memcached and apache part of the stack all by itself. It's a dedicated virtual server with Red Hat EL and 1GB RAM. As traffic to the site has ramped up over time, I've begun to

Re: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-20 Thread Igor Cicimov
Have you considered increasing the open files limit? Have a look at the ulimit command. The default is sometimes not enough. Cheers, Igor On Wed, May 20, 2009 at 4:00 PM, Ben Welsh wrote: > Hello, > > I'm managing a pgsql --> django --> mod_python --> apache machine that > serves no media, but

Re: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-20 Thread palewire
For what it's worth, I've taken the additional step of running `lsof |grep httpd > http.dump` and posting it online. My understanding is that this shows all of the httpd open files that are adding up to my high totals. http://gist.github.com/114943 If anyone who understands this better than me h

Re: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-20 Thread palewire
Igor Cicimov wrote: > > Have you considered increasing the open files limit? Have a look at the > ulimit command. The default is sometimes not enough. > > Cheers, > > Igor > Thx for your time and attention. I have considered fiddling with the open files liimits, but frankly before I go too

Re: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-20 Thread André Warnier
palewire wrote: For what it's worth, I've taken the additional step of running `lsof |grep httpd > http.dump` and posting it online. My understanding is that this shows all of the httpd open files that are adding up to my high totals. http://gist.github.com/114943 If anyone who understands this

Re: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-20 Thread Ben Welsh
Thanks for your thoughts Andre. As I recollect, I think some of the these errors started cropping up around the time I compiled the python module numpy on the server. And I think that fortran and a lot of those deadweight file objects are its children. Perhaps if I scaled by the server to no longer

Re: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-20 Thread André Warnier
Ben Welsh wrote: Thanks for your thoughts Andre. As I recollect, I think some of the these errors started cropping up around the time I compiled the python module numpy on the server. And I think that fortran and a lot of those deadweight file objects are its children. Perhaps if I scaled by the

Re: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-20 Thread palewire
awarnier wrote: > > I could not say that. Only you really know which modules you load with > the LoadModule directives, and which ones you really need for your > configuration and applications. > > Taking into account that libraries (stuff like the > /usr/lib/libkrb5.so.3.3 for instance) ar

Re: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-20 Thread Sean Conner
It was thus said that the Great Andr Warnier once stated: > > Another thing : it looks from your lsof list, that you are using the > Apache "prefork" model. > I don't remember precisely your configuration or the kind of load or > processes you are running, but you might try the "worker" (threade

Re: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-20 Thread palewire
Sean -- Thanks for the lengthy and thoughtful response. What I've found most successful so far is hunting through the lsof | grep http and finding ways to eliminate unnecessary files. And I'm also exploring an expansion of my production resources to include a second tier machine that houses the

RE: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-21 Thread Mick Sheppard
ctivity to databases that this will increase the number of sockets, and therefore open files, used for each page served. Mick -Original Message- From: palewire [mailto:ben.we...@gmail.com] Sent: 20 May 2009 23:55 To: users@httpd.apache.org Subject: Re: [us...@httpd] What is the best way

RE: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-21 Thread palewire
pen files, used for each > page served. > > Mick > > -Original Message- > From: palewire [mailto:ben.we...@gmail.com] > Sent: 20 May 2009 23:55 > To: users@httpd.apache.org > Subject: Re: [us...@httpd] What is the best way to handle "too many open > fi

Re: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-21 Thread Sean Conner
It was thus said that the Great Mick Sheppard once stated: > Hi, > > Just to throw a slight spanner in the works here. My understanding of > 'open files' is open file descriptors. As far as a file descriptor is > concerned there is no real difference between a physical file on disk > and a socket

Re: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-22 Thread Ben Welsh
I just had something of a "slap your forehead" moment on this one. It had be eating me about where all those extra lsof processes came from. And then it hit me. It's the "developer tools" kit from CentOS. Duh. http://www.linuxtopia.org/online_books/centos_linux_guides/centos_linux_developer_tools_

Re: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-23 Thread André Warnier
Sean Conner wrote: But I would wait a few hours for a real expert to comment, which I'm sure one will do if I wrote something really stupid above. Thanks, Sean, for the clarification. Usweful for me too. - The official Use

Re: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-23 Thread Sean Conner
It was thus said that the Great Ben Welsh once stated: > I just had something of a "slap your forehead" moment on this one. It had be > eating me about where all those extra lsof processes came from. And then it > hit me. It's the "developer tools" kit from CentOS. Duh. > > http://www.linuxtopia.o

Re: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-23 Thread Ben Welsh
When I study my lsof logs on httpd instances, I'm seeing the use of a number of image libraries, or instance, that I cannot imagine any use for in my applications. then when I look at the yum log the DV provisioned by my host I see those same libraries as default installs. To me there seem to be tw

Re: [us...@httpd] What is the best way to handle "too many open files" errors?

2009-05-26 Thread Sean Conner
It was thus said that the Great Ben Welsh once stated: > When I study my lsof logs on httpd instances, I'm seeing the use of a number > of image libraries, or instance, that I cannot imagine any use for in my > applications. then when I look at the yum log the DV provisioned by my host > I see thos