Re: apache & log files

2004-11-04 Thread Martin Kos
On 04.11.2004 23:09, Craig Sanders wrote:
For ErrorLog you can pipe to a suitable program which does the same.
but this doesn't.  unless apache has added this feature since i last looked
into this (about six months ago) the "suitable program" has no way of
separating the error logs for each virtual host, because it's just STDERR with
no vhost prefix on each line.
does somebody know if this thing was changed in apache 2 ?
greets
 KoS
--
Martin Kos   +41-76-384-93-33
http://kos.liSay NO to HTML in mail ICQ# 13556143
Proudly running Debian GNU/Linux
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]


Re: apache & log files

2004-11-04 Thread Craig Sanders
On Thu, Nov 04, 2004 at 11:19:22AM +0100, Francesco P. Lovergine wrote:
> I personally prefer a single CustomLog file with a suitable domain
> prefix for every domain. That allows a nice grepping to extract 
> information and avoid resources wasting. 

yes, this works.

> For ErrorLog you can pipe to a suitable program which does the same.

but this doesn't.  unless apache has added this feature since i last looked
into this (about six months ago) the "suitable program" has no way of
separating the error logs for each virtual host, because it's just STDERR with
no vhost prefix on each line.

craig

-- 
craig sanders <[EMAIL PROTECTED]>   (part time cyborg)


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]



Re: apache & log files

2004-11-04 Thread Russell Coker
On Thursday 04 November 2004 09:11, Marek Podmaka <[EMAIL PROTECTED]> wrote:
>   I have apache 1.3 webserver hosting about 150 domains (more than 400
>   virtual hosts). Now I have separate error log for each domain

My clftools package allows you to split and mangle the log files if you have 
Apache configured for a single log file...

-- 
http://www.coker.com.au/selinux/   My NSA Security Enhanced Linux packages
http://www.coker.com.au/bonnie++/  Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/Postal SMTP/POP benchmark
http://www.coker.com.au/~russell/  My home page


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]



Re: apache & log files

2004-11-04 Thread Soenke von Stamm
Instead of setting ulimit in an init script, you can set it in the 
file /etc/security/limits.conf

like this:
www-data   hardnofile  4096
www-data   softnofile  8192

I'm not sure what hard and soft really mean, they're discribed as:
# can have the two values:
#- "soft" for enforcing the soft limits
#- "hard" for enforcing hard limits

but I don't know what kind of a difference they make.

/proc/sys/fs/file-max is 26214 on that system, I'm not sure if it's influenced 
by the contents of limits.conf ... On another system it's even higher though 
I've not changed anything. Anybody knows what changes this value?


 Sönke

Am Mittwoch, 3. November 2004 23:42 schrieb Craig Sanders:
> On Wed, Nov 03, 2004 at 11:11:13PM +0100, Marek Podmaka wrote:
> >   I have apache 1.3 webserver hosting about 150 domains (more than 400
> >   virtual hosts). Now I have separate error log for each domain
> >   (something.sk) and separate combined log for each virtual host (for
> >   example www.abcq.sk and new.abcq.sk). This has many positives for
> >   me: easy to find some data related to each virtual host and that I
> >   can make seaparate statistics for each virtual host. I use awstats.
> >   And now the bad side - the number of open files in each apache
> >   process is more than 500 just for these log files. It's no problem
> >   for now, but with more domains in future it will hit the 1024 per
> >   process limit of open files.
> >
> >   And now the questions :)
> >   1) Where does that 1024 open files limit come from? Is it somewhere
> >   configurable?
>
> edit /etc/init.d/apache and add a line like "ulimit -n 4096"
>
> "-n" is the maximum number of open file descriptors per process.  use a
> value that's about twice as much as you think you'll need.
>
> you also need to set /proc/sys/fs/file-max to some suitably high value
> (again, calculate how many you think you'll need and double it).  this can
> be set in /etc/sysctl.conf
>
> > Or do you think it's totally bad idea to have such number of log
> > files?
>
> until recently, i ran a web server with about 600 virtual hosts on it, each
> with its own access.log and error.log files.
>
> with 200 max apache children, that worked out as up to about 240,000 (600 x
> 200 x 2) file handles opened by apache processes for logging at any given
> time. this was on a dual p3-866 with 512MB RAM.   it worked.
>
> it bothered me a little that it wasn't really scalable and that eventually
> i'd have to do something about logging.  i had some ideas on what to do,
> but was limited by the fact that i wanted to have separate error.log files
> for each virtual host.  overall, though, my attitude was "it aint broke, so
> don't fix it".
>
> this wouldn't be a problem if apache could be configured to prefix each
> error.log line with the virtual host's domain name..then you could have
> a single pipe to an error logging script which wrote the error lines to the
> right file, same as you can do for the access.log.
>
> but apache can't be configured to do that, and i never bothered looking at
> the source to see how easy it would be to hack it in, so that means you
> either have a shared error.log for all vhosts or you put up with having
> lots of open file handles.  i chose the latter, and occasionally increased
> both "ulimit -n" and /proc/sys/fs/file-max as requiredi never did run
> into any limit.
>
>
> craig
>
> --
> craig sanders <[EMAIL PROTECTED]>   (part time cyborg)

-- 
Sönke von Stamm
System Administration

[EMAIL PROTECTED]
Tel.: +49 511 - 35 77 19 - 29
Fax.: +49 511 - 35 77 19 - 19

Imedia Gesellschaft für neue Medien mbH
Limmerstr. 46 | 30451 Hannover



Re: apache & log files

2004-11-04 Thread Francesco P. Lovergine
On Wed, Nov 03, 2004 at 11:11:13PM +0100, Marek Podmaka wrote:
> Hello,
> 
>   I have apache 1.3 webserver hosting about 150 domains (more than 400
>   virtual hosts). Now I have separate error log for each domain
>   (something.sk) and separate combined log for each virtual host (for
>   example www.abcq.sk and new.abcq.sk). This has many positives for
>   me: easy to find some data related to each virtual host and that I
>   can make seaparate statistics for each virtual host. I use awstats.
>   And now the bad side - the number of open files in each apache
>   process is more than 500 just for these log files. It's no problem
>   for now, but with more domains in future it will hit the 1024 per
>   process limit of open files.
> 
>   3) Or you maybe use some other solution to this problem?
> 

I personally prefer a single CustomLog file with a suitable domain
prefix for every domain. That allows a nice grepping to extract 
information and avoid resources wasting. For ErrorLog you can pipe
to a suitable program which does the same.

-- 
Francesco P. Lovergine


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]