Lamar Owen wrote:
Anyway, Syslog is not an option for us. We need flat files.
Ok, riddle me this:
If I have PostgreSQL set to log to syslog facility LOCAL0, and a local0.none
on /var/log/messages and local0.* to /var/log/pgsql (assuming only one
postmaster, unfortunately) then you get a flat
On Wednesday 24 March 2004 12:31, Bruce Momjian wrote:
Andrew Dunstan wrote:
This thread seems to have died without a conclusion. AFAICS, we have 5
options:
. the apache program - see below
pro: robust, portable, extremely well tested, no effort to import
con: possible license
Robert Treat wrote:
On Wednesday 24 March 2004 12:31, Bruce Momjian wrote:
Andrew Dunstan wrote:
This thread seems to have died without a conclusion. AFAICS, we have 5
options:
. the apache program - see below
pro: robust, portable, extremely well tested, no effort to import
This thread seems to have died without a conclusion. AFAICS, we have 5
options:
. the apache program - see below
pro: robust, portable, extremely well tested, no effort to import
con: possible license issues, limited features
. Peter Eisentraut's program
pro: portable, better featured, no
On Wed, Mar 24, 2004 at 09:55:34AM -0500, Andrew Dunstan wrote:
This thread seems to have died without a conclusion. AFAICS, we have 5
options:
. the apache program - see below
pro: robust, portable, extremely well tested, no effort to import
con: possible license issues, limited
On Wed, 24 Mar 2004, Andrew Dunstan wrote:
. Peter Eisentraut's program
pro: portable, better featured, no license issues
con: code state uncertain, less well tested
Where is Peter's code available, I'd like to try it out.
---(end of
Andrew Dunstan wrote:
This thread seems to have died without a conclusion. AFAICS, we have 5
options:
. the apache program - see below
pro: robust, portable, extremely well tested, no effort to import
con: possible license issues, limited features
. Peter Eisentraut's program
On Sunday 14 March 2004 1:00 pm, Tom Lane wrote:
...
So it seems fairly likely that the fsync-by-default business is
indeed a Linux-ism not shared by other Unixen.
Excerpt from the Postfix 2.0.8 README_FILES/LINUX_README file in case
it proves interesting:
-
LINUX syslogd uses
Fernando Nasser wrote:
Bruce Momjian wrote:
Manfred Spraul wrote:
Bruce Momjian wrote:
Which basically shows one fsync, no O_SYNC's, and setting of the flag
only for klog reads.
Which sysklogd do you look at? The version from RedHat 9 contains this
block:
I looked on
On Sat, Mar 13, 2004 at 10:45:35AM -0500, Rod Taylor wrote:
Not that I'm volunteering, but I think the biggest issue is many users
simply don't know how to approach the problem. Some docs on using
syslog, cron, etc. with PostgreSQL to accomplish maintenace jobs would
probably be enough.
There
Andrew Sullivan wrote:
On Sat, Mar 13, 2004 at 10:45:35AM -0500, Rod Taylor wrote:
Not that I'm volunteering, but I think the biggest issue is many users
simply don't know how to approach the problem. Some docs on using
syslog, cron, etc. with PostgreSQL to accomplish maintenace jobs would
Lamar Owen wrote:
On Saturday 13 March 2004 10:36 am, Fernando Nasser wrote:
The problem is that sysloging has more overhead than a plain append to a
file. There are some very strict response time AppServer applications
where we want to keep this things out of the picture.
Well, I have a
Lamar Owen wrote:
On Saturday 13 March 2004 01:00 pm, Fernando Nasser wrote:
There are some applicatons which run in servers with very strict
response times and any syscall, network packet that can be saved counts.
Ok, what about pipe overhead? If we're gong to split hairs, let's split all
of
Fernando Nasser [EMAIL PROTECTED] writes:
Lamar Owen wrote:
Third, it seems that you don't have enough profiling data to support
a 'syslog is bad' position.
That is true. It is from hearsay, from people who run production
environments. It may be a belief based on old experiences though.
On Sunday 14 March 2004 12:19, Tom Lane wrote:
Fernando Nasser [EMAIL PROTECTED] writes:
Lamar Owen wrote:
Third, it seems that you don't have enough profiling data to support
a 'syslog is bad' position.
That is true. It is from hearsay, from people who run production
environments.
Tom Lane wrote:
Fernando Nasser [EMAIL PROTECTED] writes:
Lamar Owen wrote:
Third, it seems that you don't have enough profiling data to support
a 'syslog is bad' position.
That is true. It is from hearsay, from people who run production
environments. It may be a belief based on
Bruce Momjian [EMAIL PROTECTED] writes:
What versions of syslog fsync, and where is the syslog.conf option. I
don't see it on FreeBSD or Linux.
It's a per-output-file option. My Linux manpage for syslogd quoth
The - may only be used to prefix a filename if you want to omit
My $0.02 worth: syslog-ng
:)
Regards,
John
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Bruce Momjian
Sent: Monday, March 15, 2004 5:50 AM
To: Tom Lane
Cc: Fernando Nasser; Lamar Owen; [EMAIL PROTECTED]
Subject: Re: [HACKERS] Log rotation
Tom Lane
Tom Lane wrote:
Bruce Momjian [EMAIL PROTECTED] writes:
What versions of syslog fsync, and where is the syslog.conf option. I
don't see it on FreeBSD or Linux.
It's a per-output-file option. My Linux manpage for syslogd quoth
The - may only be used to prefix a filename if you
Bruce Momjian [EMAIL PROTECTED] writes:
I wonder if this fsync for PostgreSQL messages is some change made to
Linux syslog.
You're missing the point: by default syslog fsyncs *all* messages.
You can turn this off on a per-output-file basis by putting - on the
desired lines of the syslog config
Tom Lane wrote:
Bruce Momjian [EMAIL PROTECTED] writes:
I wonder if this fsync for PostgreSQL messages is some change made to
Linux syslog.
You're missing the point: by default syslog fsyncs *all* messages.
You can turn this off on a per-output-file basis by putting - on the
desired
I wrote:
strace'ing syslogd on my HPUX box shows that it doesn't issue explicit
fsync calls either, but I don't know of any way to tell whether it's got
the log files opened O_SYNC.
It occurred to me to SIGHUP syslogd and watch it reopen its log files
with strace. No O_SYNC.
So it seems
Fernando Nasser wrote:
Lamar Owen wrote:
I am not opposed to including a small logrotator for stderr logging.
I just think it is redundant when a good highly configurable logging
facility already exists. But, if Red Hat wants to pay Tom to do
it... :-)
Maybe it is a question of
Bruce Momjian wrote:
Which basically shows one fsync, no O_SYNC's, and setting of the flag
only for klog reads.
Which sysklogd do you look at? The version from RedHat 9 contains this
block:
/*
* Crack a configuration file line
*/
void cfline(line, f)
char *line;
register
Manfred Spraul wrote:
Bruce Momjian wrote:
Which basically shows one fsync, no O_SYNC's, and setting of the flag
only for klog reads.
Which sysklogd do you look at? The version from RedHat 9 contains this
block:
I looked on NetBSD, FreeBSD, and BSD/OS.
--
Bruce Momjian
On Friday 12 March 2004 03:21 pm, Fernando Nasser wrote:
Lamar Owen wrote:
Uh, we have many many many different ways to use syslog. So my other
message on the topic.
Which other message?
The one I didn't post. :-)
Anyway, Syslog is not an option for us. We need flat files.
Ok, riddle
On Saturday 13 March 2004 09:36, Lamar Owen wrote:
On Friday 12 March 2004 03:21 pm, Fernando Nasser wrote:
Lamar Owen wrote:
Uh, we have many many many different ways to use syslog. So my other
message on the topic.
Which other message?
The one I didn't post. :-)
Anyway, Syslog
Bruno Wolff III wrote:
On Fri, Mar 12, 2004 at 15:19:29 -0500,
Fernando Nasser [EMAIL PROTECTED] wrote:
Bruno Wolff III wrote:
I can see their problem with making a dependency to all of apache or
including
multilog in their distribution. But they probably could include something
that is only a
Lamar Owen wrote:
Anyway, Syslog is not an option for us. We need flat files.
Ok, riddle me this:
If I have PostgreSQL set to log to syslog facility LOCAL0, and a local0.none
on /var/log/messages and local0.* to /var/log/pgsql (assuming only one
postmaster, unfortunately) then you get a
I can see that in a multipostmaster setting how you might want some
differentiation between postmasters, but ISTM that the tool reading these
logs should be trained in how to separate loglines out. snip
Different postmasters = different conf files. Just set your syslog_facility
and/or
Robert Treat wrote:
Different postmasters = different conf files. Just set your syslog_facility
and/or your syslog_ident differently and it should be pretty easy to seperate
the logs. Actually, now that I have started using syslog fairly regularly, I
find it hard to believe it would be worth
Rod Taylor wrote:
I can see that in a multipostmaster setting how you might want some
differentiation between postmasters, but ISTM that the tool reading these
logs should be trained in how to separate loglines out. snip
Different postmasters = different conf files. Just set your syslog_facility
On Sat, Mar 13, 2004 at 10:36:23AM -0500, Fernando Nasser wrote:
Lamar Owen wrote:
Ok, riddle me this:
If I have PostgreSQL set to log to syslog facility LOCAL0, and a
local0.none on /var/log/messages and local0.* to /var/log/pgsql (assuming
only one postmaster, unfortunately) then you
Patrick Welche wrote:
On Sat, Mar 13, 2004 at 10:36:23AM -0500, Fernando Nasser wrote:
Lamar Owen wrote:
Ok, riddle me this:
If I have PostgreSQL set to log to syslog facility LOCAL0, and a
local0.none on /var/log/messages and local0.* to /var/log/pgsql (assuming
only one postmaster,
On Saturday 13 March 2004 10:36 am, Fernando Nasser wrote:
The problem is that sysloging has more overhead than a plain append to a
file. There are some very strict response time AppServer applications
where we want to keep this things out of the picture.
Well, I have a couple of ideas on
On Saturday 13 March 2004 01:00 pm, Fernando Nasser wrote:
There are some applicatons which run in servers with very strict
response times and any syscall, network packet that can be saved counts.
Ok, what about pipe overhead? If we're gong to split hairs, let's split all
of them. The design
Tom Lane wrote:
Andrew Dunstan [EMAIL PROTECTED] writes:
Did anything ever come from this thread?
http://archives.postgresql.org/pgsql-hackers/2003-05/msg00603.php
(Heading: Plan B for log rotation support: borrow Apache code)
Only an entry on my depressingly long personal to-do list
Tom Lane wrote:
I did take a look at the Apache rotator program, and found that it
was probably more trouble to adopt than it's worth. It seemed to
depend on a lot of configuration and library-routine infrastructure
that we don't share.
Here's a log rotation program that does not share those
Tom Lane wrote:
Fernando Nasser [EMAIL PROTECTED] writes:
Please remind me again why the postmaster cannot close and open the log
file when it receives a SIGHUP (to re-read configuration)?
(a) Because it never opened it in the first place --- the log file is
whatever was passed as stderr.
(b)
On Fri, Mar 12, 2004 at 09:24:28 -0500,
Fernando Nasser [EMAIL PROTECTED] wrote:
I don't really care on how its done, but IMO an enterprise class
database must be able to do log rotation. Logging to Syslog is not an
option (specially with our verbosity) -- users must be able to use flat
Bruno Wolff III [EMAIL PROTECTED] writes:
Are you suggesting the that postgres project develop their own logger
rather than people just using one that has already been developed
by some other group?
The problem from the point of view of Red Hat is to not introduce a
dependency from the
On Fri, Mar 12, 2004 at 13:17:50 -0500,
Tom Lane [EMAIL PROTECTED] wrote:
Bruno Wolff III [EMAIL PROTECTED] writes:
Are you suggesting the that postgres project develop their own logger
rather than people just using one that has already been developed
by some other group?
The problem
On Friday 12 March 2004 09:24 am, Fernando Nasser wrote:
I don't really care on how its done, but IMO an enterprise class
database must be able to do log rotation. Logging to Syslog is not an
option (specially with our verbosity) -- users must be able to use flat
files for logging.
Uh, we
Bruno Wolff III wrote:
I can see their problem with making a dependency to all of apache or including
multilog in their distribution. But they probably could include something
that is only a logger either using some project that is only a logger or
splitting out the logger that is bundled with
Hi Lamar,
Lamar Owen wrote:
On Friday 12 March 2004 09:24 am, Fernando Nasser wrote:
I don't really care on how its done, but IMO an enterprise class
database must be able to do log rotation. Logging to Syslog is not an
option (specially with our verbosity) -- users must be able to use flat
On Fri, Mar 12, 2004 at 15:19:29 -0500,
Fernando Nasser [EMAIL PROTECTED] wrote:
Bruno Wolff III wrote:
I can see their problem with making a dependency to all of apache or
including
multilog in their distribution. But they probably could include something
that is only a logger either
Andrew Dunstan [EMAIL PROTECTED] writes:
Did anything ever come from this thread?
http://archives.postgresql.org/pgsql-hackers/2003-05/msg00603.php
(Heading: Plan B for log rotation support: borrow Apache code)
Only an entry on my depressingly long personal to-do list :-(
I did take a look
Hi,
Please remind me again why the postmaster cannot close and open the log
file when it receives a SIGHUP (to re-read configuration)? This was
discussed before but I cannot remember if and why this was not possible
or if the arguments are still valid after -l was added.
If this was possible
Seems posting to this list from the office didn't work...
The patch is attached as requested - this is just a quick hack, written to
do what I needed at the time.. consider it just as a starting point for
further work.
I've tested it on Solaris 9 with Sun's compiler (Sun Studio 8 Compilers)
but
Hi,
What's the best way to do log rolling with pg_autovacuum? It doesn't
seem to have any syslog options, etc. Using 'tee' maybe?
Chris
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
Christopher Kings-Lynne wrote:
What's the best way to do log rolling with pg_autovacuum? It doesn't
seem to have any syslog options, etc. Using 'tee' maybe?
I got an email from Mark Hollow saying that he had implemented a syslog
patch for pg_autovacuum. Don't know how good it is, but it
Hi Peter,
I've been playing with a little program I wrote whose sole purpose is to
write its stdin to a file and close and reopen that file when it receives
a signal. I figured this could work well when integrated transparently
into pg_ctl.
So, is log rotation a concern? Is this a
Christopher Kings-Lynne writes:
Yeah, I use FreeBSD's wonderful newsyslog utility, and I do my logging like
this:
su -l pgsql -c '[ -d ${PGDATA} ] exec /usr/local/bin/pg_ctl
start -s -w -o -i -l /var/log/pgsql.log'
And my /etc/newsyslog.conf entry:
/var/log/pgsql.log pgsql:pgsql
Christopher Kings-Lynne writes:
What's the problem with using newsyslog or logrotate at the moment? (ie.
use the system log rotator)
The postmaster will never close the output file, so you can rotate all you
want, the original file will never be abandoned.
--
Peter Eisentraut [EMAIL
Tom Lane writes:
Aren't there log-rotation utilities out there already? (I seem to
recall mention that Apache has one, for instance.) Seems like this
is a wheel we shouldn't have to reinvent.
I'm aware of the Apache rotatelogs utility, but I'm not completely
satisfied with it.
1. It
Thus spake Tom Lane
Also, I kinda thought the long-range solution was to encourage everyone
to migrate to syslog logging ...
And no, use syslog doesn't count.
Why not?
Well, one why not might be that syslog is not a guaranteed delivery
logging system. It might be good enough for some
Peter Eisentraut [EMAIL PROTECTED] writes:
Tom Lane writes:
Aren't there log-rotation utilities out there already? (I seem to
recall mention that Apache has one, for instance.) Seems like this
is a wheel we shouldn't have to reinvent.
I'm aware of the Apache rotatelogs utility, but I'm
At 08:54 PM 9/5/2001 -0700, Ian Lance Taylor wrote:
Tom Lane [EMAIL PROTECTED] writes:
And no, use syslog doesn't count.
Why not?
The standard implementations of syslog lose log entries under heavy
load, because they rely on a daemon which reads from a named pipe with
a limited buffer
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]On Behalf Of Peter Eisentraut
Sent: Thursday, 6 September 2001 6:04 PM
To: Christopher Kings-Lynne
Cc: PostgreSQL Development
Subject: Re: [HACKERS] Log rotation?
Christopher Kings-Lynne writes:
What's the problem
Peter Eisentraut [EMAIL PROTECTED] writes:
With all the great work put into allowing true 24/7 operation, as
distributed we're still unable to rotate the log file. While the log file
tends to be smaller than the database system as a whole, this is still
going to annoy people because they
Tom Lane [EMAIL PROTECTED] writes:
And no, use syslog doesn't count.
Why not?
The standard implementations of syslog lose log entries under heavy
load, because they rely on a daemon which reads from a named pipe with
a limited buffer space. This is not acceptable in a production
system,
With all the great work put into allowing true 24/7 operation, as
distributed we're still unable to rotate the log file. While the log file
tends to be smaller than the database system as a whole, this is still
going to annoy people because they can't control disk usage without taking
the server
So, is log rotation a concern? Is this a reasonable solution? Other
ideas?
(No Grand Unified Logging Solutions please. And no, use syslog doesn't
count.)
What's the problem with using newsyslog or logrotate at the moment? (ie.
use the system log rotator)
Chris
63 matches
Mail list logo
