Re: [PORTS] Logging (was Re: [GENERAL] PostgreSQL 7.0-2 RPMset released.)
Tom Lane wrote: > > Lamar Owen <[EMAIL PROTECTED]> writes: > > The real problem with redirecting the postmaster output is the issue > > of log rolling, which is impossible to do in the 'classic' > > stderr/stdout redirect UNLESS you throw down postmaster when rolling > > the log (unless you know a trick I don't). I think I do ;-) read on... > Yes. I think ultimately we will have to do some logging support code of > our own to make this work the way we want. My thought at the moment is > there's nothing wrong with logging to stderr, as long as there's some > code somewhere that periodically closes stderr and reopens it to a new > log file. There needn't be a lot of code involved, we just need a > well-thought-out spec for how it should work. Comments anyone? > > regards, tom lane I really enjoy using apache's rotatelogs program. stderr is redirected through a pipe to a very small and robust C program, rotatelogs, that takes as arguments number of seconds between log rotates and the log filename. Logs are rotated every argv[2] seconds. The rotatelogs program takes care of closing and reopening, and nothing has to done from the application, just start postmaster with '2>&1 | rotatelogs ...' at the end, and log to stderr. Also, BSD license! :) For reference, I enclose the program as an attachment; it's less than 100 lines. Also, here's the man page: Name rotatelogs - rotate Apache logs without having to kill the server Synopsis rotatelogs logfile rotationtime Description rotatelogs is a simple program for use in conjunction with Apache's piped logfile feature which can be used like this: TransferLog"|rotatelogs /path/to/logs/access_log 86400" This creates the files /path/to/logs/access_log. where is the system time at which the log nominally starts (this time will always be a multiple of the rotation time, so you can synchronize cron scripts with it). At the end of each rotation time (here after 24 hours) a new log is started. Options logfile The path plus basename of the logfile. The suffix . is automatically added. rotationtime The rotation time in seconds. See Also httpd(8) /* * Simple program to rotate Apache logs without having to kill the server. * * Contributed by Ben Laurie <[EMAIL PROTECTED]> * * 12 Mar 1996 */ #define BUFSIZE 65536 #define MAX_PATH1024 #include "ap_config.h" #include #include #include int main (int argc, char **argv) { char buf[BUFSIZE], buf2[MAX_PATH]; time_t tLogEnd = 0; time_t tRotation; int nLogFD = -1; int nRead; char *szLogRoot; if (argc != 3) { fprintf(stderr, "%s \n\n", argv[0]); #ifdef OS2 fprintf(stderr, "Add this:\n\nTransferLog \"|%s.exe /some/where 86400\"\n\n", argv[0]); #else fprintf(stderr, "Add this:\n\nTransferLog \"|%s /some/where 86400\"\n\n", argv[0]); #endif fprintf(stderr, "to httpd.conf. The generated name will be /some/where. " "where is the\nsystem time at which the log nominally " "starts (N.B. this time will always be a\nmultiple of the " "rotation time, so you can synchronize cron scripts with it).\n" "At the end of each rotation time a new log is started.\n"); exit(1); } szLogRoot = argv[1]; tRotation = atoi(argv[2]); if (tRotation <= 0) { fprintf(stderr, "Rotation time must be > 0\n"); exit(6); } for (;;) { nRead = read(0, buf, sizeof buf); if (nRead == 0) exit(3); if (nRead < 0) if (errno != EINTR) exit(4); if (nLogFD >= 0 && (time(NULL) >= tLogEnd || nRead < 0)) { close(nLogFD); nLogFD = -1; } if (nLogFD < 0) { time_t tLogStart = (time(NULL) / tRotation) * tRotation; sprintf(buf2, "%s.%010d", szLogRoot, (int) tLogStart); tLogEnd = tLogStart + tRotation; nLogFD = open(buf2, O_WRONLY | O_CREAT | O_APPEND, 0666); if (nLogFD < 0) { perror(buf2); exit(2); } } if (write(nLogFD, buf, nRead) != nRead) { perror(buf2); exit(5); } } }
Re: Logging (was Re: [GENERAL] PostgreSQL 7.0-2 RPMset released.)
On Mon, 22 May 2000 00:19:45 -0400 Tom Lane wrote: > There needn't be a lot of code involved, we just need a > well-thought-out spec for how it should work. Comments anyone? I run postmaster under Dan Bernstein's "daemontools", which include logging facilities: http://cr.yp.to/daemontools.html The summary of this setup is that postmaster runs in the forground writing error messages to standard error, and standard error is a pipe to another process. The second process is responsible for selecting messages to write, writing them, and rotating the log file. More traditional Unix solutions would involve teaching postmaster what the name of its log file is, and to reopen it on receipt of some signal. Usually SIGHUP is used since SIGHUP is unlikely to be useful to a daemon running in the background. There are issues for logging errors that many applications handle badly. What happens when: o there is an I/O error writing to a log file? o the log file is at maximum size? o the filesystem the log file is in is full? o a write to a log file blocks? To take a not random example, syslogd is OK for log file rotation but makes a mess and a muddle of things otherwise including the points I list. Regards, Giles
Logging (was Re: [GENERAL] PostgreSQL 7.0-2 RPMset released.)
Lamar Owen <[EMAIL PROTECTED]> writes: > The real problem with redirecting the postmaster output is the issue > of log rolling, which is impossible to do in the 'classic' > stderr/stdout redirect UNLESS you throw down postmaster when rolling > the log (unless you know a trick I don't). Yes. I think ultimately we will have to do some logging support code of our own to make this work the way we want. My thought at the moment is there's nothing wrong with logging to stderr, as long as there's some code somewhere that periodically closes stderr and reopens it to a new log file. There needn't be a lot of code involved, we just need a well-thought-out spec for how it should work. Comments anyone? regards, tom lane
Re: [GENERAL] PostgreSQL 7.0-2 RPMset released.
On Sun, 21 May 2000, Thomas Good wrote: > On the GENERAL list the issue of firing up a server, and the silent flag > used by the default redhatter 'postgresql' script in init.d came up. Yes, I read the thread. I didn't write the original initscript -- but hopefully have changed it to more your liking (see below). > I redirect output to /var/lib/pgsql/postlog after I rm the -S from the > call to the server...not having pg complain when I screw up my CGI > scripts is no good to me. > If I were to have a vote, I'd urge whomever to add a comment to 'postgresql > the script' to offer logging in the manner described above. The 7.0 RPM's /etc/rc.d/init.d/postgresql script uses pg_ctl, rather than directly starting postmaster (and has since 7.0beta2, IIRC) -- and the PGDATA/postmaster.opts.default (which, by default, only has '-i' -- no -S) file is used for postmaster startup options, rather than passing them on the command line. The changelog notice for this was buried back in the beta cycle release announcements -- I should have duplicated all notices for the 7.0-1 release announcement. More documentation will be written as I have time (or input to README.rpm, or patches to README.rpm). Look at the new initscript, then let me know about possible improvements (of which I am sure improvements can be made!). Currently stderr and stdout from pg_ctl are piped to /dev/null, but that is easy enough to change. And, by changing the PGDATA/pg_options file's contents, you can turn on syslog -- edit /etc/syslog.conf to get syslogging working -- just watch out for long queries! Logging is one of the hot issues in the RPMset right now, as the comments about syslog in the -2 release announcement show. The real problem with redirecting the postmaster output is the issue of log rolling, which is impossible to do in the 'classic' stderr/stdout redirect UNLESS you throw down postmaster when rolling the log (unless you know a trick I don't). I am trying to get _real_ logging, by way of syslog, rather than with redirects -- however, the redhat syslog dies under long queries (such as the one issued by psql in response to a \d directive). Since some things were missed in the beta cycle's announcements (which only were sent to pgsql-hackers), notice that the new 7.0 RPMset will create a new PGDATA in /var/lib/pgsql/data instead of /var/lib/pgsql. There are other changes -- read /usr/doc/postgresql-7.0/README.rpm and the pgsql-hackers archives on the subject. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Re: [GENERAL] PostgreSQL 7.0-2 RPMset released.
On Sat, 20 May 2000, Lamar Owen wrote: > The 7.0-2 RPMset fixes the following: > 1.) SPI headers are now the 7.0 set, and not the 6.5.3 set; > 2.) pg_options default to NOT enable syslog, or extended query logging, as > syslogd has some issues with long queries (such as issued by psql's \d > command!); > 3.) Alpha patches have returned! > > As usual, read '/usr/doc/postgresql-7.0/README.rpm' for more information. Lamar, On the GENERAL list the issue of firing up a server, and the silent flag used by the default redhatter 'postgresql' script in init.d came up. I redirect output to /var/lib/pgsql/postlog after I rm the -S from the call to the server...not having pg complain when I screw up my CGI scripts is no good to me. If I were to have a vote, I'd urge whomever to add a comment to 'postgresql the script' to offer logging in the manner described above. I do this on BSD, UnixWare and Linux (a few flavours) and have never had a problem. Other than with my own code! To be a bit clearer (tough this early): I rm the -S muzzle, >> stdout and stderr to /var/lib/pgsql/postlog, then run the whole enchilada in the background ( 2>&1 &' ). This works well. I call the script from rc.local and I still get the [OK] in brilliant green followed by the pid. Nothing appears broken *and* I get a log full of insensitive complaints about my programming skills. Who could ask for more? Here is what I do: su -l postgres -c '/usr/bin/postmaster -i -D/var/lib/pgsql >> /var/lib/pgsql/postlog 2>&1 &' Cheers, Tom > **initdb required for those still running releases prior to 7.0RC5!*** > > Users running 6.5.x (or earlier!) need to thoroughly read and understand the > README.rpm before installing (it is available on the ftp site as README in the > RPM distribution directory, as well as in the 'unpacked' subdirectory). > > The spec file for this release, as well as all patches and supplemental > programs are available in the 'unpacked' subdirectory. > > RPMset's are available at: > ftp://ftp.postgresql.org/pub/binary/v7.0/redhat-RPM > > Further information available at http://www.ramifordistat.net/postgres, as > usual; or by e-mail at [EMAIL PROTECTED] (i prefer RPM questions to go > to the list instead of directly to me). > > -- > Lamar Owen > WGCR Internet Radio > 1 Peter 4:11 > --- North Richmond Community Mental Health Center --- Thomas Good, MIS Coordinatortomg@ { admin | q8 } .nrnet.org Phone: 718-354-5528 Fax: 718-354-5056 --- North Richmond Systems PostgreSQL s l a c k w a r e Are Powered By: RDBMS |-- linux ---
[GENERAL] PostgreSQL 7.0-2 RPMset released.
The 7.0-2 RPMset fixes the following: 1.) SPI headers are now the 7.0 set, and not the 6.5.3 set; 2.) pg_options default to NOT enable syslog, or extended query logging, as syslogd has some issues with long queries (such as issued by psql's \d command!); 3.) Alpha patches have returned! As usual, read '/usr/doc/postgresql-7.0/README.rpm' for more information. **initdb required for those still running releases prior to 7.0RC5!*** Users running 6.5.x (or earlier!) need to thoroughly read and understand the README.rpm before installing (it is available on the ftp site as README in the RPM distribution directory, as well as in the 'unpacked' subdirectory). The spec file for this release, as well as all patches and supplemental programs are available in the 'unpacked' subdirectory. RPMset's are available at: ftp://ftp.postgresql.org/pub/binary/v7.0/redhat-RPM Further information available at http://www.ramifordistat.net/postgres, as usual; or by e-mail at [EMAIL PROTECTED] (i prefer RPM questions to go to the list instead of directly to me). -- Lamar Owen WGCR Internet Radio 1 Peter 4:11