Re: [HACKERS] serverlog rotation/functions

2004-07-16 Thread Andreas Pflug
Bruce Momjian wrote:
Andreas Pflug wrote:
You do something that splits the value into directory name and file name
and removes every letter after %.
/var/log
postgresql.log.%-%-%_%%%
Another idea is to allow filename wildcards in the listing so it
becomes:
SELECT *
FROM dir_listing('/var/log/postgresql.log.*-*-*_***') AS dir
While that is nice, it doesn't match the functionality of opendir so we
are perhaps better with one that doesn't handle wildcards and we just do
the wildcard processing in the WHERE clause.
Uh, this looks ugly.
How about
pg_logfile_list() RETURNS setof timestamp -- to list available logfiles
pg_logfile_filename(timestamp) to return filename for that logfile

I don't see the need to return timestamps. If you select any empty
directory, you can just return the file names.  The only reason you
might need a pattern is to distinguish pg log files from other log
files.  If you want, create a server-side function that returns the file
name with the strftime() patterns converted to '*'.

and generic
pg_dir(wildcard_text)

Maybe pg_dir_ls().
OK, it would be nice if we could do a sed operation like:
sed 's/%./*/g'
but I don't know a way to do that without defining a function or pulling
in a procedural language, but if we could do it we could do:
pg_dir(echo log_destination | sed 's/%./*/g')
Ar ever used sed on win32?!? And how should the timestamp be 
represented in client tools? Date/time interpretation is always a source 
of problems, so *please* let the server do that.

Rethinking all this, I'd like the pg_logfile_list to return a complex type:
CREATE TYPE pg_logfile_list AS (
filedate timestamp,
filename text,
backendpid int,
inuse bool)
and
pg_logfile_list() RETURNS SETOF pg_logfile_list
which would enable
SELECT  filename,
pg_file_unlink(filename)
  FROM  pg_logfile_list()
 WHERE  filedate  current_timestamp - '3 months'::interval
AND NOT inuse
In-use check is easy for the backend, if the syslog process publishes 
the current logfile's timestamp in sharedmem.

We can use a GUC variable for the log_directory (not log_destination); 
anyway, I'd like the filenames to be selected by the server.

Regards,
Andreas
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
 subscribe-nomail command to [EMAIL PROTECTED] so that your
 message can get through to the mailing list cleanly


Re: [HACKERS] serverlog rotation/functions

2004-07-16 Thread Tom Lane
Andreas Pflug [EMAIL PROTECTED] writes:
 In-use check is easy for the backend, if the syslog process publishes 
 the current logfile's timestamp in sharedmem.

You really haven't absorbed any of the objections I've raised, have you?
I don't want the log process connected to shared mem at *all*, and see
no particularly good reason why it should be.

 We can use a GUC variable for the log_directory (not log_destination); 
 anyway, I'd like the filenames to be selected by the server.

The directory should definitely be a GUC variable.  The individual
filenames should probably be of the form prefixtimestamp, where
the server dictates the format of the timestamp (and we choose it so
that the names sort correctly).  We could let the prefix be
user-selectable or make it hard-wired; I don't have a strong feeling
about that either way.

regards, tom lane

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] serverlog rotation/functions

2004-07-16 Thread Bruce Momjian
Andreas Pflug wrote:
  OK, it would be nice if we could do a sed operation like:
  
  sed 's/%./*/g'
  
  but I don't know a way to do that without defining a function or pulling
  in a procedural language, but if we could do it we could do:
  
  pg_dir(echo log_destination | sed 's/%./*/g')
  
 
 Ar ever used sed on win32?!? And how should the timestamp be 
 represented in client tools? Date/time interpretation is always a source 
 of problems, so *please* let the server do that.

I am thinking of these all being server-side functions.

 Rethinking all this, I'd like the pg_logfile_list to return a complex type:
 
 CREATE TYPE pg_logfile_list AS (
   filedate timestamp,
   filename text,
   backendpid int,
   inuse bool)
 
 and
 
 pg_logfile_list() RETURNS SETOF pg_logfile_list
 
 which would enable
 
 SELECT  filename,
  pg_file_unlink(filename)
FROM  pg_logfile_list()
   WHERE  filedate  current_timestamp - '3 months'::interval
  AND NOT inuse
 
 In-use check is easy for the backend, if the syslog process publishes 
 the current logfile's timestamp in sharedmem.
 
 We can use a GUC variable for the log_directory (not log_destination); 
 anyway, I'd like the filenames to be selected by the server.

This seems quite involved.  Can we get the basic functionality I
described first?  Also I am not sure how all this information is going
to be passed from the logging process to the backend requesting the
information, and it seems overly complicated.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])


Re: [HACKERS] serverlog rotation/functions

2004-07-16 Thread Andreas Pflug
Bruce Momjian wrote:
This seems quite involved.  Can we get the basic functionality I
described first?
On the way.
Also I am not sure how all this information is going
to be passed from the logging process to the backend requesting the
information, and it seems overly complicated.
There's *no* information passing from the logging process, with the 
single exception of the latest logfile timestamp (if allowed). I'd 
rather like to have that information from the logger, to be safe in case 
the system time was manipulated and the last logfile is not the current one.
The rest is just a reworked version of pg_dir_ls, with internal 
knowledge of how the timestamp is formatted.

Regards,
Andreas


---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] serverlog rotation/functions

2004-07-16 Thread Andreas Pflug
Tom Lane wrote:
Andreas Pflug [EMAIL PROTECTED] writes:
In-use check is easy for the backend, if the syslog process publishes 
the current logfile's timestamp in sharedmem.

You really haven't absorbed any of the objections I've raised, have you?
I don't want the log process connected to shared mem at *all*, and see
no particularly good reason why it should be.
Why shouldn't the process announce the logfile timestamp and its pid 
*writeonly*, so other backends know about it?

At least the pid must be distributed like this, just as bgwriter does.
I understand perfectly that postmaster and logger are very critical 
processes, so they should be dependent on as few resources as possible. 
The logger works without shmem in general, but how to reach it if its 
pid is unknown?


The directory should definitely be a GUC variable.  The individual
filenames should probably be of the form prefixtimestamp, where
the server dictates the format of the timestamp (and we choose it so
that the names sort correctly).  We could let the prefix be
user-selectable or make it hard-wired; I don't have a strong feeling
about that either way.
Agreed.
Regard,
Andreas
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] serverlog rotation/functions

2004-07-16 Thread Bruce Momjian
Andreas Pflug wrote:
 Bruce Momjian wrote:
 
  
  This seems quite involved.  Can we get the basic functionality I
  described first?
 
 On the way.
 
  Also I am not sure how all this information is going
  to be passed from the logging process to the backend requesting the
  information, and it seems overly complicated.
 
 There's *no* information passing from the logging process, with the 
 single exception of the latest logfile timestamp (if allowed). I'd 
 rather like to have that information from the logger, to be safe in case 
 the system time was manipulated and the last logfile is not the current one.
 The rest is just a reworked version of pg_dir_ls, with internal 
 knowledge of how the timestamp is formatted.

Oh, so you are hardcoding the logfile name so you can interpret the
timestamp from that?  It seems cleaner to allow the admin to specify
whatever log pattern the want.

However, you idea of expiring the log files based on timestamp values is
pretty powerful.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])


Re: [HACKERS] serverlog rotation/functions

2004-07-16 Thread Bruce Momjian
Andreas Pflug wrote:
 Bruce Momjian wrote:
 
  
  This seems quite involved.  Can we get the basic functionality I
  described first? 
 
 Current workable patch.
 
 Some questions/limitations:
 - How's the official way to restrict pg_* functions to superuser only

Very crudely  :-)

static int pg_signal_backend(int pid, int sig)
{
if (!superuser())
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
 (errmsg(only superuser can signal other backends;

 - I've restricted pg_file_read to 50k max. What's a reasonable limit for 
 a generic function?

Uh, that seems fine.  You already check to see it is within the limit. 
I think a bigger question is should we limit it at all?  Do we limit
pg_largeobject?  Is that similar?

 - pg_file_read and pg_file_write read/write text only; should have 
 binary versions too.

I guess we could but no one is asking for that yet so I would leave it
for later.

 Very open question:
 - How should a backend know the logger's pid if it's not in shmem. Write 
 a magic string to the pipe?

I think it has to and in fact the pid is being written by the
postmaster, not by the logger process, so that should be OK.  The issue
is that the logger shouldn't _attach_ to shared memory unless it has to.

As far as recording the current log timestamp, I think that will be a
problem.  I would much rather see us forget about doing timestamp
processing with these log files and keep it simple at this point and see
what needs we have for 7.6.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] serverlog rotation/functions

2004-07-16 Thread Andreas Pflug
Bruce Momjian wrote:
- How's the official way to restrict pg_* functions to superuser only

Very crudely  :-)
Got it.
'nother question: Is reading the logfile a task that may be allowed to 
superusers only? I don't think so, though acls might apply.


Uh, that seems fine.  You already check to see it is within the limit. 
I think a bigger question is should we limit it at all?  Do we limit
pg_largeobject?  Is that similar?
Ok, no limit (but a default maximum of 50k remains). And since it's 
superuser only, he hopefully knows what he does.


Very open question:
- How should a backend know the logger's pid if it's not in shmem. Write 
a magic string to the pipe?

I think it has to and in fact the pid is being written by the
postmaster, not by the logger process, so that should be OK.  The issue
is that the logger shouldn't _attach_ to shared memory unless it has to.
It doesn't. It inherits the unnamed shared mem segment from the 
postmaster, as all subprocesses.

As far as recording the current log timestamp, I think that will be a
problem.  I would much rather see us forget about doing timestamp
processing with these log files and keep it simple at this point and see
what needs we have for 7.6.
I'm a bit insisting on this point. Remember, this all started from the 
attempt to display the serverlog on the client side. To do this, I need 
a way to retrieve the current logfile properties (size, and in case of 
rotation timestamp too) in a low-overhead way, or at least get to know 
something has changed. Scanning a whole directory and interpreting the 
data isn't low overhead any more.

There's no locking on the shmem, and the single dependence on shmem is 
the existence of it at the time of rotation. If the shmem is gone, 
postmaster is probably dead anyway.

Regards,
Andreas
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] serverlog rotation/functions

2004-07-16 Thread Bruce Momjian
Andreas Pflug wrote:
 Bruce Momjian wrote:
 - How's the official way to restrict pg_* functions to superuser only
  
  
  Very crudely  :-)
 
 Got it.
 
 'nother question: Is reading the logfile a task that may be allowed to 
 superusers only? I don't think so, though acls might apply.

Yes, the log file might contain SQL queries issued by others.  It is a
super-user only capability.

  Uh, that seems fine.  You already check to see it is within the limit. 
  I think a bigger question is should we limit it at all?  Do we limit
  pg_largeobject?  Is that similar?
 
 Ok, no limit (but a default maximum of 50k remains). And since it's 
 superuser only, he hopefully knows what he does.

Huh?  Why have a default maximum?

 Very open question:
 - How should a backend know the logger's pid if it's not in shmem. Write 
 a magic string to the pipe?
  
  
  I think it has to and in fact the pid is being written by the
  postmaster, not by the logger process, so that should be OK.  The issue
  is that the logger shouldn't _attach_ to shared memory unless it has to.
 
 It doesn't. It inherits the unnamed shared mem segment from the 
 postmaster, as all subprocesses.

Ah, I think it needs to close that as soon as it starts.  Don't other
subprocesses do that?  That shared memory is very fragile and we don't
want an errant pointer poking in there.

  As far as recording the current log timestamp, I think that will be a
  problem.  I would much rather see us forget about doing timestamp
  processing with these log files and keep it simple at this point and see
  what needs we have for 7.6.
 
 I'm a bit insisting on this point. Remember, this all started from the 
 attempt to display the serverlog on the client side. To do this, I need 
 a way to retrieve the current logfile properties (size, and in case of 
 rotation timestamp too) in a low-overhead way, or at least get to know 
 something has changed. Scanning a whole directory and interpreting the 
 data isn't low overhead any more.

This seems clean and fast enough to me:

SELECT filename
FROM pg_dir_ls('/var/log')
ORDER BY 1 DESC
LIMIT 1

Considering that any query from a client is going to have to go through
the parser and be executed, an 'ls' in a directory just isn't a
measurable performance hit.

If you want run a test that does an 'ls' and one that doesn't to see
that there is no measurable performance difference.

I would not worry about the clock going backward.  PostgreSQL would have
enough problems with timestamp columns moving backward that the file log
times are the least of our problems.

 There's no locking on the shmem, and the single dependence on shmem is 
 the existence of it at the time of rotation. If the shmem is gone, 
 postmaster is probably dead anyway.

You can't know that you aren't reading corrupt data if you read shared
memory without a lock.  What if the write is happening as you read?

The only clean solution I can think of is to write an operating system
file that contains the current log filename and read from that.  I
believe such writes are atomic.  But again, this seems like overkill to
me.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] serverlog rotation/functions

2004-07-16 Thread Andreas Pflug
Bruce Momjian wrote:
Ok, no limit (but a default maximum of 50k remains). And since it's 
superuser only, he hopefully knows what he does.

Huh?  Why have a default maximum?
Just for convenience. Both start and size are optional parameters, but 
with start=0 and size=5. Well, it's a very special function anyway, 
so we could require the user to supply all parameters. I'll remove it.


Ah, I think it needs to close that as soon as it starts.  Don't other
subprocesses do that?  That shared memory is very fragile and we don't
want an errant pointer poking in there.
The result of an errant pointer writing to that shred mem would be
1) wrong pid for SysLogger, so it can't be signalled to rotate from backends
2) wrong timestamp, so backends don't know the latest logfile.
Nothing particularly crash prone really.

This seems clean and fast enough to me:
SELECT filename
FROM pg_dir_ls('/var/log')
ORDER BY 1 DESC
LIMIT 1
For a logfile listing function, this would look
SELECT MAX(startdate)
  FROM pg_logfile_ls()

Considering that any query from a client is going to have to go through
the parser and be executed, an 'ls' in a directory just isn't a
measurable performance hit.
If you want run a test that does an 'ls' and one that doesn't to see
that there is no measurable performance difference.
So while a simple PG_RETURN_TIMESTAMP(logfiletimestamp) compared to a 
lengthy setof returning function is drastically faster, this appears 
much less drastic with parser overhead.

I would not worry about the clock going backward.  PostgreSQL would have
enough problems with timestamp columns moving backward that the file log
times are the least of our problems.
I see, so the admin is in trouble anyway (what about PITR? Data column 
deviations appear harmless compared to restoration based on timestamps).


You can't know that you aren't reading corrupt data if you read shared
memory without a lock.  What if the write is happening as you read?
I thought about this quite a while.
If the shmem fields aren't written atomically (one is 32bit, one 64 bit, 
probably on dword boundaries so writing will happen at least processor 
bus wide, do we support any 16 bit processor?) the corruption 
consequences as above apply. In the case of the timestamp, the high word 
will rarely change anyway, only every 2^32 seconds...

Concurrent access on the logger pid would mean to call 
pg_logfile_rotate() while a killed logger is being restarted, which is 
creating a new logfile then anyway. This would send a SIGINT into outer 
space, maybe to the bgwriter triggering a checkpoint, or the postmaster 
shutting it down (gracefully, still unwanted).

BTW, the consequences of a trigger flag in shmem would be less because 
all that could happen was a log rotation (which appends to existing 
files, just in case syslogger died in the milliseconds after a rotation).

The only clean solution I can think of is to write an operating system
file that contains the current log filename and read from that.  I
believe such writes are atomic.  But again, this seems like overkill to
me.
Ah wait.
Digging further behind SIGUSR1 I now *do* see a solution without pid in 
shmem, using SendPostmasterSignal. Well, a little hint from gurus would 
have helped...

I'll convert to this, *dropping* all shmem.
Regards,
Andreas
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
   (send unregister YourEmailAddressHere to [EMAIL PROTECTED])


Re: [HACKERS] serverlog rotation/functions

2004-07-16 Thread Andreas Pflug
Andreas Pflug wrote:
Ah wait.
Digging further behind SIGUSR1 I now *do* see a solution without pid in 
shmem, using SendPostmasterSignal. Well, a little hint from gurus would 
have helped...

Oops, SendPostmasterSignal uses shmem
At least, this enables syslogger.c to be free from shmem stuff, except 
for PGSharedMemDetach.

Regards,
Andreas
---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] serverlog rotation/functions

2004-07-16 Thread Bruce Momjian
Andreas Pflug wrote:
 Bruce Momjian wrote:
 Ok, no limit (but a default maximum of 50k remains). And since it's 
 superuser only, he hopefully knows what he does.
  
  
  Huh?  Why have a default maximum?
 
 Just for convenience. Both start and size are optional parameters, but 
 with start=0 and size=5. Well, it's a very special function anyway, 
 so we could require the user to supply all parameters. I'll remove it.

Agreed, and maybe a zero value gets the entire file.

  Ah, I think it needs to close that as soon as it starts.  Don't other
  subprocesses do that?  That shared memory is very fragile and we don't
  want an errant pointer poking in there.
 
 The result of an errant pointer writing to that shred mem would be
 
 1) wrong pid for SysLogger, so it can't be signalled to rotate from backends
 2) wrong timestamp, so backends don't know the latest logfile.
 
 Nothing particularly crash prone really.

No, I am thinking the program goes crazy and writes everywhere.

  This seems clean and fast enough to me:
  
  SELECT filename
  FROM pg_dir_ls('/var/log')
  ORDER BY 1 DESC
  LIMIT 1
 
 For a logfile listing function, this would look
 
 SELECT MAX(startdate)
FROM pg_logfile_ls()
 
 
  
  Considering that any query from a client is going to have to go through
  the parser and be executed, an 'ls' in a directory just isn't a
  measurable performance hit.
  
  If you want run a test that does an 'ls' and one that doesn't to see
  that there is no measurable performance difference.
  
 So while a simple PG_RETURN_TIMESTAMP(logfiletimestamp) compared to a 
 lengthy setof returning function is drastically faster, this appears 
 much less drastic with parser overhead.
 
  I would not worry about the clock going backward.  PostgreSQL would have
  enough problems with timestamp columns moving backward that the file log
  times are the least of our problems.
 
 I see, so the admin is in trouble anyway (what about PITR? Data column 
 deviations appear harmless compared to restoration based on timestamps).

PITR uses WAL numbering so it would be fine, but the timestamps on the
commit records would have problems.

  You can't know that you aren't reading corrupt data if you read shared
  memory without a lock.  What if the write is happening as you read?
 
 I thought about this quite a while.
 
 If the shmem fields aren't written atomically (one is 32bit, one 64 bit, 
 probably on dword boundaries so writing will happen at least processor 
 bus wide, do we support any 16 bit processor?) the corruption 
 consequences as above apply. In the case of the timestamp, the high word 
 will rarely change anyway, only every 2^32 seconds...
 
 Concurrent access on the logger pid would mean to call 
 pg_logfile_rotate() while a killed logger is being restarted, which is 
 creating a new logfile then anyway. This would send a SIGINT into outer 
 space, maybe to the bgwriter triggering a checkpoint, or the postmaster 
 shutting it down (gracefully, still unwanted).
 
 BTW, the consequences of a trigger flag in shmem would be less because 
 all that could happen was a log rotation (which appends to existing 
 files, just in case syslogger died in the milliseconds after a rotation).
 
  
  The only clean solution I can think of is to write an operating system
  file that contains the current log filename and read from that.  I
  believe such writes are atomic.  But again, this seems like overkill to
  me.
 
 Ah wait.
 Digging further behind SIGUSR1 I now *do* see a solution without pid in 
 shmem, using SendPostmasterSignal. Well, a little hint from gurus would 
 have helped...
 
 I'll convert to this, *dropping* all shmem.

Yes, that is the usual method.  We signal the postmaster and it then
does the signalling to the logger.  I thought you had looked at other
backend signalling examples so I didn't explain it.

Now, one really good efficiency would be to use LISTEN/NOTIFY so clients
could know when new data has appeared in the log, or the log file is
rotated.  Now that's an efficiency!   However, let's get this
infrastructure completed first.   One wacky idea would be for the
clients to LISTEN on 'pg_new_logfile' and have the logger do
system('psql -c NOTIFY pg_new_logfile template1') or something like
that.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] serverlog rotation/functions

2004-07-16 Thread Bruce Momjian
Andreas Pflug wrote:
 Andreas Pflug wrote:
 
  
  Ah wait.
  Digging further behind SIGUSR1 I now *do* see a solution without pid in 
  shmem, using SendPostmasterSignal. Well, a little hint from gurus would 
  have helped...
  
 
 Oops, SendPostmasterSignal uses shmem
 
 At least, this enables syslogger.c to be free from shmem stuff, except 
 for PGSharedMemDetach.

Right.  We already have to use shared mem for the backends and
postmaster.  It is the logger we are worried about.

Tom brought up the point that if the logger used shared memory, we would
have to kill/restart it if we need to reinitialize shared memory,
meaning we would loose logging info at a time we really need it ---
again a good reason not to use shared memory in the logger.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] serverlog rotation/functions

2004-07-16 Thread Bruce Momjian
Andreas Pflug wrote:
 Bruce Momjian wrote:
  Andreas Pflug wrote:
 
  Right.  We already have to use shared mem for the backends and
  postmaster.  It is the logger we are worried about.
  
  Tom brought up the point that if the logger used shared memory, we would
  have to kill/restart it if we need to reinitialize shared memory,
 
 I don't know why that particular segment should ever be renewed. Anyway, 
 it's gone.

As I remember, we have one big shared memory segment.  Where you
creating a special one just for this timestamp?  If you were, I see why
your approach was safer, but as you said, it doesn't buy us much anyway.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] serverlog rotation/functions

2004-07-16 Thread Andreas Pflug
Bruce Momjian wrote:
Andreas Pflug wrote:

Right.  We already have to use shared mem for the backends and
postmaster.  It is the logger we are worried about.
Tom brought up the point that if the logger used shared memory, we would
have to kill/restart it if we need to reinitialize shared memory,
I don't know why that particular segment should ever be renewed. Anyway, 
it's gone.

Regards,
Andreas
---(end of broadcast)---
TIP 6: Have you searched our list archives?
  http://archives.postgresql.org


Re: [HACKERS] serverlog rotation/functions

2004-07-16 Thread Andreas Pflug
Bruce Momjian wrote:
Andreas Pflug wrote:
Just for convenience. Both start and size are optional parameters, but 
with start=0 and size=5. Well, it's a very special function anyway, 
so we could require the user to supply all parameters. I'll remove it.

Agreed, and maybe a zero value gets the entire file.
Which is a default param back again, maybe on a 100MB file? Better not. 
Lets leave it to the admin to do sick stuff as 
pg_read_file('base/5000/5002', 0, 1) ...

No, I am thinking the program goes crazy and writes everywhere.
What I described was just that situation.

Yes, that is the usual method.  We signal the postmaster and it then
does the signalling to the logger.  I thought you had looked at other
backend signalling examples so I didn't explain it.
Well if you know the places where backends do signal stuff to the 
postmaster... Still, somebody could have yelled use the standard way 
before reinventing the wheel.

Now, one really good efficiency would be to use LISTEN/NOTIFY so clients
could know when new data has appeared in the log, or the log file is
rotated.  Now that's an efficiency!   However, let's get this
infrastructure completed first.   One wacky idea would be for the
clients to LISTEN on 'pg_new_logfile' and have the logger do
system('psql -c NOTIFY pg_new_logfile template1') or something like
that.
No, certainly not. This would mean that every time a log is done, psql 
is fired up. Tom wouldn't accept this as KISS, I believe. And h*ll, that 
would cause traffic (just imagine a single log message on client startup...)

What you saw on LinuxTag was pgAdmin3 polling once a second if the 
logfile length changed, which is the fastest setting possible.

Regards,
Andreas


---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
  http://www.postgresql.org/docs/faqs/FAQ.html