Re: [PATCHES] WIP patch - INSERT-able log statements

2007-03-27 Thread Bruce Momjian
What is the status of this patch? --- FAST PostgreSQL wrote: > Hi, > > I've been working on the following TODO item and attached is an initial > patch. (It is only partial and not yet completely functional) > > "Allow ser

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-21 Thread Greg Smith
On Thu, 22 Feb 2007, FAST PostgreSQL wrote: As we are triggering the sql output in log_destination, if the user gives 'syslog,sql' as options he is going to get two different looking logs (in terms of contents) depending upon his settings. Yes, exactly; it's a good thing. People add and remo

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-21 Thread daveg
On Thu, Feb 22, 2007 at 11:50:06AM +1100, FAST PostgreSQL wrote: > - The log output will be in COPY format and will include the following > information, irrespective of the log_line_prefix setting. > ( timestamp_with_milliseconds,  timestamp, username,  databasename, > sessionid,  host_and_port,

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-20 Thread FAST PostgreSQL
On Wed, 21 Feb 2007 14:59, Greg Smith wrote: > On Tue, 20 Feb 2007, Tom Lane wrote: > > A smaller problem is that this forces people to incur a gettimeofday > > call for every message logged > > I'm stumped trying to think of an application that would require importing > the logs into a database to

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-20 Thread Greg Smith
On Tue, 20 Feb 2007, Tom Lane wrote: A smaller problem is that this forces people to incur a gettimeofday call for every message logged I'm stumped trying to think of an application that would require importing the logs into a database to analyze them, but not need the timestamp. I'd expect

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-20 Thread FAST PostgreSQL
On Wed, 21 Feb 2007 12:08, Tom Lane wrote: > "FAST PostgreSQL" <[EMAIL PROTECTED]> writes: > > - The log output will be in COPY format and will include the following > > information, irrespective of the log_line_prefix setting. > > ( timestamp_with_milliseconds,  timestamp, username,  databasename,

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-20 Thread Tom Lane
"FAST PostgreSQL" <[EMAIL PROTECTED]> writes: > - The log output will be in COPY format and will include the following > information, irrespective of the log_line_prefix setting. > ( timestamp_with_milliseconds,  timestamp, username,  databasename, > sessionid,  host_and_port, host, proc_id, comm

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-20 Thread FAST PostgreSQL
Ok. Summarizing the key changes required on my patch, based on the discussions so far are : - The log_destination will include a new option 'sql'. This can be given with other combinations of stderr, syslog or eventlog. - The sql logs will be written in log_directory in a file log_filename.

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-20 Thread Guillaume Smet
On 2/20/07, Tom Lane <[EMAIL PROTECTED]> wrote: Of course, the other side of that coin is that syslog is known to drop messages altogether under sufficient load. (At least on some platforms; dunno about yours.) Yes I know. That's one of the reason why I asked for the bahaviour of 7.4 log_durat

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-19 Thread Alvaro Herrera
Tom Lane escribió: > "Guillaume Smet" <[EMAIL PROTECTED]> writes: > > (each query is run on a different backend). With stderr you don't have > > sufficient information to know the query you should append the text > > to. Syslog adds useful context information (pid, command line, command > > number)

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-19 Thread Tom Lane
"Guillaume Smet" <[EMAIL PROTECTED]> writes: > (each query is run on a different backend). With stderr you don't have > sufficient information to know the query you should append the text > to. Syslog adds useful context information (pid, command line, command > number) which allows you to guarante

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-19 Thread Alvaro Herrera
Guillaume Smet escribió: > On 2/19/07, Alvaro Herrera <[EMAIL PROTECTED]> wrote: > >So add the session ID (%c) to log_line_prefix. > > It could work if log_line_prefix was added before every line but it's > definitely not the case: > myuser mydb 45d9d615.4abe LOG: duration : 185.223 ms, statement

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-19 Thread Guillaume Smet
On 2/19/07, Alvaro Herrera <[EMAIL PROTECTED]> wrote: So add the session ID (%c) to log_line_prefix. It could work if log_line_prefix was added before every line but it's definitely not the case: myuser mydb 45d9d615.4abe LOG: duration : 185.223 ms, statement : SELECT * FROM lieu

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-19 Thread Alvaro Herrera
Guillaume Smet escribió: > On 2/19/07, Alvaro Herrera <[EMAIL PROTECTED]> wrote: > >I don't think the syslog option is so nice to use these days; the > >redirect_stderr stuff is more powerful and easy to use. > > Did you already analyze logs of a highly loaded platform using stderr? > It's impossi

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-19 Thread Guillaume Smet
On 2/19/07, Alvaro Herrera <[EMAIL PROTECTED]> wrote: I don't think the syslog option is so nice to use these days; the redirect_stderr stuff is more powerful and easy to use. Did you already analyze logs of a highly loaded platform using stderr? It's impossible to guarantee the consistency of

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-19 Thread Alvaro Herrera
Guillaume Smet escribió: > On 2/19/07, Alvaro Herrera <[EMAIL PROTECTED]> wrote: > >Please don't do that. We already have a "combined GUC option" that is > >used to change two different things (DateStyle) and I regularly see > >people confused about how to use it. > > Perhaps I don't understand y

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-19 Thread Guillaume Smet
On 2/19/07, Alvaro Herrera <[EMAIL PROTECTED]> wrote: Please don't do that. We already have a "combined GUC option" that is used to change two different things (DateStyle) and I regularly see people confused about how to use it. Perhaps I don't understand your "combined GUC option" but ISTM it

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-19 Thread Alvaro Herrera
Guillaume Smet escribió: > On 2/19/07, Greg Smith <[EMAIL PROTECTED]> wrote: > >log_destination = 'stderr,sql' # Valid values are combinations of > > # stderr, syslog, sql, and eventlog, > > # depending on platform. > > +1 Please don't do that. We alre

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-19 Thread Guillaume Smet
On 2/19/07, Greg Smith <[EMAIL PROTECTED]> wrote: log_destination = 'stderr,sql' # Valid values are combinations of # stderr, syslog, sql, and eventlog, # depending on platform. +1 # These are relevant when logging to sql: log_sql_table = 'pg_log'

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-18 Thread Greg Smith
On Tue, 20 Feb 2007, FAST PostgreSQL wrote: I think adding the 'format' of the log as an option in the 'destination' may be confusing. We can have a new boolean variable like 'output_sql_log' or 'log_sql_format' which will trigger the output of INSERT-able log in addition to syslog/stderr/eve

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-18 Thread FAST PostgreSQL
On Sun, 18 Feb 2007 18:25, Greg Smith wrote: > On Sat, 17 Feb 2007, FAST PostgreSQL wrote: > > #log_output_type = 'text' #Valid values are 'SQL' or 'text' > > Defaults to 'text' which is status quo. If it is set to 'SQL' log will > > be output as INSERT commands. > > This assumes someone wants eith

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-18 Thread Guillaume Smet
Added -hackers to CC:. On 2/18/07, Greg Smith <[EMAIL PROTECTED]> wrote: I've thought a bit about how to implement this TODO already (I have a log file parser and I hate maintaining it) Any problem using pgFouine? Also, I feel that supporting the whole log_line_prefix syntax for this feature

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-17 Thread Greg Smith
On Sat, 17 Feb 2007, FAST PostgreSQL wrote: #log_output_type = 'text' #Valid values are 'SQL' or 'text' Defaults to 'text' which is status quo. If it is set to 'SQL' log will be output as INSERT commands. This assumes someone wants either the INSERT-able logs or the current, easily readable

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-16 Thread Joshua D. Drake
FAST PostgreSQL wrote: > On Fri, 16 Feb 2007 11:50, Tom Lane wrote: >> "FAST PostgreSQL" <[EMAIL PROTECTED]> writes: >>> The second variable is of interest. We need to specify a table in the >>> insert command. My preferred option is for the user to give one and he >>> can create it if and when he

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-15 Thread Alvaro Herrera
Tom Lane wrote: > "FAST PostgreSQL" <[EMAIL PROTECTED]> writes: > > On Fri, 16 Feb 2007 11:50, Tom Lane wrote: > >> Why not output the data in COPY format instead? > > > Yeah, that was my initial idea too... But because the TODO item clearly > > mentions INSERT, I thought maybe there was some ver

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-15 Thread Tom Lane
"FAST PostgreSQL" <[EMAIL PROTECTED]> writes: > On Fri, 16 Feb 2007 11:50, Tom Lane wrote: >> Why not output the data in COPY format instead? > Yeah, that was my initial idea too... But because the TODO item clearly > mentions INSERT, I thought maybe there was some very specific reason for the >

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-15 Thread FAST PostgreSQL
On Fri, 16 Feb 2007 11:50, Tom Lane wrote: > "FAST PostgreSQL" <[EMAIL PROTECTED]> writes: > > The second variable is of interest. We need to specify a table in the > > insert command. My preferred option is for the user to give one and he > > can create it if and when he wants to. The alternative

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-15 Thread Tom Lane
"FAST PostgreSQL" <[EMAIL PROTECTED]> writes: > The second variable is of interest. We need to specify a table in the insert > command. My preferred option is for the user to give one and he can create it > if and when he wants to. The alternative is we decide the table name and make > initdb to

[PATCHES] WIP patch - INSERT-able log statements

2007-02-15 Thread FAST PostgreSQL
Hi, I've been working on the following TODO item and attached is an initial patch. (It is only partial and not yet completely functional) "Allow server log information to be output as INSERT statements This would allow server log information to be easily loaded into a database for analysis. "