Itagaki, * Itagaki Takahiro (itagaki.takah...@gmail.com) wrote: > ==== Discussions ==== > * How about "csvlog_fields" rather than "log_csv_fields"? > Since we have variables with syslog_ prefix, csvlog_ prefix > seems to be better.
Sure, not a big deal, to be honest, as long as we can actually agree on
something... Not changed in the patch, but I can if people want.
> * We use %<what> syntax to specify fields in logs for log_line_prefix,
> but will use long field names for log_csv_fields. Do you have any
> better idea to share the names in both options? At least I want to
> share the short description for them in postgresql.conf.
No, I don't, and that's going well beyond what I feel makes sense for
this patch at this time. We could review that for 9.2 or later, but
we've had quite enough expanding-of-requirements for this patch
already, imnsho.
> * log_csv_fields's GUC context is PGC_POSTMASTER. Is it by design?
> PGC_SIGHUP would be more consistent compared with log_line_prefix.
> However, the csv format will not be valid because the column
> definitions will be changed in the middle of file.
Doing SIGHUP would require addressing how to get all of the backends to
close the old log file and open the new one, because we don't want to
have a given log file which has two different CSV formats in it (we
wouldn't be able to load it into the database...). This was
specifically addressed in the thread leading up to this patch...
> * "none" is not so useful for the initial "role_name" field.
> Should we use user_name instead of the empty role_name?
none is what it is, however, if you query 'show role'. I would rather
be consistant with that than log something else.
> ==== Comments for codes ====
> Some of the items are trivial, though.
>
> * What objects do you want to allocate in TopMemoryContext in
> assign_log_csv_fields() ? AFAICS, we don't have to keep rawstring
> and column_list in long-term context. Or, if you need TopMemoryContext,
> those variables should be pfree'ed at the end of function.
You're right, rawstring and column_list don't need to be in
TopMemoryContext. I just moved the switch to Top to be after those are
allocated.
> * appendStringInfoChar() calls in write_csvlog() seem to be wrong
> when the last field is not application_name.
Urgh, right, fixed to *not* include a trailing comma on the last column.
> * Docs need more cross-reference hyper-links for "see also" items.
>
> * Docs need some tags for itemized elements or pre-formatted codes.
> They looks itemized in the sgml files, but will be flattened in
> complied HTML files.
Not sure what you're referring to here...? Can you elaborate? I'm not
great with the docs. :/
> * A declaration of assign_log_csv_fields() at the top of elog.c
> needs "extern".
Err, no, I don't think it does. None of the other exported functions
from elog.c have extern declarations in elog.c... I did realize that I
probably shouldnt have the declaration at the top of elog.c for
assign_loc_csv_fields() or build_default_csvlog_list().
> * There is a duplicated declaration for build_default_csvlog_list().
Removed the duplicate in elog.c.
> * list_free() is NIL-safe. You don't have to check whether the list
> is NIL before call the function.
Fixed.
Updated patch attached.
Thanks!
Stephen
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
***************
*** 3504,3510 **** local0.* /var/log/postgresql
</row>
<row>
<entry><literal>%u</literal></entry>
! <entry>User name</entry>
<entry>yes</entry>
</row>
<row>
--- 3504,3523 ----
</row>
<row>
<entry><literal>%u</literal></entry>
! <entry>Session user name, typically the user name which was used
! to authenticate to <productname>PostgreSQL</productname> with,
! but can be changed by a superuser, see <command>SET SESSION
! AUTHORIZATION</></entry>
! <entry>yes</entry>
! </row>
! <row>
! <entry><literal>%U</literal></entry>
! <entry>Current role name, when set with <command>SET ROLE</>;
! the current role identifier is relevant for permission checking;
! Returns 'none' if the current role matches the session user.
! Note: Log messages from inside <literal>SECURITY DEFINER</>
! functions will show the calling role, not the effective role
! inside the <literal>SECURITY DEFINER</> function</entry>
<entry>yes</entry>
</row>
<row>
***************
*** 3621,3626 **** FROM pg_stat_activity;
--- 3634,3662 ----
</listitem>
</varlistentry>
+ <varlistentry id="guc-log-csv-fields" xreflabel="log_csv_fields">
+ <term><varname>log_csv_fields</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>log_csv_fields</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Controls the set and order of the fields which are written out in
+ the CSV-format log file.
+
+ The default is: log_time, user_name, database_name, process_id,
+ connection_from, session_id, session_line_num, command_tag,
+ session_start_time, virtual_transaction_id, transaction_id,
+ error_severity, sql_state_code, message, detail, hint,
+ internal_query, internal_query_pos, context, query, query_pos,
+ location, application_name
+
+ For details on what these fields are, refer to the log_line_prefix
+ and CSV logging documentation.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-log-lock-waits" xreflabel="log_lock_waits">
<term><varname>log_lock_waits</varname> (<type>boolean</type>)</term>
<indexterm>
***************
*** 3728,3761 **** FROM pg_stat_activity;
Including <literal>csvlog</> in the <varname>log_destination</> list
provides a convenient way to import log files into a database table.
This option emits log lines in comma-separated-values
! (<acronym>CSV</>) format,
! with these columns:
! timestamp with milliseconds,
! user name,
! database name,
! process ID,
! client host:port number,
! session ID,
! per-session line number,
! command tag,
! session start time,
! virtual transaction ID,
! regular transaction ID,
! error severity,
! SQLSTATE code,
! error message,
! error message detail,
! hint,
! internal query that led to the error (if any),
! character count of the error position therein,
! error context,
! user query that led to the error (if any and enabled by
! <varname>log_min_error_statement</>),
! character count of the error position therein,
! location of the error in the PostgreSQL source code
(if <varname>log_error_verbosity</> is set to <literal>verbose</>),
! and application name.
! Here is a sample table definition for storing CSV-format log output:
<programlisting>
CREATE TABLE postgres_log
--- 3764,3833 ----
Including <literal>csvlog</> in the <varname>log_destination</> list
provides a convenient way to import log files into a database table.
This option emits log lines in comma-separated-values
! (<acronym>CSV</>) format. These columns may be included in the CSV
! output:
! log_time, # timestamp with milliseconds
! user_name, # session user name
! role_name, # current role name
! database_name, # database name
! process_id, # process ID
! connection_from, # client host:port number
! session_id, # session ID
! session_line_number, # per-session line number
! command_tag, # command tag
! session_start_time, # session start time
! virtual_transaction_id, # virtual transaction ID
! transaction_id, # regular transaction ID
! error_severity, # error severity
! sql_state_code, # SQLSTATE code
! message, # error message
! detail, # error message detail
! hint, # hint
! internal_query, # internal query that led to the error (if any)
! internal_query_pos, # character count of the error position therein
! context, # error context
! query, # user query that led to the error (if any and enabled by
! <varname>log_min_error_statement</>)
! query_pos, # character count of the error position therein
! location, # location of the error in the PostgreSQL source code
(if <varname>log_error_verbosity</> is set to <literal>verbose</>),
! application_name # application name
!
! The default set of columns does not include current role name, and
! is currently:
!
! log_time,
! user_name,
! database_name,
! process_id,
! connection_from,
! session_id,
! session_line_num,
! command_tag,
! session_start_time,
! virtual_transaction_id,
! transaction_id,
! error_severity,
! sql_state_code,
! message,
! detail,
! hint,
! internal_query,
! internal_query_pos,
! context,
! query,
! query_pos,
! location,
! application_name
!
! The set of columns to be included, and their order, in the CSV
! output can be controlled using the <varname>log_csv_fields</> option.
!
! For additional details on the definition of the above columns, refer
! to the documentation for <varname>log_line_prefix</>.
!
! Here is a sample table definition for storing the default CSV-format
! log output:
<programlisting>
CREATE TABLE postgres_log
*** a/src/backend/commands/variable.c
--- b/src/backend/commands/variable.c
***************
*** 760,765 **** assign_session_authorization(const char *value, bool doit, GucSource source)
--- 760,770 ----
return result;
}
+ /*
+ * function to return the stored session username, needed because we
+ * can't do catalog lookups when possibly being called after an error,
+ * eg: from elog.c or part of GUC handling.
+ */
const char *
show_session_authorization(void)
{
***************
*** 885,890 **** assign_role(const char *value, bool doit, GucSource source)
--- 890,900 ----
return result;
}
+ /*
+ * function to return the stored role username, needed because we
+ * can't do catalog lookups when possibly being called after an error,
+ * eg: from elog.c or part of GUC handling.
+ */
const char *
show_role(void)
{
*** a/src/backend/utils/error/elog.c
--- b/src/backend/utils/error/elog.c
***************
*** 3,8 ****
--- 3,17 ----
* elog.c
* error logging and reporting
*
+ * A few comments about situations where error processing is called:
+ *
+ * We need to be cautious of both a performance hit when logging, since
+ * log messages can be generated at a huge rate if every command is being
+ * logged and we also need to watch out for what can happen when we are
+ * trying to log from an aborted transaction. Specifically, attempting to
+ * do SysCache lookups and possibly use other usually available backend
+ * systems will fail badly when logging from an aborted transaction.
+ *
* Some notes about recursion and errors during error processing:
*
* We need to be robust about recursive-error scenarios --- for example,
***************
*** 59,73 ****
--- 68,85 ----
#include "access/transam.h"
#include "access/xact.h"
+ #include "commands/variable.h"
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
+ #include "nodes/pg_list.h"
#include "postmaster/postmaster.h"
#include "postmaster/syslogger.h"
#include "storage/ipc.h"
#include "storage/proc.h"
#include "tcop/tcopprot.h"
+ #include "utils/builtins.h"
#include "utils/guc.h"
#include "utils/memutils.h"
#include "utils/ps_status.h"
***************
*** 93,98 **** extern bool redirection_done;
--- 105,113 ----
int Log_error_verbosity = PGERROR_VERBOSE;
char *Log_line_prefix = NULL; /* format for extra log line info */
int Log_destination = LOG_DESTINATION_STDERR;
+ char *Log_csv_fields = NULL;
+
+ static List *csv_log_fields = NIL;
#ifdef HAVE_SYSLOG
***************
*** 161,166 **** static void write_csvlog(ErrorData *edata);
--- 176,186 ----
static void setup_formatted_log_time(void);
static void setup_formatted_start_time(void);
+ /* extern'd and used from guc.c... */
+ const char *
+ assign_log_csv_fields(const char *newval, bool doit, GucSource source);
+
+
/*
* in_error_recursion_trouble --- are we at risk of infinite error recursion?
***************
*** 1817,1831 **** log_line_prefix(StringInfo buf, ErrorData *edata)
}
break;
case 'u':
- if (MyProcPort)
{
! const char *username = MyProcPort->user_name;
!
! if (username == NULL || *username == '\0')
! username = _("[unknown]");
! appendStringInfoString(buf, username);
}
break;
case 'd':
if (MyProcPort)
{
--- 1837,1860 ----
}
break;
case 'u':
{
! const char *session_auth = show_session_authorization();
! if (*session_auth != '\0')
! appendStringInfoString(buf, session_auth);
! else
! if (MyProcPort)
! {
! const char *username = MyProcPort->user_name;
!
! if (username == NULL || *username == '\0')
! username = _("[unknown]");
! appendStringInfoString(buf, username);
! }
}
break;
+ case 'U':
+ appendStringInfoString(buf, show_role());
+ break;
case 'd':
if (MyProcPort)
{
***************
*** 1921,1926 **** log_line_prefix(StringInfo buf, ErrorData *edata)
--- 1950,2112 ----
}
/*
+ * Build up the default set of CSV fields to output, in case we need it before
+ * GUC processing is done.
+ *
+ * This is more of a 'safety valve' than anything else,
+ * since GUC processing really should happen before we do any error logging.
+ * We might even want to change this eventually to just not log CSV format logs
+ * if this ever happens, to avoid a discrepency in the CSV log file which would
+ * make it difficult to load into PG.
+ */
+ void
+ build_default_csvlog_list(void)
+ {
+ List *new_csv_fields = NIL;
+ MemoryContext oldcontext;
+
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_LOG_TIME);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_USER_NAME);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_DATABASE_NAME);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_PROCESS_ID);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_CONNECTION_FROM);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_SESSION_ID);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_SESSION_LINE_NUM);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_COMMAND_TAG);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_SESSION_START_TIME);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_VIRTUAL_TRANSACTION_ID);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_TRANSACTION_ID);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_ERROR_SEVERITY);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_SQL_STATE_CODE);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_MESSAGE);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_DETAIL);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_HINT);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_INTERNAL_QUERY);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_INTERNAL_QUERY_POS);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_CONTEXT);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_QUERY);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_QUERY_POS);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_LOCATION);
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_APPLICATION_NAME);
+
+ /* put new list in place */
+ csv_log_fields = new_csv_fields;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return;
+ }
+
+
+ /*
+ * Process the list of fields to be sent to the CSV log file
+ */
+ const char *
+ assign_log_csv_fields(const char *newval, bool doit, GucSource source)
+ {
+ /* Verify the list is valid */
+ List *new_csv_fields = NIL;
+ List *column_list = NIL;
+ ListCell *l;
+ char *rawstring;
+ MemoryContext oldcontext;
+
+ /* Need a modifyable version to pass to SplitIdentifierString */
+ rawstring = pstrdup(newval);
+
+ /* Parse string into list of identifiers */
+ if (!SplitIdentifierString(rawstring, ',', &column_list))
+ {
+ list_free(column_list);
+ return NULL;
+ }
+
+ /*
+ * We need the allocations done for the csv_log_fields list to
+ * be preserved, so allocate them in TopMemoryContext.
+ */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+
+ /*
+ * Loop through all of the fields provided by the user and build
+ * up our new_csv_fields list which will be processed by write_csvlog
+ */
+ foreach(l, column_list)
+ {
+ if (pg_strcasecmp(lfirst(l),"log_time") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_LOG_TIME);
+ else if (pg_strcasecmp(lfirst(l),"user_name") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_USER_NAME);
+ else if (pg_strcasecmp(lfirst(l),"role_name") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_ROLE_NAME);
+ else if (pg_strcasecmp(lfirst(l),"database_name") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_DATABASE_NAME);
+ else if (pg_strcasecmp(lfirst(l),"process_id") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_PROCESS_ID);
+ else if (pg_strcasecmp(lfirst(l),"connection_from") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_CONNECTION_FROM);
+ else if (pg_strcasecmp(lfirst(l),"session_id") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_SESSION_ID);
+ else if (pg_strcasecmp(lfirst(l),"session_line_num") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_SESSION_LINE_NUM);
+ else if (pg_strcasecmp(lfirst(l),"command_tag") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_COMMAND_TAG);
+ else if (pg_strcasecmp(lfirst(l),"session_start_time") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_SESSION_START_TIME);
+ else if (pg_strcasecmp(lfirst(l),"virtual_transaction_id") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_VIRTUAL_TRANSACTION_ID);
+ else if (pg_strcasecmp(lfirst(l),"transaction_id") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_TRANSACTION_ID);
+ else if (pg_strcasecmp(lfirst(l),"error_severity") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_ERROR_SEVERITY);
+ else if (pg_strcasecmp(lfirst(l),"sql_state_code") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_SQL_STATE_CODE);
+ else if (pg_strcasecmp(lfirst(l),"message") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_MESSAGE);
+ else if (pg_strcasecmp(lfirst(l),"detail") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_DETAIL);
+ else if (pg_strcasecmp(lfirst(l),"hint") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_HINT);
+ else if (pg_strcasecmp(lfirst(l),"internal_query") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_INTERNAL_QUERY);
+ else if (pg_strcasecmp(lfirst(l),"internal_query_pos") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_INTERNAL_QUERY_POS);
+ else if (pg_strcasecmp(lfirst(l),"context") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_CONTEXT);
+ else if (pg_strcasecmp(lfirst(l),"query") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_QUERY);
+ else if (pg_strcasecmp(lfirst(l),"query_pos") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_QUERY_POS);
+ else if (pg_strcasecmp(lfirst(l),"location") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_LOCATION);
+ else if (pg_strcasecmp(lfirst(l),"application_name") == 0)
+ new_csv_fields = lappend_int(new_csv_fields,CSVLOG_APPLICATION_NAME);
+ else
+ {
+ /* handle error, might need to do better than this */
+ return NULL;
+ }
+ }
+
+ if (doit)
+ {
+ /* put new list in place */
+ List *old_list = csv_log_fields;
+
+ csv_log_fields = new_csv_fields;
+
+ list_free(old_list);
+ }
+
+ /* Switch back to the calling context */
+ MemoryContextSwitchTo(oldcontext);
+
+ return newval;
+ }
+
+ /*
* append a CSV'd version of a string to a StringInfo
* We use the PostgreSQL defaults for CSV, i.e. quote = escape = '"'
* If it's NULL, append nothing.
***************
*** 1946,1957 **** appendCSVLiteral(StringInfo buf, const char *data)
}
/*
! * Constructs the error message, depending on the Errordata it gets, in a CSV
! * format which is described in doc/src/sgml/config.sgml.
*/
static void
write_csvlog(ErrorData *edata)
{
StringInfoData buf;
bool print_stmt = false;
--- 2132,2145 ----
}
/*
! * Constructs the error message, depending on the Errordata it gets, in the CSV
! * format requested by the user, based on the log_csv_fields GUC.
*/
static void
write_csvlog(ErrorData *edata)
{
+ int num_fields;
+ int curr_field = 0;
StringInfoData buf;
bool print_stmt = false;
***************
*** 1961,1966 **** write_csvlog(ErrorData *edata)
--- 2149,2158 ----
/* has counter been reset in current process? */
static int log_my_pid = 0;
+ ListCell *l;
+
+ const char *session_auth = show_session_authorization();
+
/*
* This is one of the few places where we'd rather not inherit a static
* variable's value from the postmaster. But since we will, reset it when
***************
*** 1977,2134 **** write_csvlog(ErrorData *edata)
initStringInfo(&buf);
/*
! * timestamp with milliseconds
! *
! * Check if the timestamp is already calculated for the syslog message,
! * and use it if so. Otherwise, get the current timestamp. This is done
! * to put same timestamp in both syslog and csvlog messages.
*/
! if (formatted_log_time[0] == '\0')
! setup_formatted_log_time();
! appendStringInfoString(&buf, formatted_log_time);
! appendStringInfoChar(&buf, ',');
! /* username */
! if (MyProcPort)
! appendCSVLiteral(&buf, MyProcPort->user_name);
! appendStringInfoChar(&buf, ',');
! /* database name */
! if (MyProcPort)
! appendCSVLiteral(&buf, MyProcPort->database_name);
! appendStringInfoChar(&buf, ',');
! /* Process id */
! if (MyProcPid != 0)
! appendStringInfo(&buf, "%d", MyProcPid);
! appendStringInfoChar(&buf, ',');
! /* Remote host and port */
! if (MyProcPort && MyProcPort->remote_host)
! {
! appendStringInfoChar(&buf, '"');
! appendStringInfoString(&buf, MyProcPort->remote_host);
! if (MyProcPort->remote_port && MyProcPort->remote_port[0] != '\0')
! {
! appendStringInfoChar(&buf, ':');
! appendStringInfoString(&buf, MyProcPort->remote_port);
! }
! appendStringInfoChar(&buf, '"');
! }
! appendStringInfoChar(&buf, ',');
! /* session id */
! appendStringInfo(&buf, "%lx.%x", (long) MyStartTime, MyProcPid);
! appendStringInfoChar(&buf, ',');
! /* Line number */
! appendStringInfo(&buf, "%ld", log_line_number);
! appendStringInfoChar(&buf, ',');
! /* PS display */
! if (MyProcPort)
! {
! StringInfoData msgbuf;
! const char *psdisp;
! int displen;
! initStringInfo(&msgbuf);
! psdisp = get_ps_display(&displen);
! appendBinaryStringInfo(&msgbuf, psdisp, displen);
! appendCSVLiteral(&buf, msgbuf.data);
! pfree(msgbuf.data);
! }
! appendStringInfoChar(&buf, ',');
! /* session start timestamp */
! if (formatted_start_time[0] == '\0')
! setup_formatted_start_time();
! appendStringInfoString(&buf, formatted_start_time);
! appendStringInfoChar(&buf, ',');
! /* Virtual transaction id */
! /* keep VXID format in sync with lockfuncs.c */
! if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
! appendStringInfo(&buf, "%d/%u", MyProc->backendId, MyProc->lxid);
! appendStringInfoChar(&buf, ',');
! /* Transaction id */
! appendStringInfo(&buf, "%u", GetTopTransactionIdIfAny());
! appendStringInfoChar(&buf, ',');
! /* Error severity */
! appendStringInfoString(&buf, error_severity(edata->elevel));
! appendStringInfoChar(&buf, ',');
! /* SQL state code */
! appendStringInfoString(&buf, unpack_sql_state(edata->sqlerrcode));
! appendStringInfoChar(&buf, ',');
! /* errmessage */
! appendCSVLiteral(&buf, edata->message);
! appendStringInfoChar(&buf, ',');
! /* errdetail or errdetail_log */
! if (edata->detail_log)
! appendCSVLiteral(&buf, edata->detail_log);
! else
! appendCSVLiteral(&buf, edata->detail);
! appendStringInfoChar(&buf, ',');
! /* errhint */
! appendCSVLiteral(&buf, edata->hint);
! appendStringInfoChar(&buf, ',');
! /* internal query */
! appendCSVLiteral(&buf, edata->internalquery);
! appendStringInfoChar(&buf, ',');
! /* if printed internal query, print internal pos too */
! if (edata->internalpos > 0 && edata->internalquery != NULL)
! appendStringInfo(&buf, "%d", edata->internalpos);
! appendStringInfoChar(&buf, ',');
! /* errcontext */
! appendCSVLiteral(&buf, edata->context);
! appendStringInfoChar(&buf, ',');
! /* user query --- only reported if not disabled by the caller */
! if (is_log_level_output(edata->elevel, log_min_error_statement) &&
! debug_query_string != NULL &&
! !edata->hide_stmt)
! print_stmt = true;
! if (print_stmt)
! appendCSVLiteral(&buf, debug_query_string);
! appendStringInfoChar(&buf, ',');
! if (print_stmt && edata->cursorpos > 0)
! appendStringInfo(&buf, "%d", edata->cursorpos);
! appendStringInfoChar(&buf, ',');
!
! /* file error location */
! if (Log_error_verbosity >= PGERROR_VERBOSE)
! {
! StringInfoData msgbuf;
!
! initStringInfo(&msgbuf);
!
! if (edata->funcname && edata->filename)
! appendStringInfo(&msgbuf, "%s, %s:%d",
! edata->funcname, edata->filename,
! edata->lineno);
! else if (edata->filename)
! appendStringInfo(&msgbuf, "%s:%d",
! edata->filename, edata->lineno);
! appendCSVLiteral(&buf, msgbuf.data);
! pfree(msgbuf.data);
! }
! appendStringInfoChar(&buf, ',');
! /* application name */
! if (application_name)
! appendCSVLiteral(&buf, application_name);
appendStringInfoChar(&buf, '\n');
--- 2169,2437 ----
initStringInfo(&buf);
/*
! * Get the number of fields, so we make sure to *not* include a comma
! * after the last field.
*/
! num_fields = list_length(csv_log_fields);
! /*
! * Loop through the fields requested by the user, in the order requested, in
! * the log_csv_fields GUC.
! */
! foreach(l, csv_log_fields)
! {
! /* Update which field we are on, needed to check if it's the last field */
! curr_field++;
! switch (lfirst_int(l))
! {
! case CSVLOG_LOG_TIME:
! {
! /*
! * timestamp with milliseconds
! *
! * Check if the timestamp is already calculated for the syslog message,
! * and use it if so. Otherwise, get the current timestamp. This is done
! * to put same timestamp in both syslog and csvlog messages.
! */
! if (formatted_log_time[0] == '\0')
! setup_formatted_log_time();
!
! appendStringInfoString(&buf, formatted_log_time);
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! }
! break;
! case CSVLOG_USER_NAME:
! {
! /* session username, as done for %u */
! if (*session_auth != '\0')
! appendCSVLiteral(&buf, session_auth);
! else
! /* username */
! if (MyProcPort)
! {
! const char *username = MyProcPort->user_name;
! if (username == NULL || *username == '\0')
! username = _("[unknown]");
! appendCSVLiteral(&buf, MyProcPort->user_name);
! }
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! }
! break;
! case CSVLOG_ROLE_NAME:
! /* current role, not updated if someone renames it in another
! * session, of course */
! appendCSVLiteral(&buf, show_role());
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! break;
! case CSVLOG_DATABASE_NAME:
! {
! /* database name */
! if (MyProcPort)
! appendCSVLiteral(&buf, MyProcPort->database_name);
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! }
! break;
! case CSVLOG_PROCESS_ID:
! {
! /* Process id */
! if (MyProcPid != 0)
! appendStringInfo(&buf, "%d", MyProcPid);
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! }
! break;
! case CSVLOG_CONNECTION_FROM:
! {
! /* Remote host and port */
! if (MyProcPort && MyProcPort->remote_host)
! {
! appendStringInfoChar(&buf, '"');
! appendStringInfoString(&buf, MyProcPort->remote_host);
! if (MyProcPort->remote_port && MyProcPort->remote_port[0] != '\0')
! {
! appendStringInfoChar(&buf, ':');
! appendStringInfoString(&buf, MyProcPort->remote_port);
! }
! appendStringInfoChar(&buf, '"');
! }
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! }
! break;
! case CSVLOG_SESSION_ID:
! /* session id */
! appendStringInfo(&buf, "%lx.%x", (long) MyStartTime, MyProcPid);
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! break;
! case CSVLOG_SESSION_LINE_NUM:
! /* Line number */
! appendStringInfo(&buf, "%ld", log_line_number);
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! break;
! case CSVLOG_COMMAND_TAG:
! {
! /* PS display */
! if (MyProcPort)
! {
! StringInfoData msgbuf;
! const char *psdisp;
! int displen;
!
! initStringInfo(&msgbuf);
!
! psdisp = get_ps_display(&displen);
! appendBinaryStringInfo(&msgbuf, psdisp, displen);
! appendCSVLiteral(&buf, msgbuf.data);
!
! pfree(msgbuf.data);
! }
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! }
! break;
! case CSVLOG_SESSION_START_TIME:
! {
! /* session start timestamp */
! if (formatted_start_time[0] == '\0')
! setup_formatted_start_time();
! appendStringInfoString(&buf, formatted_start_time);
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! }
! break;
! case CSVLOG_VIRTUAL_TRANSACTION_ID:
! {
! /* Virtual transaction id */
! /* keep VXID format in sync with lockfuncs.c */
! if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
! appendStringInfo(&buf, "%d/%u", MyProc->backendId, MyProc->lxid);
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! }
! break;
! case CSVLOG_TRANSACTION_ID:
! /* Transaction id */
! appendStringInfo(&buf, "%u", GetTopTransactionIdIfAny());
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! break;
! case CSVLOG_ERROR_SEVERITY:
! /* Error severity */
! appendStringInfoString(&buf, error_severity(edata->elevel));
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! break;
! case CSVLOG_SQL_STATE_CODE:
! /* SQL state code */
! appendStringInfoString(&buf, unpack_sql_state(edata->sqlerrcode));
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! break;
! case CSVLOG_MESSAGE:
! /* errmessage */
! appendCSVLiteral(&buf, edata->message);
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! break;
! case CSVLOG_DETAIL:
! {
! /* errdetail or errdetail_log */
! if (edata->detail_log)
! appendCSVLiteral(&buf, edata->detail_log);
! else
! appendCSVLiteral(&buf, edata->detail);
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! }
! break;
! case CSVLOG_HINT:
! /* errhint */
! appendCSVLiteral(&buf, edata->hint);
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! break;
! case CSVLOG_INTERNAL_QUERY:
! /* internal query */
! appendCSVLiteral(&buf, edata->internalquery);
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! break;
! case CSVLOG_INTERNAL_QUERY_POS:
! {
! /* if printed internal query, print internal pos too */
! if (edata->internalpos > 0 && edata->internalquery != NULL)
! appendStringInfo(&buf, "%d", edata->internalpos);
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! }
! break;
! case CSVLOG_CONTEXT:
! /* errcontext */
! appendCSVLiteral(&buf, edata->context);
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! break;
! case CSVLOG_QUERY:
! {
! /* user query --- only reported if not disabled by the caller */
! if (is_log_level_output(edata->elevel, log_min_error_statement) &&
! debug_query_string != NULL &&
! !edata->hide_stmt)
! print_stmt = true;
! if (print_stmt)
! appendCSVLiteral(&buf, debug_query_string);
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! }
! break;
! case CSVLOG_QUERY_POS:
! {
! if (print_stmt && edata->cursorpos > 0)
! appendStringInfo(&buf, "%d", edata->cursorpos);
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! }
! break;
! case CSVLOG_LOCATION:
! {
! /* file error location */
! if (Log_error_verbosity >= PGERROR_VERBOSE)
! {
! StringInfoData msgbuf;
!
! initStringInfo(&msgbuf);
!
! if (edata->funcname && edata->filename)
! appendStringInfo(&msgbuf, "%s, %s:%d",
! edata->funcname, edata->filename,
! edata->lineno);
! else if (edata->filename)
! appendStringInfo(&msgbuf, "%s:%d",
! edata->filename, edata->lineno);
! appendCSVLiteral(&buf, msgbuf.data);
! pfree(msgbuf.data);
! }
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! }
! break;
!
! case CSVLOG_APPLICATION_NAME:
! {
! /* application name */
! if (application_name)
! appendCSVLiteral(&buf, application_name);
! if (curr_field != num_fields) appendStringInfoChar(&buf, ',');
! }
! break;
! }
! }
appendStringInfoChar(&buf, '\n');
***************
*** 2139,2144 **** write_csvlog(ErrorData *edata)
--- 2442,2449 ----
write_pipe_chunks(buf.data, buf.len, LOG_DESTINATION_CSVLOG);
pfree(buf.data);
+
+ return;
}
/*
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
***************
*** 63,68 ****
--- 63,69 ----
#include "tsearch/ts_cache.h"
#include "utils/builtins.h"
#include "utils/bytea.h"
+ #include "utils/elog.h"
#include "utils/guc_tables.h"
#include "utils/memutils.h"
#include "utils/pg_locale.h"
***************
*** 190,195 **** static char *config_enum_get_options(struct config_enum * record,
--- 191,199 ----
const char *prefix, const char *suffix,
const char *separator);
+ /* Needs to be defined here because elog.h can't #include guc.h */
+ extern const char *assign_log_csv_fields(const char *newval,
+ bool doit, GucSource source);
/*
* Options for enum values defined in this module.
***************
*** 2287,2292 **** static struct config_string ConfigureNamesString[] =
--- 2291,2307 ----
},
{
+ {"log_csv_fields", PGC_POSTMASTER, LOGGING_WHAT,
+ gettext_noop("Controls fields logged to CSV logfiles."),
+ gettext_noop("If blank, the default set of fields is used."),
+ GUC_LIST_INPUT
+ },
+ &Log_csv_fields,
+ "log_time, user_name, database_name, process_id, connection_from, session_id, session_line_num, command_tag, session_start_time, virtual_transaction_id, transaction_id, error_severity, sql_state_code, message, detail, hint, internal_query, internal_query_pos, context, query, query_pos, location, application_name",
+ assign_log_csv_fields, NULL
+ },
+
+ {
{"log_timezone", PGC_SIGHUP, LOGGING_WHAT,
gettext_noop("Sets the time zone to use in log messages."),
NULL
***************
*** 3436,3441 **** InitializeGUCOptions(void)
--- 3451,3462 ----
pg_timezone_pre_initialize();
/*
+ * Ditto for log_csv_fields, have to set it to something before we get
+ * too far along.
+ */
+ build_default_csvlog_list();
+
+ /*
* Build sorted array of all GUC variables.
*/
build_guc_variables();
*** a/src/backend/utils/misc/postgresql.conf.sample
--- b/src/backend/utils/misc/postgresql.conf.sample
***************
*** 377,382 ****
--- 377,386 ----
# processes
# %% = '%'
# e.g. '<%u%%%d> '
+
+ # fields to include in the CSV log output
+ #log_csv_fields = 'log_time, user_name, database_name, process_id, connection_from, session_id, session_line_num, command_tag, session_start_time, virtual_transaction_id, transaction_id, error_severity, sql_state_code, message, detail, hint, internal_query, internal_query_pos, context, query, query_pos, location, application_name'
+
#log_lock_waits = off # log lock waits >= deadlock_timeout
#log_statement = 'none' # none, ddl, mod, all
#log_temp_files = -1 # log temporary files equal or larger
*** a/src/include/utils/elog.h
--- b/src/include/utils/elog.h
***************
*** 330,337 **** typedef enum
--- 330,366 ----
extern int Log_error_verbosity;
extern char *Log_line_prefix;
+ extern char *Log_csv_fields;
extern int Log_destination;
+ typedef enum LogCSVFields
+ {
+ CSVLOG_LOG_TIME,
+ CSVLOG_USER_NAME,
+ CSVLOG_ROLE_NAME,
+ CSVLOG_DATABASE_NAME,
+ CSVLOG_PROCESS_ID,
+ CSVLOG_CONNECTION_FROM,
+ CSVLOG_SESSION_ID,
+ CSVLOG_SESSION_LINE_NUM,
+ CSVLOG_COMMAND_TAG,
+ CSVLOG_SESSION_START_TIME,
+ CSVLOG_VIRTUAL_TRANSACTION_ID,
+ CSVLOG_TRANSACTION_ID,
+ CSVLOG_ERROR_SEVERITY,
+ CSVLOG_SQL_STATE_CODE,
+ CSVLOG_MESSAGE,
+ CSVLOG_DETAIL,
+ CSVLOG_HINT,
+ CSVLOG_INTERNAL_QUERY,
+ CSVLOG_INTERNAL_QUERY_POS,
+ CSVLOG_CONTEXT,
+ CSVLOG_QUERY,
+ CSVLOG_QUERY_POS,
+ CSVLOG_LOCATION,
+ CSVLOG_APPLICATION_NAME
+ } LogCSVFields;
+
/* Log destination bitmap */
#define LOG_DESTINATION_STDERR 1
#define LOG_DESTINATION_SYSLOG 2
***************
*** 343,348 **** extern void DebugFileOpen(void);
--- 372,382 ----
extern char *unpack_sql_state(int sql_state);
extern bool in_error_recursion_trouble(void);
+ /* Used by guc.c to set up the default set of
+ * csv fields to log
+ */
+ extern void build_default_csvlog_list(void);
+
#ifdef HAVE_SYSLOG
extern void set_syslog_parameters(const char *ident, int facility);
#endif
*** a/src/tools/pgindent/typedefs.list
--- b/src/tools/pgindent/typedefs.list
***************
*** 854,859 **** LockTagType
--- 854,860 ----
LockTupleMode
LockingClause
LogStmtLevel
+ LogCSVFields
LogicalTape
LogicalTapeSet
MAGIC
signature.asc
Description: Digital signature
