Re: [HACKERS] Non-empty default log_line_prefix
On Fri, Oct 14, 2016 at 7:04 AM, Christoph Berg wrote: > Re: Stephen Frost 2016-10-12 <20161012190732.gj13...@tamriel.snowman.net> >> For my 2c, I'd rather have %m, but I definitely agree with Robert that >> we need to do *something* here and if the only thing holding us back is >> %t vs. %m, then let's just pick one and move on. I'll just hold my nose >> when I see the default and change it to %m. > > Here's the very same patch with %m instead of %t. Pick one :) > > (Fwiw, I'm still leaning towards %t, but my eyes are becoming more and > more accustomed to %m as well. I'd be fine with it as well. (I'd > rather want to try to get rid of the timezone identifier there...)) So, surveying this thread, what I see is: People who prefer %m: Jeff Janes, Devrim Gündüz, Stephen Frost People who prefer %t: Christoph Berg, Tom Lane (I think) We can kibitz this more later, but for now I've committed it with %m. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Non-empty default log_line_prefix
Re: Stephen Frost 2016-10-12 <20161012190732.gj13...@tamriel.snowman.net> > For my 2c, I'd rather have %m, but I definitely agree with Robert that > we need to do *something* here and if the only thing holding us back is > %t vs. %m, then let's just pick one and move on. I'll just hold my nose > when I see the default and change it to %m. Here's the very same patch with %m instead of %t. Pick one :) (Fwiw, I'm still leaning towards %t, but my eyes are becoming more and more accustomed to %m as well. I'd be fine with it as well. (I'd rather want to try to get rid of the timezone identifier there...)) Christoph diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml new file mode 100644 index e826c19..bec9483 *** a/doc/src/sgml/config.sgml --- b/doc/src/sgml/config.sgml *** local0.*/var/log/postgresql *** 5004,5010 value will pad on the left. Padding can be useful to aid human readability in log files. This parameter can only be set in the postgresql.conf ! file or on the server command line. The default is an empty string. --- 5004,5011 value will pad on the left. Padding can be useful to aid human readability in log files. This parameter can only be set in the postgresql.conf ! file or on the server command line. The default is ! %m [%p] which logs a time stamp and the process ID. *** FROM pg_stat_activity; *** 5142,5147 --- 5143,5159 include those escapes if you are logging to syslog. + + + + The %q escape is useful when including information that is + only available in session (backend) context like user or database + name. An example would be: + + log_line_prefix = '%m [%p] %q%u@%d/%a ' + + + diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c new file mode 100644 index 622279b..65660c1 *** a/src/backend/utils/misc/guc.c --- b/src/backend/utils/misc/guc.c *** static struct config_string ConfigureNam *** 3014,3020 gettext_noop("If blank, no prefix is used.") }, &Log_line_prefix, ! "", NULL, NULL, NULL }, --- 3014,3020 gettext_noop("If blank, no prefix is used.") }, &Log_line_prefix, ! "%m [%p] ", NULL, NULL, NULL }, diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample new file mode 100644 index 05b1373..159ada3 *** a/src/backend/utils/misc/postgresql.conf.sample --- b/src/backend/utils/misc/postgresql.conf.sample *** *** 430,436 #log_duration = off #log_error_verbosity = default# terse, default, or verbose messages #log_hostname = off ! #log_line_prefix = '' # special values: # %a = application name # %u = user name # %d = database name --- 430,436 #log_duration = off #log_error_verbosity = default# terse, default, or verbose messages #log_hostname = off ! #log_line_prefix = '%m [%p] ' # special values: # %a = application name # %u = user name # %d = database name -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Non-empty default log_line_prefix
Peter Eisentraut wrote: > On 10/12/16 11:58 AM, Christoph Berg wrote: > > (Yes, the '' default might be fine for syslog, but I don't think > > that's a good argument for sticking with it for default installs. I've > > seen way too many useless log files out there, and at worst we'll have > > syslogs with two timestamps.) > > We'd have to think of a way to sort this out during upgrades. Sure, but we don't have to do that in this patch, do we? Some systems such as Debian copy the postgresql.conf file from the old installation to the new one, and then have specific code to adjust settings that no longer exist in the new version. I'm not a fan of that approach myself precisely because if PGDG installs new defaults (such as in this case), the sites using that upgrade tool don't benefit. A better approach might be to copy over the non-default settings from the old install to the new. If Debian were to do that then we wouldn't need to do anything here. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Non-empty default log_line_prefix
On 10/12/16 11:58 AM, Christoph Berg wrote: > (Yes, the '' default might be fine for syslog, but I don't think > that's a good argument for sticking with it for default installs. I've > seen way too many useless log files out there, and at worst we'll have > syslogs with two timestamps.) We'd have to think of a way to sort this out during upgrades. Documentation might be enough. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Non-empty default log_line_prefix
On 10/12/16 11:40 AM, Tom Lane wrote: > There would be some value in the complexity you're thinking of for > installations that log to multiple targets concurrently, but really, > who does that? I see that a lot. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Non-empty default log_line_prefix
* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Oct 12, 2016 at 11:20 AM, Christoph Berg wrote: > > Re: Jeff Janes 2016-10-12 > > > >> Do you think the pushback will come from people who just accept the > >> defaults? > > > > I'm concerned about readability. "2016-10-12 20:14:30.449 CEST" is a > > lot of digits. My eyes can parse "20:14:30" as a timestamp, but > > "20:14:30.449" looks more like an IP address. (Admittedly I don't have > > experience with reading %m logs.) > > > > Overall, I'd prefer %t but %m would be ok as well. > > I'm fine with either! Both are much better than the empty string. > One of the problems with the status quo is that many users don't even > realize that log_line_prefix exists, so they don't configure it at > all. They don't even realize that they have the option to add a > prefix. I think configuring a non-empty default will be both better > by default and more likely to make people realize that they have > choices. For my 2c, I'd rather have %m, but I definitely agree with Robert that we need to do *something* here and if the only thing holding us back is %t vs. %m, then let's just pick one and move on. I'll just hold my nose when I see the default and change it to %m. Thanks! Stephen signature.asc Description: Digital signature
Re: [HACKERS] Non-empty default log_line_prefix
Hi, On Wed, 2016-10-12 at 13:11 -0400, Tom Lane wrote: > > What is the cost of using %m, other than 4 (rather compressible) bytes per > > log entry? > > More log I/O, which is not free ... FWIW, we've been setting log_line_prefix to '< %m > ' for quite a long time in PGDG RPMs, and did not get any complaints. I'd vote for %m for default. Regards, -- Devrim GÜNDÜZ EnterpriseDB: http://www.enterprisedb.com PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer Twitter: @DevrimGunduz , @DevrimGunduzTR signature.asc Description: This is a digitally signed message part
Re: [HACKERS] Non-empty default log_line_prefix
On Wed, Oct 12, 2016 at 11:20 AM, Christoph Berg wrote: > Re: Jeff Janes 2016-10-12 > >> Do you think the pushback will come from people who just accept the >> defaults? > > I'm concerned about readability. "2016-10-12 20:14:30.449 CEST" is a > lot of digits. My eyes can parse "20:14:30" as a timestamp, but > "20:14:30.449" looks more like an IP address. (Admittedly I don't have > experience with reading %m logs.) > > Overall, I'd prefer %t but %m would be ok as well. I'm fine with either! Both are much better than the empty string. One of the problems with the status quo is that many users don't even realize that log_line_prefix exists, so they don't configure it at all. They don't even realize that they have the option to add a prefix. I think configuring a non-empty default will be both better by default and more likely to make people realize that they have choices. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Non-empty default log_line_prefix
Re: Jeff Janes 2016-10-12 > Do you think the pushback will come from people who just accept the > defaults? I'm concerned about readability. "2016-10-12 20:14:30.449 CEST" is a lot of digits. My eyes can parse "20:14:30" as a timestamp, but "20:14:30.449" looks more like an IP address. (Admittedly I don't have experience with reading %m logs.) Overall, I'd prefer %t but %m would be ok as well. Christoph -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Non-empty default log_line_prefix
On Wed, Oct 12, 2016 at 10:11 AM, Tom Lane wrote: > Jeff Janes writes: > > On Sun, Oct 2, 2016 at 1:20 PM, Christoph Berg wrote: > >> Patch attached. (Still using %t, I don't think %m makes sense for the > >> default.) > > > What is the cost of using %m, other than 4 (rather compressible) bytes > per > > log entry? > > More log I/O, which is not free ... and that remark about compressibility > is bogus for anyone who doesn't pipe their postmaster stderr into gzip. > I'm already afraid that adding the timestamps will get us some pushback > about log volume. > I don't pipe them into gzip, but every few months I go and pxz any of them more than few months old. Do you think the pushback will come from people who just accept the defaults? Cheers, Jeff
Re: [HACKERS] Non-empty default log_line_prefix
Jeff Janes writes: > On Sun, Oct 2, 2016 at 1:20 PM, Christoph Berg wrote: >> Patch attached. (Still using %t, I don't think %m makes sense for the >> default.) > What is the cost of using %m, other than 4 (rather compressible) bytes per > log entry? More log I/O, which is not free ... and that remark about compressibility is bogus for anyone who doesn't pipe their postmaster stderr into gzip. I'm already afraid that adding the timestamps will get us some pushback about log volume. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Non-empty default log_line_prefix
On Sun, Oct 2, 2016 at 1:20 PM, Christoph Berg wrote: > Re: Tom Lane 2016-09-29 <18642.1475159...@sss.pgh.pa.us> > > > Possibly the longer version could be added as an example in the > > > documentation. > > > > I suspect that simply having a nonempty default in the first place > > is going to do more to raise peoples' awareness than anything we > > could do in the documentation. But perhaps an example along these > > lines would be useful for showing proper use of %q. > > Patch attached. (Still using %t, I don't think %m makes sense for the > default.) > I don't agree with that part. When looking at sections of log files that people post on help forums, I've often wished people had shared milliseconds, and I've never wished they had truncated them off. If two messages are separated by 0.950 seconds, it can have entirely different implications than if they are separated by 0.002 seconds. What is the cost of using %m, other than 4 (rather compressible) bytes per log entry? Cheers, Jeff
Re: [HACKERS] Non-empty default log_line_prefix
Re: Peter Eisentraut 2016-10-12 <0caa6d7f-deb6-9a43-2b38-60e63af93...@2ndquadrant.com> > >> > is going to do more to raise peoples' awareness than anything we > >> > could do in the documentation. But perhaps an example along these > >> > lines would be useful for showing proper use of %q. > > Patch attached. (Still using %t, I don't think %m makes sense for the > > default.) > > That still doesn't address what to do about syslog and eventlog users. > We would need either a separate prefix setting for those, or maybe > something like %q that says, skip to here if using syslog. (I don't > know eventlog, so I don't know if a common setting for syslog and > eventlog would work.) This patch simply tries to fix the default (stderr + '') which wasn't useful for anyone. Note that there is already a hint in the documentation that says timestamps and PID are not useful for syslog. (Yes, the '' default might be fine for syslog, but I don't think that's a good argument for sticking with it for default installs. I've seen way too many useless log files out there, and at worst we'll have syslogs with two timestamps.) Christoph -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Non-empty default log_line_prefix
Peter Eisentraut writes: > That still doesn't address what to do about syslog and eventlog users. > We would need either a separate prefix setting for those, or maybe > something like %q that says, skip to here if using syslog. (I don't > know eventlog, so I don't know if a common setting for syslog and > eventlog would work.) Meh. Those aren't default log output targets, so I don't think the default prefix needs to cater for them. People who adjust one setting can adjust the other too. There would be some value in the complexity you're thinking of for installations that log to multiple targets concurrently, but really, who does that? Most production installations already begrudge the I/O volume for a single log target. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Non-empty default log_line_prefix
On 10/2/16 4:20 PM, Christoph Berg wrote: >> I suspect that simply having a nonempty default in the first place >> > is going to do more to raise peoples' awareness than anything we >> > could do in the documentation. But perhaps an example along these >> > lines would be useful for showing proper use of %q. > Patch attached. (Still using %t, I don't think %m makes sense for the > default.) That still doesn't address what to do about syslog and eventlog users. We would need either a separate prefix setting for those, or maybe something like %q that says, skip to here if using syslog. (I don't know eventlog, so I don't know if a common setting for syslog and eventlog would work.) -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Non-empty default log_line_prefix
Re: Tom Lane 2016-09-29 <18642.1475159...@sss.pgh.pa.us> > > Possibly the longer version could be added as an example in the > > documentation. > > I suspect that simply having a nonempty default in the first place > is going to do more to raise peoples' awareness than anything we > could do in the documentation. But perhaps an example along these > lines would be useful for showing proper use of %q. Patch attached. (Still using %t, I don't think %m makes sense for the default.) Christoph diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml new file mode 100644 index e826c19..a4d8b74 *** a/doc/src/sgml/config.sgml --- b/doc/src/sgml/config.sgml *** local0.*/var/log/postgresql *** 5004,5010 value will pad on the left. Padding can be useful to aid human readability in log files. This parameter can only be set in the postgresql.conf ! file or on the server command line. The default is an empty string. --- 5004,5011 value will pad on the left. Padding can be useful to aid human readability in log files. This parameter can only be set in the postgresql.conf ! file or on the server command line. The default is ! %t [%p] which logs a time stamp and the process ID. *** FROM pg_stat_activity; *** 5142,5147 --- 5143,5159 include those escapes if you are logging to syslog. + + + + The %q escape is useful when including information that is + only available in session (backend) context like user or database + name. An example would be: + + log_line_prefix = '%t [%p] %q%u@%d/%a ' + + + diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c new file mode 100644 index cced814..b71fa93 *** a/src/backend/utils/misc/guc.c --- b/src/backend/utils/misc/guc.c *** static struct config_string ConfigureNam *** 3014,3020 gettext_noop("If blank, no prefix is used.") }, &Log_line_prefix, ! "", NULL, NULL, NULL }, --- 3014,3020 gettext_noop("If blank, no prefix is used.") }, &Log_line_prefix, ! "%t [%p] ", NULL, NULL, NULL }, diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample new file mode 100644 index 05b1373..9eaa23e *** a/src/backend/utils/misc/postgresql.conf.sample --- b/src/backend/utils/misc/postgresql.conf.sample *** *** 430,436 #log_duration = off #log_error_verbosity = default# terse, default, or verbose messages #log_hostname = off ! #log_line_prefix = '' # special values: # %a = application name # %u = user name # %d = database name --- 430,436 #log_duration = off #log_error_verbosity = default# terse, default, or verbose messages #log_hostname = off ! #log_line_prefix = '%t [%p] ' # special values: # %a = application name # %u = user name # %d = database name -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers