Bug#849531: Possible security problem, new logwatch sends mails with charset UTF-8
Hi, Am 2016-12-28 um 10:09 schrieb Klaus Ethgen: > Package: logwatch > Version: 7.4.3+git20161207-1 > Severity: critical > > Current logwatch did change from sending mails with charset iso-8859-1 > to UTF-8. This openes up a potential security hole as UTF-8 is not able > to display all 8bit data. in order to fix this bug for stretch (next Debian stable release), I will upload a version of logwatch that reverts the change from iso-8859-1 to utf-8 for mails sent by logwatch. This is not my preferred way to fix this issue, but any better fix I can imagine will require some new configuration options, which I don't want to introduce in Debian before they are committed upstream. Bye Willi
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem, new logwatch sends mails with charset UTF-8
Hi Klaus, Am 2017-01-15 um 17:43 schrieb Klaus Ethgen: > Hi Willi, > > Am Sa den 14. Jan 2017 um 16:43 schrieb Willi Mann: >> in order to come closer to a fix for this issue, I propose the following >> two patches: > > >> 0001-Add-outputencoding-parameter.patch > >> This patch allows to configure the value for the charset in the >> Content-Type line in mail output. This should address Klaus Ethgen's >> original concern. > > Well, partly. It will, in fact, let me configure it the way, that it > solves my issue. > > However, It does not fix the issue itself as it does not ensure that the > output in mail is that charset. As I mentioned before, this is exactly > the issue. The mail could be easily in UTF-8. But then there has to be > logic to keep sure that the mail is in that charset. Without, you could > still end with logical incorrect mails that are (partly) in different > charsets that is illegal for UTF-8. I understand that the illegal UTF-8 is bad. But I have to admit that I'm not yet sure why it is bad in terms of security, especially since we are talking about mail - and anyone could send you an e-mail with illegal UTF-8 sequences. >> Since most people use UTF-8, I left the default at UTF-8. > > No objection about that. But you cannot be sure that log stuff is in the > same charset. Yeah, that I'm aware of. >> 0002-Use-pager-on-stdout-output-to-terminal.patch > [...] >> Let me know what you think about these patches. > > Patches looks good for me for fixing the bug partly. Nevertheless I > would expect some use of iconv for really ensure the choosed charset. > Maybe even encguess can come to use for guessing the input. Does anybody have a good idea on how to implement this? Concerning the second part (guessing the input encoding), I'm not really sure at what point logwatch should try to guess and assume a particular charset. Especially for the first part, does anybody know some example code? Maybe some other project that had to solve the same issue in Perl? Willi
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem, new logwatch sends mails with charset UTF-8
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi Willi, Am Sa den 14. Jan 2017 um 16:43 schrieb Willi Mann: > in order to come closer to a fix for this issue, I propose the following > two patches: > > > 0001-Add-outputencoding-parameter.patch > > This patch allows to configure the value for the charset in the > Content-Type line in mail output. This should address Klaus Ethgen's > original concern. Well, partly. It will, in fact, let me configure it the way, that it solves my issue. However, It does not fix the issue itself as it does not ensure that the output in mail is that charset. As I mentioned before, this is exactly the issue. The mail could be easily in UTF-8. But then there has to be logic to keep sure that the mail is in that charset. Without, you could still end with logical incorrect mails that are (partly) in different charsets that is illegal for UTF-8. > Since most people use UTF-8, I left the default at UTF-8. No objection about that. But you cannot be sure that log stuff is in the same charset. > 0002-Use-pager-on-stdout-output-to-terminal.patch [...] > Let me know what you think about these patches. Patches looks good for me for fixing the bug partly. Nevertheless I would expect some use of iconv for really ensure the choosed charset. Maybe even encguess can come to use for guessing the input. Regards Klaus - -- Klaus Ethgen http://www.ethgen.ch/ pub 4096R/4E20AF1C 2011-05-16Klaus Ethgen Fingerprint: 85D4 CA42 952C 949B 1753 62B3 79D0 B06F 4E20 AF1C -BEGIN PGP SIGNATURE- Comment: Charset: ISO-8859-1 iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlh7pqYACgkQpnwKsYAZ 9qzURAwAvO09xg/6VmP4PyI4jJd5lnULPJtbCcIsxypOut7fYuZATF6RrUfe1at1 GyTKhQ2aayFK1TIzwMifkD96fj3wbCV35WZm2LBAAiGkKBFs1ZasN+Zj1JkGuYiL frFHBmO49TP5YKWWRCNGrg16mxfn6QdyIqKOpPqRJeNUhuhWE05vhecjG+MzJZ9C d25JO12IooHk3np7BzXHmLn4HtJawHoKHdob6WlnyIhg2rJG8WgE44zrNYlYrO5T IbRXuiLRTGmuxuNfPqEuwLohkFbP/tPGhpgeUPV6G26THhPy0aZBENKzPoLvwJ8/ 1/RxpIe++gr5uil9xS/0zjFYZwn+TrD3rkobuIArkpmEH17v733tP9922t1YWLTU bxRtKWg6sGQMY0QsFC+0+GATJD5FHQOHQQ+fHByU+kvLHYnKvD/gPprzNuWr3Mye QF/6zHSqSzdGTDfPyl3dYz438iSOnZemsuPBTEtzL9L+0+KuRRKUgPgV5omLt4oT BZ9YB7JA =qup2 -END PGP SIGNATURE-
Bug#849531: [Logwatch-devel] Bug#849531: Bug#849531: Possible security problem, new logwatch sends mails with charset UTF-8
Hi, Am 2017-01-14 um 16:48 schrieb Mike Tremaine: > >> 0002-Use-pager-on-stdout-output-to-terminal.patch >> >> Use pager less if output is on terminal. This should address the issues >> associated with escape sequences in logs that may mess with your >> terminal. Less seems to be good at filtering these escape sequences. > > > Sounds good, but I’d make it a config switch and leave the default as is. > Some “lazy” destro's let cron handle terminal output to mail directly… :/ But, then "-t STDOUT" is false (see patch) - in this case, the pager will not be invoked. I also made the pager configurable and it can be set to am empty string to disable). I updated the help text in the second patch to document that better. Willi >From 162f8b2e7fae134e65761dc64e0a8d29c9c172d4 Mon Sep 17 00:00:00 2001 From: Willi Mann Date: Sat, 14 Jan 2017 15:32:39 +0100 Subject: [PATCH 1/2] Add outputencoding parameter --- conf/logwatch.conf | 8 scripts/logwatch.pl | 8 +--- 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/conf/logwatch.conf b/conf/logwatch.conf index 2b61ec6..0d0e31b 100644 --- a/conf/logwatch.conf +++ b/conf/logwatch.conf @@ -126,4 +126,12 @@ mailer = "/usr/sbin/sendmail -t" # #HostLimit = myhost +# +# With this option, the output encoding for mail headers can be set. It +# defaults to utf-8. Be aware that the default value may change to autodetect +# (take the system's encoding). Note that the configured value will be +# upper-cased. +# +# OutputEncoding = utf-8 + # vi: shiftwidth=3 tabstop=3 et diff --git a/scripts/logwatch.pl b/scripts/logwatch.pl index 0167755..0cff511 100755 --- a/scripts/logwatch.pl +++ b/scripts/logwatch.pl @@ -90,6 +90,7 @@ $Config{'pathtobzcat'} = "bzcat"; $Config{'output'} = "stdout"; #8.0 $Config{'format'} = "text"; #8.0 $Config{'encode'} = "none"; #8.0 +$Config{'outputencoding'} = "utf-8"; $Config{'hostformat'} = "none"; #8.0 $Config{'html_wrap'} = 80; $Config{'supress_ignores'} = 0; @@ -1160,11 +1161,12 @@ sub initprint { } else { $out_mime .= "Content-Transfer-Encoding: 7bit\n"; } - #Config{output} html + #Config{output} html, Config{outputenconding} + my $encoding = uc($Config{outputencoding}); if ( $Config{'format'} eq "html" ) { -$out_mime .= "Content-Type: text/html; charset=\"UTF-8\"\n\n"; +$out_mime .= "Content-Type: text/html; charset=\"$encoding\"\n\n"; } else { -$out_mime .= "Content-Type: text/plain; charset=\"UTF-8\"\n\n"; +$out_mime .= "Content-Type: text/plain; charset=\"$encoding\"\n\n"; } if ($Config{'hostformat'} eq "split") { #8.0 check hostlimit also? or ne none? -- 2.1.4 >From 0f52b6b4a35e8aa33d2827f90d515491c15a3d41 Mon Sep 17 00:00:00 2001 From: Willi Mann Date: Sat, 14 Jan 2017 16:25:30 +0100 Subject: [PATCH 2/2] Use pager on stdout output to terminal --- conf/logwatch.conf | 5 + scripts/logwatch.pl | 7 ++- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/conf/logwatch.conf b/conf/logwatch.conf index 0d0e31b..c46b5d6 100644 --- a/conf/logwatch.conf +++ b/conf/logwatch.conf @@ -134,4 +134,9 @@ mailer = "/usr/sbin/sendmail -t" # # OutputEncoding = utf-8 +# +# For output type stdout, logwatch uses the pager Pager if STDOUT is a terminal. +# Am empty string disables the pager. +#Pager = /usr/bin/less + # vi: shiftwidth=3 tabstop=3 et diff --git a/scripts/logwatch.pl b/scripts/logwatch.pl index 0cff511..f053cb3 100755 --- a/scripts/logwatch.pl +++ b/scripts/logwatch.pl @@ -88,6 +88,7 @@ $Config{'pathtocat'} = "cat"; $Config{'pathtozcat'} = "zcat"; $Config{'pathtobzcat'} = "bzcat"; $Config{'output'} = "stdout"; #8.0 +$Config{'pager'} = "/usr/bin/less"; #8.0 $Config{'format'} = "text"; #8.0 $Config{'encode'} = "none"; #8.0 $Config{'outputencoding'} = "utf-8"; @@ -1127,7 +1128,11 @@ sub initprint { $OStitle = "Solaris" if ($OSname eq "SunOS" && $release >= 2); if ($Config{'output'} eq "stdout") { #8.0 start with others? - *OUTFILE = *STDOUT; + if($Config{'pager'} and -t STDOUT) { + open(OUTFILE,"|$Config{'pager'}") or die "could not open pager: $Config{'pager'} $!"; + } else { + *OUTFILE = *STDOUT; + } } elsif ($Config{'output'} eq "file") { open(OUTFILE,">>" . $Config{'filename'}) or die "Can't open output file: $Config{'filename'} $!\n"; } else { -- 2.1.4
Bug#849531: [Logwatch-devel] Bug#849531: Bug#849531: Possible security problem, new logwatch sends mails with charset UTF-8
> On Jan 14, 2017, at 7:43 AM, Willi Mann wrote: > > Hi, > > in order to come closer to a fix for this issue, I propose the following > two patches: > > > 0001-Add-outputencoding-parameter.patch > > This patch allows to configure the value for the charset in the > Content-Type line in mail output. This should address Klaus Ethgen's > original concern. Since most people use UTF-8, I left the default at UTF-8. Sounds good. > > 0002-Use-pager-on-stdout-output-to-terminal.patch > > Use pager less if output is on terminal. This should address the issues > associated with escape sequences in logs that may mess with your > terminal. Less seems to be good at filtering these escape sequences. Sounds good, but I’d make it a config switch and leave the default as is. Some “lazy” destro's let cron handle terminal output to mail directly… :/ -Mike > > > Let me know what you think about these patches. > > Willi > > Am 2017-01-01 um 23:01 schrieb Mike Tremaine: >> >>> >>> The fail-safe default before was ISO-8859-1. So I suggest to use it >>> again. >>> >> >> >> If stream converted output it s require please consider making it a >> configurable module in the code base that can be turned on and off and >> modified (the module) as needed. Leave the default as is, that way DESTRO’s >> and users can configure to their liking. But it will work as intended all >> the way back to the stone-age tools that we started this project with. (I’m >> looking at you Solaris 7/8) >> >> >> -Mike >> > > <0001-Add-outputencoding-parameter.patch><0002-Use-pager-on-stdout-output-to-terminal.patch>-- > Developer Access Program for Intel Xeon Phi Processors > Access to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. > http://sdm.link/xeonphi___ > Logwatch-devel mailing list > logwatch-de...@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/logwatch-devel
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem, new logwatch sends mails with charset UTF-8
Hi, in order to come closer to a fix for this issue, I propose the following two patches: 0001-Add-outputencoding-parameter.patch This patch allows to configure the value for the charset in the Content-Type line in mail output. This should address Klaus Ethgen's original concern. Since most people use UTF-8, I left the default at UTF-8. 0002-Use-pager-on-stdout-output-to-terminal.patch Use pager less if output is on terminal. This should address the issues associated with escape sequences in logs that may mess with your terminal. Less seems to be good at filtering these escape sequences. Let me know what you think about these patches. Willi Am 2017-01-01 um 23:01 schrieb Mike Tremaine: > >> >> The fail-safe default before was ISO-8859-1. So I suggest to use it >> again. >> > > > If stream converted output it s require please consider making it a > configurable module in the code base that can be turned on and off and > modified (the module) as needed. Leave the default as is, that way DESTRO’s > and users can configure to their liking. But it will work as intended all the > way back to the stone-age tools that we started this project with. (I’m > looking at you Solaris 7/8) > > > -Mike > >From 162f8b2e7fae134e65761dc64e0a8d29c9c172d4 Mon Sep 17 00:00:00 2001 From: Willi Mann Date: Sat, 14 Jan 2017 15:32:39 +0100 Subject: [PATCH 1/2] Add outputencoding parameter --- conf/logwatch.conf | 8 scripts/logwatch.pl | 8 +--- 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/conf/logwatch.conf b/conf/logwatch.conf index 2b61ec6..0d0e31b 100644 --- a/conf/logwatch.conf +++ b/conf/logwatch.conf @@ -126,4 +126,12 @@ mailer = "/usr/sbin/sendmail -t" # #HostLimit = myhost +# +# With this option, the output encoding for mail headers can be set. It +# defaults to utf-8. Be aware that the default value may change to autodetect +# (take the system's encoding). Note that the configured value will be +# upper-cased. +# +# OutputEncoding = utf-8 + # vi: shiftwidth=3 tabstop=3 et diff --git a/scripts/logwatch.pl b/scripts/logwatch.pl index 0167755..0cff511 100755 --- a/scripts/logwatch.pl +++ b/scripts/logwatch.pl @@ -90,6 +90,7 @@ $Config{'pathtobzcat'} = "bzcat"; $Config{'output'} = "stdout"; #8.0 $Config{'format'} = "text"; #8.0 $Config{'encode'} = "none"; #8.0 +$Config{'outputencoding'} = "utf-8"; $Config{'hostformat'} = "none"; #8.0 $Config{'html_wrap'} = 80; $Config{'supress_ignores'} = 0; @@ -1160,11 +1161,12 @@ sub initprint { } else { $out_mime .= "Content-Transfer-Encoding: 7bit\n"; } - #Config{output} html + #Config{output} html, Config{outputenconding} + my $encoding = uc($Config{outputencoding}); if ( $Config{'format'} eq "html" ) { -$out_mime .= "Content-Type: text/html; charset=\"UTF-8\"\n\n"; +$out_mime .= "Content-Type: text/html; charset=\"$encoding\"\n\n"; } else { -$out_mime .= "Content-Type: text/plain; charset=\"UTF-8\"\n\n"; +$out_mime .= "Content-Type: text/plain; charset=\"$encoding\"\n\n"; } if ($Config{'hostformat'} eq "split") { #8.0 check hostlimit also? or ne none? -- 2.1.4 >From 4e2579f41f1b94db97d265a72f67309cbf1737fa Mon Sep 17 00:00:00 2001 From: Willi Mann Date: Sat, 14 Jan 2017 16:25:30 +0100 Subject: [PATCH 2/2] Use pager on stdout output to terminal --- conf/logwatch.conf | 4 scripts/logwatch.pl | 7 ++- 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/conf/logwatch.conf b/conf/logwatch.conf index 0d0e31b..e15ed3b 100644 --- a/conf/logwatch.conf +++ b/conf/logwatch.conf @@ -134,4 +134,8 @@ mailer = "/usr/sbin/sendmail -t" # # OutputEncoding = utf-8 +# +# For output type stdout, use pager if the stdout is a terminal. +#Pager = /usr/bin/less + # vi: shiftwidth=3 tabstop=3 et diff --git a/scripts/logwatch.pl b/scripts/logwatch.pl index 0cff511..f053cb3 100755 --- a/scripts/logwatch.pl +++ b/scripts/logwatch.pl @@ -88,6 +88,7 @@ $Config{'pathtocat'} = "cat"; $Config{'pathtozcat'} = "zcat"; $Config{'pathtobzcat'} = "bzcat"; $Config{'output'} = "stdout"; #8.0 +$Config{'pager'} = "/usr/bin/less"; #8.0 $Config{'format'} = "text"; #8.0 $Config{'encode'} = "none"; #8.0 $Config{'outputencoding'} = "utf-8"; @@ -1127,7 +1128,11 @@ sub initprint { $OStitle = "Solaris" if ($OSname eq "SunOS" && $release >= 2); if ($Config{'output'} eq "stdout") { #8.0 start with others? - *OUTFILE = *STDOUT; + if($Config{'pager'} and -t STDOUT) { + open(OUTFILE,"|$Config{'pager'}") or die "could not open pager: $Config{'pager'} $!"; + } else { + *OUTFILE = *STDOUT; + } } elsif ($Config{'output'} eq "file") { open(OUTFILE,">>" . $Config{'filename'}) or die "Can't open output file: $Config{'filename'} $!\n"; } else { -- 2.1.4
Bug#849531: Possible security problem, new logwatch sends mails with charset UTF-8
On Dec/31, Willi Mann wrote: > I would like to get your input on bug #849531 [1]. > [...] > So my question is: Is it a security issue if a script sends e-mails > with encoding=utf-8, but potentially containing invalid utf-8 strings? > If yes, what would be the (minimum) requirements to address this > problem? Reading all the bug comments, I feel you have the issue pretty much in check. Invalid utf-8 input can indeed cause problems (the xterm example was mentioned), but we'll leave it to upstream to figure out exactly *how* this should be resolved. Cheers, --Seb
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem,new logwatch sends mails with charset UTF-8
> -Original Message- > From: 'Klaus Ethgen' [mailto:kl...@ethgen.de] > Sent: Sunday, January 01, 2017 14:42 > To: Jason Pyeron > Cc: 'Willi Mann'; 849...@bugs.debian.org; > logwatch-de...@lists.sourceforge.net > Subject: Re: Bug#849531: [Logwatch-devel] Bug#849531: > Possible security problem,new logwatch sends mails with charset UTF-8 > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > Am So den 1. Jan 2017 um 20:24 schrieb Jason Pyeron: > > Yes, 8-bit ASCII, here is a translation table. > > ASCII is 7 bit. > > All 8 bit are different encodings. Sorry, "extended" ascii (code page 437), been using it since 1984 and old naming habits die hard. > > > > Below 128 it is not a problem as UTF-8 is transparent in this > > > range. But > > > above 128 it _is_ a problem. > > > > No, only when not ENCODED properly. > > Well, exactly what I said. > > > UTF-8 is very, very, likely going to stay. > > I would wish that it would die as it is bad designed and have some bad > behaviour, technical wise. > > However, I am realistic enough to know that it will not be possible to > replace UTF-8 with better design's. It is like IPv6, there is > no Plan B. > > Just I will only use UTF-8 where it is cleary specified (like XML) or > where it is the only possible way to go. With everything else I stay > with latin1. But this is just my personal view. > > Regards >Klaus > - -- > Klaus Ethgen > http://www.ethgen.ch/ > pub 4096R/4E20AF1C 2011-05-16Klaus Ethgen > > Fingerprint: 85D4 CA42 952C 949B 1753 62B3 79D0 B06F 4E20 AF1C > -BEGIN PGP SIGNATURE- > Comment: Charset: ISO-8859-1 > > iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlhpW38ACgkQpnwKsYAZ > 9qzrIgwAlGkTGVpJEp1XKYNcpUF2hR7Ie6vlA3xFBfCacCDAjJx5gIrGLkX5d8Ay > rus3hblp/Gv+r+IpzUrRyk5gifBAfaDWVnuj1j8Qi9dLFyhnQaD0LX8/MBgV6wOn > A2/C+XedX1geoZxk7c/m6b59XaWAuN36JK8PJjpoWX/8MNqDoNYAr31gMrZkHwmw > IYZygbLom22qcSq4ZeowWY5ZUX9nqku0/9NNzBYtegB/YIWxbzxWNaOIPhgPouZ5 > Ejjd14NBap4ZT263Bc55L+tvCBX2BU1iFipmKa5pRslexF3UdscR1e20b0wWVTaP > HTkcNJggsNQVUpPYyxp6xUWkmaItZCMSSHC77sv4ur6jnRY8IolUlh+x68ZEayIn > yU5O/rjg3tMbNaZHTOZcNZNtOisNOMm19SOkwNCezKdods8BhAI0z9I7rnDaOumT > HefdprK0CTuylGKdAvBxJv6io3AKkNdTaZ32bFnfE2HtaX77ZNDTgXkmutE1og6t > UeF6fw/T > =Mr6M > -END PGP SIGNATURE- >
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem, new logwatch sends mails with charset UTF-8
> > The fail-safe default before was ISO-8859-1. So I suggest to use it > again. > If stream converted output it s require please consider making it a configurable module in the code base that can be turned on and off and modified (the module) as needed. Leave the default as is, that way DESTRO’s and users can configure to their liking. But it will work as intended all the way back to the stone-age tools that we started this project with. (I’m looking at you Solaris 7/8) -Mike
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem,new logwatch sends mails with charset UTF-8
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Am So den 1. Jan 2017 um 20:24 schrieb Jason Pyeron: > Yes, 8-bit ASCII, here is a translation table. ASCII is 7 bit. All 8 bit are different encodings. > > Below 128 it is not a problem as UTF-8 is transparent in this > > range. But > > above 128 it _is_ a problem. > > No, only when not ENCODED properly. Well, exactly what I said. > UTF-8 is very, very, likely going to stay. I would wish that it would die as it is bad designed and have some bad behaviour, technical wise. However, I am realistic enough to know that it will not be possible to replace UTF-8 with better design's. It is like IPv6, there is no Plan B. Just I will only use UTF-8 where it is cleary specified (like XML) or where it is the only possible way to go. With everything else I stay with latin1. But this is just my personal view. Regards Klaus - -- Klaus Ethgen http://www.ethgen.ch/ pub 4096R/4E20AF1C 2011-05-16Klaus Ethgen Fingerprint: 85D4 CA42 952C 949B 1753 62B3 79D0 B06F 4E20 AF1C -BEGIN PGP SIGNATURE- Comment: Charset: ISO-8859-1 iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlhpW38ACgkQpnwKsYAZ 9qzrIgwAlGkTGVpJEp1XKYNcpUF2hR7Ie6vlA3xFBfCacCDAjJx5gIrGLkX5d8Ay rus3hblp/Gv+r+IpzUrRyk5gifBAfaDWVnuj1j8Qi9dLFyhnQaD0LX8/MBgV6wOn A2/C+XedX1geoZxk7c/m6b59XaWAuN36JK8PJjpoWX/8MNqDoNYAr31gMrZkHwmw IYZygbLom22qcSq4ZeowWY5ZUX9nqku0/9NNzBYtegB/YIWxbzxWNaOIPhgPouZ5 Ejjd14NBap4ZT263Bc55L+tvCBX2BU1iFipmKa5pRslexF3UdscR1e20b0wWVTaP HTkcNJggsNQVUpPYyxp6xUWkmaItZCMSSHC77sv4ur6jnRY8IolUlh+x68ZEayIn yU5O/rjg3tMbNaZHTOZcNZNtOisNOMm19SOkwNCezKdods8BhAI0z9I7rnDaOumT HefdprK0CTuylGKdAvBxJv6io3AKkNdTaZ32bFnfE2HtaX77ZNDTgXkmutE1og6t UeF6fw/T =Mr6M -END PGP SIGNATURE-
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem,new logwatch sends mails with charset UTF-8
> -Original Message- > From: 'Klaus Ethgen' [mailto:kl...@ethgen.de] > Sent: Sunday, January 01, 2017 12:23 > To: Jason Pyeron > Cc: 'Willi Mann'; 849...@bugs.debian.org; > logwatch-de...@lists.sourceforge.net > Subject: Re: Bug#849531: [Logwatch-devel] Bug#849531: > Possible security problem,new logwatch sends mails with charset UTF-8 > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > Am So den 1. Jan 2017 um 17:38 schrieb Jason Pyeron: > > > What do you want to say with that? Your input is not in UTF-8. > > > > That is the point. The OP complaines about ASCII being sent > when labeld as UTF8, as such it created invalid UTF8 sequences. > > No, I complain about non UTF-8 being send as UTF-8. > > Namelly, bytes values 0-255. Yes, 8-bit ASCII, here is a translation table. ASCII|UTF-8 00 | 00 01 | 01 ... 7E | 7e 7F | 7f 80 | c2 80 81 | c2 81 ... BE | c2 be BF | c2 bf C0 | c3 80 C1 | c3 81 ... FE | c3 be FF | c3 bf > > Below 128 it is not a problem as UTF-8 is transparent in this > range. But > above 128 it _is_ a problem. No, only when not ENCODED properly. UTF-8 is very, very, likely going to stay. -Jason
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem,new logwatch sends mails with charset UTF-8
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Am So den 1. Jan 2017 um 17:38 schrieb Jason Pyeron: > > What do you want to say with that? Your input is not in UTF-8. > > That is the point. The OP complaines about ASCII being sent when labeld as > UTF8, as such it created invalid UTF8 sequences. No, I complain about non UTF-8 being send as UTF-8. Namelly, bytes values 0-255. Below 128 it is not a problem as UTF-8 is transparent in this range. But above 128 it _is_ a problem. Regards Klaus - -- Klaus Ethgen http://www.ethgen.ch/ pub 4096R/4E20AF1C 2011-05-16Klaus Ethgen Fingerprint: 85D4 CA42 952C 949B 1753 62B3 79D0 B06F 4E20 AF1C -BEGIN PGP SIGNATURE- Comment: Charset: ISO-8859-1 iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlhpOtAACgkQpnwKsYAZ 9qyfhQwAnpi6NbFfuS+i49Surxqsdqax3TNG7poe0I0/VOqYgIeUAzLVagI3Q3rz 7SWhHuVgRn+TdZoqJaOPoSwdKOuSh4H4U6YVa3Hke66tW5h+Yx+U6HZCCtpqs8L8 kqGzyif84XkpSNdZzqk1cfLgBeTljVL5L+5VpkOpAmrgLYLpn6e4NaPWzZl2ZUKh YBFUuIUnjO3dQysV6OAm94eYaZbqs+9nzWJhdMOxR3j33gfRyAIrto7VPu/pBgPK ErVhIpyytRe2ztvqM2is7NRv9hpPiMrXtdZNUtFltKIPQPPHrjm/sgYEFi/FFc7R 5aAgFL4XIAOcbDpw2lmLMNob3Kdzx8MWYx1aI2Ox/avlkFnTONQvDHDCLDfHOCeu CQhsAQSrzNARVy9VE7p7RFaPBeQ2iyOA1J/TGvUgf+VU7+gwR4uH3ASgfmfnJcnA nYNeIQCXeMt/GvW0yScVBr/EY6KOWWC3rEdeh/YhZ8nvRG8RJb188UjYdZ+LWnHa 3lkxpDZK =UtDa -END PGP SIGNATURE-
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem,new logwatch sends mails with charset UTF-8
> -Original Message- > From: Willi Mann [mailto:wi...@debian.org] > Sent: Sunday, January 01, 2017 08:27 > To: Jason Pyeron; 849...@bugs.debian.org > Cc: 'Klaus Ethgen'; logwatch-de...@lists.sourceforge.net > Subject: Re: Bug#849531: [Logwatch-devel] Bug#849531: > Possible security problem,new logwatch sends mails with charset UTF-8 > > Hi, > > Am 2017-01-01 um 00:20 schrieb Jason Pyeron: > > Not exactly a valid test, besides it works for me. The > issue is internal ascii data being written as ascii but > instructing consumers > > it is uft8. > > > > $ cat utf8_test.pl > > #!/usr/bin/perl > > # > > use strict; > > use File::Slurp; > > > > my $inputfile = @ARGV[0]; > > my $logfilecontent = read_file($inputfile); > > binmode(STDOUT, ":utf8"); > > print $logfilecontent; > > > > $ ./utf8_test.pl testlog.txt > > übersät > > > > $ ./utf8_test.pl testlog.txt | hexdump -C > > c3 bc 62 65 72 73 c3 a4 74 0a > |..bers..t.| > > 000a > > > > $ hexdump.exe -C testlog.txt > > fc 62 65 72 73 e4 74 0a > |.bers.t.| > > 0008 > > What do you want to say with that? Your input is not in UTF-8. That is the point. The OP complaines about ASCII being sent when labeld as UTF8, as such it created invalid UTF8 sequences. Quoting https://en.wikipedia.org/wiki/UTF-8#Codepage_layout > Invalid byte sequences[edit] > > Not all sequences of bytes are valid UTF-8. A UTF-8 decoder should be > prepared for: > * the red invalid bytes in the above table [192,103,245-255] > * an unexpected continuation byte > * a leading byte not followed by enough continuation bytes (can happen in > simple > string truncation, when a string is too long to fit when copying it) > * an overlong encoding as described above > * a sequence that decodes to an invalid code point as described below > > Many earlier decoders would happily try to decode these. Carefully crafted > invalid > UTF-8 could make them either skip or create ASCII characters such as NUL, > slash, > or quotes. Invalid UTF-8 has been used to bypass security validations in > high-profile > products including Microsoft's IIS web server[14] and Apache's Tomcat servlet > container.[15] > > > Just for the record, the output what I posted originally: We have differences > > % ./utf8_test.pl testlog > übersät > % ./utf8_test.pl testlog | hexdump -C > c3 83 c2 bc 62 65 72 73 c3 83 c2 a4 74 0a > |berst.| > 000e > % hexdump -C testlog > c3 bc 62 65 72 73 c3 a4 74 0a > |..bers..t.| > 000a $ hexdump.exe -C testlog.txt fc 62 65 72 73 e4 74 0a |.bers.t.| 0008 > > Bye > Willi >
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem,new logwatch sends mails with charset UTF-8
Hi, Am 2017-01-01 um 00:20 schrieb Jason Pyeron: > Not exactly a valid test, besides it works for me. The issue is internal > ascii data being written as ascii but instructing consumers > it is uft8. > > $ cat utf8_test.pl > #!/usr/bin/perl > # > use strict; > use File::Slurp; > > my $inputfile = @ARGV[0]; > my $logfilecontent = read_file($inputfile); > binmode(STDOUT, ":utf8"); > print $logfilecontent; > > $ ./utf8_test.pl testlog.txt > übersät > > $ ./utf8_test.pl testlog.txt | hexdump -C > c3 bc 62 65 72 73 c3 a4 74 0a|..bers..t.| > 000a > > $ hexdump.exe -C testlog.txt > fc 62 65 72 73 e4 74 0a |.bers.t.| > 0008 What do you want to say with that? Your input is not in UTF-8. Just for the record, the output what I posted originally: % ./utf8_test.pl testlog übersät % ./utf8_test.pl testlog | hexdump -C c3 83 c2 bc 62 65 72 73 c3 83 c2 a4 74 0a|berst.| 000e % hexdump -C testlog c3 bc 62 65 72 73 c3 a4 74 0a|..bers..t.| 000a Bye Willi
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem,new logwatch sends mails with charset UTF-8
> -Original Message- > From: Klaus Ethgen > Sent: Saturday, December 31, 2016 08:48 > To: Willi Mann > Cc: Jason Pyeron; 849...@bugs.debian.org; logwatch-de...@lists.sourceforge.net > > Hi, > > Am Sa den 31. Dez 2016 um 14:28 schrieb Willi Mann: > > thanks for your test cases. However, I don't think that > binmode provides > > an acceptable solution, at least not alone. While it > ensures that the > > strings are valid utf-8 strings, it will convert any valid utf-8 > > character to two "garbage" characters. Try Not exactly a valid test, besides it works for me. The issue is internal ascii data being written as ascii but instructing consumers it is uft8. $ cat utf8_test.pl #!/usr/bin/perl # use strict; use File::Slurp; my $inputfile = @ARGV[0]; my $logfilecontent = read_file($inputfile); binmode(STDOUT, ":utf8"); print $logfilecontent; $ ./utf8_test.pl testlog.txt übersät $ ./utf8_test.pl testlog.txt | hexdump -C c3 bc 62 65 72 73 c3 a4 74 0a|..bers..t.| 000a $ hexdump.exe -C testlog.txt fc 62 65 72 73 e4 74 0a |.bers.t.| 0008 > > Well, that "garbage" is by design for UTF-8. If you don't want that, > stay on latin1. > > It is a no-go to set the mime type to UTF-8 but still send latin1. (As > it does the current version.) Setting header to UTF-8 doesn't > change the > content of the mail. It just open up for troubles. > > Regards >Klaus > - -- > Klaus Ethgen > http://www.ethgen.ch/ > pub 4096R/4E20AF1C 2011-05-16Klaus Ethgen > > Fingerprint: 85D4 CA42 952C 949B 1753 62B3 79D0 B06F 4E20 AF1C > -BEGIN PGP SIGNATURE- > Comment: Charset: ISO-8859-1 > > iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlhntxMACgkQpnwKsYAZ > 9qyD4gv/ThmNQDCI9QeXYGvwafNDzcDtaHUpeGhOqJI4NjE/UxvPDGIJsMAmS3fI > w69zDuHmy9d1AsCm4I8ipF9l1LD1GHo8Fh9g2Uiv4l6d5e4jYmMi/L/pJxqbAqIt > A1LjNQUNGMLk97OHLqR5/9lnfOzahdzgEVNP/Fi5ygVXi3vJFdwfFFbWk39CfYUy > jcKQUdDzbQUzyFLl7I+1pZm19HCDH4v5fIzqwQW8bz4VXpTIUZjXJSV2n5gN1Lo9 > 99utKdR1b1UQScdGs2zV/QhVN/IJJsNNzK4Zylisdjw0ZgvnSW3gt461d62FAH1o > R4UwerUZYWzCGLZHpGwPw/1/s7YOAlPlO46UzSslqC0J0mmcCPG5eBz4iX2F03U3 > uoz3gscPsjFAf/eqlkp6MHXeNqSV2cCwQLnqZ17/py5DiMUxS61dFXRmcrLOotC0 > KmDBRC7Gft8dcr4bjqYG3jIv0ppOEdvA1izQQ+q2WNQ4E7AprDPJ94MgibQ8BBYX > iGbaxnj2 > =af5+ > -END PGP SIGNATURE- >
Bug#849531: Possible security problem, new logwatch sends mails with charset UTF-8
Dear Security Team, I would like to get your input on bug #849531 [1]. A short summary: Logwatch is a log summarizer that parses various logfiles and reports a summary, either via e-mail or to stdout. Parts of the input are copied verbatim w.r.t. to their encoding to the output (e.g., usernames, URLs, etc.) However, e-mails were sent with a hard-coded Content-Type: ... encoding=ISO-8859-1. This meant that non-ascii UTF-8 characters were not displayed correctly. As part of a recent change that is already in Debian testing/unstable, the Content-Type line was modified to say that the encoding is UTF-8, obviously to ensure that utf-8 characters are displayed correctly. However, logwatch does not ensure that the output is correct utf-8, and that is claimed to be a security problem. So my question is: Is it a security issue if a script sends e-mails with encoding=utf-8, but potentially containing invalid utf-8 strings? If yes, what would be the (minimum) requirements to address this problem? thank you for your time Willi [1] https://bugs.debian.org/849531
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem,new logwatch sends mails with charset UTF-8
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi, Am Sa den 31. Dez 2016 um 14:28 schrieb Willi Mann: > thanks for your test cases. However, I don't think that binmode provides > an acceptable solution, at least not alone. While it ensures that the > strings are valid utf-8 strings, it will convert any valid utf-8 > character to two "garbage" characters. Try Well, that "garbage" is by design for UTF-8. If you don't want that, stay on latin1. It is a no-go to set the mime type to UTF-8 but still send latin1. (As it does the current version.) Setting header to UTF-8 doesn't change the content of the mail. It just open up for troubles. Regards Klaus - -- Klaus Ethgen http://www.ethgen.ch/ pub 4096R/4E20AF1C 2011-05-16Klaus Ethgen Fingerprint: 85D4 CA42 952C 949B 1753 62B3 79D0 B06F 4E20 AF1C -BEGIN PGP SIGNATURE- Comment: Charset: ISO-8859-1 iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlhntxMACgkQpnwKsYAZ 9qyD4gv/ThmNQDCI9QeXYGvwafNDzcDtaHUpeGhOqJI4NjE/UxvPDGIJsMAmS3fI w69zDuHmy9d1AsCm4I8ipF9l1LD1GHo8Fh9g2Uiv4l6d5e4jYmMi/L/pJxqbAqIt A1LjNQUNGMLk97OHLqR5/9lnfOzahdzgEVNP/Fi5ygVXi3vJFdwfFFbWk39CfYUy jcKQUdDzbQUzyFLl7I+1pZm19HCDH4v5fIzqwQW8bz4VXpTIUZjXJSV2n5gN1Lo9 99utKdR1b1UQScdGs2zV/QhVN/IJJsNNzK4Zylisdjw0ZgvnSW3gt461d62FAH1o R4UwerUZYWzCGLZHpGwPw/1/s7YOAlPlO46UzSslqC0J0mmcCPG5eBz4iX2F03U3 uoz3gscPsjFAf/eqlkp6MHXeNqSV2cCwQLnqZ17/py5DiMUxS61dFXRmcrLOotC0 KmDBRC7Gft8dcr4bjqYG3jIv0ppOEdvA1izQQ+q2WNQ4E7AprDPJ94MgibQ8BBYX iGbaxnj2 =af5+ -END PGP SIGNATURE-
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem,new logwatch sends mails with charset UTF-8
Hi Jason, thanks for your test cases. However, I don't think that binmode provides an acceptable solution, at least not alone. While it ensures that the strings are valid utf-8 strings, it will convert any valid utf-8 character to two "garbage" characters. Try $ ./utf8_test.pl testlog (see attached files) I'm not really sure what a proper solution is. But I'm actually not yet fully convinced that there is a problem logwatch should solve. I will ask Debian's security team for advice. WM Am 2016-12-30 um 20:26 schrieb Jason Pyeron: > A very rudimentary test: > > /projects/logwatch > $ perl -e 'for ($i=0; $i<256; ++$i) {print chr($i);}' | hexdump.exe -C > 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f || > 0010 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f || > 0020 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f | !"#$%&'()*+,-./| > 0030 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f |0123456789:;<=>?| > 0040 40 41 42 43 44 45 46 47 48 49 4a 4b 4c 4d 4e 4f |@ABCDEFGHIJKLMNO| > 0050 50 51 52 53 54 55 56 57 58 59 5a 5b 5c 5d 5e 5f |PQRSTUVWXYZ[\]^_| > 0060 60 61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f |`abcdefghijklmno| > 0070 70 71 72 73 74 75 76 77 78 79 7a 7b 7c 7d 7e 7f |pqrstuvwxyz{|}~.| > 0080 80 81 82 83 84 85 86 87 88 89 8a 8b 8c 8d 8e 8f || > 0090 90 91 92 93 94 95 96 97 98 99 9a 9b 9c 9d 9e 9f || > 00a0 a0 a1 a2 a3 a4 a5 a6 a7 a8 a9 aa ab ac ad ae af || > 00b0 b0 b1 b2 b3 b4 b5 b6 b7 b8 b9 ba bb bc bd be bf || > 00c0 c0 c1 c2 c3 c4 c5 c6 c7 c8 c9 ca cb cc cd ce cf || > 00d0 d0 d1 d2 d3 d4 d5 d6 d7 d8 d9 da db dc dd de df || > 00e0 e0 e1 e2 e3 e4 e5 e6 e7 e8 e9 ea eb ec ed ee ef || > 00f0 f0 f1 f2 f3 f4 f5 f6 f7 f8 f9 fa fb fc fd fe ff || > 0100 > > /projects/logwatch > $ perl -e 'binmode(STDOUT, ":utf8"); for ($i=0; $i<256; ++$i) {print STDOUT > chr($i);}' | hexdump.exe -C > 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f || > 0010 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f || > 0020 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f | !"#$%&'()*+,-./| > 0030 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f |0123456789:;<=>?| > 0040 40 41 42 43 44 45 46 47 48 49 4a 4b 4c 4d 4e 4f |@ABCDEFGHIJKLMNO| > 0050 50 51 52 53 54 55 56 57 58 59 5a 5b 5c 5d 5e 5f |PQRSTUVWXYZ[\]^_| > 0060 60 61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f |`abcdefghijklmno| > 0070 70 71 72 73 74 75 76 77 78 79 7a 7b 7c 7d 7e 7f |pqrstuvwxyz{|}~.| > 0080 c2 80 c2 81 c2 82 c2 83 c2 84 c2 85 c2 86 c2 87 || > 0090 c2 88 c2 89 c2 8a c2 8b c2 8c c2 8d c2 8e c2 8f || > 00a0 c2 90 c2 91 c2 92 c2 93 c2 94 c2 95 c2 96 c2 97 || > 00b0 c2 98 c2 99 c2 9a c2 9b c2 9c c2 9d c2 9e c2 9f || > 00c0 c2 a0 c2 a1 c2 a2 c2 a3 c2 a4 c2 a5 c2 a6 c2 a7 || > 00d0 c2 a8 c2 a9 c2 aa c2 ab c2 ac c2 ad c2 ae c2 af || > 00e0 c2 b0 c2 b1 c2 b2 c2 b3 c2 b4 c2 b5 c2 b6 c2 b7 || > 00f0 c2 b8 c2 b9 c2 ba c2 bb c2 bc c2 bd c2 be c2 bf || > 0100 c3 80 c3 81 c3 82 c3 83 c3 84 c3 85 c3 86 c3 87 || > 0110 c3 88 c3 89 c3 8a c3 8b c3 8c c3 8d c3 8e c3 8f || > 0120 c3 90 c3 91 c3 92 c3 93 c3 94 c3 95 c3 96 c3 97 || > 0130 c3 98 c3 99 c3 9a c3 9b c3 9c c3 9d c3 9e c3 9f || > 0140 c3 a0 c3 a1 c3 a2 c3 a3 c3 a4 c3 a5 c3 a6 c3 a7 || > 0150 c3 a8 c3 a9 c3 aa c3 ab c3 ac c3 ad c3 ae c3 af || > 0160 c3 b0 c3 b1 c3 b2 c3 b3 c3 b4 c3 b5 c3 b6 c3 b7 || > 0170 c3 b8 c3 b9 c3 ba c3 bb c3 bc c3 bd c3 be c3 bf || > 0180 > > This confirms that binmode utf8 is needed to print out the full ASCII range. > >> -Original Message- >> From: Jason Pyeron [mailto:jpye...@pdinc.us] >> Sent: Friday, December 30, 2016 14:03 >> To: Jason Pyeron; 'Willi Mann'; logwatch-de...@lists.sourceforge.net >> Cc: 849...@bugs.debian.org; 849531-forwar...@bugs.debian.org; >> 'Klaus Ethgen' >> Subject: RE: [Logwatch-devel] Bug#849531: Possible security >> problem,new logwatch sends mails with charset UTF-8 >> >> I have opened https://sourceforge.net/p/logwatch/bugs/56/ . >> >> I am working a test
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem, new logwatch sends mails with charset UTF-8
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Am Fr den 30. Dez 2016 um 22:53 schrieb Jason Pyeron: > You would have the same issue with cat /var/log/x True. That is the reason I always tell the people not to use cat for that. (There is only little you should use cat for ever.) I seen many problems occure because an admin not listening. Regards Klaus - -- Klaus Ethgen http://www.ethgen.ch/ pub 4096R/4E20AF1C 2011-05-16Klaus Ethgen Fingerprint: 85D4 CA42 952C 949B 1753 62B3 79D0 B06F 4E20 AF1C -BEGIN PGP SIGNATURE- Comment: Charset: ISO-8859-1 iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlhm39wACgkQpnwKsYAZ 9qwDZAwAtKQHpIpI+l+rVG+/1TUs5xguzj3Yox7ZCc3irYbDiq5Q6qty95ZRiyv5 HYMJhTQjYjV7Spz7EBYllxfZ7ZvSxxKRmH7Zvbw3RGZFOGVRFUQ0uAJIylzh9x7U Ugh5LyhYhJsxeK5PRrWuKj6ARYIHivyaNEce/bQ6XzP2rK6mEZJJZhJJ8q1djei4 8cCCvfOBjjug1qQmbdwc8uGDr4fPz32gc1DhdeZY4HUSBXa9Ib7dl3XIP27vU28K FId+ZtkdE8ObHIuw/c8w8odB8k3D/20Q6axrCUAYZVmdXeqbXhL88Iz6K5Ugx9uo NxK+4xG2kez9J06s2RDtZzPwwS4Kaa5un8v5hItZg39zikVYFtLiNLJM+IIpSos0 tszgLrFRKmM+IFXkj3eZabE/dYU230AT62vxg6wqzNh3zNTJucPan2i3ad7vCKcs Sjy+O238p1CSjmZsI6A9iodMPnsbm3pTFUm9p7WM5x9MjgfBMMNPPVeASWBEdry0 ezxxPXPJ =Jl6P -END PGP SIGNATURE-
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem, new logwatch sends mails with charset UTF-8
> -Original Message- > From: Willi Mann > Sent: Friday, December 30, 2016 16:21 > To: Klaus Ethgen; 849...@bugs.debian.org > Cc: logwatch-de...@lists.sourceforge.net > Subject: Re: [Logwatch-devel] Bug#849531: Possible security > problem, new logwatch sends mails with charset UTF-8 > > Hi Klaus, > > Am 2016-12-30 um 18:36 schrieb Klaus Ethgen: > > Hi Willi, > > > > Am Fr den 30. Dez 2016 um 18:18 schrieb Willi Mann: > >> can you elaborate how this could be exploited? > > > > Well, log principally contains untrusted data that could be injected > > from untrusted source. That is no security hole itself. > > > > But when that data gets displayed with the wrong charset, that can > > trigger problems in window managers (for example). See > xterm which can > > be controlled via ansii sequences. Even more, it could > trigger stream > > conversion problems if the UTF-8 implementation is not really fully > > tested with broken streams. You would have the same issue with cat /var/log/x > > So far, I cannot see that the change you mentioned would be > problematic. Adding the binmode(OUTFILE, ":utf8"); fixes your primary report. > What I do see is that it might be wise to sanitize the output of > logwatch. A possible way to go might be to remove any byte > with value < > 0x20 - unless it is a newline or tab. But that is independent of the > ISO-8859-15 to utf-8 change. Please open a new bug for this enhancement, as it a different issue. -Jason
Bug#849531: Possible security problem, new logwatch sends mails with charset UTF-8
Hi Klaus, Am 2016-12-30 um 18:36 schrieb Klaus Ethgen: > Hi Willi, > > Am Fr den 30. Dez 2016 um 18:18 schrieb Willi Mann: >> can you elaborate how this could be exploited? > > Well, log principally contains untrusted data that could be injected > from untrusted source. That is no security hole itself. > > But when that data gets displayed with the wrong charset, that can > trigger problems in window managers (for example). See xterm which can > be controlled via ansii sequences. Even more, it could trigger stream > conversion problems if the UTF-8 implementation is not really fully > tested with broken streams. OK, I understand that text from untrusted sources is dangerous and could be harmful to your terminal - and that it is better to assume logfiles to be such an untrusted source. However, the change only affects mailers, right? For mailers, I would expect them to not trust any mails they get, and therefore to remove any dangerous byte sequences. Otherwise, the mailer contains the security hole - and an attacker does not need to go via logfiles and logwatch, he just sends you mail. When running logwatch such that it writes its output to the terminal, the change does not have any effect on the security, right?. If there is a security issue with untrusted data, it was already there before. So far, I cannot see that the change you mentioned would be problematic. What I do see is that it might be wise to sanitize the output of logwatch. A possible way to go might be to remove any byte with value < 0x20 - unless it is a newline or tab. But that is independent of the ISO-8859-15 to utf-8 change. Bye Willi
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem,new logwatch sends mails with charset UTF-8
A very rudimentary test: /projects/logwatch $ perl -e 'for ($i=0; $i<256; ++$i) {print chr($i);}' | hexdump.exe -C 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f || 0010 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f || 0020 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f | !"#$%&'()*+,-./| 0030 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f |0123456789:;<=>?| 0040 40 41 42 43 44 45 46 47 48 49 4a 4b 4c 4d 4e 4f |@ABCDEFGHIJKLMNO| 0050 50 51 52 53 54 55 56 57 58 59 5a 5b 5c 5d 5e 5f |PQRSTUVWXYZ[\]^_| 0060 60 61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f |`abcdefghijklmno| 0070 70 71 72 73 74 75 76 77 78 79 7a 7b 7c 7d 7e 7f |pqrstuvwxyz{|}~.| 0080 80 81 82 83 84 85 86 87 88 89 8a 8b 8c 8d 8e 8f || 0090 90 91 92 93 94 95 96 97 98 99 9a 9b 9c 9d 9e 9f || 00a0 a0 a1 a2 a3 a4 a5 a6 a7 a8 a9 aa ab ac ad ae af || 00b0 b0 b1 b2 b3 b4 b5 b6 b7 b8 b9 ba bb bc bd be bf || 00c0 c0 c1 c2 c3 c4 c5 c6 c7 c8 c9 ca cb cc cd ce cf || 00d0 d0 d1 d2 d3 d4 d5 d6 d7 d8 d9 da db dc dd de df || 00e0 e0 e1 e2 e3 e4 e5 e6 e7 e8 e9 ea eb ec ed ee ef || 00f0 f0 f1 f2 f3 f4 f5 f6 f7 f8 f9 fa fb fc fd fe ff || 0100 /projects/logwatch $ perl -e 'binmode(STDOUT, ":utf8"); for ($i=0; $i<256; ++$i) {print STDOUT chr($i);}' | hexdump.exe -C 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f || 0010 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f || 0020 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f | !"#$%&'()*+,-./| 0030 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f |0123456789:;<=>?| 0040 40 41 42 43 44 45 46 47 48 49 4a 4b 4c 4d 4e 4f |@ABCDEFGHIJKLMNO| 0050 50 51 52 53 54 55 56 57 58 59 5a 5b 5c 5d 5e 5f |PQRSTUVWXYZ[\]^_| 0060 60 61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f |`abcdefghijklmno| 0070 70 71 72 73 74 75 76 77 78 79 7a 7b 7c 7d 7e 7f |pqrstuvwxyz{|}~.| 0080 c2 80 c2 81 c2 82 c2 83 c2 84 c2 85 c2 86 c2 87 || 0090 c2 88 c2 89 c2 8a c2 8b c2 8c c2 8d c2 8e c2 8f || 00a0 c2 90 c2 91 c2 92 c2 93 c2 94 c2 95 c2 96 c2 97 || 00b0 c2 98 c2 99 c2 9a c2 9b c2 9c c2 9d c2 9e c2 9f || 00c0 c2 a0 c2 a1 c2 a2 c2 a3 c2 a4 c2 a5 c2 a6 c2 a7 || 00d0 c2 a8 c2 a9 c2 aa c2 ab c2 ac c2 ad c2 ae c2 af || 00e0 c2 b0 c2 b1 c2 b2 c2 b3 c2 b4 c2 b5 c2 b6 c2 b7 || 00f0 c2 b8 c2 b9 c2 ba c2 bb c2 bc c2 bd c2 be c2 bf || 0100 c3 80 c3 81 c3 82 c3 83 c3 84 c3 85 c3 86 c3 87 || 0110 c3 88 c3 89 c3 8a c3 8b c3 8c c3 8d c3 8e c3 8f || 0120 c3 90 c3 91 c3 92 c3 93 c3 94 c3 95 c3 96 c3 97 || 0130 c3 98 c3 99 c3 9a c3 9b c3 9c c3 9d c3 9e c3 9f || 0140 c3 a0 c3 a1 c3 a2 c3 a3 c3 a4 c3 a5 c3 a6 c3 a7 || 0150 c3 a8 c3 a9 c3 aa c3 ab c3 ac c3 ad c3 ae c3 af || 0160 c3 b0 c3 b1 c3 b2 c3 b3 c3 b4 c3 b5 c3 b6 c3 b7 || 0170 c3 b8 c3 b9 c3 ba c3 bb c3 bc c3 bd c3 be c3 bf || 0180 This confirms that binmode utf8 is needed to print out the full ASCII range. > -Original Message- > From: Jason Pyeron [mailto:jpye...@pdinc.us] > Sent: Friday, December 30, 2016 14:03 > To: Jason Pyeron; 'Willi Mann'; logwatch-de...@lists.sourceforge.net > Cc: 849...@bugs.debian.org; 849531-forwar...@bugs.debian.org; > 'Klaus Ethgen' > Subject: RE: [Logwatch-devel] Bug#849531: Possible security > problem,new logwatch sends mails with charset UTF-8 > > I have opened https://sourceforge.net/p/logwatch/bugs/56/ . > > I am working a test case for this right now. > > As I see it, there are 3 paths to test. > > Output as STDOUT, file, and email. In each case does an 8bit > value (0x00..0xff unsigned) result in a valid UTF-8 character. > > Is binmode(STDOUT, ":utf8") needed? Does it fix the issue if > it was needed? > > > > -Original Message- > > > From: Willi Mann > > > Sent: Friday, December 30, 2016 12:18 > > > To: logwatch-devel > > > Cc: 849...@bugs.debian.org; > 849531-forwar...@bugs.debian.org; Klaus Ethgen > > > What would be your suggested fix? > > > $ git show f9db5949c58321175bda66310156f43ae607109f > commit f9db5949c58321175bda66310156f43ae607109f > Author: bjorn > Date: Sat Oct 15 17:38:40 2016 -0700 > > Changed encoding to UTF-8, as suggested by
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem,new logwatch sends mails with charset UTF-8
I have opened https://sourceforge.net/p/logwatch/bugs/56/ . I am working a test case for this right now. As I see it, there are 3 paths to test. Output as STDOUT, file, and email. In each case does an 8bit value (0x00..0xff unsigned) result in a valid UTF-8 character. Is binmode(STDOUT, ":utf8") needed? Does it fix the issue if it was needed? > > -Original Message- > > From: Willi Mann > > Sent: Friday, December 30, 2016 12:18 > > To: logwatch-devel > > Cc: 849...@bugs.debian.org; 849531-forwar...@bugs.debian.org; Klaus Ethgen > > What would be your suggested fix? $ git show f9db5949c58321175bda66310156f43ae607109f commit f9db5949c58321175bda66310156f43ae607109f Author: bjorn Date: Sat Oct 15 17:38:40 2016 -0700 Changed encoding to UTF-8, as suggested by Goran Uddeborg. diff --git a/scripts/logwatch.pl b/scripts/logwatch.pl index 0f863dc..0167755 100755 --- a/scripts/logwatch.pl +++ b/scripts/logwatch.pl @@ -1162,9 +1162,9 @@ sub initprint { } #Config{output} html if ( $Config{'format'} eq "html" ) { -$out_mime .= "Content-Type: text/html; charset=\"iso-8859-1\"\n\n"; +$out_mime .= "Content-Type: text/html; charset=\"UTF-8\"\n\n"; } else { -$out_mime .= "Content-Type: text/plain; charset=\"iso-8859-1\"\n\n"; +$out_mime .= "Content-Type: text/plain; charset=\"UTF-8\"\n\n"; } if ($Config{'hostformat'} eq "split") { #8.0 check hostlimit also? or ne none?
Bug#849531: [Logwatch-devel] Bug#849531: Possible security problem, new logwatch sends mails with charset UTF-8
> -Original Message- > From: Willi Mann [mailto:wi...@debian.org] > Sent: Friday, December 30, 2016 12:18 > To: logwatch-de...@lists.sourceforge.net > Cc: 849...@bugs.debian.org; 849531-forwar...@bugs.debian.org; > Klaus Ethgen > Subject: Re: [Logwatch-devel] Bug#849531: Possible security > problem, new logwatch sends mails with charset UTF-8 > > Hi Klaus, > > can you elaborate how this could be exploited? It does not make the list at http://unicode.org/reports/tr36/ , but bad utf-8 **may** cause email programs to behave badly. A google for unicode crash brings up the iPhone message processing issues... > What would be your suggested fix? Not sure it is a high risk issue, but perl should use binmode(STDOUT, ":utf8"); to encode the perl strings as utf-8. > > I'm including the upstream mailing list in the conversation. > > thanks you > Willi > > Am 2016-12-28 um 10:09 schrieb Klaus Ethgen: > > Package: logwatch > > Version: 7.4.3+git20161207-1 > > Severity: critical > > > > Current logwatch did change from sending mails with charset iso-8859-1 > > to UTF-8. This openes up a potential security hole as UTF-8 is not able To give context... commit f9db5949c58321175bda66310156f43ae607109f Author: bjorn Date: Sat Oct 15 17:38:40 2016 -0700 Changed encoding to UTF-8, as suggested by Goran Uddeborg. diff --git a/scripts/logwatch.pl b/scripts/logwatch.pl index 0f863dc..0167755 100755 --- a/scripts/logwatch.pl +++ b/scripts/logwatch.pl @@ -1162,9 +1162,9 @@ sub initprint { } #Config{output} html if ( $Config{'format'} eq "html" ) { -$out_mime .= "Content-Type: text/html; charset=\"iso-8859-1\"\n\n"; +$out_mime .= "Content-Type: text/html; charset=\"UTF-8\"\n\n"; } else { -$out_mime .= "Content-Type: text/plain; charset=\"iso-8859-1\"\n\n"; +$out_mime .= "Content-Type: text/plain; charset=\"UTF-8\"\n\n"; } if ($Config{'hostformat'} eq "split") { #8.0 check hostlimit also? or ne none? > > to display all 8bit data. > > > > This is especially true as the output from logwatch is from > untrusted > > source where there could easily put some malicious content > in. Logwatch > > does nothing to cleanup the mail content or convert it from > the native > > charset to UTF-8. > > > > Note that this bug went in recently as 7.4.0 did not have this bug > > (neither does 7.4.1). I do not find any upstream changelog in the > > package and when I download it from upstream directly, I > cannot find any > > note of this breaking change. > > > > -- System Information: > > Debian Release: stretch/sid > > APT prefers unstable > > APT policy: (500, 'unstable'), (500, 'testing'), (1, > 'experimental') > > Architecture: amd64 (x86_64) > > Foreign Architectures: i386 > > > > Kernel: Linux 4.7.10 (SMP w/8 CPU cores) > > Locale: LANG=de_DE, LC_CTYPE=de_DE (charmap=ISO-8859-1) > > Shell: /bin/sh linked to /bin/dash > > Init: sysvinit (via /sbin/init) > > > > Versions of packages logwatch depends on: > > ii exim4-daemon-light [mail-transport-agent] 4.88~RC6-2 > > pn perl:any > > > > Versions of packages logwatch recommends: > > ii libdate-manip-perl 6.56-1 > > ii libsys-cpu-perl 0.61-2+b1 > > pn libsys-meminfo-perl > > > > Versions of packages logwatch suggests: > > ii fortune-mod 1:1.99.1-7 > > > > -- no debconf information > > > > > > > -- > > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > ___ > Logwatch-devel mailing list > logwatch-de...@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/logwatch-devel >
Bug#849531: Possible security problem, new logwatch sends mails with charset UTF-8
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi Willi, Am Fr den 30. Dez 2016 um 18:18 schrieb Willi Mann: > can you elaborate how this could be exploited? Well, log principally contains untrusted data that could be injected from untrusted source. That is no security hole itself. But when that data gets displayed with the wrong charset, that can trigger problems in window managers (for example). See xterm which can be controlled via ansii sequences. Even more, it could trigger stream conversion problems if the UTF-8 implementation is not really fully tested with broken streams. > What would be your suggested fix? Send the data with a char set that cover the full byte, not only a part of it like UTF-8 or convert it somehow to UTF-8 what would be impossible as you don't know the source char set. The fail-safe default before was ISO-8859-1. So I suggest to use it again. Regards Klaus - -- Klaus Ethgen http://www.ethgen.ch/ pub 4096R/4E20AF1C 2011-05-16Klaus Ethgen Fingerprint: 85D4 CA42 952C 949B 1753 62B3 79D0 B06F 4E20 AF1C -BEGIN PGP SIGNATURE- Comment: Charset: ISO-8859-1 iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlhmmyAACgkQpnwKsYAZ 9qzhKQv9Gecm9WzvdohmcSwFTX9mMpsSN34r1zcMbWgMEYw2PRIYtBBsPvf8gZ5q vb1qWmipWoYE4HGPsNftm5+VhdkIdvU5fyB2UIcSAtOgjNqY7pTpe3yC3o0DHFcq 7SFuBerH8peJM6HvuPXzsRQVxk0/YOBG+OvprKyi6fWYbDN3wC6xfv1gB4BAI0b7 2kBmBxgV7RhN1qgttbGOXmDCvxvlVydArmotmaWG7DUqfFle/T9Y9FOOFBvdEy+z JGdQSNX6TNryiVXAnAJJepa7GCLV0/1CuSV0327WV5vBJXAJCqcW4zbI2+MZp79A 2WCoOzy8JdeAxacK2XV2xynVzvqfIrw41E9QFMdoo/944zR6S3VqwZh4PF97FJW+ tDt+vAnYKglOzufjbFxKojjffba357TumQ3oH1+4JAAKZIKeeJXa2iVt8stD1eBN O34FmWY5qffn8oeP3AbB54LHJBoNmzpJErZqEZYv/1gZi3pSu0Ie4BrMu0IFQr3y cOLr8pkg =pzp3 -END PGP SIGNATURE-
Bug#849531: Possible security problem, new logwatch sends mails with charset UTF-8
Hi Klaus, can you elaborate how this could be exploited? What would be your suggested fix? I'm including the upstream mailing list in the conversation. thanks you Willi Am 2016-12-28 um 10:09 schrieb Klaus Ethgen: > Package: logwatch > Version: 7.4.3+git20161207-1 > Severity: critical > > Current logwatch did change from sending mails with charset iso-8859-1 > to UTF-8. This openes up a potential security hole as UTF-8 is not able > to display all 8bit data. > > This is especially true as the output from logwatch is from untrusted > source where there could easily put some malicious content in. Logwatch > does nothing to cleanup the mail content or convert it from the native > charset to UTF-8. > > Note that this bug went in recently as 7.4.0 did not have this bug > (neither does 7.4.1). I do not find any upstream changelog in the > package and when I download it from upstream directly, I cannot find any > note of this breaking change. > > -- System Information: > Debian Release: stretch/sid > APT prefers unstable > APT policy: (500, 'unstable'), (500, 'testing'), (1, 'experimental') > Architecture: amd64 (x86_64) > Foreign Architectures: i386 > > Kernel: Linux 4.7.10 (SMP w/8 CPU cores) > Locale: LANG=de_DE, LC_CTYPE=de_DE (charmap=ISO-8859-1) > Shell: /bin/sh linked to /bin/dash > Init: sysvinit (via /sbin/init) > > Versions of packages logwatch depends on: > ii exim4-daemon-light [mail-transport-agent] 4.88~RC6-2 > pn perl:any > > Versions of packages logwatch recommends: > ii libdate-manip-perl 6.56-1 > ii libsys-cpu-perl 0.61-2+b1 > pn libsys-meminfo-perl > > Versions of packages logwatch suggests: > ii fortune-mod 1:1.99.1-7 > > -- no debconf information > >
Bug#849531: Possible security problem, new logwatch sends mails with charset UTF-8
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Package: logwatch Version: 7.4.3+git20161207-1 Severity: critical Current logwatch did change from sending mails with charset iso-8859-1 to UTF-8. This openes up a potential security hole as UTF-8 is not able to display all 8bit data. This is especially true as the output from logwatch is from untrusted source where there could easily put some malicious content in. Logwatch does nothing to cleanup the mail content or convert it from the native charset to UTF-8. Note that this bug went in recently as 7.4.0 did not have this bug (neither does 7.4.1). I do not find any upstream changelog in the package and when I download it from upstream directly, I cannot find any note of this breaking change. - -- System Information: Debian Release: stretch/sid APT prefers unstable APT policy: (500, 'unstable'), (500, 'testing'), (1, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 4.7.10 (SMP w/8 CPU cores) Locale: LANG=de_DE, LC_CTYPE=de_DE (charmap=ISO-8859-1) Shell: /bin/sh linked to /bin/dash Init: sysvinit (via /sbin/init) Versions of packages logwatch depends on: ii exim4-daemon-light [mail-transport-agent] 4.88~RC6-2 pn perl:any Versions of packages logwatch recommends: ii libdate-manip-perl 6.56-1 ii libsys-cpu-perl 0.61-2+b1 pn libsys-meminfo-perl Versions of packages logwatch suggests: ii fortune-mod 1:1.99.1-7 - -- no debconf information - -- Klaus Ethgen http://www.ethgen.ch/ pub 4096R/4E20AF1C 2011-05-16Klaus Ethgen Fingerprint: 85D4 CA42 952C 949B 1753 62B3 79D0 B06F 4E20 AF1C -BEGIN PGP SIGNATURE- Comment: Charset: ISO-8859-1 iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlhjgU0ACgkQpnwKsYAZ 9qzmVgwApEE7ee5wf0J7W3ibcGSGiPE0WHRDdrYhuE4Bew7uIlefvj1vil2RgbzN nm4SSn0CyCfSnvbWZA1SROaGWVApJLP7TOJRn3KioJm6N29SqXwJbGq6XD1HRNea woBsTGugHFoBOjVbpMe72dO2batal1xl2e8wQKKHuqSkkeGwAgl0oA7OgKgZ51gi 9A9fZaNfsWekMJzlGd8m3bmPQp32qRywxtkAQ6t+DEwABgdvPv05HB42CXBpbzrh QrXm6a64v/GPSs2uq4+Fjpi9/uXSExUTSqj/M2pJ14u10rD3n9Yghmkwc2290CIJ xHYQgdCm2EMpPRyb9pcJknIzE43oQkdNCTcqMyw62FO6hKKX3j0/b9md9AfH/tZn xbEkjd8HSyCY158QTPNHEro7klxoznjCLTj1dLaZH3HWTYpovpoBbJ9ecABaj4YJ tphX/wy46GL35PLJUnDcGgEgNavsbPpt/jiBYy2Q/FCPEg5DTJAXIh6RDNrCHsoY oH/vHcPf =Zlgb -END PGP SIGNATURE-