Re: [exim] "allow_insecure_tainted_data = yes" - was: tainted data issues

2021-05-08 Thread Heiko Schlittermann via Exim-users 
Chris Edwards via Exim-users  (Sa 08 Mai 2021 13:15:45 
CEST):
> On Tue, 6 Apr 2021, Heiko Schlittermann via Exim-users wrote:
> 
> > Currently I'm running this on a production systems without any issues so
> > far. You're invited to do tests in your systems too.
> 
> Trying this version, with allow_insecure_tainted_data set, then this:
> 
>   testlist:
> driver = redirect
> data = :include:/some/where/${local_part}
> 
> fails with error:
> 
>  LOG: MAIN PANIC DIE
>   Taint mismatch, Ustrncpy: parse_forward_list 1393
> 
> It looks like the :include: might be the issue.
> 
> Not a problem here as I've now detainted this, but thought to report back.

Thanks, I'll try to reproduce it, and fix it.

-- 
Heiko


signature.asc
Description: PGP signature
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] "allow_insecure_tainted_data = yes" - was: tainted data issues

2021-05-08 Thread Chris Edwards via Exim-users 

On Tue, 6 Apr 2021, Heiko Schlittermann via Exim-users wrote:


"ALLOW_INSECURE_TAINTED_DATA", currently enabled. Using this build time
option provides a new runtime option "allow_insecure_tainted_data", which
turns taint errors into warnings (and spams your log file).


[...]


Currently I'm running this on a production systems without any issues so
far. You're invited to do tests in your systems too.


Trying this version, with allow_insecure_tainted_data set, then this:

  testlist:
driver = redirect
data = :include:/some/where/${local_part}

fails with error:

 LOG: MAIN PANIC DIE
  Taint mismatch, Ustrncpy: parse_forward_list 1393

It looks like the :include: might be the issue.

Not a problem here as I've now detainted this, but thought to report back.

Cheers

Chris

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] "allow_insecure_tainted_data = yes" - was: tainted data issues

2021-04-25 Thread Heiko Schlittermann via Exim-users 
Thank you for spending your time :)

Andreas Metzler via Exim-users  (So 25 Apr 2021 08:12:58 
CEST):
>  void
> -openlogs();
> +open_logs(const char *m);
> is the proper fix?

It is one possible fix. But the char* isn't used anymore (was there for
debugging). I updated the branch.

> log.c: In function 'set_file_path':
> log.c:654:45: warning: pointer type mismatch in conditional expression
>   654 | uschar *ss = *log_file_path ? log_file_path : LOG_FILE_PATH;

Same here. Fixed.

> In file included from exim.h:486,
>  from log.c:13:
> log.c:657:31: warning: passing argument 1 of 'string_nextinlist_trc' from 
> incompatible pointer type [-Wincompatible-pointer-types]
>   657 | while ((s = string_nextinlist(, , log_buffer, 
> LOG_BUFFER_SIZE)))
> functions.h:560:25: note: in definition of macro 'string_nextinlist'
>   560 |  string_nextinlist_trc((lp), (sp), (b), (l), US __FUNCTION__, 
> __LINE__)

ditto.
And finally I set my compiler options to be about the same as yours.

-- 
Heiko


signature.asc
Description: PGP signature
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] "allow_insecure_tainted_data = yes" - was: tainted data issues

2021-04-25 Thread Andreas Metzler via Exim-users 
On 2021-04-25 Andreas Metzler  wrote:
> On 2021-04-24 Heiko Schlittermann  wrote:
> > I believe, the issue is fixed now. I'd be happy, if you **or anybody
> > else** can give it a try. To avoid cluttering the official Exim repo,
> > this branch is still only in my private but public repositories:
> [...]

> Good morning Heiko,

> thank you. Will upload to Debian/experimental.
[...]

Hello,

I forgot to confirm that the updated patchset fixes the error I had
reported. ;-)

cu Andreas

-- 
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'

-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] "allow_insecure_tainted_data = yes" - was: tainted data issues

2021-04-25 Thread Andreas Metzler via Exim-users 
On 2021-04-24 Heiko Schlittermann  wrote:
> I believe, the issue is fixed now. I'd be happy, if you **or anybody
> else** can give it a try. To avoid cluttering the official Exim repo,
> this branch is still only in my private but public repositories:
[...]

Good morning Heiko,

thank you. Will upload to Debian/experimental.

Compiler throws two new warnings:

appendfile.c: In function 'appendfile_transport_setup':
appendfile.c:238:1: warning: implicit declaration of function 'open_logs'; did 
you mean 'openlogs'? [-Wimplicit-function-declaration]
  238 | open_logs("appendfile");
  | ^
  | openlogs

I guess

 void
-openlogs();
+open_logs(const char *m);

is the proper fix?


log.c: In function 'set_file_path':
log.c:654:45: warning: pointer type mismatch in conditional expression
  654 | uschar *ss = *log_file_path ? log_file_path : LOG_FILE_PATH;
  | ^
In file included from exim.h:486,
 from log.c:13:
log.c:657:31: warning: passing argument 1 of 'string_nextinlist_trc' from 
incompatible pointer type [-Wincompatible-pointer-types]
  657 | while ((s = string_nextinlist(, , log_buffer, LOG_BUFFER_SIZE)))
functions.h:560:25: note: in definition of macro 'string_nextinlist'
  560 |  string_nextinlist_trc((lp), (sp), (b), (l), US __FUNCTION__, __LINE__)
  | ^~
functions.h:561:53: note: expected 'const uschar **' {aka 'const unsigned char 
**'} but argument is of type 'uschar **' {aka 'unsigned char **'}
  561 | extern uschar *string_nextinlist_trc(const uschar **listptr, int 
*separator, uschar *buffer, int buflen,
  |  ~~~^~~

cu Andreas

-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] "allow_insecure_tainted_data = yes" - was: tainted data issues

2021-04-24 Thread Heiko Schlittermann via Exim-users 
Hi Andreas,

I believe, the issue is fixed now. I'd be happy, if you **or anybody
else** can give it a try. To avoid cluttering the official Exim repo,
this branch is still only in my private but public repositories:


https://git.exim.org/users/heiko/exim.git/shortlog/refs/heads/exim-4.94+fixes+taintwarn

https://gitea.schlittermann.de/heiko/exim/src/branch/exim-4.94+fixes+taintwarn

Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
--
 SCHLITTERMANN.de  internet & unix support -
 Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
 gnupg encrypted messages are welcome --- key ID: F69376CE -


signature.asc
Description: PGP signature
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] "allow_insecure_tainted_data = yes" - was: tainted data issues

2021-04-12 Thread Heiko Schlittermann via Exim-users 
Hi Andreas,

the problem isn't caused by the new allow_insecure_tainted_data, but
these warnings trigger the issue.

We're in progress fixing it.

-- 
Heiko


signature.asc
Description: PGP signature
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] "allow_insecure_tainted_data = yes" - was: tainted data issues

2021-04-11 Thread Heiko Schlittermann via Exim-users 
Heiko Schlittermann via Exim-users  (So 11 Apr 2021 
09:08:10 CEST):
> Hi Andreas,
> 
> which commit ID your build is based on? I'd like to reproduce it
> locally.

I can reproduce it using a minimal config, going to check it now.
(The version I'm running on production systems doesn't do local
delivery.)

allow_insecure_tainted_data = yes

log_selector = +pid
acl_smtp_rcpt = accept

begin routers

accept:
driver = accept
check_local_user
transport = local

begin transports

local:
driver = appendfile
group = mail
file = /opt/exim/spool/mail/$local_part

-- 
Heiko


signature.asc
Description: PGP signature
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] "allow_insecure_tainted_data = yes" - was: tainted data issues

2021-04-11 Thread Heiko Schlittermann via Exim-users 
Hi Andreas,

which commit ID your build is based on? I'd like to reproduce it
locally.

Andreas Metzler via Exim-users  (So 11 Apr 2021 08:51:48 
CEST):
> On 2021-04-06 Heiko Schlittermann via Exim-users  wrote:
> [...]
> > .ifdef _OPT_MAIN_ALLOW_INSECURE_TAINTED_DATA
> > allow_insecure_tainted_data = yes
> > .endif
> 
> Hello,
> 
> I just did a test build on the fixes branch, added the
> allow_insecure_tainted_data setting and changed the mail_spool
> transport:
> - file = /var/mail/$local_part_data
> + file = /var/mail/$local_part
> 
> Success was limited though. Without the patch the message delivery is
> deferred. With the patch the message is frozen for
> "allow_insecure_tainted_data = yes" (log file excerpt below).
> 
> ==> /var/log/exim4/mainlog <==
> 2021-04-11 08:26:08 1lVTXs-000F7W-0D <= ametz...@bebt.de H=localhost 
> (argenau.bebt.de) [::1] P=esmtp S=476 id=20210411082607.058...@argenau.bebt.de
> 2021-04-11 08:26:08 1lVTXs-000F7W-0D failed to read delivery status for 
> ametzler@localhost from delivery subprocess
> 
> Debug log:
…
> 08:26:08 58130 ╰──(tainted)
> 08:26:08 58130 LOG: MAIN
> 08:26:08 58130   Warning: Tainted '/var/mail/ametzler' (file or directory 
> name for mail_spool transport) not permitted
> 2021-04-11 08:26:08 1lVTXs-000F7W-0D Warning: Tainted '/var/mail/ametzler' 
> (file or directory name for mail_spool transport) not permitted
…
> 08:26:08 58130 lock name: /var/mail/ametzler.lock
> 08:26:08 58130 hitch name: 
> /var/mail/ametzler.lock.argenau.bebt.de.60729680.e312
> 08:26:08 58130 LOG: MAIN
> 08:26:08 58130   Warning: Tainted filename 
> '/var/mail/ametzler.lock.argenau.bebt.de.60729680.e312'

> 08:26:08 58128 LOG: MAIN PANIC
> 08:26:08 58128   failed to read delivery status for ametzler@localhost from 
> delivery subprocess

Is there any indication that the child (delivery process) crashed?

> BTW the build-log with patch is very noisy:
> ---
> cc -c -g -O2 -ffile-prefix-map=/dev/shm/EXIM4/exim-4.94=. 
> -fstack-protector-strong -Wformat -Werror=format-security  
> -D_LARGEFILE_SOURCE -fno-strict-aliasing -Wall -Wdate-time 
> -D_FORTIFY_SOURCE=2 -fvisibility=hidden  -DCOMPILE_UTILITY -o util-spool_in.o 
> spool_in.c
> In file included from exim.h:486,

I'll check that noise. Thx.

-- 
Heiko


signature.asc
Description: PGP signature
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] "allow_insecure_tainted_data = yes" - was: tainted data issues

2021-04-11 Thread Andreas Metzler via Exim-users 
On 2021-04-06 Heiko Schlittermann via Exim-users  wrote:
[...]
> .ifdef _OPT_MAIN_ALLOW_INSECURE_TAINTED_DATA
> allow_insecure_tainted_data = yes
> .endif

Hello,

I just did a test build on the fixes branch, added the
allow_insecure_tainted_data setting and changed the mail_spool
transport:
- file = /var/mail/$local_part_data
+ file = /var/mail/$local_part

Success was limited though. Without the patch the message delivery is
deferred. With the patch the message is frozen for
"allow_insecure_tainted_data = yes" (log file excerpt below).

==> /var/log/exim4/mainlog <==
2021-04-11 08:26:08 1lVTXs-000F7W-0D <= ametz...@bebt.de H=localhost 
(argenau.bebt.de) [::1] P=esmtp S=476 id=20210411082607.058...@argenau.bebt.de
2021-04-11 08:26:08 1lVTXs-000F7W-0D failed to read delivery status for 
ametzler@localhost from delivery subprocess

Debug log:
08:26:08 58128 R: local_user for ametzler@localhost
08:26:08 58128 calling local_user router
08:26:08 58128 local_user router called for ametzler@localhost
08:26:08 58128   domain = localhost
08:26:08 58128 set transport mail_spool
08:26:08 58128 queued for mail_spool transport: local_part = ametzler
08:26:08 58128 domain = localhost
08:26:08 58128   errors_to=NULL
08:26:08 58128   domain_data=localhost local_part_data=ametzler
08:26:08 58128 routed by local_user router
08:26:08 58128   envelope to: ametzler@localhost
08:26:08 58128   transport: mail_spool
08:26:08 58128 >>
08:26:08 58128 After routing:
08:26:08 58128   Local deliveries:
08:26:08 58128 ametzler@localhost
08:26:08 58128   Remote deliveries:
08:26:08 58128   Failed addresses:
08:26:08 58128   Deferred addresses:
08:26:08 58128 search_tidyup called
08:26:08 58128  Local deliveries 
08:26:08 58128 > ametzler@localhost <
08:26:08 58128  locking /var/spool/exim4/db/retry.lockfile
08:26:08 58128  locked  /var/spool/exim4/db/retry.lockfile
08:26:08 58128  EXIM_DBOPEN: file  dir 
 flags=O_RDONLY
08:26:08 58128  returned from EXIM_DBOPEN: 0x55693f0b8380
08:26:08 58128  opened hints database /var/spool/exim4/db/retry: flags=O_RDONLY
08:26:08 58128  dbfn_read: key=T:ametzler@localhost
08:26:08 58128 retry record exists: age=5m11s (max 1w)
08:26:08 58128   time to retry = 9m49s expired = 0
08:26:08 58128  EXIM_DBCLOSE(0x55693f0b8380)
08:26:08 58128  closed hints database and lockfile
08:26:08 58128 search_tidyup called
08:26:08 58128 daemon-accept-delivery forking for delivery-local
08:26:08 58128 daemon-accept-delivery forked for delivery-local: 58130
08:26:08 58130 postfork: delivery-local
08:26:08 58130 changed uid/gid: local delivery to ametzler  
transport=mail_spool
08:26:08 58130   uid=1001 gid=8 pid=58130
08:26:08 58130   auxiliary group list: 
08:26:08 58130   home=/home/ametzler current=/home/ametzler
08:26:08 58130 set_process_info: 58130 delivering 1lVTXs-000F7W-0D to ametzler 
using mail_spool
08:26:08 58130  ╭considering: T: appendfile for $local_part@$domain
08:26:08 58130  ├──expanding: T: appendfile for $local_part@$domain
08:26:08 58130  ╰─result: T: appendfile for ametzler@localhost
08:26:08 58130 ╰──(tainted)
08:26:08 58130 T: appendfile for ametzler@localhost
08:26:08 58130 appendfile transport entered
08:26:08 58130  ╭considering: /var/mail/$local_part
08:26:08 58130  ├──expanding: /var/mail/$local_part
08:26:08 58130  ╰─result: /var/mail/ametzler
08:26:08 58130 ╰──(tainted)
08:26:08 58130 LOG: MAIN
08:26:08 58130   Warning: Tainted '/var/mail/ametzler' (file or directory name 
for mail_spool transport) not permitted
2021-04-11 08:26:08 1lVTXs-000F7W-0D Warning: Tainted '/var/mail/ametzler' 
(file or directory name for mail_spool transport) not permitted
08:26:08 58130 appendfile: mode=660 notify_comsat=0 quota=0 warning=0
08:26:08 58130   file=/var/mail/ametzler format=unix
08:26:08 58130   message_prefix=From ${if 
def:return_path{$return_path}{MAILER-DAEMON}} ${tod_bsdinbox}\n
08:26:08 58130   message_suffix=\n
08:26:08 58130   maildir_use_size_file=no
08:26:08 58130   locking by lockfile fcntl
08:26:08 58130 lock name: /var/mail/ametzler.lock
08:26:08 58130 hitch name: 
/var/mail/ametzler.lock.argenau.bebt.de.60729680.e312
08:26:08 58130 LOG: MAIN
08:26:08 58130   Warning: Tainted filename 
'/var/mail/ametzler.lock.argenau.bebt.de.60729680.e312'
08:26:08 58128 LOG: MAIN PANIC
08:26:08 58128   failed to read delivery status for ametzler@localhost from 
delivery subprocess
08:26:08 58128 LOG: MAIN PANIC
08:26:08 58128   appendfile transport process returned non-zero status 0x0100: 
exit code 1
08:26:08 58128 mail_spool transport returned DEFER for ametzler@localhost
08:26:08 58128 added retry item for T:ametzler@localhost: errno=-1 more_errno=0 
flags=0
08:26:08 58128 post-process ametzler@localhost (1)
08:26:08 58128 LOG: MAIN
08:26:08 58128   == ametzler@localhost R=local_user T=mail_spool defer (-1)

BTW the build-log with

Re: [exim] "allow_insecure_tainted_data = yes" - was: tainted data issues

2021-04-10 Thread Heiko Schlittermann via Exim-users 
Andreas Metzler via Exim-users  (Sa 10 Apr 2021 18:06:05 
CEST):
> On 2021-04-06 Heiko Schlittermann via Exim-users  wrote:
> [...]
> > .ifdef _OPT_MAIN_ALLOW_INSECURE_TAINTED_DATA
> > allow_insecure_tainted_data = yes
> > .endif
> [...]
> > Suggestions, question, remarks are welcome.
> 
> Nitpicks:
> * The changes to doc/NewStuff should not be on +fixes.
> * typos in spec.xftp: s/acessing/accessing/

Ok, I'll fix that, thank you.

-- 
Heiko


signature.asc
Description: PGP signature
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] "allow_insecure_tainted_data = yes" - was: tainted data issues

2021-04-10 Thread Heiko Schlittermann via Exim-users 
Andreas Metzler via Exim-users  (Sa 10 Apr 2021 17:37:56 
CEST):
> On 2021-04-06 Heiko Schlittermann via Exim-users  wrote:
> [...]
> > .ifdef _OPT_MAIN_ALLOW_INSECURE_TAINTED_DATA
> > allow_insecure_tainted_data = yes
> > .endif
> [...]
> > But as soon as the work stabilizes, it will be merged into the upstream
> > source. (For now, please expect changes in the commit history!)
> [...]
> > Suggestions, question, remarks are welcome.
> 
> Thank you Heiko!
> 
> I plan to add this to the next Debian release but without "taintwarn:
> set allow_insecure_data = true for 4.94+fixes". - I think it will work
> out better if we have a big fat warning

It would be good if we find more testers.
Anybody out there?

Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
--
 SCHLITTERMANN.de  internet & unix support -
 Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
 gnupg encrypted messages are welcome --- key ID: F69376CE -


signature.asc
Description: PGP signature
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] "allow_insecure_tainted_data = yes" - was: tainted data issues

2021-04-10 Thread Andreas Metzler via Exim-users 
On 2021-04-06 Heiko Schlittermann via Exim-users  wrote:
[...]
> .ifdef _OPT_MAIN_ALLOW_INSECURE_TAINTED_DATA
> allow_insecure_tainted_data = yes
> .endif
[...]
> Suggestions, question, remarks are welcome.

Nitpicks:
* The changes to doc/NewStuff should not be on +fixes.
* typos in spec.xftp: s/acessing/accessing/

cu Andreas

-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] "allow_insecure_tainted_data = yes" - was: tainted data issues

2021-04-10 Thread Andreas Metzler via Exim-users 
On 2021-04-06 Heiko Schlittermann via Exim-users  wrote:
[...]
> .ifdef _OPT_MAIN_ALLOW_INSECURE_TAINTED_DATA
> allow_insecure_tainted_data = yes
> .endif
[...]
> But as soon as the work stabilizes, it will be merged into the upstream
> source. (For now, please expect changes in the commit history!)
[...]
> Suggestions, question, remarks are welcome.

Thank you Heiko!

I plan to add this to the next Debian release but without "taintwarn:
set allow_insecure_data = true for 4.94+fixes". - I think it will work
out better if we have a big fat warning

| Consider this a major exim release, almost all customized configurations
| will require changes ...

and a note on how to *temporary* work around this by setting
allow_insecure_tainted_data in advance.

If I do not do this I expect a neverending list of reports about either
spammed logfile or breakage reports on 4.95.

cu Andreas
-- 
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'

-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/