Re: exim4 and TLS Once Again
The problem (actually 2 problems) are solved. The one I was actually posting about turned out to be that I had put a line which says protocol = smtps in the wrong place in a file named /etc/exim4/conf.d/transport/30_exim4-config_remote_smtp_smarthost This caused dpkg-reconfigure exim4-config to throw an error about not being able to generate the master configuration file named /var/lib/exim4/config.autogenerated. It turns out that it's the very last line in the file just as it was before and I had seen where I thought it should go which was wrong. After that, the login worked. Port 465 is supposed to start encrypted and 587 starts in the clear. What I found out, though, was that there was another issue, that of the user id that suddenlink needs to see versus the user ID based totally on the user and machine name. There is a specific debian fix for this problem which makes life light years easier and it is /etc/email-addresses and it installs along with your debian version of exim4 and starts life empty of anything but comments and explanation. One simply puts in a line containing their local user ID followed by a : and then the user ID that the smarthost knows you as. exim4 rewrites all the local lines such as Sender: to be the remote user ID and then you can start your happy dance because it just works. You can have multiple smarthosts and multiple lines in /etc/email-addresses so if there are multiple users on the debian system or you have multiple smarthosts to deliver to, the user ID that gets sent over the network is right each time. This is getting easier with each upgrade but you need to stick to very recent postings to get the best information. Anyway, it now works again and I am simply sending it through exim4 in the normal manner. Thank you to everybody who helped. Martin McCormick WB5AGZ
Re: exim4 and TLS Once Again
On Fri 01 Jun 2018 at 17:18:39 +0100, Brian wrote: > TLS, so the remote server sees no point in going any further. > > Create /etc/exim4/conf.d/main/00_my_custom_macros and in it put > > TLS_ON_CONNECT = true > > To the end of > /etc/exim4/conf.d/transport/30_exim4-config_remote_smtp_smarthost > add > > protocol = smtps > > 'update-exim4.conf' and restart the exim4 service. TLS_ON_CONNECT is unnecessary. However, be aware that a mail server might expect the envelope From to correspond to the mail account name, otherwise the mail will not be authorised. Gmail doesn't do this but others do. By default, Exim will use the login name on the machine plus the content of /etc/mailname for the From address. -- Brian.
Re: exim4 and TLS Once Again
On Tue 29 May 2018 at 12:00:20 -0500, David Wright wrote: > On Mon 28 May 2018 at 21:26:59 (-0500), Martin McCormick wrote: > > > > David Wright writes: > > > And exim's logs say …? > > > > 2018-05-28 14:56:38 1fNL0J-0003CQ-70 H=smtp.suddenlink.net [208.180.40.68]: > > Remote host closed connection in response to initial connection > > 2018-05-28 14:56:38 1fNL0J-0003CQ-70 == mar...@okstate.edu R=smarthost > > T=remote_smtp_smarthost defer (-18) H=smtp.suddenlink.net [208.180.40.68]: > > Remote host closed connection in response to initial connection > > > > That is 100% of the diagnostic help. With the -d flag, that part > > still doesn't change. All one gets with the -d flag is lots of > > file descriptors opening and closing. > > > > The only thing the log seems to show is that the > > authentication process goes wrong instantly but no clue whether > > or what causes that. > > I don't see anything in the log about authentication, only > "initial connection". That would suggest you haven't even got > as far as HELO/EHLO. Indeed, it does mean that. Exim does not admit to knowing anything about TLS, so the remote server sees no point in going any further. Create /etc/exim4/conf.d/main/00_my_custom_macros and in it put TLS_ON_CONNECT = true To the end of /etc/exim4/conf.d/transport/30_exim4-config_remote_smtp_smarthost add protocol = smtps 'update-exim4.conf' and restart the exim4 service. -- Brian.
Re: exim4 and TLS Once Again
On Wed 30 May 2018 at 20:58:12 -0400, Michael Stone wrote: > On Wed, May 30, 2018 at 06:22:49PM -0500, David Wright wrote: > > AIUI 587 is the standard email submission port and 465 is now > > deprecated but often still in use. I think they differ in the > > details of how they handle encrypting the session. > > > From a protocol standpoint 587/tcp is identical to 25/tcp, with the > distinction that it is designated for a end-users to submit messages for > delivery rather than accepting mail for delivery from external mail relays. > The expectation is that there is authentication of the submission, either > via allowed IPs, SMTP AUTH, or some other mechanism. Networks can block > port 25 to reduce spam originating from the network, but allow 587 for > visitors to submit email to their provider for delivery. Encryption is > activated with STARTTLS. TLS is not offered by suddenlink.net on port 587: brian@stretch:~# nc smtp.suddenlink.net 587 220 omta01.suddenlink.net ESMTP server (InterMail vM.8.04.03.22 201-2389-100-167-20150619) ready Fri, 1 Jun 2018 06:45:17 -0500 ehlo test 250-omta01.suddenlink.net 250-HELP 250-XREMOTEQUEUE 250-ETRN 250-AUTH=LOGIN PLAIN 250-AUTH LOGIN PLAIN 250-PIPELINING 250-DSN 250-8BITMIME 250 SIZE 52428800 Exim will have to use "AUTH_CLIENT_ALLOW_NOTLS_PASSWORDS = true". -- Brian.
Re: exim4 and TLS Once Again
On Wed, May 30, 2018 at 06:22:49PM -0500, David Wright wrote: AIUI 587 is the standard email submission port and 465 is now deprecated but often still in use. I think they differ in the details of how they handle encrypting the session. From a protocol standpoint 587/tcp is identical to 25/tcp, with the distinction that it is designated for a end-users to submit messages for delivery rather than accepting mail for delivery from external mail relays. The expectation is that there is authentication of the submission, either via allowed IPs, SMTP AUTH, or some other mechanism. Networks can block port 25 to reduce spam originating from the network, but allow 587 for visitors to submit email to their provider for delivery. Encryption is activated with STARTTLS. 465/tcp was at one time assigned to SMTP over TLS; that is, it is an alway-encrypted channel like 443/tcp rather than a clear text channel with encryption upgrade via STARTTLS. 465/tcp has been reassigned to another protocol (a stupid decision, but that's water under the bridge) and really shouldn't be used anymore. It would be a very old or odd installation that supported only 465/tcp and not 587/tcp. Mike Stone
Re: exim4 and TLS Once Again
On Tue 29 May 2018 at 12:25:03 (-0500), Martin McCormick wrote: > > David Wright writes: > > My previous post posed two posers. Port? > > It is supposed to be port 465. That's what the > conf.autogenerated file showed it should be but it could be doing > something contrary like port 25-- anything but the right one. Perhaps my earlier attachment got lost, so here's the text: --✁ Enter Outgoing Server Settings information and click Create. SMTP Server: smtp.suddenlink.net User Name: Your Suddenlink email address Password: The password for your Suddenlink email account Outgoing Mail Port: 25 or 587 --✃ > I have joined the exim-users mailing list as of a few > hours ago and if I find anything useful, I will let the list, > here, know as I suspect that something changed and the fix is > non-intuitive but probably not difficult if one knows what to > tweak. The server at our internet service provider is fairly > standards-based so we shouldn't have to boil the ocean to make it > work again. AIUI 587 is the standard email submission port and 465 is now deprecated but often still in use. I think they differ in the details of how they handle encrypting the session. Cheers, David.
Re: exim4 and TLS Once Again
David Wright writes: > My previous post posed two posers. Port? It is supposed to be port 465. That's what the conf.autogenerated file showed it should be but it could be doing something contrary like port 25-- anything but the right one. I have joined the exim-users mailing list as of a few hours ago and if I find anything useful, I will let the list, here, know as I suspect that something changed and the fix is non-intuitive but probably not difficult if one knows what to tweak. The server at our internet service provider is fairly standards-based so we shouldn't have to boil the ocean to make it work again. Thanks. Martin McCormick
Re: exim4 and TLS Once Again
On Mon 28 May 2018 at 21:26:59 (-0500), Martin McCormick wrote: > > David Wright writes: > > And exim's logs say …? > > 2018-05-28 14:56:38 1fNL0J-0003CQ-70 H=smtp.suddenlink.net [208.180.40.68]: > Remote host closed connection in response to initial connection > 2018-05-28 14:56:38 1fNL0J-0003CQ-70 == mar...@okstate.edu R=smarthost > T=remote_smtp_smarthost defer (-18) H=smtp.suddenlink.net [208.180.40.68]: > Remote host closed connection in response to initial connection > > That is 100% of the diagnostic help. With the -d flag, that part > still doesn't change. All one gets with the -d flag is lots of > file descriptors opening and closing. > > The only thing the log seems to show is that the > authentication process goes wrong instantly but no clue whether > or what causes that. I don't see anything in the log about authentication, only "initial connection". That would suggest you haven't even got as far as HELO/EHLO. My previous post posed two posers. Port? Cheers, David.
Re: exim4 and TLS Once Again
David Wright writes: > And exim's logs say � 2018-05-28 14:56:38 1fNL0J-0003CQ-70 H=smtp.suddenlink.net [208.180.40.68]: Remote host closed connection in response to initial connection 2018-05-28 14:56:38 1fNL0J-0003CQ-70 == mar...@okstate.edu R=smarthost T=remote_smtp_smarthost defer (-18) H=smtp.suddenlink.net [208.180.40.68]: Remote host closed connection in response to initial connection That is 100% of the diagnostic help. With the -d flag, that part still doesn't change. All one gets with the -d flag is lots of file descriptors opening and closing. The only thing the log seems to show is that the authentication process goes wrong instantly but no clue whether or what causes that. I have looked in the man page and the word smarthost doesn't appear. /usr/share/doc/exim4-base/ has nothing with one file briefly mentioning a smarthost but no best practices or sample configuration examples. I also know experimentally that no part of /etc/exim4/passwd.client appears in the /var/lib/exim4/config.autogenerated file so exim4 must give that out on demand or maybe I should say it is part of the output generated during the session. Either the passwd.client file needs something else or another file needs modification as the user name and password are known to be good. The experiment was to generate a new conf.autogenerated file after modifying passwd.client and using diff between it and the previous config.autogenerated file. They were a perfect match. If there is a document somewhere describing this setup for exim4.80 or above, it should be useful as the changes apparently took place around version 4.79 or 4.80 and this version is 4.89. The fact that you are reading this message is proof that the user ID and password still work with smtp.suddenlink.net, just not in exim4 yet. Thank you. Martin McCormick
Re: exim4 and TLS Once Again
On Mon 28 May 2018 at 15:32:44 (-0500), Martin McCormick wrote: > > After about two weeks of going down all sorts of rabbit holes > and wasting tons of time, I am at a loss trying to get > exim4 to resume the ability to send messages via the smarthost > used by our ISP. The real trouble here is that all one really > knows is that something's broken. It happens as soon as exim4 > contacts the server. The server immediately aborts. It's the > ultimate "Check engine" light. 30-thousand moving parts and one > is bad. Go figure. > > It was all working fine for nearly 3 years save for a > hiccup of some kind at the ISP's site last January but this time, > it is on my end and I know that for sure. > > Connections are made using TLS on port 465. Why do I read the attached then? > Originally, what one did was to enter the user name and > password in to a file called /etc/exim4/passwd.client as follows: > > # password file used when the local exim is authenticating to a remote > # host as a client. > # > # see exim4_passwd_client(5) for more documentation > # > # Example: > ### target.mail.server.example:login:password > *.suddenlink.net:marti...@suddenlink.net:deepsecret > > The other modification was to a file called > /etc/exim4/update-exim4.conf.conf which is a debian-specific file > that configures the configurations hence .conf.conf. > > One added a couple of lines to indicate we are using the > protocol called smtps. > > After that, one ran dpkg-reconfigure exim4-config and > selected to send mail through a smarthost and receive via > fetchmail. > > It all worked until I upgraded to stretch at which point > the smarthost began dumping the connection upon the login > attempt. At least that is what appears to happen. > > Actually, I couldn't even run the reconfigure because it > complained about the protocol = smtps line and refused to build > /var/lib/exim4/conf.autogenerated file. I had to remove that > line and then it built a dead-on-arrival conf.autogenerated file > that still allows exim4 to deliver local mail but always gets the boot > from the suddenlink server. And exim's logs say …? Cheers, David.
exim4 and TLS Once Again
After about two weeks of going down all sorts of rabbit holes and wasting tons of time, I am at a loss trying to get exim4 to resume the ability to send messages via the smarthost used by our ISP. The real trouble here is that all one really knows is that something's broken. It happens as soon as exim4 contacts the server. The server immediately aborts. It's the ultimate "Check engine" light. 30-thousand moving parts and one is bad. Go figure. It was all working fine for nearly 3 years save for a hiccup of some kind at the ISP's site last January but this time, it is on my end and I know that for sure. Connections are made using TLS on port 465. Originally, what one did was to enter the user name and password in to a file called /etc/exim4/passwd.client as follows: # password file used when the local exim is authenticating to a remote # host as a client. # # see exim4_passwd_client(5) for more documentation # # Example: ### target.mail.server.example:login:password *.suddenlink.net:marti...@suddenlink.net:deepsecret The other modification was to a file called /etc/exim4/update-exim4.conf.conf which is a debian-specific file that configures the configurations hence .conf.conf. One added a couple of lines to indicate we are using the protocol called smtps. After that, one ran dpkg-reconfigure exim4-config and selected to send mail through a smarthost and receive via fetchmail. It all worked until I upgraded to stretch at which point the smarthost began dumping the connection upon the login attempt. At least that is what appears to happen. Actually, I couldn't even run the reconfigure because it complained about the protocol = smtps line and refused to build /var/lib/exim4/conf.autogenerated file. I had to remove that line and then it built a dead-on-arrival conf.autogenerated file that still allows exim4 to deliver local mail but always gets the boot from the suddenlink server. After a week of web searches, there is a bundle of out-dated how-to's that used to work but nothing useful for what to do to fix it now. So, you might ask, How am I sending this message? It's a proper question, all right. There has been a program called msmtp that only does one thing well and that is to make a smtps connection to a smtp smarthost. One sets up a configuration file called ~/.msmtprc and there is where you put the smarthost's fully-qualified domain name plus one's login credentials. It works just fine. You link the name sendmail to /usr/bin/msmtp and call it via that link with the -t flag as you pipe your message to it. The message must be a properly-formatted email message and msmtp makes the login to the smarthost and delivers the message. The trouble is that you lose local mail on the system so you don't get all the gory detains when a shell script run by cron blows up, etc. I've got a few of those kinds of scripts and it is nice to know when they misbehave. The real problem is that there does not appear to be a reasonable way to listen to the transaction when you need it most. Since the encryption is end-to-end, wireshark or some third-party monitoring device will just spout garbage when what one needs is a clear-text feed of what exim4 and the server are saying to each other. If exim4 could do that, we would know what the last words were before the wheels came off. At least I know the login credentials have not changed since msmtp works but this business of manually piping a formatted message to msmtp is for the birds but at least it beats nothing by a hair. exim -d -M on a message puts out about 400 plus lines of mostly useless information, a full-color check engine light so to speak. Any constructive ideas are much appreciated. Thank you. Martin McCormick
