On Sun, Mar 12, 2023 at 03:10:54PM +0200, Victor Ustugov via Exim-users wrote: > https://packages.debian.org/source/sid/exim4 to it. I mean 75_*.patch > > Script get_oauth2_access_token-v.corvax.t...@gmail.com.sh is based on > https://github.com/pcw11211/exim4-oauth2/blob/main/get_bearer_tocken.sh > > This is just a prototype. I'm going to cache responses to requests to > get an access_token, taking into account the expires_in value, in order > to reduce the number of HTTP requests.
For what it is worth, I am doing this already in my Oauth script for use with MS servers. I use gawk to do that part. It would be on Gitlab already if it were not for real life in the way of a failing car battery getting in the way. > Client id, client secret and refresh token will be stored in the > database, so there will be no need to create a script to get an access > token for each user. But I am not doing that, so far at least. For information, I will attach the README.md for my script as it stands today. ael
# Introduction This minimal project allows exim4 to send email to an SMTP server which only allows **OAUTH2**. At least for SMTP servers in the MS orbit. It has been tested in conjuction with [M365-IMAP](https://github.com/UvA-FNWI/M365-IMAP) which provides a similar service for collecting email from an MS IMAP server using [offlineimap](https://github.com/OfflineIMAP/offlineimap3). At the time of writing exim4 does not support OAUTH2, so it needs a separate utility to authenticate. That is the purpose of this program. The only similar utility that the author could find was [exim4-oauth2](https://github.com/pcw11211/exim4-oauth2), but that was aimed at a gmail server, and could not be modified to work with an MS server. But the basic ideas from exim4-oath2 have been used here. ## A sample OAUTH2 transmission Below is an example of successful XAOUTH transmission of an email to an office365 SMTP server. This example, gathered using the exim -v option, shows the main features which may help to explain a little of the terminology and how to configure exim. SMTP<< 220 LO4P123CA0173.outlook.office365.com Microsoft ESMTP MAIL Service ready at Sat, 4 Mar 2023 11 :14:06 +0000 SMTP>> EHLO this.machine SMTP<< 250-LO4P123CA0173.outlook.office365.com Hello [source_ip] 250-SIZE 157286400 250-PIPELINING 250-DSN 250-ENHANCEDSTATUSCODES 250-STARTTLS 250-8BITMIME 250-BINARYMIME 250-CHUNKING 250 SMTPUTF8 The server advertises options, and exim will choose STARTTLS below. This is because the exim transport uses driver = smtp protocol = smtp hosts_require_tls = * This TLS encryption is needed before the server will offer XOAUTH2. SMTP>> STARTTLS SMTP<< 220 2.0.0 SMTP server ready SMTP>> EHLO this.machine SMTP<< 250-LO4P123CA0173.outlook.office365.com Hello [source_ip] 250-SIZE 157286400 250-PIPELINING 250-DSN 250-ENHANCEDSTATUSCODES 250-AUTH LOGIN XOAUTH2 250-8BITMIME 250-BINARYMIME 250-CHUNKING 250 SMTPUTF8 One of the offers above is *AUTH LOGIN XOAUTH2*, so exim will reply *AUTH XOAUTH2* and will deliver (base84 encoded) the access token and more which it obtains from the script *exim_ms_tokens* which is provided by this utility. This is because the exim configuration must have a matching authenicator of the form: login_oauth2: driver = plaintext public_name = XOAUTH2 client_send = ${run{ SOMEWHERE/exim_ms_tokens } {$value} fail } It is that *public_name = XOAUTH2* which allows exim to select the OAUTH2 option. *plaintext* is safe because TLS is already in operation. In passing, note that if you need to talk to other hosts using OAUTH2, such as gmail, you will need a conditional in *login_oauth2* to select another script to provide the relevant token, but that is not covered here. SMTP>> AUTH XOAUTH2 ************ [.. snip.. The token and more are sent] SMTP<< 235 2.7.0 Authentication successful SMTP|> MAIL FROM:<me@test> SIZE=1378 AUTH=ddd@this.machine SMTP|> RCPT TO:<foo@fizz> will write message using CHUNKING SMTP+> BDAT 369 LAST SMTP>> QUIT SMTP<< 250 2.1.0 Sender OK SMTP<< 250 2.1.5 Recipient OK SMTP<< 250 2.0.0 OK <E1pYPpI-0002cc-2b@this.machine> [Hostname=NNN265MB0919.GBRP265.PROD.OUTLOOK.COM] SMTP<< 221 2.0.0 Service closing transmission channel That is a successful transmission. Note that at least some versions of exim may report an error as below which is probably spurious: it has been suggested that the gnutls library is complaining about a early closure which is actually legal behaviour. So it should be safe to ignore reports like the one below. LOG: MAIN H=outlook.xx.office365.com [nn.nn.nnn.nnn] TLS error on connection (recv): Error in the pull function. # Prerequisites 1. A *bash* compatible shell. *dash* is ideal. 2. [curl](https://curl.se) 3. [gawk](http://www.gnu.org/software/gawk/) 4. *coreutilties* providing basic utilities like *cut* and *date*. 5. Authority to collect tokens: see later. # Configuring exim A router along these lines is needed: exampleoauth2: debug_print = "R: exampleoauth2 for $local_part@$domain" driver = manualroute domains = ! +local_domains transport = example_t route_list = * "outlook.office365.com::587" byname host_find_failed = ignore same_domain_copy_routing = yes no_more If you use several "smarthosts", that is a variety of remote SMTP servers, you will probably need to include a *condition* to decide when to select the 365 server. The transport *example_t* could be example_t: debug_print = "T: example_t for $local_part@$domain" driver = smtp multi_domain hosts_require_tls = * hosts_try_auth = $host_address protocol = smtp Last, an authenticator, as before: login_oauth2: driver = plaintext public_name = XOAUTH2 client_send = ${run{ SOMEWHERE/exim_ms_tokens } {$value} fail } *SOMEWHERE* is the directory where you decide to place *exim_ms_tokens*, perhaps /usr/local/etc/exim. It must be executable by the exim user: it might be wise to restrict ordinary users from executing or reading it. Reading it will reveal the location of the cached tokens, although if those files are properly protected, this might not be of particular concern. ## Placing the router, transport and authenicator You probably don't need to be told that the router should be somewhere in the section starting *begin routers*, and similarly *begin transports* and *begin authenicators*. If you are using the Debian exim4 package, you will probably already have an *exim.conf.template* that you can adapt. Set it up to use a "smarthost" which will be outlook.office365.com::587. Depending on whether you choose a "split" configuration or not, edit exim.conf.template or files under /etc/exim/conf.d . Once you have done the above, you will find that you already have a router and transport set up, and basically just need to add the authenicator. Just check those automatically generated router and transport: you may not need to adjust them. Note that when the debian package is updated, you probably want to avoid the dpkg-reconfiguration overwriting your changes. You may need to update the configuration manually. vimdiff with folding makes this very easy. # Editing *exim_ms_tokens* This file is a shell script, and includes some documentation in the comments especially towards the end. It handles two files which cache the refresh and access tokens. The first task is to decide on the location of these small files. The access token typically has a lifetime of around an hour: this from observation. The refresh token has a much longer lifetime which some documentation suggests may be around 90 days. *exim_ms_tokens* updates the cached refresh token every month or so. This is to save write cycles especially on flash media. The access token is likely to be written more frequently, so it may be a good idea to place it on a *tmpfs* if SSD wear matters. This is the main reason for keeping the tokens in different files. However, since the access token is unlikely to be written more once an hour, this is perhaps over cautious. 1. The refresh token is cached in REFRESH_FILE. This should only be readable by the exim user. Obtaining the initial refresh token to be placed in this file is covered later. 2. ACCESS_FILE contains the access token, and again should only be readable by the exim user. The exim user must also be able to write both of the above files. Edit *exim_ms_tokens* to set *REFRESH_FILE* and *ACCESS_FILE*. ## Getting the initial tokens First you need a *client_id* and a *client_secret* which are issued by MS. Here we try to avoid disappearing into a nightmare of complexity and cloud tenancies. If you are already a slave, eh, tenant of MS, then you may well have an *id* and *secret*, and not need much more guidance. But otherwise, it is much easier to use the thunderbird client values. All of this and more is covered in the [M365-IMAP](https://github.com/UvA-FNWI/M365-IMAP) README.md file. Since you are using exim4 to handle your outgoing mail at least, you may perhaps be fetching your MS-hosted email via IMAP. M365-IMAP handles that in conjunction with offlineimap. If that is the case, I suggest installing M365-IMAP and setting it up which will involve fetching the refresh token which you can use here. But even if you do not use offlineimap or any IMAP, I still suggest installing M365-IMAP and following the instructions there to run *get_token.py*. You will then have a file, *imap_smtp_refresh_token*, which contains your initial refresh token. All you need to do is to copy that *imap_smtp_refresh_token* to *REFRESH_FILE*. That is, copy the contents to the location you selected above in the *exim_ms_tokens* file. It may be easiest to do this as root (or using sudo) to avoid permission problems. But that *REFRESH_FILE* is not quite ready: it needs an expiration date in addition to the token itself. Open the file with your favorite text editor, and insert a leading "1234,". So the file consists of a single line, starting with a fake expiration date way in the past of 1234, then a "," which acts as a separator followed by the refresh token and nothing else. This will be rewritten when *exim_ms_tokens* is called from exim. Or directly by root when testing... It only remains to edit *exim_ms_tokens* to insert your *client-id* and *client_secret* values in the obvious "???...???" places between the quotes. Set *email_addr* to your email address that you use when logging in to the MS system: it may be different from any of the email addresses used by exim. Ensure that *exim_ms_tokens* permits execution by the exim user. chmod ug+x *exim_ms_tokens* if it is owned by that user, although it is probably harmless to allow execution by other users since it should fail if the permissions on the token files have been set properly. # Summary Edit *exim_ms_tokens* to change the values of 1. *REFRESH_FILE* 2. *ACCESS_FILE* 3. *client_id* 4. *client_secret* and 5. *email_addr* . chmod to allow the exim user to execute *exim_ms_tokens*. Prepare the *REFRESH_FILE* to contain a leading fake expiry date, a "," and then the refresh token from *imap_smtp_refresh_token*, or elsewhere. "1,tttttttt...." will do where ttttt... is the refresh token and the quotes are not part of the file. Check that the *REFRESH_FILE* can only be read only by the exim user: Debian-exim on Debian. Note: if *exim_ms_token* is not run for 90 days or so and the refresh token expires, it cannot recover. You will need to obtain a new refresh token by running *get_token.py* or otherwise, and then set up the *REFRESH_FILE* again. # Testing If you find that exim is failing to send email, and the *mainlog* does not identify the problem, you might consider below. If, and only if, you are happy to execute *exim_ms_tokens* as root, then # ./exim_ms_tokens|hexdump -C |less in the right directory will allow you to check that the script is working. You can inspect the *ACCESS_FILE* and the *REFRESH_FILE* to see if they have been modified. But this is definitely only for experienced system admins who thoroughly understand the risks. Otherwise, you can carry out a similar test, but will have to arrange all the permisions and restore them afterwards which is complex and messy.
-- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
