Re: Dovecot Gmail OAuth2.0 Setting Question
> On 19/01/2021 07:17 福田泰葵 wrote:
>
>
> Dear Sir or Madam
> Unable to build OAuth2.0 authentication to Gmail using dovecot as proxy.
> I have a question about how to use dovecot as a proxy to perform OAuth 2.0
> authentication to Gmail using a mail client.
Mail client is required, in this case, to provide valid oauth2 bearer token. I
don't think google supports other ways.
> 1. Is the following all I need to do to authenticate to Gmail using dovecot
> as a proxy?
> * passdb
> passdb {
> driver = oauth2
> mechanisms = oauthbearer xoauth2
> args = /etc/dovecot/dovecot-oauth2.token.conf.ext
> }
> passdb {
> driver = oauth2
> mechanisms = plain login
> args = /etc/dovecot/dovecot-oauth2.plain.conf.ext
> }
>
The plain config is a way to do 'password grant' authentication. This is when
username and password is used to acquire a bearer token.
> * create dovecot-oauth2.token.conf.ext and dovecot-oauth2.plain.conf.ext
> * create gmail service account api
> 2. grant_url in dovecot-oauth2.token.conf.ext and
> dovecot-oauth2.plain.conf.ext is URL for obtaining a Google access token for
> a web server that I have built myself?
> 3. I use a Gmail service account, so I don’t need a client ID and secret
> ID, right?
> 4. Do I set introspection_url to the URL of my own web server with the
> access token used for authentication to Google as the response?
No. The introspection URL needs to point to a location where dovecot can figure
out more information about the user with token. If I recall correctly, the
token endpoint
For gmail, you need to use https://www.googleapis.com/oauth2/v2/userinfo
> 5. The documentation says “pass_attrs = host=127.0.0.1”, but if you are
> authenticating to Gmail, I should use
> “pass_attrs = proxy=y host=%{if;%s;eq;imap;imap.gmail.com
> (http://imap.gmail.com);%{if;%s;eq;pop3;smtp .gmail.com
> (http://gmail.com);pop.gmail.com (http://pop.gmail.com)}}
> port=%{if;%s;eq;imap;993;%{if;%s;eq;pop3;587;465}} proxy_mech=xoauth2
> pass=%{oauth2:access_token} user=%{oauth2:email oauth2:email}”?
I would use something more readable, like passwd-file driver with
username_format=%s
The access token is also imported as %{token} in passdb.
> 6. What is the difference between dovecot-oauth2.token.conf.ext and
> dovecot-oauth2.plain.conf.ext ? Do I need to configure both?
> I used
> https://doc.dovecot.org/configuration_manual/authentication/oauth2/#proxy as
> a reference.
> I would appreciate your reply.
> Yours faithfully,
> --
> e-mail: taiki.fuk...@justsystems.com
> TEL: 03-5324-7900
> mobile: 080-6198-7328
> --
So this might work
/etc/dovecot/oauth2-token.conf.ext
introspection_url = https://www.googleapis.com/oauth2/v2/userinfo
introspection_mode = auth
username_attribute = email
pass_attrs = proxy=y proxy_mech=xoauth2
/etc/dovecot/dovecot.conf
auth_mechanisms = xoauth2 oauthbearer
passdb {
driver = oauth2
args = /etc/dovecot/oauth2-token.conf.ext
result_success = continue-ok
}
passdb {
driver = passwd-file
args = username_format=%s /etc/dovecot/endpoints
skip = unauthenticated
}
/etc/dovecot/endpoints
imap::: host=imap.gmail.com
pop3::: host=pop3.gmail.com
submission::: host=smtp.gmail.com
Aki
Dovecot Gmail OAuth2.0 Setting Question
Dear Sir or Madam
Unable to build OAuth2.0 authentication to Gmail using dovecot as proxy.
I have a question about how to use dovecot as a proxy to perform OAuth 2.0
authentication to Gmail using a mail client.
1.
Is the following all I need to do to authenticate to Gmail using dovecot
as a proxy?
-
passdb
passdb {
driver = oauth2
mechanisms = oauthbearer xoauth2
args = /etc/dovecot/dovecot-oauth2.token.conf.ext
}
passdb {
driver = oauth2
mechanisms = plain login
args = /etc/dovecot/dovecot-oauth2.plain.conf.ext
}
-
create dovecot-oauth2.token.conf.ext and dovecot-oauth2.plain.conf.ext
- create gmail service account api
2.
grant_url in dovecot-oauth2.token.conf.ext and
dovecot-oauth2.plain.conf.ext is URL for obtaining a Google access token
for a web server that I have built myself?
3.
I use a Gmail service account, so I don’t need a client ID and secret
ID, right?
4.
Do I set introspection_url to the URL of my own web server with the
access token used for authentication to Google as the response?
5.
The documentation says “pass_attrs = host=127.0.0.1”, but if you are
authenticating to Gmail, I should use
“pass_attrs = proxy=y
host=%{if;%s;eq;imap;imap.gmail.com;%{if;%s;eq;pop3;smtp
.gmail.com;pop.gmail.com}}
port=%{if;%s;eq;imap;993;%{if;%s;eq;pop3;587;465}} proxy_mech=xoauth2
pass=%{oauth2:access_token} user=%{oauth2:email oauth2:email}”?
6.
What is the difference between dovecot-oauth2.token.conf.ext and
dovecot-oauth2.plain.conf.ext ? Do I need to configure both?
I used
https://doc.dovecot.org/configuration_manual/authentication/oauth2/#proxy
as a reference.
I would appreciate your reply.
Yours faithfully,
--
e-mail: taiki.fuk...@justsystems.com
TEL: 03-5324-7900
mobile: 080-6198-7328
--
Dovecot Gmail OAuth2.0 Setting Question
Dear Sir or Madam
Unable to build OAuth2.0 authentication to Gmail using dovecot as proxy.
I have a question about how to use dovecot as a proxy to perform OAuth 2.0
authentication to Gmail using a mail client.
1.
Is the following all I need to do to authenticate to Gmail using dovecot
as a proxy?
-
passdb
passdb {
driver = oauth2
mechanisms = oauthbearer xoauth2
args = /etc/dovecot/dovecot-oauth2.token.conf.ext
}
passdb {
driver = oauth2
mechanisms = plain login
args = /etc/dovecot/dovecot-oauth2.plain.conf.ext
}
-
create dovecot-oauth2.token.conf.ext and dovecot-oauth2.plain.conf.ext
- create gmail service account api
2.
grant_url in dovecot-oauth2.token.conf.ext and
dovecot-oauth2.plain.conf.ext is URL for obtaining a Google access token
for a web server that I have built myself?
3.
I use a Gmail service account, so I don’t need a client ID and secret
ID, right?
4.
Do I set introspection_url to the URL of my own web server with the
access token used for authentication to Google as the response?
5.
The documentation says “pass_attrs = host=127.0.0.1”, but if you are
authenticating to Gmail, I should use
“pass_attrs = proxy=y
host=%{if;%s;eq;imap;imap.gmail.com;%{if;%s;eq;pop3;smtp
.gmail.com;pop.gmail.com}}
port=%{if;%s;eq;imap;993;%{if;%s;eq;pop3;587;465}} proxy_mech=xoauth2
pass=%{oauth2:access_token} user=%{oauth2:email oauth2:email}”?
6.
What is the difference between dovecot-oauth2.token.conf.ext and
dovecot-oauth2.plain.conf.ext ? Do I need to configure both?
I used
https://doc.dovecot.org/configuration_manual/authentication/oauth2/#proxy as
a reference.
I would appreciate your reply.
Yours faithfully,
--
e-mail: taiki.fuk...@justsystems.com
TEL: 03-5324-7900
mobile: 080-6198-7328
--
Re: dovecot 2.3.13 : make check FAILURE : Assert failed: buffer_append_full_file
On 1/18/21 11:22 AM, Aki Tuomi wrote:
line 52 does chmod to
lines 54-55 attempts to read this file now, and expects it to fail
line 56 ensures that there was actual error provided
Oh yes, of course you are correct. I was confused by the MAX_SIZE
parameter.
I wrapped that test with
my_euid = geteuid();
if (0 == my_euid) { /* am I running as root ? */
seteuid(1); /* set effective uid to non-root */
}
[ run test ]
if (0 == my_euid) { /* was I running as root ? */
seteuid(0); /* reinstate effective uid to root again */
}
and now it does not fail.
Aki
I also discovered the cause of the ld errors from running some of the
tests - the gcc link command was failing because the openssl libs
were either not included in the command line or placed too early in the
command line, before the libssl_iostream_openssl.so which references
them. I had same problem with the make but fixed it up, but the
same treatment is needed for the make check. My openssl libs are in
/opt/openssl/lib, and I did follow the build instructions and specify
SSL_CFLAGS="-I/opt/openssl/include" SSL_LIBS="-L/opt/openssl/lib"
LDFLAGS="-L/opt/openssl/lib"
but libtool did not do the right thing in some places. dovecot seems to
have an unusual way of "linking" the openssl libs.
Eventually I think all is well. Thanks
Cheers, John Lumby
.SSL_CFLAGS="-I/opt/openssl/include" SSL_LIBS="-L/opt/openssl/lib"
LDFLAGS="-L/opt/openssl/lib
Re: Obtaining the IMAP GUID from a sieve script
On 18 Jan 2021, at 04:12, Jochen Bern wrote: (Also, you can legally have several e-mails with the same Message-ID in your mailbox; e.g., someone addressed it to two aliases that both expand to you, just to name one possibilty where *both* go through *sieve* as well.) On 18.01.21 12:18, @lbutlr wrote: I delete duplicates before they are delivered to a mailbox. From using doveadm-deduplicate, I remember that with message-Id as criterion, it would delete too much for my taste, of what makes a duplicate. -- peter
Re: Obtaining the IMAP GUID from a sieve script
On 18.01.21 12:18, @lbutlr wrote: > On 18 Jan 2021, at 04:12, Jochen Bern wrote: >> (Also, you can legally have several e-mails with the same Message-ID in >> your mailbox; e.g., someone addressed it to two aliases that both expand >> to you, just to name one possibilty where *both* go through *sieve* as >> well.) > > I delete duplicates before they are delivered to a mailbox. ... do I really need to elaborate on why I wrote "just to name *one* possibility"? Your own mails to a (simple) mailinglist, given that you'll likely already have a copy in your "Sent" folder as the on-list version comes back to you? Received mail matching several topics at once, and the user first copies it to archive folder A, then moves the original to folder B, all via IMAP? Versions of an e-mail that are *marked* deleted, but not yet expunged, and can still be seen/accessed/undeleted/moved/copied/... in *some* IMAP clients? Regards, -- Jochen Bern Systemingenieur Binect GmbH smime.p7s Description: S/MIME Cryptographic Signature
Re: dovecot 2.3.13 : make check FAILURE : Assert failed: buffer_append_full_file
> On 18/01/2021 17:21 J Lumby wrote: > > > On 1/18/21 12:25 AM, Aki Tuomi wrote: > >> On 18/01/2021 00:35 J Lumby wrote: > >> > >> > >> Thanks Aki. > >> > >> > >> On 1/16/21 10:18 AM, Aki Tuomi wrote: > On 16/01/2021 16:36 J Lumby wrote: > > > > make check had one failure as follows : > > test-buffer-istream.c:54: Assert failed: buffer_append_full_file(result, > TEST_FILENAME, SIZE_MAX, &error) == BUFFER_APPEND_READ_ERROR > test-buffer-istream.c:56: Assert failed: error != NULL && *error != '\0' > buffer_append_full_file .. : > FAILED > > >>> > >> And (I assume) it is best to run the make check under same userid, i.e. > >> root, otherwise it is not testing what will actually be running. > >> > >> But anyway I tried running make check under non-root and the result was > >> much worse - some indeterminate FAIL much earlier : > >> > >> Making check in lib-ssl-iostream > >> make[2]: Entering directory > >> '/mnt/julywext/wextmisc/fed30GBroot/ahcombld/dovecot-2.3.13/src/lib-ssl-iostream' > >> make check-local > >> make[3]: Entering directory > >> '/mnt/julywext/wextmisc/fed30GBroot/ahcombld/dovecot-2.3.13/src/lib-ssl-iostream' > >> for bin in test-iostream-ssl; do \ > >> if ! /bin/sh ../../run-test.sh ../.. ./$bin; then exit 1; fi; \ > >> done > >> collect2: error: ld returned 213 exit status > >> Failed to run: ./test-iostream-ssl > >> > > This is because you have also compiled the source as root and fails to > > write to those directories. You need to build as non-root, too. > No, for this attempt I ran all steps as non-root including make. > > > >> > >> > >> May we re-visit the FAIL I reported originally ? Why does it FAIL > >> when run under userid root? > >> > > Because it's testing that it cannot read a file it has no permissions to > > read. root can read all files. You can see it does a chmod few lines before. > > You are in the wrong place -in the file - you are looking at line > 61,my error occurred at line 54 - see error message earlier. > > It is make some kind of test for size of record being read. > > > > > And, what code could I add to test-buffer-istream.c to make it print > > out the offending errno? > > > > Or, perhaps easier - is it safe to ignore this one FAIL? > Hoping you will answer these ... > >>> Aki > >>> . > >> Cheers, John Lumby > > Aki > > . line 52 does chmod to lines 54-55 attempts to read this file now, and expects it to fail line 56 ensures that there was actual error provided Aki
Re: Changing the default delivery mailbox to something other than INBOX
On Jan 18, 2021, at 8:11 AM, Aki Tuomi wrote: > How about using IMAP Sieve for Spam filter training? > https://doc.dovecot.org/configuration_manual/howto/antispam_with_sieve/ That is in fact exactly what I am doing. > Also, sieve rules are processed during Delivery (lmtp or lda), not if users > or spam filters move messages around. IMAP sieve is different thing, and > needs to be separately configured. Not sure what "infinite loop" you are > seeing here? Ah. I did not realize that sieve and IMAP sieve were two different things. I though they were synonyms. I’m not actually *seeing* an infinite loop, I was merely anticipating one. But that was based on what is apparently a false assumption. So I need to go back and re-do my design analysis with this in mind. Thanks for the help! rg
Re: Changing the default delivery mailbox to something other than INBOX
> On 18/01/2021 18:05 Ron Garret wrote:
>
>
> On Jan 17, 2021, at 11:06 PM, Aki Tuomi wrote:
>
> >
> >> On 18/01/2021 08:56 Ron Garret wrote:
> >>
> >>
> >> On Jan 17, 2021, at 10:48 PM, Aki Tuomi wrote:
> >>
> >>> Don't touch the INBOX setting (leave it out), use the -m parameter for
> >>> dovecot-lda.
> >>
> >> I don’t think that will work. My MTA is postfix and it’s connected to
> >> dovecot via LMTP, so AFAIK postfix is talking to dovecot over a socket,
> >> and dovecot-lda isn’t being invoked so there is no place to pass that the
> >> -m parameter. But we are at the limits of my understanding of how all
> >> this stuff actually works under the hood so please correct me if I’m wrong
> >> here.
> >>
> >> rg
> >
> > Ah, you are using LMTP... then the easiest is to use Sieve here.
>
> I was afraid of that :-(
>
> What about just using postfix as the LDA?
>
> > You can use simple global script, e.g. /etc/dovecot/default-mbox.sieve
> >
> > ```
> > require ["fileinto", "mailbox"];
> > fileinto "OtherBox";
> > ```
> >
> > then sievec the script, and use
> >
> > plugin {
> > sieve_before = /etc/dovecot/default-mbox.sieve
> > }
> >
> > this way it will change the default mailbox, but allows per-user scripts to
> > change it.
>
> The problem is that the spam filter needs to put good messages back in INBOX.
> That would cause an infinite loop (unless sieve has a mechanism to prevent
> this that I don’t know about). So to prevent this I’d need to make the rule
> conditional on some header that the spam filter adds before moving the
> message back. The problem with *that* is that it plays badly with the
> strategy for training the filter, which is for the user to move
> false-positive messages from SPAM to INBOX. A user might then also move a
> good message from OtherBox to INBOX rather than wait for the spam filter to
> get around to processing it, and then sieve would (presumably) move it right
> back. So now I need *another* header that *sieve* adds before moving the
> message to OtherBox and make the rule conditional on *that* header. It all
> starts to seem very complicated and fragile.
>
> rg
How about using IMAP Sieve for Spam filter training?
https://doc.dovecot.org/configuration_manual/howto/antispam_with_sieve/
Also, sieve rules are processed during Delivery (lmtp or lda), not if users or
spam filters move messages around. IMAP sieve is different thing, and needs to
be separately configured. Not sure what "infinite loop" you are seeing here?
Aki
Re: Changing the default delivery mailbox to something other than INBOX
On Jan 17, 2021, at 11:06 PM, Aki Tuomi wrote:
>
>> On 18/01/2021 08:56 Ron Garret wrote:
>>
>>
>> On Jan 17, 2021, at 10:48 PM, Aki Tuomi wrote:
>>
>>> Don't touch the INBOX setting (leave it out), use the -m parameter for
>>> dovecot-lda.
>>
>> I don’t think that will work. My MTA is postfix and it’s connected to
>> dovecot via LMTP, so AFAIK postfix is talking to dovecot over a socket, and
>> dovecot-lda isn’t being invoked so there is no place to pass that the -m
>> parameter. But we are at the limits of my understanding of how all this
>> stuff actually works under the hood so please correct me if I’m wrong here.
>>
>> rg
>
> Ah, you are using LMTP... then the easiest is to use Sieve here.
I was afraid of that :-(
What about just using postfix as the LDA?
> You can use simple global script, e.g. /etc/dovecot/default-mbox.sieve
>
> ```
> require ["fileinto", "mailbox"];
> fileinto "OtherBox";
> ```
>
> then sievec the script, and use
>
> plugin {
> sieve_before = /etc/dovecot/default-mbox.sieve
> }
>
> this way it will change the default mailbox, but allows per-user scripts to
> change it.
The problem is that the spam filter needs to put good messages back in INBOX.
That would cause an infinite loop (unless sieve has a mechanism to prevent this
that I don’t know about). So to prevent this I’d need to make the rule
conditional on some header that the spam filter adds before moving the message
back. The problem with *that* is that it plays badly with the strategy for
training the filter, which is for the user to move false-positive messages from
SPAM to INBOX. A user might then also move a good message from OtherBox to
INBOX rather than wait for the spam filter to get around to processing it, and
then sieve would (presumably) move it right back. So now I need *another*
header that *sieve* adds before moving the message to OtherBox and make the
rule conditional on *that* header. It all starts to seem very complicated and
fragile.
rg
Re: dovecot 2.3.13 : make check FAILURE : Assert failed: buffer_append_full_file
On 1/18/21 12:25 AM, Aki Tuomi wrote: On 18/01/2021 00:35 J Lumby wrote: Thanks Aki. On 1/16/21 10:18 AM, Aki Tuomi wrote: On 16/01/2021 16:36 J Lumby wrote: make check had one failure as follows : test-buffer-istream.c:54: Assert failed: buffer_append_full_file(result, TEST_FILENAME, SIZE_MAX, &error) == BUFFER_APPEND_READ_ERROR test-buffer-istream.c:56: Assert failed: error != NULL && *error != '\0' buffer_append_full_file .. : FAILED And (I assume) it is best to run the make check under same userid, i.e. root, otherwise it is not testing what will actually be running. But anyway I tried running make check under non-root and the result was much worse - some indeterminate FAIL much earlier : Making check in lib-ssl-iostream make[2]: Entering directory '/mnt/julywext/wextmisc/fed30GBroot/ahcombld/dovecot-2.3.13/src/lib-ssl-iostream' make check-local make[3]: Entering directory '/mnt/julywext/wextmisc/fed30GBroot/ahcombld/dovecot-2.3.13/src/lib-ssl-iostream' for bin in test-iostream-ssl; do \ if ! /bin/sh ../../run-test.sh ../.. ./$bin; then exit 1; fi; \ done collect2: error: ld returned 213 exit status Failed to run: ./test-iostream-ssl This is because you have also compiled the source as root and fails to write to those directories. You need to build as non-root, too. No, for this attempt I ran all steps as non-root including make. May we re-visit the FAIL I reported originally ? Why does it FAIL when run under userid root? Because it's testing that it cannot read a file it has no permissions to read. root can read all files. You can see it does a chmod few lines before. You are in the wrong place -in the file - you are looking at line 61,my error occurred at line 54 - see error message earlier. It is make some kind of test for size of record being read. And, what code could I add to test-buffer-istream.c to make it print out the offending errno? Or, perhaps easier - is it safe to ignore this one FAIL? Hoping you will answer these ... Aki . Cheers, John Lumby Aki .
Re: Obtaining the IMAP GUID from a sieve script
On 18 Jan 2021, at 04:12, Jochen Bern wrote: > On 16.01.21 07:42, Ron Garret wrote: >> Because not every email has one. RFC5322 doesn’t require them. >> >> On Jan 15, 2021, at 6:30 PM, @lbutlr wrote: >>> Why not simply use the message-id? > (Also, you can legally have several e-mails with the same Message-ID in > your mailbox; e.g., someone addressed it to two aliases that both expand > to you, just to name one possibilty where *both* go through *sieve* as > well.) I delete duplicates before they are delivered to a mailbox. --
Re: Obtaining the IMAP GUID from a sieve script
On 16.01.21 07:42, Ron Garret wrote: > Because not every email has one. RFC5322 doesn’t require them. > > On Jan 15, 2021, at 6:30 PM, @lbutlr wrote: >> Why not simply use the message-id? (Also, you can legally have several e-mails with the same Message-ID in your mailbox; e.g., someone addressed it to two aliases that both expand to you, just to name one possibilty where *both* go through *sieve* as well.) Regards, -- Jochen Bern Systemingenieur Binect GmbH smime.p7s Description: S/MIME Cryptographic Signature
RE: mbox to pst advice
> > > On 2021-01-17 18:43, Odhiambo Washington wrote: > > > Personally, I would not bother looking for a script or even > asking > > anyone if they knew how to convert > > mbox2pst. > > there is always alternatives :-) > > aid4mail > > You have good experience with this (and are not in any way affiliated with this company/product)? Because there are so many of these 'our super-hero-mail-converter tools' are the best.
