Doveadm backup stateful synchronization trouble

[dovecot]

Hi,

The non-blocking error is:
doveadm(user1s@domain): Error: Saved sync state is invalid, falling back to 
full sync: Invalid base64 data

This error occurs every time on my test machine (no traffic, only a few test 
messages). That error sometime also occurs on my production system, but not 
each time. 

The command I used for the first time is : doveadm backup -A -s "" -d > 
state.txt
After that there is a daily calls (as vmail user, my config only handle virtual 
users) to : 
doveadm backup -A -s "$( state.txt.tmp && mv state.txt.tmp 
state.txt

doveconf -n output is available at 
https://www.f-hamelin.fr/pub/doveadm-backup-doveconf.txt

Any help will be welcomed 
Best regards,
Franck 

Re: OAUTH2 local validation

[Aki Tuomi]

 
 
  
    
   
   
   
On 16/11/2022 17:31 EET Felix Auringer  wrote:

   
 

   
 

   
Hello,

   
 

   


   
describes how to set up local validation for OAUTH2 with dovecot. This

   
works fine as long as the keys are not rotated. In my experience, it is

   
common for a client to try to validate a token with the cached key and

   
update the cached keys when the local validation fails (e.g. via the

   
/auth/realms//protocol/openid-connect/certs endpoint in

   
Keycloak). This way, the client does not need to fetch new keys

   
periodically but only when the old ones expired. If I understand it

   
correctly, Dovecot reads the keys from a defined path but does not care

   
how to update them. Did you have a nice way in mind how the keys should

   
be refreshed when deciding not to do that in dovecot? Are you planning

   
to add automatic refreshing of local validation keys to dovecot?

   
I am running dovecot in Docker and one way would be a cron job on the

   
host that fetches new keys and updates the files inside the container

   
via docker exec. It would work but it's not really a nice solution

   
because from outside dovecot, the information whether the old keys are

   
no longer valid, is not available and the whole process needs to run

   
periodically (and thus way more often than actually necessary).

   
 

   
Best regards,

   
Felix Auringer

   
---

   
Gesellschaft für interkulturelles

   
Zusammenleben gGmbH (GIZ)

   
Felix Auringer

   
IT

   
Reformationsplatz 2

   
13597 Berlin

   
 

   
Tel: 030/513 0100 00; Fax: 030/513 0100 09

   
www.giz.berlin; felix.auringer@giz.berlin

   
 

   
Amtsgericht Charlottenburg HRB 200872 B

   
Geschäftsführerin: Dr. Britta Marschke

   
  
    
   
  
   You could also mount a key volume and only update that with cron. Alternatively you need a dict protocol based solution.
   
   
   ---
Aki Tuomi 
  
 

OAUTH2 local validation

[Felix Auringer]

Hello,

 
describes how to set up local validation for OAUTH2 with dovecot. This 
works fine as long as the keys are not rotated. In my experience, it is 
common for a client to try to validate a token with the cached key and 
update the cached keys when the local validation fails (e.g. via the 
/auth/realms//protocol/openid-connect/certs endpoint in 
Keycloak). This way, the client does not need to fetch new keys 
periodically but only when the old ones expired. If I understand it 
correctly, Dovecot reads the keys from a defined path but does not care 
how to update them. Did you have a nice way in mind how the keys should 
be refreshed when deciding not to do that in dovecot? Are you planning 
to add automatic refreshing of local validation keys to dovecot?
I am running dovecot in Docker and one way would be a cron job on the 
host that fetches new keys and updates the files inside the container 
via docker exec. It would work but it's not really a nice solution 
because from outside dovecot, the information whether the old keys are 
no longer valid, is not available and the whole process needs to run 
periodically (and thus way more often than actually necessary).


Best regards,
Felix Auringer
---
Gesellschaft für interkulturelles
Zusammenleben gGmbH (GIZ)
Felix Auringer
IT
Reformationsplatz 2
13597 Berlin

Tel: 030/513 0100 00; Fax: 030/513 0100 09 
www.giz.berlin; felix.auringer@giz.berlin


Amtsgericht Charlottenburg HRB 200872 B
Geschäftsführerin: Dr. Britta Marschke

Re: bug: ARGON2 hash selection incompatible with LDAP

[Krisztián Szegi]

"Krisztián Szegi" k@mszk.eu – 15 November 2022 20:18
> "Michael Ströder" mich...@stroeder.com – 15 November 2022 15:00
> > On 11/15/22 13:45, Krisztián Szegi wrote:
> >> I'd like to report that non-binding auth to (Open)LDAP doesn't work
> >> if the latter hashes passwords with ARGON2.
> > Could you please elaborate why using LDAP bind is a problem for you?
> >  
> > Ciao, Michael.
> >  
> > 
>  
> Fair enough question!
>  
> I cannot specify bind_dn template due to mismatched mail addresses and user 
> DNs, and I thought that that would be suboptimal due to re-binding.
> I am a bit confused about how to optimize LDAP lookups now (static files not 
> option :), re-reading the docs it just made me question more things
> - auth_bind_dn cannot be given in my case, as a fixed starting point
> - auth_bind adds a temporary binding (using pass_filter)
> - can I use userdb prefetch? Docs say I cannot if I use bind with template, 
> but I am not using the latter. So the search for the user's dn during auth IS 
> the passdb lookup?
> - assuming I am correct, I should give back stuff with passdb lookup: or do I?
>   - Must I give back userid an guid? 10-mail.conf has "vmail" for both, as 
> mail accounts don't have UNIX ones linked to them...
>   - same for home? There is no default I've given until userdb lookup. Just 
> specify a global mail_home with variables, and get on with life?
>   -if I should give back one, should I pass it with default_fields = 
> userdb_home (currently I specify it under default_fields:home in userdb 
> lookup as LDAP doesn't override home).
> 
> The docs are confusing around userdb. The main thing what is not clear that 
> they CAN override fields on a per-user basis, but must they provide them for 
> non-extra fields, when there are global settings for those?
> 
> Thanks!
> 
> BTW, thanks for the great software all of you.
> Michael, I've come across some of your work, you have my respect!
> 

On second though:
I switched to auth_bind = yes, (I'll start a new thread on optimizing passdb 
and userdb, because the scattered documentation has some holes in it I think) 
but my patch is still needed - if I understand correctly - because I use 
postfix with dovecot as LMTP and auth backend.
 

Re: access dict in lua auth script

[Aki Tuomi]

> On 16/11/2022 13:46 EET Tobias Florek  wrote:
> 
>  
> Hi!
> 
> I want to access a configured dictionary in a lua authentication script.
> 
> Unfortunately the documentation 
>  does mention a dict object 
> but no way to get one.
> 
> I also did not find any lua tests in dovecot's source code.  Is using 
> the built-in dict not supported?
> 
> Cheers,
>   Tobias Florek

Hi!

It's not yet possible to get dict objects on demand in Lua scripts.

Aki

access dict in lua auth script

[Tobias Florek]

Hi!

I want to access a configured dictionary in a lua authentication script.

Unfortunately the documentation 
 does mention a dict object 
but no way to get one.


I also did not find any lua tests in dovecot's source code.  Is using 
the built-in dict not supported?


Cheers,
 Tobias Florek

Re: bug: ARGON2 hash selection incompatible with LDAP

[Aki Tuomi]

> On 15/11/2022 21:17 EET Krisztián Szegi  wrote:
> 
>  
> "Michael Ströder" mich...@stroeder.com – 15 November 2022 15:00
> > On 11/15/22 13:45, Krisztián Szegi wrote:
> >> I'd like to report that non-binding auth to (Open)LDAP doesn't work
> >> if the latter hashes passwords with ARGON2.
> > Could you please elaborate why using LDAP bind is a problem for you?
> >  
> > Ciao, Michael.
> >  
> > 
>  
> Fair enough question!
>  
> I cannot specify bind_dn template due to mismatched mail addresses and user 
> DNs, and I thought that that would be suboptimal due to re-binding.
> I am a bit confused about how to optimize LDAP lookups now (static files not 
> option :), re-reading the docs it just made me question more things
> - auth_bind_dn cannot be given in my case, as a fixed starting point
> - auth_bind adds a temporary binding (using pass_filter)
> - can I use userdb prefetch? Docs say I cannot if I use bind with template, 
> but I am not using the latter. So the search for the user's dn during auth IS 
> the passdb lookup?

prefetch userdb does not in fact fetch anything. It mainly looks if passdb 
result contains userdb_* field(s) and shortcuts the lookup there.

> - assuming I am correct, I should give back stuff with passdb lookup: or do I?
>   - Must I give back userid an guid? 10-mail.conf has "vmail" for both, as 
> mail accounts don't have UNIX ones linked to them...
>   - same for home? There is no default I've given until userdb lookup. Just 
> specify a global mail_home with variables, and get on with life?
>   -if I should give back one, should I pass it with default_fields = 
> userdb_home (currently I specify it under default_fields:home in userdb 
> lookup as LDAP doesn't override home).
> 
> The docs are confusing around userdb. The main thing what is not clear that 
> they CAN override fields on a per-user basis, but must they provide them for 
> non-extra fields, when there are global settings for those?

mail_home, mail_gid, mail_uid etc. can be just templated out in config file, 
providing them in userdb reply is optional. 

If you don't need anything special for the userdb, it might already be enough 
to just have ldap passdb.

> 
> Thanks!
> 
> BTW, thanks for the great software all of you.
> Michael, I've come across some of your work, you have my respect!

Aki