Re: New Dovecot version? github repo without new updates?

[Aki Tuomi via dovecot]

> On 19/07/2023 01:22 EEST Jorge Bastos  wrote:
> 
> 
> Hi Guys,
> Just asking, i don't see a new dovecot version since last December, and the 
> girhub repo has no new requests/updates.
> Is there a new place for dovecot versions, or it's happening something?
> Thanks in advanced,
> Jorge,

Our github repo has been last updated 2 days ago, so not sure what you mean... 
We are working on new major release, so this usually causes a longer pause 
between releases.

Aki
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Replication going away?

[Gerald Galster]

>> While I understand it takes effort to maintain the replication plugin, this 
>> is especially problematic for small active/active high-availability 
>> deployments.
>> I guess there are lots of servers that use replication for just 50 or 100 
>> mailboxes. Cloudstorage (like S3) would be overkill for these.
> 
> Even without active/active, it's super useful for the simple
> active/backup configuration which I use on my personal mail server

This depends heavily on individual usage. Coming from an active/active
deployment it's a major step backwards though: usually two servers
are running independently in geographically dispersed datacenters.
High-availabilty is achieved by a simple DNS entry that returns two
ip addresses, one from each datacenter. Under normal circumstances
that gives you 50/50 loadbalancing without loadbalancers, without
additional components that can fail. In case one datacenter goes down,
and that happens to every datacenter at some time, the other datacenter
takes over - automatically, without any configuration changes.
Additionally mail user agents (Outlook, Thunderbird, ...) don't need
special configuration. If one ip address is unrechable they connect
to the other one obtained via DNS and users can quite seemlessly send
and receive email again. After the outage ceased and the other
datacenter is back online again, there is nothing to do.
No configuration changes, no error prone manual synchronization or
promoting passive to active - it just works and heals itself.
Being used to a carefree setup like that you don't want to go back.

Of course there are other possibilities like nfs, glusterfs, gfs2,
zfs snapshots, ceph, minio or dsync backup but they all have their own
drawbacks. For small mailservers that want high availability dsync
replication is quite the perfect solution.


> setup (one colo box, one home server) and a small company mail
> server; as such I'm pretty sad to see it go. Still, it is up
> to OX where they want to put their resources.

Well, it seems the dsync replication function is still there,
just the replication plugin that notifies what to replicate
is deprectated. Of course it's OX's decision, I'm just hoping
they were not aware how useful replication is in the before
mentioned scenario.

Moreover I'm quite sure this kind of small-scale replication
does not have any impact on customers upgrading to the new
cloud architecture. Big customers will go for cloud because
it scales way better and does not have replication induced
performance penalties and small customers probably can't
afford to upgrade because it's too pricey.


> I guess losing repl probably doesn't affect larger ISP type setups
> so much; it seems a bit more common to use shared storage (e.g.
> maildirs on an nfs appliance or similar) in those cases if they're
> actually running their own storage.
> 
>> Do you provide dovecot pro subscriptions for such small deployments?
> 
> Unless I misunderstood the message (and I don't think I did), repl
> was removed in pro too. (I don't expect that pro is available on my
> usual choice of OS anyway..).

As I understood it dsync is still working. Replication configured via
ssh is calling dsync under the hood, so if local storage and index/log
formats don't change for single deployments, it seems to be more of
a political decision. I know maintenance is not for free, that's why
I suggested to think about a dovecot small/medium business edition
with a more affordable price tag.

Best regards,
Gerald
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

New Dovecot version? github repo without new updates?

[Jorge Bastos]

Hi Guys,

Just asking, i don't see a new dovecot version since last December, and 
the girhub repo has no new requests/updates.

Is there a new place for dovecot versions, or it's happening something?

Thanks in advanced,
Jorge,___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Replication going away?

[Stuart Henderson]

On 2023-07-18, Gerald Galster  wrote:
> While I understand it takes effort to maintain the replication plugin, this 
> is especially problematic for small active/active high-availability 
> deployments.
> I guess there are lots of servers that use replication for just 50 or 100 
> mailboxes. Cloudstorage (like S3) would be overkill for these.

Even without active/active, it's super useful for the simple
active/backup configuration which I use on my personal mail server
setup (one colo box, one home server) and a small company mail
server; as such I'm pretty sad to see it go. Still, it is up
to OX where they want to put their resources.

I guess losing repl probably doesn't affect larger ISP type setups
so much; it seems a bit more common to use shared storage (e.g.
maildirs on an nfs appliance or similar) in those cases if they're
actually running their own storage.

> Do you provide dovecot pro subscriptions for such small deployments?

Unless I misunderstood the message (and I don't think I did), repl
was removed in pro too. (I don't expect that pro is available on my
usual choice of OS anyway..).


___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Fatal error in http-client.c when passdb is oauth2 and userdb is sql

[Евгений]

- все Test reply. Sorry. 18.07.2023, 15:37, "Aki Tuomi via dovecot" :  On 18/07/2023 15:10 EEST Евгений Бахтин  wrote:Dovecot version: 2.3.20  Dear Colleagues! passdb is set to oauth2 and userdb is set to sql. On attempt of authorization a fatal error is rising. The case reproducible 100%. The following suggested configuration files allow you to reproduce the case. Thank you! Can you send auth_debug=yes and debug=yes (in oauth2 config) logs?Aki___dovecot mailing list -- dovecot@dovecot.orgTo unsubscribe send an email to dovecot-le...@dovecot.org___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Options to track performance?

[Christian]

Hi there,
after upgrading my dovecot on a bookworm container, I now have a weird
delay when imap clients like Evolution connect the first time. 

Is there any performance logging configuration I could enable, to see
what dovecot is doing in which timing? I suspect some timeout or delay
somewhere, but unable to find it so far.

Kind regards
  Chris
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

-d ${user}@${nexthop} or -d ${recipient}?

[rixati6186]

What's the differency between these two lines;

1- flags=DORhu user=vmail:vmail argv=/usr/local/libexec/dovecot/dovecot-lda -f 
${sender} -r ${recipient} -d ${user}@${nexthop}
2- flags=DRhu user=vmail:vmail argv=/usr/local/libexec/dovecot/dovecot-lda -f 
${sender} -d ${recipient}

and which one should I use, based on a setup like; Dovecot, Postfix, 
Postfixadmin, ManageSieve and virtual users hosted in SQL?

Thanks in advance.
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: dovecot: lda(info@mydomain) - vacation action: discarding vacation response to mailinglist recipient?

[rixati6186]

Weird, but I think I solved it, I had in my postfix main.cf, a line like:

"header_checks = regexp:/etc/postfix/list_unsub_header" 

to have a header like: List-Unsubscribe: 
mailto:i...@mydomain.com?subject=unsubscribe, and when I removed that line, the 
auto-reply seems worked.

BUT, why's so? The sender is Gmail, List-Unsubscribe header is my own mailbox 
header, not the Gmail's.. Why it detects the SENDER e-mail account as 
"mailinglist"?
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Replication going away?

[Steven Varco]

> While I understand it takes effort to maintain the replication plugin, this 
> is especially problematic for small active/active high-availability 
> deployments.
> I guess there are lots of servers that use replication for just 50 or 100 
> mailboxes. Cloudstorage (like S3) would be overkill for these.
> 
> Do you provide dovecot pro subscriptions for such small deployments?

I’m running such a setup for 5-10 mailboxes.. :D


-- 
https://steven.varco.ch/ 

___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Replication going away?

[Gerald Galster]

>> Just to understand that correctly: I could setup a (cron) based process for 
>> doveadm sync, but no longer a setup like 
>> plugin { 
>>  mail_replica = tcp:$IMAP_REPLICA_SERVER:$IMAP_REPLICA_PORT 
>> } 
>> where the cron would lead to some delay and would have to check for 
>> concurrent jobs?
> 
> You can also have that too.
> 
> doveadm sync -d 
> 
> makes it use mail_replica setting.

@Aki:

Is it possible to monitor actions that would have triggered replication in 
dovecot < 2.4, e.g. parsing logs or a lua script to imitate the previous 
behaviour?


@Michael Slusarz:

While I understand it takes effort to maintain the replication plugin, this is 
especially problematic for small active/active high-availability deployments.
I guess there are lots of servers that use replication for just 50 or 100 
mailboxes. Cloudstorage (like S3) would be overkill for these.

Do you provide dovecot pro subscriptions for such small deployments?

The basic replication functionality with dsync seems to be available in future 
versions. Would you consider releasing a "dovecot smb" version for small/medium 
businesses that maintains the replication plugin (without director) for a 
yearly subscription fee? Otherwise those affected would have to look for 
alternatives. The recent release of rust-written Stalwart mailserver comes to 
mind, that bundles a lot of functionality (smtp, pop, imap, jmap, s3, sieve, 
dkim/arc/spf/dmarc, dane, mta-sas, ...): https://stalw.art/blog/

Best regards,
Gerald
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Fatal error in http-client.c when passdb is oauth2 and userdb is sql

[Eugene Su]

I think this mail related to the problem.

https://dovecot.org/mailman3/archives/list/dovecot@dovecot.org/message/X5PBA2RVC7ZDIUNMXJQZRFIDS7MZNGJH/
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Fatal error in http-client.c when passdb is oauth2 and userdb is sql

[Eugene Su]

Please, delete this wrong thread.
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Fatal error in http-client.c when passdb is oauth2 and userdb is sql

[mail]

Please, delete this wrong thread.
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Fatal error in http-client.c when passdb is oauth2 and userdb is sql

[mail]

Aki Tuomi wrote:
> > Can you send auth_debug=yes and debug=yes (in oauth2 config) logs?

This is with debug.
 
Jul 18 15:29:14 mail01 dovecot: master: Dovecot v2.3.20 (80a5ac675d) starting 
up for imap (core dumps disabled)
Jul 18 15:29:21 mail01 dovecot: auth: Debug: Loading modules from directory: 
/usr/lib/dovecot/modules/auth
Jul 18 15:29:21 mail01 dovecot: auth: Debug: Module loaded: 
/usr/lib/dovecot/modules/auth/lib20_auth_var_expand_crypt.so
Jul 18 15:29:21 mail01 dovecot: auth: Debug: Module loaded: 
/usr/lib/dovecot/modules/auth/libdriver_pgsql.so
Jul 18 15:29:21 mail01 dovecot: auth: Debug: sqlpool(pgsql): Creating new 
connection
Jul 18 15:29:21 mail01 dovecot: auth: Debug: Read auth token secret from 
/run/dovecot/auth-token-secret.dat
Jul 18 15:29:21 mail01 dovecot: auth: Debug: sqlpool(pgsql): Creating new 
connection
Jul 18 15:29:21 mail01 dovecot: auth: Panic: file http-client.c: line 646 
(http_client_context_close): assertion failed: (cctx->clients_list == NULL)
Jul 18 15:29:21 mail01 dovecot: auth: Error: Raw backtrace: 
/usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x3d) [0x7f8bd1f0c85d] -> 
/usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f8bd1f0c97e] -> 
/usr/lib/dovecot/libdovecot.so.0(+0x10091b) [0x7f8bd1f1991b] -> 
/usr/lib/dovecot/libdovecot.so.0(+0x1009b1) [0x7f8bd1f199b1] -> 
/usr/lib/dovecot/libdovecot.so.0(+0x54b7c) [0x7f8bd1e6db7c] -> 
/usr/lib/dovecot/libdovecot.so.0(+0x4d208) [0x7f8bd1e66208] -> 
/usr/lib/dovecot/libdovecot.so.0(io_loop_set_current+0x75) [0x7f8bd1f2eca5] -> 
/usr/lib/dovecot/modules/auth/libdriver_pgsql.so(+0x4686) [0x7f8bd1b43686] -> 
/usr/lib/dovecot/modules/auth/libdriver_pgsql.so(+0x49b7) [0x7f8bd1b439b7] -> 
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7f8bd1f2fe59] -> 
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x131) 
[0x7f8bd1f31481] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) 
[0x7f8bd1f2fefc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f8
 bd1f30080] -> /usr/lib/dovecot/modules/auth/libdriver_pgsql.so(+0x57ec) 
[0x7f8bd1b447ec] -> /usr/lib/dovecot/modules/auth/libdriver_pgsql.so(+0x405b) 
[0x7f8bd1b4305b] -> dovecot/auth(+0x4728b) [0x649266d1628b] -> 
dovecot/auth(+0x36ca1) [0x649266d05ca1] -> dovecot/auth(userdb_init+0x1a) 
[0x649266d0370a] -> dovecot/auth(auths_init+0xc9) [0x649266ce4ce9] -> 
dovecot/auth(main+0x398) [0x649266ce3b78] -> 
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f8bd1bc709b] -> 
dovecot/auth(_start+0x2a) [0x649266ce3d8a]
Jul 18 15:29:21 mail01 dovecot: master: Error: service(auth): command startup 
failed, throttling for 2.000 secs
Jul 18 15:29:21 mail01 dovecot: auth: Fatal: master: service(auth): child 12988 
killed with signal 6 (core dumps disabled - 
https://dovecot.org/bugreport.html#coredumps)
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Tons of imap-login processes despite client_limit very high

[D D]

Awesome, thank you for the information Aki. :) We'll set vsz_limit and keep 
service_count = 0, as service_count >= 1 is not an option for us due to the 
high memory consumption associated with having many imap-login processes 
running.

The new process_shutdown_filter option looks interesting, though if it 
shutdowns the process "after finishing the current connections", as the doc 
says, that may not resolve the issue in our case due to imap-login processes 
often having lasting connections (preventing the process to be recycled when 
service_count > 1 in the first place).

Thanks again, very helpful!
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Replication going away?

[Aki Tuomi via dovecot]

> On 18/07/2023 15:19 EEST i...@joergschulz.de wrote:
> 
>  
> Just to understand that correctly: I could setup a (cron) based process for 
> doveadm sync, but no longer a setup like 
> plugin { 
>   mail_replica = tcp:$IMAP_REPLICA_SERVER:$IMAP_REPLICA_PORT 
> } 
> where the cron would lead to some delay and would have to check for 
> concurrent jobs?

You can also have that too.

doveadm sync -d 

makes it use mail_replica setting.

Aki
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Fatal error in http-client.c when passdb is oauth2 and userdb is sql

[Евгений Бахтин]

Can you send auth_debug=yes and debug=yes (in oauth2 config) logs?This is with debug. Jul 18 15:29:14 mail01 dovecot: master: Dovecot v2.3.20 (80a5ac675d) starting up for imap (core dumps disabled)Jul 18 15:29:21 mail01 dovecot: auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/authJul 18 15:29:21 mail01 dovecot: auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/lib20_auth_var_expand_crypt.soJul 18 15:29:21 mail01 dovecot: auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_pgsql.soJul 18 15:29:21 mail01 dovecot: auth: Debug: sqlpool(pgsql): Creating new connectionJul 18 15:29:21 mail01 dovecot: auth: Debug: Read auth token secret from /run/dovecot/auth-token-secret.datJul 18 15:29:21 mail01 dovecot: auth: Debug: sqlpool(pgsql): Creating new connectionJul 18 15:29:21 mail01 dovecot: auth: Panic: file http-client.c: line 646 (http_client_context_close): assertion failed: (cctx->clients_list == NULL)Jul 18 15:29:21 mail01 dovecot: auth: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x3d) [0x7f8bd1f0c85d] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f8bd1f0c97e] -> /usr/lib/dovecot/libdovecot.so.0(+0x10091b) [0x7f8bd1f1991b] -> /usr/lib/dovecot/libdovecot.so.0(+0x1009b1) [0x7f8bd1f199b1] -> /usr/lib/dovecot/libdovecot.so.0(+0x54b7c) [0x7f8bd1e6db7c] -> /usr/lib/dovecot/libdovecot.so.0(+0x4d208) [0x7f8bd1e66208] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_set_current+0x75) [0x7f8bd1f2eca5] -> /usr/lib/dovecot/modules/auth/libdriver_pgsql.so(+0x4686) [0x7f8bd1b43686] -> /usr/lib/dovecot/modules/auth/libdriver_pgsql.so(+0x49b7) [0x7f8bd1b439b7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7f8bd1f2fe59] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x131) [0x7f8bd1f31481] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f8bd1f2fefc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f8bd1f30080] -> /usr/lib/dovecot/modules/auth/libdriver_pgsql.so(+0x57ec) [0x7f8bd1b447ec] -> /usr/lib/dovecot/modules/auth/libdriver_pgsql.so(+0x405b) [0x7f8bd1b4305b] -> dovecot/auth(+0x4728b) [0x649266d1628b] -> dovecot/auth(+0x36ca1) [0x649266d05ca1] -> dovecot/auth(userdb_init+0x1a) [0x649266d0370a] -> dovecot/auth(auths_init+0xc9) [0x649266ce4ce9] -> dovecot/auth(main+0x398) [0x649266ce3b78] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f8bd1bc709b] -> dovecot/auth(_start+0x2a) [0x649266ce3d8a]Jul 18 15:29:21 mail01 dovecot: master: Error: service(auth): command startup failed, throttling for 2.000 secsJul 18 15:29:21 mail01 dovecot: auth: Fatal: master: service(auth): child 12988 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) ___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

dovecot: lda(info@mydomain) - vacation action: discarding vacation response to mailinglist recipient?

[rixati6186]

Hello,

I'm trying to set auto reply for my email account, through Roundcube and 
ManageSieve plugin (the plugin itself and its regular filtering works fine), 
however the auto-reply doesn't arrive to the sender's inbox, when I tested it 
with sending an email to myself/my server, from my Gmail account, my maillog 
has a line, stating;

Jul 18 15:28:06 mailsrv dovecot: 
lda(i...@mywebsite.com)<44812>: sieve: 
msgid=: 
vacation action: discarding vacation response to mailinglist recipient 


Any idea would be much appreciated, thank you!
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Fatal error in http-client.c when passdb is oauth2 and userdb is sql

[Евгений Бахтин]

Can you send auth_debug=yes and debug=yes (in oauth2 config) logs?This is with debug. Jul 18 15:29:14 mail01 dovecot: master: Dovecot v2.3.20 (80a5ac675d) starting up for imap (core dumps disabled)Jul 18 15:29:21 mail01 dovecot: auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/authJul 18 15:29:21 mail01 dovecot: auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/lib20_auth_var_expand_crypt.soJul 18 15:29:21 mail01 dovecot: auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_pgsql.soJul 18 15:29:21 mail01 dovecot: auth: Debug: sqlpool(pgsql): Creating new connectionJul 18 15:29:21 mail01 dovecot: auth: Debug: Read auth token secret from /run/dovecot/auth-token-secret.datJul 18 15:29:21 mail01 dovecot: auth: Debug: sqlpool(pgsql): Creating new connectionJul 18 15:29:21 mail01 dovecot: auth: Panic: file http-client.c: line 646 (http_client_context_close): assertion failed: (cctx->clients_list == NULL)Jul 18 15:29:21 mail01 dovecot: auth: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x3d) [0x7f8bd1f0c85d] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f8bd1f0c97e] -> /usr/lib/dovecot/libdovecot.so.0(+0x10091b) [0x7f8bd1f1991b] -> /usr/lib/dovecot/libdovecot.so.0(+0x1009b1) [0x7f8bd1f199b1] -> /usr/lib/dovecot/libdovecot.so.0(+0x54b7c) [0x7f8bd1e6db7c] -> /usr/lib/dovecot/libdovecot.so.0(+0x4d208) [0x7f8bd1e66208] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_set_current+0x75) [0x7f8bd1f2eca5] -> /usr/lib/dovecot/modules/auth/libdriver_pgsql.so(+0x4686) [0x7f8bd1b43686] -> /usr/lib/dovecot/modules/auth/libdriver_pgsql.so(+0x49b7) [0x7f8bd1b439b7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7f8bd1f2fe59] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x131) [0x7f8bd1f31481] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f8bd1f2fefc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f8bd1f30080] -> /usr/lib/dovecot/modules/auth/libdriver_pgsql.so(+0x57ec) [0x7f8bd1b447ec] -> /usr/lib/dovecot/modules/auth/libdriver_pgsql.so(+0x405b) [0x7f8bd1b4305b] -> dovecot/auth(+0x4728b) [0x649266d1628b] -> dovecot/auth(+0x36ca1) [0x649266d05ca1] -> dovecot/auth(userdb_init+0x1a) [0x649266d0370a] -> dovecot/auth(auths_init+0xc9) [0x649266ce4ce9] -> dovecot/auth(main+0x398) [0x649266ce3b78] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f8bd1bc709b] -> dovecot/auth(_start+0x2a) [0x649266ce3d8a]Jul 18 15:29:21 mail01 dovecot: master: Error: service(auth): command startup failed, throttling for 2.000 secsJul 18 15:29:21 mail01 dovecot: auth: Fatal: master: service(auth): child 12988 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) ___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Replication going away?

[info]

Just to understand that correctly: I could setup a (cron) based process for 
doveadm sync, but no longer a setup like 
plugin { 
  mail_replica = tcp:$IMAP_REPLICA_SERVER:$IMAP_REPLICA_PORT 
} 
where the cron would lead to some delay and would have to check for concurrent 
jobs?
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Fatal error in http-client.c when passdb is oauth2 and userdb is sql

[Aki Tuomi via dovecot]

> On 18/07/2023 15:10 EEST Евгений Бахтин  wrote:
> 
> 
> 
> Dovecot version: 2.3.20
> 
> Dear Colleagues!
> passdb is set to oauth2 and userdb is set to sql. On attempt of authorization 
> a fatal error is rising. The case reproducible 100%. The following suggested 
> configuration files allow you to reproduce the case.
> Thank you!
> 

Can you send auth_debug=yes and debug=yes (in oauth2 config) logs?

Aki
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Fatal error in http-client.c when passdb is oauth2 and userdb is sql

[Евгений Бахтин]

 Dovecot version: 2.3.20 Dear Colleagues!passdb is set to oauth2 and userdb is set to sql. On attempt of authorization a fatal error is rising. The case reproducible 100%. The following suggested configuration files allow you to reproduce the case.Thank you! dovecot.conf```protocols = imapauth_mechanisms = plain xoauth2 oauthbearerdisable_plaintext_auth = no passdb {  driver = oauth2  mechanisms = xoauth2 oauthbearer  args = /etc/dovecot/oauth2.conf.ext} userdb {  driver = sql  args = /etc/dovecot/userdb.conf.ext} mail_home=/home/test/%Lumail_location=sdbox:~/Mail namespace {  inbox = yes  separator = /}``` /etc/dovecot/oauth2.conf.ext```introspection_mode = postintrospection_url = https://admin:admin@authserver:9443/oauth2/introspectusername_attribute = usernameactive_attribute = activeactive_value = truetls_allow_invalid_cert = yes``` /etc/dovecot/userdb.conf.ext```driver = pgsqlconnect = host=dbhost port=5432 dbname=test user=test password=testuser_query = SELECT 'something' AS something``` Error log:```Jul 18 13:04:51 mail01 dovecot: auth: Panic: file http-client.c: line 646 (http_client_context_close): assertion failed: (cctx->clients_list == NULL)Jul 18 13:04:51 mail01 dovecot: auth: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x3d) [0x733f846a085d] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x733f846a097e] -> /usr/lib/dovecot/libdovecot.so.0(+0x10091b) [0x733f846ad91b] -> /usr/lib/dovecot/libdovecot.so.0(+0x1009b1) [0x733f846ad9b1] -> /usr/lib/dovecot/libdovecot.so.0(+0x54b7c) [0x733f84601b7c] -> /usr/lib/dovecot/libdovecot.so.0(+0x4d208) [0x733f845fa208] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_set_current+0x75) [0x733f846c2ca5] -> /usr/lib/dovecot/modules/auth/libdriver_pgsql.so(+0x4686) [0x733f842d7686] -> /usr/lib/dovecot/modules/auth/libdriver_pgsql.so(+0x49b7) [0x733f842d79b7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x733f846c3e59] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x131) [0x733f846c5481] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x733f846c3efc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x733f846c4080] -> /usr/lib/dovecot/modules/auth/libdriver_pgsql.so(+0x57ec) [0x733f842d87ec] -> /usr/lib/dovecot/modules/auth/libdriver_pgsql.so(+0x405b) [0x733f842d705b] -> dovecot/auth(+0x4728b) [0x5a35cac1128b] -> dovecot/auth(+0x36ca1) [0x5a35cac00ca1] -> dovecot/auth(userdb_init+0x1a) [0x5a35cabfe70a] -> dovecot/auth(auths_init+0xc9) [0x5a35cabdfce9] -> dovecot/auth(main+0x398) [0x5a35cabdeb78] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x733f8435b09b] -> dovecot/auth(_start+0x2a) [0x5a35cabded8a]Jul 18 13:04:51 mail01 dovecot: master: Error: service(auth): command startup failed, throttling for 32.000 secsJul 18 13:04:51 mail01 dovecot: auth: Fatal: master: service(auth): child 32434 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps)```___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: cram-md5, sasl and lua (inheriting technical debt)

[Aki Tuomi via dovecot]

> On 18/07/2023 13:59 EEST tk...@tunenet.dk wrote:
> 
>  
> Hello,
> 
> I am exploring the posibility of migrating an exsisting setup to 
> postfix+dovecot.
> The issue being that many clients are currently configured for cram-md5 
> authentication.
> I am fully aware that this is a really, really, really bad idea, but re 
> configuring all clients at once is not feasible with limited end user support 
> resources.
> I have a setup running with LUA for the passdb, and everything works with 
> PLAIN login.
> 
> To keep compatibility with the PLAIN login mechanism i have tried to store 
> {PLAIN} passwords in the DB, since proper secure password storage 
> is incompatible with CRAM-MD5. 
> 
> My issue is that the LUA function auth_password_verify(req, pass) not even 
> seems to be called for cram-md5 logins.
> Reading through the documentation also seems to indicate that the callenge is 
> not passed to the LUA function making it impossible to compute the hash 
> in LUA or the function  req.password_verify(req, row.password, pass).
> 
> Is my assumption correct that cram-md5 can not work with a LUA script ?
> 
> Kind regards,
> Peter K.


Hi!

CRAM-MD5 only works if you return a credential, so you need to use 
auth_passdb_lookup to return it with {PLAIN} prefix. It's not possible to use 
Lua script to calculate it yourself.

There is also CRAM-MD5 password scheme, but calculating that is slightly 
difficult, so i'd just return the PLAIN password from auth_passdb_lookup 
instead.

Aki

Aki
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

cram-md5, sasl and lua (inheriting technical debt)

[tknb8]

Hello,

I am exploring the posibility of migrating an exsisting setup to 
postfix+dovecot.
The issue being that many clients are currently configured for cram-md5 
authentication.
I am fully aware that this is a really, really, really bad idea, but re 
configuring all clients at once is not feasible with limited end user support 
resources.
I have a setup running with LUA for the passdb, and everything works with PLAIN 
login.

To keep compatibility with the PLAIN login mechanism i have tried to store 
{PLAIN} passwords in the DB, since proper secure password storage 
is incompatible with CRAM-MD5. 

My issue is that the LUA function auth_password_verify(req, pass) not even 
seems to be called for cram-md5 logins.
Reading through the documentation also seems to indicate that the callenge is 
not passed to the LUA function making it impossible to compute the hash 
in LUA or the function  req.password_verify(req, row.password, pass).

Is my assumption correct that cram-md5 can not work with a LUA script ?

Kind regards,
Peter K.
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Tons of imap-login processes despite client_limit very high

[D D]

It seems to us that the ideal solution would be that once service_count is 
reached, a new process is spawned and the remaining connections are moved to 
that new process so that the old one can die quickly. But I suspect that's not 
a simple change to do.
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Tons of imap-login processes despite client_limit very high

[Aki Tuomi via dovecot]

> On 18/07/2023 12:11 EEST D D  wrote:
> 
>  
> After further testing we realized that it was due to service_count = 100. We 
> suspect that when the service count is reached, a new process is spawned, 
> explaining the large number of imap-login processes.
> 
> With service_count = 0 we stick with only 4 processes (process_min_avail). 
> However, we're concerned with having these processes run indefinitely in case 
> of memory leaks.
> 
> Any insights on that?

It's unlikely that they leak memory, but in case that worries, you can do two 
things:

configure vsz_limit for the service(s), or configure service_count for the 
service(s). The first one will slay the login processes, disconnecting 
everyone, if there is a leak, the second one will recycle the process at some 
point.

Since 2.3.19, you can also use 
https://doc.dovecot.org/settings/core/#core_setting-process_shutdown_filter to 
shutdown the process if it's memory consumption goes too high.

Aki
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Tons of imap-login processes despite client_limit very high

[D D]

Thank you Joseph and Aki!

You got it right, the issue was indeed with this service_count=100. With 
service_count=0 it works as intended (only 4 imap-login processes), though now 
we're concerned about possible memory leaks with this config.

What you described Jospeh 
(https://www.mail-archive.com/dovecot%40dovecot.org/msg85850.html) is what 
we've observed as well. In addition, service_count > 1 + high process_limit 
consumes much more mermory because of all those imap-login processes handling 
juste a few lasting connections. We're consuming about 4x less memory with 
service_count=0, it's day and night.

There's something somewhat close documented on 
https://doc.dovecot.org/configuration_manual/service_configuration/#service-limits:

"Otherwise when the service_count is beginning to be reached, the total number 
of available connections will shrink. With very bad luck that could mean that 
all the processes are simply waiting for the existing connections to die away 
before the process can die and a new one can be created. "

Though not the focus of the discussion, it does say that processes don't die 
until their connections have died.

It could perhaps benefit from mentioning a few more things like:
- service_count = 0 has no protection against potential memory leaks.
- service_count > 1 + high process_limit coud produce many processes since 
these don't actually die until their connections have all died, which consumes 
isgnificantly more memory.

One workaround to the lack of memory leaks protection could be to set 
process_limit close to process_min_avail while keeping service_count > 1. But 
we'd end up in the risky case described in the docs: 

"With very bad luck that could mean that all the processes are simply waiting 
for the existing connections to die away before the process can die and a new 
one can be created."

So for now we don't see any way out of service_count = 0 and it's associated 
memory leak risk.
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Tons of imap-login processes despite client_limit very high

[D D]

After further testing we realized that it was due to service_count = 100. We 
suspect that when the service count is reached, a new process is spawned, 
explaining the large number of imap-login processes.

With service_count = 0 we stick with only 4 processes (process_min_avail). 
However, we're concerned with having these processes run indefinitely in case 
of memory leaks.

Any insights on that?
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Tons of imap-login processes despite client_limit very high

[Aki Tuomi via dovecot]

> On 18/07/2023 09:18 EEST Joseph Tam  wrote:
> 
>  
> https://www.mail-archive.com/dovecot%40dovecot.org/msg85850.html
> 
> 
> From: D D 
> 
> > We're seeing a ton of imap-login processes running even when using high 
> > performance mode 
> > (https://doc.dovecot.org/admin_manual/login_processes/#high-performance-mode).
> >  According to the docs:
> >
> > "process_min_avail should be set to be at least the number of CPU cores in 
> > the system, so that all of them will be used. Otherwise new processes are 
> > created only once an existing one’s connection count reaches client_limit"
> >
> > We have process_min_avail=4, client_limit=0 and 
> > default_client_limit=20. So we'd expect seeing only 4 imap-login 
> > processes serving a ton of connections each. Yet, we see thousands of 
> > imap-login processes (more than half of all the imap processes):
> > ...
> >
> > Is having so many imap-login processes normal with our config? Did we 
> > misunderstand the docs or is there something wrong here?
> >
> >
> > default_client_limit = 1048576
> > default_process_limit = 20
> >
> > service imap-login {
> >   # client_limit = 0 # default is 0
> >   # process_limit = 0 # default is 0
> >   service_count = 100
> 
> This service limit might be your culprit.
> 
> I wrote about the strange interaction between service_count and
> process_limit here:
> 
> https://www.mail-archive.com/dovecot%40dovecot.org/msg85850.html
> 
> This gotcha should really be documented.
> 
> Joseph Tam 

Did you check the 
https://doc.dovecot.org/configuration_manual/service_configuration/#service-limits
 to see if it is documented? A pull request would be appreciated if it's still 
wrong.

Aki
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Tons of imap-login processes despite client_limit very high

[Joseph Tam]

https://www.mail-archive.com/dovecot%40dovecot.org/msg85850.html


From: D D 

> We're seeing a ton of imap-login processes running even when using high 
> performance mode 
> (https://doc.dovecot.org/admin_manual/login_processes/#high-performance-mode).
>  According to the docs:
>
> "process_min_avail should be set to be at least the number of CPU cores in 
> the system, so that all of them will be used. Otherwise new processes are 
> created only once an existing one’s connection count reaches client_limit"
>
> We have process_min_avail=4, client_limit=0 and default_client_limit=20. 
> So we'd expect seeing only 4 imap-login processes serving a ton of 
> connections each. Yet, we see thousands of imap-login processes (more than 
> half of all the imap processes):
> ...
>
> Is having so many imap-login processes normal with our config? Did we 
> misunderstand the docs or is there something wrong here?
>
>
> default_client_limit = 1048576
> default_process_limit = 20
>
> service imap-login {
>   # client_limit = 0 # default is 0
>   # process_limit = 0 # default is 0
>   service_count = 100

This service limit might be your culprit.

I wrote about the strange interaction between service_count and
process_limit here:

https://www.mail-archive.com/dovecot%40dovecot.org/msg85850.html

This gotcha should really be documented.

Joseph Tam 
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org