Re: Orphaned processes after doveadm log reopen
On 8.7.2019 14.54, Tom Sommer via dovecot wrote: > > On 2019-07-08 13:36, Tom Sommer via dovecot wrote: >> I rotate logs every night on my Director, running "doveadm log reopen" > > It happens on "/etc/init.d/dovecot restart", not "doveadm log reopen" > - sorry > > So it happens when you run killproc on dovecot > > --- > Tom Do you happen to have shutdown_clients=no ? Aki
Re: JMAP support
No ETA yet. Aki On 7.7.2019 1.12, Martynas Bendorius via dovecot wrote: > Hello, > > Is there any ETA set for JMAP support? > > Thank you! > >>> On 11/27/2016 5:28 AM, Aki Tuomi wrote: >>> Hi! >>> We are working on including JMAP support to Dovecot. At this moment I >>> cannot give any promise for exact version, but hopefully it will be part of >>> v2.3 >>> >>> Aki Tuomi >>> Dovecot Oy > -- > Best regards, > Martynas Bendorius > >
Re: Dovecot 2.3.6 on Solaris10: build issues, segfaults
On 9.7.2019 3.02, Joseph Tam via dovecot wrote:
>
> Hopefully, there is some fix for issue 3 which is beyond my
> skill to fix.
>
> Issue 1) Need recent gcc version
>
> Building Dovecot versions <=2.2.x using gcc 3.4.4 worked,
> but this gcc version fails to build 2.3.x properly: symptoms
> include compile failures and executable crashes that depended
> on the amount of optimization used, which is usually a sign of
> compiler bugs. (It could also be issue 3 in disguise.)
>
> Either way, I updated to gcc 9.1.0.
>
> Issue 2) Cannot build with --enable-hardening
>
> Using gcc 9.1.0, "configure" step fails because fd passing was
> broken, but the real problem was a compilation failure when
> "--enable-hardening" is used. Demonstration:
>
> # echo 'int main(){char a[1]; strcpy(a,a);} ' | gcc -w
> -fstack-protector-strong -x c -
> Undefined first referenced
> symbol in file
> __stack_chk_guard /var/tmp//cc12L9zV.o (symbol
> scope specifies local binding)
> ld: fatal: symbol referencing errors. No output written to a.out
> collect2: error: ld returned 1 exit status
>
> I'm not sure if this is a Solaris10 fumble, but configuring
> "--disable-hardening" removes the "-fstack-protector-strong"
> compiler option, which resolves this issue.
>
> Issue 3) dovecot/doveconf segfaults on startup
>
> It crashes here while processing dovecot.conf, as does "doveconf"
>
>
Just to be sure ...
You did gmake clean; ./configure ; gmake
We'll look at the 3rd issue, the 2nd issue looks really odd, and I'm
verging on compiler bug there. As for 1st, isn't gcc 3 rather old?
Aki
Dovecot 2.3.6 on Solaris10: build issues, segfaults
Hopefully, there is some fix for issue 3 which is beyond my
skill to fix.
Issue 1) Need recent gcc version
Building Dovecot versions <=2.2.x using gcc 3.4.4 worked,
but this gcc version fails to build 2.3.x properly: symptoms
include compile failures and executable crashes that depended
on the amount of optimization used, which is usually a sign of
compiler bugs. (It could also be issue 3 in disguise.)
Either way, I updated to gcc 9.1.0.
Issue 2) Cannot build with --enable-hardening
Using gcc 9.1.0, "configure" step fails because fd passing was
broken, but the real problem was a compilation failure when
"--enable-hardening" is used. Demonstration:
# echo 'int main(){char a[1]; strcpy(a,a);} ' | gcc -w
-fstack-protector-strong -x c -
Undefined first referenced
symbol in file
__stack_chk_guard /var/tmp//cc12L9zV.o (symbol scope
specifies local binding)
ld: fatal: symbol referencing errors. No output written to a.out
collect2: error: ld returned 1 exit status
I'm not sure if this is a Solaris10 fumble, but configuring
"--disable-hardening" removes the "-fstack-protector-strong"
compiler option, which resolves this issue.
Issue 3) dovecot/doveconf segfaults on startup
It crashes here while processing dovecot.conf, as does "doveconf"
(settings-parser.c:1519 in setting_copy())
*dest_size = *src_size;
It appears *src_size is not an 8-byte address aligned (0x5597c).
It inherits this value from the calling routine as the sum of
"set" (8-byte aligned) + "def->offset"=20 => misaligned address.
(settings-parser.c:1597 in settings_dup_full())
src = CONST_PTR_OFFSET(set, def->offset);
(gdb) p set
$2 = (const void *) 0x55968
(gdb) p *def
$3 = {type = SET_SIZE, key = 0x2d548
"submission_max_mail_size", offset = 20, list_info = 0x0}
(gdb) bt full
#0 0xff190690 in setting_copy (type=SET_SIZE, src=0x5798c,
dest=0xf7ed4, pool=0xf29a0,
keep_values=false) at settings-parser.c:1519
src_size = 0x5798c
dest_size = 0xf7ed4
__func__ = "setting_copy"
#1 0xff190bc4 in settings_dup_full (info=0x2d9ac
, set=0x57978,
pool=0xf29a0, keep_values=false) at settings-parser.c:1600
def = 0x2d7d4
src = 0x5798c
dest_set = 0xf7ec0
dest = 0xf7ed4
children = 0x2c
i = 1105688
count = 4279837964
#2 0xff192724 in settings_parser_dup (old_ctx=0x9ec08,
new_pool=0xf29a0) at settings-parser.c:1946
new_ctx = 0x10def0
iter = 0x0
links = {_table = 0xa23c0, _key = 0xa23c0, _keyp = 0xa23c0,
_const_key = 0xa23c0,
_value = 0xa23c0, _valuep = 0xa23c0}
new_link = 0x10b488
value = 0x0
key = 0x0
i = 0
parser_pool = 0x10ded8
keep_values = false
__func__ = "settings_parser_dup"
#3 0x0001fc0c in config_filter_parsers_get (ctx=0x657a8, pool=0xf29a0,
modules=0x0,
filter=0x583e8, parsers_r=0xffbff9c4, output_r=0xffbff9bc,
error_r=0xffbffa44)
at config-filter.c:372
src = 0xf29d8
dest = 0xf2a48
error = 0x0
error_p = 0x52cfc
i = 26
count = 33
__func__ = "config_filter_parsers_get"
#4 0x00021be4 in config_all_parsers_check (ctx=0xffbffac8,
new_filter=0x657a8, error_r=0xffbffa44)
at config-parser.c:441
parsers = 0x58180
tmp_parsers = 0x4
output = {specific_services = 0xf2a20, service_uses_local =
false,
service_uses_remote = false, used_local = false, used_remote
= false,
permission_denied = false}
i = 0
count = 3
ssl_set = 0x4
global_ssl_set = 0x32188 ""
tmp_pool = 0xf29a0
ssl_warned = false
ret = 0
__func__ = "config_all_parsers_check"
#5 0x00022ec8 in config_parse_finish (ctx=0xffbffac8,
error_r=0xffbffb98) at config-parser.c:747
new_filter = 0x657a8
error = 0x33c22 "/user=%s"
ret = 0
#6 0x00024230 in config_parse_file (path=0x50468
"/local/dovecot/etc/dovecot/dovecot.conf",
expand_values=false, modules=0x0, error_r=0xffbffb98) at
config-parser.c:1064
root = {prev = 0x0, input = 0x0, path = 0x50468
"/local/dovecot/etc/d
Re: Applying Dovecot for a large / deep folder-hierarchy archive - BUG REPORTS!
Hello Aki, Thanks for looking into these. I will as requested attempt the relevant procedures under Dovecot 2.3.6. To make the test fair, I will need to fork the relevant production groupware stack (which is now stable and in operation, with our enterprise (email) data successfully migrated from Microsoft Exchange) to a new staging server; given that the current staging server is now of a materially different configuration to the production server (where the most controlled observations of these bugs were made). Kindly give me some time, as I have an urgent internal openstack deployment project to kick-off, that has been delayed by an overrun of this internal groupware stack deployment project. Best regards, Arnold Opio Oree Chief Executive Officer Parallax Digital Technologies arnoldo...@parallaxdt.com http://www.parallaxdt.com tel : +44 (0) 333 577 8587 fax : +44 (0) 20 8711 2477 Parallax Digital Technologies is a trading name of Parallax Global Limited. U.K. Co. No. 08836288 The contents of this e-mail are confidential. If you are not the intended recipient you are to delete this e-mail immediately, disregard its contents and disclose them to no other persons. -Original Message- From: Aki Tuomi via dovecot Reply-To: Aki Tuomi To: arnoldo...@parallaxict.com, Arnold Opio Oree < arnold.o...@parallaxict.com>, Arnold Opio Oree via dovecot < dovecot@dovecot.org> Cc: debian-rele...@lists.debian.org Subject: Re: Applying Dovecot for a large / deep folder-hierarchy archive - BUG REPORTS! Date: Mon, 8 Jul 2019 08:48:08 +0300 (EEST) Hi! Thank you for reporting these. We will look into them. In the mean time, can you see if any of these are fixed in 2.3.6? Aki > On 07/07/2019 18:12 Arnold Opio Oree via dovecot > wrote: > > > Dovecot Team, > > I'd like to report a number of bugs, that are to my view all > critical. > > System: Replicated on multiple Debian 10 (Buster) systems > Dovecot Version(s): 2.3.4.1 > > doveadm-sync -1/general > > 1) If DIRNAMEs are not different between command line and > mail_location doveadm sync will fail, saying that the source and > destination directories are the same > > 2) The -n / -N flags do not work, and a sync will fail strangely if > location is specified in the namespace definition > > 3) Adds mbox to path name under mailbox directory (where syncing from > an mbox source) > > 4) Not having the mailboxes at source named the same as those at > destination causes errors and partial sync > > 5) Not having the target mailboxes formatted to receive the sync > (//DIRNAME/) will cause sync errors. > > doveadm-sync > > 1) With large synchronizations UIDs are corrupted where multiple > syncs are executed and the program can no longer synchronize > > dovecot > > 1) Panics and fails to expand ~ to user home: observed cases are > where multiple namespaces are being used > > Please let me know if you need me to elaborate or to provide any > further information that you may need to replicate the bugs, or if I > can help in any other way. > > With regards to the last error that I requested help on i.e. > \Noselect. This has been resolved more-or-less by the workarounds > that I have implemented for the bugs reported above. > > I have seen a number of threads whilst researching the \Noselect > issue where people have been very confused. My finding was that > \Noselect is a function of the IMAP specification server-side > implementation RFC3501 ( > https://tools.ietf.org/html/rfc3501#section-6.3.6). And for me the > server was returning directories with \Noselect because the mailboxes > were malformed on account of dovadm-sync errors. In order to fix this > I formed a bash command to transverse the mailbox hierarchy and > create the missing folders critical to the sdbox format, namely > DIRNAME. > > Kind regards, > > Arnold Opio Oree > Chief Executive Officer > Parallax Digital Technologies > > arnoldo...@parallaxdt.com > > http://www.parallaxdt.com > > tel : +44 (0) 333 577 8587 > fax : +44 (0) 20 8711 2477 > > Parallax Digital Technologies is a trading name of Parallax Global > Limited. U.K. Co. No. 08836288 > > The contents of this e-mail are confidential. If you are not the > intended recipient you are to delete this e-mail immediately, > disregard its contents and disclose them to no other persons. > > -Original Message- > From: Arnold Opio Oree via dovecot < dovecot@dovecot.org> > Reply-To: arnoldo...@parallaxict.com, Arnold Opio Oree < > arnold.o...@parallaxict.com> > To: dovecot@dovecot.org > Cc: r...@sys4.de, aki.tu...@open-xchange.com > Subject: Re: Applying Dovecot for a large / deep folder-hierarchy > archive. > Date: Thu, 04 Jul 2019 14:52:28 +0100 > > > Hi all, > > The guidance provided so far has been really helpful, and has helped > a great deal to bringing down wasted energy on finding and executing > a viable path. I am now at the final due action to complete our > Dovecot application to our use-case, but am stuck
Re: Orphaned processes after doveadm log reopen
On 2019-07-08 13:36, Tom Sommer via dovecot wrote: I rotate logs every night on my Director, running "doveadm log reopen" It happens on "/etc/init.d/dovecot restart", not "doveadm log reopen" - sorry So it happens when you run killproc on dovecot --- Tom
Re: Orphaned processes after doveadm log reopen
On 2019-07-08 13:36, Tom Sommer via dovecot wrote: I rotate logs every night on my Director, running "doveadm log reopen" I've noticed a problem with processes not closing correctly, they can be open for days and cause corrupt index files because they are connected to different backends (so it writes to the wrong server when the process disconnects) 23518 ?S140:53 dovecot/anvil [4 connections] 23519 ?S444:45 dovecot/log 23520 ?S497:19 dovecot/imap-login [8 pre-login + 27 proxies] 23531 ?S 0:27 dovecot/config 26201 ?S551:53 dovecot/imap-login [7 pre-login + 33 proxies] 22126 ?S960:15 dovecot/pop3-login [28 pre-login + 396 proxies] 59515 ?S218:32 dovecot/pop3-login [44 pre-login + 468 proxies] Is there any way to force these orphaned processes to terminate after a certain time? I ran "strace -f -p" on 59515 which caused the process to wake up and terminate? Very strange. I can send the dump from strace if you want, but since it contains addresses I would prefer it to be done off-list
Orphaned processes after doveadm log reopen
I rotate logs every night on my Director, running "doveadm log reopen" I've noticed a problem with processes not closing correctly, they can be open for days and cause corrupt index files because they are connected to different backends (so it writes to the wrong server when the process disconnects) 23518 ?S140:53 dovecot/anvil [4 connections] 23519 ?S444:45 dovecot/log 23520 ?S497:19 dovecot/imap-login [8 pre-login + 27 proxies] 23531 ?S 0:27 dovecot/config 26201 ?S551:53 dovecot/imap-login [7 pre-login + 33 proxies] 22126 ?S960:15 dovecot/pop3-login [28 pre-login + 396 proxies] 59515 ?S218:32 dovecot/pop3-login [44 pre-login + 468 proxies] Is there any way to force these orphaned processes to terminate after a certain time? -- Tom
