Re: [SOGo] Tuning SOGo environment

2018-11-27 Thread goetz.reini...@filmakademie.de 
Hi Ludovic,

I got may be an other one with more output:

#0  0x7f3fc2f387e0 in __read_nocancel () from /lib64/libpthread.so.0
#1  0x7f3fc0476ffb in sock_read () from /lib64/libcrypto.so.10
#2  0x7f3fc047503b in BIO_read () from /lib64/libcrypto.so.10
#3  0x7f3fbf2963e4 in ssl3_read_n () from /lib64/libssl.so.10
#4  0x7f3fbf297ccd in ssl3_read_bytes () from /lib64/libssl.so.10
#5  0x7f3fbf2946f4 in ssl3_read_internal () from /lib64/libssl.so.10
#6  0x7f3fc6451e73 in -[NGActiveSSLSocket readBytes:count:] 
(self=0x555fcfdfb6c0, _cmd=0x7f3fc666ad00 <_OBJC_SELECTOR_TABLE+416>, 
_buf=0x555fcf1ac0c0, _len=512) at NGActiveSSLSocket.m:263
#7  0x7f3fc6437dab in -[NGBufferedStream readBytes:count:] 
(self=0x555fce209e10, _cmd=0x7f3fc666ad00 <_OBJC_SELECTOR_TABLE+416>, 
_buf=0x7ffecf176b8f, _len=1) at NGBufferedStream.m:291
#8  0x7f3fc643f385 in NGReadByteFromStream (_stream=0x555fce209e10) at 
NGStream.m:160
#9  0x7f3fc643edcd in -[NGStream readByte] (self=0x555fce209e10, 
_cmd=0x7f3fc6662f10 <_OBJC_SELECTOR_TABLE+464>) at NGStream.m:90
#10 0x7f3fc6438100 in -[NGBufferedStream readByte] (self=0x555fce209e10, 
_cmd=0x7f3fc6671be0 <_OBJC_SELECTOR_TABLE+352>) at NGBufferedStream.m:336
#11 0x7f3fc6443b8d in -[NGByteBuffer la:] (self=0x555fc9285f60, 
_cmd=0x7f3fc6c0ae40 <_OBJC_SELECTOR_TABLE+288>, _la=0) at NGByteBuffer.m:217
#12 0x7f3fc6962640 in _la (self=0x555fcef070e0, _laCnt=0) at 
NGImap4ResponseParser.m:67
#13 0x7f3fc696308e in -[NGImap4ResponseParser 
parseResponseForTagId:exception:] (self=0x555fcef070e0, _cmd=0x7f3fc6c0dc30 
<_OBJC_SELECTOR_TABLE+880>, _tag=9, ex_=0x7ffecf176dd8)
at NGImap4ResponseParser.m:273
#14 0x7f3fc697435e in -[NGImap4Client 
processCommand:withTag:withNotification:logText:] (self=0x555fcf1e9db0, 
_cmd=0x7f3fc6c0e580 <_OBJC_SELECTOR_TABLE+3264>, _command=0x555fcf120130, 
_tag=1 '\001', _notification=1 '\001', _txt=0x555fcf120130) at 
NGImap4Client.m:1772
#15 0x7f3fc69748da in -[NGImap4Client processCommand:] 
(self=0x555fcf1e9db0, _cmd=0x7f3fc6c0dc70 <_OBJC_SELECTOR_TABLE+944>, 
_command=0x555fcf120130) at NGImap4Client.m:1845
#16 0x7f3fc69710ed in -[NGImap4Client fetchUids:parts:] 
(self=0x555fcf1e9db0, _cmd=0x7f3fc6c22bb0 <_OBJC_SELECTOR_TABLE+1584>, 
_uids=0x555fcb905cd0, _parts=0x555fcf9d4780)
at NGImap4Client.m:1108
#17 0x7f3fc699e869 in -[NGImap4Connection fetchUIDs:inURL:parts:] 
(self=0x555fc8cf8160, _cmd=0x7f3fb33fb110 <_OBJC_SELECTOR_TABLE+2416>, 
_uids=0x555fd21300c0, _url=0x555fcf6ea950, 
_parts=0x555fcf9d4780) at NGImap4Connection.m:711
#18 0x7f3fb31c8d8f in -[SOGoMailFolder fetchUIDs:parts:] 
(self=0x555fc8e4f370, _cmd=0x7f3fb11df5f0 <_OBJC_SELECTOR_TABLE+1648>, 
_uids=0x555fd21300c0, _parts=0x555fcf9d4780)
at SOGoMailFolder.m:986
#19 0x7f3fb0fbe282 in -[UIxMailListActions getHeadersForUIDs:inFolder:] 
(self=0x555fcdcd63a0, _cmd=0x7f3fb11df520 <_OBJC_SELECTOR_TABLE+1440>, 
uids=0x555fd21300c0, 
mailFolder=0x555fc8e4f370) at UIxMailListActions.m:799
---Type  to continue, or q  to quit---
#20 0x7f3fb0fbdd04 in -[UIxMailListActions getUIDsInFolder:withHeaders:] 
(self=0x555fcdcd63a0, _cmd=0x7f3fb11df590 <_OBJC_SELECTOR_TABLE+1552>, 
folder=0x555fc8e4f370, 
includeHeaders=1 '\001') at UIxMailListActions.m:673
#21 0x7f3fb0fbe068 in -[UIxMailListActions getUIDsAction] 
(self=0x555fcdcd63a0, _cmd=0x555fc88f57c0) at UIxMailListActions.m:765
#22 0x7f3fc6d6b20b in -[WODirectAction performActionNamed:] 
(self=0x555fcdcd63a0, _cmd=0x7f3fb25ddcf0 <_OBJC_SELECTOR_TABLE+720>, 
_actionName=0x555fcf2cbdf0) at WODirectAction.m:97
#23 0x7f3fb23cb199 in -[SOGoDirectAction performActionNamed:] 
(self=0x555fcdcd63a0, _cmd=0x7f3fc7132880 <_OBJC_SELECTOR_TABLE+928>, 
_actionName=0x555fc81d6640) at SOGoDirectAction.m:254
#24 0x7f3fc6df9640 in -[SoActionInvocation 
callOnObject:withPositionalParametersWhenNotNil:inContext:] 
(self=0x555fcdf5ebb0, _cmd=0x7f3fc71328b0 <_OBJC_SELECTOR_TABLE+976>, 
_client=0x555fc8e4f370, _positionalArgs=0x0, _ctx=0x555fc91a1260) at 
SoActionInvocation.m:300
#25 0x7f3fc6df9765 in -[SoActionInvocation callOnObject:inContext:] 
(self=0x555fcdf5ebb0, _cmd=0x7f3fc71327d0 <_OBJC_SELECTOR_TABLE+752>, 
_client=0x555fc8e4f370, _ctx=0x555fc91a1260)
at SoActionInvocation.m:316
#26 0x7f3fc6df93ee in -[SoActionInvocation 
callOnObject:withPositionalParametersWhenNotNil:inContext:] 
(self=0x555fc81d65a0, _cmd=0x7f3fc71328b0 <_OBJC_SELECTOR_TABLE+976>, 
_client=0x555fc8e4f370, _positionalArgs=0x0, _ctx=0x555fc91a1260) at 
SoActionInvocation.m:259
#27 0x7f3fc6df9765 in -[SoActionInvocation callOnObject:inContext:] 
(self=0x555fc81d65a0, _cmd=0x7f3fc712ee20 <_OBJC_SELECTOR_TABLE+672>, 
_client=0x555fc8e4f370, _ctx=0x555fc91a1260)
at SoActionInvocation.m:316
#28 0x7f3fc6df3ec4 in -[SoObjectMethodDispatcher dispatchInContext:] 
(self=0x555fcfbd93c0, _cmd=0x7f3fc7130500 <_OBJC_SELECTOR_TABLE+1536>, 
_ctx=0x555fc91a1260)
at

Re: [SOGo] Tuning SOGo environment

2018-11-23 Thread goetz.reini...@filmakademie.de 


> Am 21.11.2018 um 14:30 schrieb Ludovic Marcotte (lmarco...@inverse.ca) 
> :
> 
> On 2018-11-21 8:28 AM, Götz Reinicke (goetz.reini...@filmakademie.de 
> ) wrote:
>> I don’t get it …
>> 
>> In the logs I have „sogod [9256]: [ERROR] <0x0x559d2ea51c00[WOWatchDog]> No 
>> child available to handle incoming request!“
>> 
>> I did use the PID 9256 of process  gdb -p 9256 and hit „bt“. The output are 
>> the lines above.
>> 
>> What am I missing?
> Look for messages like "process has been hanging for X minutes.." and grab 
> the pid. Use that pid for gd.
> 
> 

Got one - hopefully the right one :)

#0  0x7f3fc0ceef0d in poll () from /lib64/libc.so.6
#1  0x7f3fc5621e36 in poll (__timeout=6536, __nfds=, 
__fds=) at /usr/include/bits/poll2.h:46
#2  -[GSRunLoopCtxt pollUntil:within:] (self=, 
_cmd=0x7f3fc59ff540 <_OBJC_SELECTOR_TABLE+1216>, milliseconds=6536, 
contexts=) at GSRunLoopCtxt.m:430
#3  0x7f3fc5561030 in -[NSRunLoop acceptInputForMode:beforeDate:] 
(self=0x555fc8220c10, _cmd=0x7f3fc59ff580 <_OBJC_SELECTOR_TABLE+1280>, 
mode=0x7f3fc5a00350 <_OBJC_INSTANCE_2>, 
limit_date=) at NSRunLoop.m:1214
#4  0x7f3fc55614a9 in -[NSRunLoop runMode:beforeDate:] 
(self=0x555fc8220c10, _cmd=0x7f3fc7080ce0 <_OBJC_SELECTOR_TABLE+1344>, 
mode=0x7f3fc5a00350 <_OBJC_INSTANCE_2>, date=0x555fd5c51910)
at NSRunLoop.m:1275
#5  0x7f3fc6d3d954 in -[WOCoreApplication run] (self=0x555fc8152390, 
_cmd=0x555fc7b63200 <_OBJC_SELECTOR_TABLE+992>) at WOCoreApplication.m:584
#6  0x555fc795a0d4 in -[SOGo run] (self=0x555fc8152390, _cmd=0x7f3fc70bedb0 
<_OBJC_SELECTOR_TABLE+624>) at SOGo.m:337
#7  0x7f3fc6d85e4d in -[WOWatchDog _runChildWithControlSocket:] 
(self=0x555fc8294c10, _cmd=0x7f3fc70bee50 <_OBJC_SELECTOR_TABLE+784>, 
controlSocket=0x555fc80fd5b0)
at WOWatchDogApplicationMain.m:523
#8  0x7f3fc6d862e0 in -[WOWatchDog _spawnChild:] (self=0x555fc8294c10, 
_cmd=0x7f3fc70bef20 <_OBJC_SELECTOR_TABLE+992>, child=0x555fc81da5a0) at 
WOWatchDogApplicationMain.m:600
#9  0x7f3fc6d869d6 in -[WOWatchDog _ensureChildren] (self=0x555fc8294c10, 
_cmd=0x7f3fc70bf0e0 <_OBJC_SELECTOR_TABLE+1440>) at 
WOWatchDogApplicationMain.m:690
#10 0x7f3fc6d87a1e in -[WOWatchDog run:argc:argv:] (self=0x555fc8294c10, 
_cmd=0x7f3fc70bf1f0 <_OBJC_SELECTOR_TABLE+1712>, newAppName=0x555fc7b61160 
<_OBJC_INSTANCE_0>, newArgC=7, 
newArgV=0x7ffecf17a308) at WOWatchDogApplicationMain.m:942
#11 0x7f3fc6d88149 in WOWatchDogApplicationMain (appName=0x555fc7b61160 
<_OBJC_INSTANCE_0>, argc=7, argv=0x7ffecf17a308) at 
WOWatchDogApplicationMain.m:1051
#12 0x555fc7958ce1 in main (argc=7, argv=0x7ffecf17a308, 
env=0x7ffecf17a348) at sogod.m:51


thx. Regards . Götz 



smime.p7s
Description: S/MIME cryptographic signature

Re: [SOGo] Tuning SOGo environment

2018-11-21 Thread Ludovic Marcotte 

On 2018-11-21 8:28 AM, Götz Reinicke (goetz.reini...@filmakademie.de) wrote:


I don’t get it …

In the logs I have „sogod [9256]: [*ERROR*] 
<0x0x559d2ea51c00[WOWatchDog]> No child available to handle incoming 
request!“


I did use the PID 9256 of process  gdb -p 9256 and hit „bt“. The 
output are the lines above.


What am I missing?


Look for messages like "process has been hanging for X minutes.." and 
grab the pid. Use that pid for gdb.


Thanks,

--
Ludovic Marcotte
lmarco...@inverse.ca  ::  +1.514.755.3630  ::  https://inverse.ca
Inverse inc. :: Leaders behind SOGo (https://sogo.nu), PacketFence 
(https://packetfence.org) and Fingerbank (https://fingerbank.org)

--
users@sogo.nu
https://inverse.ca/sogo/lists

Re: [SOGo] Tuning SOGo environment

2018-11-21 Thread goetz.reini...@filmakademie.de 


> Am 21.11.2018 um 13:14 schrieb Ludovic Marcotte (lmarco...@inverse.ca) 
> :
> 
> On 2018-11-21 5:10 AM, Götz Reinicke (goetz.reini...@filmakademie.de 
> ) wrote:
>> 
>> <…>
>> 
>> #0  0x7f21b3284ef0 in __poll_nocancel () from /lib64/libc.so.6
>> #1  0x7f21b7bb7e36 in poll (__timeout=499, __nfds=, 
>> __fds=) at /usr/include/bits/poll2.h:46
>> #2  -[GSRunLoopCtxt pollUntil:within:] (self=, 
>> _cmd=0x7f21b7f95540 <_OBJC_SELECTOR_TABLE+1216>, milliseconds=499, 
>> contexts=) at GSRunLoopCtxt.m:430
>> #3  0x7f21b7af7030 in -[NSRunLoop acceptInputForMode:beforeDate:] 
>> (self=0x559d2e9ddc10, _cmd=0x7f21b7f95580 <_OBJC_SELECTOR_TABLE+1280>, 
>> mode=0x7f21b7f96350 <_OBJC_INSTANCE_2>, limit_date=) at 
>> NSRunLoop.m:1214
>> #4  0x7f21b7af74a9 in -[NSRunLoop runMode:beforeDate:] 
>> (self=0x559d2e9ddc10, _cmd=0x7f21b9655100 <_OBJC_SELECTOR_TABLE+1472>, 
>> mode=0x7f21b7f96350 <_OBJC_INSTANCE_2>, date=0x559d2e8c76f0) at 
>> NSRunLoop.m:1275
>> #5  0x7f21b931da8b in -[WOWatchDog run:argc:argv:] (self=0x559d2ea51c00, 
>> _cmd=0x7f21b96551f0 <_OBJC_SELECTOR_TABLE+1712>, newAppName=0x559d2d3bc160 
>> <_OBJC_INSTANCE_0>, newArgC=7, newArgV=0x7ffcefa4e2d8) at 
>> WOWatchDogApplicationMain.m:945
>> #6  0x7f21b931e149 in WOWatchDogApplicationMain (appName=0x559d2d3bc160 
>> <_OBJC_INSTANCE_0>, argc=7, argv=0x7ffcefa4e2d8) at 
>> WOWatchDogApplicationMain.m:1051
>> #7  0x559d2d1b3ce1 in main (argc=7, argv=0x7ffcefa4e2d8, 
>> env=0x7ffcefa4e318) at sogod.m:51
>> 
> That's the master sogod process, not a child doing work.

I don’t get it …

In the logs I have „sogod [9256]: [ERROR] <0x0x559d2ea51c00[WOWatchDog]> No 
child available to handle incoming request!“

I did use the PID 9256 of process  gdb -p 9256 and hit „bt“. The output are the 
lines above.

What am I missing?

/Götz 



smime.p7s
Description: S/MIME cryptographic signature

Re: [SOGo] Tuning SOGo environment

2018-11-21 Thread Ludovic Marcotte 

On 2018-11-21 5:10 AM, Götz Reinicke (goetz.reini...@filmakademie.de) wrote:



<…>

#0 0x7f21b3284ef0 in __poll_nocancel () from /lib64/libc.so.6
#1 0x7f21b7bb7e36 in poll (__timeout=499, __nfds=, 
__fds=) at /usr/include/bits/poll2.h:46
#2 -[GSRunLoopCtxt pollUntil:within:] (self=, 
_cmd=0x7f21b7f95540 <_OBJC_SELECTOR_TABLE+1216>, milliseconds=499, 
contexts=) at GSRunLoopCtxt.m:430
#3 0x7f21b7af7030 in -[NSRunLoop acceptInputForMode:beforeDate:] 
(self=0x559d2e9ddc10, _cmd=0x7f21b7f95580 <_OBJC_SELECTOR_TABLE+1280>, 
mode=0x7f21b7f96350 <_OBJC_INSTANCE_2>, limit_date=) at 
NSRunLoop.m:1214
#4 0x7f21b7af74a9 in -[NSRunLoop runMode:beforeDate:] 
(self=0x559d2e9ddc10, _cmd=0x7f21b9655100 <_OBJC_SELECTOR_TABLE+1472>, 
mode=0x7f21b7f96350 <_OBJC_INSTANCE_2>, date=0x559d2e8c76f0) at 
NSRunLoop.m:1275
#5 0x7f21b931da8b in -[WOWatchDog run:argc:argv:] 
(self=0x559d2ea51c00, _cmd=0x7f21b96551f0 <_OBJC_SELECTOR_TABLE+1712>, 
newAppName=0x559d2d3bc160 <_OBJC_INSTANCE_0>, newArgC=7, 
newArgV=0x7ffcefa4e2d8) at WOWatchDogApplicationMain.m:945
#6 0x7f21b931e149 in WOWatchDogApplicationMain 
(appName=0x559d2d3bc160 <_OBJC_INSTANCE_0>, argc=7, 
argv=0x7ffcefa4e2d8) at WOWatchDogApplicationMain.m:1051
#7 0x559d2d1b3ce1 in main (argc=7, argv=0x7ffcefa4e2d8, 
env=0x7ffcefa4e318) at sogod.m:51



That's the master sogod process, not a child doing work.

--
Ludovic Marcotte
lmarco...@inverse.ca  ::  +1.514.755.3630  ::  https://inverse.ca
Inverse inc. :: Leaders behind SOGo (https://sogo.nu), PacketFence 
(https://packetfence.org) and Fingerbank (https://fingerbank.org)

--
users@sogo.nu
https://inverse.ca/sogo/lists

Re: [SOGo] Tuning SOGo environment

2018-11-21 Thread goetz.reini...@filmakademie.de 
Hi,

> Am 20.11.2018 um 20:54 schrieb Ludovic Marcotte (lmarco...@inverse.ca) 
> :
> 
> On 2018-11-20 5:10 AM, Götz Reinicke (goetz.reini...@filmakademie.de 
> ) wrote:
>> 
>> Checking the sogo log I can currently only see the old friend „No child 
>> available to handle incoming request!“ … but no other clues.
> When you see this, try to attach to the process using gdb (so "gdb -p 
> PID_OF_THE_PROCESS") and produce a stacktrace (by typing "bt"). Quit gdb 
> after and send us the stacktrace. It'll tell us where it's stuck. Install 
> SOPE/SOGo/GNUstep debugging symbols before doing that. See 
> https://sogo.nu/support/faq/how-do-i-debug-sogo.html 
>  for all nice details.

I’ll try to have a closer look ASAP.

>> 
>> That error shows up from time to time and than all sogod processes are using 
>> from 40 to 100% of cpu  …. 
>> 
>> My PREFORK is at 12, the server is a 12 GB 4 CPU VM. MySQL and Dovecot are 
>> on different VMs.
> My wild guess based on the symptoms you mention is your IMAP server is 
> running out of available connections. Or, you have EAS clients consuming all 
> sogod workers.

Hm, I’ll check the IMAP server too. 

As I currently don’t know how might use Active Sync (EAS), is there a easy way 
to see the clients in SOGo logs ?

Thx. Götz





smime.p7s
Description: S/MIME cryptographic signature

Re: [SOGo] Tuning SOGo environment

2018-11-20 Thread Ludovic Marcotte 

On 2018-11-20 5:10 AM, Götz Reinicke (goetz.reini...@filmakademie.de) wrote:



Checking the sogo log I can currently only see the old friend „No 
child available to handle incoming request!“ … but no other clues.
When you see this, try to attach to the process using gdb (so "gdb -p 
PID_OF_THE_PROCESS") and produce a stacktrace (by typing "bt"). Quit gdb 
after and send us the stacktrace. It'll tell us where it's stuck. 
Install SOPE/SOGo/GNUstep debugging symbols before doing that. See 
https://sogo.nu/support/faq/how-do-i-debug-sogo.html for all nice details.


That error shows up from time to time and than all sogod processes are 
using from 40 to 100% of cpu  ….


My PREFORK is at 12, the server is a 12 GB 4 CPU VM. MySQL and Dovecot 
are on different VMs.


My wild guess based on the symptoms you mention is your IMAP server is 
running out of available connections. Or, you have EAS clients consuming 
all sogod workers.


Thanks,

--
Ludovic Marcotte
lmarco...@inverse.ca  ::  +1.514.755.3630  ::  https://inverse.ca
Inverse inc. :: Leaders behind SOGo (https://sogo.nu), PacketFence 
(https://packetfence.org) and Fingerbank (https://fingerbank.org)

--
users@sogo.nu
https://inverse.ca/sogo/lists

[SOGo] Tuning SOGo environment

2018-11-20 Thread goetz.reini...@filmakademie.de 
Hi,

we are faced with some complains from different users which I can partly 
reproduce or understand.

E.G. they log in to the mail Web Userinterface and try to open a mail, which 
takes 5 - 10 seconds, sometimes the mail will not be loaded at all.

Sometimes I can reproduce that for a mail with +- 10kb in size with my account. 
Klick on it… it loads … klick some minutes later on the same mail … it will not 
load at all.

Some users told me, that opening mails, writing some or add calendars is very 
slow while opening the modules or entries in the browser.

This is independent from chrome to firefox or safari on modern new devices and 
some folders….

e.g. my sandbox has about 25.000 mails and shows up from my POV quit quick on 
an iPhones and in safari on my macbook.

Otter users have to wait a few seconds till the list shows up in mailbox 
folders with about 1.000 mails ...

Native mailclients do not show such a „strange“ behavior.



Checking the sogo log I can currently only see the old friend „No child 
available to handle incoming request!“ … but no other clues.

That error shows up from time to time and than all sogod processes are using 
from 40 to 100% of cpu  …. 

My PREFORK is at 12, the server is a 12 GB 4 CPU VM. MySQL and Dovecot are on 
different VMs.


Where and how may I start to get better performance or debug and find the 
„real“ bottleneck?

E.g. what is a special sogo process doing with 100% cpu time?


Thanks for hints and suggestions. Regards . Götz

smime.p7s
Description: S/MIME cryptographic signature