Re: [SOGo] sogod hanging in the same request for ... minutes
On 03/09/2012 02:07 AM, Patrick Ben Koetter wrote: Anyone from Inverse? * Patrick Ben Koetterusers@sogo.nu: Anyone? Hello Patrick. I don't know if this will help you or not, but I too recently saw sogod hanging. For me it started after upgrading to 1.3.12 and almost simultaneously adding additional users. In my case, the issue appears to have been resolved by dealing with some resource limits being reached in the virtual server I had SOGo running in. You've probably already looked at this kind of thing, but just passing this on in case. -- Clay -- users@sogo.nu https://inverse.ca/sogo/lists
Re: [SOGo] sogod hanging in the same request for ... minutes
On 3/15/2012 11:27 AM, Clay Wright wrote: On 03/09/2012 02:07 AM, Patrick Ben Koetter wrote: Anyone from Inverse? * Patrick Ben Koetterusers@sogo.nu: Anyone? Hello Patrick. I don't know if this will help you or not, but I too recently saw sogod hanging. For me it started after upgrading to 1.3.12 and almost simultaneously adding additional users. In my case, the issue appears to have been resolved by dealing with some resource limits being reached in the virtual server I had SOGo running in. You've probably already looked at this kind of thing, but just passing this on in case. I have dealt with similar issues in the past. It started rearing its head about the time we had the last batch of upgrades. Today I have had to power off and restart the virtual server that houses SOGo twice. Both time the load gets so astronomically high that it is unreachable. Then there is an out of memory error on the console when I have been able to get to that. If we can't find a workaround Iam going to try reinstalling fresh and just restore the calendars/mail/etc from the sogo-tool backup. Anyone have any ideas? Donny B. -- users@sogo.nu https://inverse.ca/sogo/lists
Re: [SOGo] sogod hanging in the same request for ... minutes
On 03/15/2012 12:58 PM, Donny Brooks wrote: I have dealt with similar issues in the past. It started rearing its head about the time we had the last batch of upgrades. Today I have had to power off and restart the virtual server that houses SOGo twice. Both time the load gets so astronomically high that it is unreachable. Then there is an out of memory error on the console when I have been able to get to that. If we can't find a workaround Iam going to try reinstalling fresh and just restore the calendars/mail/etc from the sogo-tool backup. Anyone have any ideas? I'd certainly try giving your VM more memory. That appears to have been the main thing that helped for me. When I saw memory limits being reached, sogod would sometimes hang and consume 100% of the CPU. -- users@sogo.nu https://inverse.ca/sogo/lists
Re: [SOGo] sogod hanging in the same request for ... minutes
On 3/15/2012 2:31 PM, Clay Wright wrote: On 03/15/2012 12:58 PM, Donny Brooks wrote: I have dealt with similar issues in the past. It started rearing its head about the time we had the last batch of upgrades. Today I have had to power off and restart the virtual server that houses SOGo twice. Both time the load gets so astronomically high that it is unreachable. Then there is an out of memory error on the console when I have been able to get to that. If we can't find a workaround Iam going to try reinstalling fresh and just restore the calendars/mail/etc from the sogo-tool backup. Anyone have any ideas? I'd certainly try giving your VM more memory. That appears to have been the main thing that helped for me. When I saw memory limits being reached, sogod would sometimes hang and consume 100% of the CPU. It started life with 4GB of memory and 4cpu cores. Recently when this started happening I gave it 6GB and 6 cores but that did not help much if any. Changing the cleanup time from 300 seconds to 90 seconds had the most impact and giving it 8 workers with 256MB each helped too. I just don't have the luxury of trying different settings since this is a production system now. If someone could explain the settings better maybe one evening I can change them to be optimal. We have approx 200 users and use SOGo with external dovecot, external MySQL, and external OpenLDAP. So it really isn't hard to replace the machine if need be. Donny B. -- users@sogo.nu https://inverse.ca/sogo/lists
Re: [SOGo] sogod hanging in the same request for ... minutes
Anyone from Inverse? * Patrick Ben Koetter users@sogo.nu: Anyone? * Patrick Ben Koetter users@sogo.nu: Ludovic, * Ludovic Marcotte users@sogo.nu: On 07/02/12 06:12, Patrick Ben Koetter wrote: Happened again today and people can't login on the Webinterface: Feb 07 11:47:48 sogod [14423]: [WARN]0x0x128e6848[WOWatchDogChild] pid 24369 has been hanging in the same request for 6 minutes Feb 07 11:48:30 sogod [14423]: [WARN]0x0x128e6608[WOWatchDogChild] pid 16354 has been hanging in the same request for 2 minutes Feb 07 11:48:31 sogod [14423]: [WARN]0x0x128e75a8[WOWatchDogChild] pid 17308 has been hanging in the same request for 2 minutes ... Feb 07 11:51:53 sogod [14423]: [WARN]0x0x128e6848[WOWatchDogChild] pid 24369 has been hanging in the same request for 2 minutes Feb 07 11:52:35 sogod [14423]: [WARN]0x0x128e6608[WOWatchDogChild] pid 16354 has been hanging in the same request for 2 minutes Make sure you have the debug symbols installed for SOGo, SOPE and GNUstep and when that happens, attach to a pid using gdb (for example, pid 24369) and produce a stack trace. It'll show you where the process has been hanging - thus the real cause of your issue. here's a series of backtraces. All of them were made by the customer on site. I did not have direct access. Can you tell what's the real cause of my issue: (gdb) b Breakpoint 1 at 0x334a6cba7f (gdb) bt #0 0x00334a6cba7f in poll () from /lib64/libc.so.6 #1 0x00335443caf6 in NGDescriptorRecv (_fd=20, _buf=0x1284d890 , _len=512, _flags=0, _timeout=12) at NGDescriptorFunctions.m:215 #2 0x003354441e3a in -[NGActiveSocket readBytes:count:] (self=0x128bc9d8, _cmd=0x33546631b0, _buf=0x1284d890, _len=512) at NGActiveSocket.m:731 #3 0x0033544312de in -[NGBufferedStream readBytes:count:] (self=0x129e5808, _cmd=0x33546631b0, _buf=0x7fffd8e825d7, _len=1) at NGBufferedStream.m:289 #4 0x003354439712 in NGReadByteFromStream (_stream=0x129e5808) at NGStream.m:160 #5 0x003354439161 in -[NGStream readByte] (self=0x129e5808, _cmd=0x335465b380) at NGStream.m:90 #6 0x003354431693 in -[NGBufferedStream readByte] (self=0x129e5808, _cmd=0x335466a070) at NGBufferedStream.m:336 #7 0x00335443e38f in -[NGByteBuffer la:] (self=0x12867d08, _cmd=0x335466a090, _la=0) at NGByteBuffer.m:215 #8 0x00335443de29 in -[NGByteBuffer readByte] (self=0x12867d08, _cmd=0x3352abbc70) at NGByteBuffer.m:130 #9 0x003352781a9d in _readByte (self=0x12b5f4b8) at NGHttpMessageParser.m:78 #10 0x00335278138a in -[NGHttpMessageParser parseRequestLine] (self=0x12b5f4b8, _cmd=0x3352abbd50) at NGHttpMessageParser.m:168 #11 0x003352781fd9 in -[NGHttpMessageParser parseStartLine] (self=0x12b5f4b8, _cmd=0x3352abbd80) at NGHttpMessageParser.m:499 #12 0x00335278208a in -[NGHttpMessageParser parsePrefix] (self=0x12b5f4b8, _cmd=0x3354f2aa80) at NGHttpMessageParser.m:506 #13 0x003354c8c456 in -[NGMimePartParser parsePart] (self=0x12b5f4b8, _cmd=0x3354f2ab00) at NGMimePartParser.m:1236 #14 0x003354c8c6dd in -[NGMimePartParser parsePartFromStream:] (self=0x12b5f4b8, _cmd=0x3352abbeb0, _stream=0x129e5808) at NGMimePartParser.m:1269 #15 0x003352782bbf in -[NGHttpMessageParser parseRequestFromStream:] (self=0x12b5f4b8, _cmd=0x3352b0c580, _stream=0x129e5808) at NGHttpMessageParser.m:697 #16 0x0033527d75cc in -[WOHttpTransaction parseRequestFromStream:] (self=0x126e0338, _cmd=0x3352b0c2f0, _in=0x129e5808) at WOHttpTransaction.m:658 #17 0x0033527d5a7f in -[WOHttpTransaction _readRequest] (self=0x126e0338, _cmd=0x3352b0c510) at WOHttpTransaction.m:400 #18 0x0033527d6f61 in -[WOHttpTransaction _run] (self=0x126e0338, _cmd=0x3352b0c550) at WOHttpTransaction.m:575 #19 0x0033527d73ea in -[WOHttpTransaction run] (self=0x126e0338, _cmd=0x3352b0a300) at WOHttpTransaction.m:634 #20 0x0033527d2716 in -[WOHttpAdaptor runConnection:] (self=0x12664c48, _cmd=0x3352b0a3a0, _socket=0x128bc9d8) at WOHttpAdaptor.m:384 #21 0x0033527d2a08 in -[WOHttpAdaptor _handleAcceptedConnection:] (self=0x12664c48, _cmd=0x3352b0a3b0, _connection=0x128bc9d8) at WOHttpAdaptor.m:418 #22 0x0033527d2f4b in -[WOHttpAdaptor _handleConnection:] (self=0x12664c48, _cmd=0x3352b0a460, connection=0x128bc9d8) at WOHttpAdaptor.m:477 #23 0x0033527d32a0 in -[WOHttpAdaptor acceptControlMessage:] (self=0x12664c48, _cmd=0x3352b0a1c0, aNotification=0x12801c98) at WOHttpAdaptor.m:516 #24 0x003350c53eb8 in -[NSNotificationCenter _postAndRelease:] (self=0x121e07d8, _cmd=0x3351113690, notification=0x12801c98) at NSNotificationCenter.m:1223 #25 0x003350c54866 in -[NSNotificationCenter postNotificationName:object:userInfo:] (self=0x121e07d8, _cmd=0x33511136a0,
Re: [SOGo] sogod hanging in the same request for ... minutes
Anyone? * Patrick Ben Koetter users@sogo.nu: Ludovic, * Ludovic Marcotte users@sogo.nu: On 07/02/12 06:12, Patrick Ben Koetter wrote: Happened again today and people can't login on the Webinterface: Feb 07 11:47:48 sogod [14423]: [WARN]0x0x128e6848[WOWatchDogChild] pid 24369 has been hanging in the same request for 6 minutes Feb 07 11:48:30 sogod [14423]: [WARN]0x0x128e6608[WOWatchDogChild] pid 16354 has been hanging in the same request for 2 minutes Feb 07 11:48:31 sogod [14423]: [WARN]0x0x128e75a8[WOWatchDogChild] pid 17308 has been hanging in the same request for 2 minutes ... Feb 07 11:51:53 sogod [14423]: [WARN]0x0x128e6848[WOWatchDogChild] pid 24369 has been hanging in the same request for 2 minutes Feb 07 11:52:35 sogod [14423]: [WARN]0x0x128e6608[WOWatchDogChild] pid 16354 has been hanging in the same request for 2 minutes Make sure you have the debug symbols installed for SOGo, SOPE and GNUstep and when that happens, attach to a pid using gdb (for example, pid 24369) and produce a stack trace. It'll show you where the process has been hanging - thus the real cause of your issue. here's a series of backtraces. All of them were made by the customer on site. I did not have direct access. Can you tell what's the real cause of my issue: (gdb) b Breakpoint 1 at 0x334a6cba7f (gdb) bt #0 0x00334a6cba7f in poll () from /lib64/libc.so.6 #1 0x00335443caf6 in NGDescriptorRecv (_fd=20, _buf=0x1284d890 , _len=512, _flags=0, _timeout=12) at NGDescriptorFunctions.m:215 #2 0x003354441e3a in -[NGActiveSocket readBytes:count:] (self=0x128bc9d8, _cmd=0x33546631b0, _buf=0x1284d890, _len=512) at NGActiveSocket.m:731 #3 0x0033544312de in -[NGBufferedStream readBytes:count:] (self=0x129e5808, _cmd=0x33546631b0, _buf=0x7fffd8e825d7, _len=1) at NGBufferedStream.m:289 #4 0x003354439712 in NGReadByteFromStream (_stream=0x129e5808) at NGStream.m:160 #5 0x003354439161 in -[NGStream readByte] (self=0x129e5808, _cmd=0x335465b380) at NGStream.m:90 #6 0x003354431693 in -[NGBufferedStream readByte] (self=0x129e5808, _cmd=0x335466a070) at NGBufferedStream.m:336 #7 0x00335443e38f in -[NGByteBuffer la:] (self=0x12867d08, _cmd=0x335466a090, _la=0) at NGByteBuffer.m:215 #8 0x00335443de29 in -[NGByteBuffer readByte] (self=0x12867d08, _cmd=0x3352abbc70) at NGByteBuffer.m:130 #9 0x003352781a9d in _readByte (self=0x12b5f4b8) at NGHttpMessageParser.m:78 #10 0x00335278138a in -[NGHttpMessageParser parseRequestLine] (self=0x12b5f4b8, _cmd=0x3352abbd50) at NGHttpMessageParser.m:168 #11 0x003352781fd9 in -[NGHttpMessageParser parseStartLine] (self=0x12b5f4b8, _cmd=0x3352abbd80) at NGHttpMessageParser.m:499 #12 0x00335278208a in -[NGHttpMessageParser parsePrefix] (self=0x12b5f4b8, _cmd=0x3354f2aa80) at NGHttpMessageParser.m:506 #13 0x003354c8c456 in -[NGMimePartParser parsePart] (self=0x12b5f4b8, _cmd=0x3354f2ab00) at NGMimePartParser.m:1236 #14 0x003354c8c6dd in -[NGMimePartParser parsePartFromStream:] (self=0x12b5f4b8, _cmd=0x3352abbeb0, _stream=0x129e5808) at NGMimePartParser.m:1269 #15 0x003352782bbf in -[NGHttpMessageParser parseRequestFromStream:] (self=0x12b5f4b8, _cmd=0x3352b0c580, _stream=0x129e5808) at NGHttpMessageParser.m:697 #16 0x0033527d75cc in -[WOHttpTransaction parseRequestFromStream:] (self=0x126e0338, _cmd=0x3352b0c2f0, _in=0x129e5808) at WOHttpTransaction.m:658 #17 0x0033527d5a7f in -[WOHttpTransaction _readRequest] (self=0x126e0338, _cmd=0x3352b0c510) at WOHttpTransaction.m:400 #18 0x0033527d6f61 in -[WOHttpTransaction _run] (self=0x126e0338, _cmd=0x3352b0c550) at WOHttpTransaction.m:575 #19 0x0033527d73ea in -[WOHttpTransaction run] (self=0x126e0338, _cmd=0x3352b0a300) at WOHttpTransaction.m:634 #20 0x0033527d2716 in -[WOHttpAdaptor runConnection:] (self=0x12664c48, _cmd=0x3352b0a3a0, _socket=0x128bc9d8) at WOHttpAdaptor.m:384 #21 0x0033527d2a08 in -[WOHttpAdaptor _handleAcceptedConnection:] (self=0x12664c48, _cmd=0x3352b0a3b0, _connection=0x128bc9d8) at WOHttpAdaptor.m:418 #22 0x0033527d2f4b in -[WOHttpAdaptor _handleConnection:] (self=0x12664c48, _cmd=0x3352b0a460, connection=0x128bc9d8) at WOHttpAdaptor.m:477 #23 0x0033527d32a0 in -[WOHttpAdaptor acceptControlMessage:] (self=0x12664c48, _cmd=0x3352b0a1c0, aNotification=0x12801c98) at WOHttpAdaptor.m:516 #24 0x003350c53eb8 in -[NSNotificationCenter _postAndRelease:] (self=0x121e07d8, _cmd=0x3351113690, notification=0x12801c98) at NSNotificationCenter.m:1223 #25 0x003350c54866 in -[NSNotificationCenter postNotificationName:object:userInfo:] (self=0x121e07d8, _cmd=0x33511136a0, name=0x33542ae100, object=0x1243e708, info=0x0) at NSNotificationCenter.m:1282 #26 0x003350c546d9 in -[NSNotificationCenter postNotificationName:object:]
Re: [SOGo] sogod hanging in the same request for ... minutes
Ludovic, * Ludovic Marcotte users@sogo.nu: On 07/02/12 06:12, Patrick Ben Koetter wrote: Happened again today and people can't login on the Webinterface: Feb 07 11:47:48 sogod [14423]: [WARN]0x0x128e6848[WOWatchDogChild] pid 24369 has been hanging in the same request for 6 minutes Feb 07 11:48:30 sogod [14423]: [WARN]0x0x128e6608[WOWatchDogChild] pid 16354 has been hanging in the same request for 2 minutes Feb 07 11:48:31 sogod [14423]: [WARN]0x0x128e75a8[WOWatchDogChild] pid 17308 has been hanging in the same request for 2 minutes ... Feb 07 11:51:53 sogod [14423]: [WARN]0x0x128e6848[WOWatchDogChild] pid 24369 has been hanging in the same request for 2 minutes Feb 07 11:52:35 sogod [14423]: [WARN]0x0x128e6608[WOWatchDogChild] pid 16354 has been hanging in the same request for 2 minutes Make sure you have the debug symbols installed for SOGo, SOPE and GNUstep and when that happens, attach to a pid using gdb (for example, pid 24369) and produce a stack trace. It'll show you where the process has been hanging - thus the real cause of your issue. here's a series of backtraces. All of them were made by the customer on site. I did not have direct access. Can you tell what's the real cause of my issue: (gdb) b Breakpoint 1 at 0x334a6cba7f (gdb) bt #0 0x00334a6cba7f in poll () from /lib64/libc.so.6 #1 0x00335443caf6 in NGDescriptorRecv (_fd=20, _buf=0x1284d890 , _len=512, _flags=0, _timeout=12) at NGDescriptorFunctions.m:215 #2 0x003354441e3a in -[NGActiveSocket readBytes:count:] (self=0x128bc9d8, _cmd=0x33546631b0, _buf=0x1284d890, _len=512) at NGActiveSocket.m:731 #3 0x0033544312de in -[NGBufferedStream readBytes:count:] (self=0x129e5808, _cmd=0x33546631b0, _buf=0x7fffd8e825d7, _len=1) at NGBufferedStream.m:289 #4 0x003354439712 in NGReadByteFromStream (_stream=0x129e5808) at NGStream.m:160 #5 0x003354439161 in -[NGStream readByte] (self=0x129e5808, _cmd=0x335465b380) at NGStream.m:90 #6 0x003354431693 in -[NGBufferedStream readByte] (self=0x129e5808, _cmd=0x335466a070) at NGBufferedStream.m:336 #7 0x00335443e38f in -[NGByteBuffer la:] (self=0x12867d08, _cmd=0x335466a090, _la=0) at NGByteBuffer.m:215 #8 0x00335443de29 in -[NGByteBuffer readByte] (self=0x12867d08, _cmd=0x3352abbc70) at NGByteBuffer.m:130 #9 0x003352781a9d in _readByte (self=0x12b5f4b8) at NGHttpMessageParser.m:78 #10 0x00335278138a in -[NGHttpMessageParser parseRequestLine] (self=0x12b5f4b8, _cmd=0x3352abbd50) at NGHttpMessageParser.m:168 #11 0x003352781fd9 in -[NGHttpMessageParser parseStartLine] (self=0x12b5f4b8, _cmd=0x3352abbd80) at NGHttpMessageParser.m:499 #12 0x00335278208a in -[NGHttpMessageParser parsePrefix] (self=0x12b5f4b8, _cmd=0x3354f2aa80) at NGHttpMessageParser.m:506 #13 0x003354c8c456 in -[NGMimePartParser parsePart] (self=0x12b5f4b8, _cmd=0x3354f2ab00) at NGMimePartParser.m:1236 #14 0x003354c8c6dd in -[NGMimePartParser parsePartFromStream:] (self=0x12b5f4b8, _cmd=0x3352abbeb0, _stream=0x129e5808) at NGMimePartParser.m:1269 #15 0x003352782bbf in -[NGHttpMessageParser parseRequestFromStream:] (self=0x12b5f4b8, _cmd=0x3352b0c580, _stream=0x129e5808) at NGHttpMessageParser.m:697 #16 0x0033527d75cc in -[WOHttpTransaction parseRequestFromStream:] (self=0x126e0338, _cmd=0x3352b0c2f0, _in=0x129e5808) at WOHttpTransaction.m:658 #17 0x0033527d5a7f in -[WOHttpTransaction _readRequest] (self=0x126e0338, _cmd=0x3352b0c510) at WOHttpTransaction.m:400 #18 0x0033527d6f61 in -[WOHttpTransaction _run] (self=0x126e0338, _cmd=0x3352b0c550) at WOHttpTransaction.m:575 #19 0x0033527d73ea in -[WOHttpTransaction run] (self=0x126e0338, _cmd=0x3352b0a300) at WOHttpTransaction.m:634 #20 0x0033527d2716 in -[WOHttpAdaptor runConnection:] (self=0x12664c48, _cmd=0x3352b0a3a0, _socket=0x128bc9d8) at WOHttpAdaptor.m:384 #21 0x0033527d2a08 in -[WOHttpAdaptor _handleAcceptedConnection:] (self=0x12664c48, _cmd=0x3352b0a3b0, _connection=0x128bc9d8) at WOHttpAdaptor.m:418 #22 0x0033527d2f4b in -[WOHttpAdaptor _handleConnection:] (self=0x12664c48, _cmd=0x3352b0a460, connection=0x128bc9d8) at WOHttpAdaptor.m:477 #23 0x0033527d32a0 in -[WOHttpAdaptor acceptControlMessage:] (self=0x12664c48, _cmd=0x3352b0a1c0, aNotification=0x12801c98) at WOHttpAdaptor.m:516 #24 0x003350c53eb8 in -[NSNotificationCenter _postAndRelease:] (self=0x121e07d8, _cmd=0x3351113690, notification=0x12801c98) at NSNotificationCenter.m:1223 #25 0x003350c54866 in -[NSNotificationCenter postNotificationName:object:userInfo:] (self=0x121e07d8, _cmd=0x33511136a0, name=0x33542ae100, object=0x1243e708, info=0x0) at NSNotificationCenter.m:1282 #26 0x003350c546d9 in -[NSNotificationCenter postNotificationName:object:] (self=0x121e07d8, _cmd=0x33542ae2f0, name=0x33542ae100, object=0x1243e708) at NSNotificationCenter.m:1262 ---Type return to continue, or q return
Re: [SOGo] sogod hanging in the same request for ... minutes
* Patrick Ben Koetter users@sogo.nu: We're running SOGo for a customer with 25k accounts. Occasionally sogod locks up and users cannot be served. In such cases, the log below is from December, 15th 2011, sogod reports it has been hanging for a while: Happened again today and people can't login on the Webinterface: Feb 07 11:47:48 sogod [14423]: [WARN] 0x0x128e6848[WOWatchDogChild] pid 24369 has been hanging in the same request for 6 minutes Feb 07 11:48:30 sogod [14423]: [WARN] 0x0x128e6608[WOWatchDogChild] pid 16354 has been hanging in the same request for 2 minutes Feb 07 11:48:31 sogod [14423]: [WARN] 0x0x128e75a8[WOWatchDogChild] pid 17308 has been hanging in the same request for 2 minutes ... Feb 07 11:51:53 sogod [14423]: [WARN] 0x0x128e6848[WOWatchDogChild] pid 24369 has been hanging in the same request for 2 minutes Feb 07 11:52:35 sogod [14423]: [WARN] 0x0x128e6608[WOWatchDogChild] pid 16354 has been hanging in the same request for 2 minutes Dec 15 10:27:06 sogod [953]: [WARN] 0x0x1757e948[WOWatchDogChild] pid 18602 has been hanging in the same request for 7 minutes Looking for the PID I get log like this: Dec 15 09:33:35 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:35 sogod [18602]: SOGoRootPage successful login for user '012345...@example.com ' - expire = -1 grace = -1 Dec 15 09:33:35 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:36 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:36 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:36 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:36 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:37 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:37 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:37 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:37 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:38 sogod [18602]: [WARN] 0x0x1a25f338[UIxEnvelopeAddressFormatter] unexpected envelope address: 0x0x1a6b37b8[NGImap4EnvelopeAddress]: mailbox='' host='MISSING_DOMAIN' Dec 15 09:33:39 sogod [18602]: [WARN] 0x0x1a25f338[UIxEnvelopeAddressFormatter] unexpected envelope address: 0x0x1a434df8[NGImap4EnvelopeAddress]: Dec 15 09:33:43 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:43 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:43 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:43 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:44 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:44 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:44 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:44 sogod [18602]: 0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:48 sogod [18602]: [ERROR] 0x0x19254068[UIxMailRenderingContext] found no viewer for MIME type: application/msword Dec 15 09:33:50 sogod [18602]: [ERROR] 0x0x186fd538[UIxMailRenderingContext] found no viewer for MIME type: application/msword Dec 15 09:33:51 sogod [18602]: [WARN] 0x0x1a6a5618[UIxEnvelopeAddressFormatter] unexpected
Re: [SOGo] sogod hanging in the same request for ... minutes
On 07/02/12 11:12, Patrick Ben Koetter wrote: * Patrick Ben Koetterusers@sogo.nu: We're running SOGo for a customer with 25k accounts. Occasionally sogod locks up and users cannot be served. In such cases, the log below is from December, 15th 2011, sogod reports it has been hanging for a while: Happened again today and people can't login on the Webinterface: Feb 07 11:47:48 sogod [14423]: [WARN]0x0x128e6848[WOWatchDogChild] pid 24369 has been hanging in the same request for 6 minutes Feb 07 11:48:30 sogod [14423]: [WARN]0x0x128e6608[WOWatchDogChild] pid 16354 has been hanging in the same request for 2 minutes Feb 07 11:48:31 sogod [14423]: [WARN]0x0x128e75a8[WOWatchDogChild] pid 17308 has been hanging in the same request for 2 minutes ... Feb 07 11:51:53 sogod [14423]: [WARN]0x0x128e6848[WOWatchDogChild] pid 24369 has been hanging in the same request for 2 minutes Feb 07 11:52:35 sogod [14423]: [WARN]0x0x128e6608[WOWatchDogChild] pid 16354 has been hanging in the same request for 2 minutes Dec 15 10:27:06 sogod [953]: [WARN]0x0x1757e948[WOWatchDogChild] pid 18602 has been hanging in the same request for 7 minutes Looking for the PID I get log like this: Dec 15 09:33:35 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:35 sogod [18602]: SOGoRootPage successful login for user '012345...@example.com ' - expire = -1 grace = -1 Dec 15 09:33:35 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:36 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:36 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:36 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:36 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:37 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:37 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:37 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:37 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:38 sogod [18602]: [WARN]0x0x1a25f338[UIxEnvelopeAddressFormatter] unexpected envelope address:0x0x1a6b37b8[NGImap4EnvelopeAddress]: mailbox='' host='MISSING_DOMAIN' Dec 15 09:33:39 sogod [18602]: [WARN]0x0x1a25f338[UIxEnvelopeAddressFormatter] unexpected envelope address:0x0x1a434df8[NGImap4EnvelopeAddress]: Dec 15 09:33:43 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:43 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:43 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:43 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:44 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:44 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:44 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:44 sogod [18602]:0x0x177f6518[SOGoCache] an error occurred when caching value for key '012345...@example.com +attributes': CLIENT ERROR Dec 15 09:33:48 sogod [18602]: [ERROR]0x0x19254068[UIxMailRenderingContext] found no viewer for MIME type: application/msword Dec 15 09:33:50 sogod [18602]: [ERROR]0x0x186fd538[UIxMailRenderingContext] found no viewer for MIME type: application/msword Dec 15 09:33:51 sogod [18602]: [WARN]0x0x1a6a5618[UIxEnvelopeAddressFormatter] unexpected envelope address:0x0x1a6c4718[NGImap4EnvelopeAddress]:
Re: [SOGo] sogod hanging in the same request for ... minutes
On 07/02/12 06:12, Patrick Ben Koetter wrote: Happened again today and people can't login on the Webinterface: Feb 07 11:47:48 sogod [14423]: [WARN]0x0x128e6848[WOWatchDogChild] pid 24369 has been hanging in the same request for 6 minutes Feb 07 11:48:30 sogod [14423]: [WARN]0x0x128e6608[WOWatchDogChild] pid 16354 has been hanging in the same request for 2 minutes Feb 07 11:48:31 sogod [14423]: [WARN]0x0x128e75a8[WOWatchDogChild] pid 17308 has been hanging in the same request for 2 minutes ... Feb 07 11:51:53 sogod [14423]: [WARN]0x0x128e6848[WOWatchDogChild] pid 24369 has been hanging in the same request for 2 minutes Feb 07 11:52:35 sogod [14423]: [WARN]0x0x128e6608[WOWatchDogChild] pid 16354 has been hanging in the same request for 2 minutes Make sure you have the debug symbols installed for SOGo, SOPE and GNUstep and when that happens, attach to a pid using gdb (for example, pid 24369) and produce a stack trace. It'll show you where the process has been hanging - thus the real cause of your issue. -- Ludovic Marcotte lmarco...@inverse.ca :: +1.514.755.3630 :: www.inverse.ca Inverse inc. :: Leaders behind SOGo (www.sogo.nu) and PacketFence (www.packetfence.org) -- users@sogo.nu https://inverse.ca/sogo/lists
