Re: [Dovecot] dovecot stats
Am 08.05.2013 11:25, schrieb Jan-Frode Myklebust: Thanks, nice graphs. I've attached a graph over LMTP delays per minute as seen from the postfix side on one of our servers. This includes delays caused by both delivery to dovecot LMTP, and also LMTP communication internally on the mailservers between postfix and amavis. Unfortunately it says nothing about the delivery time to each individual dovecot backend, since these are hiding behind dovecot director, and therefor we have no way of knowing which of our backends are slow (if any). -jf i am not sure, i ll have to read more doku perhaps it makes sense to use delay stuff in log for analyse i.e after all i am not using directors, i have real loadbalancers private/dovecot-lmtp], delay=1.4, delays=1.2/0/0/0.17, dsn=2.0.0, status=sent Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Re: [Dovecot] dovecot stats
On Mon, May 6, 2013 at 4:37 PM, Timo Sirainen t...@iki.fi wrote: On 29.4.2013, at 15.30, Jan-Frode Myklebust janfr...@tanso.net wrote: Is it possible to collect info about POP3 and LMTP commands also ? No. I think they would be pretty boring statistics, since with POP3 pretty much everything causing disk I/O or CPU usage would be RETRs and with LMTP everything would be DATAs. I think knowing the timings of writing messages to disk / reading from disk would be very interesting and relevant data. Especially for us with mostly POP3 clients, where LMTP DATAs and POP3 RETRs probably is accounting for major parts of the server load. Also, is doveadm stats dump command telling me the results of all commands that has finished the last stats_command_min_time, or will it maybe contain much more than 1 minute of activity ? It can contain much more. The stats process will keep as much data in memory as possible until it reaches the stats_memory_limit. The doveadm stats dump lists everything that the stats process knows. Ok, then I guess we'll need to limit our stats dumps based on last_seen. -jf
Re: [Dovecot] dovecot stats
Am 08.05.2013 10:25, schrieb Jan-Frode Myklebust: On Mon, May 6, 2013 at 4:37 PM, Timo Sirainen t...@iki.fi wrote: On 29.4.2013, at 15.30, Jan-Frode Myklebust janfr...@tanso.net wrote: Is it possible to collect info about POP3 and LMTP commands also ? No. I think they would be pretty boring statistics, since with POP3 pretty much everything causing disk I/O or CPU usage would be RETRs and with LMTP everything would be DATAs. I think knowing the timings of writing messages to disk / reading from disk would be very interesting and relevant data. Especially for us with mostly POP3 clients, where LMTP DATAs and POP3 RETRs probably is accounting for major parts of the server load. no urgent need for stats i count pop3 logins with xymon out of syslog in realtime, also logwatch reports lmtp/pop3/imap daily for each user by cron in daily terms perhaps have a look at http://sys4.de/de/blog/2013/01/10/xymon-dovecot-count-imap-pop3-logins-graph-central-rsyslog-server-ubuntu-lucid/ if you have xymon monitoring etc installed you have other stuff like cpu , mem, disk io too i ll plan to write some solution for xymon retrieve data out of dovecot stats too Also, is doveadm stats dump command telling me the results of all commands that has finished the last stats_command_min_time, or will it maybe contain much more than 1 minute of activity ? It can contain much more. The stats process will keep as much data in memory as possible until it reaches the stats_memory_limit. The doveadm stats dump lists everything that the stats process knows. Ok, then I guess we'll need to limit our stats dumps based on last_seen. -jf Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Re: [Dovecot] dovecot stats
Thanks, nice graphs. I've attached a graph over LMTP delays per minute as seen from the postfix side on one of our servers. This includes delays caused by both delivery to dovecot LMTP, and also LMTP communication internally on the mailservers between postfix and amavis. Unfortunately it says nothing about the delivery time to each individual dovecot backend, since these are hiding behind dovecot director, and therefor we have no way of knowing which of our backends are slow (if any). -jf attachment: lmtp-delays.png
Re: [Dovecot] dovecot stats
On 29.4.2013, at 15.30, Jan-Frode Myklebust janfr...@tanso.net wrote: I just upgraded one of our servers to dovecot v2.1.16 (ee), and am looking into the stats feature. Am I interpreting the wiki correct in reading that the doveadm stats dump command only returns statistics about IMAP commands? Right. Is it possible to collect info about POP3 and LMTP commands also ? No. I think they would be pretty boring statistics, since with POP3 pretty much everything causing disk I/O or CPU usage would be RETRs and with LMTP everything would be DATAs. Also, is doveadm stats dump command telling me the results of all commands that has finished the last stats_command_min_time, or will it maybe contain much more than 1 minute of activity ? It can contain much more. The stats process will keep as much data in memory as possible until it reaches the stats_memory_limit. The doveadm stats dump lists everything that the stats process knows.
[Dovecot] dovecot stats
I just upgraded one of our servers to dovecot v2.1.16 (ee), and am looking into the stats feature. Am I interpreting the wiki correct in reading that the doveadm stats dump command only returns statistics about IMAP commands? Is it possible to collect info about POP3 and LMTP commands also ? Also, is doveadm stats dump command telling me the results of all commands that has finished the last stats_command_min_time, or will it maybe contain much more than 1 minute of activity ? -jf
Re: [Dovecot] dovecot stats: useful data to gather
On 06/02/12 17:10, Daniel Parthey wrote: Patrick Ben Koetter wrote: following our discussion on dovecot stats at the LinuxTag 2012 my team and I sat down and put together a list of stat items we think to be useful in daily dovecot usage. Besides pulling together all the data we also think it would be useful to have an SNMP interface to access the stats. Our offer to create and contribute a standalone web interface for dovecot stats stands. This should be done via SNMP subagent, but how could you differentiate different dovecot instances on the same machine, different snmp ports for the subagent, or different snmp trees? I'd suggest some additional performance metrics like min/max/avg time to authenicate, establish a proxy session and perhaps include auth failure causes counters as well. I personally wouldn't want to see this implemented as an SNMP subagent but so long as the stats would be available off a local socket directly I think everyone would be happy. -K
Re: [Dovecot] dovecot stats error
On 6/22/2012 6:34 AM, Timo Sirainen wrote: Which Dovecot version? I thought I fixed this already.. I'm seeing these errors running 2.1.8 -- Kelsey Cummings - k...@corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407
Re: [Dovecot] dovecot stats error
On 8/25/2012 12:14 PM, Kelsey Cummings wrote: On 6/22/2012 6:34 AM, Timo Sirainen wrote: Which Dovecot version? I thought I fixed this already.. I'm seeing these errors running 2.1.8 Examples below, let me know if I can provide any other info Timo. In other news, we're finally migrated to dovecot from courier. WHOOO H Aug 25 12:53:37 a dovecot: stats: Error: Mail server input error: UPDATE-SESSION: stats shrank: mcache 331 332 Aug 25 12:53:37 a dovecot: stats: Error: Mail server input error: UPDATE-SESSION: stats shrank: mrbytes 180435729 204849088 Aug 25 12:53:38 a dovecot: stats: Error: Mail server input error: UPDATE-SESSION: stats shrank: mrbytes 50757363 62351358 Aug 25 12:53:38 d dovecot: stats: Error: Mail server input error: UPDATE-SESSION: stats shrank: mlpath 17451 20067 Aug 25 12:53:41 d dovecot: stats: Error: Mail server input error: UPDATE-SESSION: stats shrank: mrbytes 40483661 42086237 Aug 25 12:53:42 b dovecot: stats: Error: Mail server input error: UPDATE-SESSION: stats shrank: mrbytes 65540465 67974537 Aug 25 12:53:42 a dovecot: stats: Error: Mail server input error: UPDATE-SESSION: stats shrank: mlpath 811 946 Aug 25 12:53:43 b dovecot: stats: Error: Mail server input error: UPDATE-SESSION: stats shrank: mrbytes 220133763 221888538 Aug 25 12:53:47 a dovecot: stats: Error: Mail server input error: UPDATE-SESSION: stats shrank: mcache 13 14 Aug 25 12:53:48 c dovecot: stats: Error: Mail server input error: UPDATE-SESSION: stats shrank: mrbytes 118702153 121714865 -- Kelsey Cummings - k...@corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407
[Dovecot] dovecot stats error
Hi Timo, any idea whats this related too ? dovecot: stats: Error: Mail server input error: UPDATE-SESSION: stats shrank: mrbytes 21703727 25193928 -- Best Regards MfG Robert Schetterer
Re: [Dovecot] dovecot stats error
On 22.6.2012, at 15.34, Robert Schetterer wrote: Hi Timo, any idea whats this related too ? dovecot: stats: Error: Mail server input error: UPDATE-SESSION: stats shrank: mrbytes 21703727 25193928 Which Dovecot version? I thought I fixed this already..
Re: [Dovecot] dovecot stats: useful data to gather
On 01.06.2012 22:58, Patrick Ben Koetter wrote: [...] I sat down and put together a list of stat items we think to be useful in daily dovecot usage. Quite a list. But I believe most of those values are quite useful and I would also love to see such a rich set of measurements being available. Besides pulling together all the data we also think it would be useful to have an SNMP interface to access the stats. Our offer to create and contribute a standalone web interface for dovecot stats stands. Yes, I second that. Otherwise quite a few installation will just hook the dovecot commands to netsnmp handlers, which is not a pretty solution. Maybe dovecot could also do the SNMP for statistics that plugins provide? I'm thinking managesieve access, sieve processing or expire here. Regards Christian smime.p7s Description: S/MIME Cryptographic Signature
Re: [Dovecot] dovecot stats: useful data to gather
Patrick Ben Koetter wrote: following our discussion on dovecot stats at the LinuxTag 2012 my team and I sat down and put together a list of stat items we think to be useful in daily dovecot usage. Besides pulling together all the data we also think it would be useful to have an SNMP interface to access the stats. Our offer to create and contribute a standalone web interface for dovecot stats stands. This should be done via SNMP subagent, but how could you differentiate different dovecot instances on the same machine, different snmp ports for the subagent, or different snmp trees? Here are the stats we believe to be useful: [...] Here are the stats which I also consider to be useful: Login/Logout: - Hits/Misses for Logins via userdb cache System resources: - detailed memory usage of dovecot services (imap, worker, userdb cache) - dovecot connections to mysql database - dovecot connections to ldap - director connections vs. backend connections Regards, Daniel
[Dovecot] dovecot stats: useful data to gather
Timo, following our discussion on dovecot stats at the LinuxTag 2012 my team and I sat down and put together a list of stat items we think to be useful in daily dovecot usage. Besides pulling together all the data we also think it would be useful to have an SNMP interface to access the stats. Our offer to create and contribute a standalone web interface for dovecot stats stands. Here are the stats we believe to be useful: Login/Logout - total number login success/time - total number login failure/time - total number per authentication mechanism - total number plain sessions - total number STARTTLS sessions - total number of currently connected users (pop3/pop3s/imap/imaps/managesieve) - login names of connected users (not really stats, but great for actions regarding those uses e.g. force logout) - total number logout commands/time - total number BYE responses (autologout) Mailbox state - Inflow rate (number incoming messages/time) - Deleted rate (number \Deleted flagged messages/time) - Expunge rate (number Expunge operations/time) - total number current messages mailboxes normal storage - total number current messages mailboxes alt storage - total number read messages mailboxes normal storage - total number read messages mailboxes alt storage - per user number current messages mailboxes normal storage - per user number current messages mailboxes alt storage - per user number read messages mailboxes normal storage - per user number read messages mailboxes alt storage Mailbox Quota - total number persons under soft-quota per quota - total number persons above or equal soft-quota per quota - total number persons above or equal hard-quota per quota Performance - minimum time to write a message - maximum time to write a message - average time to write a message - minimum time to modify a message - maximum time to modify a message - average time to modify a message - minimum time to delete a message - maximum time to delete a message - average time to delete a message - minimum time search operations - maximum time search operations - average time search operations Regards, p@rick -- state of mind () http://www.state-of-mind.de Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666 Amtsgericht MünchenPartnerschaftsregister PR 563
Re: [Dovecot] dovecot stats: useful data to gather
On 1.6.2012, at 23.58, Patrick Ben Koetter wrote: Besides pulling together all the data we also think it would be useful to have an SNMP interface to access the stats. I had thought about SNMP before also, but for the current kind of stats that are exported I couldn't think of any reasonable way to export them. Here are the stats we believe to be useful: Login/Logout - total number login success/time - total number login failure/time .. I'll look at these later in more detail, but some important questions / design decisions: Currently stats process only remembers things after Dovecot was started. I don't think getting these kind of numbers would really work like that. Perhaps all of the statistics should be permanently dumped to disk every ~minute or so + at shutdown and loaded at startup, so the numbers would at least normally always just increase since the first time Dovecot was started? Mailbox state - Inflow rate (number incoming messages/time) - Deleted rate (number \Deleted flagged messages/time) These operations/time type of things I had hoped to be able to externalize :) If stats process simply gives the raw stats, the reader could do this kind of summing up. Otherwise .. well, I guess it could maybe keep track of the current ops/last 60 secs and the reader would then have to read the value about once a minute or half or something. It wouldn't give exact results though. Performance - minimum time to write a message - maximum time to write a message - average time to write a message Within last .. day? hour? minute? ..
Re: [Dovecot] dovecot stats: useful data to gather
* Timo Sirainen dovecot@dovecot.org: On 1.6.2012, at 23.58, Patrick Ben Koetter wrote: Besides pulling together all the data we also think it would be useful to have an SNMP interface to access the stats. I had thought about SNMP before also, but for the current kind of stats that are exported I couldn't think of any reasonable way to export them. I am not an expert on SNMP, others in my office are, but as I understand it there's no need for Dovecot to export the data. AFAIK Dovecot would have to offer a subagent, which could be queried by a SNMP server. If we need more knowledge on SNMP I can ask my folks on the team to give some guidance. For the moment I found this: http://net-snmp.sourceforge.net/wiki/index.php/TUT:Writing_a_Subagent Here are the stats we believe to be useful: Login/Logout - total number login success/time - total number login failure/time .. I'll look at these later in more detail, but some important questions / design decisions: Currently stats process only remembers things after Dovecot was started. I don't think getting these kind of numbers would really work like that. Perhaps all of the statistics should be permanently dumped to disk every ~minute or so + at shutdown and loaded at startup, so the numbers would at least normally always just increase since the first time Dovecot was started? ACK. My understanding is: Statistical data are moments in time. The application provides these snapshots. It is up to other protocols (e.g. SNMP) and software (e.g. RRD) to gather and create time series and also to relate data to each other in order to come up with ratios, timelines etc. This might be a good opportunity to check out Howard's MDB database (in order to get around potential future law suits concerning BDB usage ...). http://highlandsun.com/hyc/mdb/ Mailbox state - Inflow rate (number incoming messages/time) - Deleted rate (number \Deleted flagged messages/time) These operations/time type of things I had hoped to be able to externalize :) If stats process simply gives the raw stats, the reader could do this kind of summing up. Otherwise .. well, I guess it could maybe keep track of the current ops/last 60 secs and the reader would then have to read the value about once a minute or half or something. It wouldn't give exact results though. ACK. I'd externalize them too. So dump the /time aspect and only give raw data at moment of query. Performance - minimum time to write a message - maximum time to write a message - average time to write a message Within last .. day? hour? minute? .. Concerning message write time: the time the last message had to be written. In general the stats update interval should be configurable in order to adapt it to the overall system performance. Makes no sense to bring down the server by gathering stats every nano second unless one likes self-induced DOS. ;) It would probably be a useful strategy to update internal data on every event and answer SNMP queries from memory but write the data to disc every once in a while to have them when the server restarts. Besides that I don't see a use case for sharing such data between processes such as exporting them to memcache or anything alike. Do you? p@rick -- state of mind () http://www.state-of-mind.de Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666 Amtsgericht MünchenPartnerschaftsregister PR 563
