Re: I/O stats for lmtp, imap and pop
On Sat, Jan 07, 2012 at 02:50:40PM +0100, Jeroen van Meeuwen (Kolab Systems) wrote: I guess what remains is to figure out what other (current?) platforms support, and if the code can be made so to be compatible across the board. I don't think we currently have a complete and exhaustive list of supported platforms (and versions), let alone points of contact in the community that can verify certain things on certain platforms for us. I can work on making this list happen, though. Can we use the wiki for this? Put up a list of platforms that it's known to work on, and any notes - like, OpenBSD will suck because it doesn't have a good enough MMAP. Bron.
Re: I/O stats for lmtp, imap and pop
getrusage() works on all platforms, AFAIK, at least in the POSIX sense: the fields are there, and they may or may not be filled in, but everything will compile. This is why I wonder if specifying a what usage ought to be logged (including nothing) wouldn't make sense. If you're on a platform that doesn't report I/O, that's your problem. If you aren't concerned with usage reporting, turn it off. :wes
Re: I/O stats for lmtp, imap and pop
On 2011-07-13 8:43, Olivier ROLAND wrote: Thanks, you are perfectly right :-) 2011/7/13 Wesley Craig wescr...@columbia.edu: I'd think you'd want to add this to telemetry_rusage(). Seems like you can get this data from getrusage() since kernel 2.6.22? Hi there, apologies for bumping the old topic, but in preparation for what I think I want to have included in 2.5 (now the master branch still), I successfully merged the two branches. I can, and would like to, push this out to the git.cyrusimap.org/cyrus-imapd master branch for inclusion as is in 2.5, if there's no objections. Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: I/O stats for lmtp, imap and pop
On Fri, Jan 06, 2012 at 05:15:48PM +0100, Jeroen van Meeuwen (Kolab Systems) wrote: On 2011-07-13 8:43, Olivier ROLAND wrote: Thanks, you are perfectly right :-) 2011/7/13 Wesley Craig wescr...@columbia.edu: I'd think you'd want to add this to telemetry_rusage(). Seems like you can get this data from getrusage() since kernel 2.6.22? Hi there, apologies for bumping the old topic, but in preparation for what I think I want to have included in 2.5 (now the master branch still), I successfully merged the two branches. I can, and would like to, push this out to the git.cyrusimap.org/cyrus-imapd master branch for inclusion as is in 2.5, if there's no objections. Kind regards, Jeroen van Meeuwen Sounds good. Apologies for not following up on that thread. Bron.
Re: I/O stats for lmtp, imap and pop
On 2012-01-06 17:47, Bron Gondwana wrote: On Fri, Jan 06, 2012 at 05:15:48PM +0100, Jeroen van Meeuwen (Kolab Systems) wrote: I can, and would like to, push this out to the git.cyrusimap.org/cyrus-imapd master branch for inclusion as is in 2.5, if there's no objections. Sounds good. Done. Apologies for not following up on that thread. Don't worry about it, it gives me something to do ;-) Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: I/O stats for lmtp, imap and pop
2011/7/13 Wesley Craig wescr...@columbia.edu: I'd think you'd want to add this to telemetry_rusage(). Seems like you can get this data from getrusage() since kernel 2.6.22? Hi there, apologies for bumping the old topic, but in preparation for what I think I want to have included in 2.5 (now the master branch still), I successfully merged the two branches. I can, and would like to, push this out to the git.cyrusimap.org/cyrus-imapd master branch for inclusion as is in 2.5, if there's no objections. Kind regards, Jeroen van Meeuwen Hi Jeroen, Just to be clear. The patch has not be modified to use telemetry_rusage() The philosophy of telemetry_rusage seem to be : must be POSIX guaranteed Don't know if we really want specific kernel feature here. But I agree with Wesley too :-) Mixed opinion :-/ Regards, Olivier ROLAND
Re: I/O stats for lmtp, imap and pop
2012/1/6 Jeroen van Meeuwen (Kolab Systems) vanmeeu...@kolabsys.com: On 2012-01-06 18:17, Olivier ROLAND wrote: Jeroen van Meeuwen wrote: 2011/7/13 Wesley Craig wescr...@columbia.edu: I'd think you'd want to add this to telemetry_rusage(). Seems like you can get this data from getrusage() since kernel 2.6.22? I can, and would like to, push this out to the git.cyrusimap.org/cyrus-imapd master branch for inclusion as is in 2.5, if there's no objections. Just to be clear. The patch has not be modified to use telemetry_rusage() The philosophy of telemetry_rusage seem to be : must be POSIX guaranteed Don't know if we really want specific kernel feature here. But I agree with Wesley too :-) Can you then please elaborate? I see one use of getrusage(), in imap/telemetry.c (line ~107), in the function telemetry_rusage(), introduced by Bron Gondwana in June 2010 (f3d1d19e). What seems to be the nature of the disagreement? When I wrote the patch I was not aware of the telemetry code and that getrusage could also get this data. Wesley is right, some could see redundancy between my code and telemetry_rusage but obviously Bron could have get much more data from getrusage() and voluntarily restrain to POSIX. To resume : 1) I read /proc/self/io but getrusage() is probably a better way to get the data 2) Potential redundancy with telemetry_rusage But if Bron is OK with my patch as is, I'm OK too :-) Regards, Olivier ROLAND
Re: I/O stats for lmtp, imap and pop
I don't see a good reason to include code that digs around in /proc when getrusage() has the exact same data. I think adding, e.g., I/O to the telemetry calls would be peachy, either as a runtime option or if someone wants to exercise their autoconf-foo, at build time as appropriate to the platform in question. When I wrote the getruage() code, I included only system user CPU time because the platform I was working on provided them -- because POSIX requires them. POSIX similarly requires that the other fields be present, but they didn't happen to have any content when I wrote that code, on the platform I was working on. The present day version of the platform in question (Linux) provides much more data, and most non-Linux platforms have provided more than just CPU usage for many years. :wes
Re: I/O stats for lmtp, imap and pop
Thanks, you are perfectly right :-) 2011/7/13 Wesley Craig wescr...@columbia.edu: I'd think you'd want to add this to telemetry_rusage(). Seems like you can get this data from getrusage() since kernel 2.6.22? :wes On 12 Jul 2011, at 12:14, Olivier ROLAND wrote: Linux kernel 2.6.20 and later supports per process I/O accounting. You can access every process/thread's I/O read/write values by using /proc filesystem. You can check if your kernel has built with I/O account by just simply checking /proc/self/io file. If it exists then you have I/O accounting built-in. I use that to have real I/O stats for lmtp, imap and pop in Cyrus. You just need ioconf = yes in your conf to activate I/O stats. The code autodetect if your kernel support I/O account and disable the stats if not. You can find the git branch with the code here : https://github.com/worldline-messaging/cyrus-imapd/tree/io-stat
Re: I/O stats for lmtp, imap and pop
I'd think you'd want to add this to telemetry_rusage(). Seems like you can get this data from getrusage() since kernel 2.6.22? :wes On 12 Jul 2011, at 12:14, Olivier ROLAND wrote: Linux kernel 2.6.20 and later supports per process I/O accounting. You can access every process/thread's I/O read/write values by using /proc filesystem. You can check if your kernel has built with I/O account by just simply checking /proc/self/io file. If it exists then you have I/O accounting built-in. I use that to have real I/O stats for lmtp, imap and pop in Cyrus. You just need ioconf = yes in your conf to activate I/O stats. The code autodetect if your kernel support I/O account and disable the stats if not. You can find the git branch with the code here : https://github.com/worldline-messaging/cyrus-imapd/tree/io-stat
