Re: Log rotation issue with runit
On Sat, 29 Dec 2018 18:33:06 + Dmitry Bogatov wrote: > [2018-12-27 08:07] Steve Litt > > > [ Dmitry Bogatov ] > > > No, it is reproducible. See end of bug thread. > > > > Hi Dmitry, > > > > Just so we're all on the same page, do you mean the end of > > https://bugs.debian.org/916230 ? Could you please provide a message > > number? I briefly looked over https://bugs.debian.org/916230, paying > > particular attention to the later half, and could see no > > reproduction sequence articulated. But sometimes I miss things. > > Sure. Message 17: > > Offending .u file is created by rename(2) call at line 532, in > logdir_open() function. It happens, when 'current' file exists, > non-executable and non-empty. > [...] OK, so you can reproduce it with echo bogus > current; chmod a-x current That's pretty bizarre logic. I can understand why an empty current would be overwritten --- what do you lose. But why only if it's non-executable? Are they using the executable bit as some sort of flag to keep processes from overwriting each others' writes to the file? I once wrote a program in which, when an invocation of my program opened a file, it set the file to read-write, and other invocations of my program would decline to try to open a read-write version of the file. Bizarre, but it worked, and no process clobbered the other. I guess what I'm asking is this: Are you sure the original poster's (OP's) .u files were caused by the rename call when non-empty non-exec current exists, or is he experiencing a different reproduction sequence. I still wouldn't mess with the existing code. I don't think anyone here is positive of the purpose of the logic you described, and therefore what side effects would happen if it were modified. The OP didn't have enough .u files to really inconvenience him, and there are other ways of getting rid of .u files. And the OP could switch to s6. SteveT Steve Litt December 2018 featured book: Rapid Learning for the 21st Century http://www.troubleshooters.com/rl21
Re: Log rotation issue with runit
El sáb., 29 dic. 2018 a las 15:33, Dmitry Bogatov escribió: > > > > And this issue happens not only on crash, it happens after every > > > termination of svlogd, due any signal. I would agree that SIGKILL is > > > crash, but issue reproduces with SIGINT and SIGTERM. > > > > SIGTERM should make svlogd exit cleanly, are you sure? It does for me, > > and when restarted, does not create any .u file (using Gentoo's runit, > > which is upstream runit plus a minor patch to the makefile). > > Wierd. I tried again, and failed to reproduced issue with SIGTERM this > time. Good. This is the expected behaviour of SIGTERM. > What do you mean, `supervision tree not not being teared down properly'? First of all, I should have written "not being torn down properly" :P With "properly" I mean with all runsv and runsvdir processes exiting cleanly. Either by sending runsvdir a SIGHUP signal, which makes it take care of its runsv children, of by sending runsvdir a SIGTERM signal and then making all runsv processes exit one by one with 'sv exit' (or SIGTERM, which is equivalent). Each runsv will then close the writing end of the logger's pipe before exiting, if there is one, making the logger finalize the 'current' file, set its permissions to 0744, and exit as well (because it detects EOF in its stdin). So next time svlogd is started, no .u file. I know this to work as described on Gentoo. If runit is the init system, process 1 sends a SIGTERM signal to the runsvdir process that /etc/runit/2 replaces itself with during the shutdown sequence, so it is the task of /etc/runit/3 to make supervised processes exit, as well as their runsv supervisors. Debian's runit-2.1.2-19 'stock' /etc/runit/3 file is this one, right? * https://salsa.debian.org/runit-team/runit/raw/a45cbc4ec73e1e293d372147ad5ab3f1d2a9/debian/contrib/3 Here, the 'sv exit /etc/service/*' that follows 'sv force-stop /etc/service/*' takes care of making the runsv processes exit. The bug reporter's /etc/runit/3 is a modified one, so that's what I would look next. G.
Re: Log rotation issue with runit
[2018-12-27 20:39] Guillermo > El jue., 27 dic. 2018 a las 6:36, Dmitry Bogatov escribió: > > > > And this issue happens not only on crash, it happens after every > > termination of svlogd, due any signal. I would agree that SIGKILL is > > crash, but issue reproduces with SIGINT and SIGTERM. > > SIGTERM should make svlogd exit cleanly, are you sure? It does for me, > and when restarted, does not create any .u file (using Gentoo's runit, > which is upstream runit plus a minor patch to the makefile). Wierd. I tried again, and failed to reproduced issue with SIGTERM this time. So seems that problem is that svlogd is killed by some other signal. It is going to be quite a long debugging session. > > /etc/runit/3 changed [not included] > > What does this mean? Is this Debian with runit as the init system and > a modified /etc/runit/3 instead of the one supplied with the Debian > package? Yes. > I'm going to make a guess here and say that the supervision tree is > not being teared down properly, resulting in svlogd processes not > exiting cleanly and producing lots of .u files, compounded with the > fact that they are never deleted during rotations because of the fix > to bug #878476... What do you mean, `supervision tree not not being teared down properly'?
Re: Log rotation issue with runit
[2018-12-27 08:07] Steve Litt > > [ Dmitry Bogatov ] > > No, it is reproducible. See end of bug thread. > > Hi Dmitry, > > Just so we're all on the same page, do you mean the end of > https://bugs.debian.org/916230 ? Could you please provide a message > number? I briefly looked over https://bugs.debian.org/916230, paying > particular attention to the later half, and could see no reproduction > sequence articulated. But sometimes I miss things. Sure. Message 17: Offending .u file is created by rename(2) call at line 532, in logdir_open() function. It happens, when 'current' file exists, non-executable and non-empty. [...]
