On possibly "finer" dependencies in s6-rc
* "opportunist" dependencies: According to `s6-rc:src/s6-rc/s6-rc-compile.c' as of current trunk, unknown services in `dependencies' will make rc-compile complain and die, which is the optimal behaviour with mandatory dependencies. Nevertheless, I think it can be helpful to also support "opportunist" dependencies: if said service is not enabled, it is silently ignored; if said service is enabled, the dependency on it is considered in the dependency resolution process. If this is supported, service definition can be distributed in a more uniform manner (cf. OpenRC's runscripts). Of course, mandatory and opportunist dependencies should be specified separately; for example, `need()' and `use()' are used respectively in OpenRC for these requirements. * Online `OR'/virtual dependencies: According to `s6-rc:doc/faq.html' as of current trunk, `OR'/virtual dependencies are handled offline. But I think online virtuals still have use that do not have offline equivalents: apart from interchangeable implementations like openssh/dropbear, there can also be multiple enabled services providing one functionality at the same time. For example, with a laptop, it's common and useful to have `eth0' and `wlan0' both providing network access, and a network-dependent service can start when either is up. In my view (and probably the opinions of quite a few other people, see discussion on the DNG list), parallelism in rc is mostly appealing to desktop, not server users. Therefore, `eth0'/`wlan0' and similar use cases are not insignificant, and thus it is worth the hassle to implement online virtuals. -- My current OpenPGP key: 4096R/0xE18262B5D9BF213A (expires: 2017.1.1) D69C 1828 2BF2 755D C383 D7B2 E182 62B5 D9BF 213A
Re: On possibly "finer" dependencies in s6-rc
On Tue, Sep 29, 2015 at 10:46:02AM +0800, Casper Ti. Vector wrote: > * "opportunist" dependencies: After some more thought, I found one may as well supply a `mandatory' and `opportunist' file for each service, and write a preprocessor program to auto-generate the `dependencies' file, like this one: > #!/bin/sh - > # Usage: echo $list_of_enabled_services | > # ./"$0" mandatory opportunist > dependencies > tr -s ' ' '\n' | > grep -xF "$2" | > cat "$1" - So this mechanism can be left out of s6-rc; however, whether it *should* be might still be arguable. > ... If this is supported, service definition can be distributed in a > more uniform manner (cf. OpenRC's runscripts). Clarification: by "distributed", I mean "provided by a distribution". In other words, distributions have differently tailored service definition repositories, but each distribution (ideally) has only one repository that is uniform to its users. -- My current OpenPGP key: 4096R/0xE18262B5D9BF213A (expires: 2017.1.1) D69C 1828 2BF2 755D C383 D7B2 E182 62B5 D9BF 213A
Re: On possibly "finer" dependencies in s6-rc
On Tue, Sep 29, 2015 at 09:29:23PM +0800, Casper Ti. Vector wrote: > tr -s ' ' '\n' | grep -xF "$2" | cat "$1" - Should be: tr -s ' ' '\n' | grep -xFf "$2" | cat "$1" - -- My current OpenPGP key: 4096R/0xE18262B5D9BF213A (expires: 2017.1.1) D69C 1828 2BF2 755D C383 D7B2 E182 62B5 D9BF 213A
Re: On possibly "finer" dependencies in s6-rc
On 29/09/2015 04:46, Casper Ti. Vector wrote: Nevertheless, I think it can be helpful to also support "opportunist" dependencies: if said service is not enabled, it is silently ignored; if said service is enabled, the dependency on it is considered in the dependency resolution process. As you subsequently wrote, this can be left out of s6-rc. An "opportunist" dependency is not a real dependency, since the service can start without it; if you want to include it, a preprocessor for source directories is the exact right thing to use. * Online `OR'/virtual dependencies: (...) For example, with a laptop, it's common and useful to have `eth0' and `wlan0' both providing network access, and a network-dependent service can start when either is up. That kind of dependency is still infrequent compared to "normal" dependencies; I don't think it's unreasonable to maintain a set of compiled databases to address that case. The size of the set may grow exponentially, but if it becomes larger than 4 or 8 databases, I tend to think it's an abuse of virtual dependencies; I can be convinced otherwise with a real-world example that cannot be solved another way. Having several compiled databases and switching between them with s6-rc-update is normal procedure. I worked on s6-rc-update for a long time to make sure that this procedure would be as painless as possible, even if it has to be done on a semi-regular basis. Switching between eth0 and wlan0 is a typical case of this, and as a matter of fact, it's *exactly* what I'm doing on my home router. :) -- Laurent
Re: On possibly "finer" dependencies in s6-rc
I see. I agree that the typical use case of this feature is almost limited to the `eth0'/`wlan0' scenario and cannot think of any other concrete use case that is compelling. Nevertheless, I still think this make s6-rc "incomplete" mechanism-wise (sorry if that appears like insulting): opportunist dependencies support can be implemented using a preprocessor, without requiring unreasonable hacks or incurring any feature loss; on the other hand, as a mechanism, the online virtual is demanding to implement in an RC system, but might be even much more difficult to implement elsewhere. More than that, this mechanism fits naturally into an RC system, so it is architecturally a reasonable part of s6-rc. Risking the possibility of being regarded as insulting, I respectfully say that your way of `eth0'/`wlan0' switching is a choice of policy, which does not make up for the incompleteness in the set of mechanism provided by s6-rc. Sorry again for my possibly irritating language. Hope these make sense. On Tue, Sep 29, 2015 at 03:45:41PM +0200, Laurent Bercot wrote: > That kind of dependency is still infrequent compared to "normal" > dependencies; I don't think it's unreasonable to maintain a set of > compiled databases to address that case. The size of the set may > grow exponentially, but if it becomes larger than 4 or 8 databases, > I tend to think it's an abuse of virtual dependencies; I can be > convinced otherwise with a real-world example that cannot be solved > another way. > > Having several compiled databases and switching between them with > s6-rc-update is normal procedure. I worked on s6-rc-update for a long > time to make sure that this procedure would be as painless as possible, > even if it has to be done on a semi-regular basis. Switching between > eth0 and wlan0 is a typical case of this, and as a matter of fact, > it's *exactly* what I'm doing on my home router. :) -- My current OpenPGP key: 4096R/0xE18262B5D9BF213A (expires: 2017.1.1) D69C 1828 2BF2 755D C383 D7B2 E182 62B5 D9BF 213A
Re: On possibly "finer" dependencies in s6-rc
Sorry, but please allow me to append my previously forgotten conclusion: "Make it simple, but not over-simple". On Tue, Sep 29, 2015 at 10:28:30PM +0800, Casper Ti. Vector wrote: > Sorry again for my possibly irritating language. Hope these make sense. -- My current OpenPGP key: 4096R/0xE18262B5D9BF213A (expires: 2017.1.1) D69C 1828 2BF2 755D C383 D7B2 E182 62B5 D9BF 213A
Re: On possibly "finer" dependencies in s6-rc
On 29/09/2015 16:28, Casper Ti. Vector wrote: as a mechanism, the online virtual is demanding to implement in an RC system, but might be even much more difficult to implement elsewhere. More than that, this mechanism fits naturally into an RC system, so it is architecturally a reasonable part of s6-rc. On a theoretical level, I tend to agree; and I will definitely think about handling virtual services in s6-rc if a real-world case happens. The main reason why I didn't do it is that it increases complexity of the dependency engine *considerably*; and in practice, as long as your online set of virtual dependencies isn't too high, it's still simpler (in terms of code maintainability) to keep a set of compiled databases around and switch them with s6-rc-update. I agree that it defers some of the burden to the user, a burden that should be handled by s6-rc itself. I could work on some scripts to automate that kind of database management, if you think it would be useful. say that your way of `eth0'/`wlan0' switching is a choice of policy, which does not make up for the incompleteness in the set of mechanism provided by s6-rc. I think you're right in principle; I just compromised here in the name of practicality. It's definitely a possibility of evolution for s6-rc if it becomes a real issue. -- Laurent
Re: On possibly "finer" dependencies in s6-rc
[Just accidentally sent as private mail; reposting verbatim.] I'm glad that possibility still exists. Thanks :) On Tue, Sep 29, 2015 at 06:00:40PM +0200, Laurent Bercot wrote: > I just compromised here in the name of practicality. It's definitely > a possibility of evolution for s6-rc if it becomes a real issue. -- My current OpenPGP key: 4096R/0xE18262B5D9BF213A (expires: 2017.1.1) D69C 1828 2BF2 755D C383 D7B2 E182 62B5 D9BF 213A
Re: On possibly "finer" dependencies in s6-rc
What about a `dnsmasq' service depending on serveral `dnscrypt-proxy' instances for failover, and can start when any of the instances become ready? On Tue, Sep 29, 2015 at 06:00:40PM +0200, Laurent Bercot wrote: > On a theoretical level, I tend to agree; and I will definitely > think about handling virtual services in s6-rc if a real-world case > happens. -- My current OpenPGP key: RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19) 7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C
Re: On possibly "finer" dependencies in s6-rc
What about a `dnsmasq' service depending on serveral `dnscrypt-proxy' instances for failover, and can start when any of the instances become ready? Call it bad faith from me, or whatever you like, but I really dislike the idea of implementing disjunctions at the service manager level. Disjunctions create nondeterministic startup sequences, which can lead to heisenbugs faster than you can say "systemd". For that use case, I would delegate the disjunction to the service itself. I would have a unique "dnscrypt-proxy" service handling the failover instances itself, and "dnsmasq" would depend on that "dnscrypt-proxy" service. The failover instances can still be supervised and managed by regular s6 primitives; s6-rc would just not see them as separate services. If that kind of setup can be generalized, I'm not opposed to adding a "failover instance manager" set of programs to s6, because I believe it's reasonably in scope. I would much rather do that than add disjunction management to s6-rc. -- Laurent
Re: On possibly "finer" dependencies in s6-rc
I just realised that we can set up a oneshot for the disjunction with its dependencies being the intersection of the dependencies of all services in the disjunction, with the `up' and `down' being something like `s6-svwait -o -[ud] srv1 srv2 ...' ([ud]: `u' for `up', `d' for `down'), so perhaps no change is need after all in s6-rc. Nevertheless, s6-svwait -o seems to also fail if some failure occurs before the wanted state is reached, eg. one `dnscrypt-proxy' instance reporting permanent failure before another one comes up successfully. So we might need to add an option to ignore such failures. On Tue, Feb 14, 2017 at 09:31:47PM +, Laurent Bercot wrote: > If that kind of setup can be generalized, I'm not opposed to adding a > "failover instance manager" set of programs to s6, because I believe it's > reasonably in scope. I would much rather do that than add disjunction > management to s6-rc. -- My current OpenPGP key: RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19) 7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C
Re: On possibly "finer" dependencies in s6-rc
Nevertheless, s6-svwait -o seems to also fail if some failure occurs before the wanted state is reached, eg. one `dnscrypt-proxy' instance reporting permanent failure before another one comes up successfully. So we might need to add an option to ignore such failures. Yes, that's a good point. I'll think about it. However, I'd like to insist that permanent failure is a big deal, because it changes the wanted state of a process without user input; so it really should not be silently ignored. If someone is explicitly waiting for a service that reports permanent failure, I'd like to use that opportunity to warn that user that something serious happened, even if the command can still succeed because the user asked for a disjunction and another service can still come up. Does a warning message to stderr when this happens sound appropriate to you? -- Laurent
Re: On possibly "finer" dependencies in s6-rc
Yes, I think this is the correct behaviour: if the user does not want it, the warnings can be somehow filtered; on the other hand, there would not be a trivial way to know such failures if the user wants it but there is no warning in the first place. By the way, if this change is done in s6-svwait, perhaps the s6-rc FAQ can mention that dynamic (i.e. without using s6-rc-update manually) virtual dependencies can be implemented using s6-svwait oneshots? On Tue, May 02, 2017 at 08:59:05AM +, Laurent Bercot wrote: > Does a warning message to stderr when this happens sound appropriate > to you? -- My current OpenPGP key: RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19) 7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C
Re: On possibly "finer" dependencies in s6-rc
By the way, if this change is done in s6-svwait, perhaps the s6-rc FAQ can mention that dynamic (i.e. without using s6-rc-update manually) virtual dependencies can be implemented using s6-svwait oneshots? Probably not. If anything, I'll think about the use of s6-svwait oneshots a bit more, and if I assess it's the correct, or as correct as it gets, way to implement disjunctions, I'll add a mechanism in s6-rc-compile to encapsulate it and auto-generate hidden oneshot services - because those "utility" services should not be managed by users - they should be part of the magic instead. In the meantime, congratulations, you've figured out a way to do it; enjoy it, and consider it a reward for your cleverness. As to people who are not as smart or as s6-rc-savvy, it's probably best if they just stay the heck away from disjunctions until they're officially supported. :P -- Laurent
Re: On possibly "finer" dependencies in s6-rc
That would be much cleaner. Thanks :) On Tue, May 02, 2017 at 02:40:56PM +, Laurent Bercot wrote: > Probably not. If anything, I'll think about the use of s6-svwait > oneshots > a bit more, and if I assess it's the correct, or as correct as it gets, > way to implement disjunctions, I'll add a mechanism in s6-rc-compile to > encapsulate it and auto-generate hidden oneshot services - because > those "utility" services should not be managed by users - they should > be part of the magic instead. -- My current OpenPGP key: RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19) 7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C
Re: On possibly "finer" dependencies in s6-rc
Well, I found this to be fairly over-simplified. If all members of the disjunction are longruns, using s6-svwait is OK; however this is not directly extensible to oneshots and bundles. Nevertheless, I think the idea of using a separate command to wait for the disjunction is still architecturally promising: no change in the s6-rc *command*, just one more internal command in the s6-rc *suite*; and as with s6-svwait, we can inspect the procedure of state transition by simply listing the processes. On Tue, May 02, 2017 at 11:33:56AM +0800, Casper Ti. Vector wrote: > perhaps no change is need after all in s6-rc. -- My current OpenPGP key: RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19) 7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C
