Re: Dependencies in S6
On Wed, 21 Oct 2020 13:26:25 + "Laurent Bercot" wrote: > "A is ready" means: >- if A is a oneshot: when the script has completed >- if A is a longrun: > * if there is a notification-fd file in the service definition > directory: when the daemon has notified readiness > * if there is no such file: as soon as the run script has started > (which is not a good readiness indicator; it is akin to starting B > _right after A_, essentially in parallel, so you may have race > conditions here - which is why you should always use readiness > notifications.) I seem to remember that s6 also has specially named shellscript to determine dependency readiness. So, for instance, process B requires a functioning network, the shellscript would contain a ping command or an ip command. I do this all the time, informally. SteveT Steve Litt Autumn 2020 featured book: Thriving in Tough Times http://www.troubleshooters.com/thrive
Re: Dependencies in S6
Hi Amaresh, As long as service B depends on service A, A and B will never start in parallel; B will always start when A is ready. "A is ready" means: - if A is a oneshot: when the script has completed - if A is a longrun: * if there is a notification-fd file in the service definition directory: when the daemon has notified readiness * if there is no such file: as soon as the run script has started (which is not a good readiness indicator; it is akin to starting B _right after A_, essentially in parallel, so you may have race conditions here - which is why you should always use readiness notifications.) Secnario 1: 1. A & B are one shot services in a s6-rc bundle. Service B is dependent on A. ( added in dependency file). 2. I think Service B will start after the completion of service A ( A then B, in serial manner ) is it correct ? or 3. Service B will start immediately after the service A as started (in a parallel manner) ? Serial manner, A then B. Secnario 2: 1. A is one shot service and B is a long run service. Service B is dependent on Service A. 2. Service B will start, once th service A as completed (in serial manner) ? or 3. Service B will start immediately after the service A as started (in a parallel manner) ? Serial manner, A then B. Secnario 3: 1. A is a long run service and B is a one shot service. Service B is dependent on Service A. 2. Service B will start immediately after the service A as started (in a parallel manner) ? 3. Do I need to use notification mechanism if I want to start service B after A service as reached particular point in execution Yes, you need notification; if you don't, then B will start immediately after A. Secnario 4: 1. A & B both are long run services. Service B is dependent on Service A. 2. Service B will start immediately after the service A as started (in a parallel manner) ? Same as above: almost-parallel manner if A does not use notification, serial manner if it does. If one shot service are dependent on other one shot services. They will be started in serial manner one after the other ? That is your Scenario 1 above. If the case is: "C depends on A and B, but A and B are independent", then: * A and B are started in parallel * when both A and B are ready, then C is started. Hope this helps, -- Laurent
Re: Dependencies in S6
On 21.10.20 12:10, Amaresh Kotekal wrote: Hi Team, I have some doubt on dependencies file in S6. Secnario 1: 1. A & B are one shot services in a s6-rc bundle. Service B is dependent on A. ( added in dependency file). 2. I think Service B will start after the completion of service A ( A then B, in serial manner ) is it correct ? or 3. Service B will start immediately after the service A as started (in a parallel manner) ? Secnario 2: 1. A is one shot service and B is a long run service. Service B is dependent on Service A. 2. Service B will start, once th service A as completed (in serial manner) ? or 3. Service B will start immediately after the service A as started (in a parallel manner) ? Secnario 3: 1. A is a long run service and B is a one shot service. Service B is dependent on Service A. 2. Service B will start immediately after the service A as started (in a parallel manner) ? 3. Do I need to use notification mechanism if I want to start service B after A service as reached particular point in execution Secnario 4: 1. A & B both are long run services. Service B is dependent on Service A. 2. Service B will start immediately after the service A as started (in a parallel manner) ? If one shot service are dependent on other one shot services. They will be started in serial manner one after the other ? The s6-rc service manager is always concurrent while respecting dependencies. It starts all services as soon as their requirements are satisfied. Having a single service depend on a single other service is just a special case of the more general dependency resolution s6-rc does.
Re: Dependencies in S6
El mié., 21 oct. 2020 a las 7:10, Amaresh Kotekal escribió: > > Secnario 1: > 1. A & B are one shot services in a s6-rc bundle. Service B is dependent on > A. ( added in dependency file). > 2. I think Service B will start after the completion of service A ( A then > B, in serial manner ) is it correct ? or > 3. Service B will start immediately after the service A as started (in a > parallel manner) ? #2. If A starts successfully, B will be started after A, i.e. serially. If A fails to start, B won't be started.at all. > Secnario 2: > 1. A is one shot service and B is a long run service. Service B is > dependent on Service A. Same as scenario #1. Whether B is a longrun or a oneshot does not change the behaviour. > Secnario 3: > 1. A is a long run service and B is a one shot service. Service B is > dependent on Service A. > [...] > 3. Do I need to use notification mechanism if I want to start service B > after A service as reached particular point in execution Same as scenario #1, service type does not affect the behaviour. What does depend on service type is the way s6-rc detects that state transition has completed, and the way it detects transition failure. For oneshots, transition completion happens when the oneshot's process terminates. And transition failure is determined based on the process' exit code, and, if a timeout was specified with a 'timeout-up' file, also on whether it terminates before the timeout. For longruns that support readiness notification, transition completion happens when the longrun's process notifies readiness. And if a timeout was specified with a 'timeout-up' file, transition failure is determined based on whether notification happens before the timeout. If there is no support for readiness notification, transition completion happens when the longrun's process has been spawned by its corresponding s6-supervise process, because there is no better way. If the process can be polled for readiness, you can use the s6-notifyoncheck program to hook to s6-supervise's readiness notification mechanism. * http://www.skarnet.org/software/s6/s6-notifyoncheck.html > Secnario 4: > 1. A & B both are long run services. Service B is dependent on Service A. Left as an excercise :) G.
Dependencies in S6
Hi Team, I have some doubt on dependencies file in S6. Secnario 1: 1. A & B are one shot services in a s6-rc bundle. Service B is dependent on A. ( added in dependency file). 2. I think Service B will start after the completion of service A ( A then B, in serial manner ) is it correct ? or 3. Service B will start immediately after the service A as started (in a parallel manner) ? Secnario 2: 1. A is one shot service and B is a long run service. Service B is dependent on Service A. 2. Service B will start, once th service A as completed (in serial manner) ? or 3. Service B will start immediately after the service A as started (in a parallel manner) ? Secnario 3: 1. A is a long run service and B is a one shot service. Service B is dependent on Service A. 2. Service B will start immediately after the service A as started (in a parallel manner) ? 3. Do I need to use notification mechanism if I want to start service B after A service as reached particular point in execution Secnario 4: 1. A & B both are long run services. Service B is dependent on Service A. 2. Service B will start immediately after the service A as started (in a parallel manner) ? If one shot service are dependent on other one shot services. They will be started in serial manner one after the other ? Regards, Amaresh
Re: On possibly "finer" dependencies in s6-rc
Well, I found this to be fairly over-simplified. If all members of the disjunction are longruns, using s6-svwait is OK; however this is not directly extensible to oneshots and bundles. Nevertheless, I think the idea of using a separate command to wait for the disjunction is still architecturally promising: no change in the s6-rc *command*, just one more internal command in the s6-rc *suite*; and as with s6-svwait, we can inspect the procedure of state transition by simply listing the processes. On Tue, May 02, 2017 at 11:33:56AM +0800, Casper Ti. Vector wrote: > perhaps no change is need after all in s6-rc. -- My current OpenPGP key: RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19) 7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C
Re: On possibly "finer" dependencies in s6-rc
That would be much cleaner. Thanks :) On Tue, May 02, 2017 at 02:40:56PM +, Laurent Bercot wrote: > Probably not. If anything, I'll think about the use of s6-svwait > oneshots > a bit more, and if I assess it's the correct, or as correct as it gets, > way to implement disjunctions, I'll add a mechanism in s6-rc-compile to > encapsulate it and auto-generate hidden oneshot services - because > those "utility" services should not be managed by users - they should > be part of the magic instead. -- My current OpenPGP key: RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19) 7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C
Re: On possibly "finer" dependencies in s6-rc
By the way, if this change is done in s6-svwait, perhaps the s6-rc FAQ can mention that dynamic (i.e. without using s6-rc-update manually) virtual dependencies can be implemented using s6-svwait oneshots? Probably not. If anything, I'll think about the use of s6-svwait oneshots a bit more, and if I assess it's the correct, or as correct as it gets, way to implement disjunctions, I'll add a mechanism in s6-rc-compile to encapsulate it and auto-generate hidden oneshot services - because those "utility" services should not be managed by users - they should be part of the magic instead. In the meantime, congratulations, you've figured out a way to do it; enjoy it, and consider it a reward for your cleverness. As to people who are not as smart or as s6-rc-savvy, it's probably best if they just stay the heck away from disjunctions until they're officially supported. :P -- Laurent
Re: On possibly "finer" dependencies in s6-rc
Yes, I think this is the correct behaviour: if the user does not want it, the warnings can be somehow filtered; on the other hand, there would not be a trivial way to know such failures if the user wants it but there is no warning in the first place. By the way, if this change is done in s6-svwait, perhaps the s6-rc FAQ can mention that dynamic (i.e. without using s6-rc-update manually) virtual dependencies can be implemented using s6-svwait oneshots? On Tue, May 02, 2017 at 08:59:05AM +, Laurent Bercot wrote: > Does a warning message to stderr when this happens sound appropriate > to you? -- My current OpenPGP key: RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19) 7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C
Re: On possibly "finer" dependencies in s6-rc
Nevertheless, s6-svwait -o seems to also fail if some failure occurs before the wanted state is reached, eg. one `dnscrypt-proxy' instance reporting permanent failure before another one comes up successfully. So we might need to add an option to ignore such failures. Yes, that's a good point. I'll think about it. However, I'd like to insist that permanent failure is a big deal, because it changes the wanted state of a process without user input; so it really should not be silently ignored. If someone is explicitly waiting for a service that reports permanent failure, I'd like to use that opportunity to warn that user that something serious happened, even if the command can still succeed because the user asked for a disjunction and another service can still come up. Does a warning message to stderr when this happens sound appropriate to you? -- Laurent
Re: On possibly "finer" dependencies in s6-rc
I just realised that we can set up a oneshot for the disjunction with its dependencies being the intersection of the dependencies of all services in the disjunction, with the `up' and `down' being something like `s6-svwait -o -[ud] srv1 srv2 ...' ([ud]: `u' for `up', `d' for `down'), so perhaps no change is need after all in s6-rc. Nevertheless, s6-svwait -o seems to also fail if some failure occurs before the wanted state is reached, eg. one `dnscrypt-proxy' instance reporting permanent failure before another one comes up successfully. So we might need to add an option to ignore such failures. On Tue, Feb 14, 2017 at 09:31:47PM +, Laurent Bercot wrote: > If that kind of setup can be generalized, I'm not opposed to adding a > "failover instance manager" set of programs to s6, because I believe it's > reasonably in scope. I would much rather do that than add disjunction > management to s6-rc. -- My current OpenPGP key: RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19) 7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C
Re: On possibly "finer" dependencies in s6-rc
What about a `dnsmasq' service depending on serveral `dnscrypt-proxy' instances for failover, and can start when any of the instances become ready? On Tue, Sep 29, 2015 at 06:00:40PM +0200, Laurent Bercot wrote: > On a theoretical level, I tend to agree; and I will definitely > think about handling virtual services in s6-rc if a real-world case > happens. -- My current OpenPGP key: RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19) 7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C
Re: On possibly "finer" dependencies in s6-rc
On Tue, Sep 29, 2015 at 10:46:02AM +0800, Casper Ti. Vector wrote: > * "opportunist" dependencies: After some more thought, I found one may as well supply a `mandatory' and `opportunist' file for each service, and write a preprocessor program to auto-generate the `dependencies' file, like this one: > #!/bin/sh - > # Usage: echo $list_of_enabled_services | > # ./"$0" mandatory opportunist > dependencies > tr -s ' ' '\n' | > grep -xF "$2" | > cat "$1" - So this mechanism can be left out of s6-rc; however, whether it *should* be might still be arguable. > ... If this is supported, service definition can be distributed in a > more uniform manner (cf. OpenRC's runscripts). Clarification: by "distributed", I mean "provided by a distribution". In other words, distributions have differently tailored service definition repositories, but each distribution (ideally) has only one repository that is uniform to its users. -- My current OpenPGP key: 4096R/0xE18262B5D9BF213A (expires: 2017.1.1) D69C 1828 2BF2 755D C383 D7B2 E182 62B5 D9BF 213A
Re: On possibly "finer" dependencies in s6-rc
On Tue, Sep 29, 2015 at 09:29:23PM +0800, Casper Ti. Vector wrote: > tr -s ' ' '\n' | grep -xF "$2" | cat "$1" - Should be: tr -s ' ' '\n' | grep -xFf "$2" | cat "$1" - -- My current OpenPGP key: 4096R/0xE18262B5D9BF213A (expires: 2017.1.1) D69C 1828 2BF2 755D C383 D7B2 E182 62B5 D9BF 213A
Re: On possibly "finer" dependencies in s6-rc
On 29/09/2015 04:46, Casper Ti. Vector wrote: Nevertheless, I think it can be helpful to also support "opportunist" dependencies: if said service is not enabled, it is silently ignored; if said service is enabled, the dependency on it is considered in the dependency resolution process. As you subsequently wrote, this can be left out of s6-rc. An "opportunist" dependency is not a real dependency, since the service can start without it; if you want to include it, a preprocessor for source directories is the exact right thing to use. * Online `OR'/virtual dependencies: (...) For example, with a laptop, it's common and useful to have `eth0' and `wlan0' both providing network access, and a network-dependent service can start when either is up. That kind of dependency is still infrequent compared to "normal" dependencies; I don't think it's unreasonable to maintain a set of compiled databases to address that case. The size of the set may grow exponentially, but if it becomes larger than 4 or 8 databases, I tend to think it's an abuse of virtual dependencies; I can be convinced otherwise with a real-world example that cannot be solved another way. Having several compiled databases and switching between them with s6-rc-update is normal procedure. I worked on s6-rc-update for a long time to make sure that this procedure would be as painless as possible, even if it has to be done on a semi-regular basis. Switching between eth0 and wlan0 is a typical case of this, and as a matter of fact, it's *exactly* what I'm doing on my home router. :) -- Laurent
Re: On possibly "finer" dependencies in s6-rc
Sorry, but please allow me to append my previously forgotten conclusion: "Make it simple, but not over-simple". On Tue, Sep 29, 2015 at 10:28:30PM +0800, Casper Ti. Vector wrote: > Sorry again for my possibly irritating language. Hope these make sense. -- My current OpenPGP key: 4096R/0xE18262B5D9BF213A (expires: 2017.1.1) D69C 1828 2BF2 755D C383 D7B2 E182 62B5 D9BF 213A
Re: On possibly "finer" dependencies in s6-rc
I see. I agree that the typical use case of this feature is almost limited to the `eth0'/`wlan0' scenario and cannot think of any other concrete use case that is compelling. Nevertheless, I still think this make s6-rc "incomplete" mechanism-wise (sorry if that appears like insulting): opportunist dependencies support can be implemented using a preprocessor, without requiring unreasonable hacks or incurring any feature loss; on the other hand, as a mechanism, the online virtual is demanding to implement in an RC system, but might be even much more difficult to implement elsewhere. More than that, this mechanism fits naturally into an RC system, so it is architecturally a reasonable part of s6-rc. Risking the possibility of being regarded as insulting, I respectfully say that your way of `eth0'/`wlan0' switching is a choice of policy, which does not make up for the incompleteness in the set of mechanism provided by s6-rc. Sorry again for my possibly irritating language. Hope these make sense. On Tue, Sep 29, 2015 at 03:45:41PM +0200, Laurent Bercot wrote: > That kind of dependency is still infrequent compared to "normal" > dependencies; I don't think it's unreasonable to maintain a set of > compiled databases to address that case. The size of the set may > grow exponentially, but if it becomes larger than 4 or 8 databases, > I tend to think it's an abuse of virtual dependencies; I can be > convinced otherwise with a real-world example that cannot be solved > another way. > > Having several compiled databases and switching between them with > s6-rc-update is normal procedure. I worked on s6-rc-update for a long > time to make sure that this procedure would be as painless as possible, > even if it has to be done on a semi-regular basis. Switching between > eth0 and wlan0 is a typical case of this, and as a matter of fact, > it's *exactly* what I'm doing on my home router. :) -- My current OpenPGP key: 4096R/0xE18262B5D9BF213A (expires: 2017.1.1) D69C 1828 2BF2 755D C383 D7B2 E182 62B5 D9BF 213A
Re: On possibly "finer" dependencies in s6-rc
On 29/09/2015 16:28, Casper Ti. Vector wrote: as a mechanism, the online virtual is demanding to implement in an RC system, but might be even much more difficult to implement elsewhere. More than that, this mechanism fits naturally into an RC system, so it is architecturally a reasonable part of s6-rc. On a theoretical level, I tend to agree; and I will definitely think about handling virtual services in s6-rc if a real-world case happens. The main reason why I didn't do it is that it increases complexity of the dependency engine *considerably*; and in practice, as long as your online set of virtual dependencies isn't too high, it's still simpler (in terms of code maintainability) to keep a set of compiled databases around and switch them with s6-rc-update. I agree that it defers some of the burden to the user, a burden that should be handled by s6-rc itself. I could work on some scripts to automate that kind of database management, if you think it would be useful. say that your way of `eth0'/`wlan0' switching is a choice of policy, which does not make up for the incompleteness in the set of mechanism provided by s6-rc. I think you're right in principle; I just compromised here in the name of practicality. It's definitely a possibility of evolution for s6-rc if it becomes a real issue. -- Laurent
Re: On possibly "finer" dependencies in s6-rc
[Just accidentally sent as private mail; reposting verbatim.] I'm glad that possibility still exists. Thanks :) On Tue, Sep 29, 2015 at 06:00:40PM +0200, Laurent Bercot wrote: > I just compromised here in the name of practicality. It's definitely > a possibility of evolution for s6-rc if it becomes a real issue. -- My current OpenPGP key: 4096R/0xE18262B5D9BF213A (expires: 2017.1.1) D69C 1828 2BF2 755D C383 D7B2 E182 62B5 D9BF 213A
