Afaik, `relation-set` and `open-port` gets "reset" by Juju if a hook fails. This means that a bash charm might think it has opened a port and published info on the relation based on the flags, but that hasn't actually happened.
Important to note here is that this only happens on a hook failure from _Juju's standpoint_. You won't experience this if you're retrying a hook a few times as part of a debug-hooks session, because that session is seen as "one continuously running hook" from Juju's standpoint. Therefore, it doesn't make sense for the reactive framework to reset everything in a debug-hooks session, because Juju doesn't reset anything. As I said in the bug report <https://github.com/juju-solutions/charms.reactive/issues/137>, a possible improvement is that `charms.reactive` somehow figures out if it's in a `debug-hooks` session, and if so, doesn't reset flags. Thoughts? PS: The structural issue is that Juju's "transactional" nature is half-assed: all communication with outside gets reset, but the internal state of the OS doesn't get reset. The ideal solution is that when Juju resets, the actual OS gets reset too, removing the need for hooks to be idempotent.. ZFS might be useful in achieving this, but I have no idea if this is even on the roadmap for Juju.. PPS: It's correct that this behavior is not consistent between python and non-python handlers, this is a bug. 2017-11-10 15:54 GMT+01:00 Konstantinos Tsakalozos < kos.tsakalo...@canonical.com>: > Hi again, > > This behaviour is not consistent between the python and bash reactive > implementation. > > Seems the setting of states occurs at the end of each method > invocation in bash, and at the end of hook invocation in the python > implementation. > > Even if it is not correct, the behaviour of the bash reactive is what > the naive ( :) ) developer expects. Can you give me a concrete example > of what can go wrong with the approach the bash reactive > implementation is taking? Stuart, you mention "changes made to the > Juju environment get rolled back on hook failure" could some of these > changes cause a bash reactive charm to misbehave, can you please give > me an example? > > Thanks, > Konstantinos > > On Tue, Oct 31, 2017 at 3:57 PM, Merlijn Sebrechts > <merlijn.sebrec...@gmail.com> wrote: > > > > 2017-10-31 14:39 GMT+01:00 fengxia <fx...@lenovo.com>: > > > >> we have been using Ansible (called from charm) for executions. > > > > > > Would you recommend this approach to new users? > > Do you have examples of such Charms? > > Do you have docs detailing the ansible-Juju integration? > > What can we do to make this integration easier? > > > > > >> > >> But ppl fear things they don't understand. Config mgt tools are like > >> giving you a set of cooking utensils and a recipe book, you can follow > the > >> book, or be an iron chef yourself, and surprisingly, many believe they > have > >> that talent to be the iron chef, though over optimistically often ~~ > > > > > > True! I think this one of the reasons why Juju is so successful for > > OpenStack. Normal config mgmt tools give you all the control, but it > > requires you to completely understand what the deployment scripts are > doing. > > OpenStack is so complex that it's almost impossible to fully understand > all > > components so few sysadmins have the knowledge required to deploy > OpenStack > > with cfg mgmt tools.. > > > > > > > > > >> > >> > >> > >> > >> > >> > >> On 10/30/2017 11:56 AM, Merlijn Sebrechts wrote: > >> > >> Hi all > >> > >> > >> Great discussion in this thread. I sense there are two issues here: > >> > >> 1. The transactional nature of charms.reactive and Juju needs to be > >> explained better. We can't change the transactional nature from the > >> charms.reactive side since this is a Juju core feature, but we can > provide a > >> lot better docs and change function names to better match their actual > >> behavior. This is already discussed for relationships as part of the > >> Endpoint PR. > >> 2. Idempotency is hard, not commonly understood outside of config mgmt > >> community and charms.reactive isn't helping. Idempotency is a solved > issue > >> in config mgmt tools. I don't think it's the job of Juju and > charms.reactive > >> to provide ways to do this because we operate on a higher level (service > >> orchestration, not config mgmt). What we should to is make it easier to > use > >> charms.reactive together with config mgmt tools like Puppet and Chef. > This > >> will keep us from reinventing the wheel and will provide a number of > >> additional benefits (such as being able to leverage existing Puppet > scripts > >> and quicker charms). > >> > >> Please create more issues if I've missed something, and add your > comments > >> to the issues. > >> > >> > >> > >> Kind regards > >> Merlijn > >> > >> 2017-10-05 20:50 GMT+02:00 fengxia <fx...@lenovo.com>: > >>> > >>> " An assumption is being made that the state changes get committed > >>> immediately, but these changes are actually transactional and > >>> following the same transactional behaviour as the Juju hook > >>> environment [1]." > >>> > >>> To chip in my experience as 6-month into learning charms and writing a > >>> few simple charms. > >>> > >>> 1. The "transactional" nature of Juju hook needs better explained. To > be > >>> honest I have no idea what this means, and what implication it has to a > >>> charm writer. Any reference would be helpful. > >>> > >>> 2. I like Mike Wilson's approach to provide a list of "set_state_xxx" > >>> functions so new writer can better guess what this function will do. > >>> Further, a different name calls for further study why they are > different, > >>> thus learning important concept of whatever Juju thinks charm writer > needs > >>> to understand. > >>> > >>> Otherwise, I will expect "set_state" will set that state/flag asap. If > >>> there is a scanning cycle (which I heard there is some kind of 5-min > cycle, > >>> which document has not sufficiently made it clear to a writer either), > charm > >>> writer needs to have better doc to learn what it means for design. I > came > >>> from embedded system world in which a timer loop is common. It calls > for a > >>> different thinking than user space script user. I think such > implication > >>> should be emphasized more. > >>> > >>> 3. idempotent > >>> > >>> Again, this is a concept me (or many new writer) will fail to grasp. > >>> Looking at "apt install" as example, my reaction was that the package > >>> manager is taking care of "doing nothing" if called multiple times. > But how > >>> this translate to my system in which charm is expected to "do > something"? > >>> Does it mean I need a gatekeeper like the package manager so to guard > these > >>> "multiple calls"? > >>> > >>> Again, this feels like a work around because "set_state" will call the > >>> same function block multiple times, which is unintuitive to writer -- > when I > >>> set a state, the action for that state is executed once, not over and > over > >>> again until I turn it off. Further, even "remove_state" doesn't take > effect > >>> immediately, so it feels arbitrary how many cycles a block of code is > >>> executed. This is a design pattern I'm afraid many are not familiar > with, so > >>> some tutorial examples will be much appreciated. > >>> > >>> Best, > >>> > >>> Feng > >>> > >>> > >>> > >>> > >>> On 10/04/2017 08:59 AM, Marco Ceppi wrote: > >>> > >>> So, I've not actually checked the logs in a while, but if visibility is > >>> an issue, it seems reasonable that the reactive framework should print > in > >>> the log that X flags are being reset due to hook failure. Things like > >>> set_flag_immediate has farther reaching consequences than simply > stating > >>> that flags only get set on success of a hook run. > >>> > >>> I know there are further reaching initiatives to alleviate this by > >>> decoupling the reactive state engine from the juju hooks system. In > this > >>> case each successive loop of the reactive runtime could better snapshot > >>> state and make failures more granular. > >>> > >>> * state is being renamed to flag in the next major version of reactive. > >>> > >>> On Wed, Oct 4, 2017 at 8:52 AM Mike Wilson <mike.wil...@canonical.com> > >>> wrote: > >>>> > >>>> So as a new charm writer coming to Juju I would first do this: > >>>> > >>>> def get_ready(): > >>>> func0() > >>>> func1_fails() > >>>> > >>>> Then I would, hopefully, test and notice issues. I would investigate > and > >>>> see that I needed to be idempotent. My next attempt would be to wrap > those > >>>> functions inside state checks with sets after they complete. This > would also > >>>> fail and now the charm creator is left with nothing in the api that > can > >>>> help. They are now off to their own devices to start doing random > things to > >>>> attempt to make this work the way they want it to work. Hopefully, the > >>>> solution is as straight-forward as touching random files, but we just > never > >>>> know. > >>>> > >>>> I would expect the name of set_state to be something like > >>>> set_state_on_success and I would further expect some sort of > immediate state > >>>> thing like set_state or set_state_immediate. This would give the user > the > >>>> tools we know that they need in order to create bug-free charms. > >>>> > >>>> Now to compound that confusion, we have the issue of a hook can call > >>>> multiple functions inside the charm code and if any of those > functions have > >>>> something that fails the whole thing is unwrapped. I understand from > a Juju > >>>> perspective why this is the case, but as a user, I would be > completely taken > >>>> by surprise here. The only real fix here is documentation so that we > can set > >>>> expectations, but people will most likely look at examples instead of > >>>> documentation. This means that we need to make sure to call out any > >>>> potential issues like this in the example charms we release. > >>>> > >>>> > >>>> On Wed, Oct 4, 2017 at 6:34 AM Stuart Bishop > >>>> <stuart.bis...@canonical.com> wrote: > >>>>> > >>>>> On 4 October 2017 at 00:51, Mike Wilson <mike.wil...@canonical.com> > >>>>> wrote: > >>>>> > So the best practice here is to touch a file and test for the > >>>>> > existence of > >>>>> > that file before running must_be_called_exactly_once()? > >>>>> > > >>>>> > I think part of the issue here is that without knowing the extent > of > >>>>> > the > >>>>> > hook it is hard to enforce idempotency as a charm writer. It's easy > >>>>> > to look > >>>>> > at the code above and say that is it idempotent since the init > >>>>> > function is > >>>>> > wrapped in a when_not and the initialized state is set at the > bottom > >>>>> > of > >>>>> > init. > >>>>> > >>>>> Individual handlers should be idempotent, so it doesn't matter about > >>>>> the extent of the hook, or even if the chained handlers being > triggers > >>>>> are running in the same hook. Assume your handlers get called > multiple > >>>>> times, because they may be. Yes, it looks idempotent but it isn't. An > >>>>> assumption is being made that the state changes get committed > >>>>> immediately, but these changes are actually transactional and > >>>>> following the same transactional behaviour as the Juju hook > >>>>> environment [1]. I think this can certainly be explained better in > the > >>>>> docs, but I can't think of a way to stop this being an easy error to > >>>>> make. > >>>>> > >>>>> [1] spot the DBA > >>>>> > >>>>> -- > >>>>> Stuart Bishop <stuart.bis...@canonical.com> > >>>> > >>>> -- > >>>> Juju mailing list > >>>> Juju@lists.ubuntu.com > >>>> Modify settings or unsubscribe at: > >>>> https://lists.ubuntu.com/mailman/listinfo/juju > >>> > >>> > >>> > >>> > >>> -- > >>> Feng Xia > >>> > >>> Advisory Engineer > >>> Datacenter Group (DCG), Lenovo US > >>> 8000 Development Dr, Morrisiville, NC 27509 > >>> W: http://www.lenovo.com > >>> > >>> > >>> -- > >>> Juju mailing list > >>> Juju@lists.ubuntu.com > >>> Modify settings or unsubscribe at: > >>> https://lists.ubuntu.com/mailman/listinfo/juju > >>> > >> > >> > >> -- > >> Feng Xia > >> > >> Advisory Engineer > >> Datacenter Group (DCG), Lenovo US > >> 8000 Development Dr, Morrisiville, NC 27509 > >> W: http://www.lenovo.com > > > > > > > > -- > > Juju mailing list > > Juju@lists.ubuntu.com > > Modify settings or unsubscribe at: > > https://lists.ubuntu.com/mailman/listinfo/juju > > >
-- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
