Re: Dynamically registering reactive handlers at runtime

[Stuart Bishop]

On 12 February 2016 at 22:55, Merlijn Sebrechts
 wrote:
> Hi all
>
>
> We have a layer that can install a number of different packages. It decides
> at runtime which packages will be installed. This layer uses the `apt` layer
> which sets a state `apt.installed.`.
>
> We want to react to this installed state, but we don't know what the
> packagename will be beforehand. This gets decided at runtime. Therefore, we
> can't put it in a @when() decorator.
>
> Is there a way to dynamically register handlers at runtime?

Not with reactive as it stands, unless by 'at runtime' you mean 'at
import time'.

I previously patched @when_file_changed to accept callables, allowing
the filenames to be generated at run time. I imagine the same thing
would need to be done for the @when and @when_not decorators.

I'd experiment by creating your own decorator, and if it turns out to
be useful propose it as a charms.reactive update.

-- 
Stuart Bishop 

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: @when('config.changed')

[Stuart Bishop]

On 18 February 2016 at 01:44, Cory Johns  wrote:

> If a charm config option has changed, the state "config.changed" will be set
> for the duration of the hook.  Additionally, specific states will be set for
> each config option that changed; that is, if option "foo" has changed, the
> state "config.changed.foo" will be set.  An example of code using this would

Will these states be set in the first hook?

I can come up with use cases for both yes and no answers to that
question. If I had done this in a layer, there would have been another
state named something like config.initial.

@when('config.changed')
def first_hook_and_config_changed(): pass

@when('config.changed')
@when_not('config.initial')
def config_changed_not_first_hook(): pass



> This provides a much cleaner way of detecting changes to config, and it is
> recommended that this be used in favor of @hook('config-changed') going
> forward, as the latter can actually run in to some situations, albeit rather
> rarely, where the charm sees new config option values before the
> config-changed hook has fired.  Using the reactive states avoids that
> completely as well as working more naturally with existing @when decorators.


Did you want to pull in the Leadership layer too? Its hard to know
when to stop :) I'd thought this would be better handled as a layer,
for no other reason than it could be done as a separate layer and keep
the basic layer thin.

-- 
Stuart Bishop 

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

[Review Queue] lis-test-charm, etcd

[Matt Bruzek]

I reviewed the HyperV lis-test-charm today that had a few boilerplate files
left inside the charm that should be removed before it passes review:
https://bugs.launchpad.net/charms/+bug/1513612


I also reviewed a change to the etcd charm. I pushed this charm to the
charm store using the new charm2 command:

https://code.launchpad.net/~lazypower/charms/trusty/etcd/bump-to-six/+merge/286412

   - Matt Bruzek 
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Layered Hadoop Core

[Cory Johns]

The big data team just updated the apache-core-batch-processing bundle in
~bigdata-dev (
https://jujucharms.com/u/bigdata-dev/apache-core-batch-processing) to use
the new layered charms.  This makes the charms much easier to understand,
maintain, and extend.  We're asking for more testing by the community,
before we move the recommended core bundle over to these charms as well.

If you are interested in seeing the source layers for these charms, they
can be found in GitHub in the following repos:

https://github.com/juju-solutions/layer-apache-hadoop-namenode
https://github.com/juju-solutions/layer-apache-hadoop-resourcemanager
https://github.com/juju-solutions/layer-apache-hadoop-slave
https://github.com/juju-solutions/layer-apache-hadoop-plugin

Creating a charm that connects to the cluster is now even easier, by
extending the hadoop-client layer:

https://github.com/juju-solutions/layer-hadoop-client

One caveat with this update is that, due to some aspects of how reactive
charms maintain state internally, in-place upgrades from the existing
charms are not recommended at this point.  You will also note that there
are some differences in the services deployed, which we made to bring them
more in line with commonly understood terminology and to prepare for fully
utilizing the HA support built-in to Hadoop 2.7.1.

We are continuing to develop these charms, as well as converting the other
big data charms, such as Spark, Zeppelin, Flume, Kafka, etc., to use layers
and reactive.  We look forward to your feedback.
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: @when('config.changed')

[Marco Ceppi]

On Wed, Feb 17, 2016 at 5:56 PM Matt Bruzek 
wrote:

> I hate to go against the love-in here. While I desperately want a
> configuration changed feature for the reactive framework. I disagree with
> the way it was implemented. I brought up issues with this implementation in
> the pull request:  https://github.com/juju-solutions/layer-basic/pull/36
>

I appreciate the feedback on the feature


> It feels wrong to lump configuration change events in to states. There
> should be a different way to handle these types of events that are
> generated by Juju. We already have @hook('config-changed') for when the
> traditional hook would be called. When a configuration changes it would be
> more natural should have a different decorator for that such as:
> @when_changed('key') or more importantly
> @when_any_changed('key1','key2','key3')
>

I'd want to avoid having /another/ primitive. We start to create our own
language and it becomes this DSL rather than a simple declaration of state.

I actually disagree. I feel this is the perfect way to describe the
situation. Under the covers the base layer determines that configuration
options have changed since last hook run, and sets the state letting the
charm author know "Hey, we've got new configuration, respond accordingly".
We really don't want people to implement @hook call if they can avoid it as
it sets up a weird precedent in processing. Hook is invoked prior to @when
decorations and could lead to a mixed state.


> Charm states are completely arbitrary and an author (who may not read this
> list) could legitimately set a 'config.changed' state that would create
> invalid changed events and break other layers. Even if we pick a different
> state name it seems we are inviting a possible collision that would be very
> difficult to diagnose and debug.
>

You're right, but this argument could be applied to /any/ state. Having a
way to review layers is the best way to curb this behavior. As a result
we've recommended authors to prefix their states with the layer/scope name.
For example, interfaces set states based on the relation name they're
responding to. The Django layer sets it's states as django.


> It feels strange that we are "reserving" a state name (or many names) for
> the configuration changes in this way. Implementing the feature with a
> different decorator seems more deterministic and not overloading the states
> with configuration events.
>

This isn't really overloading, we're processing data and setting the state
of the charm at that time. This is very similar to how the leadership layer
and apt layer work that stub has created.

I feel this fits well with the state of the charm. Having a feature
specific to configuration means we should do the same for things like
storage, actions, relations, etc which all still fit in this current model.


>
> Thoughts?
>

I understand your point of view, but I think if you look at this from a new
charmer perspective, one without years of charm authoring to date, they're
going to be looking to respond to states, and needing to address
configuration changes is a state much like all other states in charm
(layer.restart as an example is an event modeled as state).

Marco
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: @when('config.changed')

[Matt Bruzek]

I hate to go against the love-in here. While I desperately want a
configuration changed feature for the reactive framework. I disagree with
the way it was implemented. I brought up issues with this implementation in
the pull request:  https://github.com/juju-solutions/layer-basic/pull/36

It feels wrong to lump configuration change events in to states. There
should be a different way to handle these types of events that are
generated by Juju. We already have @hook('config-changed') for when the
traditional hook would be called. When a configuration changes it would be
more natural should have a different decorator for that such as:
@when_changed('key') or more importantly
@when_any_changed('key1','key2','key3')

Charm states are completely arbitrary and an author (who may not read this
list) could legitimately set a 'config.changed' state that would create
invalid changed events and break other layers. Even if we pick a different
state name it seems we are inviting a possible collision that would be very
difficult to diagnose and debug.

It feels strange that we are "reserving" a state name (or many names) for
the configuration changes in this way. Implementing the feature with a
different decorator seems more deterministic and not overloading the states
with configuration events.

Thoughts?

   - Matt Bruzek 

On Wed, Feb 17, 2016 at 4:00 PM, Merlijn Sebrechts <
merlijn.sebrec...@gmail.com> wrote:

> Great! Very usefull! Thanks, Cory!
>
>
> Op woensdag 17 februari 2016 heeft Cory Johns 
> het volgende geschreven:
> > Just wanted to give a heads-up about a feature that landed in the
> reactive base layer yesterday.
> > If a charm config option has changed, the state "config.changed" will be
> set for the duration of the hook.  Additionally, specific states will be
> set for each config option that changed; that is, if option "foo" has
> changed, the state "config.changed.foo" will be set.  An example of code
> using this would be:
> > @when('myservice.started', 'config.changed')
> > def update_config():
> > update_config_files()
> > restart_myservice()
> > This provides a much cleaner way of detecting changes to config, and it
> is recommended that this be used in favor of @hook('config-changed') going
> forward, as the latter can actually run in to some situations, albeit
> rather rarely, where the charm sees new config option values before the
> config-changed hook has fired.  Using the reactive states avoids that
> completely as well as working more naturally with existing @when decorators.
> > Please note that, while we are not aware of any charms currently using
> "config.changed" as a state, there is some risk of the state set by the
> base layer conflicting with it if set by the charm layer.  The
> recommendation is to always prefix your states by the name of the layer
> setting them,  or the relation name for interface layers.
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: @when('config.changed')

[Nick Moffitt]

That's a very good point.

Marco Ceppi:
> Relations are best served being managed by the interface layer. Which
> should consume the data and raise states as appropriate.
> 
> On Wed, Feb 17, 2016 at 3:45 PM Nick Moffitt 
> wrote:
> 
> > Cory Johns:
> > > If a charm config option has changed, the state "config.changed" will
> > > be set for the duration of the hook.  Additionally, specific states
> > > will be set for each config option that changed; that is, if option
> > > "foo" has changed, the state "config.changed.foo" will be set.
> >
> > Ah, this is perfectly natural syntax, and I have been wishing for
> > something along this line for a while!
> >
> > Could we do something similar for relation values, or is the namespace
> > just too complicated with all the dimensions it affords?
> >
> > --
> > Nick Moffitt
> >
> > --
> > Juju mailing list
> > Juju@lists.ubuntu.com
> > Modify settings or unsubscribe at:
> > https://lists.ubuntu.com/mailman/listinfo/juju
> >

-- 
Nick Moffitt

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: @when('config.changed')

[Tom Barber]

Cool! I'll work that into my charms tomorrow. Might as well go all out on
the new stuff ;)

Tom
On 17 Feb 2016 18:48, "Marco Ceppi"  wrote:

> This is awesome, glad to see this wrapped in the reactive framework. Will
> make a lot of my layers much simpler!
>
> Marco
>
> On Wed, Feb 17, 2016 at 1:44 PM Cory Johns 
> wrote:
>
>> Just wanted to give a heads-up about a feature that landed in the
>> reactive base layer yesterday.
>>
>> If a charm config option has changed, the state "config.changed" will be
>> set for the duration of the hook.  Additionally, specific states will be
>> set for each config option that changed; that is, if option "foo" has
>> changed, the state "config.changed.foo" will be set.  An example of code
>> using this would be:
>>
>> @when('myservice.started', 'config.changed')
>> def update_config():
>> update_config_files()
>> restart_myservice()
>>
>> This provides a much cleaner way of detecting changes to config, and it
>> is recommended that this be used in favor of @hook('config-changed') going
>> forward, as the latter can actually run in to some situations, albeit
>> rather rarely, where the charm sees new config option values before the
>> config-changed hook has fired.  Using the reactive states avoids that
>> completely as well as working more naturally with existing @when decorators.
>>
>> Please note that, while we are not aware of any charms currently using
>> "config.changed" as a state, there is some risk of the state set by the
>> base layer conflicting with it if set by the charm layer.  The
>> recommendation is to always prefix your states by the name of the layer
>> setting them,  or the relation name for interface layers.
>> --
>> Juju mailing list
>> Juju@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

@when('config.changed')

[Cory Johns]

Just wanted to give a heads-up about a feature that landed in the reactive
base layer yesterday.

If a charm config option has changed, the state "config.changed" will be
set for the duration of the hook.  Additionally, specific states will be
set for each config option that changed; that is, if option "foo" has
changed, the state "config.changed.foo" will be set.  An example of code
using this would be:

@when('myservice.started', 'config.changed')
def update_config():
update_config_files()
restart_myservice()

This provides a much cleaner way of detecting changes to config, and it is
recommended that this be used in favor of @hook('config-changed') going
forward, as the latter can actually run in to some situations, albeit
rather rarely, where the charm sees new config option values before the
config-changed hook has fired.  Using the reactive states avoids that
completely as well as working more naturally with existing @when decorators.

Please note that, while we are not aware of any charms currently using
"config.changed" as a state, there is some risk of the state set by the
base layer conflicting with it if set by the charm layer.  The
recommendation is to always prefix your states by the name of the layer
setting them,  or the relation name for interface layers.
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Introducing, juju resources!

[Adam Collard]

Hi Moonstone!

On Fri, 12 Feb 2016 at 21:27 Katherine Cox-Buday <
katherine.cox-bu...@canonical.com> wrote:

> Moonstone have been working hard on a new feature coming up in Juju 2.0
> called "Juju Resources", and we're now at a point where we can share the
> goodness and call for bugs/feedback! As noted in the video linked below,
> the feature isn't quite complete, so expect some rough edges, and for some
> of the CLI details to shift just a bit.
>

After watching the demo (thanks for sharing!) it struck me that use of the
update-status hook to demonstrate this was pointing to a missing feature
(IMHO).

Is there a hook that fires when a resource is made available? If I have an
install hook that requires a resource, I could make it transition to
blocked when that resource is missing. But when that resource is provided,
how do I know to try again?

Thanks,

Adam
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Units & resources: are units homogeneous?

[Mark Shuttleworth]

I think it's fine for units to self-organise into specific functions.

We'll add a means for these functions to be described to the user, later.

The only rule is that the config and resources presented to units are
consistent, or trend to consistency. In other words, when I set config
or modify a resource, I am doing that for ALL units. While the units
will individually make the transition, they will ultimately ALL make the
transition. So at some level Juju knows that units are briefly
different, depending on which ones have processed the messages
associated with the new config or new resource, but it thinks of the
entire group of units as having the same target config / resources.

If a unit asks for a resource, it should get the same answer that any
other unit asking for that resource would get at that same moment.

Mark

On 17/02/16 12:03, Rick Harding wrote:
> I wanted to add that the reason we're curious about this is because we're
> working on how Juju can help provide insights into things that could be
> off/wrong between units. If we expect the units to be the same, then things
> like warning users that a unit hasn't yet gotten a resource that the others
> have seems like a good idea. However, if you're expecting different units
> to appear differently, then it's like having an error dialog always be
> there that you end up ignoring because that's the way it's meant to be.
>
> It influences the design of how Juju thinks about the resources across the
> units if we're expecting them to look the same or not. There have been
> discussions of how unit entropy is something to try to work on cutting out
> of the system. If I add-unit, it should be the same code at the same
> revision as the one I started previously, even if it's been months since I
> deployed that unit.
>
> The landscape example is interesting, but I can't help but feel like that
> it's abusing the system a bit. The end user doesn't really know what's
> running where if it's a self-adapting system based on the number of units.
> It's really interesting because I guess the end user just wants to know
> they've got landscape running and working properly, but the tech person in
> me is a bit scared that it's not clear what services I'm running where
> setup and exposed in what ways.
>
> On Tue, Feb 16, 2016 at 1:20 PM Katherine Cox-Buday <
> katherine.cox-bu...@canonical.com> wrote:
>
>> Hey All,
>>
>> The team is looking closely at some of our CLI surrounding resources, and
>> an interesting question came up: should units be considered homogeneous?
>>
>> My understanding is that it's a goal to make the management of units more
>> consistent, and making the units more homogeneous would support this, but
>> I'm wondering from a workload perspective if this is also true? One example
>> I could think of to support the discussion is a unit being elected leader
>> and thus taking a different path through it's workflow than the other
>> units. When it comes to resources, maybe this means it pulls a different
>> sub-set of the declared resources, or maybe doesn't pull resources at all
>> (e.g. it's coordinating the rest of the units or something).
>>
>> I'm curious what people are seeing out in the field, and hearing opinions
>> too.
>>
>> Thanks :)
>>
>>
>> -
>> Katherine
>>
>>
>
>

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Units & resources: are units homogeneous?

[Rick Harding]

I wanted to add that the reason we're curious about this is because we're
working on how Juju can help provide insights into things that could be
off/wrong between units. If we expect the units to be the same, then things
like warning users that a unit hasn't yet gotten a resource that the others
have seems like a good idea. However, if you're expecting different units
to appear differently, then it's like having an error dialog always be
there that you end up ignoring because that's the way it's meant to be.

It influences the design of how Juju thinks about the resources across the
units if we're expecting them to look the same or not. There have been
discussions of how unit entropy is something to try to work on cutting out
of the system. If I add-unit, it should be the same code at the same
revision as the one I started previously, even if it's been months since I
deployed that unit.

The landscape example is interesting, but I can't help but feel like that
it's abusing the system a bit. The end user doesn't really know what's
running where if it's a self-adapting system based on the number of units.
It's really interesting because I guess the end user just wants to know
they've got landscape running and working properly, but the tech person in
me is a bit scared that it's not clear what services I'm running where
setup and exposed in what ways.

On Tue, Feb 16, 2016 at 1:20 PM Katherine Cox-Buday <
katherine.cox-bu...@canonical.com> wrote:

> Hey All,
>
> The team is looking closely at some of our CLI surrounding resources, and
> an interesting question came up: should units be considered homogeneous?
>
> My understanding is that it's a goal to make the management of units more
> consistent, and making the units more homogeneous would support this, but
> I'm wondering from a workload perspective if this is also true? One example
> I could think of to support the discussion is a unit being elected leader
> and thus taking a different path through it's workflow than the other
> units. When it comes to resources, maybe this means it pulls a different
> sub-set of the declared resources, or maybe doesn't pull resources at all
> (e.g. it's coordinating the rest of the units or something).
>
> I'm curious what people are seeing out in the field, and hearing opinions
> too.
>
> Thanks :)
>
>
> -
> Katherine
>
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev