Re: Juju and TOSCA

[Gustavo Niemeyer]

Thanks for these details Melijn,

So you want to be able to describe a topology without deploying it, and you
also want to be able to record that topology so that it can be reused.

Both of these are good ideas, and the juju concept that supports both of
these at once is called "stacks". It's not yet implemented, but it's being
debated for a long time (since the first sprint, to be honest) and will
come eventually.

Bundles are a weak form of stack that was introduced organically into the
project while the full fledged concept isn't in place. If it depended just
on me bundles wouldn't exist, to be honest, but they solved a few real
problems while stacks aren't available. Unfortunately they also took away
some of the urge to have the proper implementation in place, but we know
what to do and it will come.




On Fri, Aug 7, 2015 at 5:41 AM, Merlijn Sebrechts <
merlijn.sebrec...@gmail.com> wrote:

> Gustavo
>
>
> Thank you for your response. I'll explain two TOSCA concepts I find very
> interesting and would like to use in Juju.
>
>
> # Separation of Topology VS Plans.
>
> In TOSCA, you can define a topology (what services you want and how to
> connect them) independent from Plans (How to implement these services, what
> Charms to use). This enables the provider (Juju) to decide what
> implementation to use. So for example, say your infrastructure needs a
> MySQL database, and you can define this need without saying how to setup
> the database. In this example, Juju has two possibilities: either it
> deploys a MySQL Charm, or it puts the database in an existing MySQL Charm.
> This flexibility isn't possible because making a topology and choosing the
> implementation of that topology is the same in Juju: You connect Charms.
>
> This concept could be used to automate a lot of tasks an administrator
> currently has to do himself.
>
>
> # Composition of service templates
>
> In TOSCA, a service can consist of a collection of services and relations.
> In Juju you only have two layers: Charms and bundles. It would be great if
> you could make a Charm that consists of a bundle. Other Charms can have
> relations with that Charm bundle as if it was a single Charm. The bundle
> itself decides how these relations translate into relations with individual
> Charms of that bundle.
>
> An example of this would be the data analytics bundle:
> https://jujucharms.com/data-analytics-with-sql-like/6
> When using this bundle as part of a setup, you have to have certain
> knowledge about what the individual Charms do. This is because you have to
> decide yourself how to plug it into the other infrastructure. This isn't
> that complex with a bundle consisting of 4 Charms, but bundles are bound to
> become more complex, with the Openstack Telemetry (19 Charms) as an
> example: https://jujucharms.com/openstack-telemetry/31
>
>
>
> I'm looking forward to reading your thoughts on this.
> Merlijn
>
>
> 2015-08-06 18:05 GMT+02:00 Gustavo Niemeyer <
> gustavo.nieme...@canonical.com>:
>
>> Hi Merlijn,
>>
>> On Thu, Aug 6, 2015 at 12:00 PM, Merlijn Sebrechts <
>> merlijn.sebrec...@gmail.com> wrote:
>>>
>>> I've been using Juju for a while now and I came across TOSCA. TOSCA is
>>> an OASIS open standard that can be used to model the full life-cycle of
>>> cloud applications.
>>>
>>> As stated before, the TOSCA concepts are very similar to Juju.
>>>
>>
>> The initial tosca gatherings happened shortly after we released working
>> juju code, so that's not too surprising.
>>
>>
>>> I'm wondering if there are any plans surrounding Juju and TOSCA. I see
>>> that Canonical is part of the TOSCA TC. Are there any plans to bring TOSCA
>>> and Juju closer together? TOSCA's move to yaml is already a big leap in the
>>> right direction...
>>>
>>> I see that a TOSCA-Juju translator project
>>> <https://github.com/juju/juju-tosca> exists, however I do not know what
>>> the state of that project is. I am also more interested in a native
>>> implementation of some of the TOSCA concepts since there are big advantages
>>> in having a tosca-like representation of a running cloud, beyond what Juju
>>> already offers.
>>>
>>
>> What real advantages do you perceive in that sense?
>>
>> We're constantly designing and improving juju, so that sort of feedback
>> is very welcome. Our last planning sprint happened just last week.
>>
>>
>> gustavo @ http://niemeyer.net
>>
>
>


-- 
gustavo @ http://niemeyer.net
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Juju and TOSCA

[Gustavo Niemeyer]

Hi Merlijn,

On Thu, Aug 6, 2015 at 12:00 PM, Merlijn Sebrechts <
merlijn.sebrec...@gmail.com> wrote:
>
> I've been using Juju for a while now and I came across TOSCA. TOSCA is an
> OASIS open standard that can be used to model the full life-cycle of cloud
> applications.
>
> As stated before, the TOSCA concepts are very similar to Juju.
>

The initial tosca gatherings happened shortly after we released working
juju code, so that's not too surprising.


> I'm wondering if there are any plans surrounding Juju and TOSCA. I see
> that Canonical is part of the TOSCA TC. Are there any plans to bring TOSCA
> and Juju closer together? TOSCA's move to yaml is already a big leap in the
> right direction...
>
> I see that a TOSCA-Juju translator project
>  exists, however I do not know what
> the state of that project is. I am also more interested in a native
> implementation of some of the TOSCA concepts since there are big advantages
> in having a tosca-like representation of a running cloud, beyond what Juju
> already offers.
>

What real advantages do you perceive in that sense?

We're constantly designing and improving juju, so that sort of feedback is
very welcome. Our last planning sprint happened just last week.


gustavo @ http://niemeyer.net
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Fwd: Coordinating actions in a service

[Gustavo Niemeyer]

Sorry, the removal of the list CC was unintended.


-- Forwarded message --
From: Gustavo Niemeyer 
Date: Fri, May 8, 2015 at 1:13 PM
Subject: Re: Coordinating actions in a service
To: John Weldon 




On Fri, May 8, 2015 at 12:42 PM, John Weldon  wrote:

> On Fri, May 8, 2015 at 8:00 AM, Gustavo Niemeyer 
> wrote:
>
>>
>> On Fri, May 8, 2015 at 11:58 AM, John Weldon 
>> wrote:
>>
>>> Actions and Hooks are two different things.
>>>
>>
>> How are they different?
>>
>
> They're similar in that they're both initiated from the uniter loop.
> Basically one of the select stanzas in the uniter loop(s) look for new
> actions as reported by a Strings Watcher that reports new Action Ids.
>
> Once an action is selected the uniter constructs an action context
> including the name and parameters for the Action and then executes the
> defined action in the charm, which is responsible for logging output and
> building a result.  Unless the charm action explicitly calls action-fail
> then the action will be marked as successfully complete once the action
> process ends.
>
>   They're both processed in the Uniter loop, so only one will run at a
>>> time, not in parallel.  I don't know what you mean about not having access
>>> to hook tools, but actions do have access to the jujuc commands.
>>>
>>
>> I mean having access to things such as config-get, etc.
>>
>
> Yes, Actions have access to those.
>

All of that sounds very much in line with what the other hooks do. It's
still not clear to me why you claim actions and hooks are different.


> There is no spec at the moment, only some high level design goals: planning
>>> doc
>>> <https://docs.google.com/document/d/1vxlszQ_gyE2j7YN5LDGD5VE3RI7ijczvYMrzMgEabTQ/edit?usp=sharing>
>>>
>>>
>>
>> Okay, can you please let me know once you have the specification in place?
>>
>>
> Will do.  My status with Canonical is still being worked out, and I have
> been focusing on other clients since Nuremberg.
>

Understood, and thanks.


gustavo @ http://niemeyer.net



-- 

gustavo @ http://niemeyer.net
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Coordinating actions in a service

[Gustavo Niemeyer]

The context for hooks is not all the same either. What is available for a
given hook depends on what the hook itself does. I honestly see no
distinction between actions and hooks in practice, and I also don't see how
driving that distinction externally would benefit users. I would rather
point out that actions are just hooks that are executed when an action is
received by the unit. If that's wrong for some reason and there's a
relevant distinction, I'd still like to learn why.


On Fri, May 8, 2015 at 1:15 PM, John Weldon  wrote:

>
> On Fri, May 8, 2015 at 9:13 AM, Gustavo Niemeyer 
> wrote:
>
>> All of that sounds very much in line with what the other hooks do. It's
>> still not clear to me why you claim actions and hooks are different.
>
>
> I'm not sure the distinction is important.  The key differences are that
> hooks have a hook context and actions have an action context, and the
> runner for each is different code.
>
> Cheers,
>
> --
> John Weldon
>



-- 

gustavo @ http://niemeyer.net
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Coordinating actions in a service

[Gustavo Niemeyer]

On Fri, May 8, 2015 at 10:44 AM, John Weldon  wrote:

> Actions are not hooks, but they are run in a similar fashion as hooks.  I
> don't know of any restriction keeping actions from communicating with peers
> before the action completes, but that would be something we'd need to
> address for Actions 2.0 certainly.
>

I'm not sure I understand what you mean. Are you suggesting actions do not
run in a hook context, and so have no access to hook tools and can run in
parallel with them?  My instinct is that this is not a great design. Am I
wrong?


> I like the concept of metering actions out to units of a service; I would
> expect that to happen when targeting an action to the service instead of
> specific units which would require the leader to decide how to fan out the
> actions.
>
The implementation details for Actions 2.0 are still very conceptual; I'll
> certainly be calling on you for opinions this go around as we're
> implementing them :)
>

Where is the spec, please?


gustavo @ http://niemeyer.net
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Coordinating actions in a service

[Gustavo Niemeyer]

On Fri, May 8, 2015 at 10:24 AM, John Weldon  wrote:

> Hi Stuart;
>
> I think this is addressed in the proposed work for Actions 2.0
>
> In the current model you'd have to manage all of this yourself.  Actions
> can only be targeted to specific units in the current implementation, so
> you'd have to manage the distribution outside of actions (or else, as you
> suggest, some sort of generalised semaphore service, and a way to find all
> units of a service and queue up actions for all of them and have the
> actions individually manage themselves whether to run or not)
>
> The plan is to allow actions to be targeted at 1) specific units in a
> service, 2) leaders only, 3) all units in a service, or 4) a subset of
> units in a service.  This would still be a little tricky for your use case,
> but you could at least manage all the logic in an action targeted at only
> the leader for example.
>

None of these seem to fix the issue, though. The requirement is to execute
on all units, but not at the same time. Also, the leader has no way to
communicate with peer units without the hook terminating, right? There
should be a way to postpone the result of an action to a moment past the
end of the hook, but I actually think the proper way to fix this is to
avoid the avalanche in the first place, by default: when dispatching an
action to all units of a service, roll out in a sane way rather than doing
all at once.


gustavo @ http://niemeyer.net
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Coordinating actions in a service

[Gustavo Niemeyer]

Hi Stuart,

I see two ways to fix this issue, and both are probably worth implementing
for their own good:

- We should be able to have an action that lives past the execution of the
action hook, upon request. This would enable the leader unit to communicate
with all peer units to do anything it wants. The support for dispatching an
action to the leader is scheduled for the upcoming work already.

- The requirement of running an action across all units but without
executing in all of them at once also sounds very common. In fact, it's so
common that it should probably be the default when somebody dispatches an
action to all units of a service.




On Fri, May 8, 2015 at 9:17 AM, Stuart Bishop 
wrote:

> Hi.
>
> I have several potentially long running and expensive database
> operations I'd like to wrap as actions, which will generally be run on
> just one unit or on all units of the service.
>
> The problem I have is running an action on all units of the service.
> For HA, I need to ensure that, if there is more than one unit, then it
> may only run on (num_units/2)-1 units at a time. ie. if I fire off an
> action on all units of a 5 unit cluster, then only two units at a time
> may run the action and the other units will block until they are done.
>
> Leadership is needed to do coordination like this, but I can't see how
> to use it with actions. The action has no way to request permission
> from the leader and no way to get a response.
>
> Can anyone tell me how to do this with the current model?
>
> I don't think it can be done, which I guess makes this a feature
> request for Actions 2.0. Or perhaps this is another use case for a
> general locking service providing semaphores (which would need some
> tricky semantics, since I'd need a semaphore that can be acquired by
> max(1,(num_units/2)-1) units at a time and num_units might change
> while waiting for the lock and before releasing it).
>
> Or is this just out of scope, and the operator needs to do this sort
> of coordination themselves? There is plenty of other coordination that
> can't be embedded in the charm as it is (eg. don't drop an old unit if
> it is still being used to bootstrap a new unit into the cluster).
>
> --
> Stuart Bishop 
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>



-- 

gustavo @ http://niemeyer.net
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Juju Resources (a tool / library)

[Gustavo Niemeyer]

I can, but that's not the right way to proceed if you were in fact
trying to implement an important feature of juju that was extensively
discussed.

1. The project has a technical lead and a manager which should have
the proper information to bootstrap this, or at least know who to talk
to.

2. The project has a roadmap. Make sure to talk to the people in (1)
to see how this fits in.

3. I'm sure there are previous documents about this, given relevance
and prior conversations. You can find these documents yourself by
searching through specs, or by asking people that participated in
prior conversations and might have a better idea about what to search
for.

4. There should be a specification about the feature, before there is
an implementation. Stakeholders should review the specification and
approve it before there is code.





On Wed, Feb 11, 2015 at 6:40 PM, Cory Johns  wrote:
> Can you be more specific on how it differs from the goals of resources
> streams?  As I mentioned in my first email, I asked around to try to
> get specific information about the proposed feature and wasn't able to
> get any concrete answers or documentation.  So I created this based on
> what I remembered from the discussions I'd heard (admittedly not much)
> and what I needed in the charms I was working on.
>
> I fully intend for this library to be subsumed / obviated by core as
> the feature develops, and I tried to make that clear in the library
> README and documentation.  I also intend to update the interface to
> match the feature as closely as possible as the proposal becomes more
> concrete.
>
> On Wed, Feb 11, 2015 at 2:33 PM, Gustavo Niemeyer  
> wrote:
>> Hi Cory,
>>
>> While it's fine and welcome to have such test bed features, it feels
>> like the proposal and implementation have quite different goals from
>> the actual resources feature we've been talking about for a while, so
>> as a very early suggestion and request, I would strongly recommend
>> renaming the feature to avoid creating ambiguity with the feature that
>> we intend juju to have in the long run. Having two resource
>> implementations and taking over important namespaces such as
>> resources.yaml might introduce unnecessary confusion down the road.
>> Instead, the project might have a nice non-generic name, and its
>> configuration file could also be named after it.
>>
>>
>> On Wed, Feb 11, 2015 at 4:17 PM, Cory Johns  wrote:
>>> Per request, the documentation is now also available on
>>> ReadTheDocs.org: http://jujuresources.readthedocs.org/
>>>
>>> On Wed, Feb 11, 2015 at 11:25 AM, Cory Johns  
>>> wrote:
>>>> Hi all,
>>>>
>>>> (cross-posting to juju & juju-dev)
>>>>
>>>> I've created a tool / library for organizing and managing resources
>>>> (binary blobs, tarballs, Python packages, and, eventually, apt
>>>> packages) required by a charm.  The idea is to be an interim tool, and
>>>> a test-bed for the resource features that have been discussed for the
>>>> future in Juju core.
>>>>
>>>> It is available on GitHub [1] and PyPI [2], and the full documentation
>>>> is on PythonHosted.org [3].
>>>>
>>>> The goals of this project are:
>>>>
>>>>   * Organize all external resource dependencies into a single
>>>> resources.yaml file
>>>>   * Provide easy, consistent interfaces to manage, install, and mirror 
>>>> resources
>>>>   * CLI and Python bindings
>>>>   * Enforce best practices, such as cryptographic hash validation
>>>>
>>>> I asked around to see if there was an existing proposal for a
>>>> resources.yaml file format, but couldn't find one.  If someone is
>>>> aware of an existing spec / proposal, I'd prefer to match that as much
>>>> as possible.
>>>>
>>>> The current version is fully functional, and is currently being used
>>>> in the framework refactor of the Apache Hadoop charms (e.g., [4]).
>>>>
>>>> Note that I created this separately from Charm Helpers primarily
>>>> because I wanted to use it to bootstrap CH, but this also makes it
>>>> easier to use in Bash charms.
>>>>
>>>> My next step is to add apt-get support, but that will requiring
>>>> cleaning up the mirror server (possibly converting it to use squid,
>>>> but I may want to keep it self-contained), and learning a bit more
>>>> about how the apt proxy settings work.  Advice here is appreciated.
>>>>
>>>>
>>>> [1] https://github.com/juju-solutions/jujuresources
>>>> [2] https://pypi.python.org/pypi/jujuresources
>>>> [3] http://pythonhosted.org/jujuresources/
>>>> [4] 
>>>> https://code.launchpad.net/~bigdata-dev/charms/trusty/apache-hadoop-hdfs-master/trunk
>>>
>>> --
>>> Juju-dev mailing list
>>> juju-...@lists.ubuntu.com
>>> Modify settings or unsubscribe at: 
>>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>
>>
>>
>> --
>>
>> gustavo @ http://niemeyer.net



-- 

gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Juju Resources (a tool / library)

[Gustavo Niemeyer]

Hi Cory,

While it's fine and welcome to have such test bed features, it feels
like the proposal and implementation have quite different goals from
the actual resources feature we've been talking about for a while, so
as a very early suggestion and request, I would strongly recommend
renaming the feature to avoid creating ambiguity with the feature that
we intend juju to have in the long run. Having two resource
implementations and taking over important namespaces such as
resources.yaml might introduce unnecessary confusion down the road.
Instead, the project might have a nice non-generic name, and its
configuration file could also be named after it.


On Wed, Feb 11, 2015 at 4:17 PM, Cory Johns  wrote:
> Per request, the documentation is now also available on
> ReadTheDocs.org: http://jujuresources.readthedocs.org/
>
> On Wed, Feb 11, 2015 at 11:25 AM, Cory Johns  wrote:
>> Hi all,
>>
>> (cross-posting to juju & juju-dev)
>>
>> I've created a tool / library for organizing and managing resources
>> (binary blobs, tarballs, Python packages, and, eventually, apt
>> packages) required by a charm.  The idea is to be an interim tool, and
>> a test-bed for the resource features that have been discussed for the
>> future in Juju core.
>>
>> It is available on GitHub [1] and PyPI [2], and the full documentation
>> is on PythonHosted.org [3].
>>
>> The goals of this project are:
>>
>>   * Organize all external resource dependencies into a single
>> resources.yaml file
>>   * Provide easy, consistent interfaces to manage, install, and mirror 
>> resources
>>   * CLI and Python bindings
>>   * Enforce best practices, such as cryptographic hash validation
>>
>> I asked around to see if there was an existing proposal for a
>> resources.yaml file format, but couldn't find one.  If someone is
>> aware of an existing spec / proposal, I'd prefer to match that as much
>> as possible.
>>
>> The current version is fully functional, and is currently being used
>> in the framework refactor of the Apache Hadoop charms (e.g., [4]).
>>
>> Note that I created this separately from Charm Helpers primarily
>> because I wanted to use it to bootstrap CH, but this also makes it
>> easier to use in Bash charms.
>>
>> My next step is to add apt-get support, but that will requiring
>> cleaning up the mirror server (possibly converting it to use squid,
>> but I may want to keep it self-contained), and learning a bit more
>> about how the apt proxy settings work.  Advice here is appreciated.
>>
>>
>> [1] https://github.com/juju-solutions/jujuresources
>> [2] https://pypi.python.org/pypi/jujuresources
>> [3] http://pythonhosted.org/jujuresources/
>> [4] 
>> https://code.launchpad.net/~bigdata-dev/charms/trusty/apache-hadoop-hdfs-master/trunk
>
> --
> Juju-dev mailing list
> juju-...@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju-dev



-- 

gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: updating a config value from within a hook

[Gustavo Niemeyer]

It can't be done. Configuration is writable only by the administrator, and
is read-only from the charm side.

What is coming, though, is the ability for the charm to report that the
configuration is somehow incorrect, so that the admin can tell that
something is not right by being told nicely via the UI, and then take
appropriate action. That sounds better than simply resetting the value
silently, which would look something like this:

   http://youtu.be/DCMZZRTbR-Q



On Thu Oct 30 2014 at 2:16:37 PM Stein Myrseth 
wrote:

> I have the need to update the value of a config key from within a hook.
> 'config-set’ is non existing. Resetting it would also work. The use case is
> that a certain value can only be added once within the lifecycle of the
> charm.
> If updated from either from within the UI or cmd line, I need to set it
> back to it’s previous value (which I have stored).
>
> Any idea how this can be done ?
>
> Stein Myrseth
> Bjørkesvingen 6J
> 3408 Tranby
> mob: +47 909 62 763
> mailto:stein.myrs...@gmail.com 
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Posts restricted to members only

[Gustavo Niemeyer]

Hey all,

Just a quick note to let you know I've changed the policy of the mailing
list from holding messages from non-members into rejecting. It's not clear
why, but the volume of spam in this list is unlike any other I moderate at
Canonical, with multiple entries a day, and comparatively rarely we have
real people trying to reach the list, which means these often end up being
held for longer than they should. So the change is supposed to be a
win-win: real people get a prompt notice telling them they should
subscribe, and spammers get handled without manual labor.

Let's hope this works well.
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: arch constraint default changed?

[Gustavo Niemeyer]

On Tue, May 13, 2014 at 8:18 AM, John Meinel  wrote:
> FWIW, I've gotten bug requests from a user that did a regular bootstrap and
> then was trying to "juju upgrade-juju --upload-tools" and was confused that
> his local machine wasn't able to upgrade tools for his environment. (he was
> running i386 locally, and bootstrap created an amd64 machine).
> And while we desperately want to not expose --upload-tools in its current
> incarnation, we haven't given an alternative for those use cases.

There's nothing wrong with tweaking constraints if the application
sees the --upload-tools option. At the same time, getting a bug from a
user is not enough reason to tweak the defaults for everybody.

> Also, while we have a reasonably clear model for "if you have the choice
> between amd64 and i386 pick amd64", I don't think people disagree with that.
> But what if you have the choice between amd64 and ppc64le and the client is
> running on ppc64le? Is it likely that they are actually thinking in a ppc
> world?

I don't see how that logic holds. Using an arm laptop as a client does
not imply I want to use all my server deployments on arm. The same
holds true for the operating system, or to the Ubuntu series, or to
pretty much anything else: I would not expect the tool to deploy a
completely different environment based on where I'm sitting this
second.

> We certainly had a lot of discussions around this between Nate and myself,
> and while I think I was reasonably convinced that we could just pick amd64
> if it was an option, it seems he was also reasonably convinced that we'd
> want to use the client arch if available. (He wrote it as "just pick amd64",
> I argued against it but ended up feeling it was a reasonable pick among
> alternatives, but then he changed it before landing.)

If I had the chance, I would submit these kinds of decisions to the
development mailing list, rather than arbitrating it back and forth in
isolation like that. This is changing the default behavior for
everybody, so sending it for the team's appraisal feels cheap.


gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: arch constraint default changed?

[Gustavo Niemeyer]

Why isn't the default tweaked by --upload-tools itself then?  We
should be optimizing these options for users, rather than for
developers, and it sounds sensible to assume that the vast majority of
users do want to deploy on amd64 rather than i386 or arm.

On Mon, May 12, 2014 at 2:53 PM, Nate Finch  wrote:
> However, the fix is slightly different than just "always choose amd64".
> Instead, we always choose the same architecture as the client machine, that
> way if the user uses --upload-tools, the tools will actually work on the
> cloud machine.
>
> This means that if you're running i386, you would still need --arch amd64 to
> get amd64 machines in the cloud.
>
>
> On Mon, May 12, 2014 at 1:50 PM, John Meinel  wrote:
>>
>> https://bugs.launchpad.net/ubuntu/+source/juju-core/+bug/1304407
>>
>> It should be fixed in 1.18.3.
>> John
>> =:->
>>
>>
>>
>> On Mon, May 12, 2014 at 9:36 PM, Henning Eggers  wrote:
>>>
>>> Hi!
>>>
>>> I just noticed that I now have specify the arch constraint if I want
>>> amd64
>>> instances. Without it, I get i386 instances. "juju help constraints"
>>> still
>>> tells me that amd64 is supposed to be the default.
>>>
>>> I am using juju 1.18.2 on EC2, precise on the instances, trusty on my
>>> workstation.
>>> Has this since been fixed? Or is it an intentional change and the help
>>> page is
>>> outdated?
>>>
>>> Cheers,
>>> Henning
>>>
>>>
>>> --
>>> Juju mailing list
>>> Juju@lists.ubuntu.com
>>> Modify settings or unsubscribe at:
>>> https://lists.ubuntu.com/mailman/listinfo/juju
>>
>>
>>
>> --
>> Juju mailing list
>> Juju@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>>
>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>



-- 

gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: goamz API stability

[Gustavo Niemeyer]

Whatever is done we should not break the API like that, as this would
instantly break every single application out there that uses goamz's s3
package.

A better way to handle that is to panic if the S3 endpoint is unset and
s3.New is called on such a region, and document that this will be the case
on that function. That's generally not a good idea, but in practice this is
already what happens today: if a region without the S3 endpoint is used, it
will eventually panic in a more subtle way, so we're just making the
behavior explicit and clear.

Then, when we're forced to break the API for a different reason, we can
take the chance to improve that API, and at that time we should change the
import path so that existent applications are unaffected by these
modifications.


gustavo @ http://niemeyer.net
On May 8, 2014 11:07 AM, "Benoît Canet"  wrote:

>
> Hi,
>
> I am adding the Outscale provider regions declarations in goamz/aws/aws.py
> Regions variable.
>
> It turn that a bunch of endpoint are not definied for the outscale cloud
> which only implements the EC2 endpoint.
>
> It set these field to "" to indicated they are not used.
>
> Now I am trying to find a way to make sure that the S3 api of goamz will
> work
> fine with this.
>
> Is it correct to change goamz S2 new constructor to return nill, false if
> the
> region has no S3 endpoint and *S3, true if the region has an endpoint.
>
> It does imply that callers need to change their way of doing.
>
> Of course I would modify the juju-core code to support this.
>
> What are your opinions ?
>
> Best regards
>
> Benoît
>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Juju Academy

[Gustavo Niemeyer]

This is _extremely_ cool, Marco. Very well done!

One tiny suggestion, for anyone interested in contributing: it would
be great to have "ls [-la]" at some point. That's the first thing most
people will type when seeing a prompt, and gives them room for
navigating to the juju configuration directory.

On Wed, May 7, 2014 at 1:16 PM, Marco Ceppi  wrote:
> Hi everyone!
>
> I was trying to keep this under wraps as I worked on it more before
> announcing to the world but I'm too excited with the progress so far so
> here's the "SUPER ALPHA BETA OMEGA" introduction to Juju Academy.
>
> I started this, http://juju.academy (http://learnjuju.com) based on my own
> experiences when trying new software. Primarily modeled after the Learn Go
> Lang webiste (http://tour.golang.org/) I set out to create an easy platform
> that emulates a terminal environment and allows a user to try Juju before
> ever having to install it. In addition I wanted to make a lightweight lesson
> framework to help guide new users in this exciting new Service Orchestration
> paradigm. Finally, the last goal of this project was to build an easy to
> embed module that could live in the docs to provide very lightweight
> terminal sessions that users could use to review what portions of the docs
> they were reading.
>
> Right now I've modeled just a hand full of lessons and only a few of the
> juju commands have actually been implemented. As this is a spare time
> project progress comes in chunks of time over the weekend and in the
> evenings. However, if you're interested in piloting the demoware and shaking
> out bugs please do so! You can view the lessons at http://juju.academy the
> source code is https://github.com/marcoceppi/juju-academy and the issue
> tracker is on that repo.
>
> Your juju environment(s) persist not only between lessons but also between
> page visits. If at anytime you wish to start anew you can do so by issuing
> the "reset" command in the terminal. I'm working on finishing
> http://help.juju.academy which will have this and other FAQ/Guide like
> questions to use the software. All Juju help can be found, as always, at
> https://juju.ubuntu.com/docs
>
> This is also a call for help! Anyone interested in writing lessons, command
> modules, fixing bugs, making this look nicer, etc - pull requests are
> welcome! The entire project aims to be modular (in that this framework could
> be used for non juju terminal lessons). Lessons are simply JSONP files that
> contain a set number of keys and commands are functions that perform some
> rudimentary validation.
>
> I eagerly await feedback and have had an immense amount of fun working on
> this so far! I'll likely follow up with a more official announcement when
> more of the commands have been implemented.
>
> Thanks,
> Marco Ceppi
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>



-- 

gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Openstack and Juju not working

[Gustavo Niemeyer]

There's probably something else not quite okay with that deployment.
Note that apparently all the interactions fail with several minutes of
delay, until apache sends the 408. Some of these operations are GET
requests, for files that tend to be very small. For example, one of
the timeouts was from:

2014-03-26 18:52:13 ERROR juju.cmd supercommand.go:300 failed to GET
object provider-state from container juju-dist

How was the event reported in the logs of Apache and Swift?



On Wed, Mar 26, 2014 at 7:24 PM, Sebastian  wrote:
> Yes!! it seems that the apache is giving the time out, for too long uploads.
>
> Maybe apache tweeks to increase time out?
>
> thank you people! :)
>
> Abs,
> Sebas.
>
>
>
> 2014-03-26 18:54 GMT-03:00 Gustavo Niemeyer
> :
>>
>> The response was cut out:
>>
>> > It's worth noting the timing between the first and the second entries
>> > in the log above. It's taking quite a while for apache to respond with
>> > the 408 timeout, which might indicate that the
>>
>> ... swift in the backend isn't communicating properly for some reason.
>>
>>
>> gustavo @ http://niemeyer.net
>>
>> --
>> Juju mailing list
>> Juju@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>
>



-- 
gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Openstack and Juju not working

[Gustavo Niemeyer]

The response was cut out:

> It's worth noting the timing between the first and the second entries
> in the log above. It's taking quite a while for apache to respond with
> the 408 timeout, which might indicate that the

... swift in the backend isn't communicating properly for some reason.


gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Openstack and Juju not working

[Gustavo Niemeyer]

Indeed swift doesn't seem to be working properly there. During the
debug run, the juju client also could not create the juju-dist bucket:

2014-03-26 19:41:36 INFO juju.provider.openstack provider.go:203
opening environment "openstack"
2014-03-26 19:55:13 WARNING juju.environs open.go:278 failed to write
bootstrap-verify file: cannot make Swift control container: failed to
create container: juju-dist
caused by: failed executing the request
http://x.x.x.x:8080/v1/AUTH_5e10983c75b64e31ba200cebd088b29f/juju-dist
caused by: Put 
http://x.x.x.x:8080/v1/AUTH_5e10983c75b64e31ba200cebd088b29f/juju-dist:
EOF

It's worth noting the timing between the first and the second entries
in the log above. It's taking quite a while for apache to respond with
the 408 timeout, which might indicate that the

So a few initial checks:

1) Is the swift service enabled?
2) Is the swift endpoint working with any other software?
3) What do the apache logs say about the event?
4) What do the swift logs say about the event?



On Wed, Mar 26, 2014 at 6:29 PM, James Page  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Hi Sebastian
>
> On 26/03/14 20:31, Sebastian wrote:
>> I work at Taller  as a cto, and we are making
>> a POC to a big gov organization here at Brazil to show how *Juju*
>> and *Openstack* can work together to manage their cloud, and change
>> their mindset to an a more efficient and automated way of deploying
>> their sites and systems.
>>
>> But! we are stuck(and frustrated), because now we are more than 1
>> month late. So this email is a desperate way asking for help to
>> make this opportunity of making a big case of success here at
>> Brazil.
>>
>> Being said that, heres our problem, we have an openstack
>> all-in-one installed through devstack script, and its working with
>> lxc hypervisor which is cool and neat, but, when we try to connect
>> juju to it, fails in two different ways, here are the logs
>> http://pastebin.com/N5ksbsY2
>
> This looks like a problem with the swift storage setup in your deployment:
>
> 2014-03-26 18:52:13 ERROR juju.cmd supercommand.go:300 failed to GET
> object provider-state from container juju-dist
> caused by: request
> (http://x.x.x.x:8080/v1/AUTH_5e10983c75b64e31ba200cebd088b29f/juju-dist/provider-state)
> returned unexpected status: 408; error info:  "-//IETF//DTD HTML 2.0//EN">
> 
> 408 Request Time-out
> 
> Request Time-out
> Server timeout waiting for the HTTP request from the client.
> 
> Apache/2.2.22 (Ubuntu) Server at x:x:x:x::1 Port 8080
> 
>
> I think this error message indicates that swift backend behind apache
> is not working correctly - could you confirm that swift is functional
> using the openstack-dashboard?
>
> - --
> James Page
> Ubuntu and Debian Developer
> james.p...@ubuntu.com
> jamesp...@debian.org
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJTM0aiAAoJEL/srsug59jDGh8QAI3nBZIAupkhJnXH7kPFFo4c
> dD6wz6ElDQWYGLwoJZfJORNK4Zl0omsvLJLDPyHgLtEO4F9pel8GcBkKk7owII1p
> 6A+XJaSJBRCYQAiH87BuI374d9uc4Mz7lWd73af2299AbXYb6ydCdet3U9cH4KvB
> zetCZptwlGAMyA+bro59m2eus6iy9UWLHDHxeGohtSV6o0HMZny+2yoLeaV4YGUK
> xNg5l7IKmozEfDr14imrNTjuQBD/2DWA/58T2xRT7apKx5pcC1uBaG7z0wkAV0M3
> HO/ciKqawCrsfUpuERnSo5brH3780BBQsWuGxfAyxCoWsTwBjOSB0d/pMLy1p0na
> qUyNt46qX/tjkvlf36TjkD+7UOyrSG7jdye26T/PFkz9bVRK0aFLpF6+RU2GpMNe
> FXUpF3vnsSmnIF2tKAQIlz0RBhOLBMUGVb2UqIJeGXO63c13IKgF5JW9IoxN3N74
> uz3BxsoS9Ufe77qQ/ALbGrMHHrURPpI1rMj4/845fKkfwcUVKPUBOHOvE6Bec8WG
> jZXc6hEwAjbG7tdjOdvZPg7/tFz8OdO8Qg0K6dZqYjyz3OUGSswYRhORjnejCSun
> W4157MlUJYSk136t2wo8RPV6c4Z2/aTEr3q3AJBhcrfdnNmSV9yMuC8JCeSVaHHo
> pmzirD1kGKjaoEjq8HzB
> =rqr+
> -END PGP SIGNATURE-
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju



-- 
gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: AWS Request Rate

[Gustavo Niemeyer]

On Thu, Feb 20, 2014 at 12:02 PM, David Britton
 wrote:
>   "2":
>   agent-state-info: '(error: cannot set up groups: Request limit 
> exceeded. (RequestLimitExceeded))'

As a side note, I'd just like to congratulate the juju team for
offering such a clear error message in the right place.


gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Juju.is redirect now in place.

[Gustavo Niemeyer]

Wow, this is great!

On Thu, Feb 6, 2014 at 4:58 AM, Jorge O. Castro  wrote:
> Thanks to Maarten Ectors and Darren Spiteri we now have http://juju.is
> as a convenient and rememberable way to search charms. It takes
> anything after the / and passes it to jujucharms.com... so
>
> http://juju.is/hadoop
> http://juju.is/rails
> http://juju.is/mysql
>
> and so on.
>
> --
> Jorge Castro
> Canonical Ltd.
> http://juju.ubuntu.com/ - Automate your Cloud Infrastructure
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju



-- 

gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: standalone units vs units in peer relationship

[Gustavo Niemeyer]

Hey Stuart,

I'm not quite if it is in the roadmap or not, but a very simple
approach was documented over the past sprint, so there might be a good
chance it gets implemented: http://goo.gl/birTcN


On Mon, Nov 25, 2013 at 8:31 AM, Stuart Bishop
 wrote:
> Whenever I deploy a new unit, the 'install' and 'config-changed' hooks
> are run and it configures it self as a standalone unit. If there are
> other units in the service, peer relation-joined and peer
> relation-changed hooks are run and the unit reconfigures itself to be
> a member of this cluster.
>
> The problem I am having is with access to an external resource defined
> by the service's yaml configuration (in my case, a Swift container
> used to store WAL logs for PITR). There is always a period where there
> are two (or more) units that believe they are standalone units and in
> control of the shared resource.
>
> I think I have a work around - use the unit name to generate a unique
> Swift container name and use that until the unit has joined the peer
> relationship. There are a few drawbacks to this approach so it isn't
> ideal.
>
> Can anyone think of a better model for this? I do notice from the logs
> that a standalone unit will join the peer relationship, although no
> peer relation joined hook is run until at least one more unit has
> joined. If the unit joined was joined to the peer relationship right
> at the start, before the initial config-changed hook is run, then it
> could detect that there are other units in the service (but I suspect
> this would still be racy).
>
> I think I recall a helper being suggested to allow a master unit to be
> elected safely which would solve my problem properly (and also remove
> a lot of complexity from my charm). Did anything like this get on the
> roadmap?
>
> --
> Stuart Bishop 
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju

-- 
gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Inactive charmers and charmers

[Gustavo Niemeyer]

It's not the first time that the "inactive" bit there creates
confusion. One idea would be to rename "charmers" to be more
descriptive, rather than having a negative team (a team that does NOT
do something). For example:

~charmers => all charmers
~charm-reviewers => only those who review; this team is a member of charmers too


On Thu, Oct 24, 2013 at 10:46 PM, Jorge O. Castro  wrote:
> On Thu, Oct 24, 2013 at 3:14 PM, Clint Byrum  wrote:
>>
>> Also it isn't clear at all what you mean by "I've removed it". Did you
>> remove inactive-charmers from charmers or did you remove the team
>> altogether?
>
>
> He removed inactive-charmers from charmers. However in hindsight I think
> what this really should be is:
>
> ~charmers < --- we maintain the charm store, we actively review and are
> responsible for the charm store submissions.
> ~non-reviewing-charmers <--- I help maintain the charm store but I don't
> actively review. We're part of ~charmers and there is stuff I care about in
> the store that I want to take care of, but not be responsible for the whole
> thing.
>
> ~inactive-charmers <--- NOT a part of ~charmers, inactive but can activate.
> I am retired so I shouldn't have access to the entire charm store.
>
> I've readded the team for now pending discussion on this list on what we
> should do.
>
> --
> Jorge Castro
> Canonical Ltd.
> http://juju.ubuntu.com/ - Automate your Cloud Infrastructure
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>



-- 

gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Sharing a DB user password among units of the app

[Gustavo Niemeyer]

On Wed, Oct 30, 2013 at 7:50 AM, Marco Ceppi  wrote:
> You can't make that assumption and we really frown upon it in charm reviews.
> I've seen cases where `juju deploy -n 5 service` results in unit 2 or really
> unit > 0 being first up. There's also scenarios where there might not
> actually be a "unit 0" there's no way to safely guarentee a leader right
> now.

As I said, we can do better, and I've already published a spec for the
"leader election" feature. That's not a reason for not starting with
something both plausible and reasonable right now, though. It doesn't
matter which unit comes first; unit/0 is still unit/0. If unit/0
doesn't come up, it won't work, so don't do that. The fix is already
specified and is easy to implement.


gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Sharing a DB user password among units of the app

[Gustavo Niemeyer]

On Wed, Oct 30, 2013 at 7:35 AM, Kapil Thangavelu
 wrote:
> Just thinking outloud, but In the active-active db scenario, there's an
> available db for the charm to store this info. ie. the charm could keep some
> bookeeping tables/db instead of using files on disk (a shared key via peer
> relation used for storing the credentials in db).

So you share the credentials for the credentials? :-)

Just share the credentials over the peer relation. Unit 0 creates them.

We can do better, but this should be good enough of a start.


gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Sharing a DB user password among units of the app

[Gustavo Niemeyer]

Hi James,

What is the inconsistency problem with trying to share the password
via relations?

On Wed, Oct 30, 2013 at 4:58 AM, James Page  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> On 29/10/13 17:12, Kapil Thangavelu wrote:
>> fwiw, the mysql charm tries to address this with a shared-db
>> interface, and a separate admin interface. ie the shared-db
>> interface shares out the same db user/password to multiple
>> services, and then for things that need admin access they can be
>> related to the admin interface.
>
> Right now the username/passwords are stored on the disk that the mysql
> datafiles reside on; currently in a clustered mysql with ceph backing
> volume this is always presented to the active mysql node so should be
> up-to-date.  Only the active node will dish out requests for passwords
> from related services.
>
> This won't work when we move to active/active mysql; in this case we
> will probably use the approach in the keystone charm where
> username/passwords are stored on local disk and synced out to peered
> service units via a unpriviledged SSH account and unison.
>
> Unless, of course, Juju grows a feature to allow peers to share data
> in a way which is more consistent that trying todo it via the peer
> relation.
>
> - --
> James Page
> Ubuntu and Debian Developer
> james.p...@ubuntu.com
> jamesp...@debian.org
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.14 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJScPRJAAoJEL/srsug59jDJ4EQALtxKhaJ/aMdRfBOctLOu8ZV
> i90Qxq8qz+HGLu7JNY43ITkyoNuzhYIRNZ82nwDkVNwTH3DQOhKEpK5e+fmMoaMZ
> CBJv0R+HibCWtSH4bZPDIdH+7emEJIamwkMkfMs2ie7AdC+t3LDhMM0OMJ4ZX2Tc
> aWenDjFsY1ZqgNERs7ZLw7VNOYIBSj042+9MJpEIDbAo1CTlhpgE/y0QrjRp7yzi
> LZksfm8CCVn4TJZb8m8ThHJkdzLkavRGrc/xe/tZWMBCwmxkwFfNDL/1ARjZ/lhc
> lo7PFXGBjI1jBbaQybjIV6thpy0pf4yjgU8n9o1I2io4BeF51yKO+lyKd+7yZJjS
> rVoeHhOGSrV3yQgHgFDG1vQWHtTrDVaozLxR7pdt0zPu7dM7xiXnOJKPdJR/ArLO
> OZ/YDK+3ZhcLa5x08levbjNqa++e8VMwHXVTP2bOXPx4hvkL0/8aoQblaauzRhX2
> c3Wg6ZuxbtE8mmD7zYB5HZXJzjKILlJpRLlnuS5FZuEiVqa/2sUX3Qmh0XkECmZh
> EuSaCqJLDGK3sdjhtsdjN/9v10ic5PE2+NCJ5v80YUkaMzKTRJb6Ia5/KMGFIKgc
> N9IMpPSMpfxvvu5GzcD4QLPzsSQnC+BqaMi5IMKr+OY9dFKMGzRqkESVHgr9Slcv
> +SHz7mvMAo12KDnky9BM
> =KcFr
> -END PGP SIGNATURE-
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju

-- 
gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: I want to change the http interface

[Gustavo Niemeyer]

Indeed, and it sounds like this is a good topic to talk through in the sprint.

There are several inner points to the topic:

- Versioning interfaces is possible

- Extending the _same_ interfaces with optional features is possible

- Providing multiple interfaces, the general one and a custom one, is
also possible

- The grand plan of properly [1] documenting interfaces never happened

- We should avoid adding new terminology and logic to an already rich
context; conventions can solve problems


[1] See http://webintents.org/share, for example


On Wed, Oct 16, 2013 at 4:57 AM, Mark Shuttleworth  wrote:
>
> When we originally discussed this, the view was that versioning would be
> handled directly in the name. So folks CAN if they want do an http-XYZ
> and own that namespace henceforth. In general, I think we should
> socialise convergence around common patterns, but allowing flexibility
> encourages healthy experimentation.
>
> Mark
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju

-- 
gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: How do I launch and m1.medium on AWS?

[Gustavo Niemeyer]

On Thu, Oct 10, 2013 at 8:31 AM, John Arbash Meinel
 wrote:
> We've gone around that discussion a *lot* about how you specify a
> value that is site-specific. Do you need a site-specific key?
(...)
> We talked about supporting a potential list and we just pick out which
> one might fit, but Canonistack and EC2 both define an "m1.tiny" that
> may or may not match well enough (and we don't provide a way to be
> more specific without just site specific meaning).

We can perhaps get away with a simple mechanism: we define a generic
"instance-type" constraint. If the name of that constraint matches
exactly something that is available in the API endpoint being used
(not just provider type, since we can have multiple API endpoints with
the same provider type yet different machine types), we use it. If it
does not match any, the script errors out so we can catch typos and
actual errors. In a second moment, we can provide a mechanism for
people to provide their own custom mappings tied to their environment
("in this environment t1.micro means 1 CPU and 512MB of memory").


gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: How do I launch and m1.medium on AWS?

[Gustavo Niemeyer]

Hey Nate,

That was quick! :-)

Have you read the previous threads on the subject?  Agree with Mark we
should do something, but would be good to keep in mind some of the
points raised (e.g. scripts that use these provider-specific
constraints shouldn't just explode when running elsewhere).

On Wed, Oct 9, 2013 at 9:24 PM, Nate Finch  wrote:
> I just implemented that tonight, pending code review.  :)
>
> On Oct 9, 2013 4:59 PM, "Mark Shuttleworth"  wrote:
>>
>>
>> We should really allow passing substrate-specific constraints like
>> instance types. No need for the voodoo in juju :)
>>
>> On 09/10/13 20:16, Kapil Thangavelu wrote:
>>
>> Unfortunately without an ec2 constraint in juju-core around instance type,
>> the overlap around capabilities within ec2 instance types means exact
>> specification via generic constraint is inexact. looks like you already
>> filed a bug on this https://bugs.launchpad.net/juju-core/+bug/1237568
>>
>>
>> On Wed, Oct 9, 2013 at 2:14 PM, Jorge O. Castro  wrote:
>>>
>>> I can't seem to figure out the right combination of constraints to get
>>> an m1.medium.
>>>
>>> juju bootstrap --constraints "cpu-cores=2 mem=3.75"
>>>
>>> launches c1.mediums and not m1's.
>>>
>>> --
>>> Jorge Castro
>>> Canonical Ltd.
>>> http://juju.ubuntu.com/charm-championship - Share your infrastructure,
>>> win a prize!
>>>
>>> --
>>> Juju mailing list
>>> Juju@lists.ubuntu.com
>>> Modify settings or unsubscribe at:
>>> https://lists.ubuntu.com/mailman/listinfo/juju
>>
>>
>>
>>
>>
>>
>> --
>> Juju mailing list
>> Juju@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>



-- 

gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: jujucharms.com not loading

[Gustavo Niemeyer]

Just as a data point, I don't have cookies blocked.

On Thu, Sep 19, 2013 at 12:19 PM, Wavy Davy  wrote:
> On 19 September 2013 16:05, Gary Poster  wrote:
>> On 09/19/2013 10:41 AM, Richard Harding wrote:
>>> On Thu, 19 Sep 2013, Gustavo Niemeyer wrote:
>>>
>>>> And it just happened again while navigating there. :-(
>> ...
>>
>> We talked about this on IRC and Gustavo can't dupe now.  We can't dupe
>> on our machines either, and haven't seen or heard about this symptom
>> since the very first jujucharms.com GUI launch.
>>
>> However, clearly there's something wrong.
>>
>> If anyone encounters this, please get in touch with us before
>> reloading your browser so we can investigate in your console.
>
> FWIW, I've had this before. I get it if I have cookies blocked (which
> I do by default in chrome). Enable cookies for jujucharms.com, and no
> problems.
>
> Thanks
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju



-- 

gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: jujucharms.com not loading

[Gustavo Niemeyer]

And it just happened again while navigating there. :-(

On Thu, Sep 19, 2013 at 11:31 AM, Gustavo Niemeyer  wrote:
> Morning,
>
> First time I tried to load http://jujucharms.com today it got stuck in
> "Connecting to the environment...":
>
> http://pbrd.co/17M5Fgy
>
> Not sure if that's a side effect of the changes being done for the new 
> release?
>
>
> gustavo @ http://niemeyer.net



-- 

gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

jujucharms.com not loading

[Gustavo Niemeyer]

Morning,

First time I tried to load http://jujucharms.com today it got stuck in
"Connecting to the environment...":

http://pbrd.co/17M5Fgy

Not sure if that's a side effect of the changes being done for the new release?


gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Beware global state in hooks

[Gustavo Niemeyer]

Doesn't really seem like an issue. It's usual to see things not
working if a process environment is arbitrarily modified
(communication with X is broken, password agent is gone,  etc). It's
even less of an issue given the original context provided, with
setuid. *Hopefully* changing to an arbitrary user will stop the
communication of the juju commands with the agent.

On Mon, Sep 16, 2013 at 8:12 PM, Kapil Thangavelu
 wrote:
> Process hierarchy
>
> unit agent -> hook -> juju-cli hook api (config-get/relation-get/set)
>
> He's saying the hook env can effect runtime of the hook cli api. Its
> debatable, but one option might be white-listing env variables that the
> hooks support, and unsetting those not related.
>
>
>
> On Tue, Sep 17, 2013 at 7:52 AM, David Cheney 
> wrote:
>>
>> > Be careful when touching process-global state when writing charm
>> > hooks. Calling out to the juju tools such as config-get will inherit
>> > the normal C environment, and juju may break in surprising ways if you
>> > don't leave it how you found it.
>>
>> I'm confused, are you saying the subprocess was able to mutate the
>> environment of the caller ? I really don't follow.
>>
>> --
>> Juju mailing list
>> Juju@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>



-- 

gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Debug problem

[Gustavo Niemeyer]

Regarding juju-run, not that I'm aware of, but it's at least a great
feature that I'm sure is still somewhere in the TODO list.

As for the debug-hooks omission of the install hook, that would be a bug.


On Thu, Sep 5, 2013 at 6:04 AM, Matt Rae  wrote:
> Just found this on the juju list, did the juju-run command ever materialize?
>
> Seems like it would be very valuable to be able to run arbritrary hooks
> rather than waiting for debug-hooks to intercept.
>
> I've noticed debug-hooks doesn't seem to work for the install hook??
>
>
> On Thu, Nov 15, 2012 at 2:34 AM, Peter M. Petrakis
>  wrote:
>>
>>
>>
>> On 11/14/2012 11:42 AM, Gustavo Niemeyer wrote:
>>>
>>> On Wed, Nov 14, 2012 at 2:36 PM, Peter M. Petrakis
>>>  wrote:
>>>>
>>>> It would be neat if we could run debug-hooks directly on the host
>>>> or just setup the entire environment on demand. This would also
>>>> help with ancillary scripts that support complex charms. Currently
>>>> I have to save away any envvars necessary at install time and
>>>> echo | sed them into local scripts to maintain context.
>>>
>>>
>>> We're actually on the way to have something very handy, but I'm not
>>> sure if it's exactly what you're looking for. We'll soon have a
>>> juju-run command that can be execute any script out-of-band, and have
>>> the full hook context so that the host can do whatever is needed even
>>> when no hooks are up.
>>>
>>> Would that help your use case?
>>
>>
>> Hmm, maybe. The use case is creating supporting scripts and include files
>> that are used by other programs on the file system. Like an upstart job
>> that calls functions included from a common include location. Currently
>> I save away the charm home at install time and template this in.
>> Alternatively
>> I would have to create a space somewhere and copy said files to new
>> location,
>> and ensure they're overwritten during install/upgrade. Keeping them in the
>> charm root ensures that they're unconditionally over written like the hook
>> scripts themselves.
>>
>> See:
>>
>> http://bazaar.launchpad.net/~peter-petrakis/charms/precise/opengrok/trunk/files/head:/
>>
>> inc/common is leveraged by two additional scripts in scripts/ that are
>> called from
>> an upstart job I install.
>>
>> juju-run is neat, but it's still client side. I would also like the
>> freedom to
>> just fool around with the get/set agents outside of the debug-hooks
>> context
>> on the target e.g. to get a better feel for hacking new config.yaml keys.
>> Currently,
>> my only alternative is to intentionally create a broken hook and run
>> debug-hooks
>> from the discomfort of a screen'ish session.
>>
>>
>>>
>>>
>>> gustavo @ http://niemeyer.net
>>>
>>
>> --
>> Juju mailing list
>> Juju@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>
>



-- 
gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Control different relation sequence

[Gustavo Niemeyer]

Exactly, that's what I would probably do as well. Once you are within
a relation you want to wait for further actions, dump the
$JUJU_RELATION_ID into a file and wait until you want to wake it up
again.  Hooks are guaranteed to be run in series, so you don't have to
worry about concurrency issues around the file.

On Wed, Sep 4, 2013 at 12:01 AM, Mike Sam  wrote:
> Thanks. Does this mean that the charm should cache the relation id's in a
> text file or something?
>
>
> On Tue, Sep 3, 2013 at 7:33 PM, Gustavo Niemeyer
>  wrote:
>>
>> The relation-set command accepts a "-r" parameter which takes the relation
>> id
>> to act upon. You can pick the relation id of an executing hook from
>> the JUJU_RELATION_ID environment variable. This way you can act across
>> relations.
>>
>> Hopefully this will be better documented at some point.
>>
>> On Tue, Sep 3, 2013 at 11:23 PM, Mike Sam  wrote:
>> > Thanks Gustavo but I did not quite get your point. The problem is that
>> > for
>> > the new unit for service A, the dependent hooks are on two different
>> > independent relationships. I mean I can control when the new unit of
>> > Service
>> > A has properly established a relation with all the units of service B on
>> > say
>> > relation x_relation_changed, but how do I make all the units of service
>> > C to
>> > now trigger the y_relation_changed hook of the Service A unit because
>> > the
>> > unit is ready to process them? How do I make y_relation_changed hook to
>> > get
>> > triggered AGAIN (in case it has already been triggered but ignored
>> > because
>> > relation with service B was not done setting up) when x_relation_changed
>> > see
>> > fit? Would you please explain your point is the Service A, B, C context
>> > of
>> > my example?
>> >
>> >
>> >
>> >
>> > On Tue, Sep 3, 2013 at 6:38 PM, Gustavo Niemeyer
>> >  wrote:
>> >>
>> >> Hi Mike,
>> >>
>> >> You cannot control the sequence in which the hooks are executed, but
>> >> you have full control of what you do when the hooks do execute. You
>> >> can choose to send nothing to the other side of the relation until its
>> >> time to report that a connection may now be established, and when you
>> >> do change the relation, the remote hook will run again to report the
>> >> change.
>> >>
>> >> On Tue, Sep 3, 2013 at 10:17 PM, Mike Sam  wrote:
>> >> > Imagine a unit needs to be added to an existing service like service
>> >> > A.
>> >> > Service A is already in relations with other services like Service B
>> >> > and
>> >> > Service C on different "requires".
>> >> >
>> >> > For the new unit on Service A to work, it needs to first process the
>> >> > relation_joined and relation_changed with the units of service B
>> >> > before
>> >> > it
>> >> > could process  relation_joined and relation_changed with the units of
>> >> > service C.
>> >> >
>> >> > Is there a way to enforce such desired sequence relationship
>> >> > establishment
>> >> > at the charm level? In other words, I do not think we can control the
>> >> > hook
>> >> > execution sequence of different relationships officially but then I
>> >> > am
>> >> > wondering how can we do a situation like above nicely?
>> >> >
>> >> > Thanks,
>> >> > Mike
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Juju-dev mailing list
>> >> > juju-...@lists.ubuntu.com
>> >> > Modify settings or unsubscribe at:
>> >> > https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> >> >
>> >>
>> >> --
>> >> gustavo @ http://niemeyer.net
>> >
>> >
>>
>>
>>
>> --
>> gustavo @ http://niemeyer.net
>
>



-- 
gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Control different relation sequence

[Gustavo Niemeyer]

The relation-set command accepts a "-r" parameter which takes the relation id
to act upon. You can pick the relation id of an executing hook from
the JUJU_RELATION_ID environment variable. This way you can act across
relations.

Hopefully this will be better documented at some point.

On Tue, Sep 3, 2013 at 11:23 PM, Mike Sam  wrote:
> Thanks Gustavo but I did not quite get your point. The problem is that for
> the new unit for service A, the dependent hooks are on two different
> independent relationships. I mean I can control when the new unit of Service
> A has properly established a relation with all the units of service B on say
> relation x_relation_changed, but how do I make all the units of service C to
> now trigger the y_relation_changed hook of the Service A unit because the
> unit is ready to process them? How do I make y_relation_changed hook to get
> triggered AGAIN (in case it has already been triggered but ignored because
> relation with service B was not done setting up) when x_relation_changed see
> fit? Would you please explain your point is the Service A, B, C context of
> my example?
>
>
>
>
> On Tue, Sep 3, 2013 at 6:38 PM, Gustavo Niemeyer
>  wrote:
>>
>> Hi Mike,
>>
>> You cannot control the sequence in which the hooks are executed, but
>> you have full control of what you do when the hooks do execute. You
>> can choose to send nothing to the other side of the relation until its
>> time to report that a connection may now be established, and when you
>> do change the relation, the remote hook will run again to report the
>> change.
>>
>> On Tue, Sep 3, 2013 at 10:17 PM, Mike Sam  wrote:
>> > Imagine a unit needs to be added to an existing service like service A.
>> > Service A is already in relations with other services like Service B and
>> > Service C on different "requires".
>> >
>> > For the new unit on Service A to work, it needs to first process the
>> > relation_joined and relation_changed with the units of service B before
>> > it
>> > could process  relation_joined and relation_changed with the units of
>> > service C.
>> >
>> > Is there a way to enforce such desired sequence relationship
>> > establishment
>> > at the charm level? In other words, I do not think we can control the
>> > hook
>> > execution sequence of different relationships officially but then I am
>> > wondering how can we do a situation like above nicely?
>> >
>> > Thanks,
>> > Mike
>> >
>> >
>> >
>> >
>> >
>> > --
>> > Juju-dev mailing list
>> > juju-...@lists.ubuntu.com
>> > Modify settings or unsubscribe at:
>> > https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> >
>>
>> --
>> gustavo @ http://niemeyer.net
>
>



-- 
gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: juju is slow to do anything

[Gustavo Niemeyer]

I was planning to mumble something about how the API would improve
performance, but that message is great John. Full of insightful
details.

Thanks!

On Fri, Aug 30, 2013 at 3:15 PM, John Arbash Meinel
 wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 2013-08-30 14:28, Peter Waller wrote:
>> For the record, I sent the link privately. The run took about 22s
>> but I have measured 30s to 1m.
>
> Some thoughts, nothing that I can give absolute confirmation on.
>
> 1) Next week we have a group sprinting on moving a lot of the command
> line operations from being evaluated by the client into being
> evaluated by the API server (running in the cloud) and then returned.
> The explicit benefits that I've seen in other commands are pretty good.
>
> 'juju status' is going to be a command that should see a rather large
> improvement. Because it does round trip queries for a lot of things
> (what machines are there, what are the details of each machine, what
> are the units running on each one, etc).
>
> I've prototyped doing those queries in parallel, or trying to do bulk
> ops, which actually helped a lot in testing (this was for hundreds of
> units/machines).
>
> Doing it on the API server means any round trips are "local" rather
> than from your machine out to Amazon.
>
> 2) From here, 'time juju status' with a single instance running on ec2
> is 10s. Which breaks down roughly 4s to lookup the IP address, 2s to
> establish the state, and 4s to "finish up". (resolution of this is 1s
> granularity)
>
> Similarly "time juju-1.13.2 get not-service" takes 8.5s to run. 4s to
> lookup the address, 2s to connect, and 3s to give the final 'not
> found' result.
>
> With trunk, "time ./juju get not-service" is 4.6s. 2s to lookup IP
> address, 2s to connect, and the not-found result is instantaneous.
>
> So I would expect the 10s of a generic "juju status" to easily drop
> down to sub 5s. Regardless of any round-trip issues.
>
> 3) We are also looking to cache the IP address of the API server, to
> shave off another ~2-4s for the common case that the address hasn't
> changed. (We'll fall back to our current discovery mechanism.)
>
> 4) There seems to be an odd timing of your screen cast. It does report
> 22s which matches the times reported in the debug output. But the
> total time of the video is 20s including typing. Is it just running
> 2:1 speed?
>
> You can see from the debug that you have 7s to lookup the address to
> connect to, and then about 1s to connect. The rest is time spent
> gathering the information.
>
> I expect it to get a whole lot faster in a couple more weeks, but I'm
> not going to guarantee that until we've finished the work.
>
> 5) If I counted correctly, you have about 23 "machines" that are being
> considered. A bunch of them down/pending/errored.
>
> I would think for the errored ones you could do some sort of "juju
> destroy-machine". It might make things better (less time spent
> checking on machines you don't care about.)
>
> What happens when you try it? (There may be other issues that make us
> think we are waiting for something to happen with a machine that we
> don't want to destroy it.)
>
>
> Anyway in summary, this should be getting better, but I won't have
> explicit numbers until the work is done.
>
> John
> =:->
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.13 (Cygwin)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iEYEARECAAYFAlIg4UIACgkQJdeBCYSNAAOfxQCeMaRQdqvdyQ11WyRnJ/WPAccp
> IysAniDrUq6IDtM0fu9SuZg+2AQto8rP
> =JaZw
> -END PGP SIGNATURE-
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju



-- 

gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: relation departure timing changes

[Gustavo Niemeyer]

On Fri, Aug 23, 2013 at 12:32 PM, William Reade
 wrote:
> True -- but I don't think the macro effects can actually be distinguished.
> Until the mooted second stage, the effect of a dying *relation* will stay
> the same: all units just run for the exit without paying any attention to
> each other :).

It's not clear what is implied in that, so I'm sorry if that sounds
repetitive: if we're improving the termination of relations, we ought
to be doing it both if a unit is dying or if a relation is dying,
rather than special casing one of them. Charm authors shouldn't have
to take into account which of the two cases they're designing for,
because it's the same operation.

> I agree; the impact on a well-written charm, that is already prepared for
> possible silent breakage of related units at any time, will be nil; but even
> a perfect and complete implementation of everything discussed here will not
> free charms from that responsibility. I don't think that's *in itself* a
> strong argument against making a minor change that moves us a step towards a
> clearer model; but I agree that it would be deeply unhelpful to misrepresent
> the guarantees we do make even by implication. I think that's about
> messaging, not about implementation, though.
(...)
> Part of what we're doing here is deciding whether it's worth investing in,
> which will then feed into the question of when :).

If the way it will work is "whether stuff runs or not is sheer luck",
then there's IMO no point in spending any time on this at all, and the
messaging should be "this is how it works; deal with it".

If, on the other hand, we're spending time on this, we should do
better than sheer luck, so the message can be "in those circumstances,
the hooks run".

(...)
> 1) peers notify peers of their own imminent departure as soon as they're
> aware of it
(...)
> 3) providers notify requirers of their own imminent departure as soon as
> they're aware of it
(...)
> It would be extremely cheap to implement (1) and (3) today, but that work
> would not allow us to guarantee any new properties about the system.

Yes, but these offer zero gain if the guy on the other end is gone by
the time the notifications arrive.

> There are legitimate concerns that, in the absence of extremely clear
> messaging, implementing (1) and (3) alone may lead charmers to infer the
> existence of guarantees that do not in fact exist.

Exactly.


gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: relation departure timing changes

[Gustavo Niemeyer]

Also, and again, this isn't about dying vs. not dying. This is about
broken relations in general, isn't it?

On Fri, Aug 23, 2013 at 7:56 AM, Gustavo Niemeyer  wrote:
> On Fri, Aug 23, 2013 at 7:37 AM, William Reade
>  wrote:
>> Currently, we *guarantee* that destroying server_service/0 will cause its
>> relationship with client_service units to be torn down before those units
>> become aware. At this stage, I'm narrowly proposing that we move the
>> synchronization point such that it's *possible* (not guaranteed) for units
>> of client_service to respond to server_service/0's destruction before it
>> materially affects them.
>
> I agree it's suboptimal, but unless we render the alternative scenario
> into a pretty strong possibility rather than mere chance, there would
> be little point in investing much time on this. Saying "oh, perhaps it
> will run if you're in a good day" isn't any better in terms of API
> than "you cannot depend on this". It would make people spend time
> trying to follow the pattern, and then wonder why it doesn't work.
>
>> It's important to realise that the *guarantees* made by the system do not in
>> fact become any stronger under the proposed model. If a unit of
>> client_service is (say) running a slow config-changed hook when (2a) comes
>> to pass, server_service/0 will *not* wait for that unit to handle depart
>> before cutting off access. It *would* in fact be possible to do this; but
>> the tradeoff in play there is whether we want an unresponsive or missing
>> unit of client_service to be capable of blocking the shutdown of
>> server_service/0. I'm +1 in theory, but nervous in practice; without
>> implementing `destroy-unit --force`, which is not entirely trivial (largely
>> but not entirely because it's blocked on "" [0]), that change could lead to
>> deadlocked environments. If you'd like the system to make this guarantee as
>> well, please let me know: I can't promise anything about scheduling
>> decisions there, but it will be useful input into those decisions ;).
>
> If we're investing on this, it definitely sounds like we should at
> least have a clear and well-defined behavior for when or it will or
> will not run, and the cases where it will not run should be mapped to
> understandable events, otherwise we're not improving the situation.
>
>
> gustavo @ http://niemeyer.net



-- 

gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: relation departure timing changes

[Gustavo Niemeyer]

On Fri, Aug 23, 2013 at 7:37 AM, William Reade
 wrote:
> Currently, we *guarantee* that destroying server_service/0 will cause its
> relationship with client_service units to be torn down before those units
> become aware. At this stage, I'm narrowly proposing that we move the
> synchronization point such that it's *possible* (not guaranteed) for units
> of client_service to respond to server_service/0's destruction before it
> materially affects them.

I agree it's suboptimal, but unless we render the alternative scenario
into a pretty strong possibility rather than mere chance, there would
be little point in investing much time on this. Saying "oh, perhaps it
will run if you're in a good day" isn't any better in terms of API
than "you cannot depend on this". It would make people spend time
trying to follow the pattern, and then wonder why it doesn't work.

> It's important to realise that the *guarantees* made by the system do not in
> fact become any stronger under the proposed model. If a unit of
> client_service is (say) running a slow config-changed hook when (2a) comes
> to pass, server_service/0 will *not* wait for that unit to handle depart
> before cutting off access. It *would* in fact be possible to do this; but
> the tradeoff in play there is whether we want an unresponsive or missing
> unit of client_service to be capable of blocking the shutdown of
> server_service/0. I'm +1 in theory, but nervous in practice; without
> implementing `destroy-unit --force`, which is not entirely trivial (largely
> but not entirely because it's blocked on "" [0]), that change could lead to
> deadlocked environments. If you'd like the system to make this guarantee as
> well, please let me know: I can't promise anything about scheduling
> decisions there, but it will be useful input into those decisions ;).

If we're investing on this, it definitely sounds like we should at
least have a clear and well-defined behavior for when or it will or
will not run, and the cases where it will not run should be mapped to
understandable events, otherwise we're not improving the situation.


gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: relation departure timing changes

[Gustavo Niemeyer]

On Fri, Aug 23, 2013 at 6:30 AM, William Reade
 wrote:
> "The unit should depart" means that related units should start running the
> -departed hook for that unit. At the moment, related units will only run
> those hooks after the dying unit has run its -broken hook, and that means
> that there is *always* a window, after the dying unit has torn down any
> relation state, in which all related units still believe it to be active.

Sounds fine and adequate to be running the departed hooks on related
units before the broken hook of the departing unit. But that sounds
unrelated to dying, and it also sounds like an interesting
synchronization problem to solve, in the sense that the departing unit
will have to be able to tell when other units are done. That's easy,
conceptually, but corner cases will make it not fun. What's the plan
for establishing that?

> ...but the problem is that, even if everyone agrees with the above, any
> potential existing relations that *don't* follow the assumed model of
> providers/requirers will find such a change unhelpful.

Unhelpful but not a problem either, right?

> Perhaps I'm conflating two problems -- that of determining the *ideal*
> behaviour around departure synchronization, and that of determining the best
> *practical* behaviour given the implicit constraints in the existing charm
> ecosystem -- but I think we need to consider the two issues together lest we
> do something entirely crazy ;).

I can't quite put my finger on where our misunderstanding is yet, but
we may be close to finding out.


gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: relation departure timing changes

[Gustavo Niemeyer]

A dying unit *is* an active unit. The unit becomes dying
instantaneously, so there's no way for it not to be active during some
of the time it is dying, and that's precisely the point of having the
state at all. It allows running polite shutdowns with awareness,
before it becomes fully "dead".

> As discussed in the bug[0], it makes sense to me that dying units should 
> depart as soon
> as they know that they're being destroyed.

Definitely in agreement. The question is simply what "the unit should
depart" means. What raised by eyebrows was the side effects mentioned
in this thread, which I still don't understand.


On Thu, Aug 22, 2013 at 6:53 PM, Matthew Wedgwood
 wrote:
> On 08/22/2013 11:57 AM, Gustavo Niemeyer wrote:
>>> requirers connect to providers and not vice versa.
>>
>> I'm not sure I understand what that means. A relation is a bilateral
>> communication channel, and it is entirely valid for a requirer to send
>> information to a provider. What does it mean for a requirer to
>> "connect" to a provider and not vice versa?
>>
>> Although I cannot yet put my finger on a specific reason, I have a bad
>> feeling about the change suggested. Reporting something as dead while
>> it is in fact still around doesn't seem great.
>>
>
> I can't think of a case where a dying unit should be treated as an active 
> unit, except possibly for cleanup procedures. From the docs[1], "A unit's 
> relation settings persist beyond its own departure from the relation". That 
> implies to me that the unit's relation settings can be queried (for cleanup 
> purposes) even after it departs. As discussed in the bug[0], it makes sense 
> to me that dying units should depart as soon as they know that they're being 
> destroyed.
>
> Additionally, I can't think of a case where a "provider" or "consumer" would 
> imply any difference in that regard.
>
>>
>>
>> On Thu, Aug 22, 2013 at 1:42 PM, William Reade
>>  wrote:
>>> Hi all
>>>
>>> This is inspired by the "relation-list reporting dying units" bug [0], which
>>> can be reasonably simply fixed by reporting peer units' departure from a
>>> relation as soon as we know that they're being destroyed (rather than
>>> *after* the unit has handled the departure as we currently do [1]).
>>>
>>> I'm not aware of any reason this measure might be controversial (please let
>>> me know if you are); but it raises an interesting question whose answer
>>> hinges on common practice across the charm community. So far, there has been
>>> no practical distinction between relation providers and requirers; we're
>>> considering introducing an asymetry in the relationship, such that providers
>>> signal departure early as above (but requirers continue to signal departure
>>> only once they have actually departed).
>>>
>>> This makes intuitive sense given the names of the roles; but it only makes
>>> practical sense if requirers and providers are written such that,
>>> essentially, requirers connect to providers and not vice versa. Many charms
>>> certainly do work this way, and would thus benefit from such a change; but
>>> it's hard to audit this behaviour across the charm ecosystem. However, I'd
>>> like to know if anyone is aware which charms (or, most likely, entire
>>> interfaces) would *not* work correctly if we were to introduce this
>>> behaviour; this will help us figure out if, when, and how to do so.
>>>
>>> Cheers
>>> William
>>>
>>>
>>> [0] https://bugs.launchpad.net/juju-core/+bug/1192433
>>>
>>> [1] https://juju.ubuntu.com/docs/authors-charms-in-action.html -- "Departing
>>> relations"
>>>
>>> --
>>> Juju mailing list
>>> Juju@lists.ubuntu.com
>>> Modify settings or unsubscribe at:
>>> https://lists.ubuntu.com/mailman/listinfo/juju
>>>
>>
>>
>>
>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju



-- 

gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: relation departure timing changes

[Gustavo Niemeyer]

On Thu, Aug 22, 2013 at 2:04 PM, Nate Finch  wrote:
> Seems like we need a third state: "Leaving" - means you shouldn't use the
> unit (treat it as dead), but anything required by that unit still counts as
> in use and required until it's actually gone.

We already have the dying state precisely for handling the shutdown
routines cleanly, and as a state that precedes the "dead" state while
the entity is still active. It's not clear why we'd need another state
representing the same thing.

> Gustavo - I think what William means about who connects to whom is who
> initiates the connection. When Wordpress connects to MySQL, it is WordPress
> that initiates the relationship & thus the connection. WordPress requires
> MySQL, but MySQL doesn't care if WordPress exists or not, it just does what
> it's told by WordPress (or anyone else).

I still don't understand how that matters for relations. So far
they've been agnostic to what kind of TCP communication, or whatever
interaction really, happens as a consequence of the relation being
established.


gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: relation departure timing changes

[Gustavo Niemeyer]

> requirers connect to providers and not vice versa.

I'm not sure I understand what that means. A relation is a bilateral
communication channel, and it is entirely valid for a requirer to send
information to a provider. What does it mean for a requirer to
"connect" to a provider and not vice versa?

Although I cannot yet put my finger on a specific reason, I have a bad
feeling about the change suggested. Reporting something as dead while
it is in fact still around doesn't seem great.



On Thu, Aug 22, 2013 at 1:42 PM, William Reade
 wrote:
> Hi all
>
> This is inspired by the "relation-list reporting dying units" bug [0], which
> can be reasonably simply fixed by reporting peer units' departure from a
> relation as soon as we know that they're being destroyed (rather than
> *after* the unit has handled the departure as we currently do [1]).
>
> I'm not aware of any reason this measure might be controversial (please let
> me know if you are); but it raises an interesting question whose answer
> hinges on common practice across the charm community. So far, there has been
> no practical distinction between relation providers and requirers; we're
> considering introducing an asymetry in the relationship, such that providers
> signal departure early as above (but requirers continue to signal departure
> only once they have actually departed).
>
> This makes intuitive sense given the names of the roles; but it only makes
> practical sense if requirers and providers are written such that,
> essentially, requirers connect to providers and not vice versa. Many charms
> certainly do work this way, and would thus benefit from such a change; but
> it's hard to audit this behaviour across the charm ecosystem. However, I'd
> like to know if anyone is aware which charms (or, most likely, entire
> interfaces) would *not* work correctly if we were to introduce this
> behaviour; this will help us figure out if, when, and how to do so.
>
> Cheers
> William
>
>
> [0] https://bugs.launchpad.net/juju-core/+bug/1192433
>
> [1] https://juju.ubuntu.com/docs/authors-charms-in-action.html -- "Departing
> relations"
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>



-- 

gustavo @ http://niemeyer.net

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju