Re: Config Management Camp, 2-3 Feb, Gent, Belgium
On Tue, Oct 29, 2013 at 4:31 PM, Jorge O. Castro wrote: > Hi everyone, > > We've been invited to join Ansible, Chef, Puppet, CFEngine, Rudder, > and SaltStack at the Config Management Camp: http://cfgmgmtcamp.eu/ > > This event takes place _after_ FOSDEM in Gent, Belgium. Not only are > we participating, but we've secured a track for us to discuss all > things Juju. I've asked around and the following people are confirmed > to want to go: > > - James Page > - Martin Packman > - Robie Basak (Not Juju specific, but having a server coredev handy > doesn't hurt) > - Roger Peppe > > Anyone else want to provide content? We'd like to fill in a day's > worth of content for participants, as well as hop into the main track > to show Juju to other communities. Hi Jorge, Depending what the others are already doing, I could do demos or workshops deploying a charm using the salt or ansible support that we've added to charm-helpers. I've got some private charms using those, so it'll be easy to apply that to a public charm (although I'd like to do some improvements to the ansible support). I could also add similar support to charm-helpers for puppet too if it's worthwhile. No problem if that kind of thing is already covered. Cheers, Michael > > -- > Jorge Castro > Canonical Ltd. > http://juju.ubuntu.com/ - Automate your Cloud Infrastructure > > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Configuration scope
On Thu, Jan 30, 2014 at 3:03 AM, Sameer Zeidat wrote: > Hello, > > Not really a question, but just a thought. Would be nice if juju had a scope > added for charm config parameters, i.e. install vs. runtime. Such that it > will make those parameters needed only during deployment of a service > mutable afterwards. Instead of people writing charms silently ignoring such > changes, while juju, on the other hand, accepting and reflecting such > changes as if the had actually happened! Hi Sameer, I've thought about this a few times with the charms I've worked on, and generally it's been possible without too much difficulty to always reflect the changes in the config_changed hook. What are a some examples of install config items that you wouldn't be able to change at runtime? (I've dealt with location of config/log files etc., versions of installed deps etc., but I'm guessing you've got something trickier). Thanks! -Michael -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Configuration scope
On Thu, Jan 30, 2014 at 9:45 AM, Stuart Bishop
wrote:
> On Thu, Jan 30, 2014 at 2:55 PM, Michael Nelson
> wrote:
>
>>> Not really a question, but just a thought. Would be nice if juju had a scope
>>> added for charm config parameters, i.e. install vs. runtime. Such that it
>>> will make those parameters needed only during deployment of a service
>>> mutable afterwards. Instead of people writing charms silently ignoring such
>>> changes, while juju, on the other hand, accepting and reflecting such
>>> changes as if the had actually happened!
>>
>> Hi Sameer,
>>
>> I've thought about this a few times with the charms I've worked on,
>> and generally it's been possible without too much difficulty to always
>> reflect the changes in the config_changed hook. What are a some
>> examples of install config items that you wouldn't be able to change
>> at runtime? (I've dealt with location of config/log files etc.,
>> versions of installed deps etc., but I'm guessing you've got something
>> trickier).
>
> I have a validate_config() function in the PostgreSQL charm up for
> review. If an invalid config change is requested, it emits an error
> message and causes the hook to fail. The main use case is the
> PostgreSQL major version to deploy. The charm does not support major
> PostgreSQL version upgrades yet, and downgrades are impossible to
> automate (and sometimes just impossible). Your charm does of course
> have to be smart enough to remember its previous state so it can
> detect changes.
>
> If there was someway for my charm to reject config changes rather than
> go into a fail state, that would be nice. I'm not sure it is possible
> though, as 1) we don't want to block a config update on the hooks
> successfully completing 2) if juju rolls back config changes, it needs
> some way of informing the user.
Interesting - so the version of an application is a config option, but
the charm cannot (for whatever reason) handle that changing once
installed. That's a good example, thanks. One idea that might handle
that situation (as well as Sameer's general situation) with a simple
UI:
1) The config.yaml supports a 'const: true' option in the config.yaml,
2) `juju set` will fail with a nice message ("The postgres charm does
not support changing postgres_version once deployed.") if the user
tries to set/change a const config option.
If at a later point, you manage to update the charm so that the
config-changed hook can handle that item changing, the const can be
removed at the same time, so that as soon as people have upgraded the
charm, they can change that config item.
It wouldn't handle a situation where you do allow one type of change,
but not another (like upgrading but not downgrading).
--
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
Re: Charns with Multiple Maintainers
On Mon, Mar 3, 2014 at 3:26 PM, Charles Butler wrote: > Greetings everyone, > > It occurs to me that having a single maintainer of a charm while functional, > doesn't offer the best route moving forward for a charms longevity. I would > like to propose a format change to the authorship line of charms allowing > for more than a single entity to maintain the charm. > > My initial thought would be to convert the entry in metadata.yaml from a > string type to a list type > > maintainer: > - Jon Doe > - Steve Doe > > This seems like it would have larger rippling effects in surrounding > projects - charmworldlib and juju-gui come to mind. > > What do you guys think? Would the above spec work to accomplish the goal? +1. I needed something along those lines the other week for the elasticsearch charm I worked on. Thanks Charles. -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Best practices for "fat" charms
On Tue, Apr 1, 2014 at 10:05 PM, Matt Bruzek
wrote:
> Thanks Jorge,
>
> Not sure we want to call them "fat" charms, maybe "enterprise" charms. Here
> is my approach when making a charm work on the enterprise or limited
> networks.
>
> 1) Find out what hook downloads the packages that we are unable to access
> (wget, curl, or special ppa repositories). The enterprise network will
> block these requests often resulting in a charm hook failing.
> 2) Download the necessary packages from system that has access.
> 3) Upload the packages to the locked down system, copying the packages to a
> directory on the local charm.
> 4) Edit the local charm hooks to check for the package in the local
> directory first and if that does not exist, the charm would continue to
> download the files (using wget, or curl, or custom ppa).
>
> I believe we could provide a charm-tools method that does something like
> this and we could use this in charms to create "enterprise" charms that are
> able to be used on limited network environments.
>
> However this creates an interesting problem that I have not figured out a
> good way to resolve yet (your feedback requested).
Hi Matt,
We've found and used a few solutions when deploying charms internally
- as we don't deploy services that rely on PPAs or external resources.
One solution which we use in the elasticsearch charm is to rely on
repositories and ensure the config has sensible defaults. This works
well for us because our IS team maintains private repos which we can
use for deployments, and in this case, elasticsearch.org have their
own repository - but it won't work when either of those aren't true
for your situation.
You'll see in that config [1] that the default apt-repository is the
one from elasticsearch.org - perfect for the charm-store, but when we
deploy that we override it with our custom internal repository.
Similarly, there's an option in the config whether to install ansible
(which is used within the charm) from a ppa which defaults to yes for
the charmstore, but we override that. For us, this has been the best
way to bring the functionality to the public charm while allowing us
to deploy it internally.
If you don't have the option of using repositories as above, you can
do something similar to the suggestions of others above, but to be
explicit, the following in the install hook:
{{{
if {{ charm_dir }}/files/my-dependency.jar doesn't exist:
download it from public url to /srv/foo/deps/my-dependency.jar
else:
copy it from {{ charm_dir }}/files/my-dependency.jar to
/srv/foo/deps/my-dependency.jar
install or use /srv/foo/deps/my-dependency.jar
}}}
So it'll work from the charm store, but a user of your charm can also
ensure that it exists in the charm before they deploy it as part of
their build/deploy process. We used to do this with our elasticsearch
charm before elasticsearch.org publicised their repos.
Another option that you have, which we also use but for different
reasons, is to add an exec.d directory to the charm only when
deploying it in an enterprise environment, and ensure that the first
thing the charm does in the install hook is to call
charmhelpers.payload.execd.execd_preinstall(), like [2]. This enables
people to customise the unit, or put specific resources in place using
whatever method they need in their environment. IMO, all charms should
be calling charmhelpers.payload.execd.execd_preinstall() within the
install hook, so that the unit can be customised to some degree in
enterprise environments - but we need to be careful that it isn't used
to do something that should be done by the charm where everyone can
benefit.
Finally, another situation we have is for a generic wsgi-app charm
which we've started using, where we (will) deploy different
code-bundles depending on which service it's being used for (I've not
yet made this one public yet as it's not very juju-like to have a
wsgi-app charm that can be used for different services, although it
may make sense with bundles). In this case, we didn't want to use any
of the above because we don't want to have to 'upgrade-charm' to
deploy a new version of our application. So instead, the charm config
has two options - code_assets_uri and a code_archive (a relative path
from the code_assets_uri), like [3]. This could be a public url to a
git tarball, or in our case, the url to an internal swift container to
which we've published the build asset during an automated deploy test.
With that we can update our application code with a `juju set` (as
well as support rolling upgrades quite easily using a single juju run
command to update individual units to the latest code).
Sorry for the brain-dump nature of the above, hope it's got some
helpful bits :-)
-Michael
[1]
http://bazaar.launchpad.net/~michael.nelson/charms/precise/elasticsearch/trunk/view/head:/config.yaml
[2]
http://bazaar.launchpad.net/~michael.nelson/charms/precise/elasticsearch/trunk/view/head:/hooks/hooks.py
[3] http://paste.ubun
Re: Detecting cowboy'd changes in a Juju Env
On Wed, May 14, 2014 at 4:06 AM, Mark Canonical Ramm-Christensen wrote: > On Tue, May 13, 2014 at 10:45 AM, Joey STANFORD wrote: >> >> Howdy, >> >> >> On Tue, May 13, 2014 at 08:04:14AM +0400, John Meinel wrote: >>> >>> I actually think this isn't about someone doing "juju set-env" but >>> someone >>> just ssh'ing into the machine and changing things with a text editor. >> >> >> Yes, this is my top concern. > > > If you want this, there are tools that can help (puppet being one, the > already mentioned tripwire being another) both of which can be used with > juju. I know it doesn't help in the general charm case, but just as an example, any charm which uses the ansible helpers can do this quite easily, as ansible allows you to check a playbook and print any diffs that it finds, which can be run on a unit: ansible-playbook --check --diff --tags=install,upgrade-charm,config-changed /path/to/playbook.yaml This is demo'd in the following (private, sorry) example, I've run the playbook for one instance using --check --diff so that all the tasks for the install, upgrade-charm and config-changed hooks are checked. The output ends with a count of tasks which were OK and those which have changed. I then manually change a configuration file and re-run to show the output with the diff of the changed file. https://pastebin.canonical.com/110164/ This could be made even simpler with `juju run` and a helper in the charm... so `juju run --all check-for-diffs` would check all units etc. It will of course not find any manual changes made to the unit that are outside the scope of the charm itself. -Michael > > Once we have full modeling of a charms storage and installed "resources" we > might be able to do *more* in juju itself. But for now the way to prevent > this would be to write charms using puppet and doing enforcement that way, > deploying a properly configured tripwire like subordinate, or for properly > "scale out" workloads, just adding a new unit and killing any unit with any > SSH sessions that might have resulted in questionable activity. > > --Mark Ramm > > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Problem deploying test charm
On Fri, May 16, 2014 at 11:15 AM, Xander Maas wrote: > Hi all, > > I am trying to create an ownCloud charm with Ceph support. (I already had a > more or less working dev version) > > When I try to deploy my latest revision, juju doesn’t seem to get the latest > (local) revision. I noticed this, as I had a typo in my config.yaml It may not be the same reason, but I have seen this when I've got multiple copies of the same charm in my local JUJU_REPOSITORY, ie. something like: ~/myrepo/precise/my-charm and ~/myrepo/precise/an-older-charm-version juju uses the first charm which matches name in the metadata, so you want to move any old charm versions out of your juju repo. Another reason can be that the version of the charm that your bootstrap node has cached is greater than the one you're deploying, but you shouldn't normally get into that situation. -Michael -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Testing of ansible-based charms
Hi people, I saw a ping from last Friday: 12:21 < jcastro> bloodearnest, noodles775: evan is asking (We're at a sprint) on how we test the ansible bits in a charm? 12:47 < bloodearnest> jcastro: not easily, other than actually running it, which is a downside to using ansible currently 12:47 < bloodearnest> jcastro: the 2 things I do do are: 12:48 < bloodearnest> 1) use ansible-playbook --syntax-check to check for typos 12:48 < bloodearnest> 2) unit test any templates standalone 12:48 < jcastro> ack, thanks, exactly what we needed to know, thanks. 12:49 < bloodearnest> jcastro: we've talked about spending some time seeing if we can write something that mocks out ansible at some level (it's all py thon, after all) and thought it was worth a followup discussion. IMO unit-testing of an ansible charm only makes sense in the context of unit-testing any logic in your templates/playbooks and or the *syntax* of your playbook (as Simon mentioned). Unit-testing any actual tasks that modify the system is going to require lots of mocking, just like it does when unit-testing straight python hooks. It'd be better to have the ansible modules unit-tested themselves (if they are custom), and focus on integration tests for the charm (which can be included in your actual playbook - see the google-groups thread below). All the arguments both ways are exchanged between Michael deHaan (ansible) and a few others on this thread: https://groups.google.com/forum/#!topic/ansible-project/7VhqDDtf6Js summarised by one of the people asking about testing here: http://sharknet.us/2013/12/12/test-driven-development-with-ansible/ I've also read about people gating their ansible changes by running these tests on travis-ci (or anywhere where you can test against a known vm state): http://renemoser.net/blog/2014/01/01/test-ansible-roles-with-travis-ci/ We'd like to do something similar for the shareable ansible roles that we're using for our charms. If anyone has other ideas or issues, please contribute :) Cheers, Michael -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Hook firing
On Wed, Sep 3, 2014 at 9:57 PM, Darryl Weaver wrote: > Hi All, > > I am still a little confused as to when Juju decides to run hooks. > Obviously hooks run when something changes. > However, I see juju rujnning config-changed hooks at times when the > configuration has not changed recently. Hi Darryl, we've been seeing this too... > > So, I'd like to know how Juju decides to run the hooks and when? Simon was asking the same question on #juju the other day, apparently juju runs config-changed after a juju agent restart, but I don't know what's triggering that in our case. More at: http://irclogs.ubuntu.com/2014/09/01/%23juju.html#t15:20 > > Any advice would be helpful, > > Thanks to everyone in advance, > > -- > Best Wishes > > Darryl Weaver > Openstack Solutions Architect, > Canonical > Desk: +44 (0)207 630 2400 ext 507516 > Mobile: +44 (0)7720088049 > GPG FPR: EA3F 3805 9347 87EC 9CBB 8C1E DADC 82C9 B16B 0403 > > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Swift middleware subordinate charms?
On Wed, Sep 24, 2014 at 7:52 PM, Andrew Wilkins
wrote:
> Hi folks,
>
> I've just started looking into writing a charm (possibly two charms?) to
> deploy some middleware to Swift; both the proxy and storage will have
> middleware added. Today was the first time I've deployed any OpenStack
> component, so my terminology could be off.
>
> I imagine a middleware charm would make most sense as a subordinate to
> swift-{proxy,storage}. Are there any existing examples of such a thing? Is
> there a friendly way for the subordinate to tell the swift services to
> restart, or is it just a matter of "sudo restart swift-object-server" or
> whatever?
Hi Andrew. If you're writing a subordinate charm, it'll be relating to
the primary charm via some relation (say middleware-changed) right?
Which I assume you'll need to add to the swift-proxy charm (ie. to
support middleware subordinates generally? Not sure.)
Anyway, normally I think you'd want the primary charm's
(swift-proxy's) relevant relation-changed hook to do the restart
itself when the middleware changes (ie. when middleware-changed is
triggered). This just makes sure that the responsibility and knowledge
of restarts stays within the charm responsible for the service.
Sometimes your subordinate might need to cause a restart even though
nothing else about the relation has changed, which you can do with a
relation-set with an additional timestamp to ensure that the
relation-changed will still be triggered in the related charm which in
turn ensures the restart.
You could of course just have your subordinate restarting the service
itself, but it may bite later on (I think this issue in reverse hit
the gunicorn subordinate charm, which changed the service name and
hence the way restarts happen, but some charms using the gunicorn
charm had done the restarts themselves rather than via the
relationship, causing compatibility issues).
-Michael
>
> Cheers,
> Andrew
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
--
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
Re: Migrating the python-django charm to Ansible
Hi Patrick,
On Thu, Nov 6, 2014 at 4:22 AM, Patrick Hetu wrote:
> Since the last UDS, I'm trying to make the python-django charm evolve
> to use Ansible for is various tasks instead of using pure Python scripts.
>
> My previous attempt:
>
> lp:~patrick-hetu/charms/precise/python-django/ansible
>
> had some problems so I decided to restart from scratch here (work in
> progress):
>
> lp:~patrick-hetu/charms/precise/python-django/ansible_reboot
>
> Based on the following observations.
>
> Use roles that are not specific to Juju
> ---
>
> If you look at an Ansible role you will probably found a file
> defaults/main.yml or vars/main.yml. For me it was stunning how close
> those are to a charm configuration file.
Specifically, this is not the config.yaml you're talking about, but
the config file which is passed during deployment right? As in,
"myconfig.yaml" in:
juju deploy --config myconfig.yaml mycharm
?
>
> So why not grab this opportunity to use the same file
> (minus the charm name at the top [1]) with both Juju and Ansible
> and make Juju use the same roles that Ansible community are using by
> introducing
> shims in the charm playbook.
This should be possible without needing to re-use the actual file
itself - if I've understood correctly what you mean: any ansible-based
juju charm can reuse any community role for free, by using a shim to
set the roles expected variable names, based on the charm config?
So looking at the examples in your branch, if your charm has a config
option django-settings (and therefore playbook and templates have {{
django_settings }} available via the current helpers), while an
ansible-community django-project role expects a "settings_module"
variable, your example shim.yml does:
set_fact: shim_settings_module="{{ django_settings }}"
and then when you use the role, you pass in the the shim_settings_module:
- role: django-project
tags:
- install
django_settings: "{{ shim_settings_module }}"
But I don't see the need for the shim at all, when you can just pass
the charm config value straight through:
django_settings: "{{ django_settings }}
(or in the general case: django_settings: "{{ whatever_my_charm_config_uses }}")
I'm not sure whether I'm missing something about what you're meaning
(re-using the actual file), or whether you may have missed that all
the config is already available in the playbook/template contexts via
the existing charmhelper ansible support (unlikely, as you've used it
before I think). Let me know what I'm missing.
> [1] Using a configuration file without that charm name at the top
> was working in the past but now doesn't. Maybe this could be
> re-enable without to much negative impact.
That might be worth a juju bug. I assume it's related to the config
file being used generally to store config for multiple services within
your setup, but I don't see any reason why that should mean that if
the config file without the charm (service) name at the top is used
for deploying a single charm, that juju couldn't still accept it as
the config for that service.
>
> Couple the playbook with Juju with a shim
> -
>
> To get the playbook to work with an external role we could use
> a shim file that translate Juju's dynamic configuration
> and relation data to roles variables.
Sounds great to be able to reuse existing external roles, but let me
know what I'm missing about why a shim is needed to do this.
>
> I was thinking using a shim imported in the pre_tasks section of
> the charm's playbook that runs on all tags.
> (see: https://github.com/ansible/ansible/issues/3157)
>
> The shim would do things like:
>
> * Translating Juju variable to role variable
That can be done in the playbook where needed as above.
> * Sanitize variable like: unit_name, relation_name, etc
That *should* be done by the existing charmhelpers (it writes out
/etc/ansible/host_vars/localhost on each hook execution with config,
relations and some other things like unit name, public and private
addresses)
> * Ensure backward compatiblity
>
> Then using tags to dispatch based on the current relation
> to the corresponding roles.
>
> Build role for re-usability
> ---
>
Yes...
> This have the potential to solve the problem of repetition of some
> high level tasks in other charm.
> Like adding ppa,
You can add PPAs with the existing apt_repository module (see the
example there):
http://docs.ansible.com/apt_repository_module.html
> fetching source code, adding ssl certificate,
> configuring backup, etc
>
> A little bit like what charmhelper is doing right now.
Yes! I think making reusable roles for common tasks in juju charms is
really worthwhile [1]. We do have a bunch of public shareable roles
that Simon Davy and I (mainly) are reusing and improving for some
internal services [2]. Please take a look and see if there's things
that could be useful -
[Review Queue] elasticsearch ec2 fix (trusty)
After reviews by Simon and tests by Tim [1] I've just pushed an update to the trusty elasticsearch charm which: * Fixes the firewall configuration so that nodes in the cluster can find each other on ec2 (on ec2 the private-address is a private dns, rather than a private IP expected by ufw) [2] * Enables the firewall to be switched off for easy testing of the admin port [3] -Michael [1] http://reports.vapour.ws/charm-tests/charm-bundle-test-5034-results [2] https://bugs.launchpad.net/charms/+source/elasticsearch/+bug/1386664 [3] https://bugs.launchpad.net/charms/+source/elasticsearch/+bug/1376396 -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Unit-get
On Fri, Nov 7, 2014 at 3:19 PM, Andrew Wilkins wrote: > Hi Sameer, > > The behaviour changed a few months ago to address a bug: > https://bugs.launchpad.net/juju-core/+bug/1353442 > > Is this causing you problems, or were you just surprised/curious? Hi Andrew. This did cause a bug in the elasticsearch charm recently [1] - I'd not realised it was related to a juju change, but thought it was just a difference on ec2, that the private-address was not an IP address (I had only tested with local, canonistack and HP). The reason it caused an issue was because we were using the private-address as part of a firewall rule which required an IP address. We've pushed a fix now, but is there a way to foresee this kind of change in the future? Maybe for these changes which might affect charms, we could trigger retests for some set of approved charms across HP, ec2 etc.? [1] https://bugs.launchpad.net/charms/+source/elasticsearch/+bug/1386664 -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Thoughts about Juju local as Dev
On Fri, Dec 5, 2014 at 12:54 PM, Sebastian wrote: > Hey guys! > > I'm using Juju as a local environment a while now, and trying to implement > Juju local a default workflow of our developers, which half of them have > generally an Ubuntu (or at least based) and the other Mac OS X, but all are > using the Vagrant workflow. > > And I have to tell you, what a challenge. Hi Sebastian! Wow - on our team we're not yet using juju for the developer workflow... we'd like to, but there are still quite a few complexities (Simon Davy has made some great progress). What we are doing currently, which is working quite well is: 1) Devs generally have a single lxc/vm for each project which has the complete developer environment for branch, test, code, commit, submit for review. 2) When a new commit lands on our trunk (after gated CI testing), an automated test deployment runs using the relevant charms for the project (basically a juju-deployer config), verifying that the deployment (and upgrade) are still working. This way, devs don't need to care about juju for development, only for deployment. Some devs do both, some only development, but the key is that it's a choice. Note that to ensure that the dev enviroment is consistent with that created by the charm, we have all dependencies defined in the code branch itself. The charm refers to the dependencies.txt of the code and apt-get installs them (or whatever package manager is relevant). Hope that helps, -Michael -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: juju vs. Puppet/Chef/Salt/Ansible/etc.
On Tue, Dec 9, 2014 at 10:35 AM, Eric Snow wrote: > The reaction I get most often from folks that aren't familiar with > juju and skim through the juju site is that it looks like a competitor > to the various configuration management tools out there like Puppet or > Salt. However, my experience is that while they have some overlap, > they sit at different layers. > > Have I grown out of touch? Conceivably those projects have or are > working on juju-like functionality that I'm not aware of. If not (or > even if so), what's the best way to educate people on what juju is and > how it will help them when they're already steeped in the lower-layer > config. management world? > > Related to that, how can we help those same folks wrap all their > existing recipes, etc. in charms? It's got to be easy enough that > they can justify the effort. Hi Eric, In online-services we've been using ansible together with juju for quite a while, and have pushed some generic helpers into charm-helpers that allow running a hook to equate to running all tasks in a playbook tagged with the hook name [1][2]. After using that for over a year, we've also developed some reusable ansible roles which make it much easier for us to maintain lots of charms, but ymmv (as they're often for specific ways which we need to do things, like deploying code from a swift container, or setting up nrpe checks etc.) [3] Using those ansible helpers within juju really won't allow you to simply re-use your ansible playbook, but it's pretty easy to adapt an ansible playbook to a charm (ie. just tagging tasks for certain relations). Cheers, Michael [1] https://micknelson.wordpress.com/2013/11/08/juju-ansible-simpler-charms/ [2] https://github.com/absoludity/charm-bootstrap-ansible [3] https://micknelson.wordpress.com/2014/06/10/reusable-ansible-roles-for-juju-charms/ -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: apache2 vhost-config relation problem
On Tue, Jan 13, 2015 at 9:10 AM, Nicolás Pace wrote: > Hi! > First of all, thanks for such a great tool! > I've been implementing a simple charm to test apache2 vhost-config relation: > https://code.launchpad.net/~nicopace/+junk/simplewebservercharm > I can add both the apache2 and simplewebserver charm, but when i add the > relation, it says: > $ juju add-relation apache2:vhost-config simplewebserver:vhost-config > ERROR no relations found Hi Nicolas. I'm not 100%, but it looks like you've trying to relate two charms which both *require* the vhost-config relation (ie. without looking at the detail, I'm guessing your charm needs to provide that, not require it). -Michael > > Any ideas? > > Thanks > > -- > Ing. Nicolás Pace > http://www.linkedin.com/in/nickar/ > http://www.unixono.com.ar/ > > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: An Open Question: Charm Dependency Management
On Wed, Jan 21, 2015 at 5:17 AM, David Britton wrote: > On Tue, Jan 20, 2015 at 05:58:24PM +, Marco Ceppi wrote: >> I don't see how a Makefile in a charm doesn't resolve this issue. > > +1 on some standard published Makefile targets. We already have some > that are highly recommended: > > - test > - lint +1 (and -1000 on any Makefile targets which run as sudo) > > Maybe: > > - test-depends or depends # to install/update dependencies needed for > testing It doesn't hurt to have an extra target like test-depends, but it should be enough to just have the test target depend on some other target that's required for whatever technology the author is using. For eg., if it was a python charm that needed some python packages: .virtualenv: virtualenv .virtualenv .virtualenv/bin/pip install -r test_requirements.txt test: .virtualenv ./virtualenv/bin/python ./unit-tests Yep, you could make that DRYer, but it might be less readable. -Michael > > Are there others that are needed/missing or that I forgot we already > have as standard? > > -- > David Britton > > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Questions/comments about better settings for aws deploys
Hi people, I'm just testing an ec2 juju deployment, and had a few questions about setup and default secgroups, which aren't addressed on the docs afaics [1] First, the docs assume that you'll put your primary (administrator) creds in your environment, which is easiest and works, but it would be safer to create a separate identity within your account which you can manage in isolation [2]. So I'm doing that, creating a separate user for the environment, assigning it as a power user [3], but wanted to check whether there is a better setting (or specific policy) to use that will allow juju to do everything it needs and no more? Second, the default secgroup for an aws account allows (by default) all tcp/udp between all instances using that same default secgroup. This secgroup is *not* associated with the juju units of the environment (afaict, which is great, because it would mean that other instances in other envs but the same account could, as aws-classic only has one default per account, afaics). Juju seems to create it's own "default-for-environment" secgroup which is applied to all the units within the environment (in addition to one per unit), which has similar rules to the above (ie. all instances can talk to each other over all ports). It also allows inbound access to ssh, 17070 and 37017 for 0.0.0.0/0, which may be a sane default given that you might be deploying from your laptop on different networks, but if you're deploying from a specific machine, it makes sense to restrict those three (no question, just comment) Finally, unlike openstack secgroups, aws (classic) doesn't allow any outbound filtering rules on the secgroup (?!). Is anyone working around this, or does it require touching iptables on each of the units? Thanks for any info, Michael [1] https://jujucharms.com/docs/config-aws [2] https://aws.amazon.com/iam/ [3] "Provides full access to AWS services and resources, but does not allow management of Users and groups." I'll create a group and use that instead once I'm sure of the best permissions. -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: juju and openstack reseting secgroups automatically overnight
On Thu, Feb 12, 2015 at 5:39 AM, Caio Begotti wrote: > Hi folks, > > I wonder if any of you have had this problem before but Juju and Openstack > are resetting my secgroup rules every night. I hope this is comprehensible > without much details as it involves private deployment info... I know this > is not strictly speaking 100% Juju but anyway... I've just checked my ec2 test deployments, and I'm seeing the same behaviour on the secgroups there. Definitely worth a bug Caio (I'll do it if you don't get around to it, I don't see one at https://bugs.launchpad.net/juju-core/?field.searchtext=secgroup ). -Michael > > Juju creates the secgroup for Nova, right? I am manually setting a nova > secgroup-add-rule for port 22 like the following: > > nova secgroup-add-rule groupname tcp 22 22 ipaddress/32 > > However, my other rules (ICMP etc) are kept between days, but SSH rules for > port 22 are being reset and disappearing overnight. Is it a known issue or > expected behavior with Juju and Openstack? > > I was told Juju or Openstack (no idea who is at faul here, really) might > reset the secgroups from time to time (when exactly?) if the specified port > in the rule is not open in the Juju units. > > Ok, so I have created this charm > https://jujucharms.com/u/caio1982/open-port/ and I confirm that now port 22 > is open in all the related units whose IPs are in the secgroup rules. Still, > all SSH rules for port 22 are being reset every single night. > > Does it make sense? > > Right now I have an extra secgroup rule for 0.0.0.0/0 too, just to see what > happens tonight. > > I would really love to understand why Juju and Openstack are not playing > nice together with my secgroup rules :-( > > — Caio Begotti [ˈka.jo | be.ˈgɔ.t͡ʃi] > > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: What's the future of Juju?
On Thu, Mar 26, 2015 at 8:24 AM, Merlijn Sebrechts wrote: > Thanks for your answer! I didn't know Windows and Centos support was coming > so soon, great to know! > > The lacking documentation is the biggest issue to me. The charm-helpers > documentation is outdated in a lot of places and that makes it seem as it > isn't being actively maintained anymore. Ofcourse, this is a side-effect of > a rapidly expanding product... > The charm-helpers documentation also lacks some good examples and > "guidelines". Things like "What's the best way to create templates, What's > the easiest way to get relation data, ..". The documentation shows you how > to do it in bash, but is really lacking for python. I had a really hard time > trying to decipher how the services framework works exactly. The services framework is quite new... I've not yet had the chance to use it yet either. > The saltstack charm-helpers integration also has few problems. I just gave > up on it and wrote the install hooks in python. ... and yes, I'll try to get some time to reevaluate the saltstack charm-helpers intergration and see whether it should be fixed or removed. We (the team I work in) used it initially for some charms, but then migrated quite soon to use the ansible support which we're using now, but the next time we write a new charm, we'll evaluate the services framework instead. So the issue with the salt support may be that there are no charms (that I'm aware of) using it and so it is not being integration-tested regularly (though I'm keen to check). I'd recommend exactly what you're doing - starting your charm in plain python. As you develop the charm, you may start to understand the need for, and use the services framework (or other charm-helper-provided support). Cheers, -Michael > > > > 2015-03-25 21:32 GMT+01:00 Nate Finch : >> >> I'm a core dev on Juju, I can answer some, but not all of these questions. >> >> First off, as far as long term commitment for Juju - Juju is a huge part >> of Canonical's long term strategy... right up there with the Ubuntu Phone >> and Ubuntu itself. The Juju team has been expanding hugely in the last >> couple years... I forget exactly the numbers we're at now, but it's an order >> of magnitude more people working on Juju than there were just a couple years >> ago. >> >> Juju is used extensively internally at Canonical. We have a mandate that >> all internal services be deployed via Juju. >> >> As far as supporting other operating systems, we actually do support >> Windows, right now (though it can be a little tricky to set up, and >> generally only works on private clouds, due to licensing restrictions on >> distributing Windows images). See here: >> http://www.cloudbase.it/windows-with-juju-and-maas/ (Cloudbase partnered >> with us to get Juju working with Windows) >> >> Cloudbase is also currently tackling CentOS support. It currently works >> and is just being cleaned up, it should be available for testing in a few >> weeks. >> >> The number of features that have landed in the last year is tremendous - >> high availability, networking, storage, major improvements in the GUI, >> support for more clouds (Google Cloud Compute support is coming out with >> 1.23, which is due any day now), Windows support, backup and restore >> >> As for bugs, there are bugs in every product, especially new and rapidly >> expanding products, like Juju. If there are particular bugs that concern >> you, we'd be happy to look into them. We try to make sure that we fix >> anything that is a regression or would majorly hinder usage we do use >> this internally after all, so believe me, we hear about it when things >> aren't working well! :) >> >> I'm sorry you find the documentation lacking. We have been putting effort >> into that recently. I, personally, am a big fan of extensive documentation, >> and I know our documentation is not nearly as extensive as it could be. >> >> I can't personally talk about big companies using Juju... I know we have >> several very large companies doing very large installations, but I don't >> think anything is public about that. Hopefully someone else can bring up a >> list of people using Juju. >> >> Hope that answers at least some of your questions. >> >> >> On Wed, Mar 25, 2015 at 4:01 PM, Merlijn Sebrechts >> wrote: >>> >>> Hi >>> >>> >>> I'm interested in what the future of Juju is. From the small experience >>> I've had with it, it seems like a product with a lot of potential. It fills >>> a gap in our project that no other technology can fill. Its biggest strength >>> is how relations between services are managed. This is something that, to my >>> knowledge, does not exist in any tool today. It enables a very modular >>> approach and solves a lot of problems we would have with other tools. >>> >>> However, I've also seen some things that worry me. Even after three >>> years, there are still a lot of bugs in the project. The documentation is >>> lacking, esp
Re: Upgrading charms to 16.01
On Fri, Jan 29, 2016 at 9:43 AM Pshem Kowalczyk wrote: > Ok, > > force-downgrade doesn't seem to work: > ubuntu@maascontroller:~$ juju upgrade-charm keystone > > > > ERROR already running latest charm "cs:trusty/keystone-33" > ubuntu@maascontroller:~$ juju upgrade-charm --force keystone > ERROR already running latest charm "cs:trusty/keystone-33" > > I ended up removing each unit and re-adding it. This has resolved the > keystone issue. > > I have tried that method with another charm (glance): > > ubuntu@maascontroller:~$ juju set glance > openstack-origin=cloud:trusty-mitaka > ubuntu@maascontroller:~$ juju upgrade-charm --force glance > Added charm "cs:trusty/glance-30" to > the environment. > > that has resulted in exactly the same error: > 2016-01-28 22:40:39 ERROR juju-log FATAL ERROR: Could not derive OpenStack > version for codename: mitaka > 2016-01-28 22:40:39 ERROR juju.worker.uniter.operation runhook.go:107 hook > "config-changed" failed: exit status 1 > > So I think the only way is to blew away a service unit and deploy a new > one. > James' point above was that you need to upgrade the charm *before* you set the config to use mitaka. The error you are seeing above is on config-changed before the charm upgrade even happens, as the old charm doesn't know about mitaka. If you haven't already blown away your glance service, you can set the openstack-origin back for glance, resolve the config-changed error, then do the upgrade first.. > > kind regards > Pshem > > > On Fri, 29 Jan 2016 at 10:49 James Page wrote: > >> >> Hi Pshem >> >> On Thu, 28 Jan 2016 at 22:39 Pshem Kowalczyk wrote: >> >>> I've tried to upgrade keystone to the new charm version (from liberty). >>> I've updated the source: >>> >>> juju set keystone openstack-origin=cloud:trusty-mitaka >>> >>> and scheduled an upgrade: >>> >>> juju upgrade charm keystone >>> >>> but the charm upgrade fails: >>> >>> 2016-01-28 21:36:13 ERROR juju-log FATAL ERROR: Could not derive >>> OpenStack version for codename: mitaka >>> >>> What am I doing wrong? >>> >> >> You need to upgrade the charm first, and then set the configuration >> option as the old version of the charm does not know about mitaka. >> >> You can resolve this by doing: >> >> juju upgrade-charm --force keystone >> juju resolved --retry keystone/0 (or whatever the unit name is that >> failed) >> >> Hopefully that should fix you up. >> >> Cheers >> >> James >> -- >> Juju mailing list >> Juju@lists.ubuntu.com >> Modify settings or unsubscribe at: >> https://lists.ubuntu.com/mailman/listinfo/juju >> > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: upgrade-charm and default configuration values
Hi there Marco, On Wed, Mar 9, 2016 at 4:18 AM Marco Ceppi wrote: > Hello everyone! > > tl;dr: upgrade-charm also updates default configuration which can break > running deployments. Should we be updating charms anyways esp since > defaults are now broken for fresh deploys? Discuss. > I think the updating of default config options is a good discussion to have, but more specifically, the question I feel needs answering is: Can a charm not only update the default config options (to point to a new major version, for eg.) but also update it's support to only handle that new default option. This is, afaiui, what seems to happen. More details below... > > We've had a problem pop up a few times and I wanted to start a discussion > about it with the community. Default configuration values and upgrade-charm. > > We have a few examples where the charm by default is broken, but we're > getting resistance on updating these because of the default action in Juju. > On upgrade-charm, if the default configuration is still set on the charm, > then it will be updated on charm upgrade. I understand why this seems like > a good idea but I have a few examples where this really hurts. > > MySQL > > There is a long standing issue with MySQL on local/LXC where because of a > faulty configuration option default value which has existed since the charm > was created, it attempts to consume 80% of available ram for a cache > instead of just a sane default (300M for example) which MySQL experts > recommend. However, because of the implication of the change for running > deployments the merge request is essentially on-hold: > > [0] https://bugs.launchpad.net/charms/+source/mysql/+bug/1373862 > > ELK > > Recently, a few configuration options were changed for Kibana[1] that > aligned it with the latest release of that software. > The issue isn't just that the default config options were changed - if you check that MP [1] you'll see kibana3 used a javascript configuration file on the unit itself (search for files/charm/config.js), whereas kibana4 uses a yml file (search for files/charm/config.yml). So once I upgrade-charm to this version, even if I've explicitly set my kibana_source and kibana_source_checksum options so it won't unexpectedly get new defaults of kibana4, it still won't work because my kibana3 won't know what to do with the yml config file - it'll still expect the js one (or perhaps it'd just keep using the old file - but the init script which runs /srv/kibana4 would fail). In that case, it may be better if I'd not set kibana_source so I would upgrade to the new version - not sure. That is, the change here to upgrade-charm would break an existing deployment. I'm not saying it was the wrong thing to land those changes, just that we need to be aware (and make sure people who use the charms for more than demo's are aware). > This, and changes to logstash mean the elasticsearch charm need to be > updated too. As it stands today, with the defaults in the charms, there is > no working ELK stack[2]. > I don't think that's true - we're using elasticsearch2, kibana4 and logstash2 for an ELK deployment currently (since late last year). The change in the kibana charm (to kibana4) meant we needed to use ES2.x and Logstash2.x. I explicitly didn't land the es2 and logstash2 charm changes in the upstream stable charms for the above reason, instead creating an elasticsearch2 branch: https://code.launchpad.net/~onlineservices-charmers/charms/trusty/elasticsearch/elasticsearch2 Not ideal - but it won't break existing deployments. If you think it's better, I can propose the elasticsearch2 charm (and the related logstash2) for merging into the main "stable" trusty charms, but we just need to be aware that we may break existing deployments if people upgrade. I don't know that you can generally update a major version of a service automatically on upgrade-charm without affecting related services. We have an existing project which currently uses ES1.x which we're upgrading to 2.x (not via upgrade-charm). If you want more details about why we can't simply upgrade-charm, feel free to reach out to Wes Mason, who's doing the upgrade. Cheers, Michael > There's a merge[3] to address this but again is being met with resistance > because this implicit issue of existing deployments get the latest > configuration if it's still default. > > [1] > https://code.launchpad.net/~chris.macnaughton/charms/trusty/kibana/version_bump/+merge/276709 > [2] https://bugs.launchpad.net/charms/+source/elasticsearch/+bug/1553256 > [3] > https://code.launchpad.net/~lazypower/charms/trusty/elasticsearch/20-bump/+merge/288423 > > I'm inclined to merge these and properly educate users on configuration > pinning and bundles to make sure the latest software works as expected > > but am interested in feedback and discussing if we need to potentially > change the default behavior of Juju to if not allow the user to address > this at least provide feedback that
Re: Default Model SG Rules
On Sat, Jan 28, 2017 at 4:34 AM James Beedy wrote: > A default SG rule generated for every model allows 22 from 0.0.0.0/0, I'm > guessing this is because we are trying to facilitate the use case for juju > deployed on a public cloud, and instances being ssh accessed from the > internet and not from behind VPN in the same address space. > > A functionality which would allow users who don't want ssh open to the > world to close it, either completely, or limit to a private address space, > would be very helpful (especially because Juju reverts any changes made to > the SG, > I created a bug about that a while back: https://bugs.launchpad.net/juju-core/+bug/1420996 As per the last change there, it was targeted for 2.1.0 until just recently. > so I couldn't even lock down port 22 if I wanted to). > > Is it possible to introduce a model config param that we could use to tell > juju where to allow ssh traffic from? > Again, an older bug, but I'd be keen to see that not just for 22/ssh, but in general when exposing services: https://bugs.launchpad.net/bugs/1401358 but that may not fit the new juju2 models since the bug was written. > > Quick fix: Introduce an 'ssh-allow' param that could be used to open and > close port 22 on the SG generated for the model? > > Better fix: Introduce a config param 'ssh-access', where default value is > 0.0.0.0/0, which could then be modified to an address space that fits the > users security needs. > > How do others feel about this? > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
