Re: Juju 2.1.0, and Conjure-up, are here!
On Fri, Feb 24, 2017 at 11:14 AM, Andrew Wilkins < andrew.wilk...@canonical.com> wrote: > On Fri, Feb 24, 2017 at 6:51 PM Mark Shuttleworth wrote: > >> On 24/02/17 11:30, Andrew Wilkins wrote: >> >> On Fri, Feb 24, 2017 at 6:15 PM Adam Collard >> wrote: >> >> On Fri, 24 Feb 2017 at 10:07 Adam Israel >> wrote: >> >> Thanks for calling this out, Simon! We should be shouting this from the >> rooftops and celebrating in the streets. >> >> >> Only if you also wave a big WARNING banner! >> >> I can definitely see value in pre-installing a bunch of things in your >> LXD images as a way of speeding up the development/testing cycle, but doing >> so might give you false confidence in your charm. It will become much >> easier to forget to list a package that you need installing, or to ensure >> that you have the correct access (PPA credentials, or proxy details etc.) >> and having your charm gracefully handle when those are missing. >> >> Juju promises charms encoding operations that can work across multiple >> cloud providers, bare metal and containers please keep that in mind :) >> >> >> Indeed, and this is the reason why it wasn't called out. We probably >> should document it for power-users/charmers, but in general I wouldn't go >> encouraging its use. Optimising for LXD is great for repeat deploys, but it >> wouldn't be great if that leads to less attention to quality on the rest of >> the providers. >> >> Anyway, I'm glad it's helping make charmers' lives easier! >> >> >> We should call this out loudly because it helps people making charms. >> >> Those people are plenty smart enough to debug a failure if they forget a >> dependency which was preinstalled in their dev images. >> > > I was thinking about deployment times more than anything else. If you > don't feel your user's pain, you're less likely to make it go away. But > anyway, that can be fixed with automation as well (CI, as you say below). > I agree there is a risk here. In my specific case, I judge the benefits to outweigh the costs, by quite some margin. But that judgment is specific to my use case, where layer-basic and IS' basenode add significant package churn on every node[1] (increasing the benefit), and we have a full mojo-based CI pipeline for bundle changes (lowering the cost). On a different tack all together, I think that reducing iteration time for charm development is a *massive* win for users. Faster iterations mean faster feature development and bug fixes, and more comprehensive testing (as it costs less). I would estimate that iteration improvement would outweigh the increased risk from a missing pre-installed package, but YMMV. [1] ok, so not every charm we deploy is layer based, but they are heading that way... Don't HIDE something that helps developers for fear of those developers >> making mistakes, TEACH them to put CI or other out-of-band tests in place >> anyway that will catch that every time. >> > > FWIW, it wasn't intentionally hidden to start with, it was just missed. I > made the changes primarily to support an external user who wanted to demo > CentOS charms on LXD; the change also enabled custom images in general, and > also slightly improved container startup time. Three birds, one stone; only > one bird-hitting was reported ;) > This is hugely appreciated. I reckon 95% of my deployments in the average week are to lxd, so improvements to the lxd provider affect my velocity considerably. Thanks -- Simon -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Juju 2.1.0, and Conjure-up, are here!
On Fri, Feb 24, 2017 at 6:51 PM Mark Shuttleworth wrote: > On 24/02/17 11:30, Andrew Wilkins wrote: > > On Fri, Feb 24, 2017 at 6:15 PM Adam Collard > wrote: > > On Fri, 24 Feb 2017 at 10:07 Adam Israel > wrote: > > Thanks for calling this out, Simon! We should be shouting this from the > rooftops and celebrating in the streets. > > > Only if you also wave a big WARNING banner! > > I can definitely see value in pre-installing a bunch of things in your LXD > images as a way of speeding up the development/testing cycle, but doing so > might give you false confidence in your charm. It will become much easier > to forget to list a package that you need installing, or to ensure that > you have the correct access (PPA credentials, or proxy details etc.) and > having your charm gracefully handle when those are missing. > > Juju promises charms encoding operations that can work across multiple > cloud providers, bare metal and containers please keep that in mind :) > > > Indeed, and this is the reason why it wasn't called out. We probably > should document it for power-users/charmers, but in general I wouldn't go > encouraging its use. Optimising for LXD is great for repeat deploys, but it > wouldn't be great if that leads to less attention to quality on the rest of > the providers. > > Anyway, I'm glad it's helping make charmers' lives easier! > > > We should call this out loudly because it helps people making charms. > > Those people are plenty smart enough to debug a failure if they forget a > dependency which was preinstalled in their dev images. > I was thinking about deployment times more than anything else. If you don't feel your user's pain, you're less likely to make it go away. But anyway, that can be fixed with automation as well (CI, as you say below). > Don't HIDE something that helps developers for fear of those developers > making mistakes, TEACH them to put CI or other out-of-band tests in place > anyway that will catch that every time. > FWIW, it wasn't intentionally hidden to start with, it was just missed. I made the changes primarily to support an external user who wanted to demo CentOS charms on LXD; the change also enabled custom images in general, and also slightly improved container startup time. Three birds, one stone; only one bird-hitting was reported ;) Cheers, Andrew -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Juju 2.1.0, and Conjure-up, are here!
On 24/02/17 11:30, Andrew Wilkins wrote: > On Fri, Feb 24, 2017 at 6:15 PM Adam Collard > mailto:adam.coll...@canonical.com>> wrote: > > On Fri, 24 Feb 2017 at 10:07 Adam Israel > mailto:adam.isr...@canonical.com>> wrote: > > Thanks for calling this out, Simon! We should be shouting this > from the rooftops and celebrating in the streets. > > > Only if you also wave a big WARNING banner! > > I can definitely see value in pre-installing a bunch of things in > your LXD images as a way of speeding up the development/testing > cycle, but doing so might give you false confidence in your charm. > It will become much easier to forget to list a package that you > need installing, or to ensure that you have the correct access > (PPA credentials, or proxy details etc.) and having your charm > gracefully handle when those are missing. > > Juju promises charms encoding operations that can work across > multiple cloud providers, bare metal and containers please keep > that in mind :) > > > Indeed, and this is the reason why it wasn't called out. We probably > should document it for power-users/charmers, but in general I wouldn't > go encouraging its use. Optimising for LXD is great for repeat > deploys, but it wouldn't be great if that leads to less attention to > quality on the rest of the providers. > > Anyway, I'm glad it's helping make charmers' lives easier! We should call this out loudly because it helps people making charms. Those people are plenty smart enough to debug a failure if they forget a dependency which was preinstalled in their dev images. Don't HIDE something that helps developers for fear of those developers making mistakes, TEACH them to put CI or other out-of-band tests in place anyway that will catch that every time. Mark -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Juju 2.1.0, and Conjure-up, are here!
On Fri, Feb 24, 2017 at 6:15 PM Adam Collard wrote: > On Fri, 24 Feb 2017 at 10:07 Adam Israel > wrote: > > Thanks for calling this out, Simon! We should be shouting this from the > rooftops and celebrating in the streets. > > > Only if you also wave a big WARNING banner! > > I can definitely see value in pre-installing a bunch of things in your LXD > images as a way of speeding up the development/testing cycle, but doing so > might give you false confidence in your charm. It will become much easier > to forget to list a package that you need installing, or to ensure that > you have the correct access (PPA credentials, or proxy details etc.) and > having your charm gracefully handle when those are missing. > > Juju promises charms encoding operations that can work across multiple > cloud providers, bare metal and containers please keep that in mind :) > Indeed, and this is the reason why it wasn't called out. We probably should document it for power-users/charmers, but in general I wouldn't go encouraging its use. Optimising for LXD is great for repeat deploys, but it wouldn't be great if that leads to less attention to quality on the rest of the providers. Anyway, I'm glad it's helping make charmers' lives easier! Cheers, Andrew > On Fri, Feb 24, 2017 at 8:42 AM Stuart Bishop > wrote: > > On 23 February 2017 at 23:20, Simon Davy wrote: > > > One thing that seems to have landed in 2.1, which is worth noting IMO, is > > the local juju lxd image aliases. > > > > tl;dr: juju 2.1 now looks for the lxd image alias juju/$series/$arch in > the > > local lxd server, and uses that if it finds it. > > > > This is amazing. I can now build a local nightly image[1] that > pre-installs > > and pre-downloads a whole set of packages[2], and my local lxd units > don't > > have to install them when they spin up. Between layer-basic and Canonical > > IS' basenode, for us that's about 111 packages that I don't need to > install > > on every machine in my 10 node bundle. Took my install hook times from > 5min+ > > each to <1min, and probably halfs my initial deploy time, on average. > > Ooh, thanks for highlighting this! I've needed this feature for a long > time for exactly the same reasons. > > > > [2] my current nightly cron: > > https://gist.github.com/bloodearnest/3474741411c4fdd6c2bb64d08dc75040 > > /me starts stealing > > -- > Stuart Bishop > > -- > Juju-dev mailing list > juju-...@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > > -- > Adam Israel, Software Engineer > Canonical // Cloud DevOps // Juju // Ecosystem > -- > Juju-dev mailing list > juju-...@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > > -- > Juju-dev mailing list > juju-...@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Juju 2.1.0, and Conjure-up, are here!
On Fri, 24 Feb 2017 at 10:07 Adam Israel wrote: > Thanks for calling this out, Simon! We should be shouting this from the > rooftops and celebrating in the streets. > Only if you also wave a big WARNING banner! I can definitely see value in pre-installing a bunch of things in your LXD images as a way of speeding up the development/testing cycle, but doing so might give you false confidence in your charm. It will become much easier to forget to list a package that you need installing, or to ensure that you have the correct access (PPA credentials, or proxy details etc.) and having your charm gracefully handle when those are missing. Juju promises charms encoding operations that can work across multiple cloud providers, bare metal and containers please keep that in mind :) > On Fri, Feb 24, 2017 at 8:42 AM Stuart Bishop > wrote: > > On 23 February 2017 at 23:20, Simon Davy wrote: > > > One thing that seems to have landed in 2.1, which is worth noting IMO, is > > the local juju lxd image aliases. > > > > tl;dr: juju 2.1 now looks for the lxd image alias juju/$series/$arch in > the > > local lxd server, and uses that if it finds it. > > > > This is amazing. I can now build a local nightly image[1] that > pre-installs > > and pre-downloads a whole set of packages[2], and my local lxd units > don't > > have to install them when they spin up. Between layer-basic and Canonical > > IS' basenode, for us that's about 111 packages that I don't need to > install > > on every machine in my 10 node bundle. Took my install hook times from > 5min+ > > each to <1min, and probably halfs my initial deploy time, on average. > > Ooh, thanks for highlighting this! I've needed this feature for a long > time for exactly the same reasons. > > > > [2] my current nightly cron: > > https://gist.github.com/bloodearnest/3474741411c4fdd6c2bb64d08dc75040 > > /me starts stealing > > -- > Stuart Bishop > > -- > Juju-dev mailing list > juju-...@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > > -- > Adam Israel, Software Engineer > Canonical // Cloud DevOps // Juju // Ecosystem > -- > Juju-dev mailing list > juju-...@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Juju 2.1.0, and Conjure-up, are here!
Thanks for calling this out, Simon! We should be shouting this from the rooftops and celebrating in the streets. On Fri, Feb 24, 2017 at 8:42 AM Stuart Bishop wrote: > On 23 February 2017 at 23:20, Simon Davy wrote: > > > One thing that seems to have landed in 2.1, which is worth noting IMO, is > > the local juju lxd image aliases. > > > > tl;dr: juju 2.1 now looks for the lxd image alias juju/$series/$arch in > the > > local lxd server, and uses that if it finds it. > > > > This is amazing. I can now build a local nightly image[1] that > pre-installs > > and pre-downloads a whole set of packages[2], and my local lxd units > don't > > have to install them when they spin up. Between layer-basic and Canonical > > IS' basenode, for us that's about 111 packages that I don't need to > install > > on every machine in my 10 node bundle. Took my install hook times from > 5min+ > > each to <1min, and probably halfs my initial deploy time, on average. > > Ooh, thanks for highlighting this! I've needed this feature for a long > time for exactly the same reasons. > > > > [2] my current nightly cron: > > https://gist.github.com/bloodearnest/3474741411c4fdd6c2bb64d08dc75040 > > /me starts stealing > > -- > Stuart Bishop > > -- > Juju-dev mailing list > juju-...@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > -- Adam Israel, Software Engineer Canonical // Cloud DevOps // Juju // Ecosystem -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Juju 2.1.0, and Conjure-up, are here!
On 23 February 2017 at 23:20, Simon Davy wrote: > One thing that seems to have landed in 2.1, which is worth noting IMO, is > the local juju lxd image aliases. > > tl;dr: juju 2.1 now looks for the lxd image alias juju/$series/$arch in the > local lxd server, and uses that if it finds it. > > This is amazing. I can now build a local nightly image[1] that pre-installs > and pre-downloads a whole set of packages[2], and my local lxd units don't > have to install them when they spin up. Between layer-basic and Canonical > IS' basenode, for us that's about 111 packages that I don't need to install > on every machine in my 10 node bundle. Took my install hook times from 5min+ > each to <1min, and probably halfs my initial deploy time, on average. Ooh, thanks for highlighting this! I've needed this feature for a long time for exactly the same reasons. > [2] my current nightly cron: > https://gist.github.com/bloodearnest/3474741411c4fdd6c2bb64d08dc75040 /me starts stealing -- Stuart Bishop -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Juju 2.1.0, and Conjure-up, are here!
On Thu, Feb 23, 2017 at 2:48 AM, Curtis Hovey-Canonical < cur...@canonical.com> wrote: > A new release of Juju, 2.1.0, and Conjure-up, are here! > > > ## What's new in 2.1.0 > > - Model migration > - Interactive `add-cloud` > - Networking changes > - Conjure-up > - LXD credential changes > - Changes to the GUI > - Instrumentation of Juju via Prometheus endpoints > - Improved OpenStack keystone v3 authentication > - New cloud-regions supported > - Additional improvements > One thing that seems to have landed in 2.1, which is worth noting IMO, is the local juju lxd image aliases. tl;dr: juju 2.1 now looks for the lxd image alias juju/$series/$arch in the local lxd server, and uses that if it finds it. This is amazing. I can now build a local nightly image[1] that pre-installs and pre-downloads a whole set of packages[2], and my local lxd units don't have to install them when they spin up. Between layer-basic and Canonical IS' basenode, for us that's about 111 packages that I don't need to install on every machine in my 10 node bundle. Took my install hook times from 5min+ each to <1min, and probably halfs my initial deploy time, on average. Oddly, I only found out about this indirectly via Andrew Wilkins' blog post[3] on CentOs images, which suggested this was possible. I had to actually look at the code[4] to figure it out. For me, this is the single biggest feature in 2.1, and will save me 30mins+ a day, per person who works with juju on my team. But more than raw time, it reduces iteration interval, and the number of context switches I'm doing as a I wait for things to deploy. ++win. I couldn't find any mention of this is the 2.1 lxd provider docs, but I think it'd be worth calling out, as it's a big speed up in local development. My thanks to the folks who did this work. Very much appreciated. [1] well, you could do this with juju 1.x, but it was messier. [2] my current nightly cron: https://gist.github.com/bloodearnest/3474741411c4fdd6c2bb64d08dc75040 [3] https://awilkins.id.au/post/juju-2.1-lxd-centos/ [4] https://github.com/juju/juju/blob/staging/tools/lxdclient/client_image.go#L117 -- Simon -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Juju 2.1.0, and Conjure-up, are here!
A new release of Juju, 2.1.0, and Conjure-up, are here! ## What's new in 2.1.0 - Model migration - Interactive `add-cloud` - Networking changes - Conjure-up - LXD credential changes - Changes to the GUI - Instrumentation of Juju via Prometheus endpoints - Improved OpenStack keystone v3 authentication - New cloud-regions supported - Additional improvements ### Model migration Model migration allows you to easily move a live model from one controller to another. The same configuration of machines, units and their relationships will be replicated on a secondary controller, while your applications continue uninterrupted. Migration is a useful alternative to upgrading a controller in place, and for moving models off a busy controller. When upgrading a controller, you can bootstrap a new controller running a newer version of Juju and then migrate each model across one at a time. This is safer than upgrading a controller while it is running many applications. Currently there are some restrictions: - The source and destination controllers need to be in the same cloud environment. - The destination controller needs to be running on the same cloud substrate as the source controller. - Destination controllers on different regions or VPCs need direct connectivity to the source controller. - The version of Juju running on the destination controller needs to be the same or newer than the version on the source controller. - The controller model cannot be migrated. To migrate a model on the current controller to a model on another controller, you simply name the model as the first argument followed by the target controller (a model with the same name cannot already exist on the target controller): juju migrate This will initiate the migration with output similar to the following: Migration started with ID "d1924666-1b00-4805-89b5-5ed5a6744426:0" You can monitor the migration progress from the output of the juju status command run against the source model. The juju show-model command also shows migration progress. If the migration fails at any point, the model will be reactivated on the original controller in the same state it was in before the migration process was started. The duration of a migration will depend on the complexity of the model, the resources it uses and the capabilities of the hosted environment. Most migrations will take minutes, and even large deployments are unlikely to take hours. When complete, the model will no longer exist on the source controller, and the model, all its applications, machines and units will be running from the target controller. Use `juju switch` to select the migrated model in the destination controller: juju switch : juju status There is more information on model migration in the Juju documentation online at <https://jujucharms.com/docs/2.1/models-migrate> ### Interactive `add-cloud` With previous versions of Juju, the `add-cloud` command would need to be fed a specifically formatted YAML file if your cloud of choice wasn't directly supported by Juju. You can still do this, but from version 2.1, you can also step through a simple interactive process that will create a working configuration for you. Typing `juju add-cloud` starts the process and produces the following output: Cloud Types maas manual openstack vsphere Select cloud type: Simply answer the three or four questions for your new cloud and Juju will do the rest. The next step is to add credentials for this new cloud, which can be done with the similarly interactive command: juju add-credentials Again, follow the prompts to add the requested information. A more detailed walkthrough of the process is published in the online Juju documentation here: <https://jujucharms.com/docs/2.1/clouds#specifying-additional-clouds> ### Networking changes A number of changes have been introduced to make the use of networks, particularly networking of containers, more efficient and consistent in Juju. Juju models networks using the primitive of "spaces". A space is made up of one or more routable subnets with common ingress and egress rules. The operator can model this topology in such a way that applications have the required network connectivity without generating network IP maps of overwhelming complexity that are not portable. The default behaviour in Juju 2.0 was that all machines might host containers and so all interfaces were bridged by default, even if a container was never placed on the machine. If a container was placed on the machine all of the network devices of that machine were made available to each container. This led to issues where the operator wanted a much cleaner model where the containers only had access to networks that the model required. Starting from Juju 2.1 this will no longer be true. Juju will only create the bridges which are necessary for a containe
