Re: Juju 2.1.0, and Conjure-up, are here!
On Fri, Feb 24, 2017 at 11:14 AM, Andrew Wilkins < andrew.wilk...@canonical.com> wrote: > On Fri, Feb 24, 2017 at 6:51 PM Mark Shuttleworth wrote: > >> On 24/02/17 11:30, Andrew Wilkins wrote: >> >> On Fri, Feb 24, 2017 at 6:15 PM Adam Collard >> wrote: >> >> On Fri, 24 Feb 2017 at 10:07 Adam Israel >> wrote: >> >> Thanks for calling this out, Simon! We should be shouting this from the >> rooftops and celebrating in the streets. >> >> >> Only if you also wave a big WARNING banner! >> >> I can definitely see value in pre-installing a bunch of things in your >> LXD images as a way of speeding up the development/testing cycle, but doing >> so might give you false confidence in your charm. It will become much >> easier to forget to list a package that you need installing, or to ensure >> that you have the correct access (PPA credentials, or proxy details etc.) >> and having your charm gracefully handle when those are missing. >> >> Juju promises charms encoding operations that can work across multiple >> cloud providers, bare metal and containers please keep that in mind :) >> >> >> Indeed, and this is the reason why it wasn't called out. We probably >> should document it for power-users/charmers, but in general I wouldn't go >> encouraging its use. Optimising for LXD is great for repeat deploys, but it >> wouldn't be great if that leads to less attention to quality on the rest of >> the providers. >> >> Anyway, I'm glad it's helping make charmers' lives easier! >> >> >> We should call this out loudly because it helps people making charms. >> >> Those people are plenty smart enough to debug a failure if they forget a >> dependency which was preinstalled in their dev images. >> > > I was thinking about deployment times more than anything else. If you > don't feel your user's pain, you're less likely to make it go away. But > anyway, that can be fixed with automation as well (CI, as you say below). > I agree there is a risk here. In my specific case, I judge the benefits to outweigh the costs, by quite some margin. But that judgment is specific to my use case, where layer-basic and IS' basenode add significant package churn on every node[1] (increasing the benefit), and we have a full mojo-based CI pipeline for bundle changes (lowering the cost). On a different tack all together, I think that reducing iteration time for charm development is a *massive* win for users. Faster iterations mean faster feature development and bug fixes, and more comprehensive testing (as it costs less). I would estimate that iteration improvement would outweigh the increased risk from a missing pre-installed package, but YMMV. [1] ok, so not every charm we deploy is layer based, but they are heading that way... Don't HIDE something that helps developers for fear of those developers >> making mistakes, TEACH them to put CI or other out-of-band tests in place >> anyway that will catch that every time. >> > > FWIW, it wasn't intentionally hidden to start with, it was just missed. I > made the changes primarily to support an external user who wanted to demo > CentOS charms on LXD; the change also enabled custom images in general, and > also slightly improved container startup time. Three birds, one stone; only > one bird-hitting was reported ;) > This is hugely appreciated. I reckon 95% of my deployments in the average week are to lxd, so improvements to the lxd provider affect my velocity considerably. Thanks -- Simon -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Juju 2.1.0, and Conjure-up, are here!
On Fri, Feb 24, 2017 at 6:51 PM Mark Shuttleworth wrote: > On 24/02/17 11:30, Andrew Wilkins wrote: > > On Fri, Feb 24, 2017 at 6:15 PM Adam Collard > wrote: > > On Fri, 24 Feb 2017 at 10:07 Adam Israel > wrote: > > Thanks for calling this out, Simon! We should be shouting this from the > rooftops and celebrating in the streets. > > > Only if you also wave a big WARNING banner! > > I can definitely see value in pre-installing a bunch of things in your LXD > images as a way of speeding up the development/testing cycle, but doing so > might give you false confidence in your charm. It will become much easier > to forget to list a package that you need installing, or to ensure that > you have the correct access (PPA credentials, or proxy details etc.) and > having your charm gracefully handle when those are missing. > > Juju promises charms encoding operations that can work across multiple > cloud providers, bare metal and containers please keep that in mind :) > > > Indeed, and this is the reason why it wasn't called out. We probably > should document it for power-users/charmers, but in general I wouldn't go > encouraging its use. Optimising for LXD is great for repeat deploys, but it > wouldn't be great if that leads to less attention to quality on the rest of > the providers. > > Anyway, I'm glad it's helping make charmers' lives easier! > > > We should call this out loudly because it helps people making charms. > > Those people are plenty smart enough to debug a failure if they forget a > dependency which was preinstalled in their dev images. > I was thinking about deployment times more than anything else. If you don't feel your user's pain, you're less likely to make it go away. But anyway, that can be fixed with automation as well (CI, as you say below). > Don't HIDE something that helps developers for fear of those developers > making mistakes, TEACH them to put CI or other out-of-band tests in place > anyway that will catch that every time. > FWIW, it wasn't intentionally hidden to start with, it was just missed. I made the changes primarily to support an external user who wanted to demo CentOS charms on LXD; the change also enabled custom images in general, and also slightly improved container startup time. Three birds, one stone; only one bird-hitting was reported ;) Cheers, Andrew -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Juju 2.1.0, and Conjure-up, are here!
On 24/02/17 11:30, Andrew Wilkins wrote: > On Fri, Feb 24, 2017 at 6:15 PM Adam Collard > mailto:adam.coll...@canonical.com>> wrote: > > On Fri, 24 Feb 2017 at 10:07 Adam Israel > mailto:adam.isr...@canonical.com>> wrote: > > Thanks for calling this out, Simon! We should be shouting this > from the rooftops and celebrating in the streets. > > > Only if you also wave a big WARNING banner! > > I can definitely see value in pre-installing a bunch of things in > your LXD images as a way of speeding up the development/testing > cycle, but doing so might give you false confidence in your charm. > It will become much easier to forget to list a package that you > need installing, or to ensure that you have the correct access > (PPA credentials, or proxy details etc.) and having your charm > gracefully handle when those are missing. > > Juju promises charms encoding operations that can work across > multiple cloud providers, bare metal and containers please keep > that in mind :) > > > Indeed, and this is the reason why it wasn't called out. We probably > should document it for power-users/charmers, but in general I wouldn't > go encouraging its use. Optimising for LXD is great for repeat > deploys, but it wouldn't be great if that leads to less attention to > quality on the rest of the providers. > > Anyway, I'm glad it's helping make charmers' lives easier! We should call this out loudly because it helps people making charms. Those people are plenty smart enough to debug a failure if they forget a dependency which was preinstalled in their dev images. Don't HIDE something that helps developers for fear of those developers making mistakes, TEACH them to put CI or other out-of-band tests in place anyway that will catch that every time. Mark -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Juju 2.1.0, and Conjure-up, are here!
On Fri, Feb 24, 2017 at 6:15 PM Adam Collard wrote: > On Fri, 24 Feb 2017 at 10:07 Adam Israel > wrote: > > Thanks for calling this out, Simon! We should be shouting this from the > rooftops and celebrating in the streets. > > > Only if you also wave a big WARNING banner! > > I can definitely see value in pre-installing a bunch of things in your LXD > images as a way of speeding up the development/testing cycle, but doing so > might give you false confidence in your charm. It will become much easier > to forget to list a package that you need installing, or to ensure that > you have the correct access (PPA credentials, or proxy details etc.) and > having your charm gracefully handle when those are missing. > > Juju promises charms encoding operations that can work across multiple > cloud providers, bare metal and containers please keep that in mind :) > Indeed, and this is the reason why it wasn't called out. We probably should document it for power-users/charmers, but in general I wouldn't go encouraging its use. Optimising for LXD is great for repeat deploys, but it wouldn't be great if that leads to less attention to quality on the rest of the providers. Anyway, I'm glad it's helping make charmers' lives easier! Cheers, Andrew > On Fri, Feb 24, 2017 at 8:42 AM Stuart Bishop > wrote: > > On 23 February 2017 at 23:20, Simon Davy wrote: > > > One thing that seems to have landed in 2.1, which is worth noting IMO, is > > the local juju lxd image aliases. > > > > tl;dr: juju 2.1 now looks for the lxd image alias juju/$series/$arch in > the > > local lxd server, and uses that if it finds it. > > > > This is amazing. I can now build a local nightly image[1] that > pre-installs > > and pre-downloads a whole set of packages[2], and my local lxd units > don't > > have to install them when they spin up. Between layer-basic and Canonical > > IS' basenode, for us that's about 111 packages that I don't need to > install > > on every machine in my 10 node bundle. Took my install hook times from > 5min+ > > each to <1min, and probably halfs my initial deploy time, on average. > > Ooh, thanks for highlighting this! I've needed this feature for a long > time for exactly the same reasons. > > > > [2] my current nightly cron: > > https://gist.github.com/bloodearnest/3474741411c4fdd6c2bb64d08dc75040 > > /me starts stealing > > -- > Stuart Bishop > > -- > Juju-dev mailing list > juju-...@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > > -- > Adam Israel, Software Engineer > Canonical // Cloud DevOps // Juju // Ecosystem > -- > Juju-dev mailing list > juju-...@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > > -- > Juju-dev mailing list > juju-...@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Juju 2.1.0, and Conjure-up, are here!
On Fri, 24 Feb 2017 at 10:07 Adam Israel wrote: > Thanks for calling this out, Simon! We should be shouting this from the > rooftops and celebrating in the streets. > Only if you also wave a big WARNING banner! I can definitely see value in pre-installing a bunch of things in your LXD images as a way of speeding up the development/testing cycle, but doing so might give you false confidence in your charm. It will become much easier to forget to list a package that you need installing, or to ensure that you have the correct access (PPA credentials, or proxy details etc.) and having your charm gracefully handle when those are missing. Juju promises charms encoding operations that can work across multiple cloud providers, bare metal and containers please keep that in mind :) > On Fri, Feb 24, 2017 at 8:42 AM Stuart Bishop > wrote: > > On 23 February 2017 at 23:20, Simon Davy wrote: > > > One thing that seems to have landed in 2.1, which is worth noting IMO, is > > the local juju lxd image aliases. > > > > tl;dr: juju 2.1 now looks for the lxd image alias juju/$series/$arch in > the > > local lxd server, and uses that if it finds it. > > > > This is amazing. I can now build a local nightly image[1] that > pre-installs > > and pre-downloads a whole set of packages[2], and my local lxd units > don't > > have to install them when they spin up. Between layer-basic and Canonical > > IS' basenode, for us that's about 111 packages that I don't need to > install > > on every machine in my 10 node bundle. Took my install hook times from > 5min+ > > each to <1min, and probably halfs my initial deploy time, on average. > > Ooh, thanks for highlighting this! I've needed this feature for a long > time for exactly the same reasons. > > > > [2] my current nightly cron: > > https://gist.github.com/bloodearnest/3474741411c4fdd6c2bb64d08dc75040 > > /me starts stealing > > -- > Stuart Bishop > > -- > Juju-dev mailing list > juju-...@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > > -- > Adam Israel, Software Engineer > Canonical // Cloud DevOps // Juju // Ecosystem > -- > Juju-dev mailing list > juju-...@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Juju 2.1.0, and Conjure-up, are here!
Thanks for calling this out, Simon! We should be shouting this from the rooftops and celebrating in the streets. On Fri, Feb 24, 2017 at 8:42 AM Stuart Bishop wrote: > On 23 February 2017 at 23:20, Simon Davy wrote: > > > One thing that seems to have landed in 2.1, which is worth noting IMO, is > > the local juju lxd image aliases. > > > > tl;dr: juju 2.1 now looks for the lxd image alias juju/$series/$arch in > the > > local lxd server, and uses that if it finds it. > > > > This is amazing. I can now build a local nightly image[1] that > pre-installs > > and pre-downloads a whole set of packages[2], and my local lxd units > don't > > have to install them when they spin up. Between layer-basic and Canonical > > IS' basenode, for us that's about 111 packages that I don't need to > install > > on every machine in my 10 node bundle. Took my install hook times from > 5min+ > > each to <1min, and probably halfs my initial deploy time, on average. > > Ooh, thanks for highlighting this! I've needed this feature for a long > time for exactly the same reasons. > > > > [2] my current nightly cron: > > https://gist.github.com/bloodearnest/3474741411c4fdd6c2bb64d08dc75040 > > /me starts stealing > > -- > Stuart Bishop > > -- > Juju-dev mailing list > juju-...@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > -- Adam Israel, Software Engineer Canonical // Cloud DevOps // Juju // Ecosystem -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Juju 2.1.0, and Conjure-up, are here!
On 23 February 2017 at 23:20, Simon Davy wrote: > One thing that seems to have landed in 2.1, which is worth noting IMO, is > the local juju lxd image aliases. > > tl;dr: juju 2.1 now looks for the lxd image alias juju/$series/$arch in the > local lxd server, and uses that if it finds it. > > This is amazing. I can now build a local nightly image[1] that pre-installs > and pre-downloads a whole set of packages[2], and my local lxd units don't > have to install them when they spin up. Between layer-basic and Canonical > IS' basenode, for us that's about 111 packages that I don't need to install > on every machine in my 10 node bundle. Took my install hook times from 5min+ > each to <1min, and probably halfs my initial deploy time, on average. Ooh, thanks for highlighting this! I've needed this feature for a long time for exactly the same reasons. > [2] my current nightly cron: > https://gist.github.com/bloodearnest/3474741411c4fdd6c2bb64d08dc75040 /me starts stealing -- Stuart Bishop -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Juju 2.1.0, and Conjure-up, are here!
On Thu, Feb 23, 2017 at 2:48 AM, Curtis Hovey-Canonical < cur...@canonical.com> wrote: > A new release of Juju, 2.1.0, and Conjure-up, are here! > > > ## What's new in 2.1.0 > > - Model migration > - Interactive `add-cloud` > - Networking changes > - Conjure-up > - LXD credential changes > - Changes to the GUI > - Instrumentation of Juju via Prometheus endpoints > - Improved OpenStack keystone v3 authentication > - New cloud-regions supported > - Additional improvements > One thing that seems to have landed in 2.1, which is worth noting IMO, is the local juju lxd image aliases. tl;dr: juju 2.1 now looks for the lxd image alias juju/$series/$arch in the local lxd server, and uses that if it finds it. This is amazing. I can now build a local nightly image[1] that pre-installs and pre-downloads a whole set of packages[2], and my local lxd units don't have to install them when they spin up. Between layer-basic and Canonical IS' basenode, for us that's about 111 packages that I don't need to install on every machine in my 10 node bundle. Took my install hook times from 5min+ each to <1min, and probably halfs my initial deploy time, on average. Oddly, I only found out about this indirectly via Andrew Wilkins' blog post[3] on CentOs images, which suggested this was possible. I had to actually look at the code[4] to figure it out. For me, this is the single biggest feature in 2.1, and will save me 30mins+ a day, per person who works with juju on my team. But more than raw time, it reduces iteration interval, and the number of context switches I'm doing as a I wait for things to deploy. ++win. I couldn't find any mention of this is the 2.1 lxd provider docs, but I think it'd be worth calling out, as it's a big speed up in local development. My thanks to the folks who did this work. Very much appreciated. [1] well, you could do this with juju 1.x, but it was messier. [2] my current nightly cron: https://gist.github.com/bloodearnest/3474741411c4fdd6c2bb64d08dc75040 [3] https://awilkins.id.au/post/juju-2.1-lxd-centos/ [4] https://github.com/juju/juju/blob/staging/tools/lxdclient/client_image.go#L117 -- Simon -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
