Re: A (Very) Minimal Charm

[Katherine Cox-Buday]

Mark Shuttleworth  writes:

> We are already aligning things like terms and authentication so that
> charms and snaps can be delivered together, though, so perhaps all
> thats required is the ability to give the charm some say in update
> control of snaps.

Interesting, thanks, Mark.

-- 
Katherine

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Free Ekanayaka]

>
> As a developer, with an xfs-backed
>

s/xfs/zfs/
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Free Ekanayaka]

On 15 December 2016 at 07:59, John Meinel  wrote:

> Right, the issue for test/development iterations is that "machine
> requested to booted in cloud" for LXD is a lot closer to 10s. Especially if
> you set "enable-os-refresh-update: false" and "enable-os-upgrade: false",
> which are also likely to be set in a testing environment.
>

+1

To re-iterate what previously stated: the point is not production
deployments, but rather development and testing cycles. The experience will
be compared to things like docker & friends where the feedback look happens
in seconds, not minutes or dozens of seconds.

As a developer, with an xfs-backed local juju lxd provider and something
like apt-cacher-ng or deb-squid-proxy installed I expect and want "juju
deploy ubuntu" (or any other minimal reactive-based charm) to take seconds.
Anything less than that gets in the way of doing proper TDD.

Free
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Mark Shuttleworth]

On 16/12/16 07:33, Katherine Cox-Buday wrote:
> Tim Penhey  writes:
>
>> Make sure you also run on LXD with a decent delay to the APT archive.
> Open question: is there any reason we shouldn't expect charm authors to take 
> a hard-right towards charms with snaps embedded as resources? I know one of 
> our long-standing conceptual problems is consistency across units which snaps 
> solves nicely.

We'll need to figure out the inherent tension between the way resources
are updated and the way snaps are updated. Resources are very similar to
snaps, but snaps are growing digital signatures that need to be
delivered together with the snap to prove integrity, and we don't want
to duplicate all that work. We are already aligning things like terms
and authentication so that charms and snaps can be delivered together,
though, so perhaps all thats required is the ability to give the charm
some say in update control of snaps.

Mark



-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Stuart Bishop]

On 16 December 2016 at 22:33, Katherine Cox-Buday <
katherine.cox-bu...@canonical.com> wrote:

> Tim Penhey  writes:
>
> > Make sure you also run on LXD with a decent delay to the APT archive.
>
> Open question: is there any reason we shouldn't expect charm authors to
> take a hard-right towards charms with snaps embedded as resources? I know
> one of our long-standing conceptual problems is consistency across units
> which snaps solves nicely.
>

https://github.com/stub42/layer-snap is how I'm expecting things to go.
There is already one charm in the ~charmers review queue using it and I'm
aware of several more in various stages of development.

More work is needed though. In particular, Juju storage is inaccessible to
snaps, because there is no way to reach it from inside the containment.

(But none of this is a reason to not optimize Juju unit provisioning times,
since we will still need an environment setup capable of running the charms
so they can install the snaps for some time yet).

-- 
Stuart Bishop 
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[David Britton]

On Fri, Dec 16, 2016 at 09:33:18AM -0600, Katherine Cox-Buday wrote:
> Tim Penhey  writes:
> 
> > Make sure you also run on LXD with a decent delay to the APT
> > archive.
> 
> Open question: is there any reason we shouldn't expect charm authors
> to take a hard-right towards charms with snaps embedded as resources?
> I know one of our long-standing conceptual problems is consistency
> across units which snaps solves nicely.

For new projects we are working this way.  We have not used resources
yet, but instead are using "fat" charms and sideloading the snap.

But, resources are the next logical progression.

-- 
David Britton 

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Katherine Cox-Buday]

Tim Penhey  writes:

> Make sure you also run on LXD with a decent delay to the APT archive.

Open question: is there any reason we shouldn't expect charm authors to take a 
hard-right towards charms with snaps embedded as resources? I know one of our 
long-standing conceptual problems is consistency across units which snaps 
solves nicely.

-- 
Katherine

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[John Meinel]

Right, the issue for test/development iterations is that "machine requested
to booted in cloud" for LXD is a lot closer to 10s. Especially if you set
"enable-os-refresh-update: false" and "enable-os-upgrade: false", which are
also likely to be set in a testing environment.

John
=:->

On Thu, Dec 15, 2016 at 7:47 AM, Tim Penhey 
wrote:

> Make sure you also run on LXD with a decent delay to the APT archive.
>
> That is what makes my local testing slow.
>
> Tim
>
> On 15/12/16 13:34, Marco Ceppi wrote:
>
>> ...


> I did this a few more times on Amazon, and the results were almost
>> identical. We have 80 seconds from machine requested to booted in cloud.
>> Less than a second for agent to initialize and 32 seconds to go from
>> install hook running to the workload being ready and active. While I'm
>> sure we can slim that down 10-15 seconds by not installing
>> build-essentials the largest time suck is still the cloud bringing up
>> the instance.
>>
>> I plan on doing this across all the clouds I have access to, and track
>> in a spreadsheet. I'll share that sheet out in a bit.
>>
>> Thanks,
>> Marco Ceppi
>>
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Tim Penhey]

Make sure you also run on LXD with a decent delay to the APT archive.

That is what makes my local testing slow.

Tim

On 15/12/16 13:34, Marco Ceppi wrote:

So, I wanted to circle back around to this thread. I think a lot of good
feedback has come from this, and we're looking into making the reactive
framework leaner and better for charm authors. However, I ran a few
deploy tests and have the following results:

15 Dec 2016 00:18:53Zworkload waitingwaiting for machine
15 Dec 2016 00:18:53Zjuju-unitallocating
15 Dec 2016 00:20:13Zworkload waitinginstalling agent
15 Dec 2016 00:20:13Zworkload waitingagent initializing
15 Dec 2016 00:20:14Zworkload maintenanceinstalling charm software
15 Dec 2016 00:20:14Zjuju-unitexecuting  running install hook
15 Dec 2016 00:20:46Zworkload active ready
15 Dec 2016 00:20:46Zjuju-unitexecuting  running leader-elected hook
15 Dec 2016 00:20:47Zjuju-unitexecuting  running start hook

I did this a few more times on Amazon, and the results were almost
identical. We have 80 seconds from machine requested to booted in cloud.
Less than a second for agent to initialize and 32 seconds to go from
install hook running to the workload being ready and active. While I'm
sure we can slim that down 10-15 seconds by not installing
build-essentials the largest time suck is still the cloud bringing up
the instance.

I plan on doing this across all the clouds I have access to, and track
in a spreadsheet. I'll share that sheet out in a bit.

Thanks,
Marco Ceppi

On Thu, Dec 1, 2016 at 5:00 AM Adam Collard mailto:adam.coll...@canonical.com>> wrote:

On Thu, 1 Dec 2016 at 04:02 Nate Finch mailto:nate.fi...@canonical.com>> wrote:

On IRC, someone was lamenting the fact that the Ubuntu charm
takes longer to deploy now, because it has been updated to
exercise more of Juju's features.  My response was - just make a
minimal charm, it's easy.  And then of course, I had to figure
out how minimal you can get.  Here it is:

It's just a directory with a metadata.yaml in it with these
contents:

name: min
summary: nope
description: nope
series:
  - xenial

(obviously you can set the series to whatever you want)
No other files or directories are needed.


This is neat, but doesn't detract from the bloat in the ubuntu charm.

IMHO the bloat in the ubuntu charm isn't from support for Juju
features, but the switch to reactive plus conflicts in layer-base
wanting to a) support lots of toolchains to allow layers above it to
be slimmer and b) be a suitable base for "just deploy me" ubuntu.
--
Juju-dev mailing list
Juju-dev@lists.ubuntu.com 
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju-dev





--
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Marco Ceppi]

So, I wanted to circle back around to this thread. I think a lot of good
feedback has come from this, and we're looking into making the reactive
framework leaner and better for charm authors. However, I ran a few deploy
tests and have the following results:

15 Dec 2016 00:18:53Z workload waiting waiting for machine
15 Dec 2016 00:18:53Z juju-unit allocating
15 Dec 2016 00:20:13Z workload waiting installing agent
15 Dec 2016 00:20:13Z workload waiting agent initializing
15 Dec 2016 00:20:14Z workload maintenance installing charm software
15 Dec 2016 00:20:14Z juju-unit executing   running install hook
15 Dec 2016 00:20:46Z workload active ready
15 Dec 2016 00:20:46Z juju-unit executing   running leader-elected hook
15 Dec 2016 00:20:47Z juju-unit executing   running start hook

I did this a few more times on Amazon, and the results were almost
identical. We have 80 seconds from machine requested to booted in cloud.
Less than a second for agent to initialize and 32 seconds to go from
install hook running to the workload being ready and active. While I'm sure
we can slim that down 10-15 seconds by not installing build-essentials the
largest time suck is still the cloud bringing up the instance.

I plan on doing this across all the clouds I have access to, and track in a
spreadsheet. I'll share that sheet out in a bit.

Thanks,
Marco Ceppi

On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
wrote:

> On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:
>
> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
> to deploy now, because it has been updated to exercise more of Juju's
> features.  My response was - just make a minimal charm, it's easy.  And
> then of course, I had to figure out how minimal you can get.  Here it is:
>
> It's just a directory with a metadata.yaml in it with these contents:
>
> name: min
> summary: nope
> description: nope
> series:
>   - xenial
>
> (obviously you can set the series to whatever you want)
> No other files or directories are needed.
>
>
> This is neat, but doesn't detract from the bloat in the ubuntu charm.
>
> IMHO the bloat in the ubuntu charm isn't from support for Juju features,
> but the switch to reactive plus conflicts in layer-base wanting to a)
> support lots of toolchains to allow layers above it to be slimmer and b) be
> a suitable base for "just deploy me" ubuntu.
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Mark Shuttleworth]

On 09/12/16 06:32, John Meinel wrote:
>
>  1. "charm push cs:~jameinel/ubuntu-lite" fails ultimately with a TLS
> timeout error:
> ERROR cannot post archive: Post
> https://api.jujucharms.com/charmstore/v5/~jameinel/ubuntu-lite/archive?..
> 
> .:
> net/http: TLS handshake timeout
> I thought the point of multi-series charms is that you didn't have
> to publish to an explicit series.
>

You should be validated that there are *some* relevant series, with one
as the default.

>  1. The result of a successful push tells you "unpublished". But the
> command to actually publish one is "charm release". Should we be
> using the verbiage of "unreleased"? Certainly there was no
> indication of what I should be doing to get the charm to a
> 'published' state from the CLI.
>

Yes, that makes sense, the feedback should use the 'release' term and
should tell you how to make that happen. It should also be possible to
have the push command also --release [channels]

>  1. ubuntu-lite does get from "pending" to "active" a fair bit faster
> than the full 'ubuntu' charm (about 1-2 minutes on my LXD provider.)
>

Sounds like we can make reactive a little leaner.

Mark
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Marco Ceppi]

I think the actual path is to streamline reactive framework, so that Ubuntu
is the simplest example of reactive. Anyone can, as Nate demonstrated,
create a minimal charm. The feedback here is great so far and we're going
to be iterating on the framework.

Marco

On Fri, Dec 9, 2016, 9:32 AM John Meinel  wrote:

> So I went ahead and did slightly more than the minimal charm:
>  cs:~jameinel/xenial/ubuntu-lite
>
> It is essentially just installing Ubuntu, and in the Start hook, I do
> "status-set active" and report the Ubuntu version in
> "application-version-set".
>
> However, some feedback about the experience:
>
>
>1. "charm push cs:~jameinel/ubuntu-lite" fails ultimately with a TLS
>timeout error:
>ERROR cannot post archive: Post
>https://api.jujucharms.com/charmstore/v5/~jameinel/ubuntu-lite/archive?...:
>net/http: TLS handshake timeout
>I thought the point of multi-series charms is that you didn't have to
>publish to an explicit series.
>2. The result of a successful push tells you "unpublished". But the
>command to actually publish one is "charm release". Should we be using the
>verbiage of "unreleased"? Certainly there was no indication of what I
>should be doing to get the charm to a 'published' state from the CLI.
>3. ubuntu-lite does get from "pending" to "active" a fair bit faster
>than the full 'ubuntu' charm (about 1-2 minutes on my LXD provider.)
>
> John
> =:->
>
> On Thu, Dec 1, 2016 at 5:01 AM, Nate Finch 
> wrote:
>
> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
> to deploy now, because it has been updated to exercise more of Juju's
> features.  My response was - just make a minimal charm, it's easy.  And
> then of course, I had to figure out how minimal you can get.  Here it is:
>
> It's just a directory with a metadata.yaml in it with these contents:
>
> name: min
> summary: nope
> description: nope
> series:
>   - xenial
>
> (obviously you can set the series to whatever you want)
> No other files or directories are needed.
>
> Figured this might be useful for others.
>
> -Nate
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[John Meinel]

So I went ahead and did slightly more than the minimal charm:
 cs:~jameinel/xenial/ubuntu-lite

It is essentially just installing Ubuntu, and in the Start hook, I do
"status-set active" and report the Ubuntu version in
"application-version-set".

However, some feedback about the experience:


   1. "charm push cs:~jameinel/ubuntu-lite" fails ultimately with a TLS
   timeout error:
   ERROR cannot post archive: Post
   https://api.jujucharms.com/charmstore/v5/~jameinel/ubuntu-lite/archive?...:
   net/http: TLS handshake timeout
   I thought the point of multi-series charms is that you didn't have to
   publish to an explicit series.
   2. The result of a successful push tells you "unpublished". But the
   command to actually publish one is "charm release". Should we be using the
   verbiage of "unreleased"? Certainly there was no indication of what I
   should be doing to get the charm to a 'published' state from the CLI.
   3. ubuntu-lite does get from "pending" to "active" a fair bit faster
   than the full 'ubuntu' charm (about 1-2 minutes on my LXD provider.)

John
=:->

On Thu, Dec 1, 2016 at 5:01 AM, Nate Finch  wrote:

> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
> to deploy now, because it has been updated to exercise more of Juju's
> features.  My response was - just make a minimal charm, it's easy.  And
> then of course, I had to figure out how minimal you can get.  Here it is:
>
> It's just a directory with a metadata.yaml in it with these contents:
>
> name: min
> summary: nope
> description: nope
> series:
>   - xenial
>
> (obviously you can set the series to whatever you want)
> No other files or directories are needed.
>
> Figured this might be useful for others.
>
> -Nate
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju-dev
>
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Free Ekanayaka]

On 2 December 2016 at 06:27, Stuart Bishop 
wrote:


>  We might be able to lower deployment times from minutes to seconds, since
> often this step is the main time sink.
>

Right. These are the time scales I meant too as well. If you want to have
quality software you need to be able to test it. If you want to test it,
you need to iterate fast. To iterate fast you want your thing to execute in
seconds, not minutes.

So when I mentioned that this will be compared to kubernetes/docker, I
mainly meant this aspect (all the rest of the goodness of juju wrt
k8s/docker notwithstanding). It's not really about production deployments,
it's about testing. And about demo'ing and playing around as well, which
are very important parts of the UX too.
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Stuart Bishop]

On 1 December 2016 at 19:53, Marco Ceppi  wrote:

> On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
> wrote:
>
>> On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:
>>
>> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
>> to deploy now, because it has been updated to exercise more of Juju's
>> features.  My response was - just make a minimal charm, it's easy.  And
>> then of course, I had to figure out how minimal you can get.  Here it is:
>>
>> It's just a directory with a metadata.yaml in it with these contents:
>>
>> name: min
>> summary: nope
>> description: nope
>> series:
>>   - xenial
>>
>> (obviously you can set the series to whatever you want)
>> No other files or directories are needed.
>>
>>
>> This is neat, but doesn't detract from the bloat in the ubuntu charm.
>>
>
> I'm happy to work though changes to the Ubuntu charm to decrease "bloat".
>
>
>> IMHO the bloat in the ubuntu charm isn't from support for Juju features,
>> but the switch to reactive plus conflicts in layer-base wanting to a)
>> support lots of toolchains to allow layers above it to be slimmer and b) be
>> a suitable base for "just deploy me" ubuntu.
>>
>
> But it is to support the reactive framework, where we utilize newer Juju
> features, like status and application-version to make the charm rich
> despite it's minimal goal set. Honestly, a handful of cached wheelhouses
> and some apt packages don't strike me as bloat, but I do want to make sure
> the Ubuntu charm works for those using it. So,
>
> What's the real problem with the Ubuntu charm today?
> How does it not achieve it's goal of providing a relatively blank Ubuntu
> machine? What are people using the Ubuntu charm for?
>
> Other than demos, hacks/workarounds, and testing I'm not clear on the
> purpose of an Ubuntu charm in a model serves.
>

The cs:ubuntu charm gets used on production to attach subordinates too. For
example, we install cs:ubuntu onto our controller nodes so we can install
subordinates like cs:ntp, cs:nrpe, cs:~telegraf-chamers/telegraf and
others. Its also used in test suites for these sort of subordinates.

The 'problem' is, like all reactive charms, the first thing it does is pull
down approximately 160MB of packages and installs them (installing pip
pulls in build-essentials, or at least a big chunk of it). Its very
noticeable when working locally, and maybe in CI environments.

If I knew how to solve this for all reactive charms, I would have suggested
it already. It could be fixed in cs:ubuntu by making it non-reactive, if
people think it is worth it (its not like it actually needs any reactive
features. A minimal metadata.yaml and an install or start hook to set the
status is all it needs).

Maybe reactive is entrenched enough as the new world order that we can get
specific cloud images spun for it, where a pile of packages are
preinstalled so we don't need to wait for cloud-init or the charm to
install them. We might be able to lower deployment times from minutes to
seconds, since often this step is the main time sink.

-- 
Stuart Bishop 
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Nate Finch]

So, the whole point of my original email was to say - if you want a minimal
charm, just make one, it's easy.
I just published my above mentioned minimal charm as cs:~natefinch/nop

It's not showing up on jujucharms.com, maybe because charm proof is failing
because it's missing everything except the metadata?  I don't know, but it
still works with juju deploy.

On Thu, Dec 1, 2016 at 10:07 PM José Antonio Rey  wrote:

> Wouldn't it be possible for us to implement a configuration flag, and
> have it as default? Going back to the general point, the idea behind the
> ubuntu charm is to have a vainilla Ubuntu where you can work on
> anything. I understand we're mostly using it for testing, and reactive
> is now a big part of the ecosystem. I find two simple approaches for this:
>
>   * Create a ubuntu-vainilla charm which doesn't install any of the
> required packages OR
>   * Implement a 'vainilla' boolean configuration flag, where you can
> specify True for the vainilla Ubuntu install, False to install all of
> the reactive/other dependencies.
>
> If we get to work around the pyyaml issue and implement a better
> solution in the long term, I think that would be amazing. However, we
> can't let one dependency drag us down, and in the meanwhile, we have to
> implement a workaround.
>
> On 12/01/2016 01:56 PM, Cory Johns wrote:
> > Marco,
> >
> > What is the issue you mentioned with using snaps where you mentioned
> > needing an "unconfined classic snap"?
> >
> > On Thu, Dec 1, 2016 at 1:13 PM, Marco Ceppi  > > wrote:
> >
> > On Thu, Dec 1, 2016 at 12:56 PM Casey Marshall
> > mailto:casey.marsh...@canonical.com>>
> > wrote:
> >
> > On Thu, Dec 1, 2016 at 6:53 AM, Marco Ceppi
> > mailto:marco.ce...@canonical.com>>
> > wrote:
> >
> > On Thu, Dec 1, 2016 at 5:00 AM Adam Collard
> >  > > wrote:
> >
> > On Thu, 1 Dec 2016 at 04:02 Nate Finch
> >  > > wrote:
> >
> > On IRC, someone was lamenting the fact that the
> > Ubuntu charm takes longer to deploy now, because it
> > has been updated to exercise more of Juju's
> > features.  My response was - just make a minimal
> > charm, it's easy.  And then of course, I had to
> > figure out how minimal you can get.  Here it is:
> >
> > It's just a directory with a metadata.yaml in it
> > with these contents:
> >
> > name: min
> > summary: nope
> > description: nope
> > series:
> >   - xenial
> >
> > (obviously you can set the series to whatever you
> want)
> > No other files or directories are needed.
> >
> >
> > This is neat, but doesn't detract from the bloat in the
> > ubuntu charm.
> >
> >
> > I'm happy to work though changes to the Ubuntu charm to
> > decrease "bloat".
> >
> >
> > IMHO the bloat in the ubuntu charm isn't from support
> > for Juju features, but the switch to reactive plus
> > conflicts in layer-base wanting to a) support lots of
> > toolchains to allow layers above it to be slimmer and b)
> > be a suitable base for "just deploy me" ubuntu.
> >
> >
> > But it is to support the reactive framework, where we
> > utilize newer Juju features, like status and
> > application-version to make the charm rich despite it's
> > minimal goal set. Honestly, a handful of cached wheelhouses
> > and some apt packages don't strike me as bloat, but I do
> > want to make sure the Ubuntu charm works for those using it.
> So,
> >
> >
> > I think a minimal wheelhouse to provide a consistent charm hook
> > runtime is very reasonable and definitely not the problem here.
> >
> > There are too many packages that get installed by default with
> > the reactive framework that most charms don't need. When I
> > deploy the ubuntu charm (but this applies to any charm built
> > with reactive and layer:basic), I also get:
> >
> > 2016-12-01 17:45:47 INFO install The following NEW packages will
> > be installed:
> > 2016-12-01 17:45:47 INFO install   binutils build-essential cpp
> > cpp-5 dpkg-dev fakeroot g++ g++-5 gc
> > c gcc-5
> > 2016-12-01 17:45:47 INFO install   libalgorithm-diff-perl
> > libalgorithm-diff-xs-perl libalgorithm-mer
> > ge-perl
> > 2016-12-01 17:45:47 INFO install   libasan2 libatomic1
> > libc-dev-bin libc6-dev libcc1-0 libcilk

Fwd: A (Very) Minimal Charm

[Adam Collard]

Oops, forgot to reply to the list.

-- Forwarded message -
From: Adam Collard 
Date: Thu, 1 Dec 2016 at 13:43
Subject: Re: A (Very) Minimal Charm
To: Marco Ceppi 


On Thu, 1 Dec 2016 at 12:53 Marco Ceppi  wrote:

On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
wrote:

On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:

On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
to deploy now, because it has been updated to exercise more of Juju's
features.  My response was - just make a minimal charm, it's easy.  And
then of course, I had to figure out how minimal you can get.  Here it is:

It's just a directory with a metadata.yaml in it with these contents:

name: min
summary: nope
description: nope
series:
  - xenial

(obviously you can set the series to whatever you want)
No other files or directories are needed.


This is neat, but doesn't detract from the bloat in the ubuntu charm.


I'm happy to work though changes to the Ubuntu charm to decrease "bloat".


Great!




IMHO the bloat in the ubuntu charm isn't from support for Juju features,
but the switch to reactive plus conflicts in layer-base wanting to a)
support lots of toolchains to allow layers above it to be slimmer and b) be
a suitable base for "just deploy me" ubuntu.


But it is to support the reactive framework, where we utilize newer Juju
features, like status and application-version to make the charm rich
despite it's minimal goal set. Honestly, a handful of cached wheelhouses
and some apt packages don't strike me as bloat, but I do want to make sure
the Ubuntu charm works for those using it. So,

What's the real problem with the Ubuntu charm today?


It apt-get installs something
It pip installs something

Both of these take non-trivial amounts of time.

How does it not achieve it's goal of providing a relatively blank Ubuntu
machine?


It does, but an additional goal (IMO) is to do so quickly and with minimal
(zero?) network requirements.


What are people using the Ubuntu charm for?


Demonstrating/finding Juju bugs whilst ruling out lots of other potential
sources of confusion.


Other than demos, hacks/workarounds, and testing I'm not clear on the
purpose of an Ubuntu charm in a model serves.


Hope this sheds some light on what we were using the Ubuntu charm for.


Marco
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Cory Johns]

Marco,

What is the issue you mentioned with using snaps where you mentioned
needing an "unconfined classic snap"?

On Thu, Dec 1, 2016 at 1:13 PM, Marco Ceppi 
wrote:

> On Thu, Dec 1, 2016 at 12:56 PM Casey Marshall <
> casey.marsh...@canonical.com> wrote:
>
> On Thu, Dec 1, 2016 at 6:53 AM, Marco Ceppi 
> wrote:
>
> On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
> wrote:
>
> On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:
>
> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
> to deploy now, because it has been updated to exercise more of Juju's
> features.  My response was - just make a minimal charm, it's easy.  And
> then of course, I had to figure out how minimal you can get.  Here it is:
>
> It's just a directory with a metadata.yaml in it with these contents:
>
> name: min
> summary: nope
> description: nope
> series:
>   - xenial
>
> (obviously you can set the series to whatever you want)
> No other files or directories are needed.
>
>
> This is neat, but doesn't detract from the bloat in the ubuntu charm.
>
>
> I'm happy to work though changes to the Ubuntu charm to decrease "bloat".
>
>
> IMHO the bloat in the ubuntu charm isn't from support for Juju features,
> but the switch to reactive plus conflicts in layer-base wanting to a)
> support lots of toolchains to allow layers above it to be slimmer and b) be
> a suitable base for "just deploy me" ubuntu.
>
>
> But it is to support the reactive framework, where we utilize newer Juju
> features, like status and application-version to make the charm rich
> despite it's minimal goal set. Honestly, a handful of cached wheelhouses
> and some apt packages don't strike me as bloat, but I do want to make sure
> the Ubuntu charm works for those using it. So,
>
>
> I think a minimal wheelhouse to provide a consistent charm hook runtime is
> very reasonable and definitely not the problem here.
>
> There are too many packages that get installed by default with the
> reactive framework that most charms don't need. When I deploy the ubuntu
> charm (but this applies to any charm built with reactive and layer:basic),
> I also get:
>
> 2016-12-01 17:45:47 INFO install The following NEW packages will be
> installed:
> 2016-12-01 17:45:47 INFO install   binutils build-essential cpp cpp-5
> dpkg-dev fakeroot g++ g++-5 gc
> c gcc-5
> 2016-12-01 17:45:47 INFO install   libalgorithm-diff-perl
> libalgorithm-diff-xs-perl libalgorithm-mer
> ge-perl
> 2016-12-01 17:45:47 INFO install   libasan2 libatomic1 libc-dev-bin
> libc6-dev libcc1-0 libcilkrts5 l
> ibdpkg-perl
> 2016-12-01 17:45:47 INFO install   libexpat1-dev libfakeroot
> libfile-fcntllock-perl libgcc-5-dev libgomp1
> 2016-12-01 17:45:47 INFO install   libisl15 libitm1 liblsan0 libmpc3
> libmpx0 libpython3-dev libpython3.5-dev
> 2016-12-01 17:45:47 INFO install   libquadmath0 libstdc++-5-dev libtsan0
> libubsan0 linux-libc-dev make
> 2016-12-01 17:45:47 INFO install   manpages-dev python-pip-whl python3-dev
> python3-pip python3-setuptools
> 2016-12-01 17:45:47 INFO install   python3-wheel python3.5-dev
>
> None of my charms need build-essential or a g++ compiler, that's a lot of
> unnecessary dependencies! Can we get rid of most of these? Would installing
> the bare minimum with --no-install-recommends help?
>
>
> This comes up a bit, and I'm really eager to figure this out. Allow me to
> explain the catch-22. It's name is pyyaml.
>
> So the wheelhouse, by default, is 3.8M in size, this is the stock
> wheelhouse we vendor in:
>
> 312K charmhelpers-0.10.0.tar.gz
> 21K  charms.reactive-0.4.5.tar.gz
> 349K Jinja2-2.8.tar.gz
> 14K  MarkupSafe-0.23.tar.gz
> 1.7M netaddr-0.7.18.tar.gz
> 1.1M pip-8.1.2.tar.gz
> 19K  pyaml-16.11.4.tar.gz
> 248K PyYAML-3.12.tar.gz
> 29K  six-1.10.0.tar.gz
> 13K  Tempita-0.5.2.tar.gz
>
> The problem child is PyYAML, which is a compiled cpyhton module, which
> requires build-essential. The latest version is 3.12, trusty is 3.10,
> xenial 3.11, and zesty (finally) 3.12 http://packages.ubuntu.
> com/search?keywords=python-yaml, CentOS 6 & 7 are 3.10
>
> So, the easy path here is to just make sure charms.reactive works with
> PyYAML 3.10 and do a `--no-install-recommends` but that's half the problem.
> There will inevitably be python modules that require build-essential to
> compile on install.
>
> We can't cache the compiled wheelhouse because of architectures (same way
> resources complicate this) so we must compile at deploy time.
>
> One path forward, after dropping PyYAML 3.12 (if feasible), would be to
> see if we can detect when a python module needs to be compiled and setting
> a flag in the rendered charm to install the build-essentials, etc.
>
> I'll file issues and spend some time seeing if we can actually detect when
> you need to compile a wheelhouse vs a straight python module that and
> lowering the requirement for PyYAML (using packaged version instead) will
> probably remove a lot of the reactive bootstrap time.
>
> Marco
>
> --
> Juju-dev mailing list
> J

Re: A (Very) Minimal Charm

[Marco Ceppi]

On Thu, Dec 1, 2016 at 12:56 PM Casey Marshall 
wrote:

On Thu, Dec 1, 2016 at 6:53 AM, Marco Ceppi 
wrote:

On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
wrote:

On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:

On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
to deploy now, because it has been updated to exercise more of Juju's
features.  My response was - just make a minimal charm, it's easy.  And
then of course, I had to figure out how minimal you can get.  Here it is:

It's just a directory with a metadata.yaml in it with these contents:

name: min
summary: nope
description: nope
series:
  - xenial

(obviously you can set the series to whatever you want)
No other files or directories are needed.


This is neat, but doesn't detract from the bloat in the ubuntu charm.


I'm happy to work though changes to the Ubuntu charm to decrease "bloat".


IMHO the bloat in the ubuntu charm isn't from support for Juju features,
but the switch to reactive plus conflicts in layer-base wanting to a)
support lots of toolchains to allow layers above it to be slimmer and b) be
a suitable base for "just deploy me" ubuntu.


But it is to support the reactive framework, where we utilize newer Juju
features, like status and application-version to make the charm rich
despite it's minimal goal set. Honestly, a handful of cached wheelhouses
and some apt packages don't strike me as bloat, but I do want to make sure
the Ubuntu charm works for those using it. So,


I think a minimal wheelhouse to provide a consistent charm hook runtime is
very reasonable and definitely not the problem here.

There are too many packages that get installed by default with the reactive
framework that most charms don't need. When I deploy the ubuntu charm (but
this applies to any charm built with reactive and layer:basic), I also get:

2016-12-01 17:45:47 INFO install The following NEW packages will be
installed:
2016-12-01 17:45:47 INFO install   binutils build-essential cpp cpp-5
dpkg-dev fakeroot g++ g++-5 gc
c gcc-5
2016-12-01 17:45:47 INFO install   libalgorithm-diff-perl
libalgorithm-diff-xs-perl libalgorithm-mer
ge-perl
2016-12-01 17:45:47 INFO install   libasan2 libatomic1 libc-dev-bin
libc6-dev libcc1-0 libcilkrts5 l
ibdpkg-perl
2016-12-01 17:45:47 INFO install   libexpat1-dev libfakeroot
libfile-fcntllock-perl libgcc-5-dev libgomp1
2016-12-01 17:45:47 INFO install   libisl15 libitm1 liblsan0 libmpc3
libmpx0 libpython3-dev libpython3.5-dev
2016-12-01 17:45:47 INFO install   libquadmath0 libstdc++-5-dev libtsan0
libubsan0 linux-libc-dev make
2016-12-01 17:45:47 INFO install   manpages-dev python-pip-whl python3-dev
python3-pip python3-setuptools
2016-12-01 17:45:47 INFO install   python3-wheel python3.5-dev

None of my charms need build-essential or a g++ compiler, that's a lot of
unnecessary dependencies! Can we get rid of most of these? Would installing
the bare minimum with --no-install-recommends help?


This comes up a bit, and I'm really eager to figure this out. Allow me to
explain the catch-22. It's name is pyyaml.

So the wheelhouse, by default, is 3.8M in size, this is the stock
wheelhouse we vendor in:

312K charmhelpers-0.10.0.tar.gz
21K  charms.reactive-0.4.5.tar.gz
349K Jinja2-2.8.tar.gz
14K  MarkupSafe-0.23.tar.gz
1.7M netaddr-0.7.18.tar.gz
1.1M pip-8.1.2.tar.gz
19K  pyaml-16.11.4.tar.gz
248K PyYAML-3.12.tar.gz
29K  six-1.10.0.tar.gz
13K  Tempita-0.5.2.tar.gz

The problem child is PyYAML, which is a compiled cpyhton module, which
requires build-essential. The latest version is 3.12, trusty is 3.10,
xenial 3.11, and zesty (finally) 3.12
http://packages.ubuntu.com/search?keywords=python-yaml, CentOS 6 & 7 are
3.10

So, the easy path here is to just make sure charms.reactive works with
PyYAML 3.10 and do a `--no-install-recommends` but that's half the problem.
There will inevitably be python modules that require build-essential to
compile on install.

We can't cache the compiled wheelhouse because of architectures (same way
resources complicate this) so we must compile at deploy time.

One path forward, after dropping PyYAML 3.12 (if feasible), would be to see
if we can detect when a python module needs to be compiled and setting a
flag in the rendered charm to install the build-essentials, etc.

I'll file issues and spend some time seeing if we can actually detect when
you need to compile a wheelhouse vs a straight python module that and
lowering the requirement for PyYAML (using packaged version instead) will
probably remove a lot of the reactive bootstrap time.

Marco
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Casey Marshall]

On Thu, Dec 1, 2016 at 6:53 AM, Marco Ceppi 
wrote:

> On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
> wrote:
>
>> On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:
>>
>> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
>> to deploy now, because it has been updated to exercise more of Juju's
>> features.  My response was - just make a minimal charm, it's easy.  And
>> then of course, I had to figure out how minimal you can get.  Here it is:
>>
>> It's just a directory with a metadata.yaml in it with these contents:
>>
>> name: min
>> summary: nope
>> description: nope
>> series:
>>   - xenial
>>
>> (obviously you can set the series to whatever you want)
>> No other files or directories are needed.
>>
>>
>> This is neat, but doesn't detract from the bloat in the ubuntu charm.
>>
>
> I'm happy to work though changes to the Ubuntu charm to decrease "bloat".
>
>
>> IMHO the bloat in the ubuntu charm isn't from support for Juju features,
>> but the switch to reactive plus conflicts in layer-base wanting to a)
>> support lots of toolchains to allow layers above it to be slimmer and b) be
>> a suitable base for "just deploy me" ubuntu.
>>
>
> But it is to support the reactive framework, where we utilize newer Juju
> features, like status and application-version to make the charm rich
> despite it's minimal goal set. Honestly, a handful of cached wheelhouses
> and some apt packages don't strike me as bloat, but I do want to make sure
> the Ubuntu charm works for those using it. So,
>

I think a minimal wheelhouse to provide a consistent charm hook runtime is
very reasonable and definitely not the problem here.

There are too many packages that get installed by default with the reactive
framework that most charms don't need. When I deploy the ubuntu charm (but
this applies to any charm built with reactive and layer:basic), I also get:

2016-12-01 17:45:47 INFO install The following NEW packages will be
installed:
2016-12-01 17:45:47 INFO install   binutils build-essential cpp cpp-5
dpkg-dev fakeroot g++ g++-5 gc
c gcc-5
2016-12-01 17:45:47 INFO install   libalgorithm-diff-perl
libalgorithm-diff-xs-perl libalgorithm-mer
ge-perl
2016-12-01 17:45:47 INFO install   libasan2 libatomic1 libc-dev-bin
libc6-dev libcc1-0 libcilkrts5 l
ibdpkg-perl
2016-12-01 17:45:47 INFO install   libexpat1-dev libfakeroot
libfile-fcntllock-perl libgcc-5-dev libgomp1
2016-12-01 17:45:47 INFO install   libisl15 libitm1 liblsan0 libmpc3
libmpx0 libpython3-dev libpython3.5-dev
2016-12-01 17:45:47 INFO install   libquadmath0 libstdc++-5-dev libtsan0
libubsan0 linux-libc-dev make
2016-12-01 17:45:47 INFO install   manpages-dev python-pip-whl python3-dev
python3-pip python3-setuptools
2016-12-01 17:45:47 INFO install   python3-wheel python3.5-dev

None of my charms need build-essential or a g++ compiler, that's a lot of
unnecessary dependencies! Can we get rid of most of these? Would installing
the bare minimum with --no-install-recommends help?


> What's the real problem with the Ubuntu charm today?
> How does it not achieve it's goal of providing a relatively blank Ubuntu
> machine? What are people using the Ubuntu charm for?
>
> Other than demos, hacks/workarounds, and testing I'm not clear on the
> purpose of an Ubuntu charm in a model serves.
>
> Marco
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju-dev
>
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Aaron Bentley]

I hope that as you implement this, you avoid making fat charms.  Can you
use "resources" for this?

Aaron

On 2016-12-01 08:39 AM, Marco Ceppi wrote:
> On Thu, Dec 1, 2016 at 8:28 AM Free Ekanayaka
> mailto:free.ekanay...@canonical.com>> wrote:
> 
> On 1 December 2016 at 13:53, Marco Ceppi  > wrote:
> 
> On Thu, Dec 1, 2016 at 5:00 AM Adam Collard
> mailto:adam.coll...@canonical.com>>
> wrote:
> 
> On Thu, 1 Dec 2016 at 04:02 Nate Finch
> mailto:nate.fi...@canonical.com>>
> wrote:
> 
> On IRC, someone was lamenting the fact that the Ubuntu
> charm takes longer to deploy now, because it has been
> updated to exercise more of Juju's features.  My
> response was - just make a minimal charm, it's easy. 
> And then of course, I had to figure out how minimal you
> can get.  Here it is:
> 
> 
> This is neat, but doesn't detract from the bloat in the
> ubuntu charm.
> 
> 
> I'm happy to work though changes to the Ubuntu charm to decrease
> "bloat".
>  
> 
> IMHO the bloat in the ubuntu charm isn't from support for
> Juju features, but the switch to reactive plus conflicts in
> layer-base wanting to a) support lots of toolchains to allow
> layers above it to be slimmer and b) be a suitable base for
> "just deploy me" ubuntu.
> 
> 
> But it is to support the reactive framework, where we utilize
> newer Juju features, like status and application-version to make
> the charm rich despite it's minimal goal set.
> 
> 
> Yeah, and I think this is a good thing.
>  
> 
> Honestly, a handful of cached wheelhouses and some apt packages
> don't strike me as bloat
> 
> 
> No it's not per-se. However I think this highlights a more general
> issue with the current implementation of the reactive stack. It's
> not only the ubuntu charm that has slowed done, it's any
> reactive-based charm, because the steps required to "setup" reactive
> take longer than they used to.
> 
> 
> The problem we're hitting with wheelhouses is exactly the one that snaps
> aim to solve:
> 
> - apt packages are not cross distro, and we have reactive centos charms
> - apt packages lag a bit meaning we can't get consistent features even
> between trusty and xenial, let alone xenial and tip
> 
> I see a couple of (possibly alternative) ways to improve the situation:
> 
> 1) Make sure the dependencies of the base reactive layer are
> packaged, that should be much faster than pip install, and fall back
> to pip only for what's not there (i.e. dependencies added by the
> consumers of the base layer). Also, package the base layer itself.
> 
> 
> I'm very keen on a development in the snap world, where an unconfined
> "classic" style package would be available. This means we could snap up
> all the dependencies of the basic layer for every architecture and skip
> the setup phase for reactive. I think this is probably our best bet
> going forward.
>  
> 
> 2) Add support for images, so when you deploy some vanilla charm
> there's an associated "pre-built" image that will be very fast. I
> guess this is in the juju road map anyways.
> 
> 
> Sure, a build step is an interesting one, but it still incurs the same
> wait for a setup, you just don't feel it as much.
>  
> 
> We always need to keep in mind that this experience will be compared
> with things like Kubernetes and Docker, and speed-y deployments
> really unlock velocity when iterating on charm development (think
> for instance running integration tests).
> 
> 
> Speed is just one facet of the experience, though an important one. For
> the speed of Kubernetes we win, hands down: 10 mins to deploy a full k8s
> cluster (with Juju) is only really outpaced by SAAS. I know that's not
> the point you were making, but it's the true speed comparison of what
> we're doing.
> 
> That being said, we're very keen on making charm development, and
> deployment, faster. Reactive 1.0 was a first leap in that direction, as
> we round off work in 2.0 and leveraging other technologies like
> unconfined snaps, we can start to speed up the bootstrap process of
> reactive charms.
> 
> I'll file bugs to track these items
> 
> Marco
> 
> 



signature.asc
Description: OpenPGP digital signature
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Free Ekanayaka]

On 1 December 2016 at 14:39, Marco Ceppi  wrote:

> On Thu, Dec 1, 2016 at 8:28 AM Free Ekanayaka <
> free.ekanay...@canonical.com> wrote:
>
>> On 1 December 2016 at 13:53, Marco Ceppi 
>> wrote:
>>
>> On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
>> wrote:
>>
>> On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:
>>
>> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
>> to deploy now, because it has been updated to exercise more of Juju's
>> features.  My response was - just make a minimal charm, it's easy.  And
>> then of course, I had to figure out how minimal you can get.  Here it is:
>>
>>
>> This is neat, but doesn't detract from the bloat in the ubuntu charm.
>>
>>
>> I'm happy to work though changes to the Ubuntu charm to decrease "bloat".
>>
>>
>> IMHO the bloat in the ubuntu charm isn't from support for Juju features,
>> but the switch to reactive plus conflicts in layer-base wanting to a)
>> support lots of toolchains to allow layers above it to be slimmer and b) be
>> a suitable base for "just deploy me" ubuntu.
>>
>>
>> But it is to support the reactive framework, where we utilize newer Juju
>> features, like status and application-version to make the charm rich
>> despite it's minimal goal set.
>>
>>
>> Yeah, and I think this is a good thing.
>>
>>
>> Honestly, a handful of cached wheelhouses and some apt packages don't
>> strike me as bloat
>>
>>
>> No it's not per-se. However I think this highlights a more general issue
>> with the current implementation of the reactive stack. It's not only the
>> ubuntu charm that has slowed done, it's any reactive-based charm, because
>> the steps required to "setup" reactive take longer than they used to.
>>
>
> The problem we're hitting with wheelhouses is exactly the one that snaps
> aim to solve:
>
> - apt packages are not cross distro, and we have reactive centos charms
> - apt packages lag a bit meaning we can't get consistent features even
> between trusty and xenial, let alone xenial and tip
>
> I see a couple of (possibly alternative) ways to improve the situation:
>>
>> 1) Make sure the dependencies of the base reactive layer are packaged,
>> that should be much faster than pip install, and fall back to pip only for
>> what's not there (i.e. dependencies added by the consumers of the base
>> layer). Also, package the base layer itself.
>>
>
> I'm very keen on a development in the snap world, where an unconfined
> "classic" style package would be available. This means we could snap up all
> the dependencies of the basic layer for every architecture and skip the
> setup phase for reactive. I think this is probably our best bet going
> forward.
>

Sounds good.
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Marco Ceppi]

On Thu, Dec 1, 2016 at 8:28 AM Free Ekanayaka 
wrote:

> On 1 December 2016 at 13:53, Marco Ceppi 
> wrote:
>
> On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
> wrote:
>
> On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:
>
> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
> to deploy now, because it has been updated to exercise more of Juju's
> features.  My response was - just make a minimal charm, it's easy.  And
> then of course, I had to figure out how minimal you can get.  Here it is:
>
>
> This is neat, but doesn't detract from the bloat in the ubuntu charm.
>
>
> I'm happy to work though changes to the Ubuntu charm to decrease "bloat".
>
>
> IMHO the bloat in the ubuntu charm isn't from support for Juju features,
> but the switch to reactive plus conflicts in layer-base wanting to a)
> support lots of toolchains to allow layers above it to be slimmer and b) be
> a suitable base for "just deploy me" ubuntu.
>
>
> But it is to support the reactive framework, where we utilize newer Juju
> features, like status and application-version to make the charm rich
> despite it's minimal goal set.
>
>
> Yeah, and I think this is a good thing.
>
>
> Honestly, a handful of cached wheelhouses and some apt packages don't
> strike me as bloat
>
>
> No it's not per-se. However I think this highlights a more general issue
> with the current implementation of the reactive stack. It's not only the
> ubuntu charm that has slowed done, it's any reactive-based charm, because
> the steps required to "setup" reactive take longer than they used to.
>

The problem we're hitting with wheelhouses is exactly the one that snaps
aim to solve:

- apt packages are not cross distro, and we have reactive centos charms
- apt packages lag a bit meaning we can't get consistent features even
between trusty and xenial, let alone xenial and tip

I see a couple of (possibly alternative) ways to improve the situation:
>
> 1) Make sure the dependencies of the base reactive layer are packaged,
> that should be much faster than pip install, and fall back to pip only for
> what's not there (i.e. dependencies added by the consumers of the base
> layer). Also, package the base layer itself.
>

I'm very keen on a development in the snap world, where an unconfined
"classic" style package would be available. This means we could snap up all
the dependencies of the basic layer for every architecture and skip the
setup phase for reactive. I think this is probably our best bet going
forward.


> 2) Add support for images, so when you deploy some vanilla charm there's
> an associated "pre-built" image that will be very fast. I guess this is in
> the juju road map anyways.
>

Sure, a build step is an interesting one, but it still incurs the same wait
for a setup, you just don't feel it as much.


> We always need to keep in mind that this experience will be compared with
> things like Kubernetes and Docker, and speed-y deployments really unlock
> velocity when iterating on charm development (think for instance running
> integration tests).
>

Speed is just one facet of the experience, though an important one. For the
speed of Kubernetes we win, hands down: 10 mins to deploy a full k8s
cluster (with Juju) is only really outpaced by SAAS. I know that's not the
point you were making, but it's the true speed comparison of what we're
doing.

That being said, we're very keen on making charm development, and
deployment, faster. Reactive 1.0 was a first leap in that direction, as we
round off work in 2.0 and leveraging other technologies like unconfined
snaps, we can start to speed up the bootstrap process of reactive charms.

I'll file bugs to track these items

Marco
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Free Ekanayaka]

On 1 December 2016 at 13:53, Marco Ceppi  wrote:

> On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
> wrote:
>
>> On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:
>>
>> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
>> to deploy now, because it has been updated to exercise more of Juju's
>> features.  My response was - just make a minimal charm, it's easy.  And
>> then of course, I had to figure out how minimal you can get.  Here it is:
>>
>> It's just a directory with a metadata.yaml in it with these contents:
>>
>> name: min
>> summary: nope
>> description: nope
>> series:
>>   - xenial
>>
>> (obviously you can set the series to whatever you want)
>> No other files or directories are needed.
>>
>>
>> This is neat, but doesn't detract from the bloat in the ubuntu charm.
>>
>
> I'm happy to work though changes to the Ubuntu charm to decrease "bloat".
>
>
>> IMHO the bloat in the ubuntu charm isn't from support for Juju features,
>> but the switch to reactive plus conflicts in layer-base wanting to a)
>> support lots of toolchains to allow layers above it to be slimmer and b) be
>> a suitable base for "just deploy me" ubuntu.
>>
>
> But it is to support the reactive framework, where we utilize newer Juju
> features, like status and application-version to make the charm rich
> despite it's minimal goal set.
>

Yeah, and I think this is a good thing.


> Honestly, a handful of cached wheelhouses and some apt packages don't
> strike me as bloat
>

No it's not per-se. However I think this highlights a more general issue
with the current implementation of the reactive stack. It's not only the
ubuntu charm that has slowed done, it's any reactive-based charm, because
the steps required to "setup" reactive take longer than they used to.

I see a couple of (possibly alternative) ways to improve the situation:

1) Make sure the dependencies of the base reactive layer are packaged, that
should be much faster than pip install, and fall back to pip only for
what's not there (i.e. dependencies added by the consumers of the base
layer). Also, package the base layer itself.

2) Add support for images, so when you deploy some vanilla charm there's an
associated "pre-built" image that will be very fast. I guess this is in the
juju road map anyways.

We always need to keep in mind that this experience will be compared with
things like Kubernetes and Docker, and speed-y deployments really unlock
velocity when iterating on charm development (think for instance running
integration tests).
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Marco Ceppi]

On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
wrote:

> On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:
>
> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
> to deploy now, because it has been updated to exercise more of Juju's
> features.  My response was - just make a minimal charm, it's easy.  And
> then of course, I had to figure out how minimal you can get.  Here it is:
>
> It's just a directory with a metadata.yaml in it with these contents:
>
> name: min
> summary: nope
> description: nope
> series:
>   - xenial
>
> (obviously you can set the series to whatever you want)
> No other files or directories are needed.
>
>
> This is neat, but doesn't detract from the bloat in the ubuntu charm.
>

I'm happy to work though changes to the Ubuntu charm to decrease "bloat".


> IMHO the bloat in the ubuntu charm isn't from support for Juju features,
> but the switch to reactive plus conflicts in layer-base wanting to a)
> support lots of toolchains to allow layers above it to be slimmer and b) be
> a suitable base for "just deploy me" ubuntu.
>

But it is to support the reactive framework, where we utilize newer Juju
features, like status and application-version to make the charm rich
despite it's minimal goal set. Honestly, a handful of cached wheelhouses
and some apt packages don't strike me as bloat, but I do want to make sure
the Ubuntu charm works for those using it. So,

What's the real problem with the Ubuntu charm today?
How does it not achieve it's goal of providing a relatively blank Ubuntu
machine? What are people using the Ubuntu charm for?

Other than demos, hacks/workarounds, and testing I'm not clear on the
purpose of an Ubuntu charm in a model serves.

Marco
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

[Adam Collard]

On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:

On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
to deploy now, because it has been updated to exercise more of Juju's
features.  My response was - just make a minimal charm, it's easy.  And
then of course, I had to figure out how minimal you can get.  Here it is:

It's just a directory with a metadata.yaml in it with these contents:

name: min
summary: nope
description: nope
series:
  - xenial

(obviously you can set the series to whatever you want)
No other files or directories are needed.


This is neat, but doesn't detract from the bloat in the ubuntu charm.

IMHO the bloat in the ubuntu charm isn't from support for Juju features,
but the switch to reactive plus conflicts in layer-base wanting to a)
support lots of toolchains to allow layers above it to be slimmer and b) be
a suitable base for "just deploy me" ubuntu.
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

A (Very) Minimal Charm

[Nate Finch]

On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
to deploy now, because it has been updated to exercise more of Juju's
features.  My response was - just make a minimal charm, it's easy.  And
then of course, I had to figure out how minimal you can get.  Here it is:

It's just a directory with a metadata.yaml in it with these contents:

name: min
summary: nope
description: nope
series:
  - xenial

(obviously you can set the series to whatever you want)
No other files or directories are needed.

Figured this might be useful for others.

-Nate
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev