Re: Juju Compose

2015-07-14 Thread Cory Johns
On Tue, Jul 14, 2015 at 4:45 PM, Marco Ceppi marco.ce...@canonical.com wrote:
 I think you're on the right train of thought as to what this is, but miss a
 few important issues this helps address. It's is trivial to copy code
 around, and we talk about it all the time in charm schools find a charm
 similar, fork, edit. What I think compose is helping solve, and correct me
 if I'm wrong Ben  co, is that those operations are a one time event, where
 compose is an operation that can be done multiple times.

 You fork the tomcat charm, you munge in your bits to deploy your tomcat
 application and now you have a charm. That's it. If the maintainer of the
 tomcat charm updates that charm, there's no way you - who now has old code,
 that possibly could contain bugs or security issues - would know to update
 it, or how to even how to update it. By having layers of charms - in the
 sense of a tomcat layer, then your bits in another layer, that get composed
 to create one final charm it would address those issues. Now we have a
 mechanism to not only know how and what files were copied from where, we
 also have the ability to potentially notify users when a base is updated
 (prompting them to re-compose their charm and test it).

I don't think this fully highlights why compose is different from, and
I think better than (for the use-case for which it is intended),
existing VCS solutions.

 With this approach we no longer need to shoe-horning huge complex ecosystems
 to use the framework charms concept which is really just an over-extension
 of the subordinate charm feature in Juju.Instead, we could build framework
 layers (tomcat, docker, django, wordpress, magento, hadoop) which themselves
 could be charms or just fragments of charms. Using compose users could start
 to mix and match so that you could theoretically combine a tomcat layer with
 a docker layer and other layers then add your own custom layer on top and
 now you've got a charm which you can update and manage when layers are
 updated.

 Another thing that this approach opens up to help solve, and it's been a
 problem for Juju since essentially day one, is relationship management.
 Potentially, instead of having to read other code to figure out how to
 reproduce a relation there could just be interface layers that model the
 relation exchange and allow you to write your layer to just respond to those
 events. While this seems complex, and would be a bit down the road, this
 pattern starts to solidify a whole new way to create, manage, and maintain
 charms.

 At the end of the day, this is just as you said. Detached version control of
 files being moved around, and we can do that already. This legitimizes that
 and makes it so that managing that code going forward isn't so tedious,
 opening up new avenues for charm authors when it comes to creating charms.

This is what it's really about, but I want to clarify exactly why we
want to do this instead of VCS forking.

Let's say you take an existing charm and fork it using your favorite
VCS, and then make some changes to extend it.  You now have a complete
copy of the original charm plus your changes.  The VCS can then help
you track and merge in upstream changes, and they do this quite well,
so this is obviously not what we're trying to accomplish, though it is
of course necessary.

The key point is that your fork is a *complete* copy of the original
charm.  While this is exactly what you want for deploying, what you're
losing (or at least what is much harder to see on its own) is what
*you* changed.  It is much more difficult to see just the things that
you have added or removed from the charm.  This makes it harder to
review the differences, and in many cases it makes it harder to
understand.

What compose is all about is allowing charm authors to maintain *just*
the layer that is different from the base.  Just their logic.  And the
base can itself be made up of multiple layers, each building on the
one below it.  Yet at the end of the day, you can tell the composer to
put all those layers together into a single, deployable, auditable,
complete charm.

Another area where VCS falls down is bringing together *multiple*
separate sources into a single resulting charm.  With compose, charm
layers become building blocks for building more complex charms.  And,
as Marco pointed out, relations can be decomposed into layers that
allow us to reuse the code so that each charm is not reimplementing
the logic (possibly incompletely).

Of course, some of this can be accomplished using libraries, but then
the end-result charm is not complete.  Library / dependency management
becomes a significant issue, as it has been for some time with charms.
And browsing the charm on jujucharms.com is more difficult because it
is harder to follow the logic into external libraries (or libraries
bundled in as compressed archives, etc.).

Compose gives us a very descriptive way to say, These are all the
common, shared bits that make up my 

Re: Juju Compose

2015-07-14 Thread Nate Finch
I'm not sure I understand the problems this is solving that aren't
adequately solved by branching from a source charm.  It seems like the hard
part of branching - maintaining changes to non-trivial code - is not helped
by this tool.  For charms that just need to add hook files or modify charm
metadata, this seems fine, but those don't seem like the difficult parts of
branching from a source charm either.  I guess my point is, this seems like
it's doing a lot of what version control already does.

On Fri, Jul 10, 2015 at 2:22 PM Benjamin Saller 
benjamin.sal...@canonical.com wrote:

 Charm Composition is a small tool facilitating a new pattern for
 developing and maintain Juju Charms. In its simplest form it lets you
 combine directories, called Layers, of files to create a charm. Rather than
 just copying files around however it keeps track of which files came from
 which Layer so that at a later time the process can be run again. This
 means that if any of the Layers have changed those bits will be reflected
 in the newly regenerated charm.

 Imaging that you have a charm you’ve forked to add some simple tweak or
 customization. You might be able to maintain your changes in a new Layer
 allowing the base you depend on to continue to evolve.

 Suppose we have layer ‘a’.

 trusty/a/

 ├── hooks/install

 ├── metadata.yaml

 └── README.md

 And a layer ‘b’

 trusty/b/

 ├── hooks/db-relation-changed

 ├── composer.yaml

 ├── metadata.yaml

 └── README.md

 If ‘b’ included ‘a’ (specified in the composer.yaml file) then composer
 might combine the layers in the following way.

 trusty/b/

 ├── hooks/install

 ├── hooks/db-relation-changed

 ├── composer.yaml

 ├── metadata.yaml

 └── README.md

 That is just a ‘cp -r’ you’re saying, and it would be except the compose
 process is able to support a variety of methods for combining layers. By
 default compose knows about Charm specific files like metadata.yaml and
 config.yaml it is able to combine them in sensible ways. For files like
 this the default is a merge with the ability to remove add/replace keys
 within the data of the file in the lower layer (in this case ‘a’ would be
 overridden if it had a key in common with ‘b’ when composed).

 Composer itself is extensible so if you’re layers need special rules
 associated with how certain types of files should be combined there is an
 interface for doing that.

 Composer has code for doing much more interesting things than just
 combining layers, but we can save that for another post. If you’re
 interested in this now and want to start thinking about what nice base
 layers might look like, check it out at
 https://github.com/bcsaller/juju-compose



 Thanks,
 Ben

 There is more that should be said about how this relates to the work with
 Relation Stubs and so on but that can be another thread.
 --
 Juju mailing list
 Juju@lists.ubuntu.com
 Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju