Re: jujucharms.com and charmstore outage in progress

[Richard Harding]

On Tue, 14 Jul 2015, Richard Harding wrote:

> FYI: there's currently an issue with the storage system that the charmstore
> and jujucharms.com are deployed to. The IS team is aware and working on
> this and we're tracking their updates to make sure to get things back asap.
> 
> This means that the jujucharms.com urls that hit the charmstore and related
> sections are not functioning.
> 
> Users of juju 1.24.0 and higher will be unable to do a deploy from the
> charmstore until the outage is resolved.
> 
> I'll reply to this email with updates and if you have any questions please
> feel free to ask me in #juju or #juju-gui on freenode. (irc nick rick_h)

The IS team has worked hard to bring up more storage and services are
responding. Note that performance isn't optimal and you might get a timeout
while the storage system rebalances, and syncs up.

If you hit any consistent issues please let us know and we'll investigate.

Thanks everyone for your patience.

-- 

Rick Harding

Juju UI Engineering
https://launchpad.net/~rharding
@mitechie

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

jujucharms.com and charmstore outage in progress

[Richard Harding]

FYI: there's currently an issue with the storage system that the charmstore
and jujucharms.com are deployed to. The IS team is aware and working on
this and we're tracking their updates to make sure to get things back asap.

This means that the jujucharms.com urls that hit the charmstore and related
sections are not functioning.

Users of juju 1.24.0 and higher will be unable to do a deploy from the
charmstore until the outage is resolved.

I'll reply to this email with updates and if you have any questions please
feel free to ask me in #juju or #juju-gui on freenode. (irc nick rick_h)


--

Rick Harding

Juju UI Engineering
https://launchpad.net/~rharding
@mitechie

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Juju Compose

[Cory Johns]

On Tue, Jul 14, 2015 at 4:45 PM, Marco Ceppi  wrote:
> I think you're on the right train of thought as to what this is, but miss a
> few important issues this helps address. It's is trivial to copy code
> around, and we talk about it all the time in charm schools "find a charm
> similar, fork, edit". What I think compose is helping solve, and correct me
> if I'm wrong Ben & co, is that those operations are a one time event, where
> compose is an operation that can be done multiple times.
>
> You fork the tomcat charm, you munge in your bits to deploy your tomcat
> application and now you have a charm. That's it. If the maintainer of the
> tomcat charm updates that charm, there's no way you - who now has old code,
> that possibly could contain bugs or security issues - would know to update
> it, or how to even how to update it. By having layers of charms - in the
> sense of a tomcat layer, then your bits in another layer, that get composed
> to create one final charm it would address those issues. Now we have a
> mechanism to not only know how and what files were copied from where, we
> also have the ability to potentially notify users when a base is updated
> (prompting them to re-compose their charm and test it).

I don't think this fully highlights why compose is different from, and
I think better than (for the use-case for which it is intended),
existing VCS solutions.

> With this approach we no longer need to shoe-horning huge complex ecosystems
> to use the "framework charms" concept which is really just an over-extension
> of the subordinate charm feature in Juju.Instead, we could build framework
> layers (tomcat, docker, django, wordpress, magento, hadoop) which themselves
> could be charms or just fragments of charms. Using compose users could start
> to mix and match so that you could theoretically combine a tomcat layer with
> a docker layer and other layers then add your own custom layer on top and
> now you've got a charm which you can update and manage when layers are
> updated.
>
> Another thing that this approach opens up to help solve, and it's been a
> problem for Juju since essentially day one, is relationship management.
> Potentially, instead of having to read other code to figure out how to
> reproduce a relation there could just be interface layers that model the
> relation exchange and allow you to write your layer to just respond to those
> events. While this seems complex, and would be a bit down the road, this
> pattern starts to solidify a whole new way to create, manage, and maintain
> charms.
>
> At the end of the day, this is just as you said. Detached version control of
> files being moved around, and we can do that already. This legitimizes that
> and makes it so that managing that code going forward isn't so tedious,
> opening up new avenues for charm authors when it comes to creating charms.

This is what it's really about, but I want to clarify exactly why we
want to do this instead of VCS forking.

Let's say you take an existing charm and fork it using your favorite
VCS, and then make some changes to extend it.  You now have a complete
copy of the original charm plus your changes.  The VCS can then help
you track and merge in upstream changes, and they do this quite well,
so this is obviously not what we're trying to accomplish, though it is
of course necessary.

The key point is that your fork is a *complete* copy of the original
charm.  While this is exactly what you want for deploying, what you're
losing (or at least what is much harder to see on its own) is what
*you* changed.  It is much more difficult to see just the things that
you have added or removed from the charm.  This makes it harder to
review the differences, and in many cases it makes it harder to
understand.

What compose is all about is allowing charm authors to maintain *just*
the layer that is different from the base.  Just their logic.  And the
base can itself be made up of multiple layers, each building on the
one below it.  Yet at the end of the day, you can tell the composer to
put all those layers together into a single, deployable, auditable,
complete charm.

Another area where VCS falls down is bringing together *multiple*
separate sources into a single resulting charm.  With compose, charm
layers become building blocks for building more complex charms.  And,
as Marco pointed out, relations can be decomposed into layers that
allow us to reuse the code so that each charm is not reimplementing
the logic (possibly incompletely).

Of course, some of this can be accomplished using libraries, but then
the end-result charm is not complete.  Library / dependency management
becomes a significant issue, as it has been for some time with charms.
And browsing the charm on jujucharms.com is more difficult because it
is harder to follow the logic into external libraries (or libraries
bundled in as compressed archives, etc.).

Compose gives us a very descriptive way to say, "These are all the
common, shared 

Re: Juju Compose

[Marco Ceppi]

Hi Ben,

I've spent some time looking through this and I think it's a great start! I
know we've talked on and off about an idea like this in the past so it's
great to see some public code around it.

On Tue, Jul 14, 2015 at 3:54 PM Nate Finch  wrote:

> I'm not sure I understand the problems this is solving that aren't
> adequately solved by branching from a source charm.  It seems like the hard
> part of branching - maintaining changes to non-trivial code - is not helped
> by this tool.  For charms that just need to add hook files or modify charm
> metadata, this seems fine, but those don't seem like the difficult parts of
> branching from a source charm either.  I guess my point is, this seems like
> it's doing a lot of what version control already does.
>

I think you're on the right train of thought as to what this is, but miss a
few important issues this helps address. It's is trivial to copy code
around, and we talk about it all the time in charm schools "find a charm
similar, fork, edit". What I think compose is helping solve, and correct me
if I'm wrong Ben & co, is that those operations are a one time event, where
compose is an operation that can be done multiple times.

You fork the tomcat charm, you munge in your bits to deploy your tomcat
application and now you have a charm. That's it. If the maintainer of the
tomcat charm updates that charm, there's no way you - who now has old code,
that possibly could contain bugs or security issues - would know to update
it, or how to even how to update it. By having layers of charms - in the
sense of a tomcat layer, then your bits in another layer, that get composed
to create one final charm it would address those issues. Now we have a
mechanism to not only know how and what files were copied from where, we
also have the ability to potentially notify users when a base is updated
(prompting them to re-compose their charm and test it).

With this approach we no longer need to shoe-horning huge
complex ecosystems to use the "framework charms" concept which is really
just an over-extension of the subordinate charm feature in Juju.Instead, we
could build framework layers (tomcat, docker, django, wordpress, magento,
hadoop) which themselves could be charms or just fragments of charms. Using
compose users could start to mix and match so that you could theoretically
combine a tomcat layer with a docker layer and other layers then add your
own custom layer on top and now you've got a charm which you can update and
manage when layers are updated.

Another thing that this approach opens up to help solve, and it's been a
problem for Juju since essentially day one, is relationship management.
Potentially, instead of having to read other code to figure out how to
reproduce a relation there could just be interface layers that model the
relation exchange and allow you to write your layer to just respond to
those events. While this seems complex, and would be a bit down the road,
this pattern starts to solidify a whole new way to create, manage, and
maintain charms.

At the end of the day, this is just as you said. Detached version control
of files being moved around, and we can do that already. This legitimizes
that and makes it so that managing that code going forward isn't so
tedious, opening up new avenues for charm authors when it comes to creating
charms.

Marco


>
> On Fri, Jul 10, 2015 at 2:22 PM Benjamin Saller <
> benjamin.sal...@canonical.com> wrote:
>
>> Charm Composition is a small tool facilitating a new pattern for
>> developing and maintain Juju Charms. In its simplest form it lets you
>> combine directories, called Layers, of files to create a charm. Rather than
>> just copying files around however it keeps track of which files came from
>> which Layer so that at a later time the process can be run again. This
>> means that if any of the Layers have changed those bits will be reflected
>> in the newly regenerated charm.
>>
>> Imaging that you have a charm you’ve forked to add some simple tweak or
>> customization. You might be able to maintain your changes in a new Layer
>> allowing the base you depend on to continue to evolve.
>>
>> Suppose we have layer ‘a’.
>>
>> trusty/a/
>>
>> ├── hooks/install
>>
>> ├── metadata.yaml
>>
>> └── README.md
>>
>> And a layer ‘b’
>>
>> trusty/b/
>>
>> ├── hooks/db-relation-changed
>>
>> ├── composer.yaml
>>
>> ├── metadata.yaml
>>
>> └── README.md
>>
>> If ‘b’ included ‘a’ (specified in the composer.yaml file) then composer
>> might combine the layers in the following way.
>>
>> trusty/b/
>>
>> ├── hooks/install
>>
>> ├── hooks/db-relation-changed
>>
>> ├── composer.yaml
>>
>> ├── metadata.yaml
>>
>> └── README.md
>>
>> That is just a ‘cp -r’ you’re saying, and it would be except the compose
>> process is able to support a variety of methods for combining layers. By
>> default compose knows about Charm specific files like metadata.yaml and
>> config.yaml it is able to combine them in sensible ways. For files like
>> thi

Re: Specify Amazon VPC/Subnet

[Mark Hannessen]

Anyone any idea if deploying inside an existing EC2 VPC/Subnet is possible?

I found someone who proposed a patch for this a long time ago but haven't been 
able to find anything like this for a more recent version.

https://bugs.launchpad.net/juju/+bug/1073133

Met vriendelijke groet, 

Mark Hannessen 


Senior System Administrator / Architect 
Tel: +31 50 5775822-9702 
Mob: 06 414 10 181 
E-mail: m.hannes...@youwe.nl

- Oorspronkelijk bericht -
Van: "Mark Hannessen" 
Aan: "juju" 
Verzonden: Maandag 13 juli 2015 10:21:03
Onderwerp: Specify Amazon VPC/Subnet

Hi,

For one of our customers we seek to deploy a juju setup inside of a Amazon VPC.
I have already configured the VPC with an internal gateway and am able to boot 
machines inside of it as expected.
Internet connectivity works as expected.

Unfortunately juju bootstrap does not seem to deploy the bootstrap node inside 
the VPC by default.
juju bootstrap --constraints "cpu-cores=1" --upload-series trusty 
--upload-tools --to zone=eu-west-1a

I suspect i need a parameter to specify the subnet of the VPC (subnet-06932f5f) 
but i couldn't find it on
https://jujucharms.com/docs/devel/config-general

You can see my current configuration below.

default: amazon

environments:
amazon:
type: "ec2"
region: "eu-west-1"
access-key: ""
secret-key: ""
image-stream: "released"
default-series: trusty


Anyone any idea as to how to make this work?

Kind Regards,

Mark

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Juju Compose

[Nate Finch]

I'm not sure I understand the problems this is solving that aren't
adequately solved by branching from a source charm.  It seems like the hard
part of branching - maintaining changes to non-trivial code - is not helped
by this tool.  For charms that just need to add hook files or modify charm
metadata, this seems fine, but those don't seem like the difficult parts of
branching from a source charm either.  I guess my point is, this seems like
it's doing a lot of what version control already does.

On Fri, Jul 10, 2015 at 2:22 PM Benjamin Saller <
benjamin.sal...@canonical.com> wrote:

> Charm Composition is a small tool facilitating a new pattern for
> developing and maintain Juju Charms. In its simplest form it lets you
> combine directories, called Layers, of files to create a charm. Rather than
> just copying files around however it keeps track of which files came from
> which Layer so that at a later time the process can be run again. This
> means that if any of the Layers have changed those bits will be reflected
> in the newly regenerated charm.
>
> Imaging that you have a charm you’ve forked to add some simple tweak or
> customization. You might be able to maintain your changes in a new Layer
> allowing the base you depend on to continue to evolve.
>
> Suppose we have layer ‘a’.
>
> trusty/a/
>
> ├── hooks/install
>
> ├── metadata.yaml
>
> └── README.md
>
> And a layer ‘b’
>
> trusty/b/
>
> ├── hooks/db-relation-changed
>
> ├── composer.yaml
>
> ├── metadata.yaml
>
> └── README.md
>
> If ‘b’ included ‘a’ (specified in the composer.yaml file) then composer
> might combine the layers in the following way.
>
> trusty/b/
>
> ├── hooks/install
>
> ├── hooks/db-relation-changed
>
> ├── composer.yaml
>
> ├── metadata.yaml
>
> └── README.md
>
> That is just a ‘cp -r’ you’re saying, and it would be except the compose
> process is able to support a variety of methods for combining layers. By
> default compose knows about Charm specific files like metadata.yaml and
> config.yaml it is able to combine them in sensible ways. For files like
> this the default is a merge with the ability to remove add/replace keys
> within the data of the file in the lower layer (in this case ‘a’ would be
> overridden if it had a key in common with ‘b’ when composed).
>
> Composer itself is extensible so if you’re layers need special rules
> associated with how certain types of files should be combined there is an
> interface for doing that.
>
> Composer has code for doing much more interesting things than just
> combining layers, but we can save that for another post. If you’re
> interested in this now and want to start thinking about what nice base
> layers might look like, check it out at
> https://github.com/bcsaller/juju-compose
>
>
>
> Thanks,
> Ben
>
> There is more that should be said about how this relates to the work with
> Relation Stubs and so on but that can be another thread.
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Specify Amazon VPC/Subnet

[Dimiter Naydenov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Mark,

It seems you're hitting a known bug:
https://bugs.launchpad.net/juju-core/+bug/1321442

I've analyzed the code and updated the bug with a rough plan to roll
out support for the non-default VPC scenario on EC2.

Unfortunately, juju supports either an EC2-Classic (no VPC) or EC2-VPC
(only with a default VPC) at the moment. There are features planned
and already in progress in feature branches that will allow greater
control and flexibility with VPC deployments, so hopefully the
situation will improve soon. Until then the proposed workaround -
having a way to specify a "vpc-id" to use in environments.yaml needs
to be implemented. So please use the quoted bug to track its progress
and add your feedback on the proposal.

Thanks,
Dimiter

On 13.07.2015 11:21, Mark Hannessen wrote:
> Hi,
> 
> For one of our customers we seek to deploy a juju setup inside of a
> Amazon VPC. I have already configured the VPC with an internal
> gateway and am able to boot machines inside of it as expected. 
> Internet connectivity works as expected.
> 
> Unfortunately juju bootstrap does not seem to deploy the bootstrap
> node inside the VPC by default. juju bootstrap --constraints
> "cpu-cores=1" --upload-series trusty --upload-tools --to
> zone=eu-west-1a
> 
> I suspect i need a parameter to specify the subnet of the VPC
> (subnet-06932f5f) but i couldn't find it on 
> https://jujucharms.com/docs/devel/config-general
> 
> You can see my current configuration below.
> 
> default: amazon
> 
> environments: amazon: type: "ec2" region: "eu-west-1" access-key:
> "" secret-key: "" image-stream: "released" 
> default-series: trusty
> 
> 
> Anyone any idea as to how to make this work?
> 
> Kind Regards,
> 
> Mark
> 


- -- 
Dimiter Naydenov 
Juju Core Sapphire team 
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJVpPxWAAoJENzxV2TbLzHw/iIH/RWm8v9mHzpowfXrUPJidH1F
Q6NfgpCm9DuFCLLcNUxMjmFOWD5/DkhcmDe/B8ib9NrE27Qd56qpCVLYxFWj/5xm
QJ/ch4s5S2xlSFixH3423VFyDqY9EQ08Ftg8Q5wCkuCHGyEm5Cg1K+HcXduOsWaC
Z7NE7+Arb9YOybUto3DSofg0Qs2WpQYhQkxBuwXyxUsYGYCYR0b14gZXlhJ5wqO9
9siUL/PxD7u1IwD+CZ5l3uGxi6MmEX0j1wLU6KGaOwOzutssTdn7/sEz3+Djw6Zj
3fJMjLitDFfBmZhXw6NKwlysB4045B3vpvrCGroFU0zg43NX6M7Cits8c9L2JuM=
=DB/0
-END PGP SIGNATURE-

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju