Re: Charm Store policy updates and refinement for 2.0

[Stuart Bishop]

On 23 March 2016 at 06:37, Jorge O. Castro  wrote:
> Thanks for the feedback Stuart, I've pushed up a new revision.
>
>> I think the acceptable software sources needs to be expanded.
>
> I've added your recommendations for this section except for:
>
>> In addition, any software sources not in the main Ubuntu or CentOS
>> archives should be listed in configuration items that can be
>> overridden rather than hard coded in the charm
>
> I've changed this to a MUST as it's not that much work to do this and
> the effort seems trivial compared to forcing users to mangle a charm
> just to get it to deploy on production systems without egress.

Yeah. And fixing it after people are using your charm in production is
a pain, which I learned the hard way :)



-- 
Stuart Bishop 

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm Store policy updates and refinement for 2.0

[Jorge O. Castro]

Thanks for the feedback Stuart, I've pushed up a new revision.

> I think the acceptable software sources needs to be expanded.

I've added your recommendations for this section except for:

> In addition, any software sources not in the main Ubuntu or CentOS
> archives should be listed in configuration items that can be
> overridden rather than hard coded in the charm

I've changed this to a MUST as it's not that much work to do this and
the effort seems trivial compared to forcing users to mangle a charm
just to get it to deploy on production systems without egress.

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm Store policy updates and refinement for 2.0

[Charles Butler]

Hey Stuart!

Thats a really good point about SSL on the interfaces service. I saw the
bug a few weeks back but it slipped my mind, and it surprised me to
discover its been there since 2015.

I'll work towards having a resolution on this in the next week or so and
will re-ping the list once its been TLS secure'd.

Thanks for beating the drum on this one, i've needed some motivation.

All the best,

On Mon, Mar 21, 2016 at 10:14 PM Stuart Bishop 
wrote:

> On 19 March 2016 at 02:58, Jorge O. Castro  wrote:
>
> > Recommendations from everyone on what we should include here would be
> > most welcome, specifically our recommendations around Windows charms
> > is non-existent.
>
> I think the acceptable software sources needs to be expanded.
> Launchpad PPAs should be acceptable as signing keys are securely
> retrieved when using 'add-apt-repository ppa:foo/bar'. Also, 3rd party
> apt repositories should be acceptable if the signing key is embedded
> in the charm (PyPi could be checked similarly, but it seems rare to
> find signed packages there).
>
> In addition, any software sources not in the main Ubuntu or CentOS
> archives should be listed in configuration items that can be
> overridden rather than hard coded in the charm, or else the charm is
> useless in network restricted environments (and yes, migrating to
> resources may be a better user experience in many cases).
>
> As examples, the PostgreSQL charm pulls non-default packages from the
> upstream PostgreSQL apt repository (PGDG, which is the source which
> flows to Debian and Ubuntu). The Cassandra charm pulls a required
> driver from a PPA I control. It also installs packages from either the
> Apache apt repository or the DataStax apt repository. Cassandra is not
> available in the Debian or Ubuntu main archives, probably as it
> required the Oracle JVM. Both charms use the
> install_sources/install_keys config items parsed by charm-helpers and
> the apt layer to make this configurable.
>
> On a side note, it is somewhat disingenuous to block charms in the
> store from pulling dependencies from untrusted sources at run time
> when we happily pull dependencies from untrusted sources at build
> time. I think the fix here is to do better at build time (Moving the
> interfaces web site to https: and ensuring clients use that address,
> only allowing https:, git+ssh: and other secure protocols for pulling
> branches, and checking GPG signatures of embedded wheels are the
> issues here I'm aware of)
>
> --
> Stuart Bishop 
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm Store policy updates and refinement for 2.0

[Stuart Bishop]

On 19 March 2016 at 02:58, Jorge O. Castro  wrote:

> Recommendations from everyone on what we should include here would be
> most welcome, specifically our recommendations around Windows charms
> is non-existent.

I think the acceptable software sources needs to be expanded.
Launchpad PPAs should be acceptable as signing keys are securely
retrieved when using 'add-apt-repository ppa:foo/bar'. Also, 3rd party
apt repositories should be acceptable if the signing key is embedded
in the charm (PyPi could be checked similarly, but it seems rare to
find signed packages there).

In addition, any software sources not in the main Ubuntu or CentOS
archives should be listed in configuration items that can be
overridden rather than hard coded in the charm, or else the charm is
useless in network restricted environments (and yes, migrating to
resources may be a better user experience in many cases).

As examples, the PostgreSQL charm pulls non-default packages from the
upstream PostgreSQL apt repository (PGDG, which is the source which
flows to Debian and Ubuntu). The Cassandra charm pulls a required
driver from a PPA I control. It also installs packages from either the
Apache apt repository or the DataStax apt repository. Cassandra is not
available in the Debian or Ubuntu main archives, probably as it
required the Oracle JVM. Both charms use the
install_sources/install_keys config items parsed by charm-helpers and
the apt layer to make this configurable.

On a side note, it is somewhat disingenuous to block charms in the
store from pulling dependencies from untrusted sources at run time
when we happily pull dependencies from untrusted sources at build
time. I think the fix here is to do better at build time (Moving the
interfaces web site to https: and ensuring clients use that address,
only allowing https:, git+ssh: and other secure protocols for pulling
branches, and checking GPG signatures of embedded wheels are the
issues here I'm aware of)

-- 
Stuart Bishop 

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm Store policy updates and refinement for 2.0

[Antonio Rosales]

On Fri, Mar 18, 2016 at 1:28 PM, Charles Butler
 wrote:
> Big +1 to the categories
>
> What i'd like to see is the policy document move to strong language where we
> can build tooling around the automated checking of policy.
>
> Refactoring the MUSTS and SHOULDS give us a strong lead on that language.
> MUST == has to be satisfied
> SHOULD = area for improvement - (proper use of warn, which wont fail charm
> proof for a change)
>
> eg: We require OSI approved licenses, we can scan the copyright file for
> that with a license bot to find the lingo of approved licenses, otherwise it
> flags for human review. It may be an acceptable license we're not aware of.
>
> The more points in policy we can convert to strongly typed lingo, the more
> targeted and automated we can make portions of policy review, and also
> scopes the mission of ~charmers. A limited resource who is responsible for
> enforcing this document.

Strongly agree. The more binary we make the policy the easier it is
for folks to understand what the need to accomplish to get it
recommended. Plus, it mapd lingo, the more
> targeted and automated we can make portions of policy review, and also
> scopes the mission of ~charmers. A limited resource who is responsible for
> enforcing this document. os well to review queue check list.  Thus in that 
> vein
- Consider dropping the "Must" from the "General Guidelines" as some
point are very objective there.
- Reference Fuctional Testing in the Deployment section
- We should be explicit for the items typically called out that are
needed in readmes, ie steps to smoke test, config options to set,
relations to set, upstream links, and how to get the software

I'll propose a branch with above suggestions, but thought I should
also mail here.

-thanks,
Antonio


>
>
>
> On Fri, Mar 18, 2016 at 11:59 AM Jorge O. Castro  wrote:
>>
>> Hello everyone,
>>
>> With 2.0 around the corner we decided to spend some time cleaning up
>> the page everyone loves to hate, the Juju Charm Store policy:
>>
>> https://jujucharms.com/docs/1.25/authors-charm-policy
>>
>> and here is what I would like to propose:
>>
>>
>> https://github.com/castrojo/docs/blob/master/src/en/authors-charm-policy.md
>>
>> I've done a few things here:
>>
>> - I've separated it from one huge paragraph to sections, General,
>> Testing and Quality requirements, Metadata requirements, and Security
>> requirements.
>> - I've split out things that a charm/bundle MUST do and what it SHOULD
>> do in each section to make it clearer on what is a hard requirement
>> and what is a recommendation.
>> - I've removed most of the Ubuntu-specific jargon and generalized it
>> to include other OSes such as CentOS.
>> - Made documenting interfaces and external dependencies a requirement.
>>
>> There are also some new policies that we need ack from ~charmers in
>> order to implement. Specifically we've made the testing and quality
>> requirements explicit. I've also added a requirement of using Juju
>> Resources (which appears to be undocumented?) for payloads.
>>
>> Recommendations from everyone on what we should include here would be
>> most welcome, specifically our recommendations around Windows charms
>> is non-existent.
>>
>>
>> --
>> Jorge Castro
>> Canonical Ltd.
>> http://jujucharms.com/ - The fastest way to model your service
>>
>> --
>> Juju mailing list
>> Juju@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>



-- 
Antonio Rosales
Ecosystem Engineering
Canonical

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm Store policy updates and refinement for 2.0

[Tom Barber]

Bundles must only use charms which are already in the store, they cannot
reference charms in personal namespaces.

I assume this apples to only bundles that get promoted to recommended
otherwise how would you enforce it?

--

Director Meteorite.bi - Saiku Analytics Founder
Tel: +44(0)5603641316

(Thanks to the Saiku community we reached our Kickstart

goal, but you can always help by sponsoring the project
)

On 18 March 2016 at 15:58, Jorge O. Castro  wrote:

> Hello everyone,
>
> With 2.0 around the corner we decided to spend some time cleaning up
> the page everyone loves to hate, the Juju Charm Store policy:
>
> https://jujucharms.com/docs/1.25/authors-charm-policy
>
> and here is what I would like to propose:
>
> https://github.com/castrojo/docs/blob/master/src/en/authors-charm-policy.md
>
> I've done a few things here:
>
> - I've separated it from one huge paragraph to sections, General,
> Testing and Quality requirements, Metadata requirements, and Security
> requirements.
> - I've split out things that a charm/bundle MUST do and what it SHOULD
> do in each section to make it clearer on what is a hard requirement
> and what is a recommendation.
> - I've removed most of the Ubuntu-specific jargon and generalized it
> to include other OSes such as CentOS.
> - Made documenting interfaces and external dependencies a requirement.
>
> There are also some new policies that we need ack from ~charmers in
> order to implement. Specifically we've made the testing and quality
> requirements explicit. I've also added a requirement of using Juju
> Resources (which appears to be undocumented?) for payloads.
>
> Recommendations from everyone on what we should include here would be
> most welcome, specifically our recommendations around Windows charms
> is non-existent.
>
>
> --
> Jorge Castro
> Canonical Ltd.
> http://jujucharms.com/ - The fastest way to model your service
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm Store policy updates and refinement for 2.0

[Jorge O. Castro]

On Fri, Mar 18, 2016 at 12:11 PM, Tom Barber  wrote:
> I assume this apples to only bundles that get promoted to recommended
> otherwise how would you enforce it?

Yes, to be clear these policies only apply to things that are in the
recommended/promulgated space. So jujucharms.com/haproxy, not
jujucharms.com/u/jorge/haproxy

As always, everyone is free to do what they like in namespaces.

-- 
Jorge Castro
Canonical Ltd.
http://jujucharms.com/ - The fastest way to model your service

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Charm Store policy updates and refinement for 2.0

[Jorge O. Castro]

Hello everyone,

With 2.0 around the corner we decided to spend some time cleaning up
the page everyone loves to hate, the Juju Charm Store policy:

https://jujucharms.com/docs/1.25/authors-charm-policy

and here is what I would like to propose:

https://github.com/castrojo/docs/blob/master/src/en/authors-charm-policy.md

I've done a few things here:

- I've separated it from one huge paragraph to sections, General,
Testing and Quality requirements, Metadata requirements, and Security
requirements.
- I've split out things that a charm/bundle MUST do and what it SHOULD
do in each section to make it clearer on what is a hard requirement
and what is a recommendation.
- I've removed most of the Ubuntu-specific jargon and generalized it
to include other OSes such as CentOS.
- Made documenting interfaces and external dependencies a requirement.

There are also some new policies that we need ack from ~charmers in
order to implement. Specifically we've made the testing and quality
requirements explicit. I've also added a requirement of using Juju
Resources (which appears to be undocumented?) for payloads.

Recommendations from everyone on what we should include here would be
most welcome, specifically our recommendations around Windows charms
is non-existent.


-- 
Jorge Castro
Canonical Ltd.
http://jujucharms.com/ - The fastest way to model your service

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm Store policy updates and refinement for 2.0

[Charles Butler]

Big +1 to the categories

What i'd like to see is the policy document move to strong language where
we can build tooling around the automated checking of policy.

Refactoring the MUSTS and SHOULDS give us a strong lead on that language.
MUST == has to be satisfied
SHOULD = area for improvement - (proper use of warn, which wont fail charm
proof for a change)

eg: We require OSI approved licenses, we can scan the copyright file for
that with a license bot to find the lingo of approved licenses, otherwise
it flags for human review. It may be an acceptable license we're not aware
of.

The more points in policy we can convert to strongly typed lingo, the more
targeted and automated we can make portions of policy review, and also
scopes the mission of ~charmers. A limited resource who is responsible for
enforcing this document.



On Fri, Mar 18, 2016 at 11:59 AM Jorge O. Castro  wrote:

> Hello everyone,
>
> With 2.0 around the corner we decided to spend some time cleaning up
> the page everyone loves to hate, the Juju Charm Store policy:
>
> https://jujucharms.com/docs/1.25/authors-charm-policy
>
> and here is what I would like to propose:
>
> https://github.com/castrojo/docs/blob/master/src/en/authors-charm-policy.md
>
> I've done a few things here:
>
> - I've separated it from one huge paragraph to sections, General,
> Testing and Quality requirements, Metadata requirements, and Security
> requirements.
> - I've split out things that a charm/bundle MUST do and what it SHOULD
> do in each section to make it clearer on what is a hard requirement
> and what is a recommendation.
> - I've removed most of the Ubuntu-specific jargon and generalized it
> to include other OSes such as CentOS.
> - Made documenting interfaces and external dependencies a requirement.
>
> There are also some new policies that we need ack from ~charmers in
> order to implement. Specifically we've made the testing and quality
> requirements explicit. I've also added a requirement of using Juju
> Resources (which appears to be undocumented?) for payloads.
>
> Recommendations from everyone on what we should include here would be
> most welcome, specifically our recommendations around Windows charms
> is non-existent.
>
>
> --
> Jorge Castro
> Canonical Ltd.
> http://jujucharms.com/ - The fastest way to model your service
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm Store policy updates and refinement for 2.0

[Tom Barber]

Cool!

--

Director Meteorite.bi - Saiku Analytics Founder
Tel: +44(0)5603641316

(Thanks to the Saiku community we reached our Kickstart

goal, but you can always help by sponsoring the project
)

On 18 March 2016 at 16:15, Jorge O. Castro  wrote:

> On Fri, Mar 18, 2016 at 12:11 PM, Tom Barber 
> wrote:
> > I assume this apples to only bundles that get promoted to recommended
> > otherwise how would you enforce it?
>
> Yes, to be clear these policies only apply to things that are in the
> recommended/promulgated space. So jujucharms.com/haproxy, not
> jujucharms.com/u/jorge/haproxy
>
> As always, everyone is free to do what they like in namespaces.
>
> --
> Jorge Castro
> Canonical Ltd.
> http://jujucharms.com/ - The fastest way to model your service
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju