Re: bind mount support for local provider
On Sun, Jan 18, 2015 at 8:01 PM, Andrew Wilkins andrew.wilk...@canonical.com wrote: On Sat, Jan 17, 2015 at 11:47 PM, Corey Bryant corey.bry...@canonical.com wrote: Excellent, thanks Ian. Do you have an estimate on when this will be available? Hi Corey, Not yet; we are working towards getting a single provider fully functional before delving too deeply into others. I'll ping you when we've got a better idea. Ok, thanks Andrew. Cheers, Andrew On Fri, Jan 16, 2015 at 10:42 PM, Ian Booth ian.bo...@canonical.com wrote: Hi Cory This is part of what we are planning to deliver for this cycle as part of the storage work. We also plan on being able to provide the container with access to block devices eg loopback, either in container's filesystem or on the host machine. On 17/01/15 02:11, Corey Bryant wrote: Hi all, Do there happen to be any plans for juju bind mount support for the local provider? For example: juju deploy mysql --bind /shared/mysql /shared which would bind mount the host /shared/mysql directory to /shared in the deployed container. -- Regards, Corey -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju -- Regards, Corey -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
[Review Queue] - BigData, squid-reverse-proxy
hdp-storm-zookeeper bundle: *pending* - The bundle did not pass proof as is - but given the age of the branch it looks like bitrot from recent changes in how the big data charms are organized. I submitted a patch inline with the review to fix up and pass - once submitted will promote to the store post haste. https://bugs.launchpad.net/charms/+bug/1373006 squid-reverse-proxy: *merged* - Add's an NRPE fixup for changing ports and monitoring with NRPE. The CI test failure and dependency management is being tracked in a different issue: https://code.launchpad.net/~jacekn/charms/trusty/squid-reverseproxy/squid-reverseproxy-nrpe-fix/+merge/245312 https://bugs.launchpad.net/charms/+source/squid-reverseproxy/+bug/1401323 -- All the best, Charles Butler charles.but...@canonical.com - Juju Charmer Come see the future of datacenter orchestration: http://jujucharms.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
[Review Queue] wildfly charms
I spent some time in the review queue today on the wildfly-ha-master and wildfly-ha-slave charms. https://bugs.launchpad.net/charms/+bug/1399239 https://bugs.launchpad.net/charms/+bug/1399228 Both are now in the recommended section of the charm store. Many thanks to *Saurabh Kumar* from Techblue for working with the review comments and landing the charms! - Matt Bruzek matthew.bru...@canonical.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: An Open Question: Charm Dependency Management
Lots of good feedback here with regard to how we want to manage it. I'm personally +1 on having a Makefile in my charm that handles these things, but was unsure if this was our defacto recommended path to completion. Thanks for such an active and rapid response on the thread. On Tue, Jan 20, 2015 at 1:20 PM, Marco Ceppi ma...@ondina.co wrote: Well there are two notions of testing, unit_test and functional_test one is largely more expensive than the other. Outside of that test-depends is a good one. Whatever it is we should identify those so we can make sure bundletester is updated to sniff these targets out (if this is the route we chose). On Tue Jan 20 2015 at 1:18:05 PM David Britton david.brit...@canonical.com wrote: On Tue, Jan 20, 2015 at 05:58:24PM +, Marco Ceppi wrote: I don't see how a Makefile in a charm doesn't resolve this issue. +1 on some standard published Makefile targets. We already have some that are highly recommended: - test - lint Maybe: - test-depends or depends # to install/update dependencies needed for testing Are there others that are needed/missing or that I forgot we already have as standard? -- David Britton david.brit...@canonical.com -- All the best, Charles Butler charles.but...@canonical.com - Juju Charmer Come see the future of datacenter orchestration: http://jujucharms.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: An Open Question: Charm Dependency Management
Well there are two notions of testing, unit_test and functional_test one is largely more expensive than the other. Outside of that test-depends is a good one. Whatever it is we should identify those so we can make sure bundletester is updated to sniff these targets out (if this is the route we chose). On Tue Jan 20 2015 at 1:18:05 PM David Britton david.brit...@canonical.com wrote: On Tue, Jan 20, 2015 at 05:58:24PM +, Marco Ceppi wrote: I don't see how a Makefile in a charm doesn't resolve this issue. +1 on some standard published Makefile targets. We already have some that are highly recommended: - test - lint Maybe: - test-depends or depends # to install/update dependencies needed for testing Are there others that are needed/missing or that I forgot we already have as standard? -- David Britton david.brit...@canonical.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: An Open Question: Charm Dependency Management
A Makefile that has a target to install dependencies suffices, but I think suggested conventions are still helpful. For example, in my case I prefer python virtualenvs over system packages. Once you establish some conventions (perhaps even just documentation conventions), A charmer can document the approach taken so that devs will be aware and can take measures. e.g. if I want to make a MR to a charm that needs system packages, I will know to do my development and testing inside of a container. On Tue, Jan 20, 2015 at 11:58 AM, Marco Ceppi ma...@ondina.co wrote: I don't see how a Makefile in a charm doesn't resolve this issue. As long as we define what targets a user should create in the Makefile, the Makefile can then do everything required: create a virtualenv and install deps, install ruby and execute bundler, npm for node, etc. Since charms are so varies in how they're written and what language they're implemented in this seems to make the most sense. Marco On Tue Jan 20 2015 at 12:37:00 PM Charles Butler charles.but...@canonical.com wrote: Greetings, If you work on charms in any capacity: this affects you, and I would love to have your feedback. While working the review queue I've encountered a few charm merges that are failing our testing infrastructure due to missing dependencies. This also has implications that reach beyond our testing infrastructure: Anyone that is submitting a new charm, Patches being accepted to existing charms, and even our documentation efforts over on the Charm Authorship Docs. There seems to be a bit of confusion about what the recommended process is to ensure all our dependencies are encapsulated in the charm. Having spoken with various members of the development community, I feel like our dependency encapsulation process for charms is still very much a grey area with several different ideologies on how to manage them, thus far I've seen: - A Virtualenv per project to manage python dependencies - make targets that sudo install packages on the host system - Zero Dependency management This is indeed a difficult topic to approach and digest as we're supporting basically everything out there. Not everyone uses the same tool-chain to accomplish the goal of dependency isolation - and several different Config Management tools have a different approach to this that assume it is installed on the host. this leaves a broken experience for: - new charm authors - CI - Anyone that comes along and runs the test targets or bundletester If we're going to ask our community to contribute to charms, is it fair to make them run down dependencies that may or may not exist on their host? It seems like we can do a better job of highlighting this, and providing a quick start style development target to install these pre-deps which would satisfy CI and Development. With this being the proposal, I follow it with some questions: - How have *we* solved this problem in other areas of our ecosystem? - How have other platforms solved this problem? - Can we emulate / improve on that pattern? - If a package management solution exists for a technology (eg: virtualenv for python, bundler for ruby, npm for node, berkshelf for chef) - can we adopt these and get started by templating in the dependency management into the charm generator template? I'm hoping this email is seen more as a conversation starter vs me being pedantic - I'm more concerned with getting the right set of information to our users/community than I am in solving some meta problem of packaging charms and their Development Environments. -- All the best, Charles Butler charles.but...@canonical.com - Juju Charmer Come see the future of datacenter orchestration: http://jujucharms.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/ mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju -- she...@pobox.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: An Open Question: Charm Dependency Management
On Tue, Jan 20, 2015 at 05:58:24PM +, Marco Ceppi wrote: I don't see how a Makefile in a charm doesn't resolve this issue. +1 on some standard published Makefile targets. We already have some that are highly recommended: - test - lint Maybe: - test-depends or depends # to install/update dependencies needed for testing Are there others that are needed/missing or that I forgot we already have as standard? -- David Britton david.brit...@canonical.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
An Open Question: Charm Dependency Management
Greetings, If you work on charms in any capacity: this affects you, and I would love to have your feedback. While working the review queue I've encountered a few charm merges that are failing our testing infrastructure due to missing dependencies. This also has implications that reach beyond our testing infrastructure: Anyone that is submitting a new charm, Patches being accepted to existing charms, and even our documentation efforts over on the Charm Authorship Docs. There seems to be a bit of confusion about what the recommended process is to ensure all our dependencies are encapsulated in the charm. Having spoken with various members of the development community, I feel like our dependency encapsulation process for charms is still very much a grey area with several different ideologies on how to manage them, thus far I've seen: - A Virtualenv per project to manage python dependencies - make targets that sudo install packages on the host system - Zero Dependency management This is indeed a difficult topic to approach and digest as we're supporting basically everything out there. Not everyone uses the same tool-chain to accomplish the goal of dependency isolation - and several different Config Management tools have a different approach to this that assume it is installed on the host. this leaves a broken experience for: - new charm authors - CI - Anyone that comes along and runs the test targets or bundletester If we're going to ask our community to contribute to charms, is it fair to make them run down dependencies that may or may not exist on their host? It seems like we can do a better job of highlighting this, and providing a quick start style development target to install these pre-deps which would satisfy CI and Development. With this being the proposal, I follow it with some questions: - How have *we* solved this problem in other areas of our ecosystem? - How have other platforms solved this problem? - Can we emulate / improve on that pattern? - If a package management solution exists for a technology (eg: virtualenv for python, bundler for ruby, npm for node, berkshelf for chef) - can we adopt these and get started by templating in the dependency management into the charm generator template? I'm hoping this email is seen more as a conversation starter vs me being pedantic - I'm more concerned with getting the right set of information to our users/community than I am in solving some meta problem of packaging charms and their Development Environments. -- All the best, Charles Butler charles.but...@canonical.com - Juju Charmer Come see the future of datacenter orchestration: http://jujucharms.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
[Review Queue]
Reviewed the python rewrite of the jenkins charm. Generally looks good but needs some minor fixes for testing and an intermittent race condition. https://code.launchpad.net/~hopem/charms/trusty/jenkins/python-redux/+merge/245769 -w -- --- D. Whit Morriss Developer, Juju Ecosystem Canonical USA -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: An Open Question: Charm Dependency Management
I don't see how a Makefile in a charm doesn't resolve this issue. As long as we define what targets a user should create in the Makefile, the Makefile can then do everything required: create a virtualenv and install deps, install ruby and execute bundler, npm for node, etc. Since charms are so varies in how they're written and what language they're implemented in this seems to make the most sense. Marco On Tue Jan 20 2015 at 12:37:00 PM Charles Butler charles.but...@canonical.com wrote: Greetings, If you work on charms in any capacity: this affects you, and I would love to have your feedback. While working the review queue I've encountered a few charm merges that are failing our testing infrastructure due to missing dependencies. This also has implications that reach beyond our testing infrastructure: Anyone that is submitting a new charm, Patches being accepted to existing charms, and even our documentation efforts over on the Charm Authorship Docs. There seems to be a bit of confusion about what the recommended process is to ensure all our dependencies are encapsulated in the charm. Having spoken with various members of the development community, I feel like our dependency encapsulation process for charms is still very much a grey area with several different ideologies on how to manage them, thus far I've seen: - A Virtualenv per project to manage python dependencies - make targets that sudo install packages on the host system - Zero Dependency management This is indeed a difficult topic to approach and digest as we're supporting basically everything out there. Not everyone uses the same tool-chain to accomplish the goal of dependency isolation - and several different Config Management tools have a different approach to this that assume it is installed on the host. this leaves a broken experience for: - new charm authors - CI - Anyone that comes along and runs the test targets or bundletester If we're going to ask our community to contribute to charms, is it fair to make them run down dependencies that may or may not exist on their host? It seems like we can do a better job of highlighting this, and providing a quick start style development target to install these pre-deps which would satisfy CI and Development. With this being the proposal, I follow it with some questions: - How have *we* solved this problem in other areas of our ecosystem? - How have other platforms solved this problem? - Can we emulate / improve on that pattern? - If a package management solution exists for a technology (eg: virtualenv for python, bundler for ruby, npm for node, berkshelf for chef) - can we adopt these and get started by templating in the dependency management into the charm generator template? I'm hoping this email is seen more as a conversation starter vs me being pedantic - I'm more concerned with getting the right set of information to our users/community than I am in solving some meta problem of packaging charms and their Development Environments. -- All the best, Charles Butler charles.but...@canonical.com - Juju Charmer Come see the future of datacenter orchestration: http://jujucharms.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/ mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
juju devel 1.21-rc1 is released
juju-core 1.21-rc1 A new development release of Juju, juju-core 1.21-rc1, is now available. This release replaces 1.21-beta5. Getting Juju juju-core 1.21-rc1 is available for utopic and backported to earlier series in the following PPA: https://launchpad.net/~juju/+archive/devel The devel packages in this archive use the devel simple-streams. You must configure the 'agent-stream' option in your environments.yaml to use the matching juju agents. agent-stream: devel Upgrading from stable releases to development releases is not supported. You can upgrade test environments to development releases to test new features and fixes, but it is not advised to upgrade production environments to 1.21-rc1. Notable Changes This releases addresses stability and performance issues. Resolved issues * Error upgrade in progress - juju functionality is limited Lp 1411502 Finally We encourage everyone to subscribe the mailing list at juju-...@lists.canonical.com, or join us on #juju-dev on freenode. -- Curtis Hovey Canonical Cloud Development and Operations http://launchpad.net/~sinzui -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: An Open Question: Charm Dependency Management
We are striving to test charms against all substrates and architectures juju supports; and are nearing completion for that goal. Manual provider is not currently tested AFAIK but will likely be in the future. On Tue Jan 20 2015 at 7:41:27 PM Andrew Wilkins andrew.wilk...@canonical.com wrote: On Wed, Jan 21, 2015 at 1:36 AM, Charles Butler charles.but...@canonical.com wrote: Greetings, If you work on charms in any capacity: this affects you, and I would love to have your feedback. While working the review queue I've encountered a few charm merges that are failing our testing infrastructure due to missing dependencies. This also has implications that reach beyond our testing infrastructure: Anyone that is submitting a new charm, Patches being accepted to existing charms, and even our documentation efforts over on the Charm Authorship Docs. On this topic, does the automated testing cover testing charms with the manual provider? There have been charms that failed (and were fixed) because they did not explicitly install dependencies. Those charms were using packages that are automatically installed as a result of cloud-init being on the machine; since the manual provider does not use cloud-init, those packages don't get automatically installed and the charms fail. IIRC, python-yaml fits into this category. There seems to be a bit of confusion about what the recommended process is to ensure all our dependencies are encapsulated in the charm. Having spoken with various members of the development community, I feel like our dependency encapsulation process for charms is still very much a grey area with several different ideologies on how to manage them, thus far I've seen: - A Virtualenv per project to manage python dependencies - make targets that sudo install packages on the host system - Zero Dependency management This is indeed a difficult topic to approach and digest as we're supporting basically everything out there. Not everyone uses the same tool-chain to accomplish the goal of dependency isolation - and several different Config Management tools have a different approach to this that assume it is installed on the host. this leaves a broken experience for: - new charm authors - CI - Anyone that comes along and runs the test targets or bundletester If we're going to ask our community to contribute to charms, is it fair to make them run down dependencies that may or may not exist on their host? It seems like we can do a better job of highlighting this, and providing a quick start style development target to install these pre-deps which would satisfy CI and Development. With this being the proposal, I follow it with some questions: - How have *we* solved this problem in other areas of our ecosystem? - How have other platforms solved this problem? - Can we emulate / improve on that pattern? - If a package management solution exists for a technology (eg: virtualenv for python, bundler for ruby, npm for node, berkshelf for chef) - can we adopt these and get started by templating in the dependency management into the charm generator template? I'm hoping this email is seen more as a conversation starter vs me being pedantic - I'm more concerned with getting the right set of information to our users/community than I am in solving some meta problem of packaging charms and their Development Environments. -- All the best, Charles Butler charles.but...@canonical.com - Juju Charmer Come see the future of datacenter orchestration: http://jujucharms.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/ mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: An Open Question: Charm Dependency Management
On Wed, Jan 21, 2015 at 8:24 AM, Michael Nelson michael.nel...@canonical.com wrote: For eg., if it was a python charm that needed some python packages: .virtualenv: virtualenv .virtualenv .virtualenv/bin/pip install -r test_requirements.txt test: .virtualenv ./virtualenv/bin/python ./unit-tests My python charm makefiles looks like this too, but one minor improvement is to suppress output from pip with '-q' which makes running dependant targets like 'test' and 'lint' much less noisy, e.g. .virtualenv: @virtualenv .virtualenv @.virtualenv/bin/pip install -q -r requirements.txt --upgrade -- Kit Randel Canonical - Ubuntu Engineering - Continuous Integration Team -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: An Open Question: Charm Dependency Management
On Wed, Jan 21, 2015 at 5:17 AM, David Britton david.brit...@canonical.com wrote: On Tue, Jan 20, 2015 at 05:58:24PM +, Marco Ceppi wrote: I don't see how a Makefile in a charm doesn't resolve this issue. +1 on some standard published Makefile targets. We already have some that are highly recommended: - test - lint +1 (and -1000 on any Makefile targets which run as sudo) Maybe: - test-depends or depends # to install/update dependencies needed for testing It doesn't hurt to have an extra target like test-depends, but it should be enough to just have the test target depend on some other target that's required for whatever technology the author is using. For eg., if it was a python charm that needed some python packages: .virtualenv: virtualenv .virtualenv .virtualenv/bin/pip install -r test_requirements.txt test: .virtualenv ./virtualenv/bin/python ./unit-tests Yep, you could make that DRYer, but it might be less readable. -Michael Are there others that are needed/missing or that I forgot we already have as standard? -- David Britton david.brit...@canonical.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
