Re: Leadership Election Tools

2016-12-14 Thread Stuart Bishop 
On 14 December 2016 at 00:39, Matthew Williams <
matthew.willi...@canonical.com> wrote:

> Hey Folks,
>
> Let's say I'm a charm author that wants to test leadership election in my
> charm. Are there any tools available that will let me force leadership
> election in juju so that I can test how my charm handles it? I was looking
> at the docs here: https://jujucharms.com/docs/stable/developer-leadership
> but couldn't see anything
>

I don't think there is any supported way of doing this.

If you don't mind an unsupported hack though, use 'juju ssh' to shut down
the unit's jujud, wait 30 seconds for the lease to expire, and you should
have a new leader. 'juju ssh' again to restart the jujud, 'juju wait' for
the hooks to clear, and failover is done. 'juju run' will hang if you use
it to shutdown jujud, so don't do that.

juju ssh ubuntu/0 'sudo systemctl stop jujud-unit-ubuntu-0.service'
sleep 30
juju ssh ubuntu/0 'sudo systemctl stop jujud-unit-ubuntu-0.service'
juju wait

Ideally, you may be able to structure things so that it doesn't matter
which unit is leader. If all state relating to leadership decisions is
stored in the leadership settings, and if you avoid using @hook, then it
doesn't matter which unit makes the decisions. Worst case is that *no* unit
is leader when hooks are run, and decisions get deferred until
leader-elected runs.

(Interesting race condition for the day: It is possible for all units in a
service to run their upgrade-charm hook and for none of them to be leader
at the time, so @hook('upgrade-charm') code guarded by is-leader may never
run. And reactive handlers have no concept of priority and might kick in
rather late for upgrade steps, requiring more creative use of reactive
states to guard 'new' code from running too soon. Not specific to
upgrade-charm hooks either, so avoid using @hook and leadership together)


-- 
Stuart Bishop 
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Leadership Election Tools

2016-12-14 Thread Cory Johns 
I think that's another nail in the coffin for @hook.  You could work around
it by having your @hook('upgrade-charm') handler unconditionally set a
state (e.g., 'upgrade-charm'), and then have the state handler do the
gating on leadership.  This would also allow you to use the decorators for
gating on leadership.is_leader, rather than using the is_leader() function.

I definitely want to move toward having the reactive framework itself
manage states like upgrade-charm, and {relation_name}.joined so that we can
remove all uses of @hook.

On Wed, Dec 14, 2016 at 6:00 AM, Stuart Bishop 
wrote:

>
>
> On 14 December 2016 at 00:39, Matthew Williams <
> matthew.willi...@canonical.com> wrote:
>
>> Hey Folks,
>>
>> Let's say I'm a charm author that wants to test leadership election in my
>> charm. Are there any tools available that will let me force leadership
>> election in juju so that I can test how my charm handles it? I was looking
>> at the docs here: https://jujucharms.com/docs/stable/developer-leadership
>> but couldn't see anything
>>
>
> I don't think there is any supported way of doing this.
>
> If you don't mind an unsupported hack though, use 'juju ssh' to shut down
> the unit's jujud, wait 30 seconds for the lease to expire, and you should
> have a new leader. 'juju ssh' again to restart the jujud, 'juju wait' for
> the hooks to clear, and failover is done. 'juju run' will hang if you use
> it to shutdown jujud, so don't do that.
>
> juju ssh ubuntu/0 'sudo systemctl stop jujud-unit-ubuntu-0.service'
> sleep 30
> juju ssh ubuntu/0 'sudo systemctl stop jujud-unit-ubuntu-0.service'
> juju wait
>
> Ideally, you may be able to structure things so that it doesn't matter
> which unit is leader. If all state relating to leadership decisions is
> stored in the leadership settings, and if you avoid using @hook, then it
> doesn't matter which unit makes the decisions. Worst case is that *no* unit
> is leader when hooks are run, and decisions get deferred until
> leader-elected runs.
>
> (Interesting race condition for the day: It is possible for all units in a
> service to run their upgrade-charm hook and for none of them to be leader
> at the time, so @hook('upgrade-charm') code guarded by is-leader may never
> run. And reactive handlers have no concept of priority and might kick in
> rather late for upgrade steps, requiring more creative use of reactive
> states to guard 'new' code from running too soon. Not specific to
> upgrade-charm hooks either, so avoid using @hook and leadership together)
>
>
> --
> Stuart Bishop 
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju-dev
>
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Representative tests failing

2016-12-14 Thread Nate Finch 
Seems like there's a likely out of disk space problem with the LXD and
windows machines that run representative tests.  For example:

http://juju-ci.vapour.ws/job/github-check-merge-juju/441/
http://juju-ci.vapour.ws/job/github-check-merge-juju/442/

Sounds like this happens often enough to be a problem.

Is this something we can automate by resetting to a snapshot or something
along those lines?
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju Show #2 Wed Dec 14th

2016-12-14 Thread Rick Harding 
Check out the recording of the show in case you missed it.

https://www.youtube.com/watch?v=FwLEMa7XE64

I want to thank everyone that joined us for the show this week. Some great
info on the new tools to aid in charm testing, reviewing, and an exciting
demo of the upcoming juju migrate feature in Juju 2.1.

With the holidays we'll be having our next Juju Show on January 4th and the
one after that will be the 18th. We'll have links and announcements out
after folks get back from the holidays.

On Tue, Dec 13, 2016 at 3:47 PM Rick Harding 
wrote:

> Come join us as we host the next Juju Show live stream tomorrow. We'll be
> going over the latest in community news, demoing the new developments in
> tools for charming, and getting a demonstration of the new model migration
> feature coming in Juju 2.1.
>
> When: Nov 30th, at 19:00 GMT, 2:00pm EST
> Where: https://www.youtube.com/watch?v=FwLEMa7XE64
> How to participate: Hang out in the #juju Freenode IRC channel
>
> Join us in the IRC channel to ask questions, get a link to the hangout to
> join the live stream, or just to listen in on the latest news.
>
> Make sure to subscribe to the Juju Youtube channel so you never miss any
> of the happenings in and around Juju!
>
> https://www.youtube.com/channel/UCSsoSZBAZ3Ivlbt_fxyjIkw
>
> See you there!
>
> Rick
> Juju Engineering
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju Show #2 Wed Dec 14th

2016-12-14 Thread Antonio Rosales 
On Tue, Dec 13, 2016 at 1:47 PM, Rick Harding
 wrote:
> Come join us as we host the next Juju Show live stream tomorrow. We'll be
> going over the latest in community news, demoing the new developments in
> tools for charming, and getting a demonstration of the new model migration
> feature coming in Juju 2.1.
>
> When: Nov 30th, at 19:00 GMT, 2:00pm EST
> Where: https://www.youtube.com/watch?v=FwLEMa7XE64
> How to participate: Hang out in the #juju Freenode IRC channel
>
> Join us in the IRC channel to ask questions, get a link to the hangout to
> join the live stream, or just to listen in on the latest news.
>
> Make sure to subscribe to the Juju Youtube channel so you never miss any of
> the happenings in and around Juju!
>
> https://www.youtube.com/channel/UCSsoSZBAZ3Ivlbt_fxyjIkw

Also if you want an easier link to remember for the Juju Channel try:
https://www.youtube.com/jujucharms

Also some folks have asked what hashtag to use for Juju. We have been
trying to use #jujucharms (https://twitter.com/hashtag/jujucharms).
So, if you have some interesting bits to tweet out please include the
#jujucharms hashtag.

-thanks,
Antonio

>
> See you there!
>
> Rick
> Juju Engineering
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

2016-12-14 Thread Marco Ceppi 
So, I wanted to circle back around to this thread. I think a lot of good
feedback has come from this, and we're looking into making the reactive
framework leaner and better for charm authors. However, I ran a few deploy
tests and have the following results:

15 Dec 2016 00:18:53Z workload waiting waiting for machine
15 Dec 2016 00:18:53Z juju-unit allocating
15 Dec 2016 00:20:13Z workload waiting installing agent
15 Dec 2016 00:20:13Z workload waiting agent initializing
15 Dec 2016 00:20:14Z workload maintenance installing charm software
15 Dec 2016 00:20:14Z juju-unit executing   running install hook
15 Dec 2016 00:20:46Z workload active ready
15 Dec 2016 00:20:46Z juju-unit executing   running leader-elected hook
15 Dec 2016 00:20:47Z juju-unit executing   running start hook

I did this a few more times on Amazon, and the results were almost
identical. We have 80 seconds from machine requested to booted in cloud.
Less than a second for agent to initialize and 32 seconds to go from
install hook running to the workload being ready and active. While I'm sure
we can slim that down 10-15 seconds by not installing build-essentials the
largest time suck is still the cloud bringing up the instance.

I plan on doing this across all the clouds I have access to, and track in a
spreadsheet. I'll share that sheet out in a bit.

Thanks,
Marco Ceppi

On Thu, Dec 1, 2016 at 5:00 AM Adam Collard 
wrote:

> On Thu, 1 Dec 2016 at 04:02 Nate Finch  wrote:
>
> On IRC, someone was lamenting the fact that the Ubuntu charm takes longer
> to deploy now, because it has been updated to exercise more of Juju's
> features.  My response was - just make a minimal charm, it's easy.  And
> then of course, I had to figure out how minimal you can get.  Here it is:
>
> It's just a directory with a metadata.yaml in it with these contents:
>
> name: min
> summary: nope
> description: nope
> series:
>   - xenial
>
> (obviously you can set the series to whatever you want)
> No other files or directories are needed.
>
>
> This is neat, but doesn't detract from the bloat in the ubuntu charm.
>
> IMHO the bloat in the ubuntu charm isn't from support for Juju features,
> but the switch to reactive plus conflicts in layer-base wanting to a)
> support lots of toolchains to allow layers above it to be slimmer and b) be
> a suitable base for "just deploy me" ubuntu.
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

2016-12-14 Thread Tim Penhey 

Make sure you also run on LXD with a decent delay to the APT archive.

That is what makes my local testing slow.

Tim

On 15/12/16 13:34, Marco Ceppi wrote:

So, I wanted to circle back around to this thread. I think a lot of good
feedback has come from this, and we're looking into making the reactive
framework leaner and better for charm authors. However, I ran a few
deploy tests and have the following results:

15 Dec 2016 00:18:53Zworkload waitingwaiting for machine
15 Dec 2016 00:18:53Zjuju-unitallocating
15 Dec 2016 00:20:13Zworkload waitinginstalling agent
15 Dec 2016 00:20:13Zworkload waitingagent initializing
15 Dec 2016 00:20:14Zworkload maintenanceinstalling charm software
15 Dec 2016 00:20:14Zjuju-unitexecuting  running install hook
15 Dec 2016 00:20:46Zworkload active ready
15 Dec 2016 00:20:46Zjuju-unitexecuting  running leader-elected hook
15 Dec 2016 00:20:47Zjuju-unitexecuting  running start hook

I did this a few more times on Amazon, and the results were almost
identical. We have 80 seconds from machine requested to booted in cloud.
Less than a second for agent to initialize and 32 seconds to go from
install hook running to the workload being ready and active. While I'm
sure we can slim that down 10-15 seconds by not installing
build-essentials the largest time suck is still the cloud bringing up
the instance.

I plan on doing this across all the clouds I have access to, and track
in a spreadsheet. I'll share that sheet out in a bit.

Thanks,
Marco Ceppi

On Thu, Dec 1, 2016 at 5:00 AM Adam Collard mailto:adam.coll...@canonical.com>> wrote:

On Thu, 1 Dec 2016 at 04:02 Nate Finch mailto:nate.fi...@canonical.com>> wrote:

On IRC, someone was lamenting the fact that the Ubuntu charm
takes longer to deploy now, because it has been updated to
exercise more of Juju's features.  My response was - just make a
minimal charm, it's easy.  And then of course, I had to figure
out how minimal you can get.  Here it is:

It's just a directory with a metadata.yaml in it with these
contents:

name: min
summary: nope
description: nope
series:
  - xenial

(obviously you can set the series to whatever you want)
No other files or directories are needed.


This is neat, but doesn't detract from the bloat in the ubuntu charm.

IMHO the bloat in the ubuntu charm isn't from support for Juju
features, but the switch to reactive plus conflicts in layer-base
wanting to a) support lots of toolchains to allow layers above it to
be slimmer and b) be a suitable base for "just deploy me" ubuntu.
--
Juju-dev mailing list
Juju-dev@lists.ubuntu.com 
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju-dev





--
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: A (Very) Minimal Charm

2016-12-14 Thread John Meinel 
Right, the issue for test/development iterations is that "machine requested
to booted in cloud" for LXD is a lot closer to 10s. Especially if you set
"enable-os-refresh-update: false" and "enable-os-upgrade: false", which are
also likely to be set in a testing environment.

John
=:->

On Thu, Dec 15, 2016 at 7:47 AM, Tim Penhey 
wrote:

> Make sure you also run on LXD with a decent delay to the APT archive.
>
> That is what makes my local testing slow.
>
> Tim
>
> On 15/12/16 13:34, Marco Ceppi wrote:
>
>> ...


> I did this a few more times on Amazon, and the results were almost
>> identical. We have 80 seconds from machine requested to booted in cloud.
>> Less than a second for agent to initialize and 32 seconds to go from
>> install hook running to the workload being ready and active. While I'm
>> sure we can slim that down 10-15 seconds by not installing
>> build-essentials the largest time suck is still the cloud bringing up
>> the instance.
>>
>> I plan on doing this across all the clouds I have access to, and track
>> in a spreadsheet. I'll share that sheet out in a bit.
>>
>> Thanks,
>> Marco Ceppi
>>
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev