Re: AZs and placement

[Martin Packman]

On 16/12/2016, Brent Clements  wrote:
> This is great info.
>
> I apologize for my ignorance but what would that, as placement for each
> unit, look like in a bundle?

For each application, it can look something like this:

  an-application:
charm: ./a-charm
num_units: 3
to:
  - zone=zone-a
  - zone=zone-a
  - zone=zone-a

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

AZs and placement

[Martin Packman]

There have been a couple of reports of Juju struggling with
availability zones recently. This isn't new code, it was introduced in
1.20, but with 2.0 it appears that we're not surfacing errors from
improper zone selection[1], which is a particular issue when we also
seem to not be selecting a valid zone[2]. This is seen with both
Openstack (datacenterd) and vSphere.

We don't have a global option to say only use a particular AZ, so the
only way at present to do this right is specify a zone placement for
every unit. I did find how to do that spelled out in the
documentation, though it's not completely intuitive:



$ juju deploy an-application -n 3 --to zone=$ZONE,zone=$ZONE,zone=$ZONE

Note that a placement directive is needed per unit.

Also, of note, we have only just fixed a bug[3] that prevented
combining constraints and placement directives in bundles. So,
currently care is required if creating a bundle that needs to specify
a zone for everything.

Martin


[1] "Openstack provider error not surfaced on bootstrap"

[2] "inconsistent results when fetching availability zones while
bootstrapping with vsphere as provider"

[3] "Juju ignores constraints set in the bundle and deploys KVMs with
default values"


-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Using juju ssh from windows clients

[Martin Packman]

Looking at juju ssh session from windows client bug:



The core issue is simple. We pass stdin straight over the ssh
connection, without coping with it being a windows text stream. So
lines are terminated with '\r\n' rather than just '\n', which then
confuses bash on the remote machine.

I have made a test build that basically wraps stdin with a translation
that strips \r characters before passing it through to the native go
ssh implementation. Obviously this doesn't give us full terminal
capabilities, but may be good enough. Are there any pitfalls I'm
missing here?

Background is juju traditionally, and still in practice on most nix
systems, will shell out to openssh client to connect to machines.
That's not an option on windows, but the fallback golang crypto
library can be used instead. That lacks a bunch of capabilities, and
is somewhat undertested.

What doesn't work:
* interactive sessions
* --proxy argument
* scp

What does work:
* single commands over ssh
* interactive sessions from cygwin

Amusingly, installing python, then doing `juju ssh $MACHINE python -i`
and ending every line with a comment sort of gets you a working remote
shell. Fixing this bug means the basic bash session is more usable
from the windows cmd prompt, though without all the nice bash features
exposed.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

New functional-endpoint-bindings tests are non-voting

[Martin Packman]

The new functional-endpoint-bindings tests on maas 2.0 and 2.1 are
non-voting while they gather a few runs and and the test environment
is tweaked.




Includes dynamic maas networking management that should make the tests
safe to run in parallel with other jobs, and form a basis for further
networking tests.

Careful watchers of the revisions test results will have noted some
fallout from the initial setup, all of that should now be resolved so
treat any new failures of maas substrate testing as significant.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

New functional-ssh-keys test is non-voting

[Martin Packman]

The new functional-ssh-keys test is non-voting while it gathers a few
runs and awaits a branch landing.



The test was written as part of fixing lp:1555727 as we lacked
functional test coverage for the key management in juju, and with some
pending updates works against 1.25 as well as master.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Uniter test failure from utils update

[Martin Packman]

Nicholas was struggling today to land his snapcraft pull request:



As a side effect this updates some dependencies, including a couple of
changes to juju/utils, which it turns out, causes
UniterSuite.TestRebootFromJujuRun to fail.

Here's a run of master, which passes:



Then the failure, just with utils updated to rev 8a9dea0:



I'm not clear on how the new exec behaviour is actually causing the
test failure:



So, any insights or suggestions welcome.


Note, the log also includes two other errors, but they appear in both
the passing and failing log, so are not actually affecting the test
outcome:

[LOG] 0:01.631 ERROR juju.apiserver Unable to prime
/tmp/check-1447535164350529303/5/logsink.log (proceeding anyway):
chown /tmp/check-1447535164350529303/5/logsink.log: operation not
permitted

This code should probably just not be being run under unit test, is
assuming root privs?

[LOG] 0:03.078 ERROR juju.worker.uniter.context updating agent status:
cannot set invalid status "rebooting"

The status setting code in uniter is a twisty confusion, I don't know
why this is happening or where the error is being swallowed.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Juju stable 1.25.6 is released

[Martin Packman]

# juju-core 1.25.6

A stable release of Juju, juju-core 1.25.6, is now available.
This release replaces version 1.25.5.


## Getting Juju

juju-core 1.25.6 is available for Xenial and backported to earlier
series in the following PPA:

https://launchpad.net/~juju/+archive/stable

Windows, Centos, and OS X users will find installers at:

https://launchpad.net/juju-core/+milestone/1.25.5


## Notable Changes

This releases addresses stability and performance issues reported by our
users over the past months. Thanks everyone for continuing to push the
limits and filing bugs.


## Resolved issues

  * Machine agent failed to register ip addresses, borks agent
Lp 1537585

  * "invalid entity name or password" error with valid credentials.
Lp 1514874

  * 'juju help' provider information is out of date
Lp 1519877

  * Cached local charms should be deleted when their service is
removed
Lp 1580418

  * Cannot obtain provisioning script
Lp 1559299

  * Uniter-hook-execution error prevents  "resolve" unit.
Lp 1486712

  * Goroutine panic launching container on xenial
Lp 1592210

  * Trusty juju 1.25.5 ha availability issues
Lp 1575448

  * Maas provider bridge script on trusty does not handle lacp bonds
Lp 1594855

  * Concurent map access in joyent
Lp 1554251

  * Juju run command failing
Lp 1598964


Finally

We encourage everyone to subscribe the mailing list at
juju-...@lists.ubuntu.com, or join us on #juju-dev on freenode.

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Juju stable 1.25.6 is released

[Martin Packman]

# juju-core 1.25.6

A stable release of Juju, juju-core 1.25.6, is now available.
This release replaces version 1.25.5.


## Getting Juju

juju-core 1.25.6 is available for Xenial and backported to earlier
series in the following PPA:

https://launchpad.net/~juju/+archive/stable

Windows, Centos, and OS X users will find installers at:

https://launchpad.net/juju-core/+milestone/1.25.5


## Notable Changes

This releases addresses stability and performance issues reported by our
users over the past months. Thanks everyone for continuing to push the
limits and filing bugs.


## Resolved issues

  * Machine agent failed to register ip addresses, borks agent
Lp 1537585

  * "invalid entity name or password" error with valid credentials.
Lp 1514874

  * 'juju help' provider information is out of date
Lp 1519877

  * Cached local charms should be deleted when their service is
removed
Lp 1580418

  * Cannot obtain provisioning script
Lp 1559299

  * Uniter-hook-execution error prevents  "resolve" unit.
Lp 1486712

  * Goroutine panic launching container on xenial
Lp 1592210

  * Trusty juju 1.25.5 ha availability issues
Lp 1575448

  * Maas provider bridge script on trusty does not handle lacp bonds
Lp 1594855

  * Concurent map access in joyent
Lp 1554251

  * Juju run command failing
Lp 1598964


Finally

We encourage everyone to subscribe the mailing list at
juju-dev@lists.ubuntu.com, or join us on #juju-dev on freenode.

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Juju stable 1.25.6 is proposed for release

[Martin Packman]

# juju-core 1.25.6

A new proposed stable release of Juju, juju-core 1.25.6, is now available.
This release may replace version 1.25.5 on Thursday July 21.


## Getting Juju

juju-core 1.25.6 is available for Xenial and backported to earlier
series in the following PPA:

https://launchpad.net/~juju/+archive/proposed

Windows, Centos, and OS X users will find installers at:

https://launchpad.net/juju-core/+milestone/1.25.6

Proposed releases use the "proposed" simple-streams. You must configure
the `agent-stream` option in your environments.yaml to use the matching
juju agents.


## Notable Changes

This releases addresses stability and performance issues reported by our
users over the past months. Thanks everyone for continuing to push the
limits and filing bugs.


## Resolved issues

  * Machine agent failed to register ip addresses, borks agent
Lp 1537585

  * "invalid entity name or password" error with valid credentials.
Lp 1514874

  * 'juju help' provider information is out of date
Lp 1519877

  * Cached local charms should be deleted when their service is
removed
Lp 1580418

  * Cannot obtain provisioning script
Lp 1559299

  * Uniter-hook-execution error prevents  "resolve" unit.
Lp 1486712

  * Goroutine panic launching container on xenial
Lp 1592210

  * Trusty juju 1.25.5 ha availability issues
Lp 1575448

  * Maas provider bridge script on trusty does not handle lacp bonds
Lp 1594855

  * Concurent map access in joyent
Lp 1554251

  * Juju run command failing
Lp 1598964


Finally

We encourage everyone to subscribe the mailing list at
juju-...@lists.ubuntu.com, or join us on #juju-dev on freenode.

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Juju stable 1.25.6 is proposed for release

[Martin Packman]

# juju-core 1.25.6

A new proposed stable release of Juju, juju-core 1.25.6, is now available.
This release may replace version 1.25.5 on Thursday July 21.


## Getting Juju

juju-core 1.25.6 is available for Xenial and backported to earlier
series in the following PPA:

https://launchpad.net/~juju/+archive/proposed

Windows, Centos, and OS X users will find installers at:

https://launchpad.net/juju-core/+milestone/1.25.6

Proposed releases use the "proposed" simple-streams. You must configure
the `agent-stream` option in your environments.yaml to use the matching
juju agents.


## Notable Changes

This releases addresses stability and performance issues reported by our
users over the past months. Thanks everyone for continuing to push the
limits and filing bugs.


## Resolved issues

  * Machine agent failed to register ip addresses, borks agent
Lp 1537585

  * "invalid entity name or password" error with valid credentials.
Lp 1514874

  * 'juju help' provider information is out of date
Lp 1519877

  * Cached local charms should be deleted when their service is
removed
Lp 1580418

  * Cannot obtain provisioning script
Lp 1559299

  * Uniter-hook-execution error prevents  "resolve" unit.
Lp 1486712

  * Goroutine panic launching container on xenial
Lp 1592210

  * Trusty juju 1.25.5 ha availability issues
Lp 1575448

  * Maas provider bridge script on trusty does not handle lacp bonds
Lp 1594855

  * Concurent map access in joyent
Lp 1554251

  * Juju run command failing
Lp 1598964


Finally

We encourage everyone to subscribe the mailing list at
juju-dev@lists.ubuntu.com, or join us on #juju-dev on freenode.

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Juju plugins should start juju-

[Martin Packman]

Landing in the next beta of Juju 2.0, plugins will not only have to
start 'juju-' but the next character must be [a-zA-Z]. For details
see:



If you've written your own juju plugin, make sure it's named something
that starts with a letter rather than a number or symbol.

Martin

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: The mean time for a CI run has risen to 33 minutes

[Martin Packman]

On 16/05/2016, David Cheney  wrote:
> This got significantly worse in the last 6 weeks. What happened ?

Either the juju tests are slower, or trusty on aws is slower. It's a
fresh cloud instance each run, and still trusty because we switched to
xenial and the lxd tests failed due to lack of isolation. Could change
back to xenial now Curtis added some lxd setup support the the
makefile I think, but that is unlikely to help speed at all.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

MAAS 2.0 in CI

[Martin Packman]

CI now has revision testing for MAAS 2.0 provider support, though
master does not yet bootstrap successfully in our environment.

End of last week start of this, Curtis and I updated finfolk and its
hosted 1.9 vmaas to xenial, and maas 2.0 (with some small adventures
along the way). Today I also switched from ppa:maas/next to
ppa:maas-maintainters/experimental3 for beta4+bzr4958 at request of
Dimiter. Please poke again if we need to pick up another new revision
for more maas fixes.

That gets us started on maas 2.0 as part of revision testing.
Currently it fails early in the bootstrap process, as our setup is a
little more complex than what's been validated with so far:

"boot resource 2.0 schema check failed: kflavor: expected string, got nothing"


I tried manually removing our centos images, which got a little further:

"filesystem 2.0 schema check failed: mount_point: expected string, got nothing"


For now, I've just put up one (non-voting) maas 2.0 job, but will fill
in the blanks with bundle testing and so on when the basics are
working:



The changes CI need to talk to maas 2.0 behind juju are here:



Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: LXD polish for xenial

[Martin Packman]

On 18/04/2016, Tycho Andersen  wrote:
>>
>> Unlike other providers, lxd exposes no way to use the daily images
>> instead of release images, so at present any machine using lxd
>> containers with juju for the first time will get the xenial beta2
>> image then upgrade basically every package. This issue goes away next
>> week, but gets in the way of testing before then.
>
> Yes, it does, jam added it. You can use the image-stream=daily option:
>
> juju set-model-config image-stream="daily"
> juju bootstrap --config image-stream=daily ...
>
> The daily streams haven't been regenerated since April 12 (another
> known issue that CPC is working on) so you'll still see some churn.

Sorry, poor wording on my part, read 'containers' instead of 'providers'.

The setup I was testing, using lxd containers as part of a bundle, not
using the lxd provider, does not work with image-stream=daily. The
release images are still selected for the lxd containers, though the
provider machines are brought up with daily images.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: LXD polish for xenial

[Martin Packman]

On 18/04/2016, Martin Packman <martin.pack...@canonical.com> wrote:
>
> "autopkgtest lxd provider tests fail for 2.0"
> <https://bugs.launchpad.net/ubuntu/+source/juju-core/+bug/1571082>
>
> So, at present we don't have confidence that the LXD provider will
> work, even with the manual configuration step, for users installing
> Xenial for the first time.

Great progress on the lxd provider side:

<http://autopkgtest.ubuntu.com/packages/j/juju-core/>

Thanks to help from everyone in the bug, we have a working autopkgtest
and juju 2.0 has migrated out of proposed into the main archive.

Along with that, Tycho has proposed a couple of branches aimed at
making the first use experience less painful.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

LXD polish for xenial

[Martin Packman]

With the LXD 2.0 release at the start of last week and the prospect of
some stability, I spent a good chunk of the week testing of Juju and
LXD.

What CI has been doing so far this cycle has been running our standard
deployment tests with the lxd provider on a pre-prepared machine with
a known-working package set. My two goals beyond this were:

- Validating the first install experience of the LXD provider on xenial
- Using lxd in place of lxc in containerised workloads across clouds

The conclusion being at present, we don't have an experience that
works for either of these.

For the lxd provider, I understand we're resigned to the user having
to manually configure a bridge for lxd before bootstrap can work.
Currently the documentation is confused as to what exactly the steps
are, the release notes refer to these two links:




But I think the latest advice is as committed with this change to our
documentation:



Note that just running dpkg-reconfigure is not enough, you have to
poke a service or run `lxc` afterwards or you get this error with
beta4:

$ juju bootstrap --config default-series=xenial lxd-test lxd
ERROR cannot find network interface "lxcbr0": route ip+net: no
such network interface
ERROR invalid config: route ip+net: no such network interface

That's probably the cause of the other confusion in the updated docs -
now we *do* want the bridge named lxdbr0 not lxcbr0. If the user
already has lxc setup in some fashion there's a different error from
juju telling them to run the dpkg-reconfigure command. That should
presumably always appear whenever there's no usable bridge.

This also presents a challenge for automated testing of the lxd
provider in a clean environment, dpkg-reconfigure isn't the nicest
thing to use non-interactively, and I can't find clear reference to
what the exact required pieces are for the juju provider.

As part of the juju 2.0 packaging for Ubuntu, we need an
autopackagetest that will run in a fresh xenial machine, so this
script is what I added to do the lxd configuration:



With the additional step afterwards to call `lxc finger` that works
(with caveats) for me. In the autopkgtest.ubuntu.com infrastructure
however it does not, and it has also failed in two different ways for
Steve Langasek and Martin Pitt:

"autopkgtest lxd provider tests fail for 2.0"


So, at present we don't have confidence that the LXD provider will
work, even with the manual configuration step, for users installing
Xenial for the first time.


When it comes to using lxd in clouds, as I understand it we've settled
on retaining the 'lxc' and 'lxd' name distinction in 2.0 - which does
mean bundles have to be manually changed at present to start using
lxd. Most of the CI bundle testing is using real bundles out of the
store, which all still say 'lxc' and therefore don't exercise the lxd
container code at all.

We do have the container networking test which uses 'juju add-machine
... lxd:0' - and fails due to the networking setup:

"container networking lxd 'Missing parent for bridged type nic'"


That is probably less interesting than the default behaviour without
the feature flag.

As a separate test, I updated one of our simple bundles just to say
'lxd' in two places where it had 'lxc' for a service before. The
deployment timed out after 24 minutes, where the normal test with lxc
takes 12 minutes.

The reason for that turns out to be pretty simple. Looking back at the
lxd provider test, it hung for over 20 minutes just updating packages
when setting up the first container:

In container /var/log/apt/history.log
Start-Date: 2016-04-15  22:11:16
...
End-Date: 2016-04-15  22:33:03

Unlike other providers, lxd exposes no way to use the daily images
instead of release images, so at present any machine using lxd
containers with juju for the first time will get the xenial beta2
image then upgrade basically every package. This issue goes away next
week, but gets in the way of testing before then.

In a related note, the lxc container handling in juju manages images
on the state server, but from what I see of the lxd code, each
deployed machine will fetch images from cloud-images.ubuntu.com and
keep a separate set of images. That makes the above problem much worse
for any bundle with multiple machines that use containers.

Finally, we'll need to update the log gathering code in CI to know how
to look inside lxd containers. At the moment, only the machine log
seems to be linked into the /var/log/lxd/ directory, so the cloud-init
logs and other pieces are currently missing. It does seem 

Re: New juju in ubuntu

[Martin Packman]

On 06/04/2016, Stuart Bishop  wrote:
>
> How do our plugins know what version of juju is in play? Can they
> assume that the 'juju' binary found on the path is the juju that
> invoked the plugin, or is there some other way to tell using
> environment variables or such? Or will all the juju plugins just fail
> if they are invoked from the non-default juju version?

The new packaging uses wrapping scripts for 'juju-1' and 'juju-2.0'
that prepend PATH with the version-specific directories. The default
/usr/bin/juju is just a symlink though, so invoking outside the
context of the wrappers will still be ambiguous. I think we do need a
better solution here.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: New juju in ubuntu

[Martin Packman]

On 05/04/2016, Adam Collard  wrote:
> Will there be a transitional release of 1.25.x that also gives us a juju1
> binary to facilitate this?

Yes, the plan entails both an update to the juju in main for the 2.0
release and a new juju 1.25 in universe that cooperates with the
changes.

For the debian packaging nitty gritty, what we had before was:

Archive:
  juju (depends juju-core)
juju-core (/usr/bin/juju etc)
  juju-local (depends juju-core, juju-mongodb, lxc bits)
  juju-local-kvm (depends juju-core, juju-mongodb, libvirt bits)

Devel PPA:
  juju2 (/usr/bin/juju2 etc)

We have the following proposed:

Main:
  juju (depends juju-2.0, suggests juju-core)
juju-2.0 (/usr/lib/juju-2.0/bin/juju, linked as /usr/bin/juju and
wrapped as /usr/bin/juju-2.0)

(The lxd provider is bundled, replacing the local packaging bits for 2.0).

Universe:
  juju-core (transitional depends juju-1.25)
  juju-1 (depends juju-1.25)
juju-1.25 (/usr/lib/juju-1.25/bin/juju, wrapped as /usr/bin/juju-1)
  juju-local (depends juju-1.25, juju-mongodb, lxc bits)
  juju-local-kvm (depends juju-1.25, juju-mongodb, libvirt bits)

When upgrading, you keep your juju package and upgrade juju-core to
the new 1.X packaging, and pick up the new 2.0 package. With a fresh
install, you just get 2.0 unless you ask for juju-1 as well or take
the suggests.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

New juju in ubuntu

[Martin Packman]

Some of you will be aware of the excitement over the last week getting
Juju 2.0 into Xenial, this is an update mail so everyone knows what
the goal is with the user experience.

The challenge here is we want Juju 2.0 and all the new functionality
to be the default on release, but not break our existing users who
have working Juju 1.X environments and no deployment upgrade path yet.
So, versions 1 and 2 have to be co-installable, and when upgrading to
xenial users should get the new version without their existing working
juju being removed.

There are several ways to accomplish that, but based on feedback from
the release team, we switched from using update-alternatives to having
'juju' on xenial always be 2.0, and exposing the 1.X client via a
'juju-1' binary wrapper. Existing scripts can either be changed to use
the new name, or add the version-specific binaries directory
'/var/lib/juju-1.25/bin' to the path.

Because the meaning of 'juju' is changing, we want to give a
on-first-run message about how to work with any existing environments
the user may have. That work is covered by:

The best way to detect seems to be, when creating the new
configuration for 2.0, if ~/.juju or JUJU_HOME is set, use the series
to suggest ways to access or install the 1.X client.

There were some other interesting points from review.

The way we do plugins both leads to large binaries with a lot of
duplication, and means the only sane way to deal with multiple juju
versions is through path manipulation. If anyone is wants to
distribute juju plugins, they'll need to watch out for this. See this
bug on the binary sizes:


Also we've had to avoid stripping the juju binaries, which is
otherwise standard distro practice, as it was unsupported and go 1.2
breaks. With the switch to go 1.6 it seems like this may now work as
expected, but needs to go through testing:


We also have some ongoing questions over golang packaging in the
distro as a whole, but have demonstrated we an split out and create
seperate golang-*-dev packages for nearly all the juju dependencies as
needed.

We hope to avoid most of the pain from this packaging process next
cycle by putting our alpha releases into the devel disto (Y 16.10),
rather than just adding them to our PPA. This will not need to replace
the 2.0 version if we have compatibility concerns with early versions
given the packaging changes from this cycle, but we will also be back
to no-breakage mode as well.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: New feature for charmers - min-juju-version

[Martin Packman]

On 23/03/2016, Ryan Beisner  wrote:
>
> To summarize:
> If we do nothing with regard to juju 1.25.x or the various tools, and if a
> relevant charm grows a series list in metadata, a load of existing
> validation and demo bundles will no longer be deployable with 1.25.x
> because `juju deploy` on 1.25.x traces when receiving a list type as a
> metadata series value.  These type of bundles often point at gh: and lp:
> charm dev/feature branches, so the charm series metadata is as it is in the
> dev charm's trunk (unmodified by the charm store).

I've landed a change on the 1.25 branch that accepts a list of series
in charm metadata and takes the first value as the default series.



Charm authors will still need to that that kind of multi series charm
and place it in named-series-directories under $JUJU_REPOSITORY to
deploy with 1.X but I think this is fine for your use case?

Martin

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Go 1.6 is now in trusty-proposed

[Martin Packman]

On 27/03/2016, David Cheney  wrote:
> Hi Martin,
>
> I was told that the Go 1.6 tests were voting, so these bugs should be
> blocking bugs. Is this not the case ?

The tests are voting, and giving blesses, so no blocking bugs, but a
lot of the remaining issues are low-occurrence failures. Basically the
unit tests pass generally given the three attempts, but overall fail a
lot from a number of issues that all happen only occasionally.

For instance, bug 1553292:



This is maybe ~5% chance of failing, but given the number of jobs now
using go 1.5+ that's still six failures in the last week.

We have enough issues like this that CI spends a lot more time
retesting on go 1.6 than we do on go 1.2 with the same unit tests.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Go 1.6 is now in trusty-proposed

[Martin Packman]

On 24/03/2016, Ian Booth  wrote:
>
> Not yet. The builders and test infrastructure all need to be updated, and
> the package needs a week to transition out of proposed.

I'd also encourage people to look again at the go 1.5/1.6 unit test
intermittent failures, as we're still often doing multiple runs in
order to get a pass.



Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

armhfentures

[Martin Packman]

There is a bug filed over Juju 2.0 not building with go 1.2 on armhf:



I believe I understand all the moving parts here, but it's a little
fiddly and we're going to have to make a choice about how we plan to
build 2.0, so wanted some input.

The bug comes from the golang.org/x/* projects being closely tied to
the core go project, and not generally attempting cross version
compatibility. So, when a new go version is released, the x/* packages
quickly pick up new functionality. In this case, the
golang.org/x/crypto package both introduces some new arm assembly code
that will not build on go 1.3 or earlier, and has also started using
some new apis from the standard library.

It looks safe to roll back to a version from before the incompatible
changes, and I've proposed that here:



However it's possible some new code (bits Rog did?) actually needs the
newer revision in github.com/juju/juju directly.

Anyway, the bigger question is, do we want keep building Juju 2.0 with
go 1.2 at all? We have work ongoing to get go 1.6 back into trusty,
using a consistent toolchain vastly simplifies what we have to test
and support. There is a wrinkle over supplying jujud binaries for
precise agents (I don't think anyone is concerned about the juju
client on precise), but it's possible we can just use the trusty build
as-is?

Mostly for my own future reference, some specifics over why
golang.org/x/crypto fails to build on armhf.

Timeline:

2015-02-06 f7445b17d61953e333441674c2d11e91ae4559d3
First problem rev, crypto package itself on longer builds with go 1.3
# golang.org/x/crypto/ocsp
ocsp/ocsp.go:482: undefined: crypto.Signer
This doesn't affect juju which has no dependency on that particular package.

2015-05-14 e3f150b4372fce47109dbd8fef5f03cd2af08700
Last working rev for go 1.2

2015-05-14 4d48e5fa3d62b5e6e71260571bf76c767198ca02
Introduction of arm assembly, new code can't be built due to syntax error?
   golang.org/x/crypto/poly1305/poly1305_arm.s:26 syntax error, last name: R9

2015-06-11 f6a608df624ae17d57958a8a294c66da81730577
Attempt to fix arm assembly compile on go 1.3, doesn't help the above issue.

2015-09-24 e74b0352e547e3830f8471e7ae9609239c52606f
Additional breakage due to use of new crypto.Signer api
golang.org/x/crypto/ssh/keys.go:492: undefined: crypto.Signer

It would be possible to address the assembly compilation by correcting
the build flags to continue using the fallback on older go versions,
with a patch something like:

http://paste.ubuntu.com/15321727/

However as tip of the project now doesn't work due to later
incompatibile changes, that doesn't get us anywhere useful.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: LXD and trusty-backports

[Martin Packman]

On 20/01/2016, John Meinel  wrote:
> So according to https://help.ubuntu.com/community/UbuntuBackports backports
> is enabled by default, as long as you explicitly request the packages from
> backports.

Yeah, that's the case for fresh machines, backports just has a lower
priority so we're safe from interfering with charm package installs.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Automatic retries of hooks

[Martin Packman]

Another common error we see in CI is apt mirrors being unhappy leading
to hook failures. Just retry later does tend to be the right option
there, though it will often be an our or two until the archive is in a
usable state again.

On 20/01/2016, William Reade  wrote:
>
> Are there any concerns that I've missed?

Automatic retries make debugging your charm harder, as James found. I
think we want an environment setting to disable this for both testing
and for charm authors.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju-CI revision testing paused

[Martin Packman]

On 24/12/2015, John George  wrote:
> As no one will be monitoring resources or test results over the holiday
> break, Juju-CI has been paused. Anything landed will be tested when the
> system is re-enabled on January 4th.

I reenabled CI this morning, and revision testing is happening again.
We don't have Joyent at present for un-juju-related reasons, but
please say if you see anything else odd.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Starting a new CI test

[Martin Packman]

I just added a cheaty way of getting started on a new CI test.

$ make new-assess name=new_feature

With 'new_feature' as the name you want for your test. This generates
a new test script, and some unit tests to go along with it, which you
can run as normal with:

$ make test p=*new_feature*

The QA team has done a fair bit of work of late both simplifying code
and updating older scripts. These templates should prove to be a
better example than looking at other random tests as we continue to
refactor things however. Input from our (many) python experts is
welcome.

Next steps, a test runner that handles both environment setup and
annoying problems like ssh config neatly.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

gomaasapi moved to github

[Martin Packman]

The gomaasapi project has been moved from launchpad to the juju
organisation on github:



As part of this, the project is using the standard juju gating, so add
the magic string $$merge$$ to a github comment after review to merge.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Upgrading minimum Go version?

[Martin Packman]

On 26/11/2015, David Cheney  wrote:
> Martin: Can you setup a unit test job that uses Go 1.5 ? I will apply
> the same process I did for the -race job, to get the build voting,
> then reenable the tests that were skipped.

We've had that since near the start of the wily development cycle, now
we have two:

http://juju-ci.vapour.ws/job/run-unit-tests-wily-amd64/
http://juju-ci.vapour.ws/job/run-unit-tests-xenial-amd64/

This is basically inherited pain from never making the tests work on
vivid (go 1.3) so the job could never become voting.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: juju bootstrap fails with vSphere 6.0

[Martin Packman]

On 05/11/2015, Forstner Michael  wrote:
>
> Response 4 (Invalid XML; attaching Hex view):
>   1f 8b 08

Aha, that's just gzip encoded. Once decoded, the response looks sane:

http://paste.debian.net/325311

So, need the http headers as well. If the vSphere server included
"Content-Encoding: gzip" response header then the client can certainly
unpack it, whether or not an Accept-Encoding request header was given.

Martin

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Juju 1.24.7 is proposed for stable release

[Martin Packman]

On 16/10/2015, Casey Marshall  wrote:
> I just upgraded to 1.24.7 and tried bootstrapping, got this message:
>
> Starting new instance for initial state server
> WARNING failed to find 1.24.7 tools, will attempt to use 1.24.6

Did you remember to set `agent-stream: proposed` in environments.yaml
(and delete any evil lingering jenv files)?

> Is there a problem with the publish to 'proposed' simple-streams?

I see 1.24.7 in the streams:



A bootstrap just worked for me:

2015-10-16 19:47:22 INFO juju.environs.bootstrap bootstrap.go:212
picked bootstrap tools version: 1.24.7

machines:
  "0":
agent-state: started
agent-version: 1.24.7

Martin

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: User creation with cloud-init

[Martin Packman]

Have run the branch through CI twice. Fixed the issues that showed up
first time so second run is blessed. It did mean reverting to the old
way of setting keys on precise specifically, as the cloud-init we have
there is too old to support 'users' configuration. What's left before
this can land is verifying the change works on CentOS.

On 11/10/2015, Antonio Rosales  wrote:
>
> I see we have https://bugs.launchpad.net/juju-core/+bug/1492066 fixed
> committed for 1.25 (thanks Dimiter), does this branch resolve any
> other known issues with deploying *Nix Specifically CentOS with Juju,
> or are there others we should identify here?

I'm not aware of any other issues with CentOS, but we've had limited
feedback from users so far. The CI team is working on getting our set
of revision tests running with CentOS on MAAS, and I've been bothering
Bogdan about getting something working on a public cloud.

What we are currently missing that I'd love is some real-world charms
that use CentOS. Our test charms are deliberately simplistic, some
real workloads to evaluate would be great.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

User creation with cloud-init

[Martin Packman]

I proposed a branch this week that alters how juju uses cloud-init to
create the ubuntu user when provisioning a machine. I've had code
review, but would appreciate further feedback:



The main aim here is to avoid depending on the /etc/cloud/cloud.cfg in
the image, but keep the logic unified across *nix systems. This
includes a CentOS behaviour change that I hope is okay, removing the
shell script use setup and just using cloud-config.

There are still a couple of things to be nervous about. The exisitng
container creation config is very much tied to ubuntu still, and I'm
not completely confident of stability across series for cloud-init
versions or default groups.

My intention is to get a reasonable amount of testing in advance of
landing the branch, but comments welcome as well.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Running upgrade steps for units

[Martin Packman]

On 16/09/2015, Tim Penhey  wrote:
>
> When is the 1.24.6 release?

We've been trying to release for a while, currently cursed by the
following bug with failing to deploy the landscape bundle:



This seems to be a regression from a partial fix on a reliability
issue reported by the landscape team:



We have the option of releasing without that change, but I'm told this
fix was one of the main motivations for doing a 1.24.6 now.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Juju devel 1.25-alpha1 is released

[Martin Packman]

# juju-core 1.25-alpha1

A new development release of Juju, juju-core 1.25-alpha1, is now available.


## Getting Juju

juju-core 1.25-alpha1 is available for Wily and backported to earlier
series in the following PPA:

https://launchpad.net/~juju/+archive/devel

Windows, Centos, and OS X users will find installers at:

https://launchpad.net/juju-core/+milestone/1.25-alpha1

Development releases use the devel simple-streams. This release
will automatically select the devel stream. You do not need to update
your config in environments.yaml to bootstrap.

Upgrading from stable releases to development releases is not
supported. You can upgrade test environments to development releases
to test new features and fixes, but it is not advised to upgrade
production environments to 1.25-alpha1.


## Notable Changes
  * (Experimental) Improved networking features for AWS
* New 'spaces' and 'subnet' commands
* New 'spaces' constraints support
  * Support for devices on MAAS 1.8+
  * Storage support for GCE and Azure providers


### (Experimental) Improved networking features for AWS

Juju has experimental support for deploying services on AWS in isolation,
taking advantage of the enhanced AWS VPC networking features. This is just a
first step towards a full-fledged networking model support in Juju.


 New 'spaces' and 'subnet' commands

Juju introduces the 'space' and 'subnet' commands to manage the networks
available to services. A Juju network space is a collection of related subnets
that have no firewalls between each other and have the same ingress and egress
policies.

You can create a new Juju space, and optionally associate existing subnets
with it by specifying their CIDRs using the 'create' subcommand. The command
looks like this:

juju space create name [CIDR1 CIDR2 …]

The list of registered spaces can be seen using the 'list' subcommand:

juju space list [--short] [--format yaml|json] [--output path]

You can add an existing AWS subnet to a space by CIDR or AWS ID, for example:

juju subnet add CIDR|AWS-subnet-id space-name

You can list all the subnets known by Juju and optionally filtering them by
space and/or availability zone like so:

juju subnet list [--zone name] [--space name] [zone1 zone2 …]

For more information about these commands, type:

juju command --help


 New 'spaces' constraints support

The new 'spaces' constraint instructs Juju to deploy a service's units in
subnets of the given space.

Similar to the 'tags' constraint, the 'spaces' constraint takes a comma-
delimited list of existing Juju network spaces. Both positive (without prefix)
and negative (prefixed by ^) spaces are allowed in the list. For this alpha
release, the first positive space name is used, the rest is ignored.

You can constrain a service to a space like this:

juju deploy charm-URL [service-name] [--constraints spaces=list]

For more information, run

juju help glossary

and

juju help constraints


### Support devices on MAAS 1.8+

MAAS 1.8 introduced a new feature called devices. This allows the
association of a device, that requires an IP address, with a parent machine
managed by MAAS. There is a view in the MAAS UI showing all devices.

With the address-allocation feature flag enabled, Juju will register lxc and
kvm containers as devices on MAAS 1.8+. They are visible in the MAAS UI. If
the environment is forcibly shut down, the IP addresses allocated to the
containers will be released by MAAS.

You can enable address-allocation is new Juju environments like so:

JUJU_DEV_FEATURE_FLAG=address-allocation juju bootstrap


### Storage support for GCE and Azure providers

Juju's storage feature can now be used with the Google Compute Engine and
Azure providers. Storage is now supported for local, AWS, Openstack, MAAS, GCE
and Azure. See https://jujucharms.com/docs/devel/storage for more information.

### Status for storage volumes

Volumes now have status associated, so provisioning progress can be observed.
To show the status of volumes, use juju storage volume list.


## Resolved issues

  * Regression: juju ssh dies with (publickey)
Lp 1472632

  * Cli rejects placement directives
Lp 1358941

  * Feature flags don't propagate to windows agents
Lp 1401368

  * Upgrade fails if no explicit version is specified
Lp 1447899

  * Provider/openstack: cinder provider should reject attempts to
create non-persistent volumes
Lp 1450737

  * Storage: loop devices leaked by lxc containers
Lp 1452649

  * Deployments fail when juju implicitly upgrade after bootstrap
Lp 1455260

  * Openstack provider should work without object-store
Lp 1456265

  * Apiserver run method doesn't validates if machines, services or
units are present on the request.
Lp 1456343

  * Upgrade fails if there's a series in streams juju doesn't
recognize
Lp 1459093

  * Init system discovery script has bashisms
Lp 1459775

  * Juju should 

Juju devel 1.25-alpha1 is released

[Martin Packman]

# juju-core 1.25-alpha1

A new development release of Juju, juju-core 1.25-alpha1, is now available.


## Getting Juju

juju-core 1.25-alpha1 is available for Wily and backported to earlier
series in the following PPA:

https://launchpad.net/~juju/+archive/devel

Windows, Centos, and OS X users will find installers at:

https://launchpad.net/juju-core/+milestone/1.25-alpha1

Development releases use the devel simple-streams. This release
will automatically select the devel stream. You do not need to update
your config in environments.yaml to bootstrap.

Upgrading from stable releases to development releases is not
supported. You can upgrade test environments to development releases
to test new features and fixes, but it is not advised to upgrade
production environments to 1.25-alpha1.


## Notable Changes
  * (Experimental) Improved networking features for AWS
* New 'spaces' and 'subnet' commands
* New 'spaces' constraints support
  * Support for devices on MAAS 1.8+
  * Storage support for GCE and Azure providers


### (Experimental) Improved networking features for AWS

Juju has experimental support for deploying services on AWS in isolation,
taking advantage of the enhanced AWS VPC networking features. This is just a
first step towards a full-fledged networking model support in Juju.


 New 'spaces' and 'subnet' commands

Juju introduces the 'space' and 'subnet' commands to manage the networks
available to services. A Juju network space is a collection of related subnets
that have no firewalls between each other and have the same ingress and egress
policies.

You can create a new Juju space, and optionally associate existing subnets
with it by specifying their CIDRs using the 'create' subcommand. The command
looks like this:

juju space create name [CIDR1 CIDR2 …]

The list of registered spaces can be seen using the 'list' subcommand:

juju space list [--short] [--format yaml|json] [--output path]

You can add an existing AWS subnet to a space by CIDR or AWS ID, for example:

juju subnet add CIDR|AWS-subnet-id space-name

You can list all the subnets known by Juju and optionally filtering them by
space and/or availability zone like so:

juju subnet list [--zone name] [--space name] [zone1 zone2 …]

For more information about these commands, type:

juju command --help


 New 'spaces' constraints support

The new 'spaces' constraint instructs Juju to deploy a service's units in
subnets of the given space.

Similar to the 'tags' constraint, the 'spaces' constraint takes a comma-
delimited list of existing Juju network spaces. Both positive (without prefix)
and negative (prefixed by ^) spaces are allowed in the list. For this alpha
release, the first positive space name is used, the rest is ignored.

You can constrain a service to a space like this:

juju deploy charm-URL [service-name] [--constraints spaces=list]

For more information, run

juju help glossary

and

juju help constraints


### Support devices on MAAS 1.8+

MAAS 1.8 introduced a new feature called devices. This allows the
association of a device, that requires an IP address, with a parent machine
managed by MAAS. There is a view in the MAAS UI showing all devices.

With the address-allocation feature flag enabled, Juju will register lxc and
kvm containers as devices on MAAS 1.8+. They are visible in the MAAS UI. If
the environment is forcibly shut down, the IP addresses allocated to the
containers will be released by MAAS.

You can enable address-allocation is new Juju environments like so:

JUJU_DEV_FEATURE_FLAG=address-allocation juju bootstrap


### Storage support for GCE and Azure providers

Juju's storage feature can now be used with the Google Compute Engine and
Azure providers. Storage is now supported for local, AWS, Openstack, MAAS, GCE
and Azure. See https://jujucharms.com/docs/devel/storage for more information.

### Status for storage volumes

Volumes now have status associated, so provisioning progress can be observed.
To show the status of volumes, use juju storage volume list.


## Resolved issues

  * Regression: juju ssh dies with (publickey)
Lp 1472632

  * Cli rejects placement directives
Lp 1358941

  * Feature flags don't propagate to windows agents
Lp 1401368

  * Upgrade fails if no explicit version is specified
Lp 1447899

  * Provider/openstack: cinder provider should reject attempts to
create non-persistent volumes
Lp 1450737

  * Storage: loop devices leaked by lxc containers
Lp 1452649

  * Deployments fail when juju implicitly upgrade after bootstrap
Lp 1455260

  * Openstack provider should work without object-store
Lp 1456265

  * Apiserver run method doesn't validates if machines, services or
units are present on the request.
Lp 1456343

  * Upgrade fails if there's a series in streams juju doesn't
recognize
Lp 1459093

  * Init system discovery script has bashisms
Lp 1459775

  * Juju should 

Re: Uniter tests for update-status hook - BLOCKER

[Martin Packman]

On 20/07/2015, Tim Penhey tim.pen...@canonical.com wrote:

 Earlier today I was investigating this CRITICAL BLOCKER bug:
   https://bugs.launchpad.net/juju-core/+bug/1475724

 At first I thought that bug was referring to a different one, which I
 fixed by skipping a part of a test that was using chmod to make unit
 enter an error state. I filed a bug for the fixing of the skip:
   https://bugs.launchpad.net/juju-core/+bug/1476060

 However on deeper looking into the first bug, it seems that it is all
 timing related.

Unfortunately tracking failures in the uniter tests is complicated.
I've seen four named tests fail much more often in recent days:

TestUniterRelations
http://reports.vapour.ws/releases/issue/559b486d749a566c38b6a7bc

TestUniterCollectMetrics
http://reports.vapour.ws/releases/issue/555b70eb749a563f74eb83ec

TestUniterSendMetrics
http://reports.vapour.ws/releases/issue/55acde69749a560b02ac8d06

TestUniterUpdateStatusHook
http://reports.vapour.ws/releases/issue/55ace932749a56072b3d3637

However it's hard to manage as we have issues in the suite in general,
and uniter test output is not easy to deal with. The logs are giant,
the actual failure lines tend to be non-informative with the real
cause several screens up in the log, multiple tests have basically the
same problems with common code...

So, I'm not sure what bugs we want to file to track the work to get
master in a good state. As best as I can work out we have:

1) A regression on windows from the 16th, probably from this metrics landing:
http://reviews.vapour.ws/r/2173/

2) An earlier regression on all platforms tracked in this bug:
https://bugs.launchpad.net/juju-core/+bug/1475724

3) A more general problem with the TestUniterSendMetrics.

I'm not even completely sure which of these your windows test skip
resolves, but I assume the first? Fun, huh.

 [1] We should seriously start thinking how to gate landings on the unit
 tests passing on amd64, ppc, and windwos.

I'd love to gate on the windows tests, but don't want to get yelled
at. Recently, three test runs at 40mins each has not been enough to
get a passing suite reliably, but maybe with this latest batch of
fixes that becomes more reasonable again.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Uniter tests for update-status hook - BLOCKER

[Martin Packman]

On 20/07/2015, Martin Packman martin.pack...@canonical.com wrote:

 So, I'm not sure what bugs we want to file to track the work to get
 master in a good state. As best as I can work out we have:

Okay, becoming clearer now,

1) Windows regression from uniter-status change:
http://reviews.vapour.ws/r/1979/
Fixed by Tim adding the skip, tracked by this bug:
https://bugs.launchpad.net/juju-core/+bug/1476060

 2) An earlier regression on all platforms tracked in this bug:
 https://bugs.launchpad.net/juju-core/+bug/1475724

3) Metrics breakage that should now be fixed by this change, not
tracked in a bug:
http://reviews.vapour.ws/r/2143/

4) Different metrics breakage fixed as part of this bug:
https://bugs.launchpad.net/juju-core/+bug/1475271

Plus unreliable tests, plus feature branches that aren't all up to
date with trunk.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Dreamhost progress

[Martin Packman]

Andrew landed a change on master to allow the Openstack provider to
track state servers using metadata on the instances rather than
object-store. This should fix the bug I raised[0] but for one small
extra change needed, and gets us onto the next problem bootstrapping
with Dreamhost.

To try out juju on Dreamhost, need an account[1] (free trial
available), then configure environments.yaml with credentials,
remembering to use project-name for tenant-name rather than tenant-id.
I also added default-series: trusty and network: private-network
which aren't strictly required.

Then recompiled master with an additional patch[2] to make goose not
complain when there is no object-store in the keystone catalog.

Next step is generate image metadata for the trusty image given by
`nova image-list` which is the same as before:

$ juju metadata generate-image -i c55094e9-699c-4da9-95b4-2e2e75f4c66e -s trusty

Then bootstrap with --upload-tools and --metadata-source pointing at
the directory with the image streams.

That got as far as trying to connect, but instances only have an IPv6
and private 10. address by default. Juju did try to ssh to
2607:f298:6050:8af8:f816:3eff:fee3:2e94 but I'm not sure that code
path would actually work even if I had IPv6 routing from canonistack
to dreamhost.

Easy fix, add use-floating-ip: true.

That gets us to the current problem though, which is the Dreamhost
image labelled Ubuntu-14.04-Trusty does not have an 'ubuntu' user.
Instead, the base user account is 'dhc-user' and cloud-init adds your
keys there. I can see three possible ways past this:

1) Get them to use our standard cloud images
2) Add config option to use a different username than ubuntu (ick)
3) Make the cloudconfig generation detect the lack of the ubuntu user and add it

Our centos path sort of does the last of these already, but we assume
all ubuntu versions are sanely set up.

Martin


[0] Openstack provider should work without object-store
https://bugs.launchpad.net/juju-core/+bug/1456265
[1] DreamCompute
https://www.dreamhost.com/cloud/computing/
[2] Patch needed to progress with bootstrap:
diff --git a/provider/openstack/provider.go b/provider/openstack/provider.go
index 4ba9a73..e84b5c9 100644
--- a/provider/openstack/provider.go
+++ b/provider/openstack/provider.go
@@ -770,7 +770,9 @@ func authClient(ecfg *environConfig) client.AuthenticatingCl
if !ecfg.SSLHostnameVerification() {
newClient = client.NewNonValidatingClient
}
-   return newClient(cred, authMode, nil)
+   client := newClient(cred, authMode, nil)
+   client.SetRequiredServiceTypes([]string{compute})
+   return client
 }

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Blocking bugs process

[Martin Packman]

On 13/07/2015, Nate Finch nate.fi...@canonical.com wrote:
 Can you put this in the wiki?

Done, with Aaron's addition:

https://github.com/juju/juju/wiki/Blocking-bugs

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Blocking bugs process

[Martin Packman]

Thank you for responding Ian.

On 13/07/2015, Ian Booth ian.bo...@canonical.com wrote:

 == Definition of blocking bugs ==
 Master and all release branches should always be in a releasable
 state. If a bug must be fixed for the next minor release, it is
 considered a ‘blocker’ and will prevent all landing on that branch.


 I don't agree with the above definition. There are always many bugs which
 must be fixed for the next minor release - these are assigned to the relevant
 milestones and marked high or critical.

So, my argument over this was a pretty strict interpretation of
must. There are lots of bugs with less-than-critical severity that
get targeted at the next minor milestone, and I think that's perfect
reasonable. However, there are comparatively few bugs that could not
be deferred if, for instance, we discovered a security issue and had
to rush out a new minor release immediately.

From my perspective, blockers are things that break CI enough to
hinder our ability to do a release, or issues that if we allow into
release code will break users in unrecoverable ways. I know Aaron
prefers a much wider interpretation however.

 In the case of bug 1468653, this has been
 under investigation for 2 weeks and even though we are holding up the 1.24.3
 release for it, if it were a blocker the whole production line would have
 stalled unnecessarily.

That's an interesting bug. It seems with a lot of pain (manually
restarting things) it is actually repairable, and does involve a
pretty specific setup. I don't think I'd have added the blocker tag to
it, but that may not be the consensus.

 With follow on changes, the problem is
 quite isolated so landing fixes for other release critical issues should not
 be prevented.

The fact it's a somewhat isolated problem is important. What I really
want to avoid is the case where we leave say, upgrades broken as a
whole, and keep landing code. That makes it impossible to tell if
following changes also have upgrade problems, or compound the existing
issue. Likewise, if we have CI tests failing, subsequent landings make
identifying and deal with further regressions vastly more painful, and
hose our release process.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Keystone error

[Martin Packman]

You may have more luck asking on the new 'openstack-charmers' list as
this sounds more like an issue with how to use those rather with juju
in general:

https://lists.ubuntu.com/mailman/listinfo/openstack-charmers

On 17/06/2015, dinesh.senap...@wipro.com dinesh.senap...@wipro.com wrote:
 The part where we relate keystone with mysql ( juju add-relation ) fails
 after we create a connection between the two. It says shared-db
 relation-changed fails. And this is the first thing we are trying to do
 before relating any other services.

For starts, you'll want to look at the unit log to find out more
details on why the hook failed. It's in /var/log/juju on the machine
containing that unit.

Martin

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: like go test, but faster

[Martin Packman]

On 15/06/2015, Nate Finch nate.fi...@canonical.com wrote:
 Russ Cox has an experimental command called gt, which replaces the go test
 tool, and caches test output when you run it.  The next time you run gt, if
 none of the files in a package have changed and none of the files of
 packages it depends on have changed, then gt will just reprint the cached
 output.

Shame that the code not having changed isn't a good indicator of
whether a passing test will fail next run with juju:

https://bugs.launchpad.net/juju-core/+bugs?field.tag=intermittent-failure

1 → 75 of 108 results...

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Rackspace deployment without os-security-groups support.

[Martin Packman]

On 11/06/2015, Mark Hannessen m.hannes...@youwe.nl wrote:

 Looking on the internet i found the setting firewall-mode which had a
 promising description for working around this.
 https://jujucharms.com/docs/devel/config-general

Unfortunately firewall-mode won't help. We still create the global
group with that set to none to ensure we can access the instances via
ssh. I filed a bug for making the setting do more what you want:

https://bugs.launchpad.net/juju-core/+bug/1464235

There may well be other issues with running on rackspace after this one.

Martin

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: REMINDER: run tests with -race before submitting for review

[Martin Packman]

On 10/06/2015, Tim Penhey tim.pen...@canonical.com wrote:

 So... for the tests that you are writing for the code you are changing,
 please run these tests with the race flag. This should at least help us
 not introduce new races.

I've linked up the race detection CI job to our outcome analyser:

http://reports.vapour.ws/releases/issue/5578525d749a567e4f99a0cb

Unfortunately the output is rather confused, as the run is also full
of tests being killed because they take longer than 10 minutes to run
and tests failing because of timing related issues. So, it's not very
useful for comparing the current result to the previous as it's so
random.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Rule #2 should die

[Martin Packman]

Currently juju-reports has a rule matching on failures where our CI
harness interrupted the test because it took too long:

http://reports.vapour.ws/releases/rules/2

This seems too generic a symptom, generally if a test is not
completing within the time we've allocated for it, there's another
indication in the log, often the final `juju status` output, that
makes it clearer why juju never finished its work.

Checking over the recent matches for rule #2:

http://reports.vapour.ws/releases/2733/job/gce-deploy-trusty-amd64/attempt/265

  2:
agent-state-info: 'sending new instance request: GCE operation
operation-1433442378645-517b54fc8fe09-e1a9dd29-092d6cff
  failed'
instance-id: pending

GCE failed to give us an instance.


http://reports.vapour.ws/releases/2732/job/canonistack-deploy-trusty-amd64/attempt/3092

  1:
agent-state: pending
dns-name: 10.55.32.175
instance-id: ee22b864-47e9-4931-8bcb-92bbbe08f05e
instance-state: ACTIVE

http://data.vapour.ws/juju-ci/products/version-2732/canonistack-deploy-trusty-amd64/build-3092/machine-1/cloud-init.log.gz

Jun  4 14:57:24 juju-canonistack-deploy-trusty-amd64-machine-1
[CLOUDINIT] util.py[DEBUG]: Running command ['eatmydata', 'apt-get',
'--option=Dpkg::Options::=--force-confold',
'--option=Dpkg::options::=--force-unsafe-io', '--assume-yes',
'--quiet', 'install', 'curl', 'cpu-checker', 'bridge-utils',
'rsyslog-gnutls', 'cloud-utils', 'cloud-image-utils', 'tmux'] with
allowed return codes [0] (shell=False, capture=False)

Super slow canonistack machine, still crawling along installing
packages when we gave up.


http://reports.vapour.ws/releases/2732/job/functional-backup-restore/attempt/2702

error: cannot re-bootstrap environment: cannot bootstrap new instance:
waited for 10m0s without being able to connect: ssh: connect to host
10.0.0.247 port 22: Connection timed out

Not the best log, but seems clear we never got a usable bootstrap
machine to restore into.


http://reports.vapour.ws/releases/2732/job/joyent-deploy-precise-amd64/attempt/2145

  1:
agent-state: pending
dns-name: 165.225.128.214
instance-id: fc67a2b4-00ab-4571-e947-ebd68fd54f9b
instance-state: running

http://data.vapour.ws/juju-ci/products/version-2732/joyent-deploy-precise-amd64/build-2145/machine-1/cloud-init-output.log.gz

Attempt 1 to download tools from
https://10.112.2.15:17070/tools/1.25-alpha1-precise-amd64...
+ curl -sSfw tools from %{url_effective} downloaded: HTTP
%{http_code}; time %{time_total}s; size %{size_download} bytes; speed
%{speed_download} bytes/s  --noproxy * --insecure -o
/var/lib/juju/tools/1.25-alpha1-precise-amd64/tools.tar.gz
https://10.112.2.15:17070/tools/1.25-alpha1-precise-amd64
curl: (7) couldn't connect to host

Joyent network issue, https://bugs.launchpad.net/juju-core/+bug/1451104


That all the recent matches for the timeout rule have more useful and
specific matches (some unfortunately needing to look at other log
files for all the details), suggests we want those as rules rather
than this.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Getting back to t1.micros

[Martin Packman]

On 14/05/2015, José Antonio Rey j...@ubuntu.com wrote:

 * When I set the same constraints at bootstrap, it just gives me an error
 saying it uses a different HDD type. Could we at least get that fixed?

There's now a pull request up that fixes the virt type of t2 instances:

http://reviews.vapour.ws/r/1704/

Do you now need ebs for t1 as well? It always used to work with pv.

 I would like to hear your opinions on getting t1.* back as a default. It
 was a great resource, and now I can't even get it by forcing it because it
 will throw errors.

As Ian mentioned, default is no good, as the old instance types just
don't exist in new regions. It's also bad to have a default that we
know is no good for anything other than dev purposes. If we can make
the constraint work when t1.micro is present, that will have to do.

Martin

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Resolving the same windows test failures again

[Martin Packman]

On 05/05/2015, Andrew Wilkins andrew.wilk...@canonical.com wrote:

 Those fixes were not made on v5, and only existed on v6-unstable.
 I later came along and merged some changes into v5 and updated
 Juju's dependencies.tsv to the latest commit on v5. That meant the
 fixes were dropped.

Not easy to track this history on github,  but as far as I can see
v6-unstable did not exist when those changes were merged. Looks like
v5 was then renamed in the web ui to v6-unstable and v5 was pushed up
from an earlier revision. When your branch went to bump the rev, it
therefore dropped those changes (and when I went to pull my v5 branch
locally, it complained of diverged branches).

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Resolving the same windows test failures again

[Martin Packman]

There was some confusion about the regression to the windows test
failures on trunk.

https://bugs.launchpad.net/juju-core/+bug/1450919

Partly my fault, Curtis initially looked at the 1.24 branch and I
looked at trunk, and each branch has a different issue. Here's what
I've just done to diagnose.


So, trunk first, I noticed the failures looked exactly the same as
before CI fiddled with the environment variable casing to work around
a juju bug:

https://launchpad.net/bugs/1446871

The fix for this bug landed on master, therefore it's easy to assume
that change actually broke the casing behaviour in such a way as to
invalidate the CI hack, rather than fixing the underlying issue.

Looking at the code:

https://github.com/juju/juju/pull/2124/files

Two similar functions now exist for merging case, mergeEnvWin which
works on map[string]string and mergeWindowsEnvironment which works on
[]string. Only mergeEnvWin has tests. Guess, mergeWindowsEnvironment
is bugged. Indeed:

-   m[strings.ToUpper(varSplit[0])] = varSplit[1]
+   k := varSplit[0]
+   if _, ok := uppers[strings.ToUpper(k)]; ok {
+   uppers[k] = varSplit[1]

It's not uppercasing the key in the assignment. So, Path is matched to
PATH, Path is assigned to, but later only PATH is pulled out.


Now for the 1.24 branch, no hints here. So, lets see what changed.
Latest working 1.24 without windows test failures:

http://reports.vapour.ws/releases/2581

Earliest 1.24 with windows test failures:

http://reports.vapour.ws/releases/2592

$ git diff fffe3e4f..95e7619f

2770 lines... Not helpful. But, error mentions cannot move the charm
archive and:

-gopkg.in/juju/charm.v5 git
6b74a2771545912f8a91a544b0f28405b99386242015-04-14T14:33:47Z
+gopkg.in/juju/charm.v5 git
779394167ac61b02ca73ca17c3012f05a5ba316c2015-04-30T02:46:55Z

$ pushd ~/go/src/gopkg.in/juju/charm.v5

Try to update and branches diverged? Wha?

$ git diff 6b74a277..77939416

Er... this includes a revert of my licence header changes, and, the
answer to the test failures, Gabriel's file closing fix:
https://github.com/juju/charm/pull/119
https://github.com/juju/charm/pull/120

So, just renaming a variable is *not* safe (if you accidentally back
out other changes when merging).


Both these regressions happened due to landing 'safe' changes while
the branch was broken, and were therefore harder to pin down than they
would otherwise have been.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Do not land code on blocked branches

[Martin Packman]

It has been great that everyone has been diving in and tackling the
backlog of bugs over the last week. However, please only merge pull
requests that are actually fixing bugs blocking that branch.

In particular, the JFDI instruction to the lander is an escape hatch
to allow for merging in extraordinary circumstances, and is not for
landing non-blocking bug fixes.

If you think the bug your pr is fixing is actually a critical blocker,
but CI is preventing it landing you should first:

* Check if the launchpad bug is correctly targeted to the series you
are landing on.
* Ask Alexis, Wes, or your team lead to escalate the bug if it's not
marked as blocking.

In the last three days, master has gone from just two failing test
cases, up to ten:

http://reports.vapour.ws/releases/2577
http://reports.vapour.ws/releases/2593

CI and the QA team has been busy with releases for the 1.22 and 1.23
branches, adding a large number of regressions to master over this
time period is unhelpful.

Curtis has filed three new bugs for these so far, and there appears to
be one or two more to come:

https://bugs.launchpad.net/juju-core/+bug/1450912
https://bugs.launchpad.net/juju-core/+bug/1450917
https://bugs.launchpad.net/juju-core/+bug/1450919

What I wanted to do was back out Nate's branch as the likely culprit
for the windows test suite regression, but with 13 other branches
landed since then that's no longer a straightforward inference.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Running Juju in Google compute engine

[Martin Packman]

On 01/05/2015, Eric Snow eric.s...@canonical.com wrote:
 We have info in the release notes [1], but looks like we have a gap in
 the docs (I'll follow up on that).  Also, it looks like the release
 notes did not get updated with the auth-file config option that
 Andrew mentioned.

So, Jorge and I walked through this and basically had three issues;

* If you some of the three split options *and* auth-file, it ignored
auth-file and complains not having all three options set. The error
should be clearer about what's mutually exclusive.

* The auth-file option is described in the source code but not
docs/release notes.

* The json file Jorge had initially was included a map of stuff to
urls to keys, but didn't include a private key block. I'm not sure how
he got it, but regenerating a json file through the web ui gave him
the right thing.

Martin

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Test suite on windows now voting on trunk

[Martin Packman]

On 23/04/2015, Martin Packman martin.pack...@canonical.com wrote:
 Well, mostly good news after Matty and I landed workarounds. There's a
 single remaining failure that has manifested since:

 TestLeadership fails on windows test slave
 https://bugs.launchpad.net/juju-core/+bug/1447595

With Bogdan's fix landed, we've had our first clean run on CI:

http://reports.vapour.ws/releases/2558/job/run-unit-tests-win2012-amd64/attempt/287

Description set: Revision build: 2558
gitbranch:master:github.com/juju/juju 73cde95d
Finished: SUCCESS

Thanks for helping out everyone!

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Public Service Annoucement

[Martin Packman]

On 24/04/2015, David Cheney david.che...@canonical.com wrote:
 Why doesn't the bot run this hook ?

It does - but there are two problems:

1) For the gating job to reject the change, the script just has not
return non-zero. Seems `go tool vet` returns 0 even if it reports
issues. So, scripts/verify.bash needs updating if you want to fail on
vet errors.

2) If I download the tarball, and run the verify script in it, I don't
get any vet issues reported:

ubuntu@go:/tmp/vet$ wget
http://data.vapour.ws/juju-ci/products/version-2558/build-revision/build-2558/juju-core_1.24-alpha1.tar.gz
ubuntu@go:/tmp/vet$ tar -xzf juju-core_1.24-alpha1.tar.gz
ubuntu@go:/tmp/vet$ cd juju-core_1.24-alpha1/src/github.com/juju/juju
ubuntu@go:/tmp/vet/juju-core_1.24-alpha1/src/github.com/juju/juju$
GOPATH=/tmp/vet/juju-core_1.24-alpha1 ./scripts/verify.bash
go version go1.2.1
checking: go fmt ...
checking: go vet ...
checking: go build ...
checking: tests are wired up ...

If I checkout that git revision (73cde95d) and run the script in tree,
I do get the vet error.

Basically, make the script fail when you guys want it to fail, and the
gating job will reject those branches.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Public Service Annoucement

[Martin Packman]

On 24/04/2015, roger peppe roger.pe...@canonical.com wrote:

 Are you sure? It seems to return a non-zero exit status for me.

I guess you're not using ubuntu packaged go, but something trunkier?

 I guess it (and the other point) might be a consequence of running
 a different version of govet.

$ dpkg -s golang-go.tools
..
Version: 0.0~hg20131126-3
...
Built-Using: golang (= 2:1.2-2ubuntu3), golang-go.net-dev (= 0.0~hg20131201-1)

Is what the bot has, it's the latest in trusty:

http://packages.ubuntu.com/trusty/golang-go.tools

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Test suite on windows now voting on trunk

[Martin Packman]

Well, mostly good news after Matty and I landed workarounds. There's a
single remaining failure that has manifested since:

TestLeadership fails on windows test slave
https://bugs.launchpad.net/juju-core/+bug/1447595

On 22/04/2015, John Meinel j...@arbash-meinel.com wrote:
 That sounds like it could be a runtime problem rather than just a testing
 problem.  It sounds surprising that we would ever reference an environment
 variable with Title case.

It does seem to be, though whether it comes up in practice will I
guess depend on your base windows image. For now the hack in the test
runner seems effective.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: 'ERROR juju.cmd supercommand.go:430 no instances found' when bootstrapping Juju on OpenStack Juno

[Martin Packman]

On 23/04/2015, Robert Day robert@metaswitch.com wrote:
 Martin, Mark,

 Thank you! That was indeed the problem - once I'd changed that entry to an
 IP address (and fixed a couple of similar problems with my metadata files
 not being accessible, where the error messages were much easier to diagnose
 from), everything started working, and I've now successfully got my charms
 working on my OpenStack environment.

Great!

I put up a branch last night to make the error handling when listing
instances in the openstack provider saner, so this kind of thing
should be easier to diagnose in future.

Martin

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Vivid init system bug status

[Martin Packman]

On 17/04/2015, Martin Packman martin.pack...@canonical.com wrote:

 I have put up a merge proposal making the error handling robust across
 go versions, and removing the map interation confusion:

 https://github.com/juju/juju/pull/2094

 This should be on the 1.23 branch shortly.

Well, this was all a bit of a disaster.

That fix resolves the case where upstart is not installed (a fresh
vivid system), but breaks the case where upstart is installed but not
enabled (an upgraded vivid system):

https://bugs.launchpad.net/juju-core/+bug/1447446

Eric has proposed a branch with a bunch more changes, but I'm not
really happy with unit tests that rely on the state of the underlying
system to cover all the cases we care about.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: 'ERROR juju.cmd supercommand.go:430 no instances found' when bootstrapping Juju on OpenStack Juno

[Martin Packman]

On 16/04/2015, Robert Day robert@metaswitch.com wrote:

 I'm trying to set up Juju on my private OpenStack Juno cloud, but 'juju
 bootstrap' is consistently failing with 'ERROR juju.cmd supercommand.go:430
 no instances found'. My juju-core package version is
 1.22.1-0ubuntu1~12.04.1~juju1.

Having looked at some logs kindly forwarded by Mark, I don't have a
definitive diagnostic, but have got an educated guess.

 auth-url: http://obuenfvm1:5000/v2.0/

Can an instance within the cloud resolve this address? Once the state
server is up, it also needs to be able to talk to the api endpoints,
and that seems to be where it's failing.

As the machine is still up after bootstrap fails, you should be able
to ssh in and confirm if the routing works. Otherwise, look in
/var/log/juju and check with upstart if jujud had issues starting up.

We have a bug filed already about the final error message being
unhelpful, that I've been meaning to get round to fixing for a while:

https://bugs.launchpad.net/juju-core/+bug/1222413

Martin

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Test suite on windows now voting on trunk

[Martin Packman]

Last week we resolved a bunch of issues with the test suite, flipped
the switch so it will now mark trunk builds with windows test failures
as not suitable for releasing.

There are three groups of failures in the latest run that still need addressing:

http://reports.vapour.ws/releases/2549/job/run-unit-tests-win2012-amd64/attempt/276


FAIL: upgrade_test.go:457: UpgradeSuite.TestUpgradeStepsHostMachine
...
upgrade_test.go:462:
go func() { c.Check(a.Run(nil), gc.IsNil) }()
... value *errors.errorString = errors.errorString{s:cannot create
juju run symlink: symlink
c:\\users\\...\\var\\lib\\juju\\tools\\machine-0\\jujud.exe
c:\\users\\...\\juju-run470635483: GetFileAttributesEx
c:\\users\\...\\var\\lib\\juju\\tools\\machine-0\\jujud.exe: The
system cannot find the path specified.} (...)

This is an unreliable test on several platforms:

https://bugs.launchpad.net/juju-core/+bug/1444576

However, there does seem to be an actual issue to resolve here. For
now, skipping it as TestUpgradeStepsStateServer already is on windows
should be fine.


Two failures in MeterStateSuite during setup, with no obvious cause in the logs:

FAIL: meterstatus_test.go:98:
MeterStateSuite.TestMeterStatusWatcherRespondstoMeterStatus
FAIL: meterstatus_test.go:105:
MeterStateSuite.TestMeterStatusWatcherRespondsToMetricsManager
...
meterstatus_test.go:117:
assertMeterStatusChanged(c, watcher)
meterstatus_test.go:125:
c.Fatalf(expected event from watcher by now)


Lots of failures on uniter tests, basically all from not finding the
juju tools within hooks:

FAIL: uniter_test.go:1352: UniterSuite.TestActionEvents
FAIL: uniter_test.go:1894: UniterSuite.TestLeadership
FAIL: uniter_test.go:1755: UniterSuite.TestReboot
FAIL: util_windows_test.go:87: UniterSuite.TestRunCommand
FAIL: uniter_test.go:1278: UniterSuite.TestUniterCollectMetrics
FAIL: uniter_test.go:291: UniterSuite.TestUniterConfigChangedHook
FAIL: uniter_test.go:399: UniterSuite.TestUniterDyingReaction
FAIL: uniter_test.go:675: UniterSuite.TestUniterErrorStateUpgrade
FAIL: uniter_test.go:365: UniterSuite.TestUniterHookSynchronisation
FAIL: uniter_test.go:165: UniterSuite.TestUniterInstallHook
FAIL: uniter_test.go:1267: UniterSuite.TestUniterMeterStatusChanged
FAIL: uniter_test.go:1205: UniterSuite.TestUniterRelationErrors
FAIL: uniter_test.go:1085: UniterSuite.TestUniterRelations
FAIL: uniter_test.go:208: UniterSuite.TestUniterStartHook
FAIL: uniter_test.go:448: UniterSuite.TestUniterSteadyStateUpgrade
FAIL: uniter_test.go:1674: UniterSuite.TestUniterSubordinates
...
[LOG] 0:03.493 INFO unit.u/0.install
c:\users\...\agents\unit-u-0\charmjuju-log.exe
deadbeef-0bad-400d-8000-4b1d0d06f00d install
[LOG] 0:03.493 INFO unit.u/0.install 'juju-log.exe' is not recognized
as an internal or external command,
[LOG] 0:03.493 INFO unit.u/0.install operable program or batch file.
...
waiting for hooks: []string{install, config-changed, start}
ctx.hooksCompleted: []string{fail-install}
...
util_test.go:718:
c.Fatalf(never got expected hooks)


The plan is to resolve these last issues and block on any regressions.
As we start running feature branches through CI, these too will be
marked as unmergable if they fail unit tests on windows.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Vivid init system bug status

[Martin Packman]

On 17/04/2015, John Meinel j...@arbash-meinel.com wrote:

 Do you know what version of Go you're using in your custom builds?

The go compiler version was the key, our CI and Oleg are using 1.3.3 on vivid.

Frustrating, this issue is in large part to vivid tests not being
reliable enough to gate on:

https://bugs.launchpad.net/juju-core/+bug/1441302

Menno's branch had good tests, that correctly failed on our
infrastructure, using go 1.3 on vivid:

http://reports.vapour.ws/releases/2543/job/run-unit-tests-vivid-amd64/attempt/455

FAIL: upstart_test.go:395: IsRunningSuite.TestUpstartNotInstalled

upstart_test.go:400:
c.Assert(err, jc.ErrorIsNil)
... value *errors.Err = errors.Err{message:,
cause:(*os.PathError)(0xc208142240),
previous:(*os.PathError)(0xc208142240),
file:github.com/juju/juju/service/upstart/upstart.go, line:43}
(fork/exec /foo/bar/not-exist: no such file or directory)
... error stack:
fork/exec /foo/bar/not-exist: no such file or directory
github.com/juju/juju/service/upstart/upstart.go:43:

OOPS: 19 passed, 1 FAILED

No one is looking at these results, because they are filled with junk
that hasn't been addressed, despite the fact we want to be doing this
release on vivid.

So, the actual issue is the error returned from the os.exec module
changed in go 1.3 - and somewhat compounded by the map ordering change
making hitting that case inconsistent and messing with our debugging
attempts.

I have put up a merge proposal making the error handling robust across
go versions, and removing the map interation confusion:

https://github.com/juju/juju/pull/2094

This should be on the 1.23 branch shortly.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Vivid init system bug status

[Martin Packman]

Update on where we're at with the local provider init system breakage on vivid.

Oleg reported the following bug againsts 1.23:

https://bugs.launchpad.net/juju-core/+bug/1443440

Short version, juju sometimes fails when trying to operate on the
mongo service, during either bootstrap or destroy-environment. The
error is trying to run an upstart file, when the vivid system doesn't
have it because instead it's using systemd:

   fork/exec /sbin/initctl: no such file or directory

Theory of the bug, the code is looking for an error when running the
script, but gets a different error than expected when the script does
not exist. The randomness isu due to a loop over a map of init
systems, if we happen to try systemd before upstart all is good,
otherwise it blows up.

Based on this, Menno landed the following fix before 1.23 to fix the issue:

https://github.com/juju/juju/pull/2083

The problem is... somehow we can still hit the issue.

Oleg has done this two ways:

* Downloading the release tarball from launchpad, then building and
packaging locally and testing
* Installing the ppa:juju/proposed packages and testing

In addition, *one* of the CI test runs on the rev that included this
fix hit the problem:

http://reports.vapour.ws/releases/2543/job/local-deploy-vivid-amd64/attempt/426

Unfortunately we can't reproduce the failure otherwise on our vivid slaves.

Further, on a vivid machine in canonistack Oleg created, with minimal
setup, we could hit the issue often with our packaged juju versions,
but not go binaries built by Menno or I with added instrumentation.

So, as far as we can tell the bug should be fixed, somehow is not, but
we really do seem to be using the correct version of the source which
includes that code change.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Merge gating for more juju subprojects

[Martin Packman]

On 25/03/2015, Rick Harding rick.hard...@canonical.com wrote:
 If you're using the github lander make sure the user has made it's org
 membership public.

Thanks Rick, that was is. Thought I'd marked the bot's membership of
go-goose at public previously, but had not.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Merge gating for more juju subprojects

[Martin Packman]

On 13/03/2015, Martin Packman martin.pack...@canonical.com wrote:

 That said, if there are any packages you think we should *not* gate
 landings on their test suite passing yet, please say now.

With the lack of objections, I've gone ahead and made the more active
projects in the juju namespace on github merge gated.

From now on, to land branches for:

juju/errors
juju/loggo
juju/names
juju/testing
juju/utils

Add $$merge$$ in a comment on the pr and wait for the bot to run tests.

Please poke me if you notice anything amiss.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Merge gating for more juju subprojects

[Martin Packman]

On 24/03/2015, Andrew Wilkins andrew.wilk...@canonical.com wrote:

 Katherine just tried to $$merge$$ a go-goose branch, and it looks like the
 script is broken: https://github.com/go-goose/goose/pull/3

 Tests passed, but merge failed. Ian merged manually.

Hmm, would have been nice to send it through again to see what the
github api was complaining about.

ERROR: Failed to merge: {u'documentation_url':
u'https://developer.github.com/v3/pulls/#merge-a-pull-request-merge-button',
u'message': u'Not Found'}

Not a super-useful error back from the merge. Maybe the bot just
doesn't have the right perms currently? Landing in the juju namespace
is working.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Merge gating for more juju subprojects

[Martin Packman]

I've put up a gating job for goose on our jenkins slave for juju:

http://juju-ci.vapour.ws/job/github-merge-goose/

It will likely need some more work, but I fake-landed Katherine's
proposed branch and it passed. The switch over steps are disabling
direct landing for most/all contributers and getting everyone using
$$merge$$ as with juju.

Many of the other juju subprojects are also suitable for sending
through automated unit tests before landing, and I can easily make a
whole bunch more jobs for whichever packages we want to gate landing
on.

There are a few caveats:
* There isn't proper isolation with lxc yet, so test suites that do
dodgy things are still an issue.
* Most packages don't use dependencies.tsv so the deps are tied to the
merge job currently.
* Anything that needs a large external dependency like mongo isn't
catered for at the moment.

That said, if there are any packages you think we should *not* gate
landings on their test suite passing yet, please say now.

Thanks!

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Trivial bugs hurting progress

[Martin Packman]

On 04/03/2015, Andrew Wilkins andrew.wilk...@canonical.com wrote:
 Hi all,

 Ian asked me to mail the list about a couple of bugs that managed to get
 past the bot; first so that we can all be mindful of these sorts of bugs,
 and second to highlight the fact that they could have been caught by the
 bot.

 The bugs were fixed in this branch: https://github.com/juju/juju/pull/1738
  - random. iteration Map is

This was being addressed as part of bug 1427149.

https://bugs.launchpad.net/juju-core/+bug/1427149
http://reviews.vapour.ws/r/1048/

Your patch means that branch now needed updating.

  - invalid printf-style formatting will cause go vet to fail

So, do we want to revisit making the bot fail on go vet errors? That's
trivial flip.

 The bot is currently running (I think) Go 1.2. I'm running 1.4, Ian's
 running 1.3, and I'm sure Dave's running tip ;)  Go 1.3+ made map iteration
 less deterministic, so these sorts of bugs are much more likely to occur
 after 1.2. It'd be good to either bump the compiler version to something
 more recent, unless we want to gate things on multiple compilers (maybe gc
 and gccgo, seeing as we currently use both).

Curtis and I have talked about also doing a ppc64 test run as part of
the gating job, that gets us the map ordering stuff as a newer go
would, and other gccgo quirks covered as well. We don't want to bump
the compiler version yet I think, as we want to be able to build with
the trusty toolchain still, and not accidentally let in regressions by
depending on newer gos.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Reports artifact links added

[Martin Packman]

I have deployed code on juju-reports that directly links all build
artifacts from the test detail page.

For instance see one of the recent runs:

http://reports.vapour.ws/releases/2312

In the Last build column darker shading indicates you can hover the
cell to reveal more links, which can include:

* Previous builds and their logs - a number of jobs may be run
multiple times in order to get a pass, the earlier failures are now
easy to access.
* Juju logs from failed tests such as the machine and unit logs.
* The binaries used in the tests for all platforms, in the build-* jobs.

The direct link to the jenkins job pages is also back, note these will
expire unlike the artifact links, so may lead nowhere for historic
runs.

I plan to make the gziped logs directly openable in the browser and
tweak the css display a little more. I'm like to hear any more
suggestions or feedback on what would be useful to see here.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Linker error on gccgo/ppc64 in test

[Martin Packman]

Landings were blocked today by a bug [1381671] where a recently landed
test file was failing to build on the ppc64 slave.

As far as I can see, there's nothing obviously broken, and the package
follows the same basic layout as its api/ siblings. So, for now, I've
skipped building this file to unblock trunk.

The linker failure itself looks similar to an issue [1289067] Dave
fixed upstream earlier in the year. However the fixed gccgo package is
indeed being used on the slave, so this requires extra investigation:

# testmain
/tmp/go-build988982231/github.com/juju/juju/api/reboot/_test/github.com/juju/juju/api/libreboot.a(reboot.o):
In function 
`github_com_juju_juju_api_reboot.ClearReboot.pN37_github_com_juju_juju_api_reboot.State':
/home/ubuntu/juju-core_1.21-alpha2/src/github.com/juju/juju/api/reboot/reboot.go:67:
multiple definition of
`github_com_juju_juju_api_reboot.ClearReboot.pN37_github_com_juju_juju_api_reboot.State'
/tmp/go-build988982231/github.com/juju/juju/api/libreboot.a(reboot.o):/home/ubuntu/juju-core_1.21-alpha2/src/github.com/juju/juju/api/reboot/reboot.go:67:
first defined here
...

I cannot reproduce the error with gccgo on amd64 or armhf, only on the
pcc64 machine.

Any ideas?

Martin


[1381671] reboot tests fail to build on gccgo
https://bugs.launchpad.net/juju-core/+bug/1381671

arm64 multiple definition of `launchpad.net_juju_core_cmd._.0
https://bugs.launchpad.net/juju-core/+bug/1289067

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Landing job uses pre-push script

[Martin Packman]

The landing job for juju now uses the pre-push script from the juju
branch before running tests. This means it's running go vet over the
source, and I've fixed a couple of errors on master.

What it doesn't do currently is fail the landing if vet reports
issues, as the script doesn't fail in this case. If we want to make go
vet issues gate the landing of code, we can just land that change to
the pre-push script.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

juju/charm master has changed

[Martin Packman]

Earlier I changed the revision that the master branch of the
github.com/juju/charm project refers to. In practical terms, I hope
this should not matter to anyone, as the vN branches are the
actively developed ones under the new versioning scheme. If there is
any issue, please let me know.

The reason for doing it, despite master basically being an abandoned
branch, is 'master' still has a special status. When fetched using go
get not using the redirector (as is the case for the 1.20 branch)
master is the initial state. As that still imported
launchpad.net/goyaml the juju tarball build process was pulling in
that as well. So, to avoid that issue I have pushed the current tip of
1.20 over master.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Lag for unblocking CI

[Martin Packman]

On 01/09/2014, Matthew Williams matthew.willi...@canonical.com wrote:
 Thanks Martin, yeah things have come up since it tried to land, I'll try
 again later today and let you know if there are any problems

Your workaround of making a new pull request was fine. Underlying
issue is the github comments api is paginated, and the lander code was
only ever fetching the first page. Your proposal just had a lot of
comments. :)

I've proposed a fix for the lander, so this shouldn't be an issue in future:

https://github.com/juju/jenkins-github-lander/pull/18

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Lag for unblocking CI

[Martin Packman]

On 01/09/2014, John Meinel j...@arbash-meinel.com wrote:

 Is there some amount of caching going on somewhere? (I also noticed it took
7minutes for the bot to notice a $$merge$$ request, so maybe it does the
 check somehow asynchronously to processing the requests?)

Maybe it was something timing related. At any rate, manually running
the script shows it's happy now.

$ python check_blockers.py master 562
No blocking bugs

Can do this by getting the script from lp:juju-ci-tools with the
params as what you want to merge into, and the pull request number.

I see the proposal has still not landed, despite some subsequent
comments with the merge directive. However, William has also left some
additional comments, so I guess we leave it till that's all addressed?

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: CI tests are installing Juju on the CI machine ...

[Martin Packman]

On 01/08/2014, David Cheney david.che...@canonical.com wrote:
 Creating config file /etc/mercurial/hgrc.d/hgext.rc with new version
 Setting up zip (3.0-4) ...
 Setting up juju-core (1.20.1-0ubuntu1~12.04.1~juju1) ...
 update-alternatives: using /usr/lib/juju-1.20.1/bin/juju to provide
 /usr/bin/juju (juju) in auto mode.
 Setting up rsyslog-gnutls (5.8.6-1ubuntu8.6) ...

 ^ it's leaking in from a dependency, this is risky to say the least.

This is because of our Makefile, which includes juju-local[1] in the
packages installed as part of the 'install-dependencies' target. I'm
very tempted to just change it to be the juju-local dependencies on
minus juju-core itself, but that put the onus on us for keeping that
in sync.

As our testing is careful to run the right copy of juju (the one just
installed, not the system package), this isn't actually causing
breakage, so I'll defer poking this for now.

Martin


[1] juju-local in trusty
http://packages.ubuntu.com/trusty/juju-local

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Possible landing bot issues

[Martin Packman]

On 31/07/2014, David Cheney david.che...@canonical.com wrote:

 1. the bot doesn't appear to NACK changes that are not gofmt'd, this
 means changes land which others cannot fix because their .gitconfig
 prevents them.

I've proposed a change to the build script to reject non-gofmt-happy changes:

https://code.launchpad.net/~gz/juju-release-tools/tarball_check_gofmt/+merge/229287

Have installed this on the bot and it seems to be working cleanly:

http://juju-ci.vapour.ws:8080/job/github-merge-juju/212/console

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Landing changes for juju subprojects

[Martin Packman]

I'm switching our various subprojects in the juju namespace on github
over to having landings gated by jenkins jobs that run their test
suites.

For most of us this should be pretty easy to deal with, the 'Merge
pull request' button will go away on the webpage as projects get
switched. When it does, use the $$merge$$ magic string to land
instead. For team leads who don't get that handy indicator, ask on IRC
or check the collaborators for the project (eg.
https://github.com/juju/juju/settings/collaboration) and avoid
manual landings when the Bots team is present.

There may be some teething issues, especially around dependency
management for some of the subprojects, if anyone runs into issues
please report them here or on IRC to me.

Thanks!

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: New flakey juju tests

[Martin Packman]

On 10/07/2014, Dimiter Naydenov dimiter.nayde...@canonical.com wrote:

 I'll take a look at both of these, since I approved the first and
 landed the second. Sorry about the flakiness :/

Thanks Dimiter!

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

New flakey juju tests

[Martin Packman]

We've done a lot of work recently improving the reliability of our
test suite. Unfortunately, feature work has been introducing new tests
that are intermittently failing.

http://juju-ci.vapour.ws:8080/job/github-merge-juju/1/console

FAIL: machine_test.go:1750: MachineSuite.TestWatchInterfaces
...
machine_test.go:1808:
wc.AssertOneChange()
testing/watcher.go:76:
c.Fatalf(watcher sent unexpected change: (_, %v), ok)
... Error: watcher sent unexpected change: (_, true)

...
FAILgithub.com/juju/juju/state  123.265s

This test was added in pr 207.

https://github.com/juju/juju/pull/207


http://juju-ci.vapour.ws:8080/job/github-merge-juju/3/console

FAIL: server_test.go:96: serverSuite.TestAPIServerCanListenOnBothIPv4AndIPv6
...
server_test.go:104:
c.Assert(err, gc.IsNil)
... value *net.OpError = net.OpError{Op:listen, Net:tcp,
Addr:(*net.TCPAddr)(0xc2107e9f00),
Err:(*os.SyscallError)(0xc21027f560)} (listen tcp :54321: bind:
address already in use)

...
FAILgithub.com/juju/juju/state/apiserver30.369s

This test was added in pr 224.

https://github.com/juju/juju/pull/224


Can we have another look at these tests and fix them up to be properly
robust? I don't want to back out changes that have been in trunk for a
while, but we can't leave unreliable tests on trunk. Thanks,

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Landing bot job changed

[Martin Packman]

I have switched the jenkins landing bot to a new job, as part of the
work on getting it faster and more reliable. If you want to check back
on old results, they are preserved for now under a different job name:

http://juju-ci.vapour.ws:8080/job/github-merge-juju-old/

I'm actually having second thoughts in writing this, maybe just
renaming the job to keep the links in existing prs correct would have
been better. I can still do that if anyone has strong opinions.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: How to show diff for a rev?

[Martin Packman]

On 18/06/2014, John Meinel j...@arbash-meinel.com wrote:

 So the only syntax that reliably gives me what I want is:
  git dif 348c104^ 348c104
 I was hoping there would be a better shortcut for it. Does anyone have some
 more voodoo that I could use to avoid having to type the same thing twice?

That's what I've always done. Often have shas (or sha heads) on my clipboard...

Seems like you could do something like this though:

$ git config --global alias.d '!sh -c git diff $1^ $1 -'
$ git d 348c104

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: naming repos on github

[Martin Packman]

On 06/06/2014, Nate Finch nate.fi...@canonical.com wrote:

 As for ditching go get... yes and no (mostly no).  go get is not really
 intended for use in contributing to an external project.  It's a download
 and install this application.  If anything, we should be making our code
 *more* happy with go get, so if Go developers want to try out juju, they
 can do so in an intuitive fashion.  It would also remove a lot of our pain
 if we stopped hitting ourselves in the head by breaking trunk of our repos
 and depending on godeps to fix up trunk.  IMO godeps should be used on
 release branches only, to pin the revision numbers for repeatable builds.
  Trunk of all our repos should always build, the fact that it doesn't is
 our own fault.

It's not just our fault tip of everything doesn't build, we do have
dependencies controlled by other people.

For instance: https://github.com/juju/juju/issues/43

To remain compatible with go 1.1 we need an old gocrypto, so, we can't
build against tip of that repo.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Membership changes on github

[Martin Packman]

I have changed our memberships on teams under the github juju
organisation, to limit commit access to the juju branch to the landing
bot. Sorry for the extra email kipple from this.

Everyone is now a member of @juju/hackers and everyone except team
leads have been removed from owners.

Branches that have gated landings have @juju/bots only as
collaborators. Branches we still need to test and merge manually have
@juju/hackers as collaborators. The juju-gui project is both - you
guys may want to switch to bot only.

Owners can still do everything. This means yes, team leads, you can
still corrupt the central repository if you do bad things with git.

As a side effect, you'll notice you've been subscribed to lots of
repositories you may not have been previously. You can go and unwatch
any you don't want to receive email about, just go to the page (such
as https://github.com/juju/jenkins-github-lander) and on the
'Unwatch' dropdown select either Not watching or Ignoring.

There are still a few issues with the current setup - most of us now
can't do useful things like add a new repository under the juju
namespace, we'll need to ask our team lead to do it instead. We also
can't bypass the bot if needed, and team leads lack a safety net by
default. One option would be to create a role account, jujuowner,
remove *everyone* from owners but that, and share the credentials to
that account. Would this be preferable to the current setup?

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Switch to github beginning

[Martin Packman]

I am going to stop the tarmac landing bot now and begin the process of
switching our juju repository from launchpad to github.

Note that any branches currently proposed will need their changes
imported into a new git branch in order to land, this can be done once
the git master is up and branched locally. Steps following from the
CONTRIBUTING setup will be something like:

$ cd $GOPATH/src
$ bzr branches launchpad.net/juju-core
* trunk
  wip_feature
$ bzr switch -d launchpad.net/juju-core wip_feature
$ (cd github.com/juju/jujugit branch)
* master
$ (cd github.com/juju/jujugit checkout -b wip_feature)
$ (cd launchpad.net/juju-corebzr diff -rancestor:co:trunk)|(cd
github.com/juju/jujupatch  -p0 --merge)

Then resolve any conflicts, stage and commit.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Juju is on github

[Martin Packman]

Juju development is now done on github:

https://github.com/juju/juju

See the updated CONTRIBUTING doc for the basics. To land code you want
the magic string $$merge$$ in a comment on the pull request so the
jenkins bot picks it up.

Note that the bot is currently taking around three times as long to
run the tests, as we can't run suites in parallel due to mongo test
flakiness. We'll work on improving that, but feel free to find and fix
slow tests while you wait. :)

As a team we're going to have some pain and discovery this week,
please share any git tips and information you find useful as you go
along. Thanks!

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

utils/ssh updates

[Martin Packman]

I've poked utils/ssh a bit during gophercon.

Unfortuately the update to use tip of go.crypto can't land right now
as it requires a couple of standard library crypto bits that go 1.1.2
lacks. Perhaps we can talk about bumping the requirement for building
trunk juju-core to go 1.2 and adding that in our golang ppa.

The branch to not generate lots of rsa keys for the tests did land,
which makes the testrun a little faster... or a lot faster on arm.

Before:
http://paste.ubuntu.com/7338971/
ok  launchpad.net/juju-core/utils/ssh   375.048s

After:
http://paste.ubuntu.com/7338978/
ok  launchpad.net/juju-core/utils/ssh   4.533s


My chromebook is now much happier... except when it has to run lbox.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Pending fix for bug 1310258

[Martin Packman]

I put up a branch to fix one of our 1.19.1 blocking bugs earlier today:

APIHostPorts returns 'localhost' and '127.0.0.1'
https://bugs.launchpad.net/juju-core/+bug/1310258
https://codereview.appspot.com/90310043/

As I'm on a plane for most of tomorrow, feel free to mark approved to
land if it gets reviewed. Otherwise, I'll try and follow up after I
land evening US (Mountain) time.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: [Canonical-juju-qa] Cursed (final): #1173 lp:juju-core r2511 (manual-deploy)

[Martin Packman]

On 31/03/2014, Curtis Hovey-Canonical cur...@canonical.com wrote:

 Aaron will capture some logs of what is wrong and investigate issues
 with CI. Can someone investigate Juju?

 Juju 1.18.0 will be based on r2510 if this bug is not fixed in time.

Looking at the diff of r2511 I'm pretty doubtful it broke deploying
wordpress. It changes one function only used in tests, and the tests
that use it, to use a different way of specifying the parameters. Have
you verified going back to r2510 actually makes the failing CI test
succeed?

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Landing bot alive again

[Martin Packman]

At second attempt, the landing bot is now running again, moved over to
lcy01. Feel free to mark any branches 'Approved' to land them.

As part of the setup, several people's keys were added to the tarmac
machine, but most management should be possible remotely by setting
juju service config now.

First source the credentials of the juju-tools-project user:

  $ source .gobotcreds

Then get the juju environment details from the swift bucket (note,
this was set up with 1.16 so may need some tweaking for a trunk
client).

  $ (cd ~/.juju/environments  swift download tarmac gobotnext.jenv)

Then fetch the tarmac service config too:

  $ swift download tarmac gobotnext.yaml

If needed update the config-changed-script config key in that file.

  $ juju set  -e gobotnext --config gobotnext.yaml tarmac

This will update packages and add new go dependencies automatically.


The bot can be setup from scratch again using the custom tarmac charm
and the service config from before.

The tarmac charm changes are mostly just adding bzr 2.6 support and a
script injection mechanism, branch is at
lp:~juju/charms/precise/tarmac/gobotnext.

Only minimal environments.yaml needed, juju can generate most things
for the jenv.

gobotnext:
type: openstack
ca-cert-path: ~/.juju/gobot-cert.pem
ca-private-key-path: ~/.juju/gobot-private-key.pem

We could leave juju to generate the keys too, rather than using those
ones from tarmac container on swift.

Then it's just:

  $ juju bootstrap -e gobotnext

  $ juju set-constraints -e gobotnext mem=4G cpu-cores=2

  $ juju deploy -e gobotnext --config=gobotnext.yaml local:precise/tarmac


Hopefully we won't need to do much more fiddling before we're able to
move to a new setup with jenkins alongside the CI work.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Landing bot back online

[Martin Packman]

Our landing bot is operational again after the canonistack lcy02
maintenance earlier today.

There were quite a few hitches along the way, some of which are
probably attributable to the fact we're still running an ancient juju
1.11.0 in that environment.

First up the state server came up without mongo from running out of
disk space... which made double sure by filling up the logs with
attempting to connect to mongo.

Then the provider-state file somehow didn't have anything in it, so I
manually uploaded a reference to the state server id to swift. That
got status working, but with instance-state: error. I added logging
to the client, mongo instance data claimed not to find the machines.
But a restart of the second machine and some other fiddling and
mysteriously everything was fine.

I then got one successful config-changed through to update the tarmac
conf - unfortunately missing one part that had been updated locally
and with a bad change to the test commands. From there, no more `juju
set` invocations actually resulted in the hook firing, though the unit
agent did pick up the first step.

(Misc note, not being able to `juju set` what you `juju get` is just
frustrating).

So, I looked at upgrading the juju version to something current... Not
a good idea. There have been so many tools location specification
changes in the mean time, I couldn't get a combination that both a
recent client and the old agents could understand. That also left the
upgrade stuck in an uncompleteable state, so I hacked the
agent-version back to what it was before.

After that, for whatever reason config-changed started firing again,
so I finally fixed up the tarmac config and confirmed changes could
land again.

We should probably set up a new environment with a current juju after
fixing up the tarmac charm to have some more of the things we needed
to manually hack in last time around.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: new dependency

[Martin Packman]

On 04/02/2014, Tim Penhey tim.pen...@canonical.com wrote:

 Docs need a little work.

Can you give us an inspirational example of an error message made
clearer at least?

I know Roger has hacked on something along these lines in the past,
I'm curious to hear his thoughts on your stab at tracebacks versus the
approach he tried.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: Juju 1.17.1 milestone

[Martin Packman]

On 21/01/2014, Curtis Hovey-Canonical cur...@canonical.com wrote:

 I believe milestone 1.17.1 has about 4 weeks of issues targeted to it.
 https://launchpad.net/juju-core/+milestone/1.17.1

I have retargetted all bugs that are not actually landed or about to
land on trunk against the new 1.18.0 milestone. This should mean we
can get a 1.17.1 release done, as desired.

In general, I think we should target bugs against the next stable
release milestone for planning, and only move them to the upcoming
development release milestone when there's actually a prospect of
something landing. Others may have a different opinion on this.

For now, if you land something in time for 1.17.1 please bump the
related bug back to that milestone.

 I doubt we can release a 1.18.0 this month without cutting scope. That
 may not be possible as API-everywhere really must be complete to
 release 1.18.0.

Playing with milestones hasn't changed the fact we have a lot of work
targeted at 1.18, unfortunately not much jumped out at me as
deferrable.

Martin

-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Re: How to get private-address in juju status output?

[Martin Packman]

On 27/11/2013, Andreas Hasenack andr...@canonical.com wrote:
 Hi,

 I have an instance on openstack (canonistack) that got a floating ip, and
 that address is now showing as public-address. I guess that's ok.

As discussed on IRC, this is the expected behaviour, though with the
limited details juju currently exposes it is surprising. For clouds
that do not hand out public ip addresses by default, juju falls back
to using a private 10. address or whatever is available. When you add
the floating ip, it realises that is in fact a public address and
starts returning that instead.

 Except it's not reachable via ssh anymore (nor juju ssh). For some reason
 (bug? deployment issue?) public addresses are not routeable from within
 that cloud itself. You have to use the private address.

That does seem like a bug, either in the cloud or possibly your ssh config.

 So, how do I get the private address in the juju status output?

Some changes still need to land to make that possible. The internal
model now understands machines can have a variety of addresses
associated, and tracks information about them beyond the rather
artifical public/private split. The tricky part is we need to get that
information into `juju status` without breaking existing tools that
expect the current forms or bloating the output too much.

Martin

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju