Re: Call for testers: preview of persistent storage support

2017-05-25 Thread Andrew Wilkins 
On Fri, May 26, 2017 at 11:26 AM Patrizio Bassi 
wrote:

> Dear Andrew,
>  what about private clouds such as maas?
>

MAAS is a bit special, because the disks are physically attached to the
machines. If and when we support something like Ceph RBD natively inside
Juju, then that could be detached from one machine and attached to another.

The new storage commands should work with private OpenStack clouds.

Cheers,
Andrew


> Patrizio
>
> Il giorno mer 24 ma 2017 alle 03:38 Andrew Wilkins <
> andrew.wilk...@canonical.com> ha scritto:
>
>> Hi folks,
>>
>> One of the things we're working on for the 2.3 release (not 2.2!) is
>> persistent storage. What this means is the ability to detach storage from a
>> unit, and reattach it to another unit keeping the storage contents intact.
>> We would like to get some feedback before this all gets set in stone.
>>
>> With the changes, removing an application unit will detach storage rather
>> than destroy it as Juju currently does. The storage will then be available
>> for attaching to another unit using "juju attach-storage  "
>> or to a new application unit using "juju deploy  --attach-storage
>> "; or for removal using "juju remove-storage ".
>>
>> For example, I can deploy postgresql on AWS with EBS storage. If I remove
>> the postgresql application, I can add another and attach the storage to it:
>>
>> $ juju deploy postgresql --storage pgdata=100G,ebs
>> Located charm "cs:postgresql-148".
>> Deploying charm "cs:postgresql-148".
>> (wait for postgresql/0 to become active)
>>
>> $ juju remove-application postgresql
>> removing application postgresql
>> - will detach storage pgdata/0
>> (wait for postgresql/0 and machine 0 to be removed)
>>
>> $ juju deploy postgresql postgresql2 --attach-storage pgdata/0 --to
>> zone=
>> Located charm "cs:postgresql-148".
>> Deploying charm "cs:postgresql-148".
>> (wait for postgresql2/0 to become active)
>>
>> If you like, you can confirm for yourself that the data is persisted by
>> logging into the first machine and runing "sudo -u postgres psql", creating
>> some data, and then checking that it is still there from the second machine.
>>
>> (The --to zone=... is required due to a limitation that we will remove by
>> the time 2.3 is released. EBS volumes and EC2 instances must be created in
>> the same AZ, and that's not automatic yet. This is fixed by
>> https://github.com/juju/juju/pull/7378 which, at the time of writing
>> this email, has not yet been merged.)
>>
>> If you have any interest in these changes, please help us make them great
>> by testing out this early release:
>>
>> $ sudo snap install --channel=edge --classic juju-axw
>> $ /snap/bin/juju-axw.juju bootstrap ...
>>
>> The new/updated commands are:
>>  - juju attach-storage  
>>  - juju detach-storage 
>>  - juju remove-storage 
>>  - juju deploy  --attach-storage 
>>
>> (We'll also be adding --attach-storage to the "juju add-unit" command
>> soon.)
>>
>> Thank you!
>>
>> Cheers,
>> Andrew
>> --
>> Juju mailing list
>> j...@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>>
> --
>
> Patrizio Bassi
> www.patriziobassi.it
> http://piazzadelpopolo.patriziobassi.it
>
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev

Weekly Development Summary

2017-05-25 Thread Andrew Wilkins 
Hi all,

It's been a busy week as we continue towards the 2.2-rc1 release. As Tim
said earlier in the week, the release will slip to next week as we continue
ironing out the kinks. One of the main things we've been focusing on this
week is improving scalability and resilience in large deployments, such as
JAAS.

One major issue [0] fixed is that if we overflow the transaction log
collection in Mongo, the controller will now automatically restart to get
all of the watchers back into the appropriate state. A new controller
config attribute, "max-txn-log-size", has been introduced, which allows you
to change the transaction log collection size from the default of 10MB.

Another critical fix is to prevent resource-get from blocking indefinitely
when fetching the resource fails [1]. Juju will now retry fetching
resources for a set number of attempts, and then abort if it continues to
fail, giving the operator a chance to fix the charm resources or charm code
itself.

Work is underway to run one log forwarding worker per model, instead of one
per controller as we have now. This will enable us to optimise the way in
which logs are managed, to improve scalability and performance. Having
per-model log forwarders will also enable future improvements to support
forwarding logs to cloud-specific logging facilities.

A handful of other notable fixes:
 - better validation of custom simplestreams metadata [2]
 - simplified mongo oplog size calculation, to prevent frequent mongo
restarts [3]
 - fixed an issue with add-storage, where the specified pool was not used
in some cases [4]
 - several fixes [5] [6] for model migration related to resources

Last week, Ian mentioned that there would be a snap forthcoming containing
a preview of storage improvements scheduled for the 2.3 release. This was
sent out earlier this week, and we're waiting for feedback. If you have a
moment, please try it out and let us know what you think. Next week we'll
aim to release an update to the snap, with the next bits: adding
--attach-storage to add-unit, and automatically placing VMs in the same AZ
as the attached volumes (at least for AWS).

Thanks for reading!

Cheers,
Andrew

---

Quick links:
  Work Pending: https://github.com/juju/juju/pulls
  Recent commits: https://github.com/juju/juju/commits/develop

[0] https://bugs.launchpad.net/juju/+bug/1692792
[1] https://bugs.launchpad.net/juju/+bug/1627127
[2] https://bugs.launchpad.net/juju/+bug/1690456
[3] https://bugs.launchpad.net/juju/+bug/1677592
[4] https://bugs.launchpad.net/juju/+bug/1692729
[5] https://bugs.launchpad.net/juju/+bug/1692610
[6] https://bugs.launchpad.net/juju/+bug/1692646
-- 
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju-dev