Re: CephFS Backend for Hadoop
2017-07-26 9:17 GMT+02:00 Mark Shuttleworth : > On 26/07/17 07:14, Patrizio Bassi wrote: > > Deploying hadoop via juju in an openstack tenant requires a separate model > (as far as i could design it). > So we may use the new juju 2.2 cross model relation to relate the hadoop > charms to the openstack ceph units. > > does it sound feasible? > > > Yes, that sounds feasible. I'm not sure how Ceph identity / permissions > will work in that case (i.e. who has access to which data, how Ceph will > correlate tenants in OpenStack both through Cinder and through a direct > relationship). In principle though, as long as the networking is arranged > so that IP addresses and routes enable traffic to flow between your tenant > network and your Ceph network, and as long as both sets of machines can see > the Juju controller, they can exchange messages and traffic. > > Mark > Dear Mark, On relation join event we may create a new ceph storage pool dedicated to the incoming unit (i.e. prefixed with the controller/model/unit/charm name by default). Can cephx proto Regarding networking openstack neutron by default block traffic from tenant VM to the admin network which it required to access the same ceph mon/osd. It requires changing neutron or implement an external nat for instance (our solution at the moment) Patrizio -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Is a set state called multiple times?
Hi Juju,
Once I set a state, set_state("here"), I want to make sure its @when
will only be executed ONCE (when "here" from False->True).
So my thought is to remove_state("here") in its @when("here") code
block. If I don't, will this @when be called multiple times if I don't
reset this state? What's the good practice here?
--
Feng xia
Engineer
Lenovo USA
Phone: 5088011794
fx...@lenovo.com
Lenovo.com
Twitter | Facebook | Instagram | Blogs | Forums
--
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
Re: call for testing: relations across Juju models
On 25/07/17 00:54, Dmitrii Shcherbakov wrote: > Hi Patrizio, > > As far as I understand it now, if you configure it right in terms of > networking, it will be possible for both single and multi-cloud cases. > Correct. You can have one application deployed to a model in a Google cloud, and another deployed to a model in AWS for example. Juju correctly determines that workload traffic needs to flow between the workloads' respective public addresses, and also takes care of opening the required firewall ports to allow workload traffic to flow from the requires side of the relation to the provides side. Future work will see Juju optimise the netwrok aspects to that the relation will be set up to use the cloud local addresses if the models and relation endpoints are deployed in a way that this is supported (eg for AWS, same region, tenant, vpc). I also plan to add cross model support to bundles, to make the k8s federation story described below easier. This is not started yet, just an udea on the fairly large cross model relations todo list. > Having only workers on the second cloud is fairly straightforward. > > However, I think the real use-case is to implement k8s federation without > having to replicate etcd across multiple data centers and using > latency-based load-balancing: > > https://kubernetes.io/docs/concepts/cluster-administration/federation/ > https://kubernetes.io/docs/tasks/federation/set-up-cluster-federation-kubefed/ > > This will require charming of the federation controller manager to > have federation control plane for multiple clouds. > > This is similar to an orchestrator use-case in the ETSI NFV architecture. > > Quite an interesting problem to solve with cross-controller relations. > > > > Best Regards, > Dmitrii Shcherbakov > > Field Software Engineer > IRC (freenode): Dmitrii-Sh > > On Mon, Jul 24, 2017 at 4:48 PM, Patrizio Bassi > wrote: > >> Hi All >> >> this is very very interesting. >> >> Is possibile to scale out some units using cross models? >> >> For instance: in a onpestack tenant i deploy a kubernates cluster. Than in >> another tenant i add k8-workers, the add-unit command will refer to the >> parent deployment to get needed params (i.e. master IP address.. juju >> config) >> >> This will be even better in a hybrid cloud environment >> Regards >> >> Patrizio >> >> >> >> 2017-07-24 15:26 GMT+02:00 Ian Booth : >> >>> >>> >>> On 24/07/17 23:12, Ian Booth wrote: On 24/07/17 20:02, Paul Gear wrote: > On 08/07/17 03:36, Rick Harding wrote: >> As I noted in The Juju Show [1] this week I've put together a blog >> post around the cross model relations feature that folks can test out >> in Juju 2.2. Please test it out and provide your feedback. >> >> http://mitechie.com/blog/2017/7/7/call-for-testing-shared-se >>> rvices-with-juju >> >> Current known limitations: >> Only works in the same model >> You need to bootstrap with the feature flag to test it out >> Does not currently work with relations to subordinates. Work is in >> progress > > Hi Rick, > > I gave this a run this afternoon. In my case, I just set up an haproxy > unit in one model and a Nagios server in another, and connected the > haproxy:reverseproxy to the nagios:website. Everything worked exactly > as expected. > > One comment about the user interface: the "juju relate" for the client > side seems a bit redundant, since "juju add-relation" could easily work > out which type of relation it was by looking at the form of the >>> provided > identifier. If we pass a URI to an offered relation in another model, > it could use a cross-model relation, and if we just use normal > service:relation-id format, it could use a normal relation. > > Anyway, just wanted to say it's great to see some progress on this, > because it solves some real operational problems for us. I can't wait > for the cross-controller, reverse-direction, highly-scalable version > which will allow us to obsolete the glue scripts needed to connect our > Nagios server to all our deployed NRPE units! :-) > > > Glad it's working. Multi-controller CMR is already available in the edge snap, but we need >>> to get a new blog post out to describe how to use it. There's also a couple of >>> branches I want to land first to fix a firewalling issue. So expect something in >>> the next few days. If you can live with the filewall issue (which will be imminently >>> fixed), give it a go. The only different with what's mentioned in the blob post >>> above is that you prefix the offer URL with the host controller name. eg, the hello world case... $ juju bootstrap aws foo $ juju deploy mysql $ juju offer mysql:db $ juju bootstrap aws bar $ juju deploy mediawiki $ juju expose mediawiki $ juju
Re: CephFS Backend for Hadoop
Hi {James, Patrizio},
Be careful with using cephfs in production before ceph Luminous though (RC
now).
Although cephfs was declared stable in Jewel,
http://ceph.com/releases/v10-2-0-jewel-released/
"CephFS:
This is the first release in which CephFS is declared stable! Several
features are disabled by default, including snapshots and multiple active
MDS servers"
having multiple active MDS servers is considered experimental for anything
prior to Luminous (12.2.x) and running in 1 active/multiple standby mode
has certain issues (scalability & performance, availability)
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2016-June/010728.html
http://docs.ceph.com/docs/kraken/cephfs/best-practices/
"For the best chance of a happy healthy filesystem, use a single active MDS
and do not use snapshots. Both of these are the default. Note that creating
multiple MDS daemons is fine, as these will simply be used as standbys.
However, for best stability you should avoid adjusting max_mds upwards, as
this would cause multiple daemons to be active at once."
http://docs.ceph.com/docs/master/cephfs/experimental-features/#multiple-active-metadata-servers
"Prior to the Luminous (12.2.x) release, running multiple active metadata
servers within a single filesystem was considered experimental. Creating
multiple active metadata servers is now permitted by default on new
filesystems..."
http://ceph.com/releases/v12-1-0-luminous-rc-released/
"Multiple active MDS daemons is now considered stable. The number
of active MDS servers may be adjusted up or down on an active CephFS file
system."
http://docs.ceph.com/docs/master/cephfs/multimds/
"Even with multiple active MDS daemons, a highly available system still
requires standby daemons to take over if any of the servers running an
active daemon fail."
As far as I can see, a ceph filesystem metadata will be sharded across
multiple MDS servers if configured. So having a multi-mds setup does not
alleviate the need for standby servers and failover - this setup provides
more parallelism but MDS high-availability is still needed for individual
shards.
http://docs.ceph.com/docs/master/cephfs/standby/
"Each CephFS filesystem has a number of ranks, one by default, which start
at zero. A rank may be thought of as a metadata shard. Controlling the
number of ranks in a filesystem is described in Configuring multiple active
MDS daemons
...
Each file system may specify a number of standby daemons to be considered
healthy. This number includes daemons in standby-replay waiting for a rank
to fail (remember that a standby-replay daemon will not be assigned to take
over a failure for another rank or a failure in a another CephFS file
system)."
Also, if you need multiple cephfs file systems, it looks like you will need
this amount of MDS instances: * *
"Each CephFS ceph-mds process (a daemon) initially starts up without a
rank. It may be assigned one by the monitor cluster. A daemon may only hold
one rank at a time. Daemons only give up a rank when the ceph-mds process
stops."
It is interesting how rank assignment is performed by the monitor cluster -
I would very much like to avoid cases where you have multiple or all ranks
of a single file system stored on one machine with multiple active MDS
daemons.
--
I think the scope of work in charm-cephfs would be to:
- implement standby MDS configuration;
- implement multi-active MDS configuration.
Best Regards,
Dmitrii Shcherbakov
Field Software Engineer
IRC (freenode): Dmitrii-Sh
On Wed, Jul 26, 2017 at 9:14 AM, Patrizio Bassi
wrote:
>
> Il giorno mer 26 lug 2017 alle 06:28 James Beedy
> ha scritto:
>
>> Hello all,
>>
>> I will be evaluating CephFS as a backend for Hadoop over the next few
>> weeks, probably start investigating how this can be delivered via the
>> charms in the morning. If anyone has ventured to this realm, or has an idea
>> on what the best way to deliver this might be, I would love to hear from
>> you.
>>
>> Thanks,
>>
>> James
>>
>>
>>
>
> I do!
>
> Probably i won't be able to test before end of the year but i plan to host
> hadoop clusters in openstack tenants and i would like to share the same
> ceph osd providing infrastructural storage to openstack nova/cinder.
>
> Deploying hadoop via juju in an openstack tenant requires a separate model
> (as far as i could design it).
> So we may use the new juju 2.2 cross model relation to relate the hadoop
> charms to the openstack ceph units.
>
> does it sound feasible?
>
> regards
>
> Patrizio
>
>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju
>
>
--
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
Re: CephFS Backend for Hadoop
On 26/07/17 07:14, Patrizio Bassi wrote: > Deploying hadoop via juju in an openstack tenant requires a separate > model (as far as i could design it). > So we may use the new juju 2.2 cross model relation to relate the > hadoop charms to the openstack ceph units. > > does it sound feasible? Yes, that sounds feasible. I'm not sure how Ceph identity / permissions will work in that case (i.e. who has access to which data, how Ceph will correlate tenants in OpenStack both through Cinder and through a direct relationship). In principle though, as long as the networking is arranged so that IP addresses and routes enable traffic to flow between your tenant network and your Ceph network, and as long as both sets of machines can see the Juju controller, they can exchange messages and traffic. Mark -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
