Cool! I'm interested in these Charms. Are they online somewhere? 2016-05-31 16:59 GMT+02:00 Tom Barber <t...@analytical-labs.com>:
> Just to clear up this thread. > > Thanks to Kos we tracked the the condition, where Drill was setting its > config before the whole quorum was available. > > For reference I ended up implementing something a bit like: > > https://github.com/juju-solutions/layer-apache-spark/blob/master/reactive/spark.py#L159 > > Thanks guys, I now have a working basic Apache Drill cluster \o/ > > Tom > > -------------- > > Director Meteorite.bi - Saiku Analytics Founder > Tel: +44(0)5603641316 > > (Thanks to the Saiku community we reached our Kickstart > <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> > goal, but you can always help by sponsoring the project > <http://www.meteorite.bi/products/saiku/sponsorship>) > > On 31 May 2016 at 13:34, Konstantinos Tsakalozos < > kos.tsakalo...@canonical.com> wrote: > >> Hm... I do not see anything wrong with this part of the code. Looking at >> [0] you should be getting all three Zookeeper units/conversations. >> >> Would it be possible to share the charm you are building so that I can >> test it locally or ping us on IRC channel #juju at freenode for a pair >> debugging session? >> >> >> [0] >> https://github.com/juju-solutions/interface-zookeeper/blob/master/requires.py#L40 >> >> On Tue, May 31, 2016 at 3:19 PM, Tom Barber <t...@analytical-labs.com> >> wrote: >> >>> Okay, slightly different question: >>> >>> zookeeper/0 active idle 2.0-beta7 8 >>> 2181/tcp,9998/tcp 10.106.143.126 Ready (3 zk units) >>> zookeeper/1 active idle 2.0-beta7 9 >>> 2181/tcp,9998/tcp 10.106.143.234 Ready (3 zk units) >>> zookeeper/2 active idle 2.0-beta7 10 >>> 2181/tcp,9998/tcp 10.106.143.133 Ready (3 zk units) >>> >>> I have 3 ZK units and I used the example here: >>> https://github.com/juju-solutions/interface-zookeeper for the client >>> config so my function looks like: >>> >>> @when('zookeeper.ready') >>> @when_not('charm.configured') >>> def configure(zookeeper): >>> zklist = '' >>> for zk_unit in zookeeper.zookeepers(): >>> zklist += add_zookeeper(zk_unit['host'], zk_unit['port']) >>> zklist = zklist[:-1] >>> >>> But it only iterates once even though 3 are listed: >>> >>> a) did I do something wrong >>> b) is this due to some leader election thing and is actually correct? >>> c) another answer? >>> >>> Thanks >>> >>> Tom >>> >>> -------------- >>> >>> Director Meteorite.bi - Saiku Analytics Founder >>> Tel: +44(0)5603641316 >>> >>> (Thanks to the Saiku community we reached our Kickstart >>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> >>> goal, but you can always help by sponsoring the project >>> <http://www.meteorite.bi/products/saiku/sponsorship>) >>> >>> On 31 May 2016 at 12:37, Tom Barber <t...@analytical-labs.com> wrote: >>> >>>> Fair enough, that resolves that then! >>>> >>>> Thanks Kos. >>>> >>>> Tom >>>> >>>> -------------- >>>> >>>> Director Meteorite.bi - Saiku Analytics Founder >>>> Tel: +44(0)5603641316 >>>> >>>> (Thanks to the Saiku community we reached our Kickstart >>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> >>>> goal, but you can always help by sponsoring the project >>>> <http://www.meteorite.bi/products/saiku/sponsorship>) >>>> >>>> On 31 May 2016 at 12:25, Konstantinos Tsakalozos < >>>> kos.tsakalo...@canonical.com> wrote: >>>> >>>>> Ah, it is clear now, thanks. >>>>> >>>>> The Apache Hadoop charms use Zookeeper when in HA mode. The namenode >>>>> is the charm that relates to Zookeeper [0,1]. So to answer your question >>>>> the Hadoop charms use ZK but they do not carry ZK under the hood. ZK >>>>> is treated as separate service/charm. >>>>> >>>>> >>>>> [0] https://github.com/juju-solutions/layer-apache-hadoop-namenode >>>>> /blob/master/metadata.yaml >>>>> [1] https://github.com/juju-solutions/layer-apache-hadoop-namenode >>>>> /blob/master/reactive/namenode_failover.py >>>>> >>>>> On Tue, May 31, 2016 at 1:50 PM, Tom Barber <t...@analytical-labs.com> >>>>> wrote: >>>>> >>>>>> No worries Kos >>>>>> >>>>>> I'm rubbish at explaining stuff properly! >>>>>> >>>>>> My query was more along the lines of "I can see the ZK charms >>>>>> available, but surely the Hadoop charms use ZK under the hood somewhere, >>>>>> do >>>>>> any of them expose the ZK relation so you can hook into that", because it >>>>>> seems like overkill running a second ZK setup if one already exists >>>>>> inside >>>>>> an existing Juju Hadoop deployment. >>>>>> >>>>>> Tom >>>>>> >>>>>> >>>>>> >>>>>> -------------- >>>>>> >>>>>> Director Meteorite.bi - Saiku Analytics Founder >>>>>> Tel: +44(0)5603641316 >>>>>> >>>>>> (Thanks to the Saiku community we reached our Kickstart >>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> >>>>>> goal, but you can always help by sponsoring the project >>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>) >>>>>> >>>>>> On 31 May 2016 at 11:47, Konstantinos Tsakalozos < >>>>>> kos.tsakalo...@canonical.com> wrote: >>>>>> >>>>>>> I am sorry Tom, I replied to your first email. >>>>>>> >>>>>>> Not sure I fully get your question. Not sure what "this" refers to >>>>>>> in "Is there a way to hook this". >>>>>>> >>>>>>> Charms interacting with a Zookeeper quorum need to use this >>>>>>> interface: http://interfaces.juju.solutions/interface/zookeeper/ . >>>>>>> Through the zookeeper interface you can be informed about zookeeper >>>>>>> units >>>>>>> joining and departing the quorum. >>>>>>> >>>>>>> You can have a single Zookeeper service that you can scale to an odd >>>>>>> number of units. Then you can have any charm "requiring" the above >>>>>>> relation >>>>>>> use the zookeeper service. Typically you would need to construct a >>>>>>> connection string or edit some configuration file within your charm in >>>>>>> order to reference the zookeeper units. Have a look at https:// >>>>>>> github.com/juju-solutions/layer-apache >>>>>>> -spark/blob/master/lib/charms/layer/apache_spark.py#L175 >>>>>>> >>>>>>> Thanks, >>>>>>> Konstantinos >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, May 31, 2016 at 1:28 PM, Konstantinos Tsakalozos < >>>>>>> kos.tsakalo...@canonical.com> wrote: >>>>>>> >>>>>>>> Hi Tom, >>>>>>>> >>>>>>>> Have you looked at: https://jujucharms.com/apache >>>>>>>> -zookeeper/trusty/3 >>>>>>>> The apache-zookeeper provides the zookeeper interface, see >>>>>>>> https://api.jujucharms.com/charmstore/v5/trusty/apache >>>>>>>> -zookeeper-3/archive/metadata.yaml >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Konstantinos >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, May 31, 2016 at 1:13 PM, Tom Barber < >>>>>>>> t...@analytical-labs.com> wrote: >>>>>>>> >>>>>>>>> Morning >>>>>>>>> >>>>>>>>> There is a zookeeper interface and a bucket load of Big Data stuff >>>>>>>>> which all must run ZK under the hood. Is there anything that provides >>>>>>>>> the >>>>>>>>> relationit so my charm can hook into ZK? I can't see anything obvious >>>>>>>>> in >>>>>>>>> the Charm Store. >>>>>>>>> >>>>>>>>> Tom >>>>>>>>> -------------- >>>>>>>>> >>>>>>>>> Director Meteorite.bi - Saiku Analytics Founder >>>>>>>>> Tel: +44(0)5603641316 >>>>>>>>> >>>>>>>>> (Thanks to the Saiku community we reached our Kickstart >>>>>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> >>>>>>>>> goal, but you can always help by sponsoring the project >>>>>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>) >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Juju mailing list >>>>>>>>> Juju@lists.ubuntu.com >>>>>>>>> Modify settings or unsubscribe at: >>>>>>>>> https://lists.ubuntu.com/mailman/listinfo/juju >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > >
-- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
