Sorry, forgot I mentioned that. There is an unstable drill charm currently because I'm hacking it around and pushing updates to the store whilst I test adding providers as hooks. Bear with me until tomorrow and I'll dump something into the stable channel. It does work, its very basic, missing auth and stuff, but the guts will spin you up a scalable drill cluster. I'll post some stuff tomorrow morning with details.
Tom -------------- Director Meteorite.bi - Saiku Analytics Founder Tel: +44(0)5603641316 (Thanks to the Saiku community we reached our Kickstart <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> goal, but you can always help by sponsoring the project <http://www.meteorite.bi/products/saiku/sponsorship>) On 31 May 2016 at 16:18, Merlijn Sebrechts <merlijn.sebrec...@gmail.com> wrote: > The Charms for a Drill cluster? Or did I misread that part? > > 2016-05-31 17:16 GMT+02:00 Tom Barber <t...@analytical-labs.com>: > >> Which charms? >> >> -------------- >> >> Director Meteorite.bi - Saiku Analytics Founder >> Tel: +44(0)5603641316 >> >> (Thanks to the Saiku community we reached our Kickstart >> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> >> goal, but you can always help by sponsoring the project >> <http://www.meteorite.bi/products/saiku/sponsorship>) >> >> On 31 May 2016 at 16:16, Merlijn Sebrechts <merlijn.sebrec...@gmail.com> >> wrote: >> >>> Cool! I'm interested in these Charms. Are they online somewhere? >>> >>> 2016-05-31 16:59 GMT+02:00 Tom Barber <t...@analytical-labs.com>: >>> >>>> Just to clear up this thread. >>>> >>>> Thanks to Kos we tracked the the condition, where Drill was setting its >>>> config before the whole quorum was available. >>>> >>>> For reference I ended up implementing something a bit like: >>>> >>>> https://github.com/juju-solutions/layer-apache-spark/blob/master/reactive/spark.py#L159 >>>> >>>> Thanks guys, I now have a working basic Apache Drill cluster \o/ >>>> >>>> Tom >>>> >>>> -------------- >>>> >>>> Director Meteorite.bi - Saiku Analytics Founder >>>> Tel: +44(0)5603641316 >>>> >>>> (Thanks to the Saiku community we reached our Kickstart >>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> >>>> goal, but you can always help by sponsoring the project >>>> <http://www.meteorite.bi/products/saiku/sponsorship>) >>>> >>>> On 31 May 2016 at 13:34, Konstantinos Tsakalozos < >>>> kos.tsakalo...@canonical.com> wrote: >>>> >>>>> Hm... I do not see anything wrong with this part of the code. Looking >>>>> at [0] you should be getting all three Zookeeper units/conversations. >>>>> >>>>> Would it be possible to share the charm you are building so that I can >>>>> test it locally or ping us on IRC channel #juju at freenode for a pair >>>>> debugging session? >>>>> >>>>> >>>>> [0] >>>>> https://github.com/juju-solutions/interface-zookeeper/blob/master/requires.py#L40 >>>>> >>>>> On Tue, May 31, 2016 at 3:19 PM, Tom Barber <t...@analytical-labs.com> >>>>> wrote: >>>>> >>>>>> Okay, slightly different question: >>>>>> >>>>>> zookeeper/0 active idle 2.0-beta7 8 >>>>>> 2181/tcp,9998/tcp 10.106.143.126 Ready (3 zk units) >>>>>> zookeeper/1 active idle 2.0-beta7 9 >>>>>> 2181/tcp,9998/tcp 10.106.143.234 Ready (3 zk units) >>>>>> zookeeper/2 active idle 2.0-beta7 10 >>>>>> 2181/tcp,9998/tcp 10.106.143.133 Ready (3 zk units) >>>>>> >>>>>> I have 3 ZK units and I used the example here: >>>>>> https://github.com/juju-solutions/interface-zookeeper for the client >>>>>> config so my function looks like: >>>>>> >>>>>> @when('zookeeper.ready') >>>>>> @when_not('charm.configured') >>>>>> def configure(zookeeper): >>>>>> zklist = '' >>>>>> for zk_unit in zookeeper.zookeepers(): >>>>>> zklist += add_zookeeper(zk_unit['host'], zk_unit['port']) >>>>>> zklist = zklist[:-1] >>>>>> >>>>>> But it only iterates once even though 3 are listed: >>>>>> >>>>>> a) did I do something wrong >>>>>> b) is this due to some leader election thing and is actually correct? >>>>>> c) another answer? >>>>>> >>>>>> Thanks >>>>>> >>>>>> Tom >>>>>> >>>>>> -------------- >>>>>> >>>>>> Director Meteorite.bi - Saiku Analytics Founder >>>>>> Tel: +44(0)5603641316 >>>>>> >>>>>> (Thanks to the Saiku community we reached our Kickstart >>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> >>>>>> goal, but you can always help by sponsoring the project >>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>) >>>>>> >>>>>> On 31 May 2016 at 12:37, Tom Barber <t...@analytical-labs.com> wrote: >>>>>> >>>>>>> Fair enough, that resolves that then! >>>>>>> >>>>>>> Thanks Kos. >>>>>>> >>>>>>> Tom >>>>>>> >>>>>>> -------------- >>>>>>> >>>>>>> Director Meteorite.bi - Saiku Analytics Founder >>>>>>> Tel: +44(0)5603641316 >>>>>>> >>>>>>> (Thanks to the Saiku community we reached our Kickstart >>>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> >>>>>>> goal, but you can always help by sponsoring the project >>>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>) >>>>>>> >>>>>>> On 31 May 2016 at 12:25, Konstantinos Tsakalozos < >>>>>>> kos.tsakalo...@canonical.com> wrote: >>>>>>> >>>>>>>> Ah, it is clear now, thanks. >>>>>>>> >>>>>>>> The Apache Hadoop charms use Zookeeper when in HA mode. The >>>>>>>> namenode is the charm that relates to Zookeeper [0,1]. So to >>>>>>>> answer your question the Hadoop charms use ZK but they do not >>>>>>>> carry ZK under the hood. ZK is treated as separate service/charm. >>>>>>>> >>>>>>>> >>>>>>>> [0] https://github.com/juju-solutions/layer-apache-hadoop-namenode >>>>>>>> /blob/master/metadata.yaml >>>>>>>> [1] https://github.com/juju-solutions/layer-apache-hadoop-namenode >>>>>>>> /blob/master/reactive/namenode_failover.py >>>>>>>> >>>>>>>> On Tue, May 31, 2016 at 1:50 PM, Tom Barber < >>>>>>>> t...@analytical-labs.com> wrote: >>>>>>>> >>>>>>>>> No worries Kos >>>>>>>>> >>>>>>>>> I'm rubbish at explaining stuff properly! >>>>>>>>> >>>>>>>>> My query was more along the lines of "I can see the ZK charms >>>>>>>>> available, but surely the Hadoop charms use ZK under the hood >>>>>>>>> somewhere, do >>>>>>>>> any of them expose the ZK relation so you can hook into that", >>>>>>>>> because it >>>>>>>>> seems like overkill running a second ZK setup if one already exists >>>>>>>>> inside >>>>>>>>> an existing Juju Hadoop deployment. >>>>>>>>> >>>>>>>>> Tom >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -------------- >>>>>>>>> >>>>>>>>> Director Meteorite.bi - Saiku Analytics Founder >>>>>>>>> Tel: +44(0)5603641316 >>>>>>>>> >>>>>>>>> (Thanks to the Saiku community we reached our Kickstart >>>>>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> >>>>>>>>> goal, but you can always help by sponsoring the project >>>>>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>) >>>>>>>>> >>>>>>>>> On 31 May 2016 at 11:47, Konstantinos Tsakalozos < >>>>>>>>> kos.tsakalo...@canonical.com> wrote: >>>>>>>>> >>>>>>>>>> I am sorry Tom, I replied to your first email. >>>>>>>>>> >>>>>>>>>> Not sure I fully get your question. Not sure what "this" refers >>>>>>>>>> to in "Is there a way to hook this". >>>>>>>>>> >>>>>>>>>> Charms interacting with a Zookeeper quorum need to use this >>>>>>>>>> interface: http://interfaces.juju.solutions/interface/zookeeper/ >>>>>>>>>> . Through the zookeeper interface you can be informed about >>>>>>>>>> zookeeper units >>>>>>>>>> joining and departing the quorum. >>>>>>>>>> >>>>>>>>>> You can have a single Zookeeper service that you can scale to an >>>>>>>>>> odd number of units. Then you can have any charm "requiring" the >>>>>>>>>> above >>>>>>>>>> relation use the zookeeper service. Typically you would need to >>>>>>>>>> construct a >>>>>>>>>> connection string or edit some configuration file within your charm >>>>>>>>>> in >>>>>>>>>> order to reference the zookeeper units. Have a look at https:// >>>>>>>>>> github.com/juju-solutions/layer-apache >>>>>>>>>> -spark/blob/master/lib/charms/layer/apache_spark.py#L175 >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Konstantinos >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, May 31, 2016 at 1:28 PM, Konstantinos Tsakalozos < >>>>>>>>>> kos.tsakalo...@canonical.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Tom, >>>>>>>>>>> >>>>>>>>>>> Have you looked at: https://jujucharms.com/apache >>>>>>>>>>> -zookeeper/trusty/3 >>>>>>>>>>> The apache-zookeeper provides the zookeeper interface, see >>>>>>>>>>> https://api.jujucharms.com/charmstore/v5/trusty/apache >>>>>>>>>>> -zookeeper-3/archive/metadata.yaml >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Konstantinos >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, May 31, 2016 at 1:13 PM, Tom Barber < >>>>>>>>>>> t...@analytical-labs.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Morning >>>>>>>>>>>> >>>>>>>>>>>> There is a zookeeper interface and a bucket load of Big Data >>>>>>>>>>>> stuff which all must run ZK under the hood. Is there anything that >>>>>>>>>>>> provides >>>>>>>>>>>> the relationit so my charm can hook into ZK? I can't see anything >>>>>>>>>>>> obvious >>>>>>>>>>>> in the Charm Store. >>>>>>>>>>>> >>>>>>>>>>>> Tom >>>>>>>>>>>> -------------- >>>>>>>>>>>> >>>>>>>>>>>> Director Meteorite.bi - Saiku Analytics Founder >>>>>>>>>>>> Tel: +44(0)5603641316 >>>>>>>>>>>> >>>>>>>>>>>> (Thanks to the Saiku community we reached our Kickstart >>>>>>>>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/> >>>>>>>>>>>> goal, but you can always help by sponsoring the project >>>>>>>>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>) >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Juju mailing list >>>>>>>>>>>> Juju@lists.ubuntu.com >>>>>>>>>>>> Modify settings or unsubscribe at: >>>>>>>>>>>> https://lists.ubuntu.com/mailman/listinfo/juju >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> -- >>>> Juju mailing list >>>> Juju@lists.ubuntu.com >>>> Modify settings or unsubscribe at: >>>> https://lists.ubuntu.com/mailman/listinfo/juju >>>> >>>> >>> >> >
-- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
