[jira] [Commented] (SLIDER-977) KOYA integration IP clearance

[ https://issues.apache.org/jira/browse/SLIDER-977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005801#comment-15005801 ] Jean-Baptiste Note commented on SLIDER-977: --- Hi, I will provide Criteo&#

Re: Add or remove component dynamically for Slider Application

Hi Paul, While i don't exactly know the specifics of your service, here we use one slider instance per user/tenant for a given service. For instance we have several hbase clusters running on separate accounts. We use chef and/or our automated deployment tools to automate this kind of per-user inst

Re: Druid on slider

Hi, I've tried to get rid of the site-specific stuff and improved the README -- druid has, in particular, a DB dependency (mysql by default). I will only be able to test the new appConfig default tomorrow, however, but the readme should set you up. Kind regards, JB

Druid on slider

Hi there, I've packaged druid to be able to run it on slider there: https://github.com/jbnote/druid-slider While the code is not ready for general use, it works good enough for my own use (realtime, brokers, coordinator and historical). Any feedback would be welcome ! Kind regards, JB

Re: registry / export question

Thanks Gour, Indeed it does help; because I can see a way to combine these to avoid polling. By monitoring the ZK registry and doing CURL whenever there's a child change in the registry it looks I can reliably track changes in the export group, so this is perfect. I'll let you know how implementat

registry / export question

Hi there, I've successfully exported some host/port dynamic combination in slider for Kafka on Yarn; they are made available under publisher/exports/servers on the appmaster (see https://github.com/jbnote/koya/). I'm now trying to access this information (really, service location) in two differen

Re: Keytab issue with Hbase secure

Hi, Thanks for the heads up. We're running on JDK8, however, it looks like 10786 is in CDH5.4.0 which we're using (and compiled against); see: http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.4.0.releasenotes.html This is so close a problem however that we'll dig in this direction. Kind

Re: Container recovery on working on CDH with yarn.component.placement.policy=1

Hi Thomas, I'm also testing on CDH5.4, so i'll be able to attempt duplicating this after my vacation (next week). I'm using the capacity scheduler on a secure cluster though. You probably should be able to see what's going on by increasing the log verbosity of the RM and/or NM -- I don't know if

[jira] [Commented] (SLIDER-868) Ability to put a Slider application on cruise control

[ https://issues.apache.org/jira/browse/SLIDER-868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539540#comment-14539540 ] Jean-Baptiste Note commented on SLIDER-868: --- Hi, As a very interested cl

Re: Packaging new apps

Hi Thomas, According to kafka's documentation: http://kafka.apache.org/07/configuration.html there should be a default value for any added property; I would expect the provided server.properties file to actually reflect those default values. Therefore, I'd look twice before overconstraining the pr

Re: Packaging new apps

There's a remark on the pull request about this, with more details than in this mail, but basically: * Other apps seem to regenerate the config files directly through a template rather than try to do a merge (you seem to be doing a SED on defined properties, however it does not work here, maybe a

Re: Packaging new apps

Hi Thomas, This is because the app_container_tag is unique under each resource. Given your two brokers are on separate resources BROKER0 and BROKER1, they get identical (1) container_tag. You should set them in the same resource (BROKER), and the numbering will be sequential. No idea how it behav

Re: Packaging new apps

Hi Thomas, Thanks a lot for the updates you brought to the main Koya repository. I saw and can see you're still declaring a resource for each broker. This is painful as it means modifying your metainfo & possibly resource.json in case you want to grow your cluster, say beyond 10 machines :) Woul

Re: Packaging new apps

Hi Steve, Thanks a lot for your reply from your very busy schedule. Actually we'll get away with a python daemon watching zookeeper and doing dynamic DNS updates. This seems easy enough and probably more palatable than duplicating a full DNS server (i'm on the operations side ;)). I'll keep you p

Re: Packaging new apps

Hi Gour, Thanks a lot for the detailed answer, and the pointer to tomcat packaging, which does half the work for httpfs. I'll try to wrap properly unpacking of the RPM & extraction of the relevant parts for slider packaging. That was my gripe; other than that, i can launch httpfs services and flex

Packaging new apps

Hi folks, Currently we're using Chef in our organization to deploy a lot of infrastructure services around Hadoop. Of course it makes a lot of sense to offer these as self-services on YARN using slider, but i'm looking at a number of challenges. So please forgive the broad range of questions :) I

Re: storm-slider-client availability

Thanks a huge lot, storm-slider is indeed in develop. What a shame, I only looked at the master and latest release tag. Thanks a lot, JB

storm-slider-client availability

Hi folks, First, thanks a lot for making slider available. This is awesome for consolidating YARN as a generic-purpose resource manager in the DC. Second, sorry to post this user request on a dev mailing list. I'm using slider on CDH5.4.0. I know, I know, hortonworks is the one investing on slide