Jean, You will see updates in the KOYA repository soon. As part of that we will move up to the latest release of Slider and also document the configuration process.
Thanks, Thomas On Thu, May 7, 2015 at 5:52 PM, Gour Saha <gs...@hortonworks.com> wrote: > Hi Jean, > > Please see answers inline. > > -Gour > > On 5/6/15, 6:16 AM, "Jean-Baptiste Note" <jbn...@gmail.com<mailto: > jbn...@gmail.com>> wrote: > > Hi folks, > > Currently we're using Chef in our organization to deploy a lot of > infrastructure services around Hadoop. Of course it makes a lot of sense to > offer these as self-services on YARN using slider, but i'm looking at a > number of challenges. So please forgive the broad range of questions :) > > I'm specifically intersted in deploying the following applications: > * HTTPFS service (see https://github.com/jbnote/httpfs-slider) & helpers > (nginx) > * Opentsdb & helpers (varnish) > * kafka (I had a look at koya) > * druid > * storm (fine, thanks !) > * hbase (fine, thanks !) > > I'm facing a lot of issues with those services which are not yet packaged > correctly: > > * httpfs/opentsdb are not released as standalone tarballs, contrary to all > services currently packaged. So i've butchered a tarball from Cloudera > RPMs, which is not satisfactory. How would you go about handling this ? > > Not sure exactly what you mean, by saying "handling this". If you are > referring to a way to create a Slider package of an app in rpm format, then > there are challenges, such as rpm install requires root access and YARN > does not allow that. If you are referring to an issue you are facing with > deploying the Slider app (now that you have created a tarball), can you > share what issues you are facing? > > You might also want to take a look at this tomcat Slider package. Caution: > It is not ready for prime-time and has few issues which needs to be > resolved. But the scripts and metadata files might be a helpful reference. > https://issues.apache.org/jira/browse/SLIDER-809 > > https://github.com/apache/incubator-slider/tree/feature/SLIDER-809-tomcat-app-package/app-packages/tomcat > > > > * KOYA has been talked a lot of, however the source i'm looking at ( > https://github.com/DataTorrent/koya) is kind of disappointing, and > activity > is a bit low -- would anyone know if dataTorrent is still committed to the > project ? > > What issues are you facing with KOYA? DataTorrent gave a presentation of > KOYA and Slider seems to have fit their need so far. They wanted few > features around data locality (strict placement) which will be there in > 0.80.0 release AND unique ids which still needs some work to be done. > > > Last but not least, I'm wondering if there would already be a plan to > expose somehow (through an internal or an external service) the registry > through DNS (that's what we really use for service location for HTTPFS & > OpenTSDB). A bash polling script would certainly be sufficient for our > needs for now, but longer-term, we'd need to have a more robust solution. > > Registry and REST APIs on registry comes directly from YARN - > https://issues.apache.org/jira/browse/YARN-913 > https://issues.apache.org/jira/browse/YARN-2948 > > http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/registry/yarn-registry.html > > > > Thanks a lot, kind regards, > JB > >
