Agreed. But I am not seeing even the host/port that is supposed to be in ZK
Thanks, Manoj On Tue, Jan 12, 2016 at 5:52 PM, Gour Saha <gs...@hortonworks.com> wrote: > In case you are expecting the data to be in zk, then you should know that > Slider stores a very small amount of metadata (primarily endpoints and > resource paths to AM) using which you can retrieve the actual data. Most > of the information is stored in AM runtime memory and not persisted > anywhere. In case of AM failure all the AM state is rebuilt. > > -Gour > > On 1/12/16, 5:24 PM, "Manoj Samel" <manojsamelt...@gmail.com> wrote: > > >Hi Gour, > > > >As you asked, the urls and their outputs > > > >http://host36:1025/ws/v1/slider/publisher/exports > > > >{"exports":{"container_log_dirs":{"description":"container_log_dirs","upda > >ted":1452647463881,"updatedTime":"Wed > >Jan 13 01:11:03 UTC > >2016","entries":{},"empty":true},"container_work_dirs":{"description":"con > >tainer_work_dirs","updated":1452647463881,"updatedTime":"Wed > >Jan 13 01:11:03 UTC 2016","entries":{},"empty":true}}} > > > >http://host36:1025/ws/v1/slider/publisher > > > >{"uris":{"slider":"http://host36:1025/ws/v1/slider/publisher/slider > ","expo > >rts":"http://host36:1025/ws/v1/slider/publisher/exports"}} > > > >http://host36:1025/ws/v1/slider/publisher/slider > > > >{"configurations":{"hdfs-site":{"description":"HDFS site > >settings","updated":0,"entries":{},"empty":true},"yarn-site":{"description > >":"YARN > >site > >settings","updated":0,"entries":{},"empty":true},"complete-config":{"descr > >iption":"Complete > >slider application > >settings","updated":0,"entries":{},"empty":true},"core-site":{"description > >":"Core > >site > >settings","updated":0,"entries":{},"empty":true},"componentinstancedata":{ > >"description":"ComponentInstanceData","updated":0,"entries":{},"empty":tru > >e}}} > > > >Thanks, > > > >Manoj > > > > > >On Tue, Jan 12, 2016 at 3:03 PM, Gour Saha <gs...@hortonworks.com> wrote: > > > >> When you do the below get, you will see an api entry for > >> "classpath:org.apache.slider.publisher.exports" > >> > >> get /registry/users/foo/services/org-apache-slider/<appname> > >> > >> Do a curl on the uri value. The uri will be in the format of > >> http://host:port/ws/v1/slider/publisher/exports. Do you find the info > >>you > >> are looking for there? > >> > >> Also do a curl on the path ws/v1/slider/publisher and its sub-paths. > >>What > >> do you see? > >> > >> -Gour > >> > >> From: Manoj Samel <manojsamelt...@gmail.com<mailto: > >> manojsamelt...@gmail.com>> > >> Reply-To: "dev@slider.incubator.apache.org<mailto: > >> dev@slider.incubator.apache.org>" > >><dev@slider.incubator.apache.org<mailto: > >> dev@slider.incubator.apache.org>> > >> Date: Tuesday, January 12, 2016 at 11:24 AM > >> To: "dev@slider.incubator.apache.org<mailto: > >> dev@slider.incubator.apache.org>" > >><dev@slider.incubator.apache.org<mailto: > >> dev@slider.incubator.apache.org>> > >> Subject: Re: Component level host:port export seen in AM but not in ZK > >> > >> Hi Josh, > >> > >> I have created a minimal use case using the jmemcached to reproduce the > >> problem. The only change from OOB jmemcached is I am passing additional > >> ports (not relevant for jmem but relevant for my ultimate use case), > >>one of > >> which I try to export. > >> > >> I have attached the 3 config files as well as a output file containing > >>the > >> slider-am URL output (which contains the component host:port) and ZK > >>output > >> (which does not contains component host:port). > >> > >> Thanks for looking into the issue & let me know if you need additional > >>info > >> > >> Manoj > >> > >> On Sun, Jan 10, 2016 at 11:26 AM, Josh Elser > >><josh.el...@gmail.com<mailto: > >> josh.el...@gmail.com>> wrote: > >> Hey Manoj, > >> > >> I'm not sure at a glance, but it might be a bug in Slider's use of the > >> registry? I'm not quite sure, I'd have to dig into the source to be > >>sure. > >> > >> Do you have an example that I could run which exhibits this? > >> > >> > >> Manoj Samel wrote: > >> Any thoughts ? > >> > >> Thanks, > >> > >> > >> > >> On Thu, Jan 7, 2016 at 10:25 AM, Manoj Samel<manojsamelt...@gmail.com > >> <mailto:manojsamelt...@gmail.com>> > >> wrote: > >> > >> Hi, > >> > >> Slider 0.80 with secured cluster. Registry is enabled *** without *** > >> security > >> > >> Export is configured as follows > >> > >> 1. No application level export > >> 2. Component level port allocated per container. In appConfig.json, *** > >> "global" *** section > >> "site.global.user1.http_port": > >> "${user1.ALLOCATED_PORT}{PER_CONTAINER}", > >> 3. Each component exports its own host/port (slider allocated). > >> metainfo.xml > >> <components> > >> <component> > >> <name>user1</name> > >> <category>MASTER</category> > >> <componentExports> > >> <componentExport> > >> <name>my_info</name> > >> <value>${THIS_HOST}:${user1.http_port}</value> > >> </componentExport> > >> </componentExports> > >> <commandScript> > >> <script>scripts/memcached.py</script> > >> <scriptType>PYTHON</scriptType> > >> </commandScript> > >> </component> > >> > >> At runtime, in > >> http://am-host:1025/ws/v1/slider/publisher/slider/componentinstancedata > , > >> the host:port info can be seen as expected > >> > >> > >>{"description":"ComponentInstanceData","updated":0,"entries":{"container_ > >>1452045276360_0005_01_000004.my_info":"host1:33786", > >> .... > >> > >> However, when I look at ZK data, > >> > >> get > >> > >>/registry/users/foo/services/org-apache-slider/<appname>/components/conta > >>iner-1452045276360-0005-01-000004, > >> I don't see the host:port info. It only shows > >> > >> { > >> "type" : "JSONServiceRecord", > >> "description" : "user1", > >> "external" : [ ], > >> "internal" : [ ], > >> "yarn:id" : "container-1452045276360-0005-01-000004", > >> "yarn:persistence" : "container" > >> } > >> > >> Any thought what I may be missing ? > >> > >> Related question - looking @ > >> > >> > https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/regist > >>ry/yarn-registry.html, > >> the spec and all examples of export list "protocol" > >> > >> e.g. REST, rmi etc. Its not clear if and how is that protocol to be > >> specified in the component export section ? Can anyone give a example ? > >> > >> Thanks, > >> > >> Manoj > >> > >> > >> > >> > >> > >> > >> > >> > >> > >
