In case you are expecting the data to be in zk, then you should know that Slider stores a very small amount of metadata (primarily endpoints and resource paths to AM) using which you can retrieve the actual data. Most of the information is stored in AM runtime memory and not persisted anywhere. In case of AM failure all the AM state is rebuilt.
-Gour On 1/12/16, 5:24 PM, "Manoj Samel" <manojsamelt...@gmail.com> wrote: >Hi Gour, > >As you asked, the urls and their outputs > >http://host36:1025/ws/v1/slider/publisher/exports > >{"exports":{"container_log_dirs":{"description":"container_log_dirs","upda >ted":1452647463881,"updatedTime":"Wed >Jan 13 01:11:03 UTC >2016","entries":{},"empty":true},"container_work_dirs":{"description":"con >tainer_work_dirs","updated":1452647463881,"updatedTime":"Wed >Jan 13 01:11:03 UTC 2016","entries":{},"empty":true}}} > >http://host36:1025/ws/v1/slider/publisher > >{"uris":{"slider":"http://host36:1025/ws/v1/slider/publisher/slider","expo >rts":"http://host36:1025/ws/v1/slider/publisher/exports"}} > >http://host36:1025/ws/v1/slider/publisher/slider > >{"configurations":{"hdfs-site":{"description":"HDFS site >settings","updated":0,"entries":{},"empty":true},"yarn-site":{"description >":"YARN >site >settings","updated":0,"entries":{},"empty":true},"complete-config":{"descr >iption":"Complete >slider application >settings","updated":0,"entries":{},"empty":true},"core-site":{"description >":"Core >site >settings","updated":0,"entries":{},"empty":true},"componentinstancedata":{ >"description":"ComponentInstanceData","updated":0,"entries":{},"empty":tru >e}}} > >Thanks, > >Manoj > > >On Tue, Jan 12, 2016 at 3:03 PM, Gour Saha <gs...@hortonworks.com> wrote: > >> When you do the below get, you will see an api entry for >> "classpath:org.apache.slider.publisher.exports" >> >> get /registry/users/foo/services/org-apache-slider/<appname> >> >> Do a curl on the uri value. The uri will be in the format of >> http://host:port/ws/v1/slider/publisher/exports. Do you find the info >>you >> are looking for there? >> >> Also do a curl on the path ws/v1/slider/publisher and its sub-paths. >>What >> do you see? >> >> -Gour >> >> From: Manoj Samel <manojsamelt...@gmail.com<mailto: >> manojsamelt...@gmail.com>> >> Reply-To: "dev@slider.incubator.apache.org<mailto: >> dev@slider.incubator.apache.org>" >><dev@slider.incubator.apache.org<mailto: >> dev@slider.incubator.apache.org>> >> Date: Tuesday, January 12, 2016 at 11:24 AM >> To: "dev@slider.incubator.apache.org<mailto: >> dev@slider.incubator.apache.org>" >><dev@slider.incubator.apache.org<mailto: >> dev@slider.incubator.apache.org>> >> Subject: Re: Component level host:port export seen in AM but not in ZK >> >> Hi Josh, >> >> I have created a minimal use case using the jmemcached to reproduce the >> problem. The only change from OOB jmemcached is I am passing additional >> ports (not relevant for jmem but relevant for my ultimate use case), >>one of >> which I try to export. >> >> I have attached the 3 config files as well as a output file containing >>the >> slider-am URL output (which contains the component host:port) and ZK >>output >> (which does not contains component host:port). >> >> Thanks for looking into the issue & let me know if you need additional >>info >> >> Manoj >> >> On Sun, Jan 10, 2016 at 11:26 AM, Josh Elser >><josh.el...@gmail.com<mailto: >> josh.el...@gmail.com>> wrote: >> Hey Manoj, >> >> I'm not sure at a glance, but it might be a bug in Slider's use of the >> registry? I'm not quite sure, I'd have to dig into the source to be >>sure. >> >> Do you have an example that I could run which exhibits this? >> >> >> Manoj Samel wrote: >> Any thoughts ? >> >> Thanks, >> >> >> >> On Thu, Jan 7, 2016 at 10:25 AM, Manoj Samel<manojsamelt...@gmail.com >> <mailto:manojsamelt...@gmail.com>> >> wrote: >> >> Hi, >> >> Slider 0.80 with secured cluster. Registry is enabled *** without *** >> security >> >> Export is configured as follows >> >> 1. No application level export >> 2. Component level port allocated per container. In appConfig.json, *** >> "global" *** section >> "site.global.user1.http_port": >> "${user1.ALLOCATED_PORT}{PER_CONTAINER}", >> 3. Each component exports its own host/port (slider allocated). >> metainfo.xml >> <components> >> <component> >> <name>user1</name> >> <category>MASTER</category> >> <componentExports> >> <componentExport> >> <name>my_info</name> >> <value>${THIS_HOST}:${user1.http_port}</value> >> </componentExport> >> </componentExports> >> <commandScript> >> <script>scripts/memcached.py</script> >> <scriptType>PYTHON</scriptType> >> </commandScript> >> </component> >> >> At runtime, in >> http://am-host:1025/ws/v1/slider/publisher/slider/componentinstancedata, >> the host:port info can be seen as expected >> >> >>{"description":"ComponentInstanceData","updated":0,"entries":{"container_ >>1452045276360_0005_01_000004.my_info":"host1:33786", >> .... >> >> However, when I look at ZK data, >> >> get >> >>/registry/users/foo/services/org-apache-slider/<appname>/components/conta >>iner-1452045276360-0005-01-000004, >> I don't see the host:port info. It only shows >> >> { >> "type" : "JSONServiceRecord", >> "description" : "user1", >> "external" : [ ], >> "internal" : [ ], >> "yarn:id" : "container-1452045276360-0005-01-000004", >> "yarn:persistence" : "container" >> } >> >> Any thought what I may be missing ? >> >> Related question - looking @ >> >>https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/regist >>ry/yarn-registry.html, >> the spec and all examples of export list "protocol" >> >> e.g. REST, rmi etc. Its not clear if and how is that protocol to be >> specified in the component export section ? Can anyone give a example ? >> >> Thanks, >> >> Manoj >> >> >> >> >> >> >> >> >>
