Hi Josh, I have created a minimal use case using the jmemcached to reproduce the problem. The only change from OOB jmemcached is I am passing additional ports (not relevant for jmem but relevant for my ultimate use case), one of which I try to export.
I have attached the 3 config files as well as a output file containing the slider-am URL output (which contains the component host:port) and ZK output (which does not contains component host:port). Thanks for looking into the issue & let me know if you need additional info Manoj On Sun, Jan 10, 2016 at 11:26 AM, Josh Elser <[email protected]> wrote: > Hey Manoj, > > I'm not sure at a glance, but it might be a bug in Slider's use of the > registry? I'm not quite sure, I'd have to dig into the source to be sure. > > Do you have an example that I could run which exhibits this? > > > Manoj Samel wrote: > >> Any thoughts ? >> >> Thanks, >> >> >> >> On Thu, Jan 7, 2016 at 10:25 AM, Manoj Samel<[email protected]> >> wrote: >> >> Hi, >>> >>> Slider 0.80 with secured cluster. Registry is enabled *** without *** >>> security >>> >>> Export is configured as follows >>> >>> 1. No application level export >>> 2. Component level port allocated per container. In appConfig.json, *** >>> "global" *** section >>> "site.global.user1.http_port": >>> "${user1.ALLOCATED_PORT}{PER_CONTAINER}", >>> 3. Each component exports its own host/port (slider allocated). >>> metainfo.xml >>> <components> >>> <component> >>> <name>user1</name> >>> <category>MASTER</category> >>> <componentExports> >>> <componentExport> >>> <name>my_info</name> >>> <value>${THIS_HOST}:${user1.http_port}</value> >>> </componentExport> >>> </componentExports> >>> <commandScript> >>> <script>scripts/memcached.py</script> >>> <scriptType>PYTHON</scriptType> >>> </commandScript> >>> </component> >>> >>> At runtime, in >>> http://am-host:1025/ws/v1/slider/publisher/slider/componentinstancedata, >>> the host:port info can be seen as expected >>> >>> {"description":"ComponentInstanceData","updated":0,"entries":{"container_1452045276360_0005_01_000004.my_info":"host1:33786", >>> .... >>> >>> However, when I look at ZK data, >>> >>> get >>> /registry/users/foo/services/org-apache-slider/<appname>/components/container-1452045276360-0005-01-000004, >>> I don't see the host:port info. It only shows >>> >>> { >>> "type" : "JSONServiceRecord", >>> "description" : "user1", >>> "external" : [ ], >>> "internal" : [ ], >>> "yarn:id" : "container-1452045276360-0005-01-000004", >>> "yarn:persistence" : "container" >>> } >>> >>> Any thought what I may be missing ? >>> >>> Related question - looking @ >>> https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/registry/yarn-registry.html, >>> the spec and all examples of export list "protocol" >>> >>> e.g. REST, rmi etc. Its not clear if and how is that protocol to be >>> specified in the component export section ? Can anyone give a example ? >>> >>> Thanks, >>> >>> Manoj >>> >>> >>> >>> >>> >>> >>> >>
appConfig.json
Description: application/json
<metainfo>
<schemaVersion>2.0</schemaVersion>
<application>
<name>ms999</name>
<comment>ms999 Memcached service deployed by Slider</comment>
<version>1.0.0</version>
<type>YARN-APP</type>
<exportedConfigs>None</exportedConfigs>
<minHadoopVersion>2.6.0</minHadoopVersion>
<components>
<component>
<name>WD2_IMPL_tenant1</name>
<category>MASTER</category>
<componentExports>
<componentExport>
<name>my_info</name>
<value>${THIS_HOST}:${site.global.WD2_IMPL_tenant1.http_port}</value>
</componentExport>
</componentExports>
<commandScript>
<script>scripts/memcached.py</script>
<scriptType>PYTHON</scriptType>
</commandScript>
</component>
</components>
<osSpecifics>
<osSpecific>
<osType>any</osType>
<packages>
<package>
<type>tarball</type>
<name>files/jmemcached-1.0.0.tar</name>
</package>
</packages>
</osSpecific>
</osSpecifics>
</application>
</metainfo>
resources.json
Description: application/json
