Sorry I did not respond sooner. The default behavior is to have the proxy
server run as part of the RM. I am not really sure why it is not doing
this in your case. If you set the config yourself to be a URI that is
different from that of the RM then you need to launch a standalone proxy
server. You can do this by running
yarn start proxy server
Without sitting down with you it is going to be somewhat difficult to
debug why this is happening. However, in retrospect it would be nice to
add in some extra logging to help indicate why the proxy server is not
functioning as desired. If you could file a JIRA to add in the logging I
would be happy to provide a patch to you and we can try and debug the
issue further. Please file it under the MAPREDUCE JIRA project.
--Bobby
On 7/6/12 3:29 AM, "Prajakta Kalmegh" <pkalm...@gmail.com> wrote:
>Re-posting as I haven't got a solution yet. Sorry for spamming. I won't be
>able to proceed in my code until I get a JSON response using AppMaster
>REST
>URL. :(
>
>Thanks,
>Prajakta
>
>
>On Wed, Jul 4, 2012 at 5:55 PM, Prajakta Kalmegh <pkalm...@gmail.com>
>wrote:
>
>> Hi Robert/Harsh
>>
>> Thanks for your reply.
>>
>> My RM is starting just fine. The problem is with the use of
>>http://<proxy httpddress:port>/proxy/{appid}/ws/v1/mapreduce
>> to get the JSON response.
>>
>> As I said before, I had not configured the yarn.web-proxy.address
>>property in yarn-site.xml. I assumed it will use the RM's
>>yarn.resourcemanager.webapp.address property value as default. However,
>>it gives me a '404-Page not found error'. Today I tried specifying a
>>value explicitly for the yarn.web-proxy.address property.
>>
>> On running the wordcount example, it even gives a url
>><http://localhost:8090>/proxy/{appid}/> to track the App Mast info.
>>However, I am still not able to get a json response.
>>
>> Also, I tried to get the data from historyserver instead of runtime
>>using the instructions given on page
>><http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yar
>>n-site/HistoryServerRest.html>
>>
>> HistoryServer REST response does not give me jobids corresponding to an
>>application. It just lists all the jobs run until now. By the way, the
>>documentation does say
>>
>> ----------
>>
>> "Both of the following URI's give you the history server information,
>>from an application id identified by the appid value.
>> * http://<history server http address:port>/ws/v1/history
>> * http://<history server http address:port>/ws/v1/history/info"
>> ---------
>>
>> But there is no provision to specify the application id with these REST
>>URLs.
>>
>> Any idea how I can get the Application Master REST working and also
>>linking jobids to application id using the HistoryServerREST API?
>>
>> Any help is appreciated. Thanks in advance.
>> Regards,
>> Prajakta
>>
>>
>>
>>
>> On Fri, Jun 29, 2012 at 8:55 PM, Robert Evans <ev...@yahoo-inc.com>
>>wrote:
>>
>>> Please don't file that JIRA. The proxy server is intended to front the
>>> web server for all calls to the AM. This is so you only have to go to
>>>a
>>> single location to get to any AM's web service. The proxy server is a
>>> very simple proxy and just forwards the extra part of the path on to
>>>the
>>> AM.
>>>
>>> If you are having issues with this please include the version you are
>>> having problems with. Also please look at the logs for the RM on
>>>startup
>>> to see if there is anything there indicating why it is not starting up.
>>>
>>> --Bobby Evans
>>>
>>> On 6/28/12 9:46 AM, "Harsh J" <ha...@cloudera.com> wrote:
>>>
>>> >As far as I can tell, the MR WebApp, as the name itself indicates on
>>> >its doc page, starts only at the MR AM (which may be running at any
>>> >NM), and it starts as an ephemeral port logged at in the AM logs
>>> >usually as:
>>> >
>>> >INFO Web app /mapreduce started at [PORT]
>>> >
>>> >That it starts its own server with an ephemeral access point makes
>>> >sense, since each job uses its own AM and having a common location may
>>> >not work with the form of REST API documented at your link. Can you
>>> >please file a JIRA to fix the doc and remove the proxy server refs,
>>> >which are misleading?
>>> >
>>> >Do correct me if I'm wrong.
>>> >
>>> >On Thu, Jun 28, 2012 at 6:13 PM, Prajakta Kalmegh <pkalm...@gmail.com>
>>> >wrote:
>>> >> Hi
>>> >>
>>> >> I am trying to get the ApplicationMaster info using the
>>><http://<proxy
>>> >>http
>>> >> address:port>/proxy/{appid}/ws/v1/mapreduce/info> link as described
>>>on
>>> >>the <
>>> >>
>>> >>
>>>
>>>http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yar
>>>n
>>> >>-site/MapredAppMasterRest.html>
>>> >> page.
>>> >>
>>> >> I am able to access and retrieve JSON response for other modules
>>> >> (ResourceManager, NodeManager and HistoryServer). However, I am
>>>getting
>>> >> 'Page not found' when I try to use my ResourceManager Http address
>>>to
>>> >> access the ApplicationMaster info. I am using <
>>> >> http://localhost:8088/proxy/{appid}/ws/v1/mapreduce/info> to
>>>retrieve
>>> >>JSON
>>> >> response.
>>> >>
>>> >> The instructions say "The application master should be accessed via
>>>the
>>> >> proxy. This proxy is configurable to run either on the resource
>>>manager
>>> >>or
>>> >> on a separate host."
>>> >>
>>> >> My yarn-default.xml contains:
>>> >> <property>
>>> >> <description>The address for the web proxy as HOST:PORT, if this
>>>is
>>> >>not
>>> >> given then the proxy will run as part of the RM</description>
>>> >> <name>yarn.web-proxy.address</name>
>>> >> <value/>
>>> >> </property>
>>> >>
>>> >> and I did not set a value explicitly in yarn-site.xml. Any idea
>>>how I
>>> >>can
>>> >> get this working? Thanks in advance.
>>> >>
>>> >> Regards,
>>> >> Prajakta
>>> >
>>> >
>>> >
>>> >--
>>> >Harsh J
>>>
>>>
>>
