[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202033#comment-13202033
]
Ahmed Radwan commented on MAPREDUCE-3436:
-
Thanks Roman, I have created MAPREDUCE-3828 to track issue#2.
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201851#comment-13201851
]
Roman Shaposhnik commented on MAPREDUCE-3436:
-
@Ahmed
I think there's a value in pushing this patch as-is and having a separate issue
tracking #2. Can you, please, do that?
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201053#comment-13201053
]
Ahmed Radwan commented on MAPREDUCE-3436:
-
@Mahadev,
There are two issues discussed here (please see my previous comment for
description of the two issues). The current patch addresses issue#1.
I am planning also on looking into how to resolve issue#2 which Siddharth
highlighted. I was busy so didn't have time to look into it, but planning to do
so in the coming few days.
So if there is urgency in pushing the current patch (which seems to be accepted
by Siddharth and Bruno), I can create another ticket to follow issue#2. If
there is no urgency then I'll investigate issue#2 in the coming days and update
the current patch. Thanks!
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200494#comment-13200494
]
Mahadev konar commented on MAPREDUCE-3436:
--
@Ahmed/Sid,
Any update on this one?
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191354#comment-13191354
]
Siddharth Seth commented on MAPREDUCE-3436:
---
Yep, two separate issues.
bq. Issue1:
The patch fixes this. (avoiding explicitly setting the history webapp address)
bq. Issue2: If the user doesn't explicitly set yarn.resourcemanager.address,
then all links (AM tracking url and jobhistory url) are broken. This issue even
exists on a single node setup as you mentioned (if the user tries to connect
from another node).
>From the jira description and yarn behaviour - I believe this was what the
>jira was created for. This gets annoying when running a single node cluster on
>a remote node. Could you please update the patch for this as well - it's a
>pretty simple change. Otherwise I can do that. Thanks.
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13190021#comment-13190021
]
Ahmed Radwan commented on MAPREDUCE-3436:
-
@Siddarth Thanks for our offline chat yesterday. I'll summarize my
understanding and please correct me if I am wrong. I think we are speaking
about two separate issues:
Issue1: When the user explicitly set yarn.resourcemanager.address and
mapreduce.jobhistory.address, the jobhistory link is broken (without the
patch), the patch basically avoids the need to explicitly set
mapreduce.jobhistory.webapp.address. This issue doesn't exist in the single
node case because all services run on the single node.
Issue2: If the user doesn't explicitly set yarn.resourcemanager.address, then
all links (AM tracking url and jobhistory url) are broken. This issue even
exists on a single node setup as you mentioned (if the user tries to connect
from another node).
However, I am not sure if Issue2 is a bug or not: For a single node setup, if
yarn.resourcemanager.address is not set, the user can still play with the web
ui if connecting from the same single node, and all links will work. On a real
cluster (multinode), it is expected/required to set the
yarn.resourcemanager.address and mapreduce.jobhistory.address for the cluster
to function, and so, with the patch, all links will also work.
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189435#comment-13189435
]
Siddharth Seth commented on MAPREDUCE-3436:
---
@Bruno - "the link was not 0.0.0.0" - that's with and without the patch,
correct ? With the patch - it goes to the correct history server (second node).
Without the patch, the history url is the same as the ResourceManager host ?
bq.
There are links to the history for each application. None of them can be
reached since they all point to the ip 0.0.0.0. For instance:
http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
@Ahmed, that's from the jira description - and I'm assuming that's what is
being fixed. This exists in a single node cluster.
On the 2 node deployment that you're using - could you try not setting
"yarn.resourcemanager.address", and "yarn.web-proxy.address" to see if the
tracker url is something other than 0.0.0.0.
For the scenario Bruno has described in his previous comment - rm address set,
history address set, history webapp address not set - the patch is good.
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189392#comment-13189392
]
Ahmed Radwan commented on MAPREDUCE-3436:
-
Thanks Siddarth, I am afraid I am misunderstanding something. To make sure we
are talking about the same issue, and since this issue doesn't exist on a
single node setup, will you be able to test the patch on a multinode setup (2
nodes for instance)? For example: running all services except HS on one node
and HS on the second node. And see if you are seeing the same reported behavior
with and without the patch (i.e. the JobHistory link is broken without the
patch). You'll need to explicitly set mapreduce.jobhistory.address in both
cases.
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189361#comment-13189361
]
Bruno Mahé commented on MAPREDUCE-3436:
---
Siddarth> "yarn.resourcemanager.address" was set and
"mapreduce.jobhistory.address" was indeed set to pointed to the second host.
"mapreduce.jobhistory.webapp.address" was not set.
And also during my tests, the link was not set to 0.0.0.0 anymore, but to the
first host.
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189351#comment-13189351
]
Siddharth Seth commented on MAPREDUCE-3436:
---
Bruno, on the node running the RM - were either of
"yarn.resourcemanager.address" or "yarn.web-proxy.address" explicitly set ?
I assume the history server address (mapreduce.jobhistory.address) was set to
point to the second node, and "mapreduce.jobhistory.webapp.address" was not set.
The changes to have the history webapp address pick the host from
"mapreduce.jobhistory.address" look good.
Other than that - I still can't see how this patch fixes the link being 0.0.0.0
.
Looks like YarnConfiguration.getProxyHostAndPort() will return the default
(0.0.0.0), unless the RM address / web-proxy address are configured. That call
is used to construct the actual proxy url - which is what causes the link to be
http://0.0.0.0:8088/proxy/* .
Setup that I'm using. Single node - "yarn.resourcemanager.address",
"yarn.web-proxy.address" not set.
If that's not the behaviour others are seing - i can create that as a separate
jira. This one is really just making sure the history webapp address host is
being up form "mapreduce.jobhistory.address" instead of
"mapreduce.jobhistory.webapp.address"
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13188712#comment-13188712
]
Robert Joseph Evans commented on MAPREDUCE-3436:
The patch looks good to me, but I would like to see a unit test for it, before
I give a +1. It might even be nice to make the code common between this and
the resource manager. IE provide an API that will take two configs, do the
split and merge between them. That is minor though. Just a nice to have.
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13188200#comment-13188200
]
Bruno Mahé commented on MAPREDUCE-3436:
---
Sorry for the delay but I can finally confirm Ahmed patch fixes the issue.
I tested this with a cluster of 2 machines where {namenode, datanode,
resourcemanager, nodemanager} run on one machine and {historyserver} run on the
other.
Without Ahmed's patch the history server link from a finished job does not
point to the correct historyserver and point to the first machine where no
historyserver run.
With Ahmed's patch, the history server link from a finished job do point to the
correct historyserver and works like a charm.
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166742#comment-13166742
]
Siddharth Seth commented on MAPREDUCE-3436:
---
Yep. Have it explicitly set to "hostname:port".
While the Job is running, the tracking URL (AM) is:
http://0.0.0.0:8088/proxy/application_1323481362374_0001/
After the Job completes, this changes to :
http://0.0.0.0:8088/proxy/application_1323481362374_0001/jobhistory/job/job_1323481362374_1_1
These links are from the http://:8088/cluster/apps page
(Tracking UI column).
If i explicitly set the proxy address (yarn.web-proxy.address) - then the
host:port changes to the value of yarn.web-proxy.address
Bruno, since you initially reported the issue, could you please verify if the
patch fixes it. Thanks.
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166721#comment-13166721
]
Ahmed Radwan commented on MAPREDUCE-3436:
-
Siddarth, when you were still seeing 0.0.0.0, did you have
mapreduce.jobhistory.address explicitly set?
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166623#comment-13166623
]
Ahmed Radwan commented on MAPREDUCE-3436:
-
Siddarth, Here is a description of what I am seeing with and without the patch:
- Without the patch, the tracking ui history link points (for example) to:
http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2,
and this link is broken if you try to follow it. (this complies with Bruno's
observation above).
- With the patch, the tracking ui history link points to:
http://:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
and if you follow this link, it is not broken, and it displays the page:
http://:19888/jobhistory/job/job_1323304600508_6_6/jobhistory/job/job_1323304600508_6_6
above is the node running both the resource manager and history
server.
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164795#comment-13164795
]
Siddharth Seth commented on MAPREDUCE-3436:
---
Ahmed, I tried the patch. The Tracking URL (AM or History) for the app on the
RM UI http://:8088/cluster/apps - is still using 0.0.0.0.
Env: No proxy address, no rm/rm webapp address, with and without
mapreduce.jobhistory.address configured.
The change I do see with the patch is for the non proxied history url - if
'mapreduce.jobhistory.address' is set , the non proxied history url uses this
instead of the host's IP address.
Am I missing something, maybe different issues ?
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164763#comment-13164763
]
Hadoop QA commented on MAPREDUCE-3436:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12506450/MAPREDUCE-3436_rev2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1408//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1408//console
This message is automatically generated.
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164142#comment-13164142
]
Siddharth Seth commented on MAPREDUCE-3436:
---
Like Tom mentioned, explicitly setting yarn.resourcemanager.webapp.address
should fix this. (yarn.resourcemanager.address also needs to be set -
YarnConfiguration.getRMWebAppHostAndPort for some reason uses the host from
this and the port from the webapp address).
This isn't limited to history urls only - you should see the same issue with
the AM address while it's running.
Another option may be to have RM startup set the address correctly - if it's a
local address. JobHistory.getHistoryUrl tries to do this.
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162985#comment-13162985
]
Bruno Mahé commented on MAPREDUCE-3436:
---
Does that mean a user cannot have this service listen on all interfaces without
breaking the ui?
But that would also mean the jobhistory link is broken by default, unless its
config is updated by the user to point to the correct ip/hostname.
I also don't understand why other web ui links are able to figure out the right
hostname/ip by themselves without any extra configuration. I haven't had time
to figure out how it's done exactly, but the jobhistory is the only place where
such configuration is needed.
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162731#comment-13162731
]
Hadoop QA commented on MAPREDUCE-3436:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12506104/MAPREDUCE-3436.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1391//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1391//console
This message is automatically generated.
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162567#comment-13162567
]
Ahmed Radwan commented on MAPREDUCE-3436:
-
Bruno, I checked this issue, the default mapreduce jobhistory webapp address is
set to 0.0.0.0:19888, and this is why you are hitting this link problem. Can
you try setting the property: mapreduce.jobhistory.webapp.address in your
cluster conf files :, hsAddress is where you started your
history server, and choose any port of your choice (doesn't need to be the
default 19888). Can you try that?
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161069#comment-13161069
]
Tom White commented on MAPREDUCE-3436:
--
You could try setting yarn.resourcemanager.webapp.address to a host-port pair.
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
> Labels: bigtop
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
