Kevin W Monroe created BIGTOP-2453:
--------------------------------------
Summary: nodemanager role requires common_mapred_app
Key: BIGTOP-2453
URL: https://issues.apache.org/jira/browse/BIGTOP-2453
Project: Bigtop
Issue Type: Bug
Components: deployment
Affects Versions: 1.1.0
Reporter: Kevin W Monroe
Priority: Minor
When deploying a standalone nodemanager, puppet apply fails in a couple
different ways:
First, /etc/hadoop/conf/core-site.xml is not created by the nodemanager class
(or any of its prereqs). When trying to install a standalone nodemanger with
the following in site.yaml:
'bigtop::roles_enabled': true
'bigtop::roles': nodemanager
The puppet apply will fail like this:
...
Error: Could not find dependency File[/etc/hadoop/conf/core-site.xml] for
Service[hadoop-yarn-nodemanager] at
/home/ubuntu/bigtop.release/bigtop-1.1.0/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp:841
...
At first glance, it may seem like we just need to add common_hdfs (the creator
of core-site.xml) to the nodemanager class. However, I tried that and ran up
against a second problem (longer trace at http://paste.ubuntu.com/16637532/):
...
java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException: Class org.apache.hadoop.mapred.ShuffleHandler
not found
...
The problem here is that yarn-site.xml is specifying a missing mapred shuffle
class, which prevents nodemanager from starting. The solution for me was to
include common_mapred_app in the nodemanager class (patch coming shortly).
This solves both problems with one line since common_mapred_app installs the
missing mapred shuffle class and includes common_hdfs to create core-site.xml.
This may not be a common problem since I suspect most people are combining
multiple slave roles (datanode, mapred-app, nodemanager) onto a single machine,
but it would bite anyone that wanted to isolate the nodemanager role by itself.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
