[ https://issues.apache.org/jira/browse/MYRIAD-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Miguel Bernadin updated MYRIAD-195: ----------------------------------- Fix Version/s: (was: Myriad 0.2.0) > Node Managers randomly die on Mesos > ----------------------------------- > > Key: MYRIAD-195 > URL: https://issues.apache.org/jira/browse/MYRIAD-195 > Project: Myriad > Issue Type: Bug > Components: Executor > Affects Versions: Myriad 0.1.0 > Environment: Ubuntu 14.04; kernel 3.13.0-66-generic; MapR 5.0 > Reporter: Miguel Bernadin > > Hello, I have been noticing that the Node Managers randomly die on mesos. > Here are the attached two logs below. The first is the mesos logs, and the > second is the node manager createNMVolume log. Looking to see if anyone else > is experiencing this. > Mesos logs: > 16/03/25 13:51:10 INFO nodemanager.NodeManager: STARTUP_MSG: > /************************************************************ STARTUP_MSG: > Starting NodeManager STARTUP_MSG: host = nodemanager/10.1.194.71 STARTUP_MSG: > args = [] STARTUP_MSG: version = 2.7.0-mapr-1506 STARTUP_MSG: classpath = > /opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop:/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop:/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-hbase-5.0.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprfs-diagnostic-tools-5.0.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jetty-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprfs-jni-5.0.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/hadoop-annotations-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-hbase-5.0.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/httpcore-4.2.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mysql-connector-java-5.1.25-bin.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/curator-framework-2.7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/hadoop-auth-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/curator-client-2.7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jsch-0.1.42.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/htrace-core-3.1.0-incubating.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-collections-3.2.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/junit-4.11.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jettison-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprfs-core-5.0.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/gson-2.2.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jersey-core-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jersey-server-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/avro-1.7.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/hamcrest-core-1.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-lang-2.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/paranamer-2.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jsr305-3.0.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/guava-13.0.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-cli-1.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/libprotodefs-5.0.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/xz-1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/zookeeper-3.4.5-mapr-1503.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/httpclient-4.2.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jsp-api-2.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-net-3.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/central-logging-5.0.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jersey-json-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/activation-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jets3t-0.9.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/xmlenc-0.52.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-digester-1.8.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-codec-1.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jetty-util-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprfs-5.0.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprutil-5.0.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/asm-3.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprfs-core-5.0.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/servlet-api-2.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-io-2.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/json-20080701.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/log4j-1.2.17.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/hadoop-common-2.7.0-mapr-1506-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/hadoop-common-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/hadoop-nfs-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/htrace-core-3.1.0-incubating.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/guava-13.0.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/asm-3.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-io-2.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/hadoop-hdfs-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/hadoop-hdfs-2.7.0-mapr-1506-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-dataformat-yaml-2.5.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jetty-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jline-0.9.94.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/curator-framework-2.7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/myriad-executor-0.1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/curator-client-2.7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/disruptor-3.3.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/eval-0.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jettison-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/mesos-0.24.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/javax.inject-1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/hibernate-validator-5.1.2.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-databind-2.5.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/log4j-1.2.16.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/netty-3.7.0.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/slf4j-api-1.7.7.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guava-13.0.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/xz-1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/zookeeper-3.4.5-mapr-1503.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/cglib-2.2.1-v20090111.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/metrics-core-3.1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/zookeeper-3.4.5-mapr-1503-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/activation-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guice-3.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/myriad-commons-0.1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/metrics-healthchecks-3.1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/myriad-scheduler-0.1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-lang3-3.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/myriad-executor-runnable-0.1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guice-multibindings-3.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/metrics-annotation-3.1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-annotations-2.5.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/asm-3.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/snakeyaml-1.12.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/gson-2.3.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guava-16.0.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-core-2.5.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/classmate-1.0.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/asm-3.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/validation-api-1.1.0.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jboss-logging-3.1.3.GA.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-common-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-common-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-registry-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-api-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-client-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-client-contrib-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.0-mapr-1506-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.7.0-mapr-1506.jar:/contrib/capacity-scheduler/*.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-common-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-common-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-registry-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-api-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-client-2.7.0-mapr-1506.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-dataformat-yaml-2.5.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jetty-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jline-0.9.94.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/curator-framework-2.7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/myriad-executor-0.1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/curator-client-2.7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/disruptor-3.3.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/eval-0.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jettison-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/mesos-0.24.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/javax.inject-1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/hibernate-validator-5.1.2.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-databind-2.5.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/log4j-1.2.16.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/netty-3.7.0.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/slf4j-api-1.7.7.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guava-13.0.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/xz-1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/zookeeper-3.4.5-mapr-1503.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/cglib-2.2.1-v20090111.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/metrics-core-3.1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/zookeeper-3.4.5-mapr-1503-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/activation-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guice-3.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/myriad-commons-0.1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/metrics-healthchecks-3.1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/myriad-scheduler-0.1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-lang3-3.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/myriad-executor-runnable-0.1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guice-multibindings-3.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/metrics-annotation-3.1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-annotations-2.5.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/asm-3.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/snakeyaml-1.12.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/gson-2.3.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guava-16.0.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-core-2.5.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/classmate-1.0.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/asm-3.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/validation-api-1.1.0.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jboss-logging-3.1.3.GA.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/nm-config/log4j.properties:/opt/mapr/lib/JPam-1.1.jar > STARTUP_MSG: build = g...@github.com:mapr/private-hadoop-common.git -r > 5264b1d5c5c2a849ee0eb09cfcbbed19fb0bfb53; compiled by 'root' on > 2015-07-02T23:46Z STARTUP_MSG: java = 1.8.0_45-internal > ************************************************************/ > 16/03/25 13:51:10 INFO nodemanager.NodeManager: registered UNIX signal > handlers for [TERM, HUP, INT] > 16/03/25 13:51:11 INFO event.AsyncDispatcher: Registering class > org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType > for class > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher > > 16/03/25 13:51:11 INFO event.AsyncDispatcher: Registering class > org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType > for class > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher > > 16/03/25 13:51:11 INFO event.AsyncDispatcher: Registering class > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType > for class > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService > > 16/03/25 13:51:11 INFO event.AsyncDispatcher: Registering class > org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType > for class > org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices > 16/03/25 13:51:11 INFO event.AsyncDispatcher: Registering class > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType > for class > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl > > 16/03/25 13:51:11 INFO event.AsyncDispatcher: Registering class > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType > for class > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher > > 16/03/25 13:51:11 INFO event.AsyncDispatcher: Registering class > org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl > > 16/03/25 13:51:11 INFO event.AsyncDispatcher: Registering class > org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class > org.apache.hadoop.yarn.server.nodemanager.NodeManager > 16/03/25 13:51:11 INFO impl.MetricsConfig: loaded properties from > hadoop-metrics2.properties > 16/03/25 13:51:11 INFO impl.MetricsSystemImpl: Scheduled snapshot period at > 10 second(s). > 16/03/25 13:51:11 INFO impl.MetricsSystemImpl: NodeManager metrics system > started > 16/03/25 13:51:11 INFO event.AsyncDispatcher: Registering class > org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType > for class > org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler > > 16/03/25 13:51:11 INFO event.AsyncDispatcher: Registering class > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType > for class > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService > > 16/03/25 13:51:11 INFO localizer.ResourceLocalizationService: per directory > file limit = 8192 > 16/03/25 13:51:11 INFO localizer.ResourceLocalizationService: usercache path > : file:///tmp/hadoop-mapr/nm-local-dir/usercache_DEL_1458939071905 > 16/03/25 13:51:11 INFO event.AsyncDispatcher: Registering class > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType > for class > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker > > 16/03/25 13:51:11 WARN containermanager.AuxServices: The Auxilurary Service > named 'mapreduce_shuffle' in the configuration is for class > org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. > Because these are not the same tools trying to send ServiceData and read > Service Meta Data may have issues unless the refer to the name in the config. > 16/03/25 13:51:11 INFO containermanager.AuxServices: Adding auxiliary service > httpshuffle, "mapreduce_shuffle" > 16/03/25 13:51:12 WARN containermanager.AuxServices: The Auxilurary Service > named 'mapr_direct_shuffle' in the configuration is for class > com.mapr.hadoop.mapred.LocalVolumeAuxService which has a name of > 'direct_shuffle'. Because these are not the same tools trying to send > ServiceData and read Service Meta Data may have issues unless the refer to > the name in the config. > 16/03/25 13:51:12 INFO containermanager.AuxServices: Adding auxiliary service > direct_shuffle, "mapr_direct_shuffle" > 16/03/25 13:51:12 INFO monitor.ContainersMonitorImpl: Using > ResourceCalculatorPlugin : > org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@2cb2fc20 > 16/03/25 13:51:12 INFO monitor.ContainersMonitorImpl: Using > ResourceCalculatorProcessTree : null > 16/03/25 13:51:12 INFO monitor.ContainersMonitorImpl: Physical memory check > enabled: true > 16/03/25 13:51:12 INFO monitor.ContainersMonitorImpl: Virtual memory check > enabled: false > 16/03/25 13:51:12 INFO nodemanager.NodeStatusUpdaterImpl: Initialized > nodemanager for null: physical-memory=12288 virtual-memory=25805 > virtual-cores=10 disks=4.0 > 16/03/25 13:51:12 INFO ipc.CallQueueManager: Using callQueue class > java.util.concurrent.LinkedBlockingQueue > 16/03/25 13:51:12 INFO ipc.Server: Starting Socket Reader #1 for port 5220 > 16/03/25 13:51:12 INFO pb.RpcServerFactoryPBImpl: Adding protocol > org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server > 16/03/25 13:51:12 INFO containermanager.ContainerManagerImpl: Blocking new > container-requests as container manager rpc server is still starting. > 16/03/25 13:51:12 INFO ipc.Server: IPC Server Responder: starting > 16/03/25 13:51:12 INFO ipc.Server: IPC Server listener on 5220: starting > 16/03/25 13:51:12 INFO security.NMContainerTokenSecretManager: Updating node > address : nodemanager.local:5220 > 16/03/25 13:51:12 INFO ipc.CallQueueManager: Using callQueue class > java.util.concurrent.LinkedBlockingQueue > 16/03/25 13:51:12 INFO ipc.Server: Starting Socket Reader #1 for port 31254 > 16/03/25 13:51:12 INFO pb.RpcServerFactoryPBImpl: Adding protocol > org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the > server > 16/03/25 13:51:12 INFO ipc.Server: IPC Server Responder: starting > 16/03/25 13:51:12 INFO ipc.Server: IPC Server listener on 31254: starting > 16/03/25 13:51:12 INFO localizer.ResourceLocalizationService: Localizer > started on port 31254 > 16/03/25 13:51:12 INFO mapred.LocalVolumeAuxService: Checking for local > volume. If volume is not present command will create and mount it. Command > invoked is : /opt/mapr/server/createTTVolume.sh nodemanager.local > /var/mapr/local/nodemanager.local/mapred > /var/mapr/local/nodemanager.local/mapred/nodeManager yarn > 16/03/25 13:51:38 ERROR mapred.LocalVolumeAuxService: Failed to create and > mount local mapreduce volume at /var/mapr/local/nodemanager.local/mapred. > Please see logs at /opt/mapr/logs/createNMVolume.log > 16/03/25 13:51:38 ERROR mapred.LocalVolumeAuxService: Command ran > /opt/mapr/server/createTTVolume.sh nodemanager.local > /var/mapr/local/nodemanager.local/mapred > /var/mapr/local/nodemanager.local/mapred/nodeManager yarn > 16/03/25 13:51:38 ERROR mapred.LocalVolumeAuxService: Command output > 16/03/25 13:51:38 INFO service.AbstractService: Service direct_shuffle failed > in state STARTED; cause: ExitCodeException exitCode=203: ExitCodeException > exitCode=203: at org.apache.hadoop.util.Shell.runCommand(Shell.java:545) at > org.apache.hadoop.util.Shell.run(Shell.java:456) at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722) at > com.mapr.hadoop.mapred.LocalVolumeAuxService.initVolume(LocalVolumeAuxService.java:260) > at > com.mapr.hadoop.mapred.LocalVolumeAuxService.serviceStart(LocalVolumeAuxService.java:229) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceStart(AuxServices.java:173) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceStart(ContainerManagerImpl.java:468) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:265) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:464) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:511) > > 16/03/25 13:51:38 INFO mapred.LocalVolumeAuxService: Shutting down Volume > Checker service > 16/03/25 13:51:38 INFO mapred.LocalVolumeAuxService: Cleanup service shutdown > 16/03/25 13:51:38 INFO service.AbstractService: Service > org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices failed > in state STARTED; cause: org.apache.hadoop.service.ServiceStateException: > ExitCodeException exitCode=203: > org.apache.hadoop.service.ServiceStateException: ExitCodeException > exitCode=203: at > org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:204) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceStart(AuxServices.java:173) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceStart(ContainerManagerImpl.java:468) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:265) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:464) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:511) > Caused by: ExitCodeException exitCode=203: at > org.apache.hadoop.util.Shell.runCommand(Shell.java:545) at > org.apache.hadoop.util.Shell.run(Shell.java:456) at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722) at > com.mapr.hadoop.mapred.LocalVolumeAuxService.initVolume(LocalVolumeAuxService.java:260) > at > com.mapr.hadoop.mapred.LocalVolumeAuxService.serviceStart(LocalVolumeAuxService.java:229) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > ... 10 more > 16/03/25 13:51:38 INFO service.AbstractService: Service > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl > failed in state STARTED; cause: > org.apache.hadoop.service.ServiceStateException: ExitCodeException > exitCode=203: org.apache.hadoop.service.ServiceStateException: > ExitCodeException exitCode=203: at > org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:204) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceStart(AuxServices.java:173) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceStart(ContainerManagerImpl.java:468) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:265) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:464) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:511) > Caused by: ExitCodeException exitCode=203: at > org.apache.hadoop.util.Shell.runCommand(Shell.java:545) at > org.apache.hadoop.util.Shell.run(Shell.java:456) at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722) at > com.mapr.hadoop.mapred.LocalVolumeAuxService.initVolume(LocalVolumeAuxService.java:260) > at > com.mapr.hadoop.mapred.LocalVolumeAuxService.serviceStart(LocalVolumeAuxService.java:229) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > ... 10 more > 16/03/25 13:51:38 INFO ipc.Server: Stopping server on 5220 > 16/03/25 13:51:38 INFO ipc.Server: Stopping IPC Server listener on 5220 > 16/03/25 13:51:38 INFO ipc.Server: Stopping IPC Server Responder > 16/03/25 13:51:38 INFO ipc.Server: Stopping server on 31254 > 16/03/25 13:51:38 INFO ipc.Server: Stopping IPC Server listener on 31254 > 16/03/25 13:51:38 INFO ipc.Server: Stopping IPC Server Responder > 16/03/25 13:51:38 INFO service.AbstractService: Service NodeManager failed in > state STARTED; cause: org.apache.hadoop.service.ServiceStateException: > ExitCodeException exitCode=203: > org.apache.hadoop.service.ServiceStateException: ExitCodeException > exitCode=203: at > org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:204) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceStart(AuxServices.java:173) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceStart(ContainerManagerImpl.java:468) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:265) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:464) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:511) > Caused by: ExitCodeException exitCode=203: at > org.apache.hadoop.util.Shell.runCommand(Shell.java:545) at > org.apache.hadoop.util.Shell.run(Shell.java:456) at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722) at > com.mapr.hadoop.mapred.LocalVolumeAuxService.initVolume(LocalVolumeAuxService.java:260) > at > com.mapr.hadoop.mapred.LocalVolumeAuxService.serviceStart(LocalVolumeAuxService.java:229) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > ... 10 more > 16/03/25 13:51:38 INFO localizer.ResourceLocalizationService: Public cache > exiting > 16/03/25 13:51:38 INFO impl.MetricsSystemImpl: Stopping NodeManager metrics > system... > 16/03/25 13:51:38 INFO impl.MetricsSystemImpl: NodeManager metrics system > stopped. > 16/03/25 13:51:38 INFO impl.MetricsSystemImpl: NodeManager metrics system > shutdown complete. > 16/03/25 13:51:38 FATAL nodemanager.NodeManager: Error starting NodeManager > org.apache.hadoop.service.ServiceStateException: ExitCodeException > exitCode=203: at > org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:204) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceStart(AuxServices.java:173) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceStart(ContainerManagerImpl.java:468) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:265) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:464) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:511) > Caused by: ExitCodeException exitCode=203: at > org.apache.hadoop.util.Shell.runCommand(Shell.java:545) at > org.apache.hadoop.util.Shell.run(Shell.java:456) at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722) at > com.mapr.hadoop.mapred.LocalVolumeAuxService.initVolume(LocalVolumeAuxService.java:260) > at > com.mapr.hadoop.mapred.LocalVolumeAuxService.serviceStart(LocalVolumeAuxService.java:229) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > ... 10 more > 16/03/25 13:51:38 INFO nodemanager.NodeManager: SHUTDOWN_MSG: > /************************************************************ SHUTDOWN_MSG: > Shutting down NodeManager at nodemanager/10.1.194.71 > ************************************************************/ > The log file below from the Node Manager on the server. > createNMVolume.2147483632.log: > ------------------------------------------------------------------------- > 2015-11-03 18:08:48 INFO The command run was: maprcli volume create -name > mapr.namemanager.local.mapred -path /var/mapr/local/namemanager.mapred > -replication 1 -localvolumehost namemanager.-localvolumeport 5660 > -shufflevolume true -rereplicationtimeoutsec 300 > 2015-11-03 18:08:48 INFO The output of the last failed command attempt: ERROR > (22) - FileServer namemanager.5660 has not heartbeated with CLDB for > 1446602926 > 2015-11-03 18:09:23 INFO This script was called with the arguments: > namemanager./var/mapr/local/namemanager.mapred > /var/mapr/local/namemanager.mapred/nodeManager yarn > 2015-11-03 18:09:23 INFO Checking if MapRFS is online > 2015-11-03 18:09:23 DEBUG Will launch command "hadoop fs -stat /" with a > command attempt timeout of 60 seconds a maximum of 1000 attempts and a sleep > time of 1 seconds between failed command attempts > 2015-11-03 18:09:23 DEBUG Launching "hadoop fs -stat /" > 2015-11-03 18:09:31 DEBUG Command attempt 1 completed successfully in 8 > seconds > 2015-11-03 18:09:31 DEBUG Command completed successfully after 1 attempts and > after 8 seconds > 2015-11-03 18:09:31 DEBUG Will launch command "hadoop fs -stat > /var/mapr/local/namemanager. with a command attempt timeout of 60 seconds a > maximum of 1000 attempts and a sleep time of 1 seconds between failed command > attempts > 2015-11-03 18:09:31 DEBUG Launching "hadoop fs -stat > /var/mapr/local/namemanager. > 2015-11-03 18:09:49 DEBUG Command attempt 1 completed successfully in 18 > seconds > 2015-11-03 18:09:49 DEBUG Command completed successfully after 1 attempts and > after 18 seconds > 2015-11-03 18:09:49 INFO MapRFS is online. Checking whether MFS on this node > is online > 2015-11-03 18:09:49 DEBUG Will launch command "/opt/mapr/server/mrconfig -p > 5660 info fsstate" with a command attempt timeout of 60 seconds a maximum of > 60 attempts and a sleep time of 3 seconds between failed command attempts > 2015-11-03 18:09:49 DEBUG Launching "/opt/mapr/server/mrconfig -p 5660 info > fsstate" > 2015-11-03 18:10:03 DEBUG Command attempt 1 completed successfully in 14 > seconds > 2015-11-03 18:10:03 DEBUG Command completed successfully after 1 attempts and > after 14 seconds > 2015-11-03 18:10:03 INFO MFS on this node is online > 2015-11-03 18:10:03 INFO Checking for a volume already mounted at the > specified mount path > 2015-11-03 18:10:03 DEBUG Will launch command "maprcli volume list -filter > [p==/var/mapr/local/namemanager.mapred]and[mt==1] -columns > volumename,mounted,mountdir" with a command attempt timeout of 60 seconds a > maximum of 1 attempts and a sleep time of 1 seconds between failed command > attempts > 2015-11-03 18:10:03 DEBUG Launching "maprcli volume list -filter > [p==/var/mapr/local/namemanager.mapred]and[mt==1] -columns > volumename,mounted,mountdir" > 2015-11-03 18:10:10 DEBUG Command attempt 1 completed successfully in 7 > seconds > 2015-11-03 18:10:10 DEBUG Command completed successfully after 1 attempts and > after 7 seconds > 2015-11-03 18:10:10 INFO The mount path is not currently being used as the > primary mount path of any existing volume > 2015-11-03 18:10:10 INFO Checking for a pre-existing NodeManager volume > 2015-11-03 18:10:10 DEBUG Will launch command "maprcli volume info -name > mapr.namemanager.local.mapred -json" with a command attempt timeout of 60 > seconds a maximum of 1 attempts and a sleep time of 1 seconds between failed > command attempts > 2015-11-03 18:10:10 DEBUG Launching "maprcli volume info -name > mapr.namemanager.local.mapred -json" > 2015-11-03 18:10:32 DEBUG Command attempt 1 failed with return code 1 after > 22 seconds, sleeping for 1 seconds > 2015-11-03 18:10:33 DEBUG Command did not complete successfully after 1 > attempts and after 23 seconds > 2015-11-03 18:10:33 INFO A pre-existing NodeManager volume could not be > found, will try to create one. > 2015-11-03 18:10:33 DEBUG Will launch command "hadoop mfs -ls > /var/mapr/local/namemanager. with a command attempt timeout of 60 seconds a > maximum of 3 attempts and a sleep time of 1 seconds between failed command > attempts > 2015-11-03 18:10:33 DEBUG Launching "hadoop mfs -ls > /var/mapr/local/namemanager. > 2015-11-03 18:10:45 DEBUG Command attempt 1 completed successfully in 12 > seconds > 2015-11-03 18:10:45 DEBUG Command completed successfully after 1 attempts and > after 12 seconds > 2015-11-03 18:10:45 INFO A new NodeManager volume will be created. > 2015-11-03 18:10:45 DEBUG Will launch command "maprcli volume create -name > mapr.namemanager.local.mapred -path /var/mapr/local/namemanager.mapred > -replication 1 -localvolumehost namemanager.-localvolumeport 5660 > -shufflevolume true -rereplicationtimeoutsec 300" with a command attempt > timeout of 60 seconds a maximum of 3 attempts and a sleep time of 1 seconds > between failed command attempts > 2015-11-03 18:10:45 DEBUG Launching "maprcli volume create -name > mapr.namemanager.local.mapred -path /var/mapr/local/namemanager.mapred > -replication 1 -localvolumehost namemanager.-localvolumeport 5660 > -shufflevolume true -rereplicationtimeoutsec 300" > 2015-11-03 18:10:57 DEBUG Command attempt 1 failed with return code 1 after > 12 seconds, sleeping for 1 seconds > 2015-11-03 18:10:58 DEBUG Launching "maprcli volume create -name > mapr.namemanager.local.mapred -path /var/mapr/local/namemanager.mapred > -replication 1 -localvolumehost namemanager.-localvolumeport 5660 > -shufflevolume true -rereplicationtimeoutsec 300" > 2015-11-03 18:11:21 DEBUG Command attempt 2 failed with return code 1 after > 23 seconds, sleeping for 1 seconds > 2015-11-03 18:11:22 DEBUG Launching "maprcli volume create -name > mapr.namemanager.local.mapred -path /var/mapr/local/namemanager.mapred > -replication 1 -localvolumehost namemanager.-localvolumeport 5660 > -shufflevolume true -rereplicationtimeoutsec 300" > 2015-11-03 18:11:34 DEBUG Command attempt 3 failed with return code 1 after > 12 seconds, sleeping for 1 seconds > 2015-11-03 18:11:35 FATAL Command did not complete successfully after 3 > attempts and after 50 seconds. > 2015-11-03 18:11:35 INFO The command run was: maprcli volume create -name > mapr.namemanager.local.mapred -path /var/mapr/local/namemanager.mapred > -replication 1 -localvolumehost namemanager.-localvolumeport 5660 > -shufflevolume true -rereplicationtimeoutsec 300 > 2015-11-03 18:11:35 INFO The output of the last failed command attempt: ERROR > (22) - FileServer namemanager.5660 has not heartbeated with CLDB for > 1446603093 > 2015-11-03 18:12:23 INFO This script was called with the arguments: > namemanager./var/mapr/local/namemanager.mapred > /var/mapr/local/namemanager.mapred/nodeManager yarn > 2015-11-03 18:12:23 INFO Checking if MapRFS is online > 2015-11-03 18:12:23 DEBUG Will launch command "hadoop fs -stat /" with a > command attempt timeout of 60 seconds a maximum of 1000 attempts and a sleep > time of 1 seconds between failed command attempts > 2015-11-03 18:12:23 DEBUG Launching "hadoop fs -stat /" > 2015-11-03 18:12:37 DEBUG Command attempt 1 completed successfully in 14 > seconds > 2015-11-03 18:12:37 DEBUG Command completed successfully after 1 attempts and > after 14 seconds > 2015-11-03 18:12:37 DEBUG Will launch command "hadoop fs -stat > /var/mapr/local/namemanager. with a command attempt timeout of 60 seconds a > maximum of 1000 attempts and a sleep time of 1 seconds between failed command > attempts > 2015-11-03 18:12:37 DEBUG Launching "hadoop fs -stat > /var/mapr/local/namemanager. > 2015-11-03 18:12:47 DEBUG Command attempt 1 completed successfully in 10 > seconds > 2015-11-03 18:12:47 DEBUG Command completed successfully after 1 attempts and > after 10 seconds > 2015-11-03 18:12:47 INFO MapRFS is online. Checking whether MFS on this node > is online > 2015-11-03 18:12:47 DEBUG Will launch command "/opt/mapr/server/mrconfig -p > 5660 info fsstate" with a command attempt timeout of 60 seconds a maximum of > 60 attempts and a sleep time of 3 seconds between failed command attempts > 2015-11-03 18:12:47 DEBUG Launching "/opt/mapr/server/mrconfig -p 5660 info > fsstate" > 2015-11-03 18:12:58 DEBUG Command attempt 1 completed successfully in 11 > seconds > 2015-11-03 18:12:58 DEBUG Command completed successfully after 1 attempts and > after 11 seconds > 2015-11-03 18:12:58 INFO MFS on this node is online > 2015-11-03 18:12:58 INFO Checking for a volume already mounted at the > specified mount path > 2015-11-03 18:12:58 DEBUG Will launch command "maprcli volume list -filter > [p==/var/mapr/local/namemanager.mapred]and[mt==1] -columns > volumename,mounted,mountdir" with a command attempt timeout of 60 seconds a > maximum of 1 attempts and a sleep time of 1 seconds between failed command > attempts > 2015-11-03 18:12:58 DEBUG Launching "maprcli volume list -filter > [p==/var/mapr/local/namemanager.mapred]and[mt==1] -columns > volumename,mounted,mountdir" > 2015-11-03 18:13:11 DEBUG Command attempt 1 completed successfully in 13 > seconds > 2015-11-03 18:13:11 DEBUG Command completed successfully after 1 attempts and > after 13 seconds > 2015-11-03 18:13:11 INFO The mount path is not currently being used as the > primary mount path of any existing volume > 2015-11-03 18:13:11 INFO Checking for a pre-existing NodeManager volume > 2015-11-03 18:13:11 DEBUG Will launch command "maprcli volume info -name > mapr.namemanager.local.mapred -json" with a command attempt timeout of 60 > seconds a maximum of 1 attempts and a sleep time of 1 seconds between failed > command attempts > 2015-11-03 18:13:11 DEBUG Launching "maprcli volume info -name > mapr.namemanager.local.mapred -json" > 2015-11-03 18:13:23 DEBUG Command attempt 1 failed with return code 1 after > 12 seconds, sleeping for 1 seconds > 2015-11-03 18:13:24 DEBUG Command did not complete successfully after 1 > attempts and after 13 seconds > 2015-11-03 18:13:24 INFO A pre-existing NodeManager volume could not be > found, will try to create one. > 2015-11-03 18:13:24 DEBUG Will launch command "hadoop mfs -ls > /var/mapr/local/namemanager. with a command attempt timeout of 60 seconds a > maximum of 3 attempts and a sleep time of 1 seconds between failed command > attempts > 2015-11-03 18:13:24 DEBUG Launching "hadoop mfs -ls > /var/mapr/local/namemanager. > 2015-11-03 18:13:42 DEBUG Command attempt 1 completed successfully in 18 > seconds > 2015-11-03 18:13:42 DEBUG Command completed successfully after 1 attempts and > after 18 seconds > 2015-11-03 18:13:42 INFO A new NodeManager volume will be created. > 2015-11-03 18:13:42 DEBUG Will launch command "maprcli volume create -name > mapr.namemanager.local.mapred -path /var/mapr/local/namemanager.mapred > -replication 1 -localvolumehost namemanager.-localvolumeport 5660 > -shufflevolume true -rereplicationtimeoutsec 300" with a command attempt > timeout of 60 seconds a maximum of 3 attempts and a sleep time of 1 seconds > between failed command attempts > 2015-11-03 18:13:42 DEBUG Launching "maprcli volume create -name > mapr.namemanager.local.mapred -path /var/mapr/local/namemanager.mapred > -replication 1 -localvolumehost namemanager.-localvolumeport 5660 > -shufflevolume true -rereplicationtimeoutsec 300" > 2015-11-03 18:13:49 DEBUG Command attempt 1 failed with return code 1 after 7 > seconds, sleeping for 1 seconds > 2015-11-03 18:13:50 DEBUG Launching "maprcli volume create -name > mapr.namemanager.local.mapred -path /var/mapr/local/namemanager.mapred > -replication 1 -localvolumehost namemanager.-localvolumeport 5660 > -shufflevolume true -rereplicationtimeoutsec 300" > 2015-11-03 18:13:57 DEBUG Command attempt 2 failed with return code 1 after 7 > seconds, sleeping for 1 seconds > 2015-11-03 18:13:58 DEBUG Launching "maprcli volume create -name > mapr.namemanager.local.mapred -path /var/mapr/local/namemanager.mapred > -replication 1 -localvolumehost namemanager.-localvolumeport 5660 > -shufflevolume true -rereplicationtimeoutsec 300" > 2015-11-03 18:14:18 DEBUG Command attempt 3 failed with return code 1 after > 20 seconds, sleeping for 1 seconds > 2015-11-03 18:14:19 FATAL Command did not complete successfully after 3 > attempts and after 37 seconds. > 2015-11-03 18:14:19 INFO The command run was: maprcli volume create -name > mapr.namemanager.local.mapred -path /var/mapr/local/namemanager.mapred > -replication 1 -localvolumehost namemanager.-localvolumeport 5660 > -shufflevolume true -rereplicationtimeoutsec 300 > 2015-11-03 18:14:19 INFO The output of the last failed command attempt: ERROR > (22) - FileServer namemanager.5660 has not heartbeated with CLDB for > 1446603257 -- This message was sent by Atlassian JIRA (v6.3.4#6332)