Hi Nikolay, I tried to comment the line IGNITE_XML_CONFIG= ,but the Ignite YARN failed.
[hongmei@c5hdpe001 config]$ hadoop jar /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar /u/hongmei/apache-ignite/config/cluster.properties WARNING: Use "yarn jar" to launch YARN applications. 16/06/09 10:31:43 INFO impl.TimelineClientImpl: Timeline service address: http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/ 16/06/09 10:31:47 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2 16/06/09 10:31:49 INFO impl.YarnClientImpl: Submitted application application_1464374946035_29710 Jun 09, 2016 10:31:49 AM org.apache.ignite.yarn.IgniteYarnClient main INFO: Submitted application. Application id: application_1464374946035_29710 Jun 09, 2016 10:31:59 AM org.apache.ignite.yarn.IgniteYarnClient main INFO: Application application_1464374946035_29710 is FAILED. On Thu, Jun 9, 2016 at 10:23 AM, Hongmei Zong <[email protected]> wrote: > Thank you very much Nikolay, I will give it a try! :) > > Hongmei > > On Thu, Jun 9, 2016 at 10:21 AM, Nikolay Tikhonov <[email protected]> > wrote: > >> You set wrong value to IGNITE_XML_CONFIG property. The property should >> contains path to ignite configuration file. For example >> >> IGNITE_XML_CONFIG=/u/hongmei/apache-ignite/config/default-config.xml >> >> I think you can comment this line in property file and ignite will start >> with default configuration. >> >> On Thu, Jun 9, 2016 at 5:08 PM, Hongmei Zong <[email protected]> wrote: >> >>> Hi nikolai, >>> >>> Thank you very much for prompt reply! >>> >>> I did not find the ignite-config.xml file under my ignite home directory( >>> /u/hongmei/apache-ignite/ ). >>> >>> I find a "default-config.xml" at the path: >>> /u/hongmei/apache-ignite/config/default-config.xml >>> >>> <?xml version="1.0" encoding="UTF-8"?> >>> >>> >>> <!-- >>> >>> Licensed to the Apache Software Foundation (ASF) under one or more >>> >>> contributor license agreements. See the NOTICE file distributed with >>> >>> this work for additional information regarding copyright ownership. >>> >>> The ASF licenses this file to You under the Apache License, Version 2.0 >>> >>> (the "License"); you may not use this file except in compliance with >>> >>> the License. You may obtain a copy of the License at >>> >>> >>> http://www.apache.org/licenses/LICENSE-2.0 >>> >>> >>> Unless required by applicable law or agreed to in writing, software >>> >>> distributed under the License is distributed on an "AS IS" BASIS, >>> >>> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or >>> implied. >>> >>> See the License for the specific language governing permissions and >>> >>> limitations under the License. >>> >>> --> >>> >>> >>> <beans xmlns="http://www.springframework.org/schema/beans"; >>> >>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >>> >>> xsi:schemaLocation=" >>> >>> http://www.springframework.org/schema/beans >>> >>> http://www.springframework.org/schema/beans/spring-beans.xsd";> >>> >>> <!-- >>> >>> Alter configuration below as needed. >>> >>> --> >>> >>> <bean id="grid.cfg" class= >>> "org.apache.ignite.configuration.IgniteConfiguration"> >>> >>> <property name="discoverySpi"> >>> >>> <bean class= >>> "org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> >>> >>> <property name="ipFinder"> >>> >>> <bean class= >>> "org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder" >>> > >>> >>> <property name= >>> "addresses"> >>> >>> <list> >>> >>> <!-- >>> >>> >>> Explicitly specifying address of a local node to let it start >>> >>> and >>> operate normally even if there is no more nodes in the cluster. >>> >>> You can >>> also optionally specify an individual port or port range. >>> >>> --> >>> >>> <!-- >>> >>> >>> <value>1.2.3.4</value> >>> >>> --> >>> >>> <!-- >>> >>> >>> IP Address and optional port range of a remote node. >>> >>> >>> You can also optionally specify an individual port and don't set >>> >>> >>> the port range at all. >>> >>> --> >>> >>> <value> >>> c5hdpe001.c5.runwaynine.com:47500..47509</value> >>> >>> <value> >>> c5hdpe002.c5.runwaynine.com:47500..47509</value> >>> >>> <value> >>> c5hdpe003.c5.runwaynine.com:47500..47509</value> >>> >>> >>> </list> >>> >>> </property> >>> >>> </bean> >>> >>> </property> >>> >>> </bean> >>> >>> </property> >>> >>> </bean> >>> >>> </beans> >>> >>> >>> >>> The "cluster.properties" at the path: >>> /u/hongmei/apache-ignite/config/cluster.properties >>> >>> # The number of nodes in the cluster. >>> >>> IGNITE_NODE_COUNT=2 >>> >>> >>> # The number of CPU Cores for each Apache Ignite node. >>> >>> IGNITE_RUN_CPU_PER_NODE=1 >>> >>> >>> # The number of Megabytes of RAM for each Apache Ignite node. >>> >>> IGNITE_MEMORY_PER_NODE=2048 >>> >>> >>> # The version ignite which will be run on nodes. >>> >>> IGNITE_VERSION=1.6.0 >>> >>> >>> # The hdfs directory which will be used for saving Apache Ignite >>> disbributives. >>> >>> IGNITE_RELEASES_DIR=/user/hongmei/ignite-yarn >>> >>> >>> #The directory which will be used for saving Apache Ignite >>> distributives(copy .jar file to it). >>> >>> IGNITE_WORKING_DIR=/user/hongmei/ignite/workdir >>> >>> >>> #The hdfs path to Apache Ignite config file. >>> >>> #IGNITE_XML_CONFIG=/user/yarn/ignite/ >>> >>> IGNITE_XML_CONFIG=/user/hongmei/ignite/ >>> >>> >>> #The hdfs path to libs which will be added to classpath. >>> >>> IGNITE_USERS_LIBS=/user/hongmei/ignite/libs/ >>> >>> >>> #The constraint on slave hosts. >>> >>> #IGNITE_HOSTENAME_CONSTRAINT= >>> >>> >>> IGNITE_LOCAL_WORK_DIR=/u/hongmei/apache-ignite >>> >>> >>> >>> Thank you very much!!!! >>> >>> Hongmei >>> >>> >>> >>> >>> >>> On Thu, Jun 9, 2016 at 9:43 AM, Nikolai Tikhonov <[email protected]> >>> wrote: >>> >>>> It seems that your ignite configuration invalid. Could you share >>>> ignite-config.xml >>>> and /u/hongmei/apache-ignite/config/cluster.properties? >>>> >>>> On Thu, Jun 9, 2016 at 4:35 PM, Hongmei Zong <[email protected]> wrote: >>>> >>>>> Thank you very much Nikolai ! >>>>> >>>>> I found another issue regarding my Ignite YARN Integration: >>>>> >>>>> I run Ignite YARN application on one client server machine >>>>> 'c5hdpe001', screenshot as the following: >>>>> Then I log onto one of the container node and the log is as the >>>>> following: >>>>> >>>>> [hongmei@c5hdpe001 apache-ignite]$ hadoop jar >>>>> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar >>>>> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar >>>>> /u/hongmei/apache-ignite/config/cluster.properties >>>>> >>>>> WARNING: Use "yarn jar" to launch YARN applications. >>>>> >>>>> 16/06/09 08:55:20 INFO impl.TimelineClientImpl: Timeline service >>>>> address: http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/ >>>>> >>>>> 16/06/09 08:55:22 INFO client.ConfiguredRMFailoverProxyProvider: >>>>> Failing over to rm2 >>>>> >>>>> 16/06/09 08:55:24 INFO impl.YarnClientImpl: Submitted application >>>>> application_1464374946035_29511 >>>>> >>>>> Jun 09, 2016 8:55:24 AM org.apache.ignite.yarn.IgniteYarnClient main >>>>> >>>>> INFO: Submitted application. Application id: >>>>> application_1464374946035_29511 >>>>> >>>>> Jun 09, 2016 8:55:27 AM org.apache.ignite.yarn.IgniteYarnClient main >>>>> *INFO: Application application_1464374946035_29511 is RUNNING.* >>>>> >>>>> >>>>> Then I log onto one of the container node and the log is as the >>>>> following: "It failed to start grid". >>>>> Any good suggestion? Thank you very much!!! >>>>> >>>>> >>>>> class org.apache.ignite.IgniteException: Failed to instantiate Spring >>>>> XML application context >>>>> [springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/, >>>>> err=Line 1 in XML document from URL >>>>> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/] >>>>> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: >>>>> 1; columnNumber: 1; Content is not allowed in prolog.] >>>>> at >>>>> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:904) >>>>> at org.apache.ignite.Ignition.start(Ignition.java:350) >>>>> at >>>>> org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:302) >>>>> *Caused by: class org.apache.ignite.IgniteCheckedException: Failed to >>>>> instantiate Spring XML application context >>>>> [springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/, >>>>> err=Line 1 in XML document from URL >>>>> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/] >>>>> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: >>>>> 1; columnNumber: 1; Content is not allowed in prolog.]* >>>>> at >>>>> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:391) >>>>> at >>>>> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104) >>>>> at >>>>> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98) >>>>> at >>>>> org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:606) >>>>> at >>>>> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:807) >>>>> at >>>>> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:716) >>>>> at >>>>> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:586) >>>>> at >>>>> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:556) >>>>> at org.apache.ignite.Ignition.start(Ignition.java:347) >>>>> ... 1 more >>>>> *Caused by: >>>>> org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: >>>>> Line >>>>> 1 in XML document from URL >>>>> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/] >>>>> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: >>>>> 1; columnNumber: 1; Content is not allowed in prolog.* >>>>> at >>>>> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:398) >>>>> at >>>>> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:335) >>>>> at >>>>> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:303) >>>>> at >>>>> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:379) >>>>> ... 9 more >>>>> *Caused by: org.xml.sax.SAXParseException; lineNumber: 1; >>>>> columnNumber: 1; Content is not allowed in prolog.* >>>>> at >>>>> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:198) >>>>> at >>>>> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177) >>>>> at >>>>> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:441) >>>>> at >>>>> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:368) >>>>> at >>>>> com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1436) >>>>> at >>>>> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:999) >>>>> at >>>>> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606) >>>>> at >>>>> com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:117) >>>>> at >>>>> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510) >>>>> at >>>>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848) >>>>> at >>>>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777) >>>>> at >>>>> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) >>>>> at >>>>> com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243) >>>>> at >>>>> com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347) >>>>> at >>>>> org.springframework.beans.factory.xml.DefaultDocumentLoader.loadDocument(DefaultDocumentLoader.java:76) >>>>> at >>>>> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadDocument(XmlBeanDefinitionReader.java:428) >>>>> at >>>>> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:390) >>>>> ... 12 more >>>>> *Failed to start grid: Failed to instantiate Spring XML application >>>>> context >>>>> *[springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/, >>>>> err=Line 1 in XML document from URL >>>>> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/] >>>>> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: >>>>> 1; columnNumber: 1; Content is not allowed in prolog.] >>>>> >>>>> >>>>> Hongmei >>>>> >>>>> >>>>> On Thu, Jun 9, 2016 at 6:04 AM, Nikolai Tikhonov <[email protected] >>>>> > wrote: >>>>> >>>>>> *your_address1:47500..47510,your_address2:47500..47510 >>>>>>> and your_address3:47500..47510 are the YARN master_host address, right?* >>>>>>> >>>>>> >>>>>> No, this addresses hosts on which deploy YARN cluster. For example, >>>>>> you have YARN cluster which contains two servers: 10.0.0.1 and 10.0.0.2. >>>>>> In >>>>>> this case you will have the following configuration: >>>>>> >>>>>> ipFinder.setAddresses(Arrays.asList("10.0.0.1:47500..47510", >>>>>> "10.0.0.2:47500..47510")); >>>>>> >>>>>> >>>>> >>>> >>> >> >
