Hi Nikolay, I tried to comment the line IGNITE_XML_CONFIG= ,but the Ignite YARN failed.
[hongmei@c5hdpe001 config]$ hadoop jar /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar /u/hongmei/apache-ignite/config/cluster.properties WARNING: Use "yarn jar" to launch YARN applications. 16/06/09 10:31:43 INFO impl.TimelineClientImpl: Timeline service address: http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/ 16/06/09 10:31:47 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2 16/06/09 10:31:49 INFO impl.YarnClientImpl: Submitted application application_1464374946035_29710 Jun 09, 2016 10:31:49 AM org.apache.ignite.yarn.IgniteYarnClient main INFO: Submitted application. Application id: application_1464374946035_29710 Jun 09, 2016 10:31:59 AM org.apache.ignite.yarn.IgniteYarnClient main INFO: Application application_1464374946035_29710 is FAILED. On Thu, Jun 9, 2016 at 10:21 AM, Nikolay Tikhonov <ntikho...@gridgain.com> wrote: > You set wrong value to IGNITE_XML_CONFIG property. The property should > contains path to ignite configuration file. For example > > IGNITE_XML_CONFIG=/u/hongmei/apache-ignite/config/default-config.xml > > I think you can comment this line in property file and ignite will start > with default configuration. > > On Thu, Jun 9, 2016 at 5:08 PM, Hongmei Zong <hong...@kayak.com> wrote: > >> Hi nikolai, >> >> Thank you very much for prompt reply! >> >> I did not find the ignite-config.xml file under my ignite home directory( >> /u/hongmei/apache-ignite/ ). >> >> I find a "default-config.xml" at the path: >> /u/hongmei/apache-ignite/config/default-config.xml >> >> <?xml version="1.0" encoding="UTF-8"?> >> >> >> <!-- >> >> Licensed to the Apache Software Foundation (ASF) under one or more >> >> contributor license agreements. See the NOTICE file distributed with >> >> this work for additional information regarding copyright ownership. >> >> The ASF licenses this file to You under the Apache License, Version 2.0 >> >> (the "License"); you may not use this file except in compliance with >> >> the License. You may obtain a copy of the License at >> >> >> http://www.apache.org/licenses/LICENSE-2.0 >> >> >> Unless required by applicable law or agreed to in writing, software >> >> distributed under the License is distributed on an "AS IS" BASIS, >> >> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. >> >> See the License for the specific language governing permissions and >> >> limitations under the License. >> >> --> >> >> >> <beans xmlns="http://www.springframework.org/schema/beans"; >> >> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >> >> xsi:schemaLocation=" >> >> http://www.springframework.org/schema/beans >> >> http://www.springframework.org/schema/beans/spring-beans.xsd";> >> >> <!-- >> >> Alter configuration below as needed. >> >> --> >> >> <bean id="grid.cfg" class= >> "org.apache.ignite.configuration.IgniteConfiguration"> >> >> <property name="discoverySpi"> >> >> <bean class= >> "org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> >> >> <property name="ipFinder"> >> >> <bean class= >> "org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder"> >> >> <property name= >> "addresses"> >> >> <list> >> >> <!-- >> >> >> Explicitly specifying address of a local node to let it start >> >> and >> operate normally even if there is no more nodes in the cluster. >> >> You can >> also optionally specify an individual port or port range. >> >> --> >> >> <!-- >> >> >> <value>1.2.3.4</value> >> >> --> >> >> <!-- >> >> >> IP Address and optional port range of a remote node. >> >> >> You can also optionally specify an individual port and don't set >> >> >> the port range at all. >> >> --> >> >> <value> >> c5hdpe001.c5.runwaynine.com:47500..47509</value> >> >> <value> >> c5hdpe002.c5.runwaynine.com:47500..47509</value> >> >> <value> >> c5hdpe003.c5.runwaynine.com:47500..47509</value> >> >> >> </list> >> >> </property> >> >> </bean> >> >> </property> >> >> </bean> >> >> </property> >> >> </bean> >> >> </beans> >> >> >> >> The "cluster.properties" at the path: >> /u/hongmei/apache-ignite/config/cluster.properties >> >> # The number of nodes in the cluster. >> >> IGNITE_NODE_COUNT=2 >> >> >> # The number of CPU Cores for each Apache Ignite node. >> >> IGNITE_RUN_CPU_PER_NODE=1 >> >> >> # The number of Megabytes of RAM for each Apache Ignite node. >> >> IGNITE_MEMORY_PER_NODE=2048 >> >> >> # The version ignite which will be run on nodes. >> >> IGNITE_VERSION=1.6.0 >> >> >> # The hdfs directory which will be used for saving Apache Ignite >> disbributives. >> >> IGNITE_RELEASES_DIR=/user/hongmei/ignite-yarn >> >> >> #The directory which will be used for saving Apache Ignite >> distributives(copy .jar file to it). >> >> IGNITE_WORKING_DIR=/user/hongmei/ignite/workdir >> >> >> #The hdfs path to Apache Ignite config file. >> >> #IGNITE_XML_CONFIG=/user/yarn/ignite/ >> >> IGNITE_XML_CONFIG=/user/hongmei/ignite/ >> >> >> #The hdfs path to libs which will be added to classpath. >> >> IGNITE_USERS_LIBS=/user/hongmei/ignite/libs/ >> >> >> #The constraint on slave hosts. >> >> #IGNITE_HOSTENAME_CONSTRAINT= >> >> >> IGNITE_LOCAL_WORK_DIR=/u/hongmei/apache-ignite >> >> >> >> Thank you very much!!!! >> >> Hongmei >> >> >> >> >> >> On Thu, Jun 9, 2016 at 9:43 AM, Nikolai Tikhonov <ntikho...@apache.org> >> wrote: >> >>> It seems that your ignite configuration invalid. Could you share >>> ignite-config.xml >>> and /u/hongmei/apache-ignite/config/cluster.properties? >>> >>> On Thu, Jun 9, 2016 at 4:35 PM, Hongmei Zong <hong...@kayak.com> wrote: >>> >>>> Thank you very much Nikolai ! >>>> >>>> I found another issue regarding my Ignite YARN Integration: >>>> >>>> I run Ignite YARN application on one client server machine >>>> 'c5hdpe001', screenshot as the following: >>>> Then I log onto one of the container node and the log is as the >>>> following: >>>> >>>> [hongmei@c5hdpe001 apache-ignite]$ hadoop jar >>>> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar >>>> /u/hongmei/apache-ignite/libs/optional/ignite-yarn/ignite-yarn-1.6.0.jar >>>> /u/hongmei/apache-ignite/config/cluster.properties >>>> >>>> WARNING: Use "yarn jar" to launch YARN applications. >>>> >>>> 16/06/09 08:55:20 INFO impl.TimelineClientImpl: Timeline service >>>> address: http://c5hdp003.c5.runwaynine.com:8188/ws/v1/timeline/ >>>> >>>> 16/06/09 08:55:22 INFO client.ConfiguredRMFailoverProxyProvider: >>>> Failing over to rm2 >>>> >>>> 16/06/09 08:55:24 INFO impl.YarnClientImpl: Submitted application >>>> application_1464374946035_29511 >>>> >>>> Jun 09, 2016 8:55:24 AM org.apache.ignite.yarn.IgniteYarnClient main >>>> >>>> INFO: Submitted application. Application id: >>>> application_1464374946035_29511 >>>> >>>> Jun 09, 2016 8:55:27 AM org.apache.ignite.yarn.IgniteYarnClient main >>>> *INFO: Application application_1464374946035_29511 is RUNNING.* >>>> >>>> >>>> Then I log onto one of the container node and the log is as the >>>> following: "It failed to start grid". >>>> Any good suggestion? Thank you very much!!! >>>> >>>> >>>> class org.apache.ignite.IgniteException: Failed to instantiate Spring >>>> XML application context >>>> [springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/, >>>> err=Line 1 in XML document from URL >>>> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/] >>>> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: >>>> 1; columnNumber: 1; Content is not allowed in prolog.] >>>> at >>>> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:904) >>>> at org.apache.ignite.Ignition.start(Ignition.java:350) >>>> at >>>> org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:302) >>>> *Caused by: class org.apache.ignite.IgniteCheckedException: Failed to >>>> instantiate Spring XML application context >>>> [springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/, >>>> err=Line 1 in XML document from URL >>>> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/] >>>> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: >>>> 1; columnNumber: 1; Content is not allowed in prolog.]* >>>> at >>>> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:391) >>>> at >>>> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:104) >>>> at >>>> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:98) >>>> at >>>> org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:606) >>>> at >>>> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:807) >>>> at >>>> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:716) >>>> at >>>> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:586) >>>> at >>>> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:556) >>>> at org.apache.ignite.Ignition.start(Ignition.java:347) >>>> ... 1 more >>>> *Caused by: >>>> org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line >>>> 1 in XML document from URL >>>> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/] >>>> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: >>>> 1; columnNumber: 1; Content is not allowed in prolog.* >>>> at >>>> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:398) >>>> at >>>> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:335) >>>> at >>>> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:303) >>>> at >>>> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:379) >>>> ... 9 more >>>> *Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: >>>> 1; Content is not allowed in prolog.* >>>> at >>>> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:198) >>>> at >>>> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177) >>>> at >>>> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:441) >>>> at >>>> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:368) >>>> at >>>> com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1436) >>>> at >>>> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:999) >>>> at >>>> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606) >>>> at >>>> com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:117) >>>> at >>>> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510) >>>> at >>>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848) >>>> at >>>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777) >>>> at >>>> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) >>>> at >>>> com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243) >>>> at >>>> com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347) >>>> at >>>> org.springframework.beans.factory.xml.DefaultDocumentLoader.loadDocument(DefaultDocumentLoader.java:76) >>>> at >>>> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadDocument(XmlBeanDefinitionReader.java:428) >>>> at >>>> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:390) >>>> ... 12 more >>>> *Failed to start grid: Failed to instantiate Spring XML application >>>> context >>>> *[springUrl=file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/, >>>> err=Line 1 in XML document from URL >>>> [file:/disk/12/hadoop/yarn/local/usercache/hongmei/appcache/application_1464374946035_27403/container_e24_1464374946035_27403_01_110493/./ignite-config.xml/] >>>> is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: >>>> 1; columnNumber: 1; Content is not allowed in prolog.] >>>> >>>> >>>> Hongmei >>>> >>>> >>>> On Thu, Jun 9, 2016 at 6:04 AM, Nikolai Tikhonov <ntikho...@apache.org> >>>> wrote: >>>> >>>>> *your_address1:47500..47510,your_address2:47500..47510 >>>>>> and your_address3:47500..47510 are the YARN master_host address, right?* >>>>>> >>>>> >>>>> No, this addresses hosts on which deploy YARN cluster. For example, >>>>> you have YARN cluster which contains two servers: 10.0.0.1 and 10.0.0.2. >>>>> In >>>>> this case you will have the following configuration: >>>>> >>>>> ipFinder.setAddresses(Arrays.asList("10.0.0.1:47500..47510", >>>>> "10.0.0.2:47500..47510")); >>>>> >>>>> >>>> >>> >> >
