Hello! I don't think you actually want to use SHMEM. How about just using @localhost:10500? -- Ilya Kasnacheev
чт, 23 мая 2019 г. в 13:15, joaquinsanroman <[email protected]>: > Hi, > > First of all, thak you very much for your help. > > I have configured one IGFS cluster without HDFS secondary filesystem > because > the intention is to use IGFS as an independent storage. > > The configuration file for all server nodes is the next: > > <?xml version="1.0" encoding="UTF-8"?> > > <beans xmlns="http://www.springframework.org/schema/beans"; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xmlns:util="http://www.springframework.org/schema/util"; > xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd > http://www.springframework.org/schema/util > http://www.springframework.org/schema/util/spring-util.xsd";> > > > <description> > Spring file for Ignite node configuration with IGFS and Apache > Hadoop map-reduce support enabled. > Ignite node will start with this configuration by default. > </description> > > > <bean id="propertyConfigurer" > > class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"> > <property name="systemPropertiesModeName" > value="SYSTEM_PROPERTIES_MODE_FALLBACK"/> > <property name="searchSystemEnvironment" value="true"/> > </bean> > > > <bean id="grid.cfg" > class="org.apache.ignite.configuration.IgniteConfiguration"> > > <property name="connectorConfiguration"> > <bean > class="org.apache.ignite.configuration.ConnectorConfiguration"> > <property name="port" value="11211"/> > </bean> > </property> > > <property name="discoverySpi"> > <bean class="org.apache.ignite.spi.discovery.zk.ZookeeperDiscoverySpi"> > <property name="zkConnectionString" > > value="sanlbeclomi0001.santander.pre.corp:2181,sanlbeclomi0002.santander.pre.corp:2181,sanlbeclomi0003.santander.pre.corp:2181"/> > <property name="sessionTimeout" value="30000"/> > <property name="zkRootPath" value="/apacheIgnite"/> > <property name="joinTimeout" value="10000"/> > </bean> > </property> > > > <property name="fileSystemConfiguration"> > <list> > <bean > class="org.apache.ignite.configuration.FileSystemConfiguration"> > > <property name="name" value="igfs"/> > <property name="ipcEndpointEnabled" > value="true"/> > > > <property name="blockSize" value="#{128 * 1024}"/> > <property name="perNodeBatchSize" value="512"/> > <property name="perNodeParallelBatchCount" value="16"/> > > > <property name="prefetchBlocks" value="32"/> > > > <property name="DataCacheConfiguration"> > <bean > class="org.apache.ignite.configuration.CacheConfiguration"> > > <property name="name" value="myDataCache"/> > > <property name="cacheMode" > value="PARTITIONED"/> > > <property name="atomicityMode" > value="TRANSACTIONAL"/> > <property name="backups" value="1"/> > </bean> > </property> > > > <property name="MetaCacheConfiguration"> > <bean > class="org.apache.ignite.configuration.CacheConfiguration"> > > <property name="name" value="myMetaCache"/> > > <property name="cacheMode" > value="PARTITIONED"/> > > <property name="atomicityMode" > value="TRANSACTIONAL"/> > <property name="backups" value="1"/> > > </bean> > </property> > > > <property name="ipcEndpointConfiguration"> > <bean > class="org.apache.ignite.igfs.IgfsIpcEndpointConfiguration"> > <property name="type" value="TCP"/> > <property name="host" value="0.0.0.0"/> > <property > name="port" value="10500"/> > </bean> > </property> > > </bean> > > </list> > </property> > </bean> > </beans> > > I can access to the IGFS through java setting the same configuration file > and adding the Ignition.setClientMode(true). > In spark, I am also able to access data remotly seting the client mode and > including the libraries into de classpath. > > My problem is when I try to get information or files directly through "hdfs > dfs". I have set the properties inside the core-site.xml: > > fs.igfs.impl > org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem > Class name mapping > > fs.AbstractFileSystem.igfs.impl > org.apache.ignite.hadoop.fs.v2.IgniteHadoopFileSystem > Class name mapping > > fs.igfs.igfs.config_path > /tmp/IGFSConfigZookeeper.xml > Class name mapping > > fs.igfs.igfs.endpoint.no_embed > true > Class name mapping > > The /tmp/IGFSConfigZookeeper.xml has the same configuration of the ignite > servers but including the property: > > <property name="clientMode" value="true"/> > > When I run the command "hdfs dfs -ls igfs://igfs@/", I get the error: > > ls: Failed to communicate with IGFS: Failed to connect to IGFS > [endpoint=igfs://igfs@, attempts=[[type=SHMEM, port=10500, > err=java.io.IOException: Failed to connect shared memory endpoint to port > (is shared memory server endpoint up and running?): 10500], [type=TCP, > host=127.0.0.1, port=10500, err=java.io.IOException: Failed to connect to > endpoint [host=127.0.0.1, port=10500]]] (ensure that IGFS is running and > have IPC endpoint enabled; ensure that ignite-shmem-1.0.0.jar is in Hadoop > classpath if you use shared memory endpoint). > > Otherwise, it runs ok when I execute: hdfs dfs -ls igfs://igfs@/host1:10500/ > > > I think the problem is I am not setting ok the client conection properties > into the /tmp/IGFSConfigZookeeper.xml file. > > Could you help me with the error? > > Thank you very much, > Regards. > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
