Hello,
I have a question regarding the Crail S3 storage tier:
https://github.com/zrlio/crail-s3
I am running Crail on Amazon EC2 and getting a connection refused error to
localhost on the client. I have the namenode and s3 datanode running on the
same EC2 instance and the client is on a separate EC2 instance, but all
instances are in the same VPC and subnet. Below is the full output on the
client. Please let me know if you have ideas what I may be missing.
$CRAIL_HOME/bin/crail iobench -t write -f /test.data -s 4096 -k 1 -w 0
18/04/05 07:24:01 INFO crail: creating singleton crail file system
18/04/05 07:24:01 INFO crail: crail.version 3004
18/04/05 07:24:01 INFO crail: crail.directorydepth 16
18/04/05 07:24:01 INFO crail: crail.tokenexpiration 10
18/04/05 07:24:01 INFO crail: crail.blocksize 65536
18/04/05 07:24:01 INFO crail: crail.cachelimit 53687091200
18/04/05 07:24:01 INFO crail: crail.cachepath /dev/hugepages/cache
18/04/05 07:24:01 INFO crail: crail.user stu
18/04/05 07:24:01 INFO crail: crail.shadowreplication 1
18/04/05 07:24:01 INFO crail: crail.debug false
18/04/05 07:24:01 INFO crail: crail.statistics true
18/04/05 07:24:01 INFO crail: crail.rpctimeout 4000
18/04/05 07:24:01 INFO crail: crail.datatimeout 1000
18/04/05 07:24:01 INFO crail: crail.buffersize 524288
18/04/05 07:24:01 INFO crail: crail.slicesize 262144
18/04/05 07:24:01 INFO crail: crail.singleton true
18/04/05 07:24:01 INFO crail: crail.regionsize 1073741824
18/04/05 07:24:01 INFO crail: crail.directoryrecord 512
18/04/05 07:24:01 INFO crail: crail.directoryrandomize true
18/04/05 07:24:01 INFO crail: crail.cacheimpl
org.apache.crail.memory.MappedBufferCache
18/04/05 07:24:01 INFO crail: crail.locationmap
18/04/05 07:24:01 INFO crail: crail.namenode.address crail://
10.1.217.199:9070
18/04/05 07:24:01 INFO crail: crail.namenode.blockselection roundrobin
18/04/05 07:24:01 INFO crail: crail.namenode.fileblocks 16
18/04/05 07:24:01 INFO crail: crail.namenode.rpctype
org.apache.crail.namenode.rpc.tcp.TcpNameNode
18/04/05 07:24:01 INFO crail: crail.namenode.log
18/04/05 07:24:01 INFO crail: crail.storage.types
com.ibm.crail.storage.object.ObjectStorageTier
18/04/05 07:24:01 INFO crail: crail.storage.classes 1
18/04/05 07:24:01 INFO crail: crail.storage.rootclass 0
18/04/05 07:24:01 INFO crail: crail.storage.keepalive 2
18/04/05 07:24:01 INFO crail: crail.storage.object.storagelimit = 1073741824
18/04/05 07:24:01 INFO crail: crail.storage.object.allocationsize = 1048576
18/04/05 07:24:01 INFO crail: crail.storage.object.datanode = localhost
18/04/05 07:24:01 INFO crail: crail.storage.object.datanodeport = 50080
18/04/05 07:24:01 INFO crail: crail.storage.object.objectprefix = crail
18/04/05 07:24:01 INFO crail: crail.storage.object.cleanuponexit = false
18/04/05 07:24:01 INFO crail: crail.storage.object.profile = false
18/04/05 07:24:01 INFO crail: crail.storage.object.s3accesskey = XXX
18/04/05 07:24:01 INFO crail: crail.storage.object.s3secretkey = XXXXX
18/04/05 07:24:01 INFO crail: crail.storage.object.s3region = us-west-2
18/04/05 07:24:01 INFO crail: crail.storage.object.s3bucket = anakli-pocket
18/04/05 07:24:01 INFO crail: crail.storage.object.s3endpoint =
s3.us-west-2.amazonaws.com
18/04/05 07:24:01 INFO crail: crail.storage.object.s3protocol = HTTP
18/04/05 07:24:01 INFO crail: crail.storage.object.s3signer = null
18/04/05 07:24:01 INFO narpc: new NaRPC server group v1.0, queueDepth 1024,
messageSize 512, nodealy true
18/04/05 07:24:01 INFO crail: crail.namenode.tcp.queueDepth 1024
18/04/05 07:24:01 INFO crail: crail.namenode.tcp.messageSize 512
18/04/05 07:24:01 INFO crail: crail.namenode.tcp.cores 1
18/04/05 07:24:01 INFO crail: connected to namenode(s) /10.1.217.199:9070
18/04/05 07:24:01 INFO crail: buffer cache, allocationCount 50, bufferCount
2048
write, filename /test.data, size 4096, loop 1, storageClass 0,
locationClass 0, buffered true
warmUp, warmupFile /test.data-972347181, operations 0
starting benchmark...
18/04/05 07:24:01 INFO crail: ERROR: failed data operation
io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
refused: localhost/127.0.0.1:50080
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at
io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:257)
at
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:291)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:640)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:575)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:489)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:451)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
at
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "main" java.util.concurrent.ExecutionException:
io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
refused: localhost/127.0.0.1:50080
at
org.apache.crail.core.CoreMetaDataOperation.get(CoreMetaDataOperation.java:98)
at org.apache.crail.tools.CrailBenchmark.write(CrailBenchmark.java:108)
at org.apache.crail.tools.CrailBenchmark.main(CrailBenchmark.java:1071)
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException:
Connection refused: localhost/127.0.0.1:50080
