I have a cluster launched with spark-ec2.
I can see a TachyonMaster process running,
but I do not seem to be able to use tachyon from the spark-shell.

if I try

rdd.saveAsTextFile("tachyon://localhost:19998/path")
I get

15/04/24 19:18:31 INFO TaskSetManager: Starting task 12.2 in stage 1.0 (TID
216, ip-10-63-69-48.ec2.internal, PROCESS_LOCAL, 1383 bytes)
15/04/24 19:18:31 WARN TaskSetManager: Lost task 32.2 in stage 1.0 (TID
177, ip-10-63-69-48.ec2.internal): java.io.IOException: Failed to connect
to master localhost/127.0.0.1:19998 after 5 attempts
        at tachyon.client.TachyonFS.connect(TachyonFS.java:293)
        at tachyon.client.TachyonFS.getUnderfsAddress(TachyonFS.java:1224)
        at tachyon.hadoop.TFS.initialize(TFS.java:289)
        at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2262)
        at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:86)
        at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2296)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2278)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:316)
        at org.apache.hadoop.fs.Path.getFileSystem(Path.java:194)
        at
org.apache.spark.SparkHadoopWriter.open(SparkHadoopWriter.scala:83)
        at
org.apache.spark.rdd.PairRDDFunctions$$anonfun$13.apply(PairRDDFunctions.scala:1068)
        at
org.apache.spark.rdd.PairRDDFunctions$$anonfun$13.apply(PairRDDFunctions.scala:1059)
        at java.lang.Thread.run(Thread.java:745)
Caused by: tachyon.org.apache.thrift.TException: Failed to connect to
master localhost/127.0.0.1:19998 after 5 attempts
        at tachyon.master.MasterClient.connect(MasterClient.java:178)
        at tachyon.client.TachyonFS.connect(TachyonFS.java:290)
        ... 17 more
Caused by: tachyon.org.apache.thrift.transport.TTransportException:
java.net.ConnectException: Connection refused
        at
tachyon.org.apache.thrift.transport.TSocket.open(TSocket.java:185)
        at
tachyon.org.apache.thrift.transport.TFramedTransport.open(TFramedTransport.java:81)
        at tachyon.master.MasterClient.connect(MasterClient.java:156)
        ... 18 more
Caused by: java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at
tachyon.org.apache.thrift.transport.TSocket.open(TSocket.java:180)
        ... 20 more


 What do I need to do before I can use tachyon?

thanks
Daniel

Reply via email to