Hello,
I'm trying to get a 1-to-1 edge connection working and running into this
exception on the yarn application master:
Vertex failed, vertexName=v1, vertexId=vertex_1472251099946_0002_1_00,
diagnostics=[Task failed, taskId=task_1472251099946_0002_1_00_000000,
diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task (
failure ) : java.lang.RuntimeException: java.io.IOException: Failed on local
exception: java.nio.channels.ClosedByInterruptException; Host Details : local
host is: "shasta.local/127.0.0.1"; destination host is: "localhost":9000;at
org.apache.hadoop.mapreduce.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:209)at
org.apache.hadoop.mapreduce.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initialize(TezGroupedSplitsInputFormat.java:156)at
org.apache.tez.mapreduce.lib.MRReaderMapReduce.setupNewRecordReader(MRReaderMapReduce.java:157)at
org.apache.tez.mapreduce.lib.MRReaderMapReduce.setSplit(MRReaderMapReduce.java:88)at
org.apache.tez.mapreduce.input.MRInput.initFromEventInternal(MRInput.java:694)at
org.apache.tez.mapreduce.input.MRInput.processSplitEvent(MRInput.java:622)at
org.apache.tez.mapreduce.input.MRInput.handleEvents(MRInput.java:586)at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.handleEvent(LogicalIOProcessorRuntimeTask.java:715)at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.access$600(LogicalIOProcessorRuntimeTask.java:105)at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$1.runInternal(LogicalIOProcessorRuntimeTask.java:792)at
org.apache.tez.common.RunnableWithNdc.run(RunnableWithNdc.java:35)at
java.lang.Thread.run(Thread.java:745)Caused by: java.io.IOException: Failed on
local exception: java.nio.channels.ClosedByInterruptException; Host Details :
local host is: "shasta.local/127.0.0.1"; destination host is:
"localhost":9000;at
org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)at
org.apache.hadoop.ipc.Client.call(Client.java:1472)at
org.apache.hadoop.ipc.Client.call(Client.java:1399)
Here is my setup:My Hadoop 2.6.4 is running in pseudo-cluster mode and I've
subclassed TezExamplesBase and setting up the DAG the following way: v1 -> v2.
>>>>>> DataSourceDescriptor dataSource = MRInput.createConfigBuilder(new
>>>>>>Configuration(tezConf), TextInputFormat.class, inputPath)
>>>>>>.groupSplits(!isDisableSplitGrouping())
>>>>>>.generateSplitsInAM(!isGenerateSplitInClient()) .build();
DataSinkDescriptor dataSink = MROutput.createConfigBuilder(new
Configuration(tezConf), TextOutputFormat.class, outputPath)
.build();
Vertex v1 = Vertex.create("v1",
ProcessorDescriptor.create(OnetoOne.V1Processor.class.getName()))
.addDataSource(OnetoOne.INPUT, dataSource);
Vertex v2 = Vertex.create("v2",
ProcessorDescriptor.create(OnetoOne.V1Processor.class.getName())).addDataSink(OnetoOne.OUTPUT,
dataSink);
UnorderedKVEdgeConfig config =
UnorderedKVEdgeConfig.newBuilder(Text.class.getName(),
Text.class.getName()).setFromConfiguration(tezConf).build();
Edge e = Edge.create(v1, v2,
config.createDefaultOneToOneEdgeProperty());
<<<<<
Pointers on resolving this exception are much appreciated!
Madhu
