Like this?
val krdd = testrdd.map(x = { try{var key =
val tmp_tocks = x.split(sep1)(0)(key,
x) }catch{ case e: Exception =
println(Exception!! = + e + |||KS1 + x)(null, x)
}})
Thanks
Best Regards
On Thu, Mar 26, 2015 at 7:45 PM, Kevin Conaway ke...@zoomdata.com wrote:
How can we catch exceptions that are thrown from custom RDDs or custom map
functions?
We have a custom RDD that is throwing an exception that we would like to
catch but the exception that is thrown back to the caller is a
*org.apache.spark.SparkException* that does not contain any useful
information about the original exception. The detail message is a string
representation of the original stack trace but its hard to do anything
useful with that.
Below is a small class that exhibits the issue. It uses a map function
instead of a custom RDD but the symptom is the same, the original
*RuntimeException* is lost. I tested this with spark 1.2.1 and 1.3.0
public class SparkErrorExample {
public static void main(String [] args) throws Exception {
SparkConf sparkConf = new
SparkConf().setAppName(SparkExample).setMaster(local[*]);
JavaSparkContext ctx = new JavaSparkContext(sparkConf);
JavaRDDString data = ctx.parallelize(Arrays.asList(1, 2,
3));
try {
data.map(line - {
throw new RuntimeException();
}).count();
} catch (Exception ex) {
System.out.println(Exception class: + ex.getClass());
System.out.println(Exception message: + ex.getMessage());
System.out.println(Exception cause: + ex.getCause());
}
}
}
