Harsh J created HADOOP-9861:
-------------------------------
Summary: Invert ReflectionUtils' stack trace
Key: HADOOP-9861
URL: https://issues.apache.org/jira/browse/HADOOP-9861
Project: Hadoop Common
Issue Type: Improvement
Components: util
Affects Versions: 2.0.5-alpha
Reporter: Harsh J
Often an MR task (as an example) may fail at the configure stage due to a
misconfiguration or whatever, and the only thing a user gets by virtue of MR
pulling limited bytes of the diagnostic error data is the top part of the
stacktrace:
{code}
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
{code}
This is absolutely useless to a user, and he also goes ahead and blames the
framework for having an issue, rather than thinking (non-intuitively) to go see
the whole task log for the full trace, especially the last part.
Hundreds of time its been a mere class thats missing, etc. but there's just too
much pain involved here to troubleshoot.
Would be much much better, if we inverted the trace. For example, here's what
Hive can return back if we did so, for a random trouble I pulled from the web:
{code}
java.lang.RuntimeException: Error in configuring object
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector.toString(StructObjectInspector.java:64)
at java.lang.String.valueOf(String.java:2826)
at java.lang.StringBuilder.append(StringBuilder.java:115)
at
org.apache.hadoop.hive.ql.exec.UnionOperator.initializeOp(UnionOperator.java:110)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:186)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at
org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:563)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:100)
... 22 more
{code}
This way the user can at least be sure what part's really failing, and not get
lost trying to work their way through reflection utils and upwards/downwards.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
