aasha commented on a change in pull request #1550:
URL: https://github.com/apache/hive/pull/1550#discussion_r499519404
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/DDLTask.java
##########
@@ -82,8 +89,32 @@ public int execute() {
throw new IllegalArgumentException("Unknown DDL request: " +
ddlDesc.getClass());
}
} catch (Throwable e) {
+ LOG.error("DDLTask failed", e);
+ int errorCode = ErrorMsg.getErrorMsg(e.getMessage()).getErrorCode();
+ try {
+ ReplicationMetricCollector metricCollector = work.getMetricCollector();
+ if (errorCode > 40000) {
+ //in case of replication related task, dumpDirectory should not be
null
+ if(work.dumpDirectory != null) {
+ Path nonRecoverableMarker = new Path(work.dumpDirectory,
ReplAck.NON_RECOVERABLE_MARKER.toString());
+ org.apache.hadoop.hive.ql.parse.repl.dump.Utils.writeStackTrace(e,
nonRecoverableMarker, conf);
+ if(metricCollector != null){
+ metricCollector.reportStageEnd(getName(), Status.FAILED_ADMIN,
nonRecoverableMarker.toString());
+ }
+ }
+ if(metricCollector != null){
Review comment:
this is needed only in replication case
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/DDLTask.java
##########
@@ -82,8 +89,32 @@ public int execute() {
throw new IllegalArgumentException("Unknown DDL request: " +
ddlDesc.getClass());
}
} catch (Throwable e) {
+ LOG.error("DDLTask failed", e);
+ int errorCode = ErrorMsg.getErrorMsg(e.getMessage()).getErrorCode();
+ try {
+ ReplicationMetricCollector metricCollector = work.getMetricCollector();
+ if (errorCode > 40000) {
+ //in case of replication related task, dumpDirectory should not be
null
+ if(work.dumpDirectory != null) {
+ Path nonRecoverableMarker = new Path(work.dumpDirectory,
ReplAck.NON_RECOVERABLE_MARKER.toString());
+ org.apache.hadoop.hive.ql.parse.repl.dump.Utils.writeStackTrace(e,
nonRecoverableMarker, conf);
+ if(metricCollector != null){
+ metricCollector.reportStageEnd(getName(), Status.FAILED_ADMIN,
nonRecoverableMarker.toString());
+ }
+ }
+ if(metricCollector != null){
+ metricCollector.reportStageEnd(getName(), Status.FAILED_ADMIN,
null);
+ }
+ } else {
+ if(metricCollector != null){
+ work.getMetricCollector().reportStageEnd(getName(), Status.FAILED);
Review comment:
use metricCollector directly
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsUpdateTask.java
##########
@@ -355,8 +360,32 @@ public int execute() {
} catch (Exception e) {
setException(e);
LOG.info("Failed to persist stats in metastore", e);
+ int errorCode = ErrorMsg.getErrorMsg(e.getMessage()).getErrorCode();
+ try {
+ ReplicationMetricCollector metricCollector = work.getMetricCollector();
+ if (errorCode > 40000) {
Review comment:
All this code can be part of a util method.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java
##########
@@ -464,14 +468,63 @@ public int execute() {
console.printInfo("\n", StringUtils.stringifyException(he),false);
}
}
-
setException(he);
+ LOG.error("MoveTask failed", he);
+ errorCode = ErrorMsg.getErrorMsg(he.getMessage()).getErrorCode();
+ try {
+ ReplicationMetricCollector metricCollector = work.getMetricCollector();
Review comment:
util method
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/DDLTask.java
##########
@@ -82,8 +89,32 @@ public int execute() {
throw new IllegalArgumentException("Unknown DDL request: " +
ddlDesc.getClass());
}
} catch (Throwable e) {
+ LOG.error("DDLTask failed", e);
Review comment:
print the DDL operation too. DDL task can be called for different
operation
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsUpdateTask.java
##########
@@ -355,8 +360,32 @@ public int execute() {
} catch (Exception e) {
setException(e);
LOG.info("Failed to persist stats in metastore", e);
+ int errorCode = ErrorMsg.getErrorMsg(e.getMessage()).getErrorCode();
+ try {
+ ReplicationMetricCollector metricCollector = work.getMetricCollector();
+ if (errorCode > 40000) {
+ //in case of replication related task, dumpDirectory should not be
null
+ String dumpDirectory = work.getDumpDirectory();
+ if(dumpDirectory != null) {
+ Path nonRecoverableMarker = new Path(dumpDirectory,
ReplAck.NON_RECOVERABLE_MARKER.toString());
+ org.apache.hadoop.hive.ql.parse.repl.dump.Utils.writeStackTrace(e,
nonRecoverableMarker, conf);
+ if(metricCollector != null){
+ metricCollector.reportStageEnd(getName(), Status.FAILED_ADMIN,
nonRecoverableMarker.toString());
+ }
+ }
+ if(metricCollector != null){
+ metricCollector.reportStageEnd(getName(), Status.FAILED_ADMIN,
null);
+ }
+ } else {
+ if(metricCollector != null){
+ work.getMetricCollector().reportStageEnd(getName(), Status.FAILED);
+ }
+ }
+ } catch (SemanticException ex) {
Review comment:
catch all exception so that metric collection failure doesn't lead to
failure of task
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
