ZihanLi58 commented on code in PR #3902:
URL: https://github.com/apache/gobblin/pull/3902#discussion_r1540173038
##########
gobblin-iceberg/src/main/java/org/apache/gobblin/iceberg/publisher/GobblinMCEPublisher.java:
##########
@@ -214,6 +214,7 @@ public static Metrics getMetrics(State state, Path path,
Configuration conf, Nam
return OrcMetrics.fromInputFile(HadoopInputFile.fromPath(path,
conf), MetricsConfig.getDefault(), mapping);
} catch (Exception e) {
//This means the table is not compatible with iceberg, so return a
dummy metric
+ //todo: throw exception here? as it's corrupted file
Review Comment:
I remove the catch exception part as I verified that the only exception is
because of we cannot open the file, which either indicating we have issue on
HDFS or means it's a corrupted file. We should be safe to just throw exception
and fail the task in this case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
