nsivabalan commented on code in PR #20452:
URL: https://github.com/apache/hudi/pull/20452#discussion_r4405490630
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieStreamingSink.scala:
##########
@@ -206,13 +206,9 @@ class HoodieStreamingSink(sqlContext: SQLContext,
match {
case Failure(e) =>
if (!ignoreFailedBatch) {
- log.error(s"Micro batch id=$batchId threw following expections," +
- s"aborting streaming app to avoid data loss: ", e)
- // spark sometimes hangs upon exceptions and keep on hold of the
executors
- // this is to force exit upon errors / exceptions and release all
executors
- // will require redeployment / supervise mode to restart the
streaming
- reset(true)
- System.exit(1)
+ // retry has already reset this sink's resources. Let Spark fail the
query and leave
+ // restart decisions to the caller without terminating other queries
in the same JVM.
+ throw new HoodieException(s"Micro batch id=$batchId failed after
exhausting write retries", e)
Review Comment:
Non-blocking: this is a behavior change for deployments that rely on the
process exiting. An app that calls `awaitTermination()` still exits non-zero,
since the `StreamingQueryException` propagates out of `main`. But an app that
starts the query and keeps the driver alive some other way (or uses YARN/K8s
supervise and expects the JVM to die) will now keep running with a failed
query. Could we mark the title as breaking (`fix(spark)!:`) or add a
release-note entry so this is visible on upgrade?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
