Github user sameeragarwal commented on a diff in the pull request:
https://github.com/apache/spark/pull/12194#discussion_r64495331
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/ScriptTransformation.scala
---
@@ -127,45 +127,78 @@ case class ScriptTransformation(
}
val mutableRow = new SpecificMutableRow(output.map(_.dataType))
+ private def checkFailureAndPropagate(cause: Throwable = null):
Unit = {
+ if (writerThread.exception.isDefined) {
+ throw writerThread.exception.get
+ }
+
+ // Checks if the proc is still alive (incase the command ran was
bad)
+ // The ideal way to do this is to use Java 8's Process#isAlive()
+ // but it cannot be used because Spark still supports Java 7.
+ // Following is a workaround used to check if a process is alive
in Java 7
+ // TODO: Once builds are switched to Java 8, this can be changed
+ try {
+ val exitCode = proc.exitValue()
+ if (exitCode != 0) {
+ logError(stderrBuffer.toString) // log the stderr circular
buffer
+ throw new SparkException(s"Subprocess exited with status
$exitCode. " +
+ s"Error: ${stderrBuffer.toString}", cause)
--- End diff --
unrelated to this PR but we should probably only print the partial contents
of the circular buffer if the number of bytes written to it are less than its
total size
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
