Github user zhichao-li commented on a diff in the pull request:
https://github.com/apache/spark/pull/6404#discussion_r31097361
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/ScriptTransformation.scala
---
@@ -58,11 +58,12 @@ case class ScriptTransformation(
child.execute().mapPartitions { iter =>
val cmd = List("/bin/bash", "-c", script)
val builder = new ProcessBuilder(cmd)
+ builder.redirectErrorStream(true)
--- End diff --
It's indeed not a good idea to mix the error and output together. Just find
a friendly API which can easily redirect the error destination of
ProcessBuilder to be the same as the current process which normally should be
stderr.
The idea is the buffer can be consuming immediately to the stderr to
prevent the hanging on the input side.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
