liupengcheng created SPARK-26529:
------------------------------------
Summary: Add logs for IOException when preparing local resource
Key: SPARK-26529
URL: https://issues.apache.org/jira/browse/SPARK-26529
Project: Spark
Issue Type: Improvement
Components: Deploy, Spark Core
Affects Versions: 2.4.0, 2.3.2
Reporter: liupengcheng
Currently, `Client#createConfArchive` do not handle IOException, and some
detail info is not provided in logs. Sometimes, this may delay the time of
locating the root cause of io error.
A case happened in our production environment is that local disk is full, and
the following exception is thrown but no detail path info provided. we have to
investigate all the local disk of the machine to find out the root cause.
{code:java}
Exception in thread "main" java.io.IOException: No space left on device
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:345)
at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:253)
at java.util.zip.ZipOutputStream.closeEntry(ZipOutputStream.java:238)
at java.util.zip.ZipOutputStream.finish(ZipOutputStream.java:343)
at java.util.zip.DeflaterOutputStream.close(DeflaterOutputStream.java:238)
at java.util.zip.ZipOutputStream.close(ZipOutputStream.java:360)
at org.apache.spark.deploy.yarn.Client.createConfArchive(Client.scala:769)
at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:657)
at
org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:895)
at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:177)
at org.apache.spark.deploy.yarn.Client.run(Client.scala:1202)
at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1261)
at org.apache.spark.deploy.yarn.Client.main(Client.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:767)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:189)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:214)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:128)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
{code}
It make sense for us to catch the IOException and print some useful information.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
