wuyi created SPARK-28302:
----------------------------
Summary: SparkLauncher: The process cannot access the file because
it is being used by another process
Key: SPARK-28302
URL: https://issues.apache.org/jira/browse/SPARK-28302
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.4.0
Environment: Windows 7
Reporter: wuyi
When using SparkLauncher to submit applications concurrently with a thread pool
under *Windows*, some apps would show that "The process cannot access the file
because it is being used by another process" and remains in LOST state at the
end.
After digging into the code, I find that, Windows cmd %RANDOM% would return the
same number if we call it instantly( < 500ms) after last call. As a result,
SparkLauncher would get same output
file(spark-class-launcher-output-%RANDOM%.txt) for apps. Then, the following
app would hit the issue when it try to write the same file which has already
been occupied by another app.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
