[
https://issues.apache.org/jira/browse/SPARK-28302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-28302:
-------------------------
Attachment: Main.scala
> SparkLauncher: The process cannot access the file because it is being used by
> another process
> ---------------------------------------------------------------------------------------------
>
> Key: SPARK-28302
> URL: https://issues.apache.org/jira/browse/SPARK-28302
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.4.0
> Environment: Windows 7
> Reporter: wuyi
> Priority: Major
> Attachments: Main.scala
>
>
> When using SparkLauncher to submit applications concurrently with a thread
> pool under *Windows*, some apps would show that "The process cannot access
> the file because it is being used by another process" and remains in LOST
> state at the end.
>
> After digging into the code, I find that, Windows cmd %RANDOM% would return
> the same number if we call it instantly( < 500ms) after last call. As a
> result, SparkLauncher would get same output
> file(spark-class-launcher-output-%RANDOM%.txt) for apps. Then, the following
> app would hit the issue when it try to write the same file which has already
> been occupied by another app.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
