[ https://issues.apache.org/jira/browse/SPARK-28302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
wuyi updated SPARK-28302: ------------------------- Attachment: Main.scala > SparkLauncher: The process cannot access the file because it is being used by > another process > --------------------------------------------------------------------------------------------- > > Key: SPARK-28302 > URL: https://issues.apache.org/jira/browse/SPARK-28302 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 2.4.0 > Environment: Windows 7 > Reporter: wuyi > Priority: Major > Attachments: Main.scala > > > When using SparkLauncher to submit applications concurrently with a thread > pool under *Windows*, some apps would show that "The process cannot access > the file because it is being used by another process" and remains in LOST > state at the end. > > After digging into the code, I find that, Windows cmd %RANDOM% would return > the same number if we call it instantly( < 500ms) after last call. As a > result, SparkLauncher would get same output > file(spark-class-launcher-output-%RANDOM%.txt) for apps. Then, the following > app would hit the issue when it try to write the same file which has already > been occupied by another app. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org