wuyi created SPARK-28302:
----------------------------

             Summary: SparkLauncher: The process cannot access the file because 
it is being used by another process
                 Key: SPARK-28302
                 URL: https://issues.apache.org/jira/browse/SPARK-28302
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 2.4.0
         Environment: Windows 7
            Reporter: wuyi


When using SparkLauncher to submit applications concurrently with a thread pool 
under *Windows*, some apps would show that "The process cannot access the file 
because it is being used by another process" and remains in LOST state at the 
end.

 

After digging into the code, I find that, Windows cmd %RANDOM% would return the 
same number if we call it  instantly( < 500ms) after last call. As a result, 
SparkLauncher would get same output 
file(spark-class-launcher-output-%RANDOM%.txt) for apps. Then, the following 
app would hit the issue when it try to write the same file which has already 
been occupied by another app.

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to