[ https://issues.apache.org/jira/browse/SYSTEMML-1127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Felix Schüler updated SYSTEMML-1127: ------------------------------------ Summary: Synchronize creation of buffer pool on remote workers (was: Distributed unique IDs are not unique) > Synchronize creation of buffer pool on remote workers > ----------------------------------------------------- > > Key: SYSTEMML-1127 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1127 > Project: SystemML > Issue Type: Bug > Components: ParFor > Reporter: Felix Schüler > > When executing a Spark parfor, the SparkParforWorker throws an exception > which states that the localtmpdir could not be created. This is due to the > fact that multiple executors are running multithreaded on the same worker. > The createDistributedUniqueID() method in the IDHander.java creates unique > IDs only per pid and host, not per thread. This could potentially be solved > by adding the threadID to the unique ID. The question is if every thread > should have its own cache or if the logic should be changed so that the first > creation will be successful and then the threads share one cache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)