[jira] [Updated] (SPARK-28843) Set OMP_NUM_THREADS to executor cores reduce Python memory consumption

2019-08-28 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28843: -- Docs Text: Pyspark workers now set the env variable OMP_NUM_THREADS (if not already set) to the

[jira] [Updated] (SPARK-28843) Set OMP_NUM_THREADS to executor cores reduce Python memory consumption

2019-08-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28843: - Labels: release-notes (was: ) > Set OMP_NUM_THREADS to executor cores reduce Python memory

[jira] [Updated] (SPARK-28843) Set OMP_NUM_THREADS to executor cores reduce Python memory consumption

2019-08-21 Thread Ryan Blue (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Blue updated SPARK-28843: -- Description: While testing hardware with more cores, we found that the amount of memory required by