[jira] [Comment Edited] (SPARK-10940) Too many open files Spark Shuffle
[ https://issues.apache.org/jira/browse/SPARK-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945652#comment-14945652 ] Sandeep Pal edited comment on SPARK-10940 at 10/6/15 7:55 PM: -- Ok, I will. One more important thing I forgot to mention. The above failed configuration of 120 cores worked on spark core code on the top of rdd. The failure is only in case of using Spark SQL. I will also update the description. was (Author: vnayak053): Ok, I will. One more important think I forgot to mention. The above failed configuration of 120 cores worked on spark core code on the top of rdd. The failure is only in case of using Spark SQL. I will also update the description. > Too many open files Spark Shuffle > - > > Key: SPARK-10940 > URL: https://issues.apache.org/jira/browse/SPARK-10940 > Project: Spark > Issue Type: Bug > Components: Shuffle, SQL >Affects Versions: 1.5.0 > Environment: 6 node standalone spark cluster with 1 master and 5 > worker nodes on Centos 6.6 for all nodes. Each node has > 100 GB memory and > 36 cores. >Reporter: Sandeep Pal > > Executing terasort by Spark-SQL on the data generated by teragen in hadoop. > Data size generated is ~456 GB. > Terasort passing with --total-executor-cores = 40, where as failing for > --total-executor-cores = 120. > I have tried to increase the ulimit to 10k but the problem persists. > Note: The above failed configuration of 120 cores worked on spark core code > on the top of rdd. The failure is only in case of using Spark SQL. > Below is the error message from one of the executor node: > java.io.FileNotFoundException: > /tmp/spark-e15993e8-51a4-452a-8b86-da0169445065/executor-0c661152-3837-4711-bba2-2abf4fd15240/blockmgr-973aab72-feb8-4c60-ba3d-1b2ee27a1cc2/3f/temp_shuffle_7741538d-3ccf-4566-869f-265655ca9c90 > (Too many open files) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-10940) Too many open files Spark Shuffle
[ https://issues.apache.org/jira/browse/SPARK-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945652#comment-14945652 ] Sandeep Pal edited comment on SPARK-10940 at 10/6/15 7:54 PM: -- Ok, I will. One more important think I forgot to mention. The above failed configuration of 120 cores worked on spark core code on the top of rdd. The failure is only in case of using Spark SQL. I will also update the description. was (Author: vnayak053): Ok, I will. One more important think I for got to mention. The above failed configuration of 120 cores worked on spark core code on the top of rdd. The failure is only in case of using Spark SQL. I will also update the description. > Too many open files Spark Shuffle > - > > Key: SPARK-10940 > URL: https://issues.apache.org/jira/browse/SPARK-10940 > Project: Spark > Issue Type: Bug > Components: Shuffle, SQL >Affects Versions: 1.5.0 > Environment: 6 node standalone spark cluster with 1 master and 5 > worker nodes on Centos 6.6 for all nodes. Each node has > 100 GB memory and > 36 cores. >Reporter: Sandeep Pal > > Executing terasort by Spark-SQL on the data generated by teragen in hadoop. > Data size generated is ~456 GB. > Terasort passing with --total-executor-cores = 40, where as failing for > --total-executor-cores = 120. > I have tried to increase the ulimit to 10k but the problem persists. > Below is the error message from one of the executor node: > java.io.FileNotFoundException: > /tmp/spark-e15993e8-51a4-452a-8b86-da0169445065/executor-0c661152-3837-4711-bba2-2abf4fd15240/blockmgr-973aab72-feb8-4c60-ba3d-1b2ee27a1cc2/3f/temp_shuffle_7741538d-3ccf-4566-869f-265655ca9c90 > (Too many open files) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-10940) Too many open files Spark Shuffle
[ https://issues.apache.org/jira/browse/SPARK-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945390#comment-14945390 ] Sandeep Pal edited comment on SPARK-10940 at 10/6/15 5:30 PM: -- Yes, I am using the default shuffle manager in spark 1.5 which is sort based. Also, the default ulimit -n is 1024 for which --total-executor-cores=60 (12 cores/executor) is successful. So, if am increasing it to 10k it should pass for 120 cores (24 cores/executor). was (Author: vnayak053): Yes, I am using the default shuffle manager in spark 1.5 which is sort based. > Too many open files Spark Shuffle > - > > Key: SPARK-10940 > URL: https://issues.apache.org/jira/browse/SPARK-10940 > Project: Spark > Issue Type: Bug > Components: Shuffle, SQL >Affects Versions: 1.5.0 > Environment: 6 node standalone spark cluster with 1 master and 5 > worker nodes on Centos 6.6 for all nodes. Each node has > 100 GB memory and > 36 cores. >Reporter: Sandeep Pal > > Executing terasort by Spark-SQL on the data generated by teragen in hadoop. > Data size generated is ~456 GB. > Terasort passing with --total-executor-cores = 40, where as failing for > --total-executor-cores = 120. > I have tried to increase the ulimit to 10k but the problem persists. > Below is the error message from one of the executor node: > java.io.FileNotFoundException: > /tmp/spark-e15993e8-51a4-452a-8b86-da0169445065/executor-0c661152-3837-4711-bba2-2abf4fd15240/blockmgr-973aab72-feb8-4c60-ba3d-1b2ee27a1cc2/3f/temp_shuffle_7741538d-3ccf-4566-869f-265655ca9c90 > (Too many open files) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
