[jira] [Updated] (SPARK-542) Cache Miss when machine have multiple hostname
[ https://issues.apache.org/jira/browse/SPARK-542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Farrellee updated SPARK-542: Component/s: Mesos Priority: Blocker Cache Miss when machine have multiple hostname -- Key: SPARK-542 URL: https://issues.apache.org/jira/browse/SPARK-542 Project: Spark Issue Type: Bug Components: Mesos Reporter: frankvictor Priority: Blocker HI, I encountered a weird runtime of pagerank in last few day. After debugging the job, I found it was caused by the DNS name. The machines of my cluster have multiple hostname, for example, slave 1 have name (c001 and c001.cm.cluster) when spark adding cache in cacheTracker, it get c001 and add cache use it. But when schedule task in SimpleJob, the msos offer give spark c001.cm.cluster. so It will never get preferred location! I thinks spark should handle the multiple hostname case(by using ip instead of hostname, or some other methods). Thanks! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-542) Cache Miss when machine have multiple hostname
[ https://issues.apache.org/jira/browse/SPARK-542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Farrellee updated SPARK-542: Priority: Minor (was: Blocker) Cache Miss when machine have multiple hostname -- Key: SPARK-542 URL: https://issues.apache.org/jira/browse/SPARK-542 Project: Spark Issue Type: Bug Components: Mesos Reporter: frankvictor Priority: Minor HI, I encountered a weird runtime of pagerank in last few day. After debugging the job, I found it was caused by the DNS name. The machines of my cluster have multiple hostname, for example, slave 1 have name (c001 and c001.cm.cluster) when spark adding cache in cacheTracker, it get c001 and add cache use it. But when schedule task in SimpleJob, the msos offer give spark c001.cm.cluster. so It will never get preferred location! I thinks spark should handle the multiple hostname case(by using ip instead of hostname, or some other methods). Thanks! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
