Yadong Qi created CARBONDATA-998: ------------------------------------ Summary: Don't request executors when we use carbon distribution Key: CARBONDATA-998 URL: https://issues.apache.org/jira/browse/CARBONDATA-998 Project: CarbonData Issue Type: Improvement Components: spark-integration Affects Versions: 1.1.0 Reporter: Yadong Qi Assignee: Yadong Qi
In the current implementation, carbon request executors if it need more executors. This brings the following questions: 1. Carbon interferes with spark dispatch to make request/remove executors confusedly. 2. Carbon will involk DistributionUtil.getDistinctNodesList and sleep some times to make sure the executors are assigned, it wastes a lot of time. We'd better make the data scan and schedule independently. -- This message was sent by Atlassian JIRA (v6.3.15#6346)