[jira] [Updated] (SPARK-9983) Local physical operators for query execution
[ https://issues.apache.org/jira/browse/SPARK-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-9983: Target Version/s: (was: 1.6.0) > Local physical operators for query execution > > > Key: SPARK-9983 > URL: https://issues.apache.org/jira/browse/SPARK-9983 > Project: Spark > Issue Type: Story > Components: SQL >Reporter: Reynold Xin >Assignee: Shixiong Zhu > > In distributed query execution, there are two kinds of operators: > (1) operators that exchange data between different executors or threads: > examples include broadcast, shuffle. > (2) operators that process data in a single thread: examples include project, > filter, group by, etc. > This ticket proposes clearly differentiating them and creating local > operators in Spark. This leads to a lot of benefits: easier to test, easier > to optimize data exchange, better design (single responsibility), and > potentially even having a hyper-optimized single-node version of DataFrame. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-9983) Local physical operators for query execution
[ https://issues.apache.org/jira/browse/SPARK-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9983: --- Description: In distributed query execution, there are two kinds of operators: (1) operators that exchange data between different executors or threads: examples include broadcast, shuffle. (2) operators that process data in a single thread: examples include project, filter, group by, etc. This ticket proposes clearly differentiating them and creating local operators in Spark. This leads to a lot of benefits: easier to test, easier to optimize data exchange, better design (single responsibility), and potentially even having a hyper-optimized single-node version of DataFrame. was: In distributed query execution, there are two kinds of operators: (1) operators that exchange data between different executors or threads: examples include broadcast, shuffle. (2) operators that process data in a single thread: examples include project, filter, group by, etc. This ticket proposes clearly differentiating them and create local operators in Spark. This leads to a lot of benefits: easier to test, easier to optimize data exchange, better design (single responsibility), and potentially even having a hyper-optimized single-node version of DataFrame. > Local physical operators for query execution > > > Key: SPARK-9983 > URL: https://issues.apache.org/jira/browse/SPARK-9983 > Project: Spark > Issue Type: Story > Components: SQL >Reporter: Reynold Xin >Assignee: Shixiong Zhu > > In distributed query execution, there are two kinds of operators: > (1) operators that exchange data between different executors or threads: > examples include broadcast, shuffle. > (2) operators that process data in a single thread: examples include project, > filter, group by, etc. > This ticket proposes clearly differentiating them and creating local > operators in Spark. This leads to a lot of benefits: easier to test, easier > to optimize data exchange, better design (single responsibility), and > potentially even having a hyper-optimized single-node version of DataFrame. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-9983) Local physical operators for query execution
[ https://issues.apache.org/jira/browse/SPARK-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9983: --- Description: In distributed query execution, there are two kinds of operators: (1) operators that exchange data between different executors or threads: examples include broadcast, shuffle. (2) operators that process data in a single thread: examples include project, filter, group by, etc. This ticket proposes clearly differentiating them and create local operators in Spark. This leads to a lot of benefits: easier to test, easier to optimize data exchange, better design (single responsibility), and potentially even having a hyper-optimized single-node version of DataFrame. was: In distributed query execution, there are two kinds of operators: (1) operators that exchange data between different executors or threads: examples include broadcast, shuffle. (2) operators that process data in a single thread: examples include project, filter, group by, etc. This ticket proposes clearly differentiating them and create local operators in Spark. This leads to a lot of benefits: easier to test, easier to optimize data exchange, and better design (single responsibility). > Local physical operators for query execution > > > Key: SPARK-9983 > URL: https://issues.apache.org/jira/browse/SPARK-9983 > Project: Spark > Issue Type: Story > Components: SQL >Reporter: Reynold Xin >Assignee: Shixiong Zhu > > In distributed query execution, there are two kinds of operators: > (1) operators that exchange data between different executors or threads: > examples include broadcast, shuffle. > (2) operators that process data in a single thread: examples include project, > filter, group by, etc. > This ticket proposes clearly differentiating them and create local operators > in Spark. This leads to a lot of benefits: easier to test, easier to optimize > data exchange, better design (single responsibility), and potentially even > having a hyper-optimized single-node version of DataFrame. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
