[
https://issues.apache.org/jira/browse/SPARK-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reynold Xin updated SPARK-7075:
-------------------------------
Description:
Based on our observation, majority of Spark workloads are not bottlenecked by
I/O or network, but rather CPU and memory. This project aims to
1. Much more efficient memory usage & robustness
2. Much more efficient execution
We will start with the DataFrame API, which gives us more application
semantics, and eventually improve core as well.
was:
I will fill more details in later, but the focus is ...
1. Much more efficient memory usage & robustness
2. Much more efficient execution
We will start with the DataFrame API, which gives us more application
semantics, and eventually improve core as well.
> Improving Physical Execution and Memory Management
> --------------------------------------------------
>
> Key: SPARK-7075
> URL: https://issues.apache.org/jira/browse/SPARK-7075
> Project: Spark
> Issue Type: Epic
> Components: Block Manager, Shuffle, Spark Core, SQL
> Reporter: Reynold Xin
> Assignee: Reynold Xin
>
> Based on our observation, majority of Spark workloads are not bottlenecked by
> I/O or network, but rather CPU and memory. This project aims to
> 1. Much more efficient memory usage & robustness
> 2. Much more efficient execution
> We will start with the DataFrame API, which gives us more application
> semantics, and eventually improve core as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
