[jira] [Commented] (SPARK-15690) Fast single-node (single-process) in-memory shuffle

2018-09-11 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610715#comment-16610715 ] Wenchen Fan commented on SPARK-15690: - I'm removing the target version, since no one is working on

[jira] [Commented] (SPARK-15690) Fast single-node (single-process) in-memory shuffle

2016-06-15 Thread Joseph Fourny (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332960#comment-15332960 ] Joseph Fourny commented on SPARK-15690: --- +1 on this. I am trying to develop single-node clusters on

[jira] [Commented] (SPARK-15690) Fast single-node (single-process) in-memory shuffle

2016-06-13 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328815#comment-15328815 ] Reynold Xin commented on SPARK-15690: - Definitely no serialization/deserialization. > Fast

[jira] [Commented] (SPARK-15690) Fast single-node (single-process) in-memory shuffle

2016-06-13 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328717#comment-15328717 ] Shivaram Venkataraman commented on SPARK-15690: --- Yeah I dont think you'll see much

[jira] [Commented] (SPARK-15690) Fast single-node (single-process) in-memory shuffle

2016-06-13 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328390#comment-15328390 ] Reynold Xin commented on SPARK-15690: - Yes there is definitely no reason to go through network for a

[jira] [Commented] (SPARK-15690) Fast single-node (single-process) in-memory shuffle

2016-06-13 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328378#comment-15328378 ] Saisai Shao commented on SPARK-15690: - I see. Since everything is in a single process, looks like

[jira] [Commented] (SPARK-15690) Fast single-node (single-process) in-memory shuffle

2016-06-13 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328264#comment-15328264 ] Reynold Xin commented on SPARK-15690: - Yup. Eventually we can also generalize this to multiple

[jira] [Commented] (SPARK-15690) Fast single-node (single-process) in-memory shuffle

2016-06-13 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328255#comment-15328255 ] Saisai Shao commented on SPARK-15690: - Hi [~rxin], what's the meaning of "single-process", is that