[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2021-12-10 Thread BoYang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17457384#comment-17457384 ] BoYang commented on SPARK-25299: I am working on a prototype to store shuffle file on external storage

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2021-05-17 Thread jiafu zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346527#comment-17346527 ] jiafu zhang commented on SPARK-25299: - Does anyone know why method

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2020-10-03 Thread BoYang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17207093#comment-17207093 ] BoYang commented on SPARK-25299: While people work on remote storage for persisting shuffle data, and

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2020-08-10 Thread Tianchen Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175117#comment-17175117 ] Tianchen Zhang commented on SPARK-25299: Hi [~mcheah], by following the history of this remote

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2019-09-06 Thread jay vyas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924474#comment-16924474 ] jay vyas commented on SPARK-25299: -- I think architecturally this is a huge step forward , in that it 

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2019-07-01 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876070#comment-16876070 ] Saisai Shao commented on SPARK-25299: - Better to post a pdf version [~mcheah] :). > Use remote

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2019-06-28 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16875106#comment-16875106 ] Matt Cheah commented on SPARK-25299: I also noticed the SPIP document wasn't ever posted on this

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2019-06-28 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16875101#comment-16875101 ] Matt Cheah commented on SPARK-25299: Let's start by making sub-issues. I have a patch staged for

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2019-06-28 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874703#comment-16874703 ] Saisai Shao commented on SPARK-25299: - Votes were passed, so what is our plan for code submission?

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2019-04-22 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823613#comment-16823613 ] zhoukang commented on SPARK-25299: -- nice work! Really looking forward thanks [~yifeih] > Use remote

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2019-04-22 Thread Yifei Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823317#comment-16823317 ] Yifei Huang commented on SPARK-25299: - You can follow the API refactor work here: 

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2019-04-22 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822997#comment-16822997 ] zhoukang commented on SPARK-25299: -- is there any progress of this task? [~yifeih] [~mcheah] > Use

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2019-03-11 Thread Hu Ziqian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789189#comment-16789189 ] Hu Ziqian commented on SPARK-25299: --- Hi [~yifeih], your google doc posted at 25/Feb/19 is mainly

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2019-02-25 Thread Yifei Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16777231#comment-16777231 ] Yifei Huang commented on SPARK-25299: - Here is an updated doc with the progress since the last

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2019-02-12 Thread Nicolas Laduguie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16766029#comment-16766029 ] Nicolas Laduguie commented on SPARK-25299: -- Hello everybody !  Could you please inform us

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2019-01-15 Thread Peiyu Zhuang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743647#comment-16743647 ] Peiyu Zhuang commented on SPARK-25299: -- Our project is now open source under Apache 2. Here is the

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2018-12-27 Thread Peiyu Zhuang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16729474#comment-16729474 ] Peiyu Zhuang commented on SPARK-25299: -- Sure.  I just create a [SPIP in google

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2018-12-26 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16729357#comment-16729357 ] Saisai Shao commented on SPARK-25299: - [~jealous] Can we have a doc about this proposed solution for

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2018-12-26 Thread Carson Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16729294#comment-16729294 ] Carson Wang commented on SPARK-25299: - I am on a vacation and will be back on January 2, 2019.

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2018-12-26 Thread Peiyu Zhuang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16729292#comment-16729292 ] Peiyu Zhuang commented on SPARK-25299: -- We are currently working on a solution that is similar to

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657543#comment-16657543 ] Apache Spark commented on SPARK-25299: -- User 'ifilonenko' has created a pull request for this

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657539#comment-16657539 ] Apache Spark commented on SPARK-25299: -- User 'ifilonenko' has created a pull request for this

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2018-09-04 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603807#comment-16603807 ] Matt Cheah commented on SPARK-25299: (Changed the title to "remote storage" for a little more