[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2019-02-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778660#comment-16778660 ] Xiangrui Meng commented on SPARK-24615: --- Attached the PDF files. > Accelerator-aware task

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2019-02-26 Thread Xingbo Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778026#comment-16778026 ] Xingbo Jiang commented on SPARK-24615: -- I updated the SPIP and Product docs, please review and

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2019-01-24 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750872#comment-16750872 ] Saisai Shao commented on SPARK-24615: - I'm really sorry about the delay. Due to some changes on my

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2019-01-23 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750751#comment-16750751 ] Felix Cheung commented on SPARK-24615: -- We are interested to know as well.   [~mengxr] touched on

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2019-01-23 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750375#comment-16750375 ] Thomas Graves commented on SPARK-24615: --- [~jerryshao]  just curious where this is at, are you

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-08-02 Thread Mingjie Tang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567393#comment-16567393 ] Mingjie Tang commented on SPARK-24615: -- >From user's perspective, user only concern about the GPU

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-31 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564562#comment-16564562 ] Saisai Shao commented on SPARK-24615: - Leveraging dynamic allocation to tear down and bring up

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-31 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564558#comment-16564558 ] Erik Erlandson commented on SPARK-24615: Am I understanding correctly that this can't assign

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-31 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563730#comment-16563730 ] Saisai Shao commented on SPARK-24615: - Hi [~tgraves], I think eval() might unnecessarily break the

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-31 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563711#comment-16563711 ] Thomas Graves commented on SPARK-24615: --- so I guess my question is this the right approach at all. 

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-24 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555080#comment-16555080 ] Saisai Shao commented on SPARK-24615: - Hi [~tgraves] [~irashid] thanks a lot for your comments.

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-24 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554742#comment-16554742 ] Imran Rashid commented on SPARK-24615: -- hi, just catching up here -- agree with a lot of Tom's

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-24 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554327#comment-16554327 ] Thomas Graves commented on SPARK-24615: --- Right so I think part of this is trying to make it more

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-23 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553679#comment-16553679 ] Saisai Shao commented on SPARK-24615: - [~Tagar] yes! > Accelerator-aware task scheduling for Spark

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-23 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553121#comment-16553121 ] Ruslan Dautkhanov commented on SPARK-24615: --- Is this work related to Spark's project

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-22 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552233#comment-16552233 ] Saisai Shao commented on SPARK-24615: - Hi [~tgraves] what you mentioned above is also what we think

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-20 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551240#comment-16551240 ] Thomas Graves commented on SPARK-24615: --- the other thing which I think I mentioned above is could

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-20 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551233#comment-16551233 ] Thomas Graves commented on SPARK-24615: --- Ok so thinking about this a bit more I slightly misread

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-20 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551198#comment-16551198 ] Mridul Muralidharan commented on SPARK-24615: - [~tgraves] This was indeed a recurring issue

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-20 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550803#comment-16550803 ] Thomas Graves commented on SPARK-24615: --- yes if any requirement can't be satisfied it would use

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550266#comment-16550266 ] Saisai Shao commented on SPARK-24615: - Hi [~tgraves] I'm still not sure how to handle memory per

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550023#comment-16550023 ] Xiangrui Meng commented on SPARK-24615: --- [~tgraves] Could you help link some past requests on

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-19 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549345#comment-16549345 ] Thomas Graves commented on SPARK-24615: --- but my point is exactly that, it shouldn't be yet another

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549276#comment-16549276 ] Saisai Shao commented on SPARK-24615: - Thanks [~tgraves] for the suggestion.  {quote}Once I get to

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-19 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549266#comment-16549266 ] Thomas Graves commented on SPARK-24615: --- I think the usage for cpu/memory is the same.  You know

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548662#comment-16548662 ] Saisai Shao commented on SPARK-24615: - Hi [~tgraves] I'm rewriting the design doc based on the

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-18 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547903#comment-16547903 ] Thomas Graves commented on SPARK-24615: --- did the design doc permissions change? I can't seem to

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-17 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547263#comment-16547263 ] Saisai Shao commented on SPARK-24615: - Sure, I will also add it as Xiangrui also suggested the same

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-17 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546617#comment-16546617 ] Thomas Graves commented on SPARK-24615: --- ok, I agree, I think this SPIP should at least cover how

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-17 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546590#comment-16546590 ] Saisai Shao commented on SPARK-24615: - Yes, currently the user is responsible for asking yarn to get

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-17 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546580#comment-16546580 ] Thomas Graves commented on SPARK-24615: --- The user is responsible for asking yarn for the right

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-16 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545962#comment-16545962 ] Saisai Shao commented on SPARK-24615: - Sorry [~tgraves] for the late response. Yes,  when requesting

[jira] [Commented] (SPARK-24615) Accelerator aware task scheduling for Spark

2018-07-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545253#comment-16545253 ] Thomas Graves commented on SPARK-24615: --- [~jerryshao] ^ > Accelerator aware task scheduling for

[jira] [Commented] (SPARK-24615) Accelerator aware task scheduling for Spark

2018-06-29 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528293#comment-16528293 ] Thomas Graves commented on SPARK-24615: --- maybe I"m missing it but how is this working with the