[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2020-06-23 Thread Itamar Turner-Trauring (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17142901#comment-17142901 ] Itamar Turner-Trauring commented on SPARK-23206: Just checking in again,

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2020-05-26 Thread Itamar Turner-Trauring (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17116803#comment-17116803 ] Itamar Turner-Trauring commented on SPARK-23206: It seems like one of th

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-12-11 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16717988#comment-16717988 ] Edwina Lu commented on SPARK-23206: --- [~irashid], [~rezasafi], [~wypoon], as discussed,

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-10-26 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16665628#comment-16665628 ] Edwina Lu commented on SPARK-23206: --- [~irashid], yes, I am planning to work on the oth

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-10-26 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16665594#comment-16665594 ] Imran Rashid commented on SPARK-23206: -- Hi [~elu], just wondering are you still pla

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-07-22 Thread James (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552087#comment-16552087 ] James commented on SPARK-23206: --- Hi  [~elu] If I want to know the CPU metrics of executor

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-06-03 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499528#comment-16499528 ] Felix Cheung commented on SPARK-23206: -- [~irashid] sorry I thought I had replied -

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-05-23 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16487901#comment-16487901 ] Imran Rashid commented on SPARK-23206: -- [~felixcheung] can you give an example of th

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-05-11 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16472405#comment-16472405 ] Edwina Lu commented on SPARK-23206: --- The design discussion for SPARK-23206 is scheduled

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-05-10 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470789#comment-16470789 ] Edwina Lu commented on SPARK-23206: --- [~irashid], I do not have the rest of the changes

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-05-10 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470777#comment-16470777 ] Felix Cheung commented on SPARK-23206: -- yes, for use network and disk IO stats. We h

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-05-09 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16469335#comment-16469335 ] Imran Rashid commented on SPARK-23206: -- Hi, I think getting together to discuss the

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-20 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446247#comment-16446247 ] Edwina Lu commented on SPARK-23206: --- The design discussion is scheduled for Monday 4/23

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-18 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16443025#comment-16443025 ] Edwina Lu commented on SPARK-23206: --- We are planning a design discussion for early next

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-18 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16442928#comment-16442928 ] Edwina Lu commented on SPARK-23206: --- [~assia6], there is an initial PR: [[Github] Pull

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-18 Thread assia ydroudj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16442379#comment-16442379 ] assia ydroudj commented on SPARK-23206: --- [~elu], thank you for the shared dos, it w

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-17 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441006#comment-16441006 ] Edwina Lu commented on SPARK-23206: --- [~assia6], could you please try the new link, [ht

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-16 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16440334#comment-16440334 ] Imran Rashid commented on SPARK-23206: -- thanks, shared doc works for me now! > Addi

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-16 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16440326#comment-16440326 ] Edwina Lu commented on SPARK-23206: --- [~smilegator], please try: https://docs.google.co

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438983#comment-16438983 ] Xiao Li commented on SPARK-23206: - No permission to access the doc  [https://docs.google

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-15 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438879#comment-16438879 ] Edwina Lu commented on SPARK-23206: --- After discussion with [~irashid] on the PR, we've

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-09 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430922#comment-16430922 ] Edwina Lu commented on SPARK-23206: --- The doc wasn't very clear what the quantile values

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-09 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430836#comment-16430836 ] Imran Rashid commented on SPARK-23206: -- ah of course, that makes sense -- quantiles

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-09 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430765#comment-16430765 ] Edwina Lu commented on SPARK-23206: --- [~irashid], this would be quantile values for peak

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430763#comment-16430763 ] Xiao Li commented on SPARK-23206: - cc [~jiangxb1987] [~Gengliang.Wang]  > Additional Mem

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-09 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430616#comment-16430616 ] Imran Rashid commented on SPARK-23206: -- I have a question about this part of the des

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-03-07 Thread assia ydroudj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389644#comment-16389644 ] assia ydroudj commented on SPARK-23206: --- [~elu] , thanks I ll be for wait! I hav

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-03-05 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387083#comment-16387083 ] Edwina Lu commented on SPARK-23206: --- [~assia6], sorry for the delay. I'm planning to su

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-03-04 Thread assia ydroudj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16385044#comment-16385044 ] assia ydroudj commented on SPARK-23206: --- Edwina Lu, Thank you I use actually Spark

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-02-27 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16378879#comment-16378879 ] Edwina Lu commented on SPARK-23206: --- [~assia6], I haven't submitted the PR yet – the co

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-02-26 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377913#comment-16377913 ] Felix Cheung commented on SPARK-23206: -- [~elu] Hi Edwina, we are interesting in this

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-02-26 Thread assia ydroudj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16376655#comment-16376655 ] assia ydroudj commented on SPARK-23206: --- Hi. Where can I find the code of this PR t

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-02-14 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16364979#comment-16364979 ] Edwina Lu commented on SPARK-23206: --- After discussion with [~cltlfcjin] and the eBay Ha

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-02-09 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358866#comment-16358866 ] Edwina Lu commented on SPARK-23206: --- [~irashid], total memory and off heap memory is al

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-02-08 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357953#comment-16357953 ] Imran Rashid commented on SPARK-23206: -- +1 on all the ideas discussed here so far.

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-02-08 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357846#comment-16357846 ] Edwina Lu commented on SPARK-23206: --- [~cltlfcjin], thanks for uploading the screenshot

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-02-08 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357844#comment-16357844 ] Lantao Jin commented on SPARK-23206: Yes, we have done the similar things: !Screen S

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-02-08 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357841#comment-16357841 ] Edwina Lu commented on SPARK-23206: --- [~jerryshao], thanks for your help and advice. [~c

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-02-08 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357824#comment-16357824 ] Saisai Shao commented on SPARK-23206: - [~cltlfcjin] from Ebay also plans to do simila

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-02-07 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355994#comment-16355994 ] Edwina Lu commented on SPARK-23206: --- We ([~jerryshao], [~zhz] and I) are planning a con

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-01-26 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16341526#comment-16341526 ] Edwina Lu commented on SPARK-23206: --- We'd like to monitor the following executor level

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-01-25 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340536#comment-16340536 ] Saisai Shao commented on SPARK-23206: - Would you please summarize what kind of metric

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-01-25 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340474#comment-16340474 ] Edwina Lu commented on SPARK-23206: --- [~jerryshao], yes  SPARK-9103 is similar, and this

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-01-24 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338559#comment-16338559 ] Edwina Lu commented on SPARK-23206: --- Thanks, [~zsxwing]. Making the new metrics availab

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-01-24 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338553#comment-16338553 ] Saisai Shao commented on SPARK-23206: - I think this Jira duplicates SPARK-9103. Also

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-01-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338427#comment-16338427 ] Shixiong Zhu commented on SPARK-23206: -- We can also just add more information to met

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-01-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338421#comment-16338421 ] Shixiong Zhu commented on SPARK-23206: -- Also cc [~jerryshao] since you were working

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-01-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338396#comment-16338396 ] Shixiong Zhu commented on SPARK-23206: -- cc [~vanzin]  > Additional Memory Tuning Me

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-01-24 Thread Ye Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338378#comment-16338378 ] Ye Zhou commented on SPARK-23206: - [~zsxwing] Hi, Can you help find some one who can help