[jira] [Updated] (SPARK-34898) Send ExecutorMetricsUpdate EventLog appropriately

2021-04-06 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34898: --- Description: In current EventLoggingListener, we won't write SparkListenerExecutorMetricsUpdate message at

[jira] [Issue Comment Deleted] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-03-28 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Comment: was deleted (was: Hi [~Baohe Zhang] , This ticket proposes a new REST API: 

[jira] [Updated] (SPARK-34092) support filtering by task status in REST API call for a specific stage

2021-03-26 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34092: --- Description: When we want to get a specific stage's detail info with task information, it will return all

[jira] [Commented] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage

2021-03-24 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308305#comment-17308305 ] Ron Hu commented on SPARK-34488: Hi [~srowen], Thanks for merging this PR.  [~angerszhuuu] and I worked

[jira] [Updated] (SPARK-34092) support filtering by task status in REST API call for a specific stage

2021-03-09 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34092: --- Description: When we want to get a specific stage's detail info with task information, it will return all

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage

2021-03-01 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution.  This

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage

2021-02-28 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution.  This

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage

2021-02-28 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution.  This

[jira] [Commented] (SPARK-34092) support filtering by task status in REST API call for a specific stage

2021-02-25 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17291067#comment-17291067 ] Ron Hu commented on SPARK-34092: We clarified the need for query parameter taskStatus for a specific

[jira] [Updated] (SPARK-34092) support filtering by task status in REST API call for a specific stage

2021-02-25 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34092: --- Description: When we want to get a specific stage's detail info with task information, it will return all

[jira] [Commented] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage

2021-02-23 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17289646#comment-17289646 ] Ron Hu commented on SPARK-34488: It should be noted that this Jira  addresses query parameter

[jira] [Commented] (SPARK-34092) support filtering by task status in REST API call for a specific stage

2021-02-23 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17289644#comment-17289644 ] Ron Hu commented on SPARK-34092: It should be noted that this Jira  addresses query parameter taskStatus

[jira] [Updated] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-21 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: [~angerszhuuu] and [~ron8hu] discussed a generic and consistent way for overall stages, i.e.

[jira] [Updated] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-21 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: [~angerszhuuu] and [~ron8hu] discussed a generic and consistent way for overall stages, i.e.

[jira] [Updated] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-21 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: [~angerszhuuu] and [~ron8hu] discussed a generic and consistent way for endpoint

[jira] [Commented] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-21 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17288026#comment-17288026 ] Ron Hu commented on SPARK-26399: Add an example of task Metrics percentile Distributions. 

[jira] [Updated] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-21 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Attachment: executorMetricsDistributions.json > Define query parameters to support various filtering

[jira] [Updated] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-21 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Attachment: taskMetricsDistributions.json > Define query parameters to support various filtering conditions

[jira] [Updated] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-21 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Attachment: taskMetricsDistributions.json > Define query parameters to support various filtering conditions

[jira] [Updated] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Summary: Define query parameters to support various filtering conditions in REST API for overall stages

[jira] [Updated] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: [~angerszhuuu] and [~ron8hu] discussed a generic and consistent way for endpoint

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Summary: Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution.  This

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution.  This

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution.  This

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Attachment: executorMetricsDistributions.json > Support task Metrics Distributions and executor Metrics

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Attachment: taskMetricsDistributions.json > Support task Metrics Distributions and executor Metrics

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution.  This

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution. This

[jira] [Created] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage

2021-02-20 Thread Ron Hu (Jira)
Ron Hu created SPARK-34488: -- Summary: Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage Key: SPARK-34488 URL:

[jira] [Updated] (SPARK-34092) support filtering by task status in REST API call for a specific stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34092: --- Summary: support filtering by task status in REST API call for a specific stage (was: Stage level RestFul

[jira] [Updated] (SPARK-34092) Stage level RestFul API support filter by task status

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34092: --- Description: Query parameter taskStatus can be used to filter the tasks meeting a specific status in the

[jira] [Updated] (SPARK-34092) Stage level RestFul API support filter by task status

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34092: --- Description: Query parameter taskStatus can be used to filter the tasks meeting a specific status in the

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-02-04 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-22 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261948#comment-17261948 ] Ron Hu edited comment on SPARK-26399 at 1/23/21, 3:39 AM: -- [~angerszhuuu] and

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-22 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-10 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261948#comment-17261948 ] Ron Hu edited comment on SPARK-26399 at 1/10/21, 6:22 PM: -- [~angerszhuuu] and

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-10 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261948#comment-17261948 ] Ron Hu edited comment on SPARK-26399 at 1/10/21, 6:22 PM: -- [~angerszhuuu] and

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-10 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261948#comment-17261948 ] Ron Hu edited comment on SPARK-26399 at 1/10/21, 6:20 PM: -- [~angerszhuuu] and

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-10 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-10 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261948#comment-17261948 ] Ron Hu edited comment on SPARK-26399 at 1/10/21, 6:18 PM: -- [~angerszhuuu] and

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-10 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-09 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261948#comment-17261948 ] Ron Hu edited comment on SPARK-26399 at 1/9/21, 9:25 PM: - [~angerszhuuu] and

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-09 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-09 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-09 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-09 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-09 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261948#comment-17261948 ] Ron Hu commented on SPARK-26399: [~angerszhuuu] and [~ron8hu] discussed a generic and consistent way for

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260856#comment-17260856 ] Ron Hu edited comment on SPARK-26399 at 1/8/21, 5:18 AM: - The initial

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260856#comment-17260856 ] Ron Hu commented on SPARK-26399: The initial description of this jira has this statement:  "filtering

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Attachment: lispark230_restapi_ex2_stages_failedTasks.json > Add new stage-level REST APIs and parameters >

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260854#comment-17260854 ] Ron Hu commented on SPARK-26399: [^lispark230_restapi_ex2_stages_failedTasks.json] > Add new

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260805#comment-17260805 ] Ron Hu edited comment on SPARK-26399 at 1/7/21, 10:23 PM: -- Dr. Elephant

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260805#comment-17260805 ] Ron Hu edited comment on SPARK-26399 at 1/7/21, 8:41 PM: - Dr. Elephant

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260805#comment-17260805 ] Ron Hu edited comment on SPARK-26399 at 1/7/21, 8:40 PM: - Dr. Elephant

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Attachment: lispark230_restapi_ex2_stages_withSummaries.json > Add new stage-level REST APIs and parameters

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260808#comment-17260808 ] Ron Hu commented on SPARK-26399: [^lispark230_restapi_ex2_stages_withSummaries.json] > Add new

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260805#comment-17260805 ] Ron Hu commented on SPARK-26399: Dr. Elephant (https://github.com/linkedin/dr-elephant) is a downstream

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257844#comment-17257844 ] Ron Hu edited comment on SPARK-26399 at 1/3/21, 9:00 PM: - [~angerszhuuu] found

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257844#comment-17257844 ] Ron Hu edited comment on SPARK-26399 at 1/3/21, 7:46 PM: - [~angerszhuuu] found

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257844#comment-17257844 ] Ron Hu edited comment on SPARK-26399 at 1/3/21, 7:45 PM: - [~angerszhuuu] found

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257844#comment-17257844 ] Ron Hu edited comment on SPARK-26399 at 1/3/21, 7:44 PM: - [~angerszhuuu] found

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Attachment: executorMetricsSummary.json > Add new stage-level REST APIs and parameters >

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257846#comment-17257846 ] Ron Hu commented on SPARK-26399: [^executorMetricsSummary.json] > Add new stage-level REST APIs and

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Attachment: stage_executorSummary_image1.png > Add new stage-level REST APIs and parameters >

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257845#comment-17257845 ] Ron Hu commented on SPARK-26399: !stage_executorSummary_image1.png! > Add new stage-level REST APIs and

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257844#comment-17257844 ] Ron Hu commented on SPARK-26399: [~angerszhuuu] found that the "executorSummary" field already exists in

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-02 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257639#comment-17257639 ] Ron Hu commented on SPARK-26399: Hi [~Baohe Zhang] , This ticket proposes a new REST API: 

[jira] [Updated] (SPARK-26341) Expose executor memory metrics at the stage level, in the Stages tab

2020-12-02 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26341: --- Affects Version/s: (was: 3.1.0) 3.0.1 > Expose executor memory metrics at the

[jira] [Comment Edited] (SPARK-21322) support histogram in filter cardinality estimation

2017-11-20 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16258601#comment-16258601 ] Ron Hu edited comment on SPARK-21322 at 11/21/17 1:43 AM: -- Pull request 19357

[jira] [Commented] (SPARK-21322) support histogram in filter cardinality estimation

2017-11-19 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16258601#comment-16258601 ] Ron Hu commented on SPARK-21322: Pull request 19357 was created while there were several dependencies

[jira] [Created] (SPARK-22164) support histogram in estimating the cardinality of aggregate (or group-by) operator

2017-09-28 Thread Ron Hu (JIRA)
Ron Hu created SPARK-22164: -- Summary: support histogram in estimating the cardinality of aggregate (or group-by) operator Key: SPARK-22164 URL: https://issues.apache.org/jira/browse/SPARK-22164 Project:

[jira] [Commented] (SPARK-17075) Cardinality Estimation of Predicate Expressions

2017-09-26 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181793#comment-16181793 ] Ron Hu commented on SPARK-17075: This jira has been resolved. Pull request 19357 was added here by

[jira] [Created] (SPARK-21322) support histogram in filter cardinality estimation

2017-07-05 Thread Ron Hu (JIRA)
Ron Hu created SPARK-21322: -- Summary: support histogram in filter cardinality estimation Key: SPARK-21322 URL: https://issues.apache.org/jira/browse/SPARK-21322 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-19408) cardinality estimation involving two columns of the same table

2017-03-31 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-19408: --- Description: In SPARK-17075, we estimate cardinality of predicate expression "column (op) literal", where

[jira] [Updated] (SPARK-19408) cardinality estimation involving two columns of the same table

2017-03-24 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-19408: --- Target Version/s: 2.2.0 (was: 2.3.0) > cardinality estimation involving two columns of the same table >

[jira] [Updated] (SPARK-19408) cardinality estimation involving two columns of the same table

2017-03-21 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-19408: --- Target Version/s: 2.3.0 (was: 2.2.0) > cardinality estimation involving two columns of the same table >

[jira] (SPARK-19408) cardinality estimation involving two columns of the same table

2017-01-30 Thread Ron Hu (JIRA)
Title: Message Title Ron Hu created an issue

[jira] [Issue Comment Deleted] (SPARK-17076) Cardinality estimation of join operator

2016-12-08 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-17076: --- Comment: was deleted (was: Hi, I am out of office 11 /15 through 11/18 with very limited Internet access.

[jira] [Commented] (SPARK-16026) Cost-based Optimizer framework

2016-11-30 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15710667#comment-15710667 ] Ron Hu commented on SPARK-16026: Hi Reynold, I previously worked on filter cardinality estimation using

[jira] [Commented] (SPARK-17076) Cardinality estimation of join operator

2016-11-14 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15665664#comment-15665664 ] Ron Hu commented on SPARK-17076: Hi, I am out of office 11 /15 through 11/18 with very limited Internet

[jira] [Issue Comment Deleted] (SPARK-16026) Cost-based Optimizer framework

2016-11-01 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Comment: was deleted (was: Design Specification of Spark Cost-Based Optimization is now posted.) >

[jira] [Commented] (SPARK-17791) Join reordering using star schema detection

2016-10-28 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616590#comment-15616590 ] Ron Hu commented on SPARK-17791: This JIRA is indeed complementary to Cost-Based Optimizer (or CBO)

[jira] [Commented] (SPARK-17626) TPC-DS performance improvements using star-schema heuristics

2016-10-08 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15559044#comment-15559044 ] Ron Hu commented on SPARK-17626: In the CBO design spec we posted in

[jira] [Commented] (SPARK-16026) Cost-based Optimizer framework

2016-09-07 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472000#comment-15472000 ] Ron Hu commented on SPARK-16026: Hi Srinath, Thank you for your comments. Let me answer them one by one.

[jira] [Issue Comment Deleted] (SPARK-16026) Cost-based Optimizer framework

2016-09-07 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Comment: was deleted (was: Hi Srinath, Thank you for your comments. Let me answer them one by one.

[jira] [Commented] (SPARK-16026) Cost-based Optimizer framework

2016-09-07 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15471986#comment-15471986 ] Ron Hu commented on SPARK-16026: Hi Srinath, Thank you for your comments. Let me answer them one by

[jira] [Updated] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Attachment: Spark_CBO_Design_Spec.pdf > Cost-based Optimizer framework > -- > >

[jira] [Updated] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Attachment: (was: Spark_CBO_Design_Spec_v_1_2.pdf) > Cost-based Optimizer framework >

[jira] [Updated] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Attachment: Spark_CBO_Design_Spec_v_1_2.pdf > Cost-based Optimizer framework >

[jira] [Updated] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Attachment: (was: Spark_CBO_Design_Spec_v_1_2.pdf) > Cost-based Optimizer framework >

[jira] [Commented] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15422375#comment-15422375 ] Ron Hu commented on SPARK-16026: Design Specification of Spark Cost-Based Optimization is now posted. >

[jira] [Issue Comment Deleted] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Comment: was deleted (was: I will post a pdf version of the design spec soon.) > Cost-based Optimizer

[jira] [Issue Comment Deleted] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Comment: was deleted (was: Design Specification of Spark Cost-Based Optimization) > Cost-based Optimizer

[jira] [Updated] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Attachment: Spark_CBO_Design_Spec_v_1_2.pdf Design Specification of Spark Cost-Based Optimization >

[jira] [Commented] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15422346#comment-15422346 ] Ron Hu commented on SPARK-16026: I will post a pdf version of the design spec soon. > Cost-based

[jira] [Commented] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15422332#comment-15422332 ] Ron Hu commented on SPARK-16026: Sean, I wrote "Design Specification of Spark Cost-Based Optimization"

[jira] [Closed] (SPARK-16990) Define the data structure to hold the statistics for CBO

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu closed SPARK-16990. -- Resolution: Duplicate > Define the data structure to hold the statistics for CBO >

  1   2   >