[jira] [Comment Edited] (SPARK-48397) Add data write time metric to FileFormatDataWriter/BasicWriteJobStatsTracker

[ https://issues.apache.org/jira/browse/SPARK-48397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848824#comment-17848824 ] Eric Yang edited comment on SPARK-48397 at 5/23/24 6:38 AM:

[jira] [Commented] (SPARK-48397) Add data write time metric to FileFormatDataWriter/BasicWriteJobStatsTracker

[ https://issues.apache.org/jira/browse/SPARK-48397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848824#comment-17848824 ] Eric Yang commented on SPARK-48397: --- I'm working on a PR for it. > Add data write tim

[jira] [Created] (SPARK-48397) Add data write time metric to FileFormatDataWriter/BasicWriteJobStatsTracker

Eric Yang created SPARK-48397: - Summary: Add data write time metric to FileFormatDataWriter/BasicWriteJobStatsTracker Key: SPARK-48397 URL: https://issues.apache.org/jira/browse/SPARK-48397 Project: Spark

[jira] [Comment Edited] (SPARK-48298) Add TCP mode to StatsdSink

[ https://issues.apache.org/jira/browse/SPARK-48298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846789#comment-17846789 ] Eric Yang edited comment on SPARK-48298 at 5/16/24 4:48 AM:

[jira] [Updated] (SPARK-48298) Add TCP mode to StatsdSink

[ https://issues.apache.org/jira/browse/SPARK-48298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated SPARK-48298: -- Summary: Add TCP mode to StatsdSink (was: StatsdSink supports TCP mode) > Add TCP mode to StatsdSink

[jira] [Updated] (SPARK-48298) StatsdSink supports TCP mode

[ https://issues.apache.org/jira/browse/SPARK-48298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated SPARK-48298: -- Description: Currently, the StatsdSink in Spark supports UDP mode only, which is the default mode of

[jira] [Created] (SPARK-48298) StatsdSink supports TCP mode

Eric Yang created SPARK-48298: - Summary: StatsdSink supports TCP mode Key: SPARK-48298 URL: https://issues.apache.org/jira/browse/SPARK-48298 Project: Spark Issue Type: New Feature Comp

[jira] [Commented] (SPARK-48298) StatsdSink supports TCP mode

[ https://issues.apache.org/jira/browse/SPARK-48298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846789#comment-17846789 ] Eric Yang commented on SPARK-48298: --- I'm preparing a PR for it. > StatsdSink supports

[jira] [Commented] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

[ https://issues.apache.org/jira/browse/SPARK-47017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844145#comment-17844145 ] Eric Yang commented on SPARK-47017: --- I'm preparing a PR for it.  > Show metrics of th

[jira] [Comment Edited] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

[ https://issues.apache.org/jira/browse/SPARK-47017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817786#comment-17817786 ] Eric Yang edited comment on SPARK-47017 at 2/15/24 9:30 PM:

[jira] [Comment Edited] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

[ https://issues.apache.org/jira/browse/SPARK-47017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817786#comment-17817786 ] Eric Yang edited comment on SPARK-47017 at 2/15/24 9:27 PM:

[jira] [Commented] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

[ https://issues.apache.org/jira/browse/SPARK-47017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817786#comment-17817786 ] Eric Yang commented on SPARK-47017: --- Here is a simple example of this issue (based on

[jira] [Updated] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

[ https://issues.apache.org/jira/browse/SPARK-47017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated SPARK-47017: -- Attachment: eventLogs-local-1708032228180.zip > Show metrics of the physical plan of RDDScanExec's int

[jira] [Updated] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

[ https://issues.apache.org/jira/browse/SPARK-47017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated SPARK-47017: -- Attachment: simple2.scala > Show metrics of the physical plan of RDDScanExec's internal RDD in the >

[jira] [Updated] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

[ https://issues.apache.org/jira/browse/SPARK-47017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated SPARK-47017: -- Attachment: ScanExistingRDD.jpg > Show metrics of the physical plan of RDDScanExec's internal RDD in t

[jira] [Updated] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

[ https://issues.apache.org/jira/browse/SPARK-47017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated SPARK-47017: -- Description: The RDDScanExec wraps an internal RDD (as below). In our environment, we find that this

[jira] [Created] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

Eric Yang created SPARK-47017: - Summary: Show metrics of the physical plan of RDDScanExec's internal RDD in the history server Key: SPARK-47017 URL: https://issues.apache.org/jira/browse/SPARK-47017 Proje

[jira] [Comment Edited] (SPARK-23717) Leverage docker support in Hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-23717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16632100#comment-16632100 ] Eric Yang edited comment on SPARK-23717 at 9/28/18 4:33 PM:

[jira] [Commented] (SPARK-23717) Leverage docker support in Hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-23717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16632100#comment-16632100 ] Eric Yang commented on SPARK-23717: --- It is possible to run standalone Spark in YARN wi

[jira] [Commented] (SPARK-25330) Permission issue after upgrade hadoop version to 2.7.7

[ https://issues.apache.org/jira/browse/SPARK-25330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16606052#comment-16606052 ] Eric Yang commented on SPARK-25330: --- {quote} user.getRealUser(): ad...@kerberos.mycom.

[jira] [Commented] (SPARK-25330) Permission issue after upgrade hadoop version to 2.7.7

[ https://issues.apache.org/jira/browse/SPARK-25330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16605114#comment-16605114 ] Eric Yang commented on SPARK-25330: --- [~yumwang] Does Hadoop 2.7.5 works? It might hel