[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-12 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22353 thanks, merging to master/2.4/2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22353 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22353 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95993/ Test PASSed. ---

[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22353 **[Test build #95993 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95993/testReport)** for PR 22353 at commit

[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22353 **[Test build #95993 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95993/testReport)** for PR 22353 at commit

[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-12 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22353 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-11 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/22353 ping @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-11 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/22353 Thank you @cloud-fan for your reminding. We’ve handled the drop message case. Agree, I will update a commit tomorrow. --- -

[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22353 So you need a way to reliably report some extra information like file path in the event logs, but don't want to show it in the UI as it maybe too long. Basically we shouldn't put such

[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-11 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/22353 Spark driver log is always distributed on various client nodes and depends on the log4j configs. In a big company, it's hard to collect them all and I think it's better to used for debug not

[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-11 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/22353 > Although event log is in JSON format, it's mostly for internal usage, to be load by history server and used to build the Spark UI. AFAIK, there are more and more projects replay event log to

[GitHub] spark issue #22353: [SPARK-25357][SQL] Add metadata to SparkPlanInfo to dump...

2018-09-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22353 Although event log is in JSON format, it's mostly for internal usage, to be load by history server and used to build the Spark UI. For compatibility, we only focus on making history to be able to