[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77698/ Test PASSed. ---

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77698 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77698/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77697/ Test PASSed. ---

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77697 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77697/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77698/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77697 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77697/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77678/ Test FAILed. ---

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77678 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77678/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77677/ Test FAILed. ---

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77677 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77677/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77678 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77678/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18159 @adrian-ionescu I've included the suggested metrics per file/partition. The `writingTime` metric is also changed to record actual time on writing data. --- If your project is set up for it, you can

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77677 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77677/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77639/ Test PASSed. ---

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77639 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77639/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-01 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18159 @adrian-ionescu Thanks! That's good points. I'll update this based on your suggestion. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-01 Thread adrian-ionescu
Github user adrian-ionescu commented on the issue: https://github.com/apache/spark/pull/18159 Also, if I understand correctly, for metrics such as "bytes of written files", "number of output rows" you currently show totals. It would be good to include avg/min/max per file and per

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-01 Thread adrian-ionescu
Github user adrian-ionescu commented on the issue: https://github.com/apache/spark/pull/18159 [Here](https://gist.github.com/adrian-ionescu/bc68783e01071419ec483bd8216757e8#file-gistfile1-txt-L20)'s a small test that shows that your `writingTime` metric actually includes the whole

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77639 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77639/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77632/ Test FAILed. ---

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77632 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77632/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-01 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18159 Looks like a large code change, but there are 200+ codes for test cases. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-06-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77632 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77632/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-05-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77616/ Test PASSed. ---

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-05-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-05-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77616 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77616/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-05-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77616 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77616/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-05-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-05-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77592/ Test FAILed. ---

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-05-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77592 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77592/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-05-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77592 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77592/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-05-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77584/ Test PASSed. ---

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-05-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18159 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-05-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77584 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77584/testReport)** for PR 18159 at commit

[GitHub] spark issue #18159: [SPARK-20703][SQL][WIP] Associate metrics with data writ...

2017-05-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18159 **[Test build #77584 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77584/testReport)** for PR 18159 at commit