[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-28 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16987 LGTM. Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16987 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73653/ Test PASSed. ---

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16987 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16987 **[Test build #73653 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73653/testReport)** for PR 16987 at commit

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16987 **[Test build #73653 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73653/testReport)** for PR 16987 at commit

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16987 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16987 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73578/ Test PASSed. ---

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16987 **[Test build #73578 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73578/testReport)** for PR 16987 at commit

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16987 **[Test build #73578 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73578/testReport)** for PR 16987 at commit

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-26 Thread lw-lin
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/16987 Rebased to master and tests updated. @zsxwing would you take another look when you've got a minute? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16987 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73492/ Test PASSed. ---

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16987 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16987 **[Test build #73492 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73492/testReport)** for PR 16987 at commit

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16987 **[Test build #73492 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73492/testReport)** for PR 16987 at commit

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16987 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73382/ Test PASSed. ---

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16987 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16987 **[Test build #73382 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73382/testReport)** for PR 16987 at commit

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16987 **[Test build #73382 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73382/testReport)** for PR 16987 at commit

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-23 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16987 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16987 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16987 **[Test build #73374 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73374/testReport)** for PR 16987 at commit

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16987 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73374/ Test FAILed. ---

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16987 **[Test build #73374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73374/testReport)** for PR 16987 at commit

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-23 Thread lw-lin
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/16987 Reopening :-) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-23 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/16987 I spoke too soon, sorry! Thinking about it more the deterministic filename solution is not great as the number of partitions could change for several reasons. Given that would you mind

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-22 Thread lw-lin
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/16987 Using deterministic file names sounds great. Thanks! I'm closing this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-21 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/16987 Thanks for working on this, however I'm not sure if we want to go with this approach. In Spark 2.2, I think we should consider deprecating the manifest files and instead use deterministic file

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-20 Thread lw-lin
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/16987 @marmbrus @zsxwing would you take a look at this? thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not