[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-28 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16370 Sorry for the delay. LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-28 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16370 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-27 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/16370 @srowen It looks like @zsxwing has no time to feed back. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-25 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/16370 @zsxwing Is there any farther feedback? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-23 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16370 I don't see a harm in ignoring these files, other comments notwithstanding. In fact, I think it's mandatory. The HDFS copy mechanism basically is copy (to a ._COPYING_ file), then move (rename).

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-22 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/16370 @zsxwing Thanks for your reminder!! In some ways, we really can evade this issue, just like not use `-cp`. But this is an user-side behaviour, we can not ensure every users know and use

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-22 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16370 A similar PR got refused as this is not the correct way to stream files: https://github.com/apache/spark/pull/3203 You should always move files instead of copying them. --- If your

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16370 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16370 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70476/ Test PASSed. ---

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16370 **[Test build #70476 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70476/testReport)** for PR 16370 at commit

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16370 **[Test build #70476 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70476/testReport)** for PR 16370 at commit

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-21 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/16370 unrelated errors, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16370 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16370 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70473/ Test FAILed. ---

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16370 **[Test build #70473 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70473/testReport)** for PR 16370 at commit

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16370 **[Test build #70473 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70473/testReport)** for PR 16370 at commit

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16370 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-21 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/16370 @AmplabJenkins retest it please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16370 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16370 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70456/ Test FAILed. ---

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16370 **[Test build #70456 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70456/testReport)** for PR 16370 at commit

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-20 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/16370 cc @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if