Github user viirya commented on the issue:
https://github.com/apache/spark/pull/15027
@frreiss Thanks for explanation!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user frreiss commented on the issue:
https://github.com/apache/spark/pull/15027
@viirya to answer your question re deleting vs moving the files: Deleting
is easier to implement, because once the .crc file is deleted, you can be sure
it won't appear again. Moving the checksum
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/15027
Merging in master/branch-2.1.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/15027
Should we just delete the crc file or rename it? As the crc file is created
for the temp file, I suppose the filesystem uses it?
---
If your project is set up for it, you can reply to this email
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15027
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67931/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15027
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15027
**[Test build #67931 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67931/consoleFull)**
for PR 15027 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15027
**[Test build #67931 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67931/consoleFull)**
for PR 15027 at commit
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/15027
It turns out there is a bug in Hadoop's FileContext that doesn't rename the
checksum file.
LGTM pending tests.
---
If your project is set up for it, you can reply to this email and have
Github user frreiss commented on the issue:
https://github.com/apache/spark/pull/15027
When I comment out line 155 in HDFSMetadataLog.scala on this branch (`if
(fileManager.exists(crcPath)) fileManager.delete(crcPath)`) and run the test
case attached to this PR, the test case fails:
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/15027
@frreiss I saw LocalFileSystem will also rename the checksum file:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15027
**[Test build #3275 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3275/consoleFull)**
for PR 15027 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15027
**[Test build #3275 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3275/consoleFull)**
for PR 15027 at commit
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/15027
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15027
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
15 matches
Mail list logo
