Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/11925#discussion_r59487598
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala
---
@@ -196,4 +195,148 @@ class HDFSMetadataLog[T:
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/11925
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201619344
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201619350
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201618700
**[Test build #54230 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54230/consoleFull)**
for PR 11925 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201611512
**[Test build #2696 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2696/consoleFull)**
for PR 11925 at commit
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201589402
Alright I am merging this to master. Thanks @zsxwing
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201572014
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201572018
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201571487
**[Test build #54219 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54219/consoleFull)**
for PR 11925 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201562753
**[Test build #54230 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54230/consoleFull)**
for PR 11925 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201561829
**[Test build #2696 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2696/consoleFull)**
for PR 11925 at commit
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201525677
LGTM except some nits
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11925#discussion_r57493848
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala
---
@@ -196,4 +195,148 @@ class HDFSMetadataLog[T:
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11925#discussion_r57493866
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala
---
@@ -196,4 +195,148 @@ class HDFSMetadataLog[T:
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11925#discussion_r57493825
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala
---
@@ -196,4 +195,148 @@ class HDFSMetadataLog[T:
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11925#discussion_r57493782
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala
---
@@ -196,4 +195,148 @@ class HDFSMetadataLog[T:
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201520324
**[Test build #54219 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54219/consoleFull)**
for PR 11925 at commit
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201519398
The last test failures were due to the changes in SharedSQLContext and
TestSQLContext. The last commit should fix that.
---
If your project is set up for it, you can
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201439988
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201439993
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201439602
**[Test build #54194 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54194/consoleFull)**
for PR 11925 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201393248
**[Test build #54194 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54194/consoleFull)**
for PR 11925 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201087805
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201087807
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201087799
**[Test build #54127 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54127/consoleFull)**
for PR 11925 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201087615
**[Test build #54127 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54127/consoleFull)**
for PR 11925 at commit
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11925#discussion_r57406100
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala
---
@@ -196,4 +195,148 @@ class HDFSMetadataLog[T:
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201066015
**[Test build #54109 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54109/consoleFull)**
for PR 11925 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201066025
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201066020
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-201065768
**[Test build #54109 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54109/consoleFull)**
for PR 11925 at commit
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/11925#discussion_r57381927
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala
---
@@ -196,4 +194,107 @@ class HDFSMetadataLog[T:
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-200973256
Looks pretty good except the `fs.rename` issue and some nits.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11925#discussion_r57369347
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLogSuite.scala
---
@@ -17,16 +17,28 @@
package
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11925#discussion_r57367862
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala
---
@@ -196,4 +194,107 @@ class HDFSMetadataLog[T:
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11925#discussion_r57367309
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala
---
@@ -196,4 +194,107 @@ class HDFSMetadataLog[T:
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-200962816
test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-200701222
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-200701133
**[Test build #54006 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54006/consoleFull)**
for PR 11925 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-200682795
**[Test build #54006 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54006/consoleFull)**
for PR 11925 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-200612231
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-200612221
**[Test build #53991 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53991/consoleFull)**
for PR 11925 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-200612229
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-200611865
**[Test build #53991 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53991/consoleFull)**
for PR 11925 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-200610889
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-200610876
**[Test build #53990 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53990/consoleFull)**
for PR 11925 at commit
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-200610470
@marmbrus @zsxwing @JoshRosen
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-200610884
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11925#issuecomment-200610164
**[Test build #53990 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53990/consoleFull)**
for PR 11925 at commit
GitHub user tdas opened a pull request:
https://github.com/apache/spark/pull/11925
[SPARK-14109][SQL] Fix HDFSMetadataLog to fallback to FileSystem
## What changes were proposed in this pull request?
HDFSMetadataLog uses newer FileContext API to achieve atomic renaming.
51 matches
Mail list logo
