Github user zjffdu closed the pull request at:
https://github.com/apache/spark/pull/9657
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enab
Github user maropu commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-190107773
This is resolved by #11384, so could you close this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If yo
Github user maropu commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-185725944
@zjffdu If you have no time to take this, is it okay I rework?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as w
Github user maropu commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-181786960
@zjffdu ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
en
Github user zjffdu commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-178487878
sorry for late response, will update the patch tomorrow.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user maropu commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-178371928
@zjffdu ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
en
Github user maropu commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-177026772
@zjffdu ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
en
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r51332909
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ---
@@ -155,6 +157,15 @@ final class DataFrameWriter private[sql](df:
DataFrame)
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-176653290
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-176653295
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/5
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-176653248
**[Test build #50364 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50364/consoleFull)**
for PR 9657 at commit
[`3645dbc`](https://git
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r51237900
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ---
@@ -155,6 +157,15 @@ final class DataFrameWriter private[sql](df:
DataFrame) {
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r51237807
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ---
@@ -155,6 +157,15 @@ final class DataFrameWriter private[sql](df:
DataFrame) {
Github user maropu commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-176649314
@rxin @liancheng Could you review this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project do
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-176646312
**[Test build #50364 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50364/consoleFull)**
for PR 9657 at commit
[`3645dbc`](https://gith
Github user zjffdu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r51233700
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ResolvedDataSource.scala
---
@@ -235,11 +236,18 @@ object ResolvedDataSource exte
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r51233418
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/WriterContainer.scala
---
@@ -239,8 +249,10 @@ private[sql] abstract class BaseWr
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r51233196
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ResolvedDataSource.scala
---
@@ -235,11 +236,18 @@ object ResolvedDataSource exte
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r51232882
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ---
@@ -155,6 +157,15 @@ final class DataFrameWriter private[sql](df:
DataFrame)
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r51232826
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ---
@@ -155,6 +157,15 @@ final class DataFrameWriter private[sql](df:
DataFrame)
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r51232808
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ---
@@ -130,6 +131,16 @@ final class DataFrameWriter private[sql](df:
DataFrame)
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-176589925
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/5
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-176589924
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-176589845
**[Test build #50347 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50347/consoleFull)**
for PR 9657 at commit
[`a8ed421`](https://git
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-176584946
**[Test build #50345 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50345/consoleFull)**
for PR 9657 at commit
[`ea70b40`](https://git
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-176584980
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/5
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-176584979
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-176583892
**[Test build #50347 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50347/consoleFull)**
for PR 9657 at commit
[`a8ed421`](https://gith
Github user zjffdu commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-176582619
@maropu Thanks for review, I update the PR to address part of your
comments. Please check my comments inline.
---
If your project is set up for it, you can reply to thi
Github user zjffdu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r51227540
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ResolvedDataSource.scala
---
@@ -235,11 +236,18 @@ object ResolvedDataSource exte
Github user zjffdu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r51227333
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ---
@@ -130,6 +131,16 @@ final class DataFrameWriter private[sql](df:
DataFrame)
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-176581521
**[Test build #50345 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50345/consoleFull)**
for PR 9657 at commit
[`ea70b40`](https://gith
Github user zjffdu commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-176560383
Will update this PR
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fea
Github user maropu commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-176556589
@zjffdu Any updates? If you keep working on it, please check my comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r51222968
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ResolvedDataSource.scala
---
@@ -235,11 +236,18 @@ object ResolvedDataSource exte
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r51222766
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelation.scala
---
@@ -57,7 +57,8 @@ import org.apache.spark.ut
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r51222742
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala
---
@@ -120,7 +120,7 @@ private[sql] object DataSourceSt
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r51222704
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ---
@@ -130,6 +131,16 @@ final class DataFrameWriter private[sql](df:
DataFrame)
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-161536165
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-161536163
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-161536062
**[Test build #47118 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47118/consoleFull)**
for PR 9657 at commit
[`657ba5a`](https://git
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-161521736
**[Test build #47118 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47118/consoleFull)**
for PR 9657 at commit
[`657ba5a`](https://gith
Github user zjffdu commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-161519077
Never mind, I change back to 1.7.0 since 1.6 is in rc1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-161203297
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-161203296
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-161203193
**[Test build #47036 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47036/consoleFull)**
for PR 9657 at commit
[`db31b8c`](https://git
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-161193474
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-161193471
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-161193217
**[Test build #47034 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47034/consoleFull)**
for PR 9657 at commit
[`896440a`](https://git
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-161184786
**[Test build #47036 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47036/consoleFull)**
for PR 9657 at commit
[`db31b8c`](https://gith
Github user zjffdu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r46377699
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/WriterContainer.scala
---
@@ -195,6 +200,11 @@ private[sql] abstract class BaseWr
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-161172865
**[Test build #47034 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47034/consoleFull)**
for PR 9657 at commit
[`896440a`](https://gith
Github user zjffdu commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-161171587
@Lewuathe Thanks for the review. I push another commit to address the
comments. Besides I change the compression feature to 1.6.0.
---
If your project is set up for it
Github user zjffdu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r46373554
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala
---
@@ -58,6 +60,14 @@ class TextSuite extends QueryTest wit
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r46373216
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/WriterContainer.scala
---
@@ -195,6 +200,11 @@ private[sql] abstract class Base
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r46372846
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala
---
@@ -58,6 +60,14 @@ class TextSuite extends QueryTest w
Github user zjffdu commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-161165602
@Lewuathe Yes, I'd like to make CsvRelation to extend HadoopFsRelation also
after this change since currently CsvRelation support compression so want to
keep its compatib
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r46372779
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala
---
@@ -17,9 +17,11 @@
package org.apache.spark
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r46372688
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/WriterContainer.scala
---
@@ -19,6 +19,10 @@ package org.apache.spark.sql.execu
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r46372602
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ResolvedDataSource.scala
---
@@ -19,6 +19,8 @@ package org.apache.spark.sql.exe
Github user zjffdu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r46372338
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala
---
@@ -17,9 +17,11 @@
package org.apache.spark.s
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r46371848
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala
---
@@ -17,9 +17,11 @@
package org.apache.spark
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r46371759
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelation.scala
---
@@ -30,7 +30,7 @@ import org.apache.spark.
Github user Lewuathe commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-161161684
@zjffdu Do you intend to remove the compression codev from `CsvRelation`
after changing `HadoopFsRelation` and inherit it? Since `spark-csv` is a
external library, it i
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/9657#discussion_r46371186
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ---
@@ -28,7 +28,7 @@ import
org.apache.spark.sql.catalyst.plans.logical.{Proje
Github user zjffdu commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-160891720
@Lewuathe Could you help review this ? This is a dependency issue for
refactor CsvRelation to extend HadoopFsRelation. CsvRelation now support to
write to compressed for
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-156086038
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-156086043
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-156085707
**[Test build #45728 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45728/consoleFull)**
for PR 9657 at commit
[`5269f4c`](https://git
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-156052895
**[Test build #45728 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45728/consoleFull)**
for PR 9657 at commit
[`5269f4c`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-156052492
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-156052526
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-156041959
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-156041965
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-156041775
**[Test build #45721 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45721/consoleFull)**
for PR 9657 at commit
[`09d39e5`](https://git
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-156033144
**[Test build #45721 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45721/consoleFull)**
for PR 9657 at commit
[`09d39e5`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-156031711
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9657#issuecomment-156031702
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
GitHub user zjffdu opened a pull request:
https://github.com/apache/spark/pull/9657
[SPARK-11691][SQL] Allow to specify compression codec in HadoopFsRelaâ¦
â¦tion when saving
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zjffd
79 matches
Mail list logo
