Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22845
@heary-cao . You had better merge in GitHub. GitHub provides merge button
for that.
Usually, merging PR is giving a credit to the person.
In Apache Spark, we don't copy someone's
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22880#discussion_r229208839
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadSupport.scala
---
@@ -49,34 +49,82 @@ import
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22880#discussion_r229204016
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRowConverter.scala
---
@@ -182,18 +182,20 @@ private[parquet]
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22880#discussion_r229211086
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRowConverter.scala
---
@@ -202,11 +204,15 @@ private[parquet]
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22844#discussion_r229211569
--- Diff: sql/core/benchmarks/JSONBenchmarks-results.txt ---
@@ -0,0 +1,33 @@
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22888
Then, you can convert the type into double or floats in Spark DataFrame.
This is super easily able to work around at Pandas DataFrame or Spark's
DataFrame. I don't think we should add this
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22844#discussion_r229210176
--- Diff: sql/core/benchmarks/JSONBenchmarks-results.txt ---
@@ -0,0 +1,33 @@
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22845
**[Test build #98259 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98259/testReport)**
for PR 22845 at commit
Github user 351zyf commented on the issue:
https://github.com/apache/spark/pull/22888
and this also have no effect on timestamp values.
tested.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22885
**[Test build #98257 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98257/testReport)**
for PR 22885 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22885
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98257/
Test FAILed.
---
Github user 351zyf commented on the issue:
https://github.com/apache/spark/pull/22888
> I think you can just manually convert from Pandas DataFrame, no?
If I'm using function toPandas, I dont think decimal to object is right.
Isn't decimal values usually a value to
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22885
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22885
**[Test build #98257 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98257/testReport)**
for PR 22885 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22845
**[Test build #98258 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98258/testReport)**
for PR 22845 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22885
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22885
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22885
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22844#discussion_r229205020
--- Diff: sql/core/benchmarks/JSONBenchmarks-results.txt ---
@@ -0,0 +1,33 @@
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22888
I think you can just manually convert from Pandas DataFrame, no?
---
-
To unsubscribe, e-mail:
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/22849#discussion_r229203395
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -240,7 +240,7 @@ class
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22888
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22888
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22844#discussion_r229202285
--- Diff: sql/core/benchmarks/JSONBenchmarks-results.txt ---
@@ -0,0 +1,33 @@
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22888
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user 351zyf opened a pull request:
https://github.com/apache/spark/pull/22888
SPARK-25881
add parametere coerce_float
https://issues.apache.org/jira/browse/SPARK-25881
## What changes were proposed in this pull request?
when using pyspark
Github user gjhkael commented on the issue:
https://github.com/apache/spark/pull/22887
test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22887
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22887
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22845#discussion_r229201094
--- Diff: sql/core/benchmarks/CSVBenchmarks-results.txt ---
@@ -0,0 +1,27 @@
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22887
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user gjhkael opened a pull request:
https://github.com/apache/spark/pull/22887
user set's hadoop conf should not overwrite by sparkcontext's conf
## What changes were proposed in this pull request?
Hadoop conf which is set by user which is use sparksql's set command
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22845#discussion_r229199547
--- Diff: sql/core/benchmarks/CSVBenchmarks-results.txt ---
@@ -0,0 +1,27 @@
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21860
**[Test build #98256 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98256/testReport)**
for PR 21860 at commit
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22845#discussion_r229199434
--- Diff: sql/core/benchmarks/CSVBenchmarks-results.txt ---
@@ -0,0 +1,27 @@
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/22849#discussion_r229198651
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -240,7 +240,7 @@ class
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22886
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gjhkael closed the pull request at:
https://github.com/apache/spark/pull/22886
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22886
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user gjhkael opened a pull request:
https://github.com/apache/spark/pull/22886
Hadoop config should overwrite by users conf
## What changes were proposed in this pull request?
Hadoop conf which is set by user which is use sparksql's set command should
not overwrite by
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22755
**[Test build #98255 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98255/testReport)**
for PR 22755 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22881
**[Test build #98249 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98249/testReport)**
for PR 22881 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22844
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22844
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98253/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22755
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22878
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22881
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22878
**[Test build #98251 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98251/testReport)**
for PR 22878 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22755
**[Test build #98252 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98252/testReport)**
for PR 22755 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22755
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22755
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98252/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22881
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98249/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22845
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98254/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22878
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98251/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22845
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22844
**[Test build #98253 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98253/testReport)**
for PR 22844 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22845
**[Test build #98254 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98254/testReport)**
for PR 22845 at commit
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/22845
@dongjoon-hyun, Well, my office machine.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21320
> https://issues.apache.org/jira/browse/SPARK-25879
>
> If we select a nested field and a top level field, the schema pruning
will fail. Here is the reproducible test,
> ...
Hi
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22755
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user yucai commented on the issue:
https://github.com/apache/spark/pull/22861
@dongjoon-hyun Tests have been passed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22845#discussion_r229190198
--- Diff: sql/core/benchmarks/CSVBenchmarks-results.txt ---
@@ -0,0 +1,27 @@
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22845
Thank you for updating and rerunning the tests, @heary-cao .
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22309
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22309
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98248/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22309
**[Test build #98248 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98248/testReport)**
for PR 22309 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22845
**[Test build #98254 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98254/testReport)**
for PR 22845 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22778
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98246/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22778
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22778
**[Test build #98246 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98246/testReport)**
for PR 22778 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22861
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98245/
Test PASSed.
---
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/21320
cc @viirya
If we select a nested field and a top level field, the schema pruning will
fail. Here is the reproducible test,
```scala
testSchemaPruning("select a single complex
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22861
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22861
**[Test build #98245 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98245/testReport)**
for PR 22861 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22844
**[Test build #98253 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98253/testReport)**
for PR 22844 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22861
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98247/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22861
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22878
Thanks @gengliangwang and @HyukjinKwon. Done in this commit.
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22861
**[Test build #98247 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98247/testReport)**
for PR 22861 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22878
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22878
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22878
**[Test build #98251 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98251/testReport)**
for PR 22878 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22755
**[Test build #98252 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98252/testReport)**
for PR 22755 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22878
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98244/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22878
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22878
**[Test build #98244 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98244/testReport)**
for PR 22878 at commit
401 - 486 of 486 matches
Mail list logo
