Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22390
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22390
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22343
@seancxmao . For Hive compatibility,
`spark.sql.hive.convertMetastoreParquet=false` looks enough to me.
---
-
To
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/22357
> FYI, @mallman I'm working on having ParquetFilter to support
IsNotNull(employer.id) to be pushed into parquet reader.
That would be pretty cool.
---
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/22357
And FYI this is the Jira issue I promised in
https://github.com/apache/spark/pull/22357#issuecomment-419940228
yesterday: https://issues.apache.org/jira/browse/SPARK-25407.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22395
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22395
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22378
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22386#discussion_r216723459
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala ---
@@ -1029,6 +1030,35 @@ class StreamSuite extends StreamTest {
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22395
**[Test build #95954 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95954/testReport)**
for PR 22395 at commit
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22391
the failure seems persistent, though unrelated to this PR
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22395
cc @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
GitHub user mgaido91 opened a pull request:
https://github.com/apache/spark/pull/22395
[SPARK-16323][SQ] Add IntegralDivide expression
## What changes were proposed in this pull request?
The PR takes over #14036 and it introduces a new expression
`IntegralDivide` in order
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22378
Merged to master/2.4.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22386#discussion_r216723249
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala ---
@@ -1029,6 +1030,35 @@ class StreamSuite extends StreamTest {
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22391
**[Test build #4335 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4335/testReport)**
for PR 22391 at commit
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22378
Thank you for review, @cloud-fan and @HyukjinKwon .
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22378#discussion_r216722571
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala ---
@@ -750,4 +751,27 @@ class InsertSuite extends QueryTest with
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22391
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22391
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95941/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22391
**[Test build #95941 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95941/testReport)**
for PR 22391 at commit
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/22394
FYI @viirya @dbtsai @gatorsmile @HyukjinKwon
Can I get someone's review of this PR please? The unmasked failures appear
to be false positives, so no changes to the tested code are
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22326
@cloud-fan Thanks for your comment.
```
IIUC, you are pulling out the join condition with python UDF and create a
filter above join. Then the join become a cross join, which usually
Github user rgbkrk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21654#discussion_r216718402
--- Diff: python/pyspark/sql/dataframe.py ---
@@ -375,6 +375,9 @@ def _truncate(self):
return int(self.sql_ctx.getConf(
Github user adambalogh commented on the issue:
https://github.com/apache/spark/pull/22289
Thank you for the detailed explanation! @vanzin
I agree with what you are saying, however I'm not sure about some of your
points about configs, so I would like to find a common ground
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/22357
This LGTM. I'm not going to submit a PR for my approach to this problem.
Thanks @viirya!
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22394
**[Test build #95952 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95952/testReport)**
for PR 22394 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22295
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22295
**[Test build #95953 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95953/testReport)**
for PR 22295 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22295
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/22357#discussion_r216714387
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala
---
@@ -110,7 +110,17 @@ private[sql]
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22390#discussion_r216713161
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala
---
@@ -263,10 +263,12 @@ object BooleanSimplification
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/22394
I'm working on fixing these test failures now. Hopefully I'll have
something pushed soon.
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22394
**[Test build #95951 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95951/testReport)**
for PR 22394 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/22357
FYI, the PR I previously mentioned about fixing the use of `withSQLConf` is
#22394.
---
-
To unsubscribe, e-mail:
GitHub user mallman opened a pull request:
https://github.com/apache/spark/pull/22394
[SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, move calls to
`withSQLConf` inside calls to `test`
(Link to Jira: https://issues.apache.org/jira/browse/SPARK-25406)
## What
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22357
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22357
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22379
**[Test build #95948 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95948/testReport)**
for PR 22379 at commit
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22357#discussion_r216708046
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala
---
@@ -110,7 +110,17 @@ private[sql]
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22357
**[Test build #95950 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95950/testReport)**
for PR 22357 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22379
**[Test build #95949 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95949/testReport)**
for PR 22379 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/22357
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22357
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22357
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95945/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22357
**[Test build #95945 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95945/testReport)**
for PR 22357 at commit
Github user squito commented on the issue:
https://github.com/apache/spark/pull/22385
@dongjoon-hyun sure thing, done
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95947 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95947/testReport)**
for PR 22192 at commit
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20611#discussion_r216703781
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -303,94 +303,44 @@ case class LoadDataCommand(
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22357#discussion_r216702003
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala
---
@@ -110,7 +110,17 @@ private[sql]
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22213
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95934/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22213
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user bersprockets commented on the issue:
https://github.com/apache/spark/pull/22192
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22213
**[Test build #95934 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95934/testReport)**
for PR 22213 at commit
Github user Fokko commented on the issue:
https://github.com/apache/spark/pull/21596
I've rebase onto master, and fixed the test. Lets wait for the CI's opinion
of the fix.
The problem was the following, introduced in
https://github.com/apache/spark/pull/21221:
``
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21596
**[Test build #95946 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95946/testReport)**
for PR 21596 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22357
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22357
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/22357
I have some bad news. The methods `testSchemaPruning` and
`testMixedCasePruning` do not set the configuration settings as expected.
Fixing that reveals 6 failing tests for the mixed case tests. One
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22357
**[Test build #95945 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95945/testReport)**
for PR 22357 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22364
**[Test build #95944 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95944/testReport)**
for PR 22364 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22364
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22364
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user sujith71955 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20611#discussion_r216694466
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -303,94 +303,44 @@ case class LoadDataCommand(
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22357#discussion_r216694201
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala
---
@@ -155,6 +163,60 @@ class
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/22357
@viirya Please amend
Github user sujith71955 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20611#discussion_r216693375
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -303,94 +303,44 @@ case class LoadDataCommand(
Github user sujith71955 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20611#discussion_r216693154
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -303,94 +303,44 @@ case class LoadDataCommand(
Github user sujith71955 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20611#discussion_r216692677
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -303,94 +303,44 @@ case class LoadDataCommand(
Github user skonto commented on the issue:
https://github.com/apache/spark/pull/22381
@srowen thanks is there someone I could call? @vanzin ?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user bersprockets commented on the issue:
https://github.com/apache/spark/pull/22382
Thanks! Closing.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user bersprockets closed the pull request at:
https://github.com/apache/spark/pull/22382
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20611#discussion_r216690734
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -303,94 +303,44 @@ case class LoadDataCommand(
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22381
I don't think I know enough about monitoring to merge this. I am concerned
about the duplication here, although I understand your argument why it's
valuable.
---
Github user skonto commented on the issue:
https://github.com/apache/spark/pull/22381
@srowen @xuanyuanking ready for another round.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/22357#discussion_r216686762
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala
---
@@ -155,6 +163,60 @@ class
Github user sujith71955 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20611#discussion_r216685077
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -303,94 +303,44 @@ case class LoadDataCommand(
Github user jiangxb1987 closed the pull request at:
https://github.com/apache/spark/pull/20414
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/22357#discussion_r216683076
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala
---
@@ -110,7 +110,17 @@ private[sql]
Github user SongYadong commented on the issue:
https://github.com/apache/spark/pull/22348
@dongjoon-hyun . You are right, DataSourceReadBenchmark result show the
benefit is too small even in some cases is covered up by fluctuation.
Java HotSpot(TM) 64-Bit Server VM
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22375#discussion_r216679499
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala
---
@@ -223,8 +223,8 @@ trait
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22375#discussion_r216676386
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala
---
@@ -223,8 +223,8 @@ trait
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21596
**[Test build #95943 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95943/testReport)**
for PR 21596 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21596
**[Test build #95942 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95942/testReport)**
for PR 21596 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22391
**[Test build #95941 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95941/testReport)**
for PR 22391 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22391
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22391
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22391
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22387
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95933/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22387
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22387
**[Test build #95933 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95933/testReport)**
for PR 22387 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22385
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95928/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22385
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22391
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95936/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22391
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22357
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
301 - 400 of 628 matches
Mail list logo
