Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22518#discussion_r232558384
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala
---
@@ -1268,4 +1269,16 @@ class SubquerySuite extends QueryTest with
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22887
looks reasonable, cc @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22693#discussion_r232556859
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala ---
@@ -115,26 +116,45 @@ class ResolveHiveSerdeTable(session:
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22944#discussion_r232556359
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala
---
@@ -1556,6 +1556,20 @@ class DatasetSuite extends QueryTest with
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23005
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23005
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22429
This is hard to review, do you mean we should add `maxFields: Option[Int]`
to all the string related methods?
---
-
To
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22976
LGTM except one comment, cc @rednaxelafx
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22955
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22976#discussion_r232552336
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateOrdering.scala
---
@@ -68,62 +68,55 @@ object
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22955
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22954
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22954
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98713/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22954
**[Test build #98713 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98713/testReport)**
for PR 22954 at commit
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22938#discussion_r232550860
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala
---
@@ -1813,6 +1817,7 @@ class JsonSuite extends
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22938#discussion_r232550733
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala
---
@@ -1115,6 +1115,7 @@ class JsonSuite extends
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22938#discussion_r232550502
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala
---
@@ -550,15 +550,23 @@ case class
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22966#discussion_r232550388
--- Diff:
external/avro/src/test/scala/org/apache/spark/sql/execution/benchmark/AvroReadBenchmark.scala
---
@@ -0,0 +1,226 @@
+/*
+ *
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22938#discussion_r232550186
--- Diff: docs/sql-migration-guide-upgrade.md ---
@@ -15,6 +15,8 @@ displayTitle: Spark SQL Upgrading Guide
- Since Spark 3.0, the
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22998
I think this is wrong. We have to zero out the bytes even writing a null
decimal, so that 2 unsafe rows with same values(including null values) are
exactly same(in binary format).
---
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23012#discussion_r232549234
--- Diff: docs/index.md ---
@@ -31,7 +31,8 @@ Spark runs on both Windows and UNIX-like systems (e.g.
Linux, Mac OS). It's easy
locally on one
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/23012#discussion_r232549062
--- Diff: docs/index.md ---
@@ -31,7 +31,8 @@ Spark runs on both Windows and UNIX-like systems (e.g.
Linux, Mac OS). It's easy
locally on one machine
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23012#discussion_r232549053
--- Diff: docs/index.md ---
@@ -31,7 +31,8 @@ Spark runs on both Windows and UNIX-like systems (e.g.
Linux, Mac OS). It's easy
locally on one
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/23012#discussion_r232548211
--- Diff: docs/index.md ---
@@ -31,7 +31,8 @@ Spark runs on both Windows and UNIX-like systems (e.g.
Linux, Mac OS). It's easy
locally on one machine
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22974
**[Test build #98719 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98719/testReport)**
for PR 22974 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22974
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22974
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23005
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23005
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98711/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23005
**[Test build #98711 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98711/testReport)**
for PR 23005 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23012
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23012
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98718/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23012
**[Test build #98718 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98718/testReport)**
for PR 23012 at commit
Github user imatiach-msft commented on the issue:
https://github.com/apache/spark/pull/23009
@shanyu can you update the name as [SPARK-26011][CORE][PYSPARK] according
to the guidelines?
---
-
To unsubscribe,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22974
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98712/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22974
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22974
**[Test build #98712 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98712/testReport)**
for PR 22974 at commit
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22939#discussion_r232540931
--- Diff: R/pkg/R/functions.R ---
@@ -2230,6 +2237,32 @@ setMethod("from_json", signature(x = "Column",
schema = "characterOrstructType")
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23012
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23012
**[Test build #98718 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98718/testReport)**
for PR 23012 at commit
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/22998
@kiszk thank you for review it.
- when writing null decimalsï¼
```
OpenJDK 64-Bit Server VM 1.8.0_163-b01 on Windows 7 6.1
Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23012
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23012
adding @srowen too.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23012
Tests probably will fail since it produces warnings.
cc @felixcheung. @shaneknapp, @viirya, @shivaram, @falaki, @mengxr,
@yanboliang FYI.
This PR is made per
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22721
**[Test build #98717 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98717/testReport)**
for PR 22721 at commit
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/22998
@mgaido91
thank you for review it. I added a test case to test "write a decimal with
16 bytes and then one with less than 8". then the current change the remaining
8 bytes would not dirty.
GitHub user HyukjinKwon opened a pull request:
https://github.com/apache/spark/pull/23012
[SPARK-26014][R] Deprecate R prior to version 3.4 in SparkR
## What changes were proposed in this pull request?
This PR proposes to bump up the minimum versions of R from 3.1 to 3.4.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22721
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22721
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user imatiach-msft commented on the issue:
https://github.com/apache/spark/pull/23009
Jenkins test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user imatiach-msft commented on the issue:
https://github.com/apache/spark/pull/23009
LGTM, nice find
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23011
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23011
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23010
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23011
**[Test build #98716 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98716/testReport)**
for PR 23011 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23010
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23011
cc @felixcheung
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
GitHub user HyukjinKwon opened a pull request:
https://github.com/apache/spark/pull/23011
[SPARK-26013][R][BUILD] Upgrade R tools version from 3.4.0 to 3.5.1 in
AppVeyor build
## What changes were proposed in this pull request?
R tools 3.5.1 is released few months ago.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23010
**[Test build #98715 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98715/testReport)**
for PR 23010 at commit
GitHub user eatoncys opened a pull request:
https://github.com/apache/spark/pull/23010
[SPARK-26012][SQL]Null and '' values should not cause dynamic partition
failure of string types
## What changes were proposed in this pull request?
Dynamic partition will fail when both
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22962
cc @jiangxb1987 @MrBago
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22962
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22962
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98714/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22962
**[Test build #98714 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98714/testReport)**
for PR 22962 at commit
Github user seancxmao commented on the issue:
https://github.com/apache/spark/pull/22184
@HyukjinKwon Thank you for your comments. Yes, this is only valid when
upgrade Spark 2.3 to 2.4. I will do it.
---
-
To
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22989
Sorry for late reply, great thanks for all reviewer's advise, will address
them soon.
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22962
Looks making sense to me in general.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22962#discussion_r232528655
--- Diff: python/pyspark/tests.py ---
@@ -618,10 +618,13 @@ def test_barrier_with_python_worker_reuse(self):
"""
Verify that
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22962
@HyukjinKwon Thanks for your review, comment address and PR
description/title changed done.
---
-
To unsubscribe, e-mail:
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22962#discussion_r232528333
--- Diff: python/pyspark/taskcontext.py ---
@@ -144,10 +144,19 @@ def __init__(self):
"""Construct a BarrierTaskContext, use get
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22962
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22962
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22962#discussion_r232527808
--- Diff: python/pyspark/tests.py ---
@@ -614,6 +614,18 @@ def context_barrier(x):
times =
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22962
**[Test build #98714 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98714/testReport)**
for PR 22962 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22954
**[Test build #98713 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98713/testReport)**
for PR 22954 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22954
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22954
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23009
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23009
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user zhengruifeng commented on the issue:
https://github.com/apache/spark/pull/22087
I also expose GMM's predictProbability.
could you please make a final pass? @srowen @felixcheung
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23009
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user shanyu opened a pull request:
https://github.com/apache/spark/pull/23009
SPARK-26011: pyspark app with "spark.jars.packages" config does not work
SparkSubmit determines pyspark app by the suffix of primary resource but
Livy
uses "spark-internal" as the primary
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22954
Yea .. I will make the followup works right away after this one get merged.
Thanks @felixcheung. Let me address the rest of comments, and wait for Arrow
release.
@BryanCutler BTW, do
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22954#discussion_r232525184
--- Diff: R/pkg/tests/fulltests/test_sparkSQL.R ---
@@ -307,6 +307,64 @@ test_that("create DataFrame from RDD", {
unsetHiveContext()
})
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22954#discussion_r232525068
--- Diff: R/pkg/tests/fulltests/test_sparkSQL.R ---
@@ -307,6 +307,64 @@ test_that("create DataFrame from RDD", {
unsetHiveContext()
})
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23008
**[Test build #98710 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98710/testReport)**
for PR 23008 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23008
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98710/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23008
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22974
**[Test build #98712 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98712/testReport)**
for PR 22974 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22974
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22974
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23005
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23005
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23005
**[Test build #98711 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98711/testReport)**
for PR 23005 at commit
Github user zhengruifeng commented on the issue:
https://github.com/apache/spark/pull/22974
@srowen I have some spare time, and will work on it.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user wangtao605 commented on the issue:
https://github.com/apache/spark/pull/21766
> @wangtao605 Do you mind documenting our behavior in our Spark SQL doc?
Yes, it's ok.
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23008
**[Test build #98710 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98710/testReport)**
for PR 23008 at commit
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22979#discussion_r232520110
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/UnivocityParser.scala
---
@@ -149,8 +156,8 @@ class UnivocityParser(
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23008
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/22764
@mgaido91 I'm on thanksgiving vacation, will be back to community to help
code review on Nov 21st. Sorry for the delay.
---
-
1 - 100 of 297 matches
Mail list logo
