Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/23194#discussion_r238256611
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
---
@@ -377,41 +377,42 @@ abstract class DDLSuite extends QueryTest
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/23194#discussion_r238256563
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
---
@@ -377,41 +377,42 @@ abstract class DDLSuite extends QueryTest
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23152
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99602/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23152
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23152
**[Test build #99602 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99602/testReport)**
for PR 23152 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23205
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23205
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99600/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23205
**[Test build #99600 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99600/testReport)**
for PR 23205 at commit
Github user golovan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23094#discussion_r238246140
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala ---
@@ -87,11 +87,32 @@ abstract class JdbcDialect extends Serializable {
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20433
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20433
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99604/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22512
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99601/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22512
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20433
**[Test build #99604 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99604/testReport)**
for PR 20433 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22512
**[Test build #99601 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99601/testReport)**
for PR 22512 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20935
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/22318
In the example @viirya described above
(https://github.com/apache/spark/pull/22318#issuecomment-426317617), I think
the interpretation is unclear to most users and I'm fairly concerned that it
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23186
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/23186
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/23094#discussion_r238239581
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala ---
@@ -87,11 +87,32 @@ abstract class JdbcDialect extends Serializable {
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/23094#discussion_r238239195
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala ---
@@ -87,11 +87,32 @@ abstract class JdbcDialect extends Serializable {
Github user golovan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23094#discussion_r238235137
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala ---
@@ -87,11 +87,32 @@ abstract class JdbcDialect extends Serializable {
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23203
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99599/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23203
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23186
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23186
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99598/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23203
**[Test build #99599 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99599/testReport)**
for PR 23203 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23186
**[Test build #99598 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99598/testReport)**
for PR 23186 at commit
Github user peter-toth commented on the issue:
https://github.com/apache/spark/pull/22318
@maropu, I don't think we can. Actually this is how we deal with [simpler
joins](https://github.com/apache/spark/pull/22318#issuecomment-427080091)
Do you think changing the behaviour is
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/23203
Not look closely at the changes yet, but I think it should be very useful.
Thanks @HyukjinKwon
---
-
To unsubscribe, e-mail:
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/23094#discussion_r238232050
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala ---
@@ -87,11 +87,32 @@ abstract class JdbcDialect extends Serializable {
Github user eatoncys commented on the issue:
https://github.com/apache/spark/pull/23010
But we may forget to filter null values when we write sql. The following
function protects this situation and writes the value of null partitions as
__HIVE_DEFAULT_PARTITION__
def
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/23094
Also, can you add tests in MySQLIntegrationSuite, too?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user eatoncys commented on the issue:
https://github.com/apache/spark/pull/23010
@cloud-fan, Thanks for review, Do you mean we should filter out invalid
partitions in sql before write?
---
-
To unsubscribe,
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/23094
@golovan thanks for your work. Can you make the title complete (... -> for
table names)?
---
-
To unsubscribe, e-mail:
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/23094
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/23195
@HeartSaVioR thanks for the review!
cc @steveloughran maybe also interested
---
-
To unsubscribe, e-mail:
Github user httfighter commented on the issue:
https://github.com/apache/spark/pull/22683
@srowen Yes. I agree with you! These places should be consistent, otherwise
it is easy to be confused. I will try to modify log statements and docs.
Should I modify it in this PR or a new one?
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/23203
I used to run pyspark test via `python python/pyspark/sql/dataframe.py`,
after setting `export PYTHONPATH="$(find "${SPARK_HOME}"/python/lib/ -name
'py4j-*-src.zip' -type f |
Github user httfighter commented on a diff in the pull request:
https://github.com/apache/spark/pull/22683#discussion_r238215152
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -1164,17 +1164,17 @@ private[spark] object Utils extends Logging {
} else
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23120
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/22468
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/23120
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/23010
The root cause is, `DynamicPartitionDataWriter` treats null and empty
string as different partition values, and creates new files. However, null and
empty string are converted to
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/23195#discussion_r238211153
--- Diff: docs/structured-streaming-kafka-integration.md ---
@@ -624,3 +624,57 @@ For experimenting on `spark-shell`, you can also use
`--packages`
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/23195#discussion_r238210009
--- Diff: docs/structured-streaming-kafka-integration.md ---
@@ -624,3 +624,57 @@ For experimenting on `spark-shell`, you can also use
`--packages`
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/23195#discussion_r238209243
--- Diff: docs/structured-streaming-kafka-integration.md ---
@@ -624,3 +624,57 @@ For experimenting on `spark-shell`, you can also use
`--packages`
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/23195#discussion_r238209492
--- Diff: docs/structured-streaming-kafka-integration.md ---
@@ -624,3 +624,57 @@ For experimenting on `spark-shell`, you can also use
`--packages`
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/23194
LGTM, cc @HyukjinKwon.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/22204
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23206
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23206
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23206
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user skambha opened a pull request:
https://github.com/apache/spark/pull/23206
[SPARK-26249][SQL] Add ability to inject a rule in order and to add a batch
via the Spark Extension Points API
## What changes were proposed in this pull request?
Add two new APIs to
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22198
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23202#discussion_r238199960
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/CSVInferSchema.scala
---
@@ -98,6 +100,7 @@ class CSVInferSchema(options:
Github user aditanase commented on the issue:
https://github.com/apache/spark/pull/23155
@dongjoon-hyun is this ok to merge? Should I squash it and ping you again?
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20433
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22198
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20433
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20433
**[Test build #99604 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99604/testReport)**
for PR 20433 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22198
**[Test build #99603 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99603/testReport)**
for PR 22198 at commit
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/22198
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/20433
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/23186
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23150#discussion_r238195206
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -1618,6 +1618,13 @@ object SQLConf {
"a SparkConf
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23150#discussion_r238195153
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -1618,6 +1618,13 @@ object SQLConf {
"a SparkConf
Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/23202#discussion_r238194142
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/CSVInferSchema.scala
---
@@ -98,6 +100,7 @@ class CSVInferSchema(options:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23152
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23152
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23152
**[Test build #99602 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99602/testReport)**
for PR 23152 at commit
Github user juliuszsompolski commented on the issue:
https://github.com/apache/spark/pull/23152
jenkins retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user sujith71955 closed the pull request at:
https://github.com/apache/spark/pull/23197
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user sujith71955 commented on the issue:
https://github.com/apache/spark/pull/23197
Closing the PR has per Li's comment. Thanks for the suggestion Li.
---
-
To unsubscribe, e-mail:
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23205
Thanks @pgandhi999 :+1:
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/23190
thanks, @cloud-fan @gatorsmile @kiszk.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22512
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22512
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user JkSelf commented on the issue:
https://github.com/apache/spark/pull/23204
**Cluster info:**
 | Master Node | Worker Nodes
-- | -- | --
Node | 1x | 4x
Processor | Intel(R) Xeon(R) Platinum 8170 CPU @ 2.10GHz | Intel(R) Xeon(R)
Platinum 8180 CPU @
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22512
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22512
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23205
**[Test build #99600 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99600/testReport)**
for PR 23205 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22512
**[Test build #99601 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99601/testReport)**
for PR 22512 at commit
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/22512#discussion_r238177033
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala ---
@@ -148,12 +156,21 @@ class SQLQueryTestSuite extends QueryTest with
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23205
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23205
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user pgandhi999 commented on the issue:
https://github.com/apache/spark/pull/23088
PR : https://github.com/apache/spark/pull/23205 for displaying the message
on empty data in task summary metrics table.
---
Github user pgandhi999 commented on the issue:
https://github.com/apache/spark/pull/23205
cc @tgravescs @abellina
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/22512#discussion_r238176184
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/InterpretedMutableProjection.scala
---
@@ -64,7 +85,7 @@ class
GitHub user pgandhi999 opened a pull request:
https://github.com/apache/spark/pull/23205
[SPARK-26253][WebUI] : Task Summary Metrics Table on Stage Page shows empty
table when no data is present
Task Summary Metrics Table on Stage Page shows empty table when no data is
present
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/22512#discussion_r238175630
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala ---
@@ -148,12 +156,21 @@ class SQLQueryTestSuite extends QueryTest with
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23204
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23204
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23204
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user JkSelf opened a pull request:
https://github.com/apache/spark/pull/23204
Revert "[SPARK-21052][SQL] Add hash map metrics to join"
Because of the performance degradation discussion in
[SPARK-26155](https://issues.apache.org/jira/browse/SPARK-26155), currently we
revert
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23203
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23203
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23190
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23203#discussion_r238173745
--- Diff: python/run-tests-with-coverage ---
@@ -50,8 +50,6 @@ export SPARK_CONF_DIR="$COVERAGE_DIR/conf"
# This environment variable enables the
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/23190
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
501 - 600 of 626 matches
Mail list logo
