Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22441#discussion_r218465898
--- Diff: dev/create-release/release-build.sh ---
@@ -111,13 +111,17 @@ fi
# different versions of Scala are supported.
BASE_PROFILES="-Pmesos
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22441#discussion_r218464733
--- Diff: dev/create-release/release-build.sh ---
@@ -414,15 +437,15 @@ if [[ "$1" == "publish-release" ]]; then
-DskipTests $PUBLISH_PROFILES
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22305
**[Test build #96184 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96184/testReport)**
for PR 22305 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22305
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22305
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user seancxmao commented on the issue:
https://github.com/apache/spark/pull/22453
@HyukjinKwon Could you please help review this?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22453
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22381
Looks like different failures. I am not sure if these are known issues. I'd
try again later today.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22453
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22453
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user seancxmao opened a pull request:
https://github.com/apache/spark/pull/22453
[SPARK-20937][DOCS] Describe spark.sql.parquet.writeLegacyFormat property
in Spark SQL, DataFrames and Datasets Guide
## What changes were proposed in this pull request?
Describe
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22441#discussion_r218458224
--- Diff: dev/create-release/release-build.sh ---
@@ -414,15 +437,15 @@ if [[ "$1" == "publish-release" ]]; then
-DskipTests $PUBLISH_PROFILES
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22441
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22452
**[Test build #96183 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96183/testReport)**
for PR 22452 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22452
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22452
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22441
Since I've verified it manually, I'm merging it to master/2.4, thanks!
---
-
To unsubscribe, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22441
The test failure is a known issue:
https://issues.apache.org/jira/browse/SPARK-25456
Actually PR builld can't verify release scripts, so we don't need to wait
for it...
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20761
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20761
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96181/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20761
**[Test build #96181 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96181/testReport)**
for PR 20761 at commit
GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/22452
[SPARK-25456][SQL][TEST] Fix PythonForeachWriterSuite
PythonForeachWriterSuite was failing because RowQueue now needs to have a
handle on a SparkEnv with a SerializerManager, so added a mock env
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21403
I'm writing release notes, and this one gets my attention. @mgaido91 can
you confirm that this patch doesn't introduce any behavior change? i.e. if it
fails previously, it still fails. If it
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22451
**[Test build #96182 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96182/testReport)**
for PR 22451 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22451
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22451
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/22415
I'm gonna merge this to master and branch-2.4 before noon if there's no
more comment.
---
-
To unsubscribe, e-mail:
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22451
[SPARK-24777][SQL] Add write benchmark for AVRO
## What changes were proposed in this pull request?
Refactor `DataSourceWriteBenchmark` and add write benchmark for AVRO.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20761
**[Test build #96181 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96181/testReport)**
for PR 20761 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22429
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96175/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22429
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22450
**[Test build #96180 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96180/testReport)**
for PR 22450 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22450
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22450
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22429
**[Test build #96175 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96175/testReport)**
for PR 22429 at commit
GitHub user mgaido91 opened a pull request:
https://github.com/apache/spark/pull/22450
[SPARK-25454][SQL] Avoid precision loss in division with decimal with
negative scale
## What changes were proposed in this pull request?
Our rules for determine decimal precision and
Github user LantaoJin commented on the issue:
https://github.com/apache/spark/pull/22411
@cloud-fan I refactor and remove the function outputPath in
```DataWritingCommand```. Besides the unit test you could see, in my local, I
added below test in ```HiveQuerySuite.scala```:
Github user szyszy commented on the issue:
https://github.com/apache/spark/pull/20761
Hi @squito, @srowen and @vanzin !
Thanks for the reviews so far!
Please see the updated code, I hope everything fixed that was commented.
@vanzin: Please also check the simplified
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20761
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20761
**[Test build #96179 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96179/testReport)**
for PR 20761 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20761
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96179/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22429
**[Test build #96178 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96178/testReport)**
for PR 22429 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20761
**[Test build #96179 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96179/testReport)**
for PR 20761 at commit
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/22419
I just linked to the previous discussion: #18106
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/22419#discussion_r218408565
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/MathUtils.scala
---
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/22364
also, cc: @dongjoon-hyun
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22364
thanks @maropu for your review! @gatorsmile do you have any comments?
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18544
**[Test build #96177 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96177/testReport)**
for PR 18544 at commit
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/22419
btw, in the title, not `UDF` but `built-in`?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/22364
Basically, this change looks good to me. I leave this to other reviewers.
---
-
To unsubscribe, e-mail:
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/22364
oh, yea, thanks! I wrongly mixed up `(AttributeSet --
Seq[Attribute]).nonEmpty` with this case.
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22165
**[Test build #96176 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96176/testReport)**
for PR 22165 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22165
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22165
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21403#discussion_r218398155
--- Diff:
sql/core/src/test/resources/sql-tests/results/subquery/in-subquery/in-basic.sql.out
---
@@ -0,0 +1,70 @@
+-- Automatically generated by
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22165
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21403#discussion_r218398060
--- Diff:
sql/core/src/test/resources/sql-tests/results/subquery/in-subquery/in-basic.sql.out
---
@@ -0,0 +1,70 @@
+-- Automatically generated by
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22449
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22449
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96174/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22449
**[Test build #96174 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96174/testReport)**
for PR 22449 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22165
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22395
Sure @cloud-fan, I'll create a JIRA and submit a PR for it.
> Looks like a use case for a legacy config.
Yes, thanks for the suggestion @rxin, I agree.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22165
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96173/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22165
**[Test build #96173 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96173/testReport)**
for PR 22165 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22316
I could check it by myself but it would take some time since I'm kind of
busy for now :-(
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22441
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96171/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22441
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22441
**[Test build #96171 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96171/testReport)**
for PR 22441 at commit
Github user steveloughran commented on the issue:
https://github.com/apache/spark/pull/21048
if this is being looked at again, it'd be nice to have a reference back end
which just did the write straight to the destination: this is exactly what all
the public cloud stores (s3,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22429
**[Test build #96175 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96175/testReport)**
for PR 22429 at commit
Github user steveloughran commented on the issue:
https://github.com/apache/spark/pull/22444
I see the reasoning here
* @jianjianjiao has a very large cluster with many thousands of history
files of past (successful) jobs.
* history server startup needs to go through all
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22449
**[Test build #96174 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96174/testReport)**
for PR 22449 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22449
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22449
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user WeichenXu123 opened a pull request:
https://github.com/apache/spark/pull/22449
[SPARK-22666][ML][FOLLOW-UP] Return a correctly formatted URI for invalid
images
## What changes were proposed in this pull request?
Change the URI returned in ImageFileFormat for an
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22448
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96170/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22448
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22448
**[Test build #96170 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96170/testReport)**
for PR 22448 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22402
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22402
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96172/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22402
**[Test build #96172 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96172/testReport)**
for PR 22402 at commit
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/21618#discussion_r218373252
--- Diff: core/src/main/java/org/apache/hadoop/fs/SparkGlobber.java ---
@@ -0,0 +1,293 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF)
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/22429
@MaxGekk please just modify simpleString it is internal API for this reason.
@rednaxelafx rope approach has the benefit that it does not create a ton of
intermediate buffers. We could do
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/18544
@stanzhai ping
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/22429
@rednaxelafx Thank you for such detailed comment.
> It would be better to pass the max fields argument down the call chain
instead of changing the Spark conf.
Sure, it would be
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22295#discussion_r218371105
--- Diff: python/pyspark/sql/tests.py ---
@@ -3654,6 +3654,107 @@ def test_jvm_default_session_already_set(self):
spark.stop()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22295#discussion_r218370828
--- Diff: python/pyspark/sql/tests.py ---
@@ -3654,6 +3654,107 @@ def test_jvm_default_session_already_set(self):
spark.stop()
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/20405
kindly ping
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22295#discussion_r218370450
--- Diff: python/pyspark/sql/tests.py ---
@@ -3654,6 +3654,107 @@ def test_jvm_default_session_already_set(self):
spark.stop()
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22440
Thank you for merging, @gatorsmile ! I like it.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user skonto commented on the issue:
https://github.com/apache/spark/pull/22381
@srowen integration tests seem so flaky! Seen this before?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22443
Thank you for confirmation, @cloud-fan .
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22375#discussion_r218362156
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelperSuite.scala
---
@@ -35,6 +36,13 @@ class
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/22448#discussion_r218360653
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
---
@@ -106,6 +108,22 @@ object TypeCoercion {
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/19773#discussion_r218358670
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -350,16 +366,11 @@ case class AlterTableChangeColumnCommand(
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/19773#discussion_r218356576
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -350,16 +366,11 @@ case class AlterTableChangeColumnCommand(
Github user zuotingbing closed the pull request at:
https://github.com/apache/spark/pull/20864
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20864
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20864
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20864
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
401 - 500 of 554 matches
Mail list logo
