Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19820
Would it be possible to add which tests caused this exception in the
description?
---
-
To unsubscribe, e-mail: reviews-unsubscr..
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19820
LGTM except one comment.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: review
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19814#discussion_r153062402
--- Diff: python/pyspark/sql/readwriter.py ---
@@ -828,8 +828,7 @@ def csv(self, path, mode=None, compression=None,
sep=None, quote=None, escape=No
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19814
Can we turn off it as documented? We could try to open a JIRA in Univocity
if this functionality is not there and incorporate the change in Spark.
---
-
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19815
LGTM except one comment
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/19815#discussion_r153061379
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/WritableColumnVector.java
---
@@ -324,24 +315,27 @@ public void putDecimal(int row
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/19817#discussion_r153060800
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -1372,19 +1372,30 @@ case class FormatString(c
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18692#discussion_r153060560
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +152,99 @@ object EliminateOuterJoin extends Rul
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18692#discussion_r153060551
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +152,99 @@ object EliminateOuterJoin extends Rul
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18692#discussion_r153060595
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +152,99 @@ object EliminateOuterJoin extends Rul
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19817#discussion_r153060489
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -1372,19 +1372,30 @@ case class FormatStr
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19815#discussion_r153060413
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/WritableColumnVector.java
---
@@ -324,24 +315,27 @@ public void putDecimal(int ro
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19815#discussion_r153060336
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/WritableColumnVector.java
---
@@ -324,24 +315,27 @@ public void putDecimal(int ro
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19816
Let me maybe try to deal with this separately ..
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For addi
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19816
I tested 16, 32 and 64 too:
16:
```
Failed
-
1. Failure: group by, agg functions (@test_sparkSQL.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19783
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84190/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19783
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19783
**[Test build #84190 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84190/testReport)**
for PR 19783 at commit
[`8e5d04e`](https://github.com/apache/spark/commit/8
Github user yucai commented on the issue:
https://github.com/apache/spark/pull/19788
Currently users need update their external shuffle service for this
feature, because we change the format of `ShuffleBlockId`, which is supposed to
be parsed by external shuffle service.
I am tryi
Github user yucai commented on a diff in the pull request:
https://github.com/apache/spark/pull/19788#discussion_r153057489
--- Diff:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolver.java
---
@@ -165,13 +165,23 @@ public ManagedBuf
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19816
Ah, makes sense. Let me at least try other numbers and be back soon anyway.
---
-
To unsubscribe, e-mail: reviews-unsubscr...
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19783
**[Test build #84190 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84190/testReport)**
for PR 19783 at commit
[`8e5d04e`](https://github.com/apache/spark/commit/8e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19754
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19754
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84189/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19754
**[Test build #84189 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84189/testReport)**
for PR 19754 at commit
[`71ec15b`](https://github.com/apache/spark/commit/7
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19754
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84188/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19754
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19754
**[Test build #84188 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84188/testReport)**
for PR 19754 at commit
[`41f4493`](https://github.com/apache/spark/commit/4
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19820
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19820
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84187/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19820
**[Test build #84187 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84187/testReport)**
for PR 19820 at commit
[`a0b6658`](https://github.com/apache/spark/commit/a
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19811
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19811
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84186/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19811
**[Test build #84186 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84186/testReport)**
for PR 19811 at commit
[`d01fcb1`](https://github.com/apache/spark/commit/d
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/19754#discussion_r153052719
--- Diff: dev/create-release/release-build.sh ---
@@ -392,6 +375,7 @@ if [[ "$1" == "publish-release" ]]; then
find . -type f |grep -v \.jar |grep
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/19816
I think likely this is a general problem with default shuffle size being
too big on limited resources when in tests (particularly on Windows/AppVeyor).
The additional test failures are expected,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19754
**[Test build #84189 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84189/testReport)**
for PR 19754 at commit
[`71ec15b`](https://github.com/apache/spark/commit/71
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19754
**[Test build #84188 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84188/testReport)**
for PR 19754 at commit
[`41f4493`](https://github.com/apache/spark/commit/41
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19820
**[Test build #84187 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84187/testReport)**
for PR 19820 at commit
[`a0b6658`](https://github.com/apache/spark/commit/a0
GitHub user srowen opened a pull request:
https://github.com/apache/spark/pull/19820
[SPARK-22607][BUILD] Set large stack size consistently for tests to avoid
StackOverflowError
## What changes were proposed in this pull request?
Set `-ea` and `-Xss4m` consistently for test
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19815
I will look this Sunday.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: review
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19811
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19811
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84185/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19811
**[Test build #84185 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84185/testReport)**
for PR 19811 at commit
[`d8a9f9e`](https://github.com/apache/spark/commit/d
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19811
**[Test build #84186 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84186/testReport)**
for PR 19811 at commit
[`d01fcb1`](https://github.com/apache/spark/commit/d0
Github user yucai commented on a diff in the pull request:
https://github.com/apache/spark/pull/19788#discussion_r153049711
--- Diff:
common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolverSuite.java
---
@@ -110,6 +110,13 @@ public void t
Github user yucai commented on a diff in the pull request:
https://github.com/apache/spark/pull/19788#discussion_r153049707
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockId.scala ---
@@ -52,8 +52,9 @@ case class RDDBlockId(rddId: Int, splitIndex: Int)
extends BlockId
Github user yucai commented on a diff in the pull request:
https://github.com/apache/spark/pull/19788#discussion_r153049650
--- Diff:
core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala ---
@@ -196,12 +196,14 @@ private[spark] class IndexShuffleBlockResolve
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19811
**[Test build #84185 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84185/testReport)**
for PR 19811 at commit
[`d8a9f9e`](https://github.com/apache/spark/commit/d8
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19798
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19798
Merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19518
> I think ldc is 2 bytes and ldc_w is 3 bytes?
You are right, thanks, updated.
---
-
To unsubscribe, e-mail: reviews-unsubscr..
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19816
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19816
**[Test build #84184 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84184/testReport)**
for PR 19816 at commit
[`41ffb54`](https://github.com/apache/spark/commit/4
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19816
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84184/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19819
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84183/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19819
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19819
**[Test build #84183 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84183/testReport)**
for PR 19819 at commit
[`aa02d89`](https://github.com/apache/spark/commit/a
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/18906
Thanks for the background Bryan :) So it sounds like from an API
perspective it makes sense to support this in the future possibly on the Pandas
UDFs (but the code isn't there on the JVM side). I'd
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19816
**[Test build #84184 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84184/testReport)**
for PR 19816 at commit
[`41ffb54`](https://github.com/apache/spark/commit/41
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19816
Not sure. Let me know if you have a preference @felixcheung.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.o
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19816
@felixcheung, I just tried to lower this by default and ran. Seems some
tests are being failed. For example, if we lower`spark.sql.shuffle.partitions`
to 5, these fail additionally:
```
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19819
**[Test build #84183 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84183/testReport)**
for PR 19819 at commit
[`aa02d89`](https://github.com/apache/spark/commit/aa
GitHub user eatoncys opened a pull request:
https://github.com/apache/spark/pull/19819
[SPARK-22606][Streaming]Add threadId to the CachedKafkaConsumer key
## What changes were proposed in this pull request?
If the value of param 'spark.streaming.concurrentJobs' is more than one,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19813
**[Test build #84182 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84182/testReport)**
for PR 19813 at commit
[`65d07d5`](https://github.com/apache/spark/commit/6
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19813
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84182/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19813
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19813
> However for whole stage codegen path, it's way more complex:
> 1. some of ctx.currentVars are just variables, their codes have already
been generated before. But some are not. For those whose co
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19813
**[Test build #84182 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84182/testReport)**
for PR 19813 at commit
[`65d07d5`](https://github.com/apache/spark/commit/65
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19813#discussion_r153040576
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala
---
@@ -236,4 +237,22 @@ class WholeStageCodegenSuite extends
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/19817
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apac
71 matches
Mail list logo
