Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20455
**[Test build #86921 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86921/testReport)**
for PR 20455 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20455
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20455
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/470/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20466
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86912/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20466
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20466
**[Test build #86912 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86912/testReport)**
for PR 20466 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20469
**[Test build #86920 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86920/testReport)**
for PR 20469 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20469
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/469/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20469
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user yaooqinn opened a pull request:
https://github.com/apache/spark/pull/20469
[SPARK-23295][Build][Minor]Exclude Waring message when generating versions
in make-distribution.sh
## What changes were proposed in this pull request?
When we specified a wrong profile
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20400
**[Test build #86919 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86919/testReport)**
for PR 20400 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20455
**[Test build #86918 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86918/testReport)**
for PR 20455 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20400
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/468/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20400
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20455
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/467/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20455
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20455
Since the map support is added, I'll do related change later.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user liufengdb commented on the issue:
https://github.com/apache/spark/pull/17886
@gatorsmile this is a great patch. The test can be improved, but I think it
is safe to merge as it.
---
-
To unsubscribe,
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/20465
Yes, the tests are being run with python3. I do prefer to have these
conditional skips removed because sometimes it is hard to tell if everything
passed or was just skipped. But since pandas
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20461
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/466/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20461
**[Test build #86917 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86917/testReport)**
for PR 20461 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20461
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/20461#discussion_r165276461
--- Diff:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/OneForOneBlockFetcher.java
---
@@ -171,7 +171,9 @@ private void
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/20468
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/20465
So, jenkins jobs run those tests with python3? If so, I feel better because
those tests are not completely skipped in Jenkins. If it is hard to make them
run with python 2. Letâs have a log to
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20422
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20422
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86907/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20422
**[Test build #86907 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86907/testReport)**
for PR 20422 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20465
Yup, there was a related discussion already. See this
https://github.com/apache/spark/pull/19884#issuecomment-351916074 and
https://github.com/apache/spark/pull/19884#issuecomment-353068446. We
Github user liufengdb commented on the issue:
https://github.com/apache/spark/pull/19219
The major issue this PR tries to cover has been fixed by
https://github.com/apache/spark/pull/20029, so I think we are good if there are
no calls to `HiveClientImpl.newSession`. We can close this
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/20465
Looking back at when pyarrow was last upgraded in #19884, pandas and
pyarrow were upgraded on all workers for python 3, but there were maybe some
concerns or difficulties with upgrading for
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20468
thanks! LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20468
**[Test build #86916 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86916/testReport)**
for PR 20468 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20468
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20468
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/465/
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/20468
cc @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
GitHub user ueshin opened a pull request:
https://github.com/apache/spark/pull/20468
[SPARK-23280][SQL][FOLLOWUP] Fix Java style check issues.
## What changes were proposed in this pull request?
This is a follow-up of #20450 which broke lint-java checks.
This pr fixes
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/20464#discussion_r165271143
--- Diff: R/pkg/R/column.R ---
@@ -169,7 +169,7 @@ setMethod("alias",
#' @note substr since 1.4.0
setMethod("substr", signature(x = "Column"),
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20422
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86906/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20422
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user zhengruifeng commented on the issue:
https://github.com/apache/spark/pull/20164
@srowen Different from the base model (like LoR), OVR and OVRModel do not
have param `rawPredictionCol`.
So if the input dataframe contains a column which has the same name as base
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20422
**[Test build #86906 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86906/testReport)**
for PR 20422 at commit
Github user huaxingao commented on a diff in the pull request:
https://github.com/apache/spark/pull/20400#discussion_r165270774
--- Diff: python/pyspark/sql/window.py ---
@@ -129,11 +131,34 @@ def rangeBetween(start, end):
:param end: boundary end, inclusive.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20422
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86905/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20422
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20422
**[Test build #86905 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86905/testReport)**
for PR 20422 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20467
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20467
Thanks, @gatorsmile.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20467
Fine. I just merged it
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20467
Sorry, actually I am hitting a network problem. Let me try it latter if
it's merged.
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20467
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19872#discussion_r165268323
--- Diff: python/pyspark/sql/tests.py ---
@@ -4353,6 +4347,446 @@ def test_unsupported_types(self):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20462
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20460
I'd target this 2.3 & master. Waiting for tests
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20460
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/462/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20462
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86904/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20462
**[Test build #86904 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86904/testReport)**
for PR 20462 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20460
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/462/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20460
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20460
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/464/
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20465
```
ImportError: Pandas >= 0.19.2 must be installed on calling Python process;
however, your version was 0.16.0.
```
I guess the RISELab boxes will need some updates...
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20465
sure, that's ok, I think we can revisit later (ie. next release) if we want
to add an env switch or something to make them optional
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20467
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86913/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20467
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20467
**[Test build #86913 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86913/testReport)**
for PR 20467 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20465
> I think there are some values in having a way to run python tests
without Arrow?
I agree, but the more important thing is to make sure jenkins runs
everything, so that we can be
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20460
**[Test build #86915 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86915/testReport)**
for PR 20460 at commit
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20460
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20454
**[Test build #86914 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86914/testReport)**
for PR 20454 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20454
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/463/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20454
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/18933
Ping. I ran into this exact issue with pandas_udf on a simple data set with
a timestamp type column.
As far as I can tell, there is no way to around this since pandas code is
running deep
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/20465
@felixcheung jenkins is actually skipping those tests (see the failure of
this pr). It makes sense to provide a way to allow developers to not run those
tests. But, I'd prefer that we run those tests
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20465
hmm, I think there are some values in having a way to run python tests
without Arrow? I mean the test.py is not just for Jenkins but for everyone
consuming the Spark release... unless we are
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20383
I agree. sorry to merge it so quickly, let me revert it.
@ssaavedra would you please submit PR again when everything is done, thanks!
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20383
my take is not that it doesn't work but some names are out of date because
it was done for the k8s fork.
I think we should revert the commit and wait till it is tested out
complete. WDYT?
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/20464#discussion_r165263961
--- Diff: R/pkg/R/column.R ---
@@ -169,7 +169,7 @@ setMethod("alias",
#' @note substr since 1.4.0
setMethod("substr", signature(x = "Column"),
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20467
**[Test build #86913 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86913/testReport)**
for PR 20467 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20457
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86909/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20457
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20467
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/462/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20457
**[Test build #86909 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86909/testReport)**
for PR 20457 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20467
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19872#discussion_r165262989
--- Diff: python/pyspark/sql/tests.py ---
@@ -4353,6 +4347,446 @@ def test_unsupported_types(self):
df.groupby('id').apply(f).collect()
GitHub user ueshin opened a pull request:
https://github.com/apache/spark/pull/20467
[SPARK-22274][PYTHON][SQL][FOLLOWUP] Use `assertRaisesRegexp` instead of
`assertRaisesRegex`.
## What changes were proposed in this pull request?
This is a follow-up pr of #19872 which
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20464
> One followup question is though, would it be difficult to match the
behaviour with substr in R when the index is 0 or minus? If i understood #20464
(comment) correctly, it sounds better to match
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/20424#discussion_r165261830
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala ---
@@ -191,7 +191,20 @@ private[spark] class
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19872#discussion_r165261550
--- Diff: python/pyspark/sql/tests.py ---
@@ -4353,6 +4347,446 @@ def test_unsupported_types(self):
df.groupby('id').apply(f).collect()
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19219
cc @liufengdb
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20466
**[Test build #86912 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86912/testReport)**
for PR 20466 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20466
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/461/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20465
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20466
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20465
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86910/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20465
**[Test build #86910 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86910/testReport)**
for PR 20465 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20466
cc @gatorsmile @rdblue @sameeragarwal
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/20466
[SPARK-23293][SQL] fix data source v2 self join
## What changes were proposed in this pull request?
`DataSourceV2Relation` should extend `MultiInstanceRelation`, to take care
of
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19219
**[Test build #86911 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86911/testReport)**
for PR 19219 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20463
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20361
Not a bug fix. This is not qualified for merging to Spark 2.3
---
-
To unsubscribe, e-mail:
1 - 100 of 624 matches
Mail list logo
