Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19222
**[Test build #82083 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82083/testReport)**
for PR 19222 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19322
**[Test build #82082 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82082/testReport)**
for PR 19322 at commit
GitHub user wangyum opened a pull request:
https://github.com/apache/spark/pull/19322
[SPARK-22102][SQL] Set ConfVars.METASTOREWAREHOUSE before constructor
CliSessionState
## What changes were proposed in this pull request?
This PR set `ConfVars.METASTOREWAREHOUSE` before
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19290
**[Test build #82081 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82081/testReport)**
for PR 19290 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19307
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82076/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19307
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19307
**[Test build #82076 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82076/testReport)**
for PR 19307 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19319
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18945
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19319
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/19308
@ala Can you backport this one to 2.2?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19222
(I have been following this out of my curiosity but I think this should be
credit to @kiszk ...)
---
-
To unsubscribe,
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19308
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19319
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82079/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19319
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19319
**[Test build #82079 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82079/testReport)**
for PR 19319 at commit
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/19308
LGTM - merging to master. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19222#discussion_r140493443
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/array/ByteArrayMethods.java
---
@@ -46,6 +47,42 @@ public static int
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19222
**[Test build #82080 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82080/testReport)**
for PR 19222 at commit
Github user ConeyLiu commented on a diff in the pull request:
https://github.com/apache/spark/pull/19285#discussion_r140490824
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -354,63 +401,30 @@ private[spark] class MemoryStore(
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19308
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19308
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82077/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19308
**[Test build #82077 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82077/testReport)**
for PR 19308 at commit
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19319#discussion_r140488306
--- Diff: python/pyspark/sql/dataframe.py ---
@@ -37,6 +37,7 @@
from pyspark.sql.column import Column, _to_seq, _to_list, _to_java_column
from
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19319
Add `Closes #18945` in the PR description.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19319
**[Test build #82079 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82079/testReport)**
for PR 19319 at commit
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19319#discussion_r140488022
--- Diff: python/pyspark/sql/dataframe.py ---
@@ -1891,14 +1892,19 @@ def toPandas(self):
"if using
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19319#discussion_r140487687
--- Diff: python/pyspark/sql/dataframe.py ---
@@ -37,6 +37,7 @@
from pyspark.sql.column import Column, _to_seq, _to_list, _to_java_column
from
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19319#discussion_r140485612
--- Diff: python/pyspark/sql/dataframe.py ---
@@ -1891,14 +1892,19 @@ def toPandas(self):
"if using
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19319#discussion_r140482349
--- Diff: python/pyspark/sql/dataframe.py ---
@@ -37,6 +37,7 @@
from pyspark.sql.column import Column, _to_seq, _to_list, _to_java_column
from
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/19312
Merged to master, branch-2.2, and branch-2.1.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18805
**[Test build #3930 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3930/testReport)**
for PR 18805 at commit
Github user stanzhai commented on the issue:
https://github.com/apache/spark/pull/19301
@viirya
Benchmark code:
```scala
val N = 500L << 22
val benchmark = new Benchmark("agg", N)
val expressions = (0 until 50).map(i => s"sum(id) as r$i")
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19020
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82075/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19020
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19020
**[Test build #82075 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82075/testReport)**
for PR 19020 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19222
**[Test build #82078 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82078/testReport)**
for PR 19222 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19144
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19144
**[Test build #3931 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3931/testReport)**
for PR 19144 at commit
Github user sathiyapk commented on a diff in the pull request:
https://github.com/apache/spark/pull/19295#discussion_r140464377
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkOptimizer.scala ---
@@ -28,12 +28,18 @@ class SparkOptimizer(
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19312
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82071/
Test PASSed.
---
Github user ala commented on the issue:
https://github.com/apache/spark/pull/19308
@hvanhovell How about this?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19312
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19312
**[Test build #82071 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82071/testReport)**
for PR 19312 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19308
**[Test build #82077 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82077/testReport)**
for PR 19308 at commit
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/19222#discussion_r140462525
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/array/ByteArrayMethods.java
---
@@ -46,6 +47,42 @@ public static int
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19307
**[Test build #82076 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82076/testReport)**
for PR 19307 at commit
Github user ala commented on a diff in the pull request:
https://github.com/apache/spark/pull/19308#discussion_r140459738
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/OffHeapColumnVector.java
---
@@ -517,6 +517,7 @@ public void
Github user steveloughran commented on the issue:
https://github.com/apache/spark/pull/19294
As I play with commit logic all the way through the stack, I can' t help
thinking everyone's lives would be better if we tagged the MRv1 commit APIs as
deprecated in Hadoop 3. and uses of the
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18015
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18015
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82072/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18015
**[Test build #82072 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82072/testReport)**
for PR 18015 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19300
I'd help review other PRs and leave a comment when someone fixes some codes
around it rather than proposing it alone, or you can buffer it in your local
and flush it.
---
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/18015
And failed, who submitted the code in question.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19321
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82073/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19321
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19321
**[Test build #82073 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82073/testReport)**
for PR 19321 at commit
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19320
In JDBC / ODBC Server page, 'job ids' are separated by braces, as shown
below:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19144
**[Test build #3931 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3931/testReport)**
for PR 19144 at commit
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19144
If this isn't getting support, it's not worth pinging indefinitely, please.
That said, I'll merge this.
---
-
To unsubscribe,
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/19242#discussion_r140450417
--- Diff: docs/configuration.md ---
@@ -740,6 +740,20 @@ Apart from these, the following properties are also
available, and may be useful
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/19242#discussion_r140450356
--- Diff: docs/configuration.md ---
@@ -740,6 +740,20 @@ Apart from these, the following properties are also
available, and may be useful
Github user sathiyapk commented on a diff in the pull request:
https://github.com/apache/spark/pull/19295#discussion_r140450459
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkOptimizer.scala ---
@@ -28,12 +28,18 @@ class SparkOptimizer(
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19300
if we find some mistakes likes this , shouldn't we to report it at first?
---
-
To unsubscribe, e-mail:
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19320
I agree with making things consistent. The question is, I guess, what the
JDBC/ODBC page does. You can match that.
---
-
To
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19300
@HyukjinKwon will make this change in another PR. We do need to push back
on tiny PRs. It's a problem because some people are actually incentivized based
on number of patches and it costs the
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19242
@HyukjinKwon @jerryshao
Help to review the code, thanks.
---
-
To unsubscribe, e-mail:
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19144
@HyukjinKwon @jerryshao
Help to review the code, thanks.
---
-
To unsubscribe, e-mail:
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19300
@cloud-fan @gatorsmile Could you please help to review this and merge it
to master? Thanks.
---
-
To unsubscribe, e-mail:
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19320
Do you agree with my PR or against my PR?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user yanboliang commented on the issue:
https://github.com/apache/spark/pull/19020
@sethah To the issue that whether huber linear regression share codebase
with ```LinearRegression```, we have discussion at
[JIRA](https://issues.apache.org/jira/browse/SPARK-3181). At last
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19301
I asked it because by considering subexpressionElimination, you may not
actually run it multiple times. So the benchmark numbers can tell if your fix
really improves the performance.
---
Github user yanboliang commented on the issue:
https://github.com/apache/spark/pull/19020
@jkbradley Thanks for your comments, I have addressed all your inline
comments. Please see replies to your other questions below:
> Echoing @WeichenXu123 's comment: Why use "epsilon" as the
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19320
In JDBC / ODBC Server page.There are no other places yet.
---
-
To unsubscribe, e-mail:
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19320
I see, you're saying the display of a bunch of job IDs is like "[1] [2]
[3]" elsewhere? consistency is the most important thing IMHO. I'd just match
whatever a similar column does elsewhere.
---
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/19295#discussion_r140440857
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkOptimizer.scala ---
@@ -28,12 +28,18 @@ class SparkOptimizer(
Github user stanzhai commented on the issue:
https://github.com/apache/spark/pull/19301
@viirya The problem is already obvious, and the same aggregate expression
will be computed multi times. I will provide a benchmark result later.
---
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/19295
why can't user just use `preOptimizationBatches`?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19320
Do you want to get rid of braces?
Then the JDBC / ODBC Server page also needs to remove the braces. But I
think there is a better distinction between brackets This is the job id.
Github user zhengruifeng commented on the issue:
https://github.com/apache/spark/pull/19229
I am not familiar with SQL source, but I think it's great to transform all
columns at a time
---
-
To unsubscribe, e-mail:
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/19020#discussion_r140440076
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala
---
@@ -998,6 +1047,172 @@ class LinearRegressionSuite
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/19020#discussion_r140439435
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -220,12 +283,12 @@ class LinearRegression @Since("1.3.0")
Github user ueshin closed the pull request at:
https://github.com/apache/spark/pull/19147
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/19020#discussion_r140439369
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -69,19 +69,57 @@ private[regression] trait
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/19147
I'd close this in favor of #18659.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19020
**[Test build #82075 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82075/testReport)**
for PR 19020 at commit
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/19020#discussion_r140439119
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/optim/aggregator/HuberAggregator.scala
---
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/19020#discussion_r140439171
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -69,19 +69,57 @@ private[regression] trait
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/19020#discussion_r140439140
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/optim/aggregator/HuberAggregator.scala
---
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19320
Why braces? just seems like noise
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18659
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/19320
I have fixed it.
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/18659
LGTM, merging to master!
We can address remaining minor comments in follow-up, and have new PRs to
remove the 0-parameter UDF and use arrow streaming protocol.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19302
**[Test build #82074 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82074/testReport)**
for PR 19302 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19302
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82074/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19302
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19302
**[Test build #82074 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82074/testReport)**
for PR 19302 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19302
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19301
Regarding performance regression, I think you should post benchmark numbers.
---
-
To unsubscribe, e-mail:
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/19302
@srowen sure, I will open a new JIRA for this.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
201 - 300 of 362 matches
Mail list logo
