Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/19204#discussion_r139312034
--- Diff: python/pyspark/ml/evaluation.py ---
@@ -328,6 +329,86 @@ def setParams(self, predictionCol="prediction",
labelCol="label",
kwargs
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/19204#discussion_r139312046
--- Diff: python/pyspark/ml/evaluation.py ---
@@ -328,6 +329,86 @@ def setParams(self, predictionCol="prediction",
labelCol="label",
kwargs
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/16578#discussion_r139312181
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala
---
@@ -0,0 +1,130 @@
+/*
+ *
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19254
I think it's a problem with a recent change, yeah. Just wait until it can
be resolved then we can retest
---
-
To unsubscribe,
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/16578#discussion_r139311738
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala
---
@@ -0,0 +1,130 @@
+/*
+ *
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/16578#discussion_r139311726
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala
---
@@ -0,0 +1,130 @@
+/*
+ *
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/16578#discussion_r139311391
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/SelectedField.scala
---
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19254
Seems to be broken on Jenkins `master` too
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test/job/spark-master-test-sbt-hadoop-2.7/3490/
---
Github user original-brownbear commented on the issue:
https://github.com/apache/spark/pull/19254
Test failure looks unrelated (even though it happened twice in a row now):
```sh
[info] org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite ***
ABORTED *** (1 second,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19254
**[Test build #3924 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3924/testReport)**
for PR 19254 at commit
Github user ConeyLiu commented on a diff in the pull request:
https://github.com/apache/spark/pull/19135#discussion_r139309495
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -190,11 +190,11 @@ private[spark] class MemoryStore(
//
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19261#discussion_r139309404
--- Diff: python/pyspark/sql/functions.py ---
@@ -793,12 +793,12 @@ def ntile(n):
# -- Date/Timestamp functions
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19261#discussion_r139309376
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -2508,6 +2508,14 @@ object functions {
def current_date(): Column =
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19261#discussion_r139309394
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -2508,6 +2508,14 @@ object functions {
def current_date(): Column =
Github user jaceklaskowski commented on a diff in the pull request:
https://github.com/apache/spark/pull/19261#discussion_r139309272
--- Diff: python/pyspark/sql/functions.py ---
@@ -793,12 +793,12 @@ def ntile(n):
# -- Date/Timestamp functions
Github user jaceklaskowski commented on a diff in the pull request:
https://github.com/apache/spark/pull/19261#discussion_r139309246
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -2508,6 +2508,14 @@ object functions {
def current_date():
Github user jaceklaskowski commented on a diff in the pull request:
https://github.com/apache/spark/pull/19261#discussion_r139309261
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -2508,6 +2508,14 @@ object functions {
def current_date():
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19261
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user mgaido91 opened a pull request:
https://github.com/apache/spark/pull/19261
[SPARK-22040] Add current_date function with timezone id
## What changes were proposed in this pull request?
Add current_date function with timezone id.
## How was this patch
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19260
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81855/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19260
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19260
**[Test build #81855 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81855/testReport)**
for PR 19260 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19260
**[Test build #81855 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81855/testReport)**
for PR 19260 at commit
GitHub user HyukjinKwon opened a pull request:
https://github.com/apache/spark/pull/19260
[SPARK-22043][PYTHON] Improves error message for show_profiles and
dump_profiles
## What changes were proposed in this pull request?
This PR proposes to improve error message from:
Github user original-brownbear commented on a diff in the pull request:
https://github.com/apache/spark/pull/19254#discussion_r139307959
--- Diff:
core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java ---
@@ -403,6 +391,16 @@ public long spill(long numBytes) throws
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19254
**[Test build #3924 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3924/testReport)**
for PR 19254 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19259
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19259
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81851/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19259
**[Test build #81851 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81851/testReport)**
for PR 19259 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19254
**[Test build #3923 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3923/testReport)**
for PR 19254 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19249
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19249
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81854/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19249
**[Test build #81854 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81854/testReport)**
for PR 19249 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19249
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19249
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81853/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19249
**[Test build #81853 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81853/testReport)**
for PR 19249 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19249
**[Test build #81854 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81854/testReport)**
for PR 19249 at commit
Github user maver1ck commented on the issue:
https://github.com/apache/spark/pull/19249
Done.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19249
LGTM too but hey @maver1ck could you add some comments around the codes and
move the worst case benchmarks into the PR description? I guess this wouldn't
be too demanding.
---
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19249
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19249
**[Test build #81853 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81853/testReport)**
for PR 19249 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19249
**[Test build #81852 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81852/testReport)**
for PR 19249 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19249
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81852/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19249
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19249
**[Test build #81852 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81852/testReport)**
for PR 19249 at commit
Github user maver1ck commented on a diff in the pull request:
https://github.com/apache/spark/pull/19249#discussion_r139306166
--- Diff: python/pyspark/sql/types.py ---
@@ -619,7 +621,8 @@ def fromInternal(self, obj):
# it's already converted by pickler
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19249#discussion_r139306038
--- Diff: python/pyspark/sql/types.py ---
@@ -619,7 +621,8 @@ def fromInternal(self, obj):
# it's already converted by pickler
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19249#discussion_r139305954
--- Diff: python/pyspark/sql/types.py ---
@@ -619,7 +621,8 @@ def fromInternal(self, obj):
# it's already converted by pickler
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19255#discussion_r139305858
--- Diff: python/pyspark/sql/types.py ---
@@ -24,6 +24,7 @@
import re
import base64
from array import array
+from functools import
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19249#discussion_r139305674
--- Diff: python/pyspark/sql/types.py ---
@@ -483,7 +483,8 @@ def __init__(self, fields=None):
self.names = [f.name for f in fields]
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19249#discussion_r139305834
--- Diff: python/pyspark/sql/types.py ---
@@ -619,7 +621,8 @@ def fromInternal(self, obj):
# it's already converted by pickler
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19230#discussion_r139305746
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnVector.java
---
@@ -16,6 +16,7 @@
*/
package
Github user original-brownbear commented on a diff in the pull request:
https://github.com/apache/spark/pull/19254#discussion_r139305653
--- Diff:
core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java ---
@@ -403,6 +391,16 @@ public long spill(long numBytes) throws
Github user original-brownbear commented on a diff in the pull request:
https://github.com/apache/spark/pull/19254#discussion_r139305647
--- Diff: core/src/main/java/org/apache/spark/memory/MemoryConsumer.java ---
@@ -116,13 +110,7 @@ public void freeArray(LongArray array) {
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19232
Yeah I wonder if this slows things down for smaller data sets, because of
the extra levels and checks, but then again, when the aggregation is small,
anything's similarly fast. The default depth is
Github user maver1ck commented on a diff in the pull request:
https://github.com/apache/spark/pull/19255#discussion_r139305244
--- Diff: python/pyspark/sql/types.py ---
@@ -24,6 +24,7 @@
import re
import base64
from array import array
+from functools import
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19254
**[Test build #3923 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3923/testReport)**
for PR 19254 at commit
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/19254#discussion_r139304891
--- Diff:
core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java ---
@@ -403,6 +391,16 @@ public long spill(long numBytes) throws IOException {
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/19254#discussion_r139304877
--- Diff: core/src/main/java/org/apache/spark/memory/MemoryConsumer.java ---
@@ -116,13 +110,7 @@ public void freeArray(LongArray array) {
protected
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19259
**[Test build #81851 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81851/testReport)**
for PR 19259 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19230
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81850/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19230
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19230
**[Test build #81850 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81850/testReport)**
for PR 19230 at commit
GitHub user wangyum opened a pull request:
https://github.com/apache/spark/pull/19259
[BACKPORT-2.1][SPARK-19318][SPARK-22041][SQL] Docker test case failure:
`SPARK-16625: General data types to be mapped to Oracle`
⦠options in case-sensitive manner.
## What changes were
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19258
Close this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user maver1ck commented on a diff in the pull request:
https://github.com/apache/spark/pull/19249#discussion_r139303509
--- Diff: python/pyspark/sql/types.py ---
@@ -619,7 +621,8 @@ def fromInternal(self, obj):
# it's already converted by pickler
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19256
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81848/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16600
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81849/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19256
**[Test build #81848 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81848/testReport)**
for PR 19256 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19256
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16600
Build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19230
**[Test build #81850 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81850/testReport)**
for PR 19230 at commit
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19256#discussion_r139302993
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/jdbc/AggregatedDialect.scala ---
@@ -41,4 +41,8 @@ private class AggregatedDialect(dialects:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19256
**[Test build #81848 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81848/testReport)**
for PR 19256 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16600
**[Test build #81849 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81849/testReport)**
for PR 16600 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19256
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user gengliangwang closed the pull request at:
https://github.com/apache/spark/pull/19253
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/19253
@gatorsmile @viirya Thanks. I should close this PR.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19218
@fjh100456 We have priority for three different inputs. Here, you just
consider one of three. Please also add the extra checks. Hopefully,
@dongjoon-hyun can help you answer your questions. He
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19218#discussion_r139302763
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala
---
@@ -101,6 +101,19 @@ case class
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19218
I see. If you set `spark.sql.hive.convertMetastoreParquet` to false, you
will also hit the issue for non-partitioned table.
---
101 - 181 of 181 matches
Mail list logo
