Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21912
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21912
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94938/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21912
**[Test build #94938 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94938/testReport)**
for PR 21912 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22143
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22143
**[Test build #94941 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94941/testReport)**
for PR 22143 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22143
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94941/
Test PASSed.
---
Github user steveloughran commented on the issue:
https://github.com/apache/spark/pull/17342
At a guess, there's possibly a mix here between hadoop hdfs JARs on your
classpath. You sure everything on the classpath is in sync? What JARs with
hadoop-hdfs are there?
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22143
**[Test build #94941 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94941/testReport)**
for PR 22143 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22085
**[Test build #94942 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94942/testReport)**
for PR 22085 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22085
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22085
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user koertkuipers commented on the issue:
https://github.com/apache/spark/pull/21273
@HyukjinKwon see the jira for the example code that reproduces the issue.
let me know if you need anything else. best, koert
---
Github user sddyljsx commented on the issue:
https://github.com/apache/spark/pull/21859
We may not know in advance how big this query is. The data at the beginning
is large, but it may be very small after filtering.
I encountered this problem while using thrift server for queries.
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22148#discussion_r211136238
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadSupport.scala
---
@@ -277,14 +291,38 @@
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21273
@koertkuipers, would you mind if I ask provide a reproducer please?
---
-
To unsubscribe, e-mail:
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/21665
Ping @uzmijnlm
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user koertkuipers commented on the issue:
https://github.com/apache/spark/pull/21273
to summarize my findings from jira:
this breaks any usage without quoting. for example we remove all characters
from our values that need to be quoted (delimiters, newlines) so we know we
Github user yucai commented on the issue:
https://github.com/apache/spark/pull/22148
LGTM.
@cloud-fan @gatorsmile Could you kindly help trigger Jenkins and review?
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22148
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21665
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22090#discussion_r211133578
--- Diff: docs/mllib-evaluation-metrics.md ---
@@ -462,13 +462,13 @@ $$rel_D(r) = \begin{cases}1 & \text{if $r \in D$}, \\
0 & \text{otherwise}.\end{
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22148
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22148
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user seancxmao opened a pull request:
https://github.com/apache/spark/pull/22148
[SPARK-25132][SQL] Case-insensitive field resolution when reading from
Parquet
## What changes were proposed in this pull request?
Spark SQL returns NULL for a column whose Hive metastore
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20637#discussion_r211132711
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelperSuite.scala
---
@@ -35,6 +35,24 @@ class
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20637#discussion_r211131717
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala
---
@@ -43,25 +45,30 @@ object
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20637#discussion_r211132393
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala
---
@@ -43,25 +45,30 @@ object
Github user seancxmao commented on the issue:
https://github.com/apache/spark/pull/22142
Split this into 2 PRs, one for Parquet and ORC respectively.
---
-
To unsubscribe, e-mail:
Github user seancxmao closed the pull request at:
https://github.com/apache/spark/pull/22142
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22133
**[Test build #94940 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94940/testReport)**
for PR 22133 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22133
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22133
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22124
@wangyum I know it's from #20020, but do you know which line of the
code/which method cause it? We must fully understand the bug before fixing it.
---
Github user yueguoguo commented on the issue:
https://github.com/apache/spark/pull/22090
@srowen Thanks Sean. Good suggestion and I have pushed new commits.
---
-
To unsubscribe, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21859
for small queries, can we just do
```
val df = table.filter(...).cache()
df.sort()
```
We should carefully make trade off between the SQL engine complexity and
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22123#discussion_r211132019
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala
---
@@ -230,7 +232,7 @@ object
Github user yueguoguo closed the pull request at:
https://github.com/apache/spark/pull/22147
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
GitHub user yueguoguo opened a pull request:
https://github.com/apache/spark/pull/22147
Fixed NDCG formula and link
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
## How was this patch tested?
(Please
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94939 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94939/testReport)**
for PR 21859 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22144
This should not be a global config but per-UDAF flag. IIRC we do have such
a flag before, but get removed later. Maybe we should bring it back.
---
Github user sddyljsx commented on a diff in the pull request:
https://github.com/apache/spark/pull/21859#discussion_r211131380
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -1207,6 +1207,13 @@ object SQLConf {
.intConf
Github user sddyljsx commented on a diff in the pull request:
https://github.com/apache/spark/pull/21859#discussion_r211131294
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -1207,6 +1207,13 @@ object SQLConf {
.intConf
Github user sddyljsx commented on a diff in the pull request:
https://github.com/apache/spark/pull/21859#discussion_r211130877
--- Diff: core/src/main/scala/org/apache/spark/Partitioner.scala ---
@@ -155,6 +156,8 @@ class RangePartitioner[K : Ordering : ClassTag, V](
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21909#discussion_r211129911
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FailureSafeParser.scala
---
@@ -56,9 +58,15 @@ class
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22121#discussion_r211128707
--- Diff: docs/avro-data-source-guide.md ---
@@ -0,0 +1,260 @@
+---
+layout: global
+title: Apache Avro Data Source Guide
+---
+
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22121#discussion_r211127696
--- Diff: docs/avro-data-source-guide.md ---
@@ -0,0 +1,260 @@
+---
+layout: global
+title: Apache Avro Data Source Guide
+---
+
Github user ajithme closed the pull request at:
https://github.com/apache/spark/pull/22120
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21912#discussion_r211126450
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala
---
@@ -735,70 +735,98 @@ class CodegenContext {
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22121#discussion_r211126239
--- Diff: docs/avro-data-source-guide.md ---
@@ -0,0 +1,260 @@
+---
+layout: global
+title: Apache Avro Data Source Guide
+---
+
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21912
**[Test build #94938 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94938/testReport)**
for PR 21912 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21912
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21912
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22146
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22146
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22146
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user onursatici opened a pull request:
https://github.com/apache/spark/pull/22146
[WIP][SPARK-24434][K8S] pod template files
## What changes were proposed in this pull request?
New feature to pass podspec files for driver and executor pods.
## How was this
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21859#discussion_r211122674
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -1207,6 +1207,13 @@ object SQLConf {
.intConf
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21859#discussion_r211122076
--- Diff: core/src/main/scala/org/apache/spark/Partitioner.scala ---
@@ -155,6 +156,8 @@ class RangePartitioner[K : Ordering : ClassTag, V](
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/spark/pull/22138#discussion_r29914
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala
---
@@ -425,70 +381,36 @@ private[kafka010]
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/22138
@koeninger
I'm not sure I got your point correctly. This patch is based on some
assumptions, so please correct me if I'm missing here. Assumptions follow:
1. There's actually no
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22143
**[Test build #94937 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94937/testReport)**
for PR 22143 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22143
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22143
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94937/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22143
**[Test build #94937 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94937/testReport)**
for PR 22143 at commit
Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/22143
Jenkins, ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/22138
If you have multiple consumers for a given key, and those consumers are at
different offsets, isn't it likely that the client code will not get the right
consumer, leading to extra seeking?
---
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21912
cc @ueshin
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22145
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22145
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94936/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22145
**[Test build #94936 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94936/testReport)**
for PR 22145 at commit
Github user ifilonenko commented on the issue:
https://github.com/apache/spark/pull/22145
This PR should fail integration tests rn, until the Jenkins OS is updated,
but error right now is in terms of the minikube environment:
`Error creating VM: virError(Code=55, Domain=19,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22145
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22145
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2308/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22145
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22145
Kubernetes integration test status failure
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2308/
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22145
**[Test build #94936 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94936/testReport)**
for PR 22145 at commit
GitHub user ifilonenko opened a pull request:
https://github.com/apache/spark/pull/22145
[SPARK-25152][K8S] Enable SparkR Integration Tests for Kubernetes
## What changes were proposed in this pull request?
Re-introduced SparkR integration tests as part of the SparkR on
Github user pgandhi999 commented on the issue:
https://github.com/apache/spark/pull/22144
@dilipbiswal This property is by default set to true so it does not effect
anything currently in the way UDAF's run. This property has been added purely
for the purpose of maintaining backward
Github user dilipbiswal commented on the issue:
https://github.com/apache/spark/pull/22144
@pgandhi999 I have a basic question. Setting this property will have a
global effect on all the aggregations ?
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22144
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22144
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22144
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user pgandhi999 opened a pull request:
https://github.com/apache/spark/pull/22144
[SPARK-24935] : Problem with Executing Hive UDF's from Spark 2.2 Onwards
A user of sketches library(https://github.com/DataSketches/sketches-hive)
reported an issue with HLL Sketch Hive UDAF
Github user vackosar commented on the issue:
https://github.com/apache/spark/pull/22143
@arunmahadevan @jose-torres @cloud-fan you may interested in this one.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22143
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22143
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22143
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user vackosar opened a pull request:
https://github.com/apache/spark/pull/22143
[SPARK-24647][SS] Report KafkaStreamWriter's written min and max offsâ¦
â¦ets via CustomMetrics.
## What changes were proposed in this pull request?
Report KafkaStreamWriter's
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22123
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94935/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22123
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22123
**[Test build #94935 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94935/testReport)**
for PR 22123 at commit
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22124
Hi @wangyum , thanks for working on this.
Can you simplify the reproducing case? E.g. Select only one column should
be enough.
Also, in the PR description, somehow there are column
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22121
@srowen Hi Sean, I will add content for new features soon. I also updated
the title.
Thanks.
---
-
To unsubscribe,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22142
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22142
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22142
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user seancxmao opened a pull request:
https://github.com/apache/spark/pull/22142
[SPARK-25132][SQL] case-insensitive field resolution when reading from
Parquet/ORC
## What changes were proposed in this pull request?
Spark SQL returns NULL for a column whose Hive
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21912
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94934/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21912
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21912
**[Test build #94934 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94934/testReport)**
for PR 21912 at commit
1 - 100 of 131 matches
Mail list logo
