Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21698
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21698
**[Test build #92526 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92526/testReport)**
for PR 21698 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21697
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/619/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21698
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21697
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21698
**[Test build #92526 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92526/testReport)**
for PR 21698 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21698
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/620/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21697
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/619/
---
Github user sidhavratha commented on the issue:
https://github.com/apache/spark/pull/21685
Thanks a lot for looking into this. Please find comments in [] below every
points.
- You're trying to commit something into 2.4 but in the test result I see
with 2.1.0 version. Have
GitHub user jiangxb1987 opened a pull request:
https://github.com/apache/spark/pull/21698
[SPARK-23243] Fix RDD.repartition() data correctness issue
## What changes were proposed in this pull request?
The RDD repartition uses a round-robin way to distribute data, thus there
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21657
@HyukjinKwon WDYT?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21697
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/619/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21697
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/618/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21697
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21697
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/618/
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21697
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/618/
---
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21685
In general `KafkaConsumer.poll` should take couple of seconds but 10+ is
extreme high. The question `why it takes so long?` has to be answered first. In
the processing time chart I see a
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21403
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21403
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/617/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21696
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/616/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21403
**[Test build #92524 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92524/testReport)**
for PR 21403 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21697
**[Test build #92525 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92525/testReport)**
for PR 21697 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21696
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21696
**[Test build #92523 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92523/testReport)**
for PR 21696 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20345
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92522/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20345
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20345
**[Test build #92522 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92522/testReport)**
for PR 20345 at commit
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21679
Actually, the unsafe package does not include user-facing classes.
---
-
To unsubscribe, e-mail:
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21657
Not sure though, the `tokenIndexArr` implementation is always faster than
the unsafe projection?
---
-
To unsubscribe, e-mail:
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21692
Thanks, I feel it'd be better to put concrete version numbers in the
description.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21692
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92521/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21692
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21692
**[Test build #92521 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92521/testReport)**
for PR 21692 at commit
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21687
IMHO we need the same fix for both cases. Also, I like simpler
implementations in expressions, so IMO it'd be better to fix in analysis phases.
---
Github user yuanboliu commented on the issue:
https://github.com/apache/spark/pull/21690
@koeninger Sorry to interrupt, would you please review my patch? Thanks in
advance.
---
-
To unsubscribe, e-mail:
Github user skonto commented on the issue:
https://github.com/apache/spark/pull/21697
@ssuchter @foxish @felixcheung pls review.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user skonto opened a pull request:
https://github.com/apache/spark/pull/21697
[SPARK-24711][K8S] Fix tags for integration tests
## What changes were proposed in this pull request?
- disables maven surfire plugin to allow tags function properly, doc here:
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21687#discussion_r199451068
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala
---
@@ -129,7 +129,7 @@ case class CaseWhen(
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21687
@viirya Yeah, it looks like the same problem, but It's worked around via
different implementation of ```IfCoercion``` rule. This rule utilizes ```!=```
operator for comparison. So if two types
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/21556#discussion_r199442189
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala
---
@@ -359,6 +369,70 @@ class
GitHub user wangyum opened a pull request:
https://github.com/apache/spark/pull/21696
[SPARK-24716][SQL] Refactor ParquetFilters
## What changes were proposed in this pull request?
Replace DataFrame schema to Parquet file schema when create
`ParquetFilters`.
more
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21687
Seems `If` has similar issue?
```scala
case class If(predicate: Expression, trueValue: Expression, falseValue:
Expression)
extends Expression {
...
override def dataType:
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21687#discussion_r199435426
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala
---
@@ -129,7 +129,7 @@ case class CaseWhen(
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21687#discussion_r199434991
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala
---
@@ -129,7 +129,7 @@ case class CaseWhen(
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21657
> Do you mean we remove the option for column pruning in csv?
I mean reverting back the index mapping - `tokenIndexArr`. In this case,
your changes in `buildReader` are not needed.
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21682#discussion_r199430250
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -42,6 +42,14 @@ private[parquet]
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21695
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21695
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21695
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user nagpall opened a pull request:
https://github.com/apache/spark/pull/21695
Maintining an order
## What is the problem?
In both IndexedRowMatrix.computeSVD and IndexedRowMatrix.multiply indices
are dropped before calling the methods from RowMatrix.
For the
Github user priyankagargnitk commented on the issue:
https://github.com/apache/spark/pull/21679
org.apache.spark.unsafe.types.CalenderInterval is already public, am i
missing something.
Also, what if i want to do some computation on any data type and return
Calender Interval..
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/21682#discussion_r199426645
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -42,6 +42,14 @@ private[parquet]
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/21682#discussion_r199426772
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -42,6 +42,14 @@ private[parquet]
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21687#discussion_r199427016
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ConditionalExpressionSuite.scala
---
@@ -113,6 +113,35 @@ class
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21687#discussion_r199426921
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala
---
@@ -129,7 +129,7 @@ case class CaseWhen(
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21687#discussion_r199425774
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala
---
@@ -129,7 +129,7 @@ case class CaseWhen(
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21692
I've seen `jline:jline:0.9.94` in `sbt dependency-tree` before this change.
---
-
To unsubscribe, e-mail:
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21692
If we don't override this, which jline version does `sbt` use? `2.12.1`?
---
-
To unsubscribe, e-mail:
Github user chia7712 commented on the issue:
https://github.com/apache/spark/pull/21694
> I'm not sure this issue should be fixed for test code qualities though
the bug is obvious but it doesn't impact the test case. At any rate, I'm ok
to decline this PR :)
---
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/21667#discussion_r199418986
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala
---
@@ -30,7 +30,7 @@ import
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/21667#discussion_r199418036
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormat.scala
---
@@ -152,6 +152,16 @@ trait FileFormat {
}
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/21667#discussion_r199416587
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
---
@@ -306,6 +306,7 @@ case class FileSourceScanExec(
}
Github user chia7712 commented on the issue:
https://github.com/apache/spark/pull/21694
> I have one question; is this only a place having this issue?`
I grepped the code base by "isInstanceOf[T]". Only one result appears and
this PR have fixed it.
---
Github user mswit-databricks commented on a diff in the pull request:
https://github.com/apache/spark/pull/21320#discussion_r199415439
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadSupport.scala
---
@@ -71,9 +80,22 @@
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21694
I'm not sure this issue should be fixed for test code qualities though, I
have one question; is this only a place having this issue?
---
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21631
oh, super quick fix ;) Thanks, @MaxGekk
In the master, do we still hit the bug when parsing csv data?
---
-
To unsubscribe,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21694
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21694
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21694
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user chia7712 opened a pull request:
https://github.com/apache/spark/pull/21694
SPARK-24714 AnalysisSuite should use ClassTag to check the runtime inâ¦
â¦stance
## What changes were proposed in this pull request?
Use scala classTag to do the type check.
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21556
@wangyum Thanks for the benchmarks!
@dongjoon-hyun In the benchmarks above, the results of ORC except for the
case `decimal(9, 2)` have worse performance values as compared to the Parquet
ones.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20345
**[Test build #92522 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92522/testReport)**
for PR 20345 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20345
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/615/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20345
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/20345
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21692
Hmm, it seems hard to compare the diff between `sbt dependency-tree` and
`mvn dependency:tree`. Any suggestion?
---
-
To
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21657
@MaxGekk Do you mean we remove the option for column pruning?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21686
@rxin, does this look okay to you? If so will check closely and get this in.
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21683
cc @koeninger
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21693
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21691
@xianbin looks mistakenly open. Mind closing this please?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21693
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21693
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21692
Nice! Mind if I ask to show the diff between `sbt dependency-tree` vs` mvn
dependency:tree`? Looks fine to me.
---
-
To
GitHub user tmnd1991 opened a pull request:
https://github.com/apache/spark/pull/21693
[SPARK 24673]
## What changes were proposed in this pull request?
Add an overloaded version to `from_utc_timestamp` and `to_utc_timestamp`
having second argument as a `Column` instead of
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21692
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/614/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21692
**[Test build #92521 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92521/testReport)**
for PR 21692 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21692
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21692
cc @dbtsai @dongjoon-hyun @HyukjinKwon
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21682
Wow .. so it was 4 years ago .. okay.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/21692
[SPARK-24715][Build] Override jline version in SBT
## What changes were proposed in this pull request?
During SPARK-24418 (Upgrade Scala to 2.11.12 and 2.12.6), we upgrade
`jline` version
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21682
It is a regression that was introduced in Spark 1.2. Almost 4 years ago.
https://issues.apache.org/jira/browse/SPARK-4453
Thus, I think no need to document it. The pushdown can be
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21669
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92520/
Test FAILed.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21682
re: https://github.com/apache/spark/pull/21682#issuecomment-401676889 - I
couldn't recall related things about it. Thanks @gatorsmile. So, is it a
regression? shall we update the migration
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21669
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21669
**[Test build #92520 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92520/testReport)**
for PR 21669 at commit
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21682
@gatorsmile aha, thanks.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21682#discussion_r199400031
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -69,6 +77,14 @@ private[parquet]
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21679
btw, can you update the title like `[SPARK-24695][SQL]...`
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21679
Since `CalendarInterval` is an internal class, I think users are not
intended to use the class directly...
---
-
To unsubscribe,
301 - 400 of 410 matches
Mail list logo
