[GitHub] spark pull request #21687: [SPARK-24165][SQL] Fixing the output data type of...

2018-07-02 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21687#discussion_r199425774 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -129,7 +129,7 @@ case class CaseWhen(

[GitHub] spark pull request #21682: [SPARK-24706][SQL] ByteType and ShortType support...

2018-07-02 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/21682#discussion_r199426772 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -42,6 +42,14 @@ private[parquet]

[GitHub] spark pull request #21687: [SPARK-24165][SQL] Fixing the output data type of...

2018-07-02 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21687#discussion_r199426921 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -129,7 +129,7 @@ case class CaseWhen(

[GitHub] spark pull request #21687: [SPARK-24165][SQL] Fixing the output data type of...

2018-07-02 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21687#discussion_r199427016 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ConditionalExpressionSuite.scala --- @@ -113,6 +113,35 @@ class

[GitHub] spark issue #21695: Maintining an order

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21695 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21695: Maintining an order

2018-07-02 Thread nagpall
GitHub user nagpall opened a pull request: https://github.com/apache/spark/pull/21695 Maintining an order ## What is the problem? In both IndexedRowMatrix.computeSVD and IndexedRowMatrix.multiply indices are dropped before calling the methods from RowMatrix. For the

[GitHub] spark issue #21695: Maintining an order

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21695 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21682: [SPARK-24706][SQL] ByteType and ShortType support...

2018-07-02 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/21682#discussion_r199426645 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -42,6 +42,14 @@ private[parquet]

[GitHub] spark issue #21695: Maintining an order

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21695 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21687: [SPARK-24165][SQL] Fixing the output data type of CaseWh...

2018-07-02 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21687 Seems `If` has similar issue? ```scala case class If(predicate: Expression, trueValue: Expression, falseValue: Expression) extends Expression { ... override def dataType:

[GitHub] spark issue #21687: [SPARK-24165][SQL] Fixing the output data type of CaseWh...

2018-07-02 Thread mn-mikke
Github user mn-mikke commented on the issue: https://github.com/apache/spark/pull/21687 @viirya Yeah, it looks like the same problem, but It's worked around via different implementation of ```IfCoercion``` rule. This rule utilizes ```!=``` operator for comparison. So if two types

[GitHub] spark pull request #21687: [SPARK-24165][SQL] Fixing the output data type of...

2018-07-02 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21687#discussion_r199451068 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -129,7 +129,7 @@ case class CaseWhen(

[GitHub] spark pull request #21697: [SPARK-24711][K8S] Fix tags for integration tests

2018-07-02 Thread skonto
GitHub user skonto opened a pull request: https://github.com/apache/spark/pull/21697 [SPARK-24711][K8S] Fix tags for integration tests ## What changes were proposed in this pull request? - disables maven surfire plugin to allow tags function properly, doc here:

[GitHub] spark issue #21697: [SPARK-24711][K8S] Fix tags for integration tests

2018-07-02 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21697 @ssuchter @foxish @felixcheung pls review. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21690: [SPARK-24713]AppMatser of spark streaming kafka OOM if t...

2018-07-02 Thread yuanboliu
Github user yuanboliu commented on the issue: https://github.com/apache/spark/pull/21690 @koeninger Sorry to interrupt, would you please review my patch? Thanks in advance. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21687: [SPARK-24165][SQL] Fixing the output data type of CaseWh...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21687 IMHO we need the same fix for both cases. Also, I like simpler implementations in expressions, so IMO it'd be better to fix in analysis phases. ---

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version in SBT

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21692 **[Test build #92521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92521/testReport)** for PR 21692 at commit

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version in SBT

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21692 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version in SBT

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21692 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92521/ Test PASSed. ---

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version in SBT

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21692 Thanks, I feel it'd be better to put concrete version numbers in the description. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21657 Not sure though, the `tokenIndexArr` implementation is always faster than the unsafe projection? --- - To unsubscribe, e-mail:

[GitHub] spark issue #21679: [SPARK-24695] [SQL]: To add support to return Calendar i...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21679 Actually, the unsafe package does not include user-facing classes. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20345 **[Test build #92522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92522/testReport)** for PR 20345 at commit

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92522/ Test PASSed. ---

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21679: SPARK-24695: To add support to return Calendar interval ...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21679 Since `CalendarInterval` is an internal class, I think users are not intended to use the class directly... --- - To unsubscribe,

[GitHub] spark pull request #21692: [SPARK-24715][Build] Override jline version in SB...

2018-07-02 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/21692 [SPARK-24715][Build] Override jline version in SBT ## What changes were proposed in this pull request? During SPARK-24418 (Upgrade Scala to 2.11.12 and 2.12.6), we upgrade `jline` version

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21682 Wow .. so it was 4 years ago .. okay. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version in SBT

2018-07-02 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21692 cc @dbtsai @dongjoon-hyun @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version in SBT

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21692 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/614/

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version in SBT

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21692 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version in SBT

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21692 **[Test build #92521 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92521/testReport)** for PR 21692 at commit

[GitHub] spark issue #21686: [SPARK-24709][SQL] schema_of_json() - schema inference f...

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21686 @rxin, does this look okay to you? If so will check closely and get this in. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20345 **[Test build #92522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92522/testReport)** for PR 20345 at commit

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/615/

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21694: SPARK-24714 AnalysisSuite should use ClassTag to ...

2018-07-02 Thread chia7712
GitHub user chia7712 opened a pull request: https://github.com/apache/spark/pull/21694 SPARK-24714 AnalysisSuite should use ClassTag to check the runtime in… …stance ## What changes were proposed in this pull request? Use scala classTag to do the type check.

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-07-02 Thread mswit-databricks
Github user mswit-databricks commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r199415439 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadSupport.scala --- @@ -71,9 +80,22 @@

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version in SBT

2018-07-02 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21692 I've seen `jline:jline:0.9.94` in `sbt dependency-tree` before this change. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to manage ...

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21648 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21682: [SPARK-24706][SQL] ByteType and ShortType support...

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21682#discussion_r199396025 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -42,6 +42,14 @@ private[parquet]

[GitHub] spark pull request #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to ...

2018-07-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21648 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21682 It is a regression that was introduced in Spark 1.2. Almost 4 years ago. https://issues.apache.org/jira/browse/SPARK-4453 Thus, I think no need to document it. The pushdown can be

[GitHub] spark issue #21683: [SPARK-24507][Documentation] Update streaming guide

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21683 cc @koeninger --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21694: SPARK-24714 AnalysisSuite should use ClassTag to check t...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21694 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21694: SPARK-24714 AnalysisSuite should use ClassTag to check t...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21694 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21667: [SPARK-24691][SQL]Add new API `supportDataType` i...

2018-07-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21667#discussion_r199418036 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormat.scala --- @@ -152,6 +152,16 @@ trait FileFormat { }

[GitHub] spark issue #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to manage ...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21648 **[Test build #92518 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92518/testReport)** for PR 21648 at commit

[GitHub] spark issue #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to manage ...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21648 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to manage ...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21648 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92518/ Test PASSed. ---

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21682 **[Test build #92519 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92519/testReport)** for PR 21682 at commit

[GitHub] spark issue #21656: [SPARK-24677][Core]MedianHeap is empty when speculation ...

2018-07-02 Thread cxzl25
Github user cxzl25 commented on the issue: https://github.com/apache/spark/pull/21656 @maropu I have added a unit test. Can you trigger a test for this? --- - To unsubscribe, e-mail:

[GitHub] spark issue #21693: [SPARK 24673]

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21693 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21691: Branch 2.2

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21691 @xianbin looks mistakenly open. Mind closing this please? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21693: [SPARK 24673]

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21693 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version in SBT

2018-07-02 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21692 Hmm, it seems hard to compare the diff between `sbt dependency-tree` and `mvn dependency:tree`. Any suggestion? --- - To

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPruning en...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21631 oh, super quick fix ;) Thanks, @MaxGekk In the master, do we still hit the bug when parsing csv data? --- - To unsubscribe,

[GitHub] spark issue #21694: SPARK-24714 AnalysisSuite should use ClassTag to check t...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21694 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21667: [SPARK-24691][SQL]Add new API `supportDataType` i...

2018-07-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21667#discussion_r199418986 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -30,7 +30,7 @@ import

[GitHub] spark issue #21679: SPARK-24695: To add support to return Calendar interval ...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21679 btw, can you update the title like `[SPARK-24695][SQL]...` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #21682: [SPARK-24706][SQL] ByteType and ShortType support...

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21682#discussion_r199400031 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -69,6 +77,14 @@ private[parquet]

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21669 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92520/ Test FAILed. ---

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21682 re: https://github.com/apache/spark/pull/21682#issuecomment-401676889 - I couldn't recall related things about it. Thanks @gatorsmile. So, is it a regression? shall we update the migration

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21669 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21667: [SPARK-24691][SQL]Add new API `supportDataType` i...

2018-07-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21667#discussion_r199416587 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -306,6 +306,7 @@ case class FileSourceScanExec( }

[GitHub] spark issue #21694: SPARK-24714 AnalysisSuite should use ClassTag to check t...

2018-07-02 Thread chia7712
Github user chia7712 commented on the issue: https://github.com/apache/spark/pull/21694 > I have one question; is this only a place having this issue?` I grepped the code base by "isInstanceOf[T]". Only one result appears and this PR have fixed it. ---

[GitHub] spark issue #21694: SPARK-24714 AnalysisSuite should use ClassTag to check t...

2018-07-02 Thread chia7712
Github user chia7712 commented on the issue: https://github.com/apache/spark/pull/21694 > I'm not sure this issue should be fixed for test code qualities though the bug is obvious but it doesn't impact the test case. At any rate, I'm ok to decline this PR :) ---

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version in SBT

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21692 If we don't override this, which jline version does `sbt` use? `2.12.1`? --- - To unsubscribe, e-mail:

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21682 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21682 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92519/ Test PASSed. ---

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21682 @gatorsmile aha, thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21669 **[Test build #92520 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92520/testReport)** for PR 21669 at commit

[GitHub] spark pull request #21693: [SPARK 24673]

2018-07-02 Thread tmnd1991
GitHub user tmnd1991 opened a pull request: https://github.com/apache/spark/pull/21693 [SPARK 24673] ## What changes were proposed in this pull request? Add an overloaded version to `from_utc_timestamp` and `to_utc_timestamp` having second argument as a `Column` instead of

[GitHub] spark issue #21693: [SPARK 24673]

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21693 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version in SBT

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21692 Nice! Mind if I ask to show the diff between `sbt dependency-tree` vs` mvn dependency:tree`? Looks fine to me. --- - To

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21657 @MaxGekk Do you mean we remove the option for column pruning? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21556 @wangyum Thanks for the benchmarks! @dongjoon-hyun In the benchmarks above, the results of ORC except for the case `decimal(9, 2)` have worse performance values as compared to the Parquet ones.

[GitHub] spark issue #21694: SPARK-24714 AnalysisSuite should use ClassTag to check t...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21694 I'm not sure this issue should be fixed for test code qualities though, I have one question; is this only a place having this issue? ---

[GitHub] spark issue #21679: [SPARK-24695] [SQL]: To add support to return Calendar i...

2018-07-02 Thread priyankagargnitk
Github user priyankagargnitk commented on the issue: https://github.com/apache/spark/pull/21679 org.apache.spark.unsafe.types.CalenderInterval is already public, am i missing something. Also, what if i want to do some computation on any data type and return Calender Interval..

[GitHub] spark pull request #21687: [SPARK-24165][SQL] Fixing the output data type of...

2018-07-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21687#discussion_r199434991 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -129,7 +129,7 @@ case class CaseWhen(

[GitHub] spark pull request #21687: [SPARK-24165][SQL] Fixing the output data type of...

2018-07-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21687#discussion_r199435426 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -129,7 +129,7 @@ case class CaseWhen(

[GitHub] spark pull request #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21696 [SPARK-24716][SQL] Refactor ParquetFilters ## What changes were proposed in this pull request? Replace DataFrame schema to Parquet file schema when create `ParquetFilters`. more

[GitHub] spark pull request #21556: [SPARK-24549][SQL] Support Decimal type push down...

2018-07-02 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21556#discussion_r199442189 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -359,6 +369,70 @@ class

[GitHub] spark pull request #21682: [SPARK-24706][SQL] ByteType and ShortType support...

2018-07-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21682#discussion_r199430250 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -42,6 +42,14 @@ private[parquet]

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-02 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/21657 > Do you mean we remove the option for column pruning in csv? I mean reverting back the index mapping - `tokenIndexArr`. In this case, your changes in `buildReader` are not needed.

[GitHub] spark issue #21403: [SPARK-24341][SQL] Support only IN subqueries with the s...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21403 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/617/

[GitHub] spark issue #21403: [SPARK-24341][SQL] Support only IN subqueries with the s...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21403 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21698: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21698 **[Test build #92526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92526/testReport)** for PR 21698 at commit

[GitHub] spark issue #21697: [SPARK-24711][K8S] Fix tags for integration tests

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21697 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21698: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21698 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/620/

[GitHub] spark issue #21697: [SPARK-24711][K8S] Fix tags for integration tests

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21697 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/619/ ---

[GitHub] spark issue #21697: [SPARK-24711][K8S] Fix tags for integration tests

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21697 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/619/

[GitHub] spark issue #21698: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21698 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21685: [SPARK-24707][DSTREAMS] Enable spark-kafka-streaming to ...

2018-07-02 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/21685 What I can't really understand is why the `Scheduler Delay` is so different. ` Scheduler delay includes time to ship the task from the scheduler to the executor, and time to send

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21696 **[Test build #92523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92523/testReport)** for PR 21696 at commit

[GitHub] spark issue #21697: [SPARK-24711][K8S] Fix tags for integration tests

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21697 **[Test build #92525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92525/testReport)** for PR 21697 at commit

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21696: [SPARK-24716][SQL] Refactor ParquetFilters

2018-07-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/616/

[GitHub] spark issue #21403: [SPARK-24341][SQL] Support only IN subqueries with the s...

2018-07-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21403 **[Test build #92524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92524/testReport)** for PR 21403 at commit

  1   2   3   4   5   >