[GitHub] spark issue #22527: [SPARK-17952][SQL] Nested Java beans support in createDa...

2018-10-05 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22527 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22419: [SPARK-23906][SQL] Add built-in UDF TRUNCATE(numb...

2018-10-03 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22419#discussion_r222539741 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -1245,3 +1245,27 @@ case class BRound(child

[GitHub] spark pull request #22527: [SPARK-17952][SQL] Nested Java beans support in c...

2018-10-03 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22527#discussion_r222520022 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -1098,16 +1098,26 @@ object SQLContext { data: Iterator

[GitHub] spark issue #22619: [SQL][MINOR] Make use of TypeCoercion.findTightestCommon...

2018-10-03 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22619 Maybe this is related to #22448. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22527: [SPARK-17952][SQL] Nested Java beans support in c...

2018-10-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22527#discussion_r222183494 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -1098,16 +1098,24 @@ object SQLContext { data: Iterator

[GitHub] spark pull request #22527: [SPARK-17952][SQL] Nested Java beans support in c...

2018-10-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22527#discussion_r222185482 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -1098,16 +1098,24 @@ object SQLContext { data: Iterator

[GitHub] spark pull request #22527: [SPARK-17952][SQL] Nested Java beans support in c...

2018-10-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22527#discussion_r221857463 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -1100,13 +1101,23 @@ object SQLContext { attrs: Seq

[GitHub] spark pull request #22425: [SPARK-23367][Build] Include python document styl...

2018-09-27 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22425#discussion_r220956727 --- Diff: dev/tox.ini --- @@ -14,6 +14,8 @@ # limitations under the License. [pycodestyle] -ignore=E226,E241,E305,E402,E722,E731,E741

[GitHub] spark issue #22545: [SPARK-25525][SQL][PYSPARK] Do not update conf for exist...

2018-09-26 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22545 I'm okay with merging this only into master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22545: [SPARK-25525][SQL][PYSPARK] Do not update conf for exist...

2018-09-26 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22545 @dongjoon-hyun I'm sorry that I haven't shared the context, but I talked with @cloud-fan about https://github.com/apache/spark/pull/22545#discussion_r220414608 off-line, and he wanted to separate

[GitHub] spark issue #22544: [SPARK-25522][SQL] Improve type promotion for input argu...

2018-09-26 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22544 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22544: [SPARK-25522][SQL] Improve type promotion for inp...

2018-09-26 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22544#discussion_r220452656 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -971,9 +971,38 @@ object TypeCoercion

[GitHub] spark pull request #22544: [SPARK-25522][SQL] Improve type promotion for inp...

2018-09-26 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22544#discussion_r220445443 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -971,9 +971,38 @@ object TypeCoercion

[GitHub] spark pull request #22544: [SPARK-25522][SQL] Improve type promotion for inp...

2018-09-26 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22544#discussion_r220452974 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -971,9 +971,38 @@ object TypeCoercion

[GitHub] spark issue #22552: [SPARK-25540][SQL][PYSPARK] Make HiveContext in PySpark ...

2018-09-26 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22552 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22552: [SPARK-25540][SQL][PYSPARK] Make HiveContext in PySpark ...

2018-09-26 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22552 cc @cloud-fan @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22552: [SPARK-25540][SQL][PYSPARK] Make HiveContext in P...

2018-09-26 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/22552 [SPARK-25540][SQL][PYSPARK] Make HiveContext in PySpark behave as the same as Scala. ## What changes were proposed in this pull request? In Scala, `HiveContext` sets a config

[GitHub] spark pull request #22545: [SPARK-25525][SQL][PYSPARK] Do not update conf fo...

2018-09-25 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/22545 [SPARK-25525][SQL][PYSPARK] Do not update conf for existing SparkContext in SparkSession.getOrCreate. ## What changes were proposed in this pull request? In [SPARK-20946](https

[GitHub] spark issue #22542: [SPARK-25519][SQL] ArrayRemove function may return incor...

2018-09-24 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22542 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22419: [SPARK-23906][SQL] Add built-in UDF TRUNCATE(numb...

2018-09-21 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22419#discussion_r219655441 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/Decimal.scala --- @@ -413,6 +413,7 @@ object Decimal { val ROUND_HALF_EVEN

[GitHub] spark issue #22407: [SPARK-25416][SQL] ArrayPosition function may return inc...

2018-09-21 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22407 LGTM, pending Jenkins. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22407: [SPARK-25416][SQL] ArrayPosition function may return inc...

2018-09-21 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22407 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22407: [SPARK-25416][SQL] ArrayPosition function may return inc...

2018-09-21 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22407 So, before #22448 (branch-2.4 if we don't backport #22448), we don't coerce between decimals, and after that (master, not merged yet though), we will do, right? I'm okay with the behavior

[GitHub] spark issue #22419: [SPARK-23906][SQL] Add built-in UDF TRUNCATE(number)

2018-09-21 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22419 On second thoughts, I'm wondering whether we can reuse `RoundBase`? I mean: ```scala case class Truncate(child: Expression, scale: Expression) extends RoundBase(child, scale

[GitHub] spark issue #22471: [SPARK-25469][SQL][Performance] Eval methods of Concat, ...

2018-09-21 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22471 @maropu Do you want to merge this as your first work as a committer? I think this can be merged into master/2.4 because this is a performance regression fix

[GitHub] spark issue #22471: [SPARK-25469][SQL][Performance] Eval methods of Concat, ...

2018-09-20 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22471 @maropu Could you take a look at this again please? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22471: [SPARK-25469][SQL][Performance] Eval methods of Concat, ...

2018-09-20 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22471 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22471: [SPARK-25470][SQL][Performance] Concat.eval should use p...

2018-09-20 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22471 @mn-mikke Sure. I'm okay with including them to this pr. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #22471: [SPARK-25470][SQL][Performance] Concat.eval shoul...

2018-09-19 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22471#discussion_r219024051 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2274,33 +2274,41 @@ case class Concat

[GitHub] spark pull request #22419: [SPARK-23906][SQL] Add built-in UDF TRUNCATE(numb...

2018-09-18 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22419#discussion_r218535402 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2214,6 +2214,24 @@ object functions { */ def radians(columnName

[GitHub] spark pull request #22419: [SPARK-23906][SQL] Add built-in UDF TRUNCATE(numb...

2018-09-18 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22419#discussion_r218534147 --- Diff: sql/core/src/test/resources/sql-tests/inputs/operators.sql --- @@ -96,3 +96,9 @@ select positive('-1.11'), positive(-1.11), negative('-1.11

[GitHub] spark pull request #22419: [SPARK-23906][SQL] Add built-in UDF TRUNCATE(numb...

2018-09-18 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22419#discussion_r218524063 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -1245,3 +1245,65 @@ case class BRound(child

[GitHub] spark pull request #22419: [SPARK-23906][SQL] Add built-in UDF TRUNCATE(numb...

2018-09-18 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22419#discussion_r218525420 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/MathExpressionsSuite.scala --- @@ -644,4 +644,31 @@ class

[GitHub] spark pull request #22419: [SPARK-23906][SQL] Add built-in UDF TRUNCATE(numb...

2018-09-18 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22419#discussion_r218531021 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -1245,3 +1245,65 @@ case class BRound(child

[GitHub] spark pull request #22419: [SPARK-23906][SQL] Add built-in UDF TRUNCATE(numb...

2018-09-18 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22419#discussion_r218525567 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/MathExpressionsSuite.scala --- @@ -644,4 +644,31 @@ class

[GitHub] spark pull request #22448: [SPARK-25417][SQL] Improve findTightestCommonType...

2018-09-18 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22448#discussion_r218337303 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -31,6 +32,7 @@ import

[GitHub] spark issue #22448: [SPARK-25417][SQL] Improve findTightestCommonType to coe...

2018-09-18 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22448 I'm just wondering we should care about the case like `decimal(3, 2)` vs. `decimal(5, 1)`? --- - To unsubscribe, e-mail: reviews

[GitHub] spark issue #22439: [SPARK-25444][SQL] Refactor GenArrayData.genCodeToCreate...

2018-09-17 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22439 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22439: [SPARK-25444][SQL] Refactor GenArrayData.genCodeToCreate...

2018-09-17 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22439 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22439: [SPARK-25444][SQL] Refactor GenArrayData.genCodeT...

2018-09-17 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22439#discussion_r218048161 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -75,87 +74,60 @@ case class CreateArray

[GitHub] spark pull request #22439: [SPARK-25444][SQL] Refactor GenArrayData.genCodeT...

2018-09-17 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22439#discussion_r218032925 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -75,87 +75,60 @@ case class CreateArray

[GitHub] spark pull request #22439: [SPARK-25444][SQL] Refactor GenArrayData.genCodeT...

2018-09-17 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22439#discussion_r218030505 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -75,87 +75,60 @@ case class CreateArray

[GitHub] spark issue #22437: [SPARK-25431][SQL][EXAMPLES] Fix function examples and t...

2018-09-17 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22437 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22439: [SPARK-25444][SQL] Refactor GenArrayData.genCodeT...

2018-09-17 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22439#discussion_r217967158 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -75,87 +75,60 @@ case class CreateArray

[GitHub] spark pull request #22439: [SPARK-25444][SQL] Refactor GenArrayData.genCodeT...

2018-09-17 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22439#discussion_r217965854 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -75,87 +75,60 @@ case class CreateArray

[GitHub] spark pull request #22439: [SPARK-25444][SQL] Refactor GenArrayData.genCodeT...

2018-09-17 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22439#discussion_r217965632 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -75,87 +75,60 @@ case class CreateArray

[GitHub] spark issue #22437: [SPARK-25431][SQL][EXAMPLES] Fix function examples and t...

2018-09-17 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22437 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22437: [SPARK-25431][SQL][EXAMPLES] Fix function example...

2018-09-16 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/22437 [SPARK-25431][SQL][EXAMPLES] Fix function examples and the example results. ## What changes were proposed in this pull request? There are some mistakes in examples of newly added functions

[GitHub] spark issue #22437: [SPARK-25431][SQL][EXAMPLES] Fix function examples and t...

2018-09-16 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22437 cc @dongjoon-hyun @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22421: [SPARK-25431][SQL][EXAMPLES] Fix function examples and u...

2018-09-14 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22421 @dongjoon-hyun Sure, reverted. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark issue #22421: [SPARK-25431][SQL][EXAMPLES] Fix function examples and u...

2018-09-14 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22421 @dongjoon-hyun I picked the string from `sql(...).show(truncate = false)` and modified a little. I'm sorry but I didn't notice that we use the `spark-sql` to pick the result. Let me send a follow-up

[GitHub] spark pull request #22410: [SPARK-25418][SQL] The metadata of DataSource tab...

2018-09-14 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22410#discussion_r217672194 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -806,6 +807,8 @@ private[spark] class HiveExternalCatalog(conf

[GitHub] spark pull request #22421: [SPARK-25431][SQL][EXAMPLES] Fix function example...

2018-09-14 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/22421 [SPARK-25431][SQL][EXAMPLES] Fix function examples and unify the format of the example results. ## What changes were proposed in this pull request? There are some mistakes in examples

[GitHub] spark pull request #22408: [SPARK-25417][SQL] ArrayContains function may ret...

2018-09-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22408#discussion_r217601459 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -735,6 +735,44 @@ class DataFrameFunctionsSuite extends

[GitHub] spark pull request #22408: [SPARK-25417][SQL] ArrayContains function may ret...

2018-09-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22408#discussion_r217593872 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -735,6 +735,44 @@ class DataFrameFunctionsSuite extends

[GitHub] spark pull request #22410: [SPARK-25418][SQL] The metadata of DataSource tab...

2018-09-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22410#discussion_r217592782 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -1309,6 +1312,8 @@ object HiveExternalCatalog

[GitHub] spark pull request #22408: [SPARK-25417][SQL] ArrayContains function may ret...

2018-09-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22408#discussion_r217277655 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -735,6 +735,44 @@ class DataFrameFunctionsSuite extends

[GitHub] spark pull request #22408: [SPARK-25417][SQL] ArrayContains function may ret...

2018-09-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22408#discussion_r217272407 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -735,6 +735,44 @@ class DataFrameFunctionsSuite extends

[GitHub] spark pull request #22408: [SPARK-25417][SQL] ArrayContains function may ret...

2018-09-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22408#discussion_r217269231 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1331,26 +1331,31 @@ case class

[GitHub] spark pull request #22408: [SPARK-25417][SQL] ArrayContains function may ret...

2018-09-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22408#discussion_r217272398 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -735,6 +735,44 @@ class DataFrameFunctionsSuite extends

[GitHub] spark issue #22407: [SPARK-25416][SQL] ArrayPosition function may return inc...

2018-09-13 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22407 @dilipbiswal Thanks! I'll take a look at it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22407: [SPARK-25416][SQL] ArrayPosition function may ret...

2018-09-12 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22407#discussion_r217266743 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -1045,6 +1045,31 @@ class DataFrameFunctionsSuite extends

[GitHub] spark pull request #22407: [SPARK-25416][SQL] ArrayPosition function may ret...

2018-09-12 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22407#discussion_r217263871 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2062,18 +2062,23 @@ case class

[GitHub] spark pull request #22410: [SPARK-25418][SQL] The metadata of DataSource tab...

2018-09-12 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/22410 [SPARK-25418][SQL] The metadata of DataSource table should not include Hive-generated storage properties. ## What changes were proposed in this pull request? When Hive support enabled

[GitHub] spark issue #22410: [SPARK-25418][SQL] The metadata of DataSource table shou...

2018-09-12 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22410 cc @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22352: [SPARK-25208][SQL][FOLLOW-UP] Reduce code size.

2018-09-06 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22352 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22352: [SPARK-25208][SQL][FOLLOW-UP] Reduce code size.

2018-09-06 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22352 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22352: [SPARK-25208][SQL][FOLLOW-UP] Reduce code size.

2018-09-06 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/22352 [SPARK-25208][SQL][FOLLOW-UP] Reduce code size. ## What changes were proposed in this pull request? When casting to decimal type, if `Cast.canNullSafeCastToDecimal()`, overflow won't

[GitHub] spark issue #22314: [SPARK-25307][SQL] ArraySort function may return an erro...

2018-09-04 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22314 @dilipbiswal Thanks for confirming! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #22315: [SPARK-25308][SQL] ArrayContains function may return a e...

2018-09-04 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22315 @dilipbiswal Thanks for the verification! I don't think we will backport the `nullSafeCodeGen` to branch-2.3, but feel free to submit a pr to backport the test cases to branch-2.3

[GitHub] spark issue #22317: [SPARK-25310][SQL] ArraysOverlap may throw a Compilation...

2018-09-03 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22317 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22317: [SPARK-25310][SQL] ArraysOverlap may throw a Compilation...

2018-09-03 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22317 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22314: [SPARK-25307][SQL] ArraySort function may return an erro...

2018-09-03 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22314 @dilipbiswal Do we need to backport this to 2.3? If so, could you submit a backport pr to branch-2.3 please? Thanks

[GitHub] spark issue #22315: [SPARK-25308][SQL] ArrayContains function may return a e...

2018-09-03 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22315 @dilipbiswal Do we need to backport this to 2.3? If so, could you submit a backport pr to branch-2.3 please? Thanks

[GitHub] spark issue #22314: [SPARK-25307][SQL] ArraySort function may return an erro...

2018-09-03 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22314 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22315: [SPARK-25308][SQL] ArrayContains function may return a e...

2018-09-03 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22315 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22315: [SPARK-25308][SQL] ArrayContains function may return a e...

2018-09-03 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22315 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22314: [SPARK-25307][SQL] ArraySort function may return an erro...

2018-09-02 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22314 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22227: [SPARK-25202] [SQL] Implements split with limit s...

2018-09-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r214526544 --- Diff: common/unsafe/src/test/java/org/apache/spark/unsafe/types/UTF8StringSuite.java --- @@ -394,12 +394,14 @@ public void substringSQL

[GitHub] spark issue #22270: [SPARK-25267][SQL][TEST] Disable ConvertToLocalRelation ...

2018-09-01 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22270 @dilipbiswal Let's fix the two functions in 2.4. Could you open a pr to fix the two functions? Thanks! --- - To unsubscribe, e

[GitHub] spark issue #22302: [SPARK-21786][SQL][FOLLOWUP] Add compressionCodec test f...

2018-08-31 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22302 ok to test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-08-31 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20637 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22227: [SPARK-25202] [SQL] Implements split with limit s...

2018-08-31 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r214497013 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -229,36 +229,74 @@ case class RLike(left

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-08-31 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20637 Note that if we miss non-primitive type tests unfortunately, we can't detect the bad UTs without `asNullable` because they won't throw any exceptions but just use the default value

[GitHub] spark pull request #22227: [SPARK-25202] [SQL] Implements split with limit s...

2018-08-30 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r213913244 --- Diff: sql/core/src/test/resources/sql-tests/inputs/string-functions.sql --- @@ -47,3 +47,7 @@ FROM ( encode(string(id + 3), 'utf-8') col4

[GitHub] spark issue #22243: [MINOR] Avoid code duplication for nullable in Higher Or...

2018-08-27 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22243 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20637: [SPARK-23466][SQL] Remove redundant null checks i...

2018-08-27 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20637#discussion_r212882312 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,9 @@ trait

[GitHub] spark issue #22227: [SPARK-25202] [Core] Implements split with limit sql fun...

2018-08-24 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/7 @phegstrom Thanks for your contribution! Btw, seems like your email address in your commits is not connected to your GitHub account. Could you connect the address to your account, or use other

[GitHub] spark pull request #22227: [SPARK-25202] [Core] Implements split with limit ...

2018-08-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r212782616 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -232,30 +232,41 @@ case class RLike(left

[GitHub] spark pull request #22227: [SPARK-25202] [Core] Implements split with limit ...

2018-08-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r212782550 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -232,30 +232,41 @@ case class RLike(left

[GitHub] spark issue #22227: [SPARK-25202] [Core] Implements split with limit sql fun...

2018-08-24 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/7 ok to test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22216: [SPARK-25223][SQL] Use a map to store values for NamedLa...

2018-08-24 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22216 I see. Let me think again. I'd close this for now. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22216: [SPARK-25223][SQL] Use a map to store values for ...

2018-08-24 Thread ueshin
Github user ueshin closed the pull request at: https://github.com/apache/spark/pull/22216 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22216: [SPARK-25223][SQL] Use a map to store values for NamedLa...

2018-08-24 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22216 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22216: [SPARK-25223][SQL] Use a map to store values for ...

2018-08-24 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/22216 [SPARK-25223][SQL] Use a map to store values for NamedLambdaVariable. ## What changes were proposed in this pull request? Currently we use `functionsForEval`, `NamedLambdaVarible`s in which

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-08-24 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20637 @cloud-fan @mgaido91 Do you have any other comments on this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22187: [SPARK-25178][SQL] Directly ship the StructType objects ...

2018-08-23 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22187 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22200: [SPARK-25208][SQL] Loosen Cast.forceNullable for Decimal...

2018-08-23 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22200 cc @gatorsmile @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22200: [SPARK-25208][SQL] Loosen Cast.forceNullable for ...

2018-08-23 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/22200 [SPARK-25208][SQL] Loosen Cast.forceNullable for DecimalType. ## What changes were proposed in this pull request? Casting to `DecimalType` is not always needed to force nullable

[GitHub] spark issue #22187: [SPARK-25178][SQL] Directly ship the StructType objects ...

2018-08-23 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22187 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

<    1   2   3   4   5   6   7   8   9   10   >