[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-16 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20659 Thanks everyone, we can move to [SPARK-23710](https://issues.apache.org/jira/browse/SPARK-23710) to discuss

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-16 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20659 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20785: [SPARK-23640][CORE] Fix hadoop config may overrid...

2018-03-15 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20785#discussion_r174980995 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2434,7 +2434,8 @@ private[spark] object Utils extends Logging

[GitHub] spark pull request #20785: [SPARK-23640][CORE] Fix hadoop config may overrid...

2018-03-15 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20785#discussion_r174979402 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2434,7 +2434,8 @@ private[spark] object Utils extends Logging

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-15 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20659 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20835: [HOT-FIX] Fix SparkOutOfMemoryError: Unable to acquire 2...

2018-03-15 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20835 cc @kiszk @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #20835: [HOT-FIX] Fix SparkOutOfMemoryError: Unable to ac...

2018-03-15 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20835 [HOT-FIX] Fix SparkOutOfMemoryError: Unable to acquire 262144 bytes of memory, got 224631 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix

[GitHub] spark pull request #20819: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-14 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/20819 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20819: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-14 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20819 [DO-NOT-MERGE] Try to update Hive to 2.3.2 ## What changes were proposed in this pull request? Check if there is any test failed. You can merge this pull request into a Git repository

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-14 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20659 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-14 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20659 > [error] /home/jenkins/workspace/SparkPullRequestBuilder/sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/HiveSessionImpl.java:825: error: cannot find symbol [er

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20659 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20803: [SPARK-23653][SQL] Show sql statement in spark SQL UI

2018-03-12 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20803 ```bash cat < test.sql select '\${a}', '\${b}'; EOF spark-sql --hiveconf a=avalue --hivevar b=bvalue -f test.sql ``` SQL text is `select ${a}, ${b}` or `select ava

[GitHub] spark issue #20803: [SPARK-23653][SQL] Show sql statement in spark SQL UI

2018-03-12 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20803 1. Double click this SQL statement can show full SQL statement: https://github.com/apache/spark/pull/6646 2. What if this SQL statement contains `--hiveconf` or `--hivevar

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-12 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20785 You are right. In fact, our cluster has two shuffle services, one for production and one for development. We configure `spark.shuffle.service.port` to decide which shuffle service to use

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-10 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20785 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-09 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20785 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20785: [SPARK-23640][CORE] Fix hadoop config may overrid...

2018-03-09 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20785 [SPARK-23640][CORE] Fix hadoop config may override spark config ## What changes were proposed in this pull request? It may be get `spark.shuffle.service.port` from https://github.com

[GitHub] spark issue #20659: [DNM] Try to update Hive to 2.3.2

2018-03-05 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20659 Yes, I'm doing it --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20659: [DNM] Try to update Hive to 2.3.2

2018-03-05 Thread wangyum
GitHub user wangyum reopened a pull request: https://github.com/apache/spark/pull/20659 [DNM] Try to update Hive to 2.3.2 ## What changes were proposed in this pull request? Check if there is any test failed. You can merge this pull request into a Git repository

[GitHub] spark pull request #20735: [MINOR][YARN] Add disable yarn.nodemanager.vmem-c...

2018-03-05 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20735 [MINOR][YARN] Add disable yarn.nodemanager.vmem-check-enabled option to memLimitExceededLogMessage ## What changes were proposed in this pull request? My spark application sometimes will throw

[GitHub] spark pull request #20734: [SPARK-23510][DOC][FOLLOW-UP] Update spark.sql.hi...

2018-03-05 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20734 [SPARK-23510][DOC][FOLLOW-UP] Update spark.sql.hive.metastore.version ## What changes were proposed in this pull request? Update `spark.sql.hive.metastore.version` to 2.3.2, same

[GitHub] spark pull request #20668: [SPARK-23510][SQL] Support Hive 2.2 and Hive 2.3 ...

2018-02-28 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/20668 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20668: [SPARK-23510][SQL] Support Hive 2.2 and Hive 2.3 metasto...

2018-02-27 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20668 Yes, If we do not add `alterPartitionsMethod`, [HiveExternalSessionCatalogSuite.alter partitions](https://github.com/apache/spark/blob/d73bb92a72fdd6c1901c070a91b70b845a034e88/sql/catalyst/src

[GitHub] spark issue #20668: [SPARK-23510][SQL] Support Hive 2.2 and Hive 2.3 metasto...

2018-02-26 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20668 Otherwise, `SessionCatalogSuite` also needs to be updated ```scala Index: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala IDEA additional

[GitHub] spark pull request #20668: [SPARK-23510][SQL] Support Hive 2.2 and Hive 2.3 ...

2018-02-25 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20668#discussion_r170450895 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -1146,3 +1146,25 @@ private[client] class Shim_v2_1 extends

[GitHub] spark pull request #20668: [SPARK-23510][SQL] Support Hive 2.2 and Hive 2.3 ...

2018-02-24 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20668#discussion_r170425667 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala --- @@ -202,7 +202,6 @@ private[spark] object HiveUtils extends Logging

[GitHub] spark pull request #20668: [SPARK-23510][SQL] Support Hive 2.2 and Hive 2.3 ...

2018-02-24 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20668#discussion_r170425631 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala --- @@ -202,8 +202,6 @@ private[spark] object HiveUtils extends Logging

[GitHub] spark pull request #20668: [SPARK-23510][SQL] Support Hive 2.2 and Hive 2.3 ...

2018-02-24 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20668#discussion_r170425408 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -1146,3 +1146,25 @@ private[client] class Shim_v2_1 extends

[GitHub] spark pull request #20668: [SPARK-23510][SQL] Support Hive 2.2 and Hive 2.3 ...

2018-02-24 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20668 [SPARK-23510][SQL] Support Hive 2.2 and Hive 2.3 metastore ## What changes were proposed in this pull request? Support Hive 2.2 and Hive 2.3 metastore. ## How was this patch

[GitHub] spark pull request #20659: [DNM] Try to update Hive to 2.3.2

2018-02-22 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/20659 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20659: [DNM] Try to update Hive to 2.3.2

2018-02-21 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20659 [DNM] Try to update Hive to 2.3.2 ## What changes were proposed in this pull request? Check if there is any test failed. You can merge this pull request into a Git repository by running

[GitHub] spark issue #20597: [MINOR][TEST] Update from 2.2.0 to 2.2.1 in HiveExternal...

2018-02-13 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20597 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-02-11 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20504 @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20557: [SPARK-23364][SQL]'desc table' command in spark-s...

2018-02-09 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20557#discussion_r167225201 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -539,15 +539,15 @@ case class DescribeTableCommand

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20521 Thanks @cloud-fan It works. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-06 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20510 jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20510 jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20510 The failure is due to flaky test suite. ``` org.apache.spark.sql.hive.client.HiveClientSuites.(It is not a test it is a sbt.testing.NestedSuiteSelector) ``` jenkins, retest

[GitHub] spark pull request #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to su...

2018-02-05 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20504#discussion_r166156332 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala --- @@ -250,11 +257,20 @@ class SQLQueryTestSuite extends QueryTest

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20510 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20510 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #20508: [SPARK-23335][SQL] Should not convert to double w...

2018-02-05 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20508#discussion_r165968094 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -327,6 +327,14 @@ object TypeCoercion

[GitHub] spark pull request #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20510 [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4 ## What changes were proposed in this pull request? This PR upgrade snappy-java to 1.1.4. release notes: - Fix a 1% performance

[GitHub] spark issue #20274: [SPARK-20120][SQL][FOLLOW-UP] Better way to support spar...

2018-02-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20274 The [Pre-build spark](https://dist.apache.org/repos/dist/dev/spark/v2.3.0-rc2-bin/) contains `kubernetes-model-2.0.0.jar`. but the below command will not contain this jar: ``` ./dev/make

[GitHub] spark issue #20274: [SPARK-20120][SQL][FOLLOW-UP] Better way to support spar...

2018-02-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20274 An interesting discovery: if `SPARK_HOME/jars` missing `kubernetes-model-2.0.0.jar`, the silent mode is broken

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-02-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20504 Thanks @hvanhovell , the major changes is `SQLQueryTestSuite.scala`: ```scala private def listTestCases(): Seq[TestCase] = { -listFilesRecursively(new File(inputFilePath)).map

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-02-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20504 After SPARK-21646, `hive/binaryComparison.sql.out`, `hive/decimalPrecision.sql.out` and `hive/promoteStrings.sql.out` seems like this: https://github.com/wangyum/spark/commit

[GitHub] spark pull request #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to su...

2018-02-04 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20504 [SPARK-23332][SQL] Update SQLQueryTestSuite to support test hive mode ## What changes were proposed in this pull request? Update `SQLQueryTestSuite` to support test hive mode

[GitHub] spark pull request #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeti...

2018-02-04 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20498#discussion_r165844408 --- Diff: sql/core/src/test/resources/sql-tests/inputs/typeCoercion/native/decimalArithmeticOperations.sql --- @@ -74,7 +75,8 @@ select

[GitHub] spark pull request #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeti...

2018-02-04 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20498#discussion_r165844386 --- Diff: sql/core/src/test/resources/sql-tests/inputs/typeCoercion/native/decimalArithmeticOperations.sql --- @@ -48,8 +48,9 @@ select

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix imperfect test

2018-02-03 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20498 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix imperfect test

2018-02-03 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20498 retest please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20498: [SPARK-22036][SQL][FOLLOWUP] Fix imperfect test

2018-02-03 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20498 [SPARK-22036][SQL][FOLLOWUP] Fix imperfect test ## What changes were proposed in this pull request? Fix imperfect test ## How was this patch tested? N/A You can merge

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2018-02-02 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/19788 Thanks @yucai , It's a great improvement for many output files. The figure below is our comparison: **Before**: https://user-images.githubusercontent.com/5399861/35762292-6b5f9f88-08cf-11e8

[GitHub] spark pull request #20430: [SPARK-23263][SQL] Create table stored as parquet...

2018-01-29 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20430 [SPARK-23263][SQL] Create table stored as parquet should update table size if automatic update table size is enabled …update table size is enabled ## What changes were proposed

[GitHub] spark pull request #20303: [SPARK-23128][SQL] A new approach to do adaptive ...

2018-01-26 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20303#discussion_r164070918 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/QueryStage.scala --- @@ -0,0 +1,222 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #18138: [SPARK-20915][SQL] Make lpad/rpad with empty pad string ...

2018-01-16 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/18138 Hive will throw `ArrayIndexOutOfBoundsException` at runtime: https://issues.apache.org/jira/browse/HIVE-17077

[GitHub] spark pull request #20274: [SPARK-20120][SQL][FOLLOW-UP] Better way to suppo...

2018-01-15 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20274 [SPARK-20120][SQL][FOLLOW-UP] Better way to support spark-sql silent mode. ## What changes were proposed in this pull request? `spark-sql` silent mode is broken now. It seems

[GitHub] spark issue #20268: [SPARK-19550][BUILD][FOLLOW-UP] Remove MaxPermSize for s...

2018-01-15 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20268 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20268: [SPARK-19550][BUILD][FOLLOW-UP] Remove MaxPermSiz...

2018-01-14 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20268 [SPARK-19550][BUILD][FOLLOW-UP] Remove MaxPermSize for sql module ## What changes were proposed in this pull request? Remove `MaxPermSize` for `sql` module ## How was this patch

[GitHub] spark issue #20248: [SPARK-23058][SQL] Show non printable field delim as uni...

2018-01-13 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20248 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20248: [SPARK-23058][SQL] Show non printable field delim as uni...

2018-01-13 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20248 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20248: [SPARK-23058][SQL] Show non printable field delim...

2018-01-12 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20248#discussion_r161364269 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -1023,7 +1023,12 @@ case class ShowCreateTableCommand(table

[GitHub] spark issue #20248: [SPARK-23058][SQL] Fix non printable field delim issue

2018-01-12 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20248 [Non printable characters](http://www.theasciicode.com.ar/): https://user-images.githubusercontent.com/5399861/34880068-33152b7a-f7ea-11e7-8203-570e61c7a21c.png

[GitHub] spark pull request #20248: [SPARK-23058][SQL] Fix non printable field delim ...

2018-01-12 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20248 [SPARK-23058][SQL] Fix non printable field delim issue ## What changes were proposed in this pull request? Create a table with non printable delim like below: ```sql CREATE

[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...

2018-01-09 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20080 @srowen @jiangxb1987 I have test on my cluster with this patch. ``` bin/spark-sql --master yarn --conf spark.dynamicAllocation.enabled=true --conf spark.shuffle.service.enabled=true --conf

[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...

2017-12-25 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20080 cc @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20080: [SPARK-22870][CORE] Dynamic allocation should all...

2017-12-25 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20080 [SPARK-22870][CORE] Dynamic allocation should allow 0 idle time ## What changes were proposed in this pull request? This pr to make `0` as a valid value

[GitHub] spark issue #20079: [SPARK-22893][SQL][HOTFIX] Fix a error message of Versio...

2017-12-25 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20079 LGTM, thanks @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #20067: [SPARK-22894][SQL] DateTimeOperations should acce...

2017-12-25 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20067#discussion_r158648321 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2760,6 +2760,17 @@ class SQLQuerySuite extends QueryTest

[GitHub] spark pull request #20067: [SPARK-22894][SQL] DateTimeOperations should acce...

2017-12-25 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20067#discussion_r158648226 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2760,6 +2760,17 @@ class SQLQuerySuite extends QueryTest

[GitHub] spark pull request #20067: [SPARK-22894][SQL] DateTimeOperations should acce...

2017-12-25 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20067#discussion_r158647982 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2760,6 +2760,17 @@ class SQLQuerySuite extends QueryTest

[GitHub] spark pull request #20067: [SPARK-22894][SQL] DateTimeOperations should acce...

2017-12-23 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20067 [SPARK-22894][SQL] DateTimeOperations should accept SQL like string type ## What changes were proposed in this pull request? `DateTimeOperations` accept [`StringType`](https://github.com

[GitHub] spark pull request #20066: [SPARK-22833][Examples][FOLLOWUP] Remove whitespa...

2017-12-23 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/20066 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20018: SPARK-22833 [Improvement] in SparkHive Scala Examples

2017-12-23 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20018 Thanks @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #20066: [SPARK-22833][Examples][FOLLOWUP] Remove whitespace to f...

2017-12-23 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20066 https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85343/console --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #20066: [SPARK-22833][Examples][FOLLOWUP] Remove whitespa...

2017-12-23 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20066 [SPARK-22833][Examples][FOLLOWUP] Remove whitespace to fix scalastyle checks failed ## What changes were proposed in this pull request? This is a followup PR for: https://github.com

[GitHub] spark issue #20064: [SPARK-22893][SQL] Unified the data type mismatch messag...

2017-12-23 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20064 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20064: [SPARK-22893][SQL] Unified the data type mismatch...

2017-12-23 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20064 [SPARK-22893][SQL] Unified the data type mismatch message ## What changes were proposed in this pull request? We should use `dataType.simpleString` to unified the data type mismatch

[GitHub] spark pull request #20061: [SPARK-22890][TEST] Basic tests for DateTimeOpera...

2017-12-22 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20061 [SPARK-22890][TEST] Basic tests for DateTimeOperations ## What changes were proposed in this pull request? Test Coverage for `DateTimeOperations`, this is a Sub-tasks for [SPARK-22722

[GitHub] spark issue #20008: [SPARK-22822][TEST] Basic tests for WindowFrameCoercion ...

2017-12-20 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20008 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20008: [SPARK-22822][TEST] Basic tests for WindowFrameCo...

2017-12-20 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20008#discussion_r158016336 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala --- @@ -252,7 +252,7 @@ case class

[GitHub] spark pull request #19804: [WIP][SPARK-22573][SQL] Shouldn't inferFilters if...

2017-12-20 Thread wangyum
GitHub user wangyum reopened a pull request: https://github.com/apache/spark/pull/19804 [WIP][SPARK-22573][SQL] Shouldn't inferFilters if it contains SubqueryExpression ## What changes were proposed in this pull request? Shouldn't inferFilters if it contains

[GitHub] spark issue #20008: [SPARK-22822][TEST] Basic tests for WindowFrameCoercion ...

2017-12-20 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20008 retest this, please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #20008: [SPARK-22822][TEST] Basic tests for FunctionArgum...

2017-12-19 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20008#discussion_r157920327 --- Diff: sql/core/src/test/resources/sql-tests/inputs/typeCoercion/native/functionArgumentConversion.sql --- @@ -25,7 +25,7 @@ SELECT array(cast(1

[GitHub] spark pull request #20006: [SPARK-22821][TEST] Basic tests for WidenSetOpera...

2017-12-19 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20006#discussion_r157691771 --- Diff: sql/core/src/test/resources/sql-tests/inputs/typeCoercion/native/booleanEquality.sql --- @@ -0,0 +1,122 @@ +-- +-- Licensed

[GitHub] spark pull request #20008: [SPARK-22822][TEST] Basic tests for FunctionArgum...

2017-12-18 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20008#discussion_r157651436 --- Diff: sql/core/src/test/resources/sql-tests/inputs/typeCoercion/native/decimalPrecision.sql --- @@ -0,0 +1,6883 @@ +-- +-- Licensed

[GitHub] spark pull request #20008: [SPARK-22822][TEST] Basic tests for FunctionArgum...

2017-12-18 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20008 [SPARK-22822][TEST] Basic tests for FunctionArgumentConversion and DecimalPrecision ## What changes were proposed in this pull request? Test Coverage for `FunctionArgumentConversion

[GitHub] spark pull request #20006: [SPARK-22821][TEST] Basic tests for WidenSetOpera...

2017-12-17 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20006 [SPARK-22821][TEST] Basic tests for WidenSetOperationTypes, BooleanEquality, StackCoercion and Division ## What changes were proposed in this pull request? Test Coverage

[GitHub] spark issue #19714: [SPARK-22489][SQL] Shouldn't change broadcast join build...

2017-12-17 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/19714 Can we backport this to branch-2.2? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark pull request #20001: [SPARK-22762][TEST] Basic tests for PromoteString...

2017-12-16 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20001 [SPARK-22762][TEST] Basic tests for PromoteStrings and InConversion ## What changes were proposed in this pull request? Test Coverage for `PromoteStrings` and `InConversion`, this is a Sub

[GitHub] spark issue #19949: [SPARK-22762][TEST] Basic tests for IfCoercion and CaseW...

2017-12-15 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/19949 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #19949: [SPARK-22762][TEST] Basic tests for IfCoercion an...

2017-12-14 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/19949#discussion_r157106530 --- Diff: sql/core/src/test/resources/sql-tests/inputs/typeCoercion/native/caseWhenCoercion.sql --- @@ -0,0 +1,200 @@ +-- +-- Licensed

[GitHub] spark issue #19949: [SPARK-22762][TEST] Basic tests for IfCoercion and CaseW...

2017-12-12 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/19949 @HyukjinKwon see: https://issues.apache.org/jira/browse/SPARK-22722 --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #19949: [SPARK-22762][TEST] Basic tests for IfCoercion an...

2017-12-12 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/19949 [SPARK-22762][TEST] Basic tests for IfCoercion and CaseWhenCoercion ## What changes were proposed in this pull request? Basic tests for IfCoercion and CaseWhenCoercion ## How

[GitHub] spark pull request #19918: [SPARK-22726] [TEST] Basic tests for Binary Compa...

2017-12-09 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/19918#discussion_r155939348 --- Diff: sql/core/src/test/resources/sql-tests/inputs/typeCoercion/native/binaryComparison.sql --- @@ -0,0 +1,287 @@ +-- +-- Licensed

[GitHub] spark pull request #19932: [SPARK-22745][SQL] read partition stats from Hive

2017-12-09 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/19932#discussion_r155935430 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -213,6 +213,29 @@ class StatisticsSuite extends

[GitHub] spark pull request #19841: [SPARK-22642][SQL] the createdTempDir will not be...

2017-12-01 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/19841#discussion_r154480032 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -104,147 +105,153 @@ case class InsertIntoHiveTable

[GitHub] spark issue #19858: [SPARK-22489][DOC][FOLLOWUP] Update broadcast behavior c...

2017-12-01 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/19858 cc @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

<    2   3   4   5   6   7   8   9   10   >