[GitHub] spark pull request #22500: [SPARK-25488][TEST] Refactor MiscBenchmark to use...

2018-09-21 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22500#discussion_r219403342 --- Diff: sql/core/benchmarks/MiscBenchmark-results.txt --- @@ -0,0 +1,132

[GitHub] spark issue #22497: [SPARK-25487][SQL][TEST] Refactor PrimitiveArrayBenchmar...

2018-09-21 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22497 Congratulation, @kiszk --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22513: [SPARK-25499][TEST]Refactor BenchmarkBase and Benchmark

2018-09-21 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22513 `KryoBenchmark` is in core, and `UnsafeProjectionBenchmark`, `HashByteArrayBenchmark` and `HashBenchmark` are in `catalyst`. If we move the benchmark base class to sql, benchmarks mentioned above

[GitHub] spark pull request #22513: [SPARK-25499][TEST]Refactor BenchmarkBase and Ben...

2018-09-20 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22513#discussion_r219388085 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/FilterPushdownBenchmark.scala --- @@ -27,7 +27,7 @@ import

[GitHub] spark pull request #22499: [SPARK-25489][ML][TEST] Refactor UDTSerialization...

2018-09-20 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22499#discussion_r219366799 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/linalg/UDTSerializationBenchmark.scala --- @@ -18,52 +18,52 @@ package

[GitHub] spark pull request #22501: [SPARK-25492][TEST] Refactor WideSchemaBenchmark ...

2018-09-20 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22501 [SPARK-25492][TEST] Refactor WideSchemaBenchmark to use main method ## What changes were proposed in this pull request? Refactor `WideSchemaBenchmark` to use main method. Generate

[GitHub] spark pull request #22500: [SPARK-25488][TEST] Refactor MiscBenchmark to use...

2018-09-20 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22500#discussion_r219219972 --- Diff: sql/core/benchmarks/MiscBenchmark-results.txt --- @@ -0,0 +1,132

[GitHub] spark pull request #22500: [SPARK-25488][TEST] Refactor MiscBenchmark to use...

2018-09-20 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22500#discussion_r219218036 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/MiscBenchmark.scala --- @@ -17,251 +17,154 @@ package

[GitHub] spark pull request #22500: [SPARK-25488][TEST] Refactor MiscBenchmark to use...

2018-09-20 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22500 [SPARK-25488][TEST] Refactor MiscBenchmark to use main method ## What changes were proposed in this pull request? Refactor `MiscBenchmark ` to use main method. Generate benchmark

[GitHub] spark issue #22497: [SPARK-25487][SQL][TEST] Refactor PrimitiveArrayBenchmar...

2018-09-20 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22497 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22491: [SPARK-25483][TEST] Refactor UnsafeArrayDataBench...

2018-09-20 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22491 [SPARK-25483][TEST] Refactor UnsafeArrayDataBenchmark to use main method ## What changes were proposed in this pull request? Refactor `UnsafeArrayDataBenchmark` to use main method

[GitHub] spark issue #22488: [SPARK-25479][TEST] Refactor DatasetBenchmark to use mai...

2018-09-20 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22488 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22484: [SPARK-25476][TEST] Refactor AggregateBenchmark t...

2018-09-20 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22484#discussion_r219104743 --- Diff: sql/core/benchmarks/AggregateBenchmark-results.txt --- @@ -0,0 +1,154

[GitHub] spark pull request #22488: [SPARK-25479][TEST] Refactor DatasetBenchmark to ...

2018-09-20 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22488 [SPARK-25479][TEST] Refactor DatasetBenchmark to use main method ## What changes were proposed in this pull request? Refactor `DatasetBenchmark` to use main method. Generate benchmark

[GitHub] spark pull request #22486: [SPARK-25478][TEST] Refactor CompressionSchemeBen...

2018-09-20 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22486 [SPARK-25478][TEST] Refactor CompressionSchemeBenchmark to use main method ## What changes were proposed in this pull request? Refactor `CompressionSchemeBenchmark` to use main method

[GitHub] spark pull request #22484: [SPARK-25476][TEST] Refactor AggregateBenchmark t...

2018-09-20 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22484 [SPARK-25476][TEST] Refactor AggregateBenchmark to use main method ## What changes were proposed in this pull request? Refactor `AggregateBenchmark` to use main method. To gererate

[GitHub] spark pull request #22419: [SPARK-23906][SQL] Add built-in UDF TRUNCATE(numb...

2018-09-19 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22419#discussion_r218749189 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -1245,3 +1245,80 @@ case class BRound(child

[GitHub] spark issue #22443: [SPARK-25339][TEST] Refactor FilterPushdownBenchmark

2018-09-19 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22443 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22443: [SPARK-25339][TEST] Refactor FilterPushdownBenchmark

2018-09-18 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22443 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22461: [SPARK-25453] OracleIntegrationSuite IllegalArgumentExce...

2018-09-18 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22461 cc @maropu --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22461: [SPARK-25453] OracleIntegrationSuite IllegalArgumentExce...

2018-09-18 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22461 Could you add `[TEST]` to title, otherwise LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22443: [SPARK-25339][TEST] Refactor FilterPushdownBenchm...

2018-09-18 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22443#discussion_r218642335 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/FilterPushdownBenchmark.scala --- @@ -17,29 +17,28 @@ package

[GitHub] spark pull request #22443: [SPARK-25339][TEST] Refactor FilterPushdownBenchm...

2018-09-18 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22443#discussion_r218308258 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/FilterPushdownBenchmark.scala --- @@ -17,29 +17,28 @@ package

[GitHub] spark issue #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JD...

2018-09-17 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22446 Yes. I can't find more references to the old JDK docs also. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JD...

2018-09-17 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22446 Some references to Java 7, Some references to Java 6. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JD...

2018-09-17 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22446 cc @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to...

2018-09-17 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22446 [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JDK8 ## What changes were proposed in this pull request? Update `tuning.md` and `building-spark.md` to use JDK8. ## How

[GitHub] spark issue #22443: [SPARK-25339][TESTS] Refactor FilterPushdownBenchmark

2018-09-17 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22443 cc @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22443: [SPARK-25339][TESTS] Refactor FilterPushdownBench...

2018-09-17 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22443 [SPARK-25339][TESTS] Refactor FilterPushdownBenchmark ## What changes were proposed in this pull request? Refactor `FilterPushdownBenchmark` use `main` method. we can use 3 ways to run

[GitHub] spark pull request #22435: [SPARK-25423][SQL] Output "dataFilters" in DataSo...

2018-09-16 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22435#discussion_r217934875 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/DataSourceScanExecRedactionSuite.scala --- @@ -83,4 +83,20 @@ class

[GitHub] spark pull request #22435: [SPARK-25423][SQL] Output "dataFilters" in DataSo...

2018-09-16 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22435 [SPARK-25423][SQL] Output "dataFilters" in DataSourceScanExec.metadata ## What changes were proposed in this pull request? Output `dataFilters` in `DataSourceScanExe

[GitHub] spark pull request #22427: [SPARK-25438][SQL][TEST] Fix FilterPushdownBenchm...

2018-09-16 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22427#discussion_r217918733 --- Diff: sql/core/benchmarks/FilterPushdownBenchmark-results.txt --- @@ -2,737 +2,669 @@ Pushdown for many distinct value case

[GitHub] spark pull request #22426: [SPARK-25436] Bump master branch version to 2.5.0...

2018-09-14 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22426#discussion_r217876251 --- Diff: docs/_config.yml --- @@ -14,8 +14,8 @@ include: # These allow the documentation to be updated with newer releases # of Spark

[GitHub] spark issue #22420: SPARK-25429

2018-09-14 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22420 Could you update the PR title. The PR title should be of the form [SPARK-][COMPONENT] Title. --- - To unsubscribe, e-mail

[GitHub] spark pull request #22419: [SPARK-23906][SQL] Add UDF TRUNCATE(number)

2018-09-14 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22419 [SPARK-23906][SQL] Add UDF TRUNCATE(number) ## What changes were proposed in this pull request? Add UDF `TRUNCATE(number)`: ```sql > SELECT TRUNCATE(1234567891.1234567891

[GitHub] spark issue #18106: [SPARK-20754][SQL] Support TRUNC (number)

2018-09-13 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/18106 @dongjoon-hyun Actually `TRUNC (number)` not resolved. I will fix it soon. https://issues.apache.org/jira/browse/SPARK-23906

[GitHub] spark pull request #18106: [SPARK-20754][SQL] Support TRUNC (number)

2018-09-13 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/18106 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComp...

2018-09-13 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22038 @mgaido91 I updated Postgres and Hive to https://github.com/apache/spark/pull/22038#issuecomment-412737994 @gatorsmile Is this change make sense

[GitHub] spark pull request #22358: [SPARK-25366][SQL]Zstd and brotli CompressionCode...

2018-09-11 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22358#discussion_r216873177 --- Diff: docs/sql-programming-guide.md --- @@ -965,6 +965,8 @@ Configuration of Parquet can be done using the `setConf` method on `SparkSession

[GitHub] spark pull request #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix I...

2018-09-11 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/22387 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22387 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to su...

2018-09-11 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/20504 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIn...

2018-09-11 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22387 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-09-11 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20504 I will close it now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22387: [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix I...

2018-09-10 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22387 [SPARK-25313][SQL][FOLLOW-UP][BACKPORT-2.3] Fix InsertIntoHiveDirCommand output schema in Parquet issue ## What changes were proposed in this pull request? Backport https://github.com

[GitHub] spark pull request #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson v...

2018-09-10 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/22372 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...

2018-09-10 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22372 I did a simple test for 2.9.6. It works well. But that pr for 3.0. It means that a simple test on branch 2.4 will fail: ```scala scala> spark.range(10).write.parquet("/tmp/spark

[GitHub] spark pull request #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson v...

2018-09-10 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22372#discussion_r216208966 --- Diff: pom.xml --- @@ -2694,6 +2694,8 @@ 3.1.0 2.12.0 3.4.9 +2.7.8 + 2.7.8 --- End

[GitHub] spark pull request #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson v...

2018-09-10 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22372#discussion_r216208793 --- Diff: pom.xml --- @@ -2694,6 +2694,8 @@ 3.1.0 2.12.0 3.4.9 +2.7.8 + 2.7.8 --- End

[GitHub] spark issue #22359: [SPARK-25313][SQL][FOLLOW-UP] Fix InsertIntoHiveDirComma...

2018-09-09 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22359 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...

2018-09-09 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22372 We do not have jenkins tests for 3.1 profile: https://github.com/apache/spark/blob/395860a986987886df6d60fd9b26afd818b2cb39/dev/run-tests.py#L307-L310

[GitHub] spark pull request #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson v...

2018-09-09 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22372 [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version to 2.7.8 ## What changes were proposed in this pull request? Upgrade Hadoop 3.1 jackson version to 2.7.8 to fix

[GitHub] spark issue #22368: [SPARK-25368][SQL] Incorrect predicate pushdown returns ...

2018-09-09 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22368 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22368: [SPARK-25368][SQL] Incorrect predicate pushdown r...

2018-09-08 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22368 [SPARK-25368][SQL] Incorrect predicate pushdown returns wrong result ## What changes were proposed in this pull request? How to reproduce: ```scala val df1 = spark.createDataFrame(Seq

[GitHub] spark pull request #22359: [SPARK-25313][SQL][FOLLOW-UP] Fix InsertIntoHiveD...

2018-09-07 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22359#discussion_r216117397 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -803,6 +803,25 @@ class HiveDDLSuite

[GitHub] spark pull request #22358: [SPARK-25366][SQL]Zstd and brotil CompressionCode...

2018-09-07 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22358#discussion_r215897048 --- Diff: docs/sql-programming-guide.md --- @@ -964,7 +964,7 @@ Configuration of Parquet can be done using the `setConf` method on `SparkSession

[GitHub] spark pull request #22358: [SPARK-25366][SQL]Zstd and brotil CompressionCode...

2018-09-07 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22358#discussion_r215874603 --- Diff: docs/sql-programming-guide.md --- @@ -964,7 +964,7 @@ Configuration of Parquet can be done using the `setConf` method on `SparkSession

[GitHub] spark issue #22359: [SPARK-25313][SQL]FOLLOW-UP] Fix InsertIntoHiveDirComman...

2018-09-07 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22359 cc @gengliangwang --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22359: [SPARK-25313][SQL]FOLLOW-UP] Fix InsertIntoHiveDi...

2018-09-07 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22359 [SPARK-25313][SQL]FOLLOW-UP] Fix InsertIntoHiveDirCommand output schema issue ## What changes were proposed in this pull request? Fix `InsertIntoHiveDirCommand` output schema issue

[GitHub] spark pull request #22287: [SPARK-25135][SQL] FileFormatWriter should respec...

2018-09-05 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/22287 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22320: [SPARK-25313][SQL]Fix regression in FileFormatWriter out...

2018-09-05 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22320 @gengliangwang We need backport this pr to branch-2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22327: [SPARK-25330][BUILD] Revert Hadoop 2.7 to 2.7.3

2018-09-05 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22327 How about revert it to branch-2.3 as we are going to release 2.3.2? We have time to fix it before releasing 2.4.0

[GitHub] spark issue #22320: [SPARK-25313][SQL]Fix regression in FileFormatWriter out...

2018-09-05 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22320 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21404: [SPARK-24360][SQL] Support Hive 3.0 metastore

2018-09-05 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21404#discussion_r215232389 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala --- @@ -99,6 +99,7 @@ private[hive] object

[GitHub] spark issue #17174: [SPARK-19145][SQL] Timestamp to String casting is slowin...

2018-09-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/17174 @tanejagagan Are you still working on? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22320: [SPARK-25313][SQL]Fix regression in FileFormatWri...

2018-09-04 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22320#discussion_r215106921 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -56,7 +56,7 @@ case class

[GitHub] spark issue #22228: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...

2018-09-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/8 This is already merged, @huaxingao Could you please close this PR? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #22179: [SPARK-23131][BUILD] Upgrade Kryo to 4.0.2

2018-09-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22179 Thanks, @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22327: [SPARK-25330][BUILD] Revert Hadoop 2.7 to 2.7.3

2018-09-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22327 Yes. This is a Hadoop thing. I try to build Hadoop 2.7.7 with [`Configuration.getRestrictParserDefault(Object resource)`](https://github.com/apache/hadoop/blob/release-2.7.7-RC0/hadoop-common

[GitHub] spark issue #22327: [SPARK-25330][BUILD] Revert Hadoop 2.7 to 2.7.3

2018-09-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22327 cc @srowen @steveloughran --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22327: [SPARK-25330][BUILD] Revert Hadoop 2.7 to 2.7.3

2018-09-04 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22327 [SPARK-25330][BUILD] Revert Hadoop 2.7 to 2.7.3 ## What changes were proposed in this pull request? Revert Hadoop 2.7 to 2.7.3 to fix permission issue. The issue occurred in this commit

[GitHub] spark pull request #22320: [SPARK-25313][SQL]Fix regression in FileFormatWri...

2018-09-03 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22320#discussion_r214786494 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -754,6 +754,47 @@ class HiveDDLSuite

[GitHub] spark issue #22179: [SPARK-23131][BUILD] Upgrade Kryo to 4.0.2

2018-09-03 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22179 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22179: [SPARK-23131][BUILD] Upgrade Kryo to 4.0.2

2018-09-03 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22179 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22179: [SPARK-23131][BUILD] Upgrade Kryo to 4.0.2

2018-09-03 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22179 Sorry @dongjoon-hyun I only reproduce one test. kryo-parametrized-type-inheritance related to language. It seems scala can't reproduce it: ```scala val ser = new KryoSerializer(new

[GitHub] spark issue #22179: [SPARK-23131][BUILD] Upgrade Kryo to 4.0.2

2018-09-01 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22179 @dongjoon-hyun I'm trying to add test cases. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21987: [SPARK-25015][BUILD] Update Hadoop 2.7 to 2.7.7

2018-08-30 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21987 It seems that this change caused permission issue: ``` export HADOOP_PROXY_USER=user_a spark-sql ``` It will create dir `/tmp/hive-$%7Buser.name%7D/user_a/`. then change to other

[GitHub] spark pull request #22124: [SPARK-25135][SQL] Insert datasource table may al...

2018-08-30 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/22124 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-30 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22124 close it. I have create a new PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22287: [SPARK-25135][SQL] FileFormatWriter should respec...

2018-08-30 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22287 [SPARK-25135][SQL] FileFormatWriter should respect the schema of Hive ## What changes were proposed in this pull request? This pr fix `FileFormatWriter's dataSchema` should respect

[GitHub] spark issue #22267: [SPARK-24716][TESTS][FOLLOW-UP] Test Hive metastore sche...

2018-08-29 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22267 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22267: [SPARK-24716][TESTS][FOLLOW-UP] Test Hive metasto...

2018-08-29 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22267 [SPARK-24716][TESTS][FOLLOW-UP] Test Hive metastore schema and parquet schema are in different letter cases ## What changes were proposed in this pull request? Since https://github.com

[GitHub] spark issue #22250: [SPARK-25259][SQL] left/right join support push down dur...

2018-08-29 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22250 Fixed by [SPARK-21479](https://issues.apache.org/jira/browse/SPARK-21479). --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #22250: [SPARK-25259][SQL] left/right join support push d...

2018-08-29 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/22250 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22263: [SPARK-25269][SQL] SQL interface support specify Storage...

2018-08-29 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22263 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #22263: [SPARK-25269][SQL] SQL interface support specify ...

2018-08-29 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22263 [SPARK-25269][SQL] SQL interface support specify StorageLevel when cache table ## What changes were proposed in this pull request? SQL interface support specify `StorageLevel` when cache

[GitHub] spark issue #22250: [SPARK-25259][SQL] left/right join support push down dur...

2018-08-28 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22250 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22177: [SPARK-25119][Web UI] stages in wrong order within job p...

2018-08-28 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22177 How about backport https://github.com/apache/spark/pull/21680? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #22250: [SPARK-25259][SQL] left/right join support push down dur...

2018-08-28 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22250 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22250: [SPARK-25259][SQL] left/right join support push d...

2018-08-28 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22250 [SPARK-25259][SQL] left/right join support push down during-join predicates ## What changes were proposed in this pull request? Prepare data: ```sql create temporary view EMPLOYEE

[GitHub] spark pull request #20020: [SPARK-22834][SQL] Make insertion commands have r...

2018-08-26 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20020#discussion_r212850054 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/DataWritingCommand.scala --- @@ -20,30 +20,32 @@ package

[GitHub] spark issue #22179: [SPARK-23131][BUILD] Upgrade Kryo to 4.0.2

2018-08-22 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22179 Thanks @srowen [SPARK-25176](https://issues.apache.org/jira/browse/SPARK-25176) has a detail description: > I'm using the latest spark version spark-core_2.11:2.3.1 which transitiv

[GitHub] spark issue #22179: [SPARK-23131][BUILD] Upgrade Kryo to 4.0.2

2018-08-22 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22179 cc @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22179: [SPARK-23131][BUILD] Upgrade Kryo to 4.0.2

2018-08-21 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22179 [SPARK-23131][BUILD] Upgrade Kryo to 4.0.2 ## What changes were proposed in this pull request? Upgrade chill to 0.9.3, Kryo to 4.0.2, to get bug fixes and improvements. More details

[GitHub] spark issue #22159: [BUILD] Close stale PRs

2018-08-21 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22159 Please add https://github.com/apache/spark/pull/18424, It has been fixed. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #22124: [SPARK-25135][SQL] Insert datasource table may al...

2018-08-20 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22124#discussion_r211447728 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -384,7 +384,12 @@ object RemoveRedundantAliases

[GitHub] spark pull request #22124: [SPARK-25135][SQL] Insert datasource table may al...

2018-08-20 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22124#discussion_r211208353 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -384,7 +384,12 @@ object RemoveRedundantAliases

[GitHub] spark issue #22148: [SPARK-25132][SQL] Case-insensitive field resolution whe...

2018-08-20 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22148 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-18 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22124 Comes from: [[SPARK-22834][SQL] Make insertion commands have real children to fix UI issues](https://github.com/apache/spark/pull/20020

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-18 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22124 The root project should be consistent with the schema of the target table. But now it is inconsistent. **Before this PR**: [dataColumns](https://github.com/apache/spark/blob

<    1   2   3   4   5   6   7   8   9   10   >