[GitHub] spark pull request #22754: [SPARK-25776][CORE]The disk write buffer size mus...

2018-10-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22754#discussion_r228601036 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillWriter.java --- @@ -62,6 +62,8 @@ public UnsafeSorterSpillWriter

[GitHub] spark pull request #22847: [SPARK-25850][SQL] Make the split threshold for t...

2018-10-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22847#discussion_r228598058 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -812,6 +812,17 @@ object SQLConf { .intConf

[GitHub] spark issue #22818: [SPARK-25827][CORE] Allocate arrays smaller than Int.Max...

2018-10-25 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22818 Thanks, would it be also possible to double-check `Integer.MAX_VALUE` if you have not checked yet? --- - To unsubscribe, e-mail

[GitHub] spark pull request #22754: [SPARK-25776][CORE]The disk write buffer size mus...

2018-10-24 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22754#discussion_r227780781 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -495,8 +495,8 @@ package object config { ConfigBuilder

[GitHub] spark pull request #22754: [SPARK-25776][CORE]The disk write buffer size mus...

2018-10-24 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22754#discussion_r227729436 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -495,8 +495,8 @@ package object config { ConfigBuilder

[GitHub] spark issue #22798: [SPARK-25803] Fix docker-image-tool.sh -n option

2018-10-23 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22798 Based on bash syntax, this change makes sense. I would like to wait for @vanzin 's comment. --- - To unsubscribe, e-mail: reviews

[GitHub] spark pull request #22754: [SPARK-25776][CORE]The disk write buffer size mus...

2018-10-23 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22754#discussion_r227363331 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -495,8 +495,8 @@ package object config { ConfigBuilder

[GitHub] spark issue #22803: change jsr305 version from 1.3.9 to 3.0.0

2018-10-23 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22803 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Sorry for my mistake. My keyboard '4' sometimes has a trouble. > I think, INT_MAX is 2147483647, so n ~= sqrt(2*2147483647) = 65

[GitHub] spark issue #22800: [SPARK-24499][SQL][DOC][follow-up] Fix spelling in doc

2018-10-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22800 cc @cloud-fan @gatorsmile @HyukjinKwon @xuanyuanking --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22800: [SPARK-24499][SQL][DOC][follow-up] Fix spelling i...

2018-10-22 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/22800 [SPARK-24499][SQL][DOC][follow-up] Fix spelling in doc ## What changes were proposed in this pull request? This PR replaces `turing` with `tuning` in files and a file name. Currently

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 One question: After this PR, what is the maximum column that we can accept? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22789: [SPARK-25767][SQL] fix inputVars preparation if outputVa...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22789 Thank you for submitting a PR. Would it be possible to add a test case, too? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #22754: [SPARK-25776][CORE][MINOR]The disk write buffer s...

2018-10-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22754#discussion_r226872137 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -495,8 +495,8 @@ package object config { ConfigBuilder

[GitHub] spark issue #22754: [SPARK-25776][CORE][MINOR]The disk write buffer size mus...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22754 Thank you for your clarification. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22754: [SPARK-25776][CORE][MINOR]The disk write buffer s...

2018-10-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22754#discussion_r226871894 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillWriter.java --- @@ -62,6 +62,8 @@ public UnsafeSorterSpillWriter

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22765: [SPARK-25757][Build] Upgrade netty-all from 4.1.17.Final...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22765 LGTM, pending Jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #22765: [SPARK-25757][Build] Upgrade netty-all from 4.1.17.Final...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22765 Retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22782: [HOTFIX] Fix PySpark pip packaging tests by non-ascii co...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22782 I will visit here tomorrow morning in Japan. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22782: [HOTFIX] Fix PySpark pip packaging tests by non-ascii co...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22782 LGTM, pending Jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #22782: [HOTFIX] Fix PySpark pip packaging tests by non-ascii co...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22782 Thank you for this hot fix. I found `0xc2` after `#` in `docker-image-tool.sh` where @HyukjinKwon fixed. ``` > git log | head -1 commit fc9ba9dcc6ad47fbd05f093b94e7e1358

[GitHub] spark pull request #22782: [HOTFIX] Fix PySpark pip packaging tests by non-a...

2018-10-20 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22782#discussion_r226833103 --- Diff: dev/run-tests.py --- @@ -551,7 +551,8 @@ def main(): if not changed_files or any(f.endswith(".

[GitHub] spark pull request #22782: [HOTFIX] Fix PySpark pip packaging tests by non-a...

2018-10-20 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22782#discussion_r226833042 --- Diff: python/pyspark/__init__.py --- @@ -16,7 +16,7 @@ # """ -PySpark is the Python API for Spark. +PySpark is

[GitHub] spark pull request #22782: [HOTFIX] Fix PySpark pip packaging tests by non-a...

2018-10-20 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22782#discussion_r226833028 --- Diff: dev/run-tests.py --- @@ -551,7 +551,8 @@ def main(): if not changed_files or any(f.endswith(".

[GitHub] spark issue #22501: [SPARK-25492][TEST] Refactor WideSchemaBenchmark to use ...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22501 Thanks, I found `0xc2` in `docker-image-tool.sh`. I will put my finding into #22782 --- - To unsubscribe, e-mail: reviews

[GitHub] spark issue #22501: [SPARK-25492][TEST] Refactor WideSchemaBenchmark to use ...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22501 Is [this](https://github.com/apache/spark/pull/22748#issuecomment-431512558) the oldest test failure related to this type of failure

[GitHub] spark issue #22501: [SPARK-25492][TEST] Refactor WideSchemaBenchmark to use ...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22501 Thanks, when it was successful, this is a part of log from [this](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97378/consoleText) ``` copying pyspark/streaming/util.py

[GitHub] spark issue #22501: [SPARK-25492][TEST] Refactor WideSchemaBenchmark to use ...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22501 I am looking at each commit from the latest to old at https://github.com/apache/spark/commits/master --- - To unsubscribe, e

[GitHub] spark issue #22750: [SPARK-25747][SQL] remove ColumnarBatchScan.needsUnsafeR...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22750 You are right. Sorry, nvm > DataSourceScanExec does not have needsUnsafeRowConversion --- - To unsubscribe, e-mail: revi

[GitHub] spark issue #22750: [SPARK-25747][SQL] remove ColumnarBatchScan.needsUnsafeR...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22750 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22750: [SPARK-25747][SQL] remove ColumnarBatchScan.needsUnsafeR...

2018-10-19 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22750 I thought about the last line > This PR removes ColumnarBatchScan.needsUnsafeRowConversion, and keep this flag only in FileSourceScanE

[GitHub] spark issue #22750: [SPARK-25747][SQL] remove ColumnarBatchScan.needsUnsafeR...

2018-10-18 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22750 Do we need to update the description? For example, `needsUnsafeRowConversion` exists in `DataSourceScanExec` now

[GitHub] spark issue #22754: [MINOR][CORE]The disk write buffer size must be greater ...

2018-10-18 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22754 Good catch. One question: can we set `12` into this property? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226263066 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -0,0 +1,520 @@ +--- +layout: global +title: Spark SQL Upgrading Guide +displayTitle

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226262995 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -0,0 +1,520 @@ +--- +layout: global +title: Spark SQL Upgrading Guide +displayTitle

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226250306 --- Diff: docs/sql-performance-turing.md --- @@ -0,0 +1,151 @@ +--- +layout: global +title: Performance Tuning +displayTitle: Performance

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226247607 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -0,0 +1,520 @@ +--- +layout: global +title: Spark SQL Upgrading Guide +displayTitle

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226246375 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -0,0 +1,520 @@ +--- +layout: global +title: Spark SQL Upgrading Guide +displayTitle

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226245945 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -0,0 +1,520 @@ +--- +layout: global +title: Spark SQL Upgrading Guide +displayTitle

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226241683 --- Diff: docs/sql-distributed-sql-engine.md --- @@ -0,0 +1,85 @@ +--- +layout: global +title: Distributed SQL Engine +displayTitle

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226239048 --- Diff: docs/sql-data-sources-parquet.md --- @@ -0,0 +1,321 @@ +--- +layout: global +title: Parquet Files +displayTitle: Parquet Files

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226237047 --- Diff: docs/sql-data-sources-parquet.md --- @@ -0,0 +1,321 @@ +--- +layout: global +title: Parquet Files +displayTitle: Parquet Files

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226235672 --- Diff: docs/sql-data-sources-load-save-functions.md --- @@ -0,0 +1,283 @@ +--- +layout: global +title: Generic Load/Save Functions

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226231876 --- Diff: docs/sql-data-sources-jdbc.md --- @@ -0,0 +1,223 @@ +--- +layout: global +title: JDBC To Other Databases +displayTitle: JDBC

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226227872 --- Diff: docs/sql-data-sources.md --- @@ -0,0 +1,42 @@ +--- +layout: global +title: Data Sources +displayTitle: Data Sources

[GitHub] spark issue #22755: [SPARK-25755][SQL][Test] Supplementation of non-CodeGen ...

2018-10-18 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22755 cc @maropu --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsa...

2018-10-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22617 Retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22729: [SPARK-25737][CORE] Remove JavaSparkContextVarargsWorkar...

2018-10-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22729 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22705: [SPARK-25704][CORE] Allocate a bit less than Int.MaxValu...

2018-10-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22705 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22705: [SPARK-25704][CORE] Allocate a bit less than Int.MaxValu...

2018-10-16 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22705 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22708: [SPARK-21402][SQL] Fix java array of structs deserializa...

2018-10-16 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22708 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22708: [SPARK-21402] Fix java array/map of structs deser...

2018-10-16 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22708#discussion_r225440522 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -30,6 +30,7 @@ import

[GitHub] spark pull request #22662: [SPARK-25627][TEST] Reduce test time for Continuo...

2018-10-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22662#discussion_r225000849 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -259,10 +259,10 @@ class ContinuousStressSuite

[GitHub] spark issue #21537: [SPARK-24505][SQL] Convert strings in codegen to blocks:...

2018-10-14 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21537 @HyukjinKwon sorry for being late. I was swampped with several things. I have just submitted it. Looking forward to seeing feedback

[GitHub] spark pull request #22662: [SPARK-25627][TEST] Reduce test time for Continuo...

2018-10-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22662#discussion_r224977443 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -259,10 +259,10 @@ class ContinuousStressSuite

[GitHub] spark pull request #22705: [SPARK-25704][CORE][WIP] Allocate a bit less than...

2018-10-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22705#discussion_r224977082 --- Diff: core/src/main/scala/org/apache/spark/util/io/ChunkedByteBuffer.scala --- @@ -195,7 +196,11 @@ object ChunkedByteBuffer { val is = new

[GitHub] spark issue #22711: [SPARK-25714][SQL][followup] improve the comment inside ...

2018-10-13 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22711 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22698: [SPARK-25710][SQL] range should report metrics correctly

2018-10-12 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22698 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22678: [SPARK-25685][BUILD] Allow running tests in Jenkins in e...

2018-10-12 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22678 LGTM, pending Jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #22678: [SPARK-25685][BUILD] Allow running tests in Jenkins in e...

2018-10-12 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22678 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22678: [SPARK-25685][BUILD] Allow running tests in Jenkins in e...

2018-10-12 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22678 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22678: [SPARK-25685][BUILD] Allow running tests in Jenki...

2018-10-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22678#discussion_r224549100 --- Diff: docs/building-spark.md --- @@ -272,3 +272,31 @@ For SBT, specify a complete scala version using (e.g. 2.12.6): ./build/sbt -Dscala.version

[GitHub] spark pull request #22678: [SPARK-25685][BUILD] Allow running tests in Jenki...

2018-10-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22678#discussion_r224549012 --- Diff: docs/building-spark.md --- @@ -272,3 +272,31 @@ For SBT, specify a complete scala version using (e.g. 2.12.6): ./build/sbt -Dscala.version

[GitHub] spark issue #22657: [SPARK-25670][TEST] Reduce number of tested timezones in...

2018-10-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22657 Sorry for bothering you again. Do we need to apply the same reduction to `CastSuite`? --- - To unsubscribe, e-mail: reviews

[GitHub] spark issue #22657: [SPARK-25670][TEST] Reduce number of tested timezones in...

2018-10-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22657 How about defining this subset in `object DateTimeTestUtils` like `ALL_TIMEZONES`? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #22657: [SPARK-25670][TEST] Reduce number of tested timezones in...

2018-10-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22657 Do we need to apply the same reduction to `CastSuite`? I believe that `DateTimeUtilsSuite` should check all of the time zones, but may skip some values between`-2` to `2

[GitHub] spark issue #22347: [SPARK-25353][SQL] executeTake in SparkPlan is modified ...

2018-10-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22347 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22678: [SPARK-25685][BUILD] Allow running tests in Jenki...

2018-10-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22678#discussion_r22445 --- Diff: docs/building-spark.md --- @@ -272,3 +272,31 @@ For SBT, specify a complete scala version using (e.g. 2.12.6): ./build/sbt -Dscala.version

[GitHub] spark pull request #22678: [SPARK-25685][BUILD] Allow running tests in Jenki...

2018-10-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22678#discussion_r224333028 --- Diff: docs/building-spark.md --- @@ -272,3 +272,31 @@ For SBT, specify a complete scala version using (e.g. 2.12.6): ./build/sbt -Dscala.version

[GitHub] spark pull request #22678: [SPARK-25685][BUILD] Allow running tests in Jenki...

2018-10-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22678#discussion_r224332828 --- Diff: docs/building-spark.md --- @@ -272,3 +272,31 @@ For SBT, specify a complete scala version using (e.g. 2.12.6): ./build/sbt -Dscala.version

[GitHub] spark pull request #22678: [SPARK-25685][BUILD] Allow running tests in Jenki...

2018-10-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22678#discussion_r224332984 --- Diff: docs/building-spark.md --- @@ -272,3 +272,31 @@ For SBT, specify a complete scala version using (e.g. 2.12.6): ./build/sbt -Dscala.version

[GitHub] spark pull request #22678: [SPARK-25685][BUILD] Allow running tests in Jenki...

2018-10-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22678#discussion_r223917766 --- Diff: dev/run-tests-jenkins.py --- @@ -39,7 +39,8 @@ def print_err(msg): def post_message_to_github(msg, ghprb_pull_id): print("Attem

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-10-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r223783598 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -221,6 +227,12 @@ trait

[GitHub] spark issue #22630: [SPARK-25497][SQL] Limit operation within whole stage co...

2018-10-09 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22630 Thanks! merging to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22630: [SPARK-25497][SQL] Limit operation within whole stage co...

2018-10-09 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22630 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22630: [SPARK-25497][SQL] Limit operation within whole s...

2018-10-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22630#discussion_r223478828 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -452,46 +452,68 @@ case class RangeExec(range

[GitHub] spark pull request #22630: [SPARK-25497][SQL] Limit operation within whole s...

2018-10-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22630#discussion_r223474906 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -345,6 +345,61 @@ trait CodegenSupport extends SparkPlan

[GitHub] spark pull request #22630: [SPARK-25497][SQL] Limit operation within whole s...

2018-10-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22630#discussion_r223474853 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -452,46 +452,73 @@ case class RangeExec(range

[GitHub] spark pull request #22630: [SPARK-25497][SQL] Limit operation within whole s...

2018-10-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22630#discussion_r223468018 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala --- @@ -518,56 +521,81 @@ class SQLMetricsSuite extends

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-10-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r223214130 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -221,6 +227,12 @@ trait

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-10-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r223169695 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -221,6 +227,12 @@ trait

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-10-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r223169637 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CodeGenerationSuite.scala --- @@ -113,7 +113,7 @@ class CodeGenerationSuite

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-10-04 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r222741323 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -221,6 +227,12 @@ trait

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-10-04 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r222740882 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -69,11 +69,17 @@ trait

[GitHub] spark issue #22375: [SPARK-25388][Test][SQL] Detect incorrect nullable of Da...

2018-10-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22375 ping @cloud-fan @mgaido91 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

2018-10-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22602 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

<    1   2   3   4   5   6   7   8   9   10   >