[GitHub] spark issue #22818: [SPARK-25904][CORE] Allocate arrays smaller than Int.Max...

2018-11-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22818 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22754: [SPARK-25776][CORE]The disk write buffer size must be gr...

2018-11-04 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22754 Thanks! merging to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

spark git commit: [SPARK-25776][CORE]The disk write buffer size must be greater than 12

2018-11-04 Thread kiszk
Repository: spark Updated Branches: refs/heads/master 463a67668 -> 6c9e5ac9d [SPARK-25776][CORE]The disk write buffer size must be greater than 12 ## What changes were proposed in this pull request? In `UnsafeSorterSpillWriter.java`, when we write a record to a spill file wtih ` void

[GitHub] spark issue #22818: [SPARK-25904][CORE] Allocate arrays smaller than Int.Max...

2018-11-04 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22818 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22818: [SPARK-25904][CORE] Allocate arrays smaller than Int.Max...

2018-11-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22818 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22754: [SPARK-25776][CORE]The disk write buffer size must be gr...

2018-11-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22754 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22754: [SPARK-25776][CORE]The disk write buffer size must be gr...

2018-11-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22754 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22898: [SPARK-25746][SQL][followup] do not add unnecessary If e...

2018-10-31 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22898 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22847: [SPARK-25850][SQL] Make the split threshold for t...

2018-10-31 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22847#discussion_r229577559 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -812,6 +812,18 @@ object SQLConf { .intConf

[GitHub] spark pull request #22847: [SPARK-25850][SQL] Make the split threshold for t...

2018-10-31 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22847#discussion_r229577345 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -812,6 +812,17 @@ object SQLConf { .intConf

[GitHub] spark issue #22891: SPARK-25881

2018-10-30 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22891 Thank you for your contribution. Could you please write appropriate title and descriptions based on http://spark.apache.org/contributing.html

[GitHub] spark pull request #22881: [SPARK-25855][CORE] Don't use erasure coding for ...

2018-10-29 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22881#discussion_r229155491 --- Diff: docs/configuration.md --- @@ -761,6 +761,17 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request #22881: [SPARK-25855][CORE] Don't use erasure coding for ...

2018-10-29 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22881#discussion_r229154733 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala --- @@ -471,4 +473,42 @@ object SparkHadoopUtil { hadoopConf.set

[GitHub] spark pull request #22877: [MINOR][SQL] Avoid hardcoded configuration keys i...

2018-10-29 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22877#discussion_r229148363 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -408,15 +408,16 @@ object SQLConf { val

[GitHub] spark issue #22879: [SPARK-25872][SQL][TEST] Add an optimizer tracker for TP...

2018-10-29 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22879 cc @maropu --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22877: [MINOR][SQL] Avoid hardcoded configuration keys i...

2018-10-29 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22877#discussion_r229034778 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -408,15 +408,16 @@ object SQLConf { val

[GitHub] spark issue #22755: [SPARK-25755][SQL][Test] Supplementation of non-CodeGen ...

2018-10-28 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22755 Is it better to apply this util method to others (e.g. `DataFrameRangeSuite.scala` and `DataFrameAggregateSuite.scala

[GitHub] spark issue #22818: [SPARK-25827][CORE] Allocate arrays smaller than Int.Max...

2018-10-28 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22818 Since this PR is not a blocker for 2.4, I think that it would be good to address these issues as possible. --- - To unsubscribe

[GitHub] spark pull request #22754: [SPARK-25776][CORE]The disk write buffer size mus...

2018-10-28 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22754#discussion_r228738630 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillWriter.java --- @@ -42,7 +42,9 @@ private final

[GitHub] spark issue #19601: [SPARK-22383][SQL] Generate code to directly get value o...

2018-10-26 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19601 Sure, let me close this --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #19601: [SPARK-22383][SQL] Generate code to directly get ...

2018-10-26 Thread kiszk
Github user kiszk closed the pull request at: https://github.com/apache/spark/pull/19601 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22754: [SPARK-25776][CORE]The disk write buffer size mus...

2018-10-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22754#discussion_r228601036 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillWriter.java --- @@ -62,6 +62,8 @@ public UnsafeSorterSpillWriter

[GitHub] spark pull request #22847: [SPARK-25850][SQL] Make the split threshold for t...

2018-10-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22847#discussion_r228598058 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -812,6 +812,17 @@ object SQLConf { .intConf

[GitHub] spark issue #22818: [SPARK-25827][CORE] Allocate arrays smaller than Int.Max...

2018-10-25 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22818 Thanks, would it be also possible to double-check `Integer.MAX_VALUE` if you have not checked yet? --- - To unsubscribe, e-mail

[GitHub] spark pull request #22754: [SPARK-25776][CORE]The disk write buffer size mus...

2018-10-24 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22754#discussion_r227780781 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -495,8 +495,8 @@ package object config { ConfigBuilder

[GitHub] spark pull request #22754: [SPARK-25776][CORE]The disk write buffer size mus...

2018-10-24 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22754#discussion_r227729436 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -495,8 +495,8 @@ package object config { ConfigBuilder

[GitHub] spark issue #22798: [SPARK-25803] Fix docker-image-tool.sh -n option

2018-10-23 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22798 Based on bash syntax, this change makes sense. I would like to wait for @vanzin 's comment. --- - To unsubscribe, e-mail: reviews

[GitHub] spark pull request #22754: [SPARK-25776][CORE]The disk write buffer size mus...

2018-10-23 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22754#discussion_r227363331 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -495,8 +495,8 @@ package object config { ConfigBuilder

[GitHub] spark issue #22803: change jsr305 version from 1.3.9 to 3.0.0

2018-10-23 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22803 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Sorry for my mistake. My keyboard '4' sometimes has a trouble. > I think, INT_MAX is 2147483647, so n ~= sqrt(2*2147483647) = 65

[GitHub] spark issue #22800: [SPARK-24499][SQL][DOC][follow-up] Fix spelling in doc

2018-10-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22800 cc @cloud-fan @gatorsmile @HyukjinKwon @xuanyuanking --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22800: [SPARK-24499][SQL][DOC][follow-up] Fix spelling i...

2018-10-22 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/22800 [SPARK-24499][SQL][DOC][follow-up] Fix spelling in doc ## What changes were proposed in this pull request? This PR replaces `turing` with `tuning` in files and a file name. Currently

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 One question: After this PR, what is the maximum column that we can accept? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 Can I clarify the description? > Because we are passing an array of size n*(n+1)/2 to the breeze library and the size cannot be more than INT_MAX. so, the maximum column size we can g

[GitHub] spark issue #22789: [SPARK-25767][SQL] fix inputVars preparation if outputVa...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22789 Thank you for submitting a PR. Would it be possible to add a test case, too? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #22754: [SPARK-25776][CORE][MINOR]The disk write buffer s...

2018-10-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22754#discussion_r226872137 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -495,8 +495,8 @@ package object config { ConfigBuilder

[GitHub] spark issue #22754: [SPARK-25776][CORE][MINOR]The disk write buffer size mus...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22754 Thank you for your clarification. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22754: [SPARK-25776][CORE][MINOR]The disk write buffer s...

2018-10-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22754#discussion_r226871894 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillWriter.java --- @@ -62,6 +62,8 @@ public UnsafeSorterSpillWriter

[GitHub] spark issue #22784: [SPARK-25790][MLLIB] PCA: Support more than 65535 column...

2018-10-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22784 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22765: [SPARK-25757][Build] Upgrade netty-all from 4.1.17.Final...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22765 LGTM, pending Jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #22765: [SPARK-25757][Build] Upgrade netty-all from 4.1.17.Final...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22765 Retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22782: [HOTFIX] Fix PySpark pip packaging tests by non-ascii co...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22782 I will visit here tomorrow morning in Japan. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22782: [HOTFIX] Fix PySpark pip packaging tests by non-ascii co...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22782 LGTM, pending Jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #22782: [HOTFIX] Fix PySpark pip packaging tests by non-ascii co...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22782 Thank you for this hot fix. I found `0xc2` after `#` in `docker-image-tool.sh` where @HyukjinKwon fixed. ``` > git log | head -1 commit fc9ba9dcc6ad47fbd05f093b94e7e1358

[GitHub] spark pull request #22782: [HOTFIX] Fix PySpark pip packaging tests by non-a...

2018-10-20 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22782#discussion_r226833103 --- Diff: dev/run-tests.py --- @@ -551,7 +551,8 @@ def main(): if not changed_files or any(f.endswith(".

[GitHub] spark pull request #22782: [HOTFIX] Fix PySpark pip packaging tests by non-a...

2018-10-20 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22782#discussion_r226833042 --- Diff: python/pyspark/__init__.py --- @@ -16,7 +16,7 @@ # """ -PySpark is the Python API for Spark. +PySpark is

[GitHub] spark pull request #22782: [HOTFIX] Fix PySpark pip packaging tests by non-a...

2018-10-20 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22782#discussion_r226833028 --- Diff: dev/run-tests.py --- @@ -551,7 +551,8 @@ def main(): if not changed_files or any(f.endswith(".

[GitHub] spark issue #22501: [SPARK-25492][TEST] Refactor WideSchemaBenchmark to use ...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22501 Thanks, I found `0xc2` in `docker-image-tool.sh`. I will put my finding into #22782 --- - To unsubscribe, e-mail: reviews

[GitHub] spark issue #22501: [SPARK-25492][TEST] Refactor WideSchemaBenchmark to use ...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22501 Is [this](https://github.com/apache/spark/pull/22748#issuecomment-431512558) the oldest test failure related to this type of failure

[GitHub] spark issue #22501: [SPARK-25492][TEST] Refactor WideSchemaBenchmark to use ...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22501 Thanks, when it was successful, this is a part of log from [this](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97378/consoleText) ``` copying pyspark/streaming/util.py

[GitHub] spark issue #22501: [SPARK-25492][TEST] Refactor WideSchemaBenchmark to use ...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22501 I am looking at each commit from the latest to old at https://github.com/apache/spark/commits/master --- - To unsubscribe, e

[GitHub] spark issue #22750: [SPARK-25747][SQL] remove ColumnarBatchScan.needsUnsafeR...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22750 You are right. Sorry, nvm > DataSourceScanExec does not have needsUnsafeRowConversion --- - To unsubscribe, e-mail: revi

[GitHub] spark issue #22750: [SPARK-25747][SQL] remove ColumnarBatchScan.needsUnsafeR...

2018-10-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22750 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22750: [SPARK-25747][SQL] remove ColumnarBatchScan.needsUnsafeR...

2018-10-19 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22750 I thought about the last line > This PR removes ColumnarBatchScan.needsUnsafeRowConversion, and keep this flag only in FileSourceScanE

[GitHub] spark issue #22750: [SPARK-25747][SQL] remove ColumnarBatchScan.needsUnsafeR...

2018-10-18 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22750 Do we need to update the description? For example, `needsUnsafeRowConversion` exists in `DataSourceScanExec` now

[GitHub] spark issue #22754: [MINOR][CORE]The disk write buffer size must be greater ...

2018-10-18 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22754 Good catch. One question: can we set `12` into this property? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226263066 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -0,0 +1,520 @@ +--- +layout: global +title: Spark SQL Upgrading Guide +displayTitle

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226262995 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -0,0 +1,520 @@ +--- +layout: global +title: Spark SQL Upgrading Guide +displayTitle

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226250306 --- Diff: docs/sql-performance-turing.md --- @@ -0,0 +1,151 @@ +--- +layout: global +title: Performance Tuning +displayTitle: Performance

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226247607 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -0,0 +1,520 @@ +--- +layout: global +title: Spark SQL Upgrading Guide +displayTitle

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226246375 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -0,0 +1,520 @@ +--- +layout: global +title: Spark SQL Upgrading Guide +displayTitle

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226245945 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -0,0 +1,520 @@ +--- +layout: global +title: Spark SQL Upgrading Guide +displayTitle

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226241683 --- Diff: docs/sql-distributed-sql-engine.md --- @@ -0,0 +1,85 @@ +--- +layout: global +title: Distributed SQL Engine +displayTitle

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226239048 --- Diff: docs/sql-data-sources-parquet.md --- @@ -0,0 +1,321 @@ +--- +layout: global +title: Parquet Files +displayTitle: Parquet Files

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226237047 --- Diff: docs/sql-data-sources-parquet.md --- @@ -0,0 +1,321 @@ +--- +layout: global +title: Parquet Files +displayTitle: Parquet Files

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226235672 --- Diff: docs/sql-data-sources-load-save-functions.md --- @@ -0,0 +1,283 @@ +--- +layout: global +title: Generic Load/Save Functions

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226231876 --- Diff: docs/sql-data-sources-jdbc.md --- @@ -0,0 +1,223 @@ +--- +layout: global +title: JDBC To Other Databases +displayTitle: JDBC

[GitHub] spark pull request #22746: [SPARK-24499][SQL][DOC] Split the page of sql-pro...

2018-10-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22746#discussion_r226227872 --- Diff: docs/sql-data-sources.md --- @@ -0,0 +1,42 @@ +--- +layout: global +title: Data Sources +displayTitle: Data Sources

[GitHub] spark issue #22755: [SPARK-25755][SQL][Test] Supplementation of non-CodeGen ...

2018-10-18 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22755 cc @maropu --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsa...

2018-10-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22617 Retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22729: [SPARK-25737][CORE] Remove JavaSparkContextVarargsWorkar...

2018-10-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22729 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22705: [SPARK-25704][CORE] Allocate a bit less than Int.MaxValu...

2018-10-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22705 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22705: [SPARK-25704][CORE] Allocate a bit less than Int.MaxValu...

2018-10-16 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22705 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22708: [SPARK-21402][SQL] Fix java array of structs deserializa...

2018-10-16 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22708 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22708: [SPARK-21402] Fix java array/map of structs deser...

2018-10-16 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22708#discussion_r225440522 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -30,6 +30,7 @@ import

[GitHub] spark pull request #22662: [SPARK-25627][TEST] Reduce test time for Continuo...

2018-10-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22662#discussion_r225000849 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -259,10 +259,10 @@ class ContinuousStressSuite

[GitHub] spark issue #21537: [SPARK-24505][SQL] Convert strings in codegen to blocks:...

2018-10-14 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21537 @HyukjinKwon sorry for being late. I was swampped with several things. I have just submitted it. Looking forward to seeing feedback

[GitHub] spark pull request #22662: [SPARK-25627][TEST] Reduce test time for Continuo...

2018-10-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22662#discussion_r224977443 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -259,10 +259,10 @@ class ContinuousStressSuite

[GitHub] spark pull request #22705: [SPARK-25704][CORE][WIP] Allocate a bit less than...

2018-10-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22705#discussion_r224977082 --- Diff: core/src/main/scala/org/apache/spark/util/io/ChunkedByteBuffer.scala --- @@ -195,7 +196,11 @@ object ChunkedByteBuffer { val is = new

[GitHub] spark issue #22711: [SPARK-25714][SQL][followup] improve the comment inside ...

2018-10-13 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22711 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22698: [SPARK-25710][SQL] range should report metrics correctly

2018-10-12 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22698 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22678: [SPARK-25685][BUILD] Allow running tests in Jenkins in e...

2018-10-12 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22678 LGTM, pending Jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #22678: [SPARK-25685][BUILD] Allow running tests in Jenkins in e...

2018-10-12 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22678 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22678: [SPARK-25685][BUILD] Allow running tests in Jenkins in e...

2018-10-12 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22678 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22678: [SPARK-25685][BUILD] Allow running tests in Jenki...

2018-10-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22678#discussion_r224549100 --- Diff: docs/building-spark.md --- @@ -272,3 +272,31 @@ For SBT, specify a complete scala version using (e.g. 2.12.6): ./build/sbt -Dscala.version

[GitHub] spark pull request #22678: [SPARK-25685][BUILD] Allow running tests in Jenki...

2018-10-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22678#discussion_r224549012 --- Diff: docs/building-spark.md --- @@ -272,3 +272,31 @@ For SBT, specify a complete scala version using (e.g. 2.12.6): ./build/sbt -Dscala.version

[GitHub] spark issue #22657: [SPARK-25670][TEST] Reduce number of tested timezones in...

2018-10-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22657 Sorry for bothering you again. Do we need to apply the same reduction to `CastSuite`? --- - To unsubscribe, e-mail: reviews

[GitHub] spark issue #22657: [SPARK-25670][TEST] Reduce number of tested timezones in...

2018-10-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22657 How about defining this subset in `object DateTimeTestUtils` like `ALL_TIMEZONES`? --- - To unsubscribe, e-mail: reviews-unsubscr

<    1   2   3   4   5   6   7   8   9   10   >