[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178427033 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/CreateJacksonParser.scala --- @@ -39,11 +40,36 @@ private[sql] object CreateJa

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178427011 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2065,29 +2065,238 @@ class JsonSuite extends

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178426994 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2065,29 +2065,238 @@ class JsonSuite extends

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178426903 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2065,29 +2065,238 @@ class JsonSuite extends

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178426879 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala --- @@ -86,14 +86,30 @@ private[sql] class JSONOptions(

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178426866 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2065,29 +2065,238 @@ class JsonSuite extends

[GitHub] spark issue #20849: [SPARK-23723] New charset option for json datasource

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20849 Let's make the point clear. There are two things, _1. one for line-by-line parsing_ and _2. JSON parsing via Jackson_. The test you pointed out looks still a bit weird because Jackson is

[GitHub] spark issue #20849: [SPARK-23723] New charset option for json datasource

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20849 From a quick look and wild guess, `UTF-16` case would be alone problematic because we are going to make the delimiter with a BOM bit `0xFF 0xFE 0x0D 0x00 0x0A 0x00`. --- -

[GitHub] spark issue #20849: [SPARK-23723] New charset option for json datasource

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20849 @MaxGekk, So to make it clear, it parses line by line correctly regardless of BOM if we set `lineSep` + `encoding` fine but it fails to parse each line as JSON via Jackson since we explicitly se

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18853 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18853 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1882/ Tes

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18853 **[Test build #88780 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88780/testReport)** for PR 18853 at commit [`81067b9`](https://github.com/apache/spark/commit/81

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178425464 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/ColumnStatsSuite.scala --- @@ -18,18 +18,35 @@ package org.apache.spark.sql.e

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178425406 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala --- @@ -323,18 +324,31 @@ private[columnar] final class Decimal

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178425525 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/ColumnarTestUtils.scala --- @@ -54,12 +54,22 @@ object ColumnarTestUtils {

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2018-03-31 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/18853 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #20954: [BACKPORT][SPARK-23040][CORE] Returns interruptible iter...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20954 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88777/ Test FAILed. ---

[GitHub] spark issue #20954: [BACKPORT][SPARK-23040][CORE] Returns interruptible iter...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20954 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18853 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18853 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88778/ Test FAILed. ---

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18853 **[Test build #88778 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88778/testReport)** for PR 18853 at commit [`81067b9`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #20954: [BACKPORT][SPARK-23040][CORE] Returns interruptible iter...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20954 **[Test build #88777 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88777/testReport)** for PR 20954 at commit [`e9a661f`](https://github.com/apache/spark/commit/e

<    1   2