[GitHub] spark pull request #18029: [SPARK-20168] [DStream] Add changes to use kinesi...

2017-12-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18029 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20081: [SPARK-22833][EXAMPLE] Improvement SparkHive Scala Examp...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20081 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85392/ Test PASSed. ---

[GitHub] spark issue #20081: [SPARK-22833][EXAMPLE] Improvement SparkHive Scala Examp...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20081 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20081: [SPARK-22833][EXAMPLE] Improvement SparkHive Scala Examp...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20081 **[Test build #85392 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85392/testReport)** for PR 20081 at commit

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-25 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/18029 Merged to master. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20081: [SPARK-22833][EXAMPLE] Improvement SparkHive Scala Examp...

2017-12-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20081 FYI, there is a JIRA for a doc about `spark.sql.parquet.writeLegacyFormat ` - https://issues-test.apache.org/jira/plugins/servlet/mobile#issue/SPARK-20937 ---

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19813 Which one is more common? A chain of arithmetic expressions? Or a deeply nested expression? I don't see strong evidence that supports statement output from the discussion. The only one possibility

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-25 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158675633 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/init-container/Dockerfile --- @@ -0,0 +1,24 @@ +# +# Licensed to the Apache

[GitHub] spark issue #20081: [SPARK-22833][EXAMPLE] Improvement SparkHive Scala Examp...

2017-12-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20081 > spark.sql.parquet.writeLegacyFormat - if you don't use this configuration, hive external table won't be able to access parquet data. Well, that's really an undocumented feature... Can

[GitHub] spark issue #20020: [SPARK-22834][SQL] Make insertion commands have real chi...

2017-12-25 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/20020 All these insertion commands are from `postHocResolutionRules`, while there are other batches after it. Skipping the batches after `postHocResolutionRules` will cause analysis error. I

[GitHub] spark issue #20020: [SPARK-22834][SQL] Make insertion commands have real chi...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20020 **[Test build #85396 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85396/testReport)** for PR 20020 at commit

[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...

2017-12-25 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20080 cc @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19929: [SPARK-22629][PYTHON] Add deterministic flag to pyspark ...

2017-12-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19929 Could you change the JIRA number to https://issues.apache.org/jira/browse/SPARK-22901 ? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20076 **[Test build #85395 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85395/testReport)** for PR 20076 at commit

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20076 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20076 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85394/ Test FAILed. ---

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20076 **[Test build #85394 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85394/testReport)** for PR 20076 at commit

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20076 **[Test build #85394 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85394/testReport)** for PR 20076 at commit

[GitHub] spark issue #20004: [Spark-22818][SQL] csv escape of quote escape

2017-12-25 Thread ep1804
Github user ep1804 commented on the issue: https://github.com/apache/spark/pull/20004 Revision followed: - comment on the default values. - applying charToEscapeQuoteEscaping using Option type. --- - To

[GitHub] spark issue #20004: [Spark-22818][SQL] csv escape of quote escape

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20004 **[Test build #85393 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85393/testReport)** for PR 20004 at commit

[GitHub] spark issue #11994: [SPARK-14151] Expose metrics Source and Sink interface

2017-12-25 Thread CodingCat
Github user CodingCat commented on the issue: https://github.com/apache/spark/pull/11994 if I understand correctly, the only issue here is that we exposed codehale's MetricsRegistry in Sink base

[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20080 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85390/ Test PASSed. ---

[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20080 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20080 **[Test build #85390 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85390/testReport)** for PR 20080 at commit

[GitHub] spark issue #20081: [SPARK-22833][EXAMPLE] Improvement SparkHive Scala Examp...

2017-12-25 Thread chetkhatri
Github user chetkhatri commented on the issue: https://github.com/apache/spark/pull/20081 @cloud-fan spark.sql.files.maxRecordsPerFile didn't worked out when i was working with mine 30 TB of Spark Hive workload whereas repartition and coalesce made sense. ---

[GitHub] spark issue #20081: [SPARK-22833][EXAMPLE] Improvement SparkHive Scala Examp...

2017-12-25 Thread chetkhatri
Github user chetkhatri commented on the issue: https://github.com/apache/spark/pull/20081 @cloud-fan Thanks for PR 4. spark.sql.parquet.writeLegacyFormat - if you don't use this configuration, hive external table won't be able to access parquet data. 5. repartition and

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19813 I think all arithmetic, predicate and bitwise expressions can benefit from it, and they are very common expressions in SQL. More importantly, allowing expressions to output statement may have

[GitHub] spark issue #20081: [SPARK-22833][EXAMPLE] Improvement SparkHive Scala Examp...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20081 **[Test build #85392 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85392/testReport)** for PR 20081 at commit

[GitHub] spark issue #20081: [SPARK-22833][EXAMPLE] Improvement SparkHive Scala Examp...

2017-12-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20081 @chetkhatri @srowen @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20081: [SPARK-22833][EXAMPLE] Improvement SparkHive Scal...

2017-12-25 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/20081 [SPARK-22833][EXAMPLE] Improvement SparkHive Scala Examples ## What changes were proposed in this pull request? Some improvements: 1. Point out we are using both Spark SQ native syntax

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2017-12-25 Thread fjh100456
Github user fjh100456 commented on the issue: https://github.com/apache/spark/pull/20076 Well, I'll revert back the renaming. Any comments? @gatorsmile --- - To unsubscribe, e-mail:

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20076 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20076 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85388/ Test PASSed. ---

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20076 **[Test build #85388 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85388/testReport)** for PR 20076 at commit

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2017-12-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20076 Sure, let's revert back the rename then. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20067: [SPARK-22894][SQL] DateTimeOperations should accept SQL ...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20067 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20067: [SPARK-22894][SQL] DateTimeOperations should accept SQL ...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20067 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85387/ Test PASSed. ---

[GitHub] spark issue #20067: [SPARK-22894][SQL] DateTimeOperations should accept SQL ...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20067 **[Test build #85387 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85387/testReport)** for PR 20067 at commit

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19813 This is only valid when by coincidence the all expressions involved can use statement as output. As I looked at the codebase, I think only few expressions can output statement. This may not apply

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19813 I did a search but can't find one in the current codebase, but I do think this is a valid idea, e.g. a simple example would be `a + b + z`, if expressions can output statement, then we just

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2017-12-25 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/20076 Thanks for the PR. Why are we complicating the PR by doing the rename? Does this actually gain anything other than minor cosmetic changes? It makes the simple PR pretty long ... ---

[GitHub] spark pull request #20076: [SPARK-21786][SQL] When acquiring 'compressionCod...

2017-12-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20076#discussion_r158663731 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/CompressionCodecSuite.scala --- @@ -0,0 +1,61 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #20076: [SPARK-21786][SQL] When acquiring 'compressionCod...

2017-12-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20076#discussion_r158663721 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/CompressionCodecSuite.scala --- @@ -0,0 +1,61 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19527 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85391/ Test PASSed. ---

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19527 **[Test build #85391 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85391/testReport)** for PR 19527 at commit

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-12-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19527 Unit tests are reformatted too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85389/ Test PASSed. ---

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19527 **[Test build #85389 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85389/testReport)** for PR 19527 at commit

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19527 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20080 **[Test build #85390 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85390/testReport)** for PR 20080 at commit

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19527 **[Test build #85391 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85391/testReport)** for PR 19527 at commit

[GitHub] spark pull request #20080: [SPARK-22870][CORE] Dynamic allocation should all...

2017-12-25 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20080 [SPARK-22870][CORE] Dynamic allocation should allow 0 idle time ## What changes were proposed in this pull request? This pr to make `0` as a valid value for

[GitHub] spark pull request #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator f...

2017-12-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19527#discussion_r158659399 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/OneHotEncoderEstimator.scala --- @@ -0,0 +1,456 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19527 **[Test build #85389 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85389/testReport)** for PR 19527 at commit

[GitHub] spark pull request #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator f...

2017-12-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19527#discussion_r158659167 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/OneHotEncoderEstimator.scala --- @@ -0,0 +1,479 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator f...

2017-12-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19527#discussion_r158659174 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/OneHotEncoderEstimator.scala --- @@ -0,0 +1,479 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20076 **[Test build #85388 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85388/testReport)** for PR 20076 at commit

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2017-12-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20076 Retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20067: [SPARK-22894][SQL] DateTimeOperations should accept SQL ...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20067 **[Test build #85387 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85387/testReport)** for PR 20067 at commit

[GitHub] spark issue #20079: [SPARK-22893][SQL][HOTFIX] Fix a error message of Versio...

2017-12-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20079 Thanks you, @gatorsmile and @wangyum --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20067: [SPARK-22894][SQL] DateTimeOperations should accept SQL ...

2017-12-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20067 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20067: [SPARK-22894][SQL] DateTimeOperations should acce...

2017-12-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20067#discussion_r158657480 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2760,6 +2760,17 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2017-12-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20023 The problem found by this PR is just one of the issues that returns NULL when SPARK SQL is unable to process it. Below is another example. I believe we can find more. ```SQL SELECT

[GitHub] spark pull request #20079: [SPARK-22893][SQL][HOTFIX] Fix a error message of...

2017-12-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20079 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20079: [SPARK-22893][SQL][HOTFIX] Fix a error message of Versio...

2017-12-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20079 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20079: [SPARK-22893][SQL][HOTFIX] Fix a error message of Versio...

2017-12-25 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20079 LGTM, thanks @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20059 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85385/ Test PASSed. ---

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20059 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20059 **[Test build #85385 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85385/testReport)** for PR 20059 at commit

[GitHub] spark issue #19954: [SPARK-22757][Kubernetes] Enable use of remote dependenc...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19954 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85384/ Test PASSed. ---

[GitHub] spark issue #19954: [SPARK-22757][Kubernetes] Enable use of remote dependenc...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19954 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19954: [SPARK-22757][Kubernetes] Enable use of remote dependenc...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19954 **[Test build #85384 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85384/testReport)** for PR 19954 at commit

[GitHub] spark issue #20079: [SPARK-22893][SQL][HOTFIX] Fix a error message of Versio...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20079 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20079: [SPARK-22893][SQL][HOTFIX] Fix a error message of Versio...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20079 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85386/ Test PASSed. ---

[GitHub] spark issue #20079: [SPARK-22893][SQL][HOTFIX] Fix a error message of Versio...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20079 **[Test build #85386 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85386/testReport)** for PR 20079 at commit

[GitHub] spark issue #20072: [SPARK-22790][SQL] add a configurable factor to describe...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20072 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20072: [SPARK-22790][SQL] add a configurable factor to describe...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20072 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85383/ Test PASSed. ---

[GitHub] spark issue #20072: [SPARK-22790][SQL] add a configurable factor to describe...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20072 **[Test build #85383 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85383/testReport)** for PR 20072 at commit

[GitHub] spark issue #19929: [SPARK-22629][PYTHON] Add deterministic flag to pyspark ...

2017-12-25 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19929 @gatorsmile, yes, the reason why seed doesn't work is in the way Python UDFs are executed, i.e. a new python process is created for each partition to evaluate the Python UDF. Thus the seed is set

[GitHub] spark issue #20079: [SPARK-22893][SQL][HOTFIX] Fix a error message of Versio...

2017-12-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20079 Thank you, @gatorsmile ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20079: [SPARK-22893][SQL][HOTFIX] Fix a error message of Versio...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20079 **[Test build #85386 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85386/testReport)** for PR 20079 at commit

[GitHub] spark issue #20079: [SPARK-22893][SQL][HOTFIX] Fix a error message of Versio...

2017-12-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20079 cc @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20064: [SPARK-22893][SQL] Unified the data type mismatch messag...

2017-12-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20064 Hi, @gatorsmile and @wangyum . This PR seems to break Jenkins tests. Please see my hotfix. --- - To unsubscribe,

[GitHub] spark pull request #20079: [SPARK-22893][SQL][HOTFIX] Fix a error message of...

2017-12-25 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/20079 [SPARK-22893][SQL][HOTFIX] Fix a error message of VersionsSuite ## What changes were proposed in this pull request? https://github.com/apache/spark/pull/20064 breaks Jenkins tests

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2017-12-25 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20023 Thanks @gatorsmile. Then should I create a follow up PR for #20008 in order to cover the cases 2 and 3 before going on with this PR or can we go on with this PR and the test cases added in this

[GitHub] spark issue #20059: [SPARK-22648][K8s] Add documentation covering init conta...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20059 **[Test build #85385 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85385/testReport)** for PR 20059 at commit

[GitHub] spark issue #19954: [SPARK-22757][Kubernetes] Enable use of remote dependenc...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19954 **[Test build #85384 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85384/testReport)** for PR 19954 at commit

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-25 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158651951 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala --- @@ -45,6 +45,59 @@

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-25 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158651945 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/rest/k8s/SparkPodInitContainer.scala --- @@ -0,0 +1,116 @@ +/*

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-25 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158651953 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala --- @@ -45,6 +45,59 @@

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19683 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85382/ Test FAILed. ---

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-25 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158651933 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -132,30 +131,84 @@ private[spark] object

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19683 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-25 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158651930 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/DriverConfigOrchestrator.scala --- @@ -98,28 +109,62 @@

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-25 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158651915 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -132,30 +131,84 @@ private[spark] object

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19683 **[Test build #85382 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85382/testReport)** for PR 19683 at commit

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-25 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158651909 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -132,30 +131,84 @@ private[spark] object

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-25 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158651386 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/rest/k8s/SparkPodInitContainer.scala --- @@ -0,0 +1,116 @@ +/*

[GitHub] spark pull request #19954: [SPARK-22757][Kubernetes] Enable use of remote de...

2017-12-25 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19954#discussion_r158651523 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala --- @@ -45,6 +45,59

  1   2   3   >