[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18810 **[Test build #80325 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80325/testReport)** for PR 18810 at commit [`7e84753`](https://github.com/apache/spark/commit/7e

[GitHub] spark pull request #17357: [SPARK-20025][CORE] Ignore SPARK_LOCAL* env, whil...

2017-08-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17357#discussion_r131582810 --- Diff: core/src/main/scala/org/apache/spark/deploy/rest/StandaloneRestServer.scala --- @@ -139,7 +139,9 @@ private[rest] class StandaloneSubmitRequestServ

[GitHub] spark pull request #18866: [SPARK-21649][SQL] Support writing data into hive...

2017-08-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18866#discussion_r131582440 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/physical/partitioning.scala --- @@ -262,7 +262,12 @@ case class HashPartitioning(e

[GitHub] spark issue #18801: SPARK-10878 Fix race condition when multiple clients res...

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18801 **[Test build #80324 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80324/testReport)** for PR 18801 at commit [`1ace5cc`](https://github.com/apache/spark/commit/1a

[GitHub] spark issue #18801: SPARK-10878 Fix race condition when multiple clients res...

2017-08-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18801 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if t

[GitHub] spark issue #18846: [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS ins...

2017-08-06 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18846 Should we also apply this change to `RpcEnv` ? @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #12147: [SPARK-14361][SQL]Window function exclude clause

2017-08-06 Thread xwu0226
Github user xwu0226 commented on the issue: https://github.com/apache/spark/pull/12147 @HyukjinKwon My rebased branch has broken most of the window exclude test cases. Trying to fix. --- If your project is set up for it, you can reply to this email and have your reply appear on GitH

[GitHub] spark issue #18846: [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS ins...

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18846 **[Test build #80323 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80323/testReport)** for PR 18846 at commit [`afc07ee`](https://github.com/apache/spark/commit/af

[GitHub] spark issue #18846: [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS ins...

2017-08-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18846 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if t

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-08-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18865 cc @gatorsmile @cloud-fan Can you help trigger Jenkins for this? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-08-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18865 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #18866: [SPARK-21649][SQL] Support writing data into hive bucket...

2017-08-06 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/18866 I added the unit test referring (https://github.com/apache/hive/blob/branch-1/ql/src/java/org/apache/hadoop/hive/ql/optimizer/AbstractBucketJoinProc.java#L393). Hive will sort bucket files by f

[GitHub] spark issue #18866: [SPARK-21649][SQL] Support writing data into hive bucket...

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18866 **[Test build #80322 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80322/testReport)** for PR 18866 at commit [`51d2c11`](https://github.com/apache/spark/commit/51

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-08-06 Thread jmchung
GitHub user jmchung reopened a pull request: https://github.com/apache/spark/pull/18865 [SPARK-21610][SQL] Corrupt records are not handled properly when creating a dataframe from a file ## What changes were proposed in this pull request? ``` echo '{"field": 1} {"field":

[GitHub] spark pull request #18866: [SPARK-21649][SQL] Support writing data into hive...

2017-08-06 Thread jinxing64
GitHub user jinxing64 opened a pull request: https://github.com/apache/spark/pull/18866 [SPARK-21649][SQL] Support writing data into hive bucket table. ## What changes were proposed in this pull request? Support writing hive bucket table. Spark internally uses Murmur3Hash fo

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-08-06 Thread jmchung
Github user jmchung closed the pull request at: https://github.com/apache/spark/pull/18865 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-08-06 Thread jmchung
GitHub user jmchung opened a pull request: https://github.com/apache/spark/pull/18865 [SPARK-21610][SQL] Corrupt records are not handled properly when creating a dataframe from a file ## What changes were proposed in this pull request? ``` echo '{"field": 1} {"field": 2}

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-08-06 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r131578995 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1121,6 +1125,30 @@ class AstBuilder(conf: SQLConf) ex

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-08-06 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r131579031 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1121,6 +1125,30 @@ class AstBuilder(conf: SQLConf) ex

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-08-06 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r131578980 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2304,7 +2304,15 @@ object functions { * @group string_funcs

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131578382 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1393,6 +1393,16 @@ def replace(self, to_replace, value=None, subset=None): |null| null| null|

[GitHub] spark issue #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18861 **[Test build #80321 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80321/testReport)** for PR 18861 at commit [`c0306d3`](https://github.com/apache/spark/commit/c0

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r131576044 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -356,6 +356,16 @@ class CodegenContex

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r131575786 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -370,6 +370,12 @@ case class WholeStageCodegenExec(c

[GitHub] spark issue #18764: [SPARK-21306][ML] For branch 2.0, OneVsRest should suppo...

2017-08-06 Thread facaiy
Github user facaiy commented on the issue: https://github.com/apache/spark/pull/18764 @SparkQA Take a test, please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] spark issue #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18861 **[Test build #80320 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80320/testReport)** for PR 18861 at commit [`413b0eb`](https://github.com/apache/spark/commit/41

[GitHub] spark issue #18864: [SPARK-21648] [SQL] Fix confusing assert failure in JDBC...

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18864 **[Test build #80319 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80319/testReport)** for PR 18864 at commit [`e4aac50`](https://github.com/apache/spark/commit/e4

[GitHub] spark issue #18864: [SPARK-21648] [SQL] Fix confusing assert failure in JDBC...

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18864 cc @zsxwing @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wish

[GitHub] spark pull request #18864: [SPARK-21648] [SQL] Fix confusing assert failure ...

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18864#discussion_r131574704 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcRelationProvider.scala --- @@ -29,17 +29,22 @@ class JdbcRelationP

[GitHub] spark pull request #18864: [SPARK-21648] [SQL] Fix confusing assert failure ...

2017-08-06 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/18864 [SPARK-21648] [SQL] Fix confusing assert failure in JDBC source when parallel fetching parameters are not properly provided. ### What changes were proposed in this pull request? ```SQL C

[GitHub] spark issue #18855: [SPARK-3151][Block Manager] DiskStore.getBytes fails for...

2017-08-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18855 Yea, please refer http://apache-spark-developers-list.1001551.n3.nabble.com/Some-PRs-not-automatically-linked-to-JIRAs-td22067.html Looks some problems related with it. --- If your project is

[GitHub] spark issue #18830: [SPARK-21621][Core] Reset numRecordsWritten after DiskBl...

2017-08-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18830 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18830: [SPARK-21621][Core] Reset numRecordsWritten after DiskBl...

2017-08-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18830 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80316/ Test PASSed. ---

[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18810 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18810 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80318/ Test FAILed. ---

[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18810 **[Test build #80318 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80318/testReport)** for PR 18810 at commit [`1b0ac5e`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #18830: [SPARK-21621][Core] Reset numRecordsWritten after DiskBl...

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18830 **[Test build #80316 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80316/testReport)** for PR 18830 at commit [`d82401d`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18810 **[Test build #80318 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80318/testReport)** for PR 18810 at commit [`1b0ac5e`](https://github.com/apache/spark/commit/1b

[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18810 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #18576: [SPARK-21351][SQL] Update nullability based on ch...

2017-08-06 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18576#discussion_r131573104 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -94,27 +94,14 @@ case class FilterExec(condition: Expre

[GitHub] spark issue #18862: [SPARK-21640][FOLLOW-UP] added errorifexists on IllegalA...

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18862 **[Test build #80317 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80317/testReport)** for PR 18862 at commit [`592ab60`](https://github.com/apache/spark/commit/59

[GitHub] spark issue #18862: [SPARK-21640][FOLLOW-UP] added errorifexists on IllegalA...

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18862 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #17583: [SPARK-20271]Add FuncTransformer to simplify custom tran...

2017-08-06 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/17583 A gentle ping since I think this is quite helpful. @jkbradley @MLnick @yanboliang @srowen @holdenk --- If your project is set up for it, you can reply to this email and have your reply appear o

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131572498 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1393,6 +1393,16 @@ def replace(self, to_replace, value=None, subset=None): |null| null|

[GitHub] spark pull request #18733: [SPARK-21535][ML]Reduce memory requirement for Cr...

2017-08-06 Thread hhbyyh
Github user hhbyyh closed the pull request at: https://github.com/apache/spark/pull/18733 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-06 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18861#discussion_r131571620 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -753,6 +753,16 @@ case class Repartitio

[GitHub] spark pull request #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-06 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18861#discussion_r131571547 --- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala --- @@ -1185,23 +1194,21 @@ class SizeBasedCoalescer(val maxSize: Int) extends PartitionCoa

[GitHub] spark pull request #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18861#discussion_r131571449 --- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala --- @@ -1185,23 +1194,21 @@ class SizeBasedCoalescer(val maxSize: Int) extends Partitio

[GitHub] spark pull request #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-06 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18861#discussion_r131571248 --- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala --- @@ -1185,23 +1194,21 @@ class SizeBasedCoalescer(val maxSize: Int) extends PartitionCoa

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131571005 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1393,6 +1393,16 @@ def replace(self, to_replace, value=None, subset=None): |null| null| null|

[GitHub] spark pull request #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18861#discussion_r131570851 --- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala --- @@ -1185,23 +1194,21 @@ class SizeBasedCoalescer(val maxSize: Int) extends Partitio

[GitHub] spark pull request #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18861#discussion_r131570879 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -753,6 +753,16 @@ case class Repart

[GitHub] spark pull request #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-06 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18861#discussion_r131570565 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -571,7 +570,8 @@ case class UnionExec(children: Seq[Spa

[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2017-08-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18576 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80315/ Test PASSed. ---

[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2017-08-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18576 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18576 **[Test build #80315 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80315/testReport)** for PR 18576 at commit [`5d2fd6d`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18861#discussion_r131570472 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -571,7 +570,8 @@ case class UnionExec(children: Seq

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131570273 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameNaFunctionsSuite.scala --- @@ -261,5 +261,18 @@ class DataFrameNaFunctionsSuite extends Q

[GitHub] spark issue #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to replace ...

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18820 cc @ueshin Could you also take a look the code changes in the Python side? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to replace ...

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18820 Could you also add a test case to cover the end-to-end use case the JIRA mentioned? Also put it in the PR description, which will be part of the PR commit. Thanks! --- If your project is set up

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131570031 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameNaFunctions.scala --- @@ -314,6 +316,7 @@ final class DataFrameNaFunctions private[sql](d

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131569954 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameNaFunctions.scala --- @@ -366,11 +370,15 @@ final class DataFrameNaFunctions private[sql]

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131569819 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/types/DataTypeSuite.scala --- @@ -145,8 +145,8 @@ class DataTypeSuite extends SparkFunSuite {

[GitHub] spark issue #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to replace ...

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18820 @bravo-zhang Could you update the PR description to explain what this PR is trying to achieve? So far, it is not clear enough to explain what you did in this PR. Thanks! --- If your project is

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131569456 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1393,6 +1393,16 @@ def replace(self, to_replace, value=None, subset=None): |null| null|

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131569039 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1393,6 +1393,16 @@ def replace(self, to_replace, value=None, subset=None): |null| null| null|

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131568706 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1393,6 +1393,16 @@ def replace(self, to_replace, value=None, subset=None): |null| null|

[GitHub] spark issue #18468: [SPARK-20783][SQL] Create CachedBatchColumnVector to abs...

2017-08-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18468 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18468: [SPARK-20783][SQL] Create CachedBatchColumnVector to abs...

2017-08-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18468 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80314/ Test PASSed. ---

[GitHub] spark issue #18468: [SPARK-20783][SQL] Create CachedBatchColumnVector to abs...

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18468 **[Test build #80314 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80314/testReport)** for PR 18468 at commit [`a26dc15`](https://github.com/apache/spark/commit/a

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131568383 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1393,6 +1393,16 @@ def replace(self, to_replace, value=None, subset=None): |null| null|

[GitHub] spark issue #18769: [SPARK-21574][SQL] Point out user to set hive config bef...

2017-08-06 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/18769 @gatorsmile Docs syntax issues was fixed by https://github.com/apache/spark/pull/18793. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as wel

[GitHub] spark pull request #18833: [SPARK-21625][SQL] sqrt(negative number) should b...

2017-08-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18833#discussion_r131568132 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/MathExpressionsSuite.scala --- @@ -403,11 +403,13 @@ class MathExpressionsSu

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131567901 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1393,6 +1393,16 @@ def replace(self, to_replace, value=None, subset=None): |null| null| null|

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131567720 --- Diff: python/pyspark/sql/tests.py --- @@ -1964,6 +1964,16 @@ def test_replace(self): .replace(False, True).first()) self.

[GitHub] spark issue #18863: [SPARK-21647] [SQL] Fix SortMergeJoin when using CROSS

2017-08-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18863 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80313/ Test PASSed. ---

[GitHub] spark issue #18863: [SPARK-21647] [SQL] Fix SortMergeJoin when using CROSS

2017-08-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18863 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18863: [SPARK-21647] [SQL] Fix SortMergeJoin when using CROSS

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18863 **[Test build #80313 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80313/testReport)** for PR 18863 at commit [`f351fb1`](https://github.com/apache/spark/commit/f

[GitHub] spark pull request #18833: [SPARK-21625][SQL] sqrt(negative number) should b...

2017-08-06 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/18833#discussion_r131566961 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/MathExpressionsSuite.scala --- @@ -403,11 +403,13 @@ class MathExpressionsS

[GitHub] spark pull request #18833: [SPARK-21625][SQL] sqrt(negative number) should b...

2017-08-06 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/18833#discussion_r131566102 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/MathExpressionsSuite.scala --- @@ -403,11 +403,13 @@ class MathExpressionsS

[GitHub] spark pull request #18833: [SPARK-21625][SQL] sqrt(negative number) should b...

2017-08-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18833#discussion_r131566009 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/MathExpressionsSuite.scala --- @@ -403,11 +403,13 @@ class MathExpressi

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131566016 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameNaFunctionsSuite.scala --- @@ -261,5 +261,18 @@ class DataFrameNaFunctionsSuite extends Query

[GitHub] spark pull request #18833: [SPARK-21625][SQL] sqrt(negative number) should b...

2017-08-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18833#discussion_r131565248 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/MathExpressionsSuite.scala --- @@ -403,11 +403,13 @@ class MathExpressionsSu

[GitHub] spark issue #18813: [SPARK-21567][SQL] Dataset should work with type alias

2017-08-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18813 ping @cloud-fan @hvanhovell Can you help to review this change? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #18641: [SPARK-21413][SQL] Fix 64KB JVM bytecode limit problem i...

2017-08-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18641 ping @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18853: [SPARK-21646][SQL] BinaryComparison shouldn't auto cast ...

2017-08-06 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18853 How about casting the `int` values into `string` ones in that case you described in the description, and then comparing them by a lexicographical order? --- If your project is set up for it, you ca

[GitHub] spark issue #18474: [SPARK-21235][TESTS] UTest should clear temp results whe...

2017-08-06 Thread wangjiaochun
Github user wangjiaochun commented on the issue: https://github.com/apache/spark/pull/18474 Yes, Running this on Windows7. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18710: [SPARK][Docs] Added note on meaning of position to subst...

2017-08-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18710 gentle ping @maclockard. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] spark pull request #18111: [SPARK-20886][CORE] HadoopMapReduceCommitProtocol...

2017-08-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18111#discussion_r131562253 --- Diff: core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala --- @@ -73,7 +73,10 @@ class HadoopMapReduceCommitProtoc

[GitHub] spark issue #18830: [SPARK-21621][Core] Reset numRecordsWritten after DiskBl...

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18830 **[Test build #80316 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80316/testReport)** for PR 18830 at commit [`d82401d`](https://github.com/apache/spark/commit/d8

[GitHub] spark issue #18474: [SPARK-21235][TESTS] UTest should clear temp results whe...

2017-08-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18474 @wangjiaochun Are you running this on Windows? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18830: [SPARK-21621][Core] Reset numRecordsWritten after DiskBl...

2017-08-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18830 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18791: [SPARK-21571][Scheduler] Spark history server leaves inc...

2017-08-06 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18791 Yea, I'm just thinking whether it is possible we can have a perfect approach that we can be confident to turn it on by default. --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18576 **[Test build #80315 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80315/testReport)** for PR 18576 at commit [`5d2fd6d`](https://github.com/apache/spark/commit/5d

[GitHub] spark issue #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to replace ...

2017-08-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18820 Other than few comments above, LGTM. Any other comments? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131559076 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1446,7 +1457,7 @@ def all_of_(xs): if isinstance(to_replace, (float, int, long, basestring

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131559178 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1460,7 +1471,8 @@ def all_of_(xs): subset = [subset] # Verify we we

[GitHub] spark issue #18468: [SPARK-20783][SQL] Create CachedBatchColumnVector to abs...

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18468 **[Test build #80314 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80314/testReport)** for PR 18468 at commit [`a26dc15`](https://github.com/apache/spark/commit/a2

[GitHub] spark issue #18863: [SPARK-21647] [SQL] Fix SortMergeJoin when using CROSS

2017-08-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18863 **[Test build #80313 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80313/testReport)** for PR 18863 at commit [`f351fb1`](https://github.com/apache/spark/commit/f3

[GitHub] spark issue #18863: [SPARK-21647] [SQL] Fix SortMergeJoin when using CROSS

2017-08-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18863 cc @cloud-fan @BoleynSu @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

  1   2   3   >