[GitHub] spark issue #18323: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18323 **[Test build #80112 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80112/testReport)** for PR 18323 at commit [`e0478f5`](https://github.com/apache/spark/commit/e0

[GitHub] spark issue #18749: [SPARK-21485][FOLLOWUP][SQL][DOCS] Describes examples an...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18749 **[Test build #80111 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80111/testReport)** for PR 18749 at commit [`389fc6e`](https://github.com/apache/spark/commit/38

[GitHub] spark issue #18337: [SPARK-21131][GraphX] Fix batch gradient bug in SVDPlusP...

2017-07-31 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18337 This seems to have a lot of superfluous change. Why is the mean gradient better? It is different just by a scale factor. Also disagree about not clipping predictions but that's separate --- If your

[GitHub] spark pull request #18792: [SPARK-21589][SQL][DOC] Add documents about Hive ...

2017-07-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18792 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #18323: [SPARK-21117][SQL] Built-in SQL Function Support ...

2017-07-31 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/18323#discussion_r130526899 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -1186,3 +1186,124 @@ case class BRound(child:

[GitHub] spark pull request #18323: [SPARK-21117][SQL] Built-in SQL Function Support ...

2017-07-31 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/18323#discussion_r130526538 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -1186,3 +1186,124 @@ case class BRound(child:

[GitHub] spark issue #18783: [SPARK-21254] [WebUI] History UI performance fixes

2017-07-31 Thread 2ooom
Github user 2ooom commented on the issue: https://github.com/apache/spark/pull/18783 Thank you @ajbozarth --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18768: [SPARK-21533][SQL] Print warning messages when override ...

2017-07-31 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18768 I think the document about these unsupported functions is enough for users, so I'll close this for now. If we have more response from users, we could revisit this again. Thanks. --- If your project

[GitHub] spark pull request #18768: [SPARK-21533][SQL] Print warning messages when ov...

2017-07-31 Thread maropu
Github user maropu closed the pull request at: https://github.com/apache/spark/pull/18768 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18792 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80109/ Test PASSed. ---

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18792 **[Test build #80109 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80109/testReport)** for PR 18792 at commit [`29f1108`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18792 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18668: [SPARK-21451][SQL]get `spark.hadoop.*` properties from s...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18668 **[Test build #80110 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80110/testReport)** for PR 18668 at commit [`9ba8f53`](https://github.com/apache/spark/commit/9b

[GitHub] spark issue #18668: [SPARK-21451][SQL]get `spark.hadoop.*` properties from s...

2017-07-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18668 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #18731: [SPARK-20990][SQL] Read all JSON documents in fil...

2017-07-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18731#discussion_r130524511 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala --- @@ -347,13 +347,18 @@ class JacksonParser( U

[GitHub] spark issue #18779: [SPARK-21580][SQL]Integers in aggregation expressions ar...

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18779 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80105/ Test PASSed. ---

[GitHub] spark issue #18779: [SPARK-21580][SQL]Integers in aggregation expressions ar...

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18779 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18779: [SPARK-21580][SQL]Integers in aggregation expressions ar...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18779 **[Test build #80105 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80105/testReport)** for PR 18779 at commit [`a5667c8`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18792 LGTM pending Jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18792 **[Test build #80109 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80109/testReport)** for PR 18792 at commit [`29f1108`](https://github.com/apache/spark/commit/29

[GitHub] spark pull request #18742: [Spark-21542][ML][Python]Python persistence helpe...

2017-07-31 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/18742#discussion_r130523538 --- Diff: python/pyspark/ml/util.py --- @@ -283,3 +289,124 @@ def numFeatures(self): Returns the number of features the model was trained o

[GitHub] spark pull request #18742: [Spark-21542][ML][Python]Python persistence helpe...

2017-07-31 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/18742#discussion_r130519716 --- Diff: python/pyspark/ml/param/__init__.py --- @@ -375,6 +375,18 @@ def copy(self, extra=None): that._defaultParamMap = {}

[GitHub] spark pull request #18742: [Spark-21542][ML][Python]Python persistence helpe...

2017-07-31 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/18742#discussion_r130521214 --- Diff: python/pyspark/ml/util.py --- @@ -283,3 +289,124 @@ def numFeatures(self): Returns the number of features the model was trained o

[GitHub] spark pull request #18742: [Spark-21542][ML][Python]Python persistence helpe...

2017-07-31 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/18742#discussion_r130521964 --- Diff: python/pyspark/ml/util.py --- @@ -283,3 +289,124 @@ def numFeatures(self): Returns the number of features the model was trained o

[GitHub] spark pull request #18742: [Spark-21542][ML][Python]Python persistence helpe...

2017-07-31 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/18742#discussion_r130520335 --- Diff: python/pyspark/ml/util.py --- @@ -283,3 +289,124 @@ def numFeatures(self): Returns the number of features the model was trained o

[GitHub] spark pull request #18742: [Spark-21542][ML][Python]Python persistence helpe...

2017-07-31 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/18742#discussion_r130522066 --- Diff: python/pyspark/ml/util.py --- @@ -283,3 +289,124 @@ def numFeatures(self): Returns the number of features the model was trained o

[GitHub] spark pull request #18792: [SPARK-21589][SQL][DOC] Add documents about Hive ...

2017-07-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18792#discussion_r130523581 --- Diff: docs/sql-programming-guide.md --- @@ -1903,6 +1903,23 @@ releases of Spark SQL. Hive can optionally merge the small files into fewer large fi

[GitHub] spark pull request #18792: [SPARK-21589][SQL][DOC] Add documents about Hive ...

2017-07-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18792#discussion_r130523169 --- Diff: docs/sql-programming-guide.md --- @@ -1903,6 +1903,23 @@ releases of Spark SQL. Hive can optionally merge the small files into fewer large fi

[GitHub] spark issue #18727: [SPARK-21522][core] Fix flakiness in LauncherServerSuite...

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18727 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18727: [SPARK-21522][core] Fix flakiness in LauncherServerSuite...

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18727 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80102/ Test PASSed. ---

[GitHub] spark issue #18727: [SPARK-21522][core] Fix flakiness in LauncherServerSuite...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18727 **[Test build #80102 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80102/testReport)** for PR 18727 at commit [`1fcbf53`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #18746: [ML][Python] Implemented UnaryTransformer in Pyth...

2017-07-31 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/18746#discussion_r130518147 --- Diff: python/pyspark/ml/base.py --- @@ -116,3 +121,44 @@ class Model(Transformer): """ __metaclass__ = ABCMeta + +

[GitHub] spark pull request #18746: [ML][Python] Implemented UnaryTransformer in Pyth...

2017-07-31 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/18746#discussion_r130518773 --- Diff: python/pyspark/ml/base.py --- @@ -116,3 +121,44 @@ class Model(Transformer): """ __metaclass__ = ABCMeta + +

[GitHub] spark issue #18787: [SPARK-21583][SQL] Create a ColumnarBatch from ArrowColu...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18787 **[Test build #80108 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80108/testReport)** for PR 18787 at commit [`f906156`](https://github.com/apache/spark/commit/f9

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18792 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18792 **[Test build #80107 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80107/testReport)** for PR 18792 at commit [`c703d57`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18792 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80107/ Test PASSed. ---

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18792 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80106/ Test PASSed. ---

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18792 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18792 **[Test build #80106 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80106/testReport)** for PR 18792 at commit [`7d07e6b`](https://github.com/apache/spark/commit/7

[GitHub] spark pull request #17419: [SPARK-19634][ML] Multivariate summarizer - dataf...

2017-07-31 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/17419#discussion_r130517322 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -0,0 +1,799 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #18695: [SPARK-12717][PYTHON] Adding thread-safe broadcas...

2017-07-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18695#discussion_r130517275 --- Diff: python/pyspark/broadcast.py --- @@ -139,6 +140,24 @@ def __reduce__(self): return _from_id, (self._jbroadcast.id(),)

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18792 **[Test build #80107 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80107/testReport)** for PR 18792 at commit [`c703d57`](https://github.com/apache/spark/commit/c7

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18792 @gatorsmile ok, fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18792 **[Test build #80106 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80106/testReport)** for PR 18792 at commit [`7d07e6b`](https://github.com/apache/spark/commit/7d

[GitHub] spark pull request #17419: [SPARK-19634][ML] Multivariate summarizer - dataf...

2017-07-31 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/17419#discussion_r130515498 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -0,0 +1,746 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #18695: [SPARK-12717][PYTHON] Adding thread-safe broadcas...

2017-07-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18695#discussion_r130515139 --- Diff: python/pyspark/broadcast.py --- @@ -139,6 +140,24 @@ def __reduce__(self): return _from_id, (self._jbroadcast.id(),) +c

[GitHub] spark issue #18695: [SPARK-12717][PYTHON] Adding thread-safe broadcast pickl...

2017-07-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18695 Will merge this one after few days if there are no more comments from @holdenk and @davies. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

[GitHub] spark issue #18695: [SPARK-12717][PYTHON] Adding thread-safe broadcast pickl...

2017-07-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18695 Last question to check if I read correctly. So, the problem is around `_prepare_for_python_RDD` in `rdd.py` basically because it adds the pickled ones into the registry when it's pickled but thi

[GitHub] spark issue #18695: [SPARK-12717][PYTHON] Adding thread-safe broadcast pickl...

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18695 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80104/ Test PASSed. ---

[GitHub] spark issue #18695: [SPARK-12717][PYTHON] Adding thread-safe broadcast pickl...

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18695 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18695: [SPARK-12717][PYTHON] Adding thread-safe broadcast pickl...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18695 **[Test build #80104 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80104/testReport)** for PR 18695 at commit [`d4d1fed`](https://github.com/apache/spark/commit/d

[GitHub] spark pull request #18695: [SPARK-12717][PYTHON] Adding thread-safe broadcas...

2017-07-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18695#discussion_r130514254 --- Diff: python/pyspark/broadcast.py --- @@ -139,6 +140,24 @@ def __reduce__(self): return _from_id, (self._jbroadcast.id(),)

[GitHub] spark pull request #18695: [SPARK-12717][PYTHON] Adding thread-safe broadcas...

2017-07-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18695#discussion_r130514229 --- Diff: python/pyspark/broadcast.py --- @@ -139,6 +140,24 @@ def __reduce__(self): return _from_id, (self._jbroadcast.id(),)

[GitHub] spark pull request #18786: [SPARK-21584][SQL][SparkR] Update R method for su...

2017-07-31 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18786#discussion_r130514013 --- Diff: R/pkg/R/DataFrame.R --- @@ -2973,15 +2974,51 @@ setMethod("describe", dataFrame(sdf) }) +#' summary

[GitHub] spark pull request #18786: [SPARK-21584][SQL][SparkR] Update R method for su...

2017-07-31 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18786#discussion_r130513953 --- Diff: R/pkg/tests/fulltests/test_sparkSQL.R --- @@ -2500,8 +2500,15 @@ test_that("describe() and summarize() on a DataFrame", { expect_equal(

[GitHub] spark pull request #18786: [SPARK-21584][SQL][SparkR] Update R method for su...

2017-07-31 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18786#discussion_r130513793 --- Diff: R/pkg/R/DataFrame.R --- @@ -2973,15 +2974,51 @@ setMethod("describe", dataFrame(sdf) }) +#' summary

[GitHub] spark pull request #18786: [SPARK-21584][SQL][SparkR] Update R method for su...

2017-07-31 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18786#discussion_r130513739 --- Diff: R/pkg/R/DataFrame.R --- @@ -2973,15 +2974,51 @@ setMethod("describe", dataFrame(sdf) }) +#' summary

[GitHub] spark pull request #18786: [SPARK-21584][SQL][SparkR] Update R method for su...

2017-07-31 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18786#discussion_r130514066 --- Diff: R/pkg/R/DataFrame.R --- @@ -2973,15 +2974,51 @@ setMethod("describe", dataFrame(sdf) }) +#' summary

[GitHub] spark pull request #18765: [SPARK-19720][CORE][BACKPORT-2.1] Redact sensitiv...

2017-07-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18765#discussion_r130513976 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2571,6 +2572,23 @@ private[spark] object Utils extends Logging { sparkJ

[GitHub] spark pull request #18605: [SparkR][SPARK-21381]:SparkR: pass on setHandleIn...

2017-07-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18605 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18605: [SparkR][SPARK-21381]:SparkR: pass on setHandleInvalid f...

2017-07-31 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18605 merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18106: [SPARK-20754][SQL] Support TRUNC (number)

2017-07-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18106 ping @wangyum --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18337: [SPARK-21131][GraphX] Fix batch gradient bug in SVDPlusP...

2017-07-31 Thread daniellaah
Github user daniellaah commented on the issue: https://github.com/apache/spark/pull/18337 Try another dataset and you will find the algorithm works well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project doe

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18792 Thanks for working on it! Just left some minor comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request #18792: [SPARK-21589][SQL][DOC] Add documents about Hive ...

2017-07-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18792#discussion_r130512423 --- Diff: docs/sql-programming-guide.md --- @@ -1903,6 +1903,25 @@ releases of Spark SQL. Hive can optionally merge the small files into fewer larg

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18792 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18792 **[Test build #80103 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80103/testReport)** for PR 18792 at commit [`1434bde`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18792 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80103/ Test PASSed. ---

[GitHub] spark pull request #18792: [SPARK-21589][SQL][DOC] Add documents about Hive ...

2017-07-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18792#discussion_r130512311 --- Diff: docs/sql-programming-guide.md --- @@ -1903,6 +1903,25 @@ releases of Spark SQL. Hive can optionally merge the small files into fewer larg

[GitHub] spark issue #18779: [SPARK-21580][SQL]Integers in aggregation expressions ar...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18779 **[Test build #80105 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80105/testReport)** for PR 18779 at commit [`a5667c8`](https://github.com/apache/spark/commit/a5

[GitHub] spark pull request #18792: [SPARK-21589][SQL][DOC] Add documents about Hive ...

2017-07-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18792#discussion_r130512174 --- Diff: docs/sql-programming-guide.md --- @@ -1903,6 +1903,25 @@ releases of Spark SQL. Hive can optionally merge the small files into fewer larg

[GitHub] spark pull request #18792: [SPARK-21589][SQL][DOC] Add documents about Hive ...

2017-07-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18792#discussion_r130512125 --- Diff: docs/sql-programming-guide.md --- @@ -1903,6 +1903,25 @@ releases of Spark SQL. Hive can optionally merge the small files into fewer larg

[GitHub] spark issue #18695: [SPARK-12717][PYTHON] Adding thread-safe broadcast pickl...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18695 **[Test build #80104 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80104/testReport)** for PR 18695 at commit [`d4d1fed`](https://github.com/apache/spark/commit/d4

[GitHub] spark pull request #18792: [SPARK-21589][SQL][DOC] Add documents about Hive ...

2017-07-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18792#discussion_r130511955 --- Diff: docs/sql-programming-guide.md --- @@ -1903,6 +1903,25 @@ releases of Spark SQL. Hive can optionally merge the small files into fewer larg

[GitHub] spark pull request #18695: [SPARK-12717][PYTHON] Adding thread-safe broadcas...

2017-07-31 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18695#discussion_r130511618 --- Diff: python/pyspark/tests.py --- @@ -858,6 +858,49 @@ def test_multiple_broadcasts(self): self.assertEqual(N, size) self.a

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18792 **[Test build #80103 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80103/testReport)** for PR 18792 at commit [`1434bde`](https://github.com/apache/spark/commit/14

[GitHub] spark issue #18792: [SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDT...

2017-07-31 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18792 @gatorsmile If you get time, could you check this? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request #18792: [SPARK-21589][SQL][DOC] Add documents about Hive ...

2017-07-31 Thread maropu
. This pr relates to #18768 and #18527. ## How was this patch tested? N/A You can merge this pull request into a Git repository by running: $ git pull https://github.com/maropu/spark HOTFIX-20170731 Alternatively you can review and apply these changes as the patch at: https

[GitHub] spark pull request #18779: [SPARK-21580][SQL]Integers in aggregation express...

2017-07-31 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/18779#discussion_r130507794 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1010,7 +1014,16 @@ class Analyzer(

[GitHub] spark pull request #18779: [SPARK-21580][SQL]Integers in aggregation express...

2017-07-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18779#discussion_r130506005 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1010,7 +1014,16 @@ class Analyzer(

[GitHub] spark issue #18790: [SPARK-21587][SS] Added pushdown through watermarks.

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18790 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18790: [SPARK-21587][SS] Added pushdown through watermarks.

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18790 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80101/ Test PASSed. ---

[GitHub] spark issue #18790: [SPARK-21587][SS] Added pushdown through watermarks.

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18790 **[Test build #80101 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80101/testReport)** for PR 18790 at commit [`0a837ae`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #18789: SPARK-20433 Bump jackson from 2.6.5 to 2.6.7.1

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18789 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18789: SPARK-20433 Bump jackson from 2.6.5 to 2.6.7.1

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18789 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80097/ Test PASSed. ---

[GitHub] spark issue #18789: SPARK-20433 Bump jackson from 2.6.5 to 2.6.7.1

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18789 **[Test build #80097 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80097/testReport)** for PR 18789 at commit [`3ee0e92`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #18779: [SPARK-21580][SQL]Integers in aggregation express...

2017-07-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18779#discussion_r130505545 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1010,7 +1014,16 @@ class Analyzer(

[GitHub] spark pull request #18779: [SPARK-21580][SQL]Integers in aggregation express...

2017-07-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18779#discussion_r130505351 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1010,7 +1014,16 @@ class Analyzer(

[GitHub] spark issue #18727: [SPARK-21522][core] Fix flakiness in LauncherServerSuite...

2017-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18727 **[Test build #80102 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80102/testReport)** for PR 18727 at commit [`1fcbf53`](https://github.com/apache/spark/commit/1f

[GitHub] spark pull request #18779: [SPARK-21580][SQL]Integers in aggregation express...

2017-07-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18779#discussion_r130504819 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1010,7 +1014,16 @@ class Analyzer(

[GitHub] spark pull request #18779: [SPARK-21580][SQL]Integers in aggregation express...

2017-07-31 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/18779#discussion_r130504796 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1010,7 +1014,16 @@ class Analyzer(

[GitHub] spark issue #18727: [SPARK-21522][core] Fix flakiness in LauncherServerSuite...

2017-07-31 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18727 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18791: [SPARK-21571][WEB UI] Spark history server leaves incomp...

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18791 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #18695: [SPARK-12717][PYTHON] Adding thread-safe broadcas...

2017-07-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18695#discussion_r130504060 --- Diff: python/pyspark/tests.py --- @@ -858,6 +858,49 @@ def test_multiple_broadcasts(self): self.assertEqual(N, size) self.assert

[GitHub] spark issue #18695: [SPARK-12717][PYTHON] Adding thread-safe broadcast pickl...

2017-07-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18695 LGTM except for one minor comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request #18791: [SPARK-21571][WEB UI] Spark history server leaves...

2017-07-31 Thread ericvandenbergfb
GitHub user ericvandenbergfb opened a pull request: https://github.com/apache/spark/pull/18791 [SPARK-21571][WEB UI] Spark history server leaves incomplete or unrea… …dable history files around forever. Fix logic 1. checkForLogs excluded 0-size files so they

[GitHub] spark pull request #18779: [SPARK-21580][SQL]Integers in aggregation express...

2017-07-31 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/18779#discussion_r130503562 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1010,7 +1014,16 @@ class Analyzer(

[GitHub] spark issue #18748: [SPARK-20679][ML] Support recommending for a subset of u...

2017-07-31 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/18748 Did you test the performance of this, I tested the performance of MLLIB recommendForUserSubset some days ago, the performance is not good. Suppose the time of recommendForAll is 35s, recommend for 1/

[GitHub] spark issue #18787: [SPARK-21583][SQL] Create a ColumnarBatch from ArrowColu...

2017-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18787 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

  1   2   3   4   >