[GitHub] spark pull request #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated S...

2016-09-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14971#discussion_r80375572 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -259,6 +260,230 @@ class StatisticsSuite extends QueryTest with

[GitHub] spark pull request #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated S...

2016-09-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14971#discussion_r80375588 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -216,16 +218,15 @@ class StatisticsSuite extends QueryTest with

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14971 **[Test build #65878 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65878/consoleFull)** for PR 14971 at commit [`50ce04e`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #15233: [SPARK-17659] [SQL] Partitioned View is Not Supported By...

2016-09-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15233 cc @cloud-fan @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15234: [SPARK-17660] [SQL] DESC FORMATTED for VIEW Lacks View D...

2016-09-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15234 cc @hvanhovell @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15233: [SPARK-17659] [SQL] Partitioned View is Not Supported By...

2016-09-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15233 When the target view is a partitioned view, `ALTER VIEW AS SELECT` and `CREATE OR REPLACE VIEW` should issue an error too. --- If your project is set up for it, you can reply to this email and h

[GitHub] spark pull request #15233: [SPARK-17659] [SQL] Partitioned View is Not Suppo...

2016-09-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15233#discussion_r80376336 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -376,6 +376,10 @@ private[hive] class HiveClientImpl(

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15235 **[Test build #65876 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65876/consoleFull)** for PR 15235 at commit [`2a76ec1`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15235 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15230: [SPARK-17657] [SQL] Disallow Users to Change Tabl...

2016-09-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15230#discussion_r80376396 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -427,6 +427,11 @@ private[hive] class HiveClientImpl(

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15235 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65876/ Test FAILed. ---

[GitHub] spark issue #15135: [pyspark][group]pyspark GroupedData can't apply agg func...

2016-09-25 Thread citoubest
Github user citoubest commented on the issue: https://github.com/apache/spark/pull/15135 @davies, what do you think about this patch? Can you give me some advice? Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request #14035: [SPARK-16356][ML] Add testImplicits for ML unit t...

2016-09-25 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/14035#discussion_r80376680 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/MultilayerPerceptronClassifierSuite.scala --- @@ -116,7 +117,7 @@ class MultilayerPerce

[GitHub] spark pull request #14035: [SPARK-16356][ML] Add testImplicits for ML unit t...

2016-09-25 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/14035#discussion_r80376849 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/util/MLUtilsSuite.scala --- @@ -282,9 +281,7 @@ class MLUtilsSuite extends SparkFunSuite with MLlib

[GitHub] spark pull request #14035: [SPARK-16356][ML] Add testImplicits for ML unit t...

2016-09-25 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/14035#discussion_r80376739 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/MinMaxScalerSuite.scala --- @@ -57,8 +58,7 @@ class MinMaxScalerSuite extends SparkFunSuite wit

[GitHub] spark pull request #14035: [SPARK-16356][ML] Add testImplicits for ML unit t...

2016-09-25 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/14035#discussion_r80376695 --- Diff: mllib/src/test/scala/org/apache/spark/ml/evaluation/RegressionEvaluatorSuite.scala --- @@ -42,9 +43,10 @@ class RegressionEvaluatorSuite

[GitHub] spark issue #14035: [SPARK-16356][ML] Add testImplicits for ML unit tests an...

2016-09-25 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/14035 @HyukjinKwon I have made a pass and this PR look good overall. Could you double check whether all ML test cases are covered? Since I found we used implicit import of different style at [```ChiSq

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15168 **[Test build #65877 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65877/consoleFull)** for PR 15168 at commit [`21e8295`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15168 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65877/ Test PASSed. ---

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15168 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.add...

2016-09-25 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/15216#discussion_r80377484 --- Diff: R/pkg/R/context.R --- @@ -231,17 +231,21 @@ setCheckpointDir <- function(sc, dirName) { #' filesystems), or an HTTP, HTTPS or FTP URI. To ac

[GitHub] spark pull request #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.add...

2016-09-25 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/15216#discussion_r80377496 --- Diff: R/pkg/R/context.R --- @@ -231,17 +231,21 @@ setCheckpointDir <- function(sc, dirName) { #' filesystems), or an HTTP, HTTPS or FTP URI. To ac

[GitHub] spark issue #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.addFile su...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15216 **[Test build #65879 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65879/consoleFull)** for PR 15216 at commit [`b2f3a59`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14971 **[Test build #65878 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65878/consoleFull)** for PR 14971 at commit [`50ce04e`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14971 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14971 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65878/ Test PASSed. ---

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-09-25 Thread eyalfa
Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/1 @cloud-fan, @hvanhovell, any update on tis? did you get to review this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark issue #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.addFile su...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15216 **[Test build #65879 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65879/consoleFull)** for PR 15216 at commit [`b2f3a59`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.addFile su...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15216 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.addFile su...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15216 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65879/ Test PASSed. ---

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] write.df API taking path optionall...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15231 **[Test build #65880 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65880/consoleFull)** for PR 15231 at commit [`c2a64db`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] write.df API taking path optionall...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15231 @felixcheung Yeap, I just added. As we don't currently have an internal datasource allowing `DataFrameWriter.save()` without a path, I just added a test with CSV data source to make sure it thro

[GitHub] spark issue #15232: [SPARK-17499][SPARKR][FOLLOWUP] Suppress warnings from i...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15232 Sure, I will take a look for this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] spark pull request #14035: [SPARK-16356][ML] Add testImplicits for ML unit t...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/14035#discussion_r80379623 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/OneVsRestSuite.scala --- @@ -55,7 +56,7 @@ class OneVsRestSuite extends SparkFunSuite w

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] write.df API taking path optionall...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15231 **[Test build #65880 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65880/consoleFull)** for PR 15231 at commit [`c2a64db`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] write.df API taking path optionall...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15231 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65880/ Test PASSed. ---

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] write.df API taking path optionall...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15231 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14035: [SPARK-16356][ML] Add testImplicits for ML unit tests an...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14035 **[Test build #65881 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65881/consoleFull)** for PR 14035 at commit [`ad9d7ac`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #14035: [SPARK-16356][ML] Add testImplicits for ML unit tests an...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14035 Thanks @yanboliang and @jaceklaskowski . I addressed comments except for few comments I am not too sure of and I think are not related changes. --- If your project is set up for it, you can re

[GitHub] spark pull request #14035: [SPARK-16356][ML] Add testImplicits for ML unit t...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/14035#discussion_r80380667 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/VectorIndexerSuite.scala --- @@ -85,11 +87,13 @@ class VectorIndexerSuite extends SparkFunSuit

[GitHub] spark issue #15232: [SPARK-17499][SPARKR][FOLLOWUP] Suppress warnings from i...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15232 Hm.. I took some issues in `testthat` and I think that is an expected behaviour(?). (Please take a look for the issue 224 in `testthat`). I asked a question to the author of that library.

[GitHub] spark issue #15232: [SPARK-17499][SPARKR][FOLLOWUP] Check null first for lay...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15232 **[Test build #65882 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65882/consoleFull)** for PR 15232 at commit [`2e88804`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #15214: [SPARK-17017][Follow-up][ML] Refactor of ChiSqSelector a...

2016-09-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15214 I'm OK with it. @mpjlu sounds like you approve? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #15053: [Doc] improve python API docstrings

2016-09-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15053 @mortada what do you think of @HyukjinKwon 's suggestions? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark issue #14035: [SPARK-16356][ML] Add testImplicits for ML unit tests an...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14035 **[Test build #65883 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65883/consoleFull)** for PR 14035 at commit [`b60c952`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #15232: [SPARK-17499][SPARKR][FOLLOWUP] Check null first for lay...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15232 **[Test build #65882 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65882/consoleFull)** for PR 15232 at commit [`2e88804`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15232: [SPARK-17499][SPARKR][FOLLOWUP] Check null first for lay...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15232 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65882/ Test PASSed. ---

[GitHub] spark issue #15232: [SPARK-17499][SPARKR][FOLLOWUP] Check null first for lay...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15232 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14035: [SPARK-16356][ML] Add testImplicits for ML unit tests an...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14035 **[Test build #65881 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65881/consoleFull)** for PR 14035 at commit [`ad9d7ac`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14035: [SPARK-16356][ML] Add testImplicits for ML unit tests an...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14035 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14035: [SPARK-16356][ML] Add testImplicits for ML unit tests an...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14035 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65881/ Test PASSed. ---

[GitHub] spark issue #14035: [SPARK-16356][ML] Add testImplicits for ML unit tests an...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14035 **[Test build #65883 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65883/consoleFull)** for PR 14035 at commit [`b60c952`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14035: [SPARK-16356][ML] Add testImplicits for ML unit tests an...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14035 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65883/ Test PASSed. ---

[GitHub] spark issue #14035: [SPARK-16356][ML] Add testImplicits for ML unit tests an...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14035 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14762: [SPARK-16962][CORE][SQL] Fix misaligned record accesses ...

2016-09-25 Thread jlhitt
Github user jlhitt commented on the issue: https://github.com/apache/spark/pull/14762 @srowen and @rxin, Sorry for the delay in getting this data to you.. Let me know if you have any questions. To see if there were any regressions we ran tests on 2-chip Broadwell E5 v4 10core

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15219 **[Test build #65884 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65884/consoleFull)** for PR 15219 at commit [`ab268f9`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #15214: [SPARK-17017][Follow-up][ML] Refactor of ChiSqSelector a...

2016-09-25 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/15214 hi @srowen . My understand of yanbo's comments here is, if user use chSqSelector like this: model1 = new ChiSqSelector().setFPR(0.05).setKBest(100).fit(data) model2 = new ChiSqSelector()

[GitHub] spark issue #15214: [SPARK-17017][Follow-up][ML] Refactor of ChiSqSelector a...

2016-09-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15214 OK, I could also support either behavior. After all, for any component, `.setFoo(x).setFoo(y)` also creates a different model if the order is swapped, so I am not so clear that's a 'problem'. ---

[GitHub] spark issue #14762: [SPARK-16962][CORE][SQL] Fix misaligned record accesses ...

2016-09-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14762 OK, it seems like a toss-up for x86. I'd expect that the difference here is very small in any event, and this at least doesn't contradict that. I'm OK with this as a result. @rxin your thoughts? --

[GitHub] spark issue #15214: [SPARK-17017][Follow-up][ML] Refactor of ChiSqSelector a...

2016-09-25 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15214 @srowen @mpjlu Another important reason for this change: it's error prone for Python ML API. ``` def __init__(self, numTopFeatures=50, featuresCol="features", outputCol=None,

[GitHub] spark issue #15214: [SPARK-17017][Follow-up][ML] Refactor of ChiSqSelector a...

2016-09-25 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15214 And you can also refer all other Estimator in ML, even you swap the arguments setting order, you still get the same model. Thanks. --- If your project is set up for it, you can reply to this ema

[GitHub] spark issue #14852: [SPARK-17138][ML][MLib] Add Python API for multinomial l...

2016-09-25 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/14852 Done. thanks for careful review :) @sethah --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark issue #14852: [SPARK-17138][ML][MLib] Add Python API for multinomial l...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14852 **[Test build #65885 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65885/consoleFull)** for PR 14852 at commit [`c47ac07`](https://github.com/apache/spark/commit/c

[GitHub] spark pull request #15233: [SPARK-17659] [SQL] Partitioned View is Not Suppo...

2016-09-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15233#discussion_r80385191 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -376,6 +376,10 @@ private[hive] class HiveClientImpl(

[GitHub] spark issue #15214: [SPARK-17017][Follow-up][ML] Refactor of ChiSqSelector a...

2016-09-25 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/15214 Thanks, this looks good to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] spark issue #14852: [SPARK-17138][ML][MLib] Add Python API for multinomial l...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14852 **[Test build #65885 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65885/consoleFull)** for PR 14852 at commit [`c47ac07`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14852: [SPARK-17138][ML][MLib] Add Python API for multinomial l...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14852 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65885/ Test PASSed. ---

[GitHub] spark issue #14852: [SPARK-17138][ML][MLib] Add Python API for multinomial l...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14852 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15097: [SPARK-17540][SparkR][Spark Core] fix SparkR array serde...

2016-09-25 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/15097 @felixcheung I think out two ways for this problem, see the PR description. which is better in your opinion? Or whether it exists better solution? --- If your project is set up for

[GitHub] spark pull request #15236: [SPARK-17017][ML][MLLIB][ML][DOC] Updated the ml ...

2016-09-25 Thread lins05
GitHub user lins05 opened a pull request: https://github.com/apache/spark/pull/15236 [SPARK-17017][ML][MLLIB][ML][DOC] Updated the ml feature selection doc for ChiSqSelector ## What changes were proposed in this pull request? A follow up for #14597 to update feature selecti

[GitHub] spark issue #15236: [SPARK-17017][ML][MLLIB][ML][DOC] Updated the ml/mllib f...

2016-09-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15236 Please add this to #15214 with @yanboliang -- this is getting spread across too many places. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #15236: [SPARK-17017][ML][MLLIB][ML][DOC] Updated the ml/mllib f...

2016-09-25 Thread lins05
Github user lins05 commented on the issue: https://github.com/apache/spark/pull/15236 Just found #15214 and #15212, I think this one need to wait until those are merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #15236: [SPARK-17017][ML][MLLIB][ML][DOC] Updated the ml/mllib f...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15236 **[Test build #65886 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65886/consoleFull)** for PR 15236 at commit [`4a2cf5e`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15219 **[Test build #65884 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65884/consoleFull)** for PR 15219 at commit [`ab268f9`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15219 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15219 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65884/ Test FAILed. ---

[GitHub] spark issue #15214: [SPARK-17017][Follow-up][ML] Refactor of ChiSqSelector a...

2016-09-25 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/15214 Hi @srowen , sorry for forgetting update the doc and python/ml/feature.py in last PR. This pr has added ml/feature.py. It looks good to me. Thanks --- If your project is set up for it,

[GitHub] spark issue #15236: [SPARK-17017][ML][MLLIB][ML][DOC] Updated the ml/mllib f...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15236 **[Test build #65886 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65886/consoleFull)** for PR 15236 at commit [`4a2cf5e`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15236: [SPARK-17017][ML][MLLIB][ML][DOC] Updated the ml/mllib f...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15236 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65886/ Test PASSed. ---

[GitHub] spark issue #15236: [SPARK-17017][ML][MLLIB][ML][DOC] Updated the ml/mllib f...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15236 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15168 Thank you for review @gatorsmile . Hi, @hvanhovell . Could you review this again? --- If your project is set up for it, you can reply to this email and have your reply appear on G

[GitHub] spark pull request #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.add...

2016-09-25 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15216#discussion_r80388454 --- Diff: R/pkg/R/context.R --- @@ -231,17 +231,21 @@ setCheckpointDir <- function(sc, dirName) { #' filesystems), or an HTTP, HTTPS or FTP URI. To a

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] write.df API taking path optionall...

2016-09-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15231 Hmm, should we hold till 12601 is merged then? Seems like we shouldn't allow this unless internal datasources are supporting this more broadly. Also, before the path parameter type is in

[GitHub] spark issue #15232: [SPARK-17499][SPARKR][FOLLOWUP] Check null first for lay...

2016-09-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15232 you mean issue 224 of testthat on github? doesn't seem like it's related? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15232: [SPARK-17499][SPARKR][FOLLOWUP] Check null first for lay...

2016-09-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15232 change LGTM to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes s

[GitHub] spark issue #15097: [SPARK-17540][SparkR][Spark Core] fix SparkR array serde...

2016-09-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15097 @WeichenXu123 do you have the user code and sample data that when run with SparkR will cause this issue? I think that will help us understand how this happens better. --- If your project is s

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15219 **[Test build #65887 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65887/consoleFull)** for PR 15219 at commit [`cd5ade5`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #15232: [SPARK-17499][SPARKR][FOLLOWUP] Check null first for lay...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15232 Oh I meant 244. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15219 **[Test build #65887 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65887/consoleFull)** for PR 15219 at commit [`cd5ade5`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15219 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15219 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65887/ Test FAILed. ---

[GitHub] spark issue #15153: [SPARK-17599] Prevent ListingFileCatalog from failing if...

2016-09-25 Thread petermaxlee
Github user petermaxlee commented on the issue: https://github.com/apache/spark/pull/15153 @brkyvz the change here only affects the serial version, and not the parallel version, does it? Wouldn't that be a problem? --- If your project is set up for it, you can reply to this

[GitHub] spark issue #15153: [SPARK-17599] Prevent ListingFileCatalog from failing if...

2016-09-25 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/15153 @petermaxlee It is true that the parallel version can fail as well, the same kind of race condition can bite people there --- If your project is set up for it, you can reply to this email and have y

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread petermaxlee
Github user petermaxlee commented on the issue: https://github.com/apache/spark/pull/15235 @brkyvz I think this also impacts the change you just did in https://github.com/apache/spark/pull/15153. This change makes both code path consistent. --- If your project is set up for it, you

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15235 **[Test build #65888 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65888/consoleFull)** for PR 15235 at commit [`5c6a640`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-09-25 Thread karlhigley
Github user karlhigley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r80392692 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-09-25 Thread karlhigley
Github user karlhigley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r80393070 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-09-25 Thread karlhigley
Github user karlhigley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r80392464 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] spark pull request #15237: [SPARK-17663] [CORE] SchedulableBuilder should ha...

2016-09-25 Thread erenavsarogullari
GitHub user erenavsarogullari opened a pull request: https://github.com/apache/spark/pull/15237 [SPARK-17663] [CORE] SchedulableBuilder should handle invalid data access via scheduler.al… ## What changes were proposed in this pull request? If `spark.scheduler.allocation.file`

[GitHub] spark issue #15237: [SPARK-17663] [CORE] SchedulableBuilder should handle in...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15237 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

  1   2   3   >