[GitHub] spark issue #12889: [SPARK-15113][PySpark][ML] Add missing num features num ...

2016-08-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/12889 ping @MLnick ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #12914: [SPARK-15130][PySpark][ML][DOCS] pyspark expose default ...

2016-08-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/12914 Now that 2.0 is truly out the door - any thoughts from anyone? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark pull request #14207: [SPARK-16552] [SQL] Store the Inferred Schemas in...

2016-08-08 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14207#discussion_r73940468 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala --- @@ -95,17 +95,39 @@ case class CreateDataSourceTable

[GitHub] spark pull request #14207: [SPARK-16552] [SQL] Store the Inferred Schemas in...

2016-08-08 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14207#discussion_r73940350 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala --- @@ -95,17 +95,39 @@ case class CreateDataSourceTable

[GitHub] spark issue #14541: [SPARK-16953] Make requestTotalExecutors public Develope...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14541 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14541: [SPARK-16953] Make requestTotalExecutors public Develope...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14541 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63369/ Test PASSed. ---

[GitHub] spark issue #14541: [SPARK-16953] Make requestTotalExecutors public Develope...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14541 **[Test build #63369 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63369/consoleFull)** for PR 14541 at commit [`5e5d67d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14544: [SPARK-16956] Make ApplicationState.MAX_NUM_RETRY config...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14544 **[Test build #63374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63374/consoleFull)** for PR 14544 at commit [`d9bab26`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #14541: [SPARK-16953] Make requestTotalExecutors public Develope...

2016-08-08 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14541 lgtm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #14544: [SPARK-16956] Make ApplicationState.MAX_NUM_RETRY...

2016-08-08 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/14544 [SPARK-16956] Make ApplicationState.MAX_NUM_RETRY configurable ## What changes were proposed in this pull request? This patch introduces a new configuration, `spark.deploy.maxExecutorRet

[GitHub] spark pull request #13701: [SPARK-15639][SPARK-16321][SQL] Push down filter ...

2016-08-08 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/13701#discussion_r73934292 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -357,10 +358,27 @@ private[sql] class Par

[GitHub] spark pull request #13701: [SPARK-15639][SPARK-16321][SQL] Push down filter ...

2016-08-08 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/13701#discussion_r73934243 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -213,7 +213,9 @@ private[sql] case class FileSourceScanExec(

[GitHub] spark issue #14175: [SPARK-16522][MESOS] Spark application throws exception ...

2016-08-08 Thread mgummelt
Github user mgummelt commented on the issue: https://github.com/apache/spark/pull/14175 Thanks @sun-rui ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, o

[GitHub] spark issue #14175: [SPARK-16522][MESOS] Spark application throws exception ...

2016-08-08 Thread mgummelt
Github user mgummelt commented on the issue: https://github.com/apache/spark/pull/14175 @srowen LGTM. Can you merge this into master/2.0? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14469: [SPARK-16700] [PYSPARK] [SQL] create DataFrame fr...

2016-08-08 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/14469#discussion_r73932092 --- Diff: python/pyspark/sql/session.py --- @@ -432,14 +430,9 @@ def createDataFrame(self, data, schema=None, samplingRatio=None): ``byte`` i

[GitHub] spark issue #14509: [SPARK-16924][SQL] - Support option("inferSchema", true)...

2016-08-08 Thread xwu0226
Github user xwu0226 commented on the issue: https://github.com/apache/spark/pull/14509 I see. So the purpose of `spark.sql.streaming.schemaInference` is to allow/disallow Spark SQL to infer schema on streaming file datasource. A bit confusion here. Many thanks! For CSV file form

[GitHub] spark pull request #14065: [SPARK-14743][YARN] Add a configurable credential...

2016-08-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14065#discussion_r73931365 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/security/CredentialUpdater.scala --- @@ -53,50 +53,58 @@ private[spark] class CredentialUpdater(

[GitHub] spark pull request #14065: [SPARK-14743][YARN] Add a configurable credential...

2016-08-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14065#discussion_r73931312 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/security/CredentialUpdater.scala --- @@ -53,50 +53,58 @@ private[spark] class CredentialUpdater(

[GitHub] spark pull request #14065: [SPARK-14743][YARN] Add a configurable credential...

2016-08-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14065#discussion_r73931240 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/security/CredentialUpdater.scala --- @@ -53,50 +53,58 @@ private[spark] class CredentialUpdater(

[GitHub] spark pull request #14065: [SPARK-14743][YARN] Add a configurable credential...

2016-08-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14065#discussion_r73931146 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/security/CredentialUpdater.scala --- @@ -53,50 +53,58 @@ private[spark] class CredentialUpdater(

[GitHub] spark pull request #14065: [SPARK-14743][YARN] Add a configurable credential...

2016-08-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14065#discussion_r73931184 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/security/CredentialUpdater.scala --- @@ -53,50 +53,58 @@ private[spark] class CredentialUpdater(

[GitHub] spark pull request #14065: [SPARK-14743][YARN] Add a configurable credential...

2016-08-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14065#discussion_r73931091 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/security/CredentialUpdater.scala --- @@ -53,50 +53,58 @@ private[spark] class CredentialUpdater(

[GitHub] spark issue #14454: [Minor] [ML] Rename TreeEnsembleModels to TreeEnsembleMo...

2016-08-08 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/14454 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

[GitHub] spark pull request #14065: [SPARK-14743][YARN] Add a configurable credential...

2016-08-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14065#discussion_r73930764 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/security/CredentialUpdater.scala --- @@ -53,50 +53,58 @@ private[spark] class CredentialUpdater(

[GitHub] spark issue #14522: [Spark-16508][SparkR] Split docs for arrange and orderBy...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14522 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63368/ Test PASSed. ---

[GitHub] spark issue #14522: [Spark-16508][SparkR] Split docs for arrange and orderBy...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14522 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bugs in C...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14531 **[Test build #63372 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63372/consoleFull)** for PR 14531 at commit [`d0e9217`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #14522: [Spark-16508][SparkR] Split docs for arrange and orderBy...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14522 **[Test build #63368 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63368/consoleFull)** for PR 14522 at commit [`f811a7d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14407: [SPARK-16779][TRIVIAL] Avoid using postfix operators whe...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14407 **[Test build #63373 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63373/consoleFull)** for PR 14407 at commit [`e4d8452`](https://github.com/apache/spark/commit/e

[GitHub] spark pull request #14065: [SPARK-14743][YARN] Add a configurable credential...

2016-08-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14065#discussion_r73930614 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/security/CredentialUpdater.scala --- @@ -53,50 +53,58 @@ private[spark] class CredentialUpdater(

[GitHub] spark pull request #14065: [SPARK-14743][YARN] Add a configurable credential...

2016-08-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14065#discussion_r73930350 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/security/CredentialUpdater.scala --- @@ -53,50 +53,58 @@ private[spark] class CredentialUpdater(

[GitHub] spark pull request #14065: [SPARK-14743][YARN] Add a configurable credential...

2016-08-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14065#discussion_r73930244 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/security/CredentialUpdater.scala --- @@ -53,50 +53,58 @@ private[spark] class CredentialUpdater(

[GitHub] spark issue #10846: [SPARK-12920][SQL] Fix high CPU usage in spark thrift se...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/10846 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63367/ Test PASSed. ---

[GitHub] spark issue #10846: [SPARK-12920][SQL] Fix high CPU usage in spark thrift se...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/10846 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #10846: [SPARK-12920][SQL] Fix high CPU usage in spark thrift se...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/10846 **[Test build #63367 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63367/consoleFull)** for PR 10846 at commit [`096ce46`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14266: [SPARK-16526][SQL] Benchmarking Performance for F...

2016-08-08 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/14266#discussion_r73928683 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/AggregateBenchmark.scala --- @@ -1078,6 +1078,146 @@ class AggregateBenchmark exte

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-08-08 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r73926983 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -25,55 +25,57 @@ import org.apache.spark.sql.

[GitHub] spark issue #14543: Bugs/mvn3.3.3

2016-08-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14543 This is already in `branch-1.6`: https://github.com/apache/spark/blob/branch-1.6/build/mvn We can't change releases/tags of course. Yeah, this is because non-current versions eventually get moved

[GitHub] spark issue #14407: [SPARK-16779][TRIVIAL] Avoid using postfix operators whe...

2016-08-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14407 Sounds like no objections on using postfix seconds operator - I'll go ahead and switch it to that. --- If your project is set up for it, you can reply to this email and have your reply appear on Gi

[GitHub] spark issue #14543: Bugs/mvn3.3.3

2016-08-08 Thread rickalm
Github user rickalm commented on the issue: https://github.com/apache/spark/pull/14543 Was considering withdrawing the PR. Here are my thoughts Tag v1.6.2 specifies mvn 3.3.3 (unless user intentionally overrides). Users who previously built wanting to replicate their previous

[GitHub] spark issue #14542: [SPARK-16930][yarn] Fix a couple of races in cluster app...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14542 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63370/ Test PASSed. ---

[GitHub] spark issue #14542: [SPARK-16930][yarn] Fix a couple of races in cluster app...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14542 **[Test build #63370 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63370/consoleFull)** for PR 14542 at commit [`854a53d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14542: [SPARK-16930][yarn] Fix a couple of races in cluster app...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14542 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14397: [SPARK-16771][SQL] WITH clause should not fall into infi...

2016-08-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14397 Hi, @hvanhovell . Sorry for late update. I updated the PR description and code. Could you review this PR again? --- If your project is set up for it, you can reply to this email and ha

[GitHub] spark pull request #14543: Bugs/mvn3.3.3

2016-08-08 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/14543#discussion_r73922880 --- Diff: build/mvn --- @@ -72,7 +72,7 @@ install_mvn() { local MVN_VERSION="3.3.3" install_app \ - "https://www.apache.org/dyn/cl

[GitHub] spark issue #14519: [SPARK-16933] [ML] Fix AFTAggregator in AFTSurvivalRegre...

2016-08-08 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/14519 LGTM. Will be nice to see the compassion of shuffle write size, and then will be ready to merge. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #14543: Bugs/mvn3.3.3

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14543 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #14543: Bugs/mvn3.3.3

2016-08-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14543 cc @srowen for build update --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wis

[GitHub] spark pull request #14519: [SPARK-16933] [ML] Fix AFTAggregator in AFTSurviv...

2016-08-08 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/14519#discussion_r73922053 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/AFTSurvivalRegression.scala --- @@ -478,21 +482,23 @@ object AFTSurvivalRegressionModel extends

[GitHub] spark pull request #14543: Bugs/mvn3.3.3

2016-08-08 Thread rickalm
GitHub user rickalm opened a pull request: https://github.com/apache/spark/pull/14543 Bugs/mvn3.3.3 ## What changes were proposed in this pull request? Changing the URI for Maven 3.3.3, version is no longer available on previous endpoint ## How was this patch teste

[GitHub] spark issue #14539: [SPARK-16947][SQL] Improve type coercion for inline tabl...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14539 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14539: [SPARK-16947][SQL] Improve type coercion for inline tabl...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14539 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63366/ Test FAILed. ---

[GitHub] spark issue #14539: [SPARK-16947][SQL] Improve type coercion for inline tabl...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14539 **[Test build #63366 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63366/consoleFull)** for PR 14539 at commit [`9a827ce`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14397: [SPARK-16771][SQL] WITH clause should not fall into infi...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14397 **[Test build #63371 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63371/consoleFull)** for PR 14397 at commit [`b10ee10`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #14542: [SPARK-16930][yarn] Fix a couple of races in cluster app...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14542 **[Test build #63370 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63370/consoleFull)** for PR 14542 at commit [`854a53d`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #5048: [SPARK-6309] [SQL] [MLlib] Implement MatrixUDT

2016-08-08 Thread karthikvadla16
Github user karthikvadla16 commented on the issue: https://github.com/apache/spark/pull/5048 Hi, Can anyone please share me a example for using MatrixUDT as datatype column in Spark SQL dataframe. I'm trying to load bunch of images into dataframe. Each image as matrix in data

[GitHub] spark pull request #14508: [SPARK-16586][core] Handle JVM errors printed to ...

2016-08-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14508 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14508: [SPARK-16586][core] Handle JVM errors printed to stdout.

2016-08-08 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14508 Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #14542: [SPARK-16930][yarn] Fix a couple of races in clus...

2016-08-08 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/14542 [SPARK-16930][yarn] Fix a couple of races in cluster app initialization. There are two narrow races that could cause the ApplicationMaster to miss when the user application instantiates the Spark

[GitHub] spark issue #14541: Make requestTotalExecutors public Developer API to be co...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14541 **[Test build #63369 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63369/consoleFull)** for PR 14541 at commit [`5e5d67d`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #14541: Make requestTotalExecutors public Developer API t...

2016-08-08 Thread tdas
GitHub user tdas opened a pull request: https://github.com/apache/spark/pull/14541 Make requestTotalExecutors public Developer API to be consistent with requestExecutors/killExecutor ## What changes were proposed in this pull request? RequestExecutors and killExecutor are p

[GitHub] spark issue #14541: Make requestTotalExecutors public Developer API to be co...

2016-08-08 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/14541 @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

[GitHub] spark pull request #14376: [SPARK-16749][SQL] Simplify processing logic in L...

2016-08-08 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14376#discussion_r73917166 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WindowExec.scala --- @@ -565,7 +566,7 @@ private[execution] abstract class WindowFunctionFrame

[GitHub] spark issue #13886: [SPARK-16185] [SQL] Better Error Messages When Creating ...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13886 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63365/ Test PASSed. ---

[GitHub] spark issue #13886: [SPARK-16185] [SQL] Better Error Messages When Creating ...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13886 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #13146: [SPARK-13081][PYSPARK][SPARK_SUBMIT]. Allow set p...

2016-08-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/13146#discussion_r73916554 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -64,6 +64,10 @@ /** Configuration key for the number of executor C

[GitHub] spark issue #13886: [SPARK-16185] [SQL] Better Error Messages When Creating ...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13886 **[Test build #63365 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63365/consoleFull)** for PR 13886 at commit [`8a4d2b2`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14518: [SPARK-16610][SQL] Add `orc.compress` as an alias for `c...

2016-08-08 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14518 I think it is fine that `compression` takes precedence. btw, is this flag used by other data sources? --- If your project is set up for it, you can reply to this email and have your reply appear on G

[GitHub] spark pull request #14518: [SPARK-16610][SQL] Add `orc.compress` as an alias...

2016-08-08 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14518#discussion_r73914641 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcOptions.scala --- @@ -31,7 +30,8 @@ private[orc] class OrcOptions( * Acceptable valu

[GitHub] spark pull request #14518: [SPARK-16610][SQL] Add `orc.compress` as an alias...

2016-08-08 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14518#discussion_r73914816 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcQuerySuite.scala --- @@ -161,6 +161,29 @@ class OrcQuerySuite extends QueryTest with Before

[GitHub] spark issue #14515: [SPARK-16926] [SQL] Remove partition columns from partit...

2016-08-08 Thread dafrista
Github user dafrista commented on the issue: https://github.com/apache/spark/pull/14515 This triggers the else case here: https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala#L368. cc: @andrewor14 Can you please ta

[GitHub] spark issue #14414: [SPARK-16809] enable history server links in dispatcher ...

2016-08-08 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/14414 I tested it with out suite. Two comments: a. Its not convenient to have only one option to add the history server uri. I had to create a properties file to pass it to the dispatcher process. I wo

[GitHub] spark pull request #14534: [SPARK-16941]Add SynchronizedMap trait with Map i...

2016-08-08 Thread markhamstra
Github user markhamstra commented on a diff in the pull request: https://github.com/apache/spark/pull/14534#discussion_r73908742 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/server/SparkSQLOperationManager.scala --- @@ -39,8 +39,10 @@ priva

[GitHub] spark issue #14533: [SPARK-16606] [CORE] isleading warning for SparkContext....

2016-08-08 Thread markhamstra
Github user markhamstra commented on the issue: https://github.com/apache/spark/pull/14533 PR title typo? Intended "misleading"? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14522: [Spark-16508][SparkR] Split docs for arrange and orderBy...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14522 **[Test build #63368 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63368/consoleFull)** for PR 14522 at commit [`f811a7d`](https://github.com/apache/spark/commit/f

[GitHub] spark pull request #14522: [Spark-16508][SparkR] Split docs for arrange and ...

2016-08-08 Thread junyangq
Github user junyangq commented on a diff in the pull request: https://github.com/apache/spark/pull/14522#discussion_r73907661 --- Diff: R/pkg/R/generics.R --- @@ -551,7 +551,7 @@ setGeneric("merge") #' @export setGeneric("mutate", function(.data, ...) {standardGeneric("mut

[GitHub] spark issue #14411: [SPARK-16804][SQL] Correlated subqueries containing non-...

2016-08-08 Thread nsyca
Github user nsyca commented on the issue: https://github.com/apache/spark/pull/14411 @hvanhovell, Thanks for getting the PR merged and sorry for causing a few hiccups before I got it right. It's my first PR. I have opened a new JIRA, SPARK-16951, to track the NOT IN i

[GitHub] spark issue #10846: [SPARK-12920][SQL] Fix high CPU usage in spark thrift se...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/10846 **[Test build #63367 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63367/consoleFull)** for PR 10846 at commit [`096ce46`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #10846: [SPARK-12920][SQL] Fix high CPU usage in spark thrift se...

2016-08-08 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/10846 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #14522: [Spark-16508][SparkR] Split docs for arrange and ...

2016-08-08 Thread junyangq
Github user junyangq commented on a diff in the pull request: https://github.com/apache/spark/pull/14522#discussion_r73904520 --- Diff: R/pkg/R/WindowSpec.R --- @@ -82,16 +82,18 @@ setMethod("partitionBy", } }) -#' orderBy +#' Ordering C

[GitHub] spark pull request #14522: [Spark-16508][SparkR] Split docs for arrange and ...

2016-08-08 Thread junyangq
Github user junyangq commented on a diff in the pull request: https://github.com/apache/spark/pull/14522#discussion_r73903804 --- Diff: R/pkg/R/DataFrame.R --- @@ -2121,7 +2121,7 @@ setMethod("arrange", }) #' @rdname arrange -#' @name orderBy --- En

[GitHub] spark issue #14539: [SPARK-16947][SQL] Improve type coercion for inline tabl...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14539 **[Test build #63366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63366/consoleFull)** for PR 14539 at commit [`9a827ce`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #14519: [SPARK-16933] [ML] Fix AFTAggregator in AFTSurvivalRegre...

2016-08-08 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14519 @yanboliang Can you do a quick test to make sure the shuffle write size is the expected size? For example, in logistic regression only the gradient should be serialized which is an array of `numFeatu

[GitHub] spark issue #13146: [SPARK-13081][PYSPARK][SPARK_SUBMIT]. Allow set pythonEx...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13146 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63361/ Test PASSed. ---

[GitHub] spark issue #13146: [SPARK-13081][PYSPARK][SPARK_SUBMIT]. Allow set pythonEx...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13146 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13146: [SPARK-13081][PYSPARK][SPARK_SUBMIT]. Allow set pythonEx...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13146 **[Test build #63361 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63361/consoleFull)** for PR 13146 at commit [`0a6a96c`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13886: [SPARK-16185] [SQL] Better Error Messages When Creating ...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13886 **[Test build #63365 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63365/consoleFull)** for PR 13886 at commit [`8a4d2b2`](https://github.com/apache/spark/commit/8

[GitHub] spark pull request #13818: [SPARK-15968][SQL] Nonempty partitioned metastore...

2016-08-08 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/13818#discussion_r73892580 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -298,6 +298,7 @@ case class InsertIntoHiveTable(

[GitHub] spark issue #14520: [SPARK-16934][ML][MLLib] Improve LogisticCostFun to avoi...

2016-08-08 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14520 Well, I suppose this won't go into another PR since the other one got merged. I think it's correct to make this match the approach taken in Linear Regression. The current code doesn't quite match tho

[GitHub] spark issue #14414: [SPARK-16809] enable history server links in dispatcher ...

2016-08-08 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/14414 I will add more comment im trying to test it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark issue #14540: [SPARK-16950] [PySpark] fromOffsets parameter support in...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14540 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63363/ Test PASSed. ---

[GitHub] spark issue #14540: [SPARK-16950] [PySpark] fromOffsets parameter support in...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14540 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14540: [SPARK-16950] [PySpark] fromOffsets parameter support in...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14540 **[Test build #63363 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63363/consoleFull)** for PR 14540 at commit [`d2a4027`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14523: [SPARK-16936] [SQL] Case Sensitivity Support for Refresh...

2016-08-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14523 thanks, merging to master and 2.0! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enable

[GitHub] spark issue #14539: [SPARK-16947][SQL] Improve type coercion for inline tabl...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14539 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63360/ Test FAILed. ---

[GitHub] spark issue #14539: [SPARK-16947][SQL] Improve type coercion for inline tabl...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14539 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14539: [SPARK-16947][SQL] Improve type coercion for inline tabl...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14539 **[Test build #63360 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63360/consoleFull)** for PR 14539 at commit [`3f3aa93`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14523: [SPARK-16936] [SQL] Case Sensitivity Support for ...

2016-08-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14523 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14517: [SPARK-16931][PYTHON] PySpark APIS for bucketBy and sort...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14517 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63364/ Test FAILed. ---

[GitHub] spark issue #14517: [SPARK-16931][PYTHON] PySpark APIS for bucketBy and sort...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14517 **[Test build #63364 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63364/consoleFull)** for PR 14517 at commit [`47d9ef7`](https://github.com/apache/spark/commit/

<    1   2   3   4   5   6   7   >