[jira] [Comment Edited] (SPARK-15227) InputStream stop-start semantics + empty implementations

2016-05-16 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286089#comment-15286089 ] Prashant Sharma edited comment on SPARK-15227 at 5/17/16 5:44 AM: -- If

[jira] [Commented] (SPARK-15227) InputStream stop-start semantics + empty implementations

2016-05-16 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286089#comment-15286089 ] Prashant Sharma commented on SPARK-15227: - If start and stop are overridden by a particular

[jira] [Commented] (SPARK-15344) Unable to set default log level for PySpark

2016-05-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286039#comment-15286039 ] Felix Cheung commented on SPARK-15344: -- This was the original change:

[jira] [Commented] (SPARK-15344) Unable to set default log level for PySpark

2016-05-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286038#comment-15286038 ] Felix Cheung commented on SPARK-15344: -- SPARK-14881 was to get pyspark and sparkR shell to match the

[jira] [Commented] (SPARK-13850) TimSort Comparison method violates its general contract

2016-05-16 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285980#comment-15285980 ] Yin Huai commented on SPARK-13850: -- Can you explain the root cause at here? > TimSort Comparison method

[jira] [Updated] (SPARK-15292) ML 2.0 QA: Scala APIs audit for classification

2016-05-16 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15292: -- Assignee: Yanbo Liang Target Version/s: 2.0.0 > ML 2.0 QA: Scala APIs

[jira] [Updated] (SPARK-15269) Creating external table leaves empty directory under warehouse directory

2016-05-16 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15269: --- Assignee: Xin Wu > Creating external table leaves empty directory under warehouse directory >

[jira] [Updated] (SPARK-15357) Cooperative spilling should check consumer memory mode

2016-05-16 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15357: -- Description: In TaskMemoryManager.java: {code} for (MemoryConsumer c: consumers) { if (c != consumer

[jira] [Created] (SPARK-15357) Cooperative spilling should check consumer memory mode

2016-05-16 Thread Andrew Or (JIRA)
Andrew Or created SPARK-15357: - Summary: Cooperative spilling should check consumer memory mode Key: SPARK-15357 URL: https://issues.apache.org/jira/browse/SPARK-15357 Project: Spark Issue Type:

[jira] [Commented] (SPARK-14752) LazilyGenerateOrdering throws NullPointerException

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285675#comment-15285675 ] Apache Spark commented on SPARK-14752: -- User 'bomeng' has created a pull request for this issue:

[jira] [Commented] (SPARK-14817) ML, Graph, R 2.0 QA: Programming guide update and migration guide

2016-05-16 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285626#comment-15285626 ] Joseph K. Bradley commented on SPARK-14817: --- Migration guide needs to note change from

[jira] [Resolved] (SPARK-14814) ML 2.0 QA: API: Java compatibility, docs

2016-05-16 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-14814. --- Resolution: Fixed Fix Version/s: 2.0.0 Given your review + the Java fix, I'll

[jira] [Commented] (SPARK-15194) Add Python ML API for MultivariateGaussian

2016-05-16 Thread praveen dareddy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285618#comment-15285618 ] praveen dareddy commented on SPARK-15194: - [~josephkb] Thanks for clarifying this. I will

[jira] [Commented] (SPARK-14810) ML, Graph 2.0 QA: API: Binary incompatible changes

2016-05-16 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285613#comment-15285613 ] Joseph K. Bradley commented on SPARK-14810: --- [~nick.pentre...@gmail.com] Thanks! Your

[jira] [Commented] (SPARK-7424) spark.ml classification, regression abstractions should add metadata to output column

2016-05-16 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285608#comment-15285608 ] Joseph K. Bradley commented on SPARK-7424: -- I'm retargeting for 2.1 since we need to focus on QA

[jira] [Updated] (SPARK-7424) spark.ml classification, regression abstractions should add metadata to output column

2016-05-16 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7424: - Target Version/s: 2.1.0 (was: 2.0.0) > spark.ml classification, regression abstractions

[jira] [Deleted] (SPARK-15356) AOL Customer Care Number @ 1800.545.7482 Help Desk Number & AOL MAIL Tech Support Phone Number

2016-05-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin deleted SPARK-15356: --- > AOL Customer Care Number @ 1800.545.7482 Help Desk Number & AOL MAIL Tech > Support Phone

[jira] [Updated] (SPARK-15328) Word2Vec import for original binary format

2016-05-16 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15328: -- Priority: Minor (was: Major) > Word2Vec import for original binary format >

[jira] [Updated] (SPARK-15328) Word2Vec import for original binary format

2016-05-16 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15328: -- Component/s: (was: MLlib) > Word2Vec import for original binary format >

[jira] [Created] (SPARK-15356) AOL Customer Care Number @ 1800.545.7482 Help Desk Number & AOL MAIL Tech Support Phone Number

2016-05-16 Thread lola pola (JIRA)
lola pola created SPARK-15356: - Summary: AOL Customer Care Number @ 1800.545.7482 Help Desk Number & AOL MAIL Tech Support Phone Number Key: SPARK-15356 URL: https://issues.apache.org/jira/browse/SPARK-15356

[jira] [Updated] (SPARK-15254) Improve ML pipeline Cross Validation Scaladoc & PyDoc

2016-05-16 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15254: -- Component/s: Documentation > Improve ML pipeline Cross Validation Scaladoc & PyDoc >

[jira] [Updated] (SPARK-15254) Improve ML pipeline Cross Validation Scaladoc & PyDoc

2016-05-16 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15254: -- Issue Type: Documentation (was: Improvement) > Improve ML pipeline Cross Validation

[jira] [Commented] (SPARK-15194) Add Python ML API for MultivariateGaussian

2016-05-16 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285595#comment-15285595 ] Joseph K. Bradley commented on SPARK-15194: --- This should be implemented using numpy, within

[jira] [Updated] (SPARK-15164) Mark classification algorithms as experimental where marked so in scala

2016-05-16 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15164: -- Target Version/s: 2.0.0 > Mark classification algorithms as experimental where marked

[jira] [Commented] (SPARK-15145) port binary classification evaluator to spark.ml

2016-05-16 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285577#comment-15285577 ] Joseph K. Bradley commented on SPARK-15145: --- [~wm624] Can you please update this JIRA title and

[jira] [Comment Edited] (SPARK-15145) port binary classification evaluator to spark.ml

2016-05-16 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285577#comment-15285577 ] Joseph K. Bradley edited comment on SPARK-15145 at 5/16/16 10:45 PM: -

[jira] [Created] (SPARK-15355) Pro-active block replenishment in case of node/executor failures

2016-05-16 Thread Shubham Chopra (JIRA)
Shubham Chopra created SPARK-15355: -- Summary: Pro-active block replenishment in case of node/executor failures Key: SPARK-15355 URL: https://issues.apache.org/jira/browse/SPARK-15355 Project: Spark

[jira] [Created] (SPARK-15354) Topology aware block replication strategies

2016-05-16 Thread Shubham Chopra (JIRA)
Shubham Chopra created SPARK-15354: -- Summary: Topology aware block replication strategies Key: SPARK-15354 URL: https://issues.apache.org/jira/browse/SPARK-15354 Project: Spark Issue Type:

[jira] [Updated] (SPARK-15353) Making peer selection for block replication pluggable

2016-05-16 Thread Shubham Chopra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shubham Chopra updated SPARK-15353: --- Attachment: BlockManagerSequenceDiagram.png Sequence diagram explaining the various calls

[jira] [Commented] (SPARK-3785) Support off-loading computations to a GPU

2016-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285539#comment-15285539 ] Sean Owen commented on SPARK-3785: -- That, and things like YARN labels are indeed a pre-requisite to be

[jira] [Created] (SPARK-15353) Making peer selection for block replication pluggable

2016-05-16 Thread Shubham Chopra (JIRA)
Shubham Chopra created SPARK-15353: -- Summary: Making peer selection for block replication pluggable Key: SPARK-15353 URL: https://issues.apache.org/jira/browse/SPARK-15353 Project: Spark

[jira] [Created] (SPARK-15352) Topology aware block replication

2016-05-16 Thread Shubham Chopra (JIRA)
Shubham Chopra created SPARK-15352: -- Summary: Topology aware block replication Key: SPARK-15352 URL: https://issues.apache.org/jira/browse/SPARK-15352 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-15100) Audit: ml.feature

2016-05-16 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285366#comment-15285366 ] Bryan Cutler commented on SPARK-15100: -- I can do a PR to update CountVectorizer and HashingTF >

[jira] [Commented] (SPARK-15230) Back quoted column with dot in it fails when running distinct on dataframe

2016-05-16 Thread Barry Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285321#comment-15285321 ] Barry Becker commented on SPARK-15230: -- I updated the description so it says distinct instead of

[jira] [Updated] (SPARK-15230) Back quoted column with dot in it fails when running distinct on dataframe

2016-05-16 Thread Barry Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Barry Becker updated SPARK-15230: - Description: When working with a dataframe columns with .'s in them must be backquoted (``) or

[jira] [Comment Edited] (SPARK-15230) Back quoted column with dot in it fails when running distinct on dataframe

2016-05-16 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285302#comment-15285302 ] Bo Meng edited comment on SPARK-15230 at 5/16/16 9:11 PM: -- In the description,

[jira] [Commented] (SPARK-15230) Back quoted column with dot in it fails when running distinct on dataframe

2016-05-16 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285302#comment-15285302 ] Bo Meng commented on SPARK-15230: - In the description, `it does not work for describe()` should be `it

[jira] [Assigned] (SPARK-15230) Back quoted column with dot in it fails when running distinct on dataframe

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15230: Assignee: (was: Apache Spark) > Back quoted column with dot in it fails when running

[jira] [Commented] (SPARK-15230) Back quoted column with dot in it fails when running distinct on dataframe

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285263#comment-15285263 ] Apache Spark commented on SPARK-15230: -- User 'bomeng' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15230) Back quoted column with dot in it fails when running distinct on dataframe

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15230: Assignee: Apache Spark > Back quoted column with dot in it fails when running distinct on

[jira] [Commented] (SPARK-3785) Support off-loading computations to a GPU

2016-05-16 Thread Bill Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285248#comment-15285248 ] Bill Zhao commented on SPARK-3785: -- Mesos has added the GPU support in 0.29 release.

[jira] [Resolved] (SPARK-14942) Reduce delay between batch construction and execution

2016-05-16 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-14942. -- Resolution: Fixed Assignee: Liwei Lin Fix Version/s: 2.0.0 > Reduce delay

[jira] [Commented] (SPARK-15072) Remove SparkSession.withHiveSupport

2016-05-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285112#comment-15285112 ] Nicholas Chammas commented on SPARK-15072: -- Brief note from [~yhuai] on the motivation behind

[jira] [Assigned] (SPARK-15186) Add user guide for Generalized Linear Regression.

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15186: Assignee: Seth Hendrickson (was: Apache Spark) > Add user guide for Generalized Linear

[jira] [Assigned] (SPARK-15186) Add user guide for Generalized Linear Regression.

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15186: Assignee: Apache Spark (was: Seth Hendrickson) > Add user guide for Generalized Linear

[jira] [Commented] (SPARK-15186) Add user guide for Generalized Linear Regression.

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285034#comment-15285034 ] Apache Spark commented on SPARK-15186: -- User 'sethah' has created a pull request for this issue:

[jira] [Resolved] (SPARK-15343) NoClassDefFoundError when initializing Spark with YARN

2016-05-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-15343. Resolution: Not A Problem Closing as "not a problem" since this is an issue with 3rd-party

[jira] [Commented] (SPARK-15343) NoClassDefFoundError when initializing Spark with YARN

2016-05-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284981#comment-15284981 ] Marcelo Vanzin commented on SPARK-15343: bq. at

[jira] [Assigned] (SPARK-15351) RowEncoder should support array as the external type for ArrayType

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15351: Assignee: Apache Spark (was: Wenchen Fan) > RowEncoder should support array as the

[jira] [Commented] (SPARK-15351) RowEncoder should support array as the external type for ArrayType

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284913#comment-15284913 ] Apache Spark commented on SPARK-15351: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15351) RowEncoder should support array as the external type for ArrayType

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15351: Assignee: Wenchen Fan (was: Apache Spark) > RowEncoder should support array as the

[jira] [Commented] (SPARK-15347) Problem select empty ORC table

2016-05-16 Thread Pedro Prado (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284907#comment-15284907 ] Pedro Prado commented on SPARK-15347: - Sorry Sean! my fault! > Problem select empty ORC table >

[jira] [Created] (SPARK-15351) RowEncoder should support array as the external type for ArrayType

2016-05-16 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-15351: --- Summary: RowEncoder should support array as the external type for ArrayType Key: SPARK-15351 URL: https://issues.apache.org/jira/browse/SPARK-15351 Project: Spark

[jira] [Commented] (SPARK-15272) DirectKafkaInputDStream doesn't work with window operation

2016-05-16 Thread Lubomir Nerad (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284776#comment-15284776 ] Lubomir Nerad commented on SPARK-15272: --- We can workaround the Kafka part of the issue. But what

[jira] [Assigned] (SPARK-15247) sqlCtx.read.parquet yields at least n_executors * n_cores tasks

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15247: Assignee: (was: Apache Spark) > sqlCtx.read.parquet yields at least n_executors *

[jira] [Commented] (SPARK-15247) sqlCtx.read.parquet yields at least n_executors * n_cores tasks

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284771#comment-15284771 ] Apache Spark commented on SPARK-15247: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15247) sqlCtx.read.parquet yields at least n_executors * n_cores tasks

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15247: Assignee: Apache Spark > sqlCtx.read.parquet yields at least n_executors * n_cores tasks

[jira] [Commented] (SPARK-15247) sqlCtx.read.parquet yields at least n_executors * n_cores tasks

2016-05-16 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284763#comment-15284763 ] Takeshi Yamamuro commented on SPARK-15247: -- I'll make a pr to fix this. > sqlCtx.read.parquet

[jira] [Comment Edited] (SPARK-15247) sqlCtx.read.parquet yields at least n_executors * n_cores tasks

2016-05-16 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284486#comment-15284486 ] Takeshi Yamamuro edited comment on SPARK-15247 at 5/16/16 3:56 PM: --- Not

[jira] [Updated] (SPARK-15350) Add unit test function for LogisticRegressionWithLBFGS in JavaLogisticRegressionSuite

2016-05-16 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-15350: --- Priority: Minor (was: Major) > Add unit test function for LogisticRegressionWithLBFGS in >

[jira] [Assigned] (SPARK-15350) Add unit test function for LogisticRegressionWithLBFGS in JavaLogisticRegressionSuite

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15350: Assignee: Apache Spark > Add unit test function for LogisticRegressionWithLBFGS in >

[jira] [Commented] (SPARK-15350) Add unit test function for LogisticRegressionWithLBFGS in JavaLogisticRegressionSuite

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284718#comment-15284718 ] Apache Spark commented on SPARK-15350: -- User 'WeichenXu123' has created a pull request for this

[jira] [Assigned] (SPARK-15350) Add unit test function for LogisticRegressionWithLBFGS in JavaLogisticRegressionSuite

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15350: Assignee: (was: Apache Spark) > Add unit test function for

[jira] [Created] (SPARK-15350) Add unit test function for LogisticRegressionWithLBFGS in JavaLogisticRegressionSuite

2016-05-16 Thread Weichen Xu (JIRA)
Weichen Xu created SPARK-15350: -- Summary: Add unit test function for LogisticRegressionWithLBFGS in JavaLogisticRegressionSuite Key: SPARK-15350 URL: https://issues.apache.org/jira/browse/SPARK-15350

[jira] [Comment Edited] (SPARK-15348) Hive ACID

2016-05-16 Thread Ran Haim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284698#comment-15284698 ] Ran Haim edited comment on SPARK-15348 at 5/16/16 3:09 PM: --- This means that if

[jira] [Commented] (SPARK-15348) Hive ACID

2016-05-16 Thread Ran Haim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284698#comment-15284698 ] Ran Haim commented on SPARK-15348: -- If I have a transnational table in hive, I cannot use spark job to

[jira] [Closed] (SPARK-15349) Hive ACID

2016-05-16 Thread Ran Haim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ran Haim closed SPARK-15349. Resolution: Duplicate > Hive ACID > - > > Key: SPARK-15349 > URL:

[jira] [Resolved] (SPARK-15347) Problem select empty ORC table

2016-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15347. --- Resolution: Duplicate Fix Version/s: (was: 1.6.0) Please have a look through JIRA first

[jira] [Commented] (SPARK-15348) Hive ACID

2016-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284688#comment-15284688 ] Sean Owen commented on SPARK-15348: --- I suspect that's waay outside the goals of the project and a huge

[jira] [Created] (SPARK-15349) Hive ACID

2016-05-16 Thread Ran Haim (JIRA)
Ran Haim created SPARK-15349: Summary: Hive ACID Key: SPARK-15349 URL: https://issues.apache.org/jira/browse/SPARK-15349 Project: Spark Issue Type: New Feature Reporter: Ran Haim

[jira] [Created] (SPARK-15348) Hive ACID

2016-05-16 Thread Ran Haim (JIRA)
Ran Haim created SPARK-15348: Summary: Hive ACID Key: SPARK-15348 URL: https://issues.apache.org/jira/browse/SPARK-15348 Project: Spark Issue Type: New Feature Reporter: Ran Haim

[jira] [Created] (SPARK-15347) Problem select empty ORC table

2016-05-16 Thread Pedro Prado (JIRA)
Pedro Prado created SPARK-15347: --- Summary: Problem select empty ORC table Key: SPARK-15347 URL: https://issues.apache.org/jira/browse/SPARK-15347 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-15346) Reduce duplicate computation in picking initial points in LocalKMeans

2016-05-16 Thread Abraham Zhan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abraham Zhan updated SPARK-15346: - Description: h2.Main Issue I found that for KMans|| in mllib, when dataset is in large scale,

[jira] [Commented] (SPARK-15343) NoClassDefFoundError when initializing Spark with YARN

2016-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284623#comment-15284623 ] Sean Owen commented on SPARK-15343: --- SInce you're executing in a cluster, I think perhaps a better and

[jira] [Comment Edited] (SPARK-15343) NoClassDefFoundError when initializing Spark with YARN

2016-05-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284619#comment-15284619 ] Maciej Bryński edited comment on SPARK-15343 at 5/16/16 2:05 PM: - Thanks.

[jira] [Commented] (SPARK-15343) NoClassDefFoundError when initializing Spark with YARN

2016-05-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284619#comment-15284619 ] Maciej Bryński commented on SPARK-15343: I set spark.hadoop.yarn.timeline-service.enabled to

[jira] [Issue Comment Deleted] (SPARK-4924) Factor out code to launch Spark applications into a separate library

2016-05-16 Thread Stephen Boesch (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Boesch updated SPARK-4924: -- Comment: was deleted (was: Chiming in here as well: three of us are now asking for commentary /

[jira] [Commented] (SPARK-12154) Upgrade to Jersey 2

2016-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284617#comment-15284617 ] Sean Owen commented on SPARK-12154: --- No, I don't think so - let's keep the discussion in one place on

[jira] [Commented] (SPARK-15343) NoClassDefFoundError when initializing Spark with YARN

2016-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284612#comment-15284612 ] Sean Owen commented on SPARK-15343: --- Yes, of course that's the change that caused the behavior you're

[jira] [Commented] (SPARK-15343) NoClassDefFoundError when initializing Spark with YARN

2016-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284538#comment-15284538 ] Sean Owen commented on SPARK-15343: --- No, it's clearly a class needed by YARN and that's where it fails

[jira] [Commented] (SPARK-12154) Upgrade to Jersey 2

2016-05-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-12154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284509#comment-15284509 ] Maciej Bryński commented on SPARK-12154: I think this upgrade break compatibility with YARN.

[jira] [Commented] (SPARK-15343) NoClassDefFoundError when initializing Spark with YARN

2016-05-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284507#comment-15284507 ] Maciej Bryński commented on SPARK-15343: And the likely reason of problem.

[jira] [Comment Edited] (SPARK-15343) NoClassDefFoundError when initializing Spark with YARN

2016-05-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284506#comment-15284506 ] Maciej Bryński edited comment on SPARK-15343 at 5/16/16 1:52 PM: - I think

[jira] [Commented] (SPARK-15343) NoClassDefFoundError when initializing Spark with YARN

2016-05-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284506#comment-15284506 ] Maciej Bryński commented on SPARK-15343: I think it's too early for that. Exception is thrown on

[jira] [Assigned] (SPARK-15346) Reduce duplicate computation in picking initial points in LocalKMeans

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15346: Assignee: Apache Spark > Reduce duplicate computation in picking initial points in

[jira] [Commented] (SPARK-15346) Reduce duplicate computation in picking initial points in LocalKMeans

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284498#comment-15284498 ] Apache Spark commented on SPARK-15346: -- User 'mouendless' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15346) Reduce duplicate computation in picking initial points in LocalKMeans

2016-05-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15346: Assignee: (was: Apache Spark) > Reduce duplicate computation in picking initial

[jira] [Created] (SPARK-15346) Reduce duplicate computation in picking initial points in LocalKMeans

2016-05-16 Thread Abraham Zhan (JIRA)
Abraham Zhan created SPARK-15346: Summary: Reduce duplicate computation in picking initial points in LocalKMeans Key: SPARK-15346 URL: https://issues.apache.org/jira/browse/SPARK-15346 Project: Spark

[jira] [Commented] (SPARK-15247) sqlCtx.read.parquet yields at least n_executors * n_cores tasks

2016-05-16 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284486#comment-15284486 ] Takeshi Yamamuro commented on SPARK-15247: -- Not yet. Actually, I'm not sure that this issue

[jira] [Commented] (SPARK-4924) Factor out code to launch Spark applications into a separate library

2016-05-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284483#comment-15284483 ] Thomas Graves commented on SPARK-4924: -- [~javadba] If you have ideas on improving the documentation

[jira] [Commented] (SPARK-15343) NoClassDefFoundError when initializing Spark with YARN

2016-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284480#comment-15284480 ] Sean Owen commented on SPARK-15343: --- Yeah, though in theory that doesn't prevent it from being pulled

[jira] [Commented] (SPARK-15343) NoClassDefFoundError when initializing Spark with YARN

2016-05-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284465#comment-15284465 ] Maciej Bryński commented on SPARK-15343: [~srowen] I found that we change version of jersey

[jira] [Commented] (SPARK-14881) pyspark and sparkR shell default log level should match spark-shell/Scala

2016-05-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-14881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284454#comment-15284454 ] Maciej Bryński commented on SPARK-14881: [~felixcheung] Could you check this ?

[jira] [Commented] (SPARK-15344) Unable to set default log level for PySpark

2016-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284449#comment-15284449 ] Sean Owen commented on SPARK-15344: --- I know, but I'm suggesting it's probably more useful to continue

[jira] [Comment Edited] (SPARK-15344) Unable to set default log level for PySpark

2016-05-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-15344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284442#comment-15284442 ] Maciej Bryński edited comment on SPARK-15344 at 5/16/16 12:44 PM: -- Yep.

[jira] [Commented] (SPARK-15344) Unable to set default log level for PySpark

2016-05-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-15344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284442#comment-15284442 ] Maciej Bryński commented on SPARK-15344: Yep. I mention PR from this Jira in description. >

[jira] [Commented] (SPARK-15344) Unable to set default log level for PySpark

2016-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284438#comment-15284438 ] Sean Owen commented on SPARK-15344: --- Comment on SPARK-14881 then maybe? this sounds like a duplicate or

[jira] [Created] (SPARK-15345) Cannot connect to Hive databases

2016-05-16 Thread Piotr Milanowski (JIRA)
Piotr Milanowski created SPARK-15345: Summary: Cannot connect to Hive databases Key: SPARK-15345 URL: https://issues.apache.org/jira/browse/SPARK-15345 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-15343) NoClassDefFoundError when initializing Spark with YARN

2016-05-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284433#comment-15284433 ] Maciej Bryński commented on SPARK-15343: CC: [~vanzin] > NoClassDefFoundError when initializing

[jira] [Commented] (SPARK-15247) sqlCtx.read.parquet yields at least n_executors * n_cores tasks

2016-05-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284432#comment-15284432 ] Sean Owen commented on SPARK-15247: --- Did you actually open a PR for this? > sqlCtx.read.parquet yields

  1   2   >