[jira] [Assigned] (SPARK-11701) YARN - dynamic allocation and speculation active task accounting wrong

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11701: Assignee: Thomas Graves (was: Apache Spark) > YARN - dynamic allocation and speculation

[jira] [Commented] (SPARK-11701) YARN - dynamic allocation and speculation active task accounting wrong

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120089#comment-15120089 ] Apache Spark commented on SPARK-11701: -- User 'tgravescs' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13045) Clean up ColumnarBatch.Row and Column.Struct

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13045: Assignee: Apache Spark > Clean up ColumnarBatch.Row and Column.Struct >

[jira] [Created] (SPARK-13049) Cannot use FIRST/LAST ignoreNulls option in 1.6 and below

2016-01-27 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-13049: - Summary: Cannot use FIRST/LAST ignoreNulls option in 1.6 and below Key: SPARK-13049 URL: https://issues.apache.org/jira/browse/SPARK-13049 Project: Spark

[jira] [Resolved] (SPARK-12896) Send only accumulator updates, not TaskMetrics, to the driver

2016-01-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-12896. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10835

[jira] [Assigned] (SPARK-11701) YARN - dynamic allocation and speculation active task accounting wrong

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11701: Assignee: Apache Spark (was: Thomas Graves) > YARN - dynamic allocation and speculation

[jira] [Updated] (SPARK-13049) Cannot use FIRST/LAST ignoreNulls option in 1.6 and below

2016-01-27 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-13049: -- Description: {{FIRST/LAST}} have an ignoreNulls option. These cannot be used in

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120110#comment-15120110 ] Apache Spark commented on SPARK-12177: -- User 'markgrover' has created a pull request for this issue:

[jira] [Created] (SPARK-13050) Scalatest tags fail builds with the addition of the sketch module

2016-01-27 Thread Alex Bozarth (JIRA)
Alex Bozarth created SPARK-13050: Summary: Scalatest tags fail builds with the addition of the sketch module Key: SPARK-13050 URL: https://issues.apache.org/jira/browse/SPARK-13050 Project: Spark

[jira] [Commented] (SPARK-13050) Scalatest tags fail builds with the addition of the sketch module

2016-01-27 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120136#comment-15120136 ] Alex Bozarth commented on SPARK-13050: -- I have a fix, will open a PR shortly > Scalatest tags fail

[jira] [Assigned] (SPARK-13050) Scalatest tags fail builds with the addition of the sketch module

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13050: Assignee: (was: Apache Spark) > Scalatest tags fail builds with the addition of the

[jira] [Commented] (SPARK-13050) Scalatest tags fail builds with the addition of the sketch module

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120144#comment-15120144 ] Apache Spark commented on SPARK-13050: -- User 'ajbozarth' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13050) Scalatest tags fail builds with the addition of the sketch module

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13050: Assignee: Apache Spark > Scalatest tags fail builds with the addition of the sketch

[jira] [Issue Comment Deleted] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

2016-01-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10945: -- Comment: was deleted (was: User 'dmarcous' has created a pull request for this issue:

[jira] [Updated] (SPARK-12811) Estimator interface for generalized linear models (GLMs)

2016-01-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-12811: -- Assignee: Yanbo Liang > Estimator interface for generalized linear models (GLMs) >

[jira] [Updated] (SPARK-12811) Estimator interface for generalized linear models (GLMs)

2016-01-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-12811: -- Shepherd: Xiangrui Meng > Estimator interface for generalized linear models (GLMs) >

[jira] [Updated] (SPARK-13048) EMLDAOptimizer deletes dependent checkpoint of DistributedLDAModel

2016-01-27 Thread Jeff Stein (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Stein updated SPARK-13048: --- Description: In EMLDAOptimizer, all checkpoints are deleted before returning the

[jira] [Resolved] (SPARK-13021) Fail fast when custom RDD's violate RDD.partition's API contract

2016-01-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-13021. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10932

[jira] [Resolved] (SPARK-12818) Implement Bloom filter and count-min sketch in DataFrames

2016-01-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12818. - Resolution: Fixed Fix Version/s: 2.0.0 > Implement Bloom filter and count-min sketch in

[jira] [Resolved] (SPARK-12938) Bloom filter DataFrame API integration

2016-01-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12938. - Resolution: Fixed Fix Version/s: 2.0.0 > Bloom filter DataFrame API integration >

[jira] [Commented] (SPARK-13049) Cannot use FIRST/LAST ignoreNulls option in 1.6 and below

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120214#comment-15120214 ] Apache Spark commented on SPARK-13049: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13049) Cannot use FIRST/LAST ignoreNulls option in 1.6 and below

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13049: Assignee: (was: Apache Spark) > Cannot use FIRST/LAST ignoreNulls option in 1.6 and

[jira] [Assigned] (SPARK-13049) Cannot use FIRST/LAST ignoreNulls option in 1.6 and below

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13049: Assignee: Apache Spark > Cannot use FIRST/LAST ignoreNulls option in 1.6 and below >

[jira] [Resolved] (SPARK-12866) Migrate the ExtendedHiveQlParser to the new parser

2016-01-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12866. - Resolution: Fixed Assignee: Herman van Hovell Fix Version/s: 2.0.0 > Migrate the

[jira] [Commented] (SPARK-13049) Cannot use FIRST/LAST ignoreNulls option in 1.6 and below

2016-01-27 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120220#comment-15120220 ] Herman van Hovell commented on SPARK-13049: --- I have opened a PR for this. This adds some API to

[jira] [Resolved] (SPARK-12865) Migrate the SparkSQLParser to the new parser

2016-01-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12865. - Resolution: Fixed Assignee: Herman van Hovell Fix Version/s: 2.0.0 > Migrate the

[jira] [Commented] (SPARK-12984) Not able to read CSV file using Spark 1.4.0

2016-01-27 Thread Jai Murugesh Rajasekaran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120229#comment-15120229 ] Jai Murugesh Rajasekaran commented on SPARK-12984: -- Thanks for your response...but it

[jira] [Commented] (SPARK-1792) Missing Spark-Shell Configure Options

2016-01-27 Thread Jim Lohse (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120248#comment-15120248 ] Jim Lohse commented on SPARK-1792: -- Was there any movement on this? I am seeing an unresolved status and

[jira] [Commented] (SPARK-13034) PySpark ml.classification support export/import

2016-01-27 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120648#comment-15120648 ] Yanbo Liang commented on SPARK-13034: - Hi Miao, Please feel free to take this one, thanks! I think

[jira] [Created] (SPARK-13056) Map column would throw NPE if value is null

2016-01-27 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-13056: --- Summary: Map column would throw NPE if value is null Key: SPARK-13056 URL: https://issues.apache.org/jira/browse/SPARK-13056 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-13041) Add a driver ui link and a mesos sandbox link on the dispatcher's ui page for each driver

2016-01-27 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-13041: Description: It would be convenient to have the uri of the driver's ui and the

[jira] [Updated] (SPARK-13041) Add a driver ui link and a mesos sandbox link on the dispatcher's ui page for each driver

2016-01-27 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-13041: Description: It would be convenient to have the uri of the driver's ui and the

[jira] [Resolved] (SPARK-5209) Jobs fail with "unexpected value" exception in certain environments

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5209. -- Resolution: Not A Problem > Jobs fail with "unexpected value" exception in certain environments >

[jira] [Resolved] (SPARK-1079) EC2 scripts should allow mounting as XFS or EXT4

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1079. -- Resolution: Won't Fix > EC2 scripts should allow mounting as XFS or EXT4 >

[jira] [Resolved] (SPARK-5851) spark_ec2.py ssh failure retry handling not always appropriate

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5851. -- Resolution: Won't Fix > spark_ec2.py ssh failure retry handling not always appropriate >

[jira] [Resolved] (SPARK-5433) Spark EC2 doesn't mount local disks for all instance types

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5433. -- Resolution: Won't Fix > Spark EC2 doesn't mount local disks for all instance types >

[jira] [Commented] (SPARK-5629) Add spark-ec2 action to return info about an existing cluster

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118972#comment-15118972 ] Sean Owen commented on SPARK-5629: -- OK, I'm not sure how to implement that. There are lots of EC2 issues

[jira] [Created] (SPARK-13040) JDBC using SPARK_CLASSPATH is deprecated but is the only way documented

2016-01-27 Thread JIRA
Sebastián Ramírez created SPARK-13040: - Summary: JDBC using SPARK_CLASSPATH is deprecated but is the only way documented Key: SPARK-13040 URL: https://issues.apache.org/jira/browse/SPARK-13040

[jira] [Updated] (SPARK-13041) Add a driver ui link and a mesos sandbox link on the dispatcher's ui page for each driver

2016-01-27 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-13041: Description: It would be convenient to have the uri of the driver's ui and the

[jira] [Updated] (SPARK-13041) Add a driver ui link and a mesos sandbox link on the dispatcher's ui page for each driver

2016-01-27 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-13041: Description: It would be convenient to have the uri of the driver's ui and the

[jira] [Updated] (SPARK-13041) Add a driver ui link and a mesos sandbox link on the dispatcher's ui page for each driver

2016-01-27 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-13041: Description: It would be convenient to have the uri of the driver's ui and the

[jira] [Comment Edited] (SPARK-13010) Survival analysis in SparkR

2016-01-27 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119054#comment-15119054 ] Yanbo Liang edited comment on SPARK-13010 at 1/27/16 11:09 AM: --- There are

[jira] [Comment Edited] (SPARK-13010) Survival analysis in SparkR

2016-01-27 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119054#comment-15119054 ] Yanbo Liang edited comment on SPARK-13010 at 1/27/16 11:09 AM: --- There are

[jira] [Commented] (SPARK-13010) Survival analysis in SparkR

2016-01-27 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119054#comment-15119054 ] Yanbo Liang commented on SPARK-13010: - There are two issues that we should discuss: 1, Support

[jira] [Updated] (SPARK-12834) Use type conversion instead of Ser/De of Pickle to transform JavaArray and JavaList

2016-01-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-12834: -- Fix Version/s: 1.6.1 > Use type conversion instead of Ser/De of Pickle to transform

[jira] [Updated] (SPARK-13029) Logistic regression returns inaccurate results when there is a column with identical value, and fit_intercept=false

2016-01-27 Thread Shuo Xiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuo Xiang updated SPARK-13029: --- Description: This is a bug that appears while fitting a Logistic Regression model with

[jira] [Commented] (SPARK-12854) Vectorize Parquet reader

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118912#comment-15118912 ] Apache Spark commented on SPARK-12854: -- User 'jaceklaskowski' has created a pull request for this

[jira] [Resolved] (SPARK-12983) Correct metrics.properties.template

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12983. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10902

[jira] [Updated] (SPARK-12983) Correct metrics.properties.template

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12983: -- Assignee: Benjamin Fradet > Correct metrics.properties.template > ---

[jira] [Updated] (SPARK-12105) Add a DataFrame.show() with argument for output PrintStream

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12105: -- Fix Version/s: (was: 2.0.0) > Add a DataFrame.show() with argument for output PrintStream >

[jira] [Resolved] (SPARK-6516) Coupling between default Hadoop versions in Spark build vs. ec2 scripts

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6516. -- Resolution: Won't Fix > Coupling between default Hadoop versions in Spark build vs. ec2 scripts >

[jira] [Resolved] (SPARK-6403) Launch master as spot instance on EC2

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6403. -- Resolution: Won't Fix > Launch master as spot instance on EC2 > - >

[jira] [Resolved] (SPARK-5189) Reorganize EC2 scripts so that nodes can be provisioned independent of Spark master

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5189. -- Resolution: Won't Fix > Reorganize EC2 scripts so that nodes can be provisioned independent of Spark >

[jira] [Resolved] (SPARK-5398) Support the eu-central-1 region for spark-ec2

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5398. -- Resolution: Won't Fix > Support the eu-central-1 region for spark-ec2 >

[jira] [Updated] (SPARK-13029) Logistic regression returns inaccurate results when there is a column with identical value, and fit_intercept=false

2016-01-27 Thread Shuo Xiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuo Xiang updated SPARK-13029: --- Description: This is a bug that appears while fitting a Logistic Regression model with

[jira] [Updated] (SPARK-13029) Logistic regression returns inaccurate results when there is a column with identical value, and fit_intercept=false

2016-01-27 Thread Shuo Xiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuo Xiang updated SPARK-13029: --- Description: This is a bug that appears while fitting a Logistic Regression model with

[jira] [Resolved] (SPARK-5149) Type mismatch when defining classes in Spark REPL

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5149. -- Resolution: Duplicate No doubt a slightly different manifestation but yes the same cause and therefore

[jira] [Resolved] (SPARK-6474) Replace image.run with connection.run_instances in spark_ec2.py

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6474. -- Resolution: Won't Fix > Replace image.run with connection.run_instances in spark_ec2.py >

[jira] [Created] (SPARK-13034) PySpark ml.classification support export/import

2016-01-27 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-13034: --- Summary: PySpark ml.classification support export/import Key: SPARK-13034 URL: https://issues.apache.org/jira/browse/SPARK-13034 Project: Spark Issue Type:

[jira] [Created] (SPARK-13035) PySpark ml.clustering support export/import

2016-01-27 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-13035: --- Summary: PySpark ml.clustering support export/import Key: SPARK-13035 URL: https://issues.apache.org/jira/browse/SPARK-13035 Project: Spark Issue Type:

[jira] [Commented] (SPARK-13004) Support Non-Volatile Data and Operations

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118916#comment-15118916 ] Sean Owen commented on SPARK-13004: --- OK, I don't think this JIRA itself is actionable then. I'd like to

[jira] [Commented] (SPARK-12684) Matrix.toString should take a format for how each cell should be printed

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118918#comment-15118918 ] Sean Owen commented on SPARK-12684: --- toString is more for utility and debugging purposes. I think you'd

[jira] [Commented] (SPARK-11560) Optimize KMeans implementation

2016-01-27 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118937#comment-15118937 ] Yanbo Liang commented on SPARK-11560: - [~yuhaoyan] I think the first step is to use BLAS Level 3

[jira] [Created] (SPARK-13041) Add a driver ui link and a mesos sandbox link on the dispatcher's ui page for each driver

2016-01-27 Thread Stavros Kontopoulos (JIRA)
Stavros Kontopoulos created SPARK-13041: --- Summary: Add a driver ui link and a mesos sandbox link on the dispatcher's ui page for each driver Key: SPARK-13041 URL:

[jira] [Updated] (SPARK-13041) Add a driver ui link and a mesos sandbox link on the dispatcher's ui page for each driver

2016-01-27 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-13041: Description: It would be convenient to have the uri of the driver's ui and the

[jira] [Created] (SPARK-13038) PySpark ml.pipeline support export/import

2016-01-27 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-13038: --- Summary: PySpark ml.pipeline support export/import Key: SPARK-13038 URL: https://issues.apache.org/jira/browse/SPARK-13038 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-13041) Add a driver ui link and a mesos sandbox link on the dispatcher's ui page for each driver

2016-01-27 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-13041: Description: It would be convenient to have the uri of the driver's ui and the

[jira] [Updated] (SPARK-13041) Add a driver ui link and a mesos sandbox link on the dispatcher's ui page for each driver

2016-01-27 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-13041: Description: It would be convenient to have the uri of the driver's ui and the

[jira] [Commented] (SPARK-12832) spark on mesos dispacher need a constraints

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119327#comment-15119327 ] Apache Spark commented on SPARK-12832: -- User 'atongen' has created a pull request for this issue:

[jira] [Updated] (SPARK-13033) PySpark ml.regression support export/import

2016-01-27 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-13033: Summary: PySpark ml.regression support export/import (was: PySpark regression support

[jira] [Created] (SPARK-13037) PySpark ml.recommendation support export/import

2016-01-27 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-13037: --- Summary: PySpark ml.recommendation support export/import Key: SPARK-13037 URL: https://issues.apache.org/jira/browse/SPARK-13037 Project: Spark Issue Type:

[jira] [Created] (SPARK-13036) PySpark ml.feature support export/import

2016-01-27 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-13036: --- Summary: PySpark ml.feature support export/import Key: SPARK-13036 URL: https://issues.apache.org/jira/browse/SPARK-13036 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-12958) Map accumulator in spark

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12958. --- Resolution: Won't Fix > Map accumulator in spark > > > Key:

[jira] [Created] (SPARK-13039) Spark Streaming with Mesos shutdown without any reason on logs

2016-01-27 Thread Luis Alves (JIRA)
Luis Alves created SPARK-13039: -- Summary: Spark Streaming with Mesos shutdown without any reason on logs Key: SPARK-13039 URL: https://issues.apache.org/jira/browse/SPARK-13039 Project: Spark

[jira] [Commented] (SPARK-9740) first/last aggregate NULL behavior

2016-01-27 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119240#comment-15119240 ] Herman van Hovell commented on SPARK-9740: -- [~yhuai] I'll have a look. > first/last aggregate

[jira] [Commented] (SPARK-12940) Partition field in Spark SQL WHERE clause causing Exception

2016-01-27 Thread Brian Wheeler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119242#comment-15119242 ] Brian Wheeler commented on SPARK-12940: --- I tried this on the binary build of

[jira] [Commented] (SPARK-12792) Refactor RRDD to support R UDF

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119162#comment-15119162 ] Apache Spark commented on SPARK-12792: -- User 'sun-rui' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12792) Refactor RRDD to support R UDF

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12792: Assignee: Apache Spark > Refactor RRDD to support R UDF > --

[jira] [Assigned] (SPARK-13040) JDBC using SPARK_CLASSPATH is deprecated but is the only way documented

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13040: Assignee: Apache Spark > JDBC using SPARK_CLASSPATH is deprecated but is the only way

[jira] [Assigned] (SPARK-13040) JDBC using SPARK_CLASSPATH is deprecated but is the only way documented

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13040: Assignee: (was: Apache Spark) > JDBC using SPARK_CLASSPATH is deprecated but is the

[jira] [Commented] (SPARK-13040) JDBC using SPARK_CLASSPATH is deprecated but is the only way documented

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119197#comment-15119197 ] Apache Spark commented on SPARK-13040: -- User 'tiangolo' has created a pull request for this issue:

[jira] [Commented] (SPARK-13040) JDBC using SPARK_CLASSPATH is deprecated but is the only way documented

2016-01-27 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119198#comment-15119198 ] Sebastián Ramírez commented on SPARK-13040: --- I submitted a PR with the parameters that ended up

[jira] [Commented] (SPARK-13033) PySpark ml.regression support export/import

2016-01-27 Thread Tommy Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119233#comment-15119233 ] Tommy Yu commented on SPARK-13033: -- I can take this one, but may need 13032 merged at first. > PySpark

[jira] [Assigned] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10945: Assignee: (was: Apache Spark) > GraphX computes Pagerank with NaN (with some

[jira] [Assigned] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10945: Assignee: Apache Spark > GraphX computes Pagerank with NaN (with some datasets) >

[jira] [Issue Comment Deleted] (SPARK-13040) JDBC using SPARK_CLASSPATH is deprecated but is the only way documented

2016-01-27 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastián Ramírez updated SPARK-13040: -- Comment: was deleted (was: I submitted a PR with the parameters that ended up working

[jira] [Assigned] (SPARK-12792) Refactor RRDD to support R UDF

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12792: Assignee: (was: Apache Spark) > Refactor RRDD to support R UDF >

[jira] [Commented] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119167#comment-15119167 ] Apache Spark commented on SPARK-10945: -- User 'dmarcous' has created a pull request for this issue:

[jira] [Created] (SPARK-13042) Adding Specificity metric (= true negative rate) to MulticlassMetrics.scala

2016-01-27 Thread Daniel Marcous (JIRA)
Daniel Marcous created SPARK-13042: -- Summary: Adding Specificity metric (= true negative rate) to MulticlassMetrics.scala Key: SPARK-13042 URL: https://issues.apache.org/jira/browse/SPARK-13042

[jira] [Assigned] (SPARK-13042) Adding Specificity metric (= true negative rate) to MulticlassMetrics.scala

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13042: Assignee: Apache Spark > Adding Specificity metric (= true negative rate) to

[jira] [Assigned] (SPARK-13042) Adding Specificity metric (= true negative rate) to MulticlassMetrics.scala

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13042: Assignee: (was: Apache Spark) > Adding Specificity metric (= true negative rate) to

[jira] [Commented] (SPARK-13042) Adding Specificity metric (= true negative rate) to MulticlassMetrics.scala

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119186#comment-15119186 ] Apache Spark commented on SPARK-13042: -- User 'dmarcous' has created a pull request for this issue:

[jira] [Updated] (SPARK-13042) Adding Specificity metric (= true negative rate) to MulticlassMetrics.scala

2016-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13042: -- Target Version/s: (was: 1.6.0) Fix Version/s: (was: 1.6.0) [~dmarc...@gmail.com] this is

[jira] [Commented] (SPARK-5189) Reorganize EC2 scripts so that nodes can be provisioned independent of Spark master

2016-01-27 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119220#comment-15119220 ] Nicholas Chammas commented on SPARK-5189: - FWIW, I found this issue to be practically unsolvable

<    1   2