[jira] [Closed] (SPARK-6424) Support user-defined aggregators in AggregateFunction

2017-03-24 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro closed SPARK-6424. --- Resolution: Duplicate > Support user-defined aggregators in AggregateFunction >

[jira] [Commented] (SPARK-6424) Support user-defined aggregators in AggregateFunction

2017-03-24 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941589#comment-15941589 ] Takeshi Yamamuro commented on SPARK-6424: - We already have fixed the Aggregation improvements in

[jira] [Commented] (SPARK-20090) Add StructType.fieldNames to Python API

2017-03-24 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941577#comment-15941577 ] Hyukjin Kwon commented on SPARK-20090: -- Hm.. don't we have {{names}}? Scala {code} scala>

[jira] [Assigned] (SPARK-17137) Add compressed support for multinomial logistic regression coefficients

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17137: Assignee: Apache Spark (was: Seth Hendrickson) > Add compressed support for multinomial

[jira] [Commented] (SPARK-17137) Add compressed support for multinomial logistic regression coefficients

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941560#comment-15941560 ] Apache Spark commented on SPARK-17137: -- User 'sethah' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17137) Add compressed support for multinomial logistic regression coefficients

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17137: Assignee: Seth Hendrickson (was: Apache Spark) > Add compressed support for multinomial

[jira] [Created] (SPARK-20091) DagScheduler should allow running concurrent attempts of a stage in case of multiple fetch failure

2017-03-24 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-20091: --- Summary: DagScheduler should allow running concurrent attempts of a stage in case of multiple fetch failure Key: SPARK-20091 URL: https://issues.apache.org/jira/browse/SPARK-20091

[jira] [Created] (SPARK-20090) Add StructType.fieldNames to Python API

2017-03-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-20090: - Summary: Add StructType.fieldNames to Python API Key: SPARK-20090 URL: https://issues.apache.org/jira/browse/SPARK-20090 Project: Spark Issue

[jira] [Commented] (SPARK-20089) Add DESC FUNCTION and DESC EXTENDED FUNCTION to SQLQueryTestSuite

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941410#comment-15941410 ] Apache Spark commented on SPARK-20089: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20089) Add DESC FUNCTION and DESC EXTENDED FUNCTION to SQLQueryTestSuite

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20089: Assignee: Apache Spark (was: Xiao Li) > Add DESC FUNCTION and DESC EXTENDED FUNCTION to

[jira] [Assigned] (SPARK-20089) Add DESC FUNCTION and DESC EXTENDED FUNCTION to SQLQueryTestSuite

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20089: Assignee: Xiao Li (was: Apache Spark) > Add DESC FUNCTION and DESC EXTENDED FUNCTION to

[jira] [Updated] (SPARK-20089) Add DESC FUNCTION and DESC EXTENDED FUNCTION to SQLQueryTestSuite

2017-03-24 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-20089: Summary: Add DESC FUNCTION and DESC EXTENDED FUNCTION to SQLQueryTestSuite (was: Added DESC FUNCTION and

[jira] [Created] (SPARK-20089) Added DESC FUNCTION and DESC EXTENDED FUNCTION to SQLQueryTestSuite

2017-03-24 Thread Xiao Li (JIRA)
Xiao Li created SPARK-20089: --- Summary: Added DESC FUNCTION and DESC EXTENDED FUNCTION to SQLQueryTestSuite Key: SPARK-20089 URL: https://issues.apache.org/jira/browse/SPARK-20089 Project: Spark

[jira] [Assigned] (SPARK-20088) Do not create new SparkContext in SparkR createSparkContext

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20088: Assignee: (was: Apache Spark) > Do not create new SparkContext in SparkR

[jira] [Assigned] (SPARK-20088) Do not create new SparkContext in SparkR createSparkContext

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20088: Assignee: Apache Spark > Do not create new SparkContext in SparkR createSparkContext >

[jira] [Commented] (SPARK-20088) Do not create new SparkContext in SparkR createSparkContext

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941394#comment-15941394 ] Apache Spark commented on SPARK-20088: -- User 'falaki' has created a pull request for this issue:

[jira] [Created] (SPARK-20088) Do not create new SparkContext in SparkR createSparkContext

2017-03-24 Thread Hossein Falaki (JIRA)
Hossein Falaki created SPARK-20088: -- Summary: Do not create new SparkContext in SparkR createSparkContext Key: SPARK-20088 URL: https://issues.apache.org/jira/browse/SPARK-20088 Project: Spark

[jira] [Updated] (SPARK-19408) cardinality estimation involving two columns of the same table

2017-03-24 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-19408: --- Target Version/s: 2.2.0 (was: 2.3.0) > cardinality estimation involving two columns of the same table >

[jira] [Created] (SPARK-20087) Include accumulators / taskMetrics when sending TaskKilled to onTaskEnd listeners

2017-03-24 Thread Charles Lewis (JIRA)
Charles Lewis created SPARK-20087: - Summary: Include accumulators / taskMetrics when sending TaskKilled to onTaskEnd listeners Key: SPARK-20087 URL: https://issues.apache.org/jira/browse/SPARK-20087

[jira] [Assigned] (SPARK-20040) Python API for ml.stat.ChiSquareTest

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20040: Assignee: Apache Spark > Python API for ml.stat.ChiSquareTest >

[jira] [Commented] (SPARK-20040) Python API for ml.stat.ChiSquareTest

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941367#comment-15941367 ] Apache Spark commented on SPARK-20040: -- User 'MrBago' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20040) Python API for ml.stat.ChiSquareTest

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20040: Assignee: (was: Apache Spark) > Python API for ml.stat.ChiSquareTest >

[jira] [Commented] (SPARK-20070) Redact datasource explain output

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941345#comment-15941345 ] Apache Spark commented on SPARK-20070: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Commented] (SPARK-19634) Feature parity for descriptive statistics in MLlib

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941322#comment-15941322 ] Apache Spark commented on SPARK-19634: -- User 'thunterdb' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19634) Feature parity for descriptive statistics in MLlib

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19634: Assignee: Apache Spark (was: Timothy Hunter) > Feature parity for descriptive statistics

[jira] [Assigned] (SPARK-19634) Feature parity for descriptive statistics in MLlib

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19634: Assignee: Timothy Hunter (was: Apache Spark) > Feature parity for descriptive statistics

[jira] [Resolved] (SPARK-19846) Add a flag to disable constraint propagation

2017-03-24 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19846. - Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.2.0 > Add a flag to

[jira] [Resolved] (SPARK-20070) Redact datasource explain output

2017-03-24 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-20070. - Resolution: Fixed Fix Version/s: 2.2.0 > Redact datasource explain output >

[jira] [Comment Edited] (SPARK-12216) Spark failed to delete temp directory

2017-03-24 Thread Jouni H (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940987#comment-15940987 ] Jouni H edited comment on SPARK-12216 at 3/24/17 10:25 PM: --- I was able to

[jira] [Updated] (SPARK-20086) issue with pyspark 2.1.0 window function

2017-03-24 Thread mandar uapdhye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mandar uapdhye updated SPARK-20086: --- Description: original post at [stackoverflow |

[jira] [Comment Edited] (SPARK-12216) Spark failed to delete temp directory

2017-03-24 Thread Jouni H (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940987#comment-15940987 ] Jouni H edited comment on SPARK-12216 at 3/24/17 9:58 PM: -- I was able to

[jira] [Comment Edited] (SPARK-12216) Spark failed to delete temp directory

2017-03-24 Thread Jouni H (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940987#comment-15940987 ] Jouni H edited comment on SPARK-12216 at 3/24/17 9:54 PM: -- I was able to

[jira] [Commented] (SPARK-19674) Ignore driver accumulator updates don't belong to the execution when merging all accumulator updates

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941213#comment-15941213 ] Apache Spark commented on SPARK-19674: -- User 'mallman' has created a pull request for this issue:

[jira] [Updated] (SPARK-20086) issue with pyspark 2.1.0 window function

2017-03-24 Thread mandar uapdhye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mandar uapdhye updated SPARK-20086: --- Description: original post at [stackoverflow |

[jira] [Updated] (SPARK-20086) issue with pyspark 2.1.0 window function

2017-03-24 Thread mandar uapdhye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mandar uapdhye updated SPARK-20086: --- Description: original post at [stackoverflow |

[jira] [Updated] (SPARK-20086) issue with pyspark 2.1.0 window function

2017-03-24 Thread mandar uapdhye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mandar uapdhye updated SPARK-20086: --- Description: original post at [stackoverflow |

[jira] [Updated] (SPARK-20086) issue with pyspark 2.1.0 window function

2017-03-24 Thread mandar uapdhye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mandar uapdhye updated SPARK-20086: --- Description: original post at [stackoverflow |

[jira] [Updated] (SPARK-20086) issue with pyspark 2.1.0 window function

2017-03-24 Thread mandar uapdhye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mandar uapdhye updated SPARK-20086: --- Description: original post at [stackoverflow |

[jira] [Comment Edited] (SPARK-12216) Spark failed to delete temp directory

2017-03-24 Thread Jouni H (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940987#comment-15940987 ] Jouni H edited comment on SPARK-12216 at 3/24/17 9:37 PM: -- I was able to

[jira] [Updated] (SPARK-20086) issue with pyspark 2.1.0 window function

2017-03-24 Thread mandar uapdhye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mandar uapdhye updated SPARK-20086: --- Description: original post at [stackoverflow |

[jira] [Updated] (SPARK-20086) issue with pyspark 2.1.0 window function

2017-03-24 Thread mandar uapdhye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mandar uapdhye updated SPARK-20086: --- Description: original post at [stackoverflow |

[jira] [Created] (SPARK-20086) issue with pyspark 2.1.0 window function

2017-03-24 Thread mandar uapdhye (JIRA)
mandar uapdhye created SPARK-20086: -- Summary: issue with pyspark 2.1.0 window function Key: SPARK-20086 URL: https://issues.apache.org/jira/browse/SPARK-20086 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-20075) Support classifier, packaging in Maven coordinates

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20075: Assignee: Sean Owen (was: Apache Spark) > Support classifier, packaging in Maven

[jira] [Commented] (SPARK-20075) Support classifier, packaging in Maven coordinates

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941155#comment-15941155 ] Apache Spark commented on SPARK-20075: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20075) Support classifier, packaging in Maven coordinates

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20075: Assignee: Apache Spark (was: Sean Owen) > Support classifier, packaging in Maven

[jira] [Commented] (SPARK-18484) case class datasets - ability to specify decimal precision and scale

2017-03-24 Thread Leif Warner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941138#comment-15941138 ] Leif Warner commented on SPARK-18484: - In the example on the ticket, the precision on the input

[jira] [Comment Edited] (SPARK-12216) Spark failed to delete temp directory

2017-03-24 Thread Jouni H (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940987#comment-15940987 ] Jouni H edited comment on SPARK-12216 at 3/24/17 8:59 PM: -- I was able to

[jira] [Commented] (SPARK-19408) cardinality estimation involving two columns of the same table

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941117#comment-15941117 ] Apache Spark commented on SPARK-19408: -- User 'ron8hu' has created a pull request for this issue:

[jira] [Commented] (SPARK-17137) Add compressed support for multinomial logistic regression coefficients

2017-03-24 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941116#comment-15941116 ] DB Tsai commented on SPARK-17137: - [~sethah] We can do the performance test during the QA time. Thanks.

[jira] [Assigned] (SPARK-19408) cardinality estimation involving two columns of the same table

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19408: Assignee: Apache Spark > cardinality estimation involving two columns of the same table >

[jira] [Assigned] (SPARK-17137) Add compressed support for multinomial logistic regression coefficients

2017-03-24 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-17137: --- Assignee: Seth Hendrickson > Add compressed support for multinomial logistic regression

[jira] [Assigned] (SPARK-19408) cardinality estimation involving two columns of the same table

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19408: Assignee: (was: Apache Spark) > cardinality estimation involving two columns of the

[jira] [Commented] (SPARK-17137) Add compressed support for multinomial logistic regression coefficients

2017-03-24 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941114#comment-15941114 ] Seth Hendrickson commented on SPARK-17137: -- I can make a PR for using this inside the MLOR code,

[jira] [Commented] (SPARK-17137) Add compressed support for multinomial logistic regression coefficients

2017-03-24 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941106#comment-15941106 ] DB Tsai commented on SPARK-17137: - Ping! SPARK-17471 is merged. Anyone interested in making this by 2.2.0

[jira] [Commented] (SPARK-19476) Running threads in Spark DataFrame foreachPartition() causes NullPointerException

2017-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941104#comment-15941104 ] Sean Owen commented on SPARK-19476: --- I don't think in general you're expected to be able to do this

[jira] [Resolved] (SPARK-17471) Add compressed method for Matrix class

2017-03-24 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-17471. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 15628

[jira] [Commented] (SPARK-19476) Running threads in Spark DataFrame foreachPartition() causes NullPointerException

2017-03-24 Thread Lucy Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941079#comment-15941079 ] Lucy Yu commented on SPARK-19476: - I believe we've seen a similar issue raised here:

[jira] [Assigned] (SPARK-20085) Configurable mesos labels for executors

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20085: Assignee: (was: Apache Spark) > Configurable mesos labels for executors >

[jira] [Assigned] (SPARK-20085) Configurable mesos labels for executors

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20085: Assignee: Apache Spark > Configurable mesos labels for executors >

[jira] [Commented] (SPARK-20085) Configurable mesos labels for executors

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941065#comment-15941065 ] Apache Spark commented on SPARK-20085: -- User 'kalvinnchau' has created a pull request for this

[jira] [Created] (SPARK-20085) Configurable mesos labels for executors

2017-03-24 Thread Kalvin Chau (JIRA)
Kalvin Chau created SPARK-20085: --- Summary: Configurable mesos labels for executors Key: SPARK-20085 URL: https://issues.apache.org/jira/browse/SPARK-20085 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-19911) Add builder interface for Kinesis DStreams

2017-03-24 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz resolved SPARK-19911. - Resolution: Fixed Assignee: Adam Budde Fix Version/s: 2.2.0 Target

[jira] [Created] (SPARK-20084) Remove internal.metrics.updatedBlockStatuses accumulator from history files

2017-03-24 Thread Ryan Blue (JIRA)
Ryan Blue created SPARK-20084: - Summary: Remove internal.metrics.updatedBlockStatuses accumulator from history files Key: SPARK-20084 URL: https://issues.apache.org/jira/browse/SPARK-20084 Project: Spark

[jira] [Assigned] (SPARK-20084) Remove internal.metrics.updatedBlockStatuses accumulator from history files

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20084: Assignee: Apache Spark > Remove internal.metrics.updatedBlockStatuses accumulator from

[jira] [Commented] (SPARK-20084) Remove internal.metrics.updatedBlockStatuses accumulator from history files

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941012#comment-15941012 ] Apache Spark commented on SPARK-20084: -- User 'rdblue' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20084) Remove internal.metrics.updatedBlockStatuses accumulator from history files

2017-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20084: Assignee: (was: Apache Spark) > Remove internal.metrics.updatedBlockStatuses

[jira] [Commented] (SPARK-12216) Spark failed to delete temp directory

2017-03-24 Thread Jouni H (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940987#comment-15940987 ] Jouni H commented on SPARK-12216: - I was able to reproduce this bug on Windows with the latest spark

[jira] [Commented] (SPARK-20082) Incremental update of LDA model, by adding initialModel as start point

2017-03-24 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940960#comment-15940960 ] yuhao yang commented on SPARK-20082: Yes, that's one of the things that we should improve for LDA. If

[jira] [Commented] (SPARK-19705) Preferred location supporting HDFS Cache for FileScanRDD

2017-03-24 Thread gagan taneja (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940897#comment-15940897 ] gagan taneja commented on SPARK-19705: -- Sandy Would you be able to help with this bug. I saw you

[jira] [Commented] (SPARK-12664) Expose raw prediction scores in MultilayerPerceptronClassificationModel

2017-03-24 Thread Ian Maloney (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940796#comment-15940796 ] Ian Maloney commented on SPARK-12664: - Does anyone know which versions this will make it into? Also,

[jira] [Created] (SPARK-20083) Change matrix toArray to not create a new array when matrix is already column major

2017-03-24 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-20083: Summary: Change matrix toArray to not create a new array when matrix is already column major Key: SPARK-20083 URL: https://issues.apache.org/jira/browse/SPARK-20083

[jira] [Updated] (SPARK-20082) Incremental update of LDA model, by adding initialModel as start point

2017-03-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-20082: -- Component/s: (was: MLlib) ML > Incremental update of LDA model,

[jira] [Updated] (SPARK-20082) Incremental update of LDA model, by adding initialModel as start point

2017-03-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-20082: -- Issue Type: New Feature (was: Wish) > Incremental update of LDA model, by adding

[jira] [Comment Edited] (SPARK-17557) SQL query on parquet table java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary

2017-03-24 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-17557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940672#comment-15940672 ] Stéphane Collot edited comment on SPARK-17557 at 3/24/17 4:37 PM: -- Hi,

[jira] [Commented] (SPARK-17557) SQL query on parquet table java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary

2017-03-24 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-17557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940672#comment-15940672 ] Stéphane Collot commented on SPARK-17557: - Hi, I got a similar issue with Spark 2.0.0 and it

[jira] [Commented] (SPARK-16745) Spark job completed however have to wait for 13 mins (data size is small)

2017-03-24 Thread Don Drake (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940654#comment-15940654 ] Don Drake commented on SPARK-16745: --- I just came across the same exception running Spark 2.1.0 on my

[jira] [Commented] (SPARK-7200) Tungsten test suites should fail if memory leak is detected

2017-03-24 Thread Jose Soltren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940604#comment-15940604 ] Jose Soltren commented on SPARK-7200: - [~joshrosen] please have a look at my PR. Is this what you had

[jira] [Commented] (SPARK-20073) Unexpected Cartesian product when using eqNullSafe in join with a derived table

2017-03-24 Thread Everett Anderson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940596#comment-15940596 ] Everett Anderson commented on SPARK-20073: -- Doesn't the join criteria convey that the user

[jira] [Assigned] (SPARK-15040) PySpark impl for ml.feature.Imputer

2017-03-24 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath reassigned SPARK-15040: -- Assignee: Nick Pentreath > PySpark impl for ml.feature.Imputer >

[jira] [Resolved] (SPARK-15040) PySpark impl for ml.feature.Imputer

2017-03-24 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-15040. Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17316

[jira] [Updated] (SPARK-20080) Spak streaming application do not throw serialisation exception in foreachRDD

2017-03-24 Thread Nick Hryhoriev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Hryhoriev updated SPARK-20080: --- Description: Step to reproduce: 1)Use SPak Streaming application. 2) inside foreachRDD of

[jira] [Updated] (SPARK-20080) Spak streaming application do not throw serialisation exception in foreachRDD

2017-03-24 Thread Nick Hryhoriev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Hryhoriev updated SPARK-20080: --- Description: Step to reproduce: 1)Use SPak Streaming application. 2) inside foreachRDD of

[jira] [Updated] (SPARK-20080) Spak streaming application do not throw serialisation exception in foreachRDD

2017-03-24 Thread Nick Hryhoriev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Hryhoriev updated SPARK-20080: --- Description: Step to reproduce: 1)Use SPak Streaming application. 2) inside foreachRDD of

[jira] [Updated] (SPARK-20080) Spak streaming application do not throw serialisation exception in foreachRDD

2017-03-24 Thread Nick Hryhoriev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Hryhoriev updated SPARK-20080: --- Description: Step to reproduce: 1)Use SPak Streaming application. 2) inside foreachRDD of

[jira] [Created] (SPARK-20082) Incremental update of LDA model, by adding initialModel as start point

2017-03-24 Thread Mathieu D (JIRA)
Mathieu D created SPARK-20082: - Summary: Incremental update of LDA model, by adding initialModel as start point Key: SPARK-20082 URL: https://issues.apache.org/jira/browse/SPARK-20082 Project: Spark

[jira] [Resolved] (SPARK-8586) SQL add jar command does not work well with Scala REPL

2017-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-8586. -- Resolution: Duplicate I think this and several JIRAs are just the same issue: add jar vs --jars and

[jira] [Resolved] (SPARK-20019) spark can not load alluxio fileSystem after adding jar

2017-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20019. --- Resolution: Duplicate > spark can not load alluxio fileSystem after adding jar >

[jira] [Commented] (SPARK-8586) SQL add jar command does not work well with Scala REPL

2017-03-24 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940185#comment-15940185 ] Takeshi Yamamuro commented on SPARK-8586: - This issue still happens in v2.1.0 and the master.

[jira] [Updated] (SPARK-8586) SQL add jar command does not work well with Scala REPL

2017-03-24 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-8586: Affects Version/s: 2.1.0 > SQL add jar command does not work well with Scala REPL >

[jira] [Created] (SPARK-20081) RandomForestClassifier doesn't seem to support more than 100 labels

2017-03-24 Thread Christian Reiniger (JIRA)
Christian Reiniger created SPARK-20081: -- Summary: RandomForestClassifier doesn't seem to support more than 100 labels Key: SPARK-20081 URL: https://issues.apache.org/jira/browse/SPARK-20081

[jira] [Commented] (SPARK-1548) Add Partial Random Forest algorithm to MLlib

2017-03-24 Thread Mohamed Baddar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940130#comment-15940130 ] Mohamed Baddar commented on SPARK-1548: --- [~manishamde] [~sowen] [~josephkb] I have small experience

[jira] [Updated] (SPARK-20080) Spak streaming application do not throw serialisation exception in foreachRDD

2017-03-24 Thread Nick Hryhoriev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Hryhoriev updated SPARK-20080: --- Description: Step to reproduce: 1)Use SPak Streaming application. 2) inside foreachRDD of

[jira] [Commented] (SPARK-20080) Spak streaming application do not throw serialisation exception in foreachRDD

2017-03-24 Thread Nick Hryhoriev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940113#comment-15940113 ] Nick Hryhoriev commented on SPARK-20080: It's clear issue. Even with a link to code. You have

[jira] [Resolved] (SPARK-20080) Spak streaming application do not throw serialisation exception in foreachRDD

2017-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20080. --- Resolution: Invalid > Spak streaming application do not throw serialisation exception in foreachRDD

[jira] [Reopened] (SPARK-20080) Spak streaming application do not throw serialisation exception in foreachRDD

2017-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-20080: --- > Spak streaming application do not throw serialisation exception in foreachRDD >

[jira] [Resolved] (SPARK-20080) Spak streaming application do not throw serialisation exception in foreachRDD

2017-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20080. --- Resolution: Fixed [~hryhoriev.nick] I can't understand what you're reporting here, and so it's not

[jira] [Commented] (SPARK-13644) Add the source file name and line into Logger when an exception occurs in the generated code

2017-03-24 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940078#comment-15940078 ] Takeshi Yamamuro commented on SPARK-13644: -- Could we close this? I think this issue has already

[jira] [Comment Edited] (SPARK-13644) Add the source file name and line into Logger when an exception occurs in the generated code

2017-03-24 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940078#comment-15940078 ] Takeshi Yamamuro edited comment on SPARK-13644 at 3/24/17 9:51 AM: ---

[jira] [Commented] (SPARK-10719) SQLImplicits.rddToDataFrameHolder is not thread safe when using Scala 2.10

2017-03-24 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940076#comment-15940076 ] Takeshi Yamamuro commented on SPARK-10719: -- I added a link to SPARK-19810 and we could close

[jira] [Updated] (SPARK-20080) Spak streaming application do not throw serialisation exception in foreachRDD

2017-03-24 Thread Nick Hryhoriev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Hryhoriev updated SPARK-20080: --- Summary: Spak streaming application do not throw serialisation exception in foreachRDD

  1   2   >