[jira] [Commented] (SPARK-11332) WeightedLeastSquares should use ml features generic Instance class instead of private

2015-10-27 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977868#comment-14977868 ] DB Tsai commented on SPARK-11332: - [~srowen] Do you know how to assign to new users in JI

[jira] [Updated] (SPARK-11332) WeightedLeastSquares should use ml features generic Instance class instead of private

2015-10-27 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-11332: Assignee: (was: DB Tsai) > WeightedLeastSquares should use ml features generic Instance class instead o

[jira] [Assigned] (SPARK-11332) WeightedLeastSquares should use ml features generic Instance class instead of private

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11332: Assignee: Apache Spark (was: DB Tsai) > WeightedLeastSquares should use ml features gener

[jira] [Assigned] (SPARK-11332) WeightedLeastSquares should use ml features generic Instance class instead of private

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11332: Assignee: DB Tsai (was: Apache Spark) > WeightedLeastSquares should use ml features gener

[jira] [Commented] (SPARK-11332) WeightedLeastSquares should use ml features generic Instance class instead of private

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977864#comment-14977864 ] Apache Spark commented on SPARK-11332: -- User 'nakul02' has created a pull request fo

[jira] [Commented] (SPARK-11313) Implement cogroup

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977846#comment-14977846 ] Apache Spark commented on SPARK-11313: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-11364) HadoopFsRelation doesn't reload the hadoop configuration for each execution

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11364: Assignee: Apache Spark > HadoopFsRelation doesn't reload the hadoop configuration for each

[jira] [Commented] (SPARK-11364) HadoopFsRelation doesn't reload the hadoop configuration for each execution

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977847#comment-14977847 ] Apache Spark commented on SPARK-11364: -- User 'chenghao-intel' has created a pull req

[jira] [Assigned] (SPARK-11364) HadoopFsRelation doesn't reload the hadoop configuration for each execution

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11364: Assignee: (was: Apache Spark) > HadoopFsRelation doesn't reload the hadoop configurati

[jira] [Created] (SPARK-11364) HadoopFsRelation doesn't reload the hadoop configuration for each execution

2015-10-27 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-11364: - Summary: HadoopFsRelation doesn't reload the hadoop configuration for each execution Key: SPARK-11364 URL: https://issues.apache.org/jira/browse/SPARK-11364 Project: Spark

[jira] [Resolved] (SPARK-11302) Multivariate Gaussian Model with Covariance matrix returns incorrect answer in some cases

2015-10-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-11302. --- Resolution: Fixed Fix Version/s: 1.5.2 1.3.2 1.4.

[jira] [Updated] (SPARK-11302) Multivariate Gaussian Model with Covariance matrix returns incorrect answer in some cases

2015-10-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11302: -- Priority: Critical (was: Minor) > Multivariate Gaussian Model with Covariance matrix returns

[jira] [Updated] (SPARK-11302) Multivariate Gaussian Model with Covariance matrix returns incorrect answer in some cases

2015-10-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11302: -- Assignee: Sean Owen Affects Version/s: 1.6.0 1.3.1

[jira] [Assigned] (SPARK-11358) Deprecate `runs` in k-means

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11358: Assignee: Apache Spark (was: Xiangrui Meng) > Deprecate `runs` in k-means > -

[jira] [Commented] (SPARK-11358) Deprecate `runs` in k-means

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977818#comment-14977818 ] Apache Spark commented on SPARK-11358: -- User 'mengxr' has created a pull request for

[jira] [Assigned] (SPARK-11358) Deprecate `runs` in k-means

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11358: Assignee: Xiangrui Meng (was: Apache Spark) > Deprecate `runs` in k-means > -

[jira] [Commented] (SPARK-11337) Make example code in user guide testable

2015-10-27 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977811#comment-14977811 ] Xusen Yin commented on SPARK-11337: --- I want to add new sub-tasks. How to assign the wor

[jira] [Commented] (SPARK-11354) Expose custom log4j to executor page in Spark standalone cluster

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977806#comment-14977806 ] Apache Spark commented on SPARK-11354: -- User 'yongjiaw' has created a pull request f

[jira] [Updated] (SPARK-10324) MLlib 1.6 Roadmap

2015-10-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10324: -- Description: Following SPARK-8445, we created this master list for MLlib features we plan to h

[jira] [Updated] (SPARK-10324) MLlib 1.6 Roadmap

2015-10-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10324: -- Description: Following SPARK-8445, we created this master list for MLlib features we plan to h

[jira] [Assigned] (SPARK-11336) Include a link to the source file in generated example code

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11336: Assignee: Apache Spark (was: Xusen Yin) > Include a link to the source file in generated

[jira] [Assigned] (SPARK-11336) Include a link to the source file in generated example code

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11336: Assignee: Xusen Yin (was: Apache Spark) > Include a link to the source file in generated

[jira] [Commented] (SPARK-11336) Include a link to the source file in generated example code

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977789#comment-14977789 ] Apache Spark commented on SPARK-11336: -- User 'yinxusen' has created a pull request f

[jira] [Assigned] (SPARK-11363) LeftSemiJoin should be LeftSemi in SparkStrategies

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11363: Assignee: Apache Spark > LeftSemiJoin should be LeftSemi in SparkStrategies >

[jira] [Assigned] (SPARK-11363) LeftSemiJoin should be LeftSemi in SparkStrategies

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11363: Assignee: (was: Apache Spark) > LeftSemiJoin should be LeftSemi in SparkStrategies > -

[jira] [Commented] (SPARK-11363) LeftSemiJoin should be LeftSemi in SparkStrategies

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977751#comment-14977751 ] Apache Spark commented on SPARK-11363: -- User 'viirya' has created a pull request for

[jira] [Created] (SPARK-11363) LeftSemiJoin should be LeftSemi in SparkStrategies

2015-10-27 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-11363: --- Summary: LeftSemiJoin should be LeftSemi in SparkStrategies Key: SPARK-11363 URL: https://issues.apache.org/jira/browse/SPARK-11363 Project: Spark Issu

[jira] [Assigned] (SPARK-10827) AppClient should not use `askWithReply` in `receiveAndReply` directly

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10827: Assignee: (was: Apache Spark) > AppClient should not use `askWithReply` in `receiveAnd

[jira] [Commented] (SPARK-10827) AppClient should not use `askWithReply` in `receiveAndReply` directly

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977744#comment-14977744 ] Apache Spark commented on SPARK-10827: -- User 'BryanCutler' has created a pull reques

[jira] [Assigned] (SPARK-10827) AppClient should not use `askWithReply` in `receiveAndReply` directly

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10827: Assignee: Apache Spark > AppClient should not use `askWithReply` in `receiveAndReply` dire

[jira] [Updated] (SPARK-10953) Benchmark declarative/codegen vs. imperative code for univariate statistics

2015-10-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10953: -- Summary: Benchmark declarative/codegen vs. imperative code for univariate statistics (was: Ben

[jira] [Assigned] (SPARK-11362) Use Spark BitSet in BroadcastNestedLoopJoin

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11362: Assignee: (was: Apache Spark) > Use Spark BitSet in BroadcastNestedLoopJoin >

[jira] [Assigned] (SPARK-11362) Use Spark BitSet in BroadcastNestedLoopJoin

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11362: Assignee: Apache Spark > Use Spark BitSet in BroadcastNestedLoopJoin > ---

[jira] [Commented] (SPARK-11362) Use Spark BitSet in BroadcastNestedLoopJoin

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977707#comment-14977707 ] Apache Spark commented on SPARK-11362: -- User 'viirya' has created a pull request for

[jira] [Created] (SPARK-11362) Use Spark BitSet in BroadcastNestedLoopJoin

2015-10-27 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-11362: --- Summary: Use Spark BitSet in BroadcastNestedLoopJoin Key: SPARK-11362 URL: https://issues.apache.org/jira/browse/SPARK-11362 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when two tables do cross join

2015-10-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-10484. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8652 [https://github.com/a

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when two tables do cross join

2015-10-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10484: - Assignee: Cheng Hao > [Spark SQL] Come across lost task(timeout) or GC OOM error when two tables > do c

[jira] [Updated] (SPARK-11361) Show scopes of RDD operations inside DStream.foreachRDD and DStream.transform in DAG viz

2015-10-27 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-11361: -- Description: Currently, when a DStream sets the scope for RDD generated by it, that scope is n

[jira] [Assigned] (SPARK-11361) Show scopes of RDD operations inside DStream.foreachRDD and DStream.transform in DAG viz

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11361: Assignee: Apache Spark (was: Tathagata Das) > Show scopes of RDD operations inside DStrea

[jira] [Assigned] (SPARK-11361) Show scopes of RDD operations inside DStream.foreachRDD and DStream.transform in DAG viz

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11361: Assignee: Tathagata Das (was: Apache Spark) > Show scopes of RDD operations inside DStrea

[jira] [Commented] (SPARK-11361) Show scopes of RDD operations inside DStream.foreachRDD and DStream.transform in DAG viz

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977606#comment-14977606 ] Apache Spark commented on SPARK-11361: -- User 'tdas' has created a pull request for t

[jira] [Created] (SPARK-11361) Show scopes of RDD operations inside DStream.foreachRDD and DStream.transform in DAG viz

2015-10-27 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-11361: - Summary: Show scopes of RDD operations inside DStream.foreachRDD and DStream.transform in DAG viz Key: SPARK-11361 URL: https://issues.apache.org/jira/browse/SPARK-11361

[jira] [Comment Edited] (SPARK-11330) Filter operation on StringType after groupBy PERSISTED brings no results

2015-10-27 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977600#comment-14977600 ] Cheng Hao edited comment on SPARK-11330 at 10/28/15 2:28 AM: -

[jira] [Commented] (SPARK-11330) Filter operation on StringType after groupBy PERSISTED brings no results

2015-10-27 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977600#comment-14977600 ] Cheng Hao commented on SPARK-11330: --- Hi, [~saif.a.ellafi], I've tried the code like bel

[jira] [Commented] (SPARK-4836) Web UI should display separate information for all stage attempts

2015-10-27 Thread Christian Kadner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977551#comment-14977551 ] Christian Kadner commented on SPARK-4836: - Hi [~joshrosen], is this still a proble

[jira] [Assigned] (SPARK-11360) Loss of nullability when writing parquet files

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11360: Assignee: (was: Apache Spark) > Loss of nullability when writing parquet files > -

[jira] [Assigned] (SPARK-11360) Loss of nullability when writing parquet files

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11360: Assignee: Apache Spark > Loss of nullability when writing parquet files >

[jira] [Commented] (SPARK-11360) Loss of nullability when writing parquet files

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977548#comment-14977548 ] Apache Spark commented on SPARK-11360: -- User 'gatorsmile' has created a pull request

[jira] [Created] (SPARK-11360) Loss of nullability when writing parquet files

2015-10-27 Thread Xiao Li (JIRA)
Xiao Li created SPARK-11360: --- Summary: Loss of nullability when writing parquet files Key: SPARK-11360 URL: https://issues.apache.org/jira/browse/SPARK-11360 Project: Spark Issue Type: Documentatio

[jira] [Issue Comment Deleted] (SPARK-11200) NettyRpcEnv endless message "cannot send ${message} because RpcEnv is closed"

2015-10-27 Thread hujiayin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiayin updated SPARK-11200: - Comment: was deleted (was: sparkscore found it happened since commit number cf2e0ae7 and resolved today.

[jira] [Commented] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

2015-10-27 Thread swetha k (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977427#comment-14977427 ] swetha k commented on SPARK-3655: - [~koert] Does this use a custom partitioner to make su

[jira] [Updated] (SPARK-11178) Improve naming around task failures in scheduler code

2015-10-27 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-11178: --- Affects Version/s: (was: 1.5.1) > Improve naming around task failures in scheduler code >

[jira] [Resolved] (SPARK-11178) Improve naming around task failures in scheduler code

2015-10-27 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-11178. Resolution: Fixed Fix Version/s: 1.6.0 > Improve naming around task failures in sche

[jira] [Created] (SPARK-11359) Kinesis receiver does not checkpoint to DynamoDB if there is no new data.

2015-10-27 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-11359: - Summary: Kinesis receiver does not checkpoint to DynamoDB if there is no new data. Key: SPARK-11359 URL: https://issues.apache.org/jira/browse/SPARK-11359 Project:

[jira] [Resolved] (SPARK-11212) Make RDD's preferred locations support the executor location and fix ReceiverTracker for multiple executors in a host

2015-10-27 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-11212. --- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 1.6.0 > Make RDD's pr

[jira] [Resolved] (SPARK-11324) Flag to close Write Ahead Log after writing

2015-10-27 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-11324. --- Resolution: Fixed Assignee: Burak Yavuz Fix Version/s: 1.6.0 > Flag to close

[jira] [Assigned] (SPARK-10658) Could pyspark provide addJars() as scala spark API?

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10658: Assignee: Apache Spark > Could pyspark provide addJars() as scala spark API? > --

[jira] [Updated] (SPARK-11358) Deprecate `runs` in k-means

2015-10-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11358: -- Component/s: (was: MLilb) MLlib > Deprecate `runs` in k-means > --

[jira] [Assigned] (SPARK-10658) Could pyspark provide addJars() as scala spark API?

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10658: Assignee: (was: Apache Spark) > Could pyspark provide addJars() as scala spark API? >

[jira] [Commented] (SPARK-10658) Could pyspark provide addJars() as scala spark API?

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977327#comment-14977327 ] Apache Spark commented on SPARK-10658: -- User 'holdenk' has created a pull request fo

[jira] [Created] (SPARK-11358) Deprecate `runs` in k-means

2015-10-27 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-11358: - Summary: Deprecate `runs` in k-means Key: SPARK-11358 URL: https://issues.apache.org/jira/browse/SPARK-11358 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-11357) Building Spark with maven doesn't add to spark-network-yarn_2.10-XXX.jar all necessary classes.

2015-10-27 Thread Nina Pakhomova (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977267#comment-14977267 ] Nina Pakhomova commented on SPARK-11357: [~srowen], I used wrong jar indeed. Sorr

[jira] [Closed] (SPARK-11357) Building Spark with maven doesn't add to spark-network-yarn_2.10-XXX.jar all necessary classes.

2015-10-27 Thread Nina Pakhomova (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nina Pakhomova closed SPARK-11357. -- Resolution: Invalid > Building Spark with maven doesn't add to spark-network-yarn_2.10-XXX.jar

[jira] [Commented] (SPARK-10592) deprecate weights and use coefficients instead in ML models

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977255#comment-14977255 ] Apache Spark commented on SPARK-10592: -- User 'vectorijk' has created a pull request

[jira] [Assigned] (SPARK-10592) deprecate weights and use coefficients instead in ML models

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10592: Assignee: Apache Spark > deprecate weights and use coefficients instead in ML models > ---

[jira] [Assigned] (SPARK-10592) deprecate weights and use coefficients instead in ML models

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10592: Assignee: (was: Apache Spark) > deprecate weights and use coefficients instead in ML m

[jira] [Updated] (SPARK-11357) Building Spark with maven doesn't add to spark-network-yarn_2.10-XXX.jar all necessary classes.

2015-10-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11357: -- Target Version/s: (was: 1.6.0) [~npakhomova] please read https://cwiki.apache.org/confluence/display

[jira] [Commented] (SPARK-11357) Building Spark with maven doesn't add to spark-network-yarn_2.10-XXX.jar all necessary classes.

2015-10-27 Thread Nina Pakhomova (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977162#comment-14977162 ] Nina Pakhomova commented on SPARK-11357: And no spark--yarn-shuffle.jar is produc

[jira] [Created] (SPARK-11357) Building Spark with maven doesn't add to spark-network-yarn_2.10-XXX.jar all necessary classes.

2015-10-27 Thread Nina Pakhomova (JIRA)
Nina Pakhomova created SPARK-11357: -- Summary: Building Spark with maven doesn't add to spark-network-yarn_2.10-XXX.jar all necessary classes. Key: SPARK-11357 URL: https://issues.apache.org/jira/browse/SPARK-113

[jira] [Commented] (SPARK-10181) HiveContext is not used with keytab principal but with user principal/unix username

2015-10-27 Thread Yu Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977151#comment-14977151 ] Yu Gao commented on SPARK-10181: No problem. Please try out the changes in the second pul

[jira] [Updated] (SPARK-10024) Python API RF and GBT related params clear up

2015-10-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10024: -- Target Version/s: 1.6.0 > Python API RF and GBT related params clear up > -

[jira] [Commented] (SPARK-10658) Could pyspark provide addJars() as scala spark API?

2015-10-27 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977147#comment-14977147 ] holdenk commented on SPARK-10658: - So this turns out be a bit complicated because of a va

[jira] [Resolved] (SPARK-10024) Python API RF and GBT related params clear up

2015-10-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-10024. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9233 [https://gi

[jira] [Updated] (SPARK-10024) Python API RF and GBT related params clear up

2015-10-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10024: -- Assignee: Kai Jiang > Python API RF and GBT related params clear up > -

[jira] [Commented] (SPARK-9492) LogisticRegression in R should provide model statistics

2015-10-27 Thread Bilind Hajer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977134#comment-14977134 ] Bilind Hajer commented on SPARK-9492: - So is there no other way to get coefficients fr

[jira] [Commented] (SPARK-11302) Multivariate Gaussian Model with Covariance matrix returns incorrect answer in some cases

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977133#comment-14977133 ] Apache Spark commented on SPARK-11302: -- User 'srowen' has created a pull request for

[jira] [Resolved] (SPARK-11347) Support for joining two datasets, returning a tuple of objects

2015-10-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-11347. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9300 [https://github.com/a

[jira] [Commented] (SPARK-10707) Set operation output columns may have incorrect nullability

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977084#comment-14977084 ] Apache Spark commented on SPARK-10707: -- User 'mbautin' has created a pull request fo

[jira] [Commented] (SPARK-10309) Some tasks failed with Unable to acquire memory

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977029#comment-14977029 ] Apache Spark commented on SPARK-10309: -- User 'davies' has created a pull request for

[jira] [Assigned] (SPARK-10309) Some tasks failed with Unable to acquire memory

2015-10-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-10309: -- Assignee: Davies Liu > Some tasks failed with Unable to acquire memory > -

[jira] [Commented] (SPARK-11239) PMML export for ML linear regression

2015-10-27 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977007#comment-14977007 ] holdenk commented on SPARK-11239: - Pretty much, I've got a draft PR out for it but I'm wa

[jira] [Closed] (SPARK-9887) After recent hive patches PySpark fails with IllegalArgumentException: Wrong FS: hdfs:

2015-10-27 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin closed SPARK-9887. - Was duplicate. > After recent hive patches PySpark fails with IllegalArgumentException: Wrong > FS:

[jira] [Commented] (SPARK-10181) HiveContext is not used with keytab principal but with user principal/unix username

2015-10-27 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976990#comment-14976990 ] Bolke de Bruin commented on SPARK-10181: I will apply it too our install if it wo

[jira] [Comment Edited] (SPARK-10181) HiveContext is not used with keytab principal but with user principal/unix username

2015-10-27 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976990#comment-14976990 ] Bolke de Bruin edited comment on SPARK-10181 at 10/27/15 7:16 PM: -

[jira] [Updated] (SPARK-11356) Option to refresh information about parquet partitions

2015-10-27 Thread JIRA
).parquet("some_location") > {code} > App 2 - example > {code} > sqlContext.read.parquet("some_location").registerTempTable("t") > sqlContext.sql("select * from t where day = 20151027").count() > {code} -- This message was sent by Atlassian JIRA (v

[jira] [Updated] (SPARK-11356) Option to refresh information about parquet partitions

2015-10-27 Thread JIRA
;some_location") > {code} > App 2 - example > {code} > sqlContext.read.parquet("some_location").registerTempTable("t") > sqlContext.sql("select * from t where day = 20151027").count() > {code} -- This message was sent by Atlassian JIRA (v

[jira] [Updated] (SPARK-11356) Option to refresh information about parquet partitions

2015-10-27 Thread JIRA
appen ? App 1 - periodically (eg. every hour) {code} df.write.partitionBy("day").mode("append").parquet("some_location") {code} App 2 - example {code} sqlContext.read.parquet("some_location").registerTempTable("t") sqlContext.sql("select *

[jira] [Updated] (SPARK-11356) Option to refresh information about parquet partitions

2015-10-27 Thread JIRA
quot;).mode("append").parquet("some_location") > {code} > App 2 > {code} > sqlContext.read.parquet("some_location").registerTempTable("t") > sqlContext.sql("select * from t where day = 20151027").count() > {code} -- This message wa

[jira] [Created] (SPARK-11356) Option to refresh information about partitions

2015-10-27 Thread JIRA
p 2 {code} sqlContext.read.parquet("some_location").registerTempTable("t") sqlContext.sql("select * from t where day = 20151027").count() {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)

[jira] [Resolved] (SPARK-11270) Add improved equality testing for TopicAndPartition from the Kafka Streaming API

2015-10-27 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-11270. --- Resolution: Fixed Fix Version/s: 1.6.0 1.5.3 > Add improved equalit

[jira] [Assigned] (SPARK-8546) PMML export for Naive Bayes

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8546: --- Assignee: Apache Spark (was: Xusen Yin) > PMML export for Naive Bayes >

[jira] [Assigned] (SPARK-8546) PMML export for Naive Bayes

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8546: --- Assignee: Xusen Yin (was: Apache Spark) > PMML export for Naive Bayes >

[jira] [Commented] (SPARK-8546) PMML export for Naive Bayes

2015-10-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976879#comment-14976879 ] Apache Spark commented on SPARK-8546: - User 'yinxusen' has created a pull request for

[jira] [Resolved] (SPARK-6488) Support addition/multiplication in PySpark's BlockMatrix

2015-10-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6488. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9139 [https://githu

[jira] [Commented] (SPARK-11346) Spark EventLog for completed applications

2015-10-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976854#comment-14976854 ] Marcelo Vanzin commented on SPARK-11346: It sounds to me like you're running the

[jira] [Commented] (SPARK-11255) R Test build should run on R 3.1.1

2015-10-27 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976850#comment-14976850 ] shane knapp commented on SPARK-11255: - this all sounds good. to make things easier,

[jira] [Resolved] (SPARK-11355) Spark 1.5.1 compile failure with scala 2.11

2015-10-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11355. --- Resolution: Cannot Reproduce I can't reproduce this, and it looks like a failure from within the comp

[jira] [Updated] (SPARK-11354) Expose custom log4j to executor page in Spark standalone cluster

2015-10-27 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated SPARK-11354: Description: Spark uses log4j, which is very flexible. However, on the executor page in standalone

[jira] [Resolved] (SPARK-11306) Executor JVM loss can lead to a hang in Standalone mode

2015-10-27 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-11306. Resolution: Fixed Fix Version/s: 1.6.0 > Executor JVM loss can lead to a hang in Sta

[jira] [Updated] (SPARK-11354) Expose custom log4j to executor page in Spark standalone cluster

2015-10-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11354: -- Priority: Minor (was: Major) > Expose custom log4j to executor page in Spark standalone cluster > ---

  1   2   3   >