[jira] [Commented] (SPARK-7189) History server will always reload the same file even when no log file is updated

2015-04-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14516852#comment-14516852 ] Sean Owen commented on SPARK-7189: -- Hm, I'd swear we had discussed this already and there

[jira] [Commented] (SPARK-7193) Spark on Mesos may need more tests for spark 1.3.1 release

2015-04-28 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14516801#comment-14516801 ] Littlestar commented on SPARK-7193: --- {noformat} 15/04/28 18:45:53 INFO

[jira] [Comment Edited] (SPARK-7193) Spark on Mesos may need more tests for spark 1.3.1 release

2015-04-28 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14516801#comment-14516801 ] Littlestar edited comment on SPARK-7193 at 4/28/15 10:51 AM: -

[jira] [Comment Edited] (SPARK-7193) Spark on Mesos may need more tests for spark 1.3.1 release

2015-04-28 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14516801#comment-14516801 ] Littlestar edited comment on SPARK-7193 at 4/28/15 10:53 AM: -

[jira] [Commented] (SPARK-7193) Spark on Mesos may need more tests for spark 1.3.1 release

2015-04-28 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14516807#comment-14516807 ] Littlestar commented on SPARK-7193: --- exception on some mesos worknode log. {noformat}

[jira] [Commented] (SPARK-7133) Implement struct, array, and map field accessor using apply in Scala and __getitem__ in Python

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14516815#comment-14516815 ] Apache Spark commented on SPARK-7133: - User 'cloud-fan' has created a pull request for

[jira] [Assigned] (SPARK-7133) Implement struct, array, and map field accessor using apply in Scala and __getitem__ in Python

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7133: --- Assignee: Apache Spark Implement struct, array, and map field accessor using apply in Scala

[jira] [Updated] (SPARK-7161) Provide REST api to download event logs from History Server

2015-04-28 Thread Kostas Sakellis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kostas Sakellis updated SPARK-7161: --- Component/s: (was: Streaming) Spark Core Provide REST api to download

[jira] [Created] (SPARK-7203) Python API for local linear algebra

2015-04-28 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-7203: Summary: Python API for local linear algebra Key: SPARK-7203 URL: https://issues.apache.org/jira/browse/SPARK-7203 Project: Spark Issue Type:

[jira] [Updated] (SPARK-7202) Add SparseMatrixPickler to SerDe

2015-04-28 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7202: - Issue Type: Sub-task (was: New Feature) Parent: SPARK-7203 Add

[jira] [Commented] (SPARK-7202) Add SparseMatrixPickler to SerDe

2015-04-28 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517905#comment-14517905 ] Joseph K. Bradley commented on SPARK-7202: -- @MechCoder I just made an umbrella

[jira] [Comment Edited] (SPARK-7202) Add SparseMatrixPickler to SerDe

2015-04-28 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517905#comment-14517905 ] Joseph K. Bradley edited comment on SPARK-7202 at 4/28/15 8:01 PM:

[jira] [Comment Edited] (SPARK-7178) Improve DataFrame documentation and code samples

2015-04-28 Thread Chris Fregly (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517858#comment-14517858 ] Chris Fregly edited comment on SPARK-7178 at 4/28/15 8:07 PM: --

[jira] [Commented] (SPARK-5182) Partitioning support for tables created by the data source API

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518305#comment-14518305 ] Apache Spark commented on SPARK-5182: - User 'liancheng' has created a pull request for

[jira] [Created] (SPARK-7215) Make repartition and coalesce a part of the query plan

2015-04-28 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-7215: -- Summary: Make repartition and coalesce a part of the query plan Key: SPARK-7215 URL: https://issues.apache.org/jira/browse/SPARK-7215 Project: Spark Issue Type:

[jira] [Created] (SPARK-7217) Add configuration to disable stopping of SparkContext when StreamingContext.stop()

2015-04-28 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-7217: Summary: Add configuration to disable stopping of SparkContext when StreamingContext.stop() Key: SPARK-7217 URL: https://issues.apache.org/jira/browse/SPARK-7217

[jira] [Resolved] (SPARK-7138) Add method to BlockGenerator to add multiple records to BlockGenerator with single callback

2015-04-28 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-7138. -- Resolution: Fixed Fix Version/s: 1.4.0 Add method to BlockGenerator to add multiple

[jira] [Assigned] (SPARK-7216) Show driver details in Mesos cluster UI

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7216: --- Assignee: (was: Apache Spark) Show driver details in Mesos cluster UI

[jira] [Resolved] (SPARK-7193) Spark on Mesos may need more tests for spark 1.3.1 release

2015-04-28 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Littlestar resolved SPARK-7193. --- Resolution: Invalid I think official document missing some notes about Spark on Mesos I worked well

[jira] [Comment Edited] (SPARK-7193) Spark on Mesos may need more tests for spark 1.3.1 release

2015-04-28 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518610#comment-14518610 ] Littlestar edited comment on SPARK-7193 at 4/29/15 2:40 AM: I

[jira] [Resolved] (SPARK-6965) StringIndexer should convert input to Strings

2015-04-28 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-6965. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5753

[jira] [Commented] (SPARK-5556) Latent Dirichlet Allocation (LDA) using Gibbs sampler

2015-04-28 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518602#comment-14518602 ] Guoqiang Li commented on SPARK-5556: I put the latest LDA code in

[jira] [Commented] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

2015-04-28 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518286#comment-14518286 ] Sandy Ryza commented on SPARK-3655: --- My opinion is that a secondary sort operator in

[jira] [Assigned] (SPARK-7215) Make repartition and coalesce a part of the query plan

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7215: --- Assignee: Apache Spark Make repartition and coalesce a part of the query plan

[jira] [Commented] (SPARK-5556) Latent Dirichlet Allocation (LDA) using Gibbs sampler

2015-04-28 Thread Pedro Rodriguez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518378#comment-14518378 ] Pedro Rodriguez commented on SPARK-5556: I will start working on it again then. It

[jira] [Commented] (SPARK-7215) Make repartition and coalesce a part of the query plan

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518379#comment-14518379 ] Apache Spark commented on SPARK-7215: - User 'brkyvz' has created a pull request for

[jira] [Assigned] (SPARK-7215) Make repartition and coalesce a part of the query plan

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7215: --- Assignee: (was: Apache Spark) Make repartition and coalesce a part of the query plan

[jira] [Assigned] (SPARK-7216) Show driver details in Mesos cluster UI

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7216: --- Assignee: Apache Spark Show driver details in Mesos cluster UI

[jira] [Created] (SPARK-7216) Show driver details in Mesos cluster UI

2015-04-28 Thread Timothy Chen (JIRA)
Timothy Chen created SPARK-7216: --- Summary: Show driver details in Mesos cluster UI Key: SPARK-7216 URL: https://issues.apache.org/jira/browse/SPARK-7216 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-7216) Show driver details in Mesos cluster UI

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518447#comment-14518447 ] Apache Spark commented on SPARK-7216: - User 'tnachen' has created a pull request for

[jira] [Commented] (SPARK-5556) Latent Dirichlet Allocation (LDA) using Gibbs sampler

2015-04-28 Thread Pedro Rodriguez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518601#comment-14518601 ] Pedro Rodriguez commented on SPARK-5556: [~gq] is the LDAGibbs line what I

[jira] [Commented] (SPARK-7156) Add randomSplit method to DataFrame

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518248#comment-14518248 ] Apache Spark commented on SPARK-7156: - User 'brkyvz' has created a pull request for

[jira] [Created] (SPARK-7214) Unrolling never evicts blocks when MemoryStore is nearly full

2015-04-28 Thread Charles Reiss (JIRA)
Charles Reiss created SPARK-7214: Summary: Unrolling never evicts blocks when MemoryStore is nearly full Key: SPARK-7214 URL: https://issues.apache.org/jira/browse/SPARK-7214 Project: Spark

[jira] [Commented] (SPARK-5556) Latent Dirichlet Allocation (LDA) using Gibbs sampler

2015-04-28 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518400#comment-14518400 ] Joseph K. Bradley commented on SPARK-5556: -- That plan sounds good. I haven't yet

[jira] [Assigned] (SPARK-4721) Improve first thread to put block failed

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4721: --- Assignee: (was: Apache Spark) Improve first thread to put block failed

[jira] [Assigned] (SPARK-4721) Improve first thread to put block failed

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4721: --- Assignee: Apache Spark Improve first thread to put block failed

[jira] [Assigned] (SPARK-7133) Implement struct, array, and map field accessor using apply in Scala and __getitem__ in Python

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7133: --- Assignee: (was: Apache Spark) Implement struct, array, and map field accessor using

[jira] [Resolved] (SPARK-7168) Update plugin versions in Maven build and centralize versions

2015-04-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7168. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5720

[jira] [Updated] (SPARK-6435) spark-shell --jars option does not add all jars to classpath

2015-04-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6435: - Assignee: Masayoshi TSUZUKI spark-shell --jars option does not add all jars to classpath

[jira] [Resolved] (SPARK-6435) spark-shell --jars option does not add all jars to classpath

2015-04-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6435. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5227

[jira] [Commented] (SPARK-5189) Reorganize EC2 scripts so that nodes can be provisioned independent of Spark master

2015-04-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14516854#comment-14516854 ] Sean Owen commented on SPARK-5189: -- [~jackli066519] You don't need to have this assigned

[jira] [Commented] (SPARK-4414) SparkContext.wholeTextFiles Doesn't work with S3 Buckets

2015-04-28 Thread Peter Marsh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14516971#comment-14516971 ] Peter Marsh commented on SPARK-4414: I managed to get this to work by re-installing

[jira] [Resolved] (SPARK-4721) Improve first thread to put block failed

2015-04-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4721. -- Resolution: Won't Fix Improve first thread to put block failed

[jira] [Resolved] (SPARK-7100) GradientBoostTrees leaks a persisted RDD

2015-04-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7100. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5669

[jira] [Updated] (SPARK-7100) GradientBoostTrees leaks a persisted RDD

2015-04-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7100: - Assignee: Jim Carroll GradientBoostTrees leaks a persisted RDD

[jira] [Commented] (SPARK-6627) Clean up of shuffle code and interfaces

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518402#comment-14518402 ] Apache Spark commented on SPARK-6627: - User 'kayousterhout' has created a pull request

[jira] [Commented] (SPARK-7169) Allow to specify metrics configuration more flexibly

2015-04-28 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518508#comment-14518508 ] Saisai Shao commented on SPARK-7169: Hi [~jlewandowski], regard to your second

[jira] [Created] (SPARK-7218) Create a real iterator with open/close for Spark SQL

2015-04-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-7218: -- Summary: Create a real iterator with open/close for Spark SQL Key: SPARK-7218 URL: https://issues.apache.org/jira/browse/SPARK-7218 Project: Spark Issue Type:

[jira] [Updated] (SPARK-5556) Latent Dirichlet Allocation (LDA) using Gibbs sampler

2015-04-28 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-5556: --- Attachment: LDA_test.xlsx Latent Dirichlet Allocation (LDA) using Gibbs sampler

[jira] [Commented] (SPARK-7189) History server will always reload the same file even when no log file is updated

2015-04-28 Thread Zhang, Liye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517136#comment-14517136 ] Zhang, Liye commented on SPARK-7189: Yes, I think the current solution is a tradeoff,

[jira] [Commented] (SPARK-7189) History server will always reload the same file even when no log file is updated

2015-04-28 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517289#comment-14517289 ] Marcelo Vanzin commented on SPARK-7189: --- Changing the {{=}} causes problems. If you

[jira] [Created] (SPARK-7194) Vectors factors method for sparse vectors should accept the output of zipWithIndex

2015-04-28 Thread Juliet Hougland (JIRA)
Juliet Hougland created SPARK-7194: -- Summary: Vectors factors method for sparse vectors should accept the output of zipWithIndex Key: SPARK-7194 URL: https://issues.apache.org/jira/browse/SPARK-7194

[jira] [Commented] (SPARK-5529) BlockManager heartbeat expiration does not kill executor

2015-04-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517280#comment-14517280 ] Sean Owen commented on SPARK-5529: -- [~arov] CDH always has the latest upstream minor

[jira] [Commented] (SPARK-5529) BlockManager heartbeat expiration does not kill executor

2015-04-28 Thread Alex Rovner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517281#comment-14517281 ] Alex Rovner commented on SPARK-5529: Applied patch to 1.3:

[jira] [Updated] (SPARK-7194) Vectors factors method for sparse vectors should accept the output of zipWithIndex

2015-04-28 Thread Juliet Hougland (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliet Hougland updated SPARK-7194: --- Description: Let's say we have an RDD of Array[Double] where zero values are explictly

[jira] [Created] (SPARK-7195) Can't start spark shell or pyspark in Windows 7

2015-04-28 Thread Mark Smiley (JIRA)
Mark Smiley created SPARK-7195: -- Summary: Can't start spark shell or pyspark in Windows 7 Key: SPARK-7195 URL: https://issues.apache.org/jira/browse/SPARK-7195 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-5529) BlockManager heartbeat expiration does not kill executor

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517285#comment-14517285 ] Apache Spark commented on SPARK-5529: - User 'alexrovner' has created a pull request

[jira] [Commented] (SPARK-5529) BlockManager heartbeat expiration does not kill executor

2015-04-28 Thread Alex Rovner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517257#comment-14517257 ] Alex Rovner commented on SPARK-5529: CDH is usually somewhat slow on picking up the

[jira] [Resolved] (SPARK-6756) Add compress() to Vector

2015-04-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6756. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5756

[jira] [Commented] (SPARK-7220) Check whether moving shared params is a compatible change

2015-04-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518747#comment-14518747 ] Xiangrui Meng commented on SPARK-7220: -- I compiled an example app that calls

[jira] [Assigned] (SPARK-7194) Vectors factors method for sparse vectors should accept the output of zipWithIndex

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7194: --- Assignee: Apache Spark Vectors factors method for sparse vectors should accept the output

[jira] [Created] (SPARK-7220) Check whether moving shared params is a compatible change

2015-04-28 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-7220: Summary: Check whether moving shared params is a compatible change Key: SPARK-7220 URL: https://issues.apache.org/jira/browse/SPARK-7220 Project: Spark

[jira] [Updated] (SPARK-5556) Latent Dirichlet Allocation (LDA) using Gibbs sampler

2015-04-28 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-5556: --- Attachment: spark-summit.pptx Latent Dirichlet Allocation (LDA) using Gibbs sampler

[jira] [Updated] (SPARK-7202) Add SparseMatrixPickler to SerDe

2015-04-28 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Kumar updated SPARK-7202: --- Priority: Major (was: Minor) Add SparseMatrixPickler to SerDe

[jira] [Commented] (SPARK-7189) History server will always reload the same file even when no log file is updated

2015-04-28 Thread Zhang, Liye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518721#comment-14518721 ] Zhang, Liye commented on SPARK-7189: Hi [~vanzin], I think using timestamp is not that

[jira] [Commented] (SPARK-5556) Latent Dirichlet Allocation (LDA) using Gibbs sampler

2015-04-28 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518618#comment-14518618 ] Guoqiang Li commented on SPARK-5556: LDA_Gibbs combines the advantages of AliasLDA,

[jira] [Commented] (SPARK-5556) Latent Dirichlet Allocation (LDA) using Gibbs sampler

2015-04-28 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518621#comment-14518621 ] Guoqiang Li commented on SPARK-5556:

[jira] [Created] (SPARK-7219) HashingTF should output ML attributes

2015-04-28 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-7219: Summary: HashingTF should output ML attributes Key: SPARK-7219 URL: https://issues.apache.org/jira/browse/SPARK-7219 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-7219) HashingTF should output ML attributes

2015-04-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7219: - Priority: Trivial (was: Major) HashingTF should output ML attributes

[jira] [Resolved] (SPARK-7208) Add Matrix, SparseMatrix to __all__ list in linalg.py

2015-04-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-7208. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5759

[jira] [Created] (SPARK-7221) Expose the current processed file name of FileInputDStream to the users

2015-04-28 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-7221: -- Summary: Expose the current processed file name of FileInputDStream to the users Key: SPARK-7221 URL: https://issues.apache.org/jira/browse/SPARK-7221 Project: Spark

[jira] [Updated] (SPARK-7221) Expose the current processed file name of FileInputDStream to the users

2015-04-28 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-7221: --- Issue Type: Wish (was: New Feature) Expose the current processed file name of FileInputDStream to

[jira] [Closed] (SPARK-7220) Check whether moving shared params is a compatible change

2015-04-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-7220. Resolution: Done Fix Version/s: 1.4.0 Check whether moving shared params is a compatible

[jira] [Commented] (SPARK-5189) Reorganize EC2 scripts so that nodes can be provisioned independent of Spark master

2015-04-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517301#comment-14517301 ] Nicholas Chammas commented on SPARK-5189: - Yeah, as Sean said you can just start

[jira] [Resolved] (SPARK-5253) LinearRegression with L1/L2 (elastic net) using OWLQN in new ML package

2015-04-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5253. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4259

[jira] [Created] (SPARK-7198) VectorAssembler should carry ML metadata

2015-04-28 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-7198: Summary: VectorAssembler should carry ML metadata Key: SPARK-7198 URL: https://issues.apache.org/jira/browse/SPARK-7198 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-7195) Can't start spark shell or pyspark in Windows 7

2015-04-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7195. -- Resolution: Duplicate Have a look around JIRA first Can't start spark shell or pyspark in Windows 7

[jira] [Commented] (SPARK-5529) BlockManager heartbeat expiration does not kill executor

2015-04-28 Thread Alex Rovner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517298#comment-14517298 ] Alex Rovner commented on SPARK-5529: Sorry to quickly pulled the trigger... Need to

[jira] [Created] (SPARK-7197) Join with DataFrame Python API not working properly with more than 1 column

2015-04-28 Thread Ali Bajwa (JIRA)
Ali Bajwa created SPARK-7197: Summary: Join with DataFrame Python API not working properly with more than 1 column Key: SPARK-7197 URL: https://issues.apache.org/jira/browse/SPARK-7197 Project: Spark

[jira] [Created] (SPARK-7196) decimal precision lost when loading DataFrame from JDBC

2015-04-28 Thread Ken Geis (JIRA)
Ken Geis created SPARK-7196: --- Summary: decimal precision lost when loading DataFrame from JDBC Key: SPARK-7196 URL: https://issues.apache.org/jira/browse/SPARK-7196 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-7140) Do not scan all values in Vector.hashCode

2015-04-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-7140. -- Resolution: Fixed Fix Version/s: 1.4.0 1.3.2 Issue resolved by pull

[jira] [Comment Edited] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

2015-04-28 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517946#comment-14517946 ] koert kuipers edited comment on SPARK-3655 at 4/28/15 8:18 PM:

[jira] [Comment Edited] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

2015-04-28 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517946#comment-14517946 ] koert kuipers edited comment on SPARK-3655 at 4/28/15 8:19 PM:

[jira] [Assigned] (SPARK-7205) Support local ivy cache in --packages

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7205: --- Assignee: (was: Apache Spark) Support local ivy cache in --packages

[jira] [Commented] (SPARK-7205) Support local ivy cache in --packages

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517988#comment-14517988 ] Apache Spark commented on SPARK-7205: - User 'brkyvz' has created a pull request for

[jira] [Assigned] (SPARK-7205) Support local ivy cache in --packages

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7205: --- Assignee: Apache Spark Support local ivy cache in --packages

[jira] [Created] (SPARK-7204) Call sites in UI are not accurate for DataFrame operations

2015-04-28 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-7204: -- Summary: Call sites in UI are not accurate for DataFrame operations Key: SPARK-7204 URL: https://issues.apache.org/jira/browse/SPARK-7204 Project: Spark

[jira] [Updated] (SPARK-5338) Support cluster mode with Mesos

2015-04-28 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5338: - Affects Version/s: 1.0.0 Support cluster mode with Mesos ---

[jira] [Commented] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

2015-04-28 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517946#comment-14517946 ] koert kuipers commented on SPARK-3655: -- since the last pullreq for this ticket i

[jira] [Closed] (SPARK-5338) Support cluster mode with Mesos

2015-04-28 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-5338. Resolution: Fixed Fix Version/s: 1.4.0 Assignee: Timothy Chen Target Version/s:

[jira] [Commented] (SPARK-6943) Graphically show RDD's included in a stage

2015-04-28 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518084#comment-14518084 ] Andrew Or commented on SPARK-6943: -- Yeah ideally we will have the job graph that

[jira] [Commented] (SPARK-5556) Latent Dirichlet Allocation (LDA) using Gibbs sampler

2015-04-28 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518141#comment-14518141 ] Joseph K. Bradley commented on SPARK-5556: -- Great! I'm not aware of blockers.

[jira] [Issue Comment Deleted] (SPARK-5014) GaussianMixture (GMM) improvements

2015-04-28 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5014: - Comment: was deleted (was: No need for umbrella JIRA) GaussianMixture (GMM)

[jira] [Assigned] (SPARK-7208) Add Matrix, SparseMatrix to __all__ list in linalg.py

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7208: --- Assignee: Apache Spark (was: Joseph K. Bradley) Add Matrix, SparseMatrix to __all__ list

[jira] [Assigned] (SPARK-7208) Add Matrix, SparseMatrix to __all__ list in linalg.py

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7208: --- Assignee: Joseph K. Bradley (was: Apache Spark) Add Matrix, SparseMatrix to __all__ list

[jira] [Created] (SPARK-7209) Adding new Manning book Spark in Action to the official Spark Webpage

2015-04-28 Thread Aleksandar Dragosavljevic (JIRA)
Aleksandar Dragosavljevic created SPARK-7209: Summary: Adding new Manning book Spark in Action to the official Spark Webpage Key: SPARK-7209 URL: https://issues.apache.org/jira/browse/SPARK-7209

[jira] [Commented] (SPARK-7208) Add Matrix, SparseMatrix to __all__ list in linalg.py

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518075#comment-14518075 ] Apache Spark commented on SPARK-7208: - User 'jkbradley' has created a pull request for

[jira] [Commented] (SPARK-7213) Exception while copying Hadoop config files due to permission issues

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518201#comment-14518201 ] Apache Spark commented on SPARK-7213: - User 'nishkamravi2' has created a pull request

[jira] [Assigned] (SPARK-7213) Exception while copying Hadoop config files due to permission issues

2015-04-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7213: --- Assignee: Apache Spark Exception while copying Hadoop config files due to permission issues

[jira] [Updated] (SPARK-7208) Add Matrix, SparseMatrix to __all__ list in linalg.py

2015-04-28 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7208: - Summary: Add Matrix, SparseMatrix to __all__ list in linalg.py (was: Add SparseMatrix to

  1   2   3   >