[jira] [Commented] (SPARK-5058) Typos and broken URL

2015-01-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288908#comment-14288908 ] Apache Spark commented on SPARK-5058: - User 'jongyoul' has created a pull request for

[jira] [Commented] (SPARK-2823) GraphX jobs throw IllegalArgumentException

2015-01-22 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288893#comment-14288893 ] Josh Rosen commented on SPARK-2823: --- I think that the Snappy error was caused by a diffe

[jira] [Commented] (SPARK-5379) Add awaitTerminationOrTimeout

2015-01-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288879#comment-14288879 ] Apache Spark commented on SPARK-5379: - User 'zsxwing' has created a pull request for t

[jira] [Updated] (SPARK-5379) Add awaitTerminationOrTimeout

2015-01-22 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-5379: Description: Add `awaitTerminationOrTimeout` to return if the waiting time elapsed: 1 `true` if it's

[jira] [Created] (SPARK-5379) Add awaitTerminationOrTimeout

2015-01-22 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-5379: --- Summary: Add awaitTerminationOrTimeout Key: SPARK-5379 URL: https://issues.apache.org/jira/browse/SPARK-5379 Project: Spark Issue Type: Improvement C

[jira] [Created] (SPARK-5378) There is no return results with 'select' operating on View

2015-01-22 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-5378: -- Summary: There is no return results with 'select' operating on View Key: SPARK-5378 URL: https://issues.apache.org/jira/browse/SPARK-5378 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-5377) Dynamically add jar into Spark Driver's classpath.

2015-01-22 Thread Xuefu Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288866#comment-14288866 ] Xuefu Zhang commented on SPARK-5377: cc [~sandyr]. > Dynamically add jar into Spark D

[jira] [Closed] (SPARK-3876) Doing a RDD map/reduce within a DStream map fails with a high enough input rate

2015-01-22 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das closed SPARK-3876. Resolution: Not a Problem > Doing a RDD map/reduce within a DStream map fails with a high enough inp

[jira] [Commented] (SPARK-3876) Doing a RDD map/reduce within a DStream map fails with a high enough input rate

2015-01-22 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288851#comment-14288851 ] Tathagata Das commented on SPARK-3876: -- This seems to be a fundamentally incorrect co

[jira] [Resolved] (SPARK-5315) reduceByWindow returns Scala DStream not JavaDStream

2015-01-22 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-5315. -- Resolution: Fixed Fix Version/s: 1.3.0 > reduceByWindow returns Scala DStream not JavaDSt

[jira] [Resolved] (SPARK-3541) Improve ALS internal storage

2015-01-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3541. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3720 [https://githu

[jira] [Resolved] (SPARK-5233) Error replay of WAL when recovered from driver failue

2015-01-22 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-5233. -- Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 > Error replay of WAL whe

[jira] [Commented] (SPARK-5376) [Mesos] MesosExecutor should have correct resources

2015-01-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288795#comment-14288795 ] Apache Spark commented on SPARK-5376: - User 'jongyoul' has created a pull request for

[jira] [Updated] (SPARK-5376) [Mesos] MesosExecutor should have correct resources

2015-01-22 Thread Jongyoul Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jongyoul Lee updated SPARK-5376: Summary: [Mesos] MesosExecutor should have correct resources (was: [Mesos] MesosExecutor should hav

[jira] [Commented] (SPARK-5377) Dynamically add jar into Spark Driver's classpath.

2015-01-22 Thread Chengxiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288783#comment-14288783 ] Chengxiang Li commented on SPARK-5377: -- cc [~xuefuz], [~rxin], [~Grace Huang]. > Dyn

[jira] [Created] (SPARK-5377) Dynamically add jar into Spark Driver's classpath.

2015-01-22 Thread Chengxiang Li (JIRA)
Chengxiang Li created SPARK-5377: Summary: Dynamically add jar into Spark Driver's classpath. Key: SPARK-5377 URL: https://issues.apache.org/jira/browse/SPARK-5377 Project: Spark Issue Type:

[jira] [Updated] (SPARK-5376) [Mesos] MesosExecutor should have correct resources.

2015-01-22 Thread Jongyoul Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jongyoul Lee updated SPARK-5376: Affects Version/s: 1.2.0 > [Mesos] MesosExecutor should have correct resources. > --

[jira] [Updated] (SPARK-5376) MesosExecutor should have correct resources.

2015-01-22 Thread Jongyoul Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jongyoul Lee updated SPARK-5376: Component/s: Mesos > MesosExecutor should have correct resources. >

[jira] [Updated] (SPARK-5376) [Mesos] MesosExecutor should have correct resources.

2015-01-22 Thread Jongyoul Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jongyoul Lee updated SPARK-5376: Summary: [Mesos] MesosExecutor should have correct resources. (was: MesosExecutor should have corre

[jira] [Created] (SPARK-5376) MesosExecutor should have correct resources.

2015-01-22 Thread Jongyoul Lee (JIRA)
Jongyoul Lee created SPARK-5376: --- Summary: MesosExecutor should have correct resources. Key: SPARK-5376 URL: https://issues.apache.org/jira/browse/SPARK-5376 Project: Spark Issue Type: Improvem

[jira] [Updated] (SPARK-5375) Specify more clearly about the max thread meaning in the ConnectionManager

2015-01-22 Thread DjvuLee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-5375: --- Description: In the ConnectionManager.scala file, there is three thread pool: handleMessageExecutor, handleR

[jira] [Updated] (SPARK-5375) Specify more clearly about the max thread meaning in the ConnectionManager

2015-01-22 Thread DjvuLee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-5375: --- Description: In the ConnectionManager.scala file, there is three thread pool: handleMessageExecutor, handleR

[jira] [Created] (SPARK-5375) Specify more clearly about the max thread meaning in the ConnectionManager

2015-01-22 Thread DjvuLee (JIRA)
DjvuLee created SPARK-5375: -- Summary: Specify more clearly about the max thread meaning in the ConnectionManager Key: SPARK-5375 URL: https://issues.apache.org/jira/browse/SPARK-5375 Project: Spark

[jira] [Commented] (SPARK-5374) abstract RDD's DAG graph iteration in DAGScheduler

2015-01-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288695#comment-14288695 ] Apache Spark commented on SPARK-5374: - User 'cloud-fan' has created a pull request for

[jira] [Updated] (SPARK-5374) abstract RDD's DAG graph iteration in DAGScheduler

2015-01-22 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-5374: --- Summary: abstract RDD's DAG graph iteration in DAGScheduler (was: abstract RDD's DAG graph iteration

[jira] [Created] (SPARK-5374) abstract RDD's DAG graph iteration

2015-01-22 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-5374: -- Summary: abstract RDD's DAG graph iteration Key: SPARK-5374 URL: https://issues.apache.org/jira/browse/SPARK-5374 Project: Spark Issue Type: Sub-task C

[jira] [Updated] (SPARK-5374) abstract RDD's DAG graph iteration in

2015-01-22 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-5374: --- Summary: abstract RDD's DAG graph iteration in (was: abstract RDD's DAG graph iteration) > abstract

[jira] [Commented] (SPARK-5373) literal in agg grouping expressioons leads to incorrect result

2015-01-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288660#comment-14288660 ] Apache Spark commented on SPARK-5373: - User 'scwf' has created a pull request for this

[jira] [Commented] (SPARK-4136) Under dynamic allocation, cancel outstanding executor requests when no longer needed

2015-01-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288649#comment-14288649 ] Apache Spark commented on SPARK-4136: - User 'sryza' has created a pull request for thi

[jira] [Updated] (SPARK-5373) literal in agg grouping expressioons leads to incorrect result

2015-01-22 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangfei updated SPARK-5373: --- Description: select key, count( * ) from src group by key, 1 will get the wrong answer! (was: select key, cou

[jira] [Created] (SPARK-5373) literal in agg grouping expressioons leads to incorrect result

2015-01-22 Thread wangfei (JIRA)
wangfei created SPARK-5373: -- Summary: literal in agg grouping expressioons leads to incorrect result Key: SPARK-5373 URL: https://issues.apache.org/jira/browse/SPARK-5373 Project: Spark Issue Type

[jira] [Commented] (SPARK-5372) Change the default storage level of window operators

2015-01-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288636#comment-14288636 ] Apache Spark commented on SPARK-5372: - User 'jerryshao' has created a pull request for

[jira] [Updated] (SPARK-4136) Under dynamic allocation, cancel outstanding executor requests when no longer needed

2015-01-22 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-4136: -- Summary: Under dynamic allocation, cancel outstanding executor requests when no longer needed (was: Und

[jira] [Commented] (SPARK-5260) Expose JsonRDD.allKeysWithValueTypes() in a utility class

2015-01-22 Thread Corey J. Nolet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288621#comment-14288621 ] Corey J. Nolet commented on SPARK-5260: --- May I be added to the proper list so that I

[jira] [Created] (SPARK-5372) Change the default storage level of window operators

2015-01-22 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-5372: -- Summary: Change the default storage level of window operators Key: SPARK-5372 URL: https://issues.apache.org/jira/browse/SPARK-5372 Project: Spark Issue Type: Bu

[jira] [Created] (SPARK-5371) SparkSQL Fails to parse Query with UNION ALL in subquery

2015-01-22 Thread David Ross (JIRA)
David Ross created SPARK-5371: - Summary: SparkSQL Fails to parse Query with UNION ALL in subquery Key: SPARK-5371 URL: https://issues.apache.org/jira/browse/SPARK-5371 Project: Spark Issue Type:

[jira] [Commented] (SPARK-5366) check for mode of private key file

2015-01-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288480#comment-14288480 ] Nicholas Chammas commented on SPARK-5366: - If the problem is no warnings and no er

[jira] [Commented] (SPARK-5358) spark.files.userClassPathFirst doesn't work correctly

2015-01-22 Thread Matthew Whelan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288430#comment-14288430 ] Matthew Whelan commented on SPARK-5358: --- The PR for 2996 issue seems to cover SPARK-

[jira] [Commented] (SPARK-5358) spark.files.userClassPathFirst doesn't work correctly

2015-01-22 Thread Matthew Whelan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288423#comment-14288423 ] Matthew Whelan commented on SPARK-5358: --- Also, Vanzin's https://github.com/apache/sp

[jira] [Commented] (SPARK-5358) spark.files.userClassPathFirst doesn't work correctly

2015-01-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288360#comment-14288360 ] Apache Spark commented on SPARK-5358: - User 'MattWhelan' has created a pull request fo

[jira] [Comment Edited] (SPARK-1529) Support setting spark.local.dirs to a hadoop FileSystem

2015-01-22 Thread Kannan Rajah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288343#comment-14288343 ] Kannan Rajah edited comment on SPARK-1529 at 1/22/15 10:30 PM: -

[jira] [Updated] (SPARK-1529) Support setting spark.local.dirs to a hadoop FileSystem

2015-01-22 Thread Kannan Rajah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kannan Rajah updated SPARK-1529: Attachment: MapRShuffleOnSpark.pdf High level proposal of the changes to make shuffle use Distribute

[jira] [Commented] (SPARK-1529) Support setting spark.local.dirs to a hadoop FileSystem

2015-01-22 Thread Kannan Rajah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288341#comment-14288341 ] Kannan Rajah commented on SPARK-1529: - [~pwendell] I would like us to consider the opt

[jira] [Comment Edited] (SPARK-5361) python tuple not supported while converting PythonRDD back to JavaRDD

2015-01-22 Thread Winston Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288331#comment-14288331 ] Winston Chen edited comment on SPARK-5361 at 1/22/15 10:23 PM: -

[jira] [Commented] (SPARK-5361) python tuple not supported while converting PythonRDD back to JavaRDD

2015-01-22 Thread Winston Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288331#comment-14288331 ] Winston Chen commented on SPARK-5361: - Found a good way to reproduce it: {noformat} f

[jira] [Commented] (SPARK-5358) spark.files.userClassPathFirst doesn't work correctly

2015-01-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288142#comment-14288142 ] Apache Spark commented on SPARK-5358: - User 'MattWhelan' has created a pull request fo

[jira] [Resolved] (SPARK-5370) Remove some unnecessary synchronization in YarnAllocator

2015-01-22 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-5370. -- Resolution: Fixed Fix Version/s: 1.3.0 > Remove some unnecessary synchronization in YarnA

[jira] [Issue Comment Deleted] (SPARK-4739) spark.files.userClassPathFirst does not work in local[*] mode

2015-01-22 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-4739: -- Comment: was deleted (was: I'm pretty sure I tested that change in local mode, explicitly overri

[jira] [Commented] (SPARK-5162) Python yarn-cluster mode

2015-01-22 Thread Jared Holmberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288072#comment-14288072 ] Jared Holmberg commented on SPARK-5162: --- Hello! My company is planning to use PySpar

[jira] [Commented] (SPARK-4739) spark.files.userClassPathFirst does not work in local[*] mode

2015-01-22 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288050#comment-14288050 ] Marcelo Vanzin commented on SPARK-4739: --- I'm pretty sure I tested that change in loc

[jira] [Commented] (SPARK-4739) spark.files.userClassPathFirst does not work in local[*] mode

2015-01-22 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288047#comment-14288047 ] Marcelo Vanzin commented on SPARK-4739: --- I'm pretty sure I tested that change in loc

[jira] [Commented] (SPARK-4877) userClassPathFirst doesn't handle user classes inheriting from parent

2015-01-22 Thread Matthew Whelan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288023#comment-14288023 ] Matthew Whelan commented on SPARK-4877: --- Looking at the PR, you're still overriding

[jira] [Commented] (SPARK-2345) ForEachDStream should have an option of running the foreachfunc on Spark

2015-01-22 Thread Hari Shreedharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288013#comment-14288013 ] Hari Shreedharan commented on SPARK-2345: - +1. Please do. > ForEachDStream should

[jira] [Commented] (SPARK-4739) spark.files.userClassPathFirst does not work in local[*] mode

2015-01-22 Thread Matthew Whelan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288012#comment-14288012 ] Matthew Whelan commented on SPARK-4739: --- I'm not sure that resolving 5358 will resol

[jira] [Commented] (SPARK-1863) Allowing user jars to take precedence over Spark jars does not work as expected

2015-01-22 Thread Matthew Whelan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288007#comment-14288007 ] Matthew Whelan commented on SPARK-1863: --- Similar issues with the same root cause: Ch

[jira] [Commented] (SPARK-5358) spark.files.userClassPathFirst doesn't work correctly

2015-01-22 Thread Matthew Whelan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287994#comment-14287994 ] Matthew Whelan commented on SPARK-5358: --- I'll mark dupes, and I'm working on a PR.

[jira] [Commented] (SPARK-5368) Support user configurable akka parameters.

2015-01-22 Thread jay vyas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287910#comment-14287910 ] jay vyas commented on SPARK-5368: - per conversations with [~willbenton] and [~tstclair] ,

[jira] [Commented] (SPARK-3622) Provide a custom transformation that can output multiple RDDs

2015-01-22 Thread Xuefu Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287901#comment-14287901 ] Xuefu Zhang commented on SPARK-3622: Unfortunately no. In Hive the problem is more gen

[jira] [Commented] (SPARK-3622) Provide a custom transformation that can output multiple RDDs

2015-01-22 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287883#comment-14287883 ] Imran Rashid commented on SPARK-3622: - Is there any chance the immediate use case is s

[jira] [Commented] (SPARK-5370) Remove some unnecessary synchronization in YarnAllocator

2015-01-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287752#comment-14287752 ] Apache Spark commented on SPARK-5370: - User 'sryza' has created a pull request for thi

[jira] [Commented] (SPARK-1714) Take advantage of AMRMClient APIs to simplify logic in YarnAllocationHandler

2015-01-22 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287753#comment-14287753 ] Sandy Ryza commented on SPARK-1714: --- Oops yeah, my bad. Filed SPARK-5370 and posted a p

[jira] [Created] (SPARK-5370) Remove some unnecessary synchronization in YarnAllocator

2015-01-22 Thread Sandy Ryza (JIRA)
Sandy Ryza created SPARK-5370: - Summary: Remove some unnecessary synchronization in YarnAllocator Key: SPARK-5370 URL: https://issues.apache.org/jira/browse/SPARK-5370 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-5368) Support user configurable akka parameters.

2015-01-22 Thread jay vyas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287698#comment-14287698 ] jay vyas edited comment on SPARK-5368 at 1/22/15 4:50 PM: -- It cou

[jira] [Created] (SPARK-5369) remove allocatedHostToContainersMap.synchronized in YarnAllocator

2015-01-22 Thread Lianhui Wang (JIRA)
Lianhui Wang created SPARK-5369: --- Summary: remove allocatedHostToContainersMap.synchronized in YarnAllocator Key: SPARK-5369 URL: https://issues.apache.org/jira/browse/SPARK-5369 Project: Spark

[jira] [Commented] (SPARK-5368) Support user configurable akka parameters.

2015-01-22 Thread Timothy St. Clair (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287697#comment-14287697 ] Timothy St. Clair commented on SPARK-5368: -- To be specific we would like to optio

[jira] [Commented] (SPARK-5368) Support user configurable akka parameters.

2015-01-22 Thread jay vyas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287698#comment-14287698 ] jay vyas commented on SPARK-5368: - It could be a possibility that we can simply inject the

[jira] [Comment Edited] (SPARK-5368) Support user configurable akka parameters.

2015-01-22 Thread Timothy St. Clair (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287697#comment-14287697 ] Timothy St. Clair edited comment on SPARK-5368 at 1/22/15 4:41 PM: -

[jira] [Updated] (SPARK-5368) Support user configurable akka parameters.

2015-01-22 Thread jay vyas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jay vyas updated SPARK-5368: Description: Spark sets up actors for akka with a set of variables which are defined in the {{AkkaUtils.sca

[jira] [Created] (SPARK-5368) Support user configurable akka parameters.

2015-01-22 Thread jay vyas (JIRA)
jay vyas created SPARK-5368: --- Summary: Support user configurable akka parameters. Key: SPARK-5368 URL: https://issues.apache.org/jira/browse/SPARK-5368 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-5365) Refactor KMeans to reduce redundant data

2015-01-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5365: - Assignee: Liang-Chi Hsieh > Refactor KMeans to reduce redundant data > ---

[jira] [Resolved] (SPARK-5365) Refactor KMeans to reduce redundant data

2015-01-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5365. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4159 [https://githu

[jira] [Commented] (SPARK-3185) SPARK launch on Hadoop 2 in EC2 throws Tachyon exception when Formatting JOURNAL_FOLDER

2015-01-22 Thread Grzegorz Dubicki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287631#comment-14287631 ] Grzegorz Dubicki commented on SPARK-3185: - [~florianverhein] can you share your wo

[jira] [Commented] (SPARK-5367) support star expression in udf

2015-01-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287612#comment-14287612 ] Apache Spark commented on SPARK-5367: - User 'scwf' has created a pull request for this

[jira] [Updated] (SPARK-5367) support star expression in udf

2015-01-22 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangfei updated SPARK-5367: --- Description: now spark sql does not support star expression in udf, the following sql will get error ``` sele

[jira] [Created] (SPARK-5367) support star expression in udf

2015-01-22 Thread wangfei (JIRA)
wangfei created SPARK-5367: -- Summary: support star expression in udf Key: SPARK-5367 URL: https://issues.apache.org/jira/browse/SPARK-5367 Project: Spark Issue Type: Bug Components: SQL

[jira] [Issue Comment Deleted] (SPARK-5366) check for mode of private key file

2015-01-22 Thread liu chang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu chang updated SPARK-5366: - Comment: was deleted (was: We should check the mode of private key. I use spark ec2, but I forget to set

[jira] [Commented] (SPARK-1714) Take advantage of AMRMClient APIs to simplify logic in YarnAllocationHandler

2015-01-22 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287582#comment-14287582 ] Thomas Graves commented on SPARK-1714: -- No it doesn't need to be synchronized, its on

[jira] [Commented] (SPARK-5366) check for mode of private key file

2015-01-22 Thread liu chang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287584#comment-14287584 ] liu chang commented on SPARK-5366: -- We should check the mode of private key. I use spark

[jira] [Commented] (SPARK-5366) check for mode of private key file

2015-01-22 Thread liu chang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287583#comment-14287583 ] liu chang commented on SPARK-5366: -- We should check the mode of private key. I use spark

[jira] [Commented] (SPARK-5366) check for mode of private key file

2015-01-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287577#comment-14287577 ] Apache Spark commented on SPARK-5366: - User 'Liuchang0812' has created a pull request

[jira] [Commented] (SPARK-5366) check for mode of private key file

2015-01-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287545#comment-14287545 ] Sean Owen commented on SPARK-5366: -- There's not much info here. What should check what pr

[jira] [Created] (SPARK-5366) check for mode of private key file

2015-01-22 Thread liu chang (JIRA)
liu chang created SPARK-5366: Summary: check for mode of private key file Key: SPARK-5366 URL: https://issues.apache.org/jira/browse/SPARK-5366 Project: Spark Issue Type: Improvement Co

[jira] [Commented] (SPARK-5308) MD5 / SHA1 hash format doesn't match standard Maven output

2015-01-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287504#comment-14287504 ] Apache Spark commented on SPARK-5308: - User 'srowen' has created a pull request for th

[jira] [Updated] (SPARK-5308) MD5 / SHA1 hash format doesn't match standard Maven output

2015-01-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5308: - Summary: MD5 / SHA1 hash format doesn't match standard Maven output (was: MD5 for 1.2.0 release in not pr

[jira] [Commented] (SPARK-5350) There are issues when combining Spark and CDK (https://github.com/egonw/cdk).

2015-01-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287499#comment-14287499 ] Sean Owen commented on SPARK-5350: -- OK, but you still need to use Spark built for Hadoop

[jira] [Commented] (SPARK-5350) There are issues when combining Spark and CDK (https://github.com/egonw/cdk).

2015-01-22 Thread Staffan Arvidsson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287494#comment-14287494 ] Staffan Arvidsson commented on SPARK-5350: -- Well I've tried to fix this for some

[jira] [Commented] (SPARK-2335) k-Nearest Neighbor classification and regression for MLLib

2015-01-22 Thread Devan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287419#comment-14287419 ] Devan commented on SPARK-2335: -- Hi, will https://github.com/kaushikranjan/knnJoin work for do

[jira] [Commented] (SPARK-5358) spark.files.userClassPathFirst doesn't work correctly

2015-01-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287399#comment-14287399 ] Sean Owen commented on SPARK-5358: -- I'm pretty sure this is the same issue as the old iss

[jira] [Commented] (SPARK-5350) There are issues when combining Spark and CDK (https://github.com/egonw/cdk).

2015-01-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287368#comment-14287368 ] Sean Owen commented on SPARK-5350: -- The problem is that you need to mark Spark as a 'prov

[jira] [Commented] (SPARK-5350) There are issues when combining Spark and CDK (https://github.com/egonw/cdk).

2015-01-22 Thread Staffan Arvidsson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287353#comment-14287353 ] Staffan Arvidsson commented on SPARK-5350: -- My guess would be that there are some

[jira] [Commented] (SPARK-5348) s3a:// protocol and hadoop-aws dependency

2015-01-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287308#comment-14287308 ] Sean Owen commented on SPARK-5348: -- OK, but I think the more basic question remains: this

[jira] [Updated] (SPARK-5348) s3a:// protocol and hadoop-aws dependency

2015-01-22 Thread Rafal Kwasny (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafal Kwasny updated SPARK-5348: Description: Hadoop 2.6.0+ includes new s3a library which is a lot better than the old one. https://

[jira] [Commented] (SPARK-5348) s3a:// protocol and hadoop-aws dependency

2015-01-22 Thread Rafal Kwasny (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287295#comment-14287295 ] Rafal Kwasny commented on SPARK-5348: - Unfortunately I am not a licensing expert Looks

[jira] [Commented] (SPARK-5349) Multiple spark shells should be able to share resources

2015-01-22 Thread Tobias Bertelsen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287276#comment-14287276 ] Tobias Bertelsen commented on SPARK-5349: - I think you're right, though it is prob

[jira] [Closed] (SPARK-2364) ShuffledDStream run tasks only when dstream has partition items

2015-01-22 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das closed SPARK-2364. Resolution: Won't Fix > ShuffledDStream run tasks only when dstream has partition items > --

[jira] [Commented] (SPARK-5349) Multiple spark shells should be able to share resources

2015-01-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287261#comment-14287261 ] Sean Owen commented on SPARK-5349: -- I think this is a subset of https://issues.apache.org

[jira] [Commented] (SPARK-2345) ForEachDStream should have an option of running the foreachfunc on Spark

2015-01-22 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287260#comment-14287260 ] Tathagata Das commented on SPARK-2345: -- If there is no further comment on this, i am

[jira] [Resolved] (SPARK-3325) Add a parameter to the method print in class DStream.

2015-01-22 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-3325. -- > Add a parameter to the method print in class DStream. > --

[jira] [Commented] (SPARK-5348) s3a:// protocol and hadoop-aws dependency

2015-01-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287243#comment-14287243 ] Sean Owen commented on SPARK-5348: -- Hm, but wouldn't Spark have the same problem distribu

[jira] [Commented] (SPARK-5350) There are issues when combining Spark and CDK (https://github.com/egonw/cdk).

2015-01-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287242#comment-14287242 ] Sean Owen commented on SPARK-5350: -- Is this really a Spark issue? My hunch is that this t

  1   2   >