[jira] [Commented] (SPARK-3995) [PYSPARK] PySpark's sample methods do not work with NumPy 1.9

2014-10-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176637#comment-14176637 ] Xiangrui Meng commented on SPARK-3995: -- [~freeman-lab] Thanks for catching the bug! P

[jira] [Updated] (SPARK-3995) [PYSPARK] PySpark's sample methods do not work with NumPy 1.9

2014-10-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3995: - Target Version/s: 1.1.1, 1.2.0 > [PYSPARK] PySpark's sample methods do not work with NumPy 1.9 > -

[jira] [Updated] (SPARK-3995) [PYSPARK] PySpark's sample methods do not work with NumPy 1.9

2014-10-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3995: - Assignee: Jeremy Freeman > [PYSPARK] PySpark's sample methods do not work with NumPy 1.9 > ---

[jira] [Commented] (SPARK-3990) kryo.KryoException caused by ALS.trainImplicit in pyspark

2014-10-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176623#comment-14176623 ] Xiangrui Meng commented on SPARK-3990: -- [~gen] Could you try the following and see wh

[jira] [Updated] (SPARK-3990) kryo.KryoException caused by ALS.trainImplicit in pyspark

2014-10-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3990: - Priority: Critical (was: Major) > kryo.KryoException caused by ALS.trainImplicit in pyspark > ---

[jira] [Assigned] (SPARK-3990) kryo.KryoException caused by ALS.trainImplicit in pyspark

2014-10-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-3990: Assignee: Xiangrui Meng > kryo.KryoException caused by ALS.trainImplicit in pyspark > -

[jira] [Updated] (SPARK-3990) kryo.KryoException caused by ALS.trainImplicit in pyspark

2014-10-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3990: - Description: When we tried ALS.trainImplicit() in pyspark environment, it only works for iteratio

[jira] [Commented] (SPARK-3922) A global UTF8 constant for Spark

2014-10-19 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176615#comment-14176615 ] Shixiong Zhu commented on SPARK-3922: - PR: https://github.com/apache/spark/pull/2781

[jira] [Updated] (SPARK-4004) add akka-persistence based recovery mechanism for Master (maybe Worker)

2014-10-19 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-4004: --- Description: Since we have upgraded akka version to 2.3.x we can utilize the features which are actually help

[jira] [Commented] (SPARK-4004) add akka-persistence based recovery mechanism for Master (maybe Worker)

2014-10-19 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176571#comment-14176571 ] Nan Zhu commented on SPARK-4004: I will post a design doc after Monday > add akka-persist

[jira] [Updated] (SPARK-4004) add akka-persistence based recovery mechanism for Master (maybe Worker)

2014-10-19 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-4004: --- Description: Since we have upgraded akka version to 2.3.x we can utilize the features which are actually help

[jira] [Created] (SPARK-4004) add akka-persistence based recovery mechanism for Master (maybe Worker)

2014-10-19 Thread Nan Zhu (JIRA)
Nan Zhu created SPARK-4004: -- Summary: add akka-persistence based recovery mechanism for Master (maybe Worker) Key: SPARK-4004 URL: https://issues.apache.org/jira/browse/SPARK-4004 Project: Spark Is

[jira] [Commented] (SPARK-4003) Add 3 types for java sql context

2014-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176568#comment-14176568 ] Apache Spark commented on SPARK-4003: - User 'adrian-wang' has created a pull request f

[jira] [Commented] (SPARK-3957) Broadcast variable memory usage not reflected in UI

2014-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176567#comment-14176567 ] Apache Spark commented on SPARK-3957: - User 'CodingCat' has created a pull request for

[jira] [Created] (SPARK-4003) Add 3 types for java sql context

2014-10-19 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-4003: -- Summary: Add 3 types for java sql context Key: SPARK-4003 URL: https://issues.apache.org/jira/browse/SPARK-4003 Project: Spark Issue Type: Bug Componen

[jira] [Resolved] (SPARK-3902) Stabilize AsyncRDDActions and expose its methods in Java API

2014-10-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3902. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2760 [https://github.com/

[jira] [Commented] (SPARK-4002) JavaKafkaStreamSuite.testKafkaStream fails on OSX

2014-10-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176540#comment-14176540 ] Saisai Shao commented on SPARK-4002: Hi Ryan, would you mind describing your issue mor

[jira] [Resolved] (SPARK-3980) GraphX Performance Issue

2014-10-19 Thread Jarred Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarred Li resolved SPARK-3980. -- Resolution: Not a Problem Resolved the issue by running the job with big cluster. > GraphX Performance

[jira] [Updated] (SPARK-3561) Allow for pluggable execution contexts in Spark

2014-10-19 Thread Oleg Zhurakousky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zhurakousky updated SPARK-3561: Description: Currently Spark provides integration with external resource-managers such as A

[jira] [Updated] (SPARK-3561) Allow for pluggable execution contexts in Spark

2014-10-19 Thread Oleg Zhurakousky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zhurakousky updated SPARK-3561: Description: Currently Spark provides integration with external resource-managers such as A

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

2014-10-19 Thread Oleg Zhurakousky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176501#comment-14176501 ] Oleg Zhurakousky commented on SPARK-3561: - Resubmitting Pull Request (squashed) wh

[jira] [Commented] (SPARK-3996) Shade Jetty in Spark deliverables

2014-10-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176485#comment-14176485 ] Patrick Wendell commented on SPARK-3996: This is a good idea if we can do it using

[jira] [Commented] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

2014-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176480#comment-14176480 ] Apache Spark commented on SPARK-3967: - User 'ryan-williams' has created a pull request

[jira] [Created] (SPARK-4002) JavaKafkaStreamSuite.testKafkaStream fails on OSX

2014-10-19 Thread Ryan Williams (JIRA)
Ryan Williams created SPARK-4002: Summary: JavaKafkaStreamSuite.testKafkaStream fails on OSX Key: SPARK-4002 URL: https://issues.apache.org/jira/browse/SPARK-4002 Project: Spark Issue Type: B

[jira] [Commented] (SPARK-2426) Quadratic Minimization for MLlib ALS

2014-10-19 Thread Debasish Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176358#comment-14176358 ] Debasish Das commented on SPARK-2426: - [~mengxr] I thought more on it and one of the r

[jira] [Commented] (SPARK-4001) Add Apriori algorithm to Spark MLlib

2014-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176262#comment-14176262 ] Apache Spark commented on SPARK-4001: - User 'jackylk' has created a pull request for t

[jira] [Created] (SPARK-4001) Add Apriori algorithm to Spark MLlib

2014-10-19 Thread Jacky Li (JIRA)
Jacky Li created SPARK-4001: --- Summary: Add Apriori algorithm to Spark MLlib Key: SPARK-4001 URL: https://issues.apache.org/jira/browse/SPARK-4001 Project: Spark Issue Type: New Feature Co

[jira] [Commented] (SPARK-3997) scalastyle should output the error location

2014-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176253#comment-14176253 ] Apache Spark commented on SPARK-3997: - User 'witgo' has created a pull request for thi

[jira] [Updated] (SPARK-3997) scalastyle should output the error location

2014-10-19 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-3997: --- Component/s: Build > scalastyle should output the error location > ---

[jira] [Updated] (SPARK-3997) scalastyle should output the error location

2014-10-19 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-3997: --- Description: {{./dev/scalastyle}} => {noformat} Scalastyle checks failed at following occurrences: jav

[jira] [Commented] (SPARK-4000) Gathers unit tests logs to Jenkins master at the end of a Jenkins build

2014-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176248#comment-14176248 ] Apache Spark commented on SPARK-4000: - User 'liancheng' has created a pull request for

[jira] [Updated] (SPARK-4000) Gathers unit tests logs to Jenkins master at the end of a Jenkins build

2014-10-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4000: -- Description: Unit tests logs can be useful for debugging Jenkins failures. Currently these logs are dele

[jira] [Updated] (SPARK-2546) Configuration object thread safety issue

2014-10-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-2546: -- Affects Version/s: 1.2.0 1.0.2 1.1.0 > Configuration objec

[jira] [Commented] (SPARK-2546) Configuration object thread safety issue

2014-10-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176245#comment-14176245 ] Josh Rosen commented on SPARK-2546: --- I've fixed this in HadoopRDD and applied my fix to

[jira] [Created] (SPARK-4000) Gathers unit tests logs to Jenkins master at the end of a Jenkins build

2014-10-19 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-4000: - Summary: Gathers unit tests logs to Jenkins master at the end of a Jenkins build Key: SPARK-4000 URL: https://issues.apache.org/jira/browse/SPARK-4000 Project: Spark

[jira] [Resolved] (SPARK-2546) Configuration object thread safety issue

2014-10-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2546. --- Resolution: Fixed Fix Version/s: 1.0.3 1.1.1 1.2.0 Issue

[jira] [Commented] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

2014-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14176236#comment-14176236 ] Apache Spark commented on SPARK-3958: - User 'JoshRosen' has created a pull request for