[jira] [Commented] (SPARK-3785) Support off-loading computations to a GPU

2014-10-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163125#comment-14163125 ] Sean Owen commented on SPARK-3785: -- Sure, but a GPU isn't going to be good at general map

[jira] [Resolved] (SPARK-3836) Spark REPL optionally propagate internal exceptions

2014-10-07 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3836. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Ahir Reddy https://github.c

[jira] [Updated] (SPARK-3807) SparkSql does not work for tables created using custom serde

2014-10-07 Thread chirag aggarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chirag aggarwal updated SPARK-3807: --- Description: SparkSql crashes on selecting tables using custom serde. Example: -

[jira] [Commented] (SPARK-2811) update algebird to 0.8.1

2014-10-07 Thread Dan Di Spaltro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163034#comment-14163034 ] Dan Di Spaltro commented on SPARK-2811: --- Looks like algebird_2.11 artifacts are on m

[jira] [Commented] (SPARK-2426) Quadratic Minimization for MLlib ALS

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163027#comment-14163027 ] Apache Spark commented on SPARK-2426: - User 'debasish83' has created a pull request fo

[jira] [Commented] (SPARK-3781) code style format

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163018#comment-14163018 ] Apache Spark commented on SPARK-3781: - User 'shijinkui' has created a pull request for

[jira] [Resolved] (SPARK-1656) Potential resource leak in HttpBroadcast, SparkSubmitArguments, FileSystemPersistenceEngine and DiskStore

2014-10-07 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-1656. - Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Already merged into master

[jira] [Commented] (SPARK-3808) PySpark fails to start in Windows

2014-10-07 Thread Masayoshi TSUZUKI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162935#comment-14162935 ] Masayoshi TSUZUKI commented on SPARK-3808: -- I verified and it works well! Thank y

[jira] [Commented] (SPARK-3005) Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask()

2014-10-07 Thread Xu Zhongxing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162913#comment-14162913 ] Xu Zhongxing commented on SPARK-3005: - Resolved in https://github.com/apache/spark/pul

[jira] [Closed] (SPARK-3005) Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask()

2014-10-07 Thread Xu Zhongxing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Zhongxing closed SPARK-3005. --- Resolution: Fixed > Spark with Mesos fine-grained mode throws UnsupportedOperationException in > Meso

[jira] [Issue Comment Deleted] (SPARK-3820) Specialize columnSimilarity() without any threshold

2014-10-07 Thread Reza Zadeh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Zadeh updated SPARK-3820: -- Comment: was deleted (was: See previous comment on resolution.) > Specialize columnSimilarity() without

[jira] [Resolved] (SPARK-3820) Specialize columnSimilarity() without any threshold

2014-10-07 Thread Reza Zadeh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Zadeh resolved SPARK-3820. --- Resolution: Fixed See previous comment on resolution. > Specialize columnSimilarity() without any thr

[jira] [Commented] (SPARK-3820) Specialize columnSimilarity() without any threshold

2014-10-07 Thread Reza Zadeh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162899#comment-14162899 ] Reza Zadeh commented on SPARK-3820: --- I ran columnSimilarities(0.0) with the random numbe

[jira] [Resolved] (SPARK-3412) Add Missing Types for Row API

2014-10-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3412. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2689 [https://github.com/

[jira] [Commented] (SPARK-3843) Cleanup scalastyle.txt at the end of running dev/scalastyle

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162867#comment-14162867 ] Apache Spark commented on SPARK-3843: - User 'sarutak' has created a pull request for t

[jira] [Created] (SPARK-3843) Cleanup scalastyle.txt at the end of running dev/scalastyle

2014-10-07 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3843: - Summary: Cleanup scalastyle.txt at the end of running dev/scalastyle Key: SPARK-3843 URL: https://issues.apache.org/jira/browse/SPARK-3843 Project: Spark

[jira] [Commented] (SPARK-3569) Add metadata field to StructField

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162853#comment-14162853 ] Apache Spark commented on SPARK-3569: - User 'mengxr' has created a pull request for th

[jira] [Created] (SPARK-3842) Remove the hacks for Python callback server in py4j

2014-10-07 Thread Davies Liu (JIRA)
Davies Liu created SPARK-3842: - Summary: Remove the hacks for Python callback server in py4j Key: SPARK-3842 URL: https://issues.apache.org/jira/browse/SPARK-3842 Project: Spark Issue Type: Impr

[jira] [Closed] (SPARK-3829) Make Spark logo image on the header of HistoryPage as a link to HistoryPage's page #1

2014-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3829. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Assignee: Kousuke Saruta > M

[jira] [Commented] (SPARK-3820) Specialize columnSimilarity() without any threshold

2014-10-07 Thread Reza Zadeh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162814#comment-14162814 ] Reza Zadeh commented on SPARK-3820: --- I will do an experiment to see if the random number

[jira] [Resolved] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-10-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3398. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2339 [https://github.com/

[jira] [Resolved] (SPARK-3832) Upgrade Breeze dependency to 0.10

2014-10-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3832. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2693 [https://githu

[jira] [Updated] (SPARK-3832) Upgrade Breeze dependency to 0.10

2014-10-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3832: - Assignee: DB Tsai > Upgrade Breeze dependency to 0.10 > - > >

[jira] [Commented] (SPARK-3841) Pretty-print Params case classes for tests

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162795#comment-14162795 ] Apache Spark commented on SPARK-3841: - User 'jkbradley' has created a pull request for

[jira] [Updated] (SPARK-3838) Python code example for Word2Vec in user guide

2014-10-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3838: - Assignee: (was: Liquan Pei) > Python code example for Word2Vec in user guide > ---

[jira] [Resolved] (SPARK-3486) Add PySpark support for Word2Vec

2014-10-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3486. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2356 [https://githu

[jira] [Created] (SPARK-3841) Pretty-print Params case classes for tests

2014-10-07 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-3841: Summary: Pretty-print Params case classes for tests Key: SPARK-3841 URL: https://issues.apache.org/jira/browse/SPARK-3841 Project: Spark Issue Type:

[jira] [Updated] (SPARK-3841) Pretty-print Params case classes for tests

2014-10-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-3841: - Description: Provide a parent class for the Params case classes used in many MLlib example

[jira] [Resolved] (SPARK-3790) CosineSimilarity via DIMSUM example

2014-10-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3790. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2622 [https://githu

[jira] [Commented] (SPARK-3840) Spark EC2 templates fail when variables are missing

2014-10-07 Thread Allan Douglas R. de Oliveira (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162783#comment-14162783 ] Allan Douglas R. de Oliveira commented on SPARK-3840: - PR: https://git

[jira] [Commented] (SPARK-3654) Implement all extended HiveQL statements/commands with a separate parser combinator

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162757#comment-14162757 ] Apache Spark commented on SPARK-3654: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2014-10-07 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162755#comment-14162755 ] Shivaram Venkataraman commented on SPARK-3821: -- 1. Yes - the same stuff is in

[jira] [Closed] (SPARK-3777) Display "Executor ID" for Tasks in Stage page

2014-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3777. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Target Version/s: 1.1.1,

[jira] [Updated] (SPARK-3777) Display "Executor ID" for Tasks in Stage page

2014-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3777: - Assignee: Shixiong Zhu > Display "Executor ID" for Tasks in Stage page > -

[jira] [Created] (SPARK-3840) Spark EC2 templates fail when variables are missing

2014-10-07 Thread Allan Douglas R. de Oliveira (JIRA)
Allan Douglas R. de Oliveira created SPARK-3840: --- Summary: Spark EC2 templates fail when variables are missing Key: SPARK-3840 URL: https://issues.apache.org/jira/browse/SPARK-3840 Projec

[jira] [Commented] (SPARK-3661) spark.*.memory is ignored in cluster mode

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162721#comment-14162721 ] Apache Spark commented on SPARK-3661: - User 'andrewor14' has created a pull request fo

[jira] [Updated] (SPARK-3661) spark.*.memory is ignored in cluster mode

2014-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3661: - Description: This is related to https://issues.apache.org/jira/browse/SPARK-3653, but for the config. Note

[jira] [Updated] (SPARK-3661) spark.*.memory is ignored in cluster mode

2014-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3661: - Description: This is related to https://issues.apache.org/jira/browse/SPARK-3653, but for the config. Note

[jira] [Updated] (SPARK-3661) spark.*.memory is ignored in cluster mode

2014-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3661: - Summary: spark.*.memory is ignored in cluster mode (was: spark.driver.memory is ignored in cluster mode)

[jira] [Created] (SPARK-3839) Reimplement HashOuterJoin to construct hash table of only one relation

2014-10-07 Thread Liquan Pei (JIRA)
Liquan Pei created SPARK-3839: - Summary: Reimplement HashOuterJoin to construct hash table of only one relation Key: SPARK-3839 URL: https://issues.apache.org/jira/browse/SPARK-3839 Project: Spark

[jira] [Created] (SPARK-3838) Python code example for Word2Vec in user guide

2014-10-07 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-3838: Summary: Python code example for Word2Vec in user guide Key: SPARK-3838 URL: https://issues.apache.org/jira/browse/SPARK-3838 Project: Spark Issue Type: Sub-

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

2014-10-07 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162685#comment-14162685 ] Sandy Ryza commented on SPARK-3174: --- bq. Maybe it makes sense to just call it `spark.dy

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2014-10-07 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162663#comment-14162663 ] Nicholas Chammas commented on SPARK-3821: - [~shivaram] / [~pwendell]: # In a Spark

[jira] [Comment Edited] (SPARK-3785) Support off-loading computations to a GPU

2014-10-07 Thread Reza Farivar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162662#comment-14162662 ] Reza Farivar edited comment on SPARK-3785 at 10/7/14 10:12 PM: -

[jira] [Comment Edited] (SPARK-3785) Support off-loading computations to a GPU

2014-10-07 Thread Reza Farivar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162662#comment-14162662 ] Reza Farivar edited comment on SPARK-3785 at 10/7/14 10:13 PM: -

[jira] [Commented] (SPARK-3785) Support off-loading computations to a GPU

2014-10-07 Thread Reza Farivar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162662#comment-14162662 ] Reza Farivar commented on SPARK-3785: - I thought to add that the project Sumatra might

[jira] [Updated] (SPARK-3682) Add helpful warnings to the UI

2014-10-07 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-3682: -- Attachment: SPARK-3682Design.pdf Posting an initial design > Add helpful warnings to the UI > -

[jira] [Commented] (SPARK-2321) Design a proper progress reporting & event listener API

2014-10-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162615#comment-14162615 ] Josh Rosen commented on SPARK-2321: --- I've opened a WIP pull request in order to discuss

[jira] [Commented] (SPARK-2321) Design a proper progress reporting & event listener API

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162614#comment-14162614 ] Apache Spark commented on SPARK-2321: - User 'JoshRosen' has created a pull request for

[jira] [Commented] (SPARK-3637) NPE in ShuffleMapTask

2014-10-07 Thread Steven Lewis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162600#comment-14162600 ] Steven Lewis commented on SPARK-3637: - I see the same thing running a Java Map reduce

[jira] [Created] (SPARK-3837) Warn when YARN is killing containers for exceeding memory limits

2014-10-07 Thread Sandy Ryza (JIRA)
Sandy Ryza created SPARK-3837: - Summary: Warn when YARN is killing containers for exceeding memory limits Key: SPARK-3837 URL: https://issues.apache.org/jira/browse/SPARK-3837 Project: Spark Iss

[jira] [Commented] (SPARK-3461) Support external groupByKey using repartitionAndSortWithinPartitions

2014-10-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162547#comment-14162547 ] Davies Liu commented on SPARK-3461: --- [~pwendell] I will start to work on this after merg

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

2014-10-07 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162514#comment-14162514 ] Marcelo Vanzin commented on SPARK-3174: --- Hi Andrew, thanks for writing this up. My

[jira] [Commented] (SPARK-3836) Spark REPL optionally propagate internal exceptions

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162470#comment-14162470 ] Apache Spark commented on SPARK-3836: - User 'ahirreddy' has created a pull request for

[jira] [Created] (SPARK-3836) Spark REPL optionally propagate internal exceptions

2014-10-07 Thread Ahir Reddy (JIRA)
Ahir Reddy created SPARK-3836: - Summary: Spark REPL optionally propagate internal exceptions Key: SPARK-3836 URL: https://issues.apache.org/jira/browse/SPARK-3836 Project: Spark Issue Type: Impr

[jira] [Resolved] (SPARK-3731) RDD caching stops working in pyspark after some time

2014-10-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3731. --- Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Fixed by Davies' PR, which I ba

[jira] [Commented] (SPARK-3828) Spark returns inconsistent results when building with different Hadoop version

2014-10-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162439#comment-14162439 ] Xiangrui Meng commented on SPARK-3828: -- I re-opened this because it may be a serious

[jira] [Reopened] (SPARK-3828) Spark returns inconsistent results when building with different Hadoop version

2014-10-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reopened SPARK-3828: -- > Spark returns inconsistent results when building with different Hadoop > version > -

[jira] [Closed] (SPARK-3825) Log more information when unrolling a block fails

2014-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3825. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 > Log more information when unrolling

[jira] [Commented] (SPARK-3816) Add configureOutputJobPropertiesForStorageHandler to JobConf in SparkHadoopWriter

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162420#comment-14162420 ] Apache Spark commented on SPARK-3816: - User 'alexliu68' has created a pull request for

[jira] [Commented] (SPARK-3790) CosineSimilarity via DIMSUM example

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162415#comment-14162415 ] Apache Spark commented on SPARK-3790: - User 'rezazadeh' has created a pull request for

[jira] [Commented] (SPARK-2759) The ability to read binary files into Spark

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162403#comment-14162403 ] Apache Spark commented on SPARK-2759: - User 'kmader' has created a pull request for th

[jira] [Commented] (SPARK-3166) Custom serialisers can't be shipped in application jars

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162409#comment-14162409 ] Apache Spark commented on SPARK-3166: - User 'GrahamDennis' has created a pull request

[jira] [Commented] (SPARK-2489) Unsupported parquet datatype optional fixed_len_byte_array

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162407#comment-14162407 ] Apache Spark commented on SPARK-2489: - User 'joesu' has created a pull request for thi

[jira] [Commented] (SPARK-3580) Add Consistent Method To Get Number of RDD Partitions Across Different Languages

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162413#comment-14162413 ] Apache Spark commented on SPARK-3580: - User 'patmcdonough' has created a pull request

[jira] [Commented] (SPARK-2016) rdd in-memory storage UI becomes unresponsive when the number of RDD partitions is large

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162405#comment-14162405 ] Apache Spark commented on SPARK-2016: - User 'carlosfuertes' has created a pull request

[jira] [Commented] (SPARK-3812) Adapt maven build to publish effective pom.

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162417#comment-14162417 ] Apache Spark commented on SPARK-3812: - User 'ScrapCodes' has created a pull request fo

[jira] [Commented] (SPARK-3338) Respect user setting of spark.submit.pyFiles

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162411#comment-14162411 ] Apache Spark commented on SPARK-3338: - User 'andrewor14' has created a pull request fo

[jira] [Commented] (SPARK-3251) Clarify learning interfaces

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162410#comment-14162410 ] Apache Spark commented on SPARK-3251: - User 'BigCrunsh' has created a pull request for

[jira] [Commented] (SPARK-2017) web ui stage page becomes unresponsive when the number of tasks is large

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162404#comment-14162404 ] Apache Spark commented on SPARK-2017: - User 'carlosfuertes' has created a pull request

[jira] [Commented] (SPARK-2803) add Kafka stream feature for fetch messages from specified starting offset position

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162402#comment-14162402 ] Apache Spark commented on SPARK-2803: - User 'pengyanhong' has created a pull request f

[jira] [Commented] (SPARK-2805) update akka to version 2.3

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162406#comment-14162406 ] Apache Spark commented on SPARK-2805: - User 'avati' has created a pull request for thi

[jira] [Commented] (SPARK-3809) make HiveThriftServer2Suite work correctly

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162418#comment-14162418 ] Apache Spark commented on SPARK-3809: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-3505) Augmenting SparkStreaming updateStateByKey API with timestamp

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162412#comment-14162412 ] Apache Spark commented on SPARK-3505: - User 'xiliu82' has created a pull request for t

[jira] [Commented] (SPARK-2960) Spark executables fail to start via symlinks

2014-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162408#comment-14162408 ] Apache Spark commented on SPARK-2960: - User 'roji' has created a pull request for this

[jira] [Created] (SPARK-3835) Spark applications that are killed should show up as "KILLED" or "CANCELLED" in the Spark UI

2014-10-07 Thread Matt Cheah (JIRA)
Matt Cheah created SPARK-3835: - Summary: Spark applications that are killed should show up as "KILLED" or "CANCELLED" in the Spark UI Key: SPARK-3835 URL: https://issues.apache.org/jira/browse/SPARK-3835

[jira] [Created] (SPARK-3834) Backticks not correctly handled in subquery aliases

2014-10-07 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-3834: --- Summary: Backticks not correctly handled in subquery aliases Key: SPARK-3834 URL: https://issues.apache.org/jira/browse/SPARK-3834 Project: Spark Issue

[jira] [Resolved] (SPARK-2582) Make Block Manager Master pluggable

2014-10-07 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2582. Resolution: Won't Fix I closed this PR a long time ago - but didn't close the asscicated JIR

[jira] [Comment Edited] (SPARK-3828) Spark returns inconsistent results when building with different Hadoop version

2014-10-07 Thread Liquan Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162374#comment-14162374 ] Liquan Pei edited comment on SPARK-3828 at 10/7/14 7:33 PM: It

[jira] [Commented] (SPARK-3828) Spark returns inconsistent results when building with different Hadoop version

2014-10-07 Thread Liquan Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162374#comment-14162374 ] Liquan Pei commented on SPARK-3828: --- It seems that this is a bug in LineRecordReader. Fo

[jira] [Resolved] (SPARK-3765) Add test information to sbt build docs

2014-10-07 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3765. Resolution: Fixed Assignee: wangfei This was resolved by: https://github.com/apache/sp

[jira] [Updated] (SPARK-3765) Add test information to sbt build docs

2014-10-07 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3765: --- Fix Version/s: 1.2.0 > Add test information to sbt build docs > --

[jira] [Commented] (SPARK-3797) Run the shuffle service inside the YARN NodeManager as an AuxiliaryService

2014-10-07 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162351#comment-14162351 ] Patrick Wendell commented on SPARK-3797: For the dependencies issue - the plan is

[jira] [Updated] (SPARK-3731) RDD caching stops working in pyspark after some time

2014-10-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3731: -- Affects Version/s: 1.0.2 > RDD caching stops working in pyspark after some time > --

[jira] [Updated] (SPARK-3731) RDD caching stops working in pyspark after some time

2014-10-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3731: -- Target Version/s: 1.1.1, 1.2.0 (was: 1.1.1, 1.2.0, 1.0.3) > RDD caching stops working in pyspark after

[jira] [Updated] (SPARK-3731) RDD caching stops working in pyspark after some time

2014-10-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3731: -- Affects Version/s: (was: 1.0.2) 1.2.0 > RDD caching stops working in pyspark

[jira] [Resolved] (SPARK-3762) clear all SparkEnv references after stop

2014-10-07 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-3762. -- Resolution: Fixed Fix Version/s: 1.2.0 > clear all SparkEnv references after stop > -

[jira] [Closed] (SPARK-3732) Yarn Client: Add option to NOT System.exit() at end of main()

2014-10-07 Thread Sotos Matzanas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sotos Matzanas closed SPARK-3732. - Resolution: Won't Fix > Yarn Client: Add option to NOT System.exit() at end of main() > --

[jira] [Commented] (SPARK-3808) PySpark fails to start in Windows

2014-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162310#comment-14162310 ] Andrew Or commented on SPARK-3808: -- Hey [~tsudukim] can you verify that pyspark, spark-sh

[jira] [Closed] (SPARK-3761) Class anonfun$1 not found exception / sbt 13.x / Scala 2.10.4

2014-10-07 Thread Igor Tkachenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Tkachenko closed SPARK-3761. - Resolution: Fixed > Class anonfun$1 not found exception / sbt 13.x / Scala 2.10.4 > --

[jira] [Closed] (SPARK-3808) PySpark fails to start in Windows

2014-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3808. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Masayoshi TSUZUKI Target Versio

[jira] [Commented] (SPARK-3761) Class anonfun$1 not found exception / sbt 13.x / Scala 2.10.4

2014-10-07 Thread Igor Tkachenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162307#comment-14162307 ] Igor Tkachenko commented on SPARK-3761: --- After I've added line sc.addJar("") it work

[jira] [Updated] (SPARK-3808) PySpark fails to start in Windows

2014-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3808: - Affects Version/s: (was: 1.1.0) 1.2.0 > PySpark fails to start in Windows > ---

[jira] [Resolved] (SPARK-3297) [Spark SQL][UI] SchemaRDD toString with many columns messes up Storage tab display

2014-10-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3297. --- Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Assignee: Hossein Falak

[jira] [Resolved] (SPARK-2915) Storage summary table UI glitch when using sparkSQL

2014-10-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2915. --- Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Assignee: Hossein Falak

[jira] [Resolved] (SPARK-3827) Very long RDD names are not rendered properly in web UI

2014-10-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3827. --- Resolution: Fixed Fix Version/s: 1.1.1 Issue resolved by pull request 2687 [https://github.com/

[jira] [Commented] (SPARK-3797) Run the shuffle service inside the YARN NodeManager as an AuxiliaryService

2014-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162254#comment-14162254 ] Andrew Or commented on SPARK-3797: -- Thanks for detailing the considerations Sandy. I agre

[jira] [Commented] (SPARK-3819) Jenkins should compile Spark against multiple versions of Hadoop

2014-10-07 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162195#comment-14162195 ] Matt Cheah commented on SPARK-3819: --- Can you elaborate as to why it is not feasible to b

[jira] [Updated] (SPARK-1297) Upgrade HBase dependency to 0.98.0

2014-10-07 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-1297: -- Attachment: spark-1297-v6.txt Patch v6 uses 0.98.5 hbase release. > Upgrade HBase dependency to 0.98.0 > --

[jira] [Commented] (SPARK-3434) Distributed block matrix

2014-10-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162156#comment-14162156 ] Xiangrui Meng commented on SPARK-3434: -- [~shivaram] and [~ConcreteVitamin] Any update

  1   2   >