[jira] [Created] (SPARK-1825) Windows Spark fails to work with Linux YARN

Taeyun Kim created SPARK-1825: - Summary: Windows Spark fails to work with Linux YARN Key: SPARK-1825 URL: https://issues.apache.org/jira/browse/SPARK-1825 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-1048) Create spark-site.xml or spark-site.yaml for configuration

[ https://issues.apache.org/jira/browse/SPARK-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shengzhe Yao resolved SPARK-1048. - Resolution: Duplicate Patrick submit a patch which introduces spark-defaults.conf to set system

[jira] [Comment Edited] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13991556#comment-13991556 ] Guoqiang Li edited comment on SPARK-1712 at 5/14/14 6:03 AM: -

[jira] [Updated] (SPARK-1823) ExternalAppendOnlyMap can still OOM if one key is very large

[ https://issues.apache.org/jira/browse/SPARK-1823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1823: - Fix Version/s: 1.1.0 ExternalAppendOnlyMap can still OOM if one key is very large

[jira] [Created] (SPARK-1828) Created forked version of hive-exec that doesn't bundle other dependencies

Patrick Wendell created SPARK-1828: -- Summary: Created forked version of hive-exec that doesn't bundle other dependencies Key: SPARK-1828 URL: https://issues.apache.org/jira/browse/SPARK-1828

[jira] [Created] (SPARK-1756) Add missing description to spark-env.sh.template

Guoqiang Li created SPARK-1756: -- Summary: Add missing description to spark-env.sh.template Key: SPARK-1756 URL: https://issues.apache.org/jira/browse/SPARK-1756 Project: Spark Issue Type:

[jira] [Created] (SPARK-1759) sbt/sbt package fail cause by directory

Jian Pan created SPARK-1759: --- Summary: sbt/sbt package fail cause by directory Key: SPARK-1759 URL: https://issues.apache.org/jira/browse/SPARK-1759 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-1765) Modify a typo in monitoring.md

Kousuke Saruta created SPARK-1765: - Summary: Modify a typo in monitoring.md Key: SPARK-1765 URL: https://issues.apache.org/jira/browse/SPARK-1765 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-1789) Multiple versions of Netty dependencies cause FlumeStreamSuite failure

[ https://issues.apache.org/jira/browse/SPARK-1789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13997317#comment-13997317 ] Sean Owen commented on SPARK-1789: -- I don't have any info either way on that. Later is

[jira] [Commented] (SPARK-1827) LICENSE and NOTICE files need a refresh to contain transitive dependency info

[ https://issues.apache.org/jira/browse/SPARK-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13997458#comment-13997458 ] Sean Owen commented on SPARK-1827: -- LICENSE and NOTICE policy is explained here:

[jira] [Resolved] (SPARK-1801) Open up some private APIs related to creating new RDDs for developers

[ https://issues.apache.org/jira/browse/SPARK-1801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson resolved SPARK-1801. --- Resolution: Fixed https://github.com/apache/spark/pull/764 Open up some private APIs

[jira] [Created] (SPARK-1830) Deploy failover, Make Persistence engine and LeaderAgent Pluggable.

Prashant Sharma created SPARK-1830: -- Summary: Deploy failover, Make Persistence engine and LeaderAgent Pluggable. Key: SPARK-1830 URL: https://issues.apache.org/jira/browse/SPARK-1830 Project: Spark

[jira] [Created] (SPARK-1827) LICENSE and NOTICE files need a refresh to contain transitive dependency info

Sean Owen created SPARK-1827: Summary: LICENSE and NOTICE files need a refresh to contain transitive dependency info Key: SPARK-1827 URL: https://issues.apache.org/jira/browse/SPARK-1827 Project: Spark

[jira] [Closed] (SPARK-1757) Support saving null primitives with .saveAsParquetFile()

[ https://issues.apache.org/jira/browse/SPARK-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash closed SPARK-1757. - Support saving null primitives with .saveAsParquetFile()

[jira] [Resolved] (SPARK-1829) Sub-second durations shouldn't round to 0 s

[ https://issues.apache.org/jira/browse/SPARK-1829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash resolved SPARK-1829. --- Resolution: Fixed Didn't make it into v1.0.0-rc6 but is on branch-1.0 so a future rc would likely

[jira] [Created] (SPARK-1833) Have an empty SparkContext constructor instead of relying on new SparkContext(new SparkConf())

Patrick Wendell created SPARK-1833: -- Summary: Have an empty SparkContext constructor instead of relying on new SparkContext(new SparkConf()) Key: SPARK-1833 URL: https://issues.apache.org/jira/browse/SPARK-1833

[jira] [Commented] (SPARK-1433) Upgrade Mesos dependency to 0.17.0

[ https://issues.apache.org/jira/browse/SPARK-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993999#comment-13993999 ] Bernardo Gomez Palacio commented on SPARK-1433: --- I agree with [~tstclair],

[jira] [Commented] (SPARK-1644) The org.datanucleus:* should not be packaged into spark-assembly-*.jar

[ https://issues.apache.org/jira/browse/SPARK-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13994131#comment-13994131 ] Guoqiang Li commented on SPARK-1644: Why {{assignee to me}} button can not find?

[jira] [Closed] (SPARK-1711) We should not use two build tools at the same time, should remove one of them

[ https://issues.apache.org/jira/browse/SPARK-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li closed SPARK-1711. -- Resolution: Duplicate We should not use two build tools at the same time, should remove one of them

[jira] [Commented] (SPARK-571) Forbid return statements when cleaning closures

[ https://issues.apache.org/jira/browse/SPARK-571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993919#comment-13993919 ] William Benton commented on SPARK-571: -- I have a patch for this and will submit a PR

[jira] [Created] (SPARK-1766) Move reduceByKey definitions next to each other in PairRDDFunctions

Sandy Ryza created SPARK-1766: - Summary: Move reduceByKey definitions next to each other in PairRDDFunctions Key: SPARK-1766 URL: https://issues.apache.org/jira/browse/SPARK-1766 Project: Spark

[jira] [Created] (SPARK-1783) Title contains html code in MLlib guide

Xiangrui Meng created SPARK-1783: Summary: Title contains html code in MLlib guide Key: SPARK-1783 URL: https://issues.apache.org/jira/browse/SPARK-1783 Project: Spark Issue Type:

[jira] [Updated] (SPARK-1806) Upgrade to Mesos 0.18.1 with Shaded Protobuf

[ https://issues.apache.org/jira/browse/SPARK-1806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1806: --- Issue Type: Dependency upgrade (was: Improvement) Upgrade to Mesos 0.18.1 with Shaded

[jira] [Updated] (SPARK-1770) repartition and coalesce(shuffle=true) put objects with the same key in the same bucket

[ https://issues.apache.org/jira/browse/SPARK-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1770: - Fix Version/s: 1.0.0 repartition and coalesce(shuffle=true) put objects with the same key in

[jira] [Updated] (SPARK-1788) Upgrade Parquet to 1.4.3

[ https://issues.apache.org/jira/browse/SPARK-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-1788: Component/s: SQL Upgrade Parquet to 1.4.3

[jira] [Commented] (SPARK-1784) Add a partitioner which partitions an RDD with each partition having specified # of keys

[ https://issues.apache.org/jira/browse/SPARK-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13994047#comment-13994047 ] Syed A. Hashmi commented on SPARK-1784: --- [~matei]: Can you please assign this JIRA

[jira] [Commented] (SPARK-1785) Streaming requires receivers to be serializable

[ https://issues.apache.org/jira/browse/SPARK-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13994100#comment-13994100 ] Hari Shreedharan commented on SPARK-1785: - The problem with sending only the class

[jira] [Resolved] (SPARK-1731) PySpark broadcast values with custom classes won't depickle properly

[ https://issues.apache.org/jira/browse/SPARK-1731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1731. Resolution: Fixed Fix Version/s: 1.0.0 Issue resolved by pull request 656

[jira] [Updated] (SPARK-1768) History Server enhancements

[ https://issues.apache.org/jira/browse/SPARK-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1768: - Assignee: Marcelo Vanzin History Server enhancements ---

[jira] [Updated] (SPARK-1823) ExternalAppendOnlyMap can still OOM if one key is very large

[ https://issues.apache.org/jira/browse/SPARK-1823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1823: - Description: If the values for one key do not collectively fit into memory, then the map will still OOM

[jira] [Created] (SPARK-1782) svd for sparse matrix using ARPACK

Li Pu created SPARK-1782: Summary: svd for sparse matrix using ARPACK Key: SPARK-1782 URL: https://issues.apache.org/jira/browse/SPARK-1782 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-937) Executors that exit cleanly should not have KILLED status

[ https://issues.apache.org/jira/browse/SPARK-937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13992631#comment-13992631 ] Kan Zhang commented on SPARK-937: - Hi Aaron, are you still working on this one? If not,

[jira] [Commented] (SPARK-1807) Modify SPARK_EXECUTOR_URI to allow for script execution in Mesos.

[ https://issues.apache.org/jira/browse/SPARK-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13995469#comment-13995469 ] William Benton commented on SPARK-1807: --- I'll be working on some related things this

[jira] [Created] (SPARK-1761) Add broadcast information on SparkUI storage tab

Andrew Or created SPARK-1761: Summary: Add broadcast information on SparkUI storage tab Key: SPARK-1761 URL: https://issues.apache.org/jira/browse/SPARK-1761 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-1769) Executor loss can cause race condition in Pool

[ https://issues.apache.org/jira/browse/SPARK-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson resolved SPARK-1769. --- Resolution: Fixed Executor loss can cause race condition in Pool

[jira] [Created] (SPARK-1834) NoSuchMethodError when invoking JavaPairRDD.reduce() in Java

John Snodgrass created SPARK-1834: - Summary: NoSuchMethodError when invoking JavaPairRDD.reduce() in Java Key: SPARK-1834 URL: https://issues.apache.org/jira/browse/SPARK-1834 Project: Spark

[jira] [Updated] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1755: --- Assignee: Andrew Or Spark-submit --name does not resolve to application name on YARN

[jira] [Resolved] (SPARK-1616) input file not found issue

[ https://issues.apache.org/jira/browse/SPARK-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Hamstra resolved SPARK-1616. - Resolution: Not a Problem input file not found issue ---

[jira] [Created] (SPARK-1788) Upgrade Parquet to 1.4.3

Michael Armbrust created SPARK-1788: --- Summary: Upgrade Parquet to 1.4.3 Key: SPARK-1788 URL: https://issues.apache.org/jira/browse/SPARK-1788 Project: Spark Issue Type: Dependency upgrade

[jira] [Created] (SPARK-1772) Spark executors do not successfully die on OOM

Aaron Davidson created SPARK-1772: - Summary: Spark executors do not successfully die on OOM Key: SPARK-1772 URL: https://issues.apache.org/jira/browse/SPARK-1772 Project: Spark Issue Type:

[jira] [Updated] (SPARK-1787) Build failure on JDK8 :: SBT fails to load build configuration file

[ https://issues.apache.org/jira/browse/SPARK-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Gomes updated SPARK-1787: - Description: SBT fails to build under JDK8. Please find steps to reproduce the error below:

[jira] [Commented] (SPARK-1675) Make clear whether computePrincipalComponents centers data

[ https://issues.apache.org/jira/browse/SPARK-1675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13998103#comment-13998103 ] Xiangrui Meng commented on SPARK-1675: -- Centering in PCA should be the standard

[jira] [Resolved] (SPARK-1644) The org.datanucleus:* should not be packaged into spark-assembly-*.jar

[ https://issues.apache.org/jira/browse/SPARK-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1644. Resolution: Fixed Issue resolved by pull request 688

[jira] [Commented] (SPARK-1768) History Server enhancements

[ https://issues.apache.org/jira/browse/SPARK-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13995375#comment-13995375 ] Marcelo Vanzin commented on SPARK-1768: --- https://github.com/apache/spark/pull/718

[jira] [Updated] (SPARK-1828) Created forked version of hive-exec that doesn't bundle other dependencies

[ https://issues.apache.org/jira/browse/SPARK-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1828: --- Fix Version/s: 1.0.0 Created forked version of hive-exec that doesn't bundle other

[jira] [Resolved] (SPARK-1828) Created forked version of hive-exec that doesn't bundle other dependencies

[ https://issues.apache.org/jira/browse/SPARK-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1828. Resolution: Fixed Issue resolved by pull request 767

[jira] [Updated] (SPARK-1784) Add a partitioner which partitions an RDD with each partition having equal # of keys

[ https://issues.apache.org/jira/browse/SPARK-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Syed A. Hashmi updated SPARK-1784: -- Summary: Add a partitioner which partitions an RDD with each partition having equal # of keys

[jira] [Commented] (SPARK-1817) RDD zip erroneous when partitions do not divide RDD count

[ https://issues.apache.org/jira/browse/SPARK-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13998153#comment-13998153 ] Kan Zhang commented on SPARK-1817: -- I opened SPARK-1837 as a specific fix for the error

[jira] [Created] (SPARK-1784) Add a partitioner which partitions an RDD with each partition having equal keys

Syed A. Hashmi created SPARK-1784: - Summary: Add a partitioner which partitions an RDD with each partition having equal keys Key: SPARK-1784 URL: https://issues.apache.org/jira/browse/SPARK-1784

[jira] [Resolved] (SPARK-1745) TaskContext.interrupted should probably not be a constructor argument

[ https://issues.apache.org/jira/browse/SPARK-1745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson resolved SPARK-1745. --- Resolution: Fixed Assignee: Andrew Or TaskContext.interrupted should probably not be

[jira] [Resolved] (SPARK-1668) Add implicit preference as an option to examples/MovieLensALS

[ https://issues.apache.org/jira/browse/SPARK-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-1668. -- Resolution: Implemented Fix Version/s: 1.0.0 Add implicit preference as an option to

[jira] [Commented] (SPARK-1779) Warning when spark.storage.memoryFraction is not between 0 and 1

[ https://issues.apache.org/jira/browse/SPARK-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13994096#comment-13994096 ] wangfei commented on SPARK-1779: i have fix it by throw an exception

[jira] [Comment Edited] (SPARK-1433) Upgrade Mesos dependency to 0.17.0

[ https://issues.apache.org/jira/browse/SPARK-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993999#comment-13993999 ] Bernardo Gomez Palacio edited comment on SPARK-1433 at 5/10/14 12:16 AM:

[jira] [Created] (SPARK-1787) Build failure on JDK8 :: SBT fails to load build configuration file

Richard Gomes created SPARK-1787: Summary: Build failure on JDK8 :: SBT fails to load build configuration file Key: SPARK-1787 URL: https://issues.apache.org/jira/browse/SPARK-1787 Project: Spark

[jira] [Updated] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

[ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1755: - Description: In YARN client mode, --name is ignored because the deploy mode is client, and the name is

[jira] [Commented] (SPARK-1760) mvn -Dsuites=* test throw an ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13992837#comment-13992837 ] Sean Owen commented on SPARK-1760: -- Yeah I think you would have to run this from the