[jira] [Created] (SPARK-2089) With YARN, preferredNodeLocalityData isn't honored

Sandy Ryza created SPARK-2089: - Summary: With YARN, preferredNodeLocalityData isn't honored Key: SPARK-2089 URL: https://issues.apache.org/jira/browse/SPARK-2089 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14013378#comment-14013378 ] Sandy Ryza commented on SPARK-1518: --- bq. I don't think there's a single release I'd call

[jira] [Created] (SPARK-1954) Make it easier to get Spark on YARN code to compile in IntelliJ

Sandy Ryza created SPARK-1954: - Summary: Make it easier to get Spark on YARN code to compile in IntelliJ Key: SPARK-1954 URL: https://issues.apache.org/jira/browse/SPARK-1954 Project: Spark

[jira] [Commented] (SPARK-1954) Make it easier to get Spark on YARN code to compile in IntelliJ

[ https://issues.apache.org/jira/browse/SPARK-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14012027#comment-14012027 ] Sandy Ryza commented on SPARK-1954: --- Cool. Your suggestion does appear to work. Make

[jira] [Commented] (SPARK-1898) In deploy.yarn.Client, use YarnClient rather than YarnClientImpl

[ https://issues.apache.org/jira/browse/SPARK-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14006494#comment-14006494 ] Sandy Ryza commented on SPARK-1898: --- The change only applies to Spark's stable/2.2+

[jira] [Commented] (SPARK-1813) Add a utility to SparkConf that makes using Kryo really easy

[ https://issues.apache.org/jira/browse/SPARK-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13998557#comment-13998557 ] Sandy Ryza commented on SPARK-1813: --- https://github.com/apache/spark/pull/789 is what I

[jira] [Created] (SPARK-1851) Upgrade Avro dependency to 1.7.6 so Spark can read Avro files

Sandy Ryza created SPARK-1851: - Summary: Upgrade Avro dependency to 1.7.6 so Spark can read Avro files Key: SPARK-1851 URL: https://issues.apache.org/jira/browse/SPARK-1851 Project: Spark Issue

[jira] [Created] (SPARK-1767) Prefer HDFS-cached replicas when scheduling data-local tasks

Sandy Ryza created SPARK-1767: - Summary: Prefer HDFS-cached replicas when scheduling data-local tasks Key: SPARK-1767 URL: https://issues.apache.org/jira/browse/SPARK-1767 Project: Spark Issue

[jira] [Commented] (SPARK-1767) Prefer HDFS-cached replicas when scheduling data-local tasks

[ https://issues.apache.org/jira/browse/SPARK-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993909#comment-13993909 ] Sandy Ryza commented on SPARK-1767: --- Currently, RDDs only support a single level of

[jira] [Created] (SPARK-1766) Move reduceByKey definitions next to each other in PairRDDFunctions

Sandy Ryza created SPARK-1766: - Summary: Move reduceByKey definitions next to each other in PairRDDFunctions Key: SPARK-1766 URL: https://issues.apache.org/jira/browse/SPARK-1766 Project: Spark

[jira] [Created] (SPARK-1813) Add a utility to SparkConf that makes using Kryo really easy

Sandy Ryza created SPARK-1813: - Summary: Add a utility to SparkConf that makes using Kryo really easy Key: SPARK-1813 URL: https://issues.apache.org/jira/browse/SPARK-1813 Project: Spark Issue

[jira] [Updated] (SPARK-1813) Add a utility to SparkConf that makes using Kryo really easy

[ https://issues.apache.org/jira/browse/SPARK-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-1813: -- Description: It would be nice to have a method in SparkConf that makes it really easy to turn on Kryo

[jira] [Updated] (SPARK-1813) Add a utility to SparkConf that makes using Kryo really easy

[ https://issues.apache.org/jira/browse/SPARK-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-1813: -- Description: It would be nice to have a method in SparkConf that makes it really easy to turn on Kryo

[jira] [Created] (SPARK-1815) SparkContext's constructor that only takes a SparkConf shouldn't be a DeveloperApi

Sandy Ryza created SPARK-1815: - Summary: SparkContext's constructor that only takes a SparkConf shouldn't be a DeveloperApi Key: SPARK-1815 URL: https://issues.apache.org/jira/browse/SPARK-1815 Project:

[jira] [Commented] (SPARK-554) Add foldByKey and aggregateByKey

[ https://issues.apache.org/jira/browse/SPARK-554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993294#comment-13993294 ] Sandy Ryza commented on SPARK-554: -- https://github.com/apache/spark/pull/705 Add

[jira] [Commented] (SPARK-1713) YarnAllocationHandler starts a thread for every executor it runs

[ https://issues.apache.org/jira/browse/SPARK-1713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990808#comment-13990808 ] Sandy Ryza commented on SPARK-1713: --- https://github.com/apache/spark/pull/663

[jira] [Created] (SPARK-1728) JavaRDDLike.mapPartitionsWithIndex requires ClassTag

Sandy Ryza created SPARK-1728: - Summary: JavaRDDLike.mapPartitionsWithIndex requires ClassTag Key: SPARK-1728 URL: https://issues.apache.org/jira/browse/SPARK-1728 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-1714) Take advantage of AMRMClient APIs to simplify logic in YarnAllocationHandler

[ https://issues.apache.org/jira/browse/SPARK-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990126#comment-13990126 ] Sandy Ryza commented on SPARK-1714: --- https://github.com/apache/spark/pull/655 Take

[jira] [Commented] (SPARK-1728) JavaRDDLike.mapPartitionsWithIndex requires ClassTag

[ https://issues.apache.org/jira/browse/SPARK-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990128#comment-13990128 ] Sandy Ryza commented on SPARK-1728: --- https://github.com/apache/spark/pull/657

[jira] [Commented] (SPARK-1707) Remove 3 second sleep before starting app on YARN

[ https://issues.apache.org/jira/browse/SPARK-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989091#comment-13989091 ] Sandy Ryza commented on SPARK-1707: --- https://github.com/apache/spark/pull/634 Remove 3

[jira] [Created] (SPARK-1713) YarnAllocationHandler starts a thread for every executor it runs

Sandy Ryza created SPARK-1713: - Summary: YarnAllocationHandler starts a thread for every executor it runs Key: SPARK-1713 URL: https://issues.apache.org/jira/browse/SPARK-1713 Project: Spark

[jira] [Created] (SPARK-1707) Remove 3 second sleep before starting app on YARN

Sandy Ryza created SPARK-1707: - Summary: Remove 3 second sleep before starting app on YARN Key: SPARK-1707 URL: https://issues.apache.org/jira/browse/SPARK-1707 Project: Spark Issue Type:

[jira] [Commented] (SPARK-1707) Remove 3 second sleep before starting app on YARN

[ https://issues.apache.org/jira/browse/SPARK-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988854#comment-13988854 ] Sandy Ryza commented on SPARK-1707: --- [~tgraves] [~mrid...@yahoo-inc.com] any thoughts on

[jira] [Commented] (SPARK-1492) running-on-yarn doc should use spark-submit script for examples

[ https://issues.apache.org/jira/browse/SPARK-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985750#comment-13985750 ] Sandy Ryza commented on SPARK-1492: --- https://github.com/apache/spark/pull/601

[jira] [Assigned] (SPARK-1492) running-on-yarn doc should use spark-submit script for examples

[ https://issues.apache.org/jira/browse/SPARK-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1492: - Assignee: Sandy Ryza running-on-yarn doc should use spark-submit script for examples

[jira] [Created] (SPARK-1675) Make clear whether computePrincipalComponents centers data

Sandy Ryza created SPARK-1675: - Summary: Make clear whether computePrincipalComponents centers data Key: SPARK-1675 URL: https://issues.apache.org/jira/browse/SPARK-1675 Project: Spark Issue

[jira] [Updated] (SPARK-1632) Avoid boxing in ExternalAppendOnlyMap compares

[ https://issues.apache.org/jira/browse/SPARK-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-1632: -- Summary: Avoid boxing in ExternalAppendOnlyMap compares (was: Avoid boxing in

[jira] [Created] (SPARK-1640) In yarn-client mode, pass preferred node locations to AM

Sandy Ryza created SPARK-1640: - Summary: In yarn-client mode, pass preferred node locations to AM Key: SPARK-1640 URL: https://issues.apache.org/jira/browse/SPARK-1640 Project: Spark Issue Type:

[jira] [Updated] (SPARK-1639) Some tidying of Spark on YARN ApplicationMaster and ExecutorLauncher

[ https://issues.apache.org/jira/browse/SPARK-1639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-1639: -- Summary: Some tidying of Spark on YARN ApplicationMaster and ExecutorLauncher (was: Some tidying of

[jira] [Created] (SPARK-1584) Upgrade Flume dependency to 1.4.0

Sandy Ryza created SPARK-1584: - Summary: Upgrade Flume dependency to 1.4.0 Key: SPARK-1584 URL: https://issues.apache.org/jira/browse/SPARK-1584 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-1584) Upgrade Flume dependency to 1.4.0

[ https://issues.apache.org/jira/browse/SPARK-1584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13977901#comment-13977901 ] Sandy Ryza commented on SPARK-1584: --- This will allow us to take advantage of features

[jira] [Commented] (SPARK-1576) Passing of JAVA_OPTS to YARN on command line

[ https://issues.apache.org/jira/browse/SPARK-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13977918#comment-13977918 ] Sandy Ryza commented on SPARK-1576: --- Thanks, mind posting a link to it on this JIRA?

[jira] [Commented] (SPARK-1576) Passing of JAVA_OPTS to YARN on command line

[ https://issues.apache.org/jira/browse/SPARK-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13977754#comment-13977754 ] Sandy Ryza commented on SPARK-1576: --- Hi [~nravi], Spark patches should be submitted as

[jira] [Resolved] (SPARK-1541) sortByKey requires all data to fit in memory

[ https://issues.apache.org/jira/browse/SPARK-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved SPARK-1541. --- Resolution: Duplicate Oops already exists sortByKey requires all data to fit in memory

[jira] [Assigned] (SPARK-1056) Header comment in Executor incorrectly implies it's not used for YARN

[ https://issues.apache.org/jira/browse/SPARK-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1056: - Assignee: Sandy Ryza (was: Sandy Pérez González) Header comment in Executor incorrectly

[jira] [Assigned] (SPARK-1033) Ask for cores in Yarn container requests

[ https://issues.apache.org/jira/browse/SPARK-1033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1033: - Assignee: Sandy Ryza (was: Sandy Pérez González) Ask for cores in Yarn container requests

[jira] [Assigned] (SPARK-1211) In ApplicationMaster, set spark.master system property to yarn-cluster

[ https://issues.apache.org/jira/browse/SPARK-1211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1211: - Assignee: Sandy Ryza (was: Sandy Pérez González) In ApplicationMaster, set spark.master system

[jira] [Assigned] (SPARK-1197) Rename yarn-standalone and fix up docs for running on YARN

[ https://issues.apache.org/jira/browse/SPARK-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1197: - Assignee: Sandy Ryza (was: Sandy Pérez González) Rename yarn-standalone and fix up docs for

[jira] [Assigned] (SPARK-1216) Add a OneHotEncoder for handling categorical features

[ https://issues.apache.org/jira/browse/SPARK-1216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1216: - Assignee: Sandy Ryza (was: Sandy Pérez González) Add a OneHotEncoder for handling categorical

[jira] [Created] (SPARK-1375) In spark-submit script, where yarn-standalone is used, accept yarn-cluster as well

Sandy Ryza created SPARK-1375: - Summary: In spark-submit script, where yarn-standalone is used, accept yarn-cluster as well Key: SPARK-1375 URL: https://issues.apache.org/jira/browse/SPARK-1375 Project:

<    1   2   3   4   5