[jira] [Updated] (SPARK-1403) java.lang.ClassNotFoundException - spark on mesos

[ https://issues.apache.org/jira/browse/SPARK-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1403: --- Priority: Critical (was: Major) > java.lang.ClassNotFoundException - spark on mesos > -

[jira] [Updated] (SPARK-1403) java.lang.ClassNotFoundException - spark on mesos

[ https://issues.apache.org/jira/browse/SPARK-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1403: --- Priority: Blocker (was: Critical) > java.lang.ClassNotFoundException - spark on mesos > ---

[jira] [Updated] (SPARK-922) Update Spark AMI to Python 2.7

[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-922: -- Issue Type: Task (was: Improvement) > Update Spark AMI to Python 2.7 >

[jira] [Resolved] (SPARK-1305) Support persisting RDD's directly to Tachyon

[ https://issues.apache.org/jira/browse/SPARK-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1305. Resolution: Fixed > Support persisting RDD's directly to Tachyon >

[jira] [Commented] (SPARK-1402) 3 more compression algorithms for in-memory columnar storage

[ https://issues.apache.org/jira/browse/SPARK-1402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13960937#comment-13960937 ] Cheng Lian commented on SPARK-1402: --- Corresponding PR: https://github.com/apache/spark/p

[jira] [Updated] (SPARK-1402) 3 more compression algorithms for in-memory columnar storage

[ https://issues.apache.org/jira/browse/SPARK-1402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-1402: -- Description: This is a followup of SPARK-1373: Compression for In-Memory Columnar storage 3 more compr

[jira] [Updated] (SPARK-1419) Apache parent POM to version 14

[ https://issues.apache.org/jira/browse/SPARK-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1419: --- Issue Type: Dependency upgrade (was: Bug) > Apache parent POM to version 14 > --

[jira] [Resolved] (SPARK-1419) Apache parent POM to version 14

[ https://issues.apache.org/jira/browse/SPARK-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1419. Resolution: Fixed Fix Version/s: 1.0.0 > Apache parent POM to version 14 > -

[jira] [Commented] (SPARK-1415) Add a minSplits parameter to wholeTextFiles

[ https://issues.apache.org/jira/browse/SPARK-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13960908#comment-13960908 ] Xusen Yin commented on SPARK-1415: -- Hi Matei, I just looked around in those Hadoop APIs.

[jira] [Assigned] (SPARK-1216) Add a OneHotEncoder for handling categorical features

[ https://issues.apache.org/jira/browse/SPARK-1216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1216: - Assignee: Sandy Ryza (was: Sandy Pérez González) > Add a OneHotEncoder for handling categorical

[jira] [Resolved] (SPARK-1414) Python API for SparkContext.wholeTextFiles

[ https://issues.apache.org/jira/browse/SPARK-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1414. -- Resolution: Fixed > Python API for SparkContext.wholeTextFiles > --

[jira] [Assigned] (SPARK-1415) Add a minSplits parameter to wholeTextFiles

[ https://issues.apache.org/jira/browse/SPARK-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin reassigned SPARK-1415: Assignee: Xusen Yin > Add a minSplits parameter to wholeTextFiles > ---

[jira] [Resolved] (SPARK-1198) Allow pipes tasks to run in different sub-directories

[ https://issues.apache.org/jira/browse/SPARK-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1198. -- Resolution: Fixed Fix Version/s: 1.0.0 > Allow pipes tasks to run in different sub-direc

[jira] [Created] (SPARK-1419) Apache parent POM to version 14

Mark Hamstra created SPARK-1419: --- Summary: Apache parent POM to version 14 Key: SPARK-1419 URL: https://issues.apache.org/jira/browse/SPARK-1419 Project: Spark Issue Type: Bug Compone

[jira] [Commented] (SPARK-1399) Reason for Stage Failure should be shown in UI

[ https://issues.apache.org/jira/browse/SPARK-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13960790#comment-13960790 ] Kay Ousterhout commented on SPARK-1399: --- FYI this outstanding pull request changes t

[jira] [Assigned] (SPARK-1399) Reason for Stage Failure should be shown in UI

[ https://issues.apache.org/jira/browse/SPARK-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu reassigned SPARK-1399: -- Assignee: Nan Zhu > Reason for Stage Failure should be shown in UI > --

[jira] [Created] (SPARK-1418) Python MLlib's _get_unmangled_rdd should uncache RDDs when training is done

Matei Zaharia created SPARK-1418: Summary: Python MLlib's _get_unmangled_rdd should uncache RDDs when training is done Key: SPARK-1418 URL: https://issues.apache.org/jira/browse/SPARK-1418 Project: Sp

[jira] [Assigned] (SPARK-1417) Spark on Yarn - spark UI link from resourcemanager is broken

[ https://issues.apache.org/jira/browse/SPARK-1417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1417: Assignee: Thomas Graves > Spark on Yarn - spark UI link from resourcemanager is broken > --

[jira] [Assigned] (SPARK-1053) Should not require SPARK_YARN_APP_JAR when running on YARN

[ https://issues.apache.org/jira/browse/SPARK-1053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1053: - Assignee: Sandy Ryza (was: Sandy Pérez González) > Should not require SPARK_YARN_APP_JAR when ru

[jira] [Assigned] (SPARK-1056) Header comment in Executor incorrectly implies it's not used for YARN

[ https://issues.apache.org/jira/browse/SPARK-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1056: - Assignee: Sandy Ryza (was: Sandy Pérez González) > Header comment in Executor incorrectly implie

[jira] [Assigned] (SPARK-1033) Ask for cores in Yarn container requests

[ https://issues.apache.org/jira/browse/SPARK-1033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1033: - Assignee: Sandy Ryza (was: Sandy Pérez González) > Ask for cores in Yarn container requests > -

[jira] [Assigned] (SPARK-1211) In ApplicationMaster, set spark.master system property to "yarn-cluster"

[ https://issues.apache.org/jira/browse/SPARK-1211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1211: - Assignee: Sandy Ryza (was: Sandy Pérez González) > In ApplicationMaster, set spark.master system

[jira] [Assigned] (SPARK-1197) Rename yarn-standalone and fix up docs for running on YARN

[ https://issues.apache.org/jira/browse/SPARK-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1197: - Assignee: Sandy Ryza (was: Sandy Pérez González) > Rename yarn-standalone and fix up docs for ru

[jira] [Assigned] (SPARK-1032) If Yarn app fails before registering, app master stays around long after

[ https://issues.apache.org/jira/browse/SPARK-1032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1032: - Assignee: Sandy Ryza (was: Sandy Pérez González) > If Yarn app fails before registering, app mas

[jira] [Assigned] (SPARK-1064) Make it possible to use cluster's Hadoop jars when running against YARN

[ https://issues.apache.org/jira/browse/SPARK-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1064: - Assignee: Sandy Ryza (was: Sandy Pérez González) > Make it possible to use cluster's Hadoop jars

[jira] [Assigned] (SPARK-1193) Inconsistent indendation between pom.xmls

[ https://issues.apache.org/jira/browse/SPARK-1193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1193: - Assignee: Sandy Ryza (was: Sandy Pérez González) > Inconsistent indendation between pom.xmls > -

[jira] [Assigned] (SPARK-782) Multiple versions of ASM being put on classpath

[ https://issues.apache.org/jira/browse/SPARK-782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-782: Assignee: Sandy Ryza (was: Sandy Pérez González) > Multiple versions of ASM being put on classpath

[jira] [Assigned] (SPARK-1051) On Yarn, executors don't doAs as submitting user

[ https://issues.apache.org/jira/browse/SPARK-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1051: - Assignee: Sandy Ryza (was: Sandy Pérez González) > On Yarn, executors don't doAs as submitting u

[jira] [Assigned] (SPARK-1183) Inconsistent meaning of "worker" in docs

[ https://issues.apache.org/jira/browse/SPARK-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-1183: - Assignee: Sandy Ryza (was: Sandy Pérez González) > Inconsistent meaning of "worker" in docs > --

[jira] [Resolved] (SPARK-1375) spark-submit script additional cleanup

[ https://issues.apache.org/jira/browse/SPARK-1375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1375. Resolution: Fixed > spark-submit script additional cleanup > --

[jira] [Created] (SPARK-1417) Spark on Yarn - spark UI link from resourcemanager is broken

Thomas Graves created SPARK-1417: Summary: Spark on Yarn - spark UI link from resourcemanager is broken Key: SPARK-1417 URL: https://issues.apache.org/jira/browse/SPARK-1417 Project: Spark I

[jira] [Updated] (SPARK-1417) Spark on Yarn - spark UI link from resourcemanager is broken

[ https://issues.apache.org/jira/browse/SPARK-1417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1417: - Description: When running spark on yarn in yarn-cluster mode, spark registers a url with the Yar

[jira] [Created] (SPARK-1416) Add support for SequenceFiles in PySpark

Matei Zaharia created SPARK-1416: Summary: Add support for SequenceFiles in PySpark Key: SPARK-1416 URL: https://issues.apache.org/jira/browse/SPARK-1416 Project: Spark Issue Type: Improvemen

[jira] [Assigned] (SPARK-1414) Python API for SparkContext.wholeTextFiles

[ https://issues.apache.org/jira/browse/SPARK-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia reassigned SPARK-1414: Assignee: Matei Zaharia > Python API for SparkContext.wholeTextFiles >

[jira] [Commented] (SPARK-1366) The sql function should be consistent between different types of SQLContext

[ https://issues.apache.org/jira/browse/SPARK-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13960292#comment-13960292 ] Michael Armbrust commented on SPARK-1366: - https://github.com/apache/spark/pull/31

[jira] [Created] (SPARK-1415) Add a minSplits parameter to wholeTextFiles

Matei Zaharia created SPARK-1415: Summary: Add a minSplits parameter to wholeTextFiles Key: SPARK-1415 URL: https://issues.apache.org/jira/browse/SPARK-1415 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-1414) Python API for SparkContext.wholeTextFiles

Matei Zaharia created SPARK-1414: Summary: Python API for SparkContext.wholeTextFiles Key: SPARK-1414 URL: https://issues.apache.org/jira/browse/SPARK-1414 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-1133) Add a new small files input for MLlib, which will return an RDD[(fileName, content)]

[ https://issues.apache.org/jira/browse/SPARK-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1133. -- Resolution: Fixed Fix Version/s: 1.0.0 > Add a new small files input for MLlib, which wi

[jira] [Resolved] (SPARK-1383) Spark-SQL: ParquetRelation improvements

[ https://issues.apache.org/jira/browse/SPARK-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andre Schumacher resolved SPARK-1383. - Resolution: Fixed Fixed by https://github.com/apache/spark/commit/fbebaedf26286ee8a75065

[jira] [Resolved] (SPARK-1404) Non-exported spark-env.sh variables are no longer present in spark-shell

[ https://issues.apache.org/jira/browse/SPARK-1404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1404. Resolution: Fixed > Non-exported spark-env.sh variables are no longer present in spark-shel

[jira] [Commented] (SPARK-1391) BlockManager cannot transfer blocks larger than 2G in size

[ https://issues.apache.org/jira/browse/SPARK-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13960100#comment-13960100 ] Shivaram Venkataraman commented on SPARK-1391: -- Thanks for the patch. I will

[jira] [Resolved] (SPARK-1350) YARN ContainerLaunchContext should use cluster's JAVA_HOME

[ https://issues.apache.org/jira/browse/SPARK-1350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1350. -- Resolution: Fixed https://github.com/apache/spark/pull/313 > YARN ContainerLaunchContext shoul

[jira] [Commented] (SPARK-1413) Parquet messes up stdout and stdin when used in Spark REPL

[ https://issues.apache.org/jira/browse/SPARK-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13959784#comment-13959784 ] witgo commented on SPARK-1413: -- Try [the PR 325|https://github.com/apache/spark/pull/325] >

[jira] [Commented] (SPARK-1394) calling system.platform on worker raises IOError

[ https://issues.apache.org/jira/browse/SPARK-1394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13959735#comment-13959735 ] Idan Zalzberg commented on SPARK-1394: -- This seems to be related to the way the handl