[jira] [Created] (SPARK-1563) Add package-info.java files for all packages that appear in Javadoc

2014-04-22 Thread Matei Zaharia (JIRA)
Matei Zaharia created SPARK-1563: Summary: Add package-info.java files for all packages that appear in Javadoc Key: SPARK-1563 URL: https://issues.apache.org/jira/browse/SPARK-1563 Project: Spark

[jira] [Created] (SPARK-1564) Add JavaScript into Javadoc to turn ::Experimental:: and such into badges

2014-04-22 Thread Matei Zaharia (JIRA)
Matei Zaharia created SPARK-1564: Summary: Add JavaScript into Javadoc to turn ::Experimental:: and such into badges Key: SPARK-1564 URL: https://issues.apache.org/jira/browse/SPARK-1564 Project:

[jira] [Resolved] (SPARK-1459) EventLoggingListener does not work with file:// target dir

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1459. Resolution: Fixed Fix Version/s: 1.0.0 EventLoggingListener does not work with

[jira] [Updated] (SPARK-1561) sbt/sbt assembly generates too many local files

2014-04-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1561: - Description: Running `find ./ | wc -l` after `sbt/sbt assembly` returned This hits the

[jira] [Commented] (SPARK-1561) sbt/sbt assembly generates too many local files

2014-04-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13976487#comment-13976487 ] Xiangrui Meng commented on SPARK-1561: -- Tried adding {code} assemblyOption in

[jira] [Updated] (SPARK-1561) sbt/sbt assembly generates too many local files

2014-04-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1561: - Description: Running `find ./ | wc -l` after `sbt/sbt assembly` returned 564365 This hits the

[jira] [Commented] (SPARK-1529) Support setting spark.local.dirs to a hadoop FileSystem

2014-04-22 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13976494#comment-13976494 ] Aaron Davidson commented on SPARK-1529: --- Hey, looking through the code a little more

[jira] [Updated] (SPARK-1565) Spark examples should be changed given spark-submit

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1565: --- Description: Example:

[jira] [Comment Edited] (SPARK-1499) Workers continuously produce failing executors

2014-04-22 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13976294#comment-13976294 ] Adrian Wang edited comment on SPARK-1499 at 4/22/14 8:35 AM: -

[jira] [Resolved] (SPARK-1560) PySpark SQL depends on Java 7 only jars

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1560. Resolution: Fixed PySpark SQL depends on Java 7 only jars

[jira] [Created] (SPARK-1567) Add language tabs to quick start guide

2014-04-22 Thread Matei Zaharia (JIRA)
Matei Zaharia created SPARK-1567: Summary: Add language tabs to quick start guide Key: SPARK-1567 URL: https://issues.apache.org/jira/browse/SPARK-1567 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-1566) Consolidate the Spark Programming Guide with tabs for all languages

2014-04-22 Thread Matei Zaharia (JIRA)
Matei Zaharia created SPARK-1566: Summary: Consolidate the Spark Programming Guide with tabs for all languages Key: SPARK-1566 URL: https://issues.apache.org/jira/browse/SPARK-1566 Project: Spark

[jira] [Updated] (SPARK-1564) Add JavaScript into Javadoc to turn ::Experimental:: and such into badges

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1564: --- Assignee: Andrew Or Add JavaScript into Javadoc to turn ::Experimental:: and such into

[jira] [Updated] (SPARK-1563) Add package-info.java files for all packages that appear in Javadoc

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1563: --- Assignee: Prashant Sharma Add package-info.java files for all packages that appear in

[jira] [Assigned] (SPARK-1505) [streaming] Add 0.9 to 1.0 migration guide for streaming receiver

2014-04-22 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-1505: Assignee: Tathagata Das [streaming] Add 0.9 to 1.0 migration guide for streaming receiver

[jira] [Resolved] (SPARK-1567) Add language tabs to quick start guide

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1567. Resolution: Fixed Add language tabs to quick start guide

[jira] [Updated] (SPARK-1563) Add package-info.java and package.scala files for all packages that appear in docs

2014-04-22 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1563: - Summary: Add package-info.java and package.scala files for all packages that appear in docs

[jira] [Updated] (SPARK-1554) Update doc overview page to not mention building if you get a pre-built distro

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1554: --- Assignee: Patrick Wendell Update doc overview page to not mention building if you get a

[jira] [Created] (SPARK-1569) Spark on Yarn, authentication broken by pr299

2014-04-22 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-1569: Summary: Spark on Yarn, authentication broken by pr299 Key: SPARK-1569 URL: https://issues.apache.org/jira/browse/SPARK-1569 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-1570) Class loading issue when using Spark SQL Java API

2014-04-22 Thread Kan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13977279#comment-13977279 ] Kan Zhang commented on SPARK-1570: -- PR: https://github.com/apache/spark/pull/484 Class

[jira] [Updated] (SPARK-1571) UnresolvedException when running JavaSparkSQL example

2014-04-22 Thread Kan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kan Zhang updated SPARK-1571: - Description: When running JavaSparkSQL example using spark-submit in local mode (this happens after

[jira] [Updated] (SPARK-1570) Class loading issue when using Spark SQL Java API

2014-04-22 Thread Kan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kan Zhang updated SPARK-1570: - Description: ClassNotFoundException in Executor when running JavaSparkSQL example using spark-submit in

[jira] [Created] (SPARK-1571) UnresolvedException when running JavaSparkSQL example

2014-04-22 Thread Kan Zhang (JIRA)
Kan Zhang created SPARK-1571: Summary: UnresolvedException when running JavaSparkSQL example Key: SPARK-1571 URL: https://issues.apache.org/jira/browse/SPARK-1571 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-1572) Uncaught IO exceptions in Pyspark kill Executor

2014-04-22 Thread Aaron Davidson (JIRA)
Aaron Davidson created SPARK-1572: - Summary: Uncaught IO exceptions in Pyspark kill Executor Key: SPARK-1572 URL: https://issues.apache.org/jira/browse/SPARK-1572 Project: Spark Issue Type:

[jira] [Created] (SPARK-1575) failing tests with master branch

2014-04-22 Thread Nishkam Ravi (JIRA)
Nishkam Ravi created SPARK-1575: --- Summary: failing tests with master branch Key: SPARK-1575 URL: https://issues.apache.org/jira/browse/SPARK-1575 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-1576) Passing of JAVA_OPTS to YARN on command line

2014-04-22 Thread Nishkam Ravi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi updated SPARK-1576: Description: JAVA_OPTS can be passed by using either env variables (i.e., SPARK_JAVA_OPTS) or as

[jira] [Created] (SPARK-1577) GraphX mapVertices with KryoSerialization

2014-04-22 Thread Joseph E. Gonzalez (JIRA)
Joseph E. Gonzalez created SPARK-1577: - Summary: GraphX mapVertices with KryoSerialization Key: SPARK-1577 URL: https://issues.apache.org/jira/browse/SPARK-1577 Project: Spark Issue

[jira] [Updated] (SPARK-1577) GraphX mapVertices with KryoSerialization

2014-04-22 Thread Joseph E. Gonzalez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph E. Gonzalez updated SPARK-1577: -- Description: If Kryo is enabled by setting: {code}

[jira] [Updated] (SPARK-1577) GraphX mapVertices with KryoSerialization

2014-04-22 Thread Joseph E. Gonzalez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph E. Gonzalez updated SPARK-1577: -- Description: If Kryo is enabled by setting: {{{quote}

[jira] [Created] (SPARK-1578) Do not require setting of cleaner TTL when creating StreamingContext

2014-04-22 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-1578: Summary: Do not require setting of cleaner TTL when creating StreamingContext Key: SPARK-1578 URL: https://issues.apache.org/jira/browse/SPARK-1578 Project: Spark

[jira] [Updated] (SPARK-1578) Do not require setting of cleaner TTL when creating StreamingContext

2014-04-22 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-1578: - Description: Since shuffles and RDDs that are out of context are automatically cleaned by Spark

[jira] [Assigned] (SPARK-1571) UnresolvedException when running JavaSparkSQL example

2014-04-22 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-1571: --- Assignee: Michael Armbrust UnresolvedException when running JavaSparkSQL example

[jira] [Updated] (SPARK-1579) PySpark should distinguish expected IOExceptions from unexpected ones in the worker

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1579: --- Description: I chatted with [~adav] a bit about this. Right now we drop IOExceptions because

[jira] [Created] (SPARK-1580) ALS: Estimate communication and computation costs given a partitioner

2014-04-22 Thread Tor Myklebust (JIRA)
Tor Myklebust created SPARK-1580: Summary: ALS: Estimate communication and computation costs given a partitioner Key: SPARK-1580 URL: https://issues.apache.org/jira/browse/SPARK-1580 Project: Spark

[jira] [Updated] (SPARK-800) Improve Quickstart Docs to Make Full Deployment More Clear

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-800: -- Reporter: Patrick Wendell (was: Patrick Cogan) Improve Quickstart Docs to Make Full

[jira] [Updated] (SPARK-800) Improve Quickstart Docs to Make Full Deployment More Clear

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-800: -- Assignee: Patrick Wendell (was: Patrick Cogan) Improve Quickstart Docs to Make Full

[jira] [Resolved] (SPARK-800) Improve Quickstart Docs to Make Full Deployment More Clear

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-800. --- Resolution: Fixed Fix Version/s: 1.0.0 This has been fixed with the recent changes to

[jira] [Updated] (SPARK-1119) Make examples and assembly jar naming consistent between maven/sbt

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1119: --- Description: Right now it's somewhat different. Also it should be consistent with what the

[jira] [Updated] (SPARK-1281) Partitioning in ALS

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1281: --- Assignee: Tor Myklebust Partitioning in ALS --- Key:

[jira] [Resolved] (SPARK-1538) SparkUI forgets about all persisted RDD's not directly associated with the Stage

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1538. Resolution: Fixed Assignee: Andrew Or SparkUI forgets about all persisted RDD's not

[jira] [Commented] (SPARK-1576) Passing of JAVA_OPTS to YARN on command line

2014-04-22 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13977754#comment-13977754 ] Sandy Ryza commented on SPARK-1576: --- Hi [~nravi], Spark patches should be submitted as

[jira] [Commented] (SPARK-1376) In the yarn-cluster submitter, rename args option to arg

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13977757#comment-13977757 ] Patrick Wendell commented on SPARK-1376: There was follow up work on this by

[jira] [Updated] (SPARK-1581) Allow One Flume Avro RPC Server for Each Worker rather than Just One Worker

2014-04-22 Thread Christophe Clapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe Clapp updated SPARK-1581: Summary: Allow One Flume Avro RPC Server for Each Worker rather than Just One Worker

[jira] [Resolved] (SPARK-1494) Hive Dependencies being checked by MIMA

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1494. Resolution: Fixed Hive Dependencies being checked by MIMA

[jira] [Updated] (SPARK-1494) Hive Dependencies being checked by MIMA

2014-04-22 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1494: --- Fix Version/s: 1.0.0 Hive Dependencies being checked by MIMA