[jira] [Created] (SPARK-1945) Add full Java examples in MLlib docs

Matei Zaharia created SPARK-1945: Summary: Add full Java examples in MLlib docs Key: SPARK-1945 URL: https://issues.apache.org/jira/browse/SPARK-1945 Project: Spark Issue Type: Sub-task

[jira] [Comment Edited] (SPARK-1153) Generalize VertexId in GraphX so that UUIDs can be used as vertex IDs.

[ https://issues.apache.org/jira/browse/SPARK-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010846#comment-14010846 ] npanj edited comment on SPARK-1153 at 5/28/14 6:48 AM: --- An

[jira] [Created] (SPARK-1946) Submit stage after executors have been registered

Zhihui created SPARK-1946: - Summary: Submit stage after executors have been registered Key: SPARK-1946 URL: https://issues.apache.org/jira/browse/SPARK-1946 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-1946) Submit stage after executors have been registered

[ https://issues.apache.org/jira/browse/SPARK-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihui updated SPARK-1946: -- Attachment: Spark Task Scheduler Optimization Proposal.pptx Submit stage after executors have been registered

[jira] [Updated] (SPARK-1946) Submit stage after executors have been registered

[ https://issues.apache.org/jira/browse/SPARK-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihui updated SPARK-1946: -- Description: Because creating TaskSetManager and registering executors are asynchronous, in most situation,

[jira] [Commented] (SPARK-1495) support leftsemijoin for sparkSQL

[ https://issues.apache.org/jira/browse/SPARK-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010932#comment-14010932 ] Adrian Wang commented on SPARK-1495: Another PR

[jira] [Commented] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010937#comment-14010937 ] Sean Owen commented on SPARK-1518: -- Re: versioning one more time, really supporting a

[jira] [Commented] (SPARK-1947) Child of SumDistinct or Average should be widened to prevent overflows the same as Sum.

[ https://issues.apache.org/jira/browse/SPARK-1947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010945#comment-14010945 ] Takuya Ueshin commented on SPARK-1947: -- PRed:

[jira] [Created] (SPARK-1948) Scalac crashes when building Spark in IntelliJ IDEA

Cheng Lian created SPARK-1948: - Summary: Scalac crashes when building Spark in IntelliJ IDEA Key: SPARK-1948 URL: https://issues.apache.org/jira/browse/SPARK-1948 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-1948) Scalac crashes when building Spark in IntelliJ IDEA

[ https://issues.apache.org/jira/browse/SPARK-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-1948: -- Attachment: scalac-crash.log Scalac crashes when building Spark in IntelliJ IDEA

[jira] [Updated] (SPARK-1951) spark on yarn can't start

[ https://issues.apache.org/jira/browse/SPARK-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1951: --- Description: {{HADOOP_CONF_DIR=/etc/hadoop/conf ./bin/spark-submit --archives

[jira] [Created] (SPARK-1950) spark on yarn can't start

Guoqiang Li created SPARK-1950: -- Summary: spark on yarn can't start Key: SPARK-1950 URL: https://issues.apache.org/jira/browse/SPARK-1950 Project: Spark Issue Type: Bug Components:

[jira] [Commented] (SPARK-1950) spark on yarn can't start

[ https://issues.apache.org/jira/browse/SPARK-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011330#comment-14011330 ] Sean Owen commented on SPARK-1950: -- (Looks like you opened this twice?

[jira] [Commented] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011350#comment-14011350 ] Colin Patrick McCabe commented on SPARK-1518: - bq. Re: versioning one more

[jira] [Updated] (SPARK-1916) SparkFlumeEvent with body bigger than 1020 bytes are not read properly

[ https://issues.apache.org/jira/browse/SPARK-1916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1916: --- Assignee: David Lemieux SparkFlumeEvent with body bigger than 1020 bytes are not read

[jira] [Commented] (SPARK-1836) REPL $outer type mismatch causes lookup() and equals() problems

[ https://issues.apache.org/jira/browse/SPARK-1836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011477#comment-14011477 ] Michael Armbrust commented on SPARK-1836: - Yeah I think its likely they are

[jira] [Commented] (SPARK-1199) Type mismatch in Spark shell when using case class defined in shell

[ https://issues.apache.org/jira/browse/SPARK-1199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011492#comment-14011492 ] Michael Malak commented on SPARK-1199: -- See also additional test cases in

[jira] [Resolved] (SPARK-1836) REPL $outer type mismatch causes lookup() and equals() problems

[ https://issues.apache.org/jira/browse/SPARK-1836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Malak resolved SPARK-1836. -- Resolution: Duplicate REPL $outer type mismatch causes lookup() and equals() problems

[jira] [Resolved] (SPARK-1936) Add apache header and remove author tags

[ https://issues.apache.org/jira/browse/SPARK-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1936. -- Resolution: Won't Fix We should not change these files' license headers because they're files

[jira] [Commented] (SPARK-1790) Update EC2 scripts to support r3 instance types

[ https://issues.apache.org/jira/browse/SPARK-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011548#comment-14011548 ] Matei Zaharia commented on SPARK-1790: -- Thanks Sujeet! Just post here when you have a

[jira] [Updated] (SPARK-1790) Update EC2 scripts to support r3 instance types

[ https://issues.apache.org/jira/browse/SPARK-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1790: - Labels: Starter (was: starter) Update EC2 scripts to support r3 instance types

[jira] [Updated] (SPARK-1952) slf4j version conflicts with pig

[ https://issues.apache.org/jira/browse/SPARK-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Compton updated SPARK-1952: Description: Upgrading from Spark-0.9.1 to Spark-1.0.0 causes all Pig scripts to fail when they

[jira] [Updated] (SPARK-1952) slf4j version conflicts with pig

[ https://issues.apache.org/jira/browse/SPARK-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Compton updated SPARK-1952: Description: Upgrading from Spark-0.9.1 to Spark-1.0.0 causes all Pig scripts to fail when they

[jira] [Updated] (SPARK-1952) slf4j version conflicts with pig

[ https://issues.apache.org/jira/browse/SPARK-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Compton updated SPARK-1952: Description: Upgrading from Spark-0.9.1 to Spark-1.0.0 causes all Pig scripts to fail when they

[jira] [Created] (SPARK-1954) Make it easier to get Spark on YARN code to compile in IntelliJ

Sandy Ryza created SPARK-1954: - Summary: Make it easier to get Spark on YARN code to compile in IntelliJ Key: SPARK-1954 URL: https://issues.apache.org/jira/browse/SPARK-1954 Project: Spark

[jira] [Created] (SPARK-1955) VertexRDD can incorrectly assume index sharing

Ankur Dave created SPARK-1955: - Summary: VertexRDD can incorrectly assume index sharing Key: SPARK-1955 URL: https://issues.apache.org/jira/browse/SPARK-1955 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-1501) Assertions in Graph.apply test are never executed

[ https://issues.apache.org/jira/browse/SPARK-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-1501. --- Resolution: Fixed Assignee: William Benton Assertions in Graph.apply test are never executed

[jira] [Commented] (SPARK-1952) slf4j version conflicts with pig

[ https://issues.apache.org/jira/browse/SPARK-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011711#comment-14011711 ] Matei Zaharia commented on SPARK-1952: -- Ryan, do you know what SLF4J version Pig

[jira] [Updated] (SPARK-1952) slf4j version conflicts with pig

[ https://issues.apache.org/jira/browse/SPARK-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1952: --- Target Version/s: 1.0.1 (was: 1.0.0) slf4j version conflicts with pig

[jira] [Commented] (SPARK-1956) Enable shuffle consolidation by default

[ https://issues.apache.org/jira/browse/SPARK-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011741#comment-14011741 ] Mridul Muralidharan commented on SPARK-1956: shuffle consolidation MUST NOT be

[jira] [Updated] (SPARK-1952) slf4j version conflicts with pig

[ https://issues.apache.org/jira/browse/SPARK-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1952: --- Fix Version/s: (was: 1.0.0) slf4j version conflicts with pig

[jira] [Resolved] (SPARK-1916) SparkFlumeEvent with body bigger than 1020 bytes are not read properly

[ https://issues.apache.org/jira/browse/SPARK-1916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1916. Resolution: Fixed Fix Version/s: 0.9.2 1.0.1 Issue resolved by

[jira] [Updated] (SPARK-1576) Passing of JAVA_OPTS to YARN on command line

[ https://issues.apache.org/jira/browse/SPARK-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1576: --- Fix Version/s: (was: 0.9.1) 0.9.2 Passing of JAVA_OPTS to YARN on

[jira] [Updated] (SPARK-1849) Broken UTF-8 encoded data gets character replacements and thus can't be fixed

[ https://issues.apache.org/jira/browse/SPARK-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1849: --- Fix Version/s: (was: 0.9.1) 0.9.2 Broken UTF-8 encoded data gets

[jira] [Updated] (SPARK-1759) sbt/sbt package fail cause by directory

[ https://issues.apache.org/jira/browse/SPARK-1759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1759: --- Fix Version/s: (was: 0.9.1) 0.9.2 sbt/sbt package fail cause by

[jira] [Resolved] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1712. -- Resolution: Fixed ParallelCollectionRDD operations hanging forever without any error messages

[jira] [Updated] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1712: - Priority: Major (was: Blocker) ParallelCollectionRDD operations hanging forever without any

[jira] [Updated] (SPARK-1817) RDD zip erroneous when partitions do not divide RDD count

[ https://issues.apache.org/jira/browse/SPARK-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1817: - Priority: Minor (was: Blocker) RDD zip erroneous when partitions do not divide RDD count

[jira] [Updated] (SPARK-1817) RDD zip erroneous when partitions do not divide RDD count

[ https://issues.apache.org/jira/browse/SPARK-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1817: - Priority: Major (was: Minor) RDD zip erroneous when partitions do not divide RDD count

[jira] [Updated] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1712: - Fix Version/s: 1.0.1 ParallelCollectionRDD operations hanging forever without any error

[jira] [Commented] (SPARK-1952) slf4j version conflicts with pig

[ https://issues.apache.org/jira/browse/SPARK-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011789#comment-14011789 ] Ryan Compton commented on SPARK-1952: - Pig depends on slf4j 1.6.1 {code}

[jira] [Commented] (SPARK-1952) slf4j version conflicts with pig

[ https://issues.apache.org/jira/browse/SPARK-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011806#comment-14011806 ] Patrick Wendell commented on SPARK-1952: Hm, unfortunately I dont' see any obvious

[jira] [Commented] (SPARK-1952) slf4j version conflicts with pig

[ https://issues.apache.org/jira/browse/SPARK-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011808#comment-14011808 ] Patrick Wendell commented on SPARK-1952: [~rcompton] - what if you modify the

[jira] [Commented] (SPARK-1954) Make it easier to get Spark on YARN code to compile in IntelliJ

[ https://issues.apache.org/jira/browse/SPARK-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011810#comment-14011810 ] Patrick Wendell commented on SPARK-1954: Have you tried running sbt/sbt gen-idea

[jira] [Resolved] (SPARK-1950) spark on yarn can't start

[ https://issues.apache.org/jira/browse/SPARK-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1950. Resolution: Duplicate spark on yarn can't start --

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011830#comment-14011830 ] Matei Zaharia commented on SPARK-1712: -- Merged the frame size check into 0.9.2 as

[jira] [Created] (SPARK-1957) Pluggable disk store for BlockManager

Raymond Liu created SPARK-1957: -- Summary: Pluggable disk store for BlockManager Key: SPARK-1957 URL: https://issues.apache.org/jira/browse/SPARK-1957 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011897#comment-14011897 ] Sean Owen commented on SPARK-1518: -- they write their app against the Spark API's in Maven

[jira] [Updated] (SPARK-1957) Pluggable disk store for BlockManager

[ https://issues.apache.org/jira/browse/SPARK-1957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Liu updated SPARK-1957: --- Issue Type: Sub-task (was: New Feature) Parent: SPARK-1733 Pluggable disk store for

[jira] [Commented] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011942#comment-14011942 ] Matei Zaharia commented on SPARK-1518: -- Sean, the model for linking to Hadoop has

[jira] [Commented] (SPARK-1112) When spark.akka.frameSize 10, task results bigger than 10MiB block execution

[ https://issues.apache.org/jira/browse/SPARK-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011968#comment-14011968 ] Kevin (Sangwoo) Kim commented on SPARK-1112: Hi all, I'm very new to Spark

[jira] [Comment Edited] (SPARK-1112) When spark.akka.frameSize 10, task results bigger than 10MiB block execution

[ https://issues.apache.org/jira/browse/SPARK-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011968#comment-14011968 ] Kevin (Sangwoo) Kim edited comment on SPARK-1112 at 5/29/14 2:01 AM:

[jira] [Updated] (SPARK-1112) When spark.akka.frameSize 10, task results bigger than 10MiB block execution

[ https://issues.apache.org/jira/browse/SPARK-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1112: - Priority: Critical (was: Blocker) When spark.akka.frameSize 10, task results bigger than

[jira] [Commented] (SPARK-1112) When spark.akka.frameSize 10, task results bigger than 10MiB block execution

[ https://issues.apache.org/jira/browse/SPARK-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011978#comment-14011978 ] Matei Zaharia commented on SPARK-1112: -- I'm curious, why did you want to make the

[jira] [Commented] (SPARK-1112) When spark.akka.frameSize 10, task results bigger than 10MiB block execution

[ https://issues.apache.org/jira/browse/SPARK-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011993#comment-14011993 ] Kevin (Sangwoo) Kim commented on SPARK-1112: [~matei] I've found the default

[jira] [Comment Edited] (SPARK-1112) When spark.akka.frameSize 10, task results bigger than 10MiB block execution

[ https://issues.apache.org/jira/browse/SPARK-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14011968#comment-14011968 ] Kevin (Sangwoo) Kim edited comment on SPARK-1112 at 5/29/14 2:50 AM:

[jira] [Created] (SPARK-1958) Calling .collect() on a SchemaRDD should call executeCollect() on the underlying query plan.

Michael Armbrust created SPARK-1958: --- Summary: Calling .collect() on a SchemaRDD should call executeCollect() on the underlying query plan. Key: SPARK-1958 URL: https://issues.apache.org/jira/browse/SPARK-1958

[jira] [Created] (SPARK-1959) String NULL is interpreted as null value

Cheng Lian created SPARK-1959: - Summary: String NULL is interpreted as null value Key: SPARK-1959 URL: https://issues.apache.org/jira/browse/SPARK-1959 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-1901) Standalone worker update exector's state ahead of executor process exit

[ https://issues.apache.org/jira/browse/SPARK-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1901: --- Fix Version/s: (was: 1.0.0) 1.0.1 Standalone worker update exector's

[jira] [Commented] (SPARK-1954) Make it easier to get Spark on YARN code to compile in IntelliJ

[ https://issues.apache.org/jira/browse/SPARK-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14012027#comment-14012027 ] Sandy Ryza commented on SPARK-1954: --- Cool. Your suggestion does appear to work. Make

[jira] [Commented] (SPARK-1959) String NULL is interpreted as null value

[ https://issues.apache.org/jira/browse/SPARK-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14012031#comment-14012031 ] Cheng Lian commented on SPARK-1959: --- The problematic line should be [this

[jira] [Commented] (SPARK-1952) slf4j version conflicts with pig

[ https://issues.apache.org/jira/browse/SPARK-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14012040#comment-14012040 ] Patrick Wendell commented on SPARK-1952: So I think the issue here is simply that

[jira] [Updated] (SPARK-1954) Make it easier to get Spark on YARN code to compile in IntelliJ

[ https://issues.apache.org/jira/browse/SPARK-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1954: --- Component/s: Build Make it easier to get Spark on YARN code to compile in IntelliJ

[jira] [Updated] (SPARK-1913) Parquet table column pruning error caused by filter pushdown

[ https://issues.apache.org/jira/browse/SPARK-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-1913: Assignee: Cheng Lian Parquet table column pruning error caused by filter pushdown

[jira] [Resolved] (SPARK-1913) Parquet table column pruning error caused by filter pushdown

[ https://issues.apache.org/jira/browse/SPARK-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-1913. - Resolution: Fixed Parquet table column pruning error caused by filter pushdown

[jira] [Updated] (SPARK-1960) EOFException when file size 0 exists when use sc.sequenceFile[K,V](path)

[ https://issues.apache.org/jira/browse/SPARK-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eunsu Yun updated SPARK-1960: - Description: java.io.EOFException throws when use sc.sequenceFile[K,V] if there is a file which size is