[jira] [Commented] (SPARK-2044) Pluggable interface for shuffles

2014-06-07 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14021115#comment-14021115 ] Matei Zaharia commented on SPARK-2044: -- Alright so I've posted my code at https://git

[jira] [Commented] (SPARK-938) OpenStack Swift Storage Support

2014-06-07 Thread Gil Vernik (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14021114#comment-14021114 ] Gil Vernik commented on SPARK-938: -- I am working on it. About to submit the patch. Almost

[jira] [Commented] (SPARK-2064) web ui should not remove executors if they are dead

2014-06-07 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14021020#comment-14021020 ] Mridul Muralidharan commented on SPARK-2064: Ah, I assumed there was a disconn

[jira] [Updated] (SPARK-2067) Spark logo in application UI uses absolute path

2014-06-07 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2067: --- Target Version/s: 1.0.1, 1.1.0 > Spark logo in application UI uses absolute path > --

[jira] [Commented] (SPARK-2064) web ui should not remove executors if they are dead

2014-06-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14021018#comment-14021018 ] Reynold Xin commented on SPARK-2064: One thing is we can help identify executors that

[jira] [Updated] (SPARK-2056) Set RDD name to input path

2014-06-07 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2056: --- Assignee: Neville Li > Set RDD name to input path > -- > >

[jira] [Resolved] (SPARK-2056) Set RDD name to input path

2014-06-07 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2056. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 992 [https://g

[jira] [Commented] (SPARK-2064) web ui should not remove executors if they are dead

2014-06-07 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14021011#comment-14021011 ] Mridul Muralidharan commented on SPARK-2064: I am probably missing the intent

[jira] [Commented] (SPARK-2064) web ui should not remove executors if they are dead

2014-06-07 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14021008#comment-14021008 ] Mridul Muralidharan commented on SPARK-2064: Unfortunately OOM is a very big i

[jira] [Comment Edited] (SPARK-2064) web ui should not remove executors if they are dead

2014-06-07 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14020976#comment-14020976 ] Patrick Wendell edited comment on SPARK-2064 at 6/7/14 9:16 PM:

[jira] [Commented] (SPARK-2064) web ui should not remove executors if they are dead

2014-06-07 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14020976#comment-14020976 ] Patrick Wendell commented on SPARK-2064: I don't think OOM is an issue here - but

[jira] [Commented] (SPARK-2064) web ui should not remove executors if they are dead

2014-06-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14020971#comment-14020971 ] Reynold Xin commented on SPARK-2064: My estimation gave you a very high bound. How oft

[jira] [Updated] (SPARK-2059) Unresolved Attributes should cause a failure before execution time

2014-06-07 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2059: Assignee: Cheng Lian > Unresolved Attributes should cause a failure before execution time >

[jira] [Updated] (SPARK-2063) Creating a SchemaRDD via sql() does not correctly resolve nested types

2014-06-07 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2063: Assignee: Cheng Lian (was: Michael Armbrust) > Creating a SchemaRDD via sql() does not cor

[jira] [Updated] (SPARK-2053) Add Catalyst expression for CASE WHEN

2014-06-07 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2053: Assignee: Zongheng Yang > Add Catalyst expression for CASE WHEN > -

[jira] [Updated] (SPARK-2060) Querying JSON Datasets with SQL and DSL in Spark SQL

2014-06-07 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2060: Assignee: Yin Huai > Querying JSON Datasets with SQL and DSL in Spark SQL > ---

[jira] [Commented] (SPARK-2063) Creating a SchemaRDD via sql() does not correctly resolve nested types

2014-06-07 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14020944#comment-14020944 ] Michael Armbrust commented on SPARK-2063: - Here is a reproducible test case: {code

[jira] [Created] (SPARK-2068) Remove other uses of @transient lazy val in physical plan nodes

2014-06-07 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-2068: --- Summary: Remove other uses of @transient lazy val in physical plan nodes Key: SPARK-2068 URL: https://issues.apache.org/jira/browse/SPARK-2068 Project: Spark

[jira] [Commented] (SPARK-2064) web ui should not remove executors if they are dead

2014-06-07 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14020936#comment-14020936 ] Mridul Muralidharan commented on SPARK-2064: It is 100 MB (or more) of memory

[jira] [Commented] (SPARK-2067) Spark logo in application UI uses absolute path

2014-06-07 Thread Neville Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14020932#comment-14020932 ] Neville Li commented on SPARK-2067: --- A simple fix: https://github.com/apache/spark/pull/

[jira] [Created] (SPARK-2067) Spark logo in application UI uses absolute path

2014-06-07 Thread Neville Li (JIRA)
Neville Li created SPARK-2067: - Summary: Spark logo in application UI uses absolute path Key: SPARK-2067 URL: https://issues.apache.org/jira/browse/SPARK-2067 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2064) web ui should not remove executors if they are dead

2014-06-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14020897#comment-14020897 ] Reynold Xin commented on SPARK-2064: Is memory really an issue here? On a 1000 node c

[jira] [Commented] (SPARK-2065) Have spark-ec2 set EC2 instance names

2014-06-07 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14020893#comment-14020893 ] Nicholas Chammas commented on SPARK-2065: - Sure, I'd love to. > Have spark-ec2 se

[jira] [Commented] (SPARK-1812) Support cross-building with Scala 2.11

2014-06-07 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14020855#comment-14020855 ] Prashant Sharma commented on SPARK-1812: We will need to release kafka, akka-zerom

[jira] [Commented] (SPARK-2064) web ui should not remove executors if they are dead

2014-06-07 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14020789#comment-14020789 ] Mridul Muralidharan commented on SPARK-2064: Depending on how long a job runs,

[jira] [Created] (SPARK-2066) org.apache.spark.sql.catalyst.errors.package$TreeNodeException: No function to evaluate expression. type: AttributeReference, tree: key#61

2014-06-07 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-2066: -- Summary: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: No function to evaluate expression. type: AttributeReference, tree: key#61 Key: SPARK-2066 URL: https://issue