[jira] [Assigned] (SPARK-8306) executionHive does not need to use switch class loader

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8306: --- Assignee: Apache Spark (was: Yin Huai) > executionHive does not need to use switch class loa

[jira] [Commented] (SPARK-8306) executionHive does not need to use switch class loader

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581532#comment-14581532 ] Apache Spark commented on SPARK-8306: - User 'yhuai' has created a pull request for thi

[jira] [Assigned] (SPARK-8306) executionHive does not need to use switch class loader

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8306: --- Assignee: Yin Huai (was: Apache Spark) > executionHive does not need to use switch class loa

[jira] [Created] (SPARK-8306) executionHive does not need to use switch class loader

2015-06-10 Thread Yin Huai (JIRA)
Yin Huai created SPARK-8306: --- Summary: executionHive does not need to use switch class loader Key: SPARK-8306 URL: https://issues.apache.org/jira/browse/SPARK-8306 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-8298) Sliding Window CrossValidator

2015-06-10 Thread Justin Yip (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Yip updated SPARK-8298: -- Description: CrossValidator only supports k-folds. It cannot prevent the validation data from look-ahea

[jira] [Commented] (SPARK-6725) Model export/import for Pipeline API

2015-06-10 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581480#comment-14581480 ] Joseph K. Bradley commented on SPARK-6725: -- No problem; it can be hard to search

[jira] [Commented] (SPARK-8298) Sliding Window CrossValidator

2015-06-10 Thread Justin Yip (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581481#comment-14581481 ] Justin Yip commented on SPARK-8298: --- I coded it up. Most of the part are copied from Cro

[jira] [Commented] (SPARK-8298) Sliding Window CrossValidator

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581475#comment-14581475 ] Apache Spark commented on SPARK-8298: - User 'yipjustin' has created a pull request for

[jira] [Assigned] (SPARK-8298) Sliding Window CrossValidator

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8298: --- Assignee: (was: Apache Spark) > Sliding Window CrossValidator > -

[jira] [Assigned] (SPARK-8298) Sliding Window CrossValidator

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8298: --- Assignee: Apache Spark > Sliding Window CrossValidator > - > >

[jira] [Commented] (SPARK-6725) Model export/import for Pipeline API

2015-06-10 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581472#comment-14581472 ] Yu Ishikawa commented on SPARK-6725: I understand. I couldn't find the issue. Sorry fo

[jira] [Commented] (SPARK-6725) Model export/import for Pipeline API

2015-06-10 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581460#comment-14581460 ] Joseph K. Bradley commented on SPARK-6725: -- This JIRA is for the spark.ml Pipelin

[jira] [Commented] (SPARK-6725) Model export/import for Pipeline API

2015-06-10 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581457#comment-14581457 ] Yu Ishikawa commented on SPARK-6725: I'm afraid I'm not familiar with the LDA implemen

[jira] [Commented] (SPARK-7879) KMeans API for spark.ml Pipelines

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581418#comment-14581418 ] Apache Spark commented on SPARK-7879: - User 'yu-iskw' has created a pull request for t

[jira] [Assigned] (SPARK-7879) KMeans API for spark.ml Pipelines

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7879: --- Assignee: (was: Apache Spark) > KMeans API for spark.ml Pipelines > -

[jira] [Assigned] (SPARK-7879) KMeans API for spark.ml Pipelines

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7879: --- Assignee: Apache Spark > KMeans API for spark.ml Pipelines >

[jira] [Assigned] (SPARK-8305) Improve codegen

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8305: --- Assignee: Davies Liu (was: Apache Spark) > Improve codegen > --- > >

[jira] [Assigned] (SPARK-8305) Improve codegen

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8305: --- Assignee: Apache Spark (was: Davies Liu) > Improve codegen > --- > >

[jira] [Commented] (SPARK-8305) Improve codegen

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581410#comment-14581410 ] Apache Spark commented on SPARK-8305: - User 'davies' has created a pull request for th

[jira] [Created] (SPARK-8305) Improve codegen

2015-06-10 Thread Davies Liu (JIRA)
Davies Liu created SPARK-8305: - Summary: Improve codegen Key: SPARK-8305 URL: https://issues.apache.org/jira/browse/SPARK-8305 Project: Spark Issue Type: Bug Components: SQL

[jira] [Created] (SPARK-8304) Table with a large number of columns

2015-06-10 Thread jaeboo jung (JIRA)
jaeboo jung created SPARK-8304: -- Summary: Table with a large number of columns Key: SPARK-8304 URL: https://issues.apache.org/jira/browse/SPARK-8304 Project: Spark Issue Type: Bug Affects Ve

[jira] [Resolved] (SPARK-8217) math function: log2

2015-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-8217. Resolution: Fixed Fix Version/s: 1.5.0 > math function: log2 > --- > >

[jira] [Updated] (SPARK-8248) string function: length

2015-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8248: --- Labels: missing-python (was: ) > string function: length > --- > >

[jira] [Updated] (SPARK-8217) math function: log2

2015-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8217: --- Labels: missing-python (was: ) > math function: log2 > --- > > Key: S

[jira] [Commented] (SPARK-8297) Scheduler backend is not notified in case node fails in YARN

2015-06-10 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581383#comment-14581383 ] Saisai Shao commented on SPARK-8297: Hi [~mridulm80], I tried with latest master branc

[jira] [Resolved] (SPARK-8248) string function: length

2015-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-8248. Resolution: Fixed Fix Version/s: 1.5.0 > string function: length > --- >

[jira] [Commented] (SPARK-8303) Move DateUtils into unsafe package

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581319#comment-14581319 ] Apache Spark commented on SPARK-8303: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-8303) Move DateUtils into unsafe package

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8303: --- Assignee: Reynold Xin (was: Apache Spark) > Move DateUtils into unsafe package > ---

[jira] [Assigned] (SPARK-8303) Move DateUtils into unsafe package

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8303: --- Assignee: Apache Spark (was: Reynold Xin) > Move DateUtils into unsafe package > ---

[jira] [Created] (SPARK-8303) Move DateUtils into unsafe package

2015-06-10 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-8303: -- Summary: Move DateUtils into unsafe package Key: SPARK-8303 URL: https://issues.apache.org/jira/browse/SPARK-8303 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-8164) transformExpressions should support nested expression sequence

2015-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-8164. Resolution: Fixed Fix Version/s: 1.5.0 Assignee: Wenchen Fan > transformExpressions

[jira] [Updated] (SPARK-8164) transformExpressions should support nested expression sequence

2015-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8164: --- Issue Type: Improvement (was: Bug) > transformExpressions should support nested expression sequence >

[jira] [Resolved] (SPARK-8285) CombineSum should be calculated as unlimited decimal first

2015-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-8285. Resolution: Fixed Fix Version/s: 1.5.0 1.4.1 Assignee: Navis > Co

[jira] [Commented] (SPARK-7550) Support setting the right schema & serde when writing to Hive metastore

2015-06-10 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581315#comment-14581315 ] Cheng Hao commented on SPARK-7550: -- I will start working on this today, sorry for the del

[jira] [Assigned] (SPARK-8302) Support heterogeneous cluster nodes on YARN

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8302: --- Assignee: Apache Spark > Support heterogeneous cluster nodes on YARN > --

[jira] [Commented] (SPARK-8302) Support heterogeneous cluster nodes on YARN

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581314#comment-14581314 ] Apache Spark commented on SPARK-8302: - User 'vanzin' has created a pull request for th

[jira] [Assigned] (SPARK-8302) Support heterogeneous cluster nodes on YARN

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8302: --- Assignee: (was: Apache Spark) > Support heterogeneous cluster nodes on YARN > ---

[jira] [Comment Edited] (SPARK-7550) Support setting the right schema & serde when writing to Hive metastore

2015-06-10 Thread Nathan McCarthy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581311#comment-14581311 ] Nathan McCarthy edited comment on SPARK-7550 at 6/11/15 1:08 AM: ---

[jira] [Commented] (SPARK-7550) Support setting the right schema & serde when writing to Hive metastore

2015-06-10 Thread Nathan McCarthy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581311#comment-14581311 ] Nathan McCarthy commented on SPARK-7550: This is a major blocker for us as we use

[jira] [Commented] (SPARK-8302) Support heterogeneous cluster nodes on YARN

2015-06-10 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581305#comment-14581305 ] Marcelo Vanzin commented on SPARK-8302: --- So I spent some time on this and tried two

[jira] [Created] (SPARK-8302) Support heterogeneous cluster nodes on YARN

2015-06-10 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-8302: - Summary: Support heterogeneous cluster nodes on YARN Key: SPARK-8302 URL: https://issues.apache.org/jira/browse/SPARK-8302 Project: Spark Issue Type: New F

[jira] [Commented] (SPARK-5594) SparkException: Failed to get broadcast (TorrentBroadcast)

2015-06-10 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581297#comment-14581297 ] Patrick Wendell commented on SPARK-5594: /cc [~zsxwing] > SparkException: Failed

[jira] [Resolved] (SPARK-8126) Use temp directory under build dir for unit tests

2015-06-10 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-8126. --- Resolution: Fixed I guess this has been backported to all wanted versions at this point. > Us

[jira] [Created] (SPARK-8301) Improve UTF8String substring/startsWith/endsWith/contains performance

2015-06-10 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-8301: -- Summary: Improve UTF8String substring/startsWith/endsWith/contains performance Key: SPARK-8301 URL: https://issues.apache.org/jira/browse/SPARK-8301 Project: Spark

[jira] [Comment Edited] (SPARK-5594) SparkException: Failed to get broadcast (TorrentBroadcast)

2015-06-10 Thread Sal Uryasev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581269#comment-14581269 ] Sal Uryasev edited comment on SPARK-5594 at 6/11/15 12:12 AM: --

[jira] [Commented] (SPARK-5594) SparkException: Failed to get broadcast (TorrentBroadcast)

2015-06-10 Thread Sal Uryasev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581269#comment-14581269 ] Sal Uryasev commented on SPARK-5594: So the HttpBroadcastFactory fix turned out to be

[jira] [Resolved] (SPARK-8189) Use 100ns precision for TimestampType

2015-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-8189. Resolution: Fixed Fix Version/s: 1.5.0 > Use 100ns precision for TimestampType >

[jira] [Commented] (SPARK-7894) Graph Union Operator

2015-06-10 Thread shijinkui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581204#comment-14581204 ] shijinkui commented on SPARK-7894: -- hey [~ilganeli] there are no union in EdgeRDD and Ver

[jira] [Commented] (SPARK-8300) DataFrame hint for broadcast join

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581198#comment-14581198 ] Apache Spark commented on SPARK-8300: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-8300) DataFrame hint for broadcast join

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8300: --- Assignee: (was: Apache Spark) > DataFrame hint for broadcast join > -

[jira] [Commented] (SPARK-7893) Complex Operators between Graphs

2015-06-10 Thread shijinkui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581199#comment-14581199 ] shijinkui commented on SPARK-7893: -- OK > Complex Operators between Graphs >

[jira] [Assigned] (SPARK-8300) DataFrame hint for broadcast join

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8300: --- Assignee: Apache Spark > DataFrame hint for broadcast join >

[jira] [Updated] (SPARK-8300) DataFrame hint for broadcast join

2015-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8300: --- Summary: DataFrame hint for broadcast join (was: Hint for broadcast join) > DataFrame hint for broadc

[jira] [Created] (SPARK-8300) Hint for broadcast join

2015-06-10 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-8300: -- Summary: Hint for broadcast join Key: SPARK-8300 URL: https://issues.apache.org/jira/browse/SPARK-8300 Project: Spark Issue Type: Sub-task Components:

[jira] [Resolved] (SPARK-8299) Improve error message reporting for DataFrame and SQL

2015-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-8299. Resolution: Fixed Fix Version/s: 1.4.0 > Improve error message reporting for DataFrame and SQ

[jira] [Created] (SPARK-8299) Improve error message reporting for DataFrame and SQL

2015-06-10 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-8299: -- Summary: Improve error message reporting for DataFrame and SQL Key: SPARK-8299 URL: https://issues.apache.org/jira/browse/SPARK-8299 Project: Spark Issue Type: S

[jira] [Updated] (SPARK-8200) Exception in StreamingLinearAlgorithm on Stream with Empty RDD.

2015-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8200: - Assignee: Paavo Parkkinen > Exception in StreamingLinearAlgorithm on Stream with Empty RDD. >

[jira] [Resolved] (SPARK-8200) Exception in StreamingLinearAlgorithm on Stream with Empty RDD.

2015-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-8200. -- Resolution: Fixed Fix Version/s: 1.4.1 1.5.0 Issue resolved by pull request 67

[jira] [Resolved] (SPARK-2774) Set preferred locations for reduce tasks

2015-06-10 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-2774. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 6652 [https://git

[jira] [Created] (SPARK-8298) Sliding Window CrossValidator

2015-06-10 Thread Justin Yip (JIRA)
Justin Yip created SPARK-8298: - Summary: Sliding Window CrossValidator Key: SPARK-8298 URL: https://issues.apache.org/jira/browse/SPARK-8298 Project: Spark Issue Type: Improvement Compo

[jira] [Commented] (SPARK-1867) Spark Documentation Error causes java.lang.IllegalStateException: unread block data

2015-06-10 Thread Dev Lakhani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581130#comment-14581130 ] Dev Lakhani commented on SPARK-1867: Worth a new JIRA to suggest this? > Spark Docume

[jira] [Assigned] (SPARK-7186) Decouple internal Row from external Row

2015-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-7186: - Assignee: Davies Liu > Decouple internal Row from external Row >

[jira] [Updated] (SPARK-8296) Not able to load Dataframe using Python throws py4j.protocol.Py4JJavaError

2015-06-10 Thread ABHISHEK CHOUDHARY (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ABHISHEK CHOUDHARY updated SPARK-8296: -- Description: While trying to load a json file using sqlcontext in prebuilt spark-1.3.1-

[jira] [Commented] (SPARK-8297) Scheduler backend is not notified in case node fails in YARN

2015-06-10 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581038#comment-14581038 ] Mridul Muralidharan commented on SPARK-8297: Spark on mesos handles this situa

[jira] [Updated] (SPARK-8297) Scheduler backend is not notified in case node fails in YARN

2015-06-10 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-8297: --- Affects Version/s: 1.5.0 > Scheduler backend is not notified in case node fails in YAR

[jira] [Closed] (SPARK-8290) spark class command builder need read SPARK_JAVA_OPTS and SPARK_DRIVER_MEMORY properly

2015-06-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-8290. Resolution: Fixed Fix Version/s: 1.5.0 Assignee: Tao Wang Target Version/s: 1.5.

[jira] [Updated] (SPARK-8290) spark class command builder need read SPARK_JAVA_OPTS and SPARK_DRIVER_MEMORY properly

2015-06-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8290: - Affects Version/s: 1.3.0 > spark class command builder need read SPARK_JAVA_OPTS and SPARK_DRIVER_MEMORY

[jira] [Closed] (SPARK-8282) Make number of threads used in RBackend configurable

2015-06-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-8282. Resolution: Fixed Fix Version/s: 1.5.0 Assignee: Hossein Falaki Target Version/s

[jira] [Closed] (SPARK-5479) PySpark on yarn mode need to support non-local python files

2015-06-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-5479. Resolution: Fixed Fix Version/s: 1.5.0 Assignee: Marcelo Vanzin Target Version/s

[jira] [Closed] (SPARK-5162) Python yarn-cluster mode

2015-06-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-5162. Resolution: Fixed Fix Version/s: 1.4.0 Assignee: Lianhui Wang Target Version/s:

[jira] [Closed] (SPARK-7527) Wrong detection of REPL mode in ClosureCleaner

2015-06-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-7527. Resolution: Fixed Fix Version/s: 1.5.0 Assignee: Shixiong Zhu Target Version/s:

[jira] [Closed] (SPARK-7261) Change default log level to WARN in the REPL

2015-06-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-7261. Resolution: Fixed Fix Version/s: 1.5.0 > Change default log level to WARN in the REPL > -

[jira] [Updated] (SPARK-8297) Scheduler backend is not notified in case node fails in YARN

2015-06-10 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-8297: --- Assignee: (was: Mridul Muralidharan) > Scheduler backend is not notified in case n

[jira] [Created] (SPARK-8297) Scheduler backend is not notified in case node fails in YARN

2015-06-10 Thread Mridul Muralidharan (JIRA)
Mridul Muralidharan created SPARK-8297: -- Summary: Scheduler backend is not notified in case node fails in YARN Key: SPARK-8297 URL: https://issues.apache.org/jira/browse/SPARK-8297 Project: Spark

[jira] [Commented] (SPARK-8103) DAGScheduler should not launch multiple concurrent attempts for one stage on fetch failures

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581019#comment-14581019 ] Apache Spark commented on SPARK-8103: - User 'squito' has created a pull request for th

[jira] [Assigned] (SPARK-8103) DAGScheduler should not launch multiple concurrent attempts for one stage on fetch failures

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8103: --- Assignee: Imran Rashid (was: Apache Spark) > DAGScheduler should not launch multiple concurr

[jira] [Assigned] (SPARK-8103) DAGScheduler should not launch multiple concurrent attempts for one stage on fetch failures

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8103: --- Assignee: Apache Spark (was: Imran Rashid) > DAGScheduler should not launch multiple concurr

[jira] [Created] (SPARK-8296) Not able to load Dataframe using Python throws py4j.protocol.Py4JJavaError

2015-06-10 Thread ABHISHEK CHOUDHARY (JIRA)
ABHISHEK CHOUDHARY created SPARK-8296: - Summary: Not able to load Dataframe using Python throws py4j.protocol.Py4JJavaError Key: SPARK-8296 URL: https://issues.apache.org/jira/browse/SPARK-8296 Pr

[jira] [Updated] (SPARK-8295) SparkContext shut down in Spark Project Streaming test suite

2015-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8295: - Priority: Minor (was: Blocker) Definitely not a blocker, but worth tracking down if it can be reproduced

[jira] [Commented] (SPARK-1867) Spark Documentation Error causes java.lang.IllegalStateException: unread block data

2015-06-10 Thread sam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580957#comment-14580957 ] sam commented on SPARK-1867: [~devlakhani] I agree. I think I mentioned a billion comments ago

[jira] [Created] (SPARK-8295) SparkContext shut down in Spark Project Streaming test suite

2015-06-10 Thread Francois-Xavier Lemire (JIRA)
Francois-Xavier Lemire created SPARK-8295: - Summary: SparkContext shut down in Spark Project Streaming test suite Key: SPARK-8295 URL: https://issues.apache.org/jira/browse/SPARK-8295 Project:

[jira] [Updated] (SPARK-3492) Clean up Yarn integration code

2015-06-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3492: - Assignee: (was: Andrew Or) > Clean up Yarn integration code > -- > >

[jira] [Created] (SPARK-8294) Break down large methods in YARN code

2015-06-10 Thread Andrew Or (JIRA)
Andrew Or created SPARK-8294: Summary: Break down large methods in YARN code Key: SPARK-8294 URL: https://issues.apache.org/jira/browse/SPARK-8294 Project: Spark Issue Type: Sub-task Co

[jira] [Created] (SPARK-8293) Add high-level java docs to important YARN classes

2015-06-10 Thread Andrew Or (JIRA)
Andrew Or created SPARK-8293: Summary: Add high-level java docs to important YARN classes Key: SPARK-8293 URL: https://issues.apache.org/jira/browse/SPARK-8293 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-7996) Deprecate the developer api SparkEnv.actorSystem

2015-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580912#comment-14580912 ] Reynold Xin commented on SPARK-7996: [~ilganeli] that one is for later versions. > D

[jira] [Resolved] (SPARK-7996) Deprecate the developer api SparkEnv.actorSystem

2015-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-7996. Resolution: Fixed Fix Version/s: 1.5.0 Assignee: Ilya Ganelin > Deprecate the develo

[jira] [Resolved] (SPARK-7814) Turn code generation on by default

2015-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-7814. --- Resolution: Duplicate Fix Version/s: 1.5.0 > Turn code generation on by default > -

[jira] [Resolved] (SPARK-6419) GenerateOrdering does not support BinaryType and complex types.

2015-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-6419. --- Resolution: Fixed Fix Version/s: 1.5.0 > GenerateOrdering does not support BinaryType and compl

[jira] [Commented] (SPARK-6419) GenerateOrdering does not support BinaryType and complex types.

2015-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580900#comment-14580900 ] Davies Liu commented on SPARK-6419: --- Fixed by SPARK-7956 > GenerateOrdering does not su

[jira] [Assigned] (SPARK-8190) ExpressionEvalHelper.checkEvaluation should also run the optimizer version

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8190: --- Assignee: Davies Liu (was: Apache Spark) > ExpressionEvalHelper.checkEvaluation should also

[jira] [Commented] (SPARK-8190) ExpressionEvalHelper.checkEvaluation should also run the optimizer version

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580898#comment-14580898 ] Apache Spark commented on SPARK-8190: - User 'davies' has created a pull request for th

[jira] [Assigned] (SPARK-8190) ExpressionEvalHelper.checkEvaluation should also run the optimizer version

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8190: --- Assignee: Apache Spark (was: Davies Liu) > ExpressionEvalHelper.checkEvaluation should also

[jira] [Commented] (SPARK-8064) Upgrade Hive to 1.2

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580886#comment-14580886 ] Apache Spark commented on SPARK-8064: - User 'steveloughran' has created a pull request

[jira] [Assigned] (SPARK-8064) Upgrade Hive to 1.2

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8064: --- Assignee: Apache Spark (was: Steve Loughran) > Upgrade Hive to 1.2 > --- > >

[jira] [Assigned] (SPARK-8064) Upgrade Hive to 1.2

2015-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8064: --- Assignee: Steve Loughran (was: Apache Spark) > Upgrade Hive to 1.2 > --- > >

[jira] [Commented] (SPARK-6284) Support framework authentication and role in Mesos framework

2015-06-10 Thread DarinJ (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580780#comment-14580780 ] DarinJ commented on SPARK-6284: --- +1 we're currently patching spark with similar code to make

[jira] [Updated] (SPARK-8212) math function: e

2015-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8212: --- Labels: missing-python (was: ) > math function: e > > > Key: SPARK-8

[jira] [Updated] (SPARK-8215) math function: pi

2015-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8215: --- Labels: missing-python (was: ) > math function: pi > - > > Key: SPARK

[jira] [Resolved] (SPARK-8215) math function: pi

2015-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-8215. Resolution: Fixed Fix Version/s: 1.5.0 > math function: pi > - > >

[jira] [Updated] (SPARK-8290) spark class command builder need read SPARK_JAVA_OPTS and SPARK_DRIVER_MEMORY properly

2015-06-10 Thread Tao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Wang updated SPARK-8290: Description: SPARK_JAVA_OPTS was missed in reconstructing the launcher part, we should add it back so spark

  1   2   >