[jira] [Created] (SPARK-7273) The SQLContext.jsonFile() api has a problem when load a format json file?

2015-04-29 Thread steven (JIRA)
steven created SPARK-7273: - Summary: The SQLContext.jsonFile() api has a problem when load a format json file? Key: SPARK-7273 URL: https://issues.apache.org/jira/browse/SPARK-7273 Project: Spark Is

[jira] [Commented] (SPARK-7217) Add configuration to disable stopping of SparkContext when StreamingContext.stop()

2015-04-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520984#comment-14520984 ] Sean Owen commented on SPARK-7217: -- OK, up to your judgment, mostly food for thought here

[jira] [Resolved] (SPARK-7265) Improving documentation for Spark SQL Hive support

2015-04-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7265. -- Resolution: Invalid Fix Version/s: (was: 1.4.0) Please review https://cwiki.apache.org/confl

[jira] [Commented] (SPARK-1406) PMML model evaluation support via MLib

2015-04-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520967#comment-14520967 ] Xiangrui Meng commented on SPARK-1406: -- The PMML model export was partially addressed

[jira] [Commented] (SPARK-6443) Support HA in standalone cluster mode

2015-04-29 Thread pankaj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520965#comment-14520965 ] pankaj commented on SPARK-6443: --- here is the detailed exception for Spark-submit in Standalo

[jira] [Created] (SPARK-7272) User guide for PMML model export

2015-04-29 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-7272: Summary: User guide for PMML model export Key: SPARK-7272 URL: https://issues.apache.org/jira/browse/SPARK-7272 Project: Spark Issue Type: Documentation

[jira] [Resolved] (SPARK-1406) PMML model evaluation support via MLib

2015-04-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-1406. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 3062 [https://githu

[jira] [Comment Edited] (SPARK-1867) Spark Documentation Error causes java.lang.IllegalStateException: unread block data

2015-04-29 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520925#comment-14520925 ] meiyoula edited comment on SPARK-1867 at 4/30/15 6:17 AM: -- Hi al

[jira] [Comment Edited] (SPARK-1867) Spark Documentation Error causes java.lang.IllegalStateException: unread block data

2015-04-29 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520925#comment-14520925 ] meiyoula edited comment on SPARK-1867 at 4/30/15 6:16 AM: -- Hi al

[jira] [Commented] (SPARK-6241) hiveql ANALYZE TABLE doesn't work for external tables

2015-04-29 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520938#comment-14520938 ] Adrian Wang commented on SPARK-6241: The NPE has been resolved by SPARK-6575 > hiveql

[jira] [Commented] (SPARK-1867) Spark Documentation Error causes java.lang.IllegalStateException: unread block data

2015-04-29 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520925#comment-14520925 ] meiyoula commented on SPARK-1867: - Hi all, My cluster information is: /opt/jdk1.8.0_40, h

[jira] [Commented] (SPARK-1830) Deploy failover, Make Persistence engine and LeaderAgent Pluggable.

2015-04-29 Thread niranda perera (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520918#comment-14520918 ] niranda perera commented on SPARK-1830: --- I'm trying to implement a custom persistenc

[jira] [Commented] (SPARK-5456) Decimal Type comparison issue

2015-04-29 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520917#comment-14520917 ] Adrian Wang commented on SPARK-5456: Hi, I just tried the code in the description of t

[jira] [Resolved] (SPARK-7225) CombineLimits optimizer does not work

2015-04-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-7225. Resolution: Fixed Fix Version/s: 1.4.0 Assignee: Zhongshuai Pei > CombineLimits opti

[jira] [Assigned] (SPARK-7242) Frequent items for DataFrames

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7242: --- Assignee: Burak Yavuz (was: Apache Spark) > Frequent items for DataFrames >

[jira] [Commented] (SPARK-7242) Frequent items for DataFrames

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520891#comment-14520891 ] Apache Spark commented on SPARK-7242: - User 'brkyvz' has created a pull request for th

[jira] [Assigned] (SPARK-7242) Frequent items for DataFrames

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7242: --- Assignee: Apache Spark (was: Burak Yavuz) > Frequent items for DataFrames >

[jira] [Closed] (SPARK-7170) Allow to add register SparkListener specified in SparkConf

2015-04-29 Thread Jacek Lewandowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Lewandowski closed SPARK-7170. Resolution: Duplicate I found that this is already implemented. I don't know how I could mis

[jira] [Created] (SPARK-7271) Redesign shuffle interface for binary processing

2015-04-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-7271: -- Summary: Redesign shuffle interface for binary processing Key: SPARK-7271 URL: https://issues.apache.org/jira/browse/SPARK-7271 Project: Spark Issue Type: Improv

[jira] [Updated] (SPARK-7270) StringType dynamic partition cast to DecimalType in Spark Sql Hive

2015-04-29 Thread Feixiang Yan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feixiang Yan updated SPARK-7270: Description: Create a hive table with two partitons,the first type is bigint and the second type is

[jira] [Created] (SPARK-7270) StringType dynamic partition cast to DecimalType in Spark Sql Hive

2015-04-29 Thread Feixiang Yan (JIRA)
Feixiang Yan created SPARK-7270: --- Summary: StringType dynamic partition cast to DecimalType in Spark Sql Hive Key: SPARK-7270 URL: https://issues.apache.org/jira/browse/SPARK-7270 Project: Spark

[jira] [Commented] (SPARK-7149) Defalt system alias problem

2015-04-29 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520764#comment-14520764 ] Adrian Wang commented on SPARK-7149: what's the schema of your testData table? I have

[jira] [Assigned] (SPARK-7269) Incorrect aggregation analysis

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7269: --- Assignee: (was: Apache Spark) > Incorrect aggregation analysis >

[jira] [Assigned] (SPARK-7269) Incorrect aggregation analysis

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7269: --- Assignee: Apache Spark > Incorrect aggregation analysis > -- > >

[jira] [Commented] (SPARK-7269) Incorrect aggregation analysis

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520730#comment-14520730 ] Apache Spark commented on SPARK-7269: - User 'chenghao-intel' has created a pull reques

[jira] [Created] (SPARK-7269) Incorrect aggregation analysis

2015-04-29 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-7269: Summary: Incorrect aggregation analysis Key: SPARK-7269 URL: https://issues.apache.org/jira/browse/SPARK-7269 Project: Spark Issue Type: Bug Components: SQ

[jira] [Commented] (SPARK-7196) decimal precision lost when loading DataFrame from JDBC

2015-04-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520715#comment-14520715 ] Reynold Xin commented on SPARK-7196: [~kgeis] can you check if this https://github.com

[jira] [Resolved] (SPARK-7156) Add randomSplit method to DataFrame

2015-04-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-7156. Resolution: Fixed Fix Version/s: 1.4.0 Assignee: Burak Yavuz Target Vers

[jira] [Commented] (SPARK-7189) History server will always reload the same file even when no log file is updated

2015-04-29 Thread Zhang, Liye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520697#comment-14520697 ] Zhang, Liye commented on SPARK-7189: Just redundant work, and we can leave it. And I m

[jira] [Commented] (SPARK-7189) History server will always reload the same file even when no log file is updated

2015-04-29 Thread Zhang, Liye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520694#comment-14520694 ] Zhang, Liye commented on SPARK-7189: *>=* is always need if we use timestamp as one of

[jira] [Commented] (SPARK-7189) History server will always reload the same file even when no log file is updated

2015-04-29 Thread Zhang, Liye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520691#comment-14520691 ] Zhang, Liye commented on SPARK-7189: Yes, we can use file size to monitor the file cha

[jira] [Updated] (SPARK-7265) Improving documentation for Spark SQL Hive support

2015-04-29 Thread Jihong MA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jihong MA updated SPARK-7265: - Priority: Minor (was: Trivial) > Improving documentation for Spark SQL Hive support > --

[jira] [Assigned] (SPARK-7267) Push down Project when it's child is Limit

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7267: --- Assignee: (was: Apache Spark) > Push down Project when it's child is Limit > ---

[jira] [Commented] (SPARK-7267) Push down Project when it's child is Limit

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520676#comment-14520676 ] Apache Spark commented on SPARK-7267: - User 'DoingDone9' has created a pull request fo

[jira] [Assigned] (SPARK-7267) Push down Project when it's child is Limit

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7267: --- Assignee: Apache Spark > Push down Project when it's child is Limit > --

[jira] [Updated] (SPARK-6939) Refactoring existing batch statistics into the new UI

2015-04-29 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-6939: - Assignee: Shixiong Zhu > Refactoring existing batch statistics into the new UI > -

[jira] [Updated] (SPARK-7267) Push down Project when it's child is Limit

2015-04-29 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-7267: -- Description: SQL {quote} select key from (select key,value from t1 limit 100) t2 limit 10 {quote

[jira] [Resolved] (SPARK-7234) When codegen on DateType defaultPrimitive will throw type mismatch exception

2015-04-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-7234. Resolution: Fixed Fix Version/s: 1.4.0 1.3.2 Assignee: Chen Song

[jira] [Resolved] (SPARK-6862) Add BatchPage to display details about a batch

2015-04-29 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-6862. -- Resolution: Fixed > Add BatchPage to display details about a batch > ---

[jira] [Updated] (SPARK-7267) Push down Project when it's child is Limit

2015-04-29 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-7267: -- Description: SQL {quote} select key from (select key,value from t1 limit 100) t2 limit 10 {quote

[jira] [Comment Edited] (SPARK-7230) Make RDD API private in SparkR for Spark 1.4

2015-04-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520635#comment-14520635 ] Patrick Wendell edited comment on SPARK-7230 at 4/30/15 1:13 AM: ---

[jira] [Comment Edited] (SPARK-7230) Make RDD API private in SparkR for Spark 1.4

2015-04-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520635#comment-14520635 ] Patrick Wendell edited comment on SPARK-7230 at 4/30/15 1:13 AM: ---

[jira] [Created] (SPARK-7268) [Spark SQL] Throw 'Shutdown hooks cannot be modified during shutdown' on YARN

2015-04-29 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-7268: -- Summary: [Spark SQL] Throw 'Shutdown hooks cannot be modified during shutdown' on YARN Key: SPARK-7268 URL: https://issues.apache.org/jira/browse/SPARK-7268 Project: Spark

[jira] [Updated] (SPARK-7228) SparkR public API for 1.4 release

2015-04-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7228: --- Priority: Blocker (was: Critical) > SparkR public API for 1.4 release > -

[jira] [Created] (SPARK-7267) Push down Project when it's child is Limit

2015-04-29 Thread Zhongshuai Pei (JIRA)
Zhongshuai Pei created SPARK-7267: - Summary: Push down Project when it's child is Limit Key: SPARK-7267 URL: https://issues.apache.org/jira/browse/SPARK-7267 Project: Spark Issue Type: Impro

[jira] [Updated] (SPARK-7266) Add ExpectsInputTypes to expressions whenever possible

2015-04-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7266: --- Description: This should gives us better analysis time error messages (rather than runtime) and autom

[jira] [Commented] (SPARK-7266) Add ExpectsInputTypes to expressions whenever possible

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520638#comment-14520638 ] Apache Spark commented on SPARK-7266: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-7266) Add ExpectsInputTypes to expressions whenever possible

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7266: --- Assignee: Reynold Xin (was: Apache Spark) > Add ExpectsInputTypes to expressions whenever po

[jira] [Assigned] (SPARK-6907) Create an isolated classloader for the Hive Client.

2015-04-29 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-6907: --- Assignee: Michael Armbrust > Create an isolated classloader for the Hive Client. > --

[jira] [Assigned] (SPARK-7266) Add ExpectsInputTypes to expressions whenever possible

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7266: --- Assignee: Apache Spark (was: Reynold Xin) > Add ExpectsInputTypes to expressions whenever po

[jira] [Commented] (SPARK-7230) Make RDD API private in SparkR for Spark 1.4

2015-04-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520635#comment-14520635 ] Patrick Wendell commented on SPARK-7230: Yes - removing API's is really difficult

[jira] [Created] (SPARK-7266) Add ExpectsInputTypes to expressions whenever possible

2015-04-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-7266: -- Summary: Add ExpectsInputTypes to expressions whenever possible Key: SPARK-7266 URL: https://issues.apache.org/jira/browse/SPARK-7266 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-7176) Add validation functionality to individual Param

2015-04-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-7176. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5740 [https://githu

[jira] [Created] (SPARK-7265) Improving documentation for Spark SQL Hive support

2015-04-29 Thread Jihong MA (JIRA)
Jihong MA created SPARK-7265: Summary: Improving documentation for Spark SQL Hive support Key: SPARK-7265 URL: https://issues.apache.org/jira/browse/SPARK-7265 Project: Spark Issue Type: Documen

[jira] [Commented] (SPARK-7156) Add randomSplit method to DataFrame

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520551#comment-14520551 ] Apache Spark commented on SPARK-7156: - User 'brkyvz' has created a pull request for th

[jira] [Resolved] (SPARK-7259) VectorIndexer: do not preserve non-ML metadata in output

2015-04-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-7259. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5789 [https

[jira] [Resolved] (SPARK-7229) SpecificMutableRow should take integer type as internal representation for DateType

2015-04-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-7229. Resolution: Fixed Fix Version/s: 1.4.0 1.3.2 Assignee: Cheng Hao

[jira] [Commented] (SPARK-7075) Project Tungsten: Improving Physical Execution and Memory Management

2015-04-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520499#comment-14520499 ] Reynold Xin commented on SPARK-7075: Yup I will post more thoughts and plans in the ne

[jira] [Commented] (SPARK-7217) Add configuration to disable stopping of SparkContext when StreamingContext.stop()

2015-04-29 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520493#comment-14520493 ] Tathagata Das commented on SPARK-7217: -- I get the analogy now. I get your point. But

[jira] [Commented] (SPARK-7230) Make RDD API private in SparkR for Spark 1.4

2015-04-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520486#comment-14520486 ] Reynold Xin commented on SPARK-7230: The existing SparkR package is still out there th

[jira] [Commented] (SPARK-7230) Make RDD API private in SparkR for Spark 1.4

2015-04-29 Thread Antonio Piccolboni (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520479#comment-14520479 ] Antonio Piccolboni commented on SPARK-7230: --- If you make a call private, you br

[jira] [Commented] (SPARK-7230) Make RDD API private in SparkR for Spark 1.4

2015-04-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520478#comment-14520478 ] Patrick Wendell commented on SPARK-7230: Yeah the goal is absolutely to support hi

[jira] [Commented] (SPARK-7261) Change default log level to WARN in the REPL

2015-04-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520467#comment-14520467 ] Patrick Wendell commented on SPARK-7261: Yeah, but SPARK-7260 is super simple, I t

[jira] [Commented] (SPARK-7230) Make RDD API private in SparkR for Spark 1.4

2015-04-29 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520465#comment-14520465 ] Shivaram Venkataraman commented on SPARK-7230: -- [~piccolbo] Thanks for your i

[jira] [Resolved] (SPARK-7155) SparkContext's newAPIHadoopFile does not support comma-separated list of files, but the other API hadoopFile does.

2015-04-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7155. -- Resolution: Fixed Fix Version/s: 1.4.0 1.3.2 Assignee: Yong Tang > Sp

[jira] [Commented] (SPARK-7155) SparkContext's newAPIHadoopFile does not support comma-separated list of files, but the other API hadoopFile does.

2015-04-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520463#comment-14520463 ] Sean Owen commented on SPARK-7155: -- Resolved by https://github.com/apache/spark/pull/5708

[jira] [Created] (SPARK-7264) SparkR API for parallel functions

2015-04-29 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-7264: Summary: SparkR API for parallel functions Key: SPARK-7264 URL: https://issues.apache.org/jira/browse/SPARK-7264 Project: Spark Issue Type: N

[jira] [Updated] (SPARK-7181) External Sorter merge with aggregation go to an infinite loop when we have a total ordering

2015-04-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7181: - Assignee: Qiping Li > External Sorter merge with aggregation go to an infinite loop when we have a > tota

[jira] [Resolved] (SPARK-7181) External Sorter merge with aggregation go to an infinite loop when we have a total ordering

2015-04-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7181. -- Resolution: Fixed Fix Version/s: 1.2.3 Issue resolved by pull request 5737 [https://github.com/ap

[jira] [Commented] (SPARK-5529) BlockManager heartbeat expiration does not kill executor

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520450#comment-14520450 ] Apache Spark commented on SPARK-5529: - User 'alexrovner' has created a pull request fo

[jira] [Commented] (SPARK-7249) Updated Hadoop dependencies due to inconsistency in the versions

2015-04-29 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-7249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520445#comment-14520445 ] Favio Vázquez commented on SPARK-7249: -- Thanks for let me know that, I wasn't sure ho

[jira] [Updated] (SPARK-7263) Add new shuffle manager which stores shuffle blocks in Parquet

2015-04-29 Thread Matt Massie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Massie updated SPARK-7263: --- Issue Type: New Feature (was: Improvement) > Add new shuffle manager which stores shuffle blocks in P

[jira] [Updated] (SPARK-7263) Add new shuffle manager which stores shuffle blocks in Parquet

2015-04-29 Thread Matt Massie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Massie updated SPARK-7263: --- Description: I have a working prototype of this feature that can be viewed at https://github.com/apach

[jira] [Assigned] (SPARK-7249) Updated Hadoop dependencies due to inconsistency in the versions

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7249: --- Assignee: Apache Spark > Updated Hadoop dependencies due to inconsistency in the versions >

[jira] [Commented] (SPARK-7263) Add new shuffle manager which stores shuffle blocks in Parquet

2015-04-29 Thread Matt Massie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520439#comment-14520439 ] Matt Massie commented on SPARK-7263: I'm happy to accept PRs against my fork of the Sp

[jira] [Assigned] (SPARK-7249) Updated Hadoop dependencies due to inconsistency in the versions

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7249: --- Assignee: (was: Apache Spark) > Updated Hadoop dependencies due to inconsistency in the

[jira] [Commented] (SPARK-7249) Updated Hadoop dependencies due to inconsistency in the versions

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520440#comment-14520440 ] Apache Spark commented on SPARK-7249: - User 'FavioVazquez' has created a pull request

[jira] [Comment Edited] (SPARK-7111) Add a tracker to track the direct (receiver-less) streams

2015-04-29 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520432#comment-14520432 ] Tathagata Das edited comment on SPARK-7111 at 4/29/15 10:40 PM:

[jira] [Commented] (SPARK-7111) Add a tracker to track the direct (receiver-less) streams

2015-04-29 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520432#comment-14520432 ] Tathagata Das commented on SPARK-7111: -- Here is the design that I propose. If this lo

[jira] [Commented] (SPARK-7189) History server will always reload the same file even when no log file is updated

2015-04-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520428#comment-14520428 ] Marcelo Vanzin commented on SPARK-7189: --- Just redundant work. Not a big deal for sma

[jira] [Commented] (SPARK-7189) History server will always reload the same file even when no log file is updated

2015-04-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520426#comment-14520426 ] Sean Owen commented on SPARK-7189: -- What's the downside of parsing them anyway, just a li

[jira] [Commented] (SPARK-7230) Make RDD API private in SparkR for Spark 1.4

2015-04-29 Thread Antonio Piccolboni (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520424#comment-14520424 ] Antonio Piccolboni commented on SPARK-7230: --- plyrmr on spark depends on the RDD

[jira] [Commented] (SPARK-7189) History server will always reload the same file even when no log file is updated

2015-04-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520421#comment-14520421 ] Marcelo Vanzin commented on SPARK-7189: --- Correct, {{>=}} is still required. The tric

[jira] [Assigned] (SPARK-7161) Provide REST api to download event logs from History Server

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7161: --- Assignee: Apache Spark > Provide REST api to download event logs from History Server > --

[jira] [Assigned] (SPARK-7161) Provide REST api to download event logs from History Server

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7161: --- Assignee: (was: Apache Spark) > Provide REST api to download event logs from History Serv

[jira] [Commented] (SPARK-7161) Provide REST api to download event logs from History Server

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520420#comment-14520420 ] Apache Spark commented on SPARK-7161: - User 'harishreedharan' has created a pull reque

[jira] [Commented] (SPARK-7189) History server will always reload the same file even when no log file is updated

2015-04-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520417#comment-14520417 ] Sean Owen commented on SPARK-7189: -- So is the outcome still that the check needs to be >=

[jira] [Commented] (SPARK-7255) spark.streaming.kafka.maxRetries not documented

2015-04-29 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520403#comment-14520403 ] Cody Koeninger commented on SPARK-7255: --- I think that should be fine to document, th

[jira] [Assigned] (SPARK-3444) Provide a way to easily change the log level in the Spark shell while running

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3444: --- Assignee: Apache Spark (was: Holden Karau) > Provide a way to easily change the log level in

[jira] [Commented] (SPARK-3444) Provide a way to easily change the log level in the Spark shell while running

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520383#comment-14520383 ] Apache Spark commented on SPARK-3444: - User 'holdenk' has created a pull request for t

[jira] [Assigned] (SPARK-3444) Provide a way to easily change the log level in the Spark shell while running

2015-04-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3444: --- Assignee: Holden Karau (was: Apache Spark) > Provide a way to easily change the log level in

[jira] [Created] (SPARK-7263) Add new shuffle manager which store shuffle block in Parquet

2015-04-29 Thread Matt Massie (JIRA)
Matt Massie created SPARK-7263: -- Summary: Add new shuffle manager which store shuffle block in Parquet Key: SPARK-7263 URL: https://issues.apache.org/jira/browse/SPARK-7263 Project: Spark Issue

[jira] [Updated] (SPARK-7263) Add new shuffle manager which stores shuffle blocks in Parquet

2015-04-29 Thread Matt Massie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Massie updated SPARK-7263: --- Summary: Add new shuffle manager which stores shuffle blocks in Parquet (was: Add new shuffle manager

[jira] [Updated] (SPARK-7262) LogisticRegression with L1/L2 (elastic net) using OWLQN in new ML package

2015-04-29 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-7262: --- Issue Type: New Feature (was: Bug) > LogisticRegression with L1/L2 (elastic net) using OWLQN in new ML packag

[jira] [Updated] (SPARK-7249) Updated Hadoop dependencies due to inconsistency in the versions

2015-04-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7249: - Priority: Minor (was: Blocker) Affects Version/s: (was: 1.3.1) > Updated Hadoop depende

[jira] [Created] (SPARK-7262) LogisticRegression with L1/L2 (elastic net) using OWLQN in new ML package

2015-04-29 Thread DB Tsai (JIRA)
DB Tsai created SPARK-7262: -- Summary: LogisticRegression with L1/L2 (elastic net) using OWLQN in new ML package Key: SPARK-7262 URL: https://issues.apache.org/jira/browse/SPARK-7262 Project: Spark

[jira] [Updated] (SPARK-7249) Updated Hadoop dependencies due to inconsistency in the versions

2015-04-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7249: - Target Version/s: (was: 1.3.1) Fix Version/s: (was: 1.3.1) Don't set Blocker priority; this d

[jira] [Commented] (SPARK-7233) ClosureCleaner#clean blocks concurrent job submitter threads

2015-04-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520368#comment-14520368 ] Patrick Wendell commented on SPARK-7233: I think we should just create a lazy val

[jira] [Commented] (SPARK-7261) Change default log level to WARN in the REPL

2015-04-29 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520366#comment-14520366 ] Matei Zaharia commented on SPARK-7261: -- IMO we can do this even without SPARK-7260 in

[jira] [Commented] (SPARK-7217) Add configuration to disable stopping of SparkContext when StreamingContext.stop()

2015-04-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520364#comment-14520364 ] Sean Owen commented on SPARK-7217: -- [~tdas] I get that. You're saying the current behavio

  1   2   3   4   >