[jira] [Updated] (SPARK-16247) Using pyspark dataframe with pipeline and cross validator

2016-06-29 Thread Edward Ma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Ma updated SPARK-16247: -- Description: I am using pyspark with dataframe. Using pipeline operation to train and predict the resu

[jira] [Commented] (SPARK-16247) Using pyspark dataframe with pipeline and cross validator

2016-06-29 Thread Edward Ma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354707#comment-15354707 ] Edward Ma commented on SPARK-16247: --- Thank you for your comment. Yes, it should be ml's

[jira] [Comment Edited] (SPARK-16247) Using pyspark dataframe with pipeline and cross validator

2016-06-29 Thread Edward Ma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354707#comment-15354707 ] Edward Ma edited comment on SPARK-16247 at 6/29/16 7:09 AM: T

[jira] [Created] (SPARK-16291) Invalid aggregate functions like MAX(COUNT(*)) are not captured by CheckAnalysis

2016-06-29 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16291: -- Summary: Invalid aggregate functions like MAX(COUNT(*)) are not captured by CheckAnalysis Key: SPARK-16291 URL: https://issues.apache.org/jira/browse/SPARK-16291 Project:

[jira] [Assigned] (SPARK-16278) Implement map_keys SQL function

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16278: Assignee: (was: Apache Spark) > Implement map_keys SQL function >

[jira] [Assigned] (SPARK-16279) Implement map_values SQL function

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16279: Assignee: Apache Spark > Implement map_values SQL function > -

[jira] [Commented] (SPARK-16279) Implement map_values SQL function

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354772#comment-15354772 ] Apache Spark commented on SPARK-16279: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Commented] (SPARK-16278) Implement map_keys SQL function

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354770#comment-15354770 ] Apache Spark commented on SPARK-16278: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Assigned] (SPARK-16279) Implement map_values SQL function

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16279: Assignee: (was: Apache Spark) > Implement map_values SQL function > --

[jira] [Assigned] (SPARK-16278) Implement map_keys SQL function

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16278: Assignee: Apache Spark > Implement map_keys SQL function > ---

[jira] [Commented] (SPARK-16279) Implement map_values SQL function

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354771#comment-15354771 ] Apache Spark commented on SPARK-16279: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Assigned] (SPARK-16279) Implement map_values SQL function

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16279: Assignee: Apache Spark > Implement map_values SQL function > -

[jira] [Commented] (SPARK-16273) NoSuchMethodError on SparkContext.rddToPairRDDFunctions

2016-06-29 Thread Fu Zhouwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354769#comment-15354769 ] Fu Zhouwang commented on SPARK-16273: - Thank you, it works! > NoSuchMethodError on S

[jira] [Commented] (SPARK-16289) Implement posexplode table generating function

2016-06-29 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354797#comment-15354797 ] Dongjoon Hyun commented on SPARK-16289: --- I'll work on this issue. > Implement pose

[jira] [Closed] (SPARK-16273) NoSuchMethodError on SparkContext.rddToPairRDDFunctions

2016-06-29 Thread Fu Zhouwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fu Zhouwang closed SPARK-16273. --- Resolution: Not A Bug > NoSuchMethodError on SparkContext.rddToPairRDDFunctions > ---

[jira] [Assigned] (SPARK-16291) Invalid aggregate functions like MAX(COUNT(*)) are not captured by CheckAnalysis

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16291: Assignee: Apache Spark (was: Cheng Lian) > Invalid aggregate functions like MAX(COUNT(*))

[jira] [Commented] (SPARK-16291) Invalid aggregate functions like MAX(COUNT(*)) are not captured by CheckAnalysis

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354819#comment-15354819 ] Apache Spark commented on SPARK-16291: -- User 'liancheng' has created a pull request

[jira] [Assigned] (SPARK-16291) Invalid aggregate functions like MAX(COUNT(*)) are not captured by CheckAnalysis

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16291: Assignee: Cheng Lian (was: Apache Spark) > Invalid aggregate functions like MAX(COUNT(*))

[jira] [Commented] (SPARK-15731) orc writer directory permissions

2016-06-29 Thread Ran Haim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354851#comment-15354851 ] Ran Haim commented on SPARK-15731: -- I have tested it under 1.6.1, and writing the data u

[jira] [Comment Edited] (SPARK-15731) orc writer directory permissions

2016-06-29 Thread Ran Haim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354851#comment-15354851 ] Ran Haim edited comment on SPARK-15731 at 6/29/16 9:14 AM: --- I h

[jira] [Updated] (SPARK-16265) Add option to SparkSubmit to ship driver JRE to YARN

2016-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16265: -- Fix Version/s: (was: 2.1.0) > Add option to SparkSubmit to ship driver JRE to YARN > --

[jira] [Created] (SPARK-16292) Failed to create spark client

2016-06-29 Thread Arcflash (JIRA)
Arcflash created SPARK-16292: Summary: Failed to create spark client Key: SPARK-16292 URL: https://issues.apache.org/jira/browse/SPARK-16292 Project: Spark Issue Type: Bug Components: S

[jira] [Assigned] (SPARK-16284) Implement reflect SQL function

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16284: Assignee: Apache Spark > Implement reflect SQL function > -- >

[jira] [Assigned] (SPARK-16284) Implement reflect SQL function

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16284: Assignee: (was: Apache Spark) > Implement reflect SQL function > -

[jira] [Resolved] (SPARK-16266) Move python DataStreamReader/Writer from pyspark.sql to pyspark.sql.streaming package

2016-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16266. --- Resolution: Fixed Resolved by https://github.com/apache/spark/pull/13955 > Move python DataStreamRea

[jira] [Resolved] (SPARK-16290) text type features column for classification

2016-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16290. --- Resolution: Invalid Fix Version/s: (was: 1.6.3) Target Version/s: (was: 1.6.2)

[jira] [Commented] (SPARK-16292) Failed to create spark client

2016-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354867#comment-15354867 ] Sean Owen commented on SPARK-16292: --- NoSuchFieldError and NoSuchMethodError are in gene

[jira] [Created] (SPARK-16293) SparkAppHandle.getState() returns wrong state in standalone mode if Spark application terminates unexpectedly

2016-06-29 Thread Gunnar Schulze (JIRA)
Gunnar Schulze created SPARK-16293: -- Summary: SparkAppHandle.getState() returns wrong state in standalone mode if Spark application terminates unexpectedly Key: SPARK-16293 URL: https://issues.apache.org/jira/bro

[jira] [Commented] (SPARK-16293) SparkAppHandle.getState() returns wrong state in standalone mode if Spark application terminates unexpectedly

2016-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354889#comment-15354889 ] Sean Owen commented on SPARK-16293: --- I think this duplicates SPARK-16095 > SparkAppHan

[jira] [Assigned] (SPARK-16289) Implement posexplode table generating function

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16289: Assignee: Apache Spark > Implement posexplode table generating function >

[jira] [Commented] (SPARK-16289) Implement posexplode table generating function

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354987#comment-15354987 ] Apache Spark commented on SPARK-16289: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Assigned] (SPARK-16289) Implement posexplode table generating function

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16289: Assignee: (was: Apache Spark) > Implement posexplode table generating function > -

[jira] [Commented] (SPARK-12329) spark-sql prints out SET commands to stdout instead of stderr

2016-06-29 Thread Kevin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355000#comment-15355000 ] Kevin commented on SPARK-12329: --- Has this been fixed so far? > spark-sql prints out SET co

[jira] [Commented] (SPARK-15944) Make spark.ml package backward compatible with spark.mllib vectors

2016-06-29 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355014#comment-15355014 ] Nick Pentreath commented on SPARK-15944: Just checking - we don't have {{asML}} /

[jira] [Commented] (SPARK-16288) Implement inline table generating function

2016-06-29 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355017#comment-15355017 ] Dongjoon Hyun commented on SPARK-16288: --- I'll start to work this issue, too. > Imp

[jira] [Created] (SPARK-16294) Labelling support for the include_example Jekyll plugin

2016-06-29 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16294: -- Summary: Labelling support for the include_example Jekyll plugin Key: SPARK-16294 URL: https://issues.apache.org/jira/browse/SPARK-16294 Project: Spark Issue Typ

[jira] [Commented] (SPARK-16235) "evaluateEachIteration" is returning wrong results when calculated for classification model.

2016-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355020#comment-15355020 ] Sean Owen commented on SPARK-16235: --- I take it back -- MSE for probabilities is just Br

[jira] [Comment Edited] (SPARK-15944) Make spark.ml package backward compatible with spark.mllib vectors

2016-06-29 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355014#comment-15355014 ] Nick Pentreath edited comment on SPARK-15944 at 6/29/16 11:12 AM: -

[jira] [Updated] (SPARK-16294) Labelling support for the include_example Jekyll plugin

2016-06-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16294: --- Description: Part of the Spark programming guide pages are using the {{include_example}} Jekyll plug

[jira] [Updated] (SPARK-16294) Labelling support for the include_example Jekyll plugin

2016-06-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16294: --- Description: Part of the Spark programming guide pages are using the {{include_example}} Jekyll plug

[jira] [Created] (SPARK-16295) Extract SQL programming guide example snippets from source files instead of hard code them

2016-06-29 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16295: -- Summary: Extract SQL programming guide example snippets from source files instead of hard code them Key: SPARK-16295 URL: https://issues.apache.org/jira/browse/SPARK-16295

[jira] [Updated] (SPARK-16294) Labelling support for the include_example Jekyll plugin

2016-06-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16294: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-16295 > Labelling support for the incl

[jira] [Updated] (SPARK-16295) Extract SQL programming guide example snippets from source files instead of hard code them

2016-06-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16295: --- Description: Currently, all example snippets in the SQL programming guide are hard-coded, which can b

[jira] [Resolved] (SPARK-16291) Invalid aggregate functions like MAX(COUNT(*)) are not captured by CheckAnalysis

2016-06-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-16291. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 2.0.0 >

[jira] [Assigned] (SPARK-16294) Labelling support for the include_example Jekyll plugin

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16294: Assignee: Apache Spark (was: Cheng Lian) > Labelling support for the include_example Jeky

[jira] [Updated] (SPARK-16157) Add New Methods for Comments in StructField and StructType

2016-06-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-16157: Assignee: Xiao Li > Add New Methods for Comments in StructField and StructType > --

[jira] [Assigned] (SPARK-16294) Labelling support for the include_example Jekyll plugin

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16294: Assignee: Cheng Lian (was: Apache Spark) > Labelling support for the include_example Jeky

[jira] [Commented] (SPARK-16294) Labelling support for the include_example Jekyll plugin

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355077#comment-15355077 ] Apache Spark commented on SPARK-16294: -- User 'liancheng' has created a pull request

[jira] [Resolved] (SPARK-16157) Add New Methods for Comments in StructField and StructType

2016-06-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-16157. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Add New Methods for Comm

[jira] [Assigned] (SPARK-16182) Utils.scala -- terminateProcess() should call Process.destroyForcibly() if and only if Process.destroy() fails

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16182: Assignee: Apache Spark > Utils.scala -- terminateProcess() should call Process.destroyForc

[jira] [Assigned] (SPARK-16182) Utils.scala -- terminateProcess() should call Process.destroyForcibly() if and only if Process.destroy() fails

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16182: Assignee: (was: Apache Spark) > Utils.scala -- terminateProcess() should call Process.

[jira] [Commented] (SPARK-16182) Utils.scala -- terminateProcess() should call Process.destroyForcibly() if and only if Process.destroy() fails

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355100#comment-15355100 ] Apache Spark commented on SPARK-16182: -- User 'srowen' has created a pull request for

[jira] [Resolved] (SPARK-16250) Can't use escapeQuotes option in DataFrameWriter.csv()

2016-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16250. --- Resolution: Duplicate > Can't use escapeQuotes option in DataFrameWriter.csv() >

[jira] [Updated] (SPARK-16261) Fix Incorrect appNames in PySpark ML Examples

2016-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16261: -- Assignee: Bryan Cutler Priority: Trivial (was: Minor) > Fix Incorrect appNames in PySpark ML Examp

[jira] [Resolved] (SPARK-16261) Fix Incorrect appNames in PySpark ML Examples

2016-06-29 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-16261. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Fix Incorrect appN

[jira] [Resolved] (SPARK-15858) "evaluateEachIteration" will fail on trying to run it on a model with 500+ trees

2016-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15858. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13624 [https://github.co

[jira] [Updated] (SPARK-15858) "evaluateEachIteration" will fail on trying to run it on a model with 500+ trees

2016-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15858: -- Assignee: Mahmoud Rawas > "evaluateEachIteration" will fail on trying to run it on a model with 500+ >

[jira] [Commented] (SPARK-15829) spark master webpage links to application UI broke when running in cluster mode

2016-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355121#comment-15355121 ] Sean Owen commented on SPARK-15829: --- I think this is a question of whether you are usin

[jira] [Resolved] (SPARK-15918) unionAll returns wrong result when two dataframes has schema in different order

2016-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15918. --- Resolution: Not A Problem > unionAll returns wrong result when two dataframes has schema in different

[jira] [Created] (SPARK-16296) add null check for key when create map data in encoder

2016-06-29 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-16296: --- Summary: add null check for key when create map data in encoder Key: SPARK-16296 URL: https://issues.apache.org/jira/browse/SPARK-16296 Project: Spark Issue Ty

[jira] [Commented] (SPARK-16296) add null check for key when create map data in encoder

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355130#comment-15355130 ] Apache Spark commented on SPARK-16296: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-16296) add null check for key when create map data in encoder

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16296: Assignee: Wenchen Fan (was: Apache Spark) > add null check for key when create map data i

[jira] [Assigned] (SPARK-16296) add null check for key when create map data in encoder

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16296: Assignee: Apache Spark (was: Wenchen Fan) > add null check for key when create map data i

[jira] [Commented] (SPARK-13587) Support virtualenv in PySpark

2016-06-29 Thread Semet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355172#comment-15355172 ] Semet commented on SPARK-13587: --- yes it looks cool! Here is what I have in mind, tell me if

[jira] [Comment Edited] (SPARK-13587) Support virtualenv in PySpark

2016-06-29 Thread Semet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355172#comment-15355172 ] Semet edited comment on SPARK-13587 at 6/29/16 12:52 PM: - yes it

[jira] [Created] (SPARK-16297) Mapping Boolean and string to BIT and NVARCHAR(MAX) for SQL Server jdbc dialect

2016-06-29 Thread Oussama Mekni (JIRA)
Oussama Mekni created SPARK-16297: - Summary: Mapping Boolean and string to BIT and NVARCHAR(MAX) for SQL Server jdbc dialect Key: SPARK-16297 URL: https://issues.apache.org/jira/browse/SPARK-16297 Pr

[jira] [Updated] (SPARK-15990) Support rolling log aggregation for Spark running on YARN

2016-06-29 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-15990: -- Assignee: Saisai Shao > Support rolling log aggregation for Spark running on YARN > ---

[jira] [Commented] (SPARK-16280) Implement histogram_numeric SQL function

2016-06-29 Thread MIN-FU YANG (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355220#comment-15355220 ] MIN-FU YANG commented on SPARK-16280: - I can working on this > Implement histogram_n

[jira] [Resolved] (SPARK-15990) Support rolling log aggregation for Spark running on YARN

2016-06-29 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-15990. --- Resolution: Fixed Fix Version/s: 2.1.0 > Support rolling log aggregation for Spark run

[jira] [Assigned] (SPARK-16297) Mapping Boolean and string to BIT and NVARCHAR(MAX) for SQL Server jdbc dialect

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16297: Assignee: (was: Apache Spark) > Mapping Boolean and string to BIT and NVARCHAR(MAX) f

[jira] [Commented] (SPARK-16282) Implement percentile SQL function

2016-06-29 Thread MIN-FU YANG (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355228#comment-15355228 ] MIN-FU YANG commented on SPARK-16282: - I can work on this, too. > Implement percenti

[jira] [Commented] (SPARK-16297) Mapping Boolean and string to BIT and NVARCHAR(MAX) for SQL Server jdbc dialect

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355227#comment-15355227 ] Apache Spark commented on SPARK-16297: -- User 'meknio' has created a pull request for

[jira] [Assigned] (SPARK-16297) Mapping Boolean and string to BIT and NVARCHAR(MAX) for SQL Server jdbc dialect

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16297: Assignee: Apache Spark > Mapping Boolean and string to BIT and NVARCHAR(MAX) for SQL Serv

[jira] [Updated] (SPARK-16297) Mapping Boolean and string to BIT and NVARCHAR(MAX) for SQL Server jdbc dialect

2016-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16297: -- Flags: (was: Patch,Important) Target Version/s: (was: 1.6.0) External issue ID:

[jira] [Created] (SPARK-16298) spark.yarn.principal not working

2016-06-29 Thread Partha Pratim Ghosh (JIRA)
Partha Pratim Ghosh created SPARK-16298: --- Summary: spark.yarn.principal not working Key: SPARK-16298 URL: https://issues.apache.org/jira/browse/SPARK-16298 Project: Spark Issue Type: Bu

[jira] [Resolved] (SPARK-12737) Decrease the redundant activeIds sent to remote mirrors in "aggregateMessagesWithActiveSet"

2016-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12737. --- Resolution: Won't Fix > Decrease the redundant activeIds sent to remote mirrors in > "aggregateMessa

[jira] [Commented] (SPARK-16287) Implement str_to_map SQL function

2016-06-29 Thread Sandeep Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355262#comment-15355262 ] Sandeep Singh commented on SPARK-16287: --- I can work on this. > Implement str_to_m

[jira] [Issue Comment Deleted] (SPARK-16287) Implement str_to_map SQL function

2016-06-29 Thread Sandeep Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-16287: -- Comment: was deleted (was: I can work on this. ) > Implement str_to_map SQL function > ---

[jira] [Commented] (SPARK-16287) Implement str_to_map SQL function

2016-06-29 Thread Sandeep Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355289#comment-15355289 ] Sandeep Singh commented on SPARK-16287: --- I can work on this. > Implement str_to_

[jira] [Commented] (SPARK-16297) Mapping Boolean and string to BIT and NVARCHAR(MAX) for SQL Server jdbc dialect

2016-06-29 Thread Oussama Mekni (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355325#comment-15355325 ] Oussama Mekni commented on SPARK-16297: --- Thanks for the guide to using JIRA. I thi

[jira] [Commented] (SPARK-16230) Executors self-killing after being assigned tasks while still in init

2016-06-29 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355339#comment-15355339 ] Tejas Patil commented on SPARK-16230: - I am using 1.6.1. I will try out with the fix

[jira] [Updated] (SPARK-16297) Mapping Boolean and string to BIT and NVARCHAR(MAX) for SQL Server jdbc dialect

2016-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16297: -- Issue Type: Bug (was: Improvement) > Mapping Boolean and string to BIT and NVARCHAR(MAX) for SQL Serv

[jira] [Created] (SPARK-16299) Capture errors from R workers in daemon.R to avoid deletion of R session temporary directory

2016-06-29 Thread Sun Rui (JIRA)
Sun Rui created SPARK-16299: --- Summary: Capture errors from R workers in daemon.R to avoid deletion of R session temporary directory Key: SPARK-16299 URL: https://issues.apache.org/jira/browse/SPARK-16299 Pr

[jira] [Created] (SPARK-16300) Capture errors from R workers in daemon.R to avoid deletion of R session temporary directory

2016-06-29 Thread Sun Rui (JIRA)
Sun Rui created SPARK-16300: --- Summary: Capture errors from R workers in daemon.R to avoid deletion of R session temporary directory Key: SPARK-16300 URL: https://issues.apache.org/jira/browse/SPARK-16300 Pr

[jira] [Commented] (SPARK-16247) Using pyspark dataframe with pipeline and cross validator

2016-06-29 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355456#comment-15355456 ] Bryan Cutler commented on SPARK-16247: -- I think you need to specify the {labelCol} i

[jira] [Commented] (SPARK-16299) Capture errors from R workers in daemon.R to avoid deletion of R session temporary directory

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355462#comment-15355462 ] Apache Spark commented on SPARK-16299: -- User 'sun-rui' has created a pull request fo

[jira] [Comment Edited] (SPARK-16247) Using pyspark dataframe with pipeline and cross validator

2016-06-29 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355456#comment-15355456 ] Bryan Cutler edited comment on SPARK-16247 at 6/29/16 3:53 PM:

[jira] [Assigned] (SPARK-16299) Capture errors from R workers in daemon.R to avoid deletion of R session temporary directory

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16299: Assignee: Apache Spark > Capture errors from R workers in daemon.R to avoid deletion of R

[jira] [Assigned] (SPARK-16299) Capture errors from R workers in daemon.R to avoid deletion of R session temporary directory

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16299: Assignee: (was: Apache Spark) > Capture errors from R workers in daemon.R to avoid del

[jira] [Assigned] (SPARK-16299) Capture errors from R workers in daemon.R to avoid deletion of R session temporary directory

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16299: Assignee: Apache Spark > Capture errors from R workers in daemon.R to avoid deletion of R

[jira] [Assigned] (SPARK-16299) Capture errors from R workers in daemon.R to avoid deletion of R session temporary directory

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16299: Assignee: (was: Apache Spark) > Capture errors from R workers in daemon.R to avoid del

[jira] [Resolved] (SPARK-16300) Capture errors from R workers in daemon.R to avoid deletion of R session temporary directory

2016-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16300. --- Resolution: Duplicate > Capture errors from R workers in daemon.R to avoid deletion of R session > t

[jira] [Assigned] (SPARK-16288) Implement inline table generating function

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16288: Assignee: (was: Apache Spark) > Implement inline table generating function > -

[jira] [Assigned] (SPARK-16288) Implement inline table generating function

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16288: Assignee: Apache Spark > Implement inline table generating function >

[jira] [Commented] (SPARK-16285) Implement sentences SQL function

2016-06-29 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355563#comment-15355563 ] Dongjoon Hyun commented on SPARK-16285: --- I'll work on this. > Implement sentences

[jira] [Commented] (SPARK-16288) Implement inline table generating function

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355560#comment-15355560 ] Apache Spark commented on SPARK-16288: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Commented] (SPARK-16301) Analyzer rule for resolving using joins should respect case sensitivity setting

2016-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355596#comment-15355596 ] Apache Spark commented on SPARK-16301: -- User 'yhuai' has created a pull request for

[jira] [Resolved] (SPARK-16140) Group k-means method in generated doc

2016-06-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-16140. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Group k-means method

[jira] [Updated] (SPARK-16144) Add a separate Rd for ML generic methods: read.ml, write.ml, summary, predict

2016-06-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16144: -- Assignee: Yanbo Liang (was: Xin Ren) > Add a separate Rd for ML generic methods: read.ml, writ

[jira] [Commented] (SPARK-16144) Add a separate Rd for ML generic methods: read.ml, write.ml, summary, predict

2016-06-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355599#comment-15355599 ] Xiangrui Meng commented on SPARK-16144: --- [~yanboliang] Do you have time to make a f

  1   2   >