[jira] [Commented] (SPARK-16235) "evaluateEachIteration" is returning wrong results when calculated for classification model.

2016-06-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352538#comment-15352538 ] Sean Owen commented on SPARK-16235: --- Although you can apply MSE to any set of numbers,

[jira] [Created] (SPARK-16249) Change visibility of Object ml.clustering.LDA to public for loading

2016-06-28 Thread yuhao yang (JIRA)
yuhao yang created SPARK-16249: -- Summary: Change visibility of Object ml.clustering.LDA to public for loading Key: SPARK-16249 URL: https://issues.apache.org/jira/browse/SPARK-16249 Project: Spark

[jira] [Assigned] (SPARK-16249) Change visibility of Object ml.clustering.LDA to public for loading

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16249: Assignee: Apache Spark > Change visibility of Object ml.clustering.LDA to public for loadi

[jira] [Commented] (SPARK-16249) Change visibility of Object ml.clustering.LDA to public for loading

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352566#comment-15352566 ] Apache Spark commented on SPARK-16249: -- User 'hhbyyh' has created a pull request for

[jira] [Assigned] (SPARK-16249) Change visibility of Object ml.clustering.LDA to public for loading

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16249: Assignee: (was: Apache Spark) > Change visibility of Object ml.clustering.LDA to publi

[jira] [Commented] (SPARK-16240) model loading backward compatibility for ml.clustering.LDA

2016-06-28 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352575#comment-15352575 ] yuhao yang commented on SPARK-16240: This still does not resolve the issue when user

[jira] [Commented] (SPARK-16224) Hive context created by HiveContext can't access Hive databases when used in a script launched be spark-submit

2016-06-28 Thread Piotr Milanowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352602#comment-15352602 ] Piotr Milanowski commented on SPARK-16224: -- Hi, Tested your changes. It seems to

[jira] [Comment Edited] (SPARK-16055) sparkR.init() can not load sparkPackages when executing an R file

2016-06-28 Thread Krishna Kalyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352051#comment-15352051 ] Krishna Kalyan edited comment on SPARK-16055 at 6/28/16 8:55 AM: --

[jira] [Comment Edited] (SPARK-16055) sparkR.init() can not load sparkPackages when executing an R file

2016-06-28 Thread Krishna Kalyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352051#comment-15352051 ] Krishna Kalyan edited comment on SPARK-16055 at 6/28/16 9:06 AM: --

[jira] [Resolved] (SPARK-16219) Unable to run Python wordcount

2016-06-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16219. --- Resolution: Not A Problem > Unable to run Python wordcount > -- > >

[jira] [Commented] (SPARK-5114) Should Evaluator be a PipelineStage

2016-06-28 Thread Max Moroz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352651#comment-15352651 ] Max Moroz commented on SPARK-5114: -- +1 for Evaluator to be a PipelineStage. What options

[jira] [Updated] (SPARK-16128) Allow setting length of characters to be truncated to, in Dataset.show function.

2016-06-28 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-16128: Summary: Allow setting length of characters to be truncated to, in Dataset.show function.

[jira] [Updated] (SPARK-16232) Getting error by making columns using DataFrame

2016-06-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16232: -- Flags: (was: Important) Target Version/s: (was: 1.5.1) > Getting error by making col

[jira] [Commented] (SPARK-11106) Should ML Models contains single models or Pipelines?

2016-06-28 Thread Max Moroz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352690#comment-15352690 ] Max Moroz commented on SPARK-11106: --- Automatic feature transformation creates a lot of

[jira] [Created] (SPARK-16250) Can't use escapeQuotes option in DataFrameWriter.csv()

2016-06-28 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-16250: Summary: Can't use escapeQuotes option in DataFrameWriter.csv() Key: SPARK-16250 URL: https://issues.apache.org/jira/browse/SPARK-16250 Project: Spark Issue

[jira] [Assigned] (SPARK-16250) Can't use escapeQuotes option in DataFrameWriter.csv()

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16250: Assignee: Apache Spark > Can't use escapeQuotes option in DataFrameWriter.csv() >

[jira] [Assigned] (SPARK-16250) Can't use escapeQuotes option in DataFrameWriter.csv()

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16250: Assignee: (was: Apache Spark) > Can't use escapeQuotes option in DataFrameWriter.csv()

[jira] [Commented] (SPARK-16250) Can't use escapeQuotes option in DataFrameWriter.csv()

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352711#comment-15352711 ] Apache Spark commented on SPARK-16250: -- User 'HyukjinKwon' has created a pull reques

[jira] [Updated] (SPARK-16251) LocalCheckpointSuite's - missing checkpoint block fails with informative message is flaky.

2016-06-28 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-16251: Summary: LocalCheckpointSuite's - missing checkpoint block fails with informative message i

[jira] [Created] (SPARK-16251) LocalCheckpointSuite's missing checkpoint block fails with informative message is flaky.

2016-06-28 Thread Prashant Sharma (JIRA)
Prashant Sharma created SPARK-16251: --- Summary: LocalCheckpointSuite's missing checkpoint block fails with informative message is flaky. Key: SPARK-16251 URL: https://issues.apache.org/jira/browse/SPARK-16251

[jira] [Created] (SPARK-16252) Full Outer join with literal column results in incorrect result

2016-06-28 Thread Jurriaan Pruis (JIRA)
Jurriaan Pruis created SPARK-16252: -- Summary: Full Outer join with literal column results in incorrect result Key: SPARK-16252 URL: https://issues.apache.org/jira/browse/SPARK-16252 Project: Spark

[jira] [Updated] (SPARK-16252) Full Outer join with literal column results in incorrect result

2016-06-28 Thread Jurriaan Pruis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jurriaan Pruis updated SPARK-16252: --- Description: {code} >>> from pyspark.sql.functions import lit, coalesce >>> data1 = [[1,2], [

[jira] [Commented] (SPARK-16251) LocalCheckpointSuite's - missing checkpoint block fails with informative message is flaky.

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352813#comment-15352813 ] Apache Spark commented on SPARK-16251: -- User 'ScrapCodes' has created a pull request

[jira] [Assigned] (SPARK-16251) LocalCheckpointSuite's - missing checkpoint block fails with informative message is flaky.

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16251: Assignee: (was: Apache Spark) > LocalCheckpointSuite's - missing checkpoint block fail

[jira] [Assigned] (SPARK-16251) LocalCheckpointSuite's - missing checkpoint block fails with informative message is flaky.

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16251: Assignee: Apache Spark > LocalCheckpointSuite's - missing checkpoint block fails with info

[jira] [Updated] (SPARK-16232) Getting error by making columns using DataFrame

2016-06-28 Thread Inam Ur Rehman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inam Ur Rehman updated SPARK-16232: --- Description: I am using pyspark in ipython notebook for analysis. I am following an example

[jira] [Commented] (SPARK-16232) Getting error by making columns using DataFrame

2016-06-28 Thread Inam Ur Rehman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352824#comment-15352824 ] Inam Ur Rehman commented on SPARK-16232: [~srowen] i have updated the error. >

[jira] [Updated] (SPARK-16232) Getting error by making columns using DataFrame

2016-06-28 Thread Inam Ur Rehman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inam Ur Rehman updated SPARK-16232: --- Description: I am using pyspark in ipython notebook for analysis. I am following an example

[jira] [Assigned] (SPARK-16128) Allow setting length of characters to be truncated to, in Dataset.show function.

2016-06-28 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma reassigned SPARK-16128: --- Assignee: Prashant Sharma > Allow setting length of characters to be truncated to, i

[jira] [Resolved] (SPARK-16128) Allow setting length of characters to be truncated to, in Dataset.show function.

2016-06-28 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma resolved SPARK-16128. - Resolution: Fixed Fix Version/s: 2.1.0 Fixed by PR https://github.com/apache/spark

[jira] [Commented] (SPARK-16233) test_sparkSQL.R is failing

2016-06-28 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352885#comment-15352885 ] Prashant Sharma commented on SPARK-16233: - Did you build spark with -Phive ? > t

[jira] [Created] (SPARK-16253) make spark sql compatible with hive sql that using python script transform like using 'xxx.py'

2016-06-28 Thread zenglinxi (JIRA)
zenglinxi created SPARK-16253: - Summary: make spark sql compatible with hive sql that using python script transform like using 'xxx.py' Key: SPARK-16253 URL: https://issues.apache.org/jira/browse/SPARK-16253

[jira] [Commented] (SPARK-16253) make spark sql compatible with hive sql that using python script transform like using 'xxx.py'

2016-06-28 Thread zenglinxi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352917#comment-15352917 ] zenglinxi commented on SPARK-16253: --- I'm working on this issue... > make spark sql com

[jira] [Created] (SPARK-16254) Spark2.0 monitor web ui->Tasks (for all stages)->the number of Succeed is more than Total

2016-06-28 Thread marymwu (JIRA)
marymwu created SPARK-16254: --- Summary: Spark2.0 monitor web ui->Tasks (for all stages)->the number of Succeed is more than Total Key: SPARK-16254 URL: https://issues.apache.org/jira/browse/SPARK-16254 Proj

[jira] [Updated] (SPARK-16254) Spark2.0 monitor web ui->Tasks (for all stages)->the number of Succeed is more than Total

2016-06-28 Thread marymwu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] marymwu updated SPARK-16254: Attachment: Reference.png > Spark2.0 monitor web ui->Tasks (for all stages)->the number of Succeed is > mo

[jira] [Updated] (SPARK-16254) Spark2.0 monitor web ui->Tasks (for all stages)->the number of Succeed is more than Total

2016-06-28 Thread marymwu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] marymwu updated SPARK-16254: Affects Version/s: 2.0.0 > Spark2.0 monitor web ui->Tasks (for all stages)->the number of Succeed is > mor

[jira] [Created] (SPARK-16255) Spark2.0 doesn't support the following SQL statement:"insert into directory "/u_qa_user/hive_testdata/test1/t1" select * from d_test_tpc_2g_txt.auction" while Hive suppo

2016-06-28 Thread marymwu (JIRA)
marymwu created SPARK-16255: --- Summary: Spark2.0 doesn't support the following SQL statement:"insert into directory "/u_qa_user/hive_testdata/test1/t1" select * from d_test_tpc_2g_txt.auction" while Hive supports Key: SPARK-16255

[jira] [Updated] (SPARK-16255) Spark2.0 doesn't support the following SQL statement:"insert into directory "/u_qa_user/hive_testdata/test1/t1" select * from d_test_tpc_2g_txt.auction" while Hive suppo

2016-06-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16255: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) > Spark2.0 doesn't support the

[jira] [Commented] (SPARK-16254) Spark2.0 monitor web ui->Tasks (for all stages)->the number of Succeed is more than Total

2016-06-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352956#comment-15352956 ] Sean Owen commented on SPARK-16254: --- The screen shot is unreadable, but, what you descr

[jira] [Resolved] (SPARK-16242) Wrap the Matrix conversion utils in Python

2016-06-28 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-16242. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Wrap the Matrix conversi

[jira] [Created] (SPARK-16256) Add Structured Streaming Programming Guide

2016-06-28 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-16256: - Summary: Add Structured Streaming Programming Guide Key: SPARK-16256 URL: https://issues.apache.org/jira/browse/SPARK-16256 Project: Spark Issue Type: Sub-

[jira] [Commented] (SPARK-16256) Add Structured Streaming Programming Guide

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353075#comment-15353075 ] Apache Spark commented on SPARK-16256: -- User 'tdas' has created a pull request for t

[jira] [Assigned] (SPARK-16256) Add Structured Streaming Programming Guide

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16256: Assignee: Tathagata Das (was: Apache Spark) > Add Structured Streaming Programming Guide

[jira] [Assigned] (SPARK-16256) Add Structured Streaming Programming Guide

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16256: Assignee: Apache Spark (was: Tathagata Das) > Add Structured Streaming Programming Guide

[jira] [Resolved] (SPARK-16224) Hive context created by HiveContext can't access Hive databases when used in a script launched be spark-submit

2016-06-28 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16224. Resolution: Fixed Fix Version/s: 2.0.1 Issue resolved by pull request 13931 [https://github.

[jira] [Created] (SPARK-16257) spark-ec2 script not updated for 1.6.2 release

2016-06-28 Thread Brian Uri (JIRA)
Brian Uri created SPARK-16257: - Summary: spark-ec2 script not updated for 1.6.2 release Key: SPARK-16257 URL: https://issues.apache.org/jira/browse/SPARK-16257 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-12922) Implement gapply() on DataFrame in SparkR

2016-06-28 Thread Narine Kokhlikyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353142#comment-15353142 ] Narine Kokhlikyan commented on SPARK-12922: --- Thank you [~timhunter] for sharing

[jira] [Comment Edited] (SPARK-12922) Implement gapply() on DataFrame in SparkR

2016-06-28 Thread Narine Kokhlikyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353142#comment-15353142 ] Narine Kokhlikyan edited comment on SPARK-12922 at 6/28/16 3:03 PM: ---

[jira] [Commented] (SPARK-16257) spark-ec2 script not updated for 1.6.2 release

2016-06-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353166#comment-15353166 ] Sean Owen commented on SPARK-16257: --- Yeah you can fix this and even put in an entry pro

[jira] [Commented] (SPARK-8425) Add blacklist mechanism for task scheduling

2016-06-28 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353277#comment-15353277 ] Imran Rashid commented on SPARK-8425: - Adding the design doc from SPARK-8426 here: ht

[jira] [Commented] (SPARK-16233) test_sparkSQL.R is failing

2016-06-28 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353340#comment-15353340 ] Xin Ren commented on SPARK-16233: - this is what I used to build sparkR, should I add "-Ph

[jira] [Commented] (SPARK-16233) test_sparkSQL.R is failing

2016-06-28 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353344#comment-15353344 ] Shivaram Venkataraman commented on SPARK-16233: --- Ideally we should skip the

[jira] [Commented] (SPARK-16233) test_sparkSQL.R is failing

2016-06-28 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353352#comment-15353352 ] Xin Ren commented on SPARK-16233: - Actually I was just following the docs here https://g

[jira] [Created] (SPARK-16258) Automatically append the grouping keys in SparkR's gapply

2016-06-28 Thread Timothy Hunter (JIRA)
Timothy Hunter created SPARK-16258: -- Summary: Automatically append the grouping keys in SparkR's gapply Key: SPARK-16258 URL: https://issues.apache.org/jira/browse/SPARK-16258 Project: Spark

[jira] [Commented] (SPARK-12922) Implement gapply() on DataFrame in SparkR

2016-06-28 Thread Timothy Hunter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353374#comment-15353374 ] Timothy Hunter commented on SPARK-12922: I opened a separate JIRA for that issue:

[jira] [Commented] (SPARK-16257) spark-ec2 script not updated for 1.6.2 release

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353376#comment-15353376 ] Apache Spark commented on SPARK-16257: -- User 'briuri' has created a pull request for

[jira] [Assigned] (SPARK-16257) spark-ec2 script not updated for 1.6.2 release

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16257: Assignee: (was: Apache Spark) > spark-ec2 script not updated for 1.6.2 release > -

[jira] [Assigned] (SPARK-16257) spark-ec2 script not updated for 1.6.2 release

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16257: Assignee: Apache Spark > spark-ec2 script not updated for 1.6.2 release >

[jira] [Commented] (SPARK-16233) test_sparkSQL.R is failing

2016-06-28 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353388#comment-15353388 ] Dongjoon Hyun commented on SPARK-16233: --- Oh, thank you for pinging me. For this iss

[jira] [Created] (SPARK-16259) Cleanup options for DataFrame reader API in Python

2016-06-28 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16259: -- Summary: Cleanup options for DataFrame reader API in Python Key: SPARK-16259 URL: https://issues.apache.org/jira/browse/SPARK-16259 Project: Spark Issue Type: Im

[jira] [Comment Edited] (SPARK-16233) test_sparkSQL.R is failing

2016-06-28 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353405#comment-15353405 ] Dongjoon Hyun edited comment on SPARK-16233 at 6/28/16 5:20 PM: ---

[jira] [Comment Edited] (SPARK-16233) test_sparkSQL.R is failing

2016-06-28 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353405#comment-15353405 ] Dongjoon Hyun edited comment on SPARK-16233 at 6/28/16 5:20 PM: ---

[jira] [Commented] (SPARK-16233) test_sparkSQL.R is failing

2016-06-28 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353405#comment-15353405 ] Dongjoon Hyun commented on SPARK-16233: --- Hi, [~shivaram]. Actually, in `DataFrameRe

[jira] [Assigned] (SPARK-16259) Cleanup options for DataFrame reader API in Python

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16259: Assignee: Apache Spark (was: Davies Liu) > Cleanup options for DataFrame reader API in Py

[jira] [Commented] (SPARK-16259) Cleanup options for DataFrame reader API in Python

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353415#comment-15353415 ] Apache Spark commented on SPARK-16259: -- User 'davies' has created a pull request for

[jira] [Assigned] (SPARK-16259) Cleanup options for DataFrame reader API in Python

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16259: Assignee: Davies Liu (was: Apache Spark) > Cleanup options for DataFrame reader API in Py

[jira] [Resolved] (SPARK-16181) Incorrect behavior for isNull filter

2016-06-28 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-16181. -- Resolution: Fixed Assignee: Wenchen Fan Fix Version/s: 2.0.0 This issue has been resolv

[jira] [Commented] (SPARK-16252) Full Outer join with literal column results in incorrect result

2016-06-28 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353431#comment-15353431 ] Dongjoon Hyun commented on SPARK-16252: --- Hi, this is fixed now. {code} >>> df1.join

[jira] [Created] (SPARK-16260) PySpark ML Example Improvements and Cleanup

2016-06-28 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-16260: Summary: PySpark ML Example Improvements and Cleanup Key: SPARK-16260 URL: https://issues.apache.org/jira/browse/SPARK-16260 Project: Spark Issue Type: Umbre

[jira] [Commented] (SPARK-16252) Full Outer join with literal column results in incorrect result

2016-06-28 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353433#comment-15353433 ] Dongjoon Hyun commented on SPARK-16252: --- I think we can close this issue as `DUPLIC

[jira] [Commented] (SPARK-16181) Incorrect behavior for isNull filter

2016-06-28 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353434#comment-15353434 ] Dongjoon Hyun commented on SPARK-16181: --- Hi, [~yhuai]. Could you check and close SP

[jira] [Commented] (SPARK-16181) Incorrect behavior for isNull filter

2016-06-28 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353439#comment-15353439 ] Yin Huai commented on SPARK-16181: -- Yea. That is the same issue. I will resolve SPARK-16

[jira] [Resolved] (SPARK-16252) Full Outer join with literal column results in incorrect result

2016-06-28 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-16252. -- Resolution: Duplicate Assignee: Wenchen Fan Fix Version/s: 2.0.0 > Full Outer join with

[jira] [Created] (SPARK-16261) Fix Incorrect appNames in PySpark ML Examples

2016-06-28 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-16261: Summary: Fix Incorrect appNames in PySpark ML Examples Key: SPARK-16261 URL: https://issues.apache.org/jira/browse/SPARK-16261 Project: Spark Issue Type: Sub

[jira] [Assigned] (SPARK-16261) Fix Incorrect appNames in PySpark ML Examples

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16261: Assignee: Apache Spark > Fix Incorrect appNames in PySpark ML Examples > -

[jira] [Commented] (SPARK-16252) Full Outer join with literal column results in incorrect result

2016-06-28 Thread Jurriaan Pruis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353471#comment-15353471 ] Jurriaan Pruis commented on SPARK-16252: Awesome! It works, indeed! > Full Outer

[jira] [Assigned] (SPARK-16261) Fix Incorrect appNames in PySpark ML Examples

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16261: Assignee: (was: Apache Spark) > Fix Incorrect appNames in PySpark ML Examples > --

[jira] [Commented] (SPARK-16261) Fix Incorrect appNames in PySpark ML Examples

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353470#comment-15353470 ] Apache Spark commented on SPARK-16261: -- User 'BryanCutler' has created a pull reques

[jira] [Commented] (SPARK-12428) Write a script to run all PySpark MLlib examples for testing

2016-06-28 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353488#comment-15353488 ] Bryan Cutler commented on SPARK-12428: -- Hey Holden, I was thinking about doing this

[jira] [Updated] (SPARK-16237) PySpark gapply

2016-06-28 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Feinberg updated SPARK-16237: -- Description: To maintain feature parity, {{gapply}} functionality should be added to P

[jira] [Updated] (SPARK-16237) PySpark gapply

2016-06-28 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Feinberg updated SPARK-16237: -- Description: To maintain feature parity, `gapply` functionality should be added to PySp

[jira] [Commented] (SPARK-16237) PySpark gapply

2016-06-28 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353495#comment-15353495 ] Vladimir Feinberg commented on SPARK-16237: --- cc [~mengxr] [~thunterdb] [~joseph

[jira] [Created] (SPARK-16262) Impossible to remake new SparkContext using SparkSession API in Pyspark

2016-06-28 Thread Vladimir Feinberg (JIRA)
Vladimir Feinberg created SPARK-16262: - Summary: Impossible to remake new SparkContext using SparkSession API in Pyspark Key: SPARK-16262 URL: https://issues.apache.org/jira/browse/SPARK-16262 Pro

[jira] [Created] (SPARK-16263) SparkSession caches configuration in an unituitive global way

2016-06-28 Thread Vladimir Feinberg (JIRA)
Vladimir Feinberg created SPARK-16263: - Summary: SparkSession caches configuration in an unituitive global way Key: SPARK-16263 URL: https://issues.apache.org/jira/browse/SPARK-16263 Project: Spar

[jira] [Updated] (SPARK-16263) SparkSession caches configuration in an unituitive global way

2016-06-28 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Feinberg updated SPARK-16263: -- Description: The following use case demonstrates the issue. cls.spark = SparkS

[jira] [Updated] (SPARK-16263) SparkSession caches configuration in an unituitive global way

2016-06-28 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Feinberg updated SPARK-16263: -- Description: The following use case demonstrates the issue. > SparkSession caches conf

[jira] [Updated] (SPARK-16263) SparkSession caches configuration in an unituitive global way

2016-06-28 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Feinberg updated SPARK-16263: -- Description: The following use case demonstrates the issue. Note that as a workaround t

[jira] [Assigned] (SPARK-15487) Spark Master UI to reverse proxy Application and Workers UI

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15487: Assignee: Apache Spark > Spark Master UI to reverse proxy Application and Workers UI > ---

[jira] [Commented] (SPARK-15487) Spark Master UI to reverse proxy Application and Workers UI

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353541#comment-15353541 ] Apache Spark commented on SPARK-15487: -- User 'gurvindersingh' has created a pull req

[jira] [Assigned] (SPARK-15487) Spark Master UI to reverse proxy Application and Workers UI

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15487: Assignee: (was: Apache Spark) > Spark Master UI to reverse proxy Application and Worke

[jira] [Commented] (SPARK-15487) Spark Master UI to reverse proxy Application and Workers UI

2016-06-28 Thread Gurvinder (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353542#comment-15353542 ] Gurvinder commented on SPARK-15487: --- Sent pull request https://github.com/apache/spark/

[jira] [Updated] (SPARK-16262) Impossible to remake new SparkContext using SparkSession API in Pyspark

2016-06-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16262: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) I don't think this is generally

[jira] [Commented] (SPARK-16247) Using pyspark dataframe with pipeline and cross validator

2016-06-28 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353555#comment-15353555 ] Bryan Cutler commented on SPARK-16247: -- I'm not sure if this is the issue, but the f

[jira] [Commented] (SPARK-16020) Fix complete mode aggregation with console sink

2016-06-28 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353579#comment-15353579 ] holdenk commented on SPARK-16020: - Do we know why this bug happened? > Fix complete mode

[jira] [Commented] (SPARK-8425) Add blacklist mechanism for task scheduling

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353587#comment-15353587 ] Apache Spark commented on SPARK-8425: - User 'squito' has created a pull request for th

[jira] [Commented] (SPARK-16262) Impossible to remake new SparkContext using SparkSession API in Pyspark

2016-06-28 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353630#comment-15353630 ] Vladimir Feinberg commented on SPARK-16262: --- What do you mean by "clearing that

[jira] [Commented] (SPARK-16020) Fix complete mode aggregation with console sink

2016-06-28 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353638#comment-15353638 ] Shixiong Zhu commented on SPARK-16020: -- Created SPARK-16264 to show the cause. > Fi

[jira] [Created] (SPARK-16264) Allow the user to use operators on the received DataFrame

2016-06-28 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-16264: Summary: Allow the user to use operators on the received DataFrame Key: SPARK-16264 URL: https://issues.apache.org/jira/browse/SPARK-16264 Project: Spark Iss

[jira] [Commented] (SPARK-16262) Impossible to remake new SparkContext using SparkSession API in Pyspark

2016-06-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353639#comment-15353639 ] Sean Owen commented on SPARK-16262: --- No, I mean it can be reset or whatever when the se

[jira] [Commented] (SPARK-16262) Impossible to remake new SparkContext using SparkSession API in Pyspark

2016-06-28 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353642#comment-15353642 ] Vladimir Feinberg commented on SPARK-16262: --- Ah, are you suggesting that line s

  1   2   3   >