[jira] [Updated] (SPARK-14098) Generate code that get a float/double value in each column from CachedBatch when DataFrame.cache() is called

2016-05-03 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-14098: - Description: When DataFrame.cache() is called, data is stored as column-oriented storage

[jira] [Created] (SPARK-15114) Column name generated by typed aggregate is super verbose

2016-05-03 Thread Yin Huai (JIRA)
Yin Huai created SPARK-15114: Summary: Column name generated by typed aggregate is super verbose Key: SPARK-15114 URL: https://issues.apache.org/jira/browse/SPARK-15114 Project: Spark Issue Type:

[jira] [Updated] (SPARK-14098) Generate code that get a float/double value in each column from CachedBatch when DataFrame.cache() is called

2016-05-03 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-14098: - Summary: Generate code that get a float/double value in each column from CachedBatch when

[jira] [Commented] (SPARK-15089) kafka-spark consumer with SSL problem

2016-05-03 Thread Mario Briggs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270203#comment-15270203 ] Mario Briggs commented on SPARK-15089: -- Kafka supports SSL only with the 0.9x Kakfa

[jira] [Commented] (SPARK-15112) Dataset filter returns garbage

2016-05-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270199#comment-15270199 ] Reynold Xin commented on SPARK-15112: - It might be JSON specific. I couldn't repro th

[jira] [Commented] (SPARK-15112) Dataset filter returns garbage

2016-05-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270198#comment-15270198 ] Yin Huai commented on SPARK-15112: -- Seems I can also reproduce the problem with parquet.

[jira] [Assigned] (SPARK-15113) Add missing numFeatures & numClasses to wrapped JavaClassificationModel

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15113: Assignee: Apache Spark > Add missing numFeatures & numClasses to wrapped JavaClassificatio

[jira] [Commented] (SPARK-15113) Add missing numFeatures & numClasses to wrapped JavaClassificationModel

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270196#comment-15270196 ] Apache Spark commented on SPARK-15113: -- User 'holdenk' has created a pull request fo

[jira] [Assigned] (SPARK-15113) Add missing numFeatures & numClasses to wrapped JavaClassificationModel

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15113: Assignee: (was: Apache Spark) > Add missing numFeatures & numClasses to wrapped JavaCl

[jira] [Updated] (SPARK-15113) Add missing numFeatures & numClasses to wrapped JavaClassificationModel

2016-05-03 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-15113: Description: As part of SPARK-14813 numFeatures and numClasses are missing in many models in PySpark ML pip

[jira] [Created] (SPARK-15113) Add missing numFeatures & numClasses to wrapped JavaClassificationModel

2016-05-03 Thread holdenk (JIRA)
holdenk created SPARK-15113: --- Summary: Add missing numFeatures & numClasses to wrapped JavaClassificationModel Key: SPARK-15113 URL: https://issues.apache.org/jira/browse/SPARK-15113 Project: Spark

[jira] [Updated] (SPARK-15112) Dataset filter returns garbage

2016-05-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15112: Attachment: demo 1 dataset - Databricks.htm > Dataset filter returns garbage >

[jira] [Created] (SPARK-15112) Dataset filter returns garbage

2016-05-03 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-15112: --- Summary: Dataset filter returns garbage Key: SPARK-15112 URL: https://issues.apache.org/jira/browse/SPARK-15112 Project: Spark Issue Type: Bug Compon

[jira] [Closed] (SPARK-15111) Programming guide Documentation

2016-05-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-15111. --- Resolution: Not A Problem Those are not meant to be consumed directly. They are built via jekyll to b

[jira] [Resolved] (SPARK-14237) De-duplicate partition value appending logic in various buildReader() implementations

2016-05-03 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14237. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12866 [https://github.

[jira] [Assigned] (SPARK-14772) Python ML Params.copy treats uid, paramMaps differently than Scala

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14772: Assignee: (was: Apache Spark) > Python ML Params.copy treats uid, paramMaps differentl

[jira] [Assigned] (SPARK-14772) Python ML Params.copy treats uid, paramMaps differently than Scala

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14772: Assignee: Apache Spark > Python ML Params.copy treats uid, paramMaps differently than Scal

[jira] [Commented] (SPARK-14772) Python ML Params.copy treats uid, paramMaps differently than Scala

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270175#comment-15270175 ] Apache Spark commented on SPARK-14772: -- User 'hujy' has created a pull request for t

[jira] [Created] (SPARK-15111) Programming guide Documentation

2016-05-03 Thread Niranjan Molkeri` (JIRA)
Niranjan Molkeri` created SPARK-15111: - Summary: Programming guide Documentation Key: SPARK-15111 URL: https://issues.apache.org/jira/browse/SPARK-15111 Project: Spark Issue Type: Documen

[jira] [Comment Edited] (SPARK-14772) Python ML Params.copy treats uid, paramMaps differently than Scala

2016-05-03 Thread hujiayin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270161#comment-15270161 ] hujiayin edited comment on SPARK-14772 at 5/4/16 6:04 AM: -- @hold

[jira] [Commented] (SPARK-14772) Python ML Params.copy treats uid, paramMaps differently than Scala

2016-05-03 Thread hujiayin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270161#comment-15270161 ] hujiayin commented on SPARK-14772: -- @holdenk, I have a code for this issue and was busy

[jira] [Commented] (SPARK-15072) Remove SparkSession.withHiveSupport

2016-05-03 Thread Sagar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270155#comment-15270155 ] Sagar commented on SPARK-15072: --- [~techaddict] Yes it fails as assembly/assembly removed,

[jira] [Resolved] (SPARK-15107) Allow running test cases with different iterations in micro-benchmark util

2016-05-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15107. - Resolution: Fixed Fix Version/s: 2.0.0 > Allow running test cases with different iteration

[jira] [Commented] (SPARK-13946) PySpark DataFrames allows you to silently use aggregate expressions derived from different table expressions

2016-05-03 Thread Niranjan Molkeri` (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270151#comment-15270151 ] Niranjan Molkeri` commented on SPARK-13946: --- Hi, I ran the following code. {n

[jira] [Commented] (SPARK-14817) ML, Graph, R 2.0 QA: Programming guide update and migration guide

2016-05-03 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270123#comment-15270123 ] Felix Cheung commented on SPARK-14817: -- perhaps this SPARK-12071 should be included?

[jira] [Commented] (SPARK-14385) Use FunctionIdentifier in FunctionRegistry/SessionCatalog

2016-05-03 Thread Niranjan Molkeri` (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270121#comment-15270121 ] Niranjan Molkeri` commented on SPARK-14385: --- Hi, I would like to take a look at

[jira] [Commented] (SPARK-15072) Remove SparkSession.withHiveSupport

2016-05-03 Thread Sandeep Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270122#comment-15270122 ] Sandeep Singh commented on SPARK-15072: --- [~snanda] the first build/sbt will fail co

[jira] [Commented] (SPARK-14539) Fetching delegation tokens in Hive-Thriftserver fails when hive.server2.enable.doAs = True

2016-05-03 Thread Niranjan Molkeri` (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270113#comment-15270113 ] Niranjan Molkeri` commented on SPARK-14539: --- Hi, Can i know which hive version

[jira] [Commented] (SPARK-10931) PySpark ML Models should contain Param values

2016-05-03 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270105#comment-15270105 ] holdenk commented on SPARK-10931: - So, just to be certain, for https://issues.apache.org/

[jira] [Commented] (SPARK-14813) ML 2.0 QA: API: Python API coverage

2016-05-03 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270107#comment-15270107 ] holdenk commented on SPARK-14813: - While starting to do this audit, a number of params ar

[jira] [Commented] (SPARK-15110) SparkR - Implement repartitionByColumn on DataFrame

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270104#comment-15270104 ] Apache Spark commented on SPARK-15110: -- User 'NarineK' has created a pull request fo

[jira] [Assigned] (SPARK-15110) SparkR - Implement repartitionByColumn on DataFrame

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15110: Assignee: Apache Spark > SparkR - Implement repartitionByColumn on DataFrame > ---

[jira] [Assigned] (SPARK-15110) SparkR - Implement repartitionByColumn on DataFrame

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15110: Assignee: (was: Apache Spark) > SparkR - Implement repartitionByColumn on DataFrame >

[jira] [Updated] (SPARK-15110) SparkR - Implement repartitionByColumn on DataFrame

2016-05-03 Thread Narine Kokhlikyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Narine Kokhlikyan updated SPARK-15110: -- Description: Implement repartitionByColumn on DataFrame. This will allow us to run R f

[jira] [Created] (SPARK-15110) SparkR - Implement repartitionByColumn on DataFrame

2016-05-03 Thread Narine Kokhlikyan (JIRA)
Narine Kokhlikyan created SPARK-15110: - Summary: SparkR - Implement repartitionByColumn on DataFrame Key: SPARK-15110 URL: https://issues.apache.org/jira/browse/SPARK-15110 Project: Spark

[jira] [Commented] (SPARK-11148) Unable to create views

2016-05-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270069#comment-15270069 ] Yin Huai commented on SPARK-11148: -- Hi [~lunendl], we have cut the 2.0 branch and we are

[jira] [Assigned] (SPARK-15109) Accept Dataset[_] in joins

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15109: Assignee: Apache Spark (was: Reynold Xin) > Accept Dataset[_] in joins >

[jira] [Commented] (SPARK-15109) Accept Dataset[_] in joins

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270068#comment-15270068 ] Apache Spark commented on SPARK-15109: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-15109) Accept Dataset[_] in joins

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15109: Assignee: Reynold Xin (was: Apache Spark) > Accept Dataset[_] in joins >

[jira] [Commented] (SPARK-13269) Expose more executor stats in stable status API

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270067#comment-15270067 ] Andrew Or commented on SPARK-13269: --- Oops actually this was already done in SPARK-14069

[jira] [Resolved] (SPARK-13269) Expose more executor stats in stable status API

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13269. --- Resolution: Fixed Assignee: Wenchen Fan Fix Version/s: 2.0.0 > Expose more executor s

[jira] [Created] (SPARK-15109) Accept Dataset[_] in joins

2016-05-03 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-15109: --- Summary: Accept Dataset[_] in joins Key: SPARK-15109 URL: https://issues.apache.org/jira/browse/SPARK-15109 Project: Spark Issue Type: Sub-task Compo

[jira] [Commented] (SPARK-15108) Function is Not Found when Describe Permanent UDTF

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270039#comment-15270039 ] Apache Spark commented on SPARK-15108: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-15108) Function is Not Found when Describe Permanent UDTF

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15108: Assignee: Apache Spark > Function is Not Found when Describe Permanent UDTF >

[jira] [Assigned] (SPARK-15108) Function is Not Found when Describe Permanent UDTF

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15108: Assignee: (was: Apache Spark) > Function is Not Found when Describe Permanent UDTF > -

[jira] [Updated] (SPARK-15108) Function is Not Found when Describe Permanent UDTF

2016-05-03 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-15108: Summary: Function is Not Found when Describe Permanent UDTF (was: Function is Not Found when Describe Perm

[jira] [Updated] (SPARK-15108) Function is Not Found when Describe Permanent UDTF

2016-05-03 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-15108: Description: When Describe UDTF, it returns a wrong result. The command is unable to find the function, whi

[jira] [Created] (SPARK-15108) Function is Not Found when Describe Permanent UDF

2016-05-03 Thread Xiao Li (JIRA)
Xiao Li created SPARK-15108: --- Summary: Function is Not Found when Describe Permanent UDF Key: SPARK-15108 URL: https://issues.apache.org/jira/browse/SPARK-15108 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-15089) kafka-spark consumer with SSL problem

2016-05-03 Thread JasonChang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270030#comment-15270030 ] JasonChang commented on SPARK-15089: Hi Sean yes, broker works with SSL I run on kaf

[jira] [Commented] (SPARK-15072) Remove SparkSession.withHiveSupport

2016-05-03 Thread Sagar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270032#comment-15270032 ] Sagar commented on SPARK-15072: --- This helps to build test.jar $ ./build/sbt -Pyarn -Phado

[jira] [Commented] (SPARK-15032) When we create a new JDBC session, we may need to create a new session of executionHive

2016-05-03 Thread Sagar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270018#comment-15270018 ] Sagar commented on SPARK-15032: --- You are right! It is safer to create new session of execut

[jira] [Commented] (SPARK-15063) filtering and joining back doesn't work

2016-05-03 Thread Sagar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270012#comment-15270012 ] Sagar commented on SPARK-15063: --- What else is required to do it in new df for each filter c

[jira] [Commented] (SPARK-15086) Update Java API once the Scala one is finalized

2016-05-03 Thread Sagar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270007#comment-15270007 ] Sagar commented on SPARK-15086: --- In order to update Java API once Scala terminates. Please

[jira] [Commented] (SPARK-15107) Allow running test cases with different iterations in micro-benchmark util

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270003#comment-15270003 ] Apache Spark commented on SPARK-15107: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-15107) Allow running test cases with different iterations in micro-benchmark util

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15107: Assignee: Reynold Xin (was: Apache Spark) > Allow running test cases with different itera

[jira] [Assigned] (SPARK-15107) Allow running test cases with different iterations in micro-benchmark util

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15107: Assignee: Apache Spark (was: Reynold Xin) > Allow running test cases with different itera

[jira] [Created] (SPARK-15107) Allow running test cases with different iterations in micro-benchmark util

2016-05-03 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-15107: --- Summary: Allow running test cases with different iterations in micro-benchmark util Key: SPARK-15107 URL: https://issues.apache.org/jira/browse/SPARK-15107 Project: Spa

[jira] [Updated] (SPARK-14645) non local Python resource doesn't work with Mesos cluster mode

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14645: -- Assignee: Timothy Chen > non local Python resource doesn't work with Mesos cluster mode > -

[jira] [Resolved] (SPARK-14414) Make error messages consistent across DDLs

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14414. --- Resolution: Fixed Fix Version/s: 2.0.0 > Make error messages consistent across DDLs >

[jira] [Resolved] (SPARK-15097) Import fails for someDataset.sqlContext.implicits._

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15097. --- Resolution: Fixed Assignee: Koert Kuipers Fix Version/s: 2.0.0 Target Ver

[jira] [Resolved] (SPARK-15084) Use builder pattern to create SparkSession in PySpark

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15084. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Use builder pattern

[jira] [Resolved] (SPARK-14645) non local Python resource doesn't work with Mesos cluster mode

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14645. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > non local Python re

[jira] [Updated] (SPARK-14422) Improve handling of optional configs in SQLConf

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14422: -- Assignee: Sandeep Singh > Improve handling of optional configs in SQLConf > ---

[jira] [Resolved] (SPARK-14422) Improve handling of optional configs in SQLConf

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14422. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Improve handling of

[jira] [Assigned] (SPARK-15106) Add package documentation for ML and remove BETA from Scala & Java for ML pipeline API.

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15106: Assignee: (was: Apache Spark) > Add package documentation for ML and remove BETA from

[jira] [Commented] (SPARK-15106) Add package documentation for ML and remove BETA from Scala & Java for ML pipeline API.

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269901#comment-15269901 ] Apache Spark commented on SPARK-15106: -- User 'holdenk' has created a pull request fo

[jira] [Assigned] (SPARK-15106) Add package documentation for ML and remove BETA from Scala & Java for ML pipeline API.

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15106: Assignee: Apache Spark > Add package documentation for ML and remove BETA from Scala & Jav

[jira] [Created] (SPARK-15106) Add package documentation for ML and remove BETA from Scala & Java for ML pipeline API.

2016-05-03 Thread holdenk (JIRA)
holdenk created SPARK-15106: --- Summary: Add package documentation for ML and remove BETA from Scala & Java for ML pipeline API. Key: SPARK-15106 URL: https://issues.apache.org/jira/browse/SPARK-15106 Project

[jira] [Commented] (SPARK-14813) ML 2.0 QA: API: Python API coverage

2016-05-03 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269827#comment-15269827 ] holdenk commented on SPARK-14813: - I'm happy to start doing a first pass on this later on

[jira] [Commented] (SPARK-14772) Python ML Params.copy treats uid, paramMaps differently than Scala

2016-05-03 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269824#comment-15269824 ] holdenk commented on SPARK-14772: - I can take a look at this if no one else is working on

[jira] [Commented] (SPARK-15096) LogisticRegression MultiClassSummarizer numClasses can fail if no valid labels are found

2016-05-03 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269776#comment-15269776 ] Miao Wang commented on SPARK-15096: --- If nobody is working on this one, I will work on t

[jira] [Commented] (SPARK-14817) ML, Graph, R 2.0 QA: Programming guide update and migration guide

2016-05-03 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269775#comment-15269775 ] Xin Ren commented on SPARK-14817: - ok, I'll start looking for new APIs. So just create

[jira] [Commented] (SPARK-15101) Audit: ml.clustering and ml.recommendation

2016-05-03 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269768#comment-15269768 ] Miao Wang commented on SPARK-15101: --- [~josephkb] I want to know how to work on these ki

[jira] [Assigned] (SPARK-14900) spark.ml classification metrics should include accuracy

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14900: Assignee: (was: Apache Spark) > spark.ml classification metrics should include accurac

[jira] [Assigned] (SPARK-14900) spark.ml classification metrics should include accuracy

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14900: Assignee: Apache Spark > spark.ml classification metrics should include accuracy > ---

[jira] [Commented] (SPARK-14900) spark.ml classification metrics should include accuracy

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269754#comment-15269754 ] Apache Spark commented on SPARK-14900: -- User 'wangmiao1981' has created a pull reque

[jira] [Commented] (SPARK-13269) Expose more executor stats in stable status API

2016-05-03 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269738#comment-15269738 ] Alex Bozarth commented on SPARK-13269: -- Hey [~andrewor14], I was interested in this

[jira] [Commented] (SPARK-15095) Drop binary mode in ThriftServer

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269719#comment-15269719 ] Apache Spark commented on SPARK-15095: -- User 'davies' has created a pull request for

[jira] [Created] (SPARK-15105) Remove HiveSessionHook from ThriftServer

2016-05-03 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15105: -- Summary: Remove HiveSessionHook from ThriftServer Key: SPARK-15105 URL: https://issues.apache.org/jira/browse/SPARK-15105 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-15102) remove delegation token from ThriftServer

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269700#comment-15269700 ] Apache Spark commented on SPARK-15102: -- User 'davies' has created a pull request for

[jira] [Resolved] (SPARK-15104) Bad spacing in log line

2016-05-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15104. - Resolution: Fixed Assignee: Andrew Ash Fix Version/s: 2.0.0 > Bad spacing in log

[jira] [Resolved] (SPARK-15102) remove delegation token from ThriftServer

2016-05-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15102. - Resolution: Fixed Fix Version/s: 2.0.0 > remove delegation token from ThriftServer > -

[jira] [Assigned] (SPARK-15104) Bad spacing in log line

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15104: Assignee: Apache Spark > Bad spacing in log line > --- > >

[jira] [Commented] (SPARK-15104) Bad spacing in log line

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269648#comment-15269648 ] Apache Spark commented on SPARK-15104: -- User 'ash211' has created a pull request for

[jira] [Assigned] (SPARK-15104) Bad spacing in log line

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15104: Assignee: (was: Apache Spark) > Bad spacing in log line > --- > >

[jira] [Created] (SPARK-15104) Bad spacing in log line

2016-05-03 Thread Andrew Ash (JIRA)
Andrew Ash created SPARK-15104: -- Summary: Bad spacing in log line Key: SPARK-15104 URL: https://issues.apache.org/jira/browse/SPARK-15104 Project: Spark Issue Type: Bug Affects Versions: 1.6

[jira] [Closed] (SPARK-9466) Flaky test: org.apache.spark.sql.hive.thriftserver.CliSuite

2016-05-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-9466. -- Resolution: Auto Closed > Flaky test: org.apache.spark.sql.hive.thriftserver.CliSuite >

[jira] [Closed] (SPARK-12008) Spark hive security authorization doesn't work as Apache hive's

2016-05-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-12008. --- Resolution: Invalid Marking this as invalid since these are unsupported for now. We might add suppor

[jira] [Commented] (SPARK-15103) Add support for batch jobs correctly inferring partitions from data written with file stream sink

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269632#comment-15269632 ] Apache Spark commented on SPARK-15103: -- User 'tdas' has created a pull request for t

[jira] [Assigned] (SPARK-15103) Add support for batch jobs correctly inferring partitions from data written with file stream sink

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15103: Assignee: Tathagata Das (was: Apache Spark) > Add support for batch jobs correctly inferr

[jira] [Assigned] (SPARK-15103) Add support for batch jobs correctly inferring partitions from data written with file stream sink

2016-05-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15103: Assignee: Apache Spark (was: Tathagata Das) > Add support for batch jobs correctly inferr

[jira] [Closed] (SPARK-12066) spark sql throw java.lang.ArrayIndexOutOfBoundsException when use table.* with join

2016-05-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-12066. --- Resolution: Cannot Reproduce Closing as cannot reproduce for now. > spark sql throw java.lang.Array

[jira] [Commented] (SPARK-15037) Use SparkSession instead of SQLContext in testsuites

2016-05-03 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269620#comment-15269620 ] Dongjoon Hyun commented on SPARK-15037: --- Sure. Go ahead if you want. This is still

[jira] [Resolved] (SPARK-15056) Parse Unsupported Sampling Syntax and Issue Better Exceptions

2016-05-03 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-15056. --- Resolution: Fixed Assignee: Xiao Li > Parse Unsupported Sampling Syntax and Iss

[jira] [Created] (SPARK-15103) Add support for batch jobs correctly inferring partitions from data written with file stream sink

2016-05-03 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-15103: - Summary: Add support for batch jobs correctly inferring partitions from data written with file stream sink Key: SPARK-15103 URL: https://issues.apache.org/jira/browse/SPARK-1510

[jira] [Resolved] (SPARK-13971) Implicit group by with distinct modifier on having raises an unexpected error

2016-05-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13971. - Resolution: Fixed Fix Version/s: 2.0.0 > Implicit group by with distinct modifier on havin

[jira] [Resolved] (SPARK-14973) The CrossValidator and TrainValidationSplit miss the seed when saving and loading

2016-05-03 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-14973. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12825 [h

[jira] [Updated] (SPARK-15102) remove delegation token from ThriftServer

2016-05-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15102: Issue Type: Sub-task (was: Bug) Parent: SPARK-14987 > remove delegation token from ThriftS

[jira] [Updated] (SPARK-15095) Drop binary mode in ThriftServer

2016-05-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15095: Issue Type: Sub-task (was: Bug) Parent: SPARK-14987 > Drop binary mode in ThriftServer > -

[jira] [Updated] (SPARK-14973) The CrossValidator and TrainValidationSplit miss the seed when saving and loading

2016-05-03 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14973: -- Shepherd: Joseph K. Bradley > The CrossValidator and TrainValidationSplit miss the seed

  1   2   3   >