[jira] [Created] (SPARK-14888) UnresolvedFunction should use FunctionIdentifier rather than just a string for function name

2016-04-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-14888: --- Summary: UnresolvedFunction should use FunctionIdentifier rather than just a string for function name Key: SPARK-14888 URL: https://issues.apache.org/jira/browse/SPARK-14888

[jira] [Commented] (SPARK-14888) UnresolvedFunction should use FunctionIdentifier rather than just a string for function name

2016-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15255994#comment-15255994 ] Apache Spark commented on SPARK-14888: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-14888) UnresolvedFunction should use FunctionIdentifier rather than just a string for function name

2016-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14888: Assignee: Reynold Xin (was: Apache Spark) > UnresolvedFunction should use FunctionIdentif

[jira] [Assigned] (SPARK-14888) UnresolvedFunction should use FunctionIdentifier rather than just a string for function name

2016-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14888: Assignee: Apache Spark (was: Reynold Xin) > UnresolvedFunction should use FunctionIdentif

[jira] [Commented] (SPARK-14597) Streaming Listener timing metrics should include time spent in JobGenerator's graph.generateJobs

2016-04-25 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15255993#comment-15255993 ] Prashant Sharma commented on SPARK-14597: - It would be good to know the usefulnes

[jira] [Commented] (SPARK-14693) Spark Streaming Context Hangs on Start

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256007#comment-15256007 ] Sean Owen commented on SPARK-14693: --- Given the end of your stack trace, it seems like i

[jira] [Updated] (SPARK-14886) RankingMetrics.ndcgAt throw java.lang.ArrayIndexOutOfBoundsException

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14886: -- Priority: Minor (was: Major) Description: {code} @Since("1.2.0") def ndcgAt(k: Int): Double

[jira] [Resolved] (SPARK-14790) Scalastyle should run on compile in sbt

2016-04-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-14790. - Resolution: Fixed Assignee: Eric Liang Fix Version/s: 2.0.0 > Scalastyle should r

[jira] [Commented] (SPARK-14886) RankingMetrics.ndcgAt throw java.lang.ArrayIndexOutOfBoundsException

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256016#comment-15256016 ] Sean Owen commented on SPARK-14886: --- Yes I think {{if (labSet.contains(pred(i))) {}} ne

[jira] [Updated] (SPARK-14802) Disable Passing to Hive the queries that can't be parsed

2016-04-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14802: Parent Issue: SPARK-14825 (was: SPARK-14776) > Disable Passing to Hive the queries that can't be p

[jira] [Updated] (SPARK-14591) Remove org.apache.spark.sql.catalyst.parser.DataTypeParser

2016-04-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14591: Parent Issue: SPARK-14825 (was: SPARK-14776) > Remove org.apache.spark.sql.catalyst.parser.DataTyp

[jira] [Resolved] (SPARK-14776) Merge HiveSqlAstBuilder and SparkSqlAstBuilder

2016-04-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-14776. - Resolution: Fixed Fix Version/s: 2.0.0 > Merge HiveSqlAstBuilder and SparkSqlAstBuilder >

[jira] [Updated] (SPARK-14806) Alias original Hive options in Spark SQL conf

2016-04-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14806: Parent Issue: SPARK-14825 (was: SPARK-14776) > Alias original Hive options in Spark SQL conf > ---

[jira] [Updated] (SPARK-14886) RankingMetrics.ndcgAt throw java.lang.ArrayIndexOutOfBoundsException

2016-04-25 Thread lichenglin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lichenglin updated SPARK-14886: --- Description: {code} @Since("1.2.0") def ndcgAt(k: Int): Double = { require(k > 0, "ranking pos

[jira] [Created] (SPARK-14889) scala.MatchError: NONE (of class scala.Enumeration$Val) when spark.scheduler.mode=NONE

2016-04-25 Thread Jacek Laskowski (JIRA)
Jacek Laskowski created SPARK-14889: --- Summary: scala.MatchError: NONE (of class scala.Enumeration$Val) when spark.scheduler.mode=NONE Key: SPARK-14889 URL: https://issues.apache.org/jira/browse/SPARK-14889

[jira] [Created] (SPARK-14890) DAGScheduler should not accept the result of a previous task attempt, since its stage has been completed.

2016-04-25 Thread yinqiang (JIRA)
yinqiang created SPARK-14890: Summary: DAGScheduler should not accept the result of a previous task attempt, since its stage has been completed. Key: SPARK-14890 URL: https://issues.apache.org/jira/browse/SPARK-14890

[jira] [Updated] (SPARK-14890) DAGScheduler should not accept the result of a previous task attempt, since its stage attempt has been completed.

2016-04-25 Thread yinqiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yinqiang updated SPARK-14890: - Summary: DAGScheduler should not accept the result of a previous task attempt, since its stage attempt ha

[jira] [Commented] (SPARK-13693) Flaky test: o.a.s.streaming.MapWithStateSuite

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256054#comment-15256054 ] Sean Owen commented on SPARK-13693: --- Weird, because this file hasn't otherwise changed

[jira] [Commented] (SPARK-14889) scala.MatchError: NONE (of class scala.Enumeration$Val) when spark.scheduler.mode=NONE

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256059#comment-15256059 ] Sean Owen commented on SPARK-14889: --- Yes I think Pool and TaskSchedulerImpl have blocks

[jira] [Commented] (SPARK-14889) scala.MatchError: NONE (of class scala.Enumeration$Val) when spark.scheduler.mode=NONE

2016-04-25 Thread ding (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256061#comment-15256061 ] ding commented on SPARK-14889: -- @Jacek Laskowski, do you mind if I take the jira and give a

[jira] [Comment Edited] (SPARK-14886) RankingMetrics.ndcgAt throw java.lang.ArrayIndexOutOfBoundsException

2016-04-25 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256065#comment-15256065 ] Nick Pentreath edited comment on SPARK-14886 at 4/25/16 8:26 AM: --

[jira] [Commented] (SPARK-14886) RankingMetrics.ndcgAt throw java.lang.ArrayIndexOutOfBoundsException

2016-04-25 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256065#comment-15256065 ] Nick Pentreath commented on SPARK-14886: Are you saying that the "maxDCG" should

[jira] [Resolved] (SPARK-14758) Add checking for StepSize and Tol

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-14758. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12530 [https://github.co

[jira] [Commented] (SPARK-14537) [CORE] SparkContext init hangs if master removes application before backend is ready.

2016-04-25 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256071#comment-15256071 ] Jacek Laskowski commented on SPARK-14537: - Hi [~drcrallen], I can't seem to figur

[jira] [Updated] (SPARK-14758) Add checking for StepSize and Tol

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14758: -- Assignee: zhengruifeng Priority: Trivial (was: Major) > Add checking for StepSize and Tol > --

[jira] [Updated] (SPARK-14664) Implement DecimalAggregates optimization for Window queries

2016-04-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-14664: -- Description: This issue aims to implement decimal aggregation optimization for window queries

[jira] [Resolved] (SPARK-14636) Spark should fail fast if executor/driver memory is too small for the StaticMemoryManager

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-14636. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12395 [https://github.co

[jira] [Updated] (SPARK-14636) Spark should fail fast if executor/driver memory is too small for the StaticMemoryManager

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14636: -- Assignee: Peter Ableda > Spark should fail fast if executor/driver memory is too small for the > Stati

[jira] [Created] (SPARK-14891) ALS in ML never validates input schema

2016-04-25 Thread Nick Pentreath (JIRA)
Nick Pentreath created SPARK-14891: -- Summary: ALS in ML never validates input schema Key: SPARK-14891 URL: https://issues.apache.org/jira/browse/SPARK-14891 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-14891) ALS in ML never validates input schema

2016-04-25 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256084#comment-15256084 ] Nick Pentreath commented on SPARK-14891: [~srowen] [~mengxr] [~josephkb] thoughts

[jira] [Commented] (SPARK-14886) RankingMetrics.ndcgAt throw java.lang.ArrayIndexOutOfBoundsException

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256096#comment-15256096 ] Sean Owen commented on SPARK-14886: --- No I don't think this concerns maxDCG in particula

[jira] [Commented] (SPARK-14891) ALS in ML never validates input schema

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256107#comment-15256107 ] Sean Owen commented on SPARK-14891: --- I've lost track of what's supported but it looks l

[jira] [Commented] (SPARK-13983) HiveThriftServer2 can not get "--hiveconf" or ''--hivevar" variables since 1.6 version (both multi-session and single session)

2016-04-25 Thread Teng Qiu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256139#comment-15256139 ] Teng Qiu commented on SPARK-13983: -- Any update would be much appreciated... we are still

[jira] [Comment Edited] (SPARK-14891) ALS in ML never validates input schema

2016-04-25 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256140#comment-15256140 ] Nick Pentreath edited comment on SPARK-14891 at 4/25/16 9:27 AM: --

[jira] [Commented] (SPARK-14891) ALS in ML never validates input schema

2016-04-25 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256140#comment-15256140 ] Nick Pentreath commented on SPARK-14891: Currently the only doc is {code} /** *

[jira] [Commented] (SPARK-13768) Set hive conf failed use --hiveconf when beeline connect to thriftserver

2016-04-25 Thread Teng Qiu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256142#comment-15256142 ] Teng Qiu commented on SPARK-13768: -- perhaps it is a beeline issue, we used the beeline f

[jira] [Commented] (SPARK-14503) spark.ml API for FPGrowth

2016-04-25 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256154#comment-15256154 ] yuhao yang commented on SPARK-14503: design doc draft https://docs.google.com/documen

[jira] [Updated] (SPARK-14409) Investigate adding a RankingEvaluator to ML

2016-04-25 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-14409: --- Shepherd: Nick Pentreath > Investigate adding a RankingEvaluator to ML >

[jira] [Commented] (SPARK-14889) scala.MatchError: NONE (of class scala.Enumeration$Val) when spark.scheduler.mode=NONE

2016-04-25 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256189#comment-15256189 ] Jacek Laskowski commented on SPARK-14889: - Go for it! Thanks. > scala.MatchError

[jira] [Updated] (SPARK-14881) pyspark and sparkR shell default log level should match spark-shell/Scala

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14881: -- Assignee: Felix Cheung > pyspark and sparkR shell default log level should match spark-shell/Scala > --

[jira] [Updated] (SPARK-13432) Add the origin of the source code into a generated Java code

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13432: -- Target Version/s: (was: 2.0.0) > Add the origin of the source code into a generated Java code > -

[jira] [Updated] (SPARK-12567) Add aes_encrypt and aes_decrypt UDFs

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12567: -- Target Version/s: (was: 2.0.0) > Add aes_encrypt and aes_decrypt UDFs > -

[jira] [Commented] (SPARK-14077) Support weighted instances in naive Bayes

2016-04-25 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256228#comment-15256228 ] zhengruifeng commented on SPARK-14077: -- Are you still working on this task? > Suppo

[jira] [Commented] (SPARK-14077) Support weighted instances in naive Bayes

2016-04-25 Thread Mohamed Baddar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256247#comment-15256247 ] Mohamed Baddar commented on SPARK-14077: I suspended working on it for the time b

[jira] [Commented] (SPARK-14412) spark.ml ALS prefered storage level Params

2016-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256250#comment-15256250 ] Apache Spark commented on SPARK-14412: -- User 'MLnick' has created a pull request for

[jira] [Updated] (SPARK-14752) LazilyGenerateOrdering throws NullPointerException

2016-04-25 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated SPARK-14752: - Summary: LazilyGenerateOrdering throws NullPointerException (was: LazilyGenerateOrdering

[jira] [Assigned] (SPARK-14752) LazilyGenerateOrdering throws NullPointerException

2016-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14752: Assignee: Apache Spark > LazilyGenerateOrdering throws NullPointerException >

[jira] [Assigned] (SPARK-14752) LazilyGenerateOrdering throws NullPointerException

2016-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14752: Assignee: (was: Apache Spark) > LazilyGenerateOrdering throws NullPointerException > -

[jira] [Commented] (SPARK-14752) LazilyGenerateOrdering throws NullPointerException

2016-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256318#comment-15256318 ] Apache Spark commented on SPARK-14752: -- User 'rajeshbalamohan' has created a pull re

[jira] [Resolved] (SPARK-14768) Remove expectedType arg for PySpark Param

2016-04-25 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-14768. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12581 [https:/

[jira] [Updated] (SPARK-14768) Remove expectedType arg for PySpark Param

2016-04-25 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-14768: --- Assignee: Jason C Lee > Remove expectedType arg for PySpark Param > -

[jira] [Resolved] (SPARK-14875) OutputWriterFactory.newInstance shouldn't be private[sql]

2016-04-25 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14875. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12652 [https://github.

[jira] [Updated] (SPARK-14844) KMeansModel in spark.ml should allow to change featureCol and predictionCol

2016-04-25 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-14844: --- Shepherd: Nick Pentreath (was: Dominik Jastrzębski) > KMeansModel in spark.ml should allow t

[jira] [Commented] (SPARK-12405) Expose featureImportances on org.apache.spark.mllib.tree.RandomForest

2016-04-25 Thread Edi Bice (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256434#comment-15256434 ] Edi Bice commented on SPARK-12405: -- This should be reopened - it's really not a duplicat

[jira] [Commented] (SPARK-14693) Spark Streaming Context Hangs on Start

2016-04-25 Thread Evan Oman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256462#comment-15256462 ] Evan Oman commented on SPARK-14693: --- Hmm, so you are thinking this looks like a Databri

[jira] [Updated] (SPARK-14098) Generate code that get a value in each column from CachedBatch when DataFrame.cache() is called

2016-04-25 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-14098: - Target Version/s: 2.0.0 > Generate code that get a value in each column from CachedBatch

[jira] [Commented] (SPARK-14817) ML 2.0 QA: Programming guide update and migration guide

2016-04-25 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256547#comment-15256547 ] Miao Wang commented on SPARK-14817: --- +1 > ML 2.0 QA: Programming guide update and migr

[jira] [Created] (SPARK-14892) Disable the HiveCompatibilitySuite test case for INPUTDRIVER and OUTPUTDRIVER

2016-04-25 Thread Xiao Li (JIRA)
Xiao Li created SPARK-14892: --- Summary: Disable the HiveCompatibilitySuite test case for INPUTDRIVER and OUTPUTDRIVER Key: SPARK-14892 URL: https://issues.apache.org/jira/browse/SPARK-14892 Project: Spark

[jira] [Assigned] (SPARK-14892) Disable the HiveCompatibilitySuite test case for INPUTDRIVER and OUTPUTDRIVER

2016-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14892: Assignee: Apache Spark > Disable the HiveCompatibilitySuite test case for INPUTDRIVER and

[jira] [Assigned] (SPARK-14892) Disable the HiveCompatibilitySuite test case for INPUTDRIVER and OUTPUTDRIVER

2016-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14892: Assignee: (was: Apache Spark) > Disable the HiveCompatibilitySuite test case for INPUT

[jira] [Commented] (SPARK-14892) Disable the HiveCompatibilitySuite test case for INPUTDRIVER and OUTPUTDRIVER

2016-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256557#comment-15256557 ] Apache Spark commented on SPARK-14892: -- User 'gatorsmile' has created a pull request

[jira] [Commented] (SPARK-14802) Disable Passing to Hive the queries that can't be parsed

2016-04-25 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256564#comment-15256564 ] Xiao Li commented on SPARK-14802: - Doing it now. Hit a couple of issues. > Disable Passi

[jira] [Commented] (SPARK-4452) Shuffle data structures can starve others on the same thread for memory

2016-04-25 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256602#comment-15256602 ] Davies Liu commented on SPARK-4452: --- We only backport critical bug fix into released bra

[jira] [Updated] (SPARK-4452) Shuffle data structures can starve others on the same thread for memory

2016-04-25 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-4452: -- Assignee: Lianhui Wang (was: Tianshuo Deng) > Shuffle data structures can starve others on the same thr

[jira] [Resolved] (SPARK-14744) Put examples packaging on a diet

2016-04-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-14744. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.0.0 > Put examp

[jira] [Commented] (SPARK-13902) Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first.

2016-04-25 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256642#comment-15256642 ] Kay Ousterhout commented on SPARK-13902: A few questions about your example: -Wh

[jira] [Commented] (SPARK-13902) Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first.

2016-04-25 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256669#comment-15256669 ] Kay Ousterhout commented on SPARK-13902: After looking at this a bit more, I thin

[jira] [Comment Edited] (SPARK-13902) Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first.

2016-04-25 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256669#comment-15256669 ] Kay Ousterhout edited comment on SPARK-13902 at 4/25/16 5:41 PM: --

[jira] [Comment Edited] (SPARK-13902) Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first.

2016-04-25 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256669#comment-15256669 ] Kay Ousterhout edited comment on SPARK-13902 at 4/25/16 5:41 PM: --

[jira] [Comment Edited] (SPARK-13902) Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first.

2016-04-25 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256669#comment-15256669 ] Kay Ousterhout edited comment on SPARK-13902 at 4/25/16 5:41 PM: --

[jira] [Assigned] (SPARK-14862) Tree and ensemble classification: do not require label metadata

2016-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14862: Assignee: Apache Spark (was: Joseph K. Bradley) > Tree and ensemble classification: do no

[jira] [Assigned] (SPARK-14862) Tree and ensemble classification: do not require label metadata

2016-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14862: Assignee: Joseph K. Bradley (was: Apache Spark) > Tree and ensemble classification: do no

[jira] [Commented] (SPARK-14862) Tree and ensemble classification: do not require label metadata

2016-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256674#comment-15256674 ] Apache Spark commented on SPARK-14862: -- User 'jkbradley' has created a pull request

[jira] [Updated] (SPARK-14433) PySpark ml GaussianMixture

2016-04-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14433: -- Assignee: Miao Wang > PySpark ml GaussianMixture > -- > >

[jira] [Resolved] (SPARK-14433) PySpark ml GaussianMixture

2016-04-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-14433. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12402 [h

[jira] [Created] (SPARK-14893) Re-enable HiveSparkSubmitSuite SPARK-8489 test after HiveContext is removed

2016-04-25 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14893: - Summary: Re-enable HiveSparkSubmitSuite SPARK-8489 test after HiveContext is removed Key: SPARK-14893 URL: https://issues.apache.org/jira/browse/SPARK-14893 Project: Spark

[jira] [Created] (SPARK-14894) Python GaussianMixture summary

2016-04-25 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-14894: - Summary: Python GaussianMixture summary Key: SPARK-14894 URL: https://issues.apache.org/jira/browse/SPARK-14894 Project: Spark Issue Type: New Feat

[jira] [Created] (SPARK-14895) SparkSession Python API

2016-04-25 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14895: - Summary: SparkSession Python API Key: SPARK-14895 URL: https://issues.apache.org/jira/browse/SPARK-14895 Project: Spark Issue Type: New Feature Component

[jira] [Created] (SPARK-14896) Remove HiveContext in Python

2016-04-25 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14896: - Summary: Remove HiveContext in Python Key: SPARK-14896 URL: https://issues.apache.org/jira/browse/SPARK-14896 Project: Spark Issue Type: Sub-task Compone

[jira] [Created] (SPARK-14897) Upgrade Jetty to latest version of 8/9

2016-04-25 Thread Adam Kramer (JIRA)
Adam Kramer created SPARK-14897: --- Summary: Upgrade Jetty to latest version of 8/9 Key: SPARK-14897 URL: https://issues.apache.org/jira/browse/SPARK-14897 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-14897) Upgrade Jetty to latest version of 8/9

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256724#comment-15256724 ] Sean Owen commented on SPARK-14897: --- You're welcome to update to the latest 8.x release

[jira] [Commented] (SPARK-14693) Spark Streaming Context Hangs on Start

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256726#comment-15256726 ] Sean Owen commented on SPARK-14693: --- .. or something merely specific to the instance it

[jira] [Commented] (SPARK-12405) Expose featureImportances on org.apache.spark.mllib.tree.RandomForest

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256744#comment-15256744 ] Sean Owen commented on SPARK-12405: --- Yes the better resolution is probably "WontFix". S

[jira] [Resolved] (SPARK-13015) Replace example code in mllib-data-types.md using include_example

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13015. --- Resolution: Won't Fix > Replace example code in mllib-data-types.md using include_example > -

[jira] [Closed] (SPARK-11399) Include_example should support labels to cut out different parts in one example code

2016-04-25 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin closed SPARK-11399. - Resolution: Won't Fix > Include_example should support labels to cut out different parts in one > exampl

[jira] [Resolved] (SPARK-11337) Make example code in user guide testable

2016-04-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11337. --- Resolution: Fixed Fix Version/s: 2.0.0 > Make example code in user guide testable > --

[jira] [Commented] (SPARK-11337) Make example code in user guide testable

2016-04-25 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256751#comment-15256751 ] Xusen Yin commented on SPARK-11337: --- [~mengxr] We can close this now. > Make example c

[jira] [Updated] (SPARK-14862) Tree and ensemble classification: do not require label metadata

2016-04-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14862: -- Target Version/s: 2.0.0 > Tree and ensemble classification: do not require label metada

[jira] [Commented] (SPARK-14894) Python GaussianMixture summary

2016-04-25 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256779#comment-15256779 ] Miao Wang commented on SPARK-14894: --- I can work on this one while I am preparing exampl

[jira] [Created] (SPARK-14898) MultivariateGaussian could use Cholesky in calculateCovarianceConstants

2016-04-25 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-14898: - Summary: MultivariateGaussian could use Cholesky in calculateCovarianceConstants Key: SPARK-14898 URL: https://issues.apache.org/jira/browse/SPARK-14898 Pro

[jira] [Updated] (SPARK-14892) Disable the HiveCompatibilitySuite test case for INPUTDRIVER and OUTPUTDRIVER

2016-04-25 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-14892: - Assignee: Xiao Li > Disable the HiveCompatibilitySuite test case for INPUTDRIVER and OUTPUTDRIVER > -

[jira] [Resolved] (SPARK-14892) Disable the HiveCompatibilitySuite test case for INPUTDRIVER and OUTPUTDRIVER

2016-04-25 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-14892. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12662 [https://github.com/

[jira] [Commented] (SPARK-14889) scala.MatchError: NONE (of class scala.Enumeration$Val) when spark.scheduler.mode=NONE

2016-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256822#comment-15256822 ] Apache Spark commented on SPARK-14889: -- User 'sbcd90' has created a pull request for

[jira] [Assigned] (SPARK-14889) scala.MatchError: NONE (of class scala.Enumeration$Val) when spark.scheduler.mode=NONE

2016-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14889: Assignee: Apache Spark > scala.MatchError: NONE (of class scala.Enumeration$Val) when > s

[jira] [Assigned] (SPARK-14889) scala.MatchError: NONE (of class scala.Enumeration$Val) when spark.scheduler.mode=NONE

2016-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14889: Assignee: (was: Apache Spark) > scala.MatchError: NONE (of class scala.Enumeration$Val

[jira] [Resolved] (SPARK-10574) HashingTF should use MurmurHash3

2016-04-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-10574. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12498 [h

[jira] [Commented] (SPARK-14820) Reduce shuffle data by pushing filter toward storage

2016-04-25 Thread Ali Tootoonchian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256826#comment-15256826 ] Ali Tootoonchian commented on SPARK-14820: -- I've listed the optimization is done

[jira] [Created] (SPARK-14899) Remove spark.ml HashingTF hashingAlg option

2016-04-25 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-14899: - Summary: Remove spark.ml HashingTF hashingAlg option Key: SPARK-14899 URL: https://issues.apache.org/jira/browse/SPARK-14899 Project: Spark Issue T

[jira] [Commented] (SPARK-14521) StackOverflowError in Kryo when executing TPC-DS

2016-04-25 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256836#comment-15256836 ] JESSE CHEN commented on SPARK-14521: This fix will allow us to use Kyro again (in spa

[jira] [Assigned] (SPARK-14853) Support LeftSemi/LeftAnti in SortMergeJoin

2016-04-25 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-14853: -- Assignee: Davies Liu > Support LeftSemi/LeftAnti in SortMergeJoin > --

  1   2   3   >