[jira] [Commented] (SPARK-15611) Got the same sequence random number in every forked worker.

2016-05-29 Thread Thomas Lau (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306204#comment-15306204 ] Thomas Lau commented on SPARK-15611: hi, can anyone review this fix? > Got the same sequence random

[jira] [Resolved] (SPARK-15611) Got the same sequence random number in every forked worker.

2016-05-29 Thread Thomas Lau (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Lau resolved SPARK-15611. Resolution: Fixed Target Version/s: 1.6.1 > Got the same sequence random number in every

[jira] [Updated] (SPARK-15604) Spark-SQL: Get com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException when runing query_1.sql of TPC-DS

2016-05-29 Thread wenjing wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wenjing wang updated SPARK-15604: - Attachment: test4.txt Fourth test details > Spark-SQL: Get

[jira] [Commented] (SPARK-15645) Fix some typos of Streaming module

2016-05-29 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306092#comment-15306092 ] Xin Ren commented on SPARK-15645: - Thank you very much for this explanation Sean, I'll try to avoid this

[jira] [Updated] (SPARK-15603) Replace SQLContext with SparkSession in ML/MLLib

2016-05-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15603: -- Target Version/s: 2.0.0 Fix Version/s: (was: 2.0.0) > Replace SQLContext with SparkSession

[jira] [Commented] (SPARK-15645) Fix some typos of Streaming module

2016-05-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306077#comment-15306077 ] Sean Owen commented on SPARK-15645: --- The guideline is really: JIRA = what should be changed and PR =

[jira] [Commented] (SPARK-15617) Clarify that fMeasure in MulticlassMetrics and MulticlassClassificationEvaluator is "micro" f1_score

2016-05-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306073#comment-15306073 ] Sean Owen commented on SPARK-15617: --- While that's true, correct me if I'm wrong, but we deprecated F1 /

[jira] [Commented] (SPARK-15641) Incorrect Completed for Incomplete applications in HistoryServer

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306033#comment-15306033 ] Apache Spark commented on SPARK-15641: -- User 'catapan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15641) Incorrect Completed for Incomplete applications in HistoryServer

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15641: Assignee: Apache Spark > Incorrect Completed for Incomplete applications in HistoryServer

[jira] [Assigned] (SPARK-15641) Incorrect Completed for Incomplete applications in HistoryServer

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15641: Assignee: (was: Apache Spark) > Incorrect Completed for Incomplete applications in

[jira] [Commented] (SPARK-14507) Decide if we should still support CREATE EXTERNAL TABLE AS SELECT

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306022#comment-15306022 ] Apache Spark commented on SPARK-14507: -- User 'yhuai' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-15641) Incorrect Completed for Incomplete applications in HistoryServer

2016-05-29 Thread Zhuo Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306021#comment-15306021 ] Zhuo Liu edited comment on SPARK-15641 at 5/29/16 6:31 PM: --- [~sowen], thanks

[jira] [Commented] (SPARK-15641) Incorrect Completed for Incomplete applications in HistoryServer

2016-05-29 Thread Zhuo Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306021#comment-15306021 ] Zhuo Liu commented on SPARK-15641: -- [~sowen], thanks Sean for the reminder, that is true. @catapan will

[jira] [Commented] (SPARK-15489) Dataset kryo encoder won't load custom user settings

2016-05-29 Thread Amit Sela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305997#comment-15305997 ] Amit Sela commented on SPARK-15489: --- [~marmbrus] you can assign this one to me. > Dataset kryo encoder

[jira] [Comment Edited] (SPARK-15645) Fix some typos of Streaming module

2016-05-29 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305971#comment-15305971 ] Xin Ren edited comment on SPARK-15645 at 5/29/16 4:21 PM: -- sorry...in this case

[jira] [Commented] (SPARK-15645) Fix some typos of Streaming module

2016-05-29 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305971#comment-15305971 ] Xin Ren commented on SPARK-15645: - sorry...in this case what should be done for some very trivial things?

[jira] [Commented] (SPARK-15648) add TeradataDialect

2016-05-29 Thread lihongli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305921#comment-15305921 ] lihongli commented on SPARK-15648: -- Have you modified the issue?Thanks very much.This is first time I

[jira] [Updated] (SPARK-15648) add TeradataDialect

2016-05-29 Thread lihongli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lihongli updated SPARK-15648: - Affects Version/s: 1.6.1 > add TeradataDialect > --- > > Key:

[jira] [Updated] (SPARK-15648) add TeradataDialect

2016-05-29 Thread lihongli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lihongli updated SPARK-15648: - Attachment: (was: TeradataDialect.scala) > add TeradataDialect > --- > >

[jira] [Updated] (SPARK-15648) add TeradataDialect

2016-05-29 Thread lihongli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lihongli updated SPARK-15648: - Affects Version/s: (was: 1.6.1) Priority: Minor (was: Major) Issue Type:

[jira] [Commented] (SPARK-13570) pyspark save with partitionBy is very slow

2016-05-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305896#comment-15305896 ] Sean Owen commented on SPARK-13570: --- It's hard to say whether this is a problem without knowing more

[jira] [Updated] (SPARK-13042) Adding Specificity metric (= true negative rate) to MulticlassMetrics.scala

2016-05-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13042: -- Shepherd: (was: Sean Owen) > Adding Specificity metric (= true negative rate) to

[jira] [Updated] (SPARK-13570) pyspark save with partitionBy is very slow

2016-05-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13570: -- Shepherd: (was: Sean Owen) > pyspark save with partitionBy is very slow >

[jira] [Updated] (SPARK-15648) add TeradataDialect

2016-05-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15648: -- Shepherd: (was: Sean Owen) Flags: (was: Patch) Target Version/s: (was:

[jira] [Commented] (SPARK-15645) Fix some typos of Streaming module

2016-05-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305871#comment-15305871 ] Sean Owen commented on SPARK-15645: --- Please don't make JIRAs for this > Fix some typos of Streaming

[jira] [Assigned] (SPARK-15490) SparkR 2.0 QA: New R APIs and API docs for non-MLib changes

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15490: Assignee: Apache Spark (was: Kai Jiang) > SparkR 2.0 QA: New R APIs and API docs for

[jira] [Commented] (SPARK-15490) SparkR 2.0 QA: New R APIs and API docs for non-MLib changes

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305855#comment-15305855 ] Apache Spark commented on SPARK-15490: -- User 'vectorijk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15490) SparkR 2.0 QA: New R APIs and API docs for non-MLib changes

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15490: Assignee: Kai Jiang (was: Apache Spark) > SparkR 2.0 QA: New R APIs and API docs for

[jira] [Commented] (SPARK-14615) Use the new ML Vector and Matrix in the ML pipeline based algorithms

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305853#comment-15305853 ] Apache Spark commented on SPARK-14615: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-15613) Incorrect days to millis conversion

2016-05-29 Thread Dmitry Bushev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bushev updated SPARK-15613: -- Description: There is an issue with {{DateTimeUtils.daysToMillis}} implementation. It

[jira] [Assigned] (SPARK-15648) add TeradataDialect

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15648: Assignee: (was: Apache Spark) > add TeradataDialect > --- > >

[jira] [Commented] (SPARK-15648) add TeradataDialect

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305805#comment-15305805 ] Apache Spark commented on SPARK-15648: -- User 'lihongliustc' has created a pull request for this

[jira] [Assigned] (SPARK-15648) add TeradataDialect

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15648: Assignee: Apache Spark > add TeradataDialect > --- > >

[jira] [Updated] (SPARK-15648) add TeradataDialect

2016-05-29 Thread lihongli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lihongli updated SPARK-15648: - Description: I found that Teradata does not have the Dialect in spark.So I want to add the

[jira] [Updated] (SPARK-15648) add TeradataDialect

2016-05-29 Thread lihongli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lihongli updated SPARK-15648: - Attachment: TeradataDialect.scala TeradataDialect > add TeradataDialect > --- > >

[jira] [Created] (SPARK-15648) add TeradataDialect

2016-05-29 Thread lihongli (JIRA)
lihongli created SPARK-15648: Summary: add TeradataDialect Key: SPARK-15648 URL: https://issues.apache.org/jira/browse/SPARK-15648 Project: Spark Issue Type: New Feature Components:

[jira] [Assigned] (SPARK-15647) Fix Boundary Cases in OptimizeCodegen Rule

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15647: Assignee: (was: Apache Spark) > Fix Boundary Cases in OptimizeCodegen Rule >

[jira] [Commented] (SPARK-15647) Fix Boundary Cases in OptimizeCodegen Rule

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305791#comment-15305791 ] Apache Spark commented on SPARK-15647: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15647) Fix Boundary Cases in OptimizeCodegen Rule

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15647: Assignee: Apache Spark > Fix Boundary Cases in OptimizeCodegen Rule >

[jira] [Commented] (SPARK-15528) conv function returns inconsistent result for the same data

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305790#comment-15305790 ] Apache Spark commented on SPARK-15528: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15528) conv function returns inconsistent result for the same data

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15528: Assignee: Apache Spark > conv function returns inconsistent result for the same data >

[jira] [Assigned] (SPARK-15528) conv function returns inconsistent result for the same data

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15528: Assignee: (was: Apache Spark) > conv function returns inconsistent result for the

[jira] [Created] (SPARK-15647) Fix Boundary Cases in OptimizeCodegen Rule

2016-05-29 Thread Xiao Li (JIRA)
Xiao Li created SPARK-15647: --- Summary: Fix Boundary Cases in OptimizeCodegen Rule Key: SPARK-15647 URL: https://issues.apache.org/jira/browse/SPARK-15647 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-15617) Clarify that fMeasure in MulticlassMetrics and MulticlassClassificationEvaluator is "micro" f1_score

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15617: Assignee: Apache Spark > Clarify that fMeasure in MulticlassMetrics and >

[jira] [Assigned] (SPARK-15617) Clarify that fMeasure in MulticlassMetrics and MulticlassClassificationEvaluator is "micro" f1_score

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15617: Assignee: (was: Apache Spark) > Clarify that fMeasure in MulticlassMetrics and >

[jira] [Commented] (SPARK-15617) Clarify that fMeasure in MulticlassMetrics and MulticlassClassificationEvaluator is "micro" f1_score

2016-05-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305773#comment-15305773 ] Apache Spark commented on SPARK-15617: -- User 'zhengruifeng' has created a pull request for this